Deep Learning with Long Short-Term Memory (LSTM)

This blog post has some recent papers about Deep Learning with Long-Short Term Memory (LSTM). To get started I recommend checking out Christopher Olah’s Understanding LSTM Networks and Andrej Karpathy’s The Unreasonable Effectiveness of Recurrent Neural Networks. This blog post is complemented by Deep Learning with Recurrent/Recursive Neural Networks (RNN) – ICLR 2017 Discoveries.

Best regards,
Amund Tveit

Year  Title Author
2016   Look, Listen and Learn-A Multimodal LSTM for Speaker Identification  J Ren, Y Hu, YW Tai, C Wang, L Xu, W Sun, Q Yan
2016   Leveraging Sentence-level Information with Encoder LSTM for Natural Language Understanding  G Kurata, B Xiang, B Zhou, M Yu
2016   Deep Convolutional and LSTM Recurrent Neural Networks for Multimodal Wearable Activity Recognition  FJ Ordóñez, D Roggen
2016   Exploiting LSTM structure in deep neural networks for speech recognition  T He, J Droppo
2016   A Comprehensive Study of Deep Bidirectional LSTM RNNs for Acoustic Modeling in Speech Recognition  A Zeyer, P Doetsch, P Voigtlaender, R Schlüter, H Ney
2016   Geometric Scene Parsing with Hierarchical LSTM  Z Peng, R Zhang, X Liang, L Lin
2016   LSTM Networks for Mobile Human Activity Recognition  Y Chen, K Zhong, J Zhang, Q Sun, X Zhao
2016   Learning Natural Language Inference using Bidirectional LSTM model and Inner-Attention  Y Liu, C Sun, L Lin, X Wang
2016   Facing Realism in Spontaneous Emotion Recognition from Speech: Feature Enhancement by Autoencoder with LSTM Neural Networks  Z Zhang, F Ringeval, J Han, J Deng, E Marchi
2016   Contextual LSTM (CLSTM) models for Large scale NLP tasks  S Ghosh, O Vinyals, B Strope, S Roy, T Dean, L Heck
2016   Bidirectional LSTM Networks Employing Stacked Bottleneck Features for Expressive Speech-Driven Head Motion Synthesis  K Haag, H Shimodaira
2016   Beyond Frame-level CNN: Saliency-aware 3D CNN with LSTM for Video Action Recognition  J Song, H Shen
2015   Learning Statistical Scripts with LSTM Recurrent Neural Networks  K Pichotta, RJ Mooney
2015   A deep bidirectional LSTM approach for video-realistic talking head  B Fan, L Xie, S Yang, L Wang, FK Soong
2015   Maxout neurons for deep convolutional and LSTM neural networks in speech recognition  M Cai, J Liu
2015   Scene Analysis by Mid-level Attribute Learning using 2D LSTM networks and an Application to Web-image Tagging  W Byeon, M Liwicki, TM Breuel
2015   Learning to Diagnose with LSTM Recurrent Neural Networks  ZC Lipton, DC Kale, C Elkan, R Wetzell
2015   Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting  SHI Xingjian, Z Chen, H Wang, DY Yeung, W Wong
Continue Reading

Deep Learning in Finance

This posting has recent publications about Deep Learning in Finance (e.g. stock market prediction)

Best regards,
Amund Tveit

Year  Title Author
2016   Genetic deep neural networks using different activation functions for financial data mining  LM Zhang
2016   Computational Intelligence and Financial Markets: A Survey and Future Directions  RC Cavalcante, RC Brasileiro, VLF Souza, JP Nobrega
2016   Classification-based Financial Markets Prediction using Deep Neural Networks  M Dixon, D Klabjan, JH Bang
2016   Exploiting Twitter Moods to Boost Financial Trend Prediction Based on Deep Network Models  Y Huang, K Huang, Y Wang, H Zhang, J Guan, S Zhou
2016 Deep Learning in Finance J. B. Heaton, N. G. Polson, J. H. Witte
2016   Deep Direct Reinforcement Learning for Financial Signal Representation and Trading.  Y Deng, F Bao, Y Kong, Z Ren, Q Dai
2015   Improving Decision Analytics with Deep Learning: The Case of Financial Disclosures  R Fehrer, S Feuerriegel
2015   An application of deep learning for trade signal prediction in financial markets  AC Turkmen, AT Cemgil
2015   Deep Learning for Multivariate Financial Time Series  G BATRES
2015   Deep Modeling Complex Couplings within Financial Markets  W Cao, L Hu, L Cao
2015   Leverage Financial News to Predict Stock Price Movements Using Word Embeddings and Deep Neural Networks  Y Peng, H Jiang
2014   GPU Implementation of a Deep Learning Network for Financial Prediction  R Kumar, AK Cheema
2016   Non-Conformity Detection in High-Dimensional Time Series of Stock Market Data  A Kasuga, Y Ohsawa, T Yoshino, S Ashida
2016   Artificial neural networks approach to the forecast of stock market price movements  L Di Persio, O Honchar
2016   Forecasting Trade Direction and Size of Future Contracts Using Deep Belief Network  A Lai, MK Li, FW Pong
Continue Reading

Deep Learning for Information Retrieval and Learning to Rank

This posting is about Deep Learning for Information Retrieval and Learning to Rank (i.e. of interest if developing search engines). The posting is complemented by the posting Deep Learning for Question Answering. To get started I recommend checking out Jianfeng Gao‘s (Deep Learning Technology Center at Microsoft Research) presentation Deep Learning for Web Search and Natural Language Processing.

Of partial relevance is the posting Deep Learning for Sentiment Analysis, the posting about Embedding for NLP with Deep Learning, the posting about Deep Learning for Natural Language Processing (ICLR 2017 discoveries), and the posting about Deep Learning for Recommender Systems

Best regards,
Amund Tveit

 

Year  Title Author
2016   Efficient Inference, Search and Evaluation for Latent Variable Models of Text with Applications to Information Retrieval and Machine Translation  K Krstovski
2016   Content-based Information Retrieval via Nearest Neighbor Search  Y Huang
2016   Information retrieval in instant messaging platforms using Recurrent Neural Networks  JHP Suorra
2016   Information Retrieval with Dimensionality Reduction using Deep Belief Networks  V Slot
2015   Deep Sentence Embedding Using the Long Short Term Memory Network: Analysis and Application to Information Retrieval  H Palangi, L Deng, Y Shen, J Gao, X He, J Chen
2014   A compositional hierarchical model for music information retrieval  M Pesek, A Leonardis, M Marolt
2014   Log-Bilinear Document Language Model for Ad-hoc Information Retrieval  X Tu, J Luo, B Li, T He
2016   Learning to rank chemical compounds based on their multiprotein activity using Random Forests  D Lesniak, M l Kowalik, P Kruk
2016   Multilevel Syntactic Parsing Based on Recursive Restricted Boltzmann Machines and Learning to Rank  J Xu, H Chen, S Zhou, B He
2016   Automatic Face Recognition Based On Learning to Rank for Image Quality Assessment  K Busa, G Tejaswi
2015   Application of Learning to Rank to protein remote homology detection  B Liu, J Chen, X Wang
Continue Reading

Deep Learning for Question Answering

This posting presents recent publications related to Deep Learning for Question Answering. Question Answering is described as “a computer science discipline within the fields of information retrieval and natural language processing (NLP), which is concerned with building systems that automatically answer questions posed by humans in a natural language”. I’ll also publish postings about Deep Learning for Information Retrieval and Learning to Rank today.

Best regards,
Amund Tveit

Year  Title Author
2016   Skipping Word: A Character-Sequential Representation based Framework for Question Answering  L Meng, Y Li, M Liu, P Shu
2016   Semantic computation in geography question answering  S Zhao, Y Zheng, C Zhu, T Zhao, S Li
2016   Open-ended visual question answering  I Masuda Mora
2016   Creating Causal Embeddings for Question Answering with Minimal Supervision  R Sharp, M Surdeanu, P Jansen, P Clark, M Hammond
2016   Deep Feature Fusion Network for Answer Quality Prediction in Community Question Answering  SP Suggu, KN Goutham, MK Chinnakotla
2016   ECNU at SemEval-2016 Task 3: Exploring traditional method and deep learning method for question retrieval and answer ranking in community question answering  G Wu, M Lan
2016   Ask Your Neurons: A Deep Learning Approach to Visual Question Answering  M Malinowski, M Rohrbach, M Fritz
2016   Revisiting Visual Question Answering Baselines  A Jabri, A Joulin, L van der Maaten
2016   End to End Long Short Term Memory Networks for Non-Factoid Question Answering  D Cohen, WB Croft
2016   Question Answering on Linked Data: Challenges and Future Directions  S Shekarpour, D Lukovnikov, AJ Kumar, K Endris
2016   Extracting Medical Knowledge from Crowdsourced Question Answering Website  Y Li, C Liu, N Du, W Fan, Q Li, J Gao, C Zhang, H Wu
2015   Building a Large-scale Multimodal Knowledge Base for Visual Question Answering  Y Zhu, C Zhang, C Ré, L Fei
2015   Answer Sequence Learning with Neural Networks for Answer Selection in Community Question Answering  X Zhou, B Hu, Q Chen, B Tang, X Wang
2015   Empirical Study on Deep Learning Models for Question Answering  Y Yu, W Zhang, CW Hang, B Zhou
2015   Simple Baseline for Visual Question Answering  B Zhou, Y Tian, S Sukhbaatar, A Szlam, R Fergus
2015   Chess Q&A: Question Answering on Chess Games  V Cirik, LP Morency, E Hovy
2015   Predicting the Quality of User-Generated Answers Using Co-Training in Community-based Question Answering Portals  B Liu, J Feng, M Liu, H Hu, X Wang
2015   Open Domain Question Answering via Semantic Enrichment  H Sun, H Ma, W Yih, CT Tsai, J Liu, MW Chang
2015   Visual Question Answering using Deep Learning  S Agrawal, A Mukherjee
2015   Learning Semantic Representation with Neural Networks for Community Question Answering Retrieval  G Zhou, Y Zhou, T He, W Wu
2015   WIKIQA: A Challenge Dataset for Open-Domain Question Answering  Y Yang, W Yih, C Meek
Continue Reading

Ensemble Deep Learning

Ensemble Based Machine Learning has been used with success in several Kaggle competitions, and this year also the Imagenet competition was dominated by ensembles in Deep Learning, e.g. Trimps-Soushen team from 3rd Research Institute of the Ministry of Public Security (China) used a combination of Inception, Inception-Resnet, Resnet and Wide Residual Network to win the Object Classification/localization challenge. This blog post has recent papers related to Ensembles in Deep Learning

Best regards,

Amund Tveit

Year  Title Author
2016   A Deep Learning Approach to Unsupervised Ensemble Learning  U Shaham, X Cheng, O Dror, A Jaffe, B Nadler
2016   Distributed Ensemble Learning for Analyzing Nationwide Health-Insurance Databases  H Ha, J Kim, J Park, S Yoon
2016   When Ensemble Learning Meets Deep Learning: a New Deep Support Vector Machine for Classification  Z Qi, B Wang, Y Tian, P Zhang
2016   Group Dropout Inspired by Ensemble Learning  K Hara, D Saitoh, T Kondou, S Suzuki, H Shouno
2016   Analysis of Dropout Learning Regarded as Ensemble Learning  K Hara, D Saitoh, H Shouno
2016   A novel methodology to predict urban traffic congestion with ensemble learning  G Asencio
2016   Glaucoma Detection Using Entropy Sampling And Ensemble Learning For Automatic Optic Cup And Disc Segmentation  J Zilly, JM Buhmann, D Mahapatra
2015   Soft sensor development for nonlinear and time‐varying processes based on supervised ensemble learning with improved process state partition  W Shao, X Tian, P Wang
2015   Face hallucination through ensemble learning  CT Tu, MC Ho, JR Luo
2015   A novel multistage deep belief network based extreme learning machine ensemble learning paradigm for credit risk assessment  L Yu, Z Yang, L Tang
2015   An Efficient Concept Detection System Via Sparse Ensemble Learning  S Tang, ZX Xu, YD Zhang, HJ Li, YT Zheng, JT Li
2015   Intro to Practical Ensemble Learning  E LeDell
2014   Pedestrian Detection with Spatially Pooled Features and Structured Ensemble Learning  S Paisitkriangkrai, C Shen, A Hengel
2014   Ensemble Learning Approaches in Speech Recognition  Y Zhao, J Xue, X Chen
Continue Reading

Deep Learning for Sentiment Analysis

Recently I published Embedding for NLP with Deep Learning (e.g. word2vec and follow-ups) and Deep Learning for Natural Language Processing – ICLR 2017 Discoveries – this posting is also mostly NLP-related since it provides recent papers related to Deep Learning for Sentiment Analysis, but also has examples of other types of sentiment (e.g. image sentiment).

Best regards,
Amund Tveit

Year  Title Author
2016   Image Sentiment Analysis using Deep Convolutional Neural Networks with Domain Specific Fine Tuning  S Jindal, S Singh
2016   Multi-Objective Model Selection (MOMS)-based Semi-Supervised Framework for Sentiment Analysis  FH Khan, U Qamar, S Bashir
2016   Real-Time Topic and Sentiment Analysis in Human-Robot Conversation  E Russell
2016   Finki at SemEval-2016 Task 4: Deep Learning Architecture for Twitter Sentiment Analysis  D Stojanovski, G Strezoski, G Madjarov, I Dimitrovski
2016   Design of Sentiment Analysis System for Hindi Content  M Yadav, V Bhojane
2016   Visual Sentiment Analysis for Social Images Using Transfer Learning Approach  J Islam, Y Zhang
2016   Unsupervised Feature Learning Assisted Visual Sentiment Analysis  Z Li, Y Fan, F Wang, W Liu
2016   Context-Aware Text Representation for Social Relation Aided Sentiment Analysis  LM Nguyen
2016   Select-Additive Learning: Improving Cross-individual Generalization in Multimodal Sentiment Analysis  H Wang, A Meghawat, LP Morency, EP Xing
2016   A semi-supervised approach to sentiment analysis using revised sentiment strength based on SentiWordNet  FH Khan, U Qamar, S Bashir
2016   SWIMS: Semi-Supervised Subjective Feature Weighting and Intelligent Model Selection for Sentiment Analysis  FH Khan, U Qamar, S Bashir
2016   Cross-modality Consistent Regression for Joint Visual-Textual Sentiment Analysis of Social Multimedia  Q You, J Luo, H Jin, J Yang
2016   INSIGHT-1 at SemEval-2016 Task 5: Deep Learning for Multilingual Aspect-based Sentiment Analysis  S Ruder12, P Ghaffari, JG Breslin
2016   SENSEI-LIF at SemEval-2016 Task 4: Polarity embedding fusion for robust sentiment analysis  M Rouvier, B Favre
2016   Leveraging Multimodal Information for Event Summarization and Concept-level Sentiment Analysis  RR Shah, Y Yu, A Verma, S Tang, AD Shaikh
2016   Visual and Textual Sentiment Analysis of a Microblog Using Deep Convolutional Neural Networks  Y Yu, H Lin, J Meng, Z Zhao
2016   Sentiment Analysis for Chinese Microblog based on Deep Neural Networks with Convolutional Extension Features  SUN Xiao, LI Chengcheng, REN Fuji
2016   PotTS at SemEval-2016 Task 4: Sentiment Analysis of Twitter Using Character-level Convolutional Neural Networks.  U Sidarenka, KL Straße
2016   Recursive Neural Conditional Random Fields for Aspect-based Sentiment Analysis  W Wang, SJ Pan, D Dahlmeier, X Xiao
2016   eSAP: A Decision Support Framework for Enhanced Sentiment Analysis and Polarity Classification  FH Khan, U Qamar, S Bashir
2016   Paragraph2Vec-based sentiment analysis on social media for business in Thailand  P Sanguansat
2016   Sentiment Analysis of Chinese Micro Blog Based on DNN and ELM and Vector Space Model  H Liu, S Li, C Jiang, H Liu
2016   Sentiment Analysis in Finance Market Forcast  D Wang
2016   Visual Sentiment Analysis with Network in Network  Z Li, Y Fan, F Wang
2015   Convolutional Neural Networks for Multimedia Sentiment Analysis  G Cai, B Xia
2015   Sentiment Analysis with Incremental Human-in-the-Loop Learning and Lexical Resource Customization  S Mishra, J Diesner, J Byrne, E Surbeck
2015   Parallel Recursive Deep Model for Sentiment Analysis  G Tian, Y Zhou
2015   Unsupervised Sentiment Analysis for Social Media Images  Y Wang, S Wang, J Tang, H Liu, B Li
2015   Prediction of changes in the stock market using twitter and sentiment analysis  IV Serban, DS González, X Wu
2015   Review Sentiment Analysis Based on Deep Learning  Z Hu, J Hu, W Ding, X Zheng
2015   Twitter Sentiment Analysis Using Deep Convolutional Neural Network  D Stojanovski, G Strezoski, G Madjarov, I Dimitrovski
2015   Sentiment-Specific Representation Learning for Document-Level Sentiment Analysis  D Tang
2015   Sentiment Analysis via Integrating Distributed Representations of Variable-length Word Sequence  Z Cui, X Shi, Y Chen
2015   Learning Higher-Level Features with Convolutional Restricted Boltzmann Machines for Sentiment Analysis  T Huynh, Y He, S Rüger
2015   Recursive Autoencoder with HowNet Lexicon for Sentence-Level Sentiment Analysis  X Fu, Y Xu
2014   Recursive Deep Learning for Sentiment Analysis over Social Data  C Li, B Xu, G Wu, S He, G Tian, H Hao
2014   Robust Image Sentiment Analysis Using Progressively Trained and Domain Transferred Deep Networks  Q You, J Luo, H Jin, J Yang
Continue Reading

Deep Learning with Gaussian Process

Gaussian Process is a statistical model where observations are in the continuous domain, to learn more check out a tutorial on gaussian process (by Univ.of Cambridge’s Zoubin G.). Gaussian Process is an infinite-dimensional generalization of multivariate normal distributions.

Researchers from University of Sheffield – Andreas C. Damanianou and Neil D. Lawrence – started using Gaussian Process with Deep Belief Networks (in 2013). This Blog post contains recent papers related to combining Deep Learning with Gaussian Process.

Best regards,
Amund Tveit

Year  Title Author
2016   Inverse Reinforcement Learning via Deep Gaussian Process  M Jin, C Spanos
2016   Annealing Gaussian into ReLU: a New Sampling Strategy for Leaky-ReLU RBM  CL Li, S Ravanbakhsh, B Poczos
2016   Large Scale Gaussian Process for Overlap-based Object Proposal Scoring  SL Pintea, S Karaoglu, JC van Gemert
2016   Gaussian Neuron in Deep Belief Network for Sentiment Prediction  Y Jin, D Du, H Zhang
2016   Fast, Exact and Multi-Scale Inference for Semantic Image Segmentation with Deep Gaussian CRFs  S Chandra, I Kokkinos
2016   The Variational Gaussian Process  D Tran, R Ranganath, DM Blei
2016   Probabilistic Feature Learning Using Gaussian Process Auto-Encoders  S Olofsson
2016   Sequential Inference for Deep Gaussian Process  Y Wang, M Brubaker, B Chaib
2016   Gaussian Copula Variational Autoencoders for Mixed Data  S Suh, S Choi
2016   Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising  K Zhang, W Zuo, Y Chen, D Meng, L Zhang
2016   Image super-resolution using non-local Gaussian process regression  H Wang, X Gao, K Zhang, J Li
2016   Gaussian Conditional Random Field Network for Semantic Segmentation  R Vemulapalli, O Tuzel, MY Liu, R Chellappa
2016   Structured and Efficient Variational Deep Learning with Matrix Gaussian Posteriors  C Louizos, M Welling
2016   Deep Gaussian Processes for Regression using Approximate Expectation Propagation  TD Bui, D Hernández
2015   Learning to Assess Terrain from Human Demonstration Using an Introspective Gaussian Process Classifier  LP Berczi, I Posner, TD Barfoot
2015   Assessing the Degree of Nativeness and Parkinson’s Condition Using Gaussian Processes and Deep Rectifier Neural Networks  T Grósz, R Busa
2015   Gaussian processes methods for nostationary regression  L Muñoz González
2015   Deep Neural Networks with Random Gaussian Weights: A Universal Classification Strategy?  R Giryes, G Sapiro, AM Bronstein
2015   Nonlinear Gaussian Belief Network based fault diagnosis for industrial processes  H Yu, F Khan, V Garaniya
2015   Interactions Between Gaussian Processes and Bayesian Estimation  YL Wang
2015   Gaussian discrete restricted Boltzmann machine: theory and its applications: a thesis presented in partial fulfilment of the requirements for the degree of Master of …  S Manoharan
2015   Prosody Generation Using Frame-based Gaussian Process Regression  T Koriyama, T Kobayashi
2015   Mean-Field Inference in Gaussian Restricted Boltzmann Machine  C Takahashi, M Yasuda
2015   Variational Auto-encoded Deep Gaussian Processes  Z Dai, A Damianou, J González, N Lawrence
2015   Training Deep Gaussian Processes using Stochastic Expectation Propagation and Probabilistic Backpropagation  TD Bui, JM Hernández
2015   Accurate Object Detection and Semantic Segmentation using Gaussian Mixture Model and CNN  S Jain, S Dehriya, YK Jain
2014   Cross Modal Deep Model and Gaussian Process Based Model for MSR-Bing Challenge  J Wang, C Kang, Y He, S Xiang, C Pan
2014   Non-negative Factor Analysis of Gaussian Mixture Model Weight Adaptation for Language and Dialect Recognition  J Glass
2014   Gaussian Process Models with Parallelization and GPU acceleration  Z Dai, A Damianou, J Hensman, N Lawrence
2014   Parametric Speech Synthesis Using Local and Global Sparse Gaussian  T Koriyama, T Nose, T Kobayashi
2014   On the Link Between Gaussian Homotopy Continuation and Convex Envelopes  H Mobahi, JW Fisher III
2014   Improving Deep Neural Networks Using State Projection Vectors Of Subspace Gaussian Mixture Model As Features  M Karthick, S Umesh
2014   A Theoretical Analysis of Optimization by Gaussian Continuation  H Mobahi, JW Fisher III
2014   Factoring Variations in Natural Images with Deep Gaussian Mixture Models  A van den Oord, B Schrauwen
2014   Feature representation with Deep Gaussian processes
Continue Reading

Deep Learning for Clustering

Previously I published an ICLR 2017 discoveries blog post about Unsupervised Deep Learning – a subset of Unsupervised methods is Clustering, and this blog post has recent publications about Deep Learning for Clustering.

Best regards,

Amund Tveit

 

Year  Title Author
2016   An Intention-Topic Model Based on Verbs Clustering and Short Texts Topic Mining  T Lu, S Hou, Z Chen, L Cui, L Zhang
2016   Speaker Identification And Clustering Using Convolutional Neural Networks  Y Lukic, C Vogt, O Dürr, T Stadelmann
2016   Building energy modeling (BEM) using clustering algorithms and semi-supervised machine learning approaches  H Naganathan, WO Chong, X Chen
2016   Clustering Based Feature Learning on Variable Stars  C Mackenzie, K Pichara, P Protopapas
2016   Deep Belief Networks Oriented Clustering  Q Yang, H Wang, T Li, Y Yang
2016   Clustering Non-Stationary Data Streams with Online Deep Learning  A Hontabat, M Rising
2016   Fast image clustering based on convolutional neural network and binary K-means  S Ke, Y Zhao, B Li, Z Wu, X Liu
2016   Extracting Bottlenecks for Reinforcement Learning Agent by Holonic Concept Clustering and Attentional Functions  B Ghazanfari, N Mozayani
2016   A Hybrid Spectral Clustering and Deep Neural Network Ensemble Algorithm for Intrusion Detection in Sensor Networks  T Ma, F Wang, J Cheng, Y Yu, X Chen
2016   Single-Channel Multi-Speaker Separation using Deep Clustering  Y Isik, JL Roux, Z Chen, S Watanabe, JR Hershey
2016   Towards K-means-friendly Spaces: Simultaneous Deep Learning and Clustering  B Yang, X Fu, ND Sidiropoulos, M Hong
2016   A Personalized Markov Clustering and Deep Learning Approach for Arabic Text Categorization  V Jindal
2016   Clustering the seoul metropolitan area by travel patterns based on a deep belief network  G Han, K Sohn
2016   An Empirical Investigation of Word Clustering Techniques for Natural Language Understanding  DA Shunmugam, P Archana
2016   Infinite Ensemble for Image Clustering  H Liu, M Shao, S Li, Y Fu
2015   Theoretical Analysis-based Distributed Load Balancing over Dynamic Overlay Clustering  H Lee, B Kwon, S Kim, I Lee, S Lee
2015   Competitive and Penalized Clustering Auto-encoder  Z Wang, Y Cheung
2015   Learning A Task-Specific Deep Architecture For Clustering  Z Wang, S Chang, J Zhou, TS Huang
2015   Pedestrian detection in thermal images using adaptive fuzzy C-means clustering and convolutional neural networks  V John, S Mita, Z Liu, B Qi
2015   Active Learning with Clustering and Unsupervised Feature Learning  S Berardo, E Favero, N Neto
2015   Clustering Noisy Signals with Structured Sparsity Using Time-Frequency Representation  T Hope, A Wagner, O Zuk
2015   Neuron Clustering for Mitigating Catastrophic Forgetting in Supervised and Reinforcement Learning  BF Goodrich
2015   Clustering Data of Mixed Categorical and Numerical Type with Unsupervised Feature Learning  D Lam, M Wei, D Wunsch
2015   Semi-supervised Hierarchical Clustering Ensemble and Its Application  W Xiao, Y Yang, H Wang, T Li, H Xing
2015   FaceNet: A Unified Embedding for Face Recognition and Clustering  F Schroff, D Kalenichenko, J Philbin
2015   Deep Transductive Semi-supervised Maximum Margin Clustering  G Chen
2015   Max-Entropy Feed-Forward Clustering Neural Network  H Xiao, X Zhu
2015   Overview of the ImageCLEF 2015 medical clustering task  MA Amin, MK Mohammed
2015   Joint Image Clustering and Labeling by Matrix Factorization  S Hong, J Choi, J Feyereisl, B Han, LS Davis
2015   Combining deep learning and unsupervised clustering to improve scene recognition performance  A Kappeler, RD Morris, AR Kamat, N Rasiwasia
2015   Experimental Study of Unsupervised Feature Learning for HEp-2 Cell Images Clustering  Y Zhao, Z Gao, L Wang, L Zhou
2015   Soft context clustering for F0 modeling in HMM-based speech synthesis  S Khorram, H Sameti, S King
2015   Semi-automatic ground truth generation using unsupervised clustering and limited manual labeling: Application to handwritten character recognition  S Vajda, Y Rangoni, H Cecotti
2015   Language discrimination and clustering via a neural network approach  A Mariano, G Parisi, S Pascazio
2015   Active Distance-Based Clustering using K-medoids  A Aghaee, M Ghadiri, MS Baghshah
2015   Convolutional Clustering for Unsupervised Learning  A Dundar, J Jin, E Culurciello
2015   Deep Learning with Nonparametric Clustering  G Chen
2015   Multi-view clustering via structured low-rank representation  D Wang, Q Yin, R He, L Wang, T Tan
2014   A Convex Formulation for Spectral Shrunk Clustering  X Chang, F Nie, Z Ma, Y Yang, X Zhou
2014   Ghent University-iMinds at MediaEval 2014 Diverse Images: Adaptive Clustering with Deep Features  B Vandersmissen, A Tomar, F Godin, W De Neve
2014   Features in Concert: Discriminative Feature Selection meets Unsupervised Clustering  M Leordeanu, A Radu, R Sukthankar
2014   Deep Embedding Network for Clustering  P Huang, Y Huang, W Wang, L Wang
2014   SoF: Soft-Cluster Matrix Factorization for Probabilistic Clustering  H Zhao, P Poupart, Y Zhang, M Lysy
Continue Reading

Deep Learning for Ultrasound Analysis


Ultrasound (also called Sonography) are sound waves with higher frequency than humans can hear, they frequently used in medical settings, e.g. for checking that pregnancy is going well with fetal ultrasound. For more about Ultrasound data formats check out Ultrasound Research Interface. This blog post has recent publications about applying Deep Learning for analyzing Ultrasound data.

Best regards,
Amund Tveit

Year  Title Authors
2016   Early-stage atherosclerosis detection using deep learning over carotid ultrasound images  RM Menchón
2016   Automatic Detection of Standard Sagittal Plane in the First Trimester of Pregnancy Using 3-D Ultrasound Data  S Nie, J Yu, P Chen, Y Wang, JQ Zhang
2016   Detection of prostate cancer using temporal sequences of ultrasound data: a large clinical feasibility study  S Azizi, F Imani, S Ghavidel, A Tahmasebi, JT Kwak
2016   Hough-CNN: Deep Learning for Segmentation of Deep Brain Regions in MRI and Ultrasound  F Milletari, SA Ahmadi, C Kroll, A Plate, V Rozanski
2016   Hybrid approach for automatic segmentation of fetal abdomen from ultrasound images using deep learning  H Ravishankar, SM Prabhu, V Vaidya, N Singhal
2016   Iterative Multi-domain Regularized Deep Learning for Anatomical Structure Detection and Segmentation from Ultrasound Images  H Chen, Y Zheng, JH Park, PA Heng, SK Zhou
2016   4D Cardiac Ultrasound Standard Plane Location by Spatial-Temporal Correlation  Y Gu, GZ Yang, J Yang, K Sun
2016   Computer-Aided Diagnosis for Breast Ultrasound Using Computerized BI-RADS Features and Machine Learning Methods  J Shan, SK Alam, B Garra, Y Zhang, T Ahmed
2016   Stacked Deep Polynomial Network Based Representation Learning for Tumor Classification with Small Ultrasound Image Dataset  J Shi, S Zhou, X Liu, Q Zhang, M Lu, T Wang
2016   Coupling Convolutional Neural Networks and Hough Voting for Robust Segmentation of Ultrasound Volumes  C Kroll, F Milletari, N Navab, SA Ahmadi
2016   Classifying Cancer Grades Using Temporal Ultrasound for Transrectal Prostate Biopsy  S Azizi, F Imani, JT Kwak, A Tahmasebi, S Xu, P Yan
2015   Tumor Classification by Deep Polynomial Network and Multiple Kernel Learning on Small Ultrasound Image Dataset  X Liu, J Shi, Q Zhang
2015   Automatic Recognition of Fetal Facial Standard Plane in Ultrasound Image via Fisher Vector  B Lei, EL Tan, S Chen, L Zhuo, S Li, D Ni, T Wang
2015   Estimation of the Arterial Diameter in Ultrasound Images of the Common Carotid Artery  RM Menchón
2015   Cell recognition based on topological sparse coding for microscopy imaging of focused ultrasound treatment  Z Wang, J Zhu, Y Xue, C Song, N Bi
2014   Mapping between ultrasound and vowel speech using DNN framework  X Zheng, J Wei, W Lu, Q Fang, J Dang
2014   High-definition 3D Image Processing Technology for Ultrasound Diagnostic Scanners  M Ogino, T Shibahara, Y Noguchi, T Tsujita
2014   Fully automatic segmentation of ultrasound common carotid artery images based on machine learning  RM Menchón
Continue Reading

Deep Learning with FPGA

This chapter presents recent papers for using FPGAs (Field Programmable Gate Arrays) for Deep Learning. FPGAs can roughly be seen as a Software-configurable Hardware, i.e you in some cases get close to dedicated hardware speed (although typically at lower clock frequency than chips, but typically with strong on-FPGA parallelism), this can be a potential good fit for e.g. Convolutional Neural Networks since they require many convolutional layer calculations (with many convolutional filters per conv.layer) with large tensors. Recommend starting with having a look at Deep Learning on FPGAs: Past, Present, and Future

Best regards,

Amund Tveit

Convolutional Neural Network

  1. Throughput-Optimized OpenCL-based FPGA Accelerator for Large-Scale Convolutional Neural Networks
    – Authors: N Suda, V Chandra, G Dasika, A Mohanty, Y M (2016)
  2. Curbing the roofline: a scalable and flexible architecture for CNNs on FPGA
    – Authors: P Meloni, G Deriu, F Conti, I Loi, L Raffo, L Benini (2016)
  3. Comprehensive Evaluation of OpenCL-based Convolutional Neural Network Accelerators in Xilinx and Altera FPGAs
    – Authors: R Tapiador, A Rios (2016)
  4. Optimizing FPGA-based Accelerator Design for Deep Convolutional Neural Networks
    – Authors: C Zhang, P Li, G Sun, Y Guan, B Xiao, J Cong (2015)
  5. Musical notes classification with Neuromorphic Auditory System using FPGA and a Convolutional Spiking Network
    – Authors: E Cerezuela (2015)
  6. CNNLab: a Novel Parallel Framework for Neural Networks using GPU and FPGA-a Practical Study with Trade-off Analysis
    – Authors: M Zhu, L Liu, C Wang, Y Xie (2016)
  7. Energy-Efficient CNN Implementation on a Deeply Pipelined FPGA Cluster
    – Authors: C Zhang, D Wu, J Sun, G Sun, G Luo, J Cong (2016)

 

Other uses of FPGA in Deep Learning

  1. Deep Neural Network Architecture Implementation on FPGAs Using a Layer Multiplexing Scheme
    – Authors: F Ortega (2016)
  2. FPGA Based Multi-core Architectures for Deep Learning Networks
    – Authors: H Chen (2016)
  3. FPGA Implementation of a Scalable and Highly Parallel Architecture for Restricted Boltzmann Machines
    – Authors: K Ueyoshi, T Marukame, T Asai, M Motomura… (2016)
  4. DLAU: A Scalable Deep Learning Accelerator Unit on FPGA
    – Authors: C Wang, Q Yu, L Gong, X Li, Y Xie, X Zhou (2016)
  5. Deep Learning on FPGAs
    – Authors: Gj Lacey (2016)
  6. DNNWEAVER: From High-Level Deep Network Models to FPGA Acceleration
    – Authors: H Sharma, J Park, E Amaro, B Thwaites, P Kotha (2016)
  7. Handwritten Digit Classification on FPGA
    – Authors: K Kudrolli, S Shah, Dj Park (2016)
  8. Fpga Based Implementation of Deep Neural Networks Using On-chip Memory Only
    – Authors: J Park, W Sung (2016)
  9. FPGA Implementation of Autoencoders Having Shared Synapse Architecture
    – Authors: A Suzuki, T Morie, H Tamukoh (2016)
  10. Programming and Runtime Support to Blaze FPGA Accelerator Deployment at Datacenter Scale
    – Authors: M Huang, D Wu, Ch Yu, Z Fang, M Interlandi, T Condie… (2016)
  11. A Deep Learning Prediction Process Accelerator Based FPGA
    – Authors: Q Yu, C Wang, X Ma, X Li, X Zhou (2015)
  12. FPGA implementation of a Deep Belief Network architecture for character recognition using stochastic computation
    – Authors: K Sanni, G Garreau, Jl Molin, Ag Andreou (2015)
  13. An FPGA-Based Multiple-Weight-and-Neuron-Fault Tolerant Digital Multilayer Perceptron (Full Version)
    – Authors: T Horita, I Takanami, M Akiba, M Terauchi, T Kanno (2015)
  14. Efficient Generation of Energy and Performance Pareto Front for FPGA Designs
    – Authors: Sr Kuppannagari, Vk Prasanna (2015)
  15. Optimizing FPGA-based Accelerator Design for Deep Convolutional Neural Networks
    – Authors: C Zhang, P Li, G Sun, Y Guan, B Xiao, J Cong (2015)
  16. Musical notes classification with Neuromorphic Auditory System using FPGA and a Convolutional Spiking Network
    – Authors: E Cerezuela (2015)
  17. FPGA Acceleration of Recurrent Neural Network based Language Model
    – Authors: S Li, C Wu, Hh Li, B Li, Y Wang, Q Qiu (2015)
  18. FPGA emulation of a spike-based, stochastic system for real-time image dewarping
    – Authors: Jl Molin, T Figliolia, K Sanni, I Doxas, A Andreou… (2015)
Continue Reading