Deep Learning with Long Short-Term Memory (LSTM)

This blog post has some recent papers about Deep Learning with Long-Short Term Memory (LSTM). To get started I recommend checking out Christopher Olah’s Understanding LSTM Networks and Andrej Karpathy’s The Unreasonable Effectiveness of Recurrent Neural Networks. This blog post is complemented by Deep Learning with Recurrent/Recursive Neural Networks (RNN) – ICLR 2017 Discoveries.

Best regards,
Amund Tveit

Year  Title Author
2016   Look, Listen and Learn-A Multimodal LSTM for Speaker Identification  J Ren, Y Hu, YW Tai, C Wang, L Xu, W Sun, Q Yan
2016   Leveraging Sentence-level Information with Encoder LSTM for Natural Language Understanding  G Kurata, B Xiang, B Zhou, M Yu
2016   Deep Convolutional and LSTM Recurrent Neural Networks for Multimodal Wearable Activity Recognition  FJ Ordóñez, D Roggen
2016   Exploiting LSTM structure in deep neural networks for speech recognition  T He, J Droppo
2016   A Comprehensive Study of Deep Bidirectional LSTM RNNs for Acoustic Modeling in Speech Recognition  A Zeyer, P Doetsch, P Voigtlaender, R Schlüter, H Ney
2016   Geometric Scene Parsing with Hierarchical LSTM  Z Peng, R Zhang, X Liang, L Lin
2016   LSTM Networks for Mobile Human Activity Recognition  Y Chen, K Zhong, J Zhang, Q Sun, X Zhao
2016   Learning Natural Language Inference using Bidirectional LSTM model and Inner-Attention  Y Liu, C Sun, L Lin, X Wang
2016   Facing Realism in Spontaneous Emotion Recognition from Speech: Feature Enhancement by Autoencoder with LSTM Neural Networks  Z Zhang, F Ringeval, J Han, J Deng, E Marchi
2016   Contextual LSTM (CLSTM) models for Large scale NLP tasks  S Ghosh, O Vinyals, B Strope, S Roy, T Dean, L Heck
2016   Bidirectional LSTM Networks Employing Stacked Bottleneck Features for Expressive Speech-Driven Head Motion Synthesis  K Haag, H Shimodaira
2016   Beyond Frame-level CNN: Saliency-aware 3D CNN with LSTM for Video Action Recognition  J Song, H Shen
2015   Learning Statistical Scripts with LSTM Recurrent Neural Networks  K Pichotta, RJ Mooney
2015   A deep bidirectional LSTM approach for video-realistic talking head  B Fan, L Xie, S Yang, L Wang, FK Soong
2015   Maxout neurons for deep convolutional and LSTM neural networks in speech recognition  M Cai, J Liu
2015   Scene Analysis by Mid-level Attribute Learning using 2D LSTM networks and an Application to Web-image Tagging  W Byeon, M Liwicki, TM Breuel
2015   Learning to Diagnose with LSTM Recurrent Neural Networks  ZC Lipton, DC Kale, C Elkan, R Wetzell
2015   Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting  SHI Xingjian, Z Chen, H Wang, DY Yeung, W Wong
Continue Reading

Deep Learning with Recurrent/Recursive Neural Networks (RNN) – ICLR 2017 Discoveries

The 5th International Conference on Learning Representation (ICLR 2017) is coming to Toulon, France (April 24-26 2017).

This blog post gives an overview of Deep Learning with Recurrent/Recursive Neural Networks (RNN) related papers submitted to ICLR 2017, see underneath for the list of papers. If you want to learn more about RNN check out Andrej Karpathy’s The Unreasonable Effectiveness of Recurrent Neural Networksand Pascanu, Gulcehre, Cho and Bengio’s How to Construct Deep Recurrent Neural Networks.

Best regards,
Amund Tveit

  1. Making Neural Programming Architectures Generalize via Recursion – Authors: Jonathon Cai, Richard Shin, Dawn Song
  2. Multi-label learning with the RNNs for Fashion Search – Authors: Taewan Kim
  3. Recursive Regression with Neural Networks: Approximating the HJI PDE Solution – Authors: Vicenç Rubies Royo
  4. SampleRNN: An Unconditional End-to-End Neural Audio Generation Model – Authors: Soroush Mehri, Kundan Kumar, Ishaan Gulrajani, Rithesh Kumar, Shubham Jain, Jose Manuel Rodriguez Sotelo, Aaron Courville, Yoshua Bengio
  5. Zoneout: Regularizing RNNs by Randomly Preserving Hidden Activations – Authors: David Krueger, Tegan Maharaj, Janos Kramar, Mohammad Pezeshki, Nicolas Ballas, Nan Rosemary Ke, Anirudh Goyal, Yoshua Bengio, Aaron Courville, Christopher Pal
  6. LR-GAN: Layered Recursive Generative Adversarial Networks for Image Generation – Authors: Jianwei Yang, Anitha Kannan, Dhruv Batra, Devi Parikh
  7. TopicRNN: A Recurrent Neural Network with Long-Range Semantic Dependency – Authors: Adji B. Dieng, Chong Wang, Jianfeng Gao, John Paisley

Sign up for Deep Learning newsletter!

Continue Reading