Deep Learning for Natural Language Processing – ICLR 2017 Discoveries

Update: 2017-Feb-03 – launched new service – (navigation and search in papers). Try e.g. out its Natural Language Processing Page.

The 5th International Conference on Learning Representation (ICLR 2017) is coming to Toulon, France (April 24-26 2017), and there is large amount of Deep Learning papers submitted to the conference, looks like it will be a great event (see word cloud below for most frequent words used in submitted paper titles).


This blog post gives an overview of Natural Language Processing related papers submitted to ICLR 2017, see underneath for the list of papers. If you want to learn about Deep Learning with NLP check out Stanford’s CS224d: Deep Learning for Natural Language Processing

Best regards,

Amund Tveit


Character/Word/Sentence Representation

  1. Character-aware Attention Residual Network for Sentence Representation – Authors: Xin Zheng, Zhenzhou Wu
  2. Program Synthesis for Character Level Language Modeling – Authors: Pavol Bielik, Veselin Raychev, Martin Vechev
  3. Words or Characters? Fine-grained Gating for Reading Comprehension – Authors: Zhilin Yang, Bhuwan Dhingra, Ye Yuan, Junjie Hu, William W. Cohen, Ruslan Salakhutdinov
  4. Deep Character-Level Neural Machine Translation By Learning Morphology – Authors: Shenjian Zhao, Zhihua Zhang
  5. Opening the vocabulary of neural language models with character-level word representations – Authors: Matthieu Labeau, Alexandre Allauzen
  6. Unsupervised sentence representation learning with adversarial auto-encoder – Authors: Shuai Tang, Hailin Jin, Chen Fang, Zhaowen Wang
  7. Offline Bilingual Word Vectors Without a Dictionary – Authors: Samuel L. Smith, David H. P. Turban, Nils Y. Hammerla, Steven Hamblin
  8. Learning Word-Like Units from Joint Audio-Visual Analylsis – Authors: David Harwath, James R. Glass
  9. Tying Word Vectors and Word Classifiers: A Loss Framework for Language Modeling – Authors: Hakan Inan, Khashayar Khosravi, Richard Socher
  10. Sentence Ordering using Recurrent Neural Networks – Authors: Lajanugen Logeswaran, Honglak Lee, Dragomir Radev

Search/Question-Answer/Recommender Systems

  1. Learning to Query, Reason, and Answer Questions On Ambiguous Texts – Authors: Xiaoxiao Guo, Tim Klinger, Clemens Rosenbaum, Joseph P. Bigus, Murray Campbell, Ban Kawas, Kartik Talamadupula, Gerry Tesauro, Satinder Singh
  2. Group Sparse CNNs for Question Sentence Classification with Answer Sets – Authors: Mingbo Ma, Liang Huang, Bing Xiang, Bowen Zhou
  3. CONTENT2VEC: Specializing Joint Representations of Product Images and Text for the task of Product Recommendation – Authors: Thomas Nedelec, Elena Smirnova, Flavian Vasile
  4. Is a picture worth a thousand words? A Deep Multi-Modal Fusion Architecture for Product Classification in e-commerce – Authors: Tom Zahavy, Alessandro Magnani, Abhinandan Krishnan, Shie Mannor

Word/Sentence Embedding

  1. A Simple but Tough-to-Beat Baseline for Sentence Embeddings – Authors: Sanjeev Arora, Yingyu Liang, Tengyu Ma
  2. Investigating Different Context Types and Representations for Learning Word Embeddings – Authors: Bofang Li, Tao Liu, Zhe Zhao, Xiaoyong Du
  3. Multi-view Recurrent Neural Acoustic Word Embeddings – Authors: Wanjia He, Weiran Wang, Karen Livescu
  4. A Self-Attentive Sentence Embedding – Authors: Zhouhan Lin, Minwei Feng, Cicero Nogueira dos Santos, Mo Yu, Bing Xiang, Bowen Zhou, Yoshua Bengio
  5. Fine-grained Analysis of Sentence Embeddings Using Auxiliary Prediction Tasks – Authors: Yossi Adi, Einat Kermany, Yonatan Belinkov, Ofer Lavi, Yoav Goldberg


  1. Neural Machine Translation with Latent Semantic of Image and Text – Authors: Joji Toyama, Masanori Misono, Masahiro Suzuki, Kotaro Nakayama, Yutaka Matsuo
  2. Beyond Bilingual: Multi-sense Word Embeddings using Multilingual Context – Authors: Shyam Upadhyay, Kai-Wei Chang, James Zhou, Matt Taddy, Adam Kalai
  3. Learning to Understand: Incorporating Local Contexts with Global Attention for Sentiment Classification – Authors: Zhigang Yuan, Yuting Hu, Yongfeng Huang
  4. Adaptive Feature Abstraction for Translating Video to Language – Authors: Yunchen Pu, Martin Renqiang Min, Zhe Gan, Lawrence Carin
  5. A Convolutional Encoder Model for Neural Machine Translation – Authors: Jonas Gehring, Michael Auli, David Grangier, Yann N. Dauphin
  6. Fuzzy paraphrases in learning word representations with a corpus and a lexicon – Authors: Yuanzhi Ke, Masafumi Hagiwara
  7. Iterative Refinement for Machine Translation – Authors: Roman Novak, Michael Auli, David Grangier
  8. Vocabulary Selection Strategies for Neural Machine Translation – Authors: Gurvan L’Hostis, David Grangier, Michael Auli

Language Models/Text Comprehension/Matching/Compression/Classification/++

  1. A Joint Many-Task Model: Growing a Neural Network for Multiple NLP Tasks – Authors: Kazuma Hashimoto, Caiming Xiong, Yoshimasa Tsuruoka, Richard Socher
  2. Gated-Attention Readers for Text Comprehension – Authors: Bhuwan Dhingra, Hanxiao Liu, Zhilin Yang, William W. Cohen, Ruslan Salakhutdinov
  3. A Compare-Aggregate Model for Matching Text Sequences – Authors: Shuohang Wang, Jing Jiang
  4. A Context-aware Attention Network for Interactive Question Answering – Authors: Huayu Li, Martin Renqiang Min, Yong Ge, Asim Kadav
  5. Compressing text classification models – Authors: Armand Joulin, Edouard Grave, Piotr Bojanowski, Matthijs Douze, Herve Jegou, Tomas Mikolov
  6. Multi-Agent Cooperation and the Emergence of (Natural) Language – Authors: Angeliki Lazaridou, Alexander Peysakhovich, Marco Baroni
  7. Learning a Natural Language Interface with Neural Programmer – Authors: Arvind Neelakantan, Quoc V. Le, Martin Abadi, Andrew McCallum, Dario Amodei
  8. Learning similarity preserving representations with neural similarity and context encoders – Authors: Franziska Horn, Klaus-Robert Müller
  9. Adversarial Training Methods for Semi-Supervised Text Classification – Authors: Takeru Miyato, Andrew M. Dai, Ian Goodfellow
  10. Multi-Label Learning using Tensor Decomposition for Large Text Corpora – Authors: Sayantan Dasgupta


Sign up for Deep Learning newsletter!

You may also like

Leave a Reply

Your email address will not be published. Required fields are marked *