Deep Learning for Question Answering

This posting presents recent publications related to Deep Learning for Question Answering. Question Answering is described as “a computer science discipline within the fields of information retrieval and natural language processing (NLP), which is concerned with building systems that automatically answer questions posed by humans in a natural language”. I’ll also publish postings about Deep Learning for Information Retrieval and Learning to Rank today.

Best regards,
Amund Tveit

Year  Title Author
2016   Skipping Word: A Character-Sequential Representation based Framework for Question Answering  L Meng, Y Li, M Liu, P Shu
2016   Semantic computation in geography question answering  S Zhao, Y Zheng, C Zhu, T Zhao, S Li
2016   Open-ended visual question answering  I Masuda Mora
2016   Creating Causal Embeddings for Question Answering with Minimal Supervision  R Sharp, M Surdeanu, P Jansen, P Clark, M Hammond
2016   Deep Feature Fusion Network for Answer Quality Prediction in Community Question Answering  SP Suggu, KN Goutham, MK Chinnakotla
2016   ECNU at SemEval-2016 Task 3: Exploring traditional method and deep learning method for question retrieval and answer ranking in community question answering  G Wu, M Lan
2016   Ask Your Neurons: A Deep Learning Approach to Visual Question Answering  M Malinowski, M Rohrbach, M Fritz
2016   Revisiting Visual Question Answering Baselines  A Jabri, A Joulin, L van der Maaten
2016   End to End Long Short Term Memory Networks for Non-Factoid Question Answering  D Cohen, WB Croft
2016   Question Answering on Linked Data: Challenges and Future Directions  S Shekarpour, D Lukovnikov, AJ Kumar, K Endris
2016   Extracting Medical Knowledge from Crowdsourced Question Answering Website  Y Li, C Liu, N Du, W Fan, Q Li, J Gao, C Zhang, H Wu
2015   Building a Large-scale Multimodal Knowledge Base for Visual Question Answering  Y Zhu, C Zhang, C Ré, L Fei
2015   Answer Sequence Learning with Neural Networks for Answer Selection in Community Question Answering  X Zhou, B Hu, Q Chen, B Tang, X Wang
2015   Empirical Study on Deep Learning Models for Question Answering  Y Yu, W Zhang, CW Hang, B Zhou
2015   Simple Baseline for Visual Question Answering  B Zhou, Y Tian, S Sukhbaatar, A Szlam, R Fergus
2015   Chess Q&A: Question Answering on Chess Games  V Cirik, LP Morency, E Hovy
2015   Predicting the Quality of User-Generated Answers Using Co-Training in Community-based Question Answering Portals  B Liu, J Feng, M Liu, H Hu, X Wang
2015   Open Domain Question Answering via Semantic Enrichment  H Sun, H Ma, W Yih, CT Tsai, J Liu, MW Chang
2015   Visual Question Answering using Deep Learning  S Agrawal, A Mukherjee
2015   Learning Semantic Representation with Neural Networks for Community Question Answering Retrieval  G Zhou, Y Zhou, T He, W Wu
2015   WIKIQA: A Challenge Dataset for Open-Domain Question Answering  Y Yang, W Yih, C Meek
Continue Reading

Deep Learning for Sentiment Analysis

Recently I published Embedding for NLP with Deep Learning (e.g. word2vec and follow-ups) and Deep Learning for Natural Language Processing – ICLR 2017 Discoveries – this posting is also mostly NLP-related since it provides recent papers related to Deep Learning for Sentiment Analysis, but also has examples of other types of sentiment (e.g. image sentiment).

Best regards,
Amund Tveit

Year  Title Author
2016   Image Sentiment Analysis using Deep Convolutional Neural Networks with Domain Specific Fine Tuning  S Jindal, S Singh
2016   Multi-Objective Model Selection (MOMS)-based Semi-Supervised Framework for Sentiment Analysis  FH Khan, U Qamar, S Bashir
2016   Real-Time Topic and Sentiment Analysis in Human-Robot Conversation  E Russell
2016   Finki at SemEval-2016 Task 4: Deep Learning Architecture for Twitter Sentiment Analysis  D Stojanovski, G Strezoski, G Madjarov, I Dimitrovski
2016   Design of Sentiment Analysis System for Hindi Content  M Yadav, V Bhojane
2016   Visual Sentiment Analysis for Social Images Using Transfer Learning Approach  J Islam, Y Zhang
2016   Unsupervised Feature Learning Assisted Visual Sentiment Analysis  Z Li, Y Fan, F Wang, W Liu
2016   Context-Aware Text Representation for Social Relation Aided Sentiment Analysis  LM Nguyen
2016   Select-Additive Learning: Improving Cross-individual Generalization in Multimodal Sentiment Analysis  H Wang, A Meghawat, LP Morency, EP Xing
2016   A semi-supervised approach to sentiment analysis using revised sentiment strength based on SentiWordNet  FH Khan, U Qamar, S Bashir
2016   SWIMS: Semi-Supervised Subjective Feature Weighting and Intelligent Model Selection for Sentiment Analysis  FH Khan, U Qamar, S Bashir
2016   Cross-modality Consistent Regression for Joint Visual-Textual Sentiment Analysis of Social Multimedia  Q You, J Luo, H Jin, J Yang
2016   INSIGHT-1 at SemEval-2016 Task 5: Deep Learning for Multilingual Aspect-based Sentiment Analysis  S Ruder12, P Ghaffari, JG Breslin
2016   SENSEI-LIF at SemEval-2016 Task 4: Polarity embedding fusion for robust sentiment analysis  M Rouvier, B Favre
2016   Leveraging Multimodal Information for Event Summarization and Concept-level Sentiment Analysis  RR Shah, Y Yu, A Verma, S Tang, AD Shaikh
2016   Visual and Textual Sentiment Analysis of a Microblog Using Deep Convolutional Neural Networks  Y Yu, H Lin, J Meng, Z Zhao
2016   Sentiment Analysis for Chinese Microblog based on Deep Neural Networks with Convolutional Extension Features  SUN Xiao, LI Chengcheng, REN Fuji
2016   PotTS at SemEval-2016 Task 4: Sentiment Analysis of Twitter Using Character-level Convolutional Neural Networks.  U Sidarenka, KL Straße
2016   Recursive Neural Conditional Random Fields for Aspect-based Sentiment Analysis  W Wang, SJ Pan, D Dahlmeier, X Xiao
2016   eSAP: A Decision Support Framework for Enhanced Sentiment Analysis and Polarity Classification  FH Khan, U Qamar, S Bashir
2016   Paragraph2Vec-based sentiment analysis on social media for business in Thailand  P Sanguansat
2016   Sentiment Analysis of Chinese Micro Blog Based on DNN and ELM and Vector Space Model  H Liu, S Li, C Jiang, H Liu
2016   Sentiment Analysis in Finance Market Forcast  D Wang
2016   Visual Sentiment Analysis with Network in Network  Z Li, Y Fan, F Wang
2015   Convolutional Neural Networks for Multimedia Sentiment Analysis  G Cai, B Xia
2015   Sentiment Analysis with Incremental Human-in-the-Loop Learning and Lexical Resource Customization  S Mishra, J Diesner, J Byrne, E Surbeck
2015   Parallel Recursive Deep Model for Sentiment Analysis  G Tian, Y Zhou
2015   Unsupervised Sentiment Analysis for Social Media Images  Y Wang, S Wang, J Tang, H Liu, B Li
2015   Prediction of changes in the stock market using twitter and sentiment analysis  IV Serban, DS González, X Wu
2015   Review Sentiment Analysis Based on Deep Learning  Z Hu, J Hu, W Ding, X Zheng
2015   Twitter Sentiment Analysis Using Deep Convolutional Neural Network  D Stojanovski, G Strezoski, G Madjarov, I Dimitrovski
2015   Sentiment-Specific Representation Learning for Document-Level Sentiment Analysis  D Tang
2015   Sentiment Analysis via Integrating Distributed Representations of Variable-length Word Sequence  Z Cui, X Shi, Y Chen
2015   Learning Higher-Level Features with Convolutional Restricted Boltzmann Machines for Sentiment Analysis  T Huynh, Y He, S Rüger
2015   Recursive Autoencoder with HowNet Lexicon for Sentence-Level Sentiment Analysis  X Fu, Y Xu
2014   Recursive Deep Learning for Sentiment Analysis over Social Data  C Li, B Xu, G Wu, S He, G Tian, H Hao
2014   Robust Image Sentiment Analysis Using Progressively Trained and Domain Transferred Deep Networks  Q You, J Luo, H Jin, J Yang
Continue Reading

Embedding for NLP with Deep Learning

Word Embedding was introduced by Bengio in early 2000s, and interest in it really accelerated when Google presented Word2Vec in 2013.

This blog post has recent papers related to embedding for Natural Language Processing with Deep Learning. Example application areas embedding is used for in the papers include finance (stock market prediction), biomedical text analysis, part-of-speech tagging, sentiment analysis, pharmacology (drug adverse effects).

I recommend you to start with the paper: In Defense of Word Embedding for Generic Text Representation

Best regards,

Amund Tveit

  1. An approach to the use of word embeddings in an opinion classification task
    – Authors: F Enríquez, Ja Troyano, T López (2016)
  2. Learning Document Embeddings by Predicting N-grams for Sentiment Classification of Long Movie Reviews
    – Authors: B Li, T Liu, X Du, D Zhang, Z Zhao (2016)
  3. A Distributed Chinese Naive Bayes Classifier Based on Word Embedding
    – Authors: M Feng, G Wu (2016)
  4. An Empirical Study on Sentiment Classification of Chinese Review using Word Embedding
    – Authors: Y Lin, H Lei, J Wu, X Li (2015)
  5. Learning Bilingual Embedding Model for Cross-Language Sentiment Classification
    – Authors: X Tang, X Wan (2014)
  6. Training word embeddings for deep learning in biomedical text mining tasks
    – Authors: Z Jiang, L Li, D Huang, L Jin (2016)
  7. Learning Dense Convolutional Embeddings For Semantic Segmentation
    – Authors: Aw Harley, Kg Derpanis, I Kokkinos (2016)
  8. Creating Causal Embeddings for Question Answering with Minimal Supervision
    – Authors: R Sharp, M Surdeanu, P Jansen, P Clark, M Hammond (2016)
  9. Discriminative Phrase Embedding for Paraphrase Identification
    – Authors: W Yin, H Schütze (2016)
  10. Word embedding based retrieval model for similar cases recommendation
    – Authors: Y Zhao, J Wang, F Wang (2016)
  11. Learning Embeddings of API Tokens to Facilitate Deep Learning Based Program Processing
    – Authors: Y Lu, G Li, R Miao, Z Jin (2016)
  12. Locally Embedding Autoencoders: A Semi-Supervised Manifold Learning Approach of Document Representation
    – Authors: C Wei, S Luo, X Ma, H Ren, J Zhang, L Pan (2016)
  13. Deep Learning Architecture for Part-of-Speech Tagging with Word and Suffix Embeddings
    – Authors: A Popov (2016)
  14. Robobarista: Learning to Manipulate Novel Objects via Deep Multimodal Embedding
    – Authors: J Sung, Sh Jin, I Lenz, A Saxena (2016)
  15. Pruning subsequence search with attention-based embedding
    – Authors: C Raffel, Dpw Ellis (2016)
  16. Sentence Embedding Evaluation Using Pyramid Annotation
    – Authors: T Baumel, R Cohen, M Elhadad (2016)
  17. Deep Sentence Embedding Using the Long Short Term Memory Network: Analysis and Application to Information Retrieval
    – Authors: H Palangi, L Deng, Y Shen, J Gao, X He, J Chen… (2015)
  18. Feedback recurrent neural network-based embedded vector and its application in topic model
    – Authors: L Li, S Gan, X Yin (2016)
  19. Enhancing Sentence Relation Modeling with Auxiliary Character-level Embedding
    – Authors: P Li, H Huang (2016)
  20. Learning Word Meta-Embeddings by Using Ensembles of Embedding Sets
    – Authors: W Yin, H Schütze (2015)
  21. Syntax-Aware Multi-Sense Word Embeddings for Deep Compositional Models of Meaning
    – Authors: J Cheng, D Kartsaklis (2015)
  22. A Deep Embedding Model for Co-occurrence Learning
    – Authors: Y Shen, R Jin, J Chen, X He, J Gao, L Deng (2015)
  23. Jointly Modeling Embedding and Translation to Bridge Video and Language
    – Authors: Y Pan, T Mei, T Yao, H Li, Y Rui (2015)
  24. Learning semantic word embeddings based on ordinal knowledge constraints
    – Authors: Q Liu, H Jiang, S Wei, Zh Ling, Y Hu (2015)
  25. Boosting Named Entity Recognition with Neural Character Embeddings
    – Authors: C Dos Santos, V Guimaraes, Rj Niterói, R De Janeiro (2015)
  26. Evaluating word embeddings and a revised corpus for part-of-speech tagging in Portuguese
    – Authors: Er Fonseca, Jlg Rosa, Sm Aluísio (2015)
  27. Temporal Embedding in Convolutional Neural Networks for Robust Learning of Abstract Snippets
    – Authors: J Liu, K Zhao, B Kusy, J Wen, R Jurdak (2015)
  28. Learning Feature Hierarchies: A Layer-wise Tag-embedded Approach
    – Authors: Z Yuan, C Xu, J Sang, S Yan, M Hossain (2015)
  29. Multi-Source Bayesian Embeddings for Learning Social Knowledge Graphs
    – Authors: Z Yang, J Tang (2015)
  30. PTE: Predictive Text Embedding through Large-scale Heterogeneous Text Networks
    – Authors: J Tang, M Qu, Q Mei (2015)
  31. Pharmacovigilance from social media: mining adverse drug reaction mentions using sequence labeling with word embedding cluster features
    – Authors: A Nikfarjam, A Sarker, K O’Connor, R Ginn, G Gonzalez (2015)
  32. Projective Label Propagation by Label Embedding
    – Authors: Z Zhang, W Jiang, F Li, L Zhang, M Zhao, L Jia (2015)
  33. An Investigation of Neural Embeddings for Coreference Resolution
    – Authors: V Godbole, W Liu, R Togneri (2015)
  34. Deep Multimodal Embedding: Manipulating Novel Objects with Point-clouds, Language and Trajectories
    – Authors: J Sung, I Lenz, A Saxena (2015)
  35. AutoExtend: Extending Word Embeddings to Embeddings for Synsets and Lexemes
    – Authors: S Rothe, H Schütze (2015)
  36. Deep Multilingual Correlation for Improved Word Embeddings
    – Authors: A Lu, W Wang, M Bansal, K Gimpel, K Livescu (2015)
  37. Leverage Financial News to Predict Stock Price Movements Using Word Embeddings and Deep Neural Networks
    – Authors: Y Peng, H Jiang (2015)
  38. The Impact of Structured Event Embeddings on Scalable Stock Forecasting Models
    – Authors: Jb Nascimento, M Cristo (2015)
  39. Representing Text for Joint Embedding of Text and Knowledge Bases
    – Authors: K Toutanova, D Chen, P Pantel, H Poon, P Choudhury… (2015)
  40. In Defense of Word Embedding for Generic Text Representation
    – Authors: G Lev, B Klein, L Wolf (2015)
  41. Learning Multi-Relational Semantics Using Neural-Embedding Models
    – Authors: B Yang, W Yih, X He, J Gao, L Deng (2014)
  42. Improving relation descriptor extraction with word embeddings and cluster features
    – Authors: T Liu, M Li (2014)
Continue Reading

Deep Learning for Natural Language Processing – ICLR 2017 Discoveries

Update: 2017-Feb-03 – launched new service – ai.amundtveit.com (navigation and search in papers). Try e.g. out its Natural Language Processing Page.


The 5th International Conference on Learning Representation (ICLR 2017) is coming to Toulon, France (April 24-26 2017), and there is large amount of Deep Learning papers submitted to the conference, looks like it will be a great event (see word cloud below for most frequent words used in submitted paper titles).

iclr2017wordcloud

This blog post gives an overview of Natural Language Processing related papers submitted to ICLR 2017, see underneath for the list of papers. If you want to learn about Deep Learning with NLP check out Stanford’s CS224d: Deep Learning for Natural Language Processing

Best regards,

Amund Tveit

ICLR 2017 – NLP PAPERs

Character/Word/Sentence Representation

  1. Character-aware Attention Residual Network for Sentence Representation – Authors: Xin Zheng, Zhenzhou Wu
  2. Program Synthesis for Character Level Language Modeling – Authors: Pavol Bielik, Veselin Raychev, Martin Vechev
  3. Words or Characters? Fine-grained Gating for Reading Comprehension – Authors: Zhilin Yang, Bhuwan Dhingra, Ye Yuan, Junjie Hu, William W. Cohen, Ruslan Salakhutdinov
  4. Deep Character-Level Neural Machine Translation By Learning Morphology – Authors: Shenjian Zhao, Zhihua Zhang
  5. Opening the vocabulary of neural language models with character-level word representations – Authors: Matthieu Labeau, Alexandre Allauzen
  6. Unsupervised sentence representation learning with adversarial auto-encoder – Authors: Shuai Tang, Hailin Jin, Chen Fang, Zhaowen Wang
  7. Offline Bilingual Word Vectors Without a Dictionary – Authors: Samuel L. Smith, David H. P. Turban, Nils Y. Hammerla, Steven Hamblin
  8. Learning Word-Like Units from Joint Audio-Visual Analylsis – Authors: David Harwath, James R. Glass
  9. Tying Word Vectors and Word Classifiers: A Loss Framework for Language Modeling – Authors: Hakan Inan, Khashayar Khosravi, Richard Socher
  10. Sentence Ordering using Recurrent Neural Networks – Authors: Lajanugen Logeswaran, Honglak Lee, Dragomir Radev

Search/Question-Answer/Recommender Systems

  1. Learning to Query, Reason, and Answer Questions On Ambiguous Texts – Authors: Xiaoxiao Guo, Tim Klinger, Clemens Rosenbaum, Joseph P. Bigus, Murray Campbell, Ban Kawas, Kartik Talamadupula, Gerry Tesauro, Satinder Singh
  2. Group Sparse CNNs for Question Sentence Classification with Answer Sets – Authors: Mingbo Ma, Liang Huang, Bing Xiang, Bowen Zhou
  3. CONTENT2VEC: Specializing Joint Representations of Product Images and Text for the task of Product Recommendation – Authors: Thomas Nedelec, Elena Smirnova, Flavian Vasile
  4. Is a picture worth a thousand words? A Deep Multi-Modal Fusion Architecture for Product Classification in e-commerce – Authors: Tom Zahavy, Alessandro Magnani, Abhinandan Krishnan, Shie Mannor

Word/Sentence Embedding

  1. A Simple but Tough-to-Beat Baseline for Sentence Embeddings – Authors: Sanjeev Arora, Yingyu Liang, Tengyu Ma
  2. Investigating Different Context Types and Representations for Learning Word Embeddings – Authors: Bofang Li, Tao Liu, Zhe Zhao, Xiaoyong Du
  3. Multi-view Recurrent Neural Acoustic Word Embeddings – Authors: Wanjia He, Weiran Wang, Karen Livescu
  4. A Self-Attentive Sentence Embedding – Authors: Zhouhan Lin, Minwei Feng, Cicero Nogueira dos Santos, Mo Yu, Bing Xiang, Bowen Zhou, Yoshua Bengio
  5. Fine-grained Analysis of Sentence Embeddings Using Auxiliary Prediction Tasks – Authors: Yossi Adi, Einat Kermany, Yonatan Belinkov, Ofer Lavi, Yoav Goldberg

Multilingual/Translation/Sentiment

  1. Neural Machine Translation with Latent Semantic of Image and Text – Authors: Joji Toyama, Masanori Misono, Masahiro Suzuki, Kotaro Nakayama, Yutaka Matsuo
  2. Beyond Bilingual: Multi-sense Word Embeddings using Multilingual Context – Authors: Shyam Upadhyay, Kai-Wei Chang, James Zhou, Matt Taddy, Adam Kalai
  3. Learning to Understand: Incorporating Local Contexts with Global Attention for Sentiment Classification – Authors: Zhigang Yuan, Yuting Hu, Yongfeng Huang
  4. Adaptive Feature Abstraction for Translating Video to Language – Authors: Yunchen Pu, Martin Renqiang Min, Zhe Gan, Lawrence Carin
  5. A Convolutional Encoder Model for Neural Machine Translation – Authors: Jonas Gehring, Michael Auli, David Grangier, Yann N. Dauphin
  6. Fuzzy paraphrases in learning word representations with a corpus and a lexicon – Authors: Yuanzhi Ke, Masafumi Hagiwara
  7. Iterative Refinement for Machine Translation – Authors: Roman Novak, Michael Auli, David Grangier
  8. Vocabulary Selection Strategies for Neural Machine Translation – Authors: Gurvan L’Hostis, David Grangier, Michael Auli

Language Models/Text Comprehension/Matching/Compression/Classification/++

  1. A Joint Many-Task Model: Growing a Neural Network for Multiple NLP Tasks – Authors: Kazuma Hashimoto, Caiming Xiong, Yoshimasa Tsuruoka, Richard Socher
  2. Gated-Attention Readers for Text Comprehension – Authors: Bhuwan Dhingra, Hanxiao Liu, Zhilin Yang, William W. Cohen, Ruslan Salakhutdinov
  3. A Compare-Aggregate Model for Matching Text Sequences – Authors: Shuohang Wang, Jing Jiang
  4. A Context-aware Attention Network for Interactive Question Answering – Authors: Huayu Li, Martin Renqiang Min, Yong Ge, Asim Kadav
  5. FastText.zip: Compressing text classification models – Authors: Armand Joulin, Edouard Grave, Piotr Bojanowski, Matthijs Douze, Herve Jegou, Tomas Mikolov
  6. Multi-Agent Cooperation and the Emergence of (Natural) Language – Authors: Angeliki Lazaridou, Alexander Peysakhovich, Marco Baroni
  7. Learning a Natural Language Interface with Neural Programmer – Authors: Arvind Neelakantan, Quoc V. Le, Martin Abadi, Andrew McCallum, Dario Amodei
  8. Learning similarity preserving representations with neural similarity and context encoders – Authors: Franziska Horn, Klaus-Robert Müller
  9. Adversarial Training Methods for Semi-Supervised Text Classification – Authors: Takeru Miyato, Andrew M. Dai, Ian Goodfellow
  10. Multi-Label Learning using Tensor Decomposition for Large Text Corpora – Authors: Sayantan Dasgupta

 

Sign up for Deep Learning newsletter!


Continue Reading