Deep Learning for Emotion Recognition and Simulation


This blog post has recent publications about applying Deep Learning methods for emotion recognition (e.g. from voice, music, visual OR EEG input) and simulation (e.g. for robots).

The quote “Emotion is what makes us human” from a Human Computer Interaction (HCI) perspective be interpreted as: “For computers to properly communicate with humans they need to recognize human emotion, and simulate the appropriate emotion when communicating with humans”. Wikipedia describes emotion as:

    Emotion is any conscious experience characterized by intense mental activity and a high degree of pleasure or displeasure.Scientific discourse has drifted to other meanings and there is no consensus on a definition. Emotion is often intertwined with mood, temperament, personality, disposition, and motivation. In some theories, cognition is an important aspect of emotion. Those acting primarily on the emotions they are feeling may seem as if they are not thinking, but mental processes are still essential, particularly in the interpretation of events. For example, the realization of our believing that we are in a dangerous situation and the subsequent arousal of our body’s nervous system (rapid heartbeat and breathing, sweating, muscle tension) is integral to the experience of our feeling afraid. Other theories, however, claim that emotion is separate from and can precede cognition.

Best regards,
Amund Tveit

Year  Title Author
2017   End-to-End Multimodal Emotion Recognition using Deep Neural Networks  P Tzirakis, G Trigeorgis, MA Nicolaou, B Schuller
2017   EEG-based emotion recognition using hierarchical network with subnetwork nodes  Y Yang, QMJ Wu, WL Zheng, BL Lu
2017   Multimodal architecture for emotion in robots using deep learning  M Ghayoumi, AK Bansal
2017   Attentive Convolutional Neural Network based Speech Emotion Recognition: A Study on the Impact of Input Features, Signal Length, and Acted Speech  M Neumann, NT Vu
2017   Learning Affective Features with a Hybrid Deep Model for Audio-Visual Emotion Recognition  S Zhang, S Zhang, T Huang, W Gao, Q Tian
2017   A Literature Review on Emotion Recognition Using Various Methods  R Khan, O Sharif
2017   Evaluating deep learning architectures for Speech Emotion Recognition  HM Fayek, M Lech, L Cavedon
2017   Emotion Recognition from Scrambled Facial Images via Many Graph Embedding  R Jiang, ATS Ho, I Cheheb, N Al
2017   Prediction-based learning for continuous emotion recognition in speech  J Han, Z Zhang, F Ringeval, B Schuller
2017   Deep learning and SVM‐based emotion recognition from Chinese speech for smart affective services  W Zhang, D Zhao, Z Chai, LT Yang, X Liu, F Gong
2017   Deep spatio-temporal features for multimodal emotion recognition  D Nguyen, K Nguyen, S Sridharan, A Ghasemi, D Dean
2017   Imitation of human expressions based on emotion estimation by mental simulation  T Horii, Y Nagai, M Asada
2017   On Line Emotion Detection Using Retrainable Deep Neural Networks  D Kollias, A Tagaris, A Stafylopatis
2017   Quantum-inspired associative memories for incorporating emotion in a humanoid/Naoki Masuyama  M Naoki
2017   Respiration-based emotion recognition with deep learning  Q Zhang, X Chen, Q Zhan, T Yang, S Xia
2017   Wearable Biosensor Network Enabled Multimodal Daily-life Emotion Recognition Employing Reputation-driven Imbalanced Fuzzy Classification  Y Dai, X Wang, P Zhang, W Zhang
2016   Towards real-time Speech Emotion Recognition using deep neural networks  HM Fayek, M Lech, L Cavedon
2016   A Multi-task Learning Framework for Emotion Recognition Using 2D Continuous Space  R Xia, Y Liu
2016   Collaborative expression representation using peak expression and intra class variation face images for practical subject-independent emotion recognition in videos  SH Lee, WJ Baddar, YM Ro
2016   TrueHappiness: Neuromorphic Emotion Recognition on TrueNorth  PU Diehl, BU Pedroni, A Cassidy, P Merolla, E Neftci
2016   Discriminatively Trained Recurrent Neural Networks for Continuous Dimensional Emotion Recognition from Audio  F Weninger, F Ringeval, E Marchi, B Schuller
2016   Feature Transfer Learning for Speech Emotion Recognition  J Deng
2016   Emotion Recognition in Speech with Deep Learning Architectures  M Erdal, M Kächele, F Schwenker
2016   Error-correcting output codes for multi-label emotion classification  C Li, Z Feng, C Xu
2016   Software Effort Estimation Framework To Improve Organization Productivity Using Emotion Recognition Of Software Engineers In …  BP Rao, PS Ramaiah
2016   How Deep Neural Networks Can Improve Emotion Recognition on Video Data  P Khorrami, TL Paine, K Brady, C Dagli, TS Huang
2016   Automatic emotion recognition in the wild using an ensemble of static and dynamic representations  MM Ghazi, HK Ekenel
2016   HoloNet: towards robust emotion recognition in the wild  A Yao, D Cai, P Hu, S Wang, L Sha, Y Chen
2016   Deep learning driven hypergraph representation for image-based emotion recognition  Y Huang, H Lu
2016   A Review on Deep Learning Algorithms for Speech and Facial Emotion Recognition  CP Latha, M Priya
2016   Novel Affective Features For Multiscale Prediction Of Emotion In Music  N Kumar, T Guha, CW Huang, C Vaz, SS Narayanan
2016   Facial emotion detection using deep learning  DL Spiers
2016   Speech Emotion Recognition Based on Deep Belief Networks and Wavelet Packet Cepstral Coefficients.  Y Huang, A Wu, G Zhang, Y Li
2016   Audio-Video Based Multimodal Emotion Recognition Using SVMs and Deep Learning  B Sun, Q Xu, J He, L Yu, L Li, Q Wei
2016   Transfer Learning of Deep Neural Network for Speech Emotion Recognition  Y Huang, M Hu, X Yu, T Wang, C Yang
2016   Feature Learning via Deep Belief Network for Chinese Speech Emotion Recognition  S Zhang, X Zhao, Y Chuang, W Guo, Y Chen
2016   Multiagent Social Influence Detection Based on Facial Emotion Recognition  P Mishra, R Hadfi, T Ito
2016   Emotion Recognition from Speech Signals Using Deep Learning Methods  S Pathak, MV Kolhe
2016   Emotion Recognition Using Facial Expression Images for a Robotic Companion  V Palade
2016   Multimodal Emotion Recognition Using Multimodal Deep Learning  W Liu, WL Zheng, BL Lu
2016   Self-Configuring Ensemble of Neural Network Classifiers for Emotion Recognition in the Intelligent Human-Machine Interaction  E Sopov, I Ivanov
2016   The Role of Emotion and Context in Musical Preference  Y Song
2016   Facing Realism in Spontaneous Emotion Recognition from Speech: Feature Enhancement by Autoencoder with LSTM Neural Networks  Z Zhang, F Ringeval, J Han, J Deng, E Marchi
2016   The University of Passau Open Emotion Recognition System for the Multimodal Emotion Challenge  J Deng, N Cummins, J Han, X Xu, Z Ren, V Pandit
2016   Building a large scale dataset for image emotion recognition: The fine print and the benchmark  Q You, J Luo, H Jin, J Yang
2016   Emotion Prediction from User-Generated Videos by Emotion Wheel Guided Deep Learning  CT Ho, YH Lin, JL Wu
2016   Emotion Recognition Using Multimodal Deep Learning  W Liu, WL Zheng, BL Lu
2016   FDBN: Design and development of Fractional Deep Belief Networks for speaker emotion recognition  K Mannepalli, PN Sastry, M Suman
2016   A novel Adaptive Fractional Deep Belief Networks for speaker emotion recognition  K Mannepalli, PN Sastry, M Suman
2016   Affect and Legal Education: Emotion in Learning and Teaching the Law  C Maughan
2016   Unsupervised domain adaptation for speech emotion recognition using PCANet  Z Huang, W Xue, Q Mao, Y Zhan
2016   Learning Auditory Neural Representations for Emotion Recognition  P Barros, C Weber, S Wermter
2016   Towards an” In-the-Wild” Emotion Dataset Using a Game-based Framework  W Li, F Abtahi, C Tsangouri, Z Zhu
2016   Deep Learning for Emotion Recognition in Faces  A Ruiz
2016   Emotion Classification on face images  M Jorda, N Miolane, A Ng
2016   Paralinguistic Speech Recognition: Classifying Emotion in Speech with Deep Learning Neural Networks  ER Segal
2016   Architecture of Emotion in Robots Using Convolutional Neural Networks  M Ghayoumi, AK Bansal
2016   Emotion recognition from face dataset using deep neural nets  D Das, A Chakrabarty
2016   Recognize the facial emotion in video sequences using eye and mouth temporal Gabor features  PI Rani, K Muneeswaran
2016   Deep Learning Based Emotion Recognition from Chinese Speech  W Zhang, D Zhao, X Chen, Y Zhang
2016   Bi-Modal Music Emotion Recognition: Novel Lyrical Features and Dataset  R Malheiro, R Panda, P Gomes, R Paiva
2016   Speech Emotion Recognition Using Voiced Segment Selection Algorithm  Y Gu, E Postma, HX Lin, J van den Herik
2015   Multi-modal Dimensional Emotion Recognition using Recurrent Neural Networks  S Chen, Q Jin
2015   Quantification of Cinematography Semiotics for Video-based Facial Emotion Recognition in the EmotiW 2015 Grand Challenge  AC Cruz
2015   EEG Based Emotion Identification Using Unsupervised Deep Feature Learning  X Li, P Zhang, D Song, G Yu, Y Hou, B Hu
2015   Pattern-Based Emotion Classification on Social Media  E Tromp, M Pechenizkiy
2015   Investigating Critical Frequency Bands and Channels for EEG-based Emotion Recognition with Deep Neural Networks  WL Zheng, BL Lu
2015   Revealing critical channels and frequency bands for emotion recognition from EEG with deep belief network  WL Zheng, HT Guo, BL Lu
2015   Analysis of Physiological for Emotion Recognition with IRS Model  C Li, C Xu, Z Feng
2015   Emotion Recognition in the Wild via Convolutional Neural Networks and Mapped Binary Patterns  G Levi, T Hassner
2015   Negative Emotion Recognition in Spoken Dialogs  X Zhang, H Wang, L Li, M Zhao, Q Li
2015   Combining Multimodal Features within a Fusion Network for Emotion Recognition in the Wild  B Sun, L Li, G Zhou, X Wu, J He, L Yu, D Li, Q Wei
2015   Recurrent Neural Networks for Emotion Recognition in Video  S Ebrahimi Kahou, V Michalski, K Konda, R Memisevic
2015   A Deep Feature based Multi-kernel Learning Approach for Video Emotion Recognition  W Li, F Abtahi, Z Zhu
2015   Learning Speech Emotion Features by Joint Disentangling-Discrimination  W Xue, Z Huang, X Luo, Q Mao
2015   Data selection for acoustic emotion recognition: Analyzing and comparing utterance and sub-utterance selection strategies  D Le, EM Provost
2015   Leveraging Inter-rater Agreement for Audio-Visual Emotion Recognition  Y Kim, EM Provost
2015   The Research on Cross-Language Emotion Recognition Algorithm for Hearing Aid  X Shulan, W Jilin
2015   Optimized multi-channel deep neural network with 2D graphical representation of acoustic speech features for emotion recognition  MN Stolar, M Lech, IS Burnett
2015   EmoNets: Multimodal deep learning approaches for emotion recognition in video  SE Kahou, X Bouthillier, P Lamblin, C Gulcehre
2015   Deep learninig of EEG signals for emotion recognition  Y Gao, HJ Lee, RM Mehmood
2015   Emotion Recognition & Classification using Neural Networks  K Koupidis, A Ioannis
2015   Emotion recognition from embedded bodily expressions and speech during dyadic interactions  PM Müller, S Amin, P Verma, M Andriluka, A Bulling
2015   Speech emotion recognition with unsupervised feature learning  Z HUANG, W XUE, Q MAO
2015   Emotion identification by facial landmarks dynamics analysis  A Bandrabur, L Florea, C Florea, M Mancas
2014   Speech Emotion Recognition Using CNN  Z Huang, M Dong, Q Mao, Y Zhan
2014   Multi-scale Temporal Modeling for Dimensional Emotion Recognition in Video  L Chao, J Tao, M Yang, Y Li, Z Wen
2014   Improving generation performance of speech emotion recognition by denoising autoencoders  L Chao, J Tao, M Yang, Y Li
2014   Acoustic emotion recognition using deep neural network  J Niu, Y Qian, K Yu
2014   Prosodic, spectral and voice quality feature selection using a long-term stopping criterion for audio-based emotion recognition  M Kächele, D Zharkov, S Meudt, F Schwenker
2014   Emotion Recognition in the Wild with Feature Fusion and Multiple Kernel Learning  JK Chen, Z Chen, Z Chi, H Fu
2014   Emotion Modeling and Machine Learning in Affective Computing  K Kim
2014   A Study of Deep Belief Network Based Chinese Speech Emotion Recognition  B Chen, Q Yin, P Guo
Continue Reading