Deep Learning for Emotion Recognition and Simulation

robotfeelings

This blog post has recent publications about applying Deep Learning methods for emotion recognition (e.g. from voice, music, visual OR EEG input) and simulation (e.g. for robots).

The quote “Emotion is what makes us human” from a Human Computer Interaction (HCI) perspective be interpreted as: “For computers to properly communicate with humans they need to recognize human emotion, and simulate the appropriate emotion when communicating with humans”. Wikipedia describes emotion as:

    Emotion is any conscious experience characterized by intense mental activity and a high degree of pleasure or displeasure.Scientific discourse has drifted to other meanings and there is no consensus on a definition. Emotion is often intertwined with mood, temperament, personality, disposition, and motivation. In some theories, cognition is an important aspect of emotion. Those acting primarily on the emotions they are feeling may seem as if they are not thinking, but mental processes are still essential, particularly in the interpretation of events. For example, the realization of our believing that we are in a dangerous situation and the subsequent arousal of our body’s nervous system (rapid heartbeat and breathing, sweating, muscle tension) is integral to the experience of our feeling afraid. Other theories, however, claim that emotion is separate from and can precede cognition.

Best regards,
Amund Tveit

Year  Title Author
2017   End-to-End Multimodal Emotion Recognition using Deep Neural Networks  P Tzirakis, G Trigeorgis, MA Nicolaou, B Schuller
2017   EEG-based emotion recognition using hierarchical network with subnetwork nodes  Y Yang, QMJ Wu, WL Zheng, BL Lu
2017   Multimodal architecture for emotion in robots using deep learning  M Ghayoumi, AK Bansal
2017   Attentive Convolutional Neural Network based Speech Emotion Recognition: A Study on the Impact of Input Features, Signal Length, and Acted Speech  M Neumann, NT Vu
2017   Learning Affective Features with a Hybrid Deep Model for Audio-Visual Emotion Recognition  S Zhang, S Zhang, T Huang, W Gao, Q Tian
2017   A Literature Review on Emotion Recognition Using Various Methods  R Khan, O Sharif
2017   Evaluating deep learning architectures for Speech Emotion Recognition  HM Fayek, M Lech, L Cavedon
2017   Emotion Recognition from Scrambled Facial Images via Many Graph Embedding  R Jiang, ATS Ho, I Cheheb, N Al
2017   Prediction-based learning for continuous emotion recognition in speech  J Han, Z Zhang, F Ringeval, B Schuller
2017   Deep learning and SVM‐based emotion recognition from Chinese speech for smart affective services  W Zhang, D Zhao, Z Chai, LT Yang, X Liu, F Gong
2017   Deep spatio-temporal features for multimodal emotion recognition  D Nguyen, K Nguyen, S Sridharan, A Ghasemi, D Dean
2017   Imitation of human expressions based on emotion estimation by mental simulation  T Horii, Y Nagai, M Asada
2017   On Line Emotion Detection Using Retrainable Deep Neural Networks  D Kollias, A Tagaris, A Stafylopatis
2017   Quantum-inspired associative memories for incorporating emotion in a humanoid/Naoki Masuyama  M Naoki
2017   Respiration-based emotion recognition with deep learning  Q Zhang, X Chen, Q Zhan, T Yang, S Xia
2017   Wearable Biosensor Network Enabled Multimodal Daily-life Emotion Recognition Employing Reputation-driven Imbalanced Fuzzy Classification  Y Dai, X Wang, P Zhang, W Zhang
2016   Towards real-time Speech Emotion Recognition using deep neural networks  HM Fayek, M Lech, L Cavedon
2016   A Multi-task Learning Framework for Emotion Recognition Using 2D Continuous Space  R Xia, Y Liu
2016   Collaborative expression representation using peak expression and intra class variation face images for practical subject-independent emotion recognition in videos  SH Lee, WJ Baddar, YM Ro
2016   TrueHappiness: Neuromorphic Emotion Recognition on TrueNorth  PU Diehl, BU Pedroni, A Cassidy, P Merolla, E Neftci
2016   Discriminatively Trained Recurrent Neural Networks for Continuous Dimensional Emotion Recognition from Audio  F Weninger, F Ringeval, E Marchi, B Schuller
2016   Feature Transfer Learning for Speech Emotion Recognition  J Deng
2016   Emotion Recognition in Speech with Deep Learning Architectures  M Erdal, M Kächele, F Schwenker
2016   Error-correcting output codes for multi-label emotion classification  C Li, Z Feng, C Xu
2016   Software Effort Estimation Framework To Improve Organization Productivity Using Emotion Recognition Of Software Engineers In …  BP Rao, PS Ramaiah
2016   How Deep Neural Networks Can Improve Emotion Recognition on Video Data  P Khorrami, TL Paine, K Brady, C Dagli, TS Huang
2016   Automatic emotion recognition in the wild using an ensemble of static and dynamic representations  MM Ghazi, HK Ekenel
2016   HoloNet: towards robust emotion recognition in the wild  A Yao, D Cai, P Hu, S Wang, L Sha, Y Chen
2016   Deep learning driven hypergraph representation for image-based emotion recognition  Y Huang, H Lu
2016   A Review on Deep Learning Algorithms for Speech and Facial Emotion Recognition  CP Latha, M Priya
2016   Novel Affective Features For Multiscale Prediction Of Emotion In Music  N Kumar, T Guha, CW Huang, C Vaz, SS Narayanan
2016   Facial emotion detection using deep learning  DL Spiers
2016   Speech Emotion Recognition Based on Deep Belief Networks and Wavelet Packet Cepstral Coefficients.  Y Huang, A Wu, G Zhang, Y Li
2016   Audio-Video Based Multimodal Emotion Recognition Using SVMs and Deep Learning  B Sun, Q Xu, J He, L Yu, L Li, Q Wei
2016   Transfer Learning of Deep Neural Network for Speech Emotion Recognition  Y Huang, M Hu, X Yu, T Wang, C Yang
2016   Feature Learning via Deep Belief Network for Chinese Speech Emotion Recognition  S Zhang, X Zhao, Y Chuang, W Guo, Y Chen
2016   Multiagent Social Influence Detection Based on Facial Emotion Recognition  P Mishra, R Hadfi, T Ito
2016   Emotion Recognition from Speech Signals Using Deep Learning Methods  S Pathak, MV Kolhe
2016   Emotion Recognition Using Facial Expression Images for a Robotic Companion  V Palade
2016   Multimodal Emotion Recognition Using Multimodal Deep Learning  W Liu, WL Zheng, BL Lu
2016   Self-Configuring Ensemble of Neural Network Classifiers for Emotion Recognition in the Intelligent Human-Machine Interaction  E Sopov, I Ivanov
2016   The Role of Emotion and Context in Musical Preference  Y Song
2016   Facing Realism in Spontaneous Emotion Recognition from Speech: Feature Enhancement by Autoencoder with LSTM Neural Networks  Z Zhang, F Ringeval, J Han, J Deng, E Marchi
2016   The University of Passau Open Emotion Recognition System for the Multimodal Emotion Challenge  J Deng, N Cummins, J Han, X Xu, Z Ren, V Pandit
2016   Building a large scale dataset for image emotion recognition: The fine print and the benchmark  Q You, J Luo, H Jin, J Yang
2016   Emotion Prediction from User-Generated Videos by Emotion Wheel Guided Deep Learning  CT Ho, YH Lin, JL Wu
2016   Emotion Recognition Using Multimodal Deep Learning  W Liu, WL Zheng, BL Lu
2016   FDBN: Design and development of Fractional Deep Belief Networks for speaker emotion recognition  K Mannepalli, PN Sastry, M Suman
2016   A novel Adaptive Fractional Deep Belief Networks for speaker emotion recognition  K Mannepalli, PN Sastry, M Suman
2016   Affect and Legal Education: Emotion in Learning and Teaching the Law  C Maughan
2016   Unsupervised domain adaptation for speech emotion recognition using PCANet  Z Huang, W Xue, Q Mao, Y Zhan
2016   Learning Auditory Neural Representations for Emotion Recognition  P Barros, C Weber, S Wermter
2016   Towards an” In-the-Wild” Emotion Dataset Using a Game-based Framework  W Li, F Abtahi, C Tsangouri, Z Zhu
2016   Deep Learning for Emotion Recognition in Faces  A Ruiz
2016   Emotion Classification on face images  M Jorda, N Miolane, A Ng
2016   Paralinguistic Speech Recognition: Classifying Emotion in Speech with Deep Learning Neural Networks  ER Segal
2016   Architecture of Emotion in Robots Using Convolutional Neural Networks  M Ghayoumi, AK Bansal
2016   Emotion recognition from face dataset using deep neural nets  D Das, A Chakrabarty
2016   Recognize the facial emotion in video sequences using eye and mouth temporal Gabor features  PI Rani, K Muneeswaran
2016   Deep Learning Based Emotion Recognition from Chinese Speech  W Zhang, D Zhao, X Chen, Y Zhang
2016   Bi-Modal Music Emotion Recognition: Novel Lyrical Features and Dataset  R Malheiro, R Panda, P Gomes, R Paiva
2016   Speech Emotion Recognition Using Voiced Segment Selection Algorithm  Y Gu, E Postma, HX Lin, J van den Herik
2015   Multi-modal Dimensional Emotion Recognition using Recurrent Neural Networks  S Chen, Q Jin
2015   Quantification of Cinematography Semiotics for Video-based Facial Emotion Recognition in the EmotiW 2015 Grand Challenge  AC Cruz
2015   EEG Based Emotion Identification Using Unsupervised Deep Feature Learning  X Li, P Zhang, D Song, G Yu, Y Hou, B Hu
2015   Pattern-Based Emotion Classification on Social Media  E Tromp, M Pechenizkiy
2015   Investigating Critical Frequency Bands and Channels for EEG-based Emotion Recognition with Deep Neural Networks  WL Zheng, BL Lu
2015   Revealing critical channels and frequency bands for emotion recognition from EEG with deep belief network  WL Zheng, HT Guo, BL Lu
2015   Analysis of Physiological for Emotion Recognition with IRS Model  C Li, C Xu, Z Feng
2015   Emotion Recognition in the Wild via Convolutional Neural Networks and Mapped Binary Patterns  G Levi, T Hassner
2015   Negative Emotion Recognition in Spoken Dialogs  X Zhang, H Wang, L Li, M Zhao, Q Li
2015   Combining Multimodal Features within a Fusion Network for Emotion Recognition in the Wild  B Sun, L Li, G Zhou, X Wu, J He, L Yu, D Li, Q Wei
2015   Recurrent Neural Networks for Emotion Recognition in Video  S Ebrahimi Kahou, V Michalski, K Konda, R Memisevic
2015   A Deep Feature based Multi-kernel Learning Approach for Video Emotion Recognition  W Li, F Abtahi, Z Zhu
2015   Learning Speech Emotion Features by Joint Disentangling-Discrimination  W Xue, Z Huang, X Luo, Q Mao
2015   Data selection for acoustic emotion recognition: Analyzing and comparing utterance and sub-utterance selection strategies  D Le, EM Provost
2015   Leveraging Inter-rater Agreement for Audio-Visual Emotion Recognition  Y Kim, EM Provost
2015   The Research on Cross-Language Emotion Recognition Algorithm for Hearing Aid  X Shulan, W Jilin
2015   Optimized multi-channel deep neural network with 2D graphical representation of acoustic speech features for emotion recognition  MN Stolar, M Lech, IS Burnett
2015   EmoNets: Multimodal deep learning approaches for emotion recognition in video  SE Kahou, X Bouthillier, P Lamblin, C Gulcehre
2015   Deep learninig of EEG signals for emotion recognition  Y Gao, HJ Lee, RM Mehmood
2015   Emotion Recognition & Classification using Neural Networks  K Koupidis, A Ioannis
2015   Emotion recognition from embedded bodily expressions and speech during dyadic interactions  PM Müller, S Amin, P Verma, M Andriluka, A Bulling
2015   Speech emotion recognition with unsupervised feature learning  Z HUANG, W XUE, Q MAO
2015   Emotion identification by facial landmarks dynamics analysis  A Bandrabur, L Florea, C Florea, M Mancas
2014   Speech Emotion Recognition Using CNN  Z Huang, M Dong, Q Mao, Y Zhan
2014   Multi-scale Temporal Modeling for Dimensional Emotion Recognition in Video  L Chao, J Tao, M Yang, Y Li, Z Wen
2014   Improving generation performance of speech emotion recognition by denoising autoencoders  L Chao, J Tao, M Yang, Y Li
2014   Acoustic emotion recognition using deep neural network  J Niu, Y Qian, K Yu
2014   Prosodic, spectral and voice quality feature selection using a long-term stopping criterion for audio-based emotion recognition  M Kächele, D Zharkov, S Meudt, F Schwenker
2014   Emotion Recognition in the Wild with Feature Fusion and Multiple Kernel Learning  JK Chen, Z Chen, Z Chi, H Fu
2014   Emotion Modeling and Machine Learning in Affective Computing  K Kim
2014   A Study of Deep Belief Network Based Chinese Speech Emotion Recognition  B Chen, Q Yin, P Guo
Continue Reading

Deep Learning for Emotion Recognition and Analysis

User interfaces can gain from getting a better understanding of human emotion. This blog post has recent papers related to Deep Learning and Emotion, note that Emotion and Deep Learning has also been previously to some degree been in previous blog posts: Deep Learning with Long Short-Term Memory (LSTM), Deep Learning for Music, Deep Learning for Alzheime Diagnostics and Decision Support and Deep Learning in combination with EEG electrical signals from the brain.

Recommend to check out Chew-Yean Yam‘s (Principal Data Scientist, Microsoft) blog post Emotion Detection and Recognition from Text using Deep Learning.

Best regards,
Amund Tveit

Year  Title Author
2016   Towards real-time Speech Emotion Recognition using deep neural networks  HM Fayek, M Lech, L Cavedon
2016   A Multi-task Learning Framework for Emotion Recognition Using 2D Continuous Space  R Xia, Y Liu
2016   TrueHappiness: Neuromorphic Emotion Recognition on TrueNorth  PU Diehl, BU Pedroni, A Cassidy, P Merolla, E Neftci
2016   Collaborative expression representation using peak expression and intra class variation face images for practical subject-independent emotion recognition in videos  SH Lee, WJ Baddar, YM Ro
2016   Discriminatively Trained Recurrent Neural Networks for Continuous Dimensional Emotion Recognition from Audio  F Weninger, F Ringeval, E Marchi, B Schuller
2016   Feature Transfer Learning for Speech Emotion Recognition  J Deng
2016   Emotion Recognition in Speech with Deep Learning Architectures  M Erdal, M Kächele, F Schwenker
2016   Error-correcting output codes for multi-label emotion classification  C Li, Z Feng, C Xu
2016   Software Effort Estimation Framework To Improve Organization Productivity Using Emotion Recognition Of Software Engineers In …  BP Rao, PS Ramaiah
2016   How Deep Neural Networks Can Improve Emotion Recognition on Video Data  P Khorrami, TL Paine, K Brady, C Dagli, TS Huang
2016   Automatic emotion recognition in the wild using an ensemble of static and dynamic representations  MM Ghazi, HK Ekenel
2016   HoloNet: towards robust emotion recognition in the wild  A Yao, D Cai, P Hu, S Wang, L Sha, Y Chen
2016   Deep learning driven hypergraph representation for image-based emotion recognition  Y Huang, H Lu
2016   A Review on Deep Learning Algorithms for Speech and Facial Emotion Recognition  CP Latha, M Priya
2016   Novel Affective Features For Multiscale Prediction Of Emotion In Music  N Kumar, T Guha, CW Huang, C Vaz, SS Narayanan
2016   Facial emotion detection using deep learning  DL Spiers
2016   Speech Emotion Recognition Based on Deep Belief Networks and Wavelet Packet Cepstral Coefficients.  Y Huang, A Wu, G Zhang, Y Li
2016   Audio-Video Based Multimodal Emotion Recognition Using SVMs and Deep Learning  B Sun, Q Xu, J He, L Yu, L Li, Q Wei
2016   Feature Learning via Deep Belief Network for Chinese Speech Emotion Recognition  S Zhang, X Zhao, Y Chuang, W Guo, Y Chen
2016   Transfer Learning of Deep Neural Network for Speech Emotion Recognition  Y Huang, M Hu, X Yu, T Wang, C Yang
2016   Multiagent Social Influence Detection Based on Facial Emotion Recognition  P Mishra, R Hadfi, T Ito
2016   Emotion Recognition Using Facial Expression Images for a Robotic Companion  V Palade
2016   Emotion Recognition from Speech Signals Using Deep Learning Methods  S Pathak, MV Kolhe
2016   Multimodal Emotion Recognition Using Multimodal Deep Learning  W Liu, WL Zheng, BL Lu
2016   Self-Configuring Ensemble of Neural Network Classifiers for Emotion Recognition in the Intelligent Human-Machine Interaction  E Sopov, I Ivanov
2016   Facing Realism in Spontaneous Emotion Recognition from Speech: Feature Enhancement by Autoencoder with LSTM Neural Networks  Z Zhang, F Ringeval, J Han, J Deng, E Marchi
2016   The University of Passau Open Emotion Recognition System for the Multimodal Emotion Challenge  J Deng, N Cummins, J Han, X Xu, Z Ren, V Pandit
2016   Building a large scale dataset for image emotion recognition: The fine print and the benchmark  Q You, J Luo, H Jin, J Yang
2016   Emotion Recognition Using Multimodal Deep Learning  W Liu, WL Zheng, BL Lu
2016   Emotion Prediction from User-Generated Videos by Emotion Wheel Guided Deep Learning  CT Ho, YH Lin, JL Wu
2016   FDBN: Design and development of Fractional Deep Belief Networks for speaker emotion recognition  K Mannepalli, PN Sastry, M Suman
2016   A novel Adaptive Fractional Deep Belief Networks for speaker emotion recognition  K Mannepalli, PN Sastry, M Suman
2016   Unsupervised domain adaptation for speech emotion recognition using PCANet  Z Huang, W Xue, Q Mao, Y Zhan
2016   Learning Auditory Neural Representations for Emotion Recognition  P Barros, C Weber, S Wermter
2016   Towards an” In-the-Wild” Emotion Dataset Using a Game-based Framework  W Li, F Abtahi, C Tsangouri, Z Zhu
2016   Deep Learning for Emotion Recognition in Faces  A Ruiz
2016   Emotion Classification on face images  M Jorda, N Miolane, A Ng
2016   Paralinguistic Speech Recognition: Classifying Emotion in Speech with Deep Learning Neural Networks  ER Segal
2016   Architecture of Emotion in Robots Using Convolutional Neural Networks  M Ghayoumi, AK Bansal
2016   Emotion recognition from face dataset using deep neural nets  D Das, A Chakrabarty
2016   Recognize the facial emotion in video sequences using eye and mouth temporal Gabor features  PI Rani, K Muneeswaran
2016   Deep Learning Based Emotion Recognition from Chinese Speech  W Zhang, D Zhao, X Chen, Y Zhang
2016   Bi-Modal Music Emotion Recognition: Novel Lyrical Features and Dataset  R Malheiro, R Panda, P Gomes, R Paiva
2016   Speech Emotion Recognition Using Voiced Segment Selection Algorithm  Y Gu, E Postma, HX Lin, J van den Herik
2015   Multi-modal Dimensional Emotion Recognition using Recurrent Neural Networks  S Chen, Q Jin
2015   Quantification of Cinematography Semiotics for Video-based Facial Emotion Recognition in the EmotiW 2015 Grand Challenge  AC Cruz
2015   EEG Based Emotion Identification Using Unsupervised Deep Feature Learning  X Li, P Zhang, D Song, G Yu, Y Hou, B Hu
2015   Pattern-Based Emotion Classification on Social Media  E Tromp, M Pechenizkiy
2015   Investigating Critical Frequency Bands and Channels for EEG-based Emotion Recognition with Deep Neural Networks  WL Zheng, BL Lu
2015   Revealing critical channels and frequency bands for emotion recognition from EEG with deep belief network  WL Zheng, HT Guo, BL Lu
2015   Analysis of Physiological for Emotion Recognition with IRS Model  C Li, C Xu, Z Feng
2015   Emotion Recognition in the Wild via Convolutional Neural Networks and Mapped Binary Patterns  G Levi, T Hassner
2015   Negative Emotion Recognition in Spoken Dialogs  X Zhang, H Wang, L Li, M Zhao, Q Li
2015   Combining Multimodal Features within a Fusion Network for Emotion Recognition in the Wild  B Sun, L Li, G Zhou, X Wu, J He, L Yu, D Li, Q Wei
2015   A Deep Feature based Multi-kernel Learning Approach for Video Emotion Recognition  W Li, F Abtahi, Z Zhu
2015   Recurrent Neural Networks for Emotion Recognition in Video  S Ebrahimi Kahou, V Michalski, K Konda, R Memisevic
2015   Learning Speech Emotion Features by Joint Disentangling-Discrimination  W Xue, Z Huang, X Luo, Q Mao
2015   Data selection for acoustic emotion recognition: Analyzing and comparing utterance and sub-utterance selection strategies  D Le, EM Provost
2015   Leveraging Inter-rater Agreement for Audio-Visual Emotion Recognition  Y Kim, EM Provost
2015   The Research on Cross-Language Emotion Recognition Algorithm for Hearing Aid  X Shulan, W Jilin
2015   Optimized multi-channel deep neural network with 2D graphical representation of acoustic speech features for emotion recognition  MN Stolar, M Lech, IS Burnett
2015   EmoNets: Multimodal deep learning approaches for emotion recognition in video  SE Kahou, X Bouthillier, P Lamblin, C Gulcehre
2015   Deep learninig of EEG signals for emotion recognition  Y Gao, HJ Lee, RM Mehmood
2015   Emotion Recognition & Classification using Neural Networks  K Koupidis, A Ioannis
2015   Emotion recognition from embedded bodily expressions and speech during dyadic interactions  PM Müller, S Amin, P Verma, M Andriluka, A Bulling
2015   Speech emotion recognition with unsupervised feature learning  Z HUANG, W XUE, Q MAO
2015   Emotion identification by facial landmarks dynamics analysis  A Bandrabur, L Florea, C Florea, M Mancas
2014   Speech Emotion Recognition Using CNN  Z Huang, M Dong, Q Mao, Y Zhan
2014   Multi-scale Temporal Modeling for Dimensional Emotion Recognition in Video  L Chao, J Tao, M Yang, Y Li, Z Wen
2014   Improving generation performance of speech emotion recognition by denoising autoencoders  L Chao, J Tao, M Yang, Y Li
2014   Acoustic emotion recognition using deep neural network  J Niu, Y Qian, K Yu
2014   Prosodic, spectral and voice quality feature selection using a long-term stopping criterion for audio-based emotion recognition  M Kächele, D Zharkov, S Meudt, F Schwenker
2014   Emotion Modeling and Machine Learning in Affective Computing  K Kim
2014   Emotion Recognition in the Wild with Feature Fusion and Multiple Kernel Learning  JK Chen, Z Chen, Z Chi, H Fu
2014   A Study of Deep Belief Network Based Chinese Speech Emotion Recognition  B Chen, Q Yin, P Guo
Continue Reading

Deep Learning for Music

Deep Learning (creative AI) might potentially be used for music analysis and music creation. Deepmind’s Wavenet is a step in that direction. This blog post presents recent papers in Deep Learning for Music.

note: If you are curious about oversight of Deep Learning topics, please consider subscribing to my Deep Learning Newsletter at the end of this blog post.

Best regards,

Amund Tveit

Generation, Analysis and Segmentation of Music

  1. Deep Learning for Music
    – Authors: 
    A HuangR Wu (2016)
  2. Live Orchestral Piano, a system for real-time orchestral music generation
    – Authors: L CrestelP Esling (2016)
  3. Unsupervised feature learning for Music Structural Analysis
    – Authors: M BuccoliM ZanoniA SartiS TubaroD Andreoletti (2016)
  4. Music staff removal with supervised pixel classification
    – Authors: J Calvo (2016)
  5. An Intelligent Musical Rhythm Variation Interface
    – Authors: R VoglP Knees (2016)
  6. Probabilistic Segmentation of Musical Sequences using Restricted Boltzmann Machines
    – Authors: S LattnerM GrachtenK AgresCec Chacan (2015)
  7. Machine Learning Applied to Musical Improvisation
    – Authors: Rm Keller (2015)
  8. A Deep Neural Network for Modeling Music
    – Authors: P ZhangX ZhengW ZhangS LiS QianW (2015)
  9. Deep symbolic learning of multiple temporal granularities for musical orchestration.
    – Authors: F JacquemardG PeetersP E. (2015)
  10. Deep Karaoke: Extracting Vocals from Musical Mixtures Using a Convolutional Deep Neural Network
    – Authors: Ajr SimpsonG RomaMd Plumbley (2015)
  11. Deep Remix: Remixing Musical Mixtures Using a Convolutional Deep Neural Network
    – Authors: Ajr SimpsonG RomaMd Plumbley (2015)
  12. Analysis of musical structure: an approach based on deep learning
    – Authors: D Andreoletti (2015)
  13. An exploration of deep learning in content-based music informatics
    – Authors: Ej Humphrey (2015)
  14. Fusing Music and Video Modalities Using Multi-timescale Shared Representations
    – Authors: B XuX WangX Tang (2014)

Music Transcription

  1. An End-to-End Neural Network for Polyphonic Piano Music Transcription
    – Authors: S SigtiaE BenetosS Dixon (2016)
  2. Music transcription modelling and composition using deep learning
    – Authors: Bl SturmJf SantosO Ben (2016)
  3. Rewind: A Music Transcription Method
    – Authors: Cd Carthen (2016)
  4. An End-to-End Neural Network for Polyphonic Music Transcription
    – Authors: S SigtiaE BenetosS Dixon (2015)

Emotion

  1. Novel Affective Features For Multiscale Prediction Of Emotion In Music
    – Authors: N KumarT GuhaCw HuangC VazSs Narayanan (2016)
  2. Demv-Matchmaker: Emotional Temporal Course Representation And Deep Similarity Matching For Automatic Music Video Generation 
    – Authors: Jc LinWl WeiHm Wang (2016)

Classification

  1. A Comparative Study on Music Genre Classification Algorithms
    – Authors: W Stokowiec (2016)
  2. Audio-Based Music Classification with A Pretrained Convolutional Network
    – Authors: E DielemanB Schrauwen (2016)
  3. Semantic Labeling of Music
    – Authors: P KneesM Schedl (2016)
  4. Classification of classic Turkish music makams by using deep belief networks
    – Authors: Mak SB Bolat (2016)
  5. Deep convolutional neural networks for predominant instrument recognition in polyphonic music
    – Authors: Y HanJ KimK Lee (2016)
  6. Deep learning, audio adversaries, and music content analysis
    – Authors: C KereliukBl SturmJ Larsen (2015)
  7. Speech Music Discrimination Using an Ensemble of Biased Classifiers
    – Authors: K KimA BaijalBs KoS LeeI HwangY Kim (2015)
  8. Deep Learning and Music Adversaries
    – Authors: C KereliukBl SturmJ Larsen (2015)
  9. Musical notes classification with Neuromorphic Auditory System using FPGA and a Convolutional Spiking Network
    – Authors: E Cerezuela (2015)
  10. A Deep Bag-of-Features Model for Music Auto-Tagging
    – Authors: J NamJ HerreraK Lee (2015)
  11. Music Genre Classification Using Convolutional Neural Network
    – Authors: Q KongX FengY Li (2014)
  12. An Associative Memorization Architecture of Extracted Musical Features from Audio Signals by Deep Learning Architecture
    – Authors: T NiwaK NaruseR OoeM KinoshitaT M (2014)
  13. Unsupervised Feature Pre-training of the Scattering Wavelet Transform for Musical Genre Recognition
    – Authors: M KD K (2014)

Search and Recommender Systems

  1. Deep learning for audio-based music recommendation
    – Authors: S Dieleman (2016)
  2. A compositional hierarchical model for music information retrieval
    – Authors: M PesekA LeonardisM Marolt (2014)

Please sign up for Amund Tveit’s Deep Learning Newsletter!


Continue Reading

Deep Learning in combination with EEG electrical signals from the brain

eeg

EEG (Electroencephalography) is the measurement of electrical signals in the brain. It has long been used for medical purposes (e.g. diagnosis of epilepsy), and has in more recent years also been used in Brain Computer Interfaces (BCI) – note: if BCI is new to you don’t get overly excited about it, since these interfaces are still in my opinion quite premature. But they are definitely interesting in a longer term perspective .

This blog post gives an overview of recent research on Deep Learning in combination with EEG, e.g. r for classification, feature representation, diagnosis, safety (cognitive state of drivers) and hybrid methods (Computer Vision or Speech Recognition together with EEG and Deep Learning).

Best regards,

Amund Tveit

Diagnosis and Medicine

  1. Classification of Epileptic EEG Signals with Stacked Sparse Autoencoder Based on Deep Learning
    – Authors: Q Lin, S Ye, X Huang, S Li, M Zhang, Y Xue, Ws Chen (2016)
  2. Advanced use of EEG in drug development and personalized medicine
    – Authors: S Simpraga, R Alvarez (2016)
  3. Low-complexity algorithms for automatic detection of sleep stages and events for use in wearable EEG systems
    – Authors: Sa Imtiaz (2016)
  4. Predicting epileptic seizure from Electroencephalography (EEG) using hilbert huang transformation and neural network
    – Authors: Mo Rahman, Mn Karim (2015)
  5. Multi-Channel EEG based Sleep Stage Classification with Joint Collaborative Representation and Multiple Kernel Learning
    – Authors: J Shi, X Liu, Y Li, Q Zhang, Y Li, S Yin (2015)
  6. Superchords: decoding EEG signals in the millisecond range
    – Authors: R Normand, Ha Ferreira (2015)
  7. A novel motor imagery EEG recognition method based on deep learning
    – Authors: M Li, M Zhang, Y Sun (2016)

Brain Computer Interfaces

  1. Convolutional Networks for EEG Signal Classification in Non-Invasive Brain-Computer Interfaces
    – Authors: E Forney, C Anderson, W Gavin, P Davies (2016)
  2. Using Deep Learning for Human Computer Interface via Electroencephalography
    – Authors: S Redkar (2016)
  3. Deep learning EEG response representation for brain computer interface
    – Authors: L Jingwei, C Yin, Z Weidong (2015)

Cognition and Emotion

  1. Single-channel EEG-based mental fatigue detection based on deep belief network
    – Authors: P Li, W Jiang, F Su (2016)
  2. EEG-based prediction of driver’s cognitive performance by deep convolutional neural network
    – Authors: M Hajinoroozi, Z Mao, Tp Jung, Ct Lin, Y Huang (2016)
  3. EEG-based Driver Fatigue Detection using Hybrid Deep Generic Model
    – Authors: Pp San, Sh Ling, R Chai, Y Tran, A Craig, Ht Nguyen (2016)
  4. Mental State Recognition via Wearable EEG
    – Authors: P Bashivan, I Rish, S Heisig (2016)
  5. Single trial prediction of normal and excessive cognitive load through EEG feature fusion
    – Authors: P Bashivan, M Yeasin, Gm Bidelman (2016)
  6. Prediction of driver’s drowsy and alert states from EEG signals with deep learning
    – Authors: M Hajinoroozi, Z Mao, Y Huang (2016)
  7. Investigating Critical Frequency Bands and Channels for EEG-based Emotion Recognition with Deep Neural Networks
    – Authors: Wl Zheng, Bl Lu (2015)
  8. Deep learninig of EEG signals for emotion recognition
    – Authors: Y Gao, Hj Lee, Rm Mehmood (2015)
  9. EEG Based Emotion Identification Using Unsupervised Deep Feature Learning
    – Authors: X Li, P Zhang, D Song, G Yu, Y Hou, B Hu (2015)
  10. Feature extraction with deep belief networks for driver’s cognitive states prediction from EEG data
    – Authors: M Hajinoroozi, Tp Jung, Ct Lin, Y Huang (2015)
  11. Measurement Of Stress Intensity Using Eeg
    – Authors: V Tóth (2015)
  12. Revealing critical channels and frequency bands for emotion recognition from EEG with deep belief network
    – Authors: Wl Zheng, Ht Guo, Bl Lu (2015)
  13. Interpretable Deep Neural Networks for Single-Trial EEG Classification
    – Authors: I Sturm, S Bach, W Samek, Kr Müller (2016)

Hybrid methods – combining Deep Learning and EEG with other Deep Learning methods

  1. Improving Electroencephalography-Based Imagined Speech Recognition with a Simultaneous Video Data Stream
    – Authors: Sj Stolze (2016)
  2. A closed-loop system for rapid face retrieval by combining EEG and computer vision
    – Authors: Y Wang, L Jiang, B Cai, Y Wang, S Zhang, X Zheng (2015)
  3. Emotional Affect Estimation Using Video and EEG Data in Deep Neural Networks
    – Authors: A Frydenlund, F Rudzicz (2015)

Feature representation and Classification

  1. Improving EEG feature learning via synchronized facial video
    – Authors: X Li, X Jia, G Xun, A Zhang (2016)
  2. Decoding EEG and LFP signals using deep learning: heading TrueNorth
    – Authors: E Nurse, Bs Mashford, Aj Yepes, I Kiral (2016)
  3. Fast and efficient rejection of background waveforms in interictal EEG
    – Authors: E Bagheri, J Jin, J Dauwels, S Cash, Mb Westover (2016)
  4. Class-wise Deep Dictionaries for EEG Classification
    – Authors: P Khurana, A Majumdar, R Ward (2016)
  5. EEG-based affect states classification using Deep Belief Networks
    – Authors: H Xu, Kn Plataniotis (2016)
  6. Feature Learning from Incomplete EEG with Denoising Autoencoder
    – Authors: J Li, Z Struzik, L Zhang, A Cichocki (2014)
  7. Joint optimization of algorithmic suites for EEG analysis
    – Authors: E Santana, Aj Brockmeier, Jc Principe (2014)
  8. Deep Extreme Learning Machine and Its Application in EEG Classification
    – Authors: S Ding, N Zhang, X Xu, L Guo, J Zhang (2014)
  9. Deep Feature Learning for EEG Recordings
    – Authors: S Stober, A Sternin, Am Owen, Ja Grahn (2015)
  10. A Multichannel Deep Belief Network for the Classification of EEG Data
    – Authors: Am Al (2015)
  11. Learning Representations from EEG with Deep Recurrent-Convolutional Neural Networks
    – Authors: P Bashivan, I Rish, M Yeasin, N Codella (2015)
Continue Reading