Overview of recent Deep Learning Bibliographies

For the last couple of months I’ve been creating bibliographies of recent academic publications in various subfields of Deep Learning on this blog. This posting gives an overview of the last 25 bibliographies posted.

Best regards,

Amund Tveit (WeChat: AmundTveit – Twitter: @atveit)

1. Deep Learning with Residual Networks

This posting is recent papers related to residual networks (i.e. very deep networks). Check out Microsoft Research’s paper Deep Residual Learning for Image Recognition and Kaiming He’s ICML 2016 Tutorial Deep Residual Learning, Deep Learning Gets Way Deeper

2. Deep Learning for Traffic Sign Detection and Recognition

Traffic Sign Detection and Recognition is key functionality for self-driving cars. This posting has recent papers in this area. Check also out related posting: Deep Learning for Vehicle Detection and Classification

3. Deep Learning for Vehicle Detection and Classification

This posting has recent papers about vehicle (e.g. car) detection and classification, e.g. for selv-driving/autonomous cars. Related: check also out Nvidia‘s End-to-End Deep Learning for Self-driving Cars and Udacity‘s Self-Driving Car Engineer (Nanodegree).

4. Deep Learning with Long Short-Term Memory (LSTM)

This blog post has some recent papers about Deep Learning with Long-Short Term Memory (LSTM). To get started I recommend checking out Christopher Olah’s Understanding LSTM Networks and Andrej Karpathy’s The Unreasonable Effectiveness of Recurrent Neural Networks. This blog post is complemented by Deep Learning with Recurrent/Recursive Neural Networks (RNN) — ICLR 2017 Discoveries.

5. Deep Learning in Finance

This posting has recent publications about Deep Learning in Finance (e.g. stock market prediction)

6. Deep Learning for Information Retrieval and Learning to Rank

This posting is about Deep Learning for Information Retrieval and Learning to Rank (i.e. of interest if developing search engines). The posting is complemented by the posting Deep Learning for Question Answering. To get started I recommend checking out Jianfeng Gao‘s (Deep Learning Technology Center at Microsoft Research) presentation Deep Learning for Web Search and Natural Language Processing.

Of partial relevance is the posting Deep Learning for Sentiment Analysis, the posting about Embedding for NLP with Deep Learning, the posting about Deep Learning for Natural Language Processing (ICLR 2017 discoveries), and the posting about Deep Learning for Recommender Systems

7. Deep Learning for Question Answering

This posting presents recent publications related to Deep Learning for Question Answering. Question Answering is described as “a computer science discipline within the fields of information retrieval and natural language processing (NLP), which is concerned with building systems that automatically answer questions posed by humans in a natural language”. I’ll also publish postings about Deep Learning for Information Retrieval and Learning to Rank today.

8. Ensemble Deep Learning

Ensemble Based Machine Learning has been used with success in several Kaggle competitions, and this year also the Imagenet competition was dominated by ensembles in Deep Learning, e.g. Trimps-Soushen team from 3rd Research Institute of the Ministry of Public Security (China) used a combination of Inception, Inception-Resnet, Resnet and Wide Residual Network to win the Object Classification/localization challenge. This blog post has recent papers related to Ensembles in Deep Learning.

9. Deep Learning for Sentiment Analysis

Recently I published Embedding for NLP with Deep Learning (e.g. word2vec and follow-ups) and Deep Learning for Natural Language Processing — ICLR 2017 Discoveries — this posting is also mostly NLP-related since it provides recent papers related to Deep Learning for Sentiment Analysis, but also has examples of other types of sentiment (e.g. image sentiment).

10. Deep Learning with Gaussian Process

Gaussian Process is a statistical model where observations are in the continuous domain, to learn more check out a tutorial on gaussian process(by Univ.of Cambridge’s Zoubin G.). Gaussian Process is an infinite-dimensional generalization of multivariate normal distributions.

Researchers from University of Sheffield — Andreas C. Damanianou and Neil D. Lawrence — started using Gaussian Process with Deep Belief Networks (in 2013). This Blog post contains recent papers related to combining Deep Learning with Gaussian Process.

11. Deep Learning for Clustering

12. Deep Learning in combination with EEG electrical signals from the brain

EEG (Electroencephalography) is the measurement of electrical signals in the brain. It has long been used for medical purposes (e.g. diagnosis of epilepsy), and has in more recent years also been used in Brain Computer Interfaces (BCI) — note: if BCI is new to you don’t get overly excited about it, since these interfaces are still in my opinion quite premature. But they are definitely interesting in a longer term perspective .

This blog post gives an overview of recent research on Deep Learning in combination with EEG, e.g. r for classification, feature representation, diagnosis, safety (cognitive state of drivers) and hybrid methods (Computer Vision or Speech Recognition together with EEG and Deep Learning).

13. Embedding for NLP with Deep Learning

Word Embedding was introduced by Bengio in early 2000s, and interest in it really accelerated when Google presented Word2Vec in 2013.

This blog post has recent papers related to embedding for Natural Language Processing with Deep Learning. Example application areas embedding is used for in the papers include finance (stock market prediction), biomedical text analysis, part-of-speech tagging, sentiment analysis, pharmacology (drug adverse effects).

I recommend you to start with the paper: In Defense of Word Embedding for Generic Text Representation

14. Zero-Shot (Deep) Learning

Zero-Shot Learning is making decisions after seing only one or few examples (as opposed to other types of learning that typically requires large amount of training examples). Recommend having a look at An embarrassingly simple approach to zero-shot learning first.

15. Deep Learning for Alzheimer Diagnostics and Decision Support

Alzheimer’s Disease is the cause of 60–70% of cases of Dementia, costs associated to diagnosis, treatment and care of patients with it is estimated to be in the range of a hundred billion dollars in USA. This blog post have some recent papers related to using Deep Learning for diagnostics and decision support related to Alzheimer’s disease.

16. Recommender Systems with Deep Learning

This blog post presents recent research in Recommender Systems (/collaborative filtering) with Deep Learning. To get started I recommend having a look at A Survey and Critique of Deep Learning in Recommender Systems.

17. Deep Learning for Ultrasound Analysis

Ultrasound (also called Sonography) are sound waves with higher frequency than humans can hear, they frequently used in medical settings, e.g. for checking that pregnancy is going well with fetal ultrasound. For more about Ultrasound data formats check out Ultrasound Research Interface. This blog post has recent publications about applying Deep Learning for analyzing Ultrasound data.

18. Deep Learning for Music

Deep Learning (creative AI) might potentially be used for music analysis and music creation. Deepmind’s Wavenet is a step in that direction. This blog post presents recent papers in Deep Learning for Music.

19. Regularized Deep Networks — ICLR 2017 Discoveries

This blog post gives an overview of papers related to using Regularization in Deep Learning submitted to ICLR 2017, see underneath for the list of papers. If you want to learn about Regularization in Deep Learning check out: www.deeplearningbook.org/contents/regularization.html

20. Unsupervised Deep Learning — ICLR 2017 Discoveries

This blog post gives an overview of papers related to Unsupervised Deep Learning submitted to ICLR 2017, see underneath for the list of papers. If you want to learn about Unsupervised Deep Learning check out: Ruslan Salkhutdinov’s video Foundations of Unsupervised Deep Learning.

21. Autoencoders in Deep Learning — ICLR 2017 Discoveries

This blog post gives an overview of papers related to autoencoders submitted to ICLR 2017, see underneath for the list of papers. If you want to learn about autoencoders check out the Stanford (UFLDL) tutorial about Autoencoders, Carl Doersch’ Tutorial on Variational Autoencoders, DeepLearning.TV’s Video tutorial on Autoencoders, or Goodfellow, Bengio and Courville’s Deep Learning book’s chapter on Autencoders.

22. Stochastic/Policy Gradients in Deep Learning — ICLR 2017 Discoveries

This blog post gives an overview of papers related to stochastic/policy gradient submitted to ICLR 2017, see underneath for the list of papers.

23. Deep Learning with Recurrent/Recursive Neural Networks (RNN) — ICLR 2017 Discoveries

This blog post gives an overview of Deep Learning with Recurrent/Recursive Neural Networks (RNN) related papers submitted to ICLR 2017, see underneath for the list of papers. If you want to learn more about RNN check out Andrej Karpathy’s The Unreasonable Effectiveness of Recurrent Neural Networks and Pascanu, Gulcehre, Cho and Bengio’s How to Construct Deep Recurrent Neural Networks.

24. Deep Learning with Generative and Generative Adverserial Networks — ICLR 2017 Discoveries

This blog post gives an overview of Deep Learning with Generative and Adverserial Networks related papers submitted to ICLR 2017, see underneath for the list of papers. Want to learn about these topics? See OpenAI’s article about Generative Models and Ian Goodfellow et.al’s paper about Generative Adversarial Networks.

25. Deep Learning for Natural Language Processing — ICLR 2017 Discoveries

This blog post gives an overview of Natural Language Processing related papers submitted to ICLR 2017, see underneath for the list of papers. If you want to learn about Deep Learning with NLP check out Stanford’s CS224d: Deep Learning for Natural Language Processing

Continue Reading

Deep Learning for Traffic Sign Detection and Recognition

Traffic Sign Detection and Recognition is key functionality for self-driving cars. This posting has recent papers in this area. Check also out related posting: Deep Learning for Vehicle Detection and Classification

Best regards,
Amund Tveit
Amund Tveit

Year  Title Author
2016   Road surface traffic sign detection with hybrid region proposal and fast R-CNN  R Qian, Q Liu, Y Yue, F Coenen, B Zhang
2016   Traffic sign classification with deep convolutional neural networks  J CREDI
2016   Real-time Traffic Sign Recognition system with deep convolutional neural network  S Jung, U Lee, J Jung, DH Shim
2016   Traffic Sign Detection and Recognition using Fully Convolutional Network Guided Proposals  Y Zhu, C Zhang, D Zhou, X Wang, X Bai, W Liu
2016   A traffic sign recognition method based on deep visual feature  F Lin, Y Lai, L Lin, Y Yuan
2016   The research on traffic sign recognition based on deep learning  C Li, C Yang
2015   Fast Traffic Sign Recognition with a Rotation Invariant Binary Pattern Based Feature  S Yin, P Ouyang, L Liu, Y Guo, S Wei
2015   Malaysia traffic sign recognition with convolutional neural network  MM Lau, KH Lim, AA Gopalai
2015   Negative-Supervised Cascaded Deep Learning for Traffic Sign Classification  K Xie, S Ge, R Yang, X Lu, L Sun
Continue Reading

Deep Learning for Vehicle Detection and Classification

This posting has recent papers about vehicle (e.g. car) detection and classification, e.g. for selv-driving/autonomous cars. Related: check also out Nvidia‘s End-to-End Deep Learning for Self-driving Cars and Udacity‘s Self-Driving Car Engineer (Nanodegree).

Best regards,

<a href=”https://amundtveit.com/about/”>Amund Tveit</a> (<a href=”https://twitter.com/atveit”>@atveit</a>)

Year  Title Author
2016   Vehicle Classification using Transferable Deep Neural Network Features  Y Zhou, NM Cheung
2016   A Hybrid Fuzzy Morphology And Connected Components Labeling Methods For Vehicle Detection And Counting System  C Fatichah, JL Buliali, A Saikhu, S Tena
2016   Evaluation of vehicle interior sound quality using a continuous restricted Boltzmann machine-based DBN  HB Huang, RX Li, ML Yang, TC Lim, WP Ding
2016   An Automated Traffic Surveillance System with Aerial Camera Arrays: Data Collection with Vehicle Tracking  X Zhao, D Dawson, WA Sarasua, ST Birchfield
2016   Vehicle type classification via adaptive feature clustering for traffic surveillance video  S Wang, F Liu, Z Gan, Z Cui
2016   Vehicle Detection in Satellite Images by Incorporating Objectness and Convolutional Neural Network  S Qu, Y Wang, G Meng, C Pan
2016   DAVE: A Unified Framework for Fast Vehicle Detection and Annotation  Y Zhou, L Liu, L Shao, M Mellor
2016   3D Fully Convolutional Network for Vehicle Detection in Point Cloud  B Li
2016   A Deep Learning-Based Approach to Progressive Vehicle Re-identification for Urban Surveillance  X Liu, W Liu, T Mei, H Ma
2016   TraCount: a deep convolutional neural network for highly overlapping vehicle counting  S Surya, RV Babu
2016   Pedestrian, bike, motorcycle, and vehicle classification via deep learning: Deep belief network and small training set  YY Wu, CM Tsai
2016   Fast Vehicle Detection in Satellite Images Using Fully Convolutional Network  J Hu, T Xu, J Zhang, Y Yang
2016   Local Tiled Deep Networks for Recognition of Vehicle Make and Model  Y Gao, HJ Lee
2016   Vehicle detection based on visual saliency and deep sparse convolution hierarchical model  Y Cai, H Wang, X Chen, L Gao, L Chen
2016   Sound quality prediction of vehicle interior noise using deep belief networks  HB Huang, XR Huang, RX Li, TC Lim, WP Ding
2016   Accurate On-Road Vehicle Detection with Deep Fully Convolutional Networks  Z Jie, WF Lu, EHF Tay
2016   Fault Detection and Identification of Vehicle Starters and Alternators Using Machine Learning Techniques  E Seddik
2016   Fault diagnosis network design for vehicle on-board equipments of high-speed railway: A deep learning approach  J Yin, W Zhao
2016   Real-time state-of-health estimation for electric vehicle batteries: A data-driven approach  G You, S Park, D Oh
2016   The Precise Vehicle Retrieval in Traffic Surveillance with Deep Convolutional Neural Networks  B Su, J Shao, J Zhou, X Zhang, L Mei, C Hu
2016   Online vehicle detection using deep neural networks and lidar based preselected image patches  S Lange, F Ulbrich, D Goehring
2016   A closer look at Faster R-CNN for vehicle detection  Q Fan, L Brown, J Smith
2016   Appearance-based Brake-Lights recognition using deep learning and vehicle detection  JG Wang, L Zhou, Y Pan, S Lee, Z Song, BS Han
2016   Night time vehicle detection algorithm based on visual saliency and deep learning  Y Cai, HW Xiaoqiang Sun, LCH Jiang
2016   Vehicle classification in WAMI imagery using deep network  M Yi, F Yang, E Blasch, C Sheaff, K Liu, G Chen, H Ling
2015   VeTrack: Real Time Vehicle Tracking in Uninstrumented Indoor Environments  M Zhao, T Ye, R Gao, F Ye, Y Wang, G Luo
2015   Vehicle Color Recognition in The Surveillance with Deep Convolutional Neural Networks  B Su, J Shao, J Zhou, X Zhang, L Mei
2015   Vehicle Speed Prediction using Deep Learning  J Lemieux, Y Ma
2015   Monza: Image Classification of Vehicle Make and Model Using Convolutional Neural Networks and Transfer Learning  D Liu, Y Wang
2015   Night Time Vehicle Sensing in Far Infrared Image with Deep Learning  H Wang, Y Cai, X Chen, L Chen
2015   A Vehicle Type Recognition Method based on Sparse Auto Encoder  HL Rong, YX Xia
2015   Occluded vehicle detection with local connected deep model  H Wang, Y Cai, X Chen, L Chen
2015   Performance Evaluation of the Neural Network based Vehicle Detection Models  K Goyal, D Kaur
2015   A Smartphone-based Connected Vehicle Solution for Winter Road Surface Condition Monitoring  MA Linton
2015   Vehicle Logo Recognition System Based on Convolutional Neural Networks With a Pretraining Strategy  Y Huang, R Wu, Y Sun, W Wang, X Ding
2015   SiftKeyPre: A Vehicle Recognition Method Based on SIFT Key-Points Preference in Car-Face Image  CY Zhang, XY Wang, J Feng, Y Cheng
2015   Vehicle Detection in Aerial Imagery: A small target detection benchmark  S Razakarivony, F Jurie
2015   Vehicle license plate recognition using visual attention model and deep learning  D Zang, Z Chai, J Zhang, D Zhang, J Cheng
2015   Domain adaption of vehicle detector based on convolutional neural networks  X Li, M Ye, M Fu, P Xu, T Li
2015   Trainable Convolutional Network Apparatus And Methods For Operating A Robotic Vehicle  P O’connor, E Izhikevich
2015   Vehicle detection and classification based on convolutional neural network  D He, C Lang, S Feng, X Du, C Zhang
2015   The AdaBoost algorithm for vehicle detection based on CNN features  X Song, T Rui, Z Zha, X Wang, H Fang
2015   Deep neural networks-based vehicle detection in satellite images  Q Jiang, L Cao, M Cheng, C Wang, J Li
2015   Vehicle License Plate Recognition Based on Extremal Regions and Restricted Boltzmann Machines  C Gou, K Wang, Y Yao, Z Li
2014   Multi-modal Sensor Registration for Vehicle Perception via Deep Neural Networks  M Giering, K Reddy, V Venugopalan
2014   Mooting within the curriculum as a vehicle for learning: student perceptions  L Jones, S Field
2014   Vehicle Type Classification Using Semi-Supervised Convolutional Neural Network  Z Dong, Y Wu, M Pei, Y Jia
2014   Vehicle License Plate Recognition With Random Convolutional Networks  D Menotti, G Chiachia, AX Falcao, VJO Neto
2014   Vehicle Type Classification Using Unsupervised Convolutional Neural Network  Z Dong, M Pei, Y He, T Liu, Y Dong, Y Jia
Continue Reading

Zero-Shot (Deep) Learning

Zero-Shot Learning is making decisions after seing only one or few examples (as opposed to other types of learning that typically requires large amount of training examples). Recommend having a look at An embarrassingly simple approach to zero-shot learning first.

Best regards,

Amund Tveit

  1. Less is more: zero-shot learning from online textual documents with noise suppression
    – Authors: R Qiao, L Liu, C Shen, A Hengel (2016)
  2. Synthesized Classifiers for Zero-Shot Learning
    – Authors: S Changpinyo, Wl Chao, B Gong, F Sha (2016)
  3. Tinkering Under The Hood: Interactive Zero-Shot Learning with Pictorial Classifiers
    – Authors: V Krishnan (2016)
  4. Active Transfer Learning with Zero-Shot Priors: Reusing Past Datasets for Future Tasks
    – Authors: E Gavves, T Mensink, T Tommasi, Cgm Snoek… (2015)
  5. Transductive Multi-view Zero-Shot Learning
    – Authors: Y Fu, Tm Hospedales, T Xiang, S Gong (2015)
  6. Predicting Deep Zero-Shot Convolutional Neural Networks using Textual Descriptions
    – Authors: J Ba, K Swersky, S Fidler, R Salakhutdinov (2015)
  7. Zero-Shot Learning with Structured Embeddings
    – Authors: Z Akata, H Lee, B Schiele (2014)
Continue Reading

Deep Learning with Generative and Generative Adverserial Networks – ICLR 2017 Discoveries

The 5th International Conference on Learning Representation (ICLR 2017) is coming to Toulon, France (April 24-26 2017).

This blog post gives an overview of Deep Learning with Generative and Adverserial Networks related papers submitted to ICLR 2017, see underneath for the list of papers. Want to learn about these topics? See OpenAI’s article about Generative Models and Ian Goodfellow et.al’s paper about Generative Adversarial Networks.

Best regards,

Amund Tveit

ICLR 2017 – Generative and Generative Adversarial Papers

  1. Unsupervised Learning Using Generative Adversarial Training And Clustering – Authors: Vittal Premachandran, Alan L. Yuille
  2. Improving Generative Adversarial Networks with Denoising Feature Matching – Authors: David Warde-Farley, Yoshua Bengio
  3. Generative Adversarial Parallelization – Authors: Daniel Jiwoong Im, He Ma, Chris Dongjoo Kim, Graham Taylor
  4. b-GAN: Unified Framework of Generative Adversarial Networks – Authors: Masatosi Uehara, Issei Sato, Masahiro Suzuki, Kotaro Nakayama, Yutaka Matsuo
  5. Generative Adversarial Networks as Variational Training of Energy Based Models – Authors: Shuangfei Zhai, Yu Cheng, Rogerio Feris, Zhongfei Zhang
  6. Boosted Generative Models – Authors: Aditya Grover, Stefano Ermon
  7. Adversarial examples for generative models – Authors: Jernej Kos, Dawn Song
  8. Mode Regularized Generative Adversarial Networks – Authors: Tong Che, Yanran Li, Athul Jacob, Yoshua Bengio, Wenjie Li
  9. Variational Recurrent Adversarial Deep Domain Adaptation – Authors: Sanjay Purushotham, Wilka Carvalho, Tanachat Nilanon, Yan Liu
  10. Structured Interpretation of Deep Generative Models – Authors: N. Siddharth, Brooks Paige, Alban Desmaison, Jan-Willem van de Meent, Frank Wood, Noah D. Goodman, Pushmeet Kohli, Philip H.S. Torr
  11. Inference and Introspection in Deep Generative Models of Sparse Data – Authors: Rahul G. Krishnan, Matthew Hoffman
  12. Generative Models and Model Criticism via Optimized Maximum Mean Discrepancy – Authors: Dougal J. Sutherland, Hsiao-Yu Tung, Heiko Strathmann, Soumyajit De, Aaditya Ramdas, Alex Smola, Arthur Gretton
  13. Unsupervised sentence representation learning with adversarial auto-encoder – Authors: Shuai Tang, Hailin Jin, Chen Fang, Zhaowen Wang
  14. Unsupervised Program Induction with Hierarchical Generative Convolutional Neural Networks – Authors: Qucheng Gong, Yuandong Tian, C. Lawrence Zitnick
  15. A Theoretical Framework for Robustness of (Deep) Classifiers against Adversarial Noise – Authors: Beilun Wang, Ji Gao, Yanjun Qi
  16. On the Quantitative Analysis of Decoder-Based Generative Models – Authors: Yuhuai Wu, Yuri Burda, Ruslan Salakhutdinov, Roger Grosse
  17. Evaluation of Defensive Methods for DNNs against Multiple Adversarial Evasion Models – Authors: Xinyun Chen, Bo Li, Yevgeniy Vorobeychik
  18. Calibrating Energy-based Generative Adversarial Networks – Authors: Zihang Dai, Amjad Almahairi, Philip Bachman, Eduard Hovy, Aaron Courville
  19. Inverse Problems in Computer Vision using Adversarial Imagination Priors – Authors: Hsiao-Yu Fish Tung, Katerina Fragkiadaki
  20. Towards Principled Methods for Training Generative Adversarial Networks – Authors: Martin Arjovsky, Leon Bottou
  21. Learning to Draw Samples: With Application to Amortized MLE for Generative Adversarial Learning – Authors: Dilin Wang, Qiang Liu
  22. Multi-view Generative Adversarial Networks – Authors: Mickaël Chen, Ludovic Denoyer
  23. LR-GAN: Layered Recursive Generative Adversarial Networks for Image Generation – Authors: Jianwei Yang, Anitha Kannan, Dhruv Batra, Devi Parikh
  24. Semi-Supervised Learning with Context-Conditional Generative Adversarial Networks – Authors: Emily Denton, Sam Gross, Rob Fergus
  25. Generative Adversarial Networks for Image Steganography – Authors: Denis Volkhonskiy, Boris Borisenko, Evgeny Burnaev
  26. Unrolled Generative Adversarial Networks – Authors: Luke Metz, Ben Poole, David Pfau, Jascha Sohl-Dickstein
  27. Generative Multi-Adversarial Networks – Authors: Ishan Durugkar, Ian Gemp, Sridhar Mahadevan
  28. Joint Multimodal Learning with Deep Generative Models – Authors: Masahiro Suzuki, Kotaro Nakayama, Yutaka Matsuo
  29. Fast Adaptation in Generative Models with Generative Matching Networks – Authors: Sergey Bartunov, Dmitry P. Vetrov
  30. Adversarially Learned Inference – Authors: Vincent Dumoulin, Ishmael Belghazi, Ben Poole, Alex Lamb, Martin Arjovsky, Olivier Mastropietro, Aaron Courville
  31. Perception Updating Networks: On architectural constraints for interpretable video generative models – Authors: Eder Santana, Jose C Principe
  32. Energy-based Generative Adversarial Networks – Authors: Junbo Zhao, Michael Mathieu, Yann LeCun
  33. Simple Black-Box Adversarial Perturbations for Deep Networks – Authors: Nina Narodytska, Shiva Kasiviswanathan
  34. Learning in Implicit Generative Models – Authors: Shakir Mohamed, Balaji Lakshminarayanan
  35. On Detecting Adversarial Perturbations – Authors: Jan Hendrik Metzen, Tim Genewein, Volker Fischer, Bastian Bischoff
  36. Delving into Transferable Adversarial Examples and Black-box Attacks – Authors: Yanpei Liu, Xinyun Chen, Chang Liu, Dawn Song
  37. Adversarial Feature Learning – Authors: Jeff Donahue, Philipp Krähenbühl, Trevor Darrell
  38. Generative Paragraph Vector – Authors: Ruqing Zhang, Jiafeng Guo, Yanyan Lan, Jun Xu, Xueqi Cheng
  39. Adversarial Machine Learning at Scale – Authors: Alexey Kurakin, Ian J. Goodfellow, Samy Bengio
  40. Adversarial Training Methods for Semi-Supervised Text Classification – Authors: Takeru Miyato, Andrew M. Dai, Ian Goodfellow
  41. Sampling Generative Networks: Notes on a Few Effective Techniques – Authors: Tom White
  42. Adversarial examples in the physical world – Authors: Alexey Kurakin, Ian J. Goodfellow, Samy Bengio
  43. Improving Sampling from Generative Autoencoders with Markov Chains – Authors: Kai Arulkumaran, Antonia Creswell, Anil Anthony Bharath
  44. Neural Photo Editing with Introspective Adversarial Networks – Authors: Andrew Brock, Theodore Lim, J.M. Ritchie, Nick Weston
  45. Learning to Protect Communications with Adversarial Neural Cryptography – Authors: Martín Abadi, David G. Andersen

Sign up for Deep Learning newsletter!


Continue Reading

Deep Learning for Mobile Personal Expression at Zedge

Wrote a blog post about Deep Learning for Mobile Personal Expression, entire blog post is available at: http://corp.zedge.net/developers-blog/deep-learning-at-zedge — and start of blog post is shown underneath.

Our main product is an app — Zedge Ringtones & Wallpapers — that provides wallpapers, ringtones, app icons, game recommendations and notification sounds customized for your mobile device. Zedge apps have been downloaded more than 200 million times for iOS and Android and is used by millions of people worldwide each month.

People use our apps for self-expression. Setting a wallpaper, ringtone or app icons on your mobile device is in many ways similar to selecting clothes, hairstyle or other fashion statements. In fact people try a wallpaper or ringtone in a similar manner as they would try clothes in a dressing room before making a purchase decision, they try different wallpapers or ringtones before deciding on one they want to keep for a while.

The decision for selecting a wallpaper is not taken lightly, since people interact and view their mobile device (and background wallpaper) a lot:

… The entire blog post is available at:

http://corp.zedge.net/developers-blog/deep-learning-at-zedge

Best regards,

Amund Tveit

Continue Reading

Why Deep Learning matters

SamsungRS80AWithPrestige

Deep Learning, or more specifically a subgroup of Deep Learning called (Deep) Convolutional Neural Networks have had impressive improvements since Alex Krizhevsky’s 2012 publication about (what is now called) AlexNet. AlexNet won the ImageNet Image Recognition competition with the (then close to jawdropping) top-5 error rate of only 17.0% (top-5 error means that if your classifier presents 5 answers at least one of them must be the correct one).

But Image Recognition accuracy have increased many times since then, i.e. from 17% in 2012 to 3.08% in 2016 (see publications in table below to see more about what the error rates mean and how they can be compared).

To put this into context: human beings perform at 5.10% error rate on this Image Recognition task, see Andrej Karpathy’s publication below (to be precise: at least 1 smart, trained and highly education human being performed at that error rate on the ImageNet task).

So what I am saying is that computers with Deep Learning can actually see and understand what is on a picture better than humans! (in some and probably most/all cases)

Year Error% Reference Author(s) Organization
2012 17.00 ImageNet Classification with Deep Convolutional Neural Networks (AlexNet) Alex Krizhevsky et. al University of Toronto
2014 6.66 Going Deeper with Convolutions Christian Szegedy et. al Google
2014 (Sep) 5.10 What I learned from competing against a ConvNet on ImageNet Andrej Karpathy Stanford University
2015 (Feb) 4.94 Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification Kaiming He et. al Microsoft Research
2015 (Dec) 3.57 Deep Residual Learning for Image Recognition Kaiming He et. al Microsoft Research
2016 (Feb) 3.08 Inception v4, Inception-ResNet and the Impact of Residual Connections on Learning Christian Szegedy et. al Google

Implications of better-than-human-level image recognition with Deep Learning?

Deep Convolutional Neural network research field has moved so fast that applications still lag behind on using this. Most robots/drones and software in servers, laptops, mobiles, wearables and medical equipment does not take advantage of these research results yet, but there is a huge untapped potential (will get back to the potential in later postings).
But there are some highly important applications already, e.g. the Samsung Medison RS80A Ultrasound Machine (see image in start of posting) that uses convolutional neural networks for Breast Cancer Diagnosis.

Typical_cnn

Next blog post is probably going to be about some (simple) analogies to explain the mechanics of Convolutional Neural Networks. Stay tuned and sign up for DeepLearning.Education mailing list below.

Best regards,
Amund Tveit

 

Continue Reading