Scaling down images is a craft, scaling up images is an art
Since in the scaling down to a lower resolution you typically need to remove pixels, but in the case of scaling up you need to invent new pixels. But some Deep Learning models with Convolutional Neural Networks (and frequently Deconvolutional layers) has shown successful to scale up images, this is called Image Super-Resolution. These models are typically trained by taking high resolution images and reducing them to lower resolution and then train in the opposite way. Partially related: Recommend also checking out Odeon et. al’s Distill.pub’s publication: Deconvolution and Checkerboard Artifacts that goes into more detail about the one the core operators used in Image Super-Resolution.
This blog post has an overview papers related to acoustic modelling primarily for speech recognition but also speech generation (synthesis). See also ai.amundtveit.com/keyword/acoustic for a broader set of (at the time of writing 73) recent Deep Learning papers related to acoustics for speech recognition and other applications of acoustics.
Acoustic Modelling is described in Wikipedia as: “An acoustic model is used in Automatic Speech Recognition to represent the relationship between an audio signal and the phonemes or other linguistic units that make up speech. The model is learned from a set of audio recordings and their corresponding transcripts”.
For the last couple of months I’ve been creating bibliographies of recent academic publications in various subfields of Deep Learning on this blog. This posting gives an overview of the last 25 bibliographies posted.
This posting presents recent publications related to Deep Learning for Question Answering. Question Answering is described as “a computer science discipline within the fields of information retrieval and natural language processing (NLP), which is concerned with building systems that automatically answer questions posed by humans in a natural language”. I’ll also publish postings about Deep Learning for Information Retrieval and Learning to Rank today.
Ensemble Based Machine Learning has been used with success in several Kaggle competitions, and this year also the Imagenet competition was dominated by ensembles in Deep Learning, e.g. Trimps-Soushen team from 3rd Research Institute of the Ministry of Public Security (China) used a combination of Inception, Inception-Resnet, Resnet and Wide Residual Network to win the Object Classification/localization challenge. This blog post has recent papers related to Ensembles in Deep Learning.
EEG (Electroencephalography) is the measurement of electrical signals in the brain. It has long been used for medical purposes (e.g. diagnosis of epilepsy), and has in more recent years also been used in Brain Computer Interfaces (BCI) — note: if BCI is new to you don’t get overly excited about it, since these interfaces are still in my opinion quite premature. But they are definitely interesting in a longer term perspective .
This blog post gives an overview of recent research on Deep Learning in combination with EEG, e.g. r for classification, feature representation, diagnosis, safety (cognitive state of drivers) and hybrid methods (Computer Vision or Speech Recognition together with EEG and Deep Learning).
This blog post has recent papers related to embedding for Natural Language Processing with Deep Learning. Example application areas embedding is used for in the papers include finance (stock market prediction), biomedical text analysis, part-of-speech tagging, sentiment analysis, pharmacology (drug adverse effects).
Alzheimer’s Disease is the cause of 60–70% of cases of Dementia, costs associated to diagnosis, treatment and care of patients with it is estimated to be in the range of a hundred billion dollars in USA. This blog post have some recent papers related to using Deep Learning for diagnostics and decision support related to Alzheimer’s disease.
Ultrasound (also called Sonography) are sound waves with higher frequency than humans can hear, they frequently used in medical settings, e.g. for checking that pregnancy is going well with fetal ultrasound. For more about Ultrasound data formats check out Ultrasound Research Interface. This blog post has recent publications about applying Deep Learning for analyzing Ultrasound data.
Deep Learning (creative AI) might potentially be used for music analysis and music creation. Deepmind’s Wavenet is a step in that direction. This blog post presents recent papers in Deep Learning for Music.
This blog post gives an overview of papers related to using Regularization in Deep Learning submitted to ICLR 2017, see underneath for the list of papers. If you want to learn about Regularization in Deep Learning check out: www.deeplearningbook.org/contents/regularization.html
This blog post gives an overview of papers related to Unsupervised Deep Learning submitted to ICLR 2017, see underneath for the list of papers. If you want to learn about Unsupervised Deep Learning check out: Ruslan Salkhutdinov’s video Foundations of Unsupervised Deep Learning.
This blog post gives an overview of Natural Language Processing related papers submitted to ICLR 2017, see underneath for the list of papers. If you want to learn about Deep Learning with NLP check out Stanford’s CS224d: Deep Learning for Natural Language Processing
Our main product is an app — Zedge Ringtones & Wallpapers — that provides wallpapers, ringtones, app icons, game recommendations and notification sounds customized for your mobile device. Zedge apps have been downloaded more than 200 million times for iOS and Android and is used by millions of people worldwide each month.
People use our apps for self-expression. Setting a wallpaper, ringtone or app icons on your mobile device is in many ways similar to selecting clothes, hairstyle or other fashion statements. In fact people try a wallpaper or ringtone in a similar manner as they would try clothes in a dressing room before making a purchase decision, they try different wallpapers or ringtones before deciding on one they want to keep for a while.
The decision for selecting a wallpaper is not taken lightly, since people interact and view their mobile device (and background wallpaper) a lot:
Android — Research by DScout (June 2016) showed that average users in their study daily touched their phone 2617 times, used their phone for 145 minutes, engaged in 76 separate phone sessions, 47% of sessions were on the lock screen (references: http://www.businessinsider.com… andhttp://blog.dscout.com/mobile-…)
Deep Learning, or more specifically a subgroup of Deep Learning called (Deep) Convolutional Neural Networks have had impressive improvements since Alex Krizhevsky’s 2012 publication about (what is now called) AlexNet. AlexNet won the ImageNet Image Recognition competition with the (then close to jawdropping) top-5 error rate of only 17.0% (top-5 error means that if your classifier presents 5 answers at least one of them must be the correct one).
But Image Recognition accuracy have increased many times since then, i.e. from 17% in 2012 to 3.08% in 2016 (see publications in table below to see more about what the error rates mean and how they can be compared).
To put this into context: human beings perform at 5.10% error rate on this Image Recognition task, see Andrej Karpathy’s publication below (to be precise: at least 1 smart, trained and highly education human being performed at that error rate on the ImageNet task).
So what I am saying is that computers with Deep Learning can actually see and understand what is on a picture better than humans! (in some and probably most/all cases)
Implications of better-than-human-level image recognition with Deep Learning?
Deep Convolutional Neural network research field has moved so fast that applications still lag behind on using this. Most robots/drones and software in servers, laptops, mobiles, wearables and medical equipment does not take advantage of these research results yet, but there is a huge untapped potential (will get back to the potential in later postings).
But there are some highly important applications already, e.g. the Samsung Medison RS80A Ultrasound Machine (see image in start of posting) that uses convolutional neural networks for Breast Cancer Diagnosis.
Next blog post is probably going to be about some (simple) analogies to explain the mechanics of Convolutional Neural Networks. Stay tuned and sign up for DeepLearning.Education mailing list below.