Long overdue update of new publications in Deep Learning Publication Navigator (ai.amundtveit.com) – for now the easiest way to discover new publications is probably to convert screenshots (number of papers) per category in the before and after update screenshots below.
Examples of keywords (from publication title) with (several) new Deep Learning publications are:
Crowd Funding is a way new projects aim to get funding from (typically many) people (per funding campaign) that are not professional investors (hence the word “crowd”), e.g. at Indigogo and Kickstarter (or services such as GoFundMe, CrowdRise, RocketHub and many others) . The diversity of crowd funding projects is very high, e.g. charity funding of people and organizations as well as funding of startups (typically for product development) in an early phase (by buying the product before it is ready). Probably the most well-known startup that got crowd funding was Virtual Reality startup Oculus VR that raised 2.5 million USD from Kickstarter in 2012, and was acquired by Facebook for 2 billion USD in 2014
1. Equity Crowd Funding
However, from a financial perspective the people or companies that help fund crowd funding campaigns get very little returns (note: not to discount the feeling of helping and making projects happen). With Equity Crowd Funding this is different, it is similar to Crowd Funding that people invest moderate amounts (at least compared to what an angel investor, venture capitalist or private equity would do), but it also gives the funders equity (stocks, options or equity-guaranteed convertible loans) in the startup. Startups are an incredibly riskyinvestment since most never succeed hence provide zero returns (just loss and not to forget opportunity loss). In a quickly moving world (due to accelerating technology change, e.g. in areas such as AI/Deep Learning and Robotics) with very low interest rates getting any kind of return on investment is very hard (without taking risk).
Let me give you examples of hard it is to get high return of investment (ROI) with low risk:
a) In the 1980s the Norwegian postal bank had a “Gullbok” (Gold Book) savings account that provided around 11-13% interest rates – which seems almost unbelievable today – but had probably relatively high risk at the time – Norway did significant devaluations of the Krone currency relative to other currencies both in May 1986 and 1993 (the latter when the Norwegian bank sector almost collapsed)
b) Recently saw an ad for a regional bank’s savings account where you had to lock more than 50 thousand USD for more than a year to get less than 2% interest rate (Norway’s Bank target inflation rate is 2.5% which roughly means that you get -0.5% annual ROI from a purchasing parity view instead of 2%) (This ROI estimate is probably less risky than the one in the 1980s)
2. Crowd Equity Funding is Very Risky
For those that are willing to take a much higher risk of loosing all their invested money Crowd Equity Funding can be an approach, but please keep in mind that Crowd Equity Funding should be considered in a similar way of considering buying tickets in the lottery, doing any kind of gambling, giving away money or as regular crowd funding, i.e. only surplus money that you can afford to loose entirely and never get any ROI from. The U.S. Securities and Exchange Commission proposed crowd equity related regulations to protect people from gambling away their money, for most people the upper bound would be maximum of either $2000 or 5% of annual income or net worth.
As opposed to regular startup funding done by angel investors and venture capitalists – where Silicon Valley is absolutely leading, my impression is that crowd equity funding is so far most common in Europe and in particular in Nordics and UK (probably due to the novelty of the previously mentioned SEC regulations for crowd equity funding, see SEC’s update from May 2017). Examples of Equity Crowd Funding platforms are:
What the Equity Crowd Funding platforms have in common is that they want to provide easy-to-use and transparent platforms for doing investing with relatively high security for both the crowd equity investors and the startup, i.e. there are quite stringent requirements for registrations (for investors) and documentation about the investment round (for startups). However, there is still significant risk involved in investing.
5. Realize Returns of Startup Investments
A challenge investing in startups is how to realize returns (despite having grown), since you typically can not sell shares directly as you could with publically listed companies on a reasonably liquid stock exchange (note that Angel List reported unrealized returns for their 2013 Syndicate, see above).
A few years back there were massive amounts of startup acquisitions – some at very early stage – performed primarily by public tech companies (e.g. Alphabet(Google), Facebook, Apple, Microsoft and others) or large late-stage startups (e.g Uber, Airbnb and other unicorn startups) – (try a web search for: list of startup acquisitions by PutCompanyNameHere to get an overview) this meant that for a lucky startup investor there was a chance for a quick realized return, however in most cases – even for successful startups (some big unicorn startups have strict regulations on share sales/purchases) – it is very hard to realize returns unless the startup does an IPO or get acquired by a bigger company (in some countries startups can be traded at listed smaller exchanges – Over The Counter (OTC) – which has less regulations than the large public stock exchanges and are typically considered much riskier than the larger exchanges wrt liquidity and pricing)
Startups want and need funding, and despite being very high risk investments Equity Crowd Funding aims to make it easier to get funds for startups and to invest for investors (and perhaps realize if there are returns in secondary markets), and it is a very interesting area to follow. But please take into consideration the immense risk if wanting to take the step into becoming a crowd equity investor, being involved the startup world can become addictive, but remember that you are playing with real money. If you want to learn more about the topic I recommend the book Equity Crowdfunding: The Complete Guide For Startups And Growing Companies (by Nathan Rose)
For easy portability I chose to run the Keras part inside docker (i.e. could e.g. use nvidia-docker for a larger model that would need a GPU to train e.g. in the cloud or on a desktop or a powerful laptop). The current choice of Keras backend was TensorFlow, but believe it should also work for other backends (e.g. CNTK, Theano or MXNet). The code for this blog post is available at github.com/atveit/keras2ios
This blog post has an overview papers related to acoustic modelling primarily for speech recognition but also speech generation (synthesis). See also ai.amundtveit.com/keyword/acoustic for a broader set of (at the time of writing 73) recent Deep Learning papers related to acoustics for speech recognition and other applications of acoustics.
Acoustic Modelling is described in Wikipedia as: “An acoustic model is used in Automatic Speech Recognition to represent the relationship between an audio signal and the phonemes or other linguistic units that make up speech. The model is learned from a set of audio recordings and their corresponding transcripts”.
This blog post has recent papers about Deep Learning for authentication, e.g. iris (eye), fingerprint and various other patterns of the user, e.g. behavior writing style (stylometry) and other user patterns. Partially related is the Quora question and answer: How can Deep Learning be used for Computer Security?
Tweets (i.e. microblogging with very short documents) is a frequent data source in machine learning, e.g. for sentiment analysis and financial (stock) predictions. Here are some recent papers related to use of Analyzing Twitter Data with Deep Learning. (note: Twitter itself also does Deep Learning on Twitter data with its Cortex Team). Many of these papers could probably also apply similar data sources such as e.g. Weibo or Facebook.
For the last couple of months I’ve been creating bibliographies of recent academic publications in various subfields of Deep Learning on this blog. This posting gives an overview of the last 25 bibliographies posted.
This posting presents recent publications related to Deep Learning for Question Answering. Question Answering is described as “a computer science discipline within the fields of information retrieval and natural language processing (NLP), which is concerned with building systems that automatically answer questions posed by humans in a natural language”. I’ll also publish postings about Deep Learning for Information Retrieval and Learning to Rank today.
Ensemble Based Machine Learning has been used with success in several Kaggle competitions, and this year also the Imagenet competition was dominated by ensembles in Deep Learning, e.g. Trimps-Soushen team from 3rd Research Institute of the Ministry of Public Security (China) used a combination of Inception, Inception-Resnet, Resnet and Wide Residual Network to win the Object Classification/localization challenge. This blog post has recent papers related to Ensembles in Deep Learning.
EEG (Electroencephalography) is the measurement of electrical signals in the brain. It has long been used for medical purposes (e.g. diagnosis of epilepsy), and has in more recent years also been used in Brain Computer Interfaces (BCI) — note: if BCI is new to you don’t get overly excited about it, since these interfaces are still in my opinion quite premature. But they are definitely interesting in a longer term perspective .
This blog post gives an overview of recent research on Deep Learning in combination with EEG, e.g. r for classification, feature representation, diagnosis, safety (cognitive state of drivers) and hybrid methods (Computer Vision or Speech Recognition together with EEG and Deep Learning).
This blog post has recent papers related to embedding for Natural Language Processing with Deep Learning. Example application areas embedding is used for in the papers include finance (stock market prediction), biomedical text analysis, part-of-speech tagging, sentiment analysis, pharmacology (drug adverse effects).
Alzheimer’s Disease is the cause of 60–70% of cases of Dementia, costs associated to diagnosis, treatment and care of patients with it is estimated to be in the range of a hundred billion dollars in USA. This blog post have some recent papers related to using Deep Learning for diagnostics and decision support related to Alzheimer’s disease.
Ultrasound (also called Sonography) are sound waves with higher frequency than humans can hear, they frequently used in medical settings, e.g. for checking that pregnancy is going well with fetal ultrasound. For more about Ultrasound data formats check out Ultrasound Research Interface. This blog post has recent publications about applying Deep Learning for analyzing Ultrasound data.
Deep Learning (creative AI) might potentially be used for music analysis and music creation. Deepmind’s Wavenet is a step in that direction. This blog post presents recent papers in Deep Learning for Music.
This blog post gives an overview of papers related to using Regularization in Deep Learning submitted to ICLR 2017, see underneath for the list of papers. If you want to learn about Regularization in Deep Learning check out: www.deeplearningbook.org/contents/regularization.html
This blog post gives an overview of papers related to Unsupervised Deep Learning submitted to ICLR 2017, see underneath for the list of papers. If you want to learn about Unsupervised Deep Learning check out: Ruslan Salkhutdinov’s video Foundations of Unsupervised Deep Learning.
This blog post gives an overview of Natural Language Processing related papers submitted to ICLR 2017, see underneath for the list of papers. If you want to learn about Deep Learning with NLP check out Stanford’s CS224d: Deep Learning for Natural Language Processing