Extending the Frontiers of Deep Learning: Data-efficiency and Continual Learning
Deep learning has revolutionised many facets of machine learning, however it suffers from a number of crucial limitations that severely limit its applicability to real world problems. For example, deep learning is data hungry requiring large numbers of labelled data points. Deep learning also fails catastrophically in continual learning scenarios, where data and tasks arrive continuously and must be leaned from in an incremental way. These failings are particularly limiting in healthcare (where data are typically scarce) and personalisation (where predictions must be incrementally updated as new data and users arrive). I will discuss how we have recently started to address these limitations by fusing deep learning with probabilistic modelling techniques. I will give a short introduction to probabilistic modelling in the context of deep learning. I will then show how the new synthesis can be used to successfully deploy deep learning to small data settings, such as few shot learning tasks from computer vision. I will also show that incremental updates, necessary for continual learning, are can be naturally handled by the same methods. In both scenarios, the new systems substantially improve the state-of-the-art.
Dr. Richard E. Turner is a Reader in Machine Learning at the University of Cambridge and a Visiting Researcher at Microsoft Research Cambridge. His research fuses probabilistic machine learning and deep learning to develop robust, data-efficient, flexible and automated learning systems. Richard helps lead Cambridge’s renowned Machine Learning Group, the Machine Learning and Machine Intelligence MPhil, the Centre for Doctoral Training in AI for Environmental Risk, and the Cambridge Big Data Strategic Initiative. He studied for his PhD at the Gatsby Computational Neuroscience Unit at UCL and spent his Postdoctoral Fellowship at New York University in the Laboratory for Computational Vision. He has been awarded the Cambridge Students' Union Teaching Award for Lecturing and his work has featured on BBC Radio 5 Live’s The Naked Scientist, BBC World Service’s Click and in Wired Magazine.