Towards Teaching Machines to Read and Reason
We are getting better at teaching machines how to answer questions about content in natural language text. However, progress has been mostly restricted to extracting answers that are directly stated in text. In this talk, I will discuss our work towards teaching machines not only to read, but also to reason with what was read. We investigate two complementary approaches. In the first, we incorporate reasoning into neural models by enabling them to manipulate memory in a hierarchical sequence of steps—just like a virtual machine that executes a program. In the second, we take black box models (neural or not) and keep their structure as is. Instead, we develop rewards and penalties that encourage the model to behave as if it reasons. I will show the effectiveness of these approaches on various benchmark datasets, including relation reasoning and solving math word problems.
Sebastian is a Reader at University College London, leads the UCL Machine Reading group and is Co-Founder and Head of Research at Bloomsbury AI. He is an Allen Distinguished Investigator and received an $1M award from the Paul Allen Foundation to 'move the needle' towards answering broad scientific questions in AI. Sebastian is generally interested in the intersection of Natural Language Processing and Machine Learning, and particularly interested in teaching machines to read and to reason with what was read.