Muppets and Transformers: The New Stars of NLP
The last few years have seen huge progress in NLP. Transformers have become a fundamental building block for impressive new NLP models. ELMo, BERT, and their descendants have achieved new state-of-the-art results on a wide variety of tasks. In this talk I'll give some history of these "new stars" of NLP, explain how they work, compare them to their predecessors, and discuss how you can apply them to your own problems.
Joel Grus is Principal Engineer at Capital Group, where he oversees the development and deployment of machine learning systems. Previously he was a research engineer at the Allen Institute for Artificial Intelligence, where he helped develop AllenNLP, a deep learning library for NLP researchers. Before that he worked as a software engineer at Google and a data scientist at a variety of startups. He is the author of the beloved book Data Science from Scratch: First Principles with Python, the beloved blog post "Fizz Buzz in Tensorflow", and the polarizing JupyterCon talk "I Don't Like Notebooks". You can find him on Twitter @joelgrus