MultiModal Conversational AI
With recent progress in deep learning, there has been an increased interest in multimodal conversational AI, which requires an AI agent to hold a meaningful conversation with humans in Natural Language about content in other modalities, e.g. pictures or videos. In this talk, I will present two case studies: one in generating responses for closed-domain task-based multimodal dialogue systems with applications in conversational multimodal search; and one case-study in selecting/ retrieving responses for open-domain multimodal systems with applications in visual dialogue and visual question answering.
Throughout my talk I will highlight open challenges, including context modelling, knowledge grounding, encoding history, multimodal fusion, evaluation techniques, and shortcomings of current datasets.
Verena Rieser is a Professor in Artificial Intelligence at Heriot-Watt University, Edinburgh, where she leads the NLP lab and is affiliated with the Interaction Lab and the Edinburgh Center for Robotics. Verena holds a PhD from Saarland University (2008) and did her PostDoc at the University of Edinburgh (2008-2011). She also has 2 children.
The ongoing theme of Verena's research is to develop intelligent conversational systems, such as chatbots and virtual personal assistants, where she researches machine learning techniques to automatically build these systems from data. She has authored over 100 peer-reviewed papers in this area. For example, Verena was one of the first researchers to introduce Reinforcement Learning to optimise task-based dialogues. For the past 2 years, Verena's team was the only UK university to enter the finals of the prestigious Amazon Alexa Challenge, which aims to build open-domain chatbots.
Verena's work was also recognised with a number of prizes, including the Dr-Eduard-Martin Prize for outstanding research, winner of the SemEval 2016 Shared Task on Arabic Sentiment Analysis, and runner-up for the Amazon Alexa Prize 2017 and 2018. In 2019 alone, Verena was invited to be the keynote speaker of 7 international conferences and workshops. Verena's outreach activities have been recognised with the EPSRC Public Engagement Award 2018, a Herald Higher Education Award 2018 for Heriot-Watt's "Year of Robotics", as well as with a nomination as one of "12 Women Shaping AI" by NESTA, "One to Watch 2019" by FutureScot and as "Pioneering Women in AI" by The Telegraph, 2019.