Building Scalable Framework and Environment for Reinforcement Learning
Deep Reinforcement Learning (DRL) has made strong progress in many tasks that are traditionally considered to be difficult, such as complete information games, navigation, architecture search, etc. Although the basic principle of DRL is quite simple and straightforward, to make it work often requires substantial efforts, compared to traditional supervised training. In this talk, we introduce our recent open-sourced ELF platform: efficient, lightweight and flexible frameworks to facilitate DRL research. We show the scalability of our platforms by reproducing and open sourcing AlphaGoZero/AlphaZero framework using 2000 GPUs and 1.5 weeks, achieving super-human performance of Go AI that beats 4 top-30 professional players with 20-0. We also show usability of our platform by training agents in real-time strategy games with only a small amount of resource. The trained agent develops interesting tactics and is able to beat rule-based AIs by a large margin. On the environment side, we propose House3D that makes multi-room navigation easy with fast frame rate. With House3D, we show that model-based agent that plans ahead with uncertain information navigate in unseen environments more successfully.
Yuandong Tian is a Research Scientist and Manager in Facebook AI Research, working on deep reinforcement learning and its applications in games, and theoretical analysis of deep models. He is the lead scientist and engineer for ELF Platform for Reinforcement Learning, OpenGo and DarkForest Go project. Prior to that, he was a researcher and engineer in Google Self-driving Car team in 2013-2014. He received Ph.D in Robotics Institute, Carnegie Mellon University on 2013, Bachelor and Master degree of Computer Science in Shanghai Jiao Tong University. He is the recipient of 2013 ICCV Marr Prize Honorable Mentions.