The ability to generalise over new tasks is incredibly important in research when aiming to build scalable reinforcement learning agents. At the Deep Learning Summit in San Francisco this January 25 & 26, Junhyuk Oh, PhD Researcher at University of Michigan will discuss the challenges and progressions on reaching task generalisation with deep reinforcement learning. He will explore how to easily train agents to categorise and generalise previously unseen tasks in a zero-shot fashion.
In the run up to the event, we spoke with Junhyuk to hear more about his current work in the field.
I was very lucky to start my PhD by collaborating with Prof. Honglak and Prof. Satinder who are experts in deep learning and reinforcement learning respectively. I was originally interested in deep generative models before I started my PhD. However, after finishing my first project on video prediction in Atari games, I became much more interested in reinforcement learning problems because I have been curious how and why humans learn and act in the world and whether we can mathematically formulate it.
I think sample complexity (data efficiency) is the main challenge for building scalable reinforcement learning agents. This is mainly because it is usually hard to get labeled data (e.g., optimal demonstrations) in many reinforcement learning domains. We have been relying on distributed computing and fast simulators to handle this issue. But, I think we need more algorithmic advances such as few-shot learning and improved generalization to overcome this challenge.
Although I am mostly working on reinforcement learning problems with simulators, which may not have an immediate positive impact, I believe that advanced techniques in RL will have a big impact on many future applications and products.
I often find that there is no proper domain/environment to evaluate different aspects of reinforcement learning agents such as generalization and planning. I sometimes had to spend a huge amount of time implementing a new environment (e.g., Minecraft) to explore new problems and challenges. Besides, such a new environment should be very fast so that we can run large-scale experiments, which requires a lot of engineering.
I am personally excited about recent advances in generative adversarial networks (GANs), because they have been shown to be able to generate extremely realistic examples. This line of work seems very useful in fashion, music, graphics, and design industry. I think machine learning techniques in general can have immediate impacts on healthcare, smart homes, and self-driving cars for the next few years.
I would say AI will make our lives easier rather than 'steal' our jobs. I agree that AI can possibly replace many existing jobs, so we should be careful and prepared for such a sudden change in the industry and the job market. Nevertheless, I believe that AI will have a huge positive impact in the long run (e.g., reduced working hours), as long as we manage such risks well.
To hear more about Junhyuk's current research and most recent progressions, join us at the Deep Learning Summit in San Francisco. In the spirit of Black Friday and Cyber Monday we're offering 25% of passes to all summits with the code CYBER25. Register before Wednesday Nov 29th to receive the discount.