Ruben Glatt

Deep Symbolic Optimization (DSO) – A Reinforcement Learning-based Framework for Combinatorial Optimization

Deep Learning and Deep Reinforcement Learning have proven successful for many difficult regression and control problems by learning models represented by neural networks. However, the complexity of neural network-based models, involving thousands of composed nonlinear operators, can render them problematic to understand, trust, and deploy. In contrast, simple tractable symbolic expressions can facilitate human understanding, while also being transparent and exhibiting predictable behavior.

In this talk, we show how the DSO framework leverages deep learning for symbolic optimization via a simple idea: use a large model to search the space of small models. Specifically, we use an autoregressive recurrent neural network to emit a distribution over tractable mathematical expressions and employ a novel risk-seeking policy gradient to train the model to generate higher-performing objects. This framework can be applied to optimize hierarchical, variable-length objects under a black box performance metric, with the ability to incorporate constraints in situ, reducing the search space as we limit exploration based on established knowledge.

Ruben is a Machine Learning Researcher at the Lawrence Livermore National Laboratory (LLNL). With a background in Mechatronics and Mechanical Engineering, he has turned to Artificial Intelligence where his main interest lies in Machine Learningresearch with a focus on Reinforcement Learning, autonomous systems, and applications in energy efficiency. He received his Ph.D. in Computer Engineering in the area of ML at the University of São Paulo (USP), Brazil, holds a master degree in Mechanical Engineering in the area of controlling mechanical systems from the Universidade Estadual Paulista Júlio de Mesquita Filho(UNESP), Brazil, and a Diplom-Ingenieur degree in Mechatronics in the area of sensors and robotics from the Karlsruhe Institute of Technology (KIT), Germany. Ruben has acquired years of professional experience before and during his studies while working in the technology and energy sector, as well as in the organization of international ML conferences. After converting from a postdoctoral position at the Lawrence Livermore National Laboratory (LLNL), USA, he is now working as a Machine Learning Researcher on a variety of RL projects to develop methods for collaborative autonomy in multi-agent systems, interpretable RL, and real-world applications. Ruben’s long-term research interest lies in successfully applying RL techniques to real-world challenges to accelerate and improve decision-making, autonomously or as a support tool for humans, preferably for applications in energy and smart mobility systems.

Buttontwitter Buttonlinkedin
This website uses cookies to ensure you get the best experience. Learn more