Divide and Conquer Networks
Many engineering and scientific tasks require solving algorithmic tasks at scale under computational constraints. In general, these constraints are hard to optimize in closed form, motivating the use of data-driven algorithmic learning. Rather than studying this as a black-box discrete regression problem with no assumption whatsoever on the input-output mapping, we concentrate on tasks that are amenable to the principle of divide and conquer, and study what are its implications in terms of learning. This principle creates a powerful inductive bias that can be leveraged with neural architectures that are defined recursively and dynamically, by learning two scale-invariant atomic operations: how to split a given input into smaller sets, and how to merge two partially solved tasks into a larger partial solution. Thanks to the dynamic aspect of such architecture, computational complexity can be incorporated as a regularization term that can be optimized by backpropagation. In this talk, we will illustrate the flexibility and efficiency of the Divide-and-Conquer Network on combinatorial and geometric tasks, such as sorting, clustering and convex hulls.
Joan Bruna graduated from Universitat Politecnica de Catalunya (Barcelona, Spain) in both Mathematics and Electrical Engineering. He obtained an M.Sc. in applied mathematics from ENS Cachan (France). He then became a research engineer in an image processing startup, developing real-time video processing algorithms. He obtained his PhD in Applied Mathematics at Ecole Polytechnique (France), under the supervision of Prof. Stephane Mallat. He was a postdoctoral researcher at the Courant Institute, NYU, and a postdoctoral fellow at Facebook AI Research. In 2015, he became Assistant Professor at UC Berkeley, Statistics Department, and starting Fall 2016 he joined the Courant Institute (NYU, New York) as Assistant Professor in Computer Science, Data Science and Mathematics (affiliated). His research interests include invariant signal representations, high-dimensional statistics and stochastic processes, deep learning and its applications to signal processing. He is co-chair of the IPAM Workshop on New Deep Learning Techniques (2018).