Deep Learning for Computer Graphics: Learning to Estimate Lighting From Photographs
We propose an automatic method to infer high dynamic range illumination from a single, limited field-of-view, low dynamic range photograph of a scene. In contrast to previous work that relies on specialized image capture, user input, and/or simple scene models, we train an end-to-end deep neural network that directly regresses a limited field-of-view photo to HDR illumination, without strong assumptions on scene geometry, material properties, or lighting. This allows us to automatically recover high-quality HDR illumination estimates that significantly outperform previous state-of-the-art methods. Consequently, using our illumination estimates for applications like 3D object insertion, we can achieve compelling, photorealistic results.
Jean-François Lalonde is an Assistant Professor in Electrical and Computer Engineering at Laval University, Quebec City, since 2013. Previously, he was a Post-Doctoral Associate at Disney Research, Pittsburgh. He received a B.Eng. degree in Computer Engineering with honors from Laval University, Canada, in 2004. He earned his M.S at the Robotics Institute at Carnegie Mellon University in 2006 under Prof. Martial Hebert and received his Ph.D., also from Carnegie Mellon, in 2011 under the supervision of Profs. Alexei A. Efros and Srinivasa G. Narasimhan. His Ph.D. thesis won the 2010-11 CMU School of Computer Science Distinguished Dissertation Award. After graduation, he became a Computer Vision Scientist at Tandent, Inc., where he helped develop LightBrush™, the first commercial intrinsic imaging application. He also introduced intrinsic videos at SIGGRAPH 2012 while at Tandent. His research focuses on lighting-aware image understanding and synthesis by leveraging deep learning and large amounts of data.