Learned Video Compression
We present an algorithm for video coding, learned end-to-end for the low-latency mode. In this setting, our approach outperforms all standard video codecs across nearly the entire bitrate range. To our knowledge, this is the first ML-based method to do so. We propose a novel architecture for video compression which generalizes motion estimation to perform any learned compensation beyond simpler translations. Our architecture allows for joint compression of motion and residual and can dynamically trade-off between them. It is also able to model multiple flow fields in the same frame. We propose an ML-based spatial rate control, which allows or model to adaptively change the bitrate across space for each frame. For the same quality traditional codecs achieve up to 60% larger code.
Lubomir Bourdev is a co-founder and the CEO of WaveOne, Inc., a startup focusing on video compression with deep learning. He is also a founding member of Facebook AI Research and he founded and led the Facebook AML Computer Vision team responsible for the image and video content recognition engine at Facebook. Prior to that he was a Sr. Research Scientist at Adobe Research where he led development of computer vision and features in Adobe's Creative Suite products. He holds a Ph.D. in Computer Science from U.C. Berkeley and M.Sc. and B.A. in Computer Science from Brown University.