Deep Learning at Scale
Deep learning has had a major impact in the last three years. Imperfect interactions with machines, such as speech, natural language, or image processing have been made robust by deep learning and deep learning holds promise in finding usable structure in large datasets. The training process is lengthy and has proven to be difficult to scale due to constraints of existing compute architectures and there is a need of standardized tools for building and scaling deep learning solutions. I will outline some of these challenges and how fundamental changes to the organization of computation and communication can lead to large advances in capabilities.
Naveen’s fascination with computation in synthetic and neural systems began around age 9 when he began learning about circuits that store information along with some AI themes prevalent in sci-fi at the time. He went on to study electrical engineering and computer science at Duke, but continued to stay in touch with biology by modeling neuromorphic circuits as a senior project. After studying computer architecture at Stanford, Naveen spent the next 10 years designing novel processors at Sun Microsystems and Teragen as well as specialized chips for wireless DSP at Caly Networks, video content delivery at Kealia, Inc, and video compression at W&W Comms. Armed with intimate knowledge of synthetic computation systems, Naveen decided to get a PhD in Neuroscience to understand how biological systems do computation better. He studied neural computation and how it relates to neural prosthetics in the lab of John Donoghue at Brown. After a stint in finance doing algorithmic trading optimization at ITG, Naveen most recently was part of the Qualcomm’s neuromorphic research group leading the effort on motor control and doing business development. It’s in Nervana’s DNA to bring together engineering disciplines and neural computational paradigms to evolve the state-of-the-art and make machines smarter.