ML System Design for Continuous Experimentation
While ML model development is a challenging process, the management of these models becomes even more complex once they're in production. Shifting data distributions, upstream pipeline failures, and model predictions that affect the very datasets they’re trained on can create thorny feedback loops between development and production.
In this talk, Gideon will: • Examine naive ML workflows that don’t take the development-production feedback loop into account and explore why they break down • Showcase system design principles that will help manage these feedback loops more effectively • Share industry case studies where teams have applied these principles to their production ML systems
Gideon Mendels is CEO and founder of Comet, an ML platform provider. He led a team that trained and deployed more than 50 NLP models in 15 languages as founder of GroupWize. He also worked on hate-speech and deception detection at Google, and he trained and put into production deep learning classifiers for 500 languages at Columbia University jointly with IBM Research.