Deploying ML in Production
An ML application in production must address all of the issues of modern software development methodology, as well as issues unique to ML and data science. Different types of ML data and modeling problems have different requirements, often driven by the different data lifecycles and sources of ground truth that are unique to each domain. Initial implementations often suffer from inherent limitations in testability, scalability across clusters, training/serving skew, and the modularity and reusability of components. In addition, ML application measurement often emphasizes top level metrics, leading to issues in model fairness as well as predictive performance across user segments. We discuss the use of ML pipeline architectures for implementing production ML applications, key lessons learned with 10+ years of deploying ML in production. In particular we review Google’s experience with TensorFlow Extended (TFX), Google’s primary platform for productionizing ML applications. Google uses TFX for a wide variety of ML applications, and offers an open-source version to the community. TFX enables strong software methodology including modularity, extensibility, scalability, and deep performance analysis.
*Typically less than 5% of the code is trainer or modeling code in production ml, 95% of the code is plumbing code putting the system together (e.g.Data transformation & processing prior to training, model evaluation and validation post training)
*ML Engineering is an upcoming discipline which is a superset of Software Engineering. Treating Data (in addition to code) as a first class citizen is key to ML Engineering.
*Portability and interoperability is an important consideration while building MLOps platform. It can be challenging to support a wide variety of devices and environments, so requires a delicate balance.
Anusha Ramesh is a Machine Learning Product Manager at Google. She works on TensorFlow Extended which is a production scale machine learning platform. Previously, Anusha was a Product Lead at a fashion tech startup that builds personalized recommendations for women's fashion. She has a Master's degree in information networking from Carnegie Mellon. She worked as a Software Engineer in computer networks before venturing into product.