An artificial intelligence (AI) system is different from a traditional software system, in that we not only need to write production level code but also need to ensure and sustain data quality. There is often a lot of emphasis on novel algorithm design, however, we believe that more emphasis should be placed on the quality of data and the reliability of the AI system. An algorithm is only a small part of an AI system.
At Marshmallow, we have designed and built a machine learning feature store in-house, which serves as the central source for common features, allowing for features to be used in both offline training and online serving. Utilising AWS SageMaker, we built a customised model training and deployment process. This has enabled our data scientists to own the whole machine learning life cycle and to stop relying on engineers for deployment.
In this talk, we will present this new Feature Store and SageMaker workflow and show how it enables us to build reliable and scalable AI systems.
Weijiang is a Machine Learning Engineer at Marshmallow Insurance, working on Developing Scalable AI Models. Particularly in terms of Mitigating Risk, Employing Speed and giving more autonomy to the Data Scientists.