Executive brief: Taking your Machine Learning efforts to the next level with MLOps

Introduction

Soon after organizations start experimenting with Machine Learning, they often realize that solving business problems with Machine Learning is not as straightforward as it seemed. In fact, according to Venturebeat almost 90% of models built by data scientists never make it into production, hence producing little to no business value. MLOps is, a new paradigm that aims at overcoming these challenges and applies DevOps practices – borrowed from software development – to the field of Machine Learning.

In this article we’ll provide an introductory overview of MLOps, its benefits and how your organization can get started with it.

What MLOps is, and what it isn’t

MLOps is not a new tool, software or platform. MLOps is a “Machine Learning engineering culture and practice that aims at unifying ML system development (Dev) and ML system operation (Ops)”1, as depicted in the visual below. It enables data teams to almost entirely automate their Machine Learning workflows. This includes testing all ML system artifacts (data, model and code), efficiently keeping track of past experiments and retraining models with new production data.

MLOps benefits

Implementing MLOps comes with numerous benefits that can be boiled down to three core elements.
First, an MLOps culture makes modern Machine Learning teams more productive. Breaking down information silos brings great benefits to modern ML teams.
Second, the delivery speed of high-quality ML models improves substantially. Indeed, automated testing makes releases safer, storing previous experiments facilitates model reproducibility, and constant monitoring allows for fast interventions when performance drops.
Third, MLOps lets organizations prioritize the optimization of business metrics. Faster model iterations lead inevitably to more competitiveness on the market.

For example, Unit8 supported a global chemical manufacturer in implementing cutting-edge MLOps practices to bring their predictive models to production in a reproducible manner. Scientists are now able to run many more experiments and test them in production, with the same amount of effort as previously, thanks to automated orchestration and experiment tracking. With this new common development and release framework of ML models, the institute was able to already deploy more than 10 advanced analytics models in production.

Step 1 – Identify your needs
Implementing an MLOps culture will differ substantially depending on the size of an organization’s Data Science department, the frequency of model retraining and other compliance & regulatory requirements. We recommend spending enough time and consideration on breaking down your specific needs before moving on.

Step 2 – Implement governance standards
Ultimately, with automation comes risk. Hence, we identify three risk categories that should be clearly governed with safeguards before moving on to a full implementation: business risk (e.g., financial losses due to a model making faulty predictions), legal risk (e.g., GDPR violations) and ethical risk (e.g., model outputs which could be seen as unfair or discriminatory). To implement such safeguards, measures could be for instance the development of a framework which classifies Machine Learning models based on the risk level associated with the aforementioned dimensions. Dedicated teams could then act as a second line of defense to review performance and documentation of risky models, and collaborate with model developers until requirements are satisfactory enough to move to production.

Step 3 – Leverage best practices from industry leaders
The MLOps journey is never really finished, and it usually requires several iterations and failures. To avoid common mistakes, here are some recommendations we give to our customers based on our experience.
First, cross-functional teams with management sponsorship tend to perform much better.
Second, tracking all experiments and their artifacts in a cataloging tool is a must to enhance reproducibility.
Finally, ensuring a standardized approach for model development and deployment that is followed and supported by business and developers alike increases the value delivered by ML projects significantly.

Summary

There is no easy or quick way to implement MLOps, yet successful organizations across industries are moving towards it and are already witnessing the benefits outlined in this article. Implementing successful MLOps practices has the potential to bring your organization’s ML capabilities to the next level. However, to get it right, the necessary expertise needs to be built up in-house or brought in with the support of an experienced implementation partner.