Beyond the Basics with Azure ML: Model Tracking

This is part four in a series on getting beyond the basics with Azure ML.

A Prerequisite: MLflow

Before we dive into this post, I ask you to read a prior post I wrote about MLflow. That post lays out four key products in MLflow and how they all work together to make model management possible.

Azure ML and MLflow

Assuming that you’ve read the linked post, there are probably a couple of questions on your mind: first, can Azure ML use MLflow? And second, have we been using MLflow all along without knowing it? The answers are yes and no, respectively.

Azure ML and MLflow

Azure Machine Learning has built-in support for MLflow. Shivani Patel announced this in October of 2020 and there’s plenty of documentation on how to train an ML project with MLflow + Azure ML. This is still in public preview but it does work. From there, you can deploy MLflow models to Azure Container Instance or Azure Kubernetes Service just like you would Azure ML models.

In order to do this within Azure ML, you will need to know the URL for your MLflow Tracking server, as that’s what Azure ML will use for logging.

Azure ML and Modeling

Assuming that you are not connecting to an MLflow server—and if you have been following along in this series and the prior series on low-code development in Azure ML, you have not connected to an MLflow server—you are not using MLflow. Instead, you are using model registration capabilities whose design purposefully emulates many of the key principles behind MLflow but which are not actually built on the product.

Let’s look at each product in MLflow and understand its Azure ML equivalent. First up, we have MLflow tracking, which allows data scientists to work with experiments. Azure ML has its own concept of experiments and runs which mirrors MLflow quite nicely.

The MLflow Tracking analog in Azure ML Experiments

Azure ML experiments allow us to see details on each run, as we saw in the first post in the series. We can also review output files, parameters passed in, and even the contents of Python scripts that we build to get everything working.

A snapshot of your code. Or at least my code.

Next up, we have MLflow Projects. MLflow Projects allow us to package code for easy and repeatable deployment. There is no direct analog to MLflow Projects; the closest thing we have in Azure ML is the notion of Pipelines. Pipelines are the topic of the next post in the series, so I’ll stay mum on them until then.

Our third item is MLflow Model, which is a standardized format for packaging models. Azure ML doesn’t really offer this capability, although it does support a variety of frameworks.

The final product in the MLflow world is the model registry, which provides the ability for data scientists to register models for deployment. Azure ML has its own model registry which serves the same purpose.

All of the penguin models you could ever want.

Each of these models contains stored artifacts, including a serialized version of the model and any helper files (for example, h5 weights) the model might need.

That model’s liver is toast.

Models can also include information on the data we used to train, including links to the particular dataset and version we used. That way, in the event that a model is not performing up to expectations, we can get a better feeling for the data we used to train it originally.

Sweet, sweet data.

Once we have a model in place, Ops teams can deploy this easily. They can deploy it through the UI:

Release the hounds!

They can also deploy via Powershell and the Azure CLI.

Conclusion

In this post, we looked at the way that Azure ML implements model tracking solutions, comparing them directly to the “industry standard” in MLflow. In the next post, we’ll dive into the idea of ML pipelines and see how Azure ML uses pipelines as the key for code development.

Advertisement