MLflow

What do you like best?

1) A single format to support all measure ML libraries such as Sklearn, Tensorflow, MXnet, Spark MLlib, Pyspark etc.

2) Capabilities to deploy on Amazon Sagemaker with just one API call

3) Flexibility to log all model params such as Accuracy, Recall, etc. along with Hyperparameter tuning support.

4) A good GUI to compare and select the best models.

5) Model registry to track Staging, Production, and Archived models.

6) Python best API

7) REST APIs supported.

8) Available out of the box in Microsoft Azure.

What do you dislike?

1) CI/CD pipeline is not supported in the open-source version

2) Recent framework so not a very large community

3) Dependent on many python libraries. It can be a problem while resolving dependencies in your existing setup.

Recommendations to others considering the product:

It cant be a complete solution for the data science/ML engineering flow. But is essential in the pipeline. It may be used with Apache Airflow to have an end to end ML ops solution. Also, it works best with Amazon sagemaker and Microsoft Azure. However, GCP AI platform support is still in the development phase.

You would also need to take care of CI/CD pipeline for ML models on your own.

What problems are you solving with the product? What benefits have you realized?

I have used it for managing the ML lifecycle, including experimentation, reproducibility, deployment, and a central model registry.

The same thing can be done in Amazon sagemaker, GCP AI Platform, Microsoft Azure etc. but it would require monthly expenses. It can be good for initial startup data science team.