Experiment tracking (MLflow) in ML Python

Introduction

Experiment tracking helps you keep a clear record of your machine learning tests. MLflow makes it easy to save and compare your model results.

You want to remember which model settings gave the best results.

You need to compare different model versions to pick the best one.

You want to share your experiment results with teammates.

You want to keep track of metrics like accuracy or loss during training.

You want to save models and their details for future use.

Syntax

ML Python

import mlflow

with mlflow.start_run():
    mlflow.log_param("param_name", param_value)
    mlflow.log_metric("metric_name", metric_value)
    mlflow.log_artifact("path_to_file")
    mlflow.sklearn.log_model(model, "model")

Use mlflow.start_run() to begin tracking an experiment.

Log parameters, metrics, artifacts, and models inside the run block.

Examples

This logs a learning rate parameter and accuracy metric for one run.

ML Python

import mlflow

with mlflow.start_run():
    mlflow.log_param("learning_rate", 0.01)
    mlflow.log_metric("accuracy", 0.85)

This logs a parameter, a metric, and saves a model file as an artifact.

ML Python

import mlflow

with mlflow.start_run():
    mlflow.log_param("max_depth", 5)
    mlflow.log_metric("rmse", 3.2)
    mlflow.log_artifact("model.pkl")

Sample Model

This program trains a simple Random Forest on iris data, logs parameters, accuracy, and the model using MLflow.

ML Python

import mlflow
import mlflow.sklearn
from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
from sklearn.ensemble import RandomForestClassifier
from sklearn.metrics import accuracy_score

# Load data
iris = load_iris()
X_train, X_test, y_train, y_test = train_test_split(iris.data, iris.target, test_size=0.3, random_state=42)

# Start MLflow experiment
with mlflow.start_run():
    # Set and log parameter
    n_estimators = 10
    mlflow.log_param("n_estimators", n_estimators)

    # Train model
    model = RandomForestClassifier(n_estimators=n_estimators, random_state=42)
    model.fit(X_train, y_train)

    # Predict and calculate accuracy
    preds = model.predict(X_test)
    acc = accuracy_score(y_test, preds)

    # Log metric
    mlflow.log_metric("accuracy", acc)

    # Log model
    mlflow.sklearn.log_model(model, "random_forest_model")

    print(f"Logged run with accuracy: {acc:.4f}")

Important Notes

MLflow saves experiments locally by default in an 'mlruns' folder.

You can view logged experiments using the MLflow UI by running mlflow ui in your terminal.

Always start a run with mlflow.start_run() to group logs together.

Summary

MLflow helps you track your machine learning experiments easily.

You can log parameters, metrics, artifacts, and models to compare results later.

Use the MLflow UI to visualize and manage your experiments.