Model metadata and lineage in MLOps - Commands & Configuration

Introduction

When you train machine learning models, you need to keep track of details like parameters, data used, and results. Model metadata and lineage help you record this information so you can understand and reproduce your models later.

When you want to know which data and code produced a specific model version

When you need to compare different model versions to pick the best one

When you want to share model details with your team for collaboration

When you want to audit model training for compliance or debugging

When you want to automate retraining by tracking dependencies

Commands

This command runs the MLflow project in the current directory, starting a training run that logs metadata and lineage automatically.

Terminal

mlflow run .

Expected Output

2024/06/01 12:00:00 INFO mlflow.projects: === Run (ID abc123def456) started ===

2024/06/01 12:00:10 INFO mlflow.projects: === Run (ID abc123def456) succeeded ===

→

--experiment-name - Sets the experiment under which the run is logged

Starts the MLflow tracking UI so you can view model metadata, parameters, metrics, and lineage in a web browser.

Terminal

mlflow ui

Expected Output

2024/06/01 12:01:00 INFO mlflow.server: Starting MLflow UI at http://127.0.0.1:5000

→

--port - Specifies the port for the UI server

Shows detailed metadata and lineage information for the specific run with ID abc123def456.

Terminal

mlflow runs describe abc123def456

Expected Output

Run ID: abc123def456

Parameters:

  learning_rate: 0.01

  epochs: 10

Metrics:

  accuracy: 0.92

Artifacts:

  model.pkl

Tags:

  mlflow.source.name: train.py

  mlflow.source.git.commit: 9f8e7d6

Key Concept

If you remember nothing else from this pattern, remember: tracking model metadata and lineage lets you reproduce, compare, and trust your machine learning models.

Code Example

MLOps

import mlflow
import mlflow.sklearn
from sklearn.ensemble import RandomForestClassifier
from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
from sklearn.metrics import accuracy_score

# Load data
iris = load_iris()
X_train, X_test, y_train, y_test = train_test_split(iris.data, iris.target, test_size=0.2, random_state=42)

# Start MLflow run
with mlflow.start_run():
    # Log parameters
    n_estimators = 100
    mlflow.log_param("n_estimators", n_estimators)

    # Train model
    model = RandomForestClassifier(n_estimators=n_estimators)
    model.fit(X_train, y_train)

    # Predict and log metric
    preds = model.predict(X_test)
    acc = accuracy_score(y_test, preds)
    mlflow.log_metric("accuracy", acc)

    # Log model artifact
    mlflow.sklearn.log_model(model, "model")

    print(f"Run completed with accuracy: {acc:.2f}")

Common Mistakes

Not logging parameters or metrics during training

Without logging, you lose important details needed to understand or reproduce the model

Use MLflow logging functions like mlflow.log_param() and mlflow.log_metric() inside your training code

Not starting the MLflow tracking server or UI

You cannot view or manage your model metadata and lineage without the UI or server running

Run 'mlflow ui' to start the tracking UI and access your runs

Summary

Use MLflow commands to run training and automatically log model metadata and lineage.

Start the MLflow UI to view and compare model runs with their parameters and metrics.

Use MLflow logging functions in your code to record parameters, metrics, and artifacts for reproducibility.

Practice

1. What is the main purpose of model metadata in MLOps?

easy

A. To clean the input data before training

B. To execute the model training automatically

C. To deploy the model to production

D. To store important details about the model's creation and performance

Solution

1. Step 1: Understand what model metadata contains

   Model metadata includes details like training parameters, performance metrics, and environment info.

2. Step 2: Identify the purpose of metadata

   This information helps track how the model was created and how well it performs.

3. Final Answer:

   To store important details about the model's creation and performance -> Option D

4. Quick Check:

   Model metadata = model details storage [OK]

Hint: Metadata stores model info, not execution or deployment [OK]

Common Mistakes:

• Confusing metadata with deployment steps
• Thinking metadata runs the model
• Mixing metadata with data cleaning

2. Which of the following is the correct way to represent model lineage?

easy

A. A graph showing connections between data, code, and model versions

B. A list of model hyperparameters only

C. A single file containing the trained model weights

D. A script that trains the model

Solution

1. Step 1: Define model lineage

   Model lineage tracks the history and relationships between data, code, and model versions.

2. Step 2: Identify correct representation

   A graph or map showing these connections is the correct way to represent lineage.

3. Final Answer:

   A graph showing connections between data, code, and model versions -> Option A

4. Quick Check:

   Lineage = connection graph [OK]

Hint: Lineage means tracking history and connections [OK]

Common Mistakes:

• Thinking lineage is just model parameters
• Confusing lineage with model files
• Assuming lineage is a training script

3. Given the following metadata record:

{"model_version": "v1.2", "accuracy": 0.92, "training_data": "dataset_v3", "code_commit": "abc123"}

What does the code_commit field represent?

medium

A. The version of the training dataset used

B. The unique identifier of the code version used to train the model

C. The accuracy score of the model

D. The deployment environment name

Solution

1. Step 1: Analyze the metadata fields

   The field code_commit usually stores the code version identifier, like a git commit hash.

2. Step 2: Match field meaning to options

   It identifies the exact code used to train the model, ensuring reproducibility.

3. Final Answer:

   The unique identifier of the code version used to train the model -> Option B

4. Quick Check:

   code_commit = code version ID [OK]

Hint: Code commit means code version ID, not data or accuracy [OK]

Common Mistakes:

• Confusing code_commit with dataset version
• Thinking it stores accuracy
• Assuming it is deployment info

4. You notice that the model lineage graph is missing links between data versions and model versions. What is the most likely cause?

medium

A. The training code commit hash is missing

B. The model accuracy was too low

C. The metadata did not record the data version used during training

D. The deployment script failed to run

Solution

1. Step 1: Understand lineage graph links

   Links between data versions and model versions require metadata recording the data version used.

2. Step 2: Identify missing metadata impact

   If data version info is missing, lineage cannot connect data to model versions.

3. Final Answer:

   The metadata did not record the data version used during training -> Option C

4. Quick Check:

   Missing data version metadata breaks lineage links [OK]

Hint: Missing data version metadata breaks lineage connections [OK]

Common Mistakes:

• Blaming model accuracy for lineage issues
• Confusing deployment errors with lineage
• Assuming code commit missing causes data link loss

5. You want to ensure full reproducibility of your ML model training. Which combination of metadata and lineage tracking is best?

hard

A. Record model hyperparameters, training data version, code commit hash, and link them in a lineage graph

B. Only save the final trained model file

C. Track deployment environment and ignore training data versions

D. Store training logs without linking to code or data versions

Solution

1. Step 1: Identify key elements for reproducibility

   Reproducibility requires knowing hyperparameters, data version, and exact code used.

2. Step 2: Understand lineage role

   Linking these elements in a lineage graph shows their relationships and history.

3. Step 3: Evaluate options

   Only Record model hyperparameters, training data version, code commit hash, and link them in a lineage graph includes all necessary metadata and lineage tracking for full reproducibility.

4. Final Answer:

   Record model hyperparameters, training data version, code commit hash, and link them in a lineage graph -> Option A

5. Quick Check:

   Full reproducibility = metadata + lineage graph [OK]

Hint: Combine metadata and lineage graph for full reproducibility [OK]

Common Mistakes:

• Saving only model files without metadata
• Ignoring data version tracking
• Not linking metadata in lineage