Pipeline versioning and reproducibility in MLOps - Commands & Configuration

Introduction

When you build machine learning pipelines, you want to make sure you can run the same steps again and get the same results. Pipeline versioning helps track changes, and reproducibility ensures your work can be repeated exactly.

When you want to share your ML pipeline with a teammate and be sure they get the same results.

When you update your pipeline and want to keep the old version for comparison.

When you need to debug why a model changed by rerunning the exact same pipeline version.

When you deploy a model and want to trace back exactly how it was created.

When you automate training and want to keep track of all pipeline runs and their versions.

Commands

This command creates a new experiment in MLflow to organize pipeline runs and their versions.

Terminal

mlflow experiments create --experiment-name ml-pipeline-versioning

Expected Output

Experiment 'ml-pipeline-versioning' created with ID 1

→

--experiment-name - Sets the name of the experiment to group pipeline runs

Runs the current ML pipeline code and logs the run under the experiment with a specific version name.

Terminal

mlflow run . --experiment-name ml-pipeline-versioning --run-name version_1

Expected Output

2024/06/01 12:00:00 INFO mlflow.projects: === Run (ID 'abc123') succeeded ===

→

--experiment-name - Specifies which experiment to log the run under

→

--run-name - Gives a human-readable name to this pipeline version run

Shows details of the pipeline run including parameters, metrics, and artifacts to verify reproducibility.

Terminal

mlflow runs describe abc123

Expected Output

Run ID: abc123

Status: FINISHED

Parameters: {'param1': 'value1'}

Metrics: {'accuracy': 0.92}

Artifacts: model.pkl

Downloads the model artifact from the specific pipeline run to reproduce or deploy the exact model version.

Terminal

mlflow artifacts download -r abc123 -d ./downloaded_model

Expected Output

Downloaded artifacts to: ./downloaded_model

→

-r - Specifies the run ID to download artifacts from

→

-d - Sets the local directory to save the downloaded artifacts

Key Concept

If you remember nothing else from this pattern, remember: version every pipeline run and log all inputs and outputs to reproduce results exactly.

Code Example

MLOps

import mlflow

mlflow.set_experiment('ml-pipeline-versioning')

with mlflow.start_run(run_name='version_1') as run:
    param1 = 'value1'
    mlflow.log_param('param1', param1)
    accuracy = 0.92
    mlflow.log_metric('accuracy', accuracy)
    with open('model.pkl', 'wb') as f:
        f.write(b'Model binary data')
    mlflow.log_artifact('model.pkl')

print(f"Run ID: {run.info.run_id} logged with accuracy {accuracy}")

Common Mistakes

Not assigning unique run names or experiment names for different pipeline versions.

It becomes hard to track which run corresponds to which pipeline version, causing confusion.

Always use meaningful experiment and run names to clearly identify pipeline versions.

Not logging all parameters and artifacts during the pipeline run.

Without complete logs, you cannot reproduce the exact pipeline results later.

Ensure all inputs, parameters, metrics, and output files are logged in each run.

Overwriting previous runs or artifacts without version control.

You lose history and cannot compare or revert to earlier pipeline versions.

Keep each run separate and download artifacts by run ID to preserve versions.

Summary

Create an MLflow experiment to group pipeline runs by version.

Run the pipeline with unique run names to log parameters, metrics, and artifacts.

Use MLflow commands to inspect runs and download artifacts for exact reproduction.

Practice

1. What is the main purpose of pipeline versioning in MLOps?

easy

A. To increase the size of the dataset used

B. To speed up the training process of machine learning models

C. To track changes in workflows and configurations over time

D. To automatically fix bugs in the code

Solution

1. Step 1: Understand pipeline versioning

   Pipeline versioning means keeping track of changes made to the steps and settings in your workflow.

2. Step 2: Identify the main goal

   This helps teams know what changed and when, making it easier to reproduce or fix issues.

3. Final Answer:

   To track changes in workflows and configurations over time -> Option C

4. Quick Check:

   Pipeline versioning = track changes [OK]

Hint: Versioning means tracking changes over time [OK]

Common Mistakes:

• Confusing versioning with speeding up training
• Thinking versioning fixes bugs automatically
• Believing versioning increases dataset size

2. Which of the following is the correct way to fix a random seed in Python for reproducibility in a pipeline?

easy

A. random.seed(42)

B. random.fix_seed(42)

C. seed.random(42)

D. fix.seed(42)

Solution

1. Step 1: Recall Python random seed syntax

   In Python, the random module uses random.seed(value) to fix the seed.

2. Step 2: Check each option

   Only random.seed(42) matches the correct syntax; others are invalid function calls.

3. Final Answer:

   random.seed(42) -> Option A

4. Quick Check:

   Fix seed in Python = random.seed() [OK]

Hint: Use random.seed(value) to fix seed in Python [OK]

Common Mistakes:

• Using incorrect function names like fix_seed or seed.random
• Confusing method order or syntax
• Missing the random module prefix

3. Given this snippet in a pipeline script:

import random
random.seed(10)
print(random.randint(1, 100))
random.seed(10)
print(random.randint(1, 100))

What will be the output?

medium

A. 67 followed by 67

B. 67 followed by a different number

C. Two different random numbers

D. Error due to repeated seed

Solution

1. Step 1: Understand seed effect on random numbers

   Setting the seed to the same value resets the random number generator to the same state.

2. Step 2: Analyze the code output

   Both calls to random.randint(1, 100) after setting seed(10) will produce the same number, 67.

3. Final Answer:

   67 followed by 67 -> Option A

4. Quick Check:

   Same seed = same random output [OK]

Hint: Same seed resets random sequence, repeat outputs [OK]

Common Mistakes:

• Assuming different outputs after resetting seed
• Thinking repeated seed causes error
• Ignoring seed effect on randomness

4. You run a pipeline but get different results each time, even though you fixed the random seed. What is the most likely cause?

medium

A. The random seed was set correctly

B. The pipeline uses non-deterministic operations or external data changes

C. The pipeline versioning is enabled

D. The code has syntax errors

Solution

1. Step 1: Understand reproducibility factors

   Fixing the random seed controls randomness but does not cover external changes or non-deterministic steps.

2. Step 2: Identify cause of varying results

   If results differ despite fixed seed, likely external data or operations like parallelism cause variation.

3. Final Answer:

   The pipeline uses non-deterministic operations or external data changes -> Option B

4. Quick Check:

   Non-determinism breaks reproducibility [OK]

Hint: Check external data and non-deterministic steps [OK]

Common Mistakes:

• Assuming seed fixes all randomness
• Confusing versioning with reproducibility
• Blaming syntax errors for result changes

5. You want to ensure your ML pipeline is fully reproducible across different machines. Which combination is best to achieve this?

hard

A. Only fix random seeds and ignore environment differences

B. Run pipeline without versioning but log outputs manually

C. Use different random seeds each run and update pipeline versions

D. Fix random seeds, use containerized environments, and version pipeline code

Solution

1. Step 1: Identify reproducibility requirements

   Reproducibility needs fixed seeds, consistent environments, and tracking code changes.

2. Step 2: Evaluate options for best practice

   Fix random seeds, use containerized environments, and version pipeline code combines fixing seeds, containerization for environment consistency, and versioning for tracking changes.

3. Final Answer:

   Fix random seeds, use containerized environments, and version pipeline code -> Option D

4. Quick Check:

   Seeds + containers + versioning = reproducibility [OK]

Hint: Combine seeds, containers, and versioning for full reproducibility [OK]

Common Mistakes:

• Ignoring environment differences
• Changing seeds each run
• Skipping pipeline versioning