Why reproducibility builds trust in ML in MLOps - Challenge Your Understanding

Challenge - 5 Problems

🧠 Conceptual

intermediate

Why is reproducibility important in ML projects?

Imagine you share your ML model with a teammate. Why must they get the same results when running your code?

A It allows the model to use less memory during training.

B It makes the code run faster on different machines.

C It guarantees the model will always improve accuracy over time.

D It ensures the model's results can be trusted and verified by others.

💡 Hint

Think about why consistent results matter when sharing work.

Reproducibility means others can run the same steps and get the same results. This builds trust because it shows the model works as claimed.

💻 Command Output

intermediate

What is the output of this ML experiment logging command?

You run this command to log your ML experiment with fixed random seed. What output confirms reproducibility?

MLOps

mlflow run . --experiment-name reproducible_test --env-manager=local
# Assume the code sets seed=42 and logs metrics

A Experiment run completed with metrics matching previous runs.

B Experiment failed due to missing data files.

C Error: random seed not set, results vary each run.

D Metrics show large differences from last run.

💡 Hint

Look for output indicating consistent metrics.

When reproducibility is ensured, experiment runs produce the same metrics, confirming trust in results.

❓ Configuration

advanced

Which Dockerfile snippet ensures reproducible ML environment?

Choose the Dockerfile snippet that best guarantees the same environment for ML training every time.

A

FROM python:3.9
RUN pip install numpy pandas scikit-learn

B

FROM python:3.9
RUN pip install numpy==1.21.0 pandas==1.3.0 scikit-learn==0.24.2

C

FROM python:latest
RUN pip install numpy pandas scikit-learn

D

FROM python:3.9
RUN pip install numpy==latest pandas==latest scikit-learn==latest

💡 Hint

Fixing package versions helps reproducibility.

Pinning exact package versions ensures the environment does not change, making results reproducible.

🔀 Workflow

advanced

What is the correct order to ensure reproducible ML training?

Arrange these steps in the right order to guarantee reproducible ML training.

A 2,1,4,3

B 2,3,1,4

C 1,2,3,4

D 3,2,1,4

💡 Hint

Think about setting seeds before training and fixing parameters early.

First set seeds, then prepare data, fix hyperparameters, and finally train and log results to ensure reproducibility.

❓ Troubleshoot

expert

Why does this ML pipeline produce different results despite fixed seeds?

You fixed random seeds in your code, but results differ each run. What is the most likely cause?

MLOps

import random
import numpy as np
random.seed(42)
np.random.seed(42)
# Training code here

A The training uses GPU operations that are non-deterministic by default.

B Random seeds must be set after training starts, not before.

C The dataset is too small to produce stable results.

D The code is missing a call to random.seed() for the OS environment.

💡 Hint

Consider hardware effects on reproducibility.

GPU computations can introduce non-determinism unless explicitly controlled, causing different results despite fixed seeds.