Experiment tracking (MLflow) in ML Python - Practice Problems & Coding Challenges

Challenge - 5 Problems

🧠 Conceptual

intermediate

What is the primary purpose of MLflow's experiment tracking?

MLflow helps data scientists keep track of their machine learning experiments. What is the main goal of using MLflow's experiment tracking feature?

A To automatically tune hyperparameters without user input

B To store and compare parameters, metrics, and models from different runs

C To deploy models directly to production without testing

D To visualize data distributions before training

💡 Hint

Think about what you want to remember and compare after running many models.

MLflow experiment tracking is designed to log and organize parameters, metrics, and models so you can compare different runs easily.

❓ Predict Output

intermediate

What output does this MLflow code produce?

Consider this Python code snippet using MLflow to log a parameter and a metric. What will be printed?

ML Python

import mlflow
with mlflow.start_run():
    mlflow.log_param('alpha', 0.5)
    mlflow.log_metric('rmse', 1.23)
    print(mlflow.active_run().info.run_id)

A The value 1.23

B The value 0.5

C A unique run ID string like '1234abcd...'

D None

💡 Hint

mlflow.active_run().info.run_id returns a string identifier for the current run.

The code prints the unique run ID string assigned by MLflow to the current experiment run.

❓ Model Choice

advanced

Which MLflow model flavor is best for saving a scikit-learn model?

You trained a scikit-learn model and want to save it with MLflow so you can load it later easily. Which MLflow model flavor should you use?

A mlflow.spark

B mlflow.tensorflow

C mlflow.pytorch

D mlflow.sklearn

💡 Hint

Match the model flavor to the library you used for training.

mlflow.sklearn is designed specifically for saving and loading scikit-learn models.

❓ Hyperparameter

advanced

Which hyperparameter logging method correctly records multiple hyperparameters in MLflow?

You want to log multiple hyperparameters at once in MLflow. Which code snippet correctly does this?

A mlflow.log_params({'learning_rate': 0.01, 'batch_size': 32})

B mlflow.log_params(['learning_rate=0.01', 'batch_size=32'])

C mlflow.log_param(['learning_rate', 0.01, 'batch_size', 32])

D mlflow.log_param({'learning_rate': 0.01, 'batch_size': 32})

💡 Hint

Check the method name and argument type for logging multiple parameters.

mlflow.log_params accepts a dictionary of parameter names and values to log multiple parameters at once.

❓ Metrics

expert

What is the value of 'accuracy' after running this MLflow code?

Given this code snippet, what will be the final logged value of the metric 'accuracy'?

ML Python

import mlflow
with mlflow.start_run():
    mlflow.log_metric('accuracy', 0.8)
    mlflow.log_metric('accuracy', 0.85)
    mlflow.log_metric('accuracy', 0.83, step=1)
    mlflow.log_metric('accuracy', 0.9, step=2)
    run = mlflow.active_run()
    metrics = mlflow.get_run(run.info.run_id).data.metrics
    print(metrics['accuracy'])

A 0.85

B 0.9

C 0.83

D 0.8

💡 Hint

Without step argument, the last logged metric overwrites previous ones at step 0.

Metrics logged without a step overwrite previous values at step 0. The last value without step is 0.85, so metrics['accuracy'] returns 0.85.

Practice

1. What is the main purpose of using MLflow in machine learning projects?

easy

A. To write machine learning algorithms from scratch

B. To visualize data without any coding

C. To track and manage machine learning experiments

D. To deploy machine learning models to mobile apps

Solution

1. Step 1: Understand MLflow's role

   MLflow is designed to help track experiments by logging parameters, metrics, and models.

2. Step 2: Compare options

   Options A, C, and D describe tasks outside MLflow's main purpose.

3. Final Answer:

   To track and manage machine learning experiments -> Option C

4. Quick Check:

   MLflow = experiment tracking [OK]

Hint: MLflow is for tracking experiments, not coding models [OK]

Common Mistakes:

• Confusing MLflow with model building libraries
• Thinking MLflow is only for visualization
• Assuming MLflow deploys models directly

2. Which MLflow function is used to start a new experiment run in Python?

easy

A. mlflow.init_experiment()

B. mlflow.start_run()

C. mlflow.create_run()

D. mlflow.begin_experiment()

Solution

1. Step 1: Recall MLflow run start syntax

   The correct function to start a run is mlflow.start_run().

2. Step 2: Check other options

   Functions like init_experiment, create_run, and begin_experiment do not exist in MLflow API.

3. Final Answer:

   mlflow.start_run() -> Option B

4. Quick Check:

   Start run = mlflow.start_run() [OK]

Hint: Use start_run() to begin tracking an experiment run [OK]

Common Mistakes:

• Using non-existent MLflow functions
• Confusing experiment and run functions
• Misspelling function names

3. What will be the output of this MLflow code snippet?

import mlflow
with mlflow.start_run():
    mlflow.log_param('alpha', 0.1)
    mlflow.log_metric('accuracy', 0.95)
    run_info = mlflow.active_run().info
print(run_info.run_id is not None)

medium

A. True

B. False

C. None

D. Error

Solution

1. Step 1: Understand mlflow.start_run() context

   The code starts a run, logs parameters and metrics, then gets the active run info.

2. Step 2: Check run_id existence

   Inside the run context, run_id is assigned and not None, so print(run_info.run_id is not None) outputs True.

3. Final Answer:

   True -> Option A

4. Quick Check:

   Active run has run_id = True [OK]

Hint: Active run always has a run_id during context [OK]

Common Mistakes:

• Assuming run_id is None before logging
• Confusing run info with experiment info
• Expecting print to show run_id string

4. Identify the error in this MLflow code snippet:

import mlflow
mlflow.log_param('learning_rate', 0.01)
with mlflow.start_run():
    mlflow.log_metric('loss', 0.2)

medium

A. log_param called outside a run context

B. start_run() missing parentheses

C. log_metric called before log_param

D. No error, code runs fine

Solution

1. Step 1: Check order of MLflow calls

   log_param is called before starting a run, which is not allowed.

2. Step 2: Validate run context requirement

   MLflow requires logging functions inside a run context started by start_run().

3. Final Answer:

   log_param called outside a run context -> Option A

4. Quick Check:

   Logging outside run = error [OK]

Hint: Always log params and metrics inside start_run() [OK]

Common Mistakes:

• Calling log_param before start_run()
• Ignoring run context for logging
• Assuming log_metric order matters

5. You want to compare two ML models' performance using MLflow. Which steps correctly describe this process?

hard

A. Only log parameters for both models, metrics are not needed for comparison

B. Log both models' metrics in the same run, then compare metrics manually

C. Use mlflow.log_artifact() to save models, no need to log metrics or parameters

D. Run each model in separate mlflow.start_run() blocks, log metrics, then use MLflow UI to compare

Solution

1. Step 1: Understand MLflow run separation

   Each model should be run and logged in its own mlflow.start_run() block to keep results separate.

2. Step 2: Use MLflow UI for comparison

   After logging metrics and parameters, MLflow UI allows easy comparison of runs.

3. Final Answer:

   Run each model in separate mlflow.start_run() blocks, log metrics, then use MLflow UI to compare -> Option D

4. Quick Check:

   Separate runs + UI = compare models [OK]

Hint: Use separate runs and MLflow UI to compare models easily [OK]

Common Mistakes:

• Logging multiple models in one run
• Skipping metric logging
• Ignoring MLflow UI for comparison