Logging parameters and metrics in MLOps - Practice Problems & Coding Challenges

Challenge - 5 Problems

💻 Command Output

intermediate

What is the output of this MLflow logging command?

Consider the following Python code snippet using MLflow to log parameters and metrics. What will be printed after running this code?

MLOps

import mlflow
with mlflow.start_run():
    mlflow.log_param("learning_rate", 0.01)
    mlflow.log_metric("accuracy", 0.95)
    print(mlflow.active_run().info.run_id)

A A valid UUID string representing the run ID

B None

C SyntaxError

D An empty string

💡 Hint

mlflow.start_run() creates a new run and mlflow.active_run() returns the current run info.

The mlflow.start_run() context manager starts a new run. mlflow.active_run() returns the current run object, and its info.run_id is a unique identifier string (UUID) for the run.

🧠 Conceptual

intermediate

Which MLflow function logs a hyperparameter for a model training run?

You want to record the value of a hyperparameter named 'batch_size' with value 32 during your ML experiment. Which MLflow function should you use?

A mlflow.log_param('batch_size', 32)

B mlflow.log_metric('batch_size', 32)

C mlflow.log_artifact('batch_size', 32)

D mlflow.set_tag('batch_size', 32)

💡 Hint

Parameters are fixed values describing the run, metrics are numeric results that can change over time.

mlflow.log_param() is used to log parameters like hyperparameters. mlflow.log_metric() is for numeric results like accuracy. mlflow.log_artifact() is for files. mlflow.set_tag() adds metadata tags.

❓ Troubleshoot

advanced

Why does this MLflow metric logging code fail to record the metric?

You run this code but the metric 'loss' does not appear in the MLflow UI after the run completes. What is the most likely reason? import mlflow mlflow.log_metric('loss', 0.25)

A The metric name 'loss' is invalid and causes an error

B The metric value 0.25 is not a valid float

C MLflow requires metrics to be logged only after the run ends

D No active MLflow run was started before logging the metric

💡 Hint

MLflow needs a run context to associate logged data with.

MLflow requires an active run context (started with mlflow.start_run()) to log parameters or metrics. Without it, the log_metric call does nothing.

🔀 Workflow

advanced

What is the correct sequence to log parameters and metrics in MLflow during a training run?

Arrange the steps in the correct order to log parameters and metrics properly in MLflow.

A 1,3,2,4

B 2,1,3,4

C 1,2,3,4

D 3,2,1,4

💡 Hint

You must start the run before logging anything.

You first start the run, then log parameters before training, then train and compute metrics, then log metrics.

✅ Best Practice

expert

Which practice ensures reliable metric logging in distributed training with MLflow?

In a distributed training setup with multiple workers, which approach best ensures metrics are logged correctly without duplication or loss?

A All workers log metrics independently during training

B Only the main worker logs metrics after aggregating results from all workers

C Metrics are logged only before training starts

D Each worker logs metrics with the same run ID simultaneously

💡 Hint

Avoid multiple workers writing to the same run at the same time.

In distributed training, only the main worker should log aggregated metrics to avoid duplication and conflicts. Other workers should not log metrics independently.

Practice

1.

What is the main purpose of logging parameters in machine learning experiments?

easy

A. To record the settings used during model training

B. To measure the model's accuracy on test data

C. To save the final trained model file

D. To visualize the model's predictions

Solution

1. Step 1: Understand what parameters are

   Parameters are the settings or configurations used to train a model, like learning rate or number of layers.

2. Step 2: Identify the purpose of logging parameters

   Logging parameters helps keep track of these settings so you can compare different training runs.

3. Final Answer:

   To record the settings used during model training -> Option A

4. Quick Check:

   Logging parameters = record training settings [OK]

Hint: Parameters = training settings, metrics = performance [OK]

Common Mistakes:

• Confusing parameters with metrics
• Thinking logging saves the model file
• Assuming logging is for visualization

2.

Which of the following is the correct way to log a metric named accuracy with value 0.95 using a typical MLOps logging function log_metric?

easy

A. log_metric('accuracy', 0.95)

B. log_metric(accuracy=0.95)

C. log_metric('accuracy': 0.95)

D. log_metric(0.95, 'accuracy')

Solution

1. Step 1: Understand typical function syntax

   Logging functions usually take the metric name as a string first, then the value as a number.

2. Step 2: Check each option's syntax

   log_metric('accuracy', 0.95) uses correct syntax: function name, string key, numeric value. log_metric(accuracy=0.95) uses keyword argument which may not be supported. log_metric('accuracy': 0.95) uses invalid syntax with colon inside parentheses. log_metric(0.95, 'accuracy') reverses arguments incorrectly.

3. Final Answer:

   log_metric('accuracy', 0.95) -> Option A

4. Quick Check:

   Function(metric_name, value) = correct syntax [OK]

Hint: Metric name first as string, then value [OK]

Common Mistakes:

• Using colon instead of comma in function call
• Passing arguments in wrong order
• Using keyword arguments when not supported

3.

Given the following code snippet, what will be the output logged for the metric loss?

log_metric('loss', 0.25)
log_metric('loss', 0.20)
log_metric('loss', 0.15)

medium

A. Only the last value 0.15 is logged for 'loss'

B. An error occurs because 'loss' is logged multiple times

C. All three values 0.25, 0.20, and 0.15 are logged separately

D. The first value 0.25 overwrites the others

Solution

1. Step 1: Understand metric logging behavior

   Most MLOps tools allow logging multiple values for the same metric over time to track progress.

2. Step 2: Analyze the code snippet

   The code logs 'loss' three times with different values. Each call records a new metric value, not overwriting previous ones.

3. Final Answer:

   All three values 0.25, 0.20, and 0.15 are logged separately -> Option C

4. Quick Check:

   Multiple logs for same metric = multiple entries [OK]

Hint: Repeated metric logs add entries, not overwrite [OK]

Common Mistakes:

• Assuming repeated logs overwrite previous values
• Expecting an error on duplicate metric names
• Thinking only one value per metric is allowed

4.

Identify the error in this code snippet for logging a parameter batch_size with value 32:

log_param(batch_size, '32')

medium

A. Function name should be log_metric instead of log_param

B. Value should be a number, not a string

C. No error, the code is correct

D. Parameter name should be a string, not a variable

Solution

1. Step 1: Check parameter name argument

   The parameter name must be a string literal like 'batch_size', not a bare variable name.

2. Step 2: Check value argument

   Value can be string or number depending on context; '32' as string is acceptable here.

3. Final Answer:

   Parameter name should be a string, not a variable -> Option D

4. Quick Check:

   Parameter name = string literal [OK]

Hint: Parameter names must be quoted strings [OK]

Common Mistakes:

• Passing parameter name without quotes
• Confusing log_param with log_metric
• Thinking value must always be numeric

5.

You want to log both parameters and metrics for a training run using the following code:

log_param('learning_rate', 0.01)
log_param('optimizer', 'adam')
log_metric('accuracy', 0.92)
log_metric('loss', 0.1)

Which of these statements is true about the logged data?

hard

A. Metrics record model settings; parameters record model performance

B. Parameters record model settings; metrics record model performance

C. Both parameters and metrics record model performance

D. Both parameters and metrics record model settings

Solution

1. Step 1: Understand the role of parameters

   Parameters like learning rate and optimizer are settings used to train the model.

2. Step 2: Understand the role of metrics

   Metrics like accuracy and loss measure how well the model performs after training.

3. Final Answer:

   Parameters record model settings; metrics record model performance -> Option B

4. Quick Check:

   Parameters = settings, Metrics = performance [OK]

Hint: Parameters = settings, metrics = results [OK]

Common Mistakes:

• Mixing up parameters and metrics roles
• Thinking metrics are settings
• Assuming parameters measure performance