Logging artifacts and models in MLOps - Step-by-Step Execution

Process Flow - Logging artifacts and models

Start Training Model

↓

Generate Artifacts

↓

Log Artifacts

↓

Save Model

↓

Log Model

↓

End Process

The flow shows starting model training, creating artifacts, logging them, saving the model, logging the model, then ending.

Execution Sample

MLOps

import mlflow
import joblib

with mlflow.start_run():
    mlflow.log_artifact('metrics.json')
    joblib.dump(model, 'model.joblib')
    mlflow.log_artifact('model.joblib')

This code logs a metrics file and a trained model to MLflow during a run.

Process Table

Step	Action	Input	Output	System State Change
1	Start MLflow run	None	Run started	Active run context created
2	Generate artifact	Training metrics	metrics.json file	Artifact file created locally
3	Log artifact	metrics.json	Artifact logged	Artifact uploaded to MLflow storage
4	Save model	Trained model object	Model saved locally	Model file created
5	Log model	Model file	Model logged	Model file uploaded to MLflow storage
6	End MLflow run	None	Run ended	Run context closed

💡 Run ends after logging artifacts and model successfully.

Status Tracker

Variable	Start	After Step 2	After Step 3	After Step 4	After Step 5	Final
run_active	False	True	True	True	True	False
artifact_file	None	metrics.json created	metrics.json logged	metrics.json logged	metrics.json logged	metrics.json logged
model_file	None	None	None	Model saved locally	Model logged	Model logged

Key Moments - 3 Insights

Why do we need to start a run before logging artifacts or models?

Starting a run creates a context to group logs. Without it, MLflow doesn't know where to save artifacts or models. See execution_table step 1 where the run starts before logging.

Is the artifact file automatically created when logging?

No, the artifact file must exist locally before logging. Step 2 shows the artifact file is created first, then step 3 logs it.

What happens if we log a model without saving it first?

The model must be saved locally before logging. Step 4 saves the model, then step 5 logs it. Skipping save means no file to log.

Visual Quiz - 3 Questions

Look at the execution table, at which step is the artifact actually uploaded to MLflow storage?

A Step 2

B Step 4

C Step 3

D Step 5

💡 Hint

Check the 'System State Change' column for artifact upload details.

According to the variable tracker, when does the model_file variable first get a value?

A After Step 3

B After Step 4

C After Step 2

D After Step 5

💡 Hint

Look at the 'model_file' row and see when it changes from None.

If the run_active variable stayed True after Step 6, what would that mean?

A The run is still open

B The run never started

C The artifact was not logged

D The model was not saved

💡 Hint

Check the 'run_active' variable in the variable tracker after Step 6.

Concept Snapshot

Logging artifacts and models:
- Start a run to create context
- Create artifact files locally
- Log artifacts to upload them
- Save model locally
- Log model to upload it
- End run to close context

Full Transcript

This visual execution shows how to log artifacts and models in MLflow. First, a run is started to create a context. Then, artifacts like metrics files are generated locally and logged to MLflow storage. Next, the trained model is saved locally and logged to MLflow. Finally, the run is ended to close the context. Variables like run_active track if the run is open. Artifacts must exist before logging. Models must be saved before logging. This step-by-step process ensures all data is properly stored and tracked.

Practice

1. What is the main purpose of logging artifacts and models in MLOps?

easy

A. To speed up model training

B. To delete old models automatically

C. To create new datasets from artifacts

D. To save files and models for tracking and reuse

Solution

1. Step 1: Understand the role of logging in MLOps

   Logging artifacts and models helps keep track of work and reuse it later.

2. Step 2: Identify the correct purpose

   Saving files and models for tracking and reuse matches the main goal of logging.

3. Final Answer:

   To save files and models for tracking and reuse -> Option D

4. Quick Check:

   Logging = Save and track work [OK]

Hint: Logging means saving work for later use [OK]

Common Mistakes:

• Thinking logging deletes models
• Confusing logging with speeding training
• Assuming logging creates new data

2. Which of the following is the correct syntax to log a file artifact using MLflow?

easy

A. mlflow.log_model('path/to/file.txt')

B. mlflow.log_artifact('path/to/file.txt')

C. mlflow.log_artifacts('path/to/file.txt')

D. mlflow.log('path/to/file.txt')

Solution

1. Step 1: Recall MLflow function for logging files

   The correct function is mlflow.log_artifact() for single files.

2. Step 2: Check syntax correctness

   mlflow.log_artifact('path/to/file.txt') matches the correct syntax.

3. Final Answer:

   mlflow.log_artifact('path/to/file.txt') -> Option B

4. Quick Check:

   Single file logging = log_artifact() [OK]

Hint: Use log_artifact() for single files [OK]

Common Mistakes:

• Using log_model() for files
• Using plural log_artifacts() incorrectly
• Using generic log() function

3. What will be the output of this code snippet?

import mlflow
with mlflow.start_run():
    mlflow.log_artifact('data.csv')
    mlflow.log_model(model, 'model')
print('Run finished')

medium

A. Error because model is not defined

B. Run finished printed; artifacts and model logged in current run

C. No output; code hangs

D. Run finished printed; but nothing logged

Solution

1. Step 1: Analyze the code snippet

   The code tries to log a file and a model inside a run.

2. Step 2: Check for errors

   The variable 'model' is not defined, so mlflow.log_model(model, 'model') causes a NameError.

3. Final Answer:

   Error because model is not defined -> Option A

4. Quick Check:

   Undefined variable causes error [OK]

Hint: Check if variables are defined before logging [OK]

Common Mistakes:

• Assuming code runs without defining model
• Thinking print means success
• Ignoring variable definitions

4. You run this code but no artifacts appear in MLflow UI:

mlflow.log_artifact('output.txt')

What is the most likely reason?

medium

A. log_artifact() only works inside a run

B. The file output.txt does not exist

C. No active MLflow run was started

D. MLflow server is down

Solution

1. Step 1: Understand MLflow run context

   Logging artifacts requires an active run to group logs.

2. Step 2: Identify missing run

   Without mlflow.start_run(), logs are not saved properly.

3. Final Answer:

   No active MLflow run was started -> Option C

4. Quick Check:

   Logging needs active run [OK]

Hint: Always start a run before logging [OK]

Common Mistakes:

• Assuming logging works without a run
• Ignoring file existence
• Blaming server without checking run

5. You want to log multiple files and a trained model in one MLflow run. Which code snippet correctly does this?

hard

A. with mlflow.start_run():
mlflow.log_artifact('file1.txt')
mlflow.log_artifact('file2.txt')
mlflow.log_model(model, 'model')

B. mlflow.log_artifact(['file1.txt', 'file2.txt'])
mlflow.log_model(model, 'model')

C. with mlflow.start_run():
mlflow.log_artifacts(['file1.txt', 'file2.txt'])
mlflow.log_model(model, 'model')

D. mlflow.start_run()
mlflow.log_artifact('file1.txt')
mlflow.log_artifacts('file2.txt')
mlflow.log_model(model, 'model')
mlflow.end_run()

Solution

1. Step 1: Identify correct way to log multiple files

   For multiple individual files, call mlflow.log_artifact() for each file.

2. Step 2: Confirm run context and model logging

   Using with mlflow.start_run(): ensures proper context; logs each file and the model correctly.

3. Final Answer:

   with mlflow.start_run(): mlflow.log_artifact('file1.txt') mlflow.log_artifact('file2.txt') mlflow.log_model(model, 'model') -> Option A

4. Quick Check:

   Multiple files = log_artifact() for each inside run [OK]

Hint: Call log_artifact() for each file inside a run [OK]

Common Mistakes:

• Passing list to log_artifacts()
• Not using a run context
• Using log_artifacts() on single file