Logging artifacts and models in MLOps - Deep Dive

Overview - Logging artifacts and models

What is it?

Logging artifacts and models means saving important files and data created during machine learning work. Artifacts can be anything like data files, plots, or reports. Models are the trained programs that make predictions. Logging helps keep track of these so you can find, share, and reuse them later.

Why it matters

Without logging artifacts and models, it is hard to know which version of a model worked best or what data was used. This can cause confusion, wasted time, and mistakes. Logging makes machine learning work reliable, repeatable, and easier to improve. It helps teams collaborate and build better systems faster.

Where it fits

Before learning this, you should understand basic machine learning workflows and version control. After this, you can learn about model deployment, monitoring, and automated pipelines. Logging artifacts and models is a key step in managing machine learning projects professionally.

Mental Model

Core Idea

Logging artifacts and models is like keeping a detailed diary of every important step and result in your machine learning journey so you can always go back, check, and build on it.

Think of it like...

Imagine baking a cake and writing down the exact recipe, ingredients, oven temperature, and baking time every time you try. This diary helps you bake the perfect cake again or share the recipe with friends.

┌───────────────┐      ┌───────────────┐      ┌───────────────┐
│  Raw Data     │─────▶│  Training     │─────▶│  Model File   │
│  Artifacts    │      │  Process      │      │  (Logged)     │
└───────────────┘      └───────────────┘      └───────────────┘
       │                      │                      │
       ▼                      ▼                      ▼
┌───────────────┐      ┌───────────────┐      ┌───────────────┐
│  Plots,       │      │  Metrics      │      │  Model Info   │
│  Reports      │      │  (Logged)     │      │  (Logged)     │
│  (Artifacts)  │      └───────────────┘      └───────────────┘
└───────────────┘

Build-Up - 7 Steps

1

Foundation

What are Artifacts and Models

Concept: Introduce what artifacts and models mean in machine learning.

Artifacts are files created during ML work like data snapshots, charts, or logs. Models are trained programs saved to make predictions later. Both are important outputs of ML experiments.

Result

Learner understands the difference between artifacts and models and why both matter.

Knowing what to save is the first step to managing ML work effectively.

2

Foundation

Why Logging is Essential

Concept: Explain the need to log artifacts and models systematically.

Without logging, files get lost or mixed up. Logging means saving files with clear names, versions, and metadata so you can find and compare them later.

Result

Learner sees logging as a way to organize and track ML outputs.

Understanding logging prevents confusion and wasted effort in ML projects.

3

Intermediate

Common Tools for Logging

🤔Before reading on: do you think logging is done manually or with special tools? Commit to your answer.

Concept: Introduce popular tools that help automate logging of artifacts and models.

Tools like MLflow, Weights & Biases, and TensorBoard help save and organize artifacts and models automatically. They provide dashboards and APIs to track experiments.

Result

Learner knows which tools to explore for logging in ML workflows.

Knowing tools lets you automate logging and focus on building models, not managing files.

4

Intermediate

How to Log Artifacts Properly

🤔Before reading on: do you think logging artifacts means saving everything or only key files? Commit to your answer.

Concept: Teach best practices for choosing and saving artifacts.

Log only important files like final datasets, evaluation plots, and configuration files. Use meaningful names and versions. Store metadata like creation time and experiment ID.

Result

Learner can log artifacts in a way that is useful and not overwhelming.

Knowing what and how to log saves storage and makes retrieval easier.

5

Intermediate

Versioning Models During Logging

🤔Before reading on: do you think models should be overwritten or versioned when logged? Commit to your answer.

Concept: Explain why and how to version models when logging.

Each model version corresponds to a training run. Use unique IDs or timestamps to save models separately. This helps compare performance and roll back if needed.

Result

Learner understands model versioning as a key part of logging.

Versioning models prevents accidental loss and supports experiment comparison.

6

Advanced

Integrating Logging into Pipelines

🤔Before reading on: do you think logging is a separate step or integrated into ML pipelines? Commit to your answer.

Concept: Show how to embed logging into automated ML workflows.

Use pipeline tools like Airflow or Kubeflow to run training and log artifacts/models automatically. This ensures consistency and reduces manual errors.

Result

Learner can design pipelines that log outputs without extra effort.

Integrating logging into pipelines makes ML projects scalable and reliable.

7

Expert

Challenges and Surprises in Logging

🤔Before reading on: do you think logging large models and artifacts is always straightforward? Commit to your answer.

Concept: Discuss difficulties like storage limits, privacy, and reproducibility in logging.

Large models need efficient storage like cloud buckets or artifact stores. Sensitive data requires encryption or anonymization. Reproducibility demands logging environment and dependencies too.

Result

Learner appreciates the complexity and tradeoffs in real-world logging.

Understanding these challenges prepares you to build robust ML logging systems.

Under the Hood

Logging systems track files and metadata by assigning unique IDs and storing them in databases or storage backends. When a model or artifact is logged, the system copies the file, records its metadata (like creation time, experiment ID, parameters), and links it to the experiment record. This allows querying and retrieval later. Some systems also track lineage, showing how artifacts relate to each other and to code versions.

Why designed this way?

Logging was designed to solve the problem of lost or untraceable ML outputs. Early ML projects had scattered files and manual notes, causing confusion. Centralized logging with metadata and versioning was chosen to enable reproducibility, collaboration, and auditability. Alternatives like manual file saving were error-prone and not scalable.

┌───────────────┐      ┌───────────────┐      ┌───────────────┐
│  ML Experiment│─────▶│ Logging System│─────▶│ Storage Backend│
│  Run Code     │      │  (API/CLI)   │      │ (Database,     │
└───────────────┘      └───────────────┘      │  File System)  │
                                               └───────────────┘
       ▲                      ▲                      ▲
       │                      │                      │
┌───────────────┐      ┌───────────────┐      ┌───────────────┐
│ Model Files   │      │ Metadata      │      │ Artifact Files│
│ (Versioned)  │      │ (Params, Time)│      │ (Plots, Data) │
└───────────────┘      └───────────────┘      └───────────────┘

Myth Busters - 4 Common Misconceptions

Quick: Do you think logging artifacts means saving every single file generated? Commit yes or no.

Common Belief: Logging means saving all files generated during training to be safe.

Reality: Only important and relevant artifacts should be logged to avoid clutter and storage waste.

Why it matters: Saving everything leads to storage overload and makes it hard to find useful files.

Quick: Do you think models can be logged by just saving the code that trains them? Commit yes or no.

Common Belief: Saving the training code is enough to reproduce and log models.

Reality: The trained model files themselves must be saved because training is stochastic and code alone doesn't reproduce exact models.

Why it matters: Without saving model files, you cannot reuse or compare exact model versions.

Quick: Do you think logging models once is enough for a project? Commit yes or no.

Common Belief: Logging a single final model is sufficient for ML projects.

Reality: Multiple model versions should be logged to track improvements and rollback if needed.

Why it matters: Not versioning models causes loss of experiment history and hinders debugging.

Quick: Do you think logging artifacts and models is only useful for big teams? Commit yes or no.

Common Belief: Logging is only necessary when many people work on the same ML project.

Reality: Logging benefits solo practitioners by organizing work and enabling reproducibility too.

Why it matters: Ignoring logging leads to lost work and confusion even for individuals.

Expert Zone

1

Some logging systems support artifact lineage, showing how each artifact was created from others, which helps trace errors deeply.

2

Efficient storage of large models often uses deduplication and compression techniques to save space without losing data.

3

Logging environment details like library versions and hardware specs is crucial for true reproducibility but often overlooked.

When NOT to use

Logging is not suitable for ephemeral or exploratory experiments where speed matters more than tracking. In such cases, lightweight local saves or notebooks may be better. Also, for extremely large datasets, specialized data versioning tools like DVC are preferred over generic artifact logging.

Production Patterns

In production, logging is integrated into CI/CD pipelines to automatically save models and artifacts after training. Teams use centralized artifact stores with access controls. Metadata is enriched with performance metrics and deployment info. Monitoring tools link back to logged models for traceability.

Connections

Version Control Systems

Builds-on

Understanding how Git tracks code versions helps grasp why model versioning is essential for tracking ML experiments.

Data Lineage in Data Engineering

Same pattern

Both logging artifacts and data lineage track origins and transformations to ensure trust and reproducibility.

Scientific Lab Notebooks

Builds-on

Logging ML artifacts and models is a digital form of keeping detailed lab notes to reproduce experiments and share findings.

Common Pitfalls

#1 Saving models without versioning overwrites previous results.

Wrong approach:

model.save('model.pkl') # overwrites every time

Correct approach:

model.save(f'model_{timestamp}.pkl') # unique versioned filename

Root cause:

Not understanding the importance of unique identifiers for each model version.

#2 Logging too many irrelevant files bloats storage and slows retrieval.

Wrong approach:

logging.log_artifact('temp/debug.log')
logging.log_artifact('intermediate_data.csv')

Correct approach:

logging.log_artifact('final_evaluation_plot.png')
logging.log_artifact('cleaned_dataset.csv')

Root cause:

Confusing all generated files as equally important artifacts.

#3 Not logging metadata like parameters or environment causes reproducibility issues.

Wrong approach:

logging.log_model(model)
# no metadata saved

Correct approach:

logging.log_model(model, metadata={'params': params, 'env': env_info})

Root cause:

Underestimating the role of metadata in reproducing results.

Key Takeaways

Logging artifacts and models means saving important files and trained programs with clear versions and metadata.

Proper logging makes machine learning work reproducible, organized, and easier to share or improve.

Use specialized tools to automate logging and avoid manual errors or lost files.

Versioning models and selecting key artifacts to log prevents confusion and storage waste.

Integrating logging into pipelines and tracking environment details ensures robust and scalable ML projects.

Practice

1. What is the main purpose of logging artifacts and models in MLOps?

easy

A. To speed up model training

B. To delete old models automatically

C. To create new datasets from artifacts

D. To save files and models for tracking and reuse

Solution

1. Step 1: Understand the role of logging in MLOps

   Logging artifacts and models helps keep track of work and reuse it later.

2. Step 2: Identify the correct purpose

   Saving files and models for tracking and reuse matches the main goal of logging.

3. Final Answer:

   To save files and models for tracking and reuse -> Option D

4. Quick Check:

   Logging = Save and track work [OK]

Hint: Logging means saving work for later use [OK]

Common Mistakes:

• Thinking logging deletes models
• Confusing logging with speeding training
• Assuming logging creates new data

2. Which of the following is the correct syntax to log a file artifact using MLflow?

easy

A. mlflow.log_model('path/to/file.txt')

B. mlflow.log_artifact('path/to/file.txt')

C. mlflow.log_artifacts('path/to/file.txt')

D. mlflow.log('path/to/file.txt')

Solution

1. Step 1: Recall MLflow function for logging files

   The correct function is mlflow.log_artifact() for single files.

2. Step 2: Check syntax correctness

   mlflow.log_artifact('path/to/file.txt') matches the correct syntax.

3. Final Answer:

   mlflow.log_artifact('path/to/file.txt') -> Option B

4. Quick Check:

   Single file logging = log_artifact() [OK]

Hint: Use log_artifact() for single files [OK]

Common Mistakes:

• Using log_model() for files
• Using plural log_artifacts() incorrectly
• Using generic log() function

3. What will be the output of this code snippet?

import mlflow
with mlflow.start_run():
    mlflow.log_artifact('data.csv')
    mlflow.log_model(model, 'model')
print('Run finished')

medium

A. Error because model is not defined

B. Run finished printed; artifacts and model logged in current run

C. No output; code hangs

D. Run finished printed; but nothing logged

Solution

1. Step 1: Analyze the code snippet

   The code tries to log a file and a model inside a run.

2. Step 2: Check for errors

   The variable 'model' is not defined, so mlflow.log_model(model, 'model') causes a NameError.

3. Final Answer:

   Error because model is not defined -> Option A

4. Quick Check:

   Undefined variable causes error [OK]

Hint: Check if variables are defined before logging [OK]

Common Mistakes:

• Assuming code runs without defining model
• Thinking print means success
• Ignoring variable definitions

4. You run this code but no artifacts appear in MLflow UI:

mlflow.log_artifact('output.txt')

What is the most likely reason?

medium

A. log_artifact() only works inside a run

B. The file output.txt does not exist

C. No active MLflow run was started

D. MLflow server is down

Solution

1. Step 1: Understand MLflow run context

   Logging artifacts requires an active run to group logs.

2. Step 2: Identify missing run

   Without mlflow.start_run(), logs are not saved properly.

3. Final Answer:

   No active MLflow run was started -> Option C

4. Quick Check:

   Logging needs active run [OK]

Hint: Always start a run before logging [OK]

Common Mistakes:

• Assuming logging works without a run
• Ignoring file existence
• Blaming server without checking run

5. You want to log multiple files and a trained model in one MLflow run. Which code snippet correctly does this?

hard

A. with mlflow.start_run():
mlflow.log_artifact('file1.txt')
mlflow.log_artifact('file2.txt')
mlflow.log_model(model, 'model')

B. mlflow.log_artifact(['file1.txt', 'file2.txt'])
mlflow.log_model(model, 'model')

C. with mlflow.start_run():
mlflow.log_artifacts(['file1.txt', 'file2.txt'])
mlflow.log_model(model, 'model')

D. mlflow.start_run()
mlflow.log_artifact('file1.txt')
mlflow.log_artifacts('file2.txt')
mlflow.log_model(model, 'model')
mlflow.end_run()

Solution

1. Step 1: Identify correct way to log multiple files

   For multiple individual files, call mlflow.log_artifact() for each file.

2. Step 2: Confirm run context and model logging

   Using with mlflow.start_run(): ensures proper context; logs each file and the model correctly.

3. Final Answer:

   with mlflow.start_run(): mlflow.log_artifact('file1.txt') mlflow.log_artifact('file2.txt') mlflow.log_model(model, 'model') -> Option A

4. Quick Check:

   Multiple files = log_artifact() for each inside run [OK]

Hint: Call log_artifact() for each file inside a run [OK]

Common Mistakes:

• Passing list to log_artifacts()
• Not using a run context
• Using log_artifacts() on single file