MLOps maturity levels - Commands & Configuration

Introduction

MLOps maturity levels help teams understand how advanced their machine learning operations are. They show the steps to improve how models are built, tested, deployed, and monitored.

When you want to know how well your ML projects are managed and where to improve.

When your team is starting to automate ML workflows and wants to track progress.

When you need to explain to stakeholders how mature your ML processes are.

When planning to add new tools or practices to your ML pipeline.

When comparing your ML operations with industry standards.

Commands

Starts the MLflow tracking server to view experiments and model runs in a web interface.

Terminal

mlflow ui

Expected Output

2024/06/01 12:00:00 INFO mlflow.server: Starting MLflow server...

2024/06/01 12:00:00 INFO mlflow.server: Listening at http://127.0.0.1:5000

→

--host - Specify the network interface to listen on

→

--port - Specify the port number for the server

Creates a new MLflow experiment to track runs related to MLOps maturity improvements.

Terminal

mlflow experiments create --experiment-name 'MLOps Maturity Study'

Expected Output

Created experiment with ID 1

→

--experiment-name - Name the experiment for easy identification

Runs the ML project in the current directory, logging parameters and metrics to track progress in maturity.

Terminal

mlflow run .

Expected Output

2024/06/01 12:01:00 INFO mlflow.projects: Running run with ID '1234567890abcdef'

Run completed successfully

Deploys the trained model from a specific run to a local REST API for testing and validation.

Terminal

mlflow models serve -m runs:/1234567890abcdef/model -p 1234

Expected Output

2024/06/01 12:02:00 INFO mlflow.models: Serving model at http://127.0.0.1:1234

2024/06/01 12:02:00 INFO mlflow.models: Use Ctrl+C to stop server

→

-m - Specify the model URI to serve

→

-p - Specify the port for the model server

Key Concept

If you remember nothing else from MLOps maturity levels, remember: they guide you step-by-step to improve how your ML models are managed and delivered.

Code Example

MLOps

import mlflow
from mlflow import log_metric, log_param, start_run

with start_run():
    log_param("maturity_level", "basic")
    log_metric("accuracy", 0.85)
    print("Logged MLOps maturity level and accuracy metric")

Common Mistakes

Not tracking experiments and runs consistently

Without tracking, you cannot measure progress or reproduce results, which blocks maturity growth.

Always use tools like MLflow to log parameters, metrics, and artifacts for every run.

Skipping model deployment testing

Deploying without testing can cause failures in production and loss of trust in ML systems.

Use MLflow model serving or similar tools to test models locally before production deployment.

Ignoring monitoring after deployment

Without monitoring, model performance can degrade unnoticed, harming business outcomes.

Set up monitoring to track model predictions and data drift continuously.

Summary

Use MLflow commands to create experiments and track ML runs.

Deploy models locally to test before production.

Track metrics and parameters to measure MLOps maturity progress.

Practice

1.

What does the first level of MLOps maturity typically represent?

Level 1 maturity means:

easy

A. Manual and ad-hoc ML processes with little automation

B. Fully automated and optimized ML pipelines

C. Real-time monitoring and feedback loops

D. Collaborative model governance and compliance

Solution

1. Step 1: Understand MLOps maturity basics

   The first level usually means starting with manual, unstructured ML workflows.

2. Step 2: Identify characteristics of Level 1

   At this stage, automation and monitoring are minimal or absent.

3. Final Answer:

   Manual and ad-hoc ML processes with little automation -> Option A

4. Quick Check:

   Level 1 = Manual workflows [OK]

Hint: Level 1 means manual work, no automation [OK]

Common Mistakes:

• Confusing Level 1 with fully automated pipelines
• Thinking monitoring is present at Level 1
• Assuming collaboration is mature at Level 1

2.

Which of the following is the correct description of Level 3 in MLOps maturity?

easy

A. No automation, manual model training and deployment

B. Basic automation of training and deployment pipelines

C. Fully optimized ML lifecycle with real-time monitoring

D. Automated pipelines with continuous integration and delivery

Solution

1. Step 1: Recall Level 3 characteristics

   Level 3 usually means automation with continuous integration and delivery (CI/CD) for ML.

2. Step 2: Match options to Level 3

   Automated pipelines with continuous integration and delivery describes automated pipelines with CI/CD, fitting Level 3 maturity.

3. Final Answer:

   Automated pipelines with continuous integration and delivery -> Option D

4. Quick Check:

   Level 3 = Automated CI/CD pipelines [OK]

Hint: Level 3 means automated CI/CD pipelines [OK]

Common Mistakes:

• Mixing Level 2 basic automation with Level 3
• Confusing Level 4 optimization with Level 3
• Assuming no automation at Level 3

3.

Given this description of an MLOps system:

"Model training is automated, deployment happens via CI/CD pipelines, but monitoring is manual and infrequent."

Which MLOps maturity level best fits this description?

medium

A. Level 3 - Automated pipelines with partial monitoring

B. Level 4 - Fully automated with real-time monitoring

C. Level 2 - Basic automation without CI/CD

D. Level 1 - Manual processes only

Solution

1. Step 1: Analyze automation and monitoring status

   Training and deployment are automated with CI/CD, but monitoring is manual and infrequent.

2. Step 2: Match description to maturity levels

   Level 3 fits automated pipelines with partial monitoring; Level 4 requires real-time monitoring.

3. Final Answer:

   Level 3 - Automated pipelines with partial monitoring -> Option A

4. Quick Check:

   Automation with manual monitoring = Level 3 [OK]

Hint: Automation + manual monitoring = Level 3 [OK]

Common Mistakes:

• Choosing Level 4 despite manual monitoring
• Confusing Level 2 with no CI/CD
• Selecting Level 1 ignoring automation

4.

Identify the error in this MLOps maturity description:

"Level 2 maturity means fully optimized ML lifecycle with real-time monitoring and feedback loops."

What is wrong with this statement?

medium

A. Level 2 is the highest maturity level

B. Level 2 means no automation at all

C. Level 2 does not include real-time monitoring and full optimization

D. Level 2 only applies to data preprocessing

Solution

1. Step 1: Recall Level 2 maturity features

   Level 2 usually means basic automation, not full optimization or real-time monitoring.

2. Step 2: Identify mismatch in statement

   The statement incorrectly assigns advanced features to Level 2 that belong to higher levels.

3. Final Answer:

   Level 2 does not include real-time monitoring and full optimization -> Option C

4. Quick Check:

   Level 2 ≠ full optimization [OK]

Hint: Level 2 = basic automation, not full optimization [OK]

Common Mistakes:

• Thinking Level 2 is highest maturity
• Assuming Level 2 means no automation
• Confusing Level 2 scope with data-only tasks

5.

Your team wants to improve from Level 2 to Level 3 maturity. Which action best supports this upgrade?

Select the best next step:

hard

A. Add manual checks for model quality after deployment

B. Implement automated CI/CD pipelines for training and deployment

C. Focus on data labeling accuracy only

D. Create documentation for manual model retraining

Solution

1. Step 1: Understand Level 2 vs Level 3 differences

   Level 3 maturity requires automated CI/CD pipelines for ML workflows, beyond basic automation.

2. Step 2: Identify the best action to reach Level 3

   Implementing automated CI/CD pipelines directly addresses this requirement.

3. Final Answer:

   Implement automated CI/CD pipelines for training and deployment -> Option B

4. Quick Check:

   CI/CD automation = Level 3 upgrade [OK]

Hint: Automate CI/CD pipelines to move to Level 3 [OK]

Common Mistakes:

• Choosing manual checks instead of automation
• Focusing only on data labeling, not pipelines
• Thinking documentation alone upgrades maturity