Performance metric tracking in MLOps - Step-by-Step Execution

Process Flow - Performance metric tracking

Start Training Model

↓

Calculate Metrics

↓

Log Metrics to Tracking System

↓

Visualize Metrics

↓

Evaluate Model Performance

↓

Decide: Improve or Deploy?

No→End

Yes↓

Adjust Model or Data

↩Back to Start Training Model

This flow shows how model training metrics are calculated, logged, visualized, and used to decide next steps.

Execution Sample

MLOps

metrics = {'accuracy': 0.85, 'loss': 0.35}
log_metrics(metrics)
visualize_metrics()
if metrics['accuracy'] > 0.8:
    deploy_model()

This code calculates metrics, logs them, visualizes, and deploys if accuracy is good.

Process Table

Step	Action	Metrics State	System State	Output/Result
1	Calculate metrics after training	{'accuracy': 0.85, 'loss': 0.35}	Metrics ready	Metrics dictionary created
2	Log metrics to tracking system	{'accuracy': 0.85, 'loss': 0.35}	Metrics logged	Metrics stored in tracking DB
3	Visualize metrics	{'accuracy': 0.85, 'loss': 0.35}	Metrics visualized	Graphs displayed on dashboard
4	Evaluate if accuracy > 0.8	{'accuracy': 0.85, 'loss': 0.35}	Decision point	Condition True
5	Deploy model	{'accuracy': 0.85, 'loss': 0.35}	Model deployed	Model available for use
6	End process	{'accuracy': 0.85, 'loss': 0.35}	Process complete	No further action

💡 Accuracy condition met, model deployed, process ends

Status Tracker

Variable	Start	After Step 1	After Step 2	After Step 3	After Step 4	Final
metrics	{}	{'accuracy': 0.85, 'loss': 0.35}	{'accuracy': 0.85, 'loss': 0.35}	{'accuracy': 0.85, 'loss': 0.35}	{'accuracy': 0.85, 'loss': 0.35}	{'accuracy': 0.85, 'loss': 0.35}
system_state	Idle	Metrics ready	Metrics logged	Metrics visualized	Decision point	Model deployed

Key Moments - 2 Insights

Why do we log metrics before visualizing them?

Logging stores metrics persistently so visualization tools can access accurate data, as shown in step 2 and 3 of the execution_table.

What happens if the accuracy is below 0.8?

The condition in step 4 would be False, so deployment would not happen and the process would loop back to improve the model.

Visual Quiz - 3 Questions

Look at the execution_table, what is the system state after logging metrics?

A Metrics ready

B Metrics logged

C Metrics visualized

D Decision point

💡 Hint

Check the 'System State' column at step 2 in the execution_table.

At which step does the model get deployed?

A Step 5

B Step 3

C Step 4

D Step 6

💡 Hint

Look for the 'Action' column mentioning deployment in the execution_table.

If accuracy was 0.75 instead of 0.85, what would happen at step 4?

A Metrics would not be logged

B Condition would be True and model deploys

C Condition would be False and deployment skipped

D Visualization would fail

💡 Hint

Refer to the 'Evaluate if accuracy > 0.8' condition in step 4 of the execution_table.

Concept Snapshot

Performance Metric Tracking:
- Calculate metrics after model training
- Log metrics to a tracking system
- Visualize metrics on dashboards
- Use metrics to decide deployment
- Loop back if performance is insufficient

Full Transcript

Performance metric tracking in MLOps involves calculating key metrics like accuracy and loss after training a model. These metrics are then logged into a tracking system to keep a record. Visualization tools read these logged metrics to display graphs and charts for easy understanding. Based on the metrics, a decision is made whether to deploy the model or improve it further. If the accuracy is above a threshold, the model is deployed; otherwise, the process repeats with adjustments. This flow ensures continuous monitoring and improvement of model performance.