Experiment tracking (MLflow) in ML Python - Model Pipeline Trace

Model Pipeline - Experiment tracking (MLflow)

This pipeline shows how MLflow helps track machine learning experiments. It records data, model training steps, and results so you can compare and improve models easily.

Data Flow - 6 Stages

1Data Loading

1000 rows x 10 columns→Load dataset from CSV file→1000 rows x 10 columns

Sample row: {"feature1": 5.1, "feature2": 3.5, ..., "label": 1}

↓

2Data Preprocessing

1000 rows x 10 columns→Fill missing values and normalize features→1000 rows x 10 columns

Normalized feature1 value: 0.45

↓

3Train/Test Split

1000 rows x 10 columns→Split data into training and testing sets (80/20)→Train: 800 rows x 10 columns, Test: 200 rows x 10 columns

Train sample label: 0, Test sample label: 1

↓

4Model Training

Train: 800 rows x 9 features→Train logistic regression model→Trained model object

Model coefficients: [0.5, -0.3, 0.1, ...]

↓

5Experiment Logging

Trained model, training metrics→Log parameters, metrics, and model to MLflow→Experiment record in MLflow server

Logged accuracy: 0.85, loss: 0.35

↓

6Model Evaluation

Test: 200 rows x 9 features→Predict and calculate accuracy→Accuracy score: 0.83

Predicted labels: [0,1,0,0,1,...]

Training Trace - Epoch by Epoch

Loss
0.7 |****
0.6 |*** 
0.5 |**  
0.4 |*   
0.3 |*   
     1 2 3 4 5 Epochs

Epoch	Loss ↓	Accuracy ↑	Observation
1	0.65	0.60	Model starts learning, loss is high, accuracy low
2	0.50	0.72	Loss decreases, accuracy improves
3	0.40	0.78	Model continues to improve
4	0.35	0.82	Loss lowers further, accuracy near good level
5	0.33	0.85	Training converges with good accuracy

Prediction Trace - 4 Layers

Layer 1: Input Features

Layer 2: Linear Combination

Layer 3: Sigmoid Activation

Layer 4: Thresholding

Model Quiz - 3 Questions

What does MLflow track during model training?

A Only the final model

B Only the training data

C Parameters, metrics, and models

D Only the test results

Key Insight

Using MLflow to track experiments helps keep all training details organized. This makes it easier to compare models, understand improvements, and reproduce results reliably.