Progress tracking and reporting in Agentic AI - ML Experiment: Train & Evaluate

Experiment - Progress tracking and reporting

Problem:You have trained an AI agent to perform a task, but you lack clear progress tracking and reporting. This makes it hard to know how well the agent is learning or improving over time.

Current Metrics:No metrics are currently tracked or reported during training or evaluation.

Issue:Without progress tracking, you cannot measure improvements or detect problems early. This slows down development and debugging.

Your Task

Implement a progress tracking and reporting system that logs training loss and accuracy after each epoch and reports validation metrics. The goal is to have clear, readable progress updates during training.

Do not change the model architecture or training data.

Use only built-in Python logging or print statements for reporting.

Keep the code simple and easy to understand.

Hint 1

Hint 2

Hint 3

Solution

Agentic AI

import numpy as np
from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler
from sklearn.neural_network import MLPClassifier
from sklearn.metrics import accuracy_score

# Load data
iris = load_iris()
X, y = iris.data, iris.target

# Split data
X_train, X_val, y_train, y_val = train_test_split(X, y, test_size=0.2, random_state=42)

# Scale features
scaler = StandardScaler()
X_train = scaler.fit_transform(X_train)
X_val = scaler.transform(X_val)

# Initialize model
model = MLPClassifier(hidden_layer_sizes=(10,), max_iter=1, warm_start=True, random_state=42)

epochs = 20

for epoch in range(1, epochs + 1):
    model.fit(X_train, y_train)  # One iteration per epoch
    train_preds = model.predict(X_train)
    val_preds = model.predict(X_val)
    train_acc = accuracy_score(y_train, train_preds)
    val_acc = accuracy_score(y_val, val_preds)
    train_loss = model.loss_
    print(f"Epoch {epoch}/{epochs} - Training Loss: {train_loss:.3f} - Training Accuracy: {train_acc:.3f} - Validation Accuracy: {val_acc:.3f}")

Added a training loop with multiple epochs using warm_start=True to continue training.

After each epoch, calculated training loss, training and validation accuracy.

Printed progress updates showing epoch number, training loss, and accuracies.

Results Interpretation

Before: No progress information was available during training.

After: Each epoch shows training loss, training and validation accuracy, helping track learning progress.

Tracking and reporting progress during training helps understand how well the model learns and if it improves over time. This is essential for debugging and improving AI agents.

Bonus Experiment

Add a simple progress bar that visually shows training progress for each epoch.

💡 Hint

Use the tqdm library or print a line with dots or hashes representing progress.