Progress tracking and reporting in Agentic AI - ML Experiment: Train & Evaluate

Experiment - Progress tracking and reporting

Problem:You have trained an AI agent to perform a task, but you lack clear progress tracking and reporting. This makes it hard to know how well the agent is learning or improving over time.

Current Metrics:No metrics are currently tracked or reported during training or evaluation.

Issue:Without progress tracking, you cannot measure improvements or detect problems early. This slows down development and debugging.

Your Task

Implement a progress tracking and reporting system that logs training loss and accuracy after each epoch and reports validation metrics. The goal is to have clear, readable progress updates during training.

Do not change the model architecture or training data.

Use only built-in Python logging or print statements for reporting.

Keep the code simple and easy to understand.

Hint 1

Hint 2

Hint 3

Solution

Agentic AI

import numpy as np
from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler
from sklearn.neural_network import MLPClassifier
from sklearn.metrics import accuracy_score

# Load data
iris = load_iris()
X, y = iris.data, iris.target

# Split data
X_train, X_val, y_train, y_val = train_test_split(X, y, test_size=0.2, random_state=42)

# Scale features
scaler = StandardScaler()
X_train = scaler.fit_transform(X_train)
X_val = scaler.transform(X_val)

# Initialize model
model = MLPClassifier(hidden_layer_sizes=(10,), max_iter=1, warm_start=True, random_state=42)

epochs = 20

for epoch in range(1, epochs + 1):
    model.fit(X_train, y_train)  # One iteration per epoch
    train_preds = model.predict(X_train)
    val_preds = model.predict(X_val)
    train_acc = accuracy_score(y_train, train_preds)
    val_acc = accuracy_score(y_val, val_preds)
    train_loss = model.loss_
    print(f"Epoch {epoch}/{epochs} - Training Loss: {train_loss:.3f} - Training Accuracy: {train_acc:.3f} - Validation Accuracy: {val_acc:.3f}")

Added a training loop with multiple epochs using warm_start=True to continue training.

After each epoch, calculated training loss, training and validation accuracy.

Printed progress updates showing epoch number, training loss, and accuracies.

Results Interpretation

Before: No progress information was available during training.

After: Each epoch shows training loss, training and validation accuracy, helping track learning progress.

Tracking and reporting progress during training helps understand how well the model learns and if it improves over time. This is essential for debugging and improving AI agents.

Bonus Experiment

Add a simple progress bar that visually shows training progress for each epoch.

💡 Hint

Use the tqdm library or print a line with dots or hashes representing progress.

Practice

1. What is the main purpose of progress tracking during machine learning model training?

easy

A. To record how the model improves over time

B. To increase the size of the training data

C. To change the model architecture automatically

D. To speed up the training hardware

Solution

1. Step 1: Understand progress tracking

   Progress tracking means keeping a record of how well the model is learning as it trains.

2. Step 2: Identify the main goal

   The goal is to see improvements over time, not to change data size or hardware.

3. Final Answer:

   To record how the model improves over time -> Option A

4. Quick Check:

   Progress tracking = record improvement [OK]

Hint: Progress tracking = recording learning progress [OK]

Common Mistakes:

• Confusing progress tracking with data augmentation
• Thinking it changes model structure automatically
• Assuming it speeds up hardware

2. Which of the following is the correct way to log training loss after each epoch in Python?

easy

A. print('Loss:', loss)

B. log('Loss:' + loss)

C. print('Loss:' loss)

D. echo 'Loss:' loss

Solution

1. Step 1: Check Python print syntax

   In Python, print() requires arguments separated by commas or concatenated as strings.

2. Step 2: Validate each option

   print('Loss:', loss) uses print with a comma, which is correct. log('Loss:' + loss) uses undefined log function. print('Loss:' loss) misses a comma. echo 'Loss:' loss uses echo, which is not Python.

3. Final Answer:

   print('Loss:', loss) -> Option A

4. Quick Check:

   Correct print syntax = print('Loss:', loss) [OK]

Hint: Use print() with commas to separate text and variables [OK]

Common Mistakes:

• Missing commas in print statements
• Using non-Python functions like echo or log
• Concatenating strings without conversion

3. Given the code below, what will be printed after training for 3 epochs?

losses = []
for epoch in range(3):
    loss = 1 / (epoch + 1)
    losses.append(loss)
    print(f'Epoch {epoch+1}, Loss: {loss:.2f}')
print('Final losses:', losses)

medium

A. Epoch 1, Loss: 1.00
Epoch 2, Loss: 0.50
Epoch 3, Loss: 0.33
Final losses: [1, 2, 3]

B. Epoch 1, Loss: 0.00
Epoch 2, Loss: 0.50
Epoch 3, Loss: 0.33
Final losses: [0, 0.5, 0.3333333333333333]

C. Epoch 1, Loss: 1.00
Epoch 2, Loss: 0.50
Epoch 3, Loss: 0.33
Final losses: [1.0, 0.5, 0.3333333333333333]

D. Epoch 1, Loss: 1.00
Epoch 2, Loss: 0.50
Epoch 3, Loss: 0.33
Final losses: [1, 0.5, 0.33]

Solution

1. Step 1: Calculate loss values for each epoch

   Epoch 1: 1/(1) = 1.0, Epoch 2: 1/(2) = 0.5, Epoch 3: 1/(3) ≈ 0.3333

2. Step 2: Check printed output and final list

   Print shows formatted loss with 2 decimals. Final losses list stores full float values.

3. Final Answer:

   Epoch 1, Loss: 1.00 Epoch 2, Loss: 0.50 Epoch 3, Loss: 0.33 Final losses: [1.0, 0.5, 0.3333333333333333] -> Option C

4. Quick Check:

   Loss calculation and print formatting = Epoch 1, Loss: 1.00 Epoch 2, Loss: 0.50 Epoch 3, Loss: 0.33 Final losses: [1.0, 0.5, 0.3333333333333333] [OK]

Hint: Calculate loss per epoch and check print formatting carefully [OK]

Common Mistakes:

• Confusing integer division with float division
• Rounding losses in the list incorrectly
• Misreading the range function output

4. The following code is meant to track accuracy after each training epoch, but it throws an error. What is the error?

accuracies = []
for epoch in range(5):
    accuracy = 0.8 + epoch * 0.03
    accuracies.append(accuracy)
print('Accuracies:', accuracies)

medium

A. SyntaxError due to missing colon in for loop

B. No error; code runs correctly

C. TypeError because accuracy is not a number

D. IndexError from accessing out-of-range list element

Solution

1. Step 1: Review the code syntax and logic

   The for loop has a colon, accuracy is calculated as a float, and appended to the list.

2. Step 2: Check for runtime errors

   No invalid operations or out-of-range accesses occur.

3. Final Answer:

   No error; code runs correctly -> Option B

4. Quick Check:

   Code syntax and logic correct = No error; code runs correctly [OK]

Hint: Check for syntax and type errors carefully [OK]

Common Mistakes:

• Assuming missing colon when it is present
• Confusing variable types
• Expecting index errors without list access

5. You want to create a report that shows both training loss and accuracy after each epoch in a clear table format. Which approach best helps you track and report this progress effectively?

hard

A. Store losses and accuracies in separate lists and print them after training

B. Print loss and accuracy inside the training loop without storing values

C. Only track loss since accuracy is less important

D. Use a dictionary to store epoch as key and a tuple of (loss, accuracy) as value, then print a formatted table after training

Solution

1. Step 1: Understand the need for clear reporting

   Clear reports require organized data storage and formatted output.

2. Step 2: Evaluate each option

   Store losses and accuracies in separate lists and print them after training stores separately but may be harder to align epochs. Print loss and accuracy inside the training loop without storing values prints without storing, losing history. Use a dictionary to store epoch as key and a tuple of (loss, accuracy) as value, then print a formatted table after training uses a dictionary to link epochs with both metrics, enabling clear table printing. Only track loss since accuracy is less important ignores accuracy, which is important.

3. Final Answer:

   Use a dictionary to store epoch as key and a tuple of (loss, accuracy) as value, then print a formatted table after training -> Option D

4. Quick Check:

   Organized storage + formatted report = Use a dictionary to store epoch as key and a tuple of (loss, accuracy) as value, then print a formatted table after training [OK]

Hint: Use dictionary with epoch keys for clear progress reports [OK]

Common Mistakes:

• Not storing metrics together per epoch
• Printing inside loop without history
• Ignoring accuracy tracking