Checkpointing agent progress in Agentic AI - ML Experiment: Train & Evaluate

Experiment - Checkpointing agent progress

Problem:You have an AI agent that learns tasks over time. The agent's progress is not saved during training, so if interrupted, all learning is lost.

Current Metrics:Agent completes 80% of tasks in a session but loses all progress if stopped early.

Issue:No checkpointing means no saved progress. Training must restart from zero after interruption.

Your Task

Implement checkpointing to save the agent's state periodically so training can resume without loss.

Use only built-in or standard libraries for saving/loading state.

Checkpoint every 5 training episodes.

Ensure the agent resumes correctly from the last checkpoint.

Hint 1

Hint 2

Hint 3

Solution

Agentic AI

import os
import pickle

class Agent:
    def __init__(self):
        self.knowledge = 0  # simple progress measure
        self.episode = 0

    def train_episode(self):
        # Simulate learning by increasing knowledge
        self.knowledge += 1
        self.episode += 1

    def save_checkpoint(self, filename='agent_checkpoint.pkl'):
        with open(filename, 'wb') as f:
            pickle.dump({'knowledge': self.knowledge, 'episode': self.episode}, f)

    def load_checkpoint(self, filename='agent_checkpoint.pkl'):
        if os.path.exists(filename):
            with open(filename, 'rb') as f:
                data = pickle.load(f)
                self.knowledge = data['knowledge']
                self.episode = data['episode']
                print(f"Loaded checkpoint: episode {self.episode}, knowledge {self.knowledge}")
        else:
            print("No checkpoint found, starting fresh.")


def train_agent(total_episodes=20, checkpoint_interval=5):
    agent = Agent()
    agent.load_checkpoint()

    for _ in range(agent.episode, total_episodes):
        agent.train_episode()
        print(f"Episode {agent.episode}: knowledge {agent.knowledge}")

        if agent.episode % checkpoint_interval == 0:
            agent.save_checkpoint()
            print(f"Checkpoint saved at episode {agent.episode}")

    # Save final checkpoint
    agent.save_checkpoint()
    print("Training complete and checkpoint saved.")


if __name__ == '__main__':
    train_agent()

Added methods to save and load agent state using pickle.

Implemented checkpoint saving every 5 episodes during training.

Modified training loop to resume from last saved episode.

Results Interpretation

Before checkpointing: Agent loses all progress if training stops early.

After checkpointing: Agent resumes from last saved state, preserving progress and saving time.

Checkpointing is essential to save learning progress in long training processes. It prevents loss of work and allows resuming training efficiently.

Bonus Experiment

Try implementing checkpointing with a human-readable format like JSON instead of pickle.

💡 Hint

Convert agent state to a dictionary and save/load using json module. Remember to handle data types correctly.

Practice

1. What is the main purpose of checkpointing in agentic AI?

easy

A. To save and restore an agent's progress during tasks

B. To speed up the agent's decision-making process

C. To increase the agent's memory capacity

D. To change the agent's learning algorithm

Solution

1. Step 1: Understand checkpointing concept

   Checkpointing means saving the current state or progress of an agent so it can continue later without losing work.

2. Step 2: Identify the main purpose

   The main purpose is to save and restore progress, not to speed up decisions or change algorithms.

3. Final Answer:

   To save and restore an agent's progress during tasks -> Option A

4. Quick Check:

   Checkpointing = Save and restore progress [OK]

Hint: Checkpointing means saving progress to continue later [OK]

Common Mistakes:

• Thinking checkpointing speeds up decisions
• Confusing checkpointing with changing algorithms
• Assuming checkpointing increases memory

2. Which method is used to save an agent's progress in checkpointing?

easy

A. load_checkpoint()

B. start_training()

C. save_checkpoint()

D. reset_agent()

Solution

1. Step 1: Recall checkpointing methods

   Checkpointing uses two main methods: save_checkpoint() to save progress and load_checkpoint() to restore it.

2. Step 2: Identify the saving method

   save_checkpoint() is the method that saves the agent's current state.

3. Final Answer:

   save_checkpoint() -> Option C

4. Quick Check:

   Save progress = save_checkpoint() [OK]

Hint: Save uses save_checkpoint(), load uses load_checkpoint() [OK]

Common Mistakes:

• Choosing load_checkpoint() to save progress
• Confusing reset_agent() with saving
• Thinking start_training() saves progress

3. Given this code snippet, what will be printed?

agent.save_checkpoint('step1.ckpt')
agent.load_checkpoint('step1.ckpt')
print(agent.progress)

medium

A. An error because load_checkpoint is missing arguments

B. The agent's progress at step1

C. None, because progress is not saved

D. The initial progress before saving

Solution

1. Step 1: Understand save_checkpoint and load_checkpoint

   save_checkpoint saves the agent's current progress to a file. load_checkpoint restores that saved progress.

2. Step 2: Analyze the code flow

   The agent saves progress to 'step1.ckpt', then immediately loads it back, so agent.progress reflects the saved state.

3. Final Answer:

   The agent's progress at step1 -> Option B

4. Quick Check:

   Save then load = restored progress [OK]

Hint: Save then load returns saved progress, not error [OK]

Common Mistakes:

• Assuming load_checkpoint causes error
• Thinking progress is lost after loading
• Confusing initial progress with saved progress

4. What is wrong with this checkpointing code?

agent.load_checkpoint('step1.ckpt')
agent.save_checkpoint('step2.ckpt')

medium

A. File names must be integers, not strings

B. save_checkpoint requires no arguments

C. load_checkpoint cannot be called twice

D. Loading before saving may restore old progress

Solution

1. Step 1: Check order of checkpoint calls

   Loading before saving means the agent restores old progress first, then saves new progress, which may not be intended.

2. Step 2: Validate method usage

   save_checkpoint requires a filename string argument, so the call is correct. load_checkpoint can be called multiple times. File names as strings are valid.

3. Final Answer:

   Loading before saving may restore old progress -> Option D

4. Quick Check:

   Load before save risks old progress [OK]

Hint: Save before load to avoid restoring old progress [OK]

Common Mistakes:

• Thinking save_checkpoint needs no arguments
• Believing load_checkpoint can't be called multiple times
• Assuming file names must be integers

5. You want to checkpoint an agent working on a long task that may stop unexpectedly. Which strategy best ensures minimal lost progress?

hard

A. Save checkpoints frequently during the task and load the latest on restart

B. Save one checkpoint only at the end of the task

C. Load checkpoints multiple times without saving

D. Avoid checkpointing to keep the agent fast

Solution

1. Step 1: Understand the problem of unexpected stops

   If the task stops unexpectedly, progress since the last save is lost unless checkpoints are saved often.

2. Step 2: Choose the best checkpointing strategy

   Saving checkpoints frequently ensures minimal lost progress. Loading the latest checkpoint on restart resumes work efficiently.

3. Final Answer:

   Save checkpoints frequently during the task and load the latest on restart -> Option A

4. Quick Check:

   Frequent saves minimize lost progress [OK]

Hint: Save often to avoid losing progress on stops [OK]

Common Mistakes:

• Saving only once at the end risks losing all progress
• Loading without saving loses new progress
• Avoiding checkpointing ignores recovery needs