Task decomposition strategies in Agentic AI - Model Metrics & Evaluation

Metrics & Evaluation - Task decomposition strategies

Which metric matters for Task Decomposition Strategies and WHY

When breaking a big task into smaller parts, we want to measure how well each part helps the whole goal. Key metrics include accuracy of each subtask, completion rate, and error propagation. These show if the parts work well alone and together. If subtasks have low accuracy, the final result suffers. So, monitoring each step's performance helps improve the whole process.

Confusion Matrix for a Subtask Example

    Subtask: Classify images into cats or dogs

          Predicted
          Cat   Dog
    True Cat  80    20
         Dog  15    85

    Total samples = 200
    TP (Cat) = 80, FP (Cat) = 15, FN (Cat) = 20, TN (Cat) = 85
    

This matrix helps calculate precision and recall for the subtask. Good subtasks have high precision and recall, so errors don't build up.

Tradeoff: Precision vs Recall in Task Decomposition

Imagine a task split into parts where one part finds important items (high recall) but sometimes mistakes others (low precision). Another part is very sure but misses some items (high precision, low recall). Balancing these is key. For example, in a medical diagnosis task, missing a disease (low recall) is worse than false alarms (low precision). So, task parts should be tuned to the goal.

Good vs Bad Metric Values for Task Decomposition

• Good: Subtasks with precision and recall above 90%, low error propagation, and consistent completion.
• Bad: Subtasks with precision or recall below 60%, causing many errors to pass on and reduce final output quality.

Good metrics mean subtasks work well alone and together. Bad metrics show weak parts hurting the whole.

Common Pitfalls in Metrics for Task Decomposition

• Ignoring error propagation: Small errors in subtasks can grow and ruin final results.
• Overfitting subtasks: Subtasks too tuned to training data may fail in real use.
• Data leakage: Subtasks accidentally use future info, inflating metrics falsely.
• Accuracy paradox: High accuracy in subtasks with imbalanced data can be misleading.

Self Check

Your task decomposition model has 98% accuracy overall but one subtask has only 12% recall on a critical class. Is it good for production?

Answer: No. The low recall means the subtask misses many important cases. This will cause the whole system to fail on those cases, despite high overall accuracy. Improving recall in that subtask is crucial before production.

Key Result

In task decomposition, monitoring precision and recall of subtasks is key to ensure errors don't accumulate and the final output stays reliable.

Practice

1. What is the main purpose of task decomposition in agentic AI?

easy

A. To make tasks more confusing and complex

B. To combine multiple tasks into one complex task

C. To skip unnecessary steps in a task

D. To break a big task into smaller, manageable steps

Solution

1. Step 1: Understand the meaning of task decomposition

   Task decomposition means splitting a big task into smaller parts that are easier to handle.

2. Step 2: Identify the correct purpose

   The goal is to make complex problems simpler by breaking them down, not to combine or skip steps.

3. Final Answer:

   To break a big task into smaller, manageable steps -> Option D

4. Quick Check:

   Task decomposition = breaking big tasks down [OK]

Hint: Think: big task split into small steps [OK]

Common Mistakes:

• Confusing decomposition with combining tasks
• Thinking it skips steps
• Believing it makes tasks more complex

2. Which of the following is the correct way to write a step in task decomposition?

easy

A. Break the task into clear, simple steps done one by one

B. Do everything at once to save time

C. Skip steps that seem hard to avoid difficulty

D. Mix unrelated tasks in one step for speed

Solution

1. Step 1: Review the principles of task decomposition

   Steps should be clear, simple, and done one after another to keep things organized.

2. Step 2: Match the correct description

   Break the task into clear, simple steps done one by one matches this idea, while others suggest skipping or mixing tasks, which is incorrect.

3. Final Answer:

   Break the task into clear, simple steps done one by one -> Option A

4. Quick Check:

   Clear, simple steps = correct decomposition [OK]

Hint: Choose clear, simple, one-by-one steps [OK]

Common Mistakes:

• Trying to do all steps at once
• Skipping difficult steps
• Mixing unrelated tasks in one step

3. Given the task: "Prepare a report", which of the following is a correct decomposition output?

medium

A. ["Review report", "Write report", "Analyze data", "Collect data"]

B. ["Collect data", "Analyze data", "Write report", "Review report"]

C. ["Write report", "Collect data", "Review report", "Analyze data"]

D. ["Analyze data", "Review report", "Collect data", "Write report"]

Solution

1. Step 1: Understand logical order of steps

   First, collect data, then analyze it, write the report, and finally review it.

2. Step 2: Check each option's order

   Only ["Collect data", "Analyze data", "Write report", "Review report"] follows this logical sequence; others mix the order incorrectly.

3. Final Answer:

   ["Collect data", "Analyze data", "Write report", "Review report"] -> Option B

4. Quick Check:

   Logical step order = ["Collect data", "Analyze data", "Write report", "Review report"] [OK]

Hint: Follow natural task order step-by-step [OK]

Common Mistakes:

• Mixing steps in wrong order
• Starting with writing before data collection
• Reviewing before writing

4. Identify the error in this task decomposition: ["Start project", "Finish project", "Plan project", "Execute project"]

medium

A. Steps are in wrong order; planning should come before starting

B. No error; steps are correct

C. Missing a step to review the project

D. Too many steps; should combine start and finish

Solution

1. Step 1: Analyze the logical order of steps

   Planning should happen before starting a project to guide the work.

2. Step 2: Identify the error

   The list starts with 'Start project' before 'Plan project', which is incorrect order.

3. Final Answer:

   Steps are in wrong order; planning should come before starting -> Option A

4. Quick Check:

   Plan before start = correct order [OK]

Hint: Plan first, then start [OK]

Common Mistakes:

• Ignoring step order
• Assuming no error if steps exist
• Thinking start can come before plan

5. You want to build an agentic AI to clean a messy room. Which task decomposition strategy is best?

hard

A. Only clean visible areas and skip hidden spots

B. Clean the whole room randomly without order

C. Divide the room into zones, then clean each zone step-by-step

D. Try to clean everything at once with no steps

Solution

1. Step 1: Understand the problem complexity

   Cleaning a messy room is complex; breaking it into zones makes it manageable.

2. Step 2: Choose the best strategy

   Divide the room into zones, then clean each zone step-by-step breaks the task into smaller parts done step-by-step, which is effective and organized.

3. Final Answer:

   Divide the room into zones, then clean each zone step-by-step -> Option C

4. Quick Check:

   Break big task into small parts = Divide the room into zones, then clean each zone step-by-step [OK]

Hint: Split big task into zones or parts [OK]

Common Mistakes:

• Cleaning randomly without plan
• Skipping parts of the task
• Trying to do everything at once