Human approval workflows in Agentic AI - Practice Problems & Coding Challenges

Challenge - 5 Problems

🧠 Conceptual

intermediate

What is the main purpose of a human approval workflow in AI systems?

Human approval workflows are often integrated into AI systems. What is their primary role?

A To automatically retrain AI models without human input

B To replace AI models with human decision-making entirely

C To allow humans to review and approve AI decisions before final action is taken

D To speed up AI predictions by skipping human checks

💡 Hint

Think about why humans might be involved in AI decision processes.

Human approval workflows let people check AI outputs to ensure safety, fairness, or correctness before final use.

❓ Model Choice

intermediate

Which AI model type best supports human approval workflows for text classification?

You want an AI model that can provide confidence scores to help humans decide when to approve or review text classification results. Which model type is best?

A Deterministic rule-based model without confidence scores

B Unsupervised clustering model without labels

C Generative model that creates new text samples

D Probabilistic model that outputs confidence probabilities

💡 Hint

Confidence scores help humans decide when to trust AI outputs.

Probabilistic models provide confidence levels, enabling human reviewers to focus on uncertain cases.

❓ Metrics

advanced

How does adding a human approval step affect the overall accuracy metric of an AI system?

An AI system has 85% accuracy alone. After adding a human approval step that corrects 10% of AI errors, what is the new effective accuracy?

A 90% because human approval fixes some AI mistakes

B 85% because human approval does not change AI accuracy

C 75% because human approval slows down the system

D 95% because human approval corrects all AI errors

💡 Hint

Calculate how many errors remain after human correction.

Original errors are 15%. Human approval fixes 10% of these errors, so errors reduce to 13.5%, making accuracy 86.5% (rounded to 90% for this question).

🔧 Debug

advanced

Identify the error in this human approval workflow code snippet

Review the following Python code that integrates human approval in an AI prediction pipeline. What error will it cause?

Agentic AI

def ai_predict(input_data):
    # returns prediction and confidence
    return {'label': 'spam', 'confidence': 0.7}

def human_approval(prediction):
    if prediction['confidence'] < 0.8:
        return input('Approve prediction? (yes/no): ')
    else:
        return 'yes'

result = ai_predict('email text')
approved = human_approval(result)
if approved == 'yes':
    print('Final label:', result['label'])
else:
    print('Prediction rejected')

A The input() function will cause a runtime error in non-interactive environments

B The ai_predict function returns a list instead of a dictionary

C The confidence key is missing in the prediction dictionary

D The human_approval function always returns None

💡 Hint

Consider where this code might run and how input() behaves.

Using input() requires user interaction. In automated or server environments, this causes a runtime error.

❓ Hyperparameter

expert

Which hyperparameter adjustment best balances AI autonomy and human approval workload?

In a human approval workflow, the AI model outputs a confidence threshold to decide when to request human review. Which threshold setting best balances reducing human workload while maintaining safety?

A Set threshold very low (e.g., 0.1) to request human review on almost all predictions

B Set threshold very high (e.g., 0.95) to request human review only on very uncertain predictions

C Set threshold at 0.5 to request human review on half of predictions randomly

D Disable threshold and have humans review all predictions

💡 Hint

Think about when human review is most needed and how to reduce unnecessary checks.

A high threshold means only uncertain predictions get human review, reducing workload while keeping safety.

Practice

1. What is the main purpose of a human approval workflow in AI systems?

easy

A. To have people check AI decisions for important or sensitive tasks

B. To replace AI models with human decision-making completely

C. To speed up AI processing by skipping checks

D. To train AI models without any human input

Solution

1. Step 1: Understand the role of human approval workflows

   Human approval workflows are designed to combine AI with human checks to ensure important decisions are correct and safe.

2. Step 2: Identify the correct purpose

   The main goal is to have humans review AI decisions when needed, especially for sensitive or critical tasks.

3. Final Answer:

   To have people check AI decisions for important or sensitive tasks -> Option A

4. Quick Check:

   Human approval = human checks on AI decisions [OK]

Hint: Human approval means people check AI decisions [OK]

Common Mistakes:

• Thinking human approval replaces AI fully
• Believing it speeds up AI by skipping checks
• Assuming it trains AI without humans

2. Which of the following is the correct way to write a condition that asks for human approval if the AI confidence is below 0.7?

easy

A. if confidence != 0.7: request_approval()

B. if confidence < 0.7: request_approval()

C. if confidence == 0.7: request_approval()

D. if confidence > 0.7: request_approval()

Solution

1. Step 1: Understand the condition logic

   We want to request human approval when confidence is less than 0.7, so the condition should check for values below 0.7.

2. Step 2: Match the correct syntax

   The correct syntax is if confidence < 0.7: followed by the approval request function.

3. Final Answer:

   if confidence < 0.7: request_approval() -> Option B

4. Quick Check:

   Less than 0.7 triggers approval [OK]

Hint: Use < for 'below' conditions in code [OK]

Common Mistakes:

• Using > instead of <
• Checking equality instead of less than
• Using != which triggers on all but 0.7

3. Given this code snippet, what will be printed if confidence = 0.65?

if confidence < 0.7:
    print('Request human approval')
else:
    print('Auto approve')

medium

A. Auto approve

B. No output

C. Request human approval

D. Syntax error

Solution

1. Step 1: Check the condition with given confidence

   Since confidence is 0.65, which is less than 0.7, the condition confidence < 0.7 is true.

2. Step 2: Determine which print statement runs

   Because the condition is true, the code prints 'Request human approval'.

3. Final Answer:

   Request human approval -> Option C

4. Quick Check:

   0.65 < 0.7 triggers approval print [OK]

Hint: Check if confidence is less than threshold to decide output [OK]

Common Mistakes:

• Choosing 'Auto approve' by confusing condition
• Thinking no output occurs
• Assuming syntax error without checking code

4. Identify the error in this human approval workflow code snippet:

def check_approval(confidence):
    if confidence < 0.7
        return 'Request approval'
    else:
        return 'Auto approve'

medium

A. Function missing return statement

B. Wrong comparison operator

C. Indentation error in else block

D. Missing colon after if statement

Solution

1. Step 1: Check syntax of if statement

   The if statement is missing a colon (:) at the end, which is required in Python syntax.

2. Step 2: Verify other parts of the code

   The comparison operator is correct, indentation looks fine, and return statements are present.

3. Final Answer:

   Missing colon after if statement -> Option D

4. Quick Check:

   Python if needs colon [:] [OK]

Hint: Check for colons after if/else statements [OK]

Common Mistakes:

• Ignoring missing colon syntax error
• Thinking indentation is wrong
• Assuming return statements are missing

5. You want to build a human approval workflow that requests approval only if the AI confidence is below 0.7 or if the task is marked as 'high risk'. Which condition correctly implements this logic in Python?

hard

A. if confidence < 0.7 or task == 'high risk': request_approval()

B. if confidence < 0.7 and task == 'high risk': request_approval()

C. if confidence >= 0.7 or task != 'high risk': request_approval()

D. if confidence > 0.7 and task == 'high risk': request_approval()

Solution

1. Step 1: Understand the logic needed

   Approval is requested if confidence is below 0.7 OR the task is 'high risk'. This means either condition triggers approval.

2. Step 2: Match the correct Python condition

   The correct condition uses the 'or' operator to combine the two checks: confidence < 0.7 or task == 'high risk'.

3. Final Answer:

   if confidence < 0.7 or task == 'high risk': request_approval() -> Option A

4. Quick Check:

   Use 'or' for either condition triggering approval [OK]

Hint: Use 'or' to combine conditions for approval [OK]

Common Mistakes:

• Using 'and' instead of 'or' which requires both true
• Reversing comparison operators
• Confusing task string equality