Tool usage (function calling) in Prompt Engineering / GenAI - ML Experiment: Train & Evaluate

Experiment - Tool usage (function calling)

Problem:You have a language model that can call external functions to get information or perform tasks. Currently, the model calls functions but does not handle the responses well, leading to incorrect or incomplete answers.

Current Metrics:Function call success rate: 90%, Correct answer rate after function call: 65%

Issue:The model calls functions correctly but often fails to use the returned data properly, causing low accuracy in final answers.

Your Task

Improve the model's ability to correctly use function call outputs to increase the correct answer rate to at least 85%, while maintaining or improving the function call success rate.

You cannot change the external functions themselves.

You must keep the function call success rate above 85%.

You can only modify the model's handling of function call responses.

Hint 1

Hint 2

Hint 3

Solution

Prompt Engineering / GenAI

def call_function_and_use_response(model, input_text, function_call):
    # Step 1: Model calls the function
    response = function_call(input_text)
    
    # Step 2: Validate response format
    if not isinstance(response, dict) or 'result' not in response:
        return "Error: Invalid function response"
    
    # Step 3: Extract useful data
    result = response['result']
    
    # Step 4: Use the result to generate final answer
    final_answer = model.generate_answer(input_text, result)
    
    return final_answer

# Example usage:
class DummyModel:
    def generate_answer(self, question, data):
        return f"Answer based on function data: {data}"

def example_function_call(query):
    # Simulate a function returning a dictionary
    return {"result": "42"}

model = DummyModel()
input_text = "What is the answer to life?"
output = call_function_and_use_response(model, input_text, example_function_call)
print(output)

Added explicit validation of the function response to ensure it contains expected data.

Extracted the 'result' field from the function response before using it.

Modified the model's answer generation to incorporate the validated function output clearly.

Results Interpretation

Before: Function call success rate: 90%, Correct answer rate: 65%

After: Function call success rate: 90%, Correct answer rate: 87%

Validating and properly using function call outputs helps the model produce more accurate answers without losing the ability to call functions successfully.

Bonus Experiment

Try implementing a fallback mechanism where if the function response is invalid, the model attempts a second call or uses a default answer.

💡 Hint

Add error handling and retry logic around the function call, and design the model to recognize when to fallback.

Practice

1. What is the main purpose of calling a function in AI tool usage?

easy

A. To perform a specific task using given inputs

B. To create new data without inputs

C. To store data permanently

D. To display the AI model architecture

Solution

1. Step 1: Understand function role in AI tools

   Functions are blocks of code designed to perform specific tasks when called with inputs.

2. Step 2: Identify the purpose of calling functions

   Calling a function means using it to do a job, usually with parameters to guide the task.

3. Final Answer:

   To perform a specific task using given inputs -> Option A

4. Quick Check:

   Function call = perform task with inputs [OK]

Hint: Functions do tasks using inputs, not just store or show data [OK]

Common Mistakes:

• Thinking functions create data without inputs
• Confusing function calls with data storage
• Assuming functions only display info

2. Which of the following is the correct way to call a function named generate_text with a parameter prompt in Python?

easy

A. generate_text = prompt

B. generate_text->prompt()

C. call generate_text(prompt)

D. generate_text(prompt)

Solution

1. Step 1: Recall Python function call syntax

   In Python, functions are called by writing the function name followed by parentheses enclosing parameters.

2. Step 2: Match syntax with options

   generate_text(prompt) uses the correct syntax: function name followed by parentheses with parameter inside.

3. Final Answer:

   generate_text(prompt) -> Option D

4. Quick Check:

   Python function call = name(params) [OK]

Hint: Use functionName(parameters) to call in Python [OK]

Common Mistakes:

• Using assignment (=) instead of call
• Adding extra keywords like 'call'
• Using wrong symbols like '->'

3. Given the code:

def add_numbers(a, b):
    return a + b

result = add_numbers(3, 5)
print(result)

What will be printed?

medium

A. None

B. 8

C. TypeError

D. 35

Solution

1. Step 1: Understand function behavior

   The function add_numbers takes two inputs and returns their sum.

2. Step 2: Calculate the sum of inputs 3 and 5

   3 + 5 equals 8, so the function returns 8, which is stored in result.

3. Final Answer:

   8 -> Option B

4. Quick Check:

   3 + 5 = 8 [OK]

Hint: Add inputs inside function to get output [OK]

Common Mistakes:

• Concatenating numbers as strings (35)
• Expecting error due to parameters
• Ignoring return value (None)

4. Identify the error in this function call:

def translate(text, language):
    return f"{text} in {language}"

result = translate("Hello")
print(result)

medium

A. Return statement syntax error

B. Function name is incorrect

C. Missing second argument 'language' in function call

D. Print statement is missing parentheses

Solution

1. Step 1: Check function definition parameters

   The function translate requires two parameters: text and language.

2. Step 2: Check function call arguments

   The call translate("Hello") provides only one argument, missing the second required parameter.

3. Final Answer:

   Missing second argument 'language' in function call -> Option C

4. Quick Check:

   All parameters must be given when calling [OK]

Hint: Count parameters in definition and match in call [OK]

Common Mistakes:

• Ignoring missing arguments
• Assuming default values without definition
• Misreading function name or print syntax

5. You want to use a function summarize_text that takes a long text and a number max_length to limit summary size. Which call correctly uses this function to summarize article with a max length of 100?

hard

A. summary = summarize_text(article, max_length=100)

B. summary = summarize_text(max_length=100, article)

C. summary = summarize_text(article max_length=100)

D. summary = summarize_text(article, 100, max_length)

Solution

1. Step 1: Understand function parameters and calling conventions

   The function expects two inputs: the text and the max_length number. Keyword arguments can be used to specify parameters by name.

2. Step 2: Identify correct syntax for positional and keyword arguments

   summary = summarize_text(article, max_length=100) correctly passes article as the first argument and uses max_length=100 as a keyword argument.

3. Final Answer:

   summary = summarize_text(article, max_length=100) -> Option A

4. Quick Check:

   Positional first, then keyword args [OK]

Hint: Use positional args first, then named args [OK]

Common Mistakes:

• Placing keyword argument before positional
• Missing commas between arguments
• Adding extra unexpected arguments