Function calling in LLMs in Agentic AI - Model Pipeline Trace

Model Pipeline - Function calling in LLMs

This pipeline shows how a large language model (LLM) uses function calling to improve responses by invoking external functions during text generation.

Data Flow - 4 Stages

1User Input

1 text string→Receive user question or command→1 text string

"What is the weather in Paris today?"

↓

2LLM Processing

1 text string→Analyze input and decide if function call is needed→1 text string or function call request

"Call weather_api(city='Paris')"

↓

3Function Call Execution

Function call request with parameters→Invoke external function (e.g., API) and get result→Function output data

{"temperature": "18°C", "condition": "Cloudy"}

↓

4LLM Response Generation

Function output data→Incorporate function output into final text response→1 text string

"The weather in Paris today is 18°C and cloudy."

Training Trace - Epoch by Epoch

Loss
1.0 |\
0.9 | \
0.8 |  \
0.7 |   \
0.6 |    \
0.5 |     \
0.4 |      \
0.3 |       \
0.2 |        \
0.1 |         \
0.0 +----------
      1 2 3 4 5
      Epochs

Epoch	Loss ↓	Accuracy ↑	Observation
1	0.85	0.60	Model starts learning to detect when to call functions.
2	0.65	0.72	Improved accuracy in predicting function calls.
3	0.50	0.81	Better integration of function outputs in responses.
4	0.38	0.88	Model effectively calls functions and generates accurate answers.
5	0.30	0.92	Training converges with high accuracy and low loss.

Prediction Trace - 4 Layers

Layer 1: Input Text

Layer 2: Function Call Decision

Layer 3: Function Execution

Layer 4: Response Generation

Model Quiz - 3 Questions

What does the LLM do after receiving the user input?

A Decides if a function call is needed

B Immediately calls an external function

C Returns a random answer

D Ignores the input

Key Insight

Function calling lets LLMs get real-time data or perform tasks by invoking external functions, making their answers more accurate and useful.

Practice

1. What is the main purpose of function calling in large language models (LLMs)?

easy

A. To prevent the LLM from understanding user questions

B. To let the LLM run specific external functions and get precise results

C. To slow down the LLM's response time intentionally

D. To make the LLM generate random text without any control

Solution

1. Step 1: Understand function calling role

   Function calling lets LLMs connect to external code or tools to perform tasks.

2. Step 2: Identify the main benefit

   This connection helps LLMs provide accurate, task-specific answers by running real functions.

3. Final Answer:

   To let the LLM run specific external functions and get precise results -> Option B

4. Quick Check:

   Function calling purpose = precise external function use [OK]

Hint: Function calling means running real code from the LLM [OK]

Common Mistakes:

• Thinking function calling makes LLMs slower
• Believing it causes random text generation
• Assuming it blocks understanding questions

2. Which of the following is the correct way to specify a function call in an LLM prompt?

easy

A. {"name": "get_weather", "parameters": {"city": "Paris"}}

B. function_call: get_weather(city='Paris')

C. call_function('get_weather', city='Paris')

D. run get_weather with city=Paris

Solution

1. Step 1: Recognize JSON format for function calls

   LLMs use structured JSON to specify function names and parameters clearly.

2. Step 2: Match the correct JSON syntax

   {"name": "get_weather", "parameters": {"city": "Paris"}} shows a JSON object with "name" and "parameters", which is the standard format.

3. Final Answer:

   {"name": "get_weather", "parameters": {"city": "Paris"}} -> Option A

4. Quick Check:

   Function call format = JSON object [OK]

Hint: Function calls in LLMs use JSON with name and parameters [OK]

Common Mistakes:

• Using plain text instead of JSON
• Trying to call functions like regular code
• Missing quotes around keys or values

3. Given this function call JSON sent to an LLM:

{"name": "calculate_sum", "parameters": {"a": 5, "b": 3}}

What should the LLM do next?

medium

A. Return the sum 8 as the function output

B. Ignore the function call and generate unrelated text

C. Ask the user to provide values for a and b

D. Throw an error because parameters are missing

Solution

1. Step 1: Understand the function call content

   The JSON specifies a function named "calculate_sum" with parameters a=5 and b=3.

2. Step 2: Determine expected LLM behavior

   The LLM should run the function with these inputs and return the result, which is 8.

3. Final Answer:

   Return the sum 8 as the function output -> Option A

4. Quick Check:

   Function call with inputs = output sum 8 [OK]

Hint: LLM runs function with given inputs and returns result [OK]

Common Mistakes:

• Ignoring the function call and chatting instead
• Requesting inputs already provided
• Assuming missing parameters cause errors

4. You wrote this function call JSON for an LLM:

{"name": "get_user_info", "params": {"user_id": 42}}

Why might the LLM fail to execute this call?

medium

A. Because the function name must be uppercase

B. Because user_id must be a string, not a number

C. Because the key should be "parameters", not "params"

D. Because the JSON is missing a closing brace

Solution

1. Step 1: Check JSON keys for function calling

   The standard key for parameters is "parameters", not "params".

2. Step 2: Identify why LLM fails

   Using "params" means the LLM won't recognize the inputs and can't run the function.

3. Final Answer:

   Because the key should be "parameters", not "params" -> Option C

4. Quick Check:

   Correct key = "parameters" [OK]

Hint: Use "parameters" key exactly in function call JSON [OK]

Common Mistakes:

• Using "params" instead of "parameters"
• Assuming number types cause failure
• Thinking function names must be uppercase

5. You want to build a chatbot that can book appointments by calling an external scheduling function. Which approach best uses function calling in LLMs to achieve this?

hard

A. Ask the user to book appointments manually without automation

B. Make the LLM guess appointment times without calling any function

C. Hardcode all appointment slots inside the LLM prompt text

D. Define a function schema with name and parameters, let LLM call it with user inputs, then run the real scheduler

Solution

1. Step 1: Understand chatbot function calling design

   Function calling lets the LLM decide when to call the scheduler with user data.

2. Step 2: Choose best integration method

   Defining a function schema and letting the LLM call it dynamically is the correct approach.

3. Final Answer:

   Define a function schema with name and parameters, let LLM call it with user inputs, then run the real scheduler -> Option D

4. Quick Check:

   Function calling enables dynamic external task calls [OK]

Hint: Use function schema and let LLM call real scheduler [OK]

Common Mistakes:

• Ignoring function calling and guessing answers
• Hardcoding data inside prompt text
• Not automating booking at all