Function calling in LLMs in Agentic AI - Model Metrics & Evaluation

Metrics & Evaluation - Function calling in LLMs

Which metric matters for Function calling in LLMs and WHY

When evaluating function calling in large language models (LLMs), the key metric is accuracy of function call prediction. This means how often the model correctly decides which function to call based on the input. It matters because the model must pick the right function to get the correct result, just like choosing the right tool for a job.

Other important metrics include precision and recall for function calls. Precision tells us how many of the called functions were actually correct, while recall tells us how many correct functions the model found out of all possible correct ones. These help balance between calling too many wrong functions and missing needed ones.

Confusion matrix for function calling

      | Predicted Function Call |
      |------------------------|
      | True Positive (TP): Model called the correct function when needed
      | False Positive (FP): Model called a wrong function
      | True Negative (TN): Model correctly did not call a function
      | False Negative (FN): Model missed calling a needed function

      Example:
      TP = 80, FP = 10, TN = 900, FN = 20
      Total samples = 1010
    

Precision vs Recall tradeoff with examples

If the model calls functions too often, it may have high recall (few missed calls) but low precision (many wrong calls). For example, calling a weather function when not asked wastes resources.

If the model is too cautious, it may have high precision (calls are mostly correct) but low recall (misses some needed calls). For example, not calling a calculator function when math is requested leads to wrong answers.

Good function calling balances precision and recall so the model calls the right functions when needed without many mistakes.

What good vs bad metric values look like

• Good: Precision and recall both above 0.85 means the model calls functions correctly most of the time and rarely misses needed calls.
• Bad: Precision below 0.5 means many wrong function calls, causing confusion or errors.
• Bad: Recall below 0.5 means the model misses many needed function calls, leading to incomplete or wrong answers.
• Accuracy alone can be misleading if most inputs do not require function calls (high TN), so precision and recall are more informative.

Common pitfalls in metrics for function calling

• Accuracy paradox: High accuracy can happen if the model rarely calls functions, but this misses many needed calls (low recall).
• Data leakage: If test data includes exact prompts seen in training, metrics may be unrealistically high.
• Overfitting: Model may memorize function calls for training prompts but fail on new inputs, causing poor generalization.
• Ignoring context: Metrics must consider if the function call was appropriate for the input context, not just if a function was called.

Self-check question

Your model has 98% accuracy but only 12% recall on needed function calls. Is it good for production?

Answer: No. The high accuracy likely comes from many true negatives (no function call needed). But 12% recall means the model misses 88% of needed function calls, so it will often fail to perform required actions. This is not good for production.

Key Result

Precision and recall are key to evaluate function calling accuracy and completeness in LLMs.