Function Calling (Tool Use)
: A capability of modern LLMs that allows them to generate structured requests to invoke external functions or APIs. Instead of returning text, the model outputs a JSON-structured function call with name and arguments, which the application executes and returns results for the model to process. This enables AI to interact with databases, APIs, and external services.
Why It Matters for AI-Coded Apps
Function calling turns LLMs from text generators into action-taking agents. Each function the AI can call is a potential security boundary. If an AI can call a database query function, prompt injection could manipulate the AI into extracting sensitive data. Proper function design with authorization checks is essential.
Real-World Example
An AI assistant can call search_users(query) and delete_user(id). A prompt injection attack might trick the AI into calling delete_user when it should only call search_users. Defense: implement authorization checks in the function implementation itself, not in the AI’s instructions, since instructions can be overridden.
How to Detect and Prevent It
Implement authorization in function handlers, not in AI instructions. Validate all function arguments server-side. Use allowlists for which functions the AI can call based on user permissions. Log all function calls for audit. Rate limit function execution. Never expose destructive functions without human confirmation.
Frequently Asked Questions
Can function calling be exploited through prompt injection?
Yes. If an attacker can influence the AI’s context (through user input, retrieved documents, or injected content), they may manipulate which functions the AI calls and with what arguments. This is why authorization must be enforced in the function implementation, not in the AI’s system prompt.
What is the difference between function calling and API calls?
Function calling is the AI model’s ability to generate structured requests in a specific format. API calls are the actual HTTP requests to external services. Function calling generates the intent (which function, what arguments); the application code executes the actual API call. The distinction matters for security because validation should happen between these steps.
How do I design secure functions for AI?
Each function should: validate all inputs independently of AI instructions, check user authorization, implement rate limiting, log calls for audit, return minimal data (no sensitive fields), and handle errors without exposing internal details. Design functions as if the AI cannot be trusted – because through prompt injection, it cannot.
Scan your app for security issues automatically
Vibe Eval checks for 200+ vulnerabilities in AI-generated code.
Try Vibe Eval