What "tool use" actually is: how an LLM calling a function really works
"The AI searched the web" or "the AI ran the code" makes it sound like the model reached out and did something. It didn't — a language model can't execute anything, open a connection, or touch a file system. Tool use is a specific, more mechanical pattern than that framing suggests, and understanding the actual request/response shape explains both why it works and where it can go wrong.
The model never executes anything — it only describes what to call
A request that includes tool use provides the model with a list of available tools: each one's name, a description of what it does, and a schema for what parameters it accepts. When the model decides a tool is needed to answer the request, its response isn't code execution — it's a structured block naming which tool to call and with what arguments, shaped to match that tool's declared schema. That's the entire scope of what the model does: produce a well-formed request to call something, not perform the call itself.
The actual round trip
Tool use is a multi-step conversation, not a single call:
- Request: your code sends the conversation plus the list of available tools.
- Model responds with a tool-use request: instead of (or alongside) plain text, the response contains a structured block — tool name, arguments — and the response stops there, signaling it's waiting on that tool's result before continuing.
- Your code executes the tool — not the model, not the API provider. If the declared tool is "search the web" or "run this SQL query," your application code is what actually calls the search API or the database, using the exact arguments the model provided.
- Your code sends the result back as a new message in the same conversation, tagged as the result of that specific tool call.
- The model continues, now with that real result in context, and either produces a final answer or requests another tool call — the loop repeats until the model has enough to respond.
This is exactly the mechanism behind coding agents and assistants that appear to "use" tools like a file editor, a terminal, or a web browser: each apparent action is this same propose-then-execute-then-report loop, run by the surrounding application, not a capability built into the model doing the generating.
Why this design, not direct execution
Keeping execution entirely outside the model is a deliberate boundary, not a limitation
waiting to be removed. It means the calling application controls exactly what's actually
allowed to run — which tools exist, what arguments are valid, whether a given call needs
human approval first — rather than a model outputting text that gets blindly executed. A
model can propose calling delete_all_files() with no permission at all to actually
invoke it; what happens next is entirely up to the code that receives that proposal.
Where this explains common confusion
- "Why didn't it actually check the current date/run the calculation?" — usually means no tool was available for that specific capability, so the model answered from its training data (or declined to) instead of requesting a tool call it didn't have access to.
- "It called the tool with the wrong arguments" — the model's output has to match the tool's declared schema exactly; a vague or under-specified schema description makes it more likely the model guesses at a parameter shape rather than getting it right.
- "It's stuck in a loop calling the same tool repeatedly" — the model only knows what your code sends back as the tool's result; if that result doesn't actually change anything the model can act on, there's nothing in the loop stopping it from trying again.
Sources: Anthropic's Claude API documentation on tool use (the request/response shape, tool schemas, and the multi-turn tool-result pattern); the general agentic tool-calling pattern used consistently across current LLM provider APIs (OpenAI, Google, Anthropic all implement the same propose/execute/report shape with provider-specific request formats).