tokenade
GitHubenfr
Install

Tool Calling (Function Calling)

Cite this page

What is tool calling?

Tool calling (also called function calling) is the mechanism by which a large language model, instead of just emitting text, emits a structured request to invoke one of the functions you've described to it — read_file, search_code, run_tests, whatever. It then waits for your runtime to execute that request, and continues reasoning with the result fed back in. The model never runs anything itself; it produces a JSON-shaped intent, your harness does the work, and the output returns as another message in the conversation.

That round-trip is the whole trick. You hand the model a list of tool schemas (name, description, JSON-schema arguments) up front. On each turn it can answer in plain text or pick a tool and fill in the arguments. Your code runs it, appends the result, and the loop continues until the model decides it's done. Every modern coding agent — Claude Code, Cursor, Codex, Copilot, Windsurf — is, underneath, a tool-calling loop wrapped around a file system and a shell.

Why tool calling matters in 2026

It matters because tool calling is what turned chatbots into agents. A model that can only talk is a very expensive autocomplete; a model that can call grep, open a file, and run your test suite can actually close a loop. This is the substrate of agentic coding and the reason MCP exists — MCP is essentially a standard way to ship tool definitions to any agent that speaks it.

Here's the part nobody puts on the marketing page: tool calling is also where your token bill quietly explodes. Three things stack up.

First, the model has to know which tools exist. Agents used to put every tool schema in the prompt on each turn; load a dozen MCP servers eagerly and you'd spent thousands of tokens before the model read a line of your code. Modern agents now defer tool loading by default (Claude Code's tool search sends only tool names and loads a schema on demand), but setups that load tools upfront still pay that on every turn.

Second, tool results get pasted back verbatim. A coding agent that calls a "list directory" or "read file" tool will cheerfully drop the raw, unfiltered output into its context window, and you pay for all of it. Then you re-pay for it every subsequent turn it stays in context.

Third, the loop is multi-turn by design, so that bloat compounds.

I spend a lot of my time on exactly this. The fixes that actually move the needle are unglamorous: make sure your agent defers tool definitions (most now do by default), and run output filtering on tool results so a 600-line directory dump becomes the eight lines the model needed. That's a big slice of what Tokenade does automatically: output filtering on shell and file results, compaction of MCP server output (with dedicated handling for GitHub, Playwright, Postgres and other popular servers), and semantic code search so the agent retrieves the right three functions instead of reading thirty files. A local dashboard (tokenade dashboard) shows the savings day by day, broken down by agent, project and tool.

It sits in front of Claude Code, Cursor, Codex, Copilot and the rest, compaction runs entirely on your own machine, and it's free and open source (MIT/Apache-2.0). If your agent loop feels expensive, the tool-calling layer is the first place I'd look — see how to reduce AI coding agent token usage.

When NOT to use it

  • One-shot text generation. If you just want a summary, a translation, or prose, don't wire up tools. The schemas cost tokens every turn and you gain nothing; plain completion is cheaper and simpler.
  • When a deterministic call would do. If your code already knows it must read config.json, just read it. Routing that decision through the model adds a network round-trip, latency, and tokens to "decide" something that wasn't actually a decision.
  • Small models with many tools. Define twenty near-identical tools and a small model will pick the wrong one or hallucinate arguments. Fewer, well-described tools beat a sprawling menu — and keep the schemas out of context until they're needed.

See also