Tool calling (also called function calling) is the mechanism by which a large language model, instead of just emitting text, emits a structured request to invoke one of the functions you've described to it — read_file, search_code, run_tests, whatever — waits for your runtime to execute it, and then continues reasoning with the result fed back in. The model never runs anything itself; it produces a JSON-shaped intent, your harness does the work, and the output returns as another message in the conversation.That round-trip is the whole trick. You hand the model a list of tool schemas (name, description, JSON-schema arguments) up front. On each turn it can answer in plain text or pick a tool and fill in the arguments. Your code runs it, appends the result, and the loop continues until the model decides it's done. Every modern coding agent — Claude Code, Cursor, Codex, Copilot, Windsurf — is, underneath, a tool-calling loop wrapped around a file system and a shell.
Why tool calling matters in 2026
It matters because tool calling is what turned chatbots into agents. A model that can only talk is a very expensive autocomplete; a model that can call grep, open a file, and run your test suite can actually close a loop. This is the substrate of agentic coding and the reason MCP exists — MCP is essentially a standard way to ship tool definitions to any agent that speaks it.Here's the part nobody puts on the marketing page: tool calling is also where your token bill quietly explodes. Three things stack up. First, every tool schema lives in the prompt — on each turn, for the whole session. Load a dozen MCP servers eagerly and you've spent thousands of tokens before the model reads a single line of your code. Second, tool results get pasted back verbatim. A coding agent that calls a "list directory" or "read file" tool will cheerfully drop the raw, unfiltered output into its context window, and you pay for all of it — then re-pay for it every subsequent turn it stays in context. Third, the loop is multi-turn by design, so that bloat compounds.I spend a lot of my time on exactly this. The fixes that actually move the needle are unglamorous: lazy-load tool definitions so an agent only pays for the MCP servers it's using this turn, and run output filtering on tool results so a 600-line directory dump becomes the eight lines the model needed. That's a big slice of what Tokenade does automatically — lazy MCP loading, output filtering, semantic code search so the agent retrieves the right three functions instead of reading thirty files, plus a dashboard that shows you the savings per session. It sits in front of Claude Code, Cursor, Codex, Copilot and the rest, it's source-available (MIT), and the free tier covers ~10M tokens a month before Pro at $19.90/mo. If your agent loop feels expensive, the tool-calling layer is the first place I'd look — see how to reduce AI coding agent token usage.
When NOT to use it
One-shot text generation. If you just want a summary, a translation, or prose, don't wire up tools. The schemas cost tokens every turn and you gain nothing; plain completion is cheaper and simpler.
When a deterministic call would do. If your code already knows it must read config.json, just read it. Routing that decision through the model adds a network round-trip, latency, and tokens to "decide" something that wasn't actually a decision.
Small models with many tools. Define twenty near-identical tools and a small model will pick the wrong one or hallucinate arguments. Fewer, well-described tools beat a sprawling menu — and keep the schemas out of context until they're needed.