Tool use and function calling

Tool use lets a model ask your code to do something it cannot do alone: look up live data, run a calculation, create a ticket. The model never executes anything itself. It emits a structured request. Your runtime decides whether to run it, enforces permissions, and returns the result as the next message.

The LLM is a planner and argument generator. Your code is the trusted executor. Get the interface right and the model becomes useful. Get it wrong and you get wrong tools, wrong arguments, expensive loops, and side effects you did not intend.

Without tools, an LLM is a closed-book essay writer: fluent, outdated on your private data, and unable to take actions. With tools, it becomes the control plane for APIs you already trust — if you treat schemas, errors and permissions like a public API consumed by a stranger.

Concepts in this section

Read them in order. Each one builds on the one before.

When many apps need the same tools, package them as an MCP server. When the model must choose sequences of tools freely, you have crossed into agents — same loop, stronger termination and observability.

A rollout that saves pain

1. Ship read-only tools behind the chat product. Measure selection quality.
2. Add writes with approval and idempotency. Measure double-submit rate (should be ~0).
3. Tighten output shapes until average tool-result tokens stop growing.
4. Only then open the loop into a fuller agent with higher step caps.

At work you will usually own three artifacts: the tool schemas (versioned like API contracts), the executor (auth, idempotency, logging), and the eval set. Frameworks help with plumbing. They do not invent good tools for you.

Walk a whiteboard through the loop, then explain why you would confirm a refund, make create_refund idempotent, and return compact JSON. That shows you understand the trust boundary, not just the SDK.

Common mistakes

• Overlapping names (get_customer and lookup_user) so the model flips between them.
• Huge tool results that blow the context window.
• Writes without an idempotency key, so a timeout double-pays.
• Trusting a user_id argument instead of the session.
• Scoring only the final sentence after a dangerous tool already fired.

Key takeaways

• The model proposes calls. Your code executes them under auth, budgets and policy.
• Tool descriptions are prompts. Write when, when-not, formats and limits.
• Prefer a small set of high-level, paginated, token-efficient tools with recoverable errors.
• Make writes idempotent. Require approval for destructive actions. Never trust tool results as instructions.
• Eval tool selection, arguments and error recovery — not only final answers.

Further learning

Writing effective tools for agents (Anthropic)Writing effective tools for AI agents—using AI agentsanthropic.com Toolformer (Schick et al.)Language models (LMs) exhibit remarkable abilities to solve new tasks from just a few examples or textual instructions, especially at scale. They also, paradoxically, struggle with basic functionality, such as arithme…arXiv.org OpenAI function calling guideLearn how function calling enables large language models to connect to external data and systems.OpenAI Developers

Start with the tool loop.