The tool loop

Tool use (also called function calling) lets a model ask your code to do something it cannot do alone: look up live data, query a database, run a calculation, create a ticket. The model never executes anything itself. It emits a structured request. Your runtime decides whether to run it, enforces permissions, and returns the result as the next message.

That split is the whole design. The LLM is a planner and argument generator. Your code is the trusted executor. Without tools, an LLM is a closed-book essay writer: fluent, outdated on your private data, and unable to take actions. Treat schemas, errors and permissions like a public API consumed by a stranger. The model is capable, non-deterministic and not entitled to your credentials.

How the loop works

Every major provider follows the same shape:

1. You send the conversation plus tool definitions: name, description, and a JSON Schema for parameters.
2. The model replies with either normal text or one or more tool calls (name + arguments + a call id).
3. Your code runs each call, or refuses it, and appends tool results keyed by those ids.
4. You call the model again with the updated transcript. Repeat until it answers without calling a tool, or you hit a budget.

Stop reasons matter. On Anthropic, stop_reason === "tool_use" means you must execute and continue. On OpenAI-compatible APIs, finish reason tool_calls means the same. Always bound the loop with a max turn count, token budget, and wall-clock timeout. An unbounded loop is a blank check.

    # Sketch of the loop — same idea on every provider
messages = [{"role": "user", "content": "Where is order ORD-12345?"}]

for turn in range(8):
    response = client.messages.create(model=MODEL, tools=TOOLS, messages=messages)
    messages.append({"role": "assistant", "content": response.content})
    if response.stop_reason != "tool_use":
        break
    results = [run_or_error(block) for block in response.content if block.type == "tool_use"]
    messages.append({"role": "user", "content": results})

Return errors as tool results, not as crashed Python. The model can often recover from "No order ORD-1234 found. IDs look like ORD-12345". It cannot recover from a stack trace that aborted the turn.

Parallel calls, forced choice, streaming

Modern models can emit multiple tool calls in one turn. If the user asks "status of ORD-1 and ORD-2 and the refund policy", the model may call get_order twice and search_help_docs once. Execute independent calls concurrently. Preserve call ids when you return results.

Forcing a tool is a product decision, not a model skill.

With streaming APIs the model may emit tool-call JSON incrementally. Do not execute on partial arguments. Buffer until the call is complete, validate, then run. Showing "Looking up order…" when the name is known is UX, not permission to query with half an id.

Common mistakes

• No max turn count. The model can loop until the bill does.
• Crashing on a tool error. Return a structured error the model can recover from.
• Executing a streamed call before the arguments are complete.

Check yourself

1. stop_reason is tool_use. What must you do before you show a final answer? Run or refuse each call, append the results, and call the model again.
2. Why return "not_found" as a tool result instead of raising? The model can fix the id or ask a question. A crash ends the turn.

Key takeaways

• The model proposes calls. Your code executes them under auth, budgets and policy.
• Bound the loop. Return errors as results.
• Parallel calls are normal. Forced tool choice is a product control.
• Never execute partial streamed arguments.

Further learning

Beginner walkthrough of the call → execute → continue loop:

OpenAI function calling guideLearn how function calling enables large language models to connect to external data and systems.OpenAI Developers

Next, how to write the contracts: tool schemas.