Agents and tools
Agent state and memory
Split conversation, artifacts, working memory and run control. Persist after each step. Reload a hot working set, not the whole transcript.
Do not dump everything into the prompt. Separate kinds of state, persist after each significant step and keep a hot working set for the model. The full raw transcript belongs in cold storage for debugging.
Concept
Before you start: compaction shrinks what the model sees. The state store is what the runtime owns so a retry does not lose paid model work.
The big idea
| Kind | Examples | Store |
|---|---|---|
| Conversation | Messages, tool results | Database or object store; compact before reloading |
| Artifacts | Diffs, reports, downloaded files | Blob storage with URIs in state |
| Working memory | Todo list, hypotheses | Structured rows the agent updates via tools |
| Long-term memory | User preferences, prior tickets | Retrieval (RAG) or keyed profiles |
| Run control | Budgets, approvals, step count | Durable workflow state or Redis with TTLs |
Version the schema of stored messages. Prompt changes should not make old runs unreadable.
Anthropic's guidance for agents applies here: the runtime should help compact, offload and reload context rather than blindly appending forever.
A practical compaction policy: after every N tool calls, summarize older tool results into a short "what we know" note, keep the last K raw results, and always keep the user goal plus todo list verbatim. When an agent creates a patch file or a PDF, put it in object storage and pass a URI. Models are bad at being filesystems. Your blob store is good at it.
Walking through a crash
The agent has finished two model calls ($0.14) and written a draft report to object storage. The worker dies.
If state was only in process memory, you replay from scratch and pay again. If you persisted after each step, the workflow reloads goal, todos, artifact_uri and budget.used = 0.14. The next activity synthesizes from the stored report. The customer is not billed twice for the same research.
Common mistakes
- The prompt as the only store. A restart loses the run. Compaction loses the goal.
- Reloading the full raw transcript on every turn. Keep it cold. Reload the working set.
- Artifacts as giant strings in the message list. Store a URI.
Check yourself
- Where should a 40-page PDF the agent generated live? Object storage. State holds the URI.
- What must survive compaction? The user goal, the todo list and recent errors, verbatim.
Key takeaways
- Split conversation, artifacts, working memory and control state.
- Persist after each significant step so retries do not lose paid work.
- Hot working set for the model. Cold archive for debugging.
- Version the message schema so old runs stay readable.
Further learning
Who is allowed to call each tool is agent permissions.
Related
Agent infrastructure
Running agents for real: sandboxes, durable execution, state, permissions and long-running tasks.
Compaction and just-in-time retrieval
Summarize old turns, trim tool dumps and re-fetch full payloads only when the agent can navigate to them.
Retrieval-augmented generation
The core RAG pipeline: ingest, chunk, embed, retrieve, generate with citations.