Production and case studies
Case study: customer support agent
End-to-end: RAG over help docs, account tools, escalation to humans, evals and cost.
Customer support is the default proving ground for LLM agents: high volume, messy language, a knowledge base you already have, and a clear economic story. It is also where unsafe tool use and confident wrong policies hurt real people. This case study puts RAG, tool use, evals, latency and cost and AI product UX in one architecture with explicit safety boundaries.
The product target: answer ~60% of in-scope tickets autonomously, never take unsafe account actions, and hand off gracefully. "60% autonomous" means resolved without a human, with a published definition (confirm / no reopen in 48h / CSAT) — not "the model replied." In-scope is an allowlist of intents. Medical advice and legal threats escalate even if the model is willing.
Concepts in this section
Read them in order. Each one builds on the one before.
| Step | Concept | What it answers |
|---|---|---|
| 1 | Support requirements | What 60% means, and what "resolved" counts |
| 2 | Support architecture | Router, read-only RAG, tool pocket, handoff |
| 3 | Support tools and permissions | Session-scoped reads, propose/confirm, injection |
| 4 | Support evals and cost | Ticket-labeled gates and the $1.62M sketch |
| 5 | Support rollout | Shadow → FAQ → reads → writes, plus the desktop |
Architecture in one picture
A single mega-agent is simpler to demo and harder to secure. Split FAQ from account actions so a prompt-injection in a ticket cannot easily reach refund tools.
Help content needs versions, updated_at and citations. When retrieval is weak, refuse and escalate. Bind identity from the session. propose_refund does not move money; apply_refund is human-only in v1.
On a whiteboard you should be able to draw that diagram, name the safety properties of each tool, and describe the eval slices that gate a prompt change. Session-bound IDs, propose-then-confirm, and a safety eval set are how you stop arbitrary refunds. Shadow-mode rollout and CSAT next to deflection are how you keep the $2.4M sketch honest. Feature-flag writes separately from the bot so a red safety slice does not require an app deploy.
Common mistakes
- Counting bot-touched tickets as contained. Finance cares about resolved-without-human.
- One unrestricted agent with refund tools on day one.
- Improvising policy when retrieval is weak.
- A cost model without CSAT and reopen. Repeat contacts eat the savings.
- An untested kill switch.
Key takeaways
- Spec autonomy, safety, handoff, freshness, latency and cost up front.
- Prefer router + RAG + bounded tool agent over one unrestricted agent.
- Tools: session-scoped reads, propose/confirm for money, humans for irreversible v1.
- Evals from historical tickets with policy and never-call cases.
- Roll out shadow → FAQ → reads → confirmed writes. Keep a kill switch.
Further learning
Patterns for Building LLM-based Systems & Products (Eugene Yan)Evals, RAG, fine-tuning, caching, guardrails, defensive UX, and collecting user feedback.eugeneyan.com
Your AI Product Needs Evals (Hamel Husain)How to construct domain-specific LLM evaluation systems.Hamel’s Blog
Start with support requirements.
Related
Retrieval-augmented generation
The core RAG pipeline: ingest, chunk, embed, retrieve, generate with citations.
Tool use and function calling
Letting models call your code: tool schemas, the tool loop, and designing tools models use well.
Evaluating LLM apps
A fixed set of inputs plus a way to score outputs, so you can tell whether a change helped.