Error analysis and failure taxonomies

Error analysis means reading real requests and writing down what went wrong. You group those notes into a few recurring categories. That set of categories is a failure taxonomy, and it becomes the list of evals you write.

Why not start with a metric?

A dashboard score like "helpfulness 7/10" does not say what broke. A check for "refund tool called without an order ID" does. Start from the failures you see, because they point to a fix.

The big idea

Error analysis has three steps.

1. Sample traces. Aim for about 100 to start. Use more if the product has distinct workflows, such as support, search and writing.
2. Write one note per trace. Describe what the user would see go wrong: a wrong fact, a refusal that should not happen, a tool that was never called. Write "ok" when nothing went wrong. Do not guess the root cause yet.
3. Cluster the notes. Group them into a handful of categories and count each one.

Domain experts beat engineers at step 2 when the product is specialized, such as legal, medical or support policy.

Walking through a real count

Say you label 80 support-agent traces and get these counts.

Sixty of 80 traces failed. Retrieval misses are the biggest group, at 22 of 80, or about 28%. Fixing retrieval will move more users than polishing the tone of the system prompt.

Each category also becomes an eval. You score each one on its own, so a gain in tone cannot hide a drop in tool calling.

From failures to test cases

Once you know the failure modes, build cases that probe them. Four sources work well.

• Real traffic. The best source. Sample across workflows and include hard cases, such as new users or long threads.
• Production failures. Turn every real incident or thumbs-down into a case within a day.
• Synthetic inputs. Ask a strong model to invent inputs for a category. Humans must check the expected answers.
• Edge cases. Empty input, a huge paste, prompt-injection strings and conflicting instructions.

Start small. Twenty cases that catch your top failure beat five hundred vague ones. Keep a held-out slice that you do not look at while you tune prompts.

Common mistakes

• Jumping to the root cause. "Wrong chunk retrieved" is a cause. Write the user-visible failure first, then the cause.
• Writing judges before the taxonomy. A judge that asks "was this helpful?" agrees with nothing and teaches nothing.
• Reading too few traces. Five traces show you anecdotes. Counts need a sample.

Check yourself

1. Why read traces before choosing metrics? Because the failures you see decide which checks are worth writing.
2. In the table above, what should you fix first? Retrieval misses. They are the largest group, at 22 of 80.

Key takeaways

• Error analysis means reading real traces and writing down what the user saw go wrong.
• Cluster the notes into a few categories and count them.
• Each category becomes one eval, scored on its own.
• Build cases from real traffic and real incidents first.

Further learning

Next, see how to grade each case in eval graders. For the whole loop, return to evaluating LLM apps.