Few-shot examples

Few-shot prompting means putting a few input-to-output pairs in the prompt. The model imitates them. This is one of the most useful tools for classification, extraction and matching a house style, especially when the label boundaries are fuzzy, like "is this urgent?"

The big idea

Think of a new support agent's first week. A manual that says "tag urgent tickets" leaves them guessing. Watching three real tickets tagged by a veteran shows what "urgent" means here. Examples carry judgment that a rule cannot easily state.

The analogy fails in one way. A new agent can generalize from a few cases. A model can copy surface features: the exact wording, the length and the sentence shape of the examples.

Walking through it

Say the classifier needs to tell billing tickets from everything else. Pick four examples that cover different situations:

<example>
<input>My card was charged twice for invoice INV-4412</input>
<output>{"category":"payment_failed","invoice_number":"INV-4412","confidence":"high"}</output>
</example>

<example>
<input>Can you resend last month's invoice?</input>
<output>{"category":"invoice_question","invoice_number":null,"confidence":"high"}</output>
</example>

<example>
<input>My app keeps crashing when I log in</input>
<output>{"category":"other","invoice_number":null,"confidence":"high"}</output>
</example>

<example>
<input>I'm not sure if the charge is mine, can someone check?</input>
<output>{"category":"invoice_question","invoice_number":null,"confidence":"low"}</output>
</example>

The four cases cover a clear case, a second clear case, a case that is not billing at all, and an uncertain one. The uncertain example teaches the model that low confidence is a valid answer.

Selection rules

• Use 3 to 5 examples for most tasks. A single example is rarely enough. A long list adds cost without adding much judgment.
• Cover the real spread of inputs: easy cases, hard cases, off-topic cases and near-misses.
• Include at least one "I don't know" example if you want that behavior in production. Otherwise the model never learns to use it.

Common mistakes

• Five near-identical examples. They teach a template. The model copies the phrasing and forces every answer into the same shape.
• Only polite examples. Short, polite examples produce short, polite outputs, even when the live input needs something blunter.
• Unlabeled examples. Without tags, the model may treat an example input as the live ticket.
• Examples used instead of a rule. When the rule is simple, state it once and use examples only for the edges.

Check yourself

• Why is a catalog of 20 near-identical examples a bad idea? It teaches the surface shape of the examples, not the policy behind them, so novel inputs get forced into the same template.
• Why include a low-confidence example? So the model learns that uncertainty is an allowed output, instead of always answering with high confidence.

Key takeaways

• Examples teach judgment that a rule cannot easily state.
• Choose 3 to 5 diverse examples, including edge cases and at least one "unknown."
• Wrap examples in tags so they cannot be mistaken for the live input.
• When examples start to over-constrain, prune them and state the general rule.

Further learning

Next, see when to ask the model to reason before it answers in chain-of-thought.