Calibrated trust

Users over-trust fluent systems. Classical software fails loudly: a 500, a validation error, an empty state you designed. LLM products fail plausibly — fluent wrong answers, confident omissions, actions the user did not intend. UX has to push toward calibrated trust, not maximum wow.

The big idea

• Say what it can and cannot do on first use (onboarding, empty state, tooltips). Not a legal wall of text. A concrete list: "can draft replies from your docs; cannot issue refunds without your click."
• Show provenance for factual claims: citations to docs, ticket ids, quote highlights. Clickable sources beat "AI said so."
• Communicate uncertainty when you have a signal: low retrieval score, conflicting sources, model self-check failed. Prefer "I'm not sure — here is the closest policy" over a confident guess.
• Avoid fake precision. A spinner labeled "thinking deeply…" that always takes three seconds teaches users to ignore status. Show real stages when the agent has them.

Google's People + AI Guidebook frames this as explaining for understanding, not completeness — progressive disclosure of why, not a dump of logits.

Words matter more than in deterministic apps. Prefer "Draft," "Suggest," "Estimate" over "Answer," "Confirm," "Will." Avoid fake confidence meters unless they are calibrated. A random 87% teaches users to ignore numbers.

Streaming text should not look like a committed message until send or accept. Visually distinguish draft from sent.

Walking through first run

First-run experience teaches users whether this is a search box, a junior coworker or an autopilot.

• Start with a concrete job, not a blank chat: "Summarize this thread," "Draft a reply," "Find the refund policy."
• Show an example output with citations and an edit affordance so people learn the recovery path before they need it.
• Name the model's role in UI copy ("Draft assistant — you send") rather than "Your AI coworker who never sleeps."
• Put advanced controls (tone, length, tools) behind a secondary panel. Defaults should be safe.

When capability changes (new tools, higher autonomy), re-onboard. A banner that says what is newly automatic prevents surprise side effects.

Memory features need clear viewing, editing and deletion of stored facts. Hidden memory feels like surveillance. A badge like "Using your tone preferences" with a link to settings makes personalization obvious. Silent personalization that changes legal wording is a liability.

If the model improves, say so briefly ("Better at citing your docs"). If you take away a power (refunds now need approval), announce it before users discover a broken flow.

Common mistakes

• A confident paragraph with no sources on a factual claim.
• Fake progress that completes before the server finishes. Users discount all status UI afterward.
• Anthropomorphic copy that implies a coworker who never errs.

Check yourself

1. Retrieval score is low. What should the UI say? That you are not sure, plus the closest policy, not a fluent guess.
2. Why is a random 87% confidence meter harmful? It is not calibrated, so users learn to ignore every number.

Key takeaways

• Calibrate trust with clear limits, citations and honest uncertainty.
• Onboard with a concrete job, an example output and a named role.
• Prefer "Draft" and "Suggest" over "Answer" and "Will."
• Make memory visible and editable. Announce capability changes.

Further learning

People + AI Guidebook (Google PAIR)A toolkit for teams building human-centered AI products.pair.withgoogle.com

Next, design what happens when the system is wrong in failure UX.