Prompting and context
Structured outputs
Getting reliable JSON: schemas, constrained decoding, validation and repair.
Downstream code does not want a paragraph. It wants a typed object: an enum category, a nullable invoice ID, a list of line items with exact amounts. If you ask a chat model for "JSON please" and call JSON.parse in a loop, you will spend your time on trailing commas, markdown fences, missing fields and renamed keys.
Structured outputs are the techniques that make a model's response checkable by a program. The goal is not prettier text. It is a contract between the model and your code.
Concepts in this section
Read them in order. Each one builds on the one before.
| Step | Concept | What it answers |
|---|---|---|
| 1 | Constrained decoding | How can the sampler be forced to stay valid? |
| 2 | Schema design | How should the shape be written so the model fills it well? |
| 3 | Validation and repair | What happens when the object is valid but wrong, or cannot be produced? |
| 4 | Field-level evals | How do you measure the fields that move money? |
The approaches, weakest to strongest
| Approach | Guarantee | Typical use |
|---|---|---|
| Prompt-only ("respond with JSON") | None | Prototypes |
| JSON mode | Valid JSON syntax, not your schema | Loose objects |
| Schema-constrained decoding | Output matches a JSON Schema | Production extraction |
| Tool call with a strict schema | Same guarantee, framed as a tool call | Agents and side effects |
Prompt-only JSON fails more often than you expect. Even 1% invalid JSON at 100,000 requests a day is 1,000 broken calls. JSON mode fixes the syntax but not the keys, types or enums. Only schema-constrained decoding and strict tools fix the shape. Even then, you still validate.
Tools fit extraction well. Define a tool such as record_invoice whose parameters are the fields you want. The model "calls" it, and your code reads the arguments. Nothing runs.
Common mistakes
- Trusting the shape means trusting the values. A valid object can still hold a wrong invoice ID.
- Assuming every provider supports the same schema features. Check the subset before you rely on a feature.
- Using JSON mode and skipping validation. Syntax is only the first gate.
Key takeaways
- Structure exists so deterministic code can trust model output.
- Climb the ladder: prompt-only JSON, then JSON mode, then schema constraints or strict tools.
- Validate every output in code, even when the provider guarantees the shape.
- Score the fields, not just whether the JSON parses.
Further learning
OpenAI's guide covers the API surface and which schema features work under strict mode.
OpenAI: Structured OutputsUnderstand how to ensure model responses follow specific JSON Schema you define.OpenAI Developers
Outlines is a common open-source library for structured generation on self-hosted models.
Outlines (structured generation library)Structured Outputs. Contribute to dottxt-ai/outlines development by creating an account on GitHub.GitHubStart with the first concept page: constrained decoding.
Related
Prompting
Clear instructions, examples, structure and chain-of-thought: prompting as specification writing.
Tool use and function calling
Letting models call your code: tool schemas, the tool loop, and designing tools models use well.
Serving engines: vLLM, SGLang and TensorRT-LLM
How modern engines are built, how they differ, and which knobs move TTFT, TPOT and tokens per GPU-second.