Guide

Transformers and language models

Fewer steps, samplers and distillation

Textbook diffusion needs hundreds of denoising steps. Better samplers, noise schedules and distilled models produce good images in a few steps, which makes interactive products practical.

138 of 187 · 3 min read

Textbook diffusion uses hundreds of denoising steps, and each step is a full pass of a large network. That is far too slow for a product. Faster samplers, better noise schedules and distilled models reach good images in a handful of steps. Most production image APIs run one of these optimized schedules, not the textbook sampler.

Faster samplers

The original DDPM sampler takes a small step for every training step. DDIM and related solvers make the reverse process partly deterministic, so you can skip steps. The model is the same, but the sampler visits fewer points along the path.

The arithmetic is simple. Going from 1,000 steps to 50 cuts the number of denoiser calls by a factor of 20. The image is not identical, and quality depends on the sampler, so compare samples at each step count.

Noise schedules

The schedule β_t sets how fast the signal disappears. Linear schedules were an early default. Cosine schedules keep more information in the early steps, and they often train better. If mid-step samples look overly noisy or overly smooth, check the schedule and the prediction target first, before you change the network.

Prediction targets

The network can predict the noise, the clean image, or a velocity that combines the two. The three are mathematically related, but they behave differently at the extremes of the schedule. Changing the target is one of the first knobs to try when training is unstable.

Distillation

Distillation trains a small, fast student to match a slow teacher. Consistency models and step-distilled students learn to jump across many steps at once. In 1 to 8 steps they give up a little quality, in exchange for interactive latency.

Trade-off

Fewer steps are not free. Each cut can lose detail, change the look of the images, or make the output less diverse. Re-tune the guidance scale after you change the step count, because the two interact.

Flow matching

Flow matching frames the problem as learning a velocity field along a path from noise to data. Sampling integrates an ODE along that path. It has the same overall picture, iterative refinement, but a different mathematical route, and it often reaches similar quality in fewer steps. It is worth following as of 2025 and 2026.

Common mistakes

  • Cutting steps without re-tuning guidance. The best scale at 50 steps is not the best at 4.
  • Assuming fewer steps is free. Compare samples on your own prompts at each step count.
  • Using the textbook sampler in production. Optimized samplers and distilled models are the usual choice.

Check yourself

  1. If a sampler uses 50 steps instead of 1,000, how many fewer denoiser calls does each image need? Denoiser calls drop by a factor of 20, from 1,000 to 50.
  2. Why does the schedule matter? It sets how fast signal disappears across the steps, so a poor schedule can produce images that are too noisy or too smooth in the middle.

Key takeaways

  • Better samplers skip steps, and the cost falls in proportion to the step count.
  • The noise schedule and the prediction target are the first things to check if mid-step samples look wrong.
  • Distilled models reach good images in a few steps, at some cost in quality.
  • Re-tune guidance whenever you change the number of steps.

Further learning

Step count and guidance both multiply the cost of each image, and the next page shows how to add it up: diffusion inference cost.

Generative models beyond text

How images, video and speech are generated. Compares the main model families, builds diffusion from first principles, and explains why diffusion inference costs so differently from chat.

Latent diffusion, guidance and conditioning

Latent diffusion runs the denoising in a small compressed space, and classifier-free guidance controls how closely the output follows the prompt. Conditioning covers the other inputs a denoiser can take.

Diffusion inference cost

Serving an image model is a different systems problem from serving an LLM. Each image runs the denoiser many times over the whole latent, so the cost is steps times denoiser cost, not one pass per token.