Chat messages and roles

Modern chat APIs are stateless. Each call must include the full conversation you want the model to see. "Memory" is your problem: you store turns in a database and resend them. The provider does not remember yesterday's thread unless you are on a specialized assistants product, and even then you should know what is stored where for compliance.

Statelessness is a feature for horizontal scaling and a footgun for cost. Every retry and every follow-up pays to re-send the prefix. Prompt caching and history budgeting exist because of this API shape.

Typical fields

Roles (names vary slightly by vendor):

• system / developer — policy, persona, non-negotiable rules
• user — human or application input
• assistant — prior model outputs you are feeding back
• tool / function — results of tool executions you ran

Message content is not always a plain string. Multimodal APIs use content blocks (text, image URL or base64, documents). Tool results are often separate block types. When you log or replay requests, serialize the structured form. Stringifying carelessly is how image bytes disappear from a repro case.

Walking through a billing call

This is a sketch of the request shape, not a live SDK:

model:  claude-sonnet-4-5        # pinned, not a floating alias
system: You are a billing assistant. Never invent invoice IDs.
messages:
  - role: user
    content: What's the status of invoice INV-2044?
tools:
  - name: get_invoice
    input_schema: { invoice_id: string }   # required

The model cannot look up INV-2044 unless get_invoice runs. The next call must resend the system prompt, the user turn, the assistant tool call, and the tool result. That is the whole conversation, every time.

System prompts deserve version control like code. A one-line policy change can alter refusal rates and tool-call frequency across the product. Put the text in a staged config, record prompt_version on every trace, and promote with the same eval gate you use for model ID changes.

Keep API keys on the server. Browser-exposed keys will be stolen.

Common mistakes

• Floating aliases in production. The snapshot behind latest moves without your consent.
• Dropping tool results from history. The next call then has no invoice, and the model invents one.
• Stringifying multimodal messages. Image bytes vanish from the repro.

Check yourself

1. The user asks a follow-up. Do you send only the new sentence? No. Resend the full conversation you want the model to see.
2. Why pin a model ID instead of a family alias? Aliases move. Your postmortem should name the snapshot that served the answer.

Key takeaways

• Chat APIs are stateless. You own history.
• Roles mark system, user, assistant and tool turns.
• Pin model IDs. Version system prompts.
• Serialize structured content. Do not stringify multimodal messages.

Further learning

Next, streaming and stop reasons. Tool rounds are covered in tool use.