Idempotency keys

Some operations can't be redesigned to be safe to repeat. "Create a payment" is one. For those, the client attaches a unique idempotency key to the logical operation and sends the same key with every attempt. The server remembers which keys it has seen and what it answered. Stripe popularized this with an Idempotency-Key HTTP header, and the IETF HTTPAPI working group has a draft to standardize it.

The client side

The client makes the key once per logical operation, usually a UUIDv4, and saves it before the first attempt. If the user taps "Pay" and the app retries three times, all three attempts carry the same key. If the user later buys something else, that's a new operation and a new key.

Generating a fresh key for each attempt is the most common bug. It defeats the entire mechanism.

The server side

The server stores one row per key, scoped to the account so two customers can't collide. The row holds a hash of the request, the status, and the final response.

CREATE TABLE idempotency_keys (
  account_id     bigint      NOT NULL,
  key            text        NOT NULL,
  request_hash   bytea       NOT NULL,
  status         text        NOT NULL,         -- 'started' or 'completed'
  response_code  int,
  response_body  jsonb,
  locked_until   timestamptz,
  PRIMARY KEY (account_id, key)
);

The handler then follows a fixed order. This is a sketch: db, reply and cardNetwork stand in for your own helpers.

async function handleCharge(req: ChargeRequest, accountId: number, key: string) {
  const hash = sha256(canonicalJson(req));
  // 1. Claim the key. The primary key makes concurrent duplicates collide here.
  const row = await claimKey(db, accountId, key, hash); // inserted, or the existing row
  if (!row.inserted) {
    if (!row.request_hash.equals(hash)) return reply(422, "Key reused with different parameters");
    if (row.status === "completed") return reply(row.response_code, row.response_body);
    if (row.locked_until > new Date()) return reply(409, "Request with this key is in progress");
    // The lock expired: a previous attempt crashed. Fall through and resume.
  }
  // 2. Do the work. Pass a derived key downstream so the card network deduplicates too.
  const charge = await cardNetwork.charge(req, { idempotencyKey: `${accountId}:${key}` });
  // 3. Record the outcome, so every retry gets the same answer.
  await markCompleted(db, accountId, key, 201, charge);
  return reply(201, charge);
}

Each step handles a specific failure:

• Request hash. If a client reuses a key for a $99 charge after using it for $49, that is a client bug. Reject it loudly with 422.
• Stored response. A retry gets the same answer as the original, including the charge ID.
• Concurrency. Two attempts that arrive together can't both insert. The second sees started and gets 409, telling the client to wait and retry.
• Crash recovery. If the server dies between steps 1 and 3, the lease expires and a later retry takes over. This is safe only because step 2 passes a key downstream. The card network returns the original charge rather than creating a second one.
• Expiry. Keys can't live forever. Stripe prunes keys after at least 24 hours. A retry after that is treated as new, so pick a window longer than your clients' retry horizon.

Common mistakes

• A new key per attempt. Retries then look like new operations.
• Not hashing the parameters. The same key with a different body silently returns the old result.
• Not passing the key downstream. Your server can be safe while the provider charges twice.
• Expiring keys too soon. A key that expires before the client stops retrying lets duplicates through.

Check yourself

1. A client reuses a key from a $49 charge for a $99 charge. What should the server do? Reject it with 422. The same key with different parameters is a client bug, and returning the $49 result would be silently wrong.

2. The server crashes after the card network charged but before it saved the result. What protects the retry? The lease lets the retry take over, and the derived key sent downstream makes the card network return the original charge.

Key takeaways

• Use one key per logical operation, saved before the first attempt.
• The server claims the key atomically, checks a request hash, and stores the response for replay.
• Concurrent duplicates get 409 while the first attempt runs.
• Pass a derived key to every downstream system that has side effects.

Further learning

Stripe's own write-up covers the design decisions behind the header.

Stripe: Designing robust and predictable APIs with idempotencyOnline payment processing for internet businesses. Stripe is a suite of payment APIs that powers commerce for businesses of all sizes.stripe.com

Implementing Stripe-like idempotency keys in Postgres (Brandur Leach)Building resilient services by identifying foreign state mutations and grouping local changes into restartable atomic phases so that every request can be driven to completion.brandur.org

Next, see how consumers apply the same idea to incoming messages in deduplicating in message consumers.