Webhooks and long-running operations

Some work takes seconds, minutes or hours. Video transcoding, bulk exports and provisioning are examples. Holding an HTTP connection open that long is fragile. Instead, the server accepts the request, returns 202 Accepted, and does the work in the background. The client learns the result in one of two ways: by polling an operation resource, or by receiving a webhook, which is an HTTP call from the server to the client.

The big idea

The async job pattern has three steps.

1. The client sends POST /v1/exports. The server validates it, puts a job on a queue, and returns 202 with an operation such as op_42.
2. The client polls GET /v1/operations/op_42 with backoff, or waits for a webhook.
3. When the job finishes, the operation holds the result or a link to it.

Webhooks flip the reliability problem. Now your server is the client, calling an endpoint you don't control. It may be slow, down or wrong.

Walking through it

A receiver must prove a webhook came from you and wasn't replayed. The sender signs the timestamp and body with a shared secret. The receiver recomputes the signature and compares.

import crypto from "node:crypto"

function sign(body, secret, ts) {
  const v1 = crypto.createHmac("sha256", secret).update(`${ts}.${body}`).digest("hex")
  return `t=${ts},v1=${v1}`
}

function verifyWebhook(rawBody, header, secret, toleranceSec = 300) {
  const parts = Object.fromEntries(header.split(",").map((p) => p.split("=")))
  const ts = Number(parts.t)
  if (Math.abs(Date.now() / 1000 - ts) > toleranceSec) return false // replay protection
  const expected = crypto.createHmac("sha256", secret).update(`${ts}.${rawBody}`).digest("hex")
  const given = Buffer.from(parts.v1 ?? "", "hex")
  const want = Buffer.from(expected, "hex")
  return given.length === want.length && crypto.timingSafeEqual(given, want)
}

const secret = "whsec_demo"
const body = '{"type":"export.completed","id":"evt_1"}'
const now = Math.floor(Date.now() / 1000)
console.log("valid:", verifyWebhook(body, sign(body, secret, now), secret))
console.log("tampered:", verifyWebhook(body + " ", sign(body, secret, now), secret))

Output:

valid: true
tampered: false

The receiver checks that the timestamp is recent, so a captured request can't be replayed later. It compares signatures with timingSafeEqual, which takes the same time whether the first byte or the last byte differs.

Webhook delivery follows four rules that Stripe, GitHub and Shopify all use:

• Retry on failure. Keep retrying non-2xx or timed-out deliveries for hours or days, then alert the customer.
• Include an event ID. Delivery is at least once, so receivers must deduplicate.
• Don't promise order. Send timestamps or versions so receivers can ignore stale events.
• Respond fast. The receiver should queue the event and return 200. Doing the work inline makes it slow and triggers retries.

Common mistakes

• Comparing signatures with ===. A plain comparison can leak timing information. Use a constant-time comparison.
• Skipping the timestamp check. Without it, anyone who captures one valid request can replay it forever.
• Doing heavy work before returning 200. The sender sees a timeout, retries, and the work runs twice.

Check yourself

1. Why does the receiver check the timestamp? It blocks replays. A captured request is rejected after the tolerance window, even though its signature is still valid.
2. Why does the webhook need an event ID? Delivery is at least once, so the same event may arrive more than once. The receiver uses the ID to ignore duplicates.

Key takeaways

• Long work returns 202 and an operation the client can poll.
• Webhooks are signed, retried, deduplicated and unordered.
• Verify the signature in constant time and reject old timestamps.
• Receivers should queue events and return 200 quickly.

Further learning

The idempotency page explains how to make the receiving side safe to run twice.