Case study: payment system

Payments are where "at-least-once delivery" becomes double charges. A checkout click, a flaky mobile network, a load balancer retry and a webhook replay can each look like a new request. The business needs something close to exactly-once money movement, built on unreliable networks and at-least-once messaging.

You rarely talk to card networks directly. A payment service provider (PSP), such as Stripe or Adyen, authorizes and settles with the banks. Your job is to model payment intents, stay idempotent, keep a trustworthy ledger, and reconcile when your records disagree with the PSP's.

Requirements

• Charge cards and wallets through a PSP. If you run a marketplace, also support refunds and seller payouts.
• Survive retries and timeouts without double-charging, and without silently dropping a successful charge.
• Keep a full audit trail: who moved what, when and why.
• Keep strongly consistent balances, using integer minor units such as cents. Never use floats.
• Reduce PCI scope, so card data stays with the PSP.

A first design leaves out full fraud models, multi-currency FX engines and bank licenses.

The main flow

A PSP exposes a payment intent: a record of one attempt to collect money, with a status that moves through a state machine. Your service creates the intent, confirms it, and learns the outcome, either now or later through a webhook. The details are in payment intents and states.

checkout → create intent → confirm (authorize) → succeeded | requires_action (3DS) | failed
                                                      ↓
                                 webhook payment_intent.succeeded → post ledger

Concepts in this topic

Read them in order. Each one builds on the last.

Keep card data out of your systems

Card numbers that touch your servers bring you into PCI DSS scope. Reduce that scope:

• Use the PSP's hosted fields, such as Stripe Elements or Adyen Drop-in, so card data goes from the browser straight to the PSP.
• Store only tokens, payment method IDs and customer IDs.
• Keep webhook and server APIs free of card numbers and CVV codes.
• Put API keys in a secrets vault, and keep card data out of logs.

Consistency and money

Use a relational primary, such as Postgres, for the accounts you debit. Use unique constraints for idempotency keys and journal IDs. Partition by merchant or customer, so one busy merchant does not lock the rest. Keep each journal's entries in one shard, so they commit together. Transfers that cross shards need a saga or an outbox, covered in distributed transactions.

Eventual consistency is fine for displays, such as a dashboard or an approximate available balance. Authorization of a payout must read strongly consistent balances.

Common mistakes

• A new idempotency key on each retry click. The retry then creates a second charge.
• Trusting the redirect alone. The browser may never return. The webhook is the source of truth.
• Marking an order paid before the ledger is posted. The books and the order can disagree.
• Floats for money. 0.1 + 0.2 is not how cents work.

Key takeaways

• Model payments as intents with explicit states, and treat webhooks as the source of completion.
• Use idempotency keys on every mutating step, and reuse them on timeout retries.
• Keep immutable double-entry journals, and derive balances from them.
• Reconcile against PSP settlements on a schedule. Unexplained gaps are incidents.
• Store money as integer minor units, and tokenize cards to shrink PCI scope.

Further learning

Stripe's post on idempotency is the canonical mental model for safe retries:

Stripe: Designing robust and predictable APIs with idempotencyOnline payment processing for internet businesses. Stripe is a suite of payment APIs that powers commerce for businesses of all sizes.stripe.com

Square's Books service is a public write-up of an immutable double-entry store:

Square: Books, an immutable double-entry accounting database serviceTracking financial transactions at scaleSquare Corner Blog

Start with payment intents and states to see the flow in detail.