Managed databases and messaging

Running a database well means backups, failover, patching and replication. Managed services take that work, so most teams choose them first. The choice that matters is which managed service fits the access pattern. Each one has a limit that shapes the design.

Relational: RDS versus Aurora

RDS runs stock Postgres or MySQL on an instance with an EBS volume. Multi-AZ adds a synchronous standby in another zone. Aurora replaces the storage layer with a distributed one. It keeps six copies of your data across three zones, so failover is often under 30 seconds, and up to 15 read replicas share that storage. It costs more, especially for I/O-heavy work.

DynamoDB and hot keys

DynamoDB has no servers to size. Items are at most 400 KB. Each partition handles roughly 3,000 reads and 1,000 writes per second.

Table average: 2,000 writes/s across 10 partitions  -> 200/s each, fine
One hot key:   1,500 writes/s to a single key         -> over one partition's 1,000/s
Result: throttled writes for that key, while the table average looks healthy

That is the classic failure. Model tables around the queries you run, not around entities, and spread hot keys.

Messaging and analytics

Messaging services decouple the parts of a system. The queues and event streaming pages cover the theory.

A common AWS pattern is SNS to SQS fan-out. Publish OrderPlaced to a topic, and billing, email and analytics each have their own queue. A slow consumer then does not block the others.

Streams are sized by throughput. A Kinesis shard accepts 1 MB/s or 1,000 records/s of writes. Ingesting 20 MB/s therefore needs at least 20 MB/s ÷ 1 MB/s = 20 shards, plus headroom for spikes.

For analytics, BigQuery bills by bytes scanned, around $6 per TiB on demand. A query over a 2 TiB table costs about 2 × $6 = $12 each time it runs. Select only the columns you need.

Common mistakes

• Using DynamoDB like a SQL database. Without access patterns, scans and hot keys follow.
• Sizing a stream by average traffic. Spikes and uneven keys need headroom.
• Scanning whole BigQuery tables for one column. You pay for every byte scanned.

Check yourself

1. Why does a hot key throttle DynamoDB even when the table average is fine? Answer: Each partition has a fixed ceiling, so one busy key can exceed its partition's limit while the average stays low.
2. How many Kinesis shards does 20 MB/s of writes need? Answer: At least 20, at 1 MB/s per shard, plus headroom for spikes.

Key takeaways

• Pick the managed database by access pattern. Relational for joins and transactions, key-value for known lookups at scale.
• Aurora's storage is replicated across three zones, which makes failover fast.
• A hot key can throttle a partitioned store even when the average looks fine.
• Size streams by peak throughput, not by the average.

Further learning

Next, decide when to run these services yourself instead: managed vs self-hosted.