Concurrency and database connections in serverless

Serverless scales by creating many parallel environments. Each environment that opens a database connection adds to the total. A busy function can use up a database's connection limit within seconds. This is one of the most common serverless outages, so it is worth doing the arithmetic before launch.

The connection arithmetic

peak connections ≈ concurrent warm environments × connections per environment

Take a function with reserved concurrency of 200. Each environment opens a pool of 5 connections.

200 environments × 5 connections = 1,000 potential connections

A small RDS instance with about 100 maximum connections cannot hold that many. The database refuses new connections, and requests start to fail.

The fixes:

• Set the pool size to 1 per environment.
• Lower the reserved concurrency so the total fits.
• Put a connection pooler between them, such as RDS Proxy, PgBouncer or Cloud SQL Auth Proxy.
• Or move the chatty API to Cloud Run or Kubernetes, with a fixed number of replicas that share one pool.

Other limits to design around

• Timeouts. Lambda allows up to 15 minutes. An API Gateway HTTP API often stops at 30 seconds. For synchronous paths, the tighter limit wins. Move long work to a queue.
• Burst quotas. Accounts have regional limits on concurrency. Load test before a launch, and request increases early.
• Ephemeral disk. /tmp is not a durable volume, and its size depends on configuration.
• No sticky sessions. Keep sessions and uploads in Redis, DynamoDB or object storage.
• Environment reuse. A warm environment keeps globals and /tmp. That helps connection reuse, but it can leak one customer's data into the next request. Clear sensitive buffers. Do not cache authorization decisions in memory without a time limit.

Reserved and provisioned concurrency

These two settings sound alike and do different things.

Confusing them burns money or availability. Reserved concurrency does not make anything faster.

Common mistakes

• Opening a pool of 5 in every environment. Multiply it by the environment count before you ship.
• Treating reserved concurrency as warm capacity. It is a cap, not a pre-warm.
• Assuming a single environment serves all requests. Each environment has its own memory and connections.

Check yourself

1. Reserved concurrency is 200, and each environment holds a pool of 5. How many connections can the database see at peak? Answer: 1,000.
2. How do reserved and provisioned concurrency differ? Answer: reserved caps how many environments a function can use. Provisioned pre-warms a set number of them.

Key takeaways

• Connections multiply by environment count, not by request count.
• Use a pooler or a pool size of 1 inside each environment.
• For synchronous paths, use the tightest timeout that applies.
• Reserved concurrency caps. Provisioned concurrency warms.
• Next, the cost crossover shows when this design costs more than containers.

Further learning

Maximum concurrent requests per instance (Cloud Run docs)Configure maximum concurrent requests for servicesGoogle Cloud Documentation