Serverless and managed runtimes

Serverless is the right default for many startup workloads and for the glue between managed services. For long-lived, steadily busy APIs with heavy connection pools, always-on containers on Kubernetes or a VM often win. Choose with numbers, not slogans.

Concepts in reading order

Start with step 1. Each later step assumes you know what the platform is doing for you.

Choosing serverless or containers

Serverless fits when:

• Traffic is uncertain or mostly idle, as in side projects, internal tools and early MVPs.
• The work is event-shaped: webhooks, file uploads, queue consumers and cron jobs.
• The team is small and does not want node upgrades or capacity tickets.
• You can keep functions thin and push state into managed data stores.

Containers or Kubernetes fit when:

• Requests are steady and high, or connections stay open for hours, such as WebSockets.
• You need many persistent database connections, GPUs or large local caches.
• Your latency target rules out cold starts, and warm capacity would cost as much as always-on.
• You already have a platform team and golden paths for deployments.

Observability and local testing

Logs are easy on serverless. The platform attaches them to each invocation. Metrics need deliberate labels: cold versus warm, duration percentiles, concurrent executions, throttles and errors. Traces need the request context passed from the API gateway or event bus into the handler. Without a correlation ID on every log line, a chain of small functions becomes hard to follow. See observability.

Local testing is the weak spot. Tools such as SAM CLI and LocalStack cover unit tests, but they do not match IAM, VPC attachment or event source mappings well. Cloud Run is the easiest to test locally, because you can run the same container with Docker. For functions, test against real event fixtures, and run integration tests in a separate cloud account.

Common mistakes

• Choosing by slogan. Compute the cost and latency for your traffic first.
• Opening unbounded database pools in functions. Use a proxy and small pools.
• Skipping dead-letter queues. Failed events need somewhere to go.
• Debugging with print statements across async retries. Use structured logs with a request ID.

In practice

An SDE2 should be able to pick FaaS, Cloud Run or Kubernetes with a one-paragraph cost and latency rationale. They should size memory and timeouts on purpose, make every handler idempotent, and know how a retry storm would unfold. Expect interview questions such as "explain cold starts and three mitigations," and "when does Lambda get more expensive than ECS or Cloud Run?"

Key takeaways

• FaaS, containers-as-a-service and PaaS are different trade-offs. Containers on a managed runtime often suit HTTP APIs best.
• Cold starts come from the platform, the runtime and your init code. Shrink them first.
• Events are at-least-once, so design for idempotency and dead-letter queues.
• Use proxies and small pools, or your database will go down first.
• Serverless wins on spiky, idle traffic. Containers win on high, steady use. Do the GB-second math.

Further learning

AWS re:Invent's talk on serverless best practices covers the Lambda lifecycle, cost levers, async patterns and where Fargate fits. It is dense and practical.

AWS Lambda Developer GuideWith AWS Lambda, you can run code without provisioning or managing servers.docs.aws.amazon.com Google Cloud Run documentationFully managed application platform to run your code, function, or container on top of Google's highly scalable infrastructure.Google Cloud Documentation