Namespaces and cgroups

A container is a Linux process that the kernel has been told to isolate. Two features do the work. Namespaces control what the process can see. Control groups (cgroups) control how much of the machine it can use.

Namespaces: what a process can see

Linux has several namespace types. Each one wraps one global resource, so the process sees its own private copy.

You can build a crude container with one command on any Linux box.

sudo unshare --pid --fork --mount-proc --uts --net bash
hostname demo
echo $$
ip link

Expected output, abridged:

1
1: lo: <LOOPBACK> mtu 65536 qdisc noop state DOWN mode DEFAULT ...

The new shell is PID 1, host processes are gone, and the only network interface is a loopback that is down. Docker adds the rest: a different root filesystem, a virtual network link to a host bridge, and resource limits.

Cgroups: what a process can use

With cgroup v2, the runtime writes each container's limits into plain files.

cat /sys/fs/cgroup/system.slice/docker-<id>.scope/memory.max
cat /sys/fs/cgroup/system.slice/docker-<id>.scope/cpu.max

Expected output:

536870912
50000 100000

The first value is a 512 MiB hard limit. The second means 50 ms of CPU time per 100 ms period, which is 0.5 CPU.

The two limits fail in different ways.

• Over the memory limit: the kernel's OOM killer sends SIGKILL. The container exits with code 137 (128 + signal 9).
• Over the CPU limit: nothing is killed. The kernel pauses the processes until the next 100 ms period. This is CPU throttling, and it looks like latency spikes, not errors.

Root's powers are also split into about 40 capabilities, and a seccomp profile blocks risky system calls. See container security.

Who sets this up

The Open Container Initiative (OCI) standardized these pieces. Two of them do the setup.

runc is the low-level runtime. It sets up namespaces, cgroups and capabilities, runs your process, and exits. containerd pulls images, unpacks layers and supervises containers. Kubernetes talks to containerd through the Container Runtime Interface. Docker-built images still run, because they are OCI images.

PID 1 and signals

Your entrypoint becomes PID 1 inside its PID namespace. The kernel gives PID 1 no default signal handlers, so a signal such as SIGTERM is ignored unless your process handles it. docker stop then waits 10 seconds and sends SIGKILL. Use the exec form, CMD ["python", "app.py"], so your app receives the signal directly.

Common mistakes

• Using the shell form of CMD. The shell becomes PID 1 and does not forward signals to your app.
• Reading exit 137 as an app bug. It usually means the OOM killer ran, so check memory first.
• Expecting CPU limits to kill. They throttle quietly, so the symptom is slow responses.

Check yourself

1. A container exits with code 137. What happened? Answer: The kernel sent SIGKILL (128 + 9), usually because the process went over its memory limit.
2. A container has cpu.max set to 0.5. Its latency spikes, but nothing crashes. Why? Answer: CPU throttling. Its processes are paused once their quota for each 100 ms period is used.

Key takeaways

• Namespaces control what a container sees. Cgroups control what it uses.
• A memory overrun ends in a kill (exit 137). A CPU overrun ends in throttling (latency).
• runc sets up the isolation, and containerd manages images and containers.
• PID 1 ignores signals it has no handler for, so handle SIGTERM in your app.

Further learning

Liz Rice builds a working container from scratch in about 200 lines of Go, live on stage. Watching namespaces and chroot appear one by one makes the "a container is just a process" idea stick.

Jérôme Petazzoni's classic talk walks through each namespace and cgroup controller, and explains how copy-on-write storage drivers work.