Operating systems and concurrency
Processes, threads and the kernel
What the OS gives every program, and how to see it on a live Linux box. Start here for the process model.
Your service doesn't run "on the CPU". It runs inside a process, a set of resources the operating system keeps for one program. The kernel (the core of the OS) makes each program believe it has a whole machine: a private memory space, a CPU it never shares, and its own list of open files. None of that is literally true. The kernel shares a few cores across many programs and maps their memory onto shared RAM.
Knowing how this works pays off at work. "Too many open files", a container that takes 30 seconds to stop, a Java app showing 3 GB of virtual memory but 400 MB of real memory, and a Python server that handles 50 requests but falls over at 5,000 are all questions about processes, threads and the kernel.
The big idea
A process is the unit of isolation. It owns a virtual address space (its private view of memory), a table of file descriptors, its user identity, its environment variables and one or more threads.
A thread is one path of execution inside a process. It has its own stack and registers. Threads in the same process share memory and open files, which is cheap but risky.
Here is how the three common units compare. The numbers are rough, but the orders of magnitude matter.
| Unit | Isolation | Creation cost | Switch cost |
|---|---|---|---|
| Process | Full: separate address space | About 100s of µs, more for big parents | Context switch plus page table switch |
| OS thread | None: shared memory | About 10–20 µs | About 1–5 µs, kernel involved |
| Goroutine or green thread | None: shared memory | Under 1 µs | About 100–200 ns, in user space |
Concept
File descriptor (fd): a small integer that stands for an open file, socket or pipe. Read the file descriptors page first, since the rest of this topic uses it.
Concepts in this section
Read them in this order. Each one builds on the one before.
| Step | Concept | What it answers |
|---|---|---|
| 1 | File descriptors | What is the integer my program holds for a file or socket, and why does the limit matter? |
| 2 | Process lifecycle | How does a process get born, run a new program and exit? |
| 3 | Virtual memory | Why does ps show two memory numbers, and what triggers the OOM killer? |
| 4 | CPU scheduling | How does the kernel share cores, and why do containers get throttled at low CPU? |
| 5 | Signals and graceful shutdown | How should a service stop without losing work, and why does PID 1 cause trouble? |
Why threads and processes are not free
Creating a process or thread costs time, and the kernel does work on every switch. Count the cost before you pick a model. Ten thousand threads, each with an 8 MiB reserved stack, reserve 80 GiB of address space. Most of that is never touched, so the real cost is lower, but the scheduler still has to juggle 10,000 threads. That is why thread-per-connection works for hundreds of connections and gets painful in the thousands.
Common mistakes
- Reading VSZ as memory use. VSZ is reserved address space. Alert on RSS or the cgroup's
memory.currentinstead. - Assuming threads are free. Each thread costs stack space and switches. Many threads on a few cores make the machine busy doing nothing useful.
- Treating a process as one thing. A process can have many threads. Check
nlwpinpsto see the thread count.
Key takeaways
- A process is an isolated address space plus kernel resources. A thread is an execution path inside one.
- The kernel hides the sharing. Virtual memory, scheduling and file descriptors are the three illusions to understand.
- Processes isolate failures. Threads share memory cheaply and inherit its hazards.
- Use the tools on a live box:
ps,top -H,/procandlsofshow what the kernel knows.
Further learning
Start with file descriptors and continue through the steps above. Julia Evans's zines are a fast way to build hands-on intuition about /proc, strace and memory.
Operating Systems: Three Easy Piecespages.cs.wisc.eduRelated
File descriptors
The small integer your program holds for each open file, socket or pipe, and why "too many open files" is either a low limit or a leak.
Process lifecycle
How a process is born with fork, takes on a new program with exec, and exits with a code. Why zombies build up when a parent never calls wait.
Virtual memory
Why every process sees its own private memory, how the kernel maps it to RAM in 4 KiB pages, why ps shows two memory numbers, and what triggers the OOM killer.
CPU scheduling
How the kernel decides which thread runs on which core, why extra threads cost time, and why a container with low average CPU can still stall.
Signals and graceful shutdown
How the kernel tells a process to stop, why SIGTERM can be caught and SIGKILL cannot, and how a service drains work before it exits. Covers the PID 1 trap in containers.