Concurrency and Processes

A server that listens (from the client-server lesson) will receive overlapping requests. If it fully finishes client A's slow database read before it even accepts client B, B waits for A's entire story. Products that serve many users need overlapping work.

Three common shapes appear in real systems. Multi-process: run several OS processes behind a load balancer; each process may still be simple inside. Multi-thread: one process hosts many threads that share memory and serve requests together. Async/event-driven: mostly one (or few) threads switch between tasks whenever a task would wait on I/O.

The hard part is rarely "can I accept a socket." The hard part is shared state: counters, caches, session maps, in-memory shopping carts. When two overlapping tasks read-modify-write the same value without a rule, you get a race: lost updates, double charges, or torn reads.

Concurrency is not the same as "use more machines." You can be concurrent on one CPU. Parallelism is when hardware truly runs steps at the same time. Both matter. They answer different questions.

Process: the OS unit of isolation. Separate address spaces make accidental memory corruption across programs harder. Communication uses explicit channels (sockets, pipes, files, shared-memory regions you opt into). Crash of one process need not crash another.

Thread: a schedulable execution context inside a process. Threads share heaps and globals. Coordination uses locks, atomics, concurrent data structures, or message queues. Deadlock is mutual waiting for locks. Starvation is never getting scheduled or never getting the lock.

Async / event loop: tasks are state machines or coroutines. When a task would block on I/O, it yields. The loop runs another ready task. This overlaps waiting well. CPU-heavy work still blocks the loop unless offloaded to workers. "Async" is a scheduling style, not automatic safety for shared data.

Race condition: correctness depends on interleaving. Classic lost update: both tasks read count=10, both write count=11, one increment vanishes. Mitigations include mutual exclusion (locks), atomic read-modify-write, compare-and-swap loops, single-writer ownership, queues that serialize mutations, and moving the truth to a datastore with transactions (see state and persistence).

Server models map to client-server load: thread-per-request, process-per-request (or prefork), and async multiplexers are different answers to "many clients, one service." Load balancers add another process boundary in front.

Example A: blocking toy server. Accept connection, read request, query DB (2 seconds), write response, then accept the next client. Under ten concurrent clients, queueing latency explodes even if the CPU is idle during DB waits.

Example B: thread pool. Each request borrows a worker thread. Overlap improves. If the pool has 50 threads and 5,000 waiting clients, acceptance still queues. Pool size is a capacity knob, not infinity.

Example C: async HTTP server. Thousands of open connections mostly wait on clients or downstream APIs. The loop shines when work is I/O bound. A single tight JSON parse loop of 200ms stalls everyone on that worker.

Example D: race. Two requests increment an in-memory "likes" counter without synchronization. Under load, the counter drifts low. Fix classes: atomic increment, lock around the update, or store likes in a database with an atomic UPDATE likes = likes + 1.

Example E: process isolation for safety. A browser site isolation story and a multi-process app server both use process boundaries so one crash or compromise is less likely to take all memory with it. Cost: more RAM and more IPC.

The GIL and language runtimes: some runtimes limit CPU parallelism of threads for interpreter reasons while still allowing I/O overlap. Read your runtime's real model before assuming multi-core speedups.

False sharing and cache-line contention make "more threads" regress on hot counters. Performance work needs measurement.

Tail latency: average looks fine while the 99th percentile request hits lock convoys or GC pauses. Concurrency bugs often show up as rare spikes.

Distributed concurrency: two server replicas are concurrent processes without shared memory. Races move to the database or cache. Soft state in memory is not shared across replicas unless you design it.

Security angle: data races can become security bugs when checks and uses interleave (TOCTOU). Authorization checks need the same careful sequencing as counters.

Over-synchronization: one giant global lock removes races and destroys overlap. Prefer smaller critical sections or ownership designs.