Connection Concurrency at Scale

The central problem this whole atlas is built around. Atlas II's concurrency-in-practice made an HTTP server and a database engine handle many independent, short-lived operations in parallel — the operations don't need to know about each other. Here, connections are long-lived and do need to know about each other: a message from one client has to reach every other client in its room, right now, without blocking the loop that's also accepting new connections and reading incoming frames.

This branch covers epoll (Linux) and kqueue (macOS/BSD) event loops vs thread-per-connection vs hybrid thread-pool designs, the C10K problem and its evolution into C10M, and — the part that actually differs from Atlas II — the data structures used to register and broadcast across thousands of simultaneously open connections, with lock-based and lock-free approaches compared under real load. v3 of the chat is built here: rooms and presence broadcasting to hundreds of simulated connections.

Planned notes

• Why thread-per-connection breaks down long before you have 10,000 users, and why it breaks down differently for chat than for HTTP
• epoll: edge-triggered vs level-triggered, and the bugs each mode invites for a WebSocket server specifically
• kqueue: the same problem, a different API — what changes and what doesn't when you port between them
• The C10K problem (Kegel) and how its answer needed revisiting for C10M-scale concurrent connections
• Thread-pool hybrids: N worker threads each running their own event loop, and how connections get assigned to one
• The connection registry: a shared, mutable table of "who's connected, in which room" under concurrent access
• Locks vs lock-free structures for that registry — measured, not assumed, the way Atlas I's concurrency branch insists on
• Broadcasting from one connection to N without blocking the event loop that owns the write
• False sharing in a per-connection state array under high connection counts
• Load-testing with hundreds of simulated concurrent WebSocket clients, and reading the latency distribution under broadcast load

Core sources

• Dan Kegel, "The C10K Problem" — the central document for this branch, continuing directly from Atlas II's http-server-from-scratch.
• man epoll, man kqueue.
• Jeff Preshing (preshing.com) — memory ordering and lock-free patterns, continued from Atlas I and II.

Connects to: TCP Fundamentals · Presence, Rooms & Fanout