39 long-form articles on production engineering — how real systems fail, why, and what actually fixed them.
Architecture decisions, distributed-systems trade-offs, and observability.
13 articles in System Design
Observability16 min read
The bell curve taught you the average is a fair summary and outliers are rare. In software both halves are false: latencies, key popularity, file sizes and follower counts are heavy-tailed power laws where the outliers…
Tradeoffs in Production15 min read
When a service buckles under load, the first instinct is to make the buffer bigger — and it's the most natural wrong answer in systems engineering.
Architecture Decisions16 min read
"Stamp each write with the current time and keep the latest" feels obvious — and it silently deletes data, because two machines can never agree on what time it is.
Architecture Decisions18 min read
Most engineers think a leader election "picks the best node." It doesn't — it gets every node to agree on the same node, and agreement under failure is consensus.
Observability10 min read
Every distributed system contains a quiet subsystem that decides which peers are still alive — and "send a ping, no reply, mark it dead" is wrong in a few interesting ways.
Architecture Decisions22 min read
A plain-language walkthrough of Pat Helland's 2007 classic "Life beyond Distributed Transactions" — the paper that named the patterns every large system already uses.
Architecture Decisions18 min read
Everyone can define read-your-writes consistency; far fewer can implement it on a master/replica cluster without melting the master.
Architecture Decisions20 min read
Every process you run will be told to stop — the only question is whether it gets a request it can act on or a bullet it never sees coming.
Tradeoffs in Production16 min read
Kernel TCP keep-alive probes the network path; app heartbeats prove the peer process is serving; HTTP Connection: keep-alive only reuses sockets.
Architecture Decisions20 min read
A senior-engineer walkthrough of the whole cache stack: CPU L1/L2/L3, TLB, OS page cache, DB buffer pool, in-process LRU (Caffeine), distributed cache (Redis/Memcached), HTTP cache + ETags, CDN edge, browser, DNS, reverse proxies, query plan cache.
Architecture Decisions20 min read
SVG-backed walkthrough: ring hotspots, successor overload when a node dies, and two ring generations during deploy.
Architecture Decisions20 min read
A beginner-friendly walkthrough: what “concurrent” means, what the C10K and C10M problems actually are, how the industry usually fixes them (event loops vs whole-system design), and what still goes wrong in production —…
Architecture Decisions8 min read
Fan-out on write vs fan-out on read — and why Twitter uses a hybrid at 500M users. A beginner-friendly breakdown.