39 long-form articles on production engineering — how real systems fail, why, and what actually fixed them.
6 articles matching “Kafka”
Kafka & Pipelines25 min read
You add 24 more workers to fix Kafka lag — and they all sit idle. Meanwhile one huge message is stuck at the front of a partition, and nothing behind it moves.
Tradeoffs in Production15 min read
When a service buckles under load, the first instinct is to make the buffer bigger — and it's the most natural wrong answer in systems engineering.
Production Incidents25 min read
Why “our queue guarantees exactly-once” is almost always a lie once your flow crosses a component boundary: Kafka EOS scope, ack-after-process, unique constraints, SQS FIFO, DB transactions with SMTP, retries without…
Observability16 min read
The bell curve taught you the average is a fair summary and outliers are rare. In software both halves are false: latencies, key popularity, file sizes and follower counts are heavy-tailed power laws where the outliers…
Architecture Decisions16 min read
"Stamp each write with the current time and keep the latest" feels obvious — and it silently deletes data, because two machines can never agree on what time it is.
Architecture Decisions18 min read
Most engineers think a leader election "picks the best node." It doesn't — it gets every node to agree on the same node, and agreement under failure is consensus.