Scaling & Infra22 min read
Serverless on AWS in Production — What Nobody Tells You Until It Hurts
Not a "what is serverless" post. The stuff you learn after the demo, from someone who's run it at scale.
Writing
39 long-form articles on production engineering — how real systems fail, why, and what actually fixed them.
17 articles matching “Production”
Schema Design12 min read
Stop Using DELETE in Production: Hard Delete vs Soft Delete in High-Concurrency Systems
Why DELETE is the most expensive DML operation in a relational database — and why a simple UPDATE SET deleted_at = NOW() is 4-8x faster under concurrent load.
WebSockets & Real-time20 min read
Slack Sends Messages Over HTTP, Not Its WebSocket — and the Durability Bug That Forced the Switch
When you press Enter in Slack, the message does not travel over the WebSocket your client is holding open — it goes out as a separate HTTPS POST to chat.
Production Incidents18 min read
When AUTO_INCREMENT Stops Incrementing — The MySQL INT Overflow Anatomy
It's 3 AM. Inserts are failing with 'Duplicate entry 2147483647' — but you never inserted a duplicate. Welcome to one of MySQL's most quietly famous outages.
Production Incidents25 min read
“We Have Exactly-Once Delivery” — The Lie Your Architecture Is Telling You
Why “our queue guarantees exactly-once” is almost always a lie once your flow crosses a component boundary: Kafka EOS scope, ack-after-process, unique constraints, SQS FIFO, DB transactions with SMTP, retries without…
Tradeoffs in Production16 min read
TCP Keep-Alive vs Application Heartbeat — Three Different Things Called “Keep-Alive”
Kernel TCP keep-alive probes the network path; app heartbeats prove the peer process is serving; HTTP Connection: keep-alive only reuses sockets.
Query Optimization5 min read
MySQL Query Optimization: Real Lessons from a 36GB Index Disaster
What happens when your index outgrows your data — and how we fixed it without downtime in production.
Production Incidents45 min read
COMMIT Returned OK. After the Power Came Back, the Row Was Gone.
A payment commits, the API returns 200, the rack loses power, and after recovery the row does not exist. Before blaming the database, six questions have to be separated.
Production Incidents28 min read
One Header Was Supposed to Stop Double Charges. Here Are 10 Ways It Didn’t.
A timeout is not a failure. Ten real ways idempotency keys break in payment systems — lost gateway responses, two tabs, racing retries, Redis evictions, expired keys, duplicate webhooks — with the fix for each, ten…
Kafka & Pipelines25 min read
How Kafka Finally Solved the “One Slow Message Blocks Everything” Problem — Share Groups, Explained Simply
You add 24 more workers to fix Kafka lag — and they all sit idle. Meanwhile one huge message is stuck at the front of a partition, and nothing behind it moves.
Query Optimization13 min read
“The Money Was There When I Checked” — Transaction Isolation Levels and the Bugs Each One Lets Through
“We sold 3 tickets but only had 1.” The code is inside a transaction, looks right, works a hundred times locally, and only breaks in production under load.
Tradeoffs in Production15 min read
The Queue That Ate Your Latency — Why Backpressure, Not a Bigger Buffer, Is How Systems Survive Overload
When a service buckles under load, the first instinct is to make the buffer bigger — and it's the most natural wrong answer in systems engineering.
Architecture Decisions20 min read
Why Your Consistent Hashing Is Still Failing in Prod — and When Range Partitions Are the Right Tool
SVG-backed walkthrough: ring hotspots, successor overload when a node dies, and two ring generations during deploy.
Architecture Decisions20 min read
C10K vs C10M: Two Scaling Problems People Confuse for One
A beginner-friendly walkthrough: what “concurrent” means, what the C10K and C10M problems actually are, how the industry usually fixes them (event loops vs whole-system design), and what still goes wrong in production —…
Production Incidents16 min read
Stop-the-World: When GC Runs, Everything Freezes
When the Garbage Collector pauses, every thread freezes — zero requests, zero responses. This post explains how GC actually works: the heap, mark-and-sweep, minor vs major collections, Stop-the-World pauses, and what…
Production Incidents15 min read
The Brutal Truth About Multi-Tenant IoT Data Filtering: Every Obvious Solution Failed
A user had access to 1 lakh devices across MongoDB, ClickHouse, and Elasticsearch. We tried seven approaches. All broke. The full journey — dead ends, edge cases, and what production systems actually do.
WebSockets & Real-time4 min read
Socket Exhaustion in High-Concurrency IoT Systems
A production case study — how a small config issue brought down an entire IoT device fleet and what fixed it.