One Header Was Supposed to Stop Double Charges. Here Are 10 Ways It Didn’t.

Every payments engineer knows the 30-second version. The client sends an Idempotency-Key header, the server remembers it, and a retry does not charge twice. That version fits on a slide. It also fails in production in at least ten ways. A key that changes on every retry. Two retries that arrive in the same millisecond. A Redis eviction that silently drops the key. A process that dies after the bank moved the money but before your database recorded it. A key that expired one day before an offline phone sent its retry.

This post is about those ten ways. For each one, it shows what happened, why it happened and the fix. Each incident also asks you a question before it gives the answer. Try to answer it first. That is how the lesson sticks. The incidents are composites. Each one is a failure pattern that payment teams report again and again, with the numbers changed. Every fact about a real company comes from that company's own documentation or engineering writing, linked at the end. After the incidents comes the question everyone asks: why not just generate the key in the browser? Then come ten myths, and then the design, the SQL and a short self-test.

Part 1The basicsWhere a request can die, and why the client cannot tell which place it was

A timeout is not a failure

A customer taps Pay ₹10,000. The app sends POST /payments. Five seconds later, the HTTP client gives up and reports a timeout. What happened to the money?

Nobody on the client side knows. The request can die in three places, and the client sees the same timeout in all three.

Stripe's engineering blog describes the same three cases. The connection fails before the server gets the request. The server fails in the middle of the operation. Or the operation succeeds and the client never gets the result. Case C is the expensive one. The bank moved the money, your server knows it, and the client thinks the payment failed.

The client now has three choices. It can ignore the error, and the customer may leave without paying. It can show the error, and the customer taps Pay again, which is a retry with extra steps. Or it can retry automatically. All three choices send a second request in practice, because people, SDKs and proxies retry whether you plan for it or not.

Why POST needs help

HTTP already defines which methods are safe to repeat. GET reads and changes nothing. PUT and DELETE are idempotent by contract: "set the balance to 500" twice leaves the balance at 500. POST has no such contract. "Charge ₹10,000" twice charges ₹20,000.

Payments are POST by nature. Each request creates something new: a charge, a refund, a payout. So the server needs another way to recognize that two requests are the same intent. That is the job of the idempotency key. The client generates a unique ID for one operation and sends it with every attempt of that operation. The server stores the key and the result of the first attempt. Every later attempt with the same key gets the stored result, and the server does no new work.

The definition that matters. An operation is idempotent when doing it many times has the same effect as doing it once. It does not promise the same response, and it does not promise that the work ran only once inside your system. Keep this sentence in mind. Half of the myths at the end come from forgetting it.

Every major payments API uses this pattern under a different header name:

Provider	Header	What the docs say
Stripe	`Idempotency-Key`	Client generates it. V4 UUID suggested. Up to 255 characters. All `POST` requests accept it. Keys can be pruned after they are at least 24 hours old.
PayPal	`PayPal-Request-Id`	The server stores the ID "for a period of time". The period depends on the API, and not every API supports the header.
Razorpay X (payouts)	`X-Payout-Idempotency`	Mandatory on every payout request since 15 March 2025. 4–36 characters. A different body with the same key returns `BAD_REQUEST`.
Adyen	`Idempotency-Key`	Supported on `POST`. Up to 64 characters. UUID recommended.
IETF draft	`Idempotency-Key`	`draft-ietf-httpapi-idempotency-key-header-07` (October 2025, now expired). It suggests 400 for a missing key, 422 for a reused key with a different payload, and 409 while the first request still runs.

That is the slide version. Now for production.

Part 2Ten real failuresWhat happened, why it happened, and the fix. Each one got past a team that "already had idempotency keys"

1. The gateway charged. The response never came back.

What happenedIt is a sale day. The gateway's p99 latency rises from 800 ms to 9 s. The payments service calls the gateway with a 5-second timeout. On a timeout, the service marks the payment FAILED and tells the app. The customer taps Pay again, and the service creates a new charge. The next morning, the settlement file shows 312 customers charged twice. Support refunds them by hand for a week.

Think first: The app sent an idempotency key with every request. So why did 312 customers still pay twice?

The answerThe app sent a key to the payments service, so the first hop was safe. The second hop was not. The service called the gateway with no key, and a timeout on that hop was case C: the money moved and the answer got lost. The service then turned "I do not know" into "it failed". That false fact caused the second charge, not the network.

The fixFour changes, and all four matter.

Use three states, not two. A payment is SUCCEEDED, FAILED or UNKNOWN. A timeout gives UNKNOWN. On UNKNOWN, the app shows "we are checking your payment". It never shows "payment failed, try again".
Send a key on every hop. Derive the downstream key from your own key, for example <key-id>:charge. Brandur Leach's Stripe-style design sends Stripe a key built from the local key record. A retry then sends the same derived key, and the gateway returns the original charge.
Check the status first on rails without keys. Many bank APIs and UPI flows have no idempotency header. There, call the status-inquiry API with your own transaction reference before any retry. The RBI sets turnaround rules for failed transactions. Under them, a UPI debit that did not reach the beneficiary gets an automatic reversal by T+1 for person-to-person payments. So "pending" is a real state that can last a day. A retry in that window can debit the customer twice.
Reconcile every day. Compare your ledger with the gateway's settlement file. Reconciliation is the last line of defense. It catches what the code missed, and in this incident it found the 312 double charges.

2. Double tap, two tabs, the back button

What happenedA customer on slow 3G taps Pay. Nothing happens for four seconds, so they tap again. Another customer opens checkout in two tabs and pays in both. A third presses Back after paying, and the browser submits the form again. The client generated a fresh key for each submit. The server saw three keys and charged three times.

Think first: Every request carried a key, and the server checked every key. Where did three charges come from?

The answerThe key identified the HTTP request, not the customer's intent. An idempotency key protects the retries of one attempt. It cannot know that two attempts with two different keys mean the same thing. Only your data model knows that.

The fixUse two layers.

Layer 1: the order is the intent. The server creates an order before the payment page opens, and the payment request points at that order. Razorpay's standard checkout works this way: your server creates an order first, and the payment attaches to its order_id. Generate the idempotency key once per order, not once per click.
Layer 2: the database says no. Add a partial unique index: one live payment per order, where live means pending or captured. Two tabs can still send two different keys. The second tab's insert then fails the constraint before any money moves, and the API returns the first payment. A declined payment is not live, so the customer can try another card.
Disable the button too. It reduces load and confusion. It is not a correctness tool, because a reload, a second tab or a network retry goes around it.

3. The key that changed on every retry

What happenedA team added idempotency keys and shipped. Double charges continued at the same rate. The key came from an axios request interceptor. The retry library called the interceptor again for every attempt, so every attempt got a brand-new UUID. A second team generated the key in a React component body, which runs on every render. A page reload lost it too.

Think first: The team added idempotency keys, and the double charges continued at the same rate. What would you check first?

The answerThe code created the key where the request starts, not where the intent starts. A key that changes between attempts is not a key. It is a request ID with a misleading header name.

Where the code creates the key	What happens on a retry	Result
In an HTTP interceptor	The retry library runs the interceptor again, so each attempt gets a new key	❌ Charged again
In a UI component's render	Each render makes a new key, and a reload loses it	❌ Charged again
In a variable inside the retry loop	Each loop pass makes a new key	❌ Charged again
On the order, when the server creates it	Every attempt, tab and reload reads the same key from the order	✅ Charged once

The fixCreate the key when the intent starts: when your server creates the order or the checkout session. Store it with that object, on the server if possible. Then write the test that teams skip: force a timeout, let the client retry, and assert that every attempt carried the same key.

4. Two retries in the same millisecond

What happenedThe mobile client used a 2-second timeout and retried at once. During a database slowdown, the server's p99 rose to 2.5 s. So the retry arrived while the first attempt was still running. The server's logic was: look up the key, and if there is no row, process the payment and insert the key. Both attempts looked up the key, found no row and charged the card. The team that used Redis GET and then SET had the same bug.

Think first: The server checks for the key before it charges. How can two requests both pass that check?

The answerCheck-then-act. A gap exists between the check and the insert. On a quiet day the gap is microseconds. Under load it grows to seconds, and that is exactly when clients retry. The isolation levels post shows the same bug with stock counts.

The fixClaim the key before any work, in one atomic step. In Postgres, use INSERT … ON CONFLICT DO NOTHING RETURNING id. In Redis, use SET key value NX PX 30000. Exactly one request wins. The winner marks the key as in progress with a lease. Every other request that arrives while the lease is live gets 409 Conflict with a Retry-After header. The IETF draft and Brandur's design both use 409 for this case. Stripe documents that it saves no result for a request that conflicts with a concurrent one, so the client can retry it.

5. Same key, different amount

What happenedA customer's ₹1,000 payment timed out. On the retry screen, the app applied a coupon, and the retry sent ₹900 with the same key. The server looked only at the key and replayed the first response. The customer saw "paid", and the bank statement showed ₹1,000. In a second incident, the server did compare bodies. It hashed JSON.stringify(body). A refactor then changed the order of the object's keys, and every honest retry got a mismatch error. The client "recovered" by generating a new key, and the double charges returned.

Think first: The retry has the right key but a different amount. Should the server replay the old answer, run the new payment, or reject it?

The answerA key without a fingerprint trusts the client to never reuse it. A fingerprint over non-canonical JSON breaks on changes that do not change the meaning.

The fixStore a hash of the method, the path and the canonical body: keys sorted, no extra whitespace. On the same key with a different hash, reject the request. The IETF draft suggests 422. Brandur's design returns 409. Stripe returns an error, and Razorpay returns BAD_REQUEST. The client must treat this error as a bug, never as a reason to make a new key automatically. A changed amount is a new intent. It needs a new key, and usually a new order.

Here is the key-order trap. Both bodies mean the same thing. Their raw JSON text differs, so a hash of the raw text differs too:

Request body as sent	Hash of the raw text	Hash after sorting keys
`{"amount":900,"currency":"INR"}`	`4b7a…`	`9c1e…`
`{"currency":"INR","amount":900}`	`e03f…` ❌ mismatch	`9c1e…` ✅ match

So canonicalize before you hash: sort object keys at every level, drop extra whitespace, and keep numbers in one format. Most languages have a library for "canonical JSON". The rule fits in one sentence: same meaning, same bytes, same hash.

6. The process died between "charged" and "saved"

What happenedA deploy sent SIGKILL to a pod in the middle of a request. The key row said in_progress. The gateway already charged the card. The process died before it wrote the result. On retry, one team's server returned 409 "in progress" forever, and the customer could never finish. Another team expired the lock and ran the handler again from the start. The handler sent a fresh gateway call with no key, and the customer paid twice.

A third team held a database transaction open across the gateway call: BEGIN, insert the payment, call the gateway, COMMIT. The commit failed after the charge succeeded. The database rolled back the only record of a real charge. The long transaction also held a pooled connection for the full gateway latency. The connection pool post shows how fast that empties a pool on a slow day.

Think first: The gateway charged the card, the process is dead, and the key row says in_progress. What should the next retry do?

The answerA call to another system cannot be rolled back. Brandur calls these calls foreign state mutations, and he counts calls inside your own infrastructure too, such as a write to Kafka. Airbnb's Orpheus framework splits each remote call into pre-RPC, RPC and post-RPC phases, and it records the request in the database before the RPC. The pattern is the same: never hide a remote call inside a local transaction.

The fixSplit the request into atomic phases, as Brandur's design does.

Each local phase is one database transaction. The same transaction also moves the key's recovery_point.
Each foreign call sits alone between two phases and carries a derived key, so it is safe to repeat.
The key row holds a lease (locked_until), not a permanent lock. A retry after the lease expires takes over and resumes from the last recovery point.
A background completer finds requests that stopped part-way and whose client went away, and pushes them to the end. Without it, a customer who closes the app leaves a charge with no order.

7. The 500 that replays forever

What happenedA bad deploy returned 500 on /payments for 20 minutes. The idempotency layer saved every response, including the 500s. After the rollback, clients retried with the same keys and got the saved 500 back. Those customers could not pay until they started a new checkout. Another team had the opposite rule: it saved nothing on any error. Retries of a declined card then went to the issuer again and again. Visa charges merchants for excessive reattempts. For a "never retry" decline, such as a closed account, Visa treats the first reattempt as excessive. Other declines have a cap of 15 reattempts in 30 days. Blind retries of a decline cost real money.

Think first: Should the server save a 500 error and replay it on retry? And what about "card declined"?

The answerErrors are not all the same. Some errors are a result: the work ran and the answer is final. Others are an accident: the work did not run, or it stopped part-way, and a retry can finish it.

The fixWrite the policy down, per class of response:

Response	Save it and replay it?	Reason
`2xx` success	Yes	The work finished. Every retry must get the same answer.
`402` card declined, and other final business errors	Yes	The work ran, and the answer is final. A retry must not hit the issuer again.
`400`/`422` validation, before any work	No	Nothing ran. Stripe also saves no result when validation fails.
`409` in progress, `429` rate limit	No	These describe the moment, not the operation.
`5xx` before any foreign call	No. Release the lease.	No money moved. A retry can run the work from the start.
`5xx` after a foreign call	No. Keep the recovery point.	The retry resumes and repeats the foreign call with the same derived key.

Stripe made a different choice, and it is worth knowing. For API v1, Stripe saves the status code and body once the endpoint begins to run, and it replays them, including 500 errors. Its API v2 changes this. A retry does not repeat successful work. Stripe retries failed or partly failed work to completion where it can, and it returns an updated response. Either policy can be right. What hurts is not knowing which one your system uses.

8. The key expired before the retry

What happenedA mobile app saved unfinished payments in a local queue and retried them when the app opened again. A user paid an electricity bill, and the app crashed while the request was in flight. The user opened the app three days later. The app retried with the original key. The server deleted keys after 24 hours, so it treated the retry as new and paid the bill a second time.

Think first: The retry carried exactly the right key. Why did the server not recognize it?

The answerThe key's lifetime was shorter than the client's retry horizon. Stripe documents the same edge. Stripe can prune keys once they are at least 24 hours old, and a key reused after pruning creates a new request. PayPal stores its request ID "for a period of time" that depends on the API.

The fixMake the server's retention longer than the longest retry horizon of any client, with margin. Brandur suggests about 72 hours, so a bug found on Friday can be fixed and completed on Monday. Then cap the client: after its horizon, the client must not retry with the key. It must ask for the order's status. Keep the unique constraint from incident 2 as the final backstop. Keys expire, but a constraint on the order does not.

9. Redis forgot the key

What happenedThe team kept keys in Redis with a 24-hour TTL. During a traffic peak, Redis reached maxmemory. The policy was volatile-lru, which evicts keys that have a TTL. Every idempotency key had a TTL, so they went first. In a second incident, the Redis primary failed. Redis replicates asynchronously, so the newest keys never reached the replica that took over. In both cases, retries found no key, and the server charged again. A third team wrote keys to the primary database and read them from a replica. Replica lag hid keys that were seconds old.

Think first: The server wrote the key successfully, and nobody deleted it. How can it disappear?

The answerMost teams configure Redis as a cache, and the key store is part of a ledger. A cache may forget. A ledger may not.

The fixKeep keys in the same database as the money, and write the key row and the payment row in one transaction. Then the two cannot disagree. Read keys from the primary, never from a replica. The replica lag post explains why. If you must use Redis for latency, give it its own instance with maxmemory-policy noeviction and persistence turned on. Treat it as a fast first check, and keep the database constraint as the truth.

10. The webhook credited the wallet twice

What happenedThe gateway sent a payment.succeeded webhook, and the handler credited the customer's wallet with ₹10,000. A slow database made the handler take 12 seconds, so the gateway's delivery timed out and the gateway sent the event again. The wallet got two credits. The same thing happened inside the company. A Kafka consumer credited a wallet and then crashed before it committed its offset. After the restart, Kafka delivered the message again.

Think first: Your API already has idempotency keys. Why did the wallet still get two credits?

The answerWebhooks and queues deliver at least once. Stripe's webhook docs say that an endpoint "might occasionally receive the same event more than once". In live mode, Stripe also retries failed deliveries for up to three days. The docs add a subtler case: sometimes Stripe generates two separate Event objects for one change. Then the event IDs differ, and only the object ID plus the event type reveals the duplicate. The idempotency key on your API protects the way in. Events coming back need their own protection. The exactly-once post explains why no broker can fix this for you.

The fixGive every consumer its own dedupe table. Insert the event ID in the same transaction as the side effect, and skip the work when the insert conflicts. Make the side effect itself safe too: "mark payment pi_123 as paid" is safe to repeat, and "add ₹10,000" is not. That covers the two-Event-objects case. Return 2xx fast and do slow work in the background, so the gateway stops resending. Airbnb's payment orchestration uses its idempotency framework in the same way: its event processors turn Kafka's at-least-once delivery into exactly-once processing.

Two more that hurt

Payouts and refunds are worse than charges. A double charge pulls money in, and a refund can fix it. A double payout pushes money out to someone else's bank account, and you may never recover it. Razorpay made its X-Payout-Idempotency header mandatory on every payout request from 15 March 2025. If your own system sends money out, make the key mandatory on every endpoint that moves money.

Keys that leak across tenants. A team stored keys in one global namespace. Two merchants' SDKs used a weak key generator, and their keys collided. One merchant got the other merchant's saved response, with the other customer's payment details. The IETF draft warns about this. When keys have low entropy, an attacker can guess other keys and read other clients' cached responses. Scope every key by account, with a unique index on (account_id, idempotency_key), as Brandur's schema does. Require a minimum length, and check the format before you look the key up. Stripe also says: never put email addresses or other personal data in a key.

Part 3The question everyone asks, and ten mythsWho should create the key, and what idempotency does not give you

"Why not just generate the key in the browser?"

A popular explanation of idempotency keys says the client should first ask the server for a key, and then send the payment with it. Another says the browser should just call crypto.randomUUID(). Both answers are half right.

The client should create the key. Stripe's docs say so, and so does the IETF draft. A separate "give me a key" call adds a round trip, and that call can time out as well. Then the call that should give you idempotency needs idempotency itself.

But notice who "the client" is in Stripe's docs. It is your backend, which calls Stripe. A server keeps its state, runs code you control and has one process per request. The browser is a much weaker client, for four reasons:

It forgets. A reload, a crash or a closed tab loses anything in memory. sessionStorage lives in one tab. localStorage lives in one browser profile. A second device knows nothing.
It multiplies. Two tabs or two devices make two keys for one intent. This is incident 2.
It is not trusted. Anyone can change code that runs in a browser. A hostile user can send a new key on every request. Idempotency keys protect against accidents, not against fraud.
Its generator may be weak. Old UUID libraries used Math.random(). Use crypto.randomUUID(), which uses a secure random source and needs a secure context (HTTPS).

The pattern that works mixes both ideas. The server creates the order first, and the browser's key belongs to that order. A duplicate "create order" call is harmless: an unpaid order costs nothing, and a cleanup job deletes it. The payment call carries the order's key on every retry. The server derives its own keys for each downstream call. The database constraint catches the cases the browser cannot, such as a second device.

So "server-issued" keys are fine when they come out of a call you make anyway, such as creating the order. They are a mistake when they need a dedicated round trip of their own.

Ten myths, and what is true

Myth	Reality
1. "Idempotent means the same response every time."	It means the same effect. The response can change. Razorpay returns the payout's latest state for a duplicate payout request, and that state moves from processing to processed.
2. "Idempotency keys give exactly-once delivery."	Delivery stays at-least-once. Keys make the effect happen once, inside the key's scope and lifetime. Outside the retention window, a retry is a new request.
3. "Delete the key when the request succeeds."	A late retry then finds no key and runs again. Keep the key and its saved response for the full retention window. That is the whole point of the key.
4. "A duplicate request should get an error."	A retry is normal. Return the original response. Add a header such as `Idempotent-Replayed: true` if you want to. Use errors only for a different payload (422) or a request still in progress (409).
5. "Hash the URL and the body. You do not need a key."	Two honest ₹100 transfers to the same friend have the same hash. A hash cannot tell a retry from a second payment. Use the hash as a fingerprint next to a key.
6. "PUT and DELETE are automatically safe."	HTTP defines them as idempotent. Your handler decides whether they really are. A `DELETE` that sends an email or writes an audit row each time is not idempotent in effect. Neither is a `PUT` that fires a webhook on every call.
7. "Generate a fresh key for each retry."	That is a request ID. One intent, one key, every attempt.
8. "Redis `SET NX` solves it."	It solves the race in incident 4. It does not solve eviction, failover, a crash between charge and save, or an expired key.
9. "Idempotency keys stop double-spend fraud."	The client chooses the key, so a hostile client sends a new one each time. Fraud needs business rules: limits, balance checks in the same transaction and unique constraints.
10. "24 hours of retention is always enough."	It is enough only when no client retries after 24 hours. Measure your clients' real retry horizon first, then set the retention above it.

Part 4What to shipThe design, the SQL, and a test for yourself

The design we would ship

Every piece below comes from one of the incidents. Remove a piece, and you reopen that incident.

Every endpoint that moves money requires a key. No key means 400, as the IETF draft suggests and as Razorpay does for payouts.
The key belongs to the intent. The server creates it with the order, and the client reuses it on every attempt (incidents 2 and 3).
Scope and check every key. Unique on (account_id, key), with a format check and a minimum length, and no personal data (tenant leak).
Claim first, atomically. One insert wins, and the others get 409 (incident 4).
Fingerprint the canonical request. A mismatch gets 422 (incident 5).
Atomic phases and recovery points. No transaction is open across a network call. Every downstream call has a derived key (incidents 1 and 6).
A written error policy. It says which responses the server saves and which ones release the lease (incident 7).
Retention is longer than any client's retry horizon. After the horizon, the client checks the status (incident 8).
Keys live in the same database as the money. Reads go to the primary (incident 9).
Every consumer dedupes on the event ID (incident 10).
Constraints and reconciliation as backstops. One live payment per order, and a daily comparison with the settlement file.

The only SQL you need to remember

The whole design rests on one table and two statements. Everything else is ordinary code in whatever language you use.

1. One table remembers every request

CREATE TABLE idempotency_keys (
    account_id       BIGINT  NOT NULL,
    idempotency_key  TEXT    NOT NULL,
    request_hash     TEXT    NOT NULL,   -- fingerprint of the request body
    recovery_point   TEXT    NOT NULL DEFAULT 'started',
    locked_until     TIMESTAMPTZ,        -- the lease
    response_code    INT,                -- the saved answer
    response_body    JSONB,
    created_at       TIMESTAMPTZ NOT NULL DEFAULT now(),
    PRIMARY KEY (account_id, idempotency_key)
);

Column	Why it exists
`account_id` in the key	Keys never leak across customers
`request_hash`	Same key with a different amount gets `422` (incident 5)
`recovery_point`	A retry continues where a crashed request stopped (incident 6)
`locked_until`	Racing retries get `409`, and a dead request's lease expires (incidents 4 and 6)
`response_code`, `response_body`	Retries get the original answer back (incident 7)
`created_at`	A cleanup job deletes finished keys after 72 hours (incident 8)

2. One statement claims the key

INSERT INTO idempotency_keys (account_id, idempotency_key, request_hash, locked_until)
VALUES (:account, :key, :hash, now() + interval '30 seconds')
ON CONFLICT DO NOTHING;

This is the heart of the design. If the insert creates a row, this request owns the key and does the work. If it creates nothing, the key already exists, and the server looks at the row to decide the answer:

The existing row shows…	The server answers…
A different `request_hash`	`422`: this key belongs to a different request
A saved `response_code`	The saved answer, again. No new work.
A live lease	`409` with `Retry-After`: still running, try again soon
An expired lease	Take it over, and continue from `recovery_point`

3. One index stops the second tab

CREATE UNIQUE INDEX one_live_payment_per_order
    ON payments (order_id)
    WHERE status IN ('pending', 'captured');

Two tabs send two different keys, so the key table cannot see the duplicate. This index can. The second tab's payment fails before any money moves. A declined payment is not live, so the customer can still try another card.

The same trick protects webhooks. Give each consumer a table with the event ID as its primary key. Insert the ID in the same transaction as the side effect. If the insert does nothing, you already processed that event (incident 10).

The request, in five steps

Claim the key with the insert above, or decide the answer from the existing row.
Commit the pending payment and move recovery_point to payment_created, in one transaction.
Call the gateway outside any transaction, with a key derived from yours, such as <key>:charge.
Commit the result and the saved answer together, in one transaction. A decline (402) is a final answer, so save it too.
If anything fails on the way, save nothing. Free the lease and answer 503. The retry continues from the recovery point.

Rules for the client

Take the key from the order. Send the same key on every attempt.
Retry only when the outcome is unknown or temporary: a network error, a timeout, 409, 429 or 5xx. Never retry 400, 402 or 422.
Wait a random time between retries, longer after each failure. Stripe's engineering blog explains why. Without the randomness, every client that failed at the same moment retries at the same moment.
Never show "payment failed" for an unknown outcome. Show "we are checking your payment", and poll the order's status.
After 48 hours, stop reusing the key. Ask the server for the order's status instead.

Test yourself

These are the questions a payments team asks in a design review, and the ones interviewers like. Answer each one in your head first. Then open the answer.

1. A client gets a timeout on POST /payments. Is a retry safe?

Only with the same idempotency key. The timeout can mean case A (nothing happened), B (half done) or C (the money moved). The client cannot tell which one. Without a key, a retry in case C charges twice. With the same key, the server either does the work once or replays the first answer.

2. A retry arrives while the first request is still running. What should it get?

409 Conflict with a Retry-After header. The IETF draft and Brandur's design both use 409 here. The server must not run the payment a second time, and it has no final answer to replay yet.

3. Same key, but the amount changed from ₹1,000 to ₹900. Replay, run or reject?

Reject. The IETF draft suggests 422, and Stripe and Razorpay also return an error. A replay would show "paid ₹900" for a ₹1,000 charge. Running the payment would break the key's promise. A new amount is a new intent, so it needs a new key.

4. The payment succeeded. Can the server delete the key now?

No. The success response may be the one that got lost, and the client's retry is still on its way. Keep the key and the saved response for the full retention window, then let the reaper delete it.

5. Why must the gateway call stay outside the database transaction?

Because the database cannot roll back a charge at the bank. If the commit fails after the charge, the rollback deletes your only record of real money moving. A long transaction also holds a pooled connection for the full gateway latency. Use separate phases: commit, call the gateway, commit again.

6. Two tabs send two different keys for one order. What stops the second charge?

Not the key table, because the keys are different. The partial unique index one_live_payment_per_order stops it. The second tab's pending payment fails the insert before any money moves.

7. Is Redis with a 24-hour TTL a good key store?

Only with care. A volatile-* eviction policy deletes keys that have a TTL first, and every key has one. Asynchronous replication can lose the newest keys in a failover. Prefer the same database as the payment. If you use Redis, run it with noeviction and keep the database constraint as the truth.

8. Do idempotency keys stop a user who tries to pay twice on purpose?

No. The client chooses the key, so a hostile client sends a new key every time. Keys protect against accidents. Fraud needs business rules: limits, balance checks in the same transaction, and unique constraints on the business object.

9. The webhook handler is idempotent on the API side. Why did the wallet get two credits?

Because the API key protects requests coming in, and webhooks are events coming back. Gateways deliver webhooks at least once. The consumer needs its own dedupe: insert the event ID into processed_events in the same transaction as the credit.

10. Should the server save a 500 and replay it?

It is a policy choice, so write it down. Stripe's API v1 saves and replays 500s once the endpoint begins to run. Many teams instead free the lease on a transient error and let the retry resume. Either way, save final business results such as a 402 decline, so that a retry never hits the issuer again.

What it gives you, and what it costs

What you get

Clients, SDKs and proxies can retry freely.
A timeout becomes "retry with the same key", not "call support".
Replays return the original answer, so the client and the server agree.
A crashed request resumes where it stopped.
Fewer refunds, fewer support tickets and fewer card-network fees.

What it costs

One extra write per request, plus a saved response body.
A completer, a reaper and alerts to run and watch.
Discipline on every client: one key per intent, an honest UNKNOWN state.
Code split into phases, which is harder to read than one function.
No help against fraud. Business rules still do that work.

The takeaway

An idempotency key is a promise from the server: "the first time I see this key, I do the work. Every time after that, I give you the same result and do nothing new." The header is the easy part. Keeping the promise is the hard part. Three points are worth carrying away:

A timeout is not a failure. It is "unknown". Most double charges start when a system turns "unknown" into "failed" and acts on it. Give UNKNOWN its own state, in the API and in the UI.
The key belongs to the intent, and every hop needs one. One key per customer intent, reused on every attempt, and a derived key on every call your server makes to someone else.
Keys are the first line, not the last. Keys expire, caches forget and browsers multiply. A unique constraint on the business object and a daily reconciliation catch what the keys miss.

If your payments service has keys today, try one exercise. Kill the process right after the gateway call returns, and then retry. If the customer pays once and gets the right answer, you have idempotency. If not, you have a header.

Sources

Stripe: Idempotent requests (API reference) and Receive Stripe events in your webhook endpoint (duplicate events, retries for up to three days).
Stripe engineering blog: Designing robust and predictable APIs with idempotency.
Brandur Leach: Implementing Stripe-like Idempotency Keys in Postgres (atomic phases, recovery points, completer and reaper).
IETF: The Idempotency-Key HTTP Header Field, draft 07 (October 2025, expired).
PayPal: API idempotency (PayPal-Request-Id).
Razorpay: Payout idempotency (X-Payout-Idempotency, mandatory since 15 March 2025).
Adyen: API idempotency.
Airbnb: Rebuilding Payment Orchestration at Airbnb and "Avoiding Double Payments in a Distributed Payments System" (Orpheus).
Visa's excessive reattempts rules, as summarized by PayPal and payment processors.

Related posts. The myth of exactly-once delivery (why the effect, not the delivery, is what you can make exactly-once), Life beyond distributed transactions (entities, activities and idempotent messages), Transaction isolation levels (the check-then-act bug in SQL) and Read-your-writes and replica lag (why the key lookup goes to the primary).