A customer clicks "place order" once. Their phone loses signal for two seconds as the request goes out. The client retries automatically. The server, which never saw the first request fail, processes both. Now there are two orders, two charges, and a support ticket that starts with "why was I billed twice."
Nobody wrote a bug to make this happen. It emerges from unreliable networks, retry logic that exists for good reasons, and an API with no concept of "I already did this." Fixing it after the fact, incident by incident, is expensive and never quite complete. Designing for it up front, with a real idempotency key strategy, closes the whole class of problem at once.
Why Retries Are Unavoidable, Not a Bug to Chase Out
Every layer between a client and a server can decide, on its own, to retry a request. Mobile networks drop packets mid-flight. Load balancers time out a slow backend and reissue the call. Client SDKs built with exponential backoff retry on anything that looks like a transient failure, including a perfectly successful request whose response never made it back. None of this is misbehavior. It is the correct response to an unreliable network, and removing it would make the system more fragile, not less.
The actual failure is on the server side: treating every incoming request as new, with no memory of what it already processed. A payment endpoint that charges a card every time it receives a well-formed charge request will happily charge it three times if three retries arrive. The fix is not to stop retries from happening. It is to make the server safe to call more than once with the same intent.
What Idempotency Actually Means at the API Level
The term gets used loosely, so it helps to separate two guarantees. HTTP already defines some methods as idempotent by convention: a PUT or DELETE should produce the same end state no matter how many times it runs, while POST is explicitly not idempotent, because creating a resource twice creates two resources. That convention, documented in the IETF's HTTP semantics specification at datatracker.ietf.org, covers method semantics, not application intent.
An idempotency key strategy solves a narrower problem: making a specific POST safe to retry by attaching a client-generated token that represents "this exact attempt." The server's job shifts from "process every request" to "process this token exactly once, and hand back the same result for any repeat." The concept traces to the mathematical definition of idempotence, summarized on Wikipedia, where applying an operation multiple times produces the same result as applying it once.
Photo by Franck V. on Unsplash
Where Duplicate Requests Actually Come From
Before designing the mechanism, it helps to know what it defends against, because the sources are more varied than "flaky wifi."
Client-side retry libraries are the most common source. Most modern HTTP clients retry on connection resets, timeouts, and 5xx responses by default, often with three attempts and exponential backoff baked in. That is good defensive engineering on the client, and it means your server will see duplicates as a matter of course, not an edge case.
Webhook senders are another major source. Payment processors, messaging platforms, and most SaaS webhook systems will redeliver an event if your endpoint doesn't return a 200 within their timeout window, or if it returns any error status. A handler that is slow under load, even briefly, can process the same event two or three times.
Proxies and load balancers add a third layer. A request that times out at the load balancer before the origin server responds gets retried by the balancer itself, invisibly to both client and origin. And on the human side, users double-click submit buttons or refresh mid-request, generating genuine duplicate intent rather than a network-level retry.
Designing the Idempotency Key Itself
The key is a token the client generates and attaches to the request, either as a header (Idempotency-Key: <uuid>) or a body field, depending on what fits your API's conventions. A v4 UUID generated fresh for each logical user action, and reused across every retry of that action, is the standard approach.
Scope matters as much as generation. A key should be unique per logical operation, not per endpoint globally. "Create this specific order" needs its own key; a second, unrelated order from the same user five minutes later needs a different one. Some teams scope keys per user plus endpoint plus a client-supplied nonce, which prevents a collision between two users who happen to generate the same UUID.
Payload matching is the detail teams skip and later regret. If a client sends the same key with a different request body, that is not a legitimate retry. It is a client bug or a key reuse mistake, and the server should treat it as a conflict rather than silently returning a stale response for a request that was never actually made.
Photo by Mahdi Bafande on Pexels
Storing and Checking Keys Without Introducing a New Bottleneck
The server needs a place to record "I have seen this key" and what it did about it. A dedicated table with the key as a unique-constrained column works well for most systems: one row per key, storing the request hash, response status and body, and a timestamp. On each request, the server checks for an existing row first, returns the stored response on a match, or does the work and writes the row in the same transaction otherwise.
Some teams reach for an in-memory store like Redis for the initial check, since lookups need to be fast on every request, not just retries. That works as a first line of defense, but durability matters too: if idempotency records live only in memory and the process restarts, a retry that arrives after the restart looks brand new and gets processed again. A relational store with a unique constraint, such as one built on PostgreSQL, gives you fast lookup and durability together, and the constraint does double duty resolving the next problem: the race condition.
Handling the Race Window Between Two Simultaneous Retries
Here is the failure mode that trips up a surprising number of idempotency implementations: two requests carrying the same key arrive close enough together that both check "does this key exist" before either has written its row. Both see no match. Both proceed to do the real work, and the idempotency layer fails at the exact moment it was supposed to matter most.
The fix is to make the check-and-reserve step atomic, not just fast. Insert a row for the key, marked "in progress," as the very first thing that happens, using the database's unique constraint to guarantee only one such insert can succeed. If the insert fails because the key already exists, the second request knows a first attempt is finished or running, and it can wait briefly and return the completed result, or return a 409 if the first attempt is still in flight. This turns a race condition into a database-level guarantee, the difference between a system that is idempotent under load and one that merely looks idempotent in a manual test.
Deciding What Happens on a True Conflict
Not every repeated key is a friendly retry. When the stored request hash doesn't match the incoming payload for the same key, the server has to pick a deliberate behavior rather than whatever the code happens to do by accident. Rejecting the request with a 409 and a clear error message is usually right. Silently processing the new payload under an old key hides a real client bug, and silently returning the old cached response for a genuinely different request produces confusing, hard-to-trace behavior.
Stripe popularized the Idempotency-Key header pattern precisely because payment retries are both common and costly to get wrong. Its approach of storing the full response for a fixed retention window, rather than forever, is a reasonable default: long enough to cover realistic retry windows, short enough not to accumulate unbounded storage.
"The teams that get burned by duplicate processing almost never lack the engineering skill to fix it. They just didn't think about retries until the incident review. Idempotency is one of those things that costs almost nothing to design in from day one and costs a lot to retrofit after a customer notices." - Dennis Traina, founder of 137Foundry
Photo by panumas nikhomkhai on Pexels
Idempotency for Webhooks Looks a Little Different Than Client APIs
Webhook consumers face the same risk from the opposite direction: you don't control when the sender retries, and most senders already include their own event ID in the payload, usable as the idempotency key instead of asking the sender to generate one. The pattern is otherwise identical: check the event ID before processing, insert a row atomically, and treat a repeat delivery as a no-op that still returns success.
The one meaningful difference is timing. Webhook senders typically expect a fast acknowledgment, often under 10 seconds, and treat a timeout as a delivery failure worth retrying even if your handler would have eventually succeeded. That pushes teams toward acknowledging receipt immediately and processing asynchronously in a background job.
Testing an Idempotency Layer for Real, Not Just on Paper
An idempotency check that has never been tested under concurrency is a check you don't actually know works. A unit test that sends one request, then the same request again, proves the happy path and nothing about the race window. A better test fires two or more identical requests concurrently with the same key and asserts that exactly one did real work, with the others returning either the cached result or a clear in-progress response. It is the test most teams skip because it needs more setup than a sequential call, and it is worth the setup: a production incident caused by a race condition your suite never simulated is a far more expensive way to find the same bug.
Building This Into a New API vs. Retrofitting an Existing One
Designing idempotency in from the start is straightforward: pick the header or field convention, add the dedup table, wire the atomic insert into the request handler. Retrofitting it onto an API with existing clients that send no key at all takes more care. A reasonable rollout treats the key as optional at first, logging when it's missing while client teams add key generation, then makes it required for endpoints where duplicate processing carries real cost, like payments and order creation.
Photo by 飞 谢 on Pexels
Where a Data Integration Partner Fits In
Idempotency strategy touches API design, database schema, and often a background job system at once, which is exactly the kind of cross-cutting work that's easy to postpone because no single feature ticket owns it. 137Foundry's data integration service has designed this pattern into payment flows, order systems, and webhook consumers across client platforms, usually as part of a broader reliability pass. The web development service team can also assess where idempotency gaps sit relative to your existing API surface. See the full services hub or the 137Foundry homepage for more.
The Short Version
Retries are a permanent feature of networked systems, not a bug that better infrastructure will eventually remove. An idempotency key strategy accepts that reality: a client-generated key scoped to one logical operation, an atomic check-and-reserve step backed by a database unique constraint to close the race window, a clear conflict response when a key is reused with a different payload, and a retention window long enough to cover real-world retry patterns. None of the pieces are exotic. The value comes from putting them in place before the first double-charge complaint arrives, not after.
