API Contract Design for Idempotent Endpoints

Most idempotency incidents are not storage bugs. They are contract bugs: the client and the server disagreed about what the key meant, how long it lived, or what a repeated call was supposed to return. This page is part of Idempotency Fundamentals & API Guarantees, and it covers the layer above the deduplication store — the wire contract that makes an endpoint safely retryable by a caller who has no idea whether the previous attempt landed.

The contract is a small set of decisions, but each one is load-bearing. Get the key syntax wrong and you inherit collisions from a careless client. Skip request fingerprinting and a client that mutates its payload between retries silently overwrites the first result. Return the wrong status code on conflict and every SDK in the ecosystem treats a safe duplicate as a hard failure.

Problem framing

An HTTP client that times out has learned exactly one thing: it did not receive a response. It has learned nothing about whether the server executed the request. The TCP connection may have died before the request left the client, after the server committed the transaction, or anywhere between. This is the fundamental asymmetry that at-least-once delivery forces on every caller, and the only honest response is to retry.

Retrying is safe for GET, PUT and DELETE because their method semantics are already idempotent. It is not safe for POST, which is exactly the method used for the operations that hurt most when duplicated: charging a card, issuing a refund, submitting an order, provisioning a resource. The contract described here is what turns an unsafe POST into one a client can retry with the same confidence it retries a GET.

What a client timeout does and does not tell you A client sends a POST request. Three parallel outcomes are shown: the request never reached the server, the request executed but the response was lost in transit, and the request executed with the response delivered. Only the third case is distinguishable from the client side; the first two are identical from the client's point of view. Client POST /charges A — never arrived no side effect B — executed, reply lost card already charged C — executed, reply seen client knows the outcome Indistinguishable A and B look the same to the client: timeout

The contract’s job is to make outcome B safe to retry. Nothing else in the stack can do it, because only the server knows that it already executed this exact request.

Guarantee model

An idempotent endpoint contract promises effectively-once execution of the side effect, and exact replay of the original response, for all requests carrying the same key within the retention window. It does not promise exactly-once delivery — the network still delivers whatever it delivers. It promises that the nth delivery has the same observable effect as the first.

Three properties make that promise precise:

  • Effect uniqueness. Within the retention window, at most one execution of the handler produces side effects for a given key. Concurrent duplicates are rejected, not queued.
  • Response stability. A replay returns the byte-identical status code and body the first execution produced, including any server-generated identifiers. A client that lost the response to a charge must be able to recover the charge ID by retrying.
  • Payload binding. A key is bound to the request that created it. Reusing the key with different content is a client bug, and the server must surface it rather than silently honour either version.

The guarantee breaks in three places, and an honest contract documents all three. It breaks after the window expires, when the same key becomes a fresh key and a very late retry executes a second time. It breaks across scopes if the key namespace is not partitioned per API credential, letting one tenant’s key collide with another’s. And it breaks when the side effect leaves the transaction — if the handler charges an external processor and then crashes before recording the key, the deduplication record and the side effect have diverged. That last case is what the transactional outbox pattern exists to close.

The request lifecycle state machine

The contract is easiest to specify as a state machine over a single key. Every request either creates the key in a pending state or observes a key that already exists, and the observed state fully determines the response.

Idempotency key lifecycle state machine States: Absent, Pending, Completed and Failed. A first request moves a key from Absent to Pending by an atomic insert. Completion moves it to Completed and stores the response. A concurrent duplicate observing Pending receives 409 with retry-after. A duplicate observing Completed receives the stored response replayed. A handler error moves the key to Failed and releases it so the client may retry. absent no record pending handler running completed response stored failed key released INSERT atomic, unique commit 5xx / rollback duplicate → 409 in_use duplicate → replay 201 key deleted — retry executes fresh

The transition from absent to pending is the only one that must be atomic; every other transition is an update to a row the request already owns. That is why the contract is implementable on top of a Redis SET NX registration or a PostgreSQL unique constraint without any change to the wire semantics.

Note the failed transition. A 5xx response must release the key, because the client is expected to retry and the retry must be allowed to execute. A 4xx response is different: the request was well-formed enough to be judged and will fail identically on retry, so the contract should store it as a completed response and replay it. Releasing keys on 4xx turns a client bug into an unbounded retry loop against your validation layer.

Specifying the header

Four headers carry the contract. Only the first is sent by the client.

POST /v1/charges HTTP/1.1
Host: api.example.com
Idempotency-Key: 01HXYZ3ABCDEFGHJKMNPQRSTVW
Content-Type: application/json

{"amount": 4500, "currency": "gbp", "source": "card_1a2b3c"}

HTTP/1.1 201 Created
Idempotent-Replay: false
Idempotency-Key-Expires: 2026-08-02T09:14:33Z
Content-Type: application/json

{"id": "ch_9f8e7d", "amount": 4500, "status": "succeeded"}

The rules that make this unambiguous:

  1. Scope the key. The lookup key is the tuple (api_credential_id, endpoint, idempotency_key), never the raw header value. Without the credential in the tuple, one tenant sending key-1 can shadow another tenant’s key-1. Without the endpoint, a client that reuses one request ID across a create and a cancel call gets the wrong replay.
  2. Bound the syntax. Accept 16 to 255 characters from the set [A-Za-z0-9_-]. Reject shorter keys with 400 Bad Request and error code idempotency_key_too_short — a 4-character key from a naive client is a collision waiting to happen, and rejecting it is cheaper than debugging the cross-customer replay it eventually causes. ULIDs and UUIDv7 both fit comfortably inside these bounds.
  3. Declare where it is required. Make the key mandatory on every state-changing POST that moves money or provisions a resource, optional on cheap creates, and forbidden on GET. Returning 400 for a missing key on a payment endpoint is far better than silently accepting an unretryable charge.
  4. Advertise the window. Idempotency-Key-Expires gives clients an absolute timestamp rather than making them infer the retention period from documentation that drifts.
  5. Flag the replay. Idempotent-Replay: true lets a client distinguish “I created this” from “this already existed” without changing the status code — which matters because changing the status code breaks the SDK contract for everyone who wrote if status == 201.

Implementation variants

Three contract shapes are in common production use. They differ in who supplies the uniqueness and how much trust the server places in the client.

Comparison matrix of three idempotent endpoint contract variants A three-column, four-row matrix. Columns are the header key, payload fingerprint, and pre-allocated token variants. Rows compare client burden, collision risk, replay fidelity, and typical use. The header key variant has moderate client burden, low collision risk, exact replay fidelity, and suits payments. The fingerprint variant has no client burden, medium collision risk, exact replay fidelity, and suits internal service calls. The token variant has high client burden, no collision risk, exact replay fidelity, and suits regulated ledger writes. Header key Payload fingerprint Pre-allocated token Client burden generate + store key none extra round trip Collision risk low (client entropy) medium — identical bodies dedupe wrongly none — server issued Payload binding needs fingerprint too intrinsic bound at issue time Typical use public payment APIs internal RPC ledger / regulated Darker cell = stronger on that row

Variant A — client-supplied header key

The default, and the one Stripe, Adyen and most payment APIs expose. The client mints a key, stores it alongside its own intent record, and resends the identical key on every retry until it gets a terminal response.

import httpx, ulid

def charge(amount_minor: int, currency: str, source: str, key: str | None = None) -> dict:
    # The key is minted ONCE per business intent and persisted by the caller,
    # not regenerated per attempt — regenerating defeats the whole mechanism.
    key = key or str(ulid.ULID())
    for attempt in range(5):
        try:
            r = httpx.post(
                "https://api.example.com/v1/charges",
                headers={"Idempotency-Key": key},
                json={"amount": amount_minor, "currency": currency, "source": source},
                timeout=10.0,
            )
        except httpx.TransportError:
            continue                      # safe: same key on the next attempt
        if r.status_code == 409 and r.json().get("code") == "idempotency_key_in_use":
            continue                      # first attempt still running
        return r.json()                   # 2xx or a terminal 4xx
    raise RuntimeError("charge unresolved after 5 attempts")

Variant B — server-derived payload fingerprint

No header. The server hashes a canonical serialisation of the request and uses the digest as the key. This removes all client burden, which makes it attractive for internal service-to-service calls where you control both ends and cannot retrofit headers into fifty callers.

func fingerprint(tenantID, route string, body []byte) string {
    var canon any
    _ = json.Unmarshal(body, &canon)          // reject non-JSON upstream
    stable, _ := json.Marshal(canon)          // Go marshals map keys sorted
    h := sha256.New()
    h.Write([]byte(tenantID))
    h.Write([]byte{0x1f})                     // unit separator: no field-boundary ambiguity
    h.Write([]byte(route))
    h.Write([]byte{0x1f})
    h.Write(stable)
    return hex.EncodeToString(h.Sum(nil))
}

Its weakness is semantic, not technical: two genuinely distinct requests that happen to be byte-identical — a customer buying the same coffee twice in one minute — collapse into one. Use it only where repeated identical intent is impossible, or add a caller-supplied nonce to the hashed material and you have quietly reinvented variant A.

Variant C — pre-allocated resource token

The client calls POST /v1/charges/intents first, receives a server-issued intent_id, and then submits the charge against that ID. The uniqueness comes entirely from the server, so a confused or malicious client cannot manufacture a collision.

-- Step 1: allocate. Cheap, no side effects, safe to repeat.
INSERT INTO charge_intents (id, tenant_id, state, created_at)
VALUES (gen_random_uuid(), $1, 'allocated', now())
RETURNING id;

-- Step 2: consume. The state guard makes execution single-shot.
UPDATE charge_intents
   SET state = 'consumed', consumed_at = now()
 WHERE id = $1 AND tenant_id = $2 AND state = 'allocated'
RETURNING id;   -- zero rows means it was already consumed: replay the stored response

The extra round trip is the price, and in regulated ledger systems it is usually worth paying because the allocation record is itself an auditable artefact.

Variant Key source Payload binding Extra round trip Best fit
A — header key Client Explicit fingerprint required No Public APIs, payment gateways, partner integrations
B — payload fingerprint Server (hash) Intrinsic to the hash No Internal RPC, event handlers, migration backfills
C — pre-allocated token Server (allocation) Bound when the token is issued Yes Ledger writes, regulated workflows, high-value transfers

Edge cases and failure scenarios

Failure Scenario Remediation Steps Observability Hooks
Client reuses one key across two genuinely different payloads — a retry sends amount: 4500 after the original sent amount: 4000 Store a SHA-256 fingerprint of the canonicalised body next to the key. On mismatch, return 422 Unprocessable Entity with code idempotency_key_payload_mismatch and execute nothing. Never honour either version silently. idempotency_payload_mismatch_total counter, labelled by tenant_id and route; alert when a single tenant exceeds 5 in 10 minutes — it means a broken client, not a broken network
Two retries arrive within the same millisecond and both observe an absent key, so both attempt to execute Make the absent → pending transition a single atomic operation — SET key value NX PX 86400000 or an INSERT protected by a unique index — and let the loser read the winner’s row. Never check-then-write across two round trips. idempotency_concurrent_conflict_total counter; idempotency_registration_duration_ms histogram, alert if p99 exceeds 15 ms
Handler crashes after charging the payment processor but before writing the completed response, leaving the key stuck in pending until its lease expires Bound pending with a lease shorter than the client’s total retry budget — 60 seconds is a workable default — and reconcile orphans against the processor by the key before releasing. See handling stale locks in distributed systems. idempotency_pending_age_seconds gauge, alert above 120 seconds; idempotency_orphan_reconciled_total counter
A client retries 30 hours after the original call, past the 24-hour window, and the endpoint executes a second charge Publish the window in Idempotency-Key-Expires and reject requests whose key has expired but whose payload matches a recently archived record, returning 409 idempotency_key_expired rather than executing. Archive completed keys for 90 days for exactly this lookup. idempotency_expired_replay_total counter; alert on any non-zero value in a payments context
Key namespace is not scoped per credential, so two tenants using sequential integers as keys replay each other’s responses Make the storage key (credential_id, route, key) and add a migration that rewrites existing rows. Treat any cross-tenant replay as a security incident, not a bug. idempotency_cross_tenant_hit_total counter (should be structurally impossible — alert on any value); audit log field idempotency_scope on every registration

Operational concerns

Retention. 24 hours of live keys is the working standard. Size the store from peak request rate: at 2,000 idempotent writes per second, a 24-hour window holds roughly 173 million keys. At an average of 180 bytes per record — key, fingerprint, status, small response body — that is about 31 GB before index overhead, which is a Redis cluster decision rather than a single-node one. Choosing TTL values for idempotency keys works through the sizing arithmetic in detail.

Response body limits. Cap the stored response at 64 KB and store a pointer for anything larger. An endpoint that returns a 4 MB report and stores every response verbatim will exhaust its dedup store long before it exhausts its database.

Index strategy. Whatever the backing store, the primary access path is an exact-match lookup on the scoped tuple plus a range scan on expires_at for the reaper. A single composite index on (credential_id, route, key) and a separate B-tree on expires_at covers both. Do not add a covering index on the response body column; it doubles the write cost for a read path that always fetches the whole row anyway.

Alert thresholds. Three signals are worth paging on. A idempotency_store_error_total rate above 0.1% of requests means the endpoint is about to start behaving non-idempotently, because most implementations fail open. A replay rate that jumps from its baseline — typically 0.5% to 2% of traffic — to above 10% means a client is stuck in a retry loop. And a p99 registration latency above 15 ms means the dedup store has become the endpoint’s bottleneck rather than its safety net.

Degradation policy. Decide, in advance and in writing, what happens when the dedup store is unreachable. Failing open accepts duplicate charges; failing closed rejects legitimate traffic. For money movement, fail closed and return 503 with Retry-After. For low-value creates, fail open and record the gap so reconciliation can catch it later. The one unacceptable answer is not having decided.

FAQ

Should the client or the server generate the idempotency key?

The client must generate it. The whole point of the key is to survive a response the client never saw — if the server minted it, the client would have no stable value to resend after a timeout. Specify a client-generated key of 16 to 255 characters and reject anything outside those bounds with 400. Variant C is the deliberate exception: there the server issues the token in a separate, safely repeatable call, so the client still holds a stable value before the risky operation begins.

What status code should a replayed request return?

Return the original status code and body verbatim — a replayed POST that first returned 201 Created should return 201 Created again, plus an Idempotent-Replay: true header so the client can distinguish a replay from a first execution. Reserve 409 Conflict for the case where the same key arrives while the first attempt is still in flight, and 422 for a payload that disagrees with the fingerprint on record.

How long should an idempotency key stay valid?

24 hours is the industry-standard retention window and comfortably exceeds any sane client retry schedule. Publish the window in your API reference and return the absolute expiry in an Idempotency-Key-Expires header so clients can reason about when a key becomes reusable. Archive expired keys — without their response bodies — for 90 days so a very late retry can be answered with 409 idempotency_key_expired instead of a second execution.

What should happen when two requests with the same key arrive concurrently?

The second request must not execute. Return 409 Conflict with a machine-readable error code such as idempotency_key_in_use, and set Retry-After to a small value like 1 second. Blocking the second caller until the first completes is the wrong default: it converts a duplicate into a held connection and amplifies load during a retry storm, exactly the dynamic described in mitigating thundering herd during retry storms.

Does an idempotent contract need a distributed lock?

No, and reaching for one is usually a sign the atomic registration step was skipped. A single conditional write — SET NX or a unique-index INSERT — already serialises the absent → pending transition. Distributed lock acquisition patterns become relevant only when the handler itself must coordinate across several resources, which is a different problem from deduplicating the request.