Idempotency Middleware & Framework Integration

Idempotency belongs in one place, not scattered across forty handlers. This page is part of Backend Implementation & Storage Patterns, and it covers the integration layer: how to express the idempotent endpoint contract as framework middleware, where in the chain it has to sit, and the framework-specific traps — body consumption, response capture, async context, transaction boundaries — that turn a correct design into a subtly broken deployment.

The appeal of middleware is obvious: one implementation, uniformly applied, auditable in a single file. The risk is equally real. Middleware sees requests before the handler has parsed them and responses after the handler has finished with them, which is exactly the position from which it is easiest to corrupt a body, swallow an error, or capture the wrong status code.

Problem framing

Handler-level idempotency does not scale past a handful of endpoints. Every new route is a chance to forget the check, every refactor a chance to move the registration outside the transaction, and every code review a chance for a reviewer to assume someone else verified it. The failure is silent: the endpoint works perfectly until the first network partition, and then it charges a customer twice.

Middleware moves the guarantee into infrastructure. A single component reads the Idempotency-Key header, scopes it to the authenticated credential, registers it atomically, runs the handler, captures the response, and stores it. Adding a new idempotent route becomes a routing-table annotation rather than a code path. The trade-off is that the middleware now has to be correct about things the handler used to control — most importantly, what counts as a response worth storing.

Where idempotency middleware sits in the request chain A horizontal chain of layers: TLS termination, authentication, rate limiting, idempotency middleware, transaction middleware, and the route handler. An inbound arrow runs left to right through all layers. An outbound arrow returns right to left and is intercepted by the idempotency layer, which captures and stores the response before it leaves. Annotations mark that the credential is available from authentication onward and that the response must be captured after handler error mapping. request in TLS termination auth credential_id rate limit per credential idempotency register + replay capture response tx begin commit / rollback handler side effect response out capture point — after error mapping, before the bytes leave the process credential available from here on

Guarantee model

Middleware provides effectively-once execution per scoped key, with response replay, on every route it wraps — and nothing at all on routes it does not wrap. That second half matters more than teams expect. A middleware that silently passes through requests without an Idempotency-Key header gives you a system where idempotency is opt-in per caller, which is indistinguishable from no idempotency at all once one client forgets the header.

The strength of the guarantee is bounded by where the key write sits relative to the side effect:

  • Same-database key storage inside the handler transaction gives a genuinely atomic guarantee. Either the row and the charge both commit, or neither does. This is the only configuration with no crash window.
  • External key store (Redis) written before the handler gives a strong guarantee against concurrent duplicates and a bounded window for crash-after-effect. The key is marked pending; if the process dies mid-handler, the lease expires and the reconciliation job must decide what happened.
  • Key written after the handler returns gives essentially nothing. Two concurrent duplicates both execute before either writes. This is the most common broken implementation, and it passes every single-threaded test.

Under network partition, a Redis-backed middleware degrades according to its configured failure policy — the same fail-open versus fail-closed decision described in Redis cache-based deduplication. Under clock skew it is unaffected, because the key state machine is causal rather than time-ordered; only lease expiry depends on wall clocks, and that dependency is one-sided.

The five obligations of the middleware

Every correct implementation, in every framework, discharges the same five obligations in the same order. The frameworks differ only in how awkward each one is.

The five obligations of idempotency middleware Five numbered stages arranged vertically. Stage one extracts and scopes the key, branching to a 400 response if the key is missing or malformed. Stage two buffers and fingerprints the request body. Stage three registers the key atomically, branching to a replay of the stored response on a completed hit and to a 409 on a pending hit. Stage four invokes the handler and captures the response. Stage five stores the response on 2xx and 4xx, or releases the key on 5xx. 1 Extract + scope the key (credential_id, route, key) 2 Buffer + fingerprint body SHA-256 of canonical JSON 3 Register atomically the only step that must be one operation 4 Invoke + capture wrap the writer, not the handler's return 5 Store or release 2xx/4xx → store · 5xx → release 400 — missing / malformed key completed hit — replay stored body pending hit — 409 + Retry-After 5xx — delete, let the retry run

Obligation 4 is where framework differences bite hardest. The middleware must capture what the client will actually receive, which means wrapping the response writer rather than inspecting the handler’s return value. A handler that returns a domain object and lets a serialisation layer turn it into JSON has not yet produced the bytes you need to store; a handler that raises an exception mapped to 422 by an error handler has not returned anything at all.

Implementation variants

Express and Fastify — wrap the writer

Node’s res.json and res.send are the last hop before the socket, so patching them captures the real payload including anything an error handler produced.

const crypto = require('node:crypto');

function idempotency(store, { ttlMs = 86_400_000, leaseMs = 60_000 } = {}) {
  return async function (req, res, next) {
    const raw = req.get('Idempotency-Key');
    if (!raw) return next();                       // route-level policy decides if this is fatal
    if (!/^[A-Za-z0-9_-]{16,255}$/.test(raw)) {
      return res.status(400).json({ code: 'idempotency_key_malformed' });
    }

    const scoped = `idem:${req.credential.id}:${req.route.path}:${raw}`;
    const fingerprint = crypto.createHash('sha256')
      .update(JSON.stringify(req.body ?? null))    // express.json() already buffered it
      .digest('hex');

    const existing = await store.registerOrGet(scoped, fingerprint, leaseMs);
    if (existing) {
      if (existing.fingerprint !== fingerprint) {
        return res.status(422).json({ code: 'idempotency_key_payload_mismatch' });
      }
      if (existing.state === 'pending') {
        return res.set('Retry-After', '1').status(409).json({ code: 'idempotency_key_in_use' });
      }
      return res.set('Idempotent-Replay', 'true')
                .status(existing.status).json(existing.body);
    }

    // Capture the real outbound payload by wrapping json(), not by trusting the handler.
    const originalJson = res.json.bind(res);
    res.json = (body) => {
      const done = res.statusCode >= 500
        ? store.release(scoped)
        : store.complete(scoped, res.statusCode, body, ttlMs);
      done.catch((err) => req.log.error({ err, scoped }, 'idempotency finalise failed'));
      return originalJson(body);
    };
    next();
  };
}

The subtlety: res.json is patched before next(), so it is in place no matter which layer eventually produces the response — the handler, a validation middleware, or the global error handler. Patching after the handler runs would miss exactly the error paths you most want recorded.

FastAPI — a dependency plus a body cache

Starlette consumes the request stream once. A dependency that reads await request.body() must cache it, or the route’s own model parsing finds an empty stream.

import hashlib, json
from fastapi import Request, HTTPException, Response

async def idempotency_guard(request: Request) -> str | None:
    key = request.headers.get("idempotency-key")
    if key is None:
        return None
    if not (16 <= len(key) <= 255):
        raise HTTPException(400, {"code": "idempotency_key_malformed"})

    body = await request.body()          # Starlette caches this on the request object,
    request._body = body                 # so the route's Pydantic model still parses fine
    fingerprint = hashlib.sha256(body).hexdigest()
    scoped = f"idem:{request.state.credential_id}:{request.url.path}:{key}"

    hit = await store.register_or_get(scoped, fingerprint, lease_ms=60_000)
    if hit is None:
        request.state.idem_key = scoped
        return scoped
    if hit.fingerprint != fingerprint:
        raise HTTPException(422, {"code": "idempotency_key_payload_mismatch"})
    if hit.state == "pending":
        raise HTTPException(409, {"code": "idempotency_key_in_use"},
                            headers={"Retry-After": "1"})
    raise ReplayResponse(hit.status, hit.body)   # caught by an exception handler

FastAPI’s dependency system cannot see the response, so completion is finished in a separate http middleware that reads request.state.idem_key. Splitting the two halves is unavoidable; keeping them in one module is what stops them drifting apart.

Spring Boot — an interceptor plus a caching wrapper

Spring’s HandlerInterceptor gives clean pre- and post-handle hooks, and ContentCachingRequestWrapper and ContentCachingResponseWrapper solve the read-once problem without hand-rolled buffering.

public class IdempotencyInterceptor implements HandlerInterceptor {

    private final IdempotencyStore store;

    @Override
    public boolean preHandle(HttpServletRequest req, HttpServletResponse res, Object handler) {
        String key = req.getHeader("Idempotency-Key");
        if (key == null) return true;

        String scoped = "idem:%s:%s:%s".formatted(credentialOf(req), req.getRequestURI(), key);
        byte[] body = ((ContentCachingRequestWrapper) req).getContentAsByteArray();
        String fingerprint = sha256Hex(body);

        Optional<Record> hit = store.registerOrGet(scoped, fingerprint, Duration.ofSeconds(60));
        if (hit.isEmpty()) {
            req.setAttribute("idem.key", scoped);
            return true;                                   // proceed to the controller
        }
        Record r = hit.get();
        if (!r.fingerprint().equals(fingerprint)) { write(res, 422, MISMATCH); return false; }
        if (r.isPending())                       { write(res, 409, IN_USE);   return false; }
        res.setHeader("Idempotent-Replay", "true");
        write(res, r.status(), r.body());
        return false;                                      // short-circuit the controller
    }

    @Override
    public void afterCompletion(HttpServletRequest req, HttpServletResponse res,
                                Object handler, Exception ex) {
        String scoped = (String) req.getAttribute("idem.key");
        if (scoped == null) return;
        byte[] out = ((ContentCachingResponseWrapper) res).getContentAsByteArray();
        if (res.getStatus() >= 500) store.release(scoped);
        else store.complete(scoped, res.getStatus(), out, Duration.ofHours(24));
    }
}

afterCompletion runs even when the controller threw, which is exactly the property that makes the 5xx release path reliable.

Go — one http.Handler, one wrapped ResponseWriter

Go’s composable handler signature makes the whole thing a single self-contained type, and a ResponseWriter wrapper captures status and body without any framework support.

type capture struct {
    http.ResponseWriter
    status int
    buf     bytes.Buffer
}

func (c *capture) WriteHeader(code int) { c.status = code; c.ResponseWriter.WriteHeader(code) }
func (c *capture) Write(p []byte) (int, error) {
    if c.status == 0 { c.status = http.StatusOK }
    c.buf.Write(p)                       // cap this in production: 64 KB is plenty
    return c.ResponseWriter.Write(p)
}

func Idempotency(store Store) func(http.Handler) http.Handler {
    return func(next http.Handler) http.Handler {
        return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
            key := r.Header.Get("Idempotency-Key")
            if key == "" { next.ServeHTTP(w, r); return }

            body, _ := io.ReadAll(io.LimitReader(r.Body, 1<<20))
            r.Body = io.NopCloser(bytes.NewReader(body))          // hand it back to the handler
            scoped := fmt.Sprintf("idem:%s:%s:%s", CredentialFrom(r), r.URL.Path, key)
            fp := fmt.Sprintf("%x", sha256.Sum256(body))

            hit, err := store.RegisterOrGet(r.Context(), scoped, fp, 60*time.Second)
            if err != nil { http.Error(w, "dedup store unavailable", 503); return }
            if hit != nil { replay(w, hit, fp); return }

            c := &capture{ResponseWriter: w}
            next.ServeHTTP(c, r)
            if c.status >= 500 {
                _ = store.Release(context.WithoutCancel(r.Context()), scoped)
            } else {
                _ = store.Complete(context.WithoutCancel(r.Context()), scoped,
                                   c.status, c.buf.Bytes(), 24*time.Hour)
            }
        })
    }
}

context.WithoutCancel matters: by the time the handler has returned, a client disconnect may already have cancelled the request context, and the finalisation write would fail exactly when the key most needs to be recorded.

Framework Body buffering Response capture Runs on handler panic / exception Notable trap
Express / Fastify express.json() already buffers Patch res.json / res.send before next() Only if the error handler emits through the patched writer Patching after next() misses error paths
FastAPI await request.body() then reassign request._body Separate http middleware reading request.state Yes, middleware wraps the whole app Dependency cannot see the response — split in two
Spring Boot ContentCachingRequestWrapper ContentCachingResponseWrapper in afterCompletion Yes, afterCompletion always runs Wrappers must be installed by a servlet filter, not the interceptor
Go net/http io.ReadAll + reset r.Body Wrapped ResponseWriter No — use defer plus recover Request context is cancelled on client disconnect

Edge cases and failure scenarios

Failure Scenario Remediation Steps Observability Hooks
Middleware buffers the request body, and a client posts a 200 MB file to an idempotent route, exhausting heap on every worker Wrap the read in a limit — io.LimitReader(r.Body, 1<<20) in Go, express.json({ limit: '1mb' }) in Node — and return 413 Payload Too Large on idempotent routes rather than growing the buffer. Route large uploads to a non-idempotent path that dedupes on an object-store key instead. idempotency_body_bytes histogram, alert if p99 exceeds 256 KB; idempotency_body_rejected_total counter by route
Handler streams a chunked response; the middleware buffers it to store, so the client sees nothing until the stream ends Exclude streaming routes from the middleware at registration time and assert the exclusion in a test. For those routes, store only a completion marker plus the resource id, and let the client re-fetch with a GET. idempotency_streamed_route_wrapped_total counter (should always be zero); a startup assertion that logs every wrapped route and its response mode
Client disconnects immediately after the handler commits; the request context is cancelled, the complete write fails, and the key is left pending until its lease expires Finalise with a context detached from cancellation (context.WithoutCancel, or a fresh 5-second context) and retry the write once. Treat a failed finalise as a paging-level error, not a debug log. idempotency_finalise_error_total counter, alert on rate above 0.05% of requests; idempotency_pending_age_seconds gauge alerting above 120 s
Two middlewares are registered — one from a shared platform library, one added locally — and the inner one always sees a key already marked pending by the outer one, so every request returns 409 Register the middleware exactly once, at the application root, and log the full middleware chain on startup. Add an integration test that asserts a single successful create for a fresh key. Startup log line enumerating the chain; idempotency_conflict_total with a stage label — a 100% conflict rate on first requests is the signature
Key store write succeeds but the handler’s database transaction rolls back, leaving a completed record for an effect that never happened Move the key write inside the handler transaction when both live in the same database. Where they cannot share a transaction, write the key as pending before the handler and only complete it after the commit returns successfully — never on handler return alone. idempotency_phantom_completion_total from a reconciliation job comparing completed keys to business rows; alert on any non-zero value
Roll the middleware out in shadow mode first A timeline in three phases. In shadow mode the middleware registers keys and records what it would have deduplicated but never short-circuits a request. After a week the recorded hit rate is compared against known client retry behaviour. Only then is enforcement enabled, so a large client reusing keys across payloads is discovered before it becomes an outage. 1 · shadow, 7 days register, record, never reject 2 · compare shadow hits vs known retry rate 3 · enforce 409 and 422 go live no surprises Enabling enforcement on day one is an outage you chose to have.

Operational concerns

TTL and lease separation. Two timers, not one. The lease bounds how long a key may sit pending — 60 seconds is a reasonable default, and it must be shorter than the client’s total retry budget so a crashed request does not lock the key past the client’s last attempt. The TTL bounds how long a completed record is replayable, and 24 hours matches the published contract. Configuring one value for both means either leases that outlive the client or completed records that vanish mid-retry.

Memory budgeting. The middleware stores a status code, a fingerprint, and a response body per key. At 2,000 idempotent requests per second with a 24-hour TTL and a 400-byte average record, that is roughly 69 GB of live data — which is a sharded Redis deployment, not a sidecar. Capping stored bodies at 64 KB and storing a resource pointer for anything larger keeps the tail from dominating.

Index strategy. For a PostgreSQL-backed store, one unique index on (credential_id, route, key) serves registration and replay; a separate B-tree on expires_at serves the reaper, which should delete in batches of 10,000 with a LIMIT-bounded DELETE ... WHERE ctid IN (...) to avoid long-held locks. Both are covered in idempotency key storage & TTL management.

Alert thresholds. Page on idempotency_finalise_error_total above 0.05% of wrapped requests — that is the metric that predicts stuck keys. Warn when the replay rate leaves its 0.5%–2% baseline band in either direction: a spike means a client retry loop, and a collapse to zero usually means the middleware stopped being applied after a routing change. Track idempotency_store_latency_ms separately from handler latency so a slow dedup store is visible as its own problem rather than as diffuse endpoint slowness.

Rollout. Deploy the middleware in shadow mode first: register keys, record what would have been deduplicated, but never short-circuit. Run for a week, compare idempotency_shadow_hit_total against known client retry behaviour, and only then enable enforcement. A middleware that starts rejecting traffic on day one because a large client reuses keys across payloads is an outage you chose to have.

FAQ

Where in the middleware chain should idempotency sit?

After authentication and rate limiting, before the route handler and before any transaction-management middleware. It needs the authenticated credential to scope the key — an unscoped key namespace lets one tenant replay another’s responses — and it must own the outermost boundary of the handler so it can capture the final response after all handler-level error mapping has run.

Should idempotency middleware share the request’s database transaction?

If the key store and the business data are in the same database, yes. Writing the key inside the handler’s transaction makes the record and the side effect atomic and removes the orphaned-pending failure mode entirely. If the key lives in Redis and the data in PostgreSQL, they cannot share a transaction, so the key must be written first as pending and reconciled after a crash — the same two-store consistency problem that wrapping database transactions for safe retries works through.

How do I make middleware idempotency work with streamed responses?

You mostly do not. Buffering a streamed response to store it defeats the point of streaming and can exhaust memory on a single large export. Exclude streaming routes from the middleware and give them an application-level check that stores only a completion marker and a resource identifier, letting the client re-fetch the stream through a normal GET.

Does idempotency middleware need to read the request body?

Yes, if you want payload binding — the fingerprint that catches a client reusing a key with different content. That means buffering the body before the handler consumes it, which every framework supports but few do by default. Cap the buffered size at 1 MB and reject larger bodies on idempotent routes rather than streaming them past the fingerprint.

Can the middleware replace a distributed lock around the handler?

For deduplication, yes — the atomic registration in obligation 3 already serialises concurrent duplicates of the same key. It does not serialise different requests that touch the same resource; two distinct keys crediting the same account still race. That is a separate concern handled by optimistic or pessimistic concurrency control on the resource itself.