Choosing TTL Values for Idempotency Keys

Part of: Idempotency Key Storage & TTL Management

An idempotency key’s TTL is not a caching decision — it is a correctness decision. Set it too short and a client’s legitimate retry arrives after the deduplication record has expired, so the request is processed a second time as if it were new. Set it too long and you accumulate storage bloat in your Redis or PostgreSQL dedup store and risk rejecting a legitimately reused key as a false duplicate. This page derives a concrete TTL from three measurable inputs rather than picking a round number and hoping.

This builds on Idempotency Key Storage & TTL Management and assumes you already have a working deduplication check, such as Redis SETNX-based deduplication, in front of your write path. The same sizing discipline applies to lock lease TTLs — both are windows that must outlive every legitimate retry without outliving it by so much that stale state accumulates.

---

The three inputs to a TTL calculation

1. Client retry budget — the longest span of time your client-side retry logic (including exponential backoff, circuit breakers, and any manual “click retry” affordance a human might use) could plausibly still send the original request. For an SDK with capped exponential backoff this might be 5 minutes; for a checkout flow where a customer emails support and support manually retries, it can stretch to 12–24 hours.

2. Broker redelivery window — if the request path involves a message broker, the maximum time a message can sit unacknowledged before redelivery. SQS visibility timeout, Kafka consumer group session timeout plus rebalance delay, or RabbitMQ’s dead-letter TTL all bound this.

3. Reconciliation cadence — how often a background job re-scans for stuck or ambiguous records (a PROCESSING row that never resolved) and either retries or marks them failed. The TTL must outlive at least one full reconciliation cycle so the job has a chance to run before the record disappears.

TTL = max(client_retry_budget, broker_redelivery_window) + reconciliation_cadence + safety_margin

For a payment capture endpoint with a 4-hour client retry budget (support-assisted retries), no broker in the critical path, and a reconciliation job that runs every 1 hour:

TTL = max(4h, 0) + 1h + 20h_safety_margin ≈ 24 hours

For an internal service-to-service call with a 30-second SDK retry budget and a Kafka redelivery window of 2 minutes:

TTL = max(30s, 120s) + 5min_reconciliation + 5min_margin = 12 minutes

The diagram below lines these windows up on a single timeline so you can see why the TTL must be the outer envelope, not the average.

---

Step-by-step implementation

Step 1 — Measure the client’s real retry budget

Instrument client SDKs to log every retry attempt with its elapsed time since the first attempt, then look at the p99 across production traffic rather than the theoretical backoff schedule.

import time

class RetryTracker:
    def __init__(self, request_id: str):
        self.request_id = request_id
        self.first_attempt_at = time.monotonic()

    def log_retry(self, attempt_number: int):
        elapsed = time.monotonic() - self.first_attempt_at
        # Emit to your metrics pipeline; aggregate p99 elapsed time weekly.
        print(f"retry request_id={self.request_id} attempt={attempt_number} elapsed_s={elapsed:.1f}")

Step 2 — Measure the broker redelivery window

Pull the configured visibility timeout or session timeout directly from the broker rather than assuming defaults.

// SQS: visibility timeout is the hard ceiling on redelivery
input := &sqs.GetQueueAttributesInput{
    QueueUrl:       aws.String(queueURL),
    AttributeNames: []types.QueueAttributeName{"VisibilityTimeout"},
}
out, err := sqsClient.GetQueueAttributes(ctx, input)
if err != nil {
    log.Fatal(err)
}
visibilityTimeoutSec := out.Attributes["VisibilityTimeout"] // e.g. "120"

Step 3 — Compute the TTL and configure Redis

Apply the formula from above, then set the TTL atomically with the key write using SET ... EX (never a separate EXPIRE call, which leaves a window where the key exists without an expiry).

import redis

r = redis.Redis(host="localhost", port=6379, decode_responses=True)

def store_idempotency_key(key: str, response_snapshot: str, ttl_seconds: int = 86_400):
    # 86_400s = 24h, derived from the payment-endpoint formula above.
    was_set = r.set(f"idem:{key}", response_snapshot, ex=ttl_seconds, nx=True)
    return was_set  # False means the key already existed — return the cached response

Step 4 — Configure a backing expiry job in PostgreSQL

If PostgreSQL is your system-of-record store rather than Redis, TTL is not native — enforce it with an indexed expires_at column and a scheduled sweep.

CREATE TABLE idempotency_keys (
    key         TEXT PRIMARY KEY,
    response    JSONB NOT NULL,
    created_at  TIMESTAMPTZ NOT NULL DEFAULT now(),
    expires_at  TIMESTAMPTZ NOT NULL
);

CREATE INDEX idx_idempotency_expires_at ON idempotency_keys (expires_at);

-- Run every 15 minutes via pg_cron or an external scheduler
DELETE FROM idempotency_keys WHERE expires_at < now();

-- pg_cron registration, run every 15 minutes
SELECT cron.schedule(
    'expire-idempotency-keys',
    '*/15 * * * *',
    $$DELETE FROM idempotency_keys WHERE expires_at < now()$$
);

Step 5 — Encode the TTL decision, don’t hardcode it per call site

Centralize the formula so every endpoint derives its TTL from documented inputs instead of a copy-pasted magic number.

const TTL_POLICY = {
  payment_capture: { clientRetryBudgetS: 4 * 3600, reconciliationS: 3600, marginS: 20 * 3600 },
  internal_rpc:    { clientRetryBudgetS: 30, brokerRedeliveryS: 120, reconciliationS: 300, marginS: 300 },
};

function computeTtlSeconds(policyName) {
  const p = TTL_POLICY[policyName];
  const base = Math.max(p.clientRetryBudgetS || 0, p.brokerRedeliveryS || 0);
  return base + p.reconciliationS + p.marginS;
}

// computeTtlSeconds('payment_capture') === 86400  (24h)
// computeTtlSeconds('internal_rpc')    === 720     (12min)

---

Verification and testing

Confirm Redis expires the key at the configured TTL, not before

redis-cli SET idem:test-key "cached-response" EX 5 NX
sleep 3 && redis-cli TTL idem:test-key   # expect ~2 (still alive, not yet expired)
sleep 3 && redis-cli EXISTS idem:test-key  # expect 0 (expired after 5s total)

Confirm the Postgres sweep respects expires_at and does not over-delete

INSERT INTO idempotency_keys (key, response, expires_at)
VALUES ('still-valid', '{}'::jsonb, now() + interval '1 hour'),
       ('expired',     '{}'::jsonb, now() - interval '1 minute');

DELETE FROM idempotency_keys WHERE expires_at < now();

SELECT key FROM idempotency_keys;
-- Expected: only 'still-valid' remains

Load-test that TTL exceeds observed retry tail

# Replay production retry logs and assert no retry's elapsed time exceeds the configured TTL
awk -F'elapsed_s=' '{print $2}' retry_log.txt | sort -n | tail -1
# Compare against your TTL_POLICY value — the max observed retry must stay under it

---

Failure scenarios and debugging

---

SRE / observability checklist

1. idempotency_key_ttl_seconds — Gauge (or static config export) recording the currently configured TTL per endpoint, so dashboards can correlate incidents with TTL changes.
2. idempotency_key_expired_before_retry_total — Counter incremented whenever a retry’s elapsed time is discovered to exceed the TTL after the fact (via log correlation). Any non-zero value means the TTL formula needs revisiting.
3. redis_evicted_keys_total — Counter; a rising rate under volatile-ttl eviction policy signals the store is undersized for the chosen TTL and key volume.
4. idempotency_keys_row_count (Postgres) — Gauge tracking table growth; page if it exceeds 5x the expected daily volume, indicating the expiry sweep has stalled.
5. pg_cron_last_run_timestamp — Gauge; alert if the expiry job has not run within 2x its scheduled interval.
6. Client retry-tail p99 — dashboard panel plotting the p99 elapsed time from RetryTracker logs against the configured TTL line, so drift is visible before it causes duplicates.

---

Related

• Idempotency Key Storage & TTL Management — parent page on storage selection and key lifecycle.
• Redis vs PostgreSQL for Idempotency State — decision guide for where these TTL-bound records should live.
• Using Redis SETNX for Distributed Request Deduplication — the atomic write this TTL configuration attaches to.
• How to Generate Cryptographically Secure Idempotency Keys — key-generation rules that keep TTL reuse assumptions valid.