Full Jitter vs Decorrelated Jitter Backoff

Every backoff implementation makes the same two promises to different parties: to the client, that it will finish reasonably soon; to the server, that it will not be hit by a synchronised wave. The four common strategies trade these off differently, and the choice is measurable rather than a matter of taste. This decision guide sits under retry logic & backoff fundamentals.

Prerequisites. The endpoint being retried must be safely retryable — see API contract design for idempotent endpoints — and you should have a retry budget capping volume, because jitter controls when retries happen, not how many.

Step 1 — Implement the four candidates

import random

BASE_MS, CAP_MS = 100, 20_000

def no_jitter(attempt: int) -> float:
    """Pure exponential. Every client that failed together retries together."""
    return min(CAP_MS, BASE_MS * 2 ** attempt)

def equal_jitter(attempt: int) -> float:
    """Half fixed, half random. Guarantees a minimum wait, halves the spread."""
    ceiling = min(CAP_MS, BASE_MS * 2 ** attempt)
    return ceiling / 2 + random.uniform(0, ceiling / 2)

def full_jitter(attempt: int) -> float:
    """Uniform over the whole window. Maximum decorrelation for a given ceiling."""
    return random.uniform(0, min(CAP_MS, BASE_MS * 2 ** attempt))

def decorrelated_jitter(previous_ms: float) -> float:
    """Random walk: the next ceiling derives from the PREVIOUS actual delay,
    so the sequence can outgrow pure exponential and never resets its spread."""
    return min(CAP_MS, random.uniform(BASE_MS, previous_ms * 3))
// Go equivalents. Seed once per process from crypto/rand, never per call.
func FullJitter(attempt int) time.Duration {
    ceiling := math.Min(float64(capMS), float64(baseMS)*math.Pow(2, float64(attempt)))
    return time.Duration(rand.Float64()*ceiling) * time.Millisecond
}

func DecorrelatedJitter(prev time.Duration) time.Duration {
    lo, hi := float64(baseMS), math.Min(float64(capMS), float64(prev.Milliseconds())*3)
    if hi < lo { hi = lo }
    return time.Duration(lo+rand.Float64()*(hi-lo)) * time.Millisecond
}
Delay distributions of four backoff strategies Four rows, one per strategy, plotted against attempt number one to six on a logarithmic delay axis. No jitter shows a single delay value per attempt at one hundred, two hundred, four hundred, eight hundred, sixteen hundred and thirty-two hundred milliseconds. Equal jitter shows a band covering the upper half of each window. Full jitter shows a band covering the entire window from zero to the ceiling. Decorrelated jitter shows overlapping bands that drift upward without a fixed per-attempt ceiling. delay window per attempt (log scale, 100 ms → 20 s) no jitter all clients land on the same instant — convoy never breaks equal jitter guaranteed minimum wait, half the spread of full jitter full jitter uniform over [0, ceiling] — maximum decorrelation, lowest contention decorrelated window derives from the previous delay — drifts upward, no fixed schedule attempt → 1 2 3 4 5 6

Step 2 — Simulate against a contended resource

Argument settles quickly under simulation. Model N clients contending for a resource that serves one request at a time, and count both total calls made and time to last completion.

import heapq, random

def simulate(strategy, clients=1000, capacity_per_ms=1.0, seed=7):
    """Discrete-event model: each client retries until it wins a slot.
    Returns (total_calls, ms_until_last_client_succeeded)."""
    random.seed(seed)
    queue = [(0.0, c, 0, 100.0) for c in range(clients)]   # (t, id, attempt, prev_delay)
    heapq.heapify(queue)
    served_until, calls = 0.0, 0

    while queue:
        t, cid, attempt, prev = heapq.heappop(queue)
        calls += 1
        if t >= served_until:                              # resource free — success
            served_until = t + 1.0 / capacity_per_ms
            continue
        delay = strategy(attempt) if strategy.__code__.co_argcount == 1 else strategy(prev)
        heapq.heappush(queue, (t + delay, cid, attempt + 1, delay))
    return calls, served_until
python3 - <<'PY'
from backoff_sim import simulate, no_jitter, equal_jitter, full_jitter, decorrelated_jitter
for name, s in [("none", no_jitter), ("equal", equal_jitter),
                ("full", full_jitter), ("decorrelated", decorrelated_jitter)]:
    calls, t = simulate(s)
    print(f"{name:<14} calls={calls:>7}  completion={t/1000:.1f}s")
PY

Representative output for 1,000 clients contending for a 1-per-millisecond resource:

Strategy Total calls Time to last success Server-side contention
No jitter ~28,000 ~31 s Severe — repeated synchronised waves
Equal jitter ~5,900 ~13 s Moderate
Full jitter ~3,400 ~12 s Lowest
Decorrelated jitter ~3,900 ~10 s Low, with a longer per-client tail

The shape of the result is robust across parameters even though exact numbers move: no jitter is dramatically worst on both axes, full jitter minimises calls, and decorrelated jitter finishes marginally sooner at the cost of slightly more calls and a wider spread on individual clients.

Step 3 — Understand why the jitter-free case is so bad

Arrival pattern with and without jitter Two arrival histograms over the same time axis. The upper one, without jitter, shows three tall narrow spikes at one hundred, three hundred and seven hundred milliseconds where every client retries simultaneously, each far above the server capacity line. The lower one, with full jitter, shows a low broad distribution that stays under the capacity line throughout. no jitter — synchronised waves capacity 100 ms 300 ms 700 ms every spike exceeds capacity — the whole wave fails and re-synchronises full jitter — spread arrivals capacity

A jitter-free schedule preserves the correlation that caused the failure. Every client that timed out at the same instant waits the same 100 ms, retries together, overloads the resource together, and waits the same 200 ms. The convoy is never broken; it just moves further apart in time while staying just as dense.

Step 4 — Pick one

Default to full jitter. It minimises server contention, is one line of code, and has no tuning parameters beyond base and cap. Every AWS SDK ships it. If you have no specific reason to choose otherwise, this is the answer.

Choose decorrelated jitter when clients retry over minutes rather than seconds — long-running batch jobs, background sync, event redelivery — and you want the delay to keep growing without maintaining an attempt counter. It has a useful property for a stateless retry loop: the only state it needs is the previous delay.

Choose equal jitter when a guaranteed minimum wait matters, for instance when a downstream circuit breaker needs a quiet period to close and an immediate retry from full jitter would reopen it.

Never choose no jitter with more than one client.

# The default, complete. Cap the delay AND bound the total window.
def retry_with_full_jitter(fn, key, *, base_ms=100, cap_ms=20_000, max_elapsed_s=60):
    started, attempt = time.monotonic(), 0
    while True:
        try:
            return fn(idempotency_key=key)
        except Transient:
            attempt += 1
            delay = random.uniform(0, min(cap_ms, base_ms * 2 ** attempt)) / 1000
            if time.monotonic() + delay - started > max_elapsed_s:
                raise
            time.sleep(delay)

The cap_ms bound stops the sixteenth attempt waiting an hour. The max_elapsed_s bound is the one people forget, and it is what keeps a retry loop from outliving the key’s 24-hour retention window or the caller’s deadline.

Verification and testing

Confirm the distribution is what you think.

python3 - <<'PY'
import statistics, random
from backoff import full_jitter
samples = [full_jitter(3) for _ in range(100_000)]      # ceiling = 800 ms
print(f"min={min(samples):.1f} mean={statistics.mean(samples):.1f} max={max(samples):.1f}")
PY
# expect: min ≈ 0, mean ≈ 400, max ≈ 800 — a uniform distribution over [0, 800]

Confirm the seed is per-process, not per-call. Reseeding inside the delay function makes every client in a fleet produce identical “random” delays.

grep -rn "random.seed\|rand.Seed\|new Random(" --include=*.py --include=*.go --include=*.java src/
# expect: at most one hit, at process startup

Confirm the total window is bounded.

def test_retry_loop_respects_max_elapsed(failing_dep):
    start = time.monotonic()
    with pytest.raises(Transient):
        retry_with_full_jitter(failing_dep, key="k", max_elapsed_s=5)
    assert time.monotonic() - start < 6.5      # never unbounded

Failure scenarios and debugging

Failure Scenario Remediation Steps Observability Hooks
Jitter is implemented but every client seeds its RNG from the same fixed value, so the fleet is still synchronised Seed once per process from crypto/rand or the OS entropy source. Assert in a test that two processes started in the same second produce different first delays. Histogram of first-retry delay across the fleet; a spike at one value proves shared seeding
Delay is capped but the total retry window is not, so a client retries for 40 minutes against a key with a 24-hour TTL and a 3-second deadline Add max_elapsed_s and derive it from the propagated deadline where one exists, per retry budgets and deadline propagation. retry_total_elapsed_seconds histogram; alert above the endpoint’s documented deadline
Decorrelated jitter chosen without a cap, so one unlucky random walk produces a 40-minute delay Always clamp with min(cap_ms, ...). The random walk has no upper bound of its own. retry_delay_ms histogram p999; a value near the cap is fine, a value above it is a missing clamp
Full jitter can return a near-zero delay, which reopens a circuit breaker immediately after it half-closes Switch that call path to equal jitter, or enforce a floor of the breaker’s quiet period. circuit_breaker_reopen_total; a rate correlated with retry attempts indicates delays below the quiet period
Simulation outcome for 1,000 contending clients Four horizontal bars showing total calls made by one thousand clients contending for a single resource. No jitter costs about twenty-eight thousand calls, equal jitter about five thousand nine hundred, full jitter about three thousand four hundred, and decorrelated jitter about three thousand nine hundred. Full jitter is the shortest bar and therefore the lowest server-side cost. no jitter ~28,000 equal jitter ~5,900 full jitter ~3,400 — lowest decorrelated ~3,900 · finishes marginally sooner

SRE / observability checklist

  1. retry_delay_ms — Histogram of actual sleep durations. Verify the shape matches the chosen strategy; a narrow distribution means jitter is not being applied.
  2. retry_attempt_number — Histogram of the attempt index at success. A rising mean is the earliest sign that a dependency is degrading.
  3. retry_total_elapsed_seconds — Histogram from first attempt to terminal outcome. Alert above the endpoint’s documented deadline; it means the window is unbounded.
  4. retry_synchronisation_index — Derived from the variance of arrival timestamps within a 100 ms bucket. A collapsing variance means the fleet has re-synchronised.
  5. backoff_strategy client label — On every retry metric, so a fleet running mixed strategies after a partial rollout is visible rather than inferred.