Before comparing lock services it is worth asking whether you need one. Deduplicating a request requires a single atomic conditional write — SET NX or a unique constraint — not a lock. A lock service earns its place when a handler coordinates across several resources, or when exactly one instance must own a partition or a cron job. This guide sits under distributed lock acquisition patterns and assumes you have established that need.
Prerequisites. Familiarity with lease semantics from lock timeout & lease management, and with fencing tokens, which decide this comparison.
Step 1 — Efficiency lock or correctness lock?
Step 2 — The acquisition primitive
// etcd: a lease plus a compare-and-swap on create-revision. The Put's revision
// is a fleet-wide monotonic fencing token, produced by Raft.
lease, _ := cli.Grant(ctx, 30) // 30-second TTL
resp, err := cli.Txn(ctx).
If(clientv3.Compare(clientv3.CreateRevision(key), "=", 0)).
Then(clientv3.OpPut(key, holderID, clientv3.WithLease(lease.ID))).
Commit()
token := resp.Header.Revision
keepAlive, _ := cli.KeepAlive(ctx, lease.ID) // heartbeat renewal
// ZooKeeper via Curator: an ephemeral sequential znode. The sequence number is
// the fencing token; the ephemeral node disappears when the session ends.
InterProcessMutex mutex = new InterProcessMutex(client, "/locks/settlement");
if (mutex.acquire(5, TimeUnit.SECONDS)) {
try {
Stat stat = client.checkExists().forPath("/locks/settlement");
long token = stat.getCzxid(); // consensus-assigned transaction id
doWork(token);
} finally {
mutex.release();
}
}
-- Redis: SET NX with a lease, plus INCR for a token. Atomic within one shard.
if redis.call('SET', KEYS[1], ARGV[1], 'NX', 'PX', tonumber(ARGV[2])) then
return redis.call('INCR', KEYS[2])
end
return 0
The three differ in what happens when the service itself fails. etcd and ZooKeeper cannot grant the same lock twice, because a grant is a committed consensus entry and a new leader has it. Redis can, because a promoted replica may never have received the grant.
Step 3 — Fencing-token quality is the deciding factor
| Property | etcd | ZooKeeper | Redis (single) | Redis (Redlock) |
|---|---|---|---|---|
| Token source | Raft revision | zxid / sequential znode |
INCR counter |
INCR on each node |
| Strictly monotonic across failover | Yes | Yes | No | No |
| Double-grant possible on failover | No | No | Yes | Yes (contested) |
| Session/lease expiry mechanism | Lease + keep-alive | Session heartbeat | Key TTL | Key TTL per node |
| Automatic release on client death | Yes, lease expiry | Yes, ephemeral node | Only on TTL | Only on TTL |
| Typical acquire latency (in-region) | 3–10 ms | 5–15 ms | 0.3–1 ms | 2–5 ms |
| Cluster size for HA | 3 or 5 | 3 or 5 | 1 + replica | 5 independent |
| Operational weight | Moderate | High (JVM, ensemble) | Low | Moderate |
| Already running it? | If you run Kubernetes | Often with Kafka/HBase | Almost always | — |
The rows that matter for a correctness lock are the first three. A token that can regress is not a fence; it merely looks like one, and the failure surfaces as data corruption during an incident rather than as an error anyone can see.
That does not make Redis useless for correctness locks. It makes Redis-plus-a-resource-fence viable, where the resource itself records the highest token it has accepted — because then a regressed token is rejected by the database, not trusted by the lock service. What is not viable is Redis alone.
Step 4 — Latency versus operational cost
# Measure acquisition latency for your own workload rather than trusting a table.
etcdctl --endpoints=$ETCD check perf --load=s # etcd's own benchmark
redis-cli --latency-history -i 5 # Redis round-trip baseline
# A fair comparison: full acquire + release cycle under contention.
def bench(lock_factory, holders=32, iterations=200):
latencies = []
with ThreadPoolExecutor(holders) as pool:
def one(_):
t0 = time.perf_counter()
lease = lock_factory.acquire("bench", lease_ms=5_000)
lock_factory.release(lease)
latencies.append((time.perf_counter() - t0) * 1000)
list(pool.map(one, range(iterations * holders)))
latencies.sort()
return latencies[len(latencies) // 2], latencies[int(len(latencies) * 0.99)]
The honest framing: etcd costs roughly 5–10 ms more per acquisition than Redis. If a lock is taken once per batch job that is irrelevant. If it is taken once per request at 2,000 requests per second, it is a design constraint — and usually a sign the design should be using an atomic conditional write instead of a lock.
If you already run Kubernetes, you already run etcd. Using it for application locks is tempting and usually wrong: application lock churn competes with the control plane for the same Raft log. Run a separate etcd cluster for application locks, and budget for it.
Step 5 — Verify liveness during leader election
# etcd: kill the leader and measure the acquisition gap.
LEADER=$(etcdctl --endpoints=$ETCD endpoint status -w json \
| jq -r '.[] | select(.Status.leader == .Status.header.member_id) | .Endpoint')
./lock_probe --interval 100ms --duration 60s & # logs every acquire attempt
sleep 10 && ssh "$LEADER" sudo systemctl stop etcd
wait
awk '/FAIL/ {n++} END {print "failed acquisitions:", n}' lock_probe.log
# expect: a contiguous run of failures ~1–5 s long, then recovery
# And the important assertion: no work proceeded without a lock.
grep -c 'proceeded without lock' app.log # expect: 0
Failure scenarios and debugging
| Failure Scenario | Remediation Steps | Observability Hooks |
|---|---|---|
| Redis chosen for a correctness lock without a resource-enforced fence, and a failover grants the same lock twice | Add a fence_token column and enforce it in the write’s WHERE clause, or move the lock service to etcd. Replication alone never closes this. |
lock_fence_rejected_total — zero while lock_double_grant_suspected_total is non-zero means no fence exists |
| Client treats a lock-service outage as “proceed without the lock” | Fail closed: an unavailable lock service means the work is deferred, not done unlocked. Make the acquire call throw rather than return a nullable lease. | lock_acquire_error_total alongside a proceeded_without_lock_total counter that must be structurally zero |
| Application locks placed in the Kubernetes control-plane etcd, and lock churn slows the API server | Run a dedicated etcd cluster for application locks. Never share the control plane’s Raft log. | etcd_server_proposals_committed_total rate correlated with application deploy activity; API-server latency |
| ZooKeeper session timeout shorter than a GC pause, so the ephemeral node vanishes while the holder still runs | Set the session timeout above the JVM’s p999 pause, and add a fence so a resumed holder cannot write regardless. | zk_session_expired_total; jvm_gc_pause_seconds p999 against the configured timeout |
| Lock held across an external HTTP call, so a slow dependency holds the lock for its full timeout and everything queues | Acquire, do local work, release; call external services outside the lock with their own idempotency key. | lock_hold_duration_seconds p99; alert above 1 s for request-path locks |
SRE / observability checklist
lock_acquire_duration_seconds— Histogram per service. etcd p99 above50 msor Redis p99 above5 msindicates the lock service is degraded.lock_acquire_error_total— Counter of failed acquisitions. Expected to spike briefly during leader election; a sustained rate is an outage.proceeded_without_lock_total— Must be structurally zero. Any value means a code path fails open on a correctness lock.lock_hold_duration_seconds— Histogram. A p99 near the lease duration means holders are routinely close to expiry and fence rejections are imminent.lock_fence_rejected_total— Counter of stale writes rejected at the resource. Non-zero proves the fence works and that holders do freeze.etcd_server_has_leader/zk_server_state— Gauge of quorum health. A quorum without a leader grants nothing, and clients must fail closed.
Related
- Distributed Lock Acquisition Patterns — the parent page on lock acquisition, contention and release semantics.
- Redlock vs Single-Instance Redis Locks — the Redis-internal comparison this page places in wider context.
- Using Fencing Tokens to Survive Lock Expiry — the mechanism that decides this comparison.
- Implementing Lease Heartbeat Renewal with etcd — the keep-alive loop behind etcd’s lease semantics.