Intermediate12 min

Thundering Herd

Many clients wake up and hit the same resource at the same instant.

By the end of this lesson, identify synchronization points that create simultaneous load and de-synchronize them.

How deep?

How the pieces actually move.

A thundering herd is many clients acting at exactly the same moment: cron jobs all firing at :00, TTLs all expiring together, every client reconnecting the instant a service recovers, tokens all minted with the same expiry. The resource sees a spike it can't absorb.

First, the whole system

First, see the whole system. Then we’ll open it up.

Synchronized wake-up

At the same instant, all clients hit the resource, producing a spike it cannot absorb.

flowchart LR
  t[Same instant] --> c1[Client 1]
  t --> c2[Client 2]
  t --> c3[Client N]
  c1 --> res[Resource: instant spike]
  c2 --> res
  c3 --> res

The cure is de-synchronization: jitter every scheduled time and TTL, stagger reconnects with randomized backoff, and randomize token lifetimes slightly. Where a herd is unavoidable, put a queue or rate limiter in front to smooth it.

Thundering herd

Trigger: Many clients synchronize on the same instant (cron, TTL expiry, recovery, token expiry).
Symptom: A sharp, brief load spike that overwhelms the target.
Blast radius: The targeted resource and its immediate dependents.
Mitigation: Jitter schedules and TTLs; stagger reconnects; rate-limit or queue the spike.

How it connects

Every pod refreshes its config on a 5-minute cron at :00 and :05. What's the fix?

Add jitter to the refresh interval so pods don't all refresh at the same second.