Cascading Failure
One slow dependency takes down everything that depends on it — and everything that depends on them.
By the end of this lesson, recognize how a single failure propagates through a system and contain the spread.
A cascading failure is when one component's failure causes its callers to fail, whose failure causes *their* callers to fail, until a small problem becomes a system-wide outage. The mechanism is usually resource exhaustion: a slow dependency holds callers' threads/connections open, so callers run out of capacity and fail too.
First, see the whole system. Then we’ll open it up.
How one slow service spreads
- A slow database blocks Service A's threads.
- Service A exhausts the gateway's connection pool.
- The web tier times out and users see a total outage.
flowchart LR db[(Slow DB)] --> svcA[Service A: threads blocked] svcA --> gw[Gateway: pool exhausted] gw --> web[Web tier: timeouts] web --> user[Users: total outage]
The key insight: latency, not errors, is the killer. An outright error frees the caller's resources fast. A slow response holds threads, connections, and memory until they run out — which is why timeouts must be aggressive and paired with isolation.
Cascading failure
- Trigger
- A dependency slows down; callers block on it and exhaust their own resources.
- Symptom
- Failure spreads outward from one component to unrelated ones sharing resources.
- Blast radius
- Potentially the entire system, far beyond the original fault.
- Mitigation
- Aggressive timeouts, circuit breakers, bulkheads to isolate pools, and load shedding.
The patterns that contain cascades:
Why is a slow dependency more dangerous than one returning fast errors?
Slow responses hold the caller's threads/connections open until exhaustion; fast errors free those resources immediately.