Intermediate12 min

Queue Backlog

Producers outpace consumers and the queue grows without bound.

By the end of this lesson, detect and control unbounded queue growth before it becomes an outage.

How deep?
How the pieces actually move.

A queue absorbs bursts — but only if consumers keep up on average. When producers persistently outrun consumers, the backlog grows without bound: latency climbs, memory or disk fills, and by the time you notice, the delay is hours. A queue hides overload right up until it can't.

Producers > consumers

Producers > consumers

  1. Producers enqueue faster than consumers dequeue, so lag grows continuously.
flowchart LR
  prod[Producers: 1000/s] --> q[(Queue: growing)]
  q --> cons[Consumers: 600/s]
  q --> lag[Lag grows 400/s]

By Little's Law, backlog and latency are set by arrival vs. service rate. Controls: backpressure (slow or reject producers when the queue is deep), autoscale consumers on lag, bounded queues (reject rather than grow forever), and alert on *lag/age*, not just queue length. A queue with no backpressure is an unbounded memory leak with extra steps.

Unbounded queue backlog

Trigger
Producers sustainably exceed consumer throughput with no backpressure.
Symptom
Ever-growing lag; stale processing; eventual memory/disk exhaustion.
Blast radius
The queue infrastructure and every consumer of its (now stale) output.
Mitigation
Backpressure, bounded queues, consumer autoscaling, lag/age alerting.
Should you alert on queue length or message age?

Message age (lag). Length alone doesn't tell you how far behind you are; age reveals the actual delay users experience.

Sometimes a backlog is caused by one message the consumer can never process. What is that message called?

Next: Poison Messages →