A cache stampede is the caching-specific thundering herd: many callers miss the same hot key and regenerate it concurrently. A cache avalanche is broader—many keys expire or disappear together—while a cold-cache event follows a restart, flush, or new fleet. The mitigations overlap, but naming the trigger helps choose the right control.
This guide explains what cache stampedes and thundering herds are, why they're so destructive, and how to prevent them.
Cache Stampede vs Thundering Herd vs Avalanche
A cache stampede (also called a dog-pile or, more broadly, a thundering herd) happens when a popular cached item expires and many concurrent requests all miss the cache at once. Because the value is gone, each request independently tries to regenerate it — running the same expensive query, API call, or computation — and they all pile onto the backend simultaneously.
The "thundering herd" name is the general pattern: a large number of waiting processes all wake up and contend for the same resource at the same time. A cache stampede is the caching-specific version, where the shared resource is the work needed to refill one hot key.
- Cache stampede: concurrent regeneration of one or a few hot missing keys.
- Cache avalanche: many keys expire together, often because they share a TTL boundary.
- Cold cache: a new or flushed cache starts with no warm working set.
The cruel irony is that the cache was protecting the backend. As long as the value was cached, the database saw almost no load. The moment it expires, the database sees the full, un-cached load — often far more than it can handle — precisely because the cache had been hiding just how popular that item was.
Why It's So Destructive
A cache stampede is dangerous because it's sudden, coordinated, and self-amplifying:
- The load arrives all at once. It isn't a gradual ramp the database can autoscale into — it's thousands of identical queries in the same few milliseconds.
- The work is redundant. Every request computes the same value. You might need that value computed once; instead it's computed thousands of times in parallel, wasting the entire effort.
- It cascades. The overwhelmed database slows down, so each regeneration takes longer, so more requests pile up waiting, so the cache stays empty longer — a feedback loop. Slow regeneration can trigger connection pool exhaustion and retry storms on top.
- It's triggered by success. The more popular an item, the worse its stampede — so your best-performing, highest-traffic pages cause the biggest outages.
A related trigger is the cold cache: after a cache restart, a flush, or a deploy that invalidates everything, every key is missing at once and the entire request load slams the backend simultaneously.
Prevent Cache Stampedes in Layers
The fix is to make sure that when a hot key expires, the work to refill it happens once, not thousands of times. The main techniques:
Request coalescing / single-flight regeneration. When a request finds the key missing, one caller becomes responsible for recomputing it. Others wait within a deadline or serve stale data. If you use a distributed lock, include expiry and ownership-safe release; a stuck lock must not turn availability into a new failure mode.
Stale-while-revalidate. Serve the slightly-expired cached value while one background task refreshes it. Users never see a miss, and the backend handles a single refresh instead of a flood. This is one of the most effective and widely used patterns.
Add jitter to expiration times. If many keys are set with the same TTL, they all expire together and stampede together. Randomizing each TTL by a few percent spreads expirations out over time so misses don't synchronize — the same trick that jitter solves for retries.
Probabilistic early expiration. Have requests randomly refresh a popular key slightly before it expires, so the value is regenerated by one unlucky request while the old value is still serving everyone else — avoiding the hard cliff entirely.
Pre-warm the cache. After a deploy, flush, or restart, proactively populate hot keys before traffic hits them so you never face a fully cold cache under load.
Treat the cache as an optimization, not a crutch. If a backend literally cannot survive its own traffic without the cache, a single eviction becomes an outage. A circuit breaker and graceful degradation give you a fallback when regeneration can't keep up.
Also cap concurrent regeneration and preserve a backend request budget. Coalescing one key does not protect the database during an avalanche involving thousands of different keys.
How Webalert Helps
A cache stampede shows up from the outside as a sudden latency spike and a burst of errors that arrives the instant a hot key (or the whole cache) expires — often with no warning from your normal capacity dashboards, because the cache had been masking the real load. Webalert helps you catch it:
- Outside-in latency monitoring that catches the sudden response-time spike a stampede causes, on the real endpoints users hit.
- Error and downtime alerts for the 5xx errors and timeouts that follow when the backend gets buried — so you find out in seconds, not from customers.
- Response-time history and threshold alerts that surface endpoint slowdowns; pair them with application metrics for p95/p99 analysis, because Webalert's published feature set documents average, minimum, and maximum history rather than native percentile tracking.
- Confirmation of recovery once you've added locking, jitter, or stale-while-revalidate, verifying the spikes are actually gone under real traffic.
Webalert won't refill your cache, but it tells you the moment an expiring key turns into a user-facing outage — and confirms when your prevention measures have tamed it.
Primary Documentation
- AWS Builders' Library: caching challenges and strategies — request coalescing, soft/hard TTLs, cold starts, and resilience testing.
- RFC 5861: stale-while-revalidate and stale-if-error — HTTP cache-control extensions for serving stale responses safely.
- Redis distributed locks — lock safety properties and failure tradeoffs.
- Redis key eviction — TTL and memory-policy behavior that can trigger misses.
Summary
A cache stampede (or thundering herd) happens when a popular cached item expires and many concurrent requests all miss at once, each independently regenerating the same value and burying the backend in redundant work. It's destructive because the load is sudden, coordinated, redundant, and self-amplifying — and it's triggered by exactly the popular items and cold-cache events you most rely on.
Prevent it by making regeneration happen once instead of thousands of times: lock or single-flight the refill, serve stale-while-revalidate, add jitter to TTLs so expirations don't synchronize, use probabilistic early expiration, pre-warm hot keys after restarts, and never let your backend depend on the cache for basic survival. Pair those defenses with outside-in latency and error monitoring so you catch the spike the instant a hot key expires.