Picture a dashboard endpoint that takes two seconds of database work to build. You cache the result for five minutes, traffic is steady, and the database sits at a comfortable 20 percent load. Then the cache entry expires at 2:00 PM on a busy afternoon. In the two seconds it takes the first request to rebuild the value, three hundred more requests arrive, see an empty cache, and each starts the same expensive query.
That is a cache stampede, sometimes called a thundering herd or dogpile. The cache did its job for four minutes and fifty-eight seconds, and then it made things worse than having no cache at all, because the load arrives in a spike instead of being spread out. The fix is rarely more hardware. It is a small amount of coordination around who is allowed to rebuild a value and what everyone else does while they wait.
Why stampedes happen even on well-tuned caches
The root cause is simple: the cache check and the rebuild are two separate steps, and nothing connects concurrent requests to each other. Each request asks "is the value there?", gets "no", and independently decides to go compute it. The more expensive the computation and the hotter the key, the worse the overlap.
Several situations make this far more likely than it looks on paper:
- Hot keys with a fixed TTL. A homepage query, a pricing table, or a feature flag set that every request touches will expire at the same instant for everyone.
- Cold starts after a deploy or restart. A fresh instance with an empty in-memory cache, or a freshly flushed Redis, sends the entire production load to the origin at once.
- Synchronized expiry. If you warm a thousand keys in one batch with the same TTL, they all expire in the same second later.
- Slow rebuilds. The longer the rebuild takes, the more requests pile up behind it. A rebuild that slows down under load makes the pileup grow, which slows it further.
The Wikipedia entry on cache stampedes covers the general shape of the problem well. What matters in practice is recognizing it in your own metrics: a sawtooth pattern in database CPU that lines up exactly with your TTL is the classic signature.
Confirm it is a stampede before you fix it
It is tempting to jump straight to a locking library, but the same symptoms show up for other reasons, and the wrong fix adds complexity for nothing. Spend ten minutes confirming the diagnosis first.
Start by correlating three things on one timeline: cache miss rate, origin request rate, and origin latency. In a stampede, misses and origin requests spike together for a few seconds and then drop back to normal. If the origin load is high all the time, you have a capacity or hit-rate problem, not a stampede.
Then look at the queries themselves. Pull the slow query log or your tracing data for one spike and count how many identical queries ran inside the same window. Fifty copies of the same statement started within a second of each other is your answer. If you want a structured way to work through this kind of production analysis, our team does exactly this kind of review as part of our web development services.

Photo by Kader D. Kahraman on Pexels
Fix one: request coalescing so only one caller rebuilds
The most direct fix is to make sure that when many requests want the same missing key, only one of them does the work and the others wait for its result. This is called request coalescing, or single-flight in some language ecosystems.
Within a single process, the implementation is a map of in-flight promises keyed by cache key. The first request to miss creates the promise and stores it in the map. Every later request that misses the same key finds the promise already there and awaits it instead of starting its own. When the promise settles, it is removed from the map and the value is written to the cache.
Two details decide whether this works in production:
- Always remove the entry in a finally block. If the rebuild throws and you only clean up on success, the failed promise stays in the map and every later request awaits a rejection forever.
- Do not share errors blindly. If the rebuild fails, all waiting callers receive that failure. That is usually correct, but decide deliberately whether they should fall back to a stale value instead (more on that below).
In-process coalescing is cheap and removes most of the damage. Its limit is that it only coordinates within one process. If you run twenty application instances, you still get up to twenty simultaneous rebuilds, which is a big improvement over three hundred but not zero.
Fix two: a distributed lock for cross-instance coordination
When twenty rebuilds is still too many, move the coordination into the shared cache. The standard pattern is a short-lived lock key set with an atomic "set if not exists" command and an expiry. In Redis, that is SET lock:key token NX PX 10000, described in the Redis documentation on distributed locks.
The flow looks like this:
- A request misses the cache and tries to acquire the lock for that key.
- The winner rebuilds the value, writes it to the cache, and releases the lock.
- The losers either wait briefly and re-check the cache, or serve a stale copy if one exists.
The lock must have an expiry that is longer than a normal rebuild but short enough to recover quickly if the winner crashes. Release it only if you still own it, by comparing the stored token, so a slow rebuild that outlives its lock cannot delete somebody else's lock. And keep the waiting side bounded: poll for a couple of hundred milliseconds with a small jitter, then give up and either rebuild anyway or return a degraded response. An unbounded wait just moves the pileup from the database into your connection pool.
"A lock around a rebuild is a safety net, not a design. If your system only survives because the lock works perfectly, the real problem is a rebuild that is too expensive to run on demand." - Dennis Traina, founder of 137Foundry

Photo by Brett Sayles on Pexels
Fix three: refresh before the key expires
Locks make a stampede survivable. Early refresh prevents most stampedes from forming in the first place, by making sure a hot key is rarely empty when a request arrives.
The simplest version is a background refresh with a soft TTL. Store the value with two timestamps: a soft expiry, after which the value is considered due for refresh, and a hard expiry, after which it is gone. When a request finds a value past its soft expiry but before its hard expiry, it returns the cached value immediately and triggers a single background rebuild. Users never wait, and the rebuild happens once.
A more mathematical variant is probabilistic early expiration. Each request that reads a value near the end of its life has a small, growing chance of refreshing it early, weighted by how long the rebuild takes. With many requests per second, one of them will almost always volunteer a little before the deadline, and the refresh is spread out without any lock at all. It is elegant, but it is harder to explain to the next engineer, so use it where the traffic justifies it.
Also add jitter to every TTL you set. Instead of a flat 300 seconds, use 300 plus or minus ten percent. This single line of code breaks up synchronized expiry across batches of keys and costs nothing.
Fix four: serve stale while you revalidate
For many endpoints, a slightly old answer is far better than a slow answer. Serving stale data while a fresh copy is computed turns the rebuild into a background concern instead of a user-facing delay.
HTTP standardized this idea in RFC 5861, which defines the stale-while-revalidate and stale-if-error cache-control extensions. If your responses pass through a CDN or reverse proxy, you can often get this behavior with a header change alone. The NGINX documentation describes the equivalent proxy cache controls, including locking so that only one request goes upstream to refresh a stale entry.
The same principle works inside your own cache layer. Keep the last known good value for a generous window past its soft expiry, and let it carry traffic through two situations: a rebuild that is in progress, and a rebuild that is failing. That second case matters more than people expect. When the origin database is struggling, stale data keeps the site up while the database recovers, instead of every request adding to the pile.
Be honest about which data can tolerate staleness. A marketing page can be minutes old. An account balance or an inventory count that gates a purchase cannot. Decide per endpoint, write the tolerance down, and keep the strict ones on a different, shorter-lived path.
Photo by Jan Antonin Kolar on Unsplash
Pre-warm after deploys and flushes
Cold starts deserve their own plan because they stampede every key at once, not just one. If a deploy or a cache flush empties the cache during peak hours, no per-key lock will save you from thousands of different keys missing together.
There are three practical defenses:
- Warm the hottest keys before taking traffic. Run a small script after deploy that requests your top fifty or hundred endpoints in a controlled, rate-limited way, and only mark the instance healthy when it finishes.
- Roll deploys gradually. If each new instance warms from a shared cache that is still populated, there is no cold start at all. Flushing the shared cache on deploy is almost never necessary and often the real cause of the incident.
- Shed load deliberately. If a stampede is already happening, a short-lived limit on concurrent rebuilds per instance protects the origin and gives the cache time to fill. A small number of slow responses is better than a database that falls over.
Handle rebuild failures without making things worse
A subtle trap is what happens when the rebuild itself fails. If failures are not cached, every request retries the failing query, and you have recreated the stampede with a failing dependency in the middle of it.
Cache the failure briefly. A negative entry with a five to ten second TTL tells the next hundred requests to back off, serve stale, or return a controlled error instead of hammering the origin. Combine it with exponential backoff on the refresh job so a flapping dependency is probed gently instead of continuously.
This is also where integrations deserve attention. When the expensive value comes from a third-party API rather than your own database, the same patterns apply, and rate limits make the cost of a stampede even higher. If that describes your setup, our guidance on data integration covers how to isolate that dependency behind a cache and a circuit breaker.
Measure so you know it worked
A stampede fix that you cannot observe is a guess. Add a handful of metrics before and after the change, and compare the same traffic period on both sides.
The ones worth tracking are the cache hit rate per key group, the number of concurrent rebuilds for the same key (this should hover at one), the rebuild duration at the 95th percentile, and the count of requests served stale. A healthy system shows a flat origin request rate even as individual keys expire and refresh underneath it. The MDN guide to HTTP caching is a good reference for the response-level controls you will adjust while tuning.
Finally, test the failure on purpose. In a staging environment, expire a hot key under load with a load testing tool and watch the origin. If the spike is gone, you are done. If it is smaller but still there, you have found the next layer to fix, whether that is a missing jitter, an in-process only lock, or a cold start path nobody warmed.
A practical order of operations
You do not need all of this at once. For most applications, the right order is cheap to expensive:
- Add TTL jitter everywhere. It takes minutes and prevents synchronized expiry.
- Add in-process request coalescing around your most expensive rebuilds.
- Serve stale with a soft TTL and a single background refresh for the hot keys.
- Add a distributed lock only if multi-instance rebuilds are still hurting.
- Pre-warm and gradually roll deploys so cold starts stop being incidents.
Each step removes a class of failure and none of them requires rewriting how your application works. If your team would rather have an experienced hand look at where your own endpoints are exposed, 137Foundry works on exactly this kind of reliability and performance problem, and you can see the broader range of what we build on our services page.