Alert flapping is repeated transition between firing and recovered states over a short period. It creates duplicate pages for one unstable condition and trains responders to distrust the signal.
This guide separates detector noise from a genuinely marginal service, then applies pending duration, confirmation checks, hysteresis, dampening, and multi-location evidence without masking a sustained outage.
What Flapping Is
Flapping is when a monitored check rapidly and repeatedly transitions between states — up and down, healthy and unhealthy — over a short window. Each transition is treated as a new event, so the result is a storm of "down!" / "recovered!" / "down!" notifications for what is really one ambiguous, borderline condition.
The defining trait is instability, not direction. A service that's genuinely down stays down; a service that flaps is hovering right at the edge of whatever the check measures, tipping back and forth across the line. The check isn't lying on any single sample — it's that the underlying state is genuinely marginal, and a naive monitor reports every flip as gospel.
Why Monitors Flap
Flapping almost always traces to a metric sitting near a threshold, or to noise in the measurement path:
- A metric hovering at the threshold. CPU oscillating around an 80% alert line, latency bouncing across a 500ms limit, error rate wobbling around 1%. Each tiny fluctuation crosses the boundary and back. This is the classic cause — and it points straight at how thresholds (and static limits in general) are set.
- Intermittent network problems. Packet loss, jitter, or a flapping route between the monitor and the target makes checks time out sporadically, then succeed — even though the service itself is fine.
- Overloaded or autoscaling systems. A service at capacity serves some requests and drops others; an autoscaler adds and removes instances; a load balancer shifts traffic. The health check catches the system mid-transition.
- Single-sample checks with no tolerance. A monitor that declares "down" on one failed probe and "up" on the next will faithfully report every blip as a state change.
- Marginal dependencies. A flaky downstream database or third-party API that's itself unstable propagates that instability into your health checks.
Notice the throughline: flapping is usually a symptom of a borderline condition plus a too-sensitive detector. Fix either and the flapping subsides.
How to Tame Flapping
The goal is to distinguish a real, sustained state change from transient noise — without slowing down detection of genuine outages so much that the monitor becomes useless. Several techniques stack together:
1. Confirmation checks (don't trust a single sample)
Before declaring an outage, require a condition to remain true for a defined duration or number of evaluations rather than alerting on one sample. Google SRE's practical alerting guidance describes a minimum duration before a rule fires specifically to reduce flapping. Choose that duration from the service's detection objective and check cadence; do not copy a universal count.
2. Multi-location verification
Confirm from more than one vantage point before alerting. If a check fails from one region but succeeds from two others, the problem is almost certainly the path to that one prober — not your service. Requiring agreement across multiple locations filters out network-induced flapping that a single-prober setup would report as real downtime.
3. Hysteresis (different up and down thresholds)
Borrowed from electronics: use a stricter threshold to recover than to fail. Trip "down" at >500ms, but only declare "up" again below 400ms. The gap between the two lines means a metric sitting at 450ms can't oscillate across a single boundary, because there isn't one — there are two, with a buffer between them.
4. Flap detection and dampening
Track how often a check has changed state recently. If transitions exceed a threshold within a window, suppress further notifications and mark the check as "flapping" — one alert that says "this is unstable" instead of fifty that each claim a definitive state. Notifications resume once the check settles. This is the explicit anti-flapping mechanism many monitoring systems ship.
5. Fix the root cause, not just the noise
Dampening hides the symptom; the borderline condition is the disease. A check that flaps is telling you something is marginal — capacity at its limit, a threshold set too tight, a dependency that's unstable. Treat persistent flapping as a signal to widen a threshold, add capacity, or fix a flaky dependency, not just something to mute.
Flapping vs a Real Outage
The danger of over-tuning is muting the real thing. Keep the distinction sharp:
| Flapping | Real outage | |
|---|---|---|
| Pattern | Rapid up/down/up/down | Sustained down |
| Confirmation checks | Pass intermittently | Fail consistently |
| Multi-location | Often differs by region | Usually fails everywhere |
| Right response | Dampen + fix root cause | Page and respond |
The techniques above are designed to keep the real outage loud while silencing the noise. Confirmation and multi-location checks let a genuine, everywhere-failing outage through quickly — it fails every consecutive check from every location — while filtering the marginal flip-flop that doesn't. Done right, you lose almost no detection speed on real incidents and shed nearly all the flapping noise. Tie escalation to incident severity so a confirmed outage still pages hard.
How Webalert Helps
Flapping is a problem Webalert is built to prevent at the source:
- Multi-region confirmation before declaring downtime — a failure seen from one location is verified against others, so a bad network path to a single prober never pages you as a fake outage.
- Confirmation checks that require sustained failure rather than reacting to a single blip, cutting the one-off flaps that cause most alert storms.
- Sustained-state alerting so you're notified about real, persistent changes — not every momentary flip across a boundary.
- Clear up/down history that makes a genuinely flapping endpoint obvious, so you can see the instability and fix the borderline condition behind it.
The result is alerts you can trust: loud when it matters, quiet when it doesn't.
Summary
Flapping is a check rapidly flip-flopping between up and down, turning one borderline condition into a storm of contradictory alerts — and it's a fast track to muted monitors and missed outages. It happens when a metric hovers at a threshold, when intermittent network problems disrupt the path to the prober, or when a single-sample check has no tolerance for noise.
Tame it by refusing to trust a single sample: require consecutive failures, confirm from multiple locations, add hysteresis so recovery and failure use different thresholds, and apply flap detection that collapses a storm into one "this is unstable" alert. Above all, treat persistent flapping as a signal that something is genuinely marginal and fix that root cause. Done well, your monitoring stays loud for real outages and quiet for noise — which is the entire point of alerting.