Prompt
How do I stop pager fatigue on a 24/7 team?
Latest observation
Pager fatigue on a 24/7 team usually comes from one of three things: too many pages, too many low-value pages, or too much personal responsibility for each page. The fix is usually a mix of reducing alert noise, tightening paging policy, and improving on-call support.
1) Start by measuring the problem
Before changing anything, get a simple breakdown for the last 2–4 weeks:
- pages per day/week
- pages by service/team
- percent that were actionable vs. noise
- pages that repeated the same issue
- pages during off-hours vs. business hours
- average time to acknowledge and resolve
A lot of “pager fatigue” is actually one noisy service generating most alerts.
2) Make paging only for urgent, user-impacting issues
A good rule:
- Page only when immediate action is needed and users are impacted or likely to be impacted soon.
- Ticket / Slack / dashboard for everything else.
- No page for purely informational alerts.
Examples of things that often should not page:
- short-lived blips
- single-instance failures with redundancy
- non-customer-facing batch job delays
- warnings without active impact
- alerts that can wait until business hours
3) Reduce alert noise aggressively
Common fixes:
- deduplicate repeated alerts
- add cooldowns / rate limits
- require sustained failure before paging
- alert on symptoms, not every internal cause
- use aggregation so one incident creates one page
- tune thresholds to match real user impact
- suppress alerts during known maintenance windows
- remove alerts nobody has acted on for months
If you have pages that are almost always false positives, they should be redesigned or deleted.
4) Create severity levels with clear routing
Not every incident needs the same response.
For example:
- SEV1: page immediately, wake someone up
- SEV2: page if no one responds quickly
- SEV3: ticket during working hours
- SEV4: log only
Make sure each severity has:
- a definition
- response expectations
- escalation path
- example scenarios
This prevents “everything is urgent” culture.
5) Improve runbooks so pages are faster to handle
A pager feels exhausting when every alert requires detective work.
For each common alert, create a runbook with:
- what the alert means
- likely causes
- how to confirm impact
- first 3 actions to take
- when to escalate
- rollback / mitigation steps
Good runbooks reduce cognitive load and shorten incidents.
6) Share the burden more fairly
If the same people get paged repeatedly, fatigue rises fast.
Consider:
- smaller, better-defined on-call rotations
- secondary backup on-call
- pairing a junior with an experienced responder
- rotating ownership of noisy systems
- ensuring on-call is part of normal work, not “extra”
Also watch for chronic overload on your strongest responders.
7) Protect recovery time
On-call only works if people can recover.
Best practices:
- no paging after a severe overnight incident unless necessary
- compensate with time off after rough on-call periods
- keep rotations humane
- avoid stacking on-call with major project deadlines
- limit consecutive nights / weekends
- allow handoff flexibility if someone had a bad week
If people can’t recover, fatigue becomes burnout.
8) Fix recurring incidents, not just symptoms
If the same alert pages repeatedly, treat it as an engineering problem.
Ask:
- Why did this page happen?
- What permanent fix would eliminate it?
- Can we add automation or self-healing?
- Is the service missing capacity, retries, circuit breakers, or failover?
A team with lots of repeat pages usually needs reliability work, not just better paging.
9) Set expectations with leadership
Pager fatigue often persists because the org accepts it as “part of the job.”
Make the cost visible:
- number of pages per on-call shift
- overnight interruptions
- time spent on false alarms
- impact on morale and retention
Then propose a target, like:
- cut pages by 50% in 60 days
- reduce false pages to under 10%
- cap overnight pages to truly urgent incidents
10) Establish a “page review” loop
Do a weekly or biweekly review of all pages and ask:
- Was this page necessary?
- Was it routed correctly?
- Did the alert fire too early or too often?
- What change prevents the next one?
This keeps the system improving instead of drifting back into alert spam.
Quick practical starting plan
If you want a simple 30-day approach:
Week 1
- collect page data
- identify top 5 noisy alerts
- classify alerts by severity
Week 2
- demote non-urgent alerts
- add dedupe/cooldowns
- write runbooks for top alerts
Week 3
- tune thresholds and grouping
- remove obsolete alerts
- improve escalation and handoff rules
Week 4
- review results
- measure page reduction
- identify top recurring incident to eliminate permanently
A useful litmus test
Ask of every alert:
“If this fires at 2 a.m., does it require a human to act right now to protect users or revenue?”
If the answer is no, it probably shouldn’t page.
If you want, I can also help you build:
- a paging policy template,
- a severity matrix, or
- a 30-day alert reduction plan for your team.