Posted by The 2 AM Wake-Up Call Nobody Signed Up For
It’s 2:17 AM. Your phone buzzes. Again. The same microservice that failed last Tuesday is down, and you’re the one holding the pager. You stumble out of bed, log in, restart the service, and crawl back under the covers. Tomorrow, or rather, today, you have three meetings and a sprint review. This isn’t an emergency. It’s just Tuesday.
If this sounds familiar, you’re not alone. Across enterprises running 24/7 services, on-call rotations have become a grinding reality. But here’s the uncomfortable truth: most organizations have built on-call models that were designed for a different era, one with fewer services, slower release cycles, and far less complexity. In 2026, with microservices sprawling across multi-cloud environments and continuous delivery pipelines pushing changes hourly, the old playbook is breaking people.
It’s time to rethink how we handle operational responsibility, not just for service reliability, but for the humans who keep the lights on.
The Problem: On-Call Was Never Meant to Scale Like This
Ten years ago, a typical enterprise application stack might have included a handful of services: a database, an app server, maybe a message queue. On-call meant someone was available if the server room caught fire (metaphorically or literally).
Today, that same enterprise runs hundreds of microservices, distributed across AWS, Azure, and on-premises data centres. Each service has its own deployment pipeline, its own failure modes, and its own alerts. What was once a manageable rotation has become a relentless stream of notifications, some critical, many not.
The consequences are real:
- Alert fatigue: When everything pages, nothing is truly urgent. Engineers start ignoring alerts or apply quick fixes instead of root cause analysis.
- Burnout: Chronic sleep disruption and the constant threat of interruption take a toll. Studies show that on-call stress contributes directly to turnover in engineering teams.
- Knowledge silos: As systems grow more complex, only a few people understand enough to respond effectively. Those people end up on-call more often, accelerating their burnout.
- Poor incident response: Tired engineers make mistakes. Cognitive performance drops sharply after interrupted sleep.
The irony? Many organizations treat on-call as a necessary evil rather than a systemic design problem.
What Good Looks Like: Reframing On-Call as a Product Problem
The best engineering organizations don’t just manage on-call; they treat it as a product with customers (the on-call engineers) and a quality bar. Here’s how they do it:
1. Build Systems That Don’t Need Heroes
Netflix is famous for its Chaos Engineering practices, but the underlying philosophy is even more important: systems should be resilient by design, not because someone wakes up at 3 AM to restart them.
Actionable shifts:
- Invest in self-healing automation. If a service restart fixes 80% of incidents, automate the restart and alert only on repeated failures.
- Design for graceful degradation. Not every component failure should be a paging event.
- Implement circuit breakers, retries, and bulkheads. These patterns reduce the blast radius of failures and often eliminate the need for human intervention.
2. Radically Reduce Alert Noise
A major European bank recently reduced their on-call pages by 60% in six months, not by ignoring problems, but by ruthlessly categorizing alerts.
Their approach:
- Actionable vs. informational: Only page for alerts that require immediate human action. Everything else goes to dashboards or asynchronous channels.
- Business impact classification: If an alert doesn’t correlate to user impact or revenue loss, it shouldn’t wake anyone up.
- Alert tuning sprints: Dedicate time every quarter to review, tune, and retire noisy alerts. Treat this like technical debt.
3. Share the Load—But Make It Fair
On-call shouldn’t be a punishment. Some teams are experimenting with models that distribute responsibility more equitably:
- Follow-the-sun rotations: For global teams, pass the pager across time zones so on-call happens during business hours.
- Tiered support: Not every engineer needs to be on-call for everything. Junior engineers can handle tier-1 issues (with runbooks), while senior staff focus on complex incidents.
- Compensation and time-off policies: If you’re paying people to be available 24/7, recognize that. Automatic comp time, on-call bonuses, or reduced project workload during on-call weeks show you value their time.
4. Build Runbooks, Not Heroics
Google’s SRE model emphasizes toil reduction, the repetitive, manual work that scales linearly with service growth. Runbooks and automation are the antidote.
Make runbooks first-class artefacts:
- Store them in version control alongside your code.
- Link them directly from alerts so the on-call engineer knows exactly what to do.
- Update them after every incident. If someone had to improvise, the runbook was incomplete.
- Automate the runbook. If a human is following a checklist, a script can probably do it faster and more reliably.
5. Measure and Improve the On-Call Experience
What gets measured gets managed. Track metrics that matter:
- Time to acknowledge: How quickly do engineers respond? Long delays signal alert fatigue.
- Time to resolve: Are incidents getting faster to fix, or slower?
- Interruption frequency: How many times per week is someone paged? Aim for a target (e.g., no more than 2-3 pages per week).
- False positive rate: How often do alerts fire with no actual issue?
- Post-incident surveys: Ask on-call engineers what went well and what didn’t. Treat their feedback as user research.
The Cultural Shift: On-Call is Everyone’s Problem
Here’s where it gets uncomfortable. On-call models often fail because of misaligned incentives. Development teams ship features quickly, and operations teams (or the unlucky on-call rotation) deal with the fallout.
The solution? Make the people who build it responsible for running it. This is the core of the DevOps and SRE philosophy, but it requires organizational change:
- Product teams own their services end-to-end, including on-call.
- Reliability and observability are part of the Definition of Done, not afterthoughts.
- Incidents trigger blameless post-mortems and action items, not finger-pointing.
When developers carry the pager for what they build, they suddenly care a lot more about error handling, observability, and automated testing.
Practical Steps to Start Today
You don’t need to overhaul everything at once. Here’s a 90-day roadmap:
Month 1: Assess
- Survey your on-call engineers. What’s painful? What wakes them up most often?
- Analyze alert data. Identify the noisiest, least actionable alerts.
- Measure baseline metrics (interruption frequency, time to resolve, etc.).
Month 2: Reduce Noise
- Tune or disable the top 10 noisy alerts.
- Automate one common remediation task (e.g., service restarts, cache clearing).
- Update or create runbooks for the top 5 incident types.
Month 3: Iterate and Improve
- Review metrics. Did interruptions decrease? Did the time to resolve improve?
- Hold a retrospective with on-call engineers. What’s better? What still sucks?
- Identify the next batch of improvements and keep going.
The Bottom Line: Reliability Starts with Sustainable Teams
Exhausted engineers don’t build reliable systems. If your on-call model is burning people out, your reliability will eventually suffer too. The best teams understand that operational excellence isn’t about heroics, it’s about designing systems, processes, and cultures that don’t need heroes in the first place.
Rethinking on-call isn’t a luxury. It’s a strategic imperative. Your services will be more reliable, your engineers will be healthier, and your organization will be better positioned to scale.
And maybe, just maybe, your team can sleep through the night.
Key Takeaways
- Traditional on-call models weren’t designed for today’s microservices sprawl and cloud complexity.
- Alert fatigue, burnout, and knowledge silos are symptoms of systemic design problems, not individual failures.
- Treat on-call as a product: measure it, improve it, and prioritize the experience of the people carrying the pager.
- Build resilient systems with self-healing automation, graceful degradation, and radically reduced alert noise.
- Make on-call responsibilities everyone’s problem by aligning them with ownership—teams that build services should also run them.
- Start small: assess pain points, reduce noise, automate common tasks, and iterate based on feedback.
Sustainable reliability comes from sustainable teams. It’s time to fix the on-call model before it breaks the people holding it together.