Table Of Contents
TL;DR
- 24/7 almost never means a 24/7 helpdesk. Customers want an availability promise and a guaranteed response time when the system is down. That is on-call for critical incidents, not night shifts.
- Your cloud provider already runs most of it. Data centers, hardware and managed services are operated around the clock. Your on-call only covers your own application and the communication with customers.
- German law allows it. On-call from home (Rufbereitschaft) counts as rest time. Only real incidents are working time. After an incident, the 11 hours of rest start again.
- You have to pay for it, but not with time off. Being on-call is paid with a flat fee. Time off is only owed for real incidents.
- You need a pool of 6-10 volunteers, not a new department. One shared rotation, one week at a time, with a primary and a secondary.
- Budget honestly. The flat fees are the small part. The larger part is the 5-10% of engineering capacity that goes into reliability work.
- Sell it, do not give it away. 24/7 belongs into a paid enterprise SLA, and you only sign that SLA after a pilot has shown that you can deliver.
I am not a lawyer and this is not legal advice. Have your on-call agreement checked by an employment lawyer before you roll it out.
What customers mean when they say “24/7”
The request usually arrives in a procurement questionnaire: “Do you offer 24/7 support?” It sounds like a call center that picks up the phone at 3 am. That is almost never what is meant. Customers want two things:
- Availability: a promise like 99.9% per month.
- Response time for critical incidents: someone qualified starts working on a total outage within, say, 30 minutes. Around the clock.
The first tip is therefore the most important one: define the scope narrowly. On-call is for priority 1 incidents only: the system is down, or data is being lost. It is triggered by monitoring, not by a customer calling a mobile number. A broken export button or a slow report waits for the next business day.
It helps to know how little downtime the usual numbers allow:
| Availability per month | Allowed downtime per month |
|---|---|
| 99.5% | about 3.6 hours |
| 99.9% | about 43 minutes |
| 99.95% | about 22 minutes |
| 99.99% | about 4 minutes |
With a response time of 30 minutes, one single night incident already consumes most of a 99.9% month. Anything above 99.9% is not an on-call question any more. It is an architecture question.
What your cloud provider already does for you
If you run on AWS, Azure or Google Cloud and use their managed services, most of the 24/7 operation is already taken care of. This is the good news that often gets forgotten in the discussion.
| Layer | Who is on duty 24/7 | Example |
|---|---|---|
| Data center, hardware, network | Cloud provider | Broken server, power failure, loss of an availability zone |
| Managed services (database, storage, load balancer) | Provider operates, you configure | Database failover, backups, patches |
| Automatic recovery (health checks, auto scaling, multiple zones) | Provider executes, you set it up | Replacing a failed instance without a human |
| Application, deployments, configuration, data | You | Bad release, wrong configuration, full database, expired certificate |
| Alerting and customer communication | You | Taking the alert, updating the status page |
Three consequences:
- The better you use managed services and multiple zones, the less often on-call gets woken up. Self-managed servers shift work back to you.
- You still need on-call. The provider watches its infrastructure, not your application. Nobody at your cloud provider fixes your bad release at 2 am.
- The promise stays with you. Towards your customers you are liable for the SLA. When the provider fails, you get service credits on your cloud bill, nothing more.
German labor law in plain words
Germany distinguishes between forms of being available. The difference decides everything else.
| Form | What it means | Working time? |
|---|---|---|
| Rufbereitschaft (on-call) | The person chooses freely where to be, is reachable and can start working within an agreed time | No. Only the actual incidents are working time |
| Bereitschaftsdienst (standby duty) | The employer dictates the location, for example the office | Yes, all of it |
For a SaaS company you want Rufbereitschaft. These are the rules that matter:
- Keep the response time relaxed. The European Court of Justice ruled in 2021 (cases C-344/19 and C-580/19) that on-call becomes working time in full when the constraints seriously limit how people can use their free time. A setup like “at the laptop within 30 minutes, no fixed location” is the usual way to stay on the safe side. “Online within 5 minutes” is not.
- Incidents are working time. They count towards the daily maximum of 10 hours and have to be recorded.
- The 11 hours of rest start again after every incident (§ 5 ArbZG). Whoever fixed something from 2 am to 3 am does not start work before 2 pm. Plan for this. It is the rule with the largest practical impact.
- Sundays and public holidays are allowed for keeping IT systems running (§ 10 ArbZG). Work on a Sunday requires a replacement rest day within two weeks, and 15 Sundays per year have to stay free (§ 11 ArbZG). In a five-day week, a Saturday that is free anyway can usually serve as the replacement day.
- You normally cannot simply order it. If the employment contract says nothing about on-call, you need a supplementary agreement with each participant.
- A works council has a say. If you have a Betriebsrat, the on-call plan is subject to co-determination (§ 87 BetrVG) and you need a works agreement.
A note on timing: at the time of writing, a reform of the Working Time Act is being discussed that would allow a weekly instead of a daily maximum under certain conditions. It is not in force. Check the current state before you finalize your agreement.
Compensation: what you owe and what you do not
A common question: if nothing happens during an on-call week, do you have to compensate it with time off? No. No working time was created, so there is nothing to balance. But on-call itself has to be paid, because people restrict their private life for it. The amount is free to agree on and may be well below the normal hourly rate.
| Situation | Time off required? | Pay |
|---|---|---|
| On-call without an incident | No | Flat fee |
| Incident during on-call | Yes, or payment as overtime | Incident time plus surcharges, as agreed |
| After an incident | The 11 hours of rest start again | Nothing extra |
Tips for the design:
- Pay an explicit flat fee. A clause like “on-call is covered by the salary” is risky. A separate, visible fee per week is cleaner and it is also the better signal to the team.
- Use surcharges for nights, Sundays and holidays. Within the limits of § 3b EStG they are tax free: 25% for night work, 50% for Sundays, 125% for public holidays. The same gross amount leaves more net for the engineer.
- Let people choose between money and time for incidents. Some want the payout, some want the day off.
- Pay holidays better. Christmas and New Year should cost you more. Otherwise the same people cover them every year.
- Consider a voluntary day off after an on-call week. It is not required. It keeps volunteers in the pool.
As an orientation, not as market data: a flat fee of 400-600 EUR per week for the primary and about half of that for the secondary is a range that works for many teams.
Designing the rotation
- One shared pool. Product teams of 4-5 engineers are too small for their own rotation. Build one pool across teams.
- 6-10 people. With 8 people, everyone is on-call every eighth week. Below 6, the load becomes too high in the long run.
- Volunteers only. Nobody should have to explain why they opt out. Small children, health and care duties are good reasons that are nobody’s business.
- Primary and secondary. The primary takes the alert. If there is no acknowledgement after 10-15 minutes, the tool escalates to the secondary automatically. Senior engineers and leads are a good secondary.
- Weekly shifts with a handover in the middle of the week. Tuesday or Wednesday works better than Monday or Friday, because public holidays and long weekends cluster there.
- Make swapping easy. A swap should take one message and one click in the tool.
- Nobody goes alone the first time. New people shadow a shift before they carry the phone.
Does the team have to grow?
For the rotation itself: no. A team of 15-30 engineers has enough people for a pool.
For your delivery speed: yes, a little. On-call binds capacity, and the flat fees are the smallest part of it. Rest time after night incidents, postmortems, better alerts, runbooks and the fixes that come out of incidents add up to roughly 5-10% of engineering capacity. If the roadmap must not slow down, that capacity has to come from somewhere.
My recommendation is to hire one platform or reliability engineer first and to decide on a second position after the pilot, with measured numbers instead of estimates. Three reasons speak against hiring for the full estimate right away:
- Part of the investment comes back. More stable systems mean less unplanned work during the day as well.
- The setup is a one-off. Alerts, runbooks, rollback and access cost a few person-months once.
- New hires need 3-6 months before they help. During the setup phase they slow you down.
The decisive number is the count of incidents outside business hours in the last 12 months. If it is more than two or three per month, fix stability first. Otherwise you will lose your volunteers within a quarter.
What it costs
An example for one rotation with a primary and a secondary. All numbers are rough orientation values.
| Item | Per year (EUR) |
|---|---|
| Flat fee primary (400-600 per week) | 21,000-31,000 |
| Flat fee secondary (about half) | 10,000-16,000 |
| Employer’s social contributions on the fees (about 20%) | 6,000-9,000 |
| Incident pay with surcharges, or time off | 5,000-10,000 |
| Paging tool, status page, monitoring | 5,000-15,000 |
| Employment lawyer and SLA contract template (one-off) | 5,000-10,000 |
| Cash in the first year | about 60,000-90,000 |
Add the capacity on top. With one additional engineer at 90,000-120,000 EUR full cost, the first year lands at about 150,000-210,000 EUR. This is the number to hold against the revenue, not the flat fees alone.
Customer contracts: sell it as an SLA
- Make it a paid tier. 24/7 is an enterprise feature. A surcharge somewhere around 10-25% of the contract value is common for premium SLAs. Calculate from how many enterprise customers on it pays off.
- Promise response times, never resolution times. You control when you start. You do not control how long the fix takes.
- Define severity levels in the contract, including who decides which level applies.
- Define how availability is measured, and exclude planned maintenance windows and outages of your cloud provider.
- Cap the service credits, for example at 10-20% of the monthly fee, and make them the sole remedy.
Rollout in four phases
- Preparation (about 6 weeks). Define scope and severity levels. Evaluate the incidents of the last 12 months. Build alerts on symptoms that customers feel. Write runbooks. Make sure a single person can roll back a release.
- Trial during business hours (4-8 weeks). Run the rotation with the paging tool, escalation and incident process, but only during the day. You find most process problems here, without anybody losing sleep.
- 24/7 pilot (3 months). Real on-call with volunteers and real pay, but without a promise to customers.
- Sell the SLA. Only now, and with the numbers from the pilot.
Define before the pilot starts what “passed” means, for example a maximum number of night pages per week and a response time that was met in every case.
Do not sign a 24/7 SLA before the pilot has run. Otherwise you are liable for something you cannot deliver yet.
Tips and tricks
- Alert on symptoms, not on causes. “Customers cannot log in” wakes somebody up. “CPU at 85%” does not.
- Every alert links to a runbook. At 3 am nobody should have to think about where to start.
- Every night page gets a postmortem, and the fix gets priority over features. An alert that fires twice for the same reason is a management failure, not bad luck.
- Track pages per shift. More than two per week outside business hours means reliability work comes before roadmap work.
- Give on-call real authority. Roll back, switch off a feature flag, scale up: all without asking anyone.
- Use the on-call person as the interrupt handler during the day. They take incidents and escalations, and the rest of the team keeps its focus.
- Test the pager. A monthly test alert including the escalation to the secondary finds silenced phones and expired logins before a real incident does.
- Prepare the customer communication. A status page and three text templates save twenty minutes of wording in the middle of the night.
- Record incident times properly. They are working time, and you need them for the rest periods and for the decision on capacity.
- Automate the rest period. Whoever had a night incident starts late, automatically and without having to ask. The secondary covers the morning.
- Consider follow-the-sun if you have teams in other time zones. If the day shift of another office covers your night, most of the labor law questions disappear.
Takeaway
24/7 sounds bigger than it is. Narrow the scope to real emergencies, let your cloud provider do what it does anyway, pay people fairly for being reachable, and respect the rest periods. Then run it for three months before you promise anything to a customer. Done this way, on-call does not burn out the team. It makes the product better, because every night page becomes a reason to fix the cause.
More
- Related: Good Products Win: What Churning My Mobile Provider Taught Me About Reliability and Pricing
- The laws mentioned: Arbeitszeitgesetz (ArbZG), § 87 BetrVG, § 3b EStG
- Awesome photo on top by Jon Tyson
