Paging that wakes a human

Embed
Preview of the card
<a href="https://watchtowerops.net/docs/paging"><img src="https://watchtowerops.net/docs/_api/public/orgs/aa5959cb-1a0b-4a9a-bc9a-913e7ac31cd9/spaces/b7aac181-9916-4805-b253-becc5fd2ca62/pages/3c69dde9-f04f-47de-a592-2afed032fed6/social-card.svg?v=3e7ae33eec8b" alt="Paging that wakes a human" width="600" height="315"></a>

A page reaches a person through their verified contact methods, chosen by their personal notification rules or by the escalation level (via). Three paths can wake someone:

Path

What the person needs

Bypasses Do Not Disturb

Push to the installed web app (PWA)

the app installed, push turned on

no (Android: high-priority notification that stays until acted on)

SMS

a verified sms method

per phone settings (add the number to favourites / emergency bypass)

Voice call

a verified voice method

per phone settings (repeated calls from a favourite ring through)

For anything that must wake someone at night, give the level via: ["voice", "sms", "push"] or put a voice method in each person's audible rules, and turn on re-notification (below).

1. Install the on-call app (PWA)

The console is an installable app. Installed, it opens on On-call, My on-call (also a home screen shortcut): whether you are on call and until when, your shifts for 14 days, open pages with one-tap Acknowledge, Escalate now and Resolve, Take on-call now (an override of 1 to 24 hours on a schedule), and push for this device.

  • Android (Chrome, Edge, Samsung Internet): open the console, menu, Install app. Open it, On-call, My on-call, Turn on push on this device.

  • Desktop (Chrome, Edge): the install icon in the address bar. Firefox and Safari on macOS take push without installing.

  • iPhone and iPad (iOS / iPadOS 16.4 or later): Safari delivers web push only to an app added to the home screen. In Safari: Share, Add to Home Screen, Add. Open Watchtower from the home screen icon, sign in, On-call, My on-call, Turn on push on this device, and allow notifications. In iOS Settings, Notifications, Watchtower, set Time Sensitive or allow it in your Focus modes; web push cannot break through Do Not Disturb on iOS, so add SMS or voice for night pages. iOS shows no buttons on web notifications: a tap opens the page in My on-call.

A page notification carries Ack and Escalate (most browsers show two buttons; Resolve is third). The buttons post to POST /v1/projects/{prj}/pages/{page}:ack|:escalate|:resolve with the session; a failed action shows a "failed, open Watchtower" notification instead of silence. Critical and high pages stay on screen until acted on and vibrate. A tap opens /oncall?view=mine&project=…&page=….

Operator config: VITE_VAPID_PUBLIC_KEY at build time and WT_VAPID_PUBLIC_KEY, WT_VAPID_PRIVATE_KEY, WT_VAPID_SUBJECT on the notifier. /sw.js must be served with Cache-Control: no-cache (the nginx template does).

2. Contact methods

Each person, under On-call, My settings: add an sms or voice method with an E.164 number (+15551230001). A 6-digit code is sent to it by SMS or call; enter it to verify (5 wrong tries lock the code; Send a new code issues another). Only verified methods are ever paged. Send test (POST /v1/projects/{prj}/me/contact-methods/{id}/test, once a minute) sends a labelled test through the real channel and records it in the delivery history, so each person can prove their own path before a shift.

3. Who sends the SMS and calls

Watchtower's accounts (default). The system contact:sms and contact:voice receivers send from the operator's Twilio and/or Vonage accounts (WT_TWILIO_*, WT_VONAGE_* on the notifier, see docs/product/notifications/channels.md). Each message or call takes one credit of the org's monthly sms_credits / voice_credits plan entitlement (notification_credits, shown under usage; at 80 % the org is warned once; at the cap SMS and voice stop with error class quota while push and email continue). Nobody has to bring a carrier, where the operator of your deployment has set those accounts up. If it has not, the system SMS and voice receivers have no platform carrier to send through: bring your own Twilio, Telnyx or Vonage account (next paragraph), which is not metered.

The org's own carrier (optional). Under On-call, Call routing, SMS and voice carriers (or /v1/orgs/{org}/paging/carriers, ManageOnCall) an org adds up to two carriers, priority 1 and 2 (2 takes over on a retryable failure of 1). With one enabled, the org's SMS and voice pages go out from its own number, are not metered, and its keypad presses, SMS replies and delivery reports come back to the notifier:

  • inbound (voice gather and incoming SMS): {WT_PUBLIC_URL}/paging/inbound/{carrier_id}

  • delivery status: {WT_PUBLIC_URL}/paging/status/{carrier_id}

Both paths are returned on the carrier (inbound_path, status_path). Every callback is verified with that carrier's own secret before anything is read (403 otherwise, also for an unknown or deleted carrier). Credentials are sealed (receiver key, aad = org ‖ carrier), write-only, never in a response, audit event or log. max_per_minute (1 to 600, default 30) caps sends per carrier and notifier replica; over it a send is retried a minute later (error class rate_limited).

Carrier (kind)

Credentials

Carrier console setup

Twilio (twilio)

account_sid, auth_token

Number, Messaging: A message comes in webhook POST to the inbound URL. Calls need nothing (TwiML is sent inline).

Twilio-compatible (twilio + base_url), e.g. SignalWire's LaML API

as Twilio, from that provider

Same as Twilio; base_url is the provider's REST origin (https://<space>.signalwire.com). Its webhook signature must be Twilio's X-Twilio-Signature.

Telnyx (telnyx)

api_key, public_key (Portal, Account, Public key: the base64 Ed25519 key), application_id for calls

Messaging profile: inbound webhook to the inbound URL. A TeXML application with an outbound voice profile, its id as application_id. Calls use inline TeXML; status comes on the per-message webhook_url and the call StatusCallback. Signatures: Telnyx-Signature-Ed25519 over `timestamp

Vonage (vonage)

api_key, api_secret, signature_secret; application_id + private_key (PEM) for calls

Enable signed webhooks (the signature secret). Voice keypad events arrive on the inbound URL (set per call). Vonage SMS replies use a different signature scheme and are not accepted: acknowledge by keypad. No delivery status.

Plivo is not supported (its voice API needs a fetched answer document and its own signature scheme); use one of the above.

4. Acting from the phone

  • Call: the call reads the page, then "Press 1 to acknowledge, 2 to escalate." The gather URL carries page, project, user, exp (10 minutes) and sig = hex(HMAC-SHA256(key("wt/provider-callback/v1"), "page|project|user|exp")); a signature is used once. 1 acknowledges, 2 escalates to the next level now; the call answers "Acknowledged" or "Escalated".

  • SMS: a page SMS ends with "Reply ACK <code> to acknowledge, ESC <code> to escalate." The code is single-use, 24 h, stored only as an HMAC, and accepted only from the verified number the page was sent to (and, for an org carrier, only for that org's pages). A Twilio-compatible carrier answers with a confirmation SMS. A failed action gives the code back.

  • Push: the notification buttons, or the page in My on-call.

Each action is forwarded to the API's internal POST /internal/pages/{id}:ack|:escalate as that user, so their own role is checked and the action is audited like a click in the console.

5. Delivery status

Every SMS and call is a delivery row (notification history: GET /v1/projects/{prj}/notifications) with its attempts, provider (the carrier's message or call id) and error class. Through an org carrier, the carrier's later verdict is added as carrier_status (delivered, undelivered, failed, delivery_failed, completed, no-answer, busy, …) and carrier_status_at.

6. Escalation policies

  • via on a level: {"delay_min": 5, "targets": [{"schedule": "sch_…"}], "via": ["voice", "sms"]} pages that level's people on every verified method of those kinds at once, instead of their personal rules (someone with none of them keeps their rules). Kinds: sms, voice, push, email, slack, teams, telegram, mattermost, webhook. The policy editor offers SMS, voice and push.

  • renotify_until_ack_min on the policy (0 to 1440): a high or critical page whose levels, repeats and fallback are used up fires its last level again every N minutes until someone acknowledges or resolves it (at most 48 times, then the page is exhausted as before). Each re-notification is a renotify event on the page timeline. Lower severities exhaust as before.

Mobile app

Watchtower has no native phone app. The installable console (PWA) is the on-call app: install it on your phone (section 1), turn on push, and add a verified SMS and voice number under On-call, My settings (section 2). Press Send test on each. There is nothing to install from the App Store or Google Play.

Updated