App and API Protection

Embed
Preview of the card
<a href="https://watchtowerops.net/docs/app-api-protection"><img src="https://watchtowerops.net/docs/_api/public/orgs/aa5959cb-1a0b-4a9a-bc9a-913e7ac31cd9/spaces/b7aac181-9916-4805-b253-becc5fd2ca62/pages/3635a37c-9acd-4084-ba3e-031ec147b3ff/social-card.svg?v=3e7ae33eec8b" alt="App and API Protection" width="600" height="315"></a>

App and API Protection (AAP) finds attacks and abuse in the traces your services already send. It can also block them inside your app through the Watchtower SDKs, and it lists the API endpoints your services expose. Everything runs on the server spans you already send. Watchtower adds no agent, sidecar or proxy.

The console page is App protection (/appsec). It has four tabs: Signals, Rules, Denylist and API inventory.

Detection from traces

Every minute the worker judges the newest closed five-minute bucket of server spans. It waits 90 seconds after the bucket ends, so late spans still count.

Pattern rules run on each request. They read these fields, each percent-decoded once:

  • the URL path and the query string;

  • the User-Agent;

  • the other request header values (http.request.header.*);

  • the request body, when a capture rule keeps http.request.body.

Threshold rules count requests per client IP or per user in each bucket.

The shipped default ruleset:

Id

Detects

Kind

Severity

sqli-tautology, sqli-union, sqli-blind, sqli-stacked, sqli-metadata

SQL injection

pattern

high (metadata: medium)

xss-script, xss-handler, xss-uri, xss-tags

Cross-site scripting

pattern

high / medium

lfi-traversal, lfi-files

Path traversal, sensitive files

pattern

high

ssrf-internal, ssrf-scheme

SSRF to internal or metadata addresses, dangerous schemes

pattern

high / medium

cmdi-chain, cmdi-shell

Command injection

pattern

high

rce-jndi, rce-template

Log4Shell lookups, template injection

pattern

critical / high

scanner-ua, scanner-paths

Security scanners, vulnerability probes

pattern

medium / low

ato-bruteforce

30+ failed logins (401-403) from one IP

threshold

high

ato-stuffing

10+ failed logins over 5+ accounts from one IP

threshold

critical

ato-targeted

10+ failed logins on one account from 3+ IPs

threshold

high

abuse-signup

20+ successful signups (POST to signup or register routes) from one IP

threshold

medium

burst-4xx, burst-5xx

200+ client errors or 50+ server errors from one IP

threshold

medium / high

A login route is any route matching login, signin, session, oauth, auth or token. A login that answers 200 but fails in your code counts as a 401 when the SDK marks it with trackLogin(login, false).

Signals

Each match becomes a signal, stored in appsec_signals_v1 for 90 days. A signal groups one rule's hits by one actor (a client IP, else the user) across buckets. It carries:

  • hit and blocked counts;

  • the routes, client IPs and users involved;

  • the matched text;

  • up to 10 attack traces.

High and critical signals raise the alert external:appsec. It is keyed by the signal and labelled with rule_id, category, service, actor and trace_id. It routes through receivers, routing trees and escalation policies like any alert. The alert resolves after 30 minutes without new hits. Low and medium signals are stored but do not alert.

Client IP comes from these span attributes, in this order: client.address, http.client_ip, appsec.client_ip (set by the SDKs), network.peer.address, net.sock.peer.addr. The user comes from enduser.id, user.id or appsec.login.

Rules and mode

GET /v1/projects/{prj}/appsec/rules returns the effective ruleset.

GET|PUT /v1/projects/{prj}/appsec/config holds three settings:

  • mode: monitor or block;

  • disabled: default rule ids that are turned off;

  • custom: up to 50 custom rules.

A write needs ManageProject and is audited (appsec.config.updated).

Custom rules use the same shape as the default rules:

  • pattern rules: fields and pattern;

  • threshold rules: group_by (ip or user), optional method and route regexes, status_min, status_max, min_count and optional min_distinct.

Patterns are matched case-insensitively. They must stay within the regex subset that Rust, RE2 (Go, ClickHouse), JavaScript and Python all support. These are refused: inline flags and any (? group other than (?:, look-around, \A, \z, Unicode and POSIX classes, and \Q…\E.

In-app WAF (SDKs)

The SDKs poll GET /v1/appsec/ruleset?service= on the edge with an ingestion key that has scope probes:read. They poll every 30 seconds with an ETag. The response has the mode, the enabled pattern rules and the denylist entries in force. Threshold rules stay on the server.

The SDKs check each request before your handler runs:

  • Match in monitor mode: the server span gets appsec.event, appsec.rule_id, appsec.category and appsec.blocked=false, and the request goes through.

  • Match in block mode: the SDK answers 403 application/problem+json and your handler never runs.

  • Denylisted IP or user: the SDK answers 403 in both modes.

Until the first ruleset arrives, every request goes through.

SDK

Start

Hook

Node (@watchtowerobservability/node/appsec)

startAppsec()

app.use(appsecMiddleware()) (Express, Connect); handleRequest(req, res) for http, Fastify onRequest, Koa

Python (watchtower.appsec)

appsec.start()

AppsecMiddleware(wsgi_app) (Flask, Django); AppsecASGIMiddleware(app) (FastAPI, Starlette)

Go (watchtower)

watchtower.StartAppsec(ctx, nil, apiURL, key, service)

built into wtnethttp.Handler; watchtower.AppsecMiddleware(h) for other routers

All three read WT_API_URL and WT_INGEST_KEY by default.

Each SDK also has these helpers:

  • trackLogin / track_login / TrackLogin: marks a login attempt and its result;

  • isUserBlocked / is_user_blocked / AppsecUserBlocked: checks a user after authentication.

Install the middleware inside the OpenTelemetry server instrumentation, so the server span is the current span.

The SDKs inspect the path, query, User-Agent and header values. The Node middleware also reads a body that a body parser has already parsed. The Python and Go middleware do not read the body. The worker still checks bodies that capture rules keep.

The SDKs take the client IP from the first X-Forwarded-For hop, then X-Real-IP, then the socket peer. Run them behind a proxy that overwrites these headers.

Denylist

The denylist API needs ManageProject for writes, and every write is audited:

  • GET /v1/projects/{prj}/appsec/denylist lists entries (cursor pagination).

  • POST /v1/projects/{prj}/appsec/denylist adds an entry: {kind: "ip"|"user", value, reason?, ttl_s?, signal_key?}. ttl_s is 60 seconds to 1 year; leave it out for no expiry.

  • DELETE /v1/projects/{prj}/appsec/denylist/{id} removes an entry.

Adding the same kind and value again replaces its reason and expiry. A project holds up to 10,000 entries. Expired entries stop applying at once and are pruned a day later. A signal's detail page can deny its actor in one click. IP entries are exact addresses (IPv4 or IPv6).

API inventory

GET /v1/projects/{prj}/appsec/inventory?from&to&service lists every HTTP endpoint (method and http.route) seen in server spans. It covers the last 7 days by default, up to 2,000 endpoints. Each endpoint shows:

  • Calls and errors.

  • Authentication. Whether requests carried an identity (enduser.id, user.id, a captured Authorization header, or appsec.auth from trackLogin), or whether the endpoint answered 401 or 403.

  • PII in responses. These kinds of the gateway's PII detectors, found in http.response.* attributes (headers, and the body excerpt that capture rules keep): email, card, ssn, phone, iban, jwt.

  • First and last seen.

  • documented. Whether the service's imported OpenAPI spec (services/{svc}/api-spec) lists the endpoint. It is null when the service has no spec.

Signals API

  • GET /v1/projects/{prj}/appsec/signals?from&to&severity&category&service lists signals, at most 500, most recently seen first.

  • GET /v1/projects/{prj}/appsec/signals/{key} returns one signal's summary, its hits per bucket and the actor's active denylist entry.

Both cover the last 7 days by default.

Limits

  • Delivery is at least once. A bucket that is judged twice writes the same rows, and the ReplacingMergeTree keeps one.

  • Each bucket reads at most 20,000 spans that pass the pattern pre-filter, plus 200,000 count groups.

  • The card check in the inventory is regex-only (no Luhn check).

Updated