Shaping rules and the match expression

A shaping rule (/v1/projects/{prj}/shaping-rules, see Ingest cost shaping) selects records with a match expression. The same expression language drives pipelines. Rules without match apply to every record of the signal.

{"signal": "traces", "kind": "sample", "match": "service == \"formflow-worker\" and span_name == \"UPDATE\" and is_root",
 "params": {"rate": 0.05}, "mode": "preview"}

Grammar

• Comparison: == != < <= > >=. Both sides are coerced: numbers and numeric strings compare as numbers, other strings compare as text, booleans compare with booleans. A missing field is null: null == null is true, every other comparison with null is false (so attrs.x != "a" is false when attrs.x is absent; add attrs.x == null or ... when you want that case).

• Logic: and, or, not (lower case words; &&, || and ! are not accepted). Precedence, loosest first: or, and, not, comparison, + -, * / %, unary minus. Use parentheses to group.

• Literals: double-quoted strings (escapes \n, \t, \", \\), integers, decimals (0.5), true, false, null. Single quotes are not strings.

• Functions: contains(s, sub), starts_with, ends_with, matches(s, "regex") (the pattern must be a literal, at most 1024 bytes), lower, upper, trim, substr, len, coalesce(a, b, ...), to_int, to_float, to_string, round, min, max, abs, replace, split, if(cond, a, b), parse_json, ip_in_cidr and a few more. They are shared with pipeline expressions.

• Limits: 4096 bytes, 512 tokens, 256 nodes, nesting depth 32. Errors name the first problem.

There is no bracket syntax (attributes["x"] fails with "unexpected character ["). Use dotted names.

Fields

attrs. and resource. never fall back to each other: a service name set on the resource is resource.service.name (or just service), not attrs.service.name. Any other bare name is an error that lists the valid ones.

Examples

Traces:

• Drop the polling loop's database spans that have no parent (migrations, schedulers): attrs.db.system.name == "postgresql" and is_root with kind drop.

• Keep 10% of fast, successful health checks: attrs.http.route == "/api/health" and status_code != "STATUS_CODE_ERROR" with kind sample, rate 0.1.

• Sample Next.js internals: starts_with(span_name, "load route module") or span_name == "start response".

• Spans over 2 seconds: duration_s > 2.

Logs:

• Debug and trace logs from one service: service == "worker" and severity <= 8.

• Noisy health logs: contains(body, "GET /healthz").

• Regex: matches(body, "^heartbeat ok").

• Everything except errors: not (severity >= 17).

Metrics:

• One metric family: starts_with(metric_name, "http.server.") (kind aggregate also takes a metric glob).

• A noisy label: metric_name == "queue.depth" and attrs.queue == "scratch".

Impact preview

POST or GET /v1/projects/{prj}/shaping-rules/{id}/impact estimates what the rule would remove. It looks at the last 24 hours by default; pass ?window= to change it. The response is {"rows", "bytes_est", "pct_of_signal", "window", "rows_scanned"}: window is the window used and rows_scanned is how many stored rows the estimate was computed from, so a small number means the project has little data in that window and the percentage is rough. Bytes are estimates.

Trace sampling rules

/v1/projects/{prj}/sampling/rules (head and tail sampling) is a different rule set with its own format; see trace sampling. Use shaping rules when you want a match expression.