OTLP metric names in PromQL

Embed
Preview of the card
<a href="https://watchtowerops.net/docs/otlp-metric-names"><img src="https://watchtowerops.net/docs/_api/public/orgs/aa5959cb-1a0b-4a9a-bc9a-913e7ac31cd9/spaces/b7aac181-9916-4805-b253-becc5fd2ca62/pages/e0a27bf6-e059-47a0-a39d-797b71c86dea/social-card.svg?v=3e7ae33eec8b" alt="OTLP metric names in PromQL" width="600" height="315"></a>

How an OTLP metric and its attributes appear to PromQL (alert rules, dashboards, the Prometheus API). The project's otlp.translation_strategy (PUT /v1/projects/{prj}/metrics/settings) decides names; the default keeps them as sent.

OTLP

PromQL (default NoTranslation)

With UnderscoreEscapingWithSuffixes

metric http.server.request.duration (unit s, histogram)

{"http.server.request.duration"} (quoted, dots kept)

http_server_request_duration_seconds

metric hermetica_outbox_lag_seconds (gauge)

hermetica_outbox_lag_seconds

hermetica_outbox_lag_seconds

monotonic sum jobs.processed

{"jobs.processed"}

jobs_processed_total

data point attribute http.route

label "http.route" (quote it: {"http.route"="/users"})

label http_route

resource service.name

label service

label service

the ingest key's environment

label env (the environment name) and env_id

same

resource service.instance.id / host.name

labels instance / host_name

same

other resource attributes

not labels; promote them with promote_resource_attributes, or join target_info

same, escaped

delta temporality

label __temporality__="delta" (hidden by rate/increase)

same

Examples with the default strategy:

max by (service) (hermetica_outbox_lag_seconds{service="hermetica-worker", env="production"})
histogram_quantile(0.95, sum by (le, "http.route") (rate({"http.server.request.duration_bucket", service="api"}[5m])))

A name with characters outside [a-zA-Z0-9_:] must be quoted inside the braces; a plain name can stay bare. env="production" and env_id="env_..." both select an environment. The metric explorer (Metrics > Explore) lists the stored names and labels of a project.

Name rules under UnderscoreEscapingWithSuffixes

The name is built in this order (NoTranslation does none of it):

  1. Escape. Each character outside [a-zA-Z0-9_:] (metric name) or [a-zA-Z0-9_] (label name) becomes _, runs of _ collapse to one, and a name starting with a digit gets a prefix: _ for a metric, key_ for a label (2xx -> _2xx, label 1a -> key_1a). A __-prefixed name is left as is.

  2. Counter _total. A monotonic sum has any trailing _total stripped here (it is re-added last), so it is never doubled.

  3. Unit suffix. The metric's UCUM unit maps to a Prometheus unit word, appended as _<word>, but only when the word is non-empty and the name does not already contain it (so ..._seconds is not added to a name that already ends _seconds). A gauge whose unit is 1 gets _ratio instead.

  4. Counter _total. A monotonic sum then gets _total appended.

So gauge http.server.request.duration (unit s) -> http_server_request_duration_seconds, and the same name as a monotonic sum -> http_server_request_duration_seconds_total.

UCUM unit -> Prometheus unit word

UCUM

Word

UCUM

Word

d

days

By

bytes

h

hours

KiBy / MiBy / GiBy / TiBy

kibibytes / mebibytes / gibibytes / tibibytes

min

minutes

KBy / MBy / GBy / TBy

kilobytes / megabytes / gigabytes / terabytes

s

seconds

m

meters

ms

milliseconds

V / A / J / W

volts / amperes / joules / watts

us

microseconds

g

grams

ns

nanoseconds

Cel

celsius

Hz

hertz

%

percent

A rate unit <unit>/<per> becomes <word>_per_<per-word>, where the per-word is second, minute, hour, day, week, month or year (from s, m, h, d, w, mo, y): By/s -> bytes_per_second. A braced annotation or 1 numerator over a rate gives just per_<per-word> ({request}/s -> per_second). A braced annotation alone ({request}), the plain 1, and any unit not in this table contribute no suffix.

Updated