NoSQL, search and key-value depth (track A2)

Agent modules that go past counters for document, search and key-value stores. Shared rules:

• No values leave the host. Document queries (MongoDB filters, explain plans) are reduced to their shape ({"status":"?","total":{"$gt":"?"}}); Redis and Memcached commands keep the command name and a key pattern (GET user:*:cart); Cypher and SQL text goes through the bind-safe normalizer.

• Statements join on db.query.hash: XXH3-64 (seed 0) as a decimal string over the normalized text, the canonical algorithm of sql-normalization.md. SQL goes through the canonical normalizer (dbm.NormalizeSQL); MongoDB shapes, Redis command patterns and Cypher text are hashed as they are, with the same XXH3.

• Samples are log records with event.name = db.query.sample, db.query.plan, db.wait, db.keyspace, db.advice or db.shard and db.system.name (blocking chains: db.blocking), ready for the Datastores area (track E) to file through the db_samples path. Bodies are JSON, at most 8 KiB.

• Every part is best effort: a missing privilege skips that part, it never fails the round.

MongoDB (mongodb)

Settings: databases (default: every non-system database, at most 20; 50 collections each), slow_ms (default 1000).

Redis, Valkey, Dragonfly, KeyDB (redis)

One connection per round. The server kind comes from INFO (redis.server.info{db.system.name}).

key_sample: keys sized per pass (default 1000; -1 turns the pass off). The ACL user needs +info +slowlog +latency +cluster|info +memory +randomkey +object|freq.

Memcached (memcached)

stats plus stats slabs and stats items on the same connection: chunk size, used/free chunks, pages, items, oldest item age, evictions, last evicted item age (only for classes that evicted) and out-of-memory refusals per slab class, and evicted-unfetched items. The stock alert fires when a class evicts items younger than a minute.

ClickHouse (clickhouse)

Over the HTTP interface, every minute, as a read-only user:

• system.query_log digests per normalized_query_hash (calls, time, rows read, memory, errors); the text label now goes through normalizeQuery, so no literal reaches a series. The 5 slowest queries of the minute are db.query.sample records, normalized in the agent with the canonical normalizer (only that text leaves the host).

• system.parts: active parts, rows and bytes per table and the worst partition's part count (the "too many parts" limit), system.merges (running merges and mutations, longest merge), system.mutations (pending, failing, oldest age), system.replicas (read-only, delay, queue, active/total).

Grant: GRANT SELECT ON system.* TO wt_monitor.

Elasticsearch and OpenSearch (elasticsearch)

The integration's enable guidance now starts this module (the http_status kind still reads _cluster/health alone). The distribution is read once from GET / and becomes db.system.name on samples.

Slowlog: the slowlog is a file on each node, not an API; tail *_index_search_slowlog.json and *_index_indexing_slowlog.json with the logs module (JSON lines). Role for the agent: monitor cluster privilege and monitor on indices.

Cassandra (cassandra) and ScyllaDB (scylladb)

Cassandra: with url pointing at a Jolokia agent (http://host:8778/jolokia, now the default) the module reads the MBeans itself; any other url is scraped as the cassandra-exporter or JMX exporter, as before.

• Per table (system keyspaces left out, at most 300 tables): local read and write p99 and counts, tombstones scanned per read (p99 and max), SSTables per read p99, pending compactions, live disk bytes.

• Node: pending and completed compactions, bytes compacted; coordinator p99 latency, timeouts, unavailables and failures per operation; pending and blocked tasks per stage; dropped messages per verb.

Stock alerts: tombstones per read past the 1000 warn threshold, compaction backlog over 100, any dropped messages.

ScyllaDB: its own Prometheus endpoint (:9180/metrics, scylla_* families: reactor utilization per shard, coordinator latency and timeouts, compactions, row cache, per-table tombstones when enabled).

etcd (etcd)

The member's /metrics (point url at the plain-HTTP --listen-metrics-urls port, 2381 by convention, when 2379 requires client certificates): etcd_server_* (leader, proposals committed/applied/pending/failed, slow applies, heartbeat send failures), etcd_mvcc_*, etcd_disk_*, etcd_network_*. The module adds:

• interval p99 of WAL fsync, backend commit, apply and per-peer round trip, from bucket deltas between scrapes (a cumulative histogram's p99 since start would hide a disk going slow now);

• etcd.db.fragmentation, etcd.db.quota_used, etcd.db.defrag_recommended (over half the file and 100 MiB free).

Stock alerts: WAL fsync p99 over 10 ms, database over 80% of the quota, defrag recommended.

Neo4j (neo4j)

HTTP transactional API (POST /db/system/tx/commit), Neo4j 4.4+/5: SHOW TRANSACTIONS gives running transactions per database, those past slow_ms (a db.query.sample each: Cypher with every string and number literal removed, running time, lock count) and lock waits; "Blocked by" statuses become blocking chains (db.blocking.chain_depth, db.sessions.blocked, one chain record per root, as for Postgres and MySQL). SHOW DATABASES gives neo4j.database.online per database and role. Page cache and store metrics: Neo4j's Prometheus endpoint (where the edition has it) through the prometheus module. The user needs SHOW TRANSACTION and SHOW DATABASE.

CouchDB (couchdb)

/_node/_local/_stats is walked generically: every counter, gauge and histogram (mean, p50, p90, p99) becomes couchdb.<section>.<name>, with HTTP status codes and methods as a key label. /_active_tasks: count, oldest age and mean progress per task type. /_scheduler/docs: replications by state (source and target are never reported; their URLs can hold credentials). Every 10th round, for up to 100 databases: file and active size, fragmentation, documents and deleted documents.

Not covered

• DuckDB and MotherDuck: DuckDB is embedded (no server to poll) and is covered by the SDK tracks (C1, C2); MotherDuck exposes no monitoring API to an agent.

• Couchbase stays on its Prometheus endpoint (couchbase module).