Each node in a CockroachDB cluster exports granular time-series metrics to two available Prometheus-compatible endpoints:
The metrics are formatted for integration with Prometheus, an open-source tool for storing, aggregating, and querying time-series data. For details on how to pull these metrics into Prometheus, refer to . The Prometheus format is human-readable and can be processed to work with other Prometheus-compatible third-party monitoring systems, such as Sysdig and Google Cloud Managed Service for Prometheus. Many of the , such as Datadog and Kibana, collect metrics from the cluster’s Prometheus endpoint.
In addition to using the exported time-series data to monitor a cluster through an external system, you can write alerting rules to ensure prompt notification of critical events or issues requiring intervention or investigation. Refer to for more details.
_status/vars
To access the _status/vars Prometheus endpoint of a cluster running on localhost:8080:
The output will be similar to the following. Note that the metric names are unique for sql_*_count*.
metrics
The metrics Prometheus endpoint is commonly used and is the default in Prometheus configurations.
To access the metrics Prometheus endpoint of a cluster running on localhost:8080:
The output will be similar to the following. Note that there is one metric name for sql_count, with static labels for query_type (with values of insert, select, update, and delete) and query_internal (with value of true).
Static labels
One common use of static labels is to allow segmentation of a metric across various facets for later querying and aggregation. For example, rather than emitting separate metrics for INSERT, SELECT, UPDATE, and DELETE statements, the single metric sql_count uses the query_type label to distinguish between these operations. This enables operators to easily aggregate across query types (e.g., summing a metric for all SQL operations) or filter for a specific type using the appropriate value for the query_type label.
The following tables contrast unlabeled metrics from the _status/vars endpoint with their labeled counterparts from the metrics endpoint:
Unlabeled metrics from the _status/vars endpoint | Labeled metrics from the metrics endpoint |
|---|
sql_insert_count | sql_count{query_type="insert"} |
sql_select_count | sql_count{query_type="select"} |
sql_update_count | sql_count{query_type="update"} |
sql_delete_count | sql_count{query_type="delete"} |
Unlabeled sum query from the _status/vars endpoint | Labeled sum query from the metrics endpoint |
|---|
sum(sql_insert_count, sql_delete_count, sql_select_count) | sum(sql_count) |
| This query must be updated if new types are added, as each will have a unique metric name. | This query is resilient to new type additions. |
| Related metrics can be found via autocomplete in a third-party tool, but the results may be ambiguous. | All label values can be found through a third-party query engine and used to easily construct a graph with individual lines for each label value. |
security_certificate_expiration is reused, with the certificate type expressed as a label. The metrics endpoint returns output similar to the following:
This approach avoids a proliferation of metric names and allows third-party tools to display each certificate’s expiration as a separate line in a single graph or table.
See also
