If you need to monitor your cluster, tune performance, or issues, you can check the CockroachDB logs, which include details about notable cluster, node, and range-level events.
Details
When a node starts, shuts down, or processes a , it produces a stream of messages about the command’s activities. Each message is composed of:
- A payload that contains events either structured in JSON or conveyed in an arbitrary string. For details on structured event types and their fields, see .
- An envelope that contains event metadata (e.g., severity, date, timestamp, channel). Depending on the log format you specify when , the envelope can be formatted either in JSON or as a flat prefix to the message.
Messages are organized into appropriate logging channels and then routed through log sinks. Each sink further processes and filters the messages before emitting them to destinations outside CockroachDB. The mapping of channels to sinks, as well as the processing and filtering done by each sink, is configurable and is intended to support multiple . For more information on configuring sinks and severity levels, see .
All support logging. However, note that most messages related to cluster operation are generated by or . Other commands generate messages related to their own execution, which are mainly useful when troubleshooting the behaviors of those commands.
Logging channels
Log messages in CockroachDB are directed into logging channels, which can in turn be assigned to output to one or more .
This allows you to group channels that log related information (e.g., operational, security, or SQL events) into their own sinks. Each sink can output to a predetermined destination where the logs can be collected and parsed. For usage examples, see .
| Channel | Description |
|---|
| Uncategorized and debug messages. |
| Process starts, stops, shutdowns, and crashes (if they can be logged); changes to cluster topology, such as node additions, removals, and decommissions. |
| Resource usage; node-node connection events, including connection errors; up- and down-replication and range unavailability. |
| Low-level storage logs from the . |
| Client connections and disconnections (when enabled via the server.auth\_log.sql\_connections.enabled); SQL authentication logins/attempts and session/query terminations (when enabled via the server.auth\_log.sql\_sessions.enabled). |
| Database, schema, table, sequence, view, and type creation; changes to table columns and sequence parameters. |
| Changes to users, roles, and authentication credentials. |
| Changes to privileges and object ownership. |
| SQL audit events (when enabled via ). |
| SQL statement executions (when enabled via the sql.trace.log\_statement\_execute) and uncaught Go panic errors during SQL statement execution. Warning: Logging cluster-wide executions by enabling the sql.trace.log\_statement\_execute cluster setting will incur considerable overhead and may have a negative performance impact. |
| SQL executions that impact performance, such as slow queries (when enabled via the sql.log.slow\_query.latency\_threshold and/or sql.log.slow\_query.experimental\_full\_table\_scans.enabled). |
| Telemetry events for internal usage, except for which may be . |
| Data distribution events, such as moving between in a cluster, adding replicas to , and removing replicas from ranges. |
Logging channels are analogous to logging facilities in Syslog or logging services in Datadog. For more details on the contents of each logging channel, see the .
Logging destinations
When using the , the events collected on each logging channel are split into log files as follows:
| Filename | Description | Channels |
|---|
cockroach.log | General CockroachDB log | DEV |
cockroach-health.log | Health log | HEALTH |
cockroach-kv-distribution.log | Data distribution log | KV\_DISTRIBUTION |
cockroach-security.log | SQL security log | PRIVILEGES + USER\_ADMIN |
cockroach-sql-audit.log | SQL access audit log | SENSITIVE\_ACCESS |
cockroach-sql-auth.log | SQL authentication log | SESSIONS |
cockroach-sql-exec.log | SQL execution log | SQL\_EXEC |
cockroach-sql-slow.log | SQL slow query log | SQL\_PERF |
cockroach-sql-schema.log | SQL schema change log | SQL\_SCHEMA |
cockroach-pebble.log | Pebble log | STORAGE |
cockroach-telemetry.log | Telemetry log | TELEMETRY |
Logging destinations and many other parameters are configurable via YAML. For details, see .
See also