- The log sinks that output logs to different locations, including over the network.
- The that are mapped to each sink.
- The used by the log messages.
- The redaction of log messages.
- The buffering of log messages.
In a future major release, certain events will be directed by default to new .To prepare for the change and assess potential downstream impacts on your logging setup and pipelines, review the cluster setting. After reviewing the documentation, set
log.channel_compatibility_mode.enabled to false in a non-production environment.Flag
To configure the logging behavior of acockroach command, include one of these flags with the command:
- , where is the YAML payload
- , where is the path to a YAML file
--log-dir to a blank directory (--log-dir=) instead of using one of the other logging flags. Do not use --log-dir=""; this creates a new directory named "" and stores log files in that directory.
To apply changes to an existing node’s logging configuration, restart the node with the updated --log flag or point it to the updated --log-config-file. CockroachDB reads these settings during startup. Before restarting, use cockroach debug check-log-config to verify the effective configuration produced by your changes.
YAML payload
All log settings for acockroach command are specified with a YAML payload in one of the following formats:
-
Block format, where each parameter is written on a separate line. For example, after creating a file
logs.yaml, pass the YAML values with either--log-config-fileor--log: -
Inline format, where all parameters are specified on one line. For example, to generate an
opslog file that collects theOPSandHEALTHchannels (overriding the file groups defined for those channels in the default configuration):Note that the inline spaces must be preserved.
When you provide a minimal logging configuration for
file-groups, CockroachDB replaces the entire default file-groups structure instead of merging only the specified fields. As a result, all logging channels flow into the default file sink, which can produce noisy logs and rapid log rotation. Partial YAML overrides that merge with the default logging configuration are not supported. To avoid misconfiguration, validate your logging settings using the cockroach debug check-log-config command.stderr. Note that each file sink has a max-group-size: 100MiB.
DEV and OPS channels from the default file sink to a new file sink called debug and increases the max-group-size for the default file sink to 1000MiB:
max-group-size for the default file sink and adds a new debug file sink with the DEV and OPS channels. However, it also removes all other file sinks, such as changefeed, health, and kv-distribution. The entire file-groups structure is replaced rather than merging only the specified fields.
Log channels cannot be completely disabled. You can configure and , but you cannot turn off a log channel entirely. For example, a channel that is not explicitly configured for a is enabled in the
default file group.cockroach debug check-log-config
The cockroach debug check-log-config command helps you validate logging settings before starting or restarting nodes. It returns the YAML definitions and a URL to a visualization of the default logging configuration.
When you run it without logging flags, it prints the default log configuration. If you pass a logging flag (for example, ), the output shows the configuration after your YAML payload is applied, so you can confirm the final sink, channel, and formatting behaviors.
Configure log sinks
Log sinks route events from specified to destinations outside CockroachDB. These destinations include:- Log files
- Fluentd -compatible servers
- HTTP servers
- OTLP -compatible servers
- The standard error stream (
stderr)
sinks:
| Parameter | Description | |
|---|---|---|
filter | Minimum severity level at which logs enter the channels selected for the sink. Accepts one of the valid or NONE, which excludes all messages from the sink output. For details, see Set logging levels. | |
format | Log message format to use for file or network sinks. Accepts one of the valid . For details, see file logging format, Fluentd logging format, HTTP logging format, and OTLP logging format. | |
format-options | Customization options for specified format. For available options for each format, see . For an example use case, see Set timezone. | |
redact | When true, enables automatic redaction of personally identifiable information (PII) from log messages. This ensures that sensitive data is not transmitted when collecting logs centrally or over a network. For details, see Redact logs. | |
redactable | When true, preserves redaction markers around fields that are considered sensitive in the log messages. The markers are recognized by and but may not be compatible with external log collectors. For details on how the markers appear in each format, see . | |
exit-on-error | When true, stops the Cockroach node if an error is encountered while writing to the sink. We recommend enabling this option on file sinks in order to avoid losing any log entries. When set to false, this can be used to mark certain sinks (such as stderr) as non-critical. | |
auditable | If true, enables exit-on-error on the sink. Also disables buffered-writes if the sink is under file-groups. This guarantees non-repudiability for any logs in the sink, but can incur a performance overhead and higher disk IOPS consumption. This setting is typically enabled for . File-based audit logging cannot coexist with the buffering configuration, so disable either buffering or auditable. |
file-defaults (for file sinks), fluent-defaults (for Fluentd sinks), http-defaults (for HTTP sinks), and otlp-defaults (for OTLP sinks).
Output to files
CockroachDB can write messages to one or more log files.file-groups specifies the channels that output to each log file, along with its output directory and other configuration details. For example:
| Parameter | Description |
|---|---|
channels | List of channels that output to this sink. Use a YAML array or string of , ALL to include all channels, or ALL EXCEPT {channels} to include all channels except the specified channel names. For more details on acceptable syntax, see Logging channel selection. |
dir | Output directory for log files generated by this sink. |
max-file-size | Approximate maximum size of individual files generated by this sink. Used to specify the maximum size that a log file can grow before a new log file is created. Accepts a valid file size, such as 2MiB. Default:10MiB |
max-group-size | Approximate maximum combined size of all files to be preserved for this sink. Configures the maximum size for a logging group (for example, cockroach, cockroach-sql-audit, cockroach-auth, cockroach-sql-exec, cockroach-pebble), after which the oldest log file is deleted. An asynchronous garbage collection removes files that cause the file set to grow beyond this specified size. Accepts a valid file size, such as 1GiB. For high-traffic deployments, or to ensure log retention over longer periods of time, consider raising this value to 500MiB or 1GiB. Default:100MiB |
file-permissions | The chmod-style permissions on generated log files, formatted as a 3-digit octal number. The executable bit must not be set. Default:640 (readable by the owner or members of the group, writable by the owner). |
buffered-writes | When true, enables buffering of writes. Set to false to flush every log entry (i.e., propagate data from the cockroach process to the OS) and synchronize writes (i.e., ask the OS to confirm the log data was written to disk). Disabling this setting provides non-repudiation guarantees, but can incur a performance overhead and higher disk IOPS consumption. This setting is typically disabled for . |
buffering | buffering is disabled by default for file log sinks. Default: buffering: NONE. Note that enabling asynchronous buffering of file log sinks is in .To configure buffering of log messages for the sink, use the following sub-parameters:
max-staleness and flush-trigger-size are used together, whichever is reached first will trigger the flush. This setting is typically disabled for . For a usage example, refer to .File-based audit logging cannot coexist with this buffering configuration, so disable either buffering or auditable. |
file-defaults (except channels, which uses the default configuration).
Log file naming
Log files are named in the following format:health will generate a log file that looks like this:
The files generated for a group named
default are named after the pattern cockroach.{metadata}.log.Access in DB Console
Log files can be accessed using the DB Console, which displays them in JSON format.- and then click in the left-hand navigation.
- Under Raw Status Endpoints (JSON), click Log Files to view the JSON of all collected logs.
- Copy one of the log filenames. Then click Specific Log File and replace the
cockroach.logplaceholder in the URL with the filename.
Known limitations
Log files can only be accessed in the DB Console if they are stored in the same directory as the file sink for theDEV channel.
Output to Fluentd-compatible network collectors
CockroachDB can send logs over the network to a Fluentd-compatible log collector (e.g., Elasticsearch, Splunk).fluent-servers specifies the channels that output to a server, along with the server configuration details. For example:
A Fluentd sink can be listed more than once with different
address values. This routes the same logs to different Fluentd servers.| Parameter | Description |
|---|---|
channels | List of channels that output to this sink. Use a YAML array or string of , ALL to include all channels, or ALL EXCEPT {channels} to include all channels except the specified channel names. For more details on acceptable syntax, see Logging channel selection. |
address | Network address and port of the log collector. |
net | Network protocol to use. Options are tcp, tcp4, tcp6, udp, udp4, udp6, or unix. Default:tcp Note: In most cases, use the default of tcp rather than udp. The UDP/IP protocol has a ~65 KB (65,507 bytes) limit per datagram. This limit is enforced by the operating system. Log lines that exceed this limit return a message too long error. |
buffering | Configures buffering of log messages for the sink, with the following sub-parameters:
max-staleness and flush-trigger-size are used together, whichever is reached first will trigger the flush. buffering is enabled by default for Fluentd-compatible log sinks. To explicitly disable log buffering, specify buffering: NONE instead. This setting is typically disabled for . See Log buffering for more details and usage. |
Output to HTTP network collectors
CockroachDB can send logs over the network to an HTTP server.http-servers specifies the channels that output to a server, along with the server configuration details. For example:
An HTTP sink can be listed more than once with different
address values. This routes the same logs to different HTTP servers.| Parameter | Description | |
|---|---|---|
channels | List of channels that output to this sink. Use a YAML array or string of , ALL to include all channels, or ALL EXCEPT {channels} to include all channels except the specified channel names. For more details on acceptable syntax, see Logging channel selection. | |
address | Network address and port of the log collector. | |
method | HTTP method to use. Can be GET or POST. Default:POST | |
unsafe-tls | When true, bypasses TLS server validation. Default:false | |
timeout | Timeout before requests are abandoned. Default:0 (no timeout) | |
disable-keep-alives | When true, disallows reuse of the server connection across requests. Default:false (reuses connections) | |
compression | Compression method for the HTTP request body. Valid values gzip or none. Default:gzip | |
headers | Map of key-value string pairs which will be appended to every request as custom HTTP headers. | |
file-based-headers | Map of key-filepath string pairs. The file specified by the filepath only contains a value that corresponds to the key. The key from the YAML configuration and the value contained in the file will be appended to every request as a custom HTTP header. For example: file-based-headers: {X-CRDB-HEADER-KEY: /some/path/to/file\_with\_value, X-ANOTHER-HEADER-KEY: /other/path/to/file\_with\_another\_value} A value in a file can be updated without restarting the cockroach process. Instead, send SIGHUP to the cockroach process to notify it to refresh values. | |
buffering | Configures buffering of log messages for the sink, with the following sub-parameters:
max-staleness and flush-trigger-size are used together, whichever is reached first will trigger the flush. buffering is enabled by default for HTTP log sinks. To explicitly disable log buffering, specify buffering: NONE instead. This setting is typically disabled for . See Log buffering for more details and usage. |
compression, headers, file-based-headers, and buffering parameters, see .
Output to OTLP-compatible network collectors
CockroachDB can send logs to an OpenTelemetry-compatible server using the OTLP protocol (e.g., Datadog agent). For a complete list of log servers that support the OTLP protocol, see the OpenTelemetry registry. Defineotlp-servers to select channels and configure connection and protocol details. For example:
| Parameter | Description |
|---|---|
channels | List of channels that output to this sink. Use a YAML array or string of , ALL, or ALL EXCEPT {channels}. See Logging channel selection for syntax. |
address | Network address of the OTLP collector endpoint for log ingestion, formatted as {host}:{port}. |
mode | Protocol used to export logs. Valid values grpc or http. Default:grpc |
headers | Map of key-value string pairs which will be appended to every request as custom gRPC or HTTP headers depending on the mode selected. For example, "x-api-key": "YOUR\_API\_KEY\_HERE". |
compression | Compression for requests. Valid values gzip or none. Default:gzip |
buffering | Configures buffering of log messages for the sink, with the following sub-parameters:
max-staleness and flush-trigger-size are used together, whichever is reached first will trigger the flush. buffering is enabled by default for OTLP log sinks. To explicitly disable log buffering, specify buffering: NONE instead. This setting is typically disabled for . See Log buffering for more details and usage. |
Output to stderr
CockroachDB can output messages to the standard error stream (stderr), which prints them to the machine’s terminal but does not store them. stderr specifies the channels that output to the stream. For example:
stderr accepts the following additional parameters:
The
format parameter for stderr can be set to any one of the valid . It is set to by default.| Parameter | Description |
|---|---|
channels | List of channels that output to this sink. Use a YAML array or string of , ALL to include all channels, or ALL EXCEPT {channels} to include all channels except the specified channel names. For more details on acceptable syntax, see Logging channel selection. |
no-color | When true, removes terminal color escape codes from the output. |
stderr with log analyzers. We recommend using file logging or network logging via Fluentd or HTTP instead of stderr when integrating with automated monitoring software.
By default,
cockroach start and cockroach start-single-node do not print any messages to stderr. However, if the cockroach process does not have access to on-disk storage, all messages are printed to stderr.Logging channel selection
Log channels cannot be completely disabled. You can configure and , but you cannot turn off a log channel entirely. For example, a channel that is not explicitly configured for a is enabled in thedefault file group.
For each sink, multiple channels can be selected. Note that:
- Spacing between items will vary according to the syntax used.
- Channel selection is case-insensitive.
filter in the sink configuration is used. However, you can specify a different severity level for each channel. For more information on severity levels, see Set logging levels.
These selections are equivalent:
all keyword:
all except keyword prefix:
Configure logging defaults
When setting up a logging configuration, it’s simplest to define shared parameters infile-defaults, fluent-defaults, http-defaults, and otlp-defaults and override specific values as needed in file-groups, fluent-servers, http-servers, otlp-servers and stderr. For a complete example, see the default configuration.
This section describes some recommended defaults.
Set file defaults
Defaults for log files are set infile-defaults, which accepts all common sink parameters and the file group parameters dir, max-file-size, max-group-size, file-permissions, and buffered-writes.
Logging directory
By default, CockroachDB adds log files to alogs subdirectory in the first on-disk (default: cockroach-data):
File logging format
The default message format for log files is . Eachcrdb-v2 log message starts with a flat prefix that contains event metadata (e.g., severity, date, timestamp, channel), followed by the event payload. For details on the metadata, see .
If you plan to read logs programmatically, you can switch to a or format:
format refers to the envelope of the log message, which contains the event metadata. This is separate from the event payload, which corresponds to its .Set Fluentd defaults
Defaults for Fluentd-compatible network sinks are set influent-defaults, which accepts all common sink parameters.
Note that the server parameters address and net are not specified in fluent-defaults:
addressmust be specified for each sink underfluent-servers.netis not required and defaults totcp.
Fluentd logging format
The default message format for network output is . Each log message is structured as a JSON payload that can be read programmatically. Thejson-fluent-compact and formats include a tag field that is required by the Fluentd protocol. For details, see .
format refers to the envelope of the log message. This is separate from the event payload, which is structured according to .Set HTTP defaults
Defaults for HTTP sinks are set inhttp-defaults, which accepts all common sink parameters.
Note that the server parameters address and method are not specified in http-defaults:
addressmust be specified for each sink underhttp-servers.methodis not required and defaults toPOST.
HTTP logging format
The default message format for HTTP output is . Each log message is structured as a JSON payload that can be read programmatically. For details, see .format refers to the envelope of the log message. This is separate from the event payload, which is structured according to .
Set OTLP defaults
Defaults for OTLP sinks are set inotlp-defaults, which accepts all common sink parameters.
Note that the server parameter addressis not specified in otlp-defaults. The address must be specified for each sink under otlp-servers.
OTLP logging format
The default message format for OTLP output is . Each log message is structured as a JSON payload that can be read programmatically. For details, see .format refers to the envelope of the log message. This is separate from the event payload, which is structured according to .
Set logging levels
Log messages are associated with a when they are generated. Each logging sink accepts messages from each logging channel at a minimum severity level. This minimum severity level can be specified per sink or by default using thefilter attribute.
Messages with severity levels below the configured threshold do not enter logging channels and are discarded.
The default configuration uses the following severity levels for and :
file-defaults,fluent-defaults, andhttp-defaultseach usefilter: INFO. SinceINFOis the lowest severity level, file and network sinks will emit all log messages.stderrusesfilter: NONEand does not emit log messages.- The
defaultfile group usesfilter: INFOfor events from theDEVandOPSchannels, andfilter: WARNINGfor all others.
All other
cockroach commands use filter: WARNING and log to stderr by default, with these exceptions:- uses
filter: INFO. - uses
filter: NONE(discards all log messages).
file-defaults, fluent-defaults, and http-defaults severity levels on a per-sink basis.
For example, this will include DEV events at severity WARNING:
filter attribute and set severity levels on a per-channel basis.
For example, this will include DEV events at severity INFO, and OPS events at severity ERROR:
Set timezone
Using theformat-options sink parameter, you may set the timezone for a specified log format.
timezone for text formats
The log output formats and support the format option timezone. When specified, the corresponding timezone is used to produce the timestamp column. The value can be any timezone name recognized by the Go standard library.
For example:
datetime field for JSON format
The log output format supports the format options datetime-format and datetime-timezone.
datetime-format controls whether a datetime field with human-readable timestamp values is included in the logs. Valid values for this option are:
none: Disable the creation of thedatetimefield. This is the default value.iso8601orrfc3339: Format the timestamp like “2006-01-02T15:04:05.999999999Z”.rfc1123: Format the timestamp like “Mon, 02 Jan 2006 15:04:05 +0000”.
timestamp field with epoch values remains unchanged for backward-compatibility.
datetime-timezone selects which timezone to use when formatting the datetime field. The value can be any timezone name recognized by the Go standard library. The default value is UTC. datetime-timezone must be combined with datetime-format set to a valid format since the default value for the datetime-format option is none and therefore the datetime field is not included by default.
For example:
Enabling the
datetime field introduces CPU overhead and is nearly always unnecessary.When sending output to a log collector (for example, via Fluent or Datadog), the log collector can be configured to transform the epoch values in the timestamp field into a human-readable format.When inspecting a formatted log file produced by CockroachDB, you can use the command to convert the log into format which includes timestamps in the rfc3339 format.Redact logs
CockroachDB can redact personally identifiable information (PII) from log messages. The logging system includes two parameters that handle this differently:redactis disabled by default. When enabled,redactautomatically redacts sensitive data from logging output. We do not recommend enabling this on theDEVchannel because it impairs our ability to troubleshoot problems.redactableis enabled by default. This places redaction markers around sensitive fields in log messages. These markers are recognized by and , which aggregate CockroachDB log files and can be instructed to redact sensitive data from their output.
redact:
cockroach debug redaction markers (< >), since they are specific to CockroachDB. To prevent this issue when using network sinks, disable redactable:
< >) to redact. In this case, enable redactable.
DEV channel
TheDEV channel is used for debug and uncategorized messages. It can therefore be noisy and contain sensitive (PII) information.
We recommend configuring DEV separately from the other logging channels. When sending logs to a Fluentd-compatible or HTTP network collector, DEV logs should also be excluded from network collection.
In this example, the dev file group is reserved for DEV logs. These are output to a cockroach-dev.log file in a custom disk dir:
Log buffering for network sinks
The network (Fluentd-compatible, HTTP, and OTLP-compatible) log sinks support the buffering of log messages by default. With log buffering configured, log messages are held in a buffer for a configurable time period or accumulated message size threshold before being written to the target log sink together as a batch. Log buffering helps to ensure consistent low-latency log message writes over the network even in high-traffic, high-contention scenarios. The following shows a basic log configuration with buffering configured for a Fluentd-compatible log sink:- CockroachDB will hold log messages in a buffer for up to
20s(themax-stalenesssetting), or up to a collected size of2MiB(theflush-trigger-sizesetting), before writing (“flushing”) the buffer to the log file. When both settings are used, whichever case is met first triggers the buffer flush. - If at any point the accumulated message size of buffer flushes (as triggered by reaching either the configured
max-stalenessorflush-trigger-sizevalue) exceeds100MiB(themax-buffer-sizesetting), all new incoming log messages received are dropped until the accumulated message size in the buffer once more falls below this value. - For the
HEALTHonly, override thefile-defaultsvalue of20sformax-staleness, instead flushing messages to the log file within up to2s.
systemd.
Alternatively, you may explicitly disable log buffering by setting buffering to NONE. The following log configuration explicitly disables log buffering for just the OPS channel on a Fluentd-compatible log sink:
Stray error capture
Certain events, such as uncaught software exceptions (panics), bypass the CockroachDB logging system. However, they can be useful in troubleshooting. For example, if CockroachDB crashes, it normally logs a stack trace to what caused the problem. To ensure that these stray errors can be tracked, CockroachDB does not send them tostderr by default. Instead, stray errors are output to a cockroach-stderr.log file in the default logging directory.
You can change these settings in capture-stray-errors:
When
capture-stray-errors is disabled, redactable cannot be enabled on the stderr sink. This is because stderr will contain both stray errors and logged events and cannot apply redaction markers in a reliable way. Note that redact can still be enabled on stderr in this case.Default logging configuration
The YAML payload below represents the default logging behavior of and .For high-traffic deployments that output log messages to files, consider raising
file-defaults: {max-group-size} to 500MiB or 1GiB to extend log retention.dir is not specified for file-defaults and capture-stray-errors:
- The default
dirforfile-defaultsis inferred from the first on-disk . See Logging directory. - The default
dirforcapture-stray-errorsis inherited formfile-defaults.

