Multi-Region Capabilities Overview

This page provides an overview of CockroachDB multi-region capabilities.

Overview

CockroachDB multi-region capabilities make it easier to run global applications. To use these capabilities effectively, you should understand the following concepts:

• Cluster region is a geographic region that you specify at node start time.
• Database region is a geographic region in which a database operates. You must choose a database region from the list of available cluster regions.
• Survival goal dictates how many simultaneous failure(s) a database can survive.
• Table locality determines how CockroachDB optimizes access to a table’s data.

At a high level, the simplest process for running a multi-region cluster is:

1. Set region information for each node in the cluster at startup using .
2. Add one or more regions to a database, making it a “multi-region” database. One of these regions must be the primary region.
3. (Optional) Change table localities (global, regional by table, regional by row). This step is optional because by default the tables in a database will be homed in the database’s primary region (as set during Step 1).
4. (Optional) Change the database’s survival goals (zone or region). This step is optional because by default multi-region databases will be configured to survive zone failures.

These steps describe the simplest case, where you accept all of the default settings. The latter two steps are optional, but table locality and survival goals have a significant impact on performance. Therefore Cockroach Labs recommends that you give these aspects some consideration .

For new clusters using the , Cockroach Labs recommends lowering the setting to 250ms. This setting is especially helpful for lowering the write latency of . Nodes can run with different values for --max-offset, but only for the purpose of updating the setting across the cluster using a rolling upgrade.

Cluster regions

You define a cluster region at the node level using the region key and the zone using the zone key in the .

For example, the following command adds us-east-1 to the list of cluster regions and us-east-1b to the list of zones:

cockroach start --locality=region=us-east-1,zone=us-east-1b # ... other required flags go here

To show all of a cluster’s regions, execute the following SQL statement:

SHOW REGIONS FROM CLUSTER;

Database regions

A database region is a high-level abstraction for a geographic region. Each region is broken into multiple zones. These terms are meant to correspond directly to the region and zone terminology used by cloud providers.

The regions added during node startup become database regions when you add them to a database. To add the first region, use the .

While the database has only one region assigned to it, it is considered a “multi-region database.” This means that all data in that database is stored within its assigned regions, and CockroachDB optimizes access to the database’s data from the primary region. If the default survival goals and table localities meet your needs, there is nothing else you need to do once you have set a database’s primary region.

To add another database region, use the .

To show all of a database’s regions, execute the .

If the default survival goals and table localities meet your needs, there is nothing else you need to do once you have set a database’s primary region.

Super regions

This feature is in and subject to change. To share feedback and/or issues, contact Support.

Super regions allow you to define a set of database regions such that the following will have all of their replicas stored only in regions that are members of the super region:

• whose home region is a member of the super region.
• Any row of a whose is a member of the super region.

The primary use case for super regions is data domiciling. As mentioned above, data from and tables will be stored only in regions that are members of the super region. Further, if the super region contains 3 or more regions and if you use , the data domiciled in the super region will remain available if you lose a region.

To use super regions, keep the following considerations in mind:

• Your cluster must be a .
• Super regions must be enabled(/docs/v26.2/alter-database#enable-super-regions).
• Super regions can only contain one or more that have already been added with .
• Each database region can only belong to one super region. In other words, given two super regions A and B, the set of database regions in A must be disjoint from the set of database regions in B.
• You cannot that is part of a super region until you either to remove it, or altogether.

For more information about how to enable and use super regions, see:

Note that super regions take a different approach to data domiciling than . Specifically, super regions make it so that all (both voting and ) are placed within the super region, whereas PLACEMENT RESTRICTED makes it so that there are no non-voting replicas.

For a demo on Super Regions, watch the following video:

For more information about data domiciling using PLACEMENT RESTRICTED, see .

If you want to do data domiciling for databases with using the higher-level multi-region abstractions, you must use super regions. using the higher-level , you must use . Using will not work for databases that are set up with region survival goals.

Super regions rely on the underlying , which was historically built for performance, not for domiciling. The replication system’s top priority is to prevent the loss of data and it may override the zone configurations if necessary to ensure data durability. For more information, see .

If you are using super regions in your cluster, there are additional constraints when using secondary regions:

• If the is in a super region, the secondary region **must** be a region within the primary’s super region.
• If the primary region is not in a super region, the secondary region **must not** be within a super region.

Secondary regions

Secondary regions allow you to define a database region that will be used for failover in the event your goes down. In other words, the secondary region will act as the primary region if the original primary region fails.

Secondary regions work as follows: when a secondary region is added to a database, a is added to the tables and indexes in that database to ensure that two are moved into the secondary region.

This behavior is an improvement over versions of CockroachDB prior to v22.2. In those versions, when the primary region failed, the would be transferred to another database region at random, which could have negative effects on performance.

For more information about how to use secondary regions, see:

If you are using super regions in your cluster, there are additional constraints when using secondary regions:

• If the is in a super region, the secondary region **must** be a region within the primary’s super region.
• If the primary region is not in a super region, the secondary region **must not** be within a super region.

are not compatible with databases containing tables. CockroachDB does not prevent you from defining secondary regions on databases with regional by row tables, but the interaction of these features is not supported. Therefore, Cockroach Labs recommends that you avoid defining secondary regions on databases that use regional by row table configurations.

Survival goals

A survival goal dictates how many simultaneous failure(s) a database can survive. All tables within the same database operate with the same survival goal. Each database can have its own survival goal setting.

For more information, refer to .

Table localities

Table locality determines how CockroachDB optimizes access to the table’s data. Every table in a multi-region database has a “table locality setting” that configures one or more home regions at the table or row level.

For more information, refer to .

Additional features

The features listed in this section make working with multi-region clusters easier.

Zone config extensions

Zone Config Extensions are a customization tool for advanced users to persistently modify the configuration generated by the standard on a per-region basis.

For more information, see .

Schema changes in multi-region clusters

To reduce latency while making online schema changes, we recommend specifying a lease_preference on the system database to a single region and running all subsequent schema changes from a node within that region. For example, if the majority of online schema changes come from machines that are geographically close to us-east1, run the following:

ALTER DATABASE system CONFIGURE ZONE USING constraints = '{"+region=us-east1": 1}', lease_preferences = '[[+region=us-east1]]';

Access to tables and built-in functions in the system database is controlled by the . The above ALTER DATABASE system statement executes regardless of the variable’s setting because it does not access tables or invoke built-in functions.

Run all subsequent schema changes from a node in the specified region.

If you do not intend to run more schema changes from that region, you can safely for the system database.

Next steps

See also