Build a Python App with CockroachDB and Django

This tutorial shows you how build a simple Python application with CockroachDB and the Django framework.

CockroachDB supports Django versions 3.1+.

The example code and instructions on this page use Python 3.9 and Django 3.1.

Step 1. Start CockroachDB

Use CockroachDB Cloud

Choose your installation method

You can create a CockroachDB Basic cluster using either the CockroachDB Cloud Console, a web-based graphical user interface (GUI) tool, or ccloud, a command-line interface (CLI) tool.

Cloud Console (GUI)

Create a free cluster

Organizations without billing information on file can only create one CockroachDB Basic cluster.

1. If you haven’t already, sign up for a CockroachDB Cloud account.

2. Log in to your CockroachDB Cloud account.

3. On the Clusters page, click Create cluster.

4. On the Select a plan page, select Basic.

5. On the Cloud & Regions page, select a cloud provider (GCP or AWS) in the Cloud provider section.

6. In the Regions section, select a region for the cluster. Refer to for the regions where CockroachDB Basic clusters can be deployed. To create a multi-region cluster, click Add region and select additional regions.

7. Click Next: Capacity.

8. On the Capacity page, select Start for free. Click Next: Finalize.

9. On the Finalize page, click Create cluster.

   Your cluster will be created in a few seconds and the Create SQL user dialog will display.

Create a SQL user

The Create SQL user dialog allows you to create a new SQL user and password.

1. Enter a username in the SQL user field or use the one provided by default.

2. Click Generate & save password.

3. Copy the generated password and save it in a secure location.

4. Click Next.

   Currently, all new SQL users are created with admin privileges. For more information and to change the default settings, see .

Get the root certificate

The Connect to cluster dialog shows information about how to connect to your cluster.

1. Select General connection string from the Select option dropdown.
2. Open a new terminal on your local machine, and run the CA Cert download command provided in the Download CA Cert section. The client driver used in this tutorial requires this certificate to connect to CockroachDB Cloud.

Get the connection string

Open the General connection string section, then copy the connection string provided and save it in a secure location.

The connection string is pre-populated with your username, password, cluster name, and other details. Your password, in particular, will be provided only once. Save it in a secure place (Cockroach Labs recommends a password manager) to connect to your cluster in the future. If you forget your password, you can reset it by going to the SQL Users page for the cluster, found at https://cockroachlabs.cloud/cluster/<CLUSTER ID>/users.

ccloud CLI

Follow these steps to create a CockroachDB Basic cluster using the ccloud CLI tool.

The ccloud CLI tool is in Preview.

Install ccloud

Mac

You can install ccloud using either Homebrew or by downloading the binary.

Use Homebrew

1. Install Homebrew.

2. Install using the ccloud tap:

   brew install cockroachdb/tap/ccloud
   

Download the binary

In a terminal, enter the following command to download and extract the ccloud binary and add it to your PATH:

curl https://binaries.cockroachdb.com/ccloud/ccloud_darwin-amd64_.tar.gz | tar -xJ && cp -i ccloud /usr/local/bin/

Use the ARM 64 binary if you have an M-series Mac:

curl https://binaries.cockroachdb.com/ccloud/ccloud_darwin-arm64_.tar.gz | tar -xJ && cp -i ccloud /usr/local/bin/

Linux

In a terminal, enter the following command to download and extract the ccloud binary and add it to your PATH:

curl https://binaries.cockroachdb.com/ccloud/ccloud_linux-amd64_.tar.gz | tar -xz && cp -i ccloud /usr/local/bin/

Windows

In a PowerShell window, enter the following command to download and extract the ccloud binary and add it to your PATH:

$ErrorActionPreference = "Stop"; [Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12; $ProgressPreference = 'SilentlyContinue'; $null = New-Item -Type Directory -Force $env:appdata/ccloud; Invoke-WebRequest -Uri https://binaries.cockroachdb.com/ccloud/ccloud_windows-amd64_.zip -OutFile ccloud.zip; Expand-Archive -Force -Path ccloud.zip; Copy-Item -Force ccloud/ccloud.exe -Destination $env:appdata/ccloud; $Env:PATH += ";$env:appdata/ccloud"; # We recommend adding ";$env:appdata/ccloud" to the Path variable for your system environment. See https://docs.microsoft.com/powershell/module/microsoft.powershell.core/about/about_environment_variables#saving-changes-to-environment-variables for more information.

Run ccloud quickstart to create a new cluster, create a SQL user, and retrieve the connection string.

The easiest way of getting started with CockroachDB Cloud is to use ccloud quickstart. The ccloud quickstart command guides you through logging in to CockroachDB Cloud, creating a new CockroachDB Basic cluster, and connecting to the new cluster. Run ccloud quickstart and follow the instructions:

ccloud quickstart

The ccloud quickstart command will open a browser window to log you in to CockroachDB Cloud. If you are new to CockroachDB Cloud, you can register using one of the single sign-on (SSO) options, or create a new account using an email address.

The ccloud quickstart command will prompt you for the cluster name, cloud provider, and cloud provider region, then ask if you want to connect to the cluster. Each prompt has default values that you can select, or change if you want a different option.

Select General connection string, then copy the connection string displayed and save it in a secure location. The connection string is the line starting postgresql://.

? How would you like to connect? General connection string
Retrieving cluster info: succeeded
 Downloading cluster cert to /Users/maxroach/.postgresql/root.crt: succeeded
postgresql://maxroach:[e-mail]:26257/defaultdb?sslmode=verify-full&sslrootcert=%2FUsers%2Fmaxroach%2F.postgresql%2Froot.crt

Use a Local Cluster

1. If you haven’t already, .

2. Run the command:

   $ cockroach start-single-node --advertise-addr 'localhost' --insecure
   

   This starts an insecure, single-node cluster.

3. Take note of the following connection information in the SQL shell welcome text:

   CockroachDB node starting at 2021-08-30 17:25:30.06524 +0000 UTC (took 4.3s)
   build:               CCL v21.1.6 @ 2021/07/20 15:33:43 (go1.15.11)
   webui:               http://localhost:8080
   sql:                 postgresql://root@localhost:26257?sslmode=disable
   

   You’ll use the sql connection string to connect to the cluster later in this tutorial.

The --insecure flag used in this tutorial is intended for non-production testing only. To run CockroachDB in production, use a secure cluster instead.

Step 2. Get the sample code

Clone the code’s GitHub repo:

$ git clone https://github.com/cockroachlabs/example-app-python-django/

The project directory structure should look like this:

├── Dockerfile
├── README.md
├── cockroach_example
│   ├── cockroach_example
│   │   ├── __init__.py
│   │   ├── asgi.py
│   │   ├── migrations
│   │   │   ├── 0001_initial.py
│   │   │   └── __init__.py
│   │   ├── models.py
│   │   ├── settings.py
│   │   ├── urls.py
│   │   ├── views.py
│   │   └── wsgi.py
│   └── manage.py
└── requirements.txt

Step 3. Install the application requirements

To use CockroachDB with Django, the following modules are required:

• django
• psycopg2 (recommended for production environments) or psycopg2-binary (recommended for development and testing).
• django-cockroachdb

The major version of django-cockroachdb must correspond to the major version of django. The minor release numbers do not need to match.

The requirements.txt file at the top level of the example-app-python-django project directory contains a list of the requirements needed to run this application:

psycopg2-binary
django==5.0.1
django-cockroachdb==5.0

This tutorial uses virtualenv for dependency management.

1. Install virtualenv:

   $ pip install virtualenv
   

2. At the top level of the app’s project directory, create and then activate a virtual environment:

   $ virtualenv env
   
   $ source env/bin/activate
   

3. Install the modules listed in requirements.txt to the virtual environment:

   $ pip install -r requirements.txt
   

Step 4. Build out the application

Configure the database connection

Open cockroach_example/cockroach_example/settings.py, and configure the DATABASES dictionary to connect to your cluster using the connection parameters that you copied earlier.

DATABASES = {
    'default': {
        'ENGINE': 'django_cockroachdb',
        'NAME': '{database}',
        'USER': '{username}',
        'PASSWORD': '{password}',
        'HOST': '{host}',
        'PORT': '{port}',
        'OPTIONS': {
            'sslmode': 'verify-full'
        },
    },
}

For more information about configuration a Django connection to CockroachDB Standard, see Connect to a CockroachDB Cluster.

After you have configured the app’s database connection, you can start building out the application.

Models

Start by building some models, defined in a file called models.py. You can copy the sample code below and paste it into a new file, or you can download the file directly.

from django.db import models

class Customers(models.Model):
    id = models.UUIDField(
        primary_key=True,
        default=uuid.uuid4,
        editable=False)
    name = models.CharField(max_length=250)

class Products(models.Model):
    id = models.UUIDField(
        primary_key=True,
        default=uuid.uuid4,
        editable=False)
    name = models.CharField(max_length=250)
    price = models.DecimalField(max_digits=18, decimal_places=2)

class Orders(models.Model):
    id = models.UUIDField(
        primary_key=True,
        default=uuid.uuid4,
        editable=False)
    subtotal = models.DecimalField(max_digits=18, decimal_places=2)
    customer = models.ForeignKey(
        Customers, on_delete=models.CASCADE, null=True)
    product = models.ManyToManyField(Products)

In this file, we define some simple classes that map to the tables in the cluster.

Views

Next, build out some class-based views for the application in a file called views.py. You can copy the sample code below and paste it into a new file, or you can download the file directly.

from django.http import JsonResponse, HttpResponse
from django.utils.decorators import method_decorator
from django.views.generic import View
from django.views.decorators.csrf import csrf_exempt
from django.db import Error, IntegrityError
from django.db.transaction import atomic
from psycopg2 import errorcodes

from .models import *

# Warning: Do not use retry_on_exception in an inner nested transaction.

def retry_on_exception(num_retries=3, on_failure=HttpResponse(status=500), delay_=0.5, backoff_=1.5):
    def retry(view):
        def wrapper(*args, **kwargs):
            delay = delay_
            for i in range(num_retries):
                try:
                    return view(*args, **kwargs)
                except IntegrityError as ex:
                    if i == num_retries - 1:
                        return on_failure
                    elif getattr(ex.__cause__, 'pgcode', '') == errorcodes.SERIALIZATION_FAILURE:
                        time.sleep(delay)
                        delay *= backoff_
                except Error as ex:
                    return on_failure
        return wrapper
    return retry

class PingView(View):
    def get(self, request, *args, **kwargs):
        return HttpResponse("python/django", status=200)

@method_decorator(csrf_exempt, name='dispatch')
class CustomersView(View):
    def get(self, request, id=None, *args, **kwargs):
        if id is None:
            customers = list(Customers.objects.values())
        else:
            customers = list(Customers.objects.filter(id=id).values())
        return JsonResponse(customers, safe=False)

    @retry_on_exception(3)
    @atomic
    def post(self, request, *args, **kwargs):
        form_data = json.loads(request.body.decode())
        name = form_data['name']
        c = Customers(name=name)
        c.save()
        return HttpResponse(status=200)

    @retry_on_exception(3)
    @atomic
    def delete(self, request, id=None, *args, **kwargs):
        if id is None:
            return HttpResponse(status=404)
        Customers.objects.filter(id=id).delete()
        return HttpResponse(status=200)

    # The PUT method is shadowed by the POST method, so there doesn't seem
    # to be a reason to include it.

@method_decorator(csrf_exempt, name='dispatch')
class ProductView(View):
    def get(self, request, id=None, *args, **kwargs):
        if id is None:
            products = list(Products.objects.values())
        else:
            products = list(Products.objects.filter(id=id).values())
        return JsonResponse(products, safe=False)

    @retry_on_exception(3)
    @atomic
    def post(self, request, *args, **kwargs):
        form_data = json.loads(request.body.decode())
        name, price = form_data['name'], form_data['price']
        p = Products(name=name, price=price)
        p.save()
        return HttpResponse(status=200)

    # The REST API outlined in the github does not say that /product/ needs
    # a PUT and DELETE method

@method_decorator(csrf_exempt, name='dispatch')
class OrdersView(View):
    def get(self, request, id=None, *args, **kwargs):
        if id is None:
            orders = list(Orders.objects.values())
        else:
            orders = list(Orders.objects.filter(id=id).values())
        return JsonResponse(orders, safe=False)

    @retry_on_exception(3)
    @atomic
    def post(self, request, *args, **kwargs):
        form_data = json.loads(request.body.decode())
        c = Customers.objects.get(id=form_data['customer']['id'])
        o = Orders(subtotal=form_data['subtotal'], customer=c)
        o.save()
        for p in form_data['products']:
            p = Products.objects.get(id=p['id'])
            o.product.add(p)
        o.save()
        return HttpResponse(status=200)

This file defines the application’s views as classes. Each view class corresponds to one of the table classes defined in models.py. The methods of these classes define read and write transactions on the tables in the database.

Importantly, the file defines a in the decorator function retry_on_exception(). This function decorates each view method, ensuring that transaction ordering guarantees meet the ANSI SERIALIZABLE isolation level. For more information about how transactions (and retries) work, see .

URL routes

Lastly, define some URL routes in a file called urls.py. You can copy the sample code below and paste it into the existing urls.py file, or you can download the file directly and replace the existing one.

from django.contrib import admin
from django.urls import path

from .views import CustomersView, OrdersView, PingView, ProductView

urlpatterns = [
    path('admin/', admin.site.urls),

    path('ping/', PingView.as_view()),

    # Endpoints for customers URL.
    path('customer/', CustomersView.as_view(), name='customers'),
    path('customer/<uuid:id/', CustomersView.as_view(), name='customers'),

    # Endpoints for customers URL.
    path('product/', ProductView.as_view(), name='product'),
    path('product/<uuid:id/', ProductView.as_view(), name='product'),

    path('order/', OrdersView.as_view(), name='order'),
]

Step 5. Initialize the database

In the top cockroach_example directory, use the manage.py script to create Django migrations that initialize the database for the application:

$ python manage.py makemigrations cockroach_example

$ python manage.py migrate

This initializes the tables defined in models.py, in addition to some other tables for the admin functionality included with Django’s starter application.

Step 6. Run the app

1. In a different terminal, navigate to the top of the cockroach_example directory, and start the app:

   $ python manage.py runserver 0.0.0.0:8000
   

   The output should look like this:

   ...
   Starting development server at http://0.0.0.0:8000/
   Quit the server with CONTROL-C.
   

   To perform simple reads and writes to the database, you can send HTTP requests to the application server listening at http://0.0.0.0:8000/.

2. In a new terminal, use curl to send a POST request to the application:

   $ curl --header "Content-Type: application/json" \
   --request POST \
   --data '{"name":"Carl"}' http://0.0.0.0:8000/customer/
   

   This request inserts a new row into the cockroach_example_customers table.

3. Send a GET request to read from the cockroach_example_customers table:

   $ curl http://0.0.0.0:8000/customer/
   
   [{"id": "bb7d6c4d-efb3-45f8-b790-9911aae7d8b2", "name": "Carl"}]
   

   You can also query the table directly in the to see the changes:

   > SELECT * FROM cockroach_example_customers;
   
                      id                  | name
   ---------------------------------------+-------
     bb7d6c4d-efb3-45f8-b790-9911aae7d8b2 | Carl
   (1 row)
   

4. Enter Ctrl+C to stop the application.

What’s next?

Read more about writing a Django app.

You might also be interested in the following pages:

• Client Connection Parameters
• Connection Pooling
• Data Replication
• CockroachDB Resilience
• Replication & Rebalancing
• Cross-Cloud Migration
• Automated Operations