How to Setup CockroachDB Cluster on Linux

CockroachDB is an open-source project which is designed to be scalable and resilient to failure. Best features of RDBMS and NoSQL are combined together in this product. You can start using CockroachDB from a single node and add more nodes as your system growth in a simple way. I know that now you want to ask how to integrate this database with your application. CockroachDB is based on PostgreSQL so you can use any client drivers or supported ORMs.

In this article, I want to show you how to setup 3 nodes of CockroachDB and show how can you monitor your system.

For this purpose, I am going use to three CentOS systems with 512 MB of RAM and 1 vCPU for each.
Before we start installing CockroachDB we have to make some changes to our systems.
First of all, ensure that ports 8080 (for web UI) and 26257 (communication port) are opened.

If you use UFW:

$ sudo ufw allow 8080/tcp

$ sudo ufw allow 26257/tcp

For firewalld:

# firewall-cmd --permanent --zone=public –-add-port=8080/tcp

# firewall-cmd --permanent --zone=public –-add-port=26257/tcp

# firewall-cmd –-reload

Note: If you are going to use it in production you will need to setup NTP server and use non-root user with superuser privileges.

Now we can start the installation. For all of the hosts we have to download the latest version of CockroachDB, unzip and move binaries to /usr/local/bin.

$ wget https://binaries.cockroachdb.com/cockroach-latest.linux-amd64.tgz?s=do

tar -xf cockroach-latest.linux-amd64.tgz?s=do --strip=1 cockroach-latest.linux-amd64/cockroach

# mv cockroach /usr/local/bin

Please note that commands are executed as root or superuser (#) and as user ($). Working as root all of the time is a bad form.

Now we can check the version

$ cockroach version

and get something like:

Build Tag: v1.0.4
Build Time: 2017/09/10 10:33:34
Distribution: CCL
Platform: linux amd64
Go Version: go1.8.3
C Compiler: gcc 6.3.0
Build SHA-1: 5b757262d33d814bda1deb2af20161a1f7749df3
Build Type: release

Seems it is working and we can start our cluster. On the first node execute:

$ cockroach start –-insecure –-background –-advertise-host=192.168.1.6

A bit about flags:

• --insecure flag will run node without SSL. Otherwise, you have to use flag --certs-dir that points to valid certificates.
• --background flag is used to run in background mode and release command line.
• --advertise-host flag is an IP address or hostname of a node.

Now we need to connect all other nodes to this host. On node two execute

$ cockroach start –-insecure –-background --advertise-host=192.168.1.80 --join=192.168.1.6:26257

And on node three execute:

$ cockroach start –-insecure –-background --advertise-host=192.168.1.147 --join=192.168.1.6:26257

Note: Please ensure you have changed --advertise-host and --join IPs

Open your browser and go to 'http://nodeURLorHOSTNAME:8080'. In my case, it is 'http://192.168.1.6:8080'

Click on 'View nodes list' in 'Summary' tab. You will get all of the systems connected together that share all of the database data.

That's all. Now you can use your cluster for keeping data that is replicated in all nodes. If any node goes offline by any reasons after reconnect it will get all the changes that were made while it had been offline.

To remove node from a cluster just execute:

$ cockroach quit

For testing purposes, you can connect to any node via ssh and execute

$ cockroach sql

This is a SQL shell where you can execute any SQL commands. After executing you should see changes in statistics and graphs. Before executing I recommend to read about CockroachDB SQL dialect on their official site. To add more nodes you can use steps for nodes two and three and easily scale your system horizontally.