So far we've been using Docker and Docker Compose for running components of the Serenity trading system, but at a certain point we will need the power of Kubernetes to orchestrate and provision resources for the trading system.
The Kubernetes in Docker project, or kind, is a more featureful alternative to minikube for running your own local Kubernetes cluster. It uses its own Docker image for a Kubernetes installation which in turn runs Docker images, giving you a full-blown Kubernetes on your local PC. This article will cover getting up and running with Kubernetes initially for TimescaleDB, but ultimately for the entire Serenity trading system.
Note all instructions here are Debian-based because my main server is based on Debian "stretch," but kind works on other Linux distributions, MacOS and Windows as well.
Installing go 1.11
Go 1.11 is available in stretch-backports, so assuming you have that configured install is simple:
apt-get install go-1.11
Installing kind
Per the kind Quick Start guide, the fastest way to install on Debian is with Go:
GO111MODULE="on" go get sigs.k8s.io/kind@v0.7.0
$HOME/go/bin/kind create cluster
Installing kubectl
The detailed instructions for installing kubectl are here but as a quick reference, the Debian install requires installing an additional repository and GPG keys as follows:
sudo apt-get update && sudo apt-get install -y apt-transport-https
curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add -
echo "deb https://apt.kubernetes.io/ kubernetes-xenial main" | sudo tee -a /etc/apt/sources.list.d/kubernetes.list
sudo apt-get update
sudo apt-get install -y kubectl
First Pod: running TimescaleDB in Kubernetes
Now that we have a kind environment we can deploy our first Docker image. Picking up from the last article, we'll use the timescale/timescaledb:1.5.1-pg11-oss image and deploy TimescaleDB, an extension to Postgres for supporting timeseries databases.
First, as a good practice, we'll store our Postgres password as a secret after first base64-encoding it to prevent issues in the YAML file (obviously pick something better than mypassword!):
$ echo mypassword | base64
bXlwYXNzd29yZAo=
And then define the secret for Kubernetes:
apiVersion: v1
kind: Secret
metadata:
name: database-secret-config
type: Opaque
data:
password: bXlwYXNzd29yZAo=
Apply it:
$ kubectl apply -f secret.yaml
secret/database-secret-config created
Next step is to create a persistent storage mapping and a claim on it, which is like a pointer to the storage which we can reference later:
kind: PersistentVolume
apiVersion: v1
metadata:
name: timescaledb-pv-volume
labels:
type: local
app: timescaledb
spec:
storageClassName: manual
capacity:
storage: 50Gi
accessModes:
- ReadWriteMany
hostPath:
path: "/mnt/raid/data/timescaledb"
---
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
name: timescaledb-pv-claim
labels:
app: timescaledb
spec:
storageClassName: manual
accessModes:
- ReadWriteMany
resources:
requests:
storage: 50Gi
Apply it:
$ kubectl apply -f timescaledb-storage.yaml
persistentvolume/timescaledb-pv-volume created
persistentvolumeclaim/timescaledb-pv-claim created
Finally you can create the database deployment itself pointing to the secret and the storage:
apiVersion: apps/v1
kind: Deployment
metadata:
name: timescaledb
labels:
app: timescaledb
spec:
replicas: 1
selector:
matchLabels:
app: timescaledb
template:
metadata:
labels:
app: timescaledb
spec:
containers:
- name: timescaledb
image: timescale/timescaledb:1.5.1-pg11-oss
ports:
- containerPort: 5432
env:
- name: POSTGRES_PASSWORD
valueFrom:
secretKeyRef:
name: database-secret-config
key: password
volumeMounts:
- mountPath: /var/lib/postgresql/data
name: timescaledb
volumes:
- name: timescaledb
persistentVolumeClaim:
claimName: timescaledb-pv-claim
again, applying it from file:
$ kubectl apply -f timescaledb.yaml
deployment.apps/timescaledb created
and then getting all Kubernetes objects shows we have a Pod, a Service, a Deployment and a ReplicaSet all created:
$ kubectl get all
NAME READY STATUS RESTARTS AGE
pod/timescaledb-66cffc7d4d-2ppt8 1/1 Running 0 49s
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 3d4h
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/timescaledb 1/1 1 1 49s
NAME DESIRED CURRENT READY AGE
replicaset.apps/timescaledb-66cffc7d4d 1 1 1 49s
Note: kind does not support the LoadBalancer service type at this time, so you cannot expose TimescaleDB to the outside world in this way. Instead you can shell into the pod:
$ kubectl get pods
NAME READY STATUS RESTARTS AGE
timescaledb-66cffc7d4d-2ppt8 1/1 Running 0 41m
$ kubectl exec -it pod/timescaledb-66cffc7d4d-2ppt8 -- /bin/bash
bash-5.0# psql -U postgres --password
Password:
psql (11.5)
Type "help" for help.
postgres=#
or alternatively, we can port forward to the pod:
$ kubectl port-forward pod/timescaledb-66cffc7d4d-2ppt8 5432:5432
Forwarding from 127.0.0.1:5432 -> 5432
Forwarding from [::1]:5432 -> 5432
and connect locally -- note we're using an 11.6 client now to talk to our 11.5 server:
$ psql -Upostgres -p 5432 -h localhost
psql (11.6 (Debian 11.6-1.pgdg90+1), server 11.5)
Type "help" for help.
postgres=#
Success! We have a running TimescaleDB in Kubernetes on our local machine. But there's a problem: take a look at our local filesystem mount path: /mnt/raid/data/timescaledb: it's empty. Further digging shows that this path exists in a Docker container called kind-control-plane. What's going on here?
The issue is kind is running Docker inside Docker: we've only mapped one level of indirection with our persistent volume: it never made it "out" to the real-world level. We need to create a custom configuration for our kind cluster:
kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
nodes:
- role: control-plane
extraMounts:
- hostPath: /mnt/raid/data/timescaledb
containerPath: /mnt/raid/data/timescaledb
and re-create our cluster:
$ kind delete cluster
$ kind create cluster --config=kind-cluster.yaml
$ kubectl apply -f database-secret-config.yaml
$ kubectl apply -f timescaledb.yaml
$ kubectl port-forward timescaledb-7f55d47c8-pspdm 5432:5432
Now let's check again. This time we have files!
$ sudo ls -l /mnt/raid/data/timescaledb/
total 120
drwx------ 5 70 70 4096 Feb 1 15:49 base
drwx------ 2 70 70 4096 Feb 1 15:50 global
drwx------ 2 70 70 4096 Feb 1 15:49 pg_commit_ts
drwx------ 2 70 70 4096 Feb 1 15:49 pg_dynshmem
-rw------- 1 70 70 4535 Feb 1 15:49 pg_hba.conf
-rw------- 1 70 70 1636 Feb 1 15:49 pg_ident.conf
drwx------ 4 70 70 4096 Feb 1 15:49 pg_logical
drwx------ 4 70 70 4096 Feb 1 15:49 pg_multixact
drwx------ 2 70 70 4096 Feb 1 15:49 pg_notify
drwx------ 2 70 70 4096 Feb 1 15:49 pg_replslot
drwx------ 2 70 70 4096 Feb 1 15:49 pg_serial
drwx------ 2 70 70 4096 Feb 1 15:49 pg_snapshots
drwx------ 2 70 70 4096 Feb 1 15:49 pg_stat
drwx------ 2 70 70 4096 Feb 1 15:50 pg_stat_tmp
drwx------ 2 70 70 4096 Feb 1 15:49 pg_subtrans
drwx------ 2 70 70 4096 Feb 1 15:49 pg_tblspc
drwx------ 2 70 70 4096 Feb 1 15:49 pg_twophase
-rw------- 1 70 70 3 Feb 1 15:49 PG_VERSION
drwx------ 3 70 70 4096 Feb 1 15:49 pg_wal
drwx------ 2 70 70 4096 Feb 1 15:49 pg_xact
-rw------- 1 70 70 88 Feb 1 15:49 postgresql.auto.conf
-rw------- 1 70 70 24014 Feb 1 15:49 postgresql.conf
-rw------- 1 70 70 24 Feb 1 15:49 postmaster.opts
-rw------- 1 70 70 94 Feb 1 15:49 postmaster.pid
