Running databases in Kubernetes can be a little bit delicate owing to the fact that data needs to be persisted incase the pods die, are restarted or are malfunctioning due to one reason or another. Due to this, most people still choose the virtual machine way to run their databases because among other things, it feels safe and you know exactly where the data is being stored/persisted. But in case you need all of your applications to reside in Kubernetes, you will be delighted to know that it can handle stateful applications such as databases. There are two ways in which you can achieve this. You can either choose to use Deployments or StatefulSets. Now the two have their pros and cons and their differences are showed in the table below:
| StatefulSet | Deployment |
| Their pod names remain the same regardless | Pods created from deployments have random names |
| If a volume is configured, each pod will be provision its own persistent volume | Pod replicas in deployments share the same persistent volume |
| Recommended for deploying Stateful applications e.g databases that need persistent data across replicas. | Recommended for deploying stateless applications for example load balancers, proxies etc. |
| A headless service is responsible for the network identity of the pods | A service has to be provisioned to interact with pods in a deployment |
| Every replica is named in sequential order, beginning from ordinal number zero. | Every replica is named randomly |
Due to the advantages that StatefulSets have compared to Deployments such as separate volumes for each of the pods created, let us go ahead and deploy Redis cluster using this kind. It is best practice to use headless services (service without an IP address) when using StatefulSets. This is because we would wish our clients to connect directly to the pods and good thing is that StatefulSets maintain the same DNS identity. So we shall create a headless service to go along with this Redis cluster.
Pre-requisites
You need to have the following to get this accomplished:
- A working Kubernetes Cluster
- kubectl installed locally or somewhere that is connected to your cluster
- Ability to access your cluster
- For this we will also require Dynamic Volume Provisioning in your cluster.
Once you are ready, we shall therefore set off.
Step 1: Create and deploy headless service
In this step, we shall simply create our headless service as follows.
$ vim redis-service.yml
apiVersion: v1
kind: Service
metadata:
name: redis-service
namespace: redis
labels:
app: redis
spec:
ports:
- port: 6379
clusterIP: None
selector:
app: redis
As you can notice, the service has its clusterIP set to “None” and hence will prevent the service from having an IP giving clients the ability to connect directly to what lies behind it.
With the clean service created, proceed to deploy it as shown below. We will create the “redis” namespace then apply the file. Remember that you can choose any namespace of your choice here.
$ kubectl create ns redis
$ kubectl apply -f redis-service.yml
service/redis-ss createdStep 2: Create and deploy the configuration via a ConfigMap
We shall use a ConfigMap to add configuration parameters dynamically to our Redis Master and Replicas. Create a ConfigMap as follows:
$ vim redis-configmap.yml
apiVersion: v1
kind: ConfigMap
metadata:
name: redis-ss-configuration
namespace: redis
labels:
app: redis
data:
master.conf: |
maxmemory 400mb
maxmemory-policy allkeys-lru
maxclients 20000
timeout 300
appendonly no
dbfilename dump.rdb
dir /data
slave.conf: |
slaveof redis-ss-0.redis-ss.redis 6379
maxmemory 400mb
maxmemory-policy allkeys-lru
maxclients 20000
timeout 300
dir /data
The data section of the ConfigMap has the two configurations for master and replica which we shall direct them to the right pods depending on the StatefulSets’ ordinal numbers. For your information, StatefulSets assign a sticky identity called an ordinal number starting from zero to each Pod instead of assigning random IDs for each replica Pod. This is as was clarified on the table in the introductory section. You can add more of your configurations on this part as needed. Once you are ready/comfortable with the configs, let’s go ahead and deploy it right away.
$ kubectl apply -f redis-configmap.yml
configmap/redis-configuration-ss createdStep 3: Create and deploy the StatefulSet
We are now in the interesting part of this meal. All of the other parts are ready and we will just plug in the engine and we will be ready to hit the road. Create a new file and fill it with the following StatefulSet configuration then we explain what it does.
vim redis-statefulset.yml
Paste and modify the contents below where applicable.
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: redis-ss
namespace: redis
spec:
serviceName: "redis-service"
replicas: 1
selector:
matchLabels:
app: redis
template:
metadata:
labels:
app: redis
spec:
initContainers:
- name: init-redis
image: redis:latest
command:
- bash
- "-c"
- |
set -ex
# Generate redis server-id from pod ordinal index.
[[ `hostname` =~ -([0-9]+)$ ]] || exit 1
ordinal=${BASH_REMATCH[1]}
# Copy appropriate redis config files from config-map to respective directories.
if [[ $ordinal -eq 0 ]]; then
cp /mnt/master.conf /etc/redis-config.conf
else
cp /mnt/slave.conf /etc/redis-config.conf
fi
volumeMounts:
- name: redis-claim
mountPath: /etc
- name: config-map
mountPath: /mnt/
containers:
- name: redis
image: redis:latest
ports:
- containerPort: 6379
name: redis-ss
command:
- redis-server
- "/etc/redis-config.conf"
volumeMounts:
- name: redis-data
mountPath: /data
- name: redis-claim
mountPath: /etc
volumes:
- name: config-map
configMap:
name: redis-ss-configuration
volumeClaimTemplates:
- metadata:
name: redis-claim
spec:
accessModes: [ "ReadWriteOnce" ]
resources:
requests:
storage: 1Gi
- metadata:
name: redis-data
spec:
accessModes: [ "ReadWriteOnce" ]
resources:
requests:
storage: 1Gi
When using StorageClass for PV creation update volumeClaimTemplates section like below.
volumeClaimTemplates:
- metadata:
name: redis-claim
spec:
accessModes: [ "ReadWriteOnce" ]
resources:
requests:
storage: 1Gi
storageClassName: rook-cephfs
- metadata:
name: redis-data
spec:
accessModes: [ "ReadWriteOnce" ]
resources:
requests:
storage: 1Gi
storageClassName: rook-cephfsYou can list configured Storage Classes in your cluster with the command below:
$ kubectl get sc
NAME PROVISIONER RECLAIMPOLICY VOLUMEBINDINGMODE ALLOWVOLUMEEXPANSION AGE
rook-ceph-block rook-ceph.rbd.csi.ceph.com Delete Immediate true 204d
rook-cephfs rook-ceph.cephfs.csi.ceph.com Delete Immediate true 204dIn our Statefulset configuration, the Ordinal Number will be 0 for the first pod here and it will be named “redis-ss-0” because the name of the StatefulSet is “redis-ss”
All the way from top, we have declared that we are creating a StatefulSet and we are picking the headless service called “redis-service” that we created in Step 1 as the main entry point. We will be using an initContainer to plug the configuration files to the pods. Init containers are run before the app containers are started and once it completes its task, it goes off and hands over the mantle to the main pod. In this case, the init container is instructed to check the ordinal number of the pod. Remember the “ordinal number” we explained before, here is one of the instances it becomes useful. So we are saying that, in case the ordinal number is zero i.e “redis-ss-0” (notice the tailing zero), pick the “master.conf” configuration from the ConfigMap (now available at /mnt in the initcontainer) and copy it to “/etc/redis-config.conf” file in the main pod. This configuration will have the redis master configs. In case it has any other ordinal number like 1 from the tailing number like “redis-ss-1” then pick the “slave.conf” (now also available at /mnt in the initcontainer) file from the ConfigMap and copy it to “/etc/redis-config.conf” of that new Pod. Pretty Cool!!
You may now be wondering and asking how the ConfigMap config is found at “/mnt” path. As you can see from the VolumeMounts section, we have the name “config-map” representing a mount point at the “/mnt” path. When we check the “volumes” section of the file as well, we see that there is a volume also known as “config-map” that is attached to the ConfigMap called “redis-ss-configuration” we created in Step 2. Basically, the contents of the ConfigMap (master.conf and slave.conf) will be dropped in the volume called “config-map” which will be later mounted at the path “/mnt” and its contents accessed. That is how we get the data/files (master.conf and slave.conf) into the initContainer. Another thing you will notice is that the initContainer and the main pod share one of the volumes and mounted on the same path. This is the “redis-claim” volume. This is correct so that the files transferred from the initContainer can be found by the main pod when it starts and mounts the volume into its directory hierarchy which it “/etc” in this case. Depending on the ordinal number of the pod, the initContainer will copy either master.conf or slave.conf into “/etc/redis-config.conf”.
Once that is done, there is a redis command that will start redis using the “/etc/redis-config.conf” file copied from the initContainer. The command here is:
redis-server /etc/redis-config.confLater in the file, we see the “volumeClaimTemplates” which is a way the StatefulSet will create persistent volumes dynamically. This is why one of the requirements for this deployment is the ability of your cluster to create persistent volumes dynamically. In case this is not possible, you will have to create persistent volumes with the same names as the ones in “volumeClaimTemplates” then apply this StatefulSet. The persistent volumes here will be used by the main pod to store its data at “/data” as well as the configurations at “/etc”. We hope the explanation was clear enough. Finally, apply the file like we have done with the others before:
$ kubectl apply -f redis-statefulset.yml
statefulset.apps/redis-ss createdStep 4: Checking if all we have done is working
By now, should be good to go. And the only way to know is to simply check if everything was successfully deployed:
Check the ConfigMap
$ kubectl get configmap -n redis
NAME DATA AGE
redis-ss-configuration 2 3h35mCheck the Service
$ kubectl get svc -n redis
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
redis-service ClusterIP None <none> 6379/TCP 3h40mCheck the StatefulSet
$ kubectl get statefulset -n redis
NAME READY AGE
redis-ss 1/1 3h43mCheck the Pods created from the StatefulSet
$ kubectl get pods -n redis
NAME READY STATUS RESTARTS AGE
redis-ss-0 1/1 Running 0 3h42mCheck the Persistent Volumes
$ kubectl get pvc -n redis
redis-claim-redis-ss-0 Bound pvc-8a30a867-8b3e-49c2-ac1d-be05d8fe3255 1Gi RWO standard 3h40m
redis-data-redis-ss-0 Bound pvc-04cd967c-93ec-457e-9cef-582aabae496a 1Gi RWO standard 3h40mLet us now exec into the pod and check that the configuration we want are the ones we get.
$ kubectl exec -it redis-ss-0 -c redis -n redis -- /bin/bash
bash-5.1#Once inside, enter into redis cli interface:
bash-5.1# redis-cliFind out if the timeout matches the one on the config file
127.0.0.1:6379> config get timeout
1) "timeout"
2) "300"Find out if the maxclients matches the one on the config file
127.0.0.1:6379> config get maxclients
1) "maxclients"
2) "20000"Cool! It seems like we are out of the woods 😄.
Step 5: See if replication works
Now that the master is functioning well, let us increase the replicas and see if the master replicates to the replica pod. Navigate to the StatefulSet file we created and edit the replicas to 2 from 1 as follows then apply the file.
$ vim redis-statefulset.yml
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: redis-ss
namespace: redis
spec:
serviceName: "redis-service”
replicas: 2 ##<<<<< Change to 2
selector:
matchLabels:
app: redis
template:
metadata:
labels:
app: redis
spec:
initContainers:
- name: init-redis
image: redis:latest
command:
- bash
- "-c"
- |
set -ex
# Generate redis server-id from pod ordinal index.
[[ `hostname` =~ -([0-9]+)$ ]] || exit 1
ordinal=${BASH_REMATCH[1]}
# Copy appropriate redis config files from config-map to respective directories.
if [[ $ordinal -eq 0 ]]; then
cp /mnt/master.conf /etc/redis-config.conf
else
cp /mnt/slave.conf /etc/redis-config.conf
fi
volumeMounts:
- name: redis-claim
mountPath: /etc
- name: config-map
mountPath: /mnt/
containers:
- name: redis
image: redis:latest
ports:
- containerPort: 6379
name: redis-ss
command:
- redis-server
- "/etc/master.conf"
volumeMounts:
- name: redis-data
mountPath: /data
- name: redis-claim
mountPath: /etc
volumes:
- name: config-map
configMap:
name: redis-ss-configuration
volumeClaimTemplates:
- metadata:
name: redis-claim
spec:
accessModes: [ "ReadWriteOnce" ]
resources:
requests:
storage: 1Gi
- metadata:
name: redis-data
spec:
accessModes: [ "ReadWriteOnce" ]
resources:
requests:
storage: 1GiApply the file
$ kubectl apply -f redis-statefulset.yml
statefulset.apps/redis-ss configuredCheck the pods to verify if a new one has been added
$ kubectl get pods -n redis
NAME READY STATUS RESTARTS AGE
redis-ss-0 1/1 Running 0 3h42m
redis-ss-1 1/1 Running 0 131mCheck the persistent volumes to verify if a new ones (for both templates) have been added for the new pod
$ kubectl get pvc -n redis
redis-claim-redis-ss-0 Bound pvc-8a30a867-8b3e-49c2-ac1d-be05d8fe3255 1Gi RWO standard 3h40m
redis-claim-redis-ss-1 Bound pvc-6574b160-03a8-48f5-a989-d780d17363fa 1Gi RWO standard 3h51m
redis-data-redis-ss-0 Bound pvc-04cd967c-93ec-457e-9cef-582aabae496a 1Gi RWO standard 3h40m
redis-data-redis-ss-1 Bound pvc-6eadf86f-36b4-4985-9b78-1c7faa402cbb 1Gi RWO standard 3h51mOh yeah!! There they are. Let us now log into the replica and check replication status information
kubectl exec -it redis-ss-1 -c redis -n redis -- /bin/bashOnce inside, run the info command to see if the replica is connected to the master successfully.
bash-5.1# redis-cli
127.0.0.1:6379> info replication
# Replication
role:slave
master_host:redis-ss-0.redis-ss.redis
master_port:6379
master_link_status:up
master_last_io_seconds_ago:5
master_sync_in_progress:0
slave_read_repl_offset:18970
slave_repl_offset:18970
slave_priority:100
slave_read_only:1
replica_announced:1
connected_slaves:0
master_failover_state:no-failover
master_replid:2cfc7fdcdf3322c981250f854235ff0b92b39dfa
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:18970
second_repl_offset:-1
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:7519
repl_backlog_histlen:11452
127.0.0.1:6379>And there we have it. One last thing to note is that clients can connect to the redis instances using their DNS Names. For example, to reach the instances deployed in this guide, the DNS names would be:
- Master DNS Name: redis-ss-0.redis-service.redis or redis-service-1.redis-ss.redis.svc.cluster.local
The format is:
[pod-name].[service-name].[namespace].svc.cluster.local or [pod-name].[service-name].[namespace]Salutations
It has been fun so far and we hope you did enjoy as much as we did. We would like to continue thanking you for the tremendous support, your readership and the positive comments we continue to receive. That said, let me wish you a wonderful time and a period of success, luck, health and happiness.
Reference:
Books For Learning Kubernetes Administration:
Other guides of similar content include:
