Install and Use Trow Container Image Registry With Kubernetes

In this era, we can all agree that containerization is almost everywhere. It allows many people and organizations around the world to run applications in a lightweight executable known as a container. The containers contain all the required dependencies and packages required to run an application. This comes with several benefits such as portability and agility for developers. There are several tools that can be used to run the containers. The most popular ones are docker, Podman, Openshift, Kubernetes etc.

Container images can be created using tools like  Buildah, kaniko, Img Image builder etc. Once created, the images can either be saved on your local machine or stored elsewhere so that they can be accessed, shared and used later when required.

A container registry can be defined as a repository or a collection of repositories that provides storage for your container images. This serves as an intermediary when sharing the images across your organization/systems. Using container registries, you are able to access the images by uploading (pushing) and downloading them(pulling) to a remote system.

There are two main container registry types:

• Public registries: used by individuals and small teams that want to spin and run registries as fast as possible. These are less secure and might cause problems such as patching, privacy, and access control for large organizations.
• Private registries: They provide advanced security features, providing the best way to incorporate security and privacy into enterprise container image storage. They can be hosted remotely or o-premise. The most popular Private Registries are Google Container Registry, Amazon Elastic Container Registry (ECR), and the Azure Container Registry

What is Trow Container Image Registry?

Do you know that there is an image management solution for Kubernetes and other orchestrators which runs inside the cluster, is simple to set up and fully integrated with Kubernetes, including support for auditing and RBAC? Yes, there is!

“Trow” is a word with several, diverse meanings. In Shetland folklore, it can be defined as a small, mischievous creature, similar to the Scandanavian troll. In England, it is an old-style of cargo boat that transported goods on rivers. But its origin is archaic to mean “to think, believe, or trust”. But you have the freedom to choose the interpretation you like.

Trow provides a secure and fast registry used to get containers running on the cluster. Its main focus is to give control to system admins so that they can define which images can run in the cluster. It prevents unauthorised and potentially insecure or malicious images from being deployed into your cluster.

The main features provided by Trow are:

• Distributed architecture for HA and scalability (planned)
• Full auditing and authentication of image access (in progress)
• Controls images running inside the cluster via approve/deny lists
• It conforms to the OCI Distribution Specification for registries

In this tutorial, we will learn how we can install and use Trow Container Image Registry With Kubernetes.

1. Set up a Kubernetes Cluster

This guide requires one to have a Kubernetes Cluster up and running. There are several ways you can use to create a Kubernetes cluster. This website provides the following options:

• Deploy HA Kubernetes Cluster on Rocky Linux 8 using RKE2
• Run Kubernetes on Debian with Minikube
• Deploy Kubernetes Cluster on Linux With k0s
• Install Kubernetes Cluster on Ubuntu using K3s
• Install Kubernetes Cluster on Rocky Linux 8 with Kubeadm & CRI-O
• Deploy k0s Kubernetes on Rocky Linux 9 using k0sctl
• Install Minikube Rocky Linux 9 and Create Kubernetes Cluster

After spinning the cluster, ensure that you have kubectl set up. You can use the below command to install it:

curl -LO "https://storage.googleapis.com/kubernetes-release/release/$(curl -s https://storage.googleapis.com/kubernetes-release/release/stable.txt)/bin/linux/amd64/kubectl"
chmod +x kubectl
sudo mv kubectl /usr/local/bin

Now to have access to the cluster, export the admin config:

##For RKE2
export PATH=$PATH:/var/lib/rancher/rke2/bin export KUBECONFIG=/etc/rancher/rke2/rke2.yaml

##For K0s
export KUBECONFIG=/var/lib/k0s/pki/admin.conf

Verify if kubectl is working:

$ kubectl get nodes
NAME      STATUS   ROLES           AGE     VERSION
master    Ready    control-plane   2m12s   v1.25.4+k0s
worker1   Ready    <none>          97s     v1.25.4+k0s
worker2   Ready    <none>          79s     v1.25.4+k0s

2. Create a Persistent Storage for Trow

The Trow Container Image Registry requires a persistent storage to store files and configurations. We will begin by creating a dedicated namespace for the installation:

kubectl create namespace trow

Set it as the default namespace:

kubectl config set-context --current --namespace trow

Manually defining Storage Class (ignore if you have already)

Once the namespace has been created, we will now create a storage class.

kubectl apply -f - <<EOF
kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
  name: trow-sc
provisioner: kubernetes.io/no-provisioner
volumeBindingMode: WaitForFirstConsumer
EOF

Now set it as a default storage class:

kubectl patch storageclass trow-sc -p '{"metadata": {"annotations":{"storageclass.kubernetes.io/is-default-class":"true"}}}'

Verify this:

$ kubectl get sc
NAME                PROVISIONER                    RECLAIMPOLICY   VOLUMEBINDINGMODE      ALLOWVOLUMEEXPANSION   AGE
trow-sc (default)   kubernetes.io/no-provisioner   Delete          WaitForFirstConsumer   false                  11s

Now create a persistent volume on this storage class:

$ vim trow-pv.yml
apiVersion: v1
kind: PersistentVolume
metadata:
  name: trow-pv
spec:
  capacity:
    storage: 20Gi
  accessModes:
  - ReadWriteOnce
  persistentVolumeReclaimPolicy: Retain
  storageClassName: trow-sc
  local:
    path: /mnt/disk/vol1
  nodeAffinity:
    required:
      nodeSelectorTerms:
      - matchExpressions:
        - key: kubernetes.io/hostname
          operator: In
          values:
          - worker1

Before we create the PV, you need to create the path on the specified node. For this case, it is worker1

sudo mkdir -p /mnt/disk/vol1
sudo chmod 777 /mnt/disk/vol1

##Also run this On Rhel-based systems#####
sudo chcon -Rt svirt_sandbox_file_t /mnt/disk/vol1

Using default existing Storage Class

To use existing storage class name configure like below.

$ vim trow-pv.yml
apiVersion: v1
kind: PersistentVolume
metadata:
  name: trow-pv
spec:
  capacity:
    storage: 20Gi
  accessModes:
  - ReadWriteOnce
  persistentVolumeReclaimPolicy: Retain

Now apply the manifest file:

kubectl create -f trow-pv.yml

3. Install Trow Container Image Registry on Kubernetes

For this guide, we will install the Trow Container Image Registry on Kubernetes using Helm. Therefore, you need to have Helm installed on your system. The easiest way to install Helm is captured in the guide below:

• Install and Use Helm 3 on Kubernetes Cluster

Once Helm has been installed, add the helm chart for trow:

helm repo add trow https://trow.io

Now you can install Trow using the added Helm Chart. During the installation, you can also use several other variables that include:

It is possible to use a few variables by declaring them in a YAML file. Below are demo variables for the ingress service.

$ vim values.yaml
ingress:
    enabled: true
    tls: 
    - hosts:
      - trow.geeksforgeeks.org
      secretName: myingress-cert

In this guide, we will not use the values since we will deploy an ingress controller separately. Begin by creating and Installing the ingress controller.

a. Install and Configure an Ingress Controller

We can install Traefik ingress from Helm charts using the aid in the guide below:

• Install and Configure Traefik Ingress Controller on k0s

Now ensure that the obtained Loadbalancer IP for the Ingress service is mapped to a domain name:

$ sudo vim /etc/hosts
192.168.205.40 trow.geeksforgeeks.org 

We will now generate TLS certs for the Loadbalancer IP. Now create your certificate secret. In this guide, I will demonstrate how to create this using Self-signed certificates. Create a simple config file for this:

$ cd ~/ && vim csr.conf
[ req ]
default_bits = 2048
prompt = no
default_md = sha256
req_extensions = req_ext
distinguished_name = dn

[ dn ]
C = KE
ST = Nairobi
L = Nairobi
O = Computingforgeeks
OU = tech
CN = trow.geeksforgeeks.org

[ req_ext ]
subjectAltName = @alt_names

[ alt_names ]
DNS.1 = kubernetes
DNS.2 = kubernetes.default
DNS.3 = kubernetes.default.svc
DNS.4 = kubernetes.default.svc.cluster
DNS.5 = kubernetes.default.svc.cluster.local
DNS.6 = trow.geeksforgeeks.org
IP.1 = <LoadBalancer_IP>

[ v3_ext ]
authorityKeyIdentifier=keyid,issuer:always
basicConstraints=CA:FALSE
keyUsage=keyEncipherment,dataEncipherment
extendedKeyUsage=serverAuth,clientAuth
subjectAltName=@alt_names

Generate a ca.key and ca.crt

openssl genrsa -out ca.key 2048
openssl req -x509 -new -nodes -key ca.key -subj "/CN=trow.geeksforgeeks.org" -days 10000 -out ca.crt

Create s server key:

openssl genrsa -out server.key 2048

Now we will create a Certificate Signing Request using the config:

openssl req -new -key server.key -out server.csr -config csr.conf

Now generate a server certificate using the ca.key, ca.crt and server.csr:

openssl x509 -req -in server.csr -CA ca.crt -CAkey ca.key \
    -CAcreateserial -out server.crt -days 10000 \
    -extensions v3_ext -extfile csr.conf -sha256

Now we will create Kubernetes secrets:

kubectl create secret generic myingress-cert --from-file=tls.crt=./server.crt --from-file=tls.key=./server.key --from-file=ca.crt=./ca.crt

Verify the secret creation:

$ kubectl get secrets
NAME                            TYPE                 DATA   AGE
myingress-cert                  Opaque               3      13s

b. Run Trow Container Image Registry

Install Trow using the variable and any other desired flags. For example here, set our desired domain name.

helm install \
    trow \
    trow/trow \
--set trow.domain=trow.geeksforgeeks.org

Sample output:

NAME: trow
LAST DEPLOYED: Fri Dec 30 15:35:25 2022
NAMESPACE: trow
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
Welcome to Trow!

Please set the DNS entry for your registry to point to Trow

If you are running Trow on http, there will need to be a TLS ingress in front of it as Trow needs to be run over TLS.
Please see the examples found on the docs at
https://github.com/ContainerSolutions/trow/blob/main/docs/HELM_INSTALL.md

To test you can run and login with the credentials for 

$ docker login https://trow.geeksforgeeks.org


Please contribute to the Trow project at https://github.com/ContainerSolutions/trow/

In case you want to use the YAML, will have your command as:

helm install \
    -f values.yaml  \
    trow \
    trow/trow \
--set trow.domain=trow.geeksforgeeks.org

Check if the pod is running:

# kubectl get pods
NAME                       READY   STATUS    RESTARTS   AGE
traefik-865f54fc64-66f5n   1/1     Running   0          25m
trow-0                     1/1     Running   0          2m35s

After this, we need to configure an ingress controller for HTTPS.

$ vim trow-ingress.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: trow-https
  annotations:
    traefik.ingress.kubernetes.io/router.entrypoints: websecure
spec:
  rules:
    - host: trow.geeksforgeeks.org
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: trow
                port:
                  number: 8000
  tls:
  - hosts:
    - trow.geeksforgeeks.org
    secretName: myingress-cert

Apply the ingress:

kubectl apply -f trow-ingress.yaml

Verify the creation:

NAME         CLASS     HOSTS                        ADDRESS   PORTS     AGE
trow-https   traefik   trow.geeksforgeeks.org             80, 443   16s

Now we have trow.geeksforgeeks.org ready to handle HTTPS connections for Trow. To verify this use:

$ curl -k https:///trow.geeksforgeeks.org
<!DOCTYPE html><html><body>
<h1>Welcome to Trow, the cluster registry</h1>
</body></html>root

4. Test Trow Container Image Registry

We can now test if Trow Container Image Registry is working as desired. By default docker only uses TLS to pull and push images, we, therefore, have one problem, we have used self-signed certs. You can set docker to use insecure Registries or trust the certs.

First, ensure that docker is installed on your system.

• How To Install Docker CE on Linux Systems

• Option 1: Allow docker to use insecure registries.

$ sudo vim /etc/docker/daemon.json
{
"insecure-registries" : ["trow.geeksforgeeks.org"]
}

Restart the service:

sudo systemctl restart docker

• Option 2: Set docker to trust the certs

For this option, copy the ca.cert for the domain name/IP Address to the docker certs directory

sudo mkdir -p /etc/docker/certs.d/trow.geeksforgeeks.org
sudo cp ~/ca.crt /etc/docker/certs.d/trow.geeksforgeeks.org/ca.crt

Now verify if everything is working:

$ docker login trow.geeksforgeeks.org
Username: user
Password: password
WARNING! Your password will be stored unencrypted in /root/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store

Login Succeeded

You can also use Podman, with the syntax:

##For Podman with HTTPS
podman login <trow_address>

#For Podman with HTTP
podman login <trow_address> --tls-verify=false

Use the default login creds if you have not configured any, to access your Trow Container Image Registry. Once logged in as above, we can now try to push images to the registry.

Create a sample docker image file:

$ vim Dockerfile
FROM ubuntu:20.04
RUN apt-get update -y
ENV DEBIAN_FRONTEND=noninteractive 
RUN apt-get install -y gnupg apt-transport-https apt-utils wget
RUN echo "deb https://notesalexp.org/tesseract-ocr5/focal/ focal main" \
|tee /etc/apt/sources.list.d/notesalexp.list > /dev/null
RUN wget -O - https://notesalexp.org/debian/alexp_key.asc | apt-key add -
RUN apt-get update -y
RUN apt-get install tesseract-ocr -y
RUN apt install imagemagick -y
ENTRYPOINT ["tesseract"]
RUN tesseract -v

Build the image:

##For Docker
docker build . -t tesseract:5

##For Podman
podman build . -t tesseract:5

We will add a tag to this built image. For example:

docker tag tesseract:5 trow.geeksforgeeks.org/library/tesseract:latest

List the images:

$ docker images
REPOSITORY                                     TAG          IMAGE ID       CREATED          SIZE
tesseract                                      5            4a8b781f348b   29 seconds ago   308MB
trow.geeksforgeeks.org/library/tesseract   latest       4a8b781f348b   29 seconds ago   308MB
ubuntu                                         20.04        d5447fc01ae6   3 weeks ago      72.8MB

Now push the image:

$ docker push trow.geeksforgeeks.org/library/tesseract:latest
The push refers to repository [trow.geeksforgeeks.org/library/tesseract]
d6b5cd35d094: Pushed 
f1337b051098: Pushed 
0ec1183a10ba: Pushed 
79a92b7eb192: Pushed 
05b2e6435d75: Pushed 
bcaa36a72e2f: Pushed 
ada7a75be815: Pushed 
0002c93bdb37: Pushed 
latest: digest: sha256:4c61af5027ef185d7d090726aae7e57a9d275a54eedd1630c4efec71cf0d161d size: 2001

You now have a new image successfully pushed to Trow Container Image Registry! We can now use the image to run a container with Docker or Kubernetes.

On Docker, use:

$ docker run --rm trow.geeksforgeeks.org/library/tesseract
Unable to find image 'trow.geeksforgeeks.org/library/tesseract:latest' locally
latest: Pulling from library/tesseract
846c0b181fff: Already exists 
b826d099326f: Pull complete 
6c4d48fea110: Pull complete 
a91c3cb5df5c: Pull complete 
f87ad742ce15: Pull complete 
d4a41703f6bb: Pull complete 
8a2c5dd1ab87: Pull complete 
e2079de4719f: Pull complete 
Usage:
  tesseract --help | --help-extra | --version
  tesseract --list-langs
  tesseract imagename outputbase [options...] [configfile...]

OCR options:
  -l LANG[+LANG]        Specify language(s) used for OCR.
NOTE: These options must occur before any configfile.

Single options:
  --help                Show this help message.
  --help-extra          Show extra help for advanced users.
  --version             Show version information.
  --list-langs          List available languages for tesseract engine.

Assuming the image does not exist locally, it will be pulled and used as shown above.

If you want to remove the trow deployment, use the command:

helm uninstall trow

Books For Learning Kubernetes Administration:

• Best Kubernetes Study books

Conclusion

That marks the end of this guide on how to install and use Trow Container Image Registry With Kubernetes. You now have secure storage for your container images. I hope this was significant.

Relates posts:

• Install Harbor Image Registry on Ubuntu
• Install Harbor Registry on Kubernetes/OpenShift
• Install Harbor Registry on CentOS / Debian / Ubuntu