Build container images from Dockerfile using kaniko in Kubernetes

In the recent past, organizations used to build Docker images outside the Kubernetes cluster. With more companies adopting the Kubernetes technology, the idea of continuous integration builds within the cluster has increased.

However, building and running containers within a cluster pose a security threat since the container has to access the file system of the worker node to connect with the docker daemon. In addition to that, the containers need to run in a privileged mode which exposes the node to innumerable security threats.

This problem has been solved by Google with the introduction of Kaniko. This tool helps one to build container images from a docker file without any access to the docker daemon. It executes all the commands within the Dockerfile completely in the userspace without allowing any access to the host file system.

Kaniko works in the following ways:

• It reads the specified Dockerfile, build context or a remote Docker registry
• Proceeds to extract the base image into the container filesystem
• Runs the commands in the Dockerfile individually.
• After every run, a snapshot of the userspace filesystem is taken
• After every snapshot, it appends only the changed image layers to the base image and updates the image metadata
• Then the image is pushed to the registry

The illustration below will help you understand how Kaniko works.

Before You Begin

This guide requires one to have a Kubernetes cluster with permissions to create, update, list, and delete a job, pods, services, and secrets. There are several guides to help you deploy a Kubernetes cluster.

• Install Kubernetes Cluster on Rocky Linux 8 with Kubeadm & CRI-O
• Install Kubernetes Cluster on Ubuntu using K3s
• Deploy Kubernetes Cluster on Linux With k0s
• Run Kubernetes on Debian with Minikube
• Install Kubernetes Cluster on Ubuntu with kubeadm

In this guide, we will build container images from Dockerfile using Kaniko in Kubernetes with Github and Docker Registry. So, you need the following:

• Github repository with a Dockerfile: we will use the repo URL as the path of the Dockerfile
• Docker hub account: to be able to authenticate and push the Docker image.
• Access to Kubernetes cluster: to be able to deploy the Kaniko pod and create the docker registry secret.

Step 1 – Create the Container Registry Secret

We will start off by setting up the container registry secret. This is useful when pushing the built image to the registry. There are several registries supported by Kaniko. These are:

• Docker Hub
• Google GCR
• Amazon ECR
• Azure Container Registry
• JFrog Container Registry/JFrog Artifactory

These registries are declared using the –destination flag in the manifest. In this guide, we will push the image to the Dockerhub registry.

I assume that you already have a Dockerhub account.

Now create the registry secret using the command:

kubectl create secret docker-registry dockercred \
    --docker-server=https://index.docker.io/v1/ \
    --docker-username=<dockerhub-username> \
    --docker-password=<dockerhub-password>\
    --docker-email=<dockerhub-email>

After this, you should have the secret deployed

$ kubectl get secrets
NAME                  TYPE                                  DATA   AGE
default-token-ncn6w   kubernetes.io/service-account-token   3      3m48s
dockercred            kubernetes.io/dockerconfigjson        1      15s

Step 2 – Configure Kaniko Build Contexts

In the build context, the following types are supported by Kaniko:

• S3 Bucket
• Local Directory
• GCS Bucket
• Azure Blob Storage
• Local Tar
• Git Repository
• Standard Input

The build context represents your directory containing the Dockerfile to be used to build your image.

You can use any of the supported types by specifying them in the manifest using the –context flag. The prefix to be used are:

In this guide, We will use a GitHub repository as our build context. So we need to configure it with the required Dockerfile.

For this guide, we will use a private Git, repository with a Dockerfile. The repo URL is:

https://github.com/neveropen/kubernetes-kaniko

The repository will have a Dockerfile with the below content

FROM ubuntu
ENTRYPOINT ["/bin/bash", "-c", "echo hello"]

The repository will have the file as shown.

Step 3 – Create the Kaniko Pod Manifest

So we will create our manifest below.

vim pod.yaml

Remember to replace the values appropriately.

apiVersion: v1
kind: Pod
metadata:
  name: kaniko
spec:
  containers:
  - name: kaniko
    image: gcr.io/kaniko-project/executor:latest
    args:
    - "--context=git://<Git API token>@github.com/neveropen/kubernetes-kaniko.git#refs/heads/master"
    - "--destination=<dockerhub-username>/kaniko-demo-image:1.0"
    volumeMounts:
    - name: kaniko-secret
      mountPath: /kaniko/.docker
  restartPolicy: Never
  volumes:
  - name: kaniko-secret
    secret:
      secretName: dockercred
      items:
        - key: .dockerconfigjson
          path: config.json

In the above file, we have used the Git API token to authenticate the private git repository. This can be avoided if you are using a public repository. The –destination is the location we want to push the image. For example in my case, the location will be klinsmann1/kaniko-demo-image:1.0.

Step 4 – Run Kaniko in Kubernetes

With the manifest created as desired, run Kaniko with the command:

kubectl apply -f pod.yaml

Follow the image build and push process.

kubectl logs kaniko --follow

Sample output:

After this, I will have my image pushed to Dockerhub.

Step 5 – Pull and Test the Image

Now the image can be used to run containers on both Kubernetes and Docker.

On Docker

Begin by installing Docker on your system. The below guide can help you achieve this:

• How To Install Docker CE on Linux Systems

Now run a container with the image using the command:

docker run -it <user-name>/<repo-name>

For example:

docker run -it klinsmann1/kaniko-demo-image:1.0

Sample Output:

On Kubernetes

The image can also be pulled and tested on Kubernetes. Create a deployment file as shown.

$ vim deploy.yml 
apiVersion: apps/v1
kind: Deployment
metadata:
  name: hello-world
spec:
  selector:
    matchLabels:
      app: hello 
  replicas: 1
  template:
    metadata:
      labels:
        app: hello
    spec:
      containers:
      - name: hello-world
        image: klinsmann1/kaniko-demo-image:1.0

Apply the manifest:

kubectl apply -f deploy.yml

Check the status:

$ kubectl get pods
NAME                           READY   STATUS      RESTARTS      AGE
hello-world-7f67c97454-xv8xs   0/1     Completed   2 (18s ago)   19s
kaniko                         0/1     Completed   0             32m

Check if the execution was successful.

$ kubectl logs hello-world-7f67c97454-xv8xs --follow
hello

Books For Learning Kubernetes Administration:

• Best Kubernetes Study books

Voila!

We have triumphantly walked through how to build container images from Dockerfile using Kaniko in Kubernetes. Now proceed and build your images and use them within your cluster.

Related posts:

• How To Export and Import Docker Images / Containers
• Scan Docker Container Images for Vulnerabilities with Trivy
• How To Build OCI & Docker Container Images With Buildah