Deploy Metrics Server in Kubernetes using Helm Chart

Kubernetes is one of the popular open-source tools used to orchestrate containerized workloads to run on a cluster of hosts. It works by distributing the workloads across a group of hosts, automating the container networking needs, storage, and persistent volumes, and continuously maintaining the desired state of container applications.

The main aim of Kubernetes is to provide both resilience and scalability. This is achieved by deploying multiple pods with different resources allocated. This helps provide redundancy for the deployed applications. Kubernetes allows users to grow and shrink their deployments manually based on their needs. But is also possible to configure autoscaling using the Horizontal Pod Autoscaler and Vertical Pod Autoscaler.

The Horizontal Pod Autoscaler mainly updates the workload resource such as deployments or StatefulSets, by automatically scaling it to match the demand. It deploys more pods on increased loads and decreased the pods when demand reduces.

The Vertical Pod Autoscaler automatically adjusts the CPU and memory reservations for your pods to match the right requirements. This ensures there is maximum resource utilization and frees up CPU and Memory to be utilized by other pods

The Kubernetes Metrics Server is a scalable source of container metrics for the Kubernetes built-in autoscaling pipelines. It scrapes resource metrics from the Kubelets and exposes them via the Metrics API. These metrics can then be used by Horizontal Pod Autoscaler and Vertical Pod Autoscaler making it easier to debug the autoscaling pipelines.

The Kubernetes Metrics Server offers the following:

• Scalable support up to 5,000 node clusters
• Single deployment that works on most clusters
• Fast and reliable autoscaling, collecting metrics every 15 seconds
• Requires fewer resources, it uses 1 milli core of CPU and 2 MB of memory for each node in a cluster.

The Kubernetes Metrics Server is mainly used in the following cases;

• Obtaining metrics such as CPU and memory utilization in horizontal autoscaling
• Automatically adjusting/suggesting resources needed by containers

For Manual installation method check: Install Metrics Server on a Kubernetes Cluster

Deploy Metrics Server in Kubernetes using Helm Chart

This guide covers all the required steps on how to deploy Metrics Server in Kubernetes using Helm Chart

#1. Perform setup pre-requisites

To be able to deploy the Metrics Server in Kubernetes, you need the following:

• A Kubernetes cluster. This can be deployed using the guides below:
  • Install Kubernetes Cluster on Ubuntu with kubeadm
  • Deploy Kubernetes Cluster on AlmaLinux 8 with Kubeadm
  • Run Kubernetes on Debian with Minikube
  • Deploy Kubernetes Cluster on Linux With k0s
  • Deploy HA Kubernetes Cluster on Rocky Linux 8 using RKE2
• You also need kubectl installed and configured to connect to your cluster.

Install kubectl using the commands:

curl -LO "https://storage.googleapis.com/kubernetes-release/release/$(curl -s https://storage.googleapis.com/kubernetes-release/release/stable.txt)/bin/linux/amd64/kubectl"
chmod +x kubectl
sudo mv kubectl /usr/local/bin

To get access to your cluster, export the config:

# For k0s
export KUBECONFIG=/var/lib/k0s/pki/admin.conf

##For RKE2
export PATH=$PATH:/var/lib/rancher/rke2/bin
export KUBECONFIG=/etc/rancher/rke2/rke2.yaml

Check the nodes in the cluster

$ kubectl get nodes
NAME     STATUS   ROLES           AGE     VERSION
master   Ready    control-plane   118s    v1.24.4+k0s
node1    Ready    <none>          2m13s   v1.24.4+k0s
node2    Ready    <none>          2m13s   v1.24.4+k0s

Install Helm on your system using the aid in the guide:

• Install and Use Helm 3 on Kubernetes Cluster

#2. Deploy Metrics Server in Kubernetes using Helm

Create a namespace for the Metrics server:

kubectl create ns metrics-server

Set it as the default namespace:

kubectl config set-context $(kubectl config current-context) --namespace=metrics-server 

Once the required tools have been set up, add the helm repo for the Metrics Server;

helm repo add metrics-server https://kubernetes-sigs.github.io/metrics-server

Update the repo:

helm repo update

Obtains values from the repo:

helm show values metrics-server/metrics-server > ~/metrics-server.values

Now edit the values as desired.

vim ~/metrics-server.values

Make the desired configuration to the file. For example replicas, you can also configure the hostNetwork and additional parameters in the args

....
hostNetwork:
  # Specifies if metrics-server should be started in hostNetwork mode.
  #
  # You would require this enabled if you use alternate overlay networking for pods and
  # API server unable to communicate with metrics-server. As an example, this is required
  # if you use Weave network on EKS
  enabled: false

replicas: 2
....
args: []
...

There are other parameters, for more explanation, refer to the Metrics Server chart page. Example on enabling insecure conenction:

....
defaultArgs:
  - --cert-dir=/tmp
  - --kubelet-preferred-address-types=InternalIP,ExternalIP,Hostname
  - --kubelet-use-node-status-port
  - --metric-resolution=15s
  - --kubelet-insecure-tls

Now deploy the Metrics server using Helm charts with the values;

helm install metrics-server metrics-server/metrics-server -n  metrics-server --values ~/metrics-server.values

Sample Output:

NAME: metrics-server
LAST DEPLOYED: Mon Aug 29 08:12:15 2022
NAMESPACE: metrics-server
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
***********************************************************************
* Metrics Server                                                      *
***********************************************************************
  Chart version: 3.8.2
  App version:   0.6.1
  Image tag:     k8s.gcr.io/metrics-server/metrics-server:v0.6.1
***********************************************************************

Once created. verify with the command:

$ helm ls -n metrics-server
NAME          	NAMESPACE     	REVISION	UPDATED                               	STATUS  	CHART               	APP VERSION
metrics-server	metrics-server	1       	2022-08-29 08:12:15.56603769 -0400 EDT	deployed	metrics-server-3.8.2	0.6.1  

View other components

$ kubectl get all -n metrics-server
NAME                                  READY   STATUS    RESTARTS   AGE
pod/metrics-server-7cd9d56884-pcptv   1/1     Running   0          56s
pod/metrics-server-7cd9d56884-zdfc8   1/1     Running   0          56s

NAME                     TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)   AGE
service/metrics-server   ClusterIP   10.105.242.23   <none>        443/TCP   56s

NAME                             READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/metrics-server   2/2     2            2           56s

NAME                                        DESIRED   CURRENT   READY   AGE
replicaset.apps/metrics-server-7cd9d56884   2         2         2       56s

Once installed, you can perform an upgrade after changing values using:

helm upgrade metrics-server metrics-server/metrics-server -n  metrics-server --values ~/metrics-server.values

#3. Test the resource Metrics API

To test if the metrics-server component is installed and working as desired, we will use the top command below:

View the node’s resources

kubectl top nodes  

You can also view the pod resources

kubectl top pods -A

Sample Output:

#4. Configure Horizontal Pod Autoscaling

This example shows how to configure Horizontal Pod Autoscaling so that the number of pods grows and shrinks automatically.

There are two ways of creating a Horizontal Pod Autoscaler. These are:

• Using a manifest file
• Using the kubectl autoscale command to an existing deployment.

A. Create Demo Deployment

We will create a deployment for Horizontal Pod Autoscaling testing.

vim deploy.yml

Add the below lines to the file:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: myapp-test
spec:
  selector:
    matchLabels:
      run: myapp-test
  replicas: 1
  template:
    metadata:
      labels:
        run: myapp-test
    spec:
      containers:
        - name: busybox
          image: busybox
          resources:
            limits:
              cpu: 50m
            requests:
              cpu: 20m
          command: ["sh", "-c"]
          args:
            - while [ 1 ]; do
              echo "Test";
              sleep 0.01;
              done

Apply the manifest.

kubectl apply -f deploy.yml

Verify if the deployment is up:

$ kubectl get deploy
NAME             READY   UP-TO-DATE   AVAILABLE   AGE
metrics-server   2/2     2            2           33m
myapp-test       1/1     1            1           7s

B. Create the Horizontal Pod Autoscaler

You can use any of the methods below to configure Horizontal Pod Autoscaler on your deployment.

Using a Manifest

Create the Horizontal Pod Autoscaler CRD:

vim crd.yaml

In the file, add the below lines:

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: my-app-hpa
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: myapp-test
  minReplicas: 1
  maxReplicas: 3
  metrics:
    - type: Resource
      resource:
        name: cpu
        target:
          type: Utilization
          averageUtilization: 50

In the file above;

• spec.scaleTargetRef: is the resource being scaled
• spec.minReplicas: is the lower limit of the number of pods to be autoscaled
• spec.maxReplicas: is the upper limit
• spec.metrics.type: this is the metrics to be used to calculate the desired replica count. For example here., the scaling will depend on the CPU with the limit set to averageUtilization: 50

Now apply the manifest:

kubectl apply -f crd.yaml

Using the kubectl autoscale command

Another option for running the HPA on your application is by using the kubectl autoscale command.

For the created deployment, we will have the command as:

kubectl autoscale deployment my-app-hpa --cpu-percent=50 --min=1 --max=3

This command creates an HPA on the myapp-test deployment with CPU max value set to 50 and scaling between 1-3 replicas

Now verify the creation:

# kubectl get hpa
NAME         REFERENCE               TARGETS    MINPODS   MAXPODS   REPLICAS   AGE
my-app-hpa   Deployment/myapp-test   160%/50%   1         3         1          10m

The TARGETS value will be <unknown>/50% for some time since the HPA needs to collect average values for a while.

We now have an HPA configured with the TARGETS column displaying the current usage%/target usage%.

You can also view details with the command:

$ kubectl describe hpa my-app-hpa
Name:                                                  my-app-hpa
Namespace:                                             metrics-server
Labels:                                                <none>
Annotations:                                           <none>
CreationTimestamp:                                     Mon, 29 Aug 2022 16:51:23 +0300
Reference:                                             Deployment/myapp-test
Metrics:                                               ( current / target )
  resource cpu on pods  (as a percentage of request):  150% (30m) / 50%
Min replicas:                                          1
Max replicas:                                          3
Deployment pods:                                       3 current / 3 desired
Conditions:
  Type            Status  Reason            Message
  ----            ------  ------            -------
  AbleToScale     True    ReadyForNewScale  recommended size matches current size
  ScalingActive   True    ValidMetricFound  the HPA was able to successfully calculate a replica count from cpu resource utilization (percentage of request)

The app will then be autoscaled depending on the demands:

# kubectl get deploy
NAME             READY   UP-TO-DATE   AVAILABLE   AGE
metrics-server   2/2     2            2           48m
myapp-test       3/3     3            3           15m

# kubectl get pods
NAME                              READY   STATUS    RESTARTS   AGE
metrics-server-7cd9d56884-pcptv   1/1     Running   0          47m
metrics-server-7cd9d56884-zdfc8   1/1     Running   0          47m
myapp-test-656d4b8df4-88rnc       1/1     Running   0          2m44s
myapp-test-656d4b8df4-fpxwb       1/1     Running   0          13m
myapp-test-656d4b8df4-pxhkt       1/1     Running   0          2m44s

Books For Learning Kubernetes Administration:

• Best Kubernetes Study books

Conclusion

That marks the end of this guide on how to deploy Metrics Server in Kubernetes using Helm Chart. We have also demonstrated how the Metrics Server can be used and also configured Horizontal Pod Autoscaler. Now explore other features on your own. I hope this was significant

See more:

• Enable Horizontal Pod Autoscaler on EKS Kubernetes Cluster
• Amazon EKS Autoscaling Based on Cluster Metrics
• Automate App Deployment updates on Kubernetes using Keel