CloudWatch is a service which collects operational and monitoring data in the form of logs, metrics, and events in AWS Cloud platform. It then visualizes the data by using automated dashboards so you can get a unified view of your AWS resources, applications, and services that run in AWS and on-premises.
After deployment of EKS or Kubernetes cluster, there is a need to configure Container Insights to send data to CloudWatch.
A pre-requisite for this setup is a functional Amazon EKS or Kubernetes cluster. If you need reference installation posts we have them available in our website.
Easily Setup Kubernetes Cluster on AWS with EKS
Install Kubernetes Cluster on Ubuntu with kubeadm
Install Kubernetes Cluster on CentOS 7 with kubeadm
Check to confirm kubectl can connect to the Kubernetes API Server.
$ kubectl get nodes
NAME STATUS ROLES AGE VERSION
ip-192-168-138-244.eu-west-1.compute.internal Ready <none> 9d v1.17.9-eks-4c6976
ip-192-168-176-247.eu-west-1.compute.internal Ready <none> 9d v1.17.9-eks-4c6976Create and Attach a policy to the IAM role of your worker nodes
Open the Amazon EC2 console at https://console.aws.amazon.com/ec2/ and select one of the Worker nodes.
Choose the IAM role in the description. On the IAM role page, choose Attach policies.
Select CloudWatchAgentServerPolicy checkbox and attach the policy.
Setup Container Insights on Amazon EKS
Once the policy has been attached to the EKS worker nodes role proceed to deploy Container Insights on Amazon EKS.
Save your cluster name and region as variables.
EKS_CLUSTER_NAME="prod-eks-cluster"
EKS_CLUSTER_REGION="eu-west-1"Download installation manifest:
curl -O https://raw.githubusercontent.com/aws-samples/amazon-cloudwatch-container-insights/latest/k8s-deployment-manifest-templates/deployment-mode/daemonset/container-insights-monitoring/quickstart/cwagent-fluentd-quickstart.yamlUpdate the file to set correct name of cluster and region.
sed -ie "s/{{cluster_name}}/${EKS_CLUSTER_NAME}/;s/{{region_name}}/${EKS_CLUSTER_REGION}/" cwagent-fluentd-quickstart.yamlInstall Container Insights on Amazon EKS with the following command.
kubectl apply -f ./cwagent-fluentd-quickstart.yamlA number of resources are created as seen in the command execution output:
namespace/amazon-cloudwatch unchanged
serviceaccount/cloudwatch-agent unchanged
clusterrole.rbac.authorization.k8s.io/cloudwatch-agent-role unchanged
clusterrolebinding.rbac.authorization.k8s.io/cloudwatch-agent-role-binding unchanged
configmap/cwagentconfig unchanged
daemonset.apps/cloudwatch-agent unchanged
configmap/cluster-info unchanged
serviceaccount/fluentd unchanged
clusterrole.rbac.authorization.k8s.io/fluentd-role unchanged
clusterrolebinding.rbac.authorization.k8s.io/fluentd-role-binding unchanged
configmap/fluentd-config unchanged
daemonset.apps/fluentd-cloudwatch unchangedGet status of all resources in amazon-cloudwatch namespace.
$ kubectl get all -n amazon-cloudwatch
NAME READY STATUS RESTARTS AGE
pod/cloudwatch-agent-lqtfw 1/1 Running 0 5m21s
pod/cloudwatch-agent-lsz92 1/1 Running 0 5m21s
pod/fluentd-cloudwatch-6j2jf 1/1 Running 0 5m17s
pod/fluentd-cloudwatch-dzrdq 1/1 Running 0 5m17s
NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGE
daemonset.apps/cloudwatch-agent 2 2 2 2 2 <none> 5m22s
daemonset.apps/fluentd-cloudwatch 2 2 2 2 2 <none> 5m18s
We can see all pods are in running state.These were created by daemon sets.
Viewing Container Insights Metrics
After you have Container Insights set up and it is collecting metrics, you can view those metrics in the CloudWatch console.
Choose Performance Monitoring in the navigation pane from where you can select resource type metrics to view.
More guides:
EKS Kubernetes Persistent Storage with EFS Storage Service
Using Horizontal Pod Autoscaler on Kubernetes EKS Cluster
