Graphite is a developer friendly monitoring tool known for its low hardware resource footprint. Graphite is created to be fault tolerant and highly available. As a developer you can quickly create an application that collects numeric time-series data to be graphed, and sends to the carbon service, a Graphite’s processing backend. The data is stored in Graphite’s specialized database and rendered on demand for graphing.
Graphite’s backend caches incoming data if the disks cannot keep up with the large number of small write operations that occur (each data point is only a few bytes, but most standard disks cannot do more than a few thousand I/O operations per second, even if they are tiny).
If you’re not writing a collector application from scratch you can check out the tools available for data collection and forwarding. In this article we will walk you through the complete installation of Graphite on CentOS 8 | RHEL 8 server by using Podman. The application will run in a container with Systemd service to manage its lifecycle.
The main graphite components are:
- carbon – a Twisted daemon that listens for time-series data
- whisper – a simple database library for storing time-series data (similar in design to RRD)
- graphite webapp – A Django webapp that renders graphs on-demand using Cairo
Install Graphite & Graphite-Web on CentOS 8 | RHEL 8
We will use Podman container runtime to run Graphite on CentOS 8 | RHEL 8. The next steps are to be followed without any customizations unless as recommended in the official project documentation.
The System needs to be up-to-date:
sudo dnf -y updateStep 1: Install Container Tools on CentOS 8 | RHEL 8
Wait for the system to come online then login and install Container Tools.
sudo dnf module install container-tools -yQuery the version of Podman installed to confirm it was successful.
$ podman version
Client: Podman Engine
Version: 4.0.2
API Version: 4.0.2
Go Version: go1.17.7
Built: Sun May 15 16:45:11 2022
OS/Arch: linux/amd64Other container management tools such as buildah and skopeo are installed:
$ buildah version
Version: 1.24.2
Go Version: go1.17.7
Image Spec: 1.0.2-dev
Runtime Spec: 1.0.2-dev
CNI Spec: 1.0.0
libcni Version: v1.0.1
image Version: 5.19.2
Git Commit:
Built: Sun May 15 16:47:19 2022
OS/Arch: linux/amd64
BuildPlatform: linux/amd64
$ skopeo --version
skopeo version 1.6.1Step 2: Run Graphite Container on CentOS 8 | RHEL 8 using Podman
A detailed guide on using Graphite Docker image is available in Docker repo for Graphite. The first step will be for us to pull the latest Docker image.
Let’s switch to root use.
sudo -iDownload the latest Graphite and Statsd container image.
# podman pull docker.io/graphiteapp/graphite-statsd
Trying to pull docker.io/graphiteapp/graphite-statsd...
Getting image source signatures
Copying blob 700de820209a done
Copying blob df20fa9351a1 done
Copying blob f9a569415da5 done
Copying blob 8f0c7d0dc99e done
Copying config 875c7f22f4 done
Writing manifest to image destination
Storing signatures
875c7f22f4cd1b4da99fe6a3ab213e84567d42ed8f544655bfe8694c8d290345
List available images:
# podman images
REPOSITORY TAG IMAGE ID CREATED SIZE
docker.io/library/ubuntu latest 9140108b62dc 2 weeks ago 75.3 MB
docker.io/graphiteapp/graphite-statsd latest 875c7f22f4cd 2 months ago 651 MBThe image we downloaded includes the following components:
- Nginx – reverse proxies the graphite dashboard
- Graphite – front-end dashboard
- Carbon – back-end
- Statsd – UDP based back-end proxy
Mapped Ports
| Host | Container | Service |
|---|---|---|
| 80 | 80 | nginx |
| 2003 | 2003 | carbon receiver – plaintext |
| 2004 | 2004 | carbon receiver – pickle |
| 2023 | 2023 | carbon aggregator – plaintext |
| 2024 | 2024 | carbon aggregator – pickle |
| 8080 | 8080 | Graphite internal gunicorn port (without Nginx proxying). |
| 8125 | 8125 | statsd |
| 8126 | 8126 | statsd admin |
Create data directories to persist container data. They’ll be mapped to container volumes when starting the container.
mkdir -p /data/graphite/{data,logs,conf,statsd_config}Change timezone to your correct setting before running the commands.
podman run -d \
--name graphite \
--restart=always \
-p 80:80 \
-p 2003-2004:2003-2004 \
-p 2023-2024:2023-2024 \
-p 8125:8125/udp \
-p 8126:8126 \
-v /data/graphite/data:/opt/graphite/storage \
-v /data/graphite/conf:/opt/graphite/conf \
-v /data/graphite/statsd_config:/opt/statsd/config \
-v /data/graphite/logs:/var/log \
-e GRAPHITE_TIME_ZONE='Africa/Nairobi' \
graphiteapp/graphite-statsdList running containers to see if graphite container is running.
# podman ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
0a79ef44ad59 docker.io/graphiteapp/graphite-statsd:latest 18 seconds ago Up 17 seconds ago 0.0.0.0:80->80/tcp, 0.0.0.0:8125->8125/udp graphite
Check the ports published and occpupied.
# podman port -l
80/tcp -> 0.0.0.0:80
2003/tcp -> 0.0.0.0:2003
2004/tcp -> 0.0.0.0:2004
2023/tcp -> 0.0.0.0:2023
2024/tcp -> 0.0.0.0:2024
8126/tcp -> 0.0.0.0:8126
8125/udp -> 0.0.0.0:8125Step 3: Open Graphite Web interface
Pu SELinux in Permissive mode:
sudo setenforce 0
sudo sed -i 's/^SELINUX=.*/SELINUX=permissive/g' /etc/selinux/configUse your Server IP address on port 80 to access Graphite web console.
The default logins creds are:
Username: root Password: root
Change this login credentials after the first login at http://ip-address/admin/auth/user/1/. Click “Change Password” link to initiate root user password change.
Logout and back with the new password set.
Step 4: Managing Graphite Container with Systemd
To ensure our container is started at system boot, let’s create new Systemd service unit file.
sudo tee /etc/systemd/system/graphite-podman.service<<EOF
[Unit]
Description=Graphite Docker Container
Documentation=https://github.com/graphite-project/docker-graphite-statsd
[Service]
Type=simple
TimeoutStartSec=0
Restart=on-failure
RestartSec=30s
ExecStartPre=-/usr/bin/podman kill graphite
ExecStartPre=-/usr/bin/podman rm graphite
ExecStartPre=/usr/bin/podman pull graphiteapp/graphite-statsd
ExecStart=/usr/bin/podman run \
--name graphite \
--restart=always \
-p 80:80 \
-p 2003-2004:2003-2004 \
-p 2023-2024:2023-2024 \
-p 8125:8125/udp \
-p 8126:8126 \
-v /data/graphite/data:/opt/graphite/storage \
-v /data/graphite/conf:/opt/graphite/conf \
-v /data/graphite/statsd_config:/opt/statsd/config \
-v /data/graphite/logs:/var/log \
-e GRAPHITE_TIME_ZONE='Africa/Nairobi' \
graphiteapp/graphite-statsd
SyslogIdentifier=graphite
ExecStop=/usr/bin/podman stop graphite
[Install]
WantedBy=multi-user.target
EOFReload Systemd to get new units.
sudo systemctl daemon-reloadConfirm graphite-podman unit file is created.
# systemctl list-unit-files graphite-podman.service
UNIT FILE STATE
graphite-podman.service disabled
1 unit files listed.Enable the service to be started when server is started.
# systemctl enable graphite-podman.service
Created symlink /etc/systemd/system/multi-user.target.wants/graphite-podman.service → /etc/systemd/system/graphite-podman.service.Kill the running container to confirm the service is working.
# podman rm -f graphite
0a79ef44ad5979ade39a3f0c14ca5351771106b429a8318b7912cc9323e3da20Confirm the container has been killed.
# podman ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMESStart Graphite Systemd service.
systemctl start graphite-podman.serviceCheck service status:
# systemctl status graphite-podman.service
● graphite-podman.service - Graphite Docker Container
Loaded: loaded (/etc/systemd/system/graphite-podman.service; enabled; vendor preset: disabled)
Active: active (running) since Sun 2020-10-11 05:50:04 CEST; 22s ago
Docs: https://github.com/graphite-project/docker-graphite-statsd
Process: 7182 ExecStartPre=/usr/bin/podman pull graphiteapp/graphite-statsd (code=exited, status=0/SUCCESS)
Process: 7171 ExecStartPre=/usr/bin/podman rm graphite (code=exited, status=1/FAILURE)
Process: 7160 ExecStartPre=/usr/bin/podman kill graphite (code=exited, status=125)
Main PID: 7194 (podman)
Tasks: 13 (limit: 24402)
Memory: 32.3M
CGroup: /system.slice/graphite-podman.service
└─7194 /usr/bin/podman run --name graphite --restart=always -p 80:80 -p 2003-2004:2003-2004 -p 2023-2024:2023-2024 -p 8125:8125/udp -p 8126:8126 -v /d>
Oct 11 05:50:16 centos-01.geeksforgeeks.org graphite[7194]: 11/10/2020 03:50:16 :: [tagdb] Tagging carbon.aggregator.45793992f2ac-a.memUsage, carbon.aggregat>
Oct 11 05:50:16 centos-01.geeksforgeeks.org graphite[7194]: 11/10/2020 03:50:16 :: [tagdb] Tagged carbon.aggregator.45793992f2ac-a.memUsage, carbon.aggregato>
Oct 11 05:50:16 centos-01.geeksforgeeks.org graphite[7194]: 11/10/2020 03:50:16 :: [tagdb] Tagging carbon.aggregator.45793992f2ac-a.blacklistMatches, carbon.>
Oct 11 05:50:17 centos-01.geeksforgeeks.org graphite[7194]: 11/10/2020 03:50:17 :: [tagdb] Tagged carbon.aggregator.45793992f2ac-a.blacklistMatches, carbon.a>
Oct 11 05:50:26 centos-01.geeksforgeeks.org graphite[7194]: 11/10/2020 03:50:26 :: [listener] MetricLineReceiver connection with 127.0.0.1:60582 established
Oct 11 05:50:26 centos-01.geeksforgeeks.org graphite[7194]: 11/10/2020 03:50:26 :: [listener] MetricLineReceiver connection with 127.0.0.1:60582 closed clean>
Oct 11 05:50:26 centos-01.geeksforgeeks.org graphite[7194]: 11/10/2020 03:50:26 :: [tagdb] Tagging carbon.aggregator.45793992f2ac-a.destinations.127_0_0_1:20>
Oct 11 05:50:26 centos-01.geeksforgeeks.org graphite[7194]: 11/10/2020 03:50:26 :: [tagdb] Tagged carbon.aggregator.45793992f2ac-a.destinations.127_0_0_1:200>
Oct 11 05:50:26 centos-01.geeksforgeeks.org graphite[7194]: 11/10/2020 03:50:26 :: [tagdb] Tagging carbon.agents.45793992f2ac-a.avgUpdateTime, stats.timers.v>
Oct 11 05:50:26 centos-01.geeksforgeeks.org graphite[7194]: 11/10/2020 03:50:26 :: [tagdb] Tagged carbon.agents.45793992f2ac-a.avgUpdateTime, stats.timers.vi>Use Podman CLI to list running containers.
# podman ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
45793992f2ac docker.io/graphiteapp/graphite-statsd:latest About a minute ago Up About a minute ago 0.0.0.0:80->80/tcp, 0.0.0.0:8125->8125/udp graphitePerform system reboot to ensure container is started if server is ever rebooted.
sudo rebootWe can confirm the container is started with different ID:
# podman ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
c57a157a3600 docker.io/graphiteapp/graphite-statsd:latest 34 seconds ago Up 33 seconds ago 0.0.0.0:80->80/tcp, 0.0.0.0:8125->8125/udp graphiteVisit the Graphite documentation for the next steps after installation of Graphite.
More monitoring guides:
