As of OpenShift 4 release, Red Hat Enterprise Linux CoreOS (RHCOS) is the the recommended and supported operating system running on all OpenShift Container Platform machines. RHCOS combines the quality standards of Red Hat Enterprise Linux (RHEL) with the automated, remote upgrade features from Container Linux.
RHCOS does not ship with package managers such as yum or dnf. This OS features transactional upgrades using the rpm-ostree system where updates are delivered by means of container images and are part of the OpenShift Container Platform update process.
Without OS package manager and support for direct download and installation of RPM packages the only way to run tools not pre-packaged with the OS is through containers. The good thing is CoreOS comes with a script called toolbox that launches a container to let you bring in your favorite debugging or admin tools.
Running toolbox container in OpenShift 4 RHCOS machines
You can start the toolbox container using the toolbox script provided. But first access the OpenShift node from where you want to run admin tools in the container.
You can use oc debug command or SSH.
### Access node with SSH ###
$ ssh core@NodeIP
### Access node with oc debug command ###
$ oc debug node/<nodename>
Access with oc debug example.
$ oc debug node/node01.ocp.geeksforgeeks.org
Starting pod/node01ocpneveropencom-debug ...
To use host binaries, run `chroot /host`The prompt comes from a special-purpose tools container that mounts the node root file system at the /host folder, and allows yoy to inspect the files from the node.
You need to start a chroot shell in the /host folder as shown in the command output. This will enable you to use host binaries in the shell.
chroot /hostYou’ll see output like below:
chroot /host
Pod IP: 10.10.30.235
If you don't see a command prompt, try pressing enter.
sh-4.2# chroot /host
sh-4.4# To start the toolbox container use the following command.
# /usr/bin/toolboxThe first time you run the script it will download the toolbox container image to your node.
Trying to pull registry.redhat.io/rhel8/support-tools...
Getting image source signatures
Copying blob ec1681b6a383 done
Copying blob c4d668e229cd done
Copying blob 6b1688d3542f done
Copying config 50b63c2aff done
Writing manifest to image destination
Storing signatures
50b63c2aff8c13f9f8594c9eaf5fc961f39c74df6d9c6ddde8ca705f78f3c14dIt will then spin a container using Podman container runtime.
Spawning a container 'toolbox-core' with image 'registry.redhat.io/rhel8/support-tools'
Detected RUN label in the container image. Using that as the default...
command: podman run -it --name toolbox-core --privileged --ipc=host --net=host --pid=host -e HOST=/host -e NAME=toolbox-core -e IMAGE=registry.redhat.io/rhel8/support-tools:latest -v /run:/run -v /var/log:/var/log -v /etc/machine-id:/etc/machine-id -v /etc/localtime:/etc/localtime -v /:/host registry.redhat.io/rhel8/support-tools:latest
Run telnet / tcpdump in OpenShift v4 CoreOS Servers
Once you’re in the container shell, you can use yum package manager to install debug and administration tools you want.
### Install network tools ###
yum -y install iproute net-tools
### Install telnet ###
yum -y install telnet
### Install tcpdump ###
# yum -y install tcpdump
### Install any other tool ###
yum -y install <packagename>Using telnet:
# telnet <destinantionhost> <serviceport>Using tcpdump:
Identify interface name – You need network tools installed.
# ip link show | head
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: ens3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP mode DEFAULT group default qlen 1000
link/ether 00:1a:4a:16:01:73 brd ff:ff:ff:ff:ff:ff
7: ovs-system: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000
link/ether 4e:66:b9:32:0d:26 brd ff:ff:ff:ff:ff:ff
8: br0: <BROADCAST,MULTICAST> mtu 1450 qdisc noop state DOWN mode DEFAULT group default qlen 1000
link/ether 72:d6:df:e8:13:48 brd ff:ff:ff:ff:ff:ff
9: vxlan_sys_4789: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 65000 qdisc noqueue master ovs-system state UNKNOWN mode DEFAULT group default qlen 1000
link/ether 4a:c4:7f:c1:85:f7 brd ff:ff:ff:ff:ff:ff
Using tcpdump.
# tcpdump <OPTIONS>This example is for capturing packets from any interface destined to port 443. We’re saving the output to a path in Node file system, accessible on /host directory.
# tcpdump -i any port 443 -s 0 -vv -s 0 -w /host/tmp/testpacketname.pcapYou can replace any with interface name, e.g:
-i ens3To end the capture press Control-C.
Running tcpdump capture from a container
Open a debug shell or SSH to a Node where the target Pod is running:
$ oc debug node/<nodename>
##OR##
$ ssh core@NodeIPIdentify the ID of the target Pod’s Process ID using the crictl ps command:
# crictl psMy container ID is 51a17d9a4b376. Let’s save this as variable.
container_id="51a17d9a4b376"Get container PID:
container_pid=$(crictl inspect --output yaml $container_id | grep 'pid:' | awk '{print $2}')Confirm value:
# echo $container_pid
1124033
# ps 1124033
PID TTY STAT TIME COMMAND
1124033 ? Ss 0:00 /bin/sleep 3650dYou can the start the tcpdump in the container’s network namespace using the command below.
# nsenter -n -t $container_pid -- tcpdump <OPTIONS>
### Example ###
# tcpdump -i any port 443 -s 0 -vv -s 0 -w /host/tmp/testpacketname.pcapPlease note that tcpdump needs to be installed in the container before running the commands.
More articles on OpenShift.
