Install Graylog 4 on Ubuntu 22.04|20.04 with Let’s Encrypt

Graylog is an opensource log aggregation and management tool which can be used to store, analyse and send alerts from the logs collected. Graylog can be used to analyse both structured and unstructured logs using ElasticSearch and MongoDB. This includes a variety of systems including Windows systems, Linux systems, different applications and micro-services etc.

Graylog makes it easier to easily analyse, and monitor these systems and applications from a single host.

Graylog has the following components:

• Graylog server
• MongoDB
• ElasticSearch

Let us quickly jump into the installation of Graylog server on an Ubuntu 22.04|20.04 host. We shall then configure SSL using Let’sEncrypt.

To achieve this, we will need to install Nginx to serve as a reverse-proxy on our system.

Similar articles: How To Forward Logs to Grafana Loki using Promtail

Setup Pre-requisites

Before we can install on your box, make sure your host meets the below minimal requirements:

• 4 CPU Cores
• 8 GB RAM
• SSD Hard Disk Space with High IOPS for Elasticsearch Log Storage
• Ubuntu 22.04|20.04 LTS installed and updated.
• All packages upgraded

With the above conditions met, let us begin the installation process.

Step 1 – Install Java on Ubuntu 22.04|20.04

Before Java installation, let’s update and upgrade our system.

sudo apt update && sudo apt -y full-upgrade

We highly recommend you perform a system reboot after the upgrade:

[ -f /var/run/reboot-required ] && sudo reboot -f

Java version 8 and above is required for Graylog installation. In this post, we shall use open JDK 11

sudo apt update
sudo apt install vim apt-transport-https openjdk-11-jre-headless uuid-runtime pwgen curl dirmngr

You can verify the java version you just installed using the java -version command:

$ java -version
openjdk version "11.0.15" 2022-04-19
OpenJDK Runtime Environment (build 11.0.15+10-Ubuntu-0ubuntu0.20.04.1)
OpenJDK 64-Bit Server VM (build 11.0.15+10-Ubuntu-0ubuntu0.20.04.1, mixed mode, sharing)

Step 2 – Install Elasticsearch on Ubuntu 22.04|20.04

Elastic search is the tool that is used to store and analyse incoming logs from external sources. It uses the web-based RESTful API.

Download and install Elasticsearch GPG signing key.

curl  -fsSL https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo gpg --dearmor -o /etc/apt/trusted.gpg.d/elastic.gpg

Add Elasticsearch repository to your sources list:

echo "deb https://artifacts.elastic.co/packages/oss-7.x/apt stable main" | sudo tee /etc/apt/sources.list.d/elastic-7.x.list

Install Elasticsearch:

sudo apt update
sudo apt install elasticsearch-oss -y

Configure cluster name for Graylog.

sudo vim /etc/elasticsearch/elasticsearch.yml

Edit the cluster name to graylog

cluster.name: graylog

Add the following information in the same file

action.auto_create_index: false

Reload daemon the start Elasticsearch service.

sudo systemctl daemon-reload
sudo systemctl restart elasticsearch
sudo systemctl enable elasticsearch

You can check for the status of the service by :

$ systemctl status elasticsearch
● elasticsearch.service - Elasticsearch
     Loaded: loaded (/lib/systemd/system/elasticsearch.service; enabled; vendor preset: enabled)
     Active: active (running) since Sun 2020-11-08 12:36:10 UTC; 14s ago
       Docs: http://www.elastic.co
   Main PID: 1352139 (java)
      Tasks: 15 (limit: 4582)
     Memory: 1.1G
     CGroup: /system.slice/elasticsearch.service
             └─1352139 /bin/java -Xms1g -Xmx1g -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -Des.>

Nov 08 12:36:10 graylog.geeksforgeeks.org systemd[1]: Started Elasticsearch.

Elasticsearch runs on port 9200 and this can be virified by curl command:

curl -X GET http://localhost:9200

You should see your cluster name in the output.

$ curl -X GET http://localhost:9200
{
  "name" : "ubuntu",
  "cluster_name" : "graylog",
  "cluster_uuid" : "RsPmdLmDQUmNGKC-E4JPmQ",
  "version" : {
    "number" : "7.10.2",
    "build_flavor" : "oss",
    "build_type" : "deb",
    "build_hash" : "747e1cc71def077253878a59143c1f785afa92b9",
    "build_date" : "2021-01-13T00:42:12.435326Z",
    "build_snapshot" : false,
    "lucene_version" : "8.7.0",
    "minimum_wire_compatibility_version" : "6.8.0",
    "minimum_index_compatibility_version" : "6.0.0-beta1"
  },
  "tagline" : "You Know, for Search"
}

Step 3 – Install MongoDB on Ubuntu 22.04|20.04

Start by importing the MongoDB 6.0 GPG key to your system:

curl -fsSL https://www.mongodb.org/static/pgp/server-6.0.asc|sudo gpg --dearmor -o /etc/apt/trusted.gpg.d/mongodb-6.gpg

Add the MongoDB 6.0 repo on the system:

echo "deb [ arch=amd64,arm64 ] https://repo.mongodb.org/apt/ubuntu $(lsb_release -cs)/mongodb-org/6.0 multiverse" | sudo tee /etc/apt/sources.list.d/mongodb-org-6.0.list

Download and install mongoDB from Ubuntu’s base repository.

sudo apt update && sudo apt install mongodb-org -y

Start MongoDB

sudo systemctl start mongod
sudo systemctl enable mongod

$ systemctl status mongod
● mongodb.service - An object/document-oriented database
     Loaded: loaded (/lib/systemd/system/mongodb.service; enabled; vendor preset: enabled)
     Active: active (running) since Sun 2020-11-08 12:45:21 UTC; 1s ago
       Docs: man:mongod(1)
   Main PID: 1352931 (mongod)
      Tasks: 3 (limit: 4582)
     Memory: 27.9M
     CGroup: /system.slice/mongodb.service
             └─1352931 /usr/bin/mongod --unixSocketPrefix=/run/mongodb --config /etc/mongodb.conf

Nov 08 12:45:21 graylog.geeksforgeeks.org systemd[1]: Started An object/document-oriented database.

Step 4 – Install Graylog Server on Ubuntu 22.04|20.04

Download and configure Graylog repository.

wget https://packages.graylog2.org/repo/packages/graylog-5.0-repository_latest.deb
sudo dpkg -i graylog-5.0-repository_latest.deb

Install Graylog server:

sudo apt update
sudo apt install graylog-server

Generate a secret to secure user passwords using pwgen command

pwgen -N 1 -s 96

The output should look like:

FFP3LhcsuSTMgfRvOx0JPcpDomJtrxovlSrbfMBG19owc13T8PZbYnH0nxyIfrTb0ANwCfH98uC8LPKFb6ZEAi55CvuZ2Aum

Edit the graylog config file to add the secret we just created:

sudo vim /etc/graylog/server/server.conf

Locate the password_secret = line and add the above created secret after it.

password_secret = FFP3LhcsuSTMgfRvOx0JPcpDomJtrxovlSrbfMBG19owc13T8PZbYnH0nxyIfrTb0ANwCfH98uC8LPKFb6ZEAi55CvuZ2Aum

If you would like to Graylog interface with Server IP Address and port, then set http_bind_address to the public host name or a public IP address of the machine you can connect to.

$ sudo vim /etc/graylog/server/server.conf
http_bind_address = 0.0.0.0:9000

The next step is to create a hash sha256 password for the administrator. This is the password you will need to login to the web interface.

$ echo -n "Enter Password: " && head -1 </dev/stdin | tr -d '\n' | sha256sum | cut -d" " -f1
Enter Password: password

You will get an output of this kind:

e3c652f0ba0b4801205814f8b6bc49672c4c74e25b497770bb89b22cdeb4e951

Edit the /etc/graylog/server/server.conf file then place the hash password at root_password_sha2 =

$ sudo vim /etc/graylog/server/server.conf
root_password_sha2 = e3c652f0ba0b4801205814f8b6bc49672c4c74e25b497770bb89b22cdeb4e951

Graylog is now configured and ready for use.

Start Graylog service:

sudo systemctl daemon-reload
sudo systemctl restart mongodb graylog-server
sudo systemctl enable mongodb graylog-server

You can check if the service has started successfully from the logs:

sudo tail -f /var/log/graylog-server/server.log

Output:

2020-11-08T13:37:55.067Z INFO  [ServerBootstrap] Graylog server up and running.

You can then access graylog web dashboard on:

http://<serverip_hostname>:9000

Step 5 – Configure Nginx with Let’s Encrypt for GrayLog

The next step is to configure SSL so that we can access Graylog web interface via HTTPS.

To achieve this, we will need the following:

• Fully qualified domain name(FQDN)
• Nginx
• Let’sEncrypt certificate

Follow steps in the guide below:

• Configure Graylog Nginx proxy with Let’s Encrypt SSL

The steps are provided below for easy reference.

1. update system and install nginx

sudo apt update
sudo apt install nginx

2. Configure firewall

sudo ufw allow 'Nginx Full'

3. Create virtualhost with your domain name

Create a file in /etc/nginx/sites-available/ e.g

sudo vim /etc/nginx/sites-available/graylog.conf

Add the following in the file:

server {
  listen 80;
  server_name           graylog.yourdomain.com;

  return 301            https://$host$request_uri;
  access_log            /var/log/nginx/graylog.yourdomain.com.access.log combined;
  error_log             /var/log/nginx/graylog.yourdomain.com.error.log;
}

Remember to substitute graylog.yourdomain.com with your FQDN.

4. Create a symlink of the file we just created in /etc/nginx/sites-available to /etc/nginx/sites-enabled

sudo ln -s /etc/nginx/sites-available/graylog.conf /etc/nginx/sites-enabled/

5. Check if nginx config is okay by running nginx -t command.

$ sudo nginx -t
nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
nginx: configuration file /etc/nginx/nginx.conf test is successful

6. Install Let’sEncrypt with certbot.

sudo apt install certbot python3-certbot-nginx

7. Run certbot for nginx

$ sudo certbot --nginx
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Plugins selected: Authenticator nginx, Installer nginx
Enter email address (used for urgent renewal and security notices) (Enter 'c' to
cancel): [email protected]

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Please read the Terms of Service at
https://letsencrypt.org/documents/LE-SA-v1.2-November-15-2017.pdf. You must
agree in order to register with the ACME server at
https://acme-v02.api.letsencrypt.org/directory
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
(A)gree/(C)ancel: A

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Would you be willing to share your email address with the Electronic Frontier
Foundation, a founding partner of the Let's Encrypt project and the non-profit
organization that develops Certbot? We'd like to send you email about our work
encrypting the web, EFF news, campaigns, and ways to support digital freedom.
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
(Y)es/(N)o: N
Which names would you like to activate HTTPS for?
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
1: graylog.geeksforgeeks.org
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Select the appropriate numbers separated by commas and/or spaces, or leave input
blank to select all options shown (Enter 'c' to cancel): 1
Obtaining a new certificate
Performing the following challenges:
http-01 challenge for graylog.geeksforgeeks.org
Waiting for verification...
Cleaning up challenges
Deploying Certificate to VirtualHost /etc/nginx/sites-enabled/graylog.neveropen.tech.conf

Please choose whether or not to redirect HTTP traffic to HTTPS, removing HTTP access.
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
1: No redirect - Make no further changes to the webserver configuration.
2: Redirect - Make all requests redirect to secure HTTPS access. Choose this for
new sites, or if you're confident your site works on HTTPS. You can undo this
change by editing your web server's configuration.
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Select the appropriate number [1-2] then [enter] (press 'c' to cancel): 2
Redirecting all traffic on port 80 to ssl in /etc/nginx/sites-enabled/graylog.neveropen.tech.conf

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Congratulations! You have successfully enabled
https://graylog.geeksforgeeks.org

You should test your configuration at:
https://www.ssllabs.com/ssltest/analyze.html?d=graylog.geeksforgeeks.org
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

IMPORTANT NOTES:
 - Congratulations! Your certificate and chain have been saved at:
   /etc/letsencrypt/live/graylog.geeksforgeeks.org/fullchain.pem
   Your key file has been saved at:
   /etc/letsencrypt/live/graylog.geeksforgeeks.org/privkey.pem
   Your cert will expire on 2021-02-06. To obtain a new or tweaked
   version of this certificate in the future, simply run certbot again
   with the "certonly" option. To non-interactively renew *all* of
   your certificates, run "certbot renew"
 - Your account credentials have been saved in your Certbot
   configuration directory at /etc/letsencrypt. You should make a
   secure backup of this folder now. This configuration directory will
   also contain certificates and private keys obtained by Certbot so
   making regular backups of this folder is ideal.
 - If you like Certbot, please consider supporting our work by:

   Donating to ISRG / Let's Encrypt:   https://letsencrypt.org/donate
   Donating to EFF:                    https://eff.org/donate-le

You have successfully obtained SSL for our domain.

The next step is to configure reverse proxy on Nginx that will be used to serve Graylog which is running on the same host on port 9000.

Edit the /etc/nginx/sites-available/graylog.conf file:

sudo vim /etc/nginx/sites-available/graylog.conf

And add the following configuration under the Location section.

location /
    {
      proxy_set_header Host $http_host;
      proxy_set_header X-Forwarded-Host $host;
      proxy_set_header X-Forwarded-Server $host;
      proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
      proxy_set_header X-Graylog-Server-URL https://$server_name/;
      proxy_pass       http://127.0.0.1:9000;
    }

The final config file should look like:

server {
  listen 80;
  server_name           graylog.example.com;

  return 301            https://$host$request_uri;
  access_log            /var/log/nginx/graylog_access.log combined;
  error_log             /var/log/nginx/graylog_error.log;
}

server {

        root /var/www/html;
        index index.html index.htm index.nginx-debian.html;
        server_name graylog.example.com;


    listen [::]:443 ssl ipv6only=on; # managed by Certbot
    listen 443 ssl; # managed by Certbot
    ssl_certificate /etc/letsencrypt/live/graylog.example.com/fullchain.pem; # managed by Certbot
    ssl_certificate_key /etc/letsencrypt/live/graylog.example.com/privkey.pem; # managed by Certbot
    include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
    ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot

location /
    {
      proxy_set_header Host $http_host;
      proxy_set_header X-Forwarded-Host $host;
      proxy_set_header X-Forwarded-Server $host;
      proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
      proxy_set_header X-Graylog-Server-URL https://$server_name/;
      proxy_pass       http://127.0.0.1:9000;
    }
}

Verify your nginx configuration using the nginx -t just to make sure your nginx configuration is okay.

Now restart nginx service.

sudo systemctl restart nginx

With the above up and running, you should be able to access your graylog dashboard by entering https://graylog.yourdomain.com.

Remember to replace graylog.yourdomain.com with your FQDN.

The default username for Graylog is admin and the password we configured in step 4 (Install Graylog server) above. For my case, this would be “Str0ngPassw0rd”

You can now start using your Graylog web dashboard configured with SSL.

Conclusion

We have successfully installed graylog server, configured SSL through Nginx as a reverse proxy and managed to login to the web interface.

Configuring SSL on Graylog server is important for securing your system.

Should you face any challenge during the setup process feel free to comment or ask any question on the comments section.

• Configure Graylog Nginx reverse proxy with Letsencrypt SSL