Install OpenStack on Rocky Linux 8 – Setup Keystone (Step 3)

In Part 1 of this OpenStack installation series on Rocky Linux 8 we covered initial pre-requisites installations. Perform the actions in Step 1 and Step 2 prior to following the steps covered here. This step will only focus on the configuration of Keystone service. Keystone is a service in OpenStack that provides API client authentication, service discovery, and distributed multi-tenant authorization by implementing OpenStack’s Identity API.

In Keystone you’ll interact with the following Services.

• Identity – Service that provides auth credential validation and data about users and groups
• Resource – Provides data about projects and domains.
• Assignment – Provides data about roles and role assignments.
• Token – Validates and manages tokens used for authenticating requests once a user’s credentials have already been verified.
• Catalog – Provides an endpoint registry used for endpoint discovery.

Step 1: Configure MariaDB database for Keystone

Login to MariaDB shell as root user.

mysql -u root -p

For all database users created for OpenStack related services, we’ll use the password OpenStackSRVDBPassw0rd. With this in mind let’s create database user for Keystone service.

CREATE DATABASE keystone; 
GRANT ALL PRIVILEGES ON keystone.* TO keystone@'localhost' IDENTIFIED BY 'OpenStackSRVDBPassw0rd'; 
GRANT ALL PRIVILEGES ON keystone.* TO keystone@'%' IDENTIFIED BY 'OpenStackSRVDBPassw0rd'; 
FLUSH PRIVILEGES;
QUIT

Step 2: Install and Configure Keystone Service

Install all package needed to configure Keystone service on OpenStack node.

sudo dnf -y install openstack-keystone python3-mod_wsgi python3-oauth2client

Also install Apache httpd web server

sudo dnf -y install httpd mod_ssl

Install OpenStack Client tools

sudo dnf -y  install python3-openstackclient 

Open keystone configuration file for editing

sudo vim /etc/keystone/keystone.conf

Set Memcached server address – for single server install you can use 127.0.0.1 or localhost. For multiple nodes used IP address of server running Memcached.

# Line 441
memcache_servers = localhost:11211
#memcache_servers = 192.168.1.10:11211

Configure MariaDB database connection

#Line 649
[database]
connection = mysql+pymysql://keystone:OpenStackSRVDBPassw0rd@127.0.0.1/keystone

Under [token] section, configure the Fernet token provider:

[token]
#Line 2587
provider = fernet

Perform a database migrations after updating the settings:

$ sudo su -s /bin/bash keystone -c "keystone-manage db_sync"
2022-10-20 00:12:40.781 16138 INFO migrate.versioning.api [-] 72 -> 73...
2022-10-20 00:12:41.789 16138 INFO migrate.versioning.api [-] done
2022-10-20 00:12:41.789 16138 INFO migrate.versioning.api [-] 73 -> 74...
2022-10-20 00:12:41.791 16138 INFO migrate.versioning.api [-] done
2022-10-20 00:12:41.792 16138 INFO migrate.versioning.api [-] 74 -> 75...
2022-10-20 00:12:41.795 16138 INFO migrate.versioning.api [-] done
2022-10-20 00:12:41.796 16138 INFO migrate.versioning.api [-] 75 -> 76...
2022-10-20 00:12:41.798 16138 INFO migrate.versioning.api [-] done
2022-10-20 00:12:41.798 16138 INFO migrate.versioning.api [-] 76 -> 77...
2022-10-20 00:12:41.801 16138 INFO migrate.versioning.api [-] done
2022-10-20 00:12:41.801 16138 INFO migrate.versioning.api [-] 77 -> 78...
2022-10-20 00:12:41.803 16138 INFO migrate.versioning.api [-] done
2022-10-20 00:12:41.804 16138 INFO migrate.versioning.api [-] 78 -> 79...
2022-10-20 00:12:41.836 16138 INFO migrate.versioning.api [-] done
2022-10-20 00:12:41.903 16138 INFO migrate.versioning.api [-] 72 -> 73...
2022-10-20 00:12:41.910 16138 INFO migrate.versioning.api [-] done
2022-10-20 00:12:41.910 16138 INFO migrate.versioning.api [-] 73 -> 74...
2022-10-20 00:12:41.913 16138 INFO migrate.versioning.api [-] done
2022-10-20 00:12:41.914 16138 INFO migrate.versioning.api [-] 74 -> 75...
2022-10-20 00:12:41.916 16138 INFO migrate.versioning.api [-] done
2022-10-20 00:12:41.916 16138 INFO migrate.versioning.api [-] 75 -> 76...
2022-10-20 00:12:41.919 16138 INFO migrate.versioning.api [-] done
2022-10-20 00:12:41.919 16138 INFO migrate.versioning.api [-] 76 -> 77...
2022-10-20 00:12:41.921 16138 INFO migrate.versioning.api [-] done
2022-10-20 00:12:41.921 16138 INFO migrate.versioning.api [-] 77 -> 78...
2022-10-20 00:12:41.924 16138 INFO migrate.versioning.api [-] done
2022-10-20 00:12:41.924 16138 INFO migrate.versioning.api [-] 78 -> 79...
2022-10-20 00:12:41.926 16138 INFO migrate.versioning.api [-] done
2022-10-20 00:12:41.986 16138 INFO migrate.versioning.api [-] 72 -> 73...
2022-10-20 00:12:41.990 16138 INFO migrate.versioning.api [-] done
2022-10-20 00:12:41.990 16138 INFO migrate.versioning.api [-] 73 -> 74...
2022-10-20 00:12:41.992 16138 INFO migrate.versioning.api [-] done
2022-10-20 00:12:41.992 16138 INFO migrate.versioning.api [-] 74 -> 75...
2022-10-20 00:12:41.995 16138 INFO migrate.versioning.api [-] done
2022-10-20 00:12:41.995 16138 INFO migrate.versioning.api [-] 75 -> 76...
2022-10-20 00:12:41.998 16138 INFO migrate.versioning.api [-] done
2022-10-20 00:12:41.998 16138 INFO migrate.versioning.api [-] 76 -> 77...
2022-10-20 00:12:42.001 16138 INFO migrate.versioning.api [-] done
2022-10-20 00:12:42.001 16138 INFO migrate.versioning.api [-] 77 -> 78...
2022-10-20 00:12:42.003 16138 INFO migrate.versioning.api [-] done
2022-10-20 00:12:42.003 16138 INFO migrate.versioning.api [-] 78 -> 79...
2022-10-20 00:12:42.006 16138 INFO migrate.versioning.api [-] done

Next initialize Fernet key repositories. First command:

$ sudo keystone-manage fernet_setup --keystone-user keystone --keystone-group keystone
2022-10-20 00:13:45.151 16206 INFO keystone.common.utils [-] /etc/keystone/fernet-keys/ does not appear to exist; attempting to create it
2022-10-20 00:13:45.152 16206 INFO keystone.common.fernet_utils [-] Created a new temporary key: /etc/keystone/fernet-keys/0.tmp
2022-10-20 00:13:45.152 16206 INFO keystone.common.fernet_utils [-] Become a valid new key: /etc/keystone/fernet-keys/0
2022-10-20 00:13:45.152 16206 INFO keystone.common.fernet_utils [-] Starting key rotation with 1 key files: ['/etc/keystone/fernet-keys/0']
2022-10-20 00:13:45.152 16206 INFO keystone.common.fernet_utils [-] Created a new temporary key: /etc/keystone/fernet-keys/0.tmp
2022-10-20 00:13:45.152 16206 INFO keystone.common.fernet_utils [-] Current primary key is: 0
2022-10-20 00:13:45.152 16206 INFO keystone.common.fernet_utils [-] Next primary key will be: 1
2022-10-20 00:13:45.153 16206 INFO keystone.common.fernet_utils [-] Promoted key 0 to be the primary: 1
2022-10-20 00:13:45.153 16206 INFO keystone.common.fernet_utils [-] Become a valid new key: /etc/keystone/fernet-keys/0

Second command:

$ sudo keystone-manage credential_setup --keystone-user keystone --keystone-group keystone
2022-10-20 00:15:09.852 16316 INFO keystone.common.utils [-] /etc/keystone/credential-keys/ does not appear to exist; attempting to create it
2022-10-20 00:15:09.853 16316 INFO keystone.common.fernet_utils [-] Created a new temporary key: /etc/keystone/credential-keys/0.tmp
2022-10-20 00:15:09.853 16316 INFO keystone.common.fernet_utils [-] Become a valid new key: /etc/keystone/credential-keys/0
2022-10-20 00:15:09.853 16316 INFO keystone.common.fernet_utils [-] Starting key rotation with 1 key files: ['/etc/keystone/credential-keys/0']
2022-10-20 00:15:09.853 16316 INFO keystone.common.fernet_utils [-] Created a new temporary key: /etc/keystone/credential-keys/0.tmp
2022-10-20 00:15:09.853 16316 INFO keystone.common.fernet_utils [-] Current primary key is: 0
2022-10-20 00:15:09.853 16316 INFO keystone.common.fernet_utils [-] Next primary key will be: 1
2022-10-20 00:15:09.853 16316 INFO keystone.common.fernet_utils [-] Promoted key 0 to be the primary: 1
2022-10-20 00:15:09.854 16316 INFO keystone.common.fernet_utils [-] Become a valid new key: /etc/keystone/credential-keys/0

Define Keystone Host; this is mostly the controller node. In my case it’s the same machine I’m running these commands.

[root@osp01 ~]# ip ad
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: eno1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel master ovs-system state UP group default qlen 1000
    link/ether 1c:69:7a:61:da:e9 brd ff:ff:ff:ff:ff:ff
    inet6 fe80::1e69:7aff:fe61:dae9/64 scope link
       valid_lft forever preferred_lft forever
3: wlp0s20f3: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether 08:71:90:27:5b:be brd ff:ff:ff:ff:ff:ff
4: ovs-system: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether 82:c5:75:7c:7a:61 brd ff:ff:ff:ff:ff:ff
5: br-ex: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000
    link/ether 1c:69:7a:61:da:e9 brd ff:ff:ff:ff:ff:ff
    inet6 fe80::b045:abff:fe47:aa48/64 scope link
       valid_lft forever preferred_lft forever
6: vlan30: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000
    link/ether 26:54:c4:5c:3d:8b brd ff:ff:ff:ff:ff:ff
    inet 172.20.30.3/24 brd 172.20.30.255 scope global vlan30
       valid_lft forever preferred_lft forever
    inet6 fe80::2454:c4ff:fe5c:3d8b/64 scope link
       valid_lft forever preferred_lft forever

Define controller host (for multi-node it should be where keystone is deployed)

export controller=172.20.30.3

Bootstrap keystone API by running the commands below.

Using HTTPS Endpoint (Recommended):

sudo keystone-manage bootstrap --bootstrap-password 'StrongAdminPassw0rd' \
--bootstrap-admin-url https://$controller:5000/v3/ \
--bootstrap-internal-url https://$controller:5000/v3/ \
--bootstrap-public-url https://$controller:5000/v3/ \
--bootstrap-region-id RegionOne

Using HTTP Endpoint:

sudo keystone-manage bootstrap --bootstrap-password 'StrongAdminPassw0rd' \
--bootstrap-admin-url http://$controller:5000/v3/ \
--bootstrap-internal-url http://$controller:5000/v3/ \
--bootstrap-public-url http://$controller:5000/v3/ \
--bootstrap-region-id RegionOne

You can replace StrongAdminPassw0rd with a suitable password for an administrative user if you desire so. Sample output is as shown:

2022-10-20 02:43:31.320 9574 INFO keystone.cmd.bootstrap [req-04170693-6366-462b-8a0e-058ab92c0fa1 - - - - -] Created domain default
2022-10-20 02:43:31.346 9574 INFO keystone.cmd.bootstrap [req-04170693-6366-462b-8a0e-058ab92c0fa1 - - - - -] Created project admin
2022-10-20 02:43:31.549 9574 INFO keystone.cmd.bootstrap [req-04170693-6366-462b-8a0e-058ab92c0fa1 - - - - -] Created user admin
2022-10-20 02:43:31.554 9574 INFO keystone.cmd.bootstrap [req-04170693-6366-462b-8a0e-058ab92c0fa1 - - - - -] Created role reader
2022-10-20 02:43:31.561 9574 INFO keystone.cmd.bootstrap [req-04170693-6366-462b-8a0e-058ab92c0fa1 - - - - -] Created role member
2022-10-20 02:43:31.570 9574 INFO keystone.cmd.bootstrap [req-04170693-6366-462b-8a0e-058ab92c0fa1 - - - - -] Created implied role where 66e3a5fcabf241b4a5c0e09597b3c848 implies 449ffcdb38394b769c92a8f7395b640a
2022-10-20 02:43:31.576 9574 INFO keystone.cmd.bootstrap [req-04170693-6366-462b-8a0e-058ab92c0fa1 - - - - -] Created role admin
2022-10-20 02:43:31.582 9574 INFO keystone.cmd.bootstrap [req-04170693-6366-462b-8a0e-058ab92c0fa1 - - - - -] Created implied role where 5fe3c0b4ee264eceb97e415cdf19be8a implies 66e3a5fcabf241b4a5c0e09597b3c848
2022-10-20 02:43:31.593 9574 INFO keystone.cmd.bootstrap [req-04170693-6366-462b-8a0e-058ab92c0fa1 - - - - -] Granted role admin on project admin to user admin.
2022-10-20 02:43:31.597 9574 INFO keystone.cmd.bootstrap [req-04170693-6366-462b-8a0e-058ab92c0fa1 - - - - -] Granted role admin on the system to user admin.
2022-10-20 02:43:31.602 9574 WARNING py.warnings [req-04170693-6366-462b-8a0e-058ab92c0fa1 - - - - -] /usr/lib/python3.6/site-packages/pycadf/identifier.py:72: UserWarning: Invalid uuid: RegionOne. To ensure interoperability, identifiers should be a valid uuid.
  'identifiers should be a valid uuid.' % (value)))

2022-10-20 02:43:31.604 9574 INFO keystone.cmd.bootstrap [req-04170693-6366-462b-8a0e-058ab92c0fa1 - - - - -] Created region RegionOne
2022-10-20 02:43:31.617 9574 INFO keystone.cmd.bootstrap [req-04170693-6366-462b-8a0e-058ab92c0fa1 - - - - -] Created public endpoint https://172.20.30.3:5000/v3/
2022-10-20 02:43:31.625 9574 INFO keystone.cmd.bootstrap [req-04170693-6366-462b-8a0e-058ab92c0fa1 - - - - -] Created internal endpoint https://172.20.30.3:5000/v3/
2022-10-20 02:43:31.634 9574 INFO keystone.cmd.bootstrap [req-04170693-6366-462b-8a0e-058ab92c0fa1 - - - - -] Created admin endpoint https://172.20.30.3:5000/v3/

Open port 5000 in the firewall if the service is active.

sudo firewall-cmd --add-port=5000/tcp
sudo firewall-cmd --reload

Step 3: Configure the Apache HTTP server

We shall configure Keystone with SSL. This is recommended to ensure access to keystone from other services is secured connectivity.

Using self-signed certificates

Create self-signed certificates for Keystone.

Set server hostname

$ sudo vim /etc/pki/tls/openssl.cnf
[example.com ]
subjectAltName = osp.example.com

Generate SSL key and certificates while replacing example.com with your own OpenStack domain.

cd /etc/pki/tls/certs
sudo -i
openssl genrsa -aes128 2048 > openstack_server.key
openssl rsa -in openstack_server.key -out openstack_server.key
openssl req -utf8 -new -key openstack_server.key -out openstack_server.csr
openssl x509 -in openstack_server.csr -out openstack_server.crt -req -signkey openstack_server.key -extfile /etc/pki/tls/openssl.cnf -extensions example.com -days 3650

Limit write permission to the private key.

chmod 600 openstack_server.key

Configure SSL in Keystone Web configuration

Add SSL / TLS settings immediately after <VirtualHost *:5000>

$ sudo vim /usr/share/keystone/wsgi-keystone.conf
<VirtualHost *:5000>
      SSLEngine on
      SSLHonorCipherOrder on
      SSLCipherSuite PROFILE=SYSTEM
      SSLProxyCipherSuite PROFILE=SYSTEM
      SSLCertificateFile /etc/pki/tls/certs/openstack_server.crt
      SSLCertificateKeyFile /etc/pki/tls/certs/openstack_server.key

Create a symlink in /etc/httpd/conf.d/ directory.

sudo ln -s /usr/share/keystone/wsgi-keystone.conf /etc/httpd/conf.d/

Restart httpd service for the changes to reflect.

sudo systemctl restart httpd

Enable the service to start with the system.

sudo systemctl enable --now httpd

Check service to confirm it’s running.

$ systemctl status httpd
● httpd.service - The Apache HTTP Server
   Loaded: loaded (/usr/lib/systemd/system/httpd.service; enabled; vendor preset: disabled)
   Active: active (running) since Thu 2022-10-20 01:40:02 EAT; 22s ago
     Docs: man:httpd.service(8)
 Main PID: 4548 (httpd)
   Status: "Running, listening on: port 5000, port 443, port 80"
    Tasks: 233 (limit: 408287)
   Memory: 86.6M
   CGroup: /system.slice/httpd.service
           ├─4548 /usr/sbin/httpd -DFOREGROUND
           ├─4551 /usr/sbin/httpd -DFOREGROUND
           ├─4552 (wsgi:keystone- -DFOREGROUND
           ├─4553 (wsgi:keystone- -DFOREGROUND
           ├─4554 (wsgi:keystone- -DFOREGROUND
           ├─4555 (wsgi:keystone- -DFOREGROUND
           ├─4556 (wsgi:keystone- -DFOREGROUND
           ├─4557 /usr/sbin/httpd -DFOREGROUND
           ├─4558 /usr/sbin/httpd -DFOREGROUND
           └─4559 /usr/sbin/httpd -DFOREGROUND

Oct 20 01:40:02 osp01.home.cloudlabske.io systemd[1]: Starting The Apache HTTP Server...
Oct 20 01:40:02 osp01.home.cloudlabske.io systemd[1]: Started The Apache HTTP Server.
Oct 20 01:40:02 osp01.home.cloudlabske.io httpd[4548]: Server configured, listening on: port 5000, port 443, port 80

Step 4: Add Service Project to Keystone

By default, the trust store contains the Mozilla CA list, including positive and negative trust. You need to update the core Mozilla CA list by including our CA used in step 3. In RHEL based systems the consolidated system-wide trust store is located in the /etc/pki/ca-trust/ and /usr/share/pki/ca-trust-source/ directories.

We can add a certificate in the simple PEM or DER file formats to the list of CAs trusted on the system by copying it to /usr/share/pki/ca-trust-source/anchors/ directory.

cp openstack /usr/share/pki/ca-trust-source/anchors/
sudo update-ca-trust

Create keystonerc file by setting the proper environmental variables:

$ vim ~/keystonerc_admin
export OS_PROJECT_DOMAIN_NAME=default
export OS_USER_DOMAIN_NAME=default
export OS_PROJECT_NAME=admin
export OS_USERNAME=admin
export OS_PASSWORD=StrongAdminPassw0rd
export OS_AUTH_URL=https://$controller:5000/v3
export OS_IDENTITY_API_VERSION=3
export OS_IMAGE_API_VERSION=2
export PS1='[\u@\h \W(keystone)]\$ '

These values shown here are the default ones created from keystone-manage bootstrap. When done source file

echo "source ~/keystonerc_admin " >> ~/.bash_profile
source ~/keystonerc_admin

Confirm it works by listing OpenStack endpoints.

$ openstack endpoint list
+----------------------------------+-----------+--------------+--------------+---------+-----------+------------------------------+
| ID                               | Region    | Service Name | Service Type | Enabled | Interface | URL                          |
+----------------------------------+-----------+--------------+--------------+---------+-----------+------------------------------+
| 12eb6a4bf3e544778a718b9fc868467d | RegionOne | keystone     | identity     | True    | public    | https://172.20.30.3:5000/v3/ |
| a355f33e9ec74c1d920e46dbed6b3309 | RegionOne | keystone     | identity     | True    | admin     | https://172.20.30.3:5000/v3/ |
| ee0b149c12e64fe58b0fbe9c011c6738 | RegionOne | keystone     | identity     | True    | internal  | https://172.20.30.3:5000/v3/ |
+----------------------------------+-----------+--------------+--------------+---------+-----------+------------------------------+

Create service project using openstack command line tool.

$ openstack project create --domain default --description "Service Project" service
+-------------+----------------------------------+
| Field       | Value                            |
+-------------+----------------------------------+
| description | Service Project                  |
| domain_id   | default                          |
| enabled     | True                             |
| id          | 07b09cf0cddc4e0b9d29db4cf8f72a58 |
| is_domain   | False                            |
| name        | service                          |
| options     | {}                               |
| parent_id   | default                          |
| tags        | []                               |
+-------------+----------------------------------+

Confirm this works by listing all projects in Keystone.

$ openstack project list
+----------------------------------+---------+
| ID                               | Name    |
+----------------------------------+---------+
| 07b09cf0cddc4e0b9d29db4cf8f72a58 | service |
| 9f9562d16c904ec4848aad6767b3858e | admin   |
+----------------------------------+---------+

Conclusion

We now have Keystone configured and can be consumed by other OpenStack services that will be configured in the steps to follow. Notice that HTTPS endpoints are optional, but highly recommended. If you choose to use HTTP API endpoints for Keystone the setup should be functional just like HTTPS. For Self-signed certificate use, remember to update CA trust list on the host system or use commercially signed certificates.

Next article: Install OpenStack on Rocky Linux 8 – Configure Glance (Step 4)