In our previous guide, we walked through how to run Synapse Matrix homeserver in Docker Containers. Today we will make the task even simpler by deploying the Matrix Server using Ansible and Docker. But before diving into this matter’s crux, we need to define and get familiar with several concepts.
What is Matrix?
Matrix can be defined as a standard protocol used in real-time message relays such as VoIP/WebRTC signalling, Internet of Things communication, messaging and video calls. It works to eradicate the complexity involved with fragmented IP communications. It acts as a generic HTTP messaging and data synchronisation system for the whole web to allow people, services and devices all over the planet to communicate easily. Matrix users are able to communicate with each other without having to care what app the other user is on.
Matrix offers the following features and benefits to users:
- Open source reference implementations of:
- Clients (Web (React), iOS, Android)
- Client SDKs (Javascript, Web (React), iOS, Android)
- Home servers (Synapse)
- Application Services (bridges to IRC, Slack, Skype, Lync e.t.c)
- The whole ecosystem and community of everyone running Matrix servers and services
- 3rd party contributions of clients, SDKs, servers and services.
- Open Standard HTTP APIs for transferring JSON messages including:
- Client-Server API: defines how clients communicate with Matrix home servers.
- Server-Server API – defines the communication between Matrix home servers(how they exchange messages and synchronise history with each other)
- Application Service API – defines how to extend the functionality of Matrix with ‘integrations’ and bridge to other networks.
- Modules – specifies features that must be implemented by particular classes of clients.
- The whole ecosystem and community of everyone running Matrix servers and services
What is Synapse?
Setting up a private home server is a good practice among friends, family or colleagues. It can act as the day-to-day medium of communication within your home server while keeping all the data will be safe and secure in your server. A homeserver can be defined as any implementation deployed on a server that can be accessed through any Matrix client such as Element.
Synapse is one of the popular Matrix home servers developed in 2014 by Matrix.org Foundation. This tool is intended to showcase the concept of Matrix and allow users to run their own home servers which will generally help bootstrap the ecosystem.
Now let’s dive in!
Setup Pre-requisites
The Matrix Server Ansible playbooks best work on an x86 server(recommended) with any of the below operating systems:
- CentOS (7 only for now; 8 is not yet supported)
- Debian (10/Buster or newer)
- Ubuntu (18.04 or newer)
- Archlinux
This guide requires the following installed and configured:
- Python installed on the server for Ansible
- Ansible to run the playbooks
- git
- make
- DNS records for your Domain
These packages can be installed in your workspace as shown:
Install Python git and make:
##On Debian/Ubuntu
sudo apt update
sudo apt install python3 python3-pip git make -y
##On RHEL/Rocky Linux/Alma Linux
sudo yum install python3 python3-pip git make -y
If you have multiple versions of Python3, set the default version as shown:
$ sudo update-alternatives --config python3
There are 2 programs which provide 'python3'.
Selection Command
-----------------------------------------------
*+ 1 /usr/bin/python3.8
2 /usr/bin/python3.9
Enter to keep the current selection[+], or type selection number: 2Verify the made changes:
$ python3 --version
Python 3.9.14You can use the installed Python-PIP to easily install ansible on your system. This can be done using the command:
sudo pip3 install ansibleYou can also install ansible from package managers:
##On Ubuntu / Debian
sudo apt install ansible
##On RHEL/Rocky Linux/Alma Linux
sudo yum install epel-release
sudo yum install ansible ansible-coreOnce the installation is complete, verify the version:
$ ansible --version
ansible [core 2.14.0]
config file = /etc/ansible/ansible.cfg
configured module search path = ['/home/alma9/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
ansible python module location = /usr/local/lib/python3.9/site-packages/ansible
ansible collection location = /home/alma9/.ansible/collections:/usr/share/ansible/collections
executable location = /bin/ansible
python version = 3.9.14 (main, Nov 7 2022, 00:00:00) [GCC 11.3.1 20220421 (Red Hat 11.3.1-2)] (/usr/bin/python3.9)
jinja version = 3.1.2
libyaml = True
1. Configure DNS for Matrix Server
For this deployment, we will use several ports. The playbook will install docker and configure the server’s internal firewall for you. But if you have another firewall enabled, you need to allow these ports through it:
- 80/tcp: HTTP webserver
- 443/tcp: HTTPS webserver
- 3478/tcp: TURN over TCP (used by Coturn)
- 3478/udp: TURN over UDP (used by Coturn)
- 5349/tcp: TURN over TCP (used by Coturn)
- 5349/udp: TURN over UDP (used by Coturn)
- 8448/tcp: Matrix Federation API HTTPS web server
- 49152-49172/udp: TURN over UDP
Now you also need to configure DNS for your server. The following DNS settings are enabled by default:
| Type | Host | Priority | Weight | Port | Target |
|---|---|---|---|---|---|
| A | matrix | – | – | – | matrix-server-IP |
| CNAME | element | – | – | – | matrix.<your-domain> |
In case you are using Cloudflare DNS, you need to disable the proxy and set all records to DNS only, otherwise fetching certificates might fail. There are some other DNS settings that are optional.
For this guide we will set up DNS by editing the /etc/host file as shown:
$ sudo vim /etc/hosts
192.168.205.2 matrix.neveropen.co.za element.neveropen.co.zaReplace 192.168.205.2 with the IP address of the server you intend to run the deployment on(managed node) in the above file. You also need to provide your desired domain name in place of neveropen.co.za. This ensures that your host will resolve requests for both matrix and element subdomains.
2. Configure Ansible for Matrix Server
We will begin by downloading the playbooks and all the configurations by cloning the repository below on our workspace:
git clone https://github.com/spantaleev/matrix-docker-ansible-deploy.gitOnce downloaded, navigate into the directory:
cd matrix-docker-ansible-deployThis directory contains all the required configurations and playbooks for our deployment. Create all the required roles with the command:
make rolesNow we will create the variables directory:
domain_name=matrix.neveropen.co.za
mkdir inventory/host_vars/$domain_nameRemember to replace your domain name appropriately. Once created, copy the sample config into it:
cp examples/vars.yml inventory/host_vars/$domain_name/vars.ymlNow open the file for editing:
vim inventory/host_vars/$domain_name/vars.ymlIn the opened file, we need to make a few adjustments:
##Your Domain. Example value: example.com
matrix_domain: matrix.neveropen.co.za
# - the `docs/configuring-playbook-IMPLEMENTATION_NAME.md` documentation page, if one is available for your implementation choice
matrix_homeserver_implementation: synapse
# A secret used as a base, for generating various other secrets.
# You can put any string here, but generating a strong one is preferred (e.g. `pwgen -s 64 1`).
matrix_homeserver_generic_secret_key: ''
# you won't be required to define this variable (see `docs/configuring-playbook-ssl-certificates.md`).
#
# Example value: [email protected]
matrix_ssl_lets_encrypt_support_email: ''
# The playbook creates additional Postgres users and databases (one for each enabled service)
##PostgreSQL password
devture_postgres_connection_password: 'password'
##SSL certificates retrieval
matrix_ssl_retrieval_method: manually-managed
##Retries when pulling images and creating containers
devture_playbook_help_container_retries_delay: 9
devture_playbook_help_container_retries_count: 9
##Delay for systemd_service
devture_systemd_service_manager_up_verification_delay_seconds: 60
##Other configurations, enable them accordingly
#matrix_nginx_proxy_https_enabled: false
#matrix_coturn_enabled: false
#matrix_nginx_proxy_base_domain_serving_enabled: trueThe matrix_homeserver_generic_secret_key can be generated using the command:
pwgen -s 64 1Above are some of the basic configurations for your deployment. In the above configuration, we have set SSL retrieval to manually-managed since we want to use our own generated certificates. This can also apply in case you have custom certificates.
If this option is commented out, the playbook retrieves and auto-renews free SSL certificates from Let’s Encrypt for the domains it needs (matrix. and possibly element.)
For the case of manually-managed certificates, you need to create the SSL certificate files in the directory specified by matrix_ssl_config_dir_path. The default path is /matrix/ssl/config.
So we will create these certificates on the host we intend to do the deployment on(managed node).
For matrix.neveropen.co.za
sudo mkdir -p /matrix/ssl/config/live/matrix.neveropen.co.za
cd /matrix/ssl/config/live/matrix.neveropen.co.zaWhile here, we will create the chain.pem and privkey.pem. Generate the self-signed certificates with the command:
sudo openssl req \
-newkey rsa:4096 -nodes -sha256 -keyout privkey.pem \
-addext "subjectAltName = DNS:matrix.neveropen.co.za" \
-x509 -days 365 -out chain.pemYou also need the fullchain.pem which can be generated with the command:
sudo openssl req -new -x509 -key privkey.pem > fullchain.pemOnce generated, proceed and create the element.neveropen.co.za certificates:
sudo mkdir -p /matrix/ssl/config/live/element.neveropen.co.za
cd /matrix/ssl/config/live/element.neveropen.co.zaGenerate the certificates here:
sudo openssl req \
-newkey rsa:4096 -nodes -sha256 -keyout privkey.pem \
-addext "subjectAltName = DNS:element.neveropen.co.za" \
-x509 -days 365 -out chain.pemGenerate the fullchain.pem certificate:
sudo openssl req -new -x509 -key privkey.pem > fullchain.pemIf you do not want to be bothered with SSL certificates, you can disable SSL certificate management in the variables file:
matrix_ssl_retrieval_method: noneBut for this case, you can use an external web server which is not Nginx and no certificates will be retrieved at all. You’re free to manage them however you want.
Once the desired configurations have been made to the variables file, close it and proceed as shown below.
3. Create the Ansible Hosts Inventory file
The Ansible Hosts inventory file comprises the managed nodes. For this case, we will work with a single node.
Create the file with the command:
vim inventory/hostsIn this file, we will add the domain name, IP address and sudo user of the managed node:
[matrix_servers]
matrix.neveropen.co.za ansible_host=192.168.205.11 ansible_ssh_user=root
[matrix_servers:vars]
ansible_python_interpreter=/usr/bin/python3For this case, we are using the root user for the deployment. Remember to permit root login on the remote host before you proceed:
$ sudo vim /etc/ssh/sshd_config
PermitRootLogin yesFor the changes to apply, restart the SSH service:
sudo systemctl restart sshNow generate the SSh keys and copy them to the managed node:
ssh-keygen -t rsa
ssh-copy-id -i ~/.ssh/id_rsa.pub root@managed-node-IPNow we should be able to log in to the managed node using the SSH keys. Verify if everything is okay:
$ ansible -m ping all -i inventory/hosts
matrix.neveropen.co.za | SUCCESS => {
"changed": false,
"ping": "pong"
}4. Installing Matrix Server using Ansible and Docker
Now we are set to perform our deployment on the remote host using the playbooks. The general command for the deployment has the syntax:
ansible-playbook -i inventory/hosts setup.yml --tags=COMMA_SEPARATED_TAGS_GO_HEREThere are several tags that can be used here:
- setup-all: runs all setup tasks including installation and uninstallation for all components, but does not start/restart services
- install-all: it is similar to setup-all, but skips uninstallation tasks. It is useful for maintaining your setup quickly when its components remain unchanged. If you adjust your vars.yml to remove components, you’d need to run setup-all though, or these components will still remain installed.
- setup-SERVICE (e.g. setup-bot-postmoogle): runs the setup tasks only for a given role, but does not start/restart services.
- install-SERVICE (e.g. install-bot-postmoogle): like setup-SERVICE, but skips uninstallation tasks.
- start – starts all systemd services and makes them start automatically in the future
- stop – stops all systemd services
- ensure-matrix-users-created: a special tag which ensures that all special users needed by the playbook (for bots, etc.) are created
There are also other tags that can be used:
- –ask-pass: applies where you don’t use SSH keys for authentication, but rather a regular password.
- –ask-become-pass: applies if you do use SSH keys for authentication, and use a non-root user to become root (sudo).
There are two ways of running the installation:
- Installing a brand new server. No data importation is required:
ansible-playbook -i inventory/hosts setup.yml --tags=setup-all,ensure-matrix-users-created,start- Installing a server into which you’ll import old data
ansible-playbook -i inventory/hosts setup.yml --tags=install-allAfter running the above command, you can then import the existing SQLite, Postgres or media store data files from an existing installation.
Now start the services with the command:
ansible-playbook -i inventory/hosts setup.yml --tags=startFor this case, we go with the first installation type. Sample output:
Verify if the containers are running on your managed host:
# docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
2671933d417d nginx:1.23.2-alpine "/docker-entrypoint.…" About a minute ago Up About a minute 80/tcp, 0.0.0.0:8448->8448/tcp, :::8448->8448/tcp, 0.0.0.0:80->8080/tcp, :::80->8080/tcp, 0.0.0.0:443->8443/tcp, :::443->8443/tcp matrix-nginx-proxy
2c836b51987c vectorim/element-web:v1.11.15 "/docker-entrypoint.…" About a minute ago Up About a minute 80/tcp matrix-client-element
5cb6d4e9e733 nginx:1.23.2-alpine "/docker-entrypoint.…" About a minute ago Up About a minute 80/tcp matrix-synapse-reverse-proxy-companion
edb82f02702a matrixdotorg/synapse:v1.72.0 "/start.py run -m sy…" About a minute ago Up About a minute (healthy) 8008-8009/tcp, 8448/tcp matrix-synapse
5fb5ea546895 devture/exim-relay:4.95-r0-4 "/sbin/tini -- exim …" About a minute ago Up About a minute 8025/tcp matrix-mailer
fb15e6b9f2ec coturn/coturn:4.6.1-r0-alpine "turnserver -c /turn…" About a minute ago Up About a minute 0.0.0.0:3478->3478/tcp, 0.0.0.0:3478->3478/udp, :::3478->3478/tcp, :::3478->3478/udp, 0.0.0.0:5349->5349/udp, :::5349->5349/udp, 0.0.0.0:5349->5349/tcp, 0.0.0.0:49152-49172->49152-49172/udp, :::5349->5349/tcp, :::49152-49172->49152-49172/udp matrix-coturn
c27ce1dcd9b8 postgres:15.1-alpine "docker-entrypoint.s…" About a minute ago Up About a minute 5432/tcp matrix-postgresIf you make any changes to the vars.yml, either removing or adding components in the future, you need to run the installation with --tags=setup-all instead of --tags=install-all
5. Create and Register Users on the Matrix Server
After the deployment, we need to access the Matrix Server. In that case, we need to create a user to be able to log in. A user can be created using an ansible command with the below syntax:
ansible-playbook -i inventory/hosts setup.yml --extra-vars='username=<your-username> password=<your-password> admin=<yes|no>' --tags=register-userFor example:
ansible-playbook -i inventory/hosts setup.yml --extra-vars='username=neveropen password=Passw0rd! admin=yes' --tags=register-userOnce the user has been created, proceed to the below step.
6. Access Synapse Matrix home server
Now with the right DNS records configured in /etc/hosts, access your matrix client. For this case, we have set up the element web client which is accessible on element.neveropen.co.za. We have a redirection configured in that when we load https://matrix.neveropen.co.za, it will redirect us to the element page
Sign in using the created user.
Once authenticated, you will see the below page.
Begin by creating a new room. Click on the +icon shown below.
Enter the preferred room name and make some other desired configs.
You will then have the room created, invite members and begin communication.
I will invite another user created on this server:
Once the user accepts the invitation, you can proceed with your communication. Below is a sample communication using my Matrix home server
Verdict
That marks the end of this guide. We have systematically walked through how to deploy Matrix Server using Ansible and Docker. This can make the deployment easier and faster when dealing with several hosts. I hope this was significant to you.
See more:
4 Best Open Source Alternative To Communication Proprietary Software
Install and Use FFmpeg on Rocky Linux 9 / AlmaLinux 9
Bypass Any Website Region Restrictions using Ivacy VPN
