Difference between revisions of "Poddery - Diaspora, Matrix and XMPP"

From Free Software Community of India
Jump to: navigation, search
(System health check)
(Reorganized steps in actual order)
Line 1: Line 1:
We run diaspora, xmpp and matrix services at [https://poddery.com poddery.com]. Diaspora username and password can be used to access xmpp and matrix services. [https://chat.poddery.com chat.poddery.com] provides riot client (accessed by a web browser), which can be used to connect to any matrix server without installing a riot app/client.
+
We run Diaspora, XMPP and Matrix services at [https://poddery.com poddery.com]. Diaspora username and password can be used to access XMPP and Matrix services. [https://chat.poddery.com chat.poddery.com] provides Riot client (accessed by a web browser), which can be used to connect to any Matrix server without installing a Riot app/client.
  
 
= Environment =
 
= Environment =
Line 8: Line 8:
 
* 8GB Memory
 
* 8GB Memory
 
* 50GB SSD Disk
 
* 50GB SSD Disk
* 1 Flexible public IPv4
+
* 1 Flexible Public IPv4
 
* 300Mbit/s Unmetered bandwidth
 
* 300Mbit/s Unmetered bandwidth
 
* 2.5Gbit/s Internal bandwidth  
 
* 2.5Gbit/s Internal bandwidth  
* €11.99 Per month
+
* €11.99 Per Month
  
  
Line 19: Line 19:
 
* '''16GB''' Memory
 
* '''16GB''' Memory
 
* 50GB SSD Disk
 
* 50GB SSD Disk
* 1 Flexible public IPv4
+
* 1 Flexible Public IPv4
 
* '''500Mbit/s''' Unmeterd bandwidth
 
* '''500Mbit/s''' Unmeterd bandwidth
 
* '''5Gbit/s''' Internal bandwidth
 
* '''5Gbit/s''' Internal bandwidth
* '''€17.99''' per month
+
* '''€17.99''' Per Month
 
* Extra '''150GB''' SSD
 
* Extra '''150GB''' SSD
* Total '''€20.99''' per month
+
* Total '''€20.99''' Per Month
  
 
== Operating System ==
 
== Operating System ==
  
We run debian 9 stretch image provided by scaleway, with latest security updates applied.
+
We run Debian 9 Stretch image provided by Scaleway, with latest security updates applied.
  
 
=== Hardening checklist ===
 
=== Hardening checklist ===
Line 39: Line 39:
 
   PasswordAuthentication no
 
   PasswordAuthentication no
 
   ...
 
   ...
* Firewall enabled with only the ports we need opened. [https://fxdata.cloud/tutorials/set-up-a-firewall-with-ufw-on-ubuntu-16-04 ufw tutorial]
+
* Firewall enabled with only the ports we need opened ([https://fxdata.cloud/tutorials/set-up-a-firewall-with-ufw-on-ubuntu-16-04 ufw tutorial])
 
   sudo ufw default deny incoming
 
   sudo ufw default deny incoming
 
   sudo ufw default allow outgoing
 
   sudo ufw default allow outgoing
Line 85: Line 85:
 
* /dev/mapper/diaspora is an ext4 file system  
 
* /dev/mapper/diaspora is an ext4 file system  
 
   sudo mkfs.ext4 /dev/mapper/diaspora
 
   sudo mkfs.ext4 /dev/mapper/diaspora
* /var/lib/diaspora should be mounted  
+
* /var/lib/diaspora should be mounted. All [[#Handling_critical_data|critical data]] should be on /var/lib/diaspora.
 
   sudo mkdir /var/lib/diaspora
 
   sudo mkdir /var/lib/diaspora
 
   sudo mount /dev/mapper/diaspora /var/lib/diaspora
 
   sudo mount /dev/mapper/diaspora /var/lib/diaspora
* All critical data should be on /var/lib/diaspora
 
  sudo /etc/init.d/mysql stop
 
  sudo mv /var/lib/mysql /var/lib/diaspora
 
  sudo ln /var/lib/diaspora/mysql /var/lib/diaspora
 
  sudo mkdir /var/lib/diaspora/uploads
 
  sudo chown -R diaspora: /var/lib/diaspora/uploads
 
  sudo ln -s /var/lib/diaspora/uploads /usr/share/diaspora/public/uploads
 
  
=== Services health check ===
+
== User Visible Services ==
 +
=== Diaspora ===
 +
 
 +
* We use diaspora-installer-mysql package from https://people.debian.org/~praveen/diaspora/README
 +
* See [https://salsa.debian.org/ruby-team/diaspora-installer/blob/debian/0.6.6.0+debian1/README /usr/share/doc/diaspora-common/README] for package specific configuration.
 +
* [https://poddery.com/statistics live statistics]
 +
 
 +
=== Chat/XMPP ===
 +
 
 +
* We use Prosody and steps for setting up Prosody is given at -> https://wiki.debian.org/Diaspora/XMPP
 +
* We have enabled all XEPs conversations expect. We use sslh to multiplex Diaspora and Prosody on port 443.
 +
 
 +
=== Chat/Matrix ===
  
  systemctl status nginx # our web server front end for diaspora, xmpp and matrix
+
* We use Synapse server for setting up the Matrix server.
  systemctl status diaspora # diaspora service
+
* We will be using https://github.com/matrix-org/synapse/#synapse-installation for setting up this instance
  systemctl status matrix-synapse.service # synapse matrix server
+
* We use nginx reverse proxy to send requests that has ''/_matrix/*'' in url to synapse on 8008. See /etc/nginx/sites-enabled/diaspora
  systemctl status prosody # prosody xmpp server
+
* We use https://git.fosscommunity.in/necessary129/synapse-diaspora-auth to authenticate synapse with Diaspora database.
  systemctl status sslh # SSL/SSH multiplexer which allow us to provide multiple services via 443 port (to bypass stupid firewalls)
 
  
== User Visible Services ==
 
 
=== Homepage ===
 
=== Homepage ===
  
Homepage and other static pages are maintained in our gitlab instance. You can change it directly in the master branch or send pull requests. You can edit it via web as well.
+
Homepage and other static pages are maintained in our Gitlab instance. You can change it directly in the master branch or send pull requests. You can edit it via web as well.
  
* poddery.com -> https://git.fosscommunity.in/community/poddery.com.
+
* poddery.com -> https://git.fosscommunity.in/community/poddery.com
  
 
* save.poddery.com -> https://git.fosscommunity.in/community/save.poddery.com
 
* save.poddery.com -> https://git.fosscommunity.in/community/save.poddery.com
Line 133: Line 136:
 
  32 2 * * 1 /etc/init.d/nginx reload
 
  32 2 * * 1 /etc/init.d/nginx reload
 
  34 2 * * 1 /etc/init.d/prosody reload
 
  34 2 * * 1 /etc/init.d/prosody reload
 
=== Diaspora ===
 
 
* We use diaspora-installer-mysql package from https://people.debian.org/~praveen/diaspora/README
 
* See [https://salsa.debian.org/ruby-team/diaspora-installer/blob/debian/0.6.6.0+debian1/README /usr/share/doc/diaspora-common/README] for package specific configuration.
 
* [https://poddery.com/statistics live statistics]
 
 
=== Chat/XMPP ===
 
 
* We use prosody and steps for setting up prosody is given at -> https://wiki.debian.org/Diaspora/XMPP
 
* We have enabled all XEPs conversations expect. We use sslh to multiplex diaspora and prosody on port 443.
 
 
=== Chat/Matrix ===
 
 
* We use synapse server for setting up the matrix server.
 
* We will be using https://github.com/matrix-org/synapse/#synapse-installation for setting up this instance
 
* We use nginx reverse proxy to send requests that has ''/_matrix/*'' in url to synapse on 8008. See /etc/nginx/sites-enabled/diaspora
 
* We use https://git.fosscommunity.in/necessary129/synapse-diaspora-auth to authenticate synapse with diaspora database.
 
  
 
== Backend Services ==
 
== Backend Services ==
 
=== nginx ===
 
=== nginx ===
  
Front end for diaspora and matrix.
+
Front-end for Diaspora and Matrix.
  
 
=== PostgreSQL ===
 
=== PostgreSQL ===
  
Backend for matrix.
+
Backend for Matrix.
  
 
=== MySQL ===
 
=== MySQL ===
  
Backend for diaspora. TODO: consider migrating to PostgreSQL to optimize resources (we can reduce one service and RAM usage).
+
Backend for Diaspora.  
 +
 
 +
TODO: consider migrating to PostgreSQL to optimize resources (we can reduce one service and RAM usage).
  
 
=== sslh ===
 
=== sslh ===
  
Port multiplexer to allow xmpp and diaspora to share 443 port. This allows us to fool stupid firewalls which blocks all ports except 80 and 443 (hence xmpp).
+
Port multiplexer to allow XMPP and Diaspora to share 443 port. This allows us to fool stupid firewalls which blocks all ports except 80 and 443 (hence XMPP).
 +
 
 +
=== Handling critical data ===
 +
  sudo /etc/init.d/mysql stop
 +
  sudo mv /var/lib/mysql /var/lib/diaspora
 +
  sudo ln /var/lib/diaspora/mysql /var/lib/diaspora
 +
  sudo mkdir /var/lib/diaspora/uploads
 +
  sudo chown -R diaspora: /var/lib/diaspora/uploads
 +
  sudo ln -s /var/lib/diaspora/uploads /usr/share/diaspora/public/uploads
 +
 
 +
=== Services health check ===
 +
 
 +
  systemctl status nginx # Our web server front-end for Diaspora, XMPP and Matrix
 +
  systemctl status diaspora # Diaspora service
 +
  systemctl status matrix-synapse.service # Synapse Matrix Server
 +
  systemctl status prosody # Prosody XMPP Server
 +
  systemctl status sslh # SSL/SSH multiplexer which allow us to provide multiple services via 443 port (to bypass stupid firewalls)
  
 
= Coordination =
 
= Coordination =
  
 
*[https://www.loomio.org/g/2bjVXqAu/fosscommunity-in-poddery-com-maintainer-s-group loomio group] - we use this for decision making.
 
*[https://www.loomio.org/g/2bjVXqAu/fosscommunity-in-poddery-com-maintainer-s-group loomio group] - we use this for decision making.
* Hangout with us in our matrix room [https://matrix.to/#/#poddery:poddery.com #poddery:poddery.com]
+
* Hangout with us in our Matrix room [https://matrix.to/#/#poddery:poddery.com #poddery:poddery.com]
  
 
=== Contact ===
 
=== Contact ===
Line 178: Line 181:
 
Email: poddery at autistici.org (alias that reaches Akhilan, Abhijith Balan, Fayad, Balasankar, Julius, Praveen, Prasobh, Sruthi, Shirish, Vamsee and Manukrishnan)
 
Email: poddery at autistici.org (alias that reaches Akhilan, Abhijith Balan, Fayad, Balasankar, Julius, Praveen, Prasobh, Sruthi, Shirish, Vamsee and Manukrishnan)
  
The following people have their gpg keys in the password file.
+
The following people have their GPG keys in the password file.
  
Praveen Arimbrathodiyil (piratepin) (ID: 0xCE1F9C674512), Balasankar C (ID: 0x96EDAB9B2E6B7171), Manu Krishnan T V (ID: 0x5D0064186AF037D9), Fayad Fami (fayad) (ID: 0x51C954405D432381), Abhijith PA (ID: 0x863D4DF2ED9C28EF), Syam G Krishnan (sgk) (ID: 0x6EF48CCD865A1FFC), Sagar Ippalpalli (ID: 0xFD49D0BC6FEAECDA)
+
Praveen Arimbrathodiyil (piratepin) (ID: 0xCE1F9C674512), Balasankar C (ID: 0x96EDAB9B2E6B7171), Manu Krishnan T V (ID: 0x5D0064186AF037D9), Fayad Fami (fayad) (ID: 0x51C954405D432381), Abhijith PA (ID: 0x863D4DF2ED9C28EF), Syam G Krishnan (sgk) (ID: 0x6EF48CCD865A1FFC), Sagar Ippalpalli (ID: 0xFD49D0BC6FEAECDA), Pirate Bady (piratesin) (ID: 0x92FDAB42A95FF20C), Kannan (ID: 0x0B1955F40C691CCE)
  
We recommend you setup [http://www.vim.org/scripts/script.php?script_id=3645 vim gpg plugin] for transparent editing. If you are new to gpg, then follow [https://www.madboa.com/geek/gpg-quickstart/ this guide].
+
We recommend you setup [http://www.vim.org/scripts/script.php?script_id=3645 Vim GPG Plugin] for transparent editing. If you are new to GPG, then follow [https://www.madboa.com/geek/gpg-quickstart/ this guide].
  
 
=== Server Access ===
 
=== Server Access ===
Line 219: Line 222:
 
  # apt-get install mysql-server
 
  # apt-get install mysql-server
  
Move mysql data directory to encrypted volume
+
Move MySQL data directory to encrypted volume
 
  # /etc/init.d/mysql stop
 
  # /etc/init.d/mysql stop
 
  # mv /var/lib/mysql /var/lib/diaspora/
 
  # mv /var/lib/mysql /var/lib/diaspora/
Line 233: Line 236:
 
  # su sshtunnel
 
  # su sshtunnel
  
Generate ssh key pair and copy public key to target system
+
Generate SSH key pair and copy public key to target system
 
  $ ssh-keygen -t rsa
 
  $ ssh-keygen -t rsa
 
  $ ssh -f sshtunnel@poddery.com -L 7777:127.0.0.1:3306 -N
 
  $ ssh -f sshtunnel@poddery.com -L 7777:127.0.0.1:3306 -N

Revision as of 11:46, 9 January 2018

We run Diaspora, XMPP and Matrix services at poddery.com. Diaspora username and password can be used to access XMPP and Matrix services. chat.poddery.com provides Riot client (accessed by a web browser), which can be used to connect to any Matrix server without installing a Riot app/client.

Environment

Hosting

We are on a C2S instance of scaleway.com bare metal cloud server.

  • 4 Dedicated x86 64bit Cores
  • 8GB Memory
  • 50GB SSD Disk
  • 1 Flexible Public IPv4
  • 300Mbit/s Unmetered bandwidth
  • 2.5Gbit/s Internal bandwidth
  • €11.99 Per Month


Due to performance issues we are migrating to a new server (C2M instance of scaleway.com) with the following specs:

  • 8 Dedicated x86 64bit Cores
  • 16GB Memory
  • 50GB SSD Disk
  • 1 Flexible Public IPv4
  • 500Mbit/s Unmeterd bandwidth
  • 5Gbit/s Internal bandwidth
  • €17.99 Per Month
  • Extra 150GB SSD
  • Total €20.99 Per Month

Operating System

We run Debian 9 Stretch image provided by Scaleway, with latest security updates applied.

Hardening checklist

  • SSH password login disabled (allow only key based logins)
  • root SSH login disabled (use a normal user with sudo)

/etc/ssh/sshd_config:

 ...
 PermitRootLogin no
 ...
 PasswordAuthentication no
 ...
  • Firewall enabled with only the ports we need opened (ufw tutorial)
 sudo ufw default deny incoming
 sudo ufw default allow outgoing
 sudo ufw allow ssh
 sudo ufw enable

Currently ufw is disabled as it is crashing the server.

  • fail2ban configured against brute force attacks

/etc/ssh/sshd_config:

 ...
 LogLevel VERBOSE
 ...
 sudo systemctl restart ssh
 sudo systemctl enable fail2ban
 sudo systemctl start fail2ban

Check /var/log/fail2ban.log for logs

Unban an IP:

 sudo fail2ban-client set sshd unbanip <banned_ip>

Here sshd is the defaut jail name, change it if you are using a different jail.

System health check

  • There should be a data disk attached (added from cloud.scaleway.com)
  • The attached disk (/dev/nbdX) should be an lvm physical volume. We cannot use it directly for encryption, so we use lvm.
 # Make sure lvm2 and udev packages are installed
 sudo apt-get install lvm2 udev
 
 # Replace X with valid number according to lsblk
 sudo pvcreate /dev/nbdX 
  • /dev/data is an lvm volume group created from /dev/nbdX
 sudo vgcreate data /dev/nbdX
  • /dev/data/diaspora is an lvm logical volume
 sudo lvcreate -n diaspora /dev/data -L <size_of_disk>
  • /dev/mapper/diaspora is an encrypted device
 # Make sure cryptsetup package is installed
 sudo apt-get install cryptsetup
 # Give disk encryption password as specified in the access repo
 sudo cryptsetup luksFormat /dev/data/diaspora
 sudo cryptsetup luksOpen /dev/data/diaspora diaspora
  • /dev/mapper/diaspora is an ext4 file system
 sudo mkfs.ext4 /dev/mapper/diaspora
  • /var/lib/diaspora should be mounted. All critical data should be on /var/lib/diaspora.
 sudo mkdir /var/lib/diaspora
 sudo mount /dev/mapper/diaspora /var/lib/diaspora

User Visible Services

Diaspora

Chat/XMPP

  • We use Prosody and steps for setting up Prosody is given at -> https://wiki.debian.org/Diaspora/XMPP
  • We have enabled all XEPs conversations expect. We use sslh to multiplex Diaspora and Prosody on port 443.

Chat/Matrix

Homepage

Homepage and other static pages are maintained in our Gitlab instance. You can change it directly in the master branch or send pull requests. You can edit it via web as well.

save.poddery.com repo is maintained as a sub module in poddery.com repo. See this tutorial -> https://chrisjean.com/git-submodules-adding-using-removing-and-updating/ for working with git submodules.

SSL/TLS certificates

# letsencrypt certonly --webroot -w /usr/share/diaspora/public  -d poddery.com -d www.poddery.com -d test.poddery.com -d groups.poddery.com -w /usr/share/diaspora/public/save -d save.poddery.com
# cp  -L /etc/letsencrypt/live/poddery.com/fullchain.pem /etc/diaspora/ssl/poddery.com-bundle.pem
# cp -L /etc/letsencrypt/live/poddery.com/privkey.pem /etc/diaspora/ssl/poddery.com.key
# chown -R root:ssl-cert /etc/letsencrypt
# chmod g+r -R /etc/letsencrypt
# chmod g+x /etc/letsencrypt/*
# ls -l /etc/prosody/certs/
total 0
lrwxrwxrwx 1 root root 40 Mar 28 01:16 poddery.com.crt -> /etc/letsencrypt/live/poddery.com/fullchain.pem
lrwxrwxrwx 1 root root 33 Mar 28 01:16 poddery.com.key -> /etc/letsencrypt/live/poddery.com/privkey.pem
# crontab -e
30 2 * * 1 letsencrypt renew  >> /var/log/le-renew.log
32 2 * * 1 /etc/init.d/nginx reload
34 2 * * 1 /etc/init.d/prosody reload

Backend Services

nginx

Front-end for Diaspora and Matrix.

PostgreSQL

Backend for Matrix.

MySQL

Backend for Diaspora.

TODO: consider migrating to PostgreSQL to optimize resources (we can reduce one service and RAM usage).

sslh

Port multiplexer to allow XMPP and Diaspora to share 443 port. This allows us to fool stupid firewalls which blocks all ports except 80 and 443 (hence XMPP).

Handling critical data

 sudo /etc/init.d/mysql stop
 sudo mv /var/lib/mysql /var/lib/diaspora
 sudo ln /var/lib/diaspora/mysql /var/lib/diaspora
 sudo mkdir /var/lib/diaspora/uploads
 sudo chown -R diaspora: /var/lib/diaspora/uploads
 sudo ln -s /var/lib/diaspora/uploads /usr/share/diaspora/public/uploads

Services health check

 systemctl status nginx # Our web server front-end for Diaspora, XMPP and Matrix
 systemctl status diaspora # Diaspora service
 systemctl status matrix-synapse.service # Synapse Matrix Server
 systemctl status prosody # Prosody XMPP Server
 systemctl status sslh # SSL/SSH multiplexer which allow us to provide multiple services via 443 port (to bypass stupid firewalls)

Coordination

Contact

Email: poddery at autistici.org (alias that reaches Akhilan, Abhijith Balan, Fayad, Balasankar, Julius, Praveen, Prasobh, Sruthi, Shirish, Vamsee and Manukrishnan)

The following people have their GPG keys in the password file.

Praveen Arimbrathodiyil (piratepin) (ID: 0xCE1F9C674512), Balasankar C (ID: 0x96EDAB9B2E6B7171), Manu Krishnan T V (ID: 0x5D0064186AF037D9), Fayad Fami (fayad) (ID: 0x51C954405D432381), Abhijith PA (ID: 0x863D4DF2ED9C28EF), Syam G Krishnan (sgk) (ID: 0x6EF48CCD865A1FFC), Sagar Ippalpalli (ID: 0xFD49D0BC6FEAECDA), Pirate Bady (piratesin) (ID: 0x92FDAB42A95FF20C), Kannan (ID: 0x0B1955F40C691CCE)

We recommend you setup Vim GPG Plugin for transparent editing. If you are new to GPG, then follow this guide.

Server Access

Maintained in a private git repo at -> https://git.fosscommunity.in/community/access

Setting up Backup

Backup was setup on a Scaleway C1 VPS (4 core ARM processor with 2GB RAM). TODO: C1 server was crashing frequently and we need to setup backup again on VPS provided by Manu.

Hostname (IP): backup.poddery.com (No public ip, access via scaleway.com web console). If you restart this machine, you may want to add poddery.com private ip in /etc/hosts

# apt-get install lvm2 cryptsetup

Directly creating luks volume on /dev/nbd1 is not working, so we use a logical volume

# pvcreate /dev/nbd1
# vgcreate data /dev/nbd1
# lvcreate -n diaspora -L 46.5G /dev/data
# cryptsetup luksFormat /dev/data/diaspora
# cryptsetup luksOpen /dev/data/diaspora diaspora

and update /etc/crypttab

# <target name> <source device>         <key file>      <options>
diaspora /dev/data/diaspora none luks


# mkfs.ext4 /dev/mapper/diaspora
# mkdir /var/lib/diaspora

and update /etc/fstab

# UNCONFIGURED FSTAB FOR BASE SYSTEM
/dev/mapper/diaspora /var/lib/diaspora ext4 defaults 0 2
# mount -a
# apt-get install mysql-server

Move MySQL data directory to encrypted volume

# /etc/init.d/mysql stop
# mv /var/lib/mysql /var/lib/diaspora/
# ln -s /var/lib/diaspora/mysql /var/lib/mysql

Follow steps in https://dev.mysql.com/doc/refman/5.5/en/replication-howto-masterbaseconfig.html for replication

Follow steps in https://www.howtoforge.com/how-to-set-up-mysql-database-replication-with-ssl-encryption-on-centos-5.4 for ssl (but ssl support is disabled in debian)

Follow steps in http://www.networkcomputing.com/storage/how-set-ssh-encrypted-mysql-replication/1111882674 to use ssh port forwarding to have encrypted replication

# adduser sshtunnel --disabled-login
# su sshtunnel

Generate SSH key pair and copy public key to target system

$ ssh-keygen -t rsa
$ ssh -f sshtunnel@poddery.com -L 7777:127.0.0.1:3306 -N

Test the connectivity

# mysql -u poddery_backup -p -P 7777 -h 127.0.0.1

Uploads are rsynced every hour

# crontab -e
# m h  dom mon dow   command
0 * * * * pgrep rsync || rsync -av --delete root@poddery.com:/var/lib/diaspora/uploads/ /var/lib/diaspora/uploads/ >/var/lib/diaspora/rsync-uploads.log


Note: Since we are not using a public ip (saves us money), backup.poddery.com connects to poddery.com via private ip. So if poddery.com is rebooted, the new ip address should be updated in /etc/hosts file of backup.poddery.com. To connect, use the web console from scaleway.com

Add more disk space

  1. Power off the machine with "ARCHIVE" option. It may take upto an hour for shutdown to complete on backup.poddery.com and poddery.com
  2. Add more disk from scaleway.com control panel . Volumes -> CREATE VOLUME
  3. Attach the newly created volume to server from Server page
  4. Power on the server
  5. Create physical volume (pvcreate /dev/nbdN)
  6. Expand volume group (vgextend data /dev/nbdN)
  7. Expand logical volume (lvresize --size=186G data/diaspora)
  8. Expand encrypted partition (cryptsetup resize diaspora)
  9. Resize file system (resize2fs /dev/mapper/diaspora)

Maintenance history

This section holds maintenance/issue history for future tracking.

When updating diaspora-installer-mysql packages, remember to recreate /usr/share/diaspora/public/uploads symlink to /var/lib/diaspora/uploads.

1. Prosody error - Failed to load private key

certmanager error SSL/TLS: Failed to load '/etc/letsencrypt/live/poddery.com/privkey.pem': Previous error (see logs), or other system error. (for poddery.com)
tls error   Unable to initialize TLS: error loading private key (system lib)
certmanager error SSL/TLS: Failed to load '/etc/letsencrypt/live/poddery.com/privkey.pem': Check that the permissions allow Prosody to read this file.

This error is usually when ssl certificate in freshly installed or renewed. Prosody user is unable to access the key file due to lack of privileges.

Note that Poddery uses Letsencrypt for ssl.

Fix:

  • Make sure that prosody user is in 'certs' group (this group may also be called ssl-certs as setup by Letencrypt)
  • /etc/letsencrypt/ is the ssl directory.
  • Prosody user should have permissions to all folders importantly archive and live folders in /etc/letsencrypt. Permissions to each folder must be 750.
  • Troubleshoot by checking if you can switch to each folder in /etc/letsencrypt as prosody user and cat the files.

If replication fails, you can restart it following the instructions here

https://dba.stackexchange.com/questions/69394/mysql-replication-error-1594