dev-libre-is/docs/_source/mail-libre-is.rst

=============
mail.libre.is
=============
Documentation for Libre mail server.

Setting up Internet mail servers is a pain.
It's nothing like just setting up a web server...

The initial Administration section is for after the server has
been set up and configured. Below it is how the server is set up.


Administration
==============
Once everything is setup working...


ispmail_userctl
---------------
The easiest way to admin is to run this script:

.. code-block:: sh

  sudo ispmail_userctl


Database
--------
To admin directly, log into the database using the mailadmin database
password:

.. code-block:: sh

  mariadb -u mailadmin -p mailserver


List Virtual Domains
--------------------

.. code-block:: sql

  SELECT * FROM virtual_domains;


List Virtual Users
------------------

.. code-block:: sql

  SELECT * FROM virtual_users;


List Virtual Aliases
--------------------

.. code-block:: sql

  SELECT * FROM virtual_aliases;


Add Virtual Domain
------------------

Add the domain:

.. code-block:: sql

  INSERT INTO virtual_domains (name) VALUES ("example.org");


Add a Mail User
---------------
Generate a password with dovecot:

.. code-block:: sh

  sudo dovecot pw -s BLF-CRYPT

Log into database as mailadmin and run this command, using the dovecot
generated password string, and set the domain and user email.

Note, upstream docs are missing "(" and ")" for VALUES.

.. code-block:: sql

  INSERT INTO virtual_users (domain_id, email, password) VALUES
    ((SELECT id FROM virtual_domains WHERE name='example.org'),
    'john@example.org','{BLF-CRYPT}$2y$05$.We…');


Add a User Alias
----------------
Example to add an alias. The first email should be the alias,
the second email is where it should go to.

.. code-block:: sql

  INSERT INTO virtual_aliases (domain_id, source, destination) VALUES
    ( (SELECT id FROM virtual_domains WHERE name='example.org'),
    'melissa@example.org', 'juila@example.net');


Change a User Password
----------------------
Generate string for new password:

.. code-block:: sh

  sudo dovecot pw -s BLF-CRYPT

Use that string:

.. code-block:: sql

  UPDATE virtual_users SET password='{BLF-CRYPT}$2y$05$.We…' WHERE email='email@address';


Delete Virtual Domain
---------------------

.. code-block:: sql

  DELETE FROM virtual_domains where name='example.org';


Delete User
-----------

.. code-block:: sql

  DELETE FROM virtual_users WHERE email='john@example.org';


Delete Alias
------------

.. code-block:: sql

  DELETE FROM virtual_aliases WHERE source='melissa@example.org';


Email Client Setup
==================
Set up mail clients, such as Thunderbird, thusly.

Using example email address username@libre.is.

.. code-block:: cfg

  Incoming Server
  Hostname: mail.libre.is
  Protocol: IMAP
  Port: 993
  Connection Security: SSL/TLS
  Authenication Method: Normal Password
  Username: username@libre.is (same as email address)
  
  Outgoing Server
  Hostname: mail.libre.is
  Port: 587
  Connection Security: STARTTLS
  Authenication Method: Normal Password
  Username: username@libre.is (same as email address)


Main Components
===============

This install is based on this guide:

  `<https://workaround.org/ispmail-bookworm/>`_

For more information and details about what is what,
refer to that site.

The main components in use:

Certbot (Let's Encrypt)

  `<https://certbot.eff.org/>`_

Debian

  `<https://debian.org/>`_

Dovecot

  `<https://dovecot.org/>`_


MariaDB

  `<https://mariadb.org/>`_

Postfix

  `<https://www.postfix.org/>`_

redis

  `<https://redis.io/>`_

rspamd

  `<https://rspamd.com/>`_

unbound

  `<https://www.unbound.net/>`_


DNS
===
Add a DNS mx record, so it returns result like this:

.. code-block:: sh

  $ host -t mx libre.is
  libre.is mail is handled by 10 mail.libre.is.

Set IP for mail.libre.is.
Set up reverse DNS records.


Debian
======
Install Debian stable (bookworm).
Install rsyslog for old school convenience:

.. code-block:: sh

  sudo apt install rsyslog


Firewall
========
Open TCP ports.

.. code-block:: sh

  # Web
  -A INPUT -p tcp --dport 80 -j ACCEPT
  -A INPUT -p tcp --dport 443 -j ACCEPT
  
  # Postfix
  -A INPUT -p tcp --dport 25 -j ACCEPT
  -A INPUT -p tcp --dport 587 -j ACCEPT
  
  # Dovecot imaps
  -A INPUT -p tcp --dport 993 -j ACCEPT
  
  # Dovecot sieve
  -A INPUT -p tcp --dport 4190 -j ACCEPT


Apache
======
The Apache webserver is used out of laziness as it allows easy
certificate updates with certbot. A webmail server won't be
running on the main mail server.

.. code-block:: sh

  sudo apt install apache2
  echo "mail.libre.is" | sudo tee /var/www/html/index.html

Open up firewall ports 80 and 443.


MariaDB
=======
The main database server.

.. code-block:: sh

  sudo apt install mariadb-server
  sudo mariadb-admin password
  mariadb -uroot -p

Add databases.
Change password to something secure.

.. code-block:: sql

  CREATE DATABASE mailserver;

  GRANT ALL ON mailserver.* TO 'mailadmin'@'localhost' IDENTIFIED BY 'password';

  GRANT SELECT ON mailserver.* TO 'mailserver'@'127.0.0.1' IDENTIFIED BY 'password';

  USE mailserver;

  CREATE TABLE IF NOT EXISTS `virtual_domains` (
   `id` int(11) NOT NULL auto_increment,
   `name` varchar(50) NOT NULL,
   PRIMARY KEY (`id`)
   ) ENGINE=InnoDB DEFAULT CHARSET=utf8;

  CREATE TABLE IF NOT EXISTS `virtual_users` (
   `id` int(11) NOT NULL auto_increment,
   `domain_id` int(11) NOT NULL,
   `email` varchar(100) NOT NULL,
   `password` varchar(150) NOT NULL,
   `quota` bigint(11) NOT NULL DEFAULT 0,
   PRIMARY KEY (`id`),
   UNIQUE KEY `email` (`email`),
   FOREIGN KEY (domain_id) REFERENCES virtual_domains(id) ON DELETE CASCADE
   ) ENGINE=InnoDB DEFAULT CHARSET=utf8;

  CREATE TABLE IF NOT EXISTS `virtual_aliases` (
   `id` int(11) NOT NULL auto_increment,
   `domain_id` int(11) NOT NULL,
   `source` varchar(100) NOT NULL,
   `destination` varchar(100) NOT NULL,
   PRIMARY KEY (`id`),
   FOREIGN KEY (domain_id) REFERENCES virtual_domains(id) ON DELETE CASCADE
   ) ENGINE=InnoDB DEFAULT CHARSET=utf8;

  EXIT


Postfix
=======
The main SMTP mail server.

.. code-block:: sh

  sudo apt install postfix
  sudo apt install postfix-mysql

Set up postfix to use MariaDB.
Edit /etc/postfix/mysql-virtual-mailbox-domains.cf
and add below, using the mailserver password used in MariaDB.

.. code-block:: cfg

  user = mailserver
  password = password
  hosts = 127.0.0.1
  dbname = mailserver
  query = SELECT 1 FROM virtual_domains WHERE name='%s'


Edit /etc/postfix/mysql-virtual-mailbox-maps.cf and add below contents:

.. code-block:: cfg

  user = mailserver
  password = password
  hosts = 127.0.0.1
  dbname = mailserver
  query = SELECT 1 FROM virtual_users WHERE email='%s'

Edit /etc/postfix/mysql-virtual-alias-maps.cf and add below:

.. code-block:: cfg

  user = mailserver
  password = password
  hosts = 127.0.0.1
  dbname = mailserver
  query = SELECT destination FROM virtual_aliases WHERE source='%s'

Edit /etc/postfix/mysql-email2email.cf and add:

.. code-block:: cfg

  user = mailserver
  password = password
  hosts = 127.0.0.1
  dbname = mailserver
  query = SELECT email FROM virtual_users WHERE email='%s'

Then run these commands:

.. code-block:: sh

  sudo postconf \
    virtual_mailbox_domains=mysql:/etc/postfix/mysql-virtual-mailbox-domains.cf
  sudo postconf \
    virtual_mailbox_maps=mysql:/etc/postfix/mysql-virtual-mailbox-maps.cf
  sudo postconf \
    virtual_alias_maps=mysql:/etc/postfix/mysql-virtual-alias-maps.cf
  sudo postconf \
    virtual_alias_maps=mysql:/etc/postfix/mysql-virtual-alias-maps.cf,mysql:/etc/postfix/mysql-email2email.cf
  sudo chgrp postfix /etc/postfix/mysql-*.cf
  sudo chmod 640 /etc/postfix/mysql-*.cf


Redis
=====
Note, the licensing of Redis has gone bad. The version in Debian
is OK. But in the future, probably replace with a fork.

.. code-block:: sh

  sudo apt install redis-server


rspamd
======
Spam control.

.. code-block:: sh

  sudo apt install rspamd


Certbot
=======
Encryption certificates with Let's Encrypt.
Not using an Apache webserver on the mail server makes getting
new certificates a bit more complex.

.. code-block:: sh

  sudo apt install certbot ca-certificates python3-certbot-apache
  sudo certbot -d mail.libre.is
  sudo systemctl restart apache2
  echo "post-hook = systemctl restart postfix dovecot apache2" | \
    sudo tee /etc/letsencrypt/cli.ini


Dovecot
=======
Just using encrypted IMAPS, not POP.

.. code-block:: sh

  sudo apt install dovecot-mysql dovecot-pop3d dovecot-imapd \
    dovecot-managesieved dovecot-lmtpd

Note, since IPv6 isn't being used, the dovecot install barfs.
Edit /etc/dovecot/dovecot.conf and add this line, where appropriate:

.. code-block:: sh

  listen = *

Note, this is removing the "::" from listen, which using IPv6.
Then re-run the install so the packages are happy. Note, the re-install
won't overwrite the "listen" change.

.. code-block:: sh

  sudo apt install --reinstall dovecot-mysql dovecot-pop3d dovecot-imapd \
    dovecot-managesieved dovecot-lmtpd

Add user and set up configs

.. code-block:: sh

  sudo groupadd -g 5000 vmail
  sudo useradd -g vmail -u 5000 vmail -d /var/vmail -m
  sudo chown -R vmail:vmail /var/vmail

  sudo sed -i -e \
    's/auth_mechanisms = plain/auth_mechanisms = plain login/g' \
    /etc/dovecot/conf.d/10-auth.conf

  sudo sed -i -e \
    's/!include auth-system.conf.ext/#!include auth-system.conf.ext/g' \
    /etc/dovecot/conf.d/10-auth.conf

  sudo sed -i -e \
    's/#!include auth-sql.conf.ext/!include auth-sql.conf.ext/g' \
    /etc/dovecot/conf.d/10-auth.conf

  sudo sed -i -e \
    's/^mail_location.*/mail_location = maildir:~\/Maildir/g' \
    /etc/dovecot/conf.d/10-mail.conf

  sudo sed -i -e \
    's/#mail_plugins =/mail_plugins = quota/g' \
    /etc/dovecot/conf.d/10-mail.conf
  
Edit /etc/dovecot/conf.d/10-master.conf and add:

.. code-block:: cfg

  # Postfix smtp-auth
  unix_listener /var/spool/postfix/private/auth {
    mode = 0660
    user = postfix
    group = postfix
  }

Edit /etc/dovecot/conf.d/10-ssl.conf, set key locations, and make it
required.

.. code-block:: cfg

  ssl = required
  ssl_cert = </etc/letsencrypt/live/mail.libre.is/fullchain.pem
  ssl_key = </etc/letsencrypt/live/mail.libre.is/privkey.pem

Edit /etc/dovecot/dovecot-sql.conf.ext file and add these lines at
the bottom, changing the password to the mailserver database password.

.. code-block:: cfg

  driver = mysql
  
  connect = \
    host=127.0.0.1 \
    dbname=mailserver \
    user=mailserver \
    password=password
  
  user_query = SELECT email as user, \
    concat('*:bytes=', quota) AS quota_rule, \
    '/var/vmail/%d/%n' AS home, \
    5000 AS uid, 5000 AS gid \
    FROM virtual_users WHERE email='%u'
  
  password_query = SELECT password FROM virtual_users WHERE email='%u'
  
  iterate_query = SELECT email AS user FROM virtual_users

Set file permissions.

.. code-block:: sh

  sudo chown root:root /etc/dovecot/dovecot-sql.conf.ext
  sudo chmod 600 /etc/dovecot/dovecot-sql.conf.ext

Edit /etc/dovecot/conf.d/10-master.conf and change to:

.. code-block:: cfg

  service lmtp {
    unix_listener /var/spool/postfix/private/dovecot-lmtp {
      group = postfix
      mode = 0600
      user = postfix
    }
  }


Restart dovecot server.

.. code-block:: sh

  sudo systemctl restart dovecot

Run this to tell postfix to deliver to dovecot:

.. code-block:: sh

  sudo postconf virtual_transport=lmtp:unix:private/dovecot-lmtp

Edit /etc/dovecot/conf.d/20-lmtp.conf and change line like this:

.. code-block:: cfg

  mail_plugins = $mail_plugins sieve


Restart dovecot again....

.. code-block:: sh

  sudo systemctl restart dovecot


More postfix
============
More postfix configuration, now that the above is set up.

Set postfix to use dovecot for authentication:

.. code-block:: sh

  sudo postconf smtpd_sasl_type=dovecot
  sudo postconf smtpd_sasl_path=private/auth
  sudo postconf smtpd_sasl_auth_enable=yes

  sudo postconf smtpd_tls_security_level=may
  sudo postconf smtpd_tls_auth_only=yes
  sudo postconf smtpd_tls_cert_file=/etc/letsencrypt/live/mail.libre.is/fullchain.pem
  sudo postconf smtpd_tls_key_file=/etc/letsencrypt/live/mail.libre.is/privkey.pem
  sudo postconf smtp_tls_security_level=may

Edit /etc/postfix/master.cf and change thusly:

.. code-block:: cfg

  submission inet n       -       y       -       -       smtpd
    -o syslog_name=postfix/submission
    -o smtpd_tls_security_level=encrypt
    -o smtpd_sasl_auth_enable=yes
    -o smtpd_tls_auth_only=yes
    -o smtpd_reject_unlisted_recipient=no
    -o smtpd_client_restrictions=
    -o smtpd_helo_restrictions=
    -o smtpd_sender_restrictions=
    -o smtpd_relay_restrictions=
    -o smtpd_recipient_restrictions=permit_sasl_authenticated,reject
    -o milter_macro_daemon_name=ORIGINATING

Run:

.. code-block:: sh

  sudo postconf smtpd_sender_login_maps=mysql:/etc/postfix/mysql-email2email.cf

Restart postfix:

.. code-block:: sh

  sudo systemctl restart postfix

Does it ever end? Edit /etc/postfix/master.cf and add to bottom of submission
section.

.. code-block:: cfg

   -o smtpd_sender_restrictions=reject_sender_login_mismatch,permit_sasl_authenticated,reject


.. code-block:: sh

  sudo systemctl restart postfix

Allow aliases to send by adding this file (XXX check OK)
/etc/postfix/aliases.cf with this contents:

.. code-block:: sql

  SELECT email FROM virtual_users WHERE email='%s' UNION SELECT destination FROM virtual_aliases WHERE source='%s'


Make sure all is good:

.. code-block:: sh

  sudo postfix check


rspamd Configuration
====================
Configure postfix for rspamd.

.. code-block:: sh

  sudo postconf smtpd_milters=inet:127.0.0.1:11332
  sudo postconf non_smtpd_milters=inet:127.0.0.1:11332
  sudo postconf milter_mail_macros="i {mail_addr} {client_addr} {client_name} {auth_authen}"

Edit /etc/rspamd/override.d/milter_headers.conf and add:

.. code-block:: cfg

  extended_spam_headers = true;

Edit /etc/dovecot/conf.d/90-sieve.conf and change:

.. code-block:: cfg

  sieve_after = /etc/dovecot/sieve-after

Create dir for new sieve filter:

.. code-block:: sh

  sudo mkdir /etc/dovecot/sieve-after

Create /etc/dovecot/sieve-after/spam-to-folder.sieve with these contents:

.. code-block:: cfg

  require ["fileinto"];
  
  if header :contains "X-Spam" "Yes" {
   fileinto "Junk";
   stop;
  }

Then compile it:

.. code-block:: sh

  sudo sievec /etc/dovecot/sieve-after/spam-to-folder.sieve

Set up redis by adding /etc/rspamd/override.d/redis.conf with this:

.. code-block:: cfg

  servers = "127.0.0.1";

Add this /etc/rspamd/override.d/classifier-bayes.conf with below contents:

.. code-block:: cfg

  autolearn = [-5, 10];

Add /etc/rspamd/local.d/classifier-bayes.conf with:

.. code-block:: cfg

  users_enabled = true;

Edit /etc/dovecot/conf.d/20-imap.conf and change:

.. code-block:: cfg

  mail_plugins = $mail_plugins quota imap_sieve

Edit /etc/dovecot/conf.d/90-sieve.conf and add below to "plugins" section:

.. code-block:: cfg

  # From elsewhere to Junk folder
  imapsieve_mailbox1_name = Junk
  imapsieve_mailbox1_causes = COPY
  imapsieve_mailbox1_before = file:/etc/dovecot/sieve/learn-spam.sieve
  
  # From Junk folder to elsewhere
  imapsieve_mailbox2_name = *
  imapsieve_mailbox2_from = Junk
  imapsieve_mailbox2_causes = COPY
  imapsieve_mailbox2_before = file:/etc/dovecot/sieve/learn-ham.sieve
  
  sieve_pipe_bin_dir = /etc/dovecot/sieve
  sieve_global_extensions = +vnd.dovecot.pipe
  sieve_plugins = sieve_imapsieve sieve_extprograms

Run:

.. code-block:: sh

  sudo mkdir /etc/dovecot/sieve

Create /etc/dovecot/sieve/learn-spam.sieve with contents:

.. code-block:: cfg

  require ["vnd.dovecot.pipe", "copy", "imapsieve"];
  pipe :copy "rspamd-learn-spam.sh";

Create /etc/dovecot/sieve/learn-ham.sieve and add:

.. code-block:: cfg

  require ["vnd.dovecot.pipe", "copy", "imapsieve", "environment", "variables"];
  if environment :matches "imap.mailbox" "*" {
    set "mailbox" "${1}";
  }
  if string :matches "${mailbox}" ["*/Trash", "Trash"] {
    stop;
  }
  pipe :copy "rspamd-learn-ham.sh";


Run this to compile:

.. code-block:: sh

  sudo sievec /etc/dovecot/sieve/learn-spam.sieve
  sudo sievec /etc/dovecot/sieve/learn-ham.sieve
  sudo chmod 600 /etc/dovecot/sieve/learn-{spam,ham}.{sieve,svbin}
  sudo chown vmail:vmail /etc/dovecot/sieve/learn-{spam,ham}.{sieve,svbin}

Create /etc/dovecot/sieve/rspamd-learn-spam.sh with contents:

.. code-block:: cfg

  #!/bin/sh
  exec /usr/bin/rspamc learn_spam

Create /etc/dovecot/sieve/rspamd-learn-ham.sh with contents:

.. code-block:: cfg

  #!/bin/sh
  exec /usr/bin/rspamc learn_ham

Set ownership and permissions on scripts:

.. code-block:: sh

  sudo chmod 700 /etc/dovecot/sieve/rspamd-learn-{spam,ham}.sh
  sudo chown vmail:vmail /etc/dovecot/sieve/rspamd-learn-{spam,ham}.sh


Unbound
=======
For proper spam filtering with rspam, unbound DNS resolver should be used.

.. code-block:: sh

  sudo apt install unbound

Change /etc/resolv.conf to:

.. code-block:: cfg

  nameserver 127.0.0.1
  options trust-ad

Also add to /etc/rspamd/local.d/options.inc

.. code-block:: cfg

  dns {
    nameserver = ["127.0.0.1"];
  }


DKIM
====
Set up DNS for DKIM.

.. code-block:: sh

  sudo apt install dnsutils
  sudo mkdir /var/lib/rspamd/dkim
  sudo chown _rspamd:_rspamd /var/lib/rspamd/dkim
  sudo rspamadm dkim_keygen -d libre.is -s 2024090101

Add a 2024090101._domainkey TXT DNS record at the ISP, with contents of the "p="
and the rest, for example:

.. code-block:: cfg

  p=MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQCxenHupkYLPmFMbJjV9dQICKUl2xH/aexSRUwCuw7TJ9dkddqIN+6tyw4VKhnW8R0/UlbzlSFLmVgMU0uUkwTtVqyDHhtSU7LV/SkVYmUst4dTUF1r+8PvhAm7vobMYKdwvRsOq27ABtZc8P4oU2XXHqqa6LU8s4sNxs12hLW9swIDAQAB

Create /etc/rspamd/local.d/dkim_signing.conf with contents:

.. code-block:: cfg

  path = "/var/lib/rspamd/dkim/$domain.$selector.key";
  selector_map = "/etc/rspamd/dkim_selectors.map";

Create /etc/rspamd/dkim_selectors.map with contents:

.. code-block:: cfg

  libre.is 2024090101

Restart again...

.. code-block:: sh

  sudo systemctl restart rspamd

Take teh contents from running dkim_keygen above and add it this file:
/var/lib/rspamd/dkim/libre.is.2024090101.key

Just add the PRIVATE KEY section, not the last two lines.
Fix it's permissions:

.. code-block:: sh

  sudo chown _rspamd /var/lib/rspamd/dkim/libre.is.2024090101.key
  sudo chmod 400 /var/lib/rspamd/dkim/libre.is.2024090101.key


SPF
===
Set up SPF.

Add a DNS TXT record like this:

.. code-block:: cfg

  v=spf1 mx a ip4:70.39.110.156/32 include:mail.libre.is -all


DMARC
=====
Create a DNS TXT record for the domain _dmarc.libre.is with contents:

.. code-block:: cfg

  v=DMARC1; p=reject; rua=mailto:postmaster@libre.is; ruf=mailto:postmaster@libre.is; fo=0; adkim=r; aspf=r; pct=100; rf=afrf; ri=86400; sp=reject


Python Admin
============
Admin with Python script.

.. code-block:: sh

  sudo apt install python3-mysqldb python3-bcrypt 
  git clone https://github.com/cgzones/ispmail-userctl
  cd ispmail-userctl/

Edit ispmail_userctl.py and change:

* USE_BCRYPT: bool = True

Under DB_CONNECTION, change user and password to match database setup:

* user='mailadmin'
* password=''

Then move the script in place.

.. code-block:: sh

  sudo cp -p ispmail_userctl.py /usr/local/sbin/ispmail_userctl
  sudo chown root:root /usr/local/sbin/ispmail_userctl
  sudo chmod 700 /usr/local/sbin/ispmail_userctl

Then just run it normally as root.

.. code-block:: sh

  sudo ispmail_userctl


Spamhaus
========
Spamhaus may automagically list the IP to be blocked when it is new.
To remove, just go to:

  `<https://www.spamhaus.org/returnc/pub/>`_

Then enter the IP address of the server and fill out the form.
They will send a confirmation email with a link.
Go to that link and it will be immediately removed.

Abuse
=====
Set up an email alias for abuse@libre.is.

Follow instructions on this site to add the abuse email to their list:

  `<https://abuse.net/addnew.phtml>`_