drmaxnix/moosync

By drmaxnix

Updated 11 days ago

Automatically Sync Mailcow Addresses to Failover Instances

Image
Networking
Integration & delivery
Databases & storage
0

909

drmaxnix/moosync repository overview

Source Code: git.tjdev.de/DrMaxNix/moosync

MooSync

Automatically Sync Mailcow Addresses to Failover Instances

The Problem

  1. You have two (or more) Mailcow servers: a main server and one (or more) failover server(s).
  2. The main server has priority over the failover server(s); eg. MX 10. The failover server(s) has/have lower priority; eg. MX 20.
  3. In normal operation, all mails will go through your main server, if the main server is unavailable, your mail will be delivered to one of your failover servers instead.
  4. Mailcow supports this out of the box (using Relay this domain in domain settings). But the native failover support has a catch: the failover server does not know, which mail addresses exist, and which don't. So you have two choices; either accept all mail and then send a "delivery failed" notice after the main server is up again (will possibly keep the sender unaware for days) or painstakingly mirror all domains, mailboxes, aliases, etc. as blind mailboxes to your failover server(s) so you can disable Relay all recipients.
  5. MooSync utilizes the second approach, but completely automating the mirroring process.

How It Works

  1. MooSync gets a list of active domains and addresses from the main ("master") system. Addresses are collected from various sources, such as mailboxes, aliases, alias domains and catchalls.
  2. Every failover ("slave") system will then be aligned with these collected settings.
  3. Domains are added as normal domains, addresses however use recipient maps that map to itself, which greatly improves performance, as this pretty much just tells Mailcow that there IS an address, but doesn't actually change anything about the mails itself (compared to stuff like aliases).

Setup Instructions

For the following instructions, we will assume following:

  • The MooSync container is running on srv0.example.com (IP address: 203.0.113.1)
  • Your master system is on mx0.example.com
  • Your slave system is on mx1.example.com (and optionally another one at mx2.example.com, etc.)
1. Set up your master system
  1. Allow DKIM private key access via the api. To do this, add $SHOW_DKIM_PRIV_KEYS = true; to data/web/inc/vars.local.inc.php. Restarting is not required.
  2. In the Admin UI, under System > Configuration > Access > Administrators, enable the read-only api.
  3. On the same page, add the ip address of srv0 to the list of allowed networks (eg. 203.0.113.1). Make sure to save changes.
  4. On the same page, if you don't already have an api key in use, generate one and keep it for later.
  5. In the Admin UI, under System > Configuration > Options > Forwarding Hosts, add mx1.example.com (and all other failover systems) as forwarding hosts. Keeping the spam filter enabled will work in most setups.
2. Set up your slave system(s)
  1. In the Admin UI, under System > Configuration > Access > Administrators, enable the read-write api.
  2. On the same page, add the ip address of srv0 to the list of allowed networks (eg. 203.0.113.1). Make sure to save changes.
  3. On the same page, if you don't already have an api key in use, generate one and keep it for later.
  4. In the Admin UI, under System > Configuration > Routing, under Transport Maps, add a new transport to deliver all mails via your master system (destination: *, next hop: mx0.example.com, username/password can be empty, Destination is a regular expression: no, active: yes). This is optional, but will prevent cases, where the mail will cycle between failover servers if there are multiple.
  5. In data/conf/postfix/extra.cf, you can add bounce_queue_lifetime = 5d (or 3d) to override the default of 1d, which may be a bit short for some use cases. This duration controls how long the failover server will try to forward its mails to the master server, before it gives up and answers with a "message could not be delivered" mail.
3. Set up MooSync

MooSync can run on any server, also one that is not any of your mail servers. As MooSync can only do its job of syncing data from the master system, it is useless when the master system is down, so you can also just put it on the master system. You can use this docker-compose.yml as a good starting point:

services:
  moosync:
    image: drmaxnix/moosync
    restart: unless-stopped
    environment:
      SYNC_DESTRUCTIVE: false # you probably want this enabled, but read about the consequences first (see section "Configuration Options")
      MASTER_ADDRESS: "mx0.example.com"
      MASTER_TOKEN: "<API KEY FOR MASTER SYSTEM>"
      SLAVE_ADDRESS_MX1: "mx1.example.com"
      SLAVE_TOKEN_MX1: "<API KEY FOR SLAVE SYSTEM MX1>"
      SLAVE_ADDRESS_MX2: "mx2.example.com"
      SLAVE_TOKEN_MX2: "<API KEY FOR SLAVE SYSTEM MX2>"
      # ...

Configuration Options

Following configuration options are available by setting environment variables:

OptionTypeDefaultDescription
MASTER_ADDRESS🔸host/ipAddress of the master mail system
SLAVE_ADDRESS_*🔸host/ipAddress of the slave mail system (make sure to use the same option name as for SLAVE_TOKEN_*)
MASTER_TOKEN🔸stringAPI token for master system's Mailcow API
SLAVE_TOKEN_*🔸stringAPI token for slave system's Mailcow API (make sure to use the same option name as for SLAVE_ADDRESS_*)
SYNC_INTERVALduration15mHow often domain and address configuration should be synced to slave system(s)
SYNC_INTERVAL_DKIMduration6hHow often dkim keys should be synced (every domain's key will be synced separately, so this can take some time on systems with many domains)
SYNC_DESTRUCTIVEboolfalseWhether MooSync should clean up everything on the slave system(s) it does not need or know about (⚠️⚠️ READ WARNING BELOW ⚠️⚠️)
API_TIMEOUTint+1000HTTP timeout for outgoing API requests to Mailcow (syncs will be aborted when a request ran into a timeout)
DOMAIN_WHITELIST_*domainemptyWhen set, only sync this domain (if whitelist is empty, all domains not in blacklist will be synced)
DOMAIN_BLACKLIST_*domainemptyDo not sync this domain (has precedence over whitelist)
DEBUGboolfalseWhether debug information should be visible in the docker log

Options marked with 🔸 are required; list options need at least one element

Options named FOO_* can be used multiple times with different value for *; all of its provided values will form a list

Warning

**Destructive Mode:** This mode grants MooSync full control over all slave systems! Don't enable this if you have other domains configured on these systems, as MooSync will happily delete them otherwise! By default (if destructive mode is disabled), MooSync will only add/modify/delete domains, dkim keys and recipient maps for domains allowed by white-/blacklist.

Explanation of types:

TypeDescriptionExample(s)
boolA boolean value, either true or falsetrue
stringOne or more character(s)TestTest 123
durationEither a number of seconds, or a number followed by a unit, which can be one of [smhd] for seconds, minutes, hours and days respectively15m = 15 minutes = 900 seconds
int+A positive integer42
host/ipA hostname, domain name, or IPv4/IPv6 addressapp
app.example.com
198.51.100.49
2001:db8::a17d:661e
domainA domain nameapp.example.com

Tag summary

Content type

Image

Digest

sha256:d68ff53b7

Size

9.2 MB

Last updated

11 days ago

docker pull drmaxnix/moosync