Today I migrated the last big part of my old server: the mail system. Since I and other people depend on this server for their day-to-day mailing, I had to switch over without losing a single e-mail. This is how I did it:
Step one: Port the configuration, make the environment run on the new server. Copy also user metadata like passwords and make sure the overall structure is working (incoming SMTP works, POP3 access, etc.)
Step two: Copy over all mails from the old server. The mails will be synced later on again, so this can happen some minutes in advance. (I actually lost a few days doing this, because I discovered unused mailboxes with 995,000 mails, >99% of them being spam. I had to ask the owner first, though, whether I could delete them.)
Now comes the time-critical path. It took me an overall 70 seconds to do steps three to five.
Step three: Stop the daemons that receive mail or give the user
access to it. For example: for s in postfix courier*; do
/etc/init.d/$s stop; done – connecting to the host will now
give a "connection refused" error message. MTAs trying to deliver mail
will usually try again ten minutes later. (So no mail gets lost.)
Step four: Sync the emails again. There might have arrived new
messages, or users have deleted some of their inbox. I used this
command: rsync -avhP --delete vmail@eris.feh.name: .
Step five: Apply iptables rules to forward connections to the new
server. This is due to the fact that DNS information is slow to
spread. For a few days I don't care whether mail.feh.name resolves
to 88.198.158.101 or to 176.9.247.89. Both will effectively talk to
the new server.
iptables -t nat -A PREROUTING -p tcp -s ! 176.9.34.52 --dport 25 \
-j DNAT --to-destination 176.9.34.52:25
iptables -t nat -A PREROUTING -p tcp -s ! 176.9.34.52 --dport 110 \
-j DNAT --to-destination 176.9.34.52:110
iptables -t nat -A POSTROUTING -d 176.9.34.52 -j MASQUERADE
iptables -A FORWARD -p tcp -d 176.9.34.52 -j ACCEPT
iptables -A FORWARD -p tcp -s 176.9.34.52 -j ACCEPT
So this establishes forwarding for SMTP and POP3. The old server will simply act as a NATing gateway to the new server.
Step six: Adjust the DNS. As said above, you can take your time for
this; but the information will eventually spread. To have an indicator of
how many connections still arrive at the old host, try iptables -t
nat -L -vn, it'll print packet and byte counts for each rule in the
NAT table.
Done! And just one minute of outage. *like*