Today I migrated the last big part of my old server: the mail system. Since I and other people depend on this server for their day-to-day mailing, I had to switch over without losing a single e-mail. This is how I did it:
Step one: Port the configuration, make the environment run on the new server. Copy also user metadata like passwords and make sure the overall structure is working (incoming SMTP works, POP3 access, etc.)
Step two: Copy over all mails from the old server. The mails will be synced later on again, so this can happen some minutes in advance. (I actually lost a few days doing this, because I discovered unused mailboxes with 995,000 mails, >99% of them being spam. I had to ask the owner first, though, whether I could delete them.)
Now comes the time-critical path. It took me an overall 70 seconds to do steps three to five.
Step three: Stop the daemons that receive mail or give the user
access to it. For example: for s in postfix courier*; do
/etc/init.d/$s stop; done – connecting to the host will now
give a "connection refused" error message. MTAs trying to deliver mail
will usually try again ten minutes later. (So no mail gets lost.)
Step four: Sync the emails again. There might have arrived new
messages, or users have deleted some of their inbox. I used this
command: rsync -avhP --delete vmail@eris.feh.name: .
Step five: Apply iptables rules to forward connections to the new
server. This is due to the fact that DNS information is slow to
spread. For a few days I don't care whether mail.feh.name resolves
to 88.198.158.101 or to 176.9.247.89. Both will effectively talk to
the new server.
iptables -t nat -A PREROUTING -p tcp -s ! 176.9.34.52 --dport 25 \
-j DNAT --to-destination 176.9.34.52:25
iptables -t nat -A PREROUTING -p tcp -s ! 176.9.34.52 --dport 110 \
-j DNAT --to-destination 176.9.34.52:110
iptables -t nat -A POSTROUTING -d 176.9.34.52 -j MASQUERADE
iptables -A FORWARD -p tcp -d 176.9.34.52 -j ACCEPT
iptables -A FORWARD -p tcp -s 176.9.34.52 -j ACCEPT
So this establishes forwarding for SMTP and POP3. The old server will simply act as a NATing gateway to the new server.
Step six: Adjust the DNS. As said above, you can take your time for
this; but the information will eventually spread. To have an indicator of
how many connections still arrive at the old host, try iptables -t
nat -L -vn, it'll print packet and byte counts for each rule in the
NAT table.
Done! And just one minute of outage. *like*
I got a call yesterday – from Egypt. It was around noon local time, i.e. shortly after Egypt's government had restored regular internet access (see also, more graphs). At the time of the call, I didn't know that, though.
The guy asked me whether I could help him set up DNS tunneling. Today, he called again and inquired how things were going. He told me he wanted to have a backup internet connection if things would get messy again.
I think, however, that DNS tunneling wouldn't be able to bypass the internet block. On the day of the shutdown (Jan 27th, shortly before midnight UTC),
"Approximately 3,500 individual BGP routes were withdrawn, leaving no valid paths by which the rest of the world could continue to exchange Internet traffic with Egypt's service providers, ... Virtually all of Egypt's Internet addresses are now unreachable, worldwide."
For a more technical analysis, see this BGPmon blog post.
What this means is that Egypt was essentially split off the net. Thus, even a DNS query from inside Egypt (which would be relayed to a server in, say, Germany) could not be forwarded by the provider's DNS servers. Neither could a DNS request from Germany reach a name server located in Egypt. This blog post claims that the providers did shut down DNS access as well (which wouldn't matter much in that situation, anyway).
So, to evade this blockade (i.e., IP won't get routed) it takes more than just trying to sneak data out using covert IP channels.
There are other methods, of course. Protesters set up a voice to tweet converter:
"Over the weekend we came up with the idea of a speak-to-tweet service – the ability for anyone to tweet using just a voice connection," they said.
Voice mail messages left at +16504194196; +390662207294 or +97316199855 will instantly be converted into text messages, referred to as tweets, and posted at Twitter with an identifying "hashtag" of
#egypt.
You could use other radio infrastructure as well to communicate with other countries, which would then feed your data stream into the net. That's not possible on a large scale and for the typical user, however.