Using IMAP Email -- and Backing It Up!

Some (many?) email users still get their messages downloaded via POP3 protocol into Outlook Express or other such client. This usually means the messages are then deleted from the server. Now the only place to view those messages is on one computer. Want to read messages later, when you're somewhere else? Forget it. Your ISP-provided webmail shows an empty Inbox.

Yes, you can tell OE to keep the messages on the server. Now you have the same messages in two places. But if you delete 10 of them in OE, you will find those same 10 are still on the server. The next time you download, your email program may be smart enough to figure out that you deleted those 10 and delete them on the server.

But now you're away from home and you go to your webmail. Hey, those 10 "deleted" messages are still on the server! That's annoying! Especially if they're spam.

A lot of folks these days just use webmail only. That solves the messages-here-but-not-there problem. But it also means you can't enjoy the benefits of a real email program — speed, message sorting, and better configuration options.

You can have it all — with IMAP. By finding a provider with IMAP protocol (used instead of POP3), you can:

• have all your messages on the server in one and only one place,
• use online folders,
• use a real email program on one or more computers,
• access your email quickly on a smartphone, and
• use webmail.

There are some good IMAP providers out there. I have used Tuffmail for many years. It's fast and reliable. Great support via email. Superb server-based anti-spam. Cost varies depending on the the size of your mailstore (ie, your entire bag of messages on the server). And everything about delivery and sending is extremely configurable — but some advanced knowledge of email is required to make those optional adjustments.

Other IMAP providers are: AOL/AIM mail (free, fast, unlimited storage, image ads on the webmail page). Easy to use and lots of pretty themes.

And Google Mail (free, 7+GB storage, very fast, text ads on the webmail page). A rather different email interface, with threaded views (no way to have views by date, etc.).

Also check out Fastmail and GMX So far, Yahoo Mail and Windows Live Mail do not have an IMAP option.

Do a web search to find out how to set up OE, Thunderbird, and other apps to connect to an IMAP server.

BTW, your outgoing email will still be handled by your ISP, so there's usually no need to change the SMTP settings. However, all of the above providers have SMTP service available.

***

Now the question comes up: "I've been using POP3 all these years and I've got my all my email in local folders. How can I get them into IMAP?"

The simplest way is to simply create the same folders in your server-based IMAP account. Then, using your local email program, copy the messages from each local folder to the corresponding IMAP folder. You can use Shift + left-click to drag-copy the messages. But it's easier and safer to use one of the menu commands, like Message | Copy ... in Thunderbird. If you are on an asymmetric DSL or cable connection, where the upload speed is a lot slower than download, then the copying will take a while.

There are some other options, like using IMAPSize (Windows only, freeware) to copy an entire local mailstore to an IMAP server.

***

Okay, so you've been using IMAP for a year and have 200 or 300 MB of messages. You would like to back those up somewhere, because, who knows, you (or a hacker) could somehow delete all your messages.   Or there is a disaster or mistake and your IMAP provider loses everything.

You can't just say:

zip aim.bk.zip imap://mail.aim.com/myusername

In an IMAP mailstore, each message consists of the actual message text and a bunch of metadata, such as sender, unique ID, date received, size, IMAP flags (the little flag or star that you can click in your email program to mark a message), and keywords (important, personal, etc.). It's a complex blob of stuff to download (and later upload).

One fairly easy way to do a backup is with the aforementioned IMAPSize. This excellent app can be used to download a whole mailstore to your computer, where each message will be in its own file, organized by your folder names. The metadata is captured in the file name and in a small data file. IMAPSize will also upload those messages to an IMAP server and restore the metadata.

What if you're on Linux?   You can have some fun with these two apps:

• Courier IMAP server installed on your PC
• imapsync

Install Courier using Synaptic or other such means. You should install courier-imap-ssl and courier-ssl as well. There are other IMAP servers to choose from, such as UWimap, dovecot, and Cyrus. But Courier is fairly simple (compared to Cyrus) and has the nice benefit of allowing a folder to contain both messages and folders (unlike UW and dovecot).

Then create a local mailstore directory (maildir) for yourself with this command (as you, not root):

cd

maildirmake Maildir

Set up an account in your email program (OE, TB, etc.) to access this new (but currently empty) local mailstore — use your Unix username and "localhost" for the IMAP server name.

Now download the latest imapsync tarball. In it you will find the excellent imapsync command, which is a perl script by Gilles Lamiral that copies an IMAP mailstore (or individual folders) to another IMAP server. You can just use the imapsync command without running make install. The command has many options, so build up a script file to hold them, such as:

perl imapsync \

—authmech1 LOGIN \

—authmech2 LOGIN \

—noauthmd5    \

—host1 imap.aim.com \

—user1 joe@aim.com \

—password1 mypassword \

—host2 localhost \

—user2 joe \

—password2 mylocalpw \

—ssl1 \

—syncinternaldates \

—subscribe \

—exclude 'Spam|Trash|Drafts' \

—expunge

The "1" settings refer to the source server. The "2" settings are the destination. Save this as imapbkup.sh. Then run:

sh imapbkup.sh

(You could make imapsync and/or the script executable with chmod, but having to use sh to run it means you can't do accidentally).

If all goes well, the messages from your Internet IMAP server will be quickly downloaded into your local mailstore. Yay!

imapsync is pretty smart, it won't download the same message again. So you can restart it and only get new messages. This means you can run it every few weeks or months and have an fast backup. But it won't delete any messages (at least not with the above imapsync options), so if you delete messages on the Internet server, they will still be in your local mailstore, if even after you re-run imapsync. That's usually not a problem.

***

Now the worst happens and your Internet IMAP server has one of those "should never happen" crashes. You can immediately access your backup in your local folders.

To upload the backup to a new Internet IMAP mailstore, just reverse the "1" and "2" settings above to produce an uploading copy:

perl imapsync \

—authmech1 LOGIN \

—authmech2 LOGIN \

—noauthmd5    \

—host2 imap.aim.com \

—user2 joe@aim.com \

—password2 mypassword \

—host1 localhost \

—user1 joe \

—password1 mylocalpw \

—ssl2 \

—syncinternaldates \

—subscribe \

—exclude 'Spam|Trash|Drafts' \

—expunge

Yet another option is to copy one Internet IMAP mailstore to another Internet IMAP mailstore. You may have one of those web-hosting accounts where you get 2,000 IMAP/POP3 accounts included. Set up one of those as your backup IMAP mailstore. Then use imapsync thusly:

perl imapsync \

—authmech1 LOGIN \

—authmech2 LOGIN \

—noauthmd5    \

—host1 imap.aim.com \

—user1 joe@aim.com \

—password1 mypassword \

—host2 imap.megahost.com \

—user2 joe@megahost.com \

—password2 mymegapw \

—ssl1 \

—ssl2 \

—syncinternaldates \

—subscribe \

—exclude 'Spam|Trash|Drafts' \

—expunge

This is the slowest of all, since it's downloading to a local temporary file, then uploading to another server. But the next time you run it, it will only get new messages and go faster.