Lion Mail vs GMail

[abortrephrase]

There's a basic problem combining Lion's Mail with GMail: if you need to keep the web interface working in a sensible manner and you use the server-side filtering and labels, then you're kind of screwed.

Either you go with the defaults Mail sets up and you lose access to everything that isn't in your inbox, or you remove the IMAP prefix setting and you get doubles of everything.

There is a way around it, but it's probably not a great idea for the non-tech people out there. You need to use offlineimap and an IMAP server on your local machine.

I use Homebrew for random open sauce applications. If you want to treat this post as a recipe, I suggest installing that. It's dead easy, just make sure you have Xcode installed, then run the one-liner they provide in their install instructions.

To install offlineimap, run (as yourself) "brew install offlineimap". I'm using Dovecot as my IMAP server, to install that run "brew install dovecot".

You next need to configure Dovecot. Go into /usr/local/etc, copy dovecot-example.conf to dovecot.conf. I changed "protocols" to "imap", "ssl" to "no", "login_user" to "_dovecot" (as this already exists), and "mail_location" to "milder:~/Maildir".

Then go into /etc/pam.d and copy "login" to "dovecot" so Dovecot can authenticate you.

Finally, do the last two steps (3 and 4) you get from "brew info dovecot" to set up and enable the IMAP service.

At this point I suggest configuring Mail to talk to your local server. Give it your GMail address and your local password, it should fail to connect to GMail (you do use different passwords, yes?), just keep on going through with it failing and it will eventually give you a dialog where you can set the username to your local username and the host to "localhost". It should then connect and find nothing.

You can confirm it connected by looking in ~/Maildir, you should now see the usual stuff you'd find in an empty maildir, e.g., "cur", "new", and "tmp" directories.

If you haven't got this much working, you need to debug it and figure out why not before going any further!

Now you want to get offlineimap to suck mail from GMail and stick it in your local IMAP folders. Drop something not entirely unlike this in ~/.offlineimaprc:


[general]
accounts = GMail
ui = Noninteractive.Quiet

[Account GMail]
localrepository = LocalImap
remoterepository = Remote
autorefresh = 10
quick = 5

[Repository LocalImap]
type = IMAP
remotehost = localhost
remoteuser = me
remotepass = password
maxconnections = 2
ssl = no

[Repository Remote]
type = IMAP
ssl = yes
remotehost = imap.gmail.com
remoteuser = me@gmail.com
remotepass = gmailpassword
holdconnectionopen = yes
maxconections = 1

# pick one of the below:
# (for a single server)
nametrans = lambda foldername: re.sub('^INBOX', 'INBOX', foldername)
folderfilter = lambda foldername: not re.search('(^\[Google Mail\])', foldername)


Then do a first run to sync it all and make sure everything's happy by running "offlineimap -u TTY.TTYUI". This may well take a very long time if you've got a fair bit of mail. That's okay, just let it run and you'll be fine.

Once it finishes syncing for the first time, follow the instructions Homebrew gave you when it finished installing offlineimap to create a launchd item. This will make sure offlineimap starts up every time you log in.

One small quirk I noticed is that offlineimap gets the timestamps on some of the maildir files wrong and Mail uses that rather than the header contents to get the received date. You can work around this after Mail does its first sync by quitting Mail, then deleting the files "Envelope Index*" in ~/Library/Mail/V2/MailData. Restart Mail, it'll think it has to import your mailbox, which just makes it reindex the cache, and then you're golden!

Comments

[ideological_cuddle]

Also, I strongly recommend adding ~/Maildir to the "Privacy" section of the Spotlight config. No point indexing that stuff, you really only want the Mail version indexed as that's actually useful.

[ideological_cuddle]

Having left it run overnight, I see offlineimap+GMail still has the same problem it had on Linux where sometimes it'll get wedged and never move on.

If you install GNU coreutils ("brew install coreutils") the following little script saved in ~/bin will replace the launchd service:


#!/bin/sh

while [ 1 -eq 1 ]; do
/usr/local/bin/gtimeout -k 1m 10m /usr/local/bin/offlineimap -o -1
sleep 300 ;
done


in ~/bin/fetch.sh.

Then edit ~/Library/LaunchAgents/org.offlineimap.plist and change the ProgramArguments key to read "/Users/yourusername/bin/fetch.sh".

Remove then reload the service. "launchctl remove org.offlineimap", "launchctl load ~/Library/LaunchAgents/org.offlineimap.plist". It should now fetch mail every five minutes and kill itself off any time it takes too long to fetch.

[thorfinn]

Hrm, all my GMail filters have "skip inbox" in them, so everything works fine for me?

Even without that, Lion's Mail.app seems to do auto-de-duplication which seems to just work? Even works cross account - which is great for me, as I have mail being stored on my own domain as well having a copy forwarded to gmail. Seems to do the right thing with "Flagged" as well as unread marks.

[ideological_cuddle]

If you're using the default config, you won't see any of the labels. You'll still get the mail, of course, but it won't be sorted. That's fine if what you want is to have Mail do that for you.

I found that if I removed the IMAP prefix things got a bit wonky. Not only did it wind up with dupes, it also seemed to "lose" a lot of mail from the inbox. It may actually have been doing dedupe and winding up confused -- I have many years of mail in there, and it's my primary mail-store...

I really don't blame any of this stuff on Mail. GMail is a pretty shit IMAP server.

[thorfinn]

I see all the labels down in the account section (particularly after pointer-over-and-clicking the hidden show button to the right of the account label)?

That's provided the label is configured in GMail Settings to be "show in imap".

I never had an IMAP Path Prefix specified at all, though, so I'm not sure what's going on for you...