As many of you, I have a couple years of experience of using Internet and as such I have created different email accounts on different websites : not because I needed it, but mostly because each provider came out with a great benefit compared to the others. Funny thing, the last email account I have is the one from my Internet service provider which doesn’t propose a webmail. Buuuuuuuuuu. You must admit it is not very convenient when you are away from home or work. So here is my need for a solution to retrieve all my emails from Yahoo, Gmail, Free, etc. and serve them with IMAP to either Thunderbird or a Webmail.

Before going into the details of configuring each services, few things to know about my current setup :

• I run Debian Sarge;
• Postfix is running as a relay host to my ISP SMTP server;
• OpenLDAP stores the addressbook and my user account,
• Apache, MySQL, PHP and its extensions (ldap, mysql) are already installed.

Fetchmail

Install fetchmail :

# apt-get install fetchmail

Edit /etc/default/fetchmail to start it as a daemon :

START_DAEMON=yes

Create /etc/fetchmailrc :

poll pop.example.com with proto pop3 
  user 'john.doe' there with password 'passwd' is 'john' here 
  keep mda 'sudo /usr/bin/procmail -m /etc/procmailrc'
poll pop.gmail.com with proto pop3 port 995 
  user 'john.doe' there with password 'password' is 'john' here 
  ssl keep mda 'sudo /usr/bin/procmail -m /etc/procmailrc'

The reason I use sudo to execute procmail is that the fetchmail daemon runs as user fetchmail, and as the fetchmail user I wouldn’t be able to deliver the email.

Edit /etc/sudoers to add this line at the end :

fetchmail	ALL=NOPASSWD: /usr/bin/procmail

Start fetchmail :

# invoke-rc.d fetchmail restart

Procmail

Install procmail :

# apt-get install procmail

Create /etc/procmailrc :

SHELL="/bin/sh"
DELIVERMAIL="/usr/sbin/cyrdeliver"
USER="john"
IMAP="$DELIVERMAIL -a $USER -m user/$USER"
LOGFILE="/var/log/mail/$USER.log"
LOGABSTRACT="all"
VERBOSE=YES

:0
|$IMAP

This is a quick and dirty version of /etc/procmailrc. Improvements have to be made to turn it system-wide (for all the users) and add some filtering and anti-spam rules.

Cyrus + SASL

Install Cyrus :

# apt-get install cyrus-imapd-2.2 cyrus-admin-2.2 cyrus-clients-2.2 sasl2-bin

Edit /etc/imapd.conf :

configdirectory: /var/lib/cyrus
partition-default: /var/spool/cyrus
admins: cyrus administrator
sievedir: /var/lib/cyrus/sieve
sendmail: /usr/sbin/sendmail
hashimapspool: true
lmtpsocket: {configdirectory}/socket/lmtp
sasl_pwcheck_method: saslauthd
sasl_mech_list: PLAIN
unixhierarchysep: yes

The user cyrus is a /etc/passwd user. administrator is an LDAP user. lmtpsocket is the same path as the one specified in /etc/cyrus.conf :

lmtpunix	cmd="lmtpd" listen="/var/lib/cyrus/socket/lmtp" prefork=0 maxchild=20

Disable POP3 server :
In /etc/cyrus.conf and the SERVICES section, comment the pop3 line as follows :

# pop3		cmd="pop3d -U 30" listen="pop3" prefork=0 maxchild=50

Create /etc/saslauthd.conf with :

ldap_servers: ldap://127.0.0.1/
ldap_search_base: dc=example,dc=com

In /etc/default/saslauthd, set the authentication mechanism :

MECHANISMS="ldap"

Start saslauthd :

# invoke-rc.d saslauthd start

Test your ldap connection with :

# testsaslauthd -u john -p password
0: OK "Success."

Start cyrus :

# invoke-rc.d cyrus start

We now have to create mailboxes and give users their rights. We have to log in Cyrus with an LDAP account set as Admins in the cyrus config file (administrator). Once in the cyrus prompt, type help to get the list of command and their arguments :

# su - administrator
$ cyradm localhost
IMAP Password: password
localhost.localdomain> cm user/john
localhost.localdomain> lm
user/john (\HasNoChildren)
localhost.localdomain> sam user/john john all
localhost.localdomain> lam user/john
john lrswipcda
localhost.localdomain> exit
$

I didn’t need it but you can also fix your quota at this step.

Now emails should be ending up in /var/spool/cyrus/j/user/john/ in file named 1. 2. 3. etc.

RoundCube

Install roundcube :

# apt-get install roundcube

Edit /etc/roundcube/apache.conf to uncomment :

Alias /roundcube /var/lib/roundcube

Reload Apache :

# invoke-rc.d apache2 reload

Point your browser to http://localhost/roundcube to make sure you can access Roundcube login page.

Edit /etc/roundcube/main.inc.php and change this option to connect RoundCube to your IMAP server (authentication and emails) :

$rcmail_config['default_host'] = "imap://localhost:143";

Edit /etc/roundcube/main.inc.php and change those options to connect RoundCube to your LDAP server (addressbook) :

$rcmail_config['ldap_public']['example.com'] = array(
'name'          => 'example.com',
'hosts'         => array('ldap.example.com'),
'port'          => 389,
'base_dn'       => 'ou=addressbook,dc=example,dc=com',
'bind_dn'       => '',
'bind_pass'     => '',
'ldap_version'  => '3',       // using LDAPv3
'search_fields' => array('mail', 'cn'),  // fields to search in
'name_field'    => 'cn',    // this field represents the contact's name
'email_field'   => 'mail',  // this field represents the contact's e-mail
'surname_field' => 'sn',    // this field represents the contact's last name
'firstname_field' => 'givenName',  // this field represents the contact's first name
'scope'         => 'sub',   // search mode: sub|base|list
'filter'        => '',      // used for basic listing (if not empty) and will be &'d with search queries. ex: (status=act)
'fuzzy_search'  => true);   // server allows wildcard search

Now try logging in to RoundCube with john/password and click on Addressbook. “example.com” should appear in the Groups list and with the search box, you should be able to get your LDAP entries.

Hotmail/Live account

When I installed GetLIve, it wasn’t available on Debian repositories, so I have downloaded it from Sourceforge: Download GetLive.

Decompress the archives :

tar zxf GetLive_0_56.tgz

Organize the differents files :

mv GetLive_0_56/*.pl /usr/local/bin/.
mv GetLive_0_56/Manual /usr/local/doc
mv GetLive_0_56.tgz /usr/local/src/.

Create a repository to store the list of downloaded email ID :

mkdir /var/lib/getlive

Create /etc/getliverc :

UserName        = jdoe
Password        = password
Downloaded      = /var/lib/getlive/downloaded.data
Processor       = grep -v 'From ' | /usr/bin/procmail -m /etc/procmailrc
Folder          = INBOX
MarkRead        = YES

The reason of the grep is explained in the Troubleshooting section below.

Add a crontab entry to execute GetLive at 7am, noon and 6pm, for example :

# crontab -e
0 7,12,18 * * * /usr/local/bin/GetLive.pl --config-file /etc/getliverc > /dev/null

Troubleshooting

Most of the problem I had were due to the “From ” (From space, not From:) line, which is the first line in an email. It raised a “Message contains invalid header” error with Cyrus and cyrdeliver. Make sure to keep messages on the server during your tests, because your downloaded emails might end up nowhere on your system.

Logs are your friends :

• /var/log/mail.log,
• /var/log/mail/john.log and
• /var/log/messages.

Check your logs and pay attention to what they say.