Building a Webmail Solution on top of Apache James Mail Server

Part of maxant’s offering to small businesses is email hosting. As well as standard POP3/SMTP access, maxant offers webmail access. A quick search on the web shows that there are several open source webmail solutions available. The problem with all of them is that they communicate with the email server through the SMTP protocol. For example, if you wish to preview a list of emails, the web application needs to access the email server and ask for details of each email (while leaving them on the email server, so they can be downloaded at a later time via POP3).

Reading all the emails is inefficient and the larger the number of emails in your inbox, the longer it takes to just see a list of emails. The solution built by maxant is based on the Java Mail API from Sun. This API lets you access individual emails in your inbox using an ID. But Apache James Mail Server (James for short) doesn’t maintain the index, if a new mail is put in the inbox, so if you have a list of all emails and decide to access one, and in the mean time you have received email, the chances are that you won’t be able to read that email!

The next problem is how to deal with keeping a copy of sent emails for your "sent items" folder. If you just use the Java Mail API, the only solution for getting a mail into your email server so that it can be downloaded later via POP3 is to send that email explicitly to yourself, or to put yourself on CC. If you put yourself on CC, it gets confusing to some mail servers when replying to you, so that you get several copies of the email. The other problem is that that email will then end up in your inbox!

Another problem is message flags and folders. The Java Mail API offers the ability to add information to email messages such as the folder which they are in or details about when they were read, replied to or forwarded. But these flags are optional – email servers to not have to implement them, and indeed James does not.

There were several steps in solving these problems. The first was to set up a database repository for James so that emails were stored to a database rather than a file system, as is default. Having the email in the database allows one to access them using their primary key – a generated ID and the username. So if other email arrives in between reading a list of all email and drilling down to the detail of an individual email, you are guaranteed that you will always access the correct email.

In order to be able to store information about folders and message flags, and additional table was created with a foreign key to the message ID in the James table. Any information that needed to be stored additionally about emails could be stored there, such as the date and time of reading or replying.

To make reading a list of emails efficient, a selection of all emails in the users account was compared to their message flags. Where message flags were missing, full email details were read from the James table and used to create an entry in the other table. Since the other table was lighter weight (it only contained the information for displaying a list of emails), it was quicker to read and there was less data transfer between database and application, which is not only quicker, it causes less memory problems in the application.

When sending an email, a copy for the "sent items" folder was just created directly in the database as opposed to sending an copy of email to James for it to place it in the database. This got over the issues discussed above.

The only problem with all of these solutions was the requirement to use the James API within the web application in order to read/write directly from the database. James stores email messages as blobs in the database and to translate them into objects requires part of the James API. This is fine until a new version of James comes out in which case the libraries might not be compatible. That makes version migration an issue for this solution because of the additional rebuilding and testing required when upgrading the email server. However, James is a very solid and reliable piece of software and it is unlikely that such migrations will occur frequently!

So, to summarise, if you want a proper webmail client which interacts with the email server properly, first choose the right email server! In reality you will probably find that you need to integrate the two to a much higher degree than you intended, to get the effects you desire.

Copyright (c) 2009 Ant Kutschera