How to automatically forward email from Exchange without loosing headers

UPDATE: I've now created a service to make this much easier to forward email. If you are interested in this, please have a look at the service's site.

I've got a million email accounts. Every time I start work on a new client site, I get given yet another email account. Its a pain in the butt to manage all of these, so wherever possible I forward the mail onto my main gmail account where it can get filtered, stored and searched easily.

This works great for sites with unix based email, but more often than not my clients use Microsoft Exchange for their mail. You can set up a redirecting rule to forward the email (as long as the server has been set up to allow this), but when it does so it strips off the To: and CC: headers from the message when it sends it on. For example, if Bob Log had sent a message to Me@mycompany.com, SomebodyElse@mycompany.com, when it was forwarded on to my gmail account it would still appear to come from Bob Log but the To: would just be me@gmail.com, not the original recipients. This won't work!

I've toyed with a number of solutions to this problem, including writing a bot that uses EWS to poll the server, re-form the message and send it on, but that requires a bot to run on the client's lan and may not be able to forward on the message as getting access to Exchange to send the message can sometimes be difficult.

I now have a solution that works without requiring any additional software to be running on the client's network. The solution involves forwarding the message to an intermediate account as an attachment (which preserves the headers), filtering the message back out of the attachment at the intermediate account, then sending that message on to gmail in all its original glory. To do this, you require an intermediate email account that you can use purely for the purposes of filtering the mail, which is capable of piping incoming mail to a perl filter script. Generally, these aren't that hard to come by. I happen to have a getting started plan with Cove here in Melbourne which fits the bill nicely. Its free to run (although they do have a $2 setup fee) and its servers are in Australia which is a benefit for me. If you live elsewhere, there are probably other options that would do just as well.

To set it up, there are three steps,

step 1: create the filter script
Below is included a perl script which takes an email address as a parameter, and reads a MIME encoded email on STDIN. It will look for a part with a content-type of message/rfc822 which is an embedded message, and then it will stream that message out to sendmail with the supplied email address paramater as the final destination. This file should be uploaded onto your server somewhere where the mail filter can get hold of it.

I originally wrote a script that used the MIME::Parser perl module, but I've found that most hosting providers don't have that module installed, so it was easier to just do it from scratch. I'm not a perl programmer really, nor have I spent a lot of time on this script, so it definitely could be improved, but it works!

#!/usr/bin/perl

my $recipient = $ARGV[0];
my $boundary = '';
my $endMarker;
my $partType;
my $sendmail = "/usr/sbin/sendmail -oi $recipient";

# Reads a line from STDIN, and makes sure it isn't the EOF
sub fetchLine {
my $txt = <STDIN> or die "Reached end of file prematurely";
chomp $txt;
return $txt;
}

# reads a message part, looking for an rfc822 message.  If it finds one, it
# forwards it on to the recipient.  When it finds the part end, it returns 1
# if there are more parts, or 0 if it is the end of the message
sub parsePart {
my $isMessage = 0;
my $returnCode = -1;

# First, read the headers, looking for a content type
while ((my $text = &fetchLine()) ne '') {
if ($text eq 'Content-Type: message/rfc822') {
$isMessage = 1;
open(SENDMAIL, "|$sendmail") or die "Cannot open $sendmail: $!";
}
}
# Then read the body, streaming it out if it is a message
# End the loop when we find a boundary or the end marker
while ($returnCode == -1) {
$text = &fetchLine();
if ($text eq $boundary) {
$returnCode = 1; # Meaning we still have parts to parse
} elsif ($text eq $endMarker) {
$returnCode = 0; # Meaning we are finished parsing the message
} elsif ($isMessage) {
print SENDMAIL "$text\n";
}
}
if ($isMessage) {
close(SENDMAIL);
}
return $returnCode;
}

# First, Read Headers, looking for the multi-part content type and
# boundary separator
while ((my $text = &fetchLine()) ne '') {
if($text =~ m/^Content-Type: (.*)/i) {
$nextline = &fetchLine();
$nextline =~ m/\s+boundary="(.*)"/i or die "Could not get boundary after content type";
$boundary = "--$1";
$endMarker = "--$1--";
}
# We don't care about any other headers
}

# Check to see that we have the right mimetype and a boundary
die "No boundary found" if $boundary eq '';

# Skip until the first part separator
while ((my $text = &fetchLine()) ne $boundary) {
}

# Parse the message, looking for a part with a type of message/rfc822
while (&parsePart()) {
}

exit 0;

step 2: set up the filter in cpanel (or whatever else you use)
On your server, you now need to set up mail filtering so that any incoming mail from your work account that isn't a bounced message gets sent to your filter script. In cpanel, I did this by setting up an email filter for all mail, which looked something like this:



step 3: enable forwarding of mail in Exchange
Now that you've got your email forwarding filter set up, all that remains is to set up exchange to forward any incoming mail to your filter account. You do this by selecting Rules and Alerts



And then setting up a rule that looks like below:

Be careful with what you put in your rule definition, as some rules are "client only" which means they will only run when outlook is open. As an example, I tried to make it also mark the message as read when it forwards, but that is "client only" which means the rules won't run unless outlook is open :(

Once you've got that set up, you can test. Send a message to your work email and see if it makes it through. If anything goes wrong, it should bounce with a message telling you what went wrong. One thing I did notice though is that if I send the message from gmail itself, it tends to ignore the message when it gets forwarded through as it already has a copy (in sent), so I send test messages from an alternate account just to be safe.

Okay, so in summary, its possible to forward messages on from exchange, but it requires a man in the middle to extract the message contents, and its a bit fidlly. If you like this tip, let me know.