Cerebrus Email Guardian for POP3, Hotmail and Yahoo Email

Cerebrus Version: 1.6.5
Document Version: 1.4
Compkarori

Contents

1. Introduction

Cerebrus is a program that guards your POP mailbox from spam and other unwanted mail, and is fairly straight forward to set up.  It works by being an intermediary between your Internet Provider’s POP3 server, and your email client.  All data that is transferred from your ISP’s POP3 server, is first examined by Cerebrus to see if it might be spam, or other unwanted mail.  Cerebrus then optionally alters the subject header so that your email client can easily divert the mail to a holding area for you to inspect at your leisure.  Cerebrus does not delete any mail so that you can not lose any mail by accident.

For example:

Million fresh email addresses

will get changed to:

[Spam?][subject] Million fresh email addresses

This allows you to use your email program to direct all email with the subject line containing [Spam?] to a holding folder. The second insertion [subject] tells you that Cerebrus decided on the basis of the subject line that the mail might be spam.  You might wish to make a rule that also acts on the second insertion eg. immediately delete mail if it has “[banned from]” in the subject line.

2. Installation

Installation is easy.  Just unzip the archive cerebrus.zip into any folder on your hard drive, and the cerebrus-data.zip file into the same directory. Just make sure that you have enough space.  Cerebrus may save a compressed copy minus attachments of all the received mail in sub directories so it will need adequate space.  It will automatically attempt to create the directories “inbox” “good” and “spam”.  You can also download optional modules that support web based email, and these go into the same directory as Cerebrus.

Cerebrus does not make any registry changes, and so if you wish to delete it, you can just remove the folder into which it was installed.  If you plan to share Cerebrus with others on a TCP/IP network, then the same just applies.  In this case, Cerebrus works best when all the users on the network share the same POP server.  However, this is not a requirement.

If you want Cerebrus to start up automatically when you start your PC, just drag a link from the Cerebrus.exe to the windows startup folder.

3. Shutting Cerebrus Down

When the GUI option is set to false( refer to the Configuration section), Cerebrus brings up a console window.  Pressing ESC will cause Cerebrus to halt.

You can use the “Quit Cerebrus” button on the Cerebrus control panel.

When the GUI option is set to true and you have closed the control panel down, then you have to wait till the POPUP appears above the system tray.  You can then click on the email statistics text, and a requester will appear giving you the option to shut Cerebrus down.

You can also use the Windows Task Manager to shut Cerebrus down.  No data will be lost.  The task name will be “Cerebrus”.

4. Download Beta test version

The Windows download site is:

5. How Does Cerebrus Work?

Cerebrus works by performing exhaustive analysis of the headers which accompany each email. Here is an example header of some spam that was received recently after it was passed thru Cerebrus. The only thing that has been changed is that the real mail box name has been altered to a fictitious one where ever it occurred in the header.

Return-Path: <Arleenls@creditcard.com.tw>
Received: from 203.79.82.38 (account joebloggs@pop.netlink.co.nz)
by compkarori.co.nz (CommuniGate Pro RPOP 3.5.9)
with RPOP id 530019 for joebloggs@netlink.co.nz; Sun, 27 Oct 2002 14:22:54 +1300
Received: from nkrxvm ([63.142.221.194]) by netlink.co.nz (8.9.3/8.9.3)
with SMTP id OAA08558 for <joebloggs@netlink.co.nz>; Sun, 27 Oct 2002 14:08:37 +1300 (NZDT)
From: Cassandra Crawhall <Arleenls@creditcard.com.tw>
To: <joebloggs@netlink.co.nz>
Subject: [spam?][subject] Webcam Site Alert for joebloggs
Date: Sun, 27 Oct 2002 08:37:35 -0500
Mime-Version: 1.0
Content-Type: text/html
Content-Transfer-Encoding: base64
Message-Id: <qxicqpkubpd@creditcard.com.tw>

Cerebrus examines each line of the header looking for clues. To help it make the decision as to whether mail is spam or not, it uses a mixture of filters based upon the subject, various other header lines, content, and probabilistic content filters.  In addition, it can do a look up on black listing DNS servers, as well as validate the “Received:” headers.

To do this properly, Cerebrus needs to know as much about your normal mail as possible.

6. The Configuration Screens

[ Image ]

7. Configuration File

Here is an example configuration file that is used by Cerebrus.  This gets added to in use so that Cerebrus learns more about the type of email that you want and do not want.  A lot has been removed to ensure readibility, and you can check your distribution to see what is there now.  The file is called “cerebrus.config”.

log? false
rename-attachments? true
strict true
listenPort 110
ISP [ "netlink.co.nz" ]
password "Persephone"
GUI true
ChangeSubject true
ReportSpam? false
invisible false
reserved-ipaddresses [ "202.0.32.194" "203.79.82.38" ]
attachments [ "vbs" "scr" ]
SpamTrap [ spamtrap@netlink.co.nz ]
ban content [ " rape" "HGH" ]
ban subject [ "!!!" "$$$" "/ADV" "ADV:" "Porn" "RAPE" "Trade secrets" ]
allow subject [ "[a-w-h]" "[amigaone]" "[Rebol]" "[Zope]" ]
allow from [ myfriend@aol.com joe@hotmail.com ]
ban from [ aol.com msn.com hotmail.com ]
bad header [ "may be forged" "nobody" "HELO Hotmail" ]
ban virus [ "Klez Worm" "AAAAAAAA2AAAAA4fug4AtAnNIbgBTM0hVGhpcyBwcm9ncmFtIGNhbm5vdCBiZSBydW4gaW4g" ]
ban server [ "148.244.70.165" "19-Oct-2002/4:27:59+13:00" "160.124.88.237" "22-Oct-2002/10:00:41+13:00" ]

We will go through each line to explain how they are used.

log?
log? can be true or false.  If true, then Cerebrus maintains a log file (”cerebrus.log”) of each operation and stores it in the same installation directory.  You would only set it true if trying to see why something was not working correctly.  When set to true, and GUI is set to false, then Cerebrus will print the log directly to a window as well to the log file.
rename-attachments?
As a final line of defense against viruses, Cerebrus defaults to renaming incoming attachments if they belong to one of the dangerous types.  Viruses, when coming in through email, pretty well only come as an infected attachment.  If the attachment is renamed to an non-executable form, they can not damage your system.  The line “attachments [ “vbs” “scr” ]” lists only two of the 20 or so included in your distribution.  So, for example, an attachment “notsoinnocent.vbs” would be renamed to “notsoinnocent.vbs.scan-me-for-viruses.scan-me-for-viruses”. The double extension of “scan-me-for-viruses” is used as sometimes email programs do not show the extension.
strict
If set to true, then Cerebrus will examine the “Received:” line which is inserted by your ISP to see if the sender is who they claim to be.
listenPort
defaults to 110.  This is the tcp port that Cerebrus listens to for commands from an email client.  If you change this, you must make sure that your email client is set to open the POP3 port on the new port number that you have chosen.
ISP
Here you need to change this to your own ISP.  If you have more than one ISP, then include them all within the parentheses. There are two reasons for this.
Message-Id: <200210230132.OAA04106@netlink.co.nz>
Received: (qmail 23716 invoked from network); 23 Oct 2002 01:32:50 -0000
Received: from unknown (HELO sharon) (203.92.66.51)
by mail2.lga.net.sg with SMTP; 23 Oct 2002 01:32:50 -0000
From: Seagate Partner Program Specialist<info@seagate-ap.com.sg>

So, this email was sent by info@seagate-ap.com.sg, but the bulk emailer used did not add a message-id header. The receiving ISP netlink.co.nz added one to the message “Message-Id: <200210230132.OAA04106@netlink.co.nz>”.

To help detect bulk email, Cerebrus checks to see if the message-id comes from your ISP, and if it does then the “From:” address should also be from your ISP.  If it is not, then it is almost certainly bulk email, but not necessarily spam as in the example above.

Return-Path: <taramb918.n@hotmail.com>
Received: from 203.79.82.38 (account spamtrap@pop.netlink.co.nz)
by compkarori.co.nz (CommuniGate Pro RPOP 3.5.9)
with RPOP id 529987 for spamtrap@compkarori.co.nz; Sun, 27 Oct 2002 11:03:53 +1300
Received: from hotmail.com ([217.141.233.66]) by netlink.co.nz (8.9.3/8.9.3)
with SMTP id KAA19911 for <spamtrap@compkarori.co.nz>; Sun, 27 Oct 2002 10:53:45 +1300 (NZDT)
From: taramb918.n@hotmail.com

This spammer is pretending to be originating from hotmail.com but a DNS lookup of 217.141.233.66 shows that it is actually host66-233.pool217141.interbusiness.it. Cerebrus does a reverse DNS lookup on hotmail.com and confirms that it is not 217.141.233.66.  Now, the most important line for Cerebrus is:

Received: from hotmail.com ([217.141.233.66]) by netlink.co.nz (8.9.3/8.9.3)
with SMTP id KAA19911 for <spamtrap@compkarori.co.nz>; Sun, 27 Oct 2002 10:53:45 +1300 (NZDT)

where our ISP “netlink.co.nz” appears.  So, knowing the ISP helps Cerebrus locate that line.

password
The default password for Cerebrus.  You should change it to something else. We will cover later on how the password is used in the “Controlling Cerebrus” section.
GUI
If true, then a windows pops up just above the system tray when receiving mail.  If you click on the text, then an option to quit Cerebrus comes up.  You can set this to “false” if running on a server when no one will see the popups appear. You might want to set it to true when first running it, and then change to false once you are happy it is working well.
ChangeSubject
If true, then Cerebrus annotates the subject line of the email as described above.  If false, then it will create a new header line of “Cerebrus-filter: “ and annotate that instead.  Additionally, if false, it will alter the CC: field to include cerebrus@localhost, so that you can filter on this “email address” instead of filtering on the subject line if your email client is unable to filter on custom headers.
ReportSpam?
If true, then after Cerebrus closes the connection to the POP server, and if you have received identified spam, then a report screen

will pop up.  Choose the “guess” button and then “send” to send off your complaints. You can customise the text for the buttons 1… 4 by creating files complaint1.txt .. complaint4.txt in the same directory as Cerebrus.

invisible
if true, then no windows pop ups will appear ( apart from the very first if GUI is true ), and no console windows will appear either.  You will need to kill Cerebrus via the task manager ( cerebrus.exe ), or via an email “Quit” command.
reserved-ipaddresses
Cerebrus does a lookup on Spamcop to see if a particular IP address is listed as a spam source.  To prevent Cerebrus looking up your own IP address, you should list your ISP’s mail servers IP addresses here.  Also, if you have a static IP address of your own, you should add it here.  If you do not know what these numbers are, you can ask your ISP’s helpdesk.  Alternatively we will soon provide a tool that can do the lookup for you.
attachments
If any incoming attachment has one of these extensions, then it will be renamed.
SpamTrap
Any email sent to these emails will be treated as spam, and email servers that were used to send these emails will be added to your banned server list.
ban content
Cerebrus examines the content of all incoming email.  If it finds base64 encoded messages, it will decode them.  It then scans them for content, and if it finds any of these words listed in the banned content list, it will mark them as spam.
ban subject
Any incoming mail that has these word fragments in the subject line will be marked as spam.
allow subject
Any incoming mail that has these word fragments in the subject line will not be examined any further. This is to allow mailing list emails in.  The allowed subject list takes precedence over the banned subject list.
allow from
Any email which matches the allowed from list will not be examined any further.  If you decide to ban all of hotmail.com, you can still allow email in from specific hotmail addresses.
ban from
Any email that matches the banned from list will be marked as spam unless they also appear in the allowed from list.
ban header
Often mail servers will add strings like “unknown” or “may be forged” to the “Received:” header strings. This does not mean that they are spam, but they often are.
ban virus
Cerebrus will scan incoming base64 encoded attachments for several common viral signatures.  At the time of writing, only those for the BugBear Virus, and the Klez Worm have been included.  This is not intended to be a substitute for a good virus scanner.
ban server
Cerebrus collects a list of IP addresses which are known spam sources.  Each time it has to do a lookup on SpamCop, and finds a positive, it adds to this list so that it does not have to do another lookup on Spamcop.  If you have a correspondent who uses a banned server, you can still let them in by adding them to your allowed from list.

8. Setting up your Email Program

Cerebrus is what is called a “POP proxy server”.  This means that it takes all the instructions that your email program would normally send to your POP server, and then sends them on itself.  Any data it receives from your POP server, it then returns to your email program but after first classifying the email.  This ensures maximum compatibility with most email programs.  Although Cerebrus is designed to work only with POP3 servers, from version 1.4.2, support was introduced for Hotmail and Yahoo web email accounts.

8.1. Setup on local PC

If you are going to run Cerebrus on your own PC, then you need to configure your email program accordingly. The procedure for Outlook Express is as follows.

Go to the properties of your email account ( Tools->Accounts->Mail->Properties ), and on the server tab, note what is there.  This is the POP3 server. Now, change “Incoming Mail (POP3):” to “localhost” instead. Now, change your account name from say, joebloggs to joebloggs@what-ever-your-pop-server-was-before.

So, if your POP3 server were pop3.paradise.net.nz, and your userid is joebloggs, then your userid is now “joebloggs@pop3.paradise.net.nz”.  This is so that Cerebrus can determine which pop server to use, especially when you might have a number of different pop accounts.

Use “joebloggs@hotmail.com”, or, “joebloggs@yahoo.com” respectively.

8.2. Setup on network PC

Do the same as for local PC, except change the server to the IP address of the networked PC that is running Cerebrus.

9. Controlling Cerebrus By Email

Cerebrus was initially conceived as running on a network PC.  Since it would be inconvenient to access the network PC to make changes, commands can be sent to Cerebrus via email! In most cases, you just send yourself an email to achieve this.

Cerebrus therefore looks for a particular subject in every email to see if it is a set of commands for itself. If it sees:

password: Persephone

it knows that the body of the email contains instructions for it.  Note that the email must be sent as plain text, and not formatted as HTML.

Body commands must be formatted so that only one command occurs on a single line.  They can be single words, or they can be words that take an argument.

9.1. help

help

“help” will display a shortened version of this help

9.2. status

status

“status” shows the current configuration, and all the data for every list

9.3. password

password newpassword

Changes the password to “newpassword”

9.4. listenPort

listenPort 110

Sets the port that Cerebrus listens to for commands from an email client.  Defaults to 110 which is the POP3 port.

9.5. repair

repair

Repairs the attachment so that “src<>” is changed back to “src=” in an email that was considered to be spam.  Just forward the email back to yourself with “repair” the only command, and “password: <Cerebrus password>” as the subject. Refer to the section “Other Anti-Spam Measures” for more information.

9.6. quit

quit

Shuts Cerebrus down. All changes are saved automatically.

9.7. strict

strict true

If strict is set to true, then Cerebrus will check the message-id to see if it has originated from your ISP, and if it has, ensure that the from address is in one of your named ISPs.

9.8. GUI

GUI true

If GUI is true, then a popup will appear above your system tray when Cerebrus has finished analysing your email. If you click on the email statistics text in the popup, then a requester will appear asking you if you wish to stop Cerebrus.

9.9. ChangeSubject

ChangeSubject true

If Changesubject is true, then the subject line will be annotated when a spam is identified.

9.10. ban content

ban content "Ivory Coast" HGH "Human growth hormone"

Adds these three phrases to the banned content lists

9.11. remove content

remove content HGH

Removes the phrase “HGH” from the banned content lists

9.12. ban from

ban from aol.com msn.com

Adds aol.com and msn.com to banned list.  All mail purporting to be coming from these domains will be labelled as spam.

9.13. remove from

remove from aol.com

Removes aol.com from the banned list

9.14. allow from

allow from mymate@msn.com

Adds mymate@msn.com to the allowed list, and so even if all mail from msn.com is blocked, it will still allow mail from mymate@msn.com to come through to your InBox.

9.15. ban subject

ban subject SEX "million emails"

Adds the phrases “SEX” and “million emails” to the banned subject list.  Note that it is a good idea to add your username to the banned subject list as spammers often place this in the subject line, whereas valid email never does.

9.16. remove subject

remove subject SEX

Removes the word SEX from the banned and allowed subject lists

9.17. allow subject

allow subject [Rebol] [spam_brigade]

Adds these two phrases to the allowed subject lists.  This format with [] surrounding a word is most often used by mailing lists, which is a form of bulk mail you wish to allow in.

9.18. ban Spamtrap

ban Spamtrap spamtrap@mydomain.com

Adds this email address to the list of SpamTrap addresses.

9.19. remove SpamTrap

remove Spamtrap spamtrap@mydomain.com

Removes this email address from the list of SpamTrap addresses.

9.20. ban server

ban server 148.244.70.165 160.124.88.237

Adds these two ip addresses to the spam-server list, and automatically places a date beside the ip address. The date is added so later on you can check with SpamCop, or other blacklisting service to see if these addresses are still being used by spammers or not.  They can then be automatically removed from the spam-server list.

9.21. remove server

remove server 160.124.88.237

Removes the server 160.124.88.237 from the spam-server list

9.22. ban header

ban header nobody

Adds the word “nobody” to the banned header strings list

9.23. remove header

remove header nobody

Removes the word “nobody” from the banned header strings list

9.24. ban virus

ban virus "BugBear Virus" QogHR0l19+lj////kIsCg8IEiQeDxwSD6QR38QHP6Uz///9eife5fwUAAIoHRyzoPAF394A/DHXy

Adds “BugBear Virus” and it’s base64 encoded signature to the virusdefs list

10. Other Anti-Spam Measures

Spam often includes offensive images.  Some email programs automatically display these images once a message has been selected.  Cerebrus decouples the images so that they no longer display in any email it has classified as spam.  If by accident, Cerebrus does this to a valid email, you can retrieve the message by replacing “src<>” with “src=” in the content of the mail.  See the “Repair” command on how to use Cerebrus to do this for you.

11. Explanation of the Reasons for Mail Classified as Spam

Mail modified by Cerebrus will have one of the following inserted after the [Spam?] in the subject line.

  1. [bulk/forged?] - The message-id has been generated by your ISP, but the from: address is in another domain
  2. [forged Hotmail] - The message claims to be from a hotmail.com user but the message format is inconsistent with a hotmail email
  3. [header] - The header of the email contains one of the strings listed in the header list
  4. [subject] - The subject line matched one of the strings listed in the blacksubject list
  5. [SpamCop] - The sender’s mail server was identified as a spam source by Spamcop
  6. [Bayes] - The body of the email was analysed and thought to be spam based upon the probabilities of the words within the body.
  7. [banned content] - The body of the email contained one of the strings listed in the banned content list
  8. [banned from] - The sender’s from address is listed in the banned list
  9. [No To Address] - The email was not addressed to anyone
  10. [SpamServer] - The sender’s email server is already listed in the server list
  11. [DNS mismatch or none] - The sending email server’s IP address does not match that identified by your ISP’s mail server, or does not have a DNS record.

12. Virus Found

Note importantly that Cerebrus is not an anti-virus product, and in most instances you should be running a virus checker. When you do, then that virus checker will usually intercept the virus before Cerebrus.

If Cerebrus does find a virus, it does not attempt a repair of the attachment ( which is often a random document taken from the infected sending computer ) but replaces the email completely with a Cerebrus warning.  The subject header becomes:

[virus=Klez Worm] rest of subject

where the virus was indentified in this instance as the Klez Worm.

At the time of writing, only two virus definitions are included with the standard distribution.

13. Trouble Shooting

13.1. Timeout

A mail client such as Outlook Express does not normally timeout even when downloading large files as data are constantly being received.  However, Cerebrus has to download the entire email first so that it can be examined, and your mail client is likely to complain and issue a timeout warning as it will not receive any data during that period.  You will have to change the timeout period in your email client to avoid these messages, or just click on the “wait” alert when it pops up.

To change the timeout period in Outlook Express, go to Tools->Accounts->Mail->Properties->Advanced and change the server timeout period to 5 mins.  Cerebrus’ own timeout period is set to 30 seconds.

14. Test Mode

Cerebrus has a built-in testmode where it will process a directory of stored email and process each mail as though it had come from your POP3 server.  In this mode, the TOP command is not currently enabled.

Create a directory named “testmail” in the same directory that you have installed Cerebrus.  You can place raw emails in this directory.  A raw email looks like the text as seen in “message source” in Outlook Express. We will provide a utility to take email from your pop server and drop it into this directory.  You may also find raw email in the cache directories ( their name will be the same as the user logon string ) unless it has been deleted by Cerebrus.

Now create an email account with the incoming POP3 server set to “localhost”, and account name set to “test@testmode”. The password can be set to anything.  When you do a “Send/receive” on this account, it will grab all email files from the testmail directory.  It will not delete any of these files which can be received indefinitely unless your email client declines to receive them as it recognises them as being already read.

15. Windows Compatibility

Cerebrus has been tested with Windows 98, NT 4, XP Home and XP Professional.

16. Email Client Compatibility (Tested)

16.1. CommunigatePro

There is no way to alter the RPOP timeout period so email collection will fail on large emails ( over 500kb ) when CommunigatePro will timeout the connection to Cerebrus.  But since the cache was implemented in version 1.0.4, CommunigatePro will collect these emails on the next connection without having to download them from POP server again.

16.2. Eudora 5.2

Does not support Change Password Function

16.3. The Bat! (v1.61)

16.4. Outlook Express V6

16.5. Yam 2.3 sp1 (Amiga)

16.6. Others …

May be supported.  Please post to the

your experience with other mailers so that they can be added here.

17. Support

We have a

where you may post support related questions.  Currently you do have to register with the discussion board software.

18. Changes Log

1.4.2 16-Mar-2003 Support added for Hotmail and Yahoo web email accounts.  Email manager provided to delete mail off pop and web accounts directly off the server.

1.2.1 13-Feb-2003 Added Changesubject flag.  Relaxed testing of email where no dns record, or record does not match that claimed.  Only flagged as spam if probability exceeds .01 whereas without this problem, probability has to exceed .9

1.0.9 9-Dec-2002 Added a listenPort setting.  Resizing of the abuse report window is now fixed.

1.0.7 1-Dec-2002 Added abuse reporting module. Activate by setting ReportSpam? flag to true.

1.0.6 27-Nov-2002 Added SpamTrap list.  Now error traps popups.  Fixed bug with not decoding html mail where no boundary was given.

1.0.5 19-Nov-2002 Added invisible flag.  When true, no popups or console will appear.  You will have to kill Cerebrus by task manager, or via email “Quit” command.  If email arrives with no “subject:” header, Cerebrus will insert one.

1.0.4 16-Nov-2002 Automatic differential updates.  Caches the email locally to overcome problem where the email client is unable to increase the timeout period.  A directory is created for each email account.

1.0.3 13-Nov-2002 Supports connection to POP proxies that require userid in email format. Support for Eudora’s CAPA command.

0.9.9 10-Nov-2002 implemented test-mode.

0.9.8 7-Nov-2002 implemented TOP command so that can scan headers for spam, and delete from server without having to download the whole message.  This will not spot spam that has banned content.  Delete from server applies to mail clients that can do this eg. Outlook Express.  Spamcop is now a DNS lookup rather than http.

0.9.4 5-Nov-2002 changed the listen loop to keep the pop port open always.  Checks for new version at start up.

0.9.1 3-Nov-2002 changed to accomodate SpamCop’s new report page.

0.9.0 3-Nov-2002 added “repair” command, and new email syntax.

0.8.9 2-Nov-2002 fixed bug with forged? method.

0.8.8 2-Nov-2002 now changes are saved automatically.  No need to explicitly issue a “Save” command.  Email from hotmail.com is now checked for consistency with Hotmail guidelines.

0.8.7 1-Nov-2002 fixed parsing of the config file