User Tools

Site Tools


how_to:whitelist_blacklist

Spamfilter Whitelist & Blacklist

The VPOP3 spamfilter has a whitelist and a blacklist for sender addresses. It also has a whitelist and blacklist for words/phrases.

You can view these lists in the VPOP3 settings by going to Settings → Spamfilter → White/Black Lists

By default, whenever you send a message, the recipient's address will be added to the whitelist, this is because VPOP3 will assume that the recipient is someone you want to be in contact with. Similarly, if you release a message from the spamfilter quarantine, the sender's address will be added to the whitelist.

What the whitelist and blacklist do

When an address is found on the whitelist or blacklist, then the spam score is adjusted by 100 x (the relevant rule weight. With the default settings, this means that an entry on the whitelist has '-10000' added to it, and an entry on the blacklist has '10000' added to it. So, if an entry is in the whitelist AND the blacklist, they will cancel each other out.

If you wish, you could adjust the rule weights so that one has precedence over the other, eg set the whitelist weight to '-150' to make the whitelist overrule the blacklist, or the blacklist weight to '150' to make the blacklist overrule the whitelist.

Blacklist

VPOP3 will never automatically add an address to the blacklist. This is because most spammers will fake the sender's email address, so it is pointless adding an address to the blacklist, and can even be harmful if the spammer has sent a message pretending to be from a legitimate user.

Because of this it is often not worthwhile putting addresses onto the Blacklist manually either. The only case that is generally useful is if you have signed up to a mailing list, and they are refusing to remove you from the list, but this is quite rare.

Fields which the Blacklist checks

The Blacklist currently checks the addresses in the Return-Path, Reply-To and From header fields.

The Blacklist will not be checked if the header contains three or more 'Reply-To' headers (this is an invalid message, and matches another spamfilter rule)

Whitelist

Local Addresses

If you put an address on the whitelist which is also in a 'local domain', then, by default VPOP3 will ignore that when checking incoming mail. This is because spammers often send spam pretending to be from the same domain as the recipient. In most cases, mail from local domains will be sent locally, not incoming, so, usually, this is a sensible rule to follow.

If you look in the messages headers of a filtered message, then VPOP3 will add a header line like:

X-VPOP3-WL: Skip Local (email.address)

if this is happening

If you want to tell VPOP3 to check the whitelist for incoming mail from local domain email addresses as well, then go to Settings → Spamfilter → General, go to the Script Configuration tab, and set CheckLocalWhitelist to 1.

Note that in many cases, external users can actually send mail through VPOP3 as if they were local users, so their mail won't be spam filtered at all.

Fields which the Whitelist checks

The Whitelist currently checks the addresses in the Return-Path, Reply-To and From header fields.

The Whitelist will not be checked if the header contains three or more 'Reply-To' headers (this is an invalid message, and matches another spamfilter rule).

The 'Reply-To' address will not be checked if it is set to 'noreply@facebookmail.com', as that would whitelist a lot of spam, because that address is often added to the whitelist by out-of-office replies.

Whitelist result info

The spam filter adds a header to the message called 'X-VPOP3-WL' which contains diagnostic information about the whitelist results. It may also add a line 'X-VPOP3-SpamWL' if the whitelist checks have been skipped for some reason.

Whitelist Words and Blacklist Words

VPOP3 has a section where you can add data to a whitelist or blacklist of 'words'. You should only add up to 200 entries here, because adding too many can affect performance.

In fact the 'words' can be phrases or regular expressions as well as simple words. VPOP3 will check the subject and any 'text' parts of the message for the 'words'. (it will not check attachments). VPOP3 will decode any quoted-printable or base64 encoded parts of the message.

Note that we would not recommend using blacklist words as a general way of adding to the spam filter, as it is relatively inflexible and slow (instead, you may want to look at writing your own rules, but the whitelist words can be a good, simple way to whitelist emails which contain your own product names, for instance.

Data Formats

By default, VPOP3 will try to find the 'words' you enter as ASCII character insensitive substring matches. This means that a word of Buy Viagra will match buy viAgra but not buy viagra (with two spaces) or buy<CRLF>viagra (split onto two lines).

You can also use regular expressions to make the checks more flexible, however this can be daunting for people who do not have experience of using regular expressions. To specify a regular expression, you write it as /<regular expression/<flags> (which is a common format for regular expressions.

So, for instance, a regular expression of /\bbuy\s+v[1i][a@]gr[a@]/i will be a case insensitive match for 'buy viagra' where 'buy' starts on a word break, there is one or more space/tab/line-break between the words, and the 'i' in viagra is an 'i' or a '1', and the 'a's are either 'a' or '@'. So, this one rule would match buy v1agr@, or buy vi@gra etc. This shows the power of regular expressions,

Character Sets

VPOP3 currently does not do any character set normalisation. This can cause issues for non-ASCII character sets. For instance, European accented characters may have several different raw forms in different character sets such as ISO-8859-1, ISO-8859-2, utf-8, etc.

This may mean you need to use several whitelist/blacklist word entries to match the several options, or use regular expressions to match them all. If you understand regular expressions, that is generally the simplest way to do.

For instance, to match naïve, the ï character is character 239(EF in hex) in ISO-8859-1, ISO-8859-3 and Unicode. Even though the character number is the same, the UTF-8 encoding means that the Unicode version will usually be transmitted as hex C3 AF, rather than as hex EF.

So, a regular expression to match this would be /na(\xef|\xc3\xaf)ve/, or if you want to be case insensitive, you could use /na(\xef|\xc3\xaf|\xcf|\xc3\x8f)ve/i (Ï is hex CF or hex C3 8F in UTF-8 encoding)

In a future version of VPOP3 we may normalise all character sets to UTF-8 encoding which will not affect ASCII characters, but will make it easier to cover all the options for non-ASCII characters.

how_to/whitelist_blacklist.txt · Last modified: 2018/11/14 10:45 (external edit)