User Tools

Site Tools


Data stream contained NUL character

Description of the problem

NUL characters (bytes with value 0) are not allowed in email messages and should not be transmitted over POP3, SMTP or IMAP4 except in certain circumstances. If an attachment contains such a byte it must be encoded using Base64 or Quoted-Printable encoding so that the transmitted data doesn't contain a NUL character.

In some cases, messages will arrive at VPOP3 which do contain such characters. This is usually due to a bug in the upstream mail server which has accepted a message with a NUL character and is passing it on unchanged. The email standards say that these characters are not allowed, so they should not be passed on.

For incoming SMTP, VPOP3 will reject a message that contains a NUL character and the sender should receive a delivery failure report. This is the correct thing to do when receiving an invalid message.

For incoming POP3, VPOP3 will not download the message, it will report an error message to the administrator, and it will stop collecting further messages. This is because there is no way for VPOP3 to automatically know what is the right thing to do in this situation. For a standards compliant remote POP3 mail server, the only way this problem could occur is if there is mailbox corruption on the remote server, so aborting the connection and reporting an error is the best thing to do to minimize further damage. For a non-standards compliant remote POP3 server, if the message is downloaded with the NUL character intact, it will make VPOP3 non-compliant also; if the NUL character is skipped or replaced, then that could cause a loophole for virus scanning; if the message is skipped it could cause an important message to be lost. So, reporting an error and aborting the connection lets the administrator work out the best solution for this particular instance - preferably by reporting the error to the remote mail server administrators and having them fix the bug or mailbox corruption in their server.

So, the ideal solution would be for the upstream mail server to sort the problem out, as the problem is coming from there (they should either reject the message, or recode the message to base64 or quoted-printable encoding).

Changing how VPOP3 handles the invalid characters

If that is not possible, then you can make VPOP3 handle the problem differently. VPOP3 can either remove the nul character totally, or replace it with a specified character (eg a space or question mark)

Version 6.17 or later

In the VPOP3 settings go to Settings → Misc Settings → Advanced and set the NULL characters in POP3 downloads option as you wish.

Version 6.16 or earlier

To do this for POP3 collection (the usual time it is needed) go to a command prompt in the VPOP3 directory and run:

vpop3settings set POP3CltNullCharacterReplacement <asciicode>

For instance, to make VPOP3 set NUL characters to spaces, use:

vpop3settings set POP3CltNullCharacterReplacement 32

After doing this, you should restart VPOP3, and check that the setting is there by running, from a command prompt:

vpop3settings get POP3CltNullCharacterReplacement

You can also use Imap4SvrNullCharacterReplacement, NNTPCltNullCharacterReplacement, and SmtpSvrNullCharacterReplacement to set the NUL character replacement for those components, but these are less likely to be needed.

If you have an incoming SMTP mail feed, then the default setting is that NUL characters in incoming messages will cause the message to be rejected, which is usually the desired behaviour.

Set the <asciicode> to '0' to have the characters rejected, '255' to have them skipped, or any other ASCII code (1 to 254) to have them replaced with that character

Note: you should use VPOP3 v6.16 or later if possible. Earlier versions had a bug which meant this setting did not work in every situation.

Keywords: null character 10506

how_to/data_stream_contained_nul_character.txt · Last modified: 2018/11/14 10:45 (external edit)