User Tools

Site Tools


how_to:vpop3_historical_logger_backlog

VPOP3 Historical Logger Backlog

This problem is primarily related to VPOP3 versions 2.5 → 2.6. Theoretically it could happen on later versions, but would be extremely unusual (see the note at the bottom if you are using version 3.0 or later)

You get an error message with subject VPOP3 Historical Logger Backlog and content:

VPOP3 has encountered an excessive backlog for the historical logger queue.
VPOP3 has more than 10000 pending items in the logger queue. To avoid
excessive resource use, VPOP3 will drop some logging records if the queue
grows above 15000 entries.

You can turn off historical logging on Logging -> Admin Logs in the VPOP3
settings if you do not want it. If you do want to keep historical logging
records on a busy server we recommend using the PostgreSQL database update
option, see Start -> Programs -> VPOP3 -> Download and Install PostgreSQL
support files.

Solution

The email message gives instructions on solving the problem, or an upgrade to version 3 or later should fix the problem.

Performing a full defragment on the VPOP3 disk may help as well, but would only be an interim solution.

Background

The problem happens when you have Historical logging enabled in the Admin Logs section. When this option is enabled, VPOP3 has to write several entries to the database whenever a message is received or sent.

The database which VPOP3 version 2.x uses by default is SQLite which is excellent for small systems but is not the fastest database when it comes to multiple inserts into a database, and does not handle multiple simultaneous inserts well at all. For this reason VPOP3 accepts the messages immediately, and queues (in memory) the requirement to update the database. In the background, another part of VPOP3 will write the queued entries to the database.

This means that messages are processed without delay, but if the database is behaving slowly, then this queue will build up and up in length. Older versions of VPOP3 would run out of memory if the database was excessively slow because the queue would fill it up, so now VPOP3 gives this warning when there are 10,000 queued entries. If the queue reaches 15,000 entries, then VPOP3 will discard any log entries which would take the queue over 15,000 entries.

VPOP3 version 2.6 has the option of installing PostgreSQL for this database as an option (see the error message above) which is a lot quicker than SQLite at inserts, especially multiple simultaneous inserts. VPOP3 versions 3 and later use PostgreSQL by default.

What if PostgreSQL (or VPOP3 v3 or later) is already in use?

If PostgreSQL is being used for the logging database, then it is extremely unlikely to encounter this problem. If it does, then it usually indicates a disk problem, eg a partially failing disk, or excessive disk fragmentation.

how_to/vpop3_historical_logger_backlog.txt · Last modified: 2018/11/14 10:45 (external edit)