Homepage > Popclean

Popclean

Table of contents

Return to table of contents Description

Fetchmail downloads ("fetches") messages from a POP3 mailserver and forwards them your local (client) machine's delivery system using SMTP. It can either leave a copy of these messages on that server or delete them entirely. Many users of fetchmail have requested the possibility to postpone that deletion to a later date. Every single user has different reasons for that request. As per the FAQ, item G5, fetchmail is not meant to implement policy; hence the second-most- requested feature for fetchmail is denied. Popclean comes to the rescue.

Popclean is a perl script that deletes messages from a POP3 server a certain amount of time after they have been received by that server. It does so by looking at the latest "Received:" header. It was designed to work in conjunction with fetchmail. By default, account information is obtained from the fetchmail configuration. Popclean will also look at the .fetchids file to determine if a message was received by the local host before deleting it from the POP server. This helps in avoiding deletion of unread messages. However, provided the correct set of arguments was specified no Fetchmail information is needed. Many options control the behaviour of popclean. Some of them actually make the presence of fetchmail optional. See the note below.

Additionally, popclean can delete messages that exceed a certain size or contain a tag in the subject line that was inserted by a spamfilter.

The current version of popclean is 1.31

Popclean makes a best effort attempt to determine the age of a message. As the POP server does not record when a message was downloaded or read, the timestamp from the Received: header is taken as reference. This could mean that, when coming into the office after the weekend, a message could be deemed old before you read it. As popclean looks at the fetchmail id file to see if it was downloaded before actually deleting it, this does not mean it is deleted from the server before you get the chance to read it. Just keep in mind what the term age means in the context of popclean. The only way properly handle this is by having fetchmail record a timestamp in the id file. As popclean should work with a vanilla version of fetchmail, this approach was not an option. The fetchmail maintainers/developers are however challenged to reconsider the policy statement in the FAQ.

Return to table of contents Features

The manual page is shipped as part of the distribution. An online version is available here.
Looking for SSL support? Let me know! SSL is implemented experimentally but has received limited testing. It does however work with GMail.

GMail note:

Be aware that, unless you're using "recent mode", the GMail POP server only ever sends the approx. 300 oldest mail ID's at one time in response to a LIST or UIDL command. This can fool either fetchmail and popclean. If you have many messages in your inbox the latest arrivals can be missed and it looks like no new mail ever arrives. This can happen easily when you use the KEEP option in your Fetchmail configuration. Popclean can help here by simply cleaning up your upstream mailbox. You may need to repeat the process until popclean reports that the number of messages checked is under approx. 300. Apply - Rinse - Repeat.

Return to table of contents Download and installation

Prerequisites

Download the latest gzipped TAR archive here. This archive includes:

You will need the latest fetchmail distribution, or at least version 5.9 (which is already considered obsolete). Earlier versions may or may not not work, but you are advised to upgrade anyway. You may download fetchmail from it's homepage.

You will probably also need some additional perl modules. Most importantly, you need a perl module named POP3Client. You can download it here. You can search for other perl modules from here.

Other perl modules that are used and may not be present on your system are: Instructions to install perl modules can be found here. Most of not allof these perl modules are available as packages in your Linux distribution where the current version is also mantained. This will ensure you will be always up to date.

To install and use popclean, do the following:

  1. Install the latest fetchmail, either from source code or binary distribution.
  2. Configure fetchmail and make shure it works properly. The configuration options UIDL and KEEP are required to provide the proper context for popclean. Keep in mind that popclean, true to its name, only cleans POP servers.
    Popclean will take the values from the following keywords in the fetchmail configuration file:
  3. Install popclean.pl in an appropriate location like /usr/local/bin. Make certain the first line in the script points to the location of your perl executable.
  4. Wrap the call to popclean.pl in a shell script with your choice of options. To avoid mailbox locking issues make shure fetchmail is not running while popclean runs. Use fetchmail -q to stop the daemon; restart it after popclean has finished.
  5. Install the man page in the appropriate location, eg. /usr/man/man1.
  6. Use cron to run the script you created in the previous steps. Consult the cron manual or your local specific cron facilities. Many allow running running cron daily by inserting a link to it in a directory like /etc/cron.daily.

Note 1: Popclean can be used on systems without fetchmail installed. In that case use at least the --server, --user and --skipuidl options to explicitly specify the account information. Keep in mind that no validation is possible to prevent deletion of messages that were never downloaded from the server.

Note 2: One could create a fetchmail configuration file just for the sake of configuring popclean. The documentation is available through the online fetchmail manual page. Make shure you specify the keyword equivalents to the options mentioned above. The --skipuidl option is still needed.

Return to table of contents Testing popclean

Popclean recognizes the --test option. This allows you to test popclean without actually deleting messages. Especially when used in conjunction with the --verbose option this is a good way to make certain popclean works as expected. Pay special attention to the parsing of the fetchmail rc file and calculation of ages.

Return to table of contents Support

Questions and suggestions can be directed to the author, Jan Klaverstijn.

Return to table of contents Credits

Copyright© 2003 Jan Klaverstijn

No doubt all this would not have been possible without the work of Eric S. Raymond who started fetchmaillogo , an invaluable component of my email infrastructure and that of many others, albeit with a little white spot. That spot is now filled in by popclean.

Many thanks to Father Peter, archdaecon < DormitionSkete AT hotmail.com> for the many valuable suggestions and patient testing.

The code for parsing the fetchmail rc file is taken from the Webmin sources. Thanks to the author(s).

This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version.

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

You should have received the file LICENSE in the source directory with a copy of the GNU General Public License along with this program; if not, write to the Free Software Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.