fetchmail problems -- duplicated mail??

[Feztaa]

I'm having this weird problem with fetchmail that's just started -- I'm getting endless duplicates of all my mail. It's almost as if fetchmail is configured to not delete old messages, and to download all old messages, but it's not (I checked). In fact, fetchmail is configured to delete old mail and not to download old mail, which should prevent this, but it isn't.

I tried to boot to windows to see if this was a problem with the server or a problem with fetchmail, but in windows I can't even ping my mail server for some damned reason.

I've set up procmail to delete duplicate mails, but that's only a temporary solution. Anybody know what's going wrong and how I can fix it? Is fetchmail going haywire? Is my mail server on crack? What's up?

[redhead]

try invoking fetchmail with the '-a' flag to flush your queue on the mail server.
Else use the '-k' and fetchmail shoudl keep count on which is new and which is old messages.

[pbharris]

here is my ~/.fetchmailrc and it works good.

# Configuration created Sun Aug 19 23:58:42 2001 by fetchmailconf
set postmaster "pbharris"
set daemon 1200
poll mail.telocity.com with proto POP3
    user 'cph013' there with password 'ImNotTelling' is cph013 here

[Feztaa]

This is my fetchmailrc and it's been just fine until yesterday:

set postmaster "postmaster"
set bouncemail
set no spambounce
set properties ""
set daemon 20
poll shawmail with proto POP3
    user 'feztaa' there with password [censored] is 'feztaa' here

poll mail with proto POP3
    user 'fezziker' there with password [censored] is 'feztaa' here
  user 'kingfeztaa' there with password [censored] is 'feztaa' here

I don't get it?

Argh. I'll try what redhead said.

[redhead]

This is my fetchmailrc and it's been just fine until yesterday:

set postmaster "postmaster"
...
set daemon 20
...

Hmm.. 20 secunds, why check so often.. If one fetchmail thread is fetching a masive mail, it can take upto a minut, then the thread will be given a HUP signal once the 20 sec. are up, this will respawn a fetchmail thread to start fetching mails again, but this thread figures an old thread is actualy fetching at the moment, either the old thread gets killed, leaving the server thinking, "something went wrong I'll keep the messages marked _not fetched_" and the new thread will start fetching them again(case 1)
Or the new thread will terminate on itself, and the old thread will finish the fetching, and tell the server to mark the messages as fetched.(case 2)

Now I belive fetchmail uses case 2, since if you execute fetchmail while it is running as a deamon, it will say "Another thread is running, terminating", but shoud it be case 1, then the lets say the server has two small messages, then a HUGE one.
fetchmail will connect, get a mail list, see theres these three messages, tell the server to start fetching them, finish the two small ones, tell the server they were successful, the server will say, marke them as seen, but not deleted.
The fetching thread will start on the HUGE message.
The new thread of fetchmail will realise theres another thread allready running tell it to stop (because the new thread will have the power to fini sh any ongoing fetching ATM.)
The fetching thread will terminate, not tellling the server that the two small messages should be deleted, since they are through, the new thread will start the fetch, see the two small messages as have been seen, yet it wont know they shouldn't be fetched, since the old fetching thread didn't get a chance to write to the fetchmail status that message number foo and bar was fetched, thus are old news. And the new thread therefor fetches the two small messages, and starts on the HUGE one, at this point the process starts all over....

Like I said, I belive fetchmail uses case 2, because case 1 is a loop hole.. But just in case.. Makes you think dosnt it ?

[Feztaa]

Interesting theory, but it doesn't work because:

a) I've downloaded huge messages in the recent past with no trouble, and

b) There are no huge messages on the server now that could be causing this.

Anyway, I just used the novice config in fetchmailconf, and I didn't realize that was in seconds, thought it was minutes

[redhead]

You might want to make a small seperate configuration file, just to flush your servers here... something like:

poll shawmail
proto POP3
user feztaa
pass [seecret]

poll mail
proto POP3
user fezziker
pass [seecret]

poll mail
proto POP3
user kingfeztaa
pass [seecret]

then use: fetchmail -v -a -F -d 0 -f <the config file>
Just to make sure all your mail queues are flushed, and that you start from scratch, if your regular setup then returns to the same pattern, then it must be a problem on the server (that is if you're getting old messages from befor this flush was issued)

Just remember to have killed the allready running fetchmail deamon here...