LMTP sporadically twice

I have a problem with DOVECOT/LMTP. Emails appear sporadically twice in the mailbox. In different maildomains and mailusers. The mailusers have no quota, forwarding or copy in the mailbox settings

RockyLinux 8
Postfix 3.7.3
Dovecot 2.3.16
Webconfig ISP-Config 3.2.8 p2

 

What shows mail log for the messages that appear twice?

 

lexmark-mail.txt is the log
i didn't see anything, doesn't say there's something there and i don't see it.

 

What do you mean by mirrored setup?

Leg it does not happen with all mails, sporadically. Sometimes 5 mails on 50 accounts. Not limited to one account. There are days when nothing happens at all. It doesn't look like a system, it looks random.

 

Do you have a ISPConfig Multiserver setup with 2 or more mailservers, and are these servers mirrored?

 

No, not mirrored.
The construct of my servers is a bit strange, I have a web server without mail with ISP config and I have a mail server with some webs with ISP config. Background was that a few customers have degraded the web server to the spam slinger and I then in a night and fog action the mail server with another IP have set up again.

 

Do the mails that are duplicate, have the same date/header information?
Can you grep a recipient in the mail.log where the issue occures and check for the line that says "saved to INBOX" for the specifid mail. Does it appear twice in the mail.log?

Are these mails duplicated in the vmail home of the user? If so can you post the headers of both of the mails (make sure to censor sensitive data).

 

The mail appear several times in the log

 

But these are all fail2ban notifications from localhost?
Does the problem happen with mails coming from an external mailserver? Or does this only happen with local mails? Are we exclusivly talking about fail2ban notifications, or is this just an example?

 

This is just an example that is extremely good, yes it also concerns external servers, but since the mails are only accepted once, I guess it is lmtp or dovecot.

 

Here is an example for external server, with log and header

 

There seems to be an error with your milter.

Code:

Nov 16 10:20:24 w3bserversystem-three postfix/smtpd[1168548]: report connect to all milters
Nov 16 10:20:24 w3bserversystem-three postfix/smtpd[1168548]: milter8_connect: non-protocol events for protocol version 6:
Nov 16 10:20:24 w3bserversystem-three postfix/smtpd[1168548]: milter8_connect: transport=inet endpoint=localhost:8891
Nov 16 10:20:24 w3bserversystem-three postfix/smtpd[1168548]: trying... [::1]
Nov 16 10:20:24 w3bserversystem-three postfix/smtpd[1168548]: Connection refused
Nov 16 10:20:24 w3bserversystem-three postfix/smtpd[1168548]: trying... [127.0.0.1]

This happens over and over again, and can be the cause of the multiple mails. Check your milter and the configuration. If you use rspamd, make sure to check that redis is working correctly too, as this can often be a cause of rspamd not working correctly.

But there are some other curios things:

Code:

Nov 16 10:20:24 w3bserversystem-three postfix/smtpd[1168548]: SSL_accept error from m204-227.eu.mailgun.net[161.38.204.227]: -1
Nov 16 10:20:24 w3bserversystem-three postfix/smtpd[1168548]: warning: TLS library problem: error:14094412:SSL routines:ssl3_read_bytes:sslv3 alert bad certificate:ssl/record/rec_layer_s3.c:1544:SSL alert number 42:

So there seems to be an error with amavis too which you should check:

Code:

Nov 16 10:20:41 w3bserversystem-three amavis[1100452]: (1100452-13) (!!)TROUBLE in process_request: Error writing an SMTP response to the socket: Broken pipe at (eval 81) line 1433.
Nov 16 10:20:41 w3bserversystem-three dovecot[2796882]: lmtp(1168792): Connect from local
Nov 16 10:20:41 w3bserversystem-three amavis[1100452]: (1100452-13) (!)Requesting process rundown after fatal error

 

I threw the milter out of the main.cf. so the error message was gone and strangely also the message from amavis, strange.....
The message with the TLS library is a problem of mailgun. thx for vour help

I will reinstall opendkim and opendmarc and check the configs, there was an update 1 month ago, maybe what Das the culprit. Although the services did not show any error, strange.

 

Sehr gut

It sometimes can be quiet a pain to detect errors like this, as the service itself is working and the misconfigured part does not make the service itself fail, but misbehave and create unexpected behaviour. Glad you figured it out!

 

Thanks anyway, for the food for thought. Strange is however, that a Milter with "milter_protocol=6" the error should go over, the content filter amavis in the nirvana tears, strange.....very strange. So I will now still observe whether mails duplicate again, or whether the main problem has also settled.

So please keep the thread open,

Thx at all, and a fine weekend

 

So after a bit of testing, commented out opendkim and opendmarc from main.cf. I still have the problem that mails appear twice in the mailbox.
Attached the logs and headers from gustini.de.

 

There still seems to be an issue with amavis

Code:

Nov 20 06:30:44 server-three amavis[2490080]: (2490080-14) Passed CLEAN {RelayedInternal}, MYNETS LOCAL [127.0.0.1] [185.4.120.53] <[email protected]> -> <[email protected]>, Message-ID: <[email protected]>, mail_id: 1HxdKO01Xjrg, Hits: 1.261, size: 60407, queued_as: 4C6CB261B4, 13595 ms
Nov 20 06:30:44 server-three amavis[2490080]: (2490080-14) (!!)TROUBLE in process_request: Error writing an SMTP response to the socket: Broken pipe at (eval 81) line 1433.
Nov 20 06:30:44 server-three amavis[2490080]: (2490080-14) (!)Requesting process rundown after fatal 

 

Thanks for the answer. So the problem is solved since yesterday.
1.)
opendkim-2.11.0-0.32 has some bugs (full tit), a downgrade to opendkim-2.11.0-0.28 and eh voilà it runs again. A crashing service opendkim pulled down amavis and postfix.
2.)
an old recycled script started spamassassin as a service(GreenIT). The consequence was that if a mail was in the queue of Amavis, Spamassassin scanned the mail again, you can see it in the esmtp entries in the log.
Conclusion
a service running amok disguised and potentiated the problem with spamassassin. And above all made the search very exciting. Honestly, mess Easter, this Easternegg was just better.

And when I'm in a good mood again, I'll have a look at the switch to rspamd and bug you guys

THX for your help