rspamd: spam status modified (and mail delayed) even though Spamfilter is set to 'not enabled'?

Discussion in 'ISPConfig 3 Priority Support' started by Etcetera, May 23, 2020.

  1. Etcetera

    Etcetera Member HowtoForge Supporter

    I have a primary mailbox on server #2 that serves as the final destination for all of my mailboxes anywhere, most of which are hosted on my server #1. The transfer happens through 'redirect' commands in Sieve scripts. Both servers are running Ubuntu Server 16.04 LTS in a "perfect server" setup with Apache, Postfix and Dovecot. Today, I switched from amavisd to rspamd.

    After making that switch, I find that mail which was received by server #1 and redirected to server #2 gets processed by rspamd even if both the mail domain and the mailbox at the receiving end on server #2 are set to 'Spamfilter: not enabled', with the spam scores (X-Spamd-Bar, X-Spam-Level) adjusted and possibly the subject line edited to suddenly include the "*** SPAM ***" notice.

    The latter happens even though none of my rspamd-related Spamfilter policies is configured to modify the subject in any event. (I've now configured those receiving mailboxes to use the 'Uncensored' filter as a workaround.)

    The spam score of course always deteriorates in the process because the SPF etc. fails for most senders when the sending server is my server #1...

    Also, I've noted a delay of roughly seven minutes in one case for such a redirect now, while I'd usually expect them to happen near-instantaneously?
     
  2. till

    till Super Moderator Staff Member ISPConfig Developer

    When you don't configure a spamfilter policy, then the defaults of rspamd get applied and the default is to filter the email but with relatively high limits. It's the same for amavis btw. If you don't want to get emails filtered out, you have to choose the uncensored policy. In any case, emails will run trough amavis or rspamd as the decision if the mail is filtered happens in amavis/rspamd. The delivery should not take 7 minutes when your server is not highly loaded or has many mails in the queue that's air to be processed.
     
  3. Etcetera

    Etcetera Member HowtoForge Supporter

    Thanks for now, that explains it. I probably never noticed with amavis because it never altered the subject without my telling it so (I knew from the headers that each mail went through it, but I never knew that it still applied actual spam filtering).

    (Shouldn't that Spamfilter option in ISPConfig be rather named something like "Default" then, instead of "not enabled", which is rather the opposite of what it really does?)

    The delay, by the way, seemed to have been due to a spam score that triggered graylisting. I guess I have to make myself more acquainted with rspamd...

    Now from the same series of tests yesterday I'm still missing two mails I guess I should have received. They never showed up in the rspamd stats/history, either. As they were generated from a web forum I do not run myself, I cannot make 100% sure they were even sent, but it's a renowned forum from a website owned by Amazon, running on Amazon's servers, and I've never missed an e-mail before from it. Could it somehow be that messages completely fail to be received?

    Perhaps in that context, I see error messages accumulating in the rspamd history on what I've been calling server #1, during the past six hours I got 8 of the first one and 2 of the second:
    Code:
    redis_cache_cb received error: Connection refused
    Code:
    learn error: Unknown statistics error, found when storing data on backend; classifier: bayes
     
    Last edited: May 23, 2020
  4. till

    till Super Moderator Staff Member ISPConfig Developer

    Have a look at the mail.log file and try to find the missing emails there. if they are not in mail.log, then your server probably never received it. if they show up in mail.log, then check which actions the server did with that email according to the log.
     
  5. Etcetera

    Etcetera Member HowtoForge Supporter

    Ok, I'll be checking that...

    In the meantime, I keep getting messages with the subject being rewritten to include the *** SPAM *** notice although I have chosen the "Header" method for both mail domain and mailbox. This is still on "server #1". Also, the same message is being processed a second time on the same server, with its subject already altered and its "to" envelope set to the redirection address by Sieve. The score changed from "8.28 / 6" to "8.19 / 15", and while the first event's action was logged as "rewrite subject", the second one now is "add header". I confess I don't completely follow what's going there, but it surely is not what I'd like to happen ;-)
     
  6. till

    till Super Moderator Staff Member ISPConfig Developer

    The message should be processed only once, there is probably something wrong in your postfix config. Please post the content of you main.cf file, you can mask the domain names and iP's in there as they don't matter for this.
     
  7. Etcetera

    Etcetera Member HowtoForge Supporter

    Code:
    # See /usr/share/postfix/main.cf.dist for a commented, more complete version
    
    
    # Debian specific:  Specifying a file name will cause the first
    # line of that file to be used as the name.  The Debian default
    # is /etc/mailname.
    #myorigin = /etc/mailname
    
    smtpd_banner = $myhostname ESMTP $mail_name (Ubuntu)
    biff = no
    
    # appending .domain is the MUA's job.
    append_dot_mydomain = no
    
    # Uncomment the next line to generate "delayed mail" warnings
    #delay_warning_time = 4h
    
    readme_directory = /usr/share/doc/postfix
    
    # TLS parameters
    smtpd_tls_cert_file = /etc/postfix/smtpd.cert
    smtpd_tls_key_file = /etc/postfix/smtpd.key
    smtpd_use_tls = yes
    smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache
    smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache
    
    # See /usr/share/doc/postfix/TLS_README.gz in the postfix-doc package for
    # information on enabling SSL in the smtp client.
    
    smtpd_relay_restrictions = permit_mynetworks permit_sasl_authenticated defer_unauth_destination
    myhostname = mailhost.######.de
    alias_maps = hash:/etc/aliases, hash:/var/lib/mailman/data/aliases
    alias_database = hash:/etc/aliases, hash:/var/lib/mailman/data/aliases
    myorigin = /etc/mailname
    mydestination = mailhost.######.de, localhost, localhost.localdomain
    relayhost =
    mynetworks = 127.0.0.0/8 [::1]/128
    mailbox_size_limit = 0
    recipient_delimiter = +
    inet_interfaces = all
    inet_protocols = all
    html_directory = /usr/share/doc/postfix/html
    virtual_alias_domains =
    virtual_alias_maps = hash:/var/lib/mailman/data/virtual-mailman, proxy:mysql:/etc/postfix/mysql-virtual_forwardings.cf, proxy:mysql:/etc/postfix/mysql-virtual_email2email.cf
    virtual_mailbox_domains = proxy:mysql:/etc/postfix/mysql-virtual_domains.cf
    virtual_mailbox_maps = proxy:mysql:/etc/postfix/mysql-virtual_mailboxes.cf
    virtual_mailbox_base = /var/vmail
    virtual_uid_maps = mysql:/etc/postfix/mysql-virtual_uids.cf
    virtual_gid_maps = mysql:/etc/postfix/mysql-virtual_gids.cf
    sender_bcc_maps = proxy:mysql:/etc/postfix/mysql-virtual_outgoing_bcc.cf
    smtpd_sasl_auth_enable = yes
    broken_sasl_auth_clients = yes
    smtpd_sasl_authenticated_header = yes
    smtpd_restriction_classes = greylisting
    greylisting = check_policy_service inet:127.0.0.1:10023
    smtpd_recipient_restrictions = permit_mynetworks, permit_sasl_authenticated, reject_unauth_destination, reject_rbl_client zen.spamhaus.org, reject_rbl_client ix.dnsbl.manitu.net, check_recipient_access mysql:/etc/postfix/mysql-virtual_recipient.cf, check_recipient_access mysql:/etc/postfix/mysql-virtual_policy_greylist.cf
    smtpd_tls_security_level = may
    transport_maps = hash:/var/lib/mailman/data/transport-mailman, proxy:mysql:/etc/postfix/mysql-virtual_transports.cf
    relay_domains = mysql:/etc/postfix/mysql-virtual_relaydomains.cf
    relay_recipient_maps = mysql:/etc/postfix/mysql-virtual_relayrecipientmaps.cf
    smtpd_sender_login_maps = proxy:mysql:/etc/postfix/mysql-virtual_sender_login_maps.cf
    proxy_read_maps = $local_recipient_maps $mydestination $virtual_alias_maps $virtual_alias_domains $sender_bcc_maps $virtual_mailbox_maps $virtual_mailbox_domains $relay_recipient_maps $relay_domains $canonical_maps $sender_canonical_maps $recipient_canonical_maps $relocated_maps $transport_maps $mynetworks $smtpd_sender_login_maps
    smtpd_helo_required = yes
    smtpd_helo_restrictions = permit_sasl_authenticated, permit_mynetworks, check_helo_access regexp:/etc/postfix/helo_access, reject_invalid_hostname, reject_non_fqdn_hostname, reject_invalid_helo_hostname, reject_unknown_helo_hostname, check_helo_access regexp:/etc/postfix/blacklist_helo
    smtpd_sender_restrictions = check_sender_access mysql:/etc/postfix/mysql-virtual_sender.cf, permit_mynetworks, permit_sasl_authenticated
    smtpd_client_restrictions = check_client_access mysql:/etc/postfix/mysql-virtual_client.cf
    smtpd_client_message_rate_limit = 100
    maildrop_destination_concurrency_limit = 1
    maildrop_destination_recipient_limit = 1
    virtual_transport = dovecot
    header_checks = regexp:/etc/postfix/header_checks
    mime_header_checks = regexp:/etc/postfix/mime_header_checks
    nested_header_checks = regexp:/etc/postfix/nested_header_checks
    body_checks = regexp:/etc/postfix/body_checks
    owner_request_special = no
    smtp_tls_security_level = may
    smtpd_tls_mandatory_protocols = !SSLv2, !SSLv3
    smtpd_tls_protocols = !SSLv2,!SSLv3
    smtp_tls_protocols = !SSLv2,!SSLv3
    smtpd_tls_exclude_ciphers = RC4, aNULL
    smtp_tls_exclude_ciphers = RC4, aNULL
    dovecot_destination_recipient_limit = 1
    smtpd_sasl_type = dovecot
    smtpd_sasl_path = private/auth
    message_size_limit = 0
    smtpd_milters = inet:localhost:11332
    non_smtpd_milters = inet:localhost:11332
    milter_protocol = 6
    milter_mail_macros = i {mail_addr} {client_addr} {client_name} {auth_authen}
    milter_default_action = accept
     
  8. Etcetera

    Etcetera Member HowtoForge Supporter

    Is there any more information I could provide to solve this?
    As of now I need to switch back to amavisd-new; I guess all I have to do for this is to re-enable the amavisd-service and flip the switch in ISPConfig's server configuration?
     
  9. till

    till Super Moderator Staff Member ISPConfig Developer

    main.cf looks fine as far as I can see.

    Yes, that should be enough.
     
  10. Etcetera

    Etcetera Member HowtoForge Supporter

    Thanks.
    (On a side note, I see rspamd inserts two headers, X-Spam-Level and X-Spamd-Bar. Shouldn't the latter rather be X-Spam-Bar, without the 'd'?)
     
  11. Etcetera

    Etcetera Member HowtoForge Supporter

    Thanks, this is solved, and sorry, my error – in the meantime I've confirmed that the missing emails never reached my server...
     
  12. Etcetera

    Etcetera Member HowtoForge Supporter

    This has been cleared up, too, in another thread, it's because by default rspamd processes both incoming and outgoing mail, therefore redirected mail will be scanned twice if I don't prevent it somehow in the configutation or by postfix-whitelisting the addresses for which mail is to be redirected.
     

Share This Page