Spamassassin does not scan all messages

Discussion in 'Installation/Configuration' started by ircf, Oct 17, 2014.

  1. ircf

    ircf Member

    Hello,

    Our 2 mail servers run Debian 7.0, ISPConfig 3.0.5.4p3, Postfix, Amavis, Spamassassin, Cluebringer.

    Something weird happens : about one message over two is not scanned by spamassassin (it randomly happens on mail1 or mail2 server).

    I can't find why and it's driving me crazy... Does anyone have a hint please ?

    We have whitelisted our own servers on spamassassin local.cf, but it seems like spamassassin randomly whitelist any server.

    Should I removed our own servers from the SA whitelist ?

    Here is a header of a mail that passed without being scanned :

    Code:
    Return-Path: <[email protected]>
    Delivered-To: [email protected]
    Received: from localhost (localhost [127.0.0.1])
    	by mail2.example.com (Postfix) with ESMTP id AD0B4C2773
    	for <[email protected]>; Fri, 17 Oct 2014 09:22:45 +0200 (CEST)
    X-Virus-Scanned: Debian amavisd-new at mail2.example.com
    Received: from mail2.example.com ([127.0.0.1])
    	by localhost (mail2.example.com [127.0.0.1]) (amavisd-new, port 10024)
    	with ESMTP id X9x9hZVAFIsW for <[email protected]>;
    	Fri, 17 Oct 2014 09:22:44 +0200 (CEST)
    Received: from mail5118.mymxserver.com (mail5118.mymxserver.com [185.15.192.53])
    	by mail2.example.com (Postfix) with ESMTP id DEAABC2775
    	for <[email protected]>; Fri, 17 Oct 2014 09:22:43 +0200 (CEST)
    Received: from localhost (localhost [127.0.0.1])
    	by localhost.mx51.mymxserver.com (Postfix) with ESMTP id 5D7A11114A0
    	for <[email protected]>; Fri, 17 Oct 2014 09:22:42 +0200 (CEST)
    X-Virus-Scanned: by Mittwald Mailscanner
    Received: from mail5118.mymxserver.com ([127.0.0.1])
    	by localhost (mx51.mymxserver.com [127.0.0.1]) (amavisd-new, port 10024)
    	with ESMTP id R8QTHBEQwm-u for <[email protected]>;
    	Fri, 17 Oct 2014 09:22:42 +0200 (CEST)
    Received: from i59F56071.versanet.de (i59F57DA4.versanet.de [89.245.125.164])
    	(Authenticated sender: p167260p1)
    	by mail5118.mymxserver.com (Postfix) with ESMTP id 79A2A1114C7
    	for <[email protected]>; Fri, 17 Oct 2014 09:22:41 +0200 (CEST)
    MIME-Version: 1.0
    From: "DELTA CHECK" <[email protected]>
    Reply-To: [email protected]
    To: [email protected]
    Subject: 71.050 star rated hotels for only 499 EUR... YES! more than 25 address packets for radically reduced prices.
    Content-Type: multipart/alternative;
    	boundary="----=_NextPart_001_546D_16414BBC.3C56719D"
    X-Mailer: Smart_Send_2_0_138
    Date: Fri, 17 Oct 2014 09:22:37 +0200
    Message-ID: <67281911676721995920491@AhmedEl-Masry>
    ...
    
    Postfix main.cf :
    Code:
    root@mail1:~# postconf -n
    alias_database = hash:/etc/aliases, hash:/var/lib/mailman/data/aliases
    alias_maps = hash:/etc/aliases, hash:/var/lib/mailman/data/aliases
    append_dot_mydomain = no
    biff = no
    body_checks = regexp:/etc/postfix/body_checks
    broken_sasl_auth_clients = yes
    config_directory = /etc/postfix
    content_filter = amavis:[127.0.0.1]:10024
    disable_vrfy_command = yes
    dovecot_destination_recipient_limit = 1
    header_checks = regexp:/etc/postfix/header_checks
    inet_interfaces = all
    inet_protocols = all
    mailbox_size_limit = 0
    message_size_limit = 262144000
    mime_header_checks = regexp:/etc/postfix/mime_header_checks
    mydestination = mail1.example.com, localhost, localhost.localdomain
    myhostname = mail1.example.com
    mynetworks = 127.0.0.0/8 [::1]/128
    myorigin = /etc/mailname
    nested_header_checks = regexp:/etc/postfix/nested_header_checks
    owner_request_special = no
    proxy_read_maps = $local_recipient_maps $mydestination $virtual_alias_maps $virtual_alias_domains $virtual_mailbox_maps $virtual_mailbox_domains $relay_recipient_maps $relay_domains $canonical_maps $sender_canonical_maps $recipient_canonical_maps $relocated_maps $transport_maps $mynetworks $virtual_mailbox_limit_maps
    receive_override_options = no_address_mappings
    recipient_delimiter = +
    relay_domains = mysql:/etc/postfix/mysql-virtual_relaydomains.cf
    relay_recipient_maps = mysql:/etc/postfix/mysql-virtual_relayrecipientmaps.cf
    smtp_connect_timeout = 10s
    smtp_helo_timeout = 100s
    smtpd_banner = $myhostname ESMTP
    smtpd_delay_reject = yes
    smtpd_end_of_data_restrictions = check_policy_service inet:127.0.0.1:10031
    smtpd_error_sleep_time = 0
    smtpd_helo_required = yes
    smtpd_recipient_restrictions = check_policy_service inet:127.0.0.1:10031, permit_mynetworks, permit_sasl_authenticated, reject_unauth_pipelining, reject_invalid_hostname, reject_non_fqdn_hostname, reject_non_fqdn_sender, reject_non_fqdn_recipient, reject_unknown_sender_domain, reject_unauth_destination, reject_unknown_recipient_domain, check_client_access mysql:/etc/postfix/mysql-virtual_client.cf, check_sender_access mysql:/etc/postfix/mysql-virtual_sender.cf, check_recipient_access mysql:/etc/postfix/mysql-virtual_recipient.cf, reject_rbl_client zen.spamhaus.org, reject_rbl_client b.barracudacentral.org, permit
    smtpd_sasl_auth_enable = yes
    smtpd_sasl_authenticated_header = yes
    smtpd_sasl_path = private/auth
    smtpd_sasl_type = dovecot
    smtpd_tls_security_level = may
    smtpd_use_tls = yes
    transport_maps = hash:/var/lib/mailman/data/transport-mailman, proxy:mysql:/etc/postfix/mysql-virtual_transports.cf
    virtual_alias_domains =
    virtual_alias_maps = proxy:mysql:/etc/postfix/mysql-virtual_forwardings.cf, proxy:mysql:/etc/postfix/mysql-virtual_email2email.cf, hash:/var/lib/mailman/data/virtual-mailman
    virtual_gid_maps = static:5000
    virtual_mailbox_base = /var/vmail
    virtual_mailbox_domains = proxy:mysql:/etc/postfix/mysql-virtual_domains.cf
    virtual_mailbox_maps = proxy:mysql:/etc/postfix/mysql-virtual_mailboxes.cf
    virtual_transport = dovecot
    virtual_uid_maps = static:5000
    
    Postfix master.cf :
    Code:
    #
    # Postfix master process configuration file.  For details on the format
    # of the file, see the master(5) manual page (command: "man 5 master").
    #
    # Do not forget to execute "postfix reload" after editing this file.
    #
    # ==========================================================================
    # service type  private unpriv  chroot  wakeup  maxproc command + args
    #               (yes)   (yes)   (yes)   (never) (100)
    # ==========================================================================
    smtp      inet  n       -       -       -       -       smtpd
    #smtp      inet  n       -       -       -       1       postscreen
    #smtpd     pass  -       -       -       -       -       smtpd
    #dnsblog   unix  -       -       -       -       0       dnsblog
    #tlsproxy  unix  -       -       -       -       0       tlsproxy
    submission inet n       -       -       -       -       smtpd
      -o content_filter=amavis:[127.0.0.1]:10026
      -o syslog_name=postfix/submission
      -o smtpd_sasl_auth_enable=yes
      -o smtpd_client_restrictions=permit_sasl_authenticated,reject
    #submission inet n       -       -       -       -       smtpd
    #  -o syslog_name=postfix/submission
    #  -o smtpd_tls_security_level=encrypt
    #  -o smtpd_sasl_auth_enable=yes
    #  -o smtpd_client_restrictions=permit_sasl_authenticated,reject
    #  -o milter_macro_daemon_name=ORIGINATING
    #smtps     inet  n       -       -       -       -       smtpd
    #  -o syslog_name=postfix/smtps
    #  -o smtpd_tls_wrappermode=yes
    #  -o smtpd_sasl_auth_enable=yes
    #  -o smtpd_client_restrictions=permit_sasl_authenticated,reject
    #  -o milter_macro_daemon_name=ORIGINATING
    #628       inet  n       -       -       -       -       qmqpd
    pickup    fifo  n       -       -       60      1       pickup
    cleanup   unix  n       -       -       -       0       cleanup
    qmgr      fifo  n       -       n       300     1       qmgr
    #qmgr     fifo  n       -       n       300     1       oqmgr
    tlsmgr    unix  -       -       -       1000?   1       tlsmgr
    rewrite   unix  -       -       -       -       -       trivial-rewrite
    bounce    unix  -       -       -       -       0       bounce
    defer     unix  -       -       -       -       0       bounce
    trace     unix  -       -       -       -       0       bounce
    verify    unix  -       -       -       -       1       verify
    flush     unix  n       -       -       1000?   0       flush
    proxymap  unix  -       -       n       -       -       proxymap
    proxywrite unix -       -       n       -       1       proxymap
    smtp      unix  -       -       -       -       -       smtp
    relay     unix  -       -       -       -       -       smtp
    #       -o smtp_helo_timeout=5 -o smtp_connect_timeout=5
    showq     unix  n       -       -       -       -       showq
    error     unix  -       -       -       -       -       error
    retry     unix  -       -       -       -       -       error
    discard   unix  -       -       -       -       -       discard
    local     unix  -       n       n       -       -       local
    virtual   unix  -       n       n       -       -       virtual
    lmtp      unix  -       -       -       -       -       lmtp
    anvil     unix  -       -       -       -       1       anvil
    scache    unix  -       -       -       -       1       scache
    #
    # ====================================================================
    # Interfaces to non-Postfix software. Be sure to examine the manual
    # pages of the non-Postfix software to find out what options it wants.
    #
    # Many of the following services use the Postfix pipe(8) delivery
    # agent.  See the pipe(8) man page for information about ${recipient}
    # and other message envelope options.
    # ====================================================================
    #
    # maildrop. See the Postfix MAILDROP_README file for details.
    # Also specify in main.cf: maildrop_destination_recipient_limit=1
    #
    maildrop  unix  -       n       n       -       -       pipe
      flags=DRhu user=vmail argv=/usr/bin/maildrop -d vmail ${extension} ${recipient} ${user} ${nexthop} ${sender}
    #
    # ====================================================================
    #
    # Recent Cyrus versions can use the existing "lmtp" master.cf entry.
    #
    # Specify in cyrus.conf:
    #   lmtp    cmd="lmtpd -a" listen="localhost:lmtp" proto=tcp4
    #
    # Specify in main.cf one or more of the following:
    #  mailbox_transport = lmtp:inet:localhost
    #  virtual_transport = lmtp:inet:localhost
    #
    # ====================================================================
    #
    # Cyrus 2.1.5 (Amos Gouaux)
    # Also specify in main.cf: cyrus_destination_recipient_limit=1
    #
    #cyrus     unix  -       n       n       -       -       pipe
    #  user=cyrus argv=/cyrus/bin/deliver -e -r ${sender} -m ${extension} ${user}
    #
    # ====================================================================
    # Old example of delivery via Cyrus.
    #
    #old-cyrus unix  -       n       n       -       -       pipe
    #  flags=R user=cyrus argv=/cyrus/bin/deliver -e -m ${extension} ${user}
    #
    # ====================================================================
    #
    # See the Postfix UUCP_README file for configuration details.
    #
    uucp      unix  -       n       n       -       -       pipe
      flags=Fqhu user=uucp argv=uux -r -n -z -a$sender - $nexthop!rmail ($recipient)
    #
    # Other external delivery methods.
    #
    ifmail    unix  -       n       n       -       -       pipe
      flags=F user=ftn argv=/usr/lib/ifmail/ifmail -r $nexthop ($recipient)
    bsmtp     unix  -       n       n       -       -       pipe
      flags=Fq. user=bsmtp argv=/usr/lib/bsmtp/bsmtp -t$nexthop -f$sender $recipient
    scalemail-backend unix	-	n	n	-	2	pipe
      flags=R user=scalemail argv=/usr/lib/scalemail/bin/scalemail-store ${nexthop} ${user} ${extension}
    mailman   unix  -       n       n       -       -       pipe
      flags=FR user=list argv=/usr/lib/mailman/bin/postfix-to-mailman.py
      ${nexthop} ${user}
    
    dovecot   unix  -       n       n       -       -       pipe
      flags=DRhu user=vmail:vmail argv=/usr/lib/dovecot/deliver -f ${sender} -d ${user}@${nexthop}
    
    amavis unix - - - - 4 smtp
            -o smtp_data_done_timeout=1200
            -o smtp_send_xforward_command=yes
    
    127.0.0.1:10025 inet n - - - - smtpd
            -o content_filter=
            -o local_recipient_maps=
            -o relay_recipient_maps=
            -o smtpd_restriction_classes=
            -o smtpd_client_restrictions=
            -o smtpd_helo_restrictions=
            -o smtpd_sender_restrictions=
            -o smtpd_recipient_restrictions=permit_mynetworks,reject
            -o mynetworks=127.0.0.0/8
            -o strict_rfc821_envelopes=yes
            -o receive_override_options=no_unknown_recipient_checks,no_header_body_checks
    
    policy-spf  unix  -       n       n       -       -       spawn
         user=nobody argv=/usr/bin/policyd-spf
    Amavis 50-user :
    Code:
    use strict;
    
    #
    # Place your configuration directives here.  They will override those in
    # earlier files.
    #
    # See /usr/share/doc/amavisd-new/ for documentation and examples of
    # the directives you can use in this file
    #
    
    @bypass_virus_checks_maps = (
       \%bypass_virus_checks, \@bypass_virus_checks_acl, \$bypass_virus_checks_re);
    
    @bypass_spam_checks_maps = (
       \%bypass_spam_checks, \@bypass_spam_checks_acl, \$bypass_spam_checks_re);
    
    #   
    # Database connection settings
    #
    
    @lookup_sql_dsn =
       ( ['DBI:mysql:database=dbispconfig;host=127.0.0.1;port=3306', 'ispconfig', 'fake-password'] );
    
    # @storage_sql_dsn = @lookup_sql_dsn;  # none, same, or separate database
    #$sql_select_policy = 'SELECT "Y" as local FROM mail_domain WHERE CONCAT("@",domain) IN (%k)';
    # $banned_files_quarantine_method = 'sql';
    # $spam_quarantine_method         = 'sql';
    
    #
    # SQL Select statements
    #
    
    $sql_select_policy =
       'SELECT *,spamfilter_users.id'.
       ' FROM spamfilter_users LEFT JOIN spamfilter_policy ON spamfilter_users.policy_id=spamfilter_policy.id'.
       ' WHERE spamfilter_users.email IN (%k) ORDER BY spamfilter_users.priority DESC';
    
    
    $sql_select_white_black_list = 'SELECT wb FROM spamfilter_wblist'.
        ' WHERE (spamfilter_wblist.rid=?) AND (spamfilter_wblist.email IN (%k))' .
        ' ORDER BY spamfilter_wblist.priority DESC';
    
    #
    # Quarantine settings
    #
    
    $final_virus_destiny = D_BOUNCE;
    $final_spam_destiny = D_DISCARD;
    $final_banned_destiny = D_BOUNCE;
    $final_bad_header_destiny = D_PASS;
    
    # Default settings, we st this very high to not filter aut emails accidently
    $sa_spam_subject_tag = '';
    $sa_tag_level_deflt  = 2.0;  # add spam info headers if at, or above that level
    $sa_tag2_level_deflt = 3.0; # add 'spam detected' headers at that level
    $sa_kill_level_deflt = 6.0; # triggers spam evasive actions
    $sa_dsn_cutoff_level = 100;   # spam level beyond which a DSN is not sent
    
    #
    # Disable spam and virus notifications for the admin user.
    # Can be overridden by the policies in mysql
    #
    
    $virus_admin = undef;
    $spam_admin = undef;
    
    
    #
    # Enable Logging
    #
    
    $DO_SYSLOG = 1;
    $LOGFILE = "/var/log/amavis.log";  # (defaults to empty, no log)
    
    # Set the log_level to 5 for debugging
    $log_level = 0;                # (defaults to 0)
    
    # Set max_servers (default: 2)
    $max_servers = 4;
    
    # SASL policy
    $inet_socket_port = [10024, 10026];
    
    $interface_policy{'10026'} = 'SASL';
    
    $policy_bank{'SASL'} = {  # mail from submission and smtps ports
       final_spam_destiny => D_BOUNCE, # send error mail
       spam_tag_level_maps => [6.0], 
       spam_kill_level_maps => [7.0],
    };
    
    #------------ Do not modify anything below this line -------------
    1;  # insure a defined return
     
    Last edited: Oct 17, 2014
  2. till

    till Super Moderator Staff Member ISPConfig Developer

    The mail you posted above has been scanned by spamassassin. the header that gets added is:

    X-Virus-Scanned: Debian amavisd-new at mail2.example.com

    as spamassasin is embedded into amavisd, so no sepaate spamassassin header is added.

    If you dnt see a spam scaore then this does not indicate that the message has not been scanned, it just indicates that the resulting spamassassin score is below the tag1 level.
     
  3. ircf

    ircf Member

    Thank you !

    Thank you Till :)

    I thought that SA would always write spam score, sorry...

    I will set a lower tag1 level to see it. Thank you again :)
     
  4. till

    till Super Moderator Staff Member ISPConfig Developer

    If you want always get a sa score, set tag1 level to e.g. -100
     

Share This Page