Spam Filtering with Amavis

Discussion in 'Installation/Configuration' started by Bob White, Sep 24, 2020.

  1. Bob White

    Bob White Member

    I'm getting a bunch of spam that's base 64 encoded. They all have links that point to a common site - hubspotfree.net. I can enter the text, encoded in base 64, into Postfix's body checks file - and this will filter it out - but I have to enter the text three different times (because it's base 64) and then have to play with it to make sure I've got the right bytes entered. Spamassassin supposedly filters after it's decoded, but we're not using SA in this configuration - we're using Amavis. I've tried digging through the Amavis configuration files to see if that's configured to filter it and where I might put the text I want to filter by. I do have spam filtering enabled in ISP Config 3.1, and I've tried entering the text as /hubspotfree/ in the ISP Config screen, but either amavis is filtering before it's decoded and therefore doesn't match, or I'm doing something else wrong. How is this supposed to work?
    Thanks,
    Bob
     
  2. till

    till Super Moderator Staff Member ISPConfig Developer

    That's not correct. You use Spamassassin as Amavis is using Spamassassin. Amavis is just a high performance wrapper which internally uses Spamassassin, this means all settings that you do for spamasssain e.g. in spamassassin config files in /etc/spamassassin/... get applied to your amavis scanner automatically.
     
  3. Bob White

    Bob White Member

    So, I don't enter the filter into ISP Config, but in the SA config files in /etc/spamassassin? I put the following into /etc/spamassassin/local.cf:
    body BW_HUBSPOTFREE /t.hubspotfree/i
    score BW_HUBSPOTFREE 4
    describe BW_HUBSPOTFREE Block stuff with links to hubspotfree​
    I sent myself an email message containing "t.hubspotfree" in the body, but it wasn't flagged when the email came through. What am I doing wrong? Is there another SA setting I need to change?
    Bob
     
  4. till

    till Super Moderator Staff Member ISPConfig Developer

    Personally, I won't do that. I just answered the question where you claimed that SpamAssassin is not used.
     
  5. Bob White

    Bob White Member

    Okay, fair enough. How would you apply spam filters to incoming email? Spam is coming through despite what Amavis is doing. I know how to filter out the stuff I don't want. Where do I apply the filter?
    Bob
     
  6. Jesse Norell

    Jesse Norell Well-Known Member Staff Member Howtoforge Staff

    /etc/spamassassin/local.cf works fine for defining your own rules. Did you restart amavis after adding your rule?
     
  7. till

    till Super Moderator Staff Member ISPConfig Developer

    Did you check the email header to see which score it got and which filters match?
    Did you lower the spam tag 2 level in the spamfilter policy that you applied to the mail account, the defaults might be too high for your case.
     
  8. Jesse Norell

    Jesse Norell Well-Known Member Staff Member Howtoforge Staff

    As to your rule, do you suspect hubspotfree.net has any legitimate use, or only spam? For those domains (including top level domains) for which uri's are only spam, use blacklist_uri_host, eg.
    Code:
    blacklist_uri_host pw
    blacklist_uri_host link
    blacklist_uri_host science
    blacklist_uri_host work
    blacklist_uri_host click
    blacklist_uri_host rocks
    ...
    blacklist_uri_host hubspotfree.net
    
    If hubspotfree.net does have legitimate use, then you need a rule that just adds a score, like you're doing. If you know it's in a uri, you can use the 'uri' test instead of 'body', maybe something like:
    Code:
    uri BW_HUBSPOTFREE /\bhubspotfree\.net($|\/)/i
    score BW_HUBSPOTFREE 4
    describe BW_HUBSPOTFREE Block stuff with links to hubspotfree.net
    
     
    till likes this.
  9. Jesse Norell

    Jesse Norell Well-Known Member Staff Member Howtoforge Staff

    If you did restart amavis and your rule still isn't working, check if you have this set in local.cf:
    Code:
    normalize_charset 1
     
  10. Bob White

    Bob White Member

    Yes, I checked the email header. That's how I know it wasn't flagging it. The emails were already getting about a 3.5 spam score, so even the default limit of 4.5 should have been exceeded if the filter was actually getting applied. I'll try it again.
     
  11. till

    till Super Moderator Staff Member ISPConfig Developer

    You should lower the tag 2 level score, I use e.g. 3.05 here for years now. 4.5 is the default and meant to be very conservative.
     
  12. Bob White

    Bob White Member

    I'm getting ready to retry the test and restart amavis before sending the email. The normalize_charset statement isn't in local.cf, but it will be shortly. Where does the blacklist_uri_host statement go? local.cf, or another file?
    Thanks!
    Bob
     
  13. Jesse Norell

    Jesse Norell Well-Known Member Staff Member Howtoforge Staff

    local.cf is fine. For general spamassassin config and rules info they have good documentation and an active users list, though you might just browse some existing/similar rules for ideas as well. Eg. look under /var/lib/spamassassin/ and examples like https://mcgrail.com/downloads/KAM.cf
     
  14. Bob White

    Bob White Member

    Okay, local.cf was set to flag /hubspotfree/i and assign it a score of 4 if found. I restarted amavis. I logged into another email account and sent myself a message with "hubspotfree" in the body of the message. It was not flagged when I received the email - here's that part of the header.

    X-Spam-Status: No, score=3.746 tagged_above=1 required=4.5
    tests=[HEADER_FROM_DIFFERENT_DOMAINS=0.249, HTML_MESSAGE=0.001,
    MISSING_MIMEOLE=1.843, NO_DNS_FOR_FROM=0.379, RDNS_NONE=1.274]
    autolearn=no autolearn_force=no
     
  15. Bob White

    Bob White Member

    From /etc/spamassassin/local.cf:
    normalize_charset 1

    body BW_HUBSPOTFREE /hubspotfree/i
    score BW_HUBSPOTFREE 4
    describe BW_HUBSPOTFREE Block stuff with links to hubspotfree
     
  16. Jesse Norell

    Jesse Norell Well-Known Member Staff Member Howtoforge Staff

    I copy/pasted your rule to local.cf, restarted amavis and tested, it worked fine for me:
    Code:
    X-Spam-Status: No, score=1.902 tagged_above=1 required=4.5
            tests=[BAYES_00=-1.9, BW_HUBSPOTFREE=4, DKIM_SIGNED=0.1,
            DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1,
            FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001,
            RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_PASS=-0.001]
            autolearn=no autolearn_force=no
    
     
  17. Bob White

    Bob White Member

    I just restarted amavis and will try again. Is there anything else I need to do?
    # service amavis restart
    # service amavis status
    ● amavis.service - LSB: Starts amavisd-new mailfilter
    Loaded: loaded (/etc/init.d/amavis; generated)
    Active: active (running) since Thu 2020-09-24 12:16:55 EDT; 4s ago
    Docs: man:systemd-sysv-generator(8)
    Process: 29314 ExecStop=/etc/init.d/amavis stop (code=exited, status=0/SUCCESS)
    Process: 29323 ExecStart=/etc/init.d/amavis start (code=exited, status=0/SUCCESS)
    Tasks: 3 (limit: 4915)
    CGroup: /system.slice/amavis.service
    ├─29339 /usr/sbin/amavisd-new (master)
    ├─29340 /usr/sbin/amavisd-new (virgin child)
    └─29341 /usr/sbin/amavisd-new (virgin child)

    Sep 24 12:16:55 fileserver amavis[29339]: No decoder for .jar
    Sep 24 12:16:55 fileserver amavis[29339]: No decoder for .lha
    Sep 24 12:16:55 fileserver amavis[29339]: No decoder for .lrz
    Sep 24 12:16:55 fileserver amavis[29339]: No decoder for .lz4
    Sep 24 12:16:55 fileserver amavis[29339]: No decoder for .rar
    Sep 24 12:16:55 fileserver amavis[29339]: No decoder for .rpm
    Sep 24 12:16:55 fileserver amavis[29339]: No decoder for .swf
    Sep 24 12:16:55 fileserver amavis[29339]: No decoder for .zoo
    Sep 24 12:16:55 fileserver amavis[29339]: Using primary internal av scanner code for ClamAV-clamd
    Sep 24 12:16:55 fileserver amavis[29339]: Found secondary av scanner ClamAV-clamscan at /usr/bin/clamscan
     
  18. Bob White

    Bob White Member

    In /etc/amavis/conf.d/15-content_filter_mode, I did make one change - I uncommented two lines because it said that spam testing would be disabled unless they were uncommented:

    #
    # Default SPAM checking mode
    # Please note, that anti-spam checking is DISABLED by
    # default.
    # If You wish to enable it, please uncomment the following lines:


    @bypass_spam_checks_maps = (
    \%bypass_spam_checks, \@bypass_spam_checks_acl, \$bypass_spam_checks_re);
     
  19. Bob White

    Bob White Member

    Well, I'm not sure what changed, but it flagged it that time.
    X-Spam-Status: Yes, score=7.747 tagged_above=1 required=4.5
    tests=[BW_HUBSPOTFREE=4, HEADER_FROM_DIFFERENT_DOMAINS=0.249,
    HTML_MESSAGE=0.001, MISSING_MIMEOLE=1.843, NO_DNS_FOR_FROM=0.379,
    RDNS_NONE=1.274, TVD_SPACE_RATIO=0.001]
    autolearn=no autolearn_force=no
     
  20. Bob White

    Bob White Member

    Just to be clear, the way to filter spam like this is to modify the SA local.cf file. There's no place in ISP Config that I could enter a regex and filter email, correct?
    Even with a successful spam filter yesterday, I had some real spam come through today. Not sure why it didn't catch it. When I forwarded the email back to myself, it was successfully flagged as spam. I just wish it would do that on the first pass!

    Bob
     

Share This Page