Clarification of Spam Filter Policies in ISPConfig3

Discussion in 'Installation/Configuration' started by atjensen11, Aug 25, 2009.

  1. atjensen11

    atjensen11 New Member

    I am hoping to get some clarification on the spam filter policies settings within ISPConfig3. I have the default policies which are:

    Non-paying
    Uncensored
    Wants all spam
    Wants viruses
    Normal
    Trigger happy
    Permissive

    Upon initial glance, there are a lot of red X icons showing. I am not sure if this is a good thing or a bad thing.

    I have some users that are complaining that they are receiving a higher number of spam emails on the new ISPConfig3 system than the previous system. The previous system was a Virtual user system on Ubuntu 8.04 LTS using a How To on this site.

    Is there a policy on the ISPConfig3 system that is similar to the system established in the virtual user How To that could be used as a default? Can someone explain they checks done by each setting? Would anyone be willing to share their other spam filter settings?
     
  2. till

    till Super Moderator Staff Member ISPConfig Developer

    The policys are the default ones from amavisd-new. So they are just examples that you can change to suit your needs. For detailed information on ecah option, please take a look at the amavisd-new documentation.

    If you receive more spam, just set the spam level to a lower value.
     
  3. atjensen11

    atjensen11 New Member

    OK, I will check the documentation and try to get up to speed.

    One last question regarding the policies...When I create the email domain, I leave the spam filter policy setting there as "-Not Enabled-". Then when I create the email account, I enable the spam filter policy "Normal".

    Users have stated that changing from "-Not Enabled-" to "Normal" at the email account level results in no discernable reduction of spam in their Inbox.

    Do I have to enable the spam filter policies at both the domain and account levels? What heirarchy applies? What are the rules that apply when spam filter policies are enabled at both levels? Does the more restrictive policy win out? Does the email account policy level always override the domain policy if configured?

    Thanks.
     
  4. till

    till Super Moderator Staff Member ISPConfig Developer

    The setting of the mailbox has priority over the domain. Please see amavisd-new documentation for details as this is all handled by amavisd-new and not ispconfig.
     
  5. atjensen11

    atjensen11 New Member

    OK. I have been doing a lot of reading on amavisd-new lately and studying the different settings within the ISPConfig3 spam filter.

    I have one email account in ISPConfig that is being targeted heavily for SPAM. Furthermore, this user forwards me every message that comes in with a note like "I got another one".

    Here is a header excerpt from one of these forwarded messages:
    Code:
    X-Envelope-From: <[sanitized]@[sanitized].org>
    X-Envelope-To: <[sanitized]@[sanitized].org>
    X-Envelope-To-Blocked: 
    X-Quarantine-ID: <pJdldNwoaAgB>
    X-Amavis-Alert: BAD HEADER SECTION Non-encoded 8-bit data (char A9 hex): From:
    	\251 VIAGRA \256 Offic[...]
    X-Spam-Flag: NO
    X-Spam-Score: 5.628
    X-Spam-Level: *****
    X-Spam-Status: No, score=5.628 tag=3 tag2=6.9 kill=6.9
    	tests=[DYN_RDNS_SHORT_HELO_HTML=0.287, HTML_IMAGE_ONLY_20=1.808,
    	HTML_IMAGE_RATIO_02=0.55, HTML_MESSAGE=0.001,
    	HTML_SHORT_LINK_IMG_3=0.556, MIME_HTML_ONLY=1.672, RDNS_DYNAMIC=0.1,
    	SPF_SOFTFAIL=0.654]
    
    Prior to receiving this email from the user, I had changed the spam filter policy to "Trigger Happy". The tag level settings for Trigger Happy are:

    Code:
    SPAM Tag Level=3
    SPAM Tag2 Level=5
    SPAM Kill Level=5
    SPAM DNS Cutoff Level=0
    SPAM Quarantine Level=0
    SPAM Modifies Subject=Yes
    SPAM Subject Tag= [POSSIBLE SPAM Score=_SCORE_] -
    SPAM Subject Tag2= [SPAM SCORE=_SCORE_]-
    
    But these settings are reflected in the X Headers of the excerpt I posted. I haven't modified any of the amavisd-new configuration files manually. I assume these settings are stored in the DB. Where can I check to verify ISPConfig3 is storing the values correctly?
     
  6. till

    till Super Moderator Staff Member ISPConfig Developer

    The settings are all written correctly, as you wont see them otherwise in ispconfig as amavisd and ispconfig read the same database table. Make sure that you have set:

    $final_spam_destiny = D_DISCARD;

    in amavaisd 50-user file in debian or amavisd.conf in other linux distributions if you want spam to be deleted when it reaches the kill level. Then restart amavisd.
     
  7. bluebirdnet

    bluebirdnet Member

    hi would be interesting to see this thread continue as I am also having issues with spam filtering and trying to grasp the whole amavisd-new and spamassissin and the configuration options in Ispconfig3.

    can anyone elaborate on the spam filter policy ? How can I improve them ?

    thanks
     
  8. frogman

    frogman New Member

    Amavisd-Spamassassin-etc.. Overview

    I agree... I have searched the forums (always awesomely helpful, BTW) for some clarity on the interworkings of of the filter settings and have not found a good complete picture as I usually do.

    I'd be happy to put one together, but I don't understand it yet!

    If there is already something out there and I am missing it, please let me know.. What I think I, and others, are looking for is something that spells out how the settings work together, such as:

    1 - domain filter setting

    2 - user filter setting

    3 - What is Priority 1-10 and the slider really do in relation to the tag levels?

    4 - What happens to the emails at the various tag levels?- I for one am concerned about being too strict on SPAM scores and then a customer wanting an email from Quarantine and then finding it was actually deleted....

    5 - Map this info to the conf files that house the settings?

    Again - sorry if this is already out there, but I could not find it. If I can start to get the answers from the awesome members here, I would be more than happy to build the document for everyone. If its already started - somebody please point me to it..

    Thanks ALL, as always great help...
     
  9. falko

    falko Super Moderator Howtoforge Staff

    The settings for each of these levels are defined under Email > Spamfilter > Policy.
     
  10. latinsud

    latinsud New Member

    Just to make it clear, what happens I define a spam policy at the domain level but leave the user in "Not Enabled"?
    I think it follows the domain policy.

    But, then, isn't he label "Not Enabled" a little confusing? Shouldn't it read "Leave default", "Don't Override", "Not defined" or something like that?
     
  11. Ramm

    Ramm New Member

    Would like more info on this thread too...
     
  12. Jesse Norell

    Jesse Norell Well-Known Member Staff Member Howtoforge Staff

    You are correct in what it does, and that the label is inaccurate/misleading. https://git.ispconfig.org/ispconfig/ispconfig3/issues/4277
     
  13. Broncosis

    Broncosis New Member

    ok I have been lurking here for a long time and until lately I have found most of what I need
    but this stuff is still not very well documented
    so I am going to start on it and hopefully others will add to it hopefully it will help me sort it out in my head and help others as well

    so as it stands now the policy set in the mail box takes priority over the domain setting which takes priority over the system default
    so if your system is set to trigger happy and you have the domain set at Normal and the mail box set to Permissive your going to get what every policy is permissive even though the ones above are possibly more restrictive

    under tag level
    SPAM tag level: this is the vale you give to set the first level of tags by default this is (***Might be Spam***) set this kinda on the low side to catch most spam as with out moving them to another folder or quarantining them or deleting them

    SPAM tag2 level: this tag level by default marks them as spam (***SPAM***) it is also the trigger for the
    Move Spam Emails to Junk directory. switch on the mail filter tab of the mailbox setup page
    allowing you to set this to move all spam at this level to the junk/spam folder

    SPAM kill level: this is the level you set to prevent them from being delivered
    if you go to the quarantine tab you can fill in the Forward spam to email: with a address you can monitor
    in case of false positives

    SPAM dsn cutoff level: this is the level at which you no longer notify the sending server that it can not be delivered
    I set this pretty close to the kill level as some spam systems will use this to identify accounts

    SPAM quarantine cutoff level: this is the point at which it no longer send the e-mail to the quarantine address
    and just deletes it

    SPAM modifies subject: this needs to be set to yes if you want to modify the subject lines if you want all of this to happen with any indication to your users then turn this off

    SPAM subject tag: this is the message set at the first tag level by default this is (***Might be Spam***)
    SPAM subject tag2: this is the tag added to the subject at Spam tag2 by default this is (***SPAM***)

    I hope this help someone I know I stumbled through a bunch of this until now finally taming the spam on my server the joys of a 20 year old domain with mail boxes almost that old to

    on my system to give you a idea of the settings I have
    my trigger happy policy is set up

    SPAM tag level: 0.80
    SPAM tag2 level: 1.40
    SPAM kill level: 3.20
    SPAM dsn cutoff level: 5.0
    SPAM quarantine cutoff level: 8.0
    SPAM modifies subject: yes
    SPAM subject tag: ***Might be Spam***
    SPAM subject tag2: ***SPAM***

    this policy is used mainly by a few accounts that receive over 20 spam per hour
    and it has reduced it to under 20 per day that get tagged might be spam
    this is probably far to aggressive for some but I have had only 2 false positives
    in the last 2 months
     
  14. florian030

    florian030 Well-Known Member HowtoForge Supporter

    SPAM kill level: 3.20 is a little bit low but this depends on your installed rulesets for amavis.
    As a side-note: "If you use an alias, the policy for the domain applies." (Spammail understand header and move mails)
     
  15. Broncosis

    Broncosis New Member

    I don't disagree the 3.2 is a little low hence why its on the tigger happy policy but its on my own e-mail account and I have been tuning it to and watching the results very closely spam has been a nagging problem for the last year I'm just finally taking a serious look at it as most of my users don't have much of a problem
     
  16. florian030

    florian030 Well-Known Member HowtoForge Supporter

    It depends on (additional) rules for spamassassin. We publish our own rules and most of them set the score to 5.
     
  17. Broncosis

    Broncosis New Member

    yeah I have been watching my stuff carefully again it was not meant as a rule more of a demonstration of how they should be tiered to prevent odd behaviour and I'm not the expert but its what is working for my system every one is going to be a bit different hence the tuning part of it
     
  18. LouTux

    LouTux Member

    Hi, I know this thread is getting old, but I am just getting started with spam filtering and was wondering if anyone would share their policy.
    Also, what log and what filter should I use to identify problems. It seems that some of the emails comming out of my ispconfig instance are being filtered by other servers.
     
  19. Jesse Norell

    Jesse Norell Well-Known Member Staff Member Howtoforge Staff

    There's an example earlier in this thread, and likely others found without much effort searching (eg. google pulled up https://www.howtoforge.com/community/threads/ispconfig3-spamfilter-policy-again.72280/ within a few seconds of looking). On a few systems I think I'm using the default that come with ISPConfig, and on other systems I've tweaked the levels.

    In general, selecting a policy with lower levels does not improve the accuracy of your spam filter, it merely makes it "fire" sooner; so both more actual spam is marked as spam, and more non-spam is marked as spam, too. The spamassassin rules that are distributed (and should be updated daily by your system via sa-update) target a threshold of 5, so when you have spam that scores under that (not uncommon), you can look at ways to improve the scanning of those messages, and when you have non-spam that scores over 5 (hopefully fairly uncommon), you look at ways to identify those correctly.

    There's lots of info on the spamassassin site and elsewhere, so no need to repeat it here. Start with https://cwiki.apache.org/confluence/display/SPAMASSASSIN/ImproveAccuracy and then browse the faq and hit the docs pages for more.
     
    Th0m likes this.
  20. Stelios

    Stelios Active Member HowtoForge Supporter

    @Jesse Norell am I right to assume that sa-update is not automatically added to run every X time during the ispconfig installation and we have to add it to the cron manually to all email servers?
     

Share This Page