rspamd whitelist

Discussion in 'Installation/Configuration' started by jnsc, Jun 8, 2020.

  1. jnsc

    jnsc rotaredoM Moderator

    Hi,
    Since I migrated to rspamd, my user complain that the whitelist is not working. And from the tests I did it seems that they are right. Is the whitelist working on your side?

    Form what I have seen the configuration files /etc/rspamd/local.d/users/spamfilter_wblist_XX.conf get generated, and looks fine, they also get read by rspamd (when I enter an invalid syntax, rspamd will complain), but for an unknown reason I believe that rspamd is not doing anything about this files.

    Is there an easy way to test this rules?
     
  2. elmacus

    elmacus Active Member

    Last edited: Jun 10, 2020
  3. jnsc

    jnsc rotaredoM Moderator

    Yes I saw this, but my problem seems to be more general, I tried with many sender/recipients but without success, I also tried to edit the config files by hand, playing with rcpt, mime_rcpt, etc. but with no success.
     
  4. jnsc

    jnsc rotaredoM Moderator

    Did someone get the whitelist working on there side?
     
  5. elmacus

    elmacus Active Member

    I do have alot of whitelist that does work.
     
    jnsc likes this.
  6. till

    till Super Moderator Staff Member ISPConfig Developer

    One thing that you might try is to empty (or move all files to a backup directory) in /etc/rspamd/local.d/users/, and I mean all, not just the whitelist files, and then use Tools > resync for all mail related functions to let ispconfig regenerate them. Maybe you have some old config files there which cause the whitelist to not work.
     
    jnsc likes this.
  7. jnsc

    jnsc rotaredoM Moderator

    Hi till,
    I'll definitively will try this.
    Thanks
     
  8. jnsc

    jnsc rotaredoM Moderator

    I did the resync, and still waiting for a real world test, but I have not much hope as a diff between the old and the new folder does not show any differences :-(

    For troubleshooting I use scan function in the ui (by pasting the message source) is this the right test methode?

    Maybe in doing things wrong, but the configuration looks so simple that I do not see what could be wrong

    The mail header :
    Code:
    To: [email protected]
    From: Name of my user via Rocketbook <[email protected]>
    The rspamd rule
    Code:
    spamfilter_wblist-16 {
            priority = 25;
            from = "[email protected]";
            rcpt = "[email protected]";
            want_spam = yes;
            apply {
                    actions {
                            reject = null;
                            "add header" = null;
                            greylist = null;
                            "rewrite subject" = null;
                    }
            }
    }
     
    Last edited: Jun 11, 2020
  9. jnsc

    jnsc rotaredoM Moderator

    Ok, I somewhat progressed,
    when using
    Code:
    echo "test" | rspamc -F "[email protected]" -r "[email protected]"
    returns
    Code:
    Spam: false
    which is what should happen with the mail, but I still don't know why when receiving the mail for real it sees it as spam.
     
  10. Jesse Norell

    Jesse Norell ISPConfig Developer Staff Member ISPConfig Developer

    You are wanting to match mail From/To headers, where the whitelist entry matches envelop from/rcpt.
     
  11. mikiquo

    mikiquo New Member

    Hi, I know is an old thread but I'm facing the same problem (I installed rspamd not long ago).
    It appears that the blacklist rules aren't evaluated.
    I find them in local.d/users/spamfilter... but mails sent from/to specified accounts do not get blocked.
    Code:
    spamfilter_wblist-31 {
            priority = 45;
            from = "mike[at]gmail.com";
            rcpt = "mike[at]xxxxxx.li";
            apply {
                    R_DUMMY = 999.0;
                    actions {
                            reject = 0.2;
                            "add header" = 0.1;
                            greylist = 0.1;
                            "rewrite subject" = 0.1;
                    }
            }
    }
    
    Sending an email from mike[at]gmail.com to mike[at]xxxxxx.li simply get the default symbol values (no R_DUMMY) and not rejected.
    Even with rspamadm configdump, I find the above directives.
    I tried to delete ...local.d/users/* and resync from ispconfig, but nothing's changed.
    Am I missing some rspamd configuration?

    I'm running debian 9, postfix 3.1.15, rspamd 2.7, ispconfig 3.2.8p1
     
  12. Jesse Norell

    Jesse Norell ISPConfig Developer Staff Member ISPConfig Developer

    What shows up in rspamd.log for this email?
     
  13. mikiquo

    mikiquo New Member

    This is a try. No mention about R_DUMMY.
    Code:
    2022-04-25 14:22:43 #11891(normal) <d2ff3e>; task; rspamd_task_write_log: id: <[email protected]>, qid: <B7A40260058>, ip: 209.85.221.49, from: <[email protected]>, (default: F (no action): [-1.12/15.00] [DMARC_POLICY_ALLOW(-0.50){gmail.com;none;},R_DKIM_ALLOW(-0.20){gmail.com:s=20210112;},R_SPF_ALLOW(-0.20){+ip4:209.85.128.0/17;},BAYES_HAM(-0.11){66.71%;},MIME_GOOD(-0.10){text/plain;},MX_GOOD(-0.01){},ARC_NA(0.00){},ASN(0.00){asn:15169, ipnet:209.85.128.0/17, country:US;},DKIM_TRACE(0.00){gmail.com:+;},DWL_DNSWL_NONE(0.00){gmail.com:dkim;},FREEMAIL_ENVFROM(0.00){gmail.com;},FREEMAIL_FROM(0.00){gmail.com;},FROM_EQ_ENVFROM(0.00){},FROM_HAS_DN(0.00){},MID_RHS_MATCH_FROM(0.00){},MIME_TRACE(0.00){0:+;},PREVIOUSLY_DELIVERED(0.00){[email protected];},RCPT_COUNT_ONE(0.00){1;},RCVD_COUNT_THREE(0.00){3;},RCVD_IN_DNSWL_NONE(0.00){209.85.221.49:from;},RCVD_TLS_ALL(0.00){},RCVD_VIA_SMTP_AUTH(0.00){},RWL_MAILSPIKE_POSSIBLE(0.00){209.85.221.49:from;},TAGGED_FROM(0.00){},TO_DN_NONE(0.00){},TO_MATCH_ENVRCPT_ALL(0.00){}]), len: 2635, time: 171.654ms, dns req: 21, digest: <515f000825b5db3dfb79896ed08bf2eb>, rcpts: <[email protected]>, mime_rcpts: <[email protected]>, settings_id: ispc_mail_user_2
     
  14. Jesse Norell

    Jesse Norell ISPConfig Developer Staff Member ISPConfig Developer

    In your example, the settings used were 'ispc_mail_user_2' which means the conditions of 'spamfilter_wblist-31' didn't match:
    Code:
           from = "mike[at]gmail.com";
           rcpt = "mike[at]xxxxxx.li";
    
    Looking at the other details in the log, everything I see does match, so my guess is that the obfuscation you did to hide the actual addresses also hid the problem. Eg. maybe you had periods or a dash in the blacklisted sender address? Gmail strips those out (see https://github.com/rspamd/rspamd/issues/2560).
     
  15. mikiquo

    mikiquo New Member

    You're right, I took the wrong line of log out of many.
    I noticed what you mentioned about the period stripped with gmail so I tried with 2 rules (with and without period), then I used another external email to test but with the same results.
    Here the relevant conf and logs
    Code:
    spamfilter_wblist-33 {
            priority = 45;
            from = "mike[at]external.email";
            rcpt = "mike[at]local.email";
            apply {
                    R_DUMMY = 999.0;
                    actions {
                            reject = 0.2;
                            "add header" = 0.1;
                            greylist = 0.1;
                            "rewrite subject" = 0.1;
                    }
            }
    }
    Code:
    2022-04-26 17:23:51 #22221(normal) <499ea4>; task; rspamd_task_write_log: id: <[email protected]>, qid: <D40F8260DD0>, ip: 195.176.176.171, from: <mike[at]external.email
    >, (default: F (no action): [-2.80/0.20] [BAYES_HAM(-1.29){90.00%;},DMARC_POLICY_ALLOW(-0.50){edu.ti.ch;quarantine;},RCVD_DKIM_ARC_DNSWL_MED(-0.50){},R_DKIM_ALLOW(-0.20){edu.ti.ch:s=mail;},R_SPF_A
    LLOW(-0.20){+ip4:195.176.176.171;},MIME_GOOD(-0.10){text/plain;},MX_GOOD(-0.01){},ARC_NA(0.00){},ASN(0.00){asn:559, ipnet:195.176.160.0/19, country:CH;},DKIM_TRACE(0.00){edu.ti.ch:+;},DWL_DNSWL_ME
    D(0.00){ti.ch:dkim;},FROM_EQ_ENVFROM(0.00){},FROM_HAS_DN(0.00){},MID_RHS_MATCH_FROM(0.00){},MIME_TRACE(0.00){0:+;},RCPT_COUNT_ONE(0.00){1;},RCVD_COUNT_TWO(0.00){2;},RCVD_IN_DNSWL_MED(0.00){195.176
    .176.171:from;},RCVD_TLS_ALL(0.00){},RCVD_VIA_SMTP_AUTH(0.00){},RWL_MAILSPIKE_GOOD(0.00){195.176.176.171:from;},TO_DN_ALL(0.00){},TO_MATCH_ENVRCPT_ALL(0.00){}]), len: 1217, time: 256.536ms, dns re
    q: 47, digest: <aa8183f9fa8955b4c4462b7637c1c1f2>, rcpts: <mike[at]local.email>, mime_rcpts: <mike[at]local.email>, settings_id: spamfilter_wblist-33
    2022-04-26 17:23:51 #22221(normal) <499ea4>; task; rspamd_protocol_http_reply: regexp statistics: 0 pcre regexps scanned, 4 regexps matched, 174 regexps total, 73 regexps cached, 0B scanned using
    pcre, 824B scanned total
    Sorry for my fault and the bad formatting...
    PS: didn't hide all the sensitive address, my bad...
     
  16. Jesse Norell

    Jesse Norell ISPConfig Developer Staff Member ISPConfig Developer

    That's not the same results, this one matched the sender/recipient:
    So now you have a match, the problem is that what happened isn't what you expect:
    The effect of that "blacklist" entry is to set an low reject score, at .2 - however that particular message scored lower than that, so no action was taken.

    The real fix for this will be https://git.ispconfig.org/ispconfig/ispconfig3/-/issues/6082 In the mean time, the quickest workaround is probably to copy rspamd_wblist.inc.conf.master to your conf-custom/install/ dir and change it to reject at an even lower score (-10 or -100 or something). Or you can use a postfix blacklist entry, but that will affect the whole server, not just the one recipient.
     
  17. mikiquo

    mikiquo New Member

    Many thanks for your time.
    What's not clear for me, is the use of "R_DUMMY = 999.0" symbol.
    Shouldn't be anywhere an action that reject mails based on this symbol?
    Why isn't that symbol applied to the incoming mail?
     
  18. Jesse Norell

    Jesse Norell ISPConfig Developer Staff Member ISPConfig Developer

    To my knowledge, the symbol is assigned a score, but there is no actual definition for a rule, so it will never match. I don't think it serves any purpose.
    Code:
    # grep R_DUMMY /var/log/rspamd/rspamd.log | head -1
    2022-04-27 03:18:13 #1544207(main) <tw8sxo>; symcache; rspamd_symcache_validate: symbol 'R_DUMMY' has its score defined but there is no corresponding rule registered
    
     
  19. mikiquo

    mikiquo New Member

    Ok, I did it.
    Manually changing the reject score to "-10" worked!
    Only note, reloading rspamd is not enough, I had to restart it (it takes minutes), there's probably some cache behind.
    Next step is changing the ispconfig custom template.
    Again, many thanks for your precious help.
     
  20. ludoc

    ludoc New Member

    Helllo,
    same problem here, solved by creating "R_DUMMY" symbol in Rspamd config.

    Create the file /etc/rspamd/rspamd.local.lua with this content :
    Code:
    rspamd_config.R_DUMMY = {
            callback = function(task)
                    return true
            end,
            score = 0,
            description = 'dummy symbol',
    }
    
    Then restart Rspamd



    I hope this will help :)
     
    variable99 likes this.

Share This Page