rspamd: second redis instance for bayes

Discussion in 'Tips/Tricks/Mods' started by Jesse Norell, Apr 29, 2021.

Tags:
  1. Jesse Norell

    Jesse Norell Well-Known Member Staff Member Howtoforge Staff

    This is a quick "howto" for setting up a second redis instance for bayes data with rspamd on a Debian 10 ISPConfig server.

    Why you would want bayes in a separate redis instance is simply for memory management (redis can consume quite a bit of memory in the default configuration, on debian at least). Redis is a key-value store, and some of the keys rspamd stores should be kept longer to improve the spam scanning, while other keys (eg. old bayes tokens) can be evicted with relatively little consequence. So we'll setup a dedicated redis instance with a max-memory limit for bayes, and use the default instance without a memory limit to store all other keys. (See https://redis.io/topics/lru-cache and https://rspamd.com/doc/tutorials/redis_replication.html for background.)

    There are various ways to setup a second redis instance, this uses the systemd support for instances provided in the debian package. ISPConfig version 3.2.4 has a setting for the bayes redis server and password, so ensure you are using that version or later. This configuration will also setup the use of unix sockets to communicate with the redis server, which both performs better and is more secure in many common server configurations.

    First we'll edit `/etc/redis/redis.conf` to set the default settings for all redis instances, then we'll override a few in `/etc/redis/redis-bayes.conf`. Open `/etc/redis/redis.conf` in your favorite text editor and change/uncomment:
    Code:
    requirepass your-random-redis-password
    port 0
    unixsocket /var/run/redis/redis-server.sock
    unixsocketperm 770
    
    Now create `/etc/redis/redis-bayes.conf` with this (set an appropriate memory limit size for use with bayes):
    Code:
    # this is the redis instance used for bayes db,
    # with volatile-ttl eviction policy
    
    # read settings from redis.conf, then override
    include /etc/redis/redis.conf
    
    requirepass some-other-random-password
    
    bind 127.0.0.1 ::1
    port 0
    unixsocket /var/run/redis-bayes/redis-server.sock
    unixsocketperm 770
    timeout 0
    pidfile /var/run/redis-bayes/redis-server.pid
    loglevel notice
    logfile /var/log/redis/redis-server-bayes.log
    
    dbfilename bayes.rdb
    
    maxmemory 1GB
    maxmemory-policy volatile-ttl
    
    lazyfree-lazy-eviction yes
    lazyfree-lazy-expire yes
    
    The passwords in your redis.conf and redis-bayes.conf will need to match the rspamd settings in ISPConfig, you can generate a new password for the bayes instance or use the same password for each if you wish.

    Set the user/group and permissions of redis-bayes.conf, restart redis, and add user `_rspamd` to the `redis` group:
    Code:
    chown --reference=/etc/redis/redis.conf /etc/redis/redis-bayes.conf
    chmod --reference=/etc/redis/redis.conf /etc/redis/redis-bayes.conf
    systemctl restart redis-server
    systemctl start redis-server@bayes
    usermod -a -G redis _rspamd
    
    Now head to the ISPConfig server config mail settings and set the socket location and redis passwords (same password in the ui as you put in redis-bayes.conf):
    Code:
    Redis Servers:  /var/run/redis/redis-server.sock
    Redis Password:  your-random-redis-password
    Redis Servers for Bayes:  /var/run/redis-bayes/redis-server.sock
    Redis Password for Bayes:  some-other-random-password
    
    And that should be it, rspamd is now using the redis-server@bayes instance. Check that `/etc/rspamd/local.d/classifier-bayes.conf` points to the correct redis-server socket, and you might watch `/var/log/rspamd/rspamd.log` to ensure there are no errors. The bayes tokens in the primary redis instance should expire eventually, or you can go clean them up now:
    Code:
    export REDISCLI_AUTH=$(grep ^requirepass /etc/redis/redis.conf | awk '{print $2}')
    redis-cli -s /var/run/redis/redis-server.sock --scan --pattern 'BAYES_*' | xargs redis-cli -s /var/run/redis/redis-server.sock del
    redis-cli -s /var/run/redis/redis-server.sock --scan --pattern 'RS*' | xargs redis-cli -s /var/run/redis/redis-server.sock del
    
    The new bayes db will of course be completely empty, so if you have a corpus of training mail, now's a good time to go train it.
     
    Last edited: May 27, 2021
    ahrasis and Th0m like this.

Share This Page