Dovecot/Roundcube/Sieve: cannot automatically forward a copy of an email to external mailbox

Discussion in 'General' started by Gwyneth Llewelyn, Aug 3, 2024.

  1. Hi there!
    I'm not quite sure if I should post this question on the 'General' forum, or on any of the others; I have read similar questions in at least different ones, so feel free to move it to the right one.
    I have the usual ISPConfig3 setup (nginx variant) — Postfix, Dovecot, Amavis, Rspamd, Roundcube, etc. according to the Perfect Server cookbook. However, it's worth noticing that this particular server started being Ubuntu 18.04 (I think — or was it 16.04?) and, over the years, it has been upgraded to 24.04 LTS. Similarly, ISPConfig3 was successfully upgraded during all these years. However, we all know how it is with "old" installations: over the years, some slight configuration details subtly change, and everything breaks apart.
    Most email addresses in my setup are simply forwarded to the user's preferred mailbox provider — almost always Gmail, with a very few exceptions. This has been the case for eternities, and it's rock-solid (well, minus the occasional SPF/DKIM/DMARC hiccup with Gmail's servers, but that's something we all have to live with). No problems there whatsoever.
    There are three "exceptional" cases, though. These are three mailboxes which are stored locally on my server, but a copy of all email also gets forwarded to a Gmail address (three different persons, three different mailboxes). According to what I read, this can be accomplished in several ways. The first, and perhaps the easiest one for the system administrator, is just to select that option from the Email Mailbox menu option ("Send copy to: [email address]") when creating the mailbox, which is exactly what I did, long ago. I cannot say when that was, or what versions I had on OS, ISPConfig3, etc. back then. Let's just say that the oldest message I found on one mailbox was from March 2020, and on the other, from January 2021. Note that this is really not saying much — earlier messages may have been deleted — and I don't keep log backups for so long.
    At some point in time, when upgrading to the new responsive skin in Roundcube (well, it was new back then), I've also made sure that the extra plugin goodies were available and functional (I'm speaking of the ISPConfig3-specific plugins and the way they neatly interface with the Roundcube settings/configuration for individual users). One of the things that it shows is the default email(s) that this mailbox redirects to — I can see that the information is correctly written there as well, for both accounts.
    In reply to a similar question from 2016, there was some mismatch between what was on the MySQL database, and what was configured via sieve. Allegedly (sorry, I know I read it here in a thread, but I can't confirm where), ISPConfig3 now does all the rules using sieve, no matter what (which makes total sense); it's just on very early installations that different mechanisms were used. Which lead me to think that perhaps there was something similarly wrong in my case. Unfortunately, however, I'm not really familiar with sieve, and, as a consequence, I never looked at what exactly ISPConfig3 is writing on disk to make its magic happen; bear with me while I try to explain what I've got so far, and please ask me for more specific information if needed.
    So, the easy bits first: doveadm user ... does not show anything out of normal. It refers to sieve being installed under /var/vmail/domain.name/user.name/.sieve — but that file (is it a file? or a director?) does not exist.
    Instead, there are two sieve-related files, .ispconfig-before.sieve and .ispconfig.sieve, and the associated compiled sieve files. For good measure, I forced the recompilation. The first of those files correctly includes the following:
    Code:
    # This sieve script is generated by ISPConfig, any changes made will be overwritten.
    # You can create and activate a per-user sieve script (manually or via managesieve),
    # which will execute after this.
    
    require ["body", "copy", "date", "envelope", "fileinto", "imap4flags", "mailbox", "regex", "reject","relational", "subaddress", "vacation", "variables"];
    
    # Move spam to spam folder
    if anyof (header :is ["X-Spam", "X-Spam-Flag"] "Yes", header :matches "X-Spam-Status" "Yes, *") {
      fileinto :create "Junk";
      # Stop here so that we do not reply on spams
      stop;
    }
    
    # Send a copy of email to
    redirect :copy "[email protected]";
    
    The last line corresponds exactly to what I have configured via the ISPConfig3 admin panel, and is what the user also sees via Roundcube (naturally, each has their own Gmail address written there).
    The file ownership is set to vmail:vmail, permissions 644
    There is a sieve directory at the same hierarchic level. This directory is empty in both cases, is also owned by vmail:vmail, but its permissions are 700. These have been created in 2021 (one in January, the other in September) but seem not to have been modified (or written to) ever since.
    So far, so good, I think. Let's see what we've got on the database...
    The mail_forwarding seems to be ok. It lists a plethora of emails that get redirected elsewhere but have no local mailboxes. The three above-mentioned ones are missing from this table. I guess that makes sense, since their forwarding is handled differently (they're real mailboxes, they copy-and-forward, etc.).
    mail_user is a bit of a mystery to me. What, exactly, should be listed here? I have on sender_cc the expected email addresses to which these two mailboxes are supposed to be sent; however, these seem to be completely ignored. Is this table relevant at all?
    From here, I went back to the Dovecot configuration, and tried to figure out what exactly it does with the database records; however, as far as I can see, there is nothing there specific in the database-related configuration that caught my attention. Dovecot itself doesn't seem to need to consult anything relevant in the database for dealing with mail forwarding.
    And now, on Postfix. Searching for mail_user and/or mail_forwarding on the SQL-related configuration files, there is quite a lot going on. Unfortunately, it's a little bit awkward to figure out what's really going on. I've tried to run some of those queries by hand, just to see what results I got; however, figuring out what to put in those placeholders is tricky!... at the end of the day, I gave up and started writing here instead :)

    Obviously, I'm expecting this to be a simple one-liner — some option to turn on or off — which possibly got changed (or forgotten) for some reason during a recent upgrade. I should also note that I do have a modification on /usr/local/ispconfig/server/conf-custom/install/dovecot_custom.conf.master (which affects /etc/dovecot/conf.d/99-ispconfig-custom-config.conf during reconfiguration of ISPConfig3's services, as per instructions at the very top of /etc/dovecot/dovecot.conf):

    Code:
    protocols = imap lmtp
    disable_plaintext_auth = no
    syslog_facility = local5
    # Move to directly retrieving certs from the Let's Encrypt directory
    # (gwyneth 20230403)
    ssl_cert = </etc/letsencrypt/my.domain.com/fullchain.cer
    ssl_key  = </etc/letsencrypt/my.domain.com/my.domain.com.key
    
    # mail_plugins = quota sieve
    protocol imap {
      mail_plugins = $mail_plugins imap_sieve imap_zlib
    }
    
    plugin {
      sieve_plugins = sieve_imapsieve sieve_extprograms
      sieve_extensions = +editheader +mboxmetadata +servermetadata +imapflags +notify +spamtest +spamtestplus +virustest
    
      # From elsewhere to Spam folder
      imapsieve_mailbox1_name = INBOX/Spam
      imapsieve_mailbox1_causes = COPY
      imapsieve_mailbox1_before = file:/usr/lib/dovecot/sieve/learn-spam.sieve
    
      # From Spam folder to elsewhere
      imapsieve_mailbox2_name = *
      imapsieve_mailbox2_from = INBOX/Spam
      imapsieve_mailbox2_causes = COPY
      imapsieve_mailbox2_before = file:/usr/lib/dovecot/sieve/learn-ham.sieve
    
      sieve_pipe_bin_dir = /usr/lib/dovecot/sieve
    
      sieve_global_extensions = +vnd.dovecot.pipe +vnd.dovecot.environment
    
      ##
      # rspamd:
      #
      # X-Spamd-Result: default: False [0.00 / 150.00]; ...
      # X-Spam: Yes
    
      ##
      # rspamd score based
      #
      # WARNING: the max score provided by rspamd is the maximum score
      # rspamd cares about, e.g. the score required for reject, not
      # necessarily the one for add_header, and therefore not a good
      # baseline for what should be considered "100%" spam.
      #
      ##
      sieve_spamtest_status_type = score
      sieve_spamtest_status_header = X-Spamd-Result: .*? \[([[:digit:]]+\.[[:digit:]]+) / [[:digit:]]+\.[[:digit:]]+\];
      ### best to use the same value as for `add_header` in rspamd
      sieve_spamtest_max_value = 6
      ### can read the "max score" from rspamd, but see warning above
      ## sieve_spamtest_max_header = X-Spamd-Result: .*? \[[[:digit:]]+\.[[:digit:]]+ / ([[:digit:]]+\.[[:digit:]]+)\];
    
      ##
      # Yes/No based
      #
      ##
      # sieve_spamtest_status_type = text
      # sieve_spamtest_status_header = X-Spam
      # sieve_spamtest_text_value1 = No
      # sieve_spamtest_text_value10 = Yes
    
      #
      # Automatically filter spam into the spam folder
      #
      sieve_before = /usr/lib/dovecot/sieve/global-spam.sieve
    }
    
    The main difference being that I wished to have the Let's Encrypt certificate + key being read from someplace else than the default. I can't recall why the rest of the lines are there, though! Probably they were the recommended changes to get rspamd working.
    I've also checked everything under /etc/dovecot/conf.d/, more specifically, the lda and lmtp protocol configuration; both, as expected, have sieve configured as a plugin.

    Where else should I turn my attention next?
     

Share This Page