I'm operating a Debian11 server with Ispconfig 3.2. DKIM signing is done by amavisd-new-2.11.1 (20181009). I've noticed that locally generated outgoing mail (directly with the php mail() function from the domain's associated website) is not signed by DKIM. It works when it's sent through SMTP (connecting to localhost:25, unauthenticated) and verifies ok so DKIM is set up properly, keys work, etc. The return-path and from headers are OK in both cases (i.e. mail is coming from webmaster@domain as they should). What am I missing to fix this? I recall that there was an earlier one liner patch to amavisd-new to fix some sort of local DKIM signing issue but I'm not sure if it's relevant.
There are multiple ways to achieve this. The Issue is that locally generated mails are not getting handed over to amavis. One way could be to change the master.cf to something like this: Code: pickup fifo n - n 60 1 pickup -o content_filter= // COMMENT THIS OUT -o receive_override_options= When you comment the line out it should work
Thank you for the quick response! Unfortunately that's not the case, the pickup line (ha ha) has no extra options and amavisd does scan it, i.e. the non dkim-signed local message has an "X-Virus-Scanned: Debian amavisd-new at hostname" header but no DKIM.
Oh well that is a different issue than. Can you post your master.cf and amavis config(i assume it is called 50_user.conf or something similar)?
Here they are - it's mostly stock except for a small change in master.cf to add policy-spf master.cf Code: # # Postfix master process configuration file. For details on the format # of the file, see the master(5) manual page (command: "man 5 master" or # on-line: http://www.postfix.org/master.5.html). # # Do not forget to execute "postfix reload" after editing this file. # # ========================================================================== # service type private unpriv chroot wakeup maxproc command + args # (yes) (yes) (no) (never) (100) # ========================================================================== smtp inet n - y - - smtpd #smtp inet n - y - 1 postscreen #smtpd pass - - y - - smtpd #dnsblog unix - - y - 0 dnsblog #tlsproxy unix - - y - 0 tlsproxy submission inet n - y - - smtpd -o syslog_name=postfix/submission -o smtpd_tls_security_level=encrypt -o smtpd_sasl_auth_enable=yes # -o smtpd_tls_auth_only=yes # -o smtpd_reject_unlisted_recipient=no # -o smtpd_client_restrictions=$mua_client_restrictions # -o smtpd_helo_restrictions=$mua_helo_restrictions # -o smtpd_sender_restrictions=$mua_sender_restrictions # -o smtpd_recipient_restrictions= -o smtpd_relay_restrictions=permit_sasl_authenticated,reject # -o milter_macro_daemon_name=ORIGINATING smtps inet n - y - - smtpd -o syslog_name=postfix/smtps -o smtpd_tls_wrappermode=yes -o smtpd_sasl_auth_enable=yes # -o smtpd_reject_unlisted_recipient=no # -o smtpd_client_restrictions=$mua_client_restrictions # -o smtpd_helo_restrictions=$mua_helo_restrictions # -o smtpd_sender_restrictions=$mua_sender_restrictions # -o smtpd_recipient_restrictions= -o smtpd_relay_restrictions=permit_sasl_authenticated,reject # -o milter_macro_daemon_name=ORIGINATING #628 inet n - y - - qmqpd pickup unix n - y 60 1 pickup cleanup unix n - y - 0 cleanup qmgr unix n - n 300 1 qmgr #qmgr unix n - n 300 1 oqmgr tlsmgr unix - - y 1000? 1 tlsmgr rewrite unix - - y - - trivial-rewrite bounce unix - - y - 0 bounce defer unix - - y - 0 bounce trace unix - - y - 0 bounce verify unix - - y - 1 verify flush unix n - y 1000? 0 flush proxymap unix - - n - - proxymap proxywrite unix - - n - 1 proxymap smtp unix - - y - - smtp relay unix - - y - - smtp -o syslog_name=postfix/$service_name # -o smtp_helo_timeout=5 -o smtp_connect_timeout=5 showq unix n - y - - showq error unix - - y - - error retry unix - - y - - error discard unix - - y - - discard local unix - n n - - local virtual unix - n n - - virtual lmtp unix - - y - - lmtp anvil unix - - y - 1 anvil scache unix - - y - 1 scache postlog unix-dgram n - n - 1 postlogd # # ==================================================================== # Interfaces to non-Postfix software. Be sure to examine the manual # pages of the non-Postfix software to find out what options it wants. # # Many of the following services use the Postfix pipe(8) delivery # agent. See the pipe(8) man page for information about ${recipient} # and other message envelope options. # ==================================================================== # # maildrop. See the Postfix MAILDROP_README file for details. # Also specify in main.cf: maildrop_destination_recipient_limit=1 # maildrop unix - n n - - pipe flags=DRXhu user=vmail argv=/usr/bin/maildrop -d ${recipient} # # ==================================================================== # # Recent Cyrus versions can use the existing "lmtp" master.cf entry. # # Specify in cyrus.conf: # lmtp cmd="lmtpd -a" listen="localhost:lmtp" proto=tcp4 # # Specify in main.cf one or more of the following: # mailbox_transport = lmtp:inet:localhost # virtual_transport = lmtp:inet:localhost # # ==================================================================== # # Cyrus 2.1.5 (Amos Gouaux) # Also specify in main.cf: cyrus_destination_recipient_limit=1 # #cyrus unix - n n - - pipe # flags=DRX user=cyrus argv=/cyrus/bin/deliver -e -r ${sender} -m ${extension} ${user} # # ==================================================================== # Old example of delivery via Cyrus. # #old-cyrus unix - n n - - pipe # flags=R user=cyrus argv=/cyrus/bin/deliver -e -m ${extension} ${user} # # ==================================================================== # # See the Postfix UUCP_README file for configuration details. # uucp unix - n n - - pipe flags=Fqhu user=uucp argv=uux -r -n -z -a$sender - $nexthop!rmail ($recipient) # # Other external delivery methods. # ifmail unix - n n - - pipe flags=F user=ftn argv=/usr/lib/ifmail/ifmail -r $nexthop ($recipient) bsmtp unix - n n - - pipe flags=Fq. user=bsmtp argv=/usr/lib/bsmtp/bsmtp -t$nexthop -f$sender $recipient scalemail-backend unix - n n - 2 pipe flags=R user=scalemail argv=/usr/lib/scalemail/bin/scalemail-store ${nexthop} ${user} ${extension} mailman unix - n n - - pipe flags=FRX user=list argv=/usr/lib/mailman/bin/postfix-to-mailman.py ${nexthop} ${user} dovecot unix - n n - - pipe flags=DRhu user=vmail:vmail argv=/usr/lib/dovecot/deliver -f ${sender} -d ${user}@${nexthop} # added by diab spfcheck unix - n n - 0 spawn user=policyd-spf argv=/usr/sbin/postfix-policyd-spf-perl amavis unix - - - - 2 smtp -o smtp_data_done_timeout=1200 -o smtp_send_xforward_command=yes -o smtp_bind_address= 127.0.0.1:10025 inet n - n - - smtpd -o content_filter= -o local_recipient_maps= -o relay_recipient_maps= -o smtpd_restriction_classes= -o smtpd_client_restrictions= -o smtpd_helo_restrictions= -o smtpd_sender_restrictions= -o smtpd_recipient_restrictions=permit_mynetworks,reject -o smtpd_end_of_data_restrictions= -o mynetworks=127.0.0.0/8 -o strict_rfc821_envelopes=yes -o receive_override_options=no_unknown_recipient_checks,no_header_body_checks -o smtp_send_xforward_command=yes -o disable_dns_lookups=yes -o address_verify_virtual_transport=$virtual_transport -o address_verify_transport_maps=$transport_maps 127.0.0.1:10027 inet n - n - - smtpd -o content_filter= -o local_recipient_maps= -o relay_recipient_maps= -o smtpd_restriction_classes= -o smtpd_client_restrictions= -o smtpd_helo_restrictions= -o smtpd_sender_restrictions= -o smtpd_recipient_restrictions=permit_mynetworks,reject -o smtpd_end_of_data_restrictions= -o mynetworks=127.0.0.0/8 -o strict_rfc821_envelopes=yes -o receive_override_options=no_unknown_recipient_checks,no_header_body_checks -o smtp_send_xforward_command=yes -o disable_dns_lookups=yes -o address_verify_virtual_transport=$virtual_transport -o address_verify_transport_maps=$transport_maps -o milter_default_action=accept -o milter_macro_daemon_name=ORIGINATING amavis 50-user Code: use strict; # # Place your configuration directives here. They will override those in # earlier files. # # See /usr/share/doc/amavisd-new/ for documentation and examples of # the directives you can use in this file # @bypass_virus_checks_maps = ( \%bypass_virus_checks, \@bypass_virus_checks_acl, \$bypass_virus_checks_re); @bypass_spam_checks_maps = ( \%bypass_spam_checks, \@bypass_spam_checks_acl, \$bypass_spam_checks_re); # # Database connection settings # @lookup_sql_dsn = ( ['DBI:mysql:database=dbispconfig;host=127.0.0.1;port=3306', 'xxxx', 'xxxx'] ); # @storage_sql_dsn = @lookup_sql_dsn; # none, same, or separate database #$sql_select_policy = 'SELECT "Y" as local FROM mail_domain WHERE CONCAT("@",domain) IN (%k)'; # $banned_files_quarantine_method = 'sql'; # $spam_quarantine_method = 'sql'; # # SQL Select statements # $sql_select_policy = 'SELECT *,spamfilter_users.id'. ' FROM spamfilter_users LEFT JOIN spamfilter_policy ON spamfilter_users.policy_id=spamfilter_policy.id'. ' WHERE spamfilter_users.email IN (%k) AND spamfilter_users.policy_id != 0'. ' ORDER BY spamfilter_users.priority DESC'; $sql_select_white_black_list = 'SELECT wb FROM spamfilter_wblist'. ' WHERE (spamfilter_wblist.rid=?) AND (spamfilter_wblist.email IN (%k)) AND (spamfilter_wblist.active="y")' . ' ORDER BY spamfilter_wblist.priority DESC'; # # Quarantine settings # $final_virus_destiny = D_BOUNCE; $final_spam_destiny = D_DISCARD; $final_banned_destiny = D_BOUNCE; $final_bad_header_destiny = D_PASS; # Default settings, we set this very high to not filter out emails accidentally $sa_spam_subject_tag = '***SPAM*** '; $sa_tag_level_deflt = 20.0; # add spam info headers if at, or above that level $sa_tag2_level_deflt = 60.0; # add 'spam detected' headers at that level $sa_kill_level_deflt = 60.0; # triggers spam evasive actions $sa_dsn_cutoff_level = 100; # spam level beyond which a DSN is not sent # # Disable spam and virus notifications for the admin user. # Can be overridden by the policies in mysql # $virus_admin = undef; $spam_admin = undef; # # Enable Logging # $DO_SYSLOG = 1; $LOGFILE = "/var/log/amavis.log"; # (defaults to empty, no log) # Set the log_level to 5 for debugging $log_level = 0; # (defaults to 0) $inet_socket_port = [10024,10026]; # :* = send to incoming Port + 1 $forward_method = 'smtp:127.0.0.1:*'; $notify_method = 'smtp:127.0.0.1:*'; $interface_policy{'10026'} = 'ORIGINATING'; $policy_bank{'ORIGINATING'} = { originating => 1, }; # IP-Addresses for internal networks => load policy MYNETS # - requires -o smtp_send_xforward_command=yes in postfix master.cf @mynetworks = qw(0.0.0.0/8 127.0.0.0/8 10.0.0.0/8 172.16.0.0/12 192.168.0.0/16 [::1] [FE80::]/10 [FEC0::]/10); # Allow SMTP access from IPs in @inet_acl to amvisd SMTP Port @inet_acl = qw( 127.0.0.1 [::1] 192.168.0.0/16 ); # DKIM $enable_dkim_verification = 1; $enable_dkim_signing = 1; # load DKIM signing code $signed_header_fields{'received'} = 0; # turn off signing of Received @dkim_signature_options_bysender_maps = ( { '.' => { ttl => 21*24*3600, c => 'relaxed/simple' } } ); #------------ Do not modify anything below this line ------------- 1; # insure a defined return amavis dkim conf Code: dkim_key('xxxx.com', 'default', '/var/lib/amavis/dkim/xxxx.com.private');
Where does the php mail() function drop the emails? If they're dropped in Postfix, are they sent to the outside world or delivered locally? I don't think emails that stay local are being signed, only if they're relayed to another host.
The PHP mail() function uses sendmail to send mail IIRC, but the issue here is that mails that are send from the machine itself are not getting signed with DKIM as they don't seem to pass amavis for outbound
Ok, let me rephrase then. Where does sendmail drop the emails generated with the php mail() function? Into Postfix' queue or somewhere else? It makes no sense that they don't get signed if they're dropped into Postfix when other outbound emails do get signed. That makes me think they don't pass through Postfix and therefor Amavis at all.
The one that gets signed (i.e. by connecting to localhost 25) has this header: Code: Received: from localhost (localhost.localdomain [127.0.0.1]) by xx (Postfix) with SMTP id 30889194D283 for <xx@xx>; Sat, 3 Dec 2022 13:20:02 +0100 (CET) the one that is not signed is like this (sent by connecting to localhost:25): Code: by localhost (xx [127.0.0.1]) (amavisd-new, port 10024) with LMTP id lNA_NRmLDnwq for <xx@xx>; Sat, 3 Dec 2022 13:18:36 +0100 (CET) Received: by xx (Postfix, from userid 5013) id 625BB194D29A; Sat, 3 Dec 2022 13:18:36 +0100 (CET) I ran an strace on the php process and it looks like mail() runs sendmail directly in the background Code: [pid 3133130] execve("/usr/sbin/sendmail", ["/usr/sbin/sendmail", "-t", "-i"], 0x55a9ecbfc6b0 /* 16 vars */) = 0
+1 this does not get signed: Code: echo -en 'From: webmaster@xx\nTo: xx@xx\nSubject: test\n\nhello\n.' | sendmail -t -i -f webmaster@xx however, this does Code: # telnet 0 25 Trying 0.0.0.0... Connected to 0. Escape character is '^]'. 220 xx ESMTP Postfix (Debian/GNU) helo localhost 250 xx mail from: webmaster@xx 250 2.1.0 Ok rcpt to: xx@xx 250 2.1.5 Ok data 354 End data with <CR><LF>.<CR><LF> abc . 250 2.0.0 Ok: queued as 8C61C194D1DE quit 221 2.0.0 Bye Connection closed by foreign host.
Trace the emails in the maillogs by id to see how they pass through Postfix (and Amavis) before going outside. The headers you posted look somewhat incomplete to me and definitely don't show the whole path.
Is there a reason why you're using Amavis and not Rspamd? Recently I had issues with Postfix and Amavis after implementing SRS where emails looped between Postfix and Amavis. Not the same as you are facing but I solved it by replacing Amavis with Rspamd: https://www.howtoforge.com/replacing-amavisd-with-rspamd-in-ispconfig/
Thanks again for the response. I use amavisd because I am familiar with it and the howto I was using to set up the server used that, too. Now that it's in production I'd prefer to fix it but I may replace it with rspamd if we can't figure it out. Anyhow, here are the full headers: Code: Return-Path: <webmaster@xxxxx> Delivered-To: xxxxx@xxxxx Received: from xxxxx by xxxxx with LMTP id 4tYtAGdHi2PX4C8Ac2pvdg (envelope-from <webmaster@xxxxx>) for <xxxxx@xxxxx>; Sat, 03 Dec 2022 13:56:07 +0100 Received: from localhost (localhost.localdomain [127.0.0.1]) by xxxxx (Postfix) with ESMTP id EE765194D29D for <xxxxx@xxxxx>; Sat, 3 Dec 2022 13:56:06 +0100 (CET) X-Virus-Scanned: Debian amavisd-new at xxxxx Received: from xxxxx ([127.0.0.1]) by localhost (xxxxx [127.0.0.1]) (amavisd-new, port 10024) with LMTP id NClEPbvz1vCt for <xxxxx@xxxxx>; Sat, 3 Dec 2022 13:56:06 +0100 (CET) Received: by xxxxx (Postfix, from userid 0) id BB9F2194D29E; Sat, 3 Dec 2022 13:56:06 +0100 (CET) From: webmaster@xxxxx To: xxxxx@xxxxx Subject: test Message-Id: <20221203125606.BB9F2194D29E@xxxxx> Date: Sat, 3 Dec 2022 13:56:06 +0100 (CET) mail server logs with ID: BB9F2194D29E Code: Dec 3 12:56:06 xxxxx postfix/pickup[3130767]: BB9F2194D29E: uid=0 from=<webmaster@xxxxx> Dec 3 12:56:06 xxxxx postfix/cleanup[3137744]: BB9F2194D29E: message-id=<20221203125606.BB9F2194D29E@xxxxx> Dec 3 12:56:06 xxxxx postfix/qmgr[2721401]: BB9F2194D29E: from=<webmaster@xxxxx>, size=307, nrcpt=1 (queue active) Dec 3 12:56:06 xxxxx postfix/cleanup[3137744]: EE765194D29D: message-id=<20221203125606.BB9F2194D29E@xxxxx> Dec 3 12:56:06 xxxxx amavis[3135506]: (3135506-01) Passed CLEAN {RelayedInbound}, [127.0.0.1] <webmaster@xxxxx> -> <xxxxx@xxxxx>, Message-ID: <20221203125606.BB9F2194D29E@xxxxx>, mail_id: NClEPbvz1vCt, Hits: -0.001, size: 305, queued_as: EE765194D29D, 201 ms Dec 3 12:56:06 xxxxx postfix/lmtp[3137746]: BB9F2194D29E: to=<xxxxx@xxxxx>, relay=127.0.0.1[127.0.0.1]:10024, delay=0.23, delays=0.02/0.01/0/0.2, dsn=2.0.0, status=sent (250 2.0.0 from MTA(smtp:[127.0.0.1]:10025): 250 2.0.0 Ok: queued as EE765194D29D) Dec 3 12:56:06 xxxxx postfix/qmgr[2721401]: BB9F2194D29E: removed Dec 3 12:56:07 xxxxx dovecot: lmtp(xxxxx@xxxxx)<3137751><4tYtAGdHi2PX4C8Ac2pvdg>: sieve: msgid=<20221203125606.BB9F2194D29E@xxxxx>: stored mail into mailbox 'INBOX' On the other hand, the one that gets signed looks like this: Code: Dec 3 12:33:36 host3 amavis[2885068]: (2885068-20) Passed CLEAN {RelayedInternal}, ORIGINATING LOCAL [127.0.0.1] <xxxxx@xxxxx> -> <xxxxx@xxxxx>, Message-ID: <20221203123332.8C61C194D1DE@xxxxx>, mail_id: PMLC8Pahuq5M, Hits: 1.974, size: 334, queued_as: 1FB4D194D29D, dkim_new=default:xxxxx, 163 ms Maybe the reason is that Amavisd treats direct mail as "relayedinbound" as opposed to "relayedinternal" and it should be somehow configured to enable dkim signing there, too.
yaay, I figured it out. Enabling debug logs in amavis showed: Code: Dec 3 13:42:08 xxxxx amavis[3148219]: (3148219-01) dkim: not signing mail which is not originating from our site So I had to enable "originating" for the "MYNETS" policy bank in amavis 50-user.conf and it started working Code: $policy_bank{'MYNETS'} = { originating => 1, }; I've verified and it's not signing random e-mails from outside so it looks like it works as intended, local mail / trusted hosts only.
