I notice the migration to deb12 server (which did NOT bring over mailman, not supported I understand) I am seeing in mail.log lots of: 2024-02-28T12:38:19.712192-05:00 ns11 postfix/local[560794]: warning: hash:/var/lib/mailman/data/aliases is unavailable. open database /var/lib/mailman/data/aliases.db: No such file or directory now /var/lib/mailman/data aliases point to /etc/mailman which does not exist! ouch! do I need to tar up that folder and bring it to the new server? or what?
This should fix it: Code: mkdir -p /var/lib/mailman/data/ touch /var/lib/mailman/data/{aliases,transport-mailman,virtual-mailman} postmap /var/lib/mailman/data/{aliases,transport-mailman,virtual-mailman} You must have done something weird that caused this in the first place, as the official upgrade guide already covers this and a fresh install shall not have such problems.
Th0m I used the automated debian installer, then the migration tool. mailman was NOT migrated (it gave errors , but then mailman needing 2.7 is not supported under debian). so on the new server, mailman was never present. migtool however reports mailman was NOT migrated over. so dont see where these errors should have come from! The folders /var/lib/mailman/data etc all exist. they symlink to /etc/mailman. THAT is what does not exist!
I'm having more issues than I had thought - apparently a lot of mail is not going out! from mail.log: 2024-02-29T08:48:05.782705-05:00 ns11 postfix/smtp[643343]: F124D1D448B09: to=<[email protected]>, relay=none, delay=739, delays=589/0.03/150/0, dsn=4.4.1, status=deferred (connect to mta7.am0.yahoodns.net[67.195.228.106]:25: Connection timed out) 2024-02-29T08:58:05.358009-05:00 ns11 postfix/smtp[643789]: C9BF61D448B49: to=<[email protected]>, relay=none, delay=1858, delays=1707/0.06/150/0, dsn=4.4.1, status=deferred (connect to mta5.am0.yahoodns.net[98.136.96.74]:25: Connection timed out) I'm getting lots of 25:connection timed out. I noticed that my ispconfig had NO firewall records set so I added: Server ns11.cdbsystems.comServer Open TCP ports 20,21,22,25,53,80,110,137,138,139,143,443,445,465,514,587,993,995,2812,3306,8008,8080,8081,8091,27017,40110:40210 Open UDP ports 53,137,138,139,3306 restarted postfix. but I'm also seeing in mailq: D04711D44866F 5741128 Thu Feb 29 08:46:26 [email protected] (connect to alt2.gmail-smtp-in.l.google.com[2a00:1450:400c:c0b::1b]:25: Network is unreachable) [email protected] all ipv6 requests come back as unreachable! I assume I should turn off ipv6 in postfix/main.cf under inet_protocols but I also note that the automatic debian installer installed ufw and I have LOTS of ufw rules (even when ispconfig had NO active firewall rules). do i need to get rid of ufw? here is the iptables --list Code: hain INPUT (policy DROP) target prot opt source destination f2b-dovecot tcp -- anywhere anywhere multiport dports pop3,pop3s,imap2,imaps,submission,submissions,sieve f2b-pure-ftpd tcp -- anywhere anywhere multiport dports ftp f2b-postfix-sasl tcp -- anywhere anywhere multiport dports smtp f2b-sshd tcp -- anywhere anywhere multiport dports ssh ufw-before-logging-input all -- anywhere anywhere ufw-before-input all -- anywhere anywhere ufw-after-input all -- anywhere anywhere ufw-after-logging-input all -- anywhere anywhere ufw-reject-input all -- anywhere anywhere ufw-track-input all -- anywhere anywhere Chain FORWARD (policy DROP) target prot opt source destination ufw-before-logging-forward all -- anywhere anywhere ufw-before-forward all -- anywhere anywhere ufw-after-forward all -- anywhere anywhere ufw-after-logging-forward all -- anywhere anywhere ufw-reject-forward all -- anywhere anywhere ufw-track-forward all -- anywhere anywhere Chain OUTPUT (policy ACCEPT) target prot opt source destination ufw-before-logging-output all -- anywhere anywhere ufw-before-output all -- anywhere anywhere ufw-after-output all -- anywhere anywhere ufw-after-logging-output all -- anywhere anywhere ufw-reject-output all -- anywhere anywhere ufw-track-output all -- anywhere anywhere Chain f2b-dovecot (1 references) target prot opt source destination RETURN all -- anywhere anywhere Chain f2b-postfix-sasl (1 references) target prot opt source destination REJECT all -- 45.129.14.128 anywhere reject-with icmp-port-unreachable REJECT all -- 45.129.14.179 anywhere reject-with icmp-port-unreachable RETURN all -- anywhere anywhere Chain f2b-pure-ftpd (1 references) target prot opt source destination RETURN all -- anywhere anywhere Chain f2b-sshd (1 references) target prot opt source destination REJECT all -- 62.234.41.123 anywhere reject-with icmp-port-unreachable REJECT all -- 193.151.154.154 anywhere reject-with icmp-port-unreachable REJECT all -- 42.193.2.121 anywhere reject-with icmp-port-unreachable REJECT all -- 180.101.88.197 anywhere reject-with icmp-port-unreachable REJECT all -- 118.195.243.192 anywhere reject-with icmp-port-unreachable REJECT all -- 159.203.44.231 anywhere reject-with icmp-port-unreachable REJECT all -- 218.92.0.29 anywhere reject-with icmp-port-unreachable REJECT all -- 180.101.88.231 anywhere reject-with icmp-port-unreachable REJECT all -- 118.195.182.56 anywhere reject-with icmp-port-unreachable REJECT all -- 211.159.184.140 anywhere reject-with icmp-port-unreachable REJECT all -- 103.100.208.59 anywhere reject-with icmp-port-unreachable RETURN all -- anywhere anywhere Chain ufw-after-forward (1 references) target prot opt source destination Chain ufw-after-input (1 references) target prot opt source destination ufw-skip-to-policy-input udp -- anywhere anywhere udp dpt:netbios-ns ufw-skip-to-policy-input udp -- anywhere anywhere udp dpt:netbios-dgm ufw-skip-to-policy-input tcp -- anywhere anywhere tcp dpt:netbios-ssn ufw-skip-to-policy-input tcp -- anywhere anywhere tcp dpt:microsoft-ds ufw-skip-to-policy-input udp -- anywhere anywhere udp dpt:bootps ufw-skip-to-policy-input udp -- anywhere anywhere udp dpt:bootpc ufw-skip-to-policy-input all -- anywhere anywhere ADDRTYPE match dst-type BROADCAST Chain ufw-after-logging-forward (1 references) target prot opt source destination LOG all -- anywhere anywhere limit: avg 3/min burst 10 LOG level warn prefix "[UFW BLOCK] " Chain ufw-after-logging-input (1 references) target prot opt source destination LOG all -- anywhere anywhere limit: avg 3/min burst 10 LOG level warn prefix "[UFW BLOCK] " Chain ufw-after-logging-output (1 references) target prot opt source destination Chain ufw-after-output (1 references) target prot opt source destination Chain ufw-before-forward (1 references) target prot opt source destination ACCEPT all -- anywhere anywhere ctstate RELATED,ESTABLISHED ACCEPT icmp -- anywhere anywhere icmp destination-unreachable ACCEPT icmp -- anywhere anywhere icmp time-exceeded ACCEPT icmp -- anywhere anywhere icmp parameter-problem ACCEPT icmp -- anywhere anywhere icmp echo-request ufw-user-forward all -- anywhere anywhere Chain ufw-before-input (1 references) target prot opt source destination ACCEPT all -- anywhere anywhere ACCEPT all -- anywhere anywhere ctstate RELATED,ESTABLISHED ufw-logging-deny all -- anywhere anywhere ctstate INVALID DROP all -- anywhere anywhere ctstate INVALID ACCEPT icmp -- anywhere anywhere icmp destination-unreachable ACCEPT icmp -- anywhere anywhere icmp time-exceeded ACCEPT icmp -- anywhere anywhere icmp parameter-problem ACCEPT icmp -- anywhere anywhere icmp echo-request ACCEPT udp -- anywhere anywhere udp spt:bootps dpt:bootpc ufw-not-local all -- anywhere anywhere ACCEPT udp -- anywhere mdns.mcast.net udp dpt:mdns ACCEPT udp -- anywhere 239.255.255.250 udp dpt:1900 ufw-user-input all -- anywhere anywhere Chain ufw-before-logging-forward (1 references) target prot opt source destination Chain ufw-before-logging-input (1 references) target prot opt source destination Chain ufw-before-logging-output (1 references) target prot opt source destination Chain ufw-before-output (1 references) target prot opt source destination ACCEPT all -- anywhere anywhere ACCEPT all -- anywhere anywhere ctstate RELATED,ESTABLISHED ufw-user-output all -- anywhere anywhere Chain ufw-logging-allow (0 references) target prot opt source destination LOG all -- anywhere anywhere limit: avg 3/min burst 10 LOG level warn prefix "[UFW ALLOW] " Chain ufw-logging-deny (2 references) target prot opt source destination RETURN all -- anywhere anywhere ctstate INVALID limit: avg 3/min burst 10 LOG all -- anywhere anywhere limit: avg 3/min burst 10 LOG level warn prefix "[UFW BLOCK] " Chain ufw-not-local (1 references) target prot opt source destination RETURN all -- anywhere anywhere ADDRTYPE match dst-type LOCAL RETURN all -- anywhere anywhere ADDRTYPE match dst-type MULTICAST RETURN all -- anywhere anywhere ADDRTYPE match dst-type BROADCAST ufw-logging-deny all -- anywhere anywhere limit: avg 3/min burst 10 DROP all -- anywhere anywhere Chain ufw-reject-forward (1 references) target prot opt source destination Chain ufw-reject-input (1 references) target prot opt source destination Chain ufw-reject-output (1 references) target prot opt source destination Chain ufw-skip-to-policy-forward (0 references) target prot opt source destination DROP all -- anywhere anywhere Chain ufw-skip-to-policy-input (7 references) target prot opt source destination DROP all -- anywhere anywhere Chain ufw-skip-to-policy-output (0 references) target prot opt source destination ACCEPT all -- anywhere anywhere Chain ufw-track-forward (1 references) target prot opt source destination Chain ufw-track-input (1 references) target prot opt source destination Chain ufw-track-output (1 references) target prot opt source destination ACCEPT tcp -- anywhere anywhere ctstate NEW ACCEPT udp -- anywhere anywhere ctstate NEW Chain ufw-user-forward (1 references) target prot opt source destination Chain ufw-user-input (1 references) target prot opt source destination ACCEPT tcp -- anywhere anywhere tcp dpt:ftp ACCEPT tcp -- anywhere anywhere tcp dpt:ssh ACCEPT tcp -- anywhere anywhere tcp dpt:smtp ACCEPT tcp -- anywhere anywhere tcp dpt:domain ACCEPT tcp -- anywhere anywhere tcp dpt:http ACCEPT tcp -- anywhere anywhere tcp dpt:pop3 ACCEPT tcp -- anywhere anywhere tcp dpt:imap2 ACCEPT tcp -- anywhere anywhere tcp dpt:https ACCEPT tcp -- anywhere anywhere tcp dpt:submissions ACCEPT tcp -- anywhere anywhere tcp dpt:submission ACCEPT tcp -- anywhere anywhere tcp dpt:imaps ACCEPT tcp -- anywhere anywhere tcp dpt:pop3s ACCEPT tcp -- anywhere anywhere tcp dpt:mysql ACCEPT tcp -- anywhere anywhere tcp dpt:http-alt ACCEPT tcp -- anywhere anywhere tcp dpt:tproxy ACCEPT tcp -- anywhere anywhere multiport dports 40110:40210 ACCEPT udp -- anywhere anywhere udp dpt:domain ACCEPT udp -- anywhere anywhere udp dpt:netbios-ns ACCEPT udp -- anywhere anywhere udp dpt:netbios-dgm ACCEPT udp -- anywhere anywhere udp dpt:139 ACCEPT udp -- anywhere anywhere udp dpt:3306 ACCEPT tcp -- anywhere anywhere tcp dpt:ftp-data ACCEPT tcp -- anywhere anywhere tcp dpt:137 ACCEPT tcp -- anywhere anywhere tcp dpt:138 ACCEPT tcp -- anywhere anywhere tcp dpt:netbios-ssn ACCEPT tcp -- anywhere anywhere tcp dpt:microsoft-ds ACCEPT tcp -- anywhere anywhere tcp dpt:shell ACCEPT tcp -- anywhere anywhere tcp dpt:2812 ACCEPT tcp -- anywhere anywhere tcp dpt:8008 ACCEPT tcp -- anywhere anywhere tcp dpt:8091 ACCEPT tcp -- anywhere anywhere tcp dpt:27017 Chain ufw-user-limit (0 references) target prot opt source destination LOG all -- anywhere anywhere limit: avg 3/min burst 5 LOG level warn prefix "[UFW LIMIT BLOCK] " REJECT all -- anywhere anywhere reject-with icmp-port-unreachable Chain ufw-user-limit-accept (0 references) target prot opt source destination ACCEPT all -- anywhere anywhere Chain ufw-user-logging-forward (0 references) target prot opt source destination Chain ufw-user-logging-input (0 references) target prot opt source destination Chain ufw-user-logging-output (0 references) target prot opt source destination Chain ufw-user-output (1 references) target prot opt source destination should I have all these rules? what is going on here? something is blocking a lot (but not all) outgoing mail. tearing hair out!
This isn't related to the use of the Migration Tool, as the Migration Tool does not alter the server This is not related to the firewall. You can add a firewall, of course, but this will not make any difference as your system runs only services that shall be publicly available anyway; that's why the firewall is disabled by default. Yes, and these rules are not related to your problem and the problem is not on your server. Most likely, your new server has a new IP address, and this new IP address is on some blacklists: https://mxtoolbox.com/problem/blacklist And if you have spf records for your domains and you have set IP addresses in the records, then take care to adjust them.
thanks till. but new server is not on any blacklists either by ip or domain name. and new server was configured using the automated deb installer, and migtool nothing 'extra' done. did I miss something? I've had this address for many months at this point and its only now that I'm seeing all the mail problems. (admittedly just moved one website 'for real' to the server. glad I repointed only one!! I did the touch commands as per th0m and not seeing complaints about mailman now... thanks th0m. SOME mails goes out just fine. just not all emails. seemingly mails to outlook.com and yahoo. are all failing. on the original server, no mail issues. all I did was repoint the dns entries for the one domain from old server to new server. so mail etc all come to new server. but clearly much is unhappy! emails to an ipv6 seem to always fail. dont think verizon fios even gives an ipv6! so I guess I need to put inet_protocols = ipv4 in /etc/postfix/main.cf also I was seeing a lot of outlook.com incoming being greylisted. I added the file /etc/postgrey/whitelist_clients.local with contents: # outlook.com /^.*\.outbound\.protection\.outlook\.com$/ /^.*\.prod\.outlook\.com$/ we dont want all outlook originating email greylisted do we? also notice migtool does NOT propagate dns differences to new server, right? the ipaddress on the new server (under ispconfig dns) was not updated to point to itself - it still points to the old server (but ns11 is NOT first or second nameserver for the domain, but maybe its getting confused??)
So you know already that it's not an issue on your server; it's an issue on the receiver side, as otherwise it would fail for all outgoing emails. Outlook, as well as Yahoo have their internal blacklists; you can not know if you are on these internal lists. Outlook has a web form where you can report delivery failures and request to get whitelisted. If you do not have a working IPv6 address but IPv6 enabled on your Linux system so postfix tries to use it, then you should disable IPv6 in postfix indeed.
but till surely that is not right! if I were blocked by outlook surely I would get a bounceback email from outlook.com?? 'your email was not delivered due to ... DAM SPAM!!' or the like! I've had plenty of customers see such emails. and worked to get them unblocked. same with yahoo. I email my own account on yahoo.com - and I only see a connection timed out. not a bounceback telling me to buzz off! Its refusing connection entirely. surely that is a different problem???
Contact Yahoo and Microsoft to get your IP whitelisted. Today, it's common that all IPs that are not known to run a mail server for quite a while are blacklisted by default.
just submitted microsoft ip delist form and they tell me I'm not blacklisted! also yahoo says they use spamhaus. spamhaus has no issues with 108.18.202.58 again they are not rejecting due to spam! they are refusing connection! I never get the CHANCE to spam them LOL
Write them again by answering that email. Microsoft always claims you are not blacklisted in the first answer. I guess it's an autoresponder to try to stop people contacting them without any test done at all (see various threads here in the forum on that topic).
have written again. but I'm really thinking there has to be another reason. apparently verizon never installed the rdns I asked for months ago... possibly having a generic rdns would be enough reason to hang up on us!
spent hours talking to verizon (its a fios ip). they swear NO PROBLEMS with this ip address! what on earth is doing this?? how hard would it be to use ns10 (cox ip) as a relay for outgoing emails from ns11? presumably that would work - as the site was originally on ns10 and had no problems delivering emails?
ENLIGHTENMENT! the emails are never getting out of my network: I went back and used traceroute. root@ns11:/etc# traceroute -n -T -p 25 hotmail-com.olc.protection.outlook.com traceroute to hotmail-com.olc.protection.outlook.com (104.47.59.161), 30 hops max, 60 byte packets 1 * * * 2 * * * 3 * * * ...etc root@ns11:/etc# traceroute -n -T -p 80 hotmail-com.olc.protection.outlook.com traceroute to hotmail-com.olc.protection.outlook.com (104.47.58.161), 30 hops max, 60 byte packets 1 * * * 2 * 10.1.144.1 9.958 ms 108.18.202.1 10.049 ms 3 100.118.110.176 16.698 ms 100.41.27.224 10.060 ms 100.118.110.176 16.678 ms 4 * 100.121.20.46 16.681 ms * ...etc port 80 packets go through fine - just not port 25 packets! Then I remembered my wonderful nice dual router. Both servers are behind it... and then... hmmm...a few years ago I had a problem when customer computers (infected) were spamming emails out while attached to the internet in my store! Got yelled at by Cox at the time! CLEVER ME says I as I blocked all outgoing port 25 packets EXCEPT those originating from ns10 (the only server at the time). ns11 at a different internal ip - ... scales fall from eyes! added firewall rule allowing outgoing 25 packets from ns11 as well as ns10 - and mail flows properly! and when I was testing emails going from ns11 to ns10, all that of course worked fine as both on same internal network. so I thought some blacklist somewhere! and server had to be configured correctly. of course it is. its the dual wan router that was blocking things! boy was I happy to find it was NOT some internal microsoft unpublished blacklist. that would be impossible to deal with wouldnt it?? yet another amusing problem for your entertainment! many thanks as always! and not sure if I remember, but does migtool propagate dns changes after the original migration?
Yes. But you must run migrate on the SOURCE server that has the dns data. Try to avoid doing changes both on SOURCE and TARGET if you still are going to run the migrate script, it does not handle well if there are changes on both ends.
