migration AFTER migration has been done... does it affect new server?

I've been running ./migration --syncjobs to sync old and new servers for a bit now.
I moved over a site manually to check out the new mail handling under deb12.

but if I do a --syncjobs now will I messup the emails on the new server?? there should be new emails on new server, but also new emails on the old server till the change percolates?
and are dns changes also updated on new server?

anything to be concerned about?

 

I suspected as much. now is it possible to do a migrate --syncjobs EXCLUDING a single domain? or for only a single domain?
and if the DNS entries are different on target and source server after a migration will a new migration propagate the differences?

 

I suspected. but if these shell commands are stored somewhere I can get at, maybe I can REMOVE the ones affecting the domain I have moved manually?

 

I'll have a whack! pinata!

oh - just noticed a nasty after migration I can no longer access ns11.cdbsystems - I get a cert error and no ability to enter an exception:
--snip--
from the cert:
Common Name (CN)
ns11.cdbsystems.com
..then..
Issued By
Common Name (CN)
R3
Organization (O)
Let's Encrypt
Organizational Unit (OU)
<Not Part Of Certificate>
Validity Period
Issued On
Wednesday, November 15, 2023 at 6:55:29 PM
Expires On
Tuesday, February 13, 2024 at 6:55:28 PM
--snip--
so the LE cert is no longer valid.
I ran ispconfig_update.sh told it to generate a new SSL and same error.
also during the update run I saw something new:

Code:

Create new ISPConfig SSL certificate (yes,no) [no]: yes

Checking / creating certificate for ns11.cdbsystems.com
Using certificate path /etc/letsencrypt/live/ns11.cdbsystems.com
sh: 1: cannot open /dev/tcp/127.0.0.1/80: No such file
Using apache for certificate validation
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Symlink ISPConfig SSL certs to Postfix? (y,n) [y]:

Symlink ISPConfig SSL certs to Pure-FTPd? Creating dhparam file may take some time. (y,n) [y]:

I have not seen the cannot open /dev/tcp/127.0.0.1/80 - whats this about??
just as well I have ssh access still LOL

 

Interesting! and perplexing. a couple of the domains I was hosting no longer exist. so obviously certbot fails for those.
however from the log just tied it again:

Code:

2024-02-24 09:04:00,166:DEBUG:certbot._internal.cert_manager:Renewal conf file /etc/letsencrypt/renewal/ns11.cdbsystems.com.conf is broken. Skipping.
2024-02-24 09:04:00,166:DEBUG:certbot._internal.cert_manager:Traceback was:
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/certbot/_internal/cert_manager.py", line 437, in _search_lineages
    candidate_lineage = storage.RenewableCert(renewal_file, cli_config)
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/certbot/_internal/storage.py", line 485, in __init__
    raise errors.CertStorageError(
certbot.errors.CertStorageError: renewal config file {} is missing a required file reference

2024-02-24 09:04:00,168:DEBUG:certbot._internal.cert_manager:Renewal conf file /etc/letsencrypt/renewal/pinnaclehealthcaredmv.com.conf is broken. Skipping.
2024-02-24 09:04:00,169:DEBUG:certbot._internal.cert_manager:Traceback was:
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/certbot/_internal/cert_manager.py", line 437, in _search_lineages
    candidate_lineage = storage.RenewableCert(renewal_file, cli_config)
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/certbot/_internal/storage.py", line 485, in __init__
    raise errors.CertStorageError(
certbot.errors.CertStorageError: renewal config file {} is missing a required file reference

2024-02-24 09:04:00,195:DEBUG:urllib3.connectionpool:Starting new HTTP connection (1): r3.o.lencr.org:80
2024-02-24 09:04:00,373:DEBUG:urllib3.connectionpool:http://r3.o.lencr.org:80 "POST / HTTP/1.1" 200 503
2024-02-24 09:04:00,374:DEBUG:certbot.ocsp:OCSP response for certificate /etc/letsencrypt/archive/ns11.cdbsystems.com-0001/cert2.pem is signed by the certificate's issuer.
2024-02-24 09:04:00,376:DEBUG:certbot.ocsp:OCSP certificate status for /etc/letsencrypt/archive/ns11.cdbsystems.com-0001/cert2.pem is: OCSPCertStatus.GOOD
2024-02-24 09:04:00,386:DEBUG:certbot._internal.display.obj:Notifying user: Certificate not yet due for renewal
2024-02-24 09:04:00,386:INFO:certbot._internal.main:Keeping the existing certificate
2024-02-24 09:04:00,386:DEBUG:certbot._internal.display.obj:Notifying user: Certificate not yet due for renewal; no action taken.

so its saying the conf file is broken and also the cert is not due for renewal? and various python errors? whats going oN?

now in the directory /etc/letsencrypt/renewal we see:
-rw-r--r-- 1 root root 762 Jan 24 16:00 ns11.cdbsystems.com-0001.conf
-rw-r--r-- 1 root root 0 Nov 20 11:04 ns11.cdbsystems.com.conf
-rw-r--r-- 1 root root 737 Nov 15 19:55 ns11.cdbsystems.com.conf~backup

so we have a zero length ns11.cdbsystems.com.conf file. but there is a 0001.conf file? and whats the conf~backup file?


also -- I cannot log into ispconfig at all on ns11 I have no option to add an exception! how do i log in with an invalid ssl?

 

Recent versions of browsers do not allow exception for failing ssl anymore. Your options are:

1. find older version of browser that does allow security excaption
2. your browser should have somewhere in the settings way to go back to the old behaviour and allow security excaption for named website
3. fix the Let's Encrypt stuff so your host does get certificate and SSL works.

Try fixing by removing the certificate with broken conf file:

Code:

certbot delete --cert-name ns11.cdbsystems.com

Then force making new certificate, with ispconfig_update.sh --force.
If you have certificates made for domains that no longer exist, at all or on your server, you should delete those certificates. Check what certificates there are with

Code:

certbot certificates

 

for once I was ahead of you!
certbot delete --cert-name ns11.cdbsystems.com-0001 (get rid of 0001 certs)
certbot delete --cert-name ns11.cdbsystems.com
ispconfig_update.sh --force still failed to make cert. complained about 'archive folder'
deleted /etc/letsencrypt/archive/ns11.cdbsystems.com, /etc/letsencrypt/live/ns11.cdbsystems.com /etc/letsencrypt/renewal/ns11.cdbsystems.com folders
reran ispconfig_update.sh --force and make cert. all ok!
I still get the
sh: 1: cannot open /dev/tcp/127.0.0.1/80: No such file
during ispconfig_update.sh I have not seen this before. any ideas?
but anyway ns11 is now accessible with good cert!

 

I've been playing with syncjobs.dat and I notice there are NO references to the vmail folder. so syncjobs does not update mail on the target? or is there something its invoking that I'm not seeing? or are you sneakily referring to vmail by some other name to confuse old programmers with poor eyesight?

 

ok but I'm analyzing my syncjobs.dat and I find no references to /var/vmail anywhere!
so the syncing must happen in migrate and is hard coded right? cant do anything about that can I?

 

I guess i could resync vmail backwards before running syncjobs but then ill lose any emails that appear before --syncjobs gets to that mail
folder. we assume you are just rsyncing the whole /var/vmail at the same time? so we cant exclude one domain?
./migrate --syncjobs --exclude-domain=domainalreadymoved.com
would be outstanding!


One oddity im seeing too i am seeing obvious spam on the new deb12 server that my customer is complaining about! Emails offering condoms that i would have thought surely rspamd would have caught!

 

I tested this out on a folder and indeed the emails move over. however roundcube (installed by instaserver) sees all the emails but cannot delete any! it gives a 'cannot move message' error
on emails moved over by migrate (and not updated with rsync) deletion works properly!
I thought it might be an ownership issue (as rsync was logging in as root?) and chown vmail:vmail -R the vmail folder. but no luck
any ideas?