Debian 10 Multiserver setup

I see where acme.sh installs, I see where self certs are created, but apparently acme.sh is not using this log when it runs.

Is there another log file that acme.sh is using?

 

Looks like out of the box the acme log is disabled, i can enable it and run force update, but we dont get the output on initial install

Code:

#LOG_FILE="/root/.acme.sh/acme.sh.log"
#LOG_LEVEL=2
AUTO_UPGRADE='2'
#NO_TIMESTAMP=2
UPGRADE_HASH='..........................'
DEFAULT_ACME_SERVER='https://acme-v02.api.letsencrypt.org/directory'

 

before i attempt, enable acme.sh logs?

 

Till as an aside, it may be worth looking at the installer to add the debug or log flag to the command running acme.sh when --debug is used on the installer.

 

I would go so far as to say during the install all logging/debugging options should be enabled.

I say this because as is shown here you could end up with a "broken" install and nothing to look at to determine the cause. Having all log files available should be the default. These log files can be purged when the installation was successful, or leave the logs that are required to solve any failed portion of the installation.

For example, ISPConfig already knows when to apply self signed so acme.sh logging/debugging should be active and purged for a successful cert creation.

One more thing that I noted. The self signed part, it happens automatically no question as to whether it should be used. What would be really useful here is a warning and an option.

Code:

acme.sh failed to generate a cert (see blah/blah.log). You will now be given the option to create a self signed certificate.

Warning: Self signed certificates prevent the creation of a Let's Encrypt certificate during updates.

Do you wish to create a self signed certificate:

With this option you can gracefully back out of a self signed, look at the logs, fix any issues and update ISPConfig.

The problem I found when I used ^c to bail out of a self signed, it actually breaks out of the installer entirely leaving the system in a state of partial configuration. IE, the ispconfig config files are not yet in place meaning that when you run an update it fails because it cannot find them.

This leaves two options:
Full removal of ISPConfig from the server (not an easy task).
Full reinstallation of the server

There is another option that could be done, for the installer. As it stands it checks for certain things and bails out if it finds them reporting ispconfig already installed. it could also check if its actually been fully configured. This way you can tell the installer to continue instead of just bailing out.

Sorry to do this here, maybe I should create an issue on git.

 

It worked 100%.. but i made changes. have wiped the server to basic. now in process of install ispconfig again and will run update with --debug using nightly.
the changes i made were IPV6 - to disable it 100%, was getting failed named lookups, even when diabled it still failed on them. so, will just do basic install, ispconfig install, the update nightly.
back in 10
Oh, i made these changes earlier..

Code:

vim /etc/sysctl.d/99-sysctl.conf
adding to end of the file
net.ipv6.conf.all.disable_ipv6 = 1
net.ipv6.conf.default.disable_ipv6 = 1
net.ipv6.conf.lo.disable_ipv6 = 1
net.ipv6.conf.tun0.disable_ipv6 = 1

Code:

vim /etc/bind/named.conf.options
//      listen-on-v6 { any; };

 

An update, so yes on initial install creates self cert.
Running

Code:

root@panel:~# ispconfig_update.sh --force


--------------------------------------------------------------------------------
 _____ ___________   _____              __ _
|_   _/  ___| ___ \ /  __ \            / _(_)
  | | \ `--.| |_/ / | /  \/ ___  _ __ | |_ _  __ _
  | |  `--. \  __/  | |    / _ \| '_ \|  _| |/ _` |
 _| |_/\__/ / |     | \__/\ (_) | | | | | | | (_| |
 \___/\____/\_|      \____/\___/|_| |_|_| |_|\__, |
                                              __/ |
                                             |___/
--------------------------------------------------------------------------------


>> Update 

Please choose the update method. For production systems select 'stable'.
WARNING: The update from GIT is only for development systems and may break your current setup. Do not use the GIT version on servers that host any live websites!
Note: On Multiserver systems, enable maintenance mode and update your master server first. Then update all slave servers, and disable maintenance mode when all servers are updated.

Select update method (stable,nightly,git-develop) [stable]: nightly

Downloading ISPConfig update.
Unpacking ISPConfig update.


--------------------------------------------------------------------------------
 _____ ___________   _____              __ _         ____
|_   _/  ___| ___ \ /  __ \            / _(_)       /__  \
  | | \ `--.| |_/ / | /  \/ ___  _ __ | |_ _  __ _    _/ /
  | |  `--. \  __/  | |    / _ \| '_ \|  _| |/ _` |  |_ |
 _| |_/\__/ / |     | \__/\ (_) | | | | | | | (_| | ___\ \
 \___/\____/\_|      \____/\___/|_| |_|_| |_|\__, | \____/
                                              __/ |
                                             |___/
--------------------------------------------------------------------------------


>> Update 

Operating System: Debian 10.0 (Buster) or compatible

This application will update ISPConfig 3 on your server.

Shall the script create a ISPConfig backup in /var/backup/ now? (yes,no) [yes]:

Creating backup of "/usr/local/ispconfig" directory...
Creating backup of "/etc" directory...
Checking ISPConfig database .. OK
Starting incremental database update.
Loading SQL patch file: /tmp/update_runner.sh.YePhdNQsdz/install/sql/incremental/upd_dev_collection.sql
Reconfigure Permissions in master database? (yes,no) [no]:

Service 'mail_server' has been detected (currently disabled) do you want to enable and configure it?  (yes,no) [no]:

Service 'dns_server' has been detected (currently disabled) do you want to enable and configure it?  (yes,no) [no]:

Reconfigure Services? (yes,no,selected) [yes]:

Configuring Pureftpd
Configuring Apache
Configuring vlogger
Configuring Apps vhost
Configuring Jailkit
Configuring Ubuntu Firewall
Configuring Database
Updating ISPConfig
ISPConfig Port [8080]:

Create new ISPConfig SSL certificate (yes,no) [no]: yes

Checking / creating certificate for panel.tlwebservices.co.uk
Using certificate path /root/.acme.sh/panel.tlwebservices.co.uk
Using apache for certificate validation
acme.sh is installed, overriding certificate path to use /root/.acme.sh/panel.tlwebservices.co.uk
Symlink ISPConfig SSL certs to Postfix? (y,n) [y]: y

Symlink ISPConfig SSL certs to Pure-FTPd? Creating dhparam file may take some time. (y,n) [y]: 

Reconfigure Crontab? (yes,no) [yes]:

Updating Crontab
Restarting services ...
Update finished.

and it works.
Awsome..
So, this was run without disabling any IPV6, verified DNS is correct(unless you spot its wrong)
Thank you devs..

 

Hi chief, nice to see you have it working.

To confirm that I have understood your last post: The installer failed and the forced update succeeded in generating the Let's Encrypt certificate.

Do you still have the install and update log files available? I appreciate that you are likely very busy and now want to move onto the next server, but, If you wouldn't mind doing so, can you detail your steps to reproduce this. You can open a direct conversation with me if you prefer.

I would like to see if i am able reproduce this behaviour. I believe this warrants some investigation because if the updater succeeds then so should the installer.

 

====EDITS=== forgot to attach files== here goes===
ispconfig_install.log
setup log
syslog

Yes, initial install fails on letsencrypt cert generation and falls back to self made. Then as Till suggested, i

Code:

ispconfig_update.sh --force

choosing

Code:

nightly

it then successfully updates and creates a letsencrypt certificate. i verified this after initial 1st install - its self signed, then ran the update and then view cert and its letsencrypt.
I have sent you my mobile number if you want to view this live remoting in, before i get it all finished and other servers activated and tied in. as a thought, if panel.tlwebservices.co.uk fails on initial cert, then also other servers will fail as well.
now i have a working method, should be able to sort.
Also as a issue i have met.. The initial debian installer when working with raid setups..
The issue i experienced whicle setting up Dell poweredge R710 - 4 x 2TB hs's - raid 10. was the automatic partitioning never worked 100%, after install, i would reboot and it, 90% of the time i failed to find boot. The error was something like cant see outside of hd0, so the solution was manual partitioning. And this works every time.

Code:

/boot 1GB and the beginning of drive
/ file system 4TB  (wherever)
/swap 10GB and the end of drive

 

That seems likely to be a compatibility issue between the virtualisation platform and the Debian installer/OS.

Searched: debian 10 VM failed to find boot on Dell R710
First result: https://www.dell.com/support/kbdoc/...-device-available-is-displayed-during-startup

I don't know if that was the issue here but it seems at first glance like it might be. I am not (at all) familiar with the platform. For my vmware/esxi platform you select a template before you select the ISO to boot from. This loads OS specific configurations to aid compatibility.

Anyway, I will take a copy of the logs and dig through them. As I am not an ISPConfig dev I will not take you up on that because it would take me too long to dig into the files if needed than they would but it could be something Till, or one of the other devs take you up on. For my part I will take a look through the logs and see if there is anything that can be seen to be causing this.

 

You miss understand.
i have 7 x real machines not virtual, looks like a bug in where debian etc puts the boot loader, it has to be at the beginning of the drive

 

====EDITS TO ABOVE====
An addition to installer, when installing web01.tlwebservices.co.uk and connecting it to panel.tlwebservices.co.uk.
The installer runs and ends creating a self signed cert. had to delete ispconfig cert from

Code:

/usr/local/ispconfig/interface/ssl/ispconfig.*

re run the

Code:

ispconfig_update.sh --force take the nightly option

and it will successfully create letsencrypt cert.
I take it all the servers who need letsencrypt cert, i will have to do the same process.

 

Yes it is looking like that is the case, this is point more to an issue with the installer than with the configurations of the server.

I am still looking through these logs. I see a lot of IPV6 lines coming from named which can't be a good thing.

Code:

Aug 18 13:00:02 panel named[18142]: network unreachable resolving 'nsp.dnsnode.net/AAAA/IN': 2a01:3f1:3032:8000::53#53

It is possible that IPV6 is breaking the cert request some how, as yet I am unable to see where that comes from. No acme.sh log file to check.

With regard to my misunderstanding, I apologise for that. I assumed you were virtualising so searching again, the same result came back first. There is a link a few lines into it about how to install on the server. Did you check Debian 10 was compatible with the R710?

 

I have read lots about disabling IPV6 for debian, so far i found these 2 things to disable [/code]
1.vim /etc/sysctl.d/99-sysctl.conf

net.ipv6.conf.all.disable_ipv6 = 1
net.ipv6.conf.default.disable_ipv6 = 1
net.ipv6.conf.lo.disable_ipv6 = 1
net.ipv6.conf.tun0.disable_ipv6 = 1

2.vim /etc/bind/named.conf.options
// listen-on-v6 { any; };[/code]
is there any mileage in me disabling it? as im only using IPV4 and not 6 anyway

 

Till has now fixed this issue in the dev channel. see this reply to the acme support thread for details and how to install from the dev channel: https://www.howtoforge.com/community/threads/acme.87423/page-2#post-426211.

I may be partially responsible here for this taking longer than it should have to resolve. Chief asked about the API credentials and I incorrectly replied with no they are not needed because until this point they were not. I had not considered that something had changed in the third party program acme.sh resulting in this new situation. Instead I assumed that the ISPConfig auto installation was doing something wrong. Technically it was but certainly it was no fault of the devs in this case.

While I foresee no further issues on this, I am going to run some tests later this evening. I will install some fresh VM's overnight, debian 10 and ubuntu 20.04 using both nginx and apache. Rather than posting a new reply to this already long thread I will instead update this reply with the result.


I have performed two new deployments.

Ubuntu 20.04 using Apache2 and Nginx. Both succeeded in obtaining a cert during Installation. I don't see it being different in Debian servers.
This is the installation command for multi server hence no mail/dns. The key part for any installation at least until 3.2.6 is released is adding the --channel=dev to obtain the latest fixes from the git stable branch. Personally I would wait until the release but if you don't want to, that's how you get it to install a cert first time around.

add --help to see more available parameters.

Code:

wget -O - https://get.ispconfig.org | sh -s -- --no-mail --no-dns --use-php=system --channel=dev

If you are here because you already have an installation that failed to get an ssh cert

Code:

ispconfig_update.sh --force

When prompted for the type of upgrade you want, type nightly

That last part is now in two or three replies on this thread, hopefully it will not be missed from now on.

 

I have not followed your fix, i ran initial install then deleted certs, then ran update with --force. have letsencrypt certs.
The above bug for DNSSEC.. when making pri nameserver, there isnt any transfer option

just sign zone (DNSSEC)
see image
and other question. adding to secondary DNS zone, the NS IP - is this the IP for secondary name server? and last box, allow zone transfers to these IP's- would this be both primary and secondary IP's?

also, If my dns records are with external, is it best practice to create all hostnames under external dns or create 2 x nameservers at external ns1+ ns2 pointing to my IP, then create tlwebservices.co.uk and other host names here?
also, the mx servers.. under base dns on ispconfig, it creates A record name as mail pointing to 212.159.153.2, should this be mx1 or mx2? how does the secondary backup mail server fit in to dns?

 

That is a new issue Chief, you should open a new post with the new situation.