SSH Service Inactive (dead) after installing ISP Config on Ubuntu 22.04

Hosting provider : Aruba Business
Machine : Hypervisor Openstack - 2 CPU - 4 GB RAM - SSD
OS distro : Ubuntu Server 22.04 LTS 64bit
ISP Config version: 3.2.9

Hi, I'm experiencing a very strange behavior with ssh service.
I can log in my machine via SSH and I can proceed successfully throughout the installation of ISP Config 3.2.9 following the HowtoForge tutorial "Perfect Server Automated ISPConfig 3 Installation on Debian 10, Debian 11, Ubuntu 20.04 and Ubuntu 22.04" with this installation arguments:
--use-php=8.1 --use-ftp-ports=40110-40210 --no-mail --no-dns --no-local-dns --no-roundcube --monit [email protected] --unattended-upgrades=autoclean

After installing ISP Config and rebooting the VPS I can Log in to the ISPConfig UI to setup the Firewall and configure everything else.
But... when I try to access the machine via SSH I receive a "Connection refused" error. Firewall is OK.

When I log in the console via hypervisor I can see the output of systemctl status ssh:

ssh. service - OpenBSD Secure Shel1 server
Loaded: loaded (/lib/systemd/system/ssh.service; enabled; vendor preset: enabled)
Active: inactive (dead) since Thu 2023-01-05 18:06:23 CET; 50min ago
Docs: man:sshd (8)
man:sshd_config (5)
Process: 728 ExecStartPre=/usr/sbin/sshd -t (code-exited, status=0/SUCCESS)
Process: 752 ExecStart=/usr/sbin/sshd -D $SSHD_OPTS (code=exited, status=0/SUCCESS)
Main PID: 752 (codesexited, status=0/SUCCESS)
CPU: 12ms
Jan 05 18:06:22 webserver-22 systemd[1]: Starting OpenBSD Secure Shel1 server...
Jan 05 18:06:22 webserver-22 sshd [752]: Server listening on 0.0.0.0 port 9190.
Jan 05 18:06:22 webserver-22 systemd [1]: Started OpenBSD Secure Shell server.
Jan 05 18:06:22 webserver-22 sshd [752]: Server listening on :: port 9190.
Jan 05 18:06:28 webserver-22 systemd[1]: Stopping OpenBSD Secure Shell server...
Jan 05 18:06:23 webserver-22 systemd[1]: ssh.service: Deactivated successfully.
Jan 05 18:06:23 webserver-22 sshd [752]: Received signal 15; terminating.
Jan 05 18:06:23 webserver-22 systemd[1]: Stopped OpenBSD Secure Shel1 server.​

If I manually enable ssh service, something (maybe ISP Config itself?) stops it immediately.

Can you please help me?
Thanks in advance!
Nico

 

Check if you swap file is also properly set in case of system runs out of memory as suggested by @till above. I noticed some took this swap thingy very lightly and ended up having avoidable problem(s) on their server.

 

Thanks everyone.
@ahrasis the swap file is ok: Total 524284 Used 0 Free 524284
@till I'm sure the problem is not with ISP Config, but it is definitely related to the autoinstaller script.
I did a fresh install, starting from scratch again.
These are the simple steps to reproduce my problem:
- Ubuntu Server 22.04 freshly installed, ssh service is active and working.
- I run the ISP Config autoinstaller script; as soon as the script finishes, without restarting the server, ssh is NOT active anymore.

I'm trying to figure out what the script installs and configures. Any help is greatly appreciated. Thanks again!

 

Hi everyone!
@till /etc/ssh/sshd_config.d/custom.conf is a one row file with:

# Created by the ISPConfig autoinstaller on 2023-01-07​

Thanks to a thorough investigation I discovered that the issue was monit!
Running the ISP Config auto installer script without the --monit and --monit-alert-email arguments the problem no longer occurs.

I still don't know why, but it just seems that installing monit breaks the ssh service on a freshly installed Ubuntu Server 22.04

Anyone able to test this behavior with me?

 

you may just need to install rkhunter if it's not installed and then run rkhunter --propupd and restart/reload monit, it may be that monit notices that the ssh files don't match the checksum it expects and so stops the service because it may be unsafe/compromised.

 

I don't see these monit params in the tutorial nor that I have tested it via cli before but I do believe this should be investigated though @nhybgtvfr suggestion may also be good.

 

@ahrasis the last version of the ISP Config autoinstaller has these monit related arguments:
--monit. Install Monit and set it up to monitor installed services. Supported services: Apache2, NGINX, MariaDB, pure-ftpd-mysql, php-fpm, ssh, named, Postfix, Dovecot, rspamd.
--monit-alert-email Set up alerts for Monit to be send to given e-mail address. e.g. [email protected]

@till I also tried on a local vm and everything works fine, but I don't know why on that Aruba VPS (freshly installed) the problem occurs every time I try to install ISP Config with the monit arguments.
I just found messages from the monit service in my spam box. I list them below:

Does not exist Service redis-server
Date: Sun, 08 Jan 2023 11:33:45
Action: restart
Description: process is not running

Execution failed Service redis-server
Date: Sun, 08 Jan 2023 11:34:15
Action: alert
Description: failed to start (exit status 5) -- '/usr/bin/systemctl start redis-server': Failed to start redis-server.service: Unit redis-server.service not found.

Timeout Service redis-server
Date: Sun, 08 Jan 2023 11:40:20
Action: unmonitor
Description: service restarted 5 times within 5 cycles(s) - unmonitor

Exists Service pure-ftpd-mysql
Date: Sun, 08 Jan 2023 11:41:22
Action: alert
Description: process is running with pid 1169

Does not exist Service sshd_dsa_key
Date: Sun, 08 Jan 2023 11:33:44
Action: restart
Description: file doesn't exist

Thanks and regards.

 

Does not exist Service sshd_dsa_key
Date: Sun, 08 Jan 2023 11:33:44
Action: restart
Description: file doesn't exist

well. that's probably why ssh isn't working..

you can probably get around that, if you can connect in any way, by removing

Code:

   depend on sshd_dsa_key

and possibly

Code:

 check file sshd_dsa_key with path /etc/ssh/ssh_host_dsa_key
   group sshd
   include /etc/monit/templates/rootstrict

from /etc/monit/conf-enabled/sshd and restart monit.

although it might then progress to complaining about some other ssh file...
no idea why that key file doesn't exist though.... also doesn't look like redis got installed/configured properly either.

wonder if there's possibly somethiing odd / non-standard in the Aruba ubuntu images...

 

The problem is that monit is killing sshd server. sshd stop litenning on port 22. If you use different port and not 22 you have change configuration on /etc/monit/conf-enabled/sshd. If it does not work yo have to remove file /etc/monit/conf-enabled/sshd

Code:

sudo rm /etc/monit/conf-enabled/sshd
sudo systemctl restart monit

 

you do not have to remove /etc/monit/conf-enabled/sshd.
you'll want that service monitored.
yes, if you're using a different port you'll want to change the configuration of the checks.
also. if any of the dependant files get changed or are missing, you'll want to check that those files were changed legitimately or why they don't exist, update the expected configuration/checksums for those files, and reload/restart monit so it uses the correct checksums.
ie check the rkhunter logs to see what was changed, when, then check if a legitimate update was applied at that time. if so, run rkhunter --propupd to update the checksums, and reload monit.. or repair/reinstall the service files with ones from a trusted repository.

 

I had the exact same issue. I used the automated installer including the --monit line, and noticed that ssh would stop functioning each time I started it. The solution that worked for me was this one mentioned earlier in this thread by nhybgtvfr. Thanks!