Problem with Passenger UstRouter

After apt update && apt upgrade, as ISPconfig said updates available, on my fine running webserver (multiserver system) Debian 10 Buster ( Debian Testing) ISPconfig 3.1.1.15p3 my websites are no longer available. Apache error.log says
[ 2020-06-15 08:58:04.9014 2951/7f8946462700 age/Cor/CoreMain.cpp:531 ]: Signal received. Gracefully shutting down... (send signal 2 more time(s) to force shutdown)
[ 2020-06-15 08:58:04.9018 2951/7f8946d51980 age/Cor/CoreMain.cpp:900 ]: Received command to shutdown gracefully. Waiting until all clients have disconnected...
[ 2020-06-15 08:58:04.9023 2951/7f8945be0700 Ser/Server.h:816 ]: [ApiServer] Freed 0 spare client objects
[ 2020-06-15 08:58:04.9023 2951/7f8945be0700 Ser/Server.h:464 ]: [ApiServer] Shutdown finished
[ 2020-06-15 08:58:04.9028 2951/7f8946462700 Ser/Server.h:816 ]: [ServerThr.1] Freed 128 spare client objects
[ 2020-06-15 08:58:04.9029 2951/7f8946462700 Ser/Server.h:464 ]: [ServerThr.1] Shutdown finished

[ 2020-06-15 08:58:04.9115 2956/7f5b3a4ab700 age/Ust/UstRouterMain.cpp:421 ]: Signal received. Gracefully shutting down... (send signal 2 more time(s) to force shutdown)
[ 2020-06-15 08:58:04.9119 2956/7f5b3a59c980 age/Ust/UstRouterMain.cpp:491 ]: Received command to shutdown gracefully. Waiting until all clients have disconnected...
[ 2020-06-15 08:58:04.9122 2956/7f5b39c29700 Ser/Server.h:816 ]: [UstRouterApiServer] Freed 0 spare client objects
[ 2020-06-15 08:58:04.9122 2956/7f5b39c29700 Ser/Server.h:464 ]: [UstRouterApiServer] Shutdown finished
[ 2020-06-15 08:58:04.9130 2956/7f5b3a4ab700 Ser/Server.h:464 ]: [UstRouter] Shutdown finished
[ 2020-06-15 08:58:04.9137 2956/7f5b3a59c980 age/Ust/UstRouterMain.cpp:523 ]: Passenger UstRouter shutdown finished
[ 2020-06-15 08:58:04.9258 2951/7f8946d51980 age/Cor/CoreMain.cpp:967 ]: Passenger core shutdown finished
[ 2020-06-15 08:58:05.1795 2979/7f5382328980 age/Wat/WatchdogMain.cpp:1291 ]: Starting Passenger watchdog...
[ 2020-06-15 08:58:05.2740 2982/7f65e57ed980 age/Cor/CoreMain.cpp:982 ]: Starting Passenger core...
[ 2020-06-15 08:58:05.2751 2982/7f65e57ed980 age/Cor/CoreMain.cpp:235 ]: Passenger core running in multi-application mode.
[ 2020-06-15 08:58:05.2795 2982/7f65e57ed980 age/Cor/CoreMain.cpp:732 ]: Passenger core online, PID 2982
[ 2020-06-15 08:58:05.3354 2987/7f85c2121980 age/Ust/UstRouterMain.cpp:529 ]: Starting Passenger UstRouter...
[ 2020-06-15 08:58:05.3396 2987/7f85c2121980 age/Ust/UstRouterMain.cpp:342 ]: Passenger UstRouter online, PID 2987
PHP Fatal error: apc_mmap: mkstemp on /tmp/apc/apc.d2usA8 failed in Unknown on line 0

[ 2020-06-15 08:58:05.4702 2982/7f65e4efe700 age/Cor/CoreMain.cpp:531 ]: Signal received. Gracefully shutting down... (send signal 2 more time(s) to force shutdown)

[ 2020-06-15 08:58:05.4703 2987/7f85c2030700 age/Ust/UstRouterMain.cpp:421 ]: Signal received. Gracefully shutting down... (send signal 2 more time(s) to force shutdown)
[ 2020-06-15 08:58:05.4708 2987/7f85c2121980 age/Ust/UstRouterMain.cpp:491 ]: Received command to shutdown gracefully. Waiting until all clients have disconnected...
[ 2020-06-15 08:58:23.5862 3068/7f5cb7ad5700 Ser/Server.h:816 ]: [UstRouterApiServer] Freed 0 spare client objects
[ 2020-06-15 08:58:23.5862 3068/7f5cb7ad5700 Ser/Server.h:464 ]: [UstRouterApiServer] Shutdown finished
[ 2020-06-15 08:58:23.5863 3062/7f1e68785980 age/Cor/CoreMain.cpp:900 ]: Received command to shutdown gracefully. Waiting until all clients have disconnected...
[ 2020-06-15 08:58:23.5866 3062/7f1e62ffd700 Ser/Server.h:816 ]: [ApiServer] Freed 0 spare client objects
[ 2020-06-15 08:58:23.5866 3062/7f1e62ffd700 Ser/Server.h:464 ]: [ApiServer] Shutdown finished
[ 2020-06-15 08:58:23.5868 3068/7f5cb8357700 Ser/Server.h:464 ]: [UstRouter] Shutdown finished
[ 2020-06-15 08:58:23.5876 3062/7f1e637fe700 Ser/Server.h:816 ]: [ServerThr.1] Freed 128 spare client objects
[ 2020-06-15 08:58:23.5876 3062/7f1e637fe700 Ser/Server.h:464 ]: [ServerThr.1] Shutdown finished
[ 2020-06-15 08:58:23.5895 3068/7f5cb8448980 age/Ust/UstRouterMain.cpp:523 ]: Passenger UstRouter shutdown finished
The logs shows an apache 2 restart after I disabled all websites,.
The file listet in the PHP error does not exist.
Searched the internet for that error but did not found some thing usefull.

Rainer

 

Did the upgrade finish? If you do

Code:

apt upgrade

does it show something is to be upgraded?
The very long log you posted looks like apache shutting down, with obscure error message in the middle. Why do you not pick the error message and post only that, why the wall of text that in no way indicates why apache does not work?
Have you tried starting apache?

Code:

systemctl start apache2

Are all services running?

Code:

systemctl --state=failed

Does rebooting fix the issue?

 

I tried these things
root@http1:/var/log/apache2# df -h^C
root@http1:/var/log/apache2# systemctl start apache2
root@http1:/var/log/apache2# systemctl --state=failed
UNIT LOAD ACTIVE SUB DESCRIPTION
0 loaded units listed.
root@http1:/var/log/apache2#
Reboot does not solve
Thats why I show the whole log restrating apache, so somebody may see something in the history.
Never heard or saw this "passenegr ust router" before nor do I know what ist does.
Search the internet for it, but did not really understand
Rainer

 

If all services are running, also Apache is running. In what way are you websites not available?
Internet Search Engines tell me Passenger UstRouter is something to do with Ruby. You could disable ruby for a website.

 

I have nothing needing Ruby as I know, surely not any of my websites. On the other hand apache seems not to run, ISPconfig Monitor says offline, systemcl status says online. can ping the website but the browse returns timeout. actually all websites are disabled just one simple only html is enabled, even this one says timeout.
The apache access log is empty, it has not even an relativ actual timestamp.
Rainer

 

Are you really accessing your host? If apache access log shows nothing, it may be name service has wrong info. You could try this to make sure you are accessing your own host: https://www.faqforge.com/linux/cont...ess-a-namebased-website-without-a-dns-record/
Is ruby now disabled on all websites?

 

Still have problem. What I did/tried
1. uninstalled Ruby, as I do not need for a web service
2. disabled all web sites, enabled just one website with only the default htmlfile ispconfig creates adding a new webseite. disabled php, unchecked suexec, ssl
3. restart apache
still only an error log, no access log. error.log updates every time apache starts/restarts.
[Thu Jun 25 06:55:36.969962 2020] [suexec:notice] [pid 1876003] AH01232: suEXEC mechanism enabled (wrapper: /usr/lib/apache2/suexec)
PHP Fatal error: apc_mmap: mkstemp on /tmp/apc/apc.KyIUsV failed in Unknown on line 0
The first line is just a notice I learned, the second line seams to be the problem.
PHP -v says ver. 7.4.5, but php ist not involved I think as the is just html code
systemctl status apache2 says running, but ispconfig monitor says apache not running.
top shows high cpu, going away with stopping apache.
Did a lot of google research, but cannot find a solution.
Do you have any hint what I still can do. Do you need further information?

Thanks Rainer

 

You wrote

but never said what exactly happens when you try to access the website.
Have you done what I told you to do in #6 to access the website?

What do other logs show?
Start with basic stuff, seems you have not done these yet:
https://www.howtoforge.com/community/threads/please-read-before-posting.58408/
How was this ISPConfig system installed? Have you altered it in some ways not specified in the Perfect Server Guide?
Since you get

have you tried disabling apc?

 

I did an apt update && apt upgrade, after this I always check all the functions on my servers. By this I noticed all the websites on that server had a timeout after about 15 seconds on every website. Thats is still the case.
typically when there is something with the web site you look at the apache logs.
After uninstall of ruby, the error was reduced to the php error shown in the last post. No access log is updated.
The system was running since weeks, it was installed in a combination of the perfect server guide for buster and the multiserversetup from the latest ispconfig manual as all my other servers. I am working with ispconfig about 10 or more years now with of 5 servers in my productive system, and 4 servers in my test system.
ispconfig is well communicating with this slave as when I change something in ispconfig regarding sites these changes are seen on the server i.e. enabeling disabeling site are seen in sites available in the corresponding apache directories.
It seams apache tries to start but hangs in case of the php error. I ist not normal seeing high cpu load when apache is running without a website is accessed.
If there is an access problem in case of wrong dns (nothing was changed the, dns run on an other server) the browser returns an error imediately , timeouts I know if the webserver ist basicly reachableb but not really responding.
The servers themselfs are all VM on a ESXi5.5 on an internal net, behind a firewall (a VM itself with 2 interfaces one to the internal net another to the internet, nothing changed there). As the server(s), my own hardware, is running in a datacenter I access them via SSH and sometime via an VMware client).

Rainer

 

Did you check the settings for php-apcu? Or did disabling apc solve the issue?

 

@Taleman no to both, as I did not change anything of php-apcu. I do not thin the update will have changed someting
@jesse I will try and report
on the other side what I do not understand where/why php is involed at all, the one and only yet active website is pure html. And even if I disable this one, restarting apache brings the error up.
Rainer

 

so I did apt remove libapache2-mod-php* and startet apache. apache status show apache starts fine.
enabeling my websites they run fine the on with pur html as the one useing a lot of PHP in case ist a TYPO3 site.
Tried to reinstall
apt install libapache2-mod-php*
Reading package lists... Done
Building dependency tree
Reading state information... Done
Note, selecting 'libapache2-mod-php' for glob 'libapache2-mod-php*'
Note, selecting 'libapache2-mod-php5' for glob 'libapache2-mod-php*'
Note, selecting 'libapache2-mod-php7.3' for glob 'libapache2-mod-php*'
Note, selecting 'libapache2-mod-php7.4' for glob 'libapache2-mod-php*'
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:

The following packages have unmet dependencies:
php-common : Breaks: php7.3-common but 7.3.14-1~deb10u1 is to be installed
E: Unable to correct problems, you have held broken packages.
Got the error above. Seams that has something to do with php 7.3 and 7.4 what is actually used. Seams mod-php not really needed. Why?
I any case than your for help.
Rainer

 

What shows

Code:

# dpkg --get-selections | grep 'hold$'

I notice the first thing I asked from you was to show what happens when

Code:

apt-get upgrade

is run to see if there is still something to upgrade.

 

There was nothing more to upgrade, as I know

 

Hello,
I allow myself to come back to the subject, following an update of my apache2 server stops regularly and after a restart the vhosts remain inaccessible.
Description: Debian GNU / Linux 10 (buster)
Ispconfig: ISPConfig Version: 3.2.2 with several versions of PHP (System> Additional PHP Versions.
Apache2 logs:

Code:

[ 2021-07-16 06:14:03.8653 26112/7f5b08d53700 age/Cor/CoreMain.cpp:532 ]: Signal received. Gracefully shutting down... (send signal 2 more time(s) to force shutdown)
[Fri Jul 16 06:14:03.866195 2021] [mpm_prefork:notice] [pid 26100] AH00169: caught SIGTERM, shutting down
[ 2021-07-16 06:14:03.8715 26112/7f5b09720980 age/Cor/CoreMain.cpp:901 ]: Received command to shutdown gracefully. Waiting until all clients have disconnected...
[ 2021-07-16 06:14:03.8719 26119/7fb266406700 age/Ust/UstRouterMain.cpp:422 ]: Signal received. Gracefully shutting down... (send signal 2 more time(s) to force shutdown)
[ 2021-07-16 06:14:03.8720 26119/7fb266505980 age/Ust/UstRouterMain.cpp:492 ]: Received command to shutdown gracefully. Waiting until all clients have disconnected...
[ 2021-07-16 06:14:03.8720 26119/7fb266406700 Ser/Server.h:464 ]: [UstRouter] Shutdown finished
[ 2021-07-16 06:14:03.8731 26119/7fb265b84700 Ser/Server.h:464 ]: [UstRouterApiServer] Shutdown finished
[ 2021-07-16 06:14:03.8737 26112/7f5b08d53700 Ser/Server.h:464 ]: [ServerThr.1] Shutdown finished
[ 2021-07-16 06:14:03.8737 26112/7f5af3fff700 Ser/Server.h:464 ]: [ServerThr.2] Shutdown finished
[ 2021-07-16 06:14:03.8738 26112/7f5af37fe700 Ser/Server.h:464 ]: [ServerThr.3] Shutdown finished
[ 2021-07-16 06:14:03.8739 26112/7f5af2ffd700 Ser/Server.h:464 ]: [ServerThr.4] Shutdown finished
[ 2021-07-16 06:14:03.8741 26112/7f5af27fc700 Ser/Server.h:464 ]: [ApiServer] Shutdown finished
[ 2021-07-16 06:14:03.8747 26119/7fb266505980 age/Ust/UstRouterMain.cpp:523 ]: Passenger UstRouter shutdown finished
[ 2021-07-16 06:14:03.8749 26112/7f5b08d53700 age/Cor/CoreMain.cpp:532 ]: Signal reçu. Fermer gracieusement... (envoyer le signal 1 fois de plus pour forcer l’arrêt)
[Fri juil 16 06:14:03.895301 2021] [fcgid:info] [pid 26105] mod_fcgid: processus /var/www/php-fcgi-scripts/ispconfig/.php-fcgi-starter(26159) exit(arrêt), terminé en appelant exit(), code de retour: 0

Code:

 [ 2021-07-16 06:14:03.9425 26112/7f5b09720980 age/Cor/CoreMain.cpp:967 ]: Arrêt du noyau passager terminé
[ 2021-07-16 06:16:07.3954 17129/7f50dd469700 age/Ust/UstRouterMain.cpp:422 ]: Signal reçu. Fermer gracieusement... (envoyer le signal 2 plus de temps pour forcer l’arrêt)
[ 2021-07-16 06:16:07.3954 17121/7f233ea21700 age/Cor/CoreMain.cpp:532 ]: Signal reçu. Fermer gracieusement... (envoyer le signal 2 plus de temps pour forcer l’arrêt)
[ 2021-07-16 06:16:07.3955 17121/7f233f3ee980 age/Cor/CoreMain.cpp:901 ]: Reçu la commande d’arrêter gracieusement. En attendant que tous les clients se sont déconnectés...
[ 2021-07-16 06:16:07.3955 17129/7f50dd568980 age/Ust/UstRouterMain.cpp:492 ]: Reçu la commande d’arrêter gracieusement. En attendant que tous les clients se sont déconnectés...
[ 2021-07-16 06:16:07.3957 17129/7f50dd469700 Ser/Server.h:464 ]: [UstRouter] Arrêt terminé
[ 2021-07-16 06:16:07.3957 17129/7f50dcbe7700 Ser/Server.h:464 ]: [UstRouterApiServer] Arrêt terminé
[ 2021-07-16 06:16:07.3959 17121/7f233e19f700 Ser/Server.h:464 ]: [ServerThr.2] Arrêt terminé
[ 2021-07-16 06:16:07.3959 17121/7f233d09b700 Ser/Server.h:464 ]: [ServerThr.4] Arrêt terminé
[ 2021-07-16 06:16:07.3960 17121/7f233ea21700 Ser/Server.h:464 ]: [ServerThr.1] Arrêt terminé
[ 2021-07-16 06:16:07.3961 17121/7f233d91d700 Ser/Server.h:464 ]: [ServerThr.3] Arrêt terminé
[ 2021-07-16 06:16:07.3962 17121/7f233c819700 Ser/Server.h:464 ]: [ApiServer] Arrêt terminé
[ 2021-07-16 06:16:07.3968 17129/7f50dd568980 âge/Ust/UstRouterMain.cpp:523 ]: Arrêt de l’ustrouter passager terminé
[ 2021-07-16 06:16:07.4591 17121/7f233f3ee980 age/Cor/CoreMain.cpp:967 ]: Arrêt du noyau passager terminé

Does this have a link with the "mod_Passenger" module?
Can we deactivate the "mod_Passenger"?
Down of the apache2 server approximately every 10 minutes then. I even made a reconciliation with the cron: /usr/local/ispconfig/server/server.sh
Thank you in advance for your answers and especially for the work accomplished.
Crémos

 

Update of what?
https://www.howtoforge.com/community/threads/please-read-before-posting.58408/

 

Thank you for your responsiveness, update upgrade (PHP, Mariadb, libapache2-mod, Bind9 ....).
Last down of the apache2 server:
panel3.in.ac-amiens.fr - 16.07.2021-06:16 - WARNING - httpd is down! Try rescue httpd (try:1)...
Crémos

 

Can you upgrade ISPconfig? Then allow it to reconfigure services.
If you already upgraded ISPConfig, try

Code:

ispconfig_update.sh --force 

and allow reconfiguring services.