Recently, all sites on my ISPConfig 3 stop serving multiple times a day. A full reboot and/or Apache 2 restart brings back functionality. I noticed an 'out of space error' in the log files (below) but df -h shows 32GB of 200GB available. ISPConfig 3 Ubuntu 16.04 LTS 16GB RAM 4 CPU THX in ADV! Code: [Tue Jun 18 19:21:57.158806 2019] [ssl:error] [pid 1601:tid 139970736625536] AH02604: Unable to configure certificate cloud001-webserver.cloud.mysite.com:8080:0 for stapling [Tue Jun 18 19:21:57.302491 2019] [:error] [pid 1601:tid 139970736625536] python_init: Python version mismatch, expected '2.7.6', found '2.7.12'. [Tue Jun 18 19:21:57.306418 2019] [:error] [pid 1601:tid 139970736625536] python_init: Python executable found '/usr/bin/python'. [Tue Jun 18 19:21:57.306448 2019] [:error] [pid 1601:tid 139970736625536] python_init: Python path being used '/usr/lib/python2.7/:/usr/lib/python2.7/plat-x86_64-linux-gnu:/usr/lib/python2.7/lib-tk:/usr/lib/python2.7/lib-old:/usr/lib/python2.7/lib-dynload'. [Tue Jun 18 19:21:57.306568 2019] [:notice] [pid 1601:tid 139970736625536] mod_python: Creating 8 session mutexes based on 6 max processes and 25 max threads. [Tue Jun 18 19:21:57.306589 2019] [:notice] [pid 1601:tid 139970736625536] mod_python: using mutex_directory /tmp [Tue Jun 18 19:21:57.333738 2019] [mpm_event:notice] [pid 1601:tid 139970736625536] AH00489: Apache/2.4.37 (Ubuntu) mod_fastcgi/mod_fastcgi-SNAP-0910052141 mod_fcgid/2.3.9 OpenSSL/1.1.1 mod_python/3.3.1 Python/2.7.12 configured -- resuming normal operations [Tue Jun 18 19:21:57.333774 2019] [core:notice] [pid 1601:tid 139970736625536] AH00094: Command line: '/usr/sbin/apache2' PHP Warning: PHP Startup: Unable to load dynamic library '/usr/lib/php/20151012/php_ldap.dll' - /usr/lib/php/20151012/php_ldap.dll: cannot open shared object file: No such file or directory in Unknown on line 0 PHP Warning: Module 'redis' already loaded in Unknown on line 0 [Tue Jun 18 19:48:43.766659 2019] [pagespeed:warn] [pid 1623:tid 139969695840000] [mod_pagespeed 1.13.35.2-0 @1623] Fetch timed out: https://mysite.com/news/draas-in-under-a-week/#post-3351 (connecting to:192.168.55.32) (1) waiting for 50 ms [Tue Jun 18 20:08:41.543060 2019] [pagespeed:warn] [pid 1624:tid 139968194266880] [mod_pagespeed 1.13.35.2-0 @1624] Fetch timed out: https://mysite.com/author/john/#post-3351 (connecting to:192.168.55.32) (1) waiting for 50 ms [Tue Jun 18 20:08:42.247866 2019] [core:notice] [pid 1601:tid 139970736625536] AH00051: child pid 1623 exit signal Segmentation fault (11), possible coredump in /etc/apache2 [Tue Jun 18 20:08:46.570756 2019] [pagespeed:warn] [pid 1624:tid 139968194266880] [mod_pagespeed 1.13.35.2-0 @1624] Fetch timed out: https://mysite.com/author/john/#post-3254 (connecting to:192.168.55.32) (1) waiting for 50 ms [Tue Jun 18 20:08:51.590039 2019] [pagespeed:warn] [pid 1624:tid 139968194266880] [mod_pagespeed 1.13.35.2-0 @1624] Fetch timed out: https://mysite.com/author/john/#post-3190 (connecting to:192.168.55.32) (1) waiting for 50 ms [Tue Jun 18 20:08:56.609075 2019] [pagespeed:warn] [pid 1624:tid 139968194266880] [mod_pagespeed 1.13.35.2-0 @1624] Fetch timed out: https://mysite.com/author/john/#post-3309 (connecting to:192.168.55.32) (1) waiting for 50 ms [Tue Jun 18 20:09:01.630202 2019] [pagespeed:warn] [pid 1624:tid 139968194266880] [mod_pagespeed 1.13.35.2-0 @1624] Fetch timed out: https://mysite.com/author/john/#post-3351 (connecting to:192.168.55.32) (1) waiting for 50 ms [Tue Jun 18 20:09:06.652518 2019] [pagespeed:warn] [pid 1624:tid 139968194266880] [mod_pagespeed 1.13.35.2-0 @1624] Fetch timed out: https://mysite.com/author/john/#post-3190 (connecting to:192.168.55.32) (1) waiting for 50 ms [Tue Jun 18 20:09:11.674616 2019] [pagespeed:warn] [pid 1624:tid 139968194266880] [mod_pagespeed 1.13.35.2-0 @1624] Fetch timed out: https://mysite.com/author/john/#post-3309 (connecting to:192.168.55.32) (1) waiting for 50 ms [Tue Jun 18 20:09:16.694825 2019] [pagespeed:warn] [pid 1624:tid 139968194266880] [mod_pagespeed 1.13.35.2-0 @1624] Fetch timed out: https://mysite.com/author/john/#post-3254 (connecting to:192.168.55.32) (1) waiting for 50 ms [Tue Jun 18 20:09:21.714532 2019] [pagespeed:warn] [pid 1624:tid 139968194266880] [mod_pagespeed 1.13.35.2-0 @1624] Fetch timed out: https://mysite.com/news/draas-in-under-a-week/#post-3351 (connecting to:192.168.55.32) (1) waiting for 50 ms [Tue Jun 18 20:09:26.765129 2019] [pagespeed:warn] [pid 1624:tid 139968194266880] [mod_pagespeed 1.13.35.2-0 @1624] Fetch timed out: https://mysite.com/news/draas-in-under-a-week/#post-3351 (connecting to:192.168.55.32) (1) waiting for 50 ms [Tue Jun 18 20:12:35.418411 2019] [pagespeed:warn] [pid 4878:tid 139968353662720] [mod_pagespeed 1.13.35.2-0 @4878] Fetch timed out: https://mysite.com/mysite/happy-holidays-from-mysite/#post-3309 (connecting to:192.168.55.32) (1) waiting for 50 ms [Tue Jun 18 20:12:40.437690 2019] [pagespeed:warn] [pid 4878:tid 139968353662720] [mod_pagespeed 1.13.35.2-0 @4878] Fetch timed out: https://mysite.com/mysite/happy-holidays-from-mysite/#post-3309 (connecting to:192.168.55.32) (1) waiting for 50 ms [Tue Jun 18 20:15:46.319163 2019] [pagespeed:warn] [pid 4878:tid 139968353662720] [mod_pagespeed 1.13.35.2-0 @4878] Fetch timed out: https://mysite.com/mysite/draas-portal-security-announcements-vsphere-6-7/#post-3254 (connecting to:192.168.55.32) (1) waiting for 50 ms [Tue Jun 18 20:15:51.346395 2019] [pagespeed:warn] [pid 4878:tid 139968353662720] [mod_pagespeed 1.13.35.2-0 @4878] Fetch timed out: https://mysite.com/mysite/draas-portal-security-announcements-vsphere-6-7/#post-3254 (connecting to:192.168.55.32) (1) waiting for 50 ms PHP Warning: PHP Startup: Unable to load dynamic library '/usr/lib/php/20151012/php_ldap.dll' - /usr/lib/php/20151012/php_ldap.dll: cannot open shared object file: No such file or directory in Unknown on line 0 PHP Warning: Module 'redis' already loaded in Unknown on line 0 PHP Warning: PHP Startup: Unable to load dynamic library '/usr/lib/php/20151012/php_ldap.dll' - /usr/lib/php/20151012/php_ldap.dll: cannot open shared object file: No such file or directory in Unknown on line 0 PHP Warning: Module 'redis' already loaded in Unknown on line 0 PHP Warning: PHP Startup: Unable to load dynamic library '/usr/lib/php/20151012/php_ldap.dll' - /usr/lib/php/20151012/php_ldap.dll: cannot open shared object file: No such file or directory in Unknown on line 0 PHP Warning: Module 'redis' already loaded in Unknown on line 0 PHP Warning: PHP Startup: Unable to load dynamic library '/usr/lib/php/20151012/php_ldap.dll' - /usr/lib/php/20151012/php_ldap.dll: cannot open shared object file: No such file or directory in Unknown on line 0 PHP Warning: Module 'redis' already loaded in Unknown on line 0 [Tue Jun 18 21:45:14.279722 2019] [pagespeed:warn] [pid 4878:tid 139968353662720] [mod_pagespeed 1.13.35.2-0 @4878] Fetch timed out: https://mysite.com/mysite/supporting-older-vsphere-edititions/#post-2860 (connecting to:192.168.55.32) (1) waiting for 50 ms [Tue Jun 18 22:10:02.342194 2019] [pagespeed:error] [pid 4878:tid 139968353662720] [mod_pagespeed 1.13.35.2-0 @4878] PageSpeed Serf fetch failure rate extremely high; only 0 of 6 recent fetches fully successful; is fetching working? [Tue Jun 18 22:30:44.083242 2019] [pagespeed:warn] [pid 1624:tid 139968194266880] [mod_pagespeed 1.13.35.2-0 @1624] Fetch timed out: https://mysite.com/portfolio_category/disaster-recovery-as-a-service/?long_events=0&ajaxCalendar=1&mo=3&yr=2019#post-2725 (connecting to:192.168.55.32) (1) waiting for 50 ms [Tue Jun 18 22:30:45.099916 2019] [core:notice] [pid 1601:tid 139970736625536] AH00051: child pid 1624 exit signal Segmentation fault (11), possible coredump in /etc/apache2 PHP Warning: PHP Startup: Unable to load dynamic library '/usr/lib/php/20151012/php_ldap.dll' - /usr/lib/php/20151012/php_ldap.dll: cannot open shared object file: No such file or directory in Unknown on line 0 PHP Warning: Module 'redis' already loaded in Unknown on line 0 PHP Warning: PHP Startup: Unable to load dynamic library '/usr/lib/php/20151012/php_ldap.dll' - /usr/lib/php/20151012/php_ldap.dll: cannot open shared object file: No such file or directory in Unknown on line 0 PHP Warning: Module 'redis' already loaded in Unknown on line 0 PHP Warning: PHP Startup: Unable to load dynamic library '/usr/lib/php/20151012/php_ldap.dll' - /usr/lib/php/20151012/php_ldap.dll: cannot open shared object file: No such file or directory in Unknown on line 0 PHP Warning: Module 'redis' already loaded in Unknown on line 0 [Wed Jun 19 00:31:01.173402 2019] [mpm_event:error] [pid 1601:tid 139970736625536] AH00484: server reached MaxRequestWorkers setting, consider raising the MaxRequestWorkers setting PHP Warning: PHP Startup: Unable to load dynamic library '/usr/lib/php/20151012/php_ldap.dll' - /usr/lib/php/20151012/php_ldap.dll: cannot open shared object file: No such file or directory in Unknown on line 0 PHP Warning: Module 'redis' already loaded in Unknown on line 0 [Wed Jun 19 00:34:09.269282 2019] [log_config:warn] [pid 15072:tid 139970090100480] (28)No space left on device: [client 36.152.65.199:40754] AH00646: Error writing to /var/log/apache2/other_vhosts_access.log [Wed Jun 19 00:34:20.761141 2019] [pagespeed:error] [pid 10068:tid 139970319828736] [mod_pagespeed 1.13.35.2-0 @10068] /var/log/pagespeed/stats_log_global: flushing file 28(No space left on device)
Are you sure that there's enough space on all partitions? In this case, the issue occurred in /var or /var/log
Pardon my ignorance here. Is there another way to list partitions? Code: root@cloud001-webserver:~# df -h Filesystem Size Used Avail Use% Mounted on udev 7.9G 0 7.9G 0% /dev tmpfs 1.6G 8.9M 1.6G 1% /run /dev/mapper/cloud001--webserver--vg-root 193G 152G 32G 83% / tmpfs 7.9G 0 7.9G 0% /dev/shm tmpfs 5.0M 4.0K 5.0M 1% /run/lock tmpfs 7.9G 0 7.9G 0% /sys/fs/cgroup /dev/sda1 472M 213M 236M 48% /boot tmpfs 1.6G 0 1.6G 0% /run/user/1000 root@cloud001-webserver:~#
No, that's ok and there seems to be enough space, you currently have just one harddisk partition. I just wanted to make sure that you checked all partitions. Maybe the harddisk health is not ok, you should consider checking it with e.g smartmontools. Or is it a virtual server?
I don't see any problem? Code: root@cloud001-webserver:/var# df -hi Filesystem Inodes IUsed IFree IUse% Mounted on udev 2.0M 421 2.0M 1% /dev tmpfs 2.0M 722 2.0M 1% /run /dev/mapper/cloud001--webserver--vg-root 13M 290K 12M 3% / tmpfs 2.0M 1 2.0M 1% /dev/shm tmpfs 2.0M 16 2.0M 1% /run/lock tmpfs 2.0M 16 2.0M 1% /sys/fs/cgroup /dev/sda1 122K 320 122K 1% /boot tmpfs 2.0M 4 2.0M 1% /run/user/1000 root@cloud001-webserver:/var#
You can e.g. install munin and monit to monitor the various health parameters of your server and to restart apache in case of a failure.
Here is the latest after it stopped delivering sites: Code: PHP Warning: PHP Startup: Unable to load dynamic library '/usr/lib/php/20151012/php_ldap.dll' - /usr/lib/php/20151012/php_ldap.dll: cannot open shared object file: No such file or directory in Unknown on line 0 PHP Warning: Module 'redis' already loaded in Unknown on line 0 PHP Warning: PHP Startup: Unable to load dynamic library '/usr/lib/php/20151012/php_ldap.dll' - /usr/lib/php/20151012/php_ldap.dll: cannot open shared object file: No such file or directory in Unknown on line 0 PHP Warning: Module 'redis' already loaded in Unknown on line 0 [Wed Jun 19 09:07:45.543246 2019] [pagespeed:warn] [pid 10127:tid 139963370821376] [mod_pagespeed 1.13.35.2-0 @10127] Fetch timed out: https://mysite.com/mysite/supporting-older-vsphere-edititions/#post-2860 (connecting to:172.31.0.32) (1) waiting for 50 ms PHP Warning: PHP Startup: Unable to load dynamic library '/usr/lib/php/20151012/php_ldap.dll' - /usr/lib/php/20151012/php_ldap.dll: cannot open shared object file: No such file or directory in Unknown on line 0 PHP Warning: Module 'redis' already loaded in Unknown on line 0 [Wed Jun 19 09:46:04.533579 2019] [pagespeed:warn] [pid 10126:tid 139965350450944] [mod_pagespeed 1.13.35.2-0 @10126] Fetch timed out: https://mysite.com/portfolio_category/servers_san_storage_veeam_vmware/#post-2725 (connecting to:172.31.0.32) (1) waiting for 50 ms [Wed Jun 19 09:46:05.583755 2019] [core:notice] [pid 10071:tid 139966184691584] AH00051: child pid 10126 exit signal Segmentation fault (11), possible coredump in /etc/apache2 PHP Warning: PHP Startup: Unable to load dynamic library '/usr/lib/php/20151012/php_ldap.dll' - /usr/lib/php/20151012/php_ldap.dll: cannot open shared object file: No such file or directory in Unknown on line 0 PHP Warning: Module 'redis' already loaded in Unknown on line 0 PHP Warning: PHP Startup: Unable to load dynamic library '/usr/lib/php/20151012/php_ldap.dll' - /usr/lib/php/20151012/php_ldap.dll: cannot open shared object file: No such file or directory in Unknown on line 0 PHP Warning: PHP Startup: Unable to load dynamic library '/usr/lib/php/20151012/php_ldap.dll' - /usr/lib/php/20151012/php_ldap.dll: cannot open shared object file: No such file or directory in Unknown on line 0 PHP Warning: PHP Startup: Unable to load dynamic library '/usr/lib/php/20151012/php_ldap.dll' - /usr/lib/php/20151012/php_ldap.dll: cannot open shared object file: No such file or directory in Unknown on line 0 PHP Warning: Module 'redis' already loaded in Unknown on line 0 PHP Warning: PHP Startup: Unable to load dynamic library '/usr/lib/php/20151012/php_ldap.dll' - /usr/lib/php/20151012/php_ldap.dll: cannot open shared object file: No such file or directory in Unknown on line 0 PHP Warning: Module 'redis' already loaded in Unknown on line 0 PHP Warning: Module 'redis' already loaded in Unknown on line 0 PHP Warning: Module 'redis' already loaded in Unknown on line 0 [Wed Jun 19 10:30:13.241924 2019] [pagespeed:warn] [pid 13404:tid 139965725894400] [mod_pagespeed 1.13.35.2-0 @13404] Fetch timed out for https://mysite.com/wp-content/themes/theme47929/images/xarrows-ffffff.png.pagespeed.ic.-Kq5Tztjjv.webp [Wed Jun 19 10:30:13.242077 2019] [pagespeed:warn] [pid 13404:tid 139965725894400] [mod_pagespeed 1.13.35.2-0 @13404] https://mysite.com/wp-content/themes/theme47929/images/xarrows-ffffff.png.pagespeed.ic.-Kq5Tztjjv.webp resource_404_count: not found (404) PHP Warning: PHP Startup: Unable to load dynamic library '/usr/lib/php/20151012/php_ldap.dll' - /usr/lib/php/20151012/php_ldap.dll: cannot open shared object file: No such file or directory in Unknown on line 0 PHP Warning: Module 'redis' already loaded in Unknown on line 0 PHP Warning: PHP Startup: Unable to load dynamic library '/usr/lib/php/20151012/php_ldap.dll' - /usr/lib/php/20151012/php_ldap.dll: cannot open shared object file: No such file or directory in Unknown on line 0 PHP Warning: Module 'redis' already loaded in Unknown on line 0 [Wed Jun 19 11:47:57.444647 2019] [mpm_event:error] [pid 10071:tid 139966184691584] AH10159: server is within MinSpareThreads of MaxRequestWorkers, consider raising the MaxRequestWorkers setting
The last error message suggests what to do: 0159: server is within MinSpareThreads of MaxRequestWorkers, consider raising the MaxRequestWorkers setting and you should chek where the ldap php module is loaded and either remove it from the config or add the missing dll.
Again, pardon the ignorance; is this one of these: Code: /etc/apache2/mods-available/mpm_prefork.conf:# MaxRequestWorkers: maximum number of server processes allowed to start /etc/apache2/mods-available/mpm_prefork.conf: MaxRequestWorkers 150 /etc/apache2/mods-available/mpm_event.conf:# MaxRequestWorkers: maximum number of worker threads /etc/apache2/mods-available/mpm_event.conf: MaxRequestWorkers 150 /etc/apache2/mods-available/mpm_worker.conf:# MaxRequestWorkers: maximum number of threads /etc/apache2/mods-available/mpm_worker.conf: MaxRequestWorkers 150 I have changed all to 300 and restarted apache2 THX
according to your error message, you are using mpm_event, so the relevant should be this: /etc/apache2/mods-available/mpm_event.conf
Improvements but still having trouble: Code: [Thu Jun 20 07:22:08.635998 2019] [core:error] [pid 5575:tid 140128372320128] AH00046: child process 25078 still did not exit, sending a SIGKILL [Thu Jun 20 07:22:08.640980 2019] [core:error] [pid 5575:tid 140128372320128] AH00046: child process 26587 still did not exit, sending a SIGKILL [Thu Jun 20 07:22:08.648702 2019] [core:error] [pid 5575:tid 140128372320128] AH00046: child process 30481 still did not exit, sending a SIGKILL [Thu Jun 20 07:22:09.653238 2019] [mpm_event:notice] [pid 5575:tid 140128372320128] AH00491: caught SIGTERM, shutting down [Thu Jun 20 07:22:10.175447 2019] [ssl:warn] [pid 13862:tid 140171226650496] AH01906: cloud001-webserver.mysite.com:8080:0 server certificate is a CA certificate (BasicConstraints: CA == TRUE !?) [Thu Jun 20 07:22:10.175641 2019] [ssl:error] [pid 13862:tid 140171226650496] AH02217: ssl_stapling_init_cert: can't retrieve issuer certificate! [subject: [email protected],CN=cloud001-webserver.mysite.com,OU=webserver,O=mysite,L=Flagstaff,ST=Arizona,C=US / issuer: [email protected],CN=cloud001-webserver.mysite.com,OU=webserver,O=mysite,L=Flagstaff,ST=Arizona,C=US / serial: B287A6A0E2C60B0F / notbefore: Aug 2 15:31:14 2017 GMT / notafter: Jul 31 15:31:14 2027 GMT] [Thu Jun 20 07:22:10.175652 2019] [ssl:error] [pid 13862:tid 140171226650496] AH02604: Unable to configure certificate cloud001-webserver.mysite.com:8080:0 for stapling [Thu Jun 20 07:22:10.289805 2019] [suexec:notice] [pid 13862:tid 140171226650496] AH01232: suEXEC mechanism enabled (wrapper: /usr/lib/apache2/suexec) [Thu Jun 20 07:22:10.332561 2019] [:notice] [pid 13866:tid 140171226650496] FastCGI: process manager initialized (pid 13866) [Thu Jun 20 07:22:10.335800 2019] [ssl:warn] [pid 13864:tid 140171226650496] AH01906: cloud001-webserver.mysite.com:8080:0 server certificate is a CA certificate (BasicConstraints: CA == TRUE !?) [Thu Jun 20 07:22:10.335988 2019] [ssl:error] [pid 13864:tid 140171226650496] AH02217: ssl_stapling_init_cert: can't retrieve issuer certificate! [subject: [email protected],CN=cloud001-webserver.mysite.com,OU=webserver,O=mysite,L=Flagstaff,ST=Arizona,C=US / issuer: [email protected],CN=cloud001-webserver.mysite.com,OU=webserver,O=mysite,L=Flagstaff,ST=Arizona,C=US / serial: B287A6A0E2C60B0F / notbefore: Aug 2 15:31:14 2017 GMT / notafter: Jul 31 15:31:14 2027 GMT] [Thu Jun 20 07:22:10.336007 2019] [ssl:error] [pid 13864:tid 140171226650496] AH02604: Unable to configure certificate cloud001-webserver.mysite.com:8080:0 for stapling [Thu Jun 20 07:22:10.455946 2019] [:error] [pid 13864:tid 140171226650496] python_init: Python version mismatch, expected '2.7.6', found '2.7.12'. [Thu Jun 20 07:22:10.456111 2019] [:error] [pid 13864:tid 140171226650496] python_init: Python executable found '/usr/bin/python'. [Thu Jun 20 07:22:10.456123 2019] [:error] [pid 13864:tid 140171226650496] python_init: Python path being used '/usr/lib/python2.7/:/usr/lib/python2.7/plat-x86_64-linux-gnu:/usr/lib/python2.7/lib-tk:/usr/lib/python2.7/lib-old:/usr/lib/python2.7/lib-dynload'. [Thu Jun 20 07:22:10.456162 2019] [:notice] [pid 13864:tid 140171226650496] mod_python: Creating 8 session mutexes based on 12 max processes and 25 max threads. [Thu Jun 20 07:22:10.456170 2019] [:notice] [pid 13864:tid 140171226650496] mod_python: using mutex_directory /tmp [Thu Jun 20 07:22:10.475844 2019] [mpm_event:notice] [pid 13864:tid 140171226650496] AH00489: Apache/2.4.39 (Ubuntu) mod_fastcgi/mod_fastcgi-SNAP-0910052141 mod_fcgid/2.3.9 OpenSSL/1.1.1c mod_python/3.3.1 Python/2.7.12 configured -- resuming normal operations [Thu Jun 20 07:22:10.475915 2019] [core:notice] [pid 13864:tid 140171226650496] AH00094: Command line: '/usr/sbin/apache2' PHP Warning: PHP Startup: Unable to load dynamic library '/usr/lib/php/20151012/php_ldap.dll' - /usr/lib/php/20151012/php_ldap.dll: cannot open shared object file: No such file or directory in Unknown on line 0 PHP Warning: Module 'redis' already loaded in Unknown on line 0 PHP Warning: PHP Startup: Unable to load dynamic library '/usr/lib/php/20151012/php_ldap.dll' - /usr/lib/php/20151012/php_ldap.dll: cannot open shared object file: No such file or directory in Unknown on line 0 PHP Warning: Module 'redis' already loaded in Unknown on line 0 [Thu Jun 20 07:33:37.494836 2019] [pagespeed:warn] [pid 13868:tid 140170367706880] [mod_pagespeed 1.13.35.2-0 @13868] Fetch timed out: https://mysite.com/author/john/#post-3351 (connecting to:172.31.0.32) (1) waiting for 50 ms [Thu Jun 20 07:33:38.228844 2019] [core:notice] [pid 13864:tid 140171226650496] AH00051: child pid 13868 exit signal Segmentation fault (11), possible coredump in /etc/apache2
You should fix the php_ldap.dll thing that I mentioned above. And regarding the segfault, that's not easy to diagnose, this can be anything from a bug in apache or one of its library to hardware problems with RAM or the motherboard. And you use the apache mpm_event which we don't use in default setups. It's faster than the default mpm_prefork that we normally use but might be less stable, so it might be worth a try using the default ispconfig setup with mpm_prefork instead of your custom setup.
OK, do php_ldap.dll is not loaded in any php.ini (that I could grep in /etc), that''s why I was going nuts! I found that it is loaded by ISPConfig in a site > Options > custom php.ini settings: Code: memory_limit = 2048M upload_max_filesize = 10240M post_max_size = 20480M extension = redis.so extension = php_ldap.dll date.timezone = "America/Phoenix" on the server, the file: php_ldap.dll does not exist, so I have removed the line from the site config.
probably someone copied it from a windows system as those files end with .so normally on Linux, .dll is a windows file extension. so removing it is a good choice as it did not do anything useful anyway.
Unfortunately, sites are still crashing. Could something with pagespeed have changed? Code: [Thu Jun 20 09:50:02.212633 2019] [pagespeed:error] [pid 21427:tid 139849612904192] [mod_pagespeed 1.13.35.2-0 @21427] PageSpeed Serf fetch failure rate extremely high; only 0 of 5 recent fetches fully successful; is fetching working? PHP Warning: Module 'redis' already loaded in Unknown on line 0 PHP Warning: Module 'redis' already loaded in Unknown on line 0 [Thu Jun 20 10:09:38.659402 2019] [pagespeed:warn] [pid 21426:tid 139849487079168] [mod_pagespeed 1.13.35.2-0 @21426] Fetch timed out: https://mysite.com/portfolio_tag/veeam/#post-2725 (connecting to:172.31.0.32) (1) waiting for 50 ms [Thu Jun 20 10:09:43.681267 2019] [pagespeed:warn] [pid 21426:tid 139849487079168] [mod_pagespeed 1.13.35.2-0 @21426] Fetch timed out: https://mysite.com/portfolio_tag/veeam/#post-2547 (connecting to:172.31.0.32) (1) waiting for 50 ms [Thu Jun 20 10:09:48.707155 2019] [pagespeed:warn] [pid 21426:tid 139849487079168] [mod_pagespeed 1.13.35.2-0 @21426] Fetch timed out: https://mysite.com/portfolio_tag/veeam/#post-2546 (connecting to:172.31.0.32) (1) waiting for 50 ms [Thu Jun 20 10:09:53.730858 2019] [pagespeed:warn] [pid 21426:tid 139849487079168] [mod_pagespeed 1.13.35.2-0 @21426] Fetch timed out: https://mysite.com/portfolio_tag/veeam/#post-2725 (connecting to:172.31.0.32) (1) waiting for 50 ms [Thu Jun 20 10:09:58.751563 2019] [pagespeed:warn] [pid 21426:tid 139849487079168] [mod_pagespeed 1.13.35.2-0 @21426] Fetch timed out: https://mysite.com/portfolio_tag/veeam/#post-2546 (connecting to:172.31.0.32) (1) waiting for 50 ms [Thu Jun 20 10:10:03.770489 2019] [pagespeed:warn] [pid 21426:tid 139849487079168] [mod_pagespeed 1.13.35.2-0 @21426] Fetch timed out: https://mysite.com/portfolio_tag/veeam/#post-2547 (connecting to:172.31.0.32) (1) waiting for 50 ms [Thu Jun 20 10:25:02.309931 2019] [pagespeed:error] [pid 21427:tid 139849612904192] [mod_pagespeed 1.13.35.2-0 @21427] PageSpeed Serf fetch failure rate extremely high; only 0 of 10 recent fetches fully successful; is fetching working? [Thu Jun 20 10:30:43.741516 2019] [pagespeed:warn] [pid 21426:tid 139849487079168] [mod_pagespeed 1.13.35.2-0 @21426] Fetch timed out: https://mysite.com/mysite/foreshadow/#post-2636 (connecting to:172.31.0.32) (1) waiting for 50 ms [Thu Jun 20 10:30:44.245725 2019] [core:notice] [pid 21422:tid 139852157757312] AH00051: child pid 21426 exit signal Segmentation fault (11), possible coredump in /etc/apache2 PHP Warning: Module 'redis' already loaded in Unknown on line 0 PHP Warning: Module 'redis' already loaded in Unknown on line 0 [Thu Jun 20 11:21:39.877169 2019] [pagespeed:warn] [pid 21427:tid 139851391239936] [mod_pagespeed 1.13.35.2-0 @21427] Fetch timed out for https://mysite.com/wp-content/plugins/events-manager/includes/css/A.events_manager.css,qver=5.95.pagespeed.cf.qRIVf-Ntc1.css [Thu Jun 20 11:21:39.877306 2019] [pagespeed:warn] [pid 21427:tid 139851391239936] [mod_pagespeed 1.13.35.2-0 @21427] https://mysite.com/wp-content/plugins/events-manager/includes/css/A.events_manager.css,qver=5.95.pagespeed.cf.qRIVf-Ntc1.css resource_404_count: not found (404) [Thu Jun 20 11:21:40.636684 2019] [pagespeed:warn] [pid 11460:tid 139851508737792] [mod_pagespeed 1.13.35.2-0 @11460] Fetch timed out for https://mysite.com/wp-content/uploads/2018/08/xVMware-Footer-1024x73.png.pagespeed.ic.aN4w9xdrsS.webp [Thu Jun 20 11:21:40.636816 2019] [pagespeed:warn] [pid 11460:tid 139851508737792] [mod_pagespeed 1.13.35.2-0 @11460] https://mysite.com/wp-content/uploads/2018/08/xVMware-Footer-1024x73.png.pagespeed.ic.aN4w9xdrsS.webp resource_404_count: not found (404) PHP Warning: Module 'redis' already loaded in Unknown on line 0 PHP Warning: Module 'redis' already loaded in Unknown on line 0 PHP Warning: Module 'redis' already loaded in Unknown on line 0 PHP Warning: Module 'redis' already loaded in Unknown on line 0 [Thu Jun 20 11:31:06.809196 2019] [pagespeed:warn] [pid 18007:tid 139850040735488] [mod_pagespeed 1.13.35.2-0 @18007] Fetch timed out: https://mysite.com/blog/#post-3351 (connecting to:172.31.0.32) (1) waiting for 50 ms [Thu Jun 20 11:31:11.828949 2019] [pagespeed:warn] [pid 18007:tid 139850040735488] [mod_pagespeed 1.13.35.2-0 @18007] Fetch timed out: https://mysite.com/blog/#post-3190 (connecting to:172.31.0.32) (1) waiting for 50 ms [Thu Jun 20 11:31:16.848721 2019] [pagespeed:warn] [pid 18007:tid 139850040735488] [mod_pagespeed 1.13.35.2-0 @18007] Fetch timed out: https://mysite.com/blog/#post-3309 (connecting to:172.31.0.32) (1) waiting for 50 ms [Thu Jun 20 11:31:21.871012 2019] [pagespeed:warn] [pid 18007:tid 139850040735488] [mod_pagespeed 1.13.35.2-0 @18007] Fetch timed out: https://mysite.com/blog/#post-3254 (connecting to:172.31.0.32) (1) waiting for 50 ms [Thu Jun 20 11:31:26.891669 2019] [pagespeed:warn] [pid 18007:tid 139850040735488] [mod_pagespeed 1.13.35.2-0 @18007] Fetch timed out: https://mysite.com/blog/page/2/#post-2860 (connecting to:172.31.0.32) (1) waiting for 50 ms [Thu Jun 20 11:31:31.911022 2019] [pagespeed:warn] [pid 18007:tid 139850040735488] [mod_pagespeed 1.13.35.2-0 @18007] Fetch timed out: https://mysite.com/blog/page/2/#post-2636 (connecting to:172.31.0.32) (1) waiting for 50 ms [Thu Jun 20 11:31:36.930615 2019] [pagespeed:warn] [pid 18007:tid 139850040735488] [mod_pagespeed 1.13.35.2-0 @18007] Fetch timed out: https://mysite.com/blog/page/2/#post-77 (connecting to:172.31.0.32) (1) waiting for 50 ms [Thu Jun 20 11:31:41.951192 2019] [pagespeed:warn] [pid 18007:tid 139850040735488] [mod_pagespeed 1.13.35.2-0 @18007] Fetch timed out: https://mysite.com/blog/page/2/#post-71 (connecting to:172.31.0.32) (1) waiting for 50 ms [Thu Jun 20 12:05:02.217608 2019] [pagespeed:error] [pid 25411:tid 139849503864576] [mod_pagespeed 1.13.35.2-0 @25411] PageSpeed Serf fetch failure rate extremely high; only 0 of 10 recent fetches fully successful; is fetching working?
