Hi, I hope somebody can help me to solve a problem I've been struggling with for a while now. The problem I am having is random very high server load. This seems to usually happen during the day several times per hour, and tends to get worse as the day goes by. All my websites start to get noticeably slower and slower, in ISP config server load it jumps from 10, to 20, 30 and sometimes reaches over 90, at this point it is at a standstill. The server seems to crush and the only solution is to stop apache, wait at least 10 minutes and restart it, than things get back to normal for a while and than it starts again, and than suddenly just stops spiking up and is normal for a long period. I did some research to check MySQL long query logs, etc but that is not the case, there is no long queries running. Looking at server logs I noticed an error mod_fcgid: can't apply process slot for. Based on research I did, the solution is to adjust mod_fcgid settings. Based o the info available I should do that under ISPCONFIG under Sites >> Mysite >> Options >> Apache Directives. So in there I put this in there, There wasn't enought time for me to see if this makes a differance, because as of now the server is running fine. But i know this issue will come back: FcgidMaxRequestsPerProcess 500 FcgidMaxProcessCount 3 FcgidMaxRequestLen 33554432 FcgidIPCCommTimeout 60 FcgidIdleTimeout 120 FcgidIdleScanInterval 30 FcgidBusyTimeout 120 FcgidBusyScanInterval 90 FcgidErrorScanInterval 3 FcgidZombieScanInterval 3 FcgidProcessLifeTime 3600 This is some info about what I'm running: Hardware: Intel Core i3 2GZ, 8GB Ram OS: CentOS 6.4 64bit ISPCONFIG: 3.0.5.3 Apache Domains: Currently hosting 6 websites Traffic: about 3000/day nothing very significant Can somebody point me in the right directions so I can finally solve this problem? Thanks
With your traffic, the default settings should work. On my servers I did not had to change them yet, but it might be that you have to adjust them on your server for some sites depending on their needs and resource usage. What you might try is to switch the sites from php-fcgi to php-fpm if you have that installed. php-fpm is a newer form of the fcgi interface and is a bit more efficient then fcgi as far as I know.
I am still having this problem despite trying to adjust the FCGI settings. It just doesn't work. This server overload happens at least once or twice per day, the server just gets overloaded jumping to over 120. Weird thing is that my website is not getting that many visits. It is around 3000 per day which is not that much, I don't see a reason why there would be such a problem for it to handle, all I'm running is a wordpress website. In my error logs when this happens I see errors: mod_fcgid: can't apply process slot for .... with reference to multiple files on my wordpress site. In ISPCONFIG i put settings to adjust FCGID, this is what I have in Apache Directives in the settings of my website: FcgidMaxRequestLen 33554432 FcgidIdleTimeout 3600 FcgidProcessLifeTime 7200 FcgidMaxProcessCount 1000 FcgidDefaultMinClassProcessCount 3 FcgidDefaultMaxClassProcessCount 1000 FcgidIPCConnectTimeout 8 FcgidIPCCommTimeout 600 FcgidBusyTimeout 300 I keep adjusting these settings but this overload is still happening. I am pulling my hair out trying to figure out the problem I don't know what else to do. Can somebody please help me.
Next time is loads up, can you get a snapshot of 'top' with all cpu's showing and an 'iotop' snapshot as well? I'm not an expert, but for your own reference check ps -aux when it happen to make sure you don't have a rogue script/bot sitting in /tmp or something that launches when it wants. (just tips that I checked when had a similar problem)...but mine was a VM and windows was the issue.
If you use suexec in all sites, you can see with "top" the users that run the fcgi processes. When you know the username, you can find the corresponding website and there you shuld lookinto the access and error.log of the site. Such temporary high loads are often caused by bots that spider your website too fast or by script kids that try a dos. From the log, you can see the IP addresses and block access to your site. Another helpful tool to automate this is the apache module mod_evasive. It can temporarily block requests if someone tries to load / reload pages from your sites too fast.
I only have 3 websites on this server and I know exactly which one is causing the problem as it has the most visits and in the error logs. The error: mod_fcgid: can't apply process slot for points to this site when viewing it's error logs, and this error is generated at roughly the same time the overload happens. This problem accoured today as well and I am attaching a screen shot of the top command during when this is happening. As far as mod_evasive, can I just install it using yum? Is there anything else I need to do it to make it work with ISPCONFIG?
Here is my my.cnf [root@svr1 ~]# vi /etc/my.cnf [mysqld] datadir=/var/lib/mysql socket=/var/lib/mysql/mysql.sock user=mysql # Disabling symbolic-links is recommended to prevent assorted security risks symbolic-links=0 [mysqld_safe] log-error=/var/log/mysqld.log pid-file=/var/run/mysqld/mysqld.pid The site is a Wordpress site
sounds like a resource issue to me. You probably need a lot more tuning in your my.cnf. I'm sure others have posted some examples here or google wordpress my.cnf .
