Hi, since a few months already my server seems to run out of memory sometimes "randomly".. All of a sudden my website is not reachable anymore, my Mail server stops delivering mails etc. I can still ssh into the machine and a reboot seems the only way I found so far to get everything back working. I think it is memory related, because if I try to restart Apache followed by MySQL, MySQL gives some error about not having enough memory available. My Website has a couple million visitors per Month and I don't see any pattern of "Oh, it always happens when X happens", sometimes it happens at rush-hour, sometimes in the middle of the night. I am running Debian 7.7 and Apache 2.2.22 using mod_wsgi - but with no big framework (like Django etc) around it - on a Digital Ocean Droplet with 4gb memory. My Apache runs in prefork mode with these settings: Code: <IfModule mpm_prefork_module> StartServers 5 MinSpareServers 5 MaxSpareServers 10 MaxClients 200 MaxRequestsPerChild 2000 </IfModule> And it appears, everytime the issue happens this line shows up in my error.log file from Apache: Code: [error] server reached MaxClients setting, consider raising the MaxClients setting Could it be that some of my apache processes are crashed/hanging and so Apache starts spawning new ones until all instances are hanging sooner or later? I assume there's one thing that causes them to hang .. but, I have no idea how to find out what it is Or are my Apache settings totally wrong? In the attached screenshot you can see my htop-output ... there's tons of apache processes, so I didn't include them all. Any tips what I could do? How can I find out what the issue is? I am totally clueless at this point Cheers, Kira PS: As a temporary work around I'd also be happy with an automatic "reboot if this happens".. is there any neat way of doing this?
you could try Use Apache MPM_worker Removed unnecessary Apache modules. Turn KeepAlive Off Consider enabling WSGIRestrictEmbedded Use daemon mode of mod_wsgi Configure the mpm_worker module. Configure mod_info and mod_status any other stuff you use? php > replace with mod_fastcgi + php-fpm How many requests/s your apache needs to handle? looks like way too much apache processes check your mysql-configuration and do the math wether you overconfigured memory maybe aswell ( not that it shouldn't start anyway, just to be sure ) same for apache. Especially MPM_prefork hogs a lot of memory even when serving just a static page. However ISPConfig is not designed ( yet ) for mpm_prefork though it works except you may need to hack the configuration for phpmyadmin/webmail/whatever you use and you may not use ruby or other mod_ packages as easy.
Hi, thanks a lot for your reply! I have KeepAlive turned off already and am using daemon mode of mod_wsgi (also have WSGIRestrictEmbedded turned off to be sure).. Over the past few days since the last reboot I checked my server regulary to see if anything out of the ordinary happens .. and for the whole time, when I did htop, there were only like 20 or so apache2 processes.. and just now, 5min ago it all crashed again, and when checking htop again, there were again tons and tons of apache instances and I got that message about having reached MaxClients in the error.log ... The only thing I haven't tried yet is to use MPM_worker since that'd be a lot more work to setup I guess.. and I am not sure it'd solve my issue? The memory shortage and "not enough workers" seems to be just a symptom, not the cause .. I think something is making my Apache instances / requests hang sometimes .. and everytime one hangs, a new one is spawned up until at some point the limit is reached and then . boom Unfortunately I have no clue how to find out WHAT is causing that issue .. could I find out what each apache process is doing or something like that? Cheers, Kira
I can recommend you to try migrate your config to the php-fpm. I should decrease both server load and page load time, and not hard to do too.
