Apache hangs without traffic

Discussion in 'General' started by Thesee, Jun 6, 2011.

  1. Thesee

    Thesee New Member

    Hi, some months ago i installed my server following the great tutorial The Perfect Server - CentOS IspConfig 3.

    Randomly apache hangs without reason, without traffic. The first few months I have not given importance to this issue, due the fact I was using the server mainly for develop and test some open source project.
    Now the server has some site on production, a very very low traffic, and apache hangs without reason, random.

    Before installing ispconfig 3 this rack was serving sites for about 15k unique visitor/day without problem. After setup with ispconfig i notice this strange behavior.

    When apache hangs the load grow up very fast, the error log write
    Code:
    server reached MaxClients setting, consider raising the MaxClients setting
    but i setted every value, from 10 to 256, i tried to setup serverlimits equal to maxclients but the problem does not disappear.

    I registered about 150 vhosts, but only 10 are in production, the other are not used.

    I'm running a sh script that check the load and then restart apache, before reload i save the server-status but nothing strange here, many times there are only 5 / 6 requests ...and some OPTIONS * request.

    It's possible it's related to some check that ispconfig do with cron job or similar?

    Have you some ideas about it?

    Yesterday i upgraded to the last version of ispconfig, (the old was 3.0.2) but i have same problems.

    Any clue?

    Thank you
     
  2. till

    till Super Moderator Staff Member ISPConfig Developer

    Please post the output of the command:

    top

    when the load is high on your server.
     
  3. Thesee

    Thesee New Member

    hello,
    i have a top screen of some day ago (i have a shell script that print the top when the load is too high)

    Yesterday i updated all package of the system. I'll print the top if the load will grow again.
    Thank you

    Code:
    Local Date & Time : Thu Jun  2 00:05:17 CEST 2011
    -------------------------------------------
    System snapshot from top:
    -------------------------------------------
    top - 00:05:26 up 23 days,  5:48,  0 users,  load average: 21.60, 17.78, 54.82
    Tasks: 234 total,  50 running, 184 sleeping,   0 stopped,   0 zombie
    Cpu(s):  6.3%us,  1.7%sy,  1.9%ni, 88.7%id,  1.2%wa,  0.0%hi,  0.1%si,  0.0%st
    Mem:   4050524k total,  3497924k used,   552600k free,   176632k buffers
    Swap:  6094840k total,   937436k used,  5157404k free,  1301964k cached
    
      PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND            
     7896 apache    19   0  680m  41m  14m R 22.9  1.0   0:05.10 /usr/sbin/httpd    
     7963 apache    19   0  680m  40m  13m R 15.9  1.0   0:05.10 /usr/sbin/httpd    
     8007 apache    19   0  681m  49m  21m R 15.9  1.3   0:11.31 /usr/sbin/httpd    
     8010 apache    17   0  678m  60m  34m R 11.8  1.5   0:04.34 /usr/sbin/httpd    
     7938 apache    17   0  680m  43m  16m R 10.9  1.1   0:03.90 /usr/sbin/httpd    
     7979 apache    17   0  680m  42m  15m R 10.9  1.1   0:03.72 /usr/sbin/httpd    
     7980 apache    17   0  676m  33m  10m R 10.7  0.9   0:01.39 /usr/sbin/httpd    
     7868 apache    16   0  677m  35m  10m R  9.9  0.9   0:01.57 /usr/sbin/httpd    
     7937 apache    17   0  677m  35m  11m R  2.5  0.9   0:02.04 /usr/sbin/httpd    
     7948 apache    17   0  624m  34m  10m R  2.5  0.9   0:02.03 /usr/sbin/httpd    
     8108 apache    17   0  620m  23m 2296 R  2.5  0.6   0:01.54 /usr/sbin/httpd    
     8139 apache    17   0  678m  40m  14m R  2.5  1.0   0:02.63 /usr/sbin/httpd    
     8138 apache    17   0  676m  29m 7092 R  2.3  0.8   0:01.60 /usr/sbin/httpd    
     7969 apache    17   0  678m  40m  14m R  2.1  1.0   0:02.44 /usr/sbin/httpd    
     7986 apache    17   0  620m  24m 2696 R  2.1  0.6   0:01.55 /usr/sbin/httpd    
     7993 apache    17   0  682m  50m  20m R  2.1  1.3   0:02.34 /usr/sbin/httpd    
     8140 apache    17   0  620m  23m 2320 R  2.1  0.6   0:01.34 /usr/sbin/httpd    
     8107 apache    18   0  678m  40m  15m R  1.7  1.0   0:03.81 /usr/sbin/httpd    
     8122 apache    18   0  677m  34m 9944 R  1.7  0.9   0:02.11 /usr/sbin/httpd    
     1337 vmail     16   0 15224 1732  808 S  1.5  0.0   0:00.29 /usr/lib/courier-im
     1386 root      15   0 34848 2840 1424 S  1.4  0.1   0:00.13 /usr/lib/courier-im
     1388 root      15   0 34848 2840 1424 S  1.4  0.1   0:00.13 /usr/lib/courier-im
     1390 root      15   0 34848 2844 1424 S  1.4  0.1   0:00.13 /usr/lib/courier-im
     7867 apache    19   0  679m  41m  16m R  1.1  1.0   0:04.00 /usr/sbin/httpd    
     7882 apache    19   0  680m  53m  26m R  1.1  1.4   0:06.89 /usr/sbin/httpd    
     7883 apache    19   0  680m  41m  14m R  1.1  1.0   0:05.83 /usr/sbin/httpd    
     7949 apache    19   0  678m  36m  11m R  1.1  0.9   0:03.04 /usr/sbin/httpd    
     7970 apache    19   0  678m  35m  10m R  1.1  0.9   0:03.89 /usr/sbin/httpd    
     7983 apache    19   0  620m  24m 2676 R  1.1  0.6   0:02.84 /usr/sbin/httpd    
     7987 apache    19   0  681m  50m  23m R  1.1  1.3   0:04.00 /usr/sbin/httpd    
     8013 apache    19   0  678m  43m  18m R  1.1  1.1   0:04.40 /usr/sbin/httpd    
     8028 apache    19   0  678m  44m  18m R  1.1  1.1   0:03.61 /usr/sbin/httpd    
     8094 apache    19   0  677m  40m  15m R  1.1  1.0   0:03.89 /usr/sbin/httpd    
     8100 apache    19   0  689m  43m  10m R  1.1  1.1   0:05.21 /usr/sbin/httpd    
     8109 apache    19   0  677m  35m  10m R  1.1  0.9   0:03.34 /usr/sbin/httpd    
     8123 apache    19   0  679m  38m  12m R  1.1  1.0   0:03.89 /usr/sbin/httpd    
     8124 apache    19   0  677m  37m  12m R  1.1  0.9   0:03.49 /usr/sbin/httpd    
     8126 apache    19   0  680m  43m  15m R  1.1  1.1   0:06.71 /usr/sbin/httpd    
     8142 apache    19   0  679m  42m  16m R  1.1  1.1   0:03.69 /usr/sbin/httpd    
     8146 apache    19   0  680m  40m  13m R  1.1  1.0   0:05.91 /usr/sbin/httpd    
     1387 vmail     16   0 14544 1032  752 S  0.6  0.0   0:00.06 /usr/lib/courier-im
     7988 apache    18   0  680m  57m  30m R  0.6  1.5   0:08.59 /usr/sbin/httpd    
     1379 root      18   0 12736 1064  708 R  0.4  0.0   0:00.04 top -b -c -n1      
      467 root      10  -5     0    0    0 S  0.1  0.0  12:15.33 [kmirrord]         
      472 root      10  -5     0    0    0 S  0.1  0.0  10:04.05 [kjournald]        
     7997 apache    15   0  679m  38m  12m S  0.1  1.0   0:02.92 /usr/sbin/httpd    
        1 root      15   0 10348  596  564 S  0.0  0.0   0:02.55 init [3]           
        2 root      RT  -5     0    0    0 S  0.0  0.0   0:05.70 [migration/0]      
        3 root      34  19     0    0    0 S  0.0  0.0   0:01.04 [ksoftirqd/0]      
        4 root      RT  -5     0    0    0 S  0.0  0.0   0:00.00 [watchdog/0]       
        5 root      RT  -5     0    0    0 S  0.0  0.0   0:05.71 [migration/1]      
        6 root      34  19     0    0    0 S  0.0  0.0   0:00.99 [ksoftirqd/1]      
        7 root      RT  -5     0    0    0 S  0.0  0.0   0:00.00 [watchdog/1]       
        8 root      10  -5     0    0    0 S  0.0  0.0   0:03.82 [events/0]         
        9 root      10  -5     0    0    0 S  0.0  0.0   0:00.70 [events/1]         
       10 root      10  -5     0    0    0 S  0.0  0.0   0:00.00 [khelper]          
       47 root      10  -5     0    0    0 S  0.0  0.0   0:00.00 [kthread]          
       52 root      10  -5     0    0    0 S  0.0  0.0   0:00.34 [kblockd/0]        
       53 root      10  -5     0    0    0 S  0.0  0.0   0:02.44 [kblockd/1]        
       54 root      14  -5     0    0    0 S  0.0  0.0   0:00.00 [kacpid]           
      160 root      14  -5     0    0    0 S  0.0  0.0   0:00.00 [cqueue/0]         
      161 root      15  -5     0    0    0 S  0.0  0.0   0:00.00 [cqueue/1]         
      164 root      11  -5     0    0    0 S  0.0  0.0   0:00.00 [khubd]            
      166 root      10  -5     0    0    0 S  0.0  0.0   0:00.00 [kseriod]          
      237 root      15   0     0    0    0 S  0.0  0.0   0:00.00 [khungtaskd]       
      240 root      10  -5     0    0    0 S  0.0  0.0   3:28.30 [kswapd0]          
      241 root      17  -5     0    0    0 S  0.0  0.0   0:00.00 [aio/0]            
      242 root      17  -5     0    0    0 S  0.0  0.0   0:00.00 [aio/1]            
      393 root      11  -5     0    0    0 S  0.0  0.0   0:00.00 [kpsmoused]        
      424 root      15  -5     0    0    0 S  0.0  0.0   0:00.00 [ata/0]            
      425 root      15  -5     0    0    0 S  0.0  0.0   0:00.00 [ata/1]            
      426 root      15  -5     0    0    0 S  0.0  0.0   0:00.00 [ata_aux]          
      430 root      10  -5     0    0    0 S  0.0  0.0   0:00.00 [scsi_eh_0]        
      431 root      10  -5     0    0    0 S  0.0  0.0   0:00.00 [scsi_eh_1]        
      432 root      10  -5     0    0    0 S  0.0  0.0   0:00.00 [scsi_eh_2]        
      433 root      10  -5     0    0    0 S  0.0  0.0   0:00.00 [scsi_eh_3]        
      440 root      12  -5     0    0    0 S  0.0  0.0   0:00.00 [kstriped]         
      453 root      13  -5     0    0    0 S  0.0  0.0   0:00.00 [ksnapd]           
      468 root      11  -5     0    0    0 S  0.0  0.0   0:00.00 [kcopyd]           
      492 vmail     15   0 15244 1620  776 S  0.0  0.0   2:12.19 /usr/lib/courier-im
      497 root      10  -5     0    0    0 S  0.0  0.0   0:03.27 [kauditd]          
      530 root      21  -4 12676  360  356 S  0.0  0.0   0:00.28 /sbin/udevd -d     
      757 vmail     16   0 15220 1676  776 S  0.0  0.0   0:36.69 /usr/lib/courier-im
     1333 root      15   0 34848 2844 1424 S  0.0  0.1   0:00.16 /usr/lib/courier-im
     1334 root      15   0 34848 2848 1424 S  0.0  0.1   0:00.17 /usr/lib/courier-im
     1335 root      15   0 34848 2848 1424 S  0.0  0.1   0:00.14 /usr/lib/courier-im
     1336 vmail     15   0 14528  948  776 S  0.0  0.0   0:00.01 /usr/lib/courier-im
     1338 vmail     15   0 14528  968  776 S  0.0  0.0   0:00.01 /usr/lib/courier-im
     1339 root      17   0 99.6m 1176  872 S  0.0  0.0   0:00.00 crond              
     1340 munin     18   0 99.6m 1176  872 S  0.0  0.0   0:00.00 crond              
     1342 root      19   0 99.6m 1176  872 S  0.0  0.0   0:00.00 crond              
     1343 root      18   0  8704 1008  852 S  0.0  0.0   0:00.01 /bin/bash /root/ale
     1352 root      19   0  8700  944  808 S  0.0  0.0   0:00.00 /bin/sh -c /usr/loc
     1353 root      18   0  8700 1000  848 S  0.0  0.0   0:00.00 /bin/bash /usr/loca
     1382 root      19   0  8700  516  364 S  0.0  0.0   0:00.00 /bin/bash /usr/loca
     1383 root      20   0   236  108   84 R  0.0  0.0   0:00.00 id -un             
     1389 vmail     16   0 14528  920  756 S  0.0  0.0   0:00.00 /usr/lib/courier-im
     1391 root      15   0 10056  568  464 S  0.0  0.0   0:00.00 /usr/lib/courier-im
     1575 root      20  -5     0    0    0 S  0.0  0.0   0:00.00 [kmpathd/0]        
     1576 root      10  -5     0    0    0 S  0.0  0.0   0:00.00 [kmpathd/1]        
     1577 root      20  -5     0    0    0 S  0.0  0.0   0:00.00 [kmpath_handlerd]  
     1639 root      10  -5     0    0    0 S  0.0  0.0   0:00.00 [kjournald]        
     2092 root      11  -4 92884  792  564 S  0.0  0.0   1:44.69 auditd             
     2094 root       7  -8 81808  852  612 S  0.0  0.0   0:14.32 /sbin/audispd      
     2124 root      15   0  5908  584  492 S  0.0  0.0   0:30.92 syslogd -m 0       
     2127 root      15   0  3804  396  340 S  0.0  0.0   0:00.00 klogd -x           
     2169 root      10  -5     0    0    0 S  0.0  0.0   2:55.72 [kondemand/0]      
     2170 root      10  -5     0    0    0 S  0.0  0.0   8:01.32 [kondemand/1]      
     2188 root      18   0 10760  320  244 R  0.0  0.0   0:18.68 irqbalance         
     2208 rpc       22   0  8052  404  400 S  0.0  0.0   0:00.00 portmap            
     2239 root      12  -5     0    0    0 S  0.0  0.0   0:00.00 [rpciod/0]         
     2240 root      12  -5     0    0    0 S  0.0  0.0   0:00.00 [rpciod/1]         
     2247 rpcuser   24   0 10160  572  568 S  0.0  0.0   0:00.00 rpc.statd          
     2254 root      23   0  3792  252  248 S  0.0  0.0   0:00.00 /usr/sbin/courierlo
     2255 root      18   0 50396 1228 1192 S  0.0  0.0   0:00.00 /usr/libexec/courie
     2286 root      21   0 55180  240  236 S  0.0  0.0   0:00.00 rpc.idmapd         
     2302 root      15   0 52600 1124  952 S  0.0  0.0   0:02.24 /usr/libexec/courie
     2303 root      15   0 52600 1124  952 S  0.0  0.0   0:02.49 /usr/libexec/courie
     2305 root      15   0 52600 1124  952 S  0.0  0.0   0:02.27 /usr/libexec/courie
     2306 root      15   0 52600 1124  952 S  0.0  0.0   0:02.43 /usr/libexec/courie
     2307 root      15   0 52600 1124  952 S  0.0  0.0   0:02.46 /usr/libexec/courie
     2314 dbus      15   0 21256  432  428 S  0.0  0.0   0:00.00 dbus-daemon --syste
     2367 root      25   0 21172  596  448 S  0.0  0.0   0:13.31 pcscd              
     2381 root      21   0  3800  460  456 S  0.0  0.0   0:00.00 /usr/sbin/acpid    
     2394 haldaemo  15   0 30452 1608 1192 S  0.0  0.0   0:00.89 hald               
     2395 root      15   0 21692  676  672 S  0.0  0.0   0:00.00 hald-runner        
     2404 haldaemo  18   0 12324  540  536 S  0.0  0.0   0:00.00 hald-addon-acpi: li
     2413 root      16   0 10228  416  396 S  0.0  0.0   2:49.23 hald-addon-storage:
     2446 root      24   0  8516  360  356 S  0.0  0.0   0:00.00 /usr/bin/hidd --ser
     2508 root      21   0  119m  996  844 S  0.0  0.0   0:07.70 automount          
     2547 root      15   0 62624  704  596 S  0.0  0.0   0:00.40 /usr/sbin/sshd     
     2578 root      18   0  130m 1116 1112 S  0.0  0.0   0:00.00 cupsd              
     2595 root      22   0 21644  544  540 S  0.0  0.0   0:00.00 xinetd -stayalive -
     2611 ntp       15   0 23388 5028 3904 S  0.0  0.1   0:00.71 ntpd -u ntp:ntp -p 
     2626 clamav    15   0  243m 124m 1172 S  0.0  3.2   2:01.59 clamd              
     2664 root      20   0 65928 1048 1044 S  0.0  0.0   0:00.01 /bin/sh /usr/bin/my
     2824 mysql     15   0 1230m 682m 4508 S  0.0 17.3 618:26.81 /usr/libexec/mysqld
     2851 nobody    15   0  5816  204  108 S  0.0  0.0   0:15.78 mydns -b           
     2923 amavis    15   0  304m  27m 2668 S  0.0  0.7   0:02.31 amavisd (master)   
     2933 root      16   0  3792  304  248 S  0.0  0.0   0:01.20 /usr/sbin/courierlo
     2934 root      15   0  7988  524  472 S  0.0  0.0   0:01.81 /usr/lib/courier-im
     2940 root      16   0  3792  304  248 S  0.0  0.0   0:00.03 /usr/sbin/courierlo
     2941 root      15   0  7988  524  472 S  0.0  0.0   0:00.06 /usr/lib/courier-im
     2946 root      17   0  3792  308  248 S  0.0  0.0   0:05.71 /usr/sbin/courierlo
     2947 root      15   0  7988  520  472 S  0.0  0.0   0:07.27 /usr/lib/courier-im
     2952 root      18   0  3792  304  248 S  0.0  0.0   0:01.09 /usr/sbin/courierlo
     2953 root      15   0  7988  520  472 S  0.0  0.0   0:01.29 /usr/lib/courier-im
     3010 root      15   0 60248 1976 1868 S  0.0  0.0   0:14.18 /usr/libexec/postfi
     3018 postfix   16   0 61272 2084 1924 S  0.0  0.1   0:01.03 qmgr -l -t fifo -u 
     3052 gearmand  15   0 19192  500  316 S  0.0  0.0   0:00.03 /usr/sbin/gearmand 
     3067 root      18   0  6452  288  268 S  0.0  0.0   0:00.00 gpm -m /dev/input/m
     3100 root      15   0  116m  508  436 S  0.0  0.0   0:02.50 pure-ftpd (SERVER) 
     3113 root      18   0 74812  700  576 S  0.0  0.0   0:07.84 crond              
     3230 xfs       18   0 20492  584  540 S  0.0  0.0   0:00.00 xfs -droppriv -daem
     3521 root      15   0  153m 3664 1652 S  0.0  0.1   0:03.91 /usr/bin/python /us
     3603 root      15   0 12916 1056  892 S  0.0  0.0   0:09.12 /usr/libexec/gam_se
     3633 root      18   0 18732  352  304 S  0.0  0.0   0:00.00 /usr/sbin/atd      
     3649 root      16   0 50684  912  908 S  0.0  0.0   0:00.25 /usr/sbin/saslauthd
     3650 root      15   0 51164  892  888 S  0.0  0.0   0:00.31 /usr/sbin/saslauthd
     3651 root      15   0 52484  892  888 S  0.0  0.0   0:00.50 /usr/sbin/saslauthd
     3652 root      15   0 50924  892  888 S  0.0  0.0   0:00.28 /usr/sbin/saslauthd
     3653 root      15   0 50684  892  888 S  0.0  0.0   0:00.25 /usr/sbin/saslauthd
     3684 avahi     15   0 23276  924  804 S  0.0  0.0   0:01.90 avahi-daemon: runni
     3685 avahi     23   0 23148  160  148 S  0.0  0.0   0:00.00 avahi-daemon: chroo
     3716 postfix   15   0 60316 2012 1900 S  0.0  0.0   0:01.92 tlsmgr -l -t unix -
     3718 root      18   0 18416  352  308 S  0.0  0.0   0:00.13 /usr/sbin/smartd -q
     3722 root      16   0  3792  400  396 S  0.0  0.0   0:00.00 /sbin/mingetty tty1
     3723 root      16   0  3792  400  396 S  0.0  0.0   0:00.00 /sbin/mingetty tty2
     3724 root      19   0  3792  400  396 S  0.0  0.0   0:00.00 /sbin/mingetty tty3
     3725 root      20   0  3792  400  396 S  0.0  0.0   0:00.00 /sbin/mingetty tty4
     3734 root      18   0  3792  400  396 S  0.0  0.0   0:00.00 /sbin/mingetty tty5
     3735 root      18   0  3792  400  396 S  0.0  0.0   0:00.00 /sbin/mingetty tty6
     4011 root      15   0     0    0    0 S  0.0  0.0   0:20.96 [pdflush]          
     4038 root      15   0     0    0    0 S  0.0  0.0   0:55.58 [pdflush]          
     7780 root      16   0 66464 7316 2732 S  0.0  0.2   0:01.52 vlogger (access log
     7781 apache    18   0  224m  16m  488 S  0.0  0.4   0:00.00 /usr/sbin/httpd    
     7865 apache    18   0  677m  36m  12m R  0.0  0.9   0:02.22 /usr/sbin/httpd    
     7866 apache    15   0  680m  70m  42m S  0.0  1.8   0:07.63 /usr/sbin/httpd    
     7875 apache    15   0  677m  37m  13m S  0.0  1.0   0:00.68 /usr/sbin/httpd    
     7879 apache    15   0  680m  45m  18m S  0.0  1.1   0:03.15 /usr/sbin/httpd    
     7897 apache    15   0  676m  34m  11m S  0.0  0.9   0:00.39 /usr/sbin/httpd    
     7917 apache    18   0  680m  54m  26m R  0.0  1.4   0:08.40 /usr/sbin/httpd    
     7947 apache    18   0  623m  31m 8016 R  0.0  0.8   0:02.21 /usr/sbin/httpd    
     7962 apache    18   0  678m  39m  14m R  0.0  1.0   0:02.11 /usr/sbin/httpd    
     7964 apache    15   0  679m  40m  14m S  0.0  1.0   0:02.80 /usr/sbin/httpd    
     7967 apache    15   0  625m  35m 9952 S  0.0  0.9   0:01.05 /usr/sbin/httpd    
     7968 apache    15   0  679m  39m  12m S  0.0  1.0   0:02.65 /usr/sbin/httpd    
     7972 apache    18   0  678m  38m  13m R  0.0  1.0   0:02.34 /usr/sbin/httpd    
     7973 apache    15   0  621m  27m 5116 S  0.0  0.7   0:00.10 /usr/sbin/httpd    
     7974 apache    15   0  622m  30m 7220 S  0.0  0.8   0:00.25 /usr/sbin/httpd    
     7975 apache    15   0  676m  30m 7392 S  0.0  0.8   0:00.31 /usr/sbin/httpd    
     7978 apache    15   0  678m  37m  13m S  0.0  1.0   0:00.82 /usr/sbin/httpd    
     7995 apache    15   0  681m  65m  36m S  0.0  1.6   0:04.90 /usr/sbin/httpd    
     7996 apache    15   0  678m  46m  20m S  0.0  1.2   0:04.16 /usr/sbin/httpd    
     7998 apache    15   0  681m  57m  30m S  0.0  1.4   0:02.42 /usr/sbin/httpd    
     7999 apache    15   0  680m  58m  30m S  0.0  1.5   0:10.65 /usr/sbin/httpd    
     8008 apache    15   0  678m  39m  14m S  0.0  1.0   0:00.90 /usr/sbin/httpd    
     8012 apache    15   0  678m  43m  17m S  0.0  1.1   0:01.62 /usr/sbin/httpd    
     8014 apache    15   0  678m  53m  27m S  0.0  1.3   0:02.56 /usr/sbin/httpd    
     8015 apache    18   0  679m  39m  14m R  0.0  1.0   0:03.53 /usr/sbin/httpd    
     8029 apache    15   0  678m  35m  10m S  0.0  0.9   0:01.05 /usr/sbin/httpd    
     8030 apache    15   0  676m  29m 6776 S  0.0  0.8   0:00.23 /usr/sbin/httpd    
     8092 apache    15   0  680m  43m  16m S  0.0  1.1   0:03.02 /usr/sbin/httpd    
     8093 apache    15   0  677m  37m  13m S  0.0  1.0   0:00.75 /usr/sbin/httpd    
     8095 apache    15   0  620m  23m 1820 S  0.0  0.6   0:00.06 /usr/sbin/httpd    
     8099 apache    15   0  620m  23m 1820 S  0.0  0.6   0:00.06 /usr/sbin/httpd    
     8101 apache    18   0  677m  36m  11m R  0.0  0.9   0:02.29 /usr/sbin/httpd    
     8125 apache    18   0  680m  44m  16m R  0.0  1.1   0:05.54 /usr/sbin/httpd    
     8128 apache    15   0  680m  40m  13m S  0.0  1.0   0:05.09 /usr/sbin/httpd    
     8129 apache    15   0  676m  29m 6780 S  0.0  0.7   0:00.22 /usr/sbin/httpd    
     8136 apache    18   0  679m  37m  11m R  0.0  1.0   0:02.34 /usr/sbin/httpd    
     8137 apache    15   0  680m  39m  12m S  0.0  1.0   0:02.67 /usr/sbin/httpd    
     8141 apache    15   0  677m  32m 8824 S  0.0  0.8   0:00.42 /usr/sbin/httpd    
     8143 apache    15   0  625m  35m 9.9m S  0.0  0.9   0:00.77 /usr/sbin/httpd    
     8144 apache    15   0  680m  41m  14m S  0.0  1.1   0:02.67 /usr/sbin/httpd    
     9327 vmail     15   0 13092 1148  912 S  0.0  0.0   0:00.25 /usr/libexec/gam_se
    29147 root      15   0 99368 8908 1704 S  0.0  0.2   0:00.53 /usr/sbin/munin-nod
    29353 amavis    16   0  324m  82m 3632 S  0.0  2.1   0:34.85 amavisd (ch7-avail)
    29447 postfix   18   0 60320 2508 1892 S  0.0  0.1   0:00.02 pickup -l -t fifo -
    29720 amavis    15   0  307m  67m 3668 S  0.0  1.7   0:04.74 amavisd (ch5-avail)
    
    
    
    
     
  4. till

    till Super Moderator Staff Member ISPConfig Developer

    I recommend that you install the software packages munin and monit on your system to monitor the server parameters. You can find tutorials for that here on howtoforge.

    Additionally you should check your apache access log if there are any unusual requests. Such a high load can e.g. be caused by a DOS. Then enable also the slow query log in mysql as slow mysql queries can bring down apache as well.

    It is unlikely that the problem is directly related to ispcnfig.
     
  5. Thesee

    Thesee New Member

    Hi till thank you for fast answer.
    Unfortunately I already checked all.
    Munin is installed, but when apache hangs the log told me nothing. All request are standard and (very strange thing) the traffic is very low (pratically null)!
    ...i'll take a look to mod_evasive to be 100 sure it cannot be a DOS attack

    Mysql is optmized and it does not load too much the system.
    just calculated, i have 3 slow query on 9241827 requests (mainly during the night backup)

    I supposed the problem could be connected to ispconfig because i checked all before without any clue!

    Thank you again for you support. i'll continue to investigate
     
  6. Thesee

    Thesee New Member

    is possible to disable (maybe uninstall ) ispconfig to make some test and then re-enable (maybe reinstall ) it?
    There will be problems ? I'll mess up the server?
     
  7. till

    till Super Moderator Staff Member ISPConfig Developer

    I'am quite sure that ISPConfig is not causing the issue. ISPConfig is just a set of scripts that write the vhost config file in the sites-enabled folder, thats all. ISPConfig is not even a daemon that can caue any load when you are not changing any sttings. Also there are about 20 - 30 thousand new ispconfig installs per month which dont have such an issue, so you can assume that its not a bug in ispconfig.

    You can uninstall ispconfig with the uninstall.php script which is in the install folder of the ispconfig tar.gz. But you will have to reinstall the whole Linux system when you want to install ispconfig again afterwards.

    Please check the apache log files again, the global apache error.log and the error.log files in the log folders of the websites. Also check the access log files if there are any unusual requests when the problems occur. Are there any peaks in the munin graphs?
     
  8. Thesee

    Thesee New Member

    munin graphs

    Ok Till, i'm not going to uninstall ispconfig !
    Last night a new load with apache down about at 23.25
    as you can see on the munin graph i cant see anything unusual (but maybe you can see something that i cannot see :) )

    Thank you
     

    Attached Files:

  9. till

    till Super Moderator Staff Member ISPConfig Developer

    Is this a virtual server? If yes, which virtualisation software is used?
     
  10. Thesee

    Thesee New Member

    No, it's a dedicated server
    Supermicro
    Intel® Pentium® D 3800 MHz
    4Gb RAM

    2.6.18-238.12.1.el5.centos.plus
    CentOS release 5.6 (Final)
     
  11. comex

    comex New Member

    Hi, I"m also facing the same issue, but on a VM with CentOs5.6 and ISPConfig3.
     
  12. Thesee

    Thesee New Member

    solved?

    Did you solved?
    I migrated all the production vhost on a second server, all is working great.
    Then I re-installed centos but without the required software setup for ispconfig3
    All was working good.
    Then i reinstalled centos with the software required for ispconfig 3:
    apache hangs without traffic!!!

    Probably is related to some software configuration (i followed the how-to guide) but i dont know what to check.

    Regards.
     
  13. till

    till Super Moderator Staff Member ISPConfig Developer

    Which errors do you get in the apache error.log file?
     
  14. Thesee

    Thesee New Member

    here the lines before crash

    Code:
    [error] server reached MaxClients setting, consider raising the MaxClients setting
    This happen because httpd hangs and then all requests are queued, within minutes the queue becomes full

    httpd config

    Code:
    Timeout 150
    KeepAlive On
    MaxKeepAliveRequests 100
    KeepAliveTimeout 8
    <IfModule prefork.c>
        StartServers          30
        MinSpareServers       10
        MaxSpareServers      20
        ServerLimit		100
        MaxClients          100
        MaxRequestsPerChild   8000
    </IfModule>
    
     
    Last edited: Sep 27, 2011
  15. till

    till Super Moderator Staff Member ISPConfig Developer

    It might be that perl modules are missing on your server which causes the apache vhost logging /vlogger) to fail. Please search the error log if there are any errors for vlogger inside.

    You can also run the command:

    vlogger -v

    to see if you get any errors.
     
  16. Thesee

    Thesee New Member

    vlogger -v
    Code:
    vlogger -v
    VLogger 1.3 (apache logfile parser)
    Written by Steve J. Kondik <[email protected]>
    
    This is free software; see the source for copying conditions.  There is NO
    warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
    
    no vlogger entries on log files, seems is working good.

    any clue?
     
  17. till

    till Super Moderator Staff Member ISPConfig Developer

    Ok. Just to be absolutely sure that not vlogger is causing this, edit the ispconfig.vhost file (which is in the sites-available folder in the /etc/httpd config folder) and comment out the custom_log directive and then restart apache.
     
  18. Thesee

    Thesee New Member

    Hi Till,
    on ispconfig.vhost file the custom_log directive is already commented out.

    there is an ispconfig.conf file with
    Code:
    CustomLog "| /usr/local/ispconfig/server/scripts/vlogger -s access.log -t \"%Y%m%d-access.log\" -d \"/etc/vlogger-dbi.conf\" /var/log/ispconfig/httpd" combined_ispconfig
    
    maybe you was referring to this file?
    should i also comment out this?
     
  19. till

    till Super Moderator Staff Member ISPConfig Developer

    you are right, i meant that file :) please comenr it out in ispconfig.conf
     
  20. Thesee

    Thesee New Member

    Ok commented out!
    Code:
    #CustomLog "| /usr/local/isp.....
    httpd restarted

    I'll let you know
     

Share This Page