CPU load locks up box. Apache or MYSQL related.

Discussion in 'General' started by crypted, Oct 12, 2010.

  1. crypted

    crypted Member

    Honestly I only skimmed through the utils site. But, I will look into setting up a web interface when I have time to find the destructions on what to do.

    It's pointless to use SSH in my opinion...more of a pain to watch.
     
  2. till

    till Super Moderator Staff Member ISPConfig Developer

    Which wordpress caching plugin do you use?
     
  3. crypted

    crypted Member

    Till, I am using W3 Total Cache.

    Mark, any recommendations on a page outlining installing/setting up collectl output to a website? Not seeing it...
     
    Last edited: Oct 13, 2010
  4. crypted

    crypted Member

    Here's the latest from tuning-primer.sh for Mysql:

    Code:
    SLOW QUERIES
    The slow query log is enabled.
    Current long_query_time = 10 sec.
    You have 14077 out of 16432161 that take longer than 10 sec. to complete
    Your long_query_time seems to be fine
    
    BINARY UPDATE LOG
    The binary update log is NOT enabled.
    You will not be able to do point in time recovery
    See http://dev.mysql.com/doc/refman/5.0/en/point-in-time-recovery.html
    
    WORKER THREADS
    Current thread_cache_size = 8
    Current threads_cached = 7
    Current threads_per_sec = 0
    Historic threads_per_sec = 0
    Your thread_cache_size is fine
    
    MAX CONNECTIONS
    Current max_connections = 200
    Current threads_connected = 4
    Historic max_used_connections = 72
    The number of used connections is 36% of the configured maximum.
    Your max_connections variable seems to be fine.
    
    INNODB STATUS
    Current InnoDB index space = 64 K
    Current InnoDB data space = 64 K
    Current InnoDB buffer pool free = 95 %
    Current innodb_buffer_pool_size = 8 M
    Depending on how much space your innodb indexes take up it may be safe
    to increase this value to up to 2 / 3 of total system memory
    
    MEMORY USAGE
    Max Memory Ever Allocated : 381 M
    Configured Max Per-thread Buffers : 899 M
    Configured Max Global Buffers : 58 M
    Configured Max Memory Limit : 957 M
    Physical Memory : 3.87 G
    Max memory limit seem to be within acceptable norms
    
    KEY BUFFER
    Current MyISAM index space = 131 M
    Current key_buffer_size = 16 M
    Key cache miss rate is 1 : 1381
    Key buffer free ratio = 0 %
    You could increase key_buffer_size
    It is safe to raise this up to 1/4 of total system memory;
    assuming this is a dedicated database server.
    
    QUERY CACHE
    Query cache is enabled
    Current query_cache_size = 32 M
    Current query_cache_used = 28 M
    Current query_cache_limit = 2 M
    Current Query cache Memory fill ratio = 88.43 %
    Current query_cache_min_res_unit = 4 K
    However, 101653 queries have been removed from the query cache due to lack of memory
    Perhaps you should raise query_cache_size
    MySQL won't cache query results that are larger than query_cache_limit in size
    
    SORT OPERATIONS
    Current sort_buffer_size = 2 M
    Current read_rnd_buffer_size = 256 K
    Sort buffer seems to be fine
    
    JOINS
    Current join_buffer_size = 2.00 M
    You have had 6 queries where a join could not use an index properly
    You should enable "log-queries-not-using-indexes"
    Then look for non indexed joins in the slow query log.
    If you are unable to optimize your queries you may want to increase your
    join_buffer_size to accommodate larger joins in one pass.
    
    Note! This script will still suggest raising the join_buffer_size when
    ANY joins not using indexes are found.
    
    OPEN FILES LIMIT
    Current open_files_limit = 1210 files
    The open_files_limit should typically be set to at least 2x-3x
    that of table_cache if you have heavy MyISAM usage.
    Your open_files_limit value seems to be fine
    
    TABLE CACHE
    Current table_cache value = 500 tables
    You have a total of 391 tables
    You have 452 open tables.
    The table_cache value seems to be fine
    
    TEMP TABLES
    Current max_heap_table_size = 80 M
    Current tmp_table_size = 80 M
    Of 49835 temp tables, 39% were created on disk
    Perhaps you should increase your tmp_table_size and/or max_heap_table_size
    to reduce the number of disk-based temporary tables
    Note! BLOB and TEXT columns are not allow in memory tables.
    If you are using these columns raising these values might not impact your 
    ratio of on disk temp tables.
    
    TABLE SCANS
    Current read_buffer_size = 128 K
    Current table scan ratio = 1207 : 1
    read_buffer_size seems to be fine
    
    TABLE LOCKING
    Current Lock Wait ratio = 1 : 1884
    You may benefit from selective use of InnoDB.
    
     
  5. crypted

    crypted Member

    Mark, my LOAD AVG was at 3.56 when I looked at TOP randomly. Started Collectl. Here's the output below up until the load avg was down to 1.10.

    Code:
    #         <--------CPU--------><-----------Memory-----------><----------Disks-----------><----------Network---------->
    #Time     cpu sys inter  ctxsw Free Buff Cach Inac Slab  Map KBRead  Reads KBWrit Writes   KBIn  PktIn  KBOut  PktOut 
    00:53:40   33  16    61   1138 162M 669M   1G   1G 129M   2G      0      0      0      0      0      6      0       4 
    00:53:41   36   9    95    587 162M 669M   1G   1G 129M   2G     60      4      0      0      3     38     20      39 
    00:53:42   44  21    52   1487 164M 669M   1G   1G 129M   2G      4      1   3760     29      4     49      4      46 
    00:53:43    0   0    14     72 164M 669M   1G   1G 129M   2G      0      0      0      0      1      7      2       6 
    00:53:44   25   8    22     94 164M 669M   1G   1G 129M   2G      0      0      0      0      1     10      5       8 
    00:53:45   81  36    67    911 164M 669M   1G   1G 129M   2G      0      0   3228     18      2     16      1       9 
    00:53:46   39  22    34   1319 164M 669M   1G   1G 129M   2G      0      0     52      8      4     30      4      25 
    00:53:47    0   0    30     74 164M 669M   1G   1G 129M   2G      0      0      0      0      2     18      7      23 
    00:53:48    2   1     5     61 306M 669M   1G   1G 129M   2G      0      0      0      0      1      9      1       7 
    00:53:49    0   0    23     76 306M 669M   1G   1G 129M   2G      0      0      0      0      0      5      0       1 
    00:53:50    0   0    20     88 306M 669M   1G   1G 129M   2G      0      0      0      0      2     20      2      18 
    00:53:51   19   1    55    169 306M 669M   1G   1G 128M   2G     12      3    216      9      1      7      0       3 
    00:53:52   50  30    39     61 306M 669M   1G   1G 128M   2G      8      2      0      0      3     44     35      42 
    00:53:53    5   1    77    178 306M 669M   1G   1G 128M   2G      0      0      0      0      1     19     12      16 
    00:53:54    1   0    27     72 306M 669M   1G   1G 128M   2G      0      0      0      0      3     46      3      35 
    00:53:55   50   3    24    175 306M 669M   1G   1G 128M   2G      0      0      0      0      1     15      5       9 
    00:53:56    3   0   147    233 306M 669M   1G   1G 128M   2G     56      4      0      0      2     30     27      31 
    00:53:57    1   0    86    104 307M 669M   1G   1G 128M   2G      4      1   3452     22      6     75     82      93 
    00:53:58   11   0    35    104 306M 669M   1G   1G 128M   2G      0      0    104     12      4     40      9      32 
    00:53:59   50  28    88    200 306M 669M   1G   1G 128M   2G      0      0      0      0      2     14      1       6 
    00:54:00   33   8    36    196 306M 669M   1G   1G 128M   2G      4      1      0      0      4     47      5      45 
    00:54:01   50  15    10    962 306M 669M   1G   1G 128M   2G      0      0      0      0      2     16      6      13 
    #         <--------CPU--------><-----------Memory-----------><----------Disks-----------><----------Network---------->
    #Time     cpu sys inter  ctxsw Free Buff Cach Inac Slab  Map KBRead  Reads KBWrit Writes   KBIn  PktIn  KBOut  PktOut 
    00:54:02   36  18    98   1210 306M 669M   1G   1G 128M   2G      0      0      0      0      1     12      1       7 
    00:54:03    0   0    42     99 306M 669M   1G   1G 128M   2G      0      0   3672     17      5     63     41      61 
    00:54:04    0   0    31     72 306M 669M   1G   1G 128M   2G      0      0      0      0      1     15      6      17 
    00:54:05   42   4    66    244 289M 669M   1G   1G 128M   2G      0      0      0      0      2     21      1       9 
    00:54:06   64  21    42    140 305M 669M   1G   1G 128M   2G      0      0      0      0      4     44      3      37 
    00:54:07   17   9    38     76 306M 669M   1G   1G 128M   2G      0      0     24      5      2     20      9      16 
    00:54:08    3   0    45    101 306M 669M   1G   1G 128M   2G      0      0   3352     21      2     21      2      15 
    00:54:09   50  19    22   1335 306M 669M   1G   1G 128M   2G      0      0    112      6      2     12      2      11 
    00:54:10   17  10     7    590 306M 669M   1G   1G 128M   2G      0      0      0      0      1      8      2       6 
    00:54:11   13   1    23     90 306M 669M   1G   1G 128M   2G      0      0      0      0      0      3      1       3 
    00:54:12    1   0   251     92 306M 669M   1G   1G 128M   2G      4      1   3668    230      2     17      2      17 
    00:54:13    1   0   151    169 306M 669M   1G   1G 128M   2G      0      0    492     61      3     39     47      50 
    00:54:14    0   0    32     94 306M 669M   1G   1G 128M   2G      0      0    220      7      2     29      2      22 
    00:54:15    1   0    44     88 306M 669M   1G   1G 128M   2G      0      0      0      0      1     14      1      12 
    00:54:16    0   0    78    190 306M 669M   1G   1G 128M   2G      0      0      0      0      3     34      9      37 
    00:54:17    0   0    37     80 306M 669M   1G   1G 128M   2G      0      0      0      0      3     37      3      36 
    00:54:18    0   0    38     97 307M 669M   1G   1G 128M   2G      0      0      0      0      2     28      2      28 
    00:54:19    1   0   162    235 306M 669M   1G   1G 128M   2G      4      1    144     10      3     30     11      25 
    00:54:20   39  15    99    112 306M 669M   1G   1G 128M   2G      0      0     88      8      7     83    146     142 
    00:54:21   67  27   125    153 306M 669M   1G   1G 128M   2G    176      2      0      0      5     75    316     225 
    00:54:22   41  23   120     87 306M 669M   1G   1G 128M   2G      0      0     32      2      5     51    168     129 
    00:54:23    0   0   105    109 306M 669M   1G   1G 128M   2G      0      0      0      0      7     90    254     187 
    #         <--------CPU--------><-----------Memory-----------><----------Disks-----------><----------Network---------->
    #Time     cpu sys inter  ctxsw Free Buff Cach Inac Slab  Map KBRead  Reads KBWrit Writes   KBIn  PktIn  KBOut  PktOut 
    00:54:24    0   0   107    128 306M 669M   1G   1G 128M   2G      0      0      0      0      4     64    221     159 
    00:54:25    1   0   148     91 306M 669M   1G   1G 128M   2G      4      1   3560     28      4     54    152     120 
    00:54:26    0   0    88     65 306M 669M   1G   1G 128M   2G      0      0      0      0      6     83    317     222 
    00:54:27    7   0   125    135 306M 669M   1G   1G 128M   2G      4      1      4      1      3     40    167     120 
    00:54:28   50  22    24    119 305M 669M   1G   1G 128M   2G      0      0      0      0      6     77    188     137 
    00:54:29   19   7   160    128 305M 669M   1G   1G 128M   2G      0      0      0      0      2     21     39      40 
    00:54:30    1   0   105    106 305M 669M   1G   1G 128M   2G     52      1      0      0      9    123    357     257 
    00:54:31    0   0    54     88 305M 669M   1G   1G 128M   2G      0      0   3456     25      4     66    191     134 
    00:54:32    0   0    31     63 305M 669M   1G   1G 128M   2G      0      0      0      0      1     14      1       7 
    00:54:33    0   0    38     80 306M 669M   1G   1G 128M   2G      0      0      0      0      2     22      2      13 
    00:54:34   32   7    30    163 305M 669M   1G   1G 128M   2G      0      0    108      7      2     18      7      17 
    00:54:35   47  27    40    526 305M 669M   1G   1G 128M   2G      0      0   3184     22      2     12      1       8 
    00:54:36    0   0    32    120 305M 669M   1G   1G 128M   2G      0      0      0      0      1     15      2      10 
    00:54:37    0   0    71     90 305M 669M   1G   1G 128M   2G    416      5      0      0      1     19      6      18 
    00:54:38    0   0    36     61 305M 669M   1G   1G 128M   2G      0      0      0      0      4     47    257     182 
    00:54:39    0   0    21     81 305M 669M   1G   1G 128M   2G      0      0      0      0      1     17    176     123 
    00:54:40    0   0    11     45 305M 669M   1G   1G 128M   2G      0      0      0      0      1     13      5      11 
    00:54:41   39   5    37    163 305M 669M   1G   1G 128M   2G      0      0    112      8      0      7      0       3 
    00:54:42  100  62    36     65 305M 669M   1G   1G 128M   2G      4      1     16      3      3     34     30      32 
    Curiously, my swap space is on hda which is the main hdd. I have an hdd which is a tarball backup (I create tarballs monthly of /var/www/ and so on). I'm thinking I should reconfigure it so that hdb has the swap. That should fix up some I/O right? It's SATA so I mean it would allow the system to use I/O on the primary hdd for primary purposes. It would then let the secondary hdd handle the swap. Right?
     
    Last edited: Oct 14, 2010
  6. crypted

    crypted Member

    Any opinion on my question above???

    Reposted:

    Curiously, my swap space is on hda which is the main hdd. I have an hdd which is a tarball backup (I create tarballs monthly of /var/www/ and so on). I'm thinking I should reconfigure it so that hdb has the swap. That should fix up some I/O right? It's SATA so I mean it would allow the system to use I/O on the primary hdd for primary purposes. It would then let the secondary hdd handle the swap. Right?

    Reason I'm reposting is that the datacenter techs believe that to be negligible. But, if it ends up swapping I figured it wouldn't be negligible at that point?
     
  7. Antennipasi

    Antennipasi ISPConfig Developer ISPConfig Developer

    Do you use any php-accelerator? especially when only couple sites get most of hits using those will have very good results.

    I have been on your seat, and i used APC and it was very sufficient. In debian install is very easy and configuration is childs play. APC can give 90% more power to Apache&php in frequently accessed sites.

    Feel free to contact via PM if you want.
     
  8. crypted

    crypted Member

    I installed xcache and memcache for PHP a couple of days ago.

    APC is another type then?

    Anyone feel free to comment on the above swap question! :)
     
  9. drewb0y

    drewb0y Member

    I think this (Alternative PHP Cache) is the APC referred to above.
    I will look into adding this to my server as well.
     
  10. crypted

    crypted Member

    XCache has improved performance for sure. I'm going to let them run for another day or so, and then switch to APC and test that out.
     
  11. drewb0y

    drewb0y Member

    I was doing some more reading on this and it seems that of the 3 accelerators, APC is about right in the middle performance wise but comes out way on top stability and future support wise.

    http://stackoverflow.com/questions/930877/apc-vs-eaccelerator-vs-xcache
    http://2bits.com/articles/benchmarking-apc-vs-eaccelerator-using-drupal.html
    http://2bits.com/articles/benchmark...hes-apc-eaccelerator-and-xcache-compared.html
    http://www.scribd.com/doc/88689/apcfacebook

    Apparently facebook uses APC and also contributes code back to the project which pretty much settles it for me.

    Install was only 2 steps (OK 3 if you count restarting Apache)

    Code:
    apt-get install php-apc
    Code:
    vi /etc/php5/conf.d/apc.ini
    add the following to apc.ini
    Code:
    extension=apc.so
    apc.enabled=1
    apc.shm_size=30
    Restart apache and done.
     
  12. Antennipasi

    Antennipasi ISPConfig Developer ISPConfig Developer

    These are reasons why we are using it too.
    Also PHP6 will have accelerator build-in, and based on APC.
     
  13. Turbanator

    Turbanator Member HowtoForge Supporter

    crypted:

    i wanted to throw in my 2 cents when it came to your mysql usage. If you're having high traffic on a few wordpress sites, I'm guessing you're having large load in mysql too.

    Can you try increasing your key_buffer_size from 16M to something closer to your index size...like 128M (now that you're at 4G Ram) and report back after a restart and 2 days usage? I like to start higher and drop it down if it's overkill.

    My other thought is your temp tables. Poor queries (and any query running over fulltext searches), will end up in temp tables which are tables written to disk. A major drain on resources. Decreasing your slow query time from 10 sec to 5 sec will probably show more of those queries so you can analyze and tweak them.
     
  14. crypted

    crypted Member

    Yeah, I've been tweaking it every day and half. What's 44 hours vs. 48 hours really? So anyway, I've changed many-a-setting with SQL and it's gotten a lot better.

    I tweaked it again this afternoon; I'll report back with those results the next day or two.

    But, RAM + PHP CACHING + SQL TWEAKS = 0.36 avg load.
     
  15. falko

    falko Super Moderator ISPConfig Developer

  16. crypted

    crypted Member

    Falko: good tutorial to have. I've been using PRIMER and TUNER. I realize they are different in their suggestions, but having both does help get a good sensibility for what's needed.

    I really recommend using caching on high PHP websites after this. SQL was the next issue and it's pretty well resolved I think.
     
  17. Turbanator

    Turbanator Member HowtoForge Supporter

  18. crypted

    crypted Member

    Today, we had a few minute lockup but I wasn't at home to observe it.

    My CPUWAIT hit 198.
    http://monitor.derekgordon.com/munin/derekgordon.com/my.derekgordon.com-cpu.html

    My processes average 198, however at the time of this occurrence the processes was over 360.

    Swap was utilized during this time. First time since the RAM increase to 4GB that this happened.
    http://monitor.derekgordon.com/munin/derekgordon.com/my.derekgordon.com-swap.html

    MYSQL has been tweaked and should be pretty good.
    Here's the output of the tuner.sh:
    Code:
    MySQL Version 5.0.51a-24+lenny4-log x86_64
    
    Uptime = 3 days 0 hrs 57 min 33 sec
    Avg. qps = 384
    Total Questions = 100897230
    Threads Connected = 10
    
    Server has been running for over 48hrs.
    It should be safe to follow these recommendations
    
    To find out more information on how each of these
    runtime variables effects performance visit:
    http://dev.mysql.com/doc/refman/5.0/en/server-system-variables.html
    Visit http://www.mysql.com/products/enterprise/advisors.html
    for info about MySQL's Enterprise Monitoring and Advisory Service
    
    SLOW QUERIES
    The slow query log is enabled.
    Current long_query_time = 10 sec.
    You have 55472 out of 100898023 that take longer than 10 sec. to complete
    Your long_query_time seems to be fine
    
    BINARY UPDATE LOG
    The binary update log is NOT enabled.
    You will not be able to do point in time recovery
    See http://dev.mysql.com/doc/refman/5.0/en/point-in-time-recovery.html
    
    WORKER THREADS
    Current thread_cache_size = 8
    Current threads_cached = 3
    Current threads_per_sec = 0
    Historic threads_per_sec = 0
    Your thread_cache_size is fine
    
    MAX CONNECTIONS
    Current max_connections = 200
    Current threads_connected = 9
    Historic max_used_connections = 76
    The number of used connections is 38% of the configured maximum.
    Your max_connections variable seems to be fine.
    
    INNODB STATUS
    Current InnoDB index space = 64 K
    Current InnoDB data space = 64 K
    Current InnoDB buffer pool free = 98 %
    Current innodb_buffer_pool_size = 64 M
    Depending on how much space your innodb indexes take up it may be safe
    to increase this value to up to 2 / 3 of total system memory
    
    MEMORY USAGE
    Max Memory Ever Allocated : 649 M
    Configured Max Per-thread Buffers : 1.07 G
    Configured Max Global Buffers : 232 M
    Configured Max Memory Limit : 1.30 G
    Physical Memory : 3.87 G
    Max memory limit seem to be within acceptable norms
    
    KEY BUFFER
    Current MyISAM index space = 134 M
    Current key_buffer_size = 16 M
    Key cache miss rate is 1 : 395
    Key buffer free ratio = 15 %
    You could increase key_buffer_size
    It is safe to raise this up to 1/4 of total system memory;
    assuming this is a dedicated database server.
    
    QUERY CACHE
    Query cache is enabled
    Current query_cache_size = 150 M
    Current query_cache_used = 54 M
    Current query_cache_limit = 24 M
    Current Query cache Memory fill ratio = 36.20 %
    Current query_cache_min_res_unit = 4 K
    MySQL won't cache query results that are larger than query_cache_limit in size
    
    SORT OPERATIONS
    Current sort_buffer_size = 2 M
    Current read_rnd_buffer_size = 256 K
    Sort buffer seems to be fine
    
    JOINS
    Current join_buffer_size = 3.00 M
    You have had 4 queries where a join could not use an index properly
    You should enable "log-queries-not-using-indexes"
    Then look for non indexed joins in the slow query log.
    If you are unable to optimize your queries you may want to increase your
    join_buffer_size to accommodate larger joins in one pass.
    
    Note! This script will still suggest raising the join_buffer_size when
    ANY joins not using indexes are found.
    
    OPEN FILES LIMIT
    Current open_files_limit = 1810 files
    The open_files_limit should typically be set to at least 2x-3x
    that of table_cache if you have heavy MyISAM usage.
    Your open_files_limit value seems to be fine
    
    TABLE CACHE
    Current table_cache value = 800 tables
    You have a total of 386 tables
    You have 498 open tables.
    The table_cache value seems to be fine
    
    TEMP TABLES
    Current max_heap_table_size = 375 M
    Current tmp_table_size = 375 M
    Of 153649 temp tables, 32% were created on disk
    Perhaps you should increase your tmp_table_size and/or max_heap_table_size
    to reduce the number of disk-based temporary tables
    Note! BLOB and TEXT columns are not allow in memory tables.
    If you are using these columns raising these values might not impact your 
    ratio of on disk temp tables.
    
    TABLE SCANS
    Current read_buffer_size = 128 K
    Current table scan ratio = 1749 : 1
    read_buffer_size seems to be fine
    
    TABLE LOCKING
    Current Lock Wait ratio = 1 : 1666
    You may benefit from selective use of InnoDB.
    I would venture a guess this is to do with rampant apache/fcgi processes. However, I changed FCGI to 1 child as Till suggested.

    Have a raw.gz collectl for that day. Honestly, I can't figure out how to get it to display information in the terminal worth anything. It just floods everything and screws up my terminal connection.

    Help appreciated.
     
  19. crypted

    crypted Member

    Apparently it's all related to fcgid. The processes continue to spawn even though I have the Max set to 1 as Till suggested.

    My load has been a constant 2-3 avg the past two hours. Namely it's php-cgi (fastcgi) that's doing the eating.

    Code:
    mysql    15381  6.7  2.9 416660 120856 pts/1   Sl   20:40   3:24 /usr/sbin/mysqld --basedir=/usr --datadir=/var/lib/mysql --user=mys
    root     15382  0.0  0.0   5040   668 pts/1    S    20:40   0:00 logger -p daemon.err -t mysqld_safe -i -t mysqld
    postfix  16961  0.0  0.0  38900  2288 ?        S    20:50   0:00 pickup -l -t fifo -u -c
    root     17097  0.0  0.4 330864 17196 ?        Ss   20:53   0:00 /usr/sbin/apache2 -k start
    root     17098  0.0  0.1  39188  6916 ?        S    20:53   0:00 vlogger (access log)
    www-data 17099  0.0  0.1 170056  5320 ?        S    20:53   0:00 /usr/sbin/apache2 -k start
    www-data 17100  0.0  0.3 332128 12396 ?        S    20:53   0:02 /usr/sbin/apache2 -k start
    web5     17105  5.5  1.8 272388 73556 ?        R    20:53   2:06 /usr/bin/php-cgi -d open_basedir=/var/www/clients/client1/web5/web:
    web5     17137  6.8  1.7 270096 71096 ?        S    20:53   2:34 /usr/bin/php-cgi -d open_basedir=/var/www/clients/client1/web5/web:
    web11    17184  0.1  1.3 266256 55208 ?        S    20:54   0:02 /usr/bin/php-cgi -d open_basedir=/var/www/clients/client1/web11/web
    web1     17212  0.2  2.4 284140 101260 ?       S    20:54   0:05 /usr/bin/php-cgi -d open_basedir=/var/www/clients/client1/web1/web:
    web7     17562  0.0  1.3 263636 55780 ?        S    20:55   0:01 /usr/bin/php-cgi -d open_basedir=/var/www/clients/client1/web7/web:
    web8     17688  6.3  1.9 279888 81276 ?        S    20:57   2:08 /usr/bin/php-cgi -d open_basedir=/var/www/clients/client1/web8/web:
    www-data 18153  0.0  0.3 332516 12564 ?        S    21:00   0:00 /usr/sbin/apache2 -k start
    web11    18279  0.0  1.3 263892 54904 ?        S    21:03   0:01 /usr/bin/php-cgi -d open_basedir=/var/www/clients/client1/web11/web
    web8     18869 12.4  1.8 271824 74228 ?        S    21:06   3:08 /usr/bin/php-cgi -d open_basedir=/var/www/clients/client1/web8/web:
    web8     18875 17.7  1.8 271808 73208 ?        S    21:06   4:27 /usr/bin/php-cgi -d open_basedir=/var/www/clients/client1/web8/web:
    web8     18878 11.1  1.8 272296 73916 ?        R    21:06   2:47 /usr/bin/php-cgi -d open_basedir=/var/www/clients/client1/web8/web:
    root     19546  0.0  0.0  10560  2196 pts/1    T    21:13   0:00 nano /home/dingle/collectl.output.log
    web9     20052  0.0  1.1 261224 47412 ?        S    21:17   0:00 /usr/bin/php-cgi -d open_basedir=/var/www/clients/client1/web9/web:
    web9     20053  0.0  1.1 260956 47264 ?        S    21:17   0:00 /usr/bin/php-cgi -d open_basedir=/var/www/clients/client1/web9/web:
    amavis   20106  0.0  0.2 192480  8524 ?        S    21:18   0:00 amavisd (virgin child)
    web29    20124  0.0  1.3 267280 56724 ?        S    21:18   0:00 /usr/bin/php-cgi -d open_basedir=/var/www/clients/client1/web29/web
    web29    20125  0.0  1.4 267272 57292 ?        S    21:18   0:00 /usr/bin/php-cgi -d open_basedir=/var/www/clients/client1/web29/web
    www-data 20174  0.0  0.3 332104 12348 ?        S    21:19   0:00 /usr/sbin/apache2 -k start
    www-data 20179  0.0  0.3 332136 12372 ?        S    21:19   0:00 /usr/sbin/apache2 -k start
    www-data 20184  0.0  0.3 332104 12324 ?        S    21:19   0:00 /usr/sbin/apache2 -k start
    www-data 20598  0.0  0.3 332504 12496 ?        S    21:21   0:00 /usr/sbin/apache2 -k start
    www-data 20599  0.0  0.3 332128 12300 ?        S    21:21   0:00 /usr/sbin/apache2 -k start
    web5     20609  1.7  1.9 276432 77288 ?        S    21:21   0:10 /usr/bin/php-cgi -d open_basedir=/var/www/clients/client1/web5/web:
    web5     20610  3.8  1.9 276444 77320 ?        S    21:21   0:22 /usr/bin/php-cgi -d open_basedir=/var/www/clients/client1/web5/web:
    nobody   20954  0.0  0.0  28184  2712 ?        S    Oct25   1:22 /usr/local/sbin/mydns -b
    web8     21141 14.2  1.7 271808 73140 ?        S    21:26   0:45 /usr/bin/php-cgi -d open_basedir=/var/www/clients/client1/web8/web:
    web8     21146  6.5  1.7 271816 73088 ?        S    21:26   0:20 /usr/bin/php-cgi -d open_basedir=/var/www/clients/client1/web8/web:
    web8     21149 23.8  1.7 271812 73088 ?        S    21:26   1:14 /usr/bin/php-cgi -d open_basedir=/var/www/clients/client1/web8/web:
    web8     21150  1.1  1.7 271812 73032 ?        S    21:26   0:03 /usr/bin/php-cgi -d open_basedir=/var/www/clients/client1/web8/web:
    web8     21151  2.2  1.7 271808 73080 ?        S    21:26   0:06 /usr/bin/php-cgi -d open_basedir=/var/www/clients/client1/web8/web:
    web8     21155 12.1  1.7 271808 73140 ?        S    21:26   0:37 /usr/bin/php-cgi -d open_basedir=/var/www/clients/client1/web8/web:
    web8     21159 16.8  1.7 271808 73128 ?        S    21:26   0:51 /usr/bin/php-cgi -d open_basedir=/var/www/clients/client1/web8/web:
    www-data 21207  0.0  0.2 332064 11988 ?        S    21:27   0:00 /usr/sbin/apache2 -k start
    www-data 21215  0.0  0.3 332112 12260 ?        S    21:27   0:00 /usr/sbin/apache2 -k start
    www-data 21216  0.0  0.2 332112 12168 ?        S    21:27   0:00 /usr/sbin/apache2 -k start
    www-data 21221  0.0  0.3 332388 12500 ?        S    21:27   0:00 /usr/sbin/apache2 -k start
    postfix  21223  0.0  0.1 106484  6112 ?        S    21:27   0:00 smtpd -n smtp -t inet -u -c -o stress 
    postfix  21226  0.0  0.0  38900  2244 ?        S    21:27   0:00 anvil -l -t unix -u -c
    web8     21238  4.1  1.9 277936 81056 ?        S    21:27   0:09 /usr/bin/php-cgi -d open_basedir=/var/www/clients/client1/web8/web:
    web8     21244  2.6  1.9 277820 80944 ?        S    21:27   0:05 /usr/bin/php-cgi -d open_basedir=/var/www/clients/client1/web8/web:
    amavis   21362  0.0  0.2 192480  8380 ?        S    21:29   0:00 amavisd (virgin child)
    postfix  21546  0.0  0.0  38900  2224 ?        S    21:30   0:00 showq -t unix -u -c
    www-data 21797  0.0  0.2 331920 11788 ?        S    21:30   0:00 /usr/sbin/apache2 -k start
    www-data 21802  0.0  0.2 331788 11768 ?        S    21:30   0:00 /usr/sbin/apache2 -k start
    root     21849  0.0  0.0  16016  1116 pts/1    R+   21:31   0:00 ps aux
    root     22363  0.0  0.0      0     0 ?        S    Oct25   0:09 [pdflush]
    
     
  20. Turbanator

    Turbanator Member HowtoForge Supporter

    Do you have any recent fcgi processes die in your error.log?

    I wonder if the processes are spawning because they are waiting on mysql. I still have concern over your long queries being over 10 sec. It tells me that those are probably mysql writing a temp table out to disk...and disk tables for a busy server like yours are deadly. (still just a theory...but it would be interesting to see your disk access read/write during those maxcpu times.) This is related to if your website has a lot of fulltext searches.

    Are these custom queries or a packaged script?

    I would also increase your key_buffer_size to at least 128M and see how it goes. I keep mine the same rough size as my index files. You can reduce it once you solve some problems.
     

Share This Page