High cpu on server

Hi,

We have a server free of load and sometimes cpu is very high. There's no process running on cpu but load grows a lot.

Intel(R) Xeon(R) CPU E5-1650 0 @ 3.20GHz, 12 cores

What's the reason load average: 36.40?

Top:

top - 06:26:01 up 2 days, 6:05, 0 users, load average: 36.40, 15.74, 5.80
Tasks: 220 total, 1 running, 219 sleeping, 0 stopped, 0 zombie
Cpu(s): 0.1%us, 0.0%sy, 0.1%ni, 99.8%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Mem: 66005212k total, 1815988k used, 64189224k free, 148904k buffers
Swap: 525308k total, 0k used, 525308k free, 488124k cached

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
1 root 20 0 8400 796 668 S 0 0.0 0:02.29 init
2 root 20 0 0 0 0 S 0 0.0 0:00.00 kthreadd
3 root 20 0 0 0 0 S 0 0.0 0:00.12 ksoftirqd/0
5 root 0 -20 0 0 0 S 0 0.0 0:00.00 kworker/0:0H
7 root 0 -20 0 0 0 S 0 0.0 0:00.00 kworker/u:0H
8 root RT 0 0 0 0 S 0 0.0 0:00.03 migration/0
9 root 20 0 0 0 0 S 0 0.0 0:00.00 rcu_bh
10 root 20 0 0 0 0 S 0 0.0 0:06.61 rcu_sched
11 root 20 0 0 0 0 S 0 0.0 0:00.05 ksoftirqd/1
12 root RT 0 0 0 0 S 0 0.0 0:00.03 migration/1
14 root 0 -20 0 0 0 S 0 0.0 0:00.00 kworker/1:0H
15 root 20 0 0 0 0 S 0 0.0 0:00.04 ksoftirqd/2
16 root RT 0 0 0 0 S 0 0.0 0:00.03 migration/2
17 root 20 0 0 0 0 S 0 0.0 0:00.00 kworker/2:0
18 root 0 -20 0 0 0 S 0 0.0 0:00.00 kworker/2:0H
19 root 20 0 0 0 0 S 0 0.0 0:00.03 ksoftirqd/3
20 root RT 0 0 0 0 S 0 0.0 0:00.03 migration/3
21 root 20 0 0 0 0 S 0 0.0 0:00.00 kworker/3:0
22 root 0 -20 0 0 0 S 0 0.0 0:00.00 kworker/3:0H
23 root 20 0 0 0 0 S 0 0.0 0:00.04 ksoftirqd/4
24 root RT 0 0 0 0 S 0 0.0 0:00.03 migration/4
25 root 20 0 0 0 0 S 0 0.0 0:00.00 kworker/4:0
26 root 0 -20 0 0 0 S 0 0.0 0:00.00 kworker/4:0H
27 root 20 0 0 0 0 S 0 0.0 0:00.02 ksoftirqd/5
28 root RT 0 0 0 0 S 0 0.0 0:00.03 migration/5
29 root 20 0 0 0 0 S 0 0.0 0:00.00 kworker/5:0
30 root 0 -20 0 0 0 S 0 0.0 0:00.00 kworker/5:0H
31 root 20 0 0 0 0 S 0 0.0 0:00.02 ksoftirqd/6
32 root RT 0 0 0 0 S 0 0.0 0:00.20 migration/6
33 root 20 0 0 0 0 S 0 0.0 0:00.00 kworker/6:0
34 root 0 -20 0 0 0 S 0 0.0 0:00.00 kworker/6:0H
35 root 20 0 0 0 0 S 0 0.0 0:00.02 ksoftirqd/7
36 root RT 0 0 0 0 S 0 0.0 0:00.24 migration/7
37 root 20 0 0 0 0 S 0 0.0 0:00.00 kworker/7:0
38 root 0 -20 0 0 0 S 0 0.0 0:00.00 kworker/7:0H
39 root 20 0 0 0 0 S 0 0.0 0:00.02 ksoftirqd/8
40 root RT 0 0 0 0 S 0 0.0 0:00.20 migration/8
41 root 20 0 0 0 0 S 0 0.0 0:00.00 kworker/8:0
42 root 0 -20 0 0 0 S 0 0.0 0:00.00 kworker/8:0H
43 root 20 0 0 0 0 S 0 0.0 0:00.02 ksoftirqd/9
44 root RT 0 0 0 0 S 0 0.0 0:00.18 migration/9
45 root 20 0 0 0 0 S 0 0.0 0:00.00 kworker/9:0
46 root 0 -20 0 0 0 S 0 0.0 0:00.00 kworker/9:0H
47 root 20 0 0 0 0 S 0 0.0 0:00.02 ksoftirqd/10
48 root RT 0 0 0 0 S 0 0.0 0:00.18 migration/10
49 root 20 0 0 0 0 S 0 0.0 0:00.00 kworker/10:0
50 root 0 -20 0 0 0 S 0 0.0 0:00.00 kworker/10:0H
51 root 20 0 0 0 0 S 0 0.0 0:00.03 ksoftirqd/11
52 root RT 0 0 0 0 S 0 0.0 0:00.19 migration/11
53 root 20 0 0 0 0 S 0 0.0 0:00.00 kworker/11:0
54 root 0 -20 0 0 0 S 0 0.0 0:00.00 kworker/11:0H
55 root 0 -20 0 0 0 S 0 0.0 0:00.00 cpuset
56 root 0 -20 0 0 0 S 0 0.0 0:00.00 khelper
57 root 20 0 0 0 0 S 0 0.0 0:00.00 kdevtmpfs
58 root 0 -20 0 0 0 S 0 0.0 0:00.00 netns
544 root 20 0 0 0 0 S 0 0.0 0:00.01 bdi-default
545 root 0 -20 0 0 0 S 0 0.0 0:00.00 kintegrityd
547 root 0 -20 0 0 0 S 0 0.0 0:00.00 kblockd
659 root 20 0 0 0 0 S 0 0.0 0:00.00 kworker/0:1
757 root 0 -20 0 0 0 S 0 0.0 0:00.00 ata_sff
767 root 20 0 0 0 0 S 0 0.0 0:00.00 khubd
777 root 0 -20 0 0 0 S 0 0.0 0:00.00 md
889 root 0 -20 0 0 0 S 0 0.0 0:00.00 rpciod
890 root 20 0 0 0 0 S 0 0.0 0:01.51 kworker/3:1
891 root 0 -20 0 0 0 S 0 0.0 0:00.00 kvm-irqfd-clean
1023 root 20 0 0 0 0 S 0 0.0 0:00.00 kswapd0
1024 root 25 5 0 0 0 S 0 0.0 0:00.00 ksmd
1167 root 20 0 0 0 0 S 0 0.0 0:00.00 fsnotify_mark
1208 root 0 -20 0 0 0 S 0 0.0 0:00.00 nfsiod
1220 root 0 -20 0 0 0 S 0 0.0 0:00.00 cifsiod
1241 root 20 0 0 0 0 S 0 0.0 0:00.00 jfsIO
1242 root 20 0 0 0 0 S 0 0.0 0:00.00 jfsCommit
1243 root 20 0 0 0 0 S 0 0.0 0:00.00 jfsCommit
1244 root 20 0 0 0 0 S 0 0.0 0:00.00 jfsCommit
1245 root 20 0 0 0 0 S 0 0.0 0:00.00 jfsCommit
1246 root 20 0 0 0 0 S 0 0.0 0:00.00 jfsCommit
1247 root 20 0 0 0 0 S 0 0.0 0:00.00 jfsCommit
1248 root 20 0 0 0 0 S 0 0.0 0:00.00 jfsCommit
1249 root 20 0 0 0 0 S 0 0.0 0:00.00 jfsCommit
1250 root 20 0 0 0 0 S 0 0.0 0:00.00 jfsCommit
1251 root 20 0 0 0 0 S 0 0.0 0:00.00 jfsCommit
1252 root 20 0 0 0 0 S 0 0.0 0:00.00 jfsCommit
1253 root 20 0 0 0 0 S 0 0.0 0:00.00 jfsCommit
1254 root 20 0 0 0 0 S 0 0.0 0:00.00 jfsSync
1268 root 0 -20 0 0 0 S 0 0.0 0:00.00 xfsalloc
1270 root 0 -20 0 0 0 S 0 0.0 0:00.00 xfs_mru_cache
1273 root 0 -20 0 0 0 S 0 0.0 0:00.00 xfslogd
1280 root 0 -20 0 0 0 S 0 0.0 0:00.00 ocfs2_wq
1283 root 0 -20 0 0 0 S 0 0.0 0:00.00 user_dlm
1302 root 0 -20 0 0 0 S 0 0.0 0:00.00 glock_workqueue
1303 root 0 -20 0 0 0 S 0 0.0 0:00.00 delete_workqueu
1311 root 0 -20 0 0 0 S 0 0.0 0:00.00 gfs_recovery
1316 root 0 -20 0 0 0 S 0 0.0 0:00.00 crypto
1348 root 0 -20 0 0 0 S 0 0.0 0:00.00 kthrotld
1775 postfix 20 0 52896 3604 2684 S 0 0.0 0:00.00 cleanup
1783 postfix 20 0 52684 3456 2576 S 0 0.0 0:00.00 trivial-rewrite
1802 postfix 20 0 56884 2988 2308 S 0 0.0 0:00.00 smtp
1809 postfix 20 0 57188 4416 3328 S 0 0.0 0:00.00 smtpd
1810 postfix 20 0 53060 3784 2656 S 0 0.0 0:00.00 local
1812 root 20 0 33308 1056 780 S 0 0.0 0:00.00 cron
1816 root 20 0 47208 20m 2212 S 0 0.0 0:00.17 monitor.pl
1883 root 20 0 13180 1240 1048 S 0 0.0 0:00.00 sh
1884 root 20 0 19112 1368 936 R 0 0.0 0:00.00 top
1927 root 20 0 0 0 0 S 0 0.0 0:01.52 kworker/1:1
1928 root 20 0 0 0 0 S 0 0.0 0:01.47 kworker/2:1
1932 root 20 0 0 0 0 S 0 0.0 0:02.66 kworker/4:1
1966 root 20 0 0 0 0 S 0 0.0 0:01.47 kworker/5:1
2020 root 0 -20 0 0 0 S 0 0.0 0:00.00 drbd-reissue
2051 root 0 -20 0 0 0 S 0 0.0 0:00.00 iscsi_eh
2067 root 20 0 0 0 0 S 0 0.0 0:00.00 scsi_eh_0
2068 root 20 0 0 0 0 S 0 0.0 0:01.23 kworker/10:1
2077 root 20 0 0 0 0 S 0 0.0 0:01.26 kworker/8:1
2097 root 20 0 0 0 0 S 0 0.0 0:01.28 kworker/7:1
2110 root 0 -20 0 0 0 S 0 0.0 0:00.00 kworker/10:1H
2116 root 20 0 0 0 0 S 0 0.0 0:00.00 scsi_eh_1
2119 root 20 0 0 0 0 S 0 0.0 0:00.00 scsi_eh_2
2122 root 20 0 0 0 0 S 0 0.0 0:00.00 scsi_eh_3
2125 root 20 0 0 0 0 S 0 0.0 0:00.00 scsi_eh_4
2128 root 20 0 0 0 0 S 0 0.0 0:00.00 scsi_eh_5
2131 root 20 0 0 0 0 S 0 0.0 0:00.00 scsi_eh_6
2135 root 20 0 0 0 0 S 0 0.0 0:01.93 kworker/u:5
2136 root 20 0 0 0 0 S 0 0.0 0:03.02 kworker/u:6
2153 root 0 -20 0 0 0 S 0 0.0 0:00.00 bond0
2198 root 20 0 0 0 0 S 0 0.0 0:01.26 kworker/11:1
2243 root 0 -20 0 0 0 S 0 0.0 0:00.00 mlx4
2261 root 0 -20 0 0 0 S 0 0.0 0:00.00 vfio-irqfd-clea
2265 root 20 0 0 0 0 S 0 0.0 0:01.35 kworker/9:1
2291 root 0 -20 0 0 0 S 0 0.0 0:00.00 kpsmoused
2293 root 20 0 0 0 0 S 0 0.0 0:00.00 kworker/1:2
2319 root 0 -20 0 0 0 S 0 0.0 0:00.00 dm_bufio_cache
2322 root 0 -20 0 0 0 S 0 0.0 0:00.00 kdelayd
2325 root 0 -20 0 0 0 S 0 0.0 0:00.00 kmpathd
2326 root 0 -20 0 0 0 S 0 0.0 0:00.00 kmpath_handlerd
2332 root 0 -20 0 0 0 S 0 0.0 0:00.00 edac-poller
2421 root 0 -20 0 0 0 S 0 0.0 0:00.00 deferwq
2422 root 20 0 0 0 0 S 0 0.0 0:01.23 kworker/6:1
2436 root 20 0 0 0 0 S 0 0.0 0:01.07 jbd2/sda1-8
2437 root 0 -20 0 0 0 S 0 0.0 0:00.00 ext4-dio-unwrit
2473 root 16 -4 16924 956 388 S 0 0.0 0:00.04 udevd
2822 root 20 0 0 0 0 S 0 0.0 0:15.34 kworker/0:2
2894 root 20 0 0 0 0 S 0 0.0 0:01.52 jbd2/sda2-8
2895 root 0 -20 0 0 0 S 0 0.0 0:00.00 ext4-dio-unwrit
3562 root 20 0 0 0 0 S 0 0.0 0:02.81 flush-8:0
3563 root 0 -20 0 0 0 S 0 0.0 0:00.53 kworker/0:1H
3636 root 20 0 180m 1520 1088 S 0 0.0 0:02.57 rsyslogd
3726 root 20 0 12728 388 248 S 0 0.0 0:00.01 mdadm
3735 daemon 20 0 18764 376 220 S 0 0.0 0:00.00 atd
3778 root 20 0 9112 608 432 S 0 0.0 0:20.17 irqbalance
3848 messageb 20 0 23312 436 204 S 0 0.0 0:00.00 dbus-daemon
3887 nobody 20 0 189m 1152 848 S 0 0.0 0:03.81 memcached
3919 bind 20 0 369m 109m 2476 S 0 0.2 0:01.98 named
3955 root 20 0 49224 1136 588 S 0 0.0 0:00.09 sshd
4169 root 20 0 9184 1384 1132 S 0 0.0 0:00.00 mysqld_safe
4298 root 20 0 22928 1000 748 S 0 0.0 0:01.11 cron
4355 mysql 20 0 303m 28m 7396 S 0 0.0 0:31.53 mysqld
4356 root 20 0 3904 596 500 S 0 0.0 0:00.00 logger
4364 root 20 0 143m 7888 2252 S 0 0.0 1:10.18 fail2ban-server
4365 root 0 -20 0 0 0 S 0 0.0 0:00.00 kworker/5:1H
4404 amavis 20 0 203m 87m 3320 S 0 0.1 0:01.69 amavisd-new
4453 root 20 0 39228 984 548 S 0 0.0 0:04.23 dovecot
4455 root 20 0 75556 4032 2596 S 0 0.0 0:01.51 dovecot-auth
4477 root 20 0 36076 1840 1244 S 0 0.0 0:00.09 pure-ftpd-mysql
4585 root 20 0 90868 21m 1872 S 0 0.0 0:02.18 miniserv.pl
4612 clamav 20 0 286m 223m 5488 S 0 0.3 0:24.28 clamd
4762 clamav 20 0 42772 2096 1192 S 0 0.0 1:02.24 freshclam
4801 root 20 0 37224 2416 1896 S 0 0.0 0:01.02 master
4818 postfix 20 0 52580 2968 2260 S 0 0.0 0:00.05 qmgr
4909 root 20 0 5980 584 492 S 0 0.0 0:00.00 getty
4910 root 20 0 5980 584 492 S 0 0.0 0:00.00 getty
4911 root 20 0 5980 584 492 S 0 0.0 0:00.00 getty
4912 root 20 0 5980 584 492 S 0 0.0 0:00.00 getty
4913 root 20 0 5980 584 492 S 0 0.0 0:00.00 getty
4914 root 20 0 5980 584 492 S 0 0.0 0:00.00 getty
4918 root 18 -2 16920 864 300 S 0 0.0 0:00.00 udevd
4919 root 18 -2 16920 860 296 S 0 0.0 0:00.00 udevd
5465 amavis 20 0 206m 88m 4048 S 0 0.1 0:00.34 amavisd-new
5466 amavis 20 0 206m 88m 4048 S 0 0.1 0:00.20 amavisd-new
5494 root 0 -20 0 0 0 S 0 0.0 0:00.00 kworker/2:1H
5649 root 0 -20 0 0 0 S 0 0.0 0:00.00 kworker/1:1H
6120 root 0 -20 0 0 0 S 0 0.0 0:00.00 kworker/4:1H
6158 postfix 20 0 41832 3236 2284 S 0 0.0 0:00.15 tlsmgr
7104 root 20 0 300m 8032 2184 S 0 0.0 0:03.83 php5-fpm
7105 web1 20 0 299m 6560 712 S 0 0.0 0:00.00 php5-fpm
7106 web1 20 0 299m 6560 712 S 0 0.0 0:00.00 php5-fpm
7107 ispapps 20 0 299m 6560 712 S 0 0.0 0:00.00 php5-fpm
7108 ispapps 20 0 299m 6560 712 S 0 0.0 0:00.00 php5-fpm
7109 www-data 20 0 302m 15m 7204 S 0 0.0 0:00.03 php5-fpm
7110 www-data 20 0 301m 13m 5896 S 0 0.0 0:00.01 php5-fpm
7111 web29 20 0 299m 6560 712 S 0 0.0 0:00.00 php5-fpm
7112 web29 20 0 299m 6560 712 S 0 0.0 0:00.00 php5-fpm
7113 ispconfi 20 0 304m 17m 8788 S 0 0.0 0:00.19 php5-fpm
7114 ispconfi 20 0 304m 20m 12m S 0 0.0 0:00.20 php5-fpm
7117 root 20 0 78032 1740 140 S 0 0.0 0:00.00 nginx
7118 www-data 20 0 79872 5172 1764 S 0 0.0 0:07.02 nginx
7119 www-data 20 0 79912 5328 1860 S 0 0.0 0:06.04 nginx
7120 www-data 20 0 80012 5192 1836 S 0 0.0 0:05.01 nginx
7121 www-data 20 0 80316 5536 1864 S 0 0.0 0:07.15 nginx
7122 www-data 20 0 80052 5296 1784 S 0 0.0 0:07.05 nginx
7123 www-data 20 0 79912 5164 1780 S 0 0.0 0:05.72 nginx
7124 www-data 20 0 79912 5136 1740 S 0 0.0 0:07.08 nginx
7125 www-data 20 0 79912 5160 1780 S 0 0.0 0:06.09 nginx
7126 www-data 20 0 80088 5276 1748 S 0 0.0 0:07.39 nginx
7346 root 0 -20 0 0 0 S 0 0.0 0:00.00 kworker/8:1H
7735 postfix 20 0 39288 2404 1868 S 0 0.0 0:00.00 pickup
8967 root 0 -20 0 0 0 S 0 0.0 0:00.00 kworker/9:1H
10151 root 0 -20 0 0 0 S 0 0.0 0:00.00 kworker/3:1H
13169 root 0 -20 0 0 0 S 0 0.0 0:00.00 kworker/7:1H
13496 dovecot 20 0 36960 2456 1928 S 0 0.0 0:00.00 pop3-login
13497 dovecot 20 0 36972 2452 1924 S 0 0.0 0:00.00 imap-login
14594 root 0 -20 0 0 0 S 0 0.0 0:00.00 kworker/11:1H
16563 dovecot 20 0 36960 2456 1928 S 0 0.0 0:00.00 pop3-login
16564 dovecot 20 0 36972 2452 1924 S 0 0.0 0:00.00 imap-login
17322 postfix 20 0 57744 4772 3240 S 0 0.0 0:00.00 smtpd
17324 postfix 20 0 52724 3228 2488 S 0 0.0 0:00.00 proxymap
17333 postfix 20 0 39288 2308 1816 S 0 0.0 0:00.00 showq
17532 dovecot 20 0 36960 2456 1928 S 0 0.0 0:00.00 pop3-login
17533 dovecot 20 0 36972 2452 1924 S 0 0.0 0:00.00 imap-login
19430 root 20 0 43648 8360 1836 S 0 0.0 0:05.87 munin-node
24430 root 20 0 75076 3624 2748 S 0 0.0 0:00.00 dovecot-auth
25819 root 0 -20 0 0 0 S 0 0.0 0:00.00 kworker/6:1H

Top:

Thanks,
Servidores Dedicados Administrados

 

what the hell u want to say to us, it is very hard to understand that process here

 

Sever load

It looks like you are facing intermittent load spike on your server.
It very hard to find out exact reason from the top result you pasted. It looks like a detailed investigation on the server will help to fix the problem.

 

Take a look at "iotop", "iftop", "netstat -ntap" while such load peaks occur.
Maybe you find something special in there.
You may have to install iotop and iftop as they are not installed by default.

 

Sever load

To know more about the problem, the results of iftop,iotop is good. Also check any cronjobs are sechduled to run at these intervals.