Hi everyone, for over two months, I am facing unexplained load increases on my server. Take a look at the screenshot. Cacti-Screenshot At about 2.00, no login is possible, and the remote service board fails to get a video signal as well, because no input is possible. Well, is there a tool like Microsoft's perfmon to caputre load, and what application causes it? Maybe I can isolate the cause and fix the problem. Thank you in advance Kind Regards ZeroEnna
Concerning the crontab, this is output of Code: for user in $(cut -f1 -d: /etc/passwd); do echo $user; crontab -u $user -l; done Code: root 0 1 * * * /var/www/kibo.fm/web/sctrans/sc_trans_040/sc_trans stop 45 * * * * MAILTO=""; lynx --dump "http://kibo.fm/meinkibo/bot.php?action=log&password=d6iKpB$" 00 01 * * 0 /root/logs_backup.sh 00 02 * * * rsync -rltDv -e ssh /backup/ [email protected]:/users/hikarimedia/ 00 03 * * 0 /root/spamtrain.sh 30 01 * * * /root/mysqlback.sh 00 03 * * 6 /root/reboot.sh 23 4 */2 * * /usr/bin/sa-update --no-gpg &> /dev/null * * * * * /usr/local/ispconfig/server/server.sh > /dev/null 2>> /var/log/ispconfig/cron.log 30 00 * * * /usr/local/ispconfig/server/cron_daily.sh > /dev/null 2>> /var/log/ispconfig/cron.log daemon no crontab for daemon bin no crontab for bin sys no crontab for sys sync no crontab for sync games no crontab for games man no crontab for man lp no crontab for lp mail no crontab for mail news no crontab for news uucp no crontab for uucp proxy no crontab for proxy www-data no crontab for www-data backup no crontab for backup list no crontab for list irc no crontab for irc gnats no crontab for gnats nobody no crontab for nobody libuuid no crontab for libuuid Debian-exim no crontab for Debian-exim statd no crontab for statd zeroenna no crontab for zeroenna sshd no crontab for sshd ntp no crontab for ntp mysql no crontab for mysql postfix no crontab for postfix clamav no crontab for clamav amavis no crontab for amavis vmail no crontab for vmail getmail */5 * * * * /usr/local/bin/run-getmail.sh > /dev/null 2>> /var/log/ispconfig/cron.log ispapps no crontab for ispapps ispconfig no crontab for ispconfig web1 no crontab for web1 shoutcast no crontab for shoutcast web3 no crontab for web3 postgrey no crontab for postgrey unreal no crontab for unreal anope no crontab for anope eggdrop # m h dom mon dow command */1 * * * * /home/eggdrop/eggdrop/scripts/botchk > /dev/null & web5 no crontab for web5 web6 no crontab for web6 web7 no crontab for web7 web8 no crontab for web8 web9 # m h dom mon dow command #00 16 * * * /var/www/kibo.fm/web/sctrans/sc_trans_040/sc_trans start web11 no crontab for web11 web12 no crontab for web12 web13 no crontab for web13 web14 no crontab for web14 web15 no crontab for web15 web16 no crontab for web16 web17 no crontab for web17 web18 no crontab for web18 davfs2 no crontab for davfs2 dellroot no crontab for dellroot web19 no crontab for web19 ircservice no crontab for ircservice web21 no crontab for web21 marcel no crontab for marcel sctrans no crontab for sctrans web24 no crontab for web24 munin no crontab for munin messagebus no crontab for messagebus snmp no crontab for snmp web26 no crontab for web26 tcpdump no crontab for tcpdump web29 no crontab for web29 paranormal # Edit this file to introduce tasks to be run by cron. # # Each task to run has to be defined through a single line # indicating with different fields when the task will be run # and what command to run for the task # # To define the time you can provide concrete values for # minute (m), hour (h), day of month (dom), month (mon), # and day of week (dow) or use '*' in these fields (for 'any').# # Notice that tasks will be started based on the cron's system # daemon's notion of time and timezones. # # Output of the crontab jobs (including errors) is sent through # email to the user the crontab file belongs to (unless redirected). # # For example, you can run a backup of all your user accounts # at 5 a.m every week with: # 0 5 * * 1 tar -zcf /var/backups/home.tgz /home/ # # For more information see the manual pages of crontab(5) and cron(8) # # m h dom mon dow command 28,58 * * * * /home/paranormal/bots/psi-bot/scripts/botchk web31 no crontab for web31 minecraft no crontab for minecraft avahi no crontab for avahi
I guess this cron job is the problem: Code: 00 02 * * * rsync -rltDv -e ssh /backup/ [email protected]:/users/hikarimedia/ It could cause high disk I/O. Can you deactivate it and see if that changes anything?
Hi, unfortunately, this is my backup job, so disabling it for about 3 weeks might cause a problem. Also, the problem occured at about 3 p.m, the task finished at about 2:40 a.m., so I don't see any relations between the increasing load and the task. A few days ago I remembered that this had happened before. But in July or August (I think) there has been an update, I don't know what it was anymore, maybe the logs will tell. Anyway, after this, I had uptimes of about 60 to 70 days, when a kernel update arrived. After installing it, my uptimes crashed! So either, one of my installed packages has some problems, or something else is going wrong with it, as a patch fixed it, and another patch caused it to happen again. Regards ZeroEnna
In your first post you said it happens at 2.00h. (Is it possible that this has something to do with daylight savings time? Do you and your server have the same time?)
I'm sorry, I was misleading. I am currently (at this very moment) having issues again. It started at about noon yesterday and increased slowly to a peak of 7 and staying! Please have a look at the munin graphs (They are accessable) I have no access to SSH anymore, The connection is established, I get a liogon prompt but then the session freezes with no possibility of entering commands. Also, the Console Redirection (DELL iDRAC) is not returning a display, it stays black. The only option I have is a reboot, but I am afraid, I will never ever find out what is causing this! Is there a possibility (as FTP and HTTP are working fine) to write a script that returns the output as a Webpage (e.g. something with exec())? Regards Zero
According to http://www.hikari-media.de/monitoring/localdomain/localhost.localdomain/diskstats_iops/index.html you have some disk IO. That could be the problem, but I'm not sure. If it's possible to change the HDDs, you should try that.
