Ubuntu 11.1 breaks down

Hi guys,

I've problems with my webserver which breaks down a couple of time a month (somtimes more than a couple of times!) and this in irregular intervals.

This morning I had again one of these crashes where my server is not reachable at all. All I can do is restart it by amazon management console (yes it's an EC2 instance).

As you can see in the attachment, there was a spike at about 03:07 and crashed at 03:09. And I have no idea why, because at this time there was no traffic no my server. It just can be a system failure...

Here are a couple of configs (at the time of crashing).

syslog:
May 9 03:04:01 web pure-ftpd: ([email protected]) [INFO] New connection from 127.0.0.1
May 9 03:04:01 web pure-ftpd: ([email protected]) [INFO] Logout.
May 9 03:05:01 web CRON[3219]: (root) CMD (/usr/local/ispconfig/server/server.sh > /dev/null 2>> /var/log/ispconfig/cron.log)
May 9 03:05:01 web pure-ftpd: ([email protected]) [INFO] New connection from 127.0.0.1
May 9 03:05:01 web pure-ftpd: ([email protected]) [INFO] Logout.
May 9 03:05:02 web pure-ftpd: ([email protected]) [INFO] New connection from 127.0.0.1
May 9 03:05:02 web pure-ftpd: ([email protected]) [INFO] Logout.
May 9 03:05:02 web sendmail[3271]: NOQUEUE: SYSERR(root): /etc/mail/sendmail.cf: line 101: fileclass: cannot open '/etc/mail/local-host-names': Group writable directory
May 9 03:06:01 web CRON[3293]: (root) CMD (/usr/local/ispconfig/server/server.sh > /dev/null 2>> /var/log/ispconfig/cron.log)
May 9 03:06:01 web pure-ftpd: ([email protected]) [INFO] New connection from 127.0.0.1
May 9 03:06:01 web pure-ftpd: ([email protected]) [INFO] Logout.
May 9 03:07:01 web CRON[3304]: (root) CMD (/usr/local/ispconfig/server/server.sh > /dev/null 2>> /var/log/ispconfig/cron.log)
May 9 03:07:01 web pure-ftpd: ([email protected]) [INFO] New connection from 127.0.0.1
May 9 03:07:01 web pure-ftpd: ([email protected]) [INFO] Logout.
May 9 03:08:01 web CRON[3344]: (root) CMD (/usr/local/ispconfig/server/server.sh > /dev/null 2>> /var/log/ispconfig/cron.log)
May 9 03:08:01 web pure-ftpd: ([email protected]) [INFO] New connection from 127.0.0.1
May 9 03:08:01 web pure-ftpd: ([email protected]) [INFO] Logout.
May 9 03:09:01 web CRON[3414]: (root) CMD ( [ -x /usr/lib/php5/maxlifetime ] && [ -d /var/lib/php5 ] && find /var/lib/php5/ -depth -mindepth 1 -maxdepth 1 -type f -cmin +$(/usr/lib/php5/maxlifetime) ! -execdir fuser -s {} 2>/dev/null \; -delete)
May 9 03:09:01 web CRON[3415]: (root) CMD (/usr/local/ispconfig/server/server.sh > /dev/null 2>> /var/log/ispconfig/cron.log)
May 9 03:09:03 web pure-ftpd: ([email protected]) [INFO] New connection from 127.0.0.1
May 9 03:09:03 web pure-ftpd: ([email protected]) [INFO] Logout.
May 9 04:36:13 web kernel: imklog 5.8.1, log source = /proc/kmsg started.
May 9 04:36:13 web rsyslogd: [origin software="rsyslogd" swVersion="5.8.1" x-pid="594" x-info="http://www.rsyslog.com"] start


Apache access.log at this time just:
127.0.0.1 - - [09/May/2012:03:09:03 +0000] "GET / HTTP/1.0" 200 452 "-" "-"

Auth.log normal as well:
May 9 03:09:03 web CRON[3412]: pam_unix(cron:session): session closed for user root

Mail.err is what i have every 5min (why that by the way?):
May 9 02:55:02 web sendmail[2912]: NOQUEUE: SYSERR(root): /etc/mail/sendmail.cf: line 101: fileclass: cannot open '/etc/mail/local-host-names': Group writable directory
May 9 03:00:12 web sendmail[3125]: NOQUEUE: SYSERR(root): /etc/mail/sendmail.cf: line 101: fileclass: cannot open '/etc/mail/local-host-names': Group writable directory
May 9 03:05:02 web sendmail[3271]: NOQUEUE: SYSERR(root): /etc/mail/sendmail.cf: line 101: fileclass: cannot open '/etc/mail/local-host-names': Group writable directory

Mysql.err ok as well.

Here's the ispconfig/cron.log:
root@web:/etc# grep -rl "#30"
setquota: Not setting block grace time on /dev/disk/by-label/cloudimg-rootfs because softlimit is not exceeded.
setquota: Not setting inode grace time on /dev/disk/by-label/cloudimg-rootfs because softlimit is not exceeded.
Warning: Truncating oversized referrer field
Warning: Truncating oversized referrer field
Warning: Truncating oversized referrer field
Warning: Truncating oversized referrer field
Warning: Truncating oversized referrer field
setquota: Not setting block grace time on /dev/disk/by-label/cloudimg-rootfs because softlimit is not exceeded.
setquota: Not setting inode grace time on /dev/disk/by-label/cloudimg-rootfs because softlimit is not exceeded.
Warning: Truncating oversized referrer field
Warning: Truncating oversized referrer field
Warning: Truncating oversized referrer field
Warning: Truncating oversized referrer field
Warning: Truncating oversized referrer field
Warning: Truncating oversized referrer field
Warning: Truncating oversized referrer field
Warning: Truncating oversized referrer field
Warning: Truncating oversized referrer field
Warning: Truncating oversized referrer field
Warning: Truncating oversized referrer field
Warning: Truncating oversized referrer field
Warning: Truncating oversized referrer field
Warning: Truncating oversized date field
Skipping bad record (47591)
Warning: Truncating oversized date field
Skipping bad record (84399)

Do you have any ideas why the heck I have all these crashes?
I also took some snapshots of my AMI (os image) and started a new instance. -> same problem.

Thank you!
Marc

 

Please move this thread to ISPConfig 3 category. Sorry for that!

 

I suggest you install munin. This should help you figure out why this happens (maybe disk I/O or something like that).