System do not boot up after update

Hello!

After ISPConfig update to 3.0.5.4p1 and kernel update for 12.04.04LTS to 3.2.0-63-generic
system do to boot up.
Problem was in fstab. ISPconfig add lines like:

Code:

/var/log/ispconfig/httpd/domain.com /var/www/clients/client23/web11/log    none    bind,nobootwait    0 0

I can boot system only in recovery mode, after fsck.

I comment all this lines and now system boot up without problem.

I try to boot from old kernels but still has problem.

Some one have same problem?

Falko and Till can you explain this situation?

Thank you.

 

I do not see anything in syslog, system do not write log to syslog only when it will be mounted and system boot up.

When system do not boot up on screen I see:

Code:

stopping read required files in advance
stopping read required files in advance
stopping read required files in advance
stopping read required files in advance

Here is fstab:

Code:

# /etc/fstab: static file system information.
#
# Use 'blkid' to print the universally unique identifier for a
# device; this may be used with UUID= as a more robust way to name devices
# that works even if disks are added and removed. See fstab(5).
#
# <file system> <mount point>   <type>  <options>       <dump>  <pass>
proc            /proc           proc    nodev,noexec,nosuid 0       0
# / was on /dev/sda5 during installation
UUID=cd08a4ef-f8d6-4f02-b534-fe31ccc073e7 /               ext4    errors=remount-ro,usrjquota=quota.user,grpjquota=quota.group,jqfmt=vfsv0 0       1
# /boot was on /dev/sda1 during installation
UUID=53e0307e-1fbf-43e3-8fb2-7949af72c5cd /boot           ext4    defaults        0       2
# /tmp was on /dev/sda7 during installation
UUID=04ea320e-34d3-4ff7-8641-9a73cc7996b0 /tmp            ext2    nodev,nosuid    0       2
# /var was on /dev/sda6 during installation
UUID=93ad8100-c99a-4bea-9a1b-c6cebacc45f1 /var            ext4    nodev,nosuid,usrjquota=quota.user,grpjquota=quota.group,jqfmt=vfsv0    0       2
# swap was on /dev/sda8 during installation
UUID=0dbc594b-153a-47ce-a268-6e308d3af64d none            swap    sw              0       0
#/var/log/ispconfig/httpd/com1.com /var/www/clients/client23/web11/log    none    bind,nobootwait    0 0
#/var/log/ispconfig/httpd/com2.com /var/www/clients/client2/web94/log    none    bind,nobootwait    0 0
#/var/log/ispconfig/httpd/com3.com /var/www/clients/client24/web89/log    none    bind,nobootwait    0 0
#/var/log/ispconfig/httpd/com4.com /var/www/clients/client17/web62/log    none    bind,nobootwait    0 0
#/var/log/ispcon

I have mounted usb ntfs drive for backups. But I unplug it and still has problem.

 

Any ideas Till?

 

I checked. All clients have log directory and in /var/log/ispconfig/httpd/ exist all webs.
But some of them is disabled in ISPConfig (Locked and Canceled).
It is production server and I can not restart to check may be problem in Locked users in fstab.

 

I don't know what to do now. All clients do not have log files now. It is not mounted to his log directories.
If I enable it in fstab, I dont know what will be after restart.

 

Till,
It is possible to remount it manually? When system is on. Because all clients do not see their log files.

Thank you.

 

I have the same problem as this, on 2 separate Ubuntu 12.04 servers, anyone worked out what exactly causes this, at the moment I have all the lines for mounting log directories in fstab commented out.

I can survive without the log files mounted, is there any way I can stop ispconfig from changing the fstab file apart from the obvious removing write access?

 

I'm running into the same issue with 3 servers running 12.04. The strange thing is that it seems to only be a problem on some sites.
Short term fix on my production servers was to boot into recovery mode, start networking, then resume boot. All folders from the fstab file were mounted correctly.
For the one that wasn't in production, I did some experimentation and found that 2 of 6 fstab entries were causing the system to hang on boot. With either of the two enabled, the system would hang on boot some of the time, but sometimes would boot without an issue. Typically, it booted once with the fstab entries enabled but would fail the second time around. When these two were both commented out, I could reboot 5-6 times in a row...

 

Thanks till. Apologies if I'm being dense here, but I don't see "_netboot" in fstab. I do, however, see "_netdev." Is that what you meant? I tried removing these and it seems to have addressed the problem, but wanted to double check.

Similarly, I don't see anything that fits the description under System > server config > web besides Network Filesystem, which was already disabled in my case. Is there a setting that will keep "_netdev" from being included in the fstab entry when new sites are created?

Thanks much!

 

I have had this problem with Ubuntu 12.04 couple of years, even currently I'm running Ubuntu 12.04.5 LTS, updated to kernel 3.13.0-65 and with all available patches apt can find. Of course this is a big problem with production server with couple of hundreds of sites inside, but I have learned to live with it and do all reboot requiring improvements at nighttime (which, of course is not optimal for me).

Previously I also have been using "recovery mode with network" trick (boot to recovery mode, choose network which mounts some parts of the filesystem and continue normal booting). But this does not work always, sometimes I have to go to recovery mode console, edit /etc/fstab and comment out swap row and do all sorts of magic tricks before finally I get the server to boot.

The problem is definately with _netdev options inside fstab. Nowadays I usually remove _netdev options with sed command before booting and the server boots flawlesly. This is not optimal solution because if the server reboots because of some sort of technical problem, it won't go online until someone goes to recovery console. One solutions might be to add the sed command to crontab and get the cron to remove netdev's like every hour or so.

The sed command I use is:
sed -i 's|,_netdev||g' /etc/fstab

I have restored a full backup of the server to insulated laboratory and upgraded lab-server to Ubuntu 14.04. The server boots just fine with 14.04 even there are _netdev options in fstab. So this problem seems to be with Ubuntu's earlier versions.

I hope this post helps some other ISPConfig users with this same problem. As an sysadmin, it's not fun to boot production server (with paying customers and SLA's) and realize that the server is not going to go back online and there seem to be no any reason for the problem. What a nightmare for some junior level sysadmin.

 

The only option that I have noticed under web tab is "Network Filesystem", but that is already untagged. Is it this the option you are referring?

 

Ok, that option is and has been always unchecked but I still get the _netdev option to new sites.. I also tried to check, save, uncheck and save and create a new site. But still I can see _netdev at the end of the new row on fstab?!? A bug? I'm running the latest p8.