Quota with LVM + RAID1

Discussion in 'Installation/Configuration' started by DrJohn, Jan 25, 2010.

  1. DrJohn

    DrJohn Member

    I'm following the perfect Ubuntu 9.10 server [IspConfig2] tutorial. I installed quota as recommended, but after rebooting I get a lot (~1/second) of faults and stack dumps related to the quota tasks. Stopping quota:
    Code:
    $ quotaoff -avug
    causes the faults to cease.
    Quota is set up on /, which is one of several LVM volumes on a single LVM volume group, installed over a RAID1 partition. Is there a different way to set up quota in this situation?
     
  2. falko

    falko Super Moderator Howtoforge Staff

    Can you post some of the error messages?

    What's in /etc/fstab?
     
  3. DrJohn

    DrJohn Member

    Here's a representative bit from syslog:
    Code:
    Jan 24 21:19:37 M3N78EM kernel: [  770.042822] ------------[ cut here ]------------
    Jan 24 21:19:37 M3N78EM kernel: [  770.042845] WARNING: at /build/buildd/linux-2.6.31/fs/quota/dquot.c:964 dquot_claim_space+0x15d/0x170()
    Jan 24 21:19:37 M3N78EM kernel: [  770.042850] Hardware name: System Product Name
    Jan 24 21:19:37 M3N78EM kernel: [  770.042854] Modules linked in: quota_v2 quota_tree snd_hda_codec_nvhdmi snd_hda_codec_realtek snd_hda_intel snd$
    Jan 24 21:19:37 M3N78EM kernel: [  770.042928] Pid: 46, comm: pdflush Not tainted 2.6.31-17-server #54-Ubuntu
    Jan 24 21:19:37 M3N78EM kernel: [  770.042933] Call Trace:
    Jan 24 21:19:37 M3N78EM kernel: [  770.042949]  [<ffffffff8105e668>] warn_slowpath_common+0x78/0xb0
    Jan 24 21:19:37 M3N78EM kernel: [  770.042956]  [<ffffffff8105e6af>] warn_slowpath_null+0xf/0x20
    Jan 24 21:19:37 M3N78EM kernel: [  770.042961]  [<ffffffff8116ebed>] dquot_claim_space+0x15d/0x170
    Jan 24 21:19:37 M3N78EM kernel: [  770.042974]  [<ffffffff811dadb6>] ext4_mb_mark_diskspace_used+0x356/0x3a0
    Jan 24 21:19:37 M3N78EM kernel: [  770.042980]  [<ffffffff811db0a9>] ext4_mb_new_blocks+0x2a9/0x540
    Jan 24 21:19:37 M3N78EM kernel: [  770.042988]  [<ffffffff811d0a59>] ? ext4_ext_find_extent+0x2b9/0x2f0
    Jan 24 21:19:37 M3N78EM kernel: [  770.042994]  [<ffffffff811d2754>] ext4_ext_get_blocks+0x4a4/0x5b0
    Jan 24 21:19:37 M3N78EM kernel: [  770.043000]  [<ffffffff811b35b9>] ext4_get_blocks+0x1d9/0x210
    Jan 24 21:19:37 M3N78EM kernel: [  770.043005]  [<ffffffff811b3f97>] mpage_da_map_blocks+0xa7/0x370
    Jan 24 21:19:37 M3N78EM kernel: [  770.043013]  [<ffffffff811e9ace>] ? jbd2_journal_start+0xae/0x100
    Jan 24 21:19:37 M3N78EM kernel: [  770.043018]  [<ffffffff811b4503>] ext4_da_writepages+0x2a3/0x500
    Jan 24 21:19:37 M3N78EM kernel: [  770.043027]  [<ffffffff810e31b8>] do_writepages+0x28/0x50
    Jan 24 21:19:37 M3N78EM kernel: [  770.043034]  [<ffffffff8113e8ec>] writeback_single_inode+0x1bc/0x450
    Jan 24 21:19:37 M3N78EM kernel: [  770.043040]  [<ffffffff8113f208>] generic_sync_sb_inodes+0x418/0x530
    Jan 24 21:19:37 M3N78EM kernel: [  770.043045]  [<ffffffff8113f42b>] writeback_inodes+0x5b/0x100
    Jan 24 21:19:37 M3N78EM kernel: [  770.043051]  [<ffffffff810e1f4c>] wb_kupdate+0xbc/0x140
    Jan 24 21:19:37 M3N78EM kernel: [  770.043056]  [<ffffffff810e39be>] __pdflush+0x13e/0x260
    Jan 24 21:19:37 M3N78EM kernel: [  770.043062]  [<ffffffff810e3ae0>] ? pdflush+0x0/0x50
    Jan 24 21:19:37 M3N78EM kernel: [  770.043069]  [<ffffffff810e3b28>] pdflush+0x48/0x50
    Jan 24 21:19:37 M3N78EM kernel: [  770.043074]  [<ffffffff810e1e90>] ? wb_kupdate+0x0/0x140
    Jan 24 21:19:37 M3N78EM kernel: [  770.043081]  [<ffffffff810e3ae0>] ? pdflush+0x0/0x50
    Jan 24 21:19:37 M3N78EM kernel: [  770.043087]  [<ffffffff81078226>] kthread+0xa6/0xb0
    Jan 24 21:19:37 M3N78EM kernel: [  770.043095]  [<ffffffff810130aa>] child_rip+0xa/0x20
    Jan 24 21:19:37 M3N78EM kernel: [  770.043100]  [<ffffffff81078180>] ? kthread+0x0/0xb0
    Jan 24 21:19:37 M3N78EM kernel: [  770.043104]  [<ffffffff810130a0>] ? child_rip+0x0/0x20
    Jan 24 21:19:37 M3N78EM kernel: [  770.043107] ---[ end trace 7808f3b36ab85b6f ]---
    
    and here's fstab, unmodified from install except for the addition of the parameters related to quota:
    Code:
    # /etc/fstab: static file system information.
    #
    # Use 'blkid -o value -s UUID' to print the universally unique identifier
    # for a device; this may be used with UUID= as a more robust way to name
    # devices that works even if disks are added and removed. See fstab(5).
    #
    # <file system> <mount point>   <type>  <options>       <dump>  <pass>
    proc            /proc           proc    defaults        0       0
    /dev/mapper/M3N78EM_VG1-M3N78EM_V2_ROOT /               ext4    errors=remount-ro,usrjquota=aquota.user,grpjquota=aquota.group,jqfmt=vfsv0 0       1
    # /boot was on /dev/md0 during installation
    UUID=dd2394e9-c13c-45ae-b503-1f8d0a1ebc14 /boot           ext4    defaults        0       2
    /dev/mapper/M3N78EM_VG1-M3N78EM_V3_HOME /home           ext4    defaults        0       2
    /dev/mapper/M3N78EM_VG1-M3N78EM_V1 none            swap    sw              0       0
    /dev/scd0       /media/cdrom0   udf,iso9660 user,noauto,exec,utf8 0       0
    /dev/fd0        /media/floppy0  auto    rw,user,noauto,exec,utf8 0       0
    
    as always your help is much appreciated!
     
  4. falko

    falko Super Moderator Howtoforge Staff

    Hm, fstab looks ok. Not sure what to make of the error messages. Is this the default Ubuntu kernel that you are using?
     
  5. DrJohn

    DrJohn Member

    The install is straight off the 9.10 server CD followed by aptitude update / safe-upgrade:
    Code:
    ~$ uname -srvmo
    Linux 2.6.31-17-server #54-Ubuntu SMP Thu Dec 10 18:06:56 UTC 2009 x86_64 GNU/Linux
    
    apparmour was disabled / removed per the tutorial.
    /etc/mtab seems OK:
    Code:
    :/etc$ cat mtab
    /dev/mapper/M3N78EM_VG1-M3N78EM_V2_ROOT / ext4 rw,errors=remount-ro,usrjquota=aquota.user,grpjquota=aquota.group,jqfmt=vfsv0 0 0
    proc /proc proc rw 0 0
    none /sys sysfs rw,noexec,nosuid,nodev 0 0
    none /sys/fs/fuse/connections fusectl rw 0 0
    none /sys/kernel/debug debugfs rw 0 0
    none /sys/kernel/security securityfs rw 0 0
    udev /dev tmpfs rw,mode=0755 0 0
    none /dev/pts devpts rw,noexec,nosuid,gid=5,mode=0620 0 0
    none /dev/shm tmpfs rw,nosuid,nodev 0 0
    none /var/run tmpfs rw,nosuid,mode=0755 0 0
    none /var/lock tmpfs rw,noexec,nosuid,nodev 0 0
    none /lib/init/rw tmpfs rw,nosuid,mode=0755 0 0
    /dev/mapper/M3N78EM_VG1-M3N78EM_V3_HOME /home ext4 rw 0 0
    /dev/md0 /boot ext4 rw 0 0
    
    I have not explicitly set any quota limits, and the quotas for root and my username are at zero (output of 'edquota root'):
    Code:
    Disk quotas for user root (uid 0):
      Filesystem                   blocks       soft       hard     inodes     soft     hard
      /dev/mapper/M3N78EM_VG1-M3N78EM_V2_ROOT    1033732          0          0      53073        0        0
    
    The result of edquota for my username shows no disk usage, which is accurate since my home folder is currently empty:
    Code:
    Disk quotas for user username (uid 1000):
      Filesystem                   blocks       soft       hard     inodes     soft     hard
      /dev/mapper/M3N78EM_VG1-M3N78EM_V2_ROOT          0          0          0          0        0        0
    
    I notice here, however, that quota is reporting the ROOT volume and not the HOME volume where my /home folder is mounted. I'm not familiar enough with quota to know if it should report both volumes here?
    I have installed, configured, and started shorewall via aptitude by enabling in /etc/apt/sources.list:
    Code:
    deb http://archive.canonical.com/ubuntu karmic partner
    deb-src http://archive.canonical.com/ubuntu karmic partner
    
    so that the only uncommented sources here are the backports. I only mention it because this was the only off-tutorial part (yet).

    BTW, this is a test install on a spare for trying out 9.10 prior to replacing the old server h/w . In a few days I will receive the new server hardware, and I plan to do the 'real' install over the weekend. Perhaps there's something I can do along the way to help discover what happened here.
     
  6. HARMANE

    HARMANE New Member

    Any resolution to this? I'm seeing the same.

    Code:
    Feb 8 21:44:52 soapbox kernel: [36770.130681] [] ? child_rip+0x0/0x20
    Feb 8 21:44:52 soapbox kernel: [36770.130683] ---[ end trace 807496c710666e46 ]---
    Feb 8 21:44:52 soapbox kernel: [36770.130686] ------------[ cut here ]------------
    Feb 8 21:44:52 soapbox kernel: [36770.130690] WARNING: at /build/buildd/linux-2.6.31/fs/quota/dquot.c:964 dquot_claim_space+0x15d/0x170()
    Feb 8 21:44:52 soapbox kernel: [36770.130693] Hardware name: eserver xSeries 236 -[884105U]-
    Feb 8 21:44:52 soapbox kernel: [36770.130695] Modules linked in: xt_multiport xt_tcpudp xt_state ipt_LOG nf_conntrack_ftp iptable_mangle iptable_nat nf_nat nf_conntrack_ipv4 nf_conntrack nf_defrag_ipv4 osst st quota_v2 quota_tree iptable_filter ip_tables radeon x_tables ttm drm i2c_algo_bit ppdev psmouse shpchp e752x_edac parport_pc lp serio_raw edac_core parport raid10 raid456 usbhid raid6_pq async_xor async_memcpy async_tx xor raid1 raid0 aic79xx ips scsi_transport_spi multipath floppy tg3 linear
    Feb 8 21:44:52 soapbox kernel: [36770.130743] Pid: 46, comm: pdflush Tainted: G W 2.6.31-19-server #56-Ubuntu
    Feb 8 21:44:52 soapbox kernel: [36770.130745] Call Trace:
    Feb 8 21:44:52 soapbox kernel: [36770.130750] [] warn_slowpath_common+0x78/0xb0
    Feb 8 21:44:52 soapbox kernel: [36770.130755] [] warn_slowpath_null+0xf/0x20
    Feb 8 21:44:52 soapbox kernel: [36770.130760] [] dquot_claim_space+0x15d/0x170
    Feb 8 21:44:52 soapbox kernel: [36770.130764] [] ext4_mb_mark_diskspace_used+0x356/0x3a0
    Feb 8 21:44:52 soapbox kernel: [36770.130769] [] ext4_mb_new_blocks+0x2a9/0x540
    Feb 8 21:44:52 soapbox kernel: [36770.130774] [] ? ext4_ext_find_extent+0x130/0x2f0
    Feb 8 21:44:52 soapbox kernel: [36770.130778] [] ext4_ext_get_blocks+0x4a4/0x5b0
    Feb 8 21:44:52 soapbox kernel: [36770.130783] [] ext4_get_blocks+0x1d9/0x210
    Feb 8 21:44:52 soapbox kernel: [36770.130787] [] mpage_da_map_blocks+0xa7/0x370
    Feb 8 21:44:52 soapbox kernel: [36770.130792] [] ? jbd2_journal_start+0xae/0x100
    Feb 8 21:44:52 soapbox kernel: [36770.130796] [] ext4_da_writepages+0x2a3/0x500
    Feb 8 21:44:52 soapbox kernel: [36770.130801] [] do_writepages+0x28/0x50
    Feb 8 21:44:52 soapbox kernel: [36770.130806] [] writeback_single_inode+0x1bc/0x450
    Feb 8 21:44:52 soapbox kernel: [36770.130811] [] generic_sync_sb_inodes+0x418/0x530
    Feb 8 21:44:52 soapbox kernel: [36770.130815] [] writeback_inodes+0x5b/0x100
    Feb 8 21:44:52 soapbox kernel: [36770.130820] [] wb_kupdate+0xbc/0x140
    Feb 8 21:44:52 soapbox kernel: [36770.130824] [] __pdflush+0x13e/0x260
    Feb 8 21:44:52 soapbox kernel: [36770.130829] [] ? pdflush+0x0/0x50
    Feb 8 21:44:52 soapbox kernel: [36770.130833] [] pdflush+0x48/0x50
    Feb 8 21:44:52 soapbox kernel: [36770.130837] [] ? wb_kupdate+0x0/0x140
    Feb 8 21:44:52 soapbox kernel: [36770.130841] [] ? pdflush+0x0/0x50
    Feb 8 21:44:52 soapbox kernel: [36770.130846] [] kthread+0xa6/0xb0
    Feb 8 21:44:52 soapbox kernel: [36770.130850] [] child_rip+0xa/0x20
    Feb 8 21:44:52 soapbox kernel: [36770.130854] [] ? kthread+0x0/0xb0
    Feb 8 21:44:52 soapbox kernel: [36770.130858] [] ? child_rip+0x0/0x20
    Feb 8 21:44:52 soapbox kernel: [36770.130860] ---[ end trace 807496c710666e47 ]---
    Feb 8 21:44:52 soapbox kernel: [36770.130985] ------------[ cut here ]------------
    Feb 8 21:44:52 soapbox kernel: [36770.130992] WARNING: at /build/buildd/linux-2.6.31/fs/quota/dquot.c:964 dquot_claim_space+0x147/0x170()
    Feb 8 21:44:52 soapbox kernel: [36770.130995] Hardware name: eserver xSeries 236 -[884105U]-
    Feb 8 21:44:52 soapbox kernel: [36770.130997] Modules linked in: xt_multiport xt_tcpudp xt_state ipt_LOG nf_conntrack_ftp iptable_mangle iptable_nat nf_nat nf_conntrack_ipv4 nf_conntrack nf_defrag_ipv4 osst st quota_v2 quota_tree iptable_filter ip_tables radeon x_tables ttm drm i2c_algo_bit ppdev psmouse shpchp e752x_edac parport_pc lp serio_raw edac_core parport raid10 raid456 usbhid raid6_pq async_xor async_memcpy async_tx xor raid1 raid0 aic79xx ips scsi_transport_spi multipath floppy tg3 linear
    Feb 8 21:44:52 soapbox kernel: [36770.131047] Pid: 46, comm: pdflush Tainted: G W 2.6.31-19-server #56-Ubuntu
    Feb 8 21:44:52 soapbox kernel: [36770.131050] Call Trace:
    Feb 8 21:44:52 soapbox kernel: [36770.131056] [] warn_slowpath_common+0x78/0xb0
    Feb 8 21:44:52 soapbox kernel: [36770.131060] [] warn_slowpath_null+0xf/0x20
    Feb 8 21:44:52 soapbox kernel: [36770.131065] [] dquot_claim_space+0x147/0x170
    Feb 8 21:44:52 soapbox kernel: [36770.131069] [] ext4_mb_mark_diskspace_used+0x356/0x3a0
    Feb 8 21:44:52 soapbox kernel: [36770.131074] [] ext4_mb_new_blocks+0x2a9/0x540
    Feb 8 21:44:52 soapbox kernel: [36770.131079] [] ? ext4_ext_find_extent+0x130/0x2f0
    Feb 8 21:44:52 soapbox kernel: [36770.131084] [] ext4_ext_get_blocks+0x4a4/0x5b0
    Feb 8 21:44:52 soapbox kernel: [36770.131088] [] ext4_get_blocks+0x1d9/0x210
    Feb 8 21:44:52 soapbox kernel: [36770.131093] [] mpage_da_map_blocks+0xa7/0x370
    Feb 8 21:44:52 soapbox kernel: [36770.131097] [] ? jbd2_journal_start+0xae/0x100
    Feb 8 21:44:52 soapbox kernel: [36770.131101] [] ext4_da_writepages+0x2a3/0x500
    Feb 8 21:44:52 soapbox kernel: [36770.131107] [] do_writepages+0x28/0x50
    Feb 8 21:44:52 soapbox kernel: [36770.131111] [] writeback_single_inode+0x1bc/0x450
    Feb 8 21:44:52 soapbox kernel: [36770.131116] [] generic_sync_sb_inodes+0x418/0x530
    Feb 8 21:44:52 soapbox kernel: [36770.131121] [] writeback_inodes+0x5b/0x100
    Feb 8 21:44:52 soapbox kernel: [36770.131125] [] wb_kupdate+0xbc/0x140
    Feb 8 21:44:52 soapbox kernel: [36770.131130] [] __pdflush+0x13e/0x260
    Feb 8 21:44:52 soapbox kernel: [36770.131134] [] ? pdflush+0x0/0x50
    Feb 8 21:44:52 soapbox kernel: [36770.131138] [] pdflush+0x48/0x50
    Feb 8 21:44:52 soapbox kernel: [36770.131142] [] ? wb_kupdate+0x0/0x140
    Feb 8 21:44:52 soapbox kernel: [36770.131147] [] ? pdflush+0x0/0x50
    Feb 8 21:44:52 soapbox kernel: [36770.131151] [] kthread+0xa6/0xb0
    Feb 8 21:44:52 soapbox kernel: [36770.131155] [] child_rip+0xa/0x20
    Feb 8 21:44:52 soapbox kernel: [36770.131160] [] ? kthread+0x0/0xb0
    Feb 8 21:44:52 soapbox kernel: [36770.131163] [] ? child_rip+0x0/0x20
    Feb 8 21:44:52 soapbox kernel: [36770.131166] ---[ end trace 807496c710666e48 ]---
    Feb 8 21:44:52 soapbox kernel: [36770.131168] ------------[ cut here ]------------
    Feb 8 21:44:52 soapbox kernel: [36770.131172] WARNING: at /build/buildd/linux-2.6.31/fs/quota/dquot.c:964 dquot_claim_space+0x15d/0x170()
    Feb 8 21:44:52 soapbox kernel: [36770.131175] Hardware name: eserver xSeries 236 -[884105U]-
    Feb 8 21:44:53 soapbox kernel: [36770.131177] Modules linked in: xt_multiport xt_tcpudp xt_state ipt_LOG nf_conntrack_ftp iptable_mangle iptable_nat nf_nat nf_conntrack_ipv4 nf_conntrack nf_defrag_ipv4 osst st quota_v2 quota_tree iptable_filter ip_tables radeon x_tables ttm drm i2c_algo_bit ppdev psmouse shpchp e752x_edac parport_pc lp serio_raw edac_core parport raid10 raid456 usbhid raid6_pq async_xor async_memcpy async_tx xor raid1 raid0 aic79xx ips scsi_transport_spi multipath floppy tg3 linear
    Feb 8 21:44:53 soapbox kernel: [36770.131226] Pid: 46, comm: pdflush Tainted: G W 2.6.31-19-server #56-Ubuntu
    Feb 8 21:44:53 soapbox kernel: [36770.131229] Call Trace:
    Feb 8 21:44:53 soapbox kernel: [36770.131234] [] warn_slowpath_common+0x78/0xb0
    Feb 8 21:44:53 soapbox kernel: [36770.131239] [] warn_slowpath_null+0xf/0x20
    Feb 8 21:44:53 soapbox kernel: [36770.131243] [] dquot_claim_space+0x15d/0x170
    Feb 8 21:44:53 soapbox kernel: [36770.131248] [] ext4_mb_mark_diskspace_used+0x356/0x3a0
    Feb 8 21:44:53 soapbox kernel: [36770.131252] [] ext4_mb_new_blocks+0x2a9/0x540
    Feb 8 21:44:53 soapbox kernel: [36770.131257] [] ? ext4_ext_find_extent+0x130/0x2f0
    Feb 8 21:44:53 soapbox kernel: [36770.131262] [] ext4_ext_get_blocks+0x4a4/0x5b0
    Feb 8 21:44:53 soapbox kernel: [36770.131266] [] ext4_get_blocks+0x1d9/0x210
    Feb 8 21:44:53 soapbox kernel: [36770.131271] [] mpage_da_map_blocks+0xa7/0x370
    Feb 8 21:44:53 soapbox kernel: [36770.131275] [] ? jbd2_journal_start+0xae/0x100
    Feb 8 21:44:53 soapbox kernel: [36770.131279] [] ext4_da_writepages+0x2a3/0x500
    Feb 8 21:44:53 soapbox kernel: [36770.131285] [] do_writepages+0x28/0x50
    Feb 8 21:44:53 soapbox kernel: [36770.131289] [] writeback_single_inode+0x1bc/0x450
    Feb 8 21:44:53 soapbox kernel: [36770.131294] [] generic_sync_sb_inodes+0x418/0x530
    Feb 8 21:44:53 soapbox kernel: [36770.131299] [] writeback_inodes+0x5b/0x100
    Feb 8 21:44:53 soapbox kernel: [36770.131303] [] wb_kupdate+0xbc/0x140
    Feb 8 21:44:53 soapbox kernel: [36770.131308] [] __pdflush+0x13e/0x260
    Feb 8 21:44:53 soapbox kernel: [36770.131312] [] ? pdflush+0x0/0x50
    Feb 8 21:44:53 soapbox kernel: [36770.131316] [] pdflush+0x48/0x50
    Feb 8 21:44:53 soapbox kernel: [36770.131320] [] ? wb_kupdate+0x0/0x140
    Feb 8 21:44:53 soapbox kernel: [36770.131325] [] ? pdflush+0x0/0x50
    Feb 8 21:44:53 soapbox kernel: [36770.131329] [] kthread+0xa6/0xb0
    Feb 8 21:44:53 soapbox kernel: [36770.131333] [] child_rip+0xa/0x20
    Feb 8 21:44:53 soapbox kernel: [36770.131338] [] ? kthread+0x0/0xb0
    Feb 8 21:44:53 soapbox kernel: [36770.131341] [] ? child_rip+0x0/0x20
    Feb 8 21:44:53 soapbox kernel: [36770.131344] ---[ end trace 807496c710666e49 ]---
    Feb 8 21:45:01 soapbox pure-ftpd: ([email protected]) [INFO] New connection from 127.0.0.1
    Feb 8 21:45:01 soapbox pure-ftpd: ([email protected]) [INFO] Logout.
     
    Powered 
     
  7. DrJohn

    DrJohn Member

    I resolved this specific problem by changing the configuration. I now have a single 318Gb LVM volume group running on top of a RAID1 device on a pair of physical drives. A small second RAID1 on the same pair is mounted as /boot (non-LVM).

    The LVM volume group has two logical drives for the host system: one as swap and the other with 20G at /root. This part of the install is pretty much static; quota is not installed. The rest of the volume group space hosts several KVM instances. Each KVM was created on separate logical volumes of 20 - 40 Gb apiece mounted at (root) ~/vm1, ~/vm2, ...

    One of them (an ISPConfig webserver) is now running quota without a problem, the other (a document management install) will have it shortly. If there is a problem I'll post it here.

    All of this is to say that the original problem remains intact as far as I know, but at this point I won't be going back to fix it.
     

Share This Page