Used mailbox quota always shows 88 kb

Hi out there,
i discovered a quite odd effect with ISPconfig 3 .1.11. When check the used email quota ISPConfig web backend shows 88kB for each mailbox. Customers with access to the backend reported the same effect.
To get ahead i removed all .quotausage files. For some minutes the ISPConfig backend shows correct quota usages. But after some minutes the displayed values change back to 88kB when the quotausage-files get created again. I tried to re-write quota settings via the backend, but that doesn't change this behavior.
Everything else works fine and as expected.

Our setup is ISPConfig 3.1.11 on Debian 8 with postfix and dovecot. We habe three mailservers as cluster:
mail1 is master. mail2 and mail3 are slaves. mail1 and mail2 use a shared /var/vmail directory which runs on a HP Storage P2000 G3 with ocfs2 as cluster filesystem. mail3 has its own /var/vmail which gets mirrored with mail1 / mail2 via unison.
Dovecot Version: 2.2.13
Postfix Version: 2.11.3

I don't know what to do, does anybody have an idea?

 

I kept an eye on this problem during the last hours. The really funny thing ist, that sometimes ISPConfig reports the correct quota usage. This lasts for some minutes. Then box by box the reported quota usage falls back to 88 kB.
I excluded the .quotausage file from unison, but this doesn't change the behavior.
Before we changed to the HP storage a few weeks ago we used gluster as shared storage and hat the same problem.

 

i already use these nfs settings ...

 

Btw, this ist my dovecot.conf:

listen = *,[::]
protocols = imap pop3
auth_mechanisms = plain login
# cram-md5
disable_plaintext_auth = no
log_timestamp = "%Y-%m-%d %H:%M:%S "
mail_privileged_group = vmail
mail_plugins = quota

# ssl_cert = </etc/postfix/smtpd.cert
# ssl_key = </etc/postfix/smtpd.key
ssl_protocols = !SSLv2 !SSLv3

ssl_cert = </etc/postfix/mailer.wildcat.media.crt
ssl_key = </etc/postfix/mailer.wildcat.media.key
ssl_ca = </etc/postfix/mailer.wildcat.media.bundle


passdb {
args = /etc/dovecot/dovecot-sql.conf
driver = sql
}
userdb {
args = /etc/dovecot/dovecot-sql.conf
driver = sql
}
plugin {
quota = dict:user::file:/var/vmail/%d/%n/.quotausage
sieve=/var/vmail/%d/%n/.sieve
}
service auth {
unix_listener /var/spool/postfix/private/auth {
group = postfix
mode = 0660
user = postfix
}
unix_listener auth-userdb {
group = vmail
mode = 0600
user = vmail
}
client_limit = 5220
user = root
}
service imap-login {
client_limit = 5120
process_limit = 5120
}
service anvil {
client_limit = 5230
}
protocol imap {
mail_plugins = quota imap_quota
}
protocol pop3 {
pop3_uidl_format = %08Xu%08Xv
mail_plugins = quota
}
protocol lda {
mail_plugins = sieve quota
}

mail_max_userip_connections = 2000

mail_temp_dir = /var/tmp
mail_fsync = always
mmap_disable = yes
# lock_method = fcntl
# mail_nfs_storage = no
# lock_method = dotlock
lock_method = flock
mail_nfs_storage = yes

 

OK, i figured out that the quota itself works fine, "quota get -u user" and "quota recalc -u user" work as expected:

# doveadm quota get -u USER1
Quota name Type Value Limit %
user STORAGE 1096310 20971520 5
user MESSAGE 8901 - 0

# doveadm quota get -u USER2
Quota name Type Value Limit %
user STORAGE 2039286 4194304 48
user MESSAGE 3885 - 0

 

... but still 88 kB used shown in ISPConfig most of the time, sometimes for a few minutes ISPConfig shows correct quota usage.

 

@till , the scripts that read the quota run on each node of the cluster, right?
Where does each node write this quota information? Into the master database or its own?

I made an experiment and modified the file /usr/local/ispconfig/server/lib/classes/cron.d/100-monitor_email_quota.inc.php on each node, i commented out line 92 - 101 and 107, so it looks like this:

foreach($mailboxes as $mb) {
$email = $mb['email'];
$email_parts = explode('@', $mb['email']);
$filename = $mb['maildir'].'/.quotausage';
// if(!file_exists($filename) && $dovecot) {
// exec('doveadm quota recalc -u '.$email);
// }
// if(file_exists($filename) && !is_link($filename)) {
// $quotafile = file($filename);
// preg_match('/storage.*?([0-9]+)/s', implode('',$quotafile), $storage_value);
// $data[$email]['used'] = $storage_value[1];
// $app->log("Mail storage $email: " . $storage_value[1], LOGLEVEL_DEBUG);
// unset($quotafile);
// } else {
exec('du -s '.escapeshellcmd($mb['maildir']), $out);
$parts = explode(' ', $out[0]);
$data[$email]['used'] = intval($parts[0])*1024;
unset($out);
unset($parts);
// }
}

The idea is to force ISPConfig to read the used quota by du -s - but: same effect as above.
Now i'll take each node of the cluster down, one by one, and see what happens. To me it seems that one of them writes garbage in the database.