Quota warnings / notifications

Discussion in 'General' started by vladki, Aug 18, 2016.

  1. vladki

    vladki New Member

    Hello all,

    I have installed 3.1dev. I have tried to set up quotas for a mailbox, web and mysql. They show correctly in the home page, mailbox and FTP quotas are correctly enforced. I have enabled server options:
    -Send quota warnings to admin
    -Send quota warnings to client
    -Send DB quota warnings to admin
    -Send DB quota warnings to client
    But do not get any warnings at all, even though the web and sql are well over quota for a few days now.
    So the question is why these warnings are not sent - where can I check what happened?
    Where can I set the admin email to be used for those warnings?

    I have managed to configure dovecot to send warnigns on his own, that's fine.
    I can send warnings using warnquota from command line, but they bounce back to root, as there is no association between webX linux user and his email. (I whould have to add that manually to /etc/aliases)

    Thanks for help
     
  2. till

    till Super Moderator Staff Member ISPConfig Developer

    System > Interface config
     
  3. vladki

    vladki New Member

    I have set the email, but still I do not get any over quota warning by email. Is there any way how to debug that?

    I have checked the cron.log, thisi is what appears an midnight:

    Fri Aug 19 00:02:31 CEST 2016 Included cronjob_quota_notify from /usr/local/ispconfig/server/lib/classes/cron.d/300-quota_notify.inc.php -> will now run job.
    Fri Aug 19 00:02:31 CEST 2016 Called run() for class cronjob_quota_notify
    Fri Aug 19 00:02:31 CEST 2016 Job has schedule: 0 0 * * *
    Fri Aug 19 00:02:31 CEST 2016 Called onPrepare() for class cronjob_quota_notify
    Fri Aug 19 00:02:31 CEST 2016 Called onBeforeRun() for class cronjob_quota_notify
    Fri Aug 19 00:02:31 CEST 2016 Jobs next run is 2016-08-19 00:00:00
    Fri Aug 19 00:02:31 CEST 2016 Date compare of 1471557600 and 1471557751 is 1
    Fri Aug 19 00:02:31 CEST 2016 Jobs next run is now 2016-08-20 0:0:0
    Fri Aug 19 00:02:31 CEST 2016 PHP Warning: Invalid argument supplied for foreach() in /usr/local/ispconfig/server/lib/classes/cron.d/300-quota_notify.inc.php on line 432
    Fri Aug 19 00:02:31 CEST 2016 PHP Warning: Invalid argument supplied for foreach() in /usr/local/ispconfig/server/lib/classes/cron.d/300-quota_notify.inc.php on line 432
    Fri Aug 19 00:02:31 CEST 2016 PHP Warning: Invalid argument supplied for foreach() in /usr/local/ispconfig/server/lib/classes/cron.d/300-quota_notify.inc.php on line 432
    Fri Aug 19 00:02:31 CEST 2016 PHP Warning: Invalid argument supplied for foreach() in /usr/local/ispconfig/server/lib/classes/cron.d/300-quota_notify.inc.php on line 432
    Fri Aug 19 00:02:31 CEST 2016 PHP Warning: Invalid argument supplied for foreach() in /usr/local/ispconfig/server/lib/classes/cron.d/300-quota_notify.inc.php on line 432
    Fri Aug 19 00:02:31 CEST 2016 PHP Warning: Invalid argument supplied for foreach() in /usr/local/ispconfig/server/lib/classes/cron.d/300-quota_notify.inc.php on line 432
    Fri Aug 19 00:02:31 CEST 2016 PHP Warning: Invalid argument supplied for foreach() in /usr/local/ispconfig/server/lib/classes/cron.d/300-quota_notify.inc.php on line 432
    Fri Aug 19 00:02:31 CEST 2016 PHP Warning: Invalid argument supplied for foreach() in /usr/local/ispconfig/server/lib/classes/cron.d/300-quota_notify.inc.php on line 432
    Fri Aug 19 00:02:31 CEST 2016 Called onRun() for class cronjob_quota_notify
    Fri Aug 19 00:02:31 CEST 2016 Called onAfterRun() for class cronjob_quota_notify
    Fri Aug 19 00:02:31 CEST 2016 Called onCompleted() for class cronjob_quota_notify
    Fri Aug 19 00:02:31 CEST 2016 run job (cronjob_quota_notify) done.
     
    Last edited: Aug 19, 2016
  4. vladki

    vladki New Member

    Can I help somehow to debug this further? Or is there something I forgot to set up?
     
  5. Jesse Norell

    Jesse Norell ISPConfig Developer Staff Member ISPConfig Developer

    This is just a guess, I don't think I have any such over quota emails going out to test, but in /usr/local/ispconfig/server/lib/classes/cron.d/300-quota_notify.inc.php:
    Code:
    430                                        foreach ($monitor_data as $cid) {
    431
    432                                                foreach($cid_data as $monitor) {
    
    The variable names on line 430 and 432 don't match ($cid vs $cid_data), try changing one to be the same as the other.
     
  6. florian030

    florian030 ISPConfig Developer ISPConfig Developer

    foreach($cid_data as $monitor) {
    should be
    foreach($cid as $monitor) {
     
  7. vladki

    vladki New Member

    Thanks florian, there are no more errors, but no notifications either.
    This is the cron.log, no evident failures. Nothing in mail.log as well.

    Wed Aug 24 00:02:24 CEST 2016 Included cronjob_quota_notify from /usr/local/ispconfig/server/lib/classes/cron.d/300-quota_notify.inc.php -> will now run job.
    Wed Aug 24 00:02:24 CEST 2016 Called run() for class cronjob_quota_notify
    Wed Aug 24 00:02:24 CEST 2016 Job has schedule: 0 0 * * *
    Wed Aug 24 00:02:24 CEST 2016 Called onPrepare() for class cronjob_quota_notify
    Wed Aug 24 00:02:24 CEST 2016 Called onBeforeRun() for class cronjob_quota_notify
    Wed Aug 24 00:02:24 CEST 2016 Jobs next run is 2016-08-24 00:00:00
    Wed Aug 24 00:02:24 CEST 2016 Date compare of 1471989600 and 1471989744 is 1
    Wed Aug 24 00:02:24 CEST 2016 Jobs next run is now 2016-08-25 0:0:0
    Wed Aug 24 00:02:24 CEST 2016 Called onRun() for class cronjob_quota_notify
    Wed Aug 24 00:02:24 CEST 2016 Called onAfterRun() for class cronjob_quota_notify
    Wed Aug 24 00:02:24 CEST 2016 Called onCompleted() for class cronjob_quota_notify
    Wed Aug 24 00:02:24 CEST 2016 run job (cronjob_quota_notify) done.

    I just wonder, whether I have to enable in system config/mail the checkbox "Use SMTP to send system mails", and fill in localhost:25, or not? I would suppose that without that setting it shoud send the emails using sendmail binary or PHP mail() function.
     
  8. florian030

    florian030 ISPConfig Developer ISPConfig Developer

    The db-quota should be working now. You need at least notify admin / client for over db-quota (see server / server config / tab web)
     
    Jesse Norell likes this.
  9. vladki

    vladki New Member

    Thank you, I have updated /usr/local/ispconfig/server/lib/classes/cron.d/300-quota_notify.inc.php from git, and will see and report tomorrow.
     
  10. vladki

    vladki New Member

    Hello, finally I've got the quota warnings, but they are quite weird. I got notification for empty databases, that they are over quota:

    The database c2owncloud is near or over quota.
    Database: c2owncloud
    Used space: NAN
    Quota: 0 MB
    Usage ratio: 0.00%

    The dashboard says quota unlimited. I tried to set the database quota to -1, but it went back to 0.

    On the other hand - for a database that is clearly over quota I got:

    The database c1test is no longer near or over quota.
    Database: c1test
    Used space: 1.52 MB
    Quota: 1048576 MB
    Usage ratio: 151.56%

    The quota is set to 1 MB (web interface shows it right), not 1 TB as written in the quota report.

    Is there a way to run the quota check by hand (or more frequently) for testing purposes?
     
  11. florian030

    florian030 ISPConfig Developer ISPConfig Developer

    The mail-function is indeed wrong. Please change line 472 to
    $this->_tools->send_notification_email('db_quota_notification', $placeholders, $recipients);
    and line 504 to
    $this->_tools->send_notification_email('db_quota_ok_notification', $placeholders, $recipients);

    I justed teste the notficy-option and get only a few mails for databases over quota. Please make sure, that you use the current version: https://git.ispconfig.org/ispconfig...r/lib/classes/cron.d/300-quota_notify.inc.php

    You can test the cron-class:
    php /usr/local/ispconfig/server/cron_debug.php --cronjob=300-quota_notify.inc.php
     
    vladki likes this.
  12. vladki

    vladki New Member

    Hello, I did not get to testing on friday, but now I have updated the 300-quota_notify.inc.php from git, and run the above command manually. I got a mail about empty databases not being over quota any more, but it also says that the database that is over quota is ok:

    The database c1test is no longer near or over quota.
    Database: c1test
    Used space: 1.52 MB
    Quota: 1048576 MB
    Usage ratio: 151.56%

    The quota is in fact set to 1 MB, so the usage ratio is calculated right, but the total quota has wrong units (should be Bytes not MB). Maybe that makes the system confused and sends ok message instead of overquota messeg.

    For empty databases I get this - seems to be OK. I get it on every run of cron-quota. Should it be so? I thought that ok message would be sent only once? However I did not get automatic mails for these empty databases.

    The database c2empty is no longer near or over quota.
    Database: c2empty
    Used space: NAN
    Quota: 0 MB
    Usage ratio: 0.00%

    During the weekend I got notified only about the overquota database every day, and only once (on saturday) about overquota website (filesystem). Is that how it is supposed to be? The website quota warning was correct.
     
  13. vladki

    vladki New Member

    I did some more testing, with the following result. I get an email "Database quota ok" for all databases, regardless if they are over quota or not. My examples are: empty database with no quota, non-empty database under quota and non-empty databes over quota. The used space and usage ratio are computed correctly, but "quota" in the mail is multiplied by 1048576. That may be why it says it is not over quota when it in fact is.
     
  14. vladki

    vladki New Member

    I look at the code and do not understand the line 445:
    Code:
      
    //* send notifications only if 90% or more of the quota are used
    if($used_ratio > 0.9 && $user_ratio != 0) {
    
    The variable $user_ratio is not used anywhere else, so I suspect a typo? But if it would be used_ratio, then the test for non-zero would be redundant, as anything that is > 0.9 is not zero. Anyway after changing it to used_ratio, I stopped getting emails for over-quota database. Ok some more testing and I think I got it right, here is the patch for 300-quota_notify.inc.php
    Code:
    --- 300-quota_notify.inc.php    2016-08-30 10:07:18.138180155 +0200
    +++ /usr/local/ispconfig/server/lib/classes/cron.d/300-quota_notify.inc.php     2016-08-30 10:17:29.082182240 +0200
    @@ -442,7 +442,7 @@
                                                                    else $used_ratio = 0;
                                                                    //* send notifications only if 90% or more of the quota are used
    -                                                               if($used_ratio > 0.9 && $user_ratio != 0) {
    +                                                               if($used_ratio > 0.9 && $used_ratio != 0) {
                                                                            //* could a notification be sent?
                                                                            $send_notification = false;
    @@ -457,7 +457,7 @@
                                                                                            '{database_name}' => $rec['database_name'],
                                                                                            '{admin_mail}' => ($global_config['admin_mail'] != ''? $global_config['admin_mail'] : 'root'),
                                                                                            '{used}' => $app->functions->formatBytes($monitor['size']),
    -                                                                                       '{quota}' => $quota.' MB',
    +                                                                                       '{quota}' => $app->functions->formatBytes($quota),
                                                                                            '{ratio}' => number_format($used_ratio * 100, 2, '.', '').'%'
                                                                                    );
    @@ -479,7 +479,7 @@
                                                                            //* reset notification date
                                                                            if($rec['last_quota_notification']) $app->dbmaster->datalogUpdate('web_database', array("last_quota_notification" => null), 'database_id', $rec['database_id']);
    -                                                                       $app->dbmaster->datalogUpdate('web_database', array("last_quota_notification" => array("SQL" => "CURDATE()")), 'database_id', $rec['database_id']);
    +                                                                       // $app->dbmaster->datalogUpdate('web_database', array("last_quota_notification" => array("SQL" => "CURDATE()")), 'database_id', $rec['database_id']);
                                                                            // send notification - everything ok again
                                                                            if($rec['last_quota_notification'] && $web_config['overquota_notify_onok'] == 'y' && ($web_config['overquota_db_notify_admin'] == 'y' || $web_config['overquota_db_notify_client'] == 'y')) {
    @@ -487,7 +487,7 @@
                                                                                            '{database_name}' => $rec['database_name'],
                                                                                            '{admin_mail}' => ($global_config['admin_mail'] != ''? $global_config['admin_mail'] : 'root'),
                                                                                            '{used}' => $app->functions->formatBytes($monitor['size']),
    -                                                                                       '{quota}' => $quota.' MB',
    +                                                                                       '{quota}' => $app->functions->formatBytes($quota),
                                                                                            '{ratio}' => number_format($used_ratio * 100, 2, '.', '').'%'
                                                                                    );
    
     
    Jesse Norell likes this.
  15. vladki

    vladki New Member

    I noticed that part of this patch was commited to git, but the rest is important too:
    - it fixes wrong display of the quota in notification emails (it was showing number in bytes but adding MB suffix)
    - further fix is that sending OK notification should reset the last_quota_notofication to null, (to stop further notifications), but it set it to current date (probably leftover from copy/pasting or debug thing).
     
  16. vladki

    vladki New Member

  17. dark alex

    dark alex New Member

  18. wiss

    wiss Member

    Hi All.
    I have ispconfig 3.1.1.p1 on Ubuntu 16.04 and i have problems whit email quotas.
    Website quota is working well but ispconfig cant send mail quotas to costumer and clients.

    In logs can't find anyting wrong...
    I haver master server with ispconfig 3.1.1p1 ubuntu 16.04 and mail.server ubuntu 16.04 with ispconfig 3.1.1p1
    If I enable email log -->debug, i received every minute an email from mail.server but not received anyting from quota mail.
    I use dovecot, and I follow perfect server ubuntu 16.04.

    How do I to solve this?. How can i do?

    Thx.
     
  19. vladki

    vladki New Member

    It is possible that there are some bugs in the mail quota warnings, similar to those that were fixed for website and database quotas. Further - ispconfig sends wernigns to admin and "client" - not to the individual mailbox that is over quota. Therefore I modified directly dovecot.conf so that the warnings are sent by dovecot itself. Unfortunately, every upgrade overwrites my changes... Look here: http://wiki2.dovecot.org/Quota/Configuration#Quota_warnings
    Here is the relevant snippet:
    Code:
    plugin {
      quota = dict:user::file:/var/vmail/%d/%n/.quotausage
      sieve=/var/vmail/%d/%n/.sieve
      sieve_max_redirects = 25
      quota_warning = storage=95%% quota-warning 95 %u
      quota_warning2 = storage=80%% quota-warning 80 %u
    }
    service quota-warning {
      executable = script /usr/local/bin/quota-warning.sh
      user = vmail
      unix_listener quota-warning {
      user = vmail
      }
    }
    
    The script /usr/local/bin/quota-warning.sh is a bit simplified version form dovecot wiki:
    Code:
    #! /bin/bash
    
    PERCENT=$1
    USER=$2
    echo Mailbox $USER is $PERCENT% full | logger
    cat << EOF | /usr/lib/dovecot/deliver -d $USER
    From: [email protected]
    Subject: quota warning
    
    Your mailbox is over $PERCENT% full.
    EOF
    
    It would be nice if this is managed directly by ISPConfig.
     

Share This Page