Monitoring has stopped

Discussion in 'Installation/Configuration' started by Anna Count, Feb 29, 2024.

  1. Anna Count

    Anna Count New Member

    I'm running ISPConfig 3.2.11p1 on a fully up to date Debian 12.

    Recently the following updates appeared in Update Status so I installed them.

    Inst imagemagick-6-common [8:6.9.11.60+dfsg-1.6] (8:6.9.11.60+dfsg-1.6+deb12u1 Debian-Security:12/stable-security [all])
    Inst libmagickcore-6.q16-6 [8:6.9.11.60+dfsg-1.6] (8:6.9.11.60+dfsg-1.6+deb12u1 Debian-Security:12/stable-security [amd64])
    Inst libmagickwand-6.q16-6 [8:6.9.11.60+dfsg-1.6] (8:6.9.11.60+dfsg-1.6+deb12u1 Debian-Security:12/stable-security [amd64])
    Inst imagemagick-6.q16 [8:6.9.11.60+dfsg-1.6] (8:6.9.11.60+dfsg-1.6+deb12u1 Debian-Security:12/stable-security [amd64])
    Inst imagemagick [8:6.9.11.60+dfsg-1.6] (8:6.9.11.60+dfsg-1.6+deb12u1 Debian-Security:12/stable-security [amd64])
    Inst libmagickcore-6.q16-6-extra [8:6.9.11.60+dfsg-1.6] (8:6.9.11.60+dfsg-1.6+deb12u1 Debian-Security:12/stable-security [amd64])
    Inst rspamd [3.8.1-1~b8a2d79ee~bookworm] (3.8.4-1~93fa4f6dc~bookworm Rspamd:rspamd.com [amd64])
    Conf imagemagick-6-common (8:6.9.11.60+dfsg-1.6+deb12u1 Debian-Security:12/stable-security [all])
    Conf libmagickcore-6.q16-6 (8:6.9.11.60+dfsg-1.6+deb12u1 Debian-Security:12/stable-security [amd64])
    Conf libmagickwand-6.q16-6 (8:6.9.11.60+dfsg-1.6+deb12u1 Debian-Security:12/stable-security [amd64])
    Conf imagemagick-6.q16 (8:6.9.11.60+dfsg-1.6+deb12u1 Debian-Security:12/stable-security [amd64])
    Conf imagemagick (8:6.9.11.60+dfsg-1.6+deb12u1 Debian-Security:12/stable-security [amd64])
    Conf libmagickcore-6.q16-6-extra (8:6.9.11.60+dfsg-1.6+deb12u1 Debian-Security:12/stable-security [amd64])
    Conf rspamd (3.8.4-1~93fa4f6dc~bookworm Rspamd:rspamd.com [amd64])

    There were no errors but I did notice the following.

    Installing new version of config file /etc/rspamd/modules.d/history_redis.conf ...
    Installing new version of config file /etc/rspamd/modules.d/rbl.conf ...
    Installing new version of config file /etc/rspamd/scores.d/surbl_group.conf ...

    There were no php updates and I haven't changed any config files.

    I'm not sure whether the monitoring problem is related to the above updates but after a few minutes I noticed that system state was still blue.
    It has remained blue ever since and nothing under server state or log files is updating any more.

    The output from crontab -l is

    crontab -l
    50 10 * * * "/root/.acme.sh"/acme.sh --cron --home "/root/.acme.sh" > /dev/null
    * * * * * /usr/local/ispconfig/server/server.sh 2>&1 | while read line; do echo `/bin/date` "$line" >> /var/log/ispconfig/cron.log; done
    * * * * * /usr/local/ispconfig/server/cron.sh 2>&1 | while read line; do echo `/bin/date` "$line" >> /var/log/ispconfig/cron.log; done
    0 0 * * * /usr/local/ispconfig/server/scripts/create_daily_nginx_access_logs.sh &> /dev/null

    And pgrep cron is green with cron active and running.

    I note that in /var/log/ispconfig/cron.log the following lines were present every 5 minutes until monitoring stopped.

    Wed 28 Feb 2024 11:55:01 PM EST PHP Warning: Undefined array key -1 in /usr/local/ispconfig/server/lib/classes/cron.d/100-monitor_clamav_log.inc.php on line 111
    Wed 28 Feb 2024 11:55:01 PM EST PHP Deprecated: strpos(): Passing null to parameter #1 ($haystack) of type string is deprecated in /usr/local/ispconfig/server/lib/classes/cron.d/100-monitor_clamav_log.inc.php on line 111
    Wed 28 Feb 2024 11:55:01 PM EST PHP Warning: Undefined array key "db_flags" in /usr/local/ispconfig/server/lib/classes/db_mysql.inc.php on line 671
    Wed 28 Feb 2024 11:55:01 PM EST PHP Warning: Undefined array key 1 in /usr/local/ispconfig/server/lib/classes/cron.d/100-monitor_mem_usage.inc.php on line 79
    Wed 28 Feb 2024 11:55:01 PM EST PHP Deprecated: trim(): Passing null to parameter #1 ($string) of type string is deprecated in /usr/local/ispconfig/server/lib/classes/cron.d/100-monitor_mem_usage.inc.php on line 79

    The above are no longer present and I'm only seeing a line such as
    Wed 28 Feb 2024 11:56:01 PM EST finished server.php
    Every 1 minute.

    So it looks like a process which was previously running every 5 minutes is no longer running at all.

    I also tried rm -f /usr/local/ispconfig/server/temp/.ispconfig_lock but no difference.

    I haven't rebooted since this happened and maybe that would fix it but it's running web and email services (with no issues) and I don't want to restart it during working hours.

    The server is otherwise working fine and all log files when opened manually are found to be correctly updating.
    mail.log no longer shows the lines containing 127.0.0.1 which used to show every 5 minutes but it's otherwise normal.

    I don't see any errors in syslog, just lack of messages associated with monitoring.

    How do I find out why monitoring isn't working and get it working again?

    Thank you
     
  2. till

    till Super Moderator Staff Member ISPConfig Developer

    There is most likely no problem at all. The package monitoring is not real-time; if I remember correctly, it's done twice a day, so you just have to wait.
     
  3. till

    till Super Moderator Staff Member ISPConfig Developer

    If you have issues with other parts of the monitoring system, run a repair on all tables in dbispconfig database and if this doe snot help. empty the table sys_cron in dbispconfig database to reset the internal cron system.
     
  4. Anna Count

    Anna Count New Member

    Hi Till,
    Thank you for your reply.
    I have absolutely no idea why but monitoring started again just after midnight, approximately 27 hours after it stopped.
    During the outage, everything on the monitor tab under Server State and Logfiles was not updating.
    Some messages in cron.log which occurred at the time it started again are included in case they are of use to anyone who knows what they mean.
    Code:
    Fri 01 Mar 2024 12:05:45 AM EST PHP Deprecated:  Creation of dynamic property cronjob_quota_notify::$_tools is deprecated in /usr/local/ispconfig/server/lib/classes/cron.d/300-quota_notify.inc.php on line 55
    Fri 01 Mar 2024 12:05:45 AM EST PHP Deprecated:  Using ${var} in strings is deprecated, use {$var} instead in /usr/local/ispconfig/server/lib/classes/backup.inc.php on line 1493
    Fri 01 Mar 2024 12:05:45 AM EST PHP Deprecated:  Using ${var} in strings is deprecated, use {$var} instead in /usr/local/ispconfig/server/lib/classes/backup.inc.php on line 1493
    Fri 01 Mar 2024 12:05:45 AM EST PHP Deprecated:  Using ${var} in strings is deprecated, use {$var} instead in /usr/local/ispconfig/server/lib/classes/backup.inc.php on line 1493
    Fri 01 Mar 2024 12:05:45 AM EST PHP Deprecated:  Using ${var} in strings is deprecated, use {$var} instead in /usr/local/ispconfig/server/lib/classes/backup.inc.php on line 1493
    Fri 01 Mar 2024 12:05:45 AM EST PHP Deprecated:  Using ${var} in strings is deprecated, use {$var} instead in /usr/local/ispconfig/server/lib/classes/backup.inc.php on line 1495
    Fri 01 Mar 2024 12:05:45 AM EST PHP Deprecated:  Using ${var} in strings is deprecated, use {$var} instead in /usr/local/ispconfig/server/lib/classes/backup.inc.php on line 1495
    Fri 01 Mar 2024 12:05:45 AM EST PHP Deprecated:  Using ${var} in strings is deprecated, use {$var} instead in /usr/local/ispconfig/server/lib/classes/backup.inc.php on line 1495
    Fri 01 Mar 2024 12:05:45 AM EST PHP Deprecated:  Using ${var} in strings is deprecated, use {$var} instead in /usr/local/ispconfig/server/lib/classes/backup.inc.php on line 1495
    Fri 01 Mar 2024 12:05:45 AM EST PHP Deprecated:  Using ${var} in strings is deprecated, use {$var} instead in /usr/local/ispconfig/server/lib/classes/backup.inc.php on line 1866
    Fri 01 Mar 2024 12:05:45 AM EST PHP Deprecated:  Using ${var} in strings is deprecated, use {$var} instead in /usr/local/ispconfig/server/lib/classes/backup.inc.php on line 1867
    Fri 01 Mar 2024 12:06:01 AM EST finished server.php.
    Fri 01 Mar 2024 12:06:01 AM EST PHP Warning:  Undefined array key -1 in /usr/local/ispconfig/server/lib/classes/cron.d/100-monitor_clamav_log.inc.php on line 111
    Fri 01 Mar 2024 12:06:01 AM EST PHP Deprecated:  strpos(): Passing null to parameter #1 ($haystack) of type string is deprecated in /usr/local/ispconfig/server/lib/classes/cron.d/100-monitor_clamav_log.inc.php on line 111
    Fri 01 Mar 2024 12:06:01 AM EST PHP Warning:  Undefined array key "db_flags" in /usr/local/ispconfig/server/lib/classes/db_mysql.inc.php on line 671
    Fri 01 Mar 2024 12:06:01 AM EST PHP Warning:  Undefined array key 1 in /usr/local/ispconfig/server/lib/classes/cron.d/100-monitor_mem_usage.inc.php on line 79
    Fri 01 Mar 2024 12:06:01 AM EST PHP Deprecated:  trim(): Passing null to parameter #1 ($string) of type string is deprecated in /usr/local/ispconfig/server/lib/classes/cron.d/100-monitor_mem_usage.inc.php on line 79
    Fri 01 Mar 2024 12:07:01 AM EST finished server.php.
    Fri 01 Mar 2024 12:08:01 AM EST finished server.php.
    Fri 01 Mar 2024 12:09:01 AM EST finished server.php.
    Fri 01 Mar 2024 12:10:01 AM EST finished server.php.
    Fri 01 Mar 2024 12:10:01 AM EST PHP Warning:  Undefined array key -1 in /usr/local/ispconfig/server/lib/classes/cron.d/100-monitor_clamav_log.inc.php on line 111
    Fri 01 Mar 2024 12:10:01 AM EST PHP Deprecated:  strpos(): Passing null to parameter #1 ($haystack) of type string is deprecated in /usr/local/ispconfig/server/lib/classes/cron.d/100-monitor_clamav_log.inc.php on line 111
    Fri 01 Mar 2024 12:10:01 AM EST PHP Warning:  Undefined array key "db_flags" in /usr/local/ispconfig/server/lib/classes/db_mysql.inc.php on line 671
    Fri 01 Mar 2024 12:10:02 AM EST PHP Warning:  Undefined array key 1 in /usr/local/ispconfig/server/lib/classes/cron.d/100-monitor_mem_usage.inc.php on line 79
    Fri 01 Mar 2024 12:10:02 AM EST PHP Deprecated:  trim(): Passing null to parameter #1 ($string) of type string is deprecated in /usr/local/ispconfig/server/lib/classes/cron.d/100-monitor_mem_usage.inc.php on line 79
    Fri 01 Mar 2024 12:11:01 AM EST finished server.php.
    Fri 01 Mar 2024 12:12:01 AM EST finished server.php.
    Fri 01 Mar 2024 12:13:01 AM EST finished server.php.
    Fri 01 Mar 2024 12:14:02 AM EST finished server.php.
    Fri 01 Mar 2024 12:15:01 AM EST finished server.php.
    Fri 01 Mar 2024 12:15:01 AM EST PHP Warning:  Undefined array key -1 in /usr/local/ispconfig/server/lib/classes/cron.d/100-monitor_clamav_log.inc.php on line 111
    Fri 01 Mar 2024 12:15:01 AM EST PHP Deprecated:  strpos(): Passing null to parameter #1 ($haystack) of type string is deprecated in /usr/local/ispconfig/server/lib/classes/cron.d/100-monitor_clamav_log.inc.php on line 111
    Fri 01 Mar 2024 12:15:01 AM EST PHP Warning:  Undefined array key "db_flags" in /usr/local/ispconfig/server/lib/classes/db_mysql.inc.php on line 671
    Fri 01 Mar 2024 12:15:01 AM EST PHP Warning:  Undefined array key 1 in /usr/local/ispconfig/server/lib/classes/cron.d/100-monitor_mem_usage.inc.php on line 79
    Fri 01 Mar 2024 12:15:01 AM EST PHP Deprecated:  trim(): Passing null to parameter #1 ($string) of type string is deprecated in /usr/local/ispconfig/server/lib/classes/cron.d/100-monitor_mem_usage.inc.php on line 79
    Fri 01 Mar 2024 12:16:02 AM EST finished server.php.
    Fri 01 Mar 2024 12:17:01 AM EST finished server.php.
    
     
  5. Taleman

    Taleman Well-Known Member HowtoForge Supporter

  6. Anna Count

    Anna Count New Member

    Ok I'll maybe do that over the weekend. Other than view log files I don't like to do too much as root while it's running fine as a production server. I assume that php would have been set up by the installer. The install command was
    wget -O - https://get.ispconfig.org | sh -s -- --use-nginx --use-ftp-ports=40110-40210 --no-firewall --no-roundcube --no-mailman
    On a fully updated Debian 12 about two months ago.
    I use firewalld as the firewall with extensive inbound blocking of countries my clients do no business with.
     

Share This Page