Hello, I noticed about a 3 weeks ago my stats stopped populating. I went back and checked everything I did that day, and the day before, and found that I probably made no changes to the server so I'm not sure why. When I look at yesterday-access.log on each website, the symlink is old and broken because it links to a missing access log from 3 weeks ago. When I run "php /usr/local/ispconfig/server/cron_debug.php --cronjob=150-awstats.inc.php" all the symlinks updates to the proper file. I spent all day fixing statistics manually with logmerge and learning how awstats works, but the next morning the symlink is out of date again and stats are not generating without manually running 150-awstats.inc.php I was having excess memory/cpu issues back then, so maybe something got reaped and is causing this problem? I checked my ISPConfig cron log and I don't see anything unusual comparing it to a couple months ago. For example this appears to run fine: Code: Wed Oct 14 11:01:02 ADT 2020 Included cronjob_awstats from /usr/local/ispconfig/server/lib/classes/cron.d/150-awstats.inc.php -> will now run job. Wed Oct 14 11:01:02 ADT 2020 Called run() for class cronjob_awstats Wed Oct 14 11:01:02 ADT 2020 Job has schedule: 0 0 * * * Wed Oct 14 11:01:02 ADT 2020 Called onPrepare() for class cronjob_awstats Wed Oct 14 11:01:02 ADT 2020 Called onBeforeRun() for class cronjob_awstats Wed Oct 14 11:01:02 ADT 2020 run job (cronjob_awstats) done. I did find in apache's error log that it restarts at night, I assume this is normal? Code: [Mon Oct 12 00:00:14.175287 2020] [mpm_prefork:notice] [pid 22466] AH00171: Graceful restart requested, doing restart [ 2020-10-12 00:00:14.3937 27179/7fa1ff678700 age/Cor/CoreMain.cpp:532 ]: Signal received. Gracefully shutting down... (send signal 2 more time(s) to force shutdown) [ 2020-10-12 00:00:14.3946 27179/7fa1fff67980 age/Cor/CoreMain.cpp:901 ]: Received command to shutdown gracefully. Waiting until all clients have disconnected... [ 2020-10-12 00:00:14.3964 27179/7fa1fedf6700 Ser/Server.h:817 ]: [ApiServer] Freed 0 spare client objects [ 2020-10-12 00:00:14.3969 27179/7fa1fedf6700 Ser/Server.h:464 ]: [ApiServer] Shutdown finished [ 2020-10-12 00:00:14.3966 27184/7f58ea23d700 age/Ust/UstRouterMain.cpp:422 ]: Signal received. Gracefully shutting down... (send signal 2 more time(s) to force shutdown) [ 2020-10-12 00:00:14.3989 27184/7f58ea33c980 age/Ust/UstRouterMain.cpp:492 ]: Received command to shutdown gracefully. Waiting until all clients have disconnected... [ 2020-10-12 00:00:14.4002 27184/7f58ea23d700 Ser/Server.h:464 ]: [UstRouter] Shutdown finished [ 2020-10-12 00:00:14.4003 27184/7f58e99bb700 Ser/Server.h:817 ]: [UstRouterApiServer] Freed 0 spare client objects [ 2020-10-12 00:00:14.4004 27184/7f58e99bb700 Ser/Server.h:464 ]: [UstRouterApiServer] Shutdown finished [ 2020-10-12 00:00:14.4046 27179/7fa1ff678700 Ser/Server.h:817 ]: [ServerThr.1] Freed 128 spare client objects [ 2020-10-12 00:00:14.4056 27179/7fa1ff678700 Ser/Server.h:464 ]: [ServerThr.1] Shutdown finished [ 2020-10-12 00:00:14.4076 27184/7f58ea33c980 age/Ust/UstRouterMain.cpp:523 ]: Passenger UstRouter shutdown finished [ 2020-10-12 00:00:14.4455 27179/7fa1fff67980 age/Cor/CoreMain.cpp:967 ]: Passenger core shutdown finished Thank you everyone for all your hard work
sorry I should have mentioned ISPConfig 3.1.14p2 Debian 10 (Buster) Apache2, PHP7.3-FPM, MariaDB Hosting many websites without complaints
Do any cronjobs related to statistics or logs in sys_cron table in ispconfig database have the status running? If yes, delete these records from sys_cron table.
Yes sir, it looks like a lot of them are stuck! Would this be related to my old memory and CPU issues? Code: Edit Edit Copy Copy Delete Delete cronjob_awstats 2020-09-25 21:09:41 2020-09-27 00:00:00 1 Edit Edit Copy Copy Delete Delete cronjob_backup 2020-10-13 21:03:20 2020-10-15 00:00:00 0 Edit Edit Copy Copy Delete Delete cronjob_backup_mail 2020-10-13 21:03:20 2020-10-15 00:00:00 0 Edit Edit Copy Copy Delete Delete cronjob_bind_dnssec 2020-10-14 00:30:01 2020-10-15 03:30:00 0 Edit Edit Copy Copy Delete Delete cronjob_cleanup 2020-10-14 13:25:01 2020-10-14 16:30:00 0 Edit Edit Copy Copy Delete Delete cronjob_ftplogfiles 2020-10-13 21:03:16 2020-10-15 00:00:00 0 Edit Edit Copy Copy Delete Delete cronjob_letsencrypt 2020-10-14 00:00:02 2020-10-15 03:00:00 0 Edit Edit Copy Copy Delete Delete cronjob_logfiles 2020-10-13 21:03:16 2020-10-15 00:00:00 0 Edit Edit Copy Copy Delete Delete cronjob_mailbox_stats 2020-10-13 21:00:01 2020-10-15 00:00:00 0 Edit Edit Copy Copy Delete Delete cronjob_monitor_clamav_log 2020-09-02 13:35:17 2020-09-02 16:40:00 1 Edit Edit Copy Copy Delete Delete cronjob_monitor_cpu 2020-09-24 18:40:19 2020-09-24 21:45:00 1 Edit Edit Copy Copy Delete Delete cronjob_monitor_database_size 2019-12-06 17:24:56 2019-12-06 17:25:00 1 Edit Edit Copy Copy Delete Delete cronjob_monitor_disk_usage 2020-09-15 05:05:11 2020-09-15 08:10:00 1 Edit Edit Copy Copy Delete Delete cronjob_monitor_email_quota 2020-08-08 19:01:45 2020-08-08 22:15:00 1 Edit Edit Copy Copy Delete Delete cronjob_monitor_fail2ban 2020-08-14 11:00:30 2020-08-14 14:05:00 1 Edit Edit Copy Copy Delete Delete cronjob_monitor_hd_quota 2020-06-02 16:05:44 2020-06-02 19:10:00 1 Edit Edit Copy Copy Delete Delete cronjob_monitor_iptables 2020-04-06 17:06:14 2020-04-06 17:10:00 1 Edit Edit Copy Copy Delete Delete cronjob_monitor_ispconfig_log 2020-09-15 09:55:37 2020-09-15 13:00:00 1 Edit Edit Copy Copy Delete Delete cronjob_monitor_ispconfig_version 2020-10-14 13:25:01 2020-10-14 16:30:00 0 Edit Edit Copy Copy Delete Delete cronjob_monitor_kernel_version 2020-10-14 13:25:01 2020-10-14 16:30:00 0 Edit Edit Copy Copy Delete Delete cronjob_monitor_mail_log 2020-08-16 05:00:14 2020-08-16 08:05:00 1 Edit Edit Copy Copy Delete Delete cronjob_monitor_mail_queue 2020-05-25 14:41:40 2020-05-25 17:45:00 1 Edit Edit Copy Copy Delete Delete cronjob_monitor_mem_usage 2020-09-28 22:50:12 2020-09-29 01:55:00 1 Edit Edit Copy Copy Delete Delete cronjob_monitor_openvz 2020-10-14 13:25:01 2020-10-14 16:30:00 0 Edit Edit Copy Copy Delete Delete cronjob_monitor_os_version 2020-10-14 13:25:01 2020-10-14 16:30:00 0 Edit Edit Copy Copy Delete Delete cronjob_monitor_raid 2020-04-17 18:05:10 2020-04-17 18:10:00 1 Edit Edit Copy Copy Delete Delete cronjob_monitor_rkhunter 2020-10-13 21:00:06 2020-10-15 00:00:00 0 Edit Edit Copy Copy Delete Delete cronjob_monitor_server 2020-09-17 13:15:52 2020-09-17 16:20:00 1 Edit Edit Copy Copy Delete Delete cronjob_monitor_services 2020-08-17 09:07:25 2020-08-17 12:10:00 1 Edit Edit Copy Copy Delete Delete cronjob_monitor_syslog 2020-05-21 08:27:37 2020-05-21 11:30:00 1 Edit Edit Copy Copy Delete Delete cronjob_monitor_system_update 2020-06-10 06:00:04 2020-06-10 10:00:00 1 Edit Edit Copy Copy Delete Delete cronjob_openvz 2020-10-13 21:03:20 2020-10-15 00:00:00 0 Edit Edit Copy Copy Delete Delete cronjob_purge_mailboxes 2019-08-26 21:57:01 0000-00-00 00:00:00 0 Edit Edit Copy Copy Delete Delete cronjob_quota_notify 2020-10-13 21:03:20 2020-10-15 00:00:00 0 Edit Edit Copy Copy Delete Delete cronjob_webalizer 2020-10-13 21:03:16 2020-10-15 00:00:00 0 I'm going to find a way to monitor my statistics so it doesn't reoccur, I'll update tomorrow and see if that fixed it
You can basically empty (not delete) the whole sys_cron table to get rid of the stale cronjobs, ispconfig will fill it again. Yes, the issue is probably related to your out of memory issues.
