Per-vhost error logs empty after nightly rotation since last ISPConfig update

Since updating ISPConfig (~1 week ago), per-vhost Apache error logs (/var/log/ispconfig/httpd/<domain>/error.log) stopped receiving entries after the nightly log rotation. The same behavior is observed on all servers where the update was applied. On some servers we only have logs during 00:00 and 00:05, on some other we only have logs between 00:00 and 02:00 AM.

Observed behavior:
- Rotated error log files now only contain entries from midnight to ~2 AM (2 hours), whereas before the update they contained a full 24 hours (2 AM to 2 AM)
- The new error.log is created correctly with proper permissions (root:clientX 640) but remains empty
- Access logs are unaffected
- No Apache/PHP/MariaDB crashes, site fully accessible

Root cause (suspected):
Apache is not being signaled to reopen its log files after rotation. It keeps the old file descriptor pointing to the rotated/compressed file and falls back to the generic error log.

Workaround:
Manually restarting apache2 after the rotation immediately restores logging to the correct per-vhost file.

Has this been identified as a known regression? Is there a fix or updated rotation script available?

 

That's weird, there wasn't any other planned operation last week. The only other thing we only did our weekly update (via apt).
How could I investigate further on this issue ? From what I understand, error log rotation is managed through ispconfig cron (checked ispconfig/cron.log, nothing here).

 

From what I understand, Apache is still writing to the deleted error log file. I'm see the "deleted" status in `lsof` output. This can happen when a log file is rotated or deleted while Apache is still running and has the file open.

As far as I understand the issue :
- 02:00 AM : the error logrotate rotates the log files and deletes the old ones, deleting the file that Apache is still writing to
- 00:00 AM (the next day) : the access log rotation properly restarts Apache, fixing the issue for the error log as well
- 02:00 AM (the next day) : the error logrotate rotates the log files again, keeping the error that occured between 00:00AM and 2:00AM only. Then the cycle continues.

Maybe I'm wrong but I don't see where 200-logfiles.inc.php is gracefully restarting Apache after log rotation.
The error.log rotation section uses gzip without the -c flag, which deletes the original file but Apache is never signaled to reopen its log files:

Code:

if(is_file($error_logfile)) {
    $app->system->exec_safe("gzip ?", $error_logfile);       // deletes original file
    rename($error_logfile . '.gz', $error_logfile . '.1.gz');
    $app->system->chgrp($error_logfile . '.1.gz', $rec['system_group']);
    $app->system->chmod($error_logfile . '.1.gz', 0640);
    $app->system->exec_safe("cat /dev/null > ?", $error_logfile); // creates new file
    $app->system->chgrp($error_logfile, $rec['system_group']);
    $app->system->chmod($error_logfile, 0640);
    // ← no apache2ctl graceful or SIGUSR1 here
}

Compare this with the cron.log rotation in the same file, which correctly uses gzip -c to keep the original file and then truncates it in place — meaning Apache's file descriptor remains valid and no signal is needed

Additionnal logs for context :

Code:

root@xxx:/# lsof -c apache2 | grep -i error
apache2 73658 www-data 2w REG 9,5 3053 12583100 /var/log/apache2/error.log
apache2 73658 www-data 13w REG 9,5 20248 12583170 /var/log/ispconfig/httpd/xxx.com/error.log (deleted)
apache2 1775325 root 2w REG 9,5 3053 12583100 /var/log/apache2/error.log
apache2 1775325 root 13w REG 9,5 20248 12583170 /var/log/ispconfig/httpd/xxx.com/error.log (deleted)
apache2 4046395 www-data 2w REG 9,5 3053 12583100 /var/log/apache2/error.log
apache2 4046395 www-data 13w REG 9,5 20248 12583170 /var/log/ispconfig/httpd/xxx.com/error.log (deleted)
apache2 4046397 www-data 2w REG 9,5 3053 12583100 /var/log/apache2/error.log
apache2 4046397 www-data 13w REG 9,5 20248 12583170 /var/log/ispconfig/httpd/xxx.com/error.log (deleted)
apache2 4048140 www-data 2w REG 9,5 3053 12583100 /var/log/apache2/error.log
apache2 4048140 www-data 13w REG 9,5 20248 12583170 /var/log/ispconfig/httpd/xxx.com/error.log (deleted)

Code:

root@xxx:/# ls -lah /var/log/ispconfig/httpd/xxx.com/
total 162M
drwxr-x--- 2 root client1 4.0K Feb 23 01:11 .
drwxr-xr-x 4 root root 4.0K Dec 8 15:45 ..
-rw-r----- 1 root client1 9.5M Feb 14 00:53 20260212-access.log.gz
-rw-r----- 1 root client1 14M Feb 15 00:54 20260213-access.log.gz
-rw-r----- 1 root client1 2.9M Feb 16 00:55 20260214-access.log.gz
-rw-r----- 1 root client1 2.7M Feb 15 23:59 20260215-access.log.gz
-rw-r----- 1 root client1 5.3M Feb 16 23:59 20260216-access.log.gz
-rw-r----- 1 root client1 9.6M Feb 17 23:59 20260217-access.log.gz
-rw-r----- 1 root client1 13M Feb 18 23:59 20260218-access.log.gz
-rw-r----- 1 root client1 28M Feb 19 23:59 20260219-access.log.gz
-rw-r----- 1 root client1 14M Feb 20 23:59 20260220-access.log.gz
-rw-r----- 1 root client1 5.9M Feb 21 23:59 20260221-access.log.gz
-rw-r----- 1 root client1 32M Feb 22 23:59 20260222-access.log
-rw-r----- 1 root client1 27M Feb 23 14:21 20260223-access.log
lrwxrwxrwx 1 root root 19 Feb 23 00:00 access.log -> 20260223-access.log
-rw-r----- 1 root client1 0 Feb 23 01:11 error.log
-rw-r----- 1 root client1 683 Feb 23 01:07 error.log.1.gz
-rw-r----- 1 root client1 9.1K Feb 14 00:53 error.log.10.gz
-rw-r----- 1 root client1 771 Feb 22 01:05 error.log.2.gz
-rw-r----- 1 root client1 762 Feb 21 00:50 error.log.3.gz
-rw-r----- 1 root client1 913 Feb 20 01:02 error.log.4.gz
-rw-r----- 1 root client1 830 Feb 19 01:03 error.log.5.gz
-rw-r----- 1 root client1 1.1K Feb 18 01:09 error.log.6.gz
-rw-r----- 1 root client1 17K Feb 17 00:51 error.log.7.gz
-rw-r----- 1 root client1 21K Feb 16 00:55 error.log.8.gz
-rw-r----- 1 root client1 20K Feb 15 00:54 error.log.9.gz
lrwxrwxrwx 1 root root 53 Feb 23 00:01 yesterday-access.log -> /var/www/clients/client1/web1/log/20260222-access.log

 

Yes, it probably has something to do with the packages update if the script did not change recently. My mistake for pointing the ISPConfig update : I was the first thing that came to mind since I only do them every few months.

I'm running ISPConfig on 20~30 servers (all Debian 12), some of them are a few years old but the logs in my previous post are from a server that was just set up in december. Checked 5 other servers and they all have the same issue with error logs.

Anyway, I can assure you I did not do anything else on this server last week. Nor did anyone else.

EDIT: It seems I cannot edit the topic title to remove the misleading ISPConfig update mention.

 

Thanks a lot for the quick replies.

Let me know if you need anything else like further testing, logs, etc.

 

I'm pushing this modification on one of my servers. I'll let you know tomorrow if it worked.