mail.log rotation

I'm sorry for crossposting, but I need a solution to a problem induced by ISPConfig.

I'm using Debian Etch with ISPC and Postfix on which by default the mail logs are rotated by syslog's cronjob around 06:30 every morning. Problem is, that at 23:00 an ISPC cronjob starts mail_logs.php which does something to mail.log (gather statistics and stuff), copying it from here to there, starting a new one, ... pretty weird and not really documented stuff.

Question:
Can't you use something more spohisticated, leaving the original log files as they are, so ISPC's cron job does not interfer with other cron jobs?

In the meantime:
Is mail_logs.php something so cool I wouldn't want to live without it running every night or may I just throw it out of my crontab?

Thx

 

You can remove it from your crontab if you don't care about email traffic statistics in ISPConfig.

 

That's cool, thank you!

BTW, do you plan to redesign this part in ISPConfig 3?

 

Traffic Statistics

What traffic statistics, how do I get that statistics you speak about?

 

That statistics for mail is in any case totally not functioning and soe time show zero value, the reason is that ISP rotate the logfiles at mindnite, causing syslog sometime NOT to work at all till the next day 06:23! when syslog can restart it. I saw these gaps in logfiles as I analyse them with some scripts but found them unreliable and full of gaps! Thus far I could not detect the villan! Gotcha!

 

Would just like to add...

We are also experiencing this problem. Each day the mail.log gets rotated at 23:59 by ISPConfig and then again by logrotate at 06:25. We're using Debian Etch, Postfix (from etch) and ISPConfig 2.2.13.

Is there a simple way to modify the script to work on a temporary copy? I would rather not deactivate the traffic statistics, customers would probably complain.

Also the mail_logs.php script won't compress old logs, this seems a bit wasteful. I noticed my maillogs for this machine are above 5 GB since about may I do monitor so i would've caught this before it became a problem, but i tend to assume logs are rotated correctly.

Kind regards
Andreas Wettergren

 

I've added this to our bugtracker.

 

I too am having problems. All of my rotated mail.log files start at 23:59 and end at 06:25. It is as if ISPC deletes all prior content in the log file when it analyses it at 23:59. Till, falko, any suggestions?
Regards,
Lewis

 

Eventually, I took the time to investigate this a bit more. My Debian box with ISPconfig is running Debian 4.0 "Etch" (pretty much a default installation) with a current ISPconfig v2.2.19.

The problem persists: /var/log/mail.log.0 shows only lines from midnight until approximately 6:30. The reason is, vaguely put, a conflict between default Debian log file rotation and ISPconfig, more specifically its cronjob

Code:

59 23 * * * /root/ispconfig/php/php /root/ispconfig/scripts/shell/mail_logs.php

BTW, if you miss your data have a look at the other mail.* log files, especially /var/log/mail.info

Whether your Debian log file rotation kicks in or not you can check with

Code:

syslogd-listfiles --weekly

It returns all files which get rotated weekly by means of /etc/cron.weekly/sysklogd which again, on my system, is run every Sunday morning at 6:47 (see /etc/crontab).

But the main problem I see -- sorry to say this -- in ISPconfig's /root/ispconfig/scripts/shell/mail_logs.php -- though the whole issue is definitely not trivial. Anyway, from my point of view, mail_logs.php needs some rethinking.

As till wrote, mail_logs.php is working on a temporary copy. I guess he referred to line 53

Code:

$mod->log->caselog("cp -f $dist_mail_log $dist_mail_log.$datum", $FILE, __LINE__);

So this is true. But a few lines further below the original data is overwritten with

Code:

$fp = fopen($dist_mail_log, "w");
fwrite($fp, "");
fclose($fp);

There our data is lost. mail_logs.php deletes everything until midnight.

I do not know enough about ISPconfig but I am afraid this here is bound to break on systems with log rotation that occurs during the day. I guess it also means that ISPconfig's mail stats are not accurate. Certainly, just disabling the lines which erase $dist_mail_log does not help.

Nevertheless, maybe there is a simpler fix since till reported that he has never seen this problem, probably for a reason.

HTH,
-- xrat

P.S.: In mail_logs.php there are references to $datum2 which I assume does not exist/is empty. It does not break the script but I recommend to fix it anyway should the code ever be used somewhere else.

 

A quick fix

When I wrote this I missed the fact that mail.log is also rotated on a daily basis at 6:25 by means of /etc/crontab -> /etc/cron.daily/sysklogd

A quick fix to the issues pointed out above is the following:

Make syslog to log all mail.* data to another log file which is then used only by ISPconfig. E.g. add to /etc/syslog.conf the line

Code:

mail.*    -/var/log/mail.log4ispc

Create a copy of the already existing log file with the new name and reload syslog's configuration with

Code:

cd /var/log
cp -a mail.log mail.log4ispc && /etc/init.d/sysklogd reload

Then change the log file name in ISPconfig. Login as admin and go to Management > Server > Settings. On tab "EMail" change field "Log File" from (I suppose, at least for Debian) /var/log/mail.log to /var/log/mail.log4ispc

Note that with these settings also /var/log/mail.log4ispc will get automagically rotated by Debian's sysklogd but only weekly. Thus, every Sunday the mail stats of ISPconfig will miss 6 hours.

Please check yourself if the here suggested changes are suitable for your system.

HTH,
-- xrat

 

Proper Cron!

Best practice would be to include the cronjobs into the /etc/cron.daily, /etc/cron.weekly etc.
If we can get all the ispc cron scripts under /etc/cron.daily that process all the daily ispc activity then the system will take care of the timing and rotations will be synchronized.

 

Is there a simple fix possible for /root/ispconfig/scripts/shell/mail_logs.php?

Lewis

 

I am having this same problem.
My mail logs do not include a full 24 hour day.

Is there a fix in the works?

 

Dude, it's up there!

 

By your own words xrat, the fix improves the situation, but does not fully rectify the problem.

Regards,

Lewis

 

Dude...that's a hack. Not a fix.

A fix means that the bug in ISPConfig is addressed so the problem does not occur.