Postfix Monitoring With Mailgraph And pflogsumm

sysklogd

Good Day Falko,

I have confirmed that every hour, on the hour, (or 1 sec past) logging stops until I issue /etc/init.d/syslogd stop then syslogd -u syslog -a /var/lib/named/dev/log to restart it.

There is nothing in crontab that indicates there is a job every hour, cron.hourly is empty.

Here is /etc/init.d/sysklogd

#! /bin/sh
# /etc/init.d/sysklogd: start the system log daemon.

PATH=/bin:/usr/bin:/sbin:/usr/sbin

pidfile=/var/run/syslogd.pid
binpath=/sbin/syslogd

test -x $binpath || exit 0
. /lib/lsb/init-functions

# Options for start/restart the daemons
# For remote UDP logging use SYSLOGD="-r"
#
#SYSLOGD="-u syslog"
SYSLOGD="-u syslog -a /var/lib/named/dev/log"

create_xconsole()
{
if [ ! -e /dev/xconsole ]; then
mknod -m 640 /dev/xconsole p
else
chmod 0640 /dev/xconsole
fi
chown root:adm /dev/xconsole
}

running()
{
# No pidfile, probably no daemon present
#
if [ ! -f $pidfile ]
then
return 1
fi

pid=`cat $pidfile`

# No pid, probably no daemon present
#
if [ -z "$pid" ]
then
return 1
fi

if [ ! -d /proc/$pid ]
then
return 1
fi

cmd=`cat /proc/$pid/cmdline | tr "\000" "\n"|head -n 1`

# No syslogd?
#
if [ "$cmd" != "$binpath" ]
then
return 1
fi

return 0
}

case "$1" in
start)
log_begin_msg "Starting system log..."
create_xconsole
start-stop-daemon --start --quiet --exec $binpath -- $SYSLOGD
log_end_msg $?
;;
stop)
log_begin_msg "Stopping system log..."
start-stop-daemon --stop --quiet --oknodo --exec $binpath --pidfile $pidfile
log_end_msg $?
;;
restart|force-reload|reload-or-restart|reload)
log_begin_msg "Restarting system log..."
start-stop-daemon --stop --quiet --exec $binpath --pidfile $pidfile
sleep 1
start-stop-daemon --start --quiet --exec $binpath -- $SYSLOGD
log_end_msg $?
;;
*)
log_success_msg "Usage: /etc/init.d/sysklogd {start|stop|reload|restart|force-reload|reload-or-restart}"
exit 1
esac

exit 0

darcy

 

Is cron job causing problem?

Further to the above, I had two ssh seesions open. I was tail-f /var/log/mail.log in one and tail -f /var/log/cron.log in the other.

as soon as these fired logging stopped in mail.log

Jan 3 10:00:01 vmail /USR/SBIN/CRON[14112]: (root) CMD (if [ -x /usr/bin/mrtg ] && [ -r /etc/mrtg.cfg ]; then env LANG=C /usr/bin/mrtg /etc/mrtg.cfg >> /var/log/mrtg/mrtg.log 2>&1; fi)
Jan 3 10:00:01 vmail /USR/SBIN/CRON[14114]: (root) CMD (/etc/webmin/webalizer/webalizer.pl /var/log/apache2/access.log)
Jan 3 10:00:01 vmail /USR/SBIN/CRON[14116]: (root) CMD (/etc/webmin/bandwidth/rotate.pl)




How can I issolate which one of these is causing the problem. Where are these cron jobs being launched from?

WaveQam

 

Found the culpret

OK, I nailed down the culpret that was killing my logging.

/etc/webmin/bandwidth/rotate.pl

#!/usr/bin/perl
open(CONF, "/etc/webmin/miniserv.conf");
while(<CONF>) {
$root = $1 if (/^root=(.*)/);
}
close(CONF);
$ENV{'WEBMIN_CONFIG'} = "/etc/webmin";
$ENV{'WEBMIN_VAR'} = "/var/webmin";
chdir("$root/bandwidth");
exec("$root/bandwidth/rotate.pl", @ARGV) || die "Failed to run $root/bandwidth/rotate.pl : $!";


I did a locate for rotate.pl and found
/etc/webmin/bandwidth/rotate.pl
/usr/share/webmin/bandwidth/rotate.pl

Don't know why this was causing my syslog to crash, but I guess I will be purging it from my system

WaveQam

 

Hi!

I just started to use pflogsumm on Debian Etch with ISPConfig, followin the howto. My problem is, that my logs all rotated logs start at 23:00 and end at somewhat between 06:00 and 07:00 (usually 06:26). The currently active log starts between 06:00 and 07:00.

The only cron jobs I can find running exactly 23:00 are ISPConfig's, but I could not find anything in them that seemed to rotate postfix' logs.

Right now I'm only getting a mail with stats from around 23 till 06xx

Output of crontab -l

Code:

#min hour daymo month daywk cmd
30 00 * * * /root/ispconfig/php/php /root/ispconfig/scripts/shell/logs.php &> /dev/null
59 23 * * * /root/ispconfig/php/php /root/ispconfig/scripts/shell/ftp_logs.php &> /dev/null
59 23 * * * /root/ispconfig/php/php /root/ispconfig/scripts/shell/mail_logs.php &> /dev/null
59 23 * * * /root/ispconfig/php/php /root/ispconfig/scripts/shell/cleanup.php &> /dev/null
0 4 * * * /root/ispconfig/php/php /root/ispconfig/scripts/shell/webalizer.php &> /dev/null
0,30 * * * * /root/ispconfig/php/php /root/ispconfig/scripts/shell/check_services.php &> /dev/null
15 3,15 * * * /root/ispconfig/php/php /root/ispconfig/scripts/shell/quota_msg.php &> /dev/null
40 00 * * * /root/ispconfig/php/php /root/ispconfig/scripts/shell/traffic.php &> /dev/null
05 02 * * * /root/ispconfig/php/php /root/ispconfig/scripts/shell/backup.php &> /dev/null
36 03 * * * /root/backup.sh &> /dev/null
01 04 * * * /root/cleanisptemp.sh &> /dev/null
10 * * * * /root/drupal-cron.sh &> /dev/null
0 1 * * * /usr/local/bin/sa-update &> > /dev/null
1 1 * * * /usr/local/sbin/rules_du_jour &> > /dev/null
0 7 * * * /usr/local/sbin/postfix_report.sh &> /dev/null

logrotate.conf :

Code:

# see "man logrotate" for details
# rotate log files weekly
weekly

# keep 4 weeks worth of backlogs
rotate 4

# create new (empty) log files after rotating old ones
create

# uncomment this if you want your log files compressed
#compress

# packages drop log rotation information into this directory
include /etc/logrotate.d

# no packages own wtmp, or btmp -- we'll rotate them here
/var/log/wtmp {
    missingok
    monthly
    create 0664 root utmp
    rotate 1
}

/var/log/btmp {
    missingok
    monthly
    create 0664 root utmp
    rotate 1
}

# system-specific logs may be configured here

/var/log/mail.log {
    missingok
    daily
    rotate 7
    create
    compress
    start 0
}

postfix_report.sh :

Code:

#!/bin/sh
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin

zcat /var/log/mail.log.0.gz | pflogsumm | formail -c -I"Subject: Mail Statistics for srv001" -I"From: [email protected]" -I"To: [email protected]" -I"Received: from srv001.mydomain.tld" | sendmail [email protected]

exit 0

 

I gave up on ISP Config

I gave up on ISP config, Mailgraph seems to be working correctly - never did get proper reporting from pflogsum

WaveQam

 

Ok, I'm currently on my second attempt to explore how things are working. Seems after the years where I had less to do with administration, I've gotten a bit rusty...

Normal rotation in /var/log is done via /etc/cron.daily/sysklogd

Changing

Code:

for LOG in `syslogd-listfiles`

into

Code:

for LOG in `syslogd-listfiles -s "mail.*"`

disables the default rotation for the mail-logs (mail.info, mail.log, mail.warn, mail.err).

Now we have no rotation at all, except of mail.log's rotation, as described in the howto at the bottom of /etc/logrotate.conf . What's missing now is rotation setting for the other logfiles. Because I like to have it all in one place and sorted, I move the howto's config out of logrotate.conf .

Instead I create a file /etc/logrotate.d/postfix and put this in it:

Code:

"/var/log/mail.log" /var/log/mail.err /var/log/mail.info /var/log/mail.warn {
  missingok
  daily
  rotate 7
  create 640 root adm
  compress
  start 0
  postrotate
        /etc/init.d/postfix reload > /dev/null
  endscript
}

Whether this works or not, tomorrow I will know for sure

P.S.:
If you're using spamassassin it will complain about a lot of stuff in the report (because it contains a lot of spamner's addresses and such stuff). For not having to scan your junk mail folder for the report it makes perfectly sense to add "whitelist_from [email protected]" to your local.cf . Better do not configure this address as real account on your system or otherwise you may end up receiving real spam sent to this address.

 

does it work ?

 

Not really - The data seems out of wack

 

Its been a while - I guess I should have reviewed the thread before responding. mailgraph seems to be working great, pflogsum does not.

WaveQam