Marge apache logs in cluster environment

Discussion in 'Installation/Configuration' started by diablo666, May 5, 2014.

  1. diablo666

    diablo666 Active Member HowtoForge Supporter

    In a cluster environment with a loadbalancer bechind nodes, 've the problem that log access for stats are splitted in each node.

    There is a way a to log or merge the log before the generation of the daily stats?
     
  2. till

    till Super Moderator Staff Member ISPConfig Developer

  3. diablo666

    diablo666 Active Member HowtoForge Supporter

    Assuming the perfect setup for cluster, with scp available without password...

    Let's say somethink like this?

    Code:
    #!/bin/bash
    server1=xxx.xxx.xxx.xxx
    server2=yyy.yyy.yyy.yyy
    
    tempdir=/tmp/stats
    
    if [ ! -e $tempdir ]; then mkdir $tempdir; fi
    
    for f in $( ls /var/www/ ); do
            cp "/var/www/"$f"/log/access.log" "/tmp/stats/"$f".access.log."$server1;
            scp "root@"$server2":/var/www/"$f"/log/access.log" "/tmp/stats/"$f".access.log."$server2;
            /usr/share/awstats/tools/logresolvemerge.pl "/tmp/stats/"$f".access.log.*" > "/var/www/"$f"/log/access.log";
    done
    Put that in crontab about 0:05 ?
     
  4. diablo666

    diablo666 Active Member HowtoForge Supporter

    This is a little improved version, using only logs for websistes and generating no error for other folders like apps, clients and similar.

    Skip also index.html

    Code:
    #!/bin/bash
    server1=xxx.xxx.xxx.xxx
    server2=yyy.yyy.yyy.yyy
    
    tempdir=/tmp/stats
    
    if [ ! -e $tempdir ]; then mkdir $tempdir; fi
    
    for f in $( ls /var/www/*\.* -d | grep -v index.html ); do
            site=$( basename $f);
            cp "/var/www/"$site"/log/access.log" "/tmp/stats/"$site".access.log."$server1;
            scp "root@"$server2":/var/www/"$site"/log/access.log" "/tmp/stats/"$site".access.log."$server2;
            /usr/share/awstats/tools/logresolvemerge.pl "/tmp/stats/"$site".access.log.*" > "/var/www/"$site"/log/access.log";
    done
    I'll try to schedule every night at 0:05 and chek what heappens
     

Share This Page