AWStats - Fix and Improvement (working)

Discussion in 'Developers' Forum' started by erosbk, Jul 29, 2011.

  1. erosbk

    erosbk New Member

    This will fix AWStats "last day" and will improve AWstats to keep track of every calculated month.

    1) New file index.php default for every site with AWStats
    Filename: /usr/local/ispconfig/server/conf/awstats/index.php.awstats
    Description: This file will create a dynamic "dropdown list" with year-month format, that you can select prev stats to see them. All directories in /stats/ are readed for generate the dropdown content, and current/prev stats are loaded in a frame. Will be copied to every vhost when user/admin select AWStats as default stat program.
    Advise:
    $aw['aw_jump_text'] = 'Jump to previous stats: '; <--------- This text can be changed to your language... could be improved in a future using ISPConfig Lang files structures...
    $aw['aw_renamed_index'] = 'fixedindex.html'; <--------- If you change this filename, remember to change it in "/usr/local/ispconfig/server/cron_daily.php" too, or this fix will not work.

    Content:

    Code:
    <script>
    function load_content(url)
    {
        var iframe = document.getElementById("content");
        iframe.src = url;
    }
    </script>
    
    <?php
    $aw['aw_jump_text'] = 'Jump to previous stats: ';
    $aw['aw_renamed_index'] = 'fixedindex.html';
    
    if ($handle = opendir('.'))
    {
            while(false !== ($file = readdir($handle)))
            {
                    if (substr($file,0,1) != "." && is_dir($file))
                    {
                            $orderkey = substr($file,0,4).substr($file,5,2);
                            if (substr($file,5,2) < 10 )
                            {
                                    $orderkey = substr($file,0,4)."0".substr($file,5,2);
                            }
                            $awprev[$orderkey] = $file;
                    }
            }
    
            $month = date("n");
            $year = date("Y");
    
            if (date("d") == 1)
            {
                    $month = date("m")-1;
                    if (date("m") == 1)
                    {
                            $year = date("Y")-1;
                            $month = "12";
                    }
            }
    
            $current = $year.$month;
            $awprev[$current] = $year."-".$month;
    
            closedir($handle);
    }
    
    echo '<div style="width: 97%; margin-left: 4px; height: 20px; background-color: #FFFFFF; position: fixed; padding: 7px; border: 2px solid #cccccc;><div align="left"><font color="#000000" size="2" face="Verdana, Arial, Helvetica,  sans-serif">' .$aw["aw_jump_text"]. '</font </div>';
    
    echo "<select name='awdate' onchange=\"load_content(this.value)\">";
    krsort($awprev);
    
    foreach ($awprev as $key => $value)
    {
            if($key == $current)
            {
                    echo "<option selected=\"selected\" value=\"".$aw['aw_renamed_index']."\"> $value</option>";
            }
            else
            {
                    echo "<option value='$value/".$aw['aw_renamed_index']."'> $value</option>";
            }
    }
    
    echo '</select></div><iframe src="'.$aw['aw_renamed_index'].'" frameborder="0" scrolling="Yes" width="100%" height="100%" style="margin-top:25px" id="content"></iframe>';
    
    ?>
    

    2) Code for AWStats must be changed for this to work (and for fixing "Last month day problem")
    Filename: /usr/local/ispconfig/server/cron_daily.php
    Description: this will force AWStats to calculated every time for a specific month/year, and not "current month/year". Default html file of awstats will be renamed as fixedindex.html instead index.html.

    Content (only AWStats part must be changed):

    Code:
    $sql = "SELECT domain_id, domain, document_root FROM web_domain WHERE stats_type = 'awstats' AND server_id = ".$conf['server_id'];
    $records = $app->db->queryAllRecords($sql);
    
    $web_config = $app->getconf->get_server_config($conf['server_id'], 'web');
    
    foreach($records as $rec) {
            //$yesterday = date('Ymd',time() - 86400);
            $yesterday = date('Ymd',strtotime("-1 day", time()));
            $logfile = escapeshellcmd($rec['document_root'].'/log/'.$yesterday.'-access.log');
            if(!@is_file($logfile)) {
                    $logfile = escapeshellcmd($rec['document_root'].'/log/'.$yesterday.'-access.log.gz');
                    if(!@is_file($logfile)) {
                            continue;
                    }
            }
    
            $domain = escapeshellcmd($rec['domain']);
            $statsdir = escapeshellcmd($rec['document_root'].'/web/stats');
            $awstats_pl = $web_config['awstats_pl'];
            $awstats_buildstaticpages_pl = $web_config['awstats_buildstaticpages_pl'];
    
            $awstats_conf_dir = $web_config['awstats_conf_dir'];
            $awstats_website_conf_file = $web_config['awstats_conf_dir'].'/awstats.'.$domain.'.conf';
    
            if(is_file($awstats_website_conf_file)) unlink($awstats_website_conf_file);
    
            if(!is_file($awstats_website_conf_file)) {
                    $awstats_conf_file_content = 'Include "'.$awstats_conf_dir.'/awstats.conf"
    LogFile="/var/log/ispconfig/httpd/'.$domain.'/yesterday-access.log"
    SiteDomain="'.$domain.'"
    HostAliases="www.'.$domain.' localhost 127.0.0.1"';
                    file_put_contents($awstats_website_conf_file,$awstats_conf_file_content);
            }
            if(!@is_dir($statsdir)) mkdir($statsdir);
            if(is_link('/var/log/ispconfig/httpd/'.$domain.'/yesterday-access.log')) unlink('/var/log/ispconfig/httpd/'.$domain.'/yesterday-access.log');
            symlink($logfile,'/var/log/ispconfig/httpd/'.$domain.'/yesterday-access.log');
    
    // BOC 1: We need to know which month/year must be used to calculate statistics.
    
    $awmonth = date("m"+1-1);
    $awyear = date("Y");
    
    if (date("d") == 1)
    {
            $awmonth = date("m")-1;
            if (date("m") == 1)
            {
                    $awyear = date("Y")-1;
                    $awmonth = "12";
            }
    }
    
    // EOC 1: We have month and year vars to calculate stats at this point, to fix the last day problem
    
            // awstats_buildstaticpages.pl -update -config=mydomain.com -lang=es -dir=/var/www/domain.com/web/stats -awstatsprog=/path/to/awstats.pl
            // $command = "$awstats_buildstaticpages_pl -update -config='$domain' -lang=es -dir='$statsdir' -awstatsprog='$awstats_pl'";
    
    // BOC 2: In day 2, we need to keep track of full stats for prev month, accesible for users when they need them.
    
            $command = "$awstats_buildstaticpages_pl -month='$awmonth' -year='$awyear' -update -config='$domain' -lang=es -dir='$statsdir' -awstatsprog='$awstats_pl'";
    
            if (date("d") == 2)
            {
                    $awmonth = date("m")-1;
                    if (date("m") == 1)
                    {
                            $awyear = date("Y")-1;
                            $awmonth = "12";
                    }
    
                    $statsdirold = $statsdir."/".$awyear."-".$awmonth."/";
                    mkdir($statsdirold);
                    $files = scandir($statsdir);
                    foreach ($files as $file)
                    {
                          if (substr($file,0,1) != "." && !is_dir($file) && substr($file,0,1) != "w" && substr($file,0,1) != "i") copy("$statsdir"."/"."$file","$statsdirold"."$file");
                    }
            }
    
    // EOC 2: At this point, current month will overwrite older stats, but a copy will be kept in a separate directory, accesible through index.php
    
            if($awstats_pl != '' && $awstats_buildstaticpages_pl != '' && fileowner($awstats_pl) == 0 && fileowner($awstats_buildstaticpages_pl) == 0) {
                    exec($command);
                    rename($rec['document_root'].'/web/stats/awstats.'.$domain.'.html',$rec['document_root'].'/web/stats/fixedindex.html');
                    $app->log('Created awstats statistics with command: '.$command,LOGLEVEL_DEBUG);
            } else {
                    $app->log("No awstats statistics created. Either $awstats_pl or $awstats_buildstaticpages_pl is not owned by root user.",LOGLEVEL_WARN);
            }
    
    }
    
    3) Every time AWStats is modified index.php must be copied to vhost /stats folder.
    Filename: /usr/local/ispconfig/server/plugins-available/apache2_plugin.inc.php
    Description: Everytime AWStats is selected for a vhost, or password for admin is changed (for vhosts stat program), this file must be copied to vhost path. Till recommended trigger this from this file (thanks xD)

    Content (At line 895, add line between "BOC 1 blocks" to "if" body)

    Code:
                    if($data["new"]["stats_type"] == 'awstats' && $data["new"]["type"] == "vhost") {
                            $this->awstats_update($data,$web_config);
    
    // BOC 1: If awstats, copy file index.php.awstats to stats folder
                            unlink($data['new']['document_root']."/web/stats/index.html");
                            copy("/usr/local/ispconfig/server/conf/awstats/index.php.awstats",$data['new']['document_root']."/web/stats/index.php");
    // BOC 1: If awstats, copy file index.php.awstats to stats folder
    
                    }
    
    4) When webalizer is set, index.php of awstats must be deleted, so, at line 134 you have to add the "unlink" command between code (in cron_daily.php):

    Code:
    
    ...
    
            $domain = escapeshellcmd($rec['domain']);
            $statsdir = escapeshellcmd($rec['document_root'].'/web/stats');
            $webalizer = '/usr/bin/webalizer';
            $webalizer_conf_main = '/etc/webalizer/webalizer.conf';
            $webalizer_conf = escapeshellcmd($rec['document_root'].'/log/webalizer.conf');
    
    // XXXXXXXXXXXXXXXX BOC 3: Delete index.php if awstats was working before
            unlink($statsdir.'/index.php');
    // XXXXXXXXXXXXXXXX BOC 3: Delete index.php if awstats was working before
    
            if(!@is_file($webalizer_conf)) {
                    copy($webalizer_conf_main,$webalizer_conf);
            }
    
    ...
    
    
    Tested and working in production server. No problems right now!!!

    If you have some errors, please post here (I am tired, maybe I forget to copy some chars... large lines are a problem with nano xD)

    Use this at your own risk! :p

    Best regards
     
    Last edited: Oct 3, 2011
  2. till

    till Super Moderator Staff Member ISPConfig Developer

    Thanks for the patch. Looks like a nice solution for the problem. I've added this to the bugtracker.
     
  3. erosbk

    erosbk New Member

    Glad to help :p

    You can see a testsite here to see it working (site with practacally no traffic, stats working, will test "last day" on day 2.

    http://www.tecatest.com.ar/stats/

    user: admin
    pass: a

    I will change pass tomorrow xD
     
  4. erosbk

    erosbk New Member

    Last day fix was tested and is working perfectly.

    In cron_daily.php, code must be change in segment "BOC/EOC 2".

    Code edited in post. (if day = 2...)
     
  5. erosbk

    erosbk New Member

    2 Ago 2011: Modifications (code in first post has been modified)

    cron_daily.php. Added check for not copy index.php, because is not needed.

    in index.php, month has now +1-1. If you use date("m") it will return 08, if you REST 1 to that date, it will cut off the zero and you will have 7... I am a little ignorant about how to resolve this, so... if sum/rest 1 to moth to everytime get month with one char and not with zero :p. This was making problems with drop down list.

    Please, be my guess to improve this code xD
     
  6. till

    till Super Moderator Staff Member ISPConfig Developer

    you couls also try this:

    $myvar = (int)date('m');

    or

    $myvar = intval(date('m');
     
  7. erosbk

    erosbk New Member

    I will try with your suggestions, but I have to think a little more about because the order in the drop down list... I want order desc for the list, if I use month of only 1 char, I will get a disorder with months 1,11,12,2,3,4,5,6...x

    Sorry, I have limited english xD

    Please help me with this... I need that, creating or modifying a client stats program, is AWStats is selected, file ".../ispconfig/server/conf/index.php.awstats" must be copied to "...clients/.../web/stats/".

    I can't find where is the function thats insert/update stat type in DB xD

    Thanks in advance
     
  8. till

    till Super Moderator Staff Member ISPConfig Developer

    The apache 2 plugin is executed when the stats type is changed. So the esaiest way is to add your code into the apache plugin in /usr/local/ispconfig/server/plugins-available/
     
  9. erosbk

    erosbk New Member

    Ok, looks like a final solution! be my guess to test it :p
     
  10. erosbk

    erosbk New Member

    The was a bug in the code, where if month one, it will be calculating all the time stats for month 12... code has been modified to fix this, just replace from BOC 1 to EOC 1

    This is the bug code:
    Code:
    // BOC 1: We need to know which month/year must be used to calculate statistics.
    
    $awmonth = date("m");
    
    if (date("d") == 1)
    {
            $awmonth = date("m")-1;
    }
    
    $awyear = date("Y");
    
    if (date("m") == 1)
    {
            $awyear = date("Y")-1;
            $awmonth = "12";
    }
    
    // EOC 1: We have month and year vars to calculate stats at this point, to fix the last day problem
    

    Replace with this code:
    Code:
    // BOC 1: We need to know which month/year must be used to calculate statistics.
    
    $awmonth = date("m"+1-1);
    $awyear = date("Y");
    
    if (date("d") == 1)
    {
            $awmonth = date("m")-1;
            if (date("m") == 1)
            {
                    $awyear = date("Y")-1;
                    $awmonth = "12";
            }
    }
    // EOC 1: We have month and year vars to calculate stats at this point, to fix the last day problem
    
    Original post has been updated with this changes.

    Regards
     
  11. till

    till Super Moderator Staff Member ISPConfig Developer

    I guess with the code:

    $awmonth = date("m"+1-1);

    you like to have the month without leading 0, so that it is 1 instead of 01? In that case it might be better to use this:

    $awmonth = date("n");
     
  12. erosbk

    erosbk New Member

    When change from Webalizer to AWStats (or AWStats to Webalizer), index.php and index.html must be deleted, if not deleted, one file has priority over the other...

    I will make the changes and post changes here tomorrow
     
  13. xaver

    xaver New Member

    hi,

    What will happen, if user has no php active?
    Maybe extra vhost entry?

    regards
    xaver
     
  14. erosbk

    erosbk New Member

    Good point, maybe with a htaccess it could be fixed, or be host must have settings to use mod_php for this in this directory...

    I will work on this...

    Regards!
     
  15. xaver

    xaver New Member

    Last edited: Sep 18, 2011
  16. erosbk

    erosbk New Member

    ** Update: Bug Fix **

    1) When awstats/webalizer are selected, older index.html of webalizer or index.php of awstats must be deleted in order to avoid conflict between them.

    So, when you select awstats, index.html is deleted (apache2_plugin.inc.php). If webalizer, index.php is deleted every time cron_daily.php is executed (cron_daily.php).

    2) index.php.awstats was modified to fix january month bug...

    Main post has been updated. Till, do you need modified the task in roadmap too?

    Regards
     
  17. till

    till Super Moderator Staff Member ISPConfig Developer

    Thanks for the update. I will add it to svn directly, no need to reopen the task in the bugtracker.
     
  18. zbuzanic

    zbuzanic Member

    I have problems with AWstats. On some domains it's working fine, on some gives 404, althought there is awstats *.html files in stats directory for all domains.
    How to check what went wrong?

    ps. using SVN
     
  19. erosbk

    erosbk New Member

    AWStats index.php (as posted) search for fixedindex.html to show stats. If the file is named differently, it will not find the file and will return 404.

    I don't have the svn version to test how the code was implemented...

    In the mean while, could you post error log here of one of your domains with the problem?

    Regards
     
  20. zbuzanic

    zbuzanic Member

    My bad, it seems there was not enough data to generate stats after 3 days
     

Share This Page