Awstats mod - uses dynamic pages (non static)

Discussion in 'Tips/Tricks/Mods' started by djtremors, Nov 8, 2006.

  1. djtremors

    djtremors ISPConfig Developer ISPConfig Developer

    did you delete

    /root/ispconfig/scripts/shell/awstats.php
    and remove the .htaccess file from all vhost sites as this is where the redirection is done?
     
  2. Ovidiu

    Ovidiu Active Member

    thx found and removed everything for now.
     
  3. Norman

    Norman Member HowtoForge Supporter

    I can invoke awstats.pl properly from commandline and it works fine to generate static pages. For some reason apache2 fails and gives me the 500 error msg.

    Any ideas?
    Did I miss out on any config in apache?
    Mod_perl has been enabled as module also.
     
  4. radim_h

    radim_h Member HowtoForge Supporter

    ISPc update

    what will happen with installed awstats in case of upgrade ISPconfig ?
    (i'd like to update from 2.2.11 to 2.2.13

    i have also installed DJ tremors patch "Make ISPConfig tree sticky between refreshes"

    will both of them still working ?
     
  5. djtremors

    djtremors ISPConfig Developer ISPConfig Developer

    they will both be erased (at least the sticky patch as it overwrites the core ispconfig files). ISPC deleted the /home/admispconfig folder from memory (can't remember as I develop on DTC from gplhost.com now)

    ISPC will most likely add my sticky tree in their main code as it's something a lot of people want and is kinda annoying to see collapse every time.
    My sticky doesn't allow the full collapse and full expand buttons to work but since the stickys work, it was enough for me anyway.
     
  6. djtremors

    djtremors ISPConfig Developer ISPConfig Developer

    500 error is an internal error. this usually means the .htaccess file has non allowed apache directives. check your "errors" log file for the virtual host or even the main apache error logs for the answers.
     
  7. radim_h

    radim_h Member HowtoForge Supporter

    problem after ispc update

    so, i have updated to 2.2.13 which soved my problem with hostingplan tepmplates

    but awstats stopped create /web/stats directory and also .htpasswd files for new webs and it worked for formerly created sites

    i also tried to run instalaltion of your awstats again, but nothing changed


    also weird thing is that files for 06/2007
    /home/admispconfig/ispconfig/web/ispc.awstats.data are created with owner root:root mod 644 instead of admispconfig:admispconfig mod 755

    stats for old webs are stil properly working, not for new ones

    please, djtremors do you have idea what to check on the server ??

    That's all waht installation said:
    [root@server2 ispc.awstats-install]# ./install.sh
    Extracting awstats files and configs
    Patching httpd.conf
    Found: Redhat compatible conf.d path
    Adding crontab settings
    WARNING: Crontab entry exists in /var/spool/cron/root. Check to be sure

    Please restart your Apache server for changes to take effect
    All websites stats are accessed via {website}/stats
    Thanks to 'martinfst' for bug fixes/notes on changes
     
    Last edited: Jun 7, 2007
  8. djtremors

    djtremors ISPConfig Developer ISPConfig Developer

    you have to be careful when reinstalling. I do not make checks for things like if the /etc/awstats folder existing to not copy the rest over. I know thats slack on my part but it was how it was done before i realised ISPC deleted all the files on upgrades.

    /home/admispconfig/ispconfig/web/ispc.awstats.data with files of 644 is correct. 755 is usually directories, those files are just data and don't execute anything so 755 is not needed on the data files.

    Are all your data files for awstats intact?

    Have you checked what happens when you run the crontab command?
    /root/ispconfig/php/php /root/ispconfig/scripts/shell/awstats.php
    (check the output).
     
  9. radim_h

    radim_h Member HowtoForge Supporter

    haleluja .) it started working... i just forget filled admin/password i was sure i did it :)
    but, does it something have to do with any other program running at midnight?
    i think it always appear after midnight

    i'm currios what was that cause of working stopped after upgrade... still don't understand mechanism witch creates .htpasswd files...
    because when ispconfig left all config files on place, it had to work even after upgrade...
     
  10. meemu

    meemu ISPConfig Developer ISPConfig Developer

    awstats and suexec

    In case you still want to get this to work here are a couple of starting points:

    * set up a dedicated vhost for awstats - e.g. awstats.localhost.localdomain with a config like this:

    Code:
            ...
            DocumentRoot /var/www/share/awstats
            SuexecUserGroup awstats awstats
            ScriptAlias / /var/www/share/awstats/
            <Directory /var/www/share/awstats>
                    Options None
                    Options ExecCGI -MultiViews +SymLinksIfOwnerMatch
                    AllowOverride None
                    Order deny,allow
                    deny from all
                    allow from 127.0.0.1
            </Directory>
            ...
    
    * setup a dedicated user/group for awstats
    * modify ISPConfig to add the awstats user to each web site group (in case you want to keep awstats data in the users' sites)

    * enable mod_proxy (a2enmod mod_proxy) and mod_proxy_http and alter the configuration

    /etc/apache2/mods-enabled/proxy.conf
    Code:
            ProxyRequests Off
            ProxyVia On
            <Proxy *>
                    AddDefaultCharset off
                    Order deny,allow
                    Allow from all
            </Proxy>
    
    * add rewrite rules to the vhosts that have awstats enabled
    Code:
    ...
    RewriteEngine On
    RewriteRule ^/awstats$ http://{SERVER_NAME}/awstats/ [R]
    RewriteRule ^/awstats/$ http://awstats.localhost.localdomain/awstats.pl?config={SERVER_NAME} [NE,QSA,NS,P,L]
    RewriteRule ^/awstats/(.+)$ http://awstats.localhost.localdomain/$1 [NE,QSA,NS,P,L]
    ...
    
    * create a cron script to update awstats after log processing is finished (you should do DNS resolution before, e.g. in webalizer.php)


    /root/ispconfig/scripts/shell/awstats.php
    Code:
    <?
    set_time_limit(0);
    
    include("/root/ispconfig/scripts/lib/config.inc.php");
    include("/root/ispconfig/scripts/lib/server.inc.php");
    
    $server_id = $mod->system->server_id;
    $server = $mod->system->server_conf;
    $path_httpd_root = stripslashes($server["server_path_httpd_root"]);
    
    $web_home = $path_httpd_root;
    
    $message = "";
    $handle = @opendir($web_home);
    while ($dir = @readdir ($handle)) {
        if ($dir != "." && $dir != "..") {
            if(@!is_link("$web_home/$dir") && substr($dir,0,3) == "web" && is_numeric(substr($dir,3))) {
            // ist kein symbolischer Link
                $trigger = "$web_home/$dir/.awstats";
                if(!file_exists($trigger))
                    continue;
                $webname = $dir;
                $web_path = $web_home . "/$webname/web";
                $stats_path = $web_path ."/awstats";
                $logfile = $web_home . "/$webname/log/web.log";
                $web_user = fileowner($web_path);
                $web_group = filegroup($web_path);
                // erstelle Stats Verzeichnis, wenn nicht vorhanden
                if(!@is_dir($stats_path)) {
                    mkdir($stats_path,0750);
                    chown($stats_path,$web_user);
                    chgrp($stats_path,$web_group);
                    $message .= "Erstelle Statistik Verzeichnis: $stats_path\n";
                }
                $web_doc_id = str_replace("web", "", $webname);
                $web_data = $mod->db->queryOneRecord("SELECT * FROM isp_isp_web WHERE doc_id = '$web_doc_id'");
                if(!empty($web_data["web_host"])){
                   $web_real_name = $web_data["web_host"].".".$web_data["web_domain"];
                } else {
                   $web_real_name = $web_data["web_domain"];
                }
                if(@is_file($logfile)) {
                    if ( !@is_file("/etc/awstats/awstats.".$web_real_name.".conf") ) {
                            exec( "echo 'Include \"/etc/awstats/awstats.model.conf\"' > /etc/awstats/awstats.".$web_real_name.".conf" );
                            exec( "echo 'LogFile=\"".$logfile."\"' >> /etc/awstats/awstats.".$web_real_name.".conf" );
                            exec( "echo 'SiteDomain=\"".$web_real_name."\"' >> /etc/awstats/awstats.".$web_real_name.".conf");
                            exec( "echo 'DirData=\"".$stats_path."\"' >> /etc/awstats/awstats.".$web_real_name.".conf");
                    }
                    $message .= exec("/var/www/share/awstats/awstats.pl -config=".$web_real_name." -Logfile=".$logfile." -update")."\n";
                }
    
    
                exec("chown -R $web_user:$web_group $stats_path &> /dev/null");
    
            }
        }
    }
    
    
    All done!
     
  11. radim_h

    radim_h Member HowtoForge Supporter

    just quick dumb question .)

    is there some reason to run both scripts in 2 hours difference as
    0 4 * * * /root/ispconfig/php/php /root/ispconfig/scripts/shell/awstats.php
    0 6 * * * /home/admispconfig/ispconfig/web/ispc.awstats/awstats_updateall.pl now -awstatsprog="/home/admispconfig/ispconfig/web/ispc.awstats/awstats.pl"

    or should i run them both at once from bash script?
     
  12. till

    till Super Moderator Staff Member ISPConfig Developer

    I have not written the awstats extension, but I guess that the first script must finish before the second script starts. I recommend to leave it as it is.
     
  13. radim_h

    radim_h Member HowtoForge Supporter

    global awstats admin password added everyday again ang again to .htpasswd

    Hello DJtremors,
    i'm going to bother you again .)

    i have noticed little problem, since you added that super option for global awstatsadmin password, which must be set in /root/ispconfig/scripts/shell/awstats.php

    there is little problem, this password is added everyday when update script runs to /home/admispconfig/ispconfig/web/ispc.awstats/.htpasswd and it doesn't delete the old one so there is lot of lines with same record like

    awstatsadmin:xxxxxxxxxxxxxxxxxxxxxxxxx
    awstatsadmin:xxxxxxxxxxxxxxxxxxxxxxxxx
    awstatsadmin:xxxxxxxxxxxxxxxxxxxxxxxxx
    awstatsadmin:xxxxxxxxxxxxxxxxxxxxxxxxx

    do you think i should fix it somehow ?

    BTW:
    i have updated ispconfig again, and it seems that only /root/ispconfig/scripts/shell/awstats.php is deleted due to ISPC update..
    which is not so terrbile.. will se tommorrow .)
     
  14. djtremors

    djtremors ISPConfig Developer ISPConfig Developer

    The reason for this is due to that each site has the admin user in their .htpasswd file and the global .htpasswd file is just a `cat` of all these files so the admin user is duplicated numerous times.... call me lazy to grep it out...

    in regards to the earlier post about "awstats_updateall.pl now", it's actually a double up as the awstats script is currently doing the updates anyway. so you have 2 options.
    1. remove the crontab for the awstats_updateall.pl line and let the awstats.php do it.. good news about that is that it only processes live sites.
    2. hash out the awstats.php line where it calls the awstats.pl command. this means awstats_updateall.pl will do all the processing. Bad thing about this is only that any old sites which have been deleted are still processed since their files most likely exist in /etc/awstats/ directory....

    i think just remove that line, if I added it i probably shouldn't have (wont hurt since the awstats.php would have the logs pruned anyway)......
     
  15. Cojun

    Cojun New Member

    SuExec Problem with AWStats

    Hello,

    I spend more than hours for a solution to get AWStats running with ISPC and suexec.
    But I found nothing exept this Adresses:

    http://www.administrator.de/awStats_-_Komische_Sache.html
    and
    http://forum.swsoft.com/showthread.php?s=&threadid=23954

    Has anyone a clue how to port this informations to the AWStats package from djremors ?

    I hope, that there is a solution regarding this, because I will not run my machine without suexec only for a cool stats site :D

    Any sugguestions?

    Daniel
     
  16. meemu

    meemu ISPConfig Developer ISPConfig Developer

    mod_rewrite and proxying to a virtual host on 127.0.0.1 is teh solution I cam up with. I have written a patch for make_vhost but it's sort of complicated to get into ispconfig as it depends upon a custom field in the form manager. If you want to do it manually just add something like this:

    First Part is adding it to ISPConfig:

    In config.lib.php (you probably don't need the patch to add the apache user to each group and change perms to 750):
    Code:
       //////////////////// admispconfig der Gruppe hinzuf�gen ENDE //////////////
    
       $apache_user = $this->apache_user;
    +         //mimo http://www.howtoforge.org/forums/showthread.php?t=4375
    +         // add www user to each new group
    +         $mod->system->add_user_to_group("web".$doc_id,$apache_user);
    +         // add awstats user to each new web group
    +         $mod->system->add_user_to_group("web".$doc_id,"awstats");
    ..
    ..
         exec("chmod 755 $web_path_realname");
    +
    +           //mimo 2nd part
    +           exec("chmod 750 $web_path");
    +           exec("chmod 750 $web_path_realname");
    
    And this is the more important bit:

    Code:
         $cgi = "";
         if($web["web_cgi"] == 1) $cgi = "ScriptAlias  /cgi-bin/ ".$mod->system->server_conf["server_path_httpd_root"]."/"."web".$web["doc_id"]."/"."cgi-bi
    n/
     AddHandler cgi-script .cgi
     AddHandler cgi-script .pl";
    +    ///   AWSTATS hack
    +    ///   add awstats rewrite rules to the vhost
    +    ///   use a custom field from the ispconfig form manager "web_custom_awstats"
    +    ///   if is set create the awstats_trigger file. This is checked by the awstats cron script
    +    ///   if the file exists and awstats are disabled for teh vhost then rename it to "..-disabled-by-ispconfig"
    +    $awstats_trigger = $mod->system->server_conf["server_path_httpd_root"]."/"."web".$web["doc_id"]."/.awstats";
    +    if($web["web_custom_awstats"]) {
    +       if(!$rewrite_rule) {
    +               $rewrite_rule = "RewriteEngine On\n";
    +       $rewrite_rule .= "RewriteRule ^/awstats$ http://$servername/awstats/ [R]\n";
    +       $rewrite_rule .= "RewriteRule ^/awstats/$ http://awstats.localhost.localdomain/awstats.pl?config=$servername [NE,QSA,NS,P,L]\n";
    +       $rewrite_rule .= "RewriteRule ^/awstats/(.+)$ http://awstats.localhost.localdomain/$1 [NE,QSA,NS,P,L]\n";
    +       if(!file_exists($awstats_trigger)) {
    +               exec("touch $awstats_trigger && chown $webadmin:web".$web["doc_id"]." $awstats_trigger");
    
    

    Part Two is the vhost in apache:


    Code:
    /etc/apache2/sites-enabled/awstats
    <VirtualHost 127.0.0.1:80>
            ServerName awstats.localhost.localdomain
            DocumentRoot /var/www/share/awstats
            SuexecUserGroup awstats awstats
            ScriptAlias / /var/www/share/awstats/
            <Directory /var/www/share/awstats>
                    Options None
                    Options ExecCGI -MultiViews +SymLinksIfOwnerMatch
                    AllowOverride None
                    Order deny,allow
                    deny from all
                    allow from 127.0.0.1
            </Directory>
    </VirtualHost>
    
    Third Part the cron script:

    Code:
    awstats.php
    
    // v1.0 2007/4 mimo/at/restoel.net
    
    set_time_limit(0);
    
    include("/root/ispconfig/scripts/lib/config.inc.php");
    include("/root/ispconfig/scripts/lib/server.inc.php");
    
    $server_id = $mod->system->server_id;
    $server = $mod->system->server_conf;
    $path_httpd_root = stripslashes($server["server_path_httpd_root"]);
    exec("ls -1 /var/www/share/awstats/awstats.pl", $awstats_location, $verify);
    
    unset($awstats_location);
    $web_home = $path_httpd_root;
    
    $message = "";
    $handle = @opendir($web_home);
    while ($dir = @readdir ($handle)) {
        if ($dir != "." && $dir != "..") {
            if(@!is_link("$web_home/$dir") && substr($dir,0,3) == "web" && is_numeric(substr($dir,3))) {
            // ist kein symbolischer Link
                $trigger = "$web_home/$dir/.awstats";
                if(!file_exists($trigger))
                    continue;
                $webname = $dir;
                $web_path = $web_home . "/$webname/web";
                $stats_path = $web_path ."/awstats";
                $logfile = $web_home . "/$webname/log/web.log";
                $web_user = fileowner($web_path);
                $web_group = filegroup($web_path);
                // erstelle Stats Verzeichnis, wenn nicht vorhanden
                if(!@is_dir($stats_path)) {
                    mkdir($stats_path,0750);
                    chown($stats_path,$web_user);
                    chgrp($stats_path,$web_group);
                    $message .= "Erstelle Statistik Verzeichnis: $stats_path\n";
                }
    
                $web_doc_id = str_replace("web", "", $webname);
                $web_data = $mod->db->queryOneRecord("SELECT * FROM isp_isp_web WHERE doc_id = '$web_doc_id'");
                if(!empty($web_data["web_host"])){
                   $web_real_name = $web_data["web_host"].".".$web_data["web_domain"];
                } else {
                   $web_real_name = $web_data["web_domain"];
                }
               if(@is_file($logfile)) {
                    if ( !@is_file("/etc/awstats/awstats.".$web_real_name.".conf") ) {
                            exec( "echo 'Include \"/etc/awstats/awstats.model.conf\"' > /etc/awstats/awstats.".$web_real_name.".conf" );
                            exec( "echo 'LogFile=\"".$logfile."\"' >> /etc/awstats/awstats.".$web_real_name.".conf" );
                            exec( "echo 'SiteDomain=\"".$web_real_name."\"' >> /etc/awstats/awstats.".$web_real_name.".conf");
                            exec( "echo 'DirData=\"".$stats_path."\"' >> /etc/awstats/awstats.".$web_real_name.".conf");
                    }
                    $message .= exec("/var/www/share/awstats/awstats.pl -config=".$web_real_name." -Logfile=".$logfile." -update")."\n";
    
               }
    
    
                exec("chown -R $web_user:$web_group $stats_path &> /dev/null");
    
            }
        }
    }
    
    
    
    Voila!
    
    I think I posted this earlier somewhere here and in more detail.
     
  17. radim_h

    radim_h Member HowtoForge Supporter

    question regarding awstats DJtremors plugin v 1.41

    First, thak you for new release!!
    looks lot better that earlier doublescript .o)


    I have viped out old AWstats installation (of course saved dir /home/admispconfig/ispconfig/web/ispc.awstats.data with all stats data)
    on my ISPConfig server and have installed your version 1.41
    (My system is on Debian Etch i386)

    Scripts works fine, stats are created and accessible, but
    i'm getting those messages on end of the script:

    Warning: array_merge(): Argument #1 is not an array in /root/ispconfig/scripts/shell/awstats.php on line 196

    Warning: array_merge(): Argument #1 is not an array in /root/ispconfig/scripts/shell/awstats.php on line 196

    Warning: array_merge(): Argument #1 is not an array in /root/ispconfig/scripts/shell/awstats.php on line 196

    Warning: array_merge(): Argument #2 is not an array in /root/ispconfig/scripts/shell/awstats.php on line 196

    Line 196 contains:
    if (is_dir($dir."/".$filename)) $directory_array = array_merge($directory_array, dir_array($dir."/".$filename));


    Should it be something with global admin password?
    as i have in .htpasswd file awstatsadmin hashed password added after each site ?

    Thank you in advance for an answer

    BTW: ho does it process deleted sites? *.conf files are kept in /etc/awstats directory ?

    -----------------------------------
    P.S i have just commented out
    ##$admin_username="awstatsadmin";
    ##$admin_password="*******";
    ##if (empty($admin_password)|| empty($admin_username) )
    ## die("Set your admin username and password!\n");
    ##$admin_password_crypted=crypt($admin_password);

    And i'm still getting same mesasges and end of the script

    Just for info: When the admin password is commented out, line with ":" is created and end of each site as:

    somesite.tld_someuser:xefryhG66i2aM
    :
    somesite2.tld_someuser:aVoE79sxGTve6
    :
     
    Last edited: Mar 31, 2008
  18. djtremors

    djtremors ISPConfig Developer ISPConfig Developer

    there was no implementation to disable the super admin from checking stats sites.
    if you want to remove that option you will need to remove not only the login but the lines that use it to make sure it's done cleanly.

    look for lines that use those variables and remark those too.

    there is no cleaning method unfortuantely, this has to be done manually. It's too dangerous to be done automatically as sites can be renamed and renamed back and technically that means the site was deleted and recreated with all the data lost.
     
  19. radim_h

    radim_h Member HowtoForge Supporter

    More serious problem

    1. I didnt wan't to disable super admin, i was the person who asked you for this feature, if you remember :)

    2. But it doesnt make sense anymnore, unfortunately
    it wasn't only the problem in the script, awstats plugin 1.4.1 killed my ispconfig, i wasn't able to make any changes (changes was writen to database, but not commited), had to reinstall IPConfig with all own customization :( :(
    as i read in other Thread, same thing happened to user SupuS

    here:
    Exactly same thing happened to me:
    http://howtoforge.com/forums/showthread.php?t=20380
     
    Last edited: Apr 2, 2008
  20. djtremors

    djtremors ISPConfig Developer ISPConfig Developer

    I cannot see how this can effect ispconfig. the original script was ripped from the webalizer.php script that was provided by ispconfig and only changed the command to call awstats.pl instead of webalizer. I then added redirects to the /stats path so they all go to a universal webstats page and pass their domain config into the url so it grabs the correct one.

    this doesn't make any changes to ispconfig as far as I'm aware, it only adds itself to the 'crontab -e' entry and thats it. if you know it was awstats, i'd like to know how you know because I cannot see how on earth it could make that much damage.

    GV
     

Share This Page