Awstats mod - uses dynamic pages (non static)

did you delete

/root/ispconfig/scripts/shell/awstats.php
and remove the .htaccess file from all vhost sites as this is where the redirection is done?

 

thx found and removed everything for now.

 

I can invoke awstats.pl properly from commandline and it works fine to generate static pages. For some reason apache2 fails and gives me the 500 error msg.

Any ideas?
Did I miss out on any config in apache?
Mod_perl has been enabled as module also.

 

ISPc update

what will happen with installed awstats in case of upgrade ISPconfig ?
(i'd like to update from 2.2.11 to 2.2.13

i have also installed DJ tremors patch "Make ISPConfig tree sticky between refreshes"

will both of them still working ?

 

they will both be erased (at least the sticky patch as it overwrites the core ispconfig files). ISPC deleted the /home/admispconfig folder from memory (can't remember as I develop on DTC from gplhost.com now)

ISPC will most likely add my sticky tree in their main code as it's something a lot of people want and is kinda annoying to see collapse every time.
My sticky doesn't allow the full collapse and full expand buttons to work but since the stickys work, it was enough for me anyway.

 

500 error is an internal error. this usually means the .htaccess file has non allowed apache directives. check your "errors" log file for the virtual host or even the main apache error logs for the answers.

 

problem after ispc update

so, i have updated to 2.2.13 which soved my problem with hostingplan tepmplates

but awstats stopped create /web/stats directory and also .htpasswd files for new webs and it worked for formerly created sites

i also tried to run instalaltion of your awstats again, but nothing changed


also weird thing is that files for 06/2007
/home/admispconfig/ispconfig/web/ispc.awstats.data are created with owner root:root mod 644 instead of admispconfig:admispconfig mod 755

stats for old webs are stil properly working, not for new ones

please, djtremors do you have idea what to check on the server ??

That's all waht installation said:
[root@server2 ispc.awstats-install]# ./install.sh
Extracting awstats files and configs
Patching httpd.conf
Found: Redhat compatible conf.d path
Adding crontab settings
WARNING: Crontab entry exists in /var/spool/cron/root. Check to be sure

Please restart your Apache server for changes to take effect
All websites stats are accessed via {website}/stats
Thanks to 'martinfst' for bug fixes/notes on changes

 

you have to be careful when reinstalling. I do not make checks for things like if the /etc/awstats folder existing to not copy the rest over. I know thats slack on my part but it was how it was done before i realised ISPC deleted all the files on upgrades.

/home/admispconfig/ispconfig/web/ispc.awstats.data with files of 644 is correct. 755 is usually directories, those files are just data and don't execute anything so 755 is not needed on the data files.

Are all your data files for awstats intact?

Have you checked what happens when you run the crontab command?
/root/ispconfig/php/php /root/ispconfig/scripts/shell/awstats.php
(check the output).

 

haleluja .) it started working... i just forget filled admin/password i was sure i did it
but, does it something have to do with any other program running at midnight?
i think it always appear after midnight

i'm currios what was that cause of working stopped after upgrade... still don't understand mechanism witch creates .htpasswd files...
because when ispconfig left all config files on place, it had to work even after upgrade...

 

awstats and suexec

In case you still want to get this to work here are a couple of starting points:

* set up a dedicated vhost for awstats - e.g. awstats.localhost.localdomain with a config like this:

Code:

        ...
        DocumentRoot /var/www/share/awstats
        SuexecUserGroup awstats awstats
        ScriptAlias / /var/www/share/awstats/
        <Directory /var/www/share/awstats>
                Options None
                Options ExecCGI -MultiViews +SymLinksIfOwnerMatch
                AllowOverride None
                Order deny,allow
                deny from all
                allow from 127.0.0.1
        </Directory>
        ...

* setup a dedicated user/group for awstats
* modify ISPConfig to add the awstats user to each web site group (in case you want to keep awstats data in the users' sites)

* enable mod_proxy (a2enmod mod_proxy) and mod_proxy_http and alter the configuration

/etc/apache2/mods-enabled/proxy.conf

Code:

        ProxyRequests Off
        ProxyVia On
        <Proxy *>
                AddDefaultCharset off
                Order deny,allow
                Allow from all
        </Proxy>

* add rewrite rules to the vhosts that have awstats enabled

Code:

...
RewriteEngine On
RewriteRule ^/awstats$ http://{SERVER_NAME}/awstats/ [R]
RewriteRule ^/awstats/$ http://awstats.localhost.localdomain/awstats.pl?config={SERVER_NAME} [NE,QSA,NS,P,L]
RewriteRule ^/awstats/(.+)$ http://awstats.localhost.localdomain/$1 [NE,QSA,NS,P,L]
...

* create a cron script to update awstats after log processing is finished (you should do DNS resolution before, e.g. in webalizer.php)


/root/ispconfig/scripts/shell/awstats.php

Code:

<?
set_time_limit(0);

include("/root/ispconfig/scripts/lib/config.inc.php");
include("/root/ispconfig/scripts/lib/server.inc.php");

$server_id = $mod->system->server_id;
$server = $mod->system->server_conf;
$path_httpd_root = stripslashes($server["server_path_httpd_root"]);

$web_home = $path_httpd_root;

$message = "";
$handle = @opendir($web_home);
while ($dir = @readdir ($handle)) {
    if ($dir != "." && $dir != "..") {
        if(@!is_link("$web_home/$dir") && substr($dir,0,3) == "web" && is_numeric(substr($dir,3))) {
        // ist kein symbolischer Link
            $trigger = "$web_home/$dir/.awstats";
            if(!file_exists($trigger))
                continue;
            $webname = $dir;
            $web_path = $web_home . "/$webname/web";
            $stats_path = $web_path ."/awstats";
            $logfile = $web_home . "/$webname/log/web.log";
            $web_user = fileowner($web_path);
            $web_group = filegroup($web_path);
            // erstelle Stats Verzeichnis, wenn nicht vorhanden
            if(!@is_dir($stats_path)) {
                mkdir($stats_path,0750);
                chown($stats_path,$web_user);
                chgrp($stats_path,$web_group);
                $message .= "Erstelle Statistik Verzeichnis: $stats_path\n";
            }
            $web_doc_id = str_replace("web", "", $webname);
            $web_data = $mod->db->queryOneRecord("SELECT * FROM isp_isp_web WHERE doc_id = '$web_doc_id'");
            if(!empty($web_data["web_host"])){
               $web_real_name = $web_data["web_host"].".".$web_data["web_domain"];
            } else {
               $web_real_name = $web_data["web_domain"];
            }
            if(@is_file($logfile)) {
                if ( !@is_file("/etc/awstats/awstats.".$web_real_name.".conf") ) {
                        exec( "echo 'Include \"/etc/awstats/awstats.model.conf\"' > /etc/awstats/awstats.".$web_real_name.".conf" );
                        exec( "echo 'LogFile=\"".$logfile."\"' >> /etc/awstats/awstats.".$web_real_name.".conf" );
                        exec( "echo 'SiteDomain=\"".$web_real_name."\"' >> /etc/awstats/awstats.".$web_real_name.".conf");
                        exec( "echo 'DirData=\"".$stats_path."\"' >> /etc/awstats/awstats.".$web_real_name.".conf");
                }
                $message .= exec("/var/www/share/awstats/awstats.pl -config=".$web_real_name." -Logfile=".$logfile." -update")."\n";
            }


            exec("chown -R $web_user:$web_group $stats_path &> /dev/null");

        }
    }
}

All done!

 

just quick dumb question .)

is there some reason to run both scripts in 2 hours difference as
0 4 * * * /root/ispconfig/php/php /root/ispconfig/scripts/shell/awstats.php
0 6 * * * /home/admispconfig/ispconfig/web/ispc.awstats/awstats_updateall.pl now -awstatsprog="/home/admispconfig/ispconfig/web/ispc.awstats/awstats.pl"

or should i run them both at once from bash script?

 

global awstats admin password added everyday again ang again to .htpasswd

Hello DJtremors,
i'm going to bother you again .)

i have noticed little problem, since you added that super option for global awstatsadmin password, which must be set in /root/ispconfig/scripts/shell/awstats.php

there is little problem, this password is added everyday when update script runs to /home/admispconfig/ispconfig/web/ispc.awstats/.htpasswd and it doesn't delete the old one so there is lot of lines with same record like

awstatsadmin:xxxxxxxxxxxxxxxxxxxxxxxxx
awstatsadmin:xxxxxxxxxxxxxxxxxxxxxxxxx
awstatsadmin:xxxxxxxxxxxxxxxxxxxxxxxxx
awstatsadmin:xxxxxxxxxxxxxxxxxxxxxxxxx

do you think i should fix it somehow ?

BTW:
i have updated ispconfig again, and it seems that only /root/ispconfig/scripts/shell/awstats.php is deleted due to ISPC update..
which is not so terrbile.. will se tommorrow .)

 

The reason for this is due to that each site has the admin user in their .htpasswd file and the global .htpasswd file is just a `cat` of all these files so the admin user is duplicated numerous times.... call me lazy to grep it out...

in regards to the earlier post about "awstats_updateall.pl now", it's actually a double up as the awstats script is currently doing the updates anyway. so you have 2 options.
1. remove the crontab for the awstats_updateall.pl line and let the awstats.php do it.. good news about that is that it only processes live sites.
2. hash out the awstats.php line where it calls the awstats.pl command. this means awstats_updateall.pl will do all the processing. Bad thing about this is only that any old sites which have been deleted are still processed since their files most likely exist in /etc/awstats/ directory....

i think just remove that line, if I added it i probably shouldn't have (wont hurt since the awstats.php would have the logs pruned anyway)......

 

SuExec Problem with AWStats

Hello,

I spend more than hours for a solution to get AWStats running with ISPC and suexec.
But I found nothing exept this Adresses:

http://www.administrator.de/awStats_-_Komische_Sache.html
and
http://forum.swsoft.com/showthread.php?s=&threadid=23954

Has anyone a clue how to port this informations to the AWStats package from djremors ?

I hope, that there is a solution regarding this, because I will not run my machine without suexec only for a cool stats site

Any sugguestions?

Daniel

 

mod_rewrite and proxying to a virtual host on 127.0.0.1 is teh solution I cam up with. I have written a patch for make_vhost but it's sort of complicated to get into ispconfig as it depends upon a custom field in the form manager. If you want to do it manually just add something like this:

First Part is adding it to ISPConfig:

In config.lib.php (you probably don't need the patch to add the apache user to each group and change perms to 750):

Code:

   //////////////////// admispconfig der Gruppe hinzuf�gen ENDE //////////////

   $apache_user = $this->apache_user;
+         //mimo http://www.howtoforge.org/forums/showthread.php?t=4375
+         // add www user to each new group
+         $mod->system->add_user_to_group("web".$doc_id,$apache_user);
+         // add awstats user to each new web group
+         $mod->system->add_user_to_group("web".$doc_id,"awstats");
..
..
     exec("chmod 755 $web_path_realname");
+
+           //mimo 2nd part
+           exec("chmod 750 $web_path");
+           exec("chmod 750 $web_path_realname");

And this is the more important bit:

Code:

     $cgi = "";
     if($web["web_cgi"] == 1) $cgi = "ScriptAlias  /cgi-bin/ ".$mod->system->server_conf["server_path_httpd_root"]."/"."web".$web["doc_id"]."/"."cgi-bi
n/
 AddHandler cgi-script .cgi
 AddHandler cgi-script .pl";
+    ///   AWSTATS hack
+    ///   add awstats rewrite rules to the vhost
+    ///   use a custom field from the ispconfig form manager "web_custom_awstats"
+    ///   if is set create the awstats_trigger file. This is checked by the awstats cron script
+    ///   if the file exists and awstats are disabled for teh vhost then rename it to "..-disabled-by-ispconfig"
+    $awstats_trigger = $mod->system->server_conf["server_path_httpd_root"]."/"."web".$web["doc_id"]."/.awstats";
+    if($web["web_custom_awstats"]) {
+       if(!$rewrite_rule) {
+               $rewrite_rule = "RewriteEngine On\n";
+       $rewrite_rule .= "RewriteRule ^/awstats$ http://$servername/awstats/ [R]\n";
+       $rewrite_rule .= "RewriteRule ^/awstats/$ http://awstats.localhost.localdomain/awstats.pl?config=$servername [NE,QSA,NS,P,L]\n";
+       $rewrite_rule .= "RewriteRule ^/awstats/(.+)$ http://awstats.localhost.localdomain/$1 [NE,QSA,NS,P,L]\n";
+       if(!file_exists($awstats_trigger)) {
+               exec("touch $awstats_trigger && chown $webadmin:web".$web["doc_id"]." $awstats_trigger");

Part Two is the vhost in apache:

Code:

/etc/apache2/sites-enabled/awstats
<VirtualHost 127.0.0.1:80>
        ServerName awstats.localhost.localdomain
        DocumentRoot /var/www/share/awstats
        SuexecUserGroup awstats awstats
        ScriptAlias / /var/www/share/awstats/
        <Directory /var/www/share/awstats>
                Options None
                Options ExecCGI -MultiViews +SymLinksIfOwnerMatch
                AllowOverride None
                Order deny,allow
                deny from all
                allow from 127.0.0.1
        </Directory>
</VirtualHost>

Third Part the cron script:

Code:

awstats.php

// v1.0 2007/4 mimo/at/restoel.net

set_time_limit(0);

include("/root/ispconfig/scripts/lib/config.inc.php");
include("/root/ispconfig/scripts/lib/server.inc.php");

$server_id = $mod->system->server_id;
$server = $mod->system->server_conf;
$path_httpd_root = stripslashes($server["server_path_httpd_root"]);
exec("ls -1 /var/www/share/awstats/awstats.pl", $awstats_location, $verify);

unset($awstats_location);
$web_home = $path_httpd_root;

$message = "";
$handle = @opendir($web_home);
while ($dir = @readdir ($handle)) {
    if ($dir != "." && $dir != "..") {
        if(@!is_link("$web_home/$dir") && substr($dir,0,3) == "web" && is_numeric(substr($dir,3))) {
        // ist kein symbolischer Link
            $trigger = "$web_home/$dir/.awstats";
            if(!file_exists($trigger))
                continue;
            $webname = $dir;
            $web_path = $web_home . "/$webname/web";
            $stats_path = $web_path ."/awstats";
            $logfile = $web_home . "/$webname/log/web.log";
            $web_user = fileowner($web_path);
            $web_group = filegroup($web_path);
            // erstelle Stats Verzeichnis, wenn nicht vorhanden
            if(!@is_dir($stats_path)) {
                mkdir($stats_path,0750);
                chown($stats_path,$web_user);
                chgrp($stats_path,$web_group);
                $message .= "Erstelle Statistik Verzeichnis: $stats_path\n";
            }

            $web_doc_id = str_replace("web", "", $webname);
            $web_data = $mod->db->queryOneRecord("SELECT * FROM isp_isp_web WHERE doc_id = '$web_doc_id'");
            if(!empty($web_data["web_host"])){
               $web_real_name = $web_data["web_host"].".".$web_data["web_domain"];
            } else {
               $web_real_name = $web_data["web_domain"];
            }
           if(@is_file($logfile)) {
                if ( !@is_file("/etc/awstats/awstats.".$web_real_name.".conf") ) {
                        exec( "echo 'Include \"/etc/awstats/awstats.model.conf\"' > /etc/awstats/awstats.".$web_real_name.".conf" );
                        exec( "echo 'LogFile=\"".$logfile."\"' >> /etc/awstats/awstats.".$web_real_name.".conf" );
                        exec( "echo 'SiteDomain=\"".$web_real_name."\"' >> /etc/awstats/awstats.".$web_real_name.".conf");
                        exec( "echo 'DirData=\"".$stats_path."\"' >> /etc/awstats/awstats.".$web_real_name.".conf");
                }
                $message .= exec("/var/www/share/awstats/awstats.pl -config=".$web_real_name." -Logfile=".$logfile." -update")."\n";

           }


            exec("chown -R $web_user:$web_group $stats_path &> /dev/null");

        }
    }
}



Voila!

I think I posted this earlier somewhere here and in more detail.

 

question regarding awstats DJtremors plugin v 1.41

First, thak you for new release!!
looks lot better that earlier doublescript .o)


I have viped out old AWstats installation (of course saved dir /home/admispconfig/ispconfig/web/ispc.awstats.data with all stats data)
on my ISPConfig server and have installed your version 1.41
(My system is on Debian Etch i386)

Scripts works fine, stats are created and accessible, but
i'm getting those messages on end of the script:

Warning: array_merge(): Argument #1 is not an array in /root/ispconfig/scripts/shell/awstats.php on line 196

Warning: array_merge(): Argument #1 is not an array in /root/ispconfig/scripts/shell/awstats.php on line 196

Warning: array_merge(): Argument #1 is not an array in /root/ispconfig/scripts/shell/awstats.php on line 196

Warning: array_merge(): Argument #2 is not an array in /root/ispconfig/scripts/shell/awstats.php on line 196

Line 196 contains:
if (is_dir($dir."/".$filename)) $directory_array = array_merge($directory_array, dir_array($dir."/".$filename));


Should it be something with global admin password?
as i have in .htpasswd file awstatsadmin hashed password added after each site ?

Thank you in advance for an answer

BTW: ho does it process deleted sites? *.conf files are kept in /etc/awstats directory ?

-----------------------------------
P.S i have just commented out
##$admin_username="awstatsadmin";
##$admin_password="*******";
##if (empty($admin_password)|| empty($admin_username) )
## die("Set your admin username and password!\n");
##$admin_password_crypted=crypt($admin_password);

And i'm still getting same mesasges and end of the script

Just for info: When the admin password is commented out, line with ":" is created and end of each site as:

somesite.tld_someuser:xefryhG66i2aM
:
somesite2.tld_someuser:aVoE79sxGTve6
:

 

there was no implementation to disable the super admin from checking stats sites.
if you want to remove that option you will need to remove not only the login but the lines that use it to make sure it's done cleanly.

look for lines that use those variables and remark those too.

there is no cleaning method unfortuantely, this has to be done manually. It's too dangerous to be done automatically as sites can be renamed and renamed back and technically that means the site was deleted and recreated with all the data lost.

 

More serious problem

1. I didnt wan't to disable super admin, i was the person who asked you for this feature, if you remember

2. But it doesnt make sense anymnore, unfortunately
it wasn't only the problem in the script, awstats plugin 1.4.1 killed my ispconfig, i wasn't able to make any changes (changes was writen to database, but not commited), had to reinstall IPConfig with all own customization
as i read in other Thread, same thing happened to user SupuS

here:
Exactly same thing happened to me:
http://howtoforge.com/forums/showthread.php?t=20380

 

I cannot see how this can effect ispconfig. the original script was ripped from the webalizer.php script that was provided by ispconfig and only changed the command to call awstats.pl instead of webalizer. I then added redirects to the /stats path so they all go to a universal webstats page and pass their domain config into the url so it grabs the correct one.

this doesn't make any changes to ispconfig as far as I'm aware, it only adds itself to the 'crontab -e' entry and thats it. if you know it was awstats, i'd like to know how you know because I cannot see how on earth it could make that much damage.

GV