Sudomains and log-files

Hello all,
we have multiple subdomains running on our "main-domain" (i.e. domain.ch), which we configured like own domains.
We do this, because we want to give customers the opportunity to have separated emails, dbs and (ftp-)users.
When we create the account we "New web" and insert there i.e. subdomain in the field Host and i.e. domain.ch in the field Domain -> subdomain.domain.ch.
It works like a charm. But the problem is, we'd like to see all the stats from the subdomains in the log of the top-domain (domain.ch).
In domain.ch/stats should all subdomains be listed. At the moment all subdomains have their own logs under subdomain.domain.ch/stats. Is it possible to list all stats (i.e. from subdomain.domain.ch) under the main domain (i.e. domain.ch)?

Thank you and with kind regards

 

You would have to copy the web.log in the logs directory of each subdomain to the web.log of the main domain between 1 and 4 AM each night. The web.log is created at 00.30 AM each night, and webalizer runs at 4 AM.

 

Hello falko,
thank you for your answer.
Is there any command to copy content directly from one file to another?
And is there any script / code to get all subdomains from a domain?

Thank you and with kind regards

 

Yes:

Code:

cat file1 >> file2

No, you must write your own one and get the subdomains from the ISPConfig database.

 

Good morning,
thank you for your answer.
I did now a file, which is based on log.php.
It looks like this (after your code, from log.php):
/root/ispconfig/scripts/shell/log_subdomains.php

PHP:

// Traffic auswerten und Symlinks für Webalizer aktualisieren
foreach($traffic as $virtual_host => $bytes) {

        if(trim($virtual_host) != '' && trim($virtual_host)!='localhost') {
                // Traffic in DB Schreiben
                // Bestimme Web-ID
                $link =  readlink("$webroot/$virtual_host");
                $parts = split("/",$link);
                $web_id = intval(substr($parts[count($parts) - 1],3));

                $domains = $mod->db->queryAllRecords("SELECT doc_id, web_host, web_domain FROM isp_isp_web WHERE doc_id = '$web_id' AND (web_host = 'www' OR web_host = '')");

                foreach( $domains as $domain )
                {
                        $subdomains = $mod->db->queryAllRecords("SELECT doc_id, web_host, web_domain
                                                                FROM isp_isp_web
                                                                WHERE web_domain = '".$domain['web_domain']."'
                                                                AND web_host != ''
                                                                AND web_host != 'www'");
                        if( !empty( $subdomains ) )
                        {
                                foreach( $subdomains as $subdomain )
                                {
                                        system("cat $webroot/".$subdomain['web_host'].".".$domain['web_domain']."/log/web.log >> $webroot/$virtual_host/log/web.log");
                                }
                        }
                }
        }
}


Is this okay? I thought, I'd run it (in cron) directly after log.php.

Thank you and with kind regards

 

Try it...
If it doesn't work, you can debug it by adding some echo statements to see the value of your variables.

 

I did that before posting and it seemed to be ok. I saved the cat'ed log-file to another file and compared it with the original. The logs from the subdomain(s) were added
But running webalizer.php showed, that it didn't recognize new records..
Does webalizer.php take the file /var/www/webX/log/web.log?

Thank you and have a nice evening

 

Yes, it does.

 

Hello,
I found out, that if I cat the files manually, by hand, we get a lot higher statistics than with the cron job (the code I posted before).
Which user executes the cron-jobs? Is it possible, that this user hasn't got the
according rights for:

Code:

system("cat $webroot/".$subdomain['web_host'].".".$domain['web_domain']."/log/web.log >> $webroot/$virtual_host/log/web.log"); 

Because if I run this command as user root, the statistics are a lot higher. Could this be the problem?
Thank you and with kind regards

 

Good evening,
I have a general question concerning webalizer now
I've did some testing and once I tried to make a cat manually as root.
So I took the logfile from a subdomain of domain.ch and cat'ed it to the domain.ch's log-file, i.e.:

Code:

cat /var/www/subdomain.domain.ch/log/web.log >> /var/www/www.domain.ch/log/web.log

Lets say, the web.log-file of domain.ch was first 50MB and the log-file of subdomain.domain.ch was about 200MB. So after the cat-command, the log-file of domain.ch was 250MB, which is absolutely correct.
Now to the stats.
Subdomain.domain.ch had on 05th of sept., lets say, 10000 hits and 5000 visitors.
Domain.ch had on the same day i.e. 5000 hits and 2500 visitors.
So after the cat'ing, I've run the webalizer.php-script.
I assumed the web.log-file of domain.ch would no have 15000 hits and 7500 visitors. But this was wrong, it had exactly the same as before. But why this?
I even checked the web.log-file and the entries from the web.log-file from subdomain.domain.ch were in there. So it should have all the stats from it or am I wrong?

Thank you and with kind regards

 

Of course you do. But you're cat'ing log files of two completely independent web sites (although they use the same domain). You wouldn't cat the logs for yahoo.com to the logs for google.com, would you?

It's root.

Webalizer has its own database, so it knows if it has already created statistics for certain log lines. I think you should delete it (back it up!) before you try this. I think it's webalizer.current or webalizer.hist.

 

Hello Falko,
thank you for your answer.
Yes, that's true, they are completely different sites, but it's our aim to catch all visitors for one domain and their subdomains on the domain. Is this somehow possible?
Can I do it otherwise than cat'ing the log-files? Because we want to have all stats from the subdomains added to the stats of the main-domain, so that we have the stats for the subdomains on one side, but on the other all the stats gathered on one domain.

With kind regards

 

Because the sudomains are sites on their own, with different files and databases. They get managed by different people, which shouldn't have access to the main domain (and their email-addresses etc.). Plus they want multiple separate email-addresses per subdomain and stats for each subdomain as well. It's a bit complicated

With kind regards

 

Ok, I think I can manage this somehow I have checked the webalizer.php-file and saw, that it gets called by:

Code:

webalizer -n www.domain.ch -s www.domain.ch -r www.domain.ch -q -T -o /var/www/webX/web/stats -c /root/ispconfig/scripts/shell/webalizer.conf /var/www/webX/log/web.log

Does this mean, that it looks just after www.domain.ch for the stats in the logfile /var/www/webX/log/web.log and ignores other (sub)domains for this file?
If so, this would be the reason why the stats don't get any higher.
Then a possible solution is to cat all logfiles from that domain and their subdomains to a big-logfile and add a custom webalizer-call for that domain, so that it outputs it i.e. to domain.ch/stats_all.
But the condition would be, that it doesn't analyze the big logfile for a specific domain but analyzes it whatever (sub)domain is in the file.
Is this possible with webalizer? If yes, how?

 

Hello,
thank you for your answer.
I tried it with mergelog, but it didn't work as expected
Do I need to set an alias for the global-log file? Or can I use absolute paths in the directive (i.e. /var/www/webX/web/log/log_all.log)? And who has to be the owner and the group for that file? Because there are different users, which need to have write-access to that file.
Is this directive ok?

Code:

CustomLog /var/www/webX/log/log_all.log common

So all subdomains and the domain would have this directive under "apache directives" in ispconfig.

Thank you and with kind regards

 

You can use absolute paths.

The Apache user and group.

Looks ok.

 

OnTheRun,
Just wondering how this directive is working. On my current server I have stats for each site however i like to call up a global report that provides me with details in the morning of the servers status.

Any other changes beside the directive?

Cheers