My website is only serving around 10 websites. One of the website is very active thus generating lots of logs. I just found out about this only when I got alerted for diskspace running out email. I tried searching in ispconfig and there are no option to disable logging for certain website. I am using ispconfig 3.0.1.6. Can I know where I can disable http logging for specified websites?? I think this is a pretty important feature and I assume it is already implemented? Maybe I didn't find properly? What about compressions on the logs? Is there any? What about log rotate? As of now, these 3 features seems like not available. Without these, the logs will build up like crazy, even for a normal website, the logs generated daily is around 1GB already. With compression, this could reduce to 100mb. Please help....thanks.
Just manage to view back one of the thread about compression of log. http://www.howtoforge.com/forums/showthread.php?t=37421&highlight=compress I looked at my cron_daily.php file and there are lines already added for compression of logs and also removal of logs more than 30 days. But seems like this is not running properly. Don't know why...anyone can take a look into this?
When I rung the cron_daily.php manually, I get error: Warning: Truncating oversized referrer field This could be the thing causing the failed access_log files processing?
No. The truncate error is normal and happens on log lines that are too long. Please update your server to ispconfig 3.0.2.2 as it has an improved log handling.
i have upgrade to 3.0.2.2. There are no additional function for handling logs per website. I am assuming the improved log handling you meant was the compression and 30 days removal script is working now? If not, please do point me to the right direction. Thanks.
i have looked at another server that is using 3.0.2.1. The logs compression is there for most of the domains except for 2. One is as11 and as11.domain.com. FYI, as11 is the server's hostname. So it's like the compression doesn't work on local hostname. Upgrade to 3.0.2.2?
I left the upgraded 3.0.2.2 server overnight. Checked the access logs in /var/log/ispconfig/httpd and no compressed access log file. The log file of the previous day is still uncompressed. Please help....
Please post the output of: ls -la /var/log/ispconfig/httpd/yourdomain.com replace yourdomain.com with the domain name of a affected website. The websites that are affected had been created in ispconfig or have you created them manually?
server 1 (3.0.2.2): - all access log also does not compress nor delete files older than 30 days: ls -la /var/log/ispconfig/httpd/127.0.0.1 Code: -rw-r--r-- 1 root root 59973494 2010-07-01 23:59 20100701-access.log.gz -rw-r--r-- 1 root root 57519689 2010-07-02 23:59 20100702-access.log.gz -rw-r--r-- 1 root root 56580180 2010-07-03 23:59 20100703-access.log.gz -rw-r--r-- 1 root root 54573798 2010-07-04 23:59 20100704-access.log.gz -rw-r--r-- 1 root root 58566076 2010-07-05 23:59 20100705-access.log.gz -rw-r--r-- 1 root root 61830847 2010-07-06 23:59 20100706-access.log.gz -rw-r--r-- 1 root root 58912995 2010-07-07 23:59 20100707-access.log.gz -rw-r--r-- 1 root root 57487132 2010-07-08 23:59 20100708-access.log.gz -rw-r--r-- 1 root root 52348721 2010-07-09 23:59 20100709-access.log.gz -rw-r--r-- 1 root root 52555526 2010-07-10 23:59 20100710-access.log.gz -rw-r--r-- 1 root root 53555419 2010-07-11 23:59 20100711-access.log.gz -rw-r--r-- 1 root root 38464667 2010-07-12 23:59 20100712-access.log.gz -rw-r--r-- 1 root root 44748612 2010-07-13 23:59 20100713-access.log.gz -rw-r--r-- 1 root root 49687442 2010-07-14 23:59 20100714-access.log.gz -rw-r--r-- 1 root root 87718804 2010-07-15 23:59 20100715-access.log.gz -rw-r--r-- 1 root root 89333655 2010-07-16 23:59 20100716-access.log.gz -rw-r--r-- 1 root root 87059890 2010-07-17 23:59 20100717-access.log.gz -rw-r--r-- 1 root root 85310862 2010-07-18 23:59 20100718-access.log.gz -rw-r--r-- 1 root root 127998030 2010-07-19 23:59 20100719-access.log.gz -rw-r--r-- 1 root root 1830110297 2010-07-20 23:59 20100720-access.log -rw-r--r-- 1 root root 2011749233 2010-07-21 23:59 20100721-access.log -rw-r--r-- 1 root root 1764172052 2010-07-22 23:59 20100722-access.log -rw-r--r-- 1 root root 1754949565 2010-07-23 23:59 20100723-access.log -rw-r--r-- 1 root root 1677808377 2010-07-24 23:59 20100724-access.log -rw-r--r-- 1 root root 1670813799 2010-07-25 23:59 20100725-access.log -rw-r--r-- 1 root root 1729431133 2010-07-26 23:59 20100726-access.log -rw-r--r-- 1 root root 1687999078 2010-07-27 23:59 20100727-access.log -rw-r--r-- 1 root root 1700423428 2010-07-28 23:59 20100728-access.log -rw-r--r-- 1 root root 1429171939 2010-07-29 17:31 20100729-access.log lrwxrwxrwx 1 root root 19 2010-07-29 12:48 access.log -> 20100729-access.log ls -la /var/log/ispconfig/httpd/vn.domain.com/ Code: -rw-r--r-- 1 root root 206761957 2010-07-16 23:59 20100716-access.log.gz -rw-r--r-- 1 root root 174249432 2010-07-17 23:59 20100717-access.log.gz -rw-r--r-- 1 root root 149160599 2010-07-18 23:59 20100718-access.log.gz -rw-r--r-- 1 root root 179652719 2010-07-19 23:59 20100719-access.log.gz -rw-r--r-- 1 root root 1323127637 2010-07-20 23:59 20100720-access.log -rw-r--r-- 1 root root 1451718336 2010-07-21 23:59 20100721-access.log -rw-r--r-- 1 root root 1439908507 2010-07-22 23:59 20100722-access.log -rw-r--r-- 1 root root 1201661855 2010-07-23 23:59 20100723-access.log -rw-r--r-- 1 root root 898080288 2010-07-24 23:59 20100724-access.log -rw-r--r-- 1 root root 874027550 2010-07-25 23:59 20100725-access.log -rw-r--r-- 1 root root 1142218307 2010-07-26 23:59 20100726-access.log -rw-r--r-- 1 root root 1137946356 2010-07-27 23:59 20100727-access.log -rw-r--r-- 1 root root 1218598540 2010-07-28 23:59 20100728-access.log -rw-r--r-- 1 root root 799965574 2010-07-29 17:35 20100729-access.log lrwxrwxrwx 1 root root 19 2010-07-29 12:48 access.log -> 20100729-access.log -rw-r--r-- 1 web2 client1 1183785 2010-07-29 17:31 error.log -rw-r--r-- 1 root root 25707 2010-07-29 00:30 webalizer.conf This file I compressed them manually. And already removed files older than July 2010. The domain of 127.0.0.1 is the default domain created manually (from ubuntu's default site) and vn.domain.com is created using ispconfig server 2 (3.0.2.1): ls -la /var/log/ispconfig/httpd/as69 Code: drwxr-xr-x 2 root root 4096 2010-07-29 00:27 . drwxr-xr-x 27 root root 4096 2010-07-23 16:51 .. -rw-r--r-- 1 root root 27627 2010-05-26 23:59 20100526-access.log -rw-r--r-- 1 root root 39069 2010-05-27 23:55 20100527-access.log -rw-r--r-- 1 root root 36137 2010-05-28 23:55 20100528-access.log -rw-r--r-- 1 root root 36178 2010-05-29 23:57 20100529-access.log -rw-r--r-- 1 root root 35886 2010-05-30 23:59 20100530-access.log -rw-r--r-- 1 root root 57538 2010-05-31 23:58 20100531-access.log -rw-r--r-- 1 root root 26358 2010-06-01 23:56 20100601-access.log -rw-r--r-- 1 root root 12807 2010-06-02 23:34 20100602-access.log -rw-r--r-- 1 root root 13814 2010-06-03 23:42 20100603-access.log -rw-r--r-- 1 root root 21864 2010-06-04 23:50 20100604-access.log -rw-r--r-- 1 root root 9855 2010-06-05 23:49 20100605-access.log -rw-r--r-- 1 root root 11110 2010-06-06 23:43 20100606-access.log -rw-r--r-- 1 root root 8399 2010-06-07 23:55 20100607-access.log -rw-r--r-- 1 root root 6039 2010-06-08 20:25 20100608-access.log -rw-r--r-- 1 root root 518 2010-06-09 07:38 20100609-access.log -rw-r--r-- 1 root root 22656 2010-06-10 17:43 20100610-access.log -rw-r--r-- 1 root root 495 2010-06-11 18:25 20100611-access.log -rw-r--r-- 1 root root 2202 2010-06-13 15:12 20100613-access.log -rw-r--r-- 1 root root 1425 2010-06-14 16:26 20100614-access.log -rw-r--r-- 1 root root 13509 2010-06-15 12:52 20100615-access.log -rw-r--r-- 1 root root 1148 2010-06-17 11:27 20100617-access.log -rw-r--r-- 1 root root 173 2010-06-18 21:55 20100618-access.log -rw-r--r-- 1 root root 1842 2010-06-20 06:33 20100620-access.log -rw-r--r-- 1 root root 1017 2010-06-21 22:19 20100621-access.log -rw-r--r-- 1 root root 258 2010-06-22 22:18 20100622-access.log -rw-r--r-- 1 root root 11764 2010-06-23 15:13 20100623-access.log -rw-r--r-- 1 root root 857 2010-06-24 23:04 20100624-access.log -rw-r--r-- 1 root root 15296 2010-06-25 21:48 20100625-access.log -rw-r--r-- 1 root root 512 2010-06-26 18:04 20100626-access.log -rw-r--r-- 1 root root 4870 2010-06-27 22:18 20100627-access.log -rw-r--r-- 1 root root 1762 2010-06-28 23:01 20100628-access.log -rw-r--r-- 1 root root 658 2010-06-29 19:58 20100629-access.log -rw-r--r-- 1 root root 13850 2010-06-30 15:26 20100630-access.log -rw-r--r-- 1 root root 335 2010-07-01 14:43 20100701-access.log -rw-r--r-- 1 root root 696 2010-07-02 23:39 20100702-access.log -rw-r--r-- 1 root root 206 2010-07-03 15:34 20100703-access.log -rw-r--r-- 1 root root 2575 2010-07-04 18:26 20100704-access.log -rw-r--r-- 1 root root 15167 2010-07-05 15:17 20100705-access.log -rw-r--r-- 1 root root 256 2010-07-06 14:00 20100706-access.log -rw-r--r-- 1 root root 528 2010-07-07 23:08 20100707-access.log -rw-r--r-- 1 root root 297 2010-07-08 23:32 20100708-access.log -rw-r--r-- 1 root root 1528 2010-07-09 23:32 20100709-access.log -rw-r--r-- 1 root root 239 2010-07-10 08:13 20100710-access.log -rw-r--r-- 1 root root 4666 2010-07-11 20:42 20100711-access.log -rw-r--r-- 1 root root 260 2010-07-12 16:47 20100712-access.log -rw-r--r-- 1 root root 333 2010-07-13 15:52 20100713-access.log -rw-r--r-- 1 root root 111 2010-07-14 23:23 20100714-access.log -rw-r--r-- 1 root root 1257 2010-07-15 22:57 20100715-access.log -rw-r--r-- 1 root root 444 2010-07-16 12:11 20100716-access.log -rw-r--r-- 1 root root 763 2010-07-17 16:33 20100717-access.log -rw-r--r-- 1 root root 2586 2010-07-18 23:34 20100718-access.log -rw-r--r-- 1 root root 2298 2010-07-19 19:39 20100719-access.log -rw-r--r-- 1 root root 128 2010-07-20 12:26 20100720-access.log -rw-r--r-- 1 root root 20287 2010-07-21 18:35 20100721-access.log -rw-r--r-- 1 root root 1942 2010-07-22 20:38 20100722-access.log -rw-r--r-- 1 root root 23792 2010-07-23 23:11 20100723-access.log -rw-r--r-- 1 root root 1925 2010-07-24 23:44 20100724-access.log -rw-r--r-- 1 root root 7010 2010-07-25 21:50 20100725-access.log -rw-r--r-- 1 root root 31775 2010-07-26 23:12 20100726-access.log -rw-r--r-- 1 root root 3219 2010-07-27 23:08 20100727-access.log -rw-r--r-- 1 root root 3348 2010-07-28 23:07 20100728-access.log -rw-r--r-- 1 root root 11323 2010-07-29 17:41 20100729-access.log lrwxrwxrwx 1 root root 19 2010-07-29 00:27 access.log -> 20100729-access.log ls -la /var/log/ispconfig/httpd/as69.domain.com Code: drwxr-xr-x 2 root root 4096 2010-05-26 11:31 . drwxr-xr-x 27 root root 4096 2010-07-23 16:51 .. -rw-r--r-- 1 root root 257362 2010-03-22 18:57 20100322-access.log -rw-r--r-- 1 root root 24944 2010-03-23 18:00 20100323-access.log -rw-r--r-- 1 root root 41341 2010-03-24 18:19 20100324-access.log -rw-r--r-- 1 root root 187878 2010-03-25 23:53 20100325-access.log -rw-r--r-- 1 root root 46191 2010-03-26 18:24 20100326-access.log -rw-r--r-- 1 root root 256 2010-03-27 10:50 20100327-access.log -rw-r--r-- 1 root root 1386 2010-03-28 06:28 20100328-access.log -rw-r--r-- 1 root root 1592 2010-03-29 17:00 20100329-access.log -rw-r--r-- 1 root root 78299 2010-03-30 18:28 20100330-access.log -rw-r--r-- 1 root root 2930 2010-03-31 20:54 20100331-access.log -rw-r--r-- 1 root root 570 2010-04-01 16:10 20100401-access.log -rw-r--r-- 1 root root 48175 2010-04-02 18:32 20100402-access.log -rw-r--r-- 1 root root 130 2010-04-03 08:46 20100403-access.log -rw-r--r-- 1 root root 1893 2010-04-04 17:22 20100404-access.log -rw-r--r-- 1 root root 11802 2010-04-05 18:45 20100405-access.log -rw-r--r-- 1 root root 110 2010-04-07 21:34 20100407-access.log -rw-r--r-- 1 root root 14720 2010-04-08 21:38 20100408-access.log -rw-r--r-- 1 root root 129 2010-04-09 17:35 20100409-access.log -rw-r--r-- 1 root root 129 2010-04-10 21:44 20100410-access.log -rw-r--r-- 1 root root 1665 2010-04-11 20:05 20100411-access.log -rw-r--r-- 1 root root 130 2010-04-12 17:22 20100412-access.log -rw-r--r-- 1 root root 1436450 2010-04-13 23:59 20100413-access.log -rw-r--r-- 1 root root 2413651 2010-04-14 23:59 20100414-access.log -rw-r--r-- 1 root root 2378880 2010-04-15 23:59 20100415-access.log -rw-r--r-- 1 root root 2399040 2010-04-16 23:59 20100416-access.log -rw-r--r-- 1 root root 2062750 2010-04-17 23:59 20100417-access.log -rw-r--r-- 1 root root 2117572 2010-04-18 23:59 20100418-access.log -rw-r--r-- 1 root root 2481353 2010-04-19 23:59 20100419-access.log -rw-r--r-- 1 root root 2487938 2010-04-20 23:59 20100420-access.log -rw-r--r-- 1 root root 2542348 2010-04-21 23:59 20100421-access.log -rw-r--r-- 1 root root 2685128 2010-04-22 23:59 20100422-access.log -rw-r--r-- 1 root root 2393308 2010-04-23 23:59 20100423-access.log -rw-r--r-- 1 root root 2131292 2010-04-24 23:59 20100424-access.log -rw-r--r-- 1 root root 2097671 2010-04-25 23:59 20100425-access.log -rw-r--r-- 1 root root 2559121 2010-04-26 23:59 20100426-access.log -rw-r--r-- 1 root root 2536506 2010-04-27 23:59 20100427-access.log -rw-r--r-- 1 root root 2425253 2010-04-28 23:59 20100428-access.log -rw-r--r-- 1 root root 2429532 2010-04-29 23:59 20100429-access.log -rw-r--r-- 1 root root 2416206 2010-04-30 23:59 20100430-access.log -rw-r--r-- 1 root root 1832299 2010-05-01 23:59 20100501-access.log -rw-r--r-- 1 root root 1916858 2010-05-02 23:59 20100502-access.log -rw-r--r-- 1 root root 2274764 2010-05-03 23:59 20100503-access.log -rw-r--r-- 1 root root 2492260 2010-05-04 23:59 20100504-access.log -rw-r--r-- 1 root root 2217581 2010-05-05 23:59 20100505-access.log -rw-r--r-- 1 root root 2439680 2010-05-06 23:59 20100506-access.log -rw-r--r-- 1 root root 2346986 2010-05-07 23:59 20100507-access.log -rw-r--r-- 1 root root 2001514 2010-05-08 23:59 20100508-access.log -rw-r--r-- 1 root root 1994963 2010-05-09 23:59 20100509-access.log -rw-r--r-- 1 root root 2495807 2010-05-10 23:59 20100510-access.log -rw-r--r-- 1 root root 2385358 2010-05-11 23:59 20100511-access.log -rw-r--r-- 1 root root 2397078 2010-05-12 23:59 20100512-access.log -rw-r--r-- 1 root root 2428856 2010-05-13 23:59 20100513-access.log -rw-r--r-- 1 root root 2251266 2010-05-14 23:59 20100514-access.log -rw-r--r-- 1 root root 1963221 2010-05-15 23:59 20100515-access.log -rw-r--r-- 1 root root 2418050 2010-05-16 23:59 20100516-access.log -rw-r--r-- 1 root root 3335868 2010-05-17 23:59 20100517-access.log -rw-r--r-- 1 root root 3493862 2010-05-18 23:59 20100518-access.log -rw-r--r-- 1 root root 2840056 2010-05-19 23:59 20100519-access.log -rw-r--r-- 1 root root 3037030 2010-05-20 23:59 20100520-access.log -rw-r--r-- 1 root root 2642671 2010-05-21 23:59 20100521-access.log -rw-r--r-- 1 root root 2373177 2010-05-22 23:59 20100522-access.log -rw-r--r-- 1 root root 2192809 2010-05-23 23:59 20100523-access.log -rw-r--r-- 1 root root 2713228 2010-05-24 23:59 20100524-access.log -rw-r--r-- 1 root root 2667320 2010-05-25 23:59 20100525-access.log -rw-r--r-- 1 root root 1520034 2010-05-26 15:52 20100526-access.log lrwxrwxrwx 1 root root 19 2010-05-26 11:31 access.log -> 20100526-access.log Both as69 and as69.domain.com also is created manually. Again this is under "000-default" under apache configuration. Other sites on server2 is no problem. The access log files are compressed and old file deleted. Thanks.
The log compression are only for sites created in ispconfig. Th examples that you posted above are not from sites that had been created in ispconfig, so the logs dont get compressed. ISPConfig can only compress logs for sites that it knows off. If you want to compress logs for non ispconfig websites, you might have to write your own scripts for that.
what about vn.domain.com? This site was created with ispconfig. Also, where does 127.0.0.1 log comes from? I can't find where it is defined. Also I cant fine where does as69 amd as69.domain.com log files defined. I have looked in 000-default and no definition of the logs. And it end up in /etc/log/ispconfig/http/.....why? How do I turn off logs for these "rogue" sites?
ISPConfig compresses yesterdays log for websites that get listed by this mysql query: SELECT domain_id, domain, document_root FROM web_domain WHERE server_id = 1 The server_id depends on your server. On single server systems, it is 1, on multiserver systems you can lookup the ID in the config.inc.php file.
ok, that explains why the compressions did not work on those domains created manually. Let's talk about server1 only. This server was 3.0.1.6 when i first install. Then upgraded to 3.0.2.2. During upgrade, I did say yes to all the questions to re-configure the settings. Now, the compress does not work at all in this server. There is only 1 very active domain in here. That is vn.domain.com. This domain was created using ispconfig. Therefore, it should compress but it isn't. The access log is like 1.5GB per day and since the logs also does not get removed after 30 days, it ended up taking up spaces. How to solve this problem? Another problem in server1 also have the default website as defined in ubuntu. This will access /var/www folder as root. This was not created by ispconfig therefore I understand why ispconfig won't compress it. But why does the access log got into ispconfig's httpd folder? The access log file is in /var/log/ispconfig/httpd/127.0.0.1/. How do I turn off access log for this 127.0.0.1? In the apache file for this default site, I have already commented access log from the beginning. But logs still go into this 127.0.0.1 folder. How so? More importantly how to disable this access log? Cause the log files are also in excess of 1GB per day. Please help.
ok, after reading about vlogger, i understand how the logging in ispconfig works. Seems like there are no options to disable logs for specified virtualhost in vlogger. This will be an issue when we need to control whether to log or not to log for specified websites. Also with manually created virtual hosts. Hmm...will have to study and see what are other options available...
