Remove mod_pagespeed from acces.log

Discussion in 'Installation/Configuration' started by luckyluk3, Jan 24, 2019.

  1. luckyluk3

    luckyluk3 New Member

    Hello,

    Im trying to remove the user agent mod_pagespeed from the acces.log of websites. This internal request gets logged with the web traffic. This causes for 1000mb plus traffic a day on the ispconfig web traffic page. Of course because it is an internal request we shouldn't log it. Below an example how it looks.
    Code:
    *ipofserver* - - [24/Jan/2019:13:40:13 +0100] "GET /wp-content/***.jpg HTTP/1.1" 200 1806 "https://***.com/*" "Serf/1.3.9 (mod_pagespeed/1.13.35.2-0)"
    *ipofserver* - - [24/Jan/2019:13:40:13 +0100] "GET /wp-content/***.jpg HTTP/1.1" 200 1806 "https://***.com/*" "Serf/1.3.9 (mod_pagespeed/1.13.35.2-0)"
    *ipofserver* - - [24/Jan/2019:13:40:13 +0100] "GET /wp-content/***.jpg HTTP/1.1" 200 1806 "https://***.com/*" "Serf/1.3.9 (mod_pagespeed/1.13.35.2-0)"

    I found this posts online on how to do it: https://bjornjohansen.no/exclude-requests-from-nginx-access-log
    The problem here is that when I add "if=$log_ua" to the end of the access_log in the nginx server block it gets removed whenever ispconfig changes the file. Is there maybe a way I can do this exclude trick globally or configure something in the nginx pagespeed module itself. I couldn't understand there were not more people having this issue with mod_pagespeed logging all the internal requests. And thus ispconfig giving false information about the web traffic a website was using.
     
  2. luckyluk3

    luckyluk3 New Member

  3. till

    till Super Moderator Staff Member ISPConfig Developer

    Copy the nginx vhost conf master file from /usr/local/ispconfig/conf/ to /usr/local/ispconfig/conf-custom/ and modify it in the way you want.
     
  4. luckyluk3

    luckyluk3 New Member

    Thanks once again, I didn't know this feature existed. For anyone wondering how I fixed it.

    Add this to your nginx http block
    Code:
    map $http_user_agent $log_ua {
        ~mod_pagespeed 0;
        default 1;
    }
    And modify the custom nginx_vhost.conf.master to this
    Code:
    <tmpl_if name='logging' op='==' value='yes'>
            error_log /var/log/ispconfig/httpd/<tmpl_var name='domain'>/error.log;
            access_log /var/log/ispconfig/httpd/<tmpl_var name='domain'>/access.log combined if=$log_ua;
    </tmpl_var>
    <tmpl_if name='logging' op='==' value='anon'>
            error_log /var/log/ispconfig/httpd/<tmpl_var name='domain'>/error.log;
            access_log /var/log/ispconfig/httpd/<tmpl_var name='domain'>/access.log anonymized if=$log_ua;
    </tmpl_var>
    Requests from mod_pagespeed will no longer show up in the access.log and thus not in the web traffic quota.
     
    Jesse Norell likes this.

Share This Page