HOWTO: Add Time To Serve to GoAccess (Debian 12 Perfect Server setup)

Introduction
I recently installed my first Debian 12 Perfect Server and wanted to include the %D metric in GoAccess so I could spot potential performance issues on specific web pages. There wasn't really a clear guide on how to implement this so I am writing up this quick tip/trick/mod post for anyone trying to add this functionality to their GoAccess statistics reporting.

1. Prerequisites
Debian 12 Perfect Server
A logged in terminal session with a user permissioned to edit files with root ownership

2. Preparing the Apache2 log format
Create a copy of ispconfig.conf located in /etc/apache2/sites-availabe

Code:

cp /etc/apache2/sites-available/ispconfig.conf /etc/apache2/sites-available/ispconfig.conf.default

Using your text editor of choice, find the following (at line 17 as of ISPConfig 3.3.0p2);

Code:

LogFormat %v %h %l %u %t \"%r\" %>s %O \"%{Referer}i\" \"%{User-Agent}i\" combined_ispconfig

Comment out and add the required format string so it looks like below;

Code:

#LogFormat %v %h %l %u %t \"%r\" %>s %O \"%{Referer}i\" \"%{User-Agent}i\" combined_ispconfig
LogFormat '%v %h %l %u %t \"%r\" %>s %O \"%{Referer}i\" \"%{User-Agent}i\" %D \"%{Content-Type}o\" \"%{SSL_PROTOCOL}x\" \"%{SSL_CIPHER}x\"' combined_ispconfig

Restart Apache;

Code:

systemctl restart apache2.service

Disclaimer: I do not know if MANUALLY editing ispconfig.conf has impact on the overall functioning of Apache2 on ISPConfig 3, any experts feel free to chime in to suggest options if required.

3. Verifying your new log format
To make sure vlogger is producing the correct log format you can tail to check its ouput for your site.

Code:

tail -f /var/www/[yoursite.com]/log/access.log

Replace [yoursite.com] with the virtual host you are trying to verify. The access log should now suffix the numeric %D value at the end of each entry.

4. Enabling %D in GoAccess
For GoAccess to process the new log format a few lines in the /etc/goaccess.conf need to be altered. Start by securing a copy of the configuration file;

Code:

cp /etc/goaccess/goaccess.conf /etc/goaccess/goaccess.conf.default

Add the following lines using your editor of choice;

Code:

log-format=%h %^[%d:%t %^] "%r" %s %b "%R" "%u" %D "%M" "%K" "%k"
date-format=%d/%b/%Y
time-format=%T

These lines can be added anywhere in the configuration file, however do make sure there aren't any duplicate parameters that can override these. Save the file and return to your terminal prompt

5. Testing GoAccess in the terminal with %D
Example commands or configuration to generate logs/reports including %D.

Code:

goaccess --log-file /var/www/[yoursite.com]/log/access.log --browsers-file /etc/goaccess/browsers.list --log-format='%h %^[%d:%t %^] "%r" %s %b "%R" "%u" %D "%M" "%K" "%k"' --date-format=%d/%b/%Y --time-format=%T --real-time-html -m

Again replace [yoursite.com] with the virtual host your are trying to check.
If all is configured properly the GoAccess real-time monitor should present you with the usual statistics -- and newly added time to serve metric.

6. Final steps
Each time a new GoAccess stats page is enabled ISPConfig should automatically copy the /etc/goaccess.conf file to the log folder of your virtual host.

Ie. /var/www/{DOMAIN}/log

Updating existing GoAccess pages can be done by overwriting the existing goaccess.conf in your virtual host log folder.

Code:

cp /etc/goaccess/goaccess.conf /var/www/{DOMAIN}/log/goaccess.conf

CAVEAT: If you have an existing goaccess.conf file already that has custom changes (eg. GeoIP database support et al) you may opt to manually add the lines from step 4 instead to prevent overwriting your custom changes.

Hope this is of use to any of you looking to expand your stats using GoAccess!


EDIT: Changed the output string of the Apache2 LogFormat and GoAccess configuration accordingly, to make everything work as intended. Also added MIME type, SSL protocol and SSL cipher support.

EDIT 2: Upgrading your ISPConfig may reset the original /etc/apache2/sites-available/ispconfig.conf with the default LogFormat again. In that instance steps 2 and 3 need to be repeated.

 

I have been using this method for a couple of days now but since the last log rollover (5 Oct, 2025) by vlogger the changes are no longer processed properly.

The GoAcccess terminal output will dump the following error message on processing access.log;

Code:

==2755517== GoAccess - version 1.9.4 - Apr  1 2025 17:08:16
==2755517== Config file: /etc/goaccess/goaccess.conf
==2755517== https://goaccess.io - <[email protected]>
==2755517== Released under the MIT License.
==2755517==
==2755517== FILE: /var/www/newsviews.online/log/test.log
==2755517== Parsed 5 lines producing the following errors:
==2755517==
==2755517== Token 'EAD' doesn't match specifier '%s'
==2755517== Token 'EAD' doesn't match specifier '%s'
==2755517== Token 'ET' doesn't match specifier '%s'
==2755517== Token 'ET' doesn't match specifier '%s'
==2755517== Token 'ET' doesn't match specifier '%s'
==2755517==
==2755517== Format Errors - Verify your log/date/time format

Eg. of an entry from access.log

Code:

192.0.118.17 - - [05/Oct/2025:07:41:59 +0000] \"GET /wp-content/uploads/2019/09/Snip20190918_9.png HTTP/1.1\" 404 135727 \"-\" \"Photon/1.0\" 527983

When I use yesterday-access.log the files are being processed without any errors.

Code:

192.0.112.113 - - [04/Oct/2025:23:57:35 +0000] \"GET /wp-content/uploads/2020/02/download-10.jpeg HTTP/1.1\" 404 135725 \"-\" \"Photon/1.0\" 578340

I have been looking at the difference between the two log files, but I can't see any difference in the LogFormat of either log file. I am wondering if vlogger somehow added EOL or other special characters that aren't processed properly by GoAccess now.

Maybe I am change blind to the difference in the log format after staring at them for a few hours now, but I can't for the life of me see any difference, any suggesstions would be greatly appreciated.

 

Basically they all depend on the web server (Apache or Nginx) LogFormat, which is what you were doing in step 2 - defining its format, so one should always check whether the GoAcccess config is aligned to them or otherwise not, which clearly not, as the logs reported errors.

One, ~h{, } is meant for multiple IP, which normally comes from multiple access, for proxy / cdn, but you did not set it properly in LogFormat. So if you do not use or need that, you can keep your current LogFormat:

Code:

LogFormat '%v %h %l %u %t \"%r\" %>s %O \"%{Referer}i\" \"%{User-Agent}i\" %D' combined_ispconfig

but should change your GoAccess config, so it will be logging only single IP, and include a virtual host name field, that you are currently missing, may be like this:

Code:

log-format=%v %h %^[%d:%t %^] "%r" %s %b "%R" "%u" %D

However if you do use or need that, change your current LogFormat, so it catches them rightly, may be like this:

Code:

LogFormat '%v %{X-Forwarded-For}i, %h %l %u %t "%r" %>s %O "%{Referer}i" "%{User-Agent}i" %D' combined_ispconfig

and should change also your GoAccess config, to include the missing a virtual host name field, may be like this:

Code:

log-format=%v ~h{, } %^[%d:%t %^] "%r" %s %b "%R" "%u" %D

I think the latter can be used for all situation but I did not test any of the above suggestions. I guess you can venture into it since you are almost there.

 

Terima kasih!

I will have a look later tonight when I have some time to check out your suggestions. I may as well try adding encryption and cipher options in the logging.

 

I don't need the virtual host in my setup, so I am ommitting that one and made some additional alterations for the LogFormat in Apache and GoAccess to process the MIME type, SSL protocol and cipher (Which should satisfy a decent bunch of GoAccess' log variables);

/etc/apache2/sites-available/ispconfig.conf

Code:

LogFormat '%v %h %l %u %t \"%r\" %>s %O \"%{Referer}i\" \"%{User-Agent}i\" %D \"%{Content-Type}o\" \"%{SSL_PROTOCOL}x\" \"%{SSL_CIPHER}x\"' combined_ispconfig

/etc/goaccess/goaccess.conf

Code:

log-format=%h %^[%d:%t %^] "%r" %s %b "%R" "%u" %D "%M" "%K" "%k"
date-format=%d/%b/%Y
time-format=%T

I wrapped the MIME type/SSL protocol/SSL cipher in double quotes since all of those seem to include characters that need to be escaped otherwise.

Again, thank you for your kind input! Much appreciated.

ps. The goaccess commandline isn't processing this properly yet, but I presume this is because the log formats aren't matching yet until the nightly log-rollover takes place. (For the record, the log format above in both Apache and GoAccess had been working for a couple of days without issue, somehow it fell over the 5th of this month.)

 

@ahrasis Thanks again so much for the feedback, I think I have everything running as intended again. (That should teach me not to blindly copy/paste formatting strings from a Google search without understanding the operators used on the variables).

ps. I had some cleaning up to do in the /var/log/ispconfig/httpd folder as well as the tilde operator on the host variable was indeed IP splitting all the logging output.

 

If you doing something like in your #5 post but you also do not wish / need virtual host name to be included in the log, you should also remove - %v - in the LogFormat, or I think you might face the same error again due to the misalignment, as your GoAccess config doesn't include it. I am also not sure about other additions you made in your GoAccess config, but just make sure they are also aligned with the LogFormat.

 

I indeed do not need the virtual host name in the statistics, however when I remove %v from the Apache LogFormat vlogger seems to stop processing the log real-time.

I presume this is somehow needed for vlogger to determine how to split the logs?

ps. The additional parameters are indeed aligned for GoAccess (the real-time terminal output shows the log processing properly).

 

I think you may keep - %v - in LogFormat so that vlogger continues working, but in that case you must tell GoAccess to ignore that first virtual host name field, with - %^ - may be like this:

Code:

log-format=%^ %h %^[%d:%t %^] "%r" %s %b "%R" "%u" %D "%M" "%K" "%k"

I think by this, vlogger may still see %v (so logs route per vhost correctly), GoAccess may skip %v and parses the rest properly; and no misalignment, no stoppage, and no format conflict. Again this is merely a suggestion and not tested.

 

I initially tried;

Code:

log-format=%v %h %^[%d:%t %^] "%r" %s %b "%R" "%u" %D "%M" "%K" "%k"

Which resulted in a parse error by the goaccess command line utility

When I try your suggestion;

Code:

log-format=%^ %h %^[%d:%t %^] "%r" %s %b "%R" "%u" %D "%M" "%K" "%k"

It also returns the same parse error (it strips the first charater from the %s variable somehow, resulting in it not understanding HEAD/POST/GET (as it reads it as EAD/OST/ET)

I suspect it doesn't like the space after %^ if there isn't a leading variable (in this instance %v) because it will work when I strip out the space

Code:

log-format=%^%h %^[%d:%t %^] "%r" %s %b "%R" "%u" %D "%M" "%K" "%k"

That said, that probably just works because there isn't a variable prefixed before %h and just reads that as an empty variable, as the split vhost log starst with the %h.

At any length it seems the %h %^[%d:%t %^] "%r" %s %b "%R" "%u" %D "%M" "%K" "%k" seems to be doing what it is supposed to be doing.

Again, your feedback is much appreciated. I wouldn't have -- or at least have taken half an eon -- been able to sort this out so quickly without your help!

 

If it already is working good for you, by all means carry on with it.

I noted it as such above because of your second post reporting error which I thought was due to misalignment, probably due the missing - %v - in GoAccess config.

If current config is fine, that earlier errrors may be due to something else, of which you might have solved it with the right ones, hopefully.

 

From a purely Apache2 LogFormat and GoAccess format it is indeed a misalignment, but the Apache log is pre-vlogger. What I deduce is happening is that the post-vlogger split log output will have truncated the %v from the log causing it to match the GoAccess log format again

 

Hurmmm... The problem of my suggested code in #9 is the second - %^ - which must be adjusted, but the first suggested field is the right field, aligned to virtual host name field, which is to be ignored, since you do not need it, but - %h - as the first field as you suggested, is definitely the wrong one, so instead of your suggestion in post #5, to keep everything aligned, the right fix may rather be like this:

Code:

log-format=%^ %h %l %u [%d:%t %^] "%r" %s %b "%R" "%u" %D "%M" "%K" "%k"

Explanation:
%^ %v (skip virtual host)
%h %h Client IP
%l %l Ident
%u %u Auth user
[%d:%t %^] %t Timestamp, ignoring timezone
"%r" %r Request line
%s %>s HTTP status
%b %O Bytes sent
"%R" %{Referer}i
"%u" %{User-Agent}i
%D %D Duration μs
"%M" %{Content-Type}o
"%K" %{SSL_PROTOCOL}x
"%k" %{SSL_CIPHER}x

Note: - %l %u - are added even though each is just usually "-", but you still need placeholders in your GoAccess format so all other fields align correctly with Apache format.

Code:

LogFormat '%v %h %l %u %t \"%r\" %>s %O \"%{Referer}i\" \"%{User-Agent}i\" %D \"%{Content-Type}o\" \"%{SSL_PROTOCOL}x\" \"%{SSL_CIPHER}x\"' combined_ispconfig

Explanation:
%v Virtual host (used by vlogger)
%h Client IP
%l Identd (usually -)
%u Authenticated user (usually -)
%t Timestamp [05/Oct/2025:07:41:59 +0000]
%r Request line "GET /file HTTP/1.1"
%>s HTTP status
%O Bytes sent
%{Referer}i Referer
%{User-Agent}i User-Agent
%D Request duration in μs
%{Content-Type}o Response Content-Type
%{SSL_PROTOCOL}x SSL protocol
%{SSL_CIPHER}x SSL cipher

Misalignment only occurs if tokens are skipped or reordered incorrectly. Again if it already is working good for you, by all means carry on with it. You do not need to break things or fix something which is not broken.

 

I have been tinkering around with the log format, while doing so I encountered a couple of issues;

• Upgrading ISPConfig seemed to have restored /etc/apache2/sites-available/ispconfig.conf to its shipped state
• Rebooting my server also result in ISPConfig restoring the /etc/apache2/sites-available/ispconfig.conf to its shipped state
• GoAccess does not seem to support %l (identd) and the equivalent for %u seems to be %e -- Not a huge issue, either %h %^ %e or %h %^ %^ should fix that
  
• After upgrading ISPConfig from 3.3.0p2 to 3.3.0p3 I can not seem to get the logs to process properly (that, or some other system updates broke it).

Anyway, the persistence of the ispconfig.conf file is forcing me to roll-back to the out-of-the-box LogFormat for the time being until I figure out how to work around this issue.

ps. Alternatively I tried using GoAccess' %x timedate formatting, which seems to select the token correctly but somehow it does not like the timestamp format;

Code:

--timedate-format '%d/%b/%Y:%H:%M:%S %z'  --tz=Europe/Amsterdam

But somehow this token still mismatches (maybe I am change blind at this point but to me it looks like it is matching just fine)

Code:

==34032== Token '13/Oct/2025:07:53:07 +0200' doesn't match specifier '%x'

I checked the system's timezone with timedatectl;

Code:

Local time: Mon 2025-10-13 16:30:30 CEST
Universal time: Mon 2025-10-13 14:30:30 UTC
RTC time: Mon 2025-10-13 14:30:30
Time zone: Europe/Amsterdam (CEST, +0200)
System clock synchronized: yes
NTP service: n/a
RTC in local TZ: no

So it seems to me that using the --tz option should invoke the proper +0200 offset.

 

I thought you are already using the famous ISPConfig conf-custom and its install folder that can survive its update. If I remember correctly, you just need to copy apache_ispconfig.conf.master from install/tpl/ to server/conf-custom/install and keep the modified version in the later folder and I would also keep a symlink to server/conf-custom if ISPConfig resync tool also affects that file.

 

Terima kasih!

I am fairly new to ISPConfig so I was unware of this hack -) I'll have a look later tonight.