web pages loading slowly after upgrade to 3.2.12p1

Hello,
I am running ISPConfig on Ubuntu servers. I have upgraded Ubuntu to 24.04 and ISPConfig to 3.2.12p1 and ever since then the webpages are loading slowly. I noticed that db servers are overloading with mariadb process being the dominant process. Do you have any suggestions for troubleshooting and can I safely tweak MariaDB settings to optimize performance?

Code:

##### SERVER #####
IP-address (as per hostname): ***.***.***.***
[WARN] could not determine server's ip address by ifconfig
[INFO] OS version is Ubuntu 24.04.1 LTS

[INFO] uptime:  16:36:48 up 1 day,  9:12,  4 users,  load average: 28.52, 23.97, 18.94

[INFO] memory:
               total        used        free      shared  buff/cache   available
Mem:           9.7Gi       9.5Gi       166Mi        84Ki       207Mi       122Mi
Swap:          3.8Gi       3.6Gi       277Mi

[INFO] systemd failed services status:
  UNIT                      LOAD      ACTIVE SUB    DESCRIPTION
● snap.lxd.activate.service not-found failed failed snap.lxd.activate.service

Legend: LOAD   → Reflects whether the unit definition was properly loaded.
        ACTIVE → The high-level unit activation state, i.e. generalization of SUB.
        SUB    → The low-level unit activation state, values depend on unit type.

1 loaded units listed.

[INFO] ISPConfig is installed.

##### ISPCONFIG #####
ISPConfig version is 3.2.12p1


##### VERSION CHECK #####

[INFO] php (cli) version is 8.3.13
[INFO] php-cgi (used for cgi php in default vhost!) is version 8.3.13

##### PORT CHECK #####

[WARN] Port 8080 (ISPConfig) seems NOT to be listening
[WARN] Port 8081 (ISPConfig Apps) seems NOT to be listening
[WARN] Port 143 (IMAP server) seems NOT to be listening
[WARN] Port 993 (IMAP server SSL) seems NOT to be listening
[WARN] Port 110 (POP3 server) seems NOT to be listening
[WARN] Port 995 (POP3 server SSL) seems NOT to be listening
[WARN] Port 465 (SMTP server SSL) seems NOT to be listening

##### MAIL SERVER CHECK #####

[WARN] I found no "submission" entry in your postfix master.cf
[INFO] this is not critical, but if you want to offer port 587 for smtp connections you have to enable this.
[WARN] I found no "smtps" entry in your postfix master.cf

 

I realize that ISPConfig version should not affect the loading speed. I did check access and error logs, and found nothing peculiar (how would I see speed of loading in them anyway?). But the fact is that multiple pages hosted on multiple web servers are having problems. DB pressure could explain the slow loading time I think. I was wondering if I can safely tune mariadb without breaking ISPConfig.

Also I did run the test script as root, is there something in the output that points to problems with privileges?

 

Code:

[INFO] uptime:  16:36:48 up 1 day,  9:12,  4 users,  load average: 28.52, 23.97, 18.94

[INFO] memory:
               total        used        free      shared  buff/cache   available
Mem:           9.7Gi       9.5Gi       166Mi        84Ki       207Mi       122Mi
Swap:          3.8Gi       3.6Gi       277Mi

Load is very high and memory is exhausted. This is why web pages load slowly.
You should monitor the host to see which process uses so much memory and what is running there that uses all CPU.
Is the disk system full or in degraded state so it has gotten slow?

 

I've determined that the problem is with db servers in multiserver setup. I added more memory, and eventually it got exhausted too. memleak shows that there is a leak in mariadb and there were a lot of these types of messages in mysql error.log:

Code:

2024-11-24  8:14:14 0 [Note] InnoDB: Memory pressure event freed 199 pages

After trying to find problems in mysql usage without any luck, I found this issue description, so I turned off THP https://jira.mariadb.org/browse/MDEV-33279. I was going to wait a bit more to post this as a solution, but it's been running with no problems for about ten hours now.
I'm still puzzled why this kicked in after Ubuntu + ISPConfig upgrade.