Hi My ISPconfig server running a lot of websites, and sometimes server CPU overloaded from time to time. Is there any tool to easily find out in the past which sites causing high load and what files accessed the most? thanks
Activate statistics for each website. Then you have history data on website access patterns. Use Internet Search Engines with Code: site:howtoforge.com activate statistics If "in the past" is not a requirement, there are tools for measuring performance and showing top users.
If you want to see it in realtime, just use top command. Each website runs as its own web[ID] user, you can see which users cause the load, then you know the website ID and can check in ISPConfig by searching for the ID which website is causing the issue.
thanks. I can locate the website by using website ID, then I need to find out what and who is creating such high load in specific time range like: in the past 2024/Jan/1 20:00 - 20:05 CPU 100% overloaded by website ID 100, I want to check what caused that exactly.
By checking the access log, I found high cpu is caused by this bot. I want to apply this blocking to all websites in the ispconfig server, is there a way to add to once and applied to all vhosts? if ($http_user_agent ~* (YisouSpider|Scrapy) ) { return 410; } 120.230.181.101 - - [06/Feb/2024:09:48:45 +0800] "GET /business/tel:28700798?page=15 HTTP/1.1" 200 24193 "-" "Scrapy/2.11.0 (+https://scrapy.org)" 112.9.6.199 - - [06/Feb/2024:09:48:46 +0800] "GET /restaurant/tel:28482848?page=16 HTTP/1.1" 200 25969 "-" "Scrapy/2.11.0 (+https://scrapy.org)" 1.191.13.126 - - [06/Feb/2024:09:48:47 +0800] "GET /medical/tel:28521702?page=2 HTTP/1.1" 200 25191 "-" "Scrapy/2.11.0 (+https://scrapy.org)" 117.143.128.70 - - [06/Feb/2024:09:48:47 +0800] "GET /shopping/tel:28334926?page=4 HTTP/1.1" 200 25506 "-" "Scrapy/2.11.0 (+https://scrapy.org)" 58.212.251.47 - - [06/Feb/2024:09:48:47 +0800] "GET /business/tel:28572259?page=8 HTTP/1.1" 200 24210 "-" "Scrapy/2.11.0 (+https://scrapy.org)" 112.7.121.78 - - [06/Feb/2024:09:48:47 +0800] "GET /community/tel:83969345?page=20 HTTP/1.1" 200 21244 "-" "Scrapy/2.11.0 (+https://scrapy.org)" 1.80.137.42 - - [06/Feb/2024:09:48:47 +0800] "GET /recommend/tel:28930263?page=2 HTTP/1.1" 200 25111 "-" "Scrapy/2.11.0 (+https://scrapy.org)"
Copy the global /usr/local/ispconfig/server/conf/nginx_vhost.conf.master file to /usr/local/ispconfig/server/conf-custom/nginx_vhost.conf.master, then edit /usr/local/ispconfig/server/conf-custom/nginx_vhost.conf.master and apply your changes. Then run Tools > resync in ISPConfig to write all website config files again.
thank you. I blocked the bot, but still have high CPU from time to time. I fount out that root auto runs below everyday during busy hours, which one of the reasons causing CPU overloaded. This is not manual input, we only run HTOP. and I wonder if this is ISPconfig auto runs it? root@s1:~# crontab -l * * * * * /usr/local/ispconfig/server/server.sh 2>&1 | while read line; do echo `/bin/date` "$line" >> /var/log/ispconfig/cron.log; done * * * * * /usr/local/ispconfig/server/cron.sh 2>&1 | while read line; do echo `/bin/date` "$line" >> /var/log/ispconfig/cron.log; done 0 0 * * * /usr/local/ispconfig/server/scripts/create_daily_nginx_access_logs.sh &> /dev/null du -s /var/www/clients/client0/web1 /var/www/clients/client0/web100 /var/www/clients/client0/web101 /var/www/clients/client0/web103 /var/www/clients/client0/web104 /var/www/clients/client0/web108 /var/www/clients/client0/web19 /var/www/clients/client0/web2 /var/www/clients/client0/web20 /var/www/clients/client0/web21 /var/www/clients/client0/web22 /var/www/clients/client0/web23 /var/www/clients/client0/web26 /var/www/clients/client0/web29 /var/www/clients/client0/web3 /var/www/clients/client0/web30 /var/www/clients/client0/web35 /var/www/clients/client0/web36 /var/www/clients/client0/web41 /var/www/clients/client0/web43 /var/www/clients/client0/web44 /var/www/clients/client0/web45 /var/www/clients/client0/web46 /var/www/clients/client0/web47 /var/www/clients/client0/web48 /var/www/clients/client0/web49 /var/www/clients/client0/web50 /var/www/clients/client0/web51 /var/www/clients/client0/web52 /var/www/clients/client0/web54 /var/www/clients/client0/web55 /var/www/clients/client0/web56 /var/www/clients/client0/web59 /var/www/clients/client0/web66 /var/www/clients/client0/web70 /var/www/clients/client0/web83 /var/www/clients/client0/web85 /var/www/clients/client0/web90 /var/www/clients/client0/web91 /var/www/clients/client0/web92 /var/www/clients/client0/web94 /var/www/clients/client0/web95 /var/www/clients/client0/web96 /var/www/clients/client0/web97 /var/www/clients/client1/web107 /var/www/clients/client1/web115 /var/www/clients/client1/web27 /var/www/clients/client2/web116_bak /var/www/clients/client2/web117 /var/www/clients/client2/web118 /var/www/clients/client3/web124 /var/www/clients/client3/web126
I audited the du, and found that something (ISPconfig?) trigger the du process every 5 mints causing server overload https://monosnap.com/file/1A6DDNPYtodG37fvj4ui0TP5IQ3RcL sudo apt install auditd sudo systemctl start auditd sudo auditctl -w /usr/bin/du -p x -k du_use ausearch -k du_use
There is no need to install any audit software, especially as it does not help you to locate or solve the issue. You know the command that causes the load, so can search for that command in scripts easily using grep: Code: root@server1:~# grep -R 'du -s' /usr/local/ispconfig/ /usr/local/ispconfig/server/lib/classes/cron.d/100-monitor_hd_quota.inc.php: $ dfData = shell_exec('du -s /var/www/clients/client*/we*'); /usr/local/ispconfig/server/lib/classes/cron.d/100-monitor_email_quota.inc.php:/ /* with dovecot we can use doveadm instead of 'du -s' /usr/local/ispconfig/server/lib/classes/cron.d/100-monitor_email_quota.inc.php:$ app->system->exec_safe('du -s ?', $mb['maildir']); and when you compare the paths, then you know that it is the script /usr/local/ispconfig/server/lib/classes/cron.d/100-monitor_hd_quota.inc.php that monitors the file system usage of the websites.
Thanks for the reply. script /usr/local/ispconfig/server/lib/classes/cron.d/100-monitor_hd_quota.inc.php that monitors the file system usage of the websites why ISPconfig by default creates du process every 5 mins? It created a lot of loading on my server CPU as well as disk read continuously as you can see from this image. https://monosnap.com/file/HbE1PZdfhVk88O1LCoA2wJrNfusJFd If I change the du process once a day, CPU and Disk Read immediately droped significantly and server performance back to normal.
