ISPConfig 3 & Proxmox sites have VERY SLOW TTFB

Discussion in 'General' started by TomStone, Feb 20, 2022.

  1. TomStone

    TomStone New Member

    Hey people!

    I have two sites running on my Dell R630, 128GB ram, 2x 10 core xeons, on a 1GBPS network connection in a datacenter a few km from me.

    Ive got two sites running, both wordpress, both get very slow with traffic, with just like 20% cpu usage, network all good too. The server has 2x Mirror NVME drives, so i doubt the hardware would be the issue. The problem is mainly TTFB, the actual loading of stuff is "relatively" ok, altough not ideal either.

    We do get quite a lot of traffic, both sites have around 1-2k active users. We did not have any problems when on Google hosting, just the fact that the servers were pretty underpowered for their price in this case, with so many plugins, which is why we switched to our own hardware.
    Compared to google, it seems to get slow even when not using 100% of the CPU. Thats whats so odd.

    Any ideas? It only seems to be a problem when theres more traffic, like there isnt enough ports or something...thank you!!!
     
  2. Taleman

    Taleman Well-Known Member HowtoForge Supporter

    You may need to optimize Wordpress, I do not know how. Maybe remove plugins that are not needed, that might help.
    To get some idea on how to make your websites faster, run monitoring and measurements to see where the time is spent or wasted. Your hardware seems very powefull, but bad websites run slow on any harware.
     
    ahrasis likes this.
  3. till

    till Super Moderator Staff Member ISPConfig Developer

    Using many plugins in WordPress is always an issue as it slows down WP.

    Do you use apache or Nginx as the web server?
    Do you use any proxies in front of the system like CloudFlare?
    Do you use a caching plugin in WordPress?

    On the options tab of the websites in ISPConfig, you can also increase the number of PHP processes of the site. Is this a big website in terms of number of pages? If yes, you might want to check MySQL with e.g. MySQLTuner to see if there are any settings to be improved in the database server part. There are also plugins that optimize WP database structure for better performance https://de.wordpress.org/plugins/index-wp-mysql-for-speed/
     
    ahrasis likes this.
  4. TomStone

    TomStone New Member

    Hey,
    I use Apache
    I do not, i tried cloudflare, didnt change a thing...
    I do use it on one of the two sites, and i managed to minify and optimise it by a lot. The problem is, whenever theres traffic, it slows BOTH sites down. Its like the server couldnt keep up, yet its only using a small percentage of its available CPU power. Like its waiting on something. My TTFB is 10-30 seconds at these times, which is WAY beyond normal lol. Thanks[/QUOTE]
     
  5. TomStone

    TomStone New Member

    Hey, i doubt it really is a problem of plugins. Sure, theres a lot on one site, but the other has like 4-5 plugins only. The TTFB of both sites gets up to 30 seconds at worst, which is abysmal, even if i were to use 100s of plugins. Like i wrote above, its like the server gets overloaded, or there is a open file limit or something along those lines. The number of open files is set to unlimited ("cat /proc/sys/fs/file-max shows unlimited"), but i have not tried other stuff. Any info is welcome, im frankly a little lost.
     
  6. till

    till Super Moderator Staff Member ISPConfig Developer

    I recommend you start checking mysql settings then, WordPress highly depends on good database speed.

    Which WP caching plugin do you use and in which mode? Try using e.g. w3total cache or LiteSpeed cache and use Memcached as caching backend in the settings of the plugins to keep the cache in memory. memcached and PHP Memcached plugins are installed by default on ISPConfig systems if you followed our install guides. Alternatively, you can also use Redis as backend, but you have to install redis then first and probably also the PHP redis module.
     
  7. TomStone

    TomStone New Member

    upload_2022-2-21_8-47-38.png
    Ok, installed W3, it does not allow me to choose memcached tho. Also, the Disk Basic seems oddly way too fast lol. Any tips/a tutorial on how to get Memcached working? Thanks !
     
  8. nhybgtvfr

    nhybgtvfr Well-Known Member HowtoForge Supporter

    TTFB in wordpress, would come down mainly to DB and php performance, it won't start sending back any data until it's built the wordpress page. caching would certainly help avoid problems and reduce load. but in this case, it may be better to turn off all caching for a while, having it on may just make it harder to find where the issue actually is.

    try installing the 'query monitor' plugin on those sites, if there's anything slowing it down in php or mysql, that should make it easy to find.
     
  9. If you're using Debian/Ubuntu, it may be as simple as doing something like:

    sudo apt-get install memcached php-memcached

    RPM-based installations should have similar packages. memcached, as far as I remember, works 'out of the box' with zero configuration, so it's always a great choice :)
     
  10. hismastersvoice_

    hismastersvoice_ New Member

    Hi,
    I know this is an old thread but did You solve Your problem? We have the same...
    thx
     
  11. What was true last year is true this year as well :)

    There is a hint on @TomStone's post: "the site seems slow even though it's not using 100% of the CPU". True. When the CPU is doing nothing but waiting for external data (especially data that comes in slowly), it will not use any resources... until the data arrives.

    Usually, the first culprit is the disk — some process is insanely writing to disk, and consuming all I/O resources. That's easily solved. If you have 128 GB of RAM, the first thing is to remove a disk-based swap file — even if you have superfast NVME/SSD disks in a RAID5 configuration, it'll be slower than good, old RAM. Next step: put everything you can in memory, making sure you do some finetuning. For instance, if you're using WordPress, make sure that MySQL/MariaDB gets all the memory it wants, and then some. I have two dozen or so websites with next-to-zero traffic, and allocated some 30-40 GBytes to MariaDB. It still grumbles and writes too much to disk, but it's certainly way, way faster than before. The same applies to the /tmp directory. It will be used so often by so many programs, it's best if you can have it on memory instead on disk; almost all Linux distributions I've recently used will use tmpfs for /tmp, but some forget to do the same for /run, which is also heavily used (and usually takes little space — and certainly does not need to 'survive' a reboot).

    iotop — or, even nicer, htop — are your friends here. These are tools that measure not only CPU & memory (like plain old top does), but they look at disk writes/reads as well, and it's easy to see the culprits.

    Next step... if the disk(s) are not being used a lot, and the CPU isn't, either, but the site remains slow... what could it be? Network. There are too many variables here to check, though; it's not only 'heavy traffic' that makes everything slower. Badly configured routers (which may or may not be under your control), or an unusual firewall + NAT configuration, and things may act weird. Forgetting to configure IPv6 properly but nevertheless having DNS pointing to your server will cause all sorts of hard-to-debug issues (as you can read on this forum!).

    Last but not least... application logic. I'm a very, very old fan of WordPress, from the early days when 'blogging' was a novelty. I love it for thousands of reasons, and hardly ever use/recommend anything else. Nevertheless, WordPress has some major design flaws which weren't obvious 20 years ago. Back then, you see, 'dynamic content websites' were all the rage, and that meant storing everything on a database and generating pages on-the-fly when users visited them. These days, we scream at the horror of such a stupid idea — but 20 years ago we were eager evangelists on how 'dynamic content' is the most wonderful invention since sliced bread.

    Well... it certainly is, of course, but it means that WordPress was not initially designed to deal with 'caches' and that sort of thing. You couldn't (and it's still very hard to do!) simply dump a static version of WordPress and let the webserver handle the traffic; it was simply never designed to do that. It also suffers from a weakness of PHP. When a page is viewed, it's not just the content that is loaded. Everything is. By that I mean that the whole range of plugins and libraries are all loaded, a page is generated, and everything gets immediately 'forgotten' until the next visitor comes in. That's how PHP was designed, too.

    It's obvious that this is not what happens in 2023, of course! There are levels upon levels of optimisations — from the way PHP went from a 'simple interpreted scripting language' to a powerful, object-oriented, general purpose, compiled programming language, while still remaining faithful to many of the original tenets. WordPress, in turn, has layers upon layers of optimisations so that frequently accessed resources are not loaded all the time, but just once, and then reused as much as possible. And, of course, then the system administrator comes in, and further optimises it. Pages, or at least snippets of pages, are stored on disk (or memory!) if their content is relatively static (i.e., it doesn't change every microsecond). You can use reverse proxies to speed up the serving of static files (CSS, JS, SVG, other media files...) or even push them into a CDN. And run your site under CloudFlare (for free) and let them handle all incoming requests on your behalf. There is a lot that can be tweaked along the way, depending on the resources you've got (memory! CPUs with lots of cores!) and how you configure them.

    If you're using the W3 Total Cache plugin, make sure you use it with, say, memcached or Redis, and make sure that things like the object cache (especially the object cache!...) uses those services instead. You'll see an immediate improvement. If you don't use W3 (I don't because it's insanely heavy on the system, but that's a personal preference), you must make sure that you have one that does both page caching and object caching; if not, there are standalone object caching plugins which will do the trick.

    But you can go further; for instance, every time anyone visits your website (not only logged in users), PHP will establish a session. Sessions are small (these days, they come embedded in cookies) but require persistent storage, and, by default, PHP uses the disk for that. Bad idea. You can push sessions to Redis easily enough (I'm not quite sure about memcached — I'd have to check on that), and that will give a huge boost to PHP in general. WordPress won't need any configuration change, it will just use whatever PHP tells it to use.

    That's just a tiny, tiny aspect of all that you can do to improve your TTFB without spending more money on your server. :) The HowToForge tutorials have many, many more tips and suggestions.

    What they do not have is a 'magic switch' that speeds everything up when you turn it on. That doesn't exist. You'll have to go with your trial-and-error tweaking and fine-tuning, and see what gives some improvement and doesn't require more maintenance that you can afford.
     
  12. Taleman

    Taleman Well-Known Member HowtoForge Supporter

    I disagree with that. Having swap makes the system faster. Althoug swap file is not recommended, better to have swap partition on disk.
    When swap space is available, system can move seldom needed processes from memory to swap, thus making more RAM available for processes that need it.
    For example this (found with Internet Search Engines): https://haydenjames.io/linux-performance-almost-always-add-swap-space/
    Furthermore, in addition to htop etc., there is dstat which in my experience helps finding where the slowness comes from. It is more complicated that htop, so needs reading man page to figure out which statistics are relevant.
     
    ahrasis likes this.
  13. ahrasis

    ahrasis Well-Known Member HowtoForge Supporter

    Me too. Even on my server with 288gb ram with old SAS.

    For WP, I converted some of my sites to github manually and keep them there instead. Only ones that are truly dynamic and really need to use WP are kept on my nginx servers with some free and simple but useful plugins.

    I rarely have to tune my WP sites so far but to note, learning and using nginx as my web server really helps me for the past 7 years more or less.
     

Share This Page