[SOLVED] Migration tool very slow on second run

Discussion in 'Plugins/Modules/Addons' started by Solipse, Jul 27, 2021.

  1. Solipse

    Solipse New Member

    Hello,
    Have to mention it's my first experience in using the Migration tool, so I'm just following the howto https://www.howtoforge.com/tutorial...-confixx-plesk-to-ispconfig-31-single-server/
    I'm migrating ispconfig from 2 VM on 2 different servers, local gigabit network, debian 9 to debian 10 and ispconfig fully updated on both sides.
    On first run of the migrate script, I noticed chmoding/chowning steps were pretty slow, but it was going at a reasonable pace (100 steps/1000 in 10 minutes). But the script failed, caused by a rewrite rule in a site which seams obsolete as apache refused to start after migration. This caused some steps to be on Error, so I though of deleting the site in the source which was obsolete by the way, and re-run the script.
    But then even though the syncing steps are nearly immediate, all the chowning takes like forever, nearly 1h per 100 steps.
    I don't really get it ...
     
  2. till

    till Super Moderator Staff Member ISPConfig Developer

    The speed of the migration depends on the speed of your systems (old and new one) and the speed of the network connection between your systems. So what's slow here is not the migration tool, if chowning on the new system is slow, then the process will take long, the migration tool has no influence on how long a chown process takes. Which filesystem does the new system use and does it use a local SSD harddisk or not? If chowning is slow, then the harddisk on the new system might e.g. be a network drive. And the more small files you have on your system, the longer a chown command needs.
     
  3. Solipse

    Solipse New Member

    What's bugging me is the speed difference between the first and second run. I didn't expect it to be slower than the first, isn it chowning just as much files ?
    On the destination I've a ZFS raid 10 pool of 4 hdd with lz4 compression enabled, the vm is using raw virtio disk format and ext4 for partitioning.
     
  4. till

    till Super Moderator Staff Member ISPConfig Developer

    At the first run, the hard disk was empty at the beginning and filled up step by step, this can make difference. You might want to check what the cpu and io load on the new system is. In any case, I doubt that you can speed this up without modifying the system setup of the new server, there is probably a bottleneck somewhere in the way the harddisk setup is configured.
     
  5. till

    till Super Moderator Staff Member ISPConfig Developer

    Just as an example (which does not match your setup though), I've seen such extremely slow chown commands (not from migration tool but from manual system config) on systems that were using GlusterFS for /var/www when the drive was filled with lots of small files, it was not low with large files and also not slow when the drive was not full.
     
  6. Solipse

    Solipse New Member

    Thank you for your answers Till,
    I think I spotted the bottleneck, as I had a RAM ballooning issue, conflicting with ZFS ARC which fills up automatically half the RAM, seems not considered as cache...
    Anyway, I wish I won't have to run the script soon !

    What relates more to Ispconfig, is the error that caused the migration script to fail. Here the culprit in a vhost conf purely generated by ispconfig. On the destination Apache failed, not recognizing the Rewrite clause.
    Code:
    RewriteEngine On
    Rewrite Base /
    RewriteCond %{HTTP_HOST} ^sub.domain.com$ [NC]
    RewriteRule ^(.*)$ https://sub.domain.com/data/ [R=301,L]
    
    I just saw the vhost has been postfixed with a .err in the source.
     
  7. Jesse Norell

    Jesse Norell ISPConfig Developer Staff Member ISPConfig Developer

    Change 'Rewrite Base' to 'RewriteBase'. In searching the code base I don't find 'Rewrite Base' anywhere (nor RewriteBase), why do you believe ISPConfig generated that?
     

Share This Page