Migration Tool - Testing berofe migrate

Discussion in 'Plugins/Modules/Addons' started by esezako, Sep 8, 2017.

  1. esezako

    esezako Member

    How can you test/view a website on the new server before finishing the migration? Ie before putting the ip of the old server on the new one.
    I know that I can access the ispconfig panel of the new server with the temporary ip, but I don't know how i can test the websites on the new server while the old server is running.
     
  2. till

    till Super Moderator Staff Member ISPConfig Developer

    budgierless likes this.
  3. budgierless

    budgierless Member HowtoForge Supporter

    I'd like to add a question.
    If I already have a website on new server in folder web1 but I have a different domain on old server in web1, will the script overwrite new server web1 or make a new web2 when migrating content?
     
  4. till

    till Super Moderator Staff Member ISPConfig Developer

    The migration tool will make a new web2 folder.
     
  5. budgierless

    budgierless Member HowtoForge Supporter

    The script is working well and has been transferring over the last 24 hours.
    What I would like to know is if their is a way once the process has completed I could then modify the generated bash script to run the process again but make rsync check for changes to data then amend where found. For example: if files deleted/removed on the source server then remove on target server if exist, also if newly made data found on source server but not found on target server it then migrates the new data, and if database size is different then rsync to its the same as source database size?

    The reason I ask this is because with the initial migration process my server will take a very long time to transfer is the data and by the time the process has completed (48+ later), lots of changed data would have been added/removed.

    UPDATE Feature for other users with large capacity data servers request:
    I understand that no matter what happens some amount of data will be lost because of transfer time, but if only a few hours data lost then fine but not few days.
    Since your script uses rsync, can it not be made to also generate an updater script so it can do what it says and actually RE-Synchronise? So in most cases the databases would be the main caveat due to being the largest block file and time consuming part of the re-sync but that's fine.
     
  6. till

    till Super Moderator Staff Member ISPConfig Developer

    You can just run the data copy script (the one that is generated by the migration tool) again on the target system to resync the data. The script uses rsync for data transfers, so it will only sync the changes and this second run will be much faster of course. Databases will be resynced completely as they need to be dumped, copied and reimported.
     
  7. budgierless

    budgierless Member HowtoForge Supporter

    thanks for that,
    I problem is that the script has not completed yet, and its been a few days, at the moment only the traget server hard drive it fully active, so im guessing its dealing with data dump, but should it take so long? is the scipt able to handle dumps mush larger then 20G+ also is thie any flag that can be used to monitor dump process?
    UPDATE: i checked the process in the system monitor, and it is the grep that is active and taking the long time, so i guess that is to do with the mysql transfar. i with i could see the procress so i know how mush is completed.
     
    Last edited: Sep 15, 2017
  8. Croydon

    Croydon ISPConfig Developer ISPConfig Developer

    The grep command that (I guess) you are seeing is to rewrite paths in website files from the old one to the new one. It would be helpful to post the actual grep command that you are seeing (you can anonymize paths to strip client information) so I can give you a more specific answer.
     
  9. budgierless

    budgierless Member HowtoForge Supporter

    grep say nothing only that its active, but I worked out what the issue is why its taking so long. For some reason over a year or so ago sub-domains that were deleted are being recoved some-how by the script, why dose the script do this (reiving incremental file list) and not just migate what is active on source server?
    UPDATE: instead it says converted, and im thinkeding its sub-sites the used the old vhost sub-domain method from ispconfig 3.0 (wehre the sub could use a different verion of php intead of the root domains php), i could be wrong, i dont know.
     
    Last edited: Sep 17, 2017
  10. till

    till Super Moderator Staff Member ISPConfig Developer

    The tool migrates what it finds in the ISPConfig database, so the sites that you that thought were deleted must still exist in the database, otherwise, they won't be migrated. Vhost subdomains can use different PHP versions and even PHP methods in both ISPConfig 3.0 and 3.1. When your old ISPConfig 3.0 server used a different PHP version, then the new one will do that as well. But @Croydon might give a more detailed answer here.

    When rsync says "receiving incremental file list" then this means that it sends a list of files from old to new server to decide what needs to be copied (to avoid to copy files twice). This does not mean that it uncovers any old files from somewhere.

    Which ISPConfig version do you run on the old server?
     
  11. Croydon

    Croydon ISPConfig Developer ISPConfig Developer

    May it be that you are using ISPConfig 2 on the source server? Then you seem to have not emptied the trash folders inside the ISPConfig UI as told by the migtool. This leads to deleted (only trashed) sites being copied over.
     
  12. budgierless

    budgierless Member HowtoForge Supporter

    Maybe better to show you, beacuse i dont think i explained very well:
     

    Attached Files:

  13. Croydon

    Croydon ISPConfig Developer ISPConfig Developer

    That's why I posted this.
     
  14. till

    till Super Moderator Staff Member ISPConfig Developer

    When you want to compare paths, then use the domain name to enter the path as web9 on the old server has not necessarily the same ID in the new one. So the way you compared it now, it might be that you compared tw different websites. To ensure that you compare the right websites, use the domain name path, e.g.:

    ls -la /var/www/yourdomain.tld/

    on old and new server to compare the directories.

    Beside that, please answer the question which ISPConfig version is used on the old server.
     
  15. budgierless

    budgierless Member HowtoForge Supporter

    Sorry forgot to say that both servers are using ispconfig 3.1, also target server was clean install with no domains.

    I did not notice any trash bin for ispconfig 3.1, never used it over the years using ispconfig haha, so maybe that's the issue, where is it located? and I will check it out when I get home later.
     
  16. till

    till Super Moderator Staff Member ISPConfig Developer

    That's only when you use ISPConfig 2 as source.

    My guess is that you either compare different websites, see post #14 above on how to compare the correct sites (do not compare by webID folder!). Or you did not delete the vhost domains in ispconfig (check on the source is there are really no vhost sub- or aliasdomains listed anymore.
     

Share This Page