(RESOLVED) Backup files: format and ownership

Discussion in 'Installation/Configuration' started by TonyG, Jan 23, 2021.

  1. TonyG

    TonyG Active Member

    Gosh, I hope this isn't a FAQ. I've found many discussions on this topic but most are very old and I don't see focused answers.

    In Server Config "Backup tmp directory for zip", there are two options:
    - Backup web files owned by web user as zip
    - Backup all files in web directory as root user
    That seems to be very limited but I'm sure I'm missing something.

    Assume backups go to /var/local/backups/ispconfig. The folders under that include web* and mail*. The backups are all zip because I chose the first option. I wanted the files to be stored in a single location on the server, not in the web directory, with web user ownership.

    But the files are all owned by root:root. And because I'm using zip files the ownership and permissions are not preserved.

    So what can we set so that we get backups saved outside of webroot, with web user ownership, and in a tar file that preserves metadata?

    Thanks.
     
  2. TonyG

    TonyG Active Member

    In a related challenge - To rsync this data from /var/local, I'm not sure what permissions a SFTP client should have. For administration my "ubuntu" user is in the "staff" group. Then I put some files in root:staff with rw perms so that the ubuntu user can work on them. I was thinking it would be convenient to put all backups under group staff for the same reason. But we can't have those files under group clientN as well as under staff. If under staff then clients wouldn't be able to restore files. An alternative there is to add the ubuntu user to any clientN group ... and at the moment since I only use this with one client1 = me, that's easy. But it's not scalable or useful for others here.

    So other than root, what user/group should we use to faciliate rsync? Or what would we use for example with SFTP where we can't sudo, other than to create a default user for every client?

    Thanks!
     
  3. TonyG

    TonyG Active Member

    I'll add in yet another post-script... I know that in Sites > Web Domain we can change Backup > Compression Options. Where the backup type is "Default: zip (deflate) or tar (gzip)" I believe it take the default from System Config as noted above. We can override this on a per-site basis. I would prefer not to set the System Config as "Backup web files owned by web user as zip" and then have to override the compression on a per-site basis. I'm hoping we can set all of this at the sytem level for all sites, now and those to come.

    With regard to zips not saving permissions. I haven't tried this but if a Restore is requested from a zip file, wouldn't that restore the file system with all files at root:root? That would be a mess. Whomever is restoring would then just need to know that they need to go in and fix all of the permissions. I'm not going to click the button to test, but is there a warning about this when someone does click it?
     
  4. TonyG

    TonyG Active Member

    bump - TY
     
  5. Th0m

    Th0m ISPConfig Developer Staff Member ISPConfig Developer

    I think the answer to most of your questions is that ISPConfig should set the correct permissions when restoring a backup.

    But your thread was kind of confusing to me so I might have misunderstood something - and what you are trying to do.
     
  6. Jesse Norell

    Jesse Norell ISPConfig Developer Staff Member ISPConfig Developer

    No, the files are not all owned as root. IIRC, zip's do not store unix file permissions and would hence loose those if backing up in a zip format.
     
  7. TonyG

    TonyG Active Member

    Yeah guys, I knew this was confusing when I wrote it. Sorry. I'll take this in smaller bites...
    That's exactly my point. Zip files have no ownership data. Files restored from a zip have the ownership of the user restoring the data. Permissions need to be manually restored, and with no permissions metadata in the zip, this becomes very difficult.

    Why isn't there a system-wide option to backup to gz where all ownership and perms are preserved?
    I'm not complaining. As usual I just think I'm missing something because I can't believe this zip option is used by anyone.

    Thanks!
     
  8. TonyG

    TonyG Active Member

    This is related. In this screenshot we see that for a single mailbox there are two daily files. The zip is the actual /var/vmail tree for the account/user. What's in the gz file, and where is it? I can't find these files in the file system. Thanks.

    upload_2021-2-4_13-41-18.png
     
  9. Jesse Norell

    Jesse Norell ISPConfig Developer Staff Member ISPConfig Developer

    All backup files should be in the same place, so if the zip files exist that's where the .tar.gz should be as well. Maybe it's just bad entries in the mail_backup table? You could try running the mail backup from the cli with debug mode enabled and see what that shows, I'd guess it's a bug.
     
  10. TonyG

    TonyG Active Member

    It was bad data. I manually deleted the backup folder last night. No need to tweak the database. As of today, only the existing files are in the list, and all files listed are in the proper folder.

    I went back through all of this and now I understand what this product is telling me. But it seems really cryptic. There are nuances about the options, details implicit in the data that is saved, unstated requirements for restore that a sysadmin just needs to figure out on his own. I didn't get that before. I do now. It's too much to write up here. But as always, if and when a documentation medium is ever decided here, I'll be happy to write up notes on this.

    This thread is Resolved. Thanks.
     

Share This Page