I'm in the process of slowly moving services from a mulitserver setup with an outdated version of ISPConfig to a mulitserver setup with the latest release and refreshed hardware. I have about 15 email domains, about 150 users. I know that there is an email import tool, but I only want to do one domain at a time because I didn't have SSL on the old server and I'm implementing it on this server so I've been calling/emailing clients individually to get their account settings updated. I have the old server doing an rsync backup nightly to a backup machine that both of the servers have access to, my practice has been to redo the rsync for the domain that I'm doing at that time and then then do an rsync from the backup server to the new ISP email server. This has worked okay, but there are a few hangups that give me cause for slight concern. 1) I have a user that has 12436 emails in /var/domain/user/Maildir/cur/ but when we login to the users account in webmail (roundcube) it only shows 9625 emails. When I do the rsync to the new server and login to webmail there, it shows 12436 emails in the inbox. I'm trying to determine why the emails aren't showing up in the existing server and/or why they are showing up in the new one. 2) Not sure if I should be doing anything with the dovecot.index files, if this is something that is automatically generated or if this is part of my display/actual number values This really isn't an ideal scenario, but I want to have a conversation with all the clients while I do this changeover about their security settings and changes that should be made. All and all, this isn't working terribly, I've done a couple of 'friendly' domains with a few users and haven't encountered any problems yet. I'm creating the mail domain/user accounts in ISPConfig first before doing the rsync.
I've done a lot of dovecot to dovecot migrations the past decade, and dovecot is pretty tolerant. Much more so than Courier or others. What I would do, considering what you wrote, is reindex and be sure the dovecot version is fairly up to date, maybe even update it to the latest possible on that server by backporting or using a custom repo (Peter Does has one, you can google him); On the old source server, as root, do: # doveadm -v index -u '*' '*' (the -v is verbose, so you see it actually do stuff), with 150 users this can take some time, but it's entirely safe to do. For assurance, also sync the caches: # doveadm force-resync -A '*' Depending on the version differences between source and target dovecot, you could leave it all as is, or exclude *all* dovecot generated files with rsync from the Maildirs and let dovecot generate them anew on the target after migration. I've often NOT done this, copied it all 1:1, and it wasn't a big deal, some dirs were not matching their content, so you have to re-index on the new server as well. The problem with not excluding dovecot files is that dovecot has changed quite a bit the last few years, so maildirs tend to fill up with dovecot files that aren't being used anymore which is of course wasting storage and parsing space.
Here's a post mail migration shell script I've often used to correct some dovecot permission issues. Note that /home/prefs and /home/attach are Squirrelmail folders, so you can omit them. Code: #!/bin/sh chown vmail:vmail /var/vmail/ -R cd /var/vmail find . -type d -name Maildir -exec chmod 700 -R {} \; find . -type f \( -name '.sieve' -o -name '.sieve.svbin' \) -exec chmod 644 {} \; find . -type f \( -name 'dovecot-uidlist' -o -name 'dovecot-uidvalidity' -o -name 'dovecot.index*' -o -name 'maildirsize' \) -exec chmod 600 {} \; find . -type f \( -name 'dovecot-uidvalidity.*' \) -exec chmod 444 {} \; find . -type f \( -name 'subscriptions' \) -exec chmod 744 {} \; chown root:www-data /home/prefs -R chown root:www-data /home/attach -R and a cron script that cleans up people's Junk and Trash, you may want to run it on the old server before migrating; Code: #!/bin/sh doveadm expunge -A mailbox Junk savedbefore 90d doveadm expunge -A mailbox Trash savedbefore 660d doveadm force-resync -A '*' exit 0 This kills Spam older than 90 days, and Trash older than ~2 years. Most users never complain if you do this..
This is a lot of good information, I appreciate the detail. I've run the doveadm index and force-resync, it doesn't seem to change this descrepency in mail counts I'm seeing between the old server and new server. It's a concern for me because one of my users reported after I cut the over that the received a bunch of email that they had previously deleted, so I feel like whatever is going on is causing emails to re-appear. With that in mind, it almost seems as though the emails aren't being fully deleted by the imap client on the old server, and for whatever reason become available again on the new server, if that makes sense. FWIW, the versions of dovecot are 2.2.9 on the old server and 2.2.22 on the new (both are the latest version available in the default repos for their distro)
Try deleting *all* the dovecotgenerated files under Maildir (per directory that is showing this discrepancy), and let docevot do its thing on the new server with # doveadm -v index -u '*' '*' Or check the dovecot wiki for doveadm, it has a lot of options to solve these things. Success guaranteed, if the ( /cur /new etc) files are actually maildir format files dovecot will index them properly. Have you cat/viewed some of the files that weren't indexed? Maybe they're not in the right format, or corrupted?
Well, I'm an idiot. Your advise was very helpful in a couple of cases, but the primary issue I was running into was roundcube viewing email in 'threaded' view vs not threaded view, resulting in a different number of displayed emails. That being said, I did use the information presented here to clean up some other minor issues I was having, so big thanks.
