Mailbox size policy & Cyrus

Hi,

I'm planning to implement a wrapper for Cyrus IMAP. I know, Cyrus is more complicated to install, but it has some advantages:

- More scalable than Courier
- Sieve mail filtering support
- Easy quota checking, e.g. for webmail clients

Integrating Cyrus support into ISPConfig won't be such a big deal, BUT:
Cyrus doesn't use system quotas, as it's running with its own user and group. This way quotas have to be setup directly in cyrus. That itself isn't a problem, as this could be handled similar to mbox files, but with mbox, you can limit the size by putting the mailboxes on a quota partition - with cyrus there is no such possibility.

This leads me to my question concerning a "mailbox size policy".

There are some different options how it could be managed:

1) The total size of all users' mailboxes (of one website) can't be more than the diskspace available to this web.
1a) This total size is subtracted from the group quota. Disadvantage: You might have used already more than the new group quota
1b) No subtraction takes place. Disadvantage: A website might use the double amount of space it was assigned.

2) Do it like the new SQL quota. Resellers have a limit_mail_space or sth like that, websites have a web_mail_space and can assign their users mailbox space from this mail space pool.

Advantage: Very flexible
Disadv.: A lot of different quotas

3) Resellers have just one total space limit, which includes sql, mail and web. Resellers then can setup webs:
sum(web) + sum(sql) + sum(web_mail_space) <= limit_space

Each website gets a web_mail_space quota, which is the maximum amount of space that can be dedicated to mailboxes:
sum(mailboxes) <= web_mail_space

Advantage: This gives (almost) maximum flexibility to resellers for their offers, but keeps handling resellers quite simple.
Disadvantage: I would have to change the sql quota handling, as currently there is a seperate reseller limit for it (but I guess as it's just been in trunk one day right now this shouldn't break anything *g*)

4) Just like 2 and 3 combined. One great total quota, but also seperate space quotas for web, mail, mysql. This way you can e.g. say "the reseller may use 1 GB of total space, but he may never assign more than 200 MB of database space"
Advantage: You can do whatever you want to Really maximum flexibility.
Disadv.: Much harder to implement.

What do you think?
I like 3), especially because it could be extended to 4) in future without breaking anything.

Hope you answer fast as I want to start as fast as possible... and I want to try getting it finished until friday.

 

I thought about replacing the Maildir switch by a Mailsystem drop down list, if possible.
But: I think it would be much easier and safer to have an option in config.inc.php, because in my opinion this should never be changed in a running system as you could screw up to much.

I'd suggest to keep the Maildir option for compatibility, but introducing an option in config.inc.php - and if the latter is set, the option in the settings tab is ignored and hidden.

Opinions?

 

Just a little question is left:
Shall I implement the quota behaviour for cyrus only? Or should mbox mailboxes be restricted by this total quota, too? (Or should I simply add another option to enable/disable the behaviour? If yes, should that be a config file option or also a setting in isp_server? I guess you'll answer the latter *g*)

 

The main reason why we hide this field for Maildir is that Maildir is in the users' homedirs and therefore falls under the group quota of the web whereas mbox mailboxes may reside on another partition without quota enabled.

I think we should also apply this to Maildir then, don't you think so?

 

I don't think this would make sense. I'm introducing a new value for limiting user_mailquota, and Maildir simply doesn't use that. Web quota of users shouldn't be touched by that, on the one hand because these are inside the boundaries of web space anyway, on the other hand because it's technically a WEB quota, that shouldn't be limited by some mail quota value.

But: As soon as Maildir would get its own mailbox quota, just like mbox and cyrus, I agree.

In short terms: I'd prefer to implement this behaviour for ALL mechanisms, but Maildir just doesn't use the value and thus isn't affected.

Another question:
As far as I understand the code, users are not really deleted in "user_delete", but only disabled, and get really purged in "web_user_clean". But the mbox file is already deleted in "user_delete"... Where should I delete cyrus mailboxes? Btw, it doesn't hurt creating mailboxes that already exist (e.g. when restoring).

And last: I'm quite sure I'll finish cyrus work today or tomorrow, cyrus interface itself is complete, just some cosmetic changes to do. But I still have to implement some imit checks.

Last question for this post: Cyrus IMAP is managed by an administrator user. This username is not hardcoded, but settable in /etc/imapd.conf. This means I need an option in ISPConfig to specify username and password of this user. Where should I add them? config.inc.php or isp_server? Maybe I should extend the installer to recognize if cyrus is installed, and if yes, ask during setup?

 

They can easily be recreated, but mails are lost when the mailbox is deleted. If I'm not completely wrong then currently mbox and Maildir behave different:

mbox: Mails are erased when deleting the user and won't be restored if you restore the mailbox.
Maildir: The mailbox is disabled when deleting, mail stays there until you empty the recycle bin.

Is this intended? Or am I wrong? (I currently have no ISPConfig ready for testing with Maildir or mbox, so I have to ask *g*)

 

You can specifiy any user as administrator in /etc/imapd.conf, so this wouldn't really help. Anyway, you have to tell ISPConfig the password somewhere.

Should I store it there also when asking for username and password during setup? If yes, should this be directly changeable through the server tab in ISPConfig? If also yes, should I hide theses two fields as long cyrus is not chosen? (Remember, if yes to the latter it would be necessary to reload the tab to show the fields)

 

Yes. Should be quite easy, if cyrdeliver (the cyrus delivery program) is installed I 'd assume Cyrus. Same should be true for some Courier binary.

I'd choose standard mode, as it is a similar question like postfix and proftpd.

I just see one problem: For automatic detection, I need to adapt the dists.txt file, but I don't have the possibility to examine the directory structure of all supported distributions. What's the best way to manage this?

 

I recently added cyrus support to subversion. I didn't touch the installer yet, cyrus has to be choosed in the isp_server tab, as I'm not really sure how the installer works.

Setting up Cyrus itself is quite easy, but it's very hard to make procmail deliver correctly, as the procmailrc files are run with users' privileges, and these are not sufficient to deliver mails to cyrus (if they were, every local user could pipe unscanned mails to every mailbox).

That means there is the choice between 2 possibilities:

1) Don't use procmail. That would definitly speed up delivery for huge amounts of mails. Filtering could be done with a postfix content_filter or by a sieve script. I'd like doing it with sieve, but that would also mean that both, prochmail AND sieve scripts, would have to be maintained in future. Should not be too hard, but I wanted to mention that Also the MailScanner would have to be replaced by something else, as a procmail script won't work with sieve. But I already found a nice peace of software that might be a perfect substitution (and even cares of partial mails, which currently get through unscanned AFAIK) - I just forgot the name, but I know where I have to look for it *g*

2) Run procmail scripts with delivery rights. Sounds simple, but it isn't that easy. Ok, running itself IS simple... but this way the users' .procmailrc files will also need to be run with these rights.
This means that users could change their .procmailrc file and do something very odd if they wanted to.
The rc files having root as owner and no write privileges for group and others doesn't help, as e.g. renaming the file and creating an own one is still possible.
Only (known) solution: Store the users' .procmailrc files (and all other files called by it) in another, not accessible folder. As all the procmail files shouldn't be changed by users in any case, this seems to be a good solution. I'd make this the standard behavoiur if cyrus is selected.

Comments please