I'm working on a project that will basically be a Constant Contact like system. Mass emails that someone creates and we are the service which sends, opt-in, opt-out. Where I need help is planning for scalability. A single server emailing is easy but I want to plan for 1 million emails a day and it could be 10,000 per hour. My thought is that I would need to setup an email cluster with some sort of load balancing deciding which server will send out a batch of emails if another is processing a different users request. Bandwidth is not an issue. Am I correct in thinking an email cluster? I've never setup ISPC with multiple emails servers. My plan was a Perfect ISPC Wheezy setup with Courier....can Courier scale? I've made plenty of mistakes having to redo implementations because I didn't ask the experts ahead of time. Any advise or guidance would be appreciated.
Postfix can handle a lot of mails without any issue. It´s up to you to tune your postfix installation. I would not use courier anymore. If you use dovecot, you can use dsync to replicate the mails between two servers. You can also build a cluster with dovecot and forward the users to a server based on load-balacing and/or use a transparent forwarding for dovecot. Setting up a mailcluster for sending mails is not very simple....
Thank you for the feedback. I should have clarified that this will be sending only by the server, no external connects or webmail or incoming ...etc. 1. ok ...ditch Courier. (I'll have to work on moving to dovecot on my other installs). 2. looks like I'm still stuck on the mailcluster, I was afraid you'd say it wasn't easy. How about this for a crude implementation. Setup multiple sending servers mail1.domain.com, mail2.domain.com, mail3.domain.com. Since they are outgoing only, in my head, this doesn't seem hard. As the website gets requests to send, we check the outgoing queue of each server to determine which to send the batch to and either send all to one, or split and send to each. Again, since there is no incoming, I think this may work.
Mabe you can use different transport_maps. Send mails for domain a, b,c to out-server1 and for d,e,f to out-serve2. Or play around with a san. But this is not a usual setup with ispconfig. You can create a out-cluster with postfix but i´m not sure if you can configure every needs with ispconfig.
yeah I had a feeling I may have some conflict somewhere in ISPC. I'll have to do some reaearch on how fast a single server can send emails, but I'd hate to config something and then have to start from scratch when I really need it. I'm sure I really need multiple threads sending at the same time so the emails get out efficiently and not stacked up waiting for the next. Thank you again Florian.
You can run multiple instances on one host. You may also want to use a fallback_relay and use a ram-disk for the queue. www.postfix.org/MULTI_INSTANCE_README.html http://www.postfix.org/TUNING_README.html#rope
that looks like the winner at least to start. Would I be correct in saying ISPC will only interact with the main postfix and I'll need to decide which postfix instance to send outgoing email through via the website process (php)?
While its great to try and do this with all open source software your going to run into problems with trying to send that much traffic to gmail and other large IP's. Gmail.com only allows 2000 emails a day from a single IP. I would look at using Amazon email services for low cost sending of that much email. Not free but very cheap. There is an open source software that will do what you want and use AWS as the back end so you don't have to worry about getting your IP's blacklisted. Take a look at http://phplist.com and another one that is not free but relatively inexpensive is http://sendy.co Would save tons to development time as well. Good Luck!
