email forwarding locally consumes all resources

Discussion in 'General' started by rdells, Feb 15, 2006.

  1. rdells

    rdells New Member

    FC4 - perfect suetup....
    Hi folks, all works great until I try to forward email to many same domain email addresses. IE: We have an email address that is used to forward the same message to all employees (from the boss to employees, the boss sends 1 email to the "staff@... email address, which in-turn forwards a copy of that email to 25+ employees).

    I was using the actual email address to forward to (25+ email addresses) and the first time, it crashed the computer! The second time around, I did forwarding to 1 user, IE: web1_user1@localhost. This worked ok.

    I then did all 25 web1_@localhost and all the computer resources got consumed, it did not crash however, however2- it took about 15 minutes to deliver all the same 25 emails!...

    Email forwarding seems ok when forwarding to email on another server.

    Tested some more:
    1) forwards fine (5 email addreses) to other email server with minimal resource usage.
    2) sending email to just 1 local email from 1 local email (just 1 email) system jumps to 100% cpu usage...

    Thanks in advance for any input,
    Robert
     
    Last edited: Feb 15, 2006
  2. till

    till Super Moderator Staff Member ISPConfig Developer

    Have you tried the email addresses of the users as destination, not their usernames. e.g. [email protected] and nit web1_tom@localhost.
     
  3. rdells

    rdells New Member

    Yes, I tried delivering to:
    1) [email protected]
    2) [email protected]
    3) web1_tom.smith@localhost
    from: Ie: [email protected]
    all 3 get delivered, however cpu load jumps to 100%.

    Under any of these scenarios, cpu load barely changes:
    FROM: [email protected]
    or FROM: [email protected]
    TO: [email protected]
    FROM: [email protected]
    TO: [email protected]
    or TO: [email protected]

    Seems only when sending emails between local accounts, Ie: from one employee to another employee of the same domain.

    (I only have 1 domain setup, so I have not tried one local domain to another local domain)

    Thanks, need a maillog posted?
    Robert
     
  4. falko

    falko Super Moderator Howtoforge Staff

    Yes, please post an excerpt of your mail log.
     
  5. rdells

    rdells New Member

    I am trying to get some of the log.
    the obvious (to me) on the mail log is the entries of:

    "ipop3d[1728]: Mailbox volnerable - directory /var/spool/mail must have 1777 protection."

    Is this normal for maillog??

    I did some more testing, on one email account:
    ON: mailscan, antivirus, spamfilter: cpu jump at 100%
    ON: mailscan, antivirus: cpu jump at 30%
    OFF: mailscan, antivirus, spamfilter: cpu jump at 30%
    Seems as though spamfilter is very intense!

    Is this normal behavior for spam filter?

    The Server is 800Mhz with Raid1, 256Mb Ram, Only one website.
    Maybe I need more power to run the spamfilter and antivirus?

    Thanks, will try and get the log for you....it's getting very large...gedit keeps crashing(I have desktop installed to help my 'rookie-ness').

    *********
    Ok, heres the maillog:
    The server = server1.myserver.tld (made this up)
    Workstation Router = 11.11.11.11 (made this up)
    I sent 1 plain txt email
    I recieved the same 1 plain txt email, plus another email

    Feb 15 12:58:59 server1 postfix/smtpd[20615]: connect from unknown[11.11.11.11]
    Feb 15 12:58:59 server1 postfix/smtpd[20615]: 30D093205FD: client=unknown[11.11.11.11], sasl_method=LOGIN, sasl_username=web1_info
    Feb 15 12:58:59 server1 postfix/cleanup[20592]: 30D093205FD: message-id=<019a01c63268$c9965ce0$6501a8c0@epm002>
    Feb 15 12:58:59 server1 postfix/qmgr[2453]: 30D093205FD: from=<[email protected]>, size=615, nrcpt=1 (queue active)
    Feb 15 12:58:59 server1 postfix/smtpd[20615]: disconnect from unknown[11.11.11.11]
    Feb 15 12:58:59 server1 sendmail[20682]: k1FKwx2i020682: from=web1_admin, size=116, class=0, nrcpts=1, msgid=<[email protected]>, relay=web1_admin@localhost
    Feb 15 12:58:59 server1 postfix/smtpd[20564]: connect from server1.myserver.tld[127.0.0.1]
    Feb 15 12:58:59 server1 postfix/smtpd[20564]: setting up TLS connection from server1.myserver.tld[127.0.0.1]
    Feb 15 12:58:59 server1 postfix/smtpd[20564]: TLS connection established from server1.myserver.tld[127.0.0.1]: TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)
    Feb 15 12:58:59 server1 sendmail[20682]: STARTTLS=client, relay=[127.0.0.1], version=TLSv1/SSLv3, verify=FAIL, cipher=DHE-RSA-AES256-SHA, bits=256/256
    Feb 15 12:58:59 server1 postfix/smtpd[20564]: A30FB320608: client=server1.myserver.tld[127.0.0.1], [email protected]
    Feb 15 12:58:59 server1 postfix/cleanup[20592]: A30FB320608: message-id=<[email protected]>
    Feb 15 12:58:59 server1 postfix/qmgr[2453]: A30FB320608: from=<[email protected]>, size=796, nrcpt=1 (queue active)
    Feb 15 12:58:59 server1 sendmail[20682]: k1FKwx2i020682: to=admispconfig@localhost, ctladdr=web1_admin (10039/10001), delay=00:00:00, xdelay=00:00:00, mailer=relay, pri=30116, relay=[127.0.0.1] [127.0.0.1], dsn=2.0.0, stat=Sent (Ok: queued as A30FB320608)
    Feb 15 12:58:59 server1 postfix/smtpd[20564]: disconnect from server1.myserver.tld[127.0.0.1]
    Feb 15 12:58:59 server1 postfix/local[20616]: 30D093205FD: to=<[email protected]>, orig_to=<[email protected]>, relay=local, delay=0, status=sent (delivered to command: /usr/bin/procmail -f-)
    Feb 15 12:58:59 server1 postfix/qmgr[2453]: 30D093205FD: removed
    Feb 15 12:58:59 server1 postfix/local[20595]: A30FB320608: to=<[email protected]>, relay=local, delay=0, status=sent (delivered to command: /usr/bin/procmail -f-)
    Feb 15 12:58:59 server1 postfix/qmgr[2453]: A30FB320608: removed
    Feb 15 12:59:07 server1 ipop3d[20709]: pop3 service init from 11.11.11.11
    Feb 15 12:59:08 server1 ipop3d[20709]: Mailbox vulnerable - directory /var/spool/mail must have 1777 protection
    Feb 15 12:59:08 server1 ipop3d[20709]: Login user=web1_admin host=[11.11.11.11] nmsgs=2/2
    Feb 15 12:59:08 server1 ipop3d[20709]: Mailbox vulnerable - directory /var/spool/mail must have 1777 protection
    Feb 15 12:59:08 server1 ipop3d[20709]: Logout user=web1_admin host=[11.11.11.11] nmsgs=0 ndele=2
     
    Last edited: Feb 15, 2006
  6. falko

    falko Super Moderator Howtoforge Staff

    Can't find anything suspicious in there.
    What's the output of
    Code:
    /home/admispconfig/ispconfig/tools/spamassassin/usr/bin/spamassassin -V
    ?

    You can run
    Code:
    top
    to track down the resource-hungry processes.

    Run
    Code:
    chmod 1777 /var/spool/mail
     
  7. rdells

    rdells New Member

    Thanks for that info:

    ran: /home/admispconfig/ispconfig/tools/spamassassin/usr/bin/spamassassin -V
    SpanAssassin version 3.1.0
    running on Perl version 5.8.6

    ran top:
    Only hungry processes:
    web1_usersname(getting the email): cpu @ 55% when spamfilter is ON, server receiving email
    postfix(getting the email): cpu @ 5% when spamfilter is OFF, server receiving email

    I turned off the spamfilter on all the accounts for now(except the one I am testing with). Having the antivirus turned on seems good(cannot tell that it's running, and it is working).
    Robert
     
  8. falko

    falko Super Moderator Howtoforge Staff

    Can you post the output of
    Code:
    uname -m
    ?
     
  9. rdells

    rdells New Member

    Here you go:
    i686
     
  10. falko

    falko Super Moderator Howtoforge Staff

    That's ok, too.
    You can try to compile SpamAssassin yourself on your system - maybe your system doesn't like the one that comes with ISPConfig. This is how you do it:

    Code:
    tar xvfz Mail-SpamAssassin-3.1.0.tar.gz
    cd Mail-SpamAssassin-3.1.0
    perl Makefile.PL PREFIX=/home/admispconfig/ispconfig/tools/spamassassin/usr SYSCONFDIR=/home/admispconfig/ispconfig/tools/spamassassin/etc
    make
    make install
     
  11. rdells

    rdells New Member

    Thanks again for your all your advise, really appreciate it.
    I did not do the previous post - I like install and leave alone (nervous)!!
    I did read about spamassissin and realize now that my machine probably is not powerful enough to have spamassassin on more than a few accounts!
    The FAQ on spamassasin told me the story:
    http://wiki.apache.org/spamassassin/FrequentlyAskedQuestions
    Again, Thanks
    Robert
     
  12. till

    till Super Moderator Staff Member ISPConfig Developer

    How many email accounts do you have and what processor and how much ram does your server has?

    Which of the FAQ articles do you refer to?
     
  13. rdells

    rdells New Member

    Hi there,

    I suppose I could do some tweaking, I just moved onto other things for now.
    One FAQ: http://wiki.apache.org/spamassassin/FasterPerformance
    talked about fine tuning. Looks useful, just have not tried yet.

    I added ram: was 256 (when had issues), now 512 (still hungry).
    Cpu is Duron 800Mhz, Shuttle AK10 mobo.
    30 email accounts. was AV + Spamfilter (when had issues), now only AV (seems good).

    2 WD IDE 80GB hard drives (raid1, hda, hdc) not using "Quota" (could not figure out how to do Quota with raid1 w/in fstab file).

    Note: I did the same setup on a P3 833Mhz, 512Mb ram, near-same all other hardware. I got the near-same results from Spamassassin enabled on the email accounts (near 100% cpu jump while sending to the email acct w/spamassassin enabled, minimal cpu% with only AV enabled).

    I may try tweaking Spamassasin in a few days, I'll be sure to let you folks know of outcome if you like, Maybe I should have tried Debian for the older hardware?? !!

    (I liked the Fedora Raid setup, was easier than debian :)).

    Thanks always,
    Robert
     
    Last edited: Feb 20, 2006
  14. falko

    falko Super Moderator Howtoforge Staff

    That would be great! :)
     
  15. MathieuMa

    MathieuMa New Member

    Same issue here

    Hi,

    I have a mail address setup to forward to 4 other mail addresses (as account-less mail list are not possible yet).

    When receiving a mail to that list, even though the antivirus is setup on it, it forwards the mail to the 4 other accounts, and because of this the mail is scanned 4 times - and passes through spamassassin 4 times as well.

    Could it be possible to have the forwards done AFTER the scans ?
    And then use added headers to not run those twice (that's just an idea - not sure what it's worth regarding security, IE if the sender set that particular header we should remove it first) ?

    The issue here is that this takes a lot of CPU (4 scans) , and when a virus is found a mail is sent 4 times (once per account) to the mail list saying a virus was found (disabled that for now).


    Thanks, Mathieu
     
  16. falko

    falko Super Moderator Howtoforge Staff

    Are those 4 email addresses on the same server, and is SpamAssassin enabled for these 4 addresses?
     
  17. MathieuMa

    MathieuMa New Member

    The 4 addresses are on the same server, and SA / Antivirus is enabled on those 4.

    Performance wise, what I'd like to have would be the following :
    - SA scan on the alias done -> no more SA scan on the ML mails
    - Antivirus scan on the alias done -> no more AV scan on the ML mails

    Or at least have virus mail not forwarded, they should go to /dev/null as soon as they hit the alias, but are forwarded before apparently.

    Oh, the forwards are done to [email protected], not user_name - but that should be correct ?
     
  18. falko

    falko Super Moderator Howtoforge Staff

    We can think about how we can accomplish this, but right now it's not possible with ISPConfig.

    Yes, that's ok.
     
  19. MathieuMa

    MathieuMa New Member

    OK, will wait for next version or better try go get an idea :D
    The issue with internal ML as of now is that they consume a lot of resources due to antivirus and antispam - I thought it could be just a processing order question for a part of the issue (antivirus before forward -> if the message is discarder it's only scanned once).
     
  20. chrno

    chrno New Member

    Hi falko,

    I'm seeing the same thing when I've SpamAssassin and ClamAV selected.

    I've got an email address that sends the same email (hourly reports) to about 4 email address (managers's address) on the same domain. Not via forwarding in ISPconfig, just notmal To: and Cc: via smtp. And of course these emails will be received at the same time by the mail server. This's when I see a spike in the CPU (going as high as 100% usage). Top at that time shows that ClamAV and SpamAssassin is using +-25% for each of the 4 email users (web3_username)... which prompted me to ask about the performance impact on the server when SpamAssassin/Mailscan/ClamAV is activated (This Thread). For now, I've deactivated SpamAssassin and ClamAV on all these 4 accounts because I thought it's because I'm using a develepment copy (ISPConfig-2.3.0-dev.tar.gz) until I saw this thread ...

    Should I try to compile my own SpamAssassin? If I compile it, will I have upgrade issues with the next release/version of ISPConfig?

    Additional Info:-
    Code:
    /home/admispconfig/ispconfig/tools/spamassassin/usr/bin/spamassassin -V
    SpamAssassin version 3.1.0
    running on Perl version 5.8.8
    
    Machine spec
    AMD Duron 1GHz with 256MB ram 
    swap 700MB
    
    OS
    Debian Sarge
    
     
    Last edited: May 1, 2006

Share This Page