The Perfect SpamSnake - Jaunty

Discussion in 'HOWTO-Related Questions' started by Tyiell, Oct 23, 2009.

  1. Tyiell

    Tyiell New Member

    Hi all,

    Having followed this guide as closely as possible, it all seems to be going wrong!! Ive already had some help from the author, but to save consistently PMing him in my ineptitude I thought Id bother you all with it as well :)

    The main issues I have here are:

    1: Section 16.1 - when I sorrectly postmap the line to hold emails for mailscanner the system holds ALL external mail, regardless of its validity.

    2: Whether I set postfix to hold the mail for mailscanner or not, all mail shows up in the Incoming Queue on the Mailwatch frontend until it fills up the memory. It doesnt, however, seem to scan it in any way, nothing is moved to quarantine, etc.

    3: If I run any reports on MailWatch I just get "Error: no rows retreived from database". Not entirely sure if this is because there is not data because its not scanning or because of an access problem with the database...

    Any thoughts would be welcome!!

    Cheers,

    Paul
     
  2. Rocky

    Rocky Member

    Hey,
    First, can you verify that mailscanner is running? If mail is being held, then mailscanner should pick it up, scan it and reinject it for delivery. Post a sample of your mail.log file.
     
  3. Tyiell

    Tyiell New Member

    Hi,

    Thanks for your reply. Im pretty sure the mailscanner is working - the process starts correctly and I installed the webmin module which says it is up and running. Whether Ive got it running correctly is clearly another matter ;)

    Theres a section of the mail.log:

    Cheers,

    Paul
     
  4. Tyiell

    Tyiell New Member

    Oh, you can probably guess, but 10.36.84.49 is the exchange server that the spamsnake forwards on to. The spamsnake itself is on 10.36.84.12.

    Many thanks,

    Paul
     
  5. Tyiell

    Tyiell New Member

    Hmm, or maybe you're right and it isnt working - sorry, it seems my knowledge of Linux isn't what it should be after these years of using it!!! below is what I get from a
    Code:
    ps -ef | grep Mail
    command:

    I think its running, but theres a few too many "defuncts" in there for me to be entirely confident!!!

    Many thanks,

    Paul
     
  6. Tyiell

    Tyiell New Member

    Here are the logs from mail.log since I checked the mysql setup, did the postmapping thing again and tweaked a few permissions that looked suspect to me:

    Now this is different to what I had before - all mail is being stopped andheld in the Inbound queue in MailWatch. It was in Outbound before :confused:

    MailWatch still doesn't show anything in the Todays Totals - implying to me that it is not actually processing them. No emails were then forwarded out of the system to be delivered.

    Any thoughts?

    Many thanks,

    Paul
     
  7. Rocky

    Rocky Member

    MailScanner defunct is no good. When did this start happening?
     
  8. Tyiell

    Tyiell New Member

    Well annoyingly, I'm not sure - after the first time I got it working, I never questioned it everytime I restarted it or whatever because it didn't give me an error. Lesson learned :(

    I am also getting the same error as Frogman in the other post - the Mail Transport Unavailable error, but my MailScanner still doesnt log any messages, and there are no rows in the database.

    Wierd :)
     
  9. Tyiell

    Tyiell New Member

    Aaaand just as annoyingly it has stopped doing this now (see output below), but occasionally when restarting the service I get an error saying "cannot touch '/var/lock/subsys/MailScanner' No such file or dir". I make the subsys directory and it works, but it disappears again in a day or so... :confused:

    Code:
    postfix  30832     1  0 08:40 ?        00:00:00 MailWatch SQL
    postfix  30834     1  0 08:40 ?        00:00:00 MailScanner: starting children
    postfix  30835 30834 12 08:40 ?        00:00:01 MailScanner: waiting for messages
    postfix  30846 30834 30 08:40 ?        00:00:01 MailScanner: waiting for messages
    root     30853 30477  0 08:40 pts/0    00:00:00 grep Mail
    But it still doesn't log anything! :(
     
  10. Rocky

    Rocky Member

    After you've recreated the subsys directory, redo section 16.2 to fix the subsys deletion problem.

    Rocky
     
  11. Tyiell

    Tyiell New Member

    Ahh. OK, done that - any further ideas on why mailscanner itself is not logging anything?

    Many thanks,

    Paul
     
  12. Rocky

    Rocky Member

    Post a sample mail.log when receiving mails. I think we're getting somewhere now.
     
  13. Tyiell

    Tyiell New Member

    Hi Rocky,

    Well I would - but I seem to have broken postfix with my playing with the master.cf file. I'm clearly a lost cause :eek:

    It doesnt seem to receive any emails whatsoever now - and nothing at all is logged. The last time it logged properly I got the foolowing:

    Code:
    Oct 27 12:49:21 BWD-SPAM-01 postfix/qmgr[30158]: 52DCFC8A105: from=<[email protected]>, size=1871, nrcpt=1 (queue active)
    Oct 27 12:49:21 BWD-SPAM-01 postfix/qmgr[30158]: warning: connect to transport smtp: Connection refused
    Oct 27 12:49:21 BWD-SPAM-01 postfix/error[3385]: 52DCFC8A105: to=<[email protected]>, relay=none, delay=2311, delays=2311/0.02/$/0.02/0/0.01, dsn=4.3.0, status=deferred (mail transport unavailable)
    Below is a section of my master.cf file (extends from the comments at the top to the relay - I havn't changed anything below this) - is this anywhere near right any more?!!


    Code:
    #submission inet n       -       -       -       -       smtpd
    #  -o smtpd_tls_security_level=encrypt
    #  -o smtpd_sasl_auth_enable=yes
    #  -o smtpd_client_restrictions=permit_sasl_authenticated,reject
    #  -o milter_macro_daemon_name=ORIGINATING
    #smtps     inet  n       -       -       -       -       smtpd
    #  -o smtpd_tls_wrappermode=yes
    #  -o smtpd_sasl_auth_enable=yes
    #  -o smtpd_client_restrictions=permit_sasl_authenticated,reject
    #  -o milter_macro_daemon_name=ORIGINATING
    #628      inet  n       -       -       -       -       qmqpd
    pickup    fifo  n       -       -       60      1       pickup
       -o content_filter=
       -o receive_override_options=no_header_body_checks
    cleanup   unix  n       -       -       -       0       cleanup
    qmgr      fifo  n       -       n       300     1       qmgr
    #qmgr     fifo  n       -       -       300     1       oqmgr
    tlsmgr    unix  -       -       -       1000?   1       tlsmgr
    rewrite   unix  -       -       -       -       -       trivial-rewrite
    bounce    unix  -       -       -       -       0       bounce
    defer     unix  -       -       -       -       0       bounce
    trace     unix  -       -       -       -       0       bounce
    verify    unix  -       -       -       -       1       verify
    flush     unix  n       -       -       1000?   0       flush
    proxymap  unix  -       -       n       -       -       proxymap
    proxywrite unix -       -       n       -       1       proxymap
    smtp      unix  -       -       n       -       -       smtp
    # When relaying mail as backup MX, disable fallback_relay to avoid MX loops
    relay     unix  -       -       -       -       -       smtp
            -o smtp_fallback_relay=
    #       -o smtp_helo_timeout=5 -o smtp_connect_timeout=5
     
  14. Tyiell

    Tyiell New Member

    Oh and I have tried changing that smtp unix - - n - - smtp line to the following with no noticable result:

    smtp inet - - n - - smtpd

    and

    smtp inet n - - - - smtpd

    I'm starting to think I'm making it worse everytime I touch it!!!
     
  15. Tyiell

    Tyiell New Member

    Ok despairing rant over!! - I have brought Postfix back to life. and the master.cf looks like this:

    Code:
    # Postfix master process configuration file.  For details on the format
    # of the file, see the master(5) manual page (command: "man 5 master").
    #
    # Do not forget to execute "postfix reload" after editing this file.
    #
    # ==========================================================================
    # service type  private unpriv  chroot  wakeup  maxproc command + args
    #               (yes)   (yes)   (yes)   (never) (100)
    # ==========================================================================
    smtp      inet   n       -       -       -       -       smtpd -v
    #submission inet n       -       -       -       -       smtpd
    #  -o smtpd_tls_security_level=encrypt
    #  -o smtpd_sasl_auth_enable=yes
    #  -o smtpd_client_restrictions=permit_sasl_authenticated,reject
    #  -o milter_macro_daemon_name=ORIGINATING
    #smtps     inet  n       -       -       -       -       smtpd
    #  -o smtpd_tls_wrappermode=yes
    #  -o smtpd_sasl_auth_enable=yes
    #  -o smtpd_client_restrictions=permit_sasl_authenticated,reject
    #  -o milter_macro_daemon_name=ORIGINATING
    #628      inet  n       -       -       -       -       qmqpd
    pickup    fifo  n       -       -       60      1       pickup -v
       -o content_filter=
       -o receive_override_options=no_header_body_checks
    cleanup   unix  n       -       -       -       0       cleanup
    qmgr      fifo  n       -       n       300     1       qmgr
    #qmgr     fifo  n       -       -       300     1       oqmgr
    tlsmgr    unix  -       -       -       1000?   1       tlsmgr
    rewrite   unix  -       -       -       -       -       trivial-rewrite
    bounce    unix  -       -       -       -       0       bounce
    defer     unix  -       -       -       -       0       bounce
    trace     unix  -       -       -       -       0       bounce
    verify    unix  -       -       -       -       1       verify
    flush     unix  n       -       -       1000?   0       flush
    proxymap  unix  -       -       n       -       -       proxymap
    proxywrite unix -       -       n       -       1       proxymap
    #smtp     unix  -       -       n       -       -       smtp -v
    # When relaying mail as backup MX, disable fallback_relay to avoid MX loops
    relay     unix  -       -       -       -       -       smtp -v
            -o smtp_fallback_relay=
    #       -o smtp_helo_timeout=5 -o smtp_connect_timeout=5
    showq     unix  n       -       -       -       -       showq
    error     unix  -       -       -       -       -       error
    retry     unix  -       -       -       -       -       error
    discard   unix  -       -       -       -       -       discard
    local     unix  -       n       n       -       -       local
    virtual   unix  -       n       n       -       -       virtual
    lmtp      unix  -       -       -       -       -       lmtp
    anvil     unix  -       -       -       -       1       anvil
    scache    unix  -       -       -       -       1       scache
    #
    # ====================================================================
    # Interfaces to non-Postfix software. Be sure to examine the manual
    # pages of the non-Postfix software to find out what options it wants.
    #
    # Many of the following services use the Postfix pipe(8) delivery
    # agent.  See the pipe(8) man page for information about ${recipient}
    # and other message envelope options.
    # ====================================================================
    #
    # maildrop. See the Postfix MAILDROP_README file for details.
    # Also specify in main.cf: maildrop_destination_recipient_limit=1
    #
    maildrop  unix  -       n       n       -       -       pipe
      flags=DRhu user=vmail argv=/usr/bin/maildrop -d ${recipient}
    #
    # See the Postfix UUCP_README file for configuration details.
    #
    uucp      unix  -       n       n       -       -       pipe
      flags=Fqhu user=uucp argv=uux -r -n -z -a$sender - $nexthop!rmail ($recipient)
    #
    # Other external delivery methods.
    #
    ifmail    unix  -       n       n       -       -       pipe
      flags=F user=ftn argv=/usr/lib/ifmail/ifmail -r $nexthop ($recipient)
    bsmtp     unix  -       n       n       -       -       pipe
      flags=Fq. user=bsmtp argv=/usr/lib/bsmtp/bsmtp -t$nexthop -f$sender $recipient
    scalemail-backend unix  -       n       n       -       2       pipe
      flags=R user=scalemail argv=/usr/lib/scalemail/bin/scalemail-store ${nexthop} ${user} ${extension}
    mailman   unix  -       n       n       -       -       pipe
      flags=FR user=list argv=/usr/lib/mailman/bin/postfix-to-mailman.py
      ${nexthop} ${user}
    
    policy unix - n n - - spawn
      user=nobody argv=/usr/bin/perl /usr/lib/postfix/policyd-spf-perl
    and the transport looks like this:

    Code:
    bullerswood.bromley.sch.uk smtp:[10.36.84.49]
    and the relay_domains looks like this:

    Code:
    bullerswood.bromley.sch.uk OK
    Im going to swap over to test it again after lunch and then il post the latest mail.log

    Many thanks!!
     
  16. Rocky

    Rocky Member

    Happy to hear you got it working. Always backup before changing anything. Below is what my master.cf looks like, keep in mind I haven't changed anything beyond the scope of the setup guide.

    #
    # Postfix master process configuration file. For details on the format
    # of the file, see the master(5) manual page (command: "man 5 master").
    #
    # Do not forget to execute "postfix reload" after editing this file.
    #
    # ==========================================================================
    # service type private unpriv chroot wakeup maxproc command + args
    # (yes) (yes) (yes) (never) (100)
    # ==========================================================================
    smtp inet n - - - - smtpd
    #submission inet n - - - - smtpd
    # -o smtpd_tls_security_level=encrypt
    # -o smtpd_sasl_auth_enable=yes
    # -o smtpd_client_restrictions=permit_sasl_authenticated,reject
    # -o milter_macro_daemon_name=ORIGINATING
    #smtps inet n - - - - smtpd
    # -o smtpd_tls_wrappermode=yes
    # -o smtpd_sasl_auth_enable=yes
    # -o smtpd_client_restrictions=permit_sasl_authenticated,reject
    # -o milter_macro_daemon_name=ORIGINATING
    #628 inet n - - - - qmqpd
    pickup fifo n - - 60 1 pickup
    -o content_filter=
    -o receive_override_options=no_header_body_checks
    cleanup unix n - - - 0 cleanup
    qmgr fifo n - n 300 1 qmgr
    #qmgr fifo n - - 300 1 oqmgr
    tlsmgr unix - - - 1000? 1 tlsmgr
    rewrite unix - - - - - trivial-rewrite
    bounce unix - - - - 0 bounce
    defer unix - - - - 0 bounce
    trace unix - - - - 0 bounce
    verify unix - - - - 1 verify
    flush unix n - - 1000? 0 flush
    proxymap unix - - n - - proxymap
    proxywrite unix - - n - 1 proxymap
    smtp unix - - - - - smtp
    # When relaying mail as backup MX, disable fallback_relay to avoid MX loops
    relay unix - - - - - smtp
    -o smtp_fallback_relay=
    # -o smtp_helo_timeout=5 -o smtp_connect_timeout=5
    showq unix n - - - - showq
    error unix - - - - - error
    retry unix - - - - - error
    discard unix - - - - - discard
    local unix - n n - - local
    virtual unix - n n - - virtual
    lmtp unix - - - - - lmtp
    anvil unix - - - - 1 anvil
    scache unix - - - - 1 scache
    #
    # ====================================================================
    # Interfaces to non-Postfix software. Be sure to examine the manual
    # pages of the non-Postfix software to find out what options it wants.
    #
    # Many of the following services use the Postfix pipe(8) delivery
    # agent. See the pipe(8) man page for information about ${recipient}
    # and other message envelope options.
    # ====================================================================
    #
    # maildrop. See the Postfix MAILDROP_README file for details.
    # Also specify in main.cf: maildrop_destination_recipient_limit=1
    #
    maildrop unix - n n - - pipe
    flags=DRhu user=vmail argv=/usr/bin/maildrop -d ${recipient}
    #
    # See the Postfix UUCP_README file for configuration details.
    #
    uucp unix - n n - - pipe
    flags=Fqhu user=uucp argv=uux -r -n -z -a$sender - $nexthop!rmail ($recipient)
    #
    # Other external delivery methods.
    #
    ifmail unix - n n - - pipe
    flags=F user=ftn argv=/usr/lib/ifmail/ifmail -r $nexthop ($recipient)
    bsmtp unix - n n - - pipe
    flags=Fq. user=bsmtp argv=/usr/lib/bsmtp/bsmtp -t$nexthop -f$sender $recipient
    scalemail-backend unix - n n - 2 pipe
    flags=R user=scalemail argv=/usr/lib/scalemail/bin/scalemail-store ${nexthop} ${user} ${extension}
    mailman unix - n n - - pipe
    flags=FR user=list argv=/usr/lib/mailman/bin/postfix-to-mailman.py
    ${nexthop} ${user}
     
  17. Tyiell

    Tyiell New Member

    Ok, postfix seems to be working correctly - when emails come in they are held ready for Mailscanner to checl them out. And there they stay because Mailscanner doesnt appear to be working. If I flush the Postfix hold queue or un-hold a message they are relayed to the exchange server and delivered correctly (and un-scanned by MailScanner).

    Any ideas whats up with Mailscanner? Below is the mail.log - it shows an incoming message being held and further down, being manually released.


    Code:
    Oct 30 14:34:18 BWD-SPAM-01 postfix/smtpd[8657]: connect from bwd-isa-01.bws-ed.bullerswood.bromley.sch.uk[10.36.84.48]
    Oct 30 14:34:18 BWD-SPAM-01 postfix/smtpd[8657]: D8A5FC8A11F: client=bwd-isa-01.bws-ed.bullerswood.bromley.sch.uk[10.36.84.48]
    Oct 30 14:34:18 BWD-SPAM-01 postfix/cleanup[8660]: D8A5FC8A11F: hold: header Received: from fg-out-1718.google.com (bwd-isa-01.bws-ed.bullerswood.bromley.sch.uk [10.36.84.48])??by BWD-SPAM-01.bws-ed.bullerswood.bromley.sch.uk (Postfix) with ESMTP id D8A5FC8A11F??for <cnewman@b from bwd-isa-01.bws-ed.bullerswood.bromley.sch.uk[10.36.84.48]; from=<[email protected]> to=<[email protected]> proto=ESMTP helo=<fg-out-1718.google.com>
    Oct 30 14:34:18 BWD-SPAM-01 postfix/cleanup[8660]: D8A5FC8A11F: message-id=<[email protected]>
    
    ...
    
    Oct 30 14:37:14 BWD-SPAM-01 postfix/postsuper[9694]: D8A5FC8A11F: released from hold
    Oct 30 14:37:14 BWD-SPAM-01 postfix/postsuper[9694]: Released from hold: 1 message
    Many thanks,

    Paul
     
  18. Tyiell

    Tyiell New Member

    And Mailscanner is definitely running - heres the output from ps -ef | grep post:

    Code:
    postfix   7771 11066  0 13:42 ?        00:00:00 pickup -l -t fifo -u -c -o content_filter  -o receive_override_options no_header_body_checks
    postfix   8658 11066  0 14:31 ?        00:00:00 tlsmgr -l -t unix -u -c
    postfix   8663     1  0 14:31 ?        00:00:00 MailWatch SQL
    postfix  10078 11122  0 14:38 ?        00:00:01 MailScanner: waiting for messages
    postfix  10249 11122  0 14:38 ?        00:00:01 MailScanner: waiting for messages
    postfix  10255 11122  0 14:38 ?        00:00:01 MailScanner: waiting for messages
    postfix  10261 11122  0 14:38 ?        00:00:01 MailScanner: waiting for messages
    postfix  10267 11122  0 14:38 ?        00:00:01 MailScanner: waiting for messages
    postfix  10869 11066  0 15:02 ?        00:00:00 showq -t unix -u -c
    root     11066     1  0 Oct29 ?        00:00:00 /usr/lib/postfix/master
    postfix  11071 11066  0 Oct29 ?        00:00:00 qmgr -l -t fifo -u
    postfix  11122     1  0 Oct29 ?        00:00:00 MailScanner: starting child
     
  19. Rocky

    Rocky Member

    Hey,

    Please delete your header_checks and header_checks.db files and then recreate header_checks. Make sure the following content is in it and postmap it when done.

    /^Received:/ HOLD

    If that doesn't work, post your MailScanner.conf file.
     
    Last edited: Nov 3, 2009
  20. Tyiell

    Tyiell New Member

    Hi Rocky,

    Ok I have deleted the header_checks files as you suggested - I had tried this before after you helped Frogman with his problem, but god knows enough has changed to make it worth another shot!! I can't test it until tomorrow morning though - the users will cheerfully assassinate me if there is any interruption to their facebook alerts :D

    Meanwhile the mailscanner.conf is attached - it wouldnt let me post it - I assume its too long, although the error the forum throws up is: "your post is too short, please lengthen your post to over 10 characters"!! :confused:

    Many thanks,

    Paul
     

    Attached Files:

Share This Page