fetchmail support

Discussion in 'ISPConfig 3 Priority Support' started by craig baker, Mar 5, 2014.

  1. craig baker

    craig baker Member HowtoForge Supporter

    I may have screwed the pooch but is fetchmail automatically installed by ispconfig3?

    I did a yum install fetchmail and it did so installing fetchmail6.3 - and ispconfig3 now has a fetchmail entry on the left pane that I dont recall seeing before. does it recognize it automatically? or was my yum install pointless?

    I setup an account, and told fetchmail where to get it - but how can I see if fetchmail is actually working? and this pop server has a LOT of emails and I cannot delete them (would cause mondo problems) - how can I check the config to make sure fetchmail wont disturb that email???

    where does it put its logs?
    thanks and love ispconfig3 as always
    cdb.

    cdb.
     
  2. craig baker

    craig baker Member HowtoForge Supporter

    fetchmail

    also where is fetchmail pop port setup if they want to use 995 and ssl?
    thanks
     
  3. craig baker

    craig baker Member HowtoForge Supporter

    see some mail flowing but no attachments

    I see some mail flowing (obviously fetchmail is working) but attachments are being stripped where is the config file?
    thanks
     
  4. till

    till Super Moderator Staff Member ISPConfig Developer

    ispconfig uses getmail, a modern fetchmail replacement and getmail is installed as part of the perfect setup guides. getmail dos not strip attachments, so your issue is mor likely related to amavisd. The getmail configuration is in /etc/getmail/ Please remoe the fetchmail rpm that you installed to avoid conflicts.
     
  5. craig baker

    craig baker Member HowtoForge Supporter

    ok - but

    is the fetchmail setting actually referring to getmail then?
    and where are getmail logs etc.
    and how do I keep amavisd from stripping innocuous attachments? (a jpg file!)
     
  6. craig baker

    craig baker Member HowtoForge Supporter

    fetchmail removed... another thing I just noticed

    yum erase fetchmail executed.

    when I do a pop3 download from my fetchedmail, thunderbird shows the correct dates (these emails date back to 2012 and the pop3 download is oldest first)
    but Outlook 2007 shows all the mail is being received today! ouch.

    anything to suggest? or just obvious outlook bug? I see no setting anywhere nor even how to ask to get the original date!
     
  7. till

    till Super Moderator Staff Member ISPConfig Developer

    Do the corect email timestaps show up in webmail client? If yes, then its most likely a outlook bug.

    If you want to sync emails between accounts, I would use the program imapsync instead of fetchmail / getmail.
     
  8. craig baker

    craig baker Member HowtoForge Supporter

    outlook bug

    outlook DOES have a 'sent' field and that shows the correct date. and yes the webmail client to the mondo message pile has right dates everywhere.

    thunderbird and outlook are alternatives not both.
    problem is that outlook at the original message site will download all 135,000 messages. and start downloading them again!

    I'm hoping that using getmail and copying them to my server will behave properly.

    thunderbird seems to have a 4gb message store limitation even in the 64bit version :(

    getmail is certainly getting mail and I see messages in the logs. and its delivering it to my local mailbox which (unfortunately) has to have the same address as the original mailbox on a different server. but we seems to be avoiding confusion.

    getmail seems to download only a few messages at a time - I had expected a huge connection time for the 135000 messages on the server! but this seems to be functioning ok.

    the attachments being stripped is a problem though they are definitely on the server (and thunderbird and outlook seem to behave the same).
    so why would amavisd strip a jpg file?

    thanks again for help!
     
  9. craig baker

    craig baker Member HowtoForge Supporter

    getmail a big problem

    have to work out why the jpgs are getting stripped - have an email with 5 attached. roundcube shows no attachments, and neither do outlook/thunderbird!

    Oh - in case anyone asks you about the date field being completely empty in roundcube - turns out thats due to no date.timezone set in the php.ini
    I just added America/New_York and all dates came back nicely!

    I dont see anything in amavis saying they strip non-infected attachments but obviously SOMEONE is - where do I look?
    thanks
     
  10. craig baker

    craig baker Member HowtoForge Supporter

    further info

    its not ALL attachemnts - pdfs seem to be fine
     
  11. craig baker

    craig baker Member HowtoForge Supporter

    possible solution - how to?

    how can I have 1and1.com (the domain where getmail is getting all this huge pile of messages) whitelisted as a SOURCE so that amavisd ignores messages download from it? (not the same as whitelisting the email-originating domains which will be a huge number of different folks)?

    or have getmail deliver mail from 1and1 directly to the mailbox on my server without going through anyone? at least temporarily?
    1and1 scans all its emails i'm sure so I'm not worried about viruses and spam - I'm sure there are none.
    well spam sure.
    but I need to get ALL the mail onto my server with all attachments!

    one last thing when I get this worked out I'll have to delete all the email in the box and start the getmail again to make sure all is got properly - if I delete the email box from inside ispconfig3, do I properly delete all the emails so I can start fresh on the 135,000 pile?

    one more issue any way to throttle getmail a bit? its still chewing on the 135,000 message (up to 35,000 or so) and its really crowbaring my email system - I've got other emails just not getting delivered.

    I notice in the perfect setup, qshape is NOT included with postfix! I see a fix somewhere:
    --snip--
    yum groupinstall perl development” once you have that all done, (and here’s the magic) run:
    “yum install postfix-perl-scripts”.
    --snip--

    but didnt want to do it till I checked any reason NOT to run these to install qshape? (and why was it removed anyway)?

    I look with postfix -q and I see lots of lines like
    D514D1B1C34 15823 Wed Mar 5 15:34:40 [email protected]
    (delivery temporarily suspended: connect to 127.0.0.1[127.0.0.1]:10024: Connection refused)
    ***mylocalmailbox from big pile replaced by this text***

    D7A9B1B2AAE 3057 Wed Mar 5 14:55:45 [email protected]
    (delivery temporarily suspended: connect to 127.0.0.1[127.0.0.1]:10024: Connection refused)
    ***mylocalmailbox from big pile replaced by this text***

    am I just being overwhelmed by the huge pile of messages?
    thanks (sigh)



    thanks in advance (again)
    cdb.
    ps Love ispconfig3!
     
    Last edited: Mar 6, 2014
  12. till

    till Super Moderator Staff Member ISPConfig Developer

    Does 1&1 offers iamo access to this account? If yes, then do the first sync with the imapsync aplication:

    http://imapsync.lamiral.info/

    with imapsync, yu can copy emails from one ima account to anothet without involving postfix, amavisd etc. Its a direct copy from one imap account to another.
     
  13. craig baker

    craig baker Member HowtoForge Supporter

    imapsync

    I'll look into it. it will download all including attachments (not just headers).
    supported by ispconfig3? (guess the answer is no alas).
    and leave all email on the original server?
    and how should I (cleanly) delete the target mailbox before doing this? it actually has the same address as the original ([email protected]) because I'll be porting over her whole domain shortly.

    also poor server is still chewing on the getmail pile (up to 70000 out of 135000 and its been hobbling my email for everyone!) anyway to stop processing them? or just wait till its done, delete target box and start over with imapsync?

    they DO support imap btw.
     
  14. till

    till Super Moderator Staff Member ISPConfig Developer

    It will do a exact 1 to 1 copy of the emails and folders of an account, incl. attachments and most likel even even imap flags like seen or important.

    the mails are in the queue. you can delete the queue with:

    postsuper -d ALL

    but be aware that this will delete all mails in the mailqueue that wait for delivery, not just the ones that you fetched with getmail.

    yes, that might be better.
     
  15. craig baker

    craig baker Member HowtoForge Supporter

    imapsync etc

    alas cant delete everything as that will kill ALL mail. I'll just let it chew its way through.

    the imapsync is supported by 1and1.com (they are also 1und1.de)- but I need the target to be a local ispconfig3 dovecot box - I have the [email protected] email box setup in ispconfig3 I can set it for IMAP. but I'd rather do pop3 if I can (long story).
    can I import everything via this and use pop3 to retrieve when done?
    and can I have the target be localhost?
    [email protected] is a valid internet address as well (ON 1and1.com they currently host both email and website) so I have to be careful to make sure it goes to my LOCAL dovecot mailbox.

    can I easily delete the box in ispconfig3 and will that delete all the associated emails so imapsync can start with a fresh slate?

    happy to pay the $50. I do my part!

    oh - I still may want to use getmail (till the website gets moved over) - so I need to deactivate it while doing imapsync I assume. and then reactivate it after all email is moved over.
     
    Last edited: Mar 6, 2014
  16. till

    till Super Moderator Staff Member ISPConfig Developer

    A mailbox in ispconfig is always a imap and pop3 mailbox, no need for any additional configuration. So just run imapsync with the login details of your 1and1 account as source and your ispconfig mailbox as target.

    You can delete the ispconfig mailbox, just click on the delete button. but you should not do it until the emails in the mailqueue are all delivered.

    yes.
     
  17. craig baker

    craig baker Member HowtoForge Supporter

    Update - crisis over - things resolved and a nifty script!

    Well - getmail works just fine. the things I thought were being stripped turned out to be inline images that outlook was reporting as attachments. they were there just fine in thunderbird etc - just NOT listed as attachments.

    also I found a nifty script to kill the 80,000 emails that I was chewing on - all I had seen was the ability to kill ALL email in the queue!

    to kill mail to a specific user in the queue:

    mailq | tail -n +2 | awk 'BEGIN { RS = "" } / user@mydomain\.com$/ { print $1 }' | tr -d '*!' | postsuper -d -

    And ImapSync worked like a charm many thanks!

    now on to my next project
     

Share This Page