How to synchronize Outlook sent emails to the server automatically?

Discussion in 'General' started by Dy-2024, May 22, 2024.

  1. ztk.me

    ztk.me Well-Known Member HowtoForge Supporter

    a wild guess: non tech savy customers og his' who get/want it their way :)
    or just to comply and make sure things are saved as per requirement. Actually does not really matter why.
    Sure enough there are reasons.
     
  2. till

    till Super Moderator Staff Member ISPConfig Developer

    If there is a tool (or, in this case, protocol) to achieve exactly what is requested here (How to synchronize Outlook sent emails to the server automatically), why don't you use it then if it's supported by your mail client? Insisting on using a protocol that is not capable of achieving this makes not much sense. And the capabilities of the pop3 protocol are unlikely to change, no matter if he insists on that or not
     
  3. ztk.me

    ztk.me Well-Known Member HowtoForge Supporter

    Reality is, some customers pay good money and have their strong opinion when it comes to changes.
    Now he was just wondering how other providers achieve this.
    Sure enough, disabling pop3 is the better solution - no doubt.

    But it is not me, or in fact anyone, to make the decision.
    The point against pop3 is clear enough, though depending on needs and stuff, old things are kept alive ;)
     
  4. till

    till Super Moderator Staff Member ISPConfig Developer

    Regarding customers, I know this of course. But insisting of doing the impossible makes no sense as there is no way to achieve this with pop3 protocol only. So his options are to use IMAP or not sync sent folder. Ok, there is another option to write your own email client and web client to sync things through a different channel, but I guess this option is out of the question.
     
  5. ztk.me

    ztk.me Well-Known Member HowtoForge Supporter

    Not sure why you want to write your own mail client when the server is captureing the mails and place them in a folder which can be remapped using sieve on webmail.
    Not saying it is an easy task either but the capture and move parts is imho not too hard todo.

    One can use incoming/outgoing custom filters, heck even a stoopid cron like this
    Code:
    import os
    import shutil
    from email.parser import BytesParser
    from email.policy import default
    
    # Dovecot Local Maildir Path
    MAILDIR_PATH = "/home/user/Maildir"
    
    def move_mail_to_folder(mail_path, folder):
        destination_folder = os.path.join(MAILDIR_PATH, folder)
        if not os.path.exists(destination_folder):
            os.makedirs(destination_folder)
        shutil.move(mail_path, os.path.join(destination_folder, os.path.basename(mail_path)))
    
    def process_mails():
        for subdir in ["new", "cur"]:
            mail_folder = os.path.join(MAILDIR_PATH, subdir)
            for root, dirs, files in os.walk(mail_folder):
                for file in files:
                    if file.endswith(".eml"):
                        mail_path = os.path.join(root, file)
                        with open(mail_path, "rb") as f:
                            msg = BytesParser(policy=default).parse(f)
                            # Check if mail is sent by the user (adjust the condition as needed)
                            if "[email protected]" in msg["From"]:
                                move_mail_to_folder(mail_path, "Sent")
                            else:
                                move_mail_to_folder(mail_path, "INBOX")
    
    if __name__ == "__main__":
        process_mails()
    
    which unlikely will work but to get a rough idea. it does not check if the mail is already there and comparing by the internal message-id would be better.

    Anyway, imho enuf said =) It's not an easy task and for a one person company who does other things, likely unmaintainable. Disable pop3 is also my conclusion. But saying it is not possible or one would need to write his own clients, I disagree.
     
  6. till

    till Super Moderator Staff Member ISPConfig Developer

    The server is not capturing emails when using external services like hotmail as the thread starter requested. When using pop with Outlook, the sent email is also not stored on the server at all, this happens only when using IMAP as I suggested. So remapping folders and moving emails via cron won't work with pop.

    Writing an SMTP stream filter and mapping this in postfix to catch all outgoing messages might work indeed, but only if webmail is installed on the same server and not connected by pop only but also has access to the harddisk e.g. via IMAP, so it is not Hotmail (as the thread starter mentioned this service). And RoundCube uses IMAP by the way, so the thread starter is already using IMAP if he uses RoundCube.

    I guess it helps users more to tell them the truth right away and give them solutions or suggestions they can use instead of claiming everything is possible, which might be by writing your own programs (like mail clients) or your own smtp processing tool to extract emails during the send process. Especially when there is the right tool (or, in this case, protocol) at hand. I'm trying here to guide users in the right direction and not to confuse them with options that aren't really an option for them. or do you really think that a user who is not aware that POP3 lacks write capabilities now starts writing a secure and working SMTP stream filter for Postfix?
     
  7. ztk.me

    ztk.me Well-Known Member HowtoForge Supporter

    My understanding is, he used the providers as example and to test the functionality and make it more clear where we could reproduce this behaviour he wanted to know how it can be done like this.
     
  8. till

    till Super Moderator Staff Member ISPConfig Developer

    Ok, so the conclusion is to connect both clients using the same protocol with write capabilities. RoundCube uses IMAP, so Outlook must use it as well.
     
  9. ztk.me

    ztk.me Well-Known Member HowtoForge Supporter

    The conclusion would be "it's still complicated" but I think we had a difference in reading what is actually requested.
    Sure enough I would not have claimed that bald statement to capture outlook mail sent to yahoo on a personal mx with no connection to it :D
     
  10. till

    till Super Moderator Staff Member ISPConfig Developer

    Yes, we likely had a different understanding of this. Just out of curiosity, there is still one thing that I'm not sure how you want to achieve this, using your setup scenario:

    In your proposed setup, with a custom SMTP stream filter, RoundCube will show an email sent in Outlook desktop client as we catched it during send process in postfix and stored it in the IMAP .Sent folder, which RoundCube uses. But when I send an email using RoundCube webmail, RoundCube will store the email in the IMAP .Sent folder as well, how do you make this email show up in Outlook (which is only connected using pop3 and therefore using a local .pst file on the hard disk of the desktop as the sent folder)?
     
  11. ztk.me

    ztk.me Well-Known Member HowtoForge Supporter

    You gave one of the options on page 1 already, sieve is a mighty thing.

    I had namespace inbox / renaming aka translating folder names / special_use in mind.

    Syncing would still be an issue, like one would have to log on webmail to delete mails for good if one would
    use two folders server side as we can not know if a client deleted an email locally, true.

    Sieve has a lot of capabilities like "duplicate" detecting messages by ID.

    So yes, basically I'd keep the mails on a special folder serverside and do some syncing back and forth.
    Terrible, maybe, better solutions possible, possible. But as a general idea, good enough?
    If a large provider implements such a thing, there might be demand for such things I guess.

    And if you are a large provider, obviously one has more resourced than my quick 5 minutes thoughts about this ;)
     
  12. ztk.me

    ztk.me Well-Known Member HowtoForge Supporter

    But in case the author really wanted to have remote pop3 accounts on his server imap;
    fetchmail ; offlineimap
     
  13. till

    till Super Moderator Staff Member ISPConfig Developer

    This does not answer my question about how you get the email sent from the server back to the desktop client. ;)

    1) You have a desktop client (no matter if Outlook or Thunderbird) connected by pop3 only, as requested by the thread starter. This means this desktop client fetches mails via pop3 and sends emails via smtp. The sent folder of this desktop client is a local folder when using pop3 (so this folder not on the server, any pop3 account you create in outlook uses such a local folder for sent emails if there is no imap connection).
    2) Now you send an email with webmail, this mail is stored on the server.
    3) How do you get this sent email to show up in the sent folder of the desktop client?

    I doubt that this was the question of the thread starter. In my understanding, he wanted synced sent folders using pop3 only and not imap.
     
  14. ztk.me

    ztk.me Well-Known Member HowtoForge Supporter

    I know, my email client ( evolution ) offers this, too.
    But I think we can agree that using a milter on postfix for sending mails is much much easier if we are the MX the client is communicating with.
    Like Junk mail we can copy that mail for "further inspection" into Imap folder to be seen by webmail.
    Also on the receiving end, one could do the same for selected or all accounts.

    As mentioned, syncing mails would still be impossible ( local delete does not trigger remote delete ).
    This is a behaviour commonly known back in the days when people did not use imap and providers like web.de
    had such things implemented, dunno but t-online did this also for a while if I remember correctly.

    One had to login into webmail and delete again.
     
  15. Dy-2024

    Dy-2024 Member

    Thank you very much "till" and "ztk.me" for the great discussion.
    I've tried hard to understand and hope to learn some tricks from your discussions.

    In my area, there are about hundreds of thousands or millions of people using this feature, so maybe we feel that this is a habit, used to this kind of function, most of the time it is for administrative purposes, and secondly, it is also a kind of data security for users, such as local Outlook or system or disk corruption, etc., at this time, they can still query the mail sent in Outlook in the past through Webmail.

    In addition, there is a habit here, due to many reasons, people here may not like to use IMAP, especially their administrators, they don't want to give higher operation authority to users through IMAP, because many users may accidentally misoperate or other reasons resulting in data loss and other reasons.

    Thank you.
     
  16. till

    till Super Moderator Staff Member ISPConfig Developer

    I wonder if you just mix up POP3 and Imap, as every point you make about why to use POP3 i in fact a point of why administrators use IMAP.

    That's a point why admins choose IMAP today. Central email backup, local system corruption does not matter etc.

    Another point for IMAP. Email data is centrally managed, and backups for emails are centrally managed by the administrator as well. If a user loses data on his local PC does not affect emails.
     
  17. Dy-2024

    Dy-2024 Member

    I see what you mean, I'm just talking about the usage habits of different regions, no matter what kind of person they are.
     
  18. till

    till Super Moderator Staff Member ISPConfig Developer

    I don't think that this is about usage habits. Users use what you tell them to use; if you say they must select IMAP when creating their mail account, they will do it the same way as if you tell them they must choose POP3 in email client settings. Besides that, there is no difference from the user side, except that the features you demand are there. So, you require certain features for your mail system, and all the features are available and implemented in the IMAP protocol, and they do not exist for POP3. These cannot even exist for POP3 as you demand a feature that requires a writable back channel, which does not exist for POP3; means a mail client with pop3 can not share sent emails with other pop3 connected clients, see the discussion above between @ztk.me and me.
     
  19. Dy-2024

    Dy-2024 Member

    I see. Thank you.
     

Share This Page