I seem to have a small issue with IMAP connections. They get very slow (taking around 40 seconds to download a single message). If I reboot the server then the IMAP connection is fast for a few hours before slowing down again. The server load average is around 0.02 - 0.05 and I can't see any problems in the mail log. Any ideas as to what's happening?
You should consider to install munin, it shows graphically many server parameters and might help to identify the source of your issue.