Serious problem with IMAP and Large mailboxes in ME 10.25

Discussion forum for Enterprise Edition.
Post Reply
kiamori
Posts: 329
Joined: Wed Nov 04, 2009 1:39 am
Contact:

Serious problem with IMAP and Large mailboxes in ME 10.25

Post by kiamori »

I have a few clients that have very large mailboxes and this has always been somewhat of an issue with performance. Recently however, we've taken on some new clients through an acquisition and they have Smartermnail and since SM has just to many issues to deal with we've migrated these clients from a SM system to ME 10.25

One of the clients has a mailbox with 170,000 emails, yes I know too many. However the server is 64core 1.5TB ram 48 ssd drive in raid 10. Peak activity on the server is 1-2%, drive activity is 20-30%. However, IMAP crashes and we've traced it back to this large mailbox with 170,000 emails using sysinternals tools.

What can be done to resolve this or do I need to move this one client to another mail server?

kiamori
Posts: 329
Joined: Wed Nov 04, 2009 1:39 am
Contact:

Re: Serious problem with IMAP and Large mailboxes in ME 10.25

Post by kiamori »

I did convince the client to archive some older mail into subfolders by year. once the inbox was below 80,000 messages the system performance was no longer an issue.

I still think this needs to be looked at in order to prevent issues like this for occuring in the future. I loaded the same inbox up via a imap migration to two other email systems and didnt have any issues at all with email clients loading the messages so the issue is specific to ME 10.25 and mailbox size in excess of 80,000 emails.

MailEnable-Ian
Site Admin
Posts: 9738
Joined: Mon Mar 22, 2004 4:44 am
Location: Melbourne, Victoria, Australia

Re: Serious problem with IMAP and Large mailboxes in ME 10.25

Post by MailEnable-Ian »

Hi,

The best way forward would be to lodge a support ticket so we can gain access to log files etc. and the server if possible. If you still have the mailbox in this state then we would recommend enabling high level debug logging as well for the IMAP service to provide more information errors when they occur. the following article explains how to enable higher level debug logging for the MailEnable services.

https://www.mailenable.com/kb/content/article.asp?ID=me020585
Regards,

Ian Margarone
MailEnable Support

dcol
Posts: 237
Joined: Fri May 26, 2017 11:25 pm

Re: Serious problem with IMAP and Large mailboxes in ME 10.25

Post by dcol »

Has there ever been a resolution to this? I have been plagued with this issue for years. Clients with large mailboxes run slow as molasses and sometimes the emails just disappear I think because they can't sync. I also have a very powerful server with all SSD drives. System performance is not the issue here. ME performance is. CPU and storage is always below 20%

I should find software that can handle large mailboxes. Any suggestions?

poweredge
Posts: 157
Joined: Sat May 29, 2021 11:16 am

Re: Serious problem with IMAP and Large mailboxes in ME 10.25

Post by poweredge »

dcol wrote:
Wed Aug 25, 2021 9:57 pm
Has there ever been a resolution to this? I have been plagued with this issue for years. Clients with large mailboxes run slow as molasses and sometimes the emails just disappear I think because they can't sync. I also have a very powerful server with all SSD drives. System performance is not the issue here. ME performance is. CPU and storage is always below 20%

I should find software that can handle large mailboxes. Any suggestions?
May I know how large is your mailbox size? and how many emails within that mailbox?

dcol
Posts: 237
Joined: Fri May 26, 2017 11:25 pm

Re: Serious problem with IMAP and Large mailboxes in ME 10.25

Post by dcol »

The largest mailbox disk usage size is 26GB with an inbox size of 35K messages and 21K sent messages, The average of each other mailboxes is between 4K and 6K messages. I already delete all messages older than 180 days and they are ok with that, but not a shorter period.

If ME can't handle this then I need to find software that can. The users are sometimes mobile so they cannot archive messages to their local computers. And I won't switch them to an online service where I lose their business.

poweredge
Posts: 157
Joined: Sat May 29, 2021 11:16 am

Re: Serious problem with IMAP and Large mailboxes in ME 10.25

Post by poweredge »

Btw, I think you mean ME 10.35 instead of 10.25 right?

dcol
Posts: 237
Joined: Fri May 26, 2017 11:25 pm

Re: Serious problem with IMAP and Large mailboxes in ME 10.25

Post by dcol »

Yes 10.35. Now I see 10.36 available, but I may wait on that. I don't see any IMAP issues fixed in the release notes.

dcol
Posts: 237
Joined: Fri May 26, 2017 11:25 pm

Re: Serious problem with IMAP and Large mailboxes in ME 10.25

Post by dcol »

This is still a problem that I need to resolve. Users are asking for more access to old emails. That would put many clients with over 250k emails. Is it time to look at other email server software. Doesn't seem ME can handle this. Any suggestions?

MailEnable-Ian
Site Admin
Posts: 9738
Joined: Mon Mar 22, 2004 4:44 am
Location: Melbourne, Victoria, Australia

Re: Serious problem with IMAP and Large mailboxes in ME 10.25

Post by MailEnable-Ian »

Hi,
This is still a problem that I need to resolve. Users are asking for more access to old emails. That would put many clients with over 250k emails. Is it time to look at other email server software. Doesn't seem ME can handle this. Any suggestions?
When you say users are asking for old emails are you copying them back into the mailbox folders and then the client needs to sync them? What email client are they using on the mobile devices? Can you submit a support request so we can gain access to the server and log files?
Regards,

Ian Margarone
MailEnable Support

dcol
Posts: 237
Joined: Fri May 26, 2017 11:25 pm

Re: Serious problem with IMAP and Large mailboxes in ME 10.25

Post by dcol »

Yes I copy them back then reindex the folder by deleting index.xml
The mobile devices are all Samsung Galaxy 10 using the MS Outlook app.
I am in the process of getting everyone to use the webmail app instead in the phone and PC.

MailEnable-Ian
Site Admin
Posts: 9738
Joined: Mon Mar 22, 2004 4:44 am
Location: Melbourne, Victoria, Australia

Re: Serious problem with IMAP and Large mailboxes in ME 10.25

Post by MailEnable-Ian »

Hi,

Copying them back into the mailbox and deleting the index.xml file will cause the client to resync the folder entirely (if large then it will take a long time). Instead of the deleting the index.xml file you need to create a _change.dty file and place it in the folder. The _change.dty file is an empty Notepad text file saved as _change.dty. When placed in the folder and the client logins it will update the existing index.xml file and add the missing items and sync them without syncing the whole folder and losing flags etc.
Regards,

Ian Margarone
MailEnable Support

rfwilliams777
Posts: 1370
Joined: Thu Nov 11, 2004 5:26 pm
Location: Kingsville, Texas

Re: Serious problem with IMAP and Large mailboxes in ME 10.25

Post by rfwilliams777 »

Reply to all users with the issue:
Every time an email is received, replied, forwarded, etc., the index file is updated--whether from webmail, mobile device, or email client like Outlook. This means that it has to update those changes reflective on the hundreds to thousands of emails that are in that folder (and I assume based on your complaint that this is occurring in the Inbox). I agree and have done the recommendation of archiving emails based on year into subfolders (and not subfolders inside the Inbox). You can even sort them by month. By doing this, it will speed up the Inbox a lot. Because ME has improved the search feature in the webmail and Outlook has a fairly good search capability, finding those emails may not pose too much of a problem. Best of all, your user's inbox will be much small and therefore much quicker to index. Yes, this is a pain and yes it requires work...but the user needs to realize that they cannot save every email including "thank you".
Robert Williams, Owner
www.WilliamsWebSolutions.com
#1 in MailEnable Business-Class Email Hosting - Switch to Williams Web Solutions and we will migrate your accounts to us for FREE!
We can be hired to help you with your Mail Enable server, too!

Post Reply