URGENT: postoffice connector extreme performance problem

Discussion forum for Enterprise Edition.
Post Reply
martkla
Posts: 9
Joined: Fri Mar 19, 2004 11:04 am

URGENT: postoffice connector extreme performance problem

Post by martkla » Sun Sep 14, 2008 3:55 pm

i have an urgent strange problem:

my server with enterprise 3.53 "hangs" and reacts slowly - websites take extremely long to open, sql server queries timeout, and the remote desktop connection cannot be used, because it even takes minutes for a menu to open.

cpu usage is pretty low, but harddisk usage is extremely high. as soon as i stop the postoffice connector service, performance is up again, and the system works as expected. when i then start the postoffice connector service, performace goes down again after a minute or two.

virus scanning is already disabled - there's no virus scanner installed on that server.

does anyone have an idea how i could get that server back to normal work? this is really urgent, and any held would be highly appreciated.

thanks a lot. martin.

MailEnable-Ben
Posts: 5858
Joined: Fri Jan 16, 2004 6:49 am
Location: Melbourne

Post by MailEnable-Ben » Mon Sep 15, 2008 12:00 am

What is the post office connector doing? Check the debug log to see if there is anything noticeable. Things to search for are many quota checks or mailbox level filters being executed.

Is the post office connector using high CPU in the Task Manager when it is running?
Regards,

Product Services
MailEnable Pty Ltd

To keep track of all ME company updates and version releases you should subscribe to the MailEnable list at http://www.mailenable.com or the RSS feed http://www.mailenable.com/rss.

Allen
Posts: 16
Joined: Thu Jan 22, 2004 12:47 pm
Location: Silver Spring, MD

Post by Allen » Wed Sep 17, 2008 8:20 pm

Hello,

First off, I would like to thank Ben, Ian and the MailEnable crew for helping us through some issues since upgrading to version 3.53. We’ve been using ME for over 5 years and they have always provided great support both in the forums and when working w/ us.

We are experiencing this same issue since upgrading to 3.53. We are seeing the same issues w/ disk I/O sometimes off the scale and usually running high due to the POP Connector. If bad enough, this can then result in a backup of the queues.

Ben spent a lot of time working on this last week and sorry Ben, but we mistakenly thought the disk I/O was due to some Helm tasks that run on the server. With further analysis and testing I’ve ruled this out and have ID’d the issue as the POP Connector.

I’m not seeing anything anomalous in the logs and will continue to try and sort this. Unfortunately this is having an effect on our clients and I hope ME can take a look at this situation and provide some assistance.

Martin, how many POs are you running?

Great product great forums – Thank you and best wishes.
Allen Burnett
Storm Technologies Group, LLC

jdraggi
Posts: 33
Joined: Tue Feb 14, 2006 12:34 am
Location: USA

Post by jdraggi » Wed Sep 17, 2008 9:29 pm

We had something similar going on too (long delays for clients connecting and very high CPU usage). I moved the database off of SQL Express and over to SQL Workgroup and my POP issues disappeared.

Now I just have to wrap up the SMTP delay issue that I just posted and we'll be back on track :)

MailEnable-Ben
Posts: 5858
Joined: Fri Jan 16, 2004 6:49 am
Location: Melbourne

Post by MailEnable-Ben » Wed Sep 17, 2008 11:41 pm

Allen, sorry I cannot agree with your evaluation here as your disk IO was still maxed out after I stopped all the MailEnable services on your server. At the time I was looking the Helm service was definitely the issue as when this was stopped the queues cleared and the D drive disk IO went down by 80%. MailEnable will be affected greatly if another process is running high disk activity as we rely on this quite a bit while writing our various files around the queues and mailbox folders.

Martins issue here maybe similar with the Post Office connector taking up lots of Hard Drive IO which can be attributed to mailbox level filters or quotas. Martin you should run Perfmon and then start disabling/stopping services on your server to see when the Disk IO goes down. If this is not the case then you should check the CPU and see what process is running high if any. If you need you can also turn on high debug logging on the MailEnable Post Office connector by adding the following registry key;

Code: Select all

Windows Registry Editor Version 5.00

[HKEY_LOCAL_MACHINE\SOFTWARE\Mail Enable\Mail Enable\Connectors\SF]

"Debug Log Level"=dword:0000000a
Regards,

Product Services
MailEnable Pty Ltd

To keep track of all ME company updates and version releases you should subscribe to the MailEnable list at http://www.mailenable.com or the RSS feed http://www.mailenable.com/rss.

deepa
Posts: 41
Joined: Tue Mar 04, 2008 8:32 am

Post by deepa » Thu Jan 15, 2009 1:58 pm

we have somewhat a similar situation where in peak hours
we can see high avg disk queue length and current disk queue length.

upon checking disk i/o reads were quite high and meta mesmtpc and mepoc are on the top.
we are using ME pro 3.53

could any one guide me what all stats shld i check further and how can this be optimized.


http://forums.webhostautomation.com/sho ... hp?t=16099

MailEnable-Ben
Posts: 5858
Joined: Fri Jan 16, 2004 6:49 am
Location: Melbourne

Post by MailEnable-Ben » Wed Jan 28, 2009 12:51 am

You are either processing 100s of thousands to millions of messages per day or you have a problem with your configuration. Is it possible you are processing this amount of messages? How many mailboxes are you hosting? If you are processing lots of messages then yes the services can be processing high amounts of hard drive IO this is a bottle neck that can only be improved on by processing less messages, hosting less mailboxes, increasing your Hard Drive subsystem.

You may need to look into turning off content filters if the MTA is doing a lot. But all this is useless information unless you look at your logs and find out what your system is doing.
Regards,

Product Services
MailEnable Pty Ltd

To keep track of all ME company updates and version releases you should subscribe to the MailEnable list at http://www.mailenable.com or the RSS feed http://www.mailenable.com/rss.

deepa
Posts: 41
Joined: Tue Mar 04, 2008 8:32 am

Post by deepa » Sat May 09, 2009 12:27 pm

MailEnable-Ben wrote:You are either processing 100s of thousands to millions of messages per day or you have a problem with your configuration. Is it possible you are processing this amount of messages? How many mailboxes are you hosting? If you are processing lots of messages then yes the services can be processing high amounts of hard drive IO this is a bottle neck that can only be improved on by processing less messages, hosting less mailboxes, increasing your Hard Drive subsystem.

You may need to look into turning off content filters if the MTA is doing a lot. But all this is useless information unless you look at your logs and find out what your system is doing.
Hi Ben,

thanks for updating me though the answer was quite confusing and I really coudlnt grasp what all was written, eventually this server hadnt been much of problem till march however disk i/o used to eb quite high.

since last week we could again notice high disk i/o where MEPOC seems to be doing lot of disk i/o, average disk queue length is upto 24 per second on week days in peak hours which evenetuall starts building up after ME is restarted. (ME pro 3.53)

Kindly let me know step by step things to check also let me know if I can use some utility to generate a report for number of mail boxes and messages being processed per day for you.

rfwilliams777
Posts: 1321
Joined: Thu Nov 11, 2004 5:26 pm
Location: Kingsville, Texas

Post by rfwilliams777 » Thu May 14, 2009 3:52 am

I have several questions:
1) How many messages are you processing in a day on average?
2) How much RAM do you have on the server and what version of OS?
3) How much free disk space and what is your page file set to?
4) How many layers of filters do you have?
5) Are you still running version 3.53?
6) Are all these POP/SMTP connections are any of these IMAP?
7) How many post offices and mail boxes do you have?
Robert Williams, Owner
www.WWSHosting.net
#1 in MailEnable Business-Class Email Hosting - Switch to Williams Web Solutions and get your first 2 months FREE!
We can be hired to help you with your Mail Enable server, too!

delta2
Posts: 42
Joined: Mon Dec 20, 2004 7:25 am

Post by delta2 » Sat May 23, 2009 7:33 am

8) What is the defrag state of the hard drive ?

I had similar symptoms on my server and a defrag worked wonders !
Stuart

Post Reply