Custom bayesian dictionary not working

Discussion forum for Enterprise Edition.
Post Reply
cassius
Posts: 338
Joined: Tue Mar 11, 2003 2:29 pm
Location: Indianapolis, IN

Custom bayesian dictionary not working

Post by cassius »

Originally we were using the default dictionary, which didn't work very well, then we switched to the alternatedict dictionary provided on the mailenable site, which ended up providing too many false positives.

So, I went through the process of making our own dictionary. Created the Spam and NoSpam folders, put a little over 1000 messages in each (appropriately spam or not-spam), then created the dictionary. Changed the dictionary setting in the MMC, and restarted the MTA service.

However, no messages are being placed in users Junk E-mail folders as they come in, they all go to the Inboxes. The previous dictionaries we used successfully put messages in the Junk E-mail folders, but since we made our own it doesn't do that.

I copied the latest dozen or so spam messages we got in the last half hour to a separate folder, and ran MESPAMCMD -v (verify) on that folder with the dictionary we created. About half of them were 90% rating or more (three or four were 100%), and our filter is set to "Mark as Spam" anything 90% or higher.

Before we switched to the new custom dictionary, it worked. Now it won't put anything in the Junk E-mail folders, even though running MESPAMCMD against those messages with the new dictionary verifies that they are getting rated properly, and most of them are 90% rating or more.

I checked the message files (.MAI) themselves, and it has the Deliver-To:Junk header. Nevertheless they all show up in Inbox.

Any ideas?

rfwilliams777
Posts: 1370
Joined: Thu Nov 11, 2004 5:26 pm
Location: Kingsville, Texas

SPAM filter

Post by rfwilliams777 »

I too was using the standard SPAM filtering and didn't see any results (I've also been looking in the SPAM folder on the server to see if any messages have been put there, and there hasn't been any. However, the computers are still receiving SPAM. I tried the tighter restriction and still I do not see any difference. Am I suppose to add stuff to the dictionary or how does the MailEnable create the SPAM dictionary. Furthermore, any suggestions and/or updates for the latest patches would be much appreciated. :D
Robert Williams, Owner
www.WilliamsWebSolutions.com
#1 in MailEnable Business-Class Email Hosting - Switch to Williams Web Solutions and we will migrate your accounts to us for FREE!
We can be hired to help you with your Mail Enable server, too!

mbatalha
Posts: 133
Joined: Thu Feb 06, 2003 5:04 pm

Post by mbatalha »

I too am trying to do this. Do we need to create a Junk folder. I noticed when i enable the filter option and add a rule in then send a bogus message it will create the folder. How do I set it up so it uses the dictionary?

mbatalha
Posts: 133
Joined: Thu Feb 06, 2003 5:04 pm

Post by mbatalha »

i am trying to create the custom dictionary and i run through this code as the instructions say:

MESPAMCMD -c C:\Progra~1\MailEn~1\Dictio~1\NewDic~1\MailEn~1.TAB C:\Progra~1\MailEn~1\Dictio~1\NewDic~1\Spam C:\Progra~1\MailEn~1\Dictio~1\NewDic~1\NoSpam

but it just never seems to finish processing. Ive let it run for 30 mins already.

This is the command for creating a dictionary correct? Has anyone else run into this problem?

ShaneV

Spam Dictionary

Post by ShaneV »

I don't know how long it takes to build a dictionary, at this point my process has been running for over 24 hours. Hopefully it will work correctly.

ShaneV

42 hours and still going

Post by ShaneV »

I was wondering, since I picked a smallish number of files (i.e. ~1200 spam, 5000+ non spam) is 42 hours a typical number of hours of CPU time to build a dictionary?

Is there some way to check the progress, I don't have a problem letting this run for weeks or so, but does it take long to add/remove spam hits from the file? I guess I could run FileMon on the server, but since it's been running at 100% for the last two days I am a little loathe to run anything more.

BTW this is a fairly small server with a small number of users, so I wonder about the scalability of the spam filtering solution in the enterprise version (just wondering).

mbatalha
Posts: 133
Joined: Thu Feb 06, 2003 5:04 pm

Post by mbatalha »

It should be pretty quick. That happened to me too. i thought it was running but it shoudl actually create the file in a matter of a minute. breeak out of it using taskmanager.

This is what i did.

first i created the spam and nospam folder inside the custom folder. Then I created the mailenable.tab file
then

mespamcmd -m [drive letter]:\progra~1\mailen~1\Dictio~1\custom\mailen~1.tab [drive letter]:\progra~1\mailen~1\Dictio~1\custom\spam [drive letter]:\progra~1\mailen~1\Dictio~1\custom\nospam

The m is for merge. For some reason it was hanging because the file wasnt created. the merge works fine. It will merge to a blank file.

[drive letter] is yor actaull drive where your install is loacted. for example C:\

let me know if this works. good luck.

cassius
Posts: 338
Joined: Tue Mar 11, 2003 2:29 pm
Location: Indianapolis, IN

Post by cassius »

Don't know what exactly was wrong, but after going through every setting one by one and changing a few things, it seems to be working now. Still not very accurate at all, but at least it's putting the e-mails it marks as >90% in the users Junk E-mail folders now.

We're having problems merging new messages into the dictionary, but I'll start another thread on that...

ShaneV

ME Spam filter

Post by ShaneV »

Well, I have my computer set to load automatic updates on Sundays at 4 am, so ME Spam building got canceled, which is a good thing. Now the Spam dictionary is printing status messages and running at 10% of cpu instead of 99%.

Guess I hit a bug of some sort.

Best,
Shane

Post Reply