MTA Pickup for spamc of SpamAssassin and Clamd client

Discussion, support and announcements for third party applications that work with MailEnable.
Post Reply
whiteknight
Posts: 19
Joined: Tue Nov 18, 2003 6:17 am
Location: Singapore
Contact:

MTA Pickup for spamc of SpamAssassin and Clamd client

Post by whiteknight » Sat Dec 03, 2005 11:32 am

Hi everybody,

I recently installed spamassassin and clamav in my unix based server and wanted to hook it up with mailenable on my windows server. Well, I have managed to get it to work. I have to create a pickup program for spamc/spamd and also create a windows client for clamd. For those who are interested you may download it from the following urls.

http://www.whiteknightconsultancy.com/d ... Pickup.zip
http://www.whiteknightconsultancy.com/d ... client.zip

You can see the usage information when you execute the program in command line.

SpamAssassinPickup
Just copy SpamAssassinPickup.exe to your MailEnable bin folder. Then configure the MTA to execute, use SpamAssassinPickup.exe [host] [maxsize].
Example:

Code: Select all

C:\Program Files\MailEnable\bin\SpamAssassinPickup.exe 123.123.1.1 250000
If you want to test SpamAssasssinPickup with your spamc and your spamd host, you just have to include two more parameters in the command line. SpamAssassinPickup.exe [host] [maxsize] [filepath] TEST
Example:

Code: Select all

SpamAssassinPickup.exe 123.123.1.1 250000 spam_message.txt TEST
Your message will be replaced with the output from spamd.
That's all.

clamdclient
Just copy clamdclient.exe to your MailEnable bin folder. Then merge the clam.reg file to your registry. After this, you should be able to see the new item in your list of AntiVirus software in MailEnable. Click "Options" and modify the command line arguments with your IP Address and Port number.
Example:

Code: Select all

"[AGENT]" /host=123.123.1.1 /port=3310 "[FILENAME]"
You can also test it by executing it in the command line, either by specifying a virus file or piping a virus file to it.
Example:

Code: Select all

clamdclient.exe /host=123.123.1.1 /port=3310 /test
clamdclient.exe /host=123.123.1.1 /port=3310 eicar.com
clamdclient.exe /host=123.123.1.1 /port=3310 /modify eicar.com
clamdclient.exe /host=123.123.1.1 /port=3310 < eicar.com
Details of the options are available with /? argument.
That's all

Finally, just to include some observations after setting up this configuration. I tested the mail server by sending ten test spam messages and ten test virus message at one go, and found that it was all processed in under 2 seconds without failure. This configuration also allows me to use 4 threads without issue (it would be an issue if used with ClamWin). I do encourage anyone to consider getting a linux box to eliminate spam and scan viruses in this way.

Regards,
Terrence
White Knight

bbennett
Posts: 17
Joined: Tue Mar 22, 2005 9:20 am

Post by bbennett » Mon Dec 05, 2005 12:10 am

Hi,

Your clamd client works great even at 32 threads. I seem to have some issues though with your spamassassin pickup event leaving messages under the inbound/messages folder. Although this issue seems to occur for all the spamassassin/spamc pickup events I have tested, even a few I had custom made by developers, but yours is the best I have tried so far.

I know setting the threads to 1 is the magic cure all, but we run a high volume mail server and 1 thread will never keep up.

Keep up the good work.

Brian

whiteknight
Posts: 19
Joined: Tue Nov 18, 2003 6:17 am
Location: Singapore
Contact:

Updated SpamAssassinPickup

Post by whiteknight » Tue Dec 06, 2005 6:59 pm

Hi bbenett... thanks for the feedback.
I checked through the codes again after looking at your post. Most of the technique that I am using is very similar to what most other people are doing. Feed the file to spamc and write the output to a temp file then rename as neccessary. Realized that this is not neccessary since I am not writing a script but a program. So I have modified the codes to be more robust. The messages are probably left there after a timeout or when some error occurs. I would have to examine more closely to see what actually goes on. I also realize that email exceeding 1 meg takes too long a time to scan, so better if we set that as the limit. To reduce the possibility of problems, there must be some strategy,

1) Keep original message intact as long as possible
2) Transition to modify message be as short as possible
3) Try not to use temp files

Now with that in mind, the pickup does the following

1) Feed the file to spamc for processing
2) Read on spamc output for data (leaves original file intact)
3) When output appears, overwrite original file.

If error occurs in 1) and 2), procedure will abort, leave original file intact and exit gracefully. Only bad if error occurs while executing step 3, but then all programs will fail if error occurs in the middle of write operation, so I think that should be fine.

You can download the latest file from the same location. I just changed the help output to display version 0.1r1 so you can identify the version. I am trying my new codes on my servers and so far so good (fingers crossed).

PS: I am using 64 threads, tried 60 emails simultaneously and all scan properly. Have not tried more... but when I checked the CPU load, looks like it just peaked for just a few seconds. However, 1 email looks a little mixed up (the senders email ended up in the body) and have 4 orphaned files. All messages got through so I still don't understand why I get the orphaned files. Still this is a more robust version. Will continue to investigate.
White Knight

whiteknight
Posts: 19
Joined: Tue Nov 18, 2003 6:17 am
Location: Singapore
Contact:

Performance of updates

Post by whiteknight » Wed Dec 07, 2005 3:36 pm

Hi again,

I just checked my server. It has processed over 500 emails today, and there are no orphaned files. Perhaps when I tried 60 emails at one go, some IO problems caused files to be orphaned. It looks way better than the first solution. bbenett, please do go ahead with the new version. It is much better.

Regards,
Terrence
White Knight

bbennett
Posts: 17
Joined: Tue Mar 22, 2005 9:20 am

Post by bbennett » Mon Dec 12, 2005 4:09 am

Hi Terrence,

I will try the spamassasin pickup again and let you know how it goes. Your improvements look good. Our mail server probably receives over 500 messages every 10-20 minutes so it will put it to the test.

Your clamd client is still working awesome with no problems and little to no CPU requirements. Perfect for high volume mail servers.

Thanks again,
Brian

sergio
Posts: 67
Joined: Thu Mar 31, 2005 10:20 pm

Post by sergio » Tue Dec 13, 2005 2:42 am

Maybe a stupid question,

The clamdclient update must be manual right ?

Thanks

whiteknight
Posts: 19
Joined: Tue Nov 18, 2003 6:17 am
Location: Singapore
Contact:

Clamd Client

Post by whiteknight » Tue Dec 13, 2005 9:39 am

No worries!

No question is stupid unless it is not meant to be a question.

The clamdclient connects to clamd for scanning services, the client itself does not do the scanning and thus does not require any updating. Clamd currently runs on unix platforms only (as far as I know) and can be updated by using freshclam on a regular basis. I don't know if clamd can work on CygWin so I cannot tell you if it can work on Windows platform or its performance in the same. If anyone have tried clamd in CygWin please advice.
White Knight

bbennett
Posts: 17
Joined: Tue Mar 22, 2005 9:20 am

Post by bbennett » Tue Dec 13, 2005 12:28 pm

We use Clamd and Spamassassin via Cygwin under Windows 2003. No UNIX needed.

Whiteknight - I tried your updated pickup event, but I still noticed some messages left in the inbound messages folder. Still one of the best I have tried though.

Post Reply