MailEnable & AWStats

Discussion, support and announcements for third party applications that work with MailEnable.
Post Reply
hallik
Posts: 77
Joined: Thu Jun 10, 2004 12:45 am

Post by hallik » Fri May 20, 2005 11:41 pm

I got this error when processing a 1GB log file with MELOGConvert:

Code: Select all

Index was outside the bounds of the array.
THe file went from the 1GB input to a 1MB output file and that was the last line in the file. Can anyone explain what this means? I am going to try with a smaller file this time to see if that helps.

<edit> I got May 1st through May 10th into the awstats. I can't get the 11th thru the 20th in. The 146MB file gets converted to 400KB file, and every file it creates says the index was outside the bounds of the array.

It keeps telling me they are already parsed records. But it is data that hasn't been pulled in!

JoshWithrow
Posts: 192
Joined: Tue Mar 22, 2005 1:18 pm

Post by JoshWithrow » Sat May 21, 2005 4:49 am

I'm actually using a file called me2awstats.exe... I haven't had any problems with that. google it.

hallik
Posts: 77
Joined: Thu Jun 10, 2004 12:45 am

Post by hallik » Fri Jun 24, 2005 7:47 am

I am totally stumped. someone please help. This is what is happening. I do it over and over and it's different everytime. First time it got about 1/2 in and the other 1/2 was corrupt. I deleted the data files and tried again. And got this:


Phase 1 : First bypass old records, searching new record...
Searching new records from beginning of log file...
Phase 2 : Now process new records (Flush history on disk after 20000 hosts)...
Jumped lines in file: 0
Parsed lines in file: 370500
Found 0 dropped records,

Found 368984 corrupted records,
Found 0 old records,
Found 1516 new qualified records.
I have daily logging turned on and all boxes checked.
date time c-ip Mail Enable Account s-ip s-port cs-method cs-uri-stem cs-uri-query s-computername sc-bytes cs-bytes time-taken (cs(User-Agent))

Please note there was one box unchecked until yesterday, and I read in this thread make sure to log everything, so I checked it (don't remember which one it is).

Actually everything worked great and imported all of this data days ago, but because we had to start over, the data was deleted and now I am stuck here.

Here is my conf file
LogFile="e:\www\stats\cgi-bin\MELog2AWStats.exe e:\www\stats\cgi-bin\mail.log | "
LogType=M
LogFormat = "%time2 %email %email_r %host %host_r %method %url %code %bytesd"
LogSeparator = " "
DNSLookup = 0
DirCgi = "../cgi-bin"
DirIcons = "../icon"
AllowToUpdateStatsFromBrowser = 0
UseFramesWhenCGI = 1
ShowFlagLinks = "en fr de it nl es"
DirData = "E:\www\stats\data"
SiteDomain = "domain.com"
HostAliases = "ALL MY DOMAINS LISTED HERE"
ValidSMTPCodes="250"
ShowMenu=1
ShowMonthStats=HB
ShowDaysOfMonthStats=HB
ShowDaysOfWeekStats=HB
ShowHoursStats=HB
ShowDomainsStats=0
ShowHostsStats=HBL
ShowAuthenticatedUsers=0
ShowRobotsStats=0
ShowEMailSenders=HBL
ShowEMailReceivers=HBL
ShowSessionsStats=0
ShowPagesStats=0
ShowFileTypesStats=0
ShowFileSizesStats=0
ShowBrowsersStats=0
ShowOSStats=0
ShowOriginStats=0
ShowKeyphrasesStats=0
ShowKeywordsStats=0
ShowMiscStats=0
ShowHTTPErrorsStats=0
ShowSMTPErrorsStats=1
ArchiveLogRecords=0
PurgeLogFile=0
mail.log is all log files compiled into one log file for ease of mass importing. I did it one at a time to make sure this wasn't an issue and still got more than 1/2 corrupted records on almost all files. Again, I don't know why it isn't working now, but it's very random that it will work great.

Help please!!!

<edit> This is what the showcorrupted switch shows me

Code: Select all

Corrupted record line 251045 (record format does not match LogFormat parameter): 2005-06-23 23:58:36  uifljwu@domain1.com   domain1 SMTP - 550 35
It looks like it is omitting the receiver email and receiver domain. I have no idea why it would start doing that.

<edit2>
Here's a good line that converts properly:

Code: Select all

2005-05-25 00:00:37 192.168.1.100 SMTP-IN  192.168.1.100 1292 MAIL MAIL+FROM:<adam@domain.com> 250+Requested+mail+action+okay,+completed MAIL 43 36  
2005-05-25 00:00:37 192.168.1.100 SMTP-IN mydomain.com 192.168.1.100 1292 RCPT RCPT+TO:<smith@mydomain.com> 550+Requested+action+not+taken:+mailbox+unavailable+or+not+local MAIL 66 30  
And one that is incorrect

Code: Select all

2005-05-25 00:00:06 192.168.1.100 SMTP-IN  192.168.1.100 1248 MAIL MAIL+FROM:+<rwarmuth@anotherdomain.com> 250+Requested+mail+action+okay,+completed MAIL 43 39  
2005-05-25 00:00:06 192.168.1.100 SMTP-IN  192.168.1.100 1248 QUIT QUIT 221+Service+closing+transmission+channel MAIL 42 6 
It looks as though the second line in the incorrect one is displaying the internal IP address instead of the domain name. Why could this be happening?

ShaneBaldacchino
Posts: 3
Joined: Sat Sep 17, 2005 4:26 am
Location: Melbourne

Help - Maybe a Sample Conf File.

Post by ShaneBaldacchino » Sat Sep 17, 2005 4:36 am

Hi Guys,

Well I really haven't got very far here. No matter what I try I am getting the following from Awstats 6.4


C:\Websites\STATSJ~1\cgi-bin>awstats.pl config=pop3.japaneseusedcars.com
Update for config "C:\Websites\STATSJ~1\cgi-bin/awstats.pop3.japaneseusedcars.co
m.conf"
With data in log file "c:/websites/statsjapaneseusedcarscom/cgi-bin/tools/melog2awstats.exe C:/Progra~1/MailEn~1/Logging/SMTP/ex05081801.log"...
Error: Couldn't open server log file "c:/websites/statsjapaneseusedcarscom/cgi-b
in/tools/melog2awstats.exe C:/Progra~1/MailEn~1/Logging/SMTP/ex05081801.log" : I
nvalid argument
Setup ('C:\Websites\STATSJ~1\cgi-bin/awstats.pop3.japaneseusedcars.com.conf' fil
e, web server or permissions) may be wrong.
Check config file, permissions and AWStats documentation (in 'docs' directory).


I am 100% the paths are correct and I have tried numerous variations.

If possible could some one please reply back or even better email me (mailto:shane@baldacchino.net) a sample conf file for Awstats 6.

I am not sure exactly what the invalid argument is as if I run just melog2awstats.exe (path to log file) it works.

C:\Websites\STATSJ~1\cgi-bin\tools>melog2awstats.exe C:/Progra~1/MailEn~1/Logging/SMTP/ex05081801.log
2005-08-18 01:07:19 ipsentrymailloop@japaneseusedcars.com ipsentrymailloop@japan
eseusedcars.com japaneseusedcars.com japaneseusedcars.com SMTP - 250 693
2005-08-18 01:08:18 Lsmuusy@altern.org rees@japaneseusedcars.com altern.org japa
neseusedcars.com SMTP - 250 2158
2005-08-18 01:23:04 www-data@mail.nectarine.info rees@japaneseusedcars.com mail.
nectarine.info japaneseusedcars.com SMTP - 250 4902
2005-08-18 01:28:04 ipsentrymailloop@japaneseusedcars.com ipsentrymailloop@japan
eseusedcars.com japaneseusedcars.com japaneseusedcars.com SMTP - 250 693
2005-08-18 01:39:55 georgeyusuf43@hotmail.com rees@japaneseusedcars.com hotmail.
com japaneseusedcars.com SMTP - 250 3879
2005-08-18 01:48:43 ipsentrymailloop@japaneseusedcars.com ipsentrymailloop@japan
eseusedcars.com japaneseusedcars.com japaneseusedcars.com SMTP - 250 693

Any help is appreciated.
Thanks
Shane Baldacchino
MCSE, MCSA, MCP, A+, CCNA

ShaneBaldacchino
Posts: 3
Joined: Sat Sep 17, 2005 4:26 am
Location: Melbourne

Issue Resolve - New problem now

Post by ShaneBaldacchino » Sat Sep 17, 2005 5:02 am

Okay I solved the top issue, silly me forgot the pipe. Now though I am trying to parse a months worth of logs, I see previous posts about this but why cant I use something like

LogFile="c:/websites/statsjapaneseusedcarscom/cgi-bin/tools/melog2awstats.exe cat-dir.pl C:/Progra~1/MailEn~1/Logging/SMTP/ |"


cat-dir essentially parses all files in the one folder to Awstats.

Any clues how I can easily parse all 800 log files I have?[/i]
MCSE, MCSA, MCP, A+, CCNA

ShaneBaldacchino
Posts: 3
Joined: Sat Sep 17, 2005 4:26 am
Location: Melbourne

Issue resolved.

Post by ShaneBaldacchino » Sat Sep 17, 2005 5:12 am

Okay getting good at solving my own issues, just created a big batch file via FSO.


Obviously my log file format is wrong as I am getting everything as corrupt records. Will post back if I can solve it.
MCSE, MCSA, MCP, A+, CCNA

AbsolutelyFreeWeb
Posts: 88
Joined: Sun Nov 21, 2004 11:44 am

Without corrupted records

Post by AbsolutelyFreeWeb » Fri Jan 13, 2006 5:22 am

Without corrupted records:

Download from
http://www.finexe.com/download/AWStats4ME.zip
Cheap domains com/net/org/info $6.99 including dns manager/forwarding
Quality Hosting from $1.99/mo
Hire IT staff When quality matters

nwelshans
Posts: 50
Joined: Wed Aug 17, 2005 3:27 am

Post by nwelshans » Mon Feb 20, 2006 10:04 pm

anyone know how to get Awstats to dump its processing off to a html file so I dont have to run the perl script from a web page. In the past there have been exploits so Im thinking this might not be a good idea. I am wanting to have a scheduled task that runs the predefined script and dumps an html file of the output to a specified directory. Any help would be greatly appreciated.

AbsolutelyFreeWeb
Posts: 88
Joined: Sun Nov 21, 2004 11:44 am

Post by AbsolutelyFreeWeb » Wed Feb 22, 2006 12:30 pm

It's all described in this thread I believe, if you read from beginning. If you like it to be installed for you, let me know.
Cheap domains com/net/org/info $6.99 including dns manager/forwarding
Quality Hosting from $1.99/mo
Hire IT staff When quality matters

Post Reply