MailEnable & AWStats

paarlberg · Post by **paarlberg** » Wed Aug 18, 2004 7:55 pm

just worked finally...

jorune · Post by **jorune** » Thu Sep 23, 2004 2:10 pm

I have AWstats on my server and read through this tread - downloaded the converter and cannot find, for the life of me, any instructions on how to set this up. I could use a step by step or at least some general instructions on what needs to be put where, etc. What am I missing?

Thanks for the information.

Mother · Post by **Mother** » Thu Sep 23, 2004 7:43 pm

Its pretty cake,

You need just a couple of things
1. Perl
2. MELogConvert.exe
3. You Mail Conf file. ie (awstats.mailenable.conf)
4. Batch file for sheduled updates(not nesseccary)

If you set perl up corretly and installed awstats, place files 2,3,4 in the cgi-bin where you have awstats.pl located. Edit the sample conf file below to match your server requirements. DO NOT EDIT THE LOG FORMAT!

awstats.mailenable.conf

Code: Select all

LogFile="C:\awstats\wwwroot\cgi-bin\MELog2AWStats.exe C:\Progra~1\MailEn~1\Logging\SMTP\ex%YY-3%MM-3%DD-3.log | "
LogType=M
LogFormat = "%time2 %email %email_r %host %host_r %method %url %code %bytesd"
LogSeparator = " "
DNSLookup = 0
DirCgi = "/cgi-bin"
DirIcons = "/icon"
AllowToUpdateStatsFromBrowser = 1
UseFramesWhenCGI = 1
ShowFlagLinks = "en fr de it nl es"
DirData = "C:\AWStats\data"
SiteDomain = "mail.yourdomain.com"
HostAliases = localhost 127.0.0.1 yourdomain.com
ValidSMTPCodes="250"
ShowMenu=1 
ShowMonthStats=HB 
ShowDaysOfMonthStats=HB 
ShowDaysOfWeekStats=HB 
ShowHoursStats=HB 
ShowDomainsStats=0 
ShowHostsStats=HBL 
ShowAuthenticatedUsers=0 
ShowRobotsStats=0 
ShowEMailSenders=HBL 
ShowEMailReceivers=HBL 
ShowSessionsStats=0 
ShowPagesStats=0 
ShowFileTypesStats=0 
ShowFileSizesStats=0 
ShowBrowsersStats=0 
ShowOSStats=0 
ShowOriginStats=0 
ShowKeyphrasesStats=0 
ShowKeywordsStats=0 
ShowMiscStats=0 
ShowHTTPErrorsStats=0 
ShowSMTPErrorsStats=1 
ArchiveLogRecords=0
PurgeLogFile=0

updatemailstats.bat

Code: Select all

C:\Perl\bin\perl.exe awstats.pl config=mailenable -update

I use this so I dont have to manually update the file.

then point your browser to http://yourdomain.com/"yourstatsvirtual ... mailenable

Thats about the sum of it. If you have any other question I'm sure someone will help push you in the right direction.

jorune · Post by **jorune** » Thu Sep 23, 2004 8:11 pm

Cool thanks! That's a big help. I will let you know if I get it working but it looks pretty simple. Just needed to see what was needed.

Thanks!

jorune · Post by **jorune** » Thu Sep 23, 2004 8:38 pm

Ran the batch and I got this spit back at me...

C:\Program Files\Advanced Communications\Hosting Controller\web\admin\AWStats>C:\Perl\bin\perl.exe a
wstats.pl config=mailenable -update
Update for config "./awstats.mailenable.conf"
With data in log file "C:\PROGRA~1\ADVANC~1\HOSTIN~1\WEB\ADMIN\AWSTATS\MELog2AWStats.exe C:\Progra~1
\MailEn~1\Logging\SMTP\ex040923.log | "...
Phase 1 : First bypass old records, searching new record...
Searching new records from beginning of log file...
Unable to open file C:\Progra~1\MailEn~1\Logging\SMTP\ex040923.log for reading
Error: Command for pipe 'C:\PROGRA~1\ADVANC~1\HOSTIN~1\WEB\ADMIN\AWSTATS\MELog2AWStats.exe C:\Progra
~1\MailEn~1\Logging\SMTP\ex040923.log | ' failed
Setup ('./awstats.mailenable.conf' file, web server or permissions) may be wrong.
See AWStats documentation in 'docs' directory for informations on how to setup awstats.

ex040923.log does not exist, but when I go check the ME\logging\SMTP I see there is a ex04092315.log

Not sure what I did wrong here.

jorune · Post by **jorune** » Thu Sep 23, 2004 8:41 pm

I'm doing hourly logging. Does that have something to do with it?

Edit:

I think it does... the format for the hourly is exyymmddhh.log

Hmmm

Edit again:

Yup. I changed my logging to daily (read back in this thread someone else had same problem) - and it works now. Just a tiny bit of data coming in now so can't really tell what is going on yet. But it looks like it's working!

Thanks!

jorune · Post by **jorune** » Thu Sep 23, 2004 8:50 pm

Question:

My mail server handles about 80 domains. Do I have to make a config file for each domain or is what I have logging all the domains? (based on the instructions above?)

Edit:

Well watching the little data I'm getting so far, I can see it's doing stats for all the domains. That's cool!

jorune · Post by **jorune** » Thu Sep 23, 2004 9:13 pm

Okay sorry I have another question:

Stats for Sender EMail (Top 20) and Receiver EMail (Top 20) are only showing the External field with information. The local field is blank on each.

Anyone know why this is?

Mother · Post by **Mother** » Thu Sep 23, 2004 9:48 pm

Well both of your questions really have just one anwser.

If you append your "HostAliases=" to have something like this >>

Code: Select all

HostAliases = localhost 127.0.0.1 yourdomain.com hostdomain2.com hostdomain3.com hostdomain4.com

You will get a more acurate return to you stats, this corrects the internal/exteral reporting. The only issue being that because you only have the one mail server handling all your domains, the results will be listed altogether. So seperating your mail stats is impossible by my estimaitons. But where there is a will there is a way I spose.

Hope that helps

jorune · Post by **jorune** » Thu Sep 23, 2004 10:04 pm

Code: Select all

HostAliases = localhost 127.0.0.1 yourdomain.com hostdomain2.com hostdomain3.com hostdomain4.com

Yup that worked. I populated the HostAliases with 3 or 4 of the more active domains and it is now populating the LOCAL fields. Interesting.

I ran the batch from the cmd prompt this time and noticed this spit out:

C:\Program Files\Advanced Communications\Hosting Controller\web\admin\AWStats>C:\Perl\bin\perl.exe a
wstats.pl config=mailenable -update
Update for config "./awstats.mailenable.conf"
With data in log file "C:\PROGRA~1\ADVANC~1\HOSTIN~1\WEB\ADMIN\AWSTATS\MELog2AWStats.exe C:\Progra~1
\MailEn~1\Logging\SMTP\ex040923.log | "...
Phase 1 : First bypass old records, searching new record...
Searching new records from beginning of log file...
Phase 2 : Now process new records (Flush history on disk after 20000 hosts)...
Jumped lines in file: 0
Parsed lines in file: 286
Found 0 dropped records,
Found 6 corrupted records,
Found 190 old records,
Found 90 new qualified records.

6 corrupted records? Is this normal?

Mother · Post by **Mother** » Thu Sep 23, 2004 11:05 pm

Yea, I think your safe up to about 50 or so corrupt records. At least thats what I recall from the Awstats online guide.

But then again I cant remmeber what today is

jorune · Post by **jorune** » Thu Sep 23, 2004 11:25 pm

Hey Mother,

You have been a great help! Thanks a lot. I actually noticed one other strangeness.

Hosts (Top 10) shows all the most active domains, right? Well it looks correct, except it's missing one domain that I know is definitely a top 10 and that is my main domain (coastaldatalink.net)! It's conspicuously missing even from the 'full list' - it's simply not there and I know it should be since mail has been coming and going from this domain in the last hours I set the stats up.

I even see Sender EMail (Top 20) and Receiver EMail (Top 20) list email addresses from coastaldatalink.net yet it's not in the Hosts (Top 10).

Any ideas?

I tried checking my HostAliases to ensure everything is right and it is (according to your post).

Mother · Post by **Mother** » Thu Sep 23, 2004 11:57 pm

No problem`mo, glad to help out.

I'm not sure why your domain is not showing up in the list. The only things that come to mind are things like your host file in your windows dir having other information regarding your site. Or you have the "skipHosts=" field in the conf file.

Although there is a small chance its just a simple DNS issue with the manner your looking up vs the manner you host. But this is an assumption because I noticed that you had HC in your path in your previous comments.

jorune · Post by **jorune** » Fri Sep 24, 2004 12:17 am

Mother wrote:No problem`mo, glad to help out.

I'm not sure why your domain is not showing up in the list. The only things that come to mind are things like your host file in your windows dir having other information regarding your site. Or you have the "skipHosts=" field in the conf file.

Just checked hosts file and it's the standard "127.0.0.1 localhost" line.

Don't see a "skipHosts=" in the conf file either.

Although there is a small chance its just a simple DNS issue with the manner your looking up vs the manner you host. But this is an assumption because I noticed that you had HC in your path in your previous comments

.

It's true I do have Hosting Controller installed on this machine and AWStats comes bundled with the installation. I currently have AWSTats 6.0 (build 1.704) and when I say "bundled" I mean HC just stuffs the AWStats in it's own sub directory instead of giving it the standard that you would expect when doing a native install.

When you say DNS issue or lookups vs manner you host, what do you mean?

Here is my DNS zone for coastaldatalink.net - wasn't able to see anything wrong here:

Code: Select all

;
;  Database file coastaldatalink.net.dns for coastaldatalink.net zone.
;      Zone version:  2003120422
;

@                       IN  SOA ns1.coastaldatalink.net.  admin.coastaldatalink.net. (
                        	2003120422   ; serial number
                        	3600         ; refresh
                        	600          ; retry
                        	86400        ; expire
                        	3600       ) ; default TTL

;
;  Zone NS records
;

@                       NS	ns1.coastaldatalink.net.
@                       NS	ns2.coastaldatalink.net.

;
;  Zone records
;

@                       A	69.13.116.140
@                       MX	15	coastaldatalink.net.
@                       MX	10	mail.coastaldatalink.net.
cdl1                    A	69.13.116.128
forum                   A	69.13.116.140
ftp.forum               CNAME	forum.coastaldatalink.net.
www.forum               CNAME	forum.coastaldatalink.net.
ftp                     CNAME	coastaldatalink.net.
mail                    A	69.13.116.140
ns1                     A	69.13.116.129
ns2                     A	69.13.116.130
pop                     A	69.13.116.140
www                     CNAME	coastaldatalink.net.

Don't know if this matters, but my regular conf file for the normal stats looks like this:

Awstats.coastaldatalink.net.conf

Code: Select all

LogFile ="C:\Webspace\cdnet\coastal\coastaldatalink.net\log\W3SVC23240\ex%YY%MM%DD.log"
LogFormat ="%time2 %other %other %other %method %url %query %other %logname %host %other %ua %other %referer %other %code %other %bytesd %other %other"
DNSLookup =0
DirData =Data
DirIcons =Icon
DirCgi = ""
AllowToUpdateStatsFromBrowser = 1
SiteDomain ="coastaldatalink.net"
HostAliases ="www\.coastaldatalink\.net coastaldatalink\.net"
PurgeLogFile = 0
ArchiveLogRecords = 0
KeepBackupOfHistoricFiles = 0
Lang = "en"
DirLang = "./lang"
DefaultFile = "index.html"
SkipHosts = "
SkipDNSLookupFor = "
SkipFiles = "\.css$ \.js$ \.class$ robots\.txt$"
OnlyFiles = ""
NotPageList = "gif jpg jpeg png bmp"
URLWithQuery = 0
WarningMessages = 1
NbOfLinesForCorruptedLog = 50
FirstDayOfWeek = 1
MaxNbOfDomain = 25
MaxNbOfHostsShown = 25
MinHitHost = 1
MaxNbOfLoginShown = 10
MinHitLogin = 1
MaxNbOfRobotShown = 25
MinHitRobot = 1
MaxNbOfPageShown = 25
MinHitFile = 1
MaxNbOfRefererShown = 25
MinHitRefer = 1
MaxNbOfKeywordsShown = 25
MinHitKeyword = 1
SplitSearchString = 0
Expires = 0
ShowHeader=1
ShowMenu=1
ShowMonthDayStats = 1
ShowDaysOfWeekStats = 1
ShowHoursStats = 1
ShowDomainsStats = 1
ShowHostsStats = 1
ShowAuthenticatedUsers = 0
ShowRobotsStats = 1
ShowPagesStats = 1
ShowCompressionStats=0
ShowFileTypesStats = 1
ShowFileSizesStats=0
ShowBrowsersStats = 1
ShowOSStats = 1
ShowOriginStats = 1
ShowKeyphrasesStats = 1
ShowKeywordsStats=1
ShowHTTPErrorsStats = 1
ShowFlagLinks = "en fr de it nl es"
ShowLinksOnUrl = 1
MaxLengthOfURL = 72
DetailedReportsOnNewWindows = 1
HTMLHeadSection = ""
HTMLEndSection = ""
BarWidth = 260
BarHeight = 180
Logo = "awstats_logo1.png"
logolink = "http://awstats.sourceforge.net"
color_Background= "FFFFFF"
color_TableBGTitle="CCCCDD"
color_TableTitle="000000"
color_TableBG="CCCCDD"
color_TableRowTitle="FFFFFF"
color_TableBGRowTitle="ECECEC"
color_TableBorder="ECECEC"
color_text="000000"
color_titletext="000000"
color_weekend="EAEAEA"
color_link="0011BB"
color_hover="605040"
color_u="FF9933"
color_v="F3F300"
color_p="4477DD"
color_h="66F0FF"
color_k="339944"
color_s="8888DD"

And finally here is my awstats.mailenable.conf

Code: Select all

LogFile="C:\PROGRA~1\ADVANC~1\HOSTIN~1\WEB\ADMIN\AWSTATS\MELog2AWStats.exe C:\Progra~1\MailEn~1\Logging\SMTP\ex%YY-3%MM-3%DD-3.log | " 
LogType=M 
LogFormat = "%time2 %email %email_r %host %host_r %method %url %code %bytesd" 
LogSeparator = " " 
DNSLookup = 0 
DirCgi = "" 
DirIcons = icon 
AllowToUpdateStatsFromBrowser = 1 
UseFramesWhenCGI = 1 
ShowFlagLinks = "en fr de it nl es" 
DirData = Data 
SiteDomain = "mail.coastaldatalink.net" 
HostAliases = localhost 127.0.0.1 coastaldatalink.net coastaldatalink.com (I removed the other domains hosted on this site for security reasons but they were listed here with a space between each one)
ValidSMTPCodes="250" 
ShowMenu=1 
ShowMonthStats=HB 
ShowDaysOfMonthStats=HB 
ShowDaysOfWeekStats=HB 
ShowHoursStats=HB 
ShowDomainsStats=0 
ShowHostsStats=HBL 
ShowAuthenticatedUsers=0 
ShowRobotsStats=0 
ShowEMailSenders=HBL 
ShowEMailReceivers=HBL 
ShowSessionsStats=0 
ShowPagesStats=0 
ShowFileTypesStats=0 
ShowFileSizesStats=0 
ShowBrowsersStats=0 
ShowOSStats=0 
ShowOriginStats=0 
ShowKeyphrasesStats=0 
ShowKeywordsStats=0 
ShowMiscStats=0 
ShowHTTPErrorsStats=0 
ShowSMTPErrorsStats=1 
ArchiveLogRecords=0 
PurgeLogFile=0

Mother · Post by **Mother** » Fri Sep 24, 2004 12:59 am

Sorry about that, sometimes I dont make very clear statements..

I was thinking of your network stack when I made the comment

But I noticed in your mail.conf you have DNSLOOKUP = 0
Have you attempted to change this to see what results it yeilds?