What are the the log file limits WebLog Expert is capable of handling?

Avatar
erichberg
  • updated
  • Answered
I'm evaluating WebLog Expert using the Lite version currently. I have 80 GB of gziped compressed log files, uncompressed they are over 300 GB across 128 files. The logs are in combinedvhost format. The application stops responding at ~97% when processing the first gz file that contains a log file size near 2 GB. The application occupies nearly 2 GB in memory when it crashes. What are the the log file limits WebLog Expert is capable of handling?
Replies 4
Avatar
0
Michael
There may be a problem with log format as the program shouldn't crash when anayzing log files. Maybe the vhost field in your logs is placed at the beginning before the IP address so the program tries to use it instead of IPs.

You can find more information on the supported log formats in the program help, Log Files > Log Formats topic.
Avatar
0
erichberg
Thanks Michael,

My log file format is in the Apache combinedvhost format:
LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\" %v" combinedvhost

A sample line looks like this:
64.15.218.72 - - [29/Nov/2010:17:00:00 -0500] "GET /pwr/content/07/56/contents.js HTTP/1.0" 200 1980 "http://www.mysite.com/product/index.j..." "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0; .NET CLR 2.0.50727; .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729)" www.mysite.com

There are over 3 million lines per log, each log is a 15 min segment of time for one data center. There are 128 logs I'd like to process. I have been successful using one smaller version of the same file type with only 2048 lines.
Avatar
0
Michael
The log format is OK. However, there may be problems with analyzing log files of more than one million lines per hour with the default settings, while the program shouldn't crash in any case but should show instructions on how it is possible to fix the issue.

What version of the program do you use? If you tried the WebLog Expert Lite only I recommend you to try the trial version too, you can download it from http://www.weblogexpert.com/download.htm
Avatar
0
erichberg
Thanks Michael,

That worked for a one hour chunk (4 logs over 7m+ lines per log), now to try a whole days worth ::cross fingers::

Customer support portal powered by UserEcho