Multi core processing

Avatar
  • updated
  • Completed
Add multi-core support. It's a shame on 8-core server for WebLog Expert analyzing millions of lines of logs for hours using only 1 core! :)
Avatar
Michael
The program already supports multi-core processing in the Standard, Professional and Enterprise editions. There is the "Use multiple processors (if available)" setting in the program options, you should check if it is enabled (it is enabled by default).
Avatar
Jonathan P
Hi! I know that this is an old topic but I'm currently having the same problem.

I'm using a 4-core for the "WebLog Expert" server and the "Use multiple processors (if available)" option is correctly checked.

If I "manually" analyse a single profile or if I use the scheduler for analysing multiple profiles, the Windows process "WLExpert.exe*32" will never use more than 25% (1/4) of the CPU.

If this could help, here is my server configuration :
-Windows Serveur 2008 R2 SP1 x64 Standard
-4-Core
-16 GB RAM

I really want to optimize this process because it's taking me approx. 24 hours to run my "monthly reports" (WebLog Expert schedule working time).

Thanks!
Avatar
Michael
Such problem may occur if most analyzed data is filtered out by filters (usually) or time range settings.

The program reads logs and applies filters using one thread, so it can use only one core for it. If there is some data to analyze, it analyzes it using multiple threads/cores to create reports (Access Statistics, Activity Statistics, etc). However, if most data is skipped, there is no work for other cores so they aren't used.
Avatar
Jonathan P
I don't use filters on any of my profiles.
However, I do use the Time Range settings : Last month + "Create HTML reports for additionnal time ranges" on all of them.
For the data to analyze --> Custom report contents : Select All

In my understanding, there should be a lot of data to analyze, using multiple threads/cores.

I currently have 39 profiles, using a total of 156GB of unzip IIS logs (for march).
I use the scheduler on the 1st of each month to create the "Last month" usage reports for the management.

Is there a work around to use multiples threads/cores?
(Do you need more information about my configuration for investigation purpose?)

Thanks!
Avatar
Michael
I recommend you to create a new profile, point it to some logs, don't use any specific time range settings, analyze it and check if it uses multiple cores.

If in this case multiple cores are used, the issue probably appears because of the profile settings or logs you analyze.

One possibility is that the program needs to analyze logs for the last month only, but it is pointed to a folder with logs for other months too, so it needs to read all the logs, while it uses data from the logs for the last month only. However, in most cases the program reads all the logs once, and then saves time range for each log to the built-in time range cache so in the future it can determine which logs to analyze. If it doesn't work, you should check that the time range cache in Options > Analysis > Log Cache is enabled.

BTW there may be problems with the time range cache because of daylight changes - usually it needs to reanalyze logs after the daylight change as it thinks that the modification time of logs have changed.