Improving performance of reporting on large data sources (79373)

Date – 03/09
Affected Product & Version – NetVault: Backup
Affected Module & Version – 8.0 and greater
OS Version – n/a
Application Information – n/a

Description:
NetVault: Backup’s report system can be inefficient when generating a report on large datas ources. There is a workaround to restrict the amount of data that will be processed by the report process, improving efficiency and decreasing system resource usage.

Symptoms:
When reporting on a large data source, for example, job logs from a 2GB logs directory, the processing required by the reporting process can consume most and potentially all system resources. nvplgreport will attempt to consume all system memory, and nvlogdaemon will attempt to consume all available CPU resources.

The NetVault: Backup reports system is based upon classes. A class is a 'SQL like' query (though is not SQL compliant). New classes can be created that include restrictions, and existing classes can be modified to include restrictions.
A restriction is more efficient to process than a filter because records that do not meet the restriction will be removed at a earlier stage in the report process. This gives substantial improvement in efficiency and reduced memory usage.

For example, modify the file <NVDIR>/reports/classes/logs to show:
----------
%NICENAME="NetVault Logs"
select * from logs where in logs restrict systime to (now-3da,now)
---------
The log messages returned by the class are restricted to a time period of (now-3da,now) i.e. the last 3 days, the time restrictions are now, today, mi, ho, da, wk. So after making that modification and running a report that uses the ‘NetVault Logs” class, only the last 3 days of logs will be passed to nvplgreport for filtering and processing.

A new class can also be created by adding new file, for example <NVDIR>/reports/classes/logsrestricted
----------
%NICENAME="NetVault Restricted Logs"
select * from logs where in logs restrict systime to (now-1wk,now)
---------
Create or modify existing reports to use the class “NetVault Restricted Logs”.