Handling Log Rotation
You may want to gather logs for which rotation is configured. This section explains how to specify the names of such logs for successful gathering.
The typical configuration for log rotation is as follows:
- There is not one log file but an array of log files. The log file names are numbered so that the oldest events are in the log with the highest index. The current log file, which is constantly updated, usually has no index. Other log files are complete, and their contents remain unchanged.
- On schedule or as soon as the current log is full, the oldest log file with the highest index is deleted.
- The other log files are renamed so that their numbering increases by one. For example, eventlog.2 becomes eventlog.3. If the current file had no index, the index is appended. Thus, for example, the current file eventlog is renamed eventlog.1.
- A new log file is created and becomes the current log file. When the eventlog file in the previous example is renamed, a new eventlog is created in its place.
In such situations, you do not want to gather the current log. On the Path to Custom Text Log step of the wizard, explicitly specify one of the backup logs that does not get updated. For example, if rotation is set up for log files log.1 to log.5, supply log.2 as the log file name.
This is required so that gathered events are not duplicated and no events are missed due to the rotation cycle.
If rotation takes place on schedule, set gathering to occur as frequently as the rotation sessions or more frequently.
If rotation happens whenever the current log is at capacity, configure the gathering to take place as often as possible. This way, you will not skip a file due to a sudden increase in the number of events. Performance will not suffer if you gather often, because InTrust does not collect the same events from the same log a second time. In such a configuration, a log file is collected only once in its entirety. All other InTrust gathering sessions simply check whether the file has been replaced.
Gathering with Custom Text Log Data Sources
To use a text log data source in a new gathering policy
- Make sure the data source you need exists and is configured properly.
- Start creating a new gathering policy.
- At the Data Sources step of the New Policy Wizard, click Add. The Add Data Source Wizard opens.
- Select the custom text log data source.
- Finish the wizard.
Examples of Custom Text Log Data Source
Basic Text Log Data Source
In this example, you will create a working data source that handles gathering of the Squid proxy server’s access log. The Squid access log can have three formats. This example uses the access log native format which is the default format.
Before running the New Data Source Wizard, make sure there is sample log file in a location that is accessible from the wizard. The directory /var/log/squid/ where the file access.log resides cannot be accessed directly from InTrust Manager. Copy the file to a local folder on the computer where InTrust is running, for example C:\Logs.
To create a data source for the Squid access log
- Run the New Data Source Wizard and start creating a custom text log data source in Basic mode.
- Specify the path to the log directory (in this case, /var/log/squid) and the file name of the log (in this case, access.log).
- Select Delimited as the formatting type. Specify the location of the sample log file that you will use to verify your settings. The actual log file is not available for preview, so use the file you prepared in advance before launching the wizard.
- Specify which log fields contain data related to the date and time.
Click Test Formatting to make sure that you have supplied proper settings. - Configure the correspondence between the names of fields that InTrust recognizes and the numbers of fields in the text log. The screenshot below shows a sample configuration.
- Provide a name and, optionally, a description for the data source, and finish the wizard.
