Basic Text Log Data Source
In Basic mode, specify the following:
- Path to the text log file and name of the log file. You can use wildcards or regular expressions.
|
|
Note: If you use regular expressions, you can click the button next to the path and file name text boxes to insert special regular expression characters. Syntax is not checked, so be careful to follow rules while composing regular expressions. Thus, if you need an expression for "one or two digits" and you use the predefined special character [:digit:], an example of correct usage is C:\Logs\log[[:digit:]]{1;2} not C:\Logs\log[:digit:]{1;2}. In situations like this, remember to use the provided button to insert brackets or type them manually. If you want to gather without agents, specify the path using the %COMPUTER_NAME% variable and a share name. You can supply the name of a special Windows share (such as \\%COMPUTER_NAME%\C$\Logs) or a regular native Windows or SMB share (such as \\%COMPUTER_NAME%\logs). However, if you want to gather text logs from an SMB share on a Unix host without an agent, make sure that this host is a member of an InTrust site in the Microsoft Windows Environment container. The gathering job must be configured for this site, so you also need to use a Windows-specific gathering policy. InTrust currently supports gathering from network shares only in Microsoft Windows Environment sites; this workaround makes InTrust aware of the share even though the processed computer is not actually running Windows. |
- Log file encoding.
- Whether the log uses delimiter characters or fixed width for formatting.
- Which row to start parsing at.
- Path to a sample log file, which you can use to test settings.
- Additional format-specific settings.
- Format for date and time information.
|
|
Caution: If you select to treat event times as local time, and you collect events for a period during which time was set back due to daylight saving time adjustments, the event times will be incorrect after the adjustment for the duration of the difference. |
- Mapping of original log fields to InTrust-stored fields.
Advanced Text Log Data Source
In Advanced mode, specify the following:
- Path to the text log file and name of the log file. You can use wildcards or regular expressions.
|
|
Note: If you use regular expressions, you can click the button next to the path and file name text boxes to insert special regular expression characters. Syntax is not checked, so be careful to follow rules while composing regular expressions. Thus, if you need an expression for "one or two digits" and you use the predefined special character [:digit:], an example of correct usage is C:\Logs\log[[:digit:]]{1;2} not C:\Logs\log[:digit:]{1;2}. In situations like this, remember to use the provided button to insert brackets or type them manually. If you want to gather without agents, specify the path using the %COMPUTER_NAME% variable and a share name. You can supply the name of a special Windows share (such as \\%COMPUTER_NAME%\C$\Logs) or a regular Windows or SMB share (such as \\%COMPUTER_NAME%\logs). However, if you want to gather text logs from an SMB share on a Unix host without an agent, make sure that this host is a member of an InTrust site in the Microsoft Windows Environment container. The gathering job must be configured for this site, so you also need to use a Windows-specific gathering policy. InTrust currently supports gathering from network shares only in Microsoft Windows Environment sites; this workaround makes InTrust aware of the share even though the processed computer is not actually running Windows. |
- Regular expressions that match portions of data from the log.
- Log file encoding.
If the log is a mixture of two or more log entry conventions, you must provide two or more regular expressions. Each expression must match only one type of string format (usually a single line) in the log. Be careful not to supply expressions that can match strings with different formats.
If you cannot completely eliminate the possibility of several formats matching one expression, use an appropriate evaluation order for expressions. If an expression matches a string, that string is not parsed again. You set the order using the arrow buttons on the Regular Expressions step of the wizard.
For the entire data source, you can select whether to skip log captions, if any.
For each string format you want to match with a regular expression, specify the following:
- The expression itself.
- Whether to treat the matched string as useful data or as comments
If the string contains data, it is processed for storing in the data store. If it contains comments, it is parsed, but the result does not go into the data store.
|
|
Note: Comments are important for identifying the portions of the log that have no meaning for auditing. Therefore, data and comment regular expressions work like including and excluding filters, respectively. A good set of regular expressions defines both data patterns and comment patterns for a complete definition of the log structure. |
- Date and time formatting conventions (only for data regular expressions). For more information about date and time data mapping, see the Understanding Fields topic.
|
|
Caution: If you select to treat event times as local time, and you collect events for a period during which time was set back due to daylight saving time adjustments, the event times will be incorrect after the adjustment for the duration of the difference. |
- Field mapping (only for data regular expressions). For more information about log data mapping, see the Understanding Fields topic.
Raw Text Log Data Source
In Raw mode, edit a predefined script using a code editor.
Understanding Fields
In both Basic and Advanced mode, the wizard provides controls for translating log data into the storage format that InTrust uses. Regular expressions break up log data into strings, which are mapped to particular event fields that InTrust stores.
Date and Time Fields
Date and time are handled differently from other fields due to their specifics.
Both data source creation modes provide two controls for date and time formatting configuration: the Log fields text box and the Date/Time format combo box.
In the Log fields text box, specify numbers of fields that contain date and time information. Enclose the numbers in angle brackets and use appropriate separators.
These log fields result from the matches that the regular expression returns. A field is made up of data matched by a regular expression fragment enclosed in a pair of parentheses.
Supply field numbers so that they result in properly ordered information after InTrust parses the log. In the Date/Time format text box, supply special formatting characters (such as H, Y, m, s, d) and compose appropriate strings with them. These strings tell InTrust how to interpret the text so that the result is date and time information. For more information about using the special characters, see Pattern Letters for Date and Time Designation.
For example, suppose the log contains date and time information in the first five fields. The first, second, third and fourth field each contains fragments of the date, and the fifth contains the time. If you specify “<5> <1> <2> <3> <4>” in the Log fields text box, a correct entry for the Date/Time format text box might be “HH:mm:ss EEE MMM dd yyyy”. When this is used to process an actual event, InTrust stores something like “15:13:30 Mon Apr 22 2005” as a result.
If date and time information in the log is incomplete (for example, it does not specify the year) and the log file name does not complement it, InTrust’s gathering engine may order data from the log incorrectly. To avoid such problems, specify missing information explicitly in the Log fields text box and supply respective pattern letters in the Date/Time format text box below.
For example, if you know that events in the log occurred on April 3 2003, you can configure the date and time fields as shown in the following screenshot:
|
|
Caution: If you select to treat event times as local time, and you collect events for a period during which time was set back due to daylight saving time adjustments, the event times will be incorrect after the adjustment for the duration of the difference. |
Other Fields
For mapping all other log data, use the controls shown in the following screenshot:
Configure the correspondence between the fields InTrust stores and strings matched by the regular expression. Specify the name of the InTrust field and a corresponding value for the field.
For InTrust fields, you can use only predefined names. However, you can supply as many insertion strings as you like, such as “Insertion String #1” or “Insertion String #15”. Edit the insertion string numbers manually.
For values, you can specify field numbers (as parsed by the regular expression), log file name expressions and literal constants.
Field numbers are enclosed in brackets. Log file name expressions put the name of the current log in InTrust fields. Literal constants are useful when you want the same string to always appear in particular InTrust fields.
The following rules apply to log file name expressions:
- Use regular expressions rather than wildcards when specifying the log file name and path. By default, wildcard notation is used.
- Use parentheses to signify parts of file name and file path strings that are matched. For example, from the file name expression (r2).log.(3) two strings are returned: r2 and 3. For actual parentheses, use \( and \).
- <LogName.0> returns the full name of the log file.
- If a number other than 0 is specified with LogName, that number stands for the number of the string match in the log file name expression you specified. So, in the previous example, <LogName.2> returns 3.
- <LogPath.0> returns the full path to the log file.
- If a number other than 0 is specified with LogPath, that number stands for the number of the string match in the log path expression you specified.