Chat now with support
Chat with Support

Foglight for Infrastructure 6.0.0 - User Guide

Using Foglight for Infrastructure Monitoring log files with Foglight Log Monitor Monitoring IBM PowerVM environments
Before you begin Managing PowerVM HMC agents Monitoring your PowerVM environment
Advanced system configuration and troubleshooting Reference
Foglight for Infrastructure views Foglight Log Monitor views Rules Metrics
Appendix: Building regular expressions in Foglight

Configuring File Log Monitor agent properties

The File Log Monitor Agent collects information from selected text-based log files. A log file consists of one or more records; a record can span multiple lines, depending on the format of the log files.

The agent includes a set of properties that you can use to define the location, name, and structure of the log files it monitors. It includes the following groups of agent properties:

•
Monitored Hosts
•
Log Files
•
File Formats
•
Record Transformations
•
Data Collection Scheduler

For a configuration example, see FileLogMonitor configuration example.

Monitored Hosts

The Monitored Host properties specify the hosts whose log files you want to monitor with this agent.

•
Hosts: A list specifying the hosts monitored by the agent instance. Typically you want a cloned list that is associated with a specific agent instance. Each entry in the list includes the following columns:
•
Host: The name of the monitored host or its IP address.
•
Host name override: The host name under which this host’s data is stored in the data model. This property is optional.
•
Host Type: Windows or Unix. This property determines how the agent connects to the host: using SSH (Unix hosts), or using WMI or WinRM (Windows hosts).
•
SSH Port: The port number used for secure connections, if applicable. For Unix and Linux hosts, this value is typically set to 22. For Windows hosts, this is not applicable, and -1 should be specified (meaning not applicable). This property is optional.
•
Operation Timeout: The maximum amount of time in seconds given to the agent for each phase of a collection attempt. This includes uploading the native executable, scanning for log entries, and retrieving log content.
•
Collect System ID: This property indicates to the agent whether or not to collect a unique system ID from this system. This is not desirable when monitoring Hyper-V systems, as some Hyper-V systems use the same ID for multiple systems, preventing them from being unique.
•
Remote Collector Executable: The name of the agent native executable on the remote monitored host. This property is optional. If not specified, a random name is used. Configure this property only if you need to set a specific name for the executable so that you can write a sudo rule for it, or to have it uploaded to a non-default directory. In that case, provide a complete a full path name along with the file name.
 
* 
TIP: By default, the executable is created on the monitored host in the %TEMP% directory (Windows) or /tmp (Unix).
•
Secure Launcher: The name and path to the sudo that enables the agent to launch on Unix and Linux machines, for example: /usr/bin/sudo. This property is optional.

Log Files

The Log Files properties allow you to specify the monitored log files on each host the agent instance connects to, and the type of log records that you want to scan.

•
Log Files: A list specifying the log files monitored by this agent. If the list is shared between agent instances, or if the agent instance is configured to connect to multiple hosts, the log file locations specified in this list are checked on every host the agent connects to. This is useful in situations when you want to scan a standard log file, for example, /var/log/messages, across multiple hosts. To do that, create one agent instance with its own Hosts list, (see Monitored Hosts), and a single row in this list.
Each entry in the list includes the following columns:
•
Directory: The directory containing the log files that you want to monitor.
•
Filename Pattern: A regular expression that specifies which log files to monitor.
 
* 
TIP: The agent supports PCRE (Perl Compatible Regular Expressions). For details about the PCRE syntax, visit http://perldoc.perl.org/perlre.html.
•
File Format Name: The name of the file format the log file uses. File format definitions are specified in the File Formats properties. The value you provide in this column must match an existing file format.
•
Patterns:
•
RegEx Match Pattern: A regular expression that the agent uses to look for specific text in the monitored log files.
* 
TIP: The agent supports PCRE (Perl Compatible Regular Expressions). For details about the PCRE syntax, visit http://perldoc.perl.org/perlre.html.
 
•
Match Severity: The severity associated with log records that match the specified regular expression, in the monitored log file. There are five available severities that you can choose from: Warning, Critical, Fatal, Debug, and Informational.
* 
NOTE: The Critical severity is assigned to any log record with the Error severity.
•
Tags: One or more comma-separated tags that you want to add to log records that match the specified regular expression, in the monitored log file. This property is optional. Tags are useful because they can help you quickly locate records with a desired tag. If set, tags are reported along with any record that matches the specified regular expression. For example, the tag security, auth can be applied to any records that match the regular expression “.*login failed.*”. This allows the agent to identify all records (regardless of file name, host, agent or content) that relate to either security or authorization, and to display them on the Log Monitor dashboard.
•
Exclude Records: Enable/disable to trigger an alarm for agents that are specified in the pattern.
•
RegEx Record Exclude Pattern: The regular expression of the specific text that the agent uses to exclude records from the monitored log files.
 
•
RegEx File Exclude Pattern: The regular expression of the specific filename or directory that the agent uses to exclude records from the monitored log files.
•
Exclude Enable: Sets to true to enable excluding records from the log files, and vice versa.
•
Hours of Backlog Records to Read: Specifies the time duration of scanning the monitored log files.

File Formats

The File Formats properties allow you to specify the format of the log files that you want to monitor.

•
File Formats: A list describing the structure of contents in the monitored log files. Any file that you monitor must have its format specified in this list. In most cases, all agent instances refer to one global list of file formats. The global list is pre-populated to describe some common log file formats.
Each entry in the list includes the following columns:
•
Name: The name of the file format.
•
Max Record Size: The maximum length of a record entry in bytes to use for pattern matching and submission to the Management Server. If a record is larger in size, it is truncated. The pattern is still applied to the entire record when searching for a match. This property is optional, and defaults to 1024 if not specified.
•
New Line Policy: The character or sequence of characters used to signify the end of a text line (EOL) in the file format. The following values are available:
CR: The carriage return character (\r), ASCII code 0x0d. This is common on Mac OS systems up to version 9.
LF: The line feed character (\n), ASCII code 0x0a. This is common on Unix and Unix-like systems (including Mac OS X systems).
CRLF: A carriage return character (\r, ASCII code 0x0d), followed by a line feed (\n, ASCII code 0x0a). This is common on Windows systems.
ANYCRLF: A carriage return character (\r, ASCII code 0x0d), a line feed (\n, ASCII code 0x0a), or both. This is common on mixed platform log files.
ANY: Any Unicode new line sequence, including CR, LF, CRLF, plus the single characters VT (vertical tab, U+000B), FF (form feed, U+000C), NEL (next line, U+0085), LS (line separator, U+2028), and PS (paragraph separator, U+2029).
•
Rollover Policy: Indicates to the agent the way the contents of the log files using this format are rotated when they reach their maximum size.
RECYCLE: The name of the active log file remains the same, while older logs are renamed by appending a ‘1’ to the name and increasing that number each time a new log file is created. For example, your system always writes log records to the same file, abc.log. When that file reaches its maximum size, the system renames it to abc1.log, and creates a new abc.log file for storing new logs. When that abc.log file becomes full, the system renames it to abc.log2 file, and creates a new abc.log file, and so on.
NEW: Each time a new log file is created, the number in the file name of the active log is increased by ‘1’. For example, your system starts writing log records to abc.log. When that file reaches its maximum size, the system creates a new log file, abc1.log, and continues to write log records to that file. When abc1.log file becomes full, the system creates an abc2.log file, and so on.
•
Record Separator RegEx: A regular expression indicating when a log record ends and a new one starts. This property is optional. The default is an empty string which indicates that each record is a single line.
 
* 
TIP: The agent supports PCRE (Perl Compatible Regular Expressions). For details about the PCRE syntax, visit http://perldoc.perl.org/perlre.html
•
Character Set: The character encoding used in the log file using this format. The character set must be installed on the remote host, and available through the iconv utility. The native character set is translated to UTF-8 when a log record is reported. This property is optional. If not specified, it defaults to UTF-8.
•
Maximum Match Count: The maximum number of records the agent can read during one collection interval. If the agent reaches the number of records before the end of the file, it continues to read the remainder of the file during the next collection interval, and so on.
Setting this value to a reasonable number of records (for example, 200) allows you to control the amount of time and resources the agent spends to read monitored logs during a single collection interval, and to prevent bottlenecks during data collection. If you do not want to specify a limit, type -1.
•
Max Processing Time(s): The amount time in seconds the agent can spend on reading all log files in one collection cycle while being connected to the remote host.
This value should be equal to or less than the Operation Timeout value in the Hosts list. For more information, see Monitored Hosts.
Setting this value to a reasonable number of seconds allows you to control the amount of time the agent spends to read monitored logs during a single collection interval, and to prevent bottlenecks during data collection. Once surpassed, the collector stops, and continues from that same point in the next collection cycle.
Related Documents

The document was helpful.

Select Rating

I easily found the information I needed.

Select Rating