Chat now with support
Chat with Support

Foglight for Infrastructure 5.9.2 - User Guide

Using Foglight for Infrastructure Monitoring log files with Foglight Log Monitor Monitoring IBM PowerVM environments
Before you begin Managing PowerVM HMC agents Monitoring your PowerVM environment
Advanced system configuration and troubleshooting Reference
Advanced System Configuration Foglight for Infrastructure views Foglight Log Monitor views Rules Metrics
Appendix: Building regular expressions in Foglight

Agent properties

When an agent connects to the Foglight Management Server, it is provided with a set of properties that it uses to configure its correct running state. For more information about working with agent properties, see Creating agent instances.

The MultiHostProcessMonitorAgent is shipped with default properties that can be modified to suit your system requirements. The properties specific to the MultiHostProcessMonitorAgent are illustrated in the following screenshot.

Figure 13. MultiHostProcessMonitorAgent properties

You can configure the following settings for this agent:

•
Properties:
•
Multiple Hosts Config: The list of hosts to be monitored. Click Edit to modify the following Multiple Host Config properties
•
OS Type: type of the operating system (Unix® or Windows®).
•
Host: host name or IP address.
•
Host name override: host name to be used to store this host’s data in the Foglight data model.
•
SSH Port: SSH port on which the agent connects. Default value = 22. Applicable to the Linux® platform only.
•
Top CPU Processes: number of top CPU processes to be monitored. Default value = 5.
•
Top Memory Processes: number of top memory processes to be monitored. Default value = 5.
•
Top IO Processes: number of top IO processes to be monitored. Default value = 5.
•
Collect Top N processes only: Default value = False. When set to True, the agent collects data for Top CPU Processes, Top Memory Processes, and Top IO Processes only.
•
Aggregate data for all instances of a program: Default value = True. When set to True, the agent collects data from all the instances of a program (for example all Oracle® instances), aggregates the information, and presents it in a unified report.
When set to False, the agent still collects aggregate data, but it also includes details about every process. This could result in a lot of data sent to the Management Server and may have a performance impact.
•
Collect additional metrics for VMWare: Default value = False. When set to True, the agent collects the following data:
•
Host: availablePagingSpace, runQueueLength, contextSwitches
•
CPU: totalHz, percentUserTime
•
Memory: capacity, consumed, pageInRate, pageOutRate, and utilization
•
Collect Top N Process Details: Default value = True. When set to True, the agent collects data for the Top CPU Processes, Top Memory Processes, and Top IO Processes. Details about these top processes are accessible from the Infrastructure Environment dashboard (for example, to see the top CPU processes, in the Monitoring tab, select a host on the Quick view, click the Explore button in the Resource Utilizations view, and click any metric indicator in the CPU area; the top CPU consumers are displayed in the CPU Details dashboard).
•
Collect process metrics: Default value = True. When set to True, the agent collects process metrics.
•
Collect declared processes only: Default value = False. When set to False, the agent collects metrics for all processes that matches the values defined in Process Availability Config. An alarm will be raised when the actual process count is lower than Expected Process Count.
When the Collect declared processes only property is set to true, only processes declared under Process Availability Config will be reported.
•
Collect System ID: Default value = True. This property indicates to the agent whether or not to collect a unique system ID from this system. This is not always desirable when monitoring Hyper-V® systems, as some Hyper-V systems use the same ID for multiple systems and are not unique.
•
Use ping to validate host availability: Default value = False. When set to True, the agent is configured to use ping to detect if the monitored host is unavailable. If the agent fails to make a connection to the monitored host, and this property is set to True, the agent sends a ping command to the host. If the host does not respond, the Host.monitored observation is set to UNAVAILABLE (for more details, see Host availability alerting).
 
* 
NOTE: When the Use ping to validate host availability property is enabled on a UNIX® platform, the sudoer file needs to configured to allow the ICMP process to run with NOPASSWD. For details, see Configuring secure launcher permissions using sudo.
•
Use commands with sudo: Default value = False. Applicable to the Linux® platform only. When set to False, the agent does not use commands that require sudo, and does not collect metrics that require root permissions. For more information about sudo commands that require root access, see Configuring secure launcher permissions using sudo.
•
Path to sudo command: The path to the sudo executable. Applicable to the Linux® platform only.
Process Availability Config: A list of monitored processes and their expected instance counts. The list contains three columns: Process Name, Command Line, and Expected Process Count, and can be edited, as required. The agent compares the number of actual processes with the number of expected processes found in this list. Results are displayed in the Processes > Processes > User Defined Processes (Process Availability Config) view (for details, see User Defined Processes (Process Availability Config)).
 
* 
TIP: This feature replaces the AppMonitor capability provided by the legacy OS cartridge.
The combination of Process Name, Command Line, and Expected Process Count consists of the expression that identifies the query criteria for filtering out the process instances. The following samples demonstrate how to find the values of Process Name and Command Line on Solaris:
* 
TIP: So far, we only support exact string matching for Process Name, while regular expression is supported for Command Line.
•
Solaris: Execute the “/usr/bin/ps -e -o uid,pid,ppid,vsz,rss,time,pcpu,sid,s,user,comm,args” command. Then you will get the following process details.
16801 21353 21351 249104 159096 01:08 0.0 2665 S murex /usr/local/java/jdk1.7.0_51/bin/java /usr/local/java/jdk1.7.0_51/bin/java -Dname=RTZSP_DEALGEN -Djava.library.path=/
In the sample above:
•
Process Name is /usr/local/java/jdk1.7.0_51/bin/java
•
Command Line can be either usr/local/java/jdk1.7.0_51/bin/java -Dname=RTZSP_DEALGEN -Djava.library.path=/ or some keywords (for example, RTZSP_DEALGEN)
•
Data Collection Scheduler
•
Collector Config: defines how quickly the agent collects data. Both Windows and Linux® provide a defaultSchedule configuration. Users can modify, clone, and delete configurations, as necessary.
Configuring secure launcher permissions using sudo

Some UnixAgents can function without root privileges, but certain metrics can only be collected by commands which must be run as root. In order to give these agents the required access, Foglight Agent Manager is configured to launch these agents using a tool such as sudo that allows privilege escalation (without a password).

To this effect, the sudo configuration file (/etc/sudoers) must be configured so that password prompts are not required for a number of executables. The commands requiring elevated privileges differ by platform. The following commands must be configured for this version of Foglight for Infrastructure.

Table 6. Sudo commands that must be configured
Agent Type
Command
Notes

Linux®

/usr/bin/find, /bin/cat

Used to read IO statistics from the /proc filesystem.

/sbin/ethtool or /usr/sbin/ethtool (depending on distribution)

/sbin/mii-tool or /usr/sbin/mii-tool (depending on distribution)

Used to determine the network card bandwidth; ethtool is favoured if it is found.

The following is an example of how to configure the /etc/sudoers file to allow the user foglight to execute Linux® commands without being prompted for a password:

foglight ALL = NOPASSWD: /usr/bin/find, /bin/cat, /sbin/ethtool

In addition, the requiretty flag must not be set in /etc/sudoers for the user, since Foglight for Infrastructure agents use non-interactive shells.

The following is an example of how to unset the requiretty flag for a single user named foglight, so that this user can run sudo commands remotely:

Defaults:foglight !requiretty
 
* 
NOTE: If requiretty flag is set, sudo can run only when the user is logged in to a real tty. When this flag is set, sudo can only be run from a login session and not via other means, such as cron or cgi-bin scripts. This flag is off (unset) by default.
Controlling log usage for sudo access

Using commands with sudo access can result in increased logging. Sudo provides the following levels of logging, each resulting in the capture of a specific type of information:

•
Log all commands that run with sudo.
•
Log only commands that run with sudo and cause errors.
•
Log all command-line output produced by the commands that run with sudo.
•
Log all command-line input provided to the commands that run with sudo.

Depending on the user’s sudo and syslog.conf configuration, sudo use may result in excess logging. To minimize the amount of log messages, ensure that sudo does not make use of the LOG_INPUT or LOG_OUTPUT tags for the commands that the UnixAgent runs. Depending on the existing monitored hosts’ configuration, any lines added to the /etc/sudoers file for Foglight monitoring may have to include NOLOG_OUTPUT or NOLOG_INPUT to override the default configuration. For example, for a user named foglight connecting to a monitored host, the following lines are required:

Linux
foglight ALL = NOLOG_INPUT: ALL, NOLOG_OUTPUT: ALL, NOPASSWD: /usr/bin/find|,
/bin/cat, [/sbin/ethtool|/usr/sbin/ethtool|sbin/mii-tool|/usr/sbin/mii-tool]

The last argument in this syntax depends on the type and location of the tool, ethtool or mii-tool, used to determine the network card bandwidth. If you are unsure which tool your system uses, you can specify all of them:

foglight ALL = NOLOG_INPUT: ALL, NOLOG_OUTPUT: ALL, NOPASSWD: /usr/bin/find|,

/bin/cat, /sbin/ethtool, /usr/sbin/ethtool, /sbin/mii-tool, /usr/sbin/mii-tool

 

Monitoring log files with Foglight Log Monitor

Foglight for Infrastructure relies on the File Log Monitor and Windows Event Log Monitor agents to collect data. These agents collect desired information from selected logs. A log file consists of one or more entries, or log records. Depending on the format of a monitored log file a log record can span multiple lines. The collected information is visualized on the Log Monitor dashboard.

Start by ensuring that Foglight for Infrastructure is installed on the Management Server, and that the agent package is deployed. For installation instructions, see the Foglight for Infrastructure Release Notes.

Next, configure the File Log Monitor and Windows Event Log Monitor agents for data collection. For more information, see Configuring monitoring agents, Configuring agent properties, and Configuring connections to remote Windows platforms.

When your monitoring agent instances are configured and are collecting data, navigate to the Log Monitor dashboard. This dashboard allows you to look at individual log records, and observe their growth rate over the selected time range. For more information, see Investigating log records.

Configuring monitoring agents

Foglight for Infrastructure uses Quest File Log Monitor Agent and Quest Windows Log Monitor Agent instances to collect information from monitored hosts. When Foglight for Infrastructure is installed on the Management Server and the Host Agents package is deployed to a desired FoglightAgent Manager host, you can create these agents and configure them for data collection.

While the Quest File Log Monitor Agent collects information from selected text files, Quest Windows Event Log Monitor Agent collects information from Windows Event Log files. Both agents can look for the text patterns you specify in the monitored logs.

To create an agent instance, activate it, and start its data collection, use the appropriate wizard (Create FileLogMonitorAgent or Create WindowsEventLogMonitorAgent), accessible from the Log Monitor dashboard.

To create a monitoring agent instance, activate it, and start its data collection:
1
Log in to the Foglight browser interface.
2
On the navigation panel, under Dashboards, click Log Monitor.
•
To create a Quest File Log Monitor Agent instance, on the Log Monitor dashboard, in the top-right corner, click File Log Monitor to launch the Create FileLogMonitorAgent wizard.
•
To create a Quest Windows Event Log Monitor Agent instance, on the Log Monitor dashboard, in the top-right corner, click Windows Event Log Monitor to launch the Create WindowsEventLogMonitorAgent wizard.
3
Select the host where you want to run the agent instance, and specify the agent name.
a
On the Agent Manager and Agent Name page in the wizard, click Agent Manager, and from the list that appears, select a host running an Agent Manager instance that you want to use to manage the agent instance that you are about to create.
b
Specify the name you want to assign to this agent instance. In the Agent Name box, type the agent name. Optionally, select the Generate Name check box to have the wizard generate the name automatically.
c
Click Next.
Quest File Log Monitor and Windows Event Log Monitor agents each come with a different set of agent properties, so the contents of this page are different, reflecting the type of the agent instance that you are creating.
File Log Monitor Agent properties
Windows Event Log Monitor Agent properties
4
Configure the agent properties, as required.
For more information about specific agent properties, see the following:
 
File Log Monitor Agent properties
•
Monitored Hosts
•
File Formats
•
Log Files
•
Data Collection Scheduler
Windows Event Log Monitor Agent properties
•
Monitored Hosts
•
Event Logs
•
Data Collection Scheduler
a
To view or edit a property, click View on the right of the list property name.
In the dialog box that appears, click Edit, and make changes to the existing entries, or add new ones, as required.
b
Some columns support regular expressions. To test a regular expression, click RegEx Tester.
 
* 
TIP: The agent supports PCRE (Perl Compatible Regular Expressions). For details about the PCRE syntax, visit http://perldoc.perl.org/perlre.html.
The Regular Expression Tester Dialog box allows you to write a regular expression, and test it against a text sample. Simply type a regular expression in the Test Regular Expression box, copy the text sample to the Test Source area, and click Match. The Test Result area displays the result of your regular expression.
To close the Regular Expression Tester Dialog box, click Cancel.
c
To save your changes to the list, in the list dialog box, click Save. Click Cancel to close it.
d
On the Agent Properties page, click Next.
The wizard checks whether the selected Agent Manager has any credentials configured for the hosts whose files you want to monitor. If it finds the credentials that grant the Agent Manager permissions to access the monitored hosts, this is indicated on the Credential Verification page.
5
Review the information on the Credential Verification page, and make any changes, if required.
•
If the page indicates that you have a valid set of credentials in place, you do not need to make any changes. This is typically the case if the hosts whose files you are about to start monitoring are already monitored by other Foglight for Infrastructure agents (a WindowsAgent or a UnixAgent).
•
If the page indicates that the Agent Manager does not have the credentials needed to access the monitored hosts, click Manage Credentials, and create a new credential, as required. For more information, see “Controlling System Access with Credentials” in the Administration and Configuration Help.
Click Next.
The Summary page reflects the newly configured settings, including agent properties. Because File Log Monitor and Windows Event Log Monitor agents each come with a different set of agent properties, the contents of this page are different, reflecting the type of the agent instance that you are creating.
Create FileLogMonitorAgent wizard
Create WindowsEventLogMonitorAgent wizard
6
Review the information on the Summary page, and click Finish to start collecting data.

Investigating log records

The Log Monitor dashboard displays the amount of log records over the monitored time range, lists the monitored log records, along with individual record details, such as its source, message, and severity, among others. Use this dashboard on a daily basis to review the monitored log records and identify potential signs that can lead to performance bottlenecks. For example, a significant increase in the number of log records can help you predict and prevent potential system-level issues.

To access this dashboard, from the navigation panel, choose Dashboards > Log Monitor.

Start by choosing the host containing the log files whose records you want to review, using the File Selector view. From here, for file logs, you can select the directory and the log file name.

For Windows Event Logs, select Event Log Files, and choose the name of the Windows Event Log (for example, Application).

 
* 
NOTE: Log names are specified in the Windows Event Log Monitor agent properties. For more information, see Event Logs.

For complete information about the data appearing in these views, see Foglight Log Monitor views.

Related Documents

The document was helpful.

Select Rating

I easily found the information I needed.

Select Rating