Foglight for Infrastructure 6.3.0

Agent properties

When an agent connects to the Foglight Management Server, it is provided with a set of properties that it uses to configure its correct running state. For more information about working with agent properties, see Creating agent instances.

The UnixAgent is shipped with default properties that can be modified to suit your system requirements. The properties specific to the UnixAgent are illustrated in the following screenshot.

Figure 12. UnixAgent properties

You can configure the following settings for this agent:

•

Properties:

•

Host: host name or IP address.

•

Host name override: host name to be used to store this host’s data in the Foglight data model.

•

Port: SSH port on which the agent connects. Default value = 22.

•

Top CPU Processes: number of top CPU processes to be monitored. Default value = 5.

•

Top Memory Processes: number of top memory processes to be monitored. Default value = 5.

•

Top IO Processes: number of top IO processes to be monitored. Default value = 5.

•

Keep Retry If Collection Failed: Default value = False. When set to True, the agent keeps trying to collect data when failed to connect to the target host during startup.

•

Collect Top N processes only: Default value = False. When set to True, the agent collects data for Top CPU Processes, Top Memory Processes, and Top IO Processes only.

•

Use native collector (if available): Default value = True.


	NOTE: The AIX® and HP-UX agents deploy a simple binary program to the remote system and execute it to gather a better collection of metrics. It runs in the same user account under which the Agent Manager connects, and is deleted from the system when the agent is deleted. This program is not required for the agent to operate, but some data become unavailable if it is turned off. HP-UX in particular requires the native binary to collect many of the standard metrics.


	IMPORTANT: Using the native collector to monitor an HP-UX system requires that the monitoring UnixAgent’s account belong to the sys and bin groups on the monitored system. Failing to add the account to these groups prevents the agent from collecting some logical disk metrics.

•

Aggregate data for all instances of a program: Default value = True. When set to True, the agent collects data from all the instances of a program (for example all Oracle® instances), aggregates the information, and presents it in a unified report.

When set to False, the agent still collects aggregate data, but it also includes details about every process. This could result in a lot of data sent to the Management Server and may have a performance impact.

•

Collect Top N Process Details: Default value = True. When set to True, the agent collects data for the Top CPU Processes, Top Memory Processes, and Top IO Processes. Details about these top processes are accessible from the Infrastructure Environment dashboard (for example, to see the top CPU processes, in the Monitoring tab, select a host on the Quick view, click the Explore button in the Resource Utilizations view, and click any metric indicator in the CPU area; the top CPU consumers are displayed in the CPU Details dashboard.)

When set to False, some extra details (owning username/domain) are unavailable for processes reported in the various “Top N” collections. Gathering this information can be expensive if the connection to the remote machine is slow or the “Top N” collections are configured to be very large.

•

Collect process metrics: Default value = True. When set to True, the agent collects process metrics.

•

Collect CPU metrics: Default value = True. When set to True, the agent collects performance metrics about the system’s CPUs.

•

Collect disk metrics: Default value = True. When set to True, the agent collects performance metrics about the system’s disks.

•

Include filesystems mounted from memory: Default value = False. This property indicates to the agent whether or not to collect information about RAM disks. This information is typically collected when monitoring Linux® and Solaris® platforms, and not collected for HPUX and AIX® platforms.

•

Include mounted remote filesystems: Default value = False. When set to True, the agent collects metrics about remotely mounted disks.

•

Collect memory metrics: Default value = True. When set to True, the agent collects performance metrics about the system’s memory.

•

Collect network metrics: Default value = True. When set to True, the agent collects performance metrics about the network.


	TIP: If you are collecting basic host metrics using Foglight for VMware, you may need to set the Collect CPU/disk/memory/network metrics options to False, to prevent Foglight for VMware and Foglight for Infrastructure from reporting different or conflicting values. For Foglight for VMware, consider setting all four flags to False.

•

Collect System ID: Default value = True. This property indicates to the agent whether or not to collect a unique system ID from this system. This is not always desirable when monitoring Hyper-V® systems, as some Hyper-V systems use the same ID for multiple systems and are not unique.

•

Collect Hypervisor metrics: Default value = False. This property indicates to the agent whether or not to collect additional metrics from hypervisor systems (for example, Solaris global Zone, AIX® LPAR, and so on).

•

Use ping to validate host availability: Default value = False. When set to True, the agent is configured to use ping to detect if the monitored host is unavailable. If the agent fails to make a connection to the monitored host, and this property is set to True, the agent sends a ping command to the host. If the host does not respond, the Host.monitored observation is set to UNAVAILABLE (for more details, see Host availability alerting).


	NOTE: When the Use ping to validate host availability property is enabled on a UNIX® platform, the sudoer file needs to configured to allow the ICMP process to run with NOPASSWD. For details, see Configuring secure launcher permissions using sudo.

•

Use commands with sudo: Default value = False. When set to False, the agent does not use commands that require sudo, and does not collect metrics that require root permissions. For more information about sudo commands that require root access, see Configuring secure launcher permissions using sudo.

•

Filter local disks based on declared filesystem types: Default value = True. When set to True, the agent enables the local filesystem type filtering.

•

Path to sudo command: The path to the sudo executable.

•

Process Availability Config: A list of monitored processes and their expected instance counts. The list contains three columns: Process Name, Command Line, and Expected Process Count, and can be edited, as required. The agent compares the number of actual processes with the number of expected processes, found in this list. Results are displayed in the Processes > User Defined Processes (Process Availability Config) view (for details, see User Defined Processes (Process Availability Config)).


	TIP: This feature replaces the AppMonitor capability provided by the legacy OS cartridge.

The combination of Process Name, Command Line, and Expected Process Count consists of the expression that identifies the query criteria for filtering out the process instances. The following samples demonstrate how to find the values of Process Name and Command Line on Solaris:


	TIP: So far, we only support exact string matching for Process Name, while regular expression is supported for Command Line.

•

Solaris: Execute the “/usr/bin/ps -e -o uid,pid,ppid,vsz,rss,time,pcpu,sid,s,user,comm,args” command. Then you will get the following process details.

16801 21353 21351 249104 159096 01:08 0.0 2665 S murex /usr/local/java/jdk1.7.0_51/bin/java /usr/local/java/jdk1.7.0_51/bin/java -Dname=RTZSP_DEALGEN -Djava.library.path=/

In the sample above:

•

Process Name is /usr/local/java/jdk1.7.0_51/bin/java

•

Command Line can be either usr/local/java/jdk1.7.0_51/bin/java -Dname=RTZSP_DEALGEN -Djava.library.path=/ or some keywords (for example, RTZSP_DEALGEN)

•

Filesystem Type Config: A list of filesystem types.

•

Exclude/Include FileSystems: The type of FileSystems list to be used for monitoring.

•

Exclude (default) indicates that the file systems listed in the FileSystems list should be excluded from monitoring.

•

Include performs system monitoring on the file system that you are defining.

•

Filesystem Config: A list of file systems that are excluded from monitoring (if the Exclude/Include FileSystems property is set to Exclude) or included in the monitoring (if the Exclude/Include FileSystems property is set to Include). You can modify, clone, and delete lists of excluded/included file systems, as necessary. The list contains three columns: MountPoint regular expression, Remote host name regular expression, and Monitored host regular expression. An entry in the list consists of three regular expressions that together identify one or more file systems that are excluded from/ included in the monitoring. For example:

Table 6. Example of file systems list
Column	Value	Meaning
MountPoint regular expression	/workspace	If set, the file systems with the matching mount point should be excluded/included. In this example: All file systems located in /workspace.
Remote host name regular expression	tor.*	This is related to the file systems which are remotely mounted. If set, the remote file systems with the matching remote host should be excluded/included. In this example: All remote hosts starting with “tor” such as tor.test.com or tor.prod.com.
Monitored host regular expression	tor.*	This refers to the hostname collected by the OS. If set, all the OS-collected metrics for the matching host should be excluded/included. In this example: All hosts starting with “tor” such as tor.test.com or tor.prod.com.


	NOTE: For complete information about the regular expressions syntax, see Appendix: Building regular expressions in Foglight.

Click Edit to modify the entries in the list. In the HostAgents - UnixAgentPlus - ListName dialog box that appears, you have several options: add or delete rows, edit the existing entries, select file systems to be excluded/included, save or revert changes. To exclude/include one or more file systems from monitoring, add table rows entries and populate them with regular expressions, as required.

Click Clone to clone the selected exclude/include list. In the Clone ListName dialog box that appears, enter a name for the new list, and click OK to save the it. The new exclude/include list is added to the drop-down list.

Click Delete to delete the selected exclude/include list. In the Delete ListName dialog box that appears, confirm the deletion by clicking Yes. The exclude/include list is deleted from the drop-down list.

•

Data Collection Scheduler:

•

Collector Config: defines how quickly the agent collects data. UNIX® provides a defaultSchedule configuration. Users can modify, clone, and delete configurations, as necessary.

Click Edit to modify the configuration selected from the drop-down list. In the HostAgents - UnixAgent - ConfigurationName dialog box, you have several options: add or remove rows (that is, collectors and their settings), modify fields, and save or revert changes.

Click Clone to clone the configuration selected from the drop-down list. In the Clone ConfigurationName dialog box, enter a name for the new configuration, and click OK to save the it. The new configuration is added to the drop-down list.

Click Delete to delete the configuration selected from the drop-down list. In the Delete ConfigurationName dialog box, confirm the deletion by clicking Yes. The configuration is deleted from the drop-down list.

Configuring secure launcher permissions using sudo

Some UnixAgents can function without root privileges, but certain metrics can only be collected by commands which must be run as root. In order to give these agents the required access, Foglight Agent Manager is configured to launch these agents using a tool such as sudo that allows privilege escalation (without a password).

To this effect, the sudo configuration file (/etc/sudoers) must be configured so that password prompts are not required for a number of executables. The commands requiring elevated privileges differ by platform. The following commands must be configured for this version of Foglight for Infrastructure.

Table 7. Sudo commands that must be configured
Agent Type	Command	Notes
AIX®	/usr/bin/svmon	Some AIX versions require elevated permissions to access virtual memory information.
Linux®	/usr/bin/find, /bin/cat	Used to read IO statistics from the /proc filesystem.
Linux®	/sbin/ethtool or /usr/sbin/ethtool (depending on distribution) /sbin/mii-tool or /usr/sbin/mii-tool (depending on distribution)	Used to determine the network card bandwidth; ethtool is favoured if it is found.
Oracle Solaris®	N/A	Oracle Solaris agents do not require sudo access.
HP-UX	N/A	HP-UX agents do not require sudo access.

The following is an example of how to configure the /etc/sudoers file to allow the user foglight to execute Linux® commands without being prompted for a password:

foglight ALL = NOPASSWD: /usr/bin/find, /bin/cat, /sbin/ethtool

In addition, the requiretty flag must not be set in /etc/sudoers for the user, since Foglight for Infrastructure agents use non-interactive shells.

The following is an example of how to unset the requiretty flag for a single user named foglight, so that this user can run sudo commands remotely:

Defaults:foglight !requiretty


	NOTE: If requiretty flag is set, sudo can run only when the user is logged in to a real tty. When this flag is set, sudo can only be run from a login session and not via other means, such as cron or cgi-bin scripts. This flag is off (unset) by default.

Controlling log usage for sudo access

Using commands with sudo access can result in increased logging. Sudo provides the following levels of logging, each resulting in the capture of a specific type of information:

•

Log all commands that run with sudo.

•

Log only commands that run with sudo and cause errors.

•

Log all command-line output produced by the commands that run with sudo.

•

Log all command-line input provided to the commands that run with sudo.

Depending on the user’s sudo and syslog.conf configuration, sudo use may result in excess logging. To minimize the amount of log messages, ensure that sudo does not make use of the LOG_INPUT or LOG_OUTPUT tags for the commands that the UnixAgent runs. Depending on the existing monitored hosts’ configuration, any lines added to the /etc/sudoers file for Foglight monitoring may have to include NOLOG_OUTPUT or NOLOG_INPUT to override the default configuration. For example, for a user named foglight connecting to a monitored host, the following lines are required:

AIX

foglight ALL = NOLOG_INPUT: ALL, NOLOG_OUTPUT: ALL, NOPASSWD: /usr/bin/svmon

Linux

foglight ALL = NOLOG_INPUT: ALL, NOLOG_OUTPUT: ALL, NOPASSWD: /usr/bin/find|,

/bin/cat, [/sbin/ethtool|/usr/sbin/ethtool|sbin/mii-tool|/usr/sbin/mii-tool]

The last argument in this syntax depends on the type and location of the tool, ethtool or mii-tool, used to determine the network card bandwidth. If you are unsure which tool your system uses, you can specify all of them:

foglight ALL = NOLOG_INPUT: ALL, NOLOG_OUTPUT: ALL, NOPASSWD: /usr/bin/find|,

/bin/cat, /sbin/ethtool, /usr/sbin/ethtool, /sbin/mii-tool, /usr/sbin/mii-tool

For complete information about sudo and /etc/sudoers, refer to the sudo and /etc/sudoers man pages.

About the MultiHostProcessMonitorAgent

The MultiHostProcessMonitorAgent monitors both Windows® and Linux® systems and collects the following information:

•

Processes running or waiting to run on a monitored host

There are views, rules, and data associated with this agent. For more information, see Reference

For more details, see these topics:

•

Supported platforms

•

Agent properties

Supported platforms

For a list of platforms supported for the MultiHostProcessMonitorAgent, see “System Requirements” in the Foglight for Infrastructure Release Notes.


	NOTE: The infrastructure UNIX® agents do not execute shell commands on the monitored system. Therefore, there are no special requirements or restrictions regarding the shells on which they are installed.

Agent properties

The MultiHostProcessMonitorAgent is shipped with default properties that can be modified to suit your system requirements. The properties specific to the MultiHostProcessMonitorAgent are illustrated in the following screenshot.

Figure 13. MultiHostProcessMonitorAgent properties

You can configure the following settings for this agent:

•

Properties:

•

Multiple Hosts Config: The list of hosts to be monitored. Click Edit to modify the following Multiple Host Config properties

•

OS Type: type of the operating system (Unix® or Windows®).

•

Host: host name or IP address.

•

Host name override: host name to be used to store this host’s data in the Foglight data model.

•

SSH Port: SSH port on which the agent connects. Default value = 22. Applicable to the Linux® platform only.

•

Top CPU Processes: number of top CPU processes to be monitored. Default value = 5.

•

Top Memory Processes: number of top memory processes to be monitored. Default value = 5.

•

Top IO Processes: number of top IO processes to be monitored. Default value = 5.

•

Keep Retry If Collection Failed: Default value = False. When set to True, the agent keeps trying to collect data when failed to connect to the target host during startup.

•

Collect Top N processes only: Default value = False. When set to True, the agent collects data for Top CPU Processes, Top Memory Processes, and Top IO Processes only.

•

Collect additional metrics for VMWare: Default value = True. When set to True, the agent collects the following data:

•

Host: availablePagingSpace, runQueueLength, contextSwitches

•

CPU: totalHz, percentUserTime

•

Memory: capacity, consumed, pageInRate, pageOutRate, and utilization

•

Collect process metrics: Default value = True. When set to True, the agent collects process metrics.

•

Collect declared processes only: Default value = False. When set to False, the agent collects metrics for all processes that matches the values defined in Process Availability Config. An alarm will be raised when the actual process count is lower than Expected Process Count.

When the Collect declared processes only property is set to true, only processes declared under Process Availability Config will be reported.

•


	NOTE: When the Use ping to validate host availability property is enabled on a UNIX® platform, the sudoer file needs to configured to allow the ICMP process to run with NOPASSWD. For details, see Configuring secure launcher permissions using sudo.

•

Use commands with sudo: Default value = False. Applicable to the Linux® platform only. When set to False, the agent does not use commands that require sudo, and does not collect metrics that require root permissions. For more information about sudo commands that require root access, see Configuring secure launcher permissions using sudo.

•

Path to sudo command: The path to the sudo executable. Applicable to the Linux® platform only.

Process Availability Config: A list of monitored processes and their expected instance counts. The list contains three columns: Process Name, Command Line, and Expected Process Count, and can be edited, as required. The agent compares the number of actual processes with the number of expected processes found in this list. Results are displayed in the Processes > Processes > User Defined Processes (Process Availability Config) view (for details, see User Defined Processes (Process Availability Config)).


	TIP: This feature replaces the AppMonitor capability provided by the legacy OS cartridge.


	TIP: So far, we only support exact string matching for Process Name, while regular expression is supported for Command Line.

•

Solaris: Execute the “/usr/bin/ps -e -o uid,pid,ppid,vsz,rss,time,pcpu,sid,s,user,comm,args” command. Then you will get the following process details.

16801 21353 21351 249104 159096 01:08 0.0 2665 S murex /usr/local/java/jdk1.7.0_51/bin/java /usr/local/java/jdk1.7.0_51/bin/java -Dname=RTZSP_DEALGEN -Djava.library.path=/

In the sample above:

•

Process Name is /usr/local/java/jdk1.7.0_51/bin/java

•

Command Line can be either usr/local/java/jdk1.7.0_51/bin/java -Dname=RTZSP_DEALGEN -Djava.library.path=/ or some keywords (for example, RTZSP_DEALGEN)

•

Data Collection Scheduler

•

Collector Config: defines how quickly the agent collects data. Both Windows and Linux® provide a defaultSchedule configuration. Users can modify, clone, and delete configurations, as necessary.

Configuring secure launcher permissions using sudo

Table 8. Sudo commands that must be configured
Agent Type	Command	Notes
Linux®	/usr/bin/find, /bin/cat	Used to read IO statistics from the /proc filesystem.
Linux®	/sbin/ethtool or /usr/sbin/ethtool (depending on distribution) /sbin/mii-tool or /usr/sbin/mii-tool (depending on distribution)	Used to determine the network card bandwidth; ethtool is favoured if it is found.

The following is an example of how to configure the /etc/sudoers file to allow the user foglight to execute Linux® commands without being prompted for a password:

foglight ALL = NOPASSWD: /usr/bin/find, /bin/cat, /sbin/ethtool

In addition, the requiretty flag must not be set in /etc/sudoers for the user, since Foglight for Infrastructure agents use non-interactive shells.

The following is an example of how to unset the requiretty flag for a single user named foglight, so that this user can run sudo commands remotely:

Defaults:foglight !requiretty


	NOTE: If requiretty flag is set, sudo can run only when the user is logged in to a real tty. When this flag is set, sudo can only be run from a login session and not via other means, such as cron or cgi-bin scripts. This flag is off (unset) by default.