Troubleshooting WinRM host monitoring (4217338)

Steps to aid in configuring & troubleshooting a Windows host for remote monitoring via WinRM (Windows Remote Management)

After following KB 4295397 and configuring WinRM using "winrm quickconfig", what additional troubleshooting steps can be followed?

The following conditions can result in a WinRM failure:

• The WinRM listener is not configured on the remote machine.
• Blocked connections from the Foglight Agent Manager host to the WinRM listener ports on the remote machine.
• The AllowUnencrypted WinRM service configuration setting is false, and HTTP is used on the remote machine.
• The expected authentication type is not enabled for the WinRM service on remote machine. For example, Basic authentication is expected to be used, but it is not enabled.
• A fully qualified domain name is not specified in the user credential created on the Management Server.
• An incorrect Kerberos configuration file is used. For example, the file does not exist on a UNIX system, or the realm is not set up for cross-realm authentication.
• Key Distribution Center (KDC) host configured in the Kerberos configuration file is not reachable on port 88 from the Foglight Agent Manager host.
• The WinRM parameter TrustedHosts is not set.

Please refer to the latest version of the Foglight Agent Manager user guide for specific details on Configuring Windows Remote Management (WinRM).

1. Navigate to Navigation Panel | Dashboards | Administration | Credentials | Manage Credentials
2. Select the credential to be used for monitoring of the remote host and select edit | Credential Properties
   1. If using a domain service account for remote monitoring, ensure the following is performed
      1. Insert the Fully Qualified Domain under the Domain field
      2. Insert the User Name without a domain under the User Name field
         example: foglight_svc
   2. ​If using a local service account for remote monitoring, ensure the following is performed
      1. ​Leave the Domain field empty

Steps to investigate from remote host

1. Remote to target host and open command line with admin privileges
2. Run the following command:
   winrm get winrm/config   (This command will display the WinRM configuration for this host)
3. Make note of the following items under "Service"
   Example:
    Service
        RootSDDL = O:NSG:BAD:P(A;;GA;;;BA)S:P(AU;FA;GA;;;WD)(AU;SA;GWGX;;;WD)
        MaxConcurrentOperations = 4294967295
        MaxConcurrentOperationsPerUser = 15
        EnumerationTimeoutms = 60000
        MaxConnections = 25
        MaxPacketRetrievalTimeSeconds = 120
        AllowUnencrypted = false
        Auth
            Basic = false
            Kerberos = true
            Negotiate = true
            Certificate = false
            CredSSP = false
            CbtHardeningLevel = Relaxed
4. Note: If the WinRM configuration does not display due to it not being configured, run the following:
   winrm qc  or winrm quickconfig (same command)

Configuring the remote host for monitoring with WinRM 

1. Using a domain service account
   1. ​winrm set winrm/config/service/auth @{Basic="false"}
      Note:  Basic is used when using a local service account
   2. winrm set winrm/config/service @{AllowUnencrypted="true"}
      Note: AllowUnencrypted determines whether or not HTTP (true) or HTTPS (false) connection is used
   3. Ensure Kerberos = true  WinRM service configuration 
   4. Ensure Negotiate = true  WinRM service configuration
2. Using a local service account
   1. ​winrm set winrm/config/service/auth @{Basic="true"}
   2. winrm set winrm/config/service/auth @{Negotiate="true"}
   3. winrm set winrm/config/service @{AllowUnencrypted="true"}
      Note: AllowUnencrypted determines whether or not HTTP (true) or HTTPS (false) connection is used

Reviewing Application Event Logs

WinRM logs activity to an event log on the target machine. This includes both success and failure messages for authentication.

To view application event logs:

1. On the target machine, right-click My Computer and select Manage.
2. In the navigation tree on the left, choose System Tools | Event Viewer | Applications and Services Logs | Microsoft | Windows | Windows Remote Management | Operational.
   The default Operational log contains the most common events.

To enable additional debug logging information:

1. Click View.
2. Click Show Analytic and Debug Logs.
3. Right-click the log file you want to view.
4. Select Enable Log.

Enabling connection type debugging from the Foglight Agent Manager (FglAM)

If the only information you are interested in is the types of connections that are being established, there is a command-line setting that enables logging the connection types.

Run the Agent Manager with the following switch:

-Dquest.debug.windowsinfo.types

NOTE: This logging occurs every time a connection is established and can be very verbose. It is recommended for debugging purposes only.

For additional debug logging refer to: How to enable Kerberos Debug at the FglAM level (4309953)

When investigating WinRM configuration problems please collect the following items:

1. Foglight Agent Manager (FglAM) support bundle
   Dashboards| Administration| Agents| FglAM Support Bundle
2. Output of the WinRM configuration
   winrm get winrm/config >> _winrm.cfg
3. Output of the WinRM listener configuration
   winrm enum winrm/config/listener >> _winrmlistener.cfg
4. FMS support bundle