Upgrade and Compatibility

Upgrade and Compatibility

The latest version of Foglight for Active Directory is 5.7.2.2. Upgrades from version 5.6.9 and later are supported.

To upgrade the Foglight for Active Directory to the latest version:

1. Deactivate all of the Active Directory agents.
2. Install version 5.7.2.2 of the cartridge as you would a new Foglight for Active Directory. Do not delete the older version of the .car file.
   For cartridge installation instructions, refer to "Installing Foglight cartridges" in the Foglight Administration and Configuration Help.
3. Deploy the agent package to each Foglight Agent Manager that hosts an Active Directory agent instance and wait for the version to update.
   Note: This may take two to three refresh cycles.
4. From the navigation panel, navigate to Dashboards > Active Directory > Active Directory Environment > Administration tab. In the Agents view select the Active Directory agents that you want to upgrade, and click Upgrade Agent.
   Note: You can specify the lockbox when upgrading the agents. The credentials for the existing agents are updated automatically.
   
5. Verify the agent properties and update the properties and collection intervals as required.
6. Activate the agents and start data collections.

Important: For a list of issues that you may encounter after upgrading the Foglight for Active Directory to version 5.7.2.2, and ways to troubleshoot these issues, see section Potential issues after upgrading the cartridge to version 5.7.2.2.

Note: If you are also running Foglight for Exchange, you must upgrade the Exchange agents as well. It is strongly recommended that you run the same version and patch level of both cartridges.

The following is a list of product versions and platforms compatible with this release.

Back to Top

System Requirements

System Requirements

Before installing Foglight for Active Directory, ensure your system meets the following minimum hardware and software requirements:

Back to Top

Prerequisites

Prerequisites

The following prerequisite conditions must be in place in order to successfully initialize an Active Directory agent. Failure to meet these prerequisites may result in missing metrics in Foglight for Active Directory dashboards.

Note: The Remote Access Diagnostics utility, provided with this product, checks the connectivity between the Foglight Agent Manager (FglAM) and Active Directory and Exchange servers that are being monitored. It also tests for the prerequisite conditions that must be met in order to initialize an Active Directory agent. For more information on running the Remote Access Diagnostics utility, see the Remote Access Diagnostics User Guide.

Permissions

Note: Make sure to give minimum required privilege to your Active Directory® or Certificate Authority agent; otherwise this agent can not start data collection.

An Active Directory account with Administrator permissions  (domain or built-in administrators) must be specified in agent properties. This is the account used to run remote scripts. Foglight for Active Directory uses the userPrincipalName in the agent properties, so the sAMAccountName and the account CN must be identical. Also, they must not contain spaces, or LDAP authentication errors may occur.

To run remote scripts, a Certificate Authority agent requires an account with relevant privileges:

• If the Certificate Authority server is a member server then agent account needs to be member of Domain Admins group.
• If the Certificate Authority is a Domain Controller then the agent account needs to be member of either Domain Administrators group or Domain Admins group.

Agent must be able to reach the target host

Server objects do not appear until at least one piece of data has been collected and recorded. If communication fails completely, you will not see objects.

Configuration steps:

1. Test Ping by IP. You must be able to ping the collection target from the FglAM hosting the agent instance. If ping by IP fails, there are routing issues.
2. Test Ping by host name. A DNS server or Hosts file must be available to the FMS server in order to resolve names. If ping by host name fails, there are DNS or Hosts file issues.
3. If a Hosts file is used it should contain an entry for each domain where hosts reside. For example:
   10.10.10.100 domain.local
   10.10.10.200 childdomain.domain.local
4. In addition, individual servers must resolve to the unqualified and fully qualified name. For example:
   10.10.10.101 server server.domain.local
   The Hosts file is located at %windir%\system\drivers\etc

Remote scripting must be enabled

Since Foglight for Active Directory uses an agent-less design, remote execution of scripts must be enabled on all domain controllers. If communication fails completely, you will not see server objects. If partial data is collected, the server object will appear in the UI and the metrics with values will be displayed.  

Distributed COM (DCOM) must be enabled on all Domain Controllers (Active Directory Servers) or all Certificate Authority Servers.

To enable Distributed COM (DCOM):

1. Click Start | Run.
2. In the Run dialog, enter dcomcnfg and click OK.
3. Expand Component Services and then Computers.
4. Right-click the My Computer object and select Properties.
5. On the Default Properties tab, check the Enable Distributed COM on this computer option.

Configuring Windows Remote Management (WinRM)

For details about this topic, refer to the "Configuring Windows Remote Management (WinRM)" section in the Foglight Agent Manager Guide.

Remote Registry Service must be running

The Remote Registry service must be running to allow agents remote access to the registry.

Windows Server 2008 R2 and Server 2012 prerequisite

The account specified in the agent properties must have Full Control permissions on the registry keys for the following:

• HKEY_CLASSES_ROOT\CLSID 72C24DD5-D70A-438B-8A42-98424B88AFB8 (Windows Script Host Shell Object)
• HKEY_CLASSES_ROOT\CLSID 76A64158-CB41-11d1-8B02-00600806D9B6 (WBEM Scripting Locator)
• HKEY_CLASSES_ROOT\CLSID 0D43FE01-F093-11CF-8940-00A0C9054228 (Windows Script FileSystem Object)

• HKEY_CLASSES_ROOT\Wow6432Node\CLSID 72C24DD5-D70A-438B-8A42-98424B88AFB8
• HKEY_CLASSES_ROOT\Wow6432Node\CLSID 76A64158-CB41-11D1-8B02-00600806D9B6
• HKEY_CLASSES_ROOT\Wow6432Node\CLSID 0D43FE01-F093-11CF-8940-00A0C9054228

Note: In Windows operating systems prior to Windows Server 2008 R2, these permissions are granted to members of the Administrators group by default. In Windows Server 2008 R2 and Windows Server 2012, these permissions must be explicitly granted.

Symptom: Depending on the prerequisite, you will encounter a complete failure to collect data (missing servers) or subsets of missing values (empty charts).

Resolution:

We recommend that you create a backup copy of the Windows Registry before making any changes so that you can revert the changes, if necessary.

To grant permissions on these keys:

1. Login to the domain controller with an Administrator account that you are comfortable having ownership over these keys.
2. Start the Windows Registry Editor (run regedit.exe).
3. If asked to allow the Regedit program to make changes to the computer, click Yes.
4. Navigate to the registry item: HKEY_CLASSES_ROOT\CLSID\{clsid} or HKEY_CLASSES_ROOT\Wow6432Node\CLSID\{clsid}, as necessary.
5. Right-click the registry key and select Permissions.
6. Click Advanced.
7. Open the Owner tab.
8. In the Change Owner to box, select one of the following entries:
   • the user account that is used by the Active Directory agent
   • the administrative group for the account you currently belong to
9. Select the Replace the owner on subcontainers and objects check box.
10. If the account is not listed, click Other user or groups to add the account.
11. Click OK.
12. Under Group or user names, select the account that will be specified in the agent properties. If the account is not listed, click Add to add the account.
13. Under Permission for account, select the Allow Full Control check box and click OK.
14. Close the Registry Editor.

Sample procedure for checking and registering a WMI class

The Extensible Storage Engine (ESE) is the database engine used by Active Directory®. Foglight for Active Directory collects metrics and will fire alarms on ESE performance. It is recommended to verify that the Win32_PerfRawData_ESENT_Database WMI class is registered on each monitored domain controller by confirming the 'Database' Performance Object within Performance Monitor (Perfmon) exists. If this class is not registered, ESE queries will fail with 0x80041010 errors.

To check and register the ESENT WMI Class:

1. Check for the Database performance object.
   1. Start | Run and enter: perfmon
   2. Click the plus (+) toolbar button.
   3. On the Add Counters dialog, click the Performance object drop-down menu and locate the Database entry.
2. If the Database entry is present, the class is registered and there is likely another problem.
3. If a Database entry is not present, register the following WMI class: Win32_PerfRawData_ESENT_Database WMI.
   1. Run the RegisterEsentWmiClass.vbs script included with Foglight for Active Directory.
   2. At a command line, enter: mofcomp %windir%\system32\wbem\scm.mof
   3. At a command line, enter: wmiadap /f

This procedure sets registry keys and refreshes the WMI database so it is aware of the change.

Kerberos configuration files

The Kerberos configuration file should be located in the following folder in the Fglam server:
    /etc/krb5/krb5.conf [Solaris®]
    %WINDOWS_ROOT%\krb5.ini [Windows]
    /etc/krb5.conf [Linux®]

The krb5.ini or krb5.conf file should contain the realm info and hostname of the KDC for this realm. For example:

[libdefaults]
default_realm = MY.REALM
[realms]
MY.REALM = {
    kdc = kdc.my.realm
}

Configure root certificates for the Agent Manager

Important: Starting with version 5.7.2.2, Foglight for Exchange trusts (by default) any certificates for secure LDAP connections, and does not require users to import the SSL certificate any longer. The only case when users need to import the certificate is when they set the vm parameter "quest.ldap.ssl.trustAnyCert" as False to disable any certificate trust.

When collecting data using LDAP through SSL communication, a new Certificate Authority must be added to the Agent Manager’s Java® Runtime Environment (JRE). The JRE includes a command-line tool keytool which can be used to add the new Certificate Authority. 

keytool -import -file <importCertPath> -alias <someName> -keystore <cacertsPath> -storepass <changeit>
keytool -list -alias <someName> -keystore <cacertsPath> -storepass <changeit>

Here are example commands that import and list a new root certificate:

<FMS_HOME>\jre\bin\keytool -import -file MySSL.cer –alias MySecuryLDAP.ca -keystore <FMS_HOME>\jre\lib\security\cacerts -storepass changeit
<FMS_HOME>\jre\bin\keytool -list -alias MySecuryLDAP.ca -keystore <FMS_HOME>\jre\lib\security\cacerts -storepass changeit

The initial password of the cacerts keystore file is changeit. System administrators should change this password and the default access permissions of this file when installing the SDK. The file can be found in the directory <FMS_HOME>\jre\lib\security\cacerts (embedded Agent Manager) or <FglAM_HOME>\jre\<JRE_VERSION>\jre\lib\security\cacerts (external Agent Manager).

Note: The certificate file that you want to import should be the public certificate for the Certificate Authority that signed the server's SSL certificate, not the SSL certificate itself. The Agent Manager must be restarted for the certificate to take effect. If security LDAP is enabled when creating the Active Directory agent via the Agent Setup wizard, the root certificate also needs to be added to the Foglight Management Server’s Java Runtime Environment (JRE).

Back to Top 

Troubleshooting

Troubleshooting

This section provides information about problems that you might encounter while monitoring your environment with Foglight for Active Directory, and describes the solutions available to troubleshoot these problems.

DCs on Windows Server 2012/2012 R2 Systems may experience high CPU usage

Symptom: In some circumstances, DCs on Windows Server 2012/2012 R2 systems may experience high CPU usage when monitored by the Active Directory agent. This issue only appears when using WinRM connections. Using WMI/DCOM connections prevents this issue.

Resolution:

If this issue is encountered, contact Support for assistance.

To troubleshoot this issue directly, use the Windows Task Manager to look for an increasing number of active conhost.exe or svchost.exe processes. If this problem is observed, the problem can be confirmed by adding the optional "Command Line" column to Task Manager (View > Select Columns > [ x ] Command Line). You should then see WinRM commands associated with the conhost.exe or svchost.exe instances.

If many of these processes are observed, increase the WinRM message envelope size from the default size of 500, as follows:

winrm set winrm/config @{MaxEnvelopeSizekb="1000"}

Microsoft® offers a workaround for this issue in the "Svchost.exe uses excessive CPU resources on a single-core Windows Server 2012 domain controller" article (KB 3118385).

Recommended best practices

The following procedure is a best practice that is recommended for optimal performance.

Disable automatic updates on Foglight Management Server

Do NOT allow the Microsoft® automatic update feature to force an update of the server hosting the Foglight Management Server. This automatic update feature does not allow enough time for the Foglight Management Server to shutdown gracefully, which may leave your agents in a broken state.

Symptom: Cartridge agents will appear to be deactivated on the Agent Status dashboard.

Resolution: Using the Agent Status dashboard, select the deactivated agent and select the Activate button. If you cannot activate the selected agent, delete and reinstall the agent.

Potential issues after upgrading the cartridge to version 5.7.2.2

Insufficient heap memory

Symptoms:

When upgrading to version 5.7.2.2, you encounter an error message similar to the following message (actual values may vary):

Error deploying package … Cause: The addition of 2097152kb to the negotiated JVM Max heap size would adjust to 2359296kb, which would exceed the total available physical memory of 1780736kb. Rejecting memory request.

Resolution:

This message indicates that the Agent Manager does not have sufficient heap memory to allocate to the requesting Foglight for Active Directory agent package. It is not possible to directly increase the amount of heap memory available to the Agent Manager, as it uses as much memory as the monitoring host can provide to it before issuing this message. The amount of memory available to be allocated to the Agent Manager must be increased, for example by adding more physical memory to the host. If the monitoring host is a virtual machine, more memory may be allocated to the VM.  

If this is not possible, consider moving some agents, or the Agent Manager and all agents, to another monitoring host which has more memory capacity.

Could not establish a connection to host xxx.xxxx.xxx

Symptoms:

1. The following exception message may be found in the Active Directory agent log.
   2013-12-19 17:57:56.129 ECHO    <ActiveDirectory/5.6.6/ActiveDirectory/ad0-dc7.domain7.local-agent INFO> [Thread-33] com.quest.agent.ad.ActiveDirectoryAgent - Validate credentials for host: dc7.domain7.local
2013-12-19 17:57:56.130 ECHO    <ActiveDirectory/5.6.6/ActiveDirectory/ad0-dc7.domain7.local-agent> ERROR [Thread-33] com.quest.agent.ad.ActiveDirectoryAgent - Could not establish a connection to host : dc7.domain7.local.
2013-12-19 17:57:56.130 ECHO    <ActiveDirectory/5.6.6/ActiveDirectory/ad0-dc7.domain7.local-agent> ERROR [Thread-33] com.quest.agent.ad.ActiveDirectoryAgent - Data collection failure.
com.quest.glue.api.services.NoCredentialsException: Could not establish a connection to host : dc7.domain7.local
at com.quest.agent.ad.ActiveDirectoryAgent.buildConfigOnCredential(ActiveDirectoryAgent.java:1290)
at com.quest.agent.ad.ActiveDirectoryAgent.access$000(ActiveDirectoryAgent.java:128)
at com.quest.agent.ad.ActiveDirectoryAgent$1.run(ActiveDirectoryAgent.java:1262)
at java.lang.Thread.run(Thread.java:662) 
2. In Credentials > Manage Credentials, the following alarm may be found: "A Credential with purpose xxxx has been encrypted with a lockbox that has not been granted to this Agent Manager".

Resolution:

1. Ensure that the lockbox has been released to the related Agent Manager (check credential clients in the Credentials > Manage Lockboxes dashboard).
2. If the Agent Manager is in the credential client list, it must be restarted to fix this issue.

Query credential timeout

Symptom:

The following exception message may be found in the Active Directory agent log.

2013-12-19 18:00:02.317 ECHO    <ActiveDirectory/5.6.6/ActiveDirectory/ad0-dc7.domain7.local-agent> ERROR [Thread-35] com.quest.agent.ad.ActiveDirectoryAgent - Data collection failure.
java.util.concurrent.TimeoutException: Time out when query AD / EXC credentials.
at com.quest.agent.service.auth.impl.CredentialQueryResultImpl.get(CredentialQueryResultImpl.java:54)
at com.quest.agent.service.auth.impl.CredentialManagerImpl.queryCredential(CredentialManagerImpl.java:56)
at com.quest.agent.ad.ActiveDirectoryAgent.buildConfigOnCredential(ActiveDirectoryAgent.java:1285)
at com.quest.agent.ad.ActiveDirectoryAgent.access$000(ActiveDirectoryAgent.java:128)
at com.quest.agent.ad.ActiveDirectoryAgent$1.run(ActiveDirectoryAgent.java:1262)
at java.lang.Thread.run(Thread.java:662)

Resolution: Re-start the data collection.

Back to Top