Could not establish a connection to host xxx.xxxx.xxx

Troubleshooting

This section provides information about problems that you might encounter while monitoring your environment with Foglight for Exchange, and describes the solutions available to troubleshoot these problems.

Foglight for Active Directory and Foglight for Exchange integration

The following domain controller specific metrics are not available in Foglight for Exchange unless an Active Directory agent is monitoring the domain controller:

• Exchange | Environment | Servers | AD Dependencies (LDAP, Replication, Database)
• Exchange | Exchange Explorer | AD Health | Domain Controllers |
  • Replication Queue Length
  • Replication Failures

Symptom: Some domain controller specific metrics do not display in the Foglight for Exchange views.

Resolution: Install Foglight for Active Directory.

Exchange Server discovery feature

Foglight for Exchange now detects when an Exchange server is added or removed. Alarms are generated for the following cases:

• A new Exchange 2007/2010 server is detected and there is no agent monitoring it.
• A new Exchange server is detected, but the Exchange version on this server is not supported for monitoring.
• An existing Exchange server is removed and an associated agent still exists.

Symptom: Alarms are not being generated when an Exchange server is added or removed.

Resolution:

There are two rules used for the Exchange Server Discovery feature. Disabling either one of these rules will disable alerting on server discovery. Ensure that the following rules are not disabled:

• EXC Server Discovery Search
• EXC Server Discovery Alert

The EXC Server Discovery Search rule fires every 24 hours and an LDAP query is made once for every domain that has an active, collecting agent. Therefore, depending on when the server was added or removed, there may be a delay in seeing the alarm. Also, if the agent is deactivated or not collecting data, the new or removed server will not be detected until the next server discovery search interval after the agent is re-activated and collecting data.

RPCs Failed (Server Too Busy) performance metric

The RPCs Failed (Server Too Busy) performance metric is a client-reported value. In order to send this type of data to the server in Outlook 2003 or later, the Exchange server’s registry must contain the ClientMonitoringReportLevel registry key with a value of either one or two.

Symptom: RPCs Failed (Server Too Busy) performance metric is not being collected.

Resolution:

Ensure that the server’s registry contains the ClientMonitoringReportLevel registry key with a value of either one or two.

• In Exchange 2007, the default behavior is to collect performance data only from Outlook clients that have high-speed network connectivity. (Functionally the same as when the value of the ClientMonitoringReportLevel registry key is set to one.)
• For clients that are using a low-bandwidth connection, set the value of the ClientMonitoringReportLevel to two.

To modify the client-side monitoring levels for Outlook 2003 or later clients:

Tip: It is recommended that you create a backup copy of the Registry that you can revert to prior to making any changes.

1. On the Exchange server that contains the client mailboxes to be monitored, run: regedit.
2. If you are asked to allow the Regedit program to make changes to the computer, click Yes.
3. Navigate to the following registry key:
   HKLM\System\CurrentControlSet\Services\MSExchangeIS\ParametersSystem
4. Right-click ParametersSystem and click New | DWORD Value.
5. Name the new DWORD value ClientMonitoringReportLevel.
6. Double-click ClientMonitoringReportLevel.
7. In the Value data field, enter the appropriate value:
   0 = do not collect data from any Outlook 2003 and later clients
   1 = collect performance data only from high-bandwidth Outlook 2003 and later clients (default)
   2 = collect performance data from all Outlook 2003 or later clients
8. Close the registry editor.
   The Exchange Information Store service automatically detects the changes. You do not need to restart the computer or any services.

Monitoring Microsoft Exchange Monitoring service

The Microsoft Exchange Monitoring service is not monitored and alarms will not be raised for this service by default. However, if you use this service in your Exchange organization, you can enable monitoring.

Symptom: Microsoft Exchange Monitoring service is not being monitored.

Resolution: Enable monitoring of this service:

1.  Navigate to Dashboards > Administration > Agents > Agent Status.
2. Under Monitor, select a Monitored Service and click Edit.
3. Click Add Row in the ExchangeMonitoring - ExchangeAgent - monitoredServicesList table.
4. Enter the Server Role and the Service Name for the service to be monitored. All entries are case sensitive.
   win2k3\MicrosoftExchangeMonitoring
   or
   win2k8\MicrosoftExchangeMonitoring
5. Click Save Changes.

Recommended best practices

The following procedure is a best practice that is recommended for optimal performance.

Disable automatic updates on Foglight Management Server

Do NOT allow Microsoft® automatic update feature to force an update of the server hosting the Foglight Management Server. This automatic update feature does not allow enough time for the Foglight Management Server to shutdown gracefully, which may leave your agents in a broken state.

Symptom: Cartridge agents will appear to be deactivated on the Agent Status dashboard.

Resolution: Using the Agent Status dashboard, select the deactivated agent and select the Activate button. If you cannot activate the selected agent, delete and reinstall the agent.

Potential issues after upgrading the cartridge to version 6.3.0

Insufficient heap memory

Symptoms:

When upgrading to version 6.3.0, you encounter an error message similar to the following message (actual values may vary):

Error deploying package … Cause: The addition of 2097152kb to the negotiated JVM Max heap size would adjust to 2359296kb, which would exceed the total available physical memory of 1780736kb. Rejecting memory request.

Resolution:

This message indicates that the Agent Manager does not have sufficient heap memory to allocate to the requesting Foglight for Exchange agent package. It is not possible to directly increase the amount of heap memory available to the Agent Manager, as it uses as much memory as the monitoring host can provide to it before issuing this message. The amount of memory available to be allocated to the Agent Manager must be increased, for example by adding more physical memory to the host. If the monitoring host is a virtual machine, more memory may be allocated to the VM.  

If this is not possible, consider moving some agents, or the Agent Manager and all agents, to another monitoring host which has more memory capacity.

Symptoms:

1. The following exception message may be found in the Exchange agent log.
   2013-12-19 13:39:12.669 ECHO    <ExchangeMonitoring/5.6.6/ExchangeAgent/EXC0-EX7.domain7.local-agent> INFO [Thread-20] com.quest.agent.service.auth.impl.CredentialManagerImpl - Begin to query credential for host:  EX7.domain7.local
2013-12-19 13:39:26.707 ECHO    <ExchangeMonitoring/5.6.6/ExchangeAgent/EXC0-EX7.domain7.local-agent> INFO [Thread-20] com.quest.agent.exc.ExchangeAgentImpl - Validate credentials for host: EX7.domain7.local
2013-12-19 13:39:26.708 ECHO    <ExchangeMonitoring/5.6.6/ExchangeAgent/EXC0-EX7.domain7.local-agent> ERROR [Thread-20] com.quest.agent.exc.ExchangeAgentImpl - Could not establish a connection to host : EX7.domain7.local.
2013-12-19 13:39:26.708 ECHO    <ExchangeMonitoring/5.6.6/ExchangeAgent/EXC0-EX7.domain7.local-agent> ERROR [Thread-20] com.quest.agent.exc.ExchangeAgentImpl - Data collection failure.
com.quest.glue.api.services.NoCredentialsException: Could not establish a connection to host : EX7.domain7.local      
at com.quest.agent.exc.ExchangeAgentImpl.buidConfig(ExchangeAgentImpl.java:815)      
at com.quest.agent.exc.ExchangeAgentImpl.buildConfigOnCredential(ExchangeAgentImpl.java:791)      
at com.quest.agent.exc.ExchangeAgentImpl.access$000(ExchangeAgentImpl.java:84)      
at com.quest.agent.exc.ExchangeAgentImpl$1.run(ExchangeAgentImpl.java:839)      
at java.lang.Thread.run(Thread.java:662)  
2. In Administrator > Credentials > Manage Credentials, the following alarm may be found: "A Credential with purpose xxxx has been encrypted with a lockbox that has not been granted to this Agent Manager".

Resolution 1:

1. Ensure that the lockbox has been released to the related Agent Manager (check credential clients in the Credentials > Manage Lockboxes dashboard).
2. If the Agent Manager is in the credential client list, it must be restarted to fix this issue.

Resolution 2: Update the Agent Manager to version 5.6.12 (or later).

Data merge error found in Foglight Management Server console

Symptom:

The following error message may be found in the Foglight Management Server console.

Failed to retain value of property instances when editing EXCADAccessDomainController object "null (EXCADAccessDomainController)" (39bb11e5-e952-4d63-8629-c4efc19a546d).
Failed to retain value of property instances when editing EXCADAccessCache object "null (EXCADAccessCache)" (16d56083-19b0-4370-af54-9b775a7f644e).
Failed to retain value of property instances when editing EXCADAccessProcessobject "null (EXCADAccessProcess)" (36b2c281-13b6-48ee-9dc0-7660e326fd50).
Failed to retain value of property instances when editing EXCDatabase object "null (EXCADAccessProcess)" (36b2c281-13b6-48ee-9dc0-7660e326fd50).

Resolution:

1. Stop the data collection.
2. Run the following groovy script in the script console, to remove old topology objects: EXCADAccessDomainController, EXCADAccessCache, EXCADAccessProcess, and EXCDatabase.
   server["TopologyService"].deleteObjects(new java.util.HashSet(#!EXCADAccessDomainController#.topologyObjects)) 
server["TopologyService"].deleteObjects(new java.util.HashSet(#!EXCADAccessCache#.topologyObjects)) 
server["TopologyService"].deleteObjects(new java.util.HashSet(#!EXCADAccessProcess #.topologyObjects)) 
server["TopologyService"].deleteObjects(new java.util.HashSet(#!EXCDatabase#.topologyObjects))  

Cannot start the Exchange agent to monitor Exchange Edge Transport server

Symptoms:

1. The monitored Exchange Edge Transport server is a stand alone host.
2. In agent properties, communication Protocol is set as "WinRm Through HTTP" or "Winrm Through HTTPS".
3.  The following exception message may be found in the Exchange agent log.
   2014-01-26 10:51:47.329 ECHO    <ExchangeMonitoring/5.6.7/ExchangeAgent/2901-agent> ERROR [Quartz[0]-10] com.quest.agent.service.winRm.WinRMEndPoint - Fail to establish the WinRM connection: com.quest.glue.api.services.RemoteConnectionException: a connection could not be established.
2014-01-26 10:51:47.329 ECHO    <ExchangeMonitoring/5.6.7/ExchangeAgent/2901-agent> INFO [Quartz[0]-10] com.quest.agent.service.auth.impl.WinRmValidator - winRm connectivity test result: Failed.
2014-01-26 10:51:47.330 ECHO    <ExchangeMonitoring/5.6.7/ExchangeAgent/2901-agent> ERROR [Quartz[0]-10] com.quest.agent.exc.ExchangeAgentImpl - Could not establish a connection to host : zhuvmfog2901. 2014-01-26 10:51:47.332 ECHO    <ExchangeMonitoring/5.6.7/ExchangeAgent/2901-agent> EERROR [Quartz[0]-10] com.quest.agent.exc.ExchangeAgentImpl - Data collection failure.
com.quest.glue.api.services.NoCredentialsException: Could not establish a connection to host : XXXXXX                        
at com.quest.agent.exc.ExchangeAgentImpl.buidConfig(ExchangeAgentImpl.java:718)                        
at com.quest.agent.exc.ExchangeAgentImpl.buildConfigOnCredential(ExchangeAgentImpl.java:701)                        
at com.quest.agent.exc.ExchangeAgentImpl.init(ExchangeAgentImpl.java:866)                        
at com.quest.agent.exc.ExchangeAgentImpl.isReady(ExchangeAgentImpl.java:741)                        
at com.quest.agent.exc.ExchangeAgentImpl.informationStoreDetailCollection(ExchangeAgentImpl.java:594)                       
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)                        
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)                        
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)                        
at java.lang.reflect.Method.invoke(Method.java:597)                        
at com.quest.glue.core.services.EquivalenceInvocationHandler.invoke(EquivalenceInvocationHandler.java:70)                        
at com.quest.glue.core.agent.AgentInteractionHandler.invoke(AgentInteractionHandler.java:186)                        
at com.sun.proxy.$Proxy51.informationStoreDetailCollection(Unknown Source)                        
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)                        
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)                        
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)                        
at java.lang.reflect.Method.invoke(Method.java:597)                        
at com.quest.glue.core.agent.scheduler.CollectorCallback.invokeCollector(CollectorCallback.java:162)                        
at com.quest.glue.core.agent.scheduler.CollectorCallback.execute(CollectorCallback.java:130)                        
at com.quest.glue.core.scheduler.quartz.QuartzScheduler$ScheduledTaskSequentialJob.execute(QuartzScheduler.java:716)                        
at org.quartz.core.JobRunShell.run(JobRunShell.java:202)                        
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)                        
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)                        
at java.lang.Thread.run(Thread.java:662)

Resolution:

• Check the credential setting in Exchange Agent properties and ensure that the user is in local account format.

Back to Top

Data merge error found in Foglight Management Server console

Troubleshooting

This section provides information about problems that you might encounter while monitoring your environment with Foglight for Exchange, and describes the solutions available to troubleshoot these problems.

Foglight for Active Directory and Foglight for Exchange integration

The following domain controller specific metrics are not available in Foglight for Exchange unless an Active Directory agent is monitoring the domain controller:

• Exchange | Environment | Servers | AD Dependencies (LDAP, Replication, Database)
• Exchange | Exchange Explorer | AD Health | Domain Controllers |
  • Replication Queue Length
  • Replication Failures

Symptom: Some domain controller specific metrics do not display in the Foglight for Exchange views.

Resolution: Install Foglight for Active Directory.

Exchange Server discovery feature

Foglight for Exchange now detects when an Exchange server is added or removed. Alarms are generated for the following cases:

• A new Exchange 2007/2010 server is detected and there is no agent monitoring it.
• A new Exchange server is detected, but the Exchange version on this server is not supported for monitoring.
• An existing Exchange server is removed and an associated agent still exists.

Symptom: Alarms are not being generated when an Exchange server is added or removed.

Resolution:

There are two rules used for the Exchange Server Discovery feature. Disabling either one of these rules will disable alerting on server discovery. Ensure that the following rules are not disabled:

• EXC Server Discovery Search
• EXC Server Discovery Alert

The EXC Server Discovery Search rule fires every 24 hours and an LDAP query is made once for every domain that has an active, collecting agent. Therefore, depending on when the server was added or removed, there may be a delay in seeing the alarm. Also, if the agent is deactivated or not collecting data, the new or removed server will not be detected until the next server discovery search interval after the agent is re-activated and collecting data.

RPCs Failed (Server Too Busy) performance metric

The RPCs Failed (Server Too Busy) performance metric is a client-reported value. In order to send this type of data to the server in Outlook 2003 or later, the Exchange server’s registry must contain the ClientMonitoringReportLevel registry key with a value of either one or two.

Symptom: RPCs Failed (Server Too Busy) performance metric is not being collected.

Resolution:

Ensure that the server’s registry contains the ClientMonitoringReportLevel registry key with a value of either one or two.

• In Exchange 2007, the default behavior is to collect performance data only from Outlook clients that have high-speed network connectivity. (Functionally the same as when the value of the ClientMonitoringReportLevel registry key is set to one.)
• For clients that are using a low-bandwidth connection, set the value of the ClientMonitoringReportLevel to two.

To modify the client-side monitoring levels for Outlook 2003 or later clients:

Tip: It is recommended that you create a backup copy of the Registry that you can revert to prior to making any changes.

1. On the Exchange server that contains the client mailboxes to be monitored, run: regedit.
2. If you are asked to allow the Regedit program to make changes to the computer, click Yes.
3. Navigate to the following registry key:
   HKLM\System\CurrentControlSet\Services\MSExchangeIS\ParametersSystem
4. Right-click ParametersSystem and click New | DWORD Value.
5. Name the new DWORD value ClientMonitoringReportLevel.
6. Double-click ClientMonitoringReportLevel.
7. In the Value data field, enter the appropriate value:
   0 = do not collect data from any Outlook 2003 and later clients
   1 = collect performance data only from high-bandwidth Outlook 2003 and later clients (default)
   2 = collect performance data from all Outlook 2003 or later clients
8. Close the registry editor.
   The Exchange Information Store service automatically detects the changes. You do not need to restart the computer or any services.

Monitoring Microsoft Exchange Monitoring service

The Microsoft Exchange Monitoring service is not monitored and alarms will not be raised for this service by default. However, if you use this service in your Exchange organization, you can enable monitoring.

Symptom: Microsoft Exchange Monitoring service is not being monitored.

Resolution: Enable monitoring of this service:

1.  Navigate to Dashboards > Administration > Agents > Agent Status.
2. Under Monitor, select a Monitored Service and click Edit.
3. Click Add Row in the ExchangeMonitoring - ExchangeAgent - monitoredServicesList table.
4. Enter the Server Role and the Service Name for the service to be monitored. All entries are case sensitive.
   win2k3\MicrosoftExchangeMonitoring
   or
   win2k8\MicrosoftExchangeMonitoring
5. Click Save Changes.

Recommended best practices

The following procedure is a best practice that is recommended for optimal performance.

Disable automatic updates on Foglight Management Server

Do NOT allow Microsoft® automatic update feature to force an update of the server hosting the Foglight Management Server. This automatic update feature does not allow enough time for the Foglight Management Server to shutdown gracefully, which may leave your agents in a broken state.

Symptom: Cartridge agents will appear to be deactivated on the Agent Status dashboard.

Resolution: Using the Agent Status dashboard, select the deactivated agent and select the Activate button. If you cannot activate the selected agent, delete and reinstall the agent.

Potential issues after upgrading the cartridge to version 6.3.0

Insufficient heap memory

Symptoms:

When upgrading to version 6.3.0, you encounter an error message similar to the following message (actual values may vary):

Error deploying package … Cause: The addition of 2097152kb to the negotiated JVM Max heap size would adjust to 2359296kb, which would exceed the total available physical memory of 1780736kb. Rejecting memory request.

Resolution:

This message indicates that the Agent Manager does not have sufficient heap memory to allocate to the requesting Foglight for Exchange agent package. It is not possible to directly increase the amount of heap memory available to the Agent Manager, as it uses as much memory as the monitoring host can provide to it before issuing this message. The amount of memory available to be allocated to the Agent Manager must be increased, for example by adding more physical memory to the host. If the monitoring host is a virtual machine, more memory may be allocated to the VM.  

If this is not possible, consider moving some agents, or the Agent Manager and all agents, to another monitoring host which has more memory capacity.

Could not establish a connection to host xxx.xxxx.xxx

Symptoms:

1. The following exception message may be found in the Exchange agent log.
   2013-12-19 13:39:12.669 ECHO    <ExchangeMonitoring/5.6.6/ExchangeAgent/EXC0-EX7.domain7.local-agent> INFO [Thread-20] com.quest.agent.service.auth.impl.CredentialManagerImpl - Begin to query credential for host:  EX7.domain7.local
2013-12-19 13:39:26.707 ECHO    <ExchangeMonitoring/5.6.6/ExchangeAgent/EXC0-EX7.domain7.local-agent> INFO [Thread-20] com.quest.agent.exc.ExchangeAgentImpl - Validate credentials for host: EX7.domain7.local
2013-12-19 13:39:26.708 ECHO    <ExchangeMonitoring/5.6.6/ExchangeAgent/EXC0-EX7.domain7.local-agent> ERROR [Thread-20] com.quest.agent.exc.ExchangeAgentImpl - Could not establish a connection to host : EX7.domain7.local.
2013-12-19 13:39:26.708 ECHO    <ExchangeMonitoring/5.6.6/ExchangeAgent/EXC0-EX7.domain7.local-agent> ERROR [Thread-20] com.quest.agent.exc.ExchangeAgentImpl - Data collection failure.
com.quest.glue.api.services.NoCredentialsException: Could not establish a connection to host : EX7.domain7.local      
at com.quest.agent.exc.ExchangeAgentImpl.buidConfig(ExchangeAgentImpl.java:815)      
at com.quest.agent.exc.ExchangeAgentImpl.buildConfigOnCredential(ExchangeAgentImpl.java:791)      
at com.quest.agent.exc.ExchangeAgentImpl.access$000(ExchangeAgentImpl.java:84)      
at com.quest.agent.exc.ExchangeAgentImpl$1.run(ExchangeAgentImpl.java:839)      
at java.lang.Thread.run(Thread.java:662)  
2. In Administrator > Credentials > Manage Credentials, the following alarm may be found: "A Credential with purpose xxxx has been encrypted with a lockbox that has not been granted to this Agent Manager".

Resolution 1:

1. Ensure that the lockbox has been released to the related Agent Manager (check credential clients in the Credentials > Manage Lockboxes dashboard).
2. If the Agent Manager is in the credential client list, it must be restarted to fix this issue.

Resolution 2: Update the Agent Manager to version 5.6.12 (or later).

Data merge error found in Foglight Management Server console

Symptom:

The following error message may be found in the Foglight Management Server console.

Failed to retain value of property instances when editing EXCADAccessDomainController object "null (EXCADAccessDomainController)" (39bb11e5-e952-4d63-8629-c4efc19a546d).
Failed to retain value of property instances when editing EXCADAccessCache object "null (EXCADAccessCache)" (16d56083-19b0-4370-af54-9b775a7f644e).
Failed to retain value of property instances when editing EXCADAccessProcessobject "null (EXCADAccessProcess)" (36b2c281-13b6-48ee-9dc0-7660e326fd50).
Failed to retain value of property instances when editing EXCDatabase object "null (EXCADAccessProcess)" (36b2c281-13b6-48ee-9dc0-7660e326fd50).

Resolution:

1. Stop the data collection.
2. Run the following groovy script in the script console, to remove old topology objects: EXCADAccessDomainController, EXCADAccessCache, EXCADAccessProcess, and EXCDatabase.
   server["TopologyService"].deleteObjects(new java.util.HashSet(#!EXCADAccessDomainController#.topologyObjects)) 
server["TopologyService"].deleteObjects(new java.util.HashSet(#!EXCADAccessCache#.topologyObjects)) 
server["TopologyService"].deleteObjects(new java.util.HashSet(#!EXCADAccessProcess #.topologyObjects)) 
server["TopologyService"].deleteObjects(new java.util.HashSet(#!EXCDatabase#.topologyObjects))  

Cannot start the Exchange agent to monitor Exchange Edge Transport server

Symptoms:

1. The monitored Exchange Edge Transport server is a stand alone host.
2. In agent properties, communication Protocol is set as "WinRm Through HTTP" or "Winrm Through HTTPS".
3.  The following exception message may be found in the Exchange agent log.
   2014-01-26 10:51:47.329 ECHO    <ExchangeMonitoring/5.6.7/ExchangeAgent/2901-agent> ERROR [Quartz[0]-10] com.quest.agent.service.winRm.WinRMEndPoint - Fail to establish the WinRM connection: com.quest.glue.api.services.RemoteConnectionException: a connection could not be established.
2014-01-26 10:51:47.329 ECHO    <ExchangeMonitoring/5.6.7/ExchangeAgent/2901-agent> INFO [Quartz[0]-10] com.quest.agent.service.auth.impl.WinRmValidator - winRm connectivity test result: Failed.
2014-01-26 10:51:47.330 ECHO    <ExchangeMonitoring/5.6.7/ExchangeAgent/2901-agent> ERROR [Quartz[0]-10] com.quest.agent.exc.ExchangeAgentImpl - Could not establish a connection to host : zhuvmfog2901. 2014-01-26 10:51:47.332 ECHO    <ExchangeMonitoring/5.6.7/ExchangeAgent/2901-agent> EERROR [Quartz[0]-10] com.quest.agent.exc.ExchangeAgentImpl - Data collection failure.
com.quest.glue.api.services.NoCredentialsException: Could not establish a connection to host : XXXXXX                        
at com.quest.agent.exc.ExchangeAgentImpl.buidConfig(ExchangeAgentImpl.java:718)                        
at com.quest.agent.exc.ExchangeAgentImpl.buildConfigOnCredential(ExchangeAgentImpl.java:701)                        
at com.quest.agent.exc.ExchangeAgentImpl.init(ExchangeAgentImpl.java:866)                        
at com.quest.agent.exc.ExchangeAgentImpl.isReady(ExchangeAgentImpl.java:741)                        
at com.quest.agent.exc.ExchangeAgentImpl.informationStoreDetailCollection(ExchangeAgentImpl.java:594)                       
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)                        
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)                        
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)                        
at java.lang.reflect.Method.invoke(Method.java:597)                        
at com.quest.glue.core.services.EquivalenceInvocationHandler.invoke(EquivalenceInvocationHandler.java:70)                        
at com.quest.glue.core.agent.AgentInteractionHandler.invoke(AgentInteractionHandler.java:186)                        
at com.sun.proxy.$Proxy51.informationStoreDetailCollection(Unknown Source)                        
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)                        
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)                        
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)                        
at java.lang.reflect.Method.invoke(Method.java:597)                        
at com.quest.glue.core.agent.scheduler.CollectorCallback.invokeCollector(CollectorCallback.java:162)                        
at com.quest.glue.core.agent.scheduler.CollectorCallback.execute(CollectorCallback.java:130)                        
at com.quest.glue.core.scheduler.quartz.QuartzScheduler$ScheduledTaskSequentialJob.execute(QuartzScheduler.java:716)                        
at org.quartz.core.JobRunShell.run(JobRunShell.java:202)                        
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)                        
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)                        
at java.lang.Thread.run(Thread.java:662)

Resolution:

• Check the credential setting in Exchange Agent properties and ensure that the user is in local account format.

Back to Top

Cannot start the Exchange agent to monitor Exchange Edge Transport server

Troubleshooting

This section provides information about problems that you might encounter while monitoring your environment with Foglight for Exchange, and describes the solutions available to troubleshoot these problems.

Foglight for Active Directory and Foglight for Exchange integration

The following domain controller specific metrics are not available in Foglight for Exchange unless an Active Directory agent is monitoring the domain controller:

• Exchange | Environment | Servers | AD Dependencies (LDAP, Replication, Database)
• Exchange | Exchange Explorer | AD Health | Domain Controllers |
  • Replication Queue Length
  • Replication Failures

Symptom: Some domain controller specific metrics do not display in the Foglight for Exchange views.

Resolution: Install Foglight for Active Directory.

Exchange Server discovery feature

Foglight for Exchange now detects when an Exchange server is added or removed. Alarms are generated for the following cases:

• A new Exchange 2007/2010 server is detected and there is no agent monitoring it.
• A new Exchange server is detected, but the Exchange version on this server is not supported for monitoring.
• An existing Exchange server is removed and an associated agent still exists.

Symptom: Alarms are not being generated when an Exchange server is added or removed.

Resolution:

There are two rules used for the Exchange Server Discovery feature. Disabling either one of these rules will disable alerting on server discovery. Ensure that the following rules are not disabled:

• EXC Server Discovery Search
• EXC Server Discovery Alert

The EXC Server Discovery Search rule fires every 24 hours and an LDAP query is made once for every domain that has an active, collecting agent. Therefore, depending on when the server was added or removed, there may be a delay in seeing the alarm. Also, if the agent is deactivated or not collecting data, the new or removed server will not be detected until the next server discovery search interval after the agent is re-activated and collecting data.

RPCs Failed (Server Too Busy) performance metric

The RPCs Failed (Server Too Busy) performance metric is a client-reported value. In order to send this type of data to the server in Outlook 2003 or later, the Exchange server’s registry must contain the ClientMonitoringReportLevel registry key with a value of either one or two.

Symptom: RPCs Failed (Server Too Busy) performance metric is not being collected.

Resolution:

Ensure that the server’s registry contains the ClientMonitoringReportLevel registry key with a value of either one or two.

• In Exchange 2007, the default behavior is to collect performance data only from Outlook clients that have high-speed network connectivity. (Functionally the same as when the value of the ClientMonitoringReportLevel registry key is set to one.)
• For clients that are using a low-bandwidth connection, set the value of the ClientMonitoringReportLevel to two.

To modify the client-side monitoring levels for Outlook 2003 or later clients:

Tip: It is recommended that you create a backup copy of the Registry that you can revert to prior to making any changes.

1. On the Exchange server that contains the client mailboxes to be monitored, run: regedit.
2. If you are asked to allow the Regedit program to make changes to the computer, click Yes.
3. Navigate to the following registry key:
   HKLM\System\CurrentControlSet\Services\MSExchangeIS\ParametersSystem
4. Right-click ParametersSystem and click New | DWORD Value.
5. Name the new DWORD value ClientMonitoringReportLevel.
6. Double-click ClientMonitoringReportLevel.
7. In the Value data field, enter the appropriate value:
   0 = do not collect data from any Outlook 2003 and later clients
   1 = collect performance data only from high-bandwidth Outlook 2003 and later clients (default)
   2 = collect performance data from all Outlook 2003 or later clients
8. Close the registry editor.
   The Exchange Information Store service automatically detects the changes. You do not need to restart the computer or any services.

Monitoring Microsoft Exchange Monitoring service

The Microsoft Exchange Monitoring service is not monitored and alarms will not be raised for this service by default. However, if you use this service in your Exchange organization, you can enable monitoring.

Symptom: Microsoft Exchange Monitoring service is not being monitored.

Resolution: Enable monitoring of this service:

1.  Navigate to Dashboards > Administration > Agents > Agent Status.
2. Under Monitor, select a Monitored Service and click Edit.
3. Click Add Row in the ExchangeMonitoring - ExchangeAgent - monitoredServicesList table.
4. Enter the Server Role and the Service Name for the service to be monitored. All entries are case sensitive.
   win2k3\MicrosoftExchangeMonitoring
   or
   win2k8\MicrosoftExchangeMonitoring
5. Click Save Changes.

Recommended best practices

The following procedure is a best practice that is recommended for optimal performance.

Disable automatic updates on Foglight Management Server

Do NOT allow Microsoft® automatic update feature to force an update of the server hosting the Foglight Management Server. This automatic update feature does not allow enough time for the Foglight Management Server to shutdown gracefully, which may leave your agents in a broken state.

Symptom: Cartridge agents will appear to be deactivated on the Agent Status dashboard.

Resolution: Using the Agent Status dashboard, select the deactivated agent and select the Activate button. If you cannot activate the selected agent, delete and reinstall the agent.

Potential issues after upgrading the cartridge to version 6.3.0

Insufficient heap memory

Symptoms:

When upgrading to version 6.3.0, you encounter an error message similar to the following message (actual values may vary):

Error deploying package … Cause: The addition of 2097152kb to the negotiated JVM Max heap size would adjust to 2359296kb, which would exceed the total available physical memory of 1780736kb. Rejecting memory request.

Resolution:

This message indicates that the Agent Manager does not have sufficient heap memory to allocate to the requesting Foglight for Exchange agent package. It is not possible to directly increase the amount of heap memory available to the Agent Manager, as it uses as much memory as the monitoring host can provide to it before issuing this message. The amount of memory available to be allocated to the Agent Manager must be increased, for example by adding more physical memory to the host. If the monitoring host is a virtual machine, more memory may be allocated to the VM.  

If this is not possible, consider moving some agents, or the Agent Manager and all agents, to another monitoring host which has more memory capacity.

Could not establish a connection to host xxx.xxxx.xxx

Symptoms:

1. The following exception message may be found in the Exchange agent log.
   2013-12-19 13:39:12.669 ECHO    <ExchangeMonitoring/5.6.6/ExchangeAgent/EXC0-EX7.domain7.local-agent> INFO [Thread-20] com.quest.agent.service.auth.impl.CredentialManagerImpl - Begin to query credential for host:  EX7.domain7.local
2013-12-19 13:39:26.707 ECHO    <ExchangeMonitoring/5.6.6/ExchangeAgent/EXC0-EX7.domain7.local-agent> INFO [Thread-20] com.quest.agent.exc.ExchangeAgentImpl - Validate credentials for host: EX7.domain7.local
2013-12-19 13:39:26.708 ECHO    <ExchangeMonitoring/5.6.6/ExchangeAgent/EXC0-EX7.domain7.local-agent> ERROR [Thread-20] com.quest.agent.exc.ExchangeAgentImpl - Could not establish a connection to host : EX7.domain7.local.
2013-12-19 13:39:26.708 ECHO    <ExchangeMonitoring/5.6.6/ExchangeAgent/EXC0-EX7.domain7.local-agent> ERROR [Thread-20] com.quest.agent.exc.ExchangeAgentImpl - Data collection failure.
com.quest.glue.api.services.NoCredentialsException: Could not establish a connection to host : EX7.domain7.local      
at com.quest.agent.exc.ExchangeAgentImpl.buidConfig(ExchangeAgentImpl.java:815)      
at com.quest.agent.exc.ExchangeAgentImpl.buildConfigOnCredential(ExchangeAgentImpl.java:791)      
at com.quest.agent.exc.ExchangeAgentImpl.access$000(ExchangeAgentImpl.java:84)      
at com.quest.agent.exc.ExchangeAgentImpl$1.run(ExchangeAgentImpl.java:839)      
at java.lang.Thread.run(Thread.java:662)  
2. In Administrator > Credentials > Manage Credentials, the following alarm may be found: "A Credential with purpose xxxx has been encrypted with a lockbox that has not been granted to this Agent Manager".

Resolution 1:

1. Ensure that the lockbox has been released to the related Agent Manager (check credential clients in the Credentials > Manage Lockboxes dashboard).
2. If the Agent Manager is in the credential client list, it must be restarted to fix this issue.

Resolution 2: Update the Agent Manager to version 5.6.12 (or later).

Data merge error found in Foglight Management Server console

Symptom:

The following error message may be found in the Foglight Management Server console.

Failed to retain value of property instances when editing EXCADAccessDomainController object "null (EXCADAccessDomainController)" (39bb11e5-e952-4d63-8629-c4efc19a546d).
Failed to retain value of property instances when editing EXCADAccessCache object "null (EXCADAccessCache)" (16d56083-19b0-4370-af54-9b775a7f644e).
Failed to retain value of property instances when editing EXCADAccessProcessobject "null (EXCADAccessProcess)" (36b2c281-13b6-48ee-9dc0-7660e326fd50).
Failed to retain value of property instances when editing EXCDatabase object "null (EXCADAccessProcess)" (36b2c281-13b6-48ee-9dc0-7660e326fd50).

Resolution:

1. Stop the data collection.
2. Run the following groovy script in the script console, to remove old topology objects: EXCADAccessDomainController, EXCADAccessCache, EXCADAccessProcess, and EXCDatabase.
   server["TopologyService"].deleteObjects(new java.util.HashSet(#!EXCADAccessDomainController#.topologyObjects)) 
server["TopologyService"].deleteObjects(new java.util.HashSet(#!EXCADAccessCache#.topologyObjects)) 
server["TopologyService"].deleteObjects(new java.util.HashSet(#!EXCADAccessProcess #.topologyObjects)) 
server["TopologyService"].deleteObjects(new java.util.HashSet(#!EXCDatabase#.topologyObjects))  

Cannot start the Exchange agent to monitor Exchange Edge Transport server

Symptoms:

1. The monitored Exchange Edge Transport server is a stand alone host.
2. In agent properties, communication Protocol is set as "WinRm Through HTTP" or "Winrm Through HTTPS".
3.  The following exception message may be found in the Exchange agent log.
   2014-01-26 10:51:47.329 ECHO    <ExchangeMonitoring/5.6.7/ExchangeAgent/2901-agent> ERROR [Quartz[0]-10] com.quest.agent.service.winRm.WinRMEndPoint - Fail to establish the WinRM connection: com.quest.glue.api.services.RemoteConnectionException: a connection could not be established.
2014-01-26 10:51:47.329 ECHO    <ExchangeMonitoring/5.6.7/ExchangeAgent/2901-agent> INFO [Quartz[0]-10] com.quest.agent.service.auth.impl.WinRmValidator - winRm connectivity test result: Failed.
2014-01-26 10:51:47.330 ECHO    <ExchangeMonitoring/5.6.7/ExchangeAgent/2901-agent> ERROR [Quartz[0]-10] com.quest.agent.exc.ExchangeAgentImpl - Could not establish a connection to host : zhuvmfog2901. 2014-01-26 10:51:47.332 ECHO    <ExchangeMonitoring/5.6.7/ExchangeAgent/2901-agent> EERROR [Quartz[0]-10] com.quest.agent.exc.ExchangeAgentImpl - Data collection failure.
com.quest.glue.api.services.NoCredentialsException: Could not establish a connection to host : XXXXXX                        
at com.quest.agent.exc.ExchangeAgentImpl.buidConfig(ExchangeAgentImpl.java:718)                        
at com.quest.agent.exc.ExchangeAgentImpl.buildConfigOnCredential(ExchangeAgentImpl.java:701)                        
at com.quest.agent.exc.ExchangeAgentImpl.init(ExchangeAgentImpl.java:866)                        
at com.quest.agent.exc.ExchangeAgentImpl.isReady(ExchangeAgentImpl.java:741)                        
at com.quest.agent.exc.ExchangeAgentImpl.informationStoreDetailCollection(ExchangeAgentImpl.java:594)                       
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)                        
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)                        
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)                        
at java.lang.reflect.Method.invoke(Method.java:597)                        
at com.quest.glue.core.services.EquivalenceInvocationHandler.invoke(EquivalenceInvocationHandler.java:70)                        
at com.quest.glue.core.agent.AgentInteractionHandler.invoke(AgentInteractionHandler.java:186)                        
at com.sun.proxy.$Proxy51.informationStoreDetailCollection(Unknown Source)                        
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)                        
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)                        
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)                        
at java.lang.reflect.Method.invoke(Method.java:597)                        
at com.quest.glue.core.agent.scheduler.CollectorCallback.invokeCollector(CollectorCallback.java:162)                        
at com.quest.glue.core.agent.scheduler.CollectorCallback.execute(CollectorCallback.java:130)                        
at com.quest.glue.core.scheduler.quartz.QuartzScheduler$ScheduledTaskSequentialJob.execute(QuartzScheduler.java:716)                        
at org.quartz.core.JobRunShell.run(JobRunShell.java:202)                        
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)                        
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)                        
at java.lang.Thread.run(Thread.java:662)

Resolution:

• Check the credential setting in Exchange Agent properties and ensure that the user is in local account format.

Back to Top

Product licensing

Product licensing

Foglight includes a licensing capability that restricts access to those features that are defined in the license. Any Management Server installation requires a license that grants access to server-specific parts of the browser interface and the features associated with them. Foglight cartridges are also license-protected. While some cartridges are covered by the base Foglight license (such as Foglight Agent Manager cartridges and the Cartridge for Infrastructure), others may require an additional license. Foglight for Exchange is covered by the Foglight Evolve Monitor, Operate, and Flex license.

To activate a trial or a purchased commercial license:

1. On the navigation panel, under Dashboards, click Administration > Setup > Manage Licenses.
2. Click Install.
3. In the Install License dialog box, click Browse.
4. In the file browser that appears, specify the location of the license file.
5. In the Install License dialog box, click Install License.

Back to Top