-
Title
After upgrading the Foglight Agent Manager to 6.1.0 the FglAM will not stay connected to the FMS -
Description
After Foglight Management Server (FMS) and Foglight Agent Managers (FglAM) have been upgraded to 6.1.0, the Windows OS FglAM will not remain connected to the FMS.
CPU processing on the FglAM increases to 100% and both powershell.exe and conhost.exe processes are found to be accumulating in the Windows Task Manager.
The number of processes is proportional to the number of database agents and hostagents using a Windows_Default_Account or "Local Account for Monitoring Windows OS" to connect to the remote hosts.
Errors like the following appear in the FglAM log file
ERROR [FglAM:RestoreAgent[21]-53] com.quest.glue.core.remoteconnection.windows.winrm.GMSAHelper - PowerShell Stderr: [A connection to the directory on which to process the request was unavailable. This is likely a transient condition.Get-ADServiceAccount : A connection to the directory on which to process the request was unavailable. This is likely a transient condition.At line:1 char:9+ $gmsa = Get-ADServiceAccount -Identity
-
Cause
FglAM is being overwhelmed with powershell commands from GMSA change in 6.1.0 -
Resolution
WORKAROUND 1 - (FOR SYSTEMS USING THE 6.1.0.1 FGLAM VERSION)
Users can add the following option to the FglAM baseline.jvmargs.config file to prevent it from performing any lookup of the gMSA password, so it won't run the PowerShell script at all. Modify the file baseline.jvmargs.config file (%FglAM_HOME%\state\default\config\baseline.jvmargs.config) with the needed format for each parameter where X represents the next highest number from the previous vmparameter lines.
Example:
vmparameter.X = "-Dfglam.disable_gmsa_lookup=true";Next make sure there is no typo or invalid character which can break the configuration file.Once the settings have been corrected restart the agent manager.
WORKAROUND 2
- Navigate to Administration | Agents | Agent Status
- Select all of the database agents that share the same database agents that use Windows_Default_Account to connect to the hosts.
- Edit the properties for the agents
- Change the connection type from Windows_Default_Account to Windows_Custom_Account or SQL Server Authentication account
- Add a named domain user or SQL Server user and a password
- Save the changes
- Restart the database agents
- Navigate to the Global View Databases Dashboard
- Select all of the SQL Server agents that use the FglAM account to connect to the remote host Windows operating systems
- Select Settings | Administration
- Use the Connection Details panel to update the database agents to use a named Windows credential to connect to the hosts instead of the FglAM account
- Verify and Save the connection
- Navigate to Administration | Credentials | Manage Credentials
- Use the Reorder button to move all credentials using "Local Account for MonitoringWindowsOS" to the bottom of the list (the higher relative order number)
WORKAROUND 3
Use the Windows Scheduler to run a batch script that includes the following two lines. The minimum duration between runs in Windows Scheduler is one minute.
TASKKILL /IM POWERSHELL.EXE /fTASKKILL /IM CONHOST.EXE /fSTATUS
This issue has been logged with R&D as defect FOG-3906 / HFIX-349 and has been fixed in the 6.1.0.1 release of the Foglight Agent Manager.
Please download and install the attached Foglight Agent Manager update and navigate to Administration | Agent Managers to upgrade the FglAM to this new release. Users must be logged into Supportlink to download this update from the Attachments area below.
-
Defect ID
FOG-3906 -
Attachments