Unable to Protect Rapid Recovery Agent Due to NTLM Restrictions Enforced by Local or Domain Policy (4378979)

Attempts to protect or pair with a Rapid Recovery Agent fail.

In the user interface, the following error is shown:

Service Host Error: Service error while handling request [POST https://localhost:8006/apprecovery/api/core/agents/validateAgentProtectionAbility]: Failed to connect to remote machine using provided credentials. This can be caused by one of the following:
    - Destination host name is incorrect or host is offline.
    - Destination host is behind firewall or similar protection software that prevents connection.
    - If you are trying to connect to Windows machine:
        - WMI service is not started or blocked by firewall.
    - If you are trying to connect to Linux machine:
        - SSH service is not started or blocked by firewall.
        - Provided SSH port is incorrect.

In the Core logs (C:\ProgramData\AppRecovery\Logs\AppRecovery.log), a related internal error appears:

ERROR - Replay.Agent.Client.AgentPairManagementAgentClient () Error while deserialize ServerError object. GET https://[AgentName]:8006/apprecovery/api/agent/pair/connect/?useNtlmOnly=True failed with HTTP status code InternalServerError: 

This indicates the Core attempted to connect to the agent using NTLM authentication, but the connection was blocked or failed on the agent side.

Symptoms

• Unable to protect or pair a Rapid Recovery Agent from the Core

• UI displays: Failed to connect to remote machine using provided credentials

• Core logs show: useNtlmOnly=True and InternalServerError

• Remote WMI tests (wbemtest or Get-WmiObject) return Access Denied

• Local WMI tests on the agent using \\localhost\root\cimv2 may succeed

• The protected machine is often a Domain Controller or hardened Windows Server

• Windows Event Viewer may show WMI/DCOM errors (e.g., Event ID 10028 or 10036)

This issue is not caused by Rapid Recovery, and no changes have been made to Rapid Recovery that could result in this behavior.

Instead, it is due to environmental conditions, specifically NTLM authentication policies or system-level security settings that are interfering with or blocking the authentication.

While some NTLM policies may appear as "Not Configured" or set to "Audit All", this does not always guarantee that authentication will be permitted in all environments. We have observed cases where connections were blocked until the policies were explicitly set to “Allow all.”

This indicates that the effective policy enforcement in some domain or local environments may be more restrictive than it appears in the UI, particularly when additional security baselines or GPOs are applied.

Update NTLM and local security policies on the protected machine to allow NTLM authentication and remote WMI access.

Step 1: Configure NTLM Policy (Local or Domain GPO)

Open gpedit.msc and navigate to:
Local Computer Policy > Computer Configuration > Windows Settings > Security Settings > Local Policies > Security Options

Set the following policies:

• Network security: Restrict NTLM: Incoming NTLM traffic → Allow all

• Network security: Restrict NTLM: Outgoing NTLM traffic to remote servers → Allow all

• Network security: Restrict NTLM: Audit NTLM authentication in this domain → Enable all

• Network security: Restrict NTLM: Audit Incoming NTLM Traffic → Enable auditing for all accounts

Run gpupdate /force to apply changes.

Step 2: Verify Local Policy Persistence

After running gpupdate /force, re-check the settings in gpedit.msc.
If the changes were reverted, a domain Group Policy Object (GPO) is enforcing restrictions.

To confirm:

• Run gpresult /h gporeport.html

• Open the report and check which policy is applying the settings

• Work with your domain administrator to adjust or create exceptions

Additional Information

• For background, see KB 4033596 – NTLM and Pass-Through Authentication

• Related WMI troubleshooting KB: KB 4033088 – Using wbemtest to Test DCOM and WMI Connections

Need More Help?

If you are still experiencing issues after following the steps in this article, please contact Quest Support for further assistance.