If self signed certificates are not allowed in the environment then SCHANNEL authentication will need to be disabled. Follow the below steps to set the forest recovery agent to use negotiate instead of SCHANNEL:
HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\Dell\Recovery Manager for Active Directory
- Create the same reg key on the DCs
- Remove the cert from agent installation folder (C:\Program Files\Dell\Recovery Manager for Active Directory Forest Edition) on the DC, either by manually deleting it, or by reinstalling the agent from the FR console.