To troubleshoot this issue, follow the suggested steps below. Note that if the tests below fail, this is something that the customer would need to rectify within their own infrastructure as it is a basic requirement that at a minimum, certain ports are required for RMAD to function correctly.
From the RMAD ServerOpen an elevated Powershell session from an administrative command prompt and run the command below:
Test-NetConnection -ComputerName <DCFQDN or IP> -port <AgentPortConfigured>
An example of the output expected is shown below testing the connectivity to domain controller tommctestdc1.tommctest.local using the default RMAD backup agent port 3843.

Click To See Full Image.
The above result shows that the TCP test to port 3843 has succeeded. If this command returns "False" then there is most likely a network issue preventing RMAD from communicating with the DC, possibly a firewall or other network ACLs in place.
From the Domain ControllerOpen an administrative command prompt and use the netstat command as described below to search for the Backup agent port within the network connections of the domain controller. The syntax of the command is below and the port can be changed accordingly if the default port is not in use.
netstat -ano | findstr "3843"
Sample output from this command is shown below. Notice that in the first screen capture, the port is shown as listening. This is the default status when the backup agent is not connected to the RMAD console and not actively performing a task. In the second screen capture, note the status is now shown as "Established". This indicates the RMAD server is connected to the Agent on the DC which would be normal during a backup.
.jpg)
Click To See Full Image.
.jpg)
Click To See Full Image.
If the output from these tests does not show "LISTENING" it would indicate that possibly the backup agent is either not running or not installed. If the connection does not change status to "ESTABLISHED" once a backup is started from the RMAD console, it would indicate there is likely a network issue preventing RMAD from communicating with the DC, possibly a firewall or other network ACLs in place.