The issue has been traced to the Core server, having multiple entries for the same agent machine. Different entries are having different Agent IDs, and only 1 ID is matching with the ID from the actual agent's registry.
Navigate to registry on the Core into HKLM\Software\AppRecovery\Core\Agents\ and check AgentIDs one by one, writing down entries with the same agent machine name or IP. Write down display names.
Check these display names in Core console and see if there are Recovery Points, that belong to them.
On the agent machine, open regedit, HKLM\Software\Apprecovery\Agent\AgentID and write down agent's ID
Back to the Core, remove all display names that having same name/IP for the URL, and whose agent ID doesn't match. Remaining machine with the proper name or IP and matching AgentID should be online. Removed agents' RPs will be moved to the Recovery Points only category.