If you want to backup with only COS backups on ESX server then follow these steps.
Upon upgradeing to 5.4 or 5.5 automatic transport is selected. This caused Advanced (SAN\Hot-add) to be attempt.
. If it is missing the right-click on the server and select "Add Host Credentials"Your jobs are now set to use the Console (COS) for backups. You may want to make sure optimize your service console by following this KB: https://support.quest.com/SolutionDetail.aspx?id=SOL77393