Backup of a VM fails with the above API error. This error is addressed at length in the following VMware KB:
This can be caused by a VMware Tools issue or a Windows VSS service issue.
There are a few steps to try before further troubleshooting. After each step attempt the backup again.
1. On the vm being backed up please be sure that the VMware Tools are up to date, update if needed. This can be confirmed from the vSphere client.
2. Restart the VMware tools service on the affected vm
3. Reboot the affected vm
If these do not resolve your issue please consult this VMware KB article for further troubleshooting steps.