ESXi replication via virtual appliance network mode fails. The application throws the error:
FATAL cant_write can't write vix:r: XXXXXXXXXX/{Target Datastore}/{Target VM.vmdk}:moref=vm-3972:snapshot-4668:3 [at slave_main:569] FATAL internal internal error [at pkt_recv_unlock:82]
Upon examining the VMs file system in the VMware client's Datastore Browser tool, the typical VMware icon (three blue boxes) is not visible on the .vmx (configuration) file for the VM (instead, the icon is a "blank sheet of paper" icon). Attempting to download the file will fail with a non-specific error. This file needs to be parsed by the vRanger hot-add process in order to obtain the disk UUIDs of the VMs that are being backed-up, and it needs to be edited by the Ranger API call to re-point the VM to the vRanger snapshot file (eg. "-000001.vmdk) while the replication is running. With the file in this current state, the file cannot be manipulated or opened, so the replication stalls and cannot proceed.This is due to some locking mechanism that VMware has on the .vmx configuration file of the VM (cause not yet known but under investigation). Even though nothing else is attempting to use the VM (i.e. another VMWare process), the Virtual Center/host is not allowing this file to be opened/edited by vRanger's API call to do so. The point of the replication where this occurs is visible in the VMWare client when one can see the task "reconfiguring virtual machine" in the Task list. If the file can be manipulated correctly, this will complete quickly. If it cannot manipulate it, the task will hang until the API call timeout is reached, and fail
A workaround is to vMotion the VM to another host (simple vMotion, not Storage vMotion), then check out the file system in Datastore Browser. If the familiar VMware icon is present adjacent to the .vmx file, attempt to download the file to the Windows desktop where the VMware client is being run. If the file downloads, the problem likely is temporarily fixed. Refresh the vRanger inventory, then re-run the replication job to test. If the job progresses past the "reconfiguring virtual machine" task in the VMware client's task bar, the job is likely working.
NOTE: At this time, vMotioning the VM back to its original location after a successful move to another machine/hot-add test will NOT remove the lock on this .vmx file. The only current workaround (under investigation) is to reboot the ESXi host to clear the lock.
