VMware backup fails with Process running on 'proxyserver' has exited unexpectedly. A message similar to the one shown on the job logs extract can be seen:
| Job Message | 2017/06/13 10:06:59 | 204 | Data Plugin | proxyserver | [VM] Backing up disk '[LUN] VM/VM_2.vmdk' | |
| Error | 2017/06/13 10:08:01 | 204 | Jobs | NV | Process running on proxyserver' has exited unexpectedly | |
| Error | 2017/06/13 10:08:01 | 204 | Jobs | NV | Setting exit status to failed (2) |
When checking the trace of the error in the job log, in the more info section, the message “Failed to read virtual disk with VIX_E_OUT_OF_MEMORY error” is displayed.
| 2 | VDDK | 1956 | 100 | 100932846638 | Disk '[LUN] VM/VM_2.vmdk' opened using mode 'nbd' | |
| 0 | VDDK | 1956 | 45 | 100932846638 | Failed to read disk data | |
| 0 | VDDK | 1956 | 138 | 100932846638 | VDDK error encountered: code 2 | |
| 2 | VDDK | 1956 | 139 | 100932846638 | Retrieving error message text | |
| 0 | VDDK | 1956 | 52 | 100932846638 | VDDK error message is 'Memory allocation failed. Out of memory.' | |
| 0 | VMWSLAVE | 1956 | 293 | 100932846638 | Failed to read virtual disk with VIX_E_OUT_OF_MEMORY error (Block size 65536). | |
| 2 | VMWSLAVE | 1956 | 294 | 100932846638 | Try to read virtual disk with block size 8192. |
This is caused by a known issue in the ESXi host software; error has been detected on ESXi 5.5, 6.0, and 6.5.
Error returned when ESXi host reaches NFC memory limit.
Previously, VDDK would hang during I/O operations if the host ran out of NFC memory. This has been fixed in this release. You must upgrade both VDDK and ESXi to get the fix. VDDK now returns the VIX_E_OUT_OF_MEMORY error, leaving the disk handle in a still-valid state. Upon receiving this error, the caller can try again with a smaller I/O buffer, retry at a later time, perform other operations on the disk, or close the disk.
To work around this problem, reduce the read block size of the VMWare plugin to 2048. Unfortunately, this will slow down the backup, but as this is a VMWare defect this is the alternative to run the NetVault backup job.
To change the read block size, browse the proxy server using the Create Selection Set dialogue box, select the VMware plugin, and from the drop down list select the new value.
To fix the bug, the ESXi hosts will need to be upgraded to a version where the issue has been fixed. Then, increase the VMware plugin read block size again.
