vRanger compressed backups running slowly (91371)

In a vRanger VM install, where the vRanger Repository is configured as a physical Raw Device Mapping and Hot-Add (LAN-free) backups are being performed, uncompressed backups are showing extremely high read/write throughputs (as much as 200MB/sec or more). When compression is enabled for the backup job, the throughput to this same Repository drops down to speeds as much as 10X slower for read/writes (possibly even slower).

Depending on the SAN equipment involved, this can be explained by newer SAN Hardware/Software having VAAI (VMWare vStorage APIs for Array Integration) support. Consult the following VMware KB article for facts about VAAI:

http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1021976

In this case, where vRanger is Hot-Adding disks to itself, and these disks are resident on the SAME SAN as the VMWare datastore LUNs, the VAAI support on the SAN is "smart" enough to bypass the ESXi hosts' VMKernel and the vRanger machine as a backup proxy altogether. It "knows" that the RDM used for the Repository is on the same storage as the Datastore information, and then uses the VMWare API which vRanger uses to backup the data to do a "direct copy" operation right across the SAN backplane. This vastly increases the throughput of the backup, since the data is never "leaving" the SAN.

When compression is enabled, however, the vStorage API used for the backup has no method via VAAI to compress the data, so the Hot-added blocks are read (proxied) through the vRanger machine, processed/compressed, then written directly to the RDM. VAAI cannot perform this function on its own via the vStorage APIs.

This is by design (currently) of VAAI, and is not a "problem" as such. It is a trade-off between having very fast backups, but those backups taking up more disk space, vs. slower backups taking up less disk space. If SAN space can be afforded, a good trade off may be to perform incremental or differential backups (but WITH VMware's Change Block Tracking Enabled. Without CBT being enabled, the data would have to proxy through the vRanger machine again. With it enabled, the direct VAAI copy should work and the data should not leave the SAN or have to proxy through the machine. Since compression is defined within specific vRanger jobs and not globally, a "hybrid" setup can also be set up, depending on the individual needs of the environment.

In order to support VAAI, the vRanger install must be 5.3.1 or later, as this revision includes the updated VMWare vStorage APIs used to perform backups which supports VAAI functionality.