Backups of all or most of your vms are running very long due to slow read speeds. Each vm may exhibit the same read speed regardless of how many vms are running.
Example: 2 vms each run at 1MB/sec for a total of 2MB/sec. Adding a 3rd vm causes the total to go to 3MB/sec with each vm exhibiting 1MB/sec.
This may be due to hardware/firmware issues. Please be sure the firmware is up to date on your ESX hosts, including all NIC and HBA/array controllers. Also check firmware on SAN where vms reside.