Replication job fails with this error:
Message: An internal error occurred during execution, please contact Vizioncore support if the error persists. Error Message: 3185 - Warning: Permanently added 'esx.name.com' (RSA) to the list of known hosts. WARN repl_undo processing undo file phys:/vmfs/volumes/XXX-XXX/VMNAMEHERE/VMNAMEHERE_1-flat.vmdk.vzundo WARN repl_undo processing undo file phys:/vmfs/volumes/XXX-XXX/VMNAMEHERE/VMNAMEHERE_2-flat.vmdk.vzundo RETRY timedout operation timed out [at packet_timeout:404] FATAL internal internal error [at pkt_recv_unlock:82]
A script called vzundu-script' is created by the replication process to roll back changes on the target VM in case of a replication failure. This script creates a file (.vzundo) per VMDK in the target VM folder at the beginning of the replication process and if successful, is removed at the end. This ".vzundo" is a temporary file that records original data of changed blocks since the replication started per each *.vmdk file replicated. In case of a replication failure such as network failure, vzundo-script can be executed to restore the *.vmdks to their original, pre replication job state. To reiterate, under normal run-time ciurcumstances, the *.undu & vzundu-script file itself that are created by the replication process should be removed when a job is successfully completed; Consequently, they should only appear / remain on the replca VM's datastore when a job has failed. If you are seeing *.undo / vzundo-script files remaining after jobs have successfully completed, please contact support.
There are 2 possible workarounds for this error. #A is applicable only if the replication does not use VA (Virtual Appliance) to replicate.Temporary work around for this issue may only allieviate the problem for next scheduled replication.
A. Manually run the replundu script,as detailed below.
B. Delete the replica and schedule then re-run (Full) replication.
These are the steps on how to run replundo script:
1. Log in to target host using putty1.
Type these commands:
su - root
cd /tmp/.vzbin
ls –la
There should be directory called “201106211240” (date stamped directory, pick newest date if 2 directories are present)
Note:If there are no vz tools listed please start a backup job of a VM sitting on this host, cancel the job after 2-5 min
cd 201106211240
Type ls -la (this should list all the files. Please look for "replundo")
2. Run this command2 with path to your VM directory:
./replundo ‘phys:/vmfs/volumes/datastore/vmname/vmname.vmdk.undo’
Example:
./replundo ‘phys:/vmfs/volumes/4a16491d-5816fb0d-8713-001ec9fce735/GPvRtesting/GPvRtesting.vmdk.undo’
Note:If this routine does not work. Go to step “B” Delete the replica and schedule/run full replication.
Note:You can Also use this routine if you wish to revert the replica after a crashed replication to the last good pass and the vzundo file still exists. This reason is the original function of the vzundo file and is what it should be used for in most cases.
Critical Failures can include:
Loss of network connection during replication. Power loss of either the source or DR site during replication. As well as others.
1To download PuTTY: A free Telnet/SSH Client go here.
2If you don't know what is your path to your VM here is how to query for your vmfs directory on the ESX host:
cd / -switch to root directory of your host.
vmware-cmd -l -lists all the VM's directories with vmx files.
/vmfs/volumes/4e67c456-5816fb0d-8713-001ec9fce735/vmnamehere/ - (copy this path to use in step #2)
Navigate back to tools directory:
CD /tmp/.vzbin/<date stamped dir>
