Appendix F: Recovery Process
The Migrator for Notes migration engine has been enhanced to track the status of processed messages during migration to help recover failed migrations.
If a critical failure occurs in the migration engine, the migration process can terminate unexpectedly. The user status must be reported in Notes Migrator.nsf as “Migration terminated abnormally.” After correcting the cause of the failure, the new recovery feature allows Migrator for Notes users to restart migrating to an Exchange mail store or to a PST file from the point of crash. It prevents the application from reprocessing messages that were migrated during the initial run of the migration and moves to the next message in the message store without attempting to reprocess the message that caused the failure.
In addition, this feature also allows Migrator for Notes users to restart the migration on a secondary workstation if the first workstation is now busy after the original failed status.
In cases where a failure is caused by API errors processing the data, from Notes using the HCL Notes API or into Outlook using the MAPI API, the source document reference (UNID) will be marked in the migration history as a bad document. These types of faults are typically caused by memory exception errors as a result of source data that the vendor's API cannot process.
Subsequent migrations will skip processing (not migrate) a source document that caused a memory exception error where the migration worker was able to obtain the document UNID.
This section outlines the common reasons for a failed migration and recovery processes for each possible scenario and provides a background to the implementation and usage of this new feature.
Implementation and Feature Functionality
By default, this feature is enabled in the Enable Recovery setting in the Advanced tab of the migration profile used on each migration workstation. With this setting enabled, during each migration, the migration engine generates a text file, CMTUProcessedNoteIDs-[username].txt, in the temp directory. This .txt file contains all the NoteIDs processed during the migration. If a crash occurs causing the migration to fail, the last entry in the .txt file is the NoteID that caused the failure. If the migration is successful, this file is deleted during cleanup.
A migration workstation, when it first starts and between user migrations, checks the temp directory (typically c:\windows\temp\) for any existing CMTUProcessedNoteIDs-[Username].txt file(s). If a file is found, the migration workstation uploads the list of NoteIDs from the file into the migrated messages table in Migrator for Notes before starting the migration. This updates the Migrator for Notes migrated message table for the failed migration, then processes the pending queue as normal. This allows any migration workstation to run follow-up migrations.
Common Failure Types
Before restarting a failed migration, you must first examine the migration status of the user and log files from the failed migration to determine the cause of the failure. If the cause of the failure is corrupted views or database design elements, restarting the migration will not help. The migration will continue to fail until the problem is resolved and the recovery process will not allow the migration to continue past the point of failure.
If the cause of the failure is a failed physical attribute that needs correcting, then provided below are the four most common types of physical attribute failures with their corresponding example log files, and possible resolutions.
Cannot Open Mail File: “Migration initialization failed”
If the mail file does not exist at the specified location, or the migration account does not have access to the mail file, Migrator for Notes reports a status of “Migration initialization failed.” This status can appear if either the source or destination server is unavailable, or if the migration account does not have access to the message store. Regardless of the number of attempts made, this type of problem cannot be successful until it is resolved. If the log file records a similar message as the following example, “Unable to open mail file CN=SERVERNAME/OU=MS/OU=SVR/O=ONE!!mail6\harr0156,” either the mail file is not present or there is an ECL alert on the workstation that is running that user. In case of ECL alerts, click through them until they stop appearing.
Corrupt Tables:
When the log file records the following statement “NSFDbGetModifiedNoteTable returned 'No documents have been modified since specified time.' while getting Note IDs of folders to process”, the tables in the mail template are corrupt. To fix this, the common practice is to create a new copy of the mail file, then replace the database design.
|
|
When you create a copy or a replica of the mail file to correct corrupt tables, you must remember that the recovery log does not work as expected with the copy or the replica. This is because all the documents in the mail file now have new NoteIDs. So, when you restart the migration, all the documents in the mail file are migrated and you will end up with duplicates. To resolve this, when you restart the migration, you must treat it as a complete redo of that user’s mail file migration, and clear the contents of the user's Exchange mailbox before proceeding. |
Corrupt Views:
When the log file contains a statement like the one shown right before the summary “Error: NSFNoteOpen returned 'You are not authorized to perform that operation' opening view note”, some of the views in the mail template are corrupt, and you need to create a new copy of the mail file, then replace design as instructed above.
Notes Crash:
If there is a critical error in the Lotus Notes client, the user’s status must be reported as “Migration Terminated Abnormally.” The can also be verified by viewing the migration log: the log file’s last statement will show this “3/2/2009 10:05:17 PM Migration preformed on workstation na svr20 workstation 1.” To resume migration, restart the migration worker application on the Migration Workstation that is stated at the end of the log, then reset the user in Migrator for Notes and migrate again. Users in this status can be restarted and the migration will be successful.
This type of crash can be easily identified if the log file does not contain a summary of the events before ending the log with the statement “Migration performed on workstation…”. In such a case, if you restart the migration with recovery enabled, the migration is allowed to continue past the point of failure. However, if the log ends without the “Migration performed on workstation…” statement, then when you restart the migration, you will need to ensure that the migration is processed by the same workstation (that processed it earlier at the time of crash) to ensure that it properly resumes from the point of failure.
Cannot Open Mail File: “Migration initialization failed”
Common Failure Types
Before restarting a failed migration, you must first examine the migration status of the user and log files from the failed migration to determine the cause of the failure. If the cause of the failure is corrupted views or database design elements, restarting the migration will not help. The migration will continue to fail until the problem is resolved and the recovery process will not allow the migration to continue past the point of failure.
If the cause of the failure is a failed physical attribute that needs correcting, then provided below are the four most common types of physical attribute failures with their corresponding example log files, and possible resolutions.
If the mail file does not exist at the specified location, or the migration account does not have access to the mail file, Migrator for Notes reports a status of “Migration initialization failed.” This status can appear if either the source or destination server is unavailable, or if the migration account does not have access to the message store. Regardless of the number of attempts made, this type of problem cannot be successful until it is resolved. If the log file records a similar message as the following example, “Unable to open mail file CN=SERVERNAME/OU=MS/OU=SVR/O=ONE!!mail6\harr0156,” either the mail file is not present or there is an ECL alert on the workstation that is running that user. In case of ECL alerts, click through them until they stop appearing.
Corrupt Tables:
When the log file records the following statement “NSFDbGetModifiedNoteTable returned 'No documents have been modified since specified time.' while getting Note IDs of folders to process”, the tables in the mail template are corrupt. To fix this, the common practice is to create a new copy of the mail file, then replace the database design.
|
|
When you create a copy or a replica of the mail file to correct corrupt tables, you must remember that the recovery log does not work as expected with the copy or the replica. This is because all the documents in the mail file now have new NoteIDs. So, when you restart the migration, all the documents in the mail file are migrated and you will end up with duplicates. To resolve this, when you restart the migration, you must treat it as a complete redo of that user’s mail file migration, and clear the contents of the user's Exchange mailbox before proceeding. |
Corrupt Views:
When the log file contains a statement like the one shown right before the summary “Error: NSFNoteOpen returned 'You are not authorized to perform that operation' opening view note”, some of the views in the mail template are corrupt, and you need to create a new copy of the mail file, then replace design as instructed above.
Notes Crash:
If there is a critical error in the Lotus Notes client, the user’s status must be reported as “Migration Terminated Abnormally.” The can also be verified by viewing the migration log: the log file’s last statement will show this “3/2/2009 10:05:17 PM Migration preformed on workstation na svr20 workstation 1.” To resume migration, restart the migration worker application on the Migration Workstation that is stated at the end of the log, then reset the user in Migrator for Notes and migrate again. Users in this status can be restarted and the migration will be successful.
This type of crash can be easily identified if the log file does not contain a summary of the events before ending the log with the statement “Migration performed on workstation…”. In such a case, if you restart the migration with recovery enabled, the migration is allowed to continue past the point of failure. However, if the log ends without the “Migration performed on workstation…” statement, then when you restart the migration, you will need to ensure that the migration is processed by the same workstation (that processed it earlier at the time of crash) to ensure that it properly resumes from the point of failure.