サポートと今すぐチャット
サポートとのチャット

Binary Tree Migrator for Notes 20.11 - User Guide

Section 1. Introduction Section 2. Pre-migration Activities Section 3. User Provisioning Section 4. Email Repliability Section 5. Migrating Mail Files Section 6. Rooms and Resources Database Migration Section 7. Mail-in Database Migration Section 8. Setting Migration Status Section 9. Access and Delegation Migration Section 10. All Accounts Section 12. Logs Appendix A: Staging Replicas Appendix B: Pre-Migration Troubleshooting Appendix C: Work with Files (Import/Export) Appendix D: Item Processing Results Appendix E: Migration Result Statuses Appendix F: Recovery Process Appendix G: Automatic Migration Restart Appendix H: Folder Processing Order Third Party Components

Configuring Automatic Restart

Overview

Automatic Migration Restart is a process by which any abnormal termination of the Migrator for Notes engine (commonly referred to as a ‘crash’) is identified, and the migration job is automatically re-queued.  If this occurs, the migration is restarted and resumed on the next document in the message store, until either the migration completes process the message store, or the maximum number of retries has been exceeded.

Automatic Migration Restart works in conjunction with the existing migration recovery features in Migrator for Notes to resume a migration on the next document immediately after the document which caused the crash. 

Terminology

This following table lists and describes the terms commonly used when describing the automatic restart process.

Term

Description

Migrator for Notes

Migrator for Notes application environment, encompassing the server instance.

Job

A migration event. Within the context of Migrator for Notes, each migration job is a unique event.

Migration server

Refers to the system running the Migrator for Notes web services. This is the primary instance where the web server and CMT Monitor are installed.

Migrator for Notes Database (or Notes Migrator.nsf)

Used for configuring settings, managing user records, and queuing users to the migration server.  

Migration worker

An instance of the CMT_MigrationWorker.exe application running on a migration workstation. This application runs continuously on the migration workstation, polls the migration server for pending migration jobs, and initializes an instance of the migration engine (CMTProxy.exe) for each migration job.  

Migration engine (or CMTProxy)

An instance of CMTProxy.exe, which is initialized and managed by the Migration worker. CMTProxy.exe performs the actual data migration.

Crash

Abnormal termination of the migration engine, resulting in a status of “Migration terminated abnormally.”

Automatic restart is controlled by a parameter in the Web.config file for the Migrator for Notes server.    This file is located in [installDirectory]\CMT for Exchange\CMT_XMLServer\Web.config , for example “C:\Program Files (x86)\BinaryTree\CMT for Exchange\CMT_XMLServer\web.config”.

The Web.config file contains the startup parameters for the Migrator for Notes server.    Located in this file is an \appSettings\MaximimumRetryCount XML node, which indicates the maximum number of times a migration will be automatically restarted.

    <appSettings>

        ---snip ----

        <add key="ScheduleTimerInterval" value="3"></add>

        <add key="UseSQL" value="1"></add>

        <add key="MaximumRetryCount" value="10"></add>

<add key="RetryJobStatus" value="#001;#002"></add>

    </appSettings>

 

The MaximumRetryCount is set to 10 by default.  A value of 0 or a negative number will disable the Automatic Restart functionality.  

RetryJobStatus is set by default to resubmit if the #001 (Notes memory exception) or #002 (Outlook memory exception) job status code is returned. 


If MaximumRetryCount or “RetryJobStatus” is changed or disabled, the Migrator for Notes server instance must be restarted for the change to take effect.  This can be done by opening a command prompt on the migration server and typing “iisreset” at the prompt, or manually restarting the IIS service.

Migrator for Notes Worker Basics

The Migrator for Notes Worker is an application that runs in the task bar of windows. This application is responsible for queuing, migrating, and updating the status of a migration job on the Migrator for Notes server.   

The Migrator for Notes Worker has three basic states:

  • Offline – the migration worker is not started.

  • Online – the service is running, and actively checking for migration  jobs

  • Migrating – a migration job is in progress.

When the Migration Worker application is online, the application periodically checks the Migrator for Notes server for pending migration jobs. If a job is available, the migration server passes back the job number from the migration queue, along with settings and user information required to start the migration. The worker then starts the migration by calling CMTProxy.exe as a separate process with the appropriate parameters, which begins the actual data migration. Once the migration starts, the migration worker application sends notification back to the Migrator for Notes server that the workstation is in Migrating status. During the course of the migration, the migration worker periodically communicates throughput and migration time statistics back to the migration server.

When the migration has completed, the Migration Worker updates the CMT server and then uploads the history of migrated messages and relevant log files.

The key concept is that the Migration Worker is responsible for checking the Migrator for Notes Control Center Server for work, triggering a migration job when required, updating the Migration Control Center Server during the migration, and finally uploading log files once the job has been completed.

Migrator for Notes Recovery Basics

During a migration, the Migrator for Notes engine reads and catalogs the unique identifier of each message it encounters before the message is processed. This identifier is stored in memory to prevent migrating a document multiple times, and is also immediately written to disk in a “ProcessedNoteID-username.txt’ file on disk, where username is the unique key of the user on the migration server.

In the event of a crash, the ProcessedNoteID file on disk contains the list of unique identifiers of each document processed during the failed migration. This file is read by the Migration Worker application, merged with any existing migrated message information residing on the server for the user, and stored back on the migration server prior to the next migration. This ensures that the migrated message table on the Migrator for Notes Server is up to date when the migration is restarted.  

When the migration engine is restarted, the migrated message table is read and all previously processed messages (including the ID of the message that was being processed when the migration crashed) are skipped in subsequent runs. This allows Migrator for Notes to resume where it left off, and prevents re-migrating documents repeatedly on subsequent runs.

 


The migrated message table is not used if the Discover History from Migrated Data option is used. In this case, the previously processed messages are read from the Exchange Mailbox directly.

Detecting abnormal termination of a migration

The Migrator for Notes Worker application is a key component in detecting when a migration crashes. When the Migrator for Notes Worker application starts a migration job, it launches the migration by calling CMTProxy.exe in a separate process.  While the migration is running, the Migrator for Notes Worker application periodically sends information back to the Migrator for Notes server indicating migration time and throughput for the current migration.   

If the migration engine (CMTProxy.exe) crashes, the handle to the migration engine becomes invalid, and the Migration Worker application knows that the currently running migration has crashed. At this point, the Migration Worker reads the ProcessedNoteID table from the temporary directory, merges it with any existing information in the Migrated Message table on the Migrator for Notes server, and updates the status of the job to “Migration Terminated Abnormally.”


The Migration Worker does not actually re-queue the job, it reports only “Migration Terminated Abnormally” status back to the Migrator for Notes server. The Migrator for Notes server is responsible for determining if the job will be re-queued.

Automatic re-queue of a migration job

The sections above describe how the Migrator for Notes Worker application identifies a crash in the migration engine and returns “Migration terminated abnormally” status.  This section describes how the Migrator for Notes server re-queues the user and resulting migration status values.

If the migration worker application returns a value of “Migration terminated abnormally” or one of the RetryJobStatus codes to the Migrator for Notes server, the migration server determines if the RetryCount exceeds the MaximumRetryCount configured for the server.     

  • If the RetryCount is less than the MaximumRetryCount, the job is re-queued by creating a new migration job in the migration queue.  This job will be a high-priority job queued for the specific workstation, ensuring that it is the next job processed by that workstation.

  • If the RetryCount for a particular migration is greater or equal to the MaximumRetryCount, the migration will be left in “Migration terminated abnormally” status or the last job status.  This allows the migration administrator to determine which migrations have exceeded the maximum allowable number of automatic restarts, and indicates that the file will need additional remediation before resetting and remigrating the user.  It should be noted that the migration server will contain a history (in the form of a migrated message table) corresponding to all of the data that has been migrated up to that point. After remediation, the migration administrator can simply reset and requeue the user to continue the migration from this point, or clear previously migrated data and clear the migrated message table to restart the migration from the beginning.

  • If a migration has been automatically restarted by this process, the Notes Migrator.nsf database will only show the status of the last migration that was restarted. Consider the situation where a migration crashed, was automatically restarted, and continued to completion without further errors. The migration status in the Migrator for Notes Database will report “Migration completed successfully” even though there was a crash on the initial run.  The migration history will indicate the first migration completed with a status of “Migration terminated abnormally” status, but this may not be evident in the Notes Migrator.nsf Lotus Notes application interface.  Advanced or customized installations of the Lotus Notes application may retrieve and summarize the complete migration history for a user, but a complete migration history is not included in the default Notes Migrator.nsf application interface.

The RetryJobStatus parameter is a string list of semicolon delimited codes that will automatically be resubmitted if the job returns with that specific status code. By default, Migrator for Notes will resubmit memory exception codes #001 (Notes memory exception) and #002 (Outlook memory exception). 

Summary

In this section, we have reviewed how the Migrator for Notes Migration Worker application detects a crash of the migration engine, how the Migrator for Notes Control Center Server processes the “Migration Terminated Abnormally” status and requeues the job if necessary, and ultimately how the migration automatically restarts on the same workstation.

In this manner, the Automatic Restart process in Migrator for Notes allows migration administrators to queue jobs to the migration server and know that if a crash were to occur the migration will automatically be restarted without intervention up to the maximum number of times indicated by the MaximumRetryCount parameter.  This feature reduces the amount of resources required to monitor Migrator for Notes migrations and eliminates the need for administrator intervention in the case where the migration engine has crashed.

Again, it should be noted that the Lotus Notes application interface (Notes Migrator.nsf) will report the last migration status for the user.  Any automatic restart events will be evident in the complete migration history but will not be reflected directly in the interface.  In the case where the maximum restart count has been exceeded, the last migration event will be reported as “Migration terminated abnormally.”

 

Migrator for Notes Worker Basics

Overview

Automatic Migration Restart is a process by which any abnormal termination of the Migrator for Notes engine (commonly referred to as a ‘crash’) is identified, and the migration job is automatically re-queued.  If this occurs, the migration is restarted and resumed on the next document in the message store, until either the migration completes process the message store, or the maximum number of retries has been exceeded.

Automatic Migration Restart works in conjunction with the existing migration recovery features in Migrator for Notes to resume a migration on the next document immediately after the document which caused the crash. 

Terminology

This following table lists and describes the terms commonly used when describing the automatic restart process.

Term

Description

Migrator for Notes

Migrator for Notes application environment, encompassing the server instance.

Job

A migration event. Within the context of Migrator for Notes, each migration job is a unique event.

Migration server

Refers to the system running the Migrator for Notes web services. This is the primary instance where the web server and CMT Monitor are installed.

Migrator for Notes Database (or Notes Migrator.nsf)

Used for configuring settings, managing user records, and queuing users to the migration server.  

Migration worker

An instance of the CMT_MigrationWorker.exe application running on a migration workstation. This application runs continuously on the migration workstation, polls the migration server for pending migration jobs, and initializes an instance of the migration engine (CMTProxy.exe) for each migration job.  

Migration engine (or CMTProxy)

An instance of CMTProxy.exe, which is initialized and managed by the Migration worker. CMTProxy.exe performs the actual data migration.

Crash

Abnormal termination of the migration engine, resulting in a status of “Migration terminated abnormally.”

Configuring Automatic Restart

Automatic restart is controlled by a parameter in the Web.config file for the Migrator for Notes server.    This file is located in [installDirectory]\CMT for Exchange\CMT_XMLServer\Web.config , for example “C:\Program Files (x86)\BinaryTree\CMT for Exchange\CMT_XMLServer\web.config”.

The Web.config file contains the startup parameters for the Migrator for Notes server.    Located in this file is an \appSettings\MaximimumRetryCount XML node, which indicates the maximum number of times a migration will be automatically restarted.

    <appSettings>

        ---snip ----

        <add key="ScheduleTimerInterval" value="3"></add>

        <add key="UseSQL" value="1"></add>

        <add key="MaximumRetryCount" value="10"></add>

<add key="RetryJobStatus" value="#001;#002"></add>

    </appSettings>

 

The MaximumRetryCount is set to 10 by default.  A value of 0 or a negative number will disable the Automatic Restart functionality.  

RetryJobStatus is set by default to resubmit if the #001 (Notes memory exception) or #002 (Outlook memory exception) job status code is returned. 


If MaximumRetryCount or “RetryJobStatus” is changed or disabled, the Migrator for Notes server instance must be restarted for the change to take effect.  This can be done by opening a command prompt on the migration server and typing “iisreset” at the prompt, or manually restarting the IIS service.

The Migrator for Notes Worker is an application that runs in the task bar of windows. This application is responsible for queuing, migrating, and updating the status of a migration job on the Migrator for Notes server.   

The Migrator for Notes Worker has three basic states:

  • Offline – the migration worker is not started.

  • Online – the service is running, and actively checking for migration  jobs

  • Migrating – a migration job is in progress.

When the Migration Worker application is online, the application periodically checks the Migrator for Notes server for pending migration jobs. If a job is available, the migration server passes back the job number from the migration queue, along with settings and user information required to start the migration. The worker then starts the migration by calling CMTProxy.exe as a separate process with the appropriate parameters, which begins the actual data migration. Once the migration starts, the migration worker application sends notification back to the Migrator for Notes server that the workstation is in Migrating status. During the course of the migration, the migration worker periodically communicates throughput and migration time statistics back to the migration server.

When the migration has completed, the Migration Worker updates the CMT server and then uploads the history of migrated messages and relevant log files.

The key concept is that the Migration Worker is responsible for checking the Migrator for Notes Control Center Server for work, triggering a migration job when required, updating the Migration Control Center Server during the migration, and finally uploading log files once the job has been completed.

Migrator for Notes Recovery Basics

During a migration, the Migrator for Notes engine reads and catalogs the unique identifier of each message it encounters before the message is processed. This identifier is stored in memory to prevent migrating a document multiple times, and is also immediately written to disk in a “ProcessedNoteID-username.txt’ file on disk, where username is the unique key of the user on the migration server.

In the event of a crash, the ProcessedNoteID file on disk contains the list of unique identifiers of each document processed during the failed migration. This file is read by the Migration Worker application, merged with any existing migrated message information residing on the server for the user, and stored back on the migration server prior to the next migration. This ensures that the migrated message table on the Migrator for Notes Server is up to date when the migration is restarted.  

When the migration engine is restarted, the migrated message table is read and all previously processed messages (including the ID of the message that was being processed when the migration crashed) are skipped in subsequent runs. This allows Migrator for Notes to resume where it left off, and prevents re-migrating documents repeatedly on subsequent runs.

 


The migrated message table is not used if the Discover History from Migrated Data option is used. In this case, the previously processed messages are read from the Exchange Mailbox directly.

Detecting abnormal termination of a migration

The Migrator for Notes Worker application is a key component in detecting when a migration crashes. When the Migrator for Notes Worker application starts a migration job, it launches the migration by calling CMTProxy.exe in a separate process.  While the migration is running, the Migrator for Notes Worker application periodically sends information back to the Migrator for Notes server indicating migration time and throughput for the current migration.   

If the migration engine (CMTProxy.exe) crashes, the handle to the migration engine becomes invalid, and the Migration Worker application knows that the currently running migration has crashed. At this point, the Migration Worker reads the ProcessedNoteID table from the temporary directory, merges it with any existing information in the Migrated Message table on the Migrator for Notes server, and updates the status of the job to “Migration Terminated Abnormally.”


The Migration Worker does not actually re-queue the job, it reports only “Migration Terminated Abnormally” status back to the Migrator for Notes server. The Migrator for Notes server is responsible for determining if the job will be re-queued.

Automatic re-queue of a migration job

The sections above describe how the Migrator for Notes Worker application identifies a crash in the migration engine and returns “Migration terminated abnormally” status.  This section describes how the Migrator for Notes server re-queues the user and resulting migration status values.

If the migration worker application returns a value of “Migration terminated abnormally” or one of the RetryJobStatus codes to the Migrator for Notes server, the migration server determines if the RetryCount exceeds the MaximumRetryCount configured for the server.     

  • If the RetryCount is less than the MaximumRetryCount, the job is re-queued by creating a new migration job in the migration queue.  This job will be a high-priority job queued for the specific workstation, ensuring that it is the next job processed by that workstation.

  • If the RetryCount for a particular migration is greater or equal to the MaximumRetryCount, the migration will be left in “Migration terminated abnormally” status or the last job status.  This allows the migration administrator to determine which migrations have exceeded the maximum allowable number of automatic restarts, and indicates that the file will need additional remediation before resetting and remigrating the user.  It should be noted that the migration server will contain a history (in the form of a migrated message table) corresponding to all of the data that has been migrated up to that point. After remediation, the migration administrator can simply reset and requeue the user to continue the migration from this point, or clear previously migrated data and clear the migrated message table to restart the migration from the beginning.

  • If a migration has been automatically restarted by this process, the Notes Migrator.nsf database will only show the status of the last migration that was restarted. Consider the situation where a migration crashed, was automatically restarted, and continued to completion without further errors. The migration status in the Migrator for Notes Database will report “Migration completed successfully” even though there was a crash on the initial run.  The migration history will indicate the first migration completed with a status of “Migration terminated abnormally” status, but this may not be evident in the Notes Migrator.nsf Lotus Notes application interface.  Advanced or customized installations of the Lotus Notes application may retrieve and summarize the complete migration history for a user, but a complete migration history is not included in the default Notes Migrator.nsf application interface.

The RetryJobStatus parameter is a string list of semicolon delimited codes that will automatically be resubmitted if the job returns with that specific status code. By default, Migrator for Notes will resubmit memory exception codes #001 (Notes memory exception) and #002 (Outlook memory exception). 

Summary

In this section, we have reviewed how the Migrator for Notes Migration Worker application detects a crash of the migration engine, how the Migrator for Notes Control Center Server processes the “Migration Terminated Abnormally” status and requeues the job if necessary, and ultimately how the migration automatically restarts on the same workstation.

In this manner, the Automatic Restart process in Migrator for Notes allows migration administrators to queue jobs to the migration server and know that if a crash were to occur the migration will automatically be restarted without intervention up to the maximum number of times indicated by the MaximumRetryCount parameter.  This feature reduces the amount of resources required to monitor Migrator for Notes migrations and eliminates the need for administrator intervention in the case where the migration engine has crashed.

Again, it should be noted that the Lotus Notes application interface (Notes Migrator.nsf) will report the last migration status for the user.  Any automatic restart events will be evident in the complete migration history but will not be reflected directly in the interface.  In the case where the maximum restart count has been exceeded, the last migration event will be reported as “Migration terminated abnormally.”

 

Migrator for Notes Recovery Basics

Overview

Automatic Migration Restart is a process by which any abnormal termination of the Migrator for Notes engine (commonly referred to as a ‘crash’) is identified, and the migration job is automatically re-queued.  If this occurs, the migration is restarted and resumed on the next document in the message store, until either the migration completes process the message store, or the maximum number of retries has been exceeded.

Automatic Migration Restart works in conjunction with the existing migration recovery features in Migrator for Notes to resume a migration on the next document immediately after the document which caused the crash. 

Terminology

This following table lists and describes the terms commonly used when describing the automatic restart process.

Term

Description

Migrator for Notes

Migrator for Notes application environment, encompassing the server instance.

Job

A migration event. Within the context of Migrator for Notes, each migration job is a unique event.

Migration server

Refers to the system running the Migrator for Notes web services. This is the primary instance where the web server and CMT Monitor are installed.

Migrator for Notes Database (or Notes Migrator.nsf)

Used for configuring settings, managing user records, and queuing users to the migration server.  

Migration worker

An instance of the CMT_MigrationWorker.exe application running on a migration workstation. This application runs continuously on the migration workstation, polls the migration server for pending migration jobs, and initializes an instance of the migration engine (CMTProxy.exe) for each migration job.  

Migration engine (or CMTProxy)

An instance of CMTProxy.exe, which is initialized and managed by the Migration worker. CMTProxy.exe performs the actual data migration.

Crash

Abnormal termination of the migration engine, resulting in a status of “Migration terminated abnormally.”

Configuring Automatic Restart

Automatic restart is controlled by a parameter in the Web.config file for the Migrator for Notes server.    This file is located in [installDirectory]\CMT for Exchange\CMT_XMLServer\Web.config , for example “C:\Program Files (x86)\BinaryTree\CMT for Exchange\CMT_XMLServer\web.config”.

The Web.config file contains the startup parameters for the Migrator for Notes server.    Located in this file is an \appSettings\MaximimumRetryCount XML node, which indicates the maximum number of times a migration will be automatically restarted.

    <appSettings>

        ---snip ----

        <add key="ScheduleTimerInterval" value="3"></add>

        <add key="UseSQL" value="1"></add>

        <add key="MaximumRetryCount" value="10"></add>

<add key="RetryJobStatus" value="#001;#002"></add>

    </appSettings>

 

The MaximumRetryCount is set to 10 by default.  A value of 0 or a negative number will disable the Automatic Restart functionality.  

RetryJobStatus is set by default to resubmit if the #001 (Notes memory exception) or #002 (Outlook memory exception) job status code is returned. 


If MaximumRetryCount or “RetryJobStatus” is changed or disabled, the Migrator for Notes server instance must be restarted for the change to take effect.  This can be done by opening a command prompt on the migration server and typing “iisreset” at the prompt, or manually restarting the IIS service.

Migrator for Notes Worker Basics

The Migrator for Notes Worker is an application that runs in the task bar of windows. This application is responsible for queuing, migrating, and updating the status of a migration job on the Migrator for Notes server.   

The Migrator for Notes Worker has three basic states:

  • Offline – the migration worker is not started.

  • Online – the service is running, and actively checking for migration  jobs

  • Migrating – a migration job is in progress.

When the Migration Worker application is online, the application periodically checks the Migrator for Notes server for pending migration jobs. If a job is available, the migration server passes back the job number from the migration queue, along with settings and user information required to start the migration. The worker then starts the migration by calling CMTProxy.exe as a separate process with the appropriate parameters, which begins the actual data migration. Once the migration starts, the migration worker application sends notification back to the Migrator for Notes server that the workstation is in Migrating status. During the course of the migration, the migration worker periodically communicates throughput and migration time statistics back to the migration server.

When the migration has completed, the Migration Worker updates the CMT server and then uploads the history of migrated messages and relevant log files.

The key concept is that the Migration Worker is responsible for checking the Migrator for Notes Control Center Server for work, triggering a migration job when required, updating the Migration Control Center Server during the migration, and finally uploading log files once the job has been completed.

During a migration, the Migrator for Notes engine reads and catalogs the unique identifier of each message it encounters before the message is processed. This identifier is stored in memory to prevent migrating a document multiple times, and is also immediately written to disk in a “ProcessedNoteID-username.txt’ file on disk, where username is the unique key of the user on the migration server.

In the event of a crash, the ProcessedNoteID file on disk contains the list of unique identifiers of each document processed during the failed migration. This file is read by the Migration Worker application, merged with any existing migrated message information residing on the server for the user, and stored back on the migration server prior to the next migration. This ensures that the migrated message table on the Migrator for Notes Server is up to date when the migration is restarted.  

When the migration engine is restarted, the migrated message table is read and all previously processed messages (including the ID of the message that was being processed when the migration crashed) are skipped in subsequent runs. This allows Migrator for Notes to resume where it left off, and prevents re-migrating documents repeatedly on subsequent runs.

 


The migrated message table is not used if the Discover History from Migrated Data option is used. In this case, the previously processed messages are read from the Exchange Mailbox directly.

Detecting abnormal termination of a migration

The Migrator for Notes Worker application is a key component in detecting when a migration crashes. When the Migrator for Notes Worker application starts a migration job, it launches the migration by calling CMTProxy.exe in a separate process.  While the migration is running, the Migrator for Notes Worker application periodically sends information back to the Migrator for Notes server indicating migration time and throughput for the current migration.   

If the migration engine (CMTProxy.exe) crashes, the handle to the migration engine becomes invalid, and the Migration Worker application knows that the currently running migration has crashed. At this point, the Migration Worker reads the ProcessedNoteID table from the temporary directory, merges it with any existing information in the Migrated Message table on the Migrator for Notes server, and updates the status of the job to “Migration Terminated Abnormally.”


The Migration Worker does not actually re-queue the job, it reports only “Migration Terminated Abnormally” status back to the Migrator for Notes server. The Migrator for Notes server is responsible for determining if the job will be re-queued.

Automatic re-queue of a migration job

The sections above describe how the Migrator for Notes Worker application identifies a crash in the migration engine and returns “Migration terminated abnormally” status.  This section describes how the Migrator for Notes server re-queues the user and resulting migration status values.

If the migration worker application returns a value of “Migration terminated abnormally” or one of the RetryJobStatus codes to the Migrator for Notes server, the migration server determines if the RetryCount exceeds the MaximumRetryCount configured for the server.     

  • If the RetryCount is less than the MaximumRetryCount, the job is re-queued by creating a new migration job in the migration queue.  This job will be a high-priority job queued for the specific workstation, ensuring that it is the next job processed by that workstation.

  • If the RetryCount for a particular migration is greater or equal to the MaximumRetryCount, the migration will be left in “Migration terminated abnormally” status or the last job status.  This allows the migration administrator to determine which migrations have exceeded the maximum allowable number of automatic restarts, and indicates that the file will need additional remediation before resetting and remigrating the user.  It should be noted that the migration server will contain a history (in the form of a migrated message table) corresponding to all of the data that has been migrated up to that point. After remediation, the migration administrator can simply reset and requeue the user to continue the migration from this point, or clear previously migrated data and clear the migrated message table to restart the migration from the beginning.

  • If a migration has been automatically restarted by this process, the Notes Migrator.nsf database will only show the status of the last migration that was restarted. Consider the situation where a migration crashed, was automatically restarted, and continued to completion without further errors. The migration status in the Migrator for Notes Database will report “Migration completed successfully” even though there was a crash on the initial run.  The migration history will indicate the first migration completed with a status of “Migration terminated abnormally” status, but this may not be evident in the Notes Migrator.nsf Lotus Notes application interface.  Advanced or customized installations of the Lotus Notes application may retrieve and summarize the complete migration history for a user, but a complete migration history is not included in the default Notes Migrator.nsf application interface.

The RetryJobStatus parameter is a string list of semicolon delimited codes that will automatically be resubmitted if the job returns with that specific status code. By default, Migrator for Notes will resubmit memory exception codes #001 (Notes memory exception) and #002 (Outlook memory exception). 

Summary

In this section, we have reviewed how the Migrator for Notes Migration Worker application detects a crash of the migration engine, how the Migrator for Notes Control Center Server processes the “Migration Terminated Abnormally” status and requeues the job if necessary, and ultimately how the migration automatically restarts on the same workstation.

In this manner, the Automatic Restart process in Migrator for Notes allows migration administrators to queue jobs to the migration server and know that if a crash were to occur the migration will automatically be restarted without intervention up to the maximum number of times indicated by the MaximumRetryCount parameter.  This feature reduces the amount of resources required to monitor Migrator for Notes migrations and eliminates the need for administrator intervention in the case where the migration engine has crashed.

Again, it should be noted that the Lotus Notes application interface (Notes Migrator.nsf) will report the last migration status for the user.  Any automatic restart events will be evident in the complete migration history but will not be reflected directly in the interface.  In the case where the maximum restart count has been exceeded, the last migration event will be reported as “Migration terminated abnormally.”

 

Detecting abnormal termination of a migration

Overview

Automatic Migration Restart is a process by which any abnormal termination of the Migrator for Notes engine (commonly referred to as a ‘crash’) is identified, and the migration job is automatically re-queued.  If this occurs, the migration is restarted and resumed on the next document in the message store, until either the migration completes process the message store, or the maximum number of retries has been exceeded.

Automatic Migration Restart works in conjunction with the existing migration recovery features in Migrator for Notes to resume a migration on the next document immediately after the document which caused the crash. 

Terminology

This following table lists and describes the terms commonly used when describing the automatic restart process.

Term

Description

Migrator for Notes

Migrator for Notes application environment, encompassing the server instance.

Job

A migration event. Within the context of Migrator for Notes, each migration job is a unique event.

Migration server

Refers to the system running the Migrator for Notes web services. This is the primary instance where the web server and CMT Monitor are installed.

Migrator for Notes Database (or Notes Migrator.nsf)

Used for configuring settings, managing user records, and queuing users to the migration server.  

Migration worker

An instance of the CMT_MigrationWorker.exe application running on a migration workstation. This application runs continuously on the migration workstation, polls the migration server for pending migration jobs, and initializes an instance of the migration engine (CMTProxy.exe) for each migration job.  

Migration engine (or CMTProxy)

An instance of CMTProxy.exe, which is initialized and managed by the Migration worker. CMTProxy.exe performs the actual data migration.

Crash

Abnormal termination of the migration engine, resulting in a status of “Migration terminated abnormally.”

Configuring Automatic Restart

Automatic restart is controlled by a parameter in the Web.config file for the Migrator for Notes server.    This file is located in [installDirectory]\CMT for Exchange\CMT_XMLServer\Web.config , for example “C:\Program Files (x86)\BinaryTree\CMT for Exchange\CMT_XMLServer\web.config”.

The Web.config file contains the startup parameters for the Migrator for Notes server.    Located in this file is an \appSettings\MaximimumRetryCount XML node, which indicates the maximum number of times a migration will be automatically restarted.

    <appSettings>

        ---snip ----

        <add key="ScheduleTimerInterval" value="3"></add>

        <add key="UseSQL" value="1"></add>

        <add key="MaximumRetryCount" value="10"></add>

<add key="RetryJobStatus" value="#001;#002"></add>

    </appSettings>

 

The MaximumRetryCount is set to 10 by default.  A value of 0 or a negative number will disable the Automatic Restart functionality.  

RetryJobStatus is set by default to resubmit if the #001 (Notes memory exception) or #002 (Outlook memory exception) job status code is returned. 


If MaximumRetryCount or “RetryJobStatus” is changed or disabled, the Migrator for Notes server instance must be restarted for the change to take effect.  This can be done by opening a command prompt on the migration server and typing “iisreset” at the prompt, or manually restarting the IIS service.

Migrator for Notes Worker Basics

The Migrator for Notes Worker is an application that runs in the task bar of windows. This application is responsible for queuing, migrating, and updating the status of a migration job on the Migrator for Notes server.   

The Migrator for Notes Worker has three basic states:

  • Offline – the migration worker is not started.

  • Online – the service is running, and actively checking for migration  jobs

  • Migrating – a migration job is in progress.

When the Migration Worker application is online, the application periodically checks the Migrator for Notes server for pending migration jobs. If a job is available, the migration server passes back the job number from the migration queue, along with settings and user information required to start the migration. The worker then starts the migration by calling CMTProxy.exe as a separate process with the appropriate parameters, which begins the actual data migration. Once the migration starts, the migration worker application sends notification back to the Migrator for Notes server that the workstation is in Migrating status. During the course of the migration, the migration worker periodically communicates throughput and migration time statistics back to the migration server.

When the migration has completed, the Migration Worker updates the CMT server and then uploads the history of migrated messages and relevant log files.

The key concept is that the Migration Worker is responsible for checking the Migrator for Notes Control Center Server for work, triggering a migration job when required, updating the Migration Control Center Server during the migration, and finally uploading log files once the job has been completed.

Migrator for Notes Recovery Basics

During a migration, the Migrator for Notes engine reads and catalogs the unique identifier of each message it encounters before the message is processed. This identifier is stored in memory to prevent migrating a document multiple times, and is also immediately written to disk in a “ProcessedNoteID-username.txt’ file on disk, where username is the unique key of the user on the migration server.

In the event of a crash, the ProcessedNoteID file on disk contains the list of unique identifiers of each document processed during the failed migration. This file is read by the Migration Worker application, merged with any existing migrated message information residing on the server for the user, and stored back on the migration server prior to the next migration. This ensures that the migrated message table on the Migrator for Notes Server is up to date when the migration is restarted.  

When the migration engine is restarted, the migrated message table is read and all previously processed messages (including the ID of the message that was being processed when the migration crashed) are skipped in subsequent runs. This allows Migrator for Notes to resume where it left off, and prevents re-migrating documents repeatedly on subsequent runs.

 


The migrated message table is not used if the Discover History from Migrated Data option is used. In this case, the previously processed messages are read from the Exchange Mailbox directly.

The Migrator for Notes Worker application is a key component in detecting when a migration crashes. When the Migrator for Notes Worker application starts a migration job, it launches the migration by calling CMTProxy.exe in a separate process.  While the migration is running, the Migrator for Notes Worker application periodically sends information back to the Migrator for Notes server indicating migration time and throughput for the current migration.   

If the migration engine (CMTProxy.exe) crashes, the handle to the migration engine becomes invalid, and the Migration Worker application knows that the currently running migration has crashed. At this point, the Migration Worker reads the ProcessedNoteID table from the temporary directory, merges it with any existing information in the Migrated Message table on the Migrator for Notes server, and updates the status of the job to “Migration Terminated Abnormally.”


The Migration Worker does not actually re-queue the job, it reports only “Migration Terminated Abnormally” status back to the Migrator for Notes server. The Migrator for Notes server is responsible for determining if the job will be re-queued.

Automatic re-queue of a migration job

The sections above describe how the Migrator for Notes Worker application identifies a crash in the migration engine and returns “Migration terminated abnormally” status.  This section describes how the Migrator for Notes server re-queues the user and resulting migration status values.

If the migration worker application returns a value of “Migration terminated abnormally” or one of the RetryJobStatus codes to the Migrator for Notes server, the migration server determines if the RetryCount exceeds the MaximumRetryCount configured for the server.     

  • If the RetryCount is less than the MaximumRetryCount, the job is re-queued by creating a new migration job in the migration queue.  This job will be a high-priority job queued for the specific workstation, ensuring that it is the next job processed by that workstation.

  • If the RetryCount for a particular migration is greater or equal to the MaximumRetryCount, the migration will be left in “Migration terminated abnormally” status or the last job status.  This allows the migration administrator to determine which migrations have exceeded the maximum allowable number of automatic restarts, and indicates that the file will need additional remediation before resetting and remigrating the user.  It should be noted that the migration server will contain a history (in the form of a migrated message table) corresponding to all of the data that has been migrated up to that point. After remediation, the migration administrator can simply reset and requeue the user to continue the migration from this point, or clear previously migrated data and clear the migrated message table to restart the migration from the beginning.

  • If a migration has been automatically restarted by this process, the Notes Migrator.nsf database will only show the status of the last migration that was restarted. Consider the situation where a migration crashed, was automatically restarted, and continued to completion without further errors. The migration status in the Migrator for Notes Database will report “Migration completed successfully” even though there was a crash on the initial run.  The migration history will indicate the first migration completed with a status of “Migration terminated abnormally” status, but this may not be evident in the Notes Migrator.nsf Lotus Notes application interface.  Advanced or customized installations of the Lotus Notes application may retrieve and summarize the complete migration history for a user, but a complete migration history is not included in the default Notes Migrator.nsf application interface.

The RetryJobStatus parameter is a string list of semicolon delimited codes that will automatically be resubmitted if the job returns with that specific status code. By default, Migrator for Notes will resubmit memory exception codes #001 (Notes memory exception) and #002 (Outlook memory exception). 

Summary

In this section, we have reviewed how the Migrator for Notes Migration Worker application detects a crash of the migration engine, how the Migrator for Notes Control Center Server processes the “Migration Terminated Abnormally” status and requeues the job if necessary, and ultimately how the migration automatically restarts on the same workstation.

In this manner, the Automatic Restart process in Migrator for Notes allows migration administrators to queue jobs to the migration server and know that if a crash were to occur the migration will automatically be restarted without intervention up to the maximum number of times indicated by the MaximumRetryCount parameter.  This feature reduces the amount of resources required to monitor Migrator for Notes migrations and eliminates the need for administrator intervention in the case where the migration engine has crashed.

Again, it should be noted that the Lotus Notes application interface (Notes Migrator.nsf) will report the last migration status for the user.  Any automatic restart events will be evident in the complete migration history but will not be reflected directly in the interface.  In the case where the maximum restart count has been exceeded, the last migration event will be reported as “Migration terminated abnormally.”

 

関連ドキュメント

The document was helpful.

評価を選択

I easily found the information I needed.

評価を選択