When Content Matrix is configured to run using Distributed Migration, there are some intermittent issues that can be encountered if migrations are scripted using PowerShell. When running migrations using Distributed Migration, and the configuring and running of the migration jobs are done directly (and only) through the Content Matrix User Interface (UI), no intermittent issues are encountered or observed. The intermittent issues only occur when the migrations are scripted to run using PowerShell.
Possible symptoms include:
The above symptoms would show up intermittently, where jobs would run on an Agent with no issues for a while, and then suddenly encounter one of the error messages. The Agent would then stop working unless the status was manually forced through in the Distributed Migration database. And after the force through, the agent would work fine again for a while.
The intermittent issues of jobs failing to run is occurring because multiple instances of the Distributed Migration Job Scheduler service are running at the same time on the Controller. When the Content Matrix console is open and running (with Distributed Migration configured), it will run an instance of the Job Scheduler automatically to look for and process any migration jobs that are being pushed out to the Agents. This allows the Agents to be sent jobs as soon as they are approved on the Controller machine.
When scripting the migration jobs via PowerShell, the PowerShell scripts also start up their own instance of the Job Scheduler to push the jobs out to the Agent machines to run them.
As a result, these two instances of the Job Scheduler service were coming into conflict when trying to process some of the PowerShell scripts, but this was happening and being reported back to Content Matrix intermittently depending on which instance of the Job Scheduler service picked up the jobs to run.
There are two ways to resolve this issue, and which one to use depends on the question: Are you keeping the Controller console open?
1. Scripting the job via PowerShell and keeping the Controller console OPEN
There are some modifications that can be made to any existing or new PowerShell scripts to ensure that these intermittent errors are no longer encountered.
In the PowerShell script find the following line of code:
$remoteJob = Copy-MLSharePointList -Source $source -Target $target -OverwriteLists -CopyListItems -CopySubFolders -RunRemotely -Certificate "JobsStalled" -AgentDatabase $agentDatabase -JobTitle ""
Now, add the attribute “-TriggerJobOnly” into the script like below:
$remoteJob = Copy-MLSharePointList -Source $source -Target $target -OverwriteLists -CopyListItems -CopySubFolders -RunRemotely -TriggerJobOnly -Certificate "JobsStalled" -AgentDatabase $agentDatabase -JobTitle ""
This will put the migration job in the “Triggered” state in the Distributed Migration Database, which will prevent a new instance of the Job Scheduler service from starting up. However, it does require that the Controller is open and running. Otherwise, the jobs will sit in the migration queue and will wait for the Controller to open so they can run.
In addition, users should also remove any instances of “[Metalogix.Actions.Remoting.RemoteJobScheduler]” from the script and related scripts. These references would no longer be needed since they are providing commands to the scheduler service, and they are already being handled by running Controller console.
For example, any instances of the following PowerShell script code could be removed:
2. Scripting the job via PowerShell and keeping the Controller console CLOSED
The second option is to continue to exclude the “-TriggerJobOnly” attribute from the PowerShell script. If following this method, then, users would need to make sure that the Controller machine is closed for the full running of the migration jobs, even to check logging. Keeping the Controller closed allows the PowerShell instances to start up their over instance of the Job Scheduler service, while preventing the conflict from a running Controller console.
This means that no changes to the PowerShell scripts should be needed, and users would need to make sure that the “-TriggerJobOnly” attribute is not listed inside the scripts.
Once the migration jobs are finished running, users would be able to open the Controller to view and manage the migration logs.
