Question

I am migrating some content to SharePoint Online/Office 365 (SPO/O365). Are migrations to Office 365 throttled? And if so, how does Metalogix Content Matrix deal with this?

Answer

Before we can answer these questions, there is some information that we need to review first.

SharePoint Online (SPO) is on the open Internet, and is a multi-tenant environment in which multiple MS customers are hosted on a single SharePoint Server Farm. In light of that, Microsoft uses a number of mechanisms to protect SPO customers' environments, and the integrity of these Server Farms.

Based on internal performance benchmarking, there is an impact of between 33% and 40% on the performance of migrations to SPO because of some of these necessary protection mechanisms. Our discussions with the SPO product group, and our experience with numerous customers who have migrated large farms to SPO have shown that the following are among some of the known factors that impact this performance:

·User-based throttling - This ensures that any single user cannot perform so many simultaneous operations on their SPO environment, that these would cause performance issues for other users within the same or other tenants. Refer to the Microsoft article Avoid getting throttled or blocked in SharePoint Online. In the past, if a migration was performed using a single username/password combination, migrations were significantly slower than migrations that involve multiple username and password combinations. However, Microsoft is starting to reduce user-based throttling and increase tenant-based throttling. Given that the rollout of throttling changes occurs over a period of time rather than all farms at once, it is difficult to say if the farm you are migrating into has been impacted by these changes yet.

·Tenant-based throttling - This ensures that no one tenant on a multi-tenant farm can use so many resources, that other tenants' performance suffers significantly. Microsoft started using this to protect tenants on multi-tenant environments. We fully expect this to be the most significant throttling mechanism in use by Microsoft moving forward.

·Farm-based throttling - This ensures that if tenants on the farm are seeing low performance across the farm, no single tenant can create significant additional workload on the farm.

You can reduce throttling during migrations by connecting to SharePoint Online using Office 365 OAuth Authentication.

Question

I am running out of storage space in my Azure private storage account. How do I remove containers and queues created by Content Matrix to free up space for more migration actions?

Answer

While you can use the Microsoft Azure Storage Explorer UI to delete items individually, you can use Azure PowerShell commands to simultaneously remove all containers or queues that were created on the same date.  A prerequisite to removing multiple containers and queues is that Azure PowerShell must be installed and configured. Refer to the Microsoft Azure PowerShell documentation for installation instructions and general information about Azure PowerShell Cmdlets.

Naming Conventions

Beginning in version 9.3, Content Matrix uses naming conventions that allow you to easily identify containers and queues created by Content Matrix during migration using the Import Pipeline.  

To remove items from your Azure blob storage account using Azure PowerShell:

1.Ensure there are no active migration actions running that use the Azure storage where the items are stored.

2.Open Azure PowerShell.

3.Connect to Azure using the New-AzStorageContext cmdlet. Your account name and account key are required parameters. Note that these are different from your Client ID and Application Secret.        $context = New-AzStorageContext -StorageAccountName $accountName -StorageAccountKey $accountKey

4.Use the information in the following table to determine the appropriate command to use.

Question

I am migrating some content using the Import Pipeline.  After some time the batches seem to simply stop. Why am I facing this issue?

Answer

Migration duration and overall performance using the the Import Pipeline may vary due to factors outside the control of Metalogix Content Matrix. Migration processing resides within Microsoft's domain and is subject to Microsoft's internal processes, such as maintenance and upgrades.

You may experience a delay of several hours before batch completion if the machines that Microsoft uses undergo maintenance/upgrades or if there are O365 service issues/outages (as can be seen in the O365 Admin Center under Service Health).

Question

What does it mean when the message ERROR, the batch will be resubmitted to the import pipeline displays in the Information section of the Log Item Details dialog when a migration is performed using the Import Pipeline?

Answer

You may see this message (which will begin with either ERROR or FATAL ERROR) during a migration when a batch fails to migrate in its entirety.  To address this issue, increase the value of the environmentsettings.xml key MaxAzureBatchRetryCount.  This key controls the maximum number of times Content Matrix will resubmit the batch until it is successfully migrated.  (The default value is 5.)

The batch retry mechanism will also be triggered for the following errors:

·Please try again. CallStack -- at Microsoft.SharePoint messages

·Access denied

·Save Conflict The system cannot find the path specified. (Exception from HRESULT: 0x80070003)

·Updates to system roles is not allowed.

·List does not exist

For any error not listed above (such as Item Does Not Exist, which is different than List does not Exist), a batch will be resubmitted only if the error message is added to the environmentalsettings.xml key ErrorMessagesForRetryAzureBatch.

IMPORTANT:  If adjusting this setting does not resolve the issue, the error may be with the CSOM connection.  See Retrying Failed Document Copies to SharePoint Online CSOM Connections for details.