Chat now with support
Chat with Support

Metalogix Content Matrix 9.3 - SharePoint Edition User Guide

Introduction Entering the License Key Content Matrix Console End User Interface Enabling Advanced Mode Connecting and Migrating Account Requirements Connecting to SharePoint Preparing for a Migration How Content Matrix Handles Various SharePoint Components
How Team Sites are Migrated to Modern Team Sites How Publishing Sites are Migrated to Communication Sites How MySites are Migrated How Records Centers are Migrated to SharePoint Online Using the Import Pipeline How Master Pages are Migrated How List Template Galleries Are Migrated How Content Types are Migrated Document Version and Checked Out File Limitations How Ghosted and Un-Ghosted Pages are Migrated How Document Sets are Migrated How Navigation is Migrated How "Share With" Metadata is Migrated to SharePoint Online Using the Import Pipeline How Web Parts are Migrated How Site Themes are Preserved How Site Features Are Migrated How Managed Metadata, Terms, and Term Stores are Migrated How Users and Permissions are Migrated How BCS/BDC Data is Migrated How OneNote Notebooks are Migrated How Customized Form Pages are Migrated How InfoPath Forms are Migrated How SharePoint Workflows are Migrated How Nintex Workflows and Forms are Migrated How Link Correction Works
Initiating a Migration Configuring Copying Options Saving or Running a Migration Job Copying SharePoint Objects as a Separate Action Converting Classic Pages to Modern Pages After a Migration Incremental Migration Using PowerShell with Content Matrix
Configuring PowerShell for Use with Content Matrix Creating and Running a PowerShell Script Content Matrix PowerShell Commandlet List
Metalogix.System.Commands Metalogix.SharePoint.Commands Metalogix.Jobs.Reporting.Commands Metalogix.SharePoint.Migration.Commands
Modifying Content Matrix Configuration and Settings Configuring Content Matrix for Distributed Migration Frequently Asked Questions
DB_Owner Permission Keyboard Shortcuts License Key Troubleshooting Determining the Status of a Migration Running in the Background MySite and User Profile Migration Issues Optimal Setup for Best Performance Migrating SharePoint On Premises Using Proxies, Load Balancing or Alternate Access Mappings Troubleshooting the Extensions Web Service Installing the Extensions Web Service on Specific Systems Extensions Web Service Installation Files Using Older Versions of the Extensions Web Service Migrating JavaScript Migrating Site Variations Migrating with SharePoint Document IDs Changing a Job Configuration for Multiple Files SharePoint 2013 and Later Site Collection Creation Issue Job List Database fails to Load After Upgrade Customized Wiki Page Web Part Zones Not Being Migrated Preserving SharePoint List Item IDs with a CSOM Connection Type Retrying Failed Document Copies to O365 CSOM Connections Migrating Content When the Source Custom List Template is Missing Are SharePoint Online Migrations Throttled? What to Expect when Migrating with StoragePoint How Do I Identify and Remove Containers and Queues from My Azure Private Storage Account? Import Pipeline Batches Getting Stuck "Error, the batch will be resubmitted" Message Displays When Using the Import Pipeline Item Count at Target is Increased When Migrating Using the Import Pipeline Custom Lists with Versioning Enabled Not Being Migrated When Using the Import Pipeline Supported Nintex Actions for SPO Migration Support for GCC High Tenants
About Us

Preserving SharePoint List Item IDs with a CSOM Connection Type

Question

Is it possible to Preserve SharePoint list item IDs when migrating with a CSOM (Client Side Object Model) or Office 365 Tenant connection type on the target?

Answer

Yes, it is possible to preserve SharePoint list item IDs when migrating content to a target with a CSOM/Tenant connection.

Item IDs and Document IDs are a little different in SharePoint (not the SharePoint 2010 or 2013 Document ID feature), and this means that how they can be accessed through a CSOM connection is a little different. Preserving Document IDs will only work if an OM (Object Model) connection is made to the target, through either a Local OM or Remote OM (Extensions Web Service) connection. There are some limitations that can be encountered when trying to preserve Item IDs that are dependent on the connection type. There are three basic connection types that we need to keep in mind when looking at these limitations. They are:

·Local or Remote OM (Object Model) Connection - This connection type is made either when the Metalogix Content Matrix Console client application is installed directly on the target SharePoint server, or a remote connection is made using the Metalogix Extensions Web Service. When this type of connection is made, Metalogix Content Matrix will be able to directly set the list item ID value in SharePoint through the Object Model. This value can be set as long as that specific value is not already being used by another list item.

·SharePoint CSOM (Client Side Object Model) or Office 365 Tenant Connection - This connection type is made when remotely connecting to an Office 365 environment. When connected to SharePoint Office 365 using a CSOM connection, Metalogix Content Matrix cannot directly set the list item ID value so another method must be used instead. The CSOM method for preserving the list item ID value is for Metalogix Content Matrix to create a temporary item and then deleting it. This creation and deletion process is done in batches and each item that is added will increment the ID value by 1. This will continue until the Metalogix Content Matrix reaches the necessary item ID value. If the target ID value is already in use, or if another item with a higher ID value exists in the list, then the item(s) will be added to the end of the target list (and not at its ID value). There are a few limitations to this method:

oOlder items that are placed into newer folders - The general case for this is if an older item, that has a lower ID number value is moved and placed into a folder that was more recently created (and has a higher ID number value). In this case, the items with the lower ID number values will be added after the folder is created, so their ID number values will not be preserved. For example, if a list item with an ID value of "12" is moved into a folder, and the folder has an ID value of "25". In this case the ID value for the item will not be preserved because the folder it is located in will not be created until ID 25 is created. So Metalogix Content Matrix will hold onto the item until the folder is created and then add it, but because the item ID has already been passed, its value cannot be preserved, and it will be given a new value by SharePoint after all other items have been added.

oLists containing default data - The general case for this scenario is when a SharePoint list (and not a Document Library) is created and SharePoint automatically adds a number of items (one or multiple) by default into that list. The default items that are created by SharePoint will also automatically be assigned an item ID value by SharePoint. When migrating this type of list their data will be deleted before the source content is migrated. The default added items will have ID values associated with them, and when the items are deleted those ID values will no longer be available, and Metalogix Content Matrix won't be able to track what the values were associated to. This can interfere with the ID value calculations that are made, and throw off the ID values that are being set during migration.
 
The most common place where users can see default list items created is when an "Announcements" list is created. Announcement list that are created by SharePoint will automatically create one item by default that is the "Welcome" message. However, when migrating with Metalogix Content Matrix and using the override option, users should not see this issue with Announcement lists because Metalogix Content Matrix will delete the list that is created by the site template, and create a new one. When the new list is created Metalogix Content Matrix is able to add the list items directly, and allows for the list item ID value to be set in the process. This is more likely to be an issue with SharePoint lists that have had their template modified to automatically generate a set of default items.

NOTE: When migrating SharePoint Task lists to a CSOM Office 365 connection, it is recommended that the Preserve IDs on items/folders in Lists option is selected. This is to help preserve the Related Items property for items in the Task list. If unchecked there is a small chance that the Related Items property may list incorrect files, because some of these files may be referred to through the Item ID value.

Retrying Failed Document Copies to O365 CSOM Connections

Question

When I try to migrate some of my content into O365 using a SharePoint CSOM or Office 365 Tenant connection type, and this content contains a large Document Library, I see that some of my documents fail to migrate. The logs for the failure indicate that a "HTTP 500 - Internal Server Error" was thrown. What is the cause of this? And how can I fix it?

Answer

There is a potential issue when migrating specifically to O365 where users can encounter an "HTTP 500" error. This is caused by a combination of the CSOM/Tenant connection adapter and SharePoint's document upload methods. To help resolve this issue there is a document retry feature that can be configured. This retry feature will attempt to upload the document(s) into the O365 target, and depending on the configured setting, will attempt to retry the upload process if it fails or times-out on the initial try.

The below steps will explain how to enable and configure this retry setting.

NOTE: This retry method is only meant for migrating to O365 targets. If you are migrating to an on premises target, then this retry feature will not benefit you since it makes use of O365 specific methods.

1.Make sure that the Metalogix Content Matrix client application is closed.

2.In the file system that the Metalogix Content Matrix Console client application is installed on, navigate to the the appropriate file location:

·If you are required to be a local administrator on the machine where Content Matrix is installed:  C:\ProgramData\Metalogix

OR

·If you are not required to be a local administrator on the machine where Content Matrix is installed: C:\Users\<username>\AppData\Roaming\Metalogix\Common

3.In this location there should be an EnvironmentSettings.xml file. Open this file in an editor program. For example, Notepad, etc.

4.There are two variables that will need to be modified in order for the retry method to work. They are:

·CSOMDocumentRetriesNumber - This value determines the number of times that Metalogix Content Matrix will make another attempt to upload/migrate any document(s) that has failed the initial document upload try, when migrating to a O365 CSOM target.

·CSOMDocumentRetriesDelay - This value determines the amount of time, in seconds (s), that Metalogix Content Matrix will wait before starting a document upload retry, and is based on the above ("CSOMDocumentRetriesNumber") variable.

NOTE: The default values for these two variables will be set as "0". Users can set these values as desired, based on what works best for their environment.

5.After the desired values have been entered in the two variables, save and close the file.

6.Restart the Metalogix Content Matrix client application. The changes should now be in Metalogix Content Matrix Console, and used when running any migration to a SharePoint Online target environment.

After these values have been set and the client application has been restarted, any documents that fail on the initial migration attempt we be retried.

IMPORTANT:  If these steps do not resolve the issue and you are using the Import Pipeline for the migration, the error may be pipeline-related.  See "Error, the batch will be resubmitted" Message Displays When Using the Import Pipeline for details.

Migrating Content When the Source Custom List Template is Missing

Question

I have a list on my source that uses a custom list template (ex. a "Fab 40" list) and I want to migrate that list to my new target SharePoint environment. However, I am unable to install that template on my target before migration. How will Metalogix Content Matrix deal with this?

Answer

If at all possible, it is recommended that users install any custom list templates on the target environment before migration. That would allow Metalogix Content Matrix to use those templates when migrating, and trying to reproduce the same results on the target. But there are some customizations within templates that Metalogix Content Matrix is unable to migrate content or metadata into, such as the "Fab 40" templates.

In some cases users are unable to install the custom templates in the target environment, such as when migrating to O365. In these cases Metalogix Content Matrix will attempt to find the base template of the source customized list. Then it will create a list on the target using that same base template, and migrate the content into that list.

To help with this two variables have been added to the list XML attributes. They are:

·BaseTemplate - This is the value for the custom list template.

·BaseType - This is the value for the default template that any custom template is based on. If the custom template does not exist on the target, this template will be used for the list creation instead. Please see http://msdn.microsoft.com/en-us/library/microsoft.sharepoint.client.basetype(v=office.14).aspx for more information on the SharePoint BaseType value.

These variables will help Metalogix Content Matrix determine which base template to use so that an "Issue Tracking" list won't be created in a case where it should be a "Custom List" template.

Let's look at an example where you have a custom list on the source, and that list has a BaseTemplate value of "1011", a BaseType of "0", and has not been installed on the target. In this case, Metalogix Content Matrix will look at the source list and find the BaseTemplate value of 1011. It will then look on the target to see if there is a template that has the same value. If it finds one, then Metalogix Content Matrix will create a list with that template and migrate the data over. But, if the BaseTemplate value is not found on the target, then Metalogix Content Matrix will look at the BaseType value instead, since this value should exist by default in the target SharePoint environment. In this example we also happen to know that a BaseType value of "0" is a "Custom List" or "Generic List" type. So Metalogix Content Matrix will create a list on the target, using the "Custom List" template, and will then migrate the source data into that custom list.

Are SharePoint Online Migrations Throttled?

Question

I am migrating some content to SharePoint Online/Office 365 (SPO/O365). Are migrations to Office 365 throttled? And if so, how does Metalogix Content Matrix deal with this?

Answer

Before we can answer these questions, there is some information that we need to review first.

SharePoint Online (SPO) is on the open Internet, and is a multi-tenant environment in which multiple MS customers are hosted on a single SharePoint Server Farm. In light of that, Microsoft uses a number of mechanisms to protect SPO customers' environments, and the integrity of these Server Farms.

Based on internal performance benchmarking, there is an impact of between 33% and 40% on the performance of migrations to SPO because of some of these necessary protection mechanisms. Our discussions with the SPO product group, and our experience with numerous customers who have migrated large farms to SPO have shown that the following are among some of the known factors that impact this performance:

·User-based throttling - This ensures that any single user cannot perform so many simultaneous operations on their SPO environment, that these would cause performance issues for other users within the same or other tenants. Refer to the Microsoft article Avoid getting throttled or blocked in SharePoint Online. In the past, if a migration was performed using a single username/password combination, migrations were significantly slower than migrations that involve multiple username and password combinations. However, Microsoft is starting to reduce user-based throttling and increase tenant-based throttling. Given that the rollout of throttling changes occurs over a period of time rather than all farms at once, it is difficult to say if the farm you are migrating into has been impacted by these changes yet.

·Tenant-based throttling - This ensures that no one tenant on a multi-tenant farm can use so many resources, that other tenants' performance suffers significantly. Microsoft started using this to protect tenants on multi-tenant environments. We fully expect this to be the most significant throttling mechanism in use by Microsoft moving forward.

·Farm-based throttling - This ensures that if tenants on the farm are seeing low performance across the farm, no single tenant can create significant additional workload on the farm.

You can reduce throttling during migrations by connecting to SharePoint Online using Office 365 OAuth Authentication.

Related Documents

The document was helpful.

Select Rating

I easily found the information I needed.

Select Rating