Functional and API Differences Between SPO and SharePoint that Impact Migrations
When migrating to an SPO target, factors to consider include:
·Different Application Programming Interfaces Available for SPO
AND
·OAuth and Migration Performance.
Different Application Programming Interfaces Available for SPO
For on-prem SharePoint, Microsoft offers a very rich API known as the SharePoint Server Object Model. This has been the most frequently used and richest API to access SharePoint on-prem.
However, due to the multi-tenant nature of SPO, Microsoft has to ensure that no single tenant can compromise other tenants environments. In light of that, Microsoft cannot expose the full SharePoint Object Model in SPO. They currently expose five APIs that are useful for migrations:
·The Client Side Object Model (CSOM)
·The SharePoint REST API
·SharePoint Online Remote PowerShell (subset of CSOM)
·Microsoft 365 Remote PowerShell (subset of CSOM)
·A migration-focused API that leverages Azure Blob Storage and batching to migrate list items and documents significantly more quickly to SPO (available both from CSOM and REST)
While the CSOM API is becoming close to the SharePoint Server Object Model in terms of feature parity, it is very slow for migrations. In light of that, Microsoft launched a Migration API using the Azure and SPO-Provided Containers, collectively known as the Import Pipeline. Today, the combination of the CSOM API and the Azure/SPO-Provided Containers is used by Content Matrix to write content to SPO. SPO and the CSOM API has some limitations that make Content Matrix and other migration tools unable to migrate all on-prem SharePoint content. For a full set of SPO limitations, see this MS web page listing. Another important page to look at is SharePoint 2013 deprecated functionality. This applies to both SharePoint on premises and SPO.
OAuth and Migration Performance
Quest recommends using OAuth Authentication when connecting to SharePoint Online for optimal migration performance. Using OAuth should result in reduced throttling of migration workloads.
For more details: https://www.quest.com/community/blogs/b/microsoft-platform-management/posts/impact-of-sharepoint-online-throttling-on-migrations.
Performance of Migrations Using the Import Pipeline
There are three primary factors that until now have impacted performance of migrations to SPO dramatically:
I.The physical distance between your on-prem SharePoint farm and your SPO tenant, and the fact that in most cases, this Wide Area Network (WAN), is across the Internet, making it very unpredictable.
II.The protection mechanisms and throttling that Microsoft has in place to protect their multi-tenant clients from abuse by other tenants on the same farm.
III.The very chatty nature of the CSOM API that is the primary API used for migrations to SharePoint Online, compounded by the distance required for each API call to travel between on-prem environments and SPO.
Microsoft developed the Import Pipeline to alleviate the throttling and protection mechanisms and the chattiness of the CSOM API:
I.Site collection structure, site structure, and list structure are still migrated using CSOM as they were before, and this happens prior to list items being migrated.
II.Instead of items being migrated with one API call at a time, they are now packaged locally into a batch with a corresponding manifest XML file to store permissions, metadata, etc. before being sent to SPO.
III.Instead of items/batches being sent directly to SPO, they are instead sent to an Azure Blob Storage location (provided by either Microsoft or the user).
IV.Once a batch has been successfully uploaded to the Azure Blob Storage location, Content Matrix uses a CSOM request to ask SPO to pull content from the Azure Blob Storage location into the previously created list in SPO.
Please see image below:
The net result is a significant increase in performance of list items, that can be anywhere between 5 to 30 times faster than these migrations were before depending on a number of factors.
In light of this, it is strongly recommended that our clients leverage the Import Pipeline for all SPO migrations. Users can either provide their own private (paid) Azure containers (as detailed later in this document), or SPO-provided (free) containers.
Refer to the Microsoft article Avoid getting throttled or blocked in SharePoint Online for tips on how to avoid throttling.