Enterprise Vault to PST
This section explains what the requirements are for an Enterprise Vault (EV) to PST migration, and then covers the basic steps that should be followed to start such a migration.
Stage 1 - Synch
As soon as a user is enabled for migration, the synch process-flow shown above starts.
Archive Shuttle Core sends a command to collect all needed metadata information such as the number of items, size, and Enterprise Vault transaction IDs for the archive of the user. The results are reported back to Archive Shuttle Core to allow item level tracking and auditing.
The Enterprise Vault Export and Native Format Import module then begin to work. This runs continuously in the background until the administrator initiates a Switch (Stage 2).
|
|
NOTE: PST files will be created by default of about 2 Gb in size. A lower size can be set in the UI if required. |
Stage 2 - Switch
After the Stage 2 ('Switch') phase to the target environment is initiated, the configured workflow starts, and by default determines the gap between the source archive and the last imported item. Archive Shuttle then synchronizes the difference one last time to the target mailbox.
Archive Shuttle performs a Zap of the mailbox to remove Enterprise Vault settings before closing the PST files and moving them to the PST Output Area. Finally, shortcuts are cleaned up in the source mailbox by deleting them, and any pending-archive items are changed back to normal items.
Requirements
The following is a list of requirements.
|
Requirement |
Description |
|---|---|
|
Modules installed on source |
The EV-type modules should be installed on each appropriate source Enterprise Vault server involved in the migration. The EV import module is not required on these machines. |
|
Modules installed on appropriate bridgehead |
The Native Format Import module, and if needed, the post processing module should be installed on a bridgehead server. |
|
AD Collector module installed |
An AD Collector module should be installed so that it can collect user-level information for the migration project. |
|
Staging Area created |
One or more staging areas should be setup, of an appropriate size and should be excluded from virus scanners. |
|
PST Output Area created |
One or more PST Output areas should be setup, of an appropriate size and should be excluded from virus scanners. It is this location where finalized PSTs will be place. |
|
PST Temporary Path created |
A PST Temporary Path should be setup, of an appropriate size and should be excluded from virus scanners. It is this location where temporary PSTs will be placed. |
|
Workflows reviewed |
The Stage 2 workflows should be reviewed, and if necessary, customized to meet the needs of the project. |
|
Apply appropriate failed item threshold |
It is best practice to find an appropriate failed item threshold for the project/customer and apply that at the links level. This way all mappings which are created will inherit this value. |
Basic steps
The following are the basic steps to start this type of migration.
|
Step |
Screen |
Description |
|---|---|---|
|
Enable Modules |
Modules |
All Archive Shuttle modules must be enabled, and optionally a schedule for them defined. It is important to verify that none of the modules have a red background (this indicates that Archive Shuttle Core has not had contact with the module) and that the module versions are as expected. |
|
Enable Domains |
Active Directory |
Select and enable one or more domains for synchronization. |
|
Add EV Environment |
EV Environment |
Add the Enterprise Vault source environment. |
|
Create PST Link |
Links |
Create a PST link, and ensure that the Target modules are associated with it. Configure the PST Output Path, and PST Temporary Path. |
|
Set Watermarks for Output Path & Temporary Path |
Links |
With the PST link selected, go to the PST tab, and set the watermarks for the PST Output location. |
|
Add Source Links |
Links |
Configure all appropriate source links. Select the links, and then click on Map Modules, create the item database(s) and enable archive gathering. Ensure there is no import module specified on the links (this will only confuse things later). |
|
Add Staging Area |
Links |
Ensure that the Default Staging Area is configured correctly. Depending on the migration, the free space, and the number of vault stores being used as the source for the migration, the high water mark can be adjusted upwards. Note: Additional space may be required on the staging area as temporary PST files are created on there, before being placed on the PST Output Path. It is recommended to use double the normal size of staging area (or use a PST Temporary Path). |
|
Configuring High and Low Watermarks |
Links |
It is recommended to set the high and low watermarks to around 25% of the available disk space. For example if there is 200 Gb of disk allocated for the staging area, and one link, then the low watermark should be 50 Gb and the high watermark should be 150 Gb. Having this additional free space allows for some room for operating with the temporary PST files that reside on the staging area. |
|
Configure Folder-Less Item Handling |
System Configuration |
All migrations allow archiving of items in a folder-less area called the Top Of Information Store. On the System Configuration screen, enter a folder name to be used for placing the items into the target archive. |
|
Configure File Name Policy |
File Name Policy |
Review and configure any required PST file name policy. This is the naming standard which will be used when placing the PST file on to the PST Output Path. |
|
Change PST Roll-Over Threshold |
Links |
The default PST Rollover threshold is 2 Gb. If it is required to adjust this, it can be done on the Links page when the PST link has been selected. |
|
Map one or more containers |
Manual Mapping/Bulk Mapping |
One or more containers can now be mapped and migration can begin. It is often easiest to use a filter on the bulk mapping screen to show only DAM as a source. |
Determining Enterprise Vault archive owner(s)
Archive Shuttle must determine the owner of a mailbox archive. This section explains how this takes place..
Archive Shuttle takes the owner information of a mailbox archive from the Auto Security Descriptor field within the Enterprise Vault Directory database. The first entry for an archive will be the owner. Take the following three examples:
Simple 1
·Bill usage to: somedomain\simple1
·Permissions tab: somedomain\simple with inherited read, write and delete permissions
Archive Shuttle will see the owner of this archive as simple1.
Simple 2
·Bill usage to: somedomain\simple2
·Manually grant an additional user read, and delete permissions on the archive
·Permissions tab: somedomain\simple2 with inherited read, write and delete permissions. somedomain\someotheruser with manually set read, and delete permissions
Archive Shuttle will see the owner of this archive as simple2.
Simple 3
·Bill usage to: somedomain\simple3
·Grant full mailbox access via Exchange Management Console or Exchange Management Shell
·Permissions tab: somedomain\simple3 with inherited read, write and delete permissions. somedomain\someotheruser also with inherited read, write and delete permissions
Archive Shuttle will see the owner of this archive as simple3.
|
|
NOTE: The order which the archives are listed in the permissions tab in Enterprise Vault does not reflect the order that the accounts are described in the Auto Security Descriptor field. The permissions tab shows the archive permissions in alphabetical order. |
More detail
1.Archive Shuttle resolves the 'Owner' of an Archive as follows:
In the EnterpriseVaultDirectory database, the following SQL is used:
select ADMbxDN, LegacyMbxDN from ExchangeMailboxEntry where DefaultVaultId=@VAULTID
2.Archive Shuttle then does a lookup for the ADMbxDN in Active Directory to get the SID:
LDAP Query: (ADMbxDN=)
If Archive Shuttle does not find the AD object, the archive will be marked as Ownerless and move to the next option.
3.Archive Shuttle then does a lookup for the LegacyMbxDN in Active Directory to get the SID:
LDAP Query: (LegacyMbxDN=)
If Archive Shuttle does not find the AD object, the archive will be marked as Ownerless and move to the next option.
4.We then compare the SIDs we got from ADMbxDN and from LegacyMbxDN, and if both of them match, we have an Owner. If they do not match, they will be marked as 'Ownerless' and move to the next option.
5.If we do not find an entry in ExchangeMailboxEntry, we resolve using the BillingOwner or the AutoSecurityDesc:
select SID from Root inner join trustee on OwningTrusteeIdentity = TrusteeIdentity where VaultEntryId=@ARCHIVEID
|
|
NOTE: Regarding OwnerUserSid versus SID fields in the EVArchive Table on in the Archive Shuttle Directory Database: ·SID is always filled out with the SID which is retrieved from the above steps ·OwnerUserSid has a foreign key constraint on the [User] Table. So Archive Shuttle only fill this out if the SID obtained from EV is actually contained in the Archive Shuttle Directory Database (i.e. has been synced from AD) |
5.The UserSidHistory table is stored Sid history for each user when they were migrated from AD1 to AD2. This SidHistory attribute is part of Active Directory, and Archive Shuttle collects this data from Active Directory directly.
Migrating an archive from Enterprise Vault to PST
This section outlines the steps that are necessary in order to complete the migration of a single test archive within Enterprise Vault to a PST file.
Prerequisites
Its essential that the following steps are complete before the migration begins:
·All Modules are enabled.
·Appropriate Active Directory Domains are enabled for scanning.
·An Enterprise Vault Environment has been added for migration.
·A Link Database has been created for the source archives.
·Module Mappings is configured for the source archives.
In addition, a PST Link needs to be created.
Creating a new PST naming policy
Finalized PST files are named according to a chosen PST naming policy. Policies are managed on the Configuration > Naming Policies page.
The Naming Policies page shows existing policies. Its used to view, edit, and delete existing policies, and create new policies.
|
Token |
Description |
|---|---|
|
*username* |
Username of the owning user (sAMAccountName) |
|
*firstname* |
First name of the owning user |
|
*lastname* |
Last name of the owning user |
|
*fullname* |
Full name of the owning user |
|
*email* |
E-mail address of the owning user |
|
*upn* |
User principal name of the owning user |
|
*pstid* |
IF of the PST file; continuous integer over all PST files |
|
*pstnumber* |
Number of PST files; continuous integer per user/mapping |
|
*archivename* |
Name of the archive |
|
*archiveID* |
The Enterprise Vault archive ID associated with the archive. |
Map containers - for the test archive
To migrate a test archive from Enterprise Vault to a PST file, the source and target containers must be mapped. To do this, follow these steps:
1.Go to Manage & Operations > Bulk Mapping.
2.Type the beginning of the archive name in the Container Name filter. Then, click the Apply button located at the far right of the filter row.
|
|
NOTE: If the test archive doesnt display, go back to the Configuration > EV Environment page and click Sync all AD Users. Then select the vault store where the source archive is located and click Run Now. |
3.Select the checkbox next to the test archive, and click Add Mappings. A brief wizard runs, gathering information relating to the mapping for the selected archive (multiple archives can also be selected).
4.On the Choose Target Link page, select the name of the PST Link from the drop-down list, and click [Next].
5.On the Workflow Policy page, select Native Import (PST) (without archive deletion), and then click [Next]. You dont need to select the ownerless workflow policy in this situation; its optional.
6.On the Filter Policy page, select Default (No Filter), and then click [Next].
7.On the PST Filename Policy page, select the name of a policy from the drop-down list, and click [Next].
8.On the Container Mapping Settings page, select Enabled for both Item Gathering Status and Migration Status, and click Next.
9.If desired, set the mappings priority, and then click [Next].
10.Review the summary page, and then click [Confirm].
Review Stage 1 Status
A few minutes after the mapping is created, modules begin the actions defined in the mapping. Check this stages progress by following these steps:
1.Go to Manage & Operations > Stage 1 (Sync Data).
2.Type the beginning of the archive name in the Container Name filter. Click the button located at the far right of the filter row to apply the filter.
3.Once the source archive displays, click Refresh to see the progress of the export of the archive and the ingestion of the data into a temporary PST file. Continue to click Refresh until both export and import are complete.
Validate exported data
If there is a large amount of data to export and import, then you can see the progress bars for the Stage 1 Status move by clicking Refresh from the navigation bar on the Stage 1 (Sync Data) page.
In addition, you can use Windows Explorer to browse the export/import storage area data on the disk.
In the case of PST Migration, in the folder on disk containing the exported data, there will be a further sub-folder containing a temporary PST file. You can copy it to a new location and attach to Outlook in order to review its contents.
Enable Stage 2
Before enabling Stage 2, the switch, for the test archive, check the Stage 1 (Sync Data) page for issues like failed item-export or item-import.
Then, to enable Stage 2 for the test archive, perform the following steps:
1.Go to the Stage 1 (Sync Data) page.
2.Select the checkbox next to the test archive, and then click Enable Stage 2 in the navigation bar.
3.When the Stage 1 (Sync Data) page is refreshed, there should be a green and white check mark in the Stage 2 Enabled column.
Stage 2 is the switch-over to the target environment. A final synchronization is performed of archived items from the source environment to the temporary PST file, before several additional migration tasks are performed.
Review Stage 2 Status
After a few minutes the progress of the test archive migration will be visible in the Stage 2 Status page, as follows:
1.Go to the Stage 2 (Switch User) page.
2.Type the beginning of the archive name in the Name filter. Click Apply at the right of the filter row.
|
|
NOTE: If the archive is not displayed, wait 1-2 minutes and then click Refresh. |
3.Once the source archive displays, click Refresh from time to time to see the progress of the final stages of the archive migration. By default, since the Native Import (PST) (without archive deletion) workflow policy was selected, the following steps will be performed:
1.Disable source mailbox from Enterprise Vault archiving.
2.Rename the source archive.
3.Collect any remaining items for migration.
4.Import the remaining items in to the temporary PST file.
5.Finalize the data in the temporary PST file.
6.Move the PST file to the PST Output Path, as defined on the PST Link, using the naming standard defined in the filename policy.
7.Delete any Enterprise Vault shortcuts in the mailbox.
8.Change any pending-archive items in the source mailbox back to normal items.
9.Zap the permissions on the source archive.
Verify data has been migrated
Once all of the Stage 2 operations have been completed, the Stage 2 (Sync Data) for the test archive will have a green and white check mark in the Finished column.
In addition, the PST file can be opened with Outlook from the PST Output Path.
PST as a source
This section explains what the requirements are for migrating from PST files to a target environment, and then covers the basic steps that should be followed to start such a migration.
Stage 1 - Synch
The first task which has to be performed is the discovery of the PSTs, followed by the assignment of those PSTs to target users.
Then when a PST is enabled for migration, the synch process-flow, shown above, starts.
Archive Shuttle Core sends a command to collect all needed metadata information from each PST file. The results are reported back to Archive Shuttle Core to allow item level tracking and auditing.
The PST Export and chosen target environment import module then start to process the items from the PSTs. This runs continuously in the background until the administrator initiates Stage 2 ('Switch') for the archive.
Stage 2 - Switch
PST To Exchange or Office 365
PST To Enterprise Vault
After the administrator has initiated a Switch (Stage 2) to the target, Archive Shuttle then synchronizes the difference one last time to the target environment. Depending on the target environment there then may be some additional steps (see the diagrams above).
Requirements
The following is a list of requirements.
|
Requirement |
Description |
|---|---|
|
Modules installed on source |
The PST Export module should be installed on an appropriate server. |
|
Modules installed on target |
The appropriate modules should be installed on each appropriate target server involved in the migration. |
|
AD Collector module installed |
An AD Collector module should be installed so that it can collect user-level information for the migration project. |
|
PSTs made available |
Ensure that the PST files that are the source for the migration are available in a high speed manner. In other words if theyre on network locations ensure that they are close to the server which is running the PST Export module. If theyre on multiple USB drives ensure as many of those are accessible, or at least the first is accessible ready to start the migration process. |
|
Workflows reviewed |
The Stage 2 workflows should be reviewed, and if necessary, customized to meet the needs of the project. |
|
Apply appropriate failed item threshold |
It is best practice to find an appropriate failed item threshold for the project/customer and apply that at the links level. This way all mappings which are created will inherit this value. |
Basic steps
The following are the basic steps to start this type of migration.
|
Step |
Screen |
Description |
|---|---|---|
|
Enable Modules |
Modules |
All Archive Shuttle modules must be enabled, and optionally a schedule for them defined. It is important to verify that none of the modules have a red background (indicating that Archive Shuttle Core has not had contact with the module) and that the module versions are as expected. |
|
Enable Domains |
Active Directory |
Select and enable one or more domains for synchronization. |
|
Create PST Link |
Links |
Create a PST link, and ensure that Source modules are associated with it. The PST Output Path in this case is not used, so any UNC Path can be given. |
|
Create Link Database |
Links |
Create a link database for the newly created PST Link. |
|
Add PST Source Folders |
Links |
Add one or more source folder locations where the PST files that are to be migrated can be found. It is not necessary to run the scan command. Newly added paths are automatically scanned. |
|
Add Target Links |
Links |
Ensure that the link for the migration target is added, and modules associated with it. |
|
Add Staging Area |
Links |
Ensure that the Default Staging Area is configured correctly. Depending on the migration, the free space, and the number of vault stores being used as the source for the migration, the high water mark can be adjusted upwards. |
|
Configure Folder-Less Item Handling |
System Configuration |
Many versions of Enterprise Vault allow archiving of items in a folder-less area called the Top Of Information Store. On the System Configuration screen, enter a folder name to be used for placing the items into the target archive. |
|
Associate owner to PST |
Bulk Mapping |
Note: It is recommended to associate an owner to each discovered PST. If this is not actioned, then the Bulk Mapping page can not be used to migrate the PST, instead the Manual Mapping screen must be used. It is easiest to use a filter on the bulk mapping screen to show only PST as a source archive type. If the PST file names match a users samAccountName, then the ownership of that PST will be shown, otherwise many PST files may show as ownerless. |
|
Map one or more containers |
Manual Mapping/Bulk Mapping |
One or more containers can now be mapped and migration can begin. |
General considerations
The following general considerations should be taken into account for this type of migration:
·Scanning PST files themselves for items can take considerable time .Exports will not begin for that PST until the scanning is complete.
Migrating from PST to a target environment
This section outlines the steps to migrate a single test PST to any supported target environment.
|
|
NOTE: The steps are specific to migrating to an Exchange 2010/2013 Personal Archive, but the steps are largely the same when migrating from source PST files to any supported target environment. |
Prerequisites
It is essential that the following steps have been completed prior to beginning the migration:
·All Modules are enabled
·Appropriate Active Directory Domains have been enabled for scanning
·The Exchange target links should be configured with a specific staging area, and with modules associated to the link.
Create a PST link
To use PST as a source for migration, a PST link needs to be created and configured:
1.Go to Configuration > Links.
2.On the PST tab click Create Link.
3.Give the link a meaningful name (for example, PST Source).
4.Specify a UNC path, in this case it can be any path, because it will not be used in the case of using PST as a source.
5.With the link still selected, click on Map Modules on the Link tab.
6.A Native Export Module should be selected underneath the Source option.
7.With the link still selected, click Create Database to create a link database.
Scan PST location(s)
The PST link needs to be configured with one or more source UNC paths to scan for PSTs, so that they can be migrated from. This can be achieved as follows:
1.Go to Configuration > Links.
2.Select the PST link that was created in the previous step.
3.On the PST tab select PST Source.
4.Click New, which is located in the center of the dialog, and enter a UNC path to scan for PSTs.
5.If required, multiple paths can be entered.
|
|
NOTE: Clicking Scan after adding a new UNC path for the module is not necessary, it will scan new paths automatically. |
Auto-assign user/owner to PST files
PST files are not assigned to users automatically after the scanning runs on the PST Export UNC path.
To automatically assign PST files to users when using the PST Export module, you must use the users primary SMTP address or email address or SAMAccount Name in the file name (in the format below), and then run the Scan on the PST Source link from the Links screen.
Here are the supported PST file name formats for auto-assignment:
·PrimarySMTPAddress_id.pst
·SAMAccountName_id.pst
For example:
·firstname.lastname@somedomain.com_1.pst
·firstinitiallastname_1.pst
|
|
NOTE: In the case when a SAM Account Name exists in multiple domains that are enabled in Archive Shuttle, PST is not assigned to an owner and you will see entry below in the PST export module log file: WARN|GetUserBySAMAccountName(PstExportScanDirectoryResultProcessor.cs:97)|More users with SAMAccountName [samaccountname], skipping auto assignment of PST owner |
Map containers - for the test archive
To migrate a test archive from one vault store to another, the source and target containers must be mapped:
1.Go to the Bulk Mapping page.
2.Change the filter for Type, so it is set to PST. This will show all the discovered PST files.
|
|
NOTE: The PSTs will show as ownerless, and will display the file path and file name. The number of items in the PST will show as zero at this stage |
3.Locate the PST you wish to use to test the migration, and select it.
4.Click on Assign Archive User and pick an Active Directory user whose mailbox will be the target of the migration.
5.With the container still selected, click Add Mappings from the navigation bar.
|
|
NOTE: A short wizard will start, which gathers information relating to the mapping for the select archive. (Multiple archives can also be selected) |
6.On the Target Container Type screen, select Exchange, and click [Next].
7.On the Choose target mailbox type screen, select Secondary (Archive) mailbox. In the second part of the dialog select Map to primary mailbox if the secondary mailbox does not exist, and then click [Next].
8.On the Target User screen, select Same User and click [Next].
9.On the Workflow Policy screen, select the PST to Exchange/Office 365(without archive deletion) option, and click [Next].
10.On the Filter Policy screen, select the Default (No Filter) option, and click [Next].
11.On the Container Mapping Settings screen, select Enabled for both Migration Status and Item Gathering Status, and click [Next].
12.Review the summary screen before clicking [Confirm].
|
|
NOTE: If the PST is not assigned an owner, it is still possible to use the Manual Mapping screen to perform a migration from PST to a target. |
Review Stage 1 Status
A few minutes after the mapping has been created, the program will instruct the appropriate modules to start the actions defined in the mapping. The progress for this stage of the migration can be reviewed by doing the following:
1.Go to the Stage 1 (Sync Data) page.
2.Type the beginning of the archive name in the Name filter. Click the Apply button at the right of the filter row.
3.Once the source archive is displayed, click Refresh from time to time to show the progress of the export of the archive and the ingesting of the data into the target Archive. Continue to click Refresh until both export and import are 100%.
If further data is added to the source archive, it will be synchronized every 24 hours to the target archive via the connection made with the mapping performed in the previous section. In addition, part of the Stage 2 Workflow is to perform a final synch. Therefore, its not necessary to ensure that export and import has reached 100% before moving on to the next steps.
Validate exported data
If there is a large amount of data to export and import, you can view the progress bars for the Stage 1 (Sync Data) move by clicking Refresh from the navigation bar on the Stage 1 (Sync Data) page of the user interface.
In addition, you can use Windows Explorer to browse the export/import storage area data on the disk.
Also at this time it will be possible to login to Outlook or Outlook Web Access as the test user. The migrated data will be present in the Personal Archive, and can be freely opened and manipulated.
|
|
NOTE: If the test user previously used the Personal Archive feature, it might be difficult to locate the migrated data. Because of this, Archive Shuttle recommends that the test user has an empty personal archive before migration. |
Enable Stage 2 Status
Before enabling Stage 2, the switch, for the test archive, the Stage 1 (Sync Data) page should be checked for issues such as failed item-export or item-import.
To enable Stage 2 for the test archive, perform the following steps:
1.Navigate to the Stage 1 (Sync Data) page.
2.Select the checkbox next to the test archive, and then click Enable Stage 2 in the navigation bar.
3.When the Stage 1 (Sync Data) page is refreshed, there should be a green and white check mark in the Stage 2 Enabled column.
Stage 2 is the switchover to the target environment. A final synchronization is performed of archived items from the source environment to the temporary PST file, before several additional migration tasks are performed.
Review Stage 2 Status
After a few minutes, the progress of the test archive migration will be visible in the Stage 2 Status page, as follows:
1.Navigate to the Stage 2 (Switch User) page.
2.Type the beginning of the archive name in the Name filter. Click the Apply button at the right of the filter row.
|
|
NOTE: If the archive is not displayed, wait one to two minutes and click Refresh in the navigation bar |
Once the source archive is displayed, you can click Refresh from time to time to show the progress of the final stages of the archive migration. By default, since the workflow policy that was selected was PST to Exchange/Office 365 (without archive deletion), the following steps will be performed:
a)Export any remaining items from the PST file
b)Import the remaining items in to the target
Verify data has been migrated
Once all of the Stage 2 operations have been completed the Stage 2 (Sync Data) for the test archive will have a green and white check mark in the Finished column.
In addition, if Outlook or Outlook Web Access is used to access the test mailbox then all of the archived items will be in the Personal Archive, and no Enterprise Vault shortcuts will remain in the source mailbox.