Issue
While migrating to Office 365 the following error appears:
Error
BTE2EMCP (Update Running Jobs)
Failed to Rationalize 1 Failed jobs in forest 'cmtdemobt.onmicrosoft.com'
-------------
Connecting to remote server failed with the following error message : [Server=BLUPR01CA008,RequestId=92d94cce-305a-4185-b538-d624cabb64ad,TimeStamp=6/5/2013 6:22:09 PM] Fail to create runspace because you have exceeded your budget to create runspace. Please wait for 0 seconds.
Policy: CN=OrganizationThrottlingPolicy61fa217e-07bd-4685-a171-cb66810c7843,CN=Global Settings,CN=Configuration,CN=CMTDemoBT.onmicrosoft.com,CN=ConfigurationUnits,DC=NAMPR01A001,DC=prod,DC=outlook,DC=com;
Snapshot: Owner: Sid~NAMPR01A001\jsha58267-1708854995~WSMan~false
BudgetType: WSMan
ActiveRunspaces: 1/3
Balance: 600000/1800000/-3000000
PowerShellCmdletsLeft: 200/200
ExchangeCmdletsLeft: 9223372036854775807/Unlimited
CmdletTimePeriod: 5
DestructiveCmdletsLeft: 60/Unlimited
DestructiveCmdletTimePeriod: Unlimited
QueueDepth: Unlimited
MaxRunspacesTimePeriod: 60
RunSpacesRemaining: 0/5
LastTimeFrameUpdate: 6/5/2013 6:22:09 PM
LastTimeFrameUpdateDestructiveCmdlets: 6/5/2013 5:13:37 PM
LastTimeFrameUpdateMaxRunspaces: 6/5/2013 6:21:10 PM
Locked: False
LockRemaining: 00:00:00 For more information, see the about_Remote_Troubleshooting Help topic.
at System.Management.Automation.Runspaces.Internal.RunspacePoolInternal.EndOpen(IAsyncResult asyncResult)
at System.Management.Automation.Runspaces.RunspacePool.Open()
at System.Management.Automation.RemoteRunspace.Open()
at BinaryTree.E2E.Library.PSHelper.#RO(String #OO, PSCredential #PO)
at BinaryTree.E2E.Library.PSHelper.OpenRemoteExchangeRunspace(String hostname, PSCredential cred, String globalCatalog)
at BinaryTree.E2E.DomainServices.DataModels.CMTEUP_Forest.OpenRemoteExchangeRunspace(String powershellHost, String globalCatalogFQDN)
at #aW.#N4.#c4(CMTEUP_Forest #8U, List`1 #hX)
at #aW.#N4.#c4()
Cause
What's going on is that when the option to automatically suspend a migration before finishing is used, the migrations do properly suspend at 95% like they should and this is reflected in the Exchange EMC Move Requests view. The problem is that it is NOT reflected in the Exchange Pro console. There, the users are only shown as "In Progress" rather than "Suspended", which means you also can't tell Exchange Pro to finish the migration because it doesn't know that they're suspended
Solution
We need to back off the polling intervals:
Go to Settings Tab within Exchange Pro and increment the values for following:
“Update Forest"
"Update Mailbox Job"
"Update Mailbox Statistic"
TURN OFF the Exchange Pro machine for 2 hours.
When you turn it back on and use Exchange Pro, things should be ok.
Basically, we query mailbox status frequently. Sometimes, O365 is busy, so it says “you are consuming too many of our resources right now” so they throttle the connection, and we fail to see the status update.
By setting those values to 90 seconds, and then backing off completely for a couple of hours, we should be ok, and should fix the issue.