Changing a size-based filter in Archive Shuttle (4301126)

Issue
When Archive Shuttle performs the EV Collection process, metadata about all items is collected and stored in the Item Database.  The Archive Shuttle Core then performs any requested filtering, eg filtering based on age or size.  This article explains what happens if the size filter is later changed.
Solution
As stated above, the first thing that happens is that the EV Collector will collect metadata about all items in the archive, the Archive Shuttle Core then performs the filtering.  The routed, exported, and imported statistics then follow that and show only the filtered count.
For example if there are 10 items less than 1 Mb in size, and 10 items which are 5 Mb in size, and a filter is used to restrict the migration to items less than 2 Mb, initially 20 items will be collected, 10 items will be routed (exported, and imported).
Item Collection and routing happens by default on all mappings every 24 hours from when it was last run.
During that time if the size filter is changed to a different figure, item collection will first of all get any delta, and, the Archive Shuttle Core will process all items based on the new filter.
Following on from the previous example if the filter is changed to migrate everything less than 10 Mb in size, the EV Collector may collect additional items, Archive Shuttle Core will apply this newly updated filter to all collected items (including ones collected on previous days). In this case it would see that 10 additional items are now eligible for migration, and it will queue them for export (and subsequent import).
 
In detail the following was observed during testing with Archive Shuttle 6.6:
EV Collector

2015-03-11 14:14:12Z|4692| 44| INFO| Get| -> Got 2 commands 
2015-03-11 14:14:12Z|4692| 44| INFO| ProcessManualCommand|Got Command CollectItemsForArchive with commandID 105 
2015-03-11 14:14:12Z|4692| 56| INFO| CollectItemsForSingleArchive|Collecting Items for Archive [size2] 
2015-03-11 14:14:12Z|4692| 44| INFO| GatherArchivedItems|*** Starting item collection for [size2] from: [01/01/1900 00:00:00] 
2015-03-11 14:14:12Z|4692| 44| INFO| ProcessManualCommand|Got Command CollectShortcutsFromMailbox with commandID 106 
2015-03-11 14:14:13Z|4692| 56| INFO| GatherArchivedItems|Finished collection of [20] items for [size2] in [0 s]

The EV Collector doesn’t apply the filter, the Core does. The collector gets ‘everything’.
In the EV Export Module log file:

2015-03-11 14:15:07Z|3760| 55| INFO| Get| -> Got 1 commands 
2015-03-11 14:15:09Z|3760| 76| INFO| QueueWorkItems|Finished queueing [size2]. Queued [10] items into the group 
2015-03-11 14:15:09Z|3760| 4| INFO| GetFreeSpaceOnStaging|FreeSpace is 39 GBytes on \\fs01\exportimport [0ec0af7b-b4cd-44a7-b134-14792eaaafc7] 
2015-03-11 14:15:17Z|3760| 31| INFO| Get|Querying Core for 9 Commands (Backlog: 1) 
2015-03-11 14:15:18Z|3760| 31| INFO| Get| -> Received no new command. Service might be out of schedule or there is nothing to do. 
2015-03-11 14:15:27Z|3760| 55| INFO| Get|Querying Core for 9 Commands (Backlog: 1) 
2015-03-11 14:15:27Z|3760| 55| INFO| Get| -> Received no new command. Service might be out of schedule or there is nothing to do. 
2015-03-11 14:15:34Z|3760| 31| INFO| OnPerformanceTimerElapsedImpl|Last 1 minute(s) statistics: Count: [0] | Items/s: [0.00] | kb/s: [0] | ACPU: [60] | MaxCPU: [100] | MinCPU: [3] | TSize: [0 kb] | ASize: [0 kb] | MaxSize: [0 kb] | | MinSize (kb): [0] | ATime: [0 ms] | MaxTime: [0 ms] | MinTime: [0 ms] 
2015-03-11 14:15:34Z|3760| 31| INFO| OnPerformanceTimerElapsedImpl|Last 1 minute(s) statistics: Free memory: [1460 MB] | Avg Free Memory: [1496 MB] | Process Memory: [81 MB] | Avg Process Memory: [75 MB] | APCPU: [2 %] | PCPU: [0 %] 
2015-03-11 14:15:37Z|3760| 4| INFO| Get|Querying Core for 9 Commands (Backlog: 1) 
2015-03-11 14:15:37Z|3760| 4| INFO| Get| -> Received no new command. Service might be out of schedule or there is nothing to do. 
2015-03-11 14:15:47Z|3760| 31| INFO| Get|Querying Core for 9 Commands (Backlog: 1) 
2015-03-11 14:15:47Z|3760| 31| INFO| Get| -> Received no new command. Service might be out of schedule or there is nothing to do. 
2015-03-11 14:15:48Z|3760| 37| INFO| b__3|Finished size2

So here you can see that 10 items were exported.
The Exchange Import Module log file:

2015-03-11 14:16:44Z|3412| 4| INFO| Get| -> Got 1 commands 
2015-03-11 14:16:44Z|3412| 4| INFO| Execute|Queueing Work for [size2@ev.local//o=First Org/ou=Exchange Administrative Group (FYDIBOHF23SPDLT)/cn=Recipients/cn=size2faf] ([Primary]). Import Command contains [10] Items 
2015-03-11 14:16:44Z|3412| 14| INFO| Ingest|[AIP][size2@ev.local (Primary)]: Starting ingestion of [10] items 
2015-03-11 14:16:46Z|3412| 14| INFO| Ingest|[AIP][size2@ev.local (Primary)]: Ingestion of [10] items [323 kb] finished successfully after [00:00:01.4371222]. 
2015-03-11 14:16:46Z|3412| 12| INFO| FireOnIdleImpl|[size2@ev.local//o=First Org/ou=Exchange Administrative Group (FYDIBOHF23SPDLT)/cn=Recipients/cn=size2faf (Primary)] finished processing.

The Stage 1 screen shows 10 items exported, and 10 items imported.
The filter was then updated, and 24 hours allowed to pass. Note: If Item Gathering is run after changing the filter, then the activities described below will happen immediately.
The EV Collector checked for a delta:

2015-03-12 14:14:15Z|4692| 4| INFO| Get| -> Got 2 commands
2015-03-12 14:14:15Z|4692| 4| INFO| ProcessManualCommand|Got Command CollectItemsForArchive with commandID 105
2015-03-12 14:14:15Z|4692| 4| INFO| CollectItemsForSingleArchive|Collecting Items for Archive [size2]
2015-03-12 14:14:15Z|4692| 4| INFO| GatherArchivedItems|*** Starting item collection for [size2] from: [03/11/2015 14:06:05]
2015-03-12 14:14:15Z|4692| 4| INFO| ProcessManualCommand|Got Command CollectShortcutsFromMailbox with commandID 106
2015-03-12 14:14:15Z|4692| 23| INFO| GatherArchivedItems|Finished collection of [7] items for [size2] in [0 s]
2015-03-12 14:14:15Z|4692| 90| INFO| CollectShortcuts|*** Starting shortcut collection for [size2]
2015-03-12 14:14:18Z|4692| 27| INFO| TransferItemMetadataToCore|Starting transmission of 1 containers containing 7 items to core
2015-03-12 14:14:18Z|4692| 35| INFO| SenderWT|0/7 items still in this transmit queue. Chunk: [132 ms] Total: [132 ms]
2015-03-12 14:14:18Z|4692| 4| INFO| SenderWT|Core Transmission finished. Success: 7 items | Failed: 0 items. [133 ms]
2015-03-12 14:14:19Z|4692| 23| INFO| CollectShortcuts|*** Finished collection of [40] shortcuts for [size2] in [4 s]

The date based part of the collection can mean that additional items are found, or not. It’s environment and data dependent.
Shortly afterwards the export module logged:

2015-03-12 14:14:20Z|3760| 4| INFO| Get| -> Got 1 commands
2015-03-12 14:14:20Z|3760| 22| INFO| DoExport|Start Work for size2. Export Command contains 10 Items
2015-03-12 14:14:20Z|3760| 22| INFO| QueueWorkItems|Finished queueing [size2]. Queued [10] items into the group
2015-03-12 14:14:26Z|3760| 30| INFO| b__3|Finished size2
2015-03-12 14:14:29Z|3760| 31| INFO| Get|Querying Core for 10 Commands (Backlog: 0)

And then the import module ingested them:

2015-03-12 14:15:19Z|3412| 5| INFO| Get| -> Got 1 commands
2015-03-12 14:15:19Z|3412| 4| INFO| Execute|Queueing Work for [size2@ev.local//o=First Org/ou=Exchange Administrative Group (FYDIBOHF23SPDLT)/cn=Recipients/cn=size2faf] ([Primary]). Import Command contains [10] Items
2015-03-12 14:15:19Z|3412| 4| INFO| Ingest|[AIP][size2@ev.local (Primary)]: Starting ingestion of [3] items
2015-03-12 14:15:19Z|3412| 4| INFO| Ingest|[AIP][size2@ev.local (Primary)]: Starting ingestion of [3] items
2015-03-12 14:15:28Z|3412| 5| INFO| Get|Querying Core for 9 Commands (Backlog: 1)
2015-03-12 14:15:28Z|3412| 5| INFO| Get| -> Received no new command. Service might be out of schedule or there is nothing to do.
2015-03-12 14:15:34Z|3412| 10| INFO| Ingest|[AIP][size2@ev.local (Primary)]: Ingestion of [3] items [11668 kb] finished successfully after [00:00:15.1765407].
2015-03-12 14:15:34Z|3412| 12| INFO| Ingest|[AIP][size2@ev.local (Primary)]: Starting ingestion of [1] items
2015-03-12 14:15:38Z|3412| 5| INFO| Get|Querying Core for 9 Commands (Backlog: 1)
2015-03-12 14:15:38Z|3412| 5| INFO| Get| -> Received no new command. Service might be out of schedule or there is nothing to do.
2015-03-12 14:15:41Z|3412| 11| INFO| Ingest|[AIP][size2@ev.local (Primary)]: Ingestion of [3] items [11668 kb] finished successfully after [00:00:21.8700489].
2015-03-12 14:15:41Z|3412| 10| INFO| Ingest|[AIP][size2@ev.local (Primary)]: Starting ingestion of [3] items
2015-03-12 14:15:41Z|3412| 12| INFO| Ingest|[AIP][size2@ev.local (Primary)]: Ingestion of [1] items [3889 kb] finished successfully after [00:00:06.8575780].
2015-03-12 14:15:45Z|3412| 4| INFO| OnLogThreadPoolStartTimerElapsed|Ingest Statistics: Working Threads: [1] Waiting Items: [3] Concurrency: [10] (Min: 5 / Max: 10)
2015-03-12 14:15:45Z|3412| 5| INFO| TransferSuccessItems|Transferring [7] successful items to core in chunks of [512] items
2015-03-12 14:15:46Z|3412| 10| INFO| Ingest|[AIP][size2@ev.local (Primary)]: Ingestion of [3] items [11668 kb] finished successfully after [00:00:04.9183262].
2015-03-12 14:15:46Z|3412| 12| INFO| FireOnIdleImpl|[size2@ev.local//o=First Org/ou=Exchange Administrative Group (FYDIBOHF23SPDLT)/cn=Recipients/cn=size2faf (Primary)] finished processing.

The Stage 1 screen shows 20 items exported, and 20 items imported.