Issue: Handling of corrupted PST files (4293900)

Issue
Sometimes a PST file is uploaded to PST Flight Deck and processed by the Repair Module or the Extraction Module, but it fails with an error that would stop the file from further processing. Generally the setup is that the file skips the Repair Module, and the first failures are in the extraction module.
Solution
The Extraction Module has an automatism that will send certain failure types through the Repair automatically. If the file is not repairable, the failure will be indicated in the Extraction Module. In the Extraction Module, typical failure events are:

• The file is not in PST Format. Size is 0 Bytes -> The file with 0 bytes in size cannot be processed. Hence set the file to verified corrupt.

Click To See Full Image.

• The file appears to be corrupted: Invalid format -> If the file appears to be corrupted, it’s recommended to send the file through the repair module.

Click To See Full Image.

In the repair module typical events are:

• InvalidOperationException: Unknown / unsupported status
• PST file [ID and Path of file] is not repairable (invalid PST file)

Click To See Full Image.

In both cases, the operations team needs to decide what to do with the file.
The normal response from the operation team is look at the file.  Some files have .PST extension but are not really PST files. If the file appears to be a real PST file, it should be sent through the repair module.  The repair module uses Microsoft’s ScanPST tool to repair the file. In most cases, the file cannot be repaired – typical numbers from production environments will show that typically +90% of all files deemed corrupt by the extraction module couldn’t be repaired. A very small percentage (typically below 1%) of corrupt files can be opened by Outlook. It may seem strange that the Microsoft repair tool cannot repair it, but Outlook can open it. There are generally two reasons for this:

• Outlook has some un-documented  “repair” mechanism.
• The user may not get to the point of failure when opening it – traversing different directories may lead to the error.

Possible options
In the case where the customer wants the data ingested, some options are available. Before using these options:

• Check that the file has not already been ingested, as customers often have corrupt files from previous local repairs on their drives.  If so, decide whether to contact the customer or to simply accept it’s corrupt.
• Check with the user if the data is really needed.

If the conclusion is that the file needs to be ingested, the options are:
1. Use 3rd Party Repair tool
This requires purchasing a third party repair tool and install that to do the “repairs.”

• Move the file to another computer.  The repair does require significant CPU and memory it is recommended to do the repair on alternate systems.
• Run the third party repair tool.
• If the repair was successful, copy the file back into the Uploads directory, using the same name as the original file.
• Set the file to retry the extraction where it failed.  Alternatively, if it failed in the repair module, set file to Migrating and then reprocess.

2. Use Outlook to rewrite the PST file 
In some cases, the file can be opened in Outlook, which allows an export into a new file.
Move the file to a repair directory, or better to an alternate computer as repair load can be significant.

• Start Outlook.
• Mount the PST file in Outlook.
• Now create a new PST file with the same name but in a different directory.
• Ensure it has the same display name as the original file.
• Now drag all the data from the original file into a new PST file.
• When finished, detach both files From Outlook.
• Copy the newly create file back into the Uploads directory.
• Set the file to retry the extraction where it failed.  Alternatively, if it failed in the repair module, set file to Migrating and then reprocess.
• Delete the original file from the repair directory.