Depending on the container type, data is sent to the archive tier by different methods. For RDA and Object containers, data is archived based on the Archive Tiering Policy. For VTL containers, exporting the cartridge from the backup application will trigger the movement of cart data to cloud.
Restoring data from an archive tier differs in some ways from a standard restore process. There are two possible methods for restoring:
- Selectively restoring backups based on need, and
- Performing a full disaster recovery by completely restoring all data from Glacier to AWS S3.
In both cases, when restoring from an archive tier, no files are saved to on-prem storage. Instead, files are copied from the archive storage (Glacier or Amazon S3 Glacier Deep Archive) to warm AWS S3 storage for a period of time specified by the Archive Retention in Warm Cloud setting. When restoring from an archive tier, consider the following:
- Restoring from archive storage is a two-step operation. First, archive data is restored to standard AWS S3 storage, then the objects are read from there.
- There are two options for restoring from Glacier storage: Batch operations and Lambda with batch operations. According to AWS, "Lambda is a compute service that lets you run code without provisioning or managing servers." Using Lambda with batch operations can help avoid certain restore failures. For more information, see What is AWS Lambda? in the AWS documentation.
- Restored objects will be ready for readback after 4-6 hours for Amazon S3 Glacier (10-12 hours for Amazon S3 Glacier Deep Archive). No notification is issued when restored objects are available. You may view the status of restore operations in the AWS Console. Refer to the Amazon S3 document Checking Archive Restore Status and Expiration Date for more information. To perform a batch restore for disaster recovery purposes, refer to Manually restoring datastores from Amazon S3 Glacier.
- When restoring objects from archive, you are charged for both the archive copy and the restore copy in warm storage. Use the Archive Retention in Warm Cloud value to minimize the duration objects are kept in warm storage.
- For restoring VTL cartridge data, the command vtl --import must be run on the QoreStor server.
Restoring files from RDS Container backups replicated to AWS S3 Glacier or Deep Archive
Backups written to an RDA container replicate to an archive tier and stubbed from on-premises storage based on the Archive Tiering Policy.
In the case of an RDA container, the process of restoring files from an archive tier differs based on direct memory access (DMA) . For more information, see the respective DMA setup guide on Quest QoreStor support portal.
Restoring selective tapes of VTL backups replicated to AWS S3 Glacier or Deep Archive
In the case of VTL containers, there are no individual backups replicated to cloud storage, but the entire cartridge is exported to the cloud. You can replicate VTL cartridges that are no longer required as on-premises storage to the cloud using VTL export.
For detailed instructions, refer to the respective DMA setup guide on the Quest Support Portal.
After you replicate and stub VTL tapes to an archive tier, to bring the data to Warm Cloud storage and restore it, use the following procedure.
To restore selective tapes of VTL backups replicated to AWS S3 Glacier or Deep Archive
- To bring data to Warm Cloud storage and initiate a restore from AWS S3 Glacier to S3 Standard storage, use the following command:
vtl --restore --name <cont_name> --barcode <barcode_of-media>
|
|
NOTE: This process typically takes approximately four hours when performed from AWS S3 Glacier storage and as many as eight to 12 hours from Glacier Deep Archive. |
- To check the Glacier to Standard S3 restore status, use the following command:
vtl --show --verbose
The restore status appears at the end of the command output, as shown in the following example:
- Restore Status: [Cart Name] [Restore current State] [Restored Data available until]
- Restore status can be:
- [Cart Name] [Restore current State] [Restored Data available until]
GV758Q001 Restore process initiated Tue 2020-03-31 09:23:50 EAT
- [Cart Name] [Restore current State] [Restored Data available until]
GV758Q001 Restore initiation successful Tue 2020-03-31 09:23:50 EAT
- [Cart Name] [Restore current State] [Restored Data available until]
GV758Q001 Restore Process InProgress Wed 2020-04-08 03:00:00 EAT
- [Cart Name] [Restore current State] [Restored Data available until]
GV758Q001 Restore SUCCESSFUL Wed 2020-04-08 03:00:00 EAT
- After the restore is done, import carts using the following command:
vtl --import_cart --name <container-name> --barcode <comma-separated-barcodes>
You can provide multiple carts during the VTL import operation.
- To view the status of the selected carts moving from the cloud to the storage_slot of the VTL tape drive, use the following command:
vtl --show --verbose
- To rescan the media and update the VTL, perform an Inventory Robot operation.
|
|
NOTE: Before you attempt to restore to a local disk, bring the newly added media online. |
There are two ways to perform a recovery from the cloud, also known as a disaster recovery. You can recover data by creating a new QoreStor instance and transferring the data, or you can perform a quick recovery, which provides a read-only version of RDA container data of your current QoreStor instance. To recover your QoreStor configuration and cloud-replicated data from the cloud, perform the steps below. Before performing these steps, make sure you have the following:
- A functional, properly licensed QoreStor server.
- In the case of Archive Tier, to bring data to S3-Warm Cloud before proceeding with a disaster recovery operation, refer to Manually restoring datastores from Amazon S3 Glacier.
- The connection string for your cloud storage account. This is different depending on your cloud provider. Refer to the appropriate section below for more information.
To perform a disaster recovery from the cloud
- On a newly installed QoreStor server, run the following recovery command using the definitions provided in the table:
maintenance --disaster_recovery --cloud_string <name> --container_name <name> --cloud_provider_type <name> --passphrase <name> --logfile <name> --quick_ro_recovery no
Table 7: Recovery command definitions
|
--cloud_string |
Cloud connection string, to connect to the cloud bucket. |
|
--container_name |
Name of the cloud bucket from where data is to be recovered. Valid values are [a-z, 0-9, '-', '.']. |
|
--cloud_provider_type |
Name of the cloud service provider, such as <AWS-S3 | Azure | Wasabi-S3 | Google-S3 | IBM-S3 | S3-Compatible>. |
|
--passphrase |
Passphrase used on original machine for encrypting the data in the cloud bucket. |
|
--logfile |
Log file path to capture the ongoing recovery activity. |
This will regenerate configuration data and populate the metadata from the cloud copy.
When completed, you will see the following message:
Filesystem disaster recovery started successfully.
Please see the /var/log/oca/qsdr.log and the logfile given in the command.
- After the data recovery process is complete, perform a filesystem repair with the command
maintenance --filesystem --repair_now
When the file system repair is finished, the process is complete.
To perform a quick recovery
- On a newly installed QoreStor server of the same install mode and configuration as your previous QoreStor server, run the following recovery command using the definitions provided in the table:
maintenance --disaster_recovery [--cloud_string <name>] [--container_name <name>] [--cloud_provider_type <name>] [--passphrase <name>] [--logfile <name>] [--quick_ro_recovery <[yes | no]>]
Table 8: Quick recovery command definitions
|
--cloud_string |
Cloud connection string, to connect to the cloud bucket. |
|
--container_name |
Name of the cloud bucket from where data is to be recovered. Valid values are [a-z, 0-9, '-', '.']. |
|
--cloud_provider_type |
Name of the cloud service provider, such as <AWS-S3 | Azure | Wasabi-S3 | Google-S3 | IBM-S3 | S3-Compatible>. |
|
--passphrase |
Passphrase used on original machine for encrypting the data in the cloud bucket. |
|
--logfile |
Log file path to capture the ongoing recovery activity. |
| --quick_ro_recovery |
Fast disaster recovery, with data in cloud bucket accessible in RO mode only.--quick_ro_recovery quick read-only recovery. |
This will regenerate configuration data and populate the metadata from the cloud copy. When used with a newly deployed QoreStor instance, the --quick_ro_recovery command lets you attach a cloud bucket and conduct read-only restores without disturbing the cloud connection of the existing QoreStor.
When completed, you will see the following message:
Filesystem disaster recovery started successfully.
Please see the /var/log/oca/qsdr.log and the logfile given in the command.
Next steps
Depending on your configuration, there may be several steps required after recovering your QoreStor server. Some actions to consider are:
- If you are using QoreStor with NetVault, you will need to add the new QoreStor as target device and add the container.
- Depending on your DMA, you may need to reconfigure DMA or client connections to reference the new QoreStor server.
- Once a disaster recovery completes, the recovered source containers will be unencrypted. Before ingesting new data into the recovered containers, you must enabled encryption on the recovered storage groups.
|
|
NOTE:The recovered containers contain only stub files. The data remains encrypted in the cloud tier. |
Performing a disaster recovery from an archive tier, involves restoring all datastores to standard Amazon S3 storage, which includes generating the manifest of objects in the cloud bucket used for archive tier and then initiating a Batch Restore from the console.
|
|
NOTE: Before performing a recovery, confirm the status of various objects and ensure that they are in warm storage for the duration the readback of data is planned. |
To manually restore datastores from Amazon S3 Glacier
- Sign in to the AWS Management Console and open the Amazon S3 Console at https://console.aws.amazon.com/s3.
- Under Buckets, select the name of the bucket for which you want to configure Amazon S3 inventory.
- Click Management.
- Under Inventory configurations, click Create inventory configuration.
- For Inventory configuration name, enter a name.
- Set the Inventory scope by entering the following details:
- For prefix, enter s3://<cloud_bucket_name>/cds.
- For object versions, select Current versions only.
- Select the following options:
Table 9: Manifest options
| Report details |
Select the location of the AWS account to which you want to save the reports. For example, This account. |
| Destination |
Complete one of the following options:
- Select the destination bucket where you want to save the reports, or
- Select a different prefix under the same bucket. For example, s3://<cloud_bucket_name>/manifest.
|
|
NOTE: The destination bucket must be in the same AWS Region as the bucket for which you are setting up the inventory. | |
| Destination bucket |
Here you can see the Destination bucket permissions added to the destination bucket policy which allows Amazon S3 to place data in that bucket. For more information, see Creating a destination bucket policy in the documentation for Amazon S3. |
| Frequency |
Select Daily.
|
|
NOTE: The first report may take 24 to 48 hours to generate. | |
| Output format |
Select CSV. |
| Status |
Select Enable. |
| Server-side encryption |
Select Disable. |
| Additional Fields |
Do not select any options, as they are not included in the Batch Job submission. |
The manifest file generates.
- To initiate the Batch Job for restoring datastores, complete the following steps:
- Navigate to AWS S3 bucket > Management > Inventory Configurations.
- Select the latest manifest file.
- In the AWS console, to initiate the batch job, click Create Job from Manifest option in AWS console.
In the QoreStor GUI, you can set up and manage data replication operations. Such replication operations include, creating new replication relationships, managing or deleting existing replication relationships, starting and stopping replication, and displaying current replication statistics.
