Streams and connections
This topic describes the differences between data streams and application connections.
Streams refer to the number of files written at the same time to QoreStor. QoreStor tracks the number of files being written and assembles the data into 4–MB chunks before processing that section of the data. If the stream count is exceeded, the data is processed out of order and overall deduplication savings can be affected.
Connections are created by applications; and, within a single connection, there can be multiple streams depending on the application and the number backup jobs running in parallel over that single connection.
Secure connect
Secure Connect encompasses a set of client and server components that creates a secure channel for QoreStor communication with WAN-connected clients that is also resilient to WAN outages.
Secure Connect uses the TLS 1.2 standard with a 4096-bit RSA key. Certificates are created automatically for both client and server, but you can use you own certificates if you chose. When using Secure Connect (which is enabled by default when using the latest QoreStor and plug-in versions), the client opens a connection to the QoreStor server over port 9443. The client sends the actual QoreStor port number to the server, which then opens a local connection to that port enabling secure communication with the client. Configuration of Secure Connect ports is done through the client and server configuration file. See Configuring Secure Connect for more information.
Secure Connect also provides a method for resilient WAN connections. Packets processed by Secure Connect clients are assigned a unique identifier and are assigned to a temporary cache before being sent to the QoreStor server. When the packet is successfully delivered to the QoreStor Secure Connect server, the packet identifier is marked as delivered and acknowledged to the Secure Connect client. If the WAN connection is lost, the client and server both continue to cache data packets. When the connection is restored, unacknowledged packets are re-sent and properly processed, avoiding data loss and process interruption.
Replication
Replication is the process by which key data is saved from storage locations, with the goal of maintaining consistency between redundant resources in data storage environments. Data replication improves the level of fault-tolerance, which improves the reliability of maintaining saved data and permits accessibility to the same stored data.
QoreStor uses an active form of replication that lets you configure a primary-backup scheme. During replication, the system processes data storage requests from a specified source to a specified replica target, which acts as a replica of the original source data.
|
|
NOTE: QoreStor includes version checking that limits replication only between other QoreStor instances or DR Series systems that run a compatible software release version. If versions are incompatible, the administrator is notified by an event. |
Replicas are read-only and are updated with new or unique data during scheduled or manual replications. QoreStor can be considered to act as a form of a storage replication process in which the backup and deduplication data is replicated in real-time or via a scheduled window in a network environment. In a replication relationship between two or three QoreStor instances or DR Series systems, this means that a relationship exists between a number of systems. One system acts as the source and the other as a replica.
Replication is done at the container level and is one directional from source to replica; however, since replication is done at the container level you can set up various containers to meet your specific replication requirements for your specific workflow. This form of replication is supported for the CIFS, NFS, Rapid CIFS, and Rapid NFS protocols and is fully handled by QoreStor.
While replication of
|
|
NOTE: The storage capacity of the target QoreStor system is directly affected by the number of source systems writing to its containers, and by the amount being written by each of the source systems. |
If the source and target systems are in different Active Directory (AD) domains, then the data that resides on the target system may not be accessible. When AD is used to perform authentication forQoreStor systems, the AD information is saved with the file. This can act to restrict user access to the data based on the type of AD permissions that are in place.
|
|
NOTE: This same authentication information is replicated to the target QoreStor system when you have replication configured. To prevent domain access issues, ensure that both the target and source systems reside in the same Active Directory domain. |
Reverse replication
Alternate ways to retrieve data
Under very specific conditions, it could be possible for replica containers to support a type of write operation whose sole function is to restore data from an archival target. For example, data could be replicated back to the remote site where a data management application (DMA), or backup software, is connected to allow this data to be restored directly.
This specific type of case applies only to configurations where data is backed up from a remote location to a local container, and then replicated over a WAN to a replica container that is backed up to tape. The data needs to be restored from the tape backup to the original location; first back to QoreStor replica container, and then back to the original source location of the data on the other side of the WAN link.
|
|
NOTE: If you choose to use this alternate workaround method, you must set up a new data storage unit in your DMA, and import the images before a restore to the original location can occur. |
To leverage this type of deduplication across the WAN, complete the following:
- Make sure that the replication operation has completed (between source and target).
- Delete the current replication relationship, and re-create a replication relationship (reversing the source and target roles).
- Restore data to the original source container (now the target).
- Make sure that the replication operation has completed.
- Delete the replication relationship and re-create a replication relationship (restoring original source and target destinations).
Under this scenario, a fraction of the data to be recovered is sent across the WAN link. This could speed up a remote restore significantly. However, there are some downsides to this type of scenario:
- If step 1 is not followed correctly, any changes not fully replicated are lost.
- During steps 2 and 3, any data that is written to the original QoreStor source container may be lost.
- During step 4, if the data is not fully replicated back before the switch is made, it may be lost.
- Create a new container on the target QoreStor instance.
- Set up replication from this container back to the source QoreStor system container.
- Set up a new disk storage unit in the DMA and make sure that the DMA is aware of any new images.
- Import the old images back into the DMA from the target QoreStor instance (the original source location).
- Use a new disk storage unit in the DMA, and then restore the data back to the original client.