Best Practices for Rollup (4036258)

When protecting your data using Replay 4 or AppAssure 5, backup snapshots are taken frequently and are saved to the repository specified by the Core. To conserve resources over time, this data can be rolled up to an incrementally less granular series of recovery points.

This article describes best practices you can apply within your organization when using rollups.

Rollup is an internal nightly maintenance procedure that enforces the defined retention policy by collapsing and eliminating dated recovery points. As data ages beyond the requirements defined in the retention policy, those recovery points are rolled up into fewer, less granular recovery points.

When considering a retention policy, it is important to balance several factors, such as

• Frequency of backups.
• Frequency of creation, destruction, and change.
•  Durability required for granular recovery points.
• Level and durability for granularity Reduction in need for granularity.
• System load for rollup.
• Schedule of nightly processes.
• Storage space in the repository.

This article describes best practices you can apply within your organization when using rollups.

You should carefully consider many factors when establishing a retention policy.  Among others, following are several factors to consider:

• Frequency of backups. How frequently you need to back up agent machines. Your protection policy may be based on complicated or very specific business rules that determine how frequently you back up, and to how many Cores.
• Frequency of creation, destruction, and change. Snapshots are deduplicated. Each captures only the changes in state (files that have been created, destroyed, or changed) for that volume. If data does not change frequently, this may reduce the need to back up frequently, which in turn may reduce the amount of rollup that needs to be performed.
• Durability required for granular recovery points. Level and durability for granularity Reduction in need for granularity. How long you need to retain the data in discrete recovery points before combining them (rolling them up) is very relevant to establishing a retention policy. If you back up several times an hour, this creates many recovery points. If your need to maintain granularity erodes quickly, this will affect how aggressively you roll up existing recovery points.
• System load for rollup. The load on system resources (time, processing power) required to roll up snapshots is an important consideration.  The more rollups required, the more time it takes to perform, which puts load on the system.
• Schedule of nightly processes. If nightly processes are taking longer and longer, and encroaching on peak times, you should monitor activity and consider reducing concurrent operations, rolling up more aggressively, or starting nightly jobs earlier.
• Storage space in the repository. The amount of space available in your repository is also a consideration, although since there is little redundancy between deduplicated recovery points, storage savings performed when rolling up may not be substantial.

Following are some best practices to apply to rollup.

1. Practice #1: Setting the default Core retention policy for the largest number of machines.
2. Practice #2: Considering compliance requirements, data type, or data source before establishing retention policy.
3. Practice #3: If using replication, setting the retention policy on the source core.
4. Practice #4: Establishing rollup for clusters.
5. Practice #5: Providing for enough storage space for the rollup process.
6. Practice #6: Freeing up space to perform a rollup.
7. Practice #7: Replay 4 only: Adding an additional Replay 4 repository.
8. Practice #8: AppAssure 5 only: Adding a new file extent to increase available free space.
9. Practice #9: Ensuring that nightly jobs are planned for a period of low system demand.

Practice #1: Setting the default Core retention policy for the largest number of machines.

Set the default retention policy on the Core to apply to the majority of machines you want to protect, rather than setting it considering the most critical data. It is easier to differentiate a smaller number of agents that should have customized retention policies. Use custom policies for more critical machines, and set the default to apply to the majority.

• Navigate to the AppAssure 5 Core Console, click the Configuration tab, and then click Retention Policy.
• Specify the primary setting (Keep all Recovery Points for n) that determines how long initial backup snapshots are retained.
• Define a cascading set of rollup requirements that determines the intervals between when recovery points should be rolled up. These indicate the interval to wait before rolling up aging recovery points.

For example, the policy identifies how many rollups to keep per hour, then how many per day, week, month, and year.

• Under Settings, enter a numeric value in the Number of simultaneous Rollups text field.

It is recommended to set this value to 1. If rollup takes too long, you can try setting this to 2, carefully monitoring performance to ensure your environment will easily support 2 simultaneous rollups.

• Click Apply to save your changes.

Practice #2: Considering compliance requirements, data type, or source when establishing retention policy.

Each organization has its own specific business rules and requirements regarding the retention of data. Some organizations must comply with industry or government regulations for data retention such as specified in Statements on Auditing Standards (SAS90), the Health Insurance Portability and Accountability Act (HIPA), or banking or other regulations. When defining a retention policy in your organization, ensure you consider regulations and compliance.

Retention requirements may differ based on the type or source of data; for example, transactional data such as an Exchange Server may have a much more urgent requirement to back up in short intervals – perhaps several times an hour. Other data types or sources (for example, a low-traffic, seldom-changed web server) may require backups substantially less often, or may require a much lower granular retention of that data.

Practice #3: If using replication, setting the retention policy on the source core.

The target core is not capable of rollup on its own. Rollup occurs strictly based on the retention policy on the source core.

Practice #4: Establishing rollup for clusters.

You can establish rollup for clusters just as you can for individual agent machines. Follow the same process, but select the cluster in the Machines tab instead of an individual agent machine, and then define the appropriate retention policy.

Practice #5: Providing for enough storage space for the rollup process

Ensure you have enough free space to perform a rollup. As a rule of thumb, allocate a minimum of 1.25 times the size of the base image for the rollup process.  When the system is rolling an incremental snapshot into a base image, it creates a new base image to hold the combined data. While this is in process, the drive needs space to hold two base images until the rollup is complete. Upon completion, the old base image can then be deleted.

Practice #6: Replay 4 only: Freeing space to perform a rollup

In Replay 4, when rollup fails due to a lack of free space on the drive on which your repository is located, if you have two or more base images, use the Clean Up Tool as follows:

Customizing Retention Policy Settings

• Navigate to the Replay 4 Core Console and then click the Recovery Points tab.
• From the Recovery Points page, click Clean up.

A list appears of all of your base images except the oldest base images.

• Select a date associated with the listed base images, and then click Clean up.

CAUTION:  This will delete all recovery points before the date you selected.

• The screen refreshes, and all base images earlier than the date you selected have been removed from the system.

This step applies to Replay 4 only. For Replay 4, rollup only works on the most recent base image and the associated incremental snapshots. In AppAssure 5, rollup functions on all recovery points, regardless of the number of base images. For this reason, there is no Cleanup tool for AppAssure 5.

In Replay 4, when rollup fails due to a lack of free space on the drive on which your repository is located, you can add an additional storage location to increase available free space.

Practice #6: AppAssure 5 only: Freeing space to perform a rollup

When using AppAssure 5, if rollup fails due to a lack of space on the repository or because of a memory contention issue, the first approach to correct is to add space or memory.

If this is not feasible in your environment, then as rollup fails, more and more recovery points will queue to be rolled up, which will put stress on your system. In this case, restart services on the core (effectively cancelling the rollup job request. Before the next nightly rollup, force a manual rollup of the individual agent machines, starting with the agent with the fewest recovery points, and working your way to the agent machine with the most recovery points. The individual rollups will then not stress the system during the next nightly job, hopefully allowing the system to perform as intended for the next rollup period.

Practice #7: Replay 4 only: Adding an Additional Replay 4 Repository

• First, obtain another storage drive and connect it in the appropriate manner so it will be recognized by the Replay 4 Core Console.
• Navigate to the Replay 4 Core Console.
• Right-click the agent name and select Properties.
• In the upper right corner of the agent properties, click the for properies.
• From the Recovery point drop-down menu, add a secondary repository location.

This will result in your backup spanning across two locations.

Practice #8: AppAssure 5 only: Adding a New File Extent to Increase Available Free Space

In AppAssure 5, when rollup fails due to a lack of free space on the drive on which your repository is located, you can add an additional storage location to increase available free space.

NOTE: An extended repository may contain up to 4,096 extents that span across different storage technologies. The maximum size of a repository is 32 exabytes. Multiple repositories can exist on a single core.

Practice #9: Ensuring that nightly jobs are planned for a period of low system demand.

Rollup occurs during the nightly job, which by default is set for midnight. Some organizations may have a peak period at that time, or the total amount of time to execute the nightly job may exceed the period of low system demand for the organization. If this is the case, modify the time the nightly job is scheduled for a period where demand is lowest.

Follow this procedure to adjust the time the nightly job initiates.

Adjusting the Nightly Job Time

• Navigate to the AppAssure 5 Core Console and click the Configuration tab, and then Settings.
• In the Nightly Jobs area, click Change.
• The Nightly Jobs dialog box displays.
• In the Start Time text box, enter a new start time.
• Click OK.

From time to time, you should monitor how long your nightly jobs take, to ensure the amount of rollup does not encroach on a peak period for your organization. To see how long your rollups take, follow the process below.

Monitoring Rollup Time Requirements

• Navigate to the AppAssure 5 Core Console and click the Machines tab, and then click the hyperlink for the machine for which you want to view events.
• Click the Events Tab.

A log of all events for current tasks and alerts appears.

• Scroll through the events to see the job “Rolling up ‘n’ protected machines,” where n is the number of machines protected in the Core.

Note the start and end times to determine how long rollup is currently taking.

Related Articles

• Understanding Retention Policy and Rollup
• Customizing Retention Policy Settings for Agent Rollup
• How to Force  Rollup on an Agent Machine
• How to Disable Rollup Using a Custom Retention Policy
• Understanding a Rollup Report