SharePlex 9.2.4 - Administration Guide

Move replication during planned failover and failback

In a planned failover of database activity to a secondary Oracle instance, you can quickly move SharePlex to the secondary system. While users continue their transactions on that system, SharePlex captures their changes and stores them until the primary system is back online and activity is moved back to that system.

Supported databases

Oracle database on Unix or Linux

Requirements

• SharePlex must be configured correctly to support high availability. For more information, see Configure replication to maintain high availability.
• There must be enough disk space where the queues reside on the secondary system to contain the data that accumulates while users are working on that system.
• You must know how to run SharePlex. For more information, see Run SharePlex.
• You must understand the SharePlex flush command. For more information, see the SharePlex Reference Guide.

Procedure

This procedure is divided into logical segments. Follow them in the order presented. Do not shut down the primary instance until prompted in the procedure.

Switch users to the secondary system

1. On the primary system, stop user access to the primary instance.

2. On the primary system, flush the data in the queues to the secondary system. This command stops Post on the secondary system and places a marker in the data stream to establish a synchronization point between the primary and secondary data.

   sp_ctrl> flush datasource

   where: datasource is the datasource specification of the primary Oracle instance, for example o.OraA.

3. On the secondary system, verify that Post stopped. (Continue to issue this command until it shows that Post stopped.)

   sp_ctrl> status

4. On the primary system, verify that there are no messages in the capture and export queues. The Number of Messages and the Backlog (messages) fields must be 0.

   sp_ctrl> qstatus

5. On the secondary system, verify that there are no messages in the post queue. The Number of Messages and the Backlog (messages) fields must be 0.

   sp_ctrl> qstatus

6. On the primary system, shut down SharePlex.

7. On the primary system, shut down the Oracle instance with the abort option. Do not use the immediate option.

   svrmgr1> shutdown abort

   Note: This resets the sequence on the primary system to the top of the cache when the database starts.

8. On the secondary system, verify that Export is stopped. This prevents user changes from being replicated to the primary system until it is back online and SharePlex is ready to receive them. Stop Export if needed.

   sp_ctrl> status

   sp_ctrl> stop export

9. On the secondary system, run the script that grants INSERT, UPDATE and DELETE access to all users.
10. On the secondary system, run the script that enables triggers and constraints on the secondary instance.
11. Run your failover procedure for relocating users to the secondary system, including starting the applications and starting jobs that were running on the primary system.
12. Move the users to the secondary system to resume working, but do not start the Export process.

Switch users back to the primary system

1. On the primary system, open the Oracle instance. The sequence on this system should now be at the top of the cache.
2. On the primary system, disable triggers on the tables, or run the sp_add_trigger.sql utility script so that the triggers ignore the SharePlex user.
3. On the primary system, disable check constraints and scheduled jobs that perform DML.
4. On the primary system, start SharePlex.

5. On the secondary system, start Export so that SharePlex sends the accumulated replicated data to the primary system.

   sp_ctrl> start export

   Note: SharePlex passes any sequence updates from the secondary system back to the primary system when Export starts.

6. On the primary system, stop Export.

   sp_ctrl> stop export

7. On the primary system, allow Post to process the message backlog that was sent from the primary system.

8. On the secondary system, stop user access to the Oracle instance.

9. On the secondary system, flush the data in the queues to the primary system.

   sp_ctrl> flush datasource

   where: datasource is the datasource specification of the secondary Oracle instance, for example o.OraB.

10. On the primary system, verify that Post stopped. (Continue to issue this command until it shows that Post stopped.)

    sp_ctrl> status

11. On the secondary system, verify that there are no messages in the capture and export queues. The Number of Messages and the Backlog (messages) fields must be 0.

    sp_ctrl> qstatus

12. On the primary system, verify that there are no messages in the post queue. The Number of Messages and the Backlog (messages) fields must be 0.

    sp_ctrl> qstatus

13. On the secondary system, shut down SharePlex.

    sp_ctrl> shutdown

14. On the secondary system, shut down the Oracle instance with the abort option. Do not use the immediate option.

    svrmgr1> shutdown abort

    Note: This resets the sequence on the secondary system to the top of the cache when the database starts.

15. On the secondary system, start the Oracle instance.

    svrmgr1> startup

    Note: The sequence on the secondary system is now at the top of the cache. When the next value is selected on the primary system, a new cache is acquired and is replicated to the secondary system. Now, the primary system is at the start of a cache, and the secondary system is at the top of a cache.

16. On the primary system, run the script that grants INSERT, UPDATE and DELETE access to all users.
17. On the primary system, run the script that enables triggers and constraints on the primary system when users begin using this system.
18. Run your failover procedure for moving users back to the primary system, including starting the applications and starting jobs that were running on the secondary system.
19. Switch the users to the primary system to resume working, but do not start the Export process. This prevents replicated data from being sent to the secondary system until SharePlex is ready to receive it there.

Resume replication to maintain the secondary instance

1. On the secondary system, disable triggers on the tables, or run the sp_add_trigger.sql utility script so that the triggers ignore the SharePlex user.
2. On the secondary system, disable check constraints and scheduled jobs that perform DML.
3. On the secondary system, start SharePlex.

4. On the secondary system, stop Export. This prevents any accidental DML on that system from being replicated to the primary system.

   sp_ctrl> stop export

5. On the primary system, start export.

   sp_ctrl> start export

6. On the secondary system, start Post.

   sp_ctrl> start post

   Replication from the primary instance to the secondary instance is now active to keep the two databases synchronized and ready for future failover when needed.

Resume replication after failure and recovery

The procedure is typically used in these situations:

• When the source system fails and replication must be switched to a standby database system
• When replication must be positioned back in time to re-read old archive logs.

Requirements to support SharePlex replication recovery

To resume replication when the source, target or both have failed, there must be the following in place at the onset of replication:

• A disaster recovery (DR) solution that provides a physically identical copy of the production source instance and another physical copy of the production target instance. Methods such as Oracle Data Guard or disk mirroring, tape backups and other methods support this requirement.
• The SP_OPO_UPDATE_SCN parameter must be set to a value of 1. This parameter directs SharePlex to keep a record of the SCNs of the transactions that it processes. When you set this parameter to 1, it also disables the Post Enhanced Performance feature.

Overview of initial setup

The following diagram depicts a DR configuration at the onset of replication. There is a source production instance and a mirrored source DR instance that is kept current by Oracle Data Guard. Similarly, there is a production target instance and a mirrored DR target instance that is kept current by Oracle Data Guard.

• The solid (blue) lines represent the Oracle Data Guard DR deployment.
• The dotted (bright green) line between the production source instance and the production target instance represent SharePlex replication under normal operating circumstances.
• The dashed lines (red, orange or aqua) show possible replication recovery paths if the source, target or both fail.

Figure 2: DR configuration at the onset of replication

Example failure/recovery scenario

This example illustrates one of the potential failure/recovery scenarios, in this case where the production target instance fails. The recovery path is shown as the diagonal, orange dotted line in the DR configuration at the onset of replication diagram.

Normal replication

The following diagram illustrates the configuration and the names that are used in this example.

• The production source is named Source PROD and the DR source is named Source DR.
• The production target is named Target PROD and the DR target is named Target DR.
• SharePlex (SPO in the diagram) replicates from Source PROD to Target PROD.

Figure 3: Normal replication and mirroring configuration

Production target fails

The Target PROD target fails, as represented by the red X across it in the following diagram. SharePlex can no longer replicate to Target PROD, as represented by the red X over the original replication data stream.

Because Target PROD is offline, Oracle Data Guard can no longer keep Target DR up to date. However, SharePlex can. SharePlex (SPO in the diagram) is able to resume replication from Source PROD to Target DR, thus resuming data availability.

Figure 4: Failure and recovery by SharePlex (SPO)

Resume replication after failover

In this procedure you will do the following to direct SharePlex to recover replication:

• Direct SharePlex to capture the correct Oracle SCN of the last committed transaction processed by each post queue.
• Direct SharePlex, through the reconcile command, to discard all transactions that were committed to the target before the failure, so that SharePlex resumes replication at the correct point in the data stream.

Note: This procedure requires the following:

• The source instance is recovered to a later point in time than the target instance; otherwise, this method will not work.
• The SP_OPO_UPDATE_SCN parameter is set to 1.

To resume replication

Note: In these instructions, the source and target systems are whichever source and target are operational after the failover.

1. Shut down SharePlex on the source system, if it is still running.

   sp_ctrl> shutdown

2. On the target, start sp_cop if it is not running already.

   $ /productdir/bin/sp_cop &

3. On the target, use the qstatus command to make certain that all of the message in the queues are posted to the target database. The command output should show 0 backlog in the post queue.

   sp_ctrl> qstatus

4. From the command line of the target, run the show_scn utility from the bin subdirectory of the SharePlex product directory. For ORACLE_SID use the ORACLE_SID of the target database.

   $ /productdir/bin/show_scn ORACLE_SID

5. Keep the output of the show_scn utility open. The output displays the complete reconcile command that you will use for each of your post queues to reposition Post to the correct transaction for recovery. It also shows the SCN to which you will activate the configuration later in these steps.

6. Shut down sp_cop on the source and target.

   sp_ctrl> shutdown

7. Run ora_cleansp on the source and target to clean out the queues.

   $ /productdir/bin/ora_cleansp

8. Start sp_cop on the source and target.

   $ /productdir/bin/sp_cop &

9. On the target, stop Post.

   sp_ctrl> stop post

10. On the source, issue the activate config command with the scn option to activate the configuration. For scn_value, use the value that is shown in the output of the show_scn utility on the line that states On source activate to scn=nnnnnnn.

    sp_ctrl> activate config configname scn=scn_value

    Example:

    sp_ctrl> activate config myconfig scn=510012416

11. On the target, copy the first reconcile command from the show_scn output and then execute it in sp_ctrl. Then do the same for the second reconcile command, and work your way down the list.

    Example:

    sp_ctrl> reconcile queue spx11 for o.ora112-o.ora112 scn 235690

    sp_ctrl> reconcile queue pq1 for o.ora112-o.ora112 scn 132436

    sp_ctrl> reconcile queue pq2 for o.ora112-o.ora112 scn 246843

    sp_ctrl> reconcile queue pq3 for o.ora112-o.ora112 scn 123457

    The reconcile command may seem stalled until new data comes in. However, the command is working.

12. On the target, start Post.

    sp_ctrl> start post

Make changes to an active replication environment

This chapter contains instructions for making datbase changes, or performing system and software maintenance, on systems where SharePlex replication is active.

Contents

Change a configuration file

Many procedures that change an element of the replication environment will also involve changing the SharePlex configuration file. The recommended method to modify an active configuration file is to make a copy of the file first using a new file name. For example, append the date to the file name. By copying the file first, you preserve the original file in case it is needed again or you encounter a problem with the new file. By copying the file first, you also can control when to activate the new configuration file.

In most cases, activation of an edited configuration is less time-consuming than the original activation if you do not deactivate the original configuration. The activation of the new configuration automatically deactivates the original configuration, and SharePlex only needs to analyze the new, changed, and removed objects. If you deactivate the original configuration before you activate a new one, SharePlex re-analyzes all of the objects.

To change a configuration file, see Add or change objects in an active configuration.