SharePlex 12.0 - Installation and Setup guide

Demonstration of conflict resolution - PostgreSQL to Oracle

Demonstration of conflict resolution - PostgreSQL to Oracle

Prerequisite: Before proceeding, make certain the SharePlex demonstration objects are installed. See Prework for the demonstration.

In this demonstration, you will configure SharePlex to use generic conflict resolution procedures to resolve a replication conflict. Generic conflict resolution allows you to use one PL/SQL procedure to resolve conflicts for multiple tables.

The following conflict-resolution strategy is demonstrated:

• Trusted-source priority – In the following steps, you will define one system as the “trusted” source that takes priority in the event of a conflict. This demonstration is based on INSERTs. All INSERTs that originate on the trusted source will override INSERTs from the other system. In this demonstration, the trusted source is peer1(i.e. PostgreSQL) and the other system is peer2(i.e. Oracle).
• Timestamp priority – This demonstration is based on UPDATEs. When there is a conflict, the row that was updated LAST prevails.

IMPORTANT! Peer-to-peer replication is not compatible with all business applications. When suitable for an environment, it requires careful analysis and execution, including the creation of custom conflict resolution procedures that are typically more complex than those in this demonstration. Do not use this demonstration as the foundation of a production peer-to-peer deployment. For more information about peer-to-peer replication, see the SharePlex Administration Guide.

Pre-work for the demonstrations

Install the demonstration objects as described below before you start the demonstrations.

Note: The below pre-work is applicable only for the Oracle peer. For the information on the pre-work of PostgreSQL peer, see Pre-work for the Demonstrations - PostgreSQL.

Install the demonstration objects

The p2p.sql and od.sql scripts install the demonstration objects that you will use during these demonstrations. These scripts are stored in the util sub-directory of the SharePlex product directory.

To run the scripts:

• Run od.sql first, then run p2p.sql.
• Run the scripts on the source and target systems that you will be using for the demonstrations.
• Run them in SQL*Plus as an existing user with the DBA role and SELECT ANY TABLE privileges.

The scripts prompt for the following items:

• The schema where you want the demonstration objects to be installed.
• The tablespace for the demonstration objects.
• Whether or not you want old demonstration objects from a previous version of SharePlex to be removed. To remove the old objects, supply the name of their schema.

Description of the demonstration objects

od_employee

od_timesheet

od_department

od_salary

od_sales_emp_data

oxc_table

Prepare the objects

Note: In this demonstration, the demonstration objects are assumed to be in the schema named demo. Substitute the actual schema, if different.

The following steps provide information on how to prepare the demonstration objects for Oracle peer. For information on how to prepare the demonstration objects of PostgreSQL peer, see Prepare the objects.

Perform these steps on Oracle peer:

1. Run sp_ctrl.

2. Shut down SharePlex.

   sp_ctrl(peer1)>shutdown

3. Run the ora_cleansp utility. This removes the queues from the previous demonstrations and deactivates the previous configuration.
4. TRUNCATE  the od_employee tables.

5. Grant the user who owns the demonstration objects the system privilege to execute the sp_cr package, which was installed in the SharePlex schema when SharePlex was first installed.
   

   SQL>grant execute on sp_cr to user_name

6. Log into SQL *Plus as the user who owns the SharePlex demonstration objects.

7. Run the p2p.sql script from the util sub-directory of the SharePlex product directory. This installs the od_employee_gen demonstration conflict resolution routine. You are prompted for the following:

   • A schema and tablespace for the procedure

   • The name of the SharePlex database user.

   • The name of the system that will be the trusted source of accurate data. As in a production deployment, operations the trusted source take priority during a conflict. This system is known as peer1 in this demonstration. The other system is known as peer2 in this demonstration.

Configure SharePlex

Perform these steps to configure SharePlex:

Note: In this configuration, Peer 1 is Postgresql database and Peer2 is Oracle database. The name of the conflict_resolution file for Oracle is conflict_resolution.sid, and for PostgreSQL it is conflict_resolution.database.

1. On each system, open the conflict_resolution file in a text editor. This file is located in the data sub-directory of the SharePlex variable-data directory.

   Note: Post checks this file when there is a replication conflict to determine if there is a resolution procedure to call.

2. On each system, create the following entries in the conflict_resolution file. Separate each column with at least a few spaces or a tab character.

   demo.od_employee

   IUD

   demo.od_employee_gen

   Note: The first component is a table, the second specifies the operation types for which a resolution routine will be called if there is a conflict on that table, and the third is the name of the resolution routine that will be used.

3. On each system, start sp_cop.
4. On each system, start sp_ctrl.

5. On peer1 (the trusted source), create a configuration file named od_peer1 that replicates the od_employee table to the od_employee table on peer2.

   sp_ctrl(peer1)> create config od_peer1

   datasource:r.dbname
   "demo"."od_employee" ("emp_no", "emp_first_name", "emp_last_name", "emp_dob", "emp_dept_no", "emp_timestamp")	"DEMO"."OD_EMPLOYEE" ("EMP_NO", "EMP_FIRST_NAME", "EMP_LAST_NAME", "EMP_DOB", "EMP_DEPT_NO", "EMP_TIMESTAMP")	peer2@o.sid

    

6. On peer2 (the secondary source), create a configuration file named od.peer2 that replicates the od_employee table to the od_employee table on peer1.

   sp_ctrl(peer2)> create config od_peer2

   datasource:o.sid
   "DEMO"."OD_EMPLOYEE" ("EMP_NO", "EMP_FIRST_NAME", "EMP_LAST_NAME", "EMP_DOB", "EMP_DEPT_NO", "EMP_TIMESTAMP")	"demo"."od_employee" ("emp_no", "emp_first_name", "emp_last_name", "emp_dob", "emp_dept_no", "emp_timestamp")	peer2@r.dbname

    

Note: In the above example, column mapping feature is used since both (source and target) tables have different case sensitivity.

7. Set the SP_OPO_SUPPRESSED_OOS parameter to 0 on peer 2.

Activate and start replication

Perform these steps to activate and start replication:

1. On peer1, activate the od_peer1 configuration.

   sp_ctrl(peer1)> activate od_peer1

2. On peer2, activate the od_peer2 configuration.

   sp_ctrl(peer2)> activate od_peer2

3. On each system, confirm that the configuration activated successfully. The name od_peer1 or od_peer2 (depending on the system) should appear under File Name, and the word Active should appear under State.

   sp_ctrl(source)>list config

Demonstrate trusted-source priority

In this demonstration, an INSERT that originates on peer1 will override a conflicting INSERT that is replicated from peer2.

1. On both systems, stop the Export process.
2. On both systems, log in to database as demo (the owner of the demonstration objects).

3. On peer1, insert a row into od_employee.

   insert into od_employee values(1, 'John', 'Doe', '1949-04-01', 1, '2022-01-01');

4. On peer2, insert the same row (same values).

   INSERT INTO OD_EMPLOYEE VALUES (1,'John','Doe',to_date ('04/01/1949','MM/DD/RRRR'),1,to_date('01/01/2022','MM/DD/RRRR'));

5. On both systems, restart the Export processes.
6. This operation should generate a conflict, which Post resolves automatically based on instructions in the conflict_resolution file.
7. On both systems, view the demo.exc_table to verify that the conflict was resolved. See View the conflict resolution results.

Demonstrate timestamp priority

In this demonstration, whenever there is a conflict for an UPDATE statement, the row with the most current value of the emp_timestamp column gets precedence.

Note: The EMP_TIMESTAMP field, along with the other columns, must be modified to a non-null value in order for timestamp priority to function. The stored procedure won't be able to resolve the conflict if the timestamp column in the UPDATE statement is not being updated or is being modified to a null value. Out-of-sync problems will then be seen.

1. On both systems, stop the Export process.
2. On both systems, log in to database as demo (the owner of the demonstration objects).
3. On peer1, update the emp_first_name and emp_timestamp column of the od_employee table as follows.
   update od_employee set emp_first_name = 'James', emp_timestamp = '2022-07-01' where emp_no = 1;
4. On peer2, update the same column using a different update value but the same key value.
   UPDATE OD_EMPLOYEE SET EMP_FIRST_NAME = 'Harry', EMP_TIMESTAMP = to_date('02/02/2017','MM/DD/RRRR') WHERE EMP_NO = 1;
5. On both systems, restart the Export processes.
6. On both systems, select the row that you updated to verify that it contains the most recent emp_ timestamp value.
7. On both systems, view the demo.exc_table to verify that the conflict was resolved. See View the conflict resolution results.

Demonstrate no conflict resolution

In this demonstration, if a DELETE statement generates a conflict, it is ignored.

1. On both systems, stop the Export process.
2. On both systems, log in to log in to database as demo (the owner of the demonstration objects).
3. On peer1, delete a row as follows.
   delete from od_employee where emp_no = 1;
4. On peer2, delete the same row.
   DELETE FROM OD_EMPLOYEE WHERE EMP_NO = 1;
5. On both systems, restart the Export processes.
6. On both systems, view the demo.exc_table to verify that the conflict is not resolved. See View the conflict resolution results.

View the conflict resolution results

A table named exc_table was installed in the schema that you specified when you installed the demonstration objects. You can view it through PSQL for PostgreSQL and SQLPLUS for Oracle to view information about each conflict. The following is the table description.

Demonstration of transformation for PostgreSQL

Note: Before proceeding, make certain the SharePlex demonstration objects are installed. See Pre-work for the demonstrations.

In this demonstration, you will use a supplied transformation procedure to have SharePlex replicate data from two separate source tables and apply it to one target table.

Prepare the objects

Note: In this demonstration, the demonstration objects are assumed to be in the schema named demo. Substitute the actual schema, if different.

Perform the following steps to prepare the objects:

1. If you ran previous demonstrations, do the following:

   1. On the source and target, run sp_ctrl and issue the following command to shut down sp_cop.

      sp_ctrl (source)shutdown

      sp_ctrl (target)shutdown

   2. Run the pg_cleansp on the source and the ora_cleansp on the target according to the instructions provided in SharePlex Reference Guide. This removes the queues from the previous demonstrations and deactivates the previous configuration.

2. On the source and target, TRUNCATE the od_employee and od_salary tables.

   SQL> truncate table demo.od_employee;

   SQL> truncate table demo.od_salary;

3. On the target, grant the user who owns the demonstration objects the system privilege to execute the sp_cr package, which was installed in the SharePlex schema when SharePlex was first installed.

   SQL>grant execute on sp_cr touser_name

4. On the target, log into SQL*Plus as the user who owns the SharePlex demonstration objects, then run the transform.sql script from the util sub-directory of the SharePlex product directory. This installs transformation routines named od_transform_employee_insert and od_transform_employee_update. You are prompted for:

   • a schema and tablespace
   • the name of the SharePlex database user

Configure SharePlex

Perform the following steps to configure SharePlex:

1. On the target, open the transformation.SID file (where SID is the ORACLE_SID of the target database) in a text editor. This file is located in the data sub-directory in the SharePlex variable-data directory.

   Note: Post checks this file to determine if there is a transformation procedure that it must call instead of posting the operation to the database.

2. Create the following entries in the transformation.SID file.

   Separate each column with at least a few spaces or a tab character.

   demo.od_employee	I	demo.od_transform_employee_insert
   demo.od_employee	U	demo.od_transform_employee_update
   demo.od_salary	I	demo.od_transform_employee_insert
   demo.od_salary	U	demo.od_transform_employee_update

   Note: The components of each entry are as follows, in the order they must appear: The target table to which a transformation procedure is assigned. The operation type for which the specified transformation procedure will be called. The name of the assigned transformation procedure to use. Multiple entries can be used to assign different procedures to different operation types for the same table.

3. On the source, create a configuration file named od.transform that replicates the od_salary and od_employee tables.

   sp_ctrl(source)>create config od_transform

4. In the text editor, build your configuration file based on the following template.

   datasource:o.r.dbid
   demo.od_salary	"DEMO"."OD_SALARY"	target_system@o.target_SID
   demo.od_employee	"DEMO"."OD_EMPLOYEE"	target_system@o.target_SID

5. Save the file, then exit the editor. SharePlex automatically saves the file in the config sub-directory.

Activate and start replication

Perform the following steps to activate and start replication:

1. On the source, activate the configuration.

   sp_ctrl(source)> activate config od_transform

2. Confirm that the configuration activated successfully. The name od_transform should appear under File Name, and the word Active should appear under State.

   sp_ctrl(source)>list config

3. On the source, log in as the demo schema owner, then execute the od_add_emps procedure to populate the od_employee and od_salary tables. Use an IN parameter of 10 to create 50 new employees in the od_sales_emp_data table.

   dbname# call od_add_emps(10);

View the transformed data

Perform the following steps to view the transformed data:

1. On the target, run SQL*Plus.

2. In SQL*Plus, select all rows from od_sales_emp_data.

3. View the transformed data. You should see the following results:

   • The EMPLOYEE_NAME column contains the first and last name of the employee. Compare this to the source od_employee table, where the first and last names are in separate columns.
   • The DEPARTMENT column contains the department name. Compare this to the source od_employee table, where the EMP_DEPT_NO column contains a number. The transformation procedure transformed the replicated department number into the department name by referencing the od_department table.
   • The SALARY column contains the salary from the od_salary table.
4. [OPTIONAL] To see how transformation works for UPDATEs, you can update the od_employee table manually. The od_transform_employee_update procedure will make the transformation. To further this demonstration, you may construct a transformation procedure for DELETEs.

Sample configuration for PostgreSQL to PostgreSQL replication

SharePlex gets its replication instructions from “configurations,” which are user-defined specifications that tell SharePlex what to do. For each group of objects that you want to replicate, you create a configuration file.

Configurations reside on the source system and define:

• The datasource (source database) — the identifier of the source database.
• The source objects — the names of the objects within the source database that contain the data to be replicated. You can replicate some or all of the tables within a database.
• The target objects — the names of the objects in the database on the target system that will receive the replicated data.
• The routing map — the name of the target system and, if the target is a database, its identifier.

To create the demonstration configuration:

1. Create a replication configuration named sample_config by issuing the create config command in sp_ctrl on the source system. This opens the default text editor, which is vi for Linux systems.

   sp_ctrl(sysA)> create config sample_config

   Refer to Template 1 below as you construct your configuration.

   Template 1: Basic demonstration configuration sample_config

   datasource:r.source_DB
   "qarun"."basic_c127"	"splex"."basic_c127"	10.250.14.105@r.sp_ad

    

2. On the first non-commented line of the file, type the following, leaving no space between any of the items.

   datasource:r.source_databasename

   (Substitute the PostgreSQL database name of the source instance for source_databasename.) This tells SharePlex where to find the table whose data will be replicated. The r. tells Share- Plex that PostgreSQL data is being replicated.

3. On the next line, enter the owner name (splex) and table name (demo_src) of the source table, separating the two items with a dot (.) but no spaces. Using the owner’s name with a table name ensures that SharePlex replicates the correct table, since different tables in different schemas in a database could have the same name.

   splex.demo_src
4. Type at least a few spaces or a tab to create a second column. Do not press Enter.

5. In the second column, enter the owner name (splex) and table name (demo_dest) of the target table, separating the two items with a dot (.) but no spaces.

   splex.demo_dest

6. Type a few spaces or a tab to create a third column. Do not press Enter.

7. In the third column, type the following items with no space between them. This creates the routing map for your configuration, telling SharePlex where to put the replicated data.

   • the name of the target system
   • the @ symbol
   • the letter r
   • a dot (.)
   • the target instance

   Example:

   sysB@r.databasename

8. Save the file and exit the editor. This returns you to the sp_ctrl prompt.

9. [OPTIONAL] To view the configuration, issue the view config command in sp_ctrl on the source system:

   sp_ctrl(sysA)>view config sample_config

10. Activate the configuration in sp_ctrl on the source system. Configuration names are case-sensitive.

    sp_ctrl(sysA)>activate config sample_config

11. To confirm that your configuration is active, type the following sp_ctrl command on the source system to display a list of all configurations. The sample_config configuration should appear under “File Name,” and the word “Active” should appear under “State.”

    sp_ctrl(sysA)>list config

Tip: If your configuration activation fails, use the view config sample_config command in sp_ctrl to view the file. Compare it to Template 1 and make sure all of the information you entered is correct. For example, check for extra spaces that are not supposed to be there, or for missing components, such as the r. before the database name.

To correct mistakes in the configuration file:

1. Run the edit config sample_config command in sp_ctrlbefore you activate the configuration file (or if the activation failed).
   This command opens the file in the text editor.
2. Make the changes by editing the file.
3. Save the changes, and re-try the activation.

Note: To change an active configuration, you must copy it to a new file first with the copy config command, and then edit and activate the copy.

Configure, activate, and verify replication on the PostgreSQL source

Note: Before proceeding, make certain the SharePlex demonstration objects are installed.

In this demonstration, you will create and activate a replication configuration, load data to the source table, and then confirm that the data was replicated successfully to the target table. You will also repair a table that is out of synchronization.

Create a configuration file

Note: In this demonstration, the demonstration objects are assumed to be in the schema named demo. Substitute the actual schema, if different.

Perform these steps on the source system:

1. Run sp_ctrl .

2. Issue the following command to create a configuration file named sample_config that replicates the sample_department, sample_salary, sample_timesheet, and sample_employee tables to target tables of the same names on the target system..

   sp_ctrl(source)>create config sample_config

3. In the text editor, build your configuration file based on the following template. Allow no spaces between the characters in a component (source specification, target specification, routing map), and place at least one space between each component.

where:

• # denotes a comment.
• source_databasename is the PostgreSQL database name of the source database.
• target_system is the name or IP address of the target system.
• target_databasename is the PostgreSQL database name of the target database.

4. Save the file, then exit the editor. SharePlex automatically saves the file in the config sub-directory.

5. In sp_ctrl, verify that the configuration file will activate successfully.

   sp_ctrl(source)>verify config sample_config

Syntax configuration

An example of a config file for PostgreSQL to PostgreSQL replication:

An example of a config file for PostgreSQL to Oracle replication:

Activate the configuration

Perform these steps on the source system. When you activate a configuration, SharePlex is ready to capture transactional changes that are made to the specified source data.

1. Activate the configuration.

   sp_ctrl(source)>activate config sample_config

   Note: Configuration names are case-sensitive.

2. Confirm that the configuration activated successfully. The name config sample_config should appear under File Name, and the word Active should appear under State.

   sp_ctrl(source)>list config