Chat now with support
Chat with Support

SharePlex 8.6.6 - Reference Guide

About this guide Conventions used in this guide SharePlex commands SharePlex parameters SharePlex utilities Appendix B: SharePlex environment variables

Configure horizontally partitioned replication

Use horizontally partitioned replication to divide the rows of a table into separate processing streams. You can use horizontally partitioned replication to:

  • Replicate a subset of rows to the target, while retaining the rest of the rows in the source.
  • Replicate different subsets of rows to different targets.
  • Divide the replication of a source table into parallel Post processes for faster posting to the target table.

Guidelines for using horizontally partitioned replication

The following are guidelines to follow if you are considering a configuration that combines horizontally partitioned replication with vertically partitioned or full-table replication.

  • You can combine horizontally partitioned and vertically partitioned replication for maximum control over which information is distributed, and to where.

    For example:

    A company has a headquarters and regional divisions. The headquarters maintains the corporate database, and each region maintains a regional database. The headquarters uses vertically partitioned replication to share some of the column data of a table to those locations, while retaining other sensitive data at headquarters. Row changes made to the shared columns are further partitioned horizontally, for replication to the appropriate regional database.
  • Horizontally partitioned replication can be used in conjunction with full-table replication for the same table, for example to route groups of rows to different reporting systems and all rows to a failover system. However, SharePlex does not support that combination if vertically partitioned replication is also used for the same table.

How it works

To implement horizontally partitioned replication, you do the following:

  1. Define partitions
  2. Define partition schemes
  3. Specify partition schemes in the configuration file

Define partitions

A partition is a subset of the rows in a table that you want to replicate. To create a partition, you specify a column condition or a hash value, depending on your objectives. The following explains these concepts.

Column condition: A column condition is based on a standard WHERE clause. Use one or more column conditions for the following purposes:

  • Use a column condition to replicate only a subset of the rows of a table. For example, you can replicate only those rows where the value of the YEAR column is greater than 2000.
  • Use multiple column conditions to partition the rows of a table so that each set of rows replicates to a different target. For example: replicate changes where the value of the REGION column is EAST to one location and rows where REGION is WEST to a different location.

  • Use multiple column conditions to divide the rows of a table into parallel processing streams (parallel Export-Import-Post streams) for faster posting to the target table. For example, you can improve the flow of replication to a heavily updated target table. This use of column conditions for this purpose is appropriate only if the table contains a column that enables you to split the processing evenly among parallel Post processes.

Hash value: A hash partition (known as hash horizontal partitioning) directs SharePlex to create row partitions by using a hash algorithm. You can use a hash value to divide the rows of a table into parallel Post processing streams if it is not practical to use a column condition for this purpose. The advantage of using a hash partition rather than a column condition is that the rows are divided evenly and automatically, without the need for you to reference table columns in WHERE clauses.

Important: The following restrictions apply:

  • When hash partitioning is in use, the table cannot be compared or repaired with the compare and repair commands.
  • Hash horizontal partitioning cannot be enabled on an index-organized table (IOT) or tables that contain LOBs or LONGs.
  • Hash horizontal partitioning cannot be combined with horizontal partitioning that uses a column condition.
  • Hash horizontal partitioning is not supported for tables with operations that may cause rows to migrate. Examples of operations that cause rows to migrate are:

    • Export or import of the table
    • ALTER TABLE with the MOVE option
    • ALTER TABLE with the SHRINK SPACE option
    • FLASHBACK TABLE
    • Split a partition or combine two partitions
    • Update a value so that it moves to a new partition
    • Online table reoraganization
    • Redefine a table by using dbms_redefinition
    • DML applied to a regular, non-partitioned table that can cause row shift: UPDATE that changes row size so that the data does not fit into the current block, DELETE of a row and then re-insert.

Note: On a Windows system, the use of numerous queues may require the number of semaphores to be increased. If Post returns the error message "shs_SEMERR: an error occurred with the semaphore," see the Post stopped topic in Solve replication problems .

Define partition schemes

A partition scheme is a logical container for related row partitions and is the element that is used in the configuration file to direct SharePlex to use horizontal partitioning. For example, a simple partition scheme named CORPORATE.SALES could have four column conditions, each replicating the appropriate data subsets of the CORPORATE.SALES table to different regional sales offices. You must also specify a partition scheme if using the hash horizontal partitioning option, even though SharePlex creates the partitions.

Specify partition schemes in the configuration file

To direct SharePlex to use a specific partition scheme for a source table, you specify the partition scheme as the routing map in the configuration file. SharePlex obeys the specifications of the partition scheme to process the row subsets.

Define partitions and schemes

These instructions help you to define your partitions and link them to partition schemes. You define and link partitions by using the SHAREPLEX_PARTITION table.

Note: This table was installed in the SharePlex schema on the source system during the installation of SharePlex.

Use one SHAREPLEX_PARTITION table for all configurations that replicate data from the same Oracle instance. Enclose case-sensitive names in double quote marks, for example "Scott"."emp".

To define a partition based on a column condition

For each group of row subsets that you want to define as a partition, use a standard INSERT statement to enter the information described in "SHAREPLEX_PARTITION columns and input into the SHAREPLEX_PARTITION table.

To define a partition based on a hash

Use a standard INSERT statement to enter the information described in "SHAREPLEX_PARTITION columns and input" into one row of the SHAREPLEX_PARTITION table. One row captures the information that SharePlex needs to build the partitions automatically with a hash algorithm.

To route a partition to multiple targets

If a row partition must be sent to multiple target tables and the owner or table names are not all identical, do the following:

  • Issue and commit a separate INSERT statement for each target table that has a different owner or name from the other target tables.
  • Each INSERT must be identical, except to specify the appropriate name values in the TARGET_TABLE_OWNER and TARGET_TABLE_NAME columns, in relation to the target system specified in the ROUTE column.
  • When you create an entry for this partition scheme in the configuration file, specify any target table. SharePlex will detect the other target names when the configuration is activated.
  • Set the SP_ORD_FIRST_FIND parameter to 0 so that SharePlex checks all of the column conditions in the partition scheme. By default SharePlex assumes that any given row change will satisfy only one column condition in the partition scheme. For more information, see the SharePlex Reference Guide.

Table 16: SHAREPLEX_PARTITION columns and input

Column Datatype Input
PARTITION_SCHEME VARCHAR2(30)

The name of the partition scheme for which you are creating a row partition.

You can:

  • Define only one partition scheme per table in the replication configuration.
  • Use any or all of your partition schemes in a configuration file. Only those that are listed in an active configuration file are active.
DESCRIPTION VARCHAR2(61) A description of the partition. You can enter a NULL value, but if you are creating numerous partitions, it helps to identify or describe each one, for example to give each partition a name.
TARGET_TABLE_OWNER VARCHAR2(30)

The owner of the target table. This can be one of the following:

  • A NULL value if there is only one target owner name across all routes, such as when you are using the partitioning to parallelize posting.
  • If you are sending the partition to multiple target tables with different owner names, specify one of the owner names. If case-sensitive, the name must be specified as required by the database. See To route a partition to multiple targets.
TARGET_TABLE_NAME VARCHAR2(30)

The name of the target table. This can be one of the following:

  • A NULL value if there is only one target table name across all routes, such as when you are using the partitioning to parallelize posting.
  • If you are sending the partition to multiple target tables with different names, specify one of the target names. If case-sensitive, the name must be specified as required by the database. See To route a partition to multiple targets.
ROUTE VARCHAR2(1024)

The routing map for this partition. This can be one of the following, depending on whether you are using a column condition or a hash to create the partitions.

If creating a partition with a column condition:

If creating a partition with a hash:

Use the following format to direct SharePlex to automatically generate the named post queues:

host:name|#[o. | r.]database

where:

  • host is the name of the target system.
  • name is the base name of the post queues that SharePlex will create. Each name is appended with an integer from 0 to the number of partitions that you specify in the OPTIONS column.
  • |# directs SharePlex to sequentially number the queues.
  • database is the name or SID of the target database, if applicable.
PRIORITY NUMBER NULL. This column is reserved for use by SharePlex.
ORDER NUMBER

NULL. This column is reserved for use by SharePlex.

OPTIONS VARCHAR2(32)

Specify one of the following, depending on how you are defining the partitions:

If defining a partition based on a column condition:

NULL

If defining a partition based on a hash:

HASH#, where # is the number of partitions (post queues and processes) that you want SharePlex to use. The numbering starts at 0. For example, HASH4 creates the post queues hash00, hash01, hash02, and hash03.

COL_CONDITIONS VARCHAR2(1024)

Specify one of the following, depending on how you are defining the partitions:

If defining a partition based on a column condition:

Use standard WHERE conditional syntax such as ((region_id = West) and region_id is not null). You can specify as many column conditions as needed. See How to create a column condition for additional instructions.

If defining a partition based on a hash:

Specify a value of ROWID. This directs SharePlex to create the hash based on the block where the row resides.

How to create a column condition

The following are guidelines for creating column conditions.

Choose appropriate columns

Use columns whose values will not change, such as PRIMARY or UNIQUE key columns. The objective is to avoid a partition shift, where changes made to the conditional columns of a partition can cause the underlying data to satisfy the conditions of a different (or no) partition.

Partition shift case 1: The column value is updated and the new value no longer satisfies any column condition:
  • SharePlex performs the operation, but future operations on that row are not replicated. The reason: the row no longer satisfies a column condition.
  • The source and target tables of the original partition are now out of synchronization, but Post returns no errors.
  • Example:

    Column condition is “C1=A and C2=B”.

    1. INSERT a row into the database. C1 and C2 values satisfy the column condition, so the row is replicated to the target system.

      C1 C2 C3 C4
      A B 1 2
    2. UPDATE ...set C1=B WHERE C1=A and C2=B

    3. The change is replicated. Now the row does not satisfy the column condition, and subsequent changes to data in the row are not replicated.

      C1 C2 C3 C4
      B B 1 2
Partition shift case 2: A row that satisfies one column condition gets updated to meet a different condition:
  • Post cannot find a matching target row. The reason: the original change was not replicated because it did not meet the column condition.
  • Post returns an out-of-sync error.
  • Example:

    Column condition is “C1=A and C2=B”.

    1. INSERT a row into the database. C1 and C2 values do not satisfy the column condition, so the row is not replicated to the target system.

      C1 C2 C3 C4
      B B 1 2
    2. UPDATE ...set C1=A WHERE C1=B and C2=B

    3. Now the row satisfies the column condition, so SharePlex replicates it, but Post cannot locate the corresponding row on the target (it was never there) and returns an out-of-sync error.

How to repair out-of-sync rows caused by changes to the values of column conditions

You can do either of the following to repair the out-of-sync rows:

  • Set the SP_ORD_HP_IN_SYNC parameter to a value of 1. This setting directs SharePlex to detect and correct the out-of-sync condition. Enabling this parameter causes some performance degradation, depending on how many tables are configured for horizontally partitioned replication. For more information about this parameter, see the SharePlex Reference Guide.
  • Use the compare command to repair the out-of-sync rows. For more information about this command, see the SharePlex Reference Guide.

Note: If you are using a column other than a key to base the column condition on, and you notice reduced performance with horizontally partitioned replication enabled, add a log group for that column.

Use supported datatypes

SharePlex supports the following datatypes in column conditions:

CHAR

DATE

NUMBER

LONG VARCHAR

VARCHAR

VARCHAR2

Notes:

  • For the DATE datatype, SharePlex uses nls_date_format, with the syntax specified as MMDDSYYYYHH24MISS. For example:

    hiredate<‘1111 2011000000’

  • Horizontally partitioned replication does not support the following:

    • Oracle TO_DATE function.
    • VARRAYs and abstract datatypes.
    • UPDATEs or INSERTs on LONG columns larger than 100k.
    • LOB columns.
    • Sequences.
    • TRUNCATEs of an Oracle partition.

Use standard conditional syntax

The following list shows the conditional syntax that SharePlex supports in a column condition, where:

  • value can be a string or a number. Enclose strings and dates within single quote marks (‘west’). Do not use quote marks for numbers (10).
  • column is the name of a column in the table that you are configuring to use horizontally partitioned replication.
column = value
not (column = value)
column > value
value > column
column < value
column <= value
column >= value
column <> value
column != value
column like value
column between value1 and value2
not (column between value1 and value2)
column is null
column is not null

Conditions can be combined into nested expressions with parentheses and the AND, OR, and NOT logical connectives.

Example column conditions

not (col1 = 5)
(col2 = 5) and not (col3 = 6)
((col1 is not null) and (col2 = 5))

Additional guidelines

  • NULLs are replicated by SharePlex in cases such as this one: not (department_id = 90). If department_id is NULL, it is replicated. To avoid replicating records with NULLs, include the column is not null syntax as part of the condition, for example: not (department_id = 90) and department_id is not null.
  • If parentheses are not used to indicate operator precedence, SharePlex supports operator precedence in the same order that Oracle does. For example, a condition NOT x AND y behaves the same way as (NOT x) AND y. A condition x AND y OR z behaves the same as (x AND y) OR z. When a condition includes parentheses, the explicit precedence is respected.
  • Do not:

    • include references to other tables in the column condition.
    • exceed the 1024 bytes maximum defined storage.
  • During the activation of a configuration that refers to partition schemes, SharePlex verifies the syntax in the column conditions of those schemes. If any syntax is incorrect, the activation fails. SharePlex prints an error to the Event Log that indicates where the error occurred.

Specify partition schemes in the configuration file

Use one configuration file for all of the data that you want to replicate from a given datasource, including tables that will have full-table replication and those that will use partitioned replication. For general instructions on creating a configuration file, see Create configuration files. To configure entries for horizontally partitioned replication, use the following syntax.

datasource_specification    
src_owner.table tgt_owner.table

!partition_scheme

    !routing_map

Where:

  • src_owner.table and tgt_owner.table are the specifications for the source and target tables, respectively.

  • ! partition_scheme is the name of the partition scheme to use for the specified source and target tables. The ! is required. The name is case-sensitive. Compound routing of multiple partition schemes is not supported, for example !schemeA+schemeB. Create a separate entry for each partition scheme that you want to use for the same source table. See Examples.
  • ! routing_map is a placeholder routing map. It is required only if a route that you used in a partition scheme is not listed somewhere in the configuration file. SharePlex requires every route to be in the configuration file even if it is listed in a partition scheme.

    Notes:

    • This component applies only to partitions using a column condition.
    • This applies to different named queue routes as well as to routes to different target hosts.
    • You can use a compound routing map if the names of all target tables are identical. See Examples.

Examples

Configuration file entry to specify a partition scheme

Datasource:o.myora    
scott.emp scott.emp_2 !partition_emp
scott.cust scott.cust_2 !partition_cust

Correct way to specify multiple partition schemes for the same source table

Datasource:o.myora    
scott.emp scott.emp_2 !partition_schemeA
scott.emp scott.emp_3 !partition_schemeB

Correct way to specify placeholder routing map*

! targsys1
! targsys2@o.ora2+targsys3@o.ora3

*Required only for partitions based on column conditions.

Change the hash algorithm

When using hash-based horizontally partitioned replication, you can change the hash algorithm from the default of rowid-based to block-based. To enable the block-based hash option, set the SP_OCF_HASH_BY_BLOCK parameter to 1.

Related Documents