Chat now with support
Chat with Support

SharePlex 11.0 - SharePlex Administration Guide

About this Guide Conventions used in this guide Revision History Overview of SharePlex Run SharePlex Run multiple instances of SharePlex Execute commands in sp_ctrl Set SharePlex parameters Configure data replication Configure replication to and from a container database Configure named queues Configure partitioned replication Configure replication to a change history target Configure a replication strategy Configure DDL replication Configure error handling Configure data transformation Configure security features Assign SharePlex users to security groups Start replication on your production systems Monitor SharePlex Prevent and solve replication problems Repair out-of-sync data Tune the Capture process Tune the Post process Recover replication after Oracle failover Make changes to an active replication environment Apply an Oracle application patch or upgrade Back up Oracle data on the source or target Troubleshooting Tips Appendix A: Peer-To-Peer Diagram Appendix B: SharePlex environment variables

Database specifications in a configuration file

A database specification is required in the following components of the configuration file:

  • the datasource (source datastore) specification
  • routing map (target datastore and location) specification
Database

Database type
notation*

 Database Identifiers
Oracle source

o.

Depending on the Oracle database configuration, use one of the following. This is the string that SharePlex will use to connect to the database.

  • The Oracle SID of a regular (non-CDB) Oracle database, as in o.ora12.
  • The TNS alias of a pluggable database (PDB) in an Oracle container database (CDB), as in o.pdb1.
  • The global TNS alias of an Oracle RAC cluster, as in o.rac1. SharePlex connects to an Oracle RAC instance through this TNS alias, which is mapped locally on each node to the Oracle SID of the local Oracle instance. For more information about creating this alias, see Installation and Setup for Oracle Cluster in the SharePlex Installation Guide.
Open Targe targets r.

Use to specify the name of an Open Target (non-Oracle) target database, as in r.mydb.

IMPORTANT! Use the actual database name. Do not use the ODBC datasource name (DNS) or database instance name. If the database name is case-sensitive, specify it that way.

Oracle change-history target c.

Use in a routing map to specify the Oracle SID, TNS alias, or global RAC TNS alias of an Oracle change history database, as in c.ORA12CH. In this configuration, SharePlex applies all source transactions as INSERTs to the target tables, to maintain a history of every operation performed.

For more information, see Configure Replication to a Change History Target.

* Note: The dot is required.

Target specifications in a configuration file

The following table shows how to specify a target table or non-table target in a configuration file.

Target Target Specification Description
Database table tgt_owner.table

The fully qualified name of a database table. For more information, see How to Qualify Object Names.

Database sequence

tgt_owner.sequence

The fully qualified name of a sequence. For more information, see How to Qualify Object Names

File !file[:tgt_owner.table]

The !file designator directs Post to write change operations to a file in SQL, XML, or JSON format. The file name is applied internally by SharePlex.

Optionally, you can specify the fully qualified name of a target table if the data will be consumed by a process that ultimately applies it to a database table.

JMS !jms[:tgt_owner.table]

The !jms designator directs Post to write change operations to a JMS queue or topic in XML format. The queue or topic name can be defined by using the target command.

Optionally, you can specify the fully qualified name of a target table if the data will be consumed by a process that ultimately applies it to a database table.

Kafka

!kafka[:tgt_owner.table]

The !Kafka designator directs Post to write change operations to a Kafka topic in XML or JSON format. The topic name can be defined by using the target command.

Optionally, you can specify the fully qualified name of a target table if the data will be consumed by a process that ultimately applies it to a database table.

Change history table

!cdc:tgt_owner.table

The !cdc designator directs Post to insert every data change to the table as a new row, rather than overlay the old data with new data. Specify the fully qualified name of the change history table.

For more information, see Configure Replication to a Change History Target.

Routing specifications in a configuration file

The following instructions show you how to build a routing map based on where you want to send the source data. A routing map sends replicated data to the correct target on the correct target system, or systems.

For details about the components of these configurations, see:

Database Specifications in a Configuration File

Target Specifications in a Configuration File

Routing to one target

A simple routing map sends replicated data from one source object to one target object.

datasource_specification
src_owner.table tgt_owner.table2 host2[@database_specification]
src_owner.table tgt_owner.table3 host3[@database_specification]

Routing to a cloud service

There are special routing requirements for database targets that are hosted by a cloud service such as EC2 and RDS on Amazon AWS, Azure SQL in Microsoft Azure, and Compute Virtual Machines in Oracle Cloud Infrastructure. Whether the service is Infrastructure as a Service (IaaS) or Platform as a Service (PaaS) makes a difference in how you install and configure SharePlex. The following explains these requirements.

IaaS targets

If replicating to a database target hosted in an IaaS cloud service, specify the full endpoint URL as the target host in the routing map.

datasource_specification
src_owner.table tgt_owner.table2 endpointURL@database_specification
src_owner.table tgt_owner.table3 endpointURL@database_specification

For example, the following routing map routes to a cloud database on Amazon EC2:

ec2-12-345-678-910.compute-1.amazonaws.com@o.myora

Alternately, you can map the private IP address of the cloud service to a short name in the local hosts file, and then specify that name as the host in the routing map, for example:

shortname@o.myora

PaaS targets

If replicating to a database target hosted in a PaaS cloud service, there are special installation, setup, and routing requirements. Because SharePlex cannot be installed directly on a PaaS cloud server, you must install SharePlex on either the source server or an intermediary server, from which Post connects to the target cloud database. For more information, see Installation and setup for cloud-hosted databases in the SharePlex Installation and Setup Guide.

Routing to multiple targets

A compound routing map sends replicated data from one source object to multiple target objects. It enables you to specify the source and target objects once for all routes, rather than type a separate configuration entry for each route. Only one target specification can be used in a compound routing map, so all of the target objects must be identical as follows:

  • All are of one type: All the same database object type or all a JMS queue or all a JMS topic or all a Kafka topic, or all a file (but no combination of these).
  • All have the same fully qualified name, including any table specifications in a JMS, Kafka, or file target specification.

  • All have identical column mappings or key mappings, if used. For more information about these mappings, see:

Notes:
datasource_specification
src_owner.table tgt_owner.table

host1[@database_specification]+host2[@database_specification][...]

Routing between objects on the same system

You can replicate between the following:

  • For Oracle, you can replicate between objects that are in the same database or in different databases on the same system. You can replicate between objects that have the same name, so long as their owners are different.

When SharePlex replicates between objects on the same system, it does not create Import and Export processes. You can force SharePlex to create Import and Export processes by using the following routing map. If you do not need the Import or Export processes, omit the host* portion of the routing map.

Configuration with replication to objects in the same or different database on the same system
datasource_specification
src_owner.table tgt_owner.table

host*host[@database_specification]

Routing limitations

  • By default, SharePlex supports replication to a maximum of 19 direct target systems. That is the maximum number of processes that can read the export queue. To replicate to more than 19 targets, use named export queues. With each additional queue that you add, you can replicate to 19 additional targets. For more information, see Configure Named Export Queues.
  • Each instance of sp_cop on a system permits a maximum of 1024 different routes. This limitation includes each route that uses a different named post queue (see Configure Named Post Queues.) If your replication strategy requires more than 1024 routes, consider using one or more intermediary systems to divide the routes among multiple sp_cop instances. For more information, see Configure Replication to Share or Distribute Data.
  • By default, each sp_cop instance allows a total of 25 queues on a system. There will always be one capture queue on a source system and one post queue on a target. Therefore, you can have as many as 24 named export queues on a source system and 24 named post queues on a target system. If a system serves as both a source and target, you will have both a capture queue and a post queue. That allows you to create up to 23 named queues of either type (or a mix of both). If system memory permits, you can change the number of allowed queues by setting the SP_QUE_MAX_QUEUES parameter. See the SharePlex Reference Guide for more information about this parameter.

Configuration examples

Configuration Examples by Data Source and Target

These are examples of basic configuration files according to each possible datasource type and target type.

Replicate from a regular Oracle instance to a regular Oracle instance

This configuration applies to on-premises and IaaS cloud deployments. See the System Requirements in the SharePlex Installation and Setup Guide for supported database versions and platforms.

Datasource:o.SID
src_owner.table tgt_owner.table

host@o.SID

Example

The following example replicates table SCOTT.EMP from Oracle instance oraA to target table SCOTT.EMP2 in Oracle instance oraB on target system sysprod.

Datasource:o.oraA
SCOTT.EMP    SCOTT.EMP2    sysprod@o.oraB

Replicate from Oracle to target Oracle in PaaS Cloud

To replicate from an on-premises or IaaS-based Oracle source to a target Oracle database hosted in a PaaS cloud, the SharePlex target components (Import and Post) must run on the source server or on an intermediary server. Post connects through a remote connection using a TNS alias. To set up this topology, see Installation and setup for cloud-hosted databases in the SharePlex Installation and Setup Guide.

Datasource:o.SID
src_owner.table tgt_owner.table

source_or_intermediary_host@o.SID

Example

The following example replicates table SCOTT.EMP from Oracle instance oraA to target table SCOTT.EMP2 in the PaaS cloud Oracle instance oraB. Post runs on intermediary target system sysprod2.

datasource:o.oraA
SCOTT.EMP     SCOTT.EMP2    sysprod2@o.oraB

Replicate from a regular Oracle instance to an open target database

This configuration applies to on-premises and IaaS cloud deployments. See the System Requirements in the SharePlex Installation and Setup Guide for supported database versions and platforms.

Datasource:o.SID
src_owner.table tgt_owner.table host@r.database_name
Example

The following example replicates table SCOTT.EMP from Oracle instance oraA to target table Scott2.Emp2 in Open Target database mydb on target system sys2. The target table is case-sensitive.

Datasource:o.oraA
SCOTT.EMP    "Scott2"."Emp2"    sys2@r.mydb

Replicate from a regular Oracle instance to a file in XML or SQL format

Datasource:o.SID
src_owner.table !file host
Example

The following example replicates table SCOTT.EMP from Oracle instance oraA to a file on target system sysprod.

Datasource:o.oraA
SCOTT.EMP   !file   sysprod

Replicate from a regular Oracle instance to a JMS queue or topic

Datasource:o.SID
src_owner.table !jms host
Example

The following example replicates table SCOTT.EMP from Oracle instance oraA to a JMS queue on target system sysprod.

Datasource:o.oraA
SCOTT.EMP   !jms   sysprod

Replicate from a regular Oracle instance to a Kafka topic

Datasource:o.SID
src_owner.table !kafka host
Example

The following example replicates table SCOTT.EMP from Oracle instance oraA to a Kafka topic using SharePlex target system sysprod.

Datasource:o.oraA
SCOTT.EMP   !kafka  sysprod

Replicate from Oracle to Kafka using SSL encryption

This configuration is applicable for the Kafka target.

Pre-requisites:

Configure the Kafka target with the following parameters to post data onto Kafka before starting the replication with SSL encryption:

  • sp_ctrl target x.kafka set kafka broker = <kafka-server-hostname>:<Kafka-server-port>
  • sp_ctrl target x.kafka set kafka security.protocol = SSL
  • sp_ctrl target x.kafka set kafka ssl.ca.location = <ca-cert-file-path>
  • sp_ctrl target x.kafka set kafka ssl.certificate.location = <.pem-file-path>
  • sp_ctrl target x.kafka set kafka ssl.key.location = <ssl-key-file-path>
  • sp_ctrl target x.kafka set kafka ssl.key.password = <key-password>
  • sp_ctrl target x.kafka set kafka sasl.mechanisms = PLAIN
  • sp_ctrl target x.kafka set kafka api.version.request = true
Notes:

  • All the values in the <> need to be replaced with actual parameters from the Kafka producer.

  • Before setting the target Kafka replication parameters, you need to first stop Poster, set the parameters, and then start the Poster.

This configuration applies to on-premises and IaaS cloud deployments. See the System Requirements in the SharePlex Installation and Setup Guide for supported database versions and platforms.

Datasource:o.SID
src_owner.table !kafka host

Example:

The following example replicates table SCOTT.EMP from Oracle instance oraA to the Kafka server using SharePlex target system sysprod

Datasource:o.oraA
SCOTT.EMP    !kafka    sysprod

Replicate from Oracle to Kafka using SASL authentication

This configuration is applicable for the Kafka target.

Pre-requisites:

Configure the Kafka target with the following parameters to post data onto Kafka before starting the replication with SASL authentication.

  • sp_ctrl target x.kafka set kafka api.version.request=true
  • sp_ctrl target x.kafka set kafka sasl.mechanisms=PLAIN
  • sp_ctrl target x.kafka set kafka sasl.username=<username>
  • sp_ctrl target x.kafka set kafka sasl.password=<password>
  • sp_ctrl target x.kafka set kafka security.protocol=SASL_PLAINTEXT
Notes:

  • All the values in the <> need to be replaced with actual parameters from the Kafka producer.

  • Before setting the target Kafka replication parameters, you need to first stop Poster, set the parameters, and then start the Poster.

This configuration applies to on-premises and IaaS cloud deployments. See the System Requirements in the SharePlex Installation and Setup Guide for supported database versions and platforms.

Datasource:o.SID
src_owner.table !kafka host

Example:

The following example replicates table SCOTT.EMP from Oracle instance oraA to the Kafka server using SharePlex target system sysprod.

Datasource:o.oraA
SCOTT.EMP    !kafka    sysprod

Replicate from Oracle to Kafka using Kerberos authentication

This configuration is applicable for the Kafka target.

Pre-requisites:

Configure the Kafka target with the following parameters to post data onto Kafka before starting the replication with kerberos authentication

  • sp_ctrl target x.kafka set kafka sasl.kerberos.keytab = <kerberos-keytab-file>
  • sp_ctrl target x.kafka set kafka sasl.kerberos.kinit.cmd = <kerberos-kinit-cmd>

Note: The {broker.name} property is no longer supported for the sp_ctrl target x.kafka set kafka sasl.kerberos.kinit.cmd = <kerberos-kinit-cmd> parameter.

  • sp_ctrl target x.kafka set kafka sasl.kerberos.min.time.before.relogin = <relogin-time>
  • sp_ctrl target x.kafka set kafka sasl.kerberos.principal = <kerberos-principal>
  • sp_ctrl target x.kafka set kafka sasl.kerberos.service.name = <kerberos-service-name>
  • sp_ctrl target x.kafka set kafka sasl.mechanisms = GSSAPI
Notes:

  • All the values in the <> need to be replaced with actual parameters from the Kafka producer.

  • Before setting the target Kafka replication parameters, you need to first stop Poster, set the parameters, and then start the Poster.

This configuration applies to on-premises and IaaS cloud deployments. See the System Requirements in the SharePlex Installation and Setup Guide for supported database versions and platforms.

Datasource:o.SID
src_owner.table !kafka host

Example:

The following example replicates table SCOTT.EMP from Oracle instance oraA to the Kafka server using SharePlex target system sysprod

Datasource:o.oraA
SCOTT.EMP    !kafka    sysprod

Replicate from Oracle to Kafka using mTLS authentication

This configuration is applicable for the Kafka target.

Note: Before setting the target Kafka replication parameters, you need to first stop Poster, set the parameters, and then start the Poster.

This configuration applies to on-premises and IaaS cloud deployments. See the System Requirements in the SharePlex Installation and Setup Guide for supported database versions and platforms.

Datasource:o.SID
src_owner.table !kafka host

Example:

The following example replicates table SCOTT.EMP from Oracle instance oraA to the Kafka server using SharePlex target system sysprod

Datasource:o.oraA
SCOTT.EMP    !kafka    sysprod

Replicate data from Oracle to Azure Event Hubs

These configurations need to be done for replicating data from source to Azure Event Hubs. SharePlex for Kafka is used to communicate with Azure Event Hubs through the Kafka Event Hubs connectors.

Pre-requisites:

Configure the SharePlex source machine with the following Kafka parameters to post data onto Azure Event Hubs before starting the replication.

  • sp_ctrl Target x.kafka set kafka api.version.request = true
  • sp_ctrl target x.kafka set kafka broker = <Azure Event Hubs namespace>:<Kafka-server-port>
  • sp_ctrl Target x.kafka set kafka sasl.mechanisms = PLAIN
  • sp_ctrl Target x.kafka set kafka sasl.username = $ConnectionString
  • sp_ctrl Target x.kafka set kafka sasl.password =<Primary key generated in Event Hubs namespace>
  • sp_ctrl Target x.kafka set kafka security.protocol = SASL_SSL
  • sp_ctrl Target x.kafka set kafka topic = <Kafka Event Hubs topic generated inside Event Hubs namespace>
Notes:

  • All the values in the <> need to be replaced with actual parameters from Azure Event Hubs.

  • Before setting the target Kafka replication parameters, you need to first stop Poster, set the parameters, and then restart the Poster.

This configuration applies to on-premises and IaaS cloud deployments. See the System Requirements in the SharePlex Installation and Setup Guide for supported database versions and platforms.

Datasource:o.SID
src_owner.table !kafka src_hostname

 

Example:

The following example replicates table SCOTT.EMP from Oracle instance oraA to the Azure Event Hub using SharePlex target system sysprod.

Datasource:o.oraA
SCOTT.EMP    !kafka    sysprod

Replicate data from Oracle to SQL Server

This configuration applies to on-premises and IaaS cloud deployments. See the System Requirements in the SharePlex Installation and Setup Guide for supported database versions and platforms.

Datasource:o.SID
src_owner.table dst_owner.table dst_hostname

Example:

The following example replicates table SCOTT.EMP from Oracle instance oraA to the MS SQL Server on host sysprod

Datasource:o.oraA
SCOTT.EMP    SCOTT.EMP   sysprod@r.sp_ss

Replicate Data from Oracle to Azure SQL database

This configuration applies to on-premises and IaaS cloud deployments. See the System Requirements in the SharePlex Installation and Setup Guide for supported database versions and platforms.

Datasource:o.SID
src_owner.table dst_owner.table dst_hostname

Example:

The following example replicates table SCOTT.EMP from Oracle instance oraA to the Azure SQL database on host sysprod

Datasource:o.oraA
SCOTT.EMP    SCOTT.EMP   sysprod@r.azuresqldb

Replicate data from Oracle to PostgreSQL database

This configuration applies to on-premises and IaaS cloud deployments. See the System Requirements in the SharePlex Installation and Setup Guide for supported database versions and platforms.

Example:

The following example replicates table SCOTT.EMP from Oracle instance oraA to PostgreSQL target server on host sysprod:

Datasource:o.oraA
SCOTT.EMP    SCOTT.EMP   sysprod@r.sp_ss

Replicate data from Oracle to MySQL database

This configuration applies to on-premises and IaaS cloud deployments. See the System Requirements in the SharePlex Installation and Setup Guide for supported database versions and platforms.

Datasource:o.SID
src_owner.table dst_owner.table dst_hostname

Example:

The following example replicates table SCOTT.EMP from Oracle instance oraA to MySQL target server on host sysprod:

Datasource:o.oraA
SCOTT.EMP    SCOTT.EMP   sysprod@r.sp_ss

Replicate from and to an Oracle pluggable database (PDB) in a container database (CDB)*

This configuration applies to on-premises and IaaS cloud deployments. See the System Requirements in the SharePlex Installation and Setup Guide for supported database versions and platforms.

Datasource:o.PDBalias
src_owner.table tgt_owner.table

host@o.PDBalias

Example

This example replicates table SCOTT.EMP from an Oracle PDB that uses the TNS alias of aliasA to target table SCOTT.EMP in an Oracle PDB that uses the TNS alias of aliasB on target system sysprod.

Datasource:o.aliasA
SSCOTT.EMP    SCOTT.EMP    sysprod@o.aliasB

* You can also replicate data from an Oracle PDB to any other supported target. For more information, see Configure Capture and Delivery.

Replicate to maintain a change history target

This configuration applies to on-premises and IaaS cloud deployments. See the System Requirements in the SharePlex Installation and Setup Guide for supported database versions and platforms.

Datasource: o.SID

src_owner.table

!cdc:tgt_owner.table

host@c.SID

Example

The following example replicates table SCOTT.EMP from Oracle instance oraA to change-history target table SCOTT.EMP2 in Oracle instance oraB on target system sysprod.

Datasource:o.oraA
SCOTT.EMP     !cdc:SCOTT.EMP2     sysprod@c.oraB

For more information, see Configure Replication to a Change History Target.

Replicate data from Oracle to Oracle using Extended Data Types

These configurations need to be done for replicating data from Oracle to Oracle using extended data types.

Pre-requisite:

Your Oracle database should be supporting the Extended Data Type.

Limitation:

SharePlex does not replicate data with Extended Data Type when target type is SQL, JMS, File, or Kafka.

This configuration applies to on-premises and IaaS cloud deployments. See the System Requirements in the SharePlex Installation and Setup Guide for supported database versions and platforms.

Datasource:o.SID
src_owner.table tgt_owner.table

host@o.SID

Example

The following example replicates table SCOTT.EMP from Oracle instance oraA to target table SCOTT.EMP2 in Oracle instance oraB on target system sysprod.

Datasource:o.oraA
SCOTT.EMP    SCOTT.EMP2    sysprod@o.oraB

Replicate data from Oracle to Snowflake

This configuration applies to on-premises and IaaS cloud deployments. See the System Requirements in the SharePlex Installation and Setup Guide for supported database versions and platforms.

Datasource:o.SID
src_schema.table dst_schema.table dst_hostname

Example:

The following example replicates table scott.emp from Oracle instance oraA to the Snowflake on host sysprod

Datasource:o.oraA
"scott"."emp"    "SCOTT"."EMP"   sysprod@r.dbname

Replicate data from PostgreSQL to PostgreSQL database

This configuration applies to on-premises and IaaS cloud deployments. See the System Requirements in the SharePlex Installation and Setup Guide for supported database versions and platforms.

Datasource:r.dbname
src_schema.table dst_schema.table dst_hostname

Example:

The following example replicates table SCOTT.EMP from a PostgreSQL instance dbnameA to a PostgreSQL target server on host hostB:

Datasource:r.dbnameA
scott.emp    scott.emp   hostb@r.mydb

Replicate data from PostgreSQL to Oracle database

This configuration applies to on-premises and IaaS cloud deployments. See the System Requirements in the SharePlex Installation and Setup Guide for supported database versions and platforms.

Datasource:r.dbname
src_schema.table dst_owner.table dst_hostname

Example:

The following example replicates table SCOTT.EMP from a PostgreSQL instance dbnameA to an Oracle target server on host hostB:

Datasource:r.dbnameA
"scott"."emp"    "scott"."emp"   hostB@o.mydb

Replicate from a PostgreSQL instance to a Kafka topic

Datasource:r.dbname
src_schema.table !kafka host
Example

The following example replicates table SCOTT.EMP from PostgreSQL instance testdbA to a Kafka topic using SharePlex target system targetHost.

Datasource:r.testdbA
scott.emp   !kafka  targetHost

Replicate data from PostgreSQL to SQL Server

This configuration applies to on-premises and IaaS cloud deployments. See the System Requirements in the SharePlex Installation and Setup Guide for supported database versions and platforms.

Datasource:r.dbname
src_schema.table dst_owner.table dst_hostname

Example:

The following example replicates table SCOTT.EMP from PostgreSQL instance testdbA to MS SQL Server on host sysprod

Datasource: r.testdbA
SCOTT.EMP    SCOTT.EMP   sysprod@r.dbname

Replicate data from PostgreSQL to Snowflake

This configuration applies to on-premises and IaaS cloud deployments. See the System Requirements in the SharePlex Installation and Setup Guide for supported database versions and platforms.

Datasource:r.dbname
src_schema.table dst_schema.table dst_hostname

Example:

The following example replicates table scott.emp from PostgreSQL instance testdbA to Snowflake on host sysprod

Datasource: r.testdbA
"scott"."emp"    "SCOTT"."EMP"   sysprod@r.dbname

Related Documents

The document was helpful.

Select Rating

I easily found the information I needed.

Select Rating