SharePlex can post replicated PostgreSQL data to an Oracle target database through an Open Database Connectivity (ODBC) interface. SharePlex supports all implementations of the PostgreSQL open-source database.
These instructions contain setup instructions that are specific to this target. Install SharePlex on the source and target according to the appropriate directions in this manual before performing these setup steps.
For the versions, data types and operations that are supported when using SharePlex to replicate to this target, see the SharePlex Release Notes.
Set up SharePlex and the database on the PostgreSQL source and an Oracle target system as follows.
On the source, create a SharePlex configuration file that specifies capture and routing information.
Datasource:r.dbname | ||
src_schema.table |
tgt_owner.table |
host@o.database_name |
where:
* Important!
Target schema or table name defined in the database is case sensitive, be certain to:
Note: This is a basic one-source, one-target configuration using no additional SharePlex configuration features. See "Configure SharePlex to replicate data" in the SharePlex Administration Guide for important information about creating a configuration file and for additional setup instructions for more complex replication scenarios. |
# data2k configuration file | ||
datasource:r.source_databasename | ||
"demo"."data2k" | "DEMO"."DATA2K" | target_system@o.target_databasename |
The SharePlex Post process can connect and write to a Kafka broker. The data can be written in JSON or XML output as a sequential series of operations as they occurred on the source, which can then be consumed by a Kafka consumer.
These instructions contain setup instructions that are specific to this target. Install SharePlex on the source and target according to the appropriate directions in this manual before performing these setup steps.
For the versions, data types and operations that are supported when using SharePlex to replicate to this target, see the SharePlex Release Notes.
Kafka
Use the below command options on the SP_CTRL for Kafka:
target x.kafka set kafka broker= 10.250.40.42:9092 (IP address of Kafka server:port number)
target x.kafka set kafka topic= kafkaTarget (topic name)
target x.kafka set format record = xml/JSON
target x.kafka show
When replicating data to Kafka, configure the source database and SharePlex on the source system as follows.
On the source, create a SharePlex configuration file that specifies capture and routing information. The structure that is required in a configuration file varies, depending on your replication strategy, but this shows you the required syntax for routing data to Kafka.
Datasource:r.dbname | ||
src_schema.table | !kafka | host |
where:
host is the name of the target system.
Note: See Configure SharePlex to replicate data in the SharePlex Administration Guide.
Datasource:r.testdb
MY_SCHEMA.MY_TABLE !kafka targetHost
These instructions configure the SharePlex Post process to connect to Kafka. You must have a running Kafka broker.
Perform these steps to configure post to Kafka:
Issue the target command to configure posting to a Kafka broker and topic. The following are example commands.
sp_ctrl> target x.kafka set kafka broker=10.250.40.42:9092
sp_ctrl> target x.kafkaset kafka topic=shareplex
See Set up replication from PostgreSQL to Kafka for command explanations and options.
SharePlex can output to either XML or JSON format as input to Kafka. XML is the default. To set the input format and specify format options, use one of the following target commands:
target x.kafka set format record=json
or:
target x.kafka set format record=xml
To view samples of these formats, see the format category of the target command documentation in the SharePlex Reference Guide.
To view current property settings for output to Kafka, use the following target command:
target x.kafka show
Table 13: Kafka target properties
Property | Input Value | Default | |
---|---|---|---|
broker=broker |
Required. The host and port number of the Kafka broker, or a comma delimited list of multiple brokers. This list is the bootstrap into the Kafka cluster. So long as Post can connect to one of these brokers, it will discover any other brokers in the cluster. |
localhost:9092 | |
client_id=ID |
Optional. A user-defined string that Post will send in each request to help trace calls. |
None | |
compression.code={none, gzip, snappy} | Optional. Controls whether data is compressed in Kafka. Options are none, gzip or snappy. | None | |
partition={number | rotate | rotate trans} |
Required. One of the following:
|
0 | |
request.required.acks=value |
Optional. This is a Kafka client parameter. By default it is set to a value of -1, which means all. Consult the Kafka documentation about this subject, because all really means all in-sync replicas. This parameter can be used in conjunction with the min.insync.replicas broker parameter to tune behavior between availability and data consistency. Important: It is possible for data to be lost between a Kafka producer (SharePlex in this case) and a Kafka cluster, depending on these settings. |
-1 | |
topic=topic_name |
Required. The name of the target Kafka topic. This string may contain the special sequences %o or %t. The %o sequence is replaced by the schema name of the table that is being replicated. The %t sequence is replaced by the table name of the table that is being replicated. This feature may be used in conjunction with a Kafka server setting of auto.create.topics.enable set to 'true'. Also view your server settings for default.replication.factor and num.partitions because these are used as defaults when topics are auto created.
|
shareplex |
* To avoid latency, if Post detects no more incoming messages, it sends the packet to Kafka immediately without waiting for the threshold to be satisfied.
SharePlex can post replicated source data to a Microsoft SQL Server target database through an Open Database Connectivity (ODBC) interface.
These instructions contain setup instructions that are specific to this target. Install SharePlex on the source and target according to the appropriate directions in this manual before performing these setup steps.
For the versions, data types and operations that are supported when using SharePlex to replicate to this target, see the SharePlex Release Notes.
When replicating data to SQL Server, configure the source database and SharePlex on the source system as follows.
On the source, create a SharePlex configuration file that specifies capture and routing information. The structure that is required in a configuration file varies, depending on your replication strategy, but this shows you the required syntax for routing data to SQL Server.
Datasource:r.dbname | ||
src_schema.table | tgt_owner.table | host |
where:
tgt_owner.table is the owner and name of the target table.
host is the name of the target system.
Note: See Configure SharePlex to replicate data in the SharePlex Admin Guide.
The following example replicates table SCOTT.EMP from PostgreSQL instance dbnameA to the MS SQL Server on host sysprod
Datasource: r.testdb SCOTT.EMP SCOTT.EMP sysprod@r.mydb
Perform the following steps to configure SharePlex on the target:
Make certain that the database setup meets all of the requirements in Open target checklist .
Overview of SharePlex Setup on Cloud for PostgreSQL
Post to PaaS Cloud from the Source Server for PostgreSQL
Post to PaaS Cloud from an Intermediary Server for PostgreSQL
© ALL RIGHTS RESERVED. Terms of Use Privacy Cookie Preference Center