Configure data replication
Configure data replication
SharePlex uses a configuration file to determine which tables to replicate and where to send the replicated data. This file also provides any special processing instructions, such as column mapping and data filtering. This chapter contains the information that you need to know in order to create a configuration file.
Once you understand the basics of how to create a configuration file and route data to your targets, you can move on to any of the more advanced configurations as needed. Documentation for those configurations is included in this guide.
Contents
Ensure compatible source-target mapping
The following guidelines help you ensure that the source and target objects that you want to map in a replication configuration are compatible.
Object names
For most replication strategies, the name and/or owner of a source object can be different from that of its target object. SharePlex replicates to the correct object because you specify it by owner and name within the configuration file. For high-availability configurations, the owner and name of a source table should be the same as the owner and name of the target table.
Source and target rows
Corresponding source and target rows must contain the same values to accurately reflect the source, unless transformation is being used.
One database in the configuration can have more or fewer rows than another database in the configuration. You can control which rows are replicated through horizontally partitioned replication. For more information, see Configure horizontally partitioned replication.
Source and target columns
Corresponding source and target columns must:
A target table can have more columns than the source table.
- If the source and related target column names are identical, SharePlex will ignore the extra columns in the target table.
- If the source and target column names are not identical, SharePlex maps a one-to-one relationship in the order that the columns are defined in each table (for example, map the first column in the source to the first column in the target, map the second column to the second column, and so forth).
- To avoid Oracle errors if the extra (non-mapped) columns are NOT NULL, define default values for those columns. For more information, see Map source and target columns.
A target table can have fewer columns than the number of columns in the source table, but you must use vertically partitioned replication to replicate only that subset of the source columns that matches the rows of the target. For more information, see Configure vertically partitioned replication.
Create a configuration file
To configure SharePlex to replicate data, you create a configuration file. This is an ASCII text file in which you specify:
- The data that you want SharePlex to replicate, including any filtering or partitioning of rows or columns
- The target name and type
- The system (and database if applicable) to which the data must be delivered.
Only a SharePlex Administrator or operator has the authority to create a configuration file.
When your configuration file is completed, you activate the configuration with the activate config command to begin replication. For more information, see Start replication on your production systems.
You can create a configuration file by hand in sp_ctrl or, if your data structure supports it, you can automate the creation of a configuration file with a script.
Create the configuration file on the system from which data is to be replicated, typically the source system, but strategies such as active-active replication require configuration files on more than one system.
To create a configuration file in sp_ctrl
- Run sp_ctrl from the bin sub-directory of the SharePlex product directory.
-
In sp_ctrl, issue the create config command.
sp_ctrl> create config config_name
This command opens a file in the default text editor that is set for the operating system. Note: You can change the default editor that sp_ctrl uses. For more information, see Set a default editor for sp_ctrl.
-
Complete the configuration file. For more information, see Structure of a configuration file.
Important! All configurations must reside in the config sub-directory of the SharePlex variable-data directory. Configuration files outside this directory cannot be activated. SharePlex places configurations in this directory by default when you create them through the sp_ctrl interface with the create config command. If you create the configuration directly through a text editor, make certain to save it to the config sub-directory.
To build a configuration file with a script
Note: Valid for Oracle only
SharePlex provides the following scripts that can be used to automate the building of your configuration file if the source and target object names are identical. These scripts support Oracle Database source and targets only.
| config.sql |
Builds a configuration file that includes all tables and sequences that are in the database. Source and target object names must be identical. |
See Configuration Scripts in the SharePlex Reference Guide. |
| build_config.sql |
Builds a configuration file that includes all tables in a schema. Source and target object names must be identical. |
See Configuration Scripts in the SharePlex Reference Guide. |
Structure of a configuration file
A basic configuration file looks like the following:
|
# comment: basic SharePlex configuration file |
|
datasource_specification |
|
#source specification |
target specification |
routing map |
|
source_owner.object1 |
target_owner.object1 |
routing_map |
|
source_owner.object2 |
target_owner.object2 |
routing_map |
|
source_owner.object3 |
target_owner.object3 |
routing_map |
The basic components of a configuration file are as follows.
| # Comments |
Lines that describe the file or provide other information about the contents to viewers, but are not used by SharePlex. Precede each comment line with a pound (#) sign. Comments may be entered anywhere in the configuration file. |
# This is a comment. |
| Datasource specification |
Syntax that specifies the source database. This component must always be the first non-commented line of a configuration file. It has the following syntax elements, all on the same line with no spaces:
|
Datasource:o.SID |
| Source specification |
The fully qualified name of a supported source object, in the form of owner.object. The owner can be a schema or a database, depending on how the database stores objects logically. See:
How to qualify object names
How to specify case-sensitive names
You can use wildcards to specify multiple objects. Owner names cannot be wildcarded. For more information, see Use Wildcards to specify multiple objects. |
src_owner.table
src_owner.sequence |
| Target specification |
The target to which the replicated data is applied. Targets supported by SharePlex are:
-
The fully qualified name of a table (or tables) in a relational database. You can use wildcards to specify multiple objects. Owner names cannot be wildcarded.
For more information, see:
- An Oracle sequence (or wildcard specification).
- A file that contains XML or SQL records.
- A JMS queue or topic.
- A Kafka topic.
- A change-history table that maintains a record of all changes made to a source table (also known as change data capture)
|
tgt_owner.table
tgt_owner.sequence
!file[:tgt_owner.table]
!jms[:tgt_owner.table]
!kafka[:tgt_owner.table]
!cdc:tgt_owner.table |
| Routing map |
One or more routes that send the data to the system that contains the target object specified with the target specification. A route consists of the following:
-
The name of the machine that hosts the target.
-
(Database targets only) The @ symbol followed by the target database specification. For more information, see Database specifications in a configuration file.
The database specification is absent if the target is JMS, Kafka, or a file.
There can be no spaces between any characters in the routing map.
For more information, see Routing specifications in a configuration file. |
host@o.SID
host@o.PDBalias
host@o.tns_alias
host@r.database_name
host
host@c.SID
Compound routing map:
host@o.SID+host@r.database[...] |
How to qualify object names
So that SharePlex can determine the correct objects to capture from, and post to, you must qualify the object names in the configuration file in the same way that the database stores them logically. The general way this is indicated in SharePlex syntax is:
owner.object
Where:
- owner is the schema or database that contains the object (or objects, if wildcarded), depending on how that container is defined by the database.
- object is the name of the object or a wildcard specification to specify multiple objects.
When defining source or target objects in the configuration file, follow these guidelines for specifying the owner component:
| Aurora |
database_name.object_name |
| MySQL |
database_name.object_name |
| Oracle |
schema_name.object_name |
| PostgreSQL |
schema_name.object_name |
| SQL Server |
schema_name.object_name |
| SAP HANA |
schema_name.object_name |
