SharePlex Connector for Hadoop replicates the tables listed in the file ConfigFile. Follow these steps to edit this list of tables.
Use the SharePlex for Oracle copy config command to make a copy of ConfigFile.
TIP: Enter command /u01/app/shareplex/prod/bin > ./sp_ctrl to open the sp_ctrl ()> prompt.
copy config ConfigFile to NewConfigFile
Use the edit config command to edit the file NewConfigFile. The file is opened in vi. List the Oracle table(s) you want captured into Hadoop, one table per line.
edit config NewConfigFile
#######################
datasource: O.OracleSID
OracleSchema.OracleTable !jms[:TargetSchema.TargetTable] IPHostPostQueue[:PostQueueName]
#####################
Example line: soo70.G_AUTHORS !jms 10.20.26.28:q2
IPHostPostQueue is the name or IP address of the host on which the SharePlex post queue is running.
For more information on PostQueueName see Configure ActiveMQ to work with SharePlex.
TIP: To verify there are no errors in the config file run command sp_ctrl ()> verify config NewConfigFile.
SharePlex for Oracle uses the post queue to send messages to the JMS queue. Stop the post queue.
Activate the new config file.
activate config NewConfigFile
I) Run conn_snapshot.sh
To enable HDFS Near Real Time Replication or HBase Real Time Replication execute the SharePlex Connector for Hadoop conn_snapshot.sh script.
Specify an Oracle table to replicate. This makes a copy of that Oracle table. Note: Only the HDFS Near Real Time Replication feature supports replication of tables that do not have a primary key.
If you declared more than one Oracle table in ConfigFile in step 2, then run conn_snapshot.sh for only those tables which are added for replication.
Note: Take a snapshot of the tables that are added for replication before you start the post queue.
The conn_snapshot.sh script is fully customizable. It is fully documented in conn_snapshot.sh.
You will be prompted to enter the Oracle password. This is the password to the Oracle username supplied during configuration. Run conn_setup.sh
II) Run conn_cdc.sh
To start capturing change data on HDFS for an Oracle table, execute the SharePlex Connector for Hadoop conn_cdc.sh script.
If you declared more than one Oracle table in ConfigFile in step 2, then run conn_cdc.sh for only those tables which are added for capturing change data.
|
Note: Run the conn_cdc.sh script before you start the post queue. |
The conn_cdc.sh script is fully customizable. It is fully documented in conn_cdc.sh (conn_cdc.sh).
Captures change data on HDFS for insert, update and delete operations.
conn_cdc.sh -t Schema.Table
Note: You will be prompted to enter the Oracle password. This is the password to the Oracle user name supplied during configuration. For more information, see Run conn_setup.sh.
Start the post queue so SharePlex for Oracle can send messages from the post queue to the JMS queue.
SharePlex Connector for Hadoop installs Apache Derby and starts the Derby Network Server.
Set the DERBY_HOME environment variable. Change the current working directory to the SharePlex Connector for Hadoop home, bin and execute $ source shareplex_hadoop_env.sh.
To start Derby, execute:
$ java -jar $DERBY_HOME/lib/derbynet.jar start –p <PORT_NUM>
To stop Derby, execute:
$ java -jar $DERBY_HOME/lib/derbynet.jar shutdown –p <PORT_NUM>
Ensure you enter PORT_NUM as it was set when you ran install.sh. The default is 1527. To look up the PORT_NUM you set, in SharePlex Connector for Hadoop home, conf, open the configuration file (by default connectorConfiguration.xml) and look up the port number entry:
<entry key="derbyPort">1527</entry>
Use this script to setup SharePlex Connector for Hadoop and provide the necessary configuration details. Setup is usually a one-time activity. This script is similar to ora_setup of SharePlex for Oracle.
Shell Script Usage
[user@host bin]$ ./conn_setup.sh [-c <INPUT_FILE>] [-r] [-n][-d][--help] [--version]
Options
Parameter |
Description |
---|---|
-c <INPUT_FILE> |
Provide an input file that defines the configuration parameters with their values. Each configuration parameter should be specified on a new line of the input file. Format - Key=Value. Example – enableRealTimeCopy=True You may like to use this parameter to modify your original configuration. When conn_setup.sh is run without parameters the file it creates is conf/connectorConfiguration.xml There is an example input file located at conf/conn_setup_template.properties |
-r |
Use to update the configuration
|
-n |
Use to update the configuration
|
-d |
Use to update the configuration
|
--help |
Show this help and exit. |
--version |
Show version information and exit. |
Note: The user must restart the connector after enabling any of the features using –r/-n/-d option.
Configuration Parameters
If conn_setup.sh is run without an input file then you will be prompted to supply a value to each of the configuration parameters. For more information, see Run conn_setup.sh.
Default values are provided within brackets. Press Enter to select the default value.
© 2024 Quest Software Inc. ALL RIGHTS RESERVED. Terms of Use Privacy Cookie Preference Center