Edit the List of Tables being Replicated
Use Case: Edit the List of Tables To Be Replicated
SharePlex Connector for Hadoop replicates the tables listed in the file ConfigFile. Follow these steps to edit this list of tables.
1. SharePlex for Oracle - Copy the config file
Use the SharePlex for Oracle copy config command to make a copy of ConfigFile.
copy config ConfigFile to NewConfigFile2. Edit the new config file
Use the edit config command to edit the file NewConfigFile. The file is opened in vi. List the Oracle table(s) you want captured into Hadoop, one table per line.
edit config NewConfigFile #######################
datasource: O.OracleSID
OracleSchema.OracleTable !jms[:TargetSchema.TargetTable] IPHostPostQueue[:PostQueueName]
#####################
Example line: soo70.G_AUTHORS !jms 10.20.26.28:q2
IPHostPostQueue is the name or IP address of the host on which the SharePlex post queue is running.
For more information on PostQueueName see Configure ActiveMQ to work with SharePlex.
sp_ctrl ()> verify config NewConfigFile.
3. Stop the post queue
SharePlex for Oracle uses the post queue to send messages to the JMS queue. Stop the post queue.
4. Activate the new config file
Activate the new config file.
activate config NewConfigFile 5. SharePlex™ Connector for Hadoop® - Run conn_snapshot.sh and/or conn_cdc.sh
I) Run conn_snapshot.sh
To enable HDFS Near Real Time Replication or HBase Real Time Replication execute the SharePlex Connector for Hadoop conn_snapshot.sh script.
Specify an Oracle table to replicate. This makes a copy of that Oracle table. Note: Only the HDFS Near Real Time Replication feature supports replication of tables that do not have a primary key.
If you declared more than one Oracle table in ConfigFile in step 2, then run conn_snapshot.sh for only those tables which are added for replication.
The conn_snapshot.sh script is fully customizable. It is fully documented in conn_snapshot.sh.
You will be prompted to enter the Oracle password. This is the password to the Oracle username supplied during configuration. Run conn_setup.sh
II) Run conn_cdc.sh
To start capturing change data on HDFS for an Oracle table, execute the SharePlex Connector for Hadoop conn_cdc.sh script.
If you declared more than one Oracle table in ConfigFile in step 2, then run conn_cdc.sh for only those tables which are added for capturing change data.
|
|
Note: Run the conn_cdc.sh script before you start the post queue. |
The conn_cdc.sh script is fully customizable. It is fully documented in conn_cdc.sh (conn_cdc.sh).
Captures change data on HDFS for insert, update and delete operations.
conn_cdc.sh -t Schema.Table
6. Start the post queue
Start the post queue so SharePlex for Oracle can send messages from the post queue to the JMS queue.
Start and Stop the Derby Network Server
Use Case: Start and Stop the Derby Network Server
SharePlex Connector for Hadoop installs Apache Derby and starts the Derby Network Server.
Set the DERBY_HOME environment variable. Change the current working directory to the SharePlex Connector for Hadoop home, bin and execute $ source shareplex_hadoop_env.sh.
To start Derby, execute:
$ java -jar $DERBY_HOME/lib/derbynet.jar start –p <PORT_NUM>
To stop Derby, execute:
$ java -jar $DERBY_HOME/lib/derbynet.jar shutdown –p <PORT_NUM>
Ensure you enter PORT_NUM as it was set when you ran install.sh. The default is 1527. To look up the PORT_NUM you set, in SharePlex Connector for Hadoop home, conf, open the configuration file (by default connectorConfiguration.xml) and look up the port number entry:
<entry key="derbyPort">1527</entry>
Command Reference
conn_setup.sh
conn_setup.sh
Use this script to setup SharePlex Connector for Hadoop and provide the necessary configuration details. Setup is usually a one-time activity. This script is similar to ora_setup of SharePlex for Oracle.
Shell Script Usage
[user@host bin]$ ./conn_setup.sh [-c <INPUT_FILE>] [-r] [-n][-d][--help] [--version]
Options
| Parameter |
Description |
|---|---|
|
-c <INPUT_FILE> |
Provide an input file that defines the configuration parameters with their values. Each configuration parameter should be specified on a new line of the input file. Format - Key=Value. Example – enableRealTimeCopy=True You may like to use this parameter to modify your original configuration. When conn_setup.sh is run without parameters the file it creates is conf/connectorConfiguration.xml There is an example input file located at conf/conn_setup_template.properties |
|
-r |
Use to update the configuration
|
|
-n |
Use to update the configuration
|
| -d |
Use to update the configuration
|
|
--help |
Show this help and exit. |
|
--version |
Show version information and exit. |
Configuration Parameters
If conn_setup.sh is run without an input file then you will be prompted to supply a value to each of the configuration parameters. For more information, see Run conn_setup.sh.
Default values are provided within brackets. Press Enter to select the default value.