Tchater maintenant avec le support
Tchattez avec un ingénieur du support

SharePlex Connector for Hadoop 1.0 - Installation and Setup Guide

Edit the list of tables being replicated

SharePlex Connector for Hadoop replicates the tables listed in the file ConfigFile. Follow these steps to edit this list of tables.

1. SharePlex for Oracle - Copy the config file

Use the SharePlex for Oracle copy config command to make a copy of ConfigFile.

Tip: Enter command /u01/app/shareplex/prod/bin > ./sp_ctrl to open the sp_ctrl ()> prompt.

» sp_ctrl ()> copy config ConfigFile to NewConfigFile

2. Edit the new config file

Use the edit config command to edit the file NewConfigFile. The file is opened in vi. List the Oracle table(s) you want captured into Hadoop, one table per line.

» sp_ctrl ()> edit config NewConfigFile

#######################

datasource: O.OracleSID

OracleSchema.OracleTable !jms[:TargetSchema.TargetTable] IPHostPostQueue[:PostQueueName]

#####################

Example line: soo70.G_AUTHORS !jms 10.20.26.28:q2

IPHostPostQueue is the name or IP address of the host on which the SharePlex post queue is running.

For more information on PostQueueName see Configure ActiveMQ to work with SharePlex (page 1).

Tip: To verify there are no errors in the config file run command sp_ctrl ()> verify config NewConfigFile

3. Stop the post queue

SharePlex for Oracle uses the post queue to send messages to the JMS queue. Stop the post queue.

» sp_ctrl ()> stop post queue PostQueueName

4. Activate the new config file

Activate the new config file.

» sp_ctrl ()> activate config NewConfigFile

5. SharePlex Connector for Hadoop - Run conn_snapshot.sh

Execute the SharePlex Connector for Hadoop conn_snapshot.sh script for each NEW Oracle table added to the list of tables in the config file. This makes a copy of the Oracle table for replication.

The conn_snapshot.sh script is fully customizable. It is fully documented in conn_snapshot.sh (page 1).

» conn_snapshot.sh -t Schema.Table -s ';'

NOTE: You will be prompted to enter the Oracle password. This is the password to the Oracle username supplied during configuration. See "Run conn_setup.sh" (page 1) for more information.

6. Start the post queue

Start the post queue so SharePlex for Oracle can send messages from the post queue to the JMS queue.

» sp_ctrl ()> start post queue PostQueueName

Apache Derby Usage

Apache Derby Usage

How to install a new SharePlex Connector for Hadoop build without removing Apache Derby

After SharePlex Connector for Hadoop is installed, to reinstall SharePlex Connector for Hadoop without losing the data in Apache Derby:

  1. Backup the connector-derby-database folder in SharePlex Connector for Hadoop home.
  2. Install SharePlex Connector for Hadoop.
  3. Copy the connector-derby-database folder (from step 1) into the new SharePlex Connector for Hadoop home directory.

How to start/stop the Derby Network Server

SharePlex Connector for Hadoop installs Apache Derby and starts the Derby Network Server.

First: Set the DERBY_HOME environment variable. Change the current working directory to the SharePlex Connector for Hadoop home, bin and execute $ source shareplex_hadoop_env.sh

To start Derby execute $ java -jar $DERBY_HOME/lib/derbynet.jar start –p <PORT_NUM> &

To stop Derby execute $ java -jar $DERBY_HOME/lib/derbynet.jar shutdown –p <PORT_NUM>

Ensure you enter PORT_NUM as it was set when you ran install.sh. The default is 1527. To look up the PORT_NUM you set, in SharePlex Connector for Hadoop home, conf open the configuration file (by default connectorConfiguration.xml) and lookup the port number entry <entry key="derbyPort">1527</entry>

SharePlex Connector for Hadoop Command Reference

conn_setup.sh

conn_setup.sh

Use this script to setup SharePlex Connector for Hadoop and provide the necessary configuration details. Setup is usually a one time activity. This script is similar to ora_setup of SharePlex for Oracle.

Shell Script Usage

[user@host bin]$ ./conn_setup.sh [-c <INPUT_FILE>] [-r] [-n] [--help] [--version]

Options

Parameter

Description

-c <INPUT_FILE>

Provide an input file that defines the configuration parameters with their values.

Each configuration parameter should be specified on a new line of the input file.

Format - Key=Value. Example – enableRealTimeCopy=True

Tip: You may like to use this parameter to modify your original configuration. When conn_setup.sh is run without parameters the file it creates is conf/connectorConfiguration.xml

There is an example input file located at conf/conn_setup_template.properties

-r

Use to update the configuration

  • Add/Update parameters for HBase Real Time Replication: the HBase column family name.
  • Tables will now replicate to HBase Real Time Replication unless otherwise specified in conn_snapshot.sh

-n

Use to update the configuration

  • Add/Update parameters for HDFS Near Real Time Replication: add the HDFS destination directory and how often do you want to copy data to HDFS? This is measured by time and number of changes.
  • Tables will now replicate to HDFS Near Real Time Replication unless otherwise specified in conn_snapshot.sh

--help

Show this help and exit.

--version

Show version information and exit.

Configuration Parameters

If conn_setup.sh is run without an input file then you will be prompted to supply a value to each of the configuration parameters. See "Run conn_setup.sh" (page 1) for more information.

NOTE: Default values are provided within brackets. Press Enter to select the default value.

Documents connexes

The document was helpful.

Sélectionner une évaluation

I easily found the information I needed.

Sélectionner une évaluation