Chat now with support
Chat with Support

SharePlex Connector for Hadoop 8.5.6 - Installation Guide

Welcome to SharePlex Connector for Hadoop

Welcome to SharePlex™ Connector for Hadoop®

SharePlex™ Connector for Hadoop® enables log-based replication of tables from Oracle to Hadoop (HDFS and HBase).

SharePlex Connector for Hadoop also supports capturing change history (formerly change data capture) of a table on HDFS / Hive. Tables can further be replicated to Hive (Hive over HDFS and/or Hive over HBase and Hive over CDC).

Get Started

Get Started with SharePlex Connector for Hadoop

  1. SharePlex Connector for Hadoop operates in conjunction with SharePlex™ for Oracle®. Ensure SharePlex for Oracle is fully installed and operational in your Oracle environment.
  2. SharePlex Connector for Hadoop replicates to Cloudera's distribution of Hadoop (CDH4.2.0 64-bit, CDH 5.0.0, and CDH 5.1.0), Hortonworks distribution (HDP 1.3), Apache Hadoop 1.2.1, and IBM InfoSphere BigInsights (IBM BI 2.1.2). Install a distribution above including libraries and configurations.

    NOTE: When using IBM BI 2.1.2 distribution, you must explicitly comment out SQOOP_HOME and SQOOP_CONF_DIR that are specified in the $BIGINSIGHTS_HOME/conf/biginsights-env.sh file.

    1. Install and configure HBase (optional) to replicate tables to HBase and Hive over HBase.
    2. Install and configure Hive (optional) to replicate tables to Hive (Hive over HDFS or Hive over HBase). Installing and configuring Hive is mandatory to use the change history feature (formerly Change Data Capture).
  3. Download and install Apache ActiveMQ. Tables are replicated through a JMS queue facilitated by Apache ActiveMQ.
    1. Install the Java Development Kit as required by Apache ActiveMQ. For more information, see Install the Java Development Kit.
    2. Install ActiveMQ to operate with SharePlex. Start ActiveMQ, start the SharePlex cop process and set up the JMS queue. For more information, see Install ActiveMQ.
  4. Install and configure SharePlex Connector for Hadoop. For more information, see Unpack the SharePlex Connector for Hadoop Archive.
  5. Follow the use case: For more information, see Use case: Set Up and Start Replication.

Manage Replication Operations

Take the following steps.

SharePlex Connector for Hadoop

Log Files

Check the ALERT log messages. Actions may be required. For more information, see SharePlex Connector for Hadoop Log Files.

Configure the logs to suit your environment.

conn_ctrl.sh Control SharePlex Connector for Hadoop operations. For more information, see conn_ctrl.sh.
conn_monitor.sh Monitor SharePlex Connector for Hadoop operations. For more information, see conn_monitor.sh.
conn_cleanup.sh Remove all data (HDFS, HBase, Hive) replicated from a specified table. For more information, see conn_cleanup.sh.
uninstall.sh Uninstall SharePlex Connector for Hadoop. For more information, see uninstall.sh.

JMS Queue

  
ActiveMQ Console

Monitor activity in the JMS queue via the ActiveMQ Console / Web Interface. For more information, see Install ActiveMQ.

SharePlex for Oracle

From the sp_ctrl ()> prompt you can:

Perform Initial Setup

Self Service Tools
Knowledge Base
Notifications & Alerts
Product Support
Software Downloads
Technical Documentation
User Forums
Video Tutorials
RSS Feed
Contact Us
Licensing Assistance
Technical Support
View All
Related Documents

The document was helpful.

Select Rating

I easily found the information I needed.

Select Rating