SharePlex Connector for Hadoop 8.5.6 - Installation Guide

Replicate tables in Hive over HDFS for Sequence files

If you intend to replicate tables in Hive over HDFS for Sequence files then complete the following additional setup steps.

Use --auxpath while entering into Hive

hive --auxpath <SHAREPLEX_HADOOP_CONNECTOR_HOME>/lib/sequence-file-serde.jar, <SHAREPLEX_HADOOP_CONNECTOR_HOME>/lib/sqoop-records.jar,<SHAREPLEX_HADOOP_CONNECTOR_HOME>/sqoop-<version>.bin__hadoop-<version>/sqoop-<version>.jar

Note: Hive shell should be restarted with the above setting each time that you take a snapshot for Hive over HDFS for sequence files.

IMPORTANT NOTE: When using connector on IBM BI Hadoop and CDH 5 distributions, jar files specified in this scenario must be appended to HIVE_AUX_JARS_PATH in HIVE_CONF_DIR/hive-env.sh file.

Replicate tables in Hive over HDFS for Avro files

If you intend to replicate tables in Hive over HDFS for Avro files, then complete the following additional setup steps.

Use --auxpath while entering into Hive

hive --auxpath <SHAREPLEX_HADOOP_CONNECTOR_HOME>/lib/avro-file-serde.jar

IMPORTANT NOTE: When using connector on IBM BI Hadoop and CDH 5 distributions, jar files specified in this scenario must be appended to HIVE_AUX_JARS_PATH in HIVE_CONF_DIR/hive-env.sh file.

Replicate tables in Hive over HDFS for Avro files with Snappy compression codec

If you intend to replicate tables in Hive over HDFS for Avro files with snappy compression codedc then complete the following additional setup steps.

Use --auxpath while entering into Hive

hive --auxpath <SHAREPLEX_HADOOP_CONNECTOR_HOME>/sqoop-<version>.bin__hadoop-<version>/lib/snappy-java-<version>.jar,<SHAREPLEX_HADOOP_CONNECTOR_HOME>/lib/avro-file-serde.jar

IMPORTANT NOTE: When using connector on IBM BI Hadoop and CDH 5 distributions, jar files specified in this scenario must be appended to HIVE_AUX_JARS_PATH in HIVE_CONF_DIR/hive-env.sh file.

Configure SharePlex Connector for Hadoop