If you intend to replicate tables in Hive over HDFS for Avro files, then complete the following additional setup steps.
Use --auxpath while entering into Hive
hive --auxpath <SHAREPLEX_HADOOP_CONNECTOR_HOME>/lib/avro-file-serde.jar
IMPORTANT NOTE: When using connector on IBM BI Hadoop and CDH 5 distributions, jar files specified in this scenario must be appended to HIVE_AUX_JARS_PATH in HIVE_CONF_DIR/hive-env.sh file.
If you intend to replicate tables in Hive over HDFS for Avro files with snappy compression codedc then complete the following additional setup steps.
Use --auxpath while entering into Hive
hive --auxpath <SHAREPLEX_HADOOP_CONNECTOR_HOME>/sqoop-<version>.bin__hadoop-<version>/lib/snappy-java-<version>.jar,<SHAREPLEX_HADOOP_CONNECTOR_HOME>/lib/avro-file-serde.jar
IMPORTANT NOTE: When using connector on IBM BI Hadoop and CDH 5 distributions, jar files specified in this scenario must be appended to HIVE_AUX_JARS_PATH in HIVE_CONF_DIR/hive-env.sh file.
SharePlex Connector for Hadoop is distributed in the archive: shareplex-hadoop-connector-version-hadoopDistributionVersion.tar.gz where version identifies the SharePlex Connector for Hadoop release.
Extract the archive with hadoopDistributionVersion appropriate to your Hadoop installation on a machine where Hadoop libraries and configurations are present. Use the command:
$tar -xzf shareplex-hadoop-connector-version-hadoopDistributionVersion.tar.gz
The archive contains the following files:
| File | Description |
|---|---|
| install.sh |
A shell script that installs/upgrades SharePlex™ Connector for Hadoop® and other programs in the archive. Installing a SharePlex Connector for Hadoop upgrade or reinstalling the software without un-installing it preserves existing data files. Before you upgrade or re-install SharePlex Connector for Hadoop, see the Release Notes for the version you are installing to familiarize yourself with any special upgrade or installation requirements. |
| shareplex-hadoop-connector-version-hadoopDistributionVersion-ga.tar | SharePlex™ Connector for Hadoop® archive. |
| db-derby-10.9.1.0-bin.tar.gz |
Apache Derby installable. SharePlex™ Connector for Hadoop® uses the Apache Derby network server and creates a database for storing metadata and status information. |
| sqoop- version.bin__hadoop-version.tar.gz |
Apache Sqoop installable. Sqoop is a tool designed to transfer bulk data between Apache Hadoop and structured data stores. |
| quest-oraoop-1.6.0-date-version.tar.gz |
Data Connector for Oracle and Hadoop archive. Data Connector for Oracle and Hadoop is an optional plugin to Sqoop. It facilitates the movement of data between Oracle and Hadoop. |
