SharePlex 12.0 - Installation and Setup guide

Set up an Oracle database to support SharePlex

Set up an Oracle database to support SharePlex

Certain Oracle database settings have an affect on replication and must be set appropriately.

Adjust OPEN_CURSORS to support Post cursors

SharePlex requires the value of the Oracle OPEN_CURSORS parameter to be set correctly on the target database. To view the OPEN_CURSORS value, query the database using the following SQL statement:

select value from V$PARAMETER where name = 'open_cursors';

The Post process reserves 10 cursors for routine calls that are closed once they finish, plus a minimum of 50 cursors per transaction when the SQL Cache feature is enabled (this is the default). For more information, see Tune SQL Caching

If you intend to disable SQL caching, then estimate the peak number of concurrent updating transactions (sessions) that your applications generate, and follow this formula:

10 + (peak number of concurrent transactions x 2) = minimum open cursors needed

The OPEN_CURSORS value can be modified or added if absent. Consult the Oracle documentation prior to making changes to Oracle parameters.

Adjust the PROCESSES parameter to support connections

For the PROCESSES and SESSIONS parameters, 65 is the minimum value required by the SharePlex Post process so that it can open enough SQL connections to the target database to handle current transaction volume. This value is determined by the default setting of the SP_OPO_THREADS_MAX parameter, plus one for the main Post thread.

The PROCESSES parameter in the init.ora file must be set to accommodate connections created by SharePlex and the database users. Its value depends on whether a database is a source database, a target database, or one that serves as both a source and target database.

Database is a source only

If a database serves only as a source, the following formula takes into account logins made by the Read process.

(peak number of source database sessions) + (background Oracle processes) + (value of SP_ORD_LDA_ARRAY_SIZE parameter +3) = setting for PROCESSES

Database is a target only

The Post process creates as many connections on the target system as there are sessions on the source system in order to maintain transaction consistency.

The PROCESSES parameter on the target system must be set high enough to accommodate all of those connections, plus:

• The background Oracle processes those connections generate
• The peak number of users expected to access the target database for queries

Use the following formula as a guide:

(peak number of source database sessions) + (peak number of target database sessions) + (background Oracle processes) = setting for PROCESSES

Database is a source and target

If a database serves as both a source and a target, the following formula takes into account connections made by:

• The Read process
• The Post process
• Background Oracle processes
• User connections

(peak number of source database sessions) + (peak number of target database sessions) + (background Oracle processes) + (value of SP_ORD_LDA_ARRAY_SIZE parameter +3) = setting for PROCESSES

Adjust the log buffer size to improve posting

The number of database writers has an impact on replication, especially when there are numerous concurrent transactions. Whenever a transaction is committed, its buffered data is flushed to disk. If most transactions are small, but the buffer is large, this can cause slow posting. When a large transaction is committed while another, more normal-sized transaction is committed, the second COMMIT must wait while the entire buffer is flushed to disk.

Reducing the size of the buffer that is flushed to disk can speed the Post process. Try decreasing the size of the log buffer to 1024 KB, or even 512 KB if possible.

Adjust the SharePlex transaction table based on user volume

SharePlex updates the SHAREPLEX_TRANS table to maintain read consistency for the target database. You might need to adjust the initrans setting of this table to improve replication performance and reduce contention on that table:

• If the production database has between 500 and 1,000 concurrent users, rebuild the SHAREPLEX_TRANS table to have an initrans of 30.
• If the production database has more than 1,000 concurrent users, rebuild the SHAREPLEX_TRANS table to have an initrans value of 40.

Control conversion of character sets

This topic explains how SharePlex handles character set conversion between an Oracle source and target, and between an Oracle source and a non-Oracle target.

Replication between an Oracle source and Oracle targets

For SharePlex to replicate all characters within the Oracle character sets that you are using, one of the following must be true:

• The character sets are identical on the source and target
• The character set of the source database is a subset of the character set of the target database (all characters contained on the source exist in the character set of the target)

The following character sets are tested and supported for SharePlex:

US7ASCII

UTF8

WE8ISO8859P1

AL16UTF16

AL32UTF8

KO16KSC5601

By default, SharePlex allows an Oracle target database to perform character conversion. Post notifies Oracle of the character encoding of the source data and Oracle performs any required conversion.

Depending on the character sets involved, the Oracle conversion might lead to data loss. For example:

Example 1: The Japanese character for 'rice' in the JA16SJIS character set has no corresponding symbol in the US7ASCII character set. If you attempt to replicate this symbol into a US7ASCII database, Oracle converts it to a '?' character.

Example 2: According to Oracle, the WE8ISO8859P1 character set is a superset of the US7ASCII character set, so it is logical to assume that any character in US7ASCII is posted unconverted into a WE8ISO8859P1 target database. This is true for characters in the range 0x00 to 0x7F. However, Oracle strips off the top bit of characters in the range 0x80 to 0xFF. This "conversion" may result in data loss while replicating to a character set that is a superset of the source.

Note: Oracle does not convert characters if the character sets are identical. Thus, posting WE8ISO8859P1 data to a database with a character set of WE8ISO8859P1 bypasses the Oracle conversion process.

To apply data without conversion:

Set the SP_OPO_NLS_CONVERSION parameter to 1 to apply the data with conversion.

Note: SharePlex will always convert NVARCHAR and NCLOB data if the NLS_NCHAR_CHARACTERSET of the source database is not the same as that of the target database.

Replication between an Oracle source and non-Oracle targets

When replicating to an Open Target target (non-Oracle target), SharePlex supports replication from any Oracle Unicode character set and the US7ASCII character set. SharePlex posts data to Open Target in the Unicode character set, and therefore if the source data is Unicode or US7ASCII, no conversion on the target is required.

However, if the following are true, conversion is required on the target:

• If the character set of the source data is anything other than Oracle Unicode or US7ASCII, you must install an Oracle client on the target to perform the conversion to Unicode for posting to the target.
• If the data must be posted to the target database in any character set other than Unicode, you must install an Oracle client on the target to perform the conversion and use the target command to identify the target character set for Post to use.
• If you are replicating LOB data, conversion is required regardless of what the source character set is.

To perform conversion with an Oracle client on Linux:

1. Install an Oracle Administrator client on the target system. The client must be the Administrator installation type. The Instant Client and Runtime installation types are not supported.
2. Set ORACLE_HOME to the client installation. Set ORACLE_SID to an alias or a non-existing SID. SharePlex does not use them and a database does not have to be running.
3. Install SharePlex using the Linux/Unix installer for your operating system.
4. Make certain the SP_OPX_NLS_CONVERSION parameter is set to the default of 1.

To perform conversion with an Oracle client on Windows:

1. Install an Oracle Administrator client on the target system. The client must be the Administrator installation type. The Instant Client and Runtime installation types are not supported.

2. In the SharePlex Registry key \HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\QuestSoftware\SharePlex\port_number, set ORACLE_HOME to the Oracle client installation location, and set ORACLE_SID to an alias or a non-existing SID. No Oracle database is required. SharePlex only needs to use the client libraries.

3. Install SharePlex using the Windows installer.

4. Make certain the SP_OPX_NLS_CONVERSION parameter is set to the default of 1.

To apply Unicode and US7ASCII data without conversion:

If the source data is Unicode or US7ASCII and you are not replicating LOB data, no conversion or Oracle client is needed. Set the SP_OPX_NLS_CONVERSION parameter to 0 to disable conversion, and then restart Post if it is running.

Set up SharePlex to support Oracle data

Set up SharePlex to support Oracle data

This topic contains setup guidelines that apply to specific Oracle data types. These guidelines should be addressed before you start replication for the first time.

LOBs, LONGs, VARRAYs, and XML

• Tables that contain a LOB or LONG should have a primary key or unique key defined on them. If a table does not have a key, SharePlex builds its own key from all of the columns except LONGs or LOBs. If a LOB or LONG is the only difference between two rows that otherwise satisfy the Post WHERE clause, then Post may update the wrong row.
• Dedicate one or more named export queues to tables that contain LOBs. This automatically creates separate Export processes and named post queues with their own Post processes. By separating the processing of LOB data types from that of other data, you can improve the overall speed of replication. For more information, see the Configure Named Export Queues section in the SharePlex Administration Guide.
• To ensure that SharePlex has enough shared memory when replicating LOBs, increase the SP_QUE_POST_SHMSIZE parameter to an initial setting of 60 MB. If SharePlex generates shared memory segment errors such as "Error: sp_cop process sp_mport/sp_opst_mt killed due to SIGSEGV," increase the setting.
  

Note: A larger shared memory segment can result in a large amount of swap space being used on the system, so make sure enough disk space is available.

Manage SharePlex LOB storage

The Database Setup utility installs some tables into a tablespace of your choosing. All but the SHAREPLEX_LOBMAP table use the default storage settings of the tablespace.

The SHAREPLEX_LOBMAP table contains entries for LOBs stored out-of-row. It is created with a 1 MB INITIAL extent, 1 MB NEXT extent, and PCTINCREASE of 10. The MAXEXTENTS is 120, allowing the table to grow to 120 MB.

Preferred action: If you enable supplemental logging for primary and unique keys, you can set the SP_OCT_ENABLE_LOBMAP parameter to 0, and nothing will be stored in the SHAREPLEX_LOBMAP table. In this case, you do not have to consider its size growth. It is recommended that you enable supplemental logging for primary and unique keys to maximize the performance of the Read process.

Alternate action: The default storage usually is sufficient for SHAREPLEX_LOBMAP, permitting more than 4 million LOB entries. If the Oracle tables to be replicated have numerous LOB columns that are inserted or updated frequently, consider increasing the size the SharePlex tablespace accordingly. Take into account that this table shares the tablespace with other SharePlex tables.

If the database uses the cost-based optimizer (CBO) and the tables that SharePlex processes include numerous LOBs, incorporate the SHAREPLEX_LOBMAP table into the analysis schedule.

Note: A new installation of SharePlex does not change storage parameters from a previous installation.

Set system process priority

If Oracle or other processes are assigned resource priority, SharePlex can be left with a default setting and little resource allocation. Oracle increases its CPU utilization during peak processing. If SharePlex loses pace with Oracle, you can try increasing its process priority.

To set process priority on Unix:

Use the nice command. Consult with the System Administrator to select an appropriate value based on the requirements of all software running on the system. A root user can modify the niceness value of any process. The SharePlex Administrator user can adjust the niceness value of SharePlex.

To set process priority on Windows

1. Run SpUtils.
2. Select the TaskMgr tab.
3. Right click the SharePlex instance that you want to prioritize.
4. Select Set Priority, then select the desired priority level.

Enable Oracle direct path loads

By default SharePlex replicates changes made to tables through a SQL*Loader direct-path load (DIRECT=TRUE keyword parameter). There can be only one load per table (PARALLEL=FALSE), although there can be simultaneous loads on different tables. The database must be in archive mode, and table logging must be enabled.

If you expect the direct-path load to be sustained for a long time on the source system, it might be more efficient to load the data to the target database directly, instead of relying on replication. A large direct-path load can cause Capture to lose pace with changes that enter the redo logs from user application activity.

After the load, you should disable check constraints. You can leave ON DELETE CASCADE constraints enabled.

The SP_OCT_REPLICATE_DLOAD parameter controls whether or not direct-path loads are replicated. To disable replication of direct-path loads, change this parameter to 0. For more information, see the SharePlex Reference Guide.

Use compression

You can enable compression to reduce the amount of data that SharePlex sends across the network. SharePlex uses LZIP lossless compression. Enabling compression on the source SharePlex instance automatically enables compression to all targets of the source SharePlex instance.

By default compression is disabled. You can enable compression by itself or in conjunction with encryption. For more information about encryption, see the SharePlex Administration Guide.

To enable compression

Set the SP_XPT_ENABLE_COMPRESSION parameter to 1.

sp_ctrl> set param SP_XPT_ENABLE_COMPRESSION 1

To activate the parameter after you set it, stop and start Export.

Configure support of Data Pump exports

When replicating Oracle Data Pump export operations, set the SP_OCT_ALLOW_DP_DDL parameter to 1, and then restart Capture.

This parameter can be enabled if SharePlex fails to replicate DDL operations that occur when running an Oracle Data Pump export/import. Occasionally, SharePlex identifies DDL in a Data Pump load as recursive DDL that should be ignored. This parameter directs SharePlex to capture that DDL.

A setting of 1 enables this parameter. After the load is finished, set this parameter back to 0 and then restart Capture.

Set up TDE Support

SharePlex uses the TDE primary Encryption Key to decrypt TDE-protected data that must be replicated. SharePlex uses the Oracle wallet password to access the TDE primary Encryption Key. If the wallet password is stored on external hardware security modules (HSM), then use the sp_hsm utility.

If the wallet opens successfully, Capture connects to the decryption module and processes the data. If the wallet does not open, Capture remains in the initialization state until either the wallet is opened or the process is stopped. The initialization state that is displayed in the show capture command is "Capture state: Waiting for open wallet."

Note:  The SharePlex copy/append command does not support TDE. For full information on the Oracle features that SharePlex supports, see SharePlex Release Notes.

Required privilege to capture TDE-protected data

To decrypt TDE-protected data from the redo log, the SharePlex Administrator must open Oracle Wallet using the wallet password. By default, only the Oracle Wallet owner-user has read and write permissions for this file. You can either start as the owner of the wallet, or you can grant read permission to the file to the dba group, because the SharePlex Administrator user is a member of that group.

Configure SharePlex to capture TDE-protected data

To configure SharePlex to support TDE-protected data, two setup tools must be run:

• (If this was not done during installation) Run Database Setup.When prompted to enable TDE replication, type "y" and then enter the fully qualified path to the TDE wallet file, including the wallet file name, when prompted. For more Information, see For more information, see Database Setup Utilities.
• Run the sp_wallet utility to provide the Oracle Wallet password to SharePlex. This utility can be run in manual or auto-open mode.

To run sp_wallet and manually supply the password:

1. On the source system, start SharePlex from the SharePlex product directory. You are prompted to run sp_wallet.

   *** To enable TDE replication, run sp_wallet and provide the wallet password ***

2. Run sp_wallet.

   ./sp_wallet [-r port_number]

   IMPORTANT! On Windows, if you installed SharePlex on any port other than the default of 2100, use the -r option to specify the port number. For example, in the following command the port number is 9400:

   ./sp_wallet -r 9400

   wallet password: walletpw

   Wallet loaded into SharePlex

To run sp_wallet in auto-open mode:

If you are using an auto-open wallet, you can configure SharePlex to open the TDE wallet automatically. This eliminates the need to run sp_wallet manually at SharePlex startup. The syntax is:

./sp_wallet --auto-open [-r port_number]

Important! Using the auto-open wallet feature has additional security considerations. See the Oracle documentation for more information. In addition, do not back up the SharePlex variable-data directory together with the Oracle wallet and the Oracle data files.

To cancel auto-open mode:

./sp_wallet --no-auto-open [-r port_number]

To change the TDE primary encryption key:

If you need to change the TDE primary Encryption Key while a SharePlex configuration is active, take the following steps to ensure that SharePlex continues to replicate the TDE-protected data after the changes.

1. Quiesce the source database.
2. Make sure that Capture finishes processing the remaining data in the redo log.
3. Shut down SharePlex.
4. Change the TDE primary Encryption Key.
5. Restart SharePlex.

6. Run the sp_wallet utility to provide SharePlex with the new TDE primary Encryption Key.

   ./sp_wallet [-r port_number]

Configure SharePlex to capture the TDE wallet key stored on external storage

To configure SharePlex to support TDE wallet key stored on HSM (external storage) , two setup tools must be run:

• (If this was not done during installation) Run the Database Setup. When prompted to enable TDE replication, type y. If asked whether the TDE keys are stored in the Oracle wallet, type n. Next, you will be prompted to enter the fully qualified path name of the HSM PKCS 11 library, including the library name. For more Information, see For more information, see Database Setup Utilities.
• Run the sp_hsm utility to provide the Oracle wallet key stored on external storage (HSM). This utility can be run in manual mode.

To run sp_wallet and manually supply the password:

1. On the source system, start SharePlex from the SharePlex product directory. You are prompted to run sp_hsm.

   *** To enable TDE replication for ORACLE SID, run sp_hsm to setup the password for the HSM ***

2. Run sp_hsm.

Set up replication from Oracle to a different target type

Set up Replication from Oracle to a Supported Target Type

Contents

About these instructions

This chapter contains instructions for configuring SharePlex to replicate from Oracle to a different type of target. This is known as heterogeneous replication.

These instructions highlight specific tasks that are pertinent to the flow of data between source and target. Refer to other topics in the SharePlex documentation as needed to complete the configuration, deploy any optional features that apply, and monitor and maintain the environment.

For additional information, see:

• For the SharePlex-supported datastores, data types and operations that are supported by SharePlex, see the "System Requirements " section of SharePlex Release Notes.
• For additional configuration options, activation steps, and monitoring information, see SharePlex Administration Guide.
• For reference documentation on SharePlex commands, parameters and utilities, see SharePlex Reference Guide.