Out of Sync or missing transaction on target due to memory defect (4274170)

There are many symptoms of Out of Sync (OOS) or missing data on target due to memory defect:

1. On source capture core dumps with UNKNOWN VLD.

2. Source event_log shows "capture recovery" without additional information.

3. Target event_log shows "out of sync" (OOS) messages.

4. On non-RAC, source event_log or ocap.log show “old scn > current scn”.

5. On RAC, source event_log or ocap.log show“old scn > current scn” on the same RAC node.

6. OOS may occur after capture has been running for a significant amount of time without restarting, this depends on the frequency of heavy transactional volume.

7. Target event_log shows unique constraint violation.

8. Target compare log indicates that inserts are missing.

Product Defect.

Memory corruption may occur when capture internal queue parameter setting is not a value of power of 2 (pow(2, n)).

Capture log reader reads messages from redo log and writes to the internal memory queue area called “MacQ”. There is a counter pointer for the reader and writer of this internal MacQ, its function is to tell writer to write to certain addresses and reader to read from certain addresses in this memory area. This MacQ is a circular queue and the issue only occurs when one of the counter reaches its max value of 4 billion. If the queue size parameter is not set to a power of 2 (2^n) then the problem may occur. With more activities taking place on busy tables, the chances of missing transactions on that table is higher than other tables.

If capture is restarted frequently then you may not hit this problem as this internal queue memory area is reset.

WORK AROUND:

For SharePlex versions below 7.0.2.35

Set these parameters on source side as described below. It is recommended to so this during a relatively quiet time to avoid complication, and capture must be restarted for these setting to take effect.

OLOG queue size – power of 2 number minus 1 on non-RAC,

power of 2 minus 1 times the # of RAC nodes for RAC

OPS queue size – power of 2 number minus 1, only needed for RAC customer

For example, to set internal queue size to be 8192 then

1) Non-RAC:

sp_ctrl> stop capture

sp_ctrl> Set param SP_OCT_OLOG_QUEUE_SIZE 8191 (capture will use 8191+1=8192)

sp_ctrl> set param SP_OCT_OLOG_LOGPAR_QSIZE 4095 (only change this if using versions below 7.0.2)

sp_ctrl> start capture

2) RAC :

olog queue size need to be divided by the number of nodes.

a) 2 node RAC :

sp_ctrl> stop capture

sp_ctrl> Set param SP_OCT_OLOG_QUEUE_SIZE 16382 (capture will use 16382/2=8191+1=8192)

sp_ctrl> Set param SP_OCT_OPS_QUEUE_SIZE 8191 (capture will use 8191+1=8192)

sp_ctrl> set param SP_OCT_OLOG_LOGPAR_QSIZE 4095 (only change this is using versions below 7.0.2)

sp_ctrl> start capture

b) 3 node RAC :

sp_ctrl> stop capture

sp_ctrl> Set param SP_OCT_OLOG_QUEUE_SIZE 24573 ( capture will use 24573/3=8191 +1=8192)

sp_ctrl> Set param SP_OCT_OPS_QUEUE_SIZE 8191 (capture will use 8191+1=8192)

sp_ctrl> set param SP_OCT_OLOG_LOGPAR_QSIZE 4095 (only change this using versions below 7.0.2)

sp_ctrl> start capture

For 7.5 and above, there is a third internal hard coded parameter that cannot be adjusted by users thus there is no work around and the users must patch to the latest SharePlex version.

STATUS:

Issue is resolved in SharePlex for Oracle versions 7.0.2.35 , 7.5.2.50 and 7.6.1.28. The latest  version of SharePlex can be downloaded at: https://support.quest.com/es-es/shareplex/download-new-releases