The sp_cop fails to come up and shows “segmentation fault” and event log shows Shared Memory errors:
Notice 2015-05-22 05:28:25.272808 124851 1 Import: Unable to initialize shs_alloc_stat (shs_SEMERR: an error occurred with the semaphore) (importing from server_name queue queue_name) [module xpt]
Error 2015-05-22 05:28:25.561890 124851 1 Import: sp_mport: failure in shared memory alloc - exiting (importing from server_name queue queue_name) [module imp]
Or
Notice 2015-05-22 07:23:01.194874 4603 1 Cop: Semaphore exists for key = 0x77bc8 [module que]
Notice 2015-05-22 07:23:01.272688 4603 1 Cop: Semaphore exists for key = 0x177bc8 [module que]
Corrupt statusdb file is causing failure to launch sp_cop.
TROUBLESHOOTING:
Set debug on sp_cop as below by appending the following debug entry in paramdb file (since it is not possible to set debug at the sp_ctrl level):
SP_COP_DEBUG "0xffffffff "
Then restart sp_cop. Once it fails to restart, the following entries will be written in the trace_log located in /vardir/log directory:
Notice 2015-05-22 06:55:36.163963 104496 1 SharePlex for Oracle RAC/OPS : "Perpetual Key"
Info 2015-05-22 06:55:36.215021 104496 1 Debug: tcp_init() [module cop]
Info 2015-05-22 06:55:44.945713 104496 1 Debug: obj_new(+PC+*+sp_cop+*) [module cop]
Info 2015-05-22 06:55:44.968418 104496 1 Debug: dirty_check [module cop]
Info 2015-05-22 06:55:44.987497 104496 1 Debug: scan_actdb() [module cop]
The trace_log ends with the following which indicates corruption in statusdb:
Info 2015-05-22 06:55:44.987497 104496 1 Debug: scan_actdb() [module cop]
RESOLUTION:
If this is a target, remove the statusdb entry (located in /vardir/data directory) and then restart sp_cop. A new statusdb will be created and sp_cop will run fine.
If this is the source, it is advisable to see if there is any backup of the /vardir/data directory and replace the corrupt statusdb with a good entry from backup. If no backup exists, contact SharePlex Support to see if the activation entry can be salvaged as renaming existing statusdb can result in loss of activation.
