Import errors out with "failed to add a subqueue header" (4301082)

Import errors out with "failed to add a subqueue header"

Import process keeps erroring out with the following messages and consequently the Export process on source remains idle:

"Process launched: sp_mport.exe (from g2mcdb12.c2itv queue G2MCDB12-2104-G2MCDW11) [pid = 2528]
Process launched: Sp_Mport.exe Process started PID = 4944
Connected to export on g2mcdb12.c2itv
Notice: Queue write recovery started, qrw_srcseq=430692859600 msg.mpseq=430692407600 [sp_mport.exe(que)/4944]
Notice: Queue write recovery complete, 433 duplicate messages skipped , 245608 bytes total [sp_mport.exe(que)/4944]
Notice: bu_realloc below lo-water [sp_mport.exe(que)/4944]
Error: failed to add a subqueue header: que_NOFREE_SQUE: No more free subques [sp_mport.exe(que)/4944]
Notice: Error que_SQUE_CRE: Could not create subqueue que_write(G2MCDB12-2104-G2MCDW11+P+o.GDS2DB-o.GDS2DW) [sp_mport.exe(rim)/4944]
Error: sp_mport: failure writing to queues - exiting [sp_mport.exe/4944]
Process exited sp_mport (from g2mcdb12.c2itv queue G2MCDB12-2104-G2MCDW11) [pid = 4944] - exit(1)"

The error occurs when a certain low water mark is hit for number of buffer availble (not currently used) in shared memory for the queues. Buffers are used to update the queue and subque header files and also for queue data.

Subqueue creation error is due to the system running out of the number of subqueues it can create based on the value of the SP_QUE_SHMSIZE and SP_QUE_SHMDBUF. Shareplex splits shared memory into 32K (this is the default but can be changed with the parameter SP_QUE_SHMDBUF) size buffers to access the queue files (queue header, sub-que header, and queue data files). Shareplex will only take up 60% of the buffers for sub-que header information and save the rest for actual data. When this upper limit is reached it will result in a can't create sub-que error. The issue can be worked around by setting the buffer size to 8k (header info only takes < 8k) this will create 4 times as many buffers and allow you to create up to 4 times as many sub-ques. Another workaround is to double the value of the parameter SP_QUE_SHMSIZE from 32M to 64M.


The following illustrates this:

On target (or where the import has problems) please do the following:
sp_ctrl>set param SP_QUE_USE_OVF 0
sp_ctrl>set param SP_QUE_POST_SHMSIZE 67108864
sp_ctrl>set param SP_QUE_SHMDBUF 8192
Then shutdown and restart shareplex for the parameters to take effect.

In this example I have assumed that the current value of SP_QUE_SHMDBUF and SP_QUE_POST_SHMSIZE were 32M and 32K respectively. Decreasing the former and increasing the latter results in more number of buffers available for Shareplex. If the current or default values for these two parameter had values other than 32M and 32K, one can decrease and increase the above parameters using trial and error. One other thing to look for is that the system should have enough resources to accomodate the increase in the value of SP_QUE_POST_SHMSIZE. This issue has been prominent in Shareplex 5.1.

Starting with SharePlex 8.6, the unit of the following parameters have changed from bytes to megabytes (MB). Hence one may want to change all references to the parameters from bytes to megabytes. Here are the latest description of the parameters:

SP_QUE_POST_SHMSIZE

This parameter determines the specific shared-memory size for the post queue.

IMPORTANT: Increasing this parameter might require an increase in the maximum shared memory segment size

in your system settings.

Default: 32 megabytes

Range of valid values: integer greater than 0

Takes effect: When sp_cop is restarted

SP_QUE_Q_SHMSIZE

This parameter determines the specific shared-memory size for the capture and export queues.

Default: 8 megabytes

Range of valid values: integer greater than 0, no maximum

Takes effect: When sp_cop is restarted

Moreover, the parameter SP_QUE_POST_SHMSIZE can now be set globally as well as at a named Post queue level. This is how it is set at a queue level:

sp_ctrl>set param SP_QUE_POST_SHMSIZE queue <post_queue_name> <value>

Then bounce the Post queue in question.