Return
-
Title
Compare runs into errors "Warning 32: Broken pipe" -
Description
When running compare, the client process on target (sp_declt) gives out the following errors as evident from *declt.log file:
000: select rowid, "PROVIDERID", "REFCNT", "PROVIDERTYPE" from <owner>.<tablename> order by 2
000: Notice: starting the second phase in the repair process
000: No data in queue, waiting... (time=1168640657)
000: No data in queue, waiting... (time=1168640658)
000: Exiting de_msg_clt_preCompareMessaging2,L395
000: SP_DEQ_PART_UPDATE : 0
000: Notice: Object ID of <owner>.<tablename> is 9843
000: Warning: Call to read 8 bytes (de_sock_read,L861), unexpectedly returned 0 bytes.
000: Process writing may have exited with an error.
000: The warnings may due to either "compare cancel"command or network errors.
000: Warning 1 calling de_sock_read (de_msg_receive_header,L190)
000: Warning 1 calling de_msg_receive_header (de_msg_receive_batch_info,L2138)
000: Warning 1 calling de_msg_receive_batch_info (de_compare_ask_batch_info,L447)
000: Warning 1 calling de_compare_ask_batch_info (de_compare_client,L1054)
000: Warning 1 calling de_compare_client (main,L2444)
Exiting
000: Warning 32 failed to write (de_sock_write,L768) (socket desc=15)
000: Warning 32: Broken pipe
000: Warning 32 calling de_sock_write (de_msg_send_question,L3140)
000: Warning 1 calling de_msg_clt_ask_server (de_msg_clt_postCompareMessaging,L484)
000: DEBUG:schema=<owner>, table name=<tablename>, column_count=4 (de_table_list_table_info,L3547). -
Cause
The problem can be due to resource or connectivity issues. -
Resolution
The following steps can be tried as a workaround:
1. Reduce the SP_DEQ_BATCHSIZE on source and target prior to running compare. The SP_DEQ_BATCHSIZE parameter controls the batch size that SharePlex uses for the compare/repair function when the compare config or compare table command is issued. Compare does not do comparison process for all rows at once. Rather it takes few rows at a time in a batch, and does the compare on such rows (which also involves sorting, etc). The parameter SP_DEQ_BATCHSIZE determines the size of the batch. If you use a smaller batch size, then it involves less resource and memory usage and perhaps in the process has less chance of "broken pipe".
Try reducing it from the default 10000 to 1000 as mentioned below and run the compare again.
On the source and target set the parameter via sp_ctrl:
sp_ctrl>list param all compare (check the setting for this parameter)
sp_ctrl> set param SP_DEQ_BATCHSIZE 1000
sp_ctrl> compare table......
To revert the parameter back to the original value:
sp_ctrl> reset param sp_deq_batchsize
2. The broken pipe could also be due to network configuration errors. Check to see if the entry for hostname (and/or alias, as applicable) for both source and target exists in source and target /etc/hosts file. Check to see if the IP or virtual IP (if applicable) are resolvable from source to target and vice versa.
3. If the firewall is configured to timeout in case of inactivity after a certain time, you may see this problem if running compare on large tables. You may have to change firewall to keep the connection alive. -
Additional Information
The parameter SP_DEQ_BATCHSIZE has been deprecated in later versions of SharePlex and is replaced by the parameter SP_DEQ_MALLOC. Here is the description of the parameter SP_DEQ_MALLOC: SP_DEQ_MALLOC This parameter controls the fetch batch size. The batch size controls the number of rows that SharePlex selects at once for comparison. Larger batch sizes increase processing speed but require more memory. The value is divided equally by the number of compare threads to be used, and then the batch size is recalculated based on all column sizes added together. Default: 500 (MB) Range of valid values: 50 to 32,767 (MB) Takes effect: Immediately available for the next comparison. Please refer to knowledge base article # 124140 for differences in these two approaches for controlling the batch size of compare/repair.