While maintaining an open ssh session to a Linux NVSD instance, the open terminal window shows messages similar to the following:
***** 18831 1327505579.419961 NET TCP connection reset or aborted on socket '10' : 104:
***** 18831 1327505579.420244 DEDUPE Dedupe Failed While in Dedupe Read to FD
***** 18831 1327505579.420272 LIBERROR ErrorStack start
***** 18831 1327505579.420296 LIBERROR dom="libdedupe", func="libdedupe_sendfile_cookie", line=2943, err=1, desc="error from below"
***** 18831 1327505579.420310 LIBERROR dom="libchunk", func="bmap_internal", line=1110, err=1, desc="error from below"
***** 18831 1327505579.420320 LIBERROR dom="libdedupe", func="do_write", line=1914, err=9, desc="error writing to socketed stream"
***** 18831 1327505579.420333 LIBERROR ErrorStack stop
***** 18831 1327505579.420359 ELEMBROK Failed to Read from Element Broker to TCP : Failed to write Deduplication cookie '7f000001e4301a4fb5dcbb714f250000b9fd0100' to socket : 'unknown error'
***** 18831 1327505579.420386 URICORE Failed to move 33554432 bytes between brokers for '/NVBU/server/backup/segments/0100007f2a1a4f1b72a9226008000003' : 'Failed'
***** 18831 1327505579.420399 URI Failed to read 33554432 bytes from URI Handler of '/NVBU/server/backup/segments/0100007f2a1a4f1b72a9226008000003'
***** 18831 1327505579.420409 DAVLAY Read task with the URI library failed
'smartdisk.sh logs' shows a HTTP GET request for the mentioned segment URI, before these messages appear, suggesting this is a restore request from NVBU.
When NVBU reads from SmartDisk it opens a TCP data connection to SmartDisk.
NVBU reads from one end and SmartDisk writes to the other.
If the reader (e.g. NVBU restore) goes away prematurely, SmartDisk fails to write to the connection.
It's clear from the messages we see that this should not be a matter for concern, as the data is already safely stored in NVSD but NVBU was interrupted while reading from it.
- Dump the content of the NVSD logs into a text file
'smartdisk.sh logs --file /usr/smartdisk/foundation/var/logd.log > /tmp/logd.out
- Using a text editor open logd.out and search for the URI mentioned in the message: e.g. here: '0100007f2a1a4f1b72a9226008000003'
- It should show the date/time the segment had been deduplicated
- make note of the date/time HTTP GET requests were made for it.
- Cross-chek the time with NVBU logs to confirm there had been a duplication, datacopy or restore job initiated around the same time.
- Check that job had indeed been aborted or unexpectedly forced to terminate