Smartdisk not deduplicating because unable to lock a lock file on Solaris (4046587)

Date: 26/01/2011
Product & Version: Smartdisk 1.2
OS Version: Solaris 10
Module & Version:
Application version:

Symptoms:
Deduplication is unavailable.
Looking at the output from "smartdisk.sh stats", Smartsdisk is not performing deduplication, even though there are no other jobs running at the same time.


[SmartDisk:Deduplication Scheduler]
Elements.Queued = 2
Data.Queued = 417497088, B
Elements.Deduplicating = 0
Data.Deduplicating = 0, B
GC Active = FALSE
Deduplication Available = FALSE
Deduplication Enabled = TRUE

Dedupe trace shows smartdisk fails to get a lock on a file.
The following error is being generated....
"Error locking lock file! (Is another chunkstored running?)"

The chunkstored process shows it is trying to issue a lock....

open("/NVSD/hans020-Pool1/store/dedupe/lock", O_WRONLY|O_CREAT, 0644) = 11
fcntl(11, F_SETLK, 0xFFFFFD7FFFDFF6C0) Err#11 EAGAIN
write(12, "\0\09CFC9A\0 F I L E S Y".., 156) = 156
getpid() = 12956 [12952]
write(2, " * * * * * ", 6) = 6
write(2, " 1 2 9 5 6", 5) = 5
write(2, " ", 1) = 1
write(2, " 1 2 8 8 6 2 0 0 9 8", 10) = 10
write(2, " .", 1) = 1
write(2, " 9 6 4 9 3 4", 6) = 6
write(2, " ", 1) = 1
write(2, " F I L E S Y S", 7) = 7
write(2, " ", 1) = 1
write(2, " F a i l e d t o o p".., 112) = 112
write(2, "\n", 1) = 1
close(11) = 0

Reason:
/NVSD is an NFS filesystem mounted from a Sun/Oracle S7210.
This appears to be a stale NFS file handle.

Solution:
- To clear the problem stop the SmartDisk processes,
- remove and recreate the lock file with the same owner:group and permissions
- start smartdisk
'smartdisk.sh stats' will now report that deduplicaiton is available and you will be able to see one of the servers actively deduping staged data.