Restituisci
-
Titolo
Event_log file encounters "Disk is full" and status shows SharePlex process with "Stopped - disk is full" though vardir is not full. -
Descrizione
You encounter the following output from 'status' in sp_ctrl with corresponding entries in the event_log file as below:sp_ctrl> statusProcess Source Target State PID---------- ------------------------ ------------ -------------------- ------Capture o.PRPVG01 Running 25425Read o.PRPVG01 Running 25428Import x4100b x4100a Stopped - disk is fullPost o.PRPVG02-x4100b o.PRPVG01 Stopping 25427Export x4100a x4100b Running 2560sp_ctrl> show log reverse02/19/20 17:46 Process exited sp_ocap (for o.PRPVG01) [pid = 25425] - exit(0)02/19/20 17:46 Notice: shutting down by request [sp_ocap/25425]02/19/20 17:46 Disk is full /nfs/prvg_vardir/vardir2200/rim02/19/20 17:46 Process exited sp_mport (from x4100b.i.abc.com queue x4100b) [pid = 2559] - exit(0)02/19/20 17:46 Disk is full /nfs/prvg_vardir/vardir2200/rim02/19/20 17:46 Disk is full /nfs/prvg_vardir/vardir2200/rim02/19/20 17:46 Disk is full /nfs/prvg_vardir/vardir2200/rim02/19/20 17:45 Warning: sp_opst didn't shutdown [sp_cop/25178]Check the available disk space in vardir using "df -k" or any equivalent Unix command and find that it is not yet 100% full.And in certain situations, it may be well below 100%. -
Causa
The available disk space may have fallen below the limit specified by the SharePlex parameter SP_SYS_VAR_FULL.
-
Risoluzione
Two parameters in SharePlex act like a safety switch and turn SharePlex processes off or on when there is a looming disk capacity situation.This is to proactively prevent queue corruption that is caused when the disk housing the vardir becomes full.Refer to KB article 22995 for details on events that can lead to queue corruption.The parameters are named SP_SYS_VAR_FULL and SP_SYS_VAR_OK.By default, they acquire values of 30M and 50M respectively.When SharePlex is running, it does periodic checks on the available disk space and compares it with the SP_SYS_VAR_FULL parameter.If the available disk space falls below the value specified for the SP_SYS_VAR_FULL parameter, the child processes of sp_cop will shutdown gracefully to prevent any queue corruption that may result if the available space is near 0 with SharePlex child processes still running.If monitoring scripts are also set up, users can be alerted about this situation (since SharePlex processes stopped) and they can take corrective measures by adding more space to the vardir.If the space added results in free space higher than the value of the SP_SYS_VAR_OK parameter, then it will display the message "Disk is no longer full" in the event_log file.To change the setting for these parameters execute the following steps:sp_ctrl> set param SP_SYS_VAR_FULL <value>sp_ctrl> set param SP_SYS_VAR_OK <value>(Where <value> is expressed in units of MB).In nutshell, this is how the parameters work in unison:If the space approaches (or is equal to) the value defined by the SP_SYS_VAR_FULL parameter, then the respective process will be suspended.When space is added to the vardir so that the free space equals to the value defined by the SP_SYS_VAR_OK parameter, then SharePlex’s child processes resume.The onus is on users to appropriately set the values for these two parameters.The value of SP_SYS_VAR_OK should always be higher than that of the SP_SYS_VAR_FULL.Important Note on Queue files and how to move or copy them to different locations:Queue files are 8MB in size.To move or copy Queue files, the UNIX pax command is preferred instead of tar.The pax command preserves file sparseness and unallocates blocks filled with zeros while tar creates an 8M file regardless of actual data in file.pax -vf (same as tar -tvf )pax -rp e -f (like tar -xvf )Please check the man pages for the valid options of the "tar" and "pax" commands for your operating system.Here are the steps to move the vardir/rim Queue files to a different location and link vardir/rim:1. Locate or create a new location (newfilesystem) with plenty of disk space.2. Tar the original vardir/rim directory from the $SP_SYS_VARDIR/rim directory: tar -cvf /rim.tar ./rim3. Restore the rim.tar file into the new location (newfilesystem): pax -rp e -f rim.tar (similar to tax -xvf rim.tar)4. Remove the original $SP_SYS_VARDIR/rim directory.5. Create a symbolic link in the $SP_SYS_VARDIR named "rim" that points to the new location - may need to be done as the root user.6. Change ownerships of the new $SP_SYS_VARDIR>/rim directory to match the ownerships of the other SharePlex directories.7. Ensure the SharePlex user has write permissions to the $SP_SYS_VARDIR/rim files. -
Informazioni aggiuntive
The way SharePlex computes the amount of space available is to make the system call statvfs() on the rim directory and then take the value for the number of blocks available times the block size. SP-QUE01001