DSP: Server Hangs Domino log Error: "Exception of type 'System.OutOfMemoryException' was thrown" (4328692)

DSP: Server Hangs Domino log Error: "Exception of type 'System.OutOfMemoryException' was thrown"

We have been experiencing Domino Server hangs on our sync server every 3-5 days. We are under the impression it could be a memory issue but wanted you to review logs to confirm.
Once the case is opened, I can upload the files.

A server hang is essentially when Domino server tasks have stopped.... but not all tasks. The Domino Server thinks it's still running but in reality it is not fully functional. So not all required Domino tasks are running but the server is still pingable and accessible.

A server crash is when the Domino server task is completely stopped causing the server to become unavailable at the OS level.

I found these errors in the console log.

[10CC:003A-20BC] 02/20/2021 12:05:11 AM  BTDirSync: (Error) Exception saving SyncReportMembership for group 12345:<value>. Exception of type 'System.OutOfMemoryException' was thrown.


 

This is indicating there is an issue with these two Domino groups. Possibly corruption or malformed, or the group has a large membership of nested groups which also contain nested groups, or duplicate groups.

These groups can be found using the following SQL query.
Open SQL management studio and run the following queue to find the two groups to idenify the group names. Once you have the group names, you can open the Domino directory, and find the two groups. The groups will need to be fixed (made smaller). Work with your Domino administrator.

SELECT *
FROM [DirSyncPro].[dbo].[BT_GroupMemberRelations] Where [BT_GroupMember_FK] = <REPLACEWITHGROUPID#>

Find the group name, then open the Domino Directory (NAB) find the groups, determine root cause and perform the proper remediation, either reducing the group members, reduce nesting, delete duplicates. etc.

If this does not resolve the issue then this is pointing to the SQL database requiring a cleanup. 

Run the following SQL queries one at a time, do not run all together.

1. Run this query first
    DELETE FROM [Dirsyncpro].[dbo].[DirSync_SyncReport] WHERE BT_Config_FK = 4

2. Once the above completes, run this next (This will take a long time to complete)
    DELETE FROM [Dirsyncpro].[dbo].[DirSync_SyncReportMembership] WHERE BT_Config_FK = 4

3 Run this query next
    DELETE FROM [Dirsyncpro].[dbo].[DirSync_SyncReportProfile] WHERE BT_Config_FK = 4

Run an iisreset from command prompt on the Windows IIS server when the above is complete.