Alarms: Rule actions do not trigger, but alarms appear in the alarms dashboard (4309641)

Rule actions do not trigger, but alarms appear in the alarms dashboard

Cause 1

1. One of the root causes of this issue is the Action-4-Threads, which is tied to rule action, is exhausted (20 threads by default).
   Thread dump excerpt shown below:

ie. Thread: Action-4-thread-12, id=439, priority=5, state=WAITING, thread group=Forge Core Threads
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:317)
(AbstractQueuedSynchronizer.java:1987)
java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:317)
org.apache.activemq.transport.FutureResponse.getResult(FutureResponse.java:40)
org.apache.activemq.transport.ResponseCorrelator.request(ResponseCorrelator.java:87)
org.apache.activemq.ActiveMQConnection.syncSendPacket(ActiveMQConnection.java:1276)
org.apache.activemq.ActiveMQSession.send(ActiveMQSession.java:1760)
org.apache.activemq.ActiveMQMessageProducer.send(ActiveMQMessageProducer.java:231)
org.apache.activemq.ActiveMQMessageProducerSupport.send(ActiveMQMessageProducerSupport.java:241)

Workaround 1

1. Generate an FMS thread dump
2. <FMS Home>\bin\fms -t
3. It will be found in <FMS home>\state directory
4. Search for Action-4-Thread, if all 20 are in a wait state, no rule actions will be triggered.
5. Contact Quest Support to identify the cause of the thread exhaustion as a FMS restart will mask the root cause.