The Foglight console is very slow. In the thread dump there are several blocked http threads that include the string "waiting to lock <address>". The string is "waiting to lock <0x00007faa518bf6a0>" in the example thread excerpt below.
"http-0.0.0.0-8080-119" daemon prio=10 tid=0x00007fa5cc3fa000 nid=0xb193 waiting for monitor entry [0x00007fa5b0583000]
java.lang.Thread.State: BLOCKED (on object monitor)
at org.apache.catalina.core.StandardWrapper.allocate(StandardWrapper.java:801)
- waiting to lock <0x00007faa518bf6a0> (a org.apache.catalina.core.StandardWrapper)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:129)
...
There is a single “http” thread like the one listed below that has locked the memory address 0x00007faa518bf6a0 that all the blocked threads are waiting for.
"http-0.0.0.0-8080-8" daemon prio=10 tid=0x0000000002734000 nid=0x735c runnable [0x00007fa5bd29d000]
java.lang.Thread.State: RUNNABLE
at java.util.HashMap.getEntry(HashMap.java:347)
at java.util.HashMap.containsKey(HashMap.java:335)
at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:146)
at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:115)
at org.apache.catalina.core.StandardWrapper.loadServlet(StandardWrapper.java:1155)
- locked <0x00007faa518bf6a0> (a org.apache.catalina.core.StandardWrapper)
at org.apache.catalina.core.StandardWrapper.allocate(StandardWrapper.java:806)
- locked <0x00007faa518bf6a0> (a org.apache.catalina.core.StandardWrapper)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:129)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:175)
at org.jboss.web.tomcat.security.SecurityAssociationValve.invoke(SecurityAssociationValve.java:182)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:524)
at org.jboss.web.tomcat.security.JaccContextValve.invoke(JaccContextValve.java:84)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
at org.jboss.web.tomcat.service.jca.CachedConnectionValve.invoke(CachedConnectionValve.java:157)
at org.apache.catalina.authenticator.SingleSignOn.invoke(SingleSignOn.java:420)
at com.quest.nitro.web.tomcat.SmAuthValve.invoke(SmAuthValve.java:234)
at com.quest.nitro.web.tomcat.DenyRemoteRequestValve.filterRequest(DenyRemoteRequestValve.java:90)
at com.quest.nitro.web.tomcat.DenyRemoteRequestValve.invoke(DenyRemoteRequestValve.java:97)
at com.quest.nitro.web.tomcat.RequestEncValve.invoke(RequestEncValve.java:61)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:262)
at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:844)
at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:583)
at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:446)
at java.lang.Thread.run(Thread.java:662)
.
The above http thread will have a very high CPU utilization because it is spinning. Following is an example of the CPU utilization in the tread dump:
.
http-0.0.0.0-8080 8: 1.6064387E8 ms (49.1%/98.4%)
