Unable to Monitor Dell Compellent array due to Getting Current System Storage Usage error (4209520)

Chat now with support

Live-Hilfe anfordern
Registrierung abschließen

Anmelden

Preisinformationen anfragen

Vertrieb kontaktieren

Zurück

Feedback übermittelt

Konnten Sie mithilfe dieses Artikels ein Problem lösen?

Bewertung auswählen

Titel

Unable to Monitor Dell Compellent array due to Getting Current System Storage Usage error
Beschreibung

The environment was very large and there were 32 Storage Centers (SC) monitored by a single Enterprise Manager (EM) host. For this reason, there was a single Storage Collector created to collect topology and performance from the target.
Agent Manager log file for the Storage agent had the following error:
2016-06-12 15:21:30.515 ERROR [Collections[1]-0] DBLOG - Compellent perf: collection of Storage Center 'SCNAME' has failed
com.compellent.api.communication.CompellentApiMethodException: Error getting system settings from SCNAME
2016-06-01 05:49:02.963 ERROR [Collections[1]-0] DBLOG - Compellent perf: collection of Storage Center 'SCNAME' has failed
com.compellent.api.communication.CompellentApiMethodException: Error Getting Current Volume Storage Usage [Storage Center: 11111] : Error getting " Pagepool " from SC [SN: 11111 ] [Name: Storage Center ] (" ")
Moreover, EM host was showing native intermittent disconnection errors to the SCs. These disconnection errors were exist when no Storage
Collector agent connected to the EM host.
Last but not least, all the EM API calls Storage make are sequential. There are no concurrent calls made to Enterprise Manager.
Ursache

EM performance characteristics seemed to be exceeded with the amount of SC's were currently being monitored and SSL threads were being blocked natively.
Lösung
STATUS
Bug 52352 has been submitted to Dell Compellent Team.
Enable EM Debug logging and deactive the Storage Collector Agent to see the native blocked SSL threads error:
...\logs\serverlogs\server_error.log
2016-08-14 05:30:50,928 ERROR [org.apache.catalina.connector] (https-pool-threads - 635398) JBWEB001018: An exception or error occurred in the container during the request processing: java.lang.IllegalStateException: JBAS018053: No security context found
at org.jboss.as.web.security.SecurityActions$6.run(SecurityActions.java:136) [jboss-as-web-7.2.0.Final.jar:7.2.0.Final]
at org.jboss.as.web.security.SecurityActions$6.run(SecurityActions.java:130) [jboss-as-web-7.2.0.Final.jar:7.2.0.Final]
at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.7.0_67]
at org.jboss.as.web.security.SecurityActions.popRunAsIdentity(SecurityActions.java:130) [jboss-as-web-7.2.0.Final.jar:7.2.0.Final]
\logs\jvmreports\JvmReport.2016.08.16.02.52.40.864.CDT.xml
<thread>
<group>ServerService ThreadGroup</group>
<name>pool-27-thread-138188</name>
<id>15872881</id>
<daemon>false</daemon>
<alive>true</alive>
<state>BLOCKED</state>
<implType>java.lang.Thread</implType>
<cpuTime>843</cpuTime>
<blockedCount>19</blockedCount>
<blockedTime>87</blockedTime>
<stackTrace>
<element>sun.security.ssl.SSLContextImpl.getDefaultCipherSuiteList(SSLContextImpl.java:287)</element>
<element>sun.security.ssl.SSLSocketImpl.init(SSLSocketImpl.java:576)</element>
<element>sun.security.ssl.SSLSocketImpl.[init](SSLSocketImpl.java:512)</element>
<element>sun.security.ssl.SSLSocketFactoryImpl.createSocket(SSLSocketFactoryImpl.java:73)</element>
<element>sun.net.www.protocol.https.HttpsClient.createSocket(HttpsClient.java:409)</element>
<element>sun.net.NetworkClient.doConnect(NetworkClient.java:162)</element>
<element>sun.net.www.http.HttpClient.openServer(HttpClient.java:432)</element>
<element>sun.net.www.http.HttpClient.openServer(HttpClient.java:527)</element>
<element>sun.net.www.protocol.https.HttpsClient.[init](HttpsClient.java:275)</element>
<element>sun.net.www.protocol.https.HttpsClient.New(HttpsClient.java:371)</element>
<element>sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.getNewHttpClient
(AbstractDelegateHttpsURLConnection.java:191)
</element>
<element>sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:932)</element>
<element>sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect
(AbstractDelegateHttpsURLConnection.java:177)
</element>
<element>sun.net.www.protocol.https.HttpsURLConnectionImpl.connect(HttpsURLConnectionImpl.java:153)</element>
WORKAROUND 1
1. Create another instance of EM host and move one SC at a time to the new EM instance while NO Storage Collector agent is connected to the EM host and monitor for blocked threads.
2. Once verified there is no blocked threads happening; delete the current Storage Collector Agent
3. Create new Storage Collector Agents with different nameand point it to the new EM host
WORKAROUND 2
Based on the amount of the SC's are being monitored such as 32 SCs
1. Create 3 additional EM hosts and monitor 8 SCs at each EM host.
2. Once verified there is no blocked threads happening; delete the current Storage Collector Agent
3. Create 4 Storage Collector Agents with different name and point it to the each EM host
Weitere Informationen

This problem may be seen when monitoring large systems. Enterprise Manager 2015 R3 [Build: 15.3.1.300]

Feedback übermittelt

Konnten Sie mithilfe dieses Artikels ein Problem lösen?

Bewertung auswählen

Request a KB Article

Empfohlener Inhalt

Produkt(e):: Foglight for Virtualization Enterprise Edition
8.5, 8.4, 8.3

Thema/Themen:: Troubleshooting

Artikelhistorie:: Erstellt am: 10/7/2016
Letzte Aktualisierung am: 5/7/2023

Alle Artikel durchsuchen

Bitte w&auml;hlen Sie Ihr Produkt aus:

Damit wir Ihnen besser helfen können, geben Sie den Grund Ihres Chats an:

Empfohlene Lösungen für Ihr Problem

Unable to Monitor Dell Compellent array due to Getting Current System Storage Usage error (4209520)

Titel

Beschreibung

Ursache

Lösung

Weitere Informationen

Leave a Comment

Bitte wählen Sie Ihr Produkt aus: