Search the file for: with .* derivation rulettes. The results should resemble the following:
The third line (187230 derivation rulettes) indicates an issue because the number of derivation rulettes is significantly larger than any other rulettes listed. Locate the complex derivation definition (located above the with .* derivation rulettes section in the file), and make a note of the topology type and metric name. For example:
where DBSS_Top_Sql is the topology type and DBSS_Total_Elapsed_Time_Per_Exec is the metric name.
If there is evidence of derivation problems, contact Quest Support with this information for further assistance and to determine if the issue has been resolved in the latest version of the affected cartridge.
The most useful part of this service is in the extra information, which lists all topology types and, for each type, the number of instances, the number of instance versions, the maximum versions, and the effective instance versions. This information helps you determine if the topology is too large, or if there is a topology churn. Look for a high number of versions of instances, as well as a high maximum versions.
Topology churn is defined as the constant changing and creation of new versions of existing topology objects. Each time a property is updated on an instance, a new version of that instance is created. Topology churn can cause high CPU usage as the Management Server propagates the changes across the rest of the topology model.
Topology growth is defined as the continuous creation of new instances of a type of topology object. Topology growth can cause high CPU usage as models and rulettes are updated, as well as increased JVM heap usage. The entire topology model is stored in memory, so as the number of objects added increases, so does the heap usage.
If the values in columns five and six in the table above are greater than 5000, examine the highest numbers and work your way down the list. Resolving issues with the higher ones can sometimes resolve other churn issues, since topology changes to one object can cause changes in other objects.
This is also an example of a good, stable model. Even though the numbers are higher, 2908 x 2 = 5816, so the numbers are in balance. Additionally, in the last 7 days, there was only 1 new object, with 2 changes. There is no large growth or churn in this example.
This is also a bad model. In the past 7 days, 5295 new instances have been created. Column 4 indicates that some stale object cleanup has been done, but unless the root cause is found, the instances will keep being created.
If there is evidence of topology problems, contact Quest Support with this information for further assistance and to determine if the issue has been resolved in the latest version of the affected cartridge.
If many (thousands) of metrics are held in memory for long periods of time, they cannot be cleaned up by a garbage collector (GC) because they are active/live objects. Therefore, a large portion of memory is used simply by data that should be written into the database instead. This lead to JVM heap exhaustion, and performance problems.
cbc82b6a-1f8c-4fa8-a88a-fcb07af2854e—topology object ID
file_physical_io_pct—name of the metric
age:259200000—length of time the metric is kept in memory (in ms)
granularity:300000—rawness of the metric value (in ms)
num values:123—number of values of this metric on this object
delay:19276—length of time the metric has been in memory
<property name='file_physical_io_pct' type='Metric' is-many='false' is-containment='true' unit-name='count'>
<annotation name='UnitEntityName' value='percents'/>
This indicates that the file_phyiscal_io_pct metric is part of the DBO topology.