Foglight Evolve 9.3 - Administration and Configuration Guide

Manage Thresholds

Foglight allows you to control access to thresholds. For each threshold you can grant or deny read, write, or control access to roles or users. For more information about security concepts in Foglight, see Managing Users and Security.

Foglight employs the following behavior when it comes to threshold permissions:

TIP: The Not Assigned icons in the Permissions columns indicate that the role does not have permissions assigned to it.

TIP: Three check marks in the Permissions columns indicate that the role already has permissions assigned to it.

You can delete thresholds that you no longer need. However, when a threshold is deleted, all references to that threshold in rule conditions or derived metric expressions become invalid. This may cause a rule to fail to evaluate. If this occurs, you must manually modify the rule condition or expression.

The Manage Thresholds dashboard lists all of the thresholds that exist in your monitoring environment. This includes the thresholds that come with the Foglight Management Server, any installed cartridges, and also any thresholds that you create.

IMPORTANT: Because each threshold level comes with a unique set of pre-defined bound levels, changing any of these values may invalidate the existing threshold bounds.

Threshold levels in metrics are useful in situations when you need to reference a specific metric value multiple times, for example in derived metrics or rules.

Different types of metrics can have different types of threshold levels such as the levels that refer to agent states, alarm severity, and others. Each threshold level comes with a unique set of pre-defined bound levels. For example, the agent state threshold level for the includes the bound levels that correspond to different agent states, such as Stopped, Started, Running, and others. A bound level value can be associated with a constant value, a registry variable, or another metric of the same topology type.

Some thresholds come included with Foglight and installed cartridges, including baselineAvailability. If the existing derived metrics do not meet your needs, you can create a new one and add it to the threshold collection.

From here, you can proceed to Add bounds to threshold levels.

Each threshold level comes with a unique set of bound levels. For example, the threshold level for agent state includes bound levels that correspond to different agent states, such as Stopped, Started, Running, and others. A bound level value can be associated with a constant value, a registry variable, or another metric of the same topology object.

Table 63. Threshold levels and their associated bound levels

Threshold Level	Threshold Bound Level
AgentHealthState	OK
	Running Unexpectedly
	Broken
	Agent Info Not Present
AgentState	Unknown
	Stopped
	Starting
	Stopping
	Running
	Collecting data
	Running but not collecting data
AlarmChangeType	Fire
	Clear
	Acknowledge
	UserDefined Data
AlarmSeverity	Undefined
	Normal
	Fire
	Warning
	Critical
	Fatal
CatalystServiceState	Stopped
	Stopping
	Starting
	Started
	Failed
	Destroyed
	Created
	Unregistered
	Registered
HostDiscoveryStatus	Active
	Inactive
IncidentChangeType	Create
	Close
	Acknowledge
	Modify
	Delete
ThresholdSeverity	Normal
	Critical
	Fatal
	Warning
UnitGroupAutoScalingType	smalestValue
	Locale

You can have one or more types of threshold bounds in a threshold level.

Table 64. There are three different types of threshold bound types:

Type	Severity level is bound to
Metric Threshold Bound	Another metric
Registry Variable Threshold Bound	A registry variable
Baseline Threshold Bound	A baseline, defined in the baseline cartridge that contain one or more baseline patterns for the selected metric
NOTE: This option is only available after installing a baseline cartridge that contains the baseline definitions for the selected metric property. A baseline illustrates the behavior of a metric over time, allowing for its categorization into repeating patterns, such as weekly, monthly, and so on, that can be charted, as required. A baseline cartridge contains the definitions of one or more baselines for a particular metric. Agent developers can write baseline cartridges followed by installing them on the Foglight Management Server. When you add or edit a threshold for a metric, if a cartridge that contains baseline definitions for that metric, the Baseline Threshold Bound option becomes available, allowing you to use one or more metric baseline definitions in the selected metric’s threshold definitions.
Constant Threshold Bound	A fixed value

Foglight evaluates threshold bounds in the order that they are listed, starting with the first one. Changing their order affects the output of actions that are associated with those threshold levels.

For example, if a threshold level includes several threshold bounds that reference standard Foglight severity levels in the ascending order such as Normal, Warning, Critical, and Fatal, and you change their order in the list to Normal, Critical, Warning, and Fatal, the Warning, the bound that is associated with the Warning level evaluates to True only after the evaluation of the Critical level.

The ranges of threshold levels are created by going over threshold bounds in the order form first to last (from top to bottom in the Threshold Bounds table) and assigning the threshold level of the bound to the range immediately above the threshold bound. In some cases, their ranges do not always follow a progression sequence. This can happen in cases where threshold bounds values are calculated during run-time. For example, a set of threshold bounds can include a mix of constant values and baseline values. As a result, the effective threshold ranges may be different at different time points. It is possible to have valid configurations when threshold ranges minimize to zero '0' or even overlap. When this happens, the default behavior causes the higher threshold level to be applied. To ensure that the appropriate threshold level is applied when their ranges overlap, you can either accept the default behavior, or define a precedence to override the level with a higher value. This can be done with the Override flag. When set for a threshold bound, this flag ensures that the threshold level associated with a particular threshold bound is applied to incoming data even though its value overlap with a higher threshold level.

Figure 96. Examples of the change to threshold behavior when bounds are changed.

Threshold Bounds	Applied Threshold Levels	Observed Behavior
		There are no threshold level overrides set for any of the threshold bounds.
		The Override flag is set for the Fatal threshold level (column four in the Threshold Bounds table on the left), causing it to take precedence over Warning.
		The Override flag is set for the Normal and Fatal threshold levels, causing it to take precedence over Warning. In this case, Normal takes precedence over Critical, and Fatal takes precedence over Warning.

NOTE: This procedure continues from View and edit thresholds or Getting started: create thresholds.

NOTE: This procedure continues from View and edit thresholds or Getting started: create thresholds.

NOTE: The Baseline Threshold Bound option is only available after installing a baseline cartridge that contains baseline definitions for the selected metric property. Agent developers can write baseline cartridges. A baseline illustrates the behavior of a metric over time, allowing for its categorization into repeating patterns, such as weekly, monthly, and so on, that can be charted. A baseline cartridge contains the definitions of one or more baselines for a particular metric. When you add or edit a threshold for a metric, if a cartridge contains baseline definitions for that metric, the Baseline Threshold Bound option becomes available, allowing you to use one or more metric baseline definitions in the selected metric’s threshold definitions.

NOTE: This procedure continues from View and edit thresholds or Getting started: create thresholds.

NOTE: This procedure continues from View and edit thresholds or Getting started: create thresholds.

NOTE: his procedure continues from any of the following sections: • To bind a threshold level to a metric: • To bind a threshold level to a registry variable: • To bind a threshold level to a constant value:

IMPORTANT: The Foglight Management Server evaluates threshold bounds in the order that they are listed, starting with the first one. Changing their order affects the output of actions that are associated with that threshold level.

Manage Registry Variables and Check Registry Value

This dashboard is a starting point for any work made to the Foglight registry. It displays a list of all registry variables that exist in your environment, including the variables that come with the Management Server and any installed cartridges. It also provides an interface for creating and managing registry variables.

By default, the following columns are displayed:

In addition to these columns, the following columns can also be displayed by clicking and selecting them from the Show Columns dwell that appears.

For more information about the New Registry Variable Wizard, see Getting started: create registry variables.

For more information about these flows, see View and edit registry variable settings and Copy or delete registry variables.

Copying a registry variable is useful in situations when you need to quickly create a modified version of an existing variable. Instead of redefining the scope and performance calendars, simply copy a similar registry variable and edit its definitions.

If you have any registry variables that are no longer referenced in rule conditions, you can delete them from the list. When a registry variable is deleted, all references to that variable in rule conditions and expressions become invalid. This may cause the rule to fail to evaluate. If this occurs, you must manually modify the rule condition or expression. For details, see Writing conditional expressions for time-, data-, or schedule-driven rules and Write a conditional expression for an event-driven rule.

IMPORTANT: You must use a unique name.

From here, you can edit the newly copied registry variable to better suit your needs. For example, you can have its value change over time, or scope it to topology types or object instances. For further information, see Using performance calendars in registry variable definitions and Scoping registry variables to topology types or object instances.

The Manage Registry Variables dashboard shows all of the registry variables that exist in your monitoring environment. This includes the variables that come with the Foglight Management Server, any installed cartridges, and any registry variables that you create.

From here, you can edit the registry variable to better suit your needs. For example, you can have its value change over time, or scope it to topology types or object instances. For further information, see Using performance calendars in registry variable definitions and Scoping registry variables to topology types or object instances.

Many registry variables come included with Foglight and installed cartridges, including AvailabilityCritical, AvailabilityFatal, AvailabilityTarget, and many others. If the existing registry variables do not meet your needs, you can create a new one and add it to the registry variable collection.

Foglight registry variables can be used in rule conditions, expressions, and actions. A registry variable can have a global value that is available to all topology types and objects. It can also have one or more values associated with specific topology types or objects, or calendar dates.

The New Registry Variable Wizard takes you to the process of creating registry variables and setting its basic properties, such as the name, data type, and initial value.

The values that you assign to a variable in the wizard must be consistent with the data type that you specify at variable creation time. For example, if you selected the data type Integer, you need to provide a valid integer number when specifying the value.

IMPORTANT: The registry variable type cannot be changed once the registry variable has been added.

Table 65. Registry value types and their descriptions, with examples.

Registry Value Type	Description	Example
Boolean	A boolean value: true or false.	true
Double	A decimal value between 2-1074 and (2-2-52) * 21023. For example, valid values include 34.1234, 35e3 (meaning 35,000.0) or 1.2E-2 (meaning 0.012).	95.0
Integer	An integer value between -231 and 231 -1. For example, valid values include 235 and -10000.	6825
Long	An integer value that is too large to fit the Integer data type. It can be in the range between -263 and 263- 1.	1099511627776
Password	A password value. Prior to storing the value of the Password data type in the registry, Foglight encrypts it and saves the encrypted value in the database. This is useful in cases where you need to secure password values when passing it to command or remote command actions. For example, you create a registry variable of the Password type, MyPassword, and set its value to demo123. Foglight encrypts the registry value, then stores it encrypted in the database (for example, 43-119-184-240-170-150-124-218-30-112-216-76-197-233-188-206). In a function call registry("MyPassword"), Foglight retrieves the registry value from the database in its encrypted form, which secures the password value.	demo123
String	A text string.	This message was generated by Foglight.
Timestamp	Contains a date and time. For example, valid values include 06/12/06, 2006-06-21 15:30:21.0, June 7, 2006 3:08:21 PM. Invalid date or time formats cannot be saved to the database and any attempts to save them results in an error. If the time is not provided, or the values are invalid, Foglight treats the time as midnight. Prior to saving the value of a variable that uses this data type, Foglight converts it to the following format and stores it in the database: yyyy-mm-dd hh:mm:ss.ds	2008-08-10 23:15:16.0

TIP: You can edit the Global Default value at a later time.

TIP: Because you did not specify the Global Default, this value appears unset. You can change it by clicking Change.

TIP: Failing to set this value causes a validation error. Null is not an acceptable value for the Global Default value in any case. If you do not want to set a value at this time, click Previous and then Finish without setting a global default.

From here, you can edit the newly created registry variable to better suit your needs. For example, you can have its value change over time, or scope it to topology types or object instances. For further information, see Using performance calendars in registry variable definitions and Scoping registry variables to topology types or object instances.

If you want the default value to change over time, add one or more schedules to the performance calendar and specify the value for each schedule.

For example, there is a simple rule that applies to the servlets in your application; an alarm fires if the request response time for a servlet exceeds a value set by the ResponseTimeTooLong registry variable that is scoped to the topology type J2EEServlet and with the default scoped value of eight seconds. However, you know that at certain times of day response times for servlet instances are expected to exceed this threshold. At these times, the acceptable response time can be as long as 15 seconds. To prevent alarms from generating as a result of false positives, leave the variable’s scoped value of eight seconds, and create a schedule that recurs daily at the times when it is acceptable for response times for the servlet instances to exceed eight seconds, and specify the alternative threshold of 15 seconds by associating this value with the schedule.

Foglight evaluates the schedules in the order they are listed in the performance calendar, starting with the first one. Changing their order affects the behavior of the actions that are associated with the variable whose value is set by the schedule-based entries in the performance calendar.

For example, if there are two schedule entries in the performance calendar that overlap in time but have two different values, the first entry listed takes precedence, causing one or more actions that are associated with that variable to make use of that entry for the duration of the schedule.

The Performance Calendar Wizard, accessible from the Performance Calendars List, allows you to associate a registry value with a schedule.

IMPORTANT: Choosing to create or edit the existing schedules causes you to exit the wizard.

IMPORTANT: The Management Server evaluates schedule-based values in the order that they are listed, starting with the first one. Changing their order affects the output of actions that are associated with the value that is associated with their schedule entries.

From here, you can proceed to Scoping registry variables to topology types or object instances.

A Foglight registry variable can have a global value that is available to all topology types and objects. It can also have one or more values associated with specific topology types or objects.

The way you define registry variables can help you to reduce the number of rules that need to be created and managed. For example, instead of creating multiple rules that follow the same logic to monitor different data types or object, you can create a single rule and reference a registry variable scoped to those entities.

TIP: Use the Customizer to display two additional columns that are hidden by default: Long Name and Monitoring Agent. Long Name consists of the object name followed by its type name. For example: Broken (AgentHealthState). Monitoring Agent contains the name of the monitoring agent. This information is important if you have monitoring agents whose definitions include objects of the same type and name, like the legacy Cartridge for Operating Systems agents. For more information about the Customizer, see the Foglight User Help.

From here, to edit a registry value and associate it with performance calendars, click the registry value in the list.

A registry variable can have a global value that is available to all topology types and objects. It can also have multiple additional values associated with specific topology types or objects, or calendar dates. To find out what is the value of a registry variable for a particular topology type or object, and, if applicable, during a specific time period, use the Check Registry Value dashboard.

IMPORTANT: By default, the SYSADMIN variable has no scoped values. They are created for the purpose of this exercise.

NOTE: The list does not include the registry variables that contain password values.

IMPORTANT: If you had previously chosen to show or hide one or more columns in the Registry Value table, the selected layout is reflected in the exported file. For example, if you display only the time at which the value was set and the registry value, only those columns appear in the export file.

Some Foglight dashboards have reports associated with them. This allows you to run a report based on the current dashboard. You can generate the report using the Reports menu in the top-right corner.

The Manage Registry Variables dashboard is associated with the Registry Variable Report. Run this report by choosing Registry Variable Report from the Reports menu, and specifying the input parameters in the report wizard.

The report wizard provides more information about the Registry Variable Report and instructions on how to set the input values. For more information about reports in Foglight, see the Foglight User Help.

Manage Schedules

Foglight allows you to control access to a schedule. For each schedule you can grant or deny read, write, or control access to roles or users. For more information about security concepts in Foglight, see Managing Users and Security.

Foglight employs the following behavior when it comes to schedule permissions:

Use the Edit Permissions button () on the Manage Schedules dashboard to navigate to the Edit Permissions for Schedule area, that allows you to add or edit permissions to roles and users, as outlined below.

TIP: Three check marks in the Permissions columns indicate that the role already has permissions assigned to it.

TIP: Three check marks in the Permissions columns indicate that the role already has permissions assigned to it.

Copying a schedule is useful in situations when you need to quickly create a modified version of an existing schedule. Instead of re-creating all of the schedule settings, simply copy an existing schedule and edit the required schedule items.

When a schedule is deleted, all references to that schedule are removed as well. Any performance calendars that are based on that schedule are removed, and the deleted schedule is removed from the list of effective and blackout schedules for rules.

A schedule contains one or more schedule items, each defining a time period and a recurrence pattern. Viewing schedule definitions allows you to find out the details about the items associated with a schedule.

You can edit an schedule by adding or removing one or more of its recurrence patterns. For example, if you have a schedule that runs indefinitely from 10:00 AM to 11:00 AM daily and on the first day of the month from 8:00 AM to 6:00 PM, but you want to edit it to also run every Saturday from 11:00 AM to 4:00 PM in May, add a schedule item for each of these time spans to the schedule.

Schedules consisting of multiple scheduled items must include a relevant start time. By default, the start time is the day the schedule is created. The start time of each schedule item must be specified to reflect the first run of the scheduled item.

A default Foglight installation includes a number of scheduled, including Always, Business hours, Business week, and many others. Rules, registry variables, and derived metrics can make use of schedules. For example, a registry variable can have multiple values, each associated with a specific schedules. If none of the existing schedules meet your needs, you can create a new schedule and add it to the schedule collection.

A schedule can contain one or more items, each describing a recurrence pattern. For example, if you want a schedule to run indefinitely from 10:00 AM to 11:00 AM daily and on the first day of the month from 8:00 AM to 6:00 PM, and also every Saturday from 11:00 AM to 4:00 PM in May, add a schedule item for each of these time spans to the schedule.

When you create a schedule, you have to specify at least one schedule item and its recurrence pattern. You can edit the schedule at a later time by adding or removing schedule items.

There are six types of patterns that you can define in a schedule item.

Table 66. Pattern types you can use in a schedule item.

Pattern Name	Pattern Description	For details, see…
Once	Starts at a specified date and time, for a specified duration, and ends at a defined end date and time	To define a schedule item that occurs once:
Periodical	Starts at a specified time and date for a certain duration, repeats at specified time periods, with or without a defined end date and time	To define a schedule item that occurs periodically:
Daily	Starts at a specified time and date, runs for a whole day or a fraction of a day, repeats at a regular interval of days, with or without a defined end date and time	To define a schedule item that occurs daily:
Weekly	Starts at a specified time and date, runs for a whole day or a fraction of a day, repeats at a regular interval of weeks on one or more days of the week, with or without a defined end date and time	To define a schedule item that occurs weekly:
Monthly	Starts at a specified time and date, runs for a whole day or a fraction of a day, repeats at a regular interval of months on one or more days of the month, with or without a defined end date and time	To define a schedule item that occurs monthly:
Yearly	Starts at a specified time and date, runs for a whole day or a fraction of a day, repeats at a regular interval of years on one or more days of the year, with or without a defined end date and time	To define a schedule item that occurs yearly:

IMPORTANT: Schedules consisting of multiple scheduled items must include a relevant start time. By default, the start time is the day the schedule is created. The start time of each schedule item must be specified to reflect the first run of the scheduled item.

TIP: The Day box accepts the values between 1 and 31. Any positive values outside that range are automatically adjusted to 31; negative values are not accepted.

TIP: The Hour box accepts the values between 0 and 23. Any positive values outside that range are automatically adjusted to 23; negative values are not accepted. The Minute box accepts the values between 0 and 59. Any positive values outside that range are automatically adjusted to 59; negative values are not accepted.

NOTE: In the Range of Occurrence area, the No End option appears disabled while the End By Date option is enabled and selected. This is because a schedule item that occurs once must have an end date.

TIP: The Day box accepts the values between 1 and 31. Any positive values outside that range are automatically adjusted to 31; negative values are not accepted.

TIP: The Hour box accepts the values between 0 and 24. Any positive values outside that range are automatically adjusted to 24; negative values are not accepted. The Minute box accepts the values between 0 and 59. Any positive values outside that range are automatically adjusted to 59; negative values are not accepted.

TIP: The Day box accepts the values between 1 and 31. Any positive values outside that range are automatically adjusted to 31; negative values are not accepted.

TIP: The Hour box accepts any positive values. The Minute box accepts the values between 0 and 59. Any positive values outside that range are automatically adjusted to 59; negative values are not accepted.

TIP: The Hour box accepts any positive values. The Minute box accepts the values between 0 and 59. Any positive values outside that range are automatically adjusted to 59; negative values are not accepted.

IMPORTANT: The recurrence period must be longer than the duration specified in <Link>step 2. For example, if the duration of the schedule item is three hours, the recurrence periods should occur at intervals that are longer than three hours.

TIP: The Hour box accepts any positive values. The Minute box accepts the values between 0 and 59. Any positive values outside that range are automatically adjusted to 59; negative values are not accepted.

TIP: Use the Calendar button on the right and use the calendar controls that appear to specify the start date.

TIP: The Day box accepts the values between 1 and 31. Any positive values outside that range are automatically adjusted to 31; negative values are not accepted. The Hour box accepts the values between 0 and 24. Any positive values outside that range are automatically adjusted to 23; negative values are not accepted. The Minute box accepts the values between 0 and 59. Any positive values outside that range are automatically adjusted to 59; negative values are not accepted.

TIP: The Day box accepts the values between 1 and 31. Any positive values outside that range are automatically adjusted to 31; negative values are not accepted.

• Specify the start time and end time of the schedule item.
	• Use the Start Time [hh:mm] area to specify the hour and minute of the start time. NOTE: The Hour box accepts the values between 0 and 23. Any positive values outside that range are automatically adjusted to 23; negative values are not accepted. The Minute box accepts the values between 0 and 59. Any positive values outside that range are automatically adjusted to 59; negative values are not accepted.
	• Select End Time [hh:mm] and specify the hour and minute of the end time. NOTE: The Duration [hh:mm] boxes appear disabled when you specify the End Time [hh:mm] option. The end time should occur after the start time. NOTE: The Hour box accepts the values between 0 and 24. Any positive values outside that range are automatically adjusted to 24; negative values are not accepted. The Minute box accepts the values between 0 and 59. Any positive values outside that range are automatically adjusted to 59; negative values are not accepted.
or
• Specify the start time and duration of the schedule item.
	• Use the Start Time [hh:mm] area to specify the hour and minute of the start time. NOTE: The Hour box accepts the values between 0 and 23. Any positive values outside that range are automatically adjusted to 23; negative values are not accepted. The Minute box accepts the values between 0 and 59. Any positive values outside that range are automatically adjusted to 59; negative values are not accepted.
	• Select Duration [hh:mm] and specify the hour and minute of the duration time. NOTE: The End Time [hh:mm] boxes appear disabled when you specify the Duration [hh:mm] option. NOTE: The Hour box accepts the values between 0 and 24. Any positive values outside that range are automatically adjusted to 24; negative values are not accepted. The Minute box accepts the values between 0 and 59. Any positive values outside that range are automatically adjusted to 59; negative values are not accepted.

TIP: The Every box accepts any positive values.

TIP: Use the Calendar button on the right and use the calendar controls that appear to specify the start date.

TIP: The Day box accepts the values between 1 and 31. Any positive values outside that range are automatically adjusted to 31; negative values are not accepted. The Hour box accepts the values between 0 and 24. Any positive values outside that range are automatically adjusted to 23; negative values are not accepted. The Minute box accepts the values between 0 and 59. Any positive values outside that range are automatically adjusted to 59; negative values are not accepted.

TIP: The Day box accepts the values between 1 and 31. Any positive values outside that range are automatically adjusted to 31; negative values are not accepted.

Table 67. Select one of the following steps to create a schedule for part of the day.

• Specify the start time and end time of the schedule item.
	• Use the Start Time [hh:mm] area to specify the hour and minute of the start time. NOTE: The Hour box accepts the values between 0 and 23. Any positive values outside that range are automatically adjusted to 23; negative values are not accepted. The Minute box accepts the values between 0 and 59. Any positive values outside that range are automatically adjusted to 59; negative values are not accepted.
	• Select the End Time [hh:mm] area and specify the hour and minute of the end time. NOTE: The Duration [hh:mm] boxes appear disabled when you specify the End Time [hh:mm] option. The end time should occur after the start time. NOTE: The Hour box accepts the values between 0 and 24. Any positive values outside that range are automatically adjusted to 24; negative values are not accepted. The Minute box accepts the values between 0 and 59. Any positive values outside that range are automatically adjusted to 59; negative values are not accepted.
or
• Specify the start time and duration of the schedule item.
	• Use the Start Time [hh:mm] area to specify the hour and minute of the start time. NOTE: The Hour box accepts the values between 0 and 23. Any positive values outside that range are automatically adjusted to 23; negative values are not accepted. The Minute box accepts the values between 0 and 59. Any positive values outside that range are automatically adjusted to 59; negative values are not accepted.
	• Select Duration [hh:mm] and specify the hour and minute of the duration time. NOTE: The End Time [hh:mm] boxes appear disabled when you specify the Duration [hh:mm] option. NOTE: The Hour box accepts the values between 0 and 24. Any positive values outside that range are automatically adjusted to 24; negative values are not accepted. The Minute box accepts the values between 0 and 59. Any positive values outside that range are automatically adjusted to 59; negative values are not accepted.

TIP: The Every box accepts any positive values.

TIP: Use the Calendar button on the right and use the calendar controls that appear to specify the start date.

TIP: The Day box accepts the values between 1 and 31. Any positive values outside that range are automatically adjusted to 31; negative values are not accepted. The Hour box accepts the values between 0 and 24. Any positive values outside that range are automatically adjusted to 23; negative values are not accepted. The Minute box accepts the values between 0 and 59. Any positive values outside that range are automatically adjusted to 59; negative values are not accepted.

TIP: The Day box accepts the values between 1 and 31. Any positive values outside that range are automatically adjusted to 31; negative values are not accepted.

• Specify the start time and end time of the schedule item.
	• Use the Start Time [hh:mm] area to specify the hour and minute of the start time. NOTE: The Hour box accepts the values between 0 and 23. Any positive values outside that range are automatically adjusted to 23; negative values are not accepted. The Minute box accepts the values between 0 and 59. Any positive values outside that range are automatically adjusted to 59; negative values are not accepted.
	• Select End Time [hh:mm] and specify the hour and minute of the end time. NOTE: The Duration [hh:mm] boxes appear disabled when you specify the End Time [hh:mm] option. The end time should occur after the start time. NOTE: The Hour box accepts the values between 0 and 24. Any positive values outside that range are automatically adjusted to 24; negative values are not accepted. The Minute box accepts the values between 0 and 59. Any positive values outside that range are automatically adjusted to 59; negative values are not accepted.
or
• Specify the start time and duration of the schedule item.
	• Use the Start Time [hh:mm] area to specify the hour and minute of the start time. NOTE: The Hour box accepts the values between 0 and 23. Any positive values outside that range are automatically adjusted to 23; negative values are not accepted. The Minute box accepts the values between 0 and 59. Any positive values outside that range are automatically adjusted to 59; negative values are not accepted.
	• Select the Duration [hh:mm] area and specify the hour and minute of the duration time. NOTE: The End Time [hh:mm] boxes appear disabled when you specify the Duration [hh:mm] option. NOTE: The Hour box accepts the values between 0 and 24. Any positive values outside that range are automatically adjusted to 24; negative values are not accepted. The Minute box accepts the values between 0 and 59. Any positive values outside that range are automatically adjusted to 59; negative values are not accepted.

TIP: The Day box accepts the values between 1 and 31. Any positive values outside that range are automatically adjusted to 31; negative values are not accepted. The every box accepts any positive values.

TIP: The Every box accepts any positive values.

TIP: The Every box accepts any positive values.

TIP: Use the Calendar button on the right and use the calendar controls that appear to specify the start date.

TIP: The Day box accepts the values between 1 and 31. Any positive values outside that range are automatically adjusted to 31; negative values are not accepted. The Hour box accepts the values between 0 and 24. Any positive values outside that range are automatically adjusted to 23; negative values are not accepted. The Minute box accepts the values between 0 and 59. Any positive values outside that range are automatically adjusted to 59; negative values are not accepted.

TIP: The Day box accepts the values between 1 and 31. Any positive values outside that range are automatically adjusted to 31; negative values are not accepted.

• Specify the start time and end time of the schedule item.
	• Use the Start Time [hh:mm] area to specify the hour and minute of the start time. NOTE: The Hour box accepts the values between 0 and 23. Any positive values outside that range are automatically adjusted to 23; negative values are not accepted. The Minute box accepts the values between 0 and 59. Any positive values outside that range are automatically adjusted to 59; negative values are not accepted.
	• Select End Time [hh:mm] and specify the hour and minute of the end time. NOTE: The Duration [hh:mm] boxes appear disabled when you specify the End Time [hh:mm] option. The end time should occur after the start time. NOTE: The Hour box accepts the values between 0 and 24. Any positive values outside that range are automatically adjusted to 24; negative values are not accepted. The Minute box accepts the values between 0 and 59. Any positive values outside that range are automatically adjusted to 59; negative values are not accepted.
or
• Specify the start time and duration of the schedule item.
	• Use the Start Time [hh:mm] area to specify the hour and minute of the start time. NOTE: The Hour box accepts the values between 0 and 23. Any positive values outside that range are automatically adjusted to 23; negative values are not accepted. The Minute box accepts the values between 0 and 59. Any positive values outside that range are automatically adjusted to 59; negative values are not accepted.
	• Select Duration [hh:mm] and specify the hour and minute of the duration time. NOTE: The End Time [hh:mm] boxes appear disabled when you specify the Duration [hh:mm] option. NOTE: The Hour box accepts the values between 0 and 24. Any positive values outside that range are automatically adjusted to 24; negative values are not accepted. The Minute box accepts the values between 0 and 59. Any positive values outside that range are automatically adjusted to 59; negative values are not accepted.

TIP: The Day box accepts the values between 1 and 31. Any positive values outside that range are automatically adjusted to 31; negative values are not accepted.

TIP: Use the Calendar button on the right and use the calendar controls that appear to specify the start date.

TIP: The Day box accepts the values between 1 and 31. Any positive values outside that range are automatically adjusted to 31; negative values are not accepted. The Hour box accepts the values between 0 and 24. Any positive values outside that range are automatically adjusted to 23; negative values are not accepted. The Minute box accepts the values between 0 and 59. Any positive values outside that range are automatically adjusted to 59; negative values are not accepted.

Manage Retention Policies

While it is theoretically possible to create any retention policy that you desire in Foglight, the design of the system constrains the easy-to-accomplish retention policies to a narrow range of options. Specifically, the database design of Foglight provides a structure that is capable of holding data in three different buckets, called generations, that are defined in <Foglight_home>/config/storage-config.xml. Each generation has a predefined period of time in which it retains data.

Without modifying database generations, there are specific rules that you must follow when assigning retention policies, to ensure that you achieve the expected results. Any changes to the database generations in <Foglight_home>/config/storage-config.xml require a Management Server restart, in order for these changes to take effect.

This provides information on the key mechanisms involved in retention policies and rules for defining retention policies that work effectively with the default database configuration.

Generations refer to the database structures that hold long-term data. For any given metric, each generation can hold one aggregation level of data (for example, raw, hourly averages, 4 hour averages, and so on). Out of the box, there are three generations:

Because the collected data is constrained to the above aggregation periods, retention policies are also constrained to a set of rules. In general, you can create retention policies that:

The data service periodically writes data from the short-term memory cache to Generation 1. The frequency in which data is written is defined in the first retention policy (for more information, see How retention policies interact with database generations). This interval should not exceed 15 minutes to prevent the Foglight Management Server memory from growing too large.

A nightly roll-up job aggregates data and writes that data to Generation 2 and Generation 3. The roll-up is only done once a day, according to the time set in the Daily Database Maintenance schedule. For more information about this and other schedules in Foglight, see Associate Metric Calculations with Schedules.

Both mechanisms for populating the repository (from memory to the database, database roll-ups) use the retention policies defined in the Retention Policies dashboard as the guidelines for how they store data.

A retention policy is the set of definitions, for a given object, that indicate how data is stored. Each definition within a policy contains two parameters:

Policies can be set at an object level; however, retention policies also adhere to the object inheritance capabilities. If a policy has not been explicitly assigned to an object, it inherits a value from a higher level in the model. The top-level object is TopologyObject.

The policy that is applied to TopologyObject, and therefore any object which does not have explicitly assigned policies, is as follows:

Table 68. Policy, duration, and meaning.

Policy		Translates to
Age	15 minutes	After 15 minutes, store 15 minute aggregate data to Generation 1 (where they are stored for three days)
Roll-up	15 minutes
Age	3 days	Data older than 3 days is eligible for roll-up to one-hour averages and persisted into Generation 2 (where they are stored for 14 days)
Roll-up	1 hour
Age	14 days	Data older than 14 days old is eligible for Roll-up to four-hour averages and persisted into Generation 3 indefinitely
Roll-up	4 hours

NOTE: Individual cartridges frequently have their own policies which must be examined on individual object level to understand that retention policy behavior.

As indicated in the above diagram, any data whose retention policies include the purge settings is stored in the Generation 3 aggregation, from which it is purged in accordance with the purge settings. Purging from the last generation is done by recreating the tables to filter out the observations that are to be purged, and it occurs once a month for each configured purge setting. For example, if there are n policies that request a purge after 12 months and m policies that request a purge after 18 months, then at the start of the month, Foglight performs a single purge on the 12-month tables and a single purge on the 18-month tables. If your business requirements dictate that any data older than x months should be purged from the database, the most efficient implementation is to edit the Generation 3 setting in <Foglight_home>/config/storage-config.xml, whose default length is set to one hundred years, to x months. For example, the following code block illustrate a Generation 3 setting in storage-config.xml that dictates the purge of any data that is older than six months:

While the browser interface does not prevent you from setting policies that are in conflict with the generations, setting policies that are outside of these boundaries does not yield the expected results. Instead, the retention policy engine finds the most optimal scheme for your data (ensuring that the lowest granularity is written to Generation 1 and that longest duration data are written to Generation 3).

The table shows how to configure retention policies, at 1, 2 or 3 levels of aggregation, following the specifications below.

Table 69. How to configure retention policies.

	Acceptable age values	Acceptable roll-up values	Explanation
Three-level policy (including a purge)
Level 1	<= 15 minutes	<= 15 minutes	Data is persisted at the roll-up interval defined in the Level 1 policy for three days.
Level 2	> 15 minutes and < 3 days	Any roll-up greater than Level 1	The age date for the Level 2 policy must be less than or equal to three days. Data is persisted at the roll-up interval defined in the Level 2 policy for 14 days.
Level 3	> Level 2 setting and < 14 days	Any roll-up greater than Level 2	The age date for the Level 3 policy must be less than or equal to 14 days. Data is persisted at this roll-up interval indefinitely. A purge policy defines a minimum length of time that data must persist before it is truncated.
Purge Policy	> Level 3 setting	Purge	Data is never purged from the system before the age value of the purge policy. Data may, however, be retained for longer than the setting as the system waits to find an acceptable time to purge data.
Two-level policy (including a purge)
Level 1	<= 15 minutes	<= 15 minutes	Data is persisted at the roll-up interval defined in the Level 1 policy for either three or 14 days, depending on the age of the Level 2 setting. If the age of the Level 2 setting is less than or equal to three days, then the data is persisted for three days. If the age of the Level 2 setting is between three and 14 days, the data is persisted for 14 days.
Level 2	<= 14 days	Any roll-up greater than Level 1	The age date for the Level 2 policy must be less than or equal to 14 days. Data is persisted at this roll-up interval indefinitely. A purge policy defines a minimum length of time that data must persist before it is truncated.
Purge Policy	> Level 2 setting	Purge	Data is never purged from the system before the age value of the purge policy. Data may, however, be retained for longer than the setting as the system waits to find an acceptable time to purge data.
One-level policy (including a purge)
Level 1	<= 15 minutes	<= 15 minutes	Data is persisted at the roll-up interval defined in the Level 1 policy indefinitely. A purge policy will define a minimum length of time that data must persist before it is truncated.
Purge Policy	> 15 minutes	Purge	Data is never purged from the system before the age value of the purge policy. Data may, however, be retained for longer than the setting as the system waits to find an acceptable time to purge data.

Use the Manage Retention Policies dashboard to view, edit, and create retention policies for topology types and properties of topology types. Each policy specifies one or more time periods after which the data is rolled up and the granularity of the roll-up.

In some cases your retention policies may cause an unacceptable increase in the database size, which typically happens if the granularity is high and the data is not frequently purged. Alternatively, the database size can be controlled by deleting specific topology objects and any metrics that they contain. This can be done through the Retention Policies dashboard. In addition to associating the default monitoring policies with data storage cycles, this dashboard allows you to inspect the topology objects and to delete them by applying specific retention policies to the collected data, or to purge specific data objects as required. For more information about this dashboard, see Monitor Server Performance.

IMPORTANT: Any changes to the retention policies that you make in the Retention Policies dashboard apply to the collected data to help you adjust the current database size. The retention policies specified in the Retention Policies dashboard continue to apply to any incoming data.

On the Retention Policies dashboard, the Age column specifies the amount of time allotted for data collection. The roll-up period defines the granularity of the collection period. For example, if age is defined as one minute, and the roll-up period is defined as five minutes, any data older than one minute is eligible to be aggregated into the five-minute roll-up period.

CAUTION: The first period in the retention policy specifies the aggregation that is performed before the raw data is persisted. Therefore, the age determines how long raw samples remain in memory before being persisted. In order to constrain the server’s memory usage, the age specified for the first roll up period should not be too large. Settings larger than 30 minutes should be carefully considered.

CAUTION: The roll-up period of the first retention policy period determines the amount of the initial aggregation. If you do not want any aggregation to be performed, this can be set to 0 ms. If not set to zero, this setting should not be too small, as it increases the amount of processing performed by the server. A setting smaller than 30 seconds should be carefully considered.

For metrics, the aggregation retains the count, minimum, maximum, sum, average, and standard deviation of the aggregated values. For other observation types, aggregation is a sampling process that retains the latest value per time slice.

The default roll-up period is 15 minutes; therefore any raw data older than 15 minutes is rolled up to the next period.

If a topology type references the default data storage cycle, its retention policies cannot be modified or deleted. The default data storage cycle can be modified using the Retention Policies dashboard. For more information about this dashboard, see the Foglight User Guide. Any types descending from that type that inherit its retention policies can have their retention policies modified or deleted. This also applies to the descendants that have custom retention policies.

NOTE: Modifying or deleting an inherited policy converts it to a custom policy.

This prevents any accidental modifications of default observation life cycles through the Manage Retention Policies dashboard. For example, the retention policies of the TopologyObject super-type reference the default cycle, and as such, cannot be modified. The TopologyMergeRule type is a descendant of TopologyObject which inherits the retention policies of the ancestor type; those policies can be modified or deleted using the Manage Retention Policies dashboard. Modifying the TopologyMergeRule‘s retention policy, inherited from TopologyObject, creates a custom policy. The TaskManager type, also a descendant of TopologyObject, has a custom retention policy; this policy can be modified or deleted as required. The Manage Retention Policies dashboard indicates whether a type directly references the default storage cycle, inherits policies from another type, or has its own custom policies.

Use the Delete Selected button on the Manage Retention Policies dashboard to delete the retention policy associated with a particular topology object, as outlined below.

If a topology type directly references the default data storage cycle, such as the TopologyObject super-type, its retention policies cannot be deleted. The default data storage cycle can be modified using the Retention Policies dashboard. In addition to associating the default monitoring policies with data storage cycles, this dashboard allows you to inspect the topology objects and to delete them by applying specific retention policies to the collected data, or to purge specific data objects as required. For more information about this dashboard, see the Foglight User Guide.

Any types descending from TopologyObject that inherited its retention policies can have their retention policies deleted. This also applies to the descendants that have custom retention policies.

IMPORTANT: Deleting an inherited policy requires a conversion of that policy and all of its sibling policy to custom policies before it can be deleted.

For example, the retention policies of the TopologyObject super-type reference the default cycle, and as such, cannot be modified. The retention policy of any TopologyObject descendants that inherit its default policy can be deleted after its conversion to a custom policy. Any custom policies are enabled for deletion by default.

TIP: Placing the mouse cursor over the Age or Roll-up Period column indicates if the policy is inherited from another topology type. Any inherited default policies cannot be selected for deletion unless they are converted to custom policies.

TIP: Saving the inherited settings without making any changes to them converts an inherited policy to a custom one.

TIP: To quickly select all periods defined for a property or type, select the check box in the left-most column of the row containing that property or type. This causes all of the periods of the selected property or type to be selected. Inherited policies cannot be selected. An inherited policy can be deleted after its conversion to a custom policy. For more information, see Step 2.

Before you get started with editing retention policies, you need to identify the topology type whose retention policies you want to edit. Editing a retention policy for a topology type, causes its descendants to inherit the newly-edited retention policy.

The Manage Retention Policies dashboard lists all of the available topology types that exists in the database schema and their properties but does not provide information on their position in the schema, such as their ancestors, descendants, or object instances. To identify the descendants of a particular topology type, use the Schema Browser dashboard. In addition to topology type ancestors, the Schema Browser dashboard can show the properties, descendants, and instances for each topology type. For complete information about the Schema Browser dashboard, see the Dashboard Support Guide.

For more information about the Schema Browser, see the Foglight User Guide.

The Manage Retention Policies dashboard allows you to edit existing retention policy periods. If a topology type directly references the default data storage cycle, such as the TopologyObject super-type, its retention policies cannot be modified. The default data storage cycle can be adjusted using the Retention Policies dashboard. In addition to associating the default monitoring policies with data storage cycles, this dashboard allows you to inspect the topology objects and to delete them by applying specific retention policies to the collected data, or to purge specific data objects as required. For more information about this dashboard, see the Foglight User Guide.

Any types descending from TopologyObject that inherited its retention policies can have their retention policies modified. This also applies to the descendants that have custom retention policies.

IMPORTANT: Modifying an inherited policy requires a conversion of that policy and all of its sibling policy to custom policies before it can be modified.

For example, the retention policies of the TopologyObject super-type reference the default cycle, and as such, cannot be modified. The retention policy of any TopologyObject descendants that inherit its default policy can be modified after its conversion to a custom policy. Any custom policies are enabled for edits by default.

TIP: Placing the mouse cursor over the Age or Roll-up Period column indicates if the policy is inherited from another topology type. Inherited default policies cannot be selected unless they are converted to custom policies.

TIP: Saving the inherited settings without making any changes to them converts an inherited policy to a custom one.

TIP: Inherited policies cannot be selected for deletion. An inherited policy can be deleted after its conversion to a custom policy. For more information, see Step 2.

You can create new retention policies for the topology types listed in the table on the Manage Retention Policies dashboard.

The super-type, TopologyObject, has a set of default retention policies that directly reference the default data storage cycle. You cannot create additional policies for this type. The default data storage cycle can be adjusted using the Retention Policies dashboard. In addition to associating default monitoring policies with data storage cycles, this dashboard allows you to inspect the topology objects and delete them by applying specific retention policies to the collected data, or to purge specific data objects as required. For more information about this dashboard, see the Foglight User Guide.

Any types descending from TopologyObject that inherited its retention policies can have their retention policies modified. This also applies to the descendants that have custom retention policies.

IMPORTANT: Adding a new policy to a set of inherited policies for a type or property converts all of the policies of that type or property to custom properties, including the existing inherited policies and the newly-created policy.

For example, the retention policies of the TopologyObject super-type reference the default cycle. You cannot add new retention policies to that topology type. The retention policy of any TopologyObject descendants that inherit its default policy can be modified to include additional policies. However, adding a new policy to a set of inherited default policies converts all of the type’s policies to custom.

NOTE: This check box appears in the Add Retention Policy dialog box only if the selected type includes any inherited retention policies.

This section contains information on how to monitor, estimate, and manage the size of your database.

NOTE: Moving the embedded MySQL or PostgreSQL database outside of the Foglight directory structure is not supported. If you need to move your embedded database, choose one of the following options: • Move the entire Foglight Management Server directory structure to a new location. • Move from the embedded database to an external database.

Foglight server comes with a rule, Catalyst Database Space Checking, which monitors the size of the database and triggers alarms at different severity levels. The following registry variables define the thresholds used by the rule:

You can optionally attach actions to the rule, such as sending out emails to a database administrator.

For more information about the Catalyst Database Space Checking rule, see Catalyst Database Space Checking rule.

Each topology type can be assigned a retention policy, which determines either the granularity at which the metric history data is rolled up or whether the data should be purged after reaching a certain age.

The default retention policy for the root topology type is to roll up to the granularity of 15 minutes after the data is 15 minutes old, to the granularity of one hour after the data is four hours old, and to the granularity of one day after the data is one week old. This policy is inherited by sub-types, unless it is overridden by non-default retention policies.

The retention policy is a useful tool for controlling the database size of the monitoring server. If a particular monitoring domain contains a large number of metrics, it would be helpful to apply a coarse-grained granularity to the retention policy of the top-level topology types, or to purge metric history data after a certain age, if appropriate.

In typical environments, metric history occupies most of the storage space in the Foglight database. However, some Foglight cartridges employ non-metric-based observations that are persisted in different ways than metric-based observations.

The following observation types are persisted in the obs_string_* tables:

Complex observations are persisted in the obs_binary_* tables.

To achieve a more accurate size estimation, you can factor in the space taken by both binary and string observation tables.

For example, if a cartridge has five complex observations and all of them are persisted in five-minute granularity for two weeks, more database space is needed, in addition to metric history.

You can check the typical record size of a table by looking at the database serviceability metrics in the browser interface. Select the Configuration > Data node in the navigation panel, and in the display area, expand the Foglight > All Data > FSMDatabases > <foglight_management_server_machine> > Tables node. You will see a list of sub-nodes that represent the tables used by Foglight.

Select and expand one of the obs_binary_* nodes and you will find several metric nodes. You can calculate the typical record size using the following formula:

(dataLength + indexLength) / numberRows

Assuming that the average record size is 1.1KB, you can get a good idea of how much space the complex observations will take using the following formula:

(1.1KB) * (# of complex observations) * (3600 / (granularity in seconds)) * 24 * (# of days) * (# of hosts)

This gives us the number of MB needed (approximately an additional 220MB) to monitor ten hosts for two weeks.

MySQL with innodb engine does not shrink the table space file, even after you truncate the tables.

NOTE: Substitute dbport, dbuser, dbpwd, and dbname with the correct port number, user name, password, and database name.

NOTE: Substitute dbport, dbuser, and dbpwd the correct port number, user name, password, and database name.

NOTE: For Oracle and Microsoft SQL Server, please ask your database administrator for assistance.

If you are using an external database, you may experience a situation where the database password for the Foglight database account has changed (for example, when password policies change). Use the following method to reconfigure Foglight to start up with the new database password. The password can be encrypted using the keyman command, and then updated in the external database and in the server.config file.

If the administrative password for the embedded database changes, use the same procedure, and update the server.database.embedded.password parameter in the server.config file.

For more information about the keyman command, see the Command-Line Reference Guide.