Foglight for Databases 6.3.0 - Administration and Configuration Guide

Manage Schedules

Foglight allows you to control access to a schedule. For each schedule you can grant or deny read, write, or control access to roles or users. For more information about security concepts in Foglight, see Managing Users and Security.

Foglight employs the following behavior when it comes to schedule permissions:

Use the Edit Permissions button () on the Manage Schedules dashboard to navigate to the Edit Permissions for Schedule area, that allows you to add or edit permissions to roles and users, as outlined below.

TIP: Three check marks in the Permissions columns indicate that the role already has permissions assigned to it.

TIP: Three check marks in the Permissions columns indicate that the role already has permissions assigned to it.

Copying a schedule is useful in situations when you need to quickly create a modified version of an existing schedule. Instead of re-creating all of the schedule settings, simply copy an existing schedule and edit the required schedule items.

When a schedule is deleted, all references to that schedule are removed as well. Any performance calendars that are based on that schedule are removed, and the deleted schedule is removed from the list of effective and blackout schedules for rules.

A schedule contains one or more schedule items, each defining a time period and a recurrence pattern. Viewing schedule definitions allows you to find out the details about the items associated with a schedule.

You can edit an schedule by adding or removing one or more of its recurrence patterns. For example, if you have a schedule that runs indefinitely from 10:00 AM to 11:00 AM daily and on the first day of the month from 8:00 AM to 6:00 PM, but you want to edit it to also run every Saturday from 11:00 AM to 4:00 PM in May, add a schedule item for each of these time spans to the schedule.

Schedules consisting of multiple scheduled items must include a relevant start time. By default, the start time is the day the schedule is created. The start time of each schedule item must be specified to reflect the first run of the scheduled item.

A default Foglight installation includes a number of scheduled, including Always, Business hours, Business week, and many others. Rules, registry variables, and derived metrics can make use of schedules. For example, a registry variable can have multiple values, each associated with a specific schedules. If none of the existing schedules meet your needs, you can create a new schedule and add it to the schedule collection.

A schedule can contain one or more items, each describing a recurrence pattern. For example, if you want a schedule to run indefinitely from 10:00 AM to 11:00 AM daily and on the first day of the month from 8:00 AM to 6:00 PM, and also every Saturday from 11:00 AM to 4:00 PM in May, add a schedule item for each of these time spans to the schedule.

When you create a schedule, you have to specify at least one schedule item and its recurrence pattern. You can edit the schedule at a later time by adding or removing schedule items.

There are six types of patterns that you can define in a schedule item.

Table 41. Pattern types you can use in a schedule item.

Pattern Name	Pattern Description	For details, see…
Once	Starts at a specified date and time, for a specified duration, and ends at a defined end date and time	To define a schedule item that occurs once:
Periodical	Starts at a specified time and date for a certain duration, repeats at specified time periods, with or without a defined end date and time	To define a schedule item that occurs periodically:
Daily	Starts at a specified time and date, runs for a whole day or a fraction of a day, repeats at a regular interval of days, with or without a defined end date and time	To define a schedule item that occurs daily:
Weekly	Starts at a specified time and date, runs for a whole day or a fraction of a day, repeats at a regular interval of weeks on one or more days of the week, with or without a defined end date and time	To define a schedule item that occurs weekly:
Monthly	Starts at a specified time and date, runs for a whole day or a fraction of a day, repeats at a regular interval of months on one or more days of the month, with or without a defined end date and time	To define a schedule item that occurs monthly:
Yearly	Starts at a specified time and date, runs for a whole day or a fraction of a day, repeats at a regular interval of years on one or more days of the year, with or without a defined end date and time	To define a schedule item that occurs yearly:

IMPORTANT: Schedules consisting of multiple scheduled items must include a relevant start time. By default, the start time is the day the schedule is created. The start time of each schedule item must be specified to reflect the first run of the scheduled item.

TIP: The Day box accepts the values between 1 and 31. Any positive values outside that range are automatically adjusted to 31; negative values are not accepted.

TIP: The Hour box accepts the values between 0 and 23. Any positive values outside that range are automatically adjusted to 23; negative values are not accepted. The Minute box accepts the values between 0 and 59. Any positive values outside that range are automatically adjusted to 59; negative values are not accepted.

NOTE: In the Range of Occurrence area, the No End option appears disabled while the End By Date option is enabled and selected. This is because a schedule item that occurs once must have an end date.

TIP: The Day box accepts the values between 1 and 31. Any positive values outside that range are automatically adjusted to 31; negative values are not accepted.

TIP: The Hour box accepts the values between 0 and 24. Any positive values outside that range are automatically adjusted to 24; negative values are not accepted. The Minute box accepts the values between 0 and 59. Any positive values outside that range are automatically adjusted to 59; negative values are not accepted.

TIP: The Day box accepts the values between 1 and 31. Any positive values outside that range are automatically adjusted to 31; negative values are not accepted.

TIP: The Hour box accepts any positive values. The Minute box accepts the values between 0 and 59. Any positive values outside that range are automatically adjusted to 59; negative values are not accepted.

TIP: The Hour box accepts any positive values. The Minute box accepts the values between 0 and 59. Any positive values outside that range are automatically adjusted to 59; negative values are not accepted.

IMPORTANT: The recurrence period must be longer than the duration specified in <Link>step 2. For example, if the duration of the schedule item is three hours, the recurrence periods should occur at intervals that are longer than three hours.

TIP: The Hour box accepts any positive values. The Minute box accepts the values between 0 and 59. Any positive values outside that range are automatically adjusted to 59; negative values are not accepted.

TIP: Use the Calendar button on the right and use the calendar controls that appear to specify the start date.

TIP: The Day box accepts the values between 1 and 31. Any positive values outside that range are automatically adjusted to 31; negative values are not accepted. The Hour box accepts the values between 0 and 24. Any positive values outside that range are automatically adjusted to 23; negative values are not accepted. The Minute box accepts the values between 0 and 59. Any positive values outside that range are automatically adjusted to 59; negative values are not accepted.

TIP: The Day box accepts the values between 1 and 31. Any positive values outside that range are automatically adjusted to 31; negative values are not accepted.

• Specify the start time and end time of the schedule item.
	• Use the Start Time [hh:mm] area to specify the hour and minute of the start time. NOTE: The Hour box accepts the values between 0 and 23. Any positive values outside that range are automatically adjusted to 23; negative values are not accepted. The Minute box accepts the values between 0 and 59. Any positive values outside that range are automatically adjusted to 59; negative values are not accepted.
	• Select End Time [hh:mm] and specify the hour and minute of the end time. NOTE: The Duration [hh:mm] boxes appear disabled when you specify the End Time [hh:mm] option. The end time should occur after the start time. NOTE: The Hour box accepts the values between 0 and 24. Any positive values outside that range are automatically adjusted to 24; negative values are not accepted. The Minute box accepts the values between 0 and 59. Any positive values outside that range are automatically adjusted to 59; negative values are not accepted.
or
• Specify the start time and duration of the schedule item.
	• Use the Start Time [hh:mm] area to specify the hour and minute of the start time. NOTE: The Hour box accepts the values between 0 and 23. Any positive values outside that range are automatically adjusted to 23; negative values are not accepted. The Minute box accepts the values between 0 and 59. Any positive values outside that range are automatically adjusted to 59; negative values are not accepted.
	• Select Duration [hh:mm] and specify the hour and minute of the duration time. NOTE: The End Time [hh:mm] boxes appear disabled when you specify the Duration [hh:mm] option. NOTE: The Hour box accepts the values between 0 and 24. Any positive values outside that range are automatically adjusted to 24; negative values are not accepted. The Minute box accepts the values between 0 and 59. Any positive values outside that range are automatically adjusted to 59; negative values are not accepted.

TIP: The Every box accepts any positive values.

TIP: Use the Calendar button on the right and use the calendar controls that appear to specify the start date.

TIP: The Day box accepts the values between 1 and 31. Any positive values outside that range are automatically adjusted to 31; negative values are not accepted. The Hour box accepts the values between 0 and 24. Any positive values outside that range are automatically adjusted to 23; negative values are not accepted. The Minute box accepts the values between 0 and 59. Any positive values outside that range are automatically adjusted to 59; negative values are not accepted.

TIP: The Day box accepts the values between 1 and 31. Any positive values outside that range are automatically adjusted to 31; negative values are not accepted.

Table 42. Select one of the following steps to create a schedule for part of the day.

• Specify the start time and end time of the schedule item.
	• Use the Start Time [hh:mm] area to specify the hour and minute of the start time. NOTE: The Hour box accepts the values between 0 and 23. Any positive values outside that range are automatically adjusted to 23; negative values are not accepted. The Minute box accepts the values between 0 and 59. Any positive values outside that range are automatically adjusted to 59; negative values are not accepted.
	• Select the End Time [hh:mm] area and specify the hour and minute of the end time. NOTE: The Duration [hh:mm] boxes appear disabled when you specify the End Time [hh:mm] option. The end time should occur after the start time. NOTE: The Hour box accepts the values between 0 and 24. Any positive values outside that range are automatically adjusted to 24; negative values are not accepted. The Minute box accepts the values between 0 and 59. Any positive values outside that range are automatically adjusted to 59; negative values are not accepted.
or
• Specify the start time and duration of the schedule item.
	• Use the Start Time [hh:mm] area to specify the hour and minute of the start time. NOTE: The Hour box accepts the values between 0 and 23. Any positive values outside that range are automatically adjusted to 23; negative values are not accepted. The Minute box accepts the values between 0 and 59. Any positive values outside that range are automatically adjusted to 59; negative values are not accepted.
	• Select Duration [hh:mm] and specify the hour and minute of the duration time. NOTE: The End Time [hh:mm] boxes appear disabled when you specify the Duration [hh:mm] option. NOTE: The Hour box accepts the values between 0 and 24. Any positive values outside that range are automatically adjusted to 24; negative values are not accepted. The Minute box accepts the values between 0 and 59. Any positive values outside that range are automatically adjusted to 59; negative values are not accepted.

TIP: The Every box accepts any positive values.

TIP: Use the Calendar button on the right and use the calendar controls that appear to specify the start date.

TIP: The Day box accepts the values between 1 and 31. Any positive values outside that range are automatically adjusted to 31; negative values are not accepted. The Hour box accepts the values between 0 and 24. Any positive values outside that range are automatically adjusted to 23; negative values are not accepted. The Minute box accepts the values between 0 and 59. Any positive values outside that range are automatically adjusted to 59; negative values are not accepted.

TIP: The Day box accepts the values between 1 and 31. Any positive values outside that range are automatically adjusted to 31; negative values are not accepted.

• Specify the start time and end time of the schedule item.
	• Use the Start Time [hh:mm] area to specify the hour and minute of the start time. NOTE: The Hour box accepts the values between 0 and 23. Any positive values outside that range are automatically adjusted to 23; negative values are not accepted. The Minute box accepts the values between 0 and 59. Any positive values outside that range are automatically adjusted to 59; negative values are not accepted.
	• Select End Time [hh:mm] and specify the hour and minute of the end time. NOTE: The Duration [hh:mm] boxes appear disabled when you specify the End Time [hh:mm] option. The end time should occur after the start time. NOTE: The Hour box accepts the values between 0 and 24. Any positive values outside that range are automatically adjusted to 24; negative values are not accepted. The Minute box accepts the values between 0 and 59. Any positive values outside that range are automatically adjusted to 59; negative values are not accepted.
or
• Specify the start time and duration of the schedule item.
	• Use the Start Time [hh:mm] area to specify the hour and minute of the start time. NOTE: The Hour box accepts the values between 0 and 23. Any positive values outside that range are automatically adjusted to 23; negative values are not accepted. The Minute box accepts the values between 0 and 59. Any positive values outside that range are automatically adjusted to 59; negative values are not accepted.
	• Select the Duration [hh:mm] area and specify the hour and minute of the duration time. NOTE: The End Time [hh:mm] boxes appear disabled when you specify the Duration [hh:mm] option. NOTE: The Hour box accepts the values between 0 and 24. Any positive values outside that range are automatically adjusted to 24; negative values are not accepted. The Minute box accepts the values between 0 and 59. Any positive values outside that range are automatically adjusted to 59; negative values are not accepted.

TIP: The Day box accepts the values between 1 and 31. Any positive values outside that range are automatically adjusted to 31; negative values are not accepted. The every box accepts any positive values.

TIP: The Every box accepts any positive values.

TIP: The Every box accepts any positive values.

TIP: Use the Calendar button on the right and use the calendar controls that appear to specify the start date.

TIP: The Day box accepts the values between 1 and 31. Any positive values outside that range are automatically adjusted to 31; negative values are not accepted. The Hour box accepts the values between 0 and 24. Any positive values outside that range are automatically adjusted to 23; negative values are not accepted. The Minute box accepts the values between 0 and 59. Any positive values outside that range are automatically adjusted to 59; negative values are not accepted.

TIP: The Day box accepts the values between 1 and 31. Any positive values outside that range are automatically adjusted to 31; negative values are not accepted.

• Specify the start time and end time of the schedule item.
	• Use the Start Time [hh:mm] area to specify the hour and minute of the start time. NOTE: The Hour box accepts the values between 0 and 23. Any positive values outside that range are automatically adjusted to 23; negative values are not accepted. The Minute box accepts the values between 0 and 59. Any positive values outside that range are automatically adjusted to 59; negative values are not accepted.
	• Select End Time [hh:mm] and specify the hour and minute of the end time. NOTE: The Duration [hh:mm] boxes appear disabled when you specify the End Time [hh:mm] option. The end time should occur after the start time. NOTE: The Hour box accepts the values between 0 and 24. Any positive values outside that range are automatically adjusted to 24; negative values are not accepted. The Minute box accepts the values between 0 and 59. Any positive values outside that range are automatically adjusted to 59; negative values are not accepted.
or
• Specify the start time and duration of the schedule item.
	• Use the Start Time [hh:mm] area to specify the hour and minute of the start time. NOTE: The Hour box accepts the values between 0 and 23. Any positive values outside that range are automatically adjusted to 23; negative values are not accepted. The Minute box accepts the values between 0 and 59. Any positive values outside that range are automatically adjusted to 59; negative values are not accepted.
	• Select Duration [hh:mm] and specify the hour and minute of the duration time. NOTE: The End Time [hh:mm] boxes appear disabled when you specify the Duration [hh:mm] option. NOTE: The Hour box accepts the values between 0 and 24. Any positive values outside that range are automatically adjusted to 24; negative values are not accepted. The Minute box accepts the values between 0 and 59. Any positive values outside that range are automatically adjusted to 59; negative values are not accepted.

TIP: The Day box accepts the values between 1 and 31. Any positive values outside that range are automatically adjusted to 31; negative values are not accepted.

TIP: Use the Calendar button on the right and use the calendar controls that appear to specify the start date.

TIP: The Day box accepts the values between 1 and 31. Any positive values outside that range are automatically adjusted to 31; negative values are not accepted. The Hour box accepts the values between 0 and 24. Any positive values outside that range are automatically adjusted to 23; negative values are not accepted. The Minute box accepts the values between 0 and 59. Any positive values outside that range are automatically adjusted to 59; negative values are not accepted.

Manage Retention Policies

While it is theoretically possible to create any retention policy that you desire in Foglight, the design of the system constrains the easy-to-accomplish retention policies to a narrow range of options. Specifically, the database design of Foglight provides a structure that is capable of holding data in three different buckets, called generations, that are defined in <Foglight_home>/config/storage-config.xml. Each generation has a predefined period of time in which it retains data.

Without modifying database generations, there are specific rules that you must follow when assigning retention policies, to ensure that you achieve the expected results. Any changes to the database generations in <Foglight_home>/config/storage-config.xml require a Management Server restart, in order for these changes to take effect.

This provides information on the key mechanisms involved in retention policies and rules for defining retention policies that work effectively with the default database configuration.

Generations refer to the database structures that hold long-term data. For any given metric, each generation can hold one aggregation level of data (for example, raw, hourly averages, 4 hour averages, and so on). Out of the box, there are three generations:

Because the collected data is constrained to the above aggregation periods, retention policies are also constrained to a set of rules. In general, you can create retention policies that:

The data service periodically writes data from the short-term memory cache to Generation 1. The frequency in which data is written is defined in the first retention policy (for more information, see How retention policies interact with database generations). This interval should not exceed 15 minutes to prevent the Foglight Management Server memory from growing too large.

A nightly roll-up job aggregates data and writes that data to Generation 2 and Generation 3. The roll-up is only done once a day, according to the time set in the Daily Database Maintenance schedule. For more information about this and other schedules in Foglight, see Associate Metric Calculations with Schedules.

Both mechanisms for populating the repository (from memory to the database, database roll-ups) use the retention policies defined in the Retention Policies dashboard as the guidelines for how they store data.

A retention policy is the set of definitions, for a given object, that indicate how data is stored. Each definition within a policy contains two parameters:

Policies can be set at an object level; however, retention policies also adhere to the object inheritance capabilities. If a policy has not been explicitly assigned to an object, it inherits a value from a higher level in the model. The top-level object is TopologyObject.

The policy that is applied to TopologyObject, and therefore any object which does not have explicitly assigned policies, is as follows:

Table 43. Policy, duration, and meaning.

Policy		Translates to
Age	15 minutes	After 15 minutes, store 15 minute aggregate data to Generation 1 (where they are stored for three days)
Roll-up	15 minutes
Age	3 days	Data older than 3 days is eligible for roll-up to one-hour averages and persisted into Generation 2 (where they are stored for 14 days)
Roll-up	1 hour
Age	14 days	Data older than 14 days old is eligible for Roll-up to four-hour averages and persisted into Generation 3 indefinitely
Roll-up	4 hours

NOTE: Individual cartridges frequently have their own policies which must be examined on individual object level to understand that retention policy behavior.

As indicated in the above diagram, any data whose retention policies include the purge settings is stored in the Generation 3 aggregation, from which it is purged in accordance with the purge settings. Purging from the last generation is done by recreating the tables to filter out the observations that are to be purged, and it occurs once a month for each configured purge setting. For example, if there are n policies that request a purge after 12 months and m policies that request a purge after 18 months, then at the start of the month, Foglight performs a single purge on the 12-month tables and a single purge on the 18-month tables. If your business requirements dictate that any data older than x months should be purged from the database, the most efficient implementation is to edit the Generation 3 setting in <Foglight_home>/config/storage-config.xml, whose default length is set to one hundred years, to x months. For example, the following code block illustrate a Generation 3 setting in storage-config.xml that dictates the purge of any data that is older than six months:

While the browser interface does not prevent you from setting policies that are in conflict with the generations, setting policies that are outside of these boundaries does not yield the expected results. Instead, the retention policy engine finds the most optimal scheme for your data (ensuring that the lowest granularity is written to Generation 1 and that longest duration data are written to Generation 3).

The table shows how to configure retention policies, at 1, 2 or 3 levels of aggregation, following the specifications below.

Table 44. How to configure retention policies.

	Acceptable age values	Acceptable roll-up values	Explanation
Three-level policy (including a purge)
Level 1	<= 15 minutes	<= 15 minutes	Data is persisted at the roll-up interval defined in the Level 1 policy for three days.
Level 2	> 15 minutes and < 3 days	Any roll-up greater than Level 1	The age date for the Level 2 policy must be less than or equal to three days. Data is persisted at the roll-up interval defined in the Level 2 policy for 14 days.
Level 3	> Level 2 setting and < 14 days	Any roll-up greater than Level 2	The age date for the Level 3 policy must be less than or equal to 14 days. Data is persisted at this roll-up interval indefinitely. A purge policy defines a minimum length of time that data must persist before it is truncated.
Purge Policy	> Level 3 setting	Purge	Data is never purged from the system before the age value of the purge policy. Data may, however, be retained for longer than the setting as the system waits to find an acceptable time to purge data.
Two-level policy (including a purge)
Level 1	<= 15 minutes	<= 15 minutes	Data is persisted at the roll-up interval defined in the Level 1 policy for either three or 14 days, depending on the age of the Level 2 setting. If the age of the Level 2 setting is less than or equal to three days, then the data is persisted for three days. If the age of the Level 2 setting is between three and 14 days, the data is persisted for 14 days.
Level 2	<= 14 days	Any roll-up greater than Level 1	The age date for the Level 2 policy must be less than or equal to 14 days. Data is persisted at this roll-up interval indefinitely. A purge policy defines a minimum length of time that data must persist before it is truncated.
Purge Policy	> Level 2 setting	Purge	Data is never purged from the system before the age value of the purge policy. Data may, however, be retained for longer than the setting as the system waits to find an acceptable time to purge data.
One-level policy (including a purge)
Level 1	<= 15 minutes	<= 15 minutes	Data is persisted at the roll-up interval defined in the Level 1 policy indefinitely. A purge policy will define a minimum length of time that data must persist before it is truncated.
Purge Policy	> 15 minutes	Purge	Data is never purged from the system before the age value of the purge policy. Data may, however, be retained for longer than the setting as the system waits to find an acceptable time to purge data.

Use the Manage Retention Policies dashboard to view, edit, and create retention policies for topology types and properties of topology types. Each policy specifies one or more time periods after which the data is rolled up and the granularity of the roll-up.

In some cases your retention policies may cause an unacceptable increase in the database size, which typically happens if the granularity is high and the data is not frequently purged. Alternatively, the database size can be controlled by deleting specific topology objects and any metrics that they contain. This can be done through the Retention Policies dashboard. In addition to associating the default monitoring policies with data storage cycles, this dashboard allows you to inspect the topology objects and to delete them by applying specific retention policies to the collected data, or to purge specific data objects as required. For more information about this dashboard, see Monitor Server Performance.

IMPORTANT: Any changes to the retention policies that you make in the Retention Policies dashboard apply to the collected data to help you adjust the current database size. The retention policies specified in the Retention Policies dashboard continue to apply to any incoming data.

On the Retention Policies dashboard, the Age column specifies the amount of time allotted for data collection. The roll-up period defines the granularity of the collection period. For example, if age is defined as one minute, and the roll-up period is defined as five minutes, any data older than one minute is eligible to be aggregated into the five-minute roll-up period.

CAUTION: The first period in the retention policy specifies the aggregation that is performed before the raw data is persisted. Therefore, the age determines how long raw samples remain in memory before being persisted. In order to constrain the server’s memory usage, the age specified for the first roll up period should not be too large. Settings larger than 30 minutes should be carefully considered.

CAUTION: The roll-up period of the first retention policy period determines the amount of the initial aggregation. If you do not want any aggregation to be performed, this can be set to 0 ms. If not set to zero, this setting should not be too small, as it increases the amount of processing performed by the server. A setting smaller than 30 seconds should be carefully considered.

For metrics, the aggregation retains the count, minimum, maximum, sum, average, and standard deviation of the aggregated values. For other observation types, aggregation is a sampling process that retains the latest value per time slice.

The default roll-up period is 15 minutes; therefore any raw data older than 15 minutes is rolled up to the next period.

If a topology type references the default data storage cycle, its retention policies cannot be modified or deleted. The default data storage cycle can be modified using the Retention Policies dashboard. For more information about this dashboard, see the Foglight User Guide. Any types descending from that type that inherit its retention policies can have their retention policies modified or deleted. This also applies to the descendants that have custom retention policies.

NOTE: Modifying or deleting an inherited policy converts it to a custom policy.

This prevents any accidental modifications of default observation life cycles through the Manage Retention Policies dashboard. For example, the retention policies of the TopologyObject super-type reference the default cycle, and as such, cannot be modified. The TopologyMergeRule type is a descendant of TopologyObject which inherits the retention policies of the ancestor type; those policies can be modified or deleted using the Manage Retention Policies dashboard. Modifying the TopologyMergeRule‘s retention policy, inherited from TopologyObject, creates a custom policy. The TaskManager type, also a descendant of TopologyObject, has a custom retention policy; this policy can be modified or deleted as required. The Manage Retention Policies dashboard indicates whether a type directly references the default storage cycle, inherits policies from another type, or has its own custom policies.

Use the Delete Selected button on the Manage Retention Policies dashboard to delete the retention policy associated with a particular topology object, as outlined below.

If a topology type directly references the default data storage cycle, such as the TopologyObject super-type, its retention policies cannot be deleted. The default data storage cycle can be modified using the Retention Policies dashboard. In addition to associating the default monitoring policies with data storage cycles, this dashboard allows you to inspect the topology objects and to delete them by applying specific retention policies to the collected data, or to purge specific data objects as required. For more information about this dashboard, see the Foglight User Guide.

Any types descending from TopologyObject that inherited its retention policies can have their retention policies deleted. This also applies to the descendants that have custom retention policies.

IMPORTANT: Deleting an inherited policy requires a conversion of that policy and all of its sibling policy to custom policies before it can be deleted.

For example, the retention policies of the TopologyObject super-type reference the default cycle, and as such, cannot be modified. The retention policy of any TopologyObject descendants that inherit its default policy can be deleted after its conversion to a custom policy. Any custom policies are enabled for deletion by default.

TIP: Placing the mouse cursor over the Age or Roll-up Period column indicates if the policy is inherited from another topology type. Any inherited default policies cannot be selected for deletion unless they are converted to custom policies.

TIP: Saving the inherited settings without making any changes to them converts an inherited policy to a custom one.

TIP: To quickly select all periods defined for a property or type, select the check box in the left-most column of the row containing that property or type. This causes all of the periods of the selected property or type to be selected. Inherited policies cannot be selected. An inherited policy can be deleted after its conversion to a custom policy. For more information, see Step 2.

Before you get started with editing retention policies, you need to identify the topology type whose retention policies you want to edit. Editing a retention policy for a topology type, causes its descendants to inherit the newly-edited retention policy.

The Manage Retention Policies dashboard lists all of the available topology types that exists in the database schema and their properties but does not provide information on their position in the schema, such as their ancestors, descendants, or object instances. To identify the descendants of a particular topology type, use the Schema Browser dashboard. In addition to topology type ancestors, the Schema Browser dashboard can show the properties, descendants, and instances for each topology type. For complete information about the Schema Browser dashboard, see the Dashboard Support Guide.

For more information about the Schema Browser, see the Foglight User Guide.

The Manage Retention Policies dashboard allows you to edit existing retention policy periods. If a topology type directly references the default data storage cycle, such as the TopologyObject super-type, its retention policies cannot be modified. The default data storage cycle can be adjusted using the Retention Policies dashboard. In addition to associating the default monitoring policies with data storage cycles, this dashboard allows you to inspect the topology objects and to delete them by applying specific retention policies to the collected data, or to purge specific data objects as required. For more information about this dashboard, see the Foglight User Guide.

Any types descending from TopologyObject that inherited its retention policies can have their retention policies modified. This also applies to the descendants that have custom retention policies.

IMPORTANT: Modifying an inherited policy requires a conversion of that policy and all of its sibling policy to custom policies before it can be modified.

For example, the retention policies of the TopologyObject super-type reference the default cycle, and as such, cannot be modified. The retention policy of any TopologyObject descendants that inherit its default policy can be modified after its conversion to a custom policy. Any custom policies are enabled for edits by default.

TIP: Placing the mouse cursor over the Age or Roll-up Period column indicates if the policy is inherited from another topology type. Inherited default policies cannot be selected unless they are converted to custom policies.

TIP: Saving the inherited settings without making any changes to them converts an inherited policy to a custom one.

TIP: Inherited policies cannot be selected for deletion. An inherited policy can be deleted after its conversion to a custom policy. For more information, see Step 2.

You can create new retention policies for the topology types listed in the table on the Manage Retention Policies dashboard.

The super-type, TopologyObject, has a set of default retention policies that directly reference the default data storage cycle. You cannot create additional policies for this type. The default data storage cycle can be adjusted using the Retention Policies dashboard. In addition to associating default monitoring policies with data storage cycles, this dashboard allows you to inspect the topology objects and delete them by applying specific retention policies to the collected data, or to purge specific data objects as required. For more information about this dashboard, see the Foglight User Guide.

Any types descending from TopologyObject that inherited its retention policies can have their retention policies modified. This also applies to the descendants that have custom retention policies.

IMPORTANT: Adding a new policy to a set of inherited policies for a type or property converts all of the policies of that type or property to custom properties, including the existing inherited policies and the newly-created policy.

For example, the retention policies of the TopologyObject super-type reference the default cycle. You cannot add new retention policies to that topology type. The retention policy of any TopologyObject descendants that inherit its default policy can be modified to include additional policies. However, adding a new policy to a set of inherited default policies converts all of the type’s policies to custom.

NOTE: This check box appears in the Add Retention Policy dialog box only if the selected type includes any inherited retention policies.

This section contains information on how to monitor, estimate, and manage the size of your database.

NOTE: Moving the embedded MySQL or PostgreSQL database outside of the Foglight directory structure is not supported. If you need to move your embedded database, choose one of the following options: • Move the entire Foglight Management Server directory structure to a new location. • Move from the embedded database to an external database.

Foglight server comes with a rule, Catalyst Database Space Checking, which monitors the size of the database and triggers alarms at different severity levels. The following registry variables define the thresholds used by the rule:

You can optionally attach actions to the rule, such as sending out emails to a database administrator.

For more information about the Catalyst Database Space Checking rule, see Catalyst Database Space Checking rule.

Each topology type can be assigned a retention policy, which determines either the granularity at which the metric history data is rolled up or whether the data should be purged after reaching a certain age.

The default retention policy for the root topology type is to roll up to the granularity of 15 minutes after the data is 15 minutes old, to the granularity of one hour after the data is four hours old, and to the granularity of one day after the data is one week old. This policy is inherited by sub-types, unless it is overridden by non-default retention policies.

The retention policy is a useful tool for controlling the database size of the monitoring server. If a particular monitoring domain contains a large number of metrics, it would be helpful to apply a coarse-grained granularity to the retention policy of the top-level topology types, or to purge metric history data after a certain age, if appropriate.

In typical environments, metric history occupies most of the storage space in the Foglight database. However, some Foglight cartridges employ non-metric-based observations that are persisted in different ways than metric-based observations.

The following observation types are persisted in the obs_string_* tables:

Complex observations are persisted in the obs_binary_* tables.

To achieve a more accurate size estimation, you can factor in the space taken by both binary and string observation tables.

For example, if a cartridge has five complex observations and all of them are persisted in five-minute granularity for two weeks, more database space is needed, in addition to metric history.

You can check the typical record size of a table by looking at the database serviceability metrics in the browser interface. Select the Configuration > Data node in the navigation panel, and in the display area, expand the Foglight > All Data > FSMDatabases > <foglight_management_server_machine> > Tables node. You will see a list of sub-nodes that represent the tables used by Foglight.

Select and expand one of the obs_binary_* nodes and you will find several metric nodes. You can calculate the typical record size using the following formula:

(dataLength + indexLength) / numberRows

Assuming that the average record size is 1.1KB, you can get a good idea of how much space the complex observations will take using the following formula:

(1.1KB) * (# of complex observations) * (3600 / (granularity in seconds)) * 24 * (# of days) * (# of hosts)

This gives us the number of MB needed (approximately an additional 220MB) to monitor ten hosts for two weeks.

MySQL with innodb engine does not shrink the table space file, even after you truncate the tables.

NOTE: Substitute dbport, dbuser, dbpwd, and dbname with the correct port number, user name, password, and database name.

NOTE: Substitute dbport, dbuser, and dbpwd the correct port number, user name, password, and database name.

NOTE: For Oracle and Microsoft SQL Server, please ask your database administrator for assistance.

If you are using an external database, you may experience a situation where the database password for the Foglight database account has changed (for example, when password policies change). Use the following method to reconfigure Foglight to start up with the new database password. The password can be encrypted using the keyman command, and then updated in the external database and in the server.config file.

If the administrative password for the embedded database changes, use the same procedure, and update the server.database.embedded.password parameter in the server.config file.

For more information about the keyman command, see the Command-Line Reference Guide.

Add Topology Types

Foglight transforms monitoring data into models. A model is a set of objects and relationships designed to represent a monitored resource and its parts. Topology describes the logical and physical relationships between data nodes in a model. At run-time, Foglight dynamically builds topology models based on data about your system that is collected by Foglight agents. Topology models provide the context for the metrics sent by the agents to the Foglight Management Server. The set of topology types that exist in your environment depends on your monitoring needs, reflected in the type and nature of cartridges that you use for data collection. If you need additional topology types, you can add them to Foglight using the Add Topology Type dashboard.

Use the XML syntax when defining a topology type. For example:

IMPORTANT: Topology types can only be removed by uninstalling the cartridge that contains the types that you want to delete.

The Add Topology Type dashboard allows you to add new topology types to your topology model and to validate those types. Validating a topology type ensures that the format of the type definition is valid.

NOTE: Use the back slash character ‘\’ as a directory separator on Windows platforms; on Unix platforms, use the forward slash ‘/’.

NOTE: Use the back slash character ‘\’ as a directory separator.

NOTE: The validation process is limited. It validates the format of the type definition. Your XML may appear to be valid and still fail on import due to other errors such as naming conventions.

 

Online-Only Topics

Learn more about: