-
Title
Understanding IntelliProfile and Baselines in Foglight -
Description
Static thresholding methods are limited in flexibility and adaptability. Users often face challenges such as:
-
Determining appropriate threshold values
-
Ensuring thresholds are valid across all times and instances
-
Maintaining and reviewing static thresholds over time
There is a need for a dynamic, data-driven approach to monitoring thresholds based on system behavior.
-
-
Cause
Traditional static or fixed thresholds are not adaptive to varying workloads and usage patterns. They:
-
Do not account for time-of-day or cyclical usage trends
-
Require manual configuration and ongoing maintenance
-
May trigger false positives or miss important deviations
-
-
Resolution
What is Intelliprofile?
IntelliProfile is Foglight’s dynamic thresholding engine. It learns the regular behavior patterns of monitored metrics (e.g., CPU, memory, I/O) and creates a baseline to identify abnormal values.
Key functions include:
-
Pattern recognition: Identifies daily, weekly, and monthly cycles. IntelliProfile assumes that the periodic activities are done daily, weekly, monthly (or other custom defined periods) and then tries to detect the metric's periodic behaviors and use them in order to establish a baseline of the activity.
-
Time segmentation: Breaks down periods (e.g., 24 hours) into smaller intervals. A day is divided into 24 one hour segments and at times, the actual metric activity is much higher what is expected. If increase in activity happens on a regular basis, IntelliProfile detects the activity and adjusts the baseline accordingly.
-
Noise filtering: Ignores short-term anomalies that do not reflect meaningful trends. If increased activity is an exception and happens only once and for a short period of time, IntelliProfile filters it (by detecting it as "noise") by using a noise filtering algorithm so it doesn't affect all of the IntelliProfile results.
-
Percentile calculation: Uses statistical analysis to define threshold levels. IntelliProfile determines which of the time frequencies calculated for the metric apply to the given timeframe. The frequencies are composed into a combined value to provide the following ranges:
-
- Green - normal behavior
- Yellow - slight deviation
- Orange – minor deviation
- Red - major deviation
(In a non-technology context, percentiles are used for example with SAT scores. If the mean SAT score is 1000, then a 95th percentile might be a score of 1540 (out of 1600), a 97 percentile might be 1580, 1590 might be a 99 percentile. Potential colleges would be interested in the top percentiles. These would be outside of the normal range and admissions groups would want to be notified about these outliers).
What is a Baseline?
A baseline represents the normal operating range of a metric based on historical data. For example, if disk usage typically stays around 30% during business hours, that value becomes part of the baseline. Deviations may indicate abnormal behavior or potential issues.
Baseline vs. Utilization Alarms
Type Description Baseline Alarm Triggered when a metric significantly deviates from its expected baseline. Utilization Alarm Triggered when usage exceed a defined limit, regardless of historical behavior. Simplified:- Baseline Alarm: "This metric isn't normal"
- Utilization Alarm: "This metric is being used a lot"
Instance calculated Baseline
- Once monitoring begins, IntelliProfile starts collecting data.
- Requires ~24 hours of data to build a usable baseline.
- Aggregates data hourly.
- Values within 0.00–97th percentile are considered normal.
- Ideal for metrics with low-frequency cycles and high signal-to-noise ratios.
As seen in the chart below, the light blue area is the Baseline minimum and maximum threshold, as aggregated for 24 hours.
The dark blue line represents the Instance workload for that time range.
The basic assumption is that the metric behaves according to certain time based patterns:
- Hour in a day
- Working day vs. non-working day
- Day (or portion of it) in a week
- Day (or portion of it) in a month
The best candidates to use metrics are those metrics that have one or more fairly low frequency repeating cycles with a high signal to noise ratio, such as the baseline range for the active time on a specified database instance.
The baseline ranges for an operating system hosting a database instance are displayed below in the blue line above the metric value bar.
Examples of Baseline Alarms
Intelliprofile within Foglight
Intelliprofile thresholds and values apply for all Agents/Instances and metrics and values influences the entire metrics for of the all Instances configured.
IntelliProfile categorizes thresholds using percentile-based levels:
Intelliprofile level Percentile 1 1% 2 3% 3 5% 4 97% 5 99% 6 99.9% Alarm Colors:
- Green: Levels 1–4 (normal)
- Yellow: Between Levels 4 and 5 (warning)
- Orange: Between Levels 5 and 6 (critical)
Default behavior: Alerts are triggered for values above the 95th percentile.
Intelliprofile Minimum and maximum Thresholds can be changed in levels 4 and 5 by navigating to Administration | Data | Intelliprofile, even though this is NOT advisable. Please note that you must have the appropriate privileges in Foglight to make these changes.
Managing Sensitivity
Baseline alarms may sometimes be too sensitive. To reduce this:
- Set IntelliProfile to Level 6 (99.9 percentile).
- Note: This will impact all agents and metrics, and is not generally recommended unless necessary.
This is however not recommended because it will affect all of the baseline rules for all agents. -
-
Additional Information
Is the algorithm used to estimate the baseline based on exponentially weighted moving average?
No. The IntelliProfile is an algorithm that analyses a metric activity and establishes the normal behavior for that metric. This is done by detecting the different periodic behavior which comprises the metric activity. The analyzed data is then used to calculate the expected activity of the metric (also known as the baseline) for different time periods. IntelliProfile assumes that any collected metric is a result of the activities which are done in certain periods. For example a metric activity is usually a combination of the daily normal work, the batch jobs which are run throughout the nights and the monthly load during certain parts of the month. IntelliProfile is based on the assumption that the periodic activities are daily, weekly, monthly (or constitute other custom defined periods). IntelliProfile tries to detect the metric periodic behaviors and use them in order to establish a baseline of the activity.
Is it based on the assumption of the data is Normally Distributed?
Yes. IntelliProfile detects frequencies (Hourly, daily and monthly time segments) in which a metric values distribution is normal and properly reflects the metric behavior.
What if the data is not Normally Distributed?
IntelliProfile tries to detect frequencies in which the values distribution is close to normal. If metric values pattern is chaotic or cannot be approximated by a normal distribution in any frequency, this impacts the quality of the baseline. This scenario is not likely to happen.
Please refer to KB 431080 for more Questions and Answers.