Delay For Alerting on DBSS Usability Connection Availability Can Be Up To 15 Minutes Or More (4307642)

When a host is simply "offline" (powered off, unplugged, or unreachable), the agent doesn't get an instant "unavailable" response.  Instead, the agent's connection attempts can hang, waiting for the network (or operating system) to time out. For usability metrics, the agent waits for every connection attempt to fully time out before reporting failure.

 

• Each agent connection attempt can hang, waiting for the OS/network layer (or database driver) to finally admit that the server is unreachable and time out.
• The usability collections (like `DBSS_Usability`) only report failure after each slow connection attempt is finished (often governed by the underlying 'socket read timeout' and not always by agent config, e.g., 900 seconds / 15 minutes if default or legacy).
• While any collection is “hung,” new usability collection jobs cannot start. This means multiple failed attempts (for metrics, change tracking, performance snapshots, etc.) can accumulate in a queue, and all must individually time out—each adding more delay.
• The usability alarm only triggers after the agent finally recognizes the collection has failed, not when the outage actually began.

Other collections and their alarms (such as "Collection Status") typically fail faster, because their timeouts are enforced at the scheduler/collection system level (for example, “abort after 1 minute”), which is separate from usability/test connection logic.

 

 

Time	Event
17:31:11	DB Connect Availability = TRUE
17:31:17	OS Connect Availability = TRUE
17:34:41	VM powered off (host becomes unreachable)
17:34:41–17:54:59	Multiple collections queued/hung due to network timeouts, no usable checks finish
17:54:59	Usability collection finally starts
17:58:29	Usability collection fails after 3.5 min, alarm triggers

• Reduce the network/socket timeout settings (listed below) in the agent configuration
• This causes the agent to give up trying to connect much sooner after actual outage so that alarms are sent quickly.

However:

• Setting network timeouts too lock might cause false alarms if your network is sometimes slow or has brief hiccups.
• Always test your timeout values in a staging environment before rolling out the changes to production.

 

 

Setitng Name	Recommended	Legacy/Slow
The amount of time to wait for network activity before timing out (seconds)	30 seconds to 60 seconds	900 seconds (15 minutes)
Connection Timeout (milliseconds) (under OS-Monitored Connection Profile)	10 seconds to 30 seconds	120 seconds
DB Connection Timeout (milliseconds)	10 seconds to 30 seconds	60 seconds
Usability Collection Timeout	60 seconds	60 seconds

 

 

 

• This design prevents the agent  from declaring that a database is down just because there's a small delay or a network is busy. It "waits" to be sure the server is truly unreachable.
• But, it meants that for hard outages (like when a server is powered off), the alarm is delayed by the timeout period plus any queue/lock wait in the agent.

 

Why do other alarms fire sooner?

• Other collection status alarms may fire sooner because the scheduler can forcibly abort these tasks.
• The usability alarm waits as long as the operating system or network forces it. Sometimes it is even longer than the timeout values if the OS is slow in releasing failed connections.

 

• When you are tuning timeouts, always balance alarm speed with network reliability.
• Test any changes with both reachable and unreachable hosts to verify the alarm behavior.
• Upgrade your documentation and your users to know how these timeouts affect the alerting.

 

Defect FOG-13087 has been logged to add a uability watchdog timer to future release of the SQL Server cartridge. This will be reviewed Product Management for consideration in future release of the SQL Server cartridge.