In FxV we are seeing many 599 status hits. Why is this happening? (38054)

In Foglight Experience Viewer (FxV) we are seeing many 599 status hits. Why is this happening? The FxV and Foglight Experience Monitor (FxM) health checks are good and the systems seem to be running fine.

The error / response code of 599 is _not_ an http response code. For FxV to say "we did not get this" is, strictly speaking, fictitious. The 599 code is a creation of Quest to describe a situation that is not currently covered an standard HTTP error codes; this is an add-on code to allow users to recognize potential issues in their environment. Please see below for further details.

This response code of 599 indicates of missing response. In other words, FxV saw the request but did not see a response. It can be:

1. Something internal to the FxV system
or
2. External connection resets and external timeouts.

The major cause of 599 errors are Timeouts and ConnectionResets.

The Collector (aka c7r) keeps metrics for these types of errors. To see what is really the underlying cause of the 599 compare the number of Timeouts and ConnectionResets from the Collector metrics to the number of 599 errors you are seeing. If they are roughly the same, that's the source of the 599s.

1. Login to the FxV web console UI
2. Go to 'Configure' | 'Superuser Tasks' | 'Metrics' | 'View Metrics' | 'Archiver' tab | 'Hit Summary' ( choosing 'Default Archiver' in the Archiver drop down )

3. For the 'Connection Resets', and 'Timeouts' take a look at those metrics.