-
Title
Foglight not accessible after backend PostgreSQL database cluster fail-over -
Description
FMS repository is configured as a 3 node PostgreSQL cluster with 1 active node (read-write) and 2 standby (read-only) nodes. Foglight is configured to connect to a VIP address that always directs to the primary node.
After a fail-over of the backend database it is not possible to access the Foglight console. The following messages are logged in the Management Server logs:
YYYY-MM-DD hh:mm:ss.SSS ERROR [DataAccess-9-thread-4] com.quest.nitro.service.persistence.obs.handler.query.AbstractBatchQueryExecutor - An error has occurred while processing the MetricBatchRetrieveLastNValues batch query.
org.postgresql.util.PSQLException: ERROR: cannot execute CREATE TABLE in a read-only transactionEstablished connections indicate that Foglight continues to access the previous primary node. -
Cause
Previous active node now read-only but it is still reachable by Foglight. -
Resolution
WORKAROUND
Note: Using a clustered PostgreSQL configuration as a FMS repository has not been validated as supported by Quest R&D.
1. Modify the database connection string parameter in '$FMS_HOME/config/server.config' to:
server.database.url = "jdbc:postgresql://node1,node2,node3/databaseName?targetServerType=primary";
2. Add the following to '$FMS_HOME/config/server.config' to reduce the time for the JVM DNS cache:
server.vm.optionX = "-Dsun.net.inetaddr.ttl=60";
Note: There are multiple PostgreSQL cluster solutions and the same configuration parameters may not work in all scenarios. The above parameters were used for a 3 node PostgreSQL cluster managed by Patroni.
STATUS
Enhancement request FOGGENERAL-I-201 has been submitted for consideration in a future release of the Foglight Management Server.
-
Defect ID
FOGGENERAL-I-201