Foglight 7.1.0 - High Availability Field Guide

Tuning the Management Server

If you encounter these HA issues, consider performing the following tuning exercise.

IMPORTANT: If the system clock is out of sync by 5 seconds or more during HA server startup, a warning message similar to the following appears in the log: Warning: system time not in sync with primary server, Mon Aug 30 11:47:57 ECT 2010 vs Mon Sug 30 11:48:52 EDT 2010.)

IMPORTANT: You must make the following changes to each server in the cluster.

NOTE: In these examples, 7800 is the port number used for TCP communication. For more information about the HA ports used by the Management Server, see Configuring High Availability mode ports .

Managing hosts with multiple network interfaces

When configuring Foglight® Management Servers that are installed on hosts with multiple network interfaces (that is, with multiple IP addresses and host names), you can specify the IP address that is to be used for communication with the server. To specify the IP address and host name, configure the following parameters in the server.config file:

Where:

If a DNS host name is not available for the bind address, the IP address can be used:

When the Management Server is configured to run in HA mode, you also need to reconfigure the restarter to perform server health check with the same host name or IP address. To do that, open the restart_monitor.config file, locate the health.check.url and server.host properties, and set their values to the one specified by the server.remote.address parameter (in server.config).

If a DNS name is available for the desired bind address, the name can be used instead of the raw address.

For more information about server.bind.address and server.remote.address, see the comments in server.config.

JDK with IPv6 on Linux

Management Servers running on a VMware® Linux® x86_64 image fail to start up and a “jave.net.BindException: Cannot assign requested address exception” is raised. (85262)

This is a JDK issue with IPv6 on Linux, not a true High Availability issue. It also impacts standalone servers. However, the issue is documented here because it occurs when the JGroup cluster service is starting up.

Workaround: Start the Management Server with the following command:

-Djava.net.preferIPv4Stack=true

For more information, see: the JGroups FAQ (http://community.jboss.org/wiki/JGroupsFAQ) and Oracle® Bugs for it (http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6521974).

Foglight Management Server automatically restarted

Sometimes, after what appears to be a normal and successful startup, an HA server is automatically shut down and restarted. The most likely reason for this is a misconfiguration of health check URL in the restart_monitor.config file.

For example, you may have reconfigured the HTTP port of the Management Server or reconfigured the IP address that the server is bound to, but forgotten to reconfigure the health check URL of the restarter. If the restarter cannot contact the Management Server for health check, it considers the Management Server not responsive and restarts it.

You can check the server_restarter_xxxx-xx-xx_xxxxx_xxx.log file to determine if this scenario is what caused the restart. If so, edit the restart_monitor.config file by locating the line beginning with health.check.url and configuring the URL properly.