Upstream cache set to -1 in MongoDB, Postgresql, MySQL, and Cassandra cartridges fill the disk space in the agent manager (4227361)

The agent manager crashes frequently.

When checking the settings, the cache folder under \state\default\cache contain files that are filling up the disk.

The upstream_data.cache file continues growing and it doesn't stop. After have cleaned the cache files and a Fglam restart, the issue still persist.

In the MongoDB, Postgresql, MySQL, and Cassandra database cartridges the following information can be seen in the agent.manifest files: max-queue-size and max-disk-space. Some of the cartridges has configured max-disk-queue = "-1". This configuration allows the Fglam to grow the cache files with no limits.

The rest of database cartridges (SQL Server, Oracle, DB2 and Sybase) do not have this configuration (max-queue-size and max-disk-space) in their agent.manifest file.

By default, the fglam.config.xml file is the one that contains the information for max-queue-size and max-disk-space.

The agent.manifest file for these cartridges override the settings configured in fglam.config.xml. 

In consequence, the cache file will grow unlimited and the agent manager could crash.

WORKAROUND:

Please only apply this change if the MySQL, Cassandra, MongoDB, and PostgreSQL agent has the max-queue-size setting in the agent manifest file (versions below 5.9.3.10) 

1. Modify the max-disk-queue to 2048 in the agent.manifest file for each agent manager where the cartridges are deployed.
   

   The agent.manifest files can be located on:

   • %FGLAM_HOME%\agents\MongoDBAgent\config
   • %FGLAM_HOME%\agents\CassandraAgent\config
   • %FGLAM_HOME%\agents\MySQLAgent\config
   • %FGLAM_HOME%\agents\PostgreSQLAgent\config
2. Stop the Agent Manager.
3. Delete the 3 cache files under %FGLAM_HOME%\state\default\cache
4. Start the agent manager

Note: this workaround needs to be applied in each agent manager affected.

STATUS:

Beginning with the 5.9.3.10 and higher cartridges the agent manifest in these cartridges no longer contains a max-queue-size value

• FOGCASS-86 for Cassandra cartridge
• FOGMONGO-158 for MongoDB cartridge
• FOGMYSQL-67 for MySQL cartridge
• FOGPGSQL-61 for Postgresql cartridge