Benchmark Factory® is a database performance testing tool that allows you to conduct database workload replay, industry-standard benchmark testing, and scalability testing.
Review the following topics for a quick overview of Benchmark Factory.
Benchmark Factory 8.2.1 is a minor release and includes resolved issues. For a complete list, see the Benchmark Factory 8.2.1 Release Notes at the Quest Support Portal.
MySQL Native Connection. Beginning with this release, Benchmark Factory now provides the ability to create a MySQL connection using Native database connectivity.
Note: The following Benchmark tests are supported: TPC-C, TPC-E, and TPC-H.
This release of Benchmark Factory includes support for the following database versions:
In release 8.1, Benchmark Factory introduced an agent which you can deploy in your Linux environment. Benchmark Factory 8.2 includes several significant enhancements to the Agent for Linux.
You can now install an agent on a remote Linux machine through the Benchmark Factory Console. This provides an easy, UI-based method for installing a remote agent on a Linux platform.
The Benchmark Factory Agent for Linux is now distributed as an RPM package. This provides a number of advantages over the previous archive file (.tar) format. This format allows BMF to provide automatic installation through the Benchmark Factory Console. You can also use YUM to install the Benchmark Factory Agent RPM package on Linux manually.
To download the RPM package for manual installation, go to https://bintray.com/quest/bmfrepo. For additional information, see "Install Remote Agent on Linux" in the Benchmark Factory Help. For best results, install the Agent for Linux automatically through the Benchmark Factory Console.
The Benchmark Factory Agent for Linux can now be used when testing against the following database types (in addition to PostgreSQL):
Oracle. If using the Benchmark Factory Agent for Linux to test against an Oracle database, ensure an Oracle Client is installed on the same Linux machine as the Agent. Review the following details:
Benchmark Factory 8.1 is a minor release and includes the following new features and enhancements. For a list of resolved issues in this release, see the Benchmark Factory 8.1 Release Notes at the Quest Support Portal.
This release includes an exciting new feature. Benchmark Factory now provides an agent which you can deploy in your Linux environment. After installing the Linux version of Benchmark Factory Agent in your Linux environment, use the Benchmark Factory Console to set up the new agent. After setup, you can select the new agent from the list of agents available for load testing. In addition, the Benchmark Factory REST API supports configuring and modifying the Benchmark Factory Agent for Linux.
Supported Databases. The Benchmark Factory Agent for Linux can be used to test workloads against PostgreSQL databases.
Note: The Benchmark Factory Agent for Linux supports the following platforms only: Red Hat 7.x (64-bit), CentOs 7.x (64-bit), and Oracle Linux 7.x (64-bit).
The Benchmark Factory 8.1 Agent for Linux is available for download from the Benchmark Factory Support Page.
This release of Benchmark Factory includes a PostgreSQL Native connection.
Then select PostgreSQL from the Connection Type drop-down list.
Enter connection information in this dialog to create a PostgreSQL native connection.
Alternatively, if you want to create a PostgreSQL ODBC connection, select the ODBC tab and enter connection information.
The following PostgreSQL database versions are supported: PostgreSQL 9.4, 9.5, 9.6 and 10, and EnterpriseDB Postgres 10.
This release includes an enhanced Licensing dialog which provides a simpler, easier-to-use interface.
To delete a license, select the license to delete and click Delete License.
Important: If you delete a license key, any features enabled by that license and not supported by another currently-active license are disabled.
Benchmark Factory 8.0.1 is a maintenance release and includes resolved issues and minor enhancements. See the Benchmark Factory 8.0.1 Release Notes for a complete list.
Benchmark Factory® 8.0 includes the following new features and enhancements.
With this release the Benchmark Factory REST API has been enhanced to allow users more control of jobs, tests, test runs, etc. This includes the ability to access the transactions in tests, agents used by jobs, test results of a test run, as well as the Benchmark Factory settings.
Support for JSON format. Responses can now be returned in JSON format. To return data using the JSON format, specify JSON in the request Accept header. Both XML and JSON are supported response or request formats.
Access to Additional REST Resources. Additional resources have been added to the Benchmark Factory REST API. For a complete list, see the Benchmark Factory Help or the BMF REST API on Toad World. The Help also includes a useful description of the REST resource structure and the URI hierarchy.
This release includes additional features that allow you to easily integrate Benchmark Factory into your Continuous Integration or Continuous Testing process.
This release includes a non-UI Benchmark Factory application, BMFServer.exe.
Benchmark Factory now includes a non-UI Agent, BMFAgent.exe.
Clear Cached Data between Iterations
You can now instruct Benchmark Factory to clear cached data between test iterations. Because cached data can improve performance, this new feature can prevent cached data from affecting subsequent iterations.
Oracle—For Oracle connections, you can instruct Benchmark Factory to clear data buffer caches or shared pool (or both). To specify these options, select Edit | Connections. Then select a connection and click the Edit button. In the connection properties dialog, select the Miscellaneous tab. Then specify the Database Flush options for this connection.
Tip: You can also specify default values to automatically apply to Oracle connections in Edit | Settings | Oracle.
Note: Specific database privileges are required to perform this action. The privilege required is dependent on the database version and the option selected. See the online Help for more information.
SQL Server®—For SQL Server connections, you can instruct Benchmark Factory to clear data buffer caches and procedure caches. To specify this option, select Edit | Connections. Then select a connection and click the Edit button. In the connection properties dialog, select the Miscellaneous tab. Then specify the Database Flush option for this connection.
Tip: You can also specify a default value to apply to SQL Server connections in Edit | Settings | SQL Server.
Note: This option is only applicable to SQL Server 2005 or later. The sysadmin fixed server role is required.
Microsoft® SQL Server 2017. This release includes support for SQL Server 2017. Benchmark Factory has been tested against SQL Server 2017 running on Windows or Linux.
IBM® DB2®. This release includes support for IBM DB2 11.1 for LUW and for z/OS.
Adding Bind Parameters
It is now easier to add a bind parameter/value pair in Test Options | Transactions.
Universal C Runtime Component
Note: If you encounter an error when installing this software, install the missing prerequisite software. For Windows 8.1 or Windows Server 2012 R2, install the April 2014 update: https://support.microsoft.com/en-us/kb/2919355. For other operating systems, see the Universal C Runtime Update Prerequisites section.
Note: The features described in the New in This Release section apply to the commercial version of Benchmark Factory and may not be available in the freeware edition.
Use the following links to videos, blogs, and community resources to learn more about Benchmark Factory, to find answers to questions, and to advance your knowledge through Benchmark Factory tips and tricks.