Release Notes
Benchmark Factory® 8.1
Release Notes
Tuesday, August 07, 2018
About Benchmark Factory
Quest Software is pleased to make available Benchmark Factory® 8.1 to meet your database workload testing needs. Benchmark Factory is a database performance and code scalability testing tool that simulates users and transactions on the database and replays production workload in non-production environments. With Benchmark Factory developers, DBAs, and QA teams can validate that their databases will scale as user loads increase, application changes are made, and platform changes are implemented.
Benchmark Factory is available for Oracle®, SQL Server®, IBM® DB2®, SAP®, MySQL, and other databases via ODBC connectivity. Benchmark Factory:
- Simulates thousands of concurrent users with a minimal amount of hardware.
- Simulates database production workloads.
- Reports tests results with built-in reporting features.
Upon completion of a test execution, all test results are collected and stored in the repository for data analysis and reporting. Benchmark Factory collects a variety of informative and detailed statistics that include:
- Overall server throughput (measured in transactions per second/minute or bytes transferred).
- Detailed transaction executed statistics by individual agent workstations producing a load.
New Features
This release of Benchmark Factory includes the following new features and enhancements.
Benchmark Factory Agent for Linux
This release includes an exciting new feature. Benchmark Factory now provides an agent which you can deploy in your Linux environment. After installing the Linux version of Benchmark Factory Agent in your Linux environment, use the Benchmark Factory Console to set up the new agent. After setup, you can select the new agent from the list of agents available for load testing. In addition, the Benchmark Factory REST API supports configuring and modifying the Benchmark Factory Agent for Linux.
Supported Databases. The Benchmark Factory Agent for Linux can be used to test workloads against PostgreSQL databases.
Note: The Benchmark Factory Agent for Linux supports the following platforms only: Red Hat 7.x (64-bit), CentOs 7.x (64-bit), and Oracle Linux 7.x (64-bit).
The Benchmark Factory Agent for Linux is available for download from the Benchmark Factory Community on Toad World.
PostgreSQL Native Connection
This release of Benchmark Factory includes a PostgreSQL Native connection.
- To create a PostgreSQL connection, click New Connection in the main toolbar.
-
Then select PostgreSQL from the Connection Type drop-down list.
-
Enter connection information in this dialog to create a PostgreSQL native connection.
Alternatively, if you want to create a PostgreSQL ODBC connection, select the ODBC tab and enter connection information.
The following PostgreSQL database versions are supported: PostgreSQL 9.4, 9.5, 9.6 and 10, and EnterpriseDB Postgres 10.
Enhanced Licensing Dialog
This release includes an enhanced Licensing dialog which provides a simpler, easier-to-use interface.
- Select Help | Licensing to open the Licensing dialog. The Licensing dialog opens and displays the list of licenses for Benchmark Factory. Benchmark Factory supports multiple licenses. For example, a Toad for Oracle Edition license and a Toad for SQL Server Edition license may be active simultaneously.
- To enter a new license, click Add and enter the license key and the site message included with the license key.
- To modify a license, select the license to update and click Modify. Then update the key and site message.
- If Benchmark Factory was activated by a Toad Edition license, the Toad license is listed. Select the license and click Modify to update it.
-
To delete a license, select the license to delete and click Delete License.
Important: If you delete a license key, any features enabled by that license and not supported by another currently-active license are disabled.
BMFServer.exe Enhancements
- The default REST API port number for BMFServer.exe is now the same as the Benchmark Factory console, port 30100.
- The BMFServer log file (BMFServer.log) is now located in: \My Benchmark Factory\<version>\<bitness>\Error Logs
See Resolved Issues for more enhancements and resolved issues in this release of Benchmark Factory.
Learn More
Learn more and connect with the community at the Benchmark Factory Community page on Toad World.
- Find interesting blogs and other useful information.
- Ask questions in the forum.
- Link to Commercial, Trial, and Freeware downloads.
- Find Benchmark Factory documentation.
Deprecated Features
SQL Server Native Client
Beginning with Benchmark Factory 8.1, the Microsoft SQL Server Native Client (11.0) is no longer distributed along with Benchmark Factory.
Resolved Issues
Resolved Issues and Enhancements
The following is a list of issues addressed and enhancements implemented in this release of Benchmark Factory.
| Feature | Resolved Issue | Defect ID |
|---|---|---|
| Agents | When attempting to create a new, additional local agent through the Benchmark Factory console, user credentials cannot be used. A warning message now informs the user when this is the case. | BMF-2767 |
| Benchmark Testing | Corrected an issue that created a slight inaccuracy in the calculation of scale factor from database size in the Scale tab in the Test Options section. This issue was encountered when creating a TPC-C or TPC-E benchmark test. | BMF-2146 |
| Benchmark Testing | Corrected an issue in which the job fails during the Create Objects step if "Create/Load objects if objects don't exist" is selected and only some objects exist (for example, tables don't exist but procedures do). | BMF-3031 |
| Benchmark Testing | The CPU and memory usage statistics now display as expected when running a job using BMFAgent.exe. | BMF-3254 |
| Benchmark Testing | Oracle: Corrected an issue that caused an ORA-01406 error when attempting to run a TPC-E test against Oracle when the NLS_DATE_FORMAT variable was set to DD-MON-YYYY HH24:MI:SS. | BMF-3759 |
| BFScripts | Corrected an issue in which using BFURand generated duplicate values. | BMF-1011 |
| BMFServer | The BMFServer.ini file is now located in C:\ProgramData\Quest Software\BMF. | BMF-2704 |
| BMFServer | The default REST API port number for BMFServer.exe is now the same as the Benchmark Factory console, port 30100. | BMF-2702 |
| BMFServer | The BMFServer log file (BMFServer.log) is now located in: \My Benchmark Factory\<version>\<bitness>\Error Logs | BMF-2703 |
| Capture and Replay | SQL Server: When creating a job and adding the "Replay load from SQL Server trace table" test, Benchmark Factory no longer automatically searches the network for hosts To search for hosts, click "Refresh" on the SQL Server Authentication page. | BMF-3371 |
| Capture and Replay | Oracle: Simplified the replay test name for an imported Oracle capture. The replay test name is now "Replay Oracle Workload" regardless of the capture mode used. | BMF-3264 |
| Capture and Replay | Oracle: Corrected an issue in which a capture using FGAC failed when the capture included SQL longer than 4K. The following warning message displayed in the Status window when the capture process was running: ORA-02003: invalid USERENV parameter. | BMF-3707 |
| Capture and Replay | When importing from the Oracle Dynamic Performance View, transactions with a very low execution rate might not be imported. When this is the case, Benchmark Factory now displays a message informing the user. | BMF-2997 |
| Capture and Replay | Enhancement: When importing from an ODBC trace file, other statements in addition to Insert, Update, Delete, and Select statements are now included (e.g., Drop statements). | BMF-3067 |
| Capture and Replay | Corrected an issue in which the user was unable to specify a start time when scheduling a capture if the user's datetime format (Regional settings) was set to Polish (24h format). | BMF-4397 |
| Custom Mix Test | In a custom mix or goal test, if no results are obtained during the test run, it could be due to the fact that not all executions were completed in the specified pre-sampling time. The console now displays a message when this is possible. | BMF-2858 |
| General | The Support Bundle now includes information about BMFServer and BMFAgent and the BMFServer log file. | BMF-2705 |
| REST API | Corrected an issue in which attempting to GET testrun information using the zero-based index id or the test run, for example http://{server}:{port}/api/testruns/{index id}, resulted in a 404 error. This issue occurred with the following resource collections: TestRuns, UserLoadResults, TransactionResults, SubTransactionResults, and NodeResults. | BMF-2700 |
| REST API | Requesting latency for a transaction in a test now returns the latency information, as expected. | BMF-2745 |
| REST API | The time format is now the same in the current results returned by the console or the BMFServer (BMFServer.exe). | BMF-2857 |
| REST API | Corrected an issue that allowed a job to post successfully even though the request body contained invalid connection information. | BMF-2904 |
| REST API | The Database Flush property is now included in the Connection request/response body when using the REST API. | BMF-2918 |
| REST API | A request for TOTALRESPONSETIME from a test run now successfully returns data. An example URI is: http://{server}:{port}/api/TestRuns/1/UserloadResults/0/transactionresults/0/totalresponsetime. | BMF-3174 |
| REST API | Corrected an issue that caused an error when attempting to GET the Settings resource using the REST API (/api/settings). | BMF-2744 |
| REST API | Corrected an issue in which the ID number of a job created through the Benchmark Factory console unexpectedly changed (increased incrementally) when a subsequent job was created and added to the queue. | BMF-2856 |
| REST API | Added a read-only resource to the REST API which returns the BMF version number. | BMF-3706 |
| Run Reports | In a Transaction Mix graph, long transaction names are no longer truncated and are displayed correctly. | BMF-4660 |
| TPC-E Benchmark Test | IBM DB2: Corrected an issue which caused a TPC-E transaction (Trade-Result Transaction) to fail in a test using a DB2 native connection. | BMF-4603 |
| User Interface | Licensing dialog. The Licensing dialog has been redesigned to streamline the process of entering and modifying Benchmark Factory license keys. | BMF-1233 |
| User Interface | If one of the directories specified in Settings (Edit | Settings | General) does not have appropriate permissions applied and Benchmark Factory attempts to write to it, an informative message reminds the user to check directory permissions. | BMF-3715 |
| User Interface | Oracle: After importing an ODBC trace file, the path and file name are now displayed in full in the Trace File Summary tab of the New Job Wizard. | BMF-2973 |
| User Interface | Corrected text display issues found in the New Job Wizard in a Windows 10 environment when text size (DPI) was set to125% or greater. | BMF-1670, BMF-4895 |