Benchmark Factory for Database 8.0

Benchmark Factory Agents reside on each physical client machine and spawn multiple virtual-user sessions. Each Agent generates an information screen with tabs that contain a variety of Agent information. Benchmark Factory can control hundreds of Agent machines.

Each Agent can simulate up to 1500 virtual users, depending upon hardware and test configuration. Virtual users simulate the load against the system-under-test. Each virtual user is a separate thread, acting independently of the other virtual users, with its own connection to the system-under-test. Each virtual user tracks its own statistics that includes transaction times and the number of times a transaction executes.

The Agents View (Pane)

The Agents view or pane in the main Benchmark Factory window displays the status of the agents currently running.

If the Agents pane is not in view, select View | Agent from the main menu.

Add and Manage Agents

To add agents or to specify default settings for agents, see Agent Settings .
To install remote agents through the console, see Install Remote Agents.
To learn about running Benchmark Factory with multiple agents, see Running Benchmark Factory with Multiple Agents.
To learn about the non-UI Agent, see BMFAgent.exe.

The Benchmark Factory Agent Component Dialog

To open the Benchmark Factory Agent dialog, go to Program Files\Quest Software\Benchmark Factory <version>\bin and double-click Agent.exe.

The Benchmark Factory Agent dialog provides the follow tabs:

Agent Test Status Tab
Agent Virtual Users Tab
Agent Transaction Tab
Agent Output Tab

Run Reports

Benchmark Factory Run Reports is a separate executable that provides a comprehensive and detailed collection of database load testing results. With Benchmark Factory you can drill down into a database to view a wide array of information and statistics that gives you accurate insight into database performance. Run Reports Viewer allows you to access Benchmark Factory load testing results.

Note: Three instances of Run Reports can be viewed at one time.

Cutting and Pasting of Run Reports

From Benchmark Factory run reports you can cut and paste test results into the Clipboard.

Testing Server Throughput

Server throughput is the measurement of database capacity. Benchmark Factory allows you to view a variety of server throughput statistics that include:

Transaction/Second (TPS)
Transaction Time
Bytes/Second (BPS)
Rows/Second (RPS)
Total Bytes
Total Errors
Total Rows
Response Time

Testing SQL Scalability

The Benchmark Factory transaction comparison graph provides users with a comparison of the individual transaction performance as they compare to each other. This is most useful when analyzing a SQL Scalability test.

Testing Real Application Clusters

Benchmark Factory allows you to load test Oracle Real Application Clusters (RAC) to determine system capacity. The cluster performance graph allows users to view the performance of individual nodes.

Repository Manager

Note: If you create a new Benchmark Factory 5.5 (or later) repository, earlier versions of Benchmark Factory will not work against this repository.

The Repository is a database where all of the test results are stored. Benchmark Factory inserts test results into the repository and provides an easy way to access the data. By default, the Repository is a SQLite database that resides on the same machine as Benchmark Factory. The Repository can reside on another database server if required.

Note: By default in Benchmark Factory 7.1.1 or earlier, a MySQL database is created and used as the Repository, unless you selected the SQLite option during installation. In Benchmark Factory 7.2 or later, by default a SQLite database is created and used as the Repository.

To change the database, select the Data Source Name of the ODBC connection for the new database. To migrate data from one database to another, click Data Migration to open the Data Migration Wizard.

Note: If the database structure does not exist on the selected database, a prompt to create the structure will appear when OK is clicked.

The supported database types are:

MySQL
SQL Server®
SQLite
SAP SQL Anywhere® (Adaptive Server® Anywhere)
Oracle®

To open the Repository Manager

To open the Repository Manager from the application, select Tools | Repository Manager. Benchmark Factory prompts you to close the application. This is necessary before opening the Repository Manager.

The Repository Manager opens. Modify connection parameters or perform maintenance functions.

Review the following for additional information:

Connection Parameters
Data Source Name	Data Source name of the ODBC connection used to connect to the repository database.
ODBC Driver	Current ODBC driver
User Name	The User Name used to log into the selected database.
Password	The Password associated with the user name used to log into the database.
Edit DSN	Displays the ODBC connection information dialog for the selected data source.
ODBC Administrator	Displays the ODBC Data Source Administrator dialog. Use this to add and edit ODBC connections.
Test Connection	Tests the connection of the currently selected ODBC Data Source.
Maintenance
Create	Creates the repository objects on the selected database.
Delete	Deletes the repository objects on the selected database. Warning: This will delete all test results stored in the Repository.
Data Migration	Launches the Data Migration Wizard.

Jobs View

The Jobs View displays a list of saved jobs, shows which jobs are currently running, and which jobs are scheduled to run.

From the Jobs View pane, you can run jobs, view job status, and view real-time testing results.

To run a job

Select the job in the Jobs View pane and click .

To view test results

To view a job's test results, select a job in the Jobs View pane. Test results display in the right pane. See Benchmark Factory Console for an overview of the Benchmark Factory console.
To compare two or more run results for a test, select the Compare Results tab. Use Ctrl+click to select multiple test runs. A comparison of the results for the various runs displays.

To edit an existing job

To edit an existing job (to modify the workload or job setup options), select the job in the Jobs View pane and click .

View Job Status

From the Jobs View pane, you can view job status.

The following job states are identified:

Scheduled: All jobs currently waiting to run or scheduled to run at a future time.
Running: Job currently running.
Completed: All completed jobs.

Jobs View Toolbar

The Jobs View toolbar provides user functionality to save or delete jobs.

	Move the selected test up in the list.
	Move the selected test down in the list.
	Delete the selected item from the Jobs View.
	Toggles tree-view on and off.