Benchmark Factory Components
Benchmark Factory Console
The Benchmark Factory console implements the database workload testing process. This interface is where:
- Jobs are created and executed.
- Users can access test results. See Benchmark Factory Test Results for more information.
- Agents are managed. See About Agents for more information.
The Benchmark Factory Console contains the following views:
- Jobs View
- Benchmark Factory Test Results
- Agents View
- Output View
Agents
The Benchmark Factory Agent is a component used in Benchmark Factory to create virtual users which simulate real-world user activity by placing transactions against the database-under-test. The Benchmark Factory Agent is installed when the Benchmark Factory Console is installed. In addition, you can install additional agents on other remote machines. Each Benchmark Factory agent can spawn multiple virtual-user sessions and Benchmark Factory can control hundreds of Agent machines.
After installing additional agents, use your Benchmark Factory Console to define a connection to each agent machine. When you create a new benchmark test through the Console, you can select which of the defined agents to use to generate the user load.
Each virtual user is a separate thread, acting independently of the other virtual users, with its own connection to the system-under-test. Each virtual user tracks its own statistics, including transaction times and the number of times a transaction executes.
View Agent Information
The Benchmark Factory Console allows you to view information about all currently running agents, provides a way to connect to and manage remote agents, and allows you to specify some global agent settings. In addition, the primary Agent, Agent.exe, includes a GUI which displays transaction and virtual user statistics during test execution.
- Select View | Agent to open the Agents pane where you can view information about agents that are currently running.
- Select Edit | Settings | Agent to view a list of all the configured agents which are available to be used in testing, as well as platform information about each agent machine. Use this page to view the agent global settings, as well.
- Double-click Agent.exe in the bin directory to open the Benchmark Factory Agent. The GUI displays virtual user statistics during test execution. Select Options | Settings to configure options for this agent.
Types of Agent Components
Benchmark Factory provides several types of agent components:
- Agent.exe—(Windows) The primary agent. This agent is installed with the Benchmark Factory client application and can be found in the bin directory. This component includes a GUI.
-
BMFAgent.exe—(Windows or Linux) The non-GUI agent. You can use BMFAgent.exe to integrate Benchmark Factory into your continuous integration or continuous testing process. See BMFAgent.exe for more information.
- (Windows) The Windows version of this agent is installed with the Benchmark Factory client application and can be found in the bin directory.
- (Linux) For a Linux platform, you can download a Linux version of this agent from the Benchmark Factory Community on Toad World. See Install Remote Agent on Linux for more information.
Using Agents
Review the following topics to learn how to use the Benchmark Factory Agent.
- Set Up New User Agent—To set up a remote agent
- Agent Settings —To view configured agents and specify default settings
- Install Remote Agent on Windows—To install a remote agent on a Windows platform
- Running Benchmark Factory with Multiple Agents—To learn how to run a test using multiple agents
- BMFAgent.exe—To learn about the non-GUI agent
- Install Remote Agent on Linux—To install a remote agent on a Linux platform
Run Reports
Benchmark Factory Run Reports is a separate executable that provides a comprehensive and detailed collection of database load testing results. With Benchmark Factory you can drill down into a database to view a wide array of information and statistics that gives you accurate insight into database performance. Run Reports Viewer allows you to access Benchmark Factory load testing results.
Note: Three instances of Run Reports can be viewed at one time.
Cutting and Pasting of Run Reports
From Benchmark Factory run reports you can cut and paste test results into the Clipboard.
Testing Server Throughput
Server throughput is the measurement of database capacity. Benchmark Factory allows you to view a variety of server throughput statistics that include:
- Transaction/Second (TPS)
- Transaction Time
- Bytes/Second (BPS)
- Rows/Second (RPS)
- Total Bytes
- Total Errors
- Total Rows
- Response Time
Testing SQL Scalability
The Benchmark Factory transaction comparison graph provides users with a comparison of the individual transaction performance as they compare to each other. This is most useful when analyzing a SQL Scalability test.
Testing Real Application Clusters
Benchmark Factory allows you to load test Oracle Real Application Clusters (RAC) to determine system capacity. The cluster performance graph allows users to view the performance of individual nodes.
