Using Benchmark Factory Run Reports
Benchmark Factory Run Reports provides a wide array of graphs, charts, and data that allow you to a analyze load testing scripts.
Navigating in Benchmark Factory Run Reports
To open a test, double-click on the test name, or select the test name and click the Show Test Results icon in the upper right corner of the dialog.
When Benchmark Factory Run Reports opens, the Results Summary displays. Click Go Back to return to the previous screen.
To navigate Benchmark Factory Run Results
- Select a section of the report to view, for example, Testbed.
- Click Details. The Testbed information displays.
- To navigate back to Results Summary, click Go Back in the upper-left corner.
Run Reports provides the following views:
Results Summary
The Results Summary provides graphs and user load statistics for the selected test. The following graphs and tables are provided:
Results Summary Graph
Note: Not all tests provide the Results Summary graph.
This is a customizable graph that allows you to view selected data points.
To add or remove datapoints to the Results Summary graph
-
In the lower section of the graph, click Customize.
-
In the upper left corner, click Click here to Add Datapoints.
The Add Datapoints dialog displays.
-
Select the required datapoints.
-
Click Ok.
The selected datapoints display.
Other Results Summary Graphs
The Results Summary view allows you to view other graphs that provide test details.
To view other Results Summary Graphs
- In the Results Summary view, click More Graphs.
The following graphs display:
Userload
Understanding how user loads affect the performance of a database is essential to end user satisfaction. The Userload graph plots user load against:
- Response Time
- Total Bytes
- Total Errors
- Total Rows
- Response Time
Reviewing these datapoints allows you to fully understand how "real-world" userloads affect database performance.
Realtime Summary
The Realtime Statistic graph allow you to view what "actually" happened during a load test. You can plot userload against:
- Transaction/Second
- Total Rows
- Errors
- Total Bytes
- Rows/Second
- Minimum Time
- Maximum Response Time
- Deadlocks
- Bytes/Second
- Average Time
- Bytes/Second
Realtime Detail
The Realtime Detail graph allows you to view what actually happened during the running of a load test. This allows you to view the actual timing events. From the Realtime Detail graph you can view:
- Average Response Time
- Average Time
- Bytes/ Second
- Deadlocks
- Maximum Response Time
- Maximum Time
- Minimum Response Time
- Minimum Time
- Rows/Second
- Total Bytes
- Errors
- Total Rows
- Transactions/Second
- User Load
Userload Statistics Table
This table is a snapshot of the workload test. You to quickly review a complete overview of database that includes, userload, transactions per second (TPS), average response time (sec).
From this table you can drill down to view userload stastics.
To drill down from the Userload Statistics table
- Click on the required Userload.
The Userload Results view displays.
- From this view, you drill down further by clicking on required User Scenario name.
This displays detailed test results broken down by row name.
Workload
The Workload view provides testing details.
Clicking Details displays the transactions mix, timing, userload, and testing options used for the test.
Testbed
The Testbed view shows data on agent configuration and processes during the running of a job.
Clicking Details displays a table with machine and operating system details.
Database Under Test
This view displays connection and database information.
Click Details to display connection and database information details.
Viewing Multiple Run Results
Benchmark Factory allows you to view and compare multiple results using Run Reports.
To view multiple Run Reports results
- Select the desired runs.
- Click the Show Test Results icon
in the upper right corner of the dialog. Run Reports displays
- Click on an individual test run to view those results.
The results for that test display.
- Click on Details for Workload, Testbed, or Database Under Test to drill down on testing results.
The following graphic shows Testbed drill down results.
- Click More Graphs in Results Summary.
The Results Summary graphs display.
View other graphs as required.
- Close Run Reports when finished viewing run results.
Exporting Test Results to Excel
You can export test results to Microsoft Excel from the Compare Results page and from the Run Reports component.
Note: Microsoft Excel must be installed to perform the following procedure.
To export load test results from Run Reports
-
In Run Reports, click the Excel icon
.
- Modify export options or accept the default settings.
- Click OK. The test results open in Excel.
- The test results open in Excel.
- You can then save the report as an Excel file.
To export test results to Excel from the Compare Results page
- Select a test run and click the Excel icon
- Modify export options or accept the default settings.
- Click OK. The test results open in Excel.
- The test results open in Excel.
- You can then save the report as an Excel file.
Exporting Test Results to Excel Using the Command Line
Benchmark Factory allows you to export testing results to Excel using the Command line.
To export testing results from Benchmark Factory to Excel using the command line:
- Select Start | All Programs | Accessories | Command Prompt.
-
Navigate to the Benchmark Factory bin directory. For example:
CD C:\Program Files\Quest Software\Benchmark Factory\bin
CD C:\Program Files\Quest Software\Benchmark Factory <version number>\bin
- Enter BMFRunHistory.exe –x 34 88 108. Benchmark Factory testing results with Run ids of 34, 88, and 108 export to Excel.