Benchmark Factory Test Results Terminology
The following is a list of terms used in Benchmark Factory test results and reports. Review the terms and definitions to aid you when interpreting your test results.
| Term | Definition |
|---|---|
|
Bytes |
The number of bytes from data received from a SQL statement. |
|
Bytes Per Second |
The number of bytes processed per second over the sampling period. This is just the Total Bytes divided by the Sampling period in seconds. |
|
Response Time |
The time it takes from when the SQL is sent to the server responds. |
|
Retrieval Time |
The time it takes from the server responds to a SQL statement till the last byte of data/results is obtained. |
|
Rows |
The number of rows received from a SQL statement. |
|
Rows Per Second (RPS) |
The number of rows received per second over the sampling period. Similar to above. |
|
Transactions Per Second (TPS) |
The transactions, or SQL statements, processed by the server per second. A transaction in Benchmark Factory can be more than a single SQL statement, such as the TPC-C transaction New Order, this transaction inserts a new order by inserting one record into the new order table and 5 – 7 items in the orderline table. |
|
Transaction Time (sometimes listed as just Time) |
The sum of the Response and Retrieval time. |
The metrics listed above might also be expressed in test results using the following:
- Average–The average of all recorded values for the statistic over the sampling period.
- Minimum–The minimum value the statistic obtained over the sampling period.
- Maximum–The maximum value the statistic obtained over the sampling period.
- 90th Percentile– This is usual associated with a timing statistic. This is the time value where 90 percent of all values recorded for a statistic fell below.
Troubleshooting
Support Bundle
You can create a Support Bundle and send it to Quest Support for review. To help troubleshoot problems, the support bundle contains information such as:
- BMF version number
- Settings
- License information (just send the key files)
- Error Logs and Result Logs from the associated directories
- Files located in the data directory. This will be not only the xml files for imported users scenarios, but also dump files.
- Script files XML configuration files
- A file that contains hardware information about the system running on it as well as the output of the agents configurations from the repository
- Information about versions of all loaded .dlls
To create a Benchmark Factory support bundle
- Select Help | Support Bundle to open the Support Bundle dialog.
- To create a support bundle for a single job, select the job in the Jobs View pane before opening the Support Bundle dialog.
- In the Support Bundle dialog, select the modules you want to send.
- If creating a support bundle for a single job, select Just for selected job.
- Then do one of the following:
- To save the bundle, click the Save icon
. Then select a location in which to save the zipped file.
- To email the bundle to Support, click Email Quest Software Support.
- To contact Support via the Support Portal, click Quest Support.
By default, the support bundle is created and saved in the following location:
C:\Users\<user name>\My Documents\My Benchmark Factory\<version number>\BMFSupportBundle.zip.
- To save the bundle, click the Save icon
Agent Connection
If the agent is having a problem connecting to the Benchmark Factory console, please check the following:
- Verify that the console is running.
- Verify that the agent is pointed to the console machine. In the Agent dialog, go to Options | Settings and check the Machine Name/IP and the Console Port fields . See Configure Agent Machine for more information.
- Verify TCP connectivity between the Agent and Console. If there is a firewall in place, allow for communication on port 4568 (or the specified console port). See Configure Firewall for Remote Agent Install/Start-Up for more information.