Benchmark Factory for Database 8.2.1 - User Guide

Getting Started Licensing Overview of Benchmark Factory Benchmark Factory Components Agents Create and Edit Connections Create and Edit Tests and Jobs Test Results and Run Reports Troubleshooting The Benchmark Factory REST API

Benchmark Factory Test Results Terminology

The following is a list of terms used in Benchmark Factory test results and reports. Review the terms and definitions to aid you when interpreting your test results.

Term Definition

Bytes

The number of bytes from data received from a SQL statement.

Bytes Per Second

The number of bytes processed per second over the sampling period.  This is just the Total Bytes divided by the Sampling period in seconds.

Response Time

The time it takes from when the SQL is sent to the server responds.

Retrieval Time

The time it takes from the server responds to a SQL statement till the last byte of data/results is obtained.

Rows

The number of rows received from a SQL statement.

Rows Per Second (RPS)

The number of rows received per second over the sampling period.  Similar to above.

Transactions Per Second (TPS)

The transactions, or SQL statements, processed by the server per second.  A transaction in Benchmark Factory can be more than a single SQL statement, such as the TPC-C transaction New Order, this transaction inserts a new order by inserting one record into the new order table and 5 – 7 items in the orderline table.

Transaction Time (sometimes listed as just Time)

The sum of the Response and Retrieval time.

The metrics listed above might also be expressed in test results using the following:

  • Average–The average of all recorded values for the statistic over the sampling period.
  • Minimum–The minimum value the statistic obtained over the sampling period.
  • Maximum–The maximum value the statistic obtained over the sampling period.
  • 90th Percentile– This is usual associated with a timing statistic.  This is the time value where 90 percent of all values recorded for a statistic fell below.

Troubleshooting

Support Bundle

You can create a Support Bundle and send it to Quest Support for review. To help troubleshoot problems, the support bundle contains information such as:

  • BMF version number
  • Settings
  • License information (just send the key files)
  • Error Logs and Result Logs from the associated directories
  • Files located in the data directory. This will be not only the xml files for imported users scenarios, but also dump files.
  • Script files XML configuration files
  • A file that contains hardware information about the system running on it as well as the output of the agents configurations from the repository
  • Information about versions of all loaded .dlls

To create a Benchmark Factory support bundle

  1. Select Help | Support Bundle to open the Support Bundle dialog.
    • To create a support bundle for a single job, select the job in the Jobs View pane before opening the Support Bundle dialog.
  2. In the Support Bundle dialog, select the modules you want to send.
  3. If creating a support bundle for a single job, select Just for selected job.
  4. Then do one of the following:
    • To save the bundle, click the Save icon . Then select a location in which to save the zipped file.
    • To email the bundle to Support, click Email Quest Software Support.
    • To contact Support via the Support Portal, click Quest Support.

    By default, the support bundle is created and saved in the following location:

    C:\Users\<user name>\My Documents\My Benchmark Factory\<version number>\BMFSupportBundle.zip.

Agent Connection

If the agent is having a problem connecting to the Benchmark Factory console, please check the following:

Related Documents