Chat now with support
Chat with Support

Benchmark Factory for Database 8.0.1 - User Guide

Getting Started Overview of Benchmark Factory Benchmark Factory Components Agents Connections Create and Edit Tests and Jobs Test Results and Run Reports Troubleshooting Licensing The Benchmark Factory REST API

Overview of Benchmark Factory

About Benchmark Factory

Benchmark Factory is a database performance and code scalability testing tool that simulates users and transactions on the database and replays production workload in non-production environments. With Benchmark Factory developers, DBAs, and QA teams can validate that their databases will scale as user loads increase, application changes are made, and platform changes are implemented.

Benchmark Factory is available for Oracle®, SQL Server®, IBM® DB2®, SAP®, MySQL, and other databases via ODBC connectivity.

What You Can Do With Benchmark Factory

Benchmark Factory places enormous stress on a database system, which is typically hard to achieve in a standard testing environment. A system typically breaks under extreme load. By identifying system capacity and performance bottlenecks before they occur, Benchmark Factory facilitates proactive testing, which in turn reduces downtime, development costs, and potential loss of revenue.

Benchmark Factory allows you to:

  • Determine system throughput and capacity for database systems
  • Simulate thousands of concurrent users with a minimal amount of hardware
  • Find applications that do not scale well with an increase in the number of users
  • Find breaking points, weak links, or bottlenecks of a system
  • Quantify application or server performance

Benchmark Factory Components

Benchmark Factory consists of the following components:

Benchmark Factory Console (Desktop Application)

The Benchmark Factory console is used to create, run, and review Jobs (tests). You can also use the console to deploy agents, access the Repository, access reports, and create connections.

The Repository

All test results are collected and stored in the repository for data analysis and reporting. Benchmark Factory collects a vast amount of statistics, including overall server throughput (measured in transactions per second, bytes transferred, etc.) and detailed transaction statistics by individual workstation producing a load. You use these statistics to measure, analyze, and predict the capacity of a system.

The test results stored in the repository are displayed when you view the Results tab for a selected job or when you use the Run Reports component (Tools | Run Reports).

Agents

Agents simulate virtual users and send transactions to the system-under-test (database).

Run Reports

Benchmark Factory Run Reports is a separate component used to view the detailed test results in a report format. You can open Run Reports from the Benchmark Factory console (Tools | Run Reports) or from the Start menu (Benchmark Factory | Run Reports).

Integration with Other Toad Products

Benchmark Factory integrates with other Toad products to allow IT departments to quickly measure the capacity and performance of their systems to ensure that users will experience fast response times. For instance, Benchmark Factory can run multiple workloads in conjunction with Spotlight™ products. This provides the ability to detect and diagnose issues, allowing users to resolve bottlenecks, slow performance, and application flaws before an application is entered into production.

Understanding Benchmark Testing

Review the following for a better understanding of how Benchmark Factory performs benchmark testing on your database (system-under-test).

Agents, Virtual Users, and Automated Testing

When you load test with Benchmark Factory, you are performing automated testing. Automated testing is the process of using software to test computer hardware or software implying that the software is used instead of actual users. The software acts as a "virtual" user. For example, say you would like to test your database with two hundred virtual users over a given period of time. Benchmark Factory lets you select two hundred virtual users and the length of time you wish to perform the test.

Benchmark Factory uses Agents to deploy the virtual users. An Agent is a software routine that waits in the background and performs an action when a specified event occurs. One Agent can simulate thousands of virtual users (limited by hardware and workload characteristics) at a time. Each virtual user has their own connection to the system under test.

Understanding Benchmarks

A benchmark is a performance test of hardware and/or software on a system-under-test. Benchmark Factory provides the option of using industry standard benchmarks during the load testing process. Benchmarks measure system peak performance when performing typical operations.

Benchmark Factory comes equipped with the following industry standard benchmarks:

  • AS3AP Benchmark
  • Scalable Hardware Benchmark
  • TPC-B Benchmark
  • TPC-C Benchmark
  • TPC-D Benchmark
  • TPC-E Benchmark
  • TPC-H Benchmark
Related Documents

The document was helpful.

Select Rating

I easily found the information I needed.

Select Rating