Chat now with support

Get Live Help
Complete Registration

Sign In

Request Pricing

Contact Sales

FREE TRIALS

Benchmark Factory for Database 9.0 - User Guide

Table of Contents

Welcome to Benchmark Factory What's New in Benchmark Factory Additional Resources Overview of Benchmark Factory

About Benchmark Factory Benchmark Factory Components Understanding Benchmark Testing Understanding Benchmark Factory Terminology How Benchmark Factory Works Feature/Edition Comparison

Benchmark Factory Components

Benchmark Factory Console Agents Run Reports Repository Manager Jobs View BMFServer.exe

Licensing and Keys Adding Virtual Users

Getting Started - the Benchmark Factory Workflow Agents

About Agents Set Up New User Agent Specify Agent Settings Install Remote Agent on Windows Install Remote Agent on Linux Run Benchmark Factory with Multiple Agents Agents View Use the Agent Component Console BMFAgent.exe Specify Max Virtual Users Configure Firewall - Remote Agent Install

Create and Edit Connections

Connections Overview Create IBM DB2 CLI Connections Create MySQL Connections Create ODBC Connections Create Oracle Connections Create PostgreSQL Connections Create SAP Connections Create SQL Server Connections Edit and Manage Connections Timeout and Reconnect Options Environment Information Performance Counters Oracle Statistics Collection Oracle Clustering Options Configure RAC Load Testing

Create and Edit Tests and Jobs

Jobs View Pane Job Wizards Quickstart: Create a New Job Edit a Job Industry Standard Benchmark Tests

Create Industry Standard Benchmark Test Create Replication Benchmark Test

Capture and Replay a Workload

About Capture/Replay Tests Capture and Replay Oracle Workload Capture and Replay SQL Server Workload Run Capture Scenario Wizard Run Capture Wizard from Cmd Line Replay Load from an Oracle Trace File Replay Load from SQL Server Trace Table Replay Load from ODBC Trace File Import from Delimited Text File Import from Oracle Dynamic Performance View Import from Dell Tools Export File Capture Oracle Workload - Privileges

Artificial Test for Desired Effect

Create Artificial Test for Desired Effects

Scalability Tests

Test SQL for Scalability Test Stored Procedures for Scalability

Create Custom Test Create Mix Test Create Replay Test Create Goal Test Create Scalability Test

Create/Delete Benchmark Objects Execute External File Test Options for Create Objects

Benchmark Scale Factor Options (Create Objects) Create Objects Using Custom SQL Table Options (Create Objects for Replication)

Test Options for Transactions

Transactions User Load Timing Advanced Options (Capture/Replay) Add SQL Transactions Add User Scenario from XML Latency

Specify Latency Interarrival Time Latency Distribution Models Replace Child Latencies

Use BFScript Wizard Run SQL Preview Stagger User Startup

Job Setup Options

Job Setup Schedule Alerts Performance Counters Global Variables

Use Global Search/Replace Save Job as BMF Script Copy Test to Replay, Mix, Goal, or Scalability Test

Overview AS3AP Benchmark Scalable Hardware Benchmark TPC-B Benchmark TPC-C Benchmark TPC-D Benchmark TPC-E Benchmark TPC-H benchmark Replication Benchmark

Import Benchmark Factory Self Signed Certificate Local Access to REST API Allow external host access REST API Create a Connection Use Run Reports Create/Delete Benchmark Objects for Load Testing Change Scaling Factors Specify the User Load Execute a File Use Benchmark Factory Scripting Capabilities Copy a Test to a Replay, Mix, or Scalability Test Run SQL Preview Replace Child Latencies Use Global Search and Replace Create an ODBC Trace File Schedule a Job Set Latency Set the Number of Users on an Agent Stagger User Startup Run Benchmark Factory with Multiple Agents SQL Scalability Testing from Toad Test Database Scalability

About General Settings Benchmarks Settings - General Benchmark Settings - Specific TPC-B Benchmark Settings Replication Benchmark Settings Timing Settings User Load Settings Latency Settings Error Handling Settings Repository Settings Statistics Settings Agent Settings Oracle Settings SQL Server Settings REST API Settings Execute File Settings

Test Results and Run Reports

Overview View Real-Time Test Results

View Test Results - Running Job Realtime Statistics Tab Run vs Userload Tab Transaction vs Load Tab Changing Graph View

View Test Results - Completed Job Benchmark Factory Run Reports

Overview Changing Graph View Run Reports Viewer Using Benchmark Factory Run Reports Viewing Multiple Run Results

Exporting Test Results to Excel Exporting Test Results to Excel Using the Command Line Benchmark Factory Test Results Terminology

About Scripts BFScript Wizard Using Scripts in Username and Password Fields Dates and Numbers

$BFCreditCardExp $BFCurrentDate $BFCurrentDateTime $BFDate

$BFFileArray

Global Variable

$BFGetGlobalVar $BFSetGlobalVar $BFSetGlobalVarRtn

Name and Address

$BFAddress $BFAddress2 $BFCity $BFCompany $BFCountry $BFEmail $BFFirstName $BFFullName $BFLastName $BFMiddleInitial $BFPhone $BFState $BFZipCode

Numerical Manipulation

$BFFormat $BFProd $BFSum

$BFRand $BFRandRange $BFURand $BFURandRange

$BFCreditCard $BFRandList $BFRandMultiList $BFRandStr $BFURandList $BFList

String Manipulation

$BFAsc $BFChr $BFConcat $BFLeft $BFLen $BFLower $BFMid $BFRight $BFTrim $BFTrimLeft $BFTrimRight $BFUpper

$BFGetVar $BFMaxNode $BFNode $BFNumberOfIterations $BFProfile $BFRunID $BFSetVar $BFSetVarRtn $BFUserCounter $BFUserID $BFUserload $BFNextUserload $BFPrevUserload

Repository Settings Repository Manager Data Repository Migration Wizard

Troubleshooting

Support Bundle Agent Connection Standard Benchmark Tests Use Benchmark Factory with SQL Server 2005 MySQL Initialization Settings

The Benchmark Factory REST API

Getting Started Resource Structure and URI Hierarchy REST API Resources

Trace Files or Fine Grained Access Control: Pros and Cons Change Graph Views Customizing List Controls Oracle Instant Client Installation Creating a SQL 2005 Trace Table Using the SQL Server Profiler Creating a SQL 2008_2008 R2 Trace Table Using the SQL Server Profiler Migrating Data Using the Command Line Wizard Store Procedure Examples Oracle Trace File Activation Create an ODBC Trace File Configure Firewall for Remote Agents

About Us User Registration Product Improvement Program Third Party Components Copyright

Adding Virtual Users System/Upgrade Requirements/Supported Databases Shortcut Keys

How Benchmark Factory Works

The following steps provide an overview of how Benchmark Factory components interact during the load testing process.

The Benchmark Factory Console implements the workload testing process and controls one or more distributed agent machines. Each agent machine can simulate thousands of users. Each simulated user executes transactions and records statistics.
Benchmark Factory About Agents machines simulate virtual users. The Agents send transactions to the system-under-test. The Agents record statistics that includes how much data the resulting transaction contained and how long it took to get the results. At the end of an iteration, each agent machine reports its findings back to Benchmark Factory.
A server (system-under-test) is the database Benchmark Factory connects to. Benchmark Factory is server neutral and network protocol independent. Benchmark Factory uses vendor client libraries of the system-under-test. Any system that the client software can support, Benchmark Factory can support. Its only requirement is that Agent machines must be able to connect to a server through an appropriate manner and it supports the test system.
The Repository Manager stores all testing data.
Results are viewed from Run Reports.

Feature/Edition Comparison

Review the following table for a comparison of features available in the commercial and freeware editions of Benchmark Factory.

Feature	Freeware Edition	Commercial Edition
Default maximum virtual user load (without additional virtual users packs)	20	100
Additional virtual user packs		(unlimited)
SQL Optimizer Integration
Oracle Capture and Replay
SQL Server Capture and Replay
Maximum number of transactions that can be run during a load test/user scenario	20	Unlimited
Scaling factors for standard benchmarks (AS3AP, TPC-B, TPC-C, TPC-D, and Scalable Hardware)	Lowest two scaling factors for each benchmark	All are available
Repository database types	Only the default SQLite repository is enabled	The following repository database types are supported: Oracle, Microsoft SQL Server, SAP SQL Anywhere (Adaptive Server Anywhere), MySQL, and SQLite
BMFServer.exe
Support	Technical support from Quest is not available for the Freeware edition.	For the Commercial edition, support from Quest is available at: https://support.quest.com

= Available

= Not available

Get the latest product information and find helpful resources at the Benchmark Factory community at: https://www.toadworld.com/products/benchmark-factory-for-databases

Benchmark Factory Components

Benchmark Factory Console

The Benchmark Factory console implements the database workload testing process. This interface is where:

Jobs are created and executed.
Users can access test results. See Benchmark Factory Test Results for more information.
Agents are managed. See About Agents for more information.

The Benchmark Factory Console contains the following views:

Related Documents

The document was helpful.

Select Rating

I easily found the information I needed.

Select Rating