Benchmark Factory for Database 9.0 - User Guide

Execute a File

You might want to add an executable file (.bat or .exe) as part of a job. This gives you the flexibility to set up your database before running a workload. For example, you could run a .bat file to set up a prepared SQL statement before running a workload.  

To add an Execute File step to a workload

1. In the New Job or Edit Job Wizard, click Workload in the left pane.
2. On the workload page, select Execute External File from the drop-down list.
3. Enter the path or browse to and select a file to execute.

4. To add a timeout, select the Enforce Timeout option and enter the number of seconds for Benchmark Factory to wait for the execution to complete before continuing the job.

   You can specify a default setting for this option in Edit | Settings | Execute File. See Execute File Settings for more information.

Use Benchmark Factory Scripting Capabilities

The Benchmark Factory scripting feature known as BFScripts allows you to insert randomized data into the load testing process. You can use BFScripts when you add SQL transactions. See Transactions Tab for more information.

Script-enabled fields have a yellow background. A field has scripting capabilities if the field's right-click menu includes the BFScript Wizard option. The BFScript Wizard is a quick and easy way to use Benchmark Factory scripts. The BFScript Wizard provides you with a list of built-in script functions, grouped by category, from which to select. Each script function has a short description included, and if applicable, the function parameters.  See About Scripts for an overview of BFScripts.

There are two features in Benchmark Factory that assist you when using scripting capabilities.

• BFScript Wizard
• Script Assist

BFScript Wizard

1. Use one of the following methods to open the BFScript Wizard:

   • From within a script-enabled field (yellow background), right-click and select BFScript Wizard.

     

   • In the SQL Statement tab, click .

2. Select a category in the left pane to display BFScripts (functions) in the right pane.

   

3. Select a BFScript name (function).
4. If the script requires parameters, click Next and enter the required parameters.
5. Click Finish to enter the script in the field.

Script Assist

1. When entering a SQL statement, enter $BF. Script Assist automatically displays a list of scripts from which you can select.

   

2. Select the desired script.
3. Press ENTER to enter the script in the field.
4. Enter parameters as required.

Copy a Test to a Replay, Mix, or Scalability Test

A test can be quickly converted from one test type to another by using the "copy" command.  A workload test is an assembled test comprised of user scenarios and/or transactions. These tests can be run with multiple virtual users. A workload test can be one of three types: mix test, replay test, goal test, or scalability test.

In Benchmark Factory you can copy the following tests:

• Replay test—A Replay Test runs multiple transactions with each one running independently on a specified number of users. The test will run until the defined number of executions for each transaction or a specified time limit is reached.
• Mixed test—A Mixed Workload test runs for a specified time at each predetermined user loads level. Each user will run a transaction mix based upon the weights defined on the transactions tab.  For example, if a test has two transactions, A and B, with A having a weight of one and B having a weight of four, on average B will run four times for every time A is run once. The run order will be randomly generated for each user so they are not all running the same transaction simultaneously. That run order is used for that user each time the test is performed to ensure reproducible results.
• Goal test—A goal test is used to find maximum throughput or response time goals. A transaction mix is executed at user load levels, determined by setting a beginning, ending, and interval value. When run, the specified goal criterion is evaluated at the end of each iteration and the test ends once the goal or maximum user load has been reached.
• Scalability—A SQL Scalability test executes each transaction individually for each userload and timing period.  For example,  a test has two transactions, A and B, and two userloads of 10 and 20, with an iteration length of one minute. Transaction A would execute continually for one minute at userload 10, then B would do the same. Next A will run at userload 20, followed again by test B, for a total time of 4 minutes.

Copying a workload test creates a new test containing all compatible settings, user scenarios, and transactions from the original test.

To copy a workload test to a Replay Test, Mix Test, Goal Test, or Scalability test

1. Right-click a test in the Jobs View pane or in the New/Edit Job Wizard. A drop-down displays.
2. Select the desired type of test you want to copy to.
3. The test is created and displays in the Jobs View or New/Edit Job Wizard.

Run SQL Preview

SQL Preview allows you to preview the result set of an individual SQL statement when adding or editing a SQL transaction.

To run SQL Preview

1. In the SQL Statement tab, click .
2. Select a connection from the drop-down list.

4. In the Maximum Returned field, enter the maximum number of rows to preview, or select the All option to preview all rows.
5. Click OK. The SQL Preview window opens.
6. Review the information.
7. Click Close.