Benchmark Factory for Database 8.0 - User Guide

Create Artificial Test for Desired Effects

Benchmark Factory allows you to design a workload based on a mixture of one of the following:

• Industry standard benchmarks
• Database application types (transaction/process types)
• Database transactions (SQL statement types)
• Hardware stressing focal points (database resources)

To create an artificial test for desired effect

1. In the New Job or Edit Job Wizard, click Workload in the left pane.
2. On the workload page, select Artificial Test for Desired Effect from the drop-down list.
3. Select one of the following test types:
   • Based on user selection of one or more industry standard benchmarks—Design a workload by specifying the percentages of industry standard benchmark (TPC-C, TPC-H, TPC-E, etc.) to use for the workload.
   • Based on user selection of one or more database application mixtures—Design a workload based on a mixture of OLTP and Data Warehouse type transactions, specified as percentages.
   • Based on user selection of one or more database transactional mixtures—Design a workload based on a mixture of SELECT, INSERT, UPDATE, and DELETE SQL statements, specified as percentages.
   • Based on user selection of one or more hardware stressing focal points—Design a workload by specifying the percentages of transactions that stress CPU, Memory, Disk IO, and Network.
4. Click the Select Test button at the bottom of the page.
5. Specify a weight and benchmark scale factor for each benchmark, transaction type, SQL type, or database resource.
6. Click Add Test.

7. The tests are added to the workload and the workload Summary page opens.

   • On the Summary page, you can click each of the links to go directly to an option to modify it.

8. You can also select the Test Options tab. Each test/step in the workload may have a different set of options available in its Test Options tab. See the following for more information.

   • Transactions Tab
   • Specify User Load
   • Timing Tab
   • Advanced Tab
   • Benchmark Scale Factor
   • Options Tab (Create Objects Step)
   • Customize SQL for Creating Objects

9. After specifying options for this test, you can add another test to the job, configure job setup options, save and close the job, run the job, or schedule the job. For more information about each of these steps, see Quickstart: Create a New Job.

Test Stored Procedures for Scalability

Stored procedures enables developers who write stored procedures in Oracle and SQL Server to test performance and scalability of the logic under varying user load levels. Benchmark Factory allows you to import SQL from database stored procedures into a Mix Load Scenario (workload). This gives you the ability to import SQL transactions, from stored procedures, for further testing under load.

To test Stored Procedures for Scalability

1. In the New Job or Edit Job Wizard, click Workload in the left pane.
2. On the workload page, select Scalability Test from the drop-down list.
3. Select the Test Stored Procedures for Scalability option.
4. Click the Select Test button.
5. In the Select Stored Procedures dialog, select the procedures from which to import SQL.
6. Click Next.
7. In the Bind Parameters page, select a SQL statement and enter a bind variable if necessary.
8. Click Finish. The SQL is imported and the Summary page displays.
9. Click the links on the Summary page to modify test and job setup options.

10. Or you can select the Test Options tab to modify all test options. Review the following for additional information:

    Tab	Description
    Transactions tab	Modify the transactions and the transaction mix for the test. See Transactions Tab for more information.
    User Load tab	Modify the number of users per test iteration. See Specify User Load for more information.
    Timing tab	Specify sampling, pre-sampling, and user start-up times for the test. See Timing Tab for more information.
    Advanced tab	Specify Repository options, error handling, and database checkpoints for the test. You can also specify a file to execute at the beginning or end of each iteration. See Advanced Tab for more information.

11. After specifying options for this test, you can add another test to the job, configure job setup options, save and close the job, run the job, or schedule the job. For more information about each of these steps, see Quickstart: Create a New Job.

Test SQL for Scalability

The SQL Scalability test allows you to execute SQL statements, allowing users to spot potential issues not seen with a single execution. Users can run variations of a SQL statement in order to find the SQL that will perform the best under a load test.

Use this procedure to create a new SQL Scalability test or a Custom Scalability test.

To create a SQL Scalability Test

1. In the New Job or Edit Job Wizard, click Workload in the left pane.
2. On the workload page, do one of the following:
   • Select Scalability Test from the drop-down list. Then select the Test SQL for Scalability option.
   • Select Custom Test. Then select the Scalability test compares ... option.
3. Click the Add Test button. The test is added to the workload and the test Summary page displays.
4. On the Summary page, click each link to modify the specific test option.

5. Or you can select the Test Options tab to modify all test options. Review the following for additional information:

   Tab	Description
   Transactions tab	Add transactions for the test. See Transactions Tab for more information.
   User Load tab	Modify the number of users per test iteration. See Specify User Load for more information.
   Timing tab	Select one of the following options: Number of executions per iteration-Each transaction is executed by each user for a specified number of times (recommended). Execution time per iteration-Executes each transaction for the specified length of time.
   Advanced tab	Specify Repository options, error handling, and database checkpoints for the test. You can also specify a file to execute at the beginning or end of each iteration. See Advanced Tab for more information.

6. After specifying options for this test, you can add another test to the job, configure job setup options, save and close the job, run the job, or schedule the job. For more information about each of these steps, see Quickstart: Create a New Job.

Custom Tests

Custom tests allow you to create workloads from user provided SQL. These tests provide flexibility for your load testing requirements. The following customs load scenarios are provided:

• Mix Test—Runs a transaction mix based upon weights for a specified time at each predetermined user load level.
• Replay Test—Runs multiple transactions with each one running independently on a specified number of users.
• Goal Test—Used to find maximum throughput or response time values. A transaction mix is executed at a range of user load levels.
• Scalability Test—Compares the performance of SQL Statement variations under a workload. Each transaction will execute individually for each user load and timing period.