Benchmark Factory for Database 9.0 - User Guide

TPC-E Benchmark

• Overview
• Certification of Transaction Processing Council (TPC) Testing Results
• Best Practices
• Scaling Factor

To learn how to create a TPC-E benchmark test in Benchmark Factory, see Create Industry Standard Benchmark Test.

Overview

The Transaction Processing Council is an organization that establishes transaction processing and database benchmark standards. Find a complete overview and detailed explanation of the TPC-E  Benchmark at: http://www.tpc.org/tpce/default.asp

Certification of Transaction Processing Council (TPC) Testing Results

Transaction Processing Council  testing results cannot be published as certified unless the testing procedure is audited and approved by the TPC organization. If not certified, the testing results can be published as a "TPC-E like" test.

Best Practices

The following provides best practices for the TPC-E Benchmark.

Load-testing against production servers

Do not load-test against a production server if possible.  Load-testing and benchmarking on a production server significantly degrades performance.  In some cases, load-testing can cause a server to fail.  However, if testing against a production server, take the following precautions:

• Perform the testing when no other users are on the system and no automated processes are running.  Users and automated processes can adversely affect testing results
• Have a recovery plan and backup all data prior to testing
• Determine how long it will take to restore a production server if it went down during load-testing
• Perform manual testing. Manual testing ensures that no unexpected outside activity takes place during the testing process

Scaling Factor

The TPC-E benchmark scales by factor of 500.

TPC-H benchmark

Review the following for information about the TPC-H benchmark.

• Overview
• Certification of Transaction Processing Council Testing Results
• Best Practices
• Stream Test

To learn how to create a TPC-H benchmark test in Benchmark Factory, see Create Industry Standard Benchmark Test.

Overview

The Transaction Processing Council is an organization that establishes transaction processing and database benchmark standards. Find a complete overview and detailed explanation of the TPC-H  Benchmark at: http://www.tpc.org/tpch/default.asp.

Certification of Transaction Processing Council Testing Results

Transaction Processing Council testing results cannot be published as certified unless the testing procedure is audited and approved by the TPC organization. If not certified, the testing results can be published as a "TPC-H like" test.

Best Practices

The following provides best practices for the TPC-H Benchmark.

Load-testing against production servers

Do not load-test against a production server if possible.  Load-testing and benchmarking on a production server significantly degrades performance.  In some cases, load-testing can cause a server to fail.  However, if testing against a production server, take the following precautions:

• Perform the testing when no other users are on the system and no automated processes are running.  Users and automated processes can adversely affect testing results
• Have a recovery plan and backup all data prior to testing
• Determine how long it will take to restore a production server if it went down during load-testing
• Perform manual testing.  Manual testing ensures that no unexpected outside activity takes place during the testing process

Reinitialize the Database

To reinitialize a testing database, run a job containing a Benchmark Object node.  

There are two ways to create a Benchmark Object node:

• Run the Benchmark Object Wizard to add the Delete Benchmark objects for TPC-H node to a job as needed.  
• Run the New Job Wizard to create a new job containing the Create Objects for the Benchmark, Associated Load Scenarios for the Benchmark, and Delete Objects for the Benchmark. Running the delete objects for 'TPC-H' job will clean the environment.

Stream Test

An option when creating a TPC-H workload is to include the TPC-H Stream Test, which is the multi-user version of the Power Test. The Steam Test, per specification, should maintain the following relationship between the scale factor and the number of Streams.

Replication Benchmark

The Replication benchmark test is a unique test developed by Benchmark Factory for testing replication in applications such as SharePlex.

This test allows you to create customized database objects, as well as a customized workload, in order to produce a repeatable database change rate and history (for example, Oracle redo logs) that can be used to test replication strategies. This test can also be used to create a model of your database and workload in order to help demonstrate replication performance.

While this benchmark was designed for testing and evaluating replication strategies and functionality in SharePlex, it can also be used to test other database replication products as well.

The following options can be customized in the Replication benchmark test:

• The number of tables in the test
• The number of columns per table and the column types (data types)
• The percentage of each column type (data type) in all tables, for example, 40% INT, 20% VARCHAR(255), etc.
• The percentage of each statement type (insert, update, delete), as well as the number of statements per commit (transaction)

See Create a Replication Benchmark Test to learn how to create a new Replication test.

How Do I...