When using Benchmark Factory to run a TPC-C load scenario, how is it possible to have two runs with approximately the same number of executions and rows and yet the response time between the runs is so different?
The TPC-C load scenario includes latency time, to reproduce what may typically happen when the various transaction types occur. For example, the latency time includes a keying delay, to simulate the delay that would typically occur when the user is keying data.
The database will always process the transactions as quickly as possible, so the response time will vary according to any tuning done at the database or hardware level. However, the transactions per second or kilobytes per second will only increase if sufficient load is placed on the server.
To increase the load, either use a larger number of virtual users (which requires a "virtual user license" purchased to increase beyond the default 100 virtual users), or alternatively reduce (or eliminate) the latency time for the transactions in the TPC-C scenario (so that the transactions are posted as quickly as possible, with no delay in-between).
Please note that reducing the latency time will mean the scenario is no longer a "standard" TPC-C test as it no longer conforms to the standards at www.tpc.org. However, this may be a useful alternative in some situations.
To reduce the latency time for a TPC-C scenario, go to the "Script" tab and expand the "TPC-C Transaction Mix" node. This should expose the various transactions ("New Order", "Payment", "Order-Status", "Delivery" and "Stock-Level"). Right-click on each transaction, go to the "Latency" tab and choose the "No Delay" checkbox. This will remove the latency and cause the transactions to be posted as quickly as possible.
