지금 지원 담당자와 채팅
지원 담당자와 채팅

Benchmark Factory for Database 9.0 - User Guide

Welcome to Benchmark Factory What's New in Benchmark Factory Additional Resources Overview of Benchmark Factory Benchmark Factory Components Licensing Getting Started - the Benchmark Factory Workflow Agents Create and Edit Connections Create and Edit Tests and Jobs
Jobs View Pane Job Wizards Quickstart: Create a New Job Edit a Job Industry Standard Benchmark Tests Capture and Replay a Workload Artificial Test for Desired Effect Scalability Tests Custom Tests Create/Delete Benchmark Objects Execute External File Test Options for Create Objects Test Options for Transactions Job Setup Options Use Global Search/Replace Save Job as BMF Script Copy Test to Replay, Mix, Goal, or Scalability Test
Benchmarks How Do I... Settings Test Results and Run Reports BFScripts Repository Troubleshooting The Benchmark Factory REST API Appendix About Us Adding Virtual Users System/Upgrade Requirements/Supported Databases Shortcut Keys

Latency

Specify Latency

Latency is delay added to the execution of a transaction to control how fast transactions are submitted to the system-under-test. Latency is used to either make the transaction execution rate more like real-world executions or control the transaction rate. This delay can be added to the beginning and/or end of a transaction execution.

To specify latency values for an individual transaction, edit the transaction using the Transactions tab of the New/Edit Job Wizard. You can do this either at the time you add transactions/scenarios to a test or any time after the job is created.

You can also specify latency for all the child transactions of a test or user scenario at one time. See Replace Child Latencies for more information.

Benchmark Factory allows you to set default latency values for the transactions you add. See Latency Settings  for more information.

Why Use Latency?

Real-World Delays: Add Keying Time and Think Time to create a workload that accurately simulates real-world users, as opposed to a workload that submits transactions as fast as possible. Using real-world delays allows for easier interpretation of the results. If you do not add delay, it might be difficult to equate N virtual users running with no delay to an accurate number of real users.

Interarrival Time: Specify Interarrival Time to create a workload in which transactions arrive at the system-under-test at a specific interval. This method allows you to set the timing relationship between the executing transactions and is commonly used when replaying a captured workload.

To modify latency values

  1. Open the New Job wizard for a new job or the Edit Job wizard for an existing job.
  2. Select the test (or the step) under Workload that contains the transactions.
  3. Select the Test Options tab. Then select the Transactions tab.
  4. Select the transaction or scenario for which you want modify latency and click Edit.
  5. In the dialog that opens, select the Latency tab. For a scenario, to modify latency for individual transactions, select a transaction and click Edit. Then select the Latency tab.
  6. Modify latency options. To specify no latency, select No Delay. To add latency, select one of the other options. Review the following for additional information:

    No Delay Select to specify no latency. See No Delay for additional information.
    Transaction Rate

    Select this option to set a transaction rate by specifying the interval between successive transactions arriving at the server. See Interarrival Time for more information.

    Interarrival Time—Select one of the following:

    • Absolute—Select to specify a fixed interarrival time. Then enter a value.
    • Distribution models—To use a distribution model for determining interarrival time, select a model and enter the requested value.

      To learn more about distribution models, see Latency Distribution Models.

    Real-World Delays

    Use this option to simulate real-world processing by inserting a delay before and/or after each transaction execution.

    For each delay type, you can insert a fixed time by selecting Absolute, or you can select one of the distribution models for specifying time. To learn more about distribution models, see Latency Distribution Models.

    Keying Time—Inserts a delay before execution to simulate the amount of time spent performing data entry. See Keying Time for additional information.

    Think Time—Inserts a delay after execution to simulate the amount of time "to think" about the results of the previous transaction. See Think Time for additional information.

  7. Click OK to save your changes and close the dialog.
  8. Then click Save/Close in the wizard to save your changes.

Latency Definitions

No Delay

No Delay means that transactions execute as fast as possible. As soon as one transaction is processed, the next transaction is issued against the server. In the case of a mixed workload test, each virtual user issues transactions as fast as possible.

The No Delay option is used when the goal of the test is to stress the system to its limits, without concern for accurately simulating users.  With No Delay specified, a relatively low number of users can stress the system to its limits. However, there is no easy way to correlate N virtual users running with no delay to some number of real users.

Keying Time

Keying Time is used to simulate the amount of time spent performing data entry (entering information) before executing a transaction. In many cases, Keying Time is used with Think Time to provide a delay both before and after a transaction executes.

Specifying a Keying Time inserts a delay (either fixed or variable) before each transaction execution.  

Think Time

Think Time is used to simulate the amount of time spent thinking about the results of the previous transaction. This could be time spent performing analysis on the results of a database query.

Specifying Think Time inserts a delay (either fixed or variable) after each transaction executes.

 

Related Links

Replace Child Latencies

Interarrival Time

Interarrival time is the time between two successive transactions arriving at the server. It is used to determine the average transaction rate as seen by the server (i.e., the rate at which transactions arrive at the server).

When you specify Interarrival time, Benchmark Factory is instructed to ensure that the transactions arrive at the server at the specified interval, regardless of how long a transaction actually takes to execute.

For example, if the Interarrival Time is set for 60 seconds and a transaction runs for 20 seconds, a delay of 40 seconds is added (60 seconds - 20 seconds = 40 seconds). If the next transaction takes 28 seconds, a delay of 32 seconds is added.

Transaction

Execution

Delay

Next Transaction

Execution

Delay

--20 sec.--

--40 sec.-- --28 sec.-- --32 sec.--

60 sec.

Interarrival Time

60 sec.

Interarrival Time

 

If a transaction takes longer than the Interarrival Time, the next interval is measured from the arrival of the next transaction.

Transaction

Execution

Next Transaction

Execution

Delay

60 sec.

Interarrival Time

 

60 sec.

Next Interarrival Time

 

 

Latency Distribution Models

If you select to apply latency to transactions in a workload, Benchmark Factory allows you to either specify a fixed value (Absolute) for the delay time or use a distribution model for determining delay time.

Using Absolute

Use Absolute when you want a fixed value for delay. For example, if a 2000 ms delay is specified for Keying Time and a 3000 ms delay is specified for Think Time, when a transaction executes, Benchmark Factory waits 2000 ms, then starts the transaction, and then waits an additional 3000 ms before deciding which transaction to execute next.  

If Interarrival Time is used with an Absolute delay of 2000 ms, Benchmark Factory marks the time, executes the transaction, and waits until two seconds has elapsed from the marked time (assuming the transaction finishes in less than two seconds) before determining which transaction to execute next.

Distribution Models

Review the following distribution models provided by Benchmark Factory.

Uniform Distribution

Selecting a Uniform delay instructs Benchmark Factory that random delay should be chosen, with an equal probability of being the minimum value, the maximum value, or any value in between. Uniform delays are chosen when it is suspected that the delay is highly random within a range or a minimal amount of statistical analysis has been performed to determine how the actual users react.

Suppose a uniform Keying Time is selected with a minimum value of 1000 ms and a maximum value of 1500 ms.  If the transaction is executed more than 500 times, there is a high probability that each possible delay has been selected at least once.  With the other delay types, this is not the case.  

If 2000 ms to 2500 ms uniform delay is set for Interarrival time, the tester essentially is setting the test so that a server sees the transaction every 2 to 2.5 seconds, instead of exactly two seconds as in the Absolute delay time.

Negative Exponential Distribution

Similar to the Normal distribution, Negative Exponential inserts a random delay based on a mathematical model.  A Negative Exponential distribution should be chosen when most users have latency of the mean latency or less. Negative Exponential is weighted heavily toward smaller latencies being selected relative to the mean and maximum latency.  

Normal Distribution

Normal distributions differ from Uniform delays in that most of the delays chosen by Benchmark Factory will be close to the average, but can vary by as much as ±10% of the mean. While a Uniform delay is used when users have latencies within equal likelihood of being anywhere between two values, Normal distributions are chosen when all users fall within a range, but most of the modeled users have a latency close to the average latency.

Poisson Distribution

A Poisson distribution is very similar to the Normal distribution and can be used most places where a Normal distribution delay could be used. The biggest difference between a Normal distribution and a Poisson distribution is that Poisson selects discreet values.

 

Related Links

관련 문서

The document was helpful.

평가 결과 선택

I easily found the information I needed.

평가 결과 선택