Chat now with support
Chat with Support

Benchmark Factory for Database 8.4 - User Guide

Welcome to Benchmark Factory What's New in Benchmark Factory Additional Resources Overview of Benchmark Factory Benchmark Factory Components Licensing Getting Started - the Benchmark Factory Workflow Agents Create and Edit Connections Create and Edit Tests and Jobs
Jobs View Pane Job Wizards Quickstart: Create a New Job Edit a Job Industry Standard Benchmark Tests Capture and Replay a Workload Artificial Test for Desired Effect Scalability Tests Custom Tests Create/Delete Benchmark Objects Execute External File Test Options for Create Objects Test Options for Transactions Job Setup Options Use Global Search/Replace Save Job as BMF Script Copy Test to Replay, Mix, Goal, or Scalability Test
Benchmarks How Do I... Settings Test Results and Run Reports BFScripts Repository Troubleshooting The Benchmark Factory REST API Appendix About Us Adding Virtual Users System/Upgrade Requirements/Supported Databases Shortcut Keys

Error Handling Settings

Use this page to specify error handling options.

To specify error handling settings

  1. Select Edit | Settings. The Settings dialog opens.

  2. Select Error Handling. Review the following.

    Group Description
    Error Handling  

    Stop test after first error

    		 Default setting for a new jobs. Terminates the test when a server error occurs.

    Stop job after first test error

    		 Terminates the job when a server error occurs.
    Thresholds  

    Stop job if Memory usage exceeds

    		 Instructs Benchmark Factory to stop the job if memory usage exceeds this threshold.
  3. Click Apply to apply changes.

 

Repository Settings

Note: If you create a new repository in Benchmark Factory 5.5 or later, earlier versions of Benchmark Factory will not work against this repository.

The repository is a database where all of the test results are stored. Benchmark Factory inserts test results into the repository and provides an easy way to access all test results data.

By default, the Repository is a SQLite database that resides on the same machine as Benchmark Factory. The Repository can reside on another database server if required. To change the database, select the Data Source Name of the ODBC connection for the new database. To migrate data from one database to another, click Data Migration to open the Data Migration Wizard.

Note: By default in Benchmark Factory 7.1.1 or earlier, a MySQL database is created and used as the Repository, unless you selected the SQLite option during installation. In Benchmark Factory 7.2 or later, by default a SQLite database is created and used as the Repository.

If you plan to store a large amount of test data in the repository, you might want to consider using a more robust database than SQLite.

The Repository Settings page allows you to edit the DSN, perform ODBC administration, and test the connection. Benchmark Factory also provides a Repository Manager and Data Repository Migration wizard to assist you with other repository management functions.

Note: If the database structure does not exist on the selected database, Benchmark Factory prompts you to create the structure.

The supported database types are:

  • MySQL
  • SQL Server®
  • SQLite
  • SAP SQL Anywhere® (Adaptive Server® Anywhere)
  • Oracle®

To specify Repository settings

  1. Select Edit | Settings. The Settings dialog opens.

  2. Select Repository. Review the following for additional information:

    Data Source Name Data Source name of the ODBC connection used to connect to the repository database.
    User Name The User Name used to log into the selected database.
    Password The Password associated with the user name used to log into the database.
    Edit DSN Click to edit the selected Data Source Name.
    ODBC Administrator Click to open the ODBC Data Source Administrator dialog. Use this to add a DSN or ODBC connection.
    Test Connection Tests the defined connection.
    Data Migration Click to open the Data Migration Wizard. See Data Repository Migration Wizard for more information.

 

Statistics Settings

You can use this page of the Settings dialog to specify default values for the statistics collection options.

Note: If you modify the default settings in the Settings dialog, the changes apply to new jobs only, not to existing jobs.

To specify Statistics settings

  1. Select Edit | Settings. The Settings dialog opens.

  2. Specify statistics collection options. Review the following for additional information:

    Save results to Repository

    Select to save test information to the Repository. Then select one or more of the following options:

    • Save Real-Time Counter Information—Saves real-time performance monitoring information to the Repository during the test. See Performance Counters Tab for more information.

      Note: To specify sampling rate, go to Edit | Settings | Statistics | Real-Time counters.

    • Save User Statistics—Saves user-level statistics.
    • Save Distribution—Saves transactions per second and response time per user distribution.

    • Log Transaction Results to disk—Logs the transaction results of a job to disk.

      Note: Checking this option imposes performance penalties.

To specify Real-Time Counter settings

  1. Select Edit | Settings. The Settings dialog opens.
  2. Expand the Statistics node and select Real-Time Counters.
  3. In the Real-time reporting interval field, specify a sampling rate.

Tip: You can over-ride the default settings and specify statistics collection options per job. In the New/Edit Job Wizard, select a test under Workload and then select the Performance Counters tab (under Job Setup tab).

  

Agent Settings

Use this page of the Settings dialog to do the following:

  • Install remote agent on Windows or Linux
  • Set up a new agent by creating a connection to the agent machine
  • Specify default settings (error handling and communication timeout) for each agent
  • Specify the default agent/agents to use when creating a new job.

To specify Agent settings

  1. Select Edit | Settings. The Settings dialog opens.

  2. Select Agent. Review the following for additional information:

    Setup New User Agent

    Click to setup a new agent or to install a remote agent on Windows or Linux.

    List of agents

    Displays the list of agents that have been set up in the console.

    • Select an agent to display platform information for the agent machine.
    • Double-click an agent to open the Edit User Agent dialog.

    Use this area to specify the default agent machines to use by selecting the check box for each default agent. Also, specify the number of agents. These default settings are used when you create a new job.

    If you do not select a default agent, the local agent is automatically selected when you create a new job in the New Job wizard.

    Note: When you run a job using one or more local agents, if Agent utilization of resources on the local machine is too high, errors could occur.

    Remove Click to remove the selected agent.
    Verify Selected Agent Verifies that the selected agent starts and communicates with console.
    Error Handling

    Stop job when agent cannot be started/connected—Cancels job if agent does not start or connection is lost.

    Do not select this option if you want to run a job using multiple agents AND you want the job to run even if one or more agents do not start.

    Agent communication timeout Specify timeout duration in seconds.
    Agent refresh interval Specify how often each agent should refresh its status.

Tips:

  • In the New/Edit Job Wizard, select Agent in the left pane of the wizard to access agent options for the selected job. You can select agents or set up new agents from this page of the wizard.
  • To open the Agent console, go to Program Files\Quest Software\Benchmark Factory <version>\bin and double-click Agent.exe. See The Benchmark Factory Agent Console for more information.

 

Related Documents

The document was helpful.

Select Rating

I easily found the information I needed.

Select Rating