Chat now with support
Chat with Support

Benchmark Factory for Database 9.0 - User Guide

Welcome to Benchmark Factory What's New in Benchmark Factory Additional Resources Overview of Benchmark Factory Benchmark Factory Components Licensing Getting Started - the Benchmark Factory Workflow Agents Create and Edit Connections Create and Edit Tests and Jobs
Jobs View Pane Job Wizards Quickstart: Create a New Job Edit a Job Industry Standard Benchmark Tests Capture and Replay a Workload Artificial Test for Desired Effect Scalability Tests Custom Tests Create/Delete Benchmark Objects Execute External File Test Options for Create Objects Test Options for Transactions Job Setup Options Use Global Search/Replace Save Job as BMF Script Copy Test to Replay, Mix, Goal, or Scalability Test
Benchmarks How Do I... Settings Test Results and Run Reports BFScripts Repository Troubleshooting The Benchmark Factory REST API Appendix About Us Adding Virtual Users System/Upgrade Requirements/Supported Databases Shortcut Keys

Third Party Components

This product contains some third party components (listed below). Copies of their licenses may be found at https://www.quest.com/legal/license-agreements.aspx. Source code for components marked with an asterisk (*) is available at https://opensource.quest.com.

Component License or Acknowledgement
cpp-httplib 0.6.6

Copyright (c) 2017 yhirose. All Rights Reserved. Released under terms of the MIT License.

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

libssh2 1.9.0

Copyright (c) 2004-2007 Sara Golemon <sarag@libssh2.org>

Copyright (c) 2005,2006 Mikhail Gusarov <dottedmag@dottedmag.net>

Copyright (c) 2006-2007 The Written Word, Inc.

Copyright (c) 2007 Eli Fant <elifantu@mail.ru>

Copyright (c) 2009-2014 Daniel Stenberg

Copyright (C) 2008, 2009 Simon Josefsson

All rights reserved.

Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:

1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.

2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.

3. Neither the name of the copyright holder nor the names of any other contributors may be used to endorse or promote products derived from this software without specific prior written permission.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

The full license text is available at https://www.quest.com/legal/third-party-licenses.aspx.

libxml2 2.9.9

Copyright (c) 1998-2012 Daniel Veillard. All Rights Reserved. Released under terms of the MIT License.

The full license text is available at https://www.quest.com/legal/third-party-licenses.aspx.

libxslt 1.1.33

Copyright (c) 2001-2002 Daniel Veillard. All Rights Reserved. Released under terms of the MIT License.

The full license text is available at https://www.quest.com/legal/third-party-licenses.aspx.

MariaDB Connector/C 3.0.7*

GNU LESSER GENERAL PUBLIC LICENSE
Version 2.1, February 1999

The full license text is available at: https://www.quest.com/legal/third-party-licenses.aspx

This is LGPL MariaDB client library that can be used to connect to MySQL or MariaDB.

This code is based on the LGPL libmysql client library from MySQL 3.23 and PHP's mysqlnd extension.

This product includes PHP software, freely available from <http://www.php.net/software/>

Source code for this component is available at https://opensource.quest.com.

OpenSSL 1.1.1g

Copyright 1998-2019 The OpenSSL Project. All rights reserved.This product includes software developed by the OpenSSL Project for use in the OpenSSL Toolkit.

Copyright 1995 -1998 Eric Young (eay@cryptsoft.com). All rights reserved. This product includes cryptographic software written by Eric Young (eay@cryptsoft.com).

Copyright 1995-1998 Tim Hudson (tjh@cryptsoft.com). All rights reserved.

Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:

1. Redistributions of source code must retain the above copyrightnotice, this list of conditions and the following disclaimer.

2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with thedistribution.

3. All advertising materials mentioning features or use of this software must display the following acknowledgment:"This product includes software developed by the OpenSSL Project for use in the OpenSSL Toolkit. (http://www.openssl.org/)"

4. The names "OpenSSL Toolkit" and "OpenSSL Project" must not be used to endorse or promote products derived from this software without prior written permission. For written permission, please contact openssl-core@openssl.org.

5. Products derived from this software may not be called "OpenSSL" nor may "OpenSSL" appear in their names without prior written permission of the OpenSSL Project.

6. Redistributions of any form whatsoever must retain the following acknowledgment: "This product includes software developed by the OpenSSL Project for use in the OpenSSL Toolkit (http://www.openssl.org/)"

THIS SOFTWARE IS PROVIDED BY THE OpenSSL PROJECT ``AS IS'' AND ANY EXPRESSED OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE OpenSSL PROJECT OR ITS CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

The full license text is available at https://www.quest.com/legal/third-party-licenses.aspx.

Pevents 1.0

Copyright (c) 2011–2015 by NeoSmart Technologies

http://neosmart.net/

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

PostgreSQL client Library (libpq) 12.0

PostgreSQL is released under the PostgreSQL License, a liberal Open Source license, similar to the BSD or MIT licenses.

PostgreSQL Database Management System

(formerly known as Postgres, then as Postgres95)

Portions Copyright © 1996-2020, The PostgreSQL Global Development Group

Portions Copyright © 1994, The Regents of the University of California

Permission to use, copy, modify, and distribute this software and its documentation for any purpose, without fee, and without a written agreement is hereby granted, provided that the above copyright notice and this paragraph and the following two paragraphs appear in all copies.

IN NO EVENT SHALL THE UNIVERSITY OF CALIFORNIA BE LIABLE TO ANY PARTY FOR DIRECT, INDIRECT, SPECIAL, INCIDENTAL, OR CONSEQUENTIAL DAMAGES, INCLUDING LOST PROFITS, ARISING OUT OF THE USE OF THIS SOFTWARE AND ITS DOCUMENTATION, EVEN IF THE UNIVERSITY OF CALIFORNIA HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

THE UNIVERSITY OF CALIFORNIA SPECIFICALLY DISCLAIMS ANY WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE SOFTWARE PROVIDED HEREUNDER IS ON AN "AS IS" BASIS, AND THE UNIVERSITY OF CALIFORNIA HAS NO OBLIGATIONS TO PROVIDE MAINTENANCE, SUPPORT, UPDATES, ENHANCEMENTS, OR MODIFICATIONS.

RapidJSON 1.0.2

Copyright (C) 2015 THL A29 Limited, a Tencent company, and Milo Yip.

All rights reserved.

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

SQLiteODBC 0.9996

This software is copyrighted by Christian Werner <chw@ch-werner.de> and other authors. The following terms apply to all files associated with the software unless explicitly disclaimed in individual files.

The authors hereby grant permission to use, copy, modify, distribute, and license this software and its documentation for any purpose, provided that existing copyright notices are retained in all copies and that this notice is included verbatim in any distributions. No written agreement, license, or royalty fee is required for any of the authorized uses.

Modifications to this software may be copyrighted by their authors and need not follow the licensing terms described here, provided that the new terms are clearly indicated on the first page of each file where they apply.

IN NO EVENT SHALL THE AUTHORS OR DISTRIBUTORS BE LIABLE TO ANY PARTY FOR DIRECT, INDIRECT, SPECIAL, INCIDENTAL, OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE USE OF THIS SOFTWARE, ITS DOCUMENTATION, OR ANY DERIVATIVES THEREOF, EVEN IF THE AUTHORS HAVE BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. THE AUTHORS AND DISTRIBUTORS SPECIFICALLY DISCLAIM ANY WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, AND NON-INFRINGEMENT. THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, AND THE AUTHORS AND DISTRIBUTORS HAVE NO OBLIGATION TO PROVIDE MAINTENANCE, SUPPORT, UPDATES, ENHANCEMENTS, OR MODIFICATIONS.

This software is released under a BSD-type license.

The full license text is available at https://www.quest.com/legal/third-party-licenses.aspx.

 

Copyright

© 2024 Quest Software Inc.
ALL RIGHTS RESERVED.

This guide contains proprietary information protected by copyright. The software described in this guide is furnished under a software license or nondisclosure agreement. This software may be used or copied only in accordance with the terms of the applicable agreement. No part of this guide may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopying and recording for any purpose other than the purchaser’s personal use without the written permission of Quest Software Inc.

The information in this document is provided in connection with Quest Software products. No license, express or implied, by estoppel or otherwise, to any intellectual property right is granted by this document or in connection with the sale of Quest Software products. EXCEPT AS SET FORTH IN THE TERMS AND CONDITIONS AS SPECIFIED IN THE LICENSE AGREEMENT FOR THIS PRODUCT, QUEST SOFTWARE ASSUMES NO LIABILITY WHATSOEVER AND DISCLAIMS ANY EXPRESS, IMPLIED OR STATUTORY WARRANTY RELATING TO ITS PRODUCTS INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTY OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, OR NON-INFRINGEMENT. IN NO EVENT SHALL QUEST SOFTWARE BE LIABLE FOR ANY DIRECT, INDIRECT, CONSEQUENTIAL, PUNITIVE, SPECIAL OR INCIDENTAL DAMAGES (INCLUDING, WITHOUT LIMITATION, DAMAGES FOR LOSS OF PROFITS, BUSINESS INTERRUPTION OR LOSS OF INFORMATION) ARISING OUT OF THE USE OR INABILITY TO USE THIS DOCUMENT, EVEN IF QUEST SOFTWARE HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES. Quest Software makes no representations or warranties with respect to the accuracy or completeness of the contents of this document and reserves the right to make changes to specifications and product descriptions at any time without notice. Quest Software does not make any commitment to update the information contained in this document.

If you have any questions regarding your potential use of this material, contact:

Quest Software Inc.
Attn: LEGAL Dept
4 Polaris Way
Aliso Viejo, CA 92656

Refer to our web site (www.quest.com) for regional and international office information.

Patents

This product includes patent pending technology. For the most current information about applicable patents for this product, please visit our website at www.quest.com/legal.

Trademarks

Quest, Quest Software, Benchmark Factory, Foglight, Spotlight, SQL Navigator, Toad, SharePlex, and the Quest logo are trademarks of Quest Software Inc. in the U.S.A. and other countries. For a complete list of Quest Software trademarks, please visit our website at www.quest.com/legal. Microsoft, Windows, Windows Server, Visual Studio, SQL Server, SharePoint, Access and Excel are either registered trademarks or trademarks of Microsoft Corporation in the United States and/or other countries. Oracle is a trademark or registered trademark of Oracle and/or its affiliates in the United States and other countries. Citrix® and XenApp™ are trademarks of Citrix Systems, Inc. and/or one or more of its subsidiaries, and may be registered in the United States Patent and Trademark Office and in other countries. SAP is the registered trademark of SAP AG in Germany and in several other countries. IBM and DB2 are registered trademarks of International Business Machines Corporation. All other trademarks, servicemarks, registered trademarks, and registered servicemarks are property of their respective owners.

Adding Virtual Users

Benchmark Factory comes with 100 virtual users by default.  Benchmark Factory Freeware comes with 20 virtual users. Virtual users simulate the load against the system-under-test. Each virtual user is a separate thread, acting independently of the other virtual users, with its own connection to the system-under-test. Each virtual user tracks its own statistics that includes transaction times and the number of times a transaction executes.

Benchmark Factory can scale up to an unlimited number of concurrent load users. Additional virtual users allow you to:

  • Accurately determine system throughput and capacity for database systems
  • Simulate thousands of concurrent users with a minimal amount of hardware
  • Find applications that do not scale well with an increase in the number of users
  • Find breaking points, weak links, or bottlenecks of a system
  • Quantify application or server performance with realistic workloads

To request additional concurrent load users, please contact your Quest Software representative, or visit the Benchmark Factory Web site.

 

System/Upgrade Requirements/Supported Databases

System Requirements

Before installing Benchmark Factory, ensure your system meets the following minimum hardware and software requirements.

Requirement Details
Repository

Benchmark Factory cannot run without a repository, and during installation a SQLite file-based repository is installed. After installation, you can select a repository of your choice. Benchmark Factory supports the following databases:

  • Oracle®
  • SQL Server®
  • MySQL
  • SQLite
  • SAP SQL Anywhere® (Adaptive Server® Anywhere)
Memory

512 MB of RAM recommended

Note: The memory required may vary based on the following:

  • Applications that are running on your system.
  • Size and complexity of the database.
  • Amount of database activity.
  • Number of concurrent users accessing the database.
Platform 1.0 GHz x86 or x64 compatible CPU
Browser Requirements Benchmark Factory requires Internet Explorer® 9 or later.
Hard Disk Space 120 MB of disk space for installation Note: The disk space required may vary based on options selected for installation.
Operating System

Microsoft Windows 10

Microsoft Windows 11

Microsoft Windows Server 2012

Microsoft Windows Server 2016

Microsoft Windows Server 2019

Microsoft Windows Server 2022

.NET Framework

.NET Framework 2.0 is required to use Oracle workloads captured using Benchmark Factory 6.9.3 or earlier.

Additional Software

The Visual Studio Runtime for Windows is required. See Visual C++ Redistributable to download this software.

Database Server

Oracle® 10g R2, 11G, 12c, 18c, 19c, and 21c

Requires Oracle 12c or later client libraries

Note: Oracle Capture/Replay of workloads does not support Oracle databases earlier than 9i.

PostgreSQL 9.5, 9.6, 10, EnterpriseDB Postgres 10, 11, 12, 13, 14, 15, and 16

SQL Server®: 2012, 2014, 2016, 2017, 2019, and 2022

  • Benchmark Factory has been tested against SQL Server 2019 and 2022 running on Windows and Linux.

Note: Beginning with Benchmark Factory 8.1, a SQL Server client is no longer distributed along with Benchmark Factory. To create a SQL Server native connection, ensure a SQL Server client is installed.

SAP® ASE 15.5, 15.7, 16

IBM® DB2® for LUW 9.x through 10.x, 11.x

IBM® DB2® for z/OS 11.1, 12, 13

MariaDB 10.x, 11.x

MySQL 5.5, 8.0.36

Note: The 32-bit version of Benchmark Factory requires the 32-bit versions of the client libraries (including ODBC drivers). The 64-bit version requires 64-bit client libraries.

ODBC Database Server Benchmark Factory supports almost all databases that you can connect to using an ODBC 3.x or later driver.
Cloud Database Service

Oracle Database as a service

AWS RDS platform for MSSQL, MySQL, and MariaDB

Azure SQL database

Note: Capture and Replay functionality is not supported on cloud databases because users do not have SYS admin privileges.

Benchmark Factory Agent for Linux (64-bit)

Databases CentOS RHEL Oracle Linux
PostgreSQL 7.x, 8.x, 9.x 7.x, 8.x, 9.x 7.x, 8.x, 9.x
Oracle 7.x, 8.x 7.x, 8.x 7.x, 8.x
MySQL 7.x, 8.x, 9.x 7.x, 8.x, 9.x 7.x, 8.x, 9.x
Microsoft SQL Server 7.x 7.x 7.x

Note: RHEL 9, CentOS 9, and Oracle Linux 9 are not supported by Oracle client.

Supported Databases for Load Testing: PostgreSQL, Oracle, MySQL and Microsoft SQL Server

Note: Testing against an Oracle database needs Oracle client installed on the same Linux machine as the Agent.

Upgrade Requirements

  • Client libraries for database types used during the workload testing process must be installed on all testing machines (Benchmark Factory and Agents).
  • There is no upgrade path for the Benchmark Factory Repository version 3.3 or earlier.
  • If you create a new Benchmark Factory 5.5 or later repository, earlier versions of Benchmark Factory will not work against this repository.

 

Related Documents

The document was helpful.

Select Rating

I easily found the information I needed.

Select Rating