Back to home page

MySQL Cross Reference

 
 

    


0001 The MySQL Benchmarks
0002 
0003 These tests require a MySQL version of at least 3.20.28 or 3.21.10.
0004 
0005 Currently the following servers are supported:
0006 MySQL 3.20 and 3.21, PostgreSQL 6.#, mSQL 2.# and Solid Server 2.2
0007 
0008 The benchmark directory contains the query files and raw data files used to
0009 populate the MySQL benchmark tables. In order to run the benchmarks, you
0010 should normally execute a command such as the following:
0011 
0012 run-all-tests --server=mysql --cmp=mysql,pg,solid --user=test --password=test --log
0013 
0014 This means that you want to run the benchmarks with MySQL. The
0015 limits should be taken from all of MySQL, PostgreSQL, and Solid.
0016 The login name and password for connecting to the server both are
0017 ``test''.  The result should be saved as a RUN file in the output
0018 directory.
0019 
0020 When run-all-tests has finished, will have the individual results and the
0021 the total RUN- file in the output directory.
0022 
0023 If you want to look at some old results, use the compare-results script.
0024 For example:
0025 
0026 compare-results --dir=Results --cmp=mysql,pg,solid
0027 compare-results --dir=Results --cmp=mysql,pg,solid --relative
0028 
0029 compare-results --dir=Results --cmp=msql,mysql,pg,solid
0030 compare-results --dir=Results --cmp=msql,mysql,pg,solid --relative
0031 
0032 compare-results --dir=Results --server=mysql --same-server --cmp=mysql,pg,solid
0033 
0034 Some of the files in the benchmark directory are:
0035 
0036 File                    Description
0037 
0038 Data/ATIS               Contains data for 29 related tables used in the ATIS tests.
0039 Data/Wisconsin          Contains data for the Wisconsin benchmark.
0040 Results                 Contains old benchmark results.
0041 Makefile.am             Automake Makefile
0042 README                  This file.
0043 test-ATIS.sh            Creation of 29 tables and a lot of selects on them.
0044 test-connect.sh         Test how fast a connection to the server is.
0045 test-create.sh          Test how fast a table is created.
0046 test-insert.sh          Test create and fill of a table.
0047 test-wisconsin.sh       A port of the PostgreSQL version of this benchmark.
0048 run-all-tests           Use this to run all tests. When all tests are run,
0049                         use the --log and --use-old options to get a RUN-file.
0050 compare-results         Generates a comparison table from different RUN files.
0051 server-cfg              Contains the limits and functions for all supported
0052                         SQL servers.  If you want to add a new server, this
0053                         should be the only file that neads to be changed.
0054 
0055 
0056 Most of the tests should use portable SQL to make it possible to
0057 compare different databases. Sometimes SQL extensions can make things
0058 a lot faster. In this case the test may use the extensions if the --fast
0059 option is used.
0060 
0061 Useful options to all test-scripts (and run-all-tests):
0062 
0063 --host=#        Hostname for MySQL server (default: localhost)
0064 --db=#          Database to use (default: test)
0065 --fast          Allow use of any non-standard SQL extension to
0066                 get things done faster.
0067 --lock-tables   Use table locking to get more speed.
0068 
0069 From a text at http://www.mgt.ncu.edu.tw/CSIM/Paper/sixth/11.html:
0070 
0071 The Wisconsin Benchmark
0072 
0073 The Wisconsin Benchmark described in [Bitton, DeWitt, and Turbyfill
0074 1983] [Boral and DeWitt 1984] [Bitton and Turbyfill 1985] [Bitton and
0075 Turbyfill 1988], and [DeWitt 1993] is the first effort to
0076 systematically measure and compare the performance of relational
0077 database systems with database machines.  The benchmark is a
0078 single-user and single-factor experiment using a synthetic database
0079 and a controlled workload.  It measures the query optimization
0080 performance of database systems with 32 query types to exercise the
0081 components of the proposed systems.  The query suites include
0082 selection, join, projection, aggregate, and simple update queries.
0083 
0084 The test database consists of four generic relations.  The tenk
0085 relation is the key table and most used. Two data types of small
0086 integer numbers and character strings are utilized.  Data values are
0087 uniformly distributed. The primary metric is the query elapsed
0088 time. The main criticisms of the benchmark include the nature of
0089 single-user workload, the simplistic database structure, and the
0090 unrealistic query tests.  A number of efforts have been made to extend
0091 the benchmark to incorporate the multi-user test.  However, they do
0092 not receive the same acceptance as the original Wisconsin benchmark
0093 except an extension work called the AS3AP benchmark.