Oltpbench and valgrind

Instructions for performance data collection for valgrind / kcachegrind from Oltpbench against Peloton

---

Requirements and preparation

1. You need an oltpbench build. Follow the steps in the installation section, here: OLTPBenchmark

2. You need a Peloton release build with debug symbols. When configuring the build type, rather than Debug or Release, you need:

cmake -DCMAKE_BUILD_TYPE=RelWithDebInfo ..

Benchmark configuration

1. Select the benchmark to be run. To simplify the discussion, we'll use YCSB as the example in these instructions.

2. For the selected benchmark:

• You MUST create a configuration file. This defines how oltpbench connects to the database server and the parameters of the benchmark such as scale, duration and mix of operations.

• Consider what you want to exercise in the database e.g. sequential scans vs. index scans. This can be controlled by adjusting the DDL (the table schemas) that oltpbench uses in running the benchmark. If you have no special considerations, you can use let oltpbench use the default DDLs.

Creating the configuration file

See the initial portion of the "Running the Benchmark" section here: OLTPBenchmark Start with the sample file for the selected benchmark from oltpbench/config, e.g. oltpbench/config/sample_ycsb_config.xml and make a copy of the file for your task.

The fields you'll want to examine and modify are:

• dbtype. In this example, it needs to be peloton
• driver
• DBUrl: modify the database type, port and database name. For Peloton, use default_database for database name.
• username
• password
• isolation
• scalefactor. In selecting scalefactor, be aware that running under valgrind slows down execution a lot, at least by an order of magnitude. If you would normally use a scalefactor of 1, you might need for instance, to use a scale factor of 0.1 to keep execution time from being excessive.
• terminals
• time
• rate

If you want to change the default mix of operations, e.g. to exercise only reads, you'll need to adjust

• weights

otherwise weights can be left alone

A portion of a config file for YCSB is shown below. It shows only the initial, modified portion of the file.

<parameters>
    <dbtype>peloton</dbtype>
    <driver>org.postgresql.Driver</driver>
    <DBUrl>jdbc:postgresql://localhost:15721/default_database</DBUrl>
    <username>postgres</username>
    <password>postgres</password>
    <isolation>TRANSACTION_SERIALIZABLE</isolation>
    <scalefactor>1</scalefactor> 
    <terminals>1</terminals>
    <works>
        <work>
          <time>20</time>
          <rate>unlimited</rate>
          <weights>50,5,15,10,10,10</weights>
        </work>
    </works>

Modifying the DDLs. These may be found in the source tree in a benchmark specific directory. For YCSB:

oltpbench/src/com/oltpbenchmark/benchmarks/ycsb/ddls

Oltpbench will select the database specific DDL if it exists, e.g. ycsb-peloton-ddl.sql, otherwise it will use the generic ddl, e.g. ycsb-ddl.sql.

The YCSB generic DDL looks like:

DROP TABLE IF EXISTS USERTABLE;
CREATE TABLE USERTABLE (
    YCSB_KEY INT PRIMARY KEY,
    FIELD1 VARCHAR(100), 
  	FIELD2 VARCHAR(100),
  	FIELD3 VARCHAR(100), 
  	FIELD4 VARCHAR(100),
  	FIELD5 VARCHAR(100), 
  	FIELD6 VARCHAR(100),
  	FIELD7 VARCHAR(100), 
  	FIELD8 VARCHAR(100),
  	FIELD9 VARCHAR(100), 
  	FIELD10 VARCHAR(100)
);

During execution of the benchmark, the field YCSB_KEY INT PRIMARY KEY will result in an index being created and index scans being used. If we wish instead, to force the database to use sequential scans, we would:

1. Modify the relevant line in the DDL to be `YCSB_KEY INT,
2. Rebuild oltpbench as follows:

• ant clean
• ant

The DDLs used during execution of the benchmark are taken from the build tree. You MUST clean and rebuild, otherwise the modification of the DDL in the source tree will NOT be used at runtime.

Running and collecting traces

1. Start Peloton (from the bin directory). This starts it with data collection disabled.

   valgrind --tool=callgrind --trace-children=yes --instr-atstart=no ./peloton

2. Now run the oltbench loading phase from the oltpbench directory, in the following style:

  ./oltpbenchmark -b ycsb -c <path to your ycsb_config.xml> --histograms
     --create=true --load=true -s 5  -o ycsb_output

Adjust your oltpbenchmark arguments as desired. For instance, you may not need --histograms.

Actual invocation e.g.

  ./oltpbenchmark -b ycsb -c ../../oltp_configs/ycsb_config.xml --histograms
     --create=true --load=true -s 5  -o ycsb_output

3. Once the loading is complete, turn on data collection:

    callgrind_control -i on

4. Now run the execution phase of the benchmark. Again, substituting the path to your config file:

   ./oltpbenchmark -b ycsb -c ../../oltp_configs/ycsb_config.xml \
   --histograms --execute=true -s 5  -o ycsb_output

5. Now terminate Peloton. Issue ps axww|grep peloton and identify the pid of the Valgrind/Peloton process

   kill -TERM <pid>

You should now have a callgrind output file in the peloton bin directory.