client side

Description

The client side consists of a driver and a controller. The driver controls all the control flow of the client's side and the controller collects DBMS knobs and metrics data.

• 1 - The driver first clears cache and restarts the database. It also checks the disk usage exceeds the maximum usage (a threshold defined by the user). If so, the driver will drop the experiment database and reload the benchmark to ensure there is enough space to carry out the experiment and for the controller to collect DBMS knobs and metrics data.
• 2 - The driver then runs the oltpbenchmark as a background job. oltpbenchmark is a benchmark suite for relational databases (https://github.com/oltpbenchmark/oltpbench) . When the benchmark is ready to begin measurement, the driver sends a signal to the controller to start the experiment and waits for the benchmark to finish. The length of the experiment's observation time is defined in the benchmark configuration file.
• 3 - The controller first collects knobs and metrics data before the observation. Then it waits until the driver sends another signal.
• 4 - After the oltpbenchmark finishes its measurement, the driver sends a terminate signal to the controller. Upon receiving the signal, the controller collects the metrics data again. The controller will format all collected knobs and metrics data along with a metadata summary (i.e. database name and version, observation length, start/end time, workload name) as json files and send those files back to the driver.
• 5 - The driver uploads all the DBMS data the controller collected to the server. The driver will receive a unique token from the server and use the token to check periodically whether the server has finished with a recommended new configuration.
• 6 - If the server has successfully generated the next configuration, the driver will query the new configuration from the server and install it to the database.

Get Started

Controller Configuration

Sample configuration files are under the directory ottertune/client/controller/config/. You need to update the info in your file. Currently you do not need to care about upload_code and upload_url in controller configuration file, because the driver will handle uploading results to the server and you will set them in driver configuration file. To specify the path of controller configuration, you may want to change controller_config in driver configuration.

Driver Configuration

ottertune/client/driver/driver_config.json

OLTP-Bench Configuration

Installation

Ubuntu

See Linux Quick Start in the Wiki Page.

Amazon Linux/CentOS

• Install and start Postgres. (Note: You can use Postgres 9.2 ~ 9.6 or Postgres 10.0. Here is an example of installing Postgres 9.3.)

sudo yum install postgresql93-devel.x86_64 postgresql93.x86_64 postgresql93-server.x86_64 postgresql-server

sudo service postgresql93 initdb

sudo service postgresql93 start

• Install git and ant

sudo yum install git.x86_64 ant.noarch

• Install fabric.

sudo pip install fabric==1.10.2

• Install java. (Note: you may need both Java 1.7 and Java 1.8. oltpbench compiles well with Java 1.7 and the driver currently only supports Java 1.8.)

sudo yum install java-1.7.0-openjdk.x86_64 java-1.7.0-openjdk-devel.x86_64 java-1.8.0-openjdk.x86_64 java-1.8.0-openjdk-devel.x86_64

• Install gradle.

wget https://services.gradle.org/distributions/gradle-3.4.1-bin.zip

sudo mkdir /opt/gradle

sudo unzip -d /opt/gradle gradle-3.4.1-bin.zip

export PATH=$PATH:/opt/gradle/gradle-3.4.1/bin