ClickHouse/programs/keeper-bench

Keeper Bench

Keeper Bench is a tool for benchmarking Keeper or any ZooKeeper compatible systems.

To run it call following command from the build folder:

./utils/keeper-bench --config benchmark_config_file.yaml

Configuration file

Keeper Bench runs need to be configured inside a yaml or XML file. An example of a configuration file can be found in ./utils/keeper-bench/example.yaml

Table of contents

• Special Types
• General settings
• Connections
• Generator
• Output

Special types

IntegerGetter

Can be defined with constant integer or as a random value from a range.

key: integer
key:
    min_value: integer
    max_value: integer

Example for a constant value:

some_key: 2

Example for random value from [10, 20]:

some_key:
    min_value: 10
    max_value: 20

StringGetter

Can be defined with constant string or as a random string of some size.

key: string
key:
    random_string:
        size: IntegerGetter

Example for a constant value:

some_key: "string"

Example for a random string with a random size from [10, 20]:

some_key:
    random_string:
        size:
            min_value: 10
            max_value: 20

PathGetter

If a section contains one or more path keys, all path keys are collected into a list.
Additionally, paths can be defined with key children_of which will add all children of some path to the list.

path: string
path:
    children_of: string

Example for defining list of paths (/path1, /path2 and children of /path3):

main:
    path:
        - "/path1"
        - "/path2"
    path:
        children_of: "/path3"

General settings

# number of parallel queries (default: 1)
concurrency: integer

# amount of queries to be executed, set 0 to disable limit (default: 0)
iterations: integer

# delay between intermediate reports in seconds, set 0 to disable reports (default: 1.0)
report_delay: double

# stop launch of queries after specified time limit, set 0 to disable limit (default: 0)
timelimit: double

# continue testing even if a query fails (default: false)
continue_on_errors: boolean

Connections

Connection definitions that will be used throughout tests defined under connections key.

Following configurations can be defined under connections key or for each specific connection.
If it's defined under connections key, it will be used by default unless a specific connection overrides it.

secure: boolean
operation_timeout_ms: integer
session_timeout_ms: integer
connection_timeout_ms: integer

Specific configuration can be defined with a string or with a detailed description.

host: string
connection:
    host: string

    # number of sessions to create for host
    sessions: integer
    # any connection configuration defined above

Example definition of 3 connections in total, 1 to localhost:9181 and 2 to localhost:9182 both will use secure connections:

connections:
    secure: true

    host: "localhost:9181"
    connection:
        host: "localhost:9182"
        sessions: 2

Generator

Main part of the benchmark is the generator itself which creates necessary nodes and defines how the requests will be generated.
It is defined under generator key.

Setup

Setup defines nodes that are needed for test, defined under setup key.

Each node is defined with a node key in the following format:

node: StringGetter

node:
    name: StringGetter
    data: StringGetter
    repeat: integer
    node: Node

If only string is defined, a node with that name will be created.
Otherwise more detailed definition could be included to set data or the children of the node.
If repeat key is set, the node definition will be used multiple times. For a repeat key to be valid, the name of the node needs to be a random string.

Example for a setup:

generator:
    setup:
        node: "node1"
            node:
                name:
                    random_string:
                        size: 20
                data: "somedata"
                repeat: 4
        node:
            name:
                random_string:
                    size: 10
            repeat: 2

We will create node /node1 with no data and 4 children of random name of size 20 and data set to somedata.
We will also create 2 nodes with no data and random name of size 10 under / node.

Requests

While benchmark is running, we are generating requests.

Request generator is defined under requests key.
For each request weight (default: 1) can be defined which defines preference for a certain request.

create

create:
    # parent path for created nodes
    path: string

    # length of the name for the create node (default: 5)
    name_length: IntegerGetter

    # data for create nodes (default: "")
    data: StringGetter

    # value in range [0.0, 1.0> denoting how often a remove request should be generated compared to create request (default: 0)
    remove_factor: double

set

set:
    # paths on which we randomly set data
    path: PathGetter

    # data to set
    data: StringGetter

get

get:
    # paths for which we randomly get data
    path: PathGetter

list

list:
    # paths for which we randomly do list request
    path: PathGetter

multi

multi:
    # any request definition defined above can be added

    # optional size for the multi request
    size: IntegerGetter

Multi request definition can contain any other request generator definitions described above.
If size key is defined, we will randomly pick size amount of requests from defined request generators.
All request generators can have a higher pick probability by using weight key.
If size is not defined, multi request with same request generators will always be generated.
Both write and read multi requests are supported.

Example

generator:
    requests:
        create:
            path: "/test_create"
            name_length:
                min_value: 10
                max_value: 20
        multi:
            weight: 20
            size: 10
            get:
                path:
                    children_of: "/test_get1"
            get:
                weight: 2
                path:
                    children_of: "/test_get2"

We defined a request geneator that will generate either a create or a multi request.
Each create request will create a node under /test_create with a randomly generated name with size from range [10, 20].
multi request will be generated 20 times more than create request.
multi request will contain 10 requests and approximately twice as much get requests to children of "/test_get2".

Output

output:
    # if defined, JSON output of results will be stored at the defined path
    file: string
    # or
    file:
        # if defined, JSON output of results will be stored at the defined path
        path: string

        # if set to true, timestamp will be appended to the output file name (default: false)
        with_timestamp: boolean

    # if set to true, output will be printed to stdout also (default: false)
    stdout: boolean