ClickHouse/utils/keeper-bench/README.md

Ignoring revisions in .git-blame-ignore-revs. Click here to bypass and see the normal blame view.

318 lines
7.4 KiB
Markdown
Raw Normal View History

2023-04-17 14:09:54 +00:00
# Keeper Bench
Keeper Bench is a tool for benchmarking Keeper or any ZooKeeper compatible systems.
To run it call following command from the build folder:
```
./utils/keeper-bench --config benchmark_config_file.yaml
```
## Configuration file
Keeper Bench runs need to be configured inside a yaml or XML file.
An example of a configuration file can be found in `./utils/keeper-bench/example.yaml`
### Table of contents
- [Special Types](#special-types)
- [General settings](#general-settings)
- [Connections](#connections)
- [Generator](#generator)
- [Output](#output)
<a name="special-types"></a>
## Special types
### IntegerGetter
Can be defined with constant integer or as a random value from a range.
```yaml
key: integer
key:
min_value: integer
max_value: integer
```
Example for a constant value:
```yaml
some_key: 2
```
Example for random value from [10, 20]:
```yaml
some_key:
min_value: 10
max_value: 20
```
### StringGetter
Can be defined with constant string or as a random string of some size.
```yaml
key: string
key:
random_string:
size: IntegerGetter
```
Example for a constant value:
```yaml
some_key: "string"
```
Example for a random string with a random size from [10, 20]:
```yaml
some_key:
random_string:
size:
min_value: 10
max_value: 20
```
### PathGetter
If a section contains one or more `path` keys, all `path` keys are collected into a list. \
Additionally, paths can be defined with key `children_of` which will add all children of some path to the list.
```yaml
path: string
path:
children_of: string
```
Example for defining list of paths (`/path1`, `/path2` and children of `/path3`):
```yaml
main:
path:
- "/path1"
- "/path2"
path:
children_of: "/path3"
```
<a name="general-settings"></a>
## General settings
```yaml
# number of parallel queries (default: 1)
concurrency: integer
# amount of queries to be executed, set 0 to disable limit (default: 0)
iterations: integer
# delay between intermediate reports in seconds, set 0 to disable reports (default: 1.0)
report_delay: double
# stop launch of queries after specified time limit, set 0 to disable limit (default: 0)
timelimit: double
# continue testing even if a query fails (default: false)
continue_on_errors: boolean
```
<a name="connections"></a>
## Connections
Connection definitions that will be used throughout tests defined under `connections` key.
Following configurations can be defined under `connections` key or for each specific connection. \
If it's defined under `connections` key, it will be used by default unless a specific connection overrides it.
```yaml
secure: boolean
operation_timeout_ms: integer
session_timeout_ms: integer
connection_timeout_ms: integer
```
Specific configuration can be defined with a string or with a detailed description.
```yaml
host: string
connection:
host: string
# number of sessions to create for host
sessions: integer
# any connection configuration defined above
```
Example definition of 3 connections in total, 1 to `localhost:9181` and 2 to `localhost:9182` both will use secure connections:
```yaml
connections:
secure: true
host: "localhost:9181"
connection:
host: "localhost:9182"
sessions: 2
```
<a name="generator"></a>
## Generator
Main part of the benchmark is the generator itself which creates necessary nodes and defines how the requests will be generated. \
It is defined under `generator` key.
### Setup
Setup defines nodes that are needed for test, defined under `setup` key.
Each node is defined with a `node` key in the following format:
```yaml
node: StringGetter
node:
name: StringGetter
data: StringGetter
repeat: integer
node: Node
```
If only string is defined, a node with that name will be created. \
Otherwise more detailed definition could be included to set data or the children of the node. \
If `repeat` key is set, the node definition will be used multiple times. For a `repeat` key to be valid, the name of the node needs to be a random string.
Example for a setup:
```yaml
generator:
setup:
node: "node1"
node:
name:
random_string:
size: 20
data: "somedata"
repeat: 4
node:
name:
random_string:
size: 10
repeat: 2
```
We will create node `/node1` with no data and 4 children of random name of size 20 and data set to `somedata`. \
We will also create 2 nodes with no data and random name of size 10 under `/` node.
### Requests
While benchmark is running, we are generating requests.
Request generator is defined under `requests` key. \
For each request `weight` (default: 1) can be defined which defines preference for a certain request.
#### `create`
```yaml
create:
# parent path for created nodes
path: string
# length of the name for the create node (default: 5)
name_length: IntegerGetter
# data for create nodes (default: "")
data: StringGetter
# value in range [0.0, 1.0> denoting how often a remove request should be generated compared to create request (default: 0)
remove_factor: double
```
#### `set`
```yaml
set:
# paths on which we randomly set data
path: PathGetter
# data to set
data: StringGetter
```
#### `get`
```yaml
get:
# paths for which we randomly get data
path: PathGetter
```
#### `list`
```yaml
list:
# paths for which we randomly do list request
path: PathGetter
```
#### `multi`
```yaml
multi:
# any request definition defined above can be added
# optional size for the multi request
size: IntegerGetter
```
Multi request definition can contain any other request generator definitions described above. \
If `size` key is defined, we will randomly pick `size` amount of requests from defined request generators. \
All request generators can have a higher pick probability by using `weight` key. \
If `size` is not defined, multi request with same request generators will always be generated. \
Both write and read multi requests are supported.
#### Example
```yaml
generator:
requests:
create:
path: "/test_create"
name_length:
min_value: 10
max_value: 20
multi:
weight: 20
size: 10
get:
path:
children_of: "/test_get1"
get:
weight: 2
path:
children_of: "/test_get2"
```
We defined a request geneator that will generate either a `create` or a `multi` request. \
Each `create` request will create a node under `/test_create` with a randomly generated name with size from range `[10, 20]`. \
`multi` request will be generated 20 times more than `create` request. \
`multi` request will contain 10 requests and approximately twice as much get requests to children of "/test_get2".
<a name="output"></a>
## Output
```yaml
output:
# if defined, JSON output of results will be stored at the defined path
file: string
# or
file:
# if defined, JSON output of results will be stored at the defined path
path: string
# if set to true, timestamp will be appended to the output file name (default: false)
with_timestamp: boolean
# if set to true, output will be printed to stdout also (default: false)
stdout: boolean
```