DOCS-173: clickhouse benchmark (#7947)

* CLICKHOUSEDOCS-173: Draft of a description.

* CLICKHOUSEDOCS-173: Table experiment.

* CLICKHOUSEDOCS-173: Further writing.

* CLICKHOUSEDOCS-173: Final.

* CLICKHOUSEDOCS-173: Updated by comments.

* CLICKHOUSEDOCS-173: Updated by comments.

* CLICKHOUSEDOCS-173: Supported other languages.

docs/en/operations/utils/clickhouse-benchmark.md

@@ -0,0 +1,146 @@
+# clickhouse-benchmark
+
+Connects to a ClickHouse server and repeatedly sends specified queries.
+
+Syntax:
+
+```bash
+$ echo "single query" | clickhouse-benchmark [keys]
+```
+or
+```bash
+$ clickhouse-benchmark [keys] <<< "single query"
+```
+
+If you want to send a set of queries, create a text file and place each query on the individual string in this file. For example:
+
+```sql
+SELECT * FROM system.numbers LIMIT 10000000
+SELECT 1
+```
+
+Then pass this file to a standard input of `clickhouse-benchmark`.
+
+```bash
+clickhouse-benchmark [keys] < queries_file
+```
+
+## Keys {#clickhouse-benchmark-keys}
+
+- `-c N`, `--concurrency=N` — Number of queries that `clickhouse-benchmark` sends simultaneously. Default value: 1.
+- `-d N`, `--delay=N` — Interval in seconds between intermediate reports (set 0 to disable reports). Default value: 1.
+- `-h WORD`, `--host=WORD` — Server host. Default value: `localhost`. For the [comparison mode](#clickhouse-benchmark-comparison-mode) you can use multiple `-h` keys.
+- `-p N`, `--port=N` — Server port. Default value: 9000. For the [comparison mode](#clickhouse-benchmark-comparison-mode) you can use multiple `-p` keys.
+- `-i N`, `--iterations=N` — Total number of queries. Default value: 0.
+- `-r`, `--randomize` — Random order of queries execution if there is more then one input query.
+- `-s`, `--secure` — Using TLS connection.
+- `-t N`, `--timelimit=N` — Time limit in seconds. `clickhouse-benchmark` stops sending queries when the specified time limit is reached. Default value: 0 (time limit disabled).
+- `--confidence=N` — Level of confidence for T-test. Possible values: 0 (80%), 1 (90%), 2 (95%), 3 (98%), 4 (99%), 5 (99.5%). Default value: 5. In the [comparison mode](#clickhouse-benchmark-comparison-mode) `clickhouse-benchmark` performs the [Independent two-sample Student's t-test](https://en.wikipedia.org/wiki/Student%27s_t-test#Independent_two-sample_t-test) test to determine whether the two distributions aren't different with the selected level of confidence.
+- `--cumulative` — Printing cumulative data instead of data per interval.
+- `--database=DATABASE_NAME` — ClickHouse database name. Default value: `default`.
+- `--json=FILEPATH` — JSON output. When the key is set, `clickhouse-benchmark` outputs a report to the specified JSON-file.
+- `--user=USERNAME` — ClickHouse user name. Default value: `default`.
+- `--password=PSWD` — ClickHouse user password. Default value: empty string.
+- `--stacktrace` — Stack traces output. When the key is set, `clickhouse-bencmark` outputs stack traces of exceptions.
+- `--stage=WORD` — Query processing stage at server. ClickHouse stops query processing and returns answer to `clickhouse-benchmark` at the specified stage. Possible values: `complete`, `fetch_columns`, `with_mergeable_state`. Default value: `complete`.
+- `--help` — Shows the help message.
+
+If you want to apply some [settings](../../operations/settings/index.md) for queries, pass them as a key `--<session setting name>= SETTING_VALUE`. For example, `--max_memory_usage=1048576`.
+
+## Output {#clickhouse-benchmark-output}
+
+By default, `clickhouse-benchmark` reports for each `--delay` interval.
+
+Example of the report:
+
+```text
+Queries executed: 10.
+
+localhost:9000, queries 10, QPS: 6.772, RPS: 67904487.440, MiB/s: 518.070, result RPS: 67721584.984, result MiB/s: 516.675.
+
+0.000%		0.145 sec.	
+10.000%		0.146 sec.	
+20.000%		0.146 sec.	
+30.000%		0.146 sec.	
+40.000%		0.147 sec.	
+50.000%		0.148 sec.	
+60.000%		0.148 sec.	
+70.000%		0.148 sec.	
+80.000%		0.149 sec.	
+90.000%		0.150 sec.	
+95.000%		0.150 sec.	
+99.000%		0.150 sec.	
+99.900%		0.150 sec.	
+99.990%		0.150 sec.	
+```
+
+In the report you can find:
+
+- Number of queries in the `Queries executed: ` field.
+- Status string containing (in order):
+
+    - Endpoint of ClickHouse server.
+    - Number of processed queries.
+    - QPS: QPS: How many queries server performed per second during a period specified in the `--delay` argument.
+    - RPS: How many rows server read per second during a period specified in the `--delay` argument.
+    - MiB/s: How many mebibytes server read per second during a period specified in the `--delay` argument.
+    - result RPS: How many rows placed by server to the result of a query per second during a period specified in the `--delay` argument.
+    - result MiB/s. How many mebibytes placed by server to the result of a query per second during a period specified in the `--delay` argument.
+
+- Percentiles of queries execution time.
+
+
+## Comparison mode {#clickhouse-benchmark-comparison-mode}
+
+`clickhouse-benchmark` can compare performances for two running ClickHouse servers. 
+
+To use the comparison mode, specify endpoints of both servers by two pairs of `--host`, `--port` keys. Keys matched together by position in arguments list, the first `--host` is matched with the first `--port` and so on. `clickhouse-benchmark` establishes connections to both servers, then sends queries. Each query addressed to a randomly selected server. The results are shown for each server separately.
+
+## Example {#clickhouse-benchmark-example}
+
+```bash
+$ echo "SELECT * FROM system.numbers LIMIT 10000000 OFFSET 10000000" | clickhouse-benchmark -i 10
+```
+```text
+Loaded 1 queries.
+
+Queries executed: 6.
+
+localhost:9000, queries 6, QPS: 6.153, RPS: 123398340.957, MiB/s: 941.455, result RPS: 61532982.200, result MiB/s: 469.459.
+
+0.000%		0.159 sec.	
+10.000%		0.159 sec.	
+20.000%		0.159 sec.	
+30.000%		0.160 sec.	
+40.000%		0.160 sec.	
+50.000%		0.162 sec.	
+60.000%		0.164 sec.	
+70.000%		0.165 sec.	
+80.000%		0.166 sec.	
+90.000%		0.166 sec.	
+95.000%		0.167 sec.	
+99.000%		0.167 sec.	
+99.900%		0.167 sec.	
+99.990%		0.167 sec.	
+
+
+
+Queries executed: 10.
+
+localhost:9000, queries 10, QPS: 6.082, RPS: 121959604.568, MiB/s: 930.478, result RPS: 60815551.642, result MiB/s: 463.986.
+
+0.000%		0.159 sec.	
+10.000%		0.159 sec.	
+20.000%		0.160 sec.	
+30.000%		0.163 sec.	
+40.000%		0.164 sec.	
+50.000%		0.165 sec.	
+60.000%		0.166 sec.	
+70.000%		0.166 sec.	
+80.000%		0.167 sec.	
+90.000%		0.167 sec.	
+95.000%		0.170 sec.	
+99.000%		0.172 sec.	
+99.900%		0.172 sec.	
+99.990%		0.172 sec.	
+```

docs/en/operations/utils/index.md

@@ -1,7 +1,7 @@
 # ClickHouse Utility
 
-* [clickhouse-local](clickhouse-local.md) — Allows running SQL queries on data without stopping the ClickHouse server, similar to how `awk` does this.
-* [clickhouse-copier](clickhouse-copier.md) — Copies (and reshards) data from one cluster to another cluster.
-
+- [clickhouse-local](clickhouse-local.md) — Allows running SQL queries on data without stopping the ClickHouse server, similar to how `awk` does this.
+- [clickhouse-copier](clickhouse-copier.md) — Copies (and reshards) data from one cluster to another cluster.
+- [clickhouse-benchmark](clickhouse-benchmark.md) — Loads server with the custom queries and settings.
 
 [Original article](https://clickhouse.yandex/docs/en/operations/utils/) <!--hide-->

docs/fa/operations/utils/clickhouse-benchmark.md

@@ -0,0 +1 @@
+../../../en/operations/utils/clickhouse-benchmark.md

docs/ja/operations/utils/clickhouse-benchmark.md

@@ -0,0 +1 @@
+../../../en/operations/utils/clickhouse-benchmark.md

docs/toc_en.yml

@@ -212,6 +212,7 @@ nav:
     - 'Overview': 'operations/utils/index.md'
     - 'clickhouse-copier': 'operations/utils/clickhouse-copier.md'
     - 'clickhouse-local': 'operations/utils/clickhouse-local.md'
+    - 'clickhouse-benchmark': 'operations/utils/clickhouse-benchmark.md'
 
 - 'Development':
   - 'hidden': 'development/index.md'

docs/toc_fa.yml

@@ -208,7 +208,8 @@ nav:
     - 'Overview': 'operations/utils/index.md'
     - 'clickhouse-copier': 'operations/utils/clickhouse-copier.md'
     - 'clickhouse-local': 'operations/utils/clickhouse-local.md'
-
+    - 'clickhouse-benchmark': 'operations/utils/clickhouse-benchmark.md'
+    
 - 'F.A.Q.':
   - 'General Questions': 'faq/general.md'
 

docs/toc_ja.yml

@@ -212,7 +212,8 @@ nav:
     - 'Overview': 'operations/utils/index.md'
     - 'clickhouse-copier': 'operations/utils/clickhouse-copier.md'
     - 'clickhouse-local': 'operations/utils/clickhouse-local.md'
-
+    - 'clickhouse-benchmark': 'operations/utils/clickhouse-benchmark.md'
+    
 - 'Development':
   - 'hidden': 'development/index.md'
   - 'Overview of ClickHouse Architecture': 'development/architecture.md'

docs/toc_zh.yml

@@ -207,7 +207,8 @@ nav:
     - '介绍': 'operations/utils/index.md'
     - 'clickhouse-copier': 'operations/utils/clickhouse-copier.md'
     - 'clickhouse-local': 'operations/utils/clickhouse-local.md'
-
+    - 'clickhouse-benchmark': 'operations/utils/clickhouse-benchmark.md'
+    
 - '常见问题':
   - '一般的问题': 'faq/general.md'
 

docs/zh/operations/utils/clickhouse-benchmark.md

@@ -0,0 +1 @@
+../../../en/operations/utils/clickhouse-benchmark.md

docs/zh/operations/utils/index.md

@@ -1,7 +1,7 @@
 # ClickHouse Utility
 
-* [clickhouse-local](clickhouse-local.md) — Allows running SQL queries on data without stopping the ClickHouse server, similar to how `awk` does this.
-* [clickhouse-copier](clickhouse-copier.md) — Copies (and reshards) data from one cluster to another cluster.
-
+- [clickhouse-local](clickhouse-local.md) — Allows running SQL queries on data without stopping the ClickHouse server, similar to how `awk` does this.
+- [clickhouse-copier](clickhouse-copier.md) — Copies (and reshards) data from one cluster to another cluster.
+- [clickhouse-benchmark](clickhouse-benchmark.md) — Loads server with the custom queries and settings.
 
 [Original article](https://clickhouse.yandex/docs/en/operations/utils/) <!--hide-->