From 048ea93c09bc4fff737c79f635de4b1e1353d36b Mon Sep 17 00:00:00 2001 From: kst-morozov Date: Fri, 31 Mar 2023 17:59:19 +0200 Subject: [PATCH 1/2] upadte doc --- .../settings.md | 2 +- .../settings/merge-tree-settings.md | 26 +++++++++++++++++++ 2 files changed, 27 insertions(+), 1 deletion(-) diff --git a/docs/en/operations/server-configuration-parameters/settings.md b/docs/en/operations/server-configuration-parameters/settings.md index 08be318f334..3d579b14366 100644 --- a/docs/en/operations/server-configuration-parameters/settings.md +++ b/docs/en/operations/server-configuration-parameters/settings.md @@ -1045,7 +1045,7 @@ Default value: `0`. Sets the number of threads performing background merges and mutations for tables with MergeTree engines. This setting is also could be applied at server startup from the `default` profile configuration for backward compatibility at the ClickHouse server start. You can only increase the number of threads at runtime. To lower the number of threads you have to restart the server. By adjusting this setting, you manage CPU and disk load. Smaller pool size utilizes less CPU and disk resources, but background processes advance slower which might eventually impact query performance. -Before changing it, please also take a look at related MergeTree settings, such as `number_of_free_entries_in_pool_to_lower_max_size_of_merge` and `number_of_free_entries_in_pool_to_execute_mutation`. +Before changing it, please also take a look at related MergeTree settings, such as [number_of_free_entries_in_pool_to_lower_max_size_of_merge](../../operations/settings/merge-tree-settings.md#number-of-free-entries-in-pool-to-lower-max-size-of-merge) and [number_of_free_entries_in_pool_to_execute_mutation](../../operations/settings/merge-tree-settings.md#number-of-free-entries-in-pool-to-execute-mutation). Possible values: diff --git a/docs/en/operations/settings/merge-tree-settings.md b/docs/en/operations/settings/merge-tree-settings.md index 5bc174727ad..f36694cb81b 100644 --- a/docs/en/operations/settings/merge-tree-settings.md +++ b/docs/en/operations/settings/merge-tree-settings.md @@ -553,6 +553,32 @@ Default value: 8192 Merge reads rows from parts in blocks of `merge_max_block_size` rows, then merges and writes the result into a new part. The read block is placed in RAM, so `merge_max_block_size` affects the size of the RAM required for the merge. Thus, merges can consume a large amount of RAM for tables with very wide rows (if the average row size is 100kb, then when merging 10 parts, (100kb * 10 * 8192) = ~ 8GB of RAM). By decreasing `merge_max_block_size`, you can reduce the amount of RAM required for a merge but slow down a merge. +## number_of_free_entries_in_pool_to_lower_max_size_of_merge {#number-of-free-entries-in-pool-to-lower-max-size-of-merge} + +When there is less than specified number of free entries in pool (or replicated queue), start to lower maximum size of merge to process (or to put in queue). +This is to allow small merges to process - not filling the pool with long running merges. + +Possible values: + +- Any positive integer. + +Default value: 8 + +## number_of_free_entries_in_pool_to_execute_mutation {#number-of-free-entries-in-pool-to-execute-mutation} + +When there is less than specified number of free entries in pool, do not execute part mutations. +This is to leave free threads for regular merges and avoid "Too many parts". + +Possible values: + +- Any positive integer. + +Default value: 20 + +**Usage** + +The value of the `number_of_free_entries_in_pool_to_execute_mutation` setting should be less than the value of the [background_pool_size](../../operations/settings/server-configuration-parameters/settings.md#background_pool_size) * [background_pool_size](../../operations/settings/server-configuration-parameters/settings.md#background_merges_mutations_concurrency_ratio). Otherwise, ClickHouse throws an exception. + ## max_part_loading_threads {#max-part-loading-threads} The maximum number of threads that read parts when ClickHouse starts. From 4cb3c9260771a31199f2d793480f7ac3fa3a42fa Mon Sep 17 00:00:00 2001 From: Rich Raposa Date: Fri, 31 Mar 2023 12:38:45 -0600 Subject: [PATCH 2/2] Update merge-tree-settings.md For some reason the links are broken - I'm attempting to fix them here --- docs/en/operations/settings/merge-tree-settings.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/en/operations/settings/merge-tree-settings.md b/docs/en/operations/settings/merge-tree-settings.md index 41e66679874..4008b71ef8b 100644 --- a/docs/en/operations/settings/merge-tree-settings.md +++ b/docs/en/operations/settings/merge-tree-settings.md @@ -577,7 +577,7 @@ Default value: 20 **Usage** -The value of the `number_of_free_entries_in_pool_to_execute_mutation` setting should be less than the value of the [background_pool_size](../../operations/settings/server-configuration-parameters/settings.md#background_pool_size) * [background_pool_size](../../operations/settings/server-configuration-parameters/settings.md#background_merges_mutations_concurrency_ratio). Otherwise, ClickHouse throws an exception. +The value of the `number_of_free_entries_in_pool_to_execute_mutation` setting should be less than the value of the [background_pool_size](/docs/en/operations/server-configuration-parameters/settings#background_pool_size) * [background_pool_size](/docs/en/operations/server-configuration-parameters/settings#background_merges_mutations_concurrency_ratio). Otherwise, ClickHouse throws an exception. ## max_part_loading_threads {#max-part-loading-threads}