From 10bcc028dfceff7a8e230f8be23fc258c5fe419a Mon Sep 17 00:00:00 2001 From: vdimir Date: Thu, 14 Sep 2023 10:41:57 +0000 Subject: [PATCH] Add cluster discovery documentation --- .../en/engines/database-engines/replicated.md | 2 + docs/en/operations/cluster-discovery.md | 171 ++++++++++++++++++ .../settings.md | 12 +- 3 files changed, 180 insertions(+), 5 deletions(-) create mode 100644 docs/en/operations/cluster-discovery.md diff --git a/docs/en/engines/database-engines/replicated.md b/docs/en/engines/database-engines/replicated.md index 1f90f2acabf..27bdc20ec57 100644 --- a/docs/en/engines/database-engines/replicated.md +++ b/docs/en/engines/database-engines/replicated.md @@ -37,6 +37,8 @@ When creating a new replica of the database, this replica creates tables by itse [`ALTER TABLE FREEZE|ATTACH|FETCH|DROP|DROP DETACHED|DETACH PARTITION|PART`](../../sql-reference/statements/alter/partition.md) queries are allowed but not replicated. The database engine will only add/fetch/remove the partition/part to the current replica. However, if the table itself uses a Replicated table engine, then the data will be replicated after using `ATTACH`. +In case you need only configure a cluster without maintaining table replication, refer to [Cluster Discovery](../../operations/cluster-discovery.md) feature. + ## Usage Example {#usage-example} Creating a cluster with three hosts: diff --git a/docs/en/operations/cluster-discovery.md b/docs/en/operations/cluster-discovery.md new file mode 100644 index 00000000000..a925afac916 --- /dev/null +++ b/docs/en/operations/cluster-discovery.md @@ -0,0 +1,171 @@ +--- +slug: /en/operations/cluster-discovery +sidebar_label: Cluster Discovery +--- +# Cluster Discovery + +## Overview + +ClickHouse's Cluster Discovery feature simplifies cluster configuration by allowing nodes to automatically discover and register themselves without the need for explicit definition in the configuration files. This is especially beneficial in cases where the manual definition of each node becomes cumbersome. + +:::note + +Cluster Discovery is an experimental feature and can be changed or removed in future versions. +To enable it include the `allow_experimental_cluster_discovery` setting in your configuration file: + +```xml + + + 1 + + +``` +::: + +## Remote Servers Configuration + +### Traditional Manual Configuration + +Traditionally, in ClickHouse, each shard and replica in the cluster needed to be manually specified in the configuration: + +```xml + + + + + node1 + 9000 + + + node2 + 9000 + + + + + node3 + 9000 + + + node4 + 9000 + + + + + +``` + +### Using Cluster Discovery + +With Cluster Discovery, rather than defining each node explicitly, you simply specify a path in ZooKeeper. All nodes that register under this path in ZooKeeper will be automatically discovered and added to the cluster. + +```xml + + + + /clickhouse/discovery/cluster_name + + + +``` + +If you want to specify a shard number for a particular node, you can include the `` tag within the `` section: + +for `node1` and `node2`: + +```xml + + /clickhouse/discovery/cluster_name + 1 + +``` + +for `node3` and `node4`: + +```xml + + /clickhouse/discovery/cluster_name + 2 + +``` + +### Observer mode + + +Nodes configured in observer mode will not register themselves as replicas. +They will solely observe and discover other active replicas in the cluster without actively participating. +To enable observer mode, include the `` tag within the `` section: + +```xml + + /clickhouse/discovery/cluster_name + + +``` + + +## Use-Cases and Limitations + +As nodes are added or removed from the specified ZooKeeper path, they are automatically discovered or removed from the cluster without the need for configuration changes or server restarts. + +However, changes affect only cluster configuration, not the data or existing databases and tables. + +Consider the following example with a cluster of 3 nodes: + + +```xml + + + + /clickhouse/discovery/default_cluster + + + +``` + +``` +SELECT * EXCEPT (default_database, errors_count, slowdowns_count, estimated_recovery_time, database_shard_name, database_replica_name) +FROM system.clusters WHERE cluster = 'default'; + +┌─cluster─┬─shard_num─┬─shard_weight─┬─replica_num─┬─host_name────┬─host_address─┬─port─┬─is_local─┬─user─┬─is_active─┐ +│ default │ 1 │ 1 │ 1 │ 92d3c04025e8 │ 172.26.0.5 │ 9000 │ 0 │ │ ᴺᵁᴸᴸ │ +│ default │ 1 │ 1 │ 2 │ a6a68731c21b │ 172.26.0.4 │ 9000 │ 1 │ │ ᴺᵁᴸᴸ │ +│ default │ 1 │ 1 │ 3 │ 8e62b9cb17a1 │ 172.26.0.2 │ 9000 │ 0 │ │ ᴺᵁᴸᴸ │ +└─────────┴───────────┴──────────────┴─────────────┴──────────────┴──────────────┴──────┴──────────┴──────┴───────────┘ +``` + +```sql +CREATE TABLE event_table ON CLUSTER default (event_time DateTime, value String) +ENGINE = ReplicatedMergeTree('/clickhouse/tables/event_table', '{replica}') +ORDER BY event_time PARTITION BY toYYYYMM(event_time); + +INSERT INTO event_table ... +``` + +Then, we add a new node to the cluster, starting a new node with the same entry in the `remote_servers` section in a configuration file: + +``` +┌─cluster─┬─shard_num─┬─shard_weight─┬─replica_num─┬─host_name────┬─host_address─┬─port─┬─is_local─┬─user─┬─is_active─┐ +│ default │ 1 │ 1 │ 1 │ 92d3c04025e8 │ 172.26.0.5 │ 9000 │ 0 │ │ ᴺᵁᴸᴸ │ +│ default │ 1 │ 1 │ 2 │ a6a68731c21b │ 172.26.0.4 │ 9000 │ 1 │ │ ᴺᵁᴸᴸ │ +│ default │ 1 │ 1 │ 3 │ 8e62b9cb17a1 │ 172.26.0.2 │ 9000 │ 0 │ │ ᴺᵁᴸᴸ │ +│ default │ 1 │ 1 │ 4 │ b0df3669b81f │ 172.26.0.6 │ 9000 │ 0 │ │ ᴺᵁᴸᴸ │ +└─────────┴───────────┴──────────────┴─────────────┴──────────────┴──────────────┴──────┴──────────┴──────┴───────────┘ +``` + +The fourth node is participating in the cluster, but table `event_table` still exists only on the first three nodes: + + +```sql +SELECT hostname(), database, table FROM clusterAllReplicas(default, system.tables) WHERE table = 'event_table' FORMAT PrettyCompactMonoBlock + +┌─hostname()───┬─database─┬─table───────┐ +│ a6a68731c21b │ default │ event_table │ +│ 92d3c04025e8 │ default │ event_table │ +│ 8e62b9cb17a1 │ default │ event_table │ +└──────────────┴──────────┴─────────────┘ +``` + +If you need to have tables replicated on all the nodes, you may use the [Replicated](../engines/database-engines/replicated.md) database engine in alternative to cluster discovery. + diff --git a/docs/en/operations/server-configuration-parameters/settings.md b/docs/en/operations/server-configuration-parameters/settings.md index 7d0ab494926..b27586e705f 100644 --- a/docs/en/operations/server-configuration-parameters/settings.md +++ b/docs/en/operations/server-configuration-parameters/settings.md @@ -88,7 +88,7 @@ Default: 2 ## background_merges_mutations_scheduling_policy -The policy on how to perform a scheduling for background merges and mutations. Possible values are: `round_robin` and `shortest_task_first`. +The policy on how to perform a scheduling for background merges and mutations. Possible values are: `round_robin` and `shortest_task_first`. ## background_merges_mutations_scheduling_policy @@ -583,7 +583,7 @@ Both the cache for `local_disk`, and temporary data will be stored in `/tiny_loc Type: String -Default: +Default: ## thread_pool_queue_size @@ -640,7 +640,7 @@ When `/disk1` is full, temporary data will be stored on `/disk2`. ``` Type: String -Default: +Default: ## uncompressed_cache_policy @@ -1948,7 +1948,7 @@ If the table does not exist, ClickHouse will create it. If the structure of the 7500 1048576 8192 - 524288 + 524288 false ``` @@ -2236,6 +2236,8 @@ For the value of the `incl` attribute, see the section “[Configuration files]( **See Also** - [skip_unavailable_shards](../../operations/settings/settings.md#settings-skip_unavailable_shards) +- [Cluster Discovery](../../operations/cluster-discovery.md) +- [Replicated database engine](../../engines/database-engines/replicated.md) ## timezone {#server_configuration_parameters-timezone} @@ -2404,7 +2406,7 @@ This section contains the following parameters: * nearest_hostname - selects a ZooKeeper node with a hostname that is most similar to the server’s hostname. * first_or_random - selects the first ZooKeeper node, if it's not available then randomly selects one of remaining ZooKeeper nodes. * round_robin - selects the first ZooKeeper node, if reconnection happens selects the next. - + **Example configuration** ``` xml