Merge remote-tracking branch 'upstream/master' into fix-race-condition-between-inserts-and-dropping-mvs

docker/docs/builder/run.sh

@@ -25,6 +25,7 @@ done
 sed -i '/onBrokenMarkdownLinks:/ s/ignore/error/g' docusaurus.config.js
 
 if [[ $# -lt 1 ]] || [[ "$1" == "--"* ]]; then
+  export CI=true 
   exec yarn build "$@"
 fi
 

docs/en/engines/table-engines/integrations/index.md

@@ -6,7 +6,7 @@ sidebar_label:  Integrations
 
 # Table Engines for Integrations
 
-ClickHouse provides various means for integrating with external systems, including table engines. Like with all other table engines, the configuration is done using `CREATE TABLE` or `ALTER TABLE` queries. Then from a user perspective, the configured integration looks like a normal table, but queries to it are proxied to the external system. This transparent querying is one of the key advantages of this approach over alternative integration methods, like external dictionaries or table functions, which require to use custom query methods on each use.
+ClickHouse provides various means for integrating with external systems, including table engines. Like with all other table engines, the configuration is done using `CREATE TABLE` or `ALTER TABLE` queries. Then from a user perspective, the configured integration looks like a normal table, but queries to it are proxied to the external system. This transparent querying is one of the key advantages of this approach over alternative integration methods, like dictionaries or table functions, which require to use custom query methods on each use.
 
 List of supported integrations:
 

docs/en/engines/table-engines/integrations/mysql.md

@@ -180,6 +180,6 @@ Default value: `300`.
 ## See Also {#see-also}
 
 -   [The mysql table function](../../../sql-reference/table-functions/mysql.md)
--   [Using MySQL as a source of external dictionary](../../../sql-reference/dictionaries/external-dictionaries/external-dicts-dict-sources.md#dicts-external_dicts_dict_sources-mysql)
+-   [Using MySQL as a dictionary source](../../../sql-reference/dictionaries/external-dictionaries/external-dicts-dict-sources.md#dicts-external_dicts_dict_sources-mysql)
 
 [Original article](https://clickhouse.com/docs/en/engines/table-engines/integrations/mysql/) <!--hide-->

docs/en/engines/table-engines/integrations/odbc.md

@@ -126,7 +126,7 @@ SELECT * FROM odbc_t
 
 ## See Also {#see-also}
 
--   [ODBC external dictionaries](../../../sql-reference/dictionaries/external-dictionaries/external-dicts-dict-sources.md#dicts-external_dicts_dict_sources-odbc)
+-   [ODBC dictionaries](../../../sql-reference/dictionaries/external-dictionaries/external-dicts-dict-sources.md#dicts-external_dicts_dict_sources-odbc)
 -   [ODBC table function](../../../sql-reference/table-functions/odbc.md)
 
 [Original article](https://clickhouse.com/docs/en/engines/table-engines/integrations/odbc/) <!--hide-->

docs/en/engines/table-engines/integrations/postgresql.md

@@ -174,6 +174,6 @@ CREATE TABLE pg_table_schema_with_dots (a UInt32)
 **See Also**
 
 -   [The `postgresql` table function](../../../sql-reference/table-functions/postgresql.md)
--   [Using PostgreSQL as a source of external dictionary](../../../sql-reference/dictionaries/external-dictionaries/external-dicts-dict-sources.md#dicts-external_dicts_dict_sources-postgresql)
+-   [Using PostgreSQL as a dictionary source](../../../sql-reference/dictionaries/external-dictionaries/external-dicts-dict-sources.md#dicts-external_dicts_dict_sources-postgresql)
 
 [Original article](https://clickhouse.com/docs/en/engines/table-engines/integrations/postgresql/) <!--hide-->

docs/en/engines/table-engines/integrations/s3.md

@@ -127,6 +127,10 @@ The following settings can be set before query execution or placed into configur
 -   `s3_min_upload_part_size` — The minimum size of part to upload during multipart upload to [S3 Multipart upload](https://docs.aws.amazon.com/AmazonS3/latest/dev/uploadobjusingmpu.html). Default value is `512Mb`.
 -   `s3_max_redirects` — Max number of S3 redirects hops allowed. Default value is `10`.
 -   `s3_single_read_retries` — The maximum number of attempts during single read. Default value is `4`.
+-   `s3_max_put_rps` — Maximum PUT requests per second rate before throttling. Default value is `0` (unlimited).
+-   `s3_max_put_burst` — Max number of requests that can be issued simultaneously before hitting request per second limit. By default (`0` value) equals to `s3_max_put_rps`.
+-   `s3_max_get_rps` — Maximum GET requests per second rate before throttling. Default value is `0` (unlimited).
+-   `s3_max_get_burst` — Max number of requests that can be issued simultaneously before hitting request per second limit. By default (`0` value) equals to `s3_max_get_rps`.
 
 Security consideration: if malicious user can specify arbitrary S3 URLs, `s3_max_redirects` must be set to zero to avoid [SSRF](https://en.wikipedia.org/wiki/Server-side_request_forgery) attacks; or alternatively, `remote_host_filter` must be specified in server configuration.
 
@@ -142,6 +146,7 @@ The following settings can be specified in configuration file for given endpoint
 -   `header` —  Adds specified HTTP header to a request to given endpoint. Optional, can be specified multiple times.
 -   `server_side_encryption_customer_key_base64` — If specified, required headers for accessing S3 objects with SSE-C encryption will be set. Optional.
 -   `max_single_read_retries` — The maximum number of attempts during single read. Default value is `4`. Optional.
+-   `max_put_rps`, `max_put_burst`, `max_get_rps` and `max_get_burst` - Throttling settings (see description above) to use for specific endpoint instead of per query. Optional.
 
 **Example:**
 

docs/en/engines/table-engines/log-family/index.md

@@ -28,7 +28,7 @@ Engines:
 
     During `INSERT` queries, the table is locked, and other queries for reading and writing data both wait for the table to unlock. If there are no data writing queries, any number of data reading queries can be performed concurrently.
 
--   Do not support [mutations](/docs/en/sql-reference/statements/alter/index.md/#alter-mutations).
+-   Do not support [mutations](/docs/en/sql-reference/statements/alter/index.md#alter-mutations).
 
 -   Do not support indexes.
 

docs/en/engines/table-engines/mergetree-family/mergetree.md

@@ -537,7 +537,7 @@ TTL time_column
 TTL time_column + interval
 ```
 
-To define `interval`, use [time interval](/docs/en/sql-reference/operators/index.md/#operators-datetime) operators, for example:
+To define `interval`, use [time interval](/docs/en/sql-reference/operators/index.md#operators-datetime) operators, for example:
 
 ``` sql
 TTL date_time + INTERVAL 1 MONTH
@@ -860,7 +860,7 @@ The number of threads performing background moves of data parts can be changed b
 In the case of `MergeTree` tables, data is getting to disk in different ways:
 
 -   As a result of an insert (`INSERT` query).
--   During background merges and [mutations](/docs/en/sql-reference/statements/alter/index.md/#alter-mutations).
+-   During background merges and [mutations](/docs/en/sql-reference/statements/alter/index.md#alter-mutations).
 -   When downloading from another replica.
 -   As a result of partition freezing [ALTER TABLE … FREEZE PARTITION](/docs/en/sql-reference/statements/alter/partition.md/#alter_freeze-partition).
 
@@ -940,6 +940,10 @@ Optional parameters:
 -   `cache_path` — Path on local FS where to store cached mark and index files. Default value is `/var/lib/clickhouse/disks/<disk_name>/cache/`.
 -   `skip_access_check` — If true, disk access checks will not be performed on disk start-up. Default value is `false`.
 -   `server_side_encryption_customer_key_base64` — If specified, required headers for accessing S3 objects with SSE-C encryption will be set.
+-   `s3_max_put_rps` — Maximum PUT requests per second rate before throttling. Default value is `0` (unlimited).
+-   `s3_max_put_burst` — Max number of requests that can be issued simultaneously before hitting request per second limit. By default (`0` value) equals to `s3_max_put_rps`.
+-   `s3_max_get_rps` — Maximum GET requests per second rate before throttling. Default value is `0` (unlimited).
+-   `s3_max_get_burst` — Max number of requests that can be issued simultaneously before hitting request per second limit. By default (`0` value) equals to `s3_max_get_rps`.
 
 S3 disk can be configured as `main` or `cold` storage:
 ``` xml

docs/en/engines/table-engines/mergetree-family/replication.md

@@ -20,7 +20,7 @@ Replication works at the level of an individual table, not the entire server. A
 
 Replication does not depend on sharding. Each shard has its own independent replication.
 
-Compressed data for `INSERT` and `ALTER` queries is replicated (for more information, see the documentation for [ALTER](/docs/en/sql-reference/statements/alter/index.md/#query_language_queries_alter)).
+Compressed data for `INSERT` and `ALTER` queries is replicated (for more information, see the documentation for [ALTER](/docs/en/sql-reference/statements/alter/index.md#query_language_queries_alter)).
 
 `CREATE`, `DROP`, `ATTACH`, `DETACH` and `RENAME` queries are executed on a single server and are not replicated:
 

docs/en/engines/table-engines/special/join.md

@@ -59,7 +59,7 @@ Main use-cases for `Join`-engine tables are following:
 
 ### Deleting Data {#deleting-data}
 
-`ALTER DELETE` queries for `Join`-engine tables are implemented as [mutations](/docs/en/sql-reference/statements/alter/index.md/#mutations). `DELETE` mutation reads filtered data and overwrites data of memory and disk.
+`ALTER DELETE` queries for `Join`-engine tables are implemented as [mutations](/docs/en/sql-reference/statements/alter/index.md#mutations). `DELETE` mutation reads filtered data and overwrites data of memory and disk.
 
 ### Limitations and Settings {#join-limitations-and-settings}
 

docs/en/getting-started/example-datasets/cell-towers.md

@@ -163,7 +163,7 @@ SELECT mcc, count() FROM cell_towers GROUP BY mcc ORDER BY count() DESC LIMIT 10
 
 Based on the above query and the [MCC list](https://en.wikipedia.org/wiki/Mobile_country_code), the countries with the most cell towers are: the USA, Germany, and Russia.
 
-You may want to create an [External Dictionary](../../sql-reference/dictionaries/external-dictionaries/external-dicts.md) in ClickHouse to decode these values.
+You may want to create a [Dictionary](../../sql-reference/dictionaries/external-dictionaries/external-dicts.md) in ClickHouse to decode these values.
 
 ## Use case: Incorporate geo data {#use-case}
 

docs/en/getting-started/example-datasets/github.md

@@ -2345,23 +2345,23 @@ WITH commit_days AS
     )
 SELECT
     author,
-    arrayMax(arrayMap(x -> length(x), arraySplit(x -> (x = 0), groupArray(consecutive_day)))) AS max_consecutive_days
+    arrayMax(arrayMap(x -> length(x), arraySplit(x -> (x = 0), groupArray(consecutive_day)))) - 1 AS max_consecutive_days
 FROM commit_days
 GROUP BY author
 ORDER BY max_consecutive_days DESC
 LIMIT 10
 
 ┌─author───────────┬─max_consecutive_days─┐
-│ kssenii          │                   33 │
-│ Alexey Milovidov │                   31 │
-│ alesapin         │                   27 │
-│ Azat Khuzhin     │                   24 │
-│ Nikolai Kochetov │                   16 │
-│ Igor Nikonov     │                   12 │
-│ feng lv          │                   12 │
-│ alexey-milovidov │                   12 │
-│ Maksim Kita      │                   12 │
-│ Nikita Vasilev   │                   12 │
+│ kssenii          │                   32 │
+│ Alexey Milovidov │                   30 │
+│ alesapin         │                   26 │
+│ Azat Khuzhin     │                   23 │
+│ Nikolai Kochetov │                   15 │
+│ feng lv          │                   11 │
+│ alexey-milovidov │                   11 │
+│ Igor Nikonov     │                   11 │
+│ Maksim Kita      │                   11 │
+│ Nikita Vasilev   │                   11 │
 └──────────────────┴──────────────────────┘
 
 10 rows in set. Elapsed: 0.025 sec. Processed 62.78 thousand rows, 395.47 KB (2.54 million rows/s., 16.02 MB/s.)

docs/en/interfaces/formats.md

@@ -5,7 +5,7 @@ sidebar_label: Input and Output Formats
 title: Formats for Input and Output Data
 ---
 
-ClickHouse can accept and return data in various formats. A format supported for input can be used to parse the data provided to `INSERT`s, to perform `SELECT`s from a file-backed table such as File, URL or HDFS, or to read an external dictionary. A format supported for output can be used to arrange the
+ClickHouse can accept and return data in various formats. A format supported for input can be used to parse the data provided to `INSERT`s, to perform `SELECT`s from a file-backed table such as File, URL or HDFS, or to read a dictionary. A format supported for output can be used to arrange the
 results of a `SELECT`, and to perform `INSERT`s into a file-backed table.
 
 The supported formats are:

docs/en/operations/named-collections.md

@@ -130,7 +130,7 @@ SHOW TABLES FROM mydatabase;
 └────────┘
 ```
 
-### Example of using named collections with an external dictionary with source MySQL
+### Example of using named collections with a dictionary with source MySQL
 
 ```sql
 CREATE DICTIONARY dict (A Int64, B String)
@@ -213,7 +213,7 @@ SHOW TABLES FROM mydatabase
 └──────┘
 ```
 
-### Example of using named collections with an external dictionary with source POSTGRESQL
+### Example of using named collections with a dictionary with source POSTGRESQL
 
 ```sql
 CREATE DICTIONARY dict (a Int64, b String)
@@ -270,7 +270,7 @@ SELECT * FROM remote(remote1, database = default, table = test);
 └───┴───┘
 ```
 
-### Example of using named collections with an external dictionary with source ClickHouse
+### Example of using named collections with a dictionary with source ClickHouse
 
 ```sql
 CREATE DICTIONARY dict(a Int64, b String)

docs/en/operations/server-configuration-parameters/settings.md

@@ -268,14 +268,14 @@ The path to the table in ZooKeeper.
 
 ## dictionaries_config {#server_configuration_parameters-dictionaries_config}
 
-The path to the config file for external dictionaries.
+The path to the config file for dictionaries.
 
 Path:
 
 -   Specify the absolute path or the path relative to the server config file.
 -   The path can contain wildcards \* and ?.
 
-See also “[External dictionaries](../../sql-reference/dictionaries/external-dictionaries/external-dicts.md)”.
+See also “[Dictionaries](../../sql-reference/dictionaries/external-dictionaries/external-dicts.md)”.
 
 **Example**
 

docs/en/operations/settings/index.md

@@ -26,7 +26,7 @@ Ways to configure settings, in order of priority:
 
     -   When starting the ClickHouse console client in non-interactive mode, set the startup parameter `--setting=value`.
     -   When using the HTTP API, pass CGI parameters (`URL?setting_1=value&setting_2=value...`).
-    -   Make settings in the [SETTINGS](../../sql-reference/statements/select/index.md#settings-in-select) clause of the SELECT query. The setting value is applied only to that query and is reset to default or previous value after the query is executed.
+    -   Make settings in the [SETTINGS](../../sql-reference/statements/select/index.md#settings-in-select-query) clause of the SELECT query. The setting value is applied only to that query and is reset to default or previous value after the query is executed.
 
 Settings that can only be made in the server config file are not covered in this section.
 

docs/en/operations/settings/permissions-for-queries.md

@@ -16,44 +16,54 @@ Queries in ClickHouse can be divided into several types:
 
 The following settings regulate user permissions by the type of query:
 
--   [readonly](#settings_readonly) — Restricts permissions for all types of queries except DDL queries.
--   [allow_ddl](#settings_allow_ddl) — Restricts permissions for DDL queries.
+## readonly
+Restricts permissions for read data, write data, and change settings queries.
 
-`KILL QUERY` can be performed with any settings.
+When set to 1, allows:
 
-## readonly {#settings_readonly}
+-   All types of read queries (like SELECT and equivalent queries).
+-   Queries that modify only session context (like USE).
 
-Restricts permissions for reading data, write data and change settings queries.
+When set to 2, allows the above plus:
+- SET and CREATE TEMPORARY TABLE
 
-See how the queries are divided into types [above](#permissions_for_queries).
+  :::tip
+  Queries like EXISTS, DESCRIBE, EXPLAIN, SHOW PROCESSLIST, etc are equivalent to SELECT, because they just do select from system tables.
+  :::
 
 Possible values:
 
--   0 — All queries are allowed.
--   1 — Only read data queries are allowed.
--   2 — Read data and change settings queries are allowed.
+-   0 — Read, Write, and Change settings queries are allowed.
+-   1 — Only Read data queries are allowed.
+-   2 — Read data and Change settings queries are allowed.
 
+Default value: 0
+
+:::note
 After setting `readonly = 1`, the user can’t change `readonly` and `allow_ddl` settings in the current session.
 
 When using the `GET` method in the [HTTP interface](../../interfaces/http.md), `readonly = 1` is set automatically. To modify data, use the `POST` method.
 
-Setting `readonly = 1` prohibit the user from changing all the settings. There is a way to prohibit the user from changing only specific settings. Also there is a way to allow changing only specific settings under `readonly = 1` restrictions. For details see [constraints on settings](../../operations/settings/constraints-on-settings.md).
+Setting `readonly = 1` prohibits the user from changing settings. There is a way to prohibit the user from changing only specific settings. Also there is a way to allow changing only specific settings under `readonly = 1` restrictions. For details see [constraints on settings](../../operations/settings/constraints-on-settings.md).
+:::
 
-Default value: 0
 
 ## allow_ddl {#settings_allow_ddl}
 
 Allows or denies [DDL](https://en.wikipedia.org/wiki/Data_definition_language) queries.
 
-See how the queries are divided into types [above](#permissions_for_queries).
-
 Possible values:
 
 -   0 — DDL queries are not allowed.
 -   1 — DDL queries are allowed.
 
-You can’t execute `SET allow_ddl = 1` if `allow_ddl = 0` for the current session.
-
 Default value: 1
 
-[Original article](https://clickhouse.com/docs/en/operations/settings/permissions_for_queries/) <!--hide-->
+:::note
+You cannot run `SET allow_ddl = 1` if `allow_ddl = 0` for the current session.
+:::
+
+
+:::note KILL QUERY
+`KILL QUERY` can be performed with any combination of readonly and allow_ddl settings.
+:::

docs/en/operations/settings/settings.md

@@ -276,7 +276,7 @@ Default value: 0.
 Enables or disables the insertion of [default values](../../sql-reference/statements/create/table.md/#create-default-values) instead of [NULL](../../sql-reference/syntax.md/#null-literal) into columns with not [nullable](../../sql-reference/data-types/nullable.md/#data_type-nullable) data type.
 If column type is not nullable and this setting is disabled, then inserting `NULL` causes an exception. If column type is nullable, then `NULL` values are inserted as is, regardless of this setting.
 
-This setting is applicable to [INSERT ... SELECT](../../sql-reference/statements/insert-into.md/#insert_query_insert-select) queries. Note that `SELECT` subqueries may be concatenated with `UNION ALL` clause.
+This setting is applicable to [INSERT ... SELECT](../../sql-reference/statements/insert-into.md/#inserting-the-results-of-select) queries. Note that `SELECT` subqueries may be concatenated with `UNION ALL` clause.
 
 Possible values:
 
@@ -1619,8 +1619,8 @@ These functions can be transformed:
 -   [length](../../sql-reference/functions/array-functions.md/#array_functions-length) to read the [size0](../../sql-reference/data-types/array.md/#array-size) subcolumn.
 -   [empty](../../sql-reference/functions/array-functions.md/#function-empty) to read the [size0](../../sql-reference/data-types/array.md/#array-size) subcolumn.
 -   [notEmpty](../../sql-reference/functions/array-functions.md/#function-notempty) to read the [size0](../../sql-reference/data-types/array.md/#array-size) subcolumn.
--   [isNull](../../sql-reference/operators/index.md/#operator-is-null) to read the [null](../../sql-reference/data-types/nullable.md/#finding-null) subcolumn.
--   [isNotNull](../../sql-reference/operators/index.md/#is-not-null) to read the [null](../../sql-reference/data-types/nullable.md/#finding-null) subcolumn.
+-   [isNull](../../sql-reference/operators/index.md#operator-is-null) to read the [null](../../sql-reference/data-types/nullable.md/#finding-null) subcolumn.
+-   [isNotNull](../../sql-reference/operators/index.md#is-not-null) to read the [null](../../sql-reference/data-types/nullable.md/#finding-null) subcolumn.
 -   [count](../../sql-reference/aggregate-functions/reference/count.md) to read the [null](../../sql-reference/data-types/nullable.md/#finding-null) subcolumn.
 -   [mapKeys](../../sql-reference/functions/tuple-map-functions.md/#mapkeys) to read the [keys](../../sql-reference/data-types/map.md/#map-subcolumns) subcolumn.
 -   [mapValues](../../sql-reference/functions/tuple-map-functions.md/#mapvalues) to read the [values](../../sql-reference/data-types/map.md/#map-subcolumns) subcolumn.
@@ -2041,7 +2041,7 @@ Default value: 16.
 
 ## validate_polygons {#validate_polygons}
 
-Enables or disables throwing an exception in the [pointInPolygon](../../sql-reference/functions/geo/index.md/#pointinpolygon) function, if the polygon is self-intersecting or self-tangent.
+Enables or disables throwing an exception in the [pointInPolygon](../../sql-reference/functions/geo/index.md#pointinpolygon) function, if the polygon is self-intersecting or self-tangent.
 
 Possible values:
 
@@ -2227,7 +2227,7 @@ Default value: `0`.
 
 ## mutations_sync {#mutations_sync}
 
-Allows to execute `ALTER TABLE ... UPDATE|DELETE` queries ([mutations](../../sql-reference/statements/alter/index.md/#mutations)) synchronously.
+Allows to execute `ALTER TABLE ... UPDATE|DELETE` queries ([mutations](../../sql-reference/statements/alter/index.md#mutations)) synchronously.
 
 Possible values:
 
@@ -2239,8 +2239,8 @@ Default value: `0`.
 
 **See Also**
 
--   [Synchronicity of ALTER Queries](../../sql-reference/statements/alter/index.md/#synchronicity-of-alter-queries)
--   [Mutations](../../sql-reference/statements/alter/index.md/#mutations)
+-   [Synchronicity of ALTER Queries](../../sql-reference/statements/alter/index.md#synchronicity-of-alter-queries)
+-   [Mutations](../../sql-reference/statements/alter/index.md#mutations)
 
 ## ttl_only_drop_parts {#ttl_only_drop_parts}
 

docs/en/operations/system-tables/crash-log.md

@@ -7,8 +7,8 @@ Contains information about stack traces for fatal errors. The table does not exi
 
 Columns:
 
--   `event_date` ([Datetime](../../sql-reference/data-types/datetime.md)) — Date of the event.
--   `event_time` ([Datetime](../../sql-reference/data-types/datetime.md)) — Time of the event.
+-   `event_date` ([DateTime](../../sql-reference/data-types/datetime.md)) — Date of the event.
+-   `event_time` ([DateTime](../../sql-reference/data-types/datetime.md)) — Time of the event.
 -   `timestamp_ns` ([UInt64](../../sql-reference/data-types/int-uint.md)) — Timestamp of the event with nanoseconds.
 -   `signal` ([Int32](../../sql-reference/data-types/int-uint.md)) — Signal number.
 -   `thread_id` ([UInt64](../../sql-reference/data-types/int-uint.md)) — Thread ID.

docs/en/operations/system-tables/dictionaries.md

@@ -3,7 +3,7 @@ slug: /en/operations/system-tables/dictionaries
 ---
 # dictionaries
 
-Contains information about [external dictionaries](../../sql-reference/dictionaries/external-dictionaries/external-dicts.md).
+Contains information about [dictionaries](../../sql-reference/dictionaries/external-dictionaries/external-dicts.md).
 
 Columns:
 
@@ -33,7 +33,7 @@ Columns:
 -   `lifetime_min` ([UInt64](../../sql-reference/data-types/int-uint.md#uint-ranges)) — Minimum [lifetime](../../sql-reference/dictionaries/external-dictionaries/external-dicts-dict-lifetime.md) of the dictionary in memory, after which ClickHouse tries to reload the dictionary (if `invalidate_query` is set, then only if it has changed). Set in seconds.
 -   `lifetime_max` ([UInt64](../../sql-reference/data-types/int-uint.md#uint-ranges)) — Maximum [lifetime](../../sql-reference/dictionaries/external-dictionaries/external-dicts-dict-lifetime.md) of the dictionary in memory, after which ClickHouse tries to reload the dictionary (if `invalidate_query` is set, then only if it has changed). Set in seconds.
 -   `loading_start_time` ([DateTime](../../sql-reference/data-types/datetime.md)) — Start time for loading the dictionary.
--   `last_successful_update_time` ([DateTime](../../sql-reference/data-types/datetime.md)) — End time for loading or updating the dictionary. Helps to monitor some troubles with external sources and investigate causes.
+-   `last_successful_update_time` ([DateTime](../../sql-reference/data-types/datetime.md)) — End time for loading or updating the dictionary. Helps to monitor some troubles with dictionary sources and investigate the causes.
 -   `loading_duration` ([Float32](../../sql-reference/data-types/float.md)) — Duration of a dictionary loading.
 -   `last_exception` ([String](../../sql-reference/data-types/string.md)) — Text of the error that occurs when creating or reloading the dictionary if the dictionary couldn’t be created.
 -   `comment` ([String](../../sql-reference/data-types/string.md)) — Text of the comment to dictionary.

docs/en/operations/system-tables/mutations.md

@@ -3,7 +3,7 @@ slug: /en/operations/system-tables/mutations
 ---
 # mutations
 
-The table contains information about [mutations](/docs/en/sql-reference/statements/alter/index.md/#mutations) of [MergeTree](/docs/en/engines/table-engines/mergetree-family/mergetree.md) tables and their progress. Each mutation command is represented by a single row.
+The table contains information about [mutations](/docs/en/sql-reference/statements/alter/index.md#mutations) of [MergeTree](/docs/en/engines/table-engines/mergetree-family/mergetree.md) tables and their progress. Each mutation command is represented by a single row.
 
 Columns:
 
@@ -15,7 +15,7 @@ Columns:
 
 -   `command` ([String](/docs/en/sql-reference/data-types/string.md)) — The mutation command string (the part of the query after `ALTER TABLE [db.]table`).
 
--   `create_time` ([Datetime](/docs/en/sql-reference/data-types/datetime.md)) —  Date and time when the mutation command was submitted for execution.
+-   `create_time` ([DateTime](/docs/en/sql-reference/data-types/datetime.md)) —  Date and time when the mutation command was submitted for execution.
 
 -   `block_numbers.partition_id` ([Array](/docs/en/sql-reference/data-types/array.md)([String](/docs/en/sql-reference/data-types/string.md))) — For mutations of replicated tables, the array contains the partitions' IDs (one record for each partition). For mutations of non-replicated tables the array is empty.
 
@@ -39,13 +39,13 @@ If there were problems with mutating some data parts, the following columns cont
 
 -   `latest_failed_part` ([String](/docs/en/sql-reference/data-types/string.md)) — The name of the most recent part that could not be mutated.
 
--   `latest_fail_time` ([Datetime](/docs/en/sql-reference/data-types/datetime.md)) — The date and time of the most recent part mutation failure.
+-   `latest_fail_time` ([DateTime](/docs/en/sql-reference/data-types/datetime.md)) — The date and time of the most recent part mutation failure.
 
 -   `latest_fail_reason` ([String](/docs/en/sql-reference/data-types/string.md)) — The exception message that caused the most recent part mutation failure.
 
 **See Also**
 
--   [Mutations](/docs/en/sql-reference/statements/alter/index.md/#mutations)
+-   [Mutations](/docs/en/sql-reference/statements/alter/index.md#mutations)
 -   [MergeTree](/docs/en/engines/table-engines/mergetree-family/mergetree.md) table engine
 -   [ReplicatedMergeTree](/docs/en/engines/table-engines/mergetree-family/replication.md) family
 

docs/en/operations/system-tables/parts.md

@@ -9,7 +9,7 @@ Each row describes one data part.
 
 Columns:
 
--   `partition` ([String](../../sql-reference/data-types/string.md)) – The partition name. To learn what a partition is, see the description of the [ALTER](../../sql-reference/statements/alter/index.md/#query_language_queries_alter) query.
+-   `partition` ([String](../../sql-reference/data-types/string.md)) – The partition name. To learn what a partition is, see the description of the [ALTER](../../sql-reference/statements/alter/index.md#query_language_queries_alter) query.
 
     Formats:
 

docs/en/operations/system-tables/parts_columns.md

@@ -9,7 +9,7 @@ Each row describes one data part.
 
 Columns:
 
--   `partition` ([String](../../sql-reference/data-types/string.md)) — The partition name. To learn what a partition is, see the description of the [ALTER](../../sql-reference/statements/alter/index.md/#query_language_queries_alter) query.
+-   `partition` ([String](../../sql-reference/data-types/string.md)) — The partition name. To learn what a partition is, see the description of the [ALTER](../../sql-reference/statements/alter/index.md#query_language_queries_alter) query.
 
     Formats:
 

docs/en/operations/system-tables/replication_queue.md

@@ -29,7 +29,7 @@ Columns:
     -   `MUTATE_PART` — Apply one or several mutations to the part.
     -   `ALTER_METADATA` — Apply alter modification according to global /metadata and /columns paths.
 
--   `create_time` ([Datetime](../../sql-reference/data-types/datetime.md)) — Date and time when the task was submitted for execution.
+-   `create_time` ([DateTime](../../sql-reference/data-types/datetime.md)) — Date and time when the task was submitted for execution.
 
 -   `required_quorum` ([UInt32](../../sql-reference/data-types/int-uint.md)) — The number of replicas waiting for the task to complete with confirmation of completion. This column is only relevant for the `GET_PARTS` task.
 
@@ -47,13 +47,13 @@ Columns:
 
 -   `last_exception` ([String](../../sql-reference/data-types/string.md)) — Text message about the last error that occurred (if any).
 
--   `last_attempt_time` ([Datetime](../../sql-reference/data-types/datetime.md)) — Date and time when the task was last attempted.
+-   `last_attempt_time` ([DateTime](../../sql-reference/data-types/datetime.md)) — Date and time when the task was last attempted.
 
 -   `num_postponed` ([UInt32](../../sql-reference/data-types/int-uint.md)) — The number of postponed tasks.
 
 -   `postpone_reason` ([String](../../sql-reference/data-types/string.md)) — The reason why the task was postponed.
 
--   `last_postpone_time` ([Datetime](../../sql-reference/data-types/datetime.md)) — Date and time when the task was last postponed.
+-   `last_postpone_time` ([DateTime](../../sql-reference/data-types/datetime.md)) — Date and time when the task was last postponed.
 
 -   `merge_type` ([String](../../sql-reference/data-types/string.md)) — Type of the current merge. Empty if it's a mutation.
 

docs/en/sql-reference/data-types/date32.md

@@ -6,7 +6,7 @@ sidebar_label: Date32
 
 # Date32
 
-A date. Supports the date range same with [Datetime64](../../sql-reference/data-types/datetime64.md). Stored in four bytes as the number of days since 1900-01-01. Allows storing values till 2299-12-31.
+A date. Supports the date range same with [DateTime64](../../sql-reference/data-types/datetime64.md). Stored in four bytes as the number of days since 1900-01-01. Allows storing values till 2299-12-31.
 
 **Examples**
 

docs/en/sql-reference/data-types/datetime.md

@@ -4,7 +4,7 @@ sidebar_position: 48
 sidebar_label: DateTime
 ---
 
-# Datetime
+# DateTime
 
 Allows to store an instant in time, that can be expressed as a calendar date and a time of a day.
 

docs/en/sql-reference/data-types/datetime64.md

@@ -4,7 +4,7 @@ sidebar_position: 49
 sidebar_label: DateTime64
 ---
 
-# Datetime64
+# DateTime64
 
 Allows to store an instant in time, that can be expressed as a calendar date and a time of a day, with defined sub-second precision
 

docs/en/sql-reference/dictionaries/external-dictionaries/_category_.yml

@@ -1,8 +1,8 @@
 position: 37
-label: 'External Dictionaries'
+label: 'Dictionaries'
 collapsible: true
 collapsed: true
 link:
   type: generated-index
-  title: External Dictionaries
+  title: Dictionaries
   slug: /en/sql-reference/dictionaries/external-dictionaries

docs/en/sql-reference/dictionaries/external-dictionaries/_snippet_dictionary_in_cloud.md

@@ -0,0 +1,4 @@
+:::tip
+If you are using a dictionary with ClickHouse Cloud please use the DDL query option to create your dictionaries, and create your dictionary as user `default`. 
+Also, verify the list of supported dictionary sources in the [Cloud Compatibility guide](/docs/en/whats-new/cloud-capabilities.md).
+:::

docs/en/sql-reference/dictionaries/external-dictionaries/external-dicts-dict-layout.md

@@ -3,6 +3,7 @@ slug: /en/sql-reference/dictionaries/external-dictionaries/external-dicts-dict-l
 sidebar_position: 41
 sidebar_label: Storing Dictionaries in Memory
 ---
+import CloudDetails from '@site/docs/en/sql-reference/dictionaries/external-dictionaries/_snippet_dictionary_in_cloud.md';
 
 # Storing Dictionaries in Memory 
 
@@ -22,7 +23,9 @@ ClickHouse generates an exception for errors with dictionaries. Examples of erro
 -   The dictionary being accessed could not be loaded.
 -   Error querying a `cached` dictionary.
 
-You can view the list of external dictionaries and their statuses in the [system.dictionaries](../../../operations/system-tables/dictionaries.md) table.
+You can view the list of dictionaries and their statuses in the [system.dictionaries](../../../operations/system-tables/dictionaries.md) table.
+
+<CloudDetails /> 
 
 The configuration looks like this:
 

docs/en/sql-reference/dictionaries/external-dictionaries/external-dicts-dict-lifetime.md

@@ -3,6 +3,7 @@ slug: /en/sql-reference/dictionaries/external-dictionaries/external-dicts-dict-l
 sidebar_position: 42
 sidebar_label: Dictionary Updates
 ---
+import CloudDetails from '@site/docs/en/sql-reference/dictionaries/external-dictionaries/_snippet_dictionary_in_cloud.md';
 
 # Dictionary Updates 
 
@@ -12,6 +13,8 @@ Dictionary updates (other than loading for first use) do not block queries. Duri
 
 Example of settings:
 
+<CloudDetails />
+
 ``` xml
 <dictionary>
     ...

docs/en/sql-reference/dictionaries/external-dictionaries/external-dicts-dict-polygon.md

@@ -4,12 +4,15 @@ sidebar_position: 46
 sidebar_label: Polygon Dictionaries With Grids
 title: "Polygon dictionaries"
 ---
+import CloudDetails from '@site/docs/en/sql-reference/dictionaries/external-dictionaries/_snippet_dictionary_in_cloud.md';
 
 Polygon dictionaries allow you to efficiently search for the polygon containing specified points.
 For example: defining a city area by geographical coordinates.
 
 Example of a polygon dictionary configuration:
 
+<CloudDetails />
+
 ``` xml
 <dictionary>
     <structure>
@@ -78,7 +81,7 @@ To respond to the query, there is a corresponding cell, and the index for the po
 
 -   `POLYGON`. Synonym to `POLYGON_INDEX_CELL`.
 
-Dictionary queries are carried out using standard [functions](../../../sql-reference/functions/ext-dict-functions.md) for working with external dictionaries.
+Dictionary queries are carried out using standard [functions](../../../sql-reference/functions/ext-dict-functions.md) for working with dictionaries.
 An important difference is that here the keys will be the points for which you want to find the polygon containing them.
 
 **Example**

docs/en/sql-reference/dictionaries/external-dictionaries/external-dicts-dict-sources.md

@@ -1,12 +1,15 @@
 ---
 slug: /en/sql-reference/dictionaries/external-dictionaries/external-dicts-dict-sources
 sidebar_position: 43
-sidebar_label: Sources of External Dictionaries
+sidebar_label: Dictionary Sources
 ---
+import CloudDetails from '@site/docs/en/sql-reference/dictionaries/external-dictionaries/_snippet_dictionary_in_cloud.md';
 
-# Sources of External Dictionaries
+# Dictionary Sources
 
-An external dictionary can be connected to ClickHouse from many different sources.
+<CloudDetails />
+
+A dictionary can be connected to ClickHouse from many different sources.
 
 If the dictionary is configured using an xml-file, the configuration looks like this:
 
@@ -65,13 +68,13 @@ Types of sources (`source_type`):
 -   [Executable Pool](#dicts-external_dicts_dict_sources-executable_pool)
 -   [HTTP(s)](#dicts-external_dicts_dict_sources-http)
 -   DBMS
-    -   [ODBC](#dicts-external_dicts_dict_sources-odbc)
-    -   [MySQL](#dicts-external_dicts_dict_sources-mysql)
-    -   [ClickHouse](#dicts-external_dicts_dict_sources-clickhouse)
-    -   [MongoDB](#dicts-external_dicts_dict_sources-mongodb)
-    -   [Redis](#dicts-external_dicts_dict_sources-redis)
-    -   [Cassandra](#dicts-external_dicts_dict_sources-cassandra)
-    -   [PostgreSQL](#dicts-external_dicts_dict_sources-postgresql)
+    -   [ODBC](#odbc)
+    -   [MySQL](#mysql)
+    -   [ClickHouse](#clickhouse)
+    -   [MongoDB](#mongodb)
+    -   [Redis](#redis)
+    -   [Cassandra](#cassandra)
+    -   [PostgreSQL](#postgresql)
 
 ## Local File
 

docs/en/sql-reference/dictionaries/external-dictionaries/external-dicts-dict-structure.md

@@ -3,9 +3,12 @@ slug: /en/sql-reference/dictionaries/external-dictionaries/external-dicts-dict-s
 sidebar_position: 44
 sidebar_label: Dictionary Key and Fields
 ---
+import CloudDetails from '@site/docs/en/sql-reference/dictionaries/external-dictionaries/_snippet_dictionary_in_cloud.md';
 
 # Dictionary Key and Fields
 
+<CloudDetails />
+
 The `structure` clause describes the dictionary key and fields available for queries.
 
 XML description:
@@ -171,5 +174,5 @@ Configuration fields:
 
 **See Also**
 
--   [Functions for working with external dictionaries](../../../sql-reference/functions/ext-dict-functions.md).
+-   [Functions for working with dictionaries](../../../sql-reference/functions/ext-dict-functions.md).
 

docs/en/sql-reference/dictionaries/external-dictionaries/external-dicts-dict.md

@@ -1,10 +1,13 @@
 ---
 slug: /en/sql-reference/dictionaries/external-dictionaries/external-dicts-dict
 sidebar_position: 40
-sidebar_label: Configuring an External Dictionary
+sidebar_label: Configuring a Dictionary
 ---
+import CloudDetails from '@site/docs/en/sql-reference/dictionaries/external-dictionaries/_snippet_dictionary_in_cloud.md';
 
-# Configuring an External Dictionary
+# Configuring a Dictionary
+
+<CloudDetails />
 
 If dictionary is configured using xml file, than dictionary configuration has the following structure:
 

docs/en/sql-reference/dictionaries/external-dictionaries/external-dicts.md

@@ -3,18 +3,23 @@ slug: /en/sql-reference/dictionaries/external-dictionaries/external-dicts
 sidebar_position: 39
 sidebar_label: General Description
 ---
+import CloudDetails from '@site/docs/en/sql-reference/dictionaries/external-dictionaries/_snippet_dictionary_in_cloud.md';
 
-# External Dictionaries 
+# Dictionaries 
 
-You can add your own dictionaries from various data sources. The data source for a dictionary can be a local text or executable file, an HTTP(s) resource, or another DBMS. For more information, see “[Sources for external dictionaries](../../../sql-reference/dictionaries/external-dictionaries/external-dicts-dict-sources.md)”.
+:::tip Tutorial
+If you are getting started with Dictionaries in ClickHouse we have a tutorial that covers that topic.  Take a look [here](/docs/en/tutorial.md).
+:::
+
+You can add your own dictionaries from various data sources. The source for a dictionary can be a ClickHouse table, a local text or executable file, an HTTP(s) resource, or another DBMS. For more information, see “[Dictionary Sources](../../../sql-reference/dictionaries/external-dictionaries/external-dicts-dict-sources.md)”.
 
 ClickHouse:
 
 -   Fully or partially stores dictionaries in RAM.
 -   Periodically updates dictionaries and dynamically loads missing values. In other words, dictionaries can be loaded dynamically.
--   Allows to create external dictionaries with xml files or [DDL queries](../../../sql-reference/statements/create/dictionary.md).
+-   Allows creating dictionaries with xml files or [DDL queries](../../../sql-reference/statements/create/dictionary.md).
 
-The configuration of external dictionaries can be located in one or more xml-files. The path to the configuration is specified in the [dictionaries_config](../../../operations/server-configuration-parameters/settings.md#server_configuration_parameters-dictionaries_config) parameter.
+The configuration of dictionaries can be located in one or more xml-files. The path to the configuration is specified in the [dictionaries_config](../../../operations/server-configuration-parameters/settings.md#server_configuration_parameters-dictionaries_config) parameter.
 
 Dictionaries can be loaded at server startup or at first use, depending on the [dictionaries_lazy_load](../../../operations/server-configuration-parameters/settings.md#server_configuration_parameters-dictionaries_lazy_load) setting.
 
@@ -24,6 +29,22 @@ The [dictionaries](../../../operations/system-tables/dictionaries.md#system_tabl
 -   Configuration parameters.
 -   Metrics like amount of RAM allocated for the dictionary or a number of queries since the dictionary was successfully loaded.
 
+<CloudDetails />
+
+## Creating a dictionary with a DDL query
+
+Dictionaries can be created with [DDL queries](../../../sql-reference/statements/create/dictionary.md), and this is the recommended method because with DDL created dictionaries:
+- No additional records are added to server configuration files
+- The dictionaries can be worked with as first-class entities, like tables or views
+- Data can be read directly, using familiar SELECT rather than dictionary table functions
+- The dictionaries can be easily renamed
+
+## Creating a dictionary with a configuration file
+
+:::note
+Creating a dictionary with a configuration file is not applicable to ClickHouse Cloud. Please use DDL (see above), and create your dictionary as user `default`.
+:::
+
 The dictionary configuration file has the following format:
 
 ``` xml
@@ -44,18 +65,17 @@ The dictionary configuration file has the following format:
 
 You can [configure](../../../sql-reference/dictionaries/external-dictionaries/external-dicts-dict.md) any number of dictionaries in the same file.
 
-[DDL queries for dictionaries](../../../sql-reference/statements/create/dictionary.md) does not require any additional records in server configuration. They allow to work with dictionaries as first-class entities, like tables or views.
 
 :::note    
-You can convert values for a small dictionary by describing it in a `SELECT` query (see the [transform](../../../sql-reference/functions/other-functions.md) function). This functionality is not related to external dictionaries.
+You can convert values for a small dictionary by describing it in a `SELECT` query (see the [transform](../../../sql-reference/functions/other-functions.md) function). This functionality is not related to dictionaries.
 :::
 
 ## See Also
 
--   [Configuring an External Dictionary](../../../sql-reference/dictionaries/external-dictionaries/external-dicts-dict.md)
+-   [Configuring a Dictionary](../../../sql-reference/dictionaries/external-dictionaries/external-dicts-dict.md)
 -   [Storing Dictionaries in Memory](../../../sql-reference/dictionaries/external-dictionaries/external-dicts-dict-layout.md)
 -   [Dictionary Updates](../../../sql-reference/dictionaries/external-dictionaries/external-dicts-dict-lifetime.md)
--   [Sources of External Dictionaries](../../../sql-reference/dictionaries/external-dictionaries/external-dicts-dict-sources.md)
+-   [Dictionary Sources](../../../sql-reference/dictionaries/external-dictionaries/external-dicts-dict-sources.md)
 -   [Dictionary Key and Fields](../../../sql-reference/dictionaries/external-dictionaries/external-dicts-dict-structure.md)
--   [Functions for Working with External Dictionaries](../../../sql-reference/functions/ext-dict-functions.md)
+-   [Functions for Working with Dictionaries](../../../sql-reference/functions/ext-dict-functions.md)
 

docs/en/sql-reference/dictionaries/index.md

@@ -12,6 +12,6 @@ ClickHouse supports special functions for working with dictionaries that can be
 
 ClickHouse supports:
 
--   [Built-in dictionaries](../../sql-reference/dictionaries/internal-dicts.md#internal_dicts) with a specific [set of functions](../../sql-reference/functions/ym-dict-functions.md).
--   [Plug-in (external) dictionaries](../../sql-reference/dictionaries/external-dictionaries/external-dicts.md#dicts-external-dicts) with a [set of functions](../../sql-reference/functions/ext-dict-functions.md).
+-   [Dictionaries](../../sql-reference/dictionaries/external-dictionaries/external-dicts.md#dicts-external-dicts) with a [set of functions](../../sql-reference/functions/ext-dict-functions.md).
+-   [Embedded dictionaries](../../sql-reference/dictionaries/internal-dicts.md#internal_dicts) with a specific [set of functions](../../sql-reference/functions/ym-dict-functions.md).
 

docs/en/sql-reference/dictionaries/internal-dicts.md

@@ -1,10 +1,13 @@
 ---
 slug: /en/sql-reference/dictionaries/internal-dicts
 sidebar_position: 39
-sidebar_label: Internal Dictionaries
+sidebar_label: Embedded Dictionaries
 ---
+import SelfManaged from '@site/docs/en/_snippets/_self_managed_only_no_roadmap.md';
 
-# Internal Dictionaries
+# Embedded Dictionaries
+
+<SelfManaged />
 
 ClickHouse contains a built-in feature for working with a geobase.
 

docs/en/sql-reference/functions/arithmetic-functions.md

@@ -65,6 +65,11 @@ An exception is thrown when dividing by zero or when dividing a minimal negative
 
 Differs from [modulo](#modulo) in that it returns zero when the divisor is zero.
 
+## positive_modulo(a, b)
+Calculates the remainder when dividing `a` by `b`. Similar to function `modulo` except that `positive_modulo` always return non-negative number.
+
+Notice that `positive_modulo` is 4-5 times slower than `modulo`. You should not use `positive_modulo` unless you want to get positive result and don't care about performance too much.
+
 ## negate(a), -a operator
 
 Calculates a number with the reverse sign. The result is always signed.

docs/en/sql-reference/functions/date-time-functions.md

@@ -550,7 +550,7 @@ Alias: `dateTrunc`.
 
 -   Value, truncated to the specified part of date.
 
-Type: [Datetime](../../sql-reference/data-types/datetime.md).
+Type: [DateTime](../../sql-reference/data-types/datetime.md).
 
 **Example**
 
@@ -881,7 +881,7 @@ now([timezone])
 
 -   Current date and time.
 
-Type: [Datetime](../../sql-reference/data-types/datetime.md).
+Type: [DateTime](../../sql-reference/data-types/datetime.md).
 
 **Example**
 
@@ -932,7 +932,7 @@ now64([scale], [timezone])
 
 -   Current date and time with sub-second precision.
 
-Type: [Datetime64](../../sql-reference/data-types/datetime64.md).
+Type: [DateTime64](../../sql-reference/data-types/datetime64.md).
 
 **Example**
 
@@ -968,7 +968,7 @@ nowInBlock([timezone])
 
 -   Current date and time at the moment of processing of each block of data.
 
-Type: [Datetime](../../sql-reference/data-types/datetime.md).
+Type: [DateTime](../../sql-reference/data-types/datetime.md).
 
 **Example**
 

docs/en/sql-reference/functions/ext-dict-functions.md

@@ -1,20 +1,20 @@
 ---
 slug: /en/sql-reference/functions/ext-dict-functions
 sidebar_position: 58
-sidebar_label: External Dictionaries
+sidebar_label: Dictionaries
 ---
 
+# Functions for Working with Dictionaries
+
 :::note    
 For dictionaries created with [DDL queries](../../sql-reference/statements/create/dictionary.md), the `dict_name` parameter must be fully specified, like `<database>.<dict_name>`. Otherwise, the current database is used.
 :::
 
-# Functions for Working with External Dictionaries
-
-For information on connecting and configuring external dictionaries, see [External dictionaries](../../sql-reference/dictionaries/external-dictionaries/external-dicts.md).
+For information on connecting and configuring dictionaries, see [Dictionaries](../../sql-reference/dictionaries/external-dictionaries/external-dicts.md).
 
 ## dictGet, dictGetOrDefault, dictGetOrNull
 
-Retrieves values from an external dictionary.
+Retrieves values from a dictionary.
 
 ``` sql
 dictGet('dict_name', attr_names, id_expr)
@@ -52,7 +52,7 @@ Create a text file `ext-dict-test.csv` containing the following:
 
 The first column is `id`, the second column is `c1`.
 
-Configure the external dictionary:
+Configure the dictionary:
 
 ``` xml
 <clickhouse>
@@ -112,7 +112,7 @@ Create a text file `ext-dict-mult.csv` containing the following:
 
 The first column is `id`, the second is `c1`, the third is `c2`.
 
-Configure the external dictionary:
+Configure the dictionary:
 
 ``` xml
 <clickhouse>
@@ -185,7 +185,7 @@ INSERT INTO range_key_dictionary_source_table VALUES(2, toDate('2019-05-20'), to
 INSERT INTO range_key_dictionary_source_table VALUES(3, toDate('2019-05-20'), toDate('2019-05-20'), 'Third', 'Third');
 ```
 
-Create the external dictionary:
+Create the dictionary:
 
 ```sql
 CREATE DICTIONARY range_key_dictionary
@@ -226,7 +226,7 @@ Result:
 
 **See Also**
 
--   [External Dictionaries](../../sql-reference/dictionaries/external-dictionaries/external-dicts.md)
+-   [Dictionaries](../../sql-reference/dictionaries/external-dictionaries/external-dicts.md)
 
 ## dictHas
 

docs/en/sql-reference/functions/math-functions.md

@@ -549,3 +549,33 @@ Result:
 │ 3.141592653589793 │
 └───────────────────┘
 ```
+
+
+## factorial(n)
+
+Computes the factorial of an integer value. It works with any native integer type including UInt(8|16|32|64) and Int(8|16|32|64). The return type is UInt64.
+
+The factorial of 0 is 1. Likewise, the factorial() function returns 1 for any negative value. The maximum positive value for the input argument is 20, a value of 21 or greater will cause exception throw.
+
+
+**Syntax**
+
+``` sql
+factorial(n)
+```
+
+**Example**
+
+Query:
+
+``` sql
+SELECT factorial(10);
+```
+
+Result:
+
+``` text
+┌─factorial(10)─┐
+│       3628800 │
+└───────────────┘
+```

docs/en/sql-reference/functions/random-functions.md

@@ -24,6 +24,11 @@ Returns a pseudo-random UInt64 number, evenly distributed among all UInt64-type
 
 Uses a linear congruential generator.
 
+## canonicalRand
+The function generates pseudo random results with independent and identically distributed uniformly distributed values in [0, 1).
+
+Non-deterministic. Return type is Float64.
+
 ## randConstant
 
 Produces a constant column with a random value.

docs/en/sql-reference/functions/ym-dict-functions.md

@@ -131,7 +131,7 @@ Type: `UInt32`.
 ### regionToPopulation(id\[, geobase\])
 
 Gets the population for a region.
-The population can be recorded in files with the geobase. See the section “External dictionaries”.
+The population can be recorded in files with the geobase. See the section “Dictionaries”.
 If the population is not recorded for the region, it returns 0.
 In the geobase, the population might be recorded for child regions, but not for parent regions.
 

docs/en/sql-reference/statements/alter/column.md

@@ -254,7 +254,7 @@ The `ALTER` query lets you create and delete separate elements (columns) in nest
 
 There is no support for deleting columns in the primary key or the sampling key (columns that are used in the `ENGINE` expression). Changing the type for columns that are included in the primary key is only possible if this change does not cause the data to be modified (for example, you are allowed to add values to an Enum or to change a type from `DateTime` to `UInt32`).
 
-If the `ALTER` query is not sufficient to make the table changes you need, you can create a new table, copy the data to it using the [INSERT SELECT](/docs/en/sql-reference/statements/insert-into.md/#insert_query_insert-select) query, then switch the tables using the [RENAME](/docs/en/sql-reference/statements/rename.md/#rename-table) query and delete the old table. You can use the [clickhouse-copier](/docs/en/operations/utilities/clickhouse-copier.md) as an alternative to the `INSERT SELECT` query.
+If the `ALTER` query is not sufficient to make the table changes you need, you can create a new table, copy the data to it using the [INSERT SELECT](/docs/en/sql-reference/statements/insert-into.md/#inserting-the-results-of-select) query, then switch the tables using the [RENAME](/docs/en/sql-reference/statements/rename.md/#rename-table) query and delete the old table. You can use the [clickhouse-copier](/docs/en/operations/utilities/clickhouse-copier.md) as an alternative to the `INSERT SELECT` query.
 
 The `ALTER` query blocks all reads and writes for the table. In other words, if a long `SELECT` is running at the time of the `ALTER` query, the `ALTER` query will wait for it to complete. At the same time, all new queries to the same table will wait while this `ALTER` is running.
 

docs/en/sql-reference/statements/alter/delete.md

@@ -10,7 +10,7 @@ sidebar_label: DELETE
 ALTER TABLE [db.]table [ON CLUSTER cluster] DELETE WHERE filter_expr
 ```
 
-Deletes data matching the specified filtering expression. Implemented as a [mutation](/docs/en/sql-reference/statements/alter/index.md/#mutations).
+Deletes data matching the specified filtering expression. Implemented as a [mutation](/docs/en/sql-reference/statements/alter/index.md#mutations).
 
 
 :::note
@@ -25,6 +25,6 @@ The synchronicity of the query processing is defined by the [mutations_sync](/do
 
 **See also**
 
--   [Mutations](/docs/en/sql-reference/statements/alter/index.md/#mutations)
--   [Synchronicity of ALTER Queries](/docs/en/sql-reference/statements/alter/index.md/#synchronicity-of-alter-queries)
+-   [Mutations](/docs/en/sql-reference/statements/alter/index.md#mutations)
+-   [Synchronicity of ALTER Queries](/docs/en/sql-reference/statements/alter/index.md#synchronicity-of-alter-queries)
 -   [mutations_sync](/docs/en/operations/settings/settings.md/#mutations_sync) setting

docs/en/sql-reference/statements/alter/partition.md

@@ -270,7 +270,7 @@ ALTER TABLE hits MOVE PARTITION '2019-09-01' TO DISK 'fast_ssd'
 
 ## UPDATE IN PARTITION
 
-Manipulates data in the specifies partition matching the specified filtering expression. Implemented as a [mutation](/docs/en/sql-reference/statements/alter/index.md/#mutations).
+Manipulates data in the specifies partition matching the specified filtering expression. Implemented as a [mutation](/docs/en/sql-reference/statements/alter/index.md#mutations).
 
 Syntax:
 
@@ -290,7 +290,7 @@ ALTER TABLE mt UPDATE x = x + 1 IN PARTITION 2 WHERE p = 2;
 
 ## DELETE IN PARTITION
 
-Deletes data in the specifies partition matching the specified filtering expression. Implemented as a [mutation](/docs/en/sql-reference/statements/alter/index.md/#mutations).
+Deletes data in the specifies partition matching the specified filtering expression. Implemented as a [mutation](/docs/en/sql-reference/statements/alter/index.md#mutations).
 
 Syntax:
 

docs/en/sql-reference/statements/alter/projection.md

@@ -2,9 +2,134 @@
 slug: /en/sql-reference/statements/alter/projection
 sidebar_position: 49
 sidebar_label: PROJECTION
-title: "Manipulating Projections"
+title: "Projections"
 ---
 
+Projections store data in a format that optimizes query execution, this feature is useful for:
+- Running queries on a column that is not a part of the primary key
+- Pre-aggregating columns, it will reduce both computation and IO
+
+You can define one or more projections for a table, and during the query analysis the projection with the least data to scan will be selected by ClickHouse without modifying the query provided by the user.
+
+## Example filtering without using primary keys
+
+Creating the table:
+```
+CREATE TABLE visits_order
+(
+   `user_id` UInt64,
+   `user_name` String,
+   `pages_visited` Nullable(Float64),
+   `user_agent` String
+)
+ENGINE = MergeTree()
+PRIMARY KEY user_agent
+```
+Using `ALTER TABLE`, we could add the Projection to an existing table:
+```
+ALTER TABLE visits_order ADD PROJECTION user_name_projection (
+SELECT
+* 
+ORDER BY user_name
+)
+
+ALTER TABLE visits_order MATERIALIZE PROJECTION user_name_projection
+```
+Inserting the data:
+```
+INSERT INTO visits_order SELECT
+    number,
+    'test',
+    1.5 * (number / 2),
+    'Android'
+FROM numbers(1, 100);
+```
+
+The Projection will allow us to filter by `user_name` fast even if in the original Table `user_name` was not defined as a `PRIMARY_KEY`.
+At query time ClickHouse determined that less data will be processed if the projection is used, as the data is ordered by `user_name`.
+```
+SELECT
+    *
+FROM visits_order
+WHERE user_name='test'
+LIMIT 2
+```
+
+To verify that a query is using the projection, we could review the `system.query_log` table. On the `projections` field we have the name of the projection used or empty if none has been used:
+```
+SELECT query, projections FROM system.query_log WHERE query_id='<query_id>'
+```
+
+## Example pre-aggregation query
+
+Creating the table with the Projection:
+```
+CREATE TABLE visits
+(
+   `user_id` UInt64,
+   `user_name` String,
+   `pages_visited` Nullable(Float64),
+   `user_agent` String,
+   PROJECTION projection_visits_by_user
+   (
+       SELECT
+           user_agent,
+           sum(pages_visited)
+       GROUP BY user_id, user_agent
+   )
+)
+ENGINE = MergeTree()
+ORDER BY user_agent
+```
+Inserting the data:
+```
+INSERT INTO visits SELECT
+    number,
+    'test',
+    1.5 * (number / 2),
+    'Android'
+FROM numbers(1, 100);
+```
+```
+INSERT INTO visits SELECT
+    number,
+    'test',
+    1. * (number / 2),
+   'IOS'
+FROM numbers(100, 500);
+```
+We will execute a first query using `GROUP BY` using the field `user_agent`, this query will not use the projection defined as the pre-aggregation does not match.
+```
+SELECT
+    user_agent,
+    count(DISTINCT user_id)
+FROM visits
+GROUP BY user_agent
+```
+
+To use the projection we could execute queries that select part of, or all of the pre-aggregation and `GROUP BY` fields.
+```
+SELECT
+    user_agent
+FROM visits
+WHERE user_id > 50 AND user_id < 150
+GROUP BY user_agent
+```
+```
+SELECT
+    user_agent,
+    sum(pages_visited)
+FROM visits
+GROUP BY user_id
+```
+
+As mentioned before, we could review the `system.query_log` table. On the `projections` field we have the name of the projection used or empty if none has been used:
+```
+SELECT query, projections FROM system.query_log WHERE query_id='<query_id>'
+```
+
+# Manipulating Projections
+
 The following operations with [projections](/docs/en/engines/table-engines/mergetree-family/mergetree.md/#projections) are available:
 
 ## ADD PROJECTION
@@ -13,15 +138,15 @@ The following operations with [projections](/docs/en/engines/table-engines/merge
 
 ## DROP PROJECTION
 
-`ALTER TABLE [db].name DROP PROJECTION name` - Removes projection description from tables metadata and deletes projection files from disk. Implemented as a [mutation](/docs/en/sql-reference/statements/alter/index.md/#mutations).
+`ALTER TABLE [db].name DROP PROJECTION name` - Removes projection description from tables metadata and deletes projection files from disk. Implemented as a [mutation](/docs/en/sql-reference/statements/alter/index.md#mutations).
 
 ## MATERIALIZE PROJECTION
 
-`ALTER TABLE [db.]table MATERIALIZE PROJECTION name IN PARTITION partition_name` - The query rebuilds the projection `name` in the partition `partition_name`. Implemented as a [mutation](/docs/en/sql-reference/statements/alter/index.md/#mutations).
+`ALTER TABLE [db.]table MATERIALIZE PROJECTION name IN PARTITION partition_name` - The query rebuilds the projection `name` in the partition `partition_name`. Implemented as a [mutation](/docs/en/sql-reference/statements/alter/index.md#mutations).
 
 ## CLEAR PROJECTION
 
-`ALTER TABLE [db.]table CLEAR PROJECTION name IN PARTITION partition_name` - Deletes projection files from disk without removing description. Implemented as a [mutation](/docs/en/sql-reference/statements/alter/index.md/#mutations).
+`ALTER TABLE [db.]table CLEAR PROJECTION name IN PARTITION partition_name` - Deletes projection files from disk without removing description. Implemented as a [mutation](/docs/en/sql-reference/statements/alter/index.md#mutations).
 
 
 The commands `ADD`, `DROP` and `CLEAR` are lightweight in a sense that they only change metadata or remove files.

docs/en/sql-reference/statements/alter/skipping-index.md

@@ -14,7 +14,7 @@ The following operations are available:
 
 -   `ALTER TABLE [db].table_name [ON CLUSTER cluster] DROP INDEX name` - Removes index description from tables metadata and deletes index files from disk.
 
--   `ALTER TABLE [db.]table_name [ON CLUSTER cluster] MATERIALIZE INDEX name [IN PARTITION partition_name]` - Rebuilds the secondary index `name` for the specified `partition_name`. Implemented as a [mutation](/docs/en/sql-reference/statements/alter/index.md/#mutations). If `IN PARTITION` part is omitted then it rebuilds the index for the whole table data.
+-   `ALTER TABLE [db.]table_name [ON CLUSTER cluster] MATERIALIZE INDEX name [IN PARTITION partition_name]` - Rebuilds the secondary index `name` for the specified `partition_name`. Implemented as a [mutation](/docs/en/sql-reference/statements/alter/index.md#mutations). If `IN PARTITION` part is omitted then it rebuilds the index for the whole table data.
 
 The first two commands are lightweight in a sense that they only change metadata or remove files.
 

docs/en/sql-reference/statements/alter/update.md

@@ -10,7 +10,7 @@ sidebar_label: UPDATE
 ALTER TABLE [db.]table [ON CLUSTER cluster] UPDATE column1 = expr1 [, ...] WHERE filter_expr
 ```
 
-Manipulates data matching the specified filtering expression. Implemented as a [mutation](/docs/en/sql-reference/statements/alter/index.md/#mutations).
+Manipulates data matching the specified filtering expression. Implemented as a [mutation](/docs/en/sql-reference/statements/alter/index.md#mutations).
 
 :::note    
 The `ALTER TABLE` prefix makes this syntax different from most other systems supporting SQL. It is intended to signify that unlike similar queries in OLTP databases this is a heavy operation not designed for frequent use.
@@ -24,7 +24,7 @@ The synchronicity of the query processing is defined by the [mutations_sync](/do
 
 **See also**
 
--   [Mutations](/docs/en/sql-reference/statements/alter/index.md/#mutations)
--   [Synchronicity of ALTER Queries](/docs/en/sql-reference/statements/alter/index.md/#synchronicity-of-alter-queries)
+-   [Mutations](/docs/en/sql-reference/statements/alter/index.md#mutations)
+-   [Synchronicity of ALTER Queries](/docs/en/sql-reference/statements/alter/index.md#synchronicity-of-alter-queries)
 -   [mutations_sync](/docs/en/operations/settings/settings.md/#mutations_sync) setting
 

docs/en/sql-reference/statements/create/dictionary.md

@@ -5,9 +5,9 @@ sidebar_label: DICTIONARY
 title: "CREATE DICTIONARY"
 ---
 
-Creates a new [external dictionary](../../../sql-reference/dictionaries/external-dictionaries/external-dicts.md) with given [structure](../../../sql-reference/dictionaries/external-dictionaries/external-dicts-dict-structure.md), [source](../../../sql-reference/dictionaries/external-dictionaries/external-dicts-dict-sources.md), [layout](../../../sql-reference/dictionaries/external-dictionaries/external-dicts-dict-layout.md) and [lifetime](../../../sql-reference/dictionaries/external-dictionaries/external-dicts-dict-lifetime.md).
+Creates a new [dictionary](../../../sql-reference/dictionaries/external-dictionaries/external-dicts.md) with given [structure](../../../sql-reference/dictionaries/external-dictionaries/external-dicts-dict-structure.md), [source](../../../sql-reference/dictionaries/external-dictionaries/external-dicts-dict-sources.md), [layout](../../../sql-reference/dictionaries/external-dictionaries/external-dicts-dict-layout.md) and [lifetime](../../../sql-reference/dictionaries/external-dictionaries/external-dicts-dict-lifetime.md).
 
-**Syntax**
+## Syntax
 
 ``` sql
 CREATE [OR REPLACE] DICTIONARY [IF NOT EXISTS] [db.]dictionary_name [ON CLUSTER cluster]
@@ -25,17 +25,21 @@ SETTINGS(setting_name = setting_value, setting_name = setting_value, ...)
 COMMENT 'Comment'
 ```
 
-External dictionary structure consists of attributes. Dictionary attributes are specified similarly to table columns. The only required attribute property is its type, all other properties may have default values.
+The dictionary structure consists of attributes. Dictionary attributes are specified similarly to table columns. The only required attribute property is its type, all other properties may have default values.
 
 `ON CLUSTER` clause allows creating dictionary on a cluster, see [Distributed DDL](../../../sql-reference/distributed-ddl.md).
 
 Depending on dictionary [layout](../../../sql-reference/dictionaries/external-dictionaries/external-dicts-dict-layout.md) one or more attributes can be specified as dictionary keys.
 
-For more information, see [External Dictionaries](../../../sql-reference/dictionaries/external-dictionaries/external-dicts.md) section.
+## SOURCE
 
-You can add a comment to the dictionary when you creating it using `COMMENT` clause.
+The source for a dictionary can be a:
+- table in the current ClickHouse service
+- table in a remote ClickHouse service
+- file available by HTTP(S)
+- another database
 
-**Example**
+### Create a dictionary from a table in the current ClickHouse service
 
 Input table `source_table`:
 
@@ -49,51 +53,81 @@ Input table `source_table`:
 Creating the dictionary:
 
 ``` sql
-CREATE DICTIONARY dictionary_with_comment
+CREATE DICTIONARY id_value_dictionary
 (
     id UInt64,
     value String
 )
 PRIMARY KEY id
-SOURCE(CLICKHOUSE(HOST 'localhost' PORT tcpPort() TABLE 'source_table'))
+SOURCE(CLICKHOUSE(TABLE 'source_table'))
 LAYOUT(FLAT())
 LIFETIME(MIN 0 MAX 1000)
-COMMENT 'The temporary dictionary';
 ```
 
 Output the dictionary:
 
 ``` sql
-SHOW CREATE DICTIONARY dictionary_with_comment;
+SHOW CREATE DICTIONARY id_value_dictionary;
 ```
 
-```text
-┌─statement───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┐
-│ CREATE DICTIONARY default.dictionary_with_comment
+```response
+CREATE DICTIONARY default.id_value_dictionary
 (
     `id` UInt64,
     `value` String
 )
 PRIMARY KEY id
-SOURCE(CLICKHOUSE(HOST 'localhost' PORT tcpPort() TABLE 'source_table'))
+SOURCE(CLICKHOUSE(TABLE 'source_table'))
 LIFETIME(MIN 0 MAX 1000)
 LAYOUT(FLAT())
-COMMENT 'The temporary dictionary' │
-└─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘
 ```
 
-Output the comment to dictionary:
+### Create a dictionary from a table in a remote ClickHouse service
+
+Input table (in the remote ClickHouse service) `source_table`:
+
+``` text
+┌─id─┬─value──┐
+│  1 │ First  │
+│  2 │ Second │
+└────┴────────┘
+```
+
+Creating the dictionary:
 
 ``` sql
-SELECT comment FROM system.dictionaries WHERE name == 'dictionary_with_comment' AND database == currentDatabase();
+CREATE DICTIONARY id_value_dictionary
+(
+    id UInt64,
+    value String
+)
+PRIMARY KEY id
+SOURCE(CLICKHOUSE(HOST 'HOSTNAME' PORT 9000 USER 'default' PASSWORD 'PASSWORD' TABLE 'source_table' DB 'default'))
+LAYOUT(FLAT())
+LIFETIME(MIN 0 MAX 1000)
 ```
 
-```text
-┌─comment──────────────────┐
-│ The temporary dictionary │
-└──────────────────────────┘
+### Create a dictionary from a file available by HTTP(S)
+
+```sql
+statement: CREATE DICTIONARY default.taxi_zone_dictionary
+(
+    `LocationID` UInt16 DEFAULT 0,
+    `Borough` String,
+    `Zone` String,
+    `service_zone` String
+)
+PRIMARY KEY LocationID
+SOURCE(HTTP(URL 'https://datasets-documentation.s3.eu-west-3.amazonaws.com/nyc-taxi/taxi_zone_lookup.csv' FORMAT 'CSVWithNames'))
+LIFETIME(MIN 0 MAX 0)
+LAYOUT(HASHED())
 ```
 
+### Create a dictionary from another database
+
+Please see the details in [Dictionary sources](/docs/en/sql-reference/dictionaries/external-dictionaries/external-dicts-dict-sources.md/#dbms).
+
 **See Also**
 
--   [system.dictionaries](../../../operations/system-tables/dictionaries.md) — This table contains information about [external dictionaries](../../../sql-reference/dictionaries/external-dictionaries/external-dicts.md).
+- For more information, see the [Dictionaries](../../../sql-reference/dictionaries/external-dictionaries/external-dicts.md) section.
+- [system.dictionaries](../../../operations/system-tables/dictionaries.md) — This table contains information about [Dictionaries](../../../sql-reference/dictionaries/external-dictionaries/external-dicts.md).

docs/en/sql-reference/statements/select/join.md

@@ -282,7 +282,7 @@ Each time a query is run with the same `JOIN`, the subquery is run again because
 
 In some cases, it is more efficient to use [IN](../../../sql-reference/operators/in.md) instead of `JOIN`.
 
-If you need a `JOIN` for joining with dimension tables (these are relatively small tables that contain dimension properties, such as names for advertising campaigns), a `JOIN` might not be very convenient due to the fact that the right table is re-accessed for every query. For such cases, there is an “external dictionaries” feature that you should use instead of `JOIN`. For more information, see the [External dictionaries](../../../sql-reference/dictionaries/external-dictionaries/external-dicts.md) section.
+If you need a `JOIN` for joining with dimension tables (these are relatively small tables that contain dimension properties, such as names for advertising campaigns), a `JOIN` might not be very convenient due to the fact that the right table is re-accessed for every query. For such cases, there is a “dictionaries” feature that you should use instead of `JOIN`. For more information, see the [Dictionaries](../../../sql-reference/dictionaries/external-dictionaries/external-dicts.md) section.
 
 ### Memory Limitations
 

docs/en/sql-reference/statements/set-role.md

@@ -41,7 +41,7 @@ Purge default roles from a user:
 SET DEFAULT ROLE NONE TO user
 ```
 
-Set all the granted roles as default excepting some of them:
+Set all the granted roles as default except for specific roles `role1` and `role2`:
 
 ``` sql
 SET DEFAULT ROLE ALL EXCEPT role1, role2 TO user

docs/en/sql-reference/statements/show.md

@@ -198,7 +198,7 @@ Result:
 
 ## SHOW DICTIONARIES
 
-Displays a list of [external dictionaries](../../sql-reference/dictionaries/external-dictionaries/external-dicts.md).
+Displays a list of [Dictionaries](../../sql-reference/dictionaries/external-dictionaries/external-dicts.md).
 
 ``` sql
 SHOW DICTIONARIES [FROM <db>] [LIKE '<pattern>'] [LIMIT <N>] [INTO OUTFILE <filename>] [FORMAT <format>]

docs/en/sql-reference/table-functions/mysql.md

@@ -110,5 +110,5 @@ SELECT * FROM mysql('localhost:3306', 'test', 'test', 'bayonet', '123');
 **See Also**
 
 -   [The ‘MySQL’ table engine](../../engines/table-engines/integrations/mysql.md)
--   [Using MySQL as a source of external dictionary](../../sql-reference/dictionaries/external-dictionaries/external-dicts-dict-sources.md#dicts-external_dicts_dict_sources-mysql)
+-   [Using MySQL as a dictionary source](../../sql-reference/dictionaries/external-dictionaries/external-dicts-dict-sources.md#dicts-external_dicts_dict_sources-mysql)
 

docs/en/sql-reference/table-functions/odbc.md

@@ -101,5 +101,5 @@ SELECT * FROM odbc('DSN=mysqlconn', 'test', 'test')
 
 ## See Also
 
--   [ODBC external dictionaries](../../sql-reference/dictionaries/external-dictionaries/external-dicts-dict-sources.md#dicts-external_dicts_dict_sources-odbc)
+-   [ODBC dictionaries](../../sql-reference/dictionaries/external-dictionaries/external-dicts-dict-sources.md#dicts-external_dicts_dict_sources-odbc)
 -   [ODBC table engine](../../engines/table-engines/integrations/odbc.md).

docs/en/sql-reference/table-functions/postgresql.md

@@ -130,6 +130,6 @@ CREATE TABLE pg_table_schema_with_dots (a UInt32)
 **See Also**
 
 -   [The PostgreSQL table engine](../../engines/table-engines/integrations/postgresql.md)
--   [Using PostgreSQL as a source of external dictionary](../../sql-reference/dictionaries/external-dictionaries/external-dicts-dict-sources.md#dicts-external_dicts_dict_sources-postgresql)
+-   [Using PostgreSQL as a dictionary source](../../sql-reference/dictionaries/external-dictionaries/external-dicts-dict-sources.md#dicts-external_dicts_dict_sources-postgresql)
 
 [Original article](https://clickhouse.com/docs/en/sql-reference/table-functions/postgresql/) <!--hide-->

docs/ru/operations/settings/index.md

@@ -24,7 +24,7 @@ slug: /ru/operations/settings/
 
     -   При запуске консольного клиента ClickHouse в не интерактивном режиме установите параметр запуска `--setting=value`.
     -   При использовании HTTP API передавайте cgi-параметры (`URL?setting_1=value&setting_2=value...`).
-    -  Укажите необходимые настройки в секции [SETTINGS](../../sql-reference/statements/select/index.md#settings-in-select) запроса SELECT. Эти настройки действуют только в рамках данного запроса, а после его выполнения сбрасываются до предыдущего значения или значения по умолчанию.
+    -  Укажите необходимые настройки в секции [SETTINGS](../../sql-reference/statements/select/index.md#settings-in-select-query) запроса SELECT. Эти настройки действуют только в рамках данного запроса, а после его выполнения сбрасываются до предыдущего значения или значения по умолчанию.
 
 Настройки, которые можно задать только в конфигурационном файле сервера, в разделе не рассматриваются.
 

docs/ru/operations/settings/settings.md

@@ -479,7 +479,7 @@ SELECT * FROM table_with_enum_column_for_tsv_insert;
 Включает или отключает вставку [значений по умолчанию](../../sql-reference/statements/create/table.md#create-default-values) вместо [NULL](../../sql-reference/syntax.md#null-literal) в столбцы, которые не позволяют [хранить NULL](../../sql-reference/data-types/nullable.md#data_type-nullable).
 Если столбец не позволяет хранить `NULL` и эта настройка отключена, то вставка `NULL` приведет к возникновению исключения. Если столбец позволяет хранить `NULL`, то значения `NULL` вставляются независимо от этой настройки.
 
-Эта настройка используется для запросов [INSERT ... SELECT](../../sql-reference/statements/insert-into.md#insert_query_insert-select). При этом подзапросы `SELECT` могут объединяться с помощью `UNION ALL`.
+Эта настройка используется для запросов [INSERT ... SELECT](../../sql-reference/statements/insert-into.md#inserting-the-results-of-select). При этом подзапросы `SELECT` могут объединяться с помощью `UNION ALL`.
 
 Возможные значения:
 

docs/ru/operations/system-tables/crash-log.md

@@ -7,8 +7,8 @@ slug: /ru/operations/system-tables/crash-log
 
 Колонки:
 
--   `event_date` ([Datetime](../../sql-reference/data-types/datetime.md)) — Дата события.
--   `event_time` ([Datetime](../../sql-reference/data-types/datetime.md)) — Время события.
+-   `event_date` ([DateTime](../../sql-reference/data-types/datetime.md)) — Дата события.
+-   `event_time` ([DateTime](../../sql-reference/data-types/datetime.md)) — Время события.
 -   `timestamp_ns` ([UInt64](../../sql-reference/data-types/int-uint.md)) — Время события с наносекундами.
 -   `signal` ([Int32](../../sql-reference/data-types/int-uint.md)) — Номер сигнала, пришедшего в поток.
 -   `thread_id` ([UInt64](../../sql-reference/data-types/int-uint.md)) — Идентификатор треда.

docs/ru/operations/system-tables/mutations.md

@@ -15,7 +15,7 @@ slug: /ru/operations/system-tables/mutations
 
 -   `command` ([String](../../sql-reference/data-types/string.md)) — команда мутации (часть запроса после `ALTER TABLE [db.]table`).
 
--   `create_time` ([Datetime](../../sql-reference/data-types/datetime.md)) — дата и время создания мутации.
+-   `create_time` ([DateTime](../../sql-reference/data-types/datetime.md)) — дата и время создания мутации.
 
 -   `block_numbers.partition_id` ([Array](../../sql-reference/data-types/array.md)([String](../../sql-reference/data-types/string.md))) — Для мутаций реплицированных таблиц массив содержит содержит номера партиций (по одной записи для каждой партиции). Для мутаций нереплицированных таблиц массив пустой.
 
@@ -39,7 +39,7 @@ slug: /ru/operations/system-tables/mutations
 
 -   `latest_failed_part` ([String](../../sql-reference/data-types/string.md)) — имя последнего куска, мутация которого не удалась.
 
--   `latest_fail_time` ([Datetime](../../sql-reference/data-types/datetime.md)) — дата и время последней ошибки мутации.
+-   `latest_fail_time` ([DateTime](../../sql-reference/data-types/datetime.md)) — дата и время последней ошибки мутации.
 
 -   `latest_fail_reason` ([String](../../sql-reference/data-types/string.md)) — причина последней ошибки мутации.
 

docs/ru/operations/system-tables/replication_queue.md

@@ -29,7 +29,7 @@ slug: /ru/operations/system-tables/replication_queue
     -   `MUTATE_PART` — применить одну или несколько мутаций к куску.
     -   `ALTER_METADATA` — применить изменения структуры таблицы в результате запросов с выражением `ALTER`.
 
--   `create_time` ([Datetime](../../sql-reference/data-types/datetime.md)) — дата и время отправки задачи на выполнение.
+-   `create_time` ([DateTime](../../sql-reference/data-types/datetime.md)) — дата и время отправки задачи на выполнение.
 
 -   `required_quorum` ([UInt32](../../sql-reference/data-types/int-uint.md)) — количество реплик, ожидающих завершения задачи, с подтверждением о завершении. Этот столбец актуален только для задачи `GET_PARTS`.
 
@@ -47,13 +47,13 @@ slug: /ru/operations/system-tables/replication_queue
 
 -   `last_exception` ([String](../../sql-reference/data-types/string.md)) — текст сообщения о последней возникшей ошибке, если таковые имеются.
 
--   `last_attempt_time` ([Datetime](../../sql-reference/data-types/datetime.md)) — дата и время последней попытки выполнить задачу.
+-   `last_attempt_time` ([DateTime](../../sql-reference/data-types/datetime.md)) — дата и время последней попытки выполнить задачу.
 
 -   `num_postponed` ([UInt32](../../sql-reference/data-types/int-uint.md)) — количество отложенных задач.
 
 -   `postpone_reason` ([String](../../sql-reference/data-types/string.md)) — причина, по которой была отложена задача.
 
--   `last_postpone_time` ([Datetime](../../sql-reference/data-types/datetime.md)) — дата и время, когда была отложена задача в последний раз.
+-   `last_postpone_time` ([DateTime](../../sql-reference/data-types/datetime.md)) — дата и время, когда была отложена задача в последний раз.
 
 -   `merge_type` ([String](../../sql-reference/data-types/string.md)) — тип текущего слияния. Пусто, если это мутация.
 

docs/ru/sql-reference/data-types/date32.md

@@ -6,7 +6,7 @@ sidebar_label: Date32
 
 # Date32 {#data_type-datetime32}
 
-Дата. Поддерживается такой же диапазон дат, как для типа [Datetime64](../../sql-reference/data-types/datetime64.md). Значение хранится в четырех байтах и соответствует числу дней с 1900-01-01 по 2299-12-31.
+Дата. Поддерживается такой же диапазон дат, как для типа [DateTime64](../../sql-reference/data-types/datetime64.md). Значение хранится в четырех байтах и соответствует числу дней с 1900-01-01 по 2299-12-31.
 
 **Пример**
 

docs/ru/sql-reference/functions/date-time-functions.md

@@ -602,7 +602,7 @@ date_trunc(unit, value[, timezone])
 
 -   Дата и время, отсеченные до указанной части.
 
-Тип: [Datetime](../../sql-reference/data-types/datetime.md).
+Тип: [DateTime](../../sql-reference/data-types/datetime.md).
 
 **Примеры**
 
@@ -913,7 +913,7 @@ now([timezone])
 
 -   Текущие дата и время.
 
-Тип: [Datetime](../../sql-reference/data-types/datetime.md).
+Тип: [DateTime](../../sql-reference/data-types/datetime.md).
 
 **Пример**
 

docs/ru/sql-reference/statements/alter/column.md

@@ -254,7 +254,7 @@ SELECT groupArray(x), groupArray(s) FROM tmp;
 
 Отсутствует возможность удалять столбцы, входящие в первичный ключ или ключ для сэмплирования (в общем, входящие в выражение `ENGINE`). Изменение типа у столбцов, входящих в первичный ключ возможно только в том случае, если это изменение не приводит к изменению данных (например, разрешено добавление значения в Enum или изменение типа с `DateTime` на `UInt32`).
 
-Если возможностей запроса `ALTER` не хватает для нужного изменения таблицы, вы можете создать новую таблицу, скопировать туда данные с помощью запроса [INSERT SELECT](../insert-into.md#insert_query_insert-select), затем поменять таблицы местами с помощью запроса [RENAME](../rename.md#rename-table), и удалить старую таблицу. В качестве альтернативы для запроса `INSERT SELECT`, можно использовать инструмент [clickhouse-copier](../../../sql-reference/statements/alter/index.md).
+Если возможностей запроса `ALTER` не хватает для нужного изменения таблицы, вы можете создать новую таблицу, скопировать туда данные с помощью запроса [INSERT SELECT](../insert-into.md#inserting-the-results-of-select), затем поменять таблицы местами с помощью запроса [RENAME](../rename.md#rename-table), и удалить старую таблицу. В качестве альтернативы для запроса `INSERT SELECT`, можно использовать инструмент [clickhouse-copier](../../../sql-reference/statements/alter/index.md).
 
 Запрос `ALTER` блокирует все чтения и записи для таблицы. То есть если на момент запроса `ALTER` выполнялся долгий `SELECT`, то запрос `ALTER` сначала дождётся его выполнения. И в это время все новые запросы к той же таблице будут ждать, пока завершится этот `ALTER`.
 

docs/ru/sql-reference/statements/insert-into.md

@@ -95,7 +95,7 @@ INSERT INTO t FORMAT TabSeparated
 
 Если в таблице объявлены [ограничения](../../sql-reference/statements/create/table.md#constraints), то их выполнимость будет проверена для каждой вставляемой строки. Если для хотя бы одной строки ограничения не будут выполнены, запрос будет остановлен.
 
-### Вставка результатов `SELECT` {#insert_query_insert-select}
+### Вставка результатов `SELECT` {#inserting-the-results-of-select}
 
 **Синтаксис**
 

docs/ru/sql-reference/statements/select/index.md

@@ -270,7 +270,7 @@ SELECT * REPLACE(i + 1 AS i) EXCEPT (j) APPLY(sum) from columns_transformers;
 └─────────────────┴────────┘
 ```
 
-## SETTINGS в запросе SELECT {#settings-in-select}
+## SETTINGS в запросе SELECT {#settings-in-select-query}
 
 Вы можете задать значения необходимых настроек непосредственно в запросе `SELECT` в секции `SETTINGS`. Эти настройки действуют только в рамках данного запроса, а после его выполнения сбрасываются до предыдущего значения или значения по умолчанию.
 

docs/zh/operations/system-tables/crash-log.md

@@ -7,8 +7,8 @@ slug: /zh/operations/system-tables/crash-log
 
 列信息:
 
--   `event_date` ([Datetime](../../sql-reference/data-types/datetime.md)) — 事件日期.
--   `event_time` ([Datetime](../../sql-reference/data-types/datetime.md)) — 事件时间.
+-   `event_date` ([DateTime](../../sql-reference/data-types/datetime.md)) — 事件日期.
+-   `event_time` ([DateTime](../../sql-reference/data-types/datetime.md)) — 事件时间.
 -   `timestamp_ns` ([UInt64](../../sql-reference/data-types/int-uint.md)) — 以纳秒为单位的事件时间戳.
 -   `signal` ([Int32](../../sql-reference/data-types/int-uint.md)) — 信号编号.
 -   `thread_id` ([UInt64](../../sql-reference/data-types/int-uint.md)) — 线程ID.

docs/zh/operations/system-tables/mutations.md

@@ -15,7 +15,7 @@ slug: /zh/operations/system-tables/mutations
 
 -   `command` ([String](../../sql-reference/data-types/string.md)) — mutation命令字符串（`ALTER TABLE [db.]table`语句之后的部分)。
 
--   `create_time` ([Datetime](../../sql-reference/data-types/datetime.md)) — mutation命令提交执行的日期和时间。
+-   `create_time` ([DateTime](../../sql-reference/data-types/datetime.md)) — mutation命令提交执行的日期和时间。
 
 -   `block_numbers.partition_id` ([Array](../../sql-reference/data-types/array.md)([String](../../sql-reference/data-types/string.md))) — 对于复制表的mutation，该数组包含分区的ID（每个分区都有一条记录）。对于非复制表的mutation，该数组为空。
 
@@ -39,7 +39,7 @@ slug: /zh/operations/system-tables/mutations
 
 -   `latest_failed_part`([String](../../sql-reference/data-types/string.md)) — 最近不能mutation的part的名称。
 
--   `latest_fail_time`([Datetime](../../sql-reference/data-types/datetime.md)) — 最近的一个mutation失败的时间。
+-   `latest_fail_time`([DateTime](../../sql-reference/data-types/datetime.md)) — 最近的一个mutation失败的时间。
 
 -   `latest_fail_reason`([String](../../sql-reference/data-types/string.md)) — 导致最近part的mutation失败的异常消息。
 

docs/zh/operations/system-tables/replication_queue.md

@@ -29,7 +29,7 @@ slug: /zh/operations/system-tables/replication_queue
     -   `MUTATE_PART` — 对分片应用一个或多个突变.
     -   `ALTER_METADATA` — 根据全局 /metadata 和 /columns 路径应用alter修改.
 
--   `create_time` ([Datetime](../../sql-reference/data-types/datetime.md)) — 提交任务执行的日期和时间.
+-   `create_time` ([DateTime](../../sql-reference/data-types/datetime.md)) — 提交任务执行的日期和时间.
 
 -   `required_quorum` ([UInt32](../../sql-reference/data-types/int-uint.md)) — 等待任务完成并确认完成的副本数. 此列仅与 `GET_PARTS` 任务相关.
 
@@ -47,13 +47,13 @@ slug: /zh/operations/system-tables/replication_queue
 
 -   `last_exception` ([String](../../sql-reference/data-types/string.md)) — 发生的最后一个错误的短信（如果有）.
 
--   `last_attempt_time` ([Datetime](../../sql-reference/data-types/datetime.md)) — 上次尝试任务的日期和时间.
+-   `last_attempt_time` ([DateTime](../../sql-reference/data-types/datetime.md)) — 上次尝试任务的日期和时间.
 
 -   `num_postponed` ([UInt32](../../sql-reference/data-types/int-uint.md)) — 延期任务数.
 
 -   `postpone_reason` ([String](../../sql-reference/data-types/string.md)) — 任务延期的原因.
 
--   `last_postpone_time` ([Datetime](../../sql-reference/data-types/datetime.md)) — 上次推迟任务的日期和时间.
+-   `last_postpone_time` ([DateTime](../../sql-reference/data-types/datetime.md)) — 上次推迟任务的日期和时间.
 
 -   `merge_type` ([String](../../sql-reference/data-types/string.md)) — 当前合并的类型. 如果是突变则为空.
 

docs/zh/sql-reference/ansi.md

@@ -152,7 +152,7 @@ sidebar_label: "ANSI\u517C\u5BB9\u6027"
 | F051-02  | TIME（时间）数据类型（并支持用于表达时间的字面量），小数秒精度至少为0                    | 否 {.text-danger}       |                                                                                                                                                                          |
 | F051-03  | 时间戳数据类型（并支持用于表达时间戳的字面量），小数秒精度至少为0和6                     | 是 {.text-danger}       |                                                                                                                                                                          |
 | F051-04  | 日期、时间和时间戳数据类型的比较谓词                                                     | 是 {.text-success}      |                                                                                                                                                                          |
-| F051-05  | Datetime 类型和字符串形式表达的时间之间的显式转换                                         | 是 {.text-success}      |                                                                                                                                                                          |
+| F051-05  | DateTime 类型和字符串形式表达的时间之间的显式转换                                         | 是 {.text-success}      |                                                                                                                                                                          |
 | F051-06  | CURRENT_DATE                                                                             | 否 {.text-danger}       | 使用`today()`替代                                                                                                                                                        |
 | F051-07  | LOCALTIME                                                                                | 否 {.text-danger}       | 使用`now()`替代                                                                                                                                                          |
 | F051-08  | LOCALTIMESTAMP                                                                           | 否 {.text-danger}       |                                                                                                                                                                          |

docs/zh/sql-reference/data-types/datetime64.md

@@ -6,7 +6,7 @@ sidebar_position: 49
 sidebar_label: DateTime64
 ---
 
-# Datetime64 {#data_type-datetime64}
+# DateTime64 {#data_type-datetime64}
 
 此类型允许以日期（date）加时间（time）的形式来存储一个时刻的时间值，具有定义的亚秒精度
 

docs/zh/sql-reference/functions/date-time-functions.md

@@ -539,7 +539,7 @@ date_trunc(unit, value[, timezone])
 
 -   按指定的单位向前取整后的DateTime。
 
-类型: [Datetime](../../sql-reference/data-types/datetime.md).
+类型: [DateTime](../../sql-reference/data-types/datetime.md).
 
 **示例**
 
@@ -850,7 +850,7 @@ now([timezone])
 
 -   当前日期和时间。
 
-类型: [Datetime](../../sql-reference/data-types/datetime.md).
+类型: [DateTime](../../sql-reference/data-types/datetime.md).
 
 **示例**
 

docs/zh/sql-reference/statements/alter.md

@@ -150,7 +150,7 @@ ALTER TABLE visits MODIFY COLUMN browser Array(String)
 
 不支持对primary key或者sampling key中的列（在 `ENGINE` 表达式中用到的列）进行删除操作。改变包含在primary key中的列的类型时，如果操作不会导致数据的变化（例如，往Enum中添加一个值，或者将`DateTime` 类型改成 `UInt32`），那么这种操作是可行的。
 
-如果 `ALTER` 操作不足以完成你想要的表变动操作，你可以创建一张新的表，通过 [INSERT SELECT](../../sql-reference/statements/insert-into.md#insert_query_insert-select)将数据拷贝进去，然后通过  [RENAME](../../sql-reference/statements/misc.md#misc_operations-rename)将新的表改成和原有表一样的名称，并删除原有的表。你可以使用 [clickhouse-copier](../../operations/utilities/clickhouse-copier.md) 代替 `INSERT SELECT`。
+如果 `ALTER` 操作不足以完成你想要的表变动操作，你可以创建一张新的表，通过 [INSERT SELECT](../../sql-reference/statements/insert-into.md#inserting-the-results-of-select)将数据拷贝进去，然后通过  [RENAME](../../sql-reference/statements/misc.md#misc_operations-rename)将新的表改成和原有表一样的名称，并删除原有的表。你可以使用 [clickhouse-copier](../../operations/utilities/clickhouse-copier.md) 代替 `INSERT SELECT`。
 
  `ALTER` 操作会阻塞对表的所有读写操作。换句话说，当一个大的 `SELECT` 语句和 `ALTER`同时执行时，`ALTER`会等待，直到 `SELECT` 执行结束。与此同时，当 `ALTER` 运行时，新的 sql 语句将会等待。
 

docs/zh/sql-reference/statements/insert-into.md

@@ -90,7 +90,7 @@ INSERT INTO t FORMAT TabSeparated
 
 如果表中有一些[限制](../../sql-reference/statements/create/table.mdx#constraints),，数据插入时会逐行进行数据校验，如果这里面包含了不符合限制条件的数据，服务将会抛出包含限制信息的异常，这个语句也会被停止执行。
 
-### 使用`SELECT`的结果写入 {#insert_query_insert-select}
+### 使用`SELECT`的结果写入 {#inserting-the-results-of-select}
 
 ``` sql
 INSERT INTO [db.]table [(c1, c2, c3)] SELECT ...

programs/git-import/git-import.cpp

@@ -351,7 +351,7 @@ struct LineChange
             ++pos;
         }
 
-        indent = std::max(255U, num_spaces);
+        indent = std::min(255U, num_spaces);
         line.assign(pos, end);
 
         if (pos == end)

programs/keeper/Keeper.cpp

@@ -149,19 +149,7 @@ std::string getUserName(uid_t user_id)
 Poco::Net::SocketAddress Keeper::socketBindListen(Poco::Net::ServerSocket & socket, const std::string & host, UInt16 port, [[maybe_unused]] bool secure) const
 {
     auto address = makeSocketAddress(host, port, &logger());
-#if !defined(POCO_CLICKHOUSE_PATCH) || POCO_VERSION < 0x01090100
-    if (secure)
-        /// Bug in old (<1.9.1) poco, listen() after bind() with reusePort param will fail because have no implementation in SecureServerSocketImpl
-        /// https://github.com/pocoproject/poco/pull/2257
-        socket.bind(address, /* reuseAddress = */ true);
-    else
-#endif
-#if POCO_VERSION < 0x01080000
-    socket.bind(address, /* reuseAddress = */ true);
-#else
     socket.bind(address, /* reuseAddress = */ true, /* reusePort = */ config().getBool("listen_reuse_port", false));
-#endif
-
     socket.listen(/* backlog = */ config().getUInt("listen_backlog", 64));
 
     return address;

programs/server/Server.cpp

@@ -342,19 +342,7 @@ Poco::Net::SocketAddress Server::socketBindListen(
     [[maybe_unused]] bool secure) const
 {
     auto address = makeSocketAddress(host, port, &logger());
-#if !defined(POCO_CLICKHOUSE_PATCH) || POCO_VERSION < 0x01090100
-    if (secure)
-        /// Bug in old (<1.9.1) poco, listen() after bind() with reusePort param will fail because have no implementation in SecureServerSocketImpl
-        /// https://github.com/pocoproject/poco/pull/2257
-        socket.bind(address, /* reuseAddress = */ true);
-    else
-#endif
-#if POCO_VERSION < 0x01080000
-    socket.bind(address, /* reuseAddress = */ true);
-#else
     socket.bind(address, /* reuseAddress = */ true, /* reusePort = */ config.getBool("listen_reuse_port", false));
-#endif
-
     /// If caller requests any available port from the OS, discover it after binding.
     if (port == 0)
     {
@@ -808,41 +796,43 @@ int Server::main(const std::vector<std::string> & /*args*/)
         /// that are interpreted (not executed) but can alter the behaviour of the program as well.
 
         /// Please keep the below log messages in-sync with the ones in daemon/BaseDaemon.cpp
-
-        String calculated_binary_hash = getHashOfLoadedBinaryHex();
-
         if (stored_binary_hash.empty())
         {
-            LOG_WARNING(log, "Integrity check of the executable skipped because the reference checksum could not be read."
-                " (calculated checksum: {})", calculated_binary_hash);
-        }
-        else if (calculated_binary_hash == stored_binary_hash)
-        {
-            LOG_INFO(log, "Integrity check of the executable successfully passed (checksum: {})", calculated_binary_hash);
+            LOG_WARNING(log, "Integrity check of the executable skipped because the reference checksum could not be read.");
         }
         else
         {
-            /// If program is run under debugger, ptrace will fail.
-            if (ptrace(PTRACE_TRACEME, 0, nullptr, nullptr) == -1)
+            String calculated_binary_hash = getHashOfLoadedBinaryHex();
+            if (calculated_binary_hash == stored_binary_hash)
             {
-                /// Program is run under debugger. Modification of it's binary image is ok for breakpoints.
-                global_context->addWarningMessage(
-                    fmt::format("Server is run under debugger and its binary image is modified (most likely with breakpoints).",
-                    calculated_binary_hash)
-                );
+                LOG_INFO(log, "Integrity check of the executable successfully passed (checksum: {})", calculated_binary_hash);
             }
             else
             {
-                throw Exception(ErrorCodes::CORRUPTED_DATA,
-                    "Calculated checksum of the executable ({0}) does not correspond"
-                    " to the reference checksum stored in the executable ({1})."
-                    " This may indicate one of the following:"
-                    " - the executable {2} was changed just after startup;"
-                    " - the executable {2} was corrupted on disk due to faulty hardware;"
-                    " - the loaded executable was corrupted in memory due to faulty hardware;"
-                    " - the file {2} was intentionally modified;"
-                    " - a logical error in the code."
-                    , calculated_binary_hash, stored_binary_hash, executable_path);
+                /// If program is run under debugger, ptrace will fail.
+                if (ptrace(PTRACE_TRACEME, 0, nullptr, nullptr) == -1)
+                {
+                    /// Program is run under debugger. Modification of it's binary image is ok for breakpoints.
+                    global_context->addWarningMessage(fmt::format(
+                        "Server is run under debugger and its binary image is modified (most likely with breakpoints).",
+                        calculated_binary_hash));
+                }
+                else
+                {
+                    throw Exception(
+                        ErrorCodes::CORRUPTED_DATA,
+                        "Calculated checksum of the executable ({0}) does not correspond"
+                        " to the reference checksum stored in the executable ({1})."
+                        " This may indicate one of the following:"
+                        " - the executable {2} was changed just after startup;"
+                        " - the executable {2} was corrupted on disk due to faulty hardware;"
+                        " - the loaded executable was corrupted in memory due to faulty hardware;"
+                        " - the file {2} was intentionally modified;"
+                        " - a logical error in the code.",
+                        calculated_binary_hash,
+                        stored_binary_hash,
+                        executable_path);
+                }
             }
         }
     }

src/Access/ContextAccess.cpp

@@ -465,6 +465,17 @@ std::shared_ptr<const AccessRights> ContextAccess::getAccessRightsWithImplicit()
 template <bool throw_if_denied, bool grant_option, typename... Args>
 bool ContextAccess::checkAccessImplHelper(AccessFlags flags, const Args &... args) const
 {
+    if (user_was_dropped)
+    {
+        /// If the current user has been dropped we always throw an exception (even if `throw_if_denied` is false)
+        /// because dropping of the current user is considered as a situation which is exceptional enough to stop
+        /// query execution.
+        throw Exception(getUserName() + ": User has been dropped", ErrorCodes::UNKNOWN_USER);
+    }
+
+    if (is_full_access)
+        return true;
+
     auto access_granted = [&]
     {
         if (trace_log)
@@ -483,12 +494,6 @@ bool ContextAccess::checkAccessImplHelper(AccessFlags flags, const Args &... arg
         return false;
     };
 
-    if (is_full_access)
-        return true;
-
-    if (user_was_dropped)
-        return access_denied("User has been dropped", ErrorCodes::UNKNOWN_USER);
-
     if (flags & AccessType::CLUSTER && !access_control->doesOnClusterQueriesRequireClusterGrant())
         flags &= ~AccessType::CLUSTER;
 

src/AggregateFunctions/AggregateFunctionAggThrow.cpp

@@ -95,7 +95,7 @@ public:
     void deserialize(AggregateDataPtr __restrict /* place */, ReadBuffer & buf, std::optional<size_t> /* version */, Arena *) const override
     {
         char c = 0;
-        buf.read(c);
+        buf.readStrict(c);
     }
 
     void insertResultInto(AggregateDataPtr __restrict, IColumn & to, Arena *) const override

src/AggregateFunctions/AggregateFunctionCategoricalInformationValue.cpp

@@ -118,7 +118,7 @@ public:
 
     void deserialize(AggregateDataPtr __restrict place, ReadBuffer & buf, std::optional<size_t> /* version */, Arena *) const override
     {
-        buf.read(place, sizeOfData());
+        buf.readStrict(place, sizeOfData());
     }
 
     DataTypePtr getReturnType() const override

src/AggregateFunctions/AggregateFunctionGroupArray.h

@@ -270,7 +270,7 @@ public:
         auto & value = this->data(place).value;
 
         value.resize(size, arena);
-        buf.read(reinterpret_cast<char *>(value.data()), size * sizeof(value[0]));
+        buf.readStrict(reinterpret_cast<char *>(value.data()), size * sizeof(value[0]));
 
         if constexpr (Trait::sampler == Sampler::RNG)
         {
@@ -343,7 +343,7 @@ struct GroupArrayNodeBase
 
         Node * node = reinterpret_cast<Node *>(arena->alignedAlloc(sizeof(Node) + size, alignof(Node)));
         node->size = size;
-        buf.read(node->data(), size);
+        buf.readStrict(node->data(), size);
         return node;
     }
 };

src/AggregateFunctions/AggregateFunctionGroupArrayMoving.h

@@ -144,7 +144,7 @@ public:
         {
             auto & value = this->data(place).value;
             value.resize(size, arena);
-            buf.read(reinterpret_cast<char *>(value.data()), size * sizeof(value[0]));
+            buf.readStrict(reinterpret_cast<char *>(value.data()), size * sizeof(value[0]));
             this->data(place).sum = value.back();
         }
     }

src/AggregateFunctions/AggregateFunctionHistogram.h

@@ -293,7 +293,7 @@ public:
         if (size > max_bins * 2)
             throw Exception("Too many bins", ErrorCodes::TOO_LARGE_ARRAY_SIZE);
 
-        buf.read(reinterpret_cast<char *>(points), size * sizeof(WeightedValue));
+        buf.readStrict(reinterpret_cast<char *>(points), size * sizeof(WeightedValue));
     }
 };
 

src/AggregateFunctions/AggregateFunctionMaxIntersections.h

@@ -130,7 +130,7 @@ public:
         auto & value = this->data(place).value;
 
         value.resize(size, arena);
-        buf.read(reinterpret_cast<char *>(value.data()), size * sizeof(value[0]));
+        buf.readStrict(reinterpret_cast<char *>(value.data()), size * sizeof(value[0]));
     }
 
     void insertResultInto(AggregateDataPtr __restrict place, IColumn & to, Arena *) const override

src/AggregateFunctions/AggregateFunctionMinMaxAny.h

@@ -515,7 +515,7 @@ public:
                 size = rhs_size;
 
                 if (size > 0)
-                    buf.read(small_data, size);
+                    buf.readStrict(small_data, size);
             }
             else
             {
@@ -527,7 +527,7 @@ public:
                 }
 
                 size = rhs_size;
-                buf.read(large_data, size);
+                buf.readStrict(large_data, size);
             }
         }
         else

src/AggregateFunctions/AggregateFunctionSequenceNextNode.h

@@ -29,6 +29,11 @@ namespace DB
 {
 struct Settings;
 
+namespace ErrorCodes
+{
+    extern const int TOO_LARGE_ARRAY_SIZE;
+}
+
 enum class SequenceDirection
 {
     Forward,
@@ -43,6 +48,9 @@ enum SequenceBase
     LastMatch,
 };
 
+/// This is for security
+static const UInt64 max_node_size_deserialize = 0xFFFFFF;
+
 /// NodeBase used to implement a linked list for storage of SequenceNextNodeImpl
 template <typename Node, size_t MaxEventsSize>
 struct NodeBase
@@ -78,10 +86,12 @@ struct NodeBase
     {
         UInt64 size;
         readVarUInt(size, buf);
+        if unlikely (size > max_node_size_deserialize)
+            throw Exception("Too large node state size", ErrorCodes::TOO_LARGE_ARRAY_SIZE);
 
         Node * node = reinterpret_cast<Node *>(arena->alignedAlloc(sizeof(Node) + size, alignof(Node)));
         node->size = size;
-        buf.read(node->data(), size);
+        buf.readStrict(node->data(), size);
 
         readBinary(node->event_time, buf);
         UInt64 ulong_bitset;

src/AggregateFunctions/AggregateFunctionUniqUpTo.h

@@ -108,7 +108,7 @@ struct AggregateFunctionUniqUpToData
         readBinary(count, rb);
 
         if (count <= threshold)
-            rb.read(data_ptr, count * sizeof(T));
+            rb.readStrict(data_ptr, count * sizeof(T));
     }
 
     /// ALWAYS_INLINE is required to have better code layout for uniqUpTo function

src/AggregateFunctions/QuantileExact.h

@@ -55,7 +55,7 @@ struct QuantileExactBase
         size_t size = 0;
         readVarUInt(size, buf);
         array.resize(size);
-        buf.read(reinterpret_cast<char *>(array.data()), size * sizeof(array[0]));
+        buf.readStrict(reinterpret_cast<char *>(array.data()), size * sizeof(array[0]));
     }
 
     Value get(Float64 level)

src/AggregateFunctions/QuantileTDigest.h

@@ -316,7 +316,7 @@ public:
 
         centroids.resize(size);
         // From now, TDigest will be in invalid state if exception is thrown.
-        buf.read(reinterpret_cast<char *>(centroids.data()), size * sizeof(centroids[0]));
+        buf.readStrict(reinterpret_cast<char *>(centroids.data()), size * sizeof(centroids[0]));
 
         for (const auto & c : centroids)
         {

src/AggregateFunctions/StatCommon.h

@@ -112,8 +112,8 @@ struct StatisticalSample
         readVarUInt(size_y, buf);
         x.resize(size_x, arena);
         y.resize(size_y, arena);
-        buf.read(reinterpret_cast<char *>(x.data()), size_x * sizeof(x[0]));
-        buf.read(reinterpret_cast<char *>(y.data()), size_y * sizeof(y[0]));
+        buf.readStrict(reinterpret_cast<char *>(x.data()), size_x * sizeof(x[0]));
+        buf.readStrict(reinterpret_cast<char *>(y.data()), size_y * sizeof(y[0]));
     }
 };
 

src/Backups/BackupEntriesCollector.cpp

@@ -442,7 +442,7 @@ void BackupEntriesCollector::gatherTablesMetadata()
                 if (it != database_info.tables.end())
                 {
                     const auto & partitions = it->second.partitions;
-                    if (partitions && !storage->supportsBackupPartition())
+                    if (partitions && storage && !storage->supportsBackupPartition())
                     {
                         throw Exception(
                             ErrorCodes::CANNOT_BACKUP_TABLE,

src/Backups/BackupIO_S3.cpp

@@ -46,7 +46,7 @@ namespace
             context->getRemoteHostFilter(),
             static_cast<unsigned>(context->getGlobalContext()->getSettingsRef().s3_max_redirects),
             context->getGlobalContext()->getSettingsRef().enable_s3_requests_logging,
-            /* for_disk_s3 = */ false);
+            /* for_disk_s3 = */ false, /* get_request_throttler = */ {}, /* put_request_throttler = */ {});
 
         client_configuration.endpointOverride = s3_uri.endpoint;
         client_configuration.maxConnections = static_cast<unsigned>(context->getSettingsRef().s3_max_connections);
@@ -86,9 +86,10 @@ BackupReaderS3::BackupReaderS3(
     const S3::URI & s3_uri_, const String & access_key_id_, const String & secret_access_key_, const ContextPtr & context_)
     : s3_uri(s3_uri_)
     , client(makeS3Client(s3_uri_, access_key_id_, secret_access_key_, context_))
-    , max_single_read_retries(context_->getSettingsRef().s3_max_single_read_retries)
     , read_settings(context_->getReadSettings())
+    , request_settings(context_->getStorageS3Settings().getSettings(s3_uri.uri.toString()).request_settings)
 {
+    request_settings.max_single_read_retries = context_->getSettingsRef().s3_max_single_read_retries; // FIXME: Avoid taking value for endpoint
 }
 
 DataSourceDescription BackupReaderS3::getDataSourceDescription() const
@@ -115,7 +116,7 @@ UInt64 BackupReaderS3::getFileSize(const String & file_name)
 std::unique_ptr<SeekableReadBuffer> BackupReaderS3::readFile(const String & file_name)
 {
     return std::make_unique<ReadBufferFromS3>(
-        client, s3_uri.bucket, fs::path(s3_uri.key) / file_name, s3_uri.version_id, max_single_read_retries, read_settings);
+        client, s3_uri.bucket, fs::path(s3_uri.key) / file_name, s3_uri.version_id, request_settings, read_settings);
 }
 
 
@@ -123,12 +124,12 @@ BackupWriterS3::BackupWriterS3(
     const S3::URI & s3_uri_, const String & access_key_id_, const String & secret_access_key_, const ContextPtr & context_)
     : s3_uri(s3_uri_)
     , client(makeS3Client(s3_uri_, access_key_id_, secret_access_key_, context_))
-    , max_single_read_retries(context_->getSettingsRef().s3_max_single_read_retries)
     , read_settings(context_->getReadSettings())
-    , rw_settings(context_->getStorageS3Settings().getSettings(s3_uri.uri.toString()).rw_settings)
+    , request_settings(context_->getStorageS3Settings().getSettings(s3_uri.uri.toString()).request_settings)
     , log(&Poco::Logger::get("BackupWriterS3"))
 {
-    rw_settings.updateFromSettingsIfEmpty(context_->getSettingsRef());
+    request_settings.updateFromSettingsIfEmpty(context_->getSettingsRef());
+    request_settings.max_single_read_retries = context_->getSettingsRef().s3_max_single_read_retries; // FIXME: Avoid taking value for endpoint
 }
 
 DataSourceDescription BackupWriterS3::getDataSourceDescription() const
@@ -216,7 +217,7 @@ void BackupWriterS3::copyObjectMultipartImpl(
     std::vector<String> part_tags;
 
     size_t position = 0;
-    size_t upload_part_size = rw_settings.min_upload_part_size;
+    size_t upload_part_size = request_settings.min_upload_part_size;
 
     for (size_t part_number = 1; position < size; ++part_number)
     {
@@ -248,10 +249,10 @@ void BackupWriterS3::copyObjectMultipartImpl(
 
         position = next_position;
 
-        if (part_number % rw_settings.upload_part_size_multiply_parts_count_threshold == 0)
+        if (part_number % request_settings.upload_part_size_multiply_parts_count_threshold == 0)
         {
-            upload_part_size *= rw_settings.upload_part_size_multiply_factor;
-            upload_part_size = std::min(upload_part_size, rw_settings.max_upload_part_size);
+            upload_part_size *= request_settings.upload_part_size_multiply_factor;
+            upload_part_size = std::min(upload_part_size, request_settings.max_upload_part_size);
         }
     }
 
@@ -294,7 +295,7 @@ void BackupWriterS3::copyFileNative(DiskPtr from_disk, const String & file_name_
         auto file_path = fs::path(s3_uri.key) / file_name_to;
 
         auto head = requestObjectHeadData(source_bucket, objects[0].absolute_path).GetResult();
-        if (static_cast<size_t>(head.GetContentLength()) < rw_settings.max_single_operation_copy_size)
+        if (static_cast<size_t>(head.GetContentLength()) < request_settings.max_single_operation_copy_size)
         {
             copyObjectImpl(
                 source_bucket, objects[0].absolute_path, s3_uri.bucket, file_path, head);
@@ -331,7 +332,7 @@ bool BackupWriterS3::fileContentsEqual(const String & file_name, const String &
     try
     {
         auto in = std::make_unique<ReadBufferFromS3>(
-            client, s3_uri.bucket, fs::path(s3_uri.key) / file_name, s3_uri.version_id, max_single_read_retries, read_settings);
+            client, s3_uri.bucket, fs::path(s3_uri.key) / file_name, s3_uri.version_id, request_settings, read_settings);
         String actual_file_contents(expected_file_contents.size(), ' ');
         return (in->read(actual_file_contents.data(), actual_file_contents.size()) == actual_file_contents.size())
             && (actual_file_contents == expected_file_contents) && in->eof();
@@ -349,7 +350,7 @@ std::unique_ptr<WriteBuffer> BackupWriterS3::writeFile(const String & file_name)
         client,
         s3_uri.bucket,
         fs::path(s3_uri.key) / file_name,
-        rw_settings,
+        request_settings,
         std::nullopt,
         DBMS_DEFAULT_BUFFER_SIZE,
         threadPoolCallbackRunner<void>(IOThreadPool::get(), "BackupWriterS3"));

src/Backups/BackupIO_S3.h

@@ -39,8 +39,8 @@ public:
 private:
     S3::URI s3_uri;
     std::shared_ptr<Aws::S3::S3Client> client;
-    UInt64 max_single_read_retries;
     ReadSettings read_settings;
+    S3Settings::RequestSettings request_settings;
 };
 
 
@@ -81,9 +81,8 @@ private:
 
     S3::URI s3_uri;
     std::shared_ptr<Aws::S3::S3Client> client;
-    UInt64 max_single_read_retries;
     ReadSettings read_settings;
-    S3Settings::ReadWriteSettings rw_settings;
+    S3Settings::RequestSettings request_settings;
     Poco::Logger * log;
 };
 

src/Client/ClientBase.cpp

@@ -1628,6 +1628,14 @@ void ClientBase::processParsedSingleQuery(const String & full_query, const Strin
                     global_context->applySettingChange(change);
             }
             global_context->resetSettingsToDefaultValue(set_query->default_settings);
+
+            /// Query parameters inside SET queries should be also saved on the client side
+            ///  to override their previous definitions set with --param_* arguments
+            ///  and for substitutions to work inside INSERT ... VALUES queries
+            for (const auto & [name, value] : set_query->query_parameters)
+                query_parameters.insert_or_assign(name, value);
+
+            global_context->addQueryParameters(set_query->query_parameters);
         }
         if (const auto * use_query = parsed_query->as<ASTUseQuery>())
         {

src/Common/Exception.h

@@ -122,11 +122,7 @@ public:
     }
 
 
-    std::string displayText() const
-#if defined(POCO_CLICKHOUSE_PATCH)
-    override
-#endif
-    ;
+    std::string displayText() const override;
 
     ssize_t getLineNumber() const { return line_number; }
     void setLineNumber(int line_number_) { line_number = line_number_;}

src/Common/ProfileEvents.cpp

@@ -62,7 +62,7 @@
     M(NetworkSendElapsedMicroseconds, "Total time spent waiting for data to send to network or sending data to network. Only ClickHouse-related network interaction is included, not by 3rd party libraries..") \
     M(NetworkReceiveBytes, "Total number of bytes received from network. Only ClickHouse-related network interaction is included, not by 3rd party libraries.") \
     M(NetworkSendBytes, "Total number of bytes send to network. Only ClickHouse-related network interaction is included, not by 3rd party libraries.") \
-    M(ThrottlerSleepMicroseconds, "Total time a query was sleeping to conform the 'max_network_bandwidth' setting.") \
+    M(ThrottlerSleepMicroseconds, "Total time a query was sleeping to conform 'max_network_bandwidth' and other throttling settings.") \
     \
     M(QueryMaskingRulesMatch, "Number of times query masking rules was successfully matched.") \
     \

src/Common/Stopwatch.h

@@ -63,6 +63,8 @@ private:
 using StopwatchUniquePtr = std::unique_ptr<Stopwatch>;
 
 
+/// Allows to obtain the elapsed time concurrently with restarting the stopwatch.
+/// Allows to atomically compare the elapsed time with a threshold and restart the watch if the elapsed time is not less.
 class AtomicStopwatch
 {
 public:

src/Common/Throttler.cpp

@@ -20,8 +20,6 @@ namespace ErrorCodes
 /// Just 10^9.
 static constexpr auto NS = 1000000000UL;
 
-static const size_t default_burst_seconds = 1;
-
 Throttler::Throttler(size_t max_speed_, const std::shared_ptr<Throttler> & parent_)
     : max_speed(max_speed_)
     , max_burst(max_speed_ * default_burst_seconds)

src/Common/Throttler.h

@@ -17,6 +17,8 @@ namespace DB
 class Throttler
 {
 public:
+    static const size_t default_burst_seconds = 1;
+
     Throttler(size_t max_speed_, size_t max_burst_, const std::shared_ptr<Throttler> & parent_ = nullptr)
         : max_speed(max_speed_), max_burst(max_burst_), limit_exceeded_exception_message(""), tokens(max_burst), parent(parent_) {}