diff --git a/docs/en/changelog/index.md b/docs/en/changelog/index.md
deleted file mode 120000
index 79b747aee1b..00000000000
--- a/docs/en/changelog/index.md
+++ /dev/null
@@ -1 +0,0 @@
-../../../CHANGELOG.md
\ No newline at end of file
diff --git a/docs/en/commercial/cloud.md b/docs/en/commercial/cloud.md
index d8ddf2167d8..3f7ead86219 100644
--- a/docs/en/commercial/cloud.md
+++ b/docs/en/commercial/cloud.md
@@ -7,7 +7,7 @@
[Yandex Managed Service for ClickHouse](https://cloud.yandex.com/services/managed-clickhouse?utm_source=referrals&utm_medium=clickhouseofficialsite&utm_campaign=link3) provides the following key features:
-- Fully managed ZooKeeper service for [ClickHouse replication](../operations/table_engines/replication.md)
+- Fully managed ZooKeeper service for [ClickHouse replication](../engines/table_engines/mergetree_family/replication.md)
- Multiple storage type choices
- Replicas in different availability zones
- Encryption and isolation
diff --git a/docs/en/commercial/index.md b/docs/en/commercial/index.md
new file mode 100644
index 00000000000..3824a634726
--- /dev/null
+++ b/docs/en/commercial/index.md
@@ -0,0 +1,5 @@
+---
+toc_folder_title: Commercial
+toc_priority: 70
+---
+
diff --git a/docs/en/data_types/special_data_types/set.md b/docs/en/data_types/special_data_types/set.md
deleted file mode 100644
index 37489c85d74..00000000000
--- a/docs/en/data_types/special_data_types/set.md
+++ /dev/null
@@ -1,5 +0,0 @@
-# Set {#set}
-
-Used for the right half of an [IN](../../query_language/select.md#select-in-operators) expression.
-
-[Original article](https://clickhouse.tech/docs/en/data_types/special_data_types/set/)
diff --git a/docs/en/development/architecture.md b/docs/en/development/architecture.md
index 5bfd228cd50..1932e3e31d8 100644
--- a/docs/en/development/architecture.md
+++ b/docs/en/development/architecture.md
@@ -1,3 +1,8 @@
+---
+toc_priority: 62
+toc_title: Overview of ClickHouse Architecture
+---
+
# Overview of ClickHouse Architecture {#overview-of-clickhouse-architecture}
ClickHouse is a true column-oriented DBMS. Data is stored by columns and during the execution of arrays (vectors or chunks of columns). Whenever possible, operations are dispatched on arrays, rather than on individual values. It is called “vectorized query execution,” and it helps lower the cost of actual data processing.
@@ -115,7 +120,7 @@ There are ordinary functions and aggregate functions. For aggregate functions, s
Ordinary functions don’t change the number of rows – they work as if they are processing each row independently. In fact, functions are not called for individual rows, but for `Block`’s of data to implement vectorized query execution.
-There are some miscellaneous functions, like [blockSize](../query_language/functions/other_functions.md#function-blocksize), [rowNumberInBlock](../query_language/functions/other_functions.md#function-rownumberinblock), and [runningAccumulate](../query_language/functions/other_functions.md#function-runningaccumulate), that exploit block processing and violate the independence of rows.
+There are some miscellaneous functions, like [blockSize](../sql_reference/functions/other_functions.md#function-blocksize), [rowNumberInBlock](../sql_reference/functions/other_functions.md#function-rownumberinblock), and [runningAccumulate](../sql_reference/functions/other_functions.md#function-runningaccumulate), that exploit block processing and violate the independence of rows.
ClickHouse has strong typing, so there’s no implicit type conversion. If a function doesn’t support a specific combination of types, it throws an exception. But functions can work (be overloaded) for many different combinations of types. For example, the `plus` function (to implement the `+` operator) works for any combination of numeric types: `UInt8` + `Float32`, `UInt16` + `Int8`, and so on. Also, some variadic functions can accept any number of arguments, such as the `concat` function.
diff --git a/docs/en/development/browse_code.md b/docs/en/development/browse_code.md
index d6994c293ac..034b3396aad 100644
--- a/docs/en/development/browse_code.md
+++ b/docs/en/development/browse_code.md
@@ -1,3 +1,8 @@
+---
+toc_priority: 63
+toc_title: Browse ClickHouse Source Code
+---
+
# Browse ClickHouse Source Code {#browse-clickhouse-source-code}
You can use **Woboq** online code browser available [here](https://clickhouse-test-reports.s3.yandex.net/codebrowser/html_report///ClickHouse/dbms/index.html). It provides code navigation and semantic highlighting, search and indexing. The code snapshot is updated daily.
diff --git a/docs/en/development/build.md b/docs/en/development/build.md
index 65558060925..b9b22c737ac 100644
--- a/docs/en/development/build.md
+++ b/docs/en/development/build.md
@@ -1,3 +1,8 @@
+---
+toc_priority: 64
+toc_title: How to Build ClickHouse on Linux
+---
+
# How to Build ClickHouse for Development {#how-to-build-clickhouse-for-development}
The following tutorial is based on the Ubuntu Linux system.
diff --git a/docs/en/development/build_cross_arm.md b/docs/en/development/build_cross_arm.md
index cf6e73308c4..24a7c8363f6 100644
--- a/docs/en/development/build_cross_arm.md
+++ b/docs/en/development/build_cross_arm.md
@@ -1,3 +1,8 @@
+---
+toc_priority: 67
+toc_title: How to Build ClickHouse on Linux for AARCH64 (ARM64)
+---
+
# How to Build ClickHouse on Linux for AARCH64 (ARM64) architecture {#how-to-build-clickhouse-on-linux-for-aarch64-arm64-architecture}
This is for the case when you have Linux machine and want to use it to build `clickhouse` binary that will run on another Linux machine with AARCH64 CPU architecture. This is intended for continuous integration checks that run on Linux servers.
diff --git a/docs/en/development/build_cross_osx.md b/docs/en/development/build_cross_osx.md
index f79fddc64d1..bf95bdb42db 100644
--- a/docs/en/development/build_cross_osx.md
+++ b/docs/en/development/build_cross_osx.md
@@ -1,3 +1,8 @@
+---
+toc_priority: 66
+toc_title: How to Build ClickHouse on Linux for Mac OS X
+---
+
# How to Build ClickHouse on Linux for Mac OS X {#how-to-build-clickhouse-on-linux-for-mac-os-x}
This is for the case when you have Linux machine and want to use it to build `clickhouse` binary that will run on OS X. This is intended for continuous integration checks that run on Linux servers. If you want to build ClickHouse directly on Mac OS X, then proceed with [another instruction](build_osx.md).
diff --git a/docs/en/development/build_osx.md b/docs/en/development/build_osx.md
index e4a26f9e6b4..281f961fc2d 100644
--- a/docs/en/development/build_osx.md
+++ b/docs/en/development/build_osx.md
@@ -1,3 +1,8 @@
+---
+toc_priority: 65
+toc_title: How to Build ClickHouse on Mac OS X
+---
+
# How to Build ClickHouse on Mac OS X {#how-to-build-clickhouse-on-mac-os-x}
Build should work on Mac OS X 10.15 (Catalina)
diff --git a/docs/en/development/contrib.md b/docs/en/development/contrib.md
index 1dc203ca89c..9c594d3e03f 100644
--- a/docs/en/development/contrib.md
+++ b/docs/en/development/contrib.md
@@ -1,7 +1,12 @@
+---
+toc_priority: 70
+toc_title: Third-Party Libraries Used
+---
+
# Third-Party Libraries Used {#third-party-libraries-used}
| Library | License |
-|---------------------|----------------------------------------------------------------------------------------------------------------------------------------------|
+|-------------|--------------------------------------------------------------------------------------|
| base64 | [BSD 2-Clause License](https://github.com/aklomp/base64/blob/a27c565d1b6c676beaf297fe503c4518185666f7/LICENSE) |
| boost | [Boost Software License 1.0](https://github.com/ClickHouse-Extras/boost-extra/blob/6883b40449f378019aec792f9983ce3afc7ff16e/LICENSE_1_0.txt) |
| brotli | [MIT](https://github.com/google/brotli/blob/master/LICENSE) |
diff --git a/docs/en/development/developer_instruction.md b/docs/en/development/developer_instruction.md
index 66294f1fbd4..5cac7203d87 100644
--- a/docs/en/development/developer_instruction.md
+++ b/docs/en/development/developer_instruction.md
@@ -1,3 +1,8 @@
+---
+toc_priority: 61
+toc_title: The Beginner ClickHouse Developer Instruction
+---
+
Building of ClickHouse is supported on Linux, FreeBSD and Mac OS X.
# If you use Windows {#if-you-use-windows}
diff --git a/docs/en/development/index.md b/docs/en/development/index.md
index a18972cdaa2..bb4158554d3 100644
--- a/docs/en/development/index.md
+++ b/docs/en/development/index.md
@@ -1,3 +1,10 @@
+---
+toc_folder_title: Development
+toc_hidden: true
+toc_priority: 58
+toc_title: hidden
+---
+
# ClickHouse Development {#clickhouse-development}
[Original article](https://clickhouse.tech/docs/en/development/)
diff --git a/docs/en/development/style.md b/docs/en/development/style.md
index 645fe5210c3..33188f4c403 100644
--- a/docs/en/development/style.md
+++ b/docs/en/development/style.md
@@ -1,3 +1,8 @@
+---
+toc_priority: 68
+toc_title: How to Write C++ Code
+---
+
# How to Write C++ Code {#how-to-write-c-code}
## General Recommendations {#general-recommendations}
diff --git a/docs/en/development/tests.md b/docs/en/development/tests.md
index ac0051fed70..6ce89277ea0 100644
--- a/docs/en/development/tests.md
+++ b/docs/en/development/tests.md
@@ -1,3 +1,8 @@
+---
+toc_priority: 69
+toc_title: How to Run ClickHouse Tests
+---
+
# ClickHouse Testing {#clickhouse-testing}
## Functional Tests {#functional-tests}
diff --git a/docs/en/database_engines/index.md b/docs/en/engines/database_engines/index.md
similarity index 60%
rename from docs/en/database_engines/index.md
rename to docs/en/engines/database_engines/index.md
index d9c7060fe93..ef8202f2249 100644
--- a/docs/en/database_engines/index.md
+++ b/docs/en/engines/database_engines/index.md
@@ -1,8 +1,14 @@
+---
+toc_folder_title: Database Engines
+toc_priority: 27
+toc_title: Introduction
+---
+
# Database Engines {#database-engines}
Database engines allow you to work with tables.
-By default, ClickHouse uses its native database engine, which provides configurable [table engines](../operations/table_engines/index.md) and an [SQL dialect](../query_language/syntax.md).
+By default, ClickHouse uses its native database engine, which provides configurable [table engines](../../engines/table_engines/index.md) and an [SQL dialect](../../sql_reference/syntax.md).
You can also use the following database engines:
diff --git a/docs/en/database_engines/lazy.md b/docs/en/engines/database_engines/lazy.md
similarity index 91%
rename from docs/en/database_engines/lazy.md
rename to docs/en/engines/database_engines/lazy.md
index e9e2bd1608e..c59abec0ba5 100644
--- a/docs/en/database_engines/lazy.md
+++ b/docs/en/engines/database_engines/lazy.md
@@ -1,3 +1,8 @@
+---
+toc_priority: 31
+toc_title: Lazy
+---
+
# Lazy {#lazy}
Keeps tables in RAM only `expiration_time_in_seconds` seconds after last access. Can be used only with \*Log tables.
diff --git a/docs/en/database_engines/mysql.md b/docs/en/engines/database_engines/mysql.md
similarity index 57%
rename from docs/en/database_engines/mysql.md
rename to docs/en/engines/database_engines/mysql.md
index 63ac8e49081..678c174e1fb 100644
--- a/docs/en/database_engines/mysql.md
+++ b/docs/en/engines/database_engines/mysql.md
@@ -1,4 +1,9 @@
-# MySQL {#mysql}
+---
+toc_priority: 30
+toc_title: MySQL
+---
+
+# Mysql {#mysql}
Allows to connect to databases on a remote MySQL server and perform `INSERT` and `SELECT` queries to exchange data between ClickHouse and MySQL.
@@ -26,27 +31,27 @@ ENGINE = MySQL('host:port', 'database', 'user', 'password')
## Data Types Support {#data_types-support}
-| MySQL | ClickHouse |
-|----------------------------------|---------------------------------------------|
-| UNSIGNED TINYINT | [UInt8](../data_types/int_uint.md) |
-| TINYINT | [Int8](../data_types/int_uint.md) |
-| UNSIGNED SMALLINT | [UInt16](../data_types/int_uint.md) |
-| SMALLINT | [Int16](../data_types/int_uint.md) |
-| UNSIGNED INT, UNSIGNED MEDIUMINT | [UInt32](../data_types/int_uint.md) |
-| INT, MEDIUMINT | [Int32](../data_types/int_uint.md) |
-| UNSIGNED BIGINT | [UInt64](../data_types/int_uint.md) |
-| BIGINT | [Int64](../data_types/int_uint.md) |
-| FLOAT | [Float32](../data_types/float.md) |
-| DOUBLE | [Float64](../data_types/float.md) |
-| DATE | [Date](../data_types/date.md) |
-| DATETIME, TIMESTAMP | [DateTime](../data_types/datetime.md) |
-| BINARY | [FixedString](../data_types/fixedstring.md) |
+| MySQL | ClickHouse |
+|----------------------|--------------------------------------|
+| UNSIGNED TINYINT | [UInt8](../../sql_reference/data_types/int_uint.md) |
+| TINYINT | [Int8](../../sql_reference/data_types/int_uint.md) |
+| UNSIGNED SMALLINT | [UInt16](../../sql_reference/data_types/int_uint.md) |
+| SMALLINT | [Int16](../../sql_reference/data_types/int_uint.md) |
+| UNSIGNED INT, UNSIGNED MEDIUMINT | [UInt32](../../sql_reference/data_types/int_uint.md) |
+| INT, MEDIUMINT | [Int32](../../sql_reference/data_types/int_uint.md) |
+| UNSIGNED BIGINT | [UInt64](../../sql_reference/data_types/int_uint.md) |
+| BIGINT | [Int64](../../sql_reference/data_types/int_uint.md) |
+| FLOAT | [Float32](../../sql_reference/data_types/float.md) |
+| DOUBLE | [Float64](../../sql_reference/data_types/float.md) |
+| DATE | [Date](../../sql_reference/data_types/date.md) |
+| DATETIME, TIMESTAMP | [DateTime](../../sql_reference/data_types/datetime.md) |
+| BINARY | [FixedString](../../sql_reference/data_types/fixedstring.md) |
-All other MySQL data types are converted into [String](../data_types/string.md).
+All other MySQL data types are converted into [String](../../sql_reference/data_types/string.md).
-[Nullable](../data_types/nullable.md) is supported.
+[Nullable](../../sql_reference/data_types/nullable.md) is supported.
-## Examples of Use {#examples-of-use}
+## Examples Of Use {#examples-of-use}
Table in MySQL:
@@ -64,11 +69,11 @@ mysql> insert into mysql_table (`int_id`, `float`) VALUES (1,2);
Query OK, 1 row affected (0,00 sec)
mysql> select * from mysql_table;
-+--------+-------+
++------+-----+
| int_id | value |
-+--------+-------+
++------+-----+
| 1 | 2 |
-+--------+-------+
++------+-----+
1 row in set (0,00 sec)
```
diff --git a/docs/en/engines/index.md b/docs/en/engines/index.md
new file mode 100644
index 00000000000..37b3f5998c8
--- /dev/null
+++ b/docs/en/engines/index.md
@@ -0,0 +1,5 @@
+---
+toc_folder_title: Engines
+toc_priority: 25
+---
+
diff --git a/docs/en/engines/table_engines/index.md b/docs/en/engines/table_engines/index.md
new file mode 100644
index 00000000000..f5ef56382c5
--- /dev/null
+++ b/docs/en/engines/table_engines/index.md
@@ -0,0 +1,83 @@
+---
+toc_folder_title: Table Engines
+toc_priority: 26
+toc_title: Introduction
+---
+
+# Table Engines {#table_engines}
+
+The table engine (type of table) determines:
+
+- How and where data is stored, where to write it to, and where to read it from.
+- Which queries are supported, and how.
+- Concurrent data access.
+- Use of indexes, if present.
+- Whether multithreaded request execution is possible.
+- Data replication parameters.
+
+## Engine Families {#engine-families}
+
+### Mergetree {#mergetree}
+
+The most universal and functional table engines for high-load tasks. The property shared by these engines is quick data insertion with subsequent background data processing. `MergeTree` family engines support data replication (with [Replicated\*](mergetree_family/replication.md) versions of engines), partitioning, and other features not supported in other engines.
+
+Engines in the family:
+
+- [MergeTree](mergetree_family/mergetree.md)
+- [ReplacingMergeTree](mergetree_family/replacingmergetree.md)
+- [SummingMergeTree](mergetree_family/summingmergetree.md)
+- [AggregatingMergeTree](mergetree_family/aggregatingmergetree.md)
+- [CollapsingMergeTree](mergetree_family/collapsingmergetree.md)
+- [VersionedCollapsingMergeTree](mergetree_family/versionedcollapsingmergetree.md)
+- [GraphiteMergeTree](mergetree_family/graphitemergetree.md)
+
+### Log {#log}
+
+Lightweight [engines](log_family/index.md) with minimum functionality. They’re the most effective when you need to quickly write many small tables (up to approximately 1 million rows) and read them later as a whole.
+
+Engines in the family:
+
+- [TinyLog](log_family/tinylog.md)
+- [StripeLog](log_family/stripelog.md)
+- [Log](log_family/log.md)
+
+### Integration Engines {#integration-engines}
+
+Engines for communicating with other data storage and processing systems.
+
+Engines in the family:
+
+- [Kafka](integrations/kafka.md)
+- [MySQL](integrations/mysql.md)
+- [ODBC](integrations/odbc.md)
+- [JDBC](integrations/jdbc.md)
+- [HDFS](integrations/hdfs.md)
+
+### Special Engines {#special-engines}
+
+Engines in the family:
+
+- [Distributed](special/distributed.md)
+- [MaterializedView](special/materializedview.md)
+- [Dictionary](special/dictionary.md)
+- [Merge](special/merge.md)
+- [File](special/file.md)
+- [Null](special/null.md)
+- [Set](special/set.md)
+- [Join](special/join.md)
+- [URL](special/url.md)
+- [View](special/view.md)
+- [Memory](special/memory.md)
+- [Buffer](special/buffer.md)
+
+## Virtual Columns {#table_engines-virtual-columns}
+
+Virtual column is an integral table engine attribute that is defined in the engine source code.
+
+You shouldn’t specify virtual columns in the `CREATE TABLE` query and you can’t see them in `SHOW CREATE TABLE` and `DESCRIBE TABLE` query results. Virtual columns are also read-only, so you can’t insert data into virtual columns.
+
+To select data from a virtual column, you must specify its name in the `SELECT` query. `SELECT *` doesn’t return values from virtual columns.
+
+If you create a table with a column that has the same name as one of the table virtual columns, the virtual column becomes inaccessible. We don’t recommend doing this. To help avoid conflicts, virtual column names are usually prefixed with an underscore.
+
+[Original article](https://clickhouse.tech/docs/en/operations/table_engines/)
diff --git a/docs/en/operations/table_engines/hdfs.md b/docs/en/engines/table_engines/integrations/hdfs.md
similarity index 90%
rename from docs/en/operations/table_engines/hdfs.md
rename to docs/en/engines/table_engines/integrations/hdfs.md
index 9038c00a252..b24313e682a 100644
--- a/docs/en/operations/table_engines/hdfs.md
+++ b/docs/en/engines/table_engines/integrations/hdfs.md
@@ -1,7 +1,12 @@
+---
+toc_priority: 36
+toc_title: HDFS
+---
+
# HDFS {#table_engines-hdfs}
This engine provides integration with [Apache Hadoop](https://en.wikipedia.org/wiki/Apache_Hadoop) ecosystem by allowing to manage data on [HDFS](https://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-hdfs/HdfsDesign.html)via ClickHouse. This engine is similar
-to the [File](file.md) and [URL](url.md) engines, but provides Hadoop-specific features.
+to the [File](../special/file.md) and [URL](../special/url.md) engines, but provides Hadoop-specific features.
## Usage {#usage}
@@ -13,7 +18,7 @@ The `URI` parameter is the whole file URI in HDFS.
The `format` parameter specifies one of the available file formats. To perform
`SELECT` queries, the format must be supported for input, and to perform
`INSERT` queries – for output. The available formats are listed in the
-[Formats](../../interfaces/formats.md#formats) section.
+[Formats](../../../interfaces/formats.md#formats) section.
The path part of `URI` may contain globs. In this case the table would be readonly.
**Example:**
@@ -60,7 +65,7 @@ Multiple path components can have globs. For being processed file should exists
- `{some_string,another_string,yet_another_one}` — Substitutes any of strings `'some_string', 'another_string', 'yet_another_one'`.
- `{N..M}` — Substitutes any number in range from N to M including both borders.
-Constructions with `{}` are similar to the [remote](../../query_language/table_functions/remote.md) table function.
+Constructions with `{}` are similar to the [remote](../../../sql_reference/table_functions/remote.md) table function.
**Example**
@@ -111,6 +116,6 @@ CREARE TABLE big_table (name String, value UInt32) ENGINE = HDFS('hdfs://hdfs1:9
**See Also**
-- [Virtual columns](https://clickhouse.tech/docs/en/operations/table_engines/#table_engines-virtual_columns)
+- [Virtual columns](../index.md#table_engines-virtual_columns)
[Original article](https://clickhouse.tech/docs/en/operations/table_engines/hdfs/)
diff --git a/docs/en/engines/table_engines/integrations/index.md b/docs/en/engines/table_engines/integrations/index.md
new file mode 100644
index 00000000000..716d00cdd98
--- /dev/null
+++ b/docs/en/engines/table_engines/integrations/index.md
@@ -0,0 +1,5 @@
+---
+toc_folder_title: Integrations
+toc_priority: 30
+---
+
diff --git a/docs/en/operations/table_engines/jdbc.md b/docs/en/engines/table_engines/integrations/jdbc.md
similarity index 87%
rename from docs/en/operations/table_engines/jdbc.md
rename to docs/en/engines/table_engines/integrations/jdbc.md
index 3deed862850..e3ba45ac06e 100644
--- a/docs/en/operations/table_engines/jdbc.md
+++ b/docs/en/engines/table_engines/integrations/jdbc.md
@@ -1,10 +1,15 @@
+---
+toc_priority: 34
+toc_title: JDBC
+---
+
# JDBC {#table-engine-jdbc}
Allows ClickHouse to connect to external databases via [JDBC](https://en.wikipedia.org/wiki/Java_Database_Connectivity).
To implement the JDBC connection, ClickHouse uses the separate program [clickhouse-jdbc-bridge](https://github.com/alex-krash/clickhouse-jdbc-bridge) that should run as a daemon.
-This engine supports the [Nullable](../../data_types/nullable.md) data type.
+This engine supports the [Nullable](../../../sql_reference/data_types/nullable.md) data type.
## Creating a Table {#creating-a-table}
@@ -44,11 +49,11 @@ mysql> insert into test (`int_id`, `float`) VALUES (1,2);
Query OK, 1 row affected (0,00 sec)
mysql> select * from test;
-+--------+--------------+-------+----------------+
++------+----------+-----+----------+
| int_id | int_nullable | float | float_nullable |
-+--------+--------------+-------+----------------+
++------+----------+-----+----------+
| 1 | NULL | 2 | NULL |
-+--------+--------------+-------+----------------+
++------+----------+-----+----------+
1 row in set (0,00 sec)
```
@@ -78,6 +83,6 @@ FROM jdbc_table
## See Also {#see-also}
-- [JDBC table function](../../query_language/table_functions/jdbc.md).
+- [JDBC table function](../../../sql_reference/table_functions/jdbc.md).
[Original article](https://clickhouse.tech/docs/en/operations/table_engines/jdbc/)
diff --git a/docs/en/operations/table_engines/kafka.md b/docs/en/engines/table_engines/integrations/kafka.md
similarity index 93%
rename from docs/en/operations/table_engines/kafka.md
rename to docs/en/engines/table_engines/integrations/kafka.md
index 2c844521e8f..0166c989800 100644
--- a/docs/en/operations/table_engines/kafka.md
+++ b/docs/en/engines/table_engines/integrations/kafka.md
@@ -1,3 +1,8 @@
+---
+toc_priority: 32
+toc_title: Kafka
+---
+
# Kafka {#kafka}
This engine works with [Apache Kafka](http://kafka.apache.org/).
@@ -33,7 +38,7 @@ Required parameters:
- `kafka_broker_list` – A comma-separated list of brokers (for example, `localhost:9092`).
- `kafka_topic_list` – A list of Kafka topics.
- `kafka_group_name` – A group of Kafka consumers. Reading margins are tracked for each group separately. If you don’t want messages to be duplicated in the cluster, use the same group name everywhere.
-- `kafka_format` – Message format. Uses the same notation as the SQL `FORMAT` function, such as `JSONEachRow`. For more information, see the [Formats](../../interfaces/formats.md) section.
+- `kafka_format` – Message format. Uses the same notation as the SQL `FORMAT` function, such as `JSONEachRow`. For more information, see the [Formats](../../../interfaces/formats.md) section.
Optional parameters:
@@ -123,7 +128,7 @@ Example:
SELECT level, sum(total) FROM daily GROUP BY level;
```
-To improve performance, received messages are grouped into blocks the size of [max\_insert\_block\_size](../settings/settings.md#settings-max_insert_block_size). If the block wasn’t formed within [stream\_flush\_interval\_ms](../settings/settings.md) milliseconds, the data will be flushed to the table regardless of the completeness of the block.
+To improve performance, received messages are grouped into blocks the size of [max\_insert\_block\_size](../../../operations/server_configuration_parameters/settings.md#settings-max_insert_block_size). If the block wasn’t formed within [stream\_flush\_interval\_ms](../../../operations/server_configuration_parameters/settings.md) milliseconds, the data will be flushed to the table regardless of the completeness of the block.
To stop receiving topic data or to change the conversion logic, detach the materialized view:
@@ -164,6 +169,6 @@ For a list of possible configuration options, see the [librdkafka configuration
**See Also**
-- [Virtual columns](index.md#table_engines-virtual_columns)
+- [Virtual columns](../index.md#table_engines-virtual_columns)
[Original article](https://clickhouse.tech/docs/en/operations/table_engines/kafka/)
diff --git a/docs/en/operations/table_engines/mysql.md b/docs/en/engines/table_engines/integrations/mysql.md
similarity index 81%
rename from docs/en/operations/table_engines/mysql.md
rename to docs/en/engines/table_engines/integrations/mysql.md
index 741930d48d8..49410887242 100644
--- a/docs/en/operations/table_engines/mysql.md
+++ b/docs/en/engines/table_engines/integrations/mysql.md
@@ -1,4 +1,9 @@
-# MySQL {#mysql}
+---
+toc_priority: 33
+toc_title: MySQL
+---
+
+# Mysql {#mysql}
The MySQL engine allows you to perform `SELECT` queries on data that is stored on a remote MySQL server.
@@ -13,12 +18,12 @@ CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster]
) ENGINE = MySQL('host:port', 'database', 'table', 'user', 'password'[, replace_query, 'on_duplicate_clause']);
```
-See a detailed description of the [CREATE TABLE](../../query_language/create.md#create-table-query) query.
+See a detailed description of the [CREATE TABLE](../../../sql_reference/statements/create.md#create-table-query) query.
The table structure can differ from the original MySQL table structure:
- Column names should be the same as in the original MySQL table, but you can use just some of these columns and in any order.
-- Column types may differ from those in the original MySQL table. ClickHouse tries to [cast](../../query_language/functions/type_conversion_functions.md#type_conversion_function-cast) values to the ClickHouse data types.
+- Column types may differ from those in the original MySQL table. ClickHouse tries to [cast](../../../sql_reference/functions/type_conversion_functions.md#type_conversion_function-cast) values to the ClickHouse data types.
**Engine Parameters**
@@ -61,11 +66,11 @@ mysql> insert into test (`int_id`, `float`) VALUES (1,2);
Query OK, 1 row affected (0,00 sec)
mysql> select * from test;
-+--------+--------------+-------+----------------+
++------+----------+-----+----------+
| int_id | int_nullable | float | float_nullable |
-+--------+--------------+-------+----------------+
++------+----------+-----+----------+
| 1 | NULL | 2 | NULL |
-+--------+--------------+-------+----------------+
++------+----------+-----+----------+
1 row in set (0,00 sec)
```
@@ -92,7 +97,7 @@ SELECT * FROM mysql_table
## See Also {#see-also}
-- [The ‘mysql’ table function](../../query_language/table_functions/mysql.md)
-- [Using MySQL as a source of external dictionary](../../query_language/dicts/external_dicts_dict_sources.md#dicts-external_dicts_dict_sources-mysql)
+- [The ‘mysql’ table function](../../../sql_reference/table_functions/mysql.md)
+- [Using MySQL as a source of external dictionary](../../../sql_reference/dictionaries/external_dictionaries/external_dicts_dict_sources.md#dicts-external_dicts_dict_sources-mysql)
[Original article](https://clickhouse.tech/docs/en/operations/table_engines/mysql/)
diff --git a/docs/en/operations/table_engines/odbc.md b/docs/en/engines/table_engines/integrations/odbc.md
similarity index 81%
rename from docs/en/operations/table_engines/odbc.md
rename to docs/en/engines/table_engines/integrations/odbc.md
index f9ff4028f5a..a537150313f 100644
--- a/docs/en/operations/table_engines/odbc.md
+++ b/docs/en/engines/table_engines/integrations/odbc.md
@@ -1,10 +1,15 @@
+---
+toc_priority: 35
+toc_title: ODBC
+---
+
# ODBC {#table-engine-odbc}
Allows ClickHouse to connect to external databases via [ODBC](https://en.wikipedia.org/wiki/Open_Database_Connectivity).
To safely implement ODBC connections, ClickHouse uses a separate program `clickhouse-odbc-bridge`. If the ODBC driver is loaded directly from `clickhouse-server`, driver problems can crash the ClickHouse server. ClickHouse automatically starts `clickhouse-odbc-bridge` when it is required. The ODBC bridge program is installed from the same package as the `clickhouse-server`.
-This engine supports the [Nullable](../../data_types/nullable.md) data type.
+This engine supports the [Nullable](../../../sql_reference/data_types/nullable.md) data type.
## Creating a Table {#creating-a-table}
@@ -18,12 +23,12 @@ CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster]
ENGINE = ODBC(connection_settings, external_database, external_table)
```
-See a detailed description of the [CREATE TABLE](../../query_language/create.md#create-table-query) query.
+See a detailed description of the [CREATE TABLE](../../../sql_reference/statements/create.md#create-table-query) query.
The table structure can differ from the source table structure:
- Column names should be the same as in the source table, but you can use just some of these columns and in any order.
-- Column types may differ from those in the source table. ClickHouse tries to [cast](../../query_language/functions/type_conversion_functions.md#type_conversion_function-cast) values to the ClickHouse data types.
+- Column types may differ from those in the source table. ClickHouse tries to [cast](../../../sql_reference/functions/type_conversion_functions.md#type_conversion_function-cast) values to the ClickHouse data types.
**Engine Parameters**
@@ -67,7 +72,7 @@ You can check the connection using the `isql` utility from the unixODBC installa
``` bash
$ isql -v mysqlconn
-+---------------------------------------+
++-------------------------+
| Connected! |
| |
...
@@ -88,11 +93,11 @@ mysql> insert into test (`int_id`, `float`) VALUES (1,2);
Query OK, 1 row affected (0,00 sec)
mysql> select * from test;
-+--------+--------------+-------+----------------+
++------+----------+-----+----------+
| int_id | int_nullable | float | float_nullable |
-+--------+--------------+-------+----------------+
++------+----------+-----+----------+
| 1 | NULL | 2 | NULL |
-+--------+--------------+-------+----------------+
++------+----------+-----+----------+
1 row in set (0,00 sec)
```
@@ -119,7 +124,7 @@ SELECT * FROM odbc_t
## See Also {#see-also}
-- [ODBC external dictionaries](../../query_language/dicts/external_dicts_dict_sources.md#dicts-external_dicts_dict_sources-odbc)
-- [ODBC table function](../../query_language/table_functions/odbc.md)
+- [ODBC external dictionaries](../../../sql_reference/dictionaries/external_dictionaries/external_dicts_dict_sources.md#dicts-external_dicts_dict_sources-odbc)
+- [ODBC table function](../../../sql_reference/table_functions/odbc.md)
[Original article](https://clickhouse.tech/docs/en/operations/table_engines/odbc/)
diff --git a/docs/en/engines/table_engines/log_family/index.md b/docs/en/engines/table_engines/log_family/index.md
new file mode 100644
index 00000000000..a64371200a6
--- /dev/null
+++ b/docs/en/engines/table_engines/log_family/index.md
@@ -0,0 +1,5 @@
+---
+toc_folder_title: Log Family
+toc_priority: 29
+---
+
diff --git a/docs/en/operations/table_engines/log.md b/docs/en/engines/table_engines/log_family/log.md
similarity index 95%
rename from docs/en/operations/table_engines/log.md
rename to docs/en/engines/table_engines/log_family/log.md
index 5e10b68f63e..b34fdd4116e 100644
--- a/docs/en/operations/table_engines/log.md
+++ b/docs/en/engines/table_engines/log_family/log.md
@@ -1,3 +1,8 @@
+---
+toc_priority: 33
+toc_title: Log
+---
+
# Log {#log}
Engine belongs to the family of log engines. See the common properties of log engines and their differences in the [Log Engine Family](log_family.md) article.
diff --git a/docs/en/operations/table_engines/log_family.md b/docs/en/engines/table_engines/log_family/log_family.md
similarity index 90%
rename from docs/en/operations/table_engines/log_family.md
rename to docs/en/engines/table_engines/log_family/log_family.md
index 8065c68f705..a238a844002 100644
--- a/docs/en/operations/table_engines/log_family.md
+++ b/docs/en/engines/table_engines/log_family/log_family.md
@@ -1,3 +1,8 @@
+---
+toc_priority: 31
+toc_title: Introduction
+---
+
# Log Engine Family {#log-engine-family}
These engines were developed for scenarios when you need to quickly write many small tables (up to about 1 million rows) and read them later as a whole.
@@ -8,7 +13,7 @@ Engines of the family:
- [Log](log.md)
- [TinyLog](tinylog.md)
-## Common properties {#common-properties}
+## Common Properties {#common-properties}
Engines:
@@ -20,7 +25,7 @@ Engines:
During `INSERT` queries, the table is locked, and other queries for reading and writing data both wait for the table to unlock. If there are no data writing queries, any number of data reading queries can be performed concurrently.
-- Do not support [mutation](../../query_language/alter.md#alter-mutations) operations.
+- Do not support [mutation](../../../sql_reference/statements/alter.md#alter-mutations) operations.
- Do not support indexes.
diff --git a/docs/en/operations/table_engines/stripelog.md b/docs/en/engines/table_engines/log_family/stripelog.md
similarity index 93%
rename from docs/en/operations/table_engines/stripelog.md
rename to docs/en/engines/table_engines/log_family/stripelog.md
index d1cd9b49cee..1aace26ceeb 100644
--- a/docs/en/operations/table_engines/stripelog.md
+++ b/docs/en/engines/table_engines/log_family/stripelog.md
@@ -1,4 +1,9 @@
-# StripeLog {#stripelog}
+---
+toc_priority: 32
+toc_title: StripeLog
+---
+
+# Stripelog {#stripelog}
This engine belongs to the family of log engines. See the common properties of log engines and their differences in the [Log Engine Family](log_family.md) article.
@@ -15,7 +20,7 @@ CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster]
) ENGINE = StripeLog
```
-See the detailed description of the [CREATE TABLE](../../query_language/create.md#create-table-query) query.
+See the detailed description of the [CREATE TABLE](../../../sql_reference/statements/create.md#create-table-query) query.
## Writing the Data {#table_engines-stripelog-writing-the-data}
@@ -32,7 +37,7 @@ The `StripeLog` engine does not support the `ALTER UPDATE` and `ALTER DELETE` op
The file with marks allows ClickHouse to parallelize the reading of data. This means that a `SELECT` query returns rows in an unpredictable order. Use the `ORDER BY` clause to sort rows.
-## Example of Use {#table_engines-stripelog-example-of-use}
+## Example Of Use {#table_engines-stripelog-example-of-use}
Creating a table:
diff --git a/docs/en/operations/table_engines/tinylog.md b/docs/en/engines/table_engines/log_family/tinylog.md
similarity index 95%
rename from docs/en/operations/table_engines/tinylog.md
rename to docs/en/engines/table_engines/log_family/tinylog.md
index ea6bd14d537..7b91becc4d7 100644
--- a/docs/en/operations/table_engines/tinylog.md
+++ b/docs/en/engines/table_engines/log_family/tinylog.md
@@ -1,3 +1,8 @@
+---
+toc_priority: 34
+toc_title: TinyLog
+---
+
# TinyLog {#tinylog}
The engine belongs to the log engine family. See [Log Engine Family](log_family.md) for common properties of log engines and their differences.
diff --git a/docs/en/operations/table_engines/aggregatingmergetree.md b/docs/en/engines/table_engines/mergetree_family/aggregatingmergetree.md
similarity index 89%
rename from docs/en/operations/table_engines/aggregatingmergetree.md
rename to docs/en/engines/table_engines/mergetree_family/aggregatingmergetree.md
index 63cda964a16..2103efe98dc 100644
--- a/docs/en/operations/table_engines/aggregatingmergetree.md
+++ b/docs/en/engines/table_engines/mergetree_family/aggregatingmergetree.md
@@ -1,10 +1,15 @@
-# AggregatingMergeTree {#aggregatingmergetree}
+---
+toc_priority: 35
+toc_title: AggregatingMergeTree
+---
+
+# Aggregatingmergetree {#aggregatingmergetree}
The engine inherits from [MergeTree](mergetree.md#table_engines-mergetree), altering the logic for data parts merging. ClickHouse replaces all rows with the same primary key (or more accurately, with the same [sorting key](mergetree.md)) with a single row (within a one data part) that stores a combination of states of aggregate functions.
You can use `AggregatingMergeTree` tables for incremental data aggregation, including for aggregated materialized views.
-The engine processes all columns with [AggregateFunction](../../data_types/nested_data_structures/aggregatefunction.md) type.
+The engine processes all columns with [AggregateFunction](../../../sql_reference/data_types/aggregatefunction.md) type.
It is appropriate to use `AggregatingMergeTree` if it reduces the number of rows by orders.
@@ -24,7 +29,7 @@ CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster]
[SETTINGS name=value, ...]
```
-For a description of request parameters, see [request description](../../query_language/create.md).
+For a description of request parameters, see [request description](../../../sql_reference/statements/create.md).
**Query clauses**
@@ -51,12 +56,12 @@ All of the parameters have the same meaning as in `MergeTree`.
## SELECT and INSERT {#select-and-insert}
-To insert data, use [INSERT SELECT](../../query_language/insert_into.md) query with aggregate -State- functions.
+To insert data, use [INSERT SELECT](../../../sql_reference/statements/insert_into.md) query with aggregate -State- functions.
When selecting data from `AggregatingMergeTree` table, use `GROUP BY` clause and the same aggregate functions as when inserting data, but using `-Merge` suffix.
In the results of `SELECT` query, the values of `AggregateFunction` type have implementation-specific binary representation for all of the ClickHouse output formats. If dump data into, for example, `TabSeparated` format with `SELECT` query then this dump can be loaded back using `INSERT` query.
-## Example of an Aggregated Materialized View {#example-of-an-aggregated-materialized-view}
+## Example Of an Aggregated Materialized View {#example-of-an-aggregated-materialized-view}
`AggregatingMergeTree` materialized view that watches the `test.visits` table:
diff --git a/docs/en/operations/table_engines/collapsingmergetree.md b/docs/en/engines/table_engines/mergetree_family/collapsingmergetree.md
similarity index 98%
rename from docs/en/operations/table_engines/collapsingmergetree.md
rename to docs/en/engines/table_engines/mergetree_family/collapsingmergetree.md
index 1af014a7e04..61e0d82d32f 100644
--- a/docs/en/operations/table_engines/collapsingmergetree.md
+++ b/docs/en/engines/table_engines/mergetree_family/collapsingmergetree.md
@@ -1,4 +1,9 @@
-# CollapsingMergeTree {#table_engine-collapsingmergetree}
+---
+toc_priority: 36
+toc_title: CollapsingMergeTree
+---
+
+# Collapsingmergetree {#table_engine-collapsingmergetree}
The engine inherits from [MergeTree](mergetree.md) and adds the logic of rows collapsing to data parts merge algorithm.
@@ -21,7 +26,7 @@ CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster]
[SETTINGS name=value, ...]
```
-For a description of query parameters, see [query description](../../query_language/create.md).
+For a description of query parameters, see [query description](../../../sql_reference/statements/create.md).
**CollapsingMergeTree Parameters**
@@ -132,7 +137,7 @@ The aggregates `count`, `sum` and `avg` could be calculated this way. The aggreg
If you need to extract data without aggregation (for example, to check whether rows are present whose newest values match certain conditions), you can use the `FINAL` modifier for the `FROM` clause. This approach is significantly less efficient.
-## Example of use {#example-of-use}
+## Example Of Use {#example-of-use}
Example data:
@@ -222,7 +227,7 @@ SELECT * FROM UAct FINAL
This way of selecting the data is very inefficient. Don’t use it for big tables.
-## Example of another approach {#example-of-another-approach}
+## Example Of Another Approach {#example-of-another-approach}
Example data:
diff --git a/docs/en/operations/table_engines/custom_partitioning_key.md b/docs/en/engines/table_engines/mergetree_family/custom_partitioning_key.md
similarity index 89%
rename from docs/en/operations/table_engines/custom_partitioning_key.md
rename to docs/en/engines/table_engines/mergetree_family/custom_partitioning_key.md
index a22149b6b9b..0d5891955a9 100644
--- a/docs/en/operations/table_engines/custom_partitioning_key.md
+++ b/docs/en/engines/table_engines/mergetree_family/custom_partitioning_key.md
@@ -1,6 +1,11 @@
+---
+toc_priority: 32
+toc_title: Custom Partitioning Key
+---
+
# Custom Partitioning Key {#custom-partitioning-key}
-Partitioning is available for the [MergeTree](mergetree.md) family tables (including [replicated](replication.md) tables). [Materialized views](materializedview.md) based on MergeTree tables support partitioning, as well.
+Partitioning is available for the [MergeTree](mergetree.md) family tables (including [replicated](replication.md) tables). [Materialized views](../special/materializedview.md) based on MergeTree tables support partitioning, as well.
A partition is a logical combination of records in a table by a specified criterion. You can set a partition by an arbitrary criterion, such as by month, by day, or by event type. Each partition is stored separately to simplify manipulations of this data. When accessing the data, ClickHouse uses the smallest subset of partitions possible.
@@ -33,7 +38,7 @@ When inserting new data to a table, this data is stored as a separate part (chun
!!! info "Info"
A merge only works for data parts that have the same value for the partitioning expression. This means **you shouldn’t make overly granular partitions** (more than about a thousand partitions). Otherwise, the `SELECT` query performs poorly because of an unreasonably large number of files in the file system and open file descriptors.
-Use the [system.parts](../system_tables.md#system_tables-parts) table to view the table parts and partitions. For example, let’s assume that we have a `visits` table with partitioning by month. Let’s perform the `SELECT` query for the `system.parts` table:
+Use the [system.parts](../../../operations/system_tables.md#system_tables-parts) table to view the table parts and partitions. For example, let’s assume that we have a `visits` table with partitioning by month. Let’s perform the `SELECT` query for the `system.parts` table:
``` sql
SELECT
@@ -72,7 +77,7 @@ Let’s break down the name of the first part: `201901_1_3_1`:
The `active` column shows the status of the part. `1` is active; `0` is inactive. The inactive parts are, for example, source parts remaining after merging to a larger part. The corrupted data parts are also indicated as inactive.
-As you can see in the example, there are several separated parts of the same partition (for example, `201901_1_3_1` and `201901_1_9_2`). This means that these parts are not merged yet. ClickHouse merges the inserted parts of data periodically, approximately 15 minutes after inserting. In addition, you can perform a non-scheduled merge using the [OPTIMIZE](../../query_language/misc.md#misc_operations-optimize) query. Example:
+As you can see in the example, there are several separated parts of the same partition (for example, `201901_1_3_1` and `201901_1_9_2`). This means that these parts are not merged yet. ClickHouse merges the inserted parts of data periodically, approximately 15 minutes after inserting. In addition, you can perform a non-scheduled merge using the [OPTIMIZE](../../../sql_reference/statements/misc.md#misc_operations-optimize) query. Example:
``` sql
OPTIMIZE TABLE visits PARTITION 201902;
@@ -111,10 +116,10 @@ drwxr-xr-x 2 clickhouse clickhouse 4096 Feb 1 16:48 detached
The folders ‘201901\_1\_1\_0’, ‘201901\_1\_7\_1’ and so on are the directories of the parts. Each part relates to a corresponding partition and contains data just for a certain month (the table in this example has partitioning by month).
-The `detached` directory contains parts that were detached from the table using the [DETACH](#alter_detach-partition) query. The corrupted parts are also moved to this directory, instead of being deleted. The server does not use the parts from the `detached` directory. You can add, delete, or modify the data in this directory at any time – the server will not know about this until you run the [ATTACH](../../query_language/alter.md#alter_attach-partition) query.
+The `detached` directory contains parts that were detached from the table using the [DETACH](#alter_detach-partition) query. The corrupted parts are also moved to this directory, instead of being deleted. The server does not use the parts from the `detached` directory. You can add, delete, or modify the data in this directory at any time – the server will not know about this until you run the [ATTACH](../../../sql_reference/statements/alter.md#alter_attach-partition) query.
Note that on the operating server, you cannot manually change the set of parts or their data on the file system, since the server will not know about it. For non-replicated tables, you can do this when the server is stopped, but it isn’t recommended. For replicated tables, the set of parts cannot be changed in any case.
-ClickHouse allows you to perform operations with the partitions: delete them, copy from one table to another, or create a backup. See the list of all operations in the section [Manipulations With Partitions and Parts](../../query_language/alter.md#alter_manipulations-with-partitions).
+ClickHouse allows you to perform operations with the partitions: delete them, copy from one table to another, or create a backup. See the list of all operations in the section [Manipulations With Partitions and Parts](../../../sql_reference/statements/alter.md#alter_manipulations-with-partitions).
[Original article](https://clickhouse.tech/docs/en/operations/table_engines/custom_partitioning_key/)
diff --git a/docs/en/operations/table_engines/graphitemergetree.md b/docs/en/engines/table_engines/mergetree_family/graphitemergetree.md
similarity index 91%
rename from docs/en/operations/table_engines/graphitemergetree.md
rename to docs/en/engines/table_engines/mergetree_family/graphitemergetree.md
index 13180b9e0bd..7785af9b6d3 100644
--- a/docs/en/operations/table_engines/graphitemergetree.md
+++ b/docs/en/engines/table_engines/mergetree_family/graphitemergetree.md
@@ -1,4 +1,9 @@
-# GraphiteMergeTree {#graphitemergetree}
+---
+toc_priority: 38
+toc_title: GraphiteMergeTree
+---
+
+# Graphitemergetree {#graphitemergetree}
This engine is designed for thinning and aggregating/averaging (rollup) [Graphite](http://graphite.readthedocs.io/en/latest/index.html) data. It may be helpful to developers who want to use ClickHouse as a data store for Graphite.
@@ -23,7 +28,7 @@ CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster]
[SETTINGS name=value, ...]
```
-See a detailed description of the [CREATE TABLE](../../query_language/create.md#create-table-query) query.
+See a detailed description of the [CREATE TABLE](../../../sql_reference/statements/create.md#create-table-query) query.
A table for the Graphite data should have the following columns for the following data:
@@ -72,9 +77,9 @@ All of the parameters excepting `config_section` have the same meaning as in `Me
-## Rollup configuration {#rollup-configuration}
+## Rollup Configuration {#rollup-configuration}
-The settings for rollup are defined by the [graphite\_rollup](../server_settings/settings.md#server_settings-graphite_rollup) parameter in the server configuration. The name of the parameter could be any. You can create several configurations and use them for different tables.
+The settings for rollup are defined by the [graphite\_rollup](../../../operations/server_configuration_parameters/settings.md#server_configuration_parameters-graphite_rollup) parameter in the server configuration. The name of the parameter could be any. You can create several configurations and use them for different tables.
Rollup configuration structure:
diff --git a/docs/en/engines/table_engines/mergetree_family/index.md b/docs/en/engines/table_engines/mergetree_family/index.md
new file mode 100644
index 00000000000..6a23ef23a8e
--- /dev/null
+++ b/docs/en/engines/table_engines/mergetree_family/index.md
@@ -0,0 +1,5 @@
+---
+toc_folder_title: MergeTree Family
+toc_priority: 28
+---
+
diff --git a/docs/en/operations/table_engines/mergetree.md b/docs/en/engines/table_engines/mergetree_family/mergetree.md
similarity index 81%
rename from docs/en/operations/table_engines/mergetree.md
rename to docs/en/engines/table_engines/mergetree_family/mergetree.md
index a057d1689ef..aabc0b45487 100644
--- a/docs/en/operations/table_engines/mergetree.md
+++ b/docs/en/engines/table_engines/mergetree_family/mergetree.md
@@ -1,4 +1,9 @@
-# MergeTree {#table_engines-mergetree}
+---
+toc_priority: 30
+toc_title: MergeTree
+---
+
+# Mergetree {#table_engines-mergetree}
The `MergeTree` engine and other engines of this family (`*MergeTree`) are the most robust ClickHouse table engines.
@@ -23,7 +28,7 @@ Main features:
If necessary, you can set the data sampling method in the table.
!!! info "Info"
- The [Merge](merge.md) engine does not belong to the `*MergeTree` family.
+ The [Merge](../special/merge.md) engine does not belong to the `*MergeTree` family.
## Creating a Table {#table_engine-mergetree-creating-a-table}
@@ -44,7 +49,7 @@ CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster]
[SETTINGS name=value, ...]
```
-For a description of parameters, see the [CREATE query description](../../query_language/create.md).
+For a description of parameters, see the [CREATE query description](../../../sql_reference/statements/create.md).
!!! note "Note"
`INDEX` is an experimental feature, see [Data Skipping Indexes](#table_engine-mergetree-data_skipping-indexes).
@@ -55,7 +60,7 @@ For a description of parameters, see the [CREATE query description](../../query_
- `PARTITION BY` — The [partitioning key](custom_partitioning_key.md).
- For partitioning by month, use the `toYYYYMM(date_column)` expression, where `date_column` is a column with a date of the type [Date](../../data_types/date.md). The partition names here have the `"YYYYMM"` format.
+ For partitioning by month, use the `toYYYYMM(date_column)` expression, where `date_column` is a column with a date of the type [Date](../../../sql_reference/data_types/date.md). The partition names here have the `"YYYYMM"` format.
- `ORDER BY` — The sorting key.
@@ -83,7 +88,7 @@ For a description of parameters, see the [CREATE query description](../../query_
- `index_granularity` — Maximum number of data rows between the marks of an index. Default value: 8192. See [Data Storage](#mergetree-data-storage).
- `index_granularity_bytes` — Maximum size of data granules in bytes. Default value: 10Mb. To restrict the granule size only by number of rows, set to 0 (not recommended). See [Data Storage](#mergetree-data-storage).
- `enable_mixed_granularity_parts` — Enables or disables transitioning to control the granule size with the `index_granularity_bytes` setting. Before version 19.11, there was only the `index_granularity` setting for restricting granule size. The `index_granularity_bytes` setting improves ClickHouse performance when selecting data from tables with big rows (tens and hundreds of megabytes). If you have tables with big rows, you can enable this setting for the tables to improve the efficiency of `SELECT` queries.
- - `use_minimalistic_part_header_in_zookeeper` — Storage method of the data parts headers in ZooKeeper. If `use_minimalistic_part_header_in_zookeeper=1`, then ZooKeeper stores less data. For more information, see the [setting description](../server_settings/settings.md#server-settings-use_minimalistic_part_header_in_zookeeper) in “Server configuration parameters”.
+ - `use_minimalistic_part_header_in_zookeeper` — Storage method of the data parts headers in ZooKeeper. If `use_minimalistic_part_header_in_zookeeper=1`, then ZooKeeper stores less data. For more information, see the [setting description](../../../operations/server_configuration_parameters/settings.md#server-settings-use_minimalistic_part_header_in_zookeeper) in “Server configuration parameters”.
- `min_merge_bytes_to_use_direct_io` — The minimum data volume for merge operation that is required for using direct I/O access to the storage disk. When merging data parts, ClickHouse calculates the total storage volume of all the data to be merged. If the volume exceeds `min_merge_bytes_to_use_direct_io` bytes, ClickHouse reads and writes the data to the storage disk using the direct I/O interface (`O_DIRECT` option). If `min_merge_bytes_to_use_direct_io = 0`, then direct I/O is disabled. Default value: `10 * 1024 * 1024 * 1024` bytes.
- `merge_with_ttl_timeout` — Minimum delay in seconds before repeating a merge with TTL. Default value: 86400 (1 day).
@@ -99,7 +104,7 @@ ENGINE MergeTree() PARTITION BY toYYYYMM(EventDate) ORDER BY (CounterID, EventDa
In the example, we set partitioning by month.
-We also set an expression for sampling as a hash by the user ID. This allows you to pseudorandomize the data in the table for each `CounterID` and `EventDate`. If you define a [SAMPLE](../../query_language/select.md#select-sample-clause) clause when selecting the data, ClickHouse will return an evenly pseudorandom data sample for a subset of users.
+We also set an expression for sampling as a hash by the user ID. This allows you to pseudorandomize the data in the table for each `CounterID` and `EventDate`. If you define a [SAMPLE](../../../sql_reference/statements/select.md#select-sample-clause) clause when selecting the data, ClickHouse will return an evenly pseudorandom data sample for a subset of users.
The `index_granularity` setting can be omitted because 8192 is the default value.
@@ -121,9 +126,9 @@ CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster]
**MergeTree() Parameters**
-- `date-column` — The name of a column of the [Date](../../data_types/date.md) type. ClickHouse automatically creates partitions by month based on this column. The partition names are in the `"YYYYMM"` format.
+- `date-column` — The name of a column of the [Date](../../../sql_reference/data_types/date.md) type. ClickHouse automatically creates partitions by month based on this column. The partition names are in the `"YYYYMM"` format.
- `sampling_expression` — An expression for sampling.
-- `(primary, key)` — Primary key. Type: [Tuple()](../../data_types/tuple.md)
+- `(primary, key)` — Primary key. Type: [Tuple()](../../../sql_reference/data_types/tuple.md)
- `index_granularity` — The granularity of an index. The number of data rows between the “marks” of an index. The value 8192 is appropriate for most tasks.
**Example**
@@ -147,11 +152,11 @@ Each data part is logically divided into granules. A granule is the smallest ind
The granule size is restricted by the `index_granularity` and `index_granularity_bytes` settings of the table engine. The number of rows in a granule lays in the `[1, index_granularity]` range, depending on the size of the rows. The size of a granule can exceed `index_granularity_bytes` if the size of a single row is greater than the value of the setting. In this case, the size of the granule equals the size of the row.
-## Primary Keys and Indexes in Queries {#primary-keys-and-indexes-in-queries}
+## Primary Keys and Indexes In Queries {#primary-keys-and-indexes-in-queries}
Take the `(CounterID, Date)` primary key as an example. In this case, the sorting and index can be illustrated as follows:
- Whole data: [-------------------------------------------------------------------------]
+ Whole data: [---------------------------------------------]
CounterID: [aaaaaaaaaaaaaaaaaabbbbcdeeeeeeeeeeeeefgggggggghhhhhhhhhiiiiiiiiikllllllll]
Date: [1111111222222233331233211111222222333211111112122222223111112223311122333]
Marks: | | | | | | | | | | |
@@ -193,7 +198,7 @@ The number of columns in the primary key is not explicitly limited. Depending on
A long primary key will negatively affect the insert performance and memory consumption, but extra columns in the primary key do not affect ClickHouse performance during `SELECT` queries.
-### Choosing a Primary Key that Differs from the Sorting Key {#choosing-a-primary-key-that-differs-from-the-sorting-key}
+### Choosing a Primary Key That Differs From the Sorting Key {#choosing-a-primary-key-that-differs-from-the-sorting-key}
It is possible to specify a primary key (an expression with values that are written in the index file for each mark) that is different from the sorting key (an expression for sorting the rows in data parts). In this case the primary key expression tuple must be a prefix of the sorting key expression tuple.
@@ -202,9 +207,9 @@ This feature is helpful when using the [SummingMergeTree](summingmergetree.md) a
In this case it makes sense to leave only a few columns in the primary key that will provide efficient range scans and add the remaining dimension columns to the sorting key tuple.
-[ALTER](../../query_language/alter.md) of the sorting key is a lightweight operation because when a new column is simultaneously added to the table and to the sorting key, existing data parts don’t need to be changed. Since the old sorting key is a prefix of the new sorting key and there is no data in the newly added column, the data is sorted by both the old and new sorting keys at the moment of table modification.
+[ALTER](../../../sql_reference/statements/alter.md) of the sorting key is a lightweight operation because when a new column is simultaneously added to the table and to the sorting key, existing data parts don’t need to be changed. Since the old sorting key is a prefix of the new sorting key and there is no data in the newly added column, the data is sorted by both the old and new sorting keys at the moment of table modification.
-### Use of Indexes and Partitions in Queries {#use-of-indexes-and-partitions-in-queries}
+### Use Of Indexes and Partitions In Queries {#use-of-indexes-and-partitions-in-queries}
For `SELECT` queries, ClickHouse analyzes whether an index can be used. An index can be used if the `WHERE/PREWHERE` clause has an expression (as one of the conjunction elements, or entirely) that represents an equality or inequality comparison operation, or if it has `IN` or `LIKE` with a fixed prefix on columns or expressions that are in the primary key or partitioning key, or on certain partially repetitive functions of these columns, or logical relationships of these expressions.
@@ -232,11 +237,11 @@ In the example below, the index can’t be used.
SELECT count() FROM table WHERE CounterID = 34 OR URL LIKE '%upyachka%'
```
-To check whether ClickHouse can use the index when running a query, use the settings [force\_index\_by\_date](../settings/settings.md#settings-force_index_by_date) and [force\_primary\_key](../settings/settings.md).
+To check whether ClickHouse can use the index when running a query, use the settings [force\_index\_by\_date](../../../operations/settings/settings.md#settings-force_index_by_date) and [force\_primary\_key](../../../operations/settings/settings.md).
The key for partitioning by month allows reading only those data blocks which contain dates from the proper range. In this case, the data block may contain data for many dates (up to an entire month). Within a block, data is sorted by primary key, which might not contain the date as the first column. Because of this, using a query with only a date condition that does not specify the primary key prefix will cause more data to be read than for a single date.
-### Use of Index for Partially-Monotonic Primary Keys {#use-of-index-for-partially-monotonic-primary-keys}
+### Use Of Index For Partially-monotonic Primary Keys {#use-of-index-for-partially-monotonic-primary-keys}
Consider, for example, the days of the month. They form a [monotonic sequence](https://en.wikipedia.org/wiki/Monotonic_function) for one month, but not monotonic for more extended periods. This is a partially-monotonic sequence. If a user creates the table with partially-monotonic primary key, ClickHouse creates a sparse index as usual. When a user selects data from this kind of table, ClickHouse analyzes the query conditions. If the user wants to get data between two marks of the index and both these marks fall within one month, ClickHouse can use the index in this particular case because it can calculate the distance between the parameters of a query and index marks.
@@ -244,7 +249,7 @@ ClickHouse cannot use an index if the values of the primary key in the query par
ClickHouse uses this logic not only for days of the month sequences, but for any primary key that represents a partially-monotonic sequence.
-### Data Skipping Indexes (Experimental) {#table_engine-mergetree-data_skipping-indexes}
+### Data Skipping Indexes (experimental) {#table_engine-mergetree-data_skipping-indexes}
The index declaration is in the columns section of the `CREATE` query.
@@ -278,7 +283,7 @@ SELECT count() FROM table WHERE s < 'z'
SELECT count() FROM table WHERE u64 * i32 == 10 AND u64 * length(s) >= 1234
```
-#### Available Types of Indices {#available-types-of-indices}
+#### Available Types Of Indices {#available-types-of-indices}
- `minmax`
@@ -307,7 +312,7 @@ SELECT count() FROM table WHERE u64 * i32 == 10 AND u64 * length(s) >= 1234
Supported data types: `Int*`, `UInt*`, `Float*`, `Enum`, `Date`, `DateTime`, `String`, `FixedString`, `Array`, `LowCardinality`, `Nullable`.
- The following functions can use it: [equals](../../query_language/functions/comparison_functions.md), [notEquals](../../query_language/functions/comparison_functions.md), [in](../../query_language/functions/in_functions.md), [notIn](../../query_language/functions/in_functions.md), [has](../../query_language/functions/array_functions.md).
+ The following functions can use it: [equals](../../../sql_reference/functions/comparison_functions.md), [notEquals](../../../sql_reference/functions/comparison_functions.md), [in](../../../sql_reference/functions/in_functions.md), [notIn](../../../sql_reference/functions/in_functions.md), [has](../../../sql_reference/functions/array_functions.md).
@@ -323,24 +328,24 @@ Conditions in the `WHERE` clause contains calls of the functions that operate wi
The `set` index can be used with all functions. Function subsets for other indexes are shown in the table below.
-| Function (operator) / Index | primary key | minmax | ngrambf\_v1 | tokenbf\_v1 | bloom\_filter |
-|----------------------------------------------------------------------------------------------------------|-------------|--------|-------------|-------------|---------------|
-| [equals (=, ==)](../../query_language/functions/comparison_functions.md#function-equals) | ✔ | ✔ | ✔ | ✔ | ✔ |
-| [notEquals(!=, \<\>)](../../query_language/functions/comparison_functions.md#function-notequals) | ✔ | ✔ | ✔ | ✔ | ✔ |
-| [like](../../query_language/functions/string_search_functions.md#function-like) | ✔ | ✔ | ✔ | ✗ | ✗ |
-| [notLike](../../query_language/functions/string_search_functions.md#function-notlike) | ✔ | ✔ | ✔ | ✗ | ✗ |
-| [startsWith](../../query_language/functions/string_functions.md#startswith) | ✔ | ✔ | ✔ | ✔ | ✗ |
-| [endsWith](../../query_language/functions/string_functions.md#endswith) | ✗ | ✗ | ✔ | ✔ | ✗ |
-| [multiSearchAny](../../query_language/functions/string_search_functions.md#function-multisearchany) | ✗ | ✗ | ✔ | ✗ | ✗ |
-| [in](../../query_language/functions/in_functions.md#in-functions) | ✔ | ✔ | ✔ | ✔ | ✔ |
-| [notIn](../../query_language/functions/in_functions.md#in-functions) | ✔ | ✔ | ✔ | ✔ | ✔ |
-| [less (\<)](../../query_language/functions/comparison_functions.md#function-less) | ✔ | ✔ | ✗ | ✗ | ✗ |
-| [greater (\>)](../../query_language/functions/comparison_functions.md#function-greater) | ✔ | ✔ | ✗ | ✗ | ✗ |
-| [lessOrEquals (\<=)](../../query_language/functions/comparison_functions.md#function-lessorequals) | ✔ | ✔ | ✗ | ✗ | ✗ |
-| [greaterOrEquals (\>=)](../../query_language/functions/comparison_functions.md#function-greaterorequals) | ✔ | ✔ | ✗ | ✗ | ✗ |
-| [empty](../../query_language/functions/array_functions.md#function-empty) | ✔ | ✔ | ✗ | ✗ | ✗ |
-| [notEmpty](../../query_language/functions/array_functions.md#function-notempty) | ✔ | ✔ | ✗ | ✗ | ✗ |
-| hasToken | ✗ | ✗ | ✗ | ✔ | ✗ |
+| Function (operator) / Index | primary key | minmax | ngrambf\_v1 | tokenbf\_v1 | bloom\_filter |
+|------------------------------------------------------------------|---------|------|---------|---------|---------|
+| [equals (=, ==)](../../../sql_reference/functions/comparison_functions.md#function-equals) | ✔ | ✔ | ✔ | ✔ | ✔ |
+| [notEquals(!=, \<\>)](../../../sql_reference/functions/comparison_functions.md#function-notequals) | ✔ | ✔ | ✔ | ✔ | ✔ |
+| [like](../../../sql_reference/functions/string_search_functions.md#function-like) | ✔ | ✔ | ✔ | ✗ | ✗ |
+| [notLike](../../../sql_reference/functions/string_search_functions.md#function-notlike) | ✔ | ✔ | ✔ | ✗ | ✗ |
+| [startsWith](../../../sql_reference/functions/string_functions.md#startswith) | ✔ | ✔ | ✔ | ✔ | ✗ |
+| [endsWith](../../../sql_reference/functions/string_functions.md#endswith) | ✗ | ✗ | ✔ | ✔ | ✗ |
+| [multiSearchAny](../../../sql_reference/functions/string_search_functions.md#function-multisearchany) | ✗ | ✗ | ✔ | ✗ | ✗ |
+| [in](../../../sql_reference/functions/in_functions.md#in-functions) | ✔ | ✔ | ✔ | ✔ | ✔ |
+| [notIn](../../../sql_reference/functions/in_functions.md#in-functions) | ✔ | ✔ | ✔ | ✔ | ✔ |
+| [less (\<)](../../../sql_reference/functions/comparison_functions.md#function-less) | ✔ | ✔ | ✗ | ✗ | ✗ |
+| [greater (\>)](../../../sql_reference/functions/comparison_functions.md#function-greater) | ✔ | ✔ | ✗ | ✗ | ✗ |
+| [lessOrEquals (\<=)](../../../sql_reference/functions/comparison_functions.md#function-lessorequals) | ✔ | ✔ | ✗ | ✗ | ✗ |
+| [greaterOrEquals (\>=)](../../../sql_reference/functions/comparison_functions.md#function-greaterorequals) | ✔ | ✔ | ✗ | ✗ | ✗ |
+| [empty](../../../sql_reference/functions/array_functions.md#function-empty) | ✔ | ✔ | ✗ | ✗ | ✗ |
+| [notEmpty](../../../sql_reference/functions/array_functions.md#function-notempty) | ✔ | ✔ | ✗ | ✗ | ✗ |
+| hasToken | ✗ | ✗ | ✗ | ✔ | ✗ |
Functions with a constant argument that is less than ngram size can’t be used by `ngrambf_v1` for query optimization.
@@ -365,13 +370,13 @@ For concurrent table access, we use multi-versioning. In other words, when a tab
Reading from a table is automatically parallelized.
-## TTL for Columns and Tables {#table_engine-mergetree-ttl}
+## TTL For Columns and Tables {#table_engine-mergetree-ttl}
Determines the lifetime of values.
The `TTL` clause can be set for the whole table and for each individual column. Table-level TTL can also specify logic of automatic move of data between disks and volumes.
-Expressions must evaluate to [Date](../../data_types/date.md) or [DateTime](../../data_types/datetime.md) data type.
+Expressions must evaluate to [Date](../../../sql_reference/data_types/date.md) or [DateTime](../../../sql_reference/data_types/datetime.md) data type.
Example:
@@ -380,7 +385,7 @@ TTL time_column
TTL time_column + interval
```
-To define `interval`, use [time interval](../../query_language/operators.md#operators-datetime) operators.
+To define `interval`, use [time interval](../../../sql_reference/operators.md#operators-datetime) operators.
``` sql
TTL date_time + INTERVAL 1 MONTH
@@ -471,26 +476,24 @@ Data with an expired TTL is removed when ClickHouse merges data parts.
When ClickHouse see that data is expired, it performs an off-schedule merge. To control the frequency of such merges, you can set [merge\_with\_ttl\_timeout](#mergetree_setting-merge_with_ttl_timeout). If the value is too low, it will perform many off-schedule merges that may consume a lot of resources.
-If you perform the `SELECT` query between merges, you may get expired data. To avoid it, use the [OPTIMIZE](../../query_language/misc.md#misc_operations-optimize) query before `SELECT`.
+If you perform the `SELECT` query between merges, you may get expired data. To avoid it, use the [OPTIMIZE](../../../sql_reference/statements/misc.md#misc_operations-optimize) query before `SELECT`.
-[Original article](https://clickhouse.tech/docs/en/operations/table_engines/mergetree/)
-
-## Using Multiple Block Devices for Data Storage {#table_engine-mergetree-multiple-volumes}
+## Using Multiple Block Devices For Data Storage {#table_engine-mergetree-multiple-volumes}
### Introduction {#introduction}
`MergeTree` family table engines can store data on multiple block devices. For example, it can be useful when the data of a certain table are implicitly split into “hot” and “cold”. The most recent data is regularly requested but requires only a small amount of space. On the contrary, the fat-tailed historical data is requested rarely. If several disks are available, the “hot” data may be located on fast disks (for example, NVMe SSDs or in memory), while the “cold” data - on relatively slow ones (for example, HDD).
-Data part is the minimum movable unit for `MergeTree`-engine tables. The data belonging to one part are stored on one disk. Data parts can be moved between disks in the background (according to user settings) as well as by means of the [ALTER](../../query_language/alter.md#alter_move-partition) queries.
+Data part is the minimum movable unit for `MergeTree`-engine tables. The data belonging to one part are stored on one disk. Data parts can be moved between disks in the background (according to user settings) as well as by means of the [ALTER](../../../sql_reference/statements/alter.md#alter_move-partition) queries.
### Terms {#terms}
- Disk — Block device mounted to the filesystem.
-- Default disk — Disk that stores the path specified in the [path](../server_settings/settings.md#server_settings-path) server setting.
+- Default disk — Disk that stores the path specified in the [path](../../../operations/server_configuration_parameters/settings.md#server_configuration_parameters-path) server setting.
- Volume — Ordered set of equal disks (similar to [JBOD](https://en.wikipedia.org/wiki/Non-RAID_drive_architectures)).
- Storage policy — Set of volumes and the rules for moving data between them.
-The names given to the described entities can be found in the system tables, [system.storage\_policies](../system_tables.md#system_tables-storage_policies) and [system.disks](../system_tables.md#system_tables-disks). To apply one of the configured storage policies for a table, use the `storage_policy` setting of `MergeTree`-engine family tables.
+The names given to the described entities can be found in the system tables, [system.storage\_policies](../../../operations/system_tables.md#system_tables-storage_policies) and [system.disks](../../../operations/system_tables.md#system_tables-disks). To apply one of the configured storage policies for a table, use the `storage_policy` setting of `MergeTree`-engine family tables.
### Configuration {#table_engine-mergetree-multiple-volumes-configure}
@@ -625,9 +628,9 @@ The `default` storage policy implies using only one volume, which consists of on
In the case of `MergeTree` tables, data is getting to disk in different ways:
- As a result of an insert (`INSERT` query).
-- During background merges and [mutations](../../query_language/alter.md#alter-mutations).
+- During background merges and [mutations](../../../sql_reference/statements/alter.md#alter-mutations).
- When downloading from another replica.
-- As a result of partition freezing [ALTER TABLE … FREEZE PARTITION](../../query_language/alter.md#alter_freeze-partition).
+- As a result of partition freezing [ALTER TABLE … FREEZE PARTITION](../../../sql_reference/statements/alter.md#alter_freeze-partition).
In all these cases except for mutations and partition freezing, a part is stored on a volume and a disk according to the given storage policy:
@@ -637,9 +640,9 @@ In all these cases except for mutations and partition freezing, a part is stored
Under the hood, mutations and partition freezing make use of [hard links](https://en.wikipedia.org/wiki/Hard_link). Hard links between different disks are not supported, therefore in such cases the resulting parts are stored on the same disks as the initial ones.
In the background, parts are moved between volumes on the basis of the amount of free space (`move_factor` parameter) according to the order the volumes are declared in the configuration file.
-Data is never transferred from the last one and into the first one. One may use system tables [system.part\_log](../system_tables.md#system_tables-part-log) (field `type = MOVE_PART`) and [system.parts](../system_tables.md#system_tables-parts) (fields `path` and `disk`) to monitor background moves. Also, the detailed information can be found in server logs.
+Data is never transferred from the last one and into the first one. One may use system tables [system.part\_log](../../../operations/system_tables.md#system_tables-part-log) (field `type = MOVE_PART`) and [system.parts](../../../operations/system_tables.md#system_tables-parts) (fields `path` and `disk`) to monitor background moves. Also, the detailed information can be found in server logs.
-User can force moving a part or a partition from one volume to another using the query [ALTER TABLE … MOVE PART\|PARTITION … TO VOLUME\|DISK …](../../query_language/alter.md#alter_move-partition), all the restrictions for background operations are taken into account. The query initiates a move on its own and does not wait for background operations to be completed. User will get an error message if not enough free space is available or if any of the required conditions are not met.
+User can force moving a part or a partition from one volume to another using the query [ALTER TABLE … MOVE PART\|PARTITION … TO VOLUME\|DISK …](../../../sql_reference/statements/alter.md#alter_move-partition), all the restrictions for background operations are taken into account. The query initiates a move on its own and does not wait for background operations to be completed. User will get an error message if not enough free space is available or if any of the required conditions are not met.
Moving data does not interfere with data replication. Therefore, different storage policies can be specified for the same table on different replicas.
diff --git a/docs/en/operations/table_engines/replacingmergetree.md b/docs/en/engines/table_engines/mergetree_family/replacingmergetree.md
similarity index 94%
rename from docs/en/operations/table_engines/replacingmergetree.md
rename to docs/en/engines/table_engines/mergetree_family/replacingmergetree.md
index b80c372b814..5bdad0d4074 100644
--- a/docs/en/operations/table_engines/replacingmergetree.md
+++ b/docs/en/engines/table_engines/mergetree_family/replacingmergetree.md
@@ -1,4 +1,9 @@
-# ReplacingMergeTree {#replacingmergetree}
+---
+toc_priority: 33
+toc_title: ReplacingMergeTree
+---
+
+# Replacingmergetree {#replacingmergetree}
The engine differs from [MergeTree](mergetree.md#table_engines-mergetree) in that it removes duplicate entries with the same primary key value (or more accurately, with the same [sorting key](mergetree.md) value).
@@ -22,7 +27,7 @@ CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster]
[SETTINGS name=value, ...]
```
-For a description of request parameters, see [request description](../../query_language/create.md).
+For a description of request parameters, see [request description](../../../sql_reference/statements/create.md).
**ReplacingMergeTree Parameters**
diff --git a/docs/en/operations/table_engines/replication.md b/docs/en/engines/table_engines/mergetree_family/replication.md
similarity index 94%
rename from docs/en/operations/table_engines/replication.md
rename to docs/en/engines/table_engines/mergetree_family/replication.md
index d38c2ec4c2b..8a4edb7ac8e 100644
--- a/docs/en/operations/table_engines/replication.md
+++ b/docs/en/engines/table_engines/mergetree_family/replication.md
@@ -1,3 +1,8 @@
+---
+toc_priority: 31
+toc_title: Data Replication
+---
+
# Data Replication {#table_engines-replication}
Replication is only supported for tables in the MergeTree family:
@@ -14,7 +19,7 @@ Replication works at the level of an individual table, not the entire server. A
Replication does not depend on sharding. Each shard has its own independent replication.
-Compressed data for `INSERT` and `ALTER` queries is replicated (for more information, see the documentation for [ALTER](../../query_language/alter.md#query_language_queries_alter)).
+Compressed data for `INSERT` and `ALTER` queries is replicated (for more information, see the documentation for [ALTER](../../../sql_reference/statements/alter.md#query_language_queries_alter)).
`CREATE`, `DROP`, `ATTACH`, `DETACH` and `RENAME` queries are executed on a single server and are not replicated:
@@ -24,7 +29,7 @@ Compressed data for `INSERT` and `ALTER` queries is replicated (for more informa
ClickHouse uses [Apache ZooKeeper](https://zookeeper.apache.org) for storing replicas meta information. Use ZooKeeper version 3.4.5 or newer.
-To use replication, set parameters in the [zookeeper](../server_settings/settings.md#server-settings_zookeeper) server configuration section.
+To use replication, set parameters in the [zookeeper](../../../operations/server_configuration_parameters/settings.md#server-settings_zookeeper) server configuration section.
!!! attention "Attention"
Don’t neglect the security setting. ClickHouse supports the `digest` [ACL scheme](https://zookeeper.apache.org/doc/current/zookeeperProgrammers.html#sc_ZooKeeperAccessControl) of the ZooKeeper security subsystem.
@@ -52,7 +57,7 @@ You can specify any existing ZooKeeper cluster and the system will use a directo
If ZooKeeper isn’t set in the config file, you can’t create replicated tables, and any existing replicated tables will be read-only.
-ZooKeeper is not used in `SELECT` queries because replication does not affect the performance of `SELECT` and queries run just as fast as they do for non-replicated tables. When querying distributed replicated tables, ClickHouse behavior is controlled by the settings [max\_replica\_delay\_for\_distributed\_queries](../settings/settings.md#settings-max_replica_delay_for_distributed_queries) and [fallback\_to\_stale\_replicas\_for\_distributed\_queries](../settings/settings.md#settings-fallback_to_stale_replicas_for_distributed_queries).
+ZooKeeper is not used in `SELECT` queries because replication does not affect the performance of `SELECT` and queries run just as fast as they do for non-replicated tables. When querying distributed replicated tables, ClickHouse behavior is controlled by the settings [max\_replica\_delay\_for\_distributed\_queries](../../../operations/settings/settings.md#settings-max_replica_delay_for_distributed_queries) and [fallback\_to\_stale\_replicas\_for\_distributed\_queries](../../../operations/settings/settings.md#settings-fallback_to_stale_replicas_for_distributed_queries).
For each `INSERT` query, approximately ten entries are added to ZooKeeper through several transactions. (To be more precise, this is for each inserted block of data; an INSERT query contains one block or one block per `max_insert_block_size = 1048576` rows.) This leads to slightly longer latencies for `INSERT` compared to non-replicated tables. But if you follow the recommendations to insert data in batches of no more than one `INSERT` per second, it doesn’t create any problems. The entire ClickHouse cluster used for coordinating one ZooKeeper cluster has a total of several hundred `INSERTs` per second. The throughput on data inserts (the number of rows per second) is just as high as for non-replicated data.
@@ -64,7 +69,7 @@ By default, an INSERT query waits for confirmation of writing the data from only
Each block of data is written atomically. The INSERT query is divided into blocks up to `max_insert_block_size = 1048576` rows. In other words, if the `INSERT` query has less than 1048576 rows, it is made atomically.
-Data blocks are deduplicated. For multiple writes of the same data block (data blocks of the same size containing the same rows in the same order), the block is only written once. The reason for this is in case of network failures when the client application doesn’t know if the data was written to the DB, so the `INSERT` query can simply be repeated. It doesn’t matter which replica INSERTs were sent to with identical data. `INSERTs` are idempotent. Deduplication parameters are controlled by [merge\_tree](../server_settings/settings.md#server_settings-merge_tree) server settings.
+Data blocks are deduplicated. For multiple writes of the same data block (data blocks of the same size containing the same rows in the same order), the block is only written once. The reason for this is in case of network failures when the client application doesn’t know if the data was written to the DB, so the `INSERT` query can simply be repeated. It doesn’t matter which replica INSERTs were sent to with identical data. `INSERTs` are idempotent. Deduplication parameters are controlled by [merge\_tree](../../../operations/server_configuration_parameters/settings.md#server_configuration_parameters-merge_tree) server settings.
During replication, only the source data to insert is transferred over the network. Further data transformation (merging) is coordinated and performed on all the replicas in the same way. This minimizes network usage, which means that replication works well when replicas reside in different datacenters. (Note that duplicating data in different datacenters is the main goal of replication.)
@@ -181,7 +186,7 @@ An alternative recovery option is to delete information about the lost replica f
There is no restriction on network bandwidth during recovery. Keep this in mind if you are restoring many replicas at once.
-## Converting from MergeTree to ReplicatedMergeTree {#converting-from-mergetree-to-replicatedmergetree}
+## Converting From Mergetree To Replicatedmergetree {#converting-from-mergetree-to-replicatedmergetree}
We use the term `MergeTree` to refer to all table engines in the `MergeTree family`, the same as for `ReplicatedMergeTree`.
@@ -193,7 +198,7 @@ Rename the existing MergeTree table, then create a `ReplicatedMergeTree` table w
Move the data from the old table to the `detached` subdirectory inside the directory with the new table data (`/var/lib/clickhouse/data/db_name/table_name/`).
Then run `ALTER TABLE ATTACH PARTITION` on one of the replicas to add these data parts to the working set.
-## Converting from ReplicatedMergeTree to MergeTree {#converting-from-replicatedmergetree-to-mergetree}
+## Converting From Replicatedmergetree To Mergetree {#converting-from-replicatedmergetree-to-mergetree}
Create a MergeTree table with a different name. Move all the data from the directory with the `ReplicatedMergeTree` table data to the new table’s data directory. Then delete the `ReplicatedMergeTree` table and restart the server.
@@ -204,7 +209,7 @@ If you want to get rid of a `ReplicatedMergeTree` table without launching the se
After this, you can launch the server, create a `MergeTree` table, move the data to its directory, and then restart the server.
-## Recovery When Metadata in The ZooKeeper Cluster is Lost or Damaged {#recovery-when-metadata-in-the-zookeeper-cluster-is-lost-or-damaged}
+## Recovery When Metadata In The Zookeeper Cluster Is Lost or Damaged {#recovery-when-metadata-in-the-zookeeper-cluster-is-lost-or-damaged}
If the data in ZooKeeper was lost or damaged, you can save data by moving it to an unreplicated table as described above.
diff --git a/docs/en/operations/table_engines/summingmergetree.md b/docs/en/engines/table_engines/mergetree_family/summingmergetree.md
similarity index 86%
rename from docs/en/operations/table_engines/summingmergetree.md
rename to docs/en/engines/table_engines/mergetree_family/summingmergetree.md
index 92705718663..5c007ee0b7a 100644
--- a/docs/en/operations/table_engines/summingmergetree.md
+++ b/docs/en/engines/table_engines/mergetree_family/summingmergetree.md
@@ -1,4 +1,9 @@
-# SummingMergeTree {#summingmergetree}
+---
+toc_priority: 34
+toc_title: SummingMergeTree
+---
+
+# Summingmergetree {#summingmergetree}
The engine inherits from [MergeTree](mergetree.md#table_engines-mergetree). The difference is that when merging data parts for `SummingMergeTree` tables ClickHouse replaces all the rows with the same primary key (or more accurately, with the same [sorting key](mergetree.md)) with one row which contains summarized values for the columns with the numeric data type. If the sorting key is composed in a way that a single key value corresponds to large number of rows, this significantly reduces storage volume and speeds up data selection.
@@ -19,7 +24,7 @@ CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster]
[SETTINGS name=value, ...]
```
-For a description of request parameters, see [request description](../../query_language/create.md).
+For a description of request parameters, see [request description](../../../sql_reference/statements/create.md).
**Parameters of SummingMergeTree**
@@ -91,9 +96,9 @@ SELECT key, sum(value) FROM summtt GROUP BY key
When data are inserted into a table, they are saved as-is. Clickhouse merges the inserted parts of data periodically and this is when rows with the same primary key are summed and replaced with one for each resulting part of data.
-ClickHouse can merge the data parts so that different resulting parts of data cat consist rows with the same primary key, i.e. the summation will be incomplete. Therefore (`SELECT`) an aggregate function [sum()](../../query_language/agg_functions/reference.md#agg_function-sum) and `GROUP BY` clause should be used in a query as described in the example above.
+ClickHouse can merge the data parts so that different resulting parts of data cat consist rows with the same primary key, i.e. the summation will be incomplete. Therefore (`SELECT`) an aggregate function [sum()](../../../sql_reference/aggregate_functions/reference.md#agg_function-sum) and `GROUP BY` clause should be used in a query as described in the example above.
-### Common rules for summation {#common-rules-for-summation}
+### Common Rules For Summation {#common-rules-for-summation}
The values in the columns with the numeric data type are summarized. The set of columns is defined by the parameter `columns`.
@@ -103,9 +108,9 @@ If column is not in the primary key and is not summarized, an arbitrary value is
The values are not summarized for columns in the primary key.
-### The Summation in the AggregateFunction Columns {#the-summation-in-the-aggregatefunction-columns}
+### The Summation In the Aggregatefunction Columns {#the-summation-in-the-aggregatefunction-columns}
-For columns of [AggregateFunction type](../../data_types/nested_data_structures/aggregatefunction.md) ClickHouse behaves as [AggregatingMergeTree](aggregatingmergetree.md) engine aggregating according to the function.
+For columns of [AggregateFunction type](../../../sql_reference/data_types/aggregatefunction.md) ClickHouse behaves as [AggregatingMergeTree](aggregatingmergetree.md) engine aggregating according to the function.
### Nested Structures {#nested-structures}
@@ -127,7 +132,7 @@ Examples:
[(1, 100), (2, 150)] + [(1, -100)] -> [(2, 150)]
```
-When requesting data, use the [sumMap(key, value)](../../query_language/agg_functions/reference.md) function for aggregation of `Map`.
+When requesting data, use the [sumMap(key, value)](../../../sql_reference/aggregate_functions/reference.md) function for aggregation of `Map`.
For nested data structure, you do not need to specify its columns in the tuple of columns for summation.
diff --git a/docs/en/operations/table_engines/versionedcollapsingmergetree.md b/docs/en/engines/table_engines/mergetree_family/versionedcollapsingmergetree.md
similarity index 98%
rename from docs/en/operations/table_engines/versionedcollapsingmergetree.md
rename to docs/en/engines/table_engines/mergetree_family/versionedcollapsingmergetree.md
index 381b56064d7..2c7a28263f8 100644
--- a/docs/en/operations/table_engines/versionedcollapsingmergetree.md
+++ b/docs/en/engines/table_engines/mergetree_family/versionedcollapsingmergetree.md
@@ -1,4 +1,9 @@
-# VersionedCollapsingMergeTree {#versionedcollapsingmergetree}
+---
+toc_priority: 37
+toc_title: VersionedCollapsingMergeTree
+---
+
+# Versionedcollapsingmergetree {#versionedcollapsingmergetree}
This engine:
@@ -24,7 +29,7 @@ CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster]
[SETTINGS name=value, ...]
```
-For a description of query parameters, see the [query description](../../query_language/create.md).
+For a description of query parameters, see the [query description](../../../sql_reference/statements/create.md).
**Engine Parameters**
@@ -136,7 +141,7 @@ The aggregates `count`, `sum` and `avg` can be calculated this way. The aggregat
If you need to extract the data with “collapsing” but without aggregation (for example, to check whether rows are present whose newest values match certain conditions), you can use the `FINAL` modifier for the `FROM` clause. This approach is inefficient and should not be used with large tables.
-## Example of Use {#example-of-use}
+## Example Of Use {#example-of-use}
Example data:
diff --git a/docs/en/operations/table_engines/buffer.md b/docs/en/engines/table_engines/special/buffer.md
similarity index 99%
rename from docs/en/operations/table_engines/buffer.md
rename to docs/en/engines/table_engines/special/buffer.md
index 2f0bd5d1bb3..12143044f21 100644
--- a/docs/en/operations/table_engines/buffer.md
+++ b/docs/en/engines/table_engines/special/buffer.md
@@ -1,3 +1,8 @@
+---
+toc_priority: 45
+toc_title: Buffer
+---
+
# Buffer {#buffer}
Buffers the data to write in RAM, periodically flushing it to another table. During the read operation, data is read from the buffer and the other table simultaneously.
diff --git a/docs/en/operations/table_engines/dictionary.md b/docs/en/engines/table_engines/special/dictionary.md
similarity index 87%
rename from docs/en/operations/table_engines/dictionary.md
rename to docs/en/engines/table_engines/special/dictionary.md
index 18b5dfabd78..86b8214c8e5 100644
--- a/docs/en/operations/table_engines/dictionary.md
+++ b/docs/en/engines/table_engines/special/dictionary.md
@@ -1,6 +1,11 @@
+---
+toc_priority: 35
+toc_title: Dictionary
+---
+
# Dictionary {#dictionary}
-The `Dictionary` engine displays the [dictionary](../../query_language/dicts/external_dicts.md) data as a ClickHouse table.
+The `Dictionary` engine displays the [dictionary](../../../sql_reference/dictionaries/external_dictionaries/external_dicts.md) data as a ClickHouse table.
As an example, consider a dictionary of `products` with the following configuration:
@@ -57,7 +62,7 @@ WHERE name = 'products'
└──────────┴──────┴────────┴─────────────────┴─────────────────┴─────────────────┴───────────────┴─────────────────┘
```
-You can use the [dictGet\*](../../query_language/functions/ext_dict_functions.md#ext_dict_functions) function to get the dictionary data in this format.
+You can use the [dictGet\*](../../../sql_reference/functions/ext_dict_functions.md#ext_dict_functions) function to get the dictionary data in this format.
This view isn’t helpful when you need to get raw data, or when performing a `JOIN` operation. For these cases, you can use the `Dictionary` engine, which displays the dictionary data in a table.
diff --git a/docs/en/operations/table_engines/distributed.md b/docs/en/engines/table_engines/special/distributed.md
similarity index 88%
rename from docs/en/operations/table_engines/distributed.md
rename to docs/en/engines/table_engines/special/distributed.md
index d357e44b981..c23aa10efb8 100644
--- a/docs/en/operations/table_engines/distributed.md
+++ b/docs/en/engines/table_engines/special/distributed.md
@@ -1,3 +1,8 @@
+---
+toc_priority: 33
+toc_title: Distributed
+---
+
# Distributed {#distributed}
**Tables with Distributed engine do not store any data by themself**, but allow distributed query processing on multiple servers.
@@ -18,7 +23,7 @@ The Distributed engine accepts parameters:
See also:
- `insert_distributed_sync` setting
- - [MergeTree](mergetree.md#table_engine-mergetree-multiple-volumes) for the examples
+ - [MergeTree](../mergetree_family/mergetree.md#table_engine-mergetree-multiple-volumes) for the examples
Example:
@@ -79,12 +84,12 @@ Cluster names must not contain dots.
The parameters `host`, `port`, and optionally `user`, `password`, `secure`, `compression` are specified for each server:
- `host` – The address of the remote server. You can use either the domain or the IPv4 or IPv6 address. If you specify the domain, the server makes a DNS request when it starts, and the result is stored as long as the server is running. If the DNS request fails, the server doesn’t start. If you change the DNS record, restart the server.
- `port` – The TCP port for messenger activity (‘tcp\_port’ in the config, usually set to 9000). Do not confuse it with http\_port.
-- `user` – Name of the user for connecting to a remote server. Default value: default. This user must have access to connect to the specified server. Access is configured in the users.xml file. For more information, see the section [Access rights](../../operations/access_rights.md).
+- `user` – Name of the user for connecting to a remote server. Default value: default. This user must have access to connect to the specified server. Access is configured in the users.xml file. For more information, see the section [Access rights](../../../operations/access_rights.md).
- `password` – The password for connecting to a remote server (not masked). Default value: empty string.
- `secure` - Use ssl for connection, usually you also should define `port` = 9440. Server should listen on 9440 and have correct certificates.
- `compression` - Use data compression. Default value: true.
-When specifying replicas, one of the available replicas will be selected for each of the shards when reading. You can configure the algorithm for load balancing (the preference for which replica to access) – see the [load\_balancing](../settings/settings.md#settings-load_balancing) setting.
+When specifying replicas, one of the available replicas will be selected for each of the shards when reading. You can configure the algorithm for load balancing (the preference for which replica to access) – see the [load\_balancing](../../../operations/settings/settings.md#settings-load_balancing) setting.
If the connection with the server is not established, there will be an attempt to connect with a short timeout. If the connection failed, the next replica will be selected, and so on for all the replicas. If the connection attempt failed for all the replicas, the attempt will be repeated the same way, several times.
This works in favour of resiliency, but does not provide complete fault tolerance: a remote server might accept the connection, but might not work, or work poorly.
@@ -96,7 +101,7 @@ To view your clusters, use the ‘system.clusters’ table.
The Distributed engine allows working with a cluster like a local server. However, the cluster is inextensible: you must write its configuration in the server config file (even better, for all the cluster’s servers).
-The Distributed engine requires writing clusters to the config file. Clusters from the config file are updated on the fly, without restarting the server. If you need to send a query to an unknown set of shards and replicas each time, you don’t need to create a Distributed table – use the ‘remote’ table function instead. See the section [Table functions](../../query_language/table_functions/index.md).
+The Distributed engine requires writing clusters to the config file. Clusters from the config file are updated on the fly, without restarting the server. If you need to send a query to an unknown set of shards and replicas each time, you don’t need to create a Distributed table – use the ‘remote’ table function instead. See the section [Table functions](../../../sql_reference/table_functions/index.md).
There are two methods for writing data to a cluster:
@@ -125,18 +130,18 @@ You should be concerned about the sharding scheme in the following cases:
- Queries are used that require joining data (IN or JOIN) by a specific key. If data is sharded by this key, you can use local IN or JOIN instead of GLOBAL IN or GLOBAL JOIN, which is much more efficient.
- A large number of servers is used (hundreds or more) with a large number of small queries (queries of individual clients - websites, advertisers, or partners). In order for the small queries to not affect the entire cluster, it makes sense to locate data for a single client on a single shard. Alternatively, as we’ve done in Yandex.Metrica, you can set up bi-level sharding: divide the entire cluster into “layers”, where a layer may consist of multiple shards. Data for a single client is located on a single layer, but shards can be added to a layer as necessary, and data is randomly distributed within them. Distributed tables are created for each layer, and a single shared distributed table is created for global queries.
-Data is written asynchronously. When inserted in the table, the data block is just written to the local file system. The data is sent to the remote servers in the background as soon as possible. The period for sending data is managed by the [distributed\_directory\_monitor\_sleep\_time\_ms](../settings/settings.md#distributed_directory_monitor_sleep_time_ms) and [distributed\_directory\_monitor\_max\_sleep\_time\_ms](../settings/settings.md#distributed_directory_monitor_max_sleep_time_ms) settings. The `Distributed` engine sends each file with inserted data separately, but you can enable batch sending of files with the [distributed\_directory\_monitor\_batch\_inserts](../settings/settings.md#distributed_directory_monitor_batch_inserts) setting. This setting improves cluster performance by better utilizing local server and network resources. You should check whether data is sent successfully by checking the list of files (data waiting to be sent) in the table directory: `/var/lib/clickhouse/data/database/table/`.
+Data is written asynchronously. When inserted in the table, the data block is just written to the local file system. The data is sent to the remote servers in the background as soon as possible. The period for sending data is managed by the [distributed\_directory\_monitor\_sleep\_time\_ms](../../../operations/settings/settings.md#distributed_directory_monitor_sleep_time_ms) and [distributed\_directory\_monitor\_max\_sleep\_time\_ms](../../../operations/settings/settings.md#distributed_directory_monitor_max_sleep_time_ms) settings. The `Distributed` engine sends each file with inserted data separately, but you can enable batch sending of files with the [distributed\_directory\_monitor\_batch\_inserts](../../../operations/settings/settings.md#distributed_directory_monitor_batch_inserts) setting. This setting improves cluster performance by better utilizing local server and network resources. You should check whether data is sent successfully by checking the list of files (data waiting to be sent) in the table directory: `/var/lib/clickhouse/data/database/table/`.
If the server ceased to exist or had a rough restart (for example, after a device failure) after an INSERT to a Distributed table, the inserted data might be lost. If a damaged data part is detected in the table directory, it is transferred to the ‘broken’ subdirectory and no longer used.
-When the max\_parallel\_replicas option is enabled, query processing is parallelized across all replicas within a single shard. For more information, see the section [max\_parallel\_replicas](../settings/settings.md#settings-max_parallel_replicas).
+When the max\_parallel\_replicas option is enabled, query processing is parallelized across all replicas within a single shard. For more information, see the section [max\_parallel\_replicas](../../../operations/settings/settings.md#settings-max_parallel_replicas).
## Virtual Columns {#virtual-columns}
-- `_shard_num` — Contains the `shard_num` (from `system.clusters`). Type: [UInt32](../../data_types/int_uint.md).
+- `_shard_num` — Contains the `shard_num` (from `system.clusters`). Type: [UInt32](../../../sql_reference/data_types/int_uint.md).
!!! note "Note"
- Since [`remote`](../../query_language/table_functions/remote.md)/`cluster` table functions internally create temporary instance of the same Distributed engine, `_shard_num` is available there too.
+ Since [`remote`](../../../sql_reference/table_functions/remote.md)/`cluster` table functions internally create temporary instance of the same Distributed engine, `_shard_num` is available there too.
**See Also**
diff --git a/docs/en/operations/table_engines/external_data.md b/docs/en/engines/table_engines/special/external_data.md
similarity index 98%
rename from docs/en/operations/table_engines/external_data.md
rename to docs/en/engines/table_engines/special/external_data.md
index e520bb6fe0a..de487aef154 100644
--- a/docs/en/operations/table_engines/external_data.md
+++ b/docs/en/engines/table_engines/special/external_data.md
@@ -1,3 +1,8 @@
+---
+toc_priority: 34
+toc_title: External data
+---
+
# External Data for Query Processing {#external-data-for-query-processing}
ClickHouse allows sending a server the data that is needed for processing a query, together with a SELECT query. This data is put in a temporary table (see the section “Temporary tables”) and can be used in the query (for example, in IN operators).
diff --git a/docs/en/operations/table_engines/file.md b/docs/en/engines/table_engines/special/file.md
similarity index 71%
rename from docs/en/operations/table_engines/file.md
rename to docs/en/engines/table_engines/special/file.md
index 3f83bfd0a66..f0a9cd4e7e6 100644
--- a/docs/en/operations/table_engines/file.md
+++ b/docs/en/engines/table_engines/special/file.md
@@ -1,7 +1,12 @@
+---
+toc_priority: 37
+toc_title: File
+---
+
# File {#table_engines-file}
The File table engine keeps the data in a file in one of the supported [file
-formats](../../interfaces/formats.md#formats) (TabSeparated, Native, etc.).
+formats](../../../interfaces/formats.md#formats) (TabSeparated, Native, etc.).
Usage examples:
@@ -9,7 +14,7 @@ Usage examples:
- Convert data from one format to another.
- Updating data in ClickHouse via editing a file on a disk.
-## Usage in ClickHouse Server {#usage-in-clickhouse-server}
+## Usage In Clickhouse Server {#usage-in-clickhouse-server}
``` sql
File(Format)
@@ -18,13 +23,13 @@ File(Format)
The `Format` parameter specifies one of the available file formats. To perform
`SELECT` queries, the format must be supported for input, and to perform
`INSERT` queries – for output. The available formats are listed in the
-[Formats](../../interfaces/formats.md#formats) section.
+[Formats](../../../interfaces/formats.md#formats) section.
-ClickHouse does not allow to specify filesystem path for`File`. It will use folder defined by [path](../server_settings/settings.md) setting in server configuration.
+ClickHouse does not allow to specify filesystem path for`File`. It will use folder defined by [path](../../../operations/server_configuration_parameters/settings.md) setting in server configuration.
When creating table using `File(Format)` it creates empty subdirectory in that folder. When data is written to that table, it’s put into `data.Format` file in that subdirectory.
-You may manually create this subfolder and file in server filesystem and then [ATTACH](../../query_language/misc.md) it to table information with matching name, so you can query data from that file.
+You may manually create this subfolder and file in server filesystem and then [ATTACH](../../../sql_reference/statements/misc.md) it to table information with matching name, so you can query data from that file.
!!! warning "Warning"
Be careful with this functionality, because ClickHouse does not keep track of external changes to such files. The result of simultaneous writes via ClickHouse and outside of ClickHouse is undefined.
@@ -60,16 +65,16 @@ SELECT * FROM file_engine_table
└──────┴───────┘
```
-## Usage in Clickhouse-local {#usage-in-clickhouse-local}
+## Usage In Clickhouse-local {#usage-in-clickhouse-local}
-In [clickhouse-local](../utils/clickhouse-local.md) File engine accepts file path in addition to `Format`. Default input/output streams can be specified using numeric or human-readable names like `0` or `stdin`, `1` or `stdout`.
+In [clickhouse-local](../../../operations/utilities/clickhouse-local.md) File engine accepts file path in addition to `Format`. Default input/output streams can be specified using numeric or human-readable names like `0` or `stdin`, `1` or `stdout`.
**Example:**
``` bash
$ echo -e "1,2\n3,4" | clickhouse-local -q "CREATE TABLE table (a Int64, b Int64) ENGINE = File(CSV, stdin); SELECT a, b FROM table; DROP TABLE table"
```
-## Details of Implementation {#details-of-implementation}
+## Details Of Implementation {#details-of-implementation}
- Multiple `SELECT` queries can be performed concurrently, but `INSERT` queries will wait each other.
- Supported creating new file by `INSERT` query.
diff --git a/docs/en/operations/table_engines/generate.md b/docs/en/engines/table_engines/special/generate.md
similarity index 76%
rename from docs/en/operations/table_engines/generate.md
rename to docs/en/engines/table_engines/special/generate.md
index e7e2cf9259e..d4b573c6839 100644
--- a/docs/en/operations/table_engines/generate.md
+++ b/docs/en/engines/table_engines/special/generate.md
@@ -1,4 +1,9 @@
-# GenerateRandom {#table_engines-generate}
+---
+toc_priority: 46
+toc_title: GenerateRandom
+---
+
+# Generaterandom {#table_engines-generate}
The GenerateRandom table engine produces random data for given table schema.
@@ -7,7 +12,7 @@ Usage examples:
- Use in test to populate reproducible large table.
- Generate random input for fuzzing tests.
-## Usage in ClickHouse Server {#usage-in-clickhouse-server}
+## Usage In Clickhouse Server {#usage-in-clickhouse-server}
``` sql
ENGINE = GenerateRandom(random_seed, max_string_length, max_array_length)
@@ -18,7 +23,7 @@ array columns and strings correspondingly in generated data.
Generate table engine supports only `SELECT` queries.
-It supports all [DataTypes](../../data_types/index.md) that can be stored in a table except `LowCardinality` and `AggregateFunction`.
+It supports all [DataTypes](../../../sql_reference/data_types/index.md) that can be stored in a table except `LowCardinality` and `AggregateFunction`.
**Example:**
@@ -42,7 +47,7 @@ SELECT * FROM generate_engine_table LIMIT 3
└──────┴────────────┘
```
-## Details of Implementation {#details-of-implementation}
+## Details Of Implementation {#details-of-implementation}
- Not supported:
- `ALTER`
diff --git a/docs/en/engines/table_engines/special/index.md b/docs/en/engines/table_engines/special/index.md
new file mode 100644
index 00000000000..22cebf295c1
--- /dev/null
+++ b/docs/en/engines/table_engines/special/index.md
@@ -0,0 +1,5 @@
+---
+toc_folder_title: Special
+toc_priority: 31
+---
+
diff --git a/docs/en/operations/table_engines/join.md b/docs/en/engines/table_engines/special/join.md
similarity index 64%
rename from docs/en/operations/table_engines/join.md
rename to docs/en/engines/table_engines/special/join.md
index 5d902519fde..5c41396141d 100644
--- a/docs/en/operations/table_engines/join.md
+++ b/docs/en/engines/table_engines/special/join.md
@@ -1,6 +1,11 @@
+---
+toc_priority: 40
+toc_title: Join
+---
+
# Join {#join}
-Prepared data structure for using in [JOIN](../../query_language/select.md#select-join) operations.
+Prepared data structure for using in [JOIN](../../../sql_reference/statements/select.md#select-join) operations.
## Creating a Table {#creating-a-table}
@@ -12,12 +17,12 @@ CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster]
) ENGINE = Join(join_strictness, join_type, k1[, k2, ...])
```
-See the detailed description of the [CREATE TABLE](../../query_language/create.md#create-table-query) query.
+See the detailed description of the [CREATE TABLE](../../../sql_reference/statements/create.md#create-table-query) query.
**Engine Parameters**
-- `join_strictness` – [JOIN strictness](../../query_language/select.md#select-join-strictness).
-- `join_type` – [JOIN type](../../query_language/select.md#select-join-types).
+- `join_strictness` – [JOIN strictness](../../../sql_reference/statements/select.md#select-join-strictness).
+- `join_type` – [JOIN type](../../../sql_reference/statements/select.md#select-join-types).
- `k1[, k2, ...]` – Key columns from the `USING` clause that the `JOIN` operation is made with.
Enter `join_strictness` and `join_type` parameters without quotes, for example, `Join(ANY, LEFT, col1)`. They must match the `JOIN` operation that the table will be used for. If the parameters don’t match, ClickHouse doesn’t throw an exception and may return incorrect data.
@@ -79,21 +84,21 @@ You can use `INSERT` queries to add data to the `Join`-engine tables. If the tab
You cannot perform a `SELECT` query directly from the table. Instead, use one of the following methods:
- Place the table to the right side in a `JOIN` clause.
-- Call the [joinGet](../../query_language/functions/other_functions.md#joinget) function, which lets you extract data from the table the same way as from a dictionary.
+- Call the [joinGet](../../../sql_reference/functions/other_functions.md#joinget) function, which lets you extract data from the table the same way as from a dictionary.
### Limitations and Settings {#join-limitations-and-settings}
When creating a table, the following settings are applied:
-- [join\_use\_nulls](../settings/settings.md#join_use_nulls)
-- [max\_rows\_in\_join](../settings/query_complexity.md#settings-max_rows_in_join)
-- [max\_bytes\_in\_join](../settings/query_complexity.md#settings-max_bytes_in_join)
-- [join\_overflow\_mode](../settings/query_complexity.md#settings-join_overflow_mode)
-- [join\_any\_take\_last\_row](../settings/settings.md#settings-join_any_take_last_row)
+- [join\_use\_nulls](../../../operations/settings/settings.md#join_use_nulls)
+- [max\_rows\_in\_join](../../../operations/settings/query_complexity.md#settings-max_rows_in_join)
+- [max\_bytes\_in\_join](../../../operations/settings/query_complexity.md#settings-max_bytes_in_join)
+- [join\_overflow\_mode](../../../operations/settings/query_complexity.md#settings-join_overflow_mode)
+- [join\_any\_take\_last\_row](../../../operations/settings/settings.md#settings-join_any_take_last_row)
The `Join`-engine tables can’t be used in `GLOBAL JOIN` operations.
-The `Join`-engine allows use [join\_use\_nulls](../settings/settings.md#join_use_nulls) setting in the `CREATE TABLE` statement. And [SELECT](../../query_language/select.md) query allows use `join_use_nulls` too. If you have different `join_use_nulls` settings, you can get an error joining table. It depends on kind of JOIN. When you use [joinGet](../../query_language/functions/other_functions.md#joinget) function, you have to use the same `join_use_nulls` setting in `CRATE TABLE` and `SELECT` statements.
+The `Join`-engine allows use [join\_use\_nulls](../../../operations/settings/settings.md#join_use_nulls) setting in the `CREATE TABLE` statement. And [SELECT](../../../sql_reference/statements/select.md) query allows use `join_use_nulls` too. If you have different `join_use_nulls` settings, you can get an error joining table. It depends on kind of JOIN. When you use [joinGet](../../../sql_reference/functions/other_functions.md#joinget) function, you have to use the same `join_use_nulls` setting in `CRATE TABLE` and `SELECT` statements.
## Data Storage {#data-storage}
diff --git a/docs/en/engines/table_engines/special/materializedview.md b/docs/en/engines/table_engines/special/materializedview.md
new file mode 100644
index 00000000000..49ad2411f83
--- /dev/null
+++ b/docs/en/engines/table_engines/special/materializedview.md
@@ -0,0 +1,10 @@
+---
+toc_priority: 43
+toc_title: MaterializedView
+---
+
+# Materializedview {#materializedview}
+
+Used for implementing materialized views (for more information, see [CREATE TABLE](../../../sql_reference/statements/create.md)). For storing data, it uses a different engine that was specified when creating the view. When reading from a table, it just uses this engine.
+
+[Original article](https://clickhouse.tech/docs/en/operations/table_engines/materializedview/)
diff --git a/docs/en/operations/table_engines/memory.md b/docs/en/engines/table_engines/special/memory.md
similarity index 96%
rename from docs/en/operations/table_engines/memory.md
rename to docs/en/engines/table_engines/special/memory.md
index b490eb1e08f..abdb07658d4 100644
--- a/docs/en/operations/table_engines/memory.md
+++ b/docs/en/engines/table_engines/special/memory.md
@@ -1,3 +1,8 @@
+---
+toc_priority: 44
+toc_title: Memory
+---
+
# Memory {#memory}
The Memory engine stores data in RAM, in uncompressed form. Data is stored in exactly the same form as it is received when read. In other words, reading from this table is completely free.
diff --git a/docs/en/operations/table_engines/merge.md b/docs/en/engines/table_engines/special/merge.md
similarity index 96%
rename from docs/en/operations/table_engines/merge.md
rename to docs/en/engines/table_engines/special/merge.md
index 04554057fff..8027266b441 100644
--- a/docs/en/operations/table_engines/merge.md
+++ b/docs/en/engines/table_engines/special/merge.md
@@ -1,3 +1,8 @@
+---
+toc_priority: 36
+toc_title: Merge
+---
+
# Merge {#merge}
The `Merge` engine (not to be confused with `MergeTree`) does not store data itself, but allows reading from any number of other tables simultaneously.
@@ -52,7 +57,7 @@ FROM WatchLog
## Virtual Columns {#virtual-columns}
-- `_table` — Contains the name of the table from which data was read. Type: [String](../../data_types/string.md).
+- `_table` — Contains the name of the table from which data was read. Type: [String](../../../sql_reference/data_types/string.md).
You can set the constant conditions on `_table` in the `WHERE/PREWHERE` clause (for example, `WHERE _table='xyz'`). In this case the read operation is performed only for that tables where the condition on `_table` is satisfied, so the `_table` column acts as an index.
diff --git a/docs/en/operations/table_engines/null.md b/docs/en/engines/table_engines/special/null.md
similarity index 88%
rename from docs/en/operations/table_engines/null.md
rename to docs/en/engines/table_engines/special/null.md
index 030f470e656..73c5a2b1ea6 100644
--- a/docs/en/operations/table_engines/null.md
+++ b/docs/en/engines/table_engines/special/null.md
@@ -1,3 +1,8 @@
+---
+toc_priority: 38
+toc_title: 'Null'
+---
+
# Null {#null}
When writing to a Null table, data is ignored. When reading from a Null table, the response is empty.
diff --git a/docs/en/operations/table_engines/set.md b/docs/en/engines/table_engines/special/set.md
similarity index 95%
rename from docs/en/operations/table_engines/set.md
rename to docs/en/engines/table_engines/special/set.md
index cc881821001..fb0c5952ae4 100644
--- a/docs/en/operations/table_engines/set.md
+++ b/docs/en/engines/table_engines/special/set.md
@@ -1,3 +1,8 @@
+---
+toc_priority: 39
+toc_title: Set
+---
+
# Set {#set}
A data set that is always in RAM. It is intended for use on the right side of the IN operator (see the section “IN operators”).
diff --git a/docs/en/operations/table_engines/url.md b/docs/en/engines/table_engines/special/url.md
similarity index 86%
rename from docs/en/operations/table_engines/url.md
rename to docs/en/engines/table_engines/special/url.md
index 060c86f4c76..3c3e8d45f4c 100644
--- a/docs/en/operations/table_engines/url.md
+++ b/docs/en/engines/table_engines/special/url.md
@@ -1,13 +1,18 @@
+---
+toc_priority: 41
+toc_title: URL
+---
+
# URL(URL, Format) {#table_engines-url}
Manages data on a remote HTTP/HTTPS server. This engine is similar
to the [File](file.md) engine.
-## Using the engine in the ClickHouse server {#using-the-engine-in-the-clickhouse-server}
+## Using the Engine In the Clickhouse Server {#using-the-engine-in-the-clickhouse-server}
The `format` must be one that ClickHouse can use in
`SELECT` queries and, if necessary, in `INSERTs`. For the full list of supported formats, see
-[Formats](../../interfaces/formats.md#formats).
+[Formats](../../../interfaces/formats.md#formats).
The `URL` must conform to the structure of a Uniform Resource Locator. The specified URL must point to a server
that uses HTTP or HTTPS. This does not require any
@@ -17,7 +22,7 @@ additional headers for getting a response from the server.
respectively. For processing `POST` requests, the remote server must support
[Chunked transfer encoding](https://en.wikipedia.org/wiki/Chunked_transfer_encoding).
-You can limit the maximum number of HTTP GET redirect hops using the [max\_http\_get\_redirects](../settings/settings.md#setting-max_http_get_redirects) setting.
+You can limit the maximum number of HTTP GET redirect hops using the [max\_http\_get\_redirects](../../../operations/settings/settings.md#setting-max_http_get_redirects) setting.
**Example:**
@@ -64,7 +69,7 @@ SELECT * FROM url_engine_table
└───────┴───────┘
```
-## Details of Implementation {#details-of-implementation}
+## Details Of Implementation {#details-of-implementation}
- Reads and writes can be parallel
- Not supported:
diff --git a/docs/en/operations/table_engines/view.md b/docs/en/engines/table_engines/special/view.md
similarity index 89%
rename from docs/en/operations/table_engines/view.md
rename to docs/en/engines/table_engines/special/view.md
index 516a4491be2..f5d74795dec 100644
--- a/docs/en/operations/table_engines/view.md
+++ b/docs/en/engines/table_engines/special/view.md
@@ -1,3 +1,8 @@
+---
+toc_priority: 42
+toc_title: View
+---
+
# View {#table_engines-view}
Used for implementing views (for more information, see the `CREATE VIEW query`). It does not store data, but only stores the specified `SELECT` query. When reading from a table, it runs this query (and deletes all unnecessary columns from the query).
diff --git a/docs/en/faq/general.md b/docs/en/faq/general.md
index 058a2e5394f..5d6fe1e7014 100644
--- a/docs/en/faq/general.md
+++ b/docs/en/faq/general.md
@@ -1,3 +1,8 @@
+---
+toc_priority: 78
+toc_title: General Questions
+---
+
# General Questions {#general-questions}
## Why Not Use Something Like MapReduce? {#why-not-use-something-like-mapreduce}
@@ -40,7 +45,7 @@ SELECT * FROM table INTO OUTFILE 'file' FORMAT CSV
### Using a File-Engine Table {#using-a-file-engine-table}
-See [File](../operations/table_engines/file.md).
+See [File](../engines/table_engines/special/file.md).
### Using Command-Line Redirection {#using-command-line-redirection}
diff --git a/docs/en/faq/index.md b/docs/en/faq/index.md
new file mode 100644
index 00000000000..2f872ea45b0
--- /dev/null
+++ b/docs/en/faq/index.md
@@ -0,0 +1,5 @@
+---
+toc_folder_title: F.A.Q.
+toc_priority: 76
+---
+
diff --git a/docs/en/getting_started/example_datasets/amplab_benchmark.md b/docs/en/getting_started/example_datasets/amplab_benchmark.md
index 79c7a61f5cb..95fe444a897 100644
--- a/docs/en/getting_started/example_datasets/amplab_benchmark.md
+++ b/docs/en/getting_started/example_datasets/amplab_benchmark.md
@@ -1,3 +1,8 @@
+---
+toc_priority: 17
+toc_title: AMPLab Big Data Benchmark
+---
+
# AMPLab Big Data Benchmark {#amplab-big-data-benchmark}
See https://amplab.cs.berkeley.edu/benchmark/
diff --git a/docs/en/getting_started/example_datasets/criteo.md b/docs/en/getting_started/example_datasets/criteo.md
index a1bca49597b..a38a1b14210 100644
--- a/docs/en/getting_started/example_datasets/criteo.md
+++ b/docs/en/getting_started/example_datasets/criteo.md
@@ -1,3 +1,8 @@
+---
+toc_priority: 19
+toc_title: Terabyte Click Logs from Criteo
+---
+
# Terabyte of Click Logs from Criteo {#terabyte-of-click-logs-from-criteo}
Download the data from http://labs.criteo.com/downloads/download-terabyte-click-logs/
diff --git a/docs/en/getting_started/example_datasets/index.md b/docs/en/getting_started/example_datasets/index.md
index 4542868d7f0..a94dff3cef8 100644
--- a/docs/en/getting_started/example_datasets/index.md
+++ b/docs/en/getting_started/example_datasets/index.md
@@ -1,3 +1,9 @@
+---
+toc_folder_title: Example Datasets
+toc_priority: 12
+toc_title: Introduction
+---
+
# Example Datasets
This section describes how to obtain example datasets and import them into ClickHouse.
diff --git a/docs/en/getting_started/example_datasets/metrica.md b/docs/en/getting_started/example_datasets/metrica.md
index d4373948c03..366531fa8e7 100644
--- a/docs/en/getting_started/example_datasets/metrica.md
+++ b/docs/en/getting_started/example_datasets/metrica.md
@@ -1,3 +1,8 @@
+---
+toc_priority: 21
+toc_title: Yandex.Metrica Data
+---
+
# Anonymized Yandex.Metrica Data {#anonymized-yandex-metrica-data}
Dataset consists of two tables containing anonymized data about hits (`hits_v1`) and visits (`visits_v1`) of Yandex.Metrica. You can read more about Yandex.Metrica in [ClickHouse history](../../introduction/history.md) section.
diff --git a/docs/en/getting_started/example_datasets/nyc_taxi.md b/docs/en/getting_started/example_datasets/nyc_taxi.md
index 84a24717024..6c7b274e7d4 100644
--- a/docs/en/getting_started/example_datasets/nyc_taxi.md
+++ b/docs/en/getting_started/example_datasets/nyc_taxi.md
@@ -1,3 +1,8 @@
+---
+toc_priority: 16
+toc_title: New York Taxi Data
+---
+
# New York Taxi Data {#new-york-taxi-data}
This dataset can be obtained in two ways:
@@ -375,7 +380,7 @@ We ran queries using a client located in a Yandex datacenter in Finland on a clu
## Summary {#summary}
| servers | Q1 | Q2 | Q3 | Q4 |
-|---------|-------|-------|-------|-------|
+|-------|-----|-----|-----|-----|
| 1 | 0.490 | 1.224 | 2.104 | 3.593 |
| 3 | 0.212 | 0.438 | 0.733 | 1.241 |
| 140 | 0.028 | 0.043 | 0.051 | 0.072 |
diff --git a/docs/en/getting_started/example_datasets/ontime.md b/docs/en/getting_started/example_datasets/ontime.md
index ebad5d98241..11c9f595eab 100644
--- a/docs/en/getting_started/example_datasets/ontime.md
+++ b/docs/en/getting_started/example_datasets/ontime.md
@@ -1,3 +1,8 @@
+---
+toc_priority: 15
+toc_title: OnTime
+---
+
# OnTime {#ontime}
This dataset can be obtained in two ways:
diff --git a/docs/en/getting_started/example_datasets/star_schema.md b/docs/en/getting_started/example_datasets/star_schema.md
index 6763ec63b61..7940807ee49 100644
--- a/docs/en/getting_started/example_datasets/star_schema.md
+++ b/docs/en/getting_started/example_datasets/star_schema.md
@@ -1,3 +1,8 @@
+---
+toc_priority: 20
+toc_title: Star Schema Benchmark
+---
+
# Star Schema Benchmark {#star-schema-benchmark}
Compiling dbgen:
diff --git a/docs/en/getting_started/example_datasets/wikistat.md b/docs/en/getting_started/example_datasets/wikistat.md
index c1448284b82..874af8c9423 100644
--- a/docs/en/getting_started/example_datasets/wikistat.md
+++ b/docs/en/getting_started/example_datasets/wikistat.md
@@ -1,3 +1,8 @@
+---
+toc_priority: 18
+toc_title: WikiStat
+---
+
# WikiStat {#wikistat}
See: http://dumps.wikimedia.org/other/pagecounts-raw/
diff --git a/docs/en/getting_started/index.md b/docs/en/getting_started/index.md
index f9e6c4bd5d3..61eab2f512b 100644
--- a/docs/en/getting_started/index.md
+++ b/docs/en/getting_started/index.md
@@ -1,3 +1,10 @@
+---
+toc_folder_title: Getting Started
+toc_hidden: true
+toc_priority: 8
+toc_title: hidden
+---
+
# Getting Started {#getting-started}
If you are new to ClickHouse and want to get a hands-on feeling of its performance, first of all, you need to go through the [installation process](install.md). After that you can:
diff --git a/docs/en/getting_started/install.md b/docs/en/getting_started/install.md
index 3af47b81c88..363fc928679 100644
--- a/docs/en/getting_started/install.md
+++ b/docs/en/getting_started/install.md
@@ -1,3 +1,8 @@
+---
+toc_priority: 11
+toc_title: Installation
+---
+
# Installation {#installation}
## System Requirements {#system-requirements}
diff --git a/docs/en/getting_started/playground.md b/docs/en/getting_started/playground.md
index b109aca7ff3..d7453e51eaa 100644
--- a/docs/en/getting_started/playground.md
+++ b/docs/en/getting_started/playground.md
@@ -1,3 +1,8 @@
+---
+toc_priority: 14
+toc_title: Playground
+---
+
# ClickHouse Playground {#clickhouse-playground}
[ClickHouse Playground](https://play.clickhouse.tech?file=welcome) allows people to experiment with ClickHouse by running queries instantly, without setting up their server or cluster.
@@ -27,7 +32,7 @@ You can make queries to playground using any HTTP client, for example [curl](htt
More information about software products that support ClickHouse is available [here](../interfaces/index.md).
| Parameter | Value |
-|:----------|:--------------------------------------|
+|:------|:------------------------|
| Endpoint | https://play-api.clickhouse.tech:8443 |
| User | `playground` |
| Password | `clickhouse` |
diff --git a/docs/en/getting_started/tutorial.md b/docs/en/getting_started/tutorial.md
index 2a64bd1e486..60a912f253a 100644
--- a/docs/en/getting_started/tutorial.md
+++ b/docs/en/getting_started/tutorial.md
@@ -1,3 +1,8 @@
+---
+toc_priority: 12
+toc_title: Tutorial
+---
+
# ClickHouse Tutorial {#clickhouse-tutorial}
## What to Expect from This Tutorial? {#what-to-expect-from-this-tutorial}
@@ -99,11 +104,11 @@ As in most databases management systems, ClickHouse logically groups tables into
clickhouse-client --query "CREATE DATABASE IF NOT EXISTS tutorial"
```
-Syntax for creating tables is way more complicated compared to databases (see [reference](../query_language/create.md). In general `CREATE TABLE` statement has to specify three key things:
+Syntax for creating tables is way more complicated compared to databases (see [reference](../sql_reference/statements/create.md). In general `CREATE TABLE` statement has to specify three key things:
1. Name of table to create.
-2. Table schema, i.e. list of columns and their [data types](../data_types/index.md).
-3. [Table engine](../operations/table_engines/index.md) and it’s settings, which determines all the details on how queries to this table will be physically executed.
+2. Table schema, i.e. list of columns and their [data types](../sql_reference/data_types/index.md).
+3. [Table engine](../engines/table_engines/index.md) and it’s settings, which determines all the details on how queries to this table will be physically executed.
Yandex.Metrica is a web analytics service, and sample dataset doesn’t cover its full functionality, so there are only two tables to create:
@@ -455,11 +460,11 @@ SETTINGS index_granularity = 8192
You can execute those queries using the interactive mode of `clickhouse-client` (just launch it in a terminal without specifying a query in advance) or try some [alternative interface](../interfaces/index.md) if you want.
-As we can see, `hits_v1` uses the [basic MergeTree engine](../operations/table_engines/mergetree.md), while the `visits_v1` uses the [Collapsing](../operations/table_engines/collapsingmergetree.md) variant.
+As we can see, `hits_v1` uses the [basic MergeTree engine](../engines/table_engines/mergetree_family/mergetree.md), while the `visits_v1` uses the [Collapsing](../engines/table_engines/mergetree_family/collapsingmergetree.md) variant.
### Import Data {#import-data}
-Data import to ClickHouse is done via [INSERT INTO](../query_language/insert_into.md) query like in many other SQL databases. However, data is usually provided in one of the [supported serialization formats](../interfaces/formats.md) instead of `VALUES` clause (which is also supported).
+Data import to ClickHouse is done via [INSERT INTO](../sql_reference/statements/insert_into.md) query like in many other SQL databases. However, data is usually provided in one of the [supported serialization formats](../interfaces/formats.md) instead of `VALUES` clause (which is also supported).
The files we downloaded earlier are in tab-separated format, so here’s how to import them via console client:
@@ -524,9 +529,9 @@ ClickHouse cluster is a homogenous cluster. Steps to set up:
1. Install ClickHouse server on all machines of the cluster
2. Set up cluster configs in configuration files
3. Create local tables on each instance
-4. Create a [Distributed table](../operations/table_engines/distributed.md)
+4. Create a [Distributed table](../engines/table_engines/special/distributed.md)
-[Distributed table](../operations/table_engines/distributed.md) is actually a kind of “view” to local tables of ClickHouse cluster. SELECT query from a distributed table executes using resources of all cluster’s shards. You may specify configs for multiple clusters and create multiple distributed tables providing views to different clusters.
+[Distributed table](../engines/table_engines/special/distributed.md) is actually a kind of “view” to local tables of ClickHouse cluster. SELECT query from a distributed table executes using resources of all cluster’s shards. You may specify configs for multiple clusters and create multiple distributed tables providing views to different clusters.
Example config for a cluster with three shards, one replica each:
@@ -568,16 +573,16 @@ CREATE TABLE tutorial.hits_all AS tutorial.hits_local
ENGINE = Distributed(perftest_3shards_1replicas, tutorial, hits_local, rand());
```
-A common practice is to create similar Distributed tables on all machines of the cluster. It allows running distributed queries on any machine of the cluster. Also there’s an alternative option to create temporary distributed table for a given SELECT query using [remote](../query_language/table_functions/remote.md) table function.
+A common practice is to create similar Distributed tables on all machines of the cluster. It allows running distributed queries on any machine of the cluster. Also there’s an alternative option to create temporary distributed table for a given SELECT query using [remote](../sql_reference/table_functions/remote.md) table function.
-Let’s run [INSERT SELECT](../query_language/insert_into.md) into the Distributed table to spread the table to multiple servers.
+Let’s run [INSERT SELECT](../sql_reference/statements/insert_into.md) into the Distributed table to spread the table to multiple servers.
``` sql
INSERT INTO tutorial.hits_all SELECT * FROM tutorial.hits_v1;
```
!!! warning "Notice"
- This approach is not suitable for the sharding of large tables. There’s a separate tool [clickhouse-copier](../operations/utils/clickhouse-copier.md) that can re-shard arbitrary large tables.
+ This approach is not suitable for the sharding of large tables. There’s a separate tool [clickhouse-copier](../operations/utilities/clickhouse-copier.md) that can re-shard arbitrary large tables.
As you could expect, computationally heavy queries run N times faster if they utilize 3 servers instead of one.
@@ -653,7 +658,7 @@ ENGINE = ReplcatedMergeTree(
...
```
-Here we use [ReplicatedMergeTree](../operations/table_engines/replication.md) table engine. In parameters we specify ZooKeeper path containing shard and replica identifiers.
+Here we use [ReplicatedMergeTree](../engines/table_engines/mergetree_family/replication.md) table engine. In parameters we specify ZooKeeper path containing shard and replica identifiers.
``` sql
INSERT INTO tutorial.hits_replica SELECT * FROM tutorial.hits_local;
diff --git a/docs/en/guides/apply_catboost_model.md b/docs/en/guides/apply_catboost_model.md
index bddde92d46d..89942a00359 100644
--- a/docs/en/guides/apply_catboost_model.md
+++ b/docs/en/guides/apply_catboost_model.md
@@ -1,3 +1,8 @@
+---
+toc_priority: 41
+toc_title: Applying CatBoost Models
+---
+
# Applying a Catboost Model in ClickHouse {#applying-catboost-model-in-clickhouse}
[CatBoost](https://catboost.ai) is a free and open-source gradient boosting library developed at [Yandex](https://yandex.com/company/) for machine learning.
@@ -109,7 +114,7 @@ FROM amazon_train
+-count()-+
| 65538 |
-+---------+
++-------+
```
## 3. Integrate CatBoost into ClickHouse {#integrate-catboost-into-clickhouse}
@@ -178,7 +183,7 @@ LIMIT 10
```
!!! note "Note"
- Function [modelEvaluate](../query_language/functions/other_functions.md#function-modelevaluate) returns tuple with per-class raw predictions for multiclass models.
+ Function [modelEvaluate](../sql_reference/functions/other_functions.md#function-modelevaluate) returns tuple with per-class raw predictions for multiclass models.
Let’s predict the probability:
@@ -201,7 +206,7 @@ LIMIT 10
```
!!! note "Note"
- More info about [exp()](../query_language/functions/math_functions.md) function.
+ More info about [exp()](../sql_reference/functions/math_functions.md) function.
Let’s calculate LogLoss on the sample:
@@ -227,6 +232,6 @@ FROM
```
!!! note "Note"
- More info about [avg()](../query_language/agg_functions/reference.md#agg_function-avg) and [log()](../query_language/functions/math_functions.md) functions.
+ More info about [avg()](../sql_reference/aggregate_functions/reference.md#agg_function-avg) and [log()](../sql_reference/functions/math_functions.md) functions.
[Original article](https://clickhouse.tech/docs/en/guides/apply_catboost_model/)
diff --git a/docs/en/guides/index.md b/docs/en/guides/index.md
index f08471a605c..e18c5fac223 100644
--- a/docs/en/guides/index.md
+++ b/docs/en/guides/index.md
@@ -1,3 +1,9 @@
+---
+toc_folder_title: Guides
+toc_priority: 38
+toc_title: Overview
+---
+
# ClickHouse Guides {#clickhouse-guides}
List of detailed step-by-step instructions that help to solve various tasks using ClickHouse:
diff --git a/docs/en/index.md b/docs/en/index.md
index 705f5165870..5d40ae9177c 100644
--- a/docs/en/index.md
+++ b/docs/en/index.md
@@ -1,3 +1,8 @@
+---
+toc_priority: 3
+toc_title: Overview
+---
+
# What is ClickHouse? {#what-is-clickhouse}
ClickHouse is a column-oriented database management system (DBMS) for online analytical processing of queries (OLAP).
@@ -5,7 +10,7 @@ ClickHouse is a column-oriented database management system (DBMS) for online ana
In a “normal” row-oriented DBMS, data is stored in this order:
| Row | WatchID | JavaEnable | Title | GoodEvent | EventTime |
-|-----|-------------|------------|--------------------|-----------|---------------------|
+|---|---------|--------|------------|-------|-------------|
| \#0 | 89354350662 | 1 | Investor Relations | 1 | 2016-05-18 05:19:20 |
| \#1 | 90329509958 | 0 | Contact us | 1 | 2016-05-18 08:10:20 |
| \#2 | 89953706054 | 1 | Mission | 1 | 2016-05-18 07:38:00 |
@@ -18,7 +23,7 @@ Examples of a row-oriented DBMS are MySQL, Postgres, and MS SQL Server.
In a column-oriented DBMS, data is stored like this:
| Row: | \#0 | \#1 | \#2 | \#N |
-|-------------|---------------------|---------------------|---------------------|-----|
+|---------|-------------|-------------|-------------|---|
| WatchID: | 89354350662 | 90329509958 | 89953706054 | … |
| JavaEnable: | 1 | 0 | 1 | … |
| Title: | Investor Relations | Contact us | Mission | … |
diff --git a/docs/en/interfaces/cli.md b/docs/en/interfaces/cli.md
index bb7e52461ef..5bad73008a6 100644
--- a/docs/en/interfaces/cli.md
+++ b/docs/en/interfaces/cli.md
@@ -1,3 +1,8 @@
+---
+toc_priority: 17
+toc_title: Command-Line Client
+---
+
# Command-line Client {#command-line-client}
ClickHouse provides a native command-line client: `clickhouse-client`. The client supports command-line options and configuration files. For more information, see [Configuring](#interfaces_cli_configuration).
@@ -82,7 +87,7 @@ Format a query as usual, then place the values that you want to pass from the ap
```
- `name` — Placeholder identifier. In the console client it should be used in app parameters as `--param_ = value`.
-- `data type` — [Data type](../data_types/index.md) of the app parameter value. For example, a data structure like `(integer, ('string', integer))` can have the `Tuple(UInt8, Tuple(String, UInt8))` data type (you can also use another [integer](../data_types/int_uint.md) types).
+- `data type` — [Data type](../sql_reference/data_types/index.md) of the app parameter value. For example, a data structure like `(integer, ('string', integer))` can have the `Tuple(UInt8, Tuple(String, UInt8))` data type (you can also use another [integer](../sql_reference/data_types/int_uint.md) types).
#### Example {#example}
diff --git a/docs/en/interfaces/cpp.md b/docs/en/interfaces/cpp.md
index 71880c31635..479b0ac87d9 100644
--- a/docs/en/interfaces/cpp.md
+++ b/docs/en/interfaces/cpp.md
@@ -1,3 +1,8 @@
+---
+toc_priority: 24
+toc_title: C++ Client Library
+---
+
# C++ Client Library {#c-client-library}
See README at [clickhouse-cpp](https://github.com/ClickHouse/clickhouse-cpp) repository.
diff --git a/docs/en/interfaces/formats.md b/docs/en/interfaces/formats.md
index 44b2af272f8..05cffd40cf5 100644
--- a/docs/en/interfaces/formats.md
+++ b/docs/en/interfaces/formats.md
@@ -1,3 +1,8 @@
+---
+toc_priority: 21
+toc_title: Input and Output Formats
+---
+
# Formats for Input and Output Data {#formats}
ClickHouse can accept and return data in various formats. A format supported for input can be used to parse the data provided to `INSERT`s, to perform `SELECT`s from a file-backed table such as File, URL or HDFS, or to read an external dictionary. A format supported for output can be used to arrange the
@@ -6,7 +11,7 @@ results of a `SELECT`, and to perform `INSERT`s into a file-backed table.
The supported formats are:
| Format | Input | Output |
-|-----------------------------------------------------------------|-------|--------|
+|---------------------------------------|-----|------|
| [TabSeparated](#tabseparated) | ✔ | ✔ |
| [TabSeparatedRaw](#tabseparatedraw) | ✗ | ✔ |
| [TabSeparatedWithNames](#tabseparatedwithnames) | ✔ | ✔ |
@@ -103,9 +108,9 @@ Only a small set of symbols are escaped. You can easily stumble onto a string va
Arrays are written as a list of comma-separated values in square brackets. Number items in the array are formatted as normally. `Date` and `DateTime` types are written in single quotes. Strings are written in single quotes with the same escaping rules as above.
-[NULL](../query_language/syntax.md) is formatted as `\N`.
+[NULL](../sql_reference/syntax.md) is formatted as `\N`.
-Each element of [Nested](../data_types/nested_data_structures/nested.md) structures is represented as array.
+Each element of [Nested](../sql_reference/data_types/nested_data_structures/nested.md) structures is represented as array.
For example:
@@ -325,7 +330,7 @@ SearchPhrase=curtain designs count()=1064
SearchPhrase=baku count()=1000
```
-[NULL](../query_language/syntax.md) is formatted as `\N`.
+[NULL](../sql_reference/syntax.md) is formatted as `\N`.
``` sql
SELECT * FROM t_null FORMAT TSKV
@@ -457,7 +462,7 @@ If the query contains GROUP BY, rows\_before\_limit\_at\_least is the exact numb
This format is only appropriate for outputting a query result, but not for parsing (retrieving data to insert in a table).
-ClickHouse supports [NULL](../query_language/syntax.md), which is displayed as `null` in the JSON output.
+ClickHouse supports [NULL](../sql_reference/syntax.md), which is displayed as `null` in the JSON output.
See also the [JSONEachRow](#jsoneachrow) format.
@@ -534,7 +539,7 @@ ClickHouse ignores spaces between elements and commas after the objects. You can
**Omitted values processing**
-ClickHouse substitutes omitted values with the default values for the corresponding [data types](../data_types/index.md).
+ClickHouse substitutes omitted values with the default values for the corresponding [data types](../sql_reference/data_types/index.md).
If `DEFAULT expr` is specified, ClickHouse uses different substitution rules depending on the [input\_format\_defaults\_for\_omitted\_fields](../operations/settings/settings.md#session_settings-input_format_defaults_for_omitted_fields) setting.
@@ -579,7 +584,7 @@ Unlike the [JSON](#json) format, there is no substitution of invalid UTF-8 seque
### Usage of Nested Structures {#jsoneachrow-nested}
-If you have a table with [Nested](../data_types/nested_data_structures/nested.md) data type columns, you can insert JSON data with the same structure. Enable this feature with the [input\_format\_import\_nested\_json](../operations/settings/settings.md#settings-input_format_import_nested_json) setting.
+If you have a table with [Nested](../sql_reference/data_types/nested_data_structures/nested.md) data type columns, you can insert JSON data with the same structure. Enable this feature with the [input\_format\_import\_nested\_json](../operations/settings/settings.md#settings-input_format_import_nested_json) setting.
For example, consider the following table:
@@ -653,7 +658,7 @@ Outputs data as Unicode-art tables, also using ANSI-escape sequences for setting
A full grid of the table is drawn, and each row occupies two lines in the terminal.
Each result block is output as a separate table. This is necessary so that blocks can be output without buffering results (buffering would be necessary in order to pre-calculate the visible width of all the values).
-[NULL](../query_language/syntax.md) is output as `ᴺᵁᴸᴸ`.
+[NULL](../sql_reference/syntax.md) is output as `ᴺᵁᴸᴸ`.
Example (shown for the [PrettyCompact](#prettycompact) format):
@@ -757,7 +762,7 @@ FixedString is represented simply as a sequence of bytes.
Array is represented as a varint length (unsigned [LEB128](https://en.wikipedia.org/wiki/LEB128)), followed by successive elements of the array.
-For [NULL](../query_language/syntax.md#null-literal) support, an additional byte containing 1 or 0 is added before each [Nullable](../data_types/nullable.md) value. If 1, then the value is `NULL` and this byte is interpreted as a separate value. If 0, the value after the byte is not `NULL`.
+For [NULL](../sql_reference/syntax.md#null-literal) support, an additional byte containing 1 or 0 is added before each [Nullable](../sql_reference/data_types/nullable.md) value. If 1, then the value is `NULL` and this byte is interpreted as a separate value. If 0, the value after the byte is not `NULL`.
## RowBinaryWithNamesAndTypes {#rowbinarywithnamesandtypes}
@@ -769,7 +774,7 @@ Similar to [RowBinary](#rowbinary), but with added header:
## Values {#data-format-values}
-Prints every row in brackets. Rows are separated by commas. There is no comma after the last row. The values inside the brackets are also comma-separated. Numbers are output in a decimal format without quotes. Arrays are output in square brackets. Strings, dates, and dates with times are output in quotes. Escaping rules and parsing are similar to the [TabSeparated](#tabseparated) format. During formatting, extra spaces aren’t inserted, but during parsing, they are allowed and skipped (except for spaces inside array values, which are not allowed). [NULL](../query_language/syntax.md) is represented as `NULL`.
+Prints every row in brackets. Rows are separated by commas. There is no comma after the last row. The values inside the brackets are also comma-separated. Numbers are output in a decimal format without quotes. Arrays are output in square brackets. Strings, dates, and dates with times are output in quotes. Escaping rules and parsing are similar to the [TabSeparated](#tabseparated) format. During formatting, extra spaces aren’t inserted, but during parsing, they are allowed and skipped (except for spaces inside array values, which are not allowed). [NULL](../sql_reference/syntax.md) is represented as `NULL`.
The minimum set of characters that you need to escape when passing data in Values format: single quotes and backslashes.
@@ -781,7 +786,7 @@ See also: [input\_format\_values\_interpret\_expressions](../operations/settings
Prints each value on a separate line with the column name specified. This format is convenient for printing just one or a few rows if each row consists of a large number of columns.
-[NULL](../query_language/syntax.md) is output as `ᴺᵁᴸᴸ`.
+[NULL](../sql_reference/syntax.md) is output as `ᴺᵁᴸᴸ`.
Example:
@@ -960,7 +965,7 @@ message MessageType {
```
ClickHouse tries to find a column named `x.y.z` (or `x_y_z` or `X.y_Z` and so on).
-Nested messages are suitable to input or output a [nested data structures](../data_types/nested_data_structures/nested.md).
+Nested messages are suitable to input or output a [nested data structures](../sql_reference/data_types/nested_data_structures/nested.md).
Default values defined in a protobuf schema like this
@@ -972,7 +977,7 @@ message MessageType {
}
```
-are not applied; the [table defaults](../query_language/create.md#create-default-values) are used instead of them.
+are not applied; the [table defaults](../sql_reference/statements/create.md#create-default-values) are used instead of them.
ClickHouse inputs and outputs protobuf messages in the `length-delimited` format.
It means before every message should be written its length as a [varint](https://developers.google.com/protocol-buffers/docs/encoding#varints).
@@ -986,23 +991,23 @@ ClickHouse Avro format supports reading and writing [Avro data files](http://avr
### Data Types Matching {#data_types-matching}
-The table below shows supported data types and how they match ClickHouse [data types](../data_types/index.md) in `INSERT` and `SELECT` queries.
+The table below shows supported data types and how they match ClickHouse [data types](../sql_reference/data_types/index.md) in `INSERT` and `SELECT` queries.
| Avro data type `INSERT` | ClickHouse data type | Avro data type `SELECT` |
-|---------------------------------------------|-------------------------------------------------------------------------------------------|------------------------------|
-| `boolean`, `int`, `long`, `float`, `double` | [Int(8\|16\|32)](../data_types/int_uint.md), [UInt(8\|16\|32)](../data_types/int_uint.md) | `int` |
-| `boolean`, `int`, `long`, `float`, `double` | [Int64](../data_types/int_uint.md), [UInt64](../data_types/int_uint.md) | `long` |
-| `boolean`, `int`, `long`, `float`, `double` | [Float32](../data_types/float.md) | `float` |
-| `boolean`, `int`, `long`, `float`, `double` | [Float64](../data_types/float.md) | `double` |
-| `bytes`, `string`, `fixed`, `enum` | [String](../data_types/string.md) | `bytes` |
-| `bytes`, `string`, `fixed` | [FixedString(N)](../data_types/fixedstring.md) | `fixed(N)` |
-| `enum` | [Enum(8\|16)](../data_types/enum.md) | `enum` |
-| `array(T)` | [Array(T)](../data_types/array.md) | `array(T)` |
-| `union(null, T)`, `union(T, null)` | [Nullable(T)](../data_types/date.md) | `union(null, T)` |
-| `null` | [Nullable(Nothing)](../data_types/special_data_types/nothing.md) | `null` |
-| `int (date)` \* | [Date](../data_types/date.md) | `int (date)` \* |
-| `long (timestamp-millis)` \* | [DateTime64(3)](../data_types/datetime.md) | `long (timestamp-millis)` \* |
-| `long (timestamp-micros)` \* | [DateTime64(6)](../data_types/datetime.md) | `long (timestamp-micros)` \* |
+|---------------------------|-------------------------------------------------------|------------------|
+| `boolean`, `int`, `long`, `float`, `double` | [Int(8\|16\|32)](../sql_reference/data_types/int_uint.md), [UInt(8\|16\|32)](../sql_reference/data_types/int_uint.md) | `int` |
+| `boolean`, `int`, `long`, `float`, `double` | [Int64](../sql_reference/data_types/int_uint.md), [UInt64](../sql_reference/data_types/int_uint.md) | `long` |
+| `boolean`, `int`, `long`, `float`, `double` | [Float32](../sql_reference/data_types/float.md) | `float` |
+| `boolean`, `int`, `long`, `float`, `double` | [Float64](../sql_reference/data_types/float.md) | `double` |
+| `bytes`, `string`, `fixed`, `enum` | [String](../sql_reference/data_types/string.md) | `bytes` |
+| `bytes`, `string`, `fixed` | [FixedString(N)](../sql_reference/data_types/fixedstring.md) | `fixed(N)` |
+| `enum` | [Enum(8\|16)](../sql_reference/data_types/enum.md) | `enum` |
+| `array(T)` | [Array(T)](../sql_reference/data_types/array.md) | `array(T)` |
+| `union(null, T)`, `union(T, null)` | [Nullable(T)](../sql_reference/data_types/date.md) | `union(null, T)` |
+| `null` | [Nullable(Nothing)](../sql_reference/data_types/special_data_types/nothing.md) | `null` |
+| `int (date)` \* | [Date](../sql_reference/data_types/date.md) | `int (date)` \* |
+| `long (timestamp-millis)` \* | [DateTime64(3)](../sql_reference/data_types/datetime.md) | `long (timestamp-millis)` \* |
+| `long (timestamp-micros)` \* | [DateTime64(6)](../sql_reference/data_types/datetime.md) | `long (timestamp-micros)` \* |
\* [Avro logical types](http://avro.apache.org/docs/current/spec.html#Logical+Types)
@@ -1056,7 +1061,7 @@ Same as [Avro](#data-format-avro)
### Usage {#usage}
-To quickly verify schema resolution you can use [kafkacat](https://github.com/edenhill/kafkacat) with [clickhouse-local](../operations/utils/clickhouse-local.md):
+To quickly verify schema resolution you can use [kafkacat](https://github.com/edenhill/kafkacat) with [clickhouse-local](../operations/utilities/clickhouse-local.md):
``` bash
$ kafkacat -b kafka-broker -C -t topic1 -o beginning -f '%s' -c 3 | clickhouse-local --input-format AvroConfluent --format_avro_schema_registry_url 'http://schema-registry' -S "field1 Int64, field2 String" -q 'select * from table'
@@ -1065,7 +1070,7 @@ $ kafkacat -b kafka-broker -C -t topic1 -o beginning -f '%s' -c 3 | clickhouse-
3 c
```
-To use `AvroConfluent` with [Kafka](../operations/table_engines/kafka.md):
+To use `AvroConfluent` with [Kafka](../engines/table_engines/integrations/kafka.md):
``` sql
CREATE TABLE topic1_stream
@@ -1094,25 +1099,25 @@ SELECT * FROM topic1_stream;
### Data Types Matching {#data_types-matching-2}
-The table below shows supported data types and how they match ClickHouse [data types](../data_types/index.md) in `INSERT` and `SELECT` queries.
+The table below shows supported data types and how they match ClickHouse [data types](../sql_reference/data_types/index.md) in `INSERT` and `SELECT` queries.
| Parquet data type (`INSERT`) | ClickHouse data type | Parquet data type (`SELECT`) |
-|------------------------------|---------------------------------------------|------------------------------|
-| `UINT8`, `BOOL` | [UInt8](../data_types/int_uint.md) | `UINT8` |
-| `INT8` | [Int8](../data_types/int_uint.md) | `INT8` |
-| `UINT16` | [UInt16](../data_types/int_uint.md) | `UINT16` |
-| `INT16` | [Int16](../data_types/int_uint.md) | `INT16` |
-| `UINT32` | [UInt32](../data_types/int_uint.md) | `UINT32` |
-| `INT32` | [Int32](../data_types/int_uint.md) | `INT32` |
-| `UINT64` | [UInt64](../data_types/int_uint.md) | `UINT64` |
-| `INT64` | [Int64](../data_types/int_uint.md) | `INT64` |
-| `FLOAT`, `HALF_FLOAT` | [Float32](../data_types/float.md) | `FLOAT` |
-| `DOUBLE` | [Float64](../data_types/float.md) | `DOUBLE` |
-| `DATE32` | [Date](../data_types/date.md) | `UINT16` |
-| `DATE64`, `TIMESTAMP` | [DateTime](../data_types/datetime.md) | `UINT32` |
-| `STRING`, `BINARY` | [String](../data_types/string.md) | `STRING` |
-| — | [FixedString](../data_types/fixedstring.md) | `STRING` |
-| `DECIMAL` | [Decimal](../data_types/decimal.md) | `DECIMAL` |
+|------------------|---------------------------|------------------|
+| `UINT8`, `BOOL` | [UInt8](../sql_reference/data_types/int_uint.md) | `UINT8` |
+| `INT8` | [Int8](../sql_reference/data_types/int_uint.md) | `INT8` |
+| `UINT16` | [UInt16](../sql_reference/data_types/int_uint.md) | `UINT16` |
+| `INT16` | [Int16](../sql_reference/data_types/int_uint.md) | `INT16` |
+| `UINT32` | [UInt32](../sql_reference/data_types/int_uint.md) | `UINT32` |
+| `INT32` | [Int32](../sql_reference/data_types/int_uint.md) | `INT32` |
+| `UINT64` | [UInt64](../sql_reference/data_types/int_uint.md) | `UINT64` |
+| `INT64` | [Int64](../sql_reference/data_types/int_uint.md) | `INT64` |
+| `FLOAT`, `HALF_FLOAT` | [Float32](../sql_reference/data_types/float.md) | `FLOAT` |
+| `DOUBLE` | [Float64](../sql_reference/data_types/float.md) | `DOUBLE` |
+| `DATE32` | [Date](../sql_reference/data_types/date.md) | `UINT16` |
+| `DATE64`, `TIMESTAMP` | [DateTime](../sql_reference/data_types/datetime.md) | `UINT32` |
+| `STRING`, `BINARY` | [String](../sql_reference/data_types/string.md) | `STRING` |
+| — | [FixedString](../sql_reference/data_types/fixedstring.md) | `STRING` |
+| `DECIMAL` | [Decimal](../sql_reference/data_types/decimal.md) | `DECIMAL` |
ClickHouse supports configurable precision of `Decimal` type. The `INSERT` query treats the Parquet `DECIMAL` type as the ClickHouse `Decimal128` type.
@@ -1134,7 +1139,7 @@ You can select data from a ClickHouse table and save them into some file in the
$ clickhouse-client --query="SELECT * FROM {some_table} FORMAT Parquet" > {some_file.pq}
```
-To exchange data with Hadoop, you can use [HDFS table engine](../operations/table_engines/hdfs.md).
+To exchange data with Hadoop, you can use [HDFS table engine](../engines/table_engines/integrations/hdfs.md).
## ORC {#data-format-orc}
@@ -1142,24 +1147,24 @@ To exchange data with Hadoop, you can use [HDFS table engine](../operations/tabl
### Data Types Matching {#data_types-matching-3}
-The table below shows supported data types and how they match ClickHouse [data types](../data_types/index.md) in `INSERT` queries.
+The table below shows supported data types and how they match ClickHouse [data types](../sql_reference/data_types/index.md) in `INSERT` queries.
| ORC data type (`INSERT`) | ClickHouse data type |
-|--------------------------|---------------------------------------|
-| `UINT8`, `BOOL` | [UInt8](../data_types/int_uint.md) |
-| `INT8` | [Int8](../data_types/int_uint.md) |
-| `UINT16` | [UInt16](../data_types/int_uint.md) |
-| `INT16` | [Int16](../data_types/int_uint.md) |
-| `UINT32` | [UInt32](../data_types/int_uint.md) |
-| `INT32` | [Int32](../data_types/int_uint.md) |
-| `UINT64` | [UInt64](../data_types/int_uint.md) |
-| `INT64` | [Int64](../data_types/int_uint.md) |
-| `FLOAT`, `HALF_FLOAT` | [Float32](../data_types/float.md) |
-| `DOUBLE` | [Float64](../data_types/float.md) |
-| `DATE32` | [Date](../data_types/date.md) |
-| `DATE64`, `TIMESTAMP` | [DateTime](../data_types/datetime.md) |
-| `STRING`, `BINARY` | [String](../data_types/string.md) |
-| `DECIMAL` | [Decimal](../data_types/decimal.md) |
+|----------------|-------------------------|
+| `UINT8`, `BOOL` | [UInt8](../sql_reference/data_types/int_uint.md) |
+| `INT8` | [Int8](../sql_reference/data_types/int_uint.md) |
+| `UINT16` | [UInt16](../sql_reference/data_types/int_uint.md) |
+| `INT16` | [Int16](../sql_reference/data_types/int_uint.md) |
+| `UINT32` | [UInt32](../sql_reference/data_types/int_uint.md) |
+| `INT32` | [Int32](../sql_reference/data_types/int_uint.md) |
+| `UINT64` | [UInt64](../sql_reference/data_types/int_uint.md) |
+| `INT64` | [Int64](../sql_reference/data_types/int_uint.md) |
+| `FLOAT`, `HALF_FLOAT` | [Float32](../sql_reference/data_types/float.md) |
+| `DOUBLE` | [Float64](../sql_reference/data_types/float.md) |
+| `DATE32` | [Date](../sql_reference/data_types/date.md) |
+| `DATE64`, `TIMESTAMP` | [DateTime](../sql_reference/data_types/datetime.md) |
+| `STRING`, `BINARY` | [String](../sql_reference/data_types/string.md) |
+| `DECIMAL` | [Decimal](../sql_reference/data_types/decimal.md) |
ClickHouse supports configurable precision of the `Decimal` type. The `INSERT` query treats the ORC `DECIMAL` type as the ClickHouse `Decimal128` type.
@@ -1175,7 +1180,7 @@ You can insert ORC data from a file into ClickHouse table by the following comma
$ cat filename.orc | clickhouse-client --query="INSERT INTO some_table FORMAT ORC"
```
-To exchange data with Hadoop, you can use [HDFS table engine](../operations/table_engines/hdfs.md).
+To exchange data with Hadoop, you can use [HDFS table engine](../engines/table_engines/integrations/hdfs.md).
## Format Schema {#formatschema}
@@ -1191,7 +1196,7 @@ can contain an absolute path or a path relative to the current directory on the
If you use the client in the [batch mode](../interfaces/cli.md#cli_usage), the path to the schema must be relative due to security reasons.
If you input or output data via the [HTTP interface](../interfaces/http.md) the file name specified in the format schema
-should be located in the directory specified in [format\_schema\_path](../operations/server_settings/settings.md#server_settings-format_schema_path)
+should be located in the directory specified in [format\_schema\_path](../operations/server_configuration_parameters/settings.md#server_configuration_parameters-format_schema_path)
in the server configuration.
## Skipping Errors {#skippingerrors}
diff --git a/docs/en/interfaces/http.md b/docs/en/interfaces/http.md
index 76f6121f5d0..df2ee5e19e5 100644
--- a/docs/en/interfaces/http.md
+++ b/docs/en/interfaces/http.md
@@ -1,10 +1,15 @@
+---
+toc_priority: 19
+toc_title: HTTP Interface
+---
+
# HTTP Interface {#http-interface}
The HTTP interface lets you use ClickHouse on any platform from any programming language. We use it for working from Java and Perl, as well as shell scripts. In other departments, the HTTP interface is used from Perl, Python, and Go. The HTTP interface is more limited than the native interface, but it has better compatibility.
By default, clickhouse-server listens for HTTP on port 8123 (this can be changed in the config).
-If you make a GET / request without parameters, it returns 200 response code and the string which defined in [http\_server\_default\_response](../operations/server_settings/settings.md#server_settings-http_server_default_response) default value “Ok.” (with a line feed at the end)
+If you make a GET / request without parameters, it returns 200 response code and the string which defined in [http\_server\_default\_response](../operations/server_configuration_parameters/settings.md#server_configuration_parameters-http_server_default_response) default value “Ok.” (with a line feed at the end)
``` bash
$ curl 'http://localhost:8123/'
diff --git a/docs/en/interfaces/index.md b/docs/en/interfaces/index.md
index b40f60a1a55..938f276fe0b 100644
--- a/docs/en/interfaces/index.md
+++ b/docs/en/interfaces/index.md
@@ -1,3 +1,9 @@
+---
+toc_folder_title: Interfaces
+toc_priority: 14
+toc_title: Introduction
+---
+
# Interfaces {#interfaces}
ClickHouse provides two network interfaces (both can be optionally wrapped in TLS for additional security):
diff --git a/docs/en/interfaces/jdbc.md b/docs/en/interfaces/jdbc.md
index af4c2949ee4..ec3fa2fe89a 100644
--- a/docs/en/interfaces/jdbc.md
+++ b/docs/en/interfaces/jdbc.md
@@ -1,3 +1,8 @@
+---
+toc_priority: 22
+toc_title: JDBC Driver
+---
+
# JDBC Driver {#jdbc-driver}
- **[Official driver](https://github.com/ClickHouse/clickhouse-jdbc)**
diff --git a/docs/en/interfaces/mysql.md b/docs/en/interfaces/mysql.md
index 088305e156e..5f469079d54 100644
--- a/docs/en/interfaces/mysql.md
+++ b/docs/en/interfaces/mysql.md
@@ -1,6 +1,11 @@
+---
+toc_priority: 20
+toc_title: MySQL Interface
+---
+
# MySQL interface {#mysql-interface}
-ClickHouse supports MySQL wire protocol. It can be enabled by [mysql\_port](../operations/server_settings/settings.md#server_settings-mysql_port) setting in configuration file:
+ClickHouse supports MySQL wire protocol. It can be enabled by [mysql\_port](../operations/server_configuration_parameters/settings.md#server_configuration_parameters-mysql_port) setting in configuration file:
``` xml
9004
diff --git a/docs/en/interfaces/odbc.md b/docs/en/interfaces/odbc.md
index f2982abf998..4fd7cd23964 100644
--- a/docs/en/interfaces/odbc.md
+++ b/docs/en/interfaces/odbc.md
@@ -1,3 +1,8 @@
+---
+toc_priority: 23
+toc_title: ODBC Driver
+---
+
# ODBC Driver {#odbc-driver}
- [Official driver](https://github.com/ClickHouse/clickhouse-odbc).
diff --git a/docs/en/interfaces/tcp.md b/docs/en/interfaces/tcp.md
index 876ccf12f8a..5d4fc4a58b1 100644
--- a/docs/en/interfaces/tcp.md
+++ b/docs/en/interfaces/tcp.md
@@ -1,3 +1,8 @@
+---
+toc_priority: 18
+toc_title: Native Interface (TCP)
+---
+
# Native Interface (TCP) {#native-interface-tcp}
The native protocol is used in the [command-line client](cli.md), for inter-server communication during distributed query processing, and also in other C++ programs. Unfortunately, native ClickHouse protocol does not have formal specification yet, but it can be reverse-engineered from ClickHouse source code (starting [around here](https://github.com/ClickHouse/ClickHouse/tree/master/dbms/Client)) and/or by intercepting and analyzing TCP traffic.
diff --git a/docs/en/interfaces/third-party/client_libraries.md b/docs/en/interfaces/third-party/client_libraries.md
index 7df61ed0adc..1f5c0e31933 100644
--- a/docs/en/interfaces/third-party/client_libraries.md
+++ b/docs/en/interfaces/third-party/client_libraries.md
@@ -1,3 +1,8 @@
+---
+toc_priority: 26
+toc_title: Client Libraries
+---
+
# Client Libraries from Third-party Developers {#client-libraries-from-third-party-developers}
!!! warning "Disclaimer"
diff --git a/docs/en/interfaces/third-party/gui.md b/docs/en/interfaces/third-party/gui.md
index 98f4bf09140..7b52140e719 100644
--- a/docs/en/interfaces/third-party/gui.md
+++ b/docs/en/interfaces/third-party/gui.md
@@ -1,3 +1,8 @@
+---
+toc_priority: 28
+toc_title: Visual Interfaces
+---
+
# Visual Interfaces from Third-party Developers {#visual-interfaces-from-third-party-developers}
## Open-Source {#open-source}
diff --git a/docs/en/interfaces/third-party/index.md b/docs/en/interfaces/third-party/index.md
new file mode 100644
index 00000000000..16d315d059c
--- /dev/null
+++ b/docs/en/interfaces/third-party/index.md
@@ -0,0 +1,5 @@
+---
+toc_folder_title: Third-Party
+toc_priority: 24
+---
+
diff --git a/docs/en/interfaces/third-party/integrations.md b/docs/en/interfaces/third-party/integrations.md
index 752086d167a..ab84b3f4c19 100644
--- a/docs/en/interfaces/third-party/integrations.md
+++ b/docs/en/interfaces/third-party/integrations.md
@@ -1,3 +1,8 @@
+---
+toc_priority: 27
+toc_title: Integrations
+---
+
# Integration Libraries from Third-party Developers {#integration-libraries-from-third-party-developers}
!!! warning "Disclaimer"
@@ -35,7 +40,7 @@
- [graphouse](https://github.com/yandex/graphouse)
- [carbon-clickhouse](https://github.com/lomik/carbon-clickhouse) +
- [graphite-clickhouse](https://github.com/lomik/graphite-clickhouse)
- - [graphite-ch-optimizer](https://github.com/innogames/graphite-ch-optimizer) - optimizes staled partitions in [\*GraphiteMergeTree](../../operations/table_engines/graphitemergetree.md#graphitemergetree) if rules from [rollup configuration](../../operations/table_engines/graphitemergetree.md#rollup-configuration) could be applied
+ - [graphite-ch-optimizer](https://github.com/innogames/graphite-ch-optimizer) - optimizes staled partitions in [\*GraphiteMergeTree](../../engines/table_engines/mergetree_family/graphitemergetree.md#graphitemergetree) if rules from [rollup configuration](../../engines/table_engines/mergetree_family/graphitemergetree.md#rollup-configuration) could be applied
- [Grafana](https://grafana.com/)
- [clickhouse-grafana](https://github.com/Vertamedia/clickhouse-grafana)
- [Prometheus](https://prometheus.io/)
@@ -72,7 +77,7 @@
- [RClickhouse](https://github.com/IMSMWU/RClickhouse) (uses [clickhouse-cpp](https://github.com/artpaul/clickhouse-cpp))
- Java
- [Hadoop](http://hadoop.apache.org)
- - [clickhouse-hdfs-loader](https://github.com/jaykelin/clickhouse-hdfs-loader) (uses [JDBC](../../query_language/table_functions/jdbc.md))
+ - [clickhouse-hdfs-loader](https://github.com/jaykelin/clickhouse-hdfs-loader) (uses [JDBC](../../sql_reference/table_functions/jdbc.md))
- Scala
- [Akka](https://akka.io)
- [clickhouse-scala-client](https://github.com/crobox/clickhouse-scala-client)
diff --git a/docs/en/interfaces/third-party/proxy.md b/docs/en/interfaces/third-party/proxy.md
index 443a229122a..fc01568e35a 100644
--- a/docs/en/interfaces/third-party/proxy.md
+++ b/docs/en/interfaces/third-party/proxy.md
@@ -1,3 +1,8 @@
+---
+toc_priority: 29
+toc_title: Proxies
+---
+
# Proxy Servers from Third-party Developers {#proxy-servers-from-third-party-developers}
## chproxy {#chproxy}
diff --git a/docs/en/introduction/adopters.md b/docs/en/introduction/adopters.md
index b17514772d3..9a1103c19a5 100644
--- a/docs/en/introduction/adopters.md
+++ b/docs/en/introduction/adopters.md
@@ -1,10 +1,15 @@
+---
+toc_priority: 8
+toc_title: Adopters
+---
+
# ClickHouse Adopters {#clickhouse-adopters}
!!! warning "Disclaimer"
The following list of companies using ClickHouse and their success stories is assembled from public sources, thus might differ from current reality. We’d appreciate it if you share the story of adopting ClickHouse in your company and [add it to the list](https://github.com/ClickHouse/ClickHouse/edit/master/docs/en/introduction/adopters.md), but please make sure you won’t have any NDA issues by doing so. Providing updates with publications from other companies is also useful.
| Company | Industry | Usecase | Cluster Size | (Un)Compressed Data Size\* | Reference |
-|-----------------------------------------------------------------------------|---------------------------------|-----------------------|------------------------------------------------------------|------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+|-----------------------------------------------|---------------------|---------------|------------------------------------|------------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------|
| [2gis](https://2gis.ru) | Maps | Monitoring | — | — | [Talk in Russian, July 2019](https://youtu.be/58sPkXfq6nw) |
| [Aloha Browser](https://alohabrowser.com/) | Mobile App | Browser backend | — | — | [Slides in Russian, May 2019](https://github.com/yandex/clickhouse-presentations/blob/master/meetup22/aloha.pdf) |
| [Amadeus](https://amadeus.com/) | Travel | Analytics | — | — | [Press Release, April 2018](https://www.altinity.com/blog/2018/4/5/amadeus-technologies-launches-investment-and-insights-tool-based-on-machine-learning-and-strategy-algorithms) |
diff --git a/docs/en/introduction/distinctive_features.md b/docs/en/introduction/distinctive_features.md
index c857fafe1e0..9704c654723 100644
--- a/docs/en/introduction/distinctive_features.md
+++ b/docs/en/introduction/distinctive_features.md
@@ -1,3 +1,8 @@
+---
+toc_priority: 4
+toc_title: Distinctive Features
+---
+
# Distinctive Features of ClickHouse {#distinctive-features-of-clickhouse}
## True Column-Oriented DBMS {#true-column-oriented-dbms}
@@ -59,6 +64,6 @@ ClickHouse provides various ways to trade accuracy for performance:
ClickHouse uses asynchronous multi-master replication. After being written to any available replica, all the remaining replicas retrieve their copy in the background. The system maintains identical data on different replicas. Recovery after most failures is performed automatically, or semi-automatically in complex cases.
-For more information, see the section [Data replication](../operations/table_engines/replication.md).
+For more information, see the section [Data replication](../engines/table_engines/mergetree_family/replication.md).
[Original article](https://clickhouse.tech/docs/en/introduction/distinctive_features/)
diff --git a/docs/en/introduction/features_considered_disadvantages.md b/docs/en/introduction/features_considered_disadvantages.md
index 9a70c2f0f6c..597dad30663 100644
--- a/docs/en/introduction/features_considered_disadvantages.md
+++ b/docs/en/introduction/features_considered_disadvantages.md
@@ -1,3 +1,8 @@
+---
+toc_priority: 5
+toc_title: ClickHouse Features that Can Be Considered Disadvantages
+---
+
# ClickHouse Features that Can be Considered Disadvantages {#clickhouse-features-that-can-be-considered-disadvantages}
1. No full-fledged transactions.
diff --git a/docs/en/introduction/history.md b/docs/en/introduction/history.md
index a56f4d640d1..247c4afc375 100644
--- a/docs/en/introduction/history.md
+++ b/docs/en/introduction/history.md
@@ -1,3 +1,8 @@
+---
+toc_priority: 7
+toc_title: History
+---
+
# ClickHouse History {#clickhouse-history}
ClickHouse has been developed initially to power [Yandex.Metrica](https://metrica.yandex.com/), [the second largest web analytics platform in the world](http://w3techs.com/technologies/overview/traffic_analysis/all), and continues to be the core component of this system. With more than 13 trillion records in the database and more than 20 billion events daily, ClickHouse allows generating custom reports on the fly directly from non-aggregated data. This article briefly covers the goals of ClickHouse in the early stages of its development.
diff --git a/docs/en/introduction/index.md b/docs/en/introduction/index.md
new file mode 100644
index 00000000000..1a089551261
--- /dev/null
+++ b/docs/en/introduction/index.md
@@ -0,0 +1,5 @@
+---
+toc_folder_title: Introduction
+toc_priority: 1
+---
+
diff --git a/docs/en/introduction/performance.md b/docs/en/introduction/performance.md
index ae8ec0dd2a1..065121f10ec 100644
--- a/docs/en/introduction/performance.md
+++ b/docs/en/introduction/performance.md
@@ -1,3 +1,8 @@
+---
+toc_priority: 6
+toc_title: Performance
+---
+
# Performance {#performance}
According to internal testing results at Yandex, ClickHouse shows the best performance (both the highest throughput for long queries and the lowest latency on short queries) for comparable operating scenarios among systems of its class that were available for testing. You can view the test results on a [separate page](https://clickhouse.tech/benchmark.html).
diff --git a/docs/en/operations/access_rights.md b/docs/en/operations/access_rights.md
index 510a23d6130..fcef4995c4b 100644
--- a/docs/en/operations/access_rights.md
+++ b/docs/en/operations/access_rights.md
@@ -1,3 +1,8 @@
+---
+toc_priority: 48
+toc_title: Access Rights
+---
+
# Access Rights {#access-rights}
Users and access rights are set up in the user config. This is usually `users.xml`.
@@ -65,7 +70,7 @@ Users are recorded in the `users` section. Here is a fragment of the `users.xml`
You can see a declaration from two users: `default`and`web`. We added the `web` user separately.
-The `default` user is chosen in cases when the username is not passed. The `default` user is also used for distributed query processing, if the configuration of the server or cluster doesn’t specify the `user` and `password` (see the section on the [Distributed](../operations/table_engines/distributed.md) engine).
+The `default` user is chosen in cases when the username is not passed. The `default` user is also used for distributed query processing, if the configuration of the server or cluster doesn’t specify the `user` and `password` (see the section on the [Distributed](../engines/table_engines/special/distributed.md) engine).
The user that is used for exchanging information between servers combined in a cluster must not have substantial restrictions or quotas – otherwise, distributed queries will fail.
diff --git a/docs/en/operations/backup.md b/docs/en/operations/backup.md
index 52f7705f568..9510ef4b709 100644
--- a/docs/en/operations/backup.md
+++ b/docs/en/operations/backup.md
@@ -1,6 +1,11 @@
+---
+toc_priority: 49
+toc_title: Data Backup
+---
+
# Data Backup {#data-backup}
-While [replication](table_engines/replication.md) provides protection from hardware failures, it does not protect against human errors: accidental deletion of data, deletion of the wrong table or a table on the wrong cluster, and software bugs that result in incorrect data processing or data corruption. In many cases mistakes like these will affect all replicas. ClickHouse has built-in safeguards to prevent some types of mistakes — for example, by default [you can’t just drop tables with a MergeTree-like engine containing more than 50 Gb of data](https://github.com/ClickHouse/ClickHouse/blob/v18.14.18-stable/programs/server/config.xml#L322-L330). However, these safeguards don’t cover all possible cases and can be circumvented.
+While [replication](../engines/table_engines/mergetree_family/replication.md) provides protection from hardware failures, it does not protect against human errors: accidental deletion of data, deletion of the wrong table or a table on the wrong cluster, and software bugs that result in incorrect data processing or data corruption. In many cases mistakes like these will affect all replicas. ClickHouse has built-in safeguards to prevent some types of mistakes — for example, by default [you can’t just drop tables with a MergeTree-like engine containing more than 50 Gb of data](https://github.com/ClickHouse/ClickHouse/blob/v18.14.18-stable/programs/server/config.xml#L322-L330). However, these safeguards don’t cover all possible cases and can be circumvented.
In order to effectively mitigate possible human errors, you should carefully prepare a strategy for backing up and restoring your data **in advance**.
@@ -15,19 +20,19 @@ Often data that is ingested into ClickHouse is delivered through some sort of pe
## Filesystem Snapshots {#filesystem-snapshots}
-Some local filesystems provide snapshot functionality (for example, [ZFS](https://en.wikipedia.org/wiki/ZFS)), but they might not be the best choice for serving live queries. A possible solution is to create additional replicas with this kind of filesystem and exclude them from the [Distributed](table_engines/distributed.md) tables that are used for `SELECT` queries. Snapshots on such replicas will be out of reach of any queries that modify data. As a bonus, these replicas might have special hardware configurations with more disks attached per server, which would be cost-effective.
+Some local filesystems provide snapshot functionality (for example, [ZFS](https://en.wikipedia.org/wiki/ZFS)), but they might not be the best choice for serving live queries. A possible solution is to create additional replicas with this kind of filesystem and exclude them from the [Distributed](../engines/table_engines/special/distributed.md) tables that are used for `SELECT` queries. Snapshots on such replicas will be out of reach of any queries that modify data. As a bonus, these replicas might have special hardware configurations with more disks attached per server, which would be cost-effective.
## clickhouse-copier {#clickhouse-copier}
-[clickhouse-copier](utils/clickhouse-copier.md) is a versatile tool that was initially created to re-shard petabyte-sized tables. It can also be used for backup and restore purposes because it reliably copies data between ClickHouse tables and clusters.
+[clickhouse-copier](utilities/clickhouse-copier.md) is a versatile tool that was initially created to re-shard petabyte-sized tables. It can also be used for backup and restore purposes because it reliably copies data between ClickHouse tables and clusters.
For smaller volumes of data, a simple `INSERT INTO ... SELECT ...` to remote tables might work as well.
-## Manipulations with Parts {#manipulations-with-parts}
+## Manipulations With Parts {#manipulations-with-parts}
ClickHouse allows using the `ALTER TABLE ... FREEZE PARTITION ...` query to create a local copy of table partitions. This is implemented using hardlinks to the `/var/lib/clickhouse/shadow/` folder, so it usually does not consume extra disk space for old data. The created copies of files are not handled by ClickHouse server, so you can just leave them there: you will have a simple backup that doesn’t require any additional external system, but it will still be prone to hardware issues. For this reason, it’s better to remotely copy them to another location and then remove the local copies. Distributed filesystems and object stores are still a good options for this, but normal attached file servers with a large enough capacity might work as well (in this case the transfer will occur via the network filesystem or maybe [rsync](https://en.wikipedia.org/wiki/Rsync)).
-For more information about queries related to partition manipulations, see the [ALTER documentation](../query_language/alter.md#alter_manipulations-with-partitions).
+For more information about queries related to partition manipulations, see the [ALTER documentation](../sql_reference/statements/alter.md#alter_manipulations-with-partitions).
A third-party tool is available to automate this approach: [clickhouse-backup](https://github.com/AlexAkulov/clickhouse-backup).
diff --git a/docs/en/operations/configuration_files.md b/docs/en/operations/configuration_files.md
index f3594b2d63f..2d6a1a163f5 100644
--- a/docs/en/operations/configuration_files.md
+++ b/docs/en/operations/configuration_files.md
@@ -1,3 +1,8 @@
+---
+toc_priority: 50
+toc_title: Configuration Files
+---
+
# Configuration Files {#configuration_files}
ClickHouse supports multi-file configuration management. The main server configuration file is `/etc/clickhouse-server/config.xml`. Other files must be in the `/etc/clickhouse-server/config.d` directory.
@@ -13,7 +18,7 @@ If `replace` is specified, it replaces the entire element with the specified one
If `remove` is specified, it deletes the element.
-The config can also define “substitutions”. If an element has the `incl` attribute, the corresponding substitution from the file will be used as the value. By default, the path to the file with substitutions is `/etc/metrika.xml`. This can be changed in the [include\_from](server_settings/settings.md#server_settings-include_from) element in the server config. The substitution values are specified in `/yandex/substitution_name` elements in this file. If a substitution specified in `incl` does not exist, it is recorded in the log. To prevent ClickHouse from logging missing substitutions, specify the `optional="true"` attribute (for example, settings for [macros](server_settings/settings.md)).
+The config can also define “substitutions”. If an element has the `incl` attribute, the corresponding substitution from the file will be used as the value. By default, the path to the file with substitutions is `/etc/metrika.xml`. This can be changed in the [include\_from](server_configuration_parameters/settings.md#server_configuration_parameters-include_from) element in the server config. The substitution values are specified in `/yandex/substitution_name` elements in this file. If a substitution specified in `incl` does not exist, it is recorded in the log. To prevent ClickHouse from logging missing substitutions, specify the `optional="true"` attribute (for example, settings for [macros](server_configuration_parameters/settings.md)).
Substitutions can also be performed from ZooKeeper. To do this, specify the attribute `from_zk = "/path/to/node"`. The element value is replaced with the contents of the node at `/path/to/node` in ZooKeeper. You can also put an entire XML subtree on the ZooKeeper node and it will be fully inserted into the source element.
diff --git a/docs/en/operations/index.md b/docs/en/operations/index.md
index 6a3d4d7975a..a23c6f3d16a 100644
--- a/docs/en/operations/index.md
+++ b/docs/en/operations/index.md
@@ -1,3 +1,9 @@
+---
+toc_folder_title: Operations
+toc_priority: 41
+toc_title: Introduction
+---
+
# Operations {#operations}
ClickHouse operations manual consists of the following major sections:
@@ -12,9 +18,9 @@ ClickHouse operations manual consists of the following major sections:
- [Configuration Files](configuration_files.md)
- [Quotas](quotas.md)
- [System Tables](system_tables.md)
-- [Server Configuration Parameters](server_settings/index.md)
+- [Server Configuration Parameters](server_configuration_parameters/index.md)
- [How To Test Your Hardware With ClickHouse](performance_test.md)
- [Settings](settings/index.md)
-- [Utilities](utils/index.md)
+- [Utilities](utilities/index.md)
[Original article](https://clickhouse.tech/docs/en/operations/)
diff --git a/docs/en/operations/monitoring.md b/docs/en/operations/monitoring.md
index 44afd58dd64..2e848dbb313 100644
--- a/docs/en/operations/monitoring.md
+++ b/docs/en/operations/monitoring.md
@@ -1,3 +1,8 @@
+---
+toc_priority: 45
+toc_title: Monitoring
+---
+
# Monitoring {#monitoring}
You can monitor:
@@ -17,20 +22,20 @@ It is highly recommended to set up monitoring for:
- Utilization of storage system, RAM and network.
-## ClickHouse Server Metrics {#clickhouse-server-metrics}
+## Clickhouse Server Metrics {#clickhouse-server-metrics}
ClickHouse server has embedded instruments for self-state monitoring.
-To track server events use server logs. See the [logger](server_settings/settings.md#server_settings-logger) section of the configuration file.
+To track server events use server logs. See the [logger](server_configuration_parameters/settings.md#server_configuration_parameters-logger) section of the configuration file.
ClickHouse collects:
- Different metrics of how the server uses computational resources.
- Common statistics on query processing.
-You can find metrics in the [system.metrics](system_tables.md#system_tables-metrics), [system.events](system_tables.md#system_tables-events), and [system.asynchronous\_metrics](system_tables.md#system_tables-asynchronous_metrics) tables.
+You can find metrics in the [system.metrics](../operations/system_tables.md#system_tables-metrics), [system.events](../operations/system_tables.md#system_tables-events), and [system.asynchronous\_metrics](../operations/system_tables.md#system_tables-asynchronous_metrics) tables.
-You can configure ClickHouse to export metrics to [Graphite](https://github.com/graphite-project). See the [Graphite section](server_settings/settings.md#server_settings-graphite) in the ClickHouse server configuration file. Before configuring export of metrics, you should set up Graphite by following their official [guide](https://graphite.readthedocs.io/en/latest/install.html).
+You can configure ClickHouse to export metrics to [Graphite](https://github.com/graphite-project). See the [Graphite section](server_configuration_parameters/settings.md#server_configuration_parameters-graphite) in the ClickHouse server configuration file. Before configuring export of metrics, you should set up Graphite by following their official [guide](https://graphite.readthedocs.io/en/latest/install.html).
Additionally, you can monitor server availability through the HTTP API. Send the `HTTP GET` request to `/ping`. If the server is available, it responds with `200 OK`.
diff --git a/docs/en/operations/optimizing_performance/index.md b/docs/en/operations/optimizing_performance/index.md
new file mode 100644
index 00000000000..4f4cbb66d24
--- /dev/null
+++ b/docs/en/operations/optimizing_performance/index.md
@@ -0,0 +1,5 @@
+---
+toc_folder_title: Optimizing Performance
+toc_priority: 52
+---
+
diff --git a/docs/en/operations/performance/sampling_query_profiler.md b/docs/en/operations/optimizing_performance/sampling_query_profiler.md
similarity index 74%
rename from docs/en/operations/performance/sampling_query_profiler.md
rename to docs/en/operations/optimizing_performance/sampling_query_profiler.md
index d2ee7125c66..120817f9c44 100644
--- a/docs/en/operations/performance/sampling_query_profiler.md
+++ b/docs/en/operations/optimizing_performance/sampling_query_profiler.md
@@ -1,12 +1,17 @@
+---
+toc_priority: 54
+toc_title: Query Profiling
+---
+
# Sampling Query Profiler {#sampling-query-profiler}
ClickHouse runs sampling profiler that allows analyzing query execution. Using profiler you can find source code routines that used the most frequently during query execution. You can trace CPU time and wall-clock time spent including idle time.
To use profiler:
-- Setup the [trace\_log](../server_settings/settings.md#server_settings-trace_log) section of the server configuration.
+- Setup the [trace\_log](../server_configuration_parameters/settings.md#server_configuration_parameters-trace_log) section of the server configuration.
- This section configures the [trace\_log](../system_tables.md#system_tables-trace_log) system table containing the results of the profiler functioning. It is configured by default. Remember that data in this table is valid only for a running server. After the server restart, ClickHouse doesn’t clean up the table and all the stored virtual memory address may become invalid.
+ This section configures the [trace\_log](../../operations/system_tables.md#system_tables-trace_log) system table containing the results of the profiler functioning. It is configured by default. Remember that data in this table is valid only for a running server. After the server restart, ClickHouse doesn’t clean up the table and all the stored virtual memory address may become invalid.
- Setup the [query\_profiler\_cpu\_time\_period\_ns](../settings/settings.md#query_profiler_cpu_time_period_ns) or [query\_profiler\_real\_time\_period\_ns](../settings/settings.md#query_profiler_real_time_period_ns) settings. Both settings can be used simultaneously.
@@ -22,7 +27,7 @@ To analyze the `trace_log` system table:
For security reasons, introspection functions are disabled by default.
-- Use the `addressToLine`, `addressToSymbol` and `demangle` [introspection functions](../../query_language/functions/introspection.md) to get function names and their positions in ClickHouse code. To get a profile for some query, you need to aggregate data from the `trace_log` table. You can aggregate data by individual functions or by the whole stack traces.
+- Use the `addressToLine`, `addressToSymbol` and `demangle` [introspection functions](../../sql_reference/functions/introspection.md) to get function names and their positions in ClickHouse code. To get a profile for some query, you need to aggregate data from the `trace_log` table. You can aggregate data by individual functions or by the whole stack traces.
If you need to visualize `trace_log` info, try [flamegraph](../../interfaces/third-party/gui/#clickhouse-flamegraph) and [speedscope](https://github.com/laplab/clickhouse-speedscope).
diff --git a/docs/en/operations/performance_test.md b/docs/en/operations/performance_test.md
index c7cb0e2d78f..ca285a64cf5 100644
--- a/docs/en/operations/performance_test.md
+++ b/docs/en/operations/performance_test.md
@@ -1,3 +1,8 @@
+---
+toc_priority: 54
+toc_title: Testing Hardware
+---
+
# How To Test Your Hardware With ClickHouse {#how-to-test-your-hardware-with-clickhouse}
With this instruction you can run basic ClickHouse performance test on any server without installation of ClickHouse packages.
diff --git a/docs/en/operations/quotas.md b/docs/en/operations/quotas.md
index 51825788f9a..aeef9f88c7c 100644
--- a/docs/en/operations/quotas.md
+++ b/docs/en/operations/quotas.md
@@ -1,3 +1,8 @@
+---
+toc_priority: 51
+toc_title: Quotas
+---
+
# Quotas {#quotas}
Quotas allow you to limit resource usage over a period of time, or simply track the use of resources.
diff --git a/docs/en/operations/requirements.md b/docs/en/operations/requirements.md
index df4ea287cbc..fc67ff7f989 100644
--- a/docs/en/operations/requirements.md
+++ b/docs/en/operations/requirements.md
@@ -1,3 +1,8 @@
+---
+toc_priority: 44
+toc_title: Requirements
+---
+
# Requirements {#requirements}
## CPU {#cpu}
@@ -17,9 +22,9 @@ The required volume of RAM depends on:
- The complexity of queries.
- The amount of data that is processed in queries.
-To calculate the required volume of RAM, you should estimate the size of temporary data for [GROUP BY](../query_language/select.md#select-group-by-clause), [DISTINCT](../query_language/select.md#select-distinct), [JOIN](../query_language/select.md#select-join) and other operations you use.
+To calculate the required volume of RAM, you should estimate the size of temporary data for [GROUP BY](../sql_reference/statements/select.md#select-group-by-clause), [DISTINCT](../sql_reference/statements/select.md#select-distinct), [JOIN](../sql_reference/statements/select.md#select-join) and other operations you use.
-ClickHouse can use external memory for temporary data. See [GROUP BY in External Memory](../query_language/select.md#select-group-by-in-external-memory) for details.
+ClickHouse can use external memory for temporary data. See [GROUP BY in External Memory](../sql_reference/statements/select.md#select-group-by-in-external-memory) for details.
## Swap File {#swap-file}
diff --git a/docs/en/operations/server_settings/index.md b/docs/en/operations/server_configuration_parameters/index.md
similarity index 80%
rename from docs/en/operations/server_settings/index.md
rename to docs/en/operations/server_configuration_parameters/index.md
index c02947be247..748f4d25345 100644
--- a/docs/en/operations/server_settings/index.md
+++ b/docs/en/operations/server_configuration_parameters/index.md
@@ -1,3 +1,9 @@
+---
+toc_folder_title: Server Configuration Parameters
+toc_priority: 54
+toc_title: Introduction
+---
+
# Server configuration parameters {#server-settings}
This section contains descriptions of server settings that cannot be changed at the session or query level.
@@ -8,4 +14,4 @@ Other settings are described in the “[Settings](../settings/index.md#settings)
Before studying the settings, read the [Configuration files](../configuration_files.md#configuration_files) section and note the use of substitutions (the `incl` and `optional` attributes).
-[Original article](https://clickhouse.tech/docs/en/operations/server_settings/)
+[Original article](https://clickhouse.tech/docs/en/operations/server_configuration_parameters/)
diff --git a/docs/en/operations/server_settings/settings.md b/docs/en/operations/server_configuration_parameters/settings.md
similarity index 80%
rename from docs/en/operations/server_settings/settings.md
rename to docs/en/operations/server_configuration_parameters/settings.md
index cfc3b3437bc..85744a039f4 100644
--- a/docs/en/operations/server_settings/settings.md
+++ b/docs/en/operations/server_configuration_parameters/settings.md
@@ -1,3 +1,8 @@
+---
+toc_priority: 57
+toc_title: Server Settings
+---
+
# Server Settings {#server-settings}
## builtin\_dictionaries\_reload\_interval {#builtin-dictionaries-reload-interval}
@@ -16,7 +21,7 @@ Default value: 3600.
## compression {#server-settings-compression}
-Data compression settings for [MergeTree](../table_engines/mergetree.md)-engine tables.
+Data compression settings for [MergeTree](../../engines/table_engines/mergetree_family/mergetree.md)-engine tables.
!!! warning "Warning"
Don’t use it if you have just started using ClickHouse.
@@ -65,7 +70,7 @@ If no conditions met for a data part, ClickHouse uses the `lz4` compression.
The default database.
-To get a list of databases, use the [SHOW DATABASES](../../query_language/show.md#show-databases) query.
+To get a list of databases, use the [SHOW DATABASES](../../sql_reference/statements/show.md#show-databases) query.
**Example**
@@ -85,7 +90,7 @@ Settings profiles are located in the file specified in the parameter `user_confi
default
```
-## dictionaries\_config {#server_settings-dictionaries_config}
+## dictionaries\_config {#server_configuration_parameters-dictionaries_config}
The path to the config file for external dictionaries.
@@ -94,7 +99,7 @@ Path:
- Specify the absolute path or the path relative to the server config file.
- The path can contain wildcards \* and ?.
-See also “[External dictionaries](../../query_language/dicts/external_dicts.md)”.
+See also “[External dictionaries](../../sql_reference/dictionaries/external_dictionaries/external_dicts.md)”.
**Example**
@@ -102,7 +107,7 @@ See also “[External dictionaries](../../query_language/dicts/external_dicts.md
*_dictionary.xml
```
-## dictionaries\_lazy\_load {#server_settings-dictionaries_lazy_load}
+## dictionaries\_lazy\_load {#server_configuration_parameters-dictionaries_lazy_load}
Lazy loading of dictionaries.
@@ -118,7 +123,7 @@ The default is `true`.
true
```
-## format\_schema\_path {#server_settings-format_schema_path}
+## format\_schema\_path {#server_configuration_parameters-format_schema_path}
The path to the directory with the schemes for the input data, such as schemas for the [CapnProto](../../interfaces/formats.md#capnproto) format.
@@ -129,7 +134,7 @@ The path to the directory with the schemes for the input data, such as schemas f
format_schemas/
```
-## graphite {#server_settings-graphite}
+## graphite {#server_configuration_parameters-graphite}
Sending data to [Graphite](https://github.com/graphite-project).
@@ -140,10 +145,10 @@ Settings:
- interval – The interval for sending, in seconds.
- timeout – The timeout for sending data, in seconds.
- root\_path – Prefix for keys.
-- metrics – Sending data from the [system.metrics](../system_tables.md#system_tables-metrics) table.
-- events – Sending deltas data accumulated for the time period from the [system.events](../system_tables.md#system_tables-events) table.
-- events\_cumulative – Sending cumulative data from the [system.events](../system_tables.md#system_tables-events) table.
-- asynchronous\_metrics – Sending data from the [system.asynchronous\_metrics](../system_tables.md#system_tables-asynchronous_metrics) table.
+- metrics – Sending data from the [system.metrics](../../operations/system_tables.md#system_tables-metrics) table.
+- events – Sending deltas data accumulated for the time period from the [system.events](../../operations/system_tables.md#system_tables-events) table.
+- events\_cumulative – Sending cumulative data from the [system.events](../../operations/system_tables.md#system_tables-events) table.
+- asynchronous\_metrics – Sending data from the [system.asynchronous\_metrics](../../operations/system_tables.md#system_tables-asynchronous_metrics) table.
You can configure multiple `` clauses. For instance, you can use this for sending different data at different intervals.
@@ -163,11 +168,11 @@ You can configure multiple `` clauses. For instance, you can use this
```
-## graphite\_rollup {#server_settings-graphite-rollup}
+## graphite\_rollup {#server_configuration_parameters-graphite-rollup}
Settings for thinning data for Graphite.
-For more details, see [GraphiteMergeTree](../table_engines/graphitemergetree.md).
+For more details, see [GraphiteMergeTree](../../engines/table_engines/mergetree_family/graphitemergetree.md).
**Example**
@@ -195,7 +200,7 @@ For more details, see [GraphiteMergeTree](../table_engines/graphitemergetree.md)
The port for connecting to the server over HTTP(s).
-If `https_port` is specified, [openSSL](#server_settings-openssl) must be configured.
+If `https_port` is specified, [openSSL](#server_configuration_parameters-openssl) must be configured.
If `http_port` is specified, the OpenSSL configuration is ignored even if it is set.
@@ -205,7 +210,7 @@ If `http_port` is specified, the OpenSSL configuration is ignored even if it is
0000
```
-## http\_server\_default\_response {#server_settings-http_server_default_response}
+## http\_server\_default\_response {#server_configuration_parameters-http_server_default_response}
The page that is shown by default when you access the ClickHouse HTTP(s) server.
The default value is “Ok.” (with a line feed at the end)
@@ -220,7 +225,7 @@ Opens `https://tabix.io/` when accessing `http://localhost: http_port`.
```
-## include\_from {#server_settings-include_from}
+## include\_from {#server_configuration_parameters-include_from}
The path to the file with substitutions.
@@ -258,7 +263,7 @@ Useful for breaking away from a specific network interface.
## interserver\_http\_credentials {#server-settings-interserver-http-credentials}
-The username and password used to authenticate during [replication](../table_engines/replication.md) with the Replicated\* engines. These credentials are used only for communication between replicas and are unrelated to credentials for ClickHouse clients. The server is checking these credentials for connecting replicas and use the same credentials when connecting to other replicas. So, these credentials should be set the same for all replicas in a cluster.
+The username and password used to authenticate during [replication](../../engines/table_engines/mergetree_family/replication.md) with the Replicated\* engines. These credentials are used only for communication between replicas and are unrelated to credentials for ClickHouse clients. The server is checking these credentials for connecting replicas and use the same credentials when connecting to other replicas. So, these credentials should be set the same for all replicas in a cluster.
By default, the authentication is not used.
This section contains the following parameters:
@@ -285,7 +290,7 @@ The number of seconds that ClickHouse waits for incoming requests before closing
3
```
-## listen\_host {#server_settings-listen_host}
+## listen\_host {#server_configuration_parameters-listen_host}
Restriction on hosts that requests can come from. If you want the server to answer all of them, specify `::`.
@@ -296,7 +301,7 @@ Examples:
127.0.0.1
```
-## logger {#server_settings-logger}
+## logger {#server_configuration_parameters-logger}
Logging settings.
@@ -349,7 +354,7 @@ Parameter substitutions for replicated tables.
Can be omitted if replicated tables are not used.
-For more information, see the section “[Creating replicated tables](../../operations/table_engines/replication.md)”.
+For more information, see the section “[Creating replicated tables](../../engines/table_engines/mergetree_family/replication.md)”.
**Example**
@@ -359,7 +364,7 @@ For more information, see the section “[Creating replicated tables](../../oper
## mark\_cache\_size {#server-mark-cache-size}
-Approximate size (in bytes) of the cache of marks used by table engines of the [MergeTree](../table_engines/mergetree.md) family.
+Approximate size (in bytes) of the cache of marks used by table engines of the [MergeTree](../../engines/table_engines/mergetree_family/mergetree.md) family.
The cache is shared for the server and memory is allocated as needed. The cache size must be at least 5368709120.
@@ -407,7 +412,7 @@ We recommend using this option in Mac OS X since the `getrlimit()` function retu
Restriction on deleting tables.
-If the size of a [MergeTree](../table_engines/mergetree.md) table exceeds `max_table_size_to_drop` (in bytes), you can’t delete it using a DROP query.
+If the size of a [MergeTree](../../engines/table_engines/mergetree_family/mergetree.md) table exceeds `max_table_size_to_drop` (in bytes), you can’t delete it using a DROP query.
If you still need to delete the table without restarting the ClickHouse server, create the `/flags/force_drop_table` file and run the DROP query.
@@ -421,9 +426,9 @@ The value 0 means that you can delete all tables without any restrictions.
0
```
-## merge\_tree {#server_settings-merge_tree}
+## merge\_tree {#server_configuration_parameters-merge_tree}
-Fine tuning for tables in the [MergeTree](../table_engines/mergetree.md).
+Fine tuning for tables in the [MergeTree](../../engines/table_engines/mergetree_family/mergetree.md).
For more information, see the MergeTreeSettings.h header file.
@@ -435,7 +440,7 @@ For more information, see the MergeTreeSettings.h header file.
```
-## openSSL {#server_settings-openssl}
+## openSSL {#server_configuration_parameters-openssl}
SSL client/server configuration.
@@ -494,17 +499,17 @@ Keys for server/client settings:
```
-## part\_log {#server_settings-part-log}
+## part\_log {#server_configuration_parameters-part-log}
-Logging events that are associated with [MergeTree](../table_engines/mergetree.md). For instance, adding or merging data. You can use the log to simulate merge algorithms and compare their characteristics. You can visualize the merge process.
+Logging events that are associated with [MergeTree](../../engines/table_engines/mergetree_family/mergetree.md). For instance, adding or merging data. You can use the log to simulate merge algorithms and compare their characteristics. You can visualize the merge process.
-Queries are logged in the [system.part\_log](../system_tables.md#system_tables-part-log) table, not in a separate file. You can configure the name of this table in the `table` parameter (see below).
+Queries are logged in the [system.part\_log](../../operations/system_tables.md#system_tables-part-log) table, not in a separate file. You can configure the name of this table in the `table` parameter (see below).
Use the following parameters to configure logging:
- `database` – Name of the database.
- `table` – Name of the system table.
-- `partition_by` – Sets a [custom partitioning key](../../operations/table_engines/custom_partitioning_key.md).
+- `partition_by` – Sets a [custom partitioning key](../../engines/table_engines/mergetree_family/custom_partitioning_key.md).
- `flush_interval_milliseconds` – Interval for flushing data from the buffer in memory to the table.
**Example**
@@ -518,7 +523,7 @@ Use the following parameters to configure logging:
```
-## path {#server_settings-path}
+## path {#server_configuration_parameters-path}
The path to the directory containing data.
@@ -531,17 +536,17 @@ The path to the directory containing data.
/var/lib/clickhouse/
```
-## query\_log {#server_settings-query-log}
+## query\_log {#server_configuration_parameters-query-log}
Setting for logging queries received with the [log\_queries=1](../settings/settings.md) setting.
-Queries are logged in the [system.query\_log](../system_tables.md#system_tables-query_log) table, not in a separate file. You can change the name of the table in the `table` parameter (see below).
+Queries are logged in the [system.query\_log](../../operations/system_tables.md#system_tables-query_log) table, not in a separate file. You can change the name of the table in the `table` parameter (see below).
Use the following parameters to configure logging:
- `database` – Name of the database.
- `table` – Name of the system table the queries will be logged in.
-- `partition_by` – Sets a [custom partitioning key](../../operations/table_engines/custom_partitioning_key.md) for a table.
+- `partition_by` – Sets a [custom partitioning key](../../engines/table_engines/mergetree_family/custom_partitioning_key.md) for a table.
- `flush_interval_milliseconds` – Interval for flushing data from the buffer in memory to the table.
If the table doesn’t exist, ClickHouse will create it. If the structure of the query log changed when the ClickHouse server was updated, the table with the old structure is renamed, and a new table is created automatically.
@@ -557,17 +562,17 @@ If the table doesn’t exist, ClickHouse will create it. If the structure of the
```
-## query\_thread\_log {#server_settings-query-thread-log}
+## query\_thread\_log {#server_configuration_parameters-query-thread-log}
Setting for logging threads of queries received with the [log\_query\_threads=1](../settings/settings.md#settings-log-query-threads) setting.
-Queries are logged in the [system.query\_thread\_log](../system_tables.md#system_tables-query-thread-log) table, not in a separate file. You can change the name of the table in the `table` parameter (see below).
+Queries are logged in the [system.query\_thread\_log](../../operations/system_tables.md#system_tables-query-thread-log) table, not in a separate file. You can change the name of the table in the `table` parameter (see below).
Use the following parameters to configure logging:
- `database` – Name of the database.
- `table` – Name of the system table the queries will be logged in.
-- `partition_by` – Sets a [custom partitioning key](../../operations/table_engines/custom_partitioning_key.md) for a system table.
+- `partition_by` – Sets a [custom partitioning key](../../engines/table_engines/mergetree_family/custom_partitioning_key.md) for a system table.
- `flush_interval_milliseconds` – Interval for flushing data from the buffer in memory to the table.
If the table doesn’t exist, ClickHouse will create it. If the structure of the query thread log changed when the ClickHouse server was updated, the table with the old structure is renamed, and a new table is created automatically.
@@ -583,15 +588,15 @@ If the table doesn’t exist, ClickHouse will create it. If the structure of the
```
-## trace\_log {#server_settings-trace_log}
+## trace\_log {#server_configuration_parameters-trace_log}
-Settings for the [trace\_log](../system_tables.md#system_tables-trace_log) system table operation.
+Settings for the [trace\_log](../../operations/system_tables.md#system_tables-trace_log) system table operation.
Parameters:
- `database` — Database for storing a table.
- `table` — Table name.
-- `partition_by` — [Custom partitioning key](../../operations/table_engines/custom_partitioning_key.md) for a system table.
+- `partition_by` — [Custom partitioning key](../../engines/table_engines/mergetree_family/custom_partitioning_key.md) for a system table.
- `flush_interval_milliseconds` — Interval for flushing data from the buffer in memory to the table.
The default server configuration file `config.xml` contains the following settings section:
@@ -638,7 +643,7 @@ nodes will be stored without masking.
## remote\_servers {#server-settings-remote-servers}
-Configuration of clusters used by the [Distributed](../../operations/table_engines/distributed.md) table engine and by the `cluster` table function.
+Configuration of clusters used by the [Distributed](../../engines/table_engines/special/distributed.md) table engine and by the `cluster` table function.
**Example**
@@ -652,7 +657,7 @@ For the value of the `incl` attribute, see the section “[Configuration files](
- [skip\_unavailable\_shards](../settings/settings.md#settings-skip_unavailable_shards)
-## timezone {#server_settings-timezone}
+## timezone {#server_configuration_parameters-timezone}
The server’s time zone.
@@ -666,7 +671,7 @@ The time zone is necessary for conversions between String and DateTime formats w
Europe/Moscow
```
-## tcp\_port {#server_settings-tcp_port}
+## tcp\_port {#server_configuration_parameters-tcp_port}
Port for communicating with clients over the TCP protocol.
@@ -676,9 +681,9 @@ Port for communicating with clients over the TCP protocol.
9000
```
-## tcp\_port\_secure {#server_settings-tcp_port-secure}
+## tcp\_port\_secure {#server_configuration_parameters-tcp_port-secure}
-TCP port for secure communication with clients. Use it with [OpenSSL](#server_settings-openssl) settings.
+TCP port for secure communication with clients. Use it with [OpenSSL](#server_configuration_parameters-openssl) settings.
**Possible values**
@@ -690,7 +695,7 @@ Positive integer.
9440
```
-## mysql\_port {#server_settings-mysql_port}
+## mysql\_port {#server_configuration_parameters-mysql_port}
Port for communicating with clients over MySQL protocol.
@@ -719,7 +724,7 @@ Path to temporary data for processing large queries.
## tmp\_policy {#server-settings-tmp-policy}
-Policy from [`storage_configuration`](../table_engines/mergetree.md#table_engine-mergetree-multiple-volumes) to store temporary files.
+Policy from [`storage_configuration`](../../engines/table_engines/mergetree_family/mergetree.md#table_engine-mergetree-multiple-volumes) to store temporary files.
If not set [`tmp_path`](#server-settings-tmp_path) is used, otherwise it is ignored.
!!! note "Note"
@@ -730,7 +735,7 @@ If not set [`tmp_path`](#server-settings-tmp_path) is used, otherwise it is igno
## uncompressed\_cache\_size {#server-settings-uncompressed_cache_size}
-Cache size (in bytes) for uncompressed data used by table engines from the [MergeTree](../table_engines/mergetree.md).
+Cache size (in bytes) for uncompressed data used by table engines from the [MergeTree](../../engines/table_engines/mergetree_family/mergetree.md).
There is one shared cache for the server. Memory is allocated on demand. The cache is used if the option [use\_uncompressed\_cache](../settings/settings.md#setting-use_uncompressed_cache) is enabled.
@@ -742,9 +747,9 @@ The uncompressed cache is advantageous for very short queries in individual case
8589934592
```
-## user\_files\_path {#server_settings-user_files_path}
+## user\_files\_path {#server_configuration_parameters-user_files_path}
-The directory with user files. Used in the table function [file()](../../query_language/table_functions/file.md).
+The directory with user files. Used in the table function [file()](../../sql_reference/table_functions/file.md).
**Example**
@@ -817,7 +822,7 @@ This section contains the following parameters:
**See Also**
-- [Replication](../../operations/table_engines/replication.md)
+- [Replication](../../engines/table_engines/mergetree_family/replication.md)
- [ZooKeeper Programmer’s Guide](http://zookeeper.apache.org/doc/current/zookeeperProgrammers.html)
## use\_minimalistic\_part\_header\_in\_zookeeper {#server-settings-use_minimalistic_part_header_in_zookeeper}
@@ -826,20 +831,20 @@ Storage method for data part headers in ZooKeeper.
This setting only applies to the `MergeTree` family. It can be specified:
-- Globally in the [merge\_tree](#server_settings-merge_tree) section of the `config.xml` file.
+- Globally in the [merge\_tree](#server_configuration_parameters-merge_tree) section of the `config.xml` file.
ClickHouse uses the setting for all the tables on the server. You can change the setting at any time. Existing tables change their behaviour when the setting changes.
- For each table.
- When creating a table, specify the corresponding [engine setting](../table_engines/mergetree.md#table_engine-mergetree-creating-a-table). The behaviour of an existing table with this setting does not change, even if the global setting changes.
+ When creating a table, specify the corresponding [engine setting](../../engines/table_engines/mergetree_family/mergetree.md#table_engine-mergetree-creating-a-table). The behaviour of an existing table with this setting does not change, even if the global setting changes.
**Possible values**
- 0 — Functionality is turned off.
- 1 — Functionality is turned on.
-If `use_minimalistic_part_header_in_zookeeper = 1`, then [replicated](../table_engines/replication.md) tables store the headers of the data parts compactly using a single `znode`. If the table contains many columns, this storage method significantly reduces the volume of the data stored in Zookeeper.
+If `use_minimalistic_part_header_in_zookeeper = 1`, then [replicated](../../engines/table_engines/mergetree_family/replication.md) tables store the headers of the data parts compactly using a single `znode`. If the table contains many columns, this storage method significantly reduces the volume of the data stored in Zookeeper.
!!! attention "Attention"
After applying `use_minimalistic_part_header_in_zookeeper = 1`, you can’t downgrade the ClickHouse server to a version that doesn’t support this setting. Be careful when upgrading ClickHouse on servers in a cluster. Don’t upgrade all the servers at once. It is safer to test new versions of ClickHouse in a test environment, or on just a few servers of a cluster.
@@ -862,4 +867,4 @@ The update is performed asynchronously, in a separate system thread.
**Default value**: 15.
-[Original article](https://clickhouse.tech/docs/en/operations/server_settings/settings/)
+[Original article](https://clickhouse.tech/docs/en/operations/server_configuration_parameters/settings/)
diff --git a/docs/en/operations/settings/constraints_on_settings.md b/docs/en/operations/settings/constraints_on_settings.md
index 3fd21c26aa7..2fe75aaad58 100644
--- a/docs/en/operations/settings/constraints_on_settings.md
+++ b/docs/en/operations/settings/constraints_on_settings.md
@@ -1,3 +1,8 @@
+---
+toc_priority: 62
+toc_title: Constraints on Settings
+---
+
# Constraints on Settings {#constraints-on-settings}
The constraints on settings can be defined in the `profiles` section of the `user.xml` configuration file and prohibit users from changing some of the settings with the `SET` query.
diff --git a/docs/en/operations/settings/index.md b/docs/en/operations/settings/index.md
index 7a8fb2fef5f..0f4a11272c6 100644
--- a/docs/en/operations/settings/index.md
+++ b/docs/en/operations/settings/index.md
@@ -1,3 +1,9 @@
+---
+toc_folder_title: Settings
+toc_priority: 55
+toc_title: Introduction
+---
+
# Settings {#settings}
There are multiple ways to make all the settings described below.
diff --git a/docs/en/operations/settings/permissions_for_queries.md b/docs/en/operations/settings/permissions_for_queries.md
index a229579f47d..4bcc065e3f1 100644
--- a/docs/en/operations/settings/permissions_for_queries.md
+++ b/docs/en/operations/settings/permissions_for_queries.md
@@ -1,3 +1,8 @@
+---
+toc_priority: 58
+toc_title: Permissions for Queries
+---
+
# Permissions for Queries {#permissions_for_queries}
Queries in ClickHouse can be divided into several types:
diff --git a/docs/en/operations/settings/query_complexity.md b/docs/en/operations/settings/query_complexity.md
index f62fa3b0008..a5144d560bf 100644
--- a/docs/en/operations/settings/query_complexity.md
+++ b/docs/en/operations/settings/query_complexity.md
@@ -1,4 +1,9 @@
-# Restrictions on Query Complexity {#restrictions-on-query-complexity}
+---
+toc_priority: 59
+toc_title: Restrictions on Query Complexity
+---
+
+# Restrictions On Query Complexity {#restrictions-on-query-complexity}
Restrictions on query complexity are part of the settings.
They are used to provide safer execution from the user interface.
@@ -75,11 +80,11 @@ Using the ‘any’ value lets you run an approximation of GROUP BY. The quality
## max\_bytes\_before\_external\_group\_by {#settings-max_bytes_before_external_group_by}
-Enables or disables execution of `GROUP BY` clauses in external memory. See [GROUP BY in external memory](../../query_language/select.md#select-group-by-in-external-memory).
+Enables or disables execution of `GROUP BY` clauses in external memory. See [GROUP BY in external memory](../../sql_reference/statements/select.md#select-group-by-in-external-memory).
Possible values:
-- Maximum volume of RAM (in bytes) that can be used by the single [GROUP BY](../../query_language/select.md#select-group-by-clause) operation.
+- Maximum volume of RAM (in bytes) that can be used by the single [GROUP BY](../../sql_reference/statements/select.md#select-group-by-clause) operation.
- 0 — `GROUP BY` in external memory disabled.
Default value: 0.
@@ -227,7 +232,7 @@ What to do when the amount of data exceeds one of the limits: ‘throw’ or ‘
Limits the number of rows in the hash table that is used when joining tables.
-This settings applies to [SELECT … JOIN](../../query_language/select.md#select-join) operations and the [Join](../table_engines/join.md) table engine.
+This settings applies to [SELECT … JOIN](../../sql_reference/statements/select.md#select-join) operations and the [Join](../../engines/table_engines/special/join.md) table engine.
If a query contains multiple joins, ClickHouse checks this setting for every intermediate result.
@@ -244,7 +249,7 @@ Default value: 0.
Limits the size in bytes of the hash table used when joining tables.
-This settings applies to [SELECT … JOIN](../../query_language/select.md#select-join) operations and [Join table engine](../table_engines/join.md).
+This settings applies to [SELECT … JOIN](../../sql_reference/statements/select.md#select-join) operations and [Join table engine](../../engines/table_engines/special/join.md).
If the query contains joins, ClickHouse checks this setting for every intermediate result.
@@ -273,8 +278,8 @@ Default value: `THROW`.
**See Also**
-- [JOIN clause](../../query_language/select.md#select-join)
-- [Join table engine](../table_engines/join.md)
+- [JOIN clause](../../sql_reference/statements/select.md#select-join)
+- [Join table engine](../../engines/table_engines/special/join.md)
## max\_partitions\_per\_insert\_block {#max-partitions-per-insert-block}
diff --git a/docs/en/operations/settings/settings.md b/docs/en/operations/settings/settings.md
index 7da2927fc43..ad8153b7e7e 100644
--- a/docs/en/operations/settings/settings.md
+++ b/docs/en/operations/settings/settings.md
@@ -1,8 +1,13 @@
+---
+toc_priority: 60
+toc_title: Settings
+---
+
# Settings {#settings}
## distributed\_product\_mode {#distributed-product-mode}
-Changes the behavior of [distributed subqueries](../../query_language/select.md).
+Changes the behavior of [distributed subqueries](../../sql_reference/statements/select.md).
ClickHouse applies this setting when the query contains the product of distributed tables, i.e. when the query for a distributed table contains a non-GLOBAL subquery for the distributed table.
@@ -11,7 +16,7 @@ Restrictions:
- Only applied for IN and JOIN subqueries.
- Only if the FROM section uses a distributed table containing more than one shard.
- If the subquery concerns a distributed table containing more than one shard.
-- Not used for a table-valued [remote](../../query_language/table_functions/remote.md) function.
+- Not used for a table-valued [remote](../../sql_reference/table_functions/remote.md) function.
Possible values:
@@ -46,7 +51,7 @@ If `enable_optimize_predicate_expression = 0`, then the execution time of the se
## fallback\_to\_stale\_replicas\_for\_distributed\_queries {#settings-fallback_to_stale_replicas_for_distributed_queries}
-Forces a query to an out-of-date replica if updated data is not available. See [Replication](../table_engines/replication.md).
+Forces a query to an out-of-date replica if updated data is not available. See [Replication](../../engines/table_engines/mergetree_family/replication.md).
ClickHouse selects the most relevant from the outdated replicas of the table.
@@ -60,7 +65,7 @@ Disables query execution if the index can’t be used by date.
Works with tables in the MergeTree family.
-If `force_index_by_date=1`, ClickHouse checks whether the query has a date key condition that can be used for restricting data ranges. If there is no suitable condition, it throws an exception. However, it does not check whether the condition reduces the amount of data to read. For example, the condition `Date != ' 2000-01-01 '` is acceptable even when it matches all the data in the table (i.e., running the query requires a full scan). For more information about ranges of data in MergeTree tables, see [MergeTree](../table_engines/mergetree.md).
+If `force_index_by_date=1`, ClickHouse checks whether the query has a date key condition that can be used for restricting data ranges. If there is no suitable condition, it throws an exception. However, it does not check whether the condition reduces the amount of data to read. For example, the condition `Date != ' 2000-01-01 '` is acceptable even when it matches all the data in the table (i.e., running the query requires a full scan). For more information about ranges of data in MergeTree tables, see [MergeTree](../../engines/table_engines/mergetree_family/mergetree.md).
## force\_primary\_key {#force-primary-key}
@@ -68,7 +73,7 @@ Disables query execution if indexing by the primary key is not possible.
Works with tables in the MergeTree family.
-If `force_primary_key=1`, ClickHouse checks to see if the query has a primary key condition that can be used for restricting data ranges. If there is no suitable condition, it throws an exception. However, it does not check whether the condition reduces the amount of data to read. For more information about data ranges in MergeTree tables, see [MergeTree](../table_engines/mergetree.md).
+If `force_primary_key=1`, ClickHouse checks to see if the query has a primary key condition that can be used for restricting data ranges. If there is no suitable condition, it throws an exception. However, it does not check whether the condition reduces the amount of data to read. For more information about data ranges in MergeTree tables, see [MergeTree](../../engines/table_engines/mergetree_family/mergetree.md).
## format\_schema {#format-schema}
@@ -129,7 +134,7 @@ Default value: 0.
## max\_http\_get\_redirects {#setting-max_http_get_redirects}
-Limits the maximum number of HTTP GET redirect hops for [URL](../table_engines/url.md)-engine tables. The setting applies to both types of tables: those created by the [CREATE TABLE](../../query_language/create/#create-table-query) query and by the [url](../../query_language/table_functions/url.md) table function.
+Limits the maximum number of HTTP GET redirect hops for [URL](../../engines/table_engines/special/url.md)-engine tables. The setting applies to both types of tables: those created by the [CREATE TABLE](../../query_language/create/#create-table-query) query and by the [url](../../sql_reference/table_functions/url.md) table function.
Possible values:
@@ -165,7 +170,7 @@ If both `input_format_allow_errors_num` and `input_format_allow_errors_ratio` ar
## input\_format\_values\_interpret\_expressions {#settings-input_format_values_interpret_expressions}
-Enables or disables the full SQL parser if the fast stream parser can’t parse the data. This setting is used only for the [Values](../../interfaces/formats.md#data-format-values) format at the data insertion. For more information about syntax parsing, see the [Syntax](../../query_language/syntax.md) section.
+Enables or disables the full SQL parser if the fast stream parser can’t parse the data. This setting is used only for the [Values](../../interfaces/formats.md#data-format-values) format at the data insertion. For more information about syntax parsing, see the [Syntax](../../sql_reference/syntax.md) section.
Possible values:
@@ -181,7 +186,7 @@ Default value: 1.
Example of Use
-Insert the [DateTime](../../data_types/datetime.md) type value with the different settings.
+Insert the [DateTime](../../sql_reference/data_types/datetime.md) type value with the different settings.
``` sql
SET input_format_values_interpret_expressions = 0;
@@ -326,7 +331,7 @@ Default value: 1.
Allows choosing a parser of the text representation of date and time.
-The setting doesn’t apply to [date and time functions](../../query_language/functions/date_time_functions.md).
+The setting doesn’t apply to [date and time functions](../../sql_reference/functions/date_time_functions.md).
Possible values:
@@ -342,12 +347,12 @@ Default value: `'basic'`.
See also:
-- [DateTime data type.](../../data_types/datetime.md)
-- [Functions for working with dates and times.](../../query_language/functions/date_time_functions.md)
+- [DateTime data type.](../../sql_reference/data_types/datetime.md)
+- [Functions for working with dates and times.](../../sql_reference/functions/date_time_functions.md)
## join\_default\_strictness {#settings-join_default_strictness}
-Sets default strictness for [JOIN clauses](../../query_language/select.md#select-join).
+Sets default strictness for [JOIN clauses](../../sql_reference/statements/select.md#select-join).
Possible values:
@@ -363,7 +368,7 @@ Default value: `ALL`.
Changes behaviour of join operations with `ANY` strictness.
!!! warning "Attention"
- This setting applies only for `JOIN` operations with [Join](../table_engines/join.md) engine tables.
+ This setting applies only for `JOIN` operations with [Join](../../engines/table_engines/special/join.md) engine tables.
Possible values:
@@ -374,18 +379,18 @@ Default value: 0.
See also:
-- [JOIN clause](../../query_language/select.md#select-join)
-- [Join table engine](../table_engines/join.md)
+- [JOIN clause](../../sql_reference/statements/select.md#select-join)
+- [Join table engine](../../engines/table_engines/special/join.md)
- [join\_default\_strictness](#settings-join_default_strictness)
## join\_use\_nulls {#join_use_nulls}
-Sets the type of [JOIN](../../query_language/select.md) behavior. When merging tables, empty cells may appear. ClickHouse fills them differently based on this setting.
+Sets the type of [JOIN](../../sql_reference/statements/select.md) behavior. When merging tables, empty cells may appear. ClickHouse fills them differently based on this setting.
Possible values:
- 0 — The empty cells are filled with the default value of the corresponding field type.
-- 1 — `JOIN` behaves the same way as in standard SQL. The type of the corresponding field is converted to [Nullable](../../data_types/nullable.md#data_type-nullable), and empty cells are filled with [NULL](../../query_language/syntax.md).
+- 1 — `JOIN` behaves the same way as in standard SQL. The type of the corresponding field is converted to [Nullable](../../sql_reference/data_types/nullable.md#data_type-nullable), and empty cells are filled with [NULL](../../sql_reference/syntax.md).
Default value: 0.
@@ -405,7 +410,7 @@ By default: 1,000,000. It only works when reading from MergeTree engines.
## merge\_tree\_min\_rows\_for\_concurrent\_read {#setting-merge-tree-min-rows-for-concurrent-read}
-If the number of rows to be read from a file of a [MergeTree](../table_engines/mergetree.md) table exceeds `merge_tree_min_rows_for_concurrent_read` then ClickHouse tries to perform a concurrent reading from this file on several threads.
+If the number of rows to be read from a file of a [MergeTree](../../engines/table_engines/mergetree_family/mergetree.md) table exceeds `merge_tree_min_rows_for_concurrent_read` then ClickHouse tries to perform a concurrent reading from this file on several threads.
Possible values:
@@ -415,7 +420,7 @@ Default value: 163840.
## merge\_tree\_min\_bytes\_for\_concurrent\_read {#setting-merge-tree-min-bytes-for-concurrent-read}
-If the number of bytes to read from one file of a [MergeTree](../table_engines/mergetree.md)-engine table exceeds `merge_tree_min_bytes_for_concurrent_read`, then ClickHouse tries to concurrently read from this file in several threads.
+If the number of bytes to read from one file of a [MergeTree](../../engines/table_engines/mergetree_family/mergetree.md)-engine table exceeds `merge_tree_min_bytes_for_concurrent_read`, then ClickHouse tries to concurrently read from this file in several threads.
Possible value:
@@ -457,7 +462,7 @@ Default value: 8.
If ClickHouse should read more than `merge_tree_max_rows_to_use_cache` rows in one query, it doesn’t use the cache of uncompressed blocks.
-The cache of uncompressed blocks stores data extracted for queries. ClickHouse uses this cache to speed up responses to repeated small queries. This setting protects the cache from trashing by queries that read a large amount of data. The [uncompressed\_cache\_size](../server_settings/settings.md#server-settings-uncompressed_cache_size) server setting defines the size of the cache of uncompressed blocks.
+The cache of uncompressed blocks stores data extracted for queries. ClickHouse uses this cache to speed up responses to repeated small queries. This setting protects the cache from trashing by queries that read a large amount of data. The [uncompressed\_cache\_size](../server_configuration_parameters/settings.md#server-settings-uncompressed_cache_size) server setting defines the size of the cache of uncompressed blocks.
Possible values:
@@ -469,7 +474,7 @@ Default value: 128 ✕ 8192.
If ClickHouse should read more than `merge_tree_max_bytes_to_use_cache` bytes in one query, it doesn’t use the cache of uncompressed blocks.
-The cache of uncompressed blocks stores data extracted for queries. ClickHouse uses this cache to speed up responses to repeated small queries. This setting protects the cache from trashing by queries that read a large amount of data. The [uncompressed\_cache\_size](../server_settings/settings.md#server-settings-uncompressed_cache_size) server setting defines the size of the cache of uncompressed blocks.
+The cache of uncompressed blocks stores data extracted for queries. ClickHouse uses this cache to speed up responses to repeated small queries. This setting protects the cache from trashing by queries that read a large amount of data. The [uncompressed\_cache\_size](../server_configuration_parameters/settings.md#server-settings-uncompressed_cache_size) server setting defines the size of the cache of uncompressed blocks.
Possible value:
@@ -494,7 +499,7 @@ Default value: 0.
Setting up query logging.
-Queries sent to ClickHouse with this setup are logged according to the rules in the [query\_log](../server_settings/settings.md#server_settings-query-log) server configuration parameter.
+Queries sent to ClickHouse with this setup are logged according to the rules in the [query\_log](../server_configuration_parameters/settings.md#server_configuration_parameters-query-log) server configuration parameter.
Example:
@@ -506,7 +511,7 @@ log_queries=1
Setting up query threads logging.
-Queries’ threads runned by ClickHouse with this setup are logged according to the rules in the [query\_thread\_log](../server_settings/settings.md#server_settings-query-thread-log) server configuration parameter.
+Queries’ threads runned by ClickHouse with this setup are logged according to the rules in the [query\_thread\_log](../server_configuration_parameters/settings.md#server_configuration_parameters-query-thread-log) server configuration parameter.
Example:
@@ -528,7 +533,7 @@ The default is slightly more than `max_block_size`. The reason for this is becau
## max\_replica\_delay\_for\_distributed\_queries {#settings-max_replica_delay_for_distributed_queries}
-Disables lagging replicas for distributed queries. See [Replication](../../operations/table_engines/replication.md).
+Disables lagging replicas for distributed queries. See [Replication](../../engines/table_engines/mergetree_family/replication.md).
Sets the time in seconds. If a replica lags more than the set value, this replica is not used.
@@ -573,7 +578,7 @@ Don’t confuse blocks for compression (a chunk of memory consisting of bytes) w
## min\_compress\_block\_size {#min-compress-block-size}
-For [MergeTree](../table_engines/mergetree.md)" tables. In order to reduce latency when processing queries, a block is compressed when writing the next mark if its size is at least ‘min\_compress\_block\_size’. By default, 65,536.
+For [MergeTree](../../engines/table_engines/mergetree_family/mergetree.md)" tables. In order to reduce latency when processing queries, a block is compressed when writing the next mark if its size is at least ‘min\_compress\_block\_size’. By default, 65,536.
The actual size of the block, if the uncompressed data is less than ‘max\_compress\_block\_size’, is no less than this value and no less than the volume of data for one mark.
@@ -651,7 +656,7 @@ For more information, see the section “Extreme values”.
## use\_uncompressed\_cache {#setting-use_uncompressed_cache}
Whether to use a cache of uncompressed blocks. Accepts 0 or 1. By default, 0 (disabled).
-Using the uncompressed cache (only for tables in the MergeTree family) can significantly reduce latency and increase throughput when working with a large number of short queries. Enable this setting for users who send frequent short requests. Also pay attention to the [uncompressed\_cache\_size](../server_settings/settings.md#server-settings-uncompressed_cache_size) configuration parameter (only set in the config file) – the size of uncompressed cache blocks. By default, it is 8 GiB. The uncompressed cache is filled in as needed and the least-used data is automatically deleted.
+Using the uncompressed cache (only for tables in the MergeTree family) can significantly reduce latency and increase throughput when working with a large number of short queries. Enable this setting for users who send frequent short requests. Also pay attention to the [uncompressed\_cache\_size](../server_configuration_parameters/settings.md#server-settings-uncompressed_cache_size) configuration parameter (only set in the config file) – the size of uncompressed cache blocks. By default, it is 8 GiB. The uncompressed cache is filled in as needed and the least-used data is automatically deleted.
For queries that read at least a somewhat large volume of data (one million rows or more), the uncompressed cache is disabled automatically to save space for truly small queries. This means that you can keep the ‘use\_uncompressed\_cache’ setting always set to 1.
@@ -685,7 +690,7 @@ ClickHouse supports the following algorithms of choosing replicas:
- [In order](#load_balancing-in_order)
- [First or random](#load_balancing-first_or_random)
-### Random (by default) {#load_balancing-random}
+### Random (by Default) {#load_balancing-random}
``` sql
load_balancing = random
@@ -862,7 +867,7 @@ Possible values:
Default value: 1.
-By default, blocks inserted into replicated tables by the `INSERT` statement are deduplicated (see \[Data Replication\] (../ table\_engines/replication.md).
+By default, blocks inserted into replicated tables by the `INSERT` statement are deduplicated (see \[Data Replication\] (../engines/table\_engines/mergetree_family/replication.md).
## deduplicate\_blocks\_in\_dependent\_materialized\_views {#settings-deduplicate-blocks-in-dependent-materialized-views}
@@ -928,15 +933,15 @@ Default value: 0.
## count\_distinct\_implementation {#settings-count_distinct_implementation}
-Specifies which of the `uniq*` functions should be used to perform the [COUNT(DISTINCT …)](../../query_language/agg_functions/reference.md#agg_function-count) construction.
+Specifies which of the `uniq*` functions should be used to perform the [COUNT(DISTINCT …)](../../sql_reference/aggregate_functions/reference.md#agg_function-count) construction.
Possible values:
-- [uniq](../../query_language/agg_functions/reference.md#agg_function-uniq)
-- [uniqCombined](../../query_language/agg_functions/reference.md#agg_function-uniqcombined)
-- [uniqCombined64](../../query_language/agg_functions/reference.md#agg_function-uniqcombined64)
-- [uniqHLL12](../../query_language/agg_functions/reference.md#agg_function-uniqhll12)
-- [uniqExact](../../query_language/agg_functions/reference.md#agg_function-uniqexact)
+- [uniq](../../sql_reference/aggregate_functions/reference.md#agg_function-uniq)
+- [uniqCombined](../../sql_reference/aggregate_functions/reference.md#agg_function-uniqcombined)
+- [uniqCombined64](../../sql_reference/aggregate_functions/reference.md#agg_function-uniqcombined64)
+- [uniqHLL12](../../sql_reference/aggregate_functions/reference.md#agg_function-uniqhll12)
+- [uniqExact](../../sql_reference/aggregate_functions/reference.md#agg_function-uniqexact)
Default value: `uniqExact`.
@@ -1001,7 +1006,7 @@ Default value: 0.
## optimize\_throw\_if\_noop {#setting-optimize_throw_if_noop}
-Enables or disables throwing an exception if an [OPTIMIZE](../../query_language/misc.md#misc_operations-optimize) query didn’t perform a merge.
+Enables or disables throwing an exception if an [OPTIMIZE](../../sql_reference/statements/misc.md#misc_operations-optimize) query didn’t perform a merge.
By default, `OPTIMIZE` returns successfully even if it didn’t do anything. This setting lets you differentiate these situations and get the reason in an exception message.
@@ -1021,7 +1026,7 @@ Controls how fast errors in distributed tables are zeroed. If a replica is unava
See also:
-- [Table engine Distributed](../../operations/table_engines/distributed.md)
+- [Table engine Distributed](../../engines/table_engines/special/distributed.md)
- [distributed\_replica\_error\_cap](#settings-distributed_replica_error_cap)
## distributed\_replica\_error\_cap {#settings-distributed_replica_error_cap}
@@ -1033,12 +1038,12 @@ Error count of each replica is capped at this value, preventing a single replica
See also:
-- [Table engine Distributed](../../operations/table_engines/distributed.md)
+- [Table engine Distributed](../../engines/table_engines/special/distributed.md)
- [distributed\_replica\_error\_half\_life](#settings-distributed_replica_error_half_life)
## distributed\_directory\_monitor\_sleep\_time\_ms {#distributed_directory_monitor_sleep_time_ms}
-Base interval for the [Distributed](../table_engines/distributed.md) table engine to send data. The actual interval grows exponentially in the event of errors.
+Base interval for the [Distributed](../../engines/table_engines/special/distributed.md) table engine to send data. The actual interval grows exponentially in the event of errors.
Possible values:
@@ -1048,7 +1053,7 @@ Default value: 100 milliseconds.
## distributed\_directory\_monitor\_max\_sleep\_time\_ms {#distributed_directory_monitor_max_sleep_time_ms}
-Maximum interval for the [Distributed](../table_engines/distributed.md) table engine to send data. Limits exponential growth of the interval set in the [distributed\_directory\_monitor\_sleep\_time\_ms](#distributed_directory_monitor_sleep_time_ms) setting.
+Maximum interval for the [Distributed](../../engines/table_engines/special/distributed.md) table engine to send data. Limits exponential growth of the interval set in the [distributed\_directory\_monitor\_sleep\_time\_ms](#distributed_directory_monitor_sleep_time_ms) setting.
Possible values:
@@ -1060,7 +1065,7 @@ Default value: 30000 milliseconds (30 seconds).
Enables/disables sending of inserted data in batches.
-When batch sending is enabled, the [Distributed](../table_engines/distributed.md) table engine tries to send multiple files of inserted data in one operation instead of sending them separately. Batch sending improves cluster performance by better-utilizing server and network resources.
+When batch sending is enabled, the [Distributed](../../engines/table_engines/special/distributed.md) table engine tries to send multiple files of inserted data in one operation instead of sending them separately. Batch sending improves cluster performance by better-utilizing server and network resources.
Possible values:
@@ -1086,7 +1091,7 @@ Default value: 0.
## query\_profiler\_real\_time\_period\_ns {#query_profiler_real_time_period_ns}
-Sets the period for a real clock timer of the [query profiler](../../operations/performance/sampling_query_profiler.md). Real clock timer counts wall-clock time.
+Sets the period for a real clock timer of the [query profiler](../../operations/optimizing_performance/sampling_query_profiler.md). Real clock timer counts wall-clock time.
Possible values:
@@ -1099,17 +1104,17 @@ Possible values:
- 0 for turning off the timer.
-Type: [UInt64](../../data_types/int_uint.md).
+Type: [UInt64](../../sql_reference/data_types/int_uint.md).
Default value: 1000000000 nanoseconds (once a second).
See also:
-- System table [trace\_log](../system_tables.md#system_tables-trace_log)
+- System table [trace\_log](../../operations/system_tables.md#system_tables-trace_log)
## query\_profiler\_cpu\_time\_period\_ns {#query_profiler_cpu_time_period_ns}
-Sets the period for a CPU clock timer of the [query profiler](../../operations/performance/sampling_query_profiler.md). This timer counts only CPU time.
+Sets the period for a CPU clock timer of the [query profiler](../../operations/optimizing_performance/sampling_query_profiler.md). This timer counts only CPU time.
Possible values:
@@ -1122,17 +1127,17 @@ Possible values:
- 0 for turning off the timer.
-Type: [UInt64](../../data_types/int_uint.md).
+Type: [UInt64](../../sql_reference/data_types/int_uint.md).
Default value: 1000000000 nanoseconds.
See also:
-- System table [trace\_log](../system_tables.md#system_tables-trace_log)
+- System table [trace\_log](../../operations/system_tables.md#system_tables-trace_log)
## allow\_introspection\_functions {#settings-allow_introspection_functions}
-Enables of disables [introspections functions](../../query_language/functions/introspection.md) for query profiling.
+Enables of disables [introspections functions](../../sql_reference/functions/introspection.md) for query profiling.
Possible values:
@@ -1143,8 +1148,8 @@ Default value: 0.
**See Also**
-- [Sampling Query Profiler](../performance/sampling_query_profiler.md)
-- System table [trace\_log](../system_tables.md#system_tables-trace_log)
+- [Sampling Query Profiler](../optimizing_performance/sampling_query_profiler.md)
+- System table [trace\_log](../../operations/system_tables.md#system_tables-trace_log)
## input\_format\_parallel\_parsing {#input-format-parallel-parsing}
diff --git a/docs/en/operations/settings/settings_profiles.md b/docs/en/operations/settings/settings_profiles.md
index 09976eda14e..ee7aef922ea 100644
--- a/docs/en/operations/settings/settings_profiles.md
+++ b/docs/en/operations/settings/settings_profiles.md
@@ -1,3 +1,8 @@
+---
+toc_priority: 61
+toc_title: Settings Profiles
+---
+
# Settings Profiles {#settings-profiles}
A settings profile is a collection of settings grouped under the same name. Each ClickHouse user has a profile.
diff --git a/docs/en/operations/settings/settings_users.md b/docs/en/operations/settings/settings_users.md
index 08bd5a4c4e7..4b398796ea8 100644
--- a/docs/en/operations/settings/settings_users.md
+++ b/docs/en/operations/settings/settings_users.md
@@ -1,3 +1,8 @@
+---
+toc_priority: 63
+toc_title: User Settings
+---
+
# User Settings {#user-settings}
The `users` section of the `user.xml` configuration file contains user settings.
@@ -136,6 +141,6 @@ The following configuration forces that user `user1` can only see the rows of `t
```
-The `filter` can be any expression resulting in a [UInt8](../../data_types/int_uint.md)-type value. It usually contains comparisons and logical operators. Rows from `database_name.table1` where filter results to 0 are not returned for this user. The filtering is incompatible with `PREWHERE` operations and disables `WHERE→PREWHERE` optimization.
+The `filter` can be any expression resulting in a [UInt8](../../sql_reference/data_types/int_uint.md)-type value. It usually contains comparisons and logical operators. Rows from `database_name.table1` where filter results to 0 are not returned for this user. The filtering is incompatible with `PREWHERE` operations and disables `WHERE→PREWHERE` optimization.
[Original article](https://clickhouse.tech/docs/en/operations/settings/settings_users/)
diff --git a/docs/en/operations/system_tables.md b/docs/en/operations/system_tables.md
index 4bd58052196..360cf3ac711 100644
--- a/docs/en/operations/system_tables.md
+++ b/docs/en/operations/system_tables.md
@@ -1,4 +1,9 @@
-# System tables {#system-tables}
+---
+toc_priority: 52
+toc_title: System Tables
+---
+
+# System Tables {#system-tables}
System tables are used for implementing part of the system’s functionality, and for providing access to information about how the system is working.
You can’t delete a system table (but you can perform DETACH).
@@ -12,8 +17,8 @@ Contains metrics that are calculated periodically in the background. For example
Columns:
-- `metric` ([String](../data_types/string.md)) — Metric name.
-- `value` ([Float64](../data_types/float.md)) — Metric value.
+- `metric` ([String](../sql_reference/data_types/string.md)) — Metric name.
+- `value` ([Float64](../sql_reference/data_types/float.md)) — Metric value.
**Example**
@@ -64,7 +69,7 @@ Please note that `errors_count` is updated once per query to the cluster, but `e
**See also**
-- [Table engine Distributed](table_engines/distributed.md)
+- [Table engine Distributed](../engines/table_engines/special/distributed.md)
- [distributed\_replica\_error\_cap setting](settings/settings.md#settings-distributed_replica_error_cap)
- [distributed\_replica\_error\_half\_life setting](settings/settings.md#settings-distributed_replica_error_half_life)
@@ -72,7 +77,7 @@ Please note that `errors_count` is updated once per query to the cluster, but `e
Contains information about columns in all the tables.
-You can use this table to get information similar to the [DESCRIBE TABLE](../query_language/misc.md#misc-describe-table) query, but for multiple tables at once.
+You can use this table to get information similar to the [DESCRIBE TABLE](../sql_reference/statements/misc.md#misc-describe-table) query, but for multiple tables at once.
The `system.columns` table contains the following columns (the column type is shown in brackets):
@@ -140,7 +145,7 @@ This system table is used for implementing the `SHOW DATABASES` query.
## system.detached\_parts {#system_tables-detached_parts}
-Contains information about detached parts of [MergeTree](table_engines/mergetree.md) tables. The `reason` column specifies why the part was detached. For user-detached parts, the reason is empty. Such parts can be attached with [ALTER TABLE ATTACH PARTITION\|PART](../query_language/query_language/alter/#alter_attach-partition) command. For the description of other columns, see [system.parts](#system_tables-parts). If part name is invalid, values of some columns may be `NULL`. Such parts can be deleted with [ALTER TABLE DROP DETACHED PART](../query_language/query_language/alter/#alter_drop-detached).
+Contains information about detached parts of [MergeTree](../engines/table_engines/mergetree_family/mergetree.md) tables. The `reason` column specifies why the part was detached. For user-detached parts, the reason is empty. Such parts can be attached with [ALTER TABLE ATTACH PARTITION\|PART](../query_language/query_language/alter/#alter_attach-partition) command. For the description of other columns, see [system.parts](#system_tables-parts). If part name is invalid, values of some columns may be `NULL`. Such parts can be deleted with [ALTER TABLE DROP DETACHED PART](../query_language/query_language/alter/#alter_drop-detached).
## system.dictionaries {#system-dictionaries}
@@ -170,9 +175,9 @@ Contains information about the number of events that have occurred in the system
Columns:
-- `event` ([String](../data_types/string.md)) — Event name.
-- `value` ([UInt64](../data_types/int_uint.md)) — Number of events occurred.
-- `description` ([String](../data_types/string.md)) — Event description.
+- `event` ([String](../sql_reference/data_types/string.md)) — Event name.
+- `value` ([UInt64](../sql_reference/data_types/int_uint.md)) — Number of events occurred.
+- `description` ([String](../sql_reference/data_types/string.md)) — Event description.
**Example**
@@ -208,7 +213,7 @@ Columns:
## system.graphite\_retentions {#system-graphite-retentions}
-Contains information about parameters [graphite\_rollup](server_settings/settings.md#server_settings-graphite_rollup) which are used in tables with [\*GraphiteMergeTree](table_engines/graphitemergetree.md) engines.
+Contains information about parameters [graphite\_rollup](server_configuration_parameters/settings.md#server_configuration_parameters-graphite_rollup) which are used in tables with [\*GraphiteMergeTree](../engines/table_engines/mergetree_family/graphitemergetree.md) engines.
Columns:
@@ -248,9 +253,9 @@ Contains metrics which can be calculated instantly, or have a current value. For
Columns:
-- `metric` ([String](../data_types/string.md)) — Metric name.
-- `value` ([Int64](../data_types/int_uint.md)) — Metric value.
-- `description` ([String](../data_types/string.md)) — Metric description.
+- `metric` ([String](../sql_reference/data_types/string.md)) — Metric name.
+- `value` ([Int64](../sql_reference/data_types/int_uint.md)) — Metric value.
+- `description` ([String](../sql_reference/data_types/string.md)) — Metric description.
The list of supported metrics you can find in the [dbms/Common/CurrentMetrics.cpp](https://github.com/ClickHouse/ClickHouse/blob/master/dbms/Common/CurrentMetrics.cpp) source file of ClickHouse.
@@ -357,13 +362,13 @@ This is similar to the DUAL table found in other DBMSs.
## system.parts {#system_tables-parts}
-Contains information about parts of [MergeTree](table_engines/mergetree.md) tables.
+Contains information about parts of [MergeTree](../engines/table_engines/mergetree_family/mergetree.md) tables.
Each row describes one data part.
Columns:
-- `partition` (String) – The partition name. To learn what a partition is, see the description of the [ALTER](../query_language/alter.md#query_language_queries_alter) query.
+- `partition` (String) – The partition name. To learn what a partition is, see the description of the [ALTER](../sql_reference/statements/alter.md#query_language_queries_alter) query.
Formats:
@@ -414,7 +419,7 @@ Columns:
- `primary_key_bytes_in_memory_allocated` (`UInt64`) – The amount of memory (in bytes) reserved for primary key values.
-- `is_frozen` (`UInt8`) – Flag that shows that a partition data backup exists. 1, the backup exists. 0, the backup doesn’t exist. For more details, see [FREEZE PARTITION](../query_language/alter.md#alter_freeze-partition)
+- `is_frozen` (`UInt8`) – Flag that shows that a partition data backup exists. 1, the backup exists. 0, the backup doesn’t exist. For more details, see [FREEZE PARTITION](../sql_reference/statements/alter.md#alter_freeze-partition)
- `database` (`String`) – Name of the database.
@@ -426,11 +431,11 @@ Columns:
- `disk` (`String`) – Name of a disk that stores the data part.
-- `hash_of_all_files` (`String`) – [sipHash128](../query_language/functions/hash_functions.md#hash_functions-siphash128) of compressed files.
+- `hash_of_all_files` (`String`) – [sipHash128](../sql_reference/functions/hash_functions.md#hash_functions-siphash128) of compressed files.
-- `hash_of_uncompressed_files` (`String`) – [sipHash128](../query_language/functions/hash_functions.md#hash_functions-siphash128) of uncompressed files (files with marks, index file etc.).
+- `hash_of_uncompressed_files` (`String`) – [sipHash128](../sql_reference/functions/hash_functions.md#hash_functions-siphash128) of uncompressed files (files with marks, index file etc.).
-- `uncompressed_hash_of_compressed_files` (`String`) – [sipHash128](../query_language/functions/hash_functions.md#hash_functions-siphash128) of data in the compressed files as if they were uncompressed.
+- `uncompressed_hash_of_compressed_files` (`String`) – [sipHash128](../sql_reference/functions/hash_functions.md#hash_functions-siphash128) of data in the compressed files as if they were uncompressed.
- `bytes` (`UInt64`) – Alias for `bytes_on_disk`.
@@ -438,9 +443,9 @@ Columns:
## system.part\_log {#system_tables-part-log}
-The `system.part_log` table is created only if the [part\_log](server_settings/settings.md#server_settings-part-log) server setting is specified.
+The `system.part_log` table is created only if the [part\_log](server_configuration_parameters/settings.md#server_configuration_parameters-part-log) server setting is specified.
-This table contains information about events that occurred with [data parts](table_engines/custom_partitioning_key.md) in the [MergeTree](table_engines/mergetree.md) family tables, such as adding or merging data.
+This table contains information about events that occurred with [data parts](../engines/table_engines/mergetree_family/custom_partitioning_key.md) in the [MergeTree](../engines/table_engines/mergetree_family/mergetree.md) family tables, such as adding or merging data.
The `system.part_log` table contains the following columns:
@@ -448,7 +453,7 @@ The `system.part_log` table contains the following columns:
- `NEW_PART` — Inserting of a new data part.
- `MERGE_PARTS` — Merging of data parts.
- `DOWNLOAD_PART` — Downloading a data part.
- - `REMOVE_PART` — Removing or detaching a data part using [DETACH PARTITION](../query_language/alter.md#alter_detach-partition).
+ - `REMOVE_PART` — Removing or detaching a data part using [DETACH PARTITION](../sql_reference/statements/alter.md#alter_detach-partition).
- `MUTATE_PART` — Mutating of a data part.
- `MOVE_PART` — Moving the data part from the one disk to another one.
- `event_date` (Date) — Event date.
@@ -519,7 +524,7 @@ Contains information about execution of queries. For each query, you can see pro
!!! note "Note"
The table doesn’t contain input data for `INSERT` queries.
-ClickHouse creates this table only if the [query\_log](server_settings/settings.md#server_settings-query-log) server parameter is specified. This parameter sets the logging rules, such as the logging interval or the name of the table the queries will be logged in.
+ClickHouse creates this table only if the [query\_log](server_configuration_parameters/settings.md#server_configuration_parameters-query-log) server parameter is specified. This parameter sets the logging rules, such as the logging interval or the name of the table the queries will be logged in.
To enable query logging, set the [log\_queries](settings/settings.md#settings-log-queries) parameter to 1. For details, see the [Settings](settings/settings.md) section.
@@ -589,20 +594,20 @@ Each query creates one or two rows in the `query_log` table, depending on the st
2. If an error occurred during query processing, two events with types 1 and 4 are created.
3. If an error occurred before launching the query, a single event with type 3 is created.
-By default, logs are added to the table at intervals of 7.5 seconds. You can set this interval in the [query\_log](server_settings/settings.md#server_settings-query-log) server setting (see the `flush_interval_milliseconds` parameter). To flush the logs forcibly from the memory buffer into the table, use the `SYSTEM FLUSH LOGS` query.
+By default, logs are added to the table at intervals of 7.5 seconds. You can set this interval in the [query\_log](server_configuration_parameters/settings.md#server_configuration_parameters-query-log) server setting (see the `flush_interval_milliseconds` parameter). To flush the logs forcibly from the memory buffer into the table, use the `SYSTEM FLUSH LOGS` query.
When the table is deleted manually, it will be automatically created on the fly. Note that all the previous logs will be deleted.
!!! note "Note"
The storage period for logs is unlimited. Logs aren’t automatically deleted from the table. You need to organize the removal of outdated logs yourself.
-You can specify an arbitrary partitioning key for the `system.query_log` table in the [query\_log](server_settings/settings.md#server_settings-query-log) server setting (see the `partition_by` parameter).
+You can specify an arbitrary partitioning key for the `system.query_log` table in the [query\_log](server_configuration_parameters/settings.md#server_configuration_parameters-query-log) server setting (see the `partition_by` parameter).
## system.query\_thread\_log {#system_tables-query-thread-log}
The table contains information about each query execution thread.
-ClickHouse creates this table only if the [query\_thread\_log](server_settings/settings.md#server_settings-query-thread-log) server parameter is specified. This parameter sets the logging rules, such as the logging interval or the name of the table the queries will be logged in.
+ClickHouse creates this table only if the [query\_thread\_log](server_configuration_parameters/settings.md#server_configuration_parameters-query-thread-log) server parameter is specified. This parameter sets the logging rules, such as the logging interval or the name of the table the queries will be logged in.
To enable query logging, set the [log\_query\_threads](settings/settings.md#settings-log-query-threads) parameter to 1. For details, see the [Settings](settings/settings.md) section.
@@ -654,43 +659,43 @@ Columns:
- `ProfileEvents.Names` (Array(String)) — Counters that measure different metrics for this thread. The description of them could be found in the table [system.events](#system_tables-events)
- `ProfileEvents.Values` (Array(UInt64)) — Values of metrics for this thread that are listed in the `ProfileEvents.Names` column.
-By default, logs are added to the table at intervals of 7.5 seconds. You can set this interval in the [query\_thread\_log](server_settings/settings.md#server_settings-query-thread-log) server setting (see the `flush_interval_milliseconds` parameter). To flush the logs forcibly from the memory buffer into the table, use the `SYSTEM FLUSH LOGS` query.
+By default, logs are added to the table at intervals of 7.5 seconds. You can set this interval in the [query\_thread\_log](server_configuration_parameters/settings.md#server_configuration_parameters-query-thread-log) server setting (see the `flush_interval_milliseconds` parameter). To flush the logs forcibly from the memory buffer into the table, use the `SYSTEM FLUSH LOGS` query.
When the table is deleted manually, it will be automatically created on the fly. Note that all the previous logs will be deleted.
!!! note "Note"
The storage period for logs is unlimited. Logs aren’t automatically deleted from the table. You need to organize the removal of outdated logs yourself.
-You can specify an arbitrary partitioning key for the `system.query_thread_log` table in the [query\_thread\_log](server_settings/settings.md#server_settings-query-thread-log) server setting (see the `partition_by` parameter).
+You can specify an arbitrary partitioning key for the `system.query_thread_log` table in the [query\_thread\_log](server_configuration_parameters/settings.md#server_configuration_parameters-query-thread-log) server setting (see the `partition_by` parameter).
## system.trace\_log {#system_tables-trace_log}
Contains stack traces collected by the sampling query profiler.
-ClickHouse creates this table when the [trace\_log](server_settings/settings.md#server_settings-trace_log) server configuration section is set. Also the [query\_profiler\_real\_time\_period\_ns](settings/settings.md#query_profiler_real_time_period_ns) and [query\_profiler\_cpu\_time\_period\_ns](settings/settings.md#query_profiler_cpu_time_period_ns) settings should be set.
+ClickHouse creates this table when the [trace\_log](server_configuration_parameters/settings.md#server_configuration_parameters-trace_log) server configuration section is set. Also the [query\_profiler\_real\_time\_period\_ns](settings/settings.md#query_profiler_real_time_period_ns) and [query\_profiler\_cpu\_time\_period\_ns](settings/settings.md#query_profiler_cpu_time_period_ns) settings should be set.
To analyze logs, use the `addressToLine`, `addressToSymbol` and `demangle` introspection functions.
Columns:
-- `event_date`([Date](../data_types/date.md)) — Date of sampling moment.
+- `event_date`([Date](../sql_reference/data_types/date.md)) — Date of sampling moment.
-- `event_time`([DateTime](../data_types/datetime.md)) — Timestamp of sampling moment.
+- `event_time`([DateTime](../sql_reference/data_types/datetime.md)) — Timestamp of sampling moment.
-- `revision`([UInt32](../data_types/int_uint.md)) — ClickHouse server build revision.
+- `revision`([UInt32](../sql_reference/data_types/int_uint.md)) — ClickHouse server build revision.
When connecting to server by `clickhouse-client`, you see the string similar to `Connected to ClickHouse server version 19.18.1 revision 54429.`. This field contains the `revision`, but not the `version` of a server.
-- `timer_type`([Enum8](../data_types/enum.md)) — Timer type:
+- `timer_type`([Enum8](../sql_reference/data_types/enum.md)) — Timer type:
- `Real` represents wall-clock time.
- `CPU` represents CPU time.
-- `thread_number`([UInt32](../data_types/int_uint.md)) — Thread identifier.
+- `thread_number`([UInt32](../sql_reference/data_types/int_uint.md)) — Thread identifier.
-- `query_id`([String](../data_types/string.md)) — Query identifier that can be used to get details about a query that was running from the [query\_log](#system_tables-query_log) system table.
+- `query_id`([String](../sql_reference/data_types/string.md)) — Query identifier that can be used to get details about a query that was running from the [query\_log](#system_tables-query_log) system table.
-- `trace`([Array(UInt64)](../data_types/array.md)) — Stack trace at the moment of sampling. Each element is a virtual memory address inside ClickHouse server process.
+- `trace`([Array(UInt64)](../sql_reference/data_types/array.md)) — Stack trace at the moment of sampling. Each element is a virtual memory address inside ClickHouse server process.
**Example**
@@ -885,10 +890,10 @@ This table contains the following columns (the column type is shown in brackets)
- `name` (String) — The name of table engine.
- `supports_settings` (UInt8) — Flag that indicates if table engine supports `SETTINGS` clause.
-- `supports_skipping_indices` (UInt8) — Flag that indicates if table engine supports [skipping indices](table_engines/mergetree/#table_engine-mergetree-data_skipping-indexes).
-- `supports_ttl` (UInt8) — Flag that indicates if table engine supports [TTL](table_engines/mergetree/#table_engine-mergetree-ttl).
+- `supports_skipping_indices` (UInt8) — Flag that indicates if table engine supports [skipping indices](../engines/table_engines/mergetree_family/mergetree.md#table_engine-mergetree-data_skipping-indexes).
+- `supports_ttl` (UInt8) — Flag that indicates if table engine supports [TTL](../engines/table_engines/mergetree_family/mergetree.md#table_engine-mergetree-ttl).
- `supports_sort_order` (UInt8) — Flag that indicates if table engine supports clauses `PARTITION_BY`, `PRIMARY_KEY`, `ORDER_BY` and `SAMPLE_BY`.
-- `supports_replication` (UInt8) — Flag that indicates if table engine supports [data replication](table_engines/replication/).
+- `supports_replication` (UInt8) — Flag that indicates if table engine supports [data replication](../engines/table_engines/mergetree_family/replication.md).
- `supports_duduplication` (UInt8) — Flag that indicates if table engine supports data deduplication.
Example:
@@ -909,9 +914,9 @@ WHERE name in ('Kafka', 'MergeTree', 'ReplicatedCollapsingMergeTree')
**See also**
-- MergeTree family [query clauses](table_engines/mergetree.md#mergetree-query-clauses)
-- Kafka [settings](table_engines/kafka.md#table_engine-kafka-creating-a-table)
-- Join [settings](table_engines/join.md#join-limitations-and-settings)
+- MergeTree family [query clauses](../engines/table_engines/mergetree_family/mergetree.md#mergetree-query-clauses)
+- Kafka [settings](../engines/table_engines/integrations/kafka.md#table_engine-kafka-creating-a-table)
+- Join [settings](../engines/table_engines/special/join.md#join-limitations-and-settings)
## system.tables {#system-tables}
@@ -920,30 +925,46 @@ Contains metadata of each table that the server knows about. Detached tables are
This table contains the following columns (the column type is shown in brackets):
- `database` (String) — The name of the database the table is in.
+
- `name` (String) — Table name.
+
- `engine` (String) — Table engine name (without parameters).
+
- `is_temporary` (UInt8) - Flag that indicates whether the table is temporary.
+
- `data_path` (String) - Path to the table data in the file system.
+
- `metadata_path` (String) - Path to the table metadata in the file system.
+
- `metadata_modification_time` (DateTime) - Time of latest modification of the table metadata.
+
- `dependencies_database` (Array(String)) - Database dependencies.
-- `dependencies_table` (Array(String)) - Table dependencies ([MaterializedView](table_engines/materializedview.md) tables based on the current table).
+
+- `dependencies_table` (Array(String)) - Table dependencies ([MaterializedView](../engines/table_engines/special/materializedview.md) tables based on the current table).
+
- `create_table_query` (String) - The query that was used to create the table.
+
- `engine_full` (String) - Parameters of the table engine.
+
- `partition_key` (String) - The partition key expression specified in the table.
+
- `sorting_key` (String) - The sorting key expression specified in the table.
+
- `primary_key` (String) - The primary key expression specified in the table.
+
- `sampling_key` (String) - The sampling key expression specified in the table.
+
- `storage_policy` (String) - The storage policy:
- - [MergeTree](table_engines/mergetree.md#table_engine-mergetree-multiple-volumes)
- - [Distributed](table_engines/distributed.md#distributed)
+ - [MergeTree](../engines/table_engines/mergetree_family/mergetree.md#table_engine-mergetree-multiple-volumes)
+ - [Distributed](../engines/table_engines/special/distributed.md#distributed)
- `total_rows` (Nullable(UInt64)) - Total number of rows, if it is possible to quickly determine exact number of rows in the table, otherwise `Null` (including underying `Buffer` table).
+
- `total_bytes` (Nullable(UInt64)) - Total number of bytes, if it is possible to quickly determine exact number of bytes for the table on storage, otherwise `Null` (**does not** includes any underlying storage).
- - If the table stores data on disk, returns used space on disk (i.e. compressed).
- - If the table stores data in memory, returns approximated number of used bytes in memory.
+ - If the table stores data on disk, returns used space on disk (i.e. compressed).
+ - If the table stores data in memory, returns approximated number of used bytes in memory.
The `system.tables` table is used in `SHOW TABLES` query implementation.
@@ -1020,7 +1041,7 @@ path: /clickhouse/tables/01-08/visits/replicas
## system.mutations {#system_tables-mutations}
-The table contains information about [mutations](../query_language/alter.md#alter-mutations) of MergeTree tables and their progress. Each mutation command is represented by a single row. The table has the following columns:
+The table contains information about [mutations](../sql_reference/statements/alter.md#alter-mutations) of MergeTree tables and their progress. Each mutation command is represented by a single row. The table has the following columns:
**database**, **table** - The name of the database and table to which the mutation was applied.
@@ -1046,28 +1067,28 @@ If there were problems with mutating some parts, the following columns contain a
## system.disks {#system_tables-disks}
-Contains information about disks defined in the [server configuration](table_engines/mergetree.md#table_engine-mergetree-multiple-volumes_configure).
+Contains information about disks defined in the [server configuration](../engines/table_engines/mergetree_family/mergetree.md#table_engine-mergetree-multiple-volumes_configure).
Columns:
-- `name` ([String](../data_types/string.md)) — Name of a disk in the server configuration.
-- `path` ([String](../data_types/string.md)) — Path to the mount point in the file system.
-- `free_space` ([UInt64](../data_types/int_uint.md)) — Free space on disk in bytes.
-- `total_space` ([UInt64](../data_types/int_uint.md)) — Disk volume in bytes.
-- `keep_free_space` ([UInt64](../data_types/int_uint.md)) — Amount of disk space that should stay free on disk in bytes. Defined in the `keep_free_space_bytes` parameter of disk configuration.
+- `name` ([String](../sql_reference/data_types/string.md)) — Name of a disk in the server configuration.
+- `path` ([String](../sql_reference/data_types/string.md)) — Path to the mount point in the file system.
+- `free_space` ([UInt64](../sql_reference/data_types/int_uint.md)) — Free space on disk in bytes.
+- `total_space` ([UInt64](../sql_reference/data_types/int_uint.md)) — Disk volume in bytes.
+- `keep_free_space` ([UInt64](../sql_reference/data_types/int_uint.md)) — Amount of disk space that should stay free on disk in bytes. Defined in the `keep_free_space_bytes` parameter of disk configuration.
## system.storage\_policies {#system_tables-storage_policies}
-Contains information about storage policies and volumes defined in the [server configuration](table_engines/mergetree.md#table_engine-mergetree-multiple-volumes_configure).
+Contains information about storage policies and volumes defined in the [server configuration](../engines/table_engines/mergetree_family/mergetree.md#table_engine-mergetree-multiple-volumes_configure).
Columns:
-- `policy_name` ([String](../data_types/string.md)) — Name of the storage policy.
-- `volume_name` ([String](../data_types/string.md)) — Volume name defined in the storage policy.
-- `volume_priority` ([UInt64](../data_types/int_uint.md)) — Volume order number in the configuration.
-- `disks` ([Array(String)](../data_types/array.md)) — Disk names, defined in the storage policy.
-- `max_data_part_size` ([UInt64](../data_types/int_uint.md)) — Maximum size of a data part that can be stored on volume disks (0 — no limit).
-- `move_factor` ([Float64](../data_types/float.md)) — Ratio of free disk space. When the ratio exceeds the value of configuration parameter, ClickHouse start to move data to the next volume in order.
+- `policy_name` ([String](../sql_reference/data_types/string.md)) — Name of the storage policy.
+- `volume_name` ([String](../sql_reference/data_types/string.md)) — Volume name defined in the storage policy.
+- `volume_priority` ([UInt64](../sql_reference/data_types/int_uint.md)) — Volume order number in the configuration.
+- `disks` ([Array(String)](../sql_reference/data_types/array.md)) — Disk names, defined in the storage policy.
+- `max_data_part_size` ([UInt64](../sql_reference/data_types/int_uint.md)) — Maximum size of a data part that can be stored on volume disks (0 — no limit).
+- `move_factor` ([Float64](../sql_reference/data_types/float.md)) — Ratio of free disk space. When the ratio exceeds the value of configuration parameter, ClickHouse start to move data to the next volume in order.
If the storage policy contains more then one volume, then information for each volume is stored in the individual row of the table.
diff --git a/docs/en/operations/table_engines/index.md b/docs/en/operations/table_engines/index.md
deleted file mode 100644
index d8bad2b5841..00000000000
--- a/docs/en/operations/table_engines/index.md
+++ /dev/null
@@ -1,77 +0,0 @@
-# Table engines {#table_engines}
-
-The table engine (type of table) determines:
-
-- How and where data is stored, where to write it to, and where to read it from.
-- Which queries are supported, and how.
-- Concurrent data access.
-- Use of indexes, if present.
-- Whether multithreaded request execution is possible.
-- Data replication parameters.
-
-## Engine Families {#engine-families}
-
-### MergeTree {#mergetree}
-
-The most universal and functional table engines for high-load tasks. The property shared by these engines is quick data insertion with subsequent background data processing. `MergeTree` family engines support data replication (with [Replicated\*](replication.md) versions of engines), partitioning, and other features not supported in other engines.
-
-Engines in the family:
-
-- [MergeTree](mergetree.md)
-- [ReplacingMergeTree](replacingmergetree.md)
-- [SummingMergeTree](summingmergetree.md)
-- [AggregatingMergeTree](aggregatingmergetree.md)
-- [CollapsingMergeTree](collapsingmergetree.md)
-- [VersionedCollapsingMergeTree](versionedcollapsingmergetree.md)
-- [GraphiteMergeTree](graphitemergetree.md)
-
-### Log {#log}
-
-Lightweight [engines](log_family.md) with minimum functionality. They’re the most effective when you need to quickly write many small tables (up to approximately 1 million rows) and read them later as a whole.
-
-Engines in the family:
-
-- [TinyLog](tinylog.md)
-- [StripeLog](stripelog.md)
-- [Log](log.md)
-
-### Integration engines {#integration-engines}
-
-Engines for communicating with other data storage and processing systems.
-
-Engines in the family:
-
-- [Kafka](kafka.md)
-- [MySQL](mysql.md)
-- [ODBC](odbc.md)
-- [JDBC](jdbc.md)
-- [HDFS](hdfs.md)
-
-### Special engines {#special-engines}
-
-Engines in the family:
-
-- [Distributed](distributed.md)
-- [MaterializedView](materializedview.md)
-- [Dictionary](dictionary.md)
-- [Merge](merge.md)
-- [File](file.md)
-- [Null](null.md)
-- [Set](set.md)
-- [Join](join.md)
-- [URL](url.md)
-- [View](view.md)
-- [Memory](memory.md)
-- [Buffer](buffer.md)
-
-## Virtual columns {#table_engines-virtual-columns}
-
-Virtual column is an integral table engine attribute that is defined in the engine source code.
-
-You shouldn’t specify virtual columns in the `CREATE TABLE` query and you can’t see them in `SHOW CREATE TABLE` and `DESCRIBE TABLE` query results. Virtual columns are also read-only, so you can’t insert data into virtual columns.
-
-To select data from a virtual column, you must specify its name in the `SELECT` query. `SELECT *` doesn’t return values from virtual columns.
-
-If you create a table with a column that has the same name as one of the table virtual columns, the virtual column becomes inaccessible. We don’t recommend doing this. To help avoid conflicts, virtual column names are usually prefixed with an underscore.
-
-[Original article](https://clickhouse.tech/docs/en/operations/table_engines/)
diff --git a/docs/en/operations/table_engines/materializedview.md b/docs/en/operations/table_engines/materializedview.md
deleted file mode 100644
index c12b9512c79..00000000000
--- a/docs/en/operations/table_engines/materializedview.md
+++ /dev/null
@@ -1,5 +0,0 @@
-# MaterializedView {#materializedview}
-
-Used for implementing materialized views (for more information, see [CREATE TABLE](../../query_language/create.md)). For storing data, it uses a different engine that was specified when creating the view. When reading from a table, it just uses this engine.
-
-[Original article](https://clickhouse.tech/docs/en/operations/table_engines/materializedview/)
diff --git a/docs/en/operations/tips.md b/docs/en/operations/tips.md
index b18ee5384bd..b18daedf3d6 100644
--- a/docs/en/operations/tips.md
+++ b/docs/en/operations/tips.md
@@ -1,3 +1,8 @@
+---
+toc_priority: 58
+toc_title: Usage Recommendations
+---
+
# Usage Recommendations {#usage-recommendations}
## CPU Scaling Governor {#cpu-scaling-governor}
diff --git a/docs/en/operations/troubleshooting.md b/docs/en/operations/troubleshooting.md
index 8520a0e5ca4..3cfcee39a22 100644
--- a/docs/en/operations/troubleshooting.md
+++ b/docs/en/operations/troubleshooting.md
@@ -1,3 +1,8 @@
+---
+toc_priority: 46
+toc_title: Troubleshooting
+---
+
# Troubleshooting {#troubleshooting}
- [Installation](#troubleshooting-installation-errors)
@@ -7,12 +12,12 @@
## Installation {#troubleshooting-installation-errors}
-### You Cannot Get Deb Packages from ClickHouse Repository With apt-get {#you-cannot-get-deb-packages-from-clickhouse-repository-with-apt-get}
+### You Cannot Get Deb Packages From Clickhouse Repository With Apt-get {#you-cannot-get-deb-packages-from-clickhouse-repository-with-apt-get}
- Check firewall settings.
- If you cannot access the repository for any reason, download packages as described in the [Getting started](../getting_started/index.md) article and install them manually using the `sudo dpkg -i ` command. You will also need the `tzdata` package.
-## Connecting to the Server {#troubleshooting-accepts-no-connections}
+## Connecting To the Server {#troubleshooting-accepts-no-connections}
Possible issues:
@@ -98,7 +103,7 @@ Check:
- Endpoint settings.
- Check [listen\_host](server_settings/settings.md#server_settings-listen_host) and [tcp\_port](server_settings/settings.md#server_settings-tcp_port) settings.
+ Check [listen\_host](server_configuration_parameters/settings.md#server_configuration_parameters-listen_host) and [tcp\_port](server_configuration_parameters/settings.md#server_configuration_parameters-tcp_port) settings.
ClickHouse server accepts localhost connections only by default.
@@ -110,8 +115,8 @@ Check:
Check:
- - The [tcp\_port\_secure](server_settings/settings.md#server_settings-tcp_port_secure) setting.
- - Settings for [SSL sertificates](server_settings/settings.md#server_settings-openssl).
+ - The [tcp\_port\_secure](server_configuration_parameters/settings.md#server_configuration_parameters-tcp_port_secure) setting.
+ - Settings for [SSL sertificates](server_configuration_parameters/settings.md#server_configuration_parameters-openssl).
Use proper parameters while connecting. For example, use the `port_secure` parameter with `clickhouse_client`.
@@ -132,7 +137,7 @@ If you start `clickhouse-client` with the `stack-trace` parameter, ClickHouse re
You might see a message about a broken connection. In this case, you can repeat the query. If the connection breaks every time you perform the query, check the server logs for errors.
-## Efficiency of Query Processing {#troubleshooting-too-slow}
+## Efficiency Of Query Processing {#troubleshooting-too-slow}
If you see that ClickHouse is working too slowly, you need to profile the load on the server resources and network for your queries.
diff --git a/docs/en/operations/update.md b/docs/en/operations/update.md
index a5594b7a34f..1131703765b 100644
--- a/docs/en/operations/update.md
+++ b/docs/en/operations/update.md
@@ -1,3 +1,8 @@
+---
+toc_priority: 47
+toc_title: ClickHouse Update
+---
+
# ClickHouse Update {#clickhouse-update}
If ClickHouse was installed from deb packages, execute the following commands on the server:
diff --git a/docs/en/operations/utils/clickhouse-benchmark.md b/docs/en/operations/utilities/clickhouse-benchmark.md
similarity index 99%
rename from docs/en/operations/utils/clickhouse-benchmark.md
rename to docs/en/operations/utilities/clickhouse-benchmark.md
index b85277e5000..8435e37b4b5 100644
--- a/docs/en/operations/utils/clickhouse-benchmark.md
+++ b/docs/en/operations/utilities/clickhouse-benchmark.md
@@ -1,3 +1,8 @@
+---
+toc_priority: 61
+toc_title: clickhouse-benchmark
+---
+
# clickhouse-benchmark {#clickhouse-benchmark}
Connects to a ClickHouse server and repeatedly sends specified queries.
diff --git a/docs/en/operations/utils/clickhouse-copier.md b/docs/en/operations/utilities/clickhouse-copier.md
similarity index 99%
rename from docs/en/operations/utils/clickhouse-copier.md
rename to docs/en/operations/utilities/clickhouse-copier.md
index a96e1709b14..d519940b8d9 100644
--- a/docs/en/operations/utils/clickhouse-copier.md
+++ b/docs/en/operations/utilities/clickhouse-copier.md
@@ -1,3 +1,8 @@
+---
+toc_priority: 59
+toc_title: clickhouse-copier
+---
+
# clickhouse-copier {#clickhouse-copier}
Copies data from the tables in one cluster to tables in another (or the same) cluster.
diff --git a/docs/en/operations/utils/clickhouse-local.md b/docs/en/operations/utilities/clickhouse-local.md
similarity index 96%
rename from docs/en/operations/utils/clickhouse-local.md
rename to docs/en/operations/utilities/clickhouse-local.md
index bc08eed23ad..38f28b432a3 100644
--- a/docs/en/operations/utils/clickhouse-local.md
+++ b/docs/en/operations/utilities/clickhouse-local.md
@@ -1,8 +1,13 @@
+---
+toc_priority: 60
+toc_title: clickhouse-local
+---
+
# clickhouse-local {#clickhouse-local}
The `clickhouse-local` program enables you to perform fast processing on local files, without having to deploy and configure the ClickHouse server.
-Accepts data that represent tables and queries them using [ClickHouse SQL dialect](../../query_language/index.md).
+Accepts data that represent tables and queries them using [ClickHouse SQL dialect](../../sql_reference/index.md).
`clickhouse-local` uses the same core as ClickHouse server, so it supports most of the features and the same set of formats and table engines.
diff --git a/docs/en/operations/utils/index.md b/docs/en/operations/utilities/index.md
similarity index 87%
rename from docs/en/operations/utils/index.md
rename to docs/en/operations/utilities/index.md
index ebc1396d031..eb5fe33f1fe 100644
--- a/docs/en/operations/utils/index.md
+++ b/docs/en/operations/utilities/index.md
@@ -1,3 +1,9 @@
+---
+toc_folder_title: Utilities
+toc_priority: 56
+toc_title: Overview
+---
+
# ClickHouse Utility {#clickhouse-utility}
- [clickhouse-local](clickhouse-local.md) — Allows running SQL queries on data without stopping the ClickHouse server, similar to how `awk` does this.
diff --git a/docs/en/query_language/functions/machine_learning_functions.md b/docs/en/query_language/functions/machine_learning_functions.md
deleted file mode 100644
index 989b20d59b4..00000000000
--- a/docs/en/query_language/functions/machine_learning_functions.md
+++ /dev/null
@@ -1,13 +0,0 @@
-# Machine learning functions {#machine-learning-functions}
-
-## evalMLMethod (prediction) {#machine_learning_methods-evalmlmethod}
-
-Prediction using fitted regression models uses `evalMLMethod` function. See link in `linearRegression`.
-
-### Stochastic Linear Regression {#stochastic-linear-regression}
-
-The [stochasticLinearRegression](../agg_functions/reference.md#agg_functions-stochasticlinearregression) aggregate function implements stochastic gradient descent method using linear model and MSE loss function. Uses `evalMLMethod` to predict on new data.
-
-### Stochastic Logistic Regression {#stochastic-logistic-regression}
-
-The [stochasticLogisticRegression](../agg_functions/reference.md#agg_functions-stochasticlogisticregression) aggregate function implements stochastic gradient descent method for binary classification problem. Uses `evalMLMethod` to predict on new data.
diff --git a/docs/en/query_language/index.md b/docs/en/query_language/index.md
deleted file mode 100644
index 024281e2ac0..00000000000
--- a/docs/en/query_language/index.md
+++ /dev/null
@@ -1,9 +0,0 @@
-# SQL Reference {#sql-reference}
-
-- [SELECT](select.md)
-- [INSERT INTO](insert_into.md)
-- [CREATE](create.md)
-- [ALTER](alter.md#query_language_queries_alter)
-- [Other types of queries](misc.md)
-
-[Original article](https://clickhouse.tech/docs/en/query_language/)
diff --git a/docs/en/query_language/select.md b/docs/en/query_language/select.md
deleted file mode 100644
index d46c0be1aaf..00000000000
--- a/docs/en/query_language/select.md
+++ /dev/null
@@ -1,1375 +0,0 @@
-# SELECT Queries Syntax {#select-queries-syntax}
-
-`SELECT` performs data retrieval.
-
-``` sql
-[WITH expr_list|(subquery)]
-SELECT [DISTINCT] expr_list
-[FROM [db.]table | (subquery) | table_function] [FINAL]
-[SAMPLE sample_coeff]
-[ARRAY JOIN ...]
-[GLOBAL] [ANY|ALL] [INNER|LEFT|RIGHT|FULL|CROSS] [OUTER] JOIN (subquery)|table USING columns_list
-[PREWHERE expr]
-[WHERE expr]
-[GROUP BY expr_list] [WITH TOTALS]
-[HAVING expr]
-[ORDER BY expr_list]
-[LIMIT [offset_value, ]n BY columns]
-[LIMIT [n, ]m]
-[UNION ALL ...]
-[INTO OUTFILE filename]
-[FORMAT format]
-```
-
-All the clauses are optional, except for the required list of expressions immediately after SELECT.
-The clauses below are described in almost the same order as in the query execution conveyor.
-
-If the query omits the `DISTINCT`, `GROUP BY` and `ORDER BY` clauses and the `IN` and `JOIN` subqueries, the query will be completely stream processed, using O(1) amount of RAM.
-Otherwise, the query might consume a lot of RAM if the appropriate restrictions are not specified: `max_memory_usage`, `max_rows_to_group_by`, `max_rows_to_sort`, `max_rows_in_distinct`, `max_bytes_in_distinct`, `max_rows_in_set`, `max_bytes_in_set`, `max_rows_in_join`, `max_bytes_in_join`, `max_bytes_before_external_sort`, `max_bytes_before_external_group_by`. For more information, see the section “Settings”. It is possible to use external sorting (saving temporary tables to a disk) and external aggregation. `The system does not have "merge join"`.
-
-### WITH Clause {#with-clause}
-
-This section provides support for Common Table Expressions ([CTE](https://en.wikipedia.org/wiki/Hierarchical_and_recursive_queries_in_SQL)), with some limitations:
-1. Recursive queries are not supported
-2. When subquery is used inside WITH section, it’s result should be scalar with exactly one row
-3. Expression’s results are not available in subqueries
-Results of WITH clause expressions can be used inside SELECT clause.
-
-Example 1: Using constant expression as “variable”
-
-``` sql
-WITH '2019-08-01 15:23:00' as ts_upper_bound
-SELECT *
-FROM hits
-WHERE
- EventDate = toDate(ts_upper_bound) AND
- EventTime <= ts_upper_bound
-```
-
-Example 2: Evicting sum(bytes) expression result from SELECT clause column list
-
-``` sql
-WITH sum(bytes) as s
-SELECT
- formatReadableSize(s),
- table
-FROM system.parts
-GROUP BY table
-ORDER BY s
-```
-
-Example 3: Using results of scalar subquery
-
-``` sql
-/* this example would return TOP 10 of most huge tables */
-WITH
- (
- SELECT sum(bytes)
- FROM system.parts
- WHERE active
- ) AS total_disk_usage
-SELECT
- (sum(bytes) / total_disk_usage) * 100 AS table_disk_usage,
- table
-FROM system.parts
-GROUP BY table
-ORDER BY table_disk_usage DESC
-LIMIT 10
-```
-
-Example 4: Re-using expression in subquery
-As a workaround for current limitation for expression usage in subqueries, you may duplicate it.
-
-``` sql
-WITH ['hello'] AS hello
-SELECT
- hello,
- *
-FROM
-(
- WITH ['hello'] AS hello
- SELECT hello
-)
-```
-
-``` text
-┌─hello─────┬─hello─────┐
-│ ['hello'] │ ['hello'] │
-└───────────┴───────────┘
-```
-
-### FROM Clause {#select-from}
-
-If the FROM clause is omitted, data will be read from the `system.one` table.
-The `system.one` table contains exactly one row (this table fulfills the same purpose as the DUAL table found in other DBMSs).
-
-The `FROM` clause specifies the source to read data from:
-
-- Table
-- Subquery
-- [Table function](table_functions/index.md)
-
-`ARRAY JOIN` and the regular `JOIN` may also be included (see below).
-
-Instead of a table, the `SELECT` subquery may be specified in parenthesis.
-In contrast to standard SQL, a synonym does not need to be specified after a subquery.
-
-To execute a query, all the columns listed in the query are extracted from the appropriate table. Any columns not needed for the external query are thrown out of the subqueries.
-If a query does not list any columns (for example, `SELECT count() FROM t`), some column is extracted from the table anyway (the smallest one is preferred), in order to calculate the number of rows.
-
-#### FINAL Modifier {#select-from-final}
-
-Applicable when selecting data from tables from the [MergeTree](../operations/table_engines/mergetree.md)-engine family other than `GraphiteMergeTree`. When `FINAL` is specified, ClickHouse fully merges the data before returning the result and thus performs all data transformations that happen during merges for the given table engine.
-
-Also supported for:
-- [Replicated](../operations/table_engines/replication.md) versions of `MergeTree` engines.
-- [View](../operations/table_engines/view.md), [Buffer](../operations/table_engines/buffer.md), [Distributed](../operations/table_engines/distributed.md), and [MaterializedView](../operations/table_engines/materializedview.md) engines that operate over other engines, provided they were created over `MergeTree`-engine tables.
-
-Queries that use `FINAL` are executed not as fast as similar queries that don’t, because:
-
-- Query is executed in a single thread and data is merged during query execution.
-- Queries with `FINAL` read primary key columns in addition to the columns specified in the query.
-
-In most cases, avoid using `FINAL`.
-
-### SAMPLE Clause {#select-sample-clause}
-
-The `SAMPLE` clause allows for approximated query processing.
-
-When data sampling is enabled, the query is not performed on all the data, but only on a certain fraction of data (sample). For example, if you need to calculate statistics for all the visits, it is enough to execute the query on the 1/10 fraction of all the visits and then multiply the result by 10.
-
-Approximated query processing can be useful in the following cases:
-
-- When you have strict timing requirements (like \<100ms) but you can’t justify the cost of additional hardware resources to meet them.
-- When your raw data is not accurate, so approximation doesn’t noticeably degrade the quality.
-- Business requirements target approximate results (for cost-effectiveness, or in order to market exact results to premium users).
-
-!!! note "Note"
- You can only use sampling with the tables in the [MergeTree](../operations/table_engines/mergetree.md) family, and only if the sampling expression was specified during table creation (see [MergeTree engine](../operations/table_engines/mergetree.md#table_engine-mergetree-creating-a-table)).
-
-The features of data sampling are listed below:
-
-- Data sampling is a deterministic mechanism. The result of the same `SELECT .. SAMPLE` query is always the same.
-- Sampling works consistently for different tables. For tables with a single sampling key, a sample with the same coefficient always selects the same subset of possible data. For example, a sample of user IDs takes rows with the same subset of all the possible user IDs from different tables. This means that you can use the sample in subqueries in the [IN](#select-in-operators) clause. Also, you can join samples using the [JOIN](#select-join) clause.
-- Sampling allows reading less data from a disk. Note that you must specify the sampling key correctly. For more information, see [Creating a MergeTree Table](../operations/table_engines/mergetree.md#table_engine-mergetree-creating-a-table).
-
-For the `SAMPLE` clause the following syntax is supported:
-
-| SAMPLE Clause Syntax | Description |
-|----------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
-| `SAMPLE k` | Here `k` is the number from 0 to 1.The query is executed on `k` fraction of data. For example, `SAMPLE 0.1` runs the query on 10% of data. [Read more](#select-sample-k) |
-| `SAMPLE n` | Here `n` is a sufficiently large integer.The query is executed on a sample of at least `n` rows (but not significantly more than this). For example, `SAMPLE 10000000` runs the query on a minimum of 10,000,000 rows. [Read more](#select-sample-n) |
-| `SAMPLE k OFFSET m` | Here `k` and `m` are the numbers from 0 to 1.The query is executed on a sample of `k` fraction of the data. The data used for the sample is offset by `m` fraction. [Read more](#select-sample-offset) |
-
-#### SAMPLE k {#select-sample-k}
-
-Here `k` is the number from 0 to 1 (both fractional and decimal notations are supported). For example, `SAMPLE 1/2` or `SAMPLE 0.5`.
-
-In a `SAMPLE k` clause, the sample is taken from the `k` fraction of data. The example is shown below:
-
-``` sql
-SELECT
- Title,
- count() * 10 AS PageViews
-FROM hits_distributed
-SAMPLE 0.1
-WHERE
- CounterID = 34
-GROUP BY Title
-ORDER BY PageViews DESC LIMIT 1000
-```
-
-In this example, the query is executed on a sample from 0.1 (10%) of data. Values of aggregate functions are not corrected automatically, so to get an approximate result, the value `count()` is manually multiplied by 10.
-
-#### SAMPLE n {#select-sample-n}
-
-Here `n` is a sufficiently large integer. For example, `SAMPLE 10000000`.
-
-In this case, the query is executed on a sample of at least `n` rows (but not significantly more than this). For example, `SAMPLE 10000000` runs the query on a minimum of 10,000,000 rows.
-
-Since the minimum unit for data reading is one granule (its size is set by the `index_granularity` setting), it makes sense to set a sample that is much larger than the size of the granule.
-
-When using the `SAMPLE n` clause, you don’t know which relative percent of data was processed. So you don’t know the coefficient the aggregate functions should be multiplied by. Use the `_sample_factor` virtual column to get the approximate result.
-
-The `_sample_factor` column contains relative coefficients that are calculated dynamically. This column is created automatically when you [create](../operations/table_engines/mergetree.md#table_engine-mergetree-creating-a-table) a table with the specified sampling key. The usage examples of the `_sample_factor` column are shown below.
-
-Let’s consider the table `visits`, which contains the statistics about site visits. The first example shows how to calculate the number of page views:
-
-``` sql
-SELECT sum(PageViews * _sample_factor)
-FROM visits
-SAMPLE 10000000
-```
-
-The next example shows how to calculate the total number of visits:
-
-``` sql
-SELECT sum(_sample_factor)
-FROM visits
-SAMPLE 10000000
-```
-
-The example below shows how to calculate the average session duration. Note that you don’t need to use the relative coefficient to calculate the average values.
-
-``` sql
-SELECT avg(Duration)
-FROM visits
-SAMPLE 10000000
-```
-
-#### SAMPLE k OFFSET m {#select-sample-offset}
-
-Here `k` and `m` are numbers from 0 to 1. Examples are shown below.
-
-**Example 1**
-
-``` sql
-SAMPLE 1/10
-```
-
-In this example, the sample is 1/10th of all data:
-
-`[++------------------]`
-
-**Example 2**
-
-``` sql
-SAMPLE 1/10 OFFSET 1/2
-```
-
-Here, a sample of 10% is taken from the second half of the data.
-
-`[----------++--------]`
-
-### ARRAY JOIN Clause {#select-array-join-clause}
-
-Allows executing `JOIN` with an array or nested data structure. The intent is similar to the [arrayJoin](functions/array_join.md#functions_arrayjoin) function, but its functionality is broader.
-
-``` sql
-SELECT
-FROM
-[LEFT] ARRAY JOIN
-[WHERE|PREWHERE ]
-...
-```
-
-You can specify only a single `ARRAY JOIN` clause in a query.
-
-The query execution order is optimized when running `ARRAY JOIN`. Although `ARRAY JOIN` must always be specified before the `WHERE/PREWHERE` clause, it can be performed either before `WHERE/PREWHERE` (if the result is needed in this clause), or after completing it (to reduce the volume of calculations). The processing order is controlled by the query optimizer.
-
-Supported types of `ARRAY JOIN` are listed below:
-
-- `ARRAY JOIN` - In this case, empty arrays are not included in the result of `JOIN`.
-- `LEFT ARRAY JOIN` - The result of `JOIN` contains rows with empty arrays. The value for an empty array is set to the default value for the array element type (usually 0, empty string or NULL).
-
-The examples below demonstrate the usage of the `ARRAY JOIN` and `LEFT ARRAY JOIN` clauses. Let’s create a table with an [Array](../data_types/array.md) type column and insert values into it:
-
-``` sql
-CREATE TABLE arrays_test
-(
- s String,
- arr Array(UInt8)
-) ENGINE = Memory;
-
-INSERT INTO arrays_test
-VALUES ('Hello', [1,2]), ('World', [3,4,5]), ('Goodbye', []);
-```
-
-``` text
-┌─s───────────┬─arr─────┐
-│ Hello │ [1,2] │
-│ World │ [3,4,5] │
-│ Goodbye │ [] │
-└─────────────┴─────────┘
-```
-
-The example below uses the `ARRAY JOIN` clause:
-
-``` sql
-SELECT s, arr
-FROM arrays_test
-ARRAY JOIN arr;
-```
-
-``` text
-┌─s─────┬─arr─┐
-│ Hello │ 1 │
-│ Hello │ 2 │
-│ World │ 3 │
-│ World │ 4 │
-│ World │ 5 │
-└───────┴─────┘
-```
-
-The next example uses the `LEFT ARRAY JOIN` clause:
-
-``` sql
-SELECT s, arr
-FROM arrays_test
-LEFT ARRAY JOIN arr;
-```
-
-``` text
-┌─s───────────┬─arr─┐
-│ Hello │ 1 │
-│ Hello │ 2 │
-│ World │ 3 │
-│ World │ 4 │
-│ World │ 5 │
-│ Goodbye │ 0 │
-└─────────────┴─────┘
-```
-
-#### Using Aliases {#using-aliases}
-
-An alias can be specified for an array in the `ARRAY JOIN` clause. In this case, an array item can be accessed by this alias, but the array itself is accessed by the original name. Example:
-
-``` sql
-SELECT s, arr, a
-FROM arrays_test
-ARRAY JOIN arr AS a;
-```
-
-``` text
-┌─s─────┬─arr─────┬─a─┐
-│ Hello │ [1,2] │ 1 │
-│ Hello │ [1,2] │ 2 │
-│ World │ [3,4,5] │ 3 │
-│ World │ [3,4,5] │ 4 │
-│ World │ [3,4,5] │ 5 │
-└───────┴─────────┴───┘
-```
-
-Using aliases, you can perform `ARRAY JOIN` with an external array. For example:
-
-``` sql
-SELECT s, arr_external
-FROM arrays_test
-ARRAY JOIN [1, 2, 3] AS arr_external;
-```
-
-``` text
-┌─s───────────┬─arr_external─┐
-│ Hello │ 1 │
-│ Hello │ 2 │
-│ Hello │ 3 │
-│ World │ 1 │
-│ World │ 2 │
-│ World │ 3 │
-│ Goodbye │ 1 │
-│ Goodbye │ 2 │
-│ Goodbye │ 3 │
-└─────────────┴──────────────┘
-```
-
-Multiple arrays can be comma-separated in the `ARRAY JOIN` clause. In this case, `JOIN` is performed with them simultaneously (the direct sum, not the cartesian product). Note that all the arrays must have the same size. Example:
-
-``` sql
-SELECT s, arr, a, num, mapped
-FROM arrays_test
-ARRAY JOIN arr AS a, arrayEnumerate(arr) AS num, arrayMap(x -> x + 1, arr) AS mapped;
-```
-
-``` text
-┌─s─────┬─arr─────┬─a─┬─num─┬─mapped─┐
-│ Hello │ [1,2] │ 1 │ 1 │ 2 │
-│ Hello │ [1,2] │ 2 │ 2 │ 3 │
-│ World │ [3,4,5] │ 3 │ 1 │ 4 │
-│ World │ [3,4,5] │ 4 │ 2 │ 5 │
-│ World │ [3,4,5] │ 5 │ 3 │ 6 │
-└───────┴─────────┴───┴─────┴────────┘
-```
-
-The example below uses the [arrayEnumerate](functions/array_functions.md#array_functions-arrayenumerate) function:
-
-``` sql
-SELECT s, arr, a, num, arrayEnumerate(arr)
-FROM arrays_test
-ARRAY JOIN arr AS a, arrayEnumerate(arr) AS num;
-```
-
-``` text
-┌─s─────┬─arr─────┬─a─┬─num─┬─arrayEnumerate(arr)─┐
-│ Hello │ [1,2] │ 1 │ 1 │ [1,2] │
-│ Hello │ [1,2] │ 2 │ 2 │ [1,2] │
-│ World │ [3,4,5] │ 3 │ 1 │ [1,2,3] │
-│ World │ [3,4,5] │ 4 │ 2 │ [1,2,3] │
-│ World │ [3,4,5] │ 5 │ 3 │ [1,2,3] │
-└───────┴─────────┴───┴─────┴─────────────────────┘
-```
-
-#### ARRAY JOIN With Nested Data Structure {#array-join-with-nested-data-structure}
-
-`ARRAY`JOIN\`\` also works with [nested data structures](../data_types/nested_data_structures/nested.md). Example:
-
-``` sql
-CREATE TABLE nested_test
-(
- s String,
- nest Nested(
- x UInt8,
- y UInt32)
-) ENGINE = Memory;
-
-INSERT INTO nested_test
-VALUES ('Hello', [1,2], [10,20]), ('World', [3,4,5], [30,40,50]), ('Goodbye', [], []);
-```
-
-``` text
-┌─s───────┬─nest.x──┬─nest.y─────┐
-│ Hello │ [1,2] │ [10,20] │
-│ World │ [3,4,5] │ [30,40,50] │
-│ Goodbye │ [] │ [] │
-└─────────┴─────────┴────────────┘
-```
-
-``` sql
-SELECT s, `nest.x`, `nest.y`
-FROM nested_test
-ARRAY JOIN nest;
-```
-
-``` text
-┌─s─────┬─nest.x─┬─nest.y─┐
-│ Hello │ 1 │ 10 │
-│ Hello │ 2 │ 20 │
-│ World │ 3 │ 30 │
-│ World │ 4 │ 40 │
-│ World │ 5 │ 50 │
-└───────┴────────┴────────┘
-```
-
-When specifying names of nested data structures in `ARRAY JOIN`, the meaning is the same as `ARRAY JOIN` with all the array elements that it consists of. Examples are listed below:
-
-``` sql
-SELECT s, `nest.x`, `nest.y`
-FROM nested_test
-ARRAY JOIN `nest.x`, `nest.y`;
-```
-
-``` text
-┌─s─────┬─nest.x─┬─nest.y─┐
-│ Hello │ 1 │ 10 │
-│ Hello │ 2 │ 20 │
-│ World │ 3 │ 30 │
-│ World │ 4 │ 40 │
-│ World │ 5 │ 50 │
-└───────┴────────┴────────┘
-```
-
-This variation also makes sense:
-
-``` sql
-SELECT s, `nest.x`, `nest.y`
-FROM nested_test
-ARRAY JOIN `nest.x`;
-```
-
-``` text
-┌─s─────┬─nest.x─┬─nest.y─────┐
-│ Hello │ 1 │ [10,20] │
-│ Hello │ 2 │ [10,20] │
-│ World │ 3 │ [30,40,50] │
-│ World │ 4 │ [30,40,50] │
-│ World │ 5 │ [30,40,50] │
-└───────┴────────┴────────────┘
-```
-
-An alias may be used for a nested data structure, in order to select either the `JOIN` result or the source array. Example:
-
-``` sql
-SELECT s, `n.x`, `n.y`, `nest.x`, `nest.y`
-FROM nested_test
-ARRAY JOIN nest AS n;
-```
-
-``` text
-┌─s─────┬─n.x─┬─n.y─┬─nest.x──┬─nest.y─────┐
-│ Hello │ 1 │ 10 │ [1,2] │ [10,20] │
-│ Hello │ 2 │ 20 │ [1,2] │ [10,20] │
-│ World │ 3 │ 30 │ [3,4,5] │ [30,40,50] │
-│ World │ 4 │ 40 │ [3,4,5] │ [30,40,50] │
-│ World │ 5 │ 50 │ [3,4,5] │ [30,40,50] │
-└───────┴─────┴─────┴─────────┴────────────┘
-```
-
-Example of using the [arrayEnumerate](functions/array_functions.md#array_functions-arrayenumerate) function:
-
-``` sql
-SELECT s, `n.x`, `n.y`, `nest.x`, `nest.y`, num
-FROM nested_test
-ARRAY JOIN nest AS n, arrayEnumerate(`nest.x`) AS num;
-```
-
-``` text
-┌─s─────┬─n.x─┬─n.y─┬─nest.x──┬─nest.y─────┬─num─┐
-│ Hello │ 1 │ 10 │ [1,2] │ [10,20] │ 1 │
-│ Hello │ 2 │ 20 │ [1,2] │ [10,20] │ 2 │
-│ World │ 3 │ 30 │ [3,4,5] │ [30,40,50] │ 1 │
-│ World │ 4 │ 40 │ [3,4,5] │ [30,40,50] │ 2 │
-│ World │ 5 │ 50 │ [3,4,5] │ [30,40,50] │ 3 │
-└───────┴─────┴─────┴─────────┴────────────┴─────┘
-```
-
-### JOIN Clause {#select-join}
-
-Joins the data in the normal [SQL JOIN](https://en.wikipedia.org/wiki/Join_(SQL)) sense.
-
-!!! info "Note"
- Not related to [ARRAY JOIN](#select-array-join-clause).
-
-``` sql
-SELECT
-FROM
-[GLOBAL] [ANY|ALL] [INNER|LEFT|RIGHT|FULL|CROSS] [OUTER] JOIN
-(ON )|(USING ) ...
-```
-
-The table names can be specified instead of `` and ``. This is equivalent to the `SELECT * FROM table` subquery, except in a special case when the table has the [Join](../operations/table_engines/join.md) engine – an array prepared for joining.
-
-#### Supported Types of `JOIN` {#select-join-types}
-
-- `INNER JOIN` (or `JOIN`)
-- `LEFT JOIN` (or `LEFT OUTER JOIN`)
-- `RIGHT JOIN` (or `RIGHT OUTER JOIN`)
-- `FULL JOIN` (or `FULL OUTER JOIN`)
-- `CROSS JOIN` (or `,` )
-
-See the standard [SQL JOIN](https://en.wikipedia.org/wiki/Join_(SQL)) description.
-
-#### Multiple JOIN {#multiple-join}
-
-Performing queries, ClickHouse rewrites multi-table joins into the sequence of two-table joins. For example, if there are four tables for join ClickHouse joins the first and the second, then joins the result with the third table, and at the last step, it joins the fourth one.
-
-If a query contains the `WHERE` clause, ClickHouse tries to pushdown filters from this clause through the intermediate join. If it cannot apply the filter to each intermediate join, ClickHouse applies the filters after all joins are completed.
-
-We recommend the `JOIN ON` or `JOIN USING` syntax for creating queries. For example:
-
-``` sql
-SELECT * FROM t1 JOIN t2 ON t1.a = t2.a JOIN t3 ON t1.a = t3.a
-```
-
-You can use comma-separated lists of tables in the `FROM` clause. For example:
-
-``` sql
-SELECT * FROM t1, t2, t3 WHERE t1.a = t2.a AND t1.a = t3.a
-```
-
-Don’t mix these syntaxes.
-
-ClickHouse doesn’t directly support syntax with commas, so we don’t recommend using them. The algorithm tries to rewrite the query in terms of `CROSS JOIN` and `INNER JOIN` clauses and then proceeds to query processing. When rewriting the query, ClickHouse tries to optimize performance and memory consumption. By default, ClickHouse treats commas as an `INNER JOIN` clause and converts `INNER JOIN` to `CROSS JOIN` when the algorithm cannot guarantee that `INNER JOIN` returns the required data.
-
-#### Strictness {#select-join-strictness}
-
-- `ALL` — If the right table has several matching rows, ClickHouse creates a [Cartesian product](https://en.wikipedia.org/wiki/Cartesian_product) from matching rows. This is the standard `JOIN` behavior in SQL.
-- `ANY` — If the right table has several matching rows, only the first one found is joined. If the right table has only one matching row, the results of queries with `ANY` and `ALL` keywords are the same.
-- `ASOF` — For joining sequences with a non-exact match. `ASOF JOIN` usage is described below.
-
-**ASOF JOIN Usage**
-
-`ASOF JOIN` is useful when you need to join records that have no exact match.
-
-Tables for `ASOF JOIN` must have an ordered sequence column. This column cannot be alone in a table, and should be one of the data types: `UInt32`, `UInt64`, `Float32`, `Float64`, `Date`, and `DateTime`.
-
-Syntax `ASOF JOIN ... ON`:
-
-``` sql
-SELECT expressions_list
-FROM table_1
-ASOF LEFT JOIN table_2
-ON equi_cond AND closest_match_cond
-```
-
-You can use any number of equality conditions and exactly one closest match condition. For example, `SELECT count() FROM table_1 ASOF LEFT JOIN table_2 ON table_1.a == table_2.b AND table_2.t <= table_1.t`.
-
-Conditions supported for the closest match: `>`, `>=`, `<`, `<=`.
-
-Syntax `ASOF JOIN ... USING`:
-
-``` sql
-SELECT expressions_list
-FROM table_1
-ASOF JOIN table_2
-USING (equi_column1, ... equi_columnN, asof_column)
-```
-
-`ASOF JOIN` uses `equi_columnX` for joining on equality and `asof_column` for joining on the closest match with the `table_1.asof_column >= table_2.asof_column` condition. The `asof_column` column always the last one in the `USING` clause.
-
-For example, consider the following tables:
-
-``` text
- table_1 table_2
-
- event | ev_time | user_id event | ev_time | user_id
-----------|---------|---------- ----------|---------|----------
- ... ...
-event_1_1 | 12:00 | 42 event_2_1 | 11:59 | 42
- ... event_2_2 | 12:30 | 42
-event_1_2 | 13:00 | 42 event_2_3 | 13:00 | 42
- ... ...
-```
-
-`ASOF JOIN` can take the timestamp of a user event from `table_1` and find an event in `table_2` where the timestamp is closest to the timestamp of the event from `table_1` corresponding to the closest match condition. Equal timestamp values are the closest if available. Here, the `user_id` column can be used for joining on equality and the `ev_time` column can be used for joining on the closest match. In our example, `event_1_1` can be joined with `event_2_1` and `event_1_2` can be joined with `event_2_3`, but `event_2_2` can’t be joined.
-
-!!! note "Note"
- `ASOF` join is **not** supported in the [Join](../operations/table_engines/join.md) table engine.
-
-To set the default strictness value, use the session configuration parameter [join\_default\_strictness](../operations/settings/settings.md#settings-join_default_strictness).
-
-#### GLOBAL JOIN {#global-join}
-
-When using a normal `JOIN`, the query is sent to remote servers. Subqueries are run on each of them in order to make the right table, and the join is performed with this table. In other words, the right table is formed on each server separately.
-
-When using `GLOBAL ... JOIN`, first the requestor server runs a subquery to calculate the right table. This temporary table is passed to each remote server, and queries are run on them using the temporary data that was transmitted.
-
-Be careful when using `GLOBAL`. For more information, see the section [Distributed subqueries](#select-distributed-subqueries).
-
-#### Usage Recommendations {#usage-recommendations}
-
-When running a `JOIN`, there is no optimization of the order of execution in relation to other stages of the query. The join (a search in the right table) is run before filtering in `WHERE` and before aggregation. In order to explicitly set the processing order, we recommend running a `JOIN` subquery with a subquery.
-
-Example:
-
-``` sql
-SELECT
- CounterID,
- hits,
- visits
-FROM
-(
- SELECT
- CounterID,
- count() AS hits
- FROM test.hits
- GROUP BY CounterID
-) ANY LEFT JOIN
-(
- SELECT
- CounterID,
- sum(Sign) AS visits
- FROM test.visits
- GROUP BY CounterID
-) USING CounterID
-ORDER BY hits DESC
-LIMIT 10
-```
-
-``` text
-┌─CounterID─┬───hits─┬─visits─┐
-│ 1143050 │ 523264 │ 13665 │
-│ 731962 │ 475698 │ 102716 │
-│ 722545 │ 337212 │ 108187 │
-│ 722889 │ 252197 │ 10547 │
-│ 2237260 │ 196036 │ 9522 │
-│ 23057320 │ 147211 │ 7689 │
-│ 722818 │ 90109 │ 17847 │
-│ 48221 │ 85379 │ 4652 │
-│ 19762435 │ 77807 │ 7026 │
-│ 722884 │ 77492 │ 11056 │
-└───────────┴────────┴────────┘
-```
-
-Subqueries don’t allow you to set names or use them for referencing a column from a specific subquery.
-The columns specified in `USING` must have the same names in both subqueries, and the other columns must be named differently. You can use aliases to change the names of columns in subqueries (the example uses the aliases `hits` and `visits`).
-
-The `USING` clause specifies one or more columns to join, which establishes the equality of these columns. The list of columns is set without brackets. More complex join conditions are not supported.
-
-The right table (the subquery result) resides in RAM. If there isn’t enough memory, you can’t run a `JOIN`.
-
-Each time a query is run with the same `JOIN`, the subquery is run again because the result is not cached. To avoid this, use the special [Join](../operations/table_engines/join.md) table engine, which is a prepared array for joining that is always in RAM.
-
-In some cases, it is more efficient to use `IN` instead of `JOIN`.
-Among the various types of `JOIN`, the most efficient is `ANY LEFT JOIN`, then `ANY INNER JOIN`. The least efficient are `ALL LEFT JOIN` and `ALL INNER JOIN`.
-
-If you need a `JOIN` for joining with dimension tables (these are relatively small tables that contain dimension properties, such as names for advertising campaigns), a `JOIN` might not be very convenient due to the fact that the right table is re-accessed for every query. For such cases, there is an “external dictionaries” feature that you should use instead of `JOIN`. For more information, see the section [External dictionaries](dicts/external_dicts.md).
-
-**Memory Limitations**
-
-ClickHouse uses the [hash join](https://en.wikipedia.org/wiki/Hash_join) algorithm. ClickHouse takes the `` and creates a hash table for it in RAM. If you need to restrict join operation memory consumption use the following settings:
-
-- [max\_rows\_in\_join](../operations/settings/query_complexity.md#settings-max_rows_in_join) — Limits number of rows in the hash table.
-- [max\_bytes\_in\_join](../operations/settings/query_complexity.md#settings-max_bytes_in_join) — Limits size of the hash table.
-
-When any of these limits is reached, ClickHouse acts as the [join\_overflow\_mode](../operations/settings/query_complexity.md#settings-join_overflow_mode) setting instructs.
-
-#### Processing of Empty or NULL Cells {#processing-of-empty-or-null-cells}
-
-While joining tables, the empty cells may appear. The setting [join\_use\_nulls](../operations/settings/settings.md#join_use_nulls) define how ClickHouse fills these cells.
-
-If the `JOIN` keys are [Nullable](../data_types/nullable.md) fields, the rows where at least one of the keys has the value [NULL](syntax.md#null-literal) are not joined.
-
-#### Syntax Limitations {#syntax-limitations}
-
-For multiple `JOIN` clauses in a single `SELECT` query:
-
-- Taking all the columns via `*` is available only if tables are joined, not subqueries.
-- The `PREWHERE` clause is not available.
-
-For `ON`, `WHERE`, and `GROUP BY` clauses:
-
-- Arbitrary expressions cannot be used in `ON`, `WHERE`, and `GROUP BY` clauses, but you can define an expression in a `SELECT` clause and then use it in these clauses via an alias.
-
-### WHERE Clause {#select-where}
-
-If there is a WHERE clause, it must contain an expression with the UInt8 type. This is usually an expression with comparison and logical operators.
-This expression will be used for filtering data before all other transformations.
-
-If indexes are supported by the database table engine, the expression is evaluated on the ability to use indexes.
-
-### PREWHERE Clause {#prewhere-clause}
-
-This clause has the same meaning as the WHERE clause. The difference is in which data is read from the table.
-When using PREWHERE, first only the columns necessary for executing PREWHERE are read. Then the other columns are read that are needed for running the query, but only those blocks where the PREWHERE expression is true.
-
-It makes sense to use PREWHERE if there are filtration conditions that are used by a minority of the columns in the query, but that provide strong data filtration. This reduces the volume of data to read.
-
-For example, it is useful to write PREWHERE for queries that extract a large number of columns, but that only have filtration for a few columns.
-
-PREWHERE is only supported by tables from the `*MergeTree` family.
-
-A query may simultaneously specify PREWHERE and WHERE. In this case, PREWHERE precedes WHERE.
-
-If the ‘optimize\_move\_to\_prewhere’ setting is set to 1 and PREWHERE is omitted, the system uses heuristics to automatically move parts of expressions from WHERE to PREWHERE.
-
-### GROUP BY Clause {#select-group-by-clause}
-
-This is one of the most important parts of a column-oriented DBMS.
-
-If there is a GROUP BY clause, it must contain a list of expressions. Each expression will be referred to here as a “key”.
-All the expressions in the SELECT, HAVING, and ORDER BY clauses must be calculated from keys or from aggregate functions. In other words, each column selected from the table must be used either in keys or inside aggregate functions.
-
-If a query contains only table columns inside aggregate functions, the GROUP BY clause can be omitted, and aggregation by an empty set of keys is assumed.
-
-Example:
-
-``` sql
-SELECT
- count(),
- median(FetchTiming > 60 ? 60 : FetchTiming),
- count() - sum(Refresh)
-FROM hits
-```
-
-However, in contrast to standard SQL, if the table doesn’t have any rows (either there aren’t any at all, or there aren’t any after using WHERE to filter), an empty result is returned, and not the result from one of the rows containing the initial values of aggregate functions.
-
-As opposed to MySQL (and conforming to standard SQL), you can’t get some value of some column that is not in a key or aggregate function (except constant expressions). To work around this, you can use the ‘any’ aggregate function (get the first encountered value) or ‘min/max’.
-
-Example:
-
-``` sql
-SELECT
- domainWithoutWWW(URL) AS domain,
- count(),
- any(Title) AS title -- getting the first occurred page header for each domain.
-FROM hits
-GROUP BY domain
-```
-
-For every different key value encountered, GROUP BY calculates a set of aggregate function values.
-
-GROUP BY is not supported for array columns.
-
-A constant can’t be specified as arguments for aggregate functions. Example: sum(1). Instead of this, you can get rid of the constant. Example: `count()`.
-
-#### NULL processing {#null-processing}
-
-For grouping, ClickHouse interprets [NULL](syntax.md) as a value, and `NULL=NULL`.
-
-Here’s an example to show what this means.
-
-Assume you have this table:
-
-``` text
-┌─x─┬────y─┐
-│ 1 │ 2 │
-│ 2 │ ᴺᵁᴸᴸ │
-│ 3 │ 2 │
-│ 3 │ 3 │
-│ 3 │ ᴺᵁᴸᴸ │
-└───┴──────┘
-```
-
-The query `SELECT sum(x), y FROM t_null_big GROUP BY y` results in:
-
-``` text
-┌─sum(x)─┬────y─┐
-│ 4 │ 2 │
-│ 3 │ 3 │
-│ 5 │ ᴺᵁᴸᴸ │
-└────────┴──────┘
-```
-
-You can see that `GROUP BY` for `y = NULL` summed up `x`, as if `NULL` is this value.
-
-If you pass several keys to `GROUP BY`, the result will give you all the combinations of the selection, as if `NULL` were a specific value.
-
-#### WITH TOTALS Modifier {#with-totals-modifier}
-
-If the WITH TOTALS modifier is specified, another row will be calculated. This row will have key columns containing default values (zeros or empty lines), and columns of aggregate functions with the values calculated across all the rows (the “total” values).
-
-This extra row is output in JSON\*, TabSeparated\*, and Pretty\* formats, separately from the other rows. In the other formats, this row is not output.
-
-In JSON\* formats, this row is output as a separate ‘totals’ field. In TabSeparated\* formats, the row comes after the main result, preceded by an empty row (after the other data). In Pretty\* formats, the row is output as a separate table after the main result.
-
-`WITH TOTALS` can be run in different ways when HAVING is present. The behavior depends on the ‘totals\_mode’ setting.
-By default, `totals_mode = 'before_having'`. In this case, ‘totals’ is calculated across all rows, including the ones that don’t pass through HAVING and ‘max\_rows\_to\_group\_by’.
-
-The other alternatives include only the rows that pass through HAVING in ‘totals’, and behave differently with the setting `max_rows_to_group_by` and `group_by_overflow_mode = 'any'`.
-
-`after_having_exclusive` – Don’t include rows that didn’t pass through `max_rows_to_group_by`. In other words, ‘totals’ will have less than or the same number of rows as it would if `max_rows_to_group_by` were omitted.
-
-`after_having_inclusive` – Include all the rows that didn’t pass through ‘max\_rows\_to\_group\_by’ in ‘totals’. In other words, ‘totals’ will have more than or the same number of rows as it would if `max_rows_to_group_by` were omitted.
-
-`after_having_auto` – Count the number of rows that passed through HAVING. If it is more than a certain amount (by default, 50%), include all the rows that didn’t pass through ‘max\_rows\_to\_group\_by’ in ‘totals’. Otherwise, do not include them.
-
-`totals_auto_threshold` – By default, 0.5. The coefficient for `after_having_auto`.
-
-If `max_rows_to_group_by` and `group_by_overflow_mode = 'any'` are not used, all variations of `after_having` are the same, and you can use any of them (for example, `after_having_auto`).
-
-You can use WITH TOTALS in subqueries, including subqueries in the JOIN clause (in this case, the respective total values are combined).
-
-#### GROUP BY in External Memory {#select-group-by-in-external-memory}
-
-You can enable dumping temporary data to the disk to restrict memory usage during `GROUP BY`.
-The [max\_bytes\_before\_external\_group\_by](../operations/settings/settings.md#settings-max_bytes_before_external_group_by) setting determines the threshold RAM consumption for dumping `GROUP BY` temporary data to the file system. If set to 0 (the default), it is disabled.
-
-When using `max_bytes_before_external_group_by`, we recommend that you set `max_memory_usage` about twice as high. This is necessary because there are two stages to aggregation: reading the date and forming intermediate data (1) and merging the intermediate data (2). Dumping data to the file system can only occur during stage 1. If the temporary data wasn’t dumped, then stage 2 might require up to the same amount of memory as in stage 1.
-
-For example, if [max\_memory\_usage](../operations/settings/settings.md#settings_max_memory_usage) was set to 10000000000 and you want to use external aggregation, it makes sense to set `max_bytes_before_external_group_by` to 10000000000, and max\_memory\_usage to 20000000000. When external aggregation is triggered (if there was at least one dump of temporary data), maximum consumption of RAM is only slightly more than `max_bytes_before_external_group_by`.
-
-With distributed query processing, external aggregation is performed on remote servers. In order for the requester server to use only a small amount of RAM, set `distributed_aggregation_memory_efficient` to 1.
-
-When merging data flushed to the disk, as well as when merging results from remote servers when the `distributed_aggregation_memory_efficient` setting is enabled, consumes up to `1/256 * the_number_of_threads` from the total amount of RAM.
-
-When external aggregation is enabled, if there was less than `max_bytes_before_external_group_by` of data (i.e. data was not flushed), the query runs just as fast as without external aggregation. If any temporary data was flushed, the run time will be several times longer (approximately three times).
-
-If you have an `ORDER BY` with a `LIMIT` after `GROUP BY`, then the amount of used RAM depends on the amount of data in `LIMIT`, not in the whole table. But if the `ORDER BY` doesn’t have `LIMIT`, don’t forget to enable external sorting (`max_bytes_before_external_sort`).
-
-### LIMIT BY Clause {#limit-by-clause}
-
-A query with the `LIMIT n BY expressions` clause selects the first `n` rows for each distinct value of `expressions`. The key for `LIMIT BY` can contain any number of [expressions](syntax.md#syntax-expressions).
-
-ClickHouse supports the following syntax:
-
-- `LIMIT [offset_value, ]n BY expressions`
-- `LIMIT n OFFSET offset_value BY expressions`
-
-During query processing, ClickHouse selects data ordered by sorting key. The sorting key is set explicitly using an [ORDER BY](#select-order-by) clause or implicitly as a property of the table engine. Then ClickHouse applies `LIMIT n BY expressions` and returns the first `n` rows for each distinct combination of `expressions`. If `OFFSET` is specified, then for each data block that belongs to a distinct combination of `expressions`, ClickHouse skips `offset_value` number of rows from the beginning of the block and returns a maximum of `n` rows as a result. If `offset_value` is bigger than the number of rows in the data block, ClickHouse returns zero rows from the block.
-
-`LIMIT BY` is not related to `LIMIT`. They can both be used in the same query.
-
-**Examples**
-
-Sample table:
-
-``` sql
-CREATE TABLE limit_by(id Int, val Int) ENGINE = Memory;
-INSERT INTO limit_by values(1, 10), (1, 11), (1, 12), (2, 20), (2, 21);
-```
-
-Queries:
-
-``` sql
-SELECT * FROM limit_by ORDER BY id, val LIMIT 2 BY id
-```
-
-``` text
-┌─id─┬─val─┐
-│ 1 │ 10 │
-│ 1 │ 11 │
-│ 2 │ 20 │
-│ 2 │ 21 │
-└────┴─────┘
-```
-
-``` sql
-SELECT * FROM limit_by ORDER BY id, val LIMIT 1, 2 BY id
-```
-
-``` text
-┌─id─┬─val─┐
-│ 1 │ 11 │
-│ 1 │ 12 │
-│ 2 │ 21 │
-└────┴─────┘
-```
-
-The `SELECT * FROM limit_by ORDER BY id, val LIMIT 2 OFFSET 1 BY id` query returns the same result.
-
-The following query returns the top 5 referrers for each `domain, device_type` pair with a maximum of 100 rows in total (`LIMIT n BY + LIMIT`).
-
-``` sql
-SELECT
- domainWithoutWWW(URL) AS domain,
- domainWithoutWWW(REFERRER_URL) AS referrer,
- device_type,
- count() cnt
-FROM hits
-GROUP BY domain, referrer, device_type
-ORDER BY cnt DESC
-LIMIT 5 BY domain, device_type
-LIMIT 100
-```
-
-### HAVING Clause {#having-clause}
-
-Allows filtering the result received after GROUP BY, similar to the WHERE clause.
-WHERE and HAVING differ in that WHERE is performed before aggregation (GROUP BY), while HAVING is performed after it.
-If aggregation is not performed, HAVING can’t be used.
-
-### ORDER BY Clause {#select-order-by}
-
-The ORDER BY clause contains a list of expressions, which can each be assigned DESC or ASC (the sorting direction). If the direction is not specified, ASC is assumed. ASC is sorted in ascending order, and DESC in descending order. The sorting direction applies to a single expression, not to the entire list. Example: `ORDER BY Visits DESC, SearchPhrase`
-
-For sorting by String values, you can specify collation (comparison). Example: `ORDER BY SearchPhrase COLLATE 'tr'` - for sorting by keyword in ascending order, using the Turkish alphabet, case insensitive, assuming that strings are UTF-8 encoded. COLLATE can be specified or not for each expression in ORDER BY independently. If ASC or DESC is specified, COLLATE is specified after it. When using COLLATE, sorting is always case-insensitive.
-
-We only recommend using COLLATE for final sorting of a small number of rows, since sorting with COLLATE is less efficient than normal sorting by bytes.
-
-Rows that have identical values for the list of sorting expressions are output in an arbitrary order, which can also be nondeterministic (different each time).
-If the ORDER BY clause is omitted, the order of the rows is also undefined, and may be nondeterministic as well.
-
-`NaN` and `NULL` sorting order:
-
-- With the modifier `NULLS FIRST` — First `NULL`, then `NaN`, then other values.
-- With the modifier `NULLS LAST` — First the values, then `NaN`, then `NULL`.
-- Default — The same as with the `NULLS LAST` modifier.
-
-Example:
-
-For the table
-
-``` text
-┌─x─┬────y─┐
-│ 1 │ ᴺᵁᴸᴸ │
-│ 2 │ 2 │
-│ 1 │ nan │
-│ 2 │ 2 │
-│ 3 │ 4 │
-│ 5 │ 6 │
-│ 6 │ nan │
-│ 7 │ ᴺᵁᴸᴸ │
-│ 6 │ 7 │
-│ 8 │ 9 │
-└───┴──────┘
-```
-
-Run the query `SELECT * FROM t_null_nan ORDER BY y NULLS FIRST` to get:
-
-``` text
-┌─x─┬────y─┐
-│ 1 │ ᴺᵁᴸᴸ │
-│ 7 │ ᴺᵁᴸᴸ │
-│ 1 │ nan │
-│ 6 │ nan │
-│ 2 │ 2 │
-│ 2 │ 2 │
-│ 3 │ 4 │
-│ 5 │ 6 │
-│ 6 │ 7 │
-│ 8 │ 9 │
-└───┴──────┘
-```
-
-When floating point numbers are sorted, NaNs are separate from the other values. Regardless of the sorting order, NaNs come at the end. In other words, for ascending sorting they are placed as if they are larger than all the other numbers, while for descending sorting they are placed as if they are smaller than the rest.
-
-Less RAM is used if a small enough LIMIT is specified in addition to ORDER BY. Otherwise, the amount of memory spent is proportional to the volume of data for sorting. For distributed query processing, if GROUP BY is omitted, sorting is partially done on remote servers, and the results are merged on the requestor server. This means that for distributed sorting, the volume of data to sort can be greater than the amount of memory on a single server.
-
-If there is not enough RAM, it is possible to perform sorting in external memory (creating temporary files on a disk). Use the setting `max_bytes_before_external_sort` for this purpose. If it is set to 0 (the default), external sorting is disabled. If it is enabled, when the volume of data to sort reaches the specified number of bytes, the collected data is sorted and dumped into a temporary file. After all data is read, all the sorted files are merged and the results are output. Files are written to the /var/lib/clickhouse/tmp/ directory in the config (by default, but you can use the ‘tmp\_path’ parameter to change this setting).
-
-Running a query may use more memory than ‘max\_bytes\_before\_external\_sort’. For this reason, this setting must have a value significantly smaller than ‘max\_memory\_usage’. As an example, if your server has 128 GB of RAM and you need to run a single query, set ‘max\_memory\_usage’ to 100 GB, and ‘max\_bytes\_before\_external\_sort’ to 80 GB.
-
-External sorting works much less effectively than sorting in RAM.
-
-### SELECT Clause {#select-select}
-
-[Expressions](syntax.md#syntax-expressions) specified in the `SELECT` clause are calculated after all the operations in the clauses described above are finished. These expressions work as if they apply to separate rows in the result. If expressions in the `SELECT` clause contain aggregate functions, then ClickHouse processes aggregate functions and expressions used as their arguments during the [GROUP BY](#select-group-by-clause) aggregation.
-
-If you want to include all columns in the result, use the asterisk (`*`) symbol. For example, `SELECT * FROM ...`.
-
-To match some columns in the result with a [re2](https://en.wikipedia.org/wiki/RE2_(software)) regular expression, you can use the `COLUMNS` expression.
-
-``` sql
-COLUMNS('regexp')
-```
-
-For example, consider the table:
-
-``` sql
-CREATE TABLE default.col_names (aa Int8, ab Int8, bc Int8) ENGINE = TinyLog
-```
-
-The following query selects data from all the columns containing the `a` symbol in their name.
-
-``` sql
-SELECT COLUMNS('a') FROM col_names
-```
-
-``` text
-┌─aa─┬─ab─┐
-│ 1 │ 1 │
-└────┴────┘
-```
-
-The selected columns are returned not in the alphabetical order.
-
-You can use multiple `COLUMNS` expressions in a query and apply functions to them.
-
-For example:
-
-``` sql
-SELECT COLUMNS('a'), COLUMNS('c'), toTypeName(COLUMNS('c')) FROM col_names
-```
-
-``` text
-┌─aa─┬─ab─┬─bc─┬─toTypeName(bc)─┐
-│ 1 │ 1 │ 1 │ Int8 │
-└────┴────┴────┴────────────────┘
-```
-
-Each column returned by the `COLUMNS` expression is passed to the function as a separate argument. Also you can pass other arguments to the function if it supports them. Be careful when using functions. If a function doesn’t support the number of arguments you have passed to it, ClickHouse throws an exception.
-
-For example:
-
-``` sql
-SELECT COLUMNS('a') + COLUMNS('c') FROM col_names
-```
-
-``` text
-Received exception from server (version 19.14.1):
-Code: 42. DB::Exception: Received from localhost:9000. DB::Exception: Number of arguments for function plus doesn't match: passed 3, should be 2.
-```
-
-In this example, `COLUMNS('a')` returns two columns: `aa` and `ab`. `COLUMNS('c')` returns the `bc` column. The `+` operator can’t apply to 3 arguments, so ClickHouse throws an exception with the relevant message.
-
-Columns that matched the `COLUMNS` expression can have different data types. If `COLUMNS` doesn’t match any columns and is the only expression in `SELECT`, ClickHouse throws an exception.
-
-### DISTINCT Clause {#select-distinct}
-
-If DISTINCT is specified, only a single row will remain out of all the sets of fully matching rows in the result.
-The result will be the same as if GROUP BY were specified across all the fields specified in SELECT without aggregate functions. But there are several differences from GROUP BY:
-
-- DISTINCT can be applied together with GROUP BY.
-- When ORDER BY is omitted and LIMIT is defined, the query stops running immediately after the required number of different rows has been read.
-- Data blocks are output as they are processed, without waiting for the entire query to finish running.
-
-DISTINCT is not supported if SELECT has at least one array column.
-
-`DISTINCT` works with [NULL](syntax.md) as if `NULL` were a specific value, and `NULL=NULL`. In other words, in the `DISTINCT` results, different combinations with `NULL` only occur once.
-
-ClickHouse supports using the `DISTINCT` and `ORDER BY` clauses for different columns in one query. The `DISTINCT` clause is executed before the `ORDER BY` clause.
-
-Example table:
-
-``` text
-┌─a─┬─b─┐
-│ 2 │ 1 │
-│ 1 │ 2 │
-│ 3 │ 3 │
-│ 2 │ 4 │
-└───┴───┘
-```
-
-When selecting data with the `SELECT DISTINCT a FROM t1 ORDER BY b ASC` query, we get the following result:
-
-``` text
-┌─a─┐
-│ 2 │
-│ 1 │
-│ 3 │
-└───┘
-```
-
-If we change the sorting direction `SELECT DISTINCT a FROM t1 ORDER BY b DESC`, we get the following result:
-
-``` text
-┌─a─┐
-│ 3 │
-│ 1 │
-│ 2 │
-└───┘
-```
-
-Row `2, 4` was cut before sorting.
-
-Take this implementation specificity into account when programming queries.
-
-### LIMIT Clause {#limit-clause}
-
-`LIMIT m` allows you to select the first `m` rows from the result.
-
-`LIMIT n, m` allows you to select the first `m` rows from the result after skipping the first `n` rows. The `LIMIT m OFFSET n` syntax is also supported.
-
-`n` and `m` must be non-negative integers.
-
-If there isn’t an `ORDER BY` clause that explicitly sorts results, the result may be arbitrary and nondeterministic.
-
-### UNION ALL Clause {#union-all-clause}
-
-You can use UNION ALL to combine any number of queries. Example:
-
-``` sql
-SELECT CounterID, 1 AS table, toInt64(count()) AS c
- FROM test.hits
- GROUP BY CounterID
-
-UNION ALL
-
-SELECT CounterID, 2 AS table, sum(Sign) AS c
- FROM test.visits
- GROUP BY CounterID
- HAVING c > 0
-```
-
-Only UNION ALL is supported. The regular UNION (UNION DISTINCT) is not supported. If you need UNION DISTINCT, you can write SELECT DISTINCT from a subquery containing UNION ALL.
-
-Queries that are parts of UNION ALL can be run simultaneously, and their results can be mixed together.
-
-The structure of results (the number and type of columns) must match for the queries. But the column names can differ. In this case, the column names for the final result will be taken from the first query. Type casting is performed for unions. For example, if two queries being combined have the same field with non-`Nullable` and `Nullable` types from a compatible type, the resulting `UNION ALL` has a `Nullable` type field.
-
-Queries that are parts of UNION ALL can’t be enclosed in brackets. ORDER BY and LIMIT are applied to separate queries, not to the final result. If you need to apply a conversion to the final result, you can put all the queries with UNION ALL in a subquery in the FROM clause.
-
-### INTO OUTFILE Clause {#into-outfile-clause}
-
-Add the `INTO OUTFILE filename` clause (where filename is a string literal) to redirect query output to the specified file.
-In contrast to MySQL, the file is created on the client side. The query will fail if a file with the same filename already exists.
-This functionality is available in the command-line client and clickhouse-local (a query sent via HTTP interface will fail).
-
-The default output format is TabSeparated (the same as in the command-line client batch mode).
-
-### FORMAT Clause {#format-clause}
-
-Specify ‘FORMAT format’ to get data in any specified format.
-You can use this for convenience, or for creating dumps.
-For more information, see the section “Formats”.
-If the FORMAT clause is omitted, the default format is used, which depends on both the settings and the interface used for accessing the DB. For the HTTP interface and the command-line client in batch mode, the default format is TabSeparated. For the command-line client in interactive mode, the default format is PrettyCompact (it has attractive and compact tables).
-
-When using the command-line client, data is passed to the client in an internal efficient format. The client independently interprets the FORMAT clause of the query and formats the data itself (thus relieving the network and the server from the load).
-
-### IN Operators {#select-in-operators}
-
-The `IN`, `NOT IN`, `GLOBAL IN`, and `GLOBAL NOT IN` operators are covered separately, since their functionality is quite rich.
-
-The left side of the operator is either a single column or a tuple.
-
-Examples:
-
-``` sql
-SELECT UserID IN (123, 456) FROM ...
-SELECT (CounterID, UserID) IN ((34, 123), (101500, 456)) FROM ...
-```
-
-If the left side is a single column that is in the index, and the right side is a set of constants, the system uses the index for processing the query.
-
-Don’t list too many values explicitly (i.e. millions). If a data set is large, put it in a temporary table (for example, see the section “External data for query processing”), then use a subquery.
-
-The right side of the operator can be a set of constant expressions, a set of tuples with constant expressions (shown in the examples above), or the name of a database table or SELECT subquery in brackets.
-
-If the right side of the operator is the name of a table (for example, `UserID IN users`), this is equivalent to the subquery `UserID IN (SELECT * FROM users)`. Use this when working with external data that is sent along with the query. For example, the query can be sent together with a set of user IDs loaded to the ‘users’ temporary table, which should be filtered.
-
-If the right side of the operator is a table name that has the Set engine (a prepared data set that is always in RAM), the data set will not be created over again for each query.
-
-The subquery may specify more than one column for filtering tuples.
-Example:
-
-``` sql
-SELECT (CounterID, UserID) IN (SELECT CounterID, UserID FROM ...) FROM ...
-```
-
-The columns to the left and right of the IN operator should have the same type.
-
-The IN operator and subquery may occur in any part of the query, including in aggregate functions and lambda functions.
-Example:
-
-``` sql
-SELECT
- EventDate,
- avg(UserID IN
- (
- SELECT UserID
- FROM test.hits
- WHERE EventDate = toDate('2014-03-17')
- )) AS ratio
-FROM test.hits
-GROUP BY EventDate
-ORDER BY EventDate ASC
-```
-
-``` text
-┌──EventDate─┬────ratio─┐
-│ 2014-03-17 │ 1 │
-│ 2014-03-18 │ 0.807696 │
-│ 2014-03-19 │ 0.755406 │
-│ 2014-03-20 │ 0.723218 │
-│ 2014-03-21 │ 0.697021 │
-│ 2014-03-22 │ 0.647851 │
-│ 2014-03-23 │ 0.648416 │
-└────────────┴──────────┘
-```
-
-For each day after March 17th, count the percentage of pageviews made by users who visited the site on March 17th.
-A subquery in the IN clause is always run just one time on a single server. There are no dependent subqueries.
-
-#### NULL processing {#null-processing-1}
-
-During request processing, the IN operator assumes that the result of an operation with [NULL](syntax.md) is always equal to `0`, regardless of whether `NULL` is on the right or left side of the operator. `NULL` values are not included in any dataset, do not correspond to each other and cannot be compared.
-
-Here is an example with the `t_null` table:
-
-``` text
-┌─x─┬────y─┐
-│ 1 │ ᴺᵁᴸᴸ │
-│ 2 │ 3 │
-└───┴──────┘
-```
-
-Running the query `SELECT x FROM t_null WHERE y IN (NULL,3)` gives you the following result:
-
-``` text
-┌─x─┐
-│ 2 │
-└───┘
-```
-
-You can see that the row in which `y = NULL` is thrown out of the query results. This is because ClickHouse can’t decide whether `NULL` is included in the `(NULL,3)` set, returns `0` as the result of the operation, and `SELECT` excludes this row from the final output.
-
-``` sql
-SELECT y IN (NULL, 3)
-FROM t_null
-```
-
-``` text
-┌─in(y, tuple(NULL, 3))─┐
-│ 0 │
-│ 1 │
-└───────────────────────┘
-```
-
-#### Distributed Subqueries {#select-distributed-subqueries}
-
-There are two options for IN-s with subqueries (similar to JOINs): normal `IN` / `JOIN` and `GLOBAL IN` / `GLOBAL JOIN`. They differ in how they are run for distributed query processing.
-
-!!! attention "Attention"
- Remember that the algorithms described below may work differently depending on the [settings](../operations/settings/settings.md) `distributed_product_mode` setting.
-
-When using the regular IN, the query is sent to remote servers, and each of them runs the subqueries in the `IN` or `JOIN` clause.
-
-When using `GLOBAL IN` / `GLOBAL JOINs`, first all the subqueries are run for `GLOBAL IN` / `GLOBAL JOINs`, and the results are collected in temporary tables. Then the temporary tables are sent to each remote server, where the queries are run using this temporary data.
-
-For a non-distributed query, use the regular `IN` / `JOIN`.
-
-Be careful when using subqueries in the `IN` / `JOIN` clauses for distributed query processing.
-
-Let’s look at some examples. Assume that each server in the cluster has a normal **local\_table**. Each server also has a **distributed\_table** table with the **Distributed** type, which looks at all the servers in the cluster.
-
-For a query to the **distributed\_table**, the query will be sent to all the remote servers and run on them using the **local\_table**.
-
-For example, the query
-
-``` sql
-SELECT uniq(UserID) FROM distributed_table
-```
-
-will be sent to all remote servers as
-
-``` sql
-SELECT uniq(UserID) FROM local_table
-```
-
-and run on each of them in parallel, until it reaches the stage where intermediate results can be combined. Then the intermediate results will be returned to the requestor server and merged on it, and the final result will be sent to the client.
-
-Now let’s examine a query with IN:
-
-``` sql
-SELECT uniq(UserID) FROM distributed_table WHERE CounterID = 101500 AND UserID IN (SELECT UserID FROM local_table WHERE CounterID = 34)
-```
-
-- Calculation of the intersection of audiences of two sites.
-
-This query will be sent to all remote servers as
-
-``` sql
-SELECT uniq(UserID) FROM local_table WHERE CounterID = 101500 AND UserID IN (SELECT UserID FROM local_table WHERE CounterID = 34)
-```
-
-In other words, the data set in the IN clause will be collected on each server independently, only across the data that is stored locally on each of the servers.
-
-This will work correctly and optimally if you are prepared for this case and have spread data across the cluster servers such that the data for a single UserID resides entirely on a single server. In this case, all the necessary data will be available locally on each server. Otherwise, the result will be inaccurate. We refer to this variation of the query as “local IN”.
-
-To correct how the query works when data is spread randomly across the cluster servers, you could specify **distributed\_table** inside a subquery. The query would look like this:
-
-``` sql
-SELECT uniq(UserID) FROM distributed_table WHERE CounterID = 101500 AND UserID IN (SELECT UserID FROM distributed_table WHERE CounterID = 34)
-```
-
-This query will be sent to all remote servers as
-
-``` sql
-SELECT uniq(UserID) FROM local_table WHERE CounterID = 101500 AND UserID IN (SELECT UserID FROM distributed_table WHERE CounterID = 34)
-```
-
-The subquery will begin running on each remote server. Since the subquery uses a distributed table, the subquery that is on each remote server will be resent to every remote server as
-
-``` sql
-SELECT UserID FROM local_table WHERE CounterID = 34
-```
-
-For example, if you have a cluster of 100 servers, executing the entire query will require 10,000 elementary requests, which is generally considered unacceptable.
-
-In such cases, you should always use GLOBAL IN instead of IN. Let’s look at how it works for the query
-
-``` sql
-SELECT uniq(UserID) FROM distributed_table WHERE CounterID = 101500 AND UserID GLOBAL IN (SELECT UserID FROM distributed_table WHERE CounterID = 34)
-```
-
-The requestor server will run the subquery
-
-``` sql
-SELECT UserID FROM distributed_table WHERE CounterID = 34
-```
-
-and the result will be put in a temporary table in RAM. Then the request will be sent to each remote server as
-
-``` sql
-SELECT uniq(UserID) FROM local_table WHERE CounterID = 101500 AND UserID GLOBAL IN _data1
-```
-
-and the temporary table `_data1` will be sent to every remote server with the query (the name of the temporary table is implementation-defined).
-
-This is more optimal than using the normal IN. However, keep the following points in mind:
-
-1. When creating a temporary table, data is not made unique. To reduce the volume of data transmitted over the network, specify DISTINCT in the subquery. (You don’t need to do this for a normal IN.)
-2. The temporary table will be sent to all the remote servers. Transmission does not account for network topology. For example, if 10 remote servers reside in a datacenter that is very remote in relation to the requestor server, the data will be sent 10 times over the channel to the remote datacenter. Try to avoid large data sets when using GLOBAL IN.
-3. When transmitting data to remote servers, restrictions on network bandwidth are not configurable. You might overload the network.
-4. Try to distribute data across servers so that you don’t need to use GLOBAL IN on a regular basis.
-5. If you need to use GLOBAL IN often, plan the location of the ClickHouse cluster so that a single group of replicas resides in no more than one data center with a fast network between them, so that a query can be processed entirely within a single data center.
-
-It also makes sense to specify a local table in the `GLOBAL IN` clause, in case this local table is only available on the requestor server and you want to use data from it on remote servers.
-
-### Extreme Values {#extreme-values}
-
-In addition to results, you can also get minimum and maximum values for the results columns. To do this, set the **extremes** setting to 1. Minimums and maximums are calculated for numeric types, dates, and dates with times. For other columns, the default values are output.
-
-An extra two rows are calculated – the minimums and maximums, respectively. These extra two rows are output in `JSON*`, `TabSeparated*`, and `Pretty*` [formats](../interfaces/formats.md), separate from the other rows. They are not output for other formats.
-
-In `JSON*` formats, the extreme values are output in a separate ‘extremes’ field. In `TabSeparated*` formats, the row comes after the main result, and after ‘totals’ if present. It is preceded by an empty row (after the other data). In `Pretty*` formats, the row is output as a separate table after the main result, and after `totals` if present.
-
-Extreme values are calculated for rows before `LIMIT`, but after `LIMIT BY`. However, when using `LIMIT offset, size`, the rows before `offset` are included in `extremes`. In stream requests, the result may also include a small number of rows that passed through `LIMIT`.
-
-### Notes {#notes}
-
-The `GROUP BY` and `ORDER BY` clauses do not support positional arguments. This contradicts MySQL, but conforms to standard SQL.
-For example, `GROUP BY 1, 2` will be interpreted as grouping by constants (i.e. aggregation of all rows into one).
-
-You can use synonyms (`AS` aliases) in any part of a query.
-
-You can put an asterisk in any part of a query instead of an expression. When the query is analyzed, the asterisk is expanded to a list of all table columns (excluding the `MATERIALIZED` and `ALIAS` columns). There are only a few cases when using an asterisk is justified:
-
-- When creating a table dump.
-- For tables containing just a few columns, such as system tables.
-- For getting information about what columns are in a table. In this case, set `LIMIT 1`. But it is better to use the `DESC TABLE` query.
-- When there is strong filtration on a small number of columns using `PREWHERE`.
-- In subqueries (since columns that aren’t needed for the external query are excluded from subqueries).
-
-In all other cases, we don’t recommend using the asterisk, since it only gives you the drawbacks of a columnar DBMS instead of the advantages. In other words using the asterisk is not recommended.
-
-[Original article](https://clickhouse.tech/docs/en/query_language/select/)
diff --git a/docs/en/query_language/table_functions/index.md b/docs/en/query_language/table_functions/index.md
deleted file mode 100644
index ea07d5a5b00..00000000000
--- a/docs/en/query_language/table_functions/index.md
+++ /dev/null
@@ -1,30 +0,0 @@
-# Table Functions {#table-functions}
-
-Table functions are methods for constructing tables.
-
-You can use table functions in:
-
-- [FROM](../select.md#select-from) clause of the `SELECT` query.
-
- The method for creating a temporary table that is available only in the current query. The table is deleted when the query finishes.
-
-- [CREATE TABLE AS \](../create.md#create-table-query) query.
-
- It's one of the methods of creating a table.
-
-!!! warning "Warning"
- You can’t use table functions if the [allow\_ddl](../../operations/settings/permissions_for_queries.md#settings_allow_ddl) setting is disabled.
-
-| Function | Description |
-|-----------------------|-----------------------------------------------------------------------------------------------------------------------------------|
-| [file](file.md) | Creates a [File](../../operations/table_engines/file.md)-engine table. |
-| [merge](merge.md) | Creates a [Merge](../../operations/table_engines/merge.md)-engine table. |
-| [numbers](numbers.md) | Creates a table with a single column filled with integer numbers. |
-| [remote](remote.md) | Allows you to access remote servers without creating a [Distributed](../../operations/table_engines/distributed.md)-engine table. |
-| [url](url.md) | Creates a [Url](../../operations/table_engines/url.md)-engine table. |
-| [mysql](mysql.md) | Creates a [MySQL](../../operations/table_engines/mysql.md)-engine table. |
-| [jdbc](jdbc.md) | Creates a [JDBC](../../operations/table_engines/jdbc.md)-engine table. |
-| [odbc](odbc.md) | Creates a [ODBC](../../operations/table_engines/odbc.md)-engine table. |
-| [hdfs](hdfs.md) | Creates a [HDFS](../../operations/table_engines/hdfs.md)-engine table. |
-
-[Original article](https://clickhouse.tech/docs/en/query_language/table_functions/)
diff --git a/docs/en/query_language/agg_functions/combinators.md b/docs/en/sql_reference/aggregate_functions/combinators.md
similarity index 94%
rename from docs/en/query_language/agg_functions/combinators.md
rename to docs/en/sql_reference/aggregate_functions/combinators.md
index e0f93263fc6..46a680a7978 100644
--- a/docs/en/query_language/agg_functions/combinators.md
+++ b/docs/en/sql_reference/aggregate_functions/combinators.md
@@ -1,4 +1,9 @@
-# Aggregate function combinators {#aggregate_functions_combinators}
+---
+toc_priority: 37
+toc_title: Aggregate function combinators
+---
+
+# Aggregate Function Combinators {#aggregate_functions_combinators}
The name of an aggregate function can have a suffix appended to it. This changes the way the aggregate function works.
@@ -26,9 +31,9 @@ If you apply this combinator, the aggregate function doesn’t return the result
To work with these states, use:
-- [AggregatingMergeTree](../../operations/table_engines/aggregatingmergetree.md) table engine.
-- [finalizeAggregation](../functions/other_functions.md#function-finalizeaggregation) function.
-- [runningAccumulate](../functions/other_functions.md#function-runningaccumulate) function.
+- [AggregatingMergeTree](../../engines/table_engines/mergetree_family/aggregatingmergetree.md) table engine.
+- [finalizeAggregation](../../sql_reference/functions/other_functions.md#function-finalizeaggregation) function.
+- [runningAccumulate](../../sql_reference/functions/other_functions.md#function-runningaccumulate) function.
- [-Merge](#aggregate_functions_combinators_merge) combinator.
- [-MergeState](#aggregate_functions_combinators_mergestate) combinator.
diff --git a/docs/en/query_language/agg_functions/index.md b/docs/en/sql_reference/aggregate_functions/index.md
similarity index 94%
rename from docs/en/query_language/agg_functions/index.md
rename to docs/en/sql_reference/aggregate_functions/index.md
index 18f1555f9dc..9a7b8d09a07 100644
--- a/docs/en/query_language/agg_functions/index.md
+++ b/docs/en/sql_reference/aggregate_functions/index.md
@@ -1,3 +1,9 @@
+---
+toc_folder_title: Aggregate Functions
+toc_priority: 33
+toc_title: Introduction
+---
+
# Aggregate functions {#aggregate-functions}
Aggregate functions work in the [normal](http://www.sql-tutorial.com/sql-aggregate-functions-sql-tutorial) way as expected by database experts.
diff --git a/docs/en/query_language/agg_functions/parametric_functions.md b/docs/en/sql_reference/aggregate_functions/parametric_functions.md
similarity index 94%
rename from docs/en/query_language/agg_functions/parametric_functions.md
rename to docs/en/sql_reference/aggregate_functions/parametric_functions.md
index c5b6a983551..fd99931a34b 100644
--- a/docs/en/query_language/agg_functions/parametric_functions.md
+++ b/docs/en/sql_reference/aggregate_functions/parametric_functions.md
@@ -1,4 +1,9 @@
-# Parametric aggregate functions {#aggregate_functions_parametric}
+---
+toc_priority: 38
+toc_title: Parametric aggregate functions
+---
+
+# Parametric Aggregate Functions {#aggregate_functions_parametric}
Some aggregate functions can accept not only argument columns (used for compression), but a set of parameters – constants for initialization. The syntax is two pairs of brackets instead of one. The first is for parameters, and the second is for arguments.
@@ -19,7 +24,7 @@ The functions uses [A Streaming Parallel Decision Tree Algorithm](http://jmlr.or
**Returned values**
-- [Array](../../data_types/array.md) of [Tuples](../../data_types/tuple.md) of the following format:
+- [Array](../../sql_reference/data_types/array.md) of [Tuples](../../sql_reference/data_types/tuple.md) of the following format:
```
[(lower_1, upper_1, height_1), ... (lower_N, upper_N, height_N)]
@@ -46,7 +51,7 @@ FROM (
└─────────────────────────────────────────────────────────────────────────┘
```
-You can visualize a histogram with the [bar](../functions/other_functions.md#function-bar) function, for example:
+You can visualize a histogram with the [bar](../../sql_reference/functions/other_functions.md#function-bar) function, for example:
``` sql
WITH histogram(5)(rand() % 100) AS hist
@@ -88,7 +93,7 @@ sequenceMatch(pattern)(timestamp, cond1, cond2, ...)
- `pattern` — Pattern string. See [Pattern syntax](#sequence-function-pattern-syntax).
-- `timestamp` — Column considered to contain time data. Typical data types are `Date` and `DateTime`. You can also use any of the supported [UInt](../../data_types/int_uint.md) data types.
+- `timestamp` — Column considered to contain time data. Typical data types are `Date` and `DateTime`. You can also use any of the supported [UInt](../../sql_reference/data_types/int_uint.md) data types.
- `cond1`, `cond2` — Conditions that describe the chain of events. Data type: `UInt8`. You can pass up to 32 condition arguments. The function takes only the events described in these conditions into account. If the sequence contains data that isn’t described in a condition, the function skips them.
@@ -175,7 +180,7 @@ sequenceCount(pattern)(timestamp, cond1, cond2, ...)
- `pattern` — Pattern string. See [Pattern syntax](#sequence-function-pattern-syntax).
-- `timestamp` — Column considered to contain time data. Typical data types are `Date` and `DateTime`. You can also use any of the supported [UInt](../../data_types/int_uint.md) data types.
+- `timestamp` — Column considered to contain time data. Typical data types are `Date` and `DateTime`. You can also use any of the supported [UInt](../../sql_reference/data_types/int_uint.md) data types.
- `cond1`, `cond2` — Conditions that describe the chain of events. Data type: `UInt8`. You can pass up to 32 condition arguments. The function takes only the events described in these conditions into account. If the sequence contains data that isn’t described in a condition, the function skips them.
@@ -239,8 +244,8 @@ windowFunnel(window, [mode])(timestamp, cond1, cond2, ..., condN)
- `window` — Length of the sliding window in seconds.
- `mode` - It is an optional argument.
- `'strict'` - When the `'strict'` is set, the windowFunnel() applies conditions only for the unique values.
-- `timestamp` — Name of the column containing the timestamp. Data types supported: [Date](../../data_types/date.md), [DateTime](../../data_types/datetime.md#data_type-datetime) and other unsigned integer types (note that even though timestamp supports the `UInt64` type, it’s value can’t exceed the Int64 maximum, which is 2^63 - 1).
-- `cond` — Conditions or data describing the chain of events. [UInt8](../../data_types/int_uint.md).
+- `timestamp` — Name of the column containing the timestamp. Data types supported: [Date](../../sql_reference/data_types/date.md), [DateTime](../../sql_reference/data_types/datetime.md#data_type-datetime) and other unsigned integer types (note that even though timestamp supports the `UInt64` type, it’s value can’t exceed the Int64 maximum, which is 2^63 - 1).
+- `cond` — Conditions or data describing the chain of events. [UInt8](../../sql_reference/data_types/int_uint.md).
**Returned value**
@@ -309,7 +314,7 @@ Result:
## retention {#retention}
The function takes as arguments a set of conditions from 1 to 32 arguments of type `UInt8` that indicate whether a certain condition was met for the event.
-Any condition can be specified as an argument (as in [WHERE](../../query_language/select.md#select-where)).
+Any condition can be specified as an argument (as in [WHERE](../../sql_reference/statements/select.md#select-where)).
The conditions, except the first, apply in pairs: the result of the second will be true if the first and second are true, of the third if the first and fird are true, etc.
diff --git a/docs/en/query_language/agg_functions/reference.md b/docs/en/sql_reference/aggregate_functions/reference.md
similarity index 92%
rename from docs/en/query_language/agg_functions/reference.md
rename to docs/en/sql_reference/aggregate_functions/reference.md
index 1f276e62417..2c9eebb803b 100644
--- a/docs/en/query_language/agg_functions/reference.md
+++ b/docs/en/sql_reference/aggregate_functions/reference.md
@@ -1,3 +1,8 @@
+---
+toc_priority: 36
+toc_title: Reference
+---
+
# Function Reference {#function-reference}
## count {#agg_function-count}
@@ -18,9 +23,9 @@ The function can take:
**Returned value**
- If the function is called without parameters it counts the number of rows.
-- If the [expression](../syntax.md#syntax-expressions) is passed, then the function counts how many times this expression returned not null. If the expression returns a [Nullable](../../data_types/nullable.md)-type value, then the result of `count` stays not `Nullable`. The function returns 0 if the expression returned `NULL` for all the rows.
+- If the [expression](../syntax.md#syntax-expressions) is passed, then the function counts how many times this expression returned not null. If the expression returns a [Nullable](../../sql_reference/data_types/nullable.md)-type value, then the result of `count` stays not `Nullable`. The function returns 0 if the expression returned `NULL` for all the rows.
-In both cases the type of the returned value is [UInt64](../../data_types/int_uint.md).
+In both cases the type of the returned value is [UInt64](../../sql_reference/data_types/int_uint.md).
**Details**
@@ -239,7 +244,7 @@ binary decimal
## groupBitmap {#groupbitmap}
-Bitmap or Aggregate calculations from a unsigned integer column, return cardinality of type UInt64, if add suffix -State, then return [bitmap object](../functions/bitmap_functions.md).
+Bitmap or Aggregate calculations from a unsigned integer column, return cardinality of type UInt64, if add suffix -State, then return [bitmap object](../../sql_reference/functions/bitmap_functions.md).
``` sql
groupBitmap(expr)
@@ -375,7 +380,7 @@ skewPop(expr)
**Returned value**
-The skewness of the given distribution. Type — [Float64](../../data_types/float.md)
+The skewness of the given distribution. Type — [Float64](../../sql_reference/data_types/float.md)
**Example**
@@ -399,7 +404,7 @@ skewSamp(expr)
**Returned value**
-The skewness of the given distribution. Type — [Float64](../../data_types/float.md). If `n <= 1` (`n` is the size of the sample), then the function returns `nan`.
+The skewness of the given distribution. Type — [Float64](../../sql_reference/data_types/float.md). If `n <= 1` (`n` is the size of the sample), then the function returns `nan`.
**Example**
@@ -421,7 +426,7 @@ kurtPop(expr)
**Returned value**
-The kurtosis of the given distribution. Type — [Float64](../../data_types/float.md)
+The kurtosis of the given distribution. Type — [Float64](../../sql_reference/data_types/float.md)
**Example**
@@ -445,7 +450,7 @@ kurtSamp(expr)
**Returned value**
-The kurtosis of the given distribution. Type — [Float64](../../data_types/float.md). If `n <= 1` (`n` is a size of the sample), then the function returns `nan`.
+The kurtosis of the given distribution. Type — [Float64](../../sql_reference/data_types/float.md). If `n <= 1` (`n` is a size of the sample), then the function returns `nan`.
**Example**
@@ -536,7 +541,7 @@ The function takes a variable number of parameters. Parameters can be `Tuple`, `
**Returned value**
-- A [UInt64](../../data_types/int_uint.md)-type number.
+- A [UInt64](../../sql_reference/data_types/int_uint.md)-type number.
**Implementation details**
@@ -577,7 +582,7 @@ The function takes a variable number of parameters. Parameters can be `Tuple`, `
**Returned value**
-- A number [UInt64](../../data_types/int_uint.md)-type number.
+- A number [UInt64](../../sql_reference/data_types/int_uint.md)-type number.
**Implementation details**
@@ -625,7 +630,7 @@ The function takes a variable number of parameters. Parameters can be `Tuple`, `
**Returned value**
-- A [UInt64](../../data_types/int_uint.md)-type number.
+- A [UInt64](../../sql_reference/data_types/int_uint.md)-type number.
**Implementation details**
@@ -867,7 +872,7 @@ Alias: `median`.
**Parameters**
- `level` — Level of quantile. Optional parameter. Constant floating-point number from 0 to 1. We recommend using a `level` value in the range of `[0.01, 0.99]`. Default value: 0.5. At `level=0.5` the function calculates [median](https://en.wikipedia.org/wiki/Median).
-- `expr` — Expression over the column values resulting in numeric [data types](../../data_types/index.md#data_types), [Date](../../data_types/date.md) or [DateTime](../../data_types/datetime.md).
+- `expr` — Expression over the column values resulting in numeric [data types](../../sql_reference/data_types/index.md#data_types), [Date](../../sql_reference/data_types/date.md) or [DateTime](../../sql_reference/data_types/datetime.md).
**Returned value**
@@ -875,9 +880,9 @@ Alias: `median`.
Type:
-- [Float64](../../data_types/float.md) for numeric data type input.
-- [Date](../../data_types/date.md) if input values have the `Date` type.
-- [DateTime](../../data_types/datetime.md) if input values have the `DateTime` type.
+- [Float64](../../sql_reference/data_types/float.md) for numeric data type input.
+- [Date](../../sql_reference/data_types/date.md) if input values have the `Date` type.
+- [DateTime](../../sql_reference/data_types/datetime.md) if input values have the `DateTime` type.
**Example**
@@ -930,7 +935,7 @@ Alias: `medianDeterministic`.
**Parameters**
- `level` — Level of quantile. Optional parameter. Constant floating-point number from 0 to 1. We recommend using a `level` value in the range of `[0.01, 0.99]`. Default value: 0.5. At `level=0.5` the function calculates [median](https://en.wikipedia.org/wiki/Median).
-- `expr` — Expression over the column values resulting in numeric [data types](../../data_types/index.md#data_types), [Date](../../data_types/date.md) or [DateTime](../../data_types/datetime.md).
+- `expr` — Expression over the column values resulting in numeric [data types](../../sql_reference/data_types/index.md#data_types), [Date](../../sql_reference/data_types/date.md) or [DateTime](../../sql_reference/data_types/datetime.md).
- `determinator` — Number whose hash is used instead of a random number generator in the reservoir sampling algorithm to make the result of sampling deterministic. As a determinator you can use any deterministic positive number, for example, a user id or an event id. If the same determinator value occures too often, the function works incorrectly.
**Returned value**
@@ -939,9 +944,9 @@ Alias: `medianDeterministic`.
Type:
-- [Float64](../../data_types/float.md) for numeric data type input.
-- [Date](../../data_types/date.md) if input values have the `Date` type.
-- [DateTime](../../data_types/datetime.md) if input values have the `DateTime` type.
+- [Float64](../../sql_reference/data_types/float.md) for numeric data type input.
+- [Date](../../sql_reference/data_types/date.md) if input values have the `Date` type.
+- [DateTime](../../sql_reference/data_types/datetime.md) if input values have the `DateTime` type.
**Example**
@@ -994,7 +999,7 @@ Alias: `medianExact`.
**Parameters**
- `level` — Level of quantile. Optional parameter. Constant floating-point number from 0 to 1. We recommend using a `level` value in the range of `[0.01, 0.99]`. Default value: 0.5. At `level=0.5` the function calculates [median](https://en.wikipedia.org/wiki/Median).
-- `expr` — Expression over the column values resulting in numeric [data types](../../data_types/index.md#data_types), [Date](../../data_types/date.md) or [DateTime](../../data_types/datetime.md).
+- `expr` — Expression over the column values resulting in numeric [data types](../../sql_reference/data_types/index.md#data_types), [Date](../../sql_reference/data_types/date.md) or [DateTime](../../sql_reference/data_types/datetime.md).
**Returned value**
@@ -1002,9 +1007,9 @@ Alias: `medianExact`.
Type:
-- [Float64](../../data_types/float.md) for numeric data type input.
-- [Date](../../data_types/date.md) if input values have the `Date` type.
-- [DateTime](../../data_types/datetime.md) if input values have the `DateTime` type.
+- [Float64](../../sql_reference/data_types/float.md) for numeric data type input.
+- [Date](../../sql_reference/data_types/date.md) if input values have the `Date` type.
+- [DateTime](../../sql_reference/data_types/datetime.md) if input values have the `DateTime` type.
**Example**
@@ -1046,7 +1051,7 @@ Alias: `medianExactWeighted`.
**Parameters**
- `level` — Level of quantile. Optional parameter. Constant floating-point number from 0 to 1. We recommend using a `level` value in the range of `[0.01, 0.99]`. Default value: 0.5. At `level=0.5` the function calculates [median](https://en.wikipedia.org/wiki/Median).
-- `expr` — Expression over the column values resulting in numeric [data types](../../data_types/index.md#data_types), [Date](../../data_types/date.md) or [DateTime](../../data_types/datetime.md).
+- `expr` — Expression over the column values resulting in numeric [data types](../../sql_reference/data_types/index.md#data_types), [Date](../../sql_reference/data_types/date.md) or [DateTime](../../sql_reference/data_types/datetime.md).
- `weight` — Column with weights of sequence members. Weight is a number of value occurrences.
**Returned value**
@@ -1055,9 +1060,9 @@ Alias: `medianExactWeighted`.
Type:
-- [Float64](../../data_types/float.md) for numeric data type input.
-- [Date](../../data_types/date.md) if input values have the `Date` type.
-- [DateTime](../../data_types/datetime.md) if input values have the `DateTime` type.
+- [Float64](../../sql_reference/data_types/float.md) for numeric data type input.
+- [Date](../../sql_reference/data_types/date.md) if input values have the `Date` type.
+- [DateTime](../../sql_reference/data_types/datetime.md) if input values have the `DateTime` type.
**Example**
@@ -1111,7 +1116,7 @@ Alias: `medianTiming`.
- `level` — Level of quantile. Optional parameter. Constant floating-point number from 0 to 1. We recommend using a `level` value in the range of `[0.01, 0.99]`. Default value: 0.5. At `level=0.5` the function calculates [median](https://en.wikipedia.org/wiki/Median).
-- `expr` — [Expression](../syntax.md#syntax-expressions) over a column values returning a [Float\*](../../data_types/float.md)-type number.
+- `expr` — [Expression](../syntax.md#syntax-expressions) over a column values returning a [Float\*](../../sql_reference/data_types/float.md)-type number.
- If negative values are passed to the function, the behavior is undefined.
- If the value is greater than 30,000 (a page loading time of more than 30 seconds), it is assumed to be 30,000.
@@ -1135,7 +1140,7 @@ Otherwise, the result of the calculation is rounded to the nearest multiple of 1
Type: `Float32`.
!!! note "Note"
- If no values are passed to the function (when using `quantileTimingIf`), [NaN](../../data_types/float.md#data_type-float-nan-inf) is returned. The purpose of this is to differentiate these cases from cases that result in zero. See [ORDER BY clause](../select.md#select-order-by) for notes on sorting `NaN` values.
+ If no values are passed to the function (when using `quantileTimingIf`), [NaN](../../sql_reference/data_types/float.md#data_type-float-nan-inf) is returned. The purpose of this is to differentiate these cases from cases that result in zero. See [ORDER BY clause](../statements/select.md#select-order-by) for notes on sorting `NaN` values.
**Example**
@@ -1194,7 +1199,7 @@ Alias: `medianTimingWeighted`.
- `level` — Level of quantile. Optional parameter. Constant floating-point number from 0 to 1. We recommend using a `level` value in the range of `[0.01, 0.99]`. Default value: 0.5. At `level=0.5` the function calculates [median](https://en.wikipedia.org/wiki/Median).
-- `expr` — [Expression](../syntax.md#syntax-expressions) over a column values returning a [Float\*](../../data_types/float.md)-type number.
+- `expr` — [Expression](../syntax.md#syntax-expressions) over a column values returning a [Float\*](../../sql_reference/data_types/float.md)-type number.
- If negative values are passed to the function, the behavior is undefined.
- If the value is greater than 30,000 (a page loading time of more than 30 seconds), it is assumed to be 30,000.
@@ -1220,7 +1225,7 @@ Otherwise, the result of the calculation is rounded to the nearest multiple of 1
Type: `Float32`.
!!! note "Note"
- If no values are passed to the function (when using `quantileTimingIf`), [NaN](../../data_types/float.md#data_type-float-nan-inf) is returned. The purpose of this is to differentiate these cases from cases that result in zero. See [ORDER BY clause](../select.md#select-order-by) for notes on sorting `NaN` values.
+ If no values are passed to the function (when using `quantileTimingIf`), [NaN](../../sql_reference/data_types/float.md#data_type-float-nan-inf) is returned. The purpose of this is to differentiate these cases from cases that result in zero. See [ORDER BY clause](../statements/select.md#select-order-by) for notes on sorting `NaN` values.
**Example**
@@ -1277,7 +1282,7 @@ Alias: `medianTDigest`.
**Parameters**
- `level` — Level of quantile. Optional parameter. Constant floating-point number from 0 to 1. We recommend using a `level` value in the range of `[0.01, 0.99]`. Default value: 0.5. At `level=0.5` the function calculates [median](https://en.wikipedia.org/wiki/Median).
-- `expr` — Expression over the column values resulting in numeric [data types](../../data_types/index.md#data_types), [Date](../../data_types/date.md) or [DateTime](../../data_types/datetime.md).
+- `expr` — Expression over the column values resulting in numeric [data types](../../sql_reference/data_types/index.md#data_types), [Date](../../sql_reference/data_types/date.md) or [DateTime](../../sql_reference/data_types/datetime.md).
**Returned value**
@@ -1285,9 +1290,9 @@ Alias: `medianTDigest`.
Type:
-- [Float64](../../data_types/float.md) for numeric data type input.
-- [Date](../../data_types/date.md) if input values have the `Date` type.
-- [DateTime](../../data_types/datetime.md) if input values have the `DateTime` type.
+- [Float64](../../sql_reference/data_types/float.md) for numeric data type input.
+- [Date](../../sql_reference/data_types/date.md) if input values have the `Date` type.
+- [DateTime](../../sql_reference/data_types/datetime.md) if input values have the `DateTime` type.
**Example**
@@ -1331,7 +1336,7 @@ Alias: `medianTDigest`.
**Parameters**
- `level` — Level of quantile. Optional parameter. Constant floating-point number from 0 to 1. We recommend using a `level` value in the range of `[0.01, 0.99]`. Default value: 0.5. At `level=0.5` the function calculates [median](https://en.wikipedia.org/wiki/Median).
-- `expr` — Expression over the column values resulting in numeric [data types](../../data_types/index.md#data_types), [Date](../../data_types/date.md) or [DateTime](../../data_types/datetime.md).
+- `expr` — Expression over the column values resulting in numeric [data types](../../sql_reference/data_types/index.md#data_types), [Date](../../sql_reference/data_types/date.md) or [DateTime](../../sql_reference/data_types/datetime.md).
- `weight` — Column with weights of sequence elements. Weight is a number of value occurrences.
**Returned value**
@@ -1340,9 +1345,9 @@ Alias: `medianTDigest`.
Type:
-- [Float64](../../data_types/float.md) for numeric data type input.
-- [Date](../../data_types/date.md) if input values have the `Date` type.
-- [DateTime](../../data_types/datetime.md) if input values have the `DateTime` type.
+- [Float64](../../sql_reference/data_types/float.md) for numeric data type input.
+- [Date](../../sql_reference/data_types/date.md) if input values have the `Date` type.
+- [DateTime](../../sql_reference/data_types/datetime.md) if input values have the `DateTime` type.
**Example**
@@ -1489,7 +1494,7 @@ topKWeighted(N)(x, weight)
**Arguments**
- `x` – The value.
-- `weight` — The weight. [UInt8](../../data_types/int_uint.md).
+- `weight` — The weight. [UInt8](../../sql_reference/data_types/int_uint.md).
**Returned value**
@@ -1700,7 +1705,7 @@ stochasticLogisticRegression(1.0, 1.0, 10, 'SGD')
## groupBitmapAnd {#groupbitmapand}
-Calculations the AND of a bitmap column, return cardinality of type UInt64, if add suffix -State, then return [bitmap object](../functions/bitmap_functions.md).
+Calculations the AND of a bitmap column, return cardinality of type UInt64, if add suffix -State, then return [bitmap object](../../sql_reference/functions/bitmap_functions.md).
``` sql
groupBitmapAnd(expr)
@@ -1743,7 +1748,7 @@ SELECT arraySort(bitmapToArray(groupBitmapAndState(z))) FROM bitmap_column_expr_
## groupBitmapOr {#groupbitmapor}
-Calculations the OR of a bitmap column, return cardinality of type UInt64, if add suffix -State, then return [bitmap object](../functions/bitmap_functions.md). This is equivalent to `groupBitmapMerge`.
+Calculations the OR of a bitmap column, return cardinality of type UInt64, if add suffix -State, then return [bitmap object](../../sql_reference/functions/bitmap_functions.md). This is equivalent to `groupBitmapMerge`.
``` sql
groupBitmapOr(expr)
@@ -1786,7 +1791,7 @@ SELECT arraySort(bitmapToArray(groupBitmapOrState(z))) FROM bitmap_column_expr_t
## groupBitmapXor {#groupbitmapxor}
-Calculations the XOR of a bitmap column, return cardinality of type UInt64, if add suffix -State, then return [bitmap object](../functions/bitmap_functions.md).
+Calculations the XOR of a bitmap column, return cardinality of type UInt64, if add suffix -State, then return [bitmap object](../../sql_reference/functions/bitmap_functions.md).
``` sql
groupBitmapOr(expr)
diff --git a/docs/en/data_types/nested_data_structures/aggregatefunction.md b/docs/en/sql_reference/data_types/aggregatefunction.md
similarity index 70%
rename from docs/en/data_types/nested_data_structures/aggregatefunction.md
rename to docs/en/sql_reference/data_types/aggregatefunction.md
index dafa4a1c2b5..57935557ba9 100644
--- a/docs/en/data_types/nested_data_structures/aggregatefunction.md
+++ b/docs/en/sql_reference/data_types/aggregatefunction.md
@@ -1,6 +1,11 @@
+---
+toc_priority: 52
+toc_title: AggregateFunction(name, types_of_arguments...)
+---
+
# AggregateFunction(name, types\_of\_arguments…) {#data-type-aggregatefunction}
-Aggregate functions can have an implementation-defined intermediate state that can be serialized to an AggregateFunction(...) data type and stored in a table, usually, by means of [a materialized view](../../query_language/select.md#create-view). The common way to produce an aggregate function state is by calling the aggregate function with the `-State` suffix. To get the final result of aggregation in the future, you must use the same aggregate function with the `-Merge`suffix.
+Aggregate functions can have an implementation-defined intermediate state that can be serialized to an AggregateFunction(...) data type and stored in a table, usually, by means of [a materialized view](../../sql_reference/statements/select.md#create-view). The common way to produce an aggregate function state is by calling the aggregate function with the `-State` suffix. To get the final result of aggregation in the future, you must use the same aggregate function with the `-Merge`suffix.
`AggregateFunction` — parametric data type.
@@ -23,7 +28,7 @@ CREATE TABLE t
) ENGINE = ...
```
-[uniq](../../query_language/agg_functions/reference.md#agg_function-uniq), anyIf ([any](../../query_language/agg_functions/reference.md#agg_function-any)+[If](../../query_language/agg_functions/combinators.md#agg-functions-combinator-if)) and [quantiles](../../query_language/agg_functions/reference.md) are the aggregate functions supported in ClickHouse.
+[uniq](../../sql_reference/aggregate_functions/reference.md#agg_function-uniq), anyIf ([any](../../sql_reference/aggregate_functions/reference.md#agg_function-any)+[If](../../sql_reference/aggregate_functions/combinators.md#agg-functions-combinator-if)) and [quantiles](../../sql_reference/aggregate_functions/reference.md) are the aggregate functions supported in ClickHouse.
## Usage {#usage}
@@ -58,6 +63,6 @@ SELECT uniqMerge(state) FROM (SELECT uniqState(UserID) AS state FROM table GROUP
## Usage Example {#usage-example}
-See [AggregatingMergeTree](../../operations/table_engines/aggregatingmergetree.md) engine description.
+See [AggregatingMergeTree](../../engines/table_engines/mergetree_family/aggregatingmergetree.md) engine description.
[Original article](https://clickhouse.tech/docs/en/data_types/nested_data_structures/aggregatefunction/)
diff --git a/docs/en/data_types/array.md b/docs/en/sql_reference/data_types/array.md
similarity index 85%
rename from docs/en/data_types/array.md
rename to docs/en/sql_reference/data_types/array.md
index a635480a48f..58b1d25df04 100644
--- a/docs/en/data_types/array.md
+++ b/docs/en/sql_reference/data_types/array.md
@@ -1,8 +1,13 @@
-# Array(T) {#data-type-array}
+---
+toc_priority: 51
+toc_title: Array(T)
+---
+
+# Array(t) {#data-type-array}
An array of `T`-type items. `T` can be any data type, including an array.
-## Creating an array {#creating-an-array}
+## Creating an Array {#creating-an-array}
You can use a function to create an array:
@@ -38,9 +43,9 @@ SELECT [1, 2] AS x, toTypeName(x)
└───────┴────────────────────┘
```
-## Working with Data Types {#working-with-data-types}
+## Working With Data Types {#working-with-data-types}
-When creating an array on the fly, ClickHouse automatically defines the argument type as the narrowest data type that can store all the listed arguments. If there are any [Nullable](nullable.md#data_type-nullable) or literal [NULL](../query_language/syntax.md#null-literal) values, the type of an array element also becomes [Nullable](nullable.md).
+When creating an array on the fly, ClickHouse automatically defines the argument type as the narrowest data type that can store all the listed arguments. If there are any [Nullable](nullable.md#data_type-nullable) or literal [NULL](../../sql_reference/syntax.md#null-literal) values, the type of an array element also becomes [Nullable](nullable.md).
If ClickHouse couldn’t determine the data type, it generates an exception. For instance, this happens when trying to create an array with strings and numbers simultaneously (`SELECT array(1, 'a')`).
diff --git a/docs/en/data_types/boolean.md b/docs/en/sql_reference/data_types/boolean.md
similarity index 82%
rename from docs/en/data_types/boolean.md
rename to docs/en/sql_reference/data_types/boolean.md
index 1d3d0d1a6a4..5890725da8f 100644
--- a/docs/en/data_types/boolean.md
+++ b/docs/en/sql_reference/data_types/boolean.md
@@ -1,3 +1,8 @@
+---
+toc_priority: 43
+toc_title: Boolean
+---
+
# Boolean Values {#boolean-values}
There is no separate type for boolean values. Use UInt8 type, restricted to the values 0 or 1.
diff --git a/docs/en/data_types/date.md b/docs/en/sql_reference/data_types/date.md
similarity index 92%
rename from docs/en/data_types/date.md
rename to docs/en/sql_reference/data_types/date.md
index 359ef17f724..fd15441aabb 100644
--- a/docs/en/data_types/date.md
+++ b/docs/en/sql_reference/data_types/date.md
@@ -1,3 +1,8 @@
+---
+toc_priority: 47
+toc_title: Date
+---
+
# Date {#date}
A date. Stored in two bytes as the number of days since 1970-01-01 (unsigned). Allows storing values from just after the beginning of the Unix Epoch to the upper threshold defined by a constant at the compilation stage (currently, this is until the year 2106, but the final fully-supported year is 2105).
diff --git a/docs/en/data_types/datetime.md b/docs/en/sql_reference/data_types/datetime.md
similarity index 76%
rename from docs/en/data_types/datetime.md
rename to docs/en/sql_reference/data_types/datetime.md
index f94d1e47bb9..ee108dfe078 100644
--- a/docs/en/data_types/datetime.md
+++ b/docs/en/sql_reference/data_types/datetime.md
@@ -1,4 +1,9 @@
-# DateTime {#data_type-datetime}
+---
+toc_priority: 48
+toc_title: DateTime
+---
+
+# Datetime {#data_type-datetime}
Allows to store an instant in time, that can be expressed as a calendar date and a time of a day.
@@ -18,13 +23,13 @@ The point in time is saved as a [Unix timestamp](https://en.wikipedia.org/wiki/U
A list of supported time zones can be found in the [IANA Time Zone Database](https://www.iana.org/time-zones).
The `tzdata` package, containing [IANA Time Zone Database](https://www.iana.org/time-zones), should be installed in the system. Use the `timedatectl list-timezones` command to list timezones known by a local system.
-You can explicitly set a time zone for `DateTime`-type columns when creating a table. If the time zone isn’t set, ClickHouse uses the value of the [timezone](../operations/server_settings/settings.md#server_settings-timezone) parameter in the server settings or the operating system settings at the moment of the ClickHouse server start.
+You can explicitly set a time zone for `DateTime`-type columns when creating a table. If the time zone isn’t set, ClickHouse uses the value of the [timezone](../../operations/server_configuration_parameters/settings.md#server_configuration_parameters-timezone) parameter in the server settings or the operating system settings at the moment of the ClickHouse server start.
-The [clickhouse-client](../interfaces/cli.md) applies the server time zone by default if a time zone isn’t explicitly set when initializing the data type. To use the client time zone, run `clickhouse-client` with the `--use_client_time_zone` parameter.
+The [clickhouse-client](../../interfaces/cli.md) applies the server time zone by default if a time zone isn’t explicitly set when initializing the data type. To use the client time zone, run `clickhouse-client` with the `--use_client_time_zone` parameter.
-ClickHouse outputs values in `YYYY-MM-DD hh:mm:ss` text format by default. You can change the output with the [formatDateTime](../query_language/functions/date_time_functions.md#formatdatetime) function.
+ClickHouse outputs values in `YYYY-MM-DD hh:mm:ss` text format by default. You can change the output with the [formatDateTime](../../sql_reference/functions/date_time_functions.md#formatdatetime) function.
-When inserting data into ClickHouse, you can use different formats of date and time strings, depending on the value of the [date\_time\_input\_format](../operations/settings/settings.md#settings-date_time_input_format) setting.
+When inserting data into ClickHouse, you can use different formats of date and time strings, depending on the value of the [date\_time\_input\_format](../../operations/settings/settings.md#settings-date_time_input_format) setting.
## Examples {#examples}
@@ -111,12 +116,12 @@ FROM dt
## See Also {#see-also}
-- [Type conversion functions](../query_language/functions/type_conversion_functions.md)
-- [Functions for working with dates and times](../query_language/functions/date_time_functions.md)
-- [Functions for working with arrays](../query_language/functions/array_functions.md)
-- [The `date_time_input_format` setting](../operations/settings/settings.md#settings-date_time_input_format)
-- [The `timezone` server configuration parameter](../operations/server_settings/settings.md#server_settings-timezone)
-- [Operators for working with dates and times](../query_language/operators.md#operators-datetime)
+- [Type conversion functions](../../sql_reference/functions/type_conversion_functions.md)
+- [Functions for working with dates and times](../../sql_reference/functions/date_time_functions.md)
+- [Functions for working with arrays](../../sql_reference/functions/array_functions.md)
+- [The `date_time_input_format` setting](../../operations/settings/settings.md#settings-date_time_input_format)
+- [The `timezone` server configuration parameter](../../operations/server_configuration_parameters/settings.md#server_configuration_parameters-timezone)
+- [Operators for working with dates and times](../../sql_reference/operators.md#operators-datetime)
- [The `Date` data type](date.md)
[Original article](https://clickhouse.tech/docs/en/data_types/datetime/)
diff --git a/docs/en/data_types/datetime64.md b/docs/en/sql_reference/data_types/datetime64.md
similarity index 83%
rename from docs/en/data_types/datetime64.md
rename to docs/en/sql_reference/data_types/datetime64.md
index a031a645b6d..f946c7aff6b 100644
--- a/docs/en/data_types/datetime64.md
+++ b/docs/en/sql_reference/data_types/datetime64.md
@@ -1,4 +1,9 @@
-# DateTime64 {#data_type-datetime64}
+---
+toc_priority: 49
+toc_title: DateTime64
+---
+
+# Datetime64 {#data_type-datetime64}
Allows to store an instant in time, that can be expressed as a calendar date and a time of a day, with defined sub-second precision
@@ -87,11 +92,11 @@ FROM dt
## See Also {#see-also}
-- [Type conversion functions](../query_language/functions/type_conversion_functions.md)
-- [Functions for working with dates and times](../query_language/functions/date_time_functions.md)
-- [Functions for working with arrays](../query_language/functions/array_functions.md)
-- [The `date_time_input_format` setting](../operations/settings/settings.md#settings-date_time_input_format)
-- [The `timezone` server configuration parameter](../operations/server_settings/settings.md#server_settings-timezone)
-- [Operators for working with dates and times](../query_language/operators.md#operators-datetime)
+- [Type conversion functions](../../sql_reference/functions/type_conversion_functions.md)
+- [Functions for working with dates and times](../../sql_reference/functions/date_time_functions.md)
+- [Functions for working with arrays](../../sql_reference/functions/array_functions.md)
+- [The `date_time_input_format` setting](../../operations/settings/settings.md#settings-date_time_input_format)
+- [The `timezone` server configuration parameter](../../operations/server_configuration_parameters/settings.md#server_configuration_parameters-timezone)
+- [Operators for working with dates and times](../../sql_reference/operators.md#operators-datetime)
- [`Date` data type](date.md)
- [`DateTime` data type](datetime.md)
diff --git a/docs/en/data_types/decimal.md b/docs/en/sql_reference/data_types/decimal.md
similarity index 98%
rename from docs/en/data_types/decimal.md
rename to docs/en/sql_reference/data_types/decimal.md
index fb52fc433d9..c689cf60310 100644
--- a/docs/en/data_types/decimal.md
+++ b/docs/en/sql_reference/data_types/decimal.md
@@ -1,3 +1,8 @@
+---
+toc_priority: 42
+toc_title: Decimal
+---
+
# Decimal(P, S), Decimal32(S), Decimal64(S), Decimal128(S) {#decimalp-s-decimal32s-decimal64s-decimal128s}
Signed fixed-point numbers that keep precision during add, subtract and multiply operations. For division least significant digits are discarded (not rounded).
diff --git a/docs/en/sql_reference/data_types/domains/index.md b/docs/en/sql_reference/data_types/domains/index.md
new file mode 100644
index 00000000000..d4496cf8d5b
--- /dev/null
+++ b/docs/en/sql_reference/data_types/domains/index.md
@@ -0,0 +1,5 @@
+---
+toc_folder_title: Domains
+toc_priority: 56
+---
+
diff --git a/docs/en/data_types/domains/ipv4.md b/docs/en/sql_reference/data_types/domains/ipv4.md
similarity index 98%
rename from docs/en/data_types/domains/ipv4.md
rename to docs/en/sql_reference/data_types/domains/ipv4.md
index 33f66a24427..d8735d70b29 100644
--- a/docs/en/data_types/domains/ipv4.md
+++ b/docs/en/sql_reference/data_types/domains/ipv4.md
@@ -1,3 +1,8 @@
+---
+toc_priority: 59
+toc_title: IPv4
+---
+
## IPv4 {#ipv4}
`IPv4` is a domain based on `UInt32` type and serves as a typed replacement for storing IPv4 values. It provides compact storage with the human-friendly input-output format and column type information on inspection.
diff --git a/docs/en/data_types/domains/ipv6.md b/docs/en/sql_reference/data_types/domains/ipv6.md
similarity index 98%
rename from docs/en/data_types/domains/ipv6.md
rename to docs/en/sql_reference/data_types/domains/ipv6.md
index ec97a0c1ac6..7fd88887acc 100644
--- a/docs/en/data_types/domains/ipv6.md
+++ b/docs/en/sql_reference/data_types/domains/ipv6.md
@@ -1,3 +1,8 @@
+---
+toc_priority: 60
+toc_title: IPv6
+---
+
## IPv6 {#ipv6}
`IPv6` is a domain based on `FixedString(16)` type and serves as a typed replacement for storing IPv6 values. It provides compact storage with the human-friendly input-output format and column type information on inspection.
diff --git a/docs/en/data_types/domains/overview.md b/docs/en/sql_reference/data_types/domains/overview.md
similarity index 96%
rename from docs/en/data_types/domains/overview.md
rename to docs/en/sql_reference/data_types/domains/overview.md
index 733c6ebecec..aea7307d048 100644
--- a/docs/en/data_types/domains/overview.md
+++ b/docs/en/sql_reference/data_types/domains/overview.md
@@ -1,3 +1,8 @@
+---
+toc_priority: 58
+toc_title: Overview
+---
+
# Domains {#domains}
Domains are special-purpose types that add some extra features atop of existing base type, but leaving on-wire and on-disc format of the underlying data type intact. At the moment, ClickHouse does not support user-defined domains.
diff --git a/docs/en/data_types/enum.md b/docs/en/sql_reference/data_types/enum.md
similarity index 96%
rename from docs/en/data_types/enum.md
rename to docs/en/sql_reference/data_types/enum.md
index 200b8990937..e31bbed3426 100644
--- a/docs/en/data_types/enum.md
+++ b/docs/en/sql_reference/data_types/enum.md
@@ -1,3 +1,8 @@
+---
+toc_priority: 50
+toc_title: Enum
+---
+
# Enum {#enum}
Enumerated type consisting of named values.
@@ -11,7 +16,7 @@ ClickHouse supports:
ClickHouse automatically chooses the type of `Enum` when data is inserted. You can also use `Enum8` or `Enum16` types to be sure in the size of storage.
-## Usage examples {#usage-examples}
+## Usage Examples {#usage-examples}
Here we create a table with an `Enum8('hello' = 1, 'world' = 2)` type column:
@@ -82,11 +87,11 @@ SELECT toTypeName(CAST('a', 'Enum(\'a\' = 1, \'b\' = 2)'))
└─────────────────────────────────────────────────────┘
```
-## General rules and usage {#general-rules-and-usage}
+## General Rules and Usage {#general-rules-and-usage}
Each of the values is assigned a number in the range `-128 ... 127` for `Enum8` or in the range `-32768 ... 32767` for `Enum16`. All the strings and numbers must be different. An empty string is allowed. If this type is specified (in a table definition), numbers can be in an arbitrary order. However, the order does not matter.
-Neither the string nor the numeric value in an `Enum` can be [NULL](../query_language/syntax.md).
+Neither the string nor the numeric value in an `Enum` can be [NULL](../../sql_reference/syntax.md).
An `Enum` can be contained in [Nullable](nullable.md) type. So if you create a table using the query
diff --git a/docs/en/data_types/fixedstring.md b/docs/en/sql_reference/data_types/fixedstring.md
similarity index 84%
rename from docs/en/data_types/fixedstring.md
rename to docs/en/sql_reference/data_types/fixedstring.md
index 602741eac43..4c8ea65b278 100644
--- a/docs/en/data_types/fixedstring.md
+++ b/docs/en/sql_reference/data_types/fixedstring.md
@@ -1,4 +1,9 @@
-# FixedString {#fixedstring}
+---
+toc_priority: 45
+toc_title: FixedString(N)
+---
+
+# Fixedstring {#fixedstring}
A fixed-length string of `N` bytes (neither characters nor code points).
@@ -51,6 +56,6 @@ WHERE a = 'b\0'
This behaviour differs from MySQL for the `CHAR` type (where strings are padded with spaces, and the spaces are removed for output).
-Note that the length of the `FixedString(N)` value is constant. The [length](../query_language/functions/array_functions.md#array_functions-length) function returns `N` even if the `FixedString(N)` value is filled only with null bytes, but the [empty](../query_language/functions/string_functions.md#empty) function returns `1` in this case.
+Note that the length of the `FixedString(N)` value is constant. The [length](../../sql_reference/functions/array_functions.md#array_functions-length) function returns `N` even if the `FixedString(N)` value is filled only with null bytes, but the [empty](../../sql_reference/functions/string_functions.md#empty) function returns `1` in this case.
[Original article](https://clickhouse.tech/docs/en/data_types/fixedstring/)
diff --git a/docs/en/data_types/float.md b/docs/en/sql_reference/data_types/float.md
similarity index 95%
rename from docs/en/data_types/float.md
rename to docs/en/sql_reference/data_types/float.md
index b02101ec2aa..646161d34fd 100644
--- a/docs/en/data_types/float.md
+++ b/docs/en/sql_reference/data_types/float.md
@@ -1,3 +1,8 @@
+---
+toc_priority: 41
+toc_title: Float32, Float64
+---
+
# Float32, Float64 {#float32-float64}
[Floating point numbers](https://en.wikipedia.org/wiki/IEEE_754).
@@ -75,6 +80,6 @@ SELECT 0 / 0
└──────────────┘
```
- See the rules for `NaN` sorting in the section [ORDER BY clause](../query_language/select.md).
+ See the rules for `NaN` sorting in the section [ORDER BY clause](../sql_reference/statements/select.md).
[Original article](https://clickhouse.tech/docs/en/data_types/float/)
diff --git a/docs/en/data_types/index.md b/docs/en/sql_reference/data_types/index.md
similarity index 78%
rename from docs/en/data_types/index.md
rename to docs/en/sql_reference/data_types/index.md
index 43884b3e3e1..da507b81acd 100644
--- a/docs/en/data_types/index.md
+++ b/docs/en/sql_reference/data_types/index.md
@@ -1,3 +1,9 @@
+---
+toc_folder_title: Data Types
+toc_priority: 37
+toc_title: Introduction
+---
+
# Data Types {#data_types}
ClickHouse can store various kinds of data in table cells.
diff --git a/docs/en/data_types/int_uint.md b/docs/en/sql_reference/data_types/int_uint.md
similarity index 86%
rename from docs/en/data_types/int_uint.md
rename to docs/en/sql_reference/data_types/int_uint.md
index 2864e79394b..84ef120af7d 100644
--- a/docs/en/data_types/int_uint.md
+++ b/docs/en/sql_reference/data_types/int_uint.md
@@ -1,3 +1,8 @@
+---
+toc_priority: 40
+toc_title: UInt8, UInt16, UInt32, UInt64, Int8, Int16, Int32, Int64
+---
+
# UInt8, UInt16, UInt32, UInt64, Int8, Int16, Int32, Int64 {#uint8-uint16-uint32-uint64-int8-int16-int32-int64}
Fixed-length integers, with or without a sign.
diff --git a/docs/en/data_types/nested_data_structures/index.md b/docs/en/sql_reference/data_types/nested_data_structures/index.md
similarity index 59%
rename from docs/en/data_types/nested_data_structures/index.md
rename to docs/en/sql_reference/data_types/nested_data_structures/index.md
index f6010f804e9..14562254e57 100644
--- a/docs/en/data_types/nested_data_structures/index.md
+++ b/docs/en/sql_reference/data_types/nested_data_structures/index.md
@@ -1,3 +1,10 @@
+---
+toc_folder_title: Nested Data Structures
+toc_hidden: true
+toc_priority: 54
+toc_title: hidden
+---
+
# Nested Data Structures {#nested-data-structures}
[Original article](https://clickhouse.tech/docs/en/data_types/nested_data_structures/)
diff --git a/docs/en/data_types/nested_data_structures/nested.md b/docs/en/sql_reference/data_types/nested_data_structures/nested.md
similarity index 94%
rename from docs/en/data_types/nested_data_structures/nested.md
rename to docs/en/sql_reference/data_types/nested_data_structures/nested.md
index 67bd7b071e2..4c95f1fd2fc 100644
--- a/docs/en/data_types/nested_data_structures/nested.md
+++ b/docs/en/sql_reference/data_types/nested_data_structures/nested.md
@@ -1,6 +1,11 @@
-# Nested(Name1 Type1, Name2 Type2, …) {#nestedname1-type1-name2-type2}
+---
+toc_priority: 57
+toc_title: Nested(Name1 Type1, Name2 Type2, ...)
+---
-A nested data structure is like a table inside a cell. The parameters of a nested data structure – the column names and types – are specified the same way as in a [CREATE TABLE](../../query_language/create.md) query. Each table row can correspond to any number of rows in a nested data structure.
+# Nested(name1 Type1, Name2 Type2, …) {#nestedname1-type1-name2-type2}
+
+A nested data structure is like a table inside a cell. The parameters of a nested data structure – the column names and types – are specified the same way as in a [CREATE TABLE](../../../sql_reference/statements/create.md) query. Each table row can correspond to any number of rows in a nested data structure.
Example:
diff --git a/docs/en/data_types/nullable.md b/docs/en/sql_reference/data_types/nullable.md
similarity index 74%
rename from docs/en/data_types/nullable.md
rename to docs/en/sql_reference/data_types/nullable.md
index ad6d3761803..7fd68592c4a 100644
--- a/docs/en/data_types/nullable.md
+++ b/docs/en/sql_reference/data_types/nullable.md
@@ -1,6 +1,11 @@
-# Nullable(TypeName) {#data_type-nullable}
+---
+toc_priority: 54
+toc_title: Nullable
+---
-Allows to store special marker ([NULL](../query_language/syntax.md)) that denotes “missing value” alongside normal values allowed by `TypeName`. For example, a `Nullable(Int8)` type column can store `Int8` type values, and the rows that don’t have a value will store `NULL`.
+# Nullable(typename) {#data_type-nullable}
+
+Allows to store special marker ([NULL](../../sql_reference/syntax.md)) that denotes “missing value” alongside normal values allowed by `TypeName`. For example, a `Nullable(Int8)` type column can store `Int8` type values, and the rows that don’t have a value will store `NULL`.
For a `TypeName`, you can’t use composite data types [Array](array.md) and [Tuple](tuple.md). Composite data types can contain `Nullable` type values, such as `Array(Nullable(Int8))`.
@@ -8,14 +13,14 @@ A `Nullable` type field can’t be included in table indexes.
`NULL` is the default value for any `Nullable` type, unless specified otherwise in the ClickHouse server configuration.
-## Storage features {#storage-features}
+## Storage Features {#storage-features}
To store `Nullable` type values in a table column, ClickHouse uses a separate file with `NULL` masks in addition to normal file with values. Entries in masks file allow ClickHouse to distinguish between `NULL` and a default value of corresponding data type for each table row. Because of an additional file, `Nullable` column consumes additional storage space compared to a similar normal one.
!!! info "Note"
Using `Nullable` almost always negatively affects performance, keep this in mind when designing your databases.
-## Usage example {#usage-example}
+## Usage Example {#usage-example}
``` sql
CREATE TABLE t_null(x Int8, y Nullable(Int8)) ENGINE TinyLog
diff --git a/docs/en/data_types/special_data_types/expression.md b/docs/en/sql_reference/data_types/special_data_types/expression.md
similarity index 81%
rename from docs/en/data_types/special_data_types/expression.md
rename to docs/en/sql_reference/data_types/special_data_types/expression.md
index 91630ae1c38..c707a01cf26 100644
--- a/docs/en/data_types/special_data_types/expression.md
+++ b/docs/en/sql_reference/data_types/special_data_types/expression.md
@@ -1,3 +1,8 @@
+---
+toc_priority: 58
+toc_title: Expression
+---
+
# Expression {#expression}
Expressions are used for representing lambdas in high-order functions.
diff --git a/docs/en/data_types/special_data_types/index.md b/docs/en/sql_reference/data_types/special_data_types/index.md
similarity index 75%
rename from docs/en/data_types/special_data_types/index.md
rename to docs/en/sql_reference/data_types/special_data_types/index.md
index 5b072e143ef..83919eee56b 100644
--- a/docs/en/data_types/special_data_types/index.md
+++ b/docs/en/sql_reference/data_types/special_data_types/index.md
@@ -1,3 +1,10 @@
+---
+toc_folder_title: Special Data Types
+toc_hidden: true
+toc_priority: 55
+toc_title: hidden
+---
+
# Special Data Types {#special-data-types}
Special data type values can’t be serialized for saving in a table or output in query results, but can be used as an intermediate result during query execution.
diff --git a/docs/en/data_types/special_data_types/interval.md b/docs/en/sql_reference/data_types/special_data_types/interval.md
similarity index 83%
rename from docs/en/data_types/special_data_types/interval.md
rename to docs/en/sql_reference/data_types/special_data_types/interval.md
index 8003e8b8522..8b71e1301a6 100644
--- a/docs/en/data_types/special_data_types/interval.md
+++ b/docs/en/sql_reference/data_types/special_data_types/interval.md
@@ -1,6 +1,11 @@
+---
+toc_priority: 61
+toc_title: Interval
+---
+
# Interval {#data-type-interval}
-The family of data types representing time and date intervals. The resulting types of the [INTERVAL](../../query_language/operators.md#operator-interval) operator.
+The family of data types representing time and date intervals. The resulting types of the [INTERVAL](../../../sql_reference/operators.md#operator-interval) operator.
!!! warning "Warning"
`Interval` data type values can’t be stored in tables.
@@ -35,7 +40,7 @@ SELECT toTypeName(INTERVAL 4 DAY)
## Usage Remarks {#data-type-interval-usage-remarks}
-You can use `Interval`-type values in arithmetical operations with [Date](../../data_types/date.md) and [DateTime](../../data_types/datetime.md)-type values. For example, you can add 4 days to the current time:
+You can use `Interval`-type values in arithmetical operations with [Date](../../../sql_reference/data_types/date.md) and [DateTime](../../../sql_reference/data_types/datetime.md)-type values. For example, you can add 4 days to the current time:
``` sql
SELECT now() as current_date_time, current_date_time + INTERVAL 4 DAY
@@ -74,5 +79,5 @@ Code: 43. DB::Exception: Received from localhost:9000. DB::Exception: Wrong argu
## See Also {#see-also}
-- [INTERVAL](../../query_language/operators.md#operator-interval) operator
-- [toInterval](../../query_language/functions/type_conversion_functions.md#function-tointerval) type convertion functions
+- [INTERVAL](../../../sql_reference/operators.md#operator-interval) operator
+- [toInterval](../../../sql_reference/functions/type_conversion_functions.md#function-tointerval) type convertion functions
diff --git a/docs/en/data_types/special_data_types/nothing.md b/docs/en/sql_reference/data_types/special_data_types/nothing.md
similarity index 68%
rename from docs/en/data_types/special_data_types/nothing.md
rename to docs/en/sql_reference/data_types/special_data_types/nothing.md
index 1067fddf4b8..054f7e2c5a8 100644
--- a/docs/en/data_types/special_data_types/nothing.md
+++ b/docs/en/sql_reference/data_types/special_data_types/nothing.md
@@ -1,8 +1,13 @@
+---
+toc_priority: 60
+toc_title: Nothing
+---
+
# Nothing {#nothing}
The only purpose of this data type is to represent cases where a value is not expected. So you can’t create a `Nothing` type value.
-For example, literal [NULL](../../query_language/syntax.md#null-literal) has type of `Nullable(Nothing)`. See more about [Nullable](../../data_types/nullable.md).
+For example, literal [NULL](../../../sql_reference/syntax.md#null-literal) has type of `Nullable(Nothing)`. See more about [Nullable](../../../sql_reference/data_types/nullable.md).
The `Nothing` type can also used to denote empty arrays:
diff --git a/docs/en/sql_reference/data_types/special_data_types/set.md b/docs/en/sql_reference/data_types/special_data_types/set.md
new file mode 100644
index 00000000000..61f3c302ce4
--- /dev/null
+++ b/docs/en/sql_reference/data_types/special_data_types/set.md
@@ -0,0 +1,10 @@
+---
+toc_priority: 59
+toc_title: Set
+---
+
+# Set {#set}
+
+Used for the right half of an [IN](../../../sql_reference/statements/select.md#select-in-operators) expression.
+
+[Original article](https://clickhouse.tech/docs/en/data_types/special_data_types/set/)
diff --git a/docs/en/data_types/string.md b/docs/en/sql_reference/data_types/string.md
similarity index 96%
rename from docs/en/data_types/string.md
rename to docs/en/sql_reference/data_types/string.md
index de3e74d96da..a990bd0742b 100644
--- a/docs/en/data_types/string.md
+++ b/docs/en/sql_reference/data_types/string.md
@@ -1,3 +1,8 @@
+---
+toc_priority: 44
+toc_title: String
+---
+
# String {#string}
Strings of an arbitrary length. The length is not limited. The value can contain an arbitrary set of bytes, including null bytes.
diff --git a/docs/en/data_types/tuple.md b/docs/en/sql_reference/data_types/tuple.md
similarity index 74%
rename from docs/en/data_types/tuple.md
rename to docs/en/sql_reference/data_types/tuple.md
index 2f9d449b207..4e73f85df63 100644
--- a/docs/en/data_types/tuple.md
+++ b/docs/en/sql_reference/data_types/tuple.md
@@ -1,12 +1,17 @@
-# Tuple(T1, T2, …) {#tuplet1-t2}
+---
+toc_priority: 53
+toc_title: Tuple(T1, T2, ...)
+---
+
+# Tuple(t1, T2, …) {#tuplet1-t2}
A tuple of elements, each having an individual [type](index.md#data_types).
-Tuples are used for temporary column grouping. Columns can be grouped when an IN expression is used in a query, and for specifying certain formal parameters of lambda functions. For more information, see the sections [IN operators](../query_language/select.md) and [Higher order functions](../query_language/functions/higher_order_functions.md).
+Tuples are used for temporary column grouping. Columns can be grouped when an IN expression is used in a query, and for specifying certain formal parameters of lambda functions. For more information, see the sections [IN operators](../../sql_reference/statements/select.md) and [Higher order functions](../../sql_reference/functions/higher_order_functions.md).
Tuples can be the result of a query. In this case, for text formats other than JSON, values are comma-separated in brackets. In JSON formats, tuples are output as arrays (in square brackets).
-## Creating a tuple {#creating-a-tuple}
+## Creating a Tuple {#creating-a-tuple}
You can use a function to create a tuple:
@@ -26,9 +31,9 @@ SELECT tuple(1,'a') AS x, toTypeName(x)
└─────────┴───────────────────────────┘
```
-## Working with data types {#working-with-data-types}
+## Working With Data Types {#working-with-data-types}
-When creating a tuple on the fly, ClickHouse automatically detects the type of each argument as the minimum of the types which can store the argument value. If the argument is [NULL](../query_language/syntax.md#null-literal), the type of the tuple element is [Nullable](nullable.md).
+When creating a tuple on the fly, ClickHouse automatically detects the type of each argument as the minimum of the types which can store the argument value. If the argument is [NULL](../../sql_reference/syntax.md#null-literal), the type of the tuple element is [Nullable](nullable.md).
Example of automatic data type detection:
diff --git a/docs/en/data_types/uuid.md b/docs/en/sql_reference/data_types/uuid.md
similarity index 74%
rename from docs/en/data_types/uuid.md
rename to docs/en/sql_reference/data_types/uuid.md
index a04ae21755a..140f86ac464 100644
--- a/docs/en/data_types/uuid.md
+++ b/docs/en/sql_reference/data_types/uuid.md
@@ -1,3 +1,8 @@
+---
+toc_priority: 46
+toc_title: UUID
+---
+
# UUID {#uuid-data-type}
A universally unique identifier (UUID) is a 16-byte number used to identify records. For detailed information about the UUID, see [Wikipedia](https://en.wikipedia.org/wiki/Universally_unique_identifier).
@@ -14,11 +19,11 @@ If you do not specify the UUID column value when inserting a new record, the UUI
00000000-0000-0000-0000-000000000000
```
-## How to generate {#how-to-generate}
+## How To Generate {#how-to-generate}
-To generate the UUID value, ClickHouse provides the [generateUUIDv4](../query_language/functions/uuid_functions.md) function.
+To generate the UUID value, ClickHouse provides the [generateUUIDv4](../../sql_reference/functions/uuid_functions.md) function.
-## Usage example {#usage-example}
+## Usage Example {#usage-example}
**Example 1**
@@ -63,8 +68,8 @@ SELECT * FROM t_uuid
## Restrictions {#restrictions}
-The UUID data type only supports functions which [String](string.md) data type also supports (for example, [min](../query_language/agg_functions/reference.md#agg_function-min), [max](../query_language/agg_functions/reference.md#agg_function-max), and [count](../query_language/agg_functions/reference.md#agg_function-count)).
+The UUID data type only supports functions which [String](string.md) data type also supports (for example, [min](../../sql_reference/aggregate_functions/reference.md#agg_function-min), [max](../../sql_reference/aggregate_functions/reference.md#agg_function-max), and [count](../../sql_reference/aggregate_functions/reference.md#agg_function-count)).
-The UUID data type is not supported by arithmetic operations (for example, [abs](../query_language/functions/arithmetic_functions.md#arithm_func-abs)) or aggregate functions, such as [sum](../query_language/agg_functions/reference.md#agg_function-sum) and [avg](../query_language/agg_functions/reference.md#agg_function-avg).
+The UUID data type is not supported by arithmetic operations (for example, [abs](../../sql_reference/functions/arithmetic_functions.md#arithm_func-abs)) or aggregate functions, such as [sum](../../sql_reference/aggregate_functions/reference.md#agg_function-sum) and [avg](../../sql_reference/aggregate_functions/reference.md#agg_function-avg).
[Original article](https://clickhouse.tech/docs/en/data_types/uuid/)
diff --git a/docs/en/query_language/dicts/external_dicts.md b/docs/en/sql_reference/dictionaries/external_dictionaries/external_dicts.md
similarity index 65%
rename from docs/en/query_language/dicts/external_dicts.md
rename to docs/en/sql_reference/dictionaries/external_dictionaries/external_dicts.md
index 6e9f2541447..446946e927e 100644
--- a/docs/en/query_language/dicts/external_dicts.md
+++ b/docs/en/sql_reference/dictionaries/external_dictionaries/external_dicts.md
@@ -1,3 +1,8 @@
+---
+toc_priority: 39
+toc_title: General Description
+---
+
# External Dictionaries {#dicts-external-dicts}
You can add your own dictionaries from various data sources. The data source for a dictionary can be a local text or executable file, an HTTP(s) resource, or another DBMS. For more information, see “[Sources for external dictionaries](external_dicts_dict_sources.md)”.
@@ -6,11 +11,11 @@ ClickHouse:
- Fully or partially stores dictionaries in RAM.
- Periodically updates dictionaries and dynamically loads missing values. In other words, dictionaries can be loaded dynamically.
-- Allows to create external dictionaries with xml files or [DDL queries](../create.md#create-dictionary-query).
+- Allows to create external dictionaries with xml files or [DDL queries](../../statements/create.md#create-dictionary-query).
-The configuration of external dictionaries can be located in one or more xml-files. The path to the configuration is specified in the [dictionaries\_config](../../operations/server_settings/settings.md#server_settings-dictionaries_config) parameter.
+The configuration of external dictionaries can be located in one or more xml-files. The path to the configuration is specified in the [dictionaries\_config](../../../operations/server_configuration_parameters/settings.md#server_configuration_parameters-dictionaries_config) parameter.
-Dictionaries can be loaded at server startup or at first use, depending on the [dictionaries\_lazy\_load](../../operations/server_settings/settings.md#server_settings-dictionaries_lazy_load) setting.
+Dictionaries can be loaded at server startup or at first use, depending on the [dictionaries\_lazy\_load](../../../operations/server_configuration_parameters/settings.md#server_configuration_parameters-dictionaries_lazy_load) setting.
The dictionary configuration file has the following format:
@@ -32,18 +37,18 @@ The dictionary configuration file has the following format:
You can [configure](external_dicts_dict.md) any number of dictionaries in the same file.
-[DDL queries for dictionaries](../create.md#create-dictionary-query) doesn’t require any additional records in server configuration. They allow to work with dictionaries as first-class entities, like tables or views.
+[DDL queries for dictionaries](../../statements/create.md#create-dictionary-query) doesn’t require any additional records in server configuration. They allow to work with dictionaries as first-class entities, like tables or views.
!!! attention "Attention"
- You can convert values for a small dictionary by describing it in a `SELECT` query (see the [transform](../functions/other_functions.md) function). This functionality is not related to external dictionaries.
+ You can convert values for a small dictionary by describing it in a `SELECT` query (see the [transform](../../../sql_reference/functions/other_functions.md) function). This functionality is not related to external dictionaries.
-## See also {#ext-dicts-see-also}
+## See Also {#ext-dicts-see-also}
- [Configuring an External Dictionary](external_dicts_dict.md)
- [Storing Dictionaries in Memory](external_dicts_dict_layout.md)
- [Dictionary Updates](external_dicts_dict_lifetime.md)
- [Sources of External Dictionaries](external_dicts_dict_sources.md)
- [Dictionary Key and Fields](external_dicts_dict_structure.md)
-- [Functions for Working with External Dictionaries](../functions/ext_dict_functions.md)
+- [Functions for Working with External Dictionaries](../../../sql_reference/functions/ext_dict_functions.md)
[Original article](https://clickhouse.tech/docs/en/query_language/dicts/external_dicts/)
diff --git a/docs/en/query_language/dicts/external_dicts_dict.md b/docs/en/sql_reference/dictionaries/external_dictionaries/external_dicts_dict.md
similarity index 88%
rename from docs/en/query_language/dicts/external_dicts_dict.md
rename to docs/en/sql_reference/dictionaries/external_dictionaries/external_dicts_dict.md
index f92c5b33791..16e9a4ada39 100644
--- a/docs/en/query_language/dicts/external_dicts_dict.md
+++ b/docs/en/sql_reference/dictionaries/external_dictionaries/external_dicts_dict.md
@@ -1,3 +1,8 @@
+---
+toc_priority: 40
+toc_title: Configuring an External Dictionary
+---
+
# Configuring an External Dictionary {#dicts-external-dicts-dict}
If dictionary is configured using xml file, than dictionary configuration has the following structure:
@@ -24,7 +29,7 @@ If dictionary is configured using xml file, than dictionary configuration has th
```
-Corresponding [DDL-query](../create.md#create-dictionary-query) has the following structure:
+Corresponding [DDL-query](../../statements/create.md#create-dictionary-query) has the following structure:
``` sql
CREATE DICTIONARY dict_name
diff --git a/docs/en/query_language/dicts/external_dicts_dict_hierarchical.md b/docs/en/sql_reference/dictionaries/external_dictionaries/external_dicts_dict_hierarchical.md
similarity index 87%
rename from docs/en/query_language/dicts/external_dicts_dict_hierarchical.md
rename to docs/en/sql_reference/dictionaries/external_dictionaries/external_dicts_dict_hierarchical.md
index 0f94eac6149..b56245539cb 100644
--- a/docs/en/query_language/dicts/external_dicts_dict_hierarchical.md
+++ b/docs/en/sql_reference/dictionaries/external_dictionaries/external_dicts_dict_hierarchical.md
@@ -1,3 +1,8 @@
+---
+toc_priority: 45
+toc_title: Hierarchical dictionaries
+---
+
# Hierarchical Dictionaries {#hierarchical-dictionaries}
ClickHouse supports hierarchical dictionaries with a [numeric key](external_dicts_dict_structure.md#ext_dict-numeric-key).
@@ -21,7 +26,7 @@ Look at the following hierarchical structure:
This hierarchy can be expressed as the following dictionary table.
| region\_id | parent\_region | region\_name |
-|------------|----------------|---------------|
+|--------|----------|---------|
| 1 | 0 | Russia |
| 2 | 1 | Moscow |
| 3 | 2 | Center |
@@ -32,7 +37,7 @@ This table contains a column `parent_region` that contains the key of the neares
ClickHouse supports the [hierarchical](external_dicts_dict_structure.md#hierarchical-dict-attr) property for [external dictionary](index.md) attributes. This property allows you to configure the hierarchical dictionary similar to described above.
-The [dictGetHierarchy](../functions/ext_dict_functions.md#dictgethierarchy) function allows you to get the parent chain of an element.
+The [dictGetHierarchy](../../../sql_reference/functions/ext_dict_functions.md#dictgethierarchy) function allows you to get the parent chain of an element.
For our example, the structure of dictionary can be the following:
diff --git a/docs/en/query_language/dicts/external_dicts_dict_layout.md b/docs/en/sql_reference/dictionaries/external_dictionaries/external_dicts_dict_layout.md
similarity index 94%
rename from docs/en/query_language/dicts/external_dicts_dict_layout.md
rename to docs/en/sql_reference/dictionaries/external_dictionaries/external_dicts_dict_layout.md
index c3394fde802..663cde42f9b 100644
--- a/docs/en/query_language/dicts/external_dicts_dict_layout.md
+++ b/docs/en/sql_reference/dictionaries/external_dictionaries/external_dicts_dict_layout.md
@@ -1,4 +1,9 @@
-# Storing Dictionaries in Memory {#dicts-external-dicts-dict-layout}
+---
+toc_priority: 41
+toc_title: Storing Dictionaries in Memory
+---
+
+# Storing Dictionaries In Memory {#dicts-external-dicts-dict-layout}
There are a variety of ways to store dictionaries in memory.
@@ -34,7 +39,7 @@ The configuration looks like this:
```
-Corresponding [DDL-query](../create.md#create-dictionary-query):
+Corresponding [DDL-query](../../statements/create.md#create-dictionary-query):
``` sql
CREATE DICTIONARY (...)
@@ -43,7 +48,7 @@ LAYOUT(LAYOUT_TYPE(param value)) -- layout settings
...
```
-## Ways to Store Dictionaries in Memory {#ways-to-store-dictionaries-in-memory}
+## Ways To Store Dictionaries In Memory {#ways-to-store-dictionaries-in-memory}
- [flat](#flat)
- [hashed](#dicts-external_dicts_dict_layout-hashed)
@@ -139,15 +144,15 @@ This storage method works the same way as hashed and allows using date/time (arb
Example: The table contains discounts for each advertiser in the format:
``` text
-+---------------|---------------------|-------------------|--------+
++---------|-------------|-------------|------+
| advertiser id | discount start date | discount end date | amount |
+===============+=====================+===================+========+
| 123 | 2015-01-01 | 2015-01-15 | 0.15 |
-+---------------|---------------------|-------------------|--------+
++---------|-------------|-------------|------+
| 123 | 2015-01-16 | 2015-01-31 | 0.25 |
-+---------------|---------------------|-------------------|--------+
++---------|-------------|-------------|------+
| 456 | 2015-01-01 | 2015-01-15 | 0.05 |
-+---------------|---------------------|-------------------|--------+
++---------|-------------|-------------|------+
```
To use a sample for date ranges, define the `range_min` and `range_max` elements in the [structure](external_dicts_dict_structure.md). These elements must contain elements `name` and`type` (if `type` is not specified, the default type will be used - Date). `type` can be any numeric type (Date / DateTime / UInt64 / Int32 / others).
@@ -296,17 +301,17 @@ This type of storage is for mapping network prefixes (IP addresses) to metadata
Example: The table contains network prefixes and their corresponding AS number and country code:
``` text
- +-----------------|-------|--------+
+ +-----------|-----|------+
| prefix | asn | cca2 |
+=================+=======+========+
| 202.79.32.0/20 | 17501 | NP |
- +-----------------|-------|--------+
+ +-----------|-----|------+
| 2620:0:870::/48 | 3856 | US |
- +-----------------|-------|--------+
+ +-----------|-----|------+
| 2a02:6b8:1::/48 | 13238 | RU |
- +-----------------|-------|--------+
+ +-----------|-----|------+
| 2001:db8::/32 | 65536 | ZZ |
- +-----------------|-------|--------+
+ +-----------|-----|------+
```
When using this type of layout, the structure must have a composite key.
diff --git a/docs/en/query_language/dicts/external_dicts_dict_lifetime.md b/docs/en/sql_reference/dictionaries/external_dictionaries/external_dicts_dict_lifetime.md
similarity index 97%
rename from docs/en/query_language/dicts/external_dicts_dict_lifetime.md
rename to docs/en/sql_reference/dictionaries/external_dictionaries/external_dicts_dict_lifetime.md
index 12fca45e39f..645c6347f66 100644
--- a/docs/en/query_language/dicts/external_dicts_dict_lifetime.md
+++ b/docs/en/sql_reference/dictionaries/external_dictionaries/external_dicts_dict_lifetime.md
@@ -1,3 +1,8 @@
+---
+toc_priority: 42
+toc_title: Dictionary Updates
+---
+
# Dictionary Updates {#dictionary-updates}
ClickHouse periodically updates the dictionaries. The update interval for fully downloaded dictionaries and the invalidation interval for cached dictionaries are defined in the `` tag in seconds.
diff --git a/docs/en/query_language/dicts/external_dicts_dict_sources.md b/docs/en/sql_reference/dictionaries/external_dictionaries/external_dicts_dict_sources.md
similarity index 92%
rename from docs/en/query_language/dicts/external_dicts_dict_sources.md
rename to docs/en/sql_reference/dictionaries/external_dictionaries/external_dicts_dict_sources.md
index 1756936febf..174341f697e 100644
--- a/docs/en/query_language/dicts/external_dicts_dict_sources.md
+++ b/docs/en/sql_reference/dictionaries/external_dictionaries/external_dicts_dict_sources.md
@@ -1,4 +1,9 @@
-# Sources of External Dictionaries {#dicts-external-dicts-dict-sources}
+---
+toc_priority: 43
+toc_title: Sources of External Dictionaries
+---
+
+# Sources Of External Dictionaries {#dicts-external-dicts-dict-sources}
An external dictionary can be connected from many different sources.
@@ -19,7 +24,7 @@ If dictionary is configured using xml-file, the configuration looks like this:
```
-In case of [DDL-query](../create.md#create-dictionary-query), equal configuration will looks like:
+In case of [DDL-query](../../statements/create.md#create-dictionary-query), equal configuration will looks like:
``` sql
CREATE DICTIONARY dict_name (...)
@@ -64,7 +69,7 @@ SOURCE(FILE(path '/opt/dictionaries/os.tsv' format 'TabSeparated'))
Setting fields:
- `path` – The absolute path to the file.
-- `format` – The file format. All the formats described in “[Formats](../../interfaces/formats.md#formats)” are supported.
+- `format` – The file format. All the formats described in “[Formats](../../../interfaces/formats.md#formats)” are supported.
## Executable File {#dicts-external_dicts_dict_sources-executable}
@@ -90,9 +95,9 @@ SOURCE(EXECUTABLE(command 'cat /opt/dictionaries/os.tsv' format 'TabSeparated'))
Setting fields:
- `command` – The absolute path to the executable file, or the file name (if the program directory is written to `PATH`).
-- `format` – The file format. All the formats described in “[Formats](../../interfaces/formats.md#formats)” are supported.
+- `format` – The file format. All the formats described in “[Formats](../../../interfaces/formats.md#formats)” are supported.
-## HTTP(s) {#dicts-external_dicts_dict_sources-http}
+## Http(s) {#dicts-external_dicts_dict_sources-http}
Working with an HTTP(s) server depends on [how the dictionary is stored in memory](external_dicts_dict_layout.md). If the dictionary is stored using `cache` and `complex_key_cache`, ClickHouse requests the necessary keys by sending a request via the `POST` method.
@@ -128,12 +133,12 @@ SOURCE(HTTP(
))
```
-In order for ClickHouse to access an HTTPS resource, you must [configure openSSL](../../operations/server_settings/settings.md#server_settings-openssl) in the server configuration.
+In order for ClickHouse to access an HTTPS resource, you must [configure openSSL](../../../operations/server_configuration_parameters/settings.md#server_configuration_parameters-openssl) in the server configuration.
Setting fields:
- `url` – The source URL.
-- `format` – The file format. All the formats described in “[Formats](../../interfaces/formats.md#formats)” are supported.
+- `format` – The file format. All the formats described in “[Formats](../../../interfaces/formats.md#formats)” are supported.
- `credentials` – Basic HTTP authentication. Optional parameter.
- `user` – Username required for the authentication.
- `password` – Password required for the authentication.
@@ -179,9 +184,9 @@ Setting fields:
ClickHouse receives quoting symbols from ODBC-driver and quote all settings in queries to driver, so it’s necessary to set table name accordingly to table name case in database.
-If you have a problems with encodings when using Oracle, see the corresponding [FAQ](../../faq/general.md#oracle-odbc-encodings) article.
+If you have a problems with encodings when using Oracle, see the corresponding [FAQ](../../../faq/general.md#oracle-odbc-encodings) article.
-### Known vulnerability of the ODBC dictionary functionality {#known-vulnerability-of-the-odbc-dictionary-functionality}
+### Known Vulnerability Of the ODBC Dictionary Functionality {#known-vulnerability-of-the-odbc-dictionary-functionality}
!!! attention "Attention"
When connecting to the database through the ODBC driver connection parameter `Servername` can be substituted. In this case values of `USERNAME` and `PASSWORD` from `odbc.ini` are sent to the remote server and can be compromised.
@@ -209,7 +214,7 @@ SELECT * FROM odbc('DSN=gregtest;Servername=some-server.com', 'test_db');
ODBC driver will send values of `USERNAME` and `PASSWORD` from `odbc.ini` to `some-server.com`.
-### Example of Connecting PostgreSQL {#example-of-connecting-postgresql}
+### Example Of Connecting Postgresql {#example-of-connecting-postgresql}
Ubuntu OS.
@@ -290,7 +295,7 @@ LIFETIME(MIN 300 MAX 360)
You may need to edit `odbc.ini` to specify the full path to the library with the driver `DRIVER=/usr/local/lib/psqlodbcw.so`.
-### Example of Connecting MS SQL Server {#example-of-connecting-ms-sql-server}
+### Example Of Connecting MS SQL Server {#example-of-connecting-ms-sql-server}
Ubuntu OS.
@@ -386,7 +391,7 @@ LIFETIME(MIN 300 MAX 360)
## DBMS {#dbms}
-### MySQL {#dicts-external_dicts_dict_sources-mysql}
+### Mysql {#dicts-external_dicts_dict_sources-mysql}
Example of settings:
@@ -483,7 +488,7 @@ SOURCE(MYSQL(
))
```
-### ClickHouse {#dicts-external_dicts_dict_sources-clickhouse}
+### Clickhouse {#dicts-external_dicts_dict_sources-clickhouse}
Example of settings:
@@ -517,7 +522,7 @@ SOURCE(CLICKHOUSE(
Setting fields:
-- `host` – The ClickHouse host. If it is a local host, the query is processed without any network activity. To improve fault tolerance, you can create a [Distributed](../../operations/table_engines/distributed.md) table and enter it in subsequent configurations.
+- `host` – The ClickHouse host. If it is a local host, the query is processed without any network activity. To improve fault tolerance, you can create a [Distributed](../../../engines/table_engines/special/distributed.md) table and enter it in subsequent configurations.
- `port` – The port on the ClickHouse server.
- `user` – Name of the ClickHouse user.
- `password` – Password of the ClickHouse user.
@@ -526,7 +531,7 @@ Setting fields:
- `where` – The selection criteria. May be omitted.
- `invalidate_query` – Query for checking the dictionary status. Optional parameter. Read more in the section [Updating dictionaries](external_dicts_dict_lifetime.md).
-### MongoDB {#dicts-external_dicts_dict_sources-mongodb}
+### Mongodb {#dicts-external_dicts_dict_sources-mongodb}
Example of settings:
diff --git a/docs/en/query_language/dicts/external_dicts_dict_structure.md b/docs/en/sql_reference/dictionaries/external_dictionaries/external_dicts_dict_structure.md
similarity index 90%
rename from docs/en/query_language/dicts/external_dicts_dict_structure.md
rename to docs/en/sql_reference/dictionaries/external_dictionaries/external_dicts_dict_structure.md
index fc13ba3512a..460320e6fa3 100644
--- a/docs/en/query_language/dicts/external_dicts_dict_structure.md
+++ b/docs/en/sql_reference/dictionaries/external_dictionaries/external_dicts_dict_structure.md
@@ -1,3 +1,8 @@
+---
+toc_priority: 44
+toc_title: Dictionary Key and Fields
+---
+
# Dictionary Key and Fields {#dictionary-key-and-fields}
The `` clause describes the dictionary key and fields available for queries.
@@ -152,17 +157,17 @@ CREATE DICTIONARY somename (
Configuration fields:
| Tag | Description | Required |
-|------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|----------|
+|----------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|------|
| `name` | Column name. | Yes |
-| `type` | ClickHouse data type.
ClickHouse tries to cast value from dictionary to the specified data type. For example, for MySQL, the field might be `TEXT`, `VARCHAR`, or `BLOB` in the MySQL source table, but it can be uploaded as `String` in ClickHouse.
[Nullable](../../data_types/nullable.md) is not supported. | Yes |
+| `type` | ClickHouse data type.
ClickHouse tries to cast value from dictionary to the specified data type. For example, for MySQL, the field might be `TEXT`, `VARCHAR`, or `BLOB` in the MySQL source table, but it can be uploaded as `String` in ClickHouse.
[Nullable](../../../sql_reference/data_types/nullable.md) is not supported. | Yes |
| `null_value` | Default value for a non-existing element.
In the example, it is an empty string. You cannot use `NULL` in this field. | Yes |
-| `expression` | [Expression](../syntax.md#syntax-expressions) that ClickHouse executes on the value.
The expression can be a column name in the remote SQL database. Thus, you can use it to create an alias for the remote column.
Default value: no expression. | No |
+| `expression` | [Expression](../../syntax.md#syntax-expressions) that ClickHouse executes on the value.
The expression can be a column name in the remote SQL database. Thus, you can use it to create an alias for the remote column.
Default value: no expression. | No |
| `hierarchical` | If `true`, the attribute contains the value of a parent key for the current key. See [Hierarchical Dictionaries](external_dicts_dict_hierarchical.md).
Default value: `false`. | No |
| `injective` | Flag that shows whether the `id -> attribute` image is [injective](https://en.wikipedia.org/wiki/Injective_function).
If `true`, ClickHouse can automatically place after the `GROUP BY` clause the requests to dictionaries with injection. Usually it significantly reduces the amount of such requests.
Default value: `false`. | No |
| `is_object_id` | Flag that shows whether the query is executed for a MongoDB document by `ObjectID`.
Default value: `false`. | No |
## See Also {#see-also}
-- [Functions for working with external dictionaries](../functions/ext_dict_functions.md).
+- [Functions for working with external dictionaries](../../../sql_reference/functions/ext_dict_functions.md).
[Original article](https://clickhouse.tech/docs/en/query_language/dicts/external_dicts_dict_structure/)
diff --git a/docs/en/sql_reference/dictionaries/external_dictionaries/index.md b/docs/en/sql_reference/dictionaries/external_dictionaries/index.md
new file mode 100644
index 00000000000..9af8b4f2f12
--- /dev/null
+++ b/docs/en/sql_reference/dictionaries/external_dictionaries/index.md
@@ -0,0 +1,5 @@
+---
+toc_folder_title: External Dictionaries
+toc_priority: 37
+---
+
diff --git a/docs/en/query_language/dicts/index.md b/docs/en/sql_reference/dictionaries/index.md
similarity index 64%
rename from docs/en/query_language/dicts/index.md
rename to docs/en/sql_reference/dictionaries/index.md
index 593b37d13ba..9806b27ed9b 100644
--- a/docs/en/query_language/dicts/index.md
+++ b/docs/en/sql_reference/dictionaries/index.md
@@ -1,3 +1,9 @@
+---
+toc_folder_title: Dictionaries
+toc_priority: 35
+toc_title: Introduction
+---
+
# Dictionaries {#dictionaries}
A dictionary is a mapping (`key -> attributes`) that is convenient for various types of reference lists.
@@ -8,7 +14,7 @@ ClickHouse supports special functions for working with dictionaries that can be
ClickHouse supports:
-- [Built-in dictionaries](internal_dicts.md#internal_dicts) with a specific [set of functions](../functions/ym_dict_functions.md).
-- [Plug-in (external) dictionaries](external_dicts.md) with a [set of functions](../functions/ext_dict_functions.md).
+- [Built-in dictionaries](internal_dicts.md#internal_dicts) with a specific [set of functions](../../sql_reference/functions/ym_dict_functions.md).
+- [Plug-in (external) dictionaries](external_dictionaries/external_dicts.md) with a [net of functions](../../sql_reference/functions/ext_dict_functions.md).
[Original article](https://clickhouse.tech/docs/en/query_language/dicts/)
diff --git a/docs/en/query_language/dicts/internal_dicts.md b/docs/en/sql_reference/dictionaries/internal_dicts.md
similarity index 98%
rename from docs/en/query_language/dicts/internal_dicts.md
rename to docs/en/sql_reference/dictionaries/internal_dicts.md
index b561fe269ff..327206c9672 100644
--- a/docs/en/query_language/dicts/internal_dicts.md
+++ b/docs/en/sql_reference/dictionaries/internal_dicts.md
@@ -1,3 +1,8 @@
+---
+toc_priority: 39
+toc_title: Internal Dictionaries
+---
+
# Internal dictionaries {#internal_dicts}
ClickHouse contains a built-in feature for working with a geobase.
diff --git a/docs/en/query_language/functions/arithmetic_functions.md b/docs/en/sql_reference/functions/arithmetic_functions.md
similarity index 98%
rename from docs/en/query_language/functions/arithmetic_functions.md
rename to docs/en/sql_reference/functions/arithmetic_functions.md
index b87d4afd646..36cb6e9ada9 100644
--- a/docs/en/query_language/functions/arithmetic_functions.md
+++ b/docs/en/sql_reference/functions/arithmetic_functions.md
@@ -1,3 +1,8 @@
+---
+toc_priority: 35
+toc_title: Arithmetic
+---
+
# Arithmetic functions {#arithmetic-functions}
For all arithmetic functions, the result type is calculated as the smallest number type that the result fits in, if there is such a type. The minimum is taken simultaneously based on the number of bits, whether it is signed, and whether it floats. If there are not enough bits, the highest bit type is taken.
diff --git a/docs/en/query_language/functions/array_functions.md b/docs/en/sql_reference/functions/array_functions.md
similarity index 95%
rename from docs/en/query_language/functions/array_functions.md
rename to docs/en/sql_reference/functions/array_functions.md
index 663b096fc3d..0b6b41d04ac 100644
--- a/docs/en/query_language/functions/array_functions.md
+++ b/docs/en/sql_reference/functions/array_functions.md
@@ -1,3 +1,8 @@
+---
+toc_priority: 46
+toc_title: Working with Arrays
+---
+
# Functions for working with arrays {#functions-for-working-with-arrays}
## empty {#function-empty}
@@ -58,7 +63,7 @@ arrayConcat(arrays)
**Parameters**
-- `arrays` – Arbitrary number of arguments of [Array](../../data_types/array.md) type.
+- `arrays` – Arbitrary number of arguments of [Array](../../sql_reference/data_types/array.md) type.
**Example**
@@ -363,7 +368,7 @@ arrayPushBack(array, single_value)
**Parameters**
- `array` – Array.
-- `single_value` – A single value. Only numbers can be added to an array with numbers, and only strings can be added to an array of strings. When adding numbers, ClickHouse automatically sets the `single_value` type for the data type of the array. For more information about the types of data in ClickHouse, see “[Data types](../../data_types/index.md#data_types)”. Can be `NULL`. The function adds a `NULL` element to an array, and the type of array elements converts to `Nullable`.
+- `single_value` – A single value. Only numbers can be added to an array with numbers, and only strings can be added to an array of strings. When adding numbers, ClickHouse automatically sets the `single_value` type for the data type of the array. For more information about the types of data in ClickHouse, see “[Data types](../../sql_reference/data_types/index.md#data_types)”. Can be `NULL`. The function adds a `NULL` element to an array, and the type of array elements converts to `Nullable`.
**Example**
@@ -388,7 +393,7 @@ arrayPushFront(array, single_value)
**Parameters**
- `array` – Array.
-- `single_value` – A single value. Only numbers can be added to an array with numbers, and only strings can be added to an array of strings. When adding numbers, ClickHouse automatically sets the `single_value` type for the data type of the array. For more information about the types of data in ClickHouse, see “[Data types](../../data_types/index.md#data_types)”. Can be `NULL`. The function adds a `NULL` element to an array, and the type of array elements converts to `Nullable`.
+- `single_value` – A single value. Only numbers can be added to an array with numbers, and only strings can be added to an array of strings. When adding numbers, ClickHouse automatically sets the `single_value` type for the data type of the array. For more information about the types of data in ClickHouse, see “[Data types](../../sql_reference/data_types/index.md#data_types)”. Can be `NULL`. The function adds a `NULL` element to an array, and the type of array elements converts to `Nullable`.
**Example**
@@ -816,8 +821,8 @@ arrayReduce(agg_func, arr1, arr2, ..., arrN)
**Parameters**
-* `agg_func` — The name of an aggregate function which should be a constant [string](../../data_types/string.md).
-* `arr` — Any number of [array](../../data_types/array.md) type columns as the parameters of the aggregation function.
+* `agg_func` — The name of an aggregate function which should be a constant [string](../../sql_reference/data_types/string.md).
+* `arr` — Any number of [array](../../sql_reference/data_types/array.md) type columns as the parameters of the aggregation function.
**Returned value**
@@ -869,9 +874,9 @@ arrayReduceInRanges(agg_func, ranges, arr1, arr2, ..., arrN)
**Parameters**
-* `agg_func` — The name of an aggregate function which should be a constant [string](../../data_types/string.md).
-* `ranges` — The ranges to aggretate which should be an [array](../../data_types/array.md) of [tuples](../../data_types/tuple.md) which containing the index and the length of each range.
-* `arr` — Any number of [array](../../data_types/array.md) type columns as the parameters of the aggregation function.
+* `agg_func` — The name of an aggregate function which should be a constant [string](../../sql_reference/data_types/string.md).
+* `ranges` — The ranges to aggretate which should be an [array](../../sql_reference/data_types/array.md) of [tuples](../../sql_reference/data_types/tuple.md) which containing the index and the length of each range.
+* `arr` — Any number of [array](../../sql_reference/data_types/array.md) type columns as the parameters of the aggregation function.
**Returned value**
@@ -932,7 +937,7 @@ Alias: `flatten`.
**Parameters**
-- `array_of_arrays` — [Array](../../data_types/array.md) of arrays. For example, `[[1,2,3], [4,5]]`.
+- `array_of_arrays` — [Array](../../sql_reference/data_types/array.md) of arrays. For example, `[[1,2,3], [4,5]]`.
**Examples**
@@ -958,7 +963,7 @@ arrayCompact(arr)
**Parameters**
-`arr` — The [array](../../data_types/array.md) to inspect.
+`arr` — The [array](../../sql_reference/data_types/array.md) to inspect.
**Returned value**
@@ -994,7 +999,7 @@ arrayZip(arr1, arr2, ..., arrN)
**Parameters**
-`arr` — Any number of [array](../../data_types/array.md) type columns to combine.
+`arr` — Any number of [array](../../sql_reference/data_types/array.md) type columns to combine.
**Returned value**
@@ -1042,7 +1047,7 @@ Result:
```text
┌─arrayAUC([0.1, 0.4, 0.35, 0.8], [0, 0, 1, 1])─┐
│ 0.75 │
-└────────────────────────────────────────-----──┘
+└────────────────────────────────────────---──┘
```
[Original article](https://clickhouse.tech/docs/en/query_language/functions/array_functions/)
diff --git a/docs/en/query_language/functions/array_join.md b/docs/en/sql_reference/functions/array_join.md
similarity index 96%
rename from docs/en/query_language/functions/array_join.md
rename to docs/en/sql_reference/functions/array_join.md
index 5f58a39dc71..f1f9a545366 100644
--- a/docs/en/query_language/functions/array_join.md
+++ b/docs/en/sql_reference/functions/array_join.md
@@ -1,3 +1,8 @@
+---
+toc_priority: 61
+toc_title: arrayJoin
+---
+
# arrayJoin function {#functions_arrayjoin}
This is a very unusual function.
diff --git a/docs/en/query_language/functions/bit_functions.md b/docs/en/sql_reference/functions/bit_functions.md
similarity index 94%
rename from docs/en/query_language/functions/bit_functions.md
rename to docs/en/sql_reference/functions/bit_functions.md
index 31ce017feb3..dbed3ac2230 100644
--- a/docs/en/query_language/functions/bit_functions.md
+++ b/docs/en/sql_reference/functions/bit_functions.md
@@ -1,3 +1,8 @@
+---
+toc_priority: 48
+toc_title: Bit
+---
+
# Bit functions {#bit-functions}
Bit functions work for any pair of types from UInt8, UInt16, UInt32, UInt64, Int8, Int16, Int32, Int64, Float32, or Float64.
@@ -216,7 +221,7 @@ bitCount(x)
**Parameters**
-- `x` — [Integer](../../data_types/int_uint.md) or [floating-point](../../data_types/float.md) number. The function uses the value representation in memory. It allows supporting floating-point numbers.
+- `x` — [Integer](../../sql_reference/data_types/int_uint.md) or [floating-point](../../sql_reference/data_types/float.md) number. The function uses the value representation in memory. It allows supporting floating-point numbers.
**Returned value**
diff --git a/docs/en/query_language/functions/bitmap_functions.md b/docs/en/sql_reference/functions/bitmap_functions.md
similarity index 96%
rename from docs/en/query_language/functions/bitmap_functions.md
rename to docs/en/sql_reference/functions/bitmap_functions.md
index 056491b3941..8e47fa24034 100644
--- a/docs/en/query_language/functions/bitmap_functions.md
+++ b/docs/en/sql_reference/functions/bitmap_functions.md
@@ -1,3 +1,8 @@
+---
+toc_priority: 49
+toc_title: Bitmap
+---
+
# Bitmap functions {#bitmap-functions}
Bitmap functions work for two bitmaps Object value calculation, it is to return new bitmap or cardinality while using formula calculation, such as and, or, xor, and not, etc.
@@ -67,8 +72,8 @@ bitmapSubsetInRange(bitmap, range_start, range_end)
**Parameters**
- `bitmap` – [Bitmap object](#bitmap_functions-bitmapbuild).
-- `range_start` – range start point. Type: [UInt32](../../data_types/int_uint.md).
-- `range_end` – range end point(excluded). Type: [UInt32](../../data_types/int_uint.md).
+- `range_start` – range start point. Type: [UInt32](../../sql_reference/data_types/int_uint.md).
+- `range_end` – range end point(excluded). Type: [UInt32](../../sql_reference/data_types/int_uint.md).
**Example**
@@ -95,8 +100,8 @@ bitmapSubsetLimit(bitmap, range_start, cardinality_limit)
**Parameters**
- `bitmap` – [Bitmap object](#bitmap_functions-bitmapbuild).
-- `range_start` – The subset starting point. Type: [UInt32](../../data_types/int_uint.md).
-- `cardinality_limit` – The subset cardinality upper limit. Type: [UInt32](../../data_types/int_uint.md).
+- `range_start` – The subset starting point. Type: [UInt32](../../sql_reference/data_types/int_uint.md).
+- `cardinality_limit` – The subset cardinality upper limit. Type: [UInt32](../../sql_reference/data_types/int_uint.md).
**Returned value**
@@ -131,7 +136,7 @@ bitmapContains(haystack, needle)
**Parameters**
- `haystack` – [Bitmap object](#bitmap_functions-bitmapbuild), where the function searches.
-- `needle` – Value that the function searches. Type: [UInt32](../../data_types/int_uint.md).
+- `needle` – Value that the function searches. Type: [UInt32](../../sql_reference/data_types/int_uint.md).
**Returned values**
diff --git a/docs/en/query_language/functions/comparison_functions.md b/docs/en/sql_reference/functions/comparison_functions.md
similarity index 95%
rename from docs/en/query_language/functions/comparison_functions.md
rename to docs/en/sql_reference/functions/comparison_functions.md
index 6865fce5ef0..368ac447dd1 100644
--- a/docs/en/query_language/functions/comparison_functions.md
+++ b/docs/en/sql_reference/functions/comparison_functions.md
@@ -1,3 +1,8 @@
+---
+toc_priority: 36
+toc_title: Comparison
+---
+
# Comparison functions {#comparison-functions}
Comparison functions always return 0 or 1 (Uint8).
diff --git a/docs/en/query_language/functions/conditional_functions.md b/docs/en/sql_reference/functions/conditional_functions.md
similarity index 99%
rename from docs/en/query_language/functions/conditional_functions.md
rename to docs/en/sql_reference/functions/conditional_functions.md
index be55c96daf3..0f9f0b78ce4 100644
--- a/docs/en/query_language/functions/conditional_functions.md
+++ b/docs/en/sql_reference/functions/conditional_functions.md
@@ -1,3 +1,8 @@
+---
+toc_priority: 43
+toc_title: 'Conditional '
+---
+
# Conditional functions {#conditional-functions}
## if {#if}
diff --git a/docs/en/query_language/functions/date_time_functions.md b/docs/en/sql_reference/functions/date_time_functions.md
similarity index 98%
rename from docs/en/query_language/functions/date_time_functions.md
rename to docs/en/sql_reference/functions/date_time_functions.md
index f01ddf2a82d..3c477665bb7 100644
--- a/docs/en/query_language/functions/date_time_functions.md
+++ b/docs/en/sql_reference/functions/date_time_functions.md
@@ -1,3 +1,8 @@
+---
+toc_priority: 39
+toc_title: Working with Dates and Times
+---
+
# Functions for working with dates and times {#functions-for-working-with-dates-and-times}
Support for time zones
@@ -215,7 +220,7 @@ This function returns the week number for date or datetime. The two-argument for
The following table describes how the mode argument works.
| Mode | First day of week | Range | Week 1 is the first week … |
-|------|-------------------|-------|-------------------------------|
+|----|-------------|-----|-------------------|
| 0 | Sunday | 0-53 | with a Sunday in this year |
| 1 | Monday | 0-53 | with 4 or more days this year |
| 2 | Sunday | 1-53 | with a Sunday in this year |
@@ -364,7 +369,7 @@ dateDiff('unit', startdate, enddate, [timezone])
Supported values:
| unit |
- | ------ |
+ | ---- |
|second |
|minute |
|hour |
@@ -374,9 +379,9 @@ dateDiff('unit', startdate, enddate, [timezone])
|quarter |
|year |
-- `startdate` — The first time value to compare. [Date](../../data_types/date.md) or [DateTime](../../data_types/datetime.md).
+- `startdate` — The first time value to compare. [Date](../../sql_reference/data_types/date.md) or [DateTime](../../sql_reference/data_types/datetime.md).
-- `enddate` — The second time value to compare. [Date](../../data_types/date.md) or [DateTime](../../data_types/datetime.md).
+- `enddate` — The second time value to compare. [Date](../../sql_reference/data_types/date.md) or [DateTime](../../sql_reference/data_types/datetime.md).
- `timezone` — Optional parameter. If specified, it is applied to both `startdate` and `enddate`. If not specified, timezones of `startdate` and `enddate` are used. If they are not the same, the result is unspecified.
@@ -416,7 +421,7 @@ Supported modifiers for Format:
(“Example” column shows formatting result for time `2018-01-02 22:33:44`)
| Modifier | Description | Example |
-|----------|---------------------------------------------------------|------------|
+|------|-----------------------------------|--------|
| %C | year divided by 100 and truncated to integer (00-99) | 20 |
| %d | day of the month, zero-padded (01-31) | 02 |
| %D | Short MM/DD/YY date, equivalent to %m/%d/%y | 01/02/18 |
diff --git a/docs/en/query_language/functions/encoding_functions.md b/docs/en/sql_reference/functions/encoding_functions.md
similarity index 90%
rename from docs/en/query_language/functions/encoding_functions.md
rename to docs/en/sql_reference/functions/encoding_functions.md
index 50b79a22b13..4802b2786c2 100644
--- a/docs/en/query_language/functions/encoding_functions.md
+++ b/docs/en/sql_reference/functions/encoding_functions.md
@@ -1,3 +1,8 @@
+---
+toc_priority: 52
+toc_title: Encoding
+---
+
# Encoding functions {#encoding-functions}
## char {#char}
@@ -12,7 +17,7 @@ char(number_1, [number_2, ..., number_n]);
**Parameters**
-- `number_1, number_2, ..., number_n` — Numerical arguments interpreted as integers. Types: [Int](../../data_types/int_uint.md), [Float](../../data_types/float.md).
+- `number_1, number_2, ..., number_n` — Numerical arguments interpreted as integers. Types: [Int](../../sql_reference/data_types/int_uint.md), [Float](../../sql_reference/data_types/float.md).
**Returned value**
@@ -104,7 +109,7 @@ Values of floating point and Decimal types are encoded as their representation i
**Parameters**
-- `arg` — A value to convert to hexadecimal. Types: [String](../../data_types/string.md), [UInt](../../data_types/int_uint.md), [Float](../../data_types/float.md), [Decimal](../../data_types/decimal.md), [Date](../../data_types/date.md) or [DateTime](../../data_types/datetime.md).
+- `arg` — A value to convert to hexadecimal. Types: [String](../../sql_reference/data_types/string.md), [UInt](../../sql_reference/data_types/int_uint.md), [Float](../../sql_reference/data_types/float.md), [Decimal](../../sql_reference/data_types/decimal.md), [Date](../../sql_reference/data_types/date.md) or [DateTime](../../sql_reference/data_types/datetime.md).
**Returned value**
diff --git a/docs/en/query_language/functions/ext_dict_functions.md b/docs/en/sql_reference/functions/ext_dict_functions.md
similarity index 78%
rename from docs/en/query_language/functions/ext_dict_functions.md
rename to docs/en/sql_reference/functions/ext_dict_functions.md
index 4a0df0e9929..fa3b0519c37 100644
--- a/docs/en/query_language/functions/ext_dict_functions.md
+++ b/docs/en/sql_reference/functions/ext_dict_functions.md
@@ -1,6 +1,11 @@
-# Functions for Working with External Dictionaries {#ext_dict_functions}
+---
+toc_priority: 58
+toc_title: Working with External Dictionaries
+---
-For information on connecting and configuring external dictionaries, see [External dictionaries](../dicts/external_dicts.md).
+# Functions For Working With External Dictionaries {#ext_dict_functions}
+
+For information on connecting and configuring external dictionaries, see [External dictionaries](../../sql_reference/dictionaries/external_dictionaries/external_dicts.md).
## dictGet {#dictget}
@@ -15,12 +20,12 @@ dictGetOrDefault('dict_name', 'attr_name', id_expr, default_value_expr)
- `dict_name` — Name of the dictionary. [String literal](../syntax.md#syntax-string-literal).
- `attr_name` — Name of the column of the dictionary. [String literal](../syntax.md#syntax-string-literal).
-- `id_expr` — Key value. [Expression](../syntax.md#syntax-expressions) returning a [UInt64](../../data_types/int_uint.md) or [Tuple](../../data_types/tuple.md)-type value depending on the dictionary configuration.
+- `id_expr` — Key value. [Expression](../syntax.md#syntax-expressions) returning a [UInt64](../../sql_reference/data_types/int_uint.md) or [Tuple](../../sql_reference/data_types/tuple.md)-type value depending on the dictionary configuration.
- `default_value_expr` — Value returned if the dictionary doesn’t contain a row with the `id_expr` key. [Expression](../syntax.md#syntax-expressions) returning the value in the data type configured for the `attr_name` attribute.
**Returned value**
-- If ClickHouse parses the attribute successfully in the [attribute’s data type](../../query_language/dicts/external_dicts_dict_structure.md#ext_dict_structure-attributes), functions return the value of the dictionary attribute that corresponds to `id_expr`.
+- If ClickHouse parses the attribute successfully in the [attribute’s data type](../../sql_reference/dictionaries/external_dictionaries/external_dicts_dict_structure.md#ext_dict_structure-attributes), functions return the value of the dictionary attribute that corresponds to `id_expr`.
- If there is no the key, corresponding to `id_expr`, in the dictionary, then:
@@ -90,7 +95,7 @@ LIMIT 3
**See Also**
-- [External Dictionaries](../dicts/external_dicts.md)
+- [External Dictionaries](../../sql_reference/dictionaries/external_dictionaries/external_dicts.md)
## dictHas {#dicthas}
@@ -103,7 +108,7 @@ dictHas('dict_name', id_expr)
**Parameters**
- `dict_name` — Name of the dictionary. [String literal](../syntax.md#syntax-string-literal).
-- `id_expr` — Key value. [Expression](../syntax.md#syntax-expressions) returning a [UInt64](../../data_types/int_uint.md)-type value.
+- `id_expr` — Key value. [Expression](../syntax.md#syntax-expressions) returning a [UInt64](../../sql_reference/data_types/int_uint.md)-type value.
**Returned value**
@@ -114,7 +119,7 @@ Type: `UInt8`.
## dictGetHierarchy {#dictgethierarchy}
-Creates an array, containing all the parents of a key in the [hierarchical dictionary](../dicts/external_dicts_dict_hierarchical.md).
+Creates an array, containing all the parents of a key in the [hierarchical dictionary](../../sql_reference/dictionaries/external_dictionaries/external_dicts_dict_hierarchical.md).
**Syntax**
@@ -125,13 +130,13 @@ dictGetHierarchy('dict_name', key)
**Parameters**
- `dict_name` — Name of the dictionary. [String literal](../syntax.md#syntax-string-literal).
-- `key` — Key value. [Expression](../syntax.md#syntax-expressions) returning a [UInt64](../../data_types/int_uint.md)-type value.
+- `key` — Key value. [Expression](../syntax.md#syntax-expressions) returning a [UInt64](../../sql_reference/data_types/int_uint.md)-type value.
**Returned value**
- Parents for the key.
-Type: [Array(UInt64)](../../data_types/array.md).
+Type: [Array(UInt64)](../../sql_reference/data_types/array.md).
## dictIsIn {#dictisin}
@@ -144,8 +149,8 @@ dictIsIn('dict_name', child_id_expr, ancestor_id_expr)
**Parameters**
- `dict_name` — Name of the dictionary. [String literal](../syntax.md#syntax-string-literal).
-- `child_id_expr` — Key to be checked. [Expression](../syntax.md#syntax-expressions) returning a [UInt64](../../data_types/int_uint.md)-type value.
-- `ancestor_id_expr` — Alleged ancestor of the `child_id_expr` key. [Expression](../syntax.md#syntax-expressions) returning a [UInt64](../../data_types/int_uint.md)-type value.
+- `child_id_expr` — Key to be checked. [Expression](../syntax.md#syntax-expressions) returning a [UInt64](../../sql_reference/data_types/int_uint.md)-type value.
+- `ancestor_id_expr` — Alleged ancestor of the `child_id_expr` key. [Expression](../syntax.md#syntax-expressions) returning a [UInt64](../../sql_reference/data_types/int_uint.md)-type value.
**Returned value**
@@ -154,7 +159,7 @@ dictIsIn('dict_name', child_id_expr, ancestor_id_expr)
Type: `UInt8`.
-## Other functions {#ext_dict_functions-other}
+## Other Functions {#ext_dict_functions-other}
ClickHouse supports specialized functions that convert dictionary attribute values to a specific data type regardless of the dictionary configuration.
@@ -181,12 +186,12 @@ dictGet[Type]OrDefault('dict_name', 'attr_name', id_expr, default_value_expr)
- `dict_name` — Name of the dictionary. [String literal](../syntax.md#syntax-string-literal).
- `attr_name` — Name of the column of the dictionary. [String literal](../syntax.md#syntax-string-literal).
-- `id_expr` — Key value. [Expression](../syntax.md#syntax-expressions) returning a [UInt64](../../data_types/int_uint.md)-type value.
+- `id_expr` — Key value. [Expression](../syntax.md#syntax-expressions) returning a [UInt64](../../sql_reference/data_types/int_uint.md)-type value.
- `default_value_expr` — Value which is returned if the dictionary doesn’t contain a row with the `id_expr` key. [Expression](../syntax.md#syntax-expressions) returning a value in the data type configured for the `attr_name` attribute.
**Returned value**
-- If ClickHouse parses the attribute successfully in the [attribute’s data type](../../query_language/dicts/external_dicts_dict_structure.md#ext_dict_structure-attributes), functions return the value of the dictionary attribute that corresponds to `id_expr`.
+- If ClickHouse parses the attribute successfully in the [attribute’s data type](../../sql_reference/dictionaries/external_dictionaries/external_dicts_dict_structure.md#ext_dict_structure-attributes), functions return the value of the dictionary attribute that corresponds to `id_expr`.
- If there is no requested `id_expr` in the dictionary then:
diff --git a/docs/en/query_language/functions/functions_for_nulls.md b/docs/en/sql_reference/functions/functions_for_nulls.md
similarity index 96%
rename from docs/en/query_language/functions/functions_for_nulls.md
rename to docs/en/sql_reference/functions/functions_for_nulls.md
index df6d2a6e98b..4ea67d1e0b5 100644
--- a/docs/en/query_language/functions/functions_for_nulls.md
+++ b/docs/en/sql_reference/functions/functions_for_nulls.md
@@ -1,3 +1,8 @@
+---
+toc_priority: 63
+toc_title: Working with Nullable arguments
+---
+
# Functions for working with Nullable aggregates {#functions-for-working-with-nullable-aggregates}
## isNull {#isnull}
@@ -204,7 +209,7 @@ SELECT nullIf(1, 2)
## assumeNotNull {#assumenotnull}
-Results in a value of type [Nullable](../../data_types/nullable.md) for a non- `Nullable`, if the value is not `NULL`.
+Results in a value of type [Nullable](../../sql_reference/data_types/nullable.md) for a non- `Nullable`, if the value is not `NULL`.
``` sql
assumeNotNull(x)
diff --git a/docs/en/query_language/functions/geo.md b/docs/en/sql_reference/functions/geo.md
similarity index 85%
rename from docs/en/query_language/functions/geo.md
rename to docs/en/sql_reference/functions/geo.md
index a12af7a0dbe..7303821d162 100644
--- a/docs/en/query_language/functions/geo.md
+++ b/docs/en/sql_reference/functions/geo.md
@@ -1,3 +1,8 @@
+---
+toc_priority: 62
+toc_title: Working with geographical coordinates
+---
+
# Functions for Working with Geographical Coordinates {#functions-for-working-with-geographical-coordinates}
## greatCircleDistance {#greatcircledistance}
@@ -78,8 +83,8 @@ pointInPolygon((x, y), [(a, b), (c, d) ...], ...)
**Input values**
-- `(x, y)` — Coordinates of a point on the plane. Data type — [Tuple](../../data_types/tuple.md) — A tuple of two numbers.
-- `[(a, b), (c, d) ...]` — Polygon vertices. Data type — [Array](../../data_types/array.md). Each vertex is represented by a pair of coordinates `(a, b)`. Vertices should be specified in a clockwise or counterclockwise order. The minimum number of vertices is 3. The polygon must be constant.
+- `(x, y)` — Coordinates of a point on the plane. Data type — [Tuple](../../sql_reference/data_types/tuple.md) — A tuple of two numbers.
+- `[(a, b), (c, d) ...]` — Polygon vertices. Data type — [Array](../../sql_reference/data_types/array.md). Each vertex is represented by a pair of coordinates `(a, b)`. Vertices should be specified in a clockwise or counterclockwise order. The minimum number of vertices is 3. The polygon must be constant.
- The function also supports polygons with holes (cut out sections). In this case, add polygons that define the cut out sections using additional arguments of the function. The function does not support non-simply-connected polygons.
**Returned values**
@@ -169,9 +174,9 @@ geoToH3(lon, lat, resolution)
**Parameters**
-- `lon` — Longitude. Type: [Float64](../../data_types/float.md).
-- `lat` — Latitude. Type: [Float64](../../data_types/float.md).
-- `resolution` — Index resolution. Range: `[0, 15]`. Type: [UInt8](../../data_types/int_uint.md).
+- `lon` — Longitude. Type: [Float64](../../sql_reference/data_types/float.md).
+- `lat` — Latitude. Type: [Float64](../../sql_reference/data_types/float.md).
+- `resolution` — Index resolution. Range: `[0, 15]`. Type: [UInt8](../../sql_reference/data_types/int_uint.md).
**Returned values**
@@ -241,11 +246,11 @@ h3GetBaseCell(index)
**Parameters**
-- `index` — Hexagon index number. Type: [UInt64](../../data_types/int_uint.md).
+- `index` — Hexagon index number. Type: [UInt64](../../sql_reference/data_types/int_uint.md).
**Returned values**
-- Hexagon base cell number. Type: [UInt8](../../data_types/int_uint.md).
+- Hexagon base cell number. Type: [UInt8](../../sql_reference/data_types/int_uint.md).
**Example**
@@ -275,11 +280,11 @@ h3HexAreaM2(resolution)
**Parameters**
-- `resolution` — Index resolution. Range: `[0, 15]`. Type: [UInt8](../../data_types/int_uint.md).
+- `resolution` — Index resolution. Range: `[0, 15]`. Type: [UInt8](../../sql_reference/data_types/int_uint.md).
**Returned values**
-- Area in m². Type: [Float64](../../data_types/float.md).
+- Area in m². Type: [Float64](../../sql_reference/data_types/float.md).
**Example**
@@ -309,12 +314,12 @@ h3IndexesAreNeighbors(index1, index2)
**Parameters**
-- `index1` — Hexagon index number. Type: [UInt64](../../data_types/int_uint.md).
-- `index2` — Hexagon index number. Type: [UInt64](../../data_types/int_uint.md).
+- `index1` — Hexagon index number. Type: [UInt64](../../sql_reference/data_types/int_uint.md).
+- `index2` — Hexagon index number. Type: [UInt64](../../sql_reference/data_types/int_uint.md).
**Returned values**
-- Returns `1` if the indexes are neighbors, `0` otherwise. Type: [UInt8](../../data_types/int_uint.md).
+- Returns `1` if the indexes are neighbors, `0` otherwise. Type: [UInt8](../../sql_reference/data_types/int_uint.md).
**Example**
@@ -344,12 +349,12 @@ h3ToChildren(index, resolution)
**Parameters**
-- `index` — Hexagon index number. Type: [UInt64](../../data_types/int_uint.md).
-- `resolution` — Index resolution. Range: `[0, 15]`. Type: [UInt8](../../data_types/int_uint.md).
+- `index` — Hexagon index number. Type: [UInt64](../../sql_reference/data_types/int_uint.md).
+- `resolution` — Index resolution. Range: `[0, 15]`. Type: [UInt8](../../sql_reference/data_types/int_uint.md).
**Returned values**
-- Array with the child H3 indexes. Array of type: [UInt64](../../data_types/int_uint.md).
+- Array with the child H3 indexes. Array of type: [UInt64](../../sql_reference/data_types/int_uint.md).
**Example**
@@ -379,12 +384,12 @@ h3ToParent(index, resolution)
**Parameters**
-- `index` — Hexagon index number. Type: [UInt64](../../data_types/int_uint.md).
-- `resolution` — Index resolution. Range: `[0, 15]`. Type: [UInt8](../../data_types/int_uint.md).
+- `index` — Hexagon index number. Type: [UInt64](../../sql_reference/data_types/int_uint.md).
+- `resolution` — Index resolution. Range: `[0, 15]`. Type: [UInt8](../../sql_reference/data_types/int_uint.md).
**Returned values**
-- Parent H3 index. Type: [UInt64](../../data_types/int_uint.md).
+- Parent H3 index. Type: [UInt64](../../sql_reference/data_types/int_uint.md).
**Example**
@@ -412,11 +417,11 @@ h3ToString(index)
**Parameters**
-- `index` — Hexagon index number. Type: [UInt64](../../data_types/int_uint.md).
+- `index` — Hexagon index number. Type: [UInt64](../../sql_reference/data_types/int_uint.md).
**Returned values**
-- String representation of the H3 index. Type: [String](../../data_types/string.md).
+- String representation of the H3 index. Type: [String](../../sql_reference/data_types/string.md).
**Example**
@@ -444,11 +449,11 @@ stringToH3(index_str)
**Parameters**
-- `index_str` — String representation of the H3 index. Type: [String](../../data_types/string.md).
+- `index_str` — String representation of the H3 index. Type: [String](../../sql_reference/data_types/string.md).
**Returned values**
-- Hexagon index number. Returns 0 on error. Type: [UInt64](../../data_types/int_uint.md).
+- Hexagon index number. Returns 0 on error. Type: [UInt64](../../sql_reference/data_types/int_uint.md).
**Example**
@@ -478,11 +483,11 @@ h3GetResolution(index)
**Parameters**
-- `index` — Hexagon index number. Type: [UInt64](../../data_types/int_uint.md).
+- `index` — Hexagon index number. Type: [UInt64](../../sql_reference/data_types/int_uint.md).
**Returned values**
-- Index resolution. Range: `[0, 15]`. Type: [UInt8](../../data_types/int_uint.md).
+- Index resolution. Range: `[0, 15]`. Type: [UInt8](../../sql_reference/data_types/int_uint.md).
**Example**
diff --git a/docs/en/query_language/functions/hash_functions.md b/docs/en/sql_reference/functions/hash_functions.md
similarity index 84%
rename from docs/en/query_language/functions/hash_functions.md
rename to docs/en/sql_reference/functions/hash_functions.md
index fce95c3d7e1..d6f4bacd9d6 100644
--- a/docs/en/query_language/functions/hash_functions.md
+++ b/docs/en/sql_reference/functions/hash_functions.md
@@ -1,10 +1,15 @@
+---
+toc_priority: 50
+toc_title: Hash
+---
+
# Hash functions {#hash-functions}
Hash functions can be used for the deterministic pseudo-random shuffling of elements.
## halfMD5 {#hash-functions-halfmd5}
-[Interprets](../../query_language/functions/type_conversion_functions.md#type_conversion_functions-reinterpretAsString) all the input parameters as strings and calculates the [MD5](https://en.wikipedia.org/wiki/MD5) hash value for each of them. Then combines hashes, takes the first 8 bytes of the hash of the resulting string, and interprets them as `UInt64` in big-endian byte order.
+[Interprets](../../sql_reference/functions/type_conversion_functions.md#type_conversion_functions-reinterpretAsString) all the input parameters as strings and calculates the [MD5](https://en.wikipedia.org/wiki/MD5) hash value for each of them. Then combines hashes, takes the first 8 bytes of the hash of the resulting string, and interprets them as `UInt64` in big-endian byte order.
``` sql
halfMD5(par1, ...)
@@ -15,11 +20,11 @@ Consider using the [sipHash64](#hash_functions-siphash64) function instead.
**Parameters**
-The function takes a variable number of input parameters. Parameters can be any of the [supported data types](../../data_types/index.md).
+The function takes a variable number of input parameters. Parameters can be any of the [supported data types](../../sql_reference/data_types/index.md).
**Returned Value**
-A [UInt64](../../data_types/int_uint.md) data type hash value.
+A [UInt64](../../sql_reference/data_types/int_uint.md) data type hash value.
**Example**
@@ -49,7 +54,7 @@ sipHash64(par1,...)
This is a cryptographic hash function. It works at least three times faster than the [MD5](#hash_functions-md5) function.
-Function [interprets](../../query_language/functions/type_conversion_functions.md#type_conversion_functions-reinterpretAsString) all the input parameters as strings and calculates the hash value for each of them. Then combines hashes by the following algorithm:
+Function [interprets](../../sql_reference/functions/type_conversion_functions.md#type_conversion_functions-reinterpretAsString) all the input parameters as strings and calculates the hash value for each of them. Then combines hashes by the following algorithm:
1. After hashing all the input parameters, the function gets the array of hashes.
2. Function takes the first and the second elements and calculates a hash for the array of them.
@@ -58,11 +63,11 @@ Function [interprets](../../query_language/functions/type_conversion_functions.m
**Parameters**
-The function takes a variable number of input parameters. Parameters can be any of the [supported data types](../../data_types/index.md).
+The function takes a variable number of input parameters. Parameters can be any of the [supported data types](../../sql_reference/data_types/index.md).
**Returned Value**
-A [UInt64](../../data_types/int_uint.md) data type hash value.
+A [UInt64](../../sql_reference/data_types/int_uint.md) data type hash value.
**Example**
@@ -94,11 +99,11 @@ This is a fast non-cryptographic hash function. It uses the CityHash algorithm f
**Parameters**
-The function takes a variable number of input parameters. Parameters can be any of the [supported data types](../../data_types/index.md).
+The function takes a variable number of input parameters. Parameters can be any of the [supported data types](../../sql_reference/data_types/index.md).
**Returned Value**
-A [UInt64](../../data_types/int_uint.md) data type hash value.
+A [UInt64](../../sql_reference/data_types/int_uint.md) data type hash value.
**Examples**
@@ -160,11 +165,11 @@ The function uses the `Hash64` method from all [available methods](https://githu
**Parameters**
-The function takes a variable number of input parameters. Parameters can be any of the [supported data types](../../data_types/index.md).
+The function takes a variable number of input parameters. Parameters can be any of the [supported data types](../../sql_reference/data_types/index.md).
**Returned Value**
-A [UInt64](../../data_types/int_uint.md) data type hash value.
+A [UInt64](../../sql_reference/data_types/int_uint.md) data type hash value.
**Example**
@@ -286,11 +291,11 @@ metroHash64(par1, ...)
**Parameters**
-The function takes a variable number of input parameters. Parameters can be any of the [supported data types](../../data_types/index.md).
+The function takes a variable number of input parameters. Parameters can be any of the [supported data types](../../sql_reference/data_types/index.md).
**Returned Value**
-A [UInt64](../../data_types/int_uint.md) data type hash value.
+A [UInt64](../../sql_reference/data_types/int_uint.md) data type hash value.
**Example**
@@ -321,12 +326,12 @@ murmurHash2_64(par1, ...)
**Parameters**
-Both functions take a variable number of input parameters. Parameters can be any of the [supported data types](../../data_types/index.md).
+Both functions take a variable number of input parameters. Parameters can be any of the [supported data types](../../sql_reference/data_types/index.md).
**Returned Value**
-- The `murmurHash2_32` function returns hash value having the [UInt32](../../data_types/int_uint.md) data type.
-- The `murmurHash2_64` function returns hash value having the [UInt64](../../data_types/int_uint.md) data type.
+- The `murmurHash2_32` function returns hash value having the [UInt32](../../sql_reference/data_types/int_uint.md) data type.
+- The `murmurHash2_64` function returns hash value having the [UInt64](../../sql_reference/data_types/int_uint.md) data type.
**Example**
@@ -351,12 +356,12 @@ murmurHash3_64(par1, ...)
**Parameters**
-Both functions take a variable number of input parameters. Parameters can be any of the [supported data types](../../data_types/index.md).
+Both functions take a variable number of input parameters. Parameters can be any of the [supported data types](../../sql_reference/data_types/index.md).
**Returned Value**
-- The `murmurHash3_32` function returns a [UInt32](../../data_types/int_uint.md) data type hash value.
-- The `murmurHash3_64` function returns a [UInt64](../../data_types/int_uint.md) data type hash value.
+- The `murmurHash3_32` function returns a [UInt32](../../sql_reference/data_types/int_uint.md) data type hash value.
+- The `murmurHash3_64` function returns a [UInt64](../../sql_reference/data_types/int_uint.md) data type hash value.
**Example**
@@ -380,11 +385,11 @@ murmurHash3_128( expr )
**Parameters**
-- `expr` — [Expressions](../syntax.md#syntax-expressions) returning a [String](../../data_types/string.md)-type value.
+- `expr` — [Expressions](../syntax.md#syntax-expressions) returning a [String](../../sql_reference/data_types/string.md)-type value.
**Returned Value**
-A [FixedString(16)](../../data_types/fixedstring.md) data type hash value.
+A [FixedString(16)](../../sql_reference/data_types/fixedstring.md) data type hash value.
**Example**
diff --git a/docs/en/query_language/functions/higher_order_functions.md b/docs/en/sql_reference/functions/higher_order_functions.md
similarity index 99%
rename from docs/en/query_language/functions/higher_order_functions.md
rename to docs/en/sql_reference/functions/higher_order_functions.md
index e58528a8f7f..99d2d8ea885 100644
--- a/docs/en/query_language/functions/higher_order_functions.md
+++ b/docs/en/sql_reference/functions/higher_order_functions.md
@@ -1,3 +1,8 @@
+---
+toc_priority: 57
+toc_title: Higher-Order
+---
+
# Higher-order functions {#higher-order-functions}
## `->` operator, lambda(params, expr) function {#operator-lambdaparams-expr-function}
diff --git a/docs/en/query_language/functions/in_functions.md b/docs/en/sql_reference/functions/in_functions.md
similarity index 82%
rename from docs/en/query_language/functions/in_functions.md
rename to docs/en/sql_reference/functions/in_functions.md
index 806ce20ef2c..b8ff809b276 100644
--- a/docs/en/query_language/functions/in_functions.md
+++ b/docs/en/sql_reference/functions/in_functions.md
@@ -1,8 +1,13 @@
-# Functions for implementing the IN operator {#functions-for-implementing-the-in-operator}
+---
+toc_priority: 60
+toc_title: Implementing the IN Operator
+---
+
+# Functions For Implementing the IN Operator {#functions-for-implementing-the-in-operator}
## in, notIn, globalIn, globalNotIn {#in-functions}
-See the section [IN operators](../select.md#select-in-operators).
+See the section [IN operators](../statements/select.md#select-in-operators).
## tuple(x, y, …), operator (x, y, …) {#tuplex-y-operator-x-y}
diff --git a/docs/en/query_language/functions/index.md b/docs/en/sql_reference/functions/index.md
similarity index 98%
rename from docs/en/query_language/functions/index.md
rename to docs/en/sql_reference/functions/index.md
index c52e56ffdb1..a69c1ec1848 100644
--- a/docs/en/query_language/functions/index.md
+++ b/docs/en/sql_reference/functions/index.md
@@ -1,3 +1,9 @@
+---
+toc_folder_title: Functions
+toc_priority: 32
+toc_title: Introduction
+---
+
# Functions {#functions}
There are at least\* two types of functions - regular functions (they are just called “functions”) and aggregate functions. These are completely different concepts. Regular functions work as if they are applied to each row separately (for each row, the result of the function doesn’t depend on the other rows). Aggregate functions accumulate a set of values from various rows (i.e. they depend on the entire set of rows).
diff --git a/docs/en/query_language/functions/introspection.md b/docs/en/sql_reference/functions/introspection.md
similarity index 94%
rename from docs/en/query_language/functions/introspection.md
rename to docs/en/sql_reference/functions/introspection.md
index 5193d3a85a7..008cc2feecf 100644
--- a/docs/en/query_language/functions/introspection.md
+++ b/docs/en/sql_reference/functions/introspection.md
@@ -1,3 +1,8 @@
+---
+toc_priority: 65
+toc_title: Introspection
+---
+
# Introspection Functions {#introspection-functions}
You can use functions described in this chapter to introspect [ELF](https://en.wikipedia.org/wiki/Executable_and_Linkable_Format) and [DWARF](https://en.wikipedia.org/wiki/DWARF) for query profiling.
@@ -29,7 +34,7 @@ addressToLine(address_of_binary_instruction)
**Parameters**
-- `address_of_binary_instruction` ([UInt64](../../data_types/int_uint.md)) — Address of instruction in a running process.
+- `address_of_binary_instruction` ([UInt64](../../sql_reference/data_types/int_uint.md)) — Address of instruction in a running process.
**Returned value**
@@ -41,7 +46,7 @@ addressToLine(address_of_binary_instruction)
- Empty string, if the address is not valid.
-Type: [String](../../data_types/string.md).
+Type: [String](../../sql_reference/data_types/string.md).
**Example**
@@ -120,14 +125,14 @@ addressToSymbol(address_of_binary_instruction)
**Parameters**
-- `address_of_binary_instruction` ([UInt64](../../data_types/int_uint.md)) — Address of instruction in a running process.
+- `address_of_binary_instruction` ([UInt64](../../sql_reference/data_types/int_uint.md)) — Address of instruction in a running process.
**Returned value**
- Symbol from ClickHouse object files.
- Empty string, if the address is not valid.
-Type: [String](../../data_types/string.md).
+Type: [String](../../sql_reference/data_types/string.md).
**Example**
@@ -217,14 +222,14 @@ demangle(symbol)
**Parameters**
-- `symbol` ([String](../../data_types/string.md)) — Symbol from an object file.
+- `symbol` ([String](../../sql_reference/data_types/string.md)) — Symbol from an object file.
**Returned value**
- Name of the C++ function.
- Empty string if a symbol is not valid.
-Type: [String](../../data_types/string.md).
+Type: [String](../../sql_reference/data_types/string.md).
**Example**
diff --git a/docs/en/query_language/functions/ip_address_functions.md b/docs/en/sql_reference/functions/ip_address_functions.md
similarity index 96%
rename from docs/en/query_language/functions/ip_address_functions.md
rename to docs/en/sql_reference/functions/ip_address_functions.md
index 91a896cbf8a..a7186a5d067 100644
--- a/docs/en/query_language/functions/ip_address_functions.md
+++ b/docs/en/sql_reference/functions/ip_address_functions.md
@@ -1,3 +1,8 @@
+---
+toc_priority: 55
+toc_title: Working with IP Addresses
+---
+
# Functions for working with IP addresses {#functions-for-working-with-ip-addresses}
## IPv4NumToString(num) {#ipv4numtostringnum}
@@ -176,7 +181,7 @@ SELECT IPv6CIDRToRange(toIPv6('2001:0db8:0000:85a3:0000:0000:ac1f:8001'), 32);
## toIPv4(string) {#toipv4string}
-An alias to `IPv4StringToNum()` that takes a string form of IPv4 address and returns value of [IPv4](../../data_types/domains/ipv4.md) type, which is binary equal to value returned by `IPv4StringToNum()`.
+An alias to `IPv4StringToNum()` that takes a string form of IPv4 address and returns value of [IPv4](../../sql_reference/data_types/domains/ipv4.md) type, which is binary equal to value returned by `IPv4StringToNum()`.
``` sql
WITH
@@ -208,7 +213,7 @@ SELECT
## toIPv6(string) {#toipv6string}
-An alias to `IPv6StringToNum()` that takes a string form of IPv6 address and returns value of [IPv6](../../data_types/domains/ipv6.md) type, which is binary equal to value returned by `IPv6StringToNum()`.
+An alias to `IPv6StringToNum()` that takes a string form of IPv6 address and returns value of [IPv6](../../sql_reference/data_types/domains/ipv6.md) type, which is binary equal to value returned by `IPv6StringToNum()`.
``` sql
WITH
diff --git a/docs/en/query_language/functions/json_functions.md b/docs/en/sql_reference/functions/json_functions.md
similarity index 99%
rename from docs/en/query_language/functions/json_functions.md
rename to docs/en/sql_reference/functions/json_functions.md
index 14098bc72a7..f1a6b6cc44a 100644
--- a/docs/en/query_language/functions/json_functions.md
+++ b/docs/en/sql_reference/functions/json_functions.md
@@ -1,3 +1,8 @@
+---
+toc_priority: 56
+toc_title: Working with JSON.
+---
+
# Functions for working with JSON {#functions-for-working-with-json}
In Yandex.Metrica, JSON is transmitted by users as session parameters. There are some special functions for working with this JSON. (Although in most of the cases, the JSONs are additionally pre-processed, and the resulting values are put in separate columns in their processed format.) All these functions are based on strong assumptions about what the JSON can be, but they try to do as little as possible to get the job done.
diff --git a/docs/en/query_language/functions/logical_functions.md b/docs/en/sql_reference/functions/logical_functions.md
similarity index 91%
rename from docs/en/query_language/functions/logical_functions.md
rename to docs/en/sql_reference/functions/logical_functions.md
index a1d9c41ec7e..c1aa30775c3 100644
--- a/docs/en/query_language/functions/logical_functions.md
+++ b/docs/en/sql_reference/functions/logical_functions.md
@@ -1,3 +1,8 @@
+---
+toc_priority: 37
+toc_title: Logical
+---
+
# Logical functions {#logical-functions}
Logical functions accept any numeric types, but return a UInt8 number equal to 0 or 1.
diff --git a/docs/en/sql_reference/functions/machine_learning_functions.md b/docs/en/sql_reference/functions/machine_learning_functions.md
new file mode 100644
index 00000000000..4ff4e592e7b
--- /dev/null
+++ b/docs/en/sql_reference/functions/machine_learning_functions.md
@@ -0,0 +1,18 @@
+---
+toc_priority: 64
+toc_title: Machine Learning Functions
+---
+
+# Machine Learning Functions {#machine-learning-functions}
+
+## evalMLMethod (prediction) {#machine_learning_methods-evalmlmethod}
+
+Prediction using fitted regression models uses `evalMLMethod` function. See link in `linearRegression`.
+
+### Stochastic Linear Regression {#stochastic-linear-regression}
+
+The [stochasticLinearRegression](../../sql_reference/aggregate_functions/reference.md#agg_functions-stochasticlinearregression) aggregate function implements stochastic gradient descent method using linear model and MSE loss function. Uses `evalMLMethod` to predict on new data.
+
+### Stochastic Logistic Regression {#stochastic-logistic-regression}
+
+The [stochasticLogisticRegression](../../sql_reference/aggregate_functions/reference.md#agg_functions-stochasticlogisticregression) aggregate function implements stochastic gradient descent method for binary classification problem. Uses `evalMLMethod` to predict on new data.
diff --git a/docs/en/query_language/functions/math_functions.md b/docs/en/sql_reference/functions/math_functions.md
similarity index 98%
rename from docs/en/query_language/functions/math_functions.md
rename to docs/en/sql_reference/functions/math_functions.md
index cc89d22d786..3f0f8623840 100644
--- a/docs/en/query_language/functions/math_functions.md
+++ b/docs/en/sql_reference/functions/math_functions.md
@@ -1,3 +1,8 @@
+---
+toc_priority: 44
+toc_title: Mathematical
+---
+
# Mathematical functions {#mathematical-functions}
All the functions return a Float64 number. The accuracy of the result is close to the maximum precision possible, but the result might not coincide with the machine representable number nearest to the corresponding real number.
diff --git a/docs/en/query_language/functions/other_functions.md b/docs/en/sql_reference/functions/other_functions.md
similarity index 96%
rename from docs/en/query_language/functions/other_functions.md
rename to docs/en/sql_reference/functions/other_functions.md
index 2f7a50eab0c..843897c21b2 100644
--- a/docs/en/query_language/functions/other_functions.md
+++ b/docs/en/sql_reference/functions/other_functions.md
@@ -1,3 +1,8 @@
+---
+toc_priority: 66
+toc_title: Other
+---
+
# Other functions {#other-functions}
## hostName() {#hostname}
@@ -48,7 +53,7 @@ basename( expr )
**Parameters**
-- `expr` — Expression resulting in a [String](../../data_types/string.md) type value. All the backslashes must be escaped in the resulting value.
+- `expr` — Expression resulting in a [String](../../sql_reference/data_types/string.md) type value. All the backslashes must be escaped in the resulting value.
**Returned Value**
@@ -194,8 +199,8 @@ Checks whether floating point value is finite.
**Parameters**
-- `x` — Value to be checked for infinity. Type: [Float\*](../../data_types/float.md).
-- `y` — Fallback value. Type: [Float\*](../../data_types/float.md).
+- `x` — Value to be checked for infinity. Type: [Float\*](../../sql_reference/data_types/float.md).
+- `y` — Fallback value. Type: [Float\*](../../sql_reference/data_types/float.md).
**Returned value**
@@ -432,7 +437,7 @@ If you make a subquery with ORDER BY and call the function from outside the subq
**Parameters**
- `column` — A column name or scalar expression.
-- `offset` — The number of rows forwards or backwards from the current row of `column`. [Int64](../../data_types/int_uint.md).
+- `offset` — The number of rows forwards or backwards from the current row of `column`. [Int64](../../sql_reference/data_types/int_uint.md).
- `default_value` — Optional. The value to be returned if offset goes beyond the scope of the block. Type of data blocks affected.
**Returned values**
@@ -617,7 +622,7 @@ Accepts a MAC address in the format AA:BB:CC:DD:EE:FF (colon-separated numbers i
## getSizeOfEnumType {#getsizeofenumtype}
-Returns the number of fields in [Enum](../../data_types/enum.md).
+Returns the number of fields in [Enum](../../sql_reference/data_types/enum.md).
``` sql
getSizeOfEnumType(value)
@@ -758,7 +763,7 @@ defaultValueOfArgumentType(expression)
- `0` for numbers.
- Empty string for strings.
-- `ᴺᵁᴸᴸ` for [Nullable](../../data_types/nullable.md).
+- `ᴺᵁᴸᴸ` for [Nullable](../../sql_reference/data_types/nullable.md).
**Example**
@@ -833,7 +838,7 @@ filesystemAvailable()
- The amount of remaining space available in bytes.
-Type: [UInt64](../../data_types/int_uint.md).
+Type: [UInt64](../../sql_reference/data_types/int_uint.md).
**Example**
@@ -865,7 +870,7 @@ filesystemFree()
- Amount of free space in bytes.
-Type: [UInt64](../../data_types/int_uint.md).
+Type: [UInt64](../../sql_reference/data_types/int_uint.md).
**Example**
@@ -885,7 +890,7 @@ Result:
## filesystemCapacity {#filesystemcapacity}
-Returns the capacity of the filesystem in bytes. For evaluation, the [path](../../operations/server_settings/settings.md#server_settings-path) to the data directory must be configured.
+Returns the capacity of the filesystem in bytes. For evaluation, the [path](../../operations/server_configuration_parameters/settings.md#server_configuration_parameters-path) to the data directory must be configured.
**Syntax**
@@ -897,7 +902,7 @@ filesystemCapacity()
- Capacity information of the filesystem in bytes.
-Type: [UInt64](../../data_types/int_uint.md).
+Type: [UInt64](../../sql_reference/data_types/int_uint.md).
**Example**
@@ -927,9 +932,9 @@ So, result of function depends on partition of data to blocks and on order of da
## joinGet {#joinget}
-The function lets you extract data from the table the same way as from a [dictionary](../../query_language/dicts/index.md).
+The function lets you extract data from the table the same way as from a [dictionary](../../sql_reference/dictionaries/index.md).
-Gets data from [Join](../../operations/table_engines/join.md#creating-a-table) tables using the specified join key.
+Gets data from [Join](../../engines/table_engines/special/join.md#creating-a-table) tables using the specified join key.
Only supports tables created with the `ENGINE = Join(ANY, LEFT, )` statement.
@@ -951,7 +956,7 @@ Returns list of values corresponded to list of keys.
If certain doesn’t exist in source table then `0` or `null` will be returned based on [join\_use\_nulls](../../operations/settings/settings.md#join_use_nulls) setting.
-More info about `join_use_nulls` in [Join operation](../../operations/table_engines/join.md).
+More info about `join_use_nulls` in [Join operation](../../engines/table_engines/special/join.md).
**Example**
@@ -1053,7 +1058,7 @@ randomPrintableASCII(length)
- String with a random set of [ASCII](https://en.wikipedia.org/wiki/ASCII#Printable_characters) printable characters.
-Type: [String](../../data_types/string.md)
+Type: [String](../../sql_reference/data_types/string.md)
**Example**
diff --git a/docs/en/query_language/functions/random_functions.md b/docs/en/sql_reference/functions/random_functions.md
similarity index 93%
rename from docs/en/query_language/functions/random_functions.md
rename to docs/en/sql_reference/functions/random_functions.md
index 744eca48a01..7990d3b6cc7 100644
--- a/docs/en/query_language/functions/random_functions.md
+++ b/docs/en/sql_reference/functions/random_functions.md
@@ -1,3 +1,8 @@
+---
+toc_priority: 51
+toc_title: Generating Pseudo-Random Numbers
+---
+
# Functions for generating pseudo-random numbers {#functions-for-generating-pseudo-random-numbers}
Non-cryptographic generators of pseudo-random numbers are used.
diff --git a/docs/en/query_language/functions/rounding_functions.md b/docs/en/sql_reference/functions/rounding_functions.md
similarity index 97%
rename from docs/en/query_language/functions/rounding_functions.md
rename to docs/en/sql_reference/functions/rounding_functions.md
index 2d0afdcb637..ce3c592acc6 100644
--- a/docs/en/query_language/functions/rounding_functions.md
+++ b/docs/en/sql_reference/functions/rounding_functions.md
@@ -1,3 +1,8 @@
+---
+toc_priority: 45
+toc_title: Rounding
+---
+
# Rounding functions {#rounding-functions}
## floor(x\[, N\]) {#floorx-n}
@@ -32,7 +37,7 @@ round(expression [, decimal_places])
**Parameters:**
-- `expression` — A number to be rounded. Can be any [expression](../syntax.md#syntax-expressions) returning the numeric [data type](../../data_types/index.md#data_types).
+- `expression` — A number to be rounded. Can be any [expression](../syntax.md#syntax-expressions) returning the numeric [data type](../../sql_reference/data_types/index.md#data_types).
- `decimal-places` — An integer value.
- If `decimal-places > 0` then the function rounds the value to the right of the decimal point.
- If `decimal-places < 0` then the function rounds the value to the left of the decimal point.
@@ -111,7 +116,7 @@ roundBankers(expression [, decimal_places])
**Parameters**
-- `expression` — A number to be rounded. Can be any [expression](../syntax.md#syntax-expressions) returning the numeric [data type](../../data_types/index.md#data_types).
+- `expression` — A number to be rounded. Can be any [expression](../syntax.md#syntax-expressions) returning the numeric [data type](../../sql_reference/data_types/index.md#data_types).
- `decimal-places` — Decimal places. An integer number.
- `decimal-places > 0` — The function rounds the number to the given position right of the decimal point. Example: `roundBankers(3.55, 1) = 3.6`.
- `decimal-places < 0` — The function rounds the number to the given position left of the decimal point. Example: `roundBankers(24.55, -1) = 20`.
diff --git a/docs/en/query_language/functions/splitting_merging_functions.md b/docs/en/sql_reference/functions/splitting_merging_functions.md
similarity index 83%
rename from docs/en/query_language/functions/splitting_merging_functions.md
rename to docs/en/sql_reference/functions/splitting_merging_functions.md
index b91cdd4ed59..9e455440293 100644
--- a/docs/en/query_language/functions/splitting_merging_functions.md
+++ b/docs/en/sql_reference/functions/splitting_merging_functions.md
@@ -1,3 +1,8 @@
+---
+toc_priority: 47
+toc_title: Splitting and Merging Strings and Arrays
+---
+
# Functions for splitting and merging strings and arrays {#functions-for-splitting-and-merging-strings-and-arrays}
## splitByChar(separator, s) {#splitbycharseparator-s}
@@ -13,8 +18,8 @@ splitByChar(, )
**Parameters**
-- `separator` — The separator which should contain exactly one character. [String](../../data_types/string.md).
-- `s` — The string to split. [String](../../data_types/string.md).
+- `separator` — The separator which should contain exactly one character. [String](../../sql_reference/data_types/string.md).
+- `s` — The string to split. [String](../../sql_reference/data_types/string.md).
**Returned value(s)**
@@ -24,7 +29,7 @@ Returns an array of selected substrings. Empty substrings may be selected when:
* There are multiple consecutive separators;
* The original string `s` is empty.
-Type: [Array](../../data_types/array.md) of [String](../../data_types/string.md).
+Type: [Array](../../sql_reference/data_types/array.md) of [String](../../sql_reference/data_types/string.md).
**Example**
@@ -50,14 +55,14 @@ splitByString(, )
**Parameters**
-- `separator` — The separator. [String](../../data_types/string.md).
-- `s` — The string to split. [String](../../data_types/string.md).
+- `separator` — The separator. [String](../../sql_reference/data_types/string.md).
+- `s` — The string to split. [String](../../sql_reference/data_types/string.md).
**Returned value(s)**
Returns an array of selected substrings. Empty substrings may be selected when:
-Type: [Array](../../data_types/array.md) of [String](../../data_types/string.md).
+Type: [Array](../../sql_reference/data_types/array.md) of [String](../../sql_reference/data_types/string.md).
* A non-empty separator occurs at the beginning or end of the string;
* There are multiple consecutive non-empty separators;
diff --git a/docs/en/query_language/functions/string_functions.md b/docs/en/sql_reference/functions/string_functions.md
similarity index 95%
rename from docs/en/query_language/functions/string_functions.md
rename to docs/en/sql_reference/functions/string_functions.md
index b17b4e8d801..5619f036d52 100644
--- a/docs/en/query_language/functions/string_functions.md
+++ b/docs/en/sql_reference/functions/string_functions.md
@@ -1,3 +1,8 @@
+---
+toc_priority: 40
+toc_title: Working with strings
+---
+
# Functions for working with strings {#functions-for-working-with-strings}
## empty {#empty}
@@ -70,7 +75,7 @@ toValidUTF8( input_string )
Parameters:
-- input\_string — Any set of bytes represented as the [String](../../data_types/string.md) data type object.
+- input\_string — Any set of bytes represented as the [String](../../sql_reference/data_types/string.md) data type object.
Returned value: Valid UTF-8 string.
@@ -98,8 +103,8 @@ repeat(s, n)
**Parameters**
-- `s` — The string to repeat. [String](../../data_types/string.md).
-- `n` — The number of times to repeat the string. [UInt](../../data_types/int_uint.md).
+- `s` — The string to repeat. [String](../../sql_reference/data_types/string.md).
+- `n` — The number of times to repeat the string. [UInt](../../sql_reference/data_types/int_uint.md).
**Returned value**
@@ -322,8 +327,8 @@ trim([[LEADING|TRAILING|BOTH] trim_character FROM] input_string)
**Parameters**
-- `trim_character` — specified characters for trim. [String](../../data_types/string.md).
-- `input_string` — string for trim. [String](../../data_types/string.md).
+- `trim_character` — specified characters for trim. [String](../../sql_reference/data_types/string.md).
+- `input_string` — string for trim. [String](../../sql_reference/data_types/string.md).
**Returned value**
@@ -361,7 +366,7 @@ Alias: `ltrim(input_string)`.
**Parameters**
-- `input_string` — string to trim. [String](../../data_types/string.md).
+- `input_string` — string to trim. [String](../../sql_reference/data_types/string.md).
**Returned value**
@@ -399,7 +404,7 @@ Alias: `rtrim(input_string)`.
**Parameters**
-- `input_string` — string to trim. [String](../../data_types/string.md).
+- `input_string` — string to trim. [String](../../sql_reference/data_types/string.md).
**Returned value**
@@ -437,7 +442,7 @@ Alias: `trim(input_string)`.
**Parameters**
-- `input_string` — string to trim. [String](../../data_types/string.md).
+- `input_string` — string to trim. [String](../../sql_reference/data_types/string.md).
**Returned value**
diff --git a/docs/en/query_language/functions/string_replace_functions.md b/docs/en/sql_reference/functions/string_replace_functions.md
similarity index 98%
rename from docs/en/query_language/functions/string_replace_functions.md
rename to docs/en/sql_reference/functions/string_replace_functions.md
index c0709fd3090..1238937d7c0 100644
--- a/docs/en/query_language/functions/string_replace_functions.md
+++ b/docs/en/sql_reference/functions/string_replace_functions.md
@@ -1,3 +1,8 @@
+---
+toc_priority: 42
+toc_title: For Replacing in Strings
+---
+
# Functions for searching and replacing in strings {#functions-for-searching-and-replacing-in-strings}
## replaceOne(haystack, pattern, replacement) {#replaceonehaystack-pattern-replacement}
diff --git a/docs/en/query_language/functions/string_search_functions.md b/docs/en/sql_reference/functions/string_search_functions.md
similarity index 99%
rename from docs/en/query_language/functions/string_search_functions.md
rename to docs/en/sql_reference/functions/string_search_functions.md
index a84899220ef..180830c0842 100644
--- a/docs/en/query_language/functions/string_search_functions.md
+++ b/docs/en/sql_reference/functions/string_search_functions.md
@@ -1,3 +1,8 @@
+---
+toc_priority: 41
+toc_title: For Searching Strings
+---
+
# Functions for Searching Strings {#functions-for-searching-strings}
The search is case-sensitive by default in all these functions. There are separate variants for case insensitive search.
diff --git a/docs/en/query_language/functions/type_conversion_functions.md b/docs/en/sql_reference/functions/type_conversion_functions.md
similarity index 89%
rename from docs/en/query_language/functions/type_conversion_functions.md
rename to docs/en/sql_reference/functions/type_conversion_functions.md
index 40e0e3cc446..a61e1e692d8 100644
--- a/docs/en/query_language/functions/type_conversion_functions.md
+++ b/docs/en/sql_reference/functions/type_conversion_functions.md
@@ -1,3 +1,8 @@
+---
+toc_priority: 38
+toc_title: Type Conversion
+---
+
# Type Conversion Functions {#type-conversion-functions}
## Common Issues of Numeric Conversions {#numeric-conversion-issues}
@@ -8,7 +13,7 @@ ClickHouse has the [same behavior as C++ programs](https://en.cppreference.com/w
## toInt(8\|16\|32\|64) {#toint8163264}
-Converts an input value to the [Int](../../data_types/int_uint.md) data type. This function family includes:
+Converts an input value to the [Int](../../sql_reference/data_types/int_uint.md) data type. This function family includes:
- `toInt8(expr)` — Results in the `Int8` data type.
- `toInt16(expr)` — Results in the `Int16` data type.
@@ -25,7 +30,7 @@ Integer value in the `Int8`, `Int16`, `Int32`, or `Int64` data type.
Functions use [rounding towards zero](https://en.wikipedia.org/wiki/Rounding#Rounding_towards_zero), meaning they truncate fractional digits of numbers.
-The behavior of functions for the [NaN and Inf](../../data_types/float.md#data_type-float-nan-inf) arguments is undefined. Remember about [numeric convertions issues](#numeric-conversion-issues), when using the functions.
+The behavior of functions for the [NaN and Inf](../../sql_reference/data_types/float.md#data_type-float-nan-inf) arguments is undefined. Remember about [numeric convertions issues](#numeric-conversion-issues), when using the functions.
**Example**
@@ -73,7 +78,7 @@ select toInt64OrNull('123123'), toInt8OrNull('123qwe123')
## toUInt(8\|16\|32\|64) {#touint8163264}
-Converts an input value to the [UInt](../../data_types/int_uint.md) data type. This function family includes:
+Converts an input value to the [UInt](../../sql_reference/data_types/int_uint.md) data type. This function family includes:
- `toUInt8(expr)` — Results in the `UInt8` data type.
- `toUInt16(expr)` — Results in the `UInt16` data type.
@@ -90,7 +95,7 @@ Integer value in the `UInt8`, `UInt16`, `UInt32`, or `UInt64` data type.
Functions use [rounding towards zero](https://en.wikipedia.org/wiki/Rounding#Rounding_towards_zero), meaning they truncate fractional digits of numbers.
-The behavior of functions for negative agruments and for the [NaN and Inf](../../data_types/float.md#data_type-float-nan-inf) arguments is undefined. If you pass a string with a negative number, for example `'-32'`, ClickHouse raises an exception. Remember about [numeric convertions issues](#numeric-conversion-issues), when using the functions.
+The behavior of functions for negative agruments and for the [NaN and Inf](../../sql_reference/data_types/float.md#data_type-float-nan-inf) arguments is undefined. If you pass a string with a negative number, for example `'-32'`, ClickHouse raises an exception. Remember about [numeric convertions issues](#numeric-conversion-issues), when using the functions.
**Example**
@@ -128,7 +133,7 @@ SELECT toUInt64(nan), toUInt32(-32), toUInt16('16'), toUInt8(8.8)
## toDecimal(32\|64\|128) {#todecimal3264128}
-Converts `value` to the [Decimal](../../data_types/decimal.md) data type with precision of `S`. The `value` can be a number or a string. The `S` (scale) parameter specifies the number of decimal places.
+Converts `value` to the [Decimal](../../sql_reference/data_types/decimal.md) data type with precision of `S`. The `value` can be a number or a string. The `S` (scale) parameter specifies the number of decimal places.
- `toDecimal32(value, S)`
- `toDecimal64(value, S)`
@@ -136,7 +141,7 @@ Converts `value` to the [Decimal](../../data_types/decimal.md) data type with pr
## toDecimal(32\|64\|128)OrNull {#todecimal3264128ornull}
-Converts an input string to a [Nullable(Decimal(P,S))](../../data_types/decimal.md) data type value. This family of functions include:
+Converts an input string to a [Nullable(Decimal(P,S))](../../sql_reference/data_types/decimal.md) data type value. This family of functions include:
- `toDecimal32OrNull(expr, S)` — Results in `Nullable(Decimal32(S))` data type.
- `toDecimal64OrNull(expr, S)` — Results in `Nullable(Decimal64(S))` data type.
@@ -146,7 +151,7 @@ These functions should be used instead of `toDecimal*()` functions, if you prefe
**Parameters**
-- `expr` — [Expression](../syntax.md#syntax-expressions), returns a value in the [String](../../data_types/string.md) data type. ClickHouse expects the textual representation of the decimal number. For example, `'1.111'`.
+- `expr` — [Expression](../syntax.md#syntax-expressions), returns a value in the [String](../../sql_reference/data_types/string.md) data type. ClickHouse expects the textual representation of the decimal number. For example, `'1.111'`.
- `S` — Scale, the number of decimal places in the resulting value.
**Returned value**
@@ -180,7 +185,7 @@ SELECT toDecimal32OrNull(toString(-1.111), 2) AS val, toTypeName(val)
## toDecimal(32\|64\|128)OrZero {#todecimal3264128orzero}
-Converts an input value to the [Decimal(P,S)](../../data_types/decimal.md) data type. This family of functions include:
+Converts an input value to the [Decimal(P,S)](../../sql_reference/data_types/decimal.md) data type. This family of functions include:
- `toDecimal32OrZero( expr, S)` — Results in `Decimal32(S)` data type.
- `toDecimal64OrZero( expr, S)` — Results in `Decimal64(S)` data type.
@@ -190,7 +195,7 @@ These functions should be used instead of `toDecimal*()` functions, if you prefe
**Parameters**
-- `expr` — [Expression](../syntax.md#syntax-expressions), returns a value in the [String](../../data_types/string.md) data type. ClickHouse expects the textual representation of the decimal number. For example, `'1.111'`.
+- `expr` — [Expression](../syntax.md#syntax-expressions), returns a value in the [String](../../sql_reference/data_types/string.md) data type. ClickHouse expects the textual representation of the decimal number. For example, `'1.111'`.
- `S` — Scale, the number of decimal places in the resulting value.
**Returned value**
@@ -335,7 +340,7 @@ SELECT
Conversion to FixedString(N) only works for arguments of type String or FixedString(N).
-Type conversion to [Nullable](../../data_types/nullable.md) and back is supported. Example:
+Type conversion to [Nullable](../../sql_reference/data_types/nullable.md) and back is supported. Example:
``` sql
SELECT toTypeName(x) FROM t_null
@@ -361,7 +366,7 @@ SELECT toTypeName(CAST(x, 'Nullable(UInt16)')) FROM t_null
## toInterval(Year\|Quarter\|Month\|Week\|Day\|Hour\|Minute\|Second) {#function-tointerval}
-Converts a Number type argument to an [Interval](../../data_types/special_data_types/interval.md) data type.
+Converts a Number type argument to an [Interval](../../sql_reference/data_types/special_data_types/interval.md) data type.
**Syntax**
@@ -404,7 +409,7 @@ SELECT
## parseDateTimeBestEffort {#parsedatetimebesteffort}
-Converts a date and time in the [String](../../data_types/string.md) representation to [DateTime](../../data_types/datetime.md#data_type-datetime) data type.
+Converts a date and time in the [String](../../sql_reference/data_types/string.md) representation to [DateTime](../../sql_reference/data_types/datetime.md#data_type-datetime) data type.
The function parses [ISO 8601](https://en.wikipedia.org/wiki/ISO_8601), [RFC 1123 - 5.2.14 RFC-822 Date and Time Specification](https://tools.ietf.org/html/rfc1123#page-55), ClickHouse's and some other date and time formats.
@@ -417,8 +422,8 @@ parseDateTimeBestEffort(time_string [, time_zone]);
**Parameters**
-- `time_string` — String containing a date and time to convert. [String](../../data_types/string.md).
-- `time_zone` — Time zone. The function parses `time_string` according to the time zone. [String](../../data_types/string.md).
+- `time_string` — String containing a date and time to convert. [String](../../sql_reference/data_types/string.md).
+- `time_zone` — Time zone. The function parses `time_string` according to the time zone. [String](../../sql_reference/data_types/string.md).
**Supported non-standard formats**
diff --git a/docs/en/query_language/functions/url_functions.md b/docs/en/sql_reference/functions/url_functions.md
similarity index 97%
rename from docs/en/query_language/functions/url_functions.md
rename to docs/en/sql_reference/functions/url_functions.md
index ee50196307f..205fb4bc068 100644
--- a/docs/en/query_language/functions/url_functions.md
+++ b/docs/en/sql_reference/functions/url_functions.md
@@ -1,3 +1,8 @@
+---
+toc_priority: 54
+toc_title: Working with URLs
+---
+
# Functions for working with URLs {#functions-for-working-with-urls}
All these functions don’t follow the RFC. They are maximally simplified for improved performance.
@@ -22,7 +27,7 @@ domain(url)
**Parameters**
-- `url` — URL. Type: [String](../../data_types/string.md).
+- `url` — URL. Type: [String](../../sql_reference/data_types/string.md).
The URL can be specified with or without a scheme. Examples:
@@ -73,7 +78,7 @@ topLevelDomain(url)
**Parameters**
-- `url` — URL. Type: [String](../../data_types/string.md).
+- `url` — URL. Type: [String](../../sql_reference/data_types/string.md).
The URL can be specified with or without a scheme. Examples:
diff --git a/docs/en/query_language/functions/uuid_functions.md b/docs/en/sql_reference/functions/uuid_functions.md
similarity index 88%
rename from docs/en/query_language/functions/uuid_functions.md
rename to docs/en/sql_reference/functions/uuid_functions.md
index 1a5a3fa79fc..9cda8f44b13 100644
--- a/docs/en/query_language/functions/uuid_functions.md
+++ b/docs/en/sql_reference/functions/uuid_functions.md
@@ -1,10 +1,15 @@
+---
+toc_priority: 53
+toc_title: Working with UUID
+---
+
# Functions for working with UUID {#functions-for-working-with-uuid}
The functions for working with UUID are listed below.
## generateUUIDv4 {#uuid-function-generate}
-Generates the [UUID](../../data_types/uuid.md) of [version 4](https://tools.ietf.org/html/rfc4122#section-4.4).
+Generates the [UUID](../../sql_reference/data_types/uuid.md) of [version 4](https://tools.ietf.org/html/rfc4122#section-4.4).
``` sql
generateUUIDv4()
@@ -58,7 +63,7 @@ SELECT toUUID('61f0c404-5cb3-11e7-907b-a6006ad3dba0') AS uuid
## UUIDStringToNum {#uuidstringtonum}
-Accepts a string containing 36 characters in the format `xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx`, and returns it as a set of bytes in a [FixedString(16)](../../data_types/fixedstring.md).
+Accepts a string containing 36 characters in the format `xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx`, and returns it as a set of bytes in a [FixedString(16)](../../sql_reference/data_types/fixedstring.md).
``` sql
UUIDStringToNum(String)
@@ -85,7 +90,7 @@ SELECT
## UUIDNumToString {#uuidnumtostring}
-Accepts a [FixedString(16)](../../data_types/fixedstring.md) value, and returns a string containing 36 characters in text format.
+Accepts a [FixedString(16)](../../sql_reference/data_types/fixedstring.md) value, and returns a string containing 36 characters in text format.
``` sql
UUIDNumToString(FixedString(16))
diff --git a/docs/en/query_language/functions/ym_dict_functions.md b/docs/en/sql_reference/functions/ym_dict_functions.md
similarity index 96%
rename from docs/en/query_language/functions/ym_dict_functions.md
rename to docs/en/sql_reference/functions/ym_dict_functions.md
index 344c850cc33..e36d460689b 100644
--- a/docs/en/query_language/functions/ym_dict_functions.md
+++ b/docs/en/sql_reference/functions/ym_dict_functions.md
@@ -1,3 +1,8 @@
+---
+toc_priority: 59
+toc_title: Working with Yandex.Metrica Dictionaries
+---
+
# Functions for working with Yandex.Metrica dictionaries {#functions-for-working-with-yandex-metrica-dictionaries}
In order for the functions below to work, the server config must specify the paths and addresses for getting all the Yandex.Metrica dictionaries. The dictionaries are loaded at the first call of any of these functions. If the reference lists can’t be loaded, an exception is thrown.
@@ -112,8 +117,8 @@ regionToTopContinent(id[, geobase]);
**Parameters**
-- `id` — Region ID from the Yandex geobase. [UInt32](../../data_types/int_uint.md).
-- `geobase` — Dictionary key. See [Multiple Geobases](#multiple-geobases). [String](../../data_types/string.md). Optional.
+- `id` — Region ID from the Yandex geobase. [UInt32](../../sql_reference/data_types/int_uint.md).
+- `geobase` — Dictionary key. See [Multiple Geobases](#multiple-geobases). [String](../../sql_reference/data_types/string.md). Optional.
**Returned value**
diff --git a/docs/en/sql_reference/index.md b/docs/en/sql_reference/index.md
new file mode 100644
index 00000000000..ae0f81336be
--- /dev/null
+++ b/docs/en/sql_reference/index.md
@@ -0,0 +1,16 @@
+---
+toc_folder_title: SQL Reference
+toc_hidden: true
+toc_priority: 28
+toc_title: hidden
+---
+
+# SQL Reference {#sql-reference}
+
+- [SELECT](statements/select.md)
+- [INSERT INTO](statements/insert_into.md)
+- [CREATE](statements/create.md)
+- [ALTER](statements/alter.md#query_language_queries_alter)
+- [Other types of queries](statements/misc.md)
+
+[Original article](https://clickhouse.tech/docs/en/query_language/)
diff --git a/docs/en/query_language/operators.md b/docs/en/sql_reference/operators.md
similarity index 85%
rename from docs/en/query_language/operators.md
rename to docs/en/sql_reference/operators.md
index 700a5c87c9a..6414a1d6469 100644
--- a/docs/en/query_language/operators.md
+++ b/docs/en/sql_reference/operators.md
@@ -1,3 +1,8 @@
+---
+toc_priority: 37
+toc_title: Operators
+---
+
# Operators {#operators}
All operators are transformed to their corresponding functions at the query parsing stage in accordance with their precedence and associativity.
@@ -53,9 +58,9 @@ Groups of operators are listed in order of priority (the higher it is in the lis
`a NOT BETWEEN b AND c` – The same as `a < b OR a > c`.
-## Operators for Working With Data Sets {#operators-for-working-with-data-sets}
+## Operators For Working With Data Sets {#operators-for-working-with-data-sets}
-*See [IN operators](select.md#select-in-operators).*
+*See [IN operators](statements/select.md#select-in-operators).*
`a IN ...` – The `in(a, b)` function.
@@ -65,7 +70,7 @@ Groups of operators are listed in order of priority (the higher it is in the lis
`a GLOBAL NOT IN ...` – The `globalNotIn(a, b)` function.
-## Operators for Working with Dates and Times {#operators-datetime}
+## Operators For Working With Dates and Times {#operators-datetime}
### EXTRACT {#operator-extract}
@@ -86,7 +91,7 @@ The `part` parameter specifies which part of the date to retrieve. The following
The `part` parameter is case-insensitive.
-The `date` parameter specifies the date or the time to process. Either [Date](../data_types/date.md) or [DateTime](../data_types/datetime.md) type is supported.
+The `date` parameter specifies the date or the time to process. Either [Date](../sql_reference/data_types/date.md) or [DateTime](../sql_reference/data_types/datetime.md) type is supported.
Examples:
@@ -133,7 +138,7 @@ You can see more examples in [tests](https://github.com/ClickHouse/ClickHouse/bl
### INTERVAL {#operator-interval}
-Creates an [Interval](../data_types/special_data_types/interval.md)-type value that should be used in arithmetical operations with [Date](../data_types/date.md) and [DateTime](../data_types/datetime.md)-type values.
+Creates an [Interval](../sql_reference/data_types/special_data_types/interval.md)-type value that should be used in arithmetical operations with [Date](../sql_reference/data_types/date.md) and [DateTime](../sql_reference/data_types/datetime.md)-type values.
Types of intervals:
- `SECOND`
@@ -162,8 +167,8 @@ SELECT now() AS current_date_time, current_date_time + INTERVAL 4 DAY + INTERVAL
**See Also**
-- [Interval](../data_types/special_data_types/interval.md) data type
-- [toInterval](functions/type_conversion_functions.md#function-tointerval) type convertion functions
+- [Interval](../sql_reference/data_types/special_data_types/interval.md) data type
+- [toInterval](../sql_reference/functions/type_conversion_functions.md#function-tointerval) type convertion functions
## Logical Negation Operator {#logical-negation-operator}
@@ -183,7 +188,7 @@ SELECT now() AS current_date_time, current_date_time + INTERVAL 4 DAY + INTERVAL
Note:
-The conditional operator calculates the values of b and c, then checks whether condition a is met, and then returns the corresponding value. If `b` or `C` is an [arrayJoin()](functions/array_join.md#functions_arrayjoin) function, each row will be replicated regardless of the “a” condition.
+The conditional operator calculates the values of b and c, then checks whether condition a is met, and then returns the corresponding value. If `b` or `C` is an [arrayJoin()](../sql_reference/functions/array_join.md#functions_arrayjoin) function, each row will be replicated regardless of the “a” condition.
## Conditional Expression {#operator_case}
@@ -226,13 +231,13 @@ Sometimes this doesn’t work the way you expect. For example, `SELECT 4 > 2 > 3
For efficiency, the `and` and `or` functions accept any number of arguments. The corresponding chains of `AND` and `OR` operators are transformed to a single call of these functions.
-## Checking for `NULL` {#checking-for-null}
+## Checking For `NULL` {#checking-for-null}
ClickHouse supports the `IS NULL` and `IS NOT NULL` operators.
### IS NULL {#operator-is-null}
-- For [Nullable](../data_types/nullable.md) type values, the `IS NULL` operator returns:
+- For [Nullable](../sql_reference/data_types/nullable.md) type values, the `IS NULL` operator returns:
- `1`, if the value is `NULL`.
- `0` otherwise.
- For other values, the `IS NULL` operator always returns `0`.
@@ -251,7 +256,7 @@ SELECT x+100 FROM t_null WHERE y IS NULL
### IS NOT NULL {#is-not-null}
-- For [Nullable](../data_types/nullable.md) type values, the `IS NOT NULL` operator returns:
+- For [Nullable](../sql_reference/data_types/nullable.md) type values, the `IS NOT NULL` operator returns:
- `0`, if the value is `NULL`.
- `1` otherwise.
- For other values, the `IS NOT NULL` operator always returns `1`.
diff --git a/docs/en/query_language/alter.md b/docs/en/sql_reference/statements/alter.md
similarity index 89%
rename from docs/en/query_language/alter.md
rename to docs/en/sql_reference/statements/alter.md
index 625b460c0ef..885dc138a51 100644
--- a/docs/en/query_language/alter.md
+++ b/docs/en/sql_reference/statements/alter.md
@@ -1,3 +1,8 @@
+---
+toc_priority: 36
+toc_title: ALTER
+---
+
## ALTER {#query_language_queries_alter}
The `ALTER` query is only supported for `*MergeTree` tables, as well as `Merge`and`Distributed`. The query has several variations.
@@ -33,7 +38,7 @@ Adds a new column to the table with the specified `name`, `type`, [`codec`](crea
If the `IF NOT EXISTS` clause is included, the query won’t return an error if the column already exists. If you specify `AFTER name_after` (the name of another column), the column is added after the specified one in the list of table columns. Otherwise, the column is added to the end of the table. Note that there is no way to add a column to the beginning of a table. For a chain of actions, `name_after` can be the name of a column that is added in one of the previous actions.
-Adding a column just changes the table structure, without performing any actions with data. The data doesn’t appear on the disk after `ALTER`. If the data is missing for a column when reading from the table, it is filled in with default values (by performing the default expression if there is one, or using zeros or empty strings). The column appears on the disk after merging data parts (see [MergeTree](../operations/table_engines/mergetree.md)).
+Adding a column just changes the table structure, without performing any actions with data. The data doesn’t appear on the disk after `ALTER`. If the data is missing for a column when reading from the table, it is filled in with default values (by performing the default expression if there is one, or using zeros or empty strings). The column appears on the disk after merging data parts (see [MergeTree](../../engines/table_engines/mergetree_family/mergetree.md)).
This approach allows us to complete the `ALTER` query instantly, without increasing the volume of old data.
@@ -107,11 +112,11 @@ This query changes the `name` column properties:
- TTL
- For examples of columns TTL modifying, see [Column TTL](../operations/table_engines/mergetree.md#mergetree-column-ttl).
+ For examples of columns TTL modifying, see [Column TTL](../engines/table_engines/mergetree_family/mergetree.md#mergetree-column-ttl).
If the `IF EXISTS` clause is specified, the query won’t return an error if the column doesn’t exist.
-When changing the type, values are converted as if the [toType](functions/type_conversion_functions.md) functions were applied to them. If only the default expression is changed, the query doesn’t do anything complex, and is completed almost instantly.
+When changing the type, values are converted as if the [toType](../../sql_reference/functions/type_conversion_functions.md) functions were applied to them. If only the default expression is changed, the query doesn’t do anything complex, and is completed almost instantly.
Example:
@@ -139,7 +144,7 @@ The `ALTER` query lets you create and delete separate elements (columns) in nest
There is no support for deleting columns in the primary key or the sampling key (columns that are used in the `ENGINE` expression). Changing the type for columns that are included in the primary key is only possible if this change does not cause the data to be modified (for example, you are allowed to add values to an Enum or to change a type from `DateTime` to `UInt32`).
-If the `ALTER` query is not sufficient to make the table changes you need, you can create a new table, copy the data to it using the [INSERT SELECT](insert_into.md#insert_query_insert-select) query, then switch the tables using the [RENAME](misc.md#misc_operations-rename) query and delete the old table. You can use the [clickhouse-copier](../operations/utils/clickhouse-copier.md) as an alternative to the `INSERT SELECT` query.
+If the `ALTER` query is not sufficient to make the table changes you need, you can create a new table, copy the data to it using the [INSERT SELECT](insert_into.md#insert_query_insert-select) query, then switch the tables using the [RENAME](misc.md#misc_operations-rename) query and delete the old table. You can use the [clickhouse-copier](../../operations/utilities/clickhouse-copier.md) as an alternative to the `INSERT SELECT` query.
The `ALTER` query blocks all reads and writes for the table. In other words, if a long `SELECT` is running at the time of the `ALTER` query, the `ALTER` query will wait for it to complete. At the same time, all new queries to the same table will wait while this `ALTER` is running.
@@ -153,9 +158,9 @@ The following command is supported:
MODIFY ORDER BY new_expression
```
-It only works for tables in the [`MergeTree`](../operations/table_engines/mergetree.md) family (including
-[replicated](../operations/table_engines/replication.md) tables). The command changes the
-[sorting key](../operations/table_engines/mergetree.md) of the table
+It only works for tables in the [`MergeTree`](../../engines/table_engines/mergetree_family/mergetree.md) family (including
+[replicated](../../engines/table_engines/mergetree_family/replication.md) tables). The command changes the
+[sorting key](../../engines/table_engines/mergetree_family/mergetree.md) of the table
to `new_expression` (an expression or a tuple of expressions). Primary key remains the same.
The command is lightweight in a sense that it only changes metadata. To keep the property that data part
@@ -164,8 +169,8 @@ to the sorting key (only columns added by the `ADD COLUMN` command in the same `
### Manipulations With Data Skipping Indices {#manipulations-with-data-skipping-indices}
-It only works for tables in the [`*MergeTree`](../operations/table_engines/mergetree.md) family (including
-[replicated](../operations/table_engines/replication.md) tables). The following operations
+It only works for tables in the [`*MergeTree`](../../engines/table_engines/mergetree_family/mergetree.md) family (including
+[replicated](../../engines/table_engines/mergetree_family/replication.md) tables). The following operations
are available:
- `ALTER TABLE [db].name ADD INDEX name expression TYPE type GRANULARITY value AFTER name [AFTER name2]` - Adds index description to tables metadata.
@@ -175,7 +180,7 @@ are available:
These commands are lightweight in a sense that they only change metadata or remove files.
Also, they are replicated (syncing indices metadata through ZooKeeper).
-### Manipulations with constraints {#manipulations-with-constraints}
+### Manipulations With Constraints {#manipulations-with-constraints}
See more on [constraints](create.md#constraints)
@@ -194,7 +199,7 @@ All changes on replicated tables are broadcasting to ZooKeeper so will be applie
### Manipulations With Partitions and Parts {#alter_manipulations-with-partitions}
-The following operations with [partitions](../operations/table_engines/custom_partitioning_key.md) are available:
+The following operations with [partitions](../../engines/table_engines/mergetree_family/custom_partitioning_key.md) are available:
- [DETACH PARTITION](#alter_detach-partition) – Moves a partition to the `detached` directory and forget it.
- [DROP PARTITION](#alter_drop-partition) – Deletes a partition.
@@ -229,7 +234,7 @@ Read about setting the partition expression in a section [How to specify the par
After the query is executed, you can do whatever you want with the data in the `detached` directory — delete it from the file system, or just leave it.
-This query is replicated – it moves the data to the `detached` directory on all replicas. Note that you can execute this query only on a leader replica. To find out if a replica is a leader, perform the `SELECT` query to the [system.replicas](../operations/system_tables.md#system_tables-replicas) table. Alternatively, it is easier to make a `DETACH` query on all replicas - all the replicas throw an exception, except the leader replica.
+This query is replicated – it moves the data to the `detached` directory on all replicas. Note that you can execute this query only on a leader replica. To find out if a replica is a leader, perform the `SELECT` query to the [system.replicas](../../operations/system_tables.md#system_tables-replicas) table. Alternatively, it is easier to make a `DETACH` query on all replicas - all the replicas throw an exception, except the leader replica.
#### DROP PARTITION {#alter_drop-partition}
@@ -345,7 +350,7 @@ At the time of execution, for a data snapshot, the query creates hardlinks to a
- `N` is the incremental number of the backup.
!!! note "Note"
- If you use [a set of disks for data storage in a table](../operations/table_engines/mergetree.md#table_engine-mergetree-multiple-volumes), the `shadow/N` directory appears on every disk, storing data parts that matched by the `PARTITION` expression.
+ If you use [a set of disks for data storage in a table](../../engines/table_engines/mergetree_family/mergetree.md#table_engine-mergetree-multiple-volumes), the `shadow/N` directory appears on every disk, storing data parts that matched by the `PARTITION` expression.
The same structure of directories is created inside the backup as inside `/var/lib/clickhouse/`. The query performs ‘chmod’ for all files, forbidding writing into them.
@@ -363,7 +368,7 @@ To restore data from a backup, do the following:
Restoring from a backup doesn’t require stopping the server.
-For more information about backups and restoring data, see the [Data Backup](../operations/backup.md) section.
+For more information about backups and restoring data, see the [Data Backup](../../operations/backup.md) section.
#### CLEAR INDEX IN PARTITION {#alter_clear-index-partition}
@@ -404,7 +409,7 @@ Although the query is called `ALTER TABLE`, it does not change the table structu
#### MOVE PARTITION\|PART {#alter_move-partition}
-Moves partitions or data parts to another volume or disk for `MergeTree`-engine tables. See [Using Multiple Block Devices for Data Storage](../operations/table_engines/mergetree.md#table_engine-mergetree-multiple-volumes).
+Moves partitions or data parts to another volume or disk for `MergeTree`-engine tables. See [Using Multiple Block Devices for Data Storage](../../engines/table_engines/mergetree_family/mergetree.md#table_engine-mergetree-multiple-volumes).
``` sql
ALTER TABLE table_name MOVE PARTITION|PART partition_expr TO DISK|VOLUME 'disk_name'
@@ -430,7 +435,7 @@ You can specify the partition expression in `ALTER ... PARTITION` queries in dif
- As a value from the `partition` column of the `system.parts` table. For example, `ALTER TABLE visits DETACH PARTITION 201901`.
- As the expression from the table column. Constants and constant expressions are supported. For example, `ALTER TABLE visits DETACH PARTITION toYYYYMM(toDate('2019-01-25'))`.
- Using the partition ID. Partition ID is a string identifier of the partition (human-readable, if possible) that is used as the names of partitions in the file system and in ZooKeeper. The partition ID must be specified in the `PARTITION ID` clause, in a single quotes. For example, `ALTER TABLE visits DETACH PARTITION ID '201901'`.
-- In the [ALTER ATTACH PART](#alter_attach-partition) and [DROP DETACHED PART](#alter_drop-detached) query, to specify the name of a part, use string literal with a value from the `name` column of the [system.detached\_parts](../operations/system_tables.md#system_tables-detached_parts) table. For example, `ALTER TABLE visits ATTACH PART '201901_1_1_0'`.
+- In the [ALTER ATTACH PART](#alter_attach-partition) and [DROP DETACHED PART](#alter_drop-detached) query, to specify the name of a part, use string literal with a value from the `name` column of the [system.detached\_parts](../../operations/system_tables.md#system_tables-detached_parts) table. For example, `ALTER TABLE visits ATTACH PART '201901_1_1_0'`.
Usage of quotes when specifying the partition depends on the type of partition expression. For example, for the `String` type, you have to specify its name in quotes (`'`). For the `Date` and `Int*` types no quotes are needed.
@@ -444,15 +449,15 @@ OPTIMIZE TABLE table_not_partitioned PARTITION tuple() FINAL;
The examples of `ALTER ... PARTITION` queries are demonstrated in the tests [`00502_custom_partitioning_local`](https://github.com/ClickHouse/ClickHouse/blob/master/tests/queries/0_stateless/00502_custom_partitioning_local.sql) and [`00502_custom_partitioning_replicated_zookeeper`](https://github.com/ClickHouse/ClickHouse/blob/master/tests/queries/0_stateless/00502_custom_partitioning_replicated_zookeeper.sql).
-### Manipulations with Table TTL {#manipulations-with-table-ttl}
+### Manipulations With Table TTL {#manipulations-with-table-ttl}
-You can change [table TTL](../operations/table_engines/mergetree.md#mergetree-table-ttl) with a request of the following form:
+You can change [table TTL](../../engines/table_engines/mergetree_family/mergetree.md#mergetree-table-ttl) with a request of the following form:
``` sql
ALTER TABLE table-name MODIFY TTL ttl-expression
```
-### Synchronicity of ALTER Queries {#synchronicity-of-alter-queries}
+### Synchronicity Of ALTER Queries {#synchronicity-of-alter-queries}
For non-replicatable tables, all `ALTER` queries are performed synchronously. For replicatable tables, the query just adds instructions for the appropriate actions to `ZooKeeper`, and the actions themselves are performed as soon as possible. However, the query can wait for these actions to be completed on all the replicas.
@@ -491,7 +496,7 @@ For \*MergeTree tables mutations execute by rewriting whole data parts. There is
Mutations are totally ordered by their creation order and are applied to each part in that order. Mutations are also partially ordered with INSERTs - data that was inserted into the table before the mutation was submitted will be mutated and data that was inserted after that will not be mutated. Note that mutations do not block INSERTs in any way.
-A mutation query returns immediately after the mutation entry is added (in case of replicated tables to ZooKeeper, for nonreplicated tables - to the filesystem). The mutation itself executes asynchronously using the system profile settings. To track the progress of mutations you can use the [`system.mutations`](../operations/system_tables.md#system_tables-mutations) table. A mutation that was successfully submitted will continue to execute even if ClickHouse servers are restarted. There is no way to roll back the mutation once it is submitted, but if the mutation is stuck for some reason it can be cancelled with the [`KILL MUTATION`](misc.md#kill-mutation) query.
+A mutation query returns immediately after the mutation entry is added (in case of replicated tables to ZooKeeper, for nonreplicated tables - to the filesystem). The mutation itself executes asynchronously using the system profile settings. To track the progress of mutations you can use the [`system.mutations`](../../operations/system_tables.md#system_tables-mutations) table. A mutation that was successfully submitted will continue to execute even if ClickHouse servers are restarted. There is no way to roll back the mutation once it is submitted, but if the mutation is stuck for some reason it can be cancelled with the [`KILL MUTATION`](misc.md#kill-mutation) query.
Entries for finished mutations are not deleted right away (the number of preserved entries is determined by the `finished_mutations_to_keep` storage engine parameter). Older mutation entries are deleted.
diff --git a/docs/en/query_language/create.md b/docs/en/sql_reference/statements/create.md
similarity index 87%
rename from docs/en/query_language/create.md
rename to docs/en/sql_reference/statements/create.md
index 066eb396b19..fbfe3eb2cfb 100644
--- a/docs/en/query_language/create.md
+++ b/docs/en/sql_reference/statements/create.md
@@ -1,3 +1,8 @@
+---
+toc_priority: 35
+toc_title: CREATE
+---
+
# CREATE Queries {#create-queries}
## CREATE DATABASE {#query-language-create-database}
@@ -23,11 +28,11 @@ CREATE DATABASE [IF NOT EXISTS] db_name [ON CLUSTER cluster] [ENGINE = engine(..
- `ENGINE`
- - [MySQL](../database_engines/mysql.md)
+ - [MySQL](../engines/database_engines/mysql.md)
Allows you to retrieve data from the remote MySQL server.
- By default, ClickHouse uses its own [database engine](../database_engines/index.md).
+ By default, ClickHouse uses its own [database engine](../engines/database_engines/index.md).
## CREATE TABLE {#create-table-query}
@@ -58,7 +63,7 @@ Creates a table with the same structure as another table. You can specify a diff
CREATE TABLE [IF NOT EXISTS] [db.]table_name AS table_function()
```
-Creates a table with the structure and data returned by a [table function](table_functions/index.md).
+Creates a table with the structure and data returned by a [table function](../table_functions/index.md).
``` sql
CREATE TABLE [IF NOT EXISTS] [db.]table_name ENGINE = engine AS SELECT ...
@@ -68,7 +73,7 @@ Creates a table with a structure like the result of the `SELECT` query, with the
In all cases, if `IF NOT EXISTS` is specified, the query won’t return an error if the table already exists. In this case, the query won’t do anything.
-There can be other clauses after the `ENGINE` clause in the query. See detailed documentation on how to create tables in the descriptions of [table engines](../operations/table_engines/index.md#table_engines).
+There can be other clauses after the `ENGINE` clause in the query. See detailed documentation on how to create tables in the descriptions of [table engines](../../engines/table_engines/index.md#table_engines).
### Default Values {#create-default-values}
@@ -125,11 +130,11 @@ Adding large amount of constraints can negatively affect performance of big `INS
### TTL Expression {#ttl-expression}
-Defines storage time for values. Can be specified only for MergeTree-family tables. For the detailed description, see [TTL for columns and tables](../operations/table_engines/mergetree.md#table_engine-mergetree-ttl).
+Defines storage time for values. Can be specified only for MergeTree-family tables. For the detailed description, see [TTL for columns and tables](../../engines/table_engines/mergetree_family/mergetree.md#table_engine-mergetree-ttl).
### Column Compression Codecs {#codecs}
-By default, ClickHouse applies the `lz4` compression method. For `MergeTree`-engine family you can change the default compression method in the [compression](../operations/server_settings/settings.md#server-settings-compression) section of a server configuration. You can also define the compression method for each individual column in the `CREATE TABLE` query.
+By default, ClickHouse applies the `lz4` compression method. For `MergeTree`-engine family you can change the default compression method in the [compression](../../operations/server_configuration_parameters/settings.md#server-settings-compression) section of a server configuration. You can also define the compression method for each individual column in the `CREATE TABLE` query.
``` sql
CREATE TABLE codec_example
@@ -151,10 +156,10 @@ If a codec is specified, the default codec doesn’t apply. Codecs can be combin
Compression is supported for the following table engines:
-- [MergeTree](../operations/table_engines/mergetree.md) family. Supports column compression codecs and selecting the default compression method by [compression](../operations/server_settings/settings.md#server-settings-compression) settings.
-- [Log](../operations/table_engines/log_family.md) family. Uses the `lz4` compression method by default and supports column compression codecs.
-- [Set](../operations/table_engines/set.md). Only supported the default compression.
-- [Join](../operations/table_engines/join.md). Only supported the default compression.
+- [MergeTree](../../engines/table_engines/mergetree_family/mergetree.md) family. Supports column compression codecs and selecting the default compression method by [compression](../../operations/server_configuration_parameters/settings.md#server-settings-compression) settings.
+- [Log](../../engines/table_engines/log_family/log_family.md) family. Uses the `lz4` compression method by default and supports column compression codecs.
+- [Set](../../engines/table_engines/special/set.md). Only supported the default compression.
+- [Join](../../engines/table_engines/special/join.md). Only supported the default compression.
ClickHouse supports common purpose codecs and specialized codecs.
@@ -180,7 +185,7 @@ CREATE TABLE codec_example
ENGINE = MergeTree()
```
-#### Common purpose codecs {#create-query-common-purpose-codecs}
+#### Common Purpose Codecs {#create-query-common-purpose-codecs}
Codecs:
@@ -215,9 +220,9 @@ CREATE TEMPORARY TABLE [IF NOT EXISTS] table_name
In most cases, temporary tables are not created manually, but when using external data for a query, or for distributed `(GLOBAL) IN`. For more information, see the appropriate sections
-It’s possible to use tables with [ENGINE = Memory](../operations/table_engines/memory.md) instead of temporary tables.
+It’s possible to use tables with [ENGINE = Memory](../../engines/table_engines/special/memory.md) instead of temporary tables.
-## Distributed DDL queries (ON CLUSTER clause) {#distributed-ddl-queries-on-cluster-clause}
+## Distributed DDL Queries (ON CLUSTER Clause) {#distributed-ddl-queries-on-cluster-clause}
The `CREATE`, `DROP`, `ALTER`, and `RENAME` queries support distributed execution on a cluster.
For example, the following query creates the `all_hits` `Distributed` table on each host in `cluster`:
@@ -291,12 +296,12 @@ LAYOUT(LAYOUT_NAME([param_name param_value]))
LIFETIME([MIN val1] MAX val2)
```
-Creates [external dictionary](dicts/external_dicts.md) with given [structure](dicts/external_dicts_dict_structure.md), [source](dicts/external_dicts_dict_sources.md), [layout](dicts/external_dicts_dict_layout.md) and [lifetime](dicts/external_dicts_dict_lifetime.md).
+Creates [external dictionary](../../sql_reference/dictionaries/external_dictionaries/external_dicts.md) with given [structure](../../sql_reference/dictionaries/external_dictionaries/external_dicts_dict_structure.md), [source](../../sql_reference/dictionaries/external_dictionaries/external_dicts_dict_sources.md), [layout](../../sql_reference/dictionaries/external_dictionaries/external_dicts_dict_layout.md) and [lifetime](../../sql_reference/dictionaries/external_dictionaries/external_dicts_dict_lifetime.md).
External dictionary structure consists of attributes. Dictionary attributes are specified similarly to table columns. The only required attribute property is its type, all other properties may have default values.
-Depending on dictionary [layout](dicts/external_dicts_dict_layout.md) one or more attributes can be specified as dictionary keys.
+Depending on dictionary [layout](../../sql_reference/dictionaries/external_dictionaries/external_dicts_dict_layout.md) one or more attributes can be specified as dictionary keys.
-For more information, see [External Dictionaries](dicts/external_dicts.md) section.
+For more information, see [External Dictionaries](../../sql_reference/dictionaries/external_dictionaries/external_dicts.md) section.
[Original article](https://clickhouse.tech/docs/en/query_language/create/)
diff --git a/docs/en/sql_reference/statements/index.md b/docs/en/sql_reference/statements/index.md
new file mode 100644
index 00000000000..1adf93a153e
--- /dev/null
+++ b/docs/en/sql_reference/statements/index.md
@@ -0,0 +1,5 @@
+---
+toc_folder_title: Statements
+toc_priority: 31
+---
+
diff --git a/docs/en/query_language/insert_into.md b/docs/en/sql_reference/statements/insert_into.md
similarity index 83%
rename from docs/en/query_language/insert_into.md
rename to docs/en/sql_reference/statements/insert_into.md
index 4220c67ec3b..0f26a37e2b9 100644
--- a/docs/en/query_language/insert_into.md
+++ b/docs/en/sql_reference/statements/insert_into.md
@@ -1,3 +1,8 @@
+---
+toc_priority: 34
+toc_title: INSERT INTO
+---
+
## INSERT {#insert}
Adding data.
@@ -13,9 +18,9 @@ The query can specify a list of columns to insert `[(c1, c2, c3)]`. In this case
- The values calculated from the `DEFAULT` expressions specified in the table definition.
- Zeros and empty strings, if `DEFAULT` expressions are not defined.
-If [strict\_insert\_defaults=1](../operations/settings/settings.md), columns that do not have `DEFAULT` defined must be listed in the query.
+If [strict\_insert\_defaults=1](../../operations/settings/settings.md), columns that do not have `DEFAULT` defined must be listed in the query.
-Data can be passed to the INSERT in any [format](../interfaces/formats.md#formats) supported by ClickHouse. The format must be specified explicitly in the query:
+Data can be passed to the INSERT in any [format](../../interfaces/formats.md#formats) supported by ClickHouse. The format must be specified explicitly in the query:
``` sql
INSERT INTO [db.]table [(c1, c2, c3)] FORMAT format_name data_set
@@ -37,13 +42,13 @@ INSERT INTO t FORMAT TabSeparated
22 Qwerty
```
-You can insert data separately from the query by using the command-line client or the HTTP interface. For more information, see the section “[Interfaces](../interfaces/index.md#interfaces)”.
+You can insert data separately from the query by using the command-line client or the HTTP interface. For more information, see the section “[Interfaces](../../interfaces/index.md#interfaces)”.
### Constraints {#constraints}
If table has [constraints](create.md#constraints), their expressions will be checked for each row of inserted data. If any of those constraints is not satisfied — server will raise an exception containing constraint name and expression, the query will be stopped.
-### Inserting The Results of `SELECT` {#insert_query_insert-select}
+### Inserting The Results Of `SELECT` {#insert_query_insert-select}
``` sql
INSERT INTO [db.]table [(c1, c2, c3)] SELECT ...
@@ -56,7 +61,7 @@ None of the data formats except Values allow setting values to expressions such
Other queries for modifying data parts are not supported: `UPDATE`, `DELETE`, `REPLACE`, `MERGE`, `UPSERT`, `INSERT UPDATE`.
However, you can delete old data using `ALTER TABLE ... DROP PARTITION`.
-`FORMAT` clause must be specified in the end of query if `SELECT` clause contains table function [input()](table_functions/input.md).
+`FORMAT` clause must be specified in the end of query if `SELECT` clause contains table function [input()](../table_functions/input.md).
### Performance Considerations {#performance-considerations}
diff --git a/docs/en/query_language/misc.md b/docs/en/sql_reference/statements/misc.md
similarity index 82%
rename from docs/en/query_language/misc.md
rename to docs/en/sql_reference/statements/misc.md
index 443a4f2ac6d..be2c07e86b8 100644
--- a/docs/en/query_language/misc.md
+++ b/docs/en/sql_reference/statements/misc.md
@@ -1,3 +1,8 @@
+---
+toc_priority: 39
+toc_title: Other
+---
+
# Miscellaneous Queries {#miscellaneous-queries}
## ATTACH {#attach}
@@ -27,17 +32,17 @@ CHECK TABLE [db.]name
The `CHECK TABLE` query compares actual file sizes with the expected values which are stored on the server. If the file sizes do not match the stored values, it means the data is corrupted. This can be caused, for example, by a system crash during query execution.
The query response contains the `result` column with a single row. The row has a value of
-[Boolean](../data_types/boolean.md) type:
+[Boolean](../../sql_reference/data_types/boolean.md) type:
- 0 - The data in the table is corrupted.
- 1 - The data maintains integrity.
The `CHECK TABLE` query supports the following table engines:
-- [Log](../operations/table_engines/log.md)
-- [TinyLog](../operations/table_engines/tinylog.md)
-- [StripeLog](../operations/table_engines/stripelog.md)
-- [MergeTree family](../operations/table_engines/mergetree.md)
+- [Log](../../engines/table_engines/log_family/log.md)
+- [TinyLog](../../engines/table_engines/log_family/tinylog.md)
+- [StripeLog](../../engines/table_engines/log_family/stripelog.md)
+- [MergeTree family](../../engines/table_engines/mergetree_family/mergetree.md)
Performed over the tables with another table engines causes an exception.
@@ -50,7 +55,7 @@ For `MergeTree` family engines, the `CHECK TABLE` query shows a check status for
If the table is corrupted, you can copy the non-corrupted data to another table. To do this:
1. Create a new table with the same structure as damaged table. To do this execute the query `CREATE TABLE AS `.
-2. Set the [max\_threads](../operations/settings/settings.md#settings-max_threads) value to 1 to process the next query in a single thread. To do this run the query `SET max_threads = 1`.
+2. Set the [max\_threads](../../operations/settings/settings.md#settings-max_threads) value to 1 to process the next query in a single thread. To do this run the query `SET max_threads = 1`.
3. Execute the query `INSERT INTO SELECT * FROM `. This request copies the non-corrupted data from the damaged table to another table. Only the data before the corrupted part will be copied.
4. Restart the `clickhouse-client` to reset the `max_threads` value.
@@ -158,7 +163,7 @@ KILL MUTATION [ON CLUSTER cluster]
[FORMAT format]
```
-Tries to cancel and remove [mutations](alter.md#alter-mutations) that are currently executing. Mutations to cancel are selected from the [`system.mutations`](../operations/system_tables.md#system_tables-mutations) table using the filter specified by the `WHERE` clause of the `KILL` query.
+Tries to cancel and remove [mutations](alter.md#alter-mutations) that are currently executing. Mutations to cancel are selected from the [`system.mutations`](../../operations/system_tables.md#system_tables-mutations) table using the filter specified by the `WHERE` clause of the `KILL` query.
A test query (`TEST`) only checks the user’s rights and displays a list of queries to stop.
@@ -182,13 +187,13 @@ Changes already made by the mutation are not rolled back.
OPTIMIZE TABLE [db.]name [ON CLUSTER cluster] [PARTITION partition | PARTITION ID 'partition_id'] [FINAL] [DEDUPLICATE]
```
-This query tries to initialize an unscheduled merge of data parts for tables with a table engine from the [MergeTree](../operations/table_engines/mergetree.md) family.
+This query tries to initialize an unscheduled merge of data parts for tables with a table engine from the [MergeTree](../../engines/table_engines/mergetree_family/mergetree.md) family.
-The `OPTMIZE` query is also supported for the [MaterializedView](../operations/table_engines/materializedview.md) and the [Buffer](../operations/table_engines/buffer.md) engines. Other table engines aren’t supported.
+The `OPTMIZE` query is also supported for the [MaterializedView](../../engines/table_engines/special/materializedview.md) and the [Buffer](../../engines/table_engines/special/buffer.md) engines. Other table engines aren’t supported.
-When `OPTIMIZE` is used with the [ReplicatedMergeTree](../operations/table_engines/replication.md) family of table engines, ClickHouse creates a task for merging and waits for execution on all nodes (if the `replication_alter_partitions_sync` setting is enabled).
+When `OPTIMIZE` is used with the [ReplicatedMergeTree](../../engines/table_engines/mergetree_family/replication.md) family of table engines, ClickHouse creates a task for merging and waits for execution on all nodes (if the `replication_alter_partitions_sync` setting is enabled).
-- If `OPTIMIZE` doesn’t perform a merge for any reason, it doesn’t notify the client. To enable notifications, use the [optimize\_throw\_if\_noop](../operations/settings/settings.md#setting-optimize_throw_if_noop) setting.
+- If `OPTIMIZE` doesn’t perform a merge for any reason, it doesn’t notify the client. To enable notifications, use the [optimize\_throw\_if\_noop](../../operations/settings/settings.md#setting-optimize_throw_if_noop) setting.
- If you specify a `PARTITION`, only the specified partition is optimized. [How to set partition expression](alter.md#alter-how-to-specify-part-expr).
- If you specify `FINAL`, optimization is performed even when all the data is already in one part.
- If you specify `DEDUPLICATE`, then completely identical rows will be deduplicated (all columns are compared), it makes sense only for the MergeTree engine.
@@ -212,7 +217,7 @@ All tables are renamed under global locking. Renaming tables is a light operatio
SET param = value
```
-Assigns `value` to the `param` [setting](../operations/settings/index.md) for the current session. You cannot change [server settings](../operations/server_settings/index.md) this way.
+Assigns `value` to the `param` [setting](../../operations/settings/index.md) for the current session. You cannot change [server settings](../../operations/server_configuration_parameters/index.md) this way.
You can also set all the values from the specified settings profile in a single query.
@@ -220,7 +225,7 @@ You can also set all the values from the specified settings profile in a single
SET profile = 'profile-name-from-the-settings-file'
```
-For more information, see [Settings](../operations/settings/settings.md).
+For more information, see [Settings](../../operations/settings/settings.md).
## TRUNCATE {#truncate}
@@ -230,7 +235,7 @@ TRUNCATE TABLE [IF EXISTS] [db.]name [ON CLUSTER cluster]
Removes all data from a table. When the clause `IF EXISTS` is omitted, the query returns an error if the table does not exist.
-The `TRUNCATE` query is not supported for [View](../operations/table_engines/view.md), [File](../operations/table_engines/file.md), [URL](../operations/table_engines/url.md) and [Null](../operations/table_engines/null.md) table engines.
+The `TRUNCATE` query is not supported for [View](../../engines/table_engines/special/view.md), [File](../../engines/table_engines/special/file.md), [URL](../../engines/table_engines/special/url.md) and [Null](../../engines/table_engines/special/null.md) table engines.
## USE {#use}
diff --git a/docs/en/sql_reference/statements/select.md b/docs/en/sql_reference/statements/select.md
new file mode 100644
index 00000000000..83ead5b26e1
--- /dev/null
+++ b/docs/en/sql_reference/statements/select.md
@@ -0,0 +1,608 @@
+---
+toc_priority: 33
+toc_title: SELECT
+---
+
+# SELECT Queries Syntax {#select-queries-syntax}
+
+`SELECT` performs data retrieval.
+
+``` sql
+[WITH expr_list|(subquery)]
+SELECT [DISTINCT] expr_list
+[FROM [db.]table | (subquery) | table_function] [FINAL]
+[SAMPLE sample_coeff]
+[ARRAY JOIN ...]
+[GLOBAL] [ANY|ALL] [INNER|LEFT|RIGHT|FULL|CROSS] [OUTER] JOIN (subquery)|table USING columns_list
+[PREWHERE expr]
+[WHERE expr]
+[GROUP BY expr_list] [WITH TOTALS]
+[HAVING expr]
+[ORDER BY expr_list]
+[LIMIT [offset_value, ]n BY columns]
+[LIMIT [n, ]m]
+[UNION ALL ...]
+[INTO OUTFILE filename]
+[FORMAT format]
+```
+
+All the clauses are optional, except for the required list of expressions immediately after SELECT.
+The clauses below are described in almost the same order as in the query execution conveyor.
+
+If the query omits the `DISTINCT`, `GROUP BY` and `ORDER BY` clauses and the `IN` and `JOIN` subqueries, the query will be completely stream processed, using O(1) amount of RAM.
+Otherwise, the query might consume a lot of RAM if the appropriate restrictions are not specified: `max_memory_usage`, `max_rows_to_group_by`, `max_rows_to_sort`, `max_rows_in_distinct`, `max_bytes_in_distinct`, `max_rows_in_set`, `max_bytes_in_set`, `max_rows_in_join`, `max_bytes_in_join`, `max_bytes_before_external_sort`, `max_bytes_before_external_group_by`. For more information, see the section “Settings”. It is possible to use external sorting (saving temporary tables to a disk) and external aggregation. `The system does not have "merge join"`.
+
+### WITH Clause {#with-clause}
+
+This section provides support for Common Table Expressions ([CTE](https://en.wikipedia.org/wiki/Hierarchical_and_recursive_queries_in_SQL)), with some limitations:
+1. Recursive queries are not supported
+2. When subquery is used inside WITH section, it’s result should be scalar with exactly one row
+3. Expression’s results are not available in subqueries
+Results of WITH clause expressions can be used inside SELECT clause.
+
+Example 1: Using constant expression as “variable”
+
+``` sql
+WITH '2019-08-01 15:23:00' as ts_upper_bound
+SELECT *
+FROM hits
+WHERE
+ EventDate = toDate(ts_upper_bound) AND
+ EventTime <= ts_upper_bound
+```
+
+Example 2: Evicting sum(bytes) expression result from SELECT clause column list
+
+``` sql
+WITH sum(bytes) as s
+SELECT
+ formatReadableSize(s),
+ table
+FROM system.parts
+GROUP BY table
+ORDER BY s
+```
+
+Example 3: Using results of scalar subquery
+
+``` sql
+/* this example would return TOP 10 of most huge tables */
+WITH
+ (
+ SELECT sum(bytes)
+ FROM system.parts
+ WHERE active
+ ) AS total_disk_usage
+SELECT
+ (sum(bytes) / total_disk_usage) * 100 AS table_disk_usage,
+ table
+FROM system.parts
+GROUP BY table
+ORDER BY table_disk_usage DESC
+LIMIT 10
+```
+
+Example 4: Re-using expression in subquery
+As a workaround for current limitation for expression usage in subqueries, you may duplicate it.
+
+``` sql
+WITH ['hello'] AS hello
+SELECT
+ hello,
+ *
+FROM
+(
+ WITH ['hello'] AS hello
+ SELECT hello
+)
+```
+
+``` text
+┌─hello─────┬─hello─────┐
+│ ['hello'] │ ['hello'] │
+└───────────┴───────────┘
+```
+
+### FROM Clause {#select-from}
+
+If the FROM clause is omitted, data will be read from the `system.one` table.
+The `system.one` table contains exactly one row (this table fulfills the same purpose as the DUAL table found in other DBMSs).
+
+The `FROM` clause specifies the source to read data from:
+
+- Table
+- Subquery
+- [Table function](../table_functions/index.md)
+
+`ARRAY JOIN` and the regular `JOIN` may also be included (see below).
+
+Instead of a table, the `SELECT` subquery may be specified in parenthesis.
+In contrast to standard SQL, a synonym does not need to be specified after a subquery.
+
+To execute a query, all the columns listed in the query are extracted from the appropriate table. Any columns not needed for the external query are thrown out of the subqueries.
+If a query does not list any columns (for example, `SELECT count() FROM t`), some column is extracted from the table anyway (the smallest one is preferred), in order to calculate the number of rows.
+
+#### FINAL Modifier {#select-from-final}
+
+Applicable when selecting data from tables from the [MergeTree](../../engines/table_engines/mergetree_family/mergetree.md)-engine family other than `GraphiteMergeTree`. When `FINAL` is specified, ClickHouse fully merges the data before returning the result and thus performs all data transformations that happen during merges for the given table engine.
+
+Also supported for:
+- [Replicated](../../engines/table_engines/mergetree_family/replication.md) versions of `MergeTree` engines.
+- [View](../../engines/table_engines/special/view.md), [Buffer](../../engines/table_engines/special/buffer.md), [Distributed](../../engines/table_engines/special/distributed.md), and [MaterializedView](../../engines/table_engines/special/materializedview.md) engines that operate over other engines, provided they were created over `MergeTree`-engine tables.
+
+Queries that use `FINAL` are executed not as fast as similar queries that don’t, because:
+
+- Query is executed in a single thread and data is merged during query execution.
+- Queries with `FINAL` read primary key columns in addition to the columns specified in the query.
+
+In most cases, avoid using `FINAL`.
+
+### SAMPLE Clause {#select-sample-clause}
+
+The `SAMPLE` clause allows for approximated query processing.
+
+When data sampling is enabled, the query is not performed on all the data, but only on a certain fraction of data (sample). For example, if you need to calculate statistics for all the visits, it is enough to execute the query on the 1/10 fraction of all the visits and then multiply the result by 10.
+
+Approximated query processing can be useful in the following cases:
+
+- When you have strict timing requirements (like \<100ms) but you can’t justify the cost of additional hardware resources to meet them.
+- When your raw data is not accurate, so approximation doesn’t noticeably degrade the quality.
+- Business requirements target approximate results (for cost-effectiveness, or in order to market exact results to premium users).
+
+!!! note "Note"
+ You can only use sampling with the tables in the [MergeTree](../../engines/table_engines/mergetree_family/mergetree.md) family, and only if the sampling expression was specified during table creation (see [MergeTree engine](../../engines/table_engines/mergetree_family/mergetree.md#table_engine-mergetree-creating-a-table)).
+
+The features of data sampling are listed below:
+
+- Data sampling is a deterministic mechanism. The result of the same `SELECT .. SAMPLE` query is always the same.
+- Sampling works consistently for different tables. For tables with a single sampling key, a sample with the same coefficient always selects the same subset of possible data. For example, a sample of user IDs takes rows with the same subset of all the possible user IDs from different tables. This means that you can use the sample in subqueries in the [IN](#select-in-operators) clause. Also, you can join samples using the [JOIN](#select-join) clause.
+- Sampling allows reading less data from a disk. Note that you must specify the sampling key correctly. For more information, see [Creating a MergeTree Table](../../engines/table_engines/mergetree_family/mergetree.md#table_engine-mergetree-creating-a-table).
+
+For the `SAMPLE` clause the following syntax is supported:
+
+| SAMPLE Clause Syntax | Description |
+|--------------|-------------------------------------------------------------------------------------------------------------------------------------------------------|
+| `SAMPLE k` | Here `k` is the number from 0 to 1.The query is executed on `k` fraction of data. For example, `SAMPLE 0.1` runs the query on 10% of data. [Read more](#select-sample-k) |
+| `SAMPLE n` | Here `n` is a sufficiently large integer.The query is executed on a sample of at least `n` rows (but not significantly more than this). For example, `SAMPLE 10000000` runs the query on a minimum of 10,000,000 rows. [Read more](#select-sample-n) |
+| `SAMPLE k OFFSET m` | Here `k` and `m` are the numbers from 0 to 1.The query is executed on a sample of `k` fraction of the data. The data used for the sample is offset by `m` fraction. [Read more](#select-sample-offset) |
+
+#### SAMPLE K {#select-sample-k}
+
+Here `k` is the number from 0 to 1 (both fractional and decimal notations are supported). For example, `SAMPLE 1/2` or `SAMPLE 0.5`.
+
+In a `SAMPLE k` clause, the sample is taken from the `k` fraction of data. The example is shown below:
+
+``` sql
+SELECT
+ Title,
+ count() * 10 AS PageViews
+FROM hits_distributed
+SAMPLE 0.1
+WHERE
+ CounterID = 34
+GROUP BY Title
+ORDER BY PageViews DESC LIMIT 1000
+```
+
+In this example, the query is executed on a sample from 0.1 (10%) of data. Values of aggregate functions are not corrected automatically, so to get an approximate result, the value `count()` is manually multiplied by 10.
+
+#### SAMPLE N {#select-sample-n}
+
+Here `n` is a sufficiently large integer. For example, `SAMPLE 10000000`.
+
+In this case, the query is executed on a sample of at least `n` rows (but not significantly more than this). For example, `SAMPLE 10000000` runs the query on a minimum of 10,000,000 rows.
+
+Since the minimum unit for data reading is one granule (its size is set by the `index_granularity` setting), it makes sense to set a sample that is much larger than the size of the granule.
+
+When using the `SAMPLE n` clause, you don’t know which relative percent of data was processed. So you don’t know the coefficient the aggregate functions should be multiplied by. Use the `_sample_factor` virtual column to get the approximate result.
+
+The `_sample_factor` column contains relative coefficients that are calculated dynamically. This column is created automatically when you [create](../../engines/table_engines/mergetree_family/mergetree.md#table_engine-mergetree-creating-a-table) a table with the specified sampling key. The usage examples of the `_sample_factor` column are shown below.
+
+Let’s consider the table `visits`, which contains the statistics about site visits. The first example shows how to calculate the number of page views:
+
+``` sql
+SELECT sum(PageViews * _sample_factor)
+FROM visits
+SAMPLE 10000000
+```
+
+The next example shows how to calculate the total number of visits:
+
+``` sql
+SELECT sum(_sample_factor)
+FROM visits
+SAMPLE 10000000
+```
+
+The example below shows how to calculate the average session duration. Note that you don’t need to use the relative coefficient to calculate the average values.
+
+``` sql
+SELECT avg(Duration)
+FROM visits
+SAMPLE 10000000
+```
+
+#### SAMPLE K OFFSET M {#select-sample-offset}
+
+Here `k` and `m` are numbers from 0 to 1. Examples are shown below.
+
+**Example 1**
+
+``` sql
+SAMPLE 1/10
+```
+
+In this example, the sample is 1/10th of all data:
+
+`[++------------]`
+
+**Example 2**
+
+``` sql
+SAMPLE 1/10 OFFSET 1/2
+```
+
+Here, a sample of 10% is taken from the second half of the data.
+
+`[------++------]`
+
+### ARRAY JOIN Clause {#select-array-join-clause}
+
+Allows executing `JOIN` with an array or nested data structure. The intent is similar to the [arrayJoin](../../sql_reference/functions/array_join.md#functions_arrayjoin) function, but its functionality is broader.
+
+``` sql
+SELECT
+FROM
+[LEFT] ARRAY JOIN
+[WHERE|PREWHERE ]
+...
+```
+
+You can specify only a single `ARRAY JOIN` clause in a query.
+
+The query execution order is optimized when running `ARRAY JOIN`. Although `ARRAY JOIN` must always be specified before the `WHERE/PREWHERE` clause, it can be performed either before `WHERE/PREWHERE` (if the result is needed in this clause), or after completing it (to reduce the volume of calculations). The processing order is controlled by the query optimizer.
+
+Supported types of `ARRAY JOIN` are listed below:
+
+- `ARRAY JOIN` - In this case, empty arrays are not included in the result of `JOIN`.
+- `LEFT ARRAY JOIN` - The result of `JOIN` contains rows with empty arrays. The value for an empty array is set to the default value for the array element type (usually 0, empty string or NULL).
+
+The examples below demonstrate the usage of the `ARRAY JOIN` and `LEFT ARRAY JOIN` clauses. Let’s create a table with an [Array](../../sql_reference/data_types/array.md) type column and insert values into it:
+
+``` sql
+CREATE TABLE arrays_test
+(
+ s String,
+ arr Array(UInt8)
+) ENGINE = Memory;
+
+INSERT INTO arrays_test
+VALUES ('Hello', [1,2]), ('World', [3,4,5]), ('Goodbye', []);
+```
+
+``` text
+┌─s───────────┬─arr─────┐
+│ Hello │ [1,2] │
+│ World │ [3,4,5] │
+│ Goodbye │ [] │
+└─────────────┴─────────┘
+```
+
+The example below uses the `ARRAY JOIN` clause:
+
+``` sql
+SELECT s, arr
+FROM arrays_test
+ARRAY JOIN arr;
+```
+
+``` text
+┌─s─────┬─arr─┐
+│ Hello │ 1 │
+│ Hello │ 2 │
+│ World │ 3 │
+│ World │ 4 │
+│ World │ 5 │
+└───────┴─────┘
+```
+
+The next example uses the `LEFT ARRAY JOIN` clause:
+
+``` sql
+SELECT s, arr
+FROM arrays_test
+LEFT ARRAY JOIN arr;
+```
+
+``` text
+┌─s───────────┬─arr─┐
+│ Hello │ 1 │
+│ Hello │ 2 │
+│ World │ 3 │
+│ World │ 4 │
+│ World │ 5 │
+│ Goodbye │ 0 │
+└─────────────┴─────┘
+```
+
+#### Using Aliases {#using-aliases}
+
+An alias can be specified for an array in the `ARRAY JOIN` clause. In this case, an array item can be accessed by this alias, but the array itself is accessed by the original name. Example:
+
+``` sql
+SELECT s, arr, a
+FROM arrays_test
+ARRAY JOIN arr AS a;
+```
+
+``` text
+┌─s─────┬─arr─────┬─a─┐
+│ Hello │ [1,2] │ 1 │
+│ Hello │ [1,2] │ 2 │
+│ World │ [3,4,5] │ 3 │
+│ World │ [3,4,5] │ 4 │
+│ World │ [3,4,5] │ 5 │
+└───────┴─────────┴───┘
+```
+
+Using aliases, you can perform `ARRAY JOIN` with an external array. For example:
+
+``` sql
+SELECT s, arr_external
+FROM arrays_test
+ARRAY JOIN [1, 2, 3] AS arr_external;
+```
+
+``` text
+┌─s───────────┬─arr_external─┐
+│ Hello │ 1 │
+│ Hello │ 2 │
+│ Hello │ 3 │
+│ World │ 1 │
+│ World │ 2 │
+│ World │ 3 │
+│ Goodbye │ 1 │
+│ Goodbye │ 2 │
+│ Goodbye │ 3 │
+└─────────────┴──────────────┘
+```
+
+Multiple arrays can be comma-separated in the `ARRAY JOIN` clause. In this case, `JOIN` is performed with them simultaneously (the direct sum, not the cartesian product). Note that all the arrays must have the same size. Example:
+
+``` sql
+SELECT s, arr, a, num, mapped
+FROM arrays_test
+ARRAY JOIN arr AS a, arrayEnumerate(arr) AS num, arrayMap(x -> x + 1, arr) AS mapped;
+```
+
+``` text
+┌─s─────┬─arr─────┬─a─┬─num─┬─mapped─┐
+│ Hello │ [1,2] │ 1 │ 1 │ 2 │
+│ Hello │ [1,2] │ 2 │ 2 │ 3 │
+│ World │ [3,4,5] │ 3 │ 1 │ 4 │
+│ World │ [3,4,5] │ 4 │ 2 │ 5 │
+│ World │ [3,4,5] │ 5 │ 3 │ 6 │
+└───────┴─────────┴───┴─────┴────────┘
+```
+
+The example below uses the [arrayEnumerate](../../sql_reference/functions/array_functions.md#array_functions-arrayenumerate) function:
+
+``` sql
+SELECT s, arr, a, num, arrayEnumerate(arr)
+FROM arrays_test
+ARRAY JOIN arr AS a, arrayEnumerate(arr) AS num;
+```
+
+``` text
+┌─s─────┬─arr─────┬─a─┬─num─┬─arrayEnumerate(arr)─┐
+│ Hello │ [1,2] │ 1 │ 1 │ [1,2] │
+│ Hello │ [1,2] │ 2 │ 2 │ [1,2] │
+│ World │ [3,4,5] │ 3 │ 1 │ [1,2,3] │
+│ World │ [3,4,5] │ 4 │ 2 │ [1,2,3] │
+│ World │ [3,4,5] │ 5 │ 3 │ [1,2,3] │
+└───────┴─────────┴───┴─────┴─────────────────────┘
+```
+
+#### ARRAY JOIN With Nested Data Structure {#array-join-with-nested-data-structure}
+
+`ARRAY`JOIN\`\` also works with [nested data structures](../../sql_reference/data_types/nested_data_structures/nested.md). Example:
+
+``` sql
+CREATE TABLE nested_test
+(
+ s String,
+ nest Nested(
+ x UInt8,
+ y UInt32)
+) ENGINE = Memory;
+
+INSERT INTO nested_test
+VALUES ('Hello', [1,2], [10,20]), ('World', [3,4,5], [30,40,50]), ('Goodbye', [], []);
+```
+
+``` text
+┌─s───────┬─nest.x──┬─nest.y─────┐
+│ Hello │ [1,2] │ [10,20] │
+│ World │ [3,4,5] │ [30,40,50] │
+│ Goodbye │ [] │ [] │
+└─────────┴─────────┴────────────┘
+```
+
+``` sql
+SELECT s, `nest.x`, `nest.y`
+FROM nested_test
+ARRAY JOIN nest;
+```
+
+``` text
+┌─s─────┬─nest.x─┬─nest.y─┐
+│ Hello │ 1 │ 10 │
+│ Hello │ 2 │ 20 │
+│ World │ 3 │ 30 │
+│ World │ 4 │ 40 │
+│ World │ 5 │ 50 │
+└───────┴────────┴────────┘
+```
+
+When specifying names of nested data structures in `ARRAY JOIN`, the meaning is the same as `ARRAY JOIN` with all the array elements that it consists of. Examples are listed below:
+
+``` sql
+SELECT s, `nest.x`, `nest.y`
+FROM nested_test
+ARRAY JOIN `nest.x`, `nest.y`;
+```
+
+``` text
+┌─s─────┬─nest.x─┬─nest.y─┐
+│ Hello │ 1 │ 10 │
+│ Hello │ 2 │ 20 │
+│ World │ 3 │ 30 │
+│ World │ 4 │ 40 │
+│ World │ 5 │ 50 │
+└───────┴────────┴────────┘
+```
+
+This variation also makes sense:
+
+``` sql
+SELECT s, `nest.x`, `nest.y`
+FROM nested_test
+ARRAY JOIN `nest.x`;
+```
+
+``` text
+┌─s─────┬─nest.x─┬─nest.y─────┐
+│ Hello │ 1 │ [10,20] │
+│ Hello │ 2 │ [10,20] │
+│ World │ 3 │ [30,40,50] │
+│ World │ 4 │ [30,40,50] │
+│ World │ 5 │ [30,40,50] │
+└───────┴────────┴────────────┘
+```
+
+An alias may be used for a nested data structure, in order to select either the `JOIN` result or the source array. Example:
+
+``` sql
+SELECT s, `n.x`, `n.y`, `nest.x`, `nest.y`
+FROM nested_test
+ARRAY JOIN nest AS n;
+```
+
+``` text
+┌─s─────┬─n.x─┬─n.y─┬─nest.x──┬─nest.y─────┐
+│ Hello │ 1 │ 10 │ [1,2] │ [10,20] │
+│ Hello │ 2 │ 20 │ [1,2] │ [10,20] │
+│ World │ 3 │ 30 │ [3,4,5] │ [30,40,50] │
+│ World │ 4 │ 40 │ [3,4,5] │ [30,40,50] │
+│ World │ 5 │ 50 │ [3,4,5] │ [30,40,50] │
+└───────┴─────┴─────┴─────────┴────────────┘
+```
+
+Example of using the [arrayEnumerate](../../sql_reference/functions/array_functions.md#array_functions-arrayenumerate) function:
+
+``` sql
+SELECT s, `n.x`, `n.y`, `nest.x`, `nest.y`, num
+FROM nested_test
+ARRAY JOIN nest AS n, arrayEnumerate(`nest.x`) AS num;
+```
+
+``` text
+┌─s─────┬─n.x─┬─n.y─┬─nest.x──┬─nest.y─────┬─num─┐
+│ Hello │ 1 │ 10 │ [1,2] │ [10,20] │ 1 │
+│ Hello │ 2 │ 20 │ [1,2] │ [10,20] │ 2 │
+│ World │ 3 │ 30 │ [3,4,5] │ [30,40,50] │ 1 │
+│ World │ 4 │ 40 │ [3,4,5] │ [30,40,50] │ 2 │
+│ World │ 5 │ 50 │ [3,4,5] │ [30,40,50] │ 3 │
+└───────┴─────┴─────┴─────────┴────────────┴─────┘
+```
+
+### JOIN Clause {#select-join}
+
+Joins the data in the normal [SQL JOIN](https://en.wikipedia.org/wiki/Join_(SQL)) sense.
+
+!!! info "Note"
+ Not related to [ARRAY JOIN](#select-array-join-clause).
+
+``` sql
+SELECT
+FROM
+[GLOBAL] [ANY|ALL] [INNER|LEFT|RIGHT|FULL|CROSS] [OUTER] JOIN
+(ON )|(USING ) ...
+```
+
+The table names can be specified instead of `` and ``. This is equivalent to the `SELECT * FROM table` subquery, except in a special case when the table has the [Join](../../engines/table_engines/special/join.md) engine – an array prepared for joining.
+
+#### Supported Types Of `JOIN` {#select-join-types}
+
+- `INNER JOIN` (or `JOIN`)
+- `LEFT JOIN` (or `LEFT OUTER JOIN`)
+- `RIGHT JOIN` (or `RIGHT OUTER JOIN`)
+- `FULL JOIN` (or `FULL OUTER JOIN`)
+- `CROSS JOIN` (or `,` )
+
+See the standard [SQL JOIN](https://en.wikipedia.org/wiki/Join_(SQL)) description.
+
+#### Multiple JOIN {#multiple-join}
+
+Performing queries, ClickHouse rewrites multi-table joins into the sequence of two-table joins. For example, if there are four tables for join ClickHouse joins the first and the second, then joins the result with the third table, and at the last step, it joins the fourth one.
+
+If a query contains the `WHERE` clause, ClickHouse tries to pushdown filters from this clause through the intermediate join. If it cannot apply the filter to each intermediate join, ClickHouse applies the filters after all joins are completed.
+
+We recommend the `JOIN ON` or `JOIN USING` syntax for creating queries. For example:
+
+``` sql
+SELECT * FROM t1 JOIN t2 ON t1.a = t2.a JOIN t3 ON t1.a = t3.a
+```
+
+You can use comma-separated lists of tables in the `FROM` clause. For example:
+
+``` sql
+SELECT * FROM t1, t2, t3 WHERE t1.a = t2.a AND t1.a = t3.a
+```
+
+Don’t mix these syntaxes.
+
+ClickHouse doesn’t directly support syntax with commas, so we don’t recommend using them. The algorithm tries to rewrite the query in terms of `CROSS JOIN` and `INNER JOIN` clauses and then proceeds to query processing. When rewriting the query, ClickHouse tries to optimize performance and memory consumption. By default, ClickHouse treats commas as an `INNER JOIN` clause and converts `INNER JOIN` to `CROSS JOIN` when the algorithm cannot guarantee that `INNER JOIN` returns the required data.
+
+#### Strictness {#select-join-strictness}
+
+- `ALL` — If the right table has several matching rows, ClickHouse creates a [Cartesian product](https://en.wikipedia.org/wiki/Cartesian_product) from matching rows. This is the standard `JOIN` behavior in SQL.
+- `ANY` — If the right table has several matching rows, only the first one found is joined. If the right table has only one matching row, the results of queries with `ANY` and `ALL` keywords are the same.
+- `ASOF` — For joining sequences with a non-exact match. `ASOF JOIN` usage is described below.
+
+**ASOF JOIN Usage**
+
+`ASOF JOIN` is useful when you need to join records that have no exact match.
+
+Tables for `ASOF JOIN` must have an ordered sequence column. This column cannot be alone in a table, and should be one of the data types: `UInt32`, `UInt64`, `Float32`, `Float64`, `Date`, and `DateTime`.
+
+Syntax `ASOF JOIN ... ON`:
+
+``` sql
+SELECT expressions_list
+FROM table_1
+ASOF LEFT JOIN table_2
+ON equi_cond AND closest_match_cond
+```
+
+You can use any number of equality conditions and exactly one closest match condition. For example, `SELECT count() FROM table_1 ASOF LEFT JOIN table_2 ON table_1.a == table_2.b AND table_2.t <= table_1.t`.
+
+Conditions supported for the closest match: `>`, `>=`, `<`, `<=`.
+
+Syntax `ASOF JOIN ... USING`:
+
+``` sql
+SELECT expressions_list
+FROM table_1
+ASOF JOIN table_2
+USING (equi_column1, ... equi_columnN, asof_column)
+```
+
+`ASOF JOIN` uses `equi_columnX` for joining on equality and `asof_column` for joining on the closest match with the `table_1.asof_column >= table_2.asof_column` condition. The `asof_column` column always the last one in the `USING` clause.
+
+For example, consider the following tables:
+
+\`\`\` text
+table\_1 table\_2
+
+event \| ev\_time \| user\_id event \| ev\_time \| user\_id
diff --git a/docs/en/query_language/show.md b/docs/en/sql_reference/statements/show.md
similarity index 88%
rename from docs/en/query_language/show.md
rename to docs/en/sql_reference/statements/show.md
index ca3366de80d..89572fc7118 100644
--- a/docs/en/query_language/show.md
+++ b/docs/en/sql_reference/statements/show.md
@@ -1,3 +1,8 @@
+---
+toc_priority: 38
+toc_title: SHOW
+---
+
# SHOW Queries {#show-queries}
## SHOW CREATE TABLE {#show-create-table}
@@ -23,7 +28,7 @@ This query is identical to `SELECT name FROM system.databases [INTO OUTFILE file
SHOW PROCESSLIST [INTO OUTFILE filename] [FORMAT format]
```
-Outputs the content of the [system.processes](../operations/system_tables.md#system_tables-processes) table, that contains a list of queries that is being processed at the moment, excepting `SHOW PROCESSLIST` queries.
+Outputs the content of the [system.processes](../../operations/system_tables.md#system_tables-processes) table, that contains a list of queries that is being processed at the moment, excepting `SHOW PROCESSLIST` queries.
The `SELECT * FROM system.processes` query returns data about all the current queries.
@@ -66,7 +71,7 @@ SHOW TABLES FROM system LIKE '%co%' LIMIT 2
## SHOW DICTIONARIES {#show-dictionaries}
-Displays a list of [external dictionaries](dicts/external_dicts.md).
+Displays a list of [external dictionaries](../../sql_reference/dictionaries/external_dictionaries/external_dicts.md).
``` sql
SHOW DICTIONARIES [FROM ] [LIKE ''] [LIMIT ] [INTO OUTFILE ] [FORMAT ]
diff --git a/docs/en/query_language/system.md b/docs/en/sql_reference/statements/system.md
similarity index 79%
rename from docs/en/query_language/system.md
rename to docs/en/sql_reference/statements/system.md
index 97f9dd710dd..ebd945f850e 100644
--- a/docs/en/query_language/system.md
+++ b/docs/en/sql_reference/statements/system.md
@@ -1,3 +1,8 @@
+---
+toc_priority: 37
+toc_title: SYSTEM
+---
+
# SYSTEM Queries {#query-language-system}
- [RELOAD DICTIONARIES](#query_language-system-reload-dictionaries)
@@ -17,10 +22,10 @@
## RELOAD DICTIONARIES {#query_language-system-reload-dictionaries}
Reloads all dictionaries that have been successfully loaded before.
-By default, dictionaries are loaded lazily (see [dictionaries\_lazy\_load](../operations/server_settings/settings.md#server_settings-dictionaries_lazy_load)), so instead of being loaded automatically at startup, they are initialized on first access through dictGet function or SELECT from tables with ENGINE = Dictionary. The `SYSTEM RELOAD DICTIONARIES` query reloads such dictionaries (LOADED).
+By default, dictionaries are loaded lazily (see [dictionaries\_lazy\_load](../../operations/server_configuration_parameters/settings.md#server_configuration_parameters-dictionaries_lazy_load)), so instead of being loaded automatically at startup, they are initialized on first access through dictGet function or SELECT from tables with ENGINE = Dictionary. The `SYSTEM RELOAD DICTIONARIES` query reloads such dictionaries (LOADED).
Always returns `Ok.` regardless of the result of the dictionary update.
-## RELOAD DICTIONARY dictionary\_name {#query_language-system-reload-dictionary}
+## RELOAD DICTIONARY Dictionary\_name {#query_language-system-reload-dictionary}
Completely reloads a dictionary `dictionary_name`, regardless of the state of the dictionary (LOADED / NOT\_LOADED / FAILED).
Always returns `Ok.` regardless of the result of updating the dictionary.
@@ -58,7 +63,7 @@ Aborts ClickHouse process (like `kill -9 {$ pid_clickhouse-server}`)
## Managing Distributed Tables {#query-language-system-distributed}
-ClickHouse can manage [distributed](../operations/table_engines/distributed.md) tables. When a user inserts data into these tables, ClickHouse first creates a queue of the data that should be sent to cluster nodes, then asynchronously sends it. You can manage queue processing with the [STOP DISTRIBUTED SENDS](#query_language-system-stop-distributed-sends), [FLUSH DISTRIBUTED](#query_language-system-flush-distributed), and [START DISTRIBUTED SENDS](#query_language-system-start-distributed-sends) queries. You can also synchronously insert distributed data with the `insert_distributed_sync` setting.
+ClickHouse can manage [distributed](../../engines/table_engines/special/distributed.md) tables. When a user inserts data into these tables, ClickHouse first creates a queue of the data that should be sent to cluster nodes, then asynchronously sends it. You can manage queue processing with the [STOP DISTRIBUTED SENDS](#query_language-system-stop-distributed-sends), [FLUSH DISTRIBUTED](#query_language-system-flush-distributed), and [START DISTRIBUTED SENDS](#query_language-system-start-distributed-sends) queries. You can also synchronously insert distributed data with the `insert_distributed_sync` setting.
### STOP DISTRIBUTED SENDS {#query_language-system-stop-distributed-sends}
diff --git a/docs/en/query_language/syntax.md b/docs/en/sql_reference/syntax.md
similarity index 95%
rename from docs/en/query_language/syntax.md
rename to docs/en/sql_reference/syntax.md
index 35a47dae1f1..12a4b9df7ef 100644
--- a/docs/en/query_language/syntax.md
+++ b/docs/en/sql_reference/syntax.md
@@ -1,3 +1,8 @@
+---
+toc_priority: 31
+toc_title: Syntax
+---
+
# Syntax {#syntax}
There are two types of parsers in the system: the full SQL parser (a recursive descent parser), and the data format parser (a fast stream parser).
@@ -69,13 +74,13 @@ A numeric literal tries to be parsed:
- Otherwise, an error is returned.
The corresponding value will have the smallest type that the value fits in.
-For example, 1 is parsed as `UInt8`, but 256 is parsed as `UInt16`. For more information, see [Data types](../data_types/index.md).
+For example, 1 is parsed as `UInt8`, but 256 is parsed as `UInt16`. For more information, see [Data types](../sql_reference/data_types/index.md).
Examples: `1`, `18446744073709551615`, `0xDEADBEEF`, `01`, `0.1`, `1e100`, `-1e-100`, `inf`, `nan`.
### String {#syntax-string-literal}
-Only string literals in single quotes are supported. The enclosed characters can be backslash-escaped. The following escape sequences have a corresponding special value: `\b`, `\f`, `\r`, `\n`, `\t`, `\0`, `\a`, `\v`, `\xHH`. In all other cases, escape sequences in the format `\c`, where `c` is any character, are converted to `c`. This means that you can use the sequences `\'`and`\\`. The value will have the [String](../data_types/string.md) type.
+Only string literals in single quotes are supported. The enclosed characters can be backslash-escaped. The following escape sequences have a corresponding special value: `\b`, `\f`, `\r`, `\n`, `\t`, `\0`, `\a`, `\v`, `\xHH`. In all other cases, escape sequences in the format `\c`, where `c` is any character, are converted to `c`. This means that you can use the sequences `\'`and`\\`. The value will have the [String](../sql_reference/data_types/string.md) type.
The minimum set of characters that you need to escape in string literals: `'` and `\`. Single quote can be escaped with the single quote, literals `'It\'s'` and `'It''s'` are equal.
@@ -84,13 +89,13 @@ The minimum set of characters that you need to escape in string literals: `'` an
Constructions are supported for arrays: `[1, 2, 3]` and tuples: `(1, 'Hello, world!', 2)`..
Actually, these are not literals, but expressions with the array creation operator and the tuple creation operator, respectively.
An array must consist of at least one item, and a tuple must have at least two items.
-Tuples have a special purpose for use in the `IN` clause of a `SELECT` query. Tuples can be obtained as the result of a query, but they can’t be saved to a database (with the exception of [Memory](../operations/table_engines/memory.md) tables).
+Tuples have a special purpose for use in the `IN` clause of a `SELECT` query. Tuples can be obtained as the result of a query, but they can’t be saved to a database (with the exception of [Memory](../engines/table_engines/special/memory.md) tables).
### NULL {#null-literal}
Indicates that the value is missing.
-In order to store `NULL` in a table field, it must be of the [Nullable](../data_types/nullable.md) type.
+In order to store `NULL` in a table field, it must be of the [Nullable](../sql_reference/data_types/nullable.md) type.
Depending on the data format (input or output), `NULL` may have a different representation. For more information, see the documentation for [data formats](../interfaces/formats.md#formats).
@@ -124,7 +129,7 @@ expr AS alias
For example, `SELECT table_name_alias.column_name FROM table_name table_name_alias`.
- In the [CAST](functions/type_conversion_functions.md#type_conversion_function-cast) function, the `AS` keyword has another meaning. See the description of the function.
+ In the [CAST](sql_reference/functions/type_conversion_functions.md#type_conversion_function-cast) function, the `AS` keyword has another meaning. See the description of the function.
- `expr` — Any expression supported by ClickHouse.
diff --git a/docs/en/query_language/table_functions/file.md b/docs/en/sql_reference/table_functions/file.md
similarity index 90%
rename from docs/en/query_language/table_functions/file.md
rename to docs/en/sql_reference/table_functions/file.md
index 12d30bdbdef..6ae7eb034f9 100644
--- a/docs/en/query_language/table_functions/file.md
+++ b/docs/en/sql_reference/table_functions/file.md
@@ -1,3 +1,8 @@
+---
+toc_priority: 37
+toc_title: file
+---
+
# file {#file}
Creates a table from a file. This table function is similar to [url](url.md) and [hdfs](hdfs.md) ones.
@@ -8,7 +13,7 @@ file(path, format, structure)
**Input parameters**
-- `path` — The relative path to the file from [user\_files\_path](../../operations/server_settings/settings.md#server_settings-user_files_path). Path to file support following globs in readonly mode: `*`, `?`, `{abc,def}` and `{N..M}` where `N`, `M` — numbers, \``'abc', 'def'` — strings.
+- `path` — The relative path to the file from [user\_files\_path](../../operations/server_configuration_parameters/settings.md#server_configuration_parameters-user_files_path). Path to file support following globs in readonly mode: `*`, `?`, `{abc,def}` and `{N..M}` where `N`, `M` — numbers, \``'abc', 'def'` — strings.
- `format` — The [format](../../interfaces/formats.md#formats) of the file.
- `structure` — Structure of the table. Format `'column1_name column1_type, column2_name column2_type, ...'`.
@@ -59,7 +64,7 @@ Multiple path components can have globs. For being processed file should exists
- `{some_string,another_string,yet_another_one}` — Substitutes any of strings `'some_string', 'another_string', 'yet_another_one'`.
- `{N..M}` — Substitutes any number in range from N to M including both borders.
-Constructions with `{}` are similar to the [remote table function](../../query_language/table_functions/remote.md)).
+Constructions with `{}` are similar to the [remote table function](../../sql_reference/table_functions/remote.md)).
**Example**
diff --git a/docs/en/query_language/table_functions/generate.md b/docs/en/sql_reference/table_functions/generate.md
similarity index 97%
rename from docs/en/query_language/table_functions/generate.md
rename to docs/en/sql_reference/table_functions/generate.md
index aef08495353..608d7c499ab 100644
--- a/docs/en/query_language/table_functions/generate.md
+++ b/docs/en/sql_reference/table_functions/generate.md
@@ -1,3 +1,8 @@
+---
+toc_priority: 47
+toc_title: generateRandom
+---
+
# generateRandom {#generaterandom}
Generates random data with given schema.
diff --git a/docs/en/query_language/table_functions/hdfs.md b/docs/en/sql_reference/table_functions/hdfs.md
similarity index 97%
rename from docs/en/query_language/table_functions/hdfs.md
rename to docs/en/sql_reference/table_functions/hdfs.md
index ca80d0e759c..730bd96758f 100644
--- a/docs/en/query_language/table_functions/hdfs.md
+++ b/docs/en/sql_reference/table_functions/hdfs.md
@@ -1,3 +1,8 @@
+---
+toc_priority: 45
+toc_title: hdfs
+---
+
# hdfs {#hdfs}
Creates a table from files in HDFS. This table function is similar to [url](url.md) and [file](file.md) ones.
@@ -42,7 +47,7 @@ Multiple path components can have globs. For being processed file should exists
- `{some_string,another_string,yet_another_one}` — Substitutes any of strings `'some_string', 'another_string', 'yet_another_one'`.
- `{N..M}` — Substitutes any number in range from N to M including both borders.
-Constructions with `{}` are similar to the [remote table function](../../query_language/table_functions/remote.md)).
+Constructions with `{}` are similar to the [remote table function](../../sql_reference/table_functions/remote.md)).
**Example**
diff --git a/docs/en/sql_reference/table_functions/index.md b/docs/en/sql_reference/table_functions/index.md
new file mode 100644
index 00000000000..18a17ce2f6f
--- /dev/null
+++ b/docs/en/sql_reference/table_functions/index.md
@@ -0,0 +1,36 @@
+---
+toc_folder_title: Table Functions
+toc_priority: 34
+toc_title: Introduction
+---
+
+# Table Functions {#table-functions}
+
+Table functions are methods for constructing tables.
+
+You can use table functions in:
+
+- [FROM](../statements/select.md#select-from) clause of the `SELECT` query.
+
+ The method for creating a temporary table that is available only in the current query. The table is deleted when the query finishes.
+
+- [CREATE TABLE AS \](../statements/create.md#create-table-query) query.
+
+ It's one of the methods of creating a table.
+
+!!! warning "Warning"
+ You can’t use table functions if the [allow\_ddl](../../operations/settings/permissions_for_queries.md#settings_allow_ddl) setting is disabled.
+
+| Function | Description |
+|---------------|----------------------------------------------------------------------------------|
+| [file](file.md) | Creates a [File](../../engines/table_engines/special/file.md)-engine table. |
+| [merge](merge.md) | Creates a [Merge](../../engines/table_engines/special/merge.md)-engine table. |
+| [numbers](numbers.md) | Creates a table with a single column filled with integer numbers. |
+| [remote](remote.md) | Allows you to access remote servers without creating a [Distributed](../../engines/table_engines/special/distributed.md)-engine table. |
+| [url](url.md) | Creates a [Url](../../engines/table_engines/special/url.md)-engine table. |
+| [mysql](mysql.md) | Creates a [MySQL](../../engines/table_engines/integrations/mysql.md)-engine table. |
+| [jdbc](jdbc.md) | Creates a [JDBC](../../engines/table_engines/integrations/jdbc.md)-engine table. |
+| [odbc](odbc.md) | Creates a [ODBC](../../engines/table_engines/integrations/odbc.md)-engine table. |
+| [hdfs](hdfs.md) | Creates a [HDFS](../../engines/table_engines/integrations/hdfs.md)-engine table. |
+
+[Original article](https://clickhouse.tech/docs/en/query_language/table_functions/)
diff --git a/docs/en/query_language/table_functions/input.md b/docs/en/sql_reference/table_functions/input.md
similarity index 97%
rename from docs/en/query_language/table_functions/input.md
rename to docs/en/sql_reference/table_functions/input.md
index f79a6f96d76..40f9f4f7f6f 100644
--- a/docs/en/query_language/table_functions/input.md
+++ b/docs/en/sql_reference/table_functions/input.md
@@ -1,3 +1,8 @@
+---
+toc_priority: 46
+toc_title: input
+---
+
# input {#input}
`input(structure)` - table function that allows effectively convert and insert data sent to the
diff --git a/docs/en/query_language/table_functions/jdbc.md b/docs/en/sql_reference/table_functions/jdbc.md
similarity index 94%
rename from docs/en/query_language/table_functions/jdbc.md
rename to docs/en/sql_reference/table_functions/jdbc.md
index a98fb1825e7..6fd53b0e794 100644
--- a/docs/en/query_language/table_functions/jdbc.md
+++ b/docs/en/sql_reference/table_functions/jdbc.md
@@ -1,3 +1,8 @@
+---
+toc_priority: 43
+toc_title: jdbc
+---
+
# jdbc {#table-function-jdbc}
`jdbc(jdbc_connection_uri, schema, table)` - returns table that is connected via JDBC driver.
diff --git a/docs/en/query_language/table_functions/merge.md b/docs/en/sql_reference/table_functions/merge.md
similarity index 89%
rename from docs/en/query_language/table_functions/merge.md
rename to docs/en/sql_reference/table_functions/merge.md
index 2cf87c40da0..7b3d88f6266 100644
--- a/docs/en/query_language/table_functions/merge.md
+++ b/docs/en/sql_reference/table_functions/merge.md
@@ -1,3 +1,8 @@
+---
+toc_priority: 38
+toc_title: merge
+---
+
# merge {#merge}
`merge(db_name, 'tables_regexp')` – Creates a temporary Merge table. For more information, see the section “Table engines, Merge”.
diff --git a/docs/en/query_language/table_functions/mysql.md b/docs/en/sql_reference/table_functions/mysql.md
similarity index 86%
rename from docs/en/query_language/table_functions/mysql.md
rename to docs/en/sql_reference/table_functions/mysql.md
index ae2590e9813..b4029c945d3 100644
--- a/docs/en/query_language/table_functions/mysql.md
+++ b/docs/en/sql_reference/table_functions/mysql.md
@@ -1,3 +1,8 @@
+---
+toc_priority: 42
+toc_title: mysql
+---
+
# mysql {#mysql}
Allows `SELECT` queries to be performed on data that is stored on a remote MySQL server.
@@ -51,11 +56,11 @@ mysql> insert into test (`int_id`, `float`) VALUES (1,2);
Query OK, 1 row affected (0,00 sec)
mysql> select * from test;
-+--------+--------------+-------+----------------+
++------+----------+-----+----------+
| int_id | int_nullable | float | float_nullable |
-+--------+--------------+-------+----------------+
++------+----------+-----+----------+
| 1 | NULL | 2 | NULL |
-+--------+--------------+-------+----------------+
++------+----------+-----+----------+
1 row in set (0,00 sec)
```
@@ -73,7 +78,7 @@ SELECT * FROM mysql('localhost:3306', 'test', 'test', 'bayonet', '123')
## See Also {#see-also}
-- [The ‘MySQL’ table engine](../../operations/table_engines/mysql.md)
-- [Using MySQL as a source of external dictionary](../dicts/external_dicts_dict_sources.md#dicts-external_dicts_dict_sources-mysql)
+- [The ‘MySQL’ table engine](../../engines/table_engines/integrations/mysql.md)
+- [Using MySQL as a source of external dictionary](../../sql_reference/dictionaries/external_dictionaries/external_dicts_dict_sources.md#dicts-external_dicts_dict_sources-mysql)
[Original article](https://clickhouse.tech/docs/en/query_language/table_functions/mysql/)
diff --git a/docs/en/query_language/table_functions/numbers.md b/docs/en/sql_reference/table_functions/numbers.md
similarity index 94%
rename from docs/en/query_language/table_functions/numbers.md
rename to docs/en/sql_reference/table_functions/numbers.md
index 4f3fe1218a5..53e4e42a2f8 100644
--- a/docs/en/query_language/table_functions/numbers.md
+++ b/docs/en/sql_reference/table_functions/numbers.md
@@ -1,3 +1,8 @@
+---
+toc_priority: 39
+toc_title: numbers
+---
+
# numbers {#numbers}
`numbers(N)` – Returns a table with the single ‘number’ column (UInt64) that contains integers from 0 to N-1.
diff --git a/docs/en/query_language/table_functions/odbc.md b/docs/en/sql_reference/table_functions/odbc.md
similarity index 88%
rename from docs/en/query_language/table_functions/odbc.md
rename to docs/en/sql_reference/table_functions/odbc.md
index 26112814fd3..f14e7808f79 100644
--- a/docs/en/query_language/table_functions/odbc.md
+++ b/docs/en/sql_reference/table_functions/odbc.md
@@ -1,3 +1,8 @@
+---
+toc_priority: 44
+toc_title: odbc
+---
+
# odbc {#table-functions-odbc}
Returns table that is connected via [ODBC](https://en.wikipedia.org/wiki/Open_Database_Connectivity).
@@ -52,7 +57,7 @@ You can check the connection using the `isql` utility from the unixODBC installa
``` bash
$ isql -v mysqlconn
-+---------------------------------------+
++-------------------------+
| Connected! |
| |
...
@@ -73,11 +78,11 @@ mysql> insert into test (`int_id`, `float`) VALUES (1,2);
Query OK, 1 row affected (0,00 sec)
mysql> select * from test;
-+--------+--------------+-------+----------------+
++------+----------+-----+----------+
| int_id | int_nullable | float | float_nullable |
-+--------+--------------+-------+----------------+
++------+----------+-----+----------+
| 1 | NULL | 2 | NULL |
-+--------+--------------+-------+----------------+
++------+----------+-----+----------+
1 row in set (0,00 sec)
```
@@ -95,7 +100,7 @@ SELECT * FROM odbc('DSN=mysqlconn', 'test', 'test')
## See Also {#see-also}
-- [ODBC external dictionaries](../../query_language/dicts/external_dicts_dict_sources.md#dicts-external_dicts_dict_sources-odbc)
-- [ODBC table engine](../../operations/table_engines/odbc.md).
+- [ODBC external dictionaries](../../sql_reference/dictionaries/external_dictionaries/external_dicts_dict_sources.md#dicts-external_dicts_dict_sources-odbc)
+- [ODBC table engine](../../engines/table_engines/integrations/odbc.md).
[Original article](https://clickhouse.tech/docs/en/query_language/table_functions/jdbc/)
diff --git a/docs/en/query_language/table_functions/remote.md b/docs/en/sql_reference/table_functions/remote.md
similarity index 94%
rename from docs/en/query_language/table_functions/remote.md
rename to docs/en/sql_reference/table_functions/remote.md
index 3d24229bca8..62ead0674b5 100644
--- a/docs/en/query_language/table_functions/remote.md
+++ b/docs/en/sql_reference/table_functions/remote.md
@@ -1,3 +1,8 @@
+---
+toc_priority: 40
+toc_title: remote
+---
+
# remote, remoteSecure {#remote-remotesecure}
Allows you to access remote servers without creating a `Distributed` table.
@@ -71,6 +76,6 @@ The `remote` table function can be useful in the following cases:
If the user is not specified, `default` is used.
If the password is not specified, an empty password is used.
-`remoteSecure` - same as `remote` but with secured connection. Default port — [tcp\_port\_secure](../../operations/server_settings/settings.md#server_settings-tcp_port_secure) from config or 9440.
+`remoteSecure` - same as `remote` but with secured connection. Default port — [tcp\_port\_secure](../../operations/server_configuration_parameters/settings.md#server_configuration_parameters-tcp_port_secure) from config or 9440.
[Original article](https://clickhouse.tech/docs/en/query_language/table_functions/remote/)
diff --git a/docs/en/query_language/table_functions/url.md b/docs/en/sql_reference/table_functions/url.md
similarity index 94%
rename from docs/en/query_language/table_functions/url.md
rename to docs/en/sql_reference/table_functions/url.md
index 0d4f85d649d..ea649e01994 100644
--- a/docs/en/query_language/table_functions/url.md
+++ b/docs/en/sql_reference/table_functions/url.md
@@ -1,3 +1,8 @@
+---
+toc_priority: 41
+toc_title: url
+---
+
# url {#url}
`url(URL, format, structure)` - returns a table created from the `URL` with given
diff --git a/docs/en/changelog/2017.md b/docs/en/whats_new/changelog/2017.md
similarity index 99%
rename from docs/en/changelog/2017.md
rename to docs/en/whats_new/changelog/2017.md
index 9bfdd83e381..68744e5334a 100644
--- a/docs/en/changelog/2017.md
+++ b/docs/en/whats_new/changelog/2017.md
@@ -1,3 +1,8 @@
+---
+toc_priority: 79
+toc_title: '2017'
+---
+
### ClickHouse release 1.1.54327, 2017-12-21 {#clickhouse-release-1-1-54327-2017-12-21}
This release contains bug fixes for the previous release 1.1.54318:
diff --git a/docs/en/changelog/2018.md b/docs/en/whats_new/changelog/2018.md
similarity index 99%
rename from docs/en/changelog/2018.md
rename to docs/en/whats_new/changelog/2018.md
index 7cb6ec834ad..f8eef5a911d 100644
--- a/docs/en/changelog/2018.md
+++ b/docs/en/whats_new/changelog/2018.md
@@ -1,3 +1,8 @@
+---
+toc_priority: 78
+toc_title: '2018'
+---
+
## ClickHouse release 18.16 {#clickhouse-release-18-16}
### ClickHouse release 18.16.1, 2018-12-21 {#clickhouse-release-18-16-1-2018-12-21}
diff --git a/docs/en/changelog/2019.md b/docs/en/whats_new/changelog/2019.md
similarity index 99%
rename from docs/en/changelog/2019.md
rename to docs/en/whats_new/changelog/2019.md
index 881078d81a1..ec2c14055cb 100644
--- a/docs/en/changelog/2019.md
+++ b/docs/en/whats_new/changelog/2019.md
@@ -1,3 +1,8 @@
+---
+toc_priority: 77
+toc_title: '2019'
+---
+
## ClickHouse release v19.17 {#clickhouse-release-v19-17}
### ClickHouse release v19.17.6.36, 2019-12-27 {#clickhouse-release-v19-17-6-36-2019-12-27}
diff --git a/docs/en/whats_new/changelog/index.md b/docs/en/whats_new/changelog/index.md
new file mode 100644
index 00000000000..bcfe62cbd0b
--- /dev/null
+++ b/docs/en/whats_new/changelog/index.md
@@ -0,0 +1,650 @@
+---
+toc_folder_title: Changelog
+toc_priority: 74
+toc_title: '2020'
+---
+
+## ClickHouse release v20.3
+
+### ClickHouse release v20.3.4.10, 2020-03-20
+
+#### Bug Fix
+* This release also contains all bug fixes from 20.1.8.41
+* Fix missing `rows_before_limit_at_least` for queries over http (with processors pipeline). This fixes [#9730](https://github.com/ClickHouse/ClickHouse/issues/9730). [#9757](https://github.com/ClickHouse/ClickHouse/pull/9757) ([Nikolai Kochetov](https://github.com/KochetovNicolai))
+
+
+### ClickHouse release v20.3.3.6, 2020-03-17
+
+#### Bug Fix
+* This release also contains all bug fixes from 20.1.7.38
+* Fix bug in a replication that doesn't allow replication to work if the user has executed mutations on the previous version. This fixes [#9645](https://github.com/ClickHouse/ClickHouse/issues/9645). [#9652](https://github.com/ClickHouse/ClickHouse/pull/9652) ([alesapin](https://github.com/alesapin)). It makes version 20.3 backward compatible again.
+* Add setting `use_compact_format_in_distributed_parts_names` which allows to write files for `INSERT` queries into `Distributed` table with more compact format. This fixes [#9647](https://github.com/ClickHouse/ClickHouse/issues/9647). [#9653](https://github.com/ClickHouse/ClickHouse/pull/9653) ([alesapin](https://github.com/alesapin)). It makes version 20.3 backward compatible again.
+
+### ClickHouse release v20.3.2.1, 2020-03-12
+
+#### Backward Incompatible Change
+
+* Fixed the issue `file name too long` when sending data for `Distributed` tables for a large number of replicas. Fixed the issue that replica credentials were exposed in the server log. The format of directory name on disk was changed to `[shard{shard_index}[_replica{replica_index}]]`. [#8911](https://github.com/ClickHouse/ClickHouse/pull/8911) ([Mikhail Korotov](https://github.com/millb)) After you upgrade to the new version, you will not be able to downgrade without manual intervention, because old server version does not recognize the new directory format. If you want to downgrade, you have to manually rename the corresponding directories to the old format. This change is relevant only if you have used asynchronous `INSERT`s to `Distributed` tables. In the version 20.3.3 we will introduce a setting that will allow you to enable the new format gradually.
+* Changed the format of replication log entries for mutation commands. You have to wait for old mutations to process before installing the new version.
+* Implement simple memory profiler that dumps stacktraces to `system.trace_log` every N bytes over soft allocation limit [#8765](https://github.com/ClickHouse/ClickHouse/pull/8765) ([Ivan](https://github.com/abyss7)) [#9472](https://github.com/ClickHouse/ClickHouse/pull/9472) ([alexey-milovidov](https://github.com/alexey-milovidov)) The column of `system.trace_log` was renamed from `timer_type` to `trace_type`. This will require changes in third-party performance analysis and flamegraph processing tools.
+* Use OS thread id everywhere instead of internal thread number. This fixes [#7477](https://github.com/ClickHouse/ClickHouse/issues/7477) Old `clickhouse-client` cannot receive logs that are send from the server when the setting `send_logs_level` is enabled, because the names and types of the structured log messages were changed. On the other hand, different server versions can send logs with different types to each other. When you don't use the `send_logs_level` setting, you should not care. [#8954](https://github.com/ClickHouse/ClickHouse/pull/8954) ([alexey-milovidov](https://github.com/alexey-milovidov))
+* Remove `indexHint` function [#9542](https://github.com/ClickHouse/ClickHouse/pull/9542) ([alexey-milovidov](https://github.com/alexey-milovidov))
+* Remove `findClusterIndex`, `findClusterValue` functions. This fixes [#8641](https://github.com/ClickHouse/ClickHouse/issues/8641). If you were using these functions, send an email to `clickhouse-feedback@yandex-team.com` [#9543](https://github.com/ClickHouse/ClickHouse/pull/9543) ([alexey-milovidov](https://github.com/alexey-milovidov))
+* Now it's not allowed to create columns or add columns with `SELECT` subquery as default expression. [#9481](https://github.com/ClickHouse/ClickHouse/pull/9481) ([alesapin](https://github.com/alesapin))
+* Require aliases for subqueries in JOIN. [#9274](https://github.com/ClickHouse/ClickHouse/pull/9274) ([Artem Zuikov](https://github.com/4ertus2))
+* Improved `ALTER MODIFY/ADD` queries logic. Now you cannot `ADD` column without type, `MODIFY` default expression doesn't change type of column and `MODIFY` type doesn't loose default expression value. Fixes [#8669](https://github.com/ClickHouse/ClickHouse/issues/8669). [#9227](https://github.com/ClickHouse/ClickHouse/pull/9227) ([alesapin](https://github.com/alesapin))
+* Require server to be restarted to apply the changes in logging configuration. This is a temporary workaround to avoid the bug where the server logs to a deleted log file (see [#8696](https://github.com/ClickHouse/ClickHouse/issues/8696)). [#8707](https://github.com/ClickHouse/ClickHouse/pull/8707) ([Alexander Kuzmenkov](https://github.com/akuzm))
+* The setting `experimental_use_processors` is enabled by default. This setting enables usage of the new query pipeline. This is internal refactoring and we expect no visible changes. If you will see any issues, set it to back zero. [#8768](https://github.com/ClickHouse/ClickHouse/pull/8768) ([alexey-milovidov](https://github.com/alexey-milovidov))
+
+#### New Feature
+* Add `Avro` and `AvroConfluent` input/output formats [#8571](https://github.com/ClickHouse/ClickHouse/pull/8571) ([Andrew Onyshchuk](https://github.com/oandrew)) [#8957](https://github.com/ClickHouse/ClickHouse/pull/8957) ([Andrew Onyshchuk](https://github.com/oandrew)) [#8717](https://github.com/ClickHouse/ClickHouse/pull/8717) ([alexey-milovidov](https://github.com/alexey-milovidov))
+* Multi-threaded and non-blocking updates of expired keys in `cache` dictionaries (with optional permission to read old ones). [#8303](https://github.com/ClickHouse/ClickHouse/pull/8303) ([Nikita Mikhaylov](https://github.com/nikitamikhaylov))
+* Add query `ALTER ... MATERIALIZE TTL`. It runs mutation that forces to remove expired data by TTL and recalculates meta-information about TTL in all parts. [#8775](https://github.com/ClickHouse/ClickHouse/pull/8775) ([Anton Popov](https://github.com/CurtizJ))
+* Switch from HashJoin to MergeJoin (on disk) if needed [#9082](https://github.com/ClickHouse/ClickHouse/pull/9082) ([Artem Zuikov](https://github.com/4ertus2))
+* Added `MOVE PARTITION` command for `ALTER TABLE` [#4729](https://github.com/ClickHouse/ClickHouse/issues/4729) [#6168](https://github.com/ClickHouse/ClickHouse/pull/6168) ([Guillaume Tassery](https://github.com/YiuRULE))
+* Reloading storage configuration from configuration file on the fly. [#8594](https://github.com/ClickHouse/ClickHouse/pull/8594) ([Vladimir Chebotarev](https://github.com/excitoon))
+* Allowed to change `storage_policy` to not less rich one. [#8107](https://github.com/ClickHouse/ClickHouse/pull/8107) ([Vladimir Chebotarev](https://github.com/excitoon))
+* Added support for globs/wildcards for S3 storage and table function. [#8851](https://github.com/ClickHouse/ClickHouse/pull/8851) ([Vladimir Chebotarev](https://github.com/excitoon))
+* Implement `bitAnd`, `bitOr`, `bitXor`, `bitNot` for `FixedString(N)` datatype. [#9091](https://github.com/ClickHouse/ClickHouse/pull/9091) ([Guillaume Tassery](https://github.com/YiuRULE))
+* Added function `bitCount`. This fixes [#8702](https://github.com/ClickHouse/ClickHouse/issues/8702). [#8708](https://github.com/ClickHouse/ClickHouse/pull/8708) ([alexey-milovidov](https://github.com/alexey-milovidov)) [#8749](https://github.com/ClickHouse/ClickHouse/pull/8749) ([ikopylov](https://github.com/ikopylov))
+* Add `generateRandom` table function to generate random rows with given schema. Allows to populate arbitrary test table with data. [#8994](https://github.com/ClickHouse/ClickHouse/pull/8994) ([Ilya Yatsishin](https://github.com/qoega))
+* `JSONEachRowFormat`: support special case when objects enclosed in top-level array. [#8860](https://github.com/ClickHouse/ClickHouse/pull/8860) ([Kruglov Pavel](https://github.com/Avogar))
+* Now it's possible to create a column with `DEFAULT` expression which depends on a column with default `ALIAS` expression. [#9489](https://github.com/ClickHouse/ClickHouse/pull/9489) ([alesapin](https://github.com/alesapin))
+* Allow to specify `--limit` more than the source data size in `clickhouse-obfuscator`. The data will repeat itself with different random seed. [#9155](https://github.com/ClickHouse/ClickHouse/pull/9155) ([alexey-milovidov](https://github.com/alexey-milovidov))
+* Added `groupArraySample` function (similar to `groupArray`) with reservior sampling algorithm. [#8286](https://github.com/ClickHouse/ClickHouse/pull/8286) ([Amos Bird](https://github.com/amosbird))
+* Now you can monitor the size of update queue in `cache`/`complex_key_cache` dictionaries via system metrics. [#9413](https://github.com/ClickHouse/ClickHouse/pull/9413) ([Nikita Mikhaylov](https://github.com/nikitamikhaylov))
+* Allow to use CRLF as a line separator in CSV output format with setting `output_format_csv_crlf_end_of_line` is set to 1 [#8934](https://github.com/ClickHouse/ClickHouse/pull/8934) [#8935](https://github.com/ClickHouse/ClickHouse/pull/8935) [#8963](https://github.com/ClickHouse/ClickHouse/pull/8963) ([Mikhail Korotov](https://github.com/millb))
+* Implement more functions of the [H3](https://github.com/uber/h3) API: `h3GetBaseCell`, `h3HexAreaM2`, `h3IndexesAreNeighbors`, `h3ToChildren`, `h3ToString` and `stringToH3` [#8938](https://github.com/ClickHouse/ClickHouse/pull/8938) ([Nico Mandery](https://github.com/nmandery))
+* New setting introduced: `max_parser_depth` to control maximum stack size and allow large complex queries. This fixes [#6681](https://github.com/ClickHouse/ClickHouse/issues/6681) and [#7668](https://github.com/ClickHouse/ClickHouse/issues/7668). [#8647](https://github.com/ClickHouse/ClickHouse/pull/8647) ([Maxim Smirnov](https://github.com/qMBQx8GH))
+* Add a setting `force_optimize_skip_unused_shards` setting to throw if skipping of unused shards is not possible [#8805](https://github.com/ClickHouse/ClickHouse/pull/8805) ([Azat Khuzhin](https://github.com/azat))
+* Allow to configure multiple disks/volumes for storing data for send in `Distributed` engine [#8756](https://github.com/ClickHouse/ClickHouse/pull/8756) ([Azat Khuzhin](https://github.com/azat))
+* Support storage policy (``) for storing temporary data. [#8750](https://github.com/ClickHouse/ClickHouse/pull/8750) ([Azat Khuzhin](https://github.com/azat))
+* Added `X-ClickHouse-Exception-Code` HTTP header that is set if exception was thrown before sending data. This implements [#4971](https://github.com/ClickHouse/ClickHouse/issues/4971). [#8786](https://github.com/ClickHouse/ClickHouse/pull/8786) ([Mikhail Korotov](https://github.com/millb))
+* Added function `ifNotFinite`. It is just a syntactic sugar: `ifNotFinite(x, y) = isFinite(x) ? x : y`. [#8710](https://github.com/ClickHouse/ClickHouse/pull/8710) ([alexey-milovidov](https://github.com/alexey-milovidov))
+* Added `last_successful_update_time` column in `system.dictionaries` table [#9394](https://github.com/ClickHouse/ClickHouse/pull/9394) ([Nikita Mikhaylov](https://github.com/nikitamikhaylov))
+* Add `blockSerializedSize` function (size on disk without compression) [#8952](https://github.com/ClickHouse/ClickHouse/pull/8952) ([Azat Khuzhin](https://github.com/azat))
+* Add function `moduloOrZero` [#9358](https://github.com/ClickHouse/ClickHouse/pull/9358) ([hcz](https://github.com/hczhcz))
+* Added system tables `system.zeros` and `system.zeros_mt` as well as tale functions `zeros()` and `zeros_mt()`. Tables (and table functions) contain single column with name `zero` and type `UInt8`. This column contains zeros. It is needed for test purposes as the fastest method to generate many rows. This fixes [#6604](https://github.com/ClickHouse/ClickHouse/issues/6604) [#9593](https://github.com/ClickHouse/ClickHouse/pull/9593) ([Nikolai Kochetov](https://github.com/KochetovNicolai))
+
+#### Experimental Feature
+* Add new compact format of parts in `MergeTree`-family tables in which all columns are stored in one file. It helps to increase performance of small and frequent inserts. The old format (one file per column) is now called wide. Data storing format is controlled by settings `min_bytes_for_wide_part` and `min_rows_for_wide_part`. [#8290](https://github.com/ClickHouse/ClickHouse/pull/8290) ([Anton Popov](https://github.com/CurtizJ))
+* Support for S3 storage for `Log`, `TinyLog` and `StripeLog` tables. [#8862](https://github.com/ClickHouse/ClickHouse/pull/8862) ([Pavel Kovalenko](https://github.com/Jokser))
+
+#### Bug Fix
+* Fixed inconsistent whitespaces in log messages. [#9322](https://github.com/ClickHouse/ClickHouse/pull/9322) ([alexey-milovidov](https://github.com/alexey-milovidov))
+* Fix bug in which arrays of unnamed tuples were flattened as Nested structures on table creation. [#8866](https://github.com/ClickHouse/ClickHouse/pull/8866) ([achulkov2](https://github.com/achulkov2))
+* Fixed the issue when "Too many open files" error may happen if there are too many files matching glob pattern in `File` table or `file` table function. Now files are opened lazily. This fixes [#8857](https://github.com/ClickHouse/ClickHouse/issues/8857) [#8861](https://github.com/ClickHouse/ClickHouse/pull/8861) ([alexey-milovidov](https://github.com/alexey-milovidov))
+* DROP TEMPORARY TABLE now drops only temporary table. [#8907](https://github.com/ClickHouse/ClickHouse/pull/8907) ([Vitaly Baranov](https://github.com/vitlibar))
+* Remove outdated partition when we shutdown the server or DETACH/ATTACH a table. [#8602](https://github.com/ClickHouse/ClickHouse/pull/8602) ([Guillaume Tassery](https://github.com/YiuRULE))
+* For how the default disk calculates the free space from `data` subdirectory. Fixed the issue when the amount of free space is not calculated correctly if the `data` directory is mounted to a separate device (rare case). This fixes [#7441](https://github.com/ClickHouse/ClickHouse/issues/7441) [#9257](https://github.com/ClickHouse/ClickHouse/pull/9257) ([Mikhail Korotov](https://github.com/millb))
+* Allow comma (cross) join with IN () inside. [#9251](https://github.com/ClickHouse/ClickHouse/pull/9251) ([Artem Zuikov](https://github.com/4ertus2))
+* Allow to rewrite CROSS to INNER JOIN if there's [NOT] LIKE operator in WHERE section. [#9229](https://github.com/ClickHouse/ClickHouse/pull/9229) ([Artem Zuikov](https://github.com/4ertus2))
+* Fix possible incorrect result after `GROUP BY` with enabled setting `distributed_aggregation_memory_efficient`. Fixes [#9134](https://github.com/ClickHouse/ClickHouse/issues/9134). [#9289](https://github.com/ClickHouse/ClickHouse/pull/9289) ([Nikolai Kochetov](https://github.com/KochetovNicolai))
+* Found keys were counted as missed in metrics of cache dictionaries. [#9411](https://github.com/ClickHouse/ClickHouse/pull/9411) ([Nikita Mikhaylov](https://github.com/nikitamikhaylov))
+* Fix replication protocol incompatibility introduced in [#8598](https://github.com/ClickHouse/ClickHouse/issues/8598). [#9412](https://github.com/ClickHouse/ClickHouse/pull/9412) ([alesapin](https://github.com/alesapin))
+* Fixed race condition on `queue_task_handle` at the startup of `ReplicatedMergeTree` tables. [#9552](https://github.com/ClickHouse/ClickHouse/pull/9552) ([alexey-milovidov](https://github.com/alexey-milovidov))
+* The token `NOT` didn't work in `SHOW TABLES NOT LIKE` query [#8727](https://github.com/ClickHouse/ClickHouse/issues/8727) [#8940](https://github.com/ClickHouse/ClickHouse/pull/8940) ([alexey-milovidov](https://github.com/alexey-milovidov))
+* Added range check to function `h3EdgeLengthM`. Without this check, buffer overflow is possible. [#8945](https://github.com/ClickHouse/ClickHouse/pull/8945) ([alexey-milovidov](https://github.com/alexey-milovidov))
+* Fixed up a bug in batched calculations of ternary logical OPs on multiple arguments (more than 10). [#8718](https://github.com/ClickHouse/ClickHouse/pull/8718) ([Alexander Kazakov](https://github.com/Akazz))
+* Fix error of PREWHERE optimization, which could lead to segfaults or `Inconsistent number of columns got from MergeTreeRangeReader` exception. [#9024](https://github.com/ClickHouse/ClickHouse/pull/9024) ([Anton Popov](https://github.com/CurtizJ))
+* Fix unexpected `Timeout exceeded while reading from socket` exception, which randomly happens on secure connection before timeout actually exceeded and when query profiler is enabled. Also add `connect_timeout_with_failover_secure_ms` settings (default 100ms), which is similar to `connect_timeout_with_failover_ms`, but is used for secure connections (because SSL handshake is slower, than ordinary TCP connection) [#9026](https://github.com/ClickHouse/ClickHouse/pull/9026) ([tavplubix](https://github.com/tavplubix))
+* Fix bug with mutations finalization, when mutation may hang in state with `parts_to_do=0` and `is_done=0`. [#9022](https://github.com/ClickHouse/ClickHouse/pull/9022) ([alesapin](https://github.com/alesapin))
+* Use new ANY JOIN logic with `partial_merge_join` setting. It's possible to make `ANY|ALL|SEMI LEFT` and `ALL INNER` joins with `partial_merge_join=1` now. [#8932](https://github.com/ClickHouse/ClickHouse/pull/8932) ([Artem Zuikov](https://github.com/4ertus2))
+* Shard now clamps the settings got from the initiator to the shard's constaints instead of throwing an exception. This fix allows to send queries to a shard with another constraints. [#9447](https://github.com/ClickHouse/ClickHouse/pull/9447) ([Vitaly Baranov](https://github.com/vitlibar))
+* Fixed memory management problem in `MergeTreeReadPool`. [#8791](https://github.com/ClickHouse/ClickHouse/pull/8791) ([Vladimir Chebotarev](https://github.com/excitoon))
+* Fix `toDecimal*OrNull()` functions family when called with string `e`. Fixes [#8312](https://github.com/ClickHouse/ClickHouse/issues/8312) [#8764](https://github.com/ClickHouse/ClickHouse/pull/8764) ([Artem Zuikov](https://github.com/4ertus2))
+* Make sure that `FORMAT Null` sends no data to the client. [#8767](https://github.com/ClickHouse/ClickHouse/pull/8767) ([Alexander Kuzmenkov](https://github.com/akuzm))
+* Fix bug that timestamp in `LiveViewBlockInputStream` will not updated. `LIVE VIEW` is an experimental feature. [#8644](https://github.com/ClickHouse/ClickHouse/pull/8644) ([vxider](https://github.com/Vxider)) [#8625](https://github.com/ClickHouse/ClickHouse/pull/8625) ([vxider](https://github.com/Vxider))
+* Fixed `ALTER MODIFY TTL` wrong behavior which did not allow to delete old TTL expressions. [#8422](https://github.com/ClickHouse/ClickHouse/pull/8422) ([Vladimir Chebotarev](https://github.com/excitoon))
+* Fixed UBSan report in MergeTreeIndexSet. This fixes [#9250](https://github.com/ClickHouse/ClickHouse/issues/9250) [#9365](https://github.com/ClickHouse/ClickHouse/pull/9365) ([alexey-milovidov](https://github.com/alexey-milovidov))
+* Fixed the behaviour of `match` and `extract` functions when haystack has zero bytes. The behaviour was wrong when haystack was constant. This fixes [#9160](https://github.com/ClickHouse/ClickHouse/issues/9160) [#9163](https://github.com/ClickHouse/ClickHouse/pull/9163) ([alexey-milovidov](https://github.com/alexey-milovidov)) [#9345](https://github.com/ClickHouse/ClickHouse/pull/9345) ([alexey-milovidov](https://github.com/alexey-milovidov))
+* Avoid throwing from destructor in Apache Avro 3rd-party library. [#9066](https://github.com/ClickHouse/ClickHouse/pull/9066) ([Andrew Onyshchuk](https://github.com/oandrew))
+* Don't commit a batch polled from `Kafka` partially as it can lead to holes in data. [#8876](https://github.com/ClickHouse/ClickHouse/pull/8876) ([filimonov](https://github.com/filimonov))
+* Fix `joinGet` with nullable return types. https://github.com/ClickHouse/ClickHouse/issues/8919 [#9014](https://github.com/ClickHouse/ClickHouse/pull/9014) ([Amos Bird](https://github.com/amosbird))
+* Fix data incompatibility when compressed with `T64` codec. [#9016](https://github.com/ClickHouse/ClickHouse/pull/9016) ([Artem Zuikov](https://github.com/4ertus2)) Fix data type ids in `T64` compression codec that leads to wrong (de)compression in affected versions. [#9033](https://github.com/ClickHouse/ClickHouse/pull/9033) ([Artem Zuikov](https://github.com/4ertus2))
+* Add setting `enable_early_constant_folding` and disable it in some cases that leads to errors. [#9010](https://github.com/ClickHouse/ClickHouse/pull/9010) ([Artem Zuikov](https://github.com/4ertus2))
+* Fix pushdown predicate optimizer with VIEW and enable the test [#9011](https://github.com/ClickHouse/ClickHouse/pull/9011) ([Winter Zhang](https://github.com/zhang2014))
+* Fix segfault in `Merge` tables, that can happen when reading from `File` storages [#9387](https://github.com/ClickHouse/ClickHouse/pull/9387) ([tavplubix](https://github.com/tavplubix))
+* Added a check for storage policy in `ATTACH PARTITION FROM`, `REPLACE PARTITION`, `MOVE TO TABLE`. Otherwise it could make data of part inaccessible after restart and prevent ClickHouse to start. [#9383](https://github.com/ClickHouse/ClickHouse/pull/9383) ([Vladimir Chebotarev](https://github.com/excitoon))
+* Fix alters if there is TTL set for table. [#8800](https://github.com/ClickHouse/ClickHouse/pull/8800) ([Anton Popov](https://github.com/CurtizJ))
+* Fix race condition that can happen when `SYSTEM RELOAD ALL DICTIONARIES` is executed while some dictionary is being modified/added/removed. [#8801](https://github.com/ClickHouse/ClickHouse/pull/8801) ([Vitaly Baranov](https://github.com/vitlibar))
+* In previous versions `Memory` database engine use empty data path, so tables are created in `path` directory (e.g. `/var/lib/clickhouse/`), not in data directory of database (e.g. `/var/lib/clickhouse/db_name`). [#8753](https://github.com/ClickHouse/ClickHouse/pull/8753) ([tavplubix](https://github.com/tavplubix))
+* Fixed wrong log messages about missing default disk or policy. [#9530](https://github.com/ClickHouse/ClickHouse/pull/9530) ([Vladimir Chebotarev](https://github.com/excitoon))
+* Fix not(has()) for the bloom_filter index of array types. [#9407](https://github.com/ClickHouse/ClickHouse/pull/9407) ([achimbab](https://github.com/achimbab))
+* Allow first column(s) in a table with `Log` engine be an alias [#9231](https://github.com/ClickHouse/ClickHouse/pull/9231) ([Ivan](https://github.com/abyss7))
+* Fix order of ranges while reading from `MergeTree` table in one thread. It could lead to exceptions from `MergeTreeRangeReader` or wrong query results. [#9050](https://github.com/ClickHouse/ClickHouse/pull/9050) ([Anton Popov](https://github.com/CurtizJ))
+* Make `reinterpretAsFixedString` to return `FixedString` instead of `String`. [#9052](https://github.com/ClickHouse/ClickHouse/pull/9052) ([Andrew Onyshchuk](https://github.com/oandrew))
+* Avoid extremely rare cases when the user can get wrong error message (`Success` instead of detailed error description). [#9457](https://github.com/ClickHouse/ClickHouse/pull/9457) ([alexey-milovidov](https://github.com/alexey-milovidov))
+* Do not crash when using `Template` format with empty row template. [#8785](https://github.com/ClickHouse/ClickHouse/pull/8785) ([Alexander Kuzmenkov](https://github.com/akuzm))
+* Metadata files for system tables could be created in wrong place [#8653](https://github.com/ClickHouse/ClickHouse/pull/8653) ([tavplubix](https://github.com/tavplubix)) Fixes [#8581](https://github.com/ClickHouse/ClickHouse/issues/8581).
+* Fix data race on exception_ptr in cache dictionary [#8303](https://github.com/ClickHouse/ClickHouse/issues/8303). [#9379](https://github.com/ClickHouse/ClickHouse/pull/9379) ([Nikita Mikhaylov](https://github.com/nikitamikhaylov))
+* Do not throw an exception for query `ATTACH TABLE IF NOT EXISTS`. Previously it was thrown if table already exists, despite the `IF NOT EXISTS` clause. [#8967](https://github.com/ClickHouse/ClickHouse/pull/8967) ([Anton Popov](https://github.com/CurtizJ))
+* Fixed missing closing paren in exception message. [#8811](https://github.com/ClickHouse/ClickHouse/pull/8811) ([alexey-milovidov](https://github.com/alexey-milovidov))
+* Avoid message `Possible deadlock avoided` at the startup of clickhouse-client in interactive mode. [#9455](https://github.com/ClickHouse/ClickHouse/pull/9455) ([alexey-milovidov](https://github.com/alexey-milovidov))
+* Fixed the issue when padding at the end of base64 encoded value can be malformed. Update base64 library. This fixes [#9491](https://github.com/ClickHouse/ClickHouse/issues/9491), closes [#9492](https://github.com/ClickHouse/ClickHouse/issues/9492) [#9500](https://github.com/ClickHouse/ClickHouse/pull/9500) ([alexey-milovidov](https://github.com/alexey-milovidov))
+* Prevent losing data in `Kafka` in rare cases when exception happens after reading suffix but before commit. Fixes [#9378](https://github.com/ClickHouse/ClickHouse/issues/9378) [#9507](https://github.com/ClickHouse/ClickHouse/pull/9507) ([filimonov](https://github.com/filimonov))
+* Fixed exception in `DROP TABLE IF EXISTS` [#8663](https://github.com/ClickHouse/ClickHouse/pull/8663) ([Nikita Vasilev](https://github.com/nikvas0))
+* Fix crash when a user tries to `ALTER MODIFY SETTING` for old-formated `MergeTree` table engines family. [#9435](https://github.com/ClickHouse/ClickHouse/pull/9435) ([alesapin](https://github.com/alesapin))
+* Support for UInt64 numbers that don't fit in Int64 in JSON-related functions. Update SIMDJSON to master. This fixes [#9209](https://github.com/ClickHouse/ClickHouse/issues/9209) [#9344](https://github.com/ClickHouse/ClickHouse/pull/9344) ([alexey-milovidov](https://github.com/alexey-milovidov))
+* Fixed execution of inversed predicates when non-strictly monotinic functional index is used. [#9223](https://github.com/ClickHouse/ClickHouse/pull/9223) ([Alexander Kazakov](https://github.com/Akazz))
+* Don't try to fold `IN` constant in `GROUP BY` [#8868](https://github.com/ClickHouse/ClickHouse/pull/8868) ([Amos Bird](https://github.com/amosbird))
+* Fix bug in `ALTER DELETE` mutations which leads to index corruption. This fixes [#9019](https://github.com/ClickHouse/ClickHouse/issues/9019) and [#8982](https://github.com/ClickHouse/ClickHouse/issues/8982). Additionally fix extremely rare race conditions in `ReplicatedMergeTree` `ALTER` queries. [#9048](https://github.com/ClickHouse/ClickHouse/pull/9048) ([alesapin](https://github.com/alesapin))
+* When the setting `compile_expressions` is enabled, you can get `unexpected column` in `LLVMExecutableFunction` when we use `Nullable` type [#8910](https://github.com/ClickHouse/ClickHouse/pull/8910) ([Guillaume Tassery](https://github.com/YiuRULE))
+* Multiple fixes for `Kafka` engine: 1) fix duplicates that were appearing during consumer group rebalance. 2) Fix rare 'holes' appeared when data were polled from several partitions with one poll and committed partially (now we always process / commit the whole polled block of messages). 3) Fix flushes by block size (before that only flushing by timeout was working properly). 4) better subscription procedure (with assignment feedback). 5) Make tests work faster (with default intervals and timeouts). Due to the fact that data was not flushed by block size before (as it should according to documentation), that PR may lead to some performance degradation with default settings (due to more often & tinier flushes which are less optimal). If you encounter the performance issue after that change - please increase `kafka_max_block_size` in the table to the bigger value ( for example `CREATE TABLE ...Engine=Kafka ... SETTINGS ... kafka_max_block_size=524288`). Fixes [#7259](https://github.com/ClickHouse/ClickHouse/issues/7259) [#8917](https://github.com/ClickHouse/ClickHouse/pull/8917) ([filimonov](https://github.com/filimonov))
+* Fix `Parameter out of bound` exception in some queries after PREWHERE optimizations. [#8914](https://github.com/ClickHouse/ClickHouse/pull/8914) ([Baudouin Giard](https://github.com/bgiard))
+* Fixed the case of mixed-constness of arguments of function `arrayZip`. [#8705](https://github.com/ClickHouse/ClickHouse/pull/8705) ([alexey-milovidov](https://github.com/alexey-milovidov))
+* When executing `CREATE` query, fold constant expressions in storage engine arguments. Replace empty database name with current database. Fixes [#6508](https://github.com/ClickHouse/ClickHouse/issues/6508), [#3492](https://github.com/ClickHouse/ClickHouse/issues/3492) [#9262](https://github.com/ClickHouse/ClickHouse/pull/9262) ([tavplubix](https://github.com/tavplubix))
+* Now it's not possible to create or add columns with simple cyclic aliases like `a DEFAULT b, b DEFAULT a`. [#9603](https://github.com/ClickHouse/ClickHouse/pull/9603) ([alesapin](https://github.com/alesapin))
+* Fixed a bug with double move which may corrupt original part. This is relevant if you use `ALTER TABLE MOVE` [#8680](https://github.com/ClickHouse/ClickHouse/pull/8680) ([Vladimir Chebotarev](https://github.com/excitoon))
+* Allow `interval` identifier to correctly parse without backticks. Fixed issue when a query cannot be executed even if the `interval` identifier is enclosed in backticks or double quotes. This fixes [#9124](https://github.com/ClickHouse/ClickHouse/issues/9124). [#9142](https://github.com/ClickHouse/ClickHouse/pull/9142) ([alexey-milovidov](https://github.com/alexey-milovidov))
+* Fixed fuzz test and incorrect behaviour of `bitTestAll`/`bitTestAny` functions. [#9143](https://github.com/ClickHouse/ClickHouse/pull/9143) ([alexey-milovidov](https://github.com/alexey-milovidov))
+* Fix possible crash/wrong number of rows in `LIMIT n WITH TIES` when there are a lot of rows equal to n'th row. [#9464](https://github.com/ClickHouse/ClickHouse/pull/9464) ([tavplubix](https://github.com/tavplubix))
+* Fix mutations with parts written with enabled `insert_quorum`. [#9463](https://github.com/ClickHouse/ClickHouse/pull/9463) ([alesapin](https://github.com/alesapin))
+* Fix data race at destruction of `Poco::HTTPServer`. It could happen when server is started and immediately shut down. [#9468](https://github.com/ClickHouse/ClickHouse/pull/9468) ([Anton Popov](https://github.com/CurtizJ))
+* Fix bug in which a misleading error message was shown when running `SHOW CREATE TABLE a_table_that_does_not_exist`. [#8899](https://github.com/ClickHouse/ClickHouse/pull/8899) ([achulkov2](https://github.com/achulkov2))
+* Fixed `Parameters are out of bound` exception in some rare cases when we have a constant in the `SELECT` clause when we have an `ORDER BY` and a `LIMIT` clause. [#8892](https://github.com/ClickHouse/ClickHouse/pull/8892) ([Guillaume Tassery](https://github.com/YiuRULE))
+* Fix mutations finalization, when already done mutation can have status `is_done=0`. [#9217](https://github.com/ClickHouse/ClickHouse/pull/9217) ([alesapin](https://github.com/alesapin))
+* Prevent from executing `ALTER ADD INDEX` for MergeTree tables with old syntax, because it doesn't work. [#8822](https://github.com/ClickHouse/ClickHouse/pull/8822) ([Mikhail Korotov](https://github.com/millb))
+* During server startup do not access table, which `LIVE VIEW` depends on, so server will be able to start. Also remove `LIVE VIEW` dependencies when detaching `LIVE VIEW`. `LIVE VIEW` is an experimental feature. [#8824](https://github.com/ClickHouse/ClickHouse/pull/8824) ([tavplubix](https://github.com/tavplubix))
+* Fix possible segfault in `MergeTreeRangeReader`, while executing `PREWHERE`. [#9106](https://github.com/ClickHouse/ClickHouse/pull/9106) ([Anton Popov](https://github.com/CurtizJ))
+* Fix possible mismatched checksums with column TTLs. [#9451](https://github.com/ClickHouse/ClickHouse/pull/9451) ([Anton Popov](https://github.com/CurtizJ))
+* Fixed a bug when parts were not being moved in background by TTL rules in case when there is only one volume. [#8672](https://github.com/ClickHouse/ClickHouse/pull/8672) ([Vladimir Chebotarev](https://github.com/excitoon))
+* Fixed the issue `Method createColumn() is not implemented for data type Set`. This fixes [#7799](https://github.com/ClickHouse/ClickHouse/issues/7799). [#8674](https://github.com/ClickHouse/ClickHouse/pull/8674) ([alexey-milovidov](https://github.com/alexey-milovidov))
+* Now we will try finalize mutations more frequently. [#9427](https://github.com/ClickHouse/ClickHouse/pull/9427) ([alesapin](https://github.com/alesapin))
+* Fix `intDiv` by minus one constant [#9351](https://github.com/ClickHouse/ClickHouse/pull/9351) ([hcz](https://github.com/hczhcz))
+* Fix possible race condition in `BlockIO`. [#9356](https://github.com/ClickHouse/ClickHouse/pull/9356) ([Nikolai Kochetov](https://github.com/KochetovNicolai))
+* Fix bug leading to server termination when trying to use / drop `Kafka` table created with wrong parameters. [#9513](https://github.com/ClickHouse/ClickHouse/pull/9513) ([filimonov](https://github.com/filimonov))
+* Added workaround if OS returns wrong result for `timer_create` function. [#8837](https://github.com/ClickHouse/ClickHouse/pull/8837) ([alexey-milovidov](https://github.com/alexey-milovidov))
+* Fixed error in usage of `min_marks_for_seek` parameter. Fixed the error message when there is no sharding key in Distributed table and we try to skip unused shards. [#8908](https://github.com/ClickHouse/ClickHouse/pull/8908) ([Azat Khuzhin](https://github.com/azat))
+
+#### Improvement
+* Implement `ALTER MODIFY/DROP` queries on top of mutations for `ReplicatedMergeTree*` engines family. Now `ALTERS` blocks only at the metadata update stage, and don't block after that. [#8701](https://github.com/ClickHouse/ClickHouse/pull/8701) ([alesapin](https://github.com/alesapin))
+* Add ability to rewrite CROSS to INNER JOINs with `WHERE` section containing unqialified names. [#9512](https://github.com/ClickHouse/ClickHouse/pull/9512) ([Artem Zuikov](https://github.com/4ertus2))
+* Make `SHOW TABLES` and `SHOW DATABASES` queries support the `WHERE` expressions and `FROM`/`IN` [#9076](https://github.com/ClickHouse/ClickHouse/pull/9076) ([sundyli](https://github.com/sundy-li))
+* Added a setting `deduplicate_blocks_in_dependent_materialized_views`. [#9070](https://github.com/ClickHouse/ClickHouse/pull/9070) ([urykhy](https://github.com/urykhy))
+* After recent changes MySQL client started to print binary strings in hex thereby making them not readable ([#9032](https://github.com/ClickHouse/ClickHouse/issues/9032)). The workaround in ClickHouse is to mark string columns as UTF-8, which is not always, but usually the case. [#9079](https://github.com/ClickHouse/ClickHouse/pull/9079) ([Yuriy Baranov](https://github.com/yurriy))
+* Add support of String and FixedString keys for `sumMap` [#8903](https://github.com/ClickHouse/ClickHouse/pull/8903) ([Baudouin Giard](https://github.com/bgiard))
+* Support string keys in SummingMergeTree maps [#8933](https://github.com/ClickHouse/ClickHouse/pull/8933) ([Baudouin Giard](https://github.com/bgiard))
+* Signal termination of thread to the thread pool even if the thread has thrown exception [#8736](https://github.com/ClickHouse/ClickHouse/pull/8736) ([Ding Xiang Fei](https://github.com/dingxiangfei2009))
+* Allow to set `query_id` in `clickhouse-benchmark` [#9416](https://github.com/ClickHouse/ClickHouse/pull/9416) ([Anton Popov](https://github.com/CurtizJ))
+* Don't allow strange expressions in `ALTER TABLE ... PARTITION partition` query. This addresses [#7192](https://github.com/ClickHouse/ClickHouse/issues/7192) [#8835](https://github.com/ClickHouse/ClickHouse/pull/8835) ([alexey-milovidov](https://github.com/alexey-milovidov))
+* The table `system.table_engines` now provides information about feature support (like `supports_ttl` or `supports_sort_order`). [#8830](https://github.com/ClickHouse/ClickHouse/pull/8830) ([Max Akhmedov](https://github.com/zlobober))
+* Enable `system.metric_log` by default. It will contain rows with values of ProfileEvents, CurrentMetrics collected with "collect_interval_milliseconds" interval (one second by default). The table is very small (usually in order of megabytes) and collecting this data by default is reasonable. [#9225](https://github.com/ClickHouse/ClickHouse/pull/9225) ([alexey-milovidov](https://github.com/alexey-milovidov))
+* Initialize query profiler for all threads in a group, e.g. it allows to fully profile insert-queries. Fixes [#6964](https://github.com/ClickHouse/ClickHouse/issues/6964) [#8874](https://github.com/ClickHouse/ClickHouse/pull/8874) ([Ivan](https://github.com/abyss7))
+* Now temporary `LIVE VIEW` is created by `CREATE LIVE VIEW name WITH TIMEOUT [42] ...` instead of `CREATE TEMPORARY LIVE VIEW ...`, because the previous syntax was not consistent with `CREATE TEMPORARY TABLE ...` [#9131](https://github.com/ClickHouse/ClickHouse/pull/9131) ([tavplubix](https://github.com/tavplubix))
+* Add text_log.level configuration parameter to limit entries that goes to `system.text_log` table [#8809](https://github.com/ClickHouse/ClickHouse/pull/8809) ([Azat Khuzhin](https://github.com/azat))
+* Allow to put downloaded part to a disks/volumes according to TTL rules [#8598](https://github.com/ClickHouse/ClickHouse/pull/8598) ([Vladimir Chebotarev](https://github.com/excitoon))
+* For external MySQL dictionaries, allow to mutualize MySQL connection pool to "share" them among dictionaries. This option significantly reduces the number of connections to MySQL servers. [#9409](https://github.com/ClickHouse/ClickHouse/pull/9409) ([Clément Rodriguez](https://github.com/clemrodriguez))
+* Show nearest query execution time for quantiles in `clickhouse-benchmark` output instead of interpolated values. It's better to show values that correspond to the execution time of some queries. [#8712](https://github.com/ClickHouse/ClickHouse/pull/8712) ([alexey-milovidov](https://github.com/alexey-milovidov))
+* Possibility to add key & timestamp for the message when inserting data to Kafka. Fixes [#7198](https://github.com/ClickHouse/ClickHouse/issues/7198) [#8969](https://github.com/ClickHouse/ClickHouse/pull/8969) ([filimonov](https://github.com/filimonov))
+* If server is run from terminal, highlight thread number, query id and log priority by colors. This is for improved readability of correlated log messages for developers. [#8961](https://github.com/ClickHouse/ClickHouse/pull/8961) ([alexey-milovidov](https://github.com/alexey-milovidov))
+* Better exception message while loading tables for `Ordinary` database. [#9527](https://github.com/ClickHouse/ClickHouse/pull/9527) ([alexey-milovidov](https://github.com/alexey-milovidov))
+* Implement `arraySlice` for arrays with aggregate function states. This fixes [#9388](https://github.com/ClickHouse/ClickHouse/issues/9388) [#9391](https://github.com/ClickHouse/ClickHouse/pull/9391) ([alexey-milovidov](https://github.com/alexey-milovidov))
+* Allow constant functions and constant arrays to be used on the right side of IN operator. [#8813](https://github.com/ClickHouse/ClickHouse/pull/8813) ([Anton Popov](https://github.com/CurtizJ))
+* If zookeeper exception has happened while fetching data for system.replicas, display it in a separate column. This implements [#9137](https://github.com/ClickHouse/ClickHouse/issues/9137) [#9138](https://github.com/ClickHouse/ClickHouse/pull/9138) ([alexey-milovidov](https://github.com/alexey-milovidov))
+* Atomically remove MergeTree data parts on destroy. [#8402](https://github.com/ClickHouse/ClickHouse/pull/8402) ([Vladimir Chebotarev](https://github.com/excitoon))
+* Support row-level security for Distributed tables. [#8926](https://github.com/ClickHouse/ClickHouse/pull/8926) ([Ivan](https://github.com/abyss7))
+* Now we recognize suffix (like KB, KiB...) in settings values. [#8072](https://github.com/ClickHouse/ClickHouse/pull/8072) ([Mikhail Korotov](https://github.com/millb))
+* Prevent out of memory while constructing result of a large JOIN. [#8637](https://github.com/ClickHouse/ClickHouse/pull/8637) ([Artem Zuikov](https://github.com/4ertus2))
+* Added names of clusters to suggestions in interactive mode in `clickhouse-client`. [#8709](https://github.com/ClickHouse/ClickHouse/pull/8709) ([alexey-milovidov](https://github.com/alexey-milovidov))
+* Initialize query profiler for all threads in a group, e.g. it allows to fully profile insert-queries [#8820](https://github.com/ClickHouse/ClickHouse/pull/8820) ([Ivan](https://github.com/abyss7))
+* Added column `exception_code` in `system.query_log` table. [#8770](https://github.com/ClickHouse/ClickHouse/pull/8770) ([Mikhail Korotov](https://github.com/millb))
+* Enabled MySQL compatibility server on port `9004` in the default server configuration file. Fixed password generation command in the example in configuration. [#8771](https://github.com/ClickHouse/ClickHouse/pull/8771) ([Yuriy Baranov](https://github.com/yurriy))
+* Prevent abort on shutdown if the filesystem is readonly. This fixes [#9094](https://github.com/ClickHouse/ClickHouse/issues/9094) [#9100](https://github.com/ClickHouse/ClickHouse/pull/9100) ([alexey-milovidov](https://github.com/alexey-milovidov))
+* Better exception message when length is required in HTTP POST query. [#9453](https://github.com/ClickHouse/ClickHouse/pull/9453) ([alexey-milovidov](https://github.com/alexey-milovidov))
+* Add `_path` and `_file` virtual columns to `HDFS` and `File` engines and `hdfs` and `file` table functions [#8489](https://github.com/ClickHouse/ClickHouse/pull/8489) ([Olga Khvostikova](https://github.com/stavrolia))
+* Fix error `Cannot find column` while inserting into `MATERIALIZED VIEW` in case if new column was added to view's internal table. [#8766](https://github.com/ClickHouse/ClickHouse/pull/8766) [#8788](https://github.com/ClickHouse/ClickHouse/pull/8788) ([vzakaznikov](https://github.com/vzakaznikov)) [#8788](https://github.com/ClickHouse/ClickHouse/issues/8788) [#8806](https://github.com/ClickHouse/ClickHouse/pull/8806) ([Nikolai Kochetov](https://github.com/KochetovNicolai)) [#8803](https://github.com/ClickHouse/ClickHouse/pull/8803) ([Nikolai Kochetov](https://github.com/KochetovNicolai))
+* Fix progress over native client-server protocol, by send progress after final update (like logs). This may be relevant only to some third-party tools that are using native protocol. [#9495](https://github.com/ClickHouse/ClickHouse/pull/9495) ([Azat Khuzhin](https://github.com/azat))
+* Add a system metric tracking the number of client connections using MySQL protocol ([#9013](https://github.com/ClickHouse/ClickHouse/issues/9013)). [#9015](https://github.com/ClickHouse/ClickHouse/pull/9015) ([Eugene Klimov](https://github.com/Slach))
+* From now on, HTTP responses will have `X-ClickHouse-Timezone` header set to the same timezone value that `SELECT timezone()` would report. [#9493](https://github.com/ClickHouse/ClickHouse/pull/9493) ([Denis Glazachev](https://github.com/traceon))
+
+#### Performance Improvement
+* Improve performance of analysing index with IN [#9261](https://github.com/ClickHouse/ClickHouse/pull/9261) ([Anton Popov](https://github.com/CurtizJ))
+* Simpler and more efficient code in Logical Functions + code cleanups. A followup to [#8718](https://github.com/ClickHouse/ClickHouse/issues/8718) [#8728](https://github.com/ClickHouse/ClickHouse/pull/8728) ([Alexander Kazakov](https://github.com/Akazz))
+* Overall performance improvement (in range of 5%..200% for affected queries) by ensuring even more strict aliasing with C++20 features. [#9304](https://github.com/ClickHouse/ClickHouse/pull/9304) ([Amos Bird](https://github.com/amosbird))
+* More strict aliasing for inner loops of comparison functions. [#9327](https://github.com/ClickHouse/ClickHouse/pull/9327) ([alexey-milovidov](https://github.com/alexey-milovidov))
+* More strict aliasing for inner loops of arithmetic functions. [#9325](https://github.com/ClickHouse/ClickHouse/pull/9325) ([alexey-milovidov](https://github.com/alexey-milovidov))
+* A ~3 times faster implementation for ColumnVector::replicate(), via which ColumnConst::convertToFullColumn() is implemented. Also will be useful in tests when materializing constants. [#9293](https://github.com/ClickHouse/ClickHouse/pull/9293) ([Alexander Kazakov](https://github.com/Akazz))
+* Another minor performance improvement to `ColumnVector::replicate()` (this speeds up the `materialize` function and higher order functions) an even further improvement to [#9293](https://github.com/ClickHouse/ClickHouse/issues/9293) [#9442](https://github.com/ClickHouse/ClickHouse/pull/9442) ([Alexander Kazakov](https://github.com/Akazz))
+* Improved performance of `stochasticLinearRegression` aggregate function. This patch is contributed by Intel. [#8652](https://github.com/ClickHouse/ClickHouse/pull/8652) ([alexey-milovidov](https://github.com/alexey-milovidov))
+* Improve performance of `reinterpretAsFixedString` function. [#9342](https://github.com/ClickHouse/ClickHouse/pull/9342) ([alexey-milovidov](https://github.com/alexey-milovidov))
+* Do not send blocks to client for `Null` format in processors pipeline. [#8797](https://github.com/ClickHouse/ClickHouse/pull/8797) ([Nikolai Kochetov](https://github.com/KochetovNicolai)) [#8767](https://github.com/ClickHouse/ClickHouse/pull/8767) ([Alexander Kuzmenkov](https://github.com/akuzm))
+
+#### Build/Testing/Packaging Improvement
+* Exception handling now works correctly on Windows Subsystem for Linux. See https://github.com/ClickHouse-Extras/libunwind/pull/3 This fixes [#6480](https://github.com/ClickHouse/ClickHouse/issues/6480) [#9564](https://github.com/ClickHouse/ClickHouse/pull/9564) ([sobolevsv](https://github.com/sobolevsv))
+* Replace `readline` with `replxx` for interactive line editing in `clickhouse-client` [#8416](https://github.com/ClickHouse/ClickHouse/pull/8416) ([Ivan](https://github.com/abyss7))
+* Better build time and less template instantiations in FunctionsComparison. [#9324](https://github.com/ClickHouse/ClickHouse/pull/9324) ([alexey-milovidov](https://github.com/alexey-milovidov))
+* Added integration with `clang-tidy` in CI. See also [#6044](https://github.com/ClickHouse/ClickHouse/issues/6044) [#9566](https://github.com/ClickHouse/ClickHouse/pull/9566) ([alexey-milovidov](https://github.com/alexey-milovidov))
+* Now we link ClickHouse in CI using `lld` even for `gcc`. [#9049](https://github.com/ClickHouse/ClickHouse/pull/9049) ([alesapin](https://github.com/alesapin))
+* Allow to randomize thread scheduling and insert glitches when `THREAD_FUZZER_*` environment variables are set. This helps testing. [#9459](https://github.com/ClickHouse/ClickHouse/pull/9459) ([alexey-milovidov](https://github.com/alexey-milovidov))
+* Enable secure sockets in stateless tests [#9288](https://github.com/ClickHouse/ClickHouse/pull/9288) ([tavplubix](https://github.com/tavplubix))
+* Make SPLIT_SHARED_LIBRARIES=OFF more robust [#9156](https://github.com/ClickHouse/ClickHouse/pull/9156) ([Azat Khuzhin](https://github.com/azat))
+* Make "performance_introspection_and_logging" test reliable to random server stuck. This may happen in CI environment. See also [#9515](https://github.com/ClickHouse/ClickHouse/issues/9515) [#9528](https://github.com/ClickHouse/ClickHouse/pull/9528) ([alexey-milovidov](https://github.com/alexey-milovidov))
+* Validate XML in style check. [#9550](https://github.com/ClickHouse/ClickHouse/pull/9550) ([alexey-milovidov](https://github.com/alexey-milovidov))
+* Fixed race condition in test `00738_lock_for_inner_table`. This test relied on sleep. [#9555](https://github.com/ClickHouse/ClickHouse/pull/9555) ([alexey-milovidov](https://github.com/alexey-milovidov))
+* Remove performance tests of type `once`. This is needed to run all performance tests in statistical comparison mode (more reliable). [#9557](https://github.com/ClickHouse/ClickHouse/pull/9557) ([alexey-milovidov](https://github.com/alexey-milovidov))
+* Added performance test for arithmetic functions. [#9326](https://github.com/ClickHouse/ClickHouse/pull/9326) ([alexey-milovidov](https://github.com/alexey-milovidov))
+* Added performance test for `sumMap` and `sumMapWithOverflow` aggregate functions. Follow-up for [#8933](https://github.com/ClickHouse/ClickHouse/issues/8933) [#8947](https://github.com/ClickHouse/ClickHouse/pull/8947) ([alexey-milovidov](https://github.com/alexey-milovidov))
+* Ensure style of ErrorCodes by style check. [#9370](https://github.com/ClickHouse/ClickHouse/pull/9370) ([alexey-milovidov](https://github.com/alexey-milovidov))
+* Add script for tests history. [#8796](https://github.com/ClickHouse/ClickHouse/pull/8796) ([alesapin](https://github.com/alesapin))
+* Add GCC warning `-Wsuggest-override` to locate and fix all places where `override` keyword must be used. [#8760](https://github.com/ClickHouse/ClickHouse/pull/8760) ([kreuzerkrieg](https://github.com/kreuzerkrieg))
+* Ignore weak symbol under Mac OS X because it must be defined [#9538](https://github.com/ClickHouse/ClickHouse/pull/9538) ([Deleted user](https://github.com/ghost))
+* Normalize running time of some queries in performance tests. This is done in preparation to run all the performance tests in comparison mode. [#9565](https://github.com/ClickHouse/ClickHouse/pull/9565) ([alexey-milovidov](https://github.com/alexey-milovidov))
+* Fix some tests to support pytest with query tests [#9062](https://github.com/ClickHouse/ClickHouse/pull/9062) ([Ivan](https://github.com/abyss7))
+* Enable SSL in build with MSan, so server will not fail at startup when running stateless tests [#9531](https://github.com/ClickHouse/ClickHouse/pull/9531) ([tavplubix](https://github.com/tavplubix))
+* Fix database substitution in test results [#9384](https://github.com/ClickHouse/ClickHouse/pull/9384) ([Ilya Yatsishin](https://github.com/qoega))
+* Build fixes for miscellaneous platforms [#9381](https://github.com/ClickHouse/ClickHouse/pull/9381) ([proller](https://github.com/proller)) [#8755](https://github.com/ClickHouse/ClickHouse/pull/8755) ([proller](https://github.com/proller)) [#8631](https://github.com/ClickHouse/ClickHouse/pull/8631) ([proller](https://github.com/proller))
+* Added disks section to stateless-with-coverage test docker image [#9213](https://github.com/ClickHouse/ClickHouse/pull/9213) ([Pavel Kovalenko](https://github.com/Jokser))
+* Get rid of in-source-tree files when building with GRPC [#9588](https://github.com/ClickHouse/ClickHouse/pull/9588) ([Amos Bird](https://github.com/amosbird))
+* Slightly faster build time by removing SessionCleaner from Context. Make the code of SessionCleaner more simple. [#9232](https://github.com/ClickHouse/ClickHouse/pull/9232) ([alexey-milovidov](https://github.com/alexey-milovidov))
+* Updated checking for hung queries in clickhouse-test script [#8858](https://github.com/ClickHouse/ClickHouse/pull/8858) ([Alexander Kazakov](https://github.com/Akazz))
+* Removed some useless files from repository. [#8843](https://github.com/ClickHouse/ClickHouse/pull/8843) ([alexey-milovidov](https://github.com/alexey-milovidov))
+* Changed type of math perftests from `once` to `loop`. [#8783](https://github.com/ClickHouse/ClickHouse/pull/8783) ([Nikolai Kochetov](https://github.com/KochetovNicolai))
+* Add docker image which allows to build interactive code browser HTML report for our codebase. [#8781](https://github.com/ClickHouse/ClickHouse/pull/8781) ([alesapin](https://github.com/alesapin)) See [Woboq Code Browser](https://clickhouse-test-reports.s3.yandex.net/codebrowser/html_report///ClickHouse/dbms/src/index.html)
+* Suppress some test failures under MSan. [#8780](https://github.com/ClickHouse/ClickHouse/pull/8780) ([Alexander Kuzmenkov](https://github.com/akuzm))
+* Speedup "exception while insert" test. This test often time out in debug-with-coverage build. [#8711](https://github.com/ClickHouse/ClickHouse/pull/8711) ([alexey-milovidov](https://github.com/alexey-milovidov))
+* Updated `libcxx` and `libcxxabi` to master. In preparation to [#9304](https://github.com/ClickHouse/ClickHouse/issues/9304) [#9308](https://github.com/ClickHouse/ClickHouse/pull/9308) ([alexey-milovidov](https://github.com/alexey-milovidov))
+* Fix flacky test `00910_zookeeper_test_alter_compression_codecs`. [#9525](https://github.com/ClickHouse/ClickHouse/pull/9525) ([alexey-milovidov](https://github.com/alexey-milovidov))
+* Clean up duplicated linker flags. Make sure the linker won't look up an unexpected symbol. [#9433](https://github.com/ClickHouse/ClickHouse/pull/9433) ([Amos Bird](https://github.com/amosbird))
+* Add `clickhouse-odbc` driver into test images. This allows to test interaction of ClickHouse with ClickHouse via its own ODBC driver. [#9348](https://github.com/ClickHouse/ClickHouse/pull/9348) ([filimonov](https://github.com/filimonov))
+* Fix several bugs in unit tests. [#9047](https://github.com/ClickHouse/ClickHouse/pull/9047) ([alesapin](https://github.com/alesapin))
+* Enable `-Wmissing-include-dirs` GCC warning to eliminate all non-existing includes - mostly as a result of CMake scripting errors [#8704](https://github.com/ClickHouse/ClickHouse/pull/8704) ([kreuzerkrieg](https://github.com/kreuzerkrieg))
+* Describe reasons if query profiler cannot work. This is intended for [#9049](https://github.com/ClickHouse/ClickHouse/issues/9049) [#9144](https://github.com/ClickHouse/ClickHouse/pull/9144) ([alexey-milovidov](https://github.com/alexey-milovidov))
+* Update OpenSSL to upstream master. Fixed the issue when TLS connections may fail with the message `OpenSSL SSL_read: error:14094438:SSL routines:ssl3_read_bytes:tlsv1 alert internal error` and `SSL Exception: error:2400006E:random number generator::error retrieving entropy`. The issue was present in version 20.1. [#8956](https://github.com/ClickHouse/ClickHouse/pull/8956) ([alexey-milovidov](https://github.com/alexey-milovidov))
+* Update Dockerfile for server [#8893](https://github.com/ClickHouse/ClickHouse/pull/8893) ([Ilya Mazaev](https://github.com/ne-ray))
+* Minor fixes in build-gcc-from-sources script [#8774](https://github.com/ClickHouse/ClickHouse/pull/8774) ([Michael Nacharov](https://github.com/mnach))
+* Replace `numbers` to `zeros` in perftests where `number` column is not used. This will lead to more clean test results. [#9600](https://github.com/ClickHouse/ClickHouse/pull/9600) ([Nikolai Kochetov](https://github.com/KochetovNicolai))
+* Fix stack overflow issue when using initializer_list in Column constructors. [#9367](https://github.com/ClickHouse/ClickHouse/pull/9367) ([Deleted user](https://github.com/ghost))
+* Upgrade librdkafka to v1.3.0. Enable bundled `rdkafka` and `gsasl` libraries on Mac OS X. [#9000](https://github.com/ClickHouse/ClickHouse/pull/9000) ([Andrew Onyshchuk](https://github.com/oandrew))
+* build fix on GCC 9.2.0 [#9306](https://github.com/ClickHouse/ClickHouse/pull/9306) ([vxider](https://github.com/Vxider))
+
+
+## ClickHouse release v20.1
+
+### ClickHouse release v20.1.8.41, 2020-03-20
+
+#### Bug Fix
+* Fix possible permanent `Cannot schedule a task` error (due to unhandled exception in `ParallelAggregatingBlockInputStream::Handler::onFinish/onFinishThread`). This fixes [#6833](https://github.com/ClickHouse/ClickHouse/issues/6833). [#9154](https://github.com/ClickHouse/ClickHouse/pull/9154) ([Azat Khuzhin](https://github.com/azat))
+* Fix excessive memory consumption in `ALTER` queries (mutations). This fixes [#9533](https://github.com/ClickHouse/ClickHouse/issues/9533) and [#9670](https://github.com/ClickHouse/ClickHouse/issues/9670). [#9754](https://github.com/ClickHouse/ClickHouse/pull/9754) ([alesapin](https://github.com/alesapin))
+* Fix bug in backquoting in external dictionaries DDL. This fixes [#9619](https://github.com/ClickHouse/ClickHouse/issues/9619). [#9734](https://github.com/ClickHouse/ClickHouse/pull/9734) ([alesapin](https://github.com/alesapin))
+
+### ClickHouse release v20.1.7.38, 2020-03-18
+
+#### Bug Fix
+* Fixed incorrect internal function names for `sumKahan` and `sumWithOverflow`. I lead to exception while using this functions in remote queries. [#9636](https://github.com/ClickHouse/ClickHouse/pull/9636) ([Azat Khuzhin](https://github.com/azat)). This issue was in all ClickHouse releases.
+* Allow `ALTER ON CLUSTER` of `Distributed` tables with internal replication. This fixes [#3268](https://github.com/ClickHouse/ClickHouse/issues/3268). [#9617](https://github.com/ClickHouse/ClickHouse/pull/9617) ([shinoi2](https://github.com/shinoi2)). This issue was in all ClickHouse releases.
+* Fix possible exceptions `Size of filter doesn't match size of column` and `Invalid number of rows in Chunk` in `MergeTreeRangeReader`. They could appear while executing `PREWHERE` in some cases. Fixes [#9132](https://github.com/ClickHouse/ClickHouse/issues/9132). [#9612](https://github.com/ClickHouse/ClickHouse/pull/9612) ([Anton Popov](https://github.com/CurtizJ))
+* Fixed the issue: timezone was not preserved if you write a simple arithmetic expression like `time + 1` (in contrast to an expression like `time + INTERVAL 1 SECOND`). This fixes [#5743](https://github.com/ClickHouse/ClickHouse/issues/5743). [#9323](https://github.com/ClickHouse/ClickHouse/pull/9323) ([alexey-milovidov](https://github.com/alexey-milovidov)). This issue was in all ClickHouse releases.
+* Now it's not possible to create or add columns with simple cyclic aliases like `a DEFAULT b, b DEFAULT a`. [#9603](https://github.com/ClickHouse/ClickHouse/pull/9603) ([alesapin](https://github.com/alesapin))
+* Fixed the issue when padding at the end of base64 encoded value can be malformed. Update base64 library. This fixes [#9491](https://github.com/ClickHouse/ClickHouse/issues/9491), closes [#9492](https://github.com/ClickHouse/ClickHouse/issues/9492) [#9500](https://github.com/ClickHouse/ClickHouse/pull/9500) ([alexey-milovidov](https://github.com/alexey-milovidov))
+* Fix data race at destruction of `Poco::HTTPServer`. It could happen when server is started and immediately shut down. [#9468](https://github.com/ClickHouse/ClickHouse/pull/9468) ([Anton Popov](https://github.com/CurtizJ))
+* Fix possible crash/wrong number of rows in `LIMIT n WITH TIES` when there are a lot of rows equal to n'th row. [#9464](https://github.com/ClickHouse/ClickHouse/pull/9464) ([tavplubix](https://github.com/tavplubix))
+* Fix possible mismatched checksums with column TTLs. [#9451](https://github.com/ClickHouse/ClickHouse/pull/9451) ([Anton Popov](https://github.com/CurtizJ))
+* Fix crash when a user tries to `ALTER MODIFY SETTING` for old-formated `MergeTree` table engines family. [#9435](https://github.com/ClickHouse/ClickHouse/pull/9435) ([alesapin](https://github.com/alesapin))
+* Now we will try finalize mutations more frequently. [#9427](https://github.com/ClickHouse/ClickHouse/pull/9427) ([alesapin](https://github.com/alesapin))
+* Fix replication protocol incompatibility introduced in [#8598](https://github.com/ClickHouse/ClickHouse/issues/8598). [#9412](https://github.com/ClickHouse/ClickHouse/pull/9412) ([alesapin](https://github.com/alesapin))
+* Fix not(has()) for the bloom_filter index of array types. [#9407](https://github.com/ClickHouse/ClickHouse/pull/9407) ([achimbab](https://github.com/achimbab))
+* Fixed the behaviour of `match` and `extract` functions when haystack has zero bytes. The behaviour was wrong when haystack was constant. This fixes [#9160](https://github.com/ClickHouse/ClickHouse/issues/9160) [#9163](https://github.com/ClickHouse/ClickHouse/pull/9163) ([alexey-milovidov](https://github.com/alexey-milovidov)) [#9345](https://github.com/ClickHouse/ClickHouse/pull/9345) ([alexey-milovidov](https://github.com/alexey-milovidov))
+
+#### Build/Testing/Packaging Improvement
+
+* Exception handling now works correctly on Windows Subsystem for Linux. See https://github.com/ClickHouse-Extras/libunwind/pull/3 This fixes [#6480](https://github.com/ClickHouse/ClickHouse/issues/6480) [#9564](https://github.com/ClickHouse/ClickHouse/pull/9564) ([sobolevsv](https://github.com/sobolevsv))
+
+
+### ClickHouse release v20.1.6.30, 2020-03-05
+
+#### Bug Fix
+
+* Fix data incompatibility when compressed with `T64` codec.
+[#9039](https://github.com/ClickHouse/ClickHouse/pull/9039) [(abyss7)](https://github.com/abyss7)
+* Fix order of ranges while reading from MergeTree table in one thread. Fixes [#8964](https://github.com/ClickHouse/ClickHouse/issues/8964).
+[#9050](https://github.com/ClickHouse/ClickHouse/pull/9050) [(CurtizJ)](https://github.com/CurtizJ)
+* Fix possible segfault in `MergeTreeRangeReader`, while executing `PREWHERE`. Fixes [#9064](https://github.com/ClickHouse/ClickHouse/issues/9064).
+[#9106](https://github.com/ClickHouse/ClickHouse/pull/9106) [(CurtizJ)](https://github.com/CurtizJ)
+* Fix `reinterpretAsFixedString` to return `FixedString` instead of `String`.
+[#9052](https://github.com/ClickHouse/ClickHouse/pull/9052) [(oandrew)](https://github.com/oandrew)
+* Fix `joinGet` with nullable return types. Fixes [#8919](https://github.com/ClickHouse/ClickHouse/issues/8919)
+[#9014](https://github.com/ClickHouse/ClickHouse/pull/9014) [(amosbird)](https://github.com/amosbird)
+* Fix fuzz test and incorrect behaviour of bitTestAll/bitTestAny functions.
+[#9143](https://github.com/ClickHouse/ClickHouse/pull/9143) [(alexey-milovidov)](https://github.com/alexey-milovidov)
+* Fix the behaviour of match and extract functions when haystack has zero bytes. The behaviour was wrong when haystack was constant. Fixes [#9160](https://github.com/ClickHouse/ClickHouse/issues/9160)
+[#9163](https://github.com/ClickHouse/ClickHouse/pull/9163) [(alexey-milovidov)](https://github.com/alexey-milovidov)
+* Fixed execution of inversed predicates when non-strictly monotinic functional index is used. Fixes [#9034](https://github.com/ClickHouse/ClickHouse/issues/9034)
+[#9223](https://github.com/ClickHouse/ClickHouse/pull/9223) [(Akazz)](https://github.com/Akazz)
+* Allow to rewrite `CROSS` to `INNER JOIN` if there's `[NOT] LIKE` operator in `WHERE` section. Fixes [#9191](https://github.com/ClickHouse/ClickHouse/issues/9191)
+[#9229](https://github.com/ClickHouse/ClickHouse/pull/9229) [(4ertus2)](https://github.com/4ertus2)
+* Allow first column(s) in a table with Log engine be an alias.
+[#9231](https://github.com/ClickHouse/ClickHouse/pull/9231) [(abyss7)](https://github.com/abyss7)
+* Allow comma join with `IN()` inside. Fixes [#7314](https://github.com/ClickHouse/ClickHouse/issues/7314).
+[#9251](https://github.com/ClickHouse/ClickHouse/pull/9251) [(4ertus2)](https://github.com/4ertus2)
+* Improve `ALTER MODIFY/ADD` queries logic. Now you cannot `ADD` column without type, `MODIFY` default expression doesn't change type of column and `MODIFY` type doesn't loose default expression value. Fixes [#8669](https://github.com/ClickHouse/ClickHouse/issues/8669).
+[#9227](https://github.com/ClickHouse/ClickHouse/pull/9227) [(alesapin)](https://github.com/alesapin)
+* Fix mutations finalization, when already done mutation can have status is_done=0.
+[#9217](https://github.com/ClickHouse/ClickHouse/pull/9217) [(alesapin)](https://github.com/alesapin)
+* Support "Processors" pipeline for system.numbers and system.numbers_mt. This also fixes the bug when `max_execution_time` is not respected.
+[#7796](https://github.com/ClickHouse/ClickHouse/pull/7796) [(KochetovNicolai)](https://github.com/KochetovNicolai)
+* Fix wrong counting of `DictCacheKeysRequestedFound` metric.
+[#9411](https://github.com/ClickHouse/ClickHouse/pull/9411) [(nikitamikhaylov)](https://github.com/nikitamikhaylov)
+* Added a check for storage policy in `ATTACH PARTITION FROM`, `REPLACE PARTITION`, `MOVE TO TABLE` which otherwise could make data of part inaccessible after restart and prevent ClickHouse to start.
+[#9383](https://github.com/ClickHouse/ClickHouse/pull/9383) [(excitoon)](https://github.com/excitoon)
+* Fixed UBSan report in `MergeTreeIndexSet`. This fixes [#9250](https://github.com/ClickHouse/ClickHouse/issues/9250)
+[#9365](https://github.com/ClickHouse/ClickHouse/pull/9365) [(alexey-milovidov)](https://github.com/alexey-milovidov)
+* Fix possible datarace in BlockIO.
+[#9356](https://github.com/ClickHouse/ClickHouse/pull/9356) [(KochetovNicolai)](https://github.com/KochetovNicolai)
+* Support for `UInt64` numbers that don't fit in Int64 in JSON-related functions. Update `SIMDJSON` to master. This fixes [#9209](https://github.com/ClickHouse/ClickHouse/issues/9209)
+[#9344](https://github.com/ClickHouse/ClickHouse/pull/9344) [(alexey-milovidov)](https://github.com/alexey-milovidov)
+* Fix the issue when the amount of free space is not calculated correctly if the data directory is mounted to a separate device. For default disk calculate the free space from data subdirectory. This fixes [#7441](https://github.com/ClickHouse/ClickHouse/issues/7441)
+[#9257](https://github.com/ClickHouse/ClickHouse/pull/9257) [(millb)](https://github.com/millb)
+* Fix the issue when TLS connections may fail with the message `OpenSSL SSL_read: error:14094438:SSL routines:ssl3_read_bytes:tlsv1 alert internal error and SSL Exception: error:2400006E:random number generator::error retrieving entropy.` Update OpenSSL to upstream master.
+[#8956](https://github.com/ClickHouse/ClickHouse/pull/8956) [(alexey-milovidov)](https://github.com/alexey-milovidov)
+* When executing `CREATE` query, fold constant expressions in storage engine arguments. Replace empty database name with current database. Fixes [#6508](https://github.com/ClickHouse/ClickHouse/issues/6508), [#3492](https://github.com/ClickHouse/ClickHouse/issues/3492). Also fix check for local address in ClickHouseDictionarySource.
+[#9262](https://github.com/ClickHouse/ClickHouse/pull/9262) [(tabplubix)](https://github.com/tavplubix)
+* Fix segfault in `StorageMerge`, which can happen when reading from StorageFile.
+[#9387](https://github.com/ClickHouse/ClickHouse/pull/9387) [(tabplubix)](https://github.com/tavplubix)
+* Prevent losing data in `Kafka` in rare cases when exception happens after reading suffix but before commit. Fixes [#9378](https://github.com/ClickHouse/ClickHouse/issues/9378). Related: [#7175](https://github.com/ClickHouse/ClickHouse/issues/7175)
+[#9507](https://github.com/ClickHouse/ClickHouse/pull/9507) [(filimonov)](https://github.com/filimonov)
+* Fix bug leading to server termination when trying to use / drop `Kafka` table created with wrong parameters. Fixes [#9494](https://github.com/ClickHouse/ClickHouse/issues/9494). Incorporates [#9507](https://github.com/ClickHouse/ClickHouse/issues/9507).
+[#9513](https://github.com/ClickHouse/ClickHouse/pull/9513) [(filimonov)](https://github.com/filimonov)
+
+#### New Feature
+* Add `deduplicate_blocks_in_dependent_materialized_views` option to control the behaviour of idempotent inserts into tables with materialized views. This new feature was added to the bugfix release by a special request from Altinity.
+[#9070](https://github.com/ClickHouse/ClickHouse/pull/9070) [(urykhy)](https://github.com/urykhy)
+
+### ClickHouse release v20.1.2.4, 2020-01-22
+
+#### Backward Incompatible Change
+* Make the setting `merge_tree_uniform_read_distribution` obsolete. The server still recognizes this setting but it has no effect. [#8308](https://github.com/ClickHouse/ClickHouse/pull/8308) ([alexey-milovidov](https://github.com/alexey-milovidov))
+* Changed return type of the function `greatCircleDistance` to `Float32` because now the result of calculation is `Float32`. [#7993](https://github.com/ClickHouse/ClickHouse/pull/7993) ([alexey-milovidov](https://github.com/alexey-milovidov))
+* Now it's expected that query parameters are represented in "escaped" format. For example, to pass string `ab` you have to write `a\tb` or `a\b` and respectively, `a%5Ctb` or `a%5C%09b` in URL. This is needed to add the possibility to pass NULL as `\N`. This fixes [#7488](https://github.com/ClickHouse/ClickHouse/issues/7488). [#8517](https://github.com/ClickHouse/ClickHouse/pull/8517) ([alexey-milovidov](https://github.com/alexey-milovidov))
+* Enable `use_minimalistic_part_header_in_zookeeper` setting for `ReplicatedMergeTree` by default. This will significantly reduce amount of data stored in ZooKeeper. This setting is supported since version 19.1 and we already use it in production in multiple services without any issues for more than half a year. Disable this setting if you have a chance to downgrade to versions older than 19.1. [#6850](https://github.com/ClickHouse/ClickHouse/pull/6850) ([alexey-milovidov](https://github.com/alexey-milovidov))
+* Data skipping indices are production ready and enabled by default. The settings `allow_experimental_data_skipping_indices`, `allow_experimental_cross_to_join_conversion` and `allow_experimental_multiple_joins_emulation` are now obsolete and do nothing. [#7974](https://github.com/ClickHouse/ClickHouse/pull/7974) ([alexey-milovidov](https://github.com/alexey-milovidov))
+* Add new `ANY JOIN` logic for `StorageJoin` consistent with `JOIN` operation. To upgrade without changes in behaviour you need add `SETTINGS any_join_distinct_right_table_keys = 1` to Engine Join tables metadata or recreate these tables after upgrade. [#8400](https://github.com/ClickHouse/ClickHouse/pull/8400) ([Artem Zuikov](https://github.com/4ertus2))
+* Require server to be restarted to apply the changes in logging configuration. This is a temporary workaround to avoid the bug where the server logs to a deleted log file (see [#8696](https://github.com/ClickHouse/ClickHouse/issues/8696)). [#8707](https://github.com/ClickHouse/ClickHouse/pull/8707) ([Alexander Kuzmenkov](https://github.com/akuzm))
+
+#### New Feature
+* Added information about part paths to `system.merges`. [#8043](https://github.com/ClickHouse/ClickHouse/pull/8043) ([Vladimir Chebotarev](https://github.com/excitoon))
+* Add ability to execute `SYSTEM RELOAD DICTIONARY` query in `ON CLUSTER` mode. [#8288](https://github.com/ClickHouse/ClickHouse/pull/8288) ([Guillaume Tassery](https://github.com/YiuRULE))
+* Add ability to execute `CREATE DICTIONARY` queries in `ON CLUSTER` mode. [#8163](https://github.com/ClickHouse/ClickHouse/pull/8163) ([alesapin](https://github.com/alesapin))
+* Now user's profile in `users.xml` can inherit multiple profiles. [#8343](https://github.com/ClickHouse/ClickHouse/pull/8343) ([Mikhail f. Shiryaev](https://github.com/Felixoid))
+* Added `system.stack_trace` table that allows to look at stack traces of all server threads. This is useful for developers to introspect server state. This fixes [#7576](https://github.com/ClickHouse/ClickHouse/issues/7576). [#8344](https://github.com/ClickHouse/ClickHouse/pull/8344) ([alexey-milovidov](https://github.com/alexey-milovidov))
+* Add `DateTime64` datatype with configurable sub-second precision. [#7170](https://github.com/ClickHouse/ClickHouse/pull/7170) ([Vasily Nemkov](https://github.com/Enmk))
+* Add table function `clusterAllReplicas` which allows to query all the nodes in the cluster. [#8493](https://github.com/ClickHouse/ClickHouse/pull/8493) ([kiran sunkari](https://github.com/kiransunkari))
+* Add aggregate function `categoricalInformationValue` which calculates the information value of a discrete feature. [#8117](https://github.com/ClickHouse/ClickHouse/pull/8117) ([hcz](https://github.com/hczhcz))
+* Speed up parsing of data files in `CSV`, `TSV` and `JSONEachRow` format by doing it in parallel. [#7780](https://github.com/ClickHouse/ClickHouse/pull/7780) ([Alexander Kuzmenkov](https://github.com/akuzm))
+* Add function `bankerRound` which performs banker's rounding. [#8112](https://github.com/ClickHouse/ClickHouse/pull/8112) ([hcz](https://github.com/hczhcz))
+* Support more languages in embedded dictionary for region names: 'ru', 'en', 'ua', 'uk', 'by', 'kz', 'tr', 'de', 'uz', 'lv', 'lt', 'et', 'pt', 'he', 'vi'. [#8189](https://github.com/ClickHouse/ClickHouse/pull/8189) ([alexey-milovidov](https://github.com/alexey-milovidov))
+* Improvements in consistency of `ANY JOIN` logic. Now `t1 ANY LEFT JOIN t2` equals `t2 ANY RIGHT JOIN t1`. [#7665](https://github.com/ClickHouse/ClickHouse/pull/7665) ([Artem Zuikov](https://github.com/4ertus2))
+* Add setting `any_join_distinct_right_table_keys` which enables old behaviour for `ANY INNER JOIN`. [#7665](https://github.com/ClickHouse/ClickHouse/pull/7665) ([Artem Zuikov](https://github.com/4ertus2))
+* Add new `SEMI` and `ANTI JOIN`. Old `ANY INNER JOIN` behaviour now available as `SEMI LEFT JOIN`. [#7665](https://github.com/ClickHouse/ClickHouse/pull/7665) ([Artem Zuikov](https://github.com/4ertus2))
+* Added `Distributed` format for `File` engine and `file` table function which allows to read from `.bin` files generated by asynchronous inserts into `Distributed` table. [#8535](https://github.com/ClickHouse/ClickHouse/pull/8535) ([Nikolai Kochetov](https://github.com/KochetovNicolai))
+* Add optional reset column argument for `runningAccumulate` which allows to reset aggregation results for each new key value. [#8326](https://github.com/ClickHouse/ClickHouse/pull/8326) ([Sergey Kononenko](https://github.com/kononencheg))
+* Add ability to use ClickHouse as Prometheus endpoint. [#7900](https://github.com/ClickHouse/ClickHouse/pull/7900) ([vdimir](https://github.com/Vdimir))
+* Add section `` in `config.xml` which restricts allowed hosts for remote table engines and table functions `URL`, `S3`, `HDFS`. [#7154](https://github.com/ClickHouse/ClickHouse/pull/7154) ([Mikhail Korotov](https://github.com/millb))
+* Added function `greatCircleAngle` which calculates the distance on a sphere in degrees. [#8105](https://github.com/ClickHouse/ClickHouse/pull/8105) ([alexey-milovidov](https://github.com/alexey-milovidov))
+* Changed Earth radius to be consistent with H3 library. [#8105](https://github.com/ClickHouse/ClickHouse/pull/8105) ([alexey-milovidov](https://github.com/alexey-milovidov))
+* Added `JSONCompactEachRow` and `JSONCompactEachRowWithNamesAndTypes` formats for input and output. [#7841](https://github.com/ClickHouse/ClickHouse/pull/7841) ([Mikhail Korotov](https://github.com/millb))
+* Added feature for file-related table engines and table functions (`File`, `S3`, `URL`, `HDFS`) which allows to read and write `gzip` files based on additional engine parameter or file extension. [#7840](https://github.com/ClickHouse/ClickHouse/pull/7840) ([Andrey Bodrov](https://github.com/apbodrov))
+* Added the `randomASCII(length)` function, generating a string with a random set of [ASCII](https://en.wikipedia.org/wiki/ASCII#Printable_characters) printable characters. [#8401](https://github.com/ClickHouse/ClickHouse/pull/8401) ([BayoNet](https://github.com/BayoNet))
+* Added function `JSONExtractArrayRaw` which returns an array on unparsed json array elements from `JSON` string. [#8081](https://github.com/ClickHouse/ClickHouse/pull/8081) ([Oleg Matrokhin](https://github.com/errx))
+* Add `arrayZip` function which allows to combine multiple arrays of equal lengths into one array of tuples. [#8149](https://github.com/ClickHouse/ClickHouse/pull/8149) ([Winter Zhang](https://github.com/zhang2014))
+* Add ability to move data between disks according to configured `TTL`-expressions for `*MergeTree` table engines family. [#8140](https://github.com/ClickHouse/ClickHouse/pull/8140) ([Vladimir Chebotarev](https://github.com/excitoon))
+* Added new aggregate function `avgWeighted` which allows to calculate weighted average. [#7898](https://github.com/ClickHouse/ClickHouse/pull/7898) ([Andrey Bodrov](https://github.com/apbodrov))
+* Now parallel parsing is enabled by default for `TSV`, `TSKV`, `CSV` and `JSONEachRow` formats. [#7894](https://github.com/ClickHouse/ClickHouse/pull/7894) ([Nikita Mikhaylov](https://github.com/nikitamikhaylov))
+* Add several geo functions from `H3` library: `h3GetResolution`, `h3EdgeAngle`, `h3EdgeLength`, `h3IsValid` and `h3kRing`. [#8034](https://github.com/ClickHouse/ClickHouse/pull/8034) ([Konstantin Malanchev](https://github.com/hombit))
+* Added support for brotli (`br`) compression in file-related storages and table functions. This fixes [#8156](https://github.com/ClickHouse/ClickHouse/issues/8156). [#8526](https://github.com/ClickHouse/ClickHouse/pull/8526) ([alexey-milovidov](https://github.com/alexey-milovidov))
+* Add `groupBit*` functions for the `SimpleAggregationFunction` type. [#8485](https://github.com/ClickHouse/ClickHouse/pull/8485) ([Guillaume Tassery](https://github.com/YiuRULE))
+
+#### Bug Fix
+* Fix rename of tables with `Distributed` engine. Fixes issue [#7868](https://github.com/ClickHouse/ClickHouse/issues/7868). [#8306](https://github.com/ClickHouse/ClickHouse/pull/8306) ([tavplubix](https://github.com/tavplubix))
+* Now dictionaries support `EXPRESSION` for attributes in arbitrary string in non-ClickHouse SQL dialect. [#8098](https://github.com/ClickHouse/ClickHouse/pull/8098) ([alesapin](https://github.com/alesapin))
+* Fix broken `INSERT SELECT FROM mysql(...)` query. This fixes [#8070](https://github.com/ClickHouse/ClickHouse/issues/8070) and [#7960](https://github.com/ClickHouse/ClickHouse/issues/7960). [#8234](https://github.com/ClickHouse/ClickHouse/pull/8234) ([tavplubix](https://github.com/tavplubix))
+* Fix error "Mismatch column sizes" when inserting default `Tuple` from `JSONEachRow`. This fixes [#5653](https://github.com/ClickHouse/ClickHouse/issues/5653). [#8606](https://github.com/ClickHouse/ClickHouse/pull/8606) ([tavplubix](https://github.com/tavplubix))
+* Now an exception will be thrown in case of using `WITH TIES` alongside `LIMIT BY`. Also add ability to use `TOP` with `LIMIT BY`. This fixes [#7472](https://github.com/ClickHouse/ClickHouse/issues/7472). [#7637](https://github.com/ClickHouse/ClickHouse/pull/7637) ([Nikita Mikhaylov](https://github.com/nikitamikhaylov))
+* Fix unintendent dependency from fresh glibc version in `clickhouse-odbc-bridge` binary. [#8046](https://github.com/ClickHouse/ClickHouse/pull/8046) ([Amos Bird](https://github.com/amosbird))
+* Fix bug in check function of `*MergeTree` engines family. Now it doesn't fail in case when we have equal amount of rows in last granule and last mark (non-final). [#8047](https://github.com/ClickHouse/ClickHouse/pull/8047) ([alesapin](https://github.com/alesapin))
+* Fix insert into `Enum*` columns after `ALTER` query, when underlying numeric type is equal to table specified type. This fixes [#7836](https://github.com/ClickHouse/ClickHouse/issues/7836). [#7908](https://github.com/ClickHouse/ClickHouse/pull/7908) ([Anton Popov](https://github.com/CurtizJ))
+* Allowed non-constant negative "size" argument for function `substring`. It was not allowed by mistake. This fixes [#4832](https://github.com/ClickHouse/ClickHouse/issues/4832). [#7703](https://github.com/ClickHouse/ClickHouse/pull/7703) ([alexey-milovidov](https://github.com/alexey-milovidov))
+* Fix parsing bug when wrong number of arguments passed to `(O|J)DBC` table engine. [#7709](https://github.com/ClickHouse/ClickHouse/pull/7709) ([alesapin](https://github.com/alesapin))
+* Using command name of the running clickhouse process when sending logs to syslog. In previous versions, empty string was used instead of command name. [#8460](https://github.com/ClickHouse/ClickHouse/pull/8460) ([Michael Nacharov](https://github.com/mnach))
+* Fix check of allowed hosts for `localhost`. This PR fixes the solution provided in [#8241](https://github.com/ClickHouse/ClickHouse/pull/8241). [#8342](https://github.com/ClickHouse/ClickHouse/pull/8342) ([Vitaly Baranov](https://github.com/vitlibar))
+* Fix rare crash in `argMin` and `argMax` functions for long string arguments, when result is used in `runningAccumulate` function. This fixes [#8325](https://github.com/ClickHouse/ClickHouse/issues/8325) [#8341](https://github.com/ClickHouse/ClickHouse/pull/8341) ([dinosaur](https://github.com/769344359))
+* Fix memory overcommit for tables with `Buffer` engine. [#8345](https://github.com/ClickHouse/ClickHouse/pull/8345) ([Azat Khuzhin](https://github.com/azat))
+* Fixed potential bug in functions that can take `NULL` as one of the arguments and return non-NULL. [#8196](https://github.com/ClickHouse/ClickHouse/pull/8196) ([alexey-milovidov](https://github.com/alexey-milovidov))
+* Better metrics calculations in thread pool for background processes for `MergeTree` table engines. [#8194](https://github.com/ClickHouse/ClickHouse/pull/8194) ([Vladimir Chebotarev](https://github.com/excitoon))
+* Fix function `IN` inside `WHERE` statement when row-level table filter is present. Fixes [#6687](https://github.com/ClickHouse/ClickHouse/issues/6687) [#8357](https://github.com/ClickHouse/ClickHouse/pull/8357) ([Ivan](https://github.com/abyss7))
+* Now an exception is thrown if the integral value is not parsed completely for settings values. [#7678](https://github.com/ClickHouse/ClickHouse/pull/7678) ([Mikhail Korotov](https://github.com/millb))
+* Fix exception when aggregate function is used in query to distributed table with more than two local shards. [#8164](https://github.com/ClickHouse/ClickHouse/pull/8164) ([小路](https://github.com/nicelulu))
+* Now bloom filter can handle zero length arrays and doesn't perform redundant calculations. [#8242](https://github.com/ClickHouse/ClickHouse/pull/8242) ([achimbab](https://github.com/achimbab))
+* Fixed checking if a client host is allowed by matching the client host to `host_regexp` specified in `users.xml`. [#8241](https://github.com/ClickHouse/ClickHouse/pull/8241) ([Vitaly Baranov](https://github.com/vitlibar))
+* Relax ambiguous column check that leads to false positives in multiple `JOIN ON` section. [#8385](https://github.com/ClickHouse/ClickHouse/pull/8385) ([Artem Zuikov](https://github.com/4ertus2))
+* Fixed possible server crash (`std::terminate`) when the server cannot send or write data in `JSON` or `XML` format with values of `String` data type (that require `UTF-8` validation) or when compressing result data with Brotli algorithm or in some other rare cases. This fixes [#7603](https://github.com/ClickHouse/ClickHouse/issues/7603) [#8384](https://github.com/ClickHouse/ClickHouse/pull/8384) ([alexey-milovidov](https://github.com/alexey-milovidov))
+* Fix race condition in `StorageDistributedDirectoryMonitor` found by CI. This fixes [#8364](https://github.com/ClickHouse/ClickHouse/issues/8364). [#8383](https://github.com/ClickHouse/ClickHouse/pull/8383) ([Nikolai Kochetov](https://github.com/KochetovNicolai))
+* Now background merges in `*MergeTree` table engines family preserve storage policy volume order more accurately. [#8549](https://github.com/ClickHouse/ClickHouse/pull/8549) ([Vladimir Chebotarev](https://github.com/excitoon))
+* Now table engine `Kafka` works properly with `Native` format. This fixes [#6731](https://github.com/ClickHouse/ClickHouse/issues/6731) [#7337](https://github.com/ClickHouse/ClickHouse/issues/7337) [#8003](https://github.com/ClickHouse/ClickHouse/issues/8003). [#8016](https://github.com/ClickHouse/ClickHouse/pull/8016) ([filimonov](https://github.com/filimonov))
+* Fixed formats with headers (like `CSVWithNames`) which were throwing exception about EOF for table engine `Kafka`. [#8016](https://github.com/ClickHouse/ClickHouse/pull/8016) ([filimonov](https://github.com/filimonov))
+* Fixed a bug with making set from subquery in right part of `IN` section. This fixes [#5767](https://github.com/ClickHouse/ClickHouse/issues/5767) and [#2542](https://github.com/ClickHouse/ClickHouse/issues/2542). [#7755](https://github.com/ClickHouse/ClickHouse/pull/7755) ([Nikita Mikhaylov](https://github.com/nikitamikhaylov))
+* Fix possible crash while reading from storage `File`. [#7756](https://github.com/ClickHouse/ClickHouse/pull/7756) ([Nikolai Kochetov](https://github.com/KochetovNicolai))
+* Fixed reading of the files in `Parquet` format containing columns of type `list`. [#8334](https://github.com/ClickHouse/ClickHouse/pull/8334) ([maxulan](https://github.com/maxulan))
+* Fix error `Not found column` for distributed queries with `PREWHERE` condition dependent on sampling key if `max_parallel_replicas > 1`. [#7913](https://github.com/ClickHouse/ClickHouse/pull/7913) ([Nikolai Kochetov](https://github.com/KochetovNicolai))
+* Fix error `Not found column` if query used `PREWHERE` dependent on table's alias and the result set was empty because of primary key condition. [#7911](https://github.com/ClickHouse/ClickHouse/pull/7911) ([Nikolai Kochetov](https://github.com/KochetovNicolai))
+* Fixed return type for functions `rand` and `randConstant` in case of `Nullable` argument. Now functions always return `UInt32` and never `Nullable(UInt32)`. [#8204](https://github.com/ClickHouse/ClickHouse/pull/8204) ([Nikolai Kochetov](https://github.com/KochetovNicolai))
+* Disabled predicate push-down for `WITH FILL` expression. This fixes [#7784](https://github.com/ClickHouse/ClickHouse/issues/7784). [#7789](https://github.com/ClickHouse/ClickHouse/pull/7789) ([Winter Zhang](https://github.com/zhang2014))
+* Fixed incorrect `count()` result for `SummingMergeTree` when `FINAL` section is used. [#3280](https://github.com/ClickHouse/ClickHouse/issues/3280) [#7786](https://github.com/ClickHouse/ClickHouse/pull/7786) ([Nikita Mikhaylov](https://github.com/nikitamikhaylov))
+* Fix possible incorrect result for constant functions from remote servers. It happened for queries with functions like `version()`, `uptime()`, etc. which returns different constant values for different servers. This fixes [#7666](https://github.com/ClickHouse/ClickHouse/issues/7666). [#7689](https://github.com/ClickHouse/ClickHouse/pull/7689) ([Nikolai Kochetov](https://github.com/KochetovNicolai))
+* Fix complicated bug in push-down predicate optimization which leads to wrong results. This fixes a lot of issues on push-down predicate optimization. [#8503](https://github.com/ClickHouse/ClickHouse/pull/8503) ([Winter Zhang](https://github.com/zhang2014))
+* Fix crash in `CREATE TABLE .. AS dictionary` query. [#8508](https://github.com/ClickHouse/ClickHouse/pull/8508) ([Azat Khuzhin](https://github.com/azat))
+* Several improvements ClickHouse grammar in `.g4` file. [#8294](https://github.com/ClickHouse/ClickHouse/pull/8294) ([taiyang-li](https://github.com/taiyang-li))
+* Fix bug that leads to crashes in `JOIN`s with tables with engine `Join`. This fixes [#7556](https://github.com/ClickHouse/ClickHouse/issues/7556) [#8254](https://github.com/ClickHouse/ClickHouse/issues/8254) [#7915](https://github.com/ClickHouse/ClickHouse/issues/7915) [#8100](https://github.com/ClickHouse/ClickHouse/issues/8100). [#8298](https://github.com/ClickHouse/ClickHouse/pull/8298) ([Artem Zuikov](https://github.com/4ertus2))
+* Fix redundant dictionaries reload on `CREATE DATABASE`. [#7916](https://github.com/ClickHouse/ClickHouse/pull/7916) ([Azat Khuzhin](https://github.com/azat))
+* Limit maximum number of streams for read from `StorageFile` and `StorageHDFS`. Fixes https://github.com/ClickHouse/ClickHouse/issues/7650. [#7981](https://github.com/ClickHouse/ClickHouse/pull/7981) ([alesapin](https://github.com/alesapin))
+* Fix bug in `ALTER ... MODIFY ... CODEC` query, when user specify both default expression and codec. Fixes [8593](https://github.com/ClickHouse/ClickHouse/issues/8593). [#8614](https://github.com/ClickHouse/ClickHouse/pull/8614) ([alesapin](https://github.com/alesapin))
+* Fix error in background merge of columns with `SimpleAggregateFunction(LowCardinality)` type. [#8613](https://github.com/ClickHouse/ClickHouse/pull/8613) ([Nikolai Kochetov](https://github.com/KochetovNicolai))
+* Fixed type check in function `toDateTime64`. [#8375](https://github.com/ClickHouse/ClickHouse/pull/8375) ([Vasily Nemkov](https://github.com/Enmk))
+* Now server do not crash on `LEFT` or `FULL JOIN` with and Join engine and unsupported `join_use_nulls` settings. [#8479](https://github.com/ClickHouse/ClickHouse/pull/8479) ([Artem Zuikov](https://github.com/4ertus2))
+* Now `DROP DICTIONARY IF EXISTS db.dict` query doesn't throw exception if `db` doesn't exist. [#8185](https://github.com/ClickHouse/ClickHouse/pull/8185) ([Vitaly Baranov](https://github.com/vitlibar))
+* Fix possible crashes in table functions (`file`, `mysql`, `remote`) caused by usage of reference to removed `IStorage` object. Fix incorrect parsing of columns specified at insertion into table function. [#7762](https://github.com/ClickHouse/ClickHouse/pull/7762) ([tavplubix](https://github.com/tavplubix))
+* Ensure network be up before starting `clickhouse-server`. This fixes [#7507](https://github.com/ClickHouse/ClickHouse/issues/7507). [#8570](https://github.com/ClickHouse/ClickHouse/pull/8570) ([Zhichang Yu](https://github.com/yuzhichang))
+* Fix timeouts handling for secure connections, so queries doesn't hang indefenitely. This fixes [#8126](https://github.com/ClickHouse/ClickHouse/issues/8126). [#8128](https://github.com/ClickHouse/ClickHouse/pull/8128) ([alexey-milovidov](https://github.com/alexey-milovidov))
+* Fix `clickhouse-copier`'s redundant contention between concurrent workers. [#7816](https://github.com/ClickHouse/ClickHouse/pull/7816) ([Ding Xiang Fei](https://github.com/dingxiangfei2009))
+* Now mutations doesn't skip attached parts, even if their mutation version were larger than current mutation version. [#7812](https://github.com/ClickHouse/ClickHouse/pull/7812) ([Zhichang Yu](https://github.com/yuzhichang)) [#8250](https://github.com/ClickHouse/ClickHouse/pull/8250) ([alesapin](https://github.com/alesapin))
+* Ignore redundant copies of `*MergeTree` data parts after move to another disk and server restart. [#7810](https://github.com/ClickHouse/ClickHouse/pull/7810) ([Vladimir Chebotarev](https://github.com/excitoon))
+* Fix crash in `FULL JOIN` with `LowCardinality` in `JOIN` key. [#8252](https://github.com/ClickHouse/ClickHouse/pull/8252) ([Artem Zuikov](https://github.com/4ertus2))
+* Forbidden to use column name more than once in insert query like `INSERT INTO tbl (x, y, x)`. This fixes [#5465](https://github.com/ClickHouse/ClickHouse/issues/5465), [#7681](https://github.com/ClickHouse/ClickHouse/issues/7681). [#7685](https://github.com/ClickHouse/ClickHouse/pull/7685) ([alesapin](https://github.com/alesapin))
+* Added fallback for detection the number of physical CPU cores for unknown CPUs (using the number of logical CPU cores). This fixes [#5239](https://github.com/ClickHouse/ClickHouse/issues/5239). [#7726](https://github.com/ClickHouse/ClickHouse/pull/7726) ([alexey-milovidov](https://github.com/alexey-milovidov))
+* Fix `There's no column` error for materialized and alias columns. [#8210](https://github.com/ClickHouse/ClickHouse/pull/8210) ([Artem Zuikov](https://github.com/4ertus2))
+* Fixed sever crash when `EXISTS` query was used without `TABLE` or `DICTIONARY` qualifier. Just like `EXISTS t`. This fixes [#8172](https://github.com/ClickHouse/ClickHouse/issues/8172). This bug was introduced in version 19.17. [#8213](https://github.com/ClickHouse/ClickHouse/pull/8213) ([alexey-milovidov](https://github.com/alexey-milovidov))
+* Fix rare bug with error `"Sizes of columns doesn't match"` that might appear when using `SimpleAggregateFunction` column. [#7790](https://github.com/ClickHouse/ClickHouse/pull/7790) ([Boris Granveaud](https://github.com/bgranvea))
+* Fix bug where user with empty `allow_databases` got access to all databases (and same for `allow_dictionaries`). [#7793](https://github.com/ClickHouse/ClickHouse/pull/7793) ([DeifyTheGod](https://github.com/DeifyTheGod))
+* Fix client crash when server already disconnected from client. [#8071](https://github.com/ClickHouse/ClickHouse/pull/8071) ([Azat Khuzhin](https://github.com/azat))
+* Fix `ORDER BY` behaviour in case of sorting by primary key prefix and non primary key suffix. [#7759](https://github.com/ClickHouse/ClickHouse/pull/7759) ([Anton Popov](https://github.com/CurtizJ))
+* Check if qualified column present in the table. This fixes [#6836](https://github.com/ClickHouse/ClickHouse/issues/6836). [#7758](https://github.com/ClickHouse/ClickHouse/pull/7758) ([Artem Zuikov](https://github.com/4ertus2))
+* Fixed behavior with `ALTER MOVE` ran immediately after merge finish moves superpart of specified. Fixes [#8103](https://github.com/ClickHouse/ClickHouse/issues/8103). [#8104](https://github.com/ClickHouse/ClickHouse/pull/8104) ([Vladimir Chebotarev](https://github.com/excitoon))
+* Fix possible server crash while using `UNION` with different number of columns. Fixes [#7279](https://github.com/ClickHouse/ClickHouse/issues/7279). [#7929](https://github.com/ClickHouse/ClickHouse/pull/7929) ([Nikolai Kochetov](https://github.com/KochetovNicolai))
+* Fix size of result substring for function `substr` with negative size. [#8589](https://github.com/ClickHouse/ClickHouse/pull/8589) ([Nikolai Kochetov](https://github.com/KochetovNicolai))
+* Now server does not execute part mutation in `MergeTree` if there are not enough free threads in background pool. [#8588](https://github.com/ClickHouse/ClickHouse/pull/8588) ([tavplubix](https://github.com/tavplubix))
+* Fix a minor typo on formatting `UNION ALL` AST. [#7999](https://github.com/ClickHouse/ClickHouse/pull/7999) ([litao91](https://github.com/litao91))
+* Fixed incorrect bloom filter results for negative numbers. This fixes [#8317](https://github.com/ClickHouse/ClickHouse/issues/8317). [#8566](https://github.com/ClickHouse/ClickHouse/pull/8566) ([Winter Zhang](https://github.com/zhang2014))
+* Fixed potential buffer overflow in decompress. Malicious user can pass fabricated compressed data that will cause read after buffer. This issue was found by Eldar Zaitov from Yandex information security team. [#8404](https://github.com/ClickHouse/ClickHouse/pull/8404) ([alexey-milovidov](https://github.com/alexey-milovidov))
+* Fix incorrect result because of integers overflow in `arrayIntersect`. [#7777](https://github.com/ClickHouse/ClickHouse/pull/7777) ([Nikolai Kochetov](https://github.com/KochetovNicolai))
+* Now `OPTIMIZE TABLE` query will not wait for offline replicas to perform the operation. [#8314](https://github.com/ClickHouse/ClickHouse/pull/8314) ([javi santana](https://github.com/javisantana))
+* Fixed `ALTER TTL` parser for `Replicated*MergeTree` tables. [#8318](https://github.com/ClickHouse/ClickHouse/pull/8318) ([Vladimir Chebotarev](https://github.com/excitoon))
+* Fix communication between server and client, so server read temporary tables info after query failure. [#8084](https://github.com/ClickHouse/ClickHouse/pull/8084) ([Azat Khuzhin](https://github.com/azat))
+* Fix `bitmapAnd` function error when intersecting an aggregated bitmap and a scalar bitmap. [#8082](https://github.com/ClickHouse/ClickHouse/pull/8082) ([Yue Huang](https://github.com/moon03432))
+* Refine the definition of `ZXid` according to the ZooKeeper Programmer's Guide which fixes bug in `clickhouse-cluster-copier`. [#8088](https://github.com/ClickHouse/ClickHouse/pull/8088) ([Ding Xiang Fei](https://github.com/dingxiangfei2009))
+* `odbc` table function now respects `external_table_functions_use_nulls` setting. [#7506](https://github.com/ClickHouse/ClickHouse/pull/7506) ([Vasily Nemkov](https://github.com/Enmk))
+* Fixed bug that lead to a rare data race. [#8143](https://github.com/ClickHouse/ClickHouse/pull/8143) ([Alexander Kazakov](https://github.com/Akazz))
+* Now `SYSTEM RELOAD DICTIONARY` reloads a dictionary completely, ignoring `update_field`. This fixes [#7440](https://github.com/ClickHouse/ClickHouse/issues/7440). [#8037](https://github.com/ClickHouse/ClickHouse/pull/8037) ([Vitaly Baranov](https://github.com/vitlibar))
+* Add ability to check if dictionary exists in create query. [#8032](https://github.com/ClickHouse/ClickHouse/pull/8032) ([alesapin](https://github.com/alesapin))
+* Fix `Float*` parsing in `Values` format. This fixes [#7817](https://github.com/ClickHouse/ClickHouse/issues/7817). [#7870](https://github.com/ClickHouse/ClickHouse/pull/7870) ([tavplubix](https://github.com/tavplubix))
+* Fix crash when we cannot reserve space in some background operations of `*MergeTree` table engines family. [#7873](https://github.com/ClickHouse/ClickHouse/pull/7873) ([Vladimir Chebotarev](https://github.com/excitoon))
+* Fix crash of merge operation when table contains `SimpleAggregateFunction(LowCardinality)` column. This fixes [#8515](https://github.com/ClickHouse/ClickHouse/issues/8515). [#8522](https://github.com/ClickHouse/ClickHouse/pull/8522) ([Azat Khuzhin](https://github.com/azat))
+* Restore support of all ICU locales and add the ability to apply collations for constant expressions. Also add language name to `system.collations` table. [#8051](https://github.com/ClickHouse/ClickHouse/pull/8051) ([alesapin](https://github.com/alesapin))
+* Fix bug when external dictionaries with zero minimal lifetime (`LIFETIME(MIN 0 MAX N)`, `LIFETIME(N)`) don't update in background. [#7983](https://github.com/ClickHouse/ClickHouse/pull/7983) ([alesapin](https://github.com/alesapin))
+* Fix crash when external dictionary with ClickHouse source has subquery in query. [#8351](https://github.com/ClickHouse/ClickHouse/pull/8351) ([Nikolai Kochetov](https://github.com/KochetovNicolai))
+* Fix incorrect parsing of file extension in table with engine `URL`. This fixes [#8157](https://github.com/ClickHouse/ClickHouse/issues/8157). [#8419](https://github.com/ClickHouse/ClickHouse/pull/8419) ([Andrey Bodrov](https://github.com/apbodrov))
+* Fix `CHECK TABLE` query for `*MergeTree` tables without key. Fixes [#7543](https://github.com/ClickHouse/ClickHouse/issues/7543). [#7979](https://github.com/ClickHouse/ClickHouse/pull/7979) ([alesapin](https://github.com/alesapin))
+* Fixed conversion of `Float64` to MySQL type. [#8079](https://github.com/ClickHouse/ClickHouse/pull/8079) ([Yuriy Baranov](https://github.com/yurriy))
+* Now if table was not completely dropped because of server crash, server will try to restore and load it. [#8176](https://github.com/ClickHouse/ClickHouse/pull/8176) ([tavplubix](https://github.com/tavplubix))
+* Fixed crash in table function `file` while inserting into file that doesn't exist. Now in this case file would be created and then insert would be processed. [#8177](https://github.com/ClickHouse/ClickHouse/pull/8177) ([Olga Khvostikova](https://github.com/stavrolia))
+* Fix rare deadlock which can happen when `trace_log` is in enabled. [#7838](https://github.com/ClickHouse/ClickHouse/pull/7838) ([filimonov](https://github.com/filimonov))
+* Add ability to work with different types besides `Date` in `RangeHashed` external dictionary created from DDL query. Fixes [7899](https://github.com/ClickHouse/ClickHouse/issues/7899). [#8275](https://github.com/ClickHouse/ClickHouse/pull/8275) ([alesapin](https://github.com/alesapin))
+* Fixes crash when `now64()` is called with result of another function. [#8270](https://github.com/ClickHouse/ClickHouse/pull/8270) ([Vasily Nemkov](https://github.com/Enmk))
+* Fixed bug with detecting client IP for connections through mysql wire protocol. [#7743](https://github.com/ClickHouse/ClickHouse/pull/7743) ([Dmitry Muzyka](https://github.com/dmitriy-myz))
+* Fix empty array handling in `arraySplit` function. This fixes [#7708](https://github.com/ClickHouse/ClickHouse/issues/7708). [#7747](https://github.com/ClickHouse/ClickHouse/pull/7747) ([hcz](https://github.com/hczhcz))
+* Fixed the issue when `pid-file` of another running `clickhouse-server` may be deleted. [#8487](https://github.com/ClickHouse/ClickHouse/pull/8487) ([Weiqing Xu](https://github.com/weiqxu))
+* Fix dictionary reload if it has `invalidate_query`, which stopped updates and some exception on previous update tries. [#8029](https://github.com/ClickHouse/ClickHouse/pull/8029) ([alesapin](https://github.com/alesapin))
+* Fixed error in function `arrayReduce` that may lead to "double free" and error in aggregate function combinator `Resample` that may lead to memory leak. Added aggregate function `aggThrow`. This function can be used for testing purposes. [#8446](https://github.com/ClickHouse/ClickHouse/pull/8446) ([alexey-milovidov](https://github.com/alexey-milovidov))
+
+#### Improvement
+* Improved logging when working with `S3` table engine. [#8251](https://github.com/ClickHouse/ClickHouse/pull/8251) ([Grigory Pervakov](https://github.com/GrigoryPervakov))
+* Printed help message when no arguments are passed when calling `clickhouse-local`. This fixes [#5335](https://github.com/ClickHouse/ClickHouse/issues/5335). [#8230](https://github.com/ClickHouse/ClickHouse/pull/8230) ([Andrey Nagorny](https://github.com/Melancholic))
+* Add setting `mutations_sync` which allows to wait `ALTER UPDATE/DELETE` queries synchronously. [#8237](https://github.com/ClickHouse/ClickHouse/pull/8237) ([alesapin](https://github.com/alesapin))
+* Allow to set up relative `user_files_path` in `config.xml` (in the way similar to `format_schema_path`). [#7632](https://github.com/ClickHouse/ClickHouse/pull/7632) ([hcz](https://github.com/hczhcz))
+* Add exception for illegal types for conversion functions with `-OrZero` postfix. [#7880](https://github.com/ClickHouse/ClickHouse/pull/7880) ([Andrey Konyaev](https://github.com/akonyaev90))
+* Simplify format of the header of data sending to a shard in a distributed query. [#8044](https://github.com/ClickHouse/ClickHouse/pull/8044) ([Vitaly Baranov](https://github.com/vitlibar))
+* `Live View` table engine refactoring. [#8519](https://github.com/ClickHouse/ClickHouse/pull/8519) ([vzakaznikov](https://github.com/vzakaznikov))
+* Add additional checks for external dictionaries created from DDL-queries. [#8127](https://github.com/ClickHouse/ClickHouse/pull/8127) ([alesapin](https://github.com/alesapin))
+* Fix error `Column ... already exists` while using `FINAL` and `SAMPLE` together, e.g. `select count() from table final sample 1/2`. Fixes [#5186](https://github.com/ClickHouse/ClickHouse/issues/5186). [#7907](https://github.com/ClickHouse/ClickHouse/pull/7907) ([Nikolai Kochetov](https://github.com/KochetovNicolai))
+* Now table the first argument of `joinGet` function can be table indentifier. [#7707](https://github.com/ClickHouse/ClickHouse/pull/7707) ([Amos Bird](https://github.com/amosbird))
+* Allow using `MaterializedView` with subqueries above `Kafka` tables. [#8197](https://github.com/ClickHouse/ClickHouse/pull/8197) ([filimonov](https://github.com/filimonov))
+* Now background moves between disks run it the seprate thread pool. [#7670](https://github.com/ClickHouse/ClickHouse/pull/7670) ([Vladimir Chebotarev](https://github.com/excitoon))
+* `SYSTEM RELOAD DICTIONARY` now executes synchronously. [#8240](https://github.com/ClickHouse/ClickHouse/pull/8240) ([Vitaly Baranov](https://github.com/vitlibar))
+* Stack traces now display physical addresses (offsets in object file) instead of virtual memory addresses (where the object file was loaded). That allows the use of `addr2line` when binary is position independent and ASLR is active. This fixes [#8360](https://github.com/ClickHouse/ClickHouse/issues/8360). [#8387](https://github.com/ClickHouse/ClickHouse/pull/8387) ([alexey-milovidov](https://github.com/alexey-milovidov))
+* Support new syntax for row-level security filters: ``. Fixes [#5779](https://github.com/ClickHouse/ClickHouse/issues/5779). [#8381](https://github.com/ClickHouse/ClickHouse/pull/8381) ([Ivan](https://github.com/abyss7))
+* Now `cityHash` function can work with `Decimal` and `UUID` types. Fixes [#5184](https://github.com/ClickHouse/ClickHouse/issues/5184). [#7693](https://github.com/ClickHouse/ClickHouse/pull/7693) ([Mikhail Korotov](https://github.com/millb))
+* Removed fixed index granularity (it was 1024) from system logs because it's obsolete after implementation of adaptive granularity. [#7698](https://github.com/ClickHouse/ClickHouse/pull/7698) ([alexey-milovidov](https://github.com/alexey-milovidov))
+* Enabled MySQL compatibility server when ClickHouse is compiled without SSL. [#7852](https://github.com/ClickHouse/ClickHouse/pull/7852) ([Yuriy Baranov](https://github.com/yurriy))
+* Now server checksums distributed batches, which gives more verbose errors in case of corrupted data in batch. [#7914](https://github.com/ClickHouse/ClickHouse/pull/7914) ([Azat Khuzhin](https://github.com/azat))
+* Support `DROP DATABASE`, `DETACH TABLE`, `DROP TABLE` and `ATTACH TABLE` for `MySQL` database engine. [#8202](https://github.com/ClickHouse/ClickHouse/pull/8202) ([Winter Zhang](https://github.com/zhang2014))
+* Add authentication in S3 table function and table engine. [#7623](https://github.com/ClickHouse/ClickHouse/pull/7623) ([Vladimir Chebotarev](https://github.com/excitoon))
+* Added check for extra parts of `MergeTree` at different disks, in order to not allow to miss data parts at undefined disks. [#8118](https://github.com/ClickHouse/ClickHouse/pull/8118) ([Vladimir Chebotarev](https://github.com/excitoon))
+* Enable SSL support for Mac client and server. [#8297](https://github.com/ClickHouse/ClickHouse/pull/8297) ([Ivan](https://github.com/abyss7))
+* Now ClickHouse can work as MySQL federated server (see https://dev.mysql.com/doc/refman/5.7/en/federated-create-server.html). [#7717](https://github.com/ClickHouse/ClickHouse/pull/7717) ([Maxim Fedotov](https://github.com/MaxFedotov))
+* `clickhouse-client` now only enable `bracketed-paste` when multiquery is on and multiline is off. This fixes (#7757)[https://github.com/ClickHouse/ClickHouse/issues/7757]. [#7761](https://github.com/ClickHouse/ClickHouse/pull/7761) ([Amos Bird](https://github.com/amosbird))
+* Support `Array(Decimal)` in `if` function. [#7721](https://github.com/ClickHouse/ClickHouse/pull/7721) ([Artem Zuikov](https://github.com/4ertus2))
+* Support Decimals in `arrayDifference`, `arrayCumSum` and `arrayCumSumNegative` functions. [#7724](https://github.com/ClickHouse/ClickHouse/pull/7724) ([Artem Zuikov](https://github.com/4ertus2))
+* Added `lifetime` column to `system.dictionaries` table. [#6820](https://github.com/ClickHouse/ClickHouse/issues/6820) [#7727](https://github.com/ClickHouse/ClickHouse/pull/7727) ([kekekekule](https://github.com/kekekekule))
+* Improved check for existing parts on different disks for `*MergeTree` table engines. Addresses [#7660](https://github.com/ClickHouse/ClickHouse/issues/7660). [#8440](https://github.com/ClickHouse/ClickHouse/pull/8440) ([Vladimir Chebotarev](https://github.com/excitoon))
+* Integration with `AWS SDK` for `S3` interactions which allows to use all S3 features out of the box. [#8011](https://github.com/ClickHouse/ClickHouse/pull/8011) ([Pavel Kovalenko](https://github.com/Jokser))
+* Added support for subqueries in `Live View` tables. [#7792](https://github.com/ClickHouse/ClickHouse/pull/7792) ([vzakaznikov](https://github.com/vzakaznikov))
+* Check for using `Date` or `DateTime` column from `TTL` expressions was removed. [#7920](https://github.com/ClickHouse/ClickHouse/pull/7920) ([Vladimir Chebotarev](https://github.com/excitoon))
+* Information about disk was added to `system.detached_parts` table. [#7833](https://github.com/ClickHouse/ClickHouse/pull/7833) ([Vladimir Chebotarev](https://github.com/excitoon))
+* Now settings `max_(table|partition)_size_to_drop` can be changed without a restart. [#7779](https://github.com/ClickHouse/ClickHouse/pull/7779) ([Grigory Pervakov](https://github.com/GrigoryPervakov))
+* Slightly better usability of error messages. Ask user not to remove the lines below `Stack trace:`. [#7897](https://github.com/ClickHouse/ClickHouse/pull/7897) ([alexey-milovidov](https://github.com/alexey-milovidov))
+* Better reading messages from `Kafka` engine in various formats after [#7935](https://github.com/ClickHouse/ClickHouse/issues/7935). [#8035](https://github.com/ClickHouse/ClickHouse/pull/8035) ([Ivan](https://github.com/abyss7))
+* Better compatibility with MySQL clients which don't support `sha2_password` auth plugin. [#8036](https://github.com/ClickHouse/ClickHouse/pull/8036) ([Yuriy Baranov](https://github.com/yurriy))
+* Support more column types in MySQL compatibility server. [#7975](https://github.com/ClickHouse/ClickHouse/pull/7975) ([Yuriy Baranov](https://github.com/yurriy))
+* Implement `ORDER BY` optimization for `Merge`, `Buffer` and `Materilized View` storages with underlying `MergeTree` tables. [#8130](https://github.com/ClickHouse/ClickHouse/pull/8130) ([Anton Popov](https://github.com/CurtizJ))
+* Now we always use POSIX implementation of `getrandom` to have better compatibility with old kernels (< 3.17). [#7940](https://github.com/ClickHouse/ClickHouse/pull/7940) ([Amos Bird](https://github.com/amosbird))
+* Better check for valid destination in a move TTL rule. [#8410](https://github.com/ClickHouse/ClickHouse/pull/8410) ([Vladimir Chebotarev](https://github.com/excitoon))
+* Better checks for broken insert batches for `Distributed` table engine. [#7933](https://github.com/ClickHouse/ClickHouse/pull/7933) ([Azat Khuzhin](https://github.com/azat))
+* Add column with array of parts name which mutations must process in future to `system.mutations` table. [#8179](https://github.com/ClickHouse/ClickHouse/pull/8179) ([alesapin](https://github.com/alesapin))
+* Parallel merge sort optimization for processors. [#8552](https://github.com/ClickHouse/ClickHouse/pull/8552) ([Nikolai Kochetov](https://github.com/KochetovNicolai))
+* The settings `mark_cache_min_lifetime` is now obsolete and does nothing. In previous versions, mark cache can grow in memory larger than `mark_cache_size` to accomodate data within `mark_cache_min_lifetime` seconds. That was leading to confusion and higher memory usage than expected, that is especially bad on memory constrained systems. If you will see performance degradation after installing this release, you should increase the `mark_cache_size`. [#8484](https://github.com/ClickHouse/ClickHouse/pull/8484) ([alexey-milovidov](https://github.com/alexey-milovidov))
+* Preparation to use `tid` everywhere. This is needed for [#7477](https://github.com/ClickHouse/ClickHouse/issues/7477). [#8276](https://github.com/ClickHouse/ClickHouse/pull/8276) ([alexey-milovidov](https://github.com/alexey-milovidov))
+
+#### Performance Improvement
+* Performance optimizations in processors pipeline. [#7988](https://github.com/ClickHouse/ClickHouse/pull/7988) ([Nikolai Kochetov](https://github.com/KochetovNicolai))
+* Non-blocking updates of expired keys in cache dictionaries (with permission to read old ones). [#8303](https://github.com/ClickHouse/ClickHouse/pull/8303) ([Nikita Mikhaylov](https://github.com/nikitamikhaylov))
+* Compile ClickHouse without `-fno-omit-frame-pointer` globally to spare one more register. [#8097](https://github.com/ClickHouse/ClickHouse/pull/8097) ([Amos Bird](https://github.com/amosbird))
+* Speedup `greatCircleDistance` function and add performance tests for it. [#7307](https://github.com/ClickHouse/ClickHouse/pull/7307) ([Olga Khvostikova](https://github.com/stavrolia))
+* Improved performance of function `roundDown`. [#8465](https://github.com/ClickHouse/ClickHouse/pull/8465) ([alexey-milovidov](https://github.com/alexey-milovidov))
+* Improved performance of `max`, `min`, `argMin`, `argMax` for `DateTime64` data type. [#8199](https://github.com/ClickHouse/ClickHouse/pull/8199) ([Vasily Nemkov](https://github.com/Enmk))
+* Improved performance of sorting without a limit or with big limit and external sorting. [#8545](https://github.com/ClickHouse/ClickHouse/pull/8545) ([alexey-milovidov](https://github.com/alexey-milovidov))
+* Improved performance of formatting floating point numbers up to 6 times. [#8542](https://github.com/ClickHouse/ClickHouse/pull/8542) ([alexey-milovidov](https://github.com/alexey-milovidov))
+* Improved performance of `modulo` function. [#7750](https://github.com/ClickHouse/ClickHouse/pull/7750) ([Amos Bird](https://github.com/amosbird))
+* Optimized `ORDER BY` and merging with single column key. [#8335](https://github.com/ClickHouse/ClickHouse/pull/8335) ([alexey-milovidov](https://github.com/alexey-milovidov))
+* Better implementation for `arrayReduce`, `-Array` and `-State` combinators. [#7710](https://github.com/ClickHouse/ClickHouse/pull/7710) ([Amos Bird](https://github.com/amosbird))
+* Now `PREWHERE` should be optimized to be at least as efficient as `WHERE`. [#7769](https://github.com/ClickHouse/ClickHouse/pull/7769) ([Amos Bird](https://github.com/amosbird))
+* Improve the way `round` and `roundBankers` handling negative numbers. [#8229](https://github.com/ClickHouse/ClickHouse/pull/8229) ([hcz](https://github.com/hczhcz))
+* Improved decoding performance of `DoubleDelta` and `Gorilla` codecs by roughly 30-40%. This fixes [#7082](https://github.com/ClickHouse/ClickHouse/issues/7082). [#8019](https://github.com/ClickHouse/ClickHouse/pull/8019) ([Vasily Nemkov](https://github.com/Enmk))
+* Improved performance of `base64` related functions. [#8444](https://github.com/ClickHouse/ClickHouse/pull/8444) ([alexey-milovidov](https://github.com/alexey-milovidov))
+* Added a function `geoDistance`. It is similar to `greatCircleDistance` but uses approximation to WGS-84 ellipsoid model. The performance of both functions are near the same. [#8086](https://github.com/ClickHouse/ClickHouse/pull/8086) ([alexey-milovidov](https://github.com/alexey-milovidov))
+* Faster `min` and `max` aggregation functions for `Decimal` data type. [#8144](https://github.com/ClickHouse/ClickHouse/pull/8144) ([Artem Zuikov](https://github.com/4ertus2))
+* Vectorize processing `arrayReduce`. [#7608](https://github.com/ClickHouse/ClickHouse/pull/7608) ([Amos Bird](https://github.com/amosbird))
+* `if` chains are now optimized as `multiIf`. [#8355](https://github.com/ClickHouse/ClickHouse/pull/8355) ([kamalov-ruslan](https://github.com/kamalov-ruslan))
+* Fix performance regression of `Kafka` table engine introduced in 19.15. This fixes [#7261](https://github.com/ClickHouse/ClickHouse/issues/7261). [#7935](https://github.com/ClickHouse/ClickHouse/pull/7935) ([filimonov](https://github.com/filimonov))
+* Removed "pie" code generation that `gcc` from Debian packages occasionally brings by default. [#8483](https://github.com/ClickHouse/ClickHouse/pull/8483) ([alexey-milovidov](https://github.com/alexey-milovidov))
+* Parallel parsing data formats [#6553](https://github.com/ClickHouse/ClickHouse/pull/6553) ([Nikita Mikhaylov](https://github.com/nikitamikhaylov))
+* Enable optimized parser of `Values` with expressions by default (`input_format_values_deduce_templates_of_expressions=1`). [#8231](https://github.com/ClickHouse/ClickHouse/pull/8231) ([tavplubix](https://github.com/tavplubix))
+
+#### Build/Testing/Packaging Improvement
+* Build fixes for `ARM` and in minimal mode. [#8304](https://github.com/ClickHouse/ClickHouse/pull/8304) ([proller](https://github.com/proller))
+* Add coverage file flush for `clickhouse-server` when std::atexit is not called. Also slightly improved logging in stateless tests with coverage. [#8267](https://github.com/ClickHouse/ClickHouse/pull/8267) ([alesapin](https://github.com/alesapin))
+* Update LLVM library in contrib. Avoid using LLVM from OS packages. [#8258](https://github.com/ClickHouse/ClickHouse/pull/8258) ([alexey-milovidov](https://github.com/alexey-milovidov))
+* Make bundled `curl` build fully quiet. [#8232](https://github.com/ClickHouse/ClickHouse/pull/8232) [#8203](https://github.com/ClickHouse/ClickHouse/pull/8203) ([Pavel Kovalenko](https://github.com/Jokser))
+* Fix some `MemorySanitizer` warnings. [#8235](https://github.com/ClickHouse/ClickHouse/pull/8235) ([Alexander Kuzmenkov](https://github.com/akuzm))
+* Use `add_warning` and `no_warning` macros in `CMakeLists.txt`. [#8604](https://github.com/ClickHouse/ClickHouse/pull/8604) ([Ivan](https://github.com/abyss7))
+* Add support of Minio S3 Compatible object (https://min.io/) for better integration tests. [#7863](https://github.com/ClickHouse/ClickHouse/pull/7863) [#7875](https://github.com/ClickHouse/ClickHouse/pull/7875) ([Pavel Kovalenko](https://github.com/Jokser))
+* Imported `libc` headers to contrib. It allows to make builds more consistent across various systems (only for `x86_64-linux-gnu`). [#5773](https://github.com/ClickHouse/ClickHouse/pull/5773) ([alexey-milovidov](https://github.com/alexey-milovidov))
+* Remove `-fPIC` from some libraries. [#8464](https://github.com/ClickHouse/ClickHouse/pull/8464) ([alexey-milovidov](https://github.com/alexey-milovidov))
+* Clean `CMakeLists.txt` for curl. See https://github.com/ClickHouse/ClickHouse/pull/8011#issuecomment-569478910 [#8459](https://github.com/ClickHouse/ClickHouse/pull/8459) ([alexey-milovidov](https://github.com/alexey-milovidov))
+* Silent warnings in `CapNProto` library. [#8220](https://github.com/ClickHouse/ClickHouse/pull/8220) ([alexey-milovidov](https://github.com/alexey-milovidov))
+* Add performance tests for short string optimized hash tables. [#7679](https://github.com/ClickHouse/ClickHouse/pull/7679) ([Amos Bird](https://github.com/amosbird))
+* Now ClickHouse will build on `AArch64` even if `MADV_FREE` is not available. This fixes [#8027](https://github.com/ClickHouse/ClickHouse/issues/8027). [#8243](https://github.com/ClickHouse/ClickHouse/pull/8243) ([Amos Bird](https://github.com/amosbird))
+* Update `zlib-ng` to fix memory sanitizer problems. [#7182](https://github.com/ClickHouse/ClickHouse/pull/7182) [#8206](https://github.com/ClickHouse/ClickHouse/pull/8206) ([Alexander Kuzmenkov](https://github.com/akuzm))
+* Enable internal MySQL library on non-Linux system, because usage of OS packages is very fragile and usually doesn't work at all. This fixes [#5765](https://github.com/ClickHouse/ClickHouse/issues/5765). [#8426](https://github.com/ClickHouse/ClickHouse/pull/8426) ([alexey-milovidov](https://github.com/alexey-milovidov))
+* Fixed build on some systems after enabling `libc++`. This supersedes [#8374](https://github.com/ClickHouse/ClickHouse/issues/8374). [#8380](https://github.com/ClickHouse/ClickHouse/pull/8380) ([alexey-milovidov](https://github.com/alexey-milovidov))
+* Make `Field` methods more type-safe to find more errors. [#7386](https://github.com/ClickHouse/ClickHouse/pull/7386) [#8209](https://github.com/ClickHouse/ClickHouse/pull/8209) ([Alexander Kuzmenkov](https://github.com/akuzm))
+* Added missing files to the `libc-headers` submodule. [#8507](https://github.com/ClickHouse/ClickHouse/pull/8507) ([alexey-milovidov](https://github.com/alexey-milovidov))
+* Fix wrong `JSON` quoting in performance test output. [#8497](https://github.com/ClickHouse/ClickHouse/pull/8497) ([Nikolai Kochetov](https://github.com/KochetovNicolai))
+* Now stack trace is displayed for `std::exception` and `Poco::Exception`. In previous versions it was available only for `DB::Exception`. This improves diagnostics. [#8501](https://github.com/ClickHouse/ClickHouse/pull/8501) ([alexey-milovidov](https://github.com/alexey-milovidov))
+* Porting `clock_gettime` and `clock_nanosleep` for fresh glibc versions. [#8054](https://github.com/ClickHouse/ClickHouse/pull/8054) ([Amos Bird](https://github.com/amosbird))
+* Enable `part_log` in example config for developers. [#8609](https://github.com/ClickHouse/ClickHouse/pull/8609) ([alexey-milovidov](https://github.com/alexey-milovidov))
+* Fix async nature of reload in `01036_no_superfluous_dict_reload_on_create_database*`. [#8111](https://github.com/ClickHouse/ClickHouse/pull/8111) ([Azat Khuzhin](https://github.com/azat))
+* Fixed codec performance tests. [#8615](https://github.com/ClickHouse/ClickHouse/pull/8615) ([Vasily Nemkov](https://github.com/Enmk))
+* Add install scripts for `.tgz` build and documentation for them. [#8612](https://github.com/ClickHouse/ClickHouse/pull/8612) [#8591](https://github.com/ClickHouse/ClickHouse/pull/8591) ([alesapin](https://github.com/alesapin))
+* Removed old `ZSTD` test (it was created in year 2016 to reproduce the bug that pre 1.0 version of ZSTD has had). This fixes [#8618](https://github.com/ClickHouse/ClickHouse/issues/8618). [#8619](https://github.com/ClickHouse/ClickHouse/pull/8619) ([alexey-milovidov](https://github.com/alexey-milovidov))
+* Fixed build on Mac OS Catalina. [#8600](https://github.com/ClickHouse/ClickHouse/pull/8600) ([meo](https://github.com/meob))
+* Increased number of rows in codec performance tests to make results noticeable. [#8574](https://github.com/ClickHouse/ClickHouse/pull/8574) ([Vasily Nemkov](https://github.com/Enmk))
+* In debug builds, treat `LOGICAL_ERROR` exceptions as assertion failures, so that they are easier to notice. [#8475](https://github.com/ClickHouse/ClickHouse/pull/8475) ([Alexander Kuzmenkov](https://github.com/akuzm))
+* Make formats-related performance test more deterministic. [#8477](https://github.com/ClickHouse/ClickHouse/pull/8477) ([alexey-milovidov](https://github.com/alexey-milovidov))
+* Update `lz4` to fix a MemorySanitizer failure. [#8181](https://github.com/ClickHouse/ClickHouse/pull/8181) ([Alexander Kuzmenkov](https://github.com/akuzm))
+* Suppress a known MemorySanitizer false positive in exception handling. [#8182](https://github.com/ClickHouse/ClickHouse/pull/8182) ([Alexander Kuzmenkov](https://github.com/akuzm))
+* Update `gcc` and `g++` to version 9 in `build/docker/build.sh` [#7766](https://github.com/ClickHouse/ClickHouse/pull/7766) ([TLightSky](https://github.com/tlightsky))
+* Add performance test case to test that `PREWHERE` is worse than `WHERE`. [#7768](https://github.com/ClickHouse/ClickHouse/pull/7768) ([Amos Bird](https://github.com/amosbird))
+* Progress towards fixing one flacky test. [#8621](https://github.com/ClickHouse/ClickHouse/pull/8621) ([alexey-milovidov](https://github.com/alexey-milovidov))
+* Avoid MemorySanitizer report for data from `libunwind`. [#8539](https://github.com/ClickHouse/ClickHouse/pull/8539) ([alexey-milovidov](https://github.com/alexey-milovidov))
+* Updated `libc++` to the latest version. [#8324](https://github.com/ClickHouse/ClickHouse/pull/8324) ([alexey-milovidov](https://github.com/alexey-milovidov))
+* Build ICU library from sources. This fixes [#6460](https://github.com/ClickHouse/ClickHouse/issues/6460). [#8219](https://github.com/ClickHouse/ClickHouse/pull/8219) ([alexey-milovidov](https://github.com/alexey-milovidov))
+* Switched from `libressl` to `openssl`. ClickHouse should support TLS 1.3 and SNI after this change. This fixes [#8171](https://github.com/ClickHouse/ClickHouse/issues/8171). [#8218](https://github.com/ClickHouse/ClickHouse/pull/8218) ([alexey-milovidov](https://github.com/alexey-milovidov))
+* Fixed UBSan report when using `chacha20_poly1305` from SSL (happens on connect to https://yandex.ru/). [#8214](https://github.com/ClickHouse/ClickHouse/pull/8214) ([alexey-milovidov](https://github.com/alexey-milovidov))
+* Fix mode of default password file for `.deb` linux distros. [#8075](https://github.com/ClickHouse/ClickHouse/pull/8075) ([proller](https://github.com/proller))
+* Improved expression for getting `clickhouse-server` PID in `clickhouse-test`. [#8063](https://github.com/ClickHouse/ClickHouse/pull/8063) ([Alexander Kazakov](https://github.com/Akazz))
+* Updated contrib/googletest to v1.10.0. [#8587](https://github.com/ClickHouse/ClickHouse/pull/8587) ([Alexander Burmak](https://github.com/Alex-Burmak))
+* Fixed ThreadSaninitizer report in `base64` library. Also updated this library to the latest version, but it doesn't matter. This fixes [#8397](https://github.com/ClickHouse/ClickHouse/issues/8397). [#8403](https://github.com/ClickHouse/ClickHouse/pull/8403) ([alexey-milovidov](https://github.com/alexey-milovidov))
+* Fix `00600_replace_running_query` for processors. [#8272](https://github.com/ClickHouse/ClickHouse/pull/8272) ([Nikolai Kochetov](https://github.com/KochetovNicolai))
+* Remove support for `tcmalloc` to make `CMakeLists.txt` simpler. [#8310](https://github.com/ClickHouse/ClickHouse/pull/8310) ([alexey-milovidov](https://github.com/alexey-milovidov))
+* Release gcc builds now use `libc++` instead of `libstdc++`. Recently `libc++` was used only with clang. This will improve consistency of build configurations and portability. [#8311](https://github.com/ClickHouse/ClickHouse/pull/8311) ([alexey-milovidov](https://github.com/alexey-milovidov))
+* Enable ICU library for build with MemorySanitizer. [#8222](https://github.com/ClickHouse/ClickHouse/pull/8222) ([alexey-milovidov](https://github.com/alexey-milovidov))
+* Suppress warnings from `CapNProto` library. [#8224](https://github.com/ClickHouse/ClickHouse/pull/8224) ([alexey-milovidov](https://github.com/alexey-milovidov))
+* Removed special cases of code for `tcmalloc`, because it's no longer supported. [#8225](https://github.com/ClickHouse/ClickHouse/pull/8225) ([alexey-milovidov](https://github.com/alexey-milovidov))
+* In CI coverage task, kill the server gracefully to allow it to save the coverage report. This fixes incomplete coverage reports we've been seeing lately. [#8142](https://github.com/ClickHouse/ClickHouse/pull/8142) ([alesapin](https://github.com/alesapin))
+* Performance tests for all codecs against `Float64` and `UInt64` values. [#8349](https://github.com/ClickHouse/ClickHouse/pull/8349) ([Vasily Nemkov](https://github.com/Enmk))
+* `termcap` is very much deprecated and lead to various problems (f.g. missing "up" cap and echoing `^J` instead of multi line) . Favor `terminfo` or bundled `ncurses`. [#7737](https://github.com/ClickHouse/ClickHouse/pull/7737) ([Amos Bird](https://github.com/amosbird))
+* Fix `test_storage_s3` integration test. [#7734](https://github.com/ClickHouse/ClickHouse/pull/7734) ([Nikolai Kochetov](https://github.com/KochetovNicolai))
+* Support `StorageFile(, null) ` to insert block into given format file without actually write to disk. This is required for performance tests. [#8455](https://github.com/ClickHouse/ClickHouse/pull/8455) ([Amos Bird](https://github.com/amosbird))
+* Added argument `--print-time` to functional tests which prints execution time per test. [#8001](https://github.com/ClickHouse/ClickHouse/pull/8001) ([Nikolai Kochetov](https://github.com/KochetovNicolai))
+* Added asserts to `KeyCondition` while evaluating RPN. This will fix warning from gcc-9. [#8279](https://github.com/ClickHouse/ClickHouse/pull/8279) ([alexey-milovidov](https://github.com/alexey-milovidov))
+* Dump cmake options in CI builds. [#8273](https://github.com/ClickHouse/ClickHouse/pull/8273) ([Alexander Kuzmenkov](https://github.com/akuzm))
+* Don't generate debug info for some fat libraries. [#8271](https://github.com/ClickHouse/ClickHouse/pull/8271) ([alexey-milovidov](https://github.com/alexey-milovidov))
+* Make `log_to_console.xml` always log to stderr, regardless of is it interactive or not. [#8395](https://github.com/ClickHouse/ClickHouse/pull/8395) ([Alexander Kuzmenkov](https://github.com/akuzm))
+* Removed some unused features from `clickhouse-performance-test` tool. [#8555](https://github.com/ClickHouse/ClickHouse/pull/8555) ([alexey-milovidov](https://github.com/alexey-milovidov))
+* Now we will also search for `lld-X` with corresponding `clang-X` version. [#8092](https://github.com/ClickHouse/ClickHouse/pull/8092) ([alesapin](https://github.com/alesapin))
+* Parquet build improvement. [#8421](https://github.com/ClickHouse/ClickHouse/pull/8421) ([maxulan](https://github.com/maxulan))
+* More GCC warnings [#8221](https://github.com/ClickHouse/ClickHouse/pull/8221) ([kreuzerkrieg](https://github.com/kreuzerkrieg))
+* Package for Arch Linux now allows to run ClickHouse server, and not only client. [#8534](https://github.com/ClickHouse/ClickHouse/pull/8534) ([Vladimir Chebotarev](https://github.com/excitoon))
+* Fix test with processors. Tiny performance fixes. [#7672](https://github.com/ClickHouse/ClickHouse/pull/7672) ([Nikolai Kochetov](https://github.com/KochetovNicolai))
+* Update contrib/protobuf. [#8256](https://github.com/ClickHouse/ClickHouse/pull/8256) ([Matwey V. Kornilov](https://github.com/matwey))
+* In preparation of switching to c++20 as a new year celebration. "May the C++ force be with ClickHouse." [#8447](https://github.com/ClickHouse/ClickHouse/pull/8447) ([Amos Bird](https://github.com/amosbird))
+
+#### Experimental Feature
+* Added experimental setting `min_bytes_to_use_mmap_io`. It allows to read big files without copying data from kernel to userspace. The setting is disabled by default. Recommended threshold is about 64 MB, because mmap/munmap is slow. [#8520](https://github.com/ClickHouse/ClickHouse/pull/8520) ([alexey-milovidov](https://github.com/alexey-milovidov))
+* Reworked quotas as a part of access control system. Added new table `system.quotas`, new functions `currentQuota`, `currentQuotaKey`, new SQL syntax `CREATE QUOTA`, `ALTER QUOTA`, `DROP QUOTA`, `SHOW QUOTA`. [#7257](https://github.com/ClickHouse/ClickHouse/pull/7257) ([Vitaly Baranov](https://github.com/vitlibar))
+* Allow skipping unknown settings with warnings instead of throwing exceptions. [#7653](https://github.com/ClickHouse/ClickHouse/pull/7653) ([Vitaly Baranov](https://github.com/vitlibar))
+* Reworked row policies as a part of access control system. Added new table `system.row_policies`, new function `currentRowPolicies()`, new SQL syntax `CREATE POLICY`, `ALTER POLICY`, `DROP POLICY`, `SHOW CREATE POLICY`, `SHOW POLICIES`. [#7808](https://github.com/ClickHouse/ClickHouse/pull/7808) ([Vitaly Baranov](https://github.com/vitlibar))
+
+#### Security Fix
+* Fixed the possibility of reading directories structure in tables with `File` table engine. This fixes [#8536](https://github.com/ClickHouse/ClickHouse/issues/8536). [#8537](https://github.com/ClickHouse/ClickHouse/pull/8537) ([alexey-milovidov](https://github.com/alexey-milovidov))
+
+## [Changelog for 2019](https://github.com/ClickHouse/ClickHouse/blob/master/docs/en/changelog/2019.md)
diff --git a/docs/en/whats_new/index.md b/docs/en/whats_new/index.md
new file mode 100644
index 00000000000..3b8886a9f0a
--- /dev/null
+++ b/docs/en/whats_new/index.md
@@ -0,0 +1,5 @@
+---
+toc_folder_title: What's New
+toc_priority: 72
+---
+
diff --git a/docs/en/roadmap.md b/docs/en/whats_new/roadmap.md
similarity index 87%
rename from docs/en/roadmap.md
rename to docs/en/whats_new/roadmap.md
index 434058d8311..e50d663fcb0 100644
--- a/docs/en/roadmap.md
+++ b/docs/en/whats_new/roadmap.md
@@ -1,3 +1,8 @@
+---
+toc_priority: 74
+toc_title: Roadmap
+---
+
# Roadmap {#roadmap}
## Q1 2020 {#q1-2020}
diff --git a/docs/en/security_changelog.md b/docs/en/whats_new/security_changelog.md
similarity index 98%
rename from docs/en/security_changelog.md
rename to docs/en/whats_new/security_changelog.md
index 9d33e1083d5..80f5d2a421a 100644
--- a/docs/en/security_changelog.md
+++ b/docs/en/whats_new/security_changelog.md
@@ -1,3 +1,8 @@
+---
+toc_priority: 76
+toc_title: Security Changelog
+---
+
## Fixed in ClickHouse Release 19.14.3.3, 2019-09-10 {#fixed-in-clickhouse-release-19-14-3-3-2019-09-10}
### CVE-2019-15024 {#cve-2019-15024}
diff --git a/docs/fa/changelog/2017.md b/docs/fa/changelog/2017.md
deleted file mode 120000
index bf4fe14279d..00000000000
--- a/docs/fa/changelog/2017.md
+++ /dev/null
@@ -1 +0,0 @@
-../../en/changelog/2017.md
\ No newline at end of file
diff --git a/docs/fa/changelog/2017.md b/docs/fa/changelog/2017.md
new file mode 100644
index 00000000000..95156754100
--- /dev/null
+++ b/docs/fa/changelog/2017.md
@@ -0,0 +1,265 @@
+---
+en_copy: true
+---
+
+### ClickHouse release 1.1.54327, 2017-12-21 {#clickhouse-release-1-1-54327-2017-12-21}
+
+This release contains bug fixes for the previous release 1.1.54318:
+
+- Fixed bug with possible race condition in replication that could lead to data loss. This issue affects versions 1.1.54310 and 1.1.54318. If you use one of these versions with Replicated tables, the update is strongly recommended. This issue shows in logs in Warning messages like `Part ... from own log doesn't exist.` The issue is relevant even if you don’t see these messages in logs.
+
+### ClickHouse release 1.1.54318, 2017-11-30 {#clickhouse-release-1-1-54318-2017-11-30}
+
+This release contains bug fixes for the previous release 1.1.54310:
+
+- Fixed incorrect row deletions during merges in the SummingMergeTree engine
+- Fixed a memory leak in unreplicated MergeTree engines
+- Fixed performance degradation with frequent inserts in MergeTree engines
+- Fixed an issue that was causing the replication queue to stop running
+- Fixed rotation and archiving of server logs
+
+### ClickHouse release 1.1.54310, 2017-11-01 {#clickhouse-release-1-1-54310-2017-11-01}
+
+#### New features: {#new-features}
+
+- Custom partitioning key for the MergeTree family of table engines.
+- [Kafka](https://clickhouse.yandex/docs/en/operations/table_engines/kafka/) table engine.
+- Added support for loading [CatBoost](https://catboost.yandex/) models and applying them to data stored in ClickHouse.
+- Added support for time zones with non-integer offsets from UTC.
+- Added support for arithmetic operations with time intervals.
+- The range of values for the Date and DateTime types is extended to the year 2105.
+- Added the `CREATE MATERIALIZED VIEW x TO y` query (specifies an existing table for storing the data of a materialized view).
+- Added the `ATTACH TABLE` query without arguments.
+- The processing logic for Nested columns with names ending in -Map in a SummingMergeTree table was extracted to the sumMap aggregate function. You can now specify such columns explicitly.
+- Max size of the IP trie dictionary is increased to 128M entries.
+- Added the getSizeOfEnumType function.
+- Added the sumWithOverflow aggregate function.
+- Added support for the Cap’n Proto input format.
+- You can now customize compression level when using the zstd algorithm.
+
+#### Backward incompatible changes: {#backward-incompatible-changes}
+
+- Creation of temporary tables with an engine other than Memory is not allowed.
+- Explicit creation of tables with the View or MaterializedView engine is not allowed.
+- During table creation, a new check verifies that the sampling key expression is included in the primary key.
+
+#### Bug fixes: {#bug-fixes}
+
+- Fixed hangups when synchronously inserting into a Distributed table.
+- Fixed nonatomic adding and removing of parts in Replicated tables.
+- Data inserted into a materialized view is not subjected to unnecessary deduplication.
+- Executing a query to a Distributed table for which the local replica is lagging and remote replicas are unavailable does not result in an error anymore.
+- Users don’t need access permissions to the `default` database to create temporary tables anymore.
+- Fixed crashing when specifying the Array type without arguments.
+- Fixed hangups when the disk volume containing server logs is full.
+- Fixed an overflow in the toRelativeWeekNum function for the first week of the Unix epoch.
+
+#### Build improvements: {#build-improvements}
+
+- Several third-party libraries (notably Poco) were updated and converted to git submodules.
+
+### ClickHouse release 1.1.54304, 2017-10-19 {#clickhouse-release-1-1-54304-2017-10-19}
+
+#### New features: {#new-features-1}
+
+- TLS support in the native protocol (to enable, set `tcp_ssl_port` in `config.xml` ).
+
+#### Bug fixes: {#bug-fixes-1}
+
+- `ALTER` for replicated tables now tries to start running as soon as possible.
+- Fixed crashing when reading data with the setting `preferred_block_size_bytes=0.`
+- Fixed crashes of `clickhouse-client` when pressing `Page Down`
+- Correct interpretation of certain complex queries with `GLOBAL IN` and `UNION ALL`
+- `FREEZE PARTITION` always works atomically now.
+- Empty POST requests now return a response with code 411.
+- Fixed interpretation errors for expressions like `CAST(1 AS Nullable(UInt8)).`
+- Fixed an error when reading `Array(Nullable(String))` columns from `MergeTree` tables.
+- Fixed crashing when parsing queries like `SELECT dummy AS dummy, dummy AS b`
+- Users are updated correctly with invalid `users.xml`
+- Correct handling when an executable dictionary returns a non-zero response code.
+
+### ClickHouse release 1.1.54292, 2017-09-20 {#clickhouse-release-1-1-54292-2017-09-20}
+
+#### New features: {#new-features-2}
+
+- Added the `pointInPolygon` function for working with coordinates on a coordinate plane.
+- Added the `sumMap` aggregate function for calculating the sum of arrays, similar to `SummingMergeTree`.
+- Added the `trunc` function. Improved performance of the rounding functions (`round`, `floor`, `ceil`, `roundToExp2`) and corrected the logic of how they work. Changed the logic of the `roundToExp2` function for fractions and negative numbers.
+- The ClickHouse executable file is now less dependent on the libc version. The same ClickHouse executable file can run on a wide variety of Linux systems. There is still a dependency when using compiled queries (with the setting `compile = 1` , which is not used by default).
+- Reduced the time needed for dynamic compilation of queries.
+
+#### Bug fixes: {#bug-fixes-2}
+
+- Fixed an error that sometimes produced `part ... intersects previous part` messages and weakened replica consistency.
+- Fixed an error that caused the server to lock up if ZooKeeper was unavailable during shutdown.
+- Removed excessive logging when restoring replicas.
+- Fixed an error in the UNION ALL implementation.
+- Fixed an error in the concat function that occurred if the first column in a block has the Array type.
+- Progress is now displayed correctly in the system.merges table.
+
+### ClickHouse release 1.1.54289, 2017-09-13 {#clickhouse-release-1-1-54289-2017-09-13}
+
+#### New features: {#new-features-3}
+
+- `SYSTEM` queries for server administration: `SYSTEM RELOAD DICTIONARY`, `SYSTEM RELOAD DICTIONARIES`, `SYSTEM DROP DNS CACHE`, `SYSTEM SHUTDOWN`, `SYSTEM KILL`.
+- Added functions for working with arrays: `concat`, `arraySlice`, `arrayPushBack`, `arrayPushFront`, `arrayPopBack`, `arrayPopFront`.
+- Added `root` and `identity` parameters for the ZooKeeper configuration. This allows you to isolate individual users on the same ZooKeeper cluster.
+- Added aggregate functions `groupBitAnd`, `groupBitOr`, and `groupBitXor` (for compatibility, they are also available under the names `BIT_AND`, `BIT_OR`, and `BIT_XOR`).
+- External dictionaries can be loaded from MySQL by specifying a socket in the filesystem.
+- External dictionaries can be loaded from MySQL over SSL (`ssl_cert`, `ssl_key`, `ssl_ca` parameters).
+- Added the `max_network_bandwidth_for_user` setting to restrict the overall bandwidth use for queries per user.
+- Support for `DROP TABLE` for temporary tables.
+- Support for reading `DateTime` values in Unix timestamp format from the `CSV` and `JSONEachRow` formats.
+- Lagging replicas in distributed queries are now excluded by default (the default threshold is 5 minutes).
+- FIFO locking is used during ALTER: an ALTER query isn’t blocked indefinitely for continuously running queries.
+- Option to set `umask` in the config file.
+- Improved performance for queries with `DISTINCT` .
+
+#### Bug fixes: {#bug-fixes-3}
+
+- Improved the process for deleting old nodes in ZooKeeper. Previously, old nodes sometimes didn’t get deleted if there were very frequent inserts, which caused the server to be slow to shut down, among other things.
+- Fixed randomization when choosing hosts for the connection to ZooKeeper.
+- Fixed the exclusion of lagging replicas in distributed queries if the replica is localhost.
+- Fixed an error where a data part in a `ReplicatedMergeTree` table could be broken after running `ALTER MODIFY` on an element in a `Nested` structure.
+- Fixed an error that could cause SELECT queries to “hang”.
+- Improvements to distributed DDL queries.
+- Fixed the query `CREATE TABLE ... AS `.
+- Resolved the deadlock in the `ALTER ... CLEAR COLUMN IN PARTITION` query for `Buffer` tables.
+- Fixed the invalid default value for `Enum` s (0 instead of the minimum) when using the `JSONEachRow` and `TSKV` formats.
+- Resolved the appearance of zombie processes when using a dictionary with an `executable` source.
+- Fixed segfault for the HEAD query.
+
+#### Improved workflow for developing and assembling ClickHouse: {#improved-workflow-for-developing-and-assembling-clickhouse}
+
+- You can use `pbuilder` to build ClickHouse.
+- You can use `libc++` instead of `libstdc++` for builds on Linux.
+- Added instructions for using static code analysis tools: `Coverage`, `clang-tidy`, `cppcheck`.
+
+#### Please note when upgrading: {#please-note-when-upgrading}
+
+- There is now a higher default value for the MergeTree setting `max_bytes_to_merge_at_max_space_in_pool` (the maximum total size of data parts to merge, in bytes): it has increased from 100 GiB to 150 GiB. This might result in large merges running after the server upgrade, which could cause an increased load on the disk subsystem. If the free space available on the server is less than twice the total amount of the merges that are running, this will cause all other merges to stop running, including merges of small data parts. As a result, INSERT queries will fail with the message “Merges are processing significantly slower than inserts.” Use the `SELECT * FROM system.merges` query to monitor the situation. You can also check the `DiskSpaceReservedForMerge` metric in the `system.metrics` table, or in Graphite. You don’t need to do anything to fix this, since the issue will resolve itself once the large merges finish. If you find this unacceptable, you can restore the previous value for the `max_bytes_to_merge_at_max_space_in_pool` setting. To do this, go to the section in config.xml, set ``` ``107374182400 ``` and restart the server.
+
+### ClickHouse release 1.1.54284, 2017-08-29 {#clickhouse-release-1-1-54284-2017-08-29}
+
+- This is a bugfix release for the previous 1.1.54282 release. It fixes leaks in the parts directory in ZooKeeper.
+
+### ClickHouse release 1.1.54282, 2017-08-23 {#clickhouse-release-1-1-54282-2017-08-23}
+
+This release contains bug fixes for the previous release 1.1.54276:
+
+- Fixed `DB::Exception: Assertion violation: !_path.empty()` when inserting into a Distributed table.
+- Fixed parsing when inserting in RowBinary format if input data starts with’;’.
+- Errors during runtime compilation of certain aggregate functions (e.g. `groupArray()`).
+
+### Clickhouse Release 1.1.54276, 2017-08-16 {#clickhouse-release-1-1-54276-2017-08-16}
+
+#### New features: {#new-features-4}
+
+- Added an optional WITH section for a SELECT query. Example query: `WITH 1+1 AS a SELECT a, a*a`
+- INSERT can be performed synchronously in a Distributed table: OK is returned only after all the data is saved on all the shards. This is activated by the setting insert\_distributed\_sync=1.
+- Added the UUID data type for working with 16-byte identifiers.
+- Added aliases of CHAR, FLOAT and other types for compatibility with the Tableau.
+- Added the functions toYYYYMM, toYYYYMMDD, and toYYYYMMDDhhmmss for converting time into numbers.
+- You can use IP addresses (together with the hostname) to identify servers for clustered DDL queries.
+- Added support for non-constant arguments and negative offsets in the function `substring(str, pos, len).`
+- Added the max\_size parameter for the `groupArray(max_size)(column)` aggregate function, and optimized its performance.
+
+#### Main changes: {#main-changes}
+
+- Security improvements: all server files are created with 0640 permissions (can be changed via config parameter).
+- Improved error messages for queries with invalid syntax.
+- Significantly reduced memory consumption and improved performance when merging large sections of MergeTree data.
+- Significantly increased the performance of data merges for the ReplacingMergeTree engine.
+- Improved performance for asynchronous inserts from a Distributed table by combining multiple source inserts. To enable this functionality, use the setting distributed\_directory\_monitor\_batch\_inserts=1.
+
+#### Backward incompatible changes: {#backward-incompatible-changes-1}
+
+- Changed the binary format of aggregate states of `groupArray(array_column)` functions for arrays.
+
+#### Complete list of changes: {#complete-list-of-changes}
+
+- Added the `output_format_json_quote_denormals` setting, which enables outputting nan and inf values in JSON format.
+- Optimized stream allocation when reading from a Distributed table.
+- Settings can be configured in readonly mode if the value doesn’t change.
+- Added the ability to retrieve non-integer granules of the MergeTree engine in order to meet restrictions on the block size specified in the preferred\_block\_size\_bytes setting. The purpose is to reduce the consumption of RAM and increase cache locality when processing queries from tables with large columns.
+- Efficient use of indexes that contain expressions like `toStartOfHour(x)` for conditions like `toStartOfHour(x) op сonstexpr.`
+- Added new settings for MergeTree engines (the merge\_tree section in config.xml):
+ - replicated\_deduplication\_window\_seconds sets the number of seconds allowed for deduplicating inserts in Replicated tables.
+ - cleanup\_delay\_period sets how often to start cleanup to remove outdated data.
+ - replicated\_can\_become\_leader can prevent a replica from becoming the leader (and assigning merges).
+- Accelerated cleanup to remove outdated data from ZooKeeper.
+- Multiple improvements and fixes for clustered DDL queries. Of particular interest is the new setting distributed\_ddl\_task\_timeout, which limits the time to wait for a response from the servers in the cluster. If a ddl request has not been performed on all hosts, a response will contain a timeout error and a request will be executed in an async mode.
+- Improved display of stack traces in the server logs.
+- Added the “none” value for the compression method.
+- You can use multiple dictionaries\_config sections in config.xml.
+- It is possible to connect to MySQL through a socket in the file system.
+- The system.parts table has a new column with information about the size of marks, in bytes.
+
+#### Bug fixes: {#bug-fixes-4}
+
+- Distributed tables using a Merge table now work correctly for a SELECT query with a condition on the `_table` field.
+- Fixed a rare race condition in ReplicatedMergeTree when checking data parts.
+- Fixed possible freezing on “leader election” when starting a server.
+- The max\_replica\_delay\_for\_distributed\_queries setting was ignored when using a local replica of the data source. This has been fixed.
+- Fixed incorrect behavior of `ALTER TABLE CLEAR COLUMN IN PARTITION` when attempting to clean a non-existing column.
+- Fixed an exception in the multiIf function when using empty arrays or strings.
+- Fixed excessive memory allocations when deserializing Native format.
+- Fixed incorrect auto-update of Trie dictionaries.
+- Fixed an exception when running queries with a GROUP BY clause from a Merge table when using SAMPLE.
+- Fixed a crash of GROUP BY when using distributed\_aggregation\_memory\_efficient=1.
+- Now you can specify the database.table in the right side of IN and JOIN.
+- Too many threads were used for parallel aggregation. This has been fixed.
+- Fixed how the “if” function works with FixedString arguments.
+- SELECT worked incorrectly from a Distributed table for shards with a weight of 0. This has been fixed.
+- Running `CREATE VIEW IF EXISTS no longer causes crashes.`
+- Fixed incorrect behavior when input\_format\_skip\_unknown\_fields=1 is set and there are negative numbers.
+- Fixed an infinite loop in the `dictGetHierarchy()` function if there is some invalid data in the dictionary.
+- Fixed `Syntax error: unexpected (...)` errors when running distributed queries with subqueries in an IN or JOIN clause and Merge tables.
+- Fixed an incorrect interpretation of a SELECT query from Dictionary tables.
+- Fixed the “Cannot mremap” error when using arrays in IN and JOIN clauses with more than 2 billion elements.
+- Fixed the failover for dictionaries with MySQL as the source.
+
+#### Improved workflow for developing and assembling ClickHouse: {#improved-workflow-for-developing-and-assembling-clickhouse-1}
+
+- Builds can be assembled in Arcadia.
+- You can use gcc 7 to compile ClickHouse.
+- Parallel builds using ccache+distcc are faster now.
+
+### ClickHouse release 1.1.54245, 2017-07-04 {#clickhouse-release-1-1-54245-2017-07-04}
+
+#### New features: {#new-features-5}
+
+- Distributed DDL (for example, `CREATE TABLE ON CLUSTER`)
+- The replicated query `ALTER TABLE CLEAR COLUMN IN PARTITION.`
+- The engine for Dictionary tables (access to dictionary data in the form of a table).
+- Dictionary database engine (this type of database automatically has Dictionary tables available for all the connected external dictionaries).
+- You can check for updates to the dictionary by sending a request to the source.
+- Qualified column names
+- Quoting identifiers using double quotation marks.
+- Sessions in the HTTP interface.
+- The OPTIMIZE query for a Replicated table can can run not only on the leader.
+
+#### Backward incompatible changes: {#backward-incompatible-changes-2}
+
+- Removed SET GLOBAL.
+
+#### Minor changes: {#minor-changes}
+
+- Now after an alert is triggered, the log prints the full stack trace.
+- Relaxed the verification of the number of damaged/extra data parts at startup (there were too many false positives).
+
+#### Bug fixes: {#bug-fixes-5}
+
+- Fixed a bad connection “sticking” when inserting into a Distributed table.
+- GLOBAL IN now works for a query from a Merge table that looks at a Distributed table.
+- The incorrect number of cores was detected on a Google Compute Engine virtual machine. This has been fixed.
+- Changes in how an executable source of cached external dictionaries works.
+- Fixed the comparison of strings containing null characters.
+- Fixed the comparison of Float32 primary key fields with constants.
+- Previously, an incorrect estimate of the size of a field could lead to overly large allocations.
+- Fixed a crash when querying a Nullable column added to a table using ALTER.
+- Fixed a crash when sorting by a Nullable column, if the number of rows is less than LIMIT.
+- Fixed an ORDER BY subquery consisting of only constant values.
+- Previously, a Replicated table could remain in the invalid state after a failed DROP TABLE.
+- Aliases for scalar subqueries with empty results are no longer lost.
+- Now a query that used compilation does not fail with an error if the .so file gets damaged.
diff --git a/docs/fa/changelog/2018.md b/docs/fa/changelog/2018.md
deleted file mode 120000
index 20799251f43..00000000000
--- a/docs/fa/changelog/2018.md
+++ /dev/null
@@ -1 +0,0 @@
-../../en/changelog/2018.md
\ No newline at end of file
diff --git a/docs/fa/changelog/2018.md b/docs/fa/changelog/2018.md
new file mode 100644
index 00000000000..49bef18cbf3
--- /dev/null
+++ b/docs/fa/changelog/2018.md
@@ -0,0 +1,1060 @@
+---
+en_copy: true
+---
+
+## ClickHouse release 18.16 {#clickhouse-release-18-16}
+
+### ClickHouse release 18.16.1, 2018-12-21 {#clickhouse-release-18-16-1-2018-12-21}
+
+#### Bug fixes: {#bug-fixes}
+
+- Fixed an error that led to problems with updating dictionaries with the ODBC source. [\#3825](https://github.com/ClickHouse/ClickHouse/issues/3825), [\#3829](https://github.com/ClickHouse/ClickHouse/issues/3829)
+- JIT compilation of aggregate functions now works with LowCardinality columns. [\#3838](https://github.com/ClickHouse/ClickHouse/issues/3838)
+
+#### Improvements: {#improvements}
+
+- Added the `low_cardinality_allow_in_native_format` setting (enabled by default). When disabled, LowCardinality columns will be converted to ordinary columns for SELECT queries and ordinary columns will be expected for INSERT queries. [\#3879](https://github.com/ClickHouse/ClickHouse/pull/3879)
+
+#### Build improvements: {#build-improvements}
+
+- Fixes for builds on macOS and ARM.
+
+### ClickHouse release 18.16.0, 2018-12-14 {#clickhouse-release-18-16-0-2018-12-14}
+
+#### New features: {#new-features}
+
+- `DEFAULT` expressions are evaluated for missing fields when loading data in semi-structured input formats (`JSONEachRow`, `TSKV`). The feature is enabled with the `insert_sample_with_metadata` setting. [\#3555](https://github.com/ClickHouse/ClickHouse/pull/3555)
+- The `ALTER TABLE` query now has the `MODIFY ORDER BY` action for changing the sorting key when adding or removing a table column. This is useful for tables in the `MergeTree` family that perform additional tasks when merging based on this sorting key, such as `SummingMergeTree`, `AggregatingMergeTree`, and so on. [\#3581](https://github.com/ClickHouse/ClickHouse/pull/3581) [\#3755](https://github.com/ClickHouse/ClickHouse/pull/3755)
+- For tables in the `MergeTree` family, now you can specify a different sorting key (`ORDER BY`) and index (`PRIMARY KEY`). The sorting key can be longer than the index. [\#3581](https://github.com/ClickHouse/ClickHouse/pull/3581)
+- Added the `hdfs` table function and the `HDFS` table engine for importing and exporting data to HDFS. [chenxing-xc](https://github.com/ClickHouse/ClickHouse/pull/3617)
+- Added functions for working with base64: `base64Encode`, `base64Decode`, `tryBase64Decode`. [Alexander Krasheninnikov](https://github.com/ClickHouse/ClickHouse/pull/3350)
+- Now you can use a parameter to configure the precision of the `uniqCombined` aggregate function (select the number of HyperLogLog cells). [\#3406](https://github.com/ClickHouse/ClickHouse/pull/3406)
+- Added the `system.contributors` table that contains the names of everyone who made commits in ClickHouse. [\#3452](https://github.com/ClickHouse/ClickHouse/pull/3452)
+- Added the ability to omit the partition for the `ALTER TABLE ... FREEZE` query in order to back up all partitions at once. [\#3514](https://github.com/ClickHouse/ClickHouse/pull/3514)
+- Added `dictGet` and `dictGetOrDefault` functions that don’t require specifying the type of return value. The type is determined automatically from the dictionary description. [Amos Bird](https://github.com/ClickHouse/ClickHouse/pull/3564)
+- Now you can specify comments for a column in the table description and change it using `ALTER`. [\#3377](https://github.com/ClickHouse/ClickHouse/pull/3377)
+- Reading is supported for `Join` type tables with simple keys. [Amos Bird](https://github.com/ClickHouse/ClickHouse/pull/3728)
+- Now you can specify the options `join_use_nulls`, `max_rows_in_join`, `max_bytes_in_join`, and `join_overflow_mode` when creating a `Join` type table. [Amos Bird](https://github.com/ClickHouse/ClickHouse/pull/3728)
+- Added the `joinGet` function that allows you to use a `Join` type table like a dictionary. [Amos Bird](https://github.com/ClickHouse/ClickHouse/pull/3728)
+- Added the `partition_key`, `sorting_key`, `primary_key`, and `sampling_key` columns to the `system.tables` table in order to provide information about table keys. [\#3609](https://github.com/ClickHouse/ClickHouse/pull/3609)
+- Added the `is_in_partition_key`, `is_in_sorting_key`, `is_in_primary_key`, and `is_in_sampling_key` columns to the `system.columns` table. [\#3609](https://github.com/ClickHouse/ClickHouse/pull/3609)
+- Added the `min_time` and `max_time` columns to the `system.parts` table. These columns are populated when the partitioning key is an expression consisting of `DateTime` columns. [Emmanuel Donin de Rosière](https://github.com/ClickHouse/ClickHouse/pull/3800)
+
+#### Bug fixes: {#bug-fixes-1}
+
+- Fixes and performance improvements for the `LowCardinality` data type. `GROUP BY` using `LowCardinality(Nullable(...))`. Getting the values of `extremes`. Processing high-order functions. `LEFT ARRAY JOIN`. Distributed `GROUP BY`. Functions that return `Array`. Execution of `ORDER BY`. Writing to `Distributed` tables (nicelulu). Backward compatibility for `INSERT` queries from old clients that implement the `Native` protocol. Support for `LowCardinality` for `JOIN`. Improved performance when working in a single stream. [\#3823](https://github.com/ClickHouse/ClickHouse/pull/3823) [\#3803](https://github.com/ClickHouse/ClickHouse/pull/3803) [\#3799](https://github.com/ClickHouse/ClickHouse/pull/3799) [\#3769](https://github.com/ClickHouse/ClickHouse/pull/3769) [\#3744](https://github.com/ClickHouse/ClickHouse/pull/3744) [\#3681](https://github.com/ClickHouse/ClickHouse/pull/3681) [\#3651](https://github.com/ClickHouse/ClickHouse/pull/3651) [\#3649](https://github.com/ClickHouse/ClickHouse/pull/3649) [\#3641](https://github.com/ClickHouse/ClickHouse/pull/3641) [\#3632](https://github.com/ClickHouse/ClickHouse/pull/3632) [\#3568](https://github.com/ClickHouse/ClickHouse/pull/3568) [\#3523](https://github.com/ClickHouse/ClickHouse/pull/3523) [\#3518](https://github.com/ClickHouse/ClickHouse/pull/3518)
+- Fixed how the `select_sequential_consistency` option works. Previously, when this setting was enabled, an incomplete result was sometimes returned after beginning to write to a new partition. [\#2863](https://github.com/ClickHouse/ClickHouse/pull/2863)
+- Databases are correctly specified when executing DDL `ON CLUSTER` queries and `ALTER UPDATE/DELETE`. [\#3772](https://github.com/ClickHouse/ClickHouse/pull/3772) [\#3460](https://github.com/ClickHouse/ClickHouse/pull/3460)
+- Databases are correctly specified for subqueries inside a VIEW. [\#3521](https://github.com/ClickHouse/ClickHouse/pull/3521)
+- Fixed a bug in `PREWHERE` with `FINAL` for `VersionedCollapsingMergeTree`. [7167bfd7](https://github.com/ClickHouse/ClickHouse/commit/7167bfd7b365538f7a91c4307ad77e552ab4e8c1)
+- Now you can use `KILL QUERY` to cancel queries that have not started yet because they are waiting for the table to be locked. [\#3517](https://github.com/ClickHouse/ClickHouse/pull/3517)
+- Corrected date and time calculations if the clocks were moved back at midnight (this happens in Iran, and happened in Moscow from 1981 to 1983). Previously, this led to the time being reset a day earlier than necessary, and also caused incorrect formatting of the date and time in text format. [\#3819](https://github.com/ClickHouse/ClickHouse/pull/3819)
+- Fixed bugs in some cases of `VIEW` and subqueries that omit the database. [Winter Zhang](https://github.com/ClickHouse/ClickHouse/pull/3521)
+- Fixed a race condition when simultaneously reading from a `MATERIALIZED VIEW` and deleting a `MATERIALIZED VIEW` due to not locking the internal `MATERIALIZED VIEW`. [\#3404](https://github.com/ClickHouse/ClickHouse/pull/3404) [\#3694](https://github.com/ClickHouse/ClickHouse/pull/3694)
+- Fixed the error `Lock handler cannot be nullptr.` [\#3689](https://github.com/ClickHouse/ClickHouse/pull/3689)
+- Fixed query processing when the `compile_expressions` option is enabled (it’s enabled by default). Nondeterministic constant expressions like the `now` function are no longer unfolded. [\#3457](https://github.com/ClickHouse/ClickHouse/pull/3457)
+- Fixed a crash when specifying a non-constant scale argument in `toDecimal32/64/128` functions.
+- Fixed an error when trying to insert an array with `NULL` elements in the `Values` format into a column of type `Array` without `Nullable` (if `input_format_values_interpret_expressions` = 1). [\#3487](https://github.com/ClickHouse/ClickHouse/pull/3487) [\#3503](https://github.com/ClickHouse/ClickHouse/pull/3503)
+- Fixed continuous error logging in `DDLWorker` if ZooKeeper is not available. [8f50c620](https://github.com/ClickHouse/ClickHouse/commit/8f50c620334988b28018213ec0092fe6423847e2)
+- Fixed the return type for `quantile*` functions from `Date` and `DateTime` types of arguments. [\#3580](https://github.com/ClickHouse/ClickHouse/pull/3580)
+- Fixed the `WITH` clause if it specifies a simple alias without expressions. [\#3570](https://github.com/ClickHouse/ClickHouse/pull/3570)
+- Fixed processing of queries with named sub-queries and qualified column names when `enable_optimize_predicate_expression` is enabled. [Winter Zhang](https://github.com/ClickHouse/ClickHouse/pull/3588)
+- Fixed the error `Attempt to attach to nullptr thread group` when working with materialized views. [Marek Vavruša](https://github.com/ClickHouse/ClickHouse/pull/3623)
+- Fixed a crash when passing certain incorrect arguments to the `arrayReverse` function. [73e3a7b6](https://github.com/ClickHouse/ClickHouse/commit/73e3a7b662161d6005e7727d8a711b930386b871)
+- Fixed the buffer overflow in the `extractURLParameter` function. Improved performance. Added correct processing of strings containing zero bytes. [141e9799](https://github.com/ClickHouse/ClickHouse/commit/141e9799e49201d84ea8e951d1bed4fb6d3dacb5)
+- Fixed buffer overflow in the `lowerUTF8` and `upperUTF8` functions. Removed the ability to execute these functions over `FixedString` type arguments. [\#3662](https://github.com/ClickHouse/ClickHouse/pull/3662)
+- Fixed a rare race condition when deleting `MergeTree` tables. [\#3680](https://github.com/ClickHouse/ClickHouse/pull/3680)
+- Fixed a race condition when reading from `Buffer` tables and simultaneously performing `ALTER` or `DROP` on the target tables. [\#3719](https://github.com/ClickHouse/ClickHouse/pull/3719)
+- Fixed a segfault if the `max_temporary_non_const_columns` limit was exceeded. [\#3788](https://github.com/ClickHouse/ClickHouse/pull/3788)
+
+#### Improvements: {#improvements-1}
+
+- The server does not write the processed configuration files to the `/etc/clickhouse-server/` directory. Instead, it saves them in the `preprocessed_configs` directory inside `path`. This means that the `/etc/clickhouse-server/` directory doesn’t have write access for the `clickhouse` user, which improves security. [\#2443](https://github.com/ClickHouse/ClickHouse/pull/2443)
+- The `min_merge_bytes_to_use_direct_io` option is set to 10 GiB by default. A merge that forms large parts of tables from the MergeTree family will be performed in `O_DIRECT` mode, which prevents excessive page cache eviction. [\#3504](https://github.com/ClickHouse/ClickHouse/pull/3504)
+- Accelerated server start when there is a very large number of tables. [\#3398](https://github.com/ClickHouse/ClickHouse/pull/3398)
+- Added a connection pool and HTTP `Keep-Alive` for connections between replicas. [\#3594](https://github.com/ClickHouse/ClickHouse/pull/3594)
+- If the query syntax is invalid, the `400 Bad Request` code is returned in the `HTTP` interface (500 was returned previously). [31bc680a](https://github.com/ClickHouse/ClickHouse/commit/31bc680ac5f4bb1d0360a8ba4696fa84bb47d6ab)
+- The `join_default_strictness` option is set to `ALL` by default for compatibility. [120e2cbe](https://github.com/ClickHouse/ClickHouse/commit/120e2cbe2ff4fbad626c28042d9b28781c805afe)
+- Removed logging to `stderr` from the `re2` library for invalid or complex regular expressions. [\#3723](https://github.com/ClickHouse/ClickHouse/pull/3723)
+- Added for the `Kafka` table engine: checks for subscriptions before beginning to read from Kafka; the kafka\_max\_block\_size setting for the table. [Marek Vavruša](https://github.com/ClickHouse/ClickHouse/pull/3396)
+- The `cityHash64`, `farmHash64`, `metroHash64`, `sipHash64`, `halfMD5`, `murmurHash2_32`, `murmurHash2_64`, `murmurHash3_32`, and `murmurHash3_64` functions now work for any number of arguments and for arguments in the form of tuples. [\#3451](https://github.com/ClickHouse/ClickHouse/pull/3451) [\#3519](https://github.com/ClickHouse/ClickHouse/pull/3519)
+- The `arrayReverse` function now works with any types of arrays. [73e3a7b6](https://github.com/ClickHouse/ClickHouse/commit/73e3a7b662161d6005e7727d8a711b930386b871)
+- Added an optional parameter: the slot size for the `timeSlots` function. [Kirill Shvakov](https://github.com/ClickHouse/ClickHouse/pull/3724)
+- For `FULL` and `RIGHT JOIN`, the `max_block_size` setting is used for a stream of non-joined data from the right table. [Amos Bird](https://github.com/ClickHouse/ClickHouse/pull/3699)
+- Added the `--secure` command line parameter in `clickhouse-benchmark` and `clickhouse-performance-test` to enable TLS. [\#3688](https://github.com/ClickHouse/ClickHouse/pull/3688) [\#3690](https://github.com/ClickHouse/ClickHouse/pull/3690)
+- Type conversion when the structure of a `Buffer` type table does not match the structure of the destination table. [Vitaly Baranov](https://github.com/ClickHouse/ClickHouse/pull/3603)
+- Added the `tcp_keep_alive_timeout` option to enable keep-alive packets after inactivity for the specified time interval. [\#3441](https://github.com/ClickHouse/ClickHouse/pull/3441)
+- Removed unnecessary quoting of values for the partition key in the `system.parts` table if it consists of a single column. [\#3652](https://github.com/ClickHouse/ClickHouse/pull/3652)
+- The modulo function works for `Date` and `DateTime` data types. [\#3385](https://github.com/ClickHouse/ClickHouse/pull/3385)
+- Added synonyms for the `POWER`, `LN`, `LCASE`, `UCASE`, `REPLACE`, `LOCATE`, `SUBSTR`, and `MID` functions. [\#3774](https://github.com/ClickHouse/ClickHouse/pull/3774) [\#3763](https://github.com/ClickHouse/ClickHouse/pull/3763) Some function names are case-insensitive for compatibility with the SQL standard. Added syntactic sugar `SUBSTRING(expr FROM start FOR length)` for compatibility with SQL. [\#3804](https://github.com/ClickHouse/ClickHouse/pull/3804)
+- Added the ability to `mlock` memory pages corresponding to `clickhouse-server` executable code to prevent it from being forced out of memory. This feature is disabled by default. [\#3553](https://github.com/ClickHouse/ClickHouse/pull/3553)
+- Improved performance when reading from `O_DIRECT` (with the `min_bytes_to_use_direct_io` option enabled). [\#3405](https://github.com/ClickHouse/ClickHouse/pull/3405)
+- Improved performance of the `dictGet...OrDefault` function for a constant key argument and a non-constant default argument. [Amos Bird](https://github.com/ClickHouse/ClickHouse/pull/3563)
+- The `firstSignificantSubdomain` function now processes the domains `gov`, `mil`, and `edu`. [Igor Hatarist](https://github.com/ClickHouse/ClickHouse/pull/3601) Improved performance. [\#3628](https://github.com/ClickHouse/ClickHouse/pull/3628)
+- Ability to specify custom environment variables for starting `clickhouse-server` using the `SYS-V init.d` script by defining `CLICKHOUSE_PROGRAM_ENV` in `/etc/default/clickhouse`.
+ [Pavlo Bashynskyi](https://github.com/ClickHouse/ClickHouse/pull/3612)
+- Correct return code for the clickhouse-server init script. [\#3516](https://github.com/ClickHouse/ClickHouse/pull/3516)
+- The `system.metrics` table now has the `VersionInteger` metric, and `system.build_options` has the added line `VERSION_INTEGER`, which contains the numeric form of the ClickHouse version, such as `18016000`. [\#3644](https://github.com/ClickHouse/ClickHouse/pull/3644)
+- Removed the ability to compare the `Date` type with a number to avoid potential errors like `date = 2018-12-17`, where quotes around the date are omitted by mistake. [\#3687](https://github.com/ClickHouse/ClickHouse/pull/3687)
+- Fixed the behavior of stateful functions like `rowNumberInAllBlocks`. They previously output a result that was one number larger due to starting during query analysis. [Amos Bird](https://github.com/ClickHouse/ClickHouse/pull/3729)
+- If the `force_restore_data` file can’t be deleted, an error message is displayed. [Amos Bird](https://github.com/ClickHouse/ClickHouse/pull/3794)
+
+#### Build improvements: {#build-improvements-1}
+
+- Updated the `jemalloc` library, which fixes a potential memory leak. [Amos Bird](https://github.com/ClickHouse/ClickHouse/pull/3557)
+- Profiling with `jemalloc` is enabled by default in order to debug builds. [2cc82f5c](https://github.com/ClickHouse/ClickHouse/commit/2cc82f5cbe266421cd4c1165286c2c47e5ffcb15)
+- Added the ability to run integration tests when only `Docker` is installed on the system. [\#3650](https://github.com/ClickHouse/ClickHouse/pull/3650)
+- Added the fuzz expression test in SELECT queries. [\#3442](https://github.com/ClickHouse/ClickHouse/pull/3442)
+- Added a stress test for commits, which performs functional tests in parallel and in random order to detect more race conditions. [\#3438](https://github.com/ClickHouse/ClickHouse/pull/3438)
+- Improved the method for starting clickhouse-server in a Docker image. [Elghazal Ahmed](https://github.com/ClickHouse/ClickHouse/pull/3663)
+- For a Docker image, added support for initializing databases using files in the `/docker-entrypoint-initdb.d` directory. [Konstantin Lebedev](https://github.com/ClickHouse/ClickHouse/pull/3695)
+- Fixes for builds on ARM. [\#3709](https://github.com/ClickHouse/ClickHouse/pull/3709)
+
+#### Backward incompatible changes: {#backward-incompatible-changes}
+
+- Removed the ability to compare the `Date` type with a number. Instead of `toDate('2018-12-18') = 17883`, you must use explicit type conversion `= toDate(17883)` [\#3687](https://github.com/ClickHouse/ClickHouse/pull/3687)
+
+## ClickHouse release 18.14 {#clickhouse-release-18-14}
+
+### ClickHouse release 18.14.19, 2018-12-19 {#clickhouse-release-18-14-19-2018-12-19}
+
+#### Bug fixes: {#bug-fixes-2}
+
+- Fixed an error that led to problems with updating dictionaries with the ODBC source. [\#3825](https://github.com/ClickHouse/ClickHouse/issues/3825), [\#3829](https://github.com/ClickHouse/ClickHouse/issues/3829)
+- Databases are correctly specified when executing DDL `ON CLUSTER` queries. [\#3460](https://github.com/ClickHouse/ClickHouse/pull/3460)
+- Fixed a segfault if the `max_temporary_non_const_columns` limit was exceeded. [\#3788](https://github.com/ClickHouse/ClickHouse/pull/3788)
+
+#### Build improvements: {#build-improvements-2}
+
+- Fixes for builds on ARM.
+
+### ClickHouse release 18.14.18, 2018-12-04 {#clickhouse-release-18-14-18-2018-12-04}
+
+#### Bug fixes: {#bug-fixes-3}
+
+- Fixed error in `dictGet...` function for dictionaries of type `range`, if one of the arguments is constant and other is not. [\#3751](https://github.com/ClickHouse/ClickHouse/pull/3751)
+- Fixed error that caused messages `netlink: '...': attribute type 1 has an invalid length` to be printed in Linux kernel log, that was happening only on fresh enough versions of Linux kernel. [\#3749](https://github.com/ClickHouse/ClickHouse/pull/3749)
+- Fixed segfault in function `empty` for argument of `FixedString` type. [Daniel, Dao Quang Minh](https://github.com/ClickHouse/ClickHouse/pull/3703)
+- Fixed excessive memory allocation when using large value of `max_query_size` setting (a memory chunk of `max_query_size` bytes was preallocated at once). [\#3720](https://github.com/ClickHouse/ClickHouse/pull/3720)
+
+#### Build changes: {#build-changes}
+
+- Fixed build with LLVM/Clang libraries of version 7 from the OS packages (these libraries are used for runtime query compilation). [\#3582](https://github.com/ClickHouse/ClickHouse/pull/3582)
+
+### ClickHouse release 18.14.17, 2018-11-30 {#clickhouse-release-18-14-17-2018-11-30}
+
+#### Bug fixes: {#bug-fixes-4}
+
+- Fixed cases when the ODBC bridge process did not terminate with the main server process. [\#3642](https://github.com/ClickHouse/ClickHouse/pull/3642)
+- Fixed synchronous insertion into the `Distributed` table with a columns list that differs from the column list of the remote table. [\#3673](https://github.com/ClickHouse/ClickHouse/pull/3673)
+- Fixed a rare race condition that can lead to a crash when dropping a MergeTree table. [\#3643](https://github.com/ClickHouse/ClickHouse/pull/3643)
+- Fixed a query deadlock in case when query thread creation fails with the `Resource temporarily unavailable` error. [\#3643](https://github.com/ClickHouse/ClickHouse/pull/3643)
+- Fixed parsing of the `ENGINE` clause when the `CREATE AS table` syntax was used and the `ENGINE` clause was specified before the `AS table` (the error resulted in ignoring the specified engine). [\#3692](https://github.com/ClickHouse/ClickHouse/pull/3692)
+
+### ClickHouse release 18.14.15, 2018-11-21 {#clickhouse-release-18-14-15-2018-11-21}
+
+#### Bug fixes: {#bug-fixes-5}
+
+- The size of memory chunk was overestimated while deserializing the column of type `Array(String)` that leads to “Memory limit exceeded” errors. The issue appeared in version 18.12.13. [\#3589](https://github.com/ClickHouse/ClickHouse/issues/3589)
+
+### ClickHouse release 18.14.14, 2018-11-20 {#clickhouse-release-18-14-14-2018-11-20}
+
+#### Bug fixes: {#bug-fixes-6}
+
+- Fixed `ON CLUSTER` queries when cluster configured as secure (flag ``). [\#3599](https://github.com/ClickHouse/ClickHouse/pull/3599)
+
+#### Build changes: {#build-changes-1}
+
+- Fixed problems (llvm-7 from system, macos) [\#3582](https://github.com/ClickHouse/ClickHouse/pull/3582)
+
+### ClickHouse release 18.14.13, 2018-11-08 {#clickhouse-release-18-14-13-2018-11-08}
+
+#### Bug fixes: {#bug-fixes-7}
+
+- Fixed the `Block structure mismatch in MergingSorted stream` error. [\#3162](https://github.com/ClickHouse/ClickHouse/issues/3162)
+- Fixed `ON CLUSTER` queries in case when secure connections were turned on in the cluster config (the `` flag). [\#3465](https://github.com/ClickHouse/ClickHouse/pull/3465)
+- Fixed an error in queries that used `SAMPLE`, `PREWHERE` and alias columns. [\#3543](https://github.com/ClickHouse/ClickHouse/pull/3543)
+- Fixed a rare `unknown compression method` error when the `min_bytes_to_use_direct_io` setting was enabled. [3544](https://github.com/ClickHouse/ClickHouse/pull/3544)
+
+#### Performance improvements: {#performance-improvements}
+
+- Fixed performance regression of queries with `GROUP BY` of columns of UInt16 or Date type when executing on AMD EPYC processors. [Igor Lapko](https://github.com/ClickHouse/ClickHouse/pull/3512)
+- Fixed performance regression of queries that process long strings. [\#3530](https://github.com/ClickHouse/ClickHouse/pull/3530)
+
+#### Build improvements: {#build-improvements-3}
+
+- Improvements for simplifying the Arcadia build. [\#3475](https://github.com/ClickHouse/ClickHouse/pull/3475), [\#3535](https://github.com/ClickHouse/ClickHouse/pull/3535)
+
+### ClickHouse release 18.14.12, 2018-11-02 {#clickhouse-release-18-14-12-2018-11-02}
+
+#### Bug fixes: {#bug-fixes-8}
+
+- Fixed a crash on joining two unnamed subqueries. [\#3505](https://github.com/ClickHouse/ClickHouse/pull/3505)
+- Fixed generating incorrect queries (with an empty `WHERE` clause) when querying external databases. [hotid](https://github.com/ClickHouse/ClickHouse/pull/3477)
+- Fixed using an incorrect timeout value in ODBC dictionaries. [Marek Vavruša](https://github.com/ClickHouse/ClickHouse/pull/3511)
+
+### ClickHouse release 18.14.11, 2018-10-29 {#clickhouse-release-18-14-11-2018-10-29}
+
+#### Bug fixes: {#bug-fixes-9}
+
+- Fixed the error `Block structure mismatch in UNION stream: different number of columns` in LIMIT queries. [\#2156](https://github.com/ClickHouse/ClickHouse/issues/2156)
+- Fixed errors when merging data in tables containing arrays inside Nested structures. [\#3397](https://github.com/ClickHouse/ClickHouse/pull/3397)
+- Fixed incorrect query results if the `merge_tree_uniform_read_distribution` setting is disabled (it is enabled by default). [\#3429](https://github.com/ClickHouse/ClickHouse/pull/3429)
+- Fixed an error on inserts to a Distributed table in Native format. [\#3411](https://github.com/ClickHouse/ClickHouse/issues/3411)
+
+### ClickHouse release 18.14.10, 2018-10-23 {#clickhouse-release-18-14-10-2018-10-23}
+
+- The `compile_expressions` setting (JIT compilation of expressions) is disabled by default. [\#3410](https://github.com/ClickHouse/ClickHouse/pull/3410)
+- The `enable_optimize_predicate_expression` setting is disabled by default.
+
+### ClickHouse release 18.14.9, 2018-10-16 {#clickhouse-release-18-14-9-2018-10-16}
+
+#### New features: {#new-features-1}
+
+- The `WITH CUBE` modifier for `GROUP BY` (the alternative syntax `GROUP BY CUBE(...)` is also available). [\#3172](https://github.com/ClickHouse/ClickHouse/pull/3172)
+- Added the `formatDateTime` function. [Alexandr Krasheninnikov](https://github.com/ClickHouse/ClickHouse/pull/2770)
+- Added the `JDBC` table engine and `jdbc` table function (requires installing clickhouse-jdbc-bridge). [Alexandr Krasheninnikov](https://github.com/ClickHouse/ClickHouse/pull/3210)
+- Added functions for working with the ISO week number: `toISOWeek`, `toISOYear`, `toStartOfISOYear`, and `toDayOfYear`. [\#3146](https://github.com/ClickHouse/ClickHouse/pull/3146)
+- Now you can use `Nullable` columns for `MySQL` and `ODBC` tables. [\#3362](https://github.com/ClickHouse/ClickHouse/pull/3362)
+- Nested data structures can be read as nested objects in `JSONEachRow` format. Added the `input_format_import_nested_json` setting. [Veloman Yunkan](https://github.com/ClickHouse/ClickHouse/pull/3144)
+- Parallel processing is available for many `MATERIALIZED VIEW`s when inserting data. See the `parallel_view_processing` setting. [Marek Vavruša](https://github.com/ClickHouse/ClickHouse/pull/3208)
+- Added the `SYSTEM FLUSH LOGS` query (forced log flushes to system tables such as `query_log`) [\#3321](https://github.com/ClickHouse/ClickHouse/pull/3321)
+- Now you can use pre-defined `database` and `table` macros when declaring `Replicated` tables. [\#3251](https://github.com/ClickHouse/ClickHouse/pull/3251)
+- Added the ability to read `Decimal` type values in engineering notation (indicating powers of ten). [\#3153](https://github.com/ClickHouse/ClickHouse/pull/3153)
+
+#### Experimental features: {#experimental-features}
+
+- Optimization of the GROUP BY clause for `LowCardinality data types.` [\#3138](https://github.com/ClickHouse/ClickHouse/pull/3138)
+- Optimized calculation of expressions for `LowCardinality data types.` [\#3200](https://github.com/ClickHouse/ClickHouse/pull/3200)
+
+#### Improvements: {#improvements-2}
+
+- Significantly reduced memory consumption for queries with `ORDER BY` and `LIMIT`. See the `max_bytes_before_remerge_sort` setting. [\#3205](https://github.com/ClickHouse/ClickHouse/pull/3205)
+- In the absence of `JOIN` (`LEFT`, `INNER`, …), `INNER JOIN` is assumed. [\#3147](https://github.com/ClickHouse/ClickHouse/pull/3147)
+- Qualified asterisks work correctly in queries with `JOIN`. [Winter Zhang](https://github.com/ClickHouse/ClickHouse/pull/3202)
+- The `ODBC` table engine correctly chooses the method for quoting identifiers in the SQL dialect of a remote database. [Alexandr Krasheninnikov](https://github.com/ClickHouse/ClickHouse/pull/3210)
+- The `compile_expressions` setting (JIT compilation of expressions) is enabled by default.
+- Fixed behavior for simultaneous DROP DATABASE/TABLE IF EXISTS and CREATE DATABASE/TABLE IF NOT EXISTS. Previously, a `CREATE DATABASE ... IF NOT EXISTS` query could return the error message “File … already exists”, and the `CREATE TABLE ... IF NOT EXISTS` and `DROP TABLE IF EXISTS` queries could return `Table ... is creating or attaching right now`. [\#3101](https://github.com/ClickHouse/ClickHouse/pull/3101)
+- LIKE and IN expressions with a constant right half are passed to the remote server when querying from MySQL or ODBC tables. [\#3182](https://github.com/ClickHouse/ClickHouse/pull/3182)
+- Comparisons with constant expressions in a WHERE clause are passed to the remote server when querying from MySQL and ODBC tables. Previously, only comparisons with constants were passed. [\#3182](https://github.com/ClickHouse/ClickHouse/pull/3182)
+- Correct calculation of row width in the terminal for `Pretty` formats, including strings with hieroglyphs. [Amos Bird](https://github.com/ClickHouse/ClickHouse/pull/3257).
+- `ON CLUSTER` can be specified for `ALTER UPDATE` queries.
+- Improved performance for reading data in `JSONEachRow` format. [\#3332](https://github.com/ClickHouse/ClickHouse/pull/3332)
+- Added synonyms for the `LENGTH` and `CHARACTER_LENGTH` functions for compatibility. The `CONCAT` function is no longer case-sensitive. [\#3306](https://github.com/ClickHouse/ClickHouse/pull/3306)
+- Added the `TIMESTAMP` synonym for the `DateTime` type. [\#3390](https://github.com/ClickHouse/ClickHouse/pull/3390)
+- There is always space reserved for query\_id in the server logs, even if the log line is not related to a query. This makes it easier to parse server text logs with third-party tools.
+- Memory consumption by a query is logged when it exceeds the next level of an integer number of gigabytes. [\#3205](https://github.com/ClickHouse/ClickHouse/pull/3205)
+- Added compatibility mode for the case when the client library that uses the Native protocol sends fewer columns by mistake than the server expects for the INSERT query. This scenario was possible when using the clickhouse-cpp library. Previously, this scenario caused the server to crash. [\#3171](https://github.com/ClickHouse/ClickHouse/pull/3171)
+- In a user-defined WHERE expression in `clickhouse-copier`, you can now use a `partition_key` alias (for additional filtering by source table partition). This is useful if the partitioning scheme changes during copying, but only changes slightly. [\#3166](https://github.com/ClickHouse/ClickHouse/pull/3166)
+- The workflow of the `Kafka` engine has been moved to a background thread pool in order to automatically reduce the speed of data reading at high loads. [Marek Vavruša](https://github.com/ClickHouse/ClickHouse/pull/3215).
+- Support for reading `Tuple` and `Nested` values of structures like `struct` in the `Cap'n'Proto format`. [Marek Vavruša](https://github.com/ClickHouse/ClickHouse/pull/3216)
+- The list of top-level domains for the `firstSignificantSubdomain` function now includes the domain `biz`. [decaseal](https://github.com/ClickHouse/ClickHouse/pull/3219)
+- In the configuration of external dictionaries, `null_value` is interpreted as the value of the default data type. [\#3330](https://github.com/ClickHouse/ClickHouse/pull/3330)
+- Support for the `intDiv` and `intDivOrZero` functions for `Decimal`. [b48402e8](https://github.com/ClickHouse/ClickHouse/commit/b48402e8712e2b9b151e0eef8193811d433a1264)
+- Support for the `Date`, `DateTime`, `UUID`, and `Decimal` types as a key for the `sumMap` aggregate function. [\#3281](https://github.com/ClickHouse/ClickHouse/pull/3281)
+- Support for the `Decimal` data type in external dictionaries. [\#3324](https://github.com/ClickHouse/ClickHouse/pull/3324)
+- Support for the `Decimal` data type in `SummingMergeTree` tables. [\#3348](https://github.com/ClickHouse/ClickHouse/pull/3348)
+- Added specializations for `UUID` in `if`. [\#3366](https://github.com/ClickHouse/ClickHouse/pull/3366)
+- Reduced the number of `open` and `close` system calls when reading from a `MergeTree table`. [\#3283](https://github.com/ClickHouse/ClickHouse/pull/3283)
+- A `TRUNCATE TABLE` query can be executed on any replica (the query is passed to the leader replica). [Kirill Shvakov](https://github.com/ClickHouse/ClickHouse/pull/3375)
+
+#### Bug fixes: {#bug-fixes-10}
+
+- Fixed an issue with `Dictionary` tables for `range_hashed` dictionaries. This error occurred in version 18.12.17. [\#1702](https://github.com/ClickHouse/ClickHouse/pull/1702)
+- Fixed an error when loading `range_hashed` dictionaries (the message `Unsupported type Nullable (...)`). This error occurred in version 18.12.17. [\#3362](https://github.com/ClickHouse/ClickHouse/pull/3362)
+- Fixed errors in the `pointInPolygon` function due to the accumulation of inaccurate calculations for polygons with a large number of vertices located close to each other. [\#3331](https://github.com/ClickHouse/ClickHouse/pull/3331) [\#3341](https://github.com/ClickHouse/ClickHouse/pull/3341)
+- If after merging data parts, the checksum for the resulting part differs from the result of the same merge in another replica, the result of the merge is deleted and the data part is downloaded from the other replica (this is the correct behavior). But after downloading the data part, it couldn’t be added to the working set because of an error that the part already exists (because the data part was deleted with some delay after the merge). This led to cyclical attempts to download the same data. [\#3194](https://github.com/ClickHouse/ClickHouse/pull/3194)
+- Fixed incorrect calculation of total memory consumption by queries (because of incorrect calculation, the `max_memory_usage_for_all_queries` setting worked incorrectly and the `MemoryTracking` metric had an incorrect value). This error occurred in version 18.12.13. [Marek Vavruša](https://github.com/ClickHouse/ClickHouse/pull/3344)
+- Fixed the functionality of `CREATE TABLE ... ON CLUSTER ... AS SELECT ...` This error occurred in version 18.12.13. [\#3247](https://github.com/ClickHouse/ClickHouse/pull/3247)
+- Fixed unnecessary preparation of data structures for `JOIN`s on the server that initiates the query if the `JOIN` is only performed on remote servers. [\#3340](https://github.com/ClickHouse/ClickHouse/pull/3340)
+- Fixed bugs in the `Kafka` engine: deadlocks after exceptions when starting to read data, and locks upon completion [Marek Vavruša](https://github.com/ClickHouse/ClickHouse/pull/3215).
+- For `Kafka` tables, the optional `schema` parameter was not passed (the schema of the `Cap'n'Proto` format). [Vojtech Splichal](https://github.com/ClickHouse/ClickHouse/pull/3150)
+- If the ensemble of ZooKeeper servers has servers that accept the connection but then immediately close it instead of responding to the handshake, ClickHouse chooses to connect another server. Previously, this produced the error `Cannot read all data. Bytes read: 0. Bytes expected: 4.` and the server couldn’t start. [8218cf3a](https://github.com/ClickHouse/ClickHouse/commit/8218cf3a5f39a43401953769d6d12a0bb8d29da9)
+- If the ensemble of ZooKeeper servers contains servers for which the DNS query returns an error, these servers are ignored. [17b8e209](https://github.com/ClickHouse/ClickHouse/commit/17b8e209221061325ad7ba0539f03c6e65f87f29)
+- Fixed type conversion between `Date` and `DateTime` when inserting data in the `VALUES` format (if `input_format_values_interpret_expressions = 1`). Previously, the conversion was performed between the numerical value of the number of days in Unix Epoch time and the Unix timestamp, which led to unexpected results. [\#3229](https://github.com/ClickHouse/ClickHouse/pull/3229)
+- Corrected type conversion between `Decimal` and integer numbers. [\#3211](https://github.com/ClickHouse/ClickHouse/pull/3211)
+- Fixed errors in the `enable_optimize_predicate_expression` setting. [Winter Zhang](https://github.com/ClickHouse/ClickHouse/pull/3231)
+- Fixed a parsing error in CSV format with floating-point numbers if a non-default CSV separator is used, such as `;` [\#3155](https://github.com/ClickHouse/ClickHouse/pull/3155)
+- Fixed the `arrayCumSumNonNegative` function (it does not accumulate negative values if the accumulator is less than zero). [Aleksey Studnev](https://github.com/ClickHouse/ClickHouse/pull/3163)
+- Fixed how `Merge` tables work on top of `Distributed` tables when using `PREWHERE`. [\#3165](https://github.com/ClickHouse/ClickHouse/pull/3165)
+- Bug fixes in the `ALTER UPDATE` query.
+- Fixed bugs in the `odbc` table function that appeared in version 18.12. [\#3197](https://github.com/ClickHouse/ClickHouse/pull/3197)
+- Fixed the operation of aggregate functions with `StateArray` combinators. [\#3188](https://github.com/ClickHouse/ClickHouse/pull/3188)
+- Fixed a crash when dividing a `Decimal` value by zero. [69dd6609](https://github.com/ClickHouse/ClickHouse/commit/69dd6609193beb4e7acd3e6ad216eca0ccfb8179)
+- Fixed output of types for operations using `Decimal` and integer arguments. [\#3224](https://github.com/ClickHouse/ClickHouse/pull/3224)
+- Fixed the segfault during `GROUP BY` on `Decimal128`. [3359ba06](https://github.com/ClickHouse/ClickHouse/commit/3359ba06c39fcd05bfdb87d6c64154819621e13a)
+- The `log_query_threads` setting (logging information about each thread of query execution) now takes effect only if the `log_queries` option (logging information about queries) is set to 1. Since the `log_query_threads` option is enabled by default, information about threads was previously logged even if query logging was disabled. [\#3241](https://github.com/ClickHouse/ClickHouse/pull/3241)
+- Fixed an error in the distributed operation of the quantiles aggregate function (the error message `Not found column quantile...`). [292a8855](https://github.com/ClickHouse/ClickHouse/commit/292a885533b8e3b41ce8993867069d14cbd5a664)
+- Fixed the compatibility problem when working on a cluster of version 18.12.17 servers and older servers at the same time. For distributed queries with GROUP BY keys of both fixed and non-fixed length, if there was a large amount of data to aggregate, the returned data was not always fully aggregated (two different rows contained the same aggregation keys). [\#3254](https://github.com/ClickHouse/ClickHouse/pull/3254)
+- Fixed handling of substitutions in `clickhouse-performance-test`, if the query contains only part of the substitutions declared in the test. [\#3263](https://github.com/ClickHouse/ClickHouse/pull/3263)
+- Fixed an error when using `FINAL` with `PREWHERE`. [\#3298](https://github.com/ClickHouse/ClickHouse/pull/3298)
+- Fixed an error when using `PREWHERE` over columns that were added during `ALTER`. [\#3298](https://github.com/ClickHouse/ClickHouse/pull/3298)
+- Added a check for the absence of `arrayJoin` for `DEFAULT` and `MATERIALIZED` expressions. Previously, `arrayJoin` led to an error when inserting data. [\#3337](https://github.com/ClickHouse/ClickHouse/pull/3337)
+- Added a check for the absence of `arrayJoin` in a `PREWHERE` clause. Previously, this led to messages like `Size ... doesn't match` or `Unknown compression method` when executing queries. [\#3357](https://github.com/ClickHouse/ClickHouse/pull/3357)
+- Fixed segfault that could occur in rare cases after optimization that replaced AND chains from equality evaluations with the corresponding IN expression. [liuyimin-bytedance](https://github.com/ClickHouse/ClickHouse/pull/3339)
+- Minor corrections to `clickhouse-benchmark`: previously, client information was not sent to the server; now the number of queries executed is calculated more accurately when shutting down and for limiting the number of iterations. [\#3351](https://github.com/ClickHouse/ClickHouse/pull/3351) [\#3352](https://github.com/ClickHouse/ClickHouse/pull/3352)
+
+#### Backward incompatible changes: {#backward-incompatible-changes-1}
+
+- Removed the `allow_experimental_decimal_type` option. The `Decimal` data type is available for default use. [\#3329](https://github.com/ClickHouse/ClickHouse/pull/3329)
+
+## ClickHouse release 18.12 {#clickhouse-release-18-12}
+
+### ClickHouse release 18.12.17, 2018-09-16 {#clickhouse-release-18-12-17-2018-09-16}
+
+#### New features: {#new-features-2}
+
+- `invalidate_query` (the ability to specify a query to check whether an external dictionary needs to be updated) is implemented for the `clickhouse` source. [\#3126](https://github.com/ClickHouse/ClickHouse/pull/3126)
+- Added the ability to use `UInt*`, `Int*`, and `DateTime` data types (along with the `Date` type) as a `range_hashed` external dictionary key that defines the boundaries of ranges. Now `NULL` can be used to designate an open range. [Vasily Nemkov](https://github.com/ClickHouse/ClickHouse/pull/3123)
+- The `Decimal` type now supports `var*` and `stddev*` aggregate functions. [\#3129](https://github.com/ClickHouse/ClickHouse/pull/3129)
+- The `Decimal` type now supports mathematical functions (`exp`, `sin` and so on.) [\#3129](https://github.com/ClickHouse/ClickHouse/pull/3129)
+- The `system.part_log` table now has the `partition_id` column. [\#3089](https://github.com/ClickHouse/ClickHouse/pull/3089)
+
+#### Bug fixes: {#bug-fixes-11}
+
+- `Merge` now works correctly on `Distributed` tables. [Winter Zhang](https://github.com/ClickHouse/ClickHouse/pull/3159)
+- Fixed incompatibility (unnecessary dependency on the `glibc` version) that made it impossible to run ClickHouse on `Ubuntu Precise` and older versions. The incompatibility arose in version 18.12.13. [\#3130](https://github.com/ClickHouse/ClickHouse/pull/3130)
+- Fixed errors in the `enable_optimize_predicate_expression` setting. [Winter Zhang](https://github.com/ClickHouse/ClickHouse/pull/3107)
+- Fixed a minor issue with backwards compatibility that appeared when working with a cluster of replicas on versions earlier than 18.12.13 and simultaneously creating a new replica of a table on a server with a newer version (shown in the message `Can not clone replica, because the ... updated to new ClickHouse version`, which is logical, but shouldn’t happen). [\#3122](https://github.com/ClickHouse/ClickHouse/pull/3122)
+
+#### Backward incompatible changes: {#backward-incompatible-changes-2}
+
+- The `enable_optimize_predicate_expression` option is enabled by default (which is rather optimistic). If query analysis errors occur that are related to searching for the column names, set `enable_optimize_predicate_expression` to 0. [Winter Zhang](https://github.com/ClickHouse/ClickHouse/pull/3107)
+
+### ClickHouse release 18.12.14, 2018-09-13 {#clickhouse-release-18-12-14-2018-09-13}
+
+#### New features: {#new-features-3}
+
+- Added support for `ALTER UPDATE` queries. [\#3035](https://github.com/ClickHouse/ClickHouse/pull/3035)
+- Added the `allow_ddl` option, which restricts the user’s access to DDL queries. [\#3104](https://github.com/ClickHouse/ClickHouse/pull/3104)
+- Added the `min_merge_bytes_to_use_direct_io` option for `MergeTree` engines, which allows you to set a threshold for the total size of the merge (when above the threshold, data part files will be handled using O\_DIRECT). [\#3117](https://github.com/ClickHouse/ClickHouse/pull/3117)
+- The `system.merges` system table now contains the `partition_id` column. [\#3099](https://github.com/ClickHouse/ClickHouse/pull/3099)
+
+#### Improvements {#improvements-3}
+
+- If a data part remains unchanged during mutation, it isn’t downloaded by replicas. [\#3103](https://github.com/ClickHouse/ClickHouse/pull/3103)
+- Autocomplete is available for names of settings when working with `clickhouse-client`. [\#3106](https://github.com/ClickHouse/ClickHouse/pull/3106)
+
+#### Bug fixes: {#bug-fixes-12}
+
+- Added a check for the sizes of arrays that are elements of `Nested` type fields when inserting. [\#3118](https://github.com/ClickHouse/ClickHouse/pull/3118)
+- Fixed an error updating external dictionaries with the `ODBC` source and `hashed` storage. This error occurred in version 18.12.13.
+- Fixed a crash when creating a temporary table from a query with an `IN` condition. [Winter Zhang](https://github.com/ClickHouse/ClickHouse/pull/3098)
+- Fixed an error in aggregate functions for arrays that can have `NULL` elements. [Winter Zhang](https://github.com/ClickHouse/ClickHouse/pull/3097)
+
+### ClickHouse release 18.12.13, 2018-09-10 {#clickhouse-release-18-12-13-2018-09-10}
+
+#### New features: {#new-features-4}
+
+- Added the `DECIMAL(digits, scale)` data type (`Decimal32(scale)`, `Decimal64(scale)`, `Decimal128(scale)`). To enable it, use the setting `allow_experimental_decimal_type`. [\#2846](https://github.com/ClickHouse/ClickHouse/pull/2846) [\#2970](https://github.com/ClickHouse/ClickHouse/pull/2970) [\#3008](https://github.com/ClickHouse/ClickHouse/pull/3008) [\#3047](https://github.com/ClickHouse/ClickHouse/pull/3047)
+- New `WITH ROLLUP` modifier for `GROUP BY` (alternative syntax: `GROUP BY ROLLUP(...)`). [\#2948](https://github.com/ClickHouse/ClickHouse/pull/2948)
+- In queries with JOIN, the star character expands to a list of columns in all tables, in compliance with the SQL standard. You can restore the old behavior by setting `asterisk_left_columns_only` to 1 on the user configuration level. [Winter Zhang](https://github.com/ClickHouse/ClickHouse/pull/2787)
+- Added support for JOIN with table functions. [Winter Zhang](https://github.com/ClickHouse/ClickHouse/pull/2907)
+- Autocomplete by pressing Tab in clickhouse-client. [Sergey Shcherbin](https://github.com/ClickHouse/ClickHouse/pull/2447)
+- Ctrl+C in clickhouse-client clears a query that was entered. [\#2877](https://github.com/ClickHouse/ClickHouse/pull/2877)
+- Added the `join_default_strictness` setting (values: `"`, `'any'`, `'all'`). This allows you to not specify `ANY` or `ALL` for `JOIN`. [\#2982](https://github.com/ClickHouse/ClickHouse/pull/2982)
+- Each line of the server log related to query processing shows the query ID. [\#2482](https://github.com/ClickHouse/ClickHouse/pull/2482)
+- Now you can get query execution logs in clickhouse-client (use the `send_logs_level` setting). With distributed query processing, logs are cascaded from all the servers. [\#2482](https://github.com/ClickHouse/ClickHouse/pull/2482)
+- The `system.query_log` and `system.processes` (`SHOW PROCESSLIST`) tables now have information about all changed settings when you run a query (the nested structure of the `Settings` data). Added the `log_query_settings` setting. [\#2482](https://github.com/ClickHouse/ClickHouse/pull/2482)
+- The `system.query_log` and `system.processes` tables now show information about the number of threads that are participating in query execution (see the `thread_numbers` column). [\#2482](https://github.com/ClickHouse/ClickHouse/pull/2482)
+- Added `ProfileEvents` counters that measure the time spent on reading and writing over the network and reading and writing to disk, the number of network errors, and the time spent waiting when network bandwidth is limited. [\#2482](https://github.com/ClickHouse/ClickHouse/pull/2482)
+- Added `ProfileEvents`counters that contain the system metrics from rusage (you can use them to get information about CPU usage in userspace and the kernel, page faults, and context switches), as well as taskstats metrics (use these to obtain information about I/O wait time, CPU wait time, and the amount of data read and recorded, both with and without page cache). [\#2482](https://github.com/ClickHouse/ClickHouse/pull/2482)
+- The `ProfileEvents` counters are applied globally and for each query, as well as for each query execution thread, which allows you to profile resource consumption by query in detail. [\#2482](https://github.com/ClickHouse/ClickHouse/pull/2482)
+- Added the `system.query_thread_log` table, which contains information about each query execution thread. Added the `log_query_threads` setting. [\#2482](https://github.com/ClickHouse/ClickHouse/pull/2482)
+- The `system.metrics` and `system.events` tables now have built-in documentation. [\#3016](https://github.com/ClickHouse/ClickHouse/pull/3016)
+- Added the `arrayEnumerateDense` function. [Amos Bird](https://github.com/ClickHouse/ClickHouse/pull/2975)
+- Added the `arrayCumSumNonNegative` and `arrayDifference` functions. [Aleksey Studnev](https://github.com/ClickHouse/ClickHouse/pull/2942)
+- Added the `retention` aggregate function. [Sundy Li](https://github.com/ClickHouse/ClickHouse/pull/2887)
+- Now you can add (merge) states of aggregate functions by using the plus operator, and multiply the states of aggregate functions by a nonnegative constant. [\#3062](https://github.com/ClickHouse/ClickHouse/pull/3062) [\#3034](https://github.com/ClickHouse/ClickHouse/pull/3034)
+- Tables in the MergeTree family now have the virtual column `_partition_id`. [\#3089](https://github.com/ClickHouse/ClickHouse/pull/3089)
+
+#### Experimental features: {#experimental-features-1}
+
+- Added the `LowCardinality(T)` data type. This data type automatically creates a local dictionary of values and allows data processing without unpacking the dictionary. [\#2830](https://github.com/ClickHouse/ClickHouse/pull/2830)
+- Added a cache of JIT-compiled functions and a counter for the number of uses before compiling. To JIT compile expressions, enable the `compile_expressions` setting. [\#2990](https://github.com/ClickHouse/ClickHouse/pull/2990) [\#3077](https://github.com/ClickHouse/ClickHouse/pull/3077)
+
+#### Improvements: {#improvements-4}
+
+- Fixed the problem with unlimited accumulation of the replication log when there are abandoned replicas. Added an effective recovery mode for replicas with a long lag.
+- Improved performance of `GROUP BY` with multiple aggregation fields when one of them is string and the others are fixed length.
+- Improved performance when using `PREWHERE` and with implicit transfer of expressions in `PREWHERE`.
+- Improved parsing performance for text formats (`CSV`, `TSV`). [Amos Bird](https://github.com/ClickHouse/ClickHouse/pull/2977) [\#2980](https://github.com/ClickHouse/ClickHouse/pull/2980)
+- Improved performance of reading strings and arrays in binary formats. [Amos Bird](https://github.com/ClickHouse/ClickHouse/pull/2955)
+- Increased performance and reduced memory consumption for queries to `system.tables` and `system.columns` when there is a very large number of tables on a single server. [\#2953](https://github.com/ClickHouse/ClickHouse/pull/2953)
+- Fixed a performance problem in the case of a large stream of queries that result in an error (the `_dl_addr` function is visible in `perf top`, but the server isn’t using much CPU). [\#2938](https://github.com/ClickHouse/ClickHouse/pull/2938)
+- Conditions are cast into the View (when `enable_optimize_predicate_expression` is enabled). [Winter Zhang](https://github.com/ClickHouse/ClickHouse/pull/2907)
+- Improvements to the functionality for the `UUID` data type. [\#3074](https://github.com/ClickHouse/ClickHouse/pull/3074) [\#2985](https://github.com/ClickHouse/ClickHouse/pull/2985)
+- The `UUID` data type is supported in The-Alchemist dictionaries. [\#2822](https://github.com/ClickHouse/ClickHouse/pull/2822)
+- The `visitParamExtractRaw` function works correctly with nested structures. [Winter Zhang](https://github.com/ClickHouse/ClickHouse/pull/2974)
+- When the `input_format_skip_unknown_fields` setting is enabled, object fields in `JSONEachRow` format are skipped correctly. [BlahGeek](https://github.com/ClickHouse/ClickHouse/pull/2958)
+- For a `CASE` expression with conditions, you can now omit `ELSE`, which is equivalent to `ELSE NULL`. [\#2920](https://github.com/ClickHouse/ClickHouse/pull/2920)
+- The operation timeout can now be configured when working with ZooKeeper. [urykhy](https://github.com/ClickHouse/ClickHouse/pull/2971)
+- You can specify an offset for `LIMIT n, m` as `LIMIT n OFFSET m`. [\#2840](https://github.com/ClickHouse/ClickHouse/pull/2840)
+- You can use the `SELECT TOP n` syntax as an alternative for `LIMIT`. [\#2840](https://github.com/ClickHouse/ClickHouse/pull/2840)
+- Increased the size of the queue to write to system tables, so the `SystemLog parameter queue is full` error doesn’t happen as often.
+- The `windowFunnel` aggregate function now supports events that meet multiple conditions. [Amos Bird](https://github.com/ClickHouse/ClickHouse/pull/2801)
+- Duplicate columns can be used in a `USING` clause for `JOIN`. [\#3006](https://github.com/ClickHouse/ClickHouse/pull/3006)
+- `Pretty` formats now have a limit on column alignment by width. Use the `output_format_pretty_max_column_pad_width` setting. If a value is wider, it will still be displayed in its entirety, but the other cells in the table will not be too wide. [\#3003](https://github.com/ClickHouse/ClickHouse/pull/3003)
+- The `odbc` table function now allows you to specify the database/schema name. [Amos Bird](https://github.com/ClickHouse/ClickHouse/pull/2885)
+- Added the ability to use a username specified in the `clickhouse-client` config file. [Vladimir Kozbin](https://github.com/ClickHouse/ClickHouse/pull/2909)
+- The `ZooKeeperExceptions` counter has been split into three counters: `ZooKeeperUserExceptions`, `ZooKeeperHardwareExceptions`, and `ZooKeeperOtherExceptions`.
+- `ALTER DELETE` queries work for materialized views.
+- Added randomization when running the cleanup thread periodically for `ReplicatedMergeTree` tables in order to avoid periodic load spikes when there are a very large number of `ReplicatedMergeTree` tables.
+- Support for `ATTACH TABLE ... ON CLUSTER` queries. [\#3025](https://github.com/ClickHouse/ClickHouse/pull/3025)
+
+#### Bug fixes: {#bug-fixes-13}
+
+- Fixed an issue with `Dictionary` tables (throws the `Size of offsets doesn't match size of column` or `Unknown compression method` exception). This bug appeared in version 18.10.3. [\#2913](https://github.com/ClickHouse/ClickHouse/issues/2913)
+- Fixed a bug when merging `CollapsingMergeTree` tables if one of the data parts is empty (these parts are formed during merge or `ALTER DELETE` if all data was deleted), and the `vertical` algorithm was used for the merge. [\#3049](https://github.com/ClickHouse/ClickHouse/pull/3049)
+- Fixed a race condition during `DROP` or `TRUNCATE` for `Memory` tables with a simultaneous `SELECT`, which could lead to server crashes. This bug appeared in version 1.1.54388. [\#3038](https://github.com/ClickHouse/ClickHouse/pull/3038)
+- Fixed the possibility of data loss when inserting in `Replicated` tables if the `Session is expired` error is returned (data loss can be detected by the `ReplicatedDataLoss` metric). This error occurred in version 1.1.54378. [\#2939](https://github.com/ClickHouse/ClickHouse/pull/2939) [\#2949](https://github.com/ClickHouse/ClickHouse/pull/2949) [\#2964](https://github.com/ClickHouse/ClickHouse/pull/2964)
+- Fixed a segfault during `JOIN ... ON`. [\#3000](https://github.com/ClickHouse/ClickHouse/pull/3000)
+- Fixed the error searching column names when the `WHERE` expression consists entirely of a qualified column name, such as `WHERE table.column`. [\#2994](https://github.com/ClickHouse/ClickHouse/pull/2994)
+- Fixed the “Not found column” error that occurred when executing distributed queries if a single column consisting of an IN expression with a subquery is requested from a remote server. [\#3087](https://github.com/ClickHouse/ClickHouse/pull/3087)
+- Fixed the `Block structure mismatch in UNION stream: different number of columns` error that occurred for distributed queries if one of the shards is local and the other is not, and optimization of the move to `PREWHERE` is triggered. [\#2226](https://github.com/ClickHouse/ClickHouse/pull/2226) [\#3037](https://github.com/ClickHouse/ClickHouse/pull/3037) [\#3055](https://github.com/ClickHouse/ClickHouse/pull/3055) [\#3065](https://github.com/ClickHouse/ClickHouse/pull/3065) [\#3073](https://github.com/ClickHouse/ClickHouse/pull/3073) [\#3090](https://github.com/ClickHouse/ClickHouse/pull/3090) [\#3093](https://github.com/ClickHouse/ClickHouse/pull/3093)
+- Fixed the `pointInPolygon` function for certain cases of non-convex polygons. [\#2910](https://github.com/ClickHouse/ClickHouse/pull/2910)
+- Fixed the incorrect result when comparing `nan` with integers. [\#3024](https://github.com/ClickHouse/ClickHouse/pull/3024)
+- Fixed an error in the `zlib-ng` library that could lead to segfault in rare cases. [\#2854](https://github.com/ClickHouse/ClickHouse/pull/2854)
+- Fixed a memory leak when inserting into a table with `AggregateFunction` columns, if the state of the aggregate function is not simple (allocates memory separately), and if a single insertion request results in multiple small blocks. [\#3084](https://github.com/ClickHouse/ClickHouse/pull/3084)
+- Fixed a race condition when creating and deleting the same `Buffer` or `MergeTree` table simultaneously.
+- Fixed the possibility of a segfault when comparing tuples made up of certain non-trivial types, such as tuples. [\#2989](https://github.com/ClickHouse/ClickHouse/pull/2989)
+- Fixed the possibility of a segfault when running certain `ON CLUSTER` queries. [Winter Zhang](https://github.com/ClickHouse/ClickHouse/pull/2960)
+- Fixed an error in the `arrayDistinct` function for `Nullable` array elements. [\#2845](https://github.com/ClickHouse/ClickHouse/pull/2845) [\#2937](https://github.com/ClickHouse/ClickHouse/pull/2937)
+- The `enable_optimize_predicate_expression` option now correctly supports cases with `SELECT *`. [Winter Zhang](https://github.com/ClickHouse/ClickHouse/pull/2929)
+- Fixed the segfault when re-initializing the ZooKeeper session. [\#2917](https://github.com/ClickHouse/ClickHouse/pull/2917)
+- Fixed potential blocking when working with ZooKeeper.
+- Fixed incorrect code for adding nested data structures in a `SummingMergeTree`.
+- When allocating memory for states of aggregate functions, alignment is correctly taken into account, which makes it possible to use operations that require alignment when implementing states of aggregate functions. [chenxing-xc](https://github.com/ClickHouse/ClickHouse/pull/2808)
+
+#### Security fix: {#security-fix}
+
+- Safe use of ODBC data sources. Interaction with ODBC drivers uses a separate `clickhouse-odbc-bridge` process. Errors in third-party ODBC drivers no longer cause problems with server stability or vulnerabilities. [\#2828](https://github.com/ClickHouse/ClickHouse/pull/2828) [\#2879](https://github.com/ClickHouse/ClickHouse/pull/2879) [\#2886](https://github.com/ClickHouse/ClickHouse/pull/2886) [\#2893](https://github.com/ClickHouse/ClickHouse/pull/2893) [\#2921](https://github.com/ClickHouse/ClickHouse/pull/2921)
+- Fixed incorrect validation of the file path in the `catBoostPool` table function. [\#2894](https://github.com/ClickHouse/ClickHouse/pull/2894)
+- The contents of system tables (`tables`, `databases`, `parts`, `columns`, `parts_columns`, `merges`, `mutations`, `replicas`, and `replication_queue`) are filtered according to the user’s configured access to databases (`allow_databases`). [Winter Zhang](https://github.com/ClickHouse/ClickHouse/pull/2856)
+
+#### Backward incompatible changes: {#backward-incompatible-changes-3}
+
+- In queries with JOIN, the star character expands to a list of columns in all tables, in compliance with the SQL standard. You can restore the old behavior by setting `asterisk_left_columns_only` to 1 on the user configuration level.
+
+#### Build changes: {#build-changes-2}
+
+- Most integration tests can now be run by commit.
+- Code style checks can also be run by commit.
+- The `memcpy` implementation is chosen correctly when building on CentOS7/Fedora. [Etienne Champetier](https://github.com/ClickHouse/ClickHouse/pull/2912)
+- When using clang to build, some warnings from `-Weverything` have been added, in addition to the regular `-Wall-Wextra -Werror`. [\#2957](https://github.com/ClickHouse/ClickHouse/pull/2957)
+- Debugging the build uses the `jemalloc` debug option.
+- The interface of the library for interacting with ZooKeeper is declared abstract. [\#2950](https://github.com/ClickHouse/ClickHouse/pull/2950)
+
+## ClickHouse release 18.10 {#clickhouse-release-18-10}
+
+### ClickHouse release 18.10.3, 2018-08-13 {#clickhouse-release-18-10-3-2018-08-13}
+
+#### New features: {#new-features-5}
+
+- HTTPS can be used for replication. [\#2760](https://github.com/ClickHouse/ClickHouse/pull/2760)
+- Added the functions `murmurHash2_64`, `murmurHash3_32`, `murmurHash3_64`, and `murmurHash3_128` in addition to the existing `murmurHash2_32`. [\#2791](https://github.com/ClickHouse/ClickHouse/pull/2791)
+- Support for Nullable types in the ClickHouse ODBC driver (`ODBCDriver2` output format). [\#2834](https://github.com/ClickHouse/ClickHouse/pull/2834)
+- Support for `UUID` in the key columns.
+
+#### Improvements: {#improvements-5}
+
+- Clusters can be removed without restarting the server when they are deleted from the config files. [\#2777](https://github.com/ClickHouse/ClickHouse/pull/2777)
+- External dictionaries can be removed without restarting the server when they are removed from config files. [\#2779](https://github.com/ClickHouse/ClickHouse/pull/2779)
+- Added `SETTINGS` support for the `Kafka` table engine. [Alexander Marshalov](https://github.com/ClickHouse/ClickHouse/pull/2781)
+- Improvements for the `UUID` data type (not yet complete). [\#2618](https://github.com/ClickHouse/ClickHouse/pull/2618)
+- Support for empty parts after merges in the `SummingMergeTree`, `CollapsingMergeTree` and `VersionedCollapsingMergeTree` engines. [\#2815](https://github.com/ClickHouse/ClickHouse/pull/2815)
+- Old records of completed mutations are deleted (`ALTER DELETE`). [\#2784](https://github.com/ClickHouse/ClickHouse/pull/2784)
+- Added the `system.merge_tree_settings` table. [Kirill Shvakov](https://github.com/ClickHouse/ClickHouse/pull/2841)
+- The `system.tables` table now has dependency columns: `dependencies_database` and `dependencies_table`. [Winter Zhang](https://github.com/ClickHouse/ClickHouse/pull/2851)
+- Added the `max_partition_size_to_drop` config option. [\#2782](https://github.com/ClickHouse/ClickHouse/pull/2782)
+- Added the `output_format_json_escape_forward_slashes` option. [Alexander Bocharov](https://github.com/ClickHouse/ClickHouse/pull/2812)
+- Added the `max_fetch_partition_retries_count` setting. [\#2831](https://github.com/ClickHouse/ClickHouse/pull/2831)
+- Added the `prefer_localhost_replica` setting for disabling the preference for a local replica and going to a local replica without inter-process interaction. [\#2832](https://github.com/ClickHouse/ClickHouse/pull/2832)
+- The `quantileExact` aggregate function returns `nan` in the case of aggregation on an empty `Float32` or `Float64` set. [Sundy Li](https://github.com/ClickHouse/ClickHouse/pull/2855)
+
+#### Bug fixes: {#bug-fixes-14}
+
+- Removed unnecessary escaping of the connection string parameters for ODBC, which made it impossible to establish a connection. This error occurred in version 18.6.0.
+- Fixed the logic for processing `REPLACE PARTITION` commands in the replication queue. If there are two `REPLACE` commands for the same partition, the incorrect logic could cause one of them to remain in the replication queue and not be executed. [\#2814](https://github.com/ClickHouse/ClickHouse/pull/2814)
+- Fixed a merge bug when all data parts were empty (parts that were formed from a merge or from `ALTER DELETE` if all data was deleted). This bug appeared in version 18.1.0. [\#2930](https://github.com/ClickHouse/ClickHouse/pull/2930)
+- Fixed an error for concurrent `Set` or `Join`. [Amos Bird](https://github.com/ClickHouse/ClickHouse/pull/2823)
+- Fixed the `Block structure mismatch in UNION stream: different number of columns` error that occurred for `UNION ALL` queries inside a sub-query if one of the `SELECT` queries contains duplicate column names. [Winter Zhang](https://github.com/ClickHouse/ClickHouse/pull/2094)
+- Fixed a memory leak if an exception occurred when connecting to a MySQL server.
+- Fixed incorrect clickhouse-client response code in case of a query error.
+- Fixed incorrect behavior of materialized views containing DISTINCT. [\#2795](https://github.com/ClickHouse/ClickHouse/issues/2795)
+
+#### Backward incompatible changes {#backward-incompatible-changes-4}
+
+- Removed support for CHECK TABLE queries for Distributed tables.
+
+#### Build changes: {#build-changes-3}
+
+- The allocator has been replaced: `jemalloc` is now used instead of `tcmalloc`. In some scenarios, this increases speed up to 20%. However, there are queries that have slowed by up to 20%. Memory consumption has been reduced by approximately 10% in some scenarios, with improved stability. With highly competitive loads, CPU usage in userspace and in system shows just a slight increase. [\#2773](https://github.com/ClickHouse/ClickHouse/pull/2773)
+- Use of libressl from a submodule. [\#1983](https://github.com/ClickHouse/ClickHouse/pull/1983) [\#2807](https://github.com/ClickHouse/ClickHouse/pull/2807)
+- Use of unixodbc from a submodule. [\#2789](https://github.com/ClickHouse/ClickHouse/pull/2789)
+- Use of mariadb-connector-c from a submodule. [\#2785](https://github.com/ClickHouse/ClickHouse/pull/2785)
+- Added functional test files to the repository that depend on the availability of test data (for the time being, without the test data itself).
+
+## ClickHouse release 18.6 {#clickhouse-release-18-6}
+
+### ClickHouse release 18.6.0, 2018-08-02 {#clickhouse-release-18-6-0-2018-08-02}
+
+#### New features: {#new-features-6}
+
+- Added support for ON expressions for the JOIN ON syntax:
+ `JOIN ON Expr([table.]column ...) = Expr([table.]column, ...) [AND Expr([table.]column, ...) = Expr([table.]column, ...) ...]`
+ The expression must be a chain of equalities joined by the AND operator. Each side of the equality can be an arbitrary expression over the columns of one of the tables. The use of fully qualified column names is supported (`table.name`, `database.table.name`, `table_alias.name`, `subquery_alias.name`) for the right table. [\#2742](https://github.com/ClickHouse/ClickHouse/pull/2742)
+- HTTPS can be enabled for replication. [\#2760](https://github.com/ClickHouse/ClickHouse/pull/2760)
+
+#### Improvements: {#improvements-6}
+
+- The server passes the patch component of its version to the client. Data about the patch version component is in `system.processes` and `query_log`. [\#2646](https://github.com/ClickHouse/ClickHouse/pull/2646)
+
+## ClickHouse release 18.5 {#clickhouse-release-18-5}
+
+### ClickHouse release 18.5.1, 2018-07-31 {#clickhouse-release-18-5-1-2018-07-31}
+
+#### New features: {#new-features-7}
+
+- Added the hash function `murmurHash2_32` [\#2756](https://github.com/ClickHouse/ClickHouse/pull/2756).
+
+#### Improvements: {#improvements-7}
+
+- Now you can use the `from_env` [\#2741](https://github.com/ClickHouse/ClickHouse/pull/2741) attribute to set values in config files from environment variables.
+- Added case-insensitive versions of the `coalesce`, `ifNull`, and `nullIf functions` [\#2752](https://github.com/ClickHouse/ClickHouse/pull/2752).
+
+#### Bug fixes: {#bug-fixes-15}
+
+- Fixed a possible bug when starting a replica [\#2759](https://github.com/ClickHouse/ClickHouse/pull/2759).
+
+## ClickHouse release 18.4 {#clickhouse-release-18-4}
+
+### ClickHouse release 18.4.0, 2018-07-28 {#clickhouse-release-18-4-0-2018-07-28}
+
+#### New features: {#new-features-8}
+
+- Added system tables: `formats`, `data_type_families`, `aggregate_function_combinators`, `table_functions`, `table_engines`, `collations` [\#2721](https://github.com/ClickHouse/ClickHouse/pull/2721).
+- Added the ability to use a table function instead of a table as an argument of a `remote` or `cluster table function` [\#2708](https://github.com/ClickHouse/ClickHouse/pull/2708).
+- Support for `HTTP Basic` authentication in the replication protocol [\#2727](https://github.com/ClickHouse/ClickHouse/pull/2727).
+- The `has` function now allows searching for a numeric value in an array of `Enum` values [Maxim Khrisanfov](https://github.com/ClickHouse/ClickHouse/pull/2699).
+- Support for adding arbitrary message separators when reading from `Kafka` [Amos Bird](https://github.com/ClickHouse/ClickHouse/pull/2701).
+
+#### Improvements: {#improvements-8}
+
+- The `ALTER TABLE t DELETE WHERE` query does not rewrite data parts that were not affected by the WHERE condition [\#2694](https://github.com/ClickHouse/ClickHouse/pull/2694).
+- The `use_minimalistic_checksums_in_zookeeper` option for `ReplicatedMergeTree` tables is enabled by default. This setting was added in version 1.1.54378, 2018-04-16. Versions that are older than 1.1.54378 can no longer be installed.
+- Support for running `KILL` and `OPTIMIZE` queries that specify `ON CLUSTER` [Winter Zhang](https://github.com/ClickHouse/ClickHouse/pull/2689).
+
+#### Bug fixes: {#bug-fixes-16}
+
+- Fixed the error `Column ... is not under an aggregate function and not in GROUP BY` for aggregation with an IN expression. This bug appeared in version 18.1.0. ([bbdd780b](https://github.com/ClickHouse/ClickHouse/commit/bbdd780be0be06a0f336775941cdd536878dd2c2))
+- Fixed a bug in the `windowFunnel aggregate function` [Winter Zhang](https://github.com/ClickHouse/ClickHouse/pull/2735).
+- Fixed a bug in the `anyHeavy` aggregate function ([a2101df2](https://github.com/ClickHouse/ClickHouse/commit/a2101df25a6a0fba99aa71f8793d762af2b801ee))
+- Fixed server crash when using the `countArray()` aggregate function.
+
+#### Backward incompatible changes: {#backward-incompatible-changes-5}
+
+- Parameters for `Kafka` engine was changed from `Kafka(kafka_broker_list, kafka_topic_list, kafka_group_name, kafka_format[, kafka_schema, kafka_num_consumers])` to `Kafka(kafka_broker_list, kafka_topic_list, kafka_group_name, kafka_format[, kafka_row_delimiter, kafka_schema, kafka_num_consumers])`. If your tables use `kafka_schema` or `kafka_num_consumers` parameters, you have to manually edit the metadata files `path/metadata/database/table.sql` and add `kafka_row_delimiter` parameter with `''` value.
+
+## ClickHouse release 18.1 {#clickhouse-release-18-1}
+
+### ClickHouse release 18.1.0, 2018-07-23 {#clickhouse-release-18-1-0-2018-07-23}
+
+#### New features: {#new-features-9}
+
+- Support for the `ALTER TABLE t DELETE WHERE` query for non-replicated MergeTree tables ([\#2634](https://github.com/ClickHouse/ClickHouse/pull/2634)).
+- Support for arbitrary types for the `uniq*` family of aggregate functions ([\#2010](https://github.com/ClickHouse/ClickHouse/issues/2010)).
+- Support for arbitrary types in comparison operators ([\#2026](https://github.com/ClickHouse/ClickHouse/issues/2026)).
+- The `users.xml` file allows setting a subnet mask in the format `10.0.0.1/255.255.255.0`. This is necessary for using masks for IPv6 networks with zeros in the middle ([\#2637](https://github.com/ClickHouse/ClickHouse/pull/2637)).
+- Added the `arrayDistinct` function ([\#2670](https://github.com/ClickHouse/ClickHouse/pull/2670)).
+- The SummingMergeTree engine can now work with AggregateFunction type columns ([Constantin S. Pan](https://github.com/ClickHouse/ClickHouse/pull/2566)).
+
+#### Improvements: {#improvements-9}
+
+- Changed the numbering scheme for release versions. Now the first part contains the year of release (A.D., Moscow timezone, minus 2000), the second part contains the number for major changes (increases for most releases), and the third part is the patch version. Releases are still backward compatible, unless otherwise stated in the changelog.
+- Faster conversions of floating-point numbers to a string ([Amos Bird](https://github.com/ClickHouse/ClickHouse/pull/2664)).
+- If some rows were skipped during an insert due to parsing errors (this is possible with the `input_allow_errors_num` and `input_allow_errors_ratio` settings enabled), the number of skipped rows is now written to the server log ([Leonardo Cecchi](https://github.com/ClickHouse/ClickHouse/pull/2669)).
+
+#### Bug fixes: {#bug-fixes-17}
+
+- Fixed the TRUNCATE command for temporary tables ([Amos Bird](https://github.com/ClickHouse/ClickHouse/pull/2624)).
+- Fixed a rare deadlock in the ZooKeeper client library that occurred when there was a network error while reading the response ([c315200](https://github.com/ClickHouse/ClickHouse/commit/c315200e64b87e44bdf740707fc857d1fdf7e947)).
+- Fixed an error during a CAST to Nullable types ([\#1322](https://github.com/ClickHouse/ClickHouse/issues/1322)).
+- Fixed the incorrect result of the `maxIntersection()` function when the boundaries of intervals coincided ([Michael Furmur](https://github.com/ClickHouse/ClickHouse/pull/2657)).
+- Fixed incorrect transformation of the OR expression chain in a function argument ([chenxing-xc](https://github.com/ClickHouse/ClickHouse/pull/2663)).
+- Fixed performance degradation for queries containing `IN (subquery)` expressions inside another subquery ([\#2571](https://github.com/ClickHouse/ClickHouse/issues/2571)).
+- Fixed incompatibility between servers with different versions in distributed queries that use a `CAST` function that isn’t in uppercase letters ([fe8c4d6](https://github.com/ClickHouse/ClickHouse/commit/fe8c4d64e434cacd4ceef34faa9005129f2190a5)).
+- Added missing quoting of identifiers for queries to an external DBMS ([\#2635](https://github.com/ClickHouse/ClickHouse/issues/2635)).
+
+#### Backward incompatible changes: {#backward-incompatible-changes-6}
+
+- Converting a string containing the number zero to DateTime does not work. Example: `SELECT toDateTime('0')`. This is also the reason that `DateTime DEFAULT '0'` does not work in tables, as well as `0` in dictionaries. Solution: replace `0` with `0000-00-00 00:00:00`.
+
+## ClickHouse release 1.1 {#clickhouse-release-1-1}
+
+### ClickHouse release 1.1.54394, 2018-07-12 {#clickhouse-release-1-1-54394-2018-07-12}
+
+#### New features: {#new-features-10}
+
+- Added the `histogram` aggregate function ([Mikhail Surin](https://github.com/ClickHouse/ClickHouse/pull/2521)).
+- Now `OPTIMIZE TABLE ... FINAL` can be used without specifying partitions for `ReplicatedMergeTree` ([Amos Bird](https://github.com/ClickHouse/ClickHouse/pull/2600)).
+
+#### Bug fixes: {#bug-fixes-18}
+
+- Fixed a problem with a very small timeout for sockets (one second) for reading and writing when sending and downloading replicated data, which made it impossible to download larger parts if there is a load on the network or disk (it resulted in cyclical attempts to download parts). This error occurred in version 1.1.54388.
+- Fixed issues when using chroot in ZooKeeper if you inserted duplicate data blocks in the table.
+- The `has` function now works correctly for an array with Nullable elements ([\#2115](https://github.com/ClickHouse/ClickHouse/issues/2115)).
+- The `system.tables` table now works correctly when used in distributed queries. The `metadata_modification_time` and `engine_full` columns are now non-virtual. Fixed an error that occurred if only these columns were queried from the table.
+- Fixed how an empty `TinyLog` table works after inserting an empty data block ([\#2563](https://github.com/ClickHouse/ClickHouse/issues/2563)).
+- The `system.zookeeper` table works if the value of the node in ZooKeeper is NULL.
+
+### ClickHouse release 1.1.54390, 2018-07-06 {#clickhouse-release-1-1-54390-2018-07-06}
+
+#### New features: {#new-features-11}
+
+- Queries can be sent in `multipart/form-data` format (in the `query` field), which is useful if external data is also sent for query processing ([Olga Hvostikova](https://github.com/ClickHouse/ClickHouse/pull/2490)).
+- Added the ability to enable or disable processing single or double quotes when reading data in CSV format. You can configure this in the `format_csv_allow_single_quotes` and `format_csv_allow_double_quotes` settings ([Amos Bird](https://github.com/ClickHouse/ClickHouse/pull/2574)).
+- Now `OPTIMIZE TABLE ... FINAL` can be used without specifying the partition for non-replicated variants of `MergeTree` ([Amos Bird](https://github.com/ClickHouse/ClickHouse/pull/2599)).
+
+#### Improvements: {#improvements-10}
+
+- Improved performance, reduced memory consumption, and correct memory consumption tracking with use of the IN operator when a table index could be used ([\#2584](https://github.com/ClickHouse/ClickHouse/pull/2584)).
+- Removed redundant checking of checksums when adding a data part. This is important when there are a large number of replicas, because in these cases the total number of checks was equal to N^2.
+- Added support for `Array(Tuple(...))` arguments for the `arrayEnumerateUniq` function ([\#2573](https://github.com/ClickHouse/ClickHouse/pull/2573)).
+- Added `Nullable` support for the `runningDifference` function ([\#2594](https://github.com/ClickHouse/ClickHouse/pull/2594)).
+- Improved query analysis performance when there is a very large number of expressions ([\#2572](https://github.com/ClickHouse/ClickHouse/pull/2572)).
+- Faster selection of data parts for merging in `ReplicatedMergeTree` tables. Faster recovery of the ZooKeeper session ([\#2597](https://github.com/ClickHouse/ClickHouse/pull/2597)).
+- The `format_version.txt` file for `MergeTree` tables is re-created if it is missing, which makes sense if ClickHouse is launched after copying the directory structure without files ([Ciprian Hacman](https://github.com/ClickHouse/ClickHouse/pull/2593)).
+
+#### Bug fixes: {#bug-fixes-19}
+
+- Fixed a bug when working with ZooKeeper that could make it impossible to recover the session and readonly states of tables before restarting the server.
+- Fixed a bug when working with ZooKeeper that could result in old nodes not being deleted if the session is interrupted.
+- Fixed an error in the `quantileTDigest` function for Float arguments (this bug was introduced in version 1.1.54388) ([Mikhail Surin](https://github.com/ClickHouse/ClickHouse/pull/2553)).
+- Fixed a bug in the index for MergeTree tables if the primary key column is located inside the function for converting types between signed and unsigned integers of the same size ([\#2603](https://github.com/ClickHouse/ClickHouse/pull/2603)).
+- Fixed segfault if `macros` are used but they aren’t in the config file ([\#2570](https://github.com/ClickHouse/ClickHouse/pull/2570)).
+- Fixed switching to the default database when reconnecting the client ([\#2583](https://github.com/ClickHouse/ClickHouse/pull/2583)).
+- Fixed a bug that occurred when the `use_index_for_in_with_subqueries` setting was disabled.
+
+#### Security fix: {#security-fix-1}
+
+- Sending files is no longer possible when connected to MySQL (`LOAD DATA LOCAL INFILE`).
+
+### ClickHouse release 1.1.54388, 2018-06-28 {#clickhouse-release-1-1-54388-2018-06-28}
+
+#### New features: {#new-features-12}
+
+- Support for the `ALTER TABLE t DELETE WHERE` query for replicated tables. Added the `system.mutations` table to track progress of this type of queries.
+- Support for the `ALTER TABLE t [REPLACE|ATTACH] PARTITION` query for \*MergeTree tables.
+- Support for the `TRUNCATE TABLE` query ([Winter Zhang](https://github.com/ClickHouse/ClickHouse/pull/2260))
+- Several new `SYSTEM` queries for replicated tables (`RESTART REPLICAS`, `SYNC REPLICA`, `[STOP|START] [MERGES|FETCHES|SENDS REPLICATED|REPLICATION QUEUES]`).
+- Added the ability to write to a table with the MySQL engine and the corresponding table function ([sundy-li](https://github.com/ClickHouse/ClickHouse/pull/2294)).
+- Added the `url()` table function and the `URL` table engine ([Alexander Sapin](https://github.com/ClickHouse/ClickHouse/pull/2501)).
+- Added the `windowFunnel` aggregate function ([sundy-li](https://github.com/ClickHouse/ClickHouse/pull/2352)).
+- New `startsWith` and `endsWith` functions for strings ([Vadim Plakhtinsky](https://github.com/ClickHouse/ClickHouse/pull/2429)).
+- The `numbers()` table function now allows you to specify the offset ([Winter Zhang](https://github.com/ClickHouse/ClickHouse/pull/2535)).
+- The password to `clickhouse-client` can be entered interactively.
+- Server logs can now be sent to syslog ([Alexander Krasheninnikov](https://github.com/ClickHouse/ClickHouse/pull/2459)).
+- Support for logging in dictionaries with a shared library source ([Alexander Sapin](https://github.com/ClickHouse/ClickHouse/pull/2472)).
+- Support for custom CSV delimiters ([Ivan Zhukov](https://github.com/ClickHouse/ClickHouse/pull/2263))
+- Added the `date_time_input_format` setting. If you switch this setting to `'best_effort'`, DateTime values will be read in a wide range of formats.
+- Added the `clickhouse-obfuscator` utility for data obfuscation. Usage example: publishing data used in performance tests.
+
+#### Experimental features: {#experimental-features-2}
+
+- Added the ability to calculate `and` arguments only where they are needed ([Anastasia Tsarkova](https://github.com/ClickHouse/ClickHouse/pull/2272))
+- JIT compilation to native code is now available for some expressions ([pyos](https://github.com/ClickHouse/ClickHouse/pull/2277)).
+
+#### Bug fixes: {#bug-fixes-20}
+
+- Duplicates no longer appear for a query with `DISTINCT` and `ORDER BY`.
+- Queries with `ARRAY JOIN` and `arrayFilter` no longer return an incorrect result.
+- Fixed an error when reading an array column from a Nested structure ([\#2066](https://github.com/ClickHouse/ClickHouse/issues/2066)).
+- Fixed an error when analyzing queries with a HAVING clause like `HAVING tuple IN (...)`.
+- Fixed an error when analyzing queries with recursive aliases.
+- Fixed an error when reading from ReplacingMergeTree with a condition in PREWHERE that filters all rows ([\#2525](https://github.com/ClickHouse/ClickHouse/issues/2525)).
+- User profile settings were not applied when using sessions in the HTTP interface.
+- Fixed how settings are applied from the command line parameters in clickhouse-local.
+- The ZooKeeper client library now uses the session timeout received from the server.
+- Fixed a bug in the ZooKeeper client library when the client waited for the server response longer than the timeout.
+- Fixed pruning of parts for queries with conditions on partition key columns ([\#2342](https://github.com/ClickHouse/ClickHouse/issues/2342)).
+- Merges are now possible after `CLEAR COLUMN IN PARTITION` ([\#2315](https://github.com/ClickHouse/ClickHouse/issues/2315)).
+- Type mapping in the ODBC table function has been fixed ([sundy-li](https://github.com/ClickHouse/ClickHouse/pull/2268)).
+- Type comparisons have been fixed for `DateTime` with and without the time zone ([Alexander Bocharov](https://github.com/ClickHouse/ClickHouse/pull/2400)).
+- Fixed syntactic parsing and formatting of the `CAST` operator.
+- Fixed insertion into a materialized view for the Distributed table engine ([Babacar Diassé](https://github.com/ClickHouse/ClickHouse/pull/2411)).
+- Fixed a race condition when writing data from the `Kafka` engine to materialized views ([Yangkuan Liu](https://github.com/ClickHouse/ClickHouse/pull/2448)).
+- Fixed SSRF in the remote() table function.
+- Fixed exit behavior of `clickhouse-client` in multiline mode ([\#2510](https://github.com/ClickHouse/ClickHouse/issues/2510)).
+
+#### Improvements: {#improvements-11}
+
+- Background tasks in replicated tables are now performed in a thread pool instead of in separate threads ([Silviu Caragea](https://github.com/ClickHouse/ClickHouse/pull/1722)).
+- Improved LZ4 compression performance.
+- Faster analysis for queries with a large number of JOINs and sub-queries.
+- The DNS cache is now updated automatically when there are too many network errors.
+- Table inserts no longer occur if the insert into one of the materialized views is not possible because it has too many parts.
+- Corrected the discrepancy in the event counters `Query`, `SelectQuery`, and `InsertQuery`.
+- Expressions like `tuple IN (SELECT tuple)` are allowed if the tuple types match.
+- A server with replicated tables can start even if you haven’t configured ZooKeeper.
+- When calculating the number of available CPU cores, limits on cgroups are now taken into account ([Atri Sharma](https://github.com/ClickHouse/ClickHouse/pull/2325)).
+- Added chown for config directories in the systemd config file ([Mikhail Shiryaev](https://github.com/ClickHouse/ClickHouse/pull/2421)).
+
+#### Build changes: {#build-changes-4}
+
+- The gcc8 compiler can be used for builds.
+- Added the ability to build llvm from submodule.
+- The version of the librdkafka library has been updated to v0.11.4.
+- Added the ability to use the system libcpuid library. The library version has been updated to 0.4.0.
+- Fixed the build using the vectorclass library ([Babacar Diassé](https://github.com/ClickHouse/ClickHouse/pull/2274)).
+- Cmake now generates files for ninja by default (like when using `-G Ninja`).
+- Added the ability to use the libtinfo library instead of libtermcap ([Georgy Kondratiev](https://github.com/ClickHouse/ClickHouse/pull/2519)).
+- Fixed a header file conflict in Fedora Rawhide ([\#2520](https://github.com/ClickHouse/ClickHouse/issues/2520)).
+
+#### Backward incompatible changes: {#backward-incompatible-changes-7}
+
+- Removed escaping in `Vertical` and `Pretty*` formats and deleted the `VerticalRaw` format.
+- If servers with version 1.1.54388 (or newer) and servers with an older version are used simultaneously in a distributed query and the query has the `cast(x, 'Type')` expression without the `AS` keyword and doesn’t have the word `cast` in uppercase, an exception will be thrown with a message like `Not found column cast(0, 'UInt8') in block`. Solution: Update the server on the entire cluster.
+
+### ClickHouse release 1.1.54385, 2018-06-01 {#clickhouse-release-1-1-54385-2018-06-01}
+
+#### Bug fixes: {#bug-fixes-21}
+
+- Fixed an error that in some cases caused ZooKeeper operations to block.
+
+### ClickHouse release 1.1.54383, 2018-05-22 {#clickhouse-release-1-1-54383-2018-05-22}
+
+#### Bug fixes: {#bug-fixes-22}
+
+- Fixed a slowdown of replication queue if a table has many replicas.
+
+### ClickHouse release 1.1.54381, 2018-05-14 {#clickhouse-release-1-1-54381-2018-05-14}
+
+#### Bug fixes: {#bug-fixes-23}
+
+- Fixed a nodes leak in ZooKeeper when ClickHouse loses connection to ZooKeeper server.
+
+### ClickHouse release 1.1.54380, 2018-04-21 {#clickhouse-release-1-1-54380-2018-04-21}
+
+#### New features: {#new-features-13}
+
+- Added the table function `file(path, format, structure)`. An example reading bytes from `/dev/urandom`: ``` ln -s /dev/urandom /var/lib/clickhouse/user_files/random``clickhouse-client -q "SELECT * FROM file('random', 'RowBinary', 'd UInt8') LIMIT 10" ```.
+
+#### Improvements: {#improvements-12}
+
+- Subqueries can be wrapped in `()` brackets to enhance query readability. For example: `(SELECT 1) UNION ALL (SELECT 1)`.
+- Simple `SELECT` queries from the `system.processes` table are not included in the `max_concurrent_queries` limit.
+
+#### Bug fixes: {#bug-fixes-24}
+
+- Fixed incorrect behavior of the `IN` operator when select from `MATERIALIZED VIEW`.
+- Fixed incorrect filtering by partition index in expressions like `partition_key_column IN (...)`.
+- Fixed inability to execute `OPTIMIZE` query on non-leader replica if `REANAME` was performed on the table.
+- Fixed the authorization error when executing `OPTIMIZE` or `ALTER` queries on a non-leader replica.
+- Fixed freezing of `KILL QUERY`.
+- Fixed an error in ZooKeeper client library which led to loss of watches, freezing of distributed DDL queue, and slowdowns in the replication queue if a non-empty `chroot` prefix is used in the ZooKeeper configuration.
+
+#### Backward incompatible changes: {#backward-incompatible-changes-8}
+
+- Removed support for expressions like `(a, b) IN (SELECT (a, b))` (you can use the equivalent expression `(a, b) IN (SELECT a, b)`). In previous releases, these expressions led to undetermined `WHERE` filtering or caused errors.
+
+### ClickHouse release 1.1.54378, 2018-04-16 {#clickhouse-release-1-1-54378-2018-04-16}
+
+#### New features: {#new-features-14}
+
+- Logging level can be changed without restarting the server.
+- Added the `SHOW CREATE DATABASE` query.
+- The `query_id` can be passed to `clickhouse-client` (elBroom).
+- New setting: `max_network_bandwidth_for_all_users`.
+- Added support for `ALTER TABLE ... PARTITION ...` for `MATERIALIZED VIEW`.
+- Added information about the size of data parts in uncompressed form in the system table.
+- Server-to-server encryption support for distributed tables (`1` in the replica config in ``).
+- Configuration of the table level for the `ReplicatedMergeTree` family in order to minimize the amount of data stored in Zookeeper: : `use_minimalistic_checksums_in_zookeeper = 1`
+- Configuration of the `clickhouse-client` prompt. By default, server names are now output to the prompt. The server’s display name can be changed. It’s also sent in the `X-ClickHouse-Display-Name` HTTP header (Kirill Shvakov).
+- Multiple comma-separated `topics` can be specified for the `Kafka` engine (Tobias Adamson)
+- When a query is stopped by `KILL QUERY` or `replace_running_query`, the client receives the `Query was canceled` exception instead of an incomplete result.
+
+#### Improvements: {#improvements-13}
+
+- `ALTER TABLE ... DROP/DETACH PARTITION` queries are run at the front of the replication queue.
+- `SELECT ... FINAL` and `OPTIMIZE ... FINAL` can be used even when the table has a single data part.
+- A `query_log` table is recreated on the fly if it was deleted manually (Kirill Shvakov).
+- The `lengthUTF8` function runs faster (zhang2014).
+- Improved performance of synchronous inserts in `Distributed` tables (`insert_distributed_sync = 1`) when there is a very large number of shards.
+- The server accepts the `send_timeout` and `receive_timeout` settings from the client and applies them when connecting to the client (they are applied in reverse order: the server socket’s `send_timeout` is set to the `receive_timeout` value received from the client, and vice versa).
+- More robust crash recovery for asynchronous insertion into `Distributed` tables.
+- The return type of the `countEqual` function changed from `UInt32` to `UInt64` (谢磊).
+
+#### Bug fixes: {#bug-fixes-25}
+
+- Fixed an error with `IN` when the left side of the expression is `Nullable`.
+- Correct results are now returned when using tuples with `IN` when some of the tuple components are in the table index.
+- The `max_execution_time` limit now works correctly with distributed queries.
+- Fixed errors when calculating the size of composite columns in the `system.columns` table.
+- Fixed an error when creating a temporary table `CREATE TEMPORARY TABLE IF NOT EXISTS.`
+- Fixed errors in `StorageKafka` (\#\#2075)
+- Fixed server crashes from invalid arguments of certain aggregate functions.
+- Fixed the error that prevented the `DETACH DATABASE` query from stopping background tasks for `ReplicatedMergeTree` tables.
+- `Too many parts` state is less likely to happen when inserting into aggregated materialized views (\#\#2084).
+- Corrected recursive handling of substitutions in the config if a substitution must be followed by another substitution on the same level.
+- Corrected the syntax in the metadata file when creating a `VIEW` that uses a query with `UNION ALL`.
+- `SummingMergeTree` now works correctly for summation of nested data structures with a composite key.
+- Fixed the possibility of a race condition when choosing the leader for `ReplicatedMergeTree` tables.
+
+#### Build changes: {#build-changes-5}
+
+- The build supports `ninja` instead of `make` and uses `ninja` by default for building releases.
+- Renamed packages: `clickhouse-server-base` in `clickhouse-common-static`; `clickhouse-server-common` in `clickhouse-server`; `clickhouse-common-dbg` in `clickhouse-common-static-dbg`. To install, use `clickhouse-server clickhouse-client`. Packages with the old names will still load in the repositories for backward compatibility.
+
+#### Backward incompatible changes: {#backward-incompatible-changes-9}
+
+- Removed the special interpretation of an IN expression if an array is specified on the left side. Previously, the expression `arr IN (set)` was interpreted as “at least one `arr` element belongs to the `set`”. To get the same behavior in the new version, write `arrayExists(x -> x IN (set), arr)`.
+- Disabled the incorrect use of the socket option `SO_REUSEPORT`, which was incorrectly enabled by default in the Poco library. Note that on Linux there is no longer any reason to simultaneously specify the addresses `::` and `0.0.0.0` for listen – use just `::`, which allows listening to the connection both over IPv4 and IPv6 (with the default kernel config settings). You can also revert to the behavior from previous versions by specifying `1` in the config.
+
+### ClickHouse release 1.1.54370, 2018-03-16 {#clickhouse-release-1-1-54370-2018-03-16}
+
+#### New features: {#new-features-15}
+
+- Added the `system.macros` table and auto updating of macros when the config file is changed.
+- Added the `SYSTEM RELOAD CONFIG` query.
+- Added the `maxIntersections(left_col, right_col)` aggregate function, which returns the maximum number of simultaneously intersecting intervals `[left; right]`. The `maxIntersectionsPosition(left, right)` function returns the beginning of the “maximum” interval. ([Michael Furmur](https://github.com/ClickHouse/ClickHouse/pull/2012)).
+
+#### Improvements: {#improvements-14}
+
+- When inserting data in a `Replicated` table, fewer requests are made to `ZooKeeper` (and most of the user-level errors have disappeared from the `ZooKeeper` log).
+- Added the ability to create aliases for data sets. Example: `WITH (1, 2, 3) AS set SELECT number IN set FROM system.numbers LIMIT 10`.
+
+#### Bug fixes: {#bug-fixes-26}
+
+- Fixed the `Illegal PREWHERE` error when reading from Merge tables for `Distributed`tables.
+- Added fixes that allow you to start clickhouse-server in IPv4-only Docker containers.
+- Fixed a race condition when reading from system `system.parts_columns tables.`
+- Removed double buffering during a synchronous insert to a `Distributed` table, which could have caused the connection to timeout.
+- Fixed a bug that caused excessively long waits for an unavailable replica before beginning a `SELECT` query.
+- Fixed incorrect dates in the `system.parts` table.
+- Fixed a bug that made it impossible to insert data in a `Replicated` table if `chroot` was non-empty in the configuration of the `ZooKeeper` cluster.
+- Fixed the vertical merging algorithm for an empty `ORDER BY` table.
+- Restored the ability to use dictionaries in queries to remote tables, even if these dictionaries are not present on the requestor server. This functionality was lost in release 1.1.54362.
+- Restored the behavior for queries like `SELECT * FROM remote('server2', default.table) WHERE col IN (SELECT col2 FROM default.table)` when the right side of the `IN` should use a remote `default.table` instead of a local one. This behavior was broken in version 1.1.54358.
+- Removed extraneous error-level logging of `Not found column ... in block`.
+
+### Clickhouse Release 1.1.54362, 2018-03-11 {#clickhouse-release-1-1-54362-2018-03-11}
+
+#### New features: {#new-features-16}
+
+- Aggregation without `GROUP BY` for an empty set (such as `SELECT count(*) FROM table WHERE 0`) now returns a result with one row with null values for aggregate functions, in compliance with the SQL standard. To restore the old behavior (return an empty result), set `empty_result_for_aggregation_by_empty_set` to 1.
+- Added type conversion for `UNION ALL`. Different alias names are allowed in `SELECT` positions in `UNION ALL`, in compliance with the SQL standard.
+- Arbitrary expressions are supported in `LIMIT BY` clauses. Previously, it was only possible to use columns resulting from `SELECT`.
+- An index of `MergeTree` tables is used when `IN` is applied to a tuple of expressions from the columns of the primary key. Example: `WHERE (UserID, EventDate) IN ((123, '2000-01-01'), ...)` (Anastasiya Tsarkova).
+- Added the `clickhouse-copier` tool for copying between clusters and resharding data (beta).
+- Added consistent hashing functions: `yandexConsistentHash`, `jumpConsistentHash`, `sumburConsistentHash`. They can be used as a sharding key in order to reduce the amount of network traffic during subsequent reshardings.
+- Added functions: `arrayAny`, `arrayAll`, `hasAny`, `hasAll`, `arrayIntersect`, `arrayResize`.
+- Added the `arrayCumSum` function (Javi Santana).
+- Added the `parseDateTimeBestEffort`, `parseDateTimeBestEffortOrZero`, and `parseDateTimeBestEffortOrNull` functions to read the DateTime from a string containing text in a wide variety of possible formats.
+- Data can be partially reloaded from external dictionaries during updating (load just the records in which the value of the specified field greater than in the previous download) (Arsen Hakobyan).
+- Added the `cluster` table function. Example: `cluster(cluster_name, db, table)`. The `remote` table function can accept the cluster name as the first argument, if it is specified as an identifier.
+- The `remote` and `cluster` table functions can be used in `INSERT` queries.
+- Added the `create_table_query` and `engine_full` virtual columns to the `system.tables`table . The `metadata_modification_time` column is virtual.
+- Added the `data_path` and `metadata_path` columns to `system.tables`and`system.databases` tables, and added the `path` column to the `system.parts` and `system.parts_columns` tables.
+- Added additional information about merges in the `system.part_log` table.
+- An arbitrary partitioning key can be used for the `system.query_log` table (Kirill Shvakov).
+- The `SHOW TABLES` query now also shows temporary tables. Added temporary tables and the `is_temporary` column to `system.tables` (zhang2014).
+- Added `DROP TEMPORARY TABLE` and `EXISTS TEMPORARY TABLE` queries (zhang2014).
+- Support for `SHOW CREATE TABLE` for temporary tables (zhang2014).
+- Added the `system_profile` configuration parameter for the settings used by internal processes.
+- Support for loading `object_id` as an attribute in `MongoDB` dictionaries (Pavel Litvinenko).
+- Reading `null` as the default value when loading data for an external dictionary with the `MongoDB` source (Pavel Litvinenko).
+- Reading `DateTime` values in the `Values` format from a Unix timestamp without single quotes.
+- Failover is supported in `remote` table functions for cases when some of the replicas are missing the requested table.
+- Configuration settings can be overridden in the command line when you run `clickhouse-server`. Example: `clickhouse-server -- --logger.level=information`.
+- Implemented the `empty` function from a `FixedString` argument: the function returns 1 if the string consists entirely of null bytes (zhang2014).
+- Added the `listen_try`configuration parameter for listening to at least one of the listen addresses without quitting, if some of the addresses can’t be listened to (useful for systems with disabled support for IPv4 or IPv6).
+- Added the `VersionedCollapsingMergeTree` table engine.
+- Support for rows and arbitrary numeric types for the `library` dictionary source.
+- `MergeTree` tables can be used without a primary key (you need to specify `ORDER BY tuple()`).
+- A `Nullable` type can be `CAST` to a non-`Nullable` type if the argument is not `NULL`.
+- `RENAME TABLE` can be performed for `VIEW`.
+- Added the `throwIf` function.
+- Added the `odbc_default_field_size` option, which allows you to extend the maximum size of the value loaded from an ODBC source (by default, it is 1024).
+- The `system.processes` table and `SHOW PROCESSLIST` now have the `is_cancelled` and `peak_memory_usage` columns.
+
+#### Improvements: {#improvements-15}
+
+- Limits and quotas on the result are no longer applied to intermediate data for `INSERT SELECT` queries or for `SELECT` subqueries.
+- Fewer false triggers of `force_restore_data` when checking the status of `Replicated` tables when the server starts.
+- Added the `allow_distributed_ddl` option.
+- Nondeterministic functions are not allowed in expressions for `MergeTree` table keys.
+- Files with substitutions from `config.d` directories are loaded in alphabetical order.
+- Improved performance of the `arrayElement` function in the case of a constant multidimensional array with an empty array as one of the elements. Example: `[[1], []][x]`.
+- The server starts faster now when using configuration files with very large substitutions (for instance, very large lists of IP networks).
+- When running a query, table valued functions run once. Previously, `remote` and `mysql` table valued functions performed the same query twice to retrieve the table structure from a remote server.
+- The `MkDocs` documentation generator is used.
+- When you try to delete a table column that `DEFAULT`/`MATERIALIZED` expressions of other columns depend on, an exception is thrown (zhang2014).
+- Added the ability to parse an empty line in text formats as the number 0 for `Float` data types. This feature was previously available but was lost in release 1.1.54342.
+- `Enum` values can be used in `min`, `max`, `sum` and some other functions. In these cases, it uses the corresponding numeric values. This feature was previously available but was lost in the release 1.1.54337.
+- Added `max_expanded_ast_elements` to restrict the size of the AST after recursively expanding aliases.
+
+#### Bug fixes: {#bug-fixes-27}
+
+- Fixed cases when unnecessary columns were removed from subqueries in error, or not removed from subqueries containing `UNION ALL`.
+- Fixed a bug in merges for `ReplacingMergeTree` tables.
+- Fixed synchronous insertions in `Distributed` tables (`insert_distributed_sync = 1`).
+- Fixed segfault for certain uses of `FULL` and `RIGHT JOIN` with duplicate columns in subqueries.
+- Fixed segfault for certain uses of `replace_running_query` and `KILL QUERY`.
+- Fixed the order of the `source` and `last_exception` columns in the `system.dictionaries` table.
+- Fixed a bug when the `DROP DATABASE` query did not delete the file with metadata.
+- Fixed the `DROP DATABASE` query for `Dictionary` databases.
+- Fixed the low precision of `uniqHLL12` and `uniqCombined` functions for cardinalities greater than 100 million items (Alex Bocharov).
+- Fixed the calculation of implicit default values when necessary to simultaneously calculate default explicit expressions in `INSERT` queries (zhang2014).
+- Fixed a rare case when a query to a `MergeTree` table couldn’t finish (chenxing-xc).
+- Fixed a crash that occurred when running a `CHECK` query for `Distributed` tables if all shards are local (chenxing.xc).
+- Fixed a slight performance regression with functions that use regular expressions.
+- Fixed a performance regression when creating multidimensional arrays from complex expressions.
+- Fixed a bug that could cause an extra `FORMAT` section to appear in an `.sql` file with metadata.
+- Fixed a bug that caused the `max_table_size_to_drop` limit to apply when trying to delete a `MATERIALIZED VIEW` looking at an explicitly specified table.
+- Fixed incompatibility with old clients (old clients were sometimes sent data with the `DateTime('timezone')` type, which they do not understand).
+- Fixed a bug when reading `Nested` column elements of structures that were added using `ALTER` but that are empty for the old partitions, when the conditions for these columns moved to `PREWHERE`.
+- Fixed a bug when filtering tables by virtual `_table` columns in queries to `Merge` tables.
+- Fixed a bug when using `ALIAS` columns in `Distributed` tables.
+- Fixed a bug that made dynamic compilation impossible for queries with aggregate functions from the `quantile` family.
+- Fixed a race condition in the query execution pipeline that occurred in very rare cases when using `Merge` tables with a large number of tables, and when using `GLOBAL` subqueries.
+- Fixed a crash when passing arrays of different sizes to an `arrayReduce` function when using aggregate functions from multiple arguments.
+- Prohibited the use of queries with `UNION ALL` in a `MATERIALIZED VIEW`.
+- Fixed an error during initialization of the `part_log` system table when the server starts (by default, `part_log` is disabled).
+
+#### Backward incompatible changes: {#backward-incompatible-changes-10}
+
+- Removed the `distributed_ddl_allow_replicated_alter` option. This behavior is enabled by default.
+- Removed the `strict_insert_defaults` setting. If you were using this functionality, write to `clickhouse-feedback@yandex-team.com`.
+- Removed the `UnsortedMergeTree` engine.
+
+### Clickhouse Release 1.1.54343, 2018-02-05 {#clickhouse-release-1-1-54343-2018-02-05}
+
+- Added macros support for defining cluster names in distributed DDL queries and constructors of Distributed tables: `CREATE TABLE distr ON CLUSTER '{cluster}' (...) ENGINE = Distributed('{cluster}', 'db', 'table')`.
+- Now queries like `SELECT ... FROM table WHERE expr IN (subquery)` are processed using the `table` index.
+- Improved processing of duplicates when inserting to Replicated tables, so they no longer slow down execution of the replication queue.
+
+### Clickhouse Release 1.1.54342, 2018-01-22 {#clickhouse-release-1-1-54342-2018-01-22}
+
+This release contains bug fixes for the previous release 1.1.54337:
+
+- Fixed a regression in 1.1.54337: if the default user has readonly access, then the server refuses to start up with the message `Cannot create database in readonly mode`.
+- Fixed a regression in 1.1.54337: on systems with systemd, logs are always written to syslog regardless of the configuration; the watchdog script still uses init.d.
+- Fixed a regression in 1.1.54337: wrong default configuration in the Docker image.
+- Fixed nondeterministic behavior of GraphiteMergeTree (you can see it in log messages `Data after merge is not byte-identical to the data on another replicas`).
+- Fixed a bug that may lead to inconsistent merges after OPTIMIZE query to Replicated tables (you may see it in log messages `Part ... intersects the previous part`).
+- Buffer tables now work correctly when MATERIALIZED columns are present in the destination table (by zhang2014).
+- Fixed a bug in implementation of NULL.
+
+### Clickhouse Release 1.1.54337, 2018-01-18 {#clickhouse-release-1-1-54337-2018-01-18}
+
+#### New features: {#new-features-17}
+
+- Added support for storage of multi-dimensional arrays and tuples (`Tuple` data type) in tables.
+- Support for table functions for `DESCRIBE` and `INSERT` queries. Added support for subqueries in `DESCRIBE`. Examples: `DESC TABLE remote('host', default.hits)`; `DESC TABLE (SELECT 1)`; `INSERT INTO TABLE FUNCTION remote('host', default.hits)`. Support for `INSERT INTO TABLE` in addition to `INSERT INTO`.
+- Improved support for time zones. The `DateTime` data type can be annotated with the timezone that is used for parsing and formatting in text formats. Example: `DateTime('Europe/Moscow')`. When timezones are specified in functions for `DateTime` arguments, the return type will track the timezone, and the value will be displayed as expected.
+- Added the functions `toTimeZone`, `timeDiff`, `toQuarter`, `toRelativeQuarterNum`. The `toRelativeHour`/`Minute`/`Second` functions can take a value of type `Date` as an argument. The `now` function name is case-sensitive.
+- Added the `toStartOfFifteenMinutes` function (Kirill Shvakov).
+- Added the `clickhouse format` tool for formatting queries.
+- Added the `format_schema_path` configuration parameter (Marek Vavruşa). It is used for specifying a schema in `Cap'n Proto` format. Schema files can be located only in the specified directory.
+- Added support for config substitutions (`incl` and `conf.d`) for configuration of external dictionaries and models (Pavel Yakunin).
+- Added a column with documentation for the `system.settings` table (Kirill Shvakov).
+- Added the `system.parts_columns` table with information about column sizes in each data part of `MergeTree` tables.
+- Added the `system.models` table with information about loaded `CatBoost` machine learning models.
+- Added the `mysql` and `odbc` table function and corresponding `MySQL` and `ODBC` table engines for accessing remote databases. This functionality is in the beta stage.
+- Added the possibility to pass an argument of type `AggregateFunction` for the `groupArray` aggregate function (so you can create an array of states of some aggregate function).
+- Removed restrictions on various combinations of aggregate function combinators. For example, you can use `avgForEachIf` as well as `avgIfForEach` aggregate functions, which have different behaviors.
+- The `-ForEach` aggregate function combinator is extended for the case of aggregate functions of multiple arguments.
+- Added support for aggregate functions of `Nullable` arguments even for cases when the function returns a non-`Nullable` result (added with the contribution of Silviu Caragea). Example: `groupArray`, `groupUniqArray`, `topK`.
+- Added the `max_client_network_bandwidth` for `clickhouse-client` (Kirill Shvakov).
+- Users with the `readonly = 2` setting are allowed to work with TEMPORARY tables (CREATE, DROP, INSERT…) (Kirill Shvakov).
+- Added support for using multiple consumers with the `Kafka` engine. Extended configuration options for `Kafka` (Marek Vavruša).
+- Added the `intExp3` and `intExp4` functions.
+- Added the `sumKahan` aggregate function.
+- Added the to \* Number\* OrNull functions, where \* Number\* is a numeric type.
+- Added support for `WITH` clauses for an `INSERT SELECT` query (author: zhang2014).
+- Added settings: `http_connection_timeout`, `http_send_timeout`, `http_receive_timeout`. In particular, these settings are used for downloading data parts for replication. Changing these settings allows for faster failover if the network is overloaded.
+- Added support for `ALTER` for tables of type `Null` (Anastasiya Tsarkova).
+- The `reinterpretAsString` function is extended for all data types that are stored contiguously in memory.
+- Added the `--silent` option for the `clickhouse-local` tool. It suppresses printing query execution info in stderr.
+- Added support for reading values of type `Date` from text in a format where the month and/or day of the month is specified using a single digit instead of two digits (Amos Bird).
+
+#### Performance optimizations: {#performance-optimizations}
+
+- Improved performance of aggregate functions `min`, `max`, `any`, `anyLast`, `anyHeavy`, `argMin`, `argMax` from string arguments.
+- Improved performance of the functions `isInfinite`, `isFinite`, `isNaN`, `roundToExp2`.
+- Improved performance of parsing and formatting `Date` and `DateTime` type values in text format.
+- Improved performance and precision of parsing floating point numbers.
+- Lowered memory usage for `JOIN` in the case when the left and right parts have columns with identical names that are not contained in `USING` .
+- Improved performance of aggregate functions `varSamp`, `varPop`, `stddevSamp`, `stddevPop`, `covarSamp`, `covarPop`, `corr` by reducing computational stability. The old functions are available under the names `varSampStable`, `varPopStable`, `stddevSampStable`, `stddevPopStable`, `covarSampStable`, `covarPopStable`, `corrStable`.
+
+#### Bug fixes: {#bug-fixes-28}
+
+- Fixed data deduplication after running a `DROP` or `DETACH PARTITION` query. In the previous version, dropping a partition and inserting the same data again was not working because inserted blocks were considered duplicates.
+- Fixed a bug that could lead to incorrect interpretation of the `WHERE` clause for `CREATE MATERIALIZED VIEW` queries with `POPULATE` .
+- Fixed a bug in using the `root_path` parameter in the `zookeeper_servers` configuration.
+- Fixed unexpected results of passing the `Date` argument to `toStartOfDay` .
+- Fixed the `addMonths` and `subtractMonths` functions and the arithmetic for `INTERVAL n MONTH` in cases when the result has the previous year.
+- Added missing support for the `UUID` data type for `DISTINCT` , `JOIN` , and `uniq` aggregate functions and external dictionaries (Evgeniy Ivanov). Support for `UUID` is still incomplete.
+- Fixed `SummingMergeTree` behavior in cases when the rows summed to zero.
+- Various fixes for the `Kafka` engine (Marek Vavruša).
+- Fixed incorrect behavior of the `Join` table engine (Amos Bird).
+- Fixed incorrect allocator behavior under FreeBSD and OS X.
+- The `extractAll` function now supports empty matches.
+- Fixed an error that blocked usage of `libressl` instead of `openssl` .
+- Fixed the `CREATE TABLE AS SELECT` query from temporary tables.
+- Fixed non-atomicity of updating the replication queue. This could lead to replicas being out of sync until the server restarts.
+- Fixed possible overflow in `gcd` , `lcm` and `modulo` (`%` operator) (Maks Skorokhod).
+- `-preprocessed` files are now created after changing `umask` (`umask` can be changed in the config).
+- Fixed a bug in the background check of parts (`MergeTreePartChecker` ) when using a custom partition key.
+- Fixed parsing of tuples (values of the `Tuple` data type) in text formats.
+- Improved error messages about incompatible types passed to `multiIf` , `array` and some other functions.
+- Redesigned support for `Nullable` types. Fixed bugs that may lead to a server crash. Fixed almost all other bugs related to `NULL` support: incorrect type conversions in INSERT SELECT, insufficient support for Nullable in HAVING and PREWHERE, `join_use_nulls` mode, Nullable types as arguments of `OR` operator, etc.
+- Fixed various bugs related to internal semantics of data types. Examples: unnecessary summing of `Enum` type fields in `SummingMergeTree` ; alignment of `Enum` types in `Pretty` formats, etc.
+- Stricter checks for allowed combinations of composite columns.
+- Fixed the overflow when specifying a very large parameter for the `FixedString` data type.
+- Fixed a bug in the `topK` aggregate function in a generic case.
+- Added the missing check for equality of array sizes in arguments of n-ary variants of aggregate functions with an `-Array` combinator.
+- Fixed a bug in `--pager` for `clickhouse-client` (author: ks1322).
+- Fixed the precision of the `exp10` function.
+- Fixed the behavior of the `visitParamExtract` function for better compliance with documentation.
+- Fixed the crash when incorrect data types are specified.
+- Fixed the behavior of `DISTINCT` in the case when all columns are constants.
+- Fixed query formatting in the case of using the `tupleElement` function with a complex constant expression as the tuple element index.
+- Fixed a bug in `Dictionary` tables for `range_hashed` dictionaries.
+- Fixed a bug that leads to excessive rows in the result of `FULL` and `RIGHT JOIN` (Amos Bird).
+- Fixed a server crash when creating and removing temporary files in `config.d` directories during config reload.
+- Fixed the `SYSTEM DROP DNS CACHE` query: the cache was flushed but addresses of cluster nodes were not updated.
+- Fixed the behavior of `MATERIALIZED VIEW` after executing `DETACH TABLE` for the table under the view (Marek Vavruša).
+
+#### Build improvements: {#build-improvements-4}
+
+- The `pbuilder` tool is used for builds. The build process is almost completely independent of the build host environment.
+- A single build is used for different OS versions. Packages and binaries have been made compatible with a wide range of Linux systems.
+- Added the `clickhouse-test` package. It can be used to run functional tests.
+- The source tarball can now be published to the repository. It can be used to reproduce the build without using GitHub.
+- Added limited integration with Travis CI. Due to limits on build time in Travis, only the debug build is tested and a limited subset of tests are run.
+- Added support for `Cap'n'Proto` in the default build.
+- Changed the format of documentation sources from `Restricted Text` to `Markdown`.
+- Added support for `systemd` (Vladimir Smirnov). It is disabled by default due to incompatibility with some OS images and can be enabled manually.
+- For dynamic code generation, `clang` and `lld` are embedded into the `clickhouse` binary. They can also be invoked as `clickhouse clang` and `clickhouse lld` .
+- Removed usage of GNU extensions from the code. Enabled the `-Wextra` option. When building with `clang` the default is `libc++` instead of `libstdc++`.
+- Extracted `clickhouse_parsers` and `clickhouse_common_io` libraries to speed up builds of various tools.
+
+#### Backward incompatible changes: {#backward-incompatible-changes-11}
+
+- The format for marks in `Log` type tables that contain `Nullable` columns was changed in a backward incompatible way. If you have these tables, you should convert them to the `TinyLog` type before starting up the new server version. To do this, replace `ENGINE = Log` with `ENGINE = TinyLog` in the corresponding `.sql` file in the `metadata` directory. If your table doesn’t have `Nullable` columns or if the type of your table is not `Log`, then you don’t need to do anything.
+- Removed the `experimental_allow_extended_storage_definition_syntax` setting. Now this feature is enabled by default.
+- The `runningIncome` function was renamed to `runningDifferenceStartingWithFirstvalue` to avoid confusion.
+- Removed the `FROM ARRAY JOIN arr` syntax when ARRAY JOIN is specified directly after FROM with no table (Amos Bird).
+- Removed the `BlockTabSeparated` format that was used solely for demonstration purposes.
+- Changed the state format for aggregate functions `varSamp`, `varPop`, `stddevSamp`, `stddevPop`, `covarSamp`, `covarPop`, `corr`. If you have stored states of these aggregate functions in tables (using the `AggregateFunction` data type or materialized views with corresponding states), please write to clickhouse-feedback@yandex-team.com.
+- In previous server versions there was an undocumented feature: if an aggregate function depends on parameters, you can still specify it without parameters in the AggregateFunction data type. Example: `AggregateFunction(quantiles, UInt64)` instead of `AggregateFunction(quantiles(0.5, 0.9), UInt64)`. This feature was lost. Although it was undocumented, we plan to support it again in future releases.
+- Enum data types cannot be used in min/max aggregate functions. This ability will be returned in the next release.
+
+#### Please note when upgrading: {#please-note-when-upgrading}
+
+- When doing a rolling update on a cluster, at the point when some of the replicas are running the old version of ClickHouse and some are running the new version, replication is temporarily stopped and the message `unknown parameter 'shard'` appears in the log. Replication will continue after all replicas of the cluster are updated.
+- If different versions of ClickHouse are running on the cluster servers, it is possible that distributed queries using the following functions will have incorrect results: `varSamp`, `varPop`, `stddevSamp`, `stddevPop`, `covarSamp`, `covarPop`, `corr`. You should update all cluster nodes.
+
+## [Changelog for 2017](https://github.com/ClickHouse/ClickHouse/blob/master/docs/en/changelog/2017.md)
diff --git a/docs/fa/changelog/2019.md b/docs/fa/changelog/2019.md
deleted file mode 120000
index 105ca144fca..00000000000
--- a/docs/fa/changelog/2019.md
+++ /dev/null
@@ -1 +0,0 @@
-../../en/changelog/2019.md
\ No newline at end of file
diff --git a/docs/fa/changelog/2019.md b/docs/fa/changelog/2019.md
new file mode 100644
index 00000000000..01a0756af14
--- /dev/null
+++ b/docs/fa/changelog/2019.md
@@ -0,0 +1,2071 @@
+---
+en_copy: true
+---
+
+## ClickHouse release v19.17 {#clickhouse-release-v19-17}
+
+### ClickHouse release v19.17.6.36, 2019-12-27 {#clickhouse-release-v19-17-6-36-2019-12-27}
+
+#### Bug Fix {#bug-fix}
+
+- Fixed potential buffer overflow in decompress. Malicious user can pass fabricated compressed data that could cause read after buffer. This issue was found by Eldar Zaitov from Yandex information security team. [\#8404](https://github.com/ClickHouse/ClickHouse/pull/8404) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Fixed possible server crash (`std::terminate`) when the server cannot send or write data in JSON or XML format with values of String data type (that require UTF-8 validation) or when compressing result data with Brotli algorithm or in some other rare cases. [\#8384](https://github.com/ClickHouse/ClickHouse/pull/8384) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Fixed dictionaries with source from a clickhouse `VIEW`, now reading such dictionaries doesn’t cause the error `There is no query`. [\#8351](https://github.com/ClickHouse/ClickHouse/pull/8351) ([Nikolai Kochetov](https://github.com/KochetovNicolai))
+- Fixed checking if a client host is allowed by host\_regexp specified in users.xml. [\#8241](https://github.com/ClickHouse/ClickHouse/pull/8241), [\#8342](https://github.com/ClickHouse/ClickHouse/pull/8342) ([Vitaly Baranov](https://github.com/vitlibar))
+- `RENAME TABLE` for a distributed table now renames the folder containing inserted data before sending to shards. This fixes an issue with successive renames `tableA->tableB`, `tableC->tableA`. [\#8306](https://github.com/ClickHouse/ClickHouse/pull/8306) ([tavplubix](https://github.com/tavplubix))
+- `range_hashed` external dictionaries created by DDL queries now allow ranges of arbitrary numeric types. [\#8275](https://github.com/ClickHouse/ClickHouse/pull/8275) ([alesapin](https://github.com/alesapin))
+- Fixed `INSERT INTO table SELECT ... FROM mysql(...)` table function. [\#8234](https://github.com/ClickHouse/ClickHouse/pull/8234) ([tavplubix](https://github.com/tavplubix))
+- Fixed segfault in `INSERT INTO TABLE FUNCTION file()` while inserting into a file which doesn’t exist. Now in this case file would be created and then insert would be processed. [\#8177](https://github.com/ClickHouse/ClickHouse/pull/8177) ([Olga Khvostikova](https://github.com/stavrolia))
+- Fixed bitmapAnd error when intersecting an aggregated bitmap and a scalar bitmap. [\#8082](https://github.com/ClickHouse/ClickHouse/pull/8082) ([Yue Huang](https://github.com/moon03432))
+- Fixed segfault when `EXISTS` query was used without `TABLE` or `DICTIONARY` qualifier, just like `EXISTS t`. [\#8213](https://github.com/ClickHouse/ClickHouse/pull/8213) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Fixed return type for functions `rand` and `randConstant` in case of nullable argument. Now functions always return `UInt32` and never `Nullable(UInt32)`. [\#8204](https://github.com/ClickHouse/ClickHouse/pull/8204) ([Nikolai Kochetov](https://github.com/KochetovNicolai))
+- Fixed `DROP DICTIONARY IF EXISTS db.dict`, now it doesn’t throw exception if `db` doesn’t exist. [\#8185](https://github.com/ClickHouse/ClickHouse/pull/8185) ([Vitaly Baranov](https://github.com/vitlibar))
+- If a table wasn’t completely dropped because of server crash, the server will try to restore and load it [\#8176](https://github.com/ClickHouse/ClickHouse/pull/8176) ([tavplubix](https://github.com/tavplubix))
+- Fixed a trivial count query for a distributed table if there are more than two shard local table. [\#8164](https://github.com/ClickHouse/ClickHouse/pull/8164) ([小路](https://github.com/nicelulu))
+- Fixed bug that lead to a data race in DB::BlockStreamProfileInfo::calculateRowsBeforeLimit() [\#8143](https://github.com/ClickHouse/ClickHouse/pull/8143) ([Alexander Kazakov](https://github.com/Akazz))
+- Fixed `ALTER table MOVE part` executed immediately after merging the specified part, which could cause moving a part which the specified part merged into. Now it correctly moves the specified part. [\#8104](https://github.com/ClickHouse/ClickHouse/pull/8104) ([Vladimir Chebotarev](https://github.com/excitoon))
+- Expressions for dictionaries can be specified as strings now. This is useful for calculation of attributes while extracting data from non-ClickHouse sources because it allows to use non-ClickHouse syntax for those expressions. [\#8098](https://github.com/ClickHouse/ClickHouse/pull/8098) ([alesapin](https://github.com/alesapin))
+- Fixed a very rare race in `clickhouse-copier` because of an overflow in ZXid. [\#8088](https://github.com/ClickHouse/ClickHouse/pull/8088) ([Ding Xiang Fei](https://github.com/dingxiangfei2009))
+- Fixed the bug when after the query failed (due to “Too many simultaneous queries” for example) it would not read external tables info, and the
+ next request would interpret this info as the beginning of the next query causing an error like `Unknown packet from client`. [\#8084](https://github.com/ClickHouse/ClickHouse/pull/8084) ([Azat Khuzhin](https://github.com/azat))
+- Avoid null dereference after “Unknown packet X from server” [\#8071](https://github.com/ClickHouse/ClickHouse/pull/8071) ([Azat Khuzhin](https://github.com/azat))
+- Restore support of all ICU locales, add the ability to apply collations for constant expressions and add language name to system.collations table. [\#8051](https://github.com/ClickHouse/ClickHouse/pull/8051) ([alesapin](https://github.com/alesapin))
+- Number of streams for read from `StorageFile` and `StorageHDFS` is now limited, to avoid exceeding the memory limit. [\#7981](https://github.com/ClickHouse/ClickHouse/pull/7981) ([alesapin](https://github.com/alesapin))
+- Fixed `CHECK TABLE` query for `*MergeTree` tables without key. [\#7979](https://github.com/ClickHouse/ClickHouse/pull/7979) ([alesapin](https://github.com/alesapin))
+- Removed the mutation number from a part name in case there were no mutations. This removing improved the compatibility with older versions. [\#8250](https://github.com/ClickHouse/ClickHouse/pull/8250) ([alesapin](https://github.com/alesapin))
+- Fixed the bug that mutations are skipped for some attached parts due to their data\_version are larger than the table mutation version. [\#7812](https://github.com/ClickHouse/ClickHouse/pull/7812) ([Zhichang Yu](https://github.com/yuzhichang))
+- Allow starting the server with redundant copies of parts after moving them to another device. [\#7810](https://github.com/ClickHouse/ClickHouse/pull/7810) ([Vladimir Chebotarev](https://github.com/excitoon))
+- Fixed the error “Sizes of columns doesn’t match” that might appear when using aggregate function columns. [\#7790](https://github.com/ClickHouse/ClickHouse/pull/7790) ([Boris Granveaud](https://github.com/bgranvea))
+- Now an exception will be thrown in case of using WITH TIES alongside LIMIT BY. And now it’s possible to use TOP with LIMIT BY. [\#7637](https://github.com/ClickHouse/ClickHouse/pull/7637) ([Nikita Mikhaylov](https://github.com/nikitamikhaylov))
+- Fix dictionary reload if it has `invalidate_query`, which stopped updates and some exception on previous update tries. [\#8029](https://github.com/ClickHouse/ClickHouse/pull/8029) ([alesapin](https://github.com/alesapin))
+
+### ClickHouse release v19.17.4.11, 2019-11-22 {#clickhouse-release-v19-17-4-11-2019-11-22}
+
+#### Backward Incompatible Change {#backward-incompatible-change}
+
+- Using column instead of AST to store scalar subquery results for better performance. Setting `enable_scalar_subquery_optimization` was added in 19.17 and it was enabled by default. It leads to errors like [this](https://github.com/ClickHouse/ClickHouse/issues/7851) during upgrade to 19.17.2 or 19.17.3 from previous versions. This setting was disabled by default in 19.17.4, to make possible upgrading from 19.16 and older versions without errors. [\#7392](https://github.com/ClickHouse/ClickHouse/pull/7392) ([Amos Bird](https://github.com/amosbird))
+
+#### New Feature {#new-feature}
+
+- Add the ability to create dictionaries with DDL queries. [\#7360](https://github.com/ClickHouse/ClickHouse/pull/7360) ([alesapin](https://github.com/alesapin))
+- Make `bloom_filter` type of index supporting `LowCardinality` and `Nullable` [\#7363](https://github.com/ClickHouse/ClickHouse/issues/7363) [\#7561](https://github.com/ClickHouse/ClickHouse/pull/7561) ([Nikolai Kochetov](https://github.com/KochetovNicolai))
+- Add function `isValidJSON` to check that passed string is a valid json. [\#5910](https://github.com/ClickHouse/ClickHouse/issues/5910) [\#7293](https://github.com/ClickHouse/ClickHouse/pull/7293) ([Vdimir](https://github.com/Vdimir))
+- Implement `arrayCompact` function [\#7328](https://github.com/ClickHouse/ClickHouse/pull/7328) ([Memo](https://github.com/Joeywzr))
+- Created function `hex` for Decimal numbers. It works like `hex(reinterpretAsString())`, but doesn’t delete last zero bytes. [\#7355](https://github.com/ClickHouse/ClickHouse/pull/7355) ([Mikhail Korotov](https://github.com/millb))
+- Add `arrayFill` and `arrayReverseFill` functions, which replace elements by other elements in front/back of them in the array. [\#7380](https://github.com/ClickHouse/ClickHouse/pull/7380) ([hcz](https://github.com/hczhcz))
+- Add `CRC32IEEE()`/`CRC64()` support [\#7480](https://github.com/ClickHouse/ClickHouse/pull/7480) ([Azat Khuzhin](https://github.com/azat))
+- Implement `char` function similar to one in [mysql](https://dev.mysql.com/doc/refman/8.0/en/string-functions.html#function_char) [\#7486](https://github.com/ClickHouse/ClickHouse/pull/7486) ([sundyli](https://github.com/sundy-li))
+- Add `bitmapTransform` function. It transforms an array of values in a bitmap to another array of values, the result is a new bitmap [\#7598](https://github.com/ClickHouse/ClickHouse/pull/7598) ([Zhichang Yu](https://github.com/yuzhichang))
+- Implemented `javaHashUTF16LE()` function [\#7651](https://github.com/ClickHouse/ClickHouse/pull/7651) ([achimbab](https://github.com/achimbab))
+- Add `_shard_num` virtual column for the Distributed engine [\#7624](https://github.com/ClickHouse/ClickHouse/pull/7624) ([Azat Khuzhin](https://github.com/azat))
+
+#### Experimental Feature {#experimental-feature}
+
+- Support for processors (new query execution pipeline) in `MergeTree`. [\#7181](https://github.com/ClickHouse/ClickHouse/pull/7181) ([Nikolai Kochetov](https://github.com/KochetovNicolai))
+
+#### Bug Fix {#bug-fix-1}
+
+- Fix incorrect float parsing in `Values` [\#7817](https://github.com/ClickHouse/ClickHouse/issues/7817) [\#7870](https://github.com/ClickHouse/ClickHouse/pull/7870) ([tavplubix](https://github.com/tavplubix))
+- Fix rare deadlock which can happen when trace\_log is enabled. [\#7838](https://github.com/ClickHouse/ClickHouse/pull/7838) ([filimonov](https://github.com/filimonov))
+- Prevent message duplication when producing Kafka table has any MVs selecting from it [\#7265](https://github.com/ClickHouse/ClickHouse/pull/7265) ([Ivan](https://github.com/abyss7))
+- Support for `Array(LowCardinality(Nullable(String)))` in `IN`. Resolves [\#7364](https://github.com/ClickHouse/ClickHouse/issues/7364) [\#7366](https://github.com/ClickHouse/ClickHouse/pull/7366) ([achimbab](https://github.com/achimbab))
+- Add handling of `SQL_TINYINT` and `SQL_BIGINT`, and fix handling of `SQL_FLOAT` data source types in ODBC Bridge. [\#7491](https://github.com/ClickHouse/ClickHouse/pull/7491) ([Denis Glazachev](https://github.com/traceon))
+- Fix aggregation (`avg` and quantiles) over empty decimal columns [\#7431](https://github.com/ClickHouse/ClickHouse/pull/7431) ([Andrey Konyaev](https://github.com/akonyaev90))
+- Fix `INSERT` into Distributed with `MATERIALIZED` columns [\#7377](https://github.com/ClickHouse/ClickHouse/pull/7377) ([Azat Khuzhin](https://github.com/azat))
+- Make `MOVE PARTITION` work if some parts of partition are already on destination disk or volume [\#7434](https://github.com/ClickHouse/ClickHouse/pull/7434) ([Vladimir Chebotarev](https://github.com/excitoon))
+- Fixed bug with hardlinks failing to be created during mutations in `ReplicatedMergeTree` in multi-disk configurations. [\#7558](https://github.com/ClickHouse/ClickHouse/pull/7558) ([Vladimir Chebotarev](https://github.com/excitoon))
+- Fixed a bug with a mutation on a MergeTree when whole part remains unchanged and best space is being found on another disk [\#7602](https://github.com/ClickHouse/ClickHouse/pull/7602) ([Vladimir Chebotarev](https://github.com/excitoon))
+- Fixed bug with `keep_free_space_ratio` not being read from disks configuration [\#7645](https://github.com/ClickHouse/ClickHouse/pull/7645) ([Vladimir Chebotarev](https://github.com/excitoon))
+- Fix bug with table contains only `Tuple` columns or columns with complex paths. Fixes [7541](https://github.com/ClickHouse/ClickHouse/issues/7541). [\#7545](https://github.com/ClickHouse/ClickHouse/pull/7545) ([alesapin](https://github.com/alesapin))
+- Do not account memory for Buffer engine in max\_memory\_usage limit [\#7552](https://github.com/ClickHouse/ClickHouse/pull/7552) ([Azat Khuzhin](https://github.com/azat))
+- Fix final mark usage in `MergeTree` tables ordered by `tuple()`. In rare cases it could lead to `Can't adjust last granule` error while select. [\#7639](https://github.com/ClickHouse/ClickHouse/pull/7639) ([Anton Popov](https://github.com/CurtizJ))
+- Fix bug in mutations that have predicate with actions that require context (for example functions for json), which may lead to crashes or strange exceptions. [\#7664](https://github.com/ClickHouse/ClickHouse/pull/7664) ([alesapin](https://github.com/alesapin))
+- Fix mismatch of database and table names escaping in `data/` and `shadow/` directories [\#7575](https://github.com/ClickHouse/ClickHouse/pull/7575) ([Alexander Burmak](https://github.com/Alex-Burmak))
+- Support duplicated keys in RIGHT\|FULL JOINs, e.g. `ON t.x = u.x AND t.x = u.y`. Fix crash in this case. [\#7586](https://github.com/ClickHouse/ClickHouse/pull/7586) ([Artem Zuikov](https://github.com/4ertus2))
+- Fix `Not found column in block` when joining on expression with RIGHT or FULL JOIN. [\#7641](https://github.com/ClickHouse/ClickHouse/pull/7641) ([Artem Zuikov](https://github.com/4ertus2))
+- One more attempt to fix infinite loop in `PrettySpace` format [\#7591](https://github.com/ClickHouse/ClickHouse/pull/7591) ([Olga Khvostikova](https://github.com/stavrolia))
+- Fix bug in `concat` function when all arguments were `FixedString` of the same size. [\#7635](https://github.com/ClickHouse/ClickHouse/pull/7635) ([alesapin](https://github.com/alesapin))
+- Fixed exception in case of using 1 argument while defining S3, URL and HDFS storages. [\#7618](https://github.com/ClickHouse/ClickHouse/pull/7618) ([Vladimir Chebotarev](https://github.com/excitoon))
+- Fix scope of the InterpreterSelectQuery for views with query [\#7601](https://github.com/ClickHouse/ClickHouse/pull/7601) ([Azat Khuzhin](https://github.com/azat))
+
+#### Improvement {#improvement}
+
+- `Nullable` columns recognized and NULL-values handled correctly by ODBC-bridge [\#7402](https://github.com/ClickHouse/ClickHouse/pull/7402) ([Vasily Nemkov](https://github.com/Enmk))
+- Write current batch for distributed send atomically [\#7600](https://github.com/ClickHouse/ClickHouse/pull/7600) ([Azat Khuzhin](https://github.com/azat))
+- Throw an exception if we cannot detect table for column name in query. [\#7358](https://github.com/ClickHouse/ClickHouse/pull/7358) ([Artem Zuikov](https://github.com/4ertus2))
+- Add `merge_max_block_size` setting to `MergeTreeSettings` [\#7412](https://github.com/ClickHouse/ClickHouse/pull/7412) ([Artem Zuikov](https://github.com/4ertus2))
+- Queries with `HAVING` and without `GROUP BY` assume group by constant. So, `SELECT 1 HAVING 1` now returns a result. [\#7496](https://github.com/ClickHouse/ClickHouse/pull/7496) ([Amos Bird](https://github.com/amosbird))
+- Support parsing `(X,)` as tuple similar to python. [\#7501](https://github.com/ClickHouse/ClickHouse/pull/7501), [\#7562](https://github.com/ClickHouse/ClickHouse/pull/7562) ([Amos Bird](https://github.com/amosbird))
+- Make `range` function behaviors almost like pythonic one. [\#7518](https://github.com/ClickHouse/ClickHouse/pull/7518) ([sundyli](https://github.com/sundy-li))
+- Add `constraints` columns to table `system.settings` [\#7553](https://github.com/ClickHouse/ClickHouse/pull/7553) ([Vitaly Baranov](https://github.com/vitlibar))
+- Better Null format for tcp handler, so that it’s possible to use `select ignore() from table format Null` for perf measure via clickhouse-client [\#7606](https://github.com/ClickHouse/ClickHouse/pull/7606) ([Amos Bird](https://github.com/amosbird))
+- Queries like `CREATE TABLE ... AS (SELECT (1, 2))` are parsed correctly [\#7542](https://github.com/ClickHouse/ClickHouse/pull/7542) ([hcz](https://github.com/hczhcz))
+
+#### Performance Improvement {#performance-improvement}
+
+- The performance of aggregation over short string keys is improved. [\#6243](https://github.com/ClickHouse/ClickHouse/pull/6243) ([Alexander Kuzmenkov](https://github.com/akuzm), [Amos Bird](https://github.com/amosbird))
+- Run another pass of syntax/expression analysis to get potential optimizations after constant predicates are folded. [\#7497](https://github.com/ClickHouse/ClickHouse/pull/7497) ([Amos Bird](https://github.com/amosbird))
+- Use storage meta info to evaluate trivial `SELECT count() FROM table;` [\#7510](https://github.com/ClickHouse/ClickHouse/pull/7510) ([Amos Bird](https://github.com/amosbird), [alexey-milovidov](https://github.com/alexey-milovidov))
+- Vectorize processing `arrayReduce` similar to Aggregator `addBatch`. [\#7608](https://github.com/ClickHouse/ClickHouse/pull/7608) ([Amos Bird](https://github.com/amosbird))
+- Minor improvements in performance of `Kafka` consumption [\#7475](https://github.com/ClickHouse/ClickHouse/pull/7475) ([Ivan](https://github.com/abyss7))
+
+#### Build/Testing/Packaging Improvement {#buildtestingpackaging-improvement}
+
+- Add support for cross-compiling to the CPU architecture AARCH64. Refactor packager script. [\#7370](https://github.com/ClickHouse/ClickHouse/pull/7370) [\#7539](https://github.com/ClickHouse/ClickHouse/pull/7539) ([Ivan](https://github.com/abyss7))
+- Unpack darwin-x86\_64 and linux-aarch64 toolchains into mounted Docker volume when building packages [\#7534](https://github.com/ClickHouse/ClickHouse/pull/7534) ([Ivan](https://github.com/abyss7))
+- Update Docker Image for Binary Packager [\#7474](https://github.com/ClickHouse/ClickHouse/pull/7474) ([Ivan](https://github.com/abyss7))
+- Fixed compile errors on MacOS Catalina [\#7585](https://github.com/ClickHouse/ClickHouse/pull/7585) ([Ernest Poletaev](https://github.com/ernestp))
+- Some refactoring in query analysis logic: split complex class into several simple ones. [\#7454](https://github.com/ClickHouse/ClickHouse/pull/7454) ([Artem Zuikov](https://github.com/4ertus2))
+- Fix build without submodules [\#7295](https://github.com/ClickHouse/ClickHouse/pull/7295) ([proller](https://github.com/proller))
+- Better `add_globs` in CMake files [\#7418](https://github.com/ClickHouse/ClickHouse/pull/7418) ([Amos Bird](https://github.com/amosbird))
+- Remove hardcoded paths in `unwind` target [\#7460](https://github.com/ClickHouse/ClickHouse/pull/7460) ([Konstantin Podshumok](https://github.com/podshumok))
+- Allow to use mysql format without ssl [\#7524](https://github.com/ClickHouse/ClickHouse/pull/7524) ([proller](https://github.com/proller))
+
+#### Other {#other}
+
+- Added ANTLR4 grammar for ClickHouse SQL dialect [\#7595](https://github.com/ClickHouse/ClickHouse/issues/7595) [\#7596](https://github.com/ClickHouse/ClickHouse/pull/7596) ([alexey-milovidov](https://github.com/alexey-milovidov))
+
+## ClickHouse release v19.16 {#clickhouse-release-v19-16}
+
+#### Clickhouse release v19.16.14.65, 2020-03-25
+
+* Fixed up a bug in batched calculations of ternary logical OPs on multiple arguments (more than 10). [#8718](https://github.com/ClickHouse/ClickHouse/pull/8718) ([Alexander Kazakov](https://github.com/Akazz)) This bugfix was backported to version 19.16 by a special request from Altinity.
+
+#### Clickhouse release v19.16.14.65, 2020-03-05 {#clickhouse-release-v19-16-14-65-2020-03-05}
+
+- Fix distributed subqueries incompatibility with older CH versions. Fixes [\#7851](https://github.com/ClickHouse/ClickHouse/issues/7851)
+ [(tabplubix)](https://github.com/tavplubix)
+- When executing `CREATE` query, fold constant expressions in storage engine arguments. Replace empty database name with current database. Fixes [\#6508](https://github.com/ClickHouse/ClickHouse/issues/6508), [\#3492](https://github.com/ClickHouse/ClickHouse/issues/3492). Also fix check for local address in `ClickHouseDictionarySource`.
+ [\#9262](https://github.com/ClickHouse/ClickHouse/pull/9262) [(tabplubix)](https://github.com/tavplubix)
+- Now background merges in `*MergeTree` table engines family preserve storage policy volume order more accurately.
+ [\#8549](https://github.com/ClickHouse/ClickHouse/pull/8549) ([Vladimir Chebotarev](https://github.com/excitoon))
+- Prevent losing data in `Kafka` in rare cases when exception happens after reading suffix but before commit. Fixes [\#9378](https://github.com/ClickHouse/ClickHouse/issues/9378). Related: [\#7175](https://github.com/ClickHouse/ClickHouse/issues/7175)
+ [\#9507](https://github.com/ClickHouse/ClickHouse/pull/9507) [(filimonov)](https://github.com/filimonov)
+- Fix bug leading to server termination when trying to use / drop `Kafka` table created with wrong parameters. Fixes [\#9494](https://github.com/ClickHouse/ClickHouse/issues/9494). Incorporates [\#9507](https://github.com/ClickHouse/ClickHouse/issues/9507).
+ [\#9513](https://github.com/ClickHouse/ClickHouse/pull/9513) [(filimonov)](https://github.com/filimonov)
+- Allow using `MaterializedView` with subqueries above `Kafka` tables.
+ [\#8197](https://github.com/ClickHouse/ClickHouse/pull/8197) ([filimonov](https://github.com/filimonov))
+
+#### New Feature {#new-feature-1}
+
+- Add `deduplicate_blocks_in_dependent_materialized_views` option to control the behaviour of idempotent inserts into tables with materialized views. This new feature was added to the bugfix release by a special request from Altinity.
+ [\#9070](https://github.com/ClickHouse/ClickHouse/pull/9070) [(urykhy)](https://github.com/urykhy)
+
+### ClickHouse release v19.16.2.2, 2019-10-30 {#clickhouse-release-v19-16-2-2-2019-10-30}
+
+#### Backward Incompatible Change {#backward-incompatible-change-1}
+
+- Add missing arity validation for count/counIf.
+ [\#7095](https://github.com/ClickHouse/ClickHouse/issues/7095)
+ [\#7298](https://github.com/ClickHouse/ClickHouse/pull/7298) ([Vdimir](https://github.com/Vdimir))
+- Remove legacy `asterisk_left_columns_only` setting (it was disabled by default).
+ [\#7335](https://github.com/ClickHouse/ClickHouse/pull/7335) ([Artem
+ Zuikov](https://github.com/4ertus2))
+- Format strings for Template data format are now specified in files.
+ [\#7118](https://github.com/ClickHouse/ClickHouse/pull/7118)
+ ([tavplubix](https://github.com/tavplubix))
+
+#### New Feature {#new-feature-2}
+
+- Introduce uniqCombined64() to calculate cardinality greater than UINT\_MAX.
+ [\#7213](https://github.com/ClickHouse/ClickHouse/pull/7213),
+ [\#7222](https://github.com/ClickHouse/ClickHouse/pull/7222) ([Azat
+ Khuzhin](https://github.com/azat))
+- Support Bloom filter indexes on Array columns.
+ [\#6984](https://github.com/ClickHouse/ClickHouse/pull/6984)
+ ([achimbab](https://github.com/achimbab))
+- Add a function `getMacro(name)` that returns String with the value of corresponding ``
+ from server configuration. [\#7240](https://github.com/ClickHouse/ClickHouse/pull/7240)
+ ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Set two configuration options for a dictionary based on an HTTP source: `credentials` and
+ `http-headers`. [\#7092](https://github.com/ClickHouse/ClickHouse/pull/7092) ([Guillaume
+ Tassery](https://github.com/YiuRULE))
+- Add a new ProfileEvent `Merge` that counts the number of launched background merges.
+ [\#7093](https://github.com/ClickHouse/ClickHouse/pull/7093) ([Mikhail
+ Korotov](https://github.com/millb))
+- Add fullHostName function that returns a fully qualified domain name.
+ [\#7263](https://github.com/ClickHouse/ClickHouse/issues/7263)
+ [\#7291](https://github.com/ClickHouse/ClickHouse/pull/7291) ([sundyli](https://github.com/sundy-li))
+- Add function `arraySplit` and `arrayReverseSplit` which split an array by “cut off”
+ conditions. They are useful in time sequence handling.
+ [\#7294](https://github.com/ClickHouse/ClickHouse/pull/7294) ([hcz](https://github.com/hczhcz))
+- Add new functions that return the Array of all matched indices in multiMatch family of functions.
+ [\#7299](https://github.com/ClickHouse/ClickHouse/pull/7299) ([Danila
+ Kutenin](https://github.com/danlark1))
+- Add a new database engine `Lazy` that is optimized for storing a large number of small -Log
+ tables. [\#7171](https://github.com/ClickHouse/ClickHouse/pull/7171) ([Nikita
+ Vasilev](https://github.com/nikvas0))
+- Add aggregate functions groupBitmapAnd, -Or, -Xor for bitmap columns. [\#7109](https://github.com/ClickHouse/ClickHouse/pull/7109) ([Zhichang
+ Yu](https://github.com/yuzhichang))
+- Add aggregate function combinators -OrNull and -OrDefault, which return null
+ or default values when there is nothing to aggregate.
+ [\#7331](https://github.com/ClickHouse/ClickHouse/pull/7331)
+ ([hcz](https://github.com/hczhcz))
+- Introduce CustomSeparated data format that supports custom escaping and
+ delimiter rules. [\#7118](https://github.com/ClickHouse/ClickHouse/pull/7118)
+ ([tavplubix](https://github.com/tavplubix))
+- Support Redis as source of external dictionary. [\#4361](https://github.com/ClickHouse/ClickHouse/pull/4361) [\#6962](https://github.com/ClickHouse/ClickHouse/pull/6962) ([comunodi](https://github.com/comunodi), [Anton
+ Popov](https://github.com/CurtizJ))
+
+#### Bug Fix {#bug-fix-2}
+
+- Fix wrong query result if it has `WHERE IN (SELECT ...)` section and `optimize_read_in_order` is
+ used. [\#7371](https://github.com/ClickHouse/ClickHouse/pull/7371) ([Anton
+ Popov](https://github.com/CurtizJ))
+- Disabled MariaDB authentication plugin, which depends on files outside of project.
+ [\#7140](https://github.com/ClickHouse/ClickHouse/pull/7140) ([Yuriy
+ Baranov](https://github.com/yurriy))
+- Fix exception `Cannot convert column ... because it is constant but values of constants are different in source and result` which could rarely happen when functions `now()`, `today()`,
+ `yesterday()`, `randConstant()` are used.
+ [\#7156](https://github.com/ClickHouse/ClickHouse/pull/7156) ([Nikolai
+ Kochetov](https://github.com/KochetovNicolai))
+- Fixed issue of using HTTP keep alive timeout instead of TCP keep alive timeout.
+ [\#7351](https://github.com/ClickHouse/ClickHouse/pull/7351) ([Vasily
+ Nemkov](https://github.com/Enmk))
+- Fixed a segmentation fault in groupBitmapOr (issue [\#7109](https://github.com/ClickHouse/ClickHouse/issues/7109)).
+ [\#7289](https://github.com/ClickHouse/ClickHouse/pull/7289) ([Zhichang
+ Yu](https://github.com/yuzhichang))
+- For materialized views the commit for Kafka is called after all data were written.
+ [\#7175](https://github.com/ClickHouse/ClickHouse/pull/7175) ([Ivan](https://github.com/abyss7))
+- Fixed wrong `duration_ms` value in `system.part_log` table. It was ten times off.
+ [\#7172](https://github.com/ClickHouse/ClickHouse/pull/7172) ([Vladimir
+ Chebotarev](https://github.com/excitoon))
+- A quick fix to resolve crash in LIVE VIEW table and re-enabling all LIVE VIEW tests.
+ [\#7201](https://github.com/ClickHouse/ClickHouse/pull/7201)
+ ([vzakaznikov](https://github.com/vzakaznikov))
+- Serialize NULL values correctly in min/max indexes of MergeTree parts.
+ [\#7234](https://github.com/ClickHouse/ClickHouse/pull/7234) ([Alexander
+ Kuzmenkov](https://github.com/akuzm))
+- Don’t put virtual columns to .sql metadata when table is created as `CREATE TABLE AS`.
+ [\#7183](https://github.com/ClickHouse/ClickHouse/pull/7183) ([Ivan](https://github.com/abyss7))
+- Fix segmentation fault in `ATTACH PART` query.
+ [\#7185](https://github.com/ClickHouse/ClickHouse/pull/7185)
+ ([alesapin](https://github.com/alesapin))
+- Fix wrong result for some queries given by the optimization of empty IN subqueries and empty
+ INNER/RIGHT JOIN. [\#7284](https://github.com/ClickHouse/ClickHouse/pull/7284) ([Nikolai
+ Kochetov](https://github.com/KochetovNicolai))
+- Fixing AddressSanitizer error in the LIVE VIEW getHeader() method.
+ [\#7271](https://github.com/ClickHouse/ClickHouse/pull/7271)
+ ([vzakaznikov](https://github.com/vzakaznikov))
+
+#### Improvement {#improvement-1}
+
+- Add a message in case of queue\_wait\_max\_ms wait takes place.
+ [\#7390](https://github.com/ClickHouse/ClickHouse/pull/7390) ([Azat
+ Khuzhin](https://github.com/azat))
+- Made setting `s3_min_upload_part_size` table-level.
+ [\#7059](https://github.com/ClickHouse/ClickHouse/pull/7059) ([Vladimir
+ Chebotarev](https://github.com/excitoon))
+- Check TTL in StorageFactory. [\#7304](https://github.com/ClickHouse/ClickHouse/pull/7304)
+ ([sundyli](https://github.com/sundy-li))
+- Squash left-hand blocks in partial merge join (optimization).
+ [\#7122](https://github.com/ClickHouse/ClickHouse/pull/7122) ([Artem
+ Zuikov](https://github.com/4ertus2))
+- Do not allow non-deterministic functions in mutations of Replicated table engines, because this
+ can introduce inconsistencies between replicas.
+ [\#7247](https://github.com/ClickHouse/ClickHouse/pull/7247) ([Alexander
+ Kazakov](https://github.com/Akazz))
+- Disable memory tracker while converting exception stack trace to string. It can prevent the loss
+ of error messages of type `Memory limit exceeded` on server, which caused the `Attempt to read after eof` exception on client. [\#7264](https://github.com/ClickHouse/ClickHouse/pull/7264)
+ ([Nikolai Kochetov](https://github.com/KochetovNicolai))
+- Miscellaneous format improvements. Resolves
+ [\#6033](https://github.com/ClickHouse/ClickHouse/issues/6033),
+ [\#2633](https://github.com/ClickHouse/ClickHouse/issues/2633),
+ [\#6611](https://github.com/ClickHouse/ClickHouse/issues/6611),
+ [\#6742](https://github.com/ClickHouse/ClickHouse/issues/6742)
+ [\#7215](https://github.com/ClickHouse/ClickHouse/pull/7215)
+ ([tavplubix](https://github.com/tavplubix))
+- ClickHouse ignores values on the right side of IN operator that are not convertible to the left
+ side type. Make it work properly for compound types – Array and Tuple.
+ [\#7283](https://github.com/ClickHouse/ClickHouse/pull/7283) ([Alexander
+ Kuzmenkov](https://github.com/akuzm))
+- Support missing inequalities for ASOF JOIN. It’s possible to join less-or-equal variant and strict
+ greater and less variants for ASOF column in ON syntax.
+ [\#7282](https://github.com/ClickHouse/ClickHouse/pull/7282) ([Artem
+ Zuikov](https://github.com/4ertus2))
+- Optimize partial merge join. [\#7070](https://github.com/ClickHouse/ClickHouse/pull/7070)
+ ([Artem Zuikov](https://github.com/4ertus2))
+- Do not use more than 98K of memory in uniqCombined functions.
+ [\#7236](https://github.com/ClickHouse/ClickHouse/pull/7236),
+ [\#7270](https://github.com/ClickHouse/ClickHouse/pull/7270) ([Azat
+ Khuzhin](https://github.com/azat))
+- Flush parts of right-hand joining table on disk in PartialMergeJoin (if there is not enough
+ memory). Load data back when needed. [\#7186](https://github.com/ClickHouse/ClickHouse/pull/7186)
+ ([Artem Zuikov](https://github.com/4ertus2))
+
+#### Performance Improvement {#performance-improvement-1}
+
+- Speed up joinGet with const arguments by avoiding data duplication.
+ [\#7359](https://github.com/ClickHouse/ClickHouse/pull/7359) ([Amos
+ Bird](https://github.com/amosbird))
+- Return early if the subquery is empty.
+ [\#7007](https://github.com/ClickHouse/ClickHouse/pull/7007) ([小路](https://github.com/nicelulu))
+- Optimize parsing of SQL expression in Values.
+ [\#6781](https://github.com/ClickHouse/ClickHouse/pull/6781)
+ ([tavplubix](https://github.com/tavplubix))
+
+#### Build/Testing/Packaging Improvement {#buildtestingpackaging-improvement-1}
+
+- Disable some contribs for cross-compilation to Mac OS.
+ [\#7101](https://github.com/ClickHouse/ClickHouse/pull/7101) ([Ivan](https://github.com/abyss7))
+- Add missing linking with PocoXML for clickhouse\_common\_io.
+ [\#7200](https://github.com/ClickHouse/ClickHouse/pull/7200) ([Azat
+ Khuzhin](https://github.com/azat))
+- Accept multiple test filter arguments in clickhouse-test.
+ [\#7226](https://github.com/ClickHouse/ClickHouse/pull/7226) ([Alexander
+ Kuzmenkov](https://github.com/akuzm))
+- Enable musl and jemalloc for ARM. [\#7300](https://github.com/ClickHouse/ClickHouse/pull/7300)
+ ([Amos Bird](https://github.com/amosbird))
+- Added `--client-option` parameter to `clickhouse-test` to pass additional parameters to client.
+ [\#7277](https://github.com/ClickHouse/ClickHouse/pull/7277) ([Nikolai
+ Kochetov](https://github.com/KochetovNicolai))
+- Preserve existing configs on rpm package upgrade.
+ [\#7103](https://github.com/ClickHouse/ClickHouse/pull/7103)
+ ([filimonov](https://github.com/filimonov))
+- Fix errors detected by PVS. [\#7153](https://github.com/ClickHouse/ClickHouse/pull/7153) ([Artem
+ Zuikov](https://github.com/4ertus2))
+- Fix build for Darwin. [\#7149](https://github.com/ClickHouse/ClickHouse/pull/7149)
+ ([Ivan](https://github.com/abyss7))
+- glibc 2.29 compatibility. [\#7142](https://github.com/ClickHouse/ClickHouse/pull/7142) ([Amos
+ Bird](https://github.com/amosbird))
+- Make sure dh\_clean does not touch potential source files.
+ [\#7205](https://github.com/ClickHouse/ClickHouse/pull/7205) ([Amos
+ Bird](https://github.com/amosbird))
+- Attempt to avoid conflict when updating from altinity rpm - it has config file packaged separately
+ in clickhouse-server-common. [\#7073](https://github.com/ClickHouse/ClickHouse/pull/7073)
+ ([filimonov](https://github.com/filimonov))
+- Optimize some header files for faster rebuilds.
+ [\#7212](https://github.com/ClickHouse/ClickHouse/pull/7212),
+ [\#7231](https://github.com/ClickHouse/ClickHouse/pull/7231) ([Alexander
+ Kuzmenkov](https://github.com/akuzm))
+- Add performance tests for Date and DateTime. [\#7332](https://github.com/ClickHouse/ClickHouse/pull/7332) ([Vasily
+ Nemkov](https://github.com/Enmk))
+- Fix some tests that contained non-deterministic mutations.
+ [\#7132](https://github.com/ClickHouse/ClickHouse/pull/7132) ([Alexander
+ Kazakov](https://github.com/Akazz))
+- Add build with MemorySanitizer to CI. [\#7066](https://github.com/ClickHouse/ClickHouse/pull/7066)
+ ([Alexander Kuzmenkov](https://github.com/akuzm))
+- Avoid use of uninitialized values in MetricsTransmitter.
+ [\#7158](https://github.com/ClickHouse/ClickHouse/pull/7158) ([Azat
+ Khuzhin](https://github.com/azat))
+- Fix some issues in Fields found by MemorySanitizer.
+ [\#7135](https://github.com/ClickHouse/ClickHouse/pull/7135),
+ [\#7179](https://github.com/ClickHouse/ClickHouse/pull/7179) ([Alexander
+ Kuzmenkov](https://github.com/akuzm)), [\#7376](https://github.com/ClickHouse/ClickHouse/pull/7376)
+ ([Amos Bird](https://github.com/amosbird))
+- Fix undefined behavior in murmurhash32. [\#7388](https://github.com/ClickHouse/ClickHouse/pull/7388) ([Amos
+ Bird](https://github.com/amosbird))
+- Fix undefined behavior in StoragesInfoStream. [\#7384](https://github.com/ClickHouse/ClickHouse/pull/7384)
+ ([tavplubix](https://github.com/tavplubix))
+- Fixed constant expressions folding for external database engines (MySQL, ODBC, JDBC). In previous
+ versions it wasn’t working for multiple constant expressions and was not working at all for Date,
+ DateTime and UUID. This fixes [\#7245](https://github.com/ClickHouse/ClickHouse/issues/7245)
+ [\#7252](https://github.com/ClickHouse/ClickHouse/pull/7252)
+ ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Fixing ThreadSanitizer data race error in the LIVE VIEW when accessing no\_users\_thread variable.
+ [\#7353](https://github.com/ClickHouse/ClickHouse/pull/7353)
+ ([vzakaznikov](https://github.com/vzakaznikov))
+- Get rid of malloc symbols in libcommon
+ [\#7134](https://github.com/ClickHouse/ClickHouse/pull/7134),
+ [\#7065](https://github.com/ClickHouse/ClickHouse/pull/7065) ([Amos
+ Bird](https://github.com/amosbird))
+- Add global flag ENABLE\_LIBRARIES for disabling all libraries.
+ [\#7063](https://github.com/ClickHouse/ClickHouse/pull/7063)
+ ([proller](https://github.com/proller))
+
+#### Code cleanup {#code-cleanup}
+
+- Generalize configuration repository to prepare for DDL for Dictionaries. [\#7155](https://github.com/ClickHouse/ClickHouse/pull/7155)
+ ([alesapin](https://github.com/alesapin))
+- Parser for dictionaries DDL without any semantic.
+ [\#7209](https://github.com/ClickHouse/ClickHouse/pull/7209)
+ ([alesapin](https://github.com/alesapin))
+- Split ParserCreateQuery into different smaller parsers.
+ [\#7253](https://github.com/ClickHouse/ClickHouse/pull/7253)
+ ([alesapin](https://github.com/alesapin))
+- Small refactoring and renaming near external dictionaries.
+ [\#7111](https://github.com/ClickHouse/ClickHouse/pull/7111)
+ ([alesapin](https://github.com/alesapin))
+- Refactor some code to prepare for role-based access control. [\#7235](https://github.com/ClickHouse/ClickHouse/pull/7235) ([Vitaly
+ Baranov](https://github.com/vitlibar))
+- Some improvements in DatabaseOrdinary code.
+ [\#7086](https://github.com/ClickHouse/ClickHouse/pull/7086) ([Nikita
+ Vasilev](https://github.com/nikvas0))
+- Do not use iterators in find() and emplace() methods of hash tables.
+ [\#7026](https://github.com/ClickHouse/ClickHouse/pull/7026) ([Alexander
+ Kuzmenkov](https://github.com/akuzm))
+- Fix getMultipleValuesFromConfig in case when parameter root is not empty. [\#7374](https://github.com/ClickHouse/ClickHouse/pull/7374)
+ ([Mikhail Korotov](https://github.com/millb))
+- Remove some copy-paste (TemporaryFile and TemporaryFileStream)
+ [\#7166](https://github.com/ClickHouse/ClickHouse/pull/7166) ([Artem
+ Zuikov](https://github.com/4ertus2))
+- Improved code readability a little bit (`MergeTreeData::getActiveContainingPart`).
+ [\#7361](https://github.com/ClickHouse/ClickHouse/pull/7361) ([Vladimir
+ Chebotarev](https://github.com/excitoon))
+- Wait for all scheduled jobs, which are using local objects, if `ThreadPool::schedule(...)` throws
+ an exception. Rename `ThreadPool::schedule(...)` to `ThreadPool::scheduleOrThrowOnError(...)` and
+ fix comments to make obvious that it may throw.
+ [\#7350](https://github.com/ClickHouse/ClickHouse/pull/7350)
+ ([tavplubix](https://github.com/tavplubix))
+
+## ClickHouse release 19.15 {#clickhouse-release-19-15}
+
+### ClickHouse release 19.15.4.10, 2019-10-31 {#clickhouse-release-19-15-4-10-2019-10-31}
+
+#### Bug Fix {#bug-fix-3}
+
+- Added handling of SQL\_TINYINT and SQL\_BIGINT, and fix handling of SQL\_FLOAT data source types in ODBC Bridge.
+ [\#7491](https://github.com/ClickHouse/ClickHouse/pull/7491) ([Denis Glazachev](https://github.com/traceon))
+- Allowed to have some parts on destination disk or volume in MOVE PARTITION.
+ [\#7434](https://github.com/ClickHouse/ClickHouse/pull/7434) ([Vladimir Chebotarev](https://github.com/excitoon))
+- Fixed NULL-values in nullable columns through ODBC-bridge.
+ [\#7402](https://github.com/ClickHouse/ClickHouse/pull/7402) ([Vasily Nemkov](https://github.com/Enmk))
+- Fixed INSERT into Distributed non local node with MATERIALIZED columns.
+ [\#7377](https://github.com/ClickHouse/ClickHouse/pull/7377) ([Azat Khuzhin](https://github.com/azat))
+- Fixed function getMultipleValuesFromConfig.
+ [\#7374](https://github.com/ClickHouse/ClickHouse/pull/7374) ([Mikhail Korotov](https://github.com/millb))
+- Fixed issue of using HTTP keep alive timeout instead of TCP keep alive timeout.
+ [\#7351](https://github.com/ClickHouse/ClickHouse/pull/7351) ([Vasily Nemkov](https://github.com/Enmk))
+- Wait for all jobs to finish on exception (fixes rare segfaults).
+ [\#7350](https://github.com/ClickHouse/ClickHouse/pull/7350) ([tavplubix](https://github.com/tavplubix))
+- Don’t push to MVs when inserting into Kafka table.
+ [\#7265](https://github.com/ClickHouse/ClickHouse/pull/7265) ([Ivan](https://github.com/abyss7))
+- Disable memory tracker for exception stack.
+ [\#7264](https://github.com/ClickHouse/ClickHouse/pull/7264) ([Nikolai Kochetov](https://github.com/KochetovNicolai))
+- Fixed bad code in transforming query for external database.
+ [\#7252](https://github.com/ClickHouse/ClickHouse/pull/7252) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Avoid use of uninitialized values in MetricsTransmitter.
+ [\#7158](https://github.com/ClickHouse/ClickHouse/pull/7158) ([Azat Khuzhin](https://github.com/azat))
+- Added example config with macros for tests ([alexey-milovidov](https://github.com/alexey-milovidov))
+
+### ClickHouse release 19.15.3.6, 2019-10-09 {#clickhouse-release-19-15-3-6-2019-10-09}
+
+#### Bug Fix {#bug-fix-4}
+
+- Fixed bad\_variant in hashed dictionary.
+ ([alesapin](https://github.com/alesapin))
+- Fixed up bug with segmentation fault in ATTACH PART query.
+ ([alesapin](https://github.com/alesapin))
+- Fixed time calculation in `MergeTreeData`.
+ ([Vladimir Chebotarev](https://github.com/excitoon))
+- Commit to Kafka explicitly after the writing is finalized.
+ [\#7175](https://github.com/ClickHouse/ClickHouse/pull/7175) ([Ivan](https://github.com/abyss7))
+- Serialize NULL values correctly in min/max indexes of MergeTree parts.
+ [\#7234](https://github.com/ClickHouse/ClickHouse/pull/7234) ([Alexander Kuzmenkov](https://github.com/akuzm))
+
+### ClickHouse release 19.15.2.2, 2019-10-01 {#clickhouse-release-19-15-2-2-2019-10-01}
+
+#### New Feature {#new-feature-3}
+
+- Tiered storage: support to use multiple storage volumes for tables with MergeTree engine. It’s possible to store fresh data on SSD and automatically move old data to HDD. ([example](https://clickhouse.github.io/clickhouse-presentations/meetup30/new_features/#12)). [\#4918](https://github.com/ClickHouse/ClickHouse/pull/4918) ([Igr](https://github.com/ObjatieGroba)) [\#6489](https://github.com/ClickHouse/ClickHouse/pull/6489) ([alesapin](https://github.com/alesapin))
+- Add table function `input` for reading incoming data in `INSERT SELECT` query. [\#5450](https://github.com/ClickHouse/ClickHouse/pull/5450) ([palasonic1](https://github.com/palasonic1)) [\#6832](https://github.com/ClickHouse/ClickHouse/pull/6832) ([Anton Popov](https://github.com/CurtizJ))
+- Add a `sparse_hashed` dictionary layout, that is functionally equivalent to the `hashed` layout, but is more memory efficient. It uses about twice as less memory at the cost of slower value retrieval. [\#6894](https://github.com/ClickHouse/ClickHouse/pull/6894) ([Azat Khuzhin](https://github.com/azat))
+- Implement ability to define list of users for access to dictionaries. Only current connected database using. [\#6907](https://github.com/ClickHouse/ClickHouse/pull/6907) ([Guillaume Tassery](https://github.com/YiuRULE))
+- Add `LIMIT` option to `SHOW` query. [\#6944](https://github.com/ClickHouse/ClickHouse/pull/6944) ([Philipp Malkovsky](https://github.com/malkfilipp))
+- Add `bitmapSubsetLimit(bitmap, range_start, limit)` function, that returns subset of the smallest `limit` values in set that is no smaller than `range_start`. [\#6957](https://github.com/ClickHouse/ClickHouse/pull/6957) ([Zhichang Yu](https://github.com/yuzhichang))
+- Add `bitmapMin` and `bitmapMax` functions. [\#6970](https://github.com/ClickHouse/ClickHouse/pull/6970) ([Zhichang Yu](https://github.com/yuzhichang))
+- Add function `repeat` related to [issue-6648](https://github.com/ClickHouse/ClickHouse/issues/6648) [\#6999](https://github.com/ClickHouse/ClickHouse/pull/6999) ([flynn](https://github.com/ucasFL))
+
+#### Experimental Feature {#experimental-feature-1}
+
+- Implement (in memory) Merge Join variant that does not change current pipeline. Result is partially sorted by merge key. Set `partial_merge_join = 1` to use this feature. The Merge Join is still in development. [\#6940](https://github.com/ClickHouse/ClickHouse/pull/6940) ([Artem Zuikov](https://github.com/4ertus2))
+- Add `S3` engine and table function. It is still in development (no authentication support yet). [\#5596](https://github.com/ClickHouse/ClickHouse/pull/5596) ([Vladimir Chebotarev](https://github.com/excitoon))
+
+#### Improvement {#improvement-2}
+
+- Every message read from Kafka is inserted atomically. This resolves almost all known issues with Kafka engine. [\#6950](https://github.com/ClickHouse/ClickHouse/pull/6950) ([Ivan](https://github.com/abyss7))
+- Improvements for failover of Distributed queries. Shorten recovery time, also it is now configurable and can be seen in `system.clusters`. [\#6399](https://github.com/ClickHouse/ClickHouse/pull/6399) ([Vasily Nemkov](https://github.com/Enmk))
+- Support numeric values for Enums directly in `IN` section. \#6766 [\#6941](https://github.com/ClickHouse/ClickHouse/pull/6941) ([dimarub2000](https://github.com/dimarub2000))
+- Support (optional, disabled by default) redirects on URL storage. [\#6914](https://github.com/ClickHouse/ClickHouse/pull/6914) ([maqroll](https://github.com/maqroll))
+- Add information message when client with an older version connects to a server. [\#6893](https://github.com/ClickHouse/ClickHouse/pull/6893) ([Philipp Malkovsky](https://github.com/malkfilipp))
+- Remove maximum backoff sleep time limit for sending data in Distributed tables [\#6895](https://github.com/ClickHouse/ClickHouse/pull/6895) ([Azat Khuzhin](https://github.com/azat))
+- Add ability to send profile events (counters) with cumulative values to graphite. It can be enabled under `` in server `config.xml`. [\#6969](https://github.com/ClickHouse/ClickHouse/pull/6969) ([Azat Khuzhin](https://github.com/azat))
+- Add automatically cast type `T` to `LowCardinality(T)` while inserting data in column of type `LowCardinality(T)` in Native format via HTTP. [\#6891](https://github.com/ClickHouse/ClickHouse/pull/6891) ([Nikolai Kochetov](https://github.com/KochetovNicolai))
+- Add ability to use function `hex` without using `reinterpretAsString` for `Float32`, `Float64`. [\#7024](https://github.com/ClickHouse/ClickHouse/pull/7024) ([Mikhail Korotov](https://github.com/millb))
+
+#### Build/Testing/Packaging Improvement {#buildtestingpackaging-improvement-2}
+
+- Add gdb-index to clickhouse binary with debug info. It will speed up startup time of `gdb`. [\#6947](https://github.com/ClickHouse/ClickHouse/pull/6947) ([alesapin](https://github.com/alesapin))
+- Speed up deb packaging with patched dpkg-deb which uses `pigz`. [\#6960](https://github.com/ClickHouse/ClickHouse/pull/6960) ([alesapin](https://github.com/alesapin))
+- Set `enable_fuzzing = 1` to enable libfuzzer instrumentation of all the project code. [\#7042](https://github.com/ClickHouse/ClickHouse/pull/7042) ([kyprizel](https://github.com/kyprizel))
+- Add split build smoke test in CI. [\#7061](https://github.com/ClickHouse/ClickHouse/pull/7061) ([alesapin](https://github.com/alesapin))
+- Add build with MemorySanitizer to CI. [\#7066](https://github.com/ClickHouse/ClickHouse/pull/7066) ([Alexander Kuzmenkov](https://github.com/akuzm))
+- Replace `libsparsehash` with `sparsehash-c11` [\#6965](https://github.com/ClickHouse/ClickHouse/pull/6965) ([Azat Khuzhin](https://github.com/azat))
+
+#### Bug Fix {#bug-fix-5}
+
+- Fixed performance degradation of index analysis on complex keys on large tables. This fixes \#6924. [\#7075](https://github.com/ClickHouse/ClickHouse/pull/7075) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Fix logical error causing segfaults when selecting from Kafka empty topic. [\#6909](https://github.com/ClickHouse/ClickHouse/pull/6909) ([Ivan](https://github.com/abyss7))
+- Fix too early MySQL connection close in `MySQLBlockInputStream.cpp`. [\#6882](https://github.com/ClickHouse/ClickHouse/pull/6882) ([Clément Rodriguez](https://github.com/clemrodriguez))
+- Returned support for very old Linux kernels (fix [\#6841](https://github.com/ClickHouse/ClickHouse/issues/6841)) [\#6853](https://github.com/ClickHouse/ClickHouse/pull/6853) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Fix possible data loss in `insert select` query in case of empty block in input stream. \#6834 \#6862 [\#6911](https://github.com/ClickHouse/ClickHouse/pull/6911) ([Nikolai Kochetov](https://github.com/KochetovNicolai))
+- Fix for function `АrrayEnumerateUniqRanked` with empty arrays in params [\#6928](https://github.com/ClickHouse/ClickHouse/pull/6928) ([proller](https://github.com/proller))
+- Fix complex queries with array joins and global subqueries. [\#6934](https://github.com/ClickHouse/ClickHouse/pull/6934) ([Ivan](https://github.com/abyss7))
+- Fix `Unknown identifier` error in ORDER BY and GROUP BY with multiple JOINs [\#7022](https://github.com/ClickHouse/ClickHouse/pull/7022) ([Artem Zuikov](https://github.com/4ertus2))
+- Fixed `MSan` warning while executing function with `LowCardinality` argument. [\#7062](https://github.com/ClickHouse/ClickHouse/pull/7062) ([Nikolai Kochetov](https://github.com/KochetovNicolai))
+
+#### Backward Incompatible Change {#backward-incompatible-change-2}
+
+- Changed serialization format of bitmap\* aggregate function states to improve performance. Serialized states of bitmap\* from previous versions cannot be read. [\#6908](https://github.com/ClickHouse/ClickHouse/pull/6908) ([Zhichang Yu](https://github.com/yuzhichang))
+
+## ClickHouse release 19.14 {#clickhouse-release-19-14}
+
+### ClickHouse release 19.14.7.15, 2019-10-02 {#clickhouse-release-19-14-7-15-2019-10-02}
+
+#### Bug Fix {#bug-fix-6}
+
+- This release also contains all bug fixes from 19.11.12.69.
+- Fixed compatibility for distributed queries between 19.14 and earlier versions. This fixes [\#7068](https://github.com/ClickHouse/ClickHouse/issues/7068). [\#7069](https://github.com/ClickHouse/ClickHouse/pull/7069) ([alexey-milovidov](https://github.com/alexey-milovidov))
+
+### ClickHouse release 19.14.6.12, 2019-09-19 {#clickhouse-release-19-14-6-12-2019-09-19}
+
+#### Bug Fix {#bug-fix-7}
+
+- Fix for function `АrrayEnumerateUniqRanked` with empty arrays in params. [\#6928](https://github.com/ClickHouse/ClickHouse/pull/6928) ([proller](https://github.com/proller))
+- Fixed subquery name in queries with `ARRAY JOIN` and `GLOBAL IN subquery` with alias. Use subquery alias for external table name if it is specified. [\#6934](https://github.com/ClickHouse/ClickHouse/pull/6934) ([Ivan](https://github.com/abyss7))
+
+#### Build/Testing/Packaging Improvement {#buildtestingpackaging-improvement-3}
+
+- Fix [flapping](https://clickhouse-test-reports.s3.yandex.net/6944/aab95fd5175a513413c7395a73a82044bdafb906/functional_stateless_tests_(debug).html) test `00715_fetch_merged_or_mutated_part_zookeeper` by rewriting it to a shell scripts because it needs to wait for mutations to apply. [\#6977](https://github.com/ClickHouse/ClickHouse/pull/6977) ([Alexander Kazakov](https://github.com/Akazz))
+- Fixed UBSan and MemSan failure in function `groupUniqArray` with emtpy array argument. It was caused by placing of empty `PaddedPODArray` into hash table zero cell because constructor for zero cell value was not called. [\#6937](https://github.com/ClickHouse/ClickHouse/pull/6937) ([Amos Bird](https://github.com/amosbird))
+
+### ClickHouse release 19.14.3.3, 2019-09-10 {#clickhouse-release-19-14-3-3-2019-09-10}
+
+#### New Feature {#new-feature-4}
+
+- `WITH FILL` modifier for `ORDER BY`. (continuation of [\#5069](https://github.com/ClickHouse/ClickHouse/issues/5069)) [\#6610](https://github.com/ClickHouse/ClickHouse/pull/6610) ([Anton Popov](https://github.com/CurtizJ))
+- `WITH TIES` modifier for `LIMIT`. (continuation of [\#5069](https://github.com/ClickHouse/ClickHouse/issues/5069)) [\#6610](https://github.com/ClickHouse/ClickHouse/pull/6610) ([Anton Popov](https://github.com/CurtizJ))
+- Parse unquoted `NULL` literal as NULL (if setting `format_csv_unquoted_null_literal_as_null=1`). Initialize null fields with default values if data type of this field is not nullable (if setting `input_format_null_as_default=1`). [\#5990](https://github.com/ClickHouse/ClickHouse/issues/5990) [\#6055](https://github.com/ClickHouse/ClickHouse/pull/6055) ([tavplubix](https://github.com/tavplubix))
+- Support for wildcards in paths of table functions `file` and `hdfs`. If the path contains wildcards, the table will be readonly. Example of usage: `select * from hdfs('hdfs://hdfs1:9000/some_dir/another_dir/*/file{0..9}{0..9}')` and `select * from file('some_dir/{some_file,another_file,yet_another}.tsv', 'TSV', 'value UInt32')`. [\#6092](https://github.com/ClickHouse/ClickHouse/pull/6092) ([Olga Khvostikova](https://github.com/stavrolia))
+- New `system.metric_log` table which stores values of `system.events` and `system.metrics` with specified time interval. [\#6363](https://github.com/ClickHouse/ClickHouse/issues/6363) [\#6467](https://github.com/ClickHouse/ClickHouse/pull/6467) ([Nikita Mikhaylov](https://github.com/nikitamikhaylov)) [\#6530](https://github.com/ClickHouse/ClickHouse/pull/6530) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Allow to write ClickHouse text logs to `system.text_log` table. [\#6037](https://github.com/ClickHouse/ClickHouse/issues/6037) [\#6103](https://github.com/ClickHouse/ClickHouse/pull/6103) ([Nikita Mikhaylov](https://github.com/nikitamikhaylov)) [\#6164](https://github.com/ClickHouse/ClickHouse/pull/6164) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Show private symbols in stack traces (this is done via parsing symbol tables of ELF files). Added information about file and line number in stack traces if debug info is present. Speedup symbol name lookup with indexing symbols present in program. Added new SQL functions for introspection: `demangle` and `addressToLine`. Renamed function `symbolizeAddress` to `addressToSymbol` for consistency. Function `addressToSymbol` will return mangled name for performance reasons and you have to apply `demangle`. Added setting `allow_introspection_functions` which is turned off by default. [\#6201](https://github.com/ClickHouse/ClickHouse/pull/6201) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Table function `values` (the name is case-insensitive). It allows to read from `VALUES` list proposed in [\#5984](https://github.com/ClickHouse/ClickHouse/issues/5984). Example: `SELECT * FROM VALUES('a UInt64, s String', (1, 'one'), (2, 'two'), (3, 'three'))`. [\#6217](https://github.com/ClickHouse/ClickHouse/issues/6217). [\#6209](https://github.com/ClickHouse/ClickHouse/pull/6209) ([dimarub2000](https://github.com/dimarub2000))
+- Added an ability to alter storage settings. Syntax: `ALTER TABLE MODIFY SETTING = `. [\#6366](https://github.com/ClickHouse/ClickHouse/pull/6366) [\#6669](https://github.com/ClickHouse/ClickHouse/pull/6669) [\#6685](https://github.com/ClickHouse/ClickHouse/pull/6685) ([alesapin](https://github.com/alesapin))
+- Support for removing of detached parts. Syntax: `ALTER TABLE DROP DETACHED PART ''`. [\#6158](https://github.com/ClickHouse/ClickHouse/pull/6158) ([tavplubix](https://github.com/tavplubix))
+- Table constraints. Allows to add constraint to table definition which will be checked at insert. [\#5273](https://github.com/ClickHouse/ClickHouse/pull/5273) ([Gleb Novikov](https://github.com/NanoBjorn)) [\#6652](https://github.com/ClickHouse/ClickHouse/pull/6652) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Suppport for cascaded materialized views. [\#6324](https://github.com/ClickHouse/ClickHouse/pull/6324) ([Amos Bird](https://github.com/amosbird))
+- Turn on query profiler by default to sample every query execution thread once a second. [\#6283](https://github.com/ClickHouse/ClickHouse/pull/6283) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Input format `ORC`. [\#6454](https://github.com/ClickHouse/ClickHouse/pull/6454) [\#6703](https://github.com/ClickHouse/ClickHouse/pull/6703) ([akonyaev90](https://github.com/akonyaev90))
+- Added two new functions: `sigmoid` and `tanh` (that are useful for machine learning applications). [\#6254](https://github.com/ClickHouse/ClickHouse/pull/6254) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Function `hasToken(haystack, token)`, `hasTokenCaseInsensitive(haystack, token)` to check if given token is in haystack. Token is a maximal length substring between two non alphanumeric ASCII characters (or boundaries of haystack). Token must be a constant string. Supported by tokenbf\_v1 index specialization. [\#6596](https://github.com/ClickHouse/ClickHouse/pull/6596), [\#6662](https://github.com/ClickHouse/ClickHouse/pull/6662) ([Vasily Nemkov](https://github.com/Enmk))
+- New function `neighbor(value, offset[, default_value])`. Allows to reach prev/next value within column in a block of data. [\#5925](https://github.com/ClickHouse/ClickHouse/pull/5925) ([Alex Krash](https://github.com/alex-krash)) [6685365ab8c5b74f9650492c88a012596eb1b0c6](https://github.com/ClickHouse/ClickHouse/commit/6685365ab8c5b74f9650492c88a012596eb1b0c6) [341e2e4587a18065c2da1ca888c73389f48ce36c](https://github.com/ClickHouse/ClickHouse/commit/341e2e4587a18065c2da1ca888c73389f48ce36c) [Alexey Milovidov](https://github.com/alexey-milovidov)
+- Created a function `currentUser()`, returning login of authorized user. Added alias `user()` for compatibility with MySQL. [\#6470](https://github.com/ClickHouse/ClickHouse/pull/6470) ([Alex Krash](https://github.com/alex-krash))
+- New aggregate functions `quantilesExactInclusive` and `quantilesExactExclusive` which were proposed in [\#5885](https://github.com/ClickHouse/ClickHouse/issues/5885). [\#6477](https://github.com/ClickHouse/ClickHouse/pull/6477) ([dimarub2000](https://github.com/dimarub2000))
+- Function `bitmapRange(bitmap, range_begin, range_end)` which returns new set with specified range (not include the `range_end`). [\#6314](https://github.com/ClickHouse/ClickHouse/pull/6314) ([Zhichang Yu](https://github.com/yuzhichang))
+- Function `geohashesInBox(longitude_min, latitude_min, longitude_max, latitude_max, precision)` which creates array of precision-long strings of geohash-boxes covering provided area. [\#6127](https://github.com/ClickHouse/ClickHouse/pull/6127) ([Vasily Nemkov](https://github.com/Enmk))
+- Implement support for INSERT query with `Kafka` tables. [\#6012](https://github.com/ClickHouse/ClickHouse/pull/6012) ([Ivan](https://github.com/abyss7))
+- Added support for `_partition` and `_timestamp` virtual columns to Kafka engine. [\#6400](https://github.com/ClickHouse/ClickHouse/pull/6400) ([Ivan](https://github.com/abyss7))
+- Possibility to remove sensitive data from `query_log`, server logs, process list with regexp-based rules. [\#5710](https://github.com/ClickHouse/ClickHouse/pull/5710) ([filimonov](https://github.com/filimonov))
+
+#### Experimental Feature {#experimental-feature-2}
+
+- Input and output data format `Template`. It allows to specify custom format string for input and output. [\#4354](https://github.com/ClickHouse/ClickHouse/issues/4354) [\#6727](https://github.com/ClickHouse/ClickHouse/pull/6727) ([tavplubix](https://github.com/tavplubix))
+- Implementation of `LIVE VIEW` tables that were originally proposed in [\#2898](https://github.com/ClickHouse/ClickHouse/pull/2898), prepared in [\#3925](https://github.com/ClickHouse/ClickHouse/issues/3925), and then updated in [\#5541](https://github.com/ClickHouse/ClickHouse/issues/5541). See [\#5541](https://github.com/ClickHouse/ClickHouse/issues/5541) for detailed description. [\#5541](https://github.com/ClickHouse/ClickHouse/issues/5541) ([vzakaznikov](https://github.com/vzakaznikov)) [\#6425](https://github.com/ClickHouse/ClickHouse/pull/6425) ([Nikolai Kochetov](https://github.com/KochetovNicolai)) [\#6656](https://github.com/ClickHouse/ClickHouse/pull/6656) ([vzakaznikov](https://github.com/vzakaznikov)) Note that `LIVE VIEW` feature may be removed in next versions.
+
+#### Bug Fix {#bug-fix-8}
+
+- This release also contains all bug fixes from 19.13 and 19.11.
+- Fix segmentation fault when the table has skip indices and vertical merge happens. [\#6723](https://github.com/ClickHouse/ClickHouse/pull/6723) ([alesapin](https://github.com/alesapin))
+- Fix per-column TTL with non-trivial column defaults. Previously in case of force TTL merge with `OPTIMIZE ... FINAL` query, expired values was replaced by type defaults instead of user-specified column defaults. [\#6796](https://github.com/ClickHouse/ClickHouse/pull/6796) ([Anton Popov](https://github.com/CurtizJ))
+- Fix Kafka messages duplication problem on normal server restart. [\#6597](https://github.com/ClickHouse/ClickHouse/pull/6597) ([Ivan](https://github.com/abyss7))
+- Fixed infinite loop when reading Kafka messages. Do not pause/resume consumer on subscription at all - otherwise it may get paused indefinitely in some scenarios. [\#6354](https://github.com/ClickHouse/ClickHouse/pull/6354) ([Ivan](https://github.com/abyss7))
+- Fix `Key expression contains comparison between inconvertible types` exception in `bitmapContains` function. [\#6136](https://github.com/ClickHouse/ClickHouse/issues/6136) [\#6146](https://github.com/ClickHouse/ClickHouse/issues/6146) [\#6156](https://github.com/ClickHouse/ClickHouse/pull/6156) ([dimarub2000](https://github.com/dimarub2000))
+- Fix segfault with enabled `optimize_skip_unused_shards` and missing sharding key. [\#6384](https://github.com/ClickHouse/ClickHouse/pull/6384) ([Anton Popov](https://github.com/CurtizJ))
+- Fixed wrong code in mutations that may lead to memory corruption. Fixed segfault with read of address `0x14c0` that may happed due to concurrent `DROP TABLE` and `SELECT` from `system.parts` or `system.parts_columns`. Fixed race condition in preparation of mutation queries. Fixed deadlock caused by `OPTIMIZE` of Replicated tables and concurrent modification operations like ALTERs. [\#6514](https://github.com/ClickHouse/ClickHouse/pull/6514) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Removed extra verbose logging in MySQL interface [\#6389](https://github.com/ClickHouse/ClickHouse/pull/6389) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Return the ability to parse boolean settings from ‘true’ and ‘false’ in the configuration file. [\#6278](https://github.com/ClickHouse/ClickHouse/pull/6278) ([alesapin](https://github.com/alesapin))
+- Fix crash in `quantile` and `median` function over `Nullable(Decimal128)`. [\#6378](https://github.com/ClickHouse/ClickHouse/pull/6378) ([Artem Zuikov](https://github.com/4ertus2))
+- Fixed possible incomplete result returned by `SELECT` query with `WHERE` condition on primary key contained conversion to Float type. It was caused by incorrect checking of monotonicity in `toFloat` function. [\#6248](https://github.com/ClickHouse/ClickHouse/issues/6248) [\#6374](https://github.com/ClickHouse/ClickHouse/pull/6374) ([dimarub2000](https://github.com/dimarub2000))
+- Check `max_expanded_ast_elements` setting for mutations. Clear mutations after `TRUNCATE TABLE`. [\#6205](https://github.com/ClickHouse/ClickHouse/pull/6205) ([Winter Zhang](https://github.com/zhang2014))
+- Fix JOIN results for key columns when used with `join_use_nulls`. Attach Nulls instead of columns defaults. [\#6249](https://github.com/ClickHouse/ClickHouse/pull/6249) ([Artem Zuikov](https://github.com/4ertus2))
+- Fix for skip indices with vertical merge and alter. Fix for `Bad size of marks file` exception. [\#6594](https://github.com/ClickHouse/ClickHouse/issues/6594) [\#6713](https://github.com/ClickHouse/ClickHouse/pull/6713) ([alesapin](https://github.com/alesapin))
+- Fix rare crash in `ALTER MODIFY COLUMN` and vertical merge when one of merged/altered parts is empty (0 rows) [\#6746](https://github.com/ClickHouse/ClickHouse/issues/6746) [\#6780](https://github.com/ClickHouse/ClickHouse/pull/6780) ([alesapin](https://github.com/alesapin))
+- Fixed bug in conversion of `LowCardinality` types in `AggregateFunctionFactory`. This fixes [\#6257](https://github.com/ClickHouse/ClickHouse/issues/6257). [\#6281](https://github.com/ClickHouse/ClickHouse/pull/6281) ([Nikolai Kochetov](https://github.com/KochetovNicolai))
+- Fix wrong behavior and possible segfaults in `topK` and `topKWeighted` aggregated functions. [\#6404](https://github.com/ClickHouse/ClickHouse/pull/6404) ([Anton Popov](https://github.com/CurtizJ))
+- Fixed unsafe code around `getIdentifier` function. [\#6401](https://github.com/ClickHouse/ClickHouse/issues/6401) [\#6409](https://github.com/ClickHouse/ClickHouse/pull/6409) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Fixed bug in MySQL wire protocol (is used while connecting to ClickHouse form MySQL client). Caused by heap buffer overflow in `PacketPayloadWriteBuffer`. [\#6212](https://github.com/ClickHouse/ClickHouse/pull/6212) ([Yuriy Baranov](https://github.com/yurriy))
+- Fixed memory leak in `bitmapSubsetInRange` function. [\#6819](https://github.com/ClickHouse/ClickHouse/pull/6819) ([Zhichang Yu](https://github.com/yuzhichang))
+- Fix rare bug when mutation executed after granularity change. [\#6816](https://github.com/ClickHouse/ClickHouse/pull/6816) ([alesapin](https://github.com/alesapin))
+- Allow protobuf message with all fields by default. [\#6132](https://github.com/ClickHouse/ClickHouse/pull/6132) ([Vitaly Baranov](https://github.com/vitlibar))
+- Resolve a bug with `nullIf` function when we send a `NULL` argument on the second argument. [\#6446](https://github.com/ClickHouse/ClickHouse/pull/6446) ([Guillaume Tassery](https://github.com/YiuRULE))
+- Fix rare bug with wrong memory allocation/deallocation in complex key cache dictionaries with string fields which leads to infinite memory consumption (looks like memory leak). Bug reproduces when string size was a power of two starting from eight (8, 16, 32, etc). [\#6447](https://github.com/ClickHouse/ClickHouse/pull/6447) ([alesapin](https://github.com/alesapin))
+- Fixed Gorilla encoding on small sequences which caused exception `Cannot write after end of buffer`. [\#6398](https://github.com/ClickHouse/ClickHouse/issues/6398) [\#6444](https://github.com/ClickHouse/ClickHouse/pull/6444) ([Vasily Nemkov](https://github.com/Enmk))
+- Allow to use not nullable types in JOINs with `join_use_nulls` enabled. [\#6705](https://github.com/ClickHouse/ClickHouse/pull/6705) ([Artem Zuikov](https://github.com/4ertus2))
+- Disable `Poco::AbstractConfiguration` substitutions in query in `clickhouse-client`. [\#6706](https://github.com/ClickHouse/ClickHouse/pull/6706) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Avoid deadlock in `REPLACE PARTITION`. [\#6677](https://github.com/ClickHouse/ClickHouse/pull/6677) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Using `arrayReduce` for constant arguments may lead to segfault. [\#6242](https://github.com/ClickHouse/ClickHouse/issues/6242) [\#6326](https://github.com/ClickHouse/ClickHouse/pull/6326) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Fix inconsistent parts which can appear if replica was restored after `DROP PARTITION`. [\#6522](https://github.com/ClickHouse/ClickHouse/issues/6522) [\#6523](https://github.com/ClickHouse/ClickHouse/pull/6523) ([tavplubix](https://github.com/tavplubix))
+- Fixed hang in `JSONExtractRaw` function. [\#6195](https://github.com/ClickHouse/ClickHouse/issues/6195) [\#6198](https://github.com/ClickHouse/ClickHouse/pull/6198) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Fix bug with incorrect skip indices serialization and aggregation with adaptive granularity. [\#6594](https://github.com/ClickHouse/ClickHouse/issues/6594). [\#6748](https://github.com/ClickHouse/ClickHouse/pull/6748) ([alesapin](https://github.com/alesapin))
+- Fix `WITH ROLLUP` and `WITH CUBE` modifiers of `GROUP BY` with two-level aggregation. [\#6225](https://github.com/ClickHouse/ClickHouse/pull/6225) ([Anton Popov](https://github.com/CurtizJ))
+- Fix bug with writing secondary indices marks with adaptive granularity. [\#6126](https://github.com/ClickHouse/ClickHouse/pull/6126) ([alesapin](https://github.com/alesapin))
+- Fix initialization order while server startup. Since `StorageMergeTree::background_task_handle` is initialized in `startup()` the `MergeTreeBlockOutputStream::write()` may try to use it before initialization. Just check if it is initialized. [\#6080](https://github.com/ClickHouse/ClickHouse/pull/6080) ([Ivan](https://github.com/abyss7))
+- Clearing the data buffer from the previous read operation that was completed with an error. [\#6026](https://github.com/ClickHouse/ClickHouse/pull/6026) ([Nikolay](https://github.com/bopohaa))
+- Fix bug with enabling adaptive granularity when creating a new replica for Replicated\*MergeTree table. [\#6394](https://github.com/ClickHouse/ClickHouse/issues/6394) [\#6452](https://github.com/ClickHouse/ClickHouse/pull/6452) ([alesapin](https://github.com/alesapin))
+- Fixed possible crash during server startup in case of exception happened in `libunwind` during exception at access to uninitialized `ThreadStatus` structure. [\#6456](https://github.com/ClickHouse/ClickHouse/pull/6456) ([Nikita Mikhaylov](https://github.com/nikitamikhaylov))
+- Fix crash in `yandexConsistentHash` function. Found by fuzz test. [\#6304](https://github.com/ClickHouse/ClickHouse/issues/6304) [\#6305](https://github.com/ClickHouse/ClickHouse/pull/6305) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Fixed the possibility of hanging queries when server is overloaded and global thread pool becomes near full. This have higher chance to happen on clusters with large number of shards (hundreds), because distributed queries allocate a thread per connection to each shard. For example, this issue may reproduce if a cluster of 330 shards is processing 30 concurrent distributed queries. This issue affects all versions starting from 19.2. [\#6301](https://github.com/ClickHouse/ClickHouse/pull/6301) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Fixed logic of `arrayEnumerateUniqRanked` function. [\#6423](https://github.com/ClickHouse/ClickHouse/pull/6423) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Fix segfault when decoding symbol table. [\#6603](https://github.com/ClickHouse/ClickHouse/pull/6603) ([Amos Bird](https://github.com/amosbird))
+- Fixed irrelevant exception in cast of `LowCardinality(Nullable)` to not-Nullable column in case if it doesn’t contain Nulls (e.g. in query like `SELECT CAST(CAST('Hello' AS LowCardinality(Nullable(String))) AS String)`. [\#6094](https://github.com/ClickHouse/ClickHouse/issues/6094) [\#6119](https://github.com/ClickHouse/ClickHouse/pull/6119) ([Nikolai Kochetov](https://github.com/KochetovNicolai))
+- Removed extra quoting of description in `system.settings` table. [\#6696](https://github.com/ClickHouse/ClickHouse/issues/6696) [\#6699](https://github.com/ClickHouse/ClickHouse/pull/6699) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Avoid possible deadlock in `TRUNCATE` of Replicated table. [\#6695](https://github.com/ClickHouse/ClickHouse/pull/6695) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Fix reading in order of sorting key. [\#6189](https://github.com/ClickHouse/ClickHouse/pull/6189) ([Anton Popov](https://github.com/CurtizJ))
+- Fix `ALTER TABLE ... UPDATE` query for tables with `enable_mixed_granularity_parts=1`. [\#6543](https://github.com/ClickHouse/ClickHouse/pull/6543) ([alesapin](https://github.com/alesapin))
+- Fix bug opened by [\#4405](https://github.com/ClickHouse/ClickHouse/pull/4405) (since 19.4.0). Reproduces in queries to Distributed tables over MergeTree tables when we doesn’t query any columns (`SELECT 1`). [\#6236](https://github.com/ClickHouse/ClickHouse/pull/6236) ([alesapin](https://github.com/alesapin))
+- Fixed overflow in integer division of signed type to unsigned type. The behaviour was exactly as in C or C++ language (integer promotion rules) that may be surprising. Please note that the overflow is still possible when dividing large signed number to large unsigned number or vice-versa (but that case is less usual). The issue existed in all server versions. [\#6214](https://github.com/ClickHouse/ClickHouse/issues/6214) [\#6233](https://github.com/ClickHouse/ClickHouse/pull/6233) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Limit maximum sleep time for throttling when `max_execution_speed` or `max_execution_speed_bytes` is set. Fixed false errors like `Estimated query execution time (inf seconds) is too long`. [\#5547](https://github.com/ClickHouse/ClickHouse/issues/5547) [\#6232](https://github.com/ClickHouse/ClickHouse/pull/6232) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Fixed issues about using `MATERIALIZED` columns and aliases in `MaterializedView`. [\#448](https://github.com/ClickHouse/ClickHouse/issues/448) [\#3484](https://github.com/ClickHouse/ClickHouse/issues/3484) [\#3450](https://github.com/ClickHouse/ClickHouse/issues/3450) [\#2878](https://github.com/ClickHouse/ClickHouse/issues/2878) [\#2285](https://github.com/ClickHouse/ClickHouse/issues/2285) [\#3796](https://github.com/ClickHouse/ClickHouse/pull/3796) ([Amos Bird](https://github.com/amosbird)) [\#6316](https://github.com/ClickHouse/ClickHouse/pull/6316) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Fix `FormatFactory` behaviour for input streams which are not implemented as processor. [\#6495](https://github.com/ClickHouse/ClickHouse/pull/6495) ([Nikolai Kochetov](https://github.com/KochetovNicolai))
+- Fixed typo. [\#6631](https://github.com/ClickHouse/ClickHouse/pull/6631) ([Alex Ryndin](https://github.com/alexryndin))
+- Typo in the error message ( is -\> are ). [\#6839](https://github.com/ClickHouse/ClickHouse/pull/6839) ([Denis Zhuravlev](https://github.com/den-crane))
+- Fixed error while parsing of columns list from string if type contained a comma (this issue was relevant for `File`, `URL`, `HDFS` storages) [\#6217](https://github.com/ClickHouse/ClickHouse/issues/6217). [\#6209](https://github.com/ClickHouse/ClickHouse/pull/6209) ([dimarub2000](https://github.com/dimarub2000))
+
+#### Security Fix {#security-fix}
+
+- This release also contains all bug security fixes from 19.13 and 19.11.
+- Fixed the possibility of a fabricated query to cause server crash due to stack overflow in SQL parser. Fixed the possibility of stack overflow in Merge and Distributed tables, materialized views and conditions for row-level security that involve subqueries. [\#6433](https://github.com/ClickHouse/ClickHouse/pull/6433) ([alexey-milovidov](https://github.com/alexey-milovidov))
+
+#### Improvement {#improvement-3}
+
+- Correct implementation of ternary logic for `AND/OR`. [\#6048](https://github.com/ClickHouse/ClickHouse/pull/6048) ([Alexander Kazakov](https://github.com/Akazz))
+- Now values and rows with expired TTL will be removed after `OPTIMIZE ... FINAL` query from old parts without TTL infos or with outdated TTL infos, e.g. after `ALTER ... MODIFY TTL` query. Added queries `SYSTEM STOP/START TTL MERGES` to disallow/allow assign merges with TTL and filter expired values in all merges. [\#6274](https://github.com/ClickHouse/ClickHouse/pull/6274) ([Anton Popov](https://github.com/CurtizJ))
+- Possibility to change the location of ClickHouse history file for client using `CLICKHOUSE_HISTORY_FILE` env. [\#6840](https://github.com/ClickHouse/ClickHouse/pull/6840) ([filimonov](https://github.com/filimonov))
+- Remove `dry_run` flag from `InterpreterSelectQuery`. … [\#6375](https://github.com/ClickHouse/ClickHouse/pull/6375) ([Nikolai Kochetov](https://github.com/KochetovNicolai))
+- Support `ASOF JOIN` with `ON` section. [\#6211](https://github.com/ClickHouse/ClickHouse/pull/6211) ([Artem Zuikov](https://github.com/4ertus2))
+- Better support of skip indexes for mutations and replication. Support for `MATERIALIZE/CLEAR INDEX ... IN PARTITION` query. `UPDATE x = x` recalculates all indices that use column `x`. [\#5053](https://github.com/ClickHouse/ClickHouse/pull/5053) ([Nikita Vasilev](https://github.com/nikvas0))
+- Allow to `ATTACH` live views (for example, at the server startup) regardless to `allow_experimental_live_view` setting. [\#6754](https://github.com/ClickHouse/ClickHouse/pull/6754) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- For stack traces gathered by query profiler, do not include stack frames generated by the query profiler itself. [\#6250](https://github.com/ClickHouse/ClickHouse/pull/6250) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Now table functions `values`, `file`, `url`, `hdfs` have support for ALIAS columns. [\#6255](https://github.com/ClickHouse/ClickHouse/pull/6255) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Throw an exception if `config.d` file doesn’t have the corresponding root element as the config file. [\#6123](https://github.com/ClickHouse/ClickHouse/pull/6123) ([dimarub2000](https://github.com/dimarub2000))
+- Print extra info in exception message for `no space left on device`. [\#6182](https://github.com/ClickHouse/ClickHouse/issues/6182), [\#6252](https://github.com/ClickHouse/ClickHouse/issues/6252) [\#6352](https://github.com/ClickHouse/ClickHouse/pull/6352) ([tavplubix](https://github.com/tavplubix))
+- When determining shards of a `Distributed` table to be covered by a read query (for `optimize_skip_unused_shards` = 1) ClickHouse now checks conditions from both `prewhere` and `where` clauses of select statement. [\#6521](https://github.com/ClickHouse/ClickHouse/pull/6521) ([Alexander Kazakov](https://github.com/Akazz))
+- Enabled `SIMDJSON` for machines without AVX2 but with SSE 4.2 and PCLMUL instruction set. [\#6285](https://github.com/ClickHouse/ClickHouse/issues/6285) [\#6320](https://github.com/ClickHouse/ClickHouse/pull/6320) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- ClickHouse can work on filesystems without `O_DIRECT` support (such as ZFS and BtrFS) without additional tuning. [\#4449](https://github.com/ClickHouse/ClickHouse/issues/4449) [\#6730](https://github.com/ClickHouse/ClickHouse/pull/6730) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Support push down predicate for final subquery. [\#6120](https://github.com/ClickHouse/ClickHouse/pull/6120) ([TCeason](https://github.com/TCeason)) [\#6162](https://github.com/ClickHouse/ClickHouse/pull/6162) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Better `JOIN ON` keys extraction [\#6131](https://github.com/ClickHouse/ClickHouse/pull/6131) ([Artem Zuikov](https://github.com/4ertus2))
+- Upated `SIMDJSON`. [\#6285](https://github.com/ClickHouse/ClickHouse/issues/6285). [\#6306](https://github.com/ClickHouse/ClickHouse/pull/6306) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Optimize selecting of smallest column for `SELECT count()` query. [\#6344](https://github.com/ClickHouse/ClickHouse/pull/6344) ([Amos Bird](https://github.com/amosbird))
+- Added `strict` parameter in `windowFunnel()`. When the `strict` is set, the `windowFunnel()` applies conditions only for the unique values. [\#6548](https://github.com/ClickHouse/ClickHouse/pull/6548) ([achimbab](https://github.com/achimbab))
+- Safer interface of `mysqlxx::Pool`. [\#6150](https://github.com/ClickHouse/ClickHouse/pull/6150) ([avasiliev](https://github.com/avasiliev))
+- Options line size when executing with `--help` option now corresponds with terminal size. [\#6590](https://github.com/ClickHouse/ClickHouse/pull/6590) ([dimarub2000](https://github.com/dimarub2000))
+- Disable “read in order” optimization for aggregation without keys. [\#6599](https://github.com/ClickHouse/ClickHouse/pull/6599) ([Anton Popov](https://github.com/CurtizJ))
+- HTTP status code for `INCORRECT_DATA` and `TYPE_MISMATCH` error codes was changed from default `500 Internal Server Error` to `400 Bad Request`. [\#6271](https://github.com/ClickHouse/ClickHouse/pull/6271) ([Alexander Rodin](https://github.com/a-rodin))
+- Move Join object from `ExpressionAction` into `AnalyzedJoin`. `ExpressionAnalyzer` and `ExpressionAction` do not know about `Join` class anymore. Its logic is hidden by `AnalyzedJoin` iface. [\#6801](https://github.com/ClickHouse/ClickHouse/pull/6801) ([Artem Zuikov](https://github.com/4ertus2))
+- Fixed possible deadlock of distributed queries when one of shards is localhost but the query is sent via network connection. [\#6759](https://github.com/ClickHouse/ClickHouse/pull/6759) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Changed semantic of multiple tables `RENAME` to avoid possible deadlocks. [\#6757](https://github.com/ClickHouse/ClickHouse/issues/6757). [\#6756](https://github.com/ClickHouse/ClickHouse/pull/6756) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Rewritten MySQL compatibility server to prevent loading full packet payload in memory. Decreased memory consumption for each connection to approximately `2 * DBMS_DEFAULT_BUFFER_SIZE` (read/write buffers). [\#5811](https://github.com/ClickHouse/ClickHouse/pull/5811) ([Yuriy Baranov](https://github.com/yurriy))
+- Move AST alias interpreting logic out of parser that doesn’t have to know anything about query semantics. [\#6108](https://github.com/ClickHouse/ClickHouse/pull/6108) ([Artem Zuikov](https://github.com/4ertus2))
+- Slightly more safe parsing of `NamesAndTypesList`. [\#6408](https://github.com/ClickHouse/ClickHouse/issues/6408). [\#6410](https://github.com/ClickHouse/ClickHouse/pull/6410) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- `clickhouse-copier`: Allow use `where_condition` from config with `partition_key` alias in query for checking partition existence (Earlier it was used only in reading data queries). [\#6577](https://github.com/ClickHouse/ClickHouse/pull/6577) ([proller](https://github.com/proller))
+- Added optional message argument in `throwIf`. ([\#5772](https://github.com/ClickHouse/ClickHouse/issues/5772)) [\#6329](https://github.com/ClickHouse/ClickHouse/pull/6329) ([Vdimir](https://github.com/Vdimir))
+- Server exception got while sending insertion data is now being processed in client as well. [\#5891](https://github.com/ClickHouse/ClickHouse/issues/5891) [\#6711](https://github.com/ClickHouse/ClickHouse/pull/6711) ([dimarub2000](https://github.com/dimarub2000))
+- Added a metric `DistributedFilesToInsert` that shows the total number of files in filesystem that are selected to send to remote servers by Distributed tables. The number is summed across all shards. [\#6600](https://github.com/ClickHouse/ClickHouse/pull/6600) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Move most of JOINs prepare logic from `ExpressionAction/ExpressionAnalyzer` to `AnalyzedJoin`. [\#6785](https://github.com/ClickHouse/ClickHouse/pull/6785) ([Artem Zuikov](https://github.com/4ertus2))
+- Fix TSan [warning](https://clickhouse-test-reports.s3.yandex.net/6399/c1c1d1daa98e199e620766f1bd06a5921050a00d/functional_stateful_tests_(thread).html) ‘lock-order-inversion’. [\#6740](https://github.com/ClickHouse/ClickHouse/pull/6740) ([Vasily Nemkov](https://github.com/Enmk))
+- Better information messages about lack of Linux capabilities. Logging fatal errors with “fatal” level, that will make it easier to find in `system.text_log`. [\#6441](https://github.com/ClickHouse/ClickHouse/pull/6441) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- When enable dumping temporary data to the disk to restrict memory usage during `GROUP BY`, `ORDER BY`, it didn’t check the free disk space. The fix add a new setting `min_free_disk_space`, when the free disk space it smaller then the threshold, the query will stop and throw `ErrorCodes::NOT_ENOUGH_SPACE`. [\#6678](https://github.com/ClickHouse/ClickHouse/pull/6678) ([Weiqing Xu](https://github.com/weiqxu)) [\#6691](https://github.com/ClickHouse/ClickHouse/pull/6691) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Removed recursive rwlock by thread. It makes no sense, because threads are reused between queries. `SELECT` query may acquire a lock in one thread, hold a lock from another thread and exit from first thread. In the same time, first thread can be reused by `DROP` query. This will lead to false “Attempt to acquire exclusive lock recursively” messages. [\#6771](https://github.com/ClickHouse/ClickHouse/pull/6771) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Split `ExpressionAnalyzer.appendJoin()`. Prepare a place in `ExpressionAnalyzer` for `MergeJoin`. [\#6524](https://github.com/ClickHouse/ClickHouse/pull/6524) ([Artem Zuikov](https://github.com/4ertus2))
+- Added `mysql_native_password` authentication plugin to MySQL compatibility server. [\#6194](https://github.com/ClickHouse/ClickHouse/pull/6194) ([Yuriy Baranov](https://github.com/yurriy))
+- Less number of `clock_gettime` calls; fixed ABI compatibility between debug/release in `Allocator` (insignificant issue). [\#6197](https://github.com/ClickHouse/ClickHouse/pull/6197) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Move `collectUsedColumns` from `ExpressionAnalyzer` to `SyntaxAnalyzer`. `SyntaxAnalyzer` makes `required_source_columns` itself now. [\#6416](https://github.com/ClickHouse/ClickHouse/pull/6416) ([Artem Zuikov](https://github.com/4ertus2))
+- Add setting `joined_subquery_requires_alias` to require aliases for subselects and table functions in `FROM` that more than one table is present (i.e. queries with JOINs). [\#6733](https://github.com/ClickHouse/ClickHouse/pull/6733) ([Artem Zuikov](https://github.com/4ertus2))
+- Extract `GetAggregatesVisitor` class from `ExpressionAnalyzer`. [\#6458](https://github.com/ClickHouse/ClickHouse/pull/6458) ([Artem Zuikov](https://github.com/4ertus2))
+- `system.query_log`: change data type of `type` column to `Enum`. [\#6265](https://github.com/ClickHouse/ClickHouse/pull/6265) ([Nikita Mikhaylov](https://github.com/nikitamikhaylov))
+- Static linking of `sha256_password` authentication plugin. [\#6512](https://github.com/ClickHouse/ClickHouse/pull/6512) ([Yuriy Baranov](https://github.com/yurriy))
+- Avoid extra dependency for the setting `compile` to work. In previous versions, the user may get error like `cannot open crti.o`, `unable to find library -lc` etc. [\#6309](https://github.com/ClickHouse/ClickHouse/pull/6309) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- More validation of the input that may come from malicious replica. [\#6303](https://github.com/ClickHouse/ClickHouse/pull/6303) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Now `clickhouse-obfuscator` file is available in `clickhouse-client` package. In previous versions it was available as `clickhouse obfuscator` (with whitespace). [\#5816](https://github.com/ClickHouse/ClickHouse/issues/5816) [\#6609](https://github.com/ClickHouse/ClickHouse/pull/6609) ([dimarub2000](https://github.com/dimarub2000))
+- Fixed deadlock when we have at least two queries that read at least two tables in different order and another query that performs DDL operation on one of tables. Fixed another very rare deadlock. [\#6764](https://github.com/ClickHouse/ClickHouse/pull/6764) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Added `os_thread_ids` column to `system.processes` and `system.query_log` for better debugging possibilities. [\#6763](https://github.com/ClickHouse/ClickHouse/pull/6763) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- A workaround for PHP mysqlnd extension bugs which occur when `sha256_password` is used as a default authentication plugin (described in [\#6031](https://github.com/ClickHouse/ClickHouse/issues/6031)). [\#6113](https://github.com/ClickHouse/ClickHouse/pull/6113) ([Yuriy Baranov](https://github.com/yurriy))
+- Remove unneeded place with changed nullability columns. [\#6693](https://github.com/ClickHouse/ClickHouse/pull/6693) ([Artem Zuikov](https://github.com/4ertus2))
+- Set default value of `queue_max_wait_ms` to zero, because current value (five seconds) makes no sense. There are rare circumstances when this settings has any use. Added settings `replace_running_query_max_wait_ms`, `kafka_max_wait_ms` and `connection_pool_max_wait_ms` for disambiguation. [\#6692](https://github.com/ClickHouse/ClickHouse/pull/6692) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Extract `SelectQueryExpressionAnalyzer` from `ExpressionAnalyzer`. Keep the last one for non-select queries. [\#6499](https://github.com/ClickHouse/ClickHouse/pull/6499) ([Artem Zuikov](https://github.com/4ertus2))
+- Removed duplicating input and output formats. [\#6239](https://github.com/ClickHouse/ClickHouse/pull/6239) ([Nikolai Kochetov](https://github.com/KochetovNicolai))
+- Allow user to override `poll_interval` and `idle_connection_timeout` settings on connection. [\#6230](https://github.com/ClickHouse/ClickHouse/pull/6230) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- `MergeTree` now has an additional option `ttl_only_drop_parts` (disabled by default) to avoid partial pruning of parts, so that they dropped completely when all the rows in a part are expired. [\#6191](https://github.com/ClickHouse/ClickHouse/pull/6191) ([Sergi Vladykin](https://github.com/svladykin))
+- Type checks for set index functions. Throw exception if function got a wrong type. This fixes fuzz test with UBSan. [\#6511](https://github.com/ClickHouse/ClickHouse/pull/6511) ([Nikita Vasilev](https://github.com/nikvas0))
+
+#### Performance Improvement {#performance-improvement-2}
+
+- Optimize queries with `ORDER BY expressions` clause, where `expressions` have coinciding prefix with sorting key in `MergeTree` tables. This optimization is controlled by `optimize_read_in_order` setting. [\#6054](https://github.com/ClickHouse/ClickHouse/pull/6054) [\#6629](https://github.com/ClickHouse/ClickHouse/pull/6629) ([Anton Popov](https://github.com/CurtizJ))
+- Allow to use multiple threads during parts loading and removal. [\#6372](https://github.com/ClickHouse/ClickHouse/issues/6372) [\#6074](https://github.com/ClickHouse/ClickHouse/issues/6074) [\#6438](https://github.com/ClickHouse/ClickHouse/pull/6438) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Implemented batch variant of updating aggregate function states. It may lead to performance benefits. [\#6435](https://github.com/ClickHouse/ClickHouse/pull/6435) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Using `FastOps` library for functions `exp`, `log`, `sigmoid`, `tanh`. FastOps is a fast vector math library from Michael Parakhin (Yandex CTO). Improved performance of `exp` and `log` functions more than 6 times. The functions `exp` and `log` from `Float32` argument will return `Float32` (in previous versions they always return `Float64`). Now `exp(nan)` may return `inf`. The result of `exp` and `log` functions may be not the nearest machine representable number to the true answer. [\#6254](https://github.com/ClickHouse/ClickHouse/pull/6254) ([alexey-milovidov](https://github.com/alexey-milovidov)) Using Danila Kutenin variant to make fastops working [\#6317](https://github.com/ClickHouse/ClickHouse/pull/6317) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Disable consecutive key optimization for `UInt8/16`. [\#6298](https://github.com/ClickHouse/ClickHouse/pull/6298) [\#6701](https://github.com/ClickHouse/ClickHouse/pull/6701) ([akuzm](https://github.com/akuzm))
+- Improved performance of `simdjson` library by getting rid of dynamic allocation in `ParsedJson::Iterator`. [\#6479](https://github.com/ClickHouse/ClickHouse/pull/6479) ([Vitaly Baranov](https://github.com/vitlibar))
+- Pre-fault pages when allocating memory with `mmap()`. [\#6667](https://github.com/ClickHouse/ClickHouse/pull/6667) ([akuzm](https://github.com/akuzm))
+- Fix performance bug in `Decimal` comparison. [\#6380](https://github.com/ClickHouse/ClickHouse/pull/6380) ([Artem Zuikov](https://github.com/4ertus2))
+
+#### Build/Testing/Packaging Improvement {#buildtestingpackaging-improvement-4}
+
+- Remove Compiler (runtime template instantiation) because we’ve win over it’s performance. [\#6646](https://github.com/ClickHouse/ClickHouse/pull/6646) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Added performance test to show degradation of performance in gcc-9 in more isolated way. [\#6302](https://github.com/ClickHouse/ClickHouse/pull/6302) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Added table function `numbers_mt`, which is multithreaded version of `numbers`. Updated performance tests with hash functions. [\#6554](https://github.com/ClickHouse/ClickHouse/pull/6554) ([Nikolai Kochetov](https://github.com/KochetovNicolai))
+- Comparison mode in `clickhouse-benchmark` [\#6220](https://github.com/ClickHouse/ClickHouse/issues/6220) [\#6343](https://github.com/ClickHouse/ClickHouse/pull/6343) ([dimarub2000](https://github.com/dimarub2000))
+- Best effort for printing stack traces. Also added `SIGPROF` as a debugging signal to print stack trace of a running thread. [\#6529](https://github.com/ClickHouse/ClickHouse/pull/6529) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Every function in its own file, part 10. [\#6321](https://github.com/ClickHouse/ClickHouse/pull/6321) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Remove doubled const `TABLE_IS_READ_ONLY`. [\#6566](https://github.com/ClickHouse/ClickHouse/pull/6566) ([filimonov](https://github.com/filimonov))
+- Formatting changes for `StringHashMap` PR [\#5417](https://github.com/ClickHouse/ClickHouse/issues/5417). [\#6700](https://github.com/ClickHouse/ClickHouse/pull/6700) ([akuzm](https://github.com/akuzm))
+- Better subquery for join creation in `ExpressionAnalyzer`. [\#6824](https://github.com/ClickHouse/ClickHouse/pull/6824) ([Artem Zuikov](https://github.com/4ertus2))
+- Remove a redundant condition (found by PVS Studio). [\#6775](https://github.com/ClickHouse/ClickHouse/pull/6775) ([akuzm](https://github.com/akuzm))
+- Separate the hash table interface for `ReverseIndex`. [\#6672](https://github.com/ClickHouse/ClickHouse/pull/6672) ([akuzm](https://github.com/akuzm))
+- Refactoring of settings. [\#6689](https://github.com/ClickHouse/ClickHouse/pull/6689) ([alesapin](https://github.com/alesapin))
+- Add comments for `set` index functions. [\#6319](https://github.com/ClickHouse/ClickHouse/pull/6319) ([Nikita Vasilev](https://github.com/nikvas0))
+- Increase OOM score in debug version on Linux. [\#6152](https://github.com/ClickHouse/ClickHouse/pull/6152) ([akuzm](https://github.com/akuzm))
+- HDFS HA now work in debug build. [\#6650](https://github.com/ClickHouse/ClickHouse/pull/6650) ([Weiqing Xu](https://github.com/weiqxu))
+- Added a test to `transform_query_for_external_database`. [\#6388](https://github.com/ClickHouse/ClickHouse/pull/6388) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Add test for multiple materialized views for Kafka table. [\#6509](https://github.com/ClickHouse/ClickHouse/pull/6509) ([Ivan](https://github.com/abyss7))
+- Make a better build scheme. [\#6500](https://github.com/ClickHouse/ClickHouse/pull/6500) ([Ivan](https://github.com/abyss7))
+- Fixed `test_external_dictionaries` integration in case it was executed under non root user. [\#6507](https://github.com/ClickHouse/ClickHouse/pull/6507) ([Nikolai Kochetov](https://github.com/KochetovNicolai))
+- The bug reproduces when total size of written packets exceeds `DBMS_DEFAULT_BUFFER_SIZE`. [\#6204](https://github.com/ClickHouse/ClickHouse/pull/6204) ([Yuriy Baranov](https://github.com/yurriy))
+- Added a test for `RENAME` table race condition [\#6752](https://github.com/ClickHouse/ClickHouse/pull/6752) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Avoid data race on Settings in `KILL QUERY`. [\#6753](https://github.com/ClickHouse/ClickHouse/pull/6753) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Add integration test for handling errors by a cache dictionary. [\#6755](https://github.com/ClickHouse/ClickHouse/pull/6755) ([Vitaly Baranov](https://github.com/vitlibar))
+- Disable parsing of ELF object files on Mac OS, because it makes no sense. [\#6578](https://github.com/ClickHouse/ClickHouse/pull/6578) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Attempt to make changelog generator better. [\#6327](https://github.com/ClickHouse/ClickHouse/pull/6327) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Adding `-Wshadow` switch to the GCC. [\#6325](https://github.com/ClickHouse/ClickHouse/pull/6325) ([kreuzerkrieg](https://github.com/kreuzerkrieg))
+- Removed obsolete code for `mimalloc` support. [\#6715](https://github.com/ClickHouse/ClickHouse/pull/6715) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- `zlib-ng` determines x86 capabilities and saves this info to global variables. This is done in defalteInit call, which may be made by different threads simultaneously. To avoid multithreaded writes, do it on library startup. [\#6141](https://github.com/ClickHouse/ClickHouse/pull/6141) ([akuzm](https://github.com/akuzm))
+- Regression test for a bug which in join which was fixed in [\#5192](https://github.com/ClickHouse/ClickHouse/issues/5192). [\#6147](https://github.com/ClickHouse/ClickHouse/pull/6147) ([Bakhtiyor Ruziev](https://github.com/theruziev))
+- Fixed MSan report. [\#6144](https://github.com/ClickHouse/ClickHouse/pull/6144) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Fix flapping TTL test. [\#6782](https://github.com/ClickHouse/ClickHouse/pull/6782) ([Anton Popov](https://github.com/CurtizJ))
+- Fixed false data race in `MergeTreeDataPart::is_frozen` field. [\#6583](https://github.com/ClickHouse/ClickHouse/pull/6583) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Fixed timeouts in fuzz test. In previous version, it managed to find false hangup in query `SELECT * FROM numbers_mt(gccMurmurHash(''))`. [\#6582](https://github.com/ClickHouse/ClickHouse/pull/6582) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Added debug checks to `static_cast` of columns. [\#6581](https://github.com/ClickHouse/ClickHouse/pull/6581) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Support for Oracle Linux in official RPM packages. [\#6356](https://github.com/ClickHouse/ClickHouse/issues/6356) [\#6585](https://github.com/ClickHouse/ClickHouse/pull/6585) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Changed json perftests from `once` to `loop` type. [\#6536](https://github.com/ClickHouse/ClickHouse/pull/6536) ([Nikolai Kochetov](https://github.com/KochetovNicolai))
+- `odbc-bridge.cpp` defines `main()` so it should not be included in `clickhouse-lib`. [\#6538](https://github.com/ClickHouse/ClickHouse/pull/6538) ([Orivej Desh](https://github.com/orivej))
+- Test for crash in `FULL|RIGHT JOIN` with nulls in right table’s keys. [\#6362](https://github.com/ClickHouse/ClickHouse/pull/6362) ([Artem Zuikov](https://github.com/4ertus2))
+- Added a test for the limit on expansion of aliases just in case. [\#6442](https://github.com/ClickHouse/ClickHouse/pull/6442) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Switched from `boost::filesystem` to `std::filesystem` where appropriate. [\#6253](https://github.com/ClickHouse/ClickHouse/pull/6253) [\#6385](https://github.com/ClickHouse/ClickHouse/pull/6385) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Added RPM packages to website. [\#6251](https://github.com/ClickHouse/ClickHouse/pull/6251) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Add a test for fixed `Unknown identifier` exception in `IN` section. [\#6708](https://github.com/ClickHouse/ClickHouse/pull/6708) ([Artem Zuikov](https://github.com/4ertus2))
+- Simplify `shared_ptr_helper` because people facing difficulties understanding it. [\#6675](https://github.com/ClickHouse/ClickHouse/pull/6675) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Added performance tests for fixed Gorilla and DoubleDelta codec. [\#6179](https://github.com/ClickHouse/ClickHouse/pull/6179) ([Vasily Nemkov](https://github.com/Enmk))
+- Split the integration test `test_dictionaries` into 4 separate tests. [\#6776](https://github.com/ClickHouse/ClickHouse/pull/6776) ([Vitaly Baranov](https://github.com/vitlibar))
+- Fix PVS-Studio warning in `PipelineExecutor`. [\#6777](https://github.com/ClickHouse/ClickHouse/pull/6777) ([Nikolai Kochetov](https://github.com/KochetovNicolai))
+- Allow to use `library` dictionary source with ASan. [\#6482](https://github.com/ClickHouse/ClickHouse/pull/6482) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Added option to generate changelog from a list of PRs. [\#6350](https://github.com/ClickHouse/ClickHouse/pull/6350) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Lock the `TinyLog` storage when reading. [\#6226](https://github.com/ClickHouse/ClickHouse/pull/6226) ([akuzm](https://github.com/akuzm))
+- Check for broken symlinks in CI. [\#6634](https://github.com/ClickHouse/ClickHouse/pull/6634) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Increase timeout for “stack overflow” test because it may take a long time in debug build. [\#6637](https://github.com/ClickHouse/ClickHouse/pull/6637) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Added a check for double whitespaces. [\#6643](https://github.com/ClickHouse/ClickHouse/pull/6643) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Fix `new/delete` memory tracking when build with sanitizers. Tracking is not clear. It only prevents memory limit exceptions in tests. [\#6450](https://github.com/ClickHouse/ClickHouse/pull/6450) ([Artem Zuikov](https://github.com/4ertus2))
+- Enable back the check of undefined symbols while linking. [\#6453](https://github.com/ClickHouse/ClickHouse/pull/6453) ([Ivan](https://github.com/abyss7))
+- Avoid rebuilding `hyperscan` every day. [\#6307](https://github.com/ClickHouse/ClickHouse/pull/6307) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Fixed UBSan report in `ProtobufWriter`. [\#6163](https://github.com/ClickHouse/ClickHouse/pull/6163) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Don’t allow to use query profiler with sanitizers because it is not compatible. [\#6769](https://github.com/ClickHouse/ClickHouse/pull/6769) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Add test for reloading a dictionary after fail by timer. [\#6114](https://github.com/ClickHouse/ClickHouse/pull/6114) ([Vitaly Baranov](https://github.com/vitlibar))
+- Fix inconsistency in `PipelineExecutor::prepareProcessor` argument type. [\#6494](https://github.com/ClickHouse/ClickHouse/pull/6494) ([Nikolai Kochetov](https://github.com/KochetovNicolai))
+- Added a test for bad URIs. [\#6493](https://github.com/ClickHouse/ClickHouse/pull/6493) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Added more checks to `CAST` function. This should get more information about segmentation fault in fuzzy test. [\#6346](https://github.com/ClickHouse/ClickHouse/pull/6346) ([Nikolai Kochetov](https://github.com/KochetovNicolai))
+- Added `gcc-9` support to `docker/builder` container that builds image locally. [\#6333](https://github.com/ClickHouse/ClickHouse/pull/6333) ([Gleb Novikov](https://github.com/NanoBjorn))
+- Test for primary key with `LowCardinality(String)`. [\#5044](https://github.com/ClickHouse/ClickHouse/issues/5044) [\#6219](https://github.com/ClickHouse/ClickHouse/pull/6219) ([dimarub2000](https://github.com/dimarub2000))
+- Fixed tests affected by slow stack traces printing. [\#6315](https://github.com/ClickHouse/ClickHouse/pull/6315) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Add a test case for crash in `groupUniqArray` fixed in [\#6029](https://github.com/ClickHouse/ClickHouse/pull/6029). [\#4402](https://github.com/ClickHouse/ClickHouse/issues/4402) [\#6129](https://github.com/ClickHouse/ClickHouse/pull/6129) ([akuzm](https://github.com/akuzm))
+- Fixed indices mutations tests. [\#6645](https://github.com/ClickHouse/ClickHouse/pull/6645) ([Nikita Vasilev](https://github.com/nikvas0))
+- In performance test, do not read query log for queries we didn’t run. [\#6427](https://github.com/ClickHouse/ClickHouse/pull/6427) ([akuzm](https://github.com/akuzm))
+- Materialized view now could be created with any low cardinality types regardless to the setting about suspicious low cardinality types. [\#6428](https://github.com/ClickHouse/ClickHouse/pull/6428) ([Olga Khvostikova](https://github.com/stavrolia))
+- Updated tests for `send_logs_level` setting. [\#6207](https://github.com/ClickHouse/ClickHouse/pull/6207) ([Nikolai Kochetov](https://github.com/KochetovNicolai))
+- Fix build under gcc-8.2. [\#6196](https://github.com/ClickHouse/ClickHouse/pull/6196) ([Max Akhmedov](https://github.com/zlobober))
+- Fix build with internal libc++. [\#6724](https://github.com/ClickHouse/ClickHouse/pull/6724) ([Ivan](https://github.com/abyss7))
+- Fix shared build with `rdkafka` library [\#6101](https://github.com/ClickHouse/ClickHouse/pull/6101) ([Ivan](https://github.com/abyss7))
+- Fixes for Mac OS build (incomplete). [\#6390](https://github.com/ClickHouse/ClickHouse/pull/6390) ([alexey-milovidov](https://github.com/alexey-milovidov)) [\#6429](https://github.com/ClickHouse/ClickHouse/pull/6429) ([alex-zaitsev](https://github.com/alex-zaitsev))
+- Fix “splitted” build. [\#6618](https://github.com/ClickHouse/ClickHouse/pull/6618) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Other build fixes: [\#6186](https://github.com/ClickHouse/ClickHouse/pull/6186) ([Amos Bird](https://github.com/amosbird)) [\#6486](https://github.com/ClickHouse/ClickHouse/pull/6486) [\#6348](https://github.com/ClickHouse/ClickHouse/pull/6348) ([vxider](https://github.com/Vxider)) [\#6744](https://github.com/ClickHouse/ClickHouse/pull/6744) ([Ivan](https://github.com/abyss7)) [\#6016](https://github.com/ClickHouse/ClickHouse/pull/6016) [\#6421](https://github.com/ClickHouse/ClickHouse/pull/6421) [\#6491](https://github.com/ClickHouse/ClickHouse/pull/6491) ([proller](https://github.com/proller))
+
+#### Backward Incompatible Change {#backward-incompatible-change-3}
+
+- Removed rarely used table function `catBoostPool` and storage `CatBoostPool`. If you have used this table function, please write email to `clickhouse-feedback@yandex-team.com`. Note that CatBoost integration remains and will be supported. [\#6279](https://github.com/ClickHouse/ClickHouse/pull/6279) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Disable `ANY RIGHT JOIN` and `ANY FULL JOIN` by default. Set `any_join_distinct_right_table_keys` setting to enable them. [\#5126](https://github.com/ClickHouse/ClickHouse/issues/5126) [\#6351](https://github.com/ClickHouse/ClickHouse/pull/6351) ([Artem Zuikov](https://github.com/4ertus2))
+
+## ClickHouse release 19.13 {#clickhouse-release-19-13}
+
+### ClickHouse release 19.13.6.51, 2019-10-02 {#clickhouse-release-19-13-6-51-2019-10-02}
+
+#### Bug Fix {#bug-fix-9}
+
+- This release also contains all bug fixes from 19.11.12.69.
+
+### ClickHouse release 19.13.5.44, 2019-09-20 {#clickhouse-release-19-13-5-44-2019-09-20}
+
+#### Bug Fix {#bug-fix-10}
+
+- This release also contains all bug fixes from 19.14.6.12.
+- Fixed possible inconsistent state of table while executing `DROP` query for replicated table while zookeeper is not accessible. [\#6045](https://github.com/ClickHouse/ClickHouse/issues/6045) [\#6413](https://github.com/ClickHouse/ClickHouse/pull/6413) ([Nikita Mikhaylov](https://github.com/nikitamikhaylov))
+- Fix for data race in StorageMerge [\#6717](https://github.com/ClickHouse/ClickHouse/pull/6717) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Fix bug introduced in query profiler which leads to endless recv from socket. [\#6386](https://github.com/ClickHouse/ClickHouse/pull/6386) ([alesapin](https://github.com/alesapin))
+- Fix excessive CPU usage while executing `JSONExtractRaw` function over a boolean value. [\#6208](https://github.com/ClickHouse/ClickHouse/pull/6208) ([Vitaly Baranov](https://github.com/vitlibar))
+- Fixes the regression while pushing to materialized view. [\#6415](https://github.com/ClickHouse/ClickHouse/pull/6415) ([Ivan](https://github.com/abyss7))
+- Table function `url` had the vulnerability allowed the attacker to inject arbitrary HTTP headers in the request. This issue was found by [Nikita Tikhomirov](https://github.com/NSTikhomirov). [\#6466](https://github.com/ClickHouse/ClickHouse/pull/6466) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Fix useless `AST` check in Set index. [\#6510](https://github.com/ClickHouse/ClickHouse/issues/6510) [\#6651](https://github.com/ClickHouse/ClickHouse/pull/6651) ([Nikita Vasilev](https://github.com/nikvas0))
+- Fixed parsing of `AggregateFunction` values embedded in query. [\#6575](https://github.com/ClickHouse/ClickHouse/issues/6575) [\#6773](https://github.com/ClickHouse/ClickHouse/pull/6773) ([Zhichang Yu](https://github.com/yuzhichang))
+- Fixed wrong behaviour of `trim` functions family. [\#6647](https://github.com/ClickHouse/ClickHouse/pull/6647) ([alexey-milovidov](https://github.com/alexey-milovidov))
+
+### ClickHouse release 19.13.4.32, 2019-09-10 {#clickhouse-release-19-13-4-32-2019-09-10}
+
+#### Bug Fix {#bug-fix-11}
+
+- This release also contains all bug security fixes from 19.11.9.52 and 19.11.10.54.
+- Fixed data race in `system.parts` table and `ALTER` query. [\#6245](https://github.com/ClickHouse/ClickHouse/issues/6245) [\#6513](https://github.com/ClickHouse/ClickHouse/pull/6513) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Fixed mismatched header in streams happened in case of reading from empty distributed table with sample and prewhere. [\#6167](https://github.com/ClickHouse/ClickHouse/issues/6167) ([Lixiang Qian](https://github.com/fancyqlx)) [\#6823](https://github.com/ClickHouse/ClickHouse/pull/6823) ([Nikolai Kochetov](https://github.com/KochetovNicolai))
+- Fixed crash when using `IN` clause with a subquery with a tuple. [\#6125](https://github.com/ClickHouse/ClickHouse/issues/6125) [\#6550](https://github.com/ClickHouse/ClickHouse/pull/6550) ([tavplubix](https://github.com/tavplubix))
+- Fix case with same column names in `GLOBAL JOIN ON` section. [\#6181](https://github.com/ClickHouse/ClickHouse/pull/6181) ([Artem Zuikov](https://github.com/4ertus2))
+- Fix crash when casting types to `Decimal` that do not support it. Throw exception instead. [\#6297](https://github.com/ClickHouse/ClickHouse/pull/6297) ([Artem Zuikov](https://github.com/4ertus2))
+- Fixed crash in `extractAll()` function. [\#6644](https://github.com/ClickHouse/ClickHouse/pull/6644) ([Artem Zuikov](https://github.com/4ertus2))
+- Query transformation for `MySQL`, `ODBC`, `JDBC` table functions now works properly for `SELECT WHERE` queries with multiple `AND` expressions. [\#6381](https://github.com/ClickHouse/ClickHouse/issues/6381) [\#6676](https://github.com/ClickHouse/ClickHouse/pull/6676) ([dimarub2000](https://github.com/dimarub2000))
+- Added previous declaration checks for MySQL 8 integration. [\#6569](https://github.com/ClickHouse/ClickHouse/pull/6569) ([Rafael David Tinoco](https://github.com/rafaeldtinoco))
+
+#### Security Fix {#security-fix-1}
+
+- Fix two vulnerabilities in codecs in decompression phase (malicious user can fabricate compressed data that will lead to buffer overflow in decompression). [\#6670](https://github.com/ClickHouse/ClickHouse/pull/6670) ([Artem Zuikov](https://github.com/4ertus2))
+
+### ClickHouse release 19.13.3.26, 2019-08-22 {#clickhouse-release-19-13-3-26-2019-08-22}
+
+#### Bug Fix {#bug-fix-12}
+
+- Fix `ALTER TABLE ... UPDATE` query for tables with `enable_mixed_granularity_parts=1`. [\#6543](https://github.com/ClickHouse/ClickHouse/pull/6543) ([alesapin](https://github.com/alesapin))
+- Fix NPE when using IN clause with a subquery with a tuple. [\#6125](https://github.com/ClickHouse/ClickHouse/issues/6125) [\#6550](https://github.com/ClickHouse/ClickHouse/pull/6550) ([tavplubix](https://github.com/tavplubix))
+- Fixed an issue that if a stale replica becomes alive, it may still have data parts that were removed by DROP PARTITION. [\#6522](https://github.com/ClickHouse/ClickHouse/issues/6522) [\#6523](https://github.com/ClickHouse/ClickHouse/pull/6523) ([tavplubix](https://github.com/tavplubix))
+- Fixed issue with parsing CSV [\#6426](https://github.com/ClickHouse/ClickHouse/issues/6426) [\#6559](https://github.com/ClickHouse/ClickHouse/pull/6559) ([tavplubix](https://github.com/tavplubix))
+- Fixed data race in system.parts table and ALTER query. This fixes [\#6245](https://github.com/ClickHouse/ClickHouse/issues/6245). [\#6513](https://github.com/ClickHouse/ClickHouse/pull/6513) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Fixed wrong code in mutations that may lead to memory corruption. Fixed segfault with read of address `0x14c0` that may happed due to concurrent `DROP TABLE` and `SELECT` from `system.parts` or `system.parts_columns`. Fixed race condition in preparation of mutation queries. Fixed deadlock caused by `OPTIMIZE` of Replicated tables and concurrent modification operations like ALTERs. [\#6514](https://github.com/ClickHouse/ClickHouse/pull/6514) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Fixed possible data loss after `ALTER DELETE` query on table with skipping index. [\#6224](https://github.com/ClickHouse/ClickHouse/issues/6224) [\#6282](https://github.com/ClickHouse/ClickHouse/pull/6282) ([Nikita Vasilev](https://github.com/nikvas0))
+
+#### Security Fix {#security-fix-2}
+
+- If the attacker has write access to ZooKeeper and is able to run custom server available from the network where ClickHouse run, it can create custom-built malicious server that will act as ClickHouse replica and register it in ZooKeeper. When another replica will fetch data part from malicious replica, it can force clickhouse-server to write to arbitrary path on filesystem. Found by Eldar Zaitov, information security team at Yandex. [\#6247](https://github.com/ClickHouse/ClickHouse/pull/6247) ([alexey-milovidov](https://github.com/alexey-milovidov))
+
+### ClickHouse release 19.13.2.19, 2019-08-14 {#clickhouse-release-19-13-2-19-2019-08-14}
+
+#### New Feature {#new-feature-5}
+
+- Sampling profiler on query level. [Example](https://gist.github.com/alexey-milovidov/92758583dd41c24c360fdb8d6a4da194). [\#4247](https://github.com/ClickHouse/ClickHouse/issues/4247) ([laplab](https://github.com/laplab)) [\#6124](https://github.com/ClickHouse/ClickHouse/pull/6124) ([alexey-milovidov](https://github.com/alexey-milovidov)) [\#6250](https://github.com/ClickHouse/ClickHouse/pull/6250) [\#6283](https://github.com/ClickHouse/ClickHouse/pull/6283) [\#6386](https://github.com/ClickHouse/ClickHouse/pull/6386)
+- Allow to specify a list of columns with `COLUMNS('regexp')` expression that works like a more sophisticated variant of `*` asterisk. [\#5951](https://github.com/ClickHouse/ClickHouse/pull/5951) ([mfridental](https://github.com/mfridental)), ([alexey-milovidov](https://github.com/alexey-milovidov))
+- `CREATE TABLE AS table_function()` is now possible [\#6057](https://github.com/ClickHouse/ClickHouse/pull/6057) ([dimarub2000](https://github.com/dimarub2000))
+- Adam optimizer for stochastic gradient descent is used by default in `stochasticLinearRegression()` and `stochasticLogisticRegression()` aggregate functions, because it shows good quality without almost any tuning. [\#6000](https://github.com/ClickHouse/ClickHouse/pull/6000) ([Quid37](https://github.com/Quid37))
+- Added functions for working with the сustom week number [\#5212](https://github.com/ClickHouse/ClickHouse/pull/5212) ([Andy Yang](https://github.com/andyyzh))
+- `RENAME` queries now work with all storages. [\#5953](https://github.com/ClickHouse/ClickHouse/pull/5953) ([Ivan](https://github.com/abyss7))
+- Now client receive logs from server with any desired level by setting `send_logs_level` regardless to the log level specified in server settings. [\#5964](https://github.com/ClickHouse/ClickHouse/pull/5964) ([Nikita Mikhaylov](https://github.com/nikitamikhaylov))
+
+#### Backward Incompatible Change {#backward-incompatible-change-4}
+
+- The setting `input_format_defaults_for_omitted_fields` is enabled by default. Inserts in Distributed tables need this setting to be the same on cluster (you need to set it before rolling update). It enables calculation of complex default expressions for omitted fields in `JSONEachRow` and `CSV*` formats. It should be the expected behavior but may lead to negligible performance difference. [\#6043](https://github.com/ClickHouse/ClickHouse/pull/6043) ([Artem Zuikov](https://github.com/4ertus2)), [\#5625](https://github.com/ClickHouse/ClickHouse/pull/5625) ([akuzm](https://github.com/akuzm))
+
+#### Experimental features {#experimental-features}
+
+- New query processing pipeline. Use `experimental_use_processors=1` option to enable it. Use for your own trouble. [\#4914](https://github.com/ClickHouse/ClickHouse/pull/4914) ([Nikolai Kochetov](https://github.com/KochetovNicolai))
+
+#### Bug Fix {#bug-fix-13}
+
+- Kafka integration has been fixed in this version.
+- Fixed `DoubleDelta` encoding of `Int64` for large `DoubleDelta` values, improved `DoubleDelta` encoding for random data for `Int32`. [\#5998](https://github.com/ClickHouse/ClickHouse/pull/5998) ([Vasily Nemkov](https://github.com/Enmk))
+- Fixed overestimation of `max_rows_to_read` if the setting `merge_tree_uniform_read_distribution` is set to 0. [\#6019](https://github.com/ClickHouse/ClickHouse/pull/6019) ([alexey-milovidov](https://github.com/alexey-milovidov))
+
+#### Improvement {#improvement-4}
+
+- Throws an exception if `config.d` file doesn’t have the corresponding root element as the config file [\#6123](https://github.com/ClickHouse/ClickHouse/pull/6123) ([dimarub2000](https://github.com/dimarub2000))
+
+#### Performance Improvement {#performance-improvement-3}
+
+- Optimize `count()`. Now it uses the smallest column (if possible). [\#6028](https://github.com/ClickHouse/ClickHouse/pull/6028) ([Amos Bird](https://github.com/amosbird))
+
+#### Build/Testing/Packaging Improvement {#buildtestingpackaging-improvement-5}
+
+- Report memory usage in performance tests. [\#5899](https://github.com/ClickHouse/ClickHouse/pull/5899) ([akuzm](https://github.com/akuzm))
+- Fix build with external `libcxx` [\#6010](https://github.com/ClickHouse/ClickHouse/pull/6010) ([Ivan](https://github.com/abyss7))
+- Fix shared build with `rdkafka` library [\#6101](https://github.com/ClickHouse/ClickHouse/pull/6101) ([Ivan](https://github.com/abyss7))
+
+## ClickHouse release 19.11 {#clickhouse-release-19-11}
+
+### ClickHouse release 19.11.13.74, 2019-11-01 {#clickhouse-release-19-11-13-74-2019-11-01}
+
+#### Bug Fix {#bug-fix-14}
+
+- Fixed rare crash in `ALTER MODIFY COLUMN` and vertical merge when one of merged/altered parts is empty (0 rows). [\#6780](https://github.com/ClickHouse/ClickHouse/pull/6780) ([alesapin](https://github.com/alesapin))
+- Manual update of `SIMDJSON`. This fixes possible flooding of stderr files with bogus json diagnostic messages. [\#7548](https://github.com/ClickHouse/ClickHouse/pull/7548) ([Alexander Kazakov](https://github.com/Akazz))
+- Fixed bug with `mrk` file extension for mutations ([alesapin](https://github.com/alesapin))
+
+### ClickHouse release 19.11.12.69, 2019-10-02 {#clickhouse-release-19-11-12-69-2019-10-02}
+
+#### Bug Fix {#bug-fix-15}
+
+- Fixed performance degradation of index analysis on complex keys on large tables. This fixes [\#6924](https://github.com/ClickHouse/ClickHouse/issues/6924). [\#7075](https://github.com/ClickHouse/ClickHouse/pull/7075) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Avoid rare SIGSEGV while sending data in tables with Distributed engine (`Failed to send batch: file with index XXXXX is absent`). [\#7032](https://github.com/ClickHouse/ClickHouse/pull/7032) ([Azat Khuzhin](https://github.com/azat))
+- Fix `Unknown identifier` with multiple joins. This fixes [\#5254](https://github.com/ClickHouse/ClickHouse/issues/5254). [\#7022](https://github.com/ClickHouse/ClickHouse/pull/7022) ([Artem Zuikov](https://github.com/4ertus2))
+
+### ClickHouse release 19.11.11.57, 2019-09-13 {#clickhouse-release-19-11-11-57-2019-09-13}
+
+- Fix logical error causing segfaults when selecting from Kafka empty topic. [\#6902](https://github.com/ClickHouse/ClickHouse/issues/6902) [\#6909](https://github.com/ClickHouse/ClickHouse/pull/6909) ([Ivan](https://github.com/abyss7))
+- Fix for function `АrrayEnumerateUniqRanked` with empty arrays in params. [\#6928](https://github.com/ClickHouse/ClickHouse/pull/6928) ([proller](https://github.com/proller))
+
+### ClickHouse release 19.11.10.54, 2019-09-10 {#clickhouse-release-19-11-10-54-2019-09-10}
+
+#### Bug Fix {#bug-fix-16}
+
+- Do store offsets for Kafka messages manually to be able to commit them all at once for all partitions. Fixes potential duplication in “one consumer - many partitions” scenario. [\#6872](https://github.com/ClickHouse/ClickHouse/pull/6872) ([Ivan](https://github.com/abyss7))
+
+### ClickHouse release 19.11.9.52, 2019-09-6 {#clickhouse-release-19-11-9-52-2019-09-6}
+
+- Improve error handling in cache dictionaries. [\#6737](https://github.com/ClickHouse/ClickHouse/pull/6737) ([Vitaly Baranov](https://github.com/vitlibar))
+- Fixed bug in function `arrayEnumerateUniqRanked`. [\#6779](https://github.com/ClickHouse/ClickHouse/pull/6779) ([proller](https://github.com/proller))
+- Fix `JSONExtract` function while extracting a `Tuple` from JSON. [\#6718](https://github.com/ClickHouse/ClickHouse/pull/6718) ([Vitaly Baranov](https://github.com/vitlibar))
+- Fixed possible data loss after `ALTER DELETE` query on table with skipping index. [\#6224](https://github.com/ClickHouse/ClickHouse/issues/6224) [\#6282](https://github.com/ClickHouse/ClickHouse/pull/6282) ([Nikita Vasilev](https://github.com/nikvas0))
+- Fixed performance test. [\#6392](https://github.com/ClickHouse/ClickHouse/pull/6392) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Parquet: Fix reading boolean columns. [\#6579](https://github.com/ClickHouse/ClickHouse/pull/6579) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Fixed wrong behaviour of `nullIf` function for constant arguments. [\#6518](https://github.com/ClickHouse/ClickHouse/pull/6518) ([Guillaume Tassery](https://github.com/YiuRULE)) [\#6580](https://github.com/ClickHouse/ClickHouse/pull/6580) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Fix Kafka messages duplication problem on normal server restart. [\#6597](https://github.com/ClickHouse/ClickHouse/pull/6597) ([Ivan](https://github.com/abyss7))
+- Fixed an issue when long `ALTER UPDATE` or `ALTER DELETE` may prevent regular merges to run. Prevent mutations from executing if there is no enough free threads available. [\#6502](https://github.com/ClickHouse/ClickHouse/issues/6502) [\#6617](https://github.com/ClickHouse/ClickHouse/pull/6617) ([tavplubix](https://github.com/tavplubix))
+- Fixed error with processing “timezone” in server configuration file. [\#6709](https://github.com/ClickHouse/ClickHouse/pull/6709) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Fix kafka tests. [\#6805](https://github.com/ClickHouse/ClickHouse/pull/6805) ([Ivan](https://github.com/abyss7))
+
+#### Security Fix {#security-fix-3}
+
+- If the attacker has write access to ZooKeeper and is able to run custom server available from the network where ClickHouse runs, it can create custom-built malicious server that will act as ClickHouse replica and register it in ZooKeeper. When another replica will fetch data part from malicious replica, it can force clickhouse-server to write to arbitrary path on filesystem. Found by Eldar Zaitov, information security team at Yandex. [\#6247](https://github.com/ClickHouse/ClickHouse/pull/6247) ([alexey-milovidov](https://github.com/alexey-milovidov))
+
+### ClickHouse release 19.11.8.46, 2019-08-22 {#clickhouse-release-19-11-8-46-2019-08-22}
+
+#### Bug Fix {#bug-fix-17}
+
+- Fix `ALTER TABLE ... UPDATE` query for tables with `enable_mixed_granularity_parts=1`. [\#6543](https://github.com/ClickHouse/ClickHouse/pull/6543) ([alesapin](https://github.com/alesapin))
+- Fix NPE when using IN clause with a subquery with a tuple. [\#6125](https://github.com/ClickHouse/ClickHouse/issues/6125) [\#6550](https://github.com/ClickHouse/ClickHouse/pull/6550) ([tavplubix](https://github.com/tavplubix))
+- Fixed an issue that if a stale replica becomes alive, it may still have data parts that were removed by DROP PARTITION. [\#6522](https://github.com/ClickHouse/ClickHouse/issues/6522) [\#6523](https://github.com/ClickHouse/ClickHouse/pull/6523) ([tavplubix](https://github.com/tavplubix))
+- Fixed issue with parsing CSV [\#6426](https://github.com/ClickHouse/ClickHouse/issues/6426) [\#6559](https://github.com/ClickHouse/ClickHouse/pull/6559) ([tavplubix](https://github.com/tavplubix))
+- Fixed data race in system.parts table and ALTER query. This fixes [\#6245](https://github.com/ClickHouse/ClickHouse/issues/6245). [\#6513](https://github.com/ClickHouse/ClickHouse/pull/6513) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Fixed wrong code in mutations that may lead to memory corruption. Fixed segfault with read of address `0x14c0` that may happed due to concurrent `DROP TABLE` and `SELECT` from `system.parts` or `system.parts_columns`. Fixed race condition in preparation of mutation queries. Fixed deadlock caused by `OPTIMIZE` of Replicated tables and concurrent modification operations like ALTERs. [\#6514](https://github.com/ClickHouse/ClickHouse/pull/6514) ([alexey-milovidov](https://github.com/alexey-milovidov))
+
+### ClickHouse release 19.11.7.40, 2019-08-14 {#clickhouse-release-19-11-7-40-2019-08-14}
+
+#### Bug fix {#bug-fix-18}
+
+- Kafka integration has been fixed in this version.
+- Fix segfault when using `arrayReduce` for constant arguments. [\#6326](https://github.com/ClickHouse/ClickHouse/pull/6326) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Fixed `toFloat()` monotonicity. [\#6374](https://github.com/ClickHouse/ClickHouse/pull/6374) ([dimarub2000](https://github.com/dimarub2000))
+- Fix segfault with enabled `optimize_skip_unused_shards` and missing sharding key. [\#6384](https://github.com/ClickHouse/ClickHouse/pull/6384) ([CurtizJ](https://github.com/CurtizJ))
+- Fixed logic of `arrayEnumerateUniqRanked` function. [\#6423](https://github.com/ClickHouse/ClickHouse/pull/6423) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Removed extra verbose logging from MySQL handler. [\#6389](https://github.com/ClickHouse/ClickHouse/pull/6389) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Fix wrong behavior and possible segfaults in `topK` and `topKWeighted` aggregated functions. [\#6404](https://github.com/ClickHouse/ClickHouse/pull/6404) ([CurtizJ](https://github.com/CurtizJ))
+- Do not expose virtual columns in `system.columns` table. This is required for backward compatibility. [\#6406](https://github.com/ClickHouse/ClickHouse/pull/6406) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Fix bug with memory allocation for string fields in complex key cache dictionary. [\#6447](https://github.com/ClickHouse/ClickHouse/pull/6447) ([alesapin](https://github.com/alesapin))
+- Fix bug with enabling adaptive granularity when creating new replica for `Replicated*MergeTree` table. [\#6452](https://github.com/ClickHouse/ClickHouse/pull/6452) ([alesapin](https://github.com/alesapin))
+- Fix infinite loop when reading Kafka messages. [\#6354](https://github.com/ClickHouse/ClickHouse/pull/6354) ([abyss7](https://github.com/abyss7))
+- Fixed the possibility of a fabricated query to cause server crash due to stack overflow in SQL parser and possibility of stack overflow in `Merge` and `Distributed` tables [\#6433](https://github.com/ClickHouse/ClickHouse/pull/6433) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Fixed Gorilla encoding error on small sequences. [\#6444](https://github.com/ClickHouse/ClickHouse/pull/6444) ([Enmk](https://github.com/Enmk))
+
+#### Improvement {#improvement-5}
+
+- Allow user to override `poll_interval` and `idle_connection_timeout` settings on connection. [\#6230](https://github.com/ClickHouse/ClickHouse/pull/6230) ([alexey-milovidov](https://github.com/alexey-milovidov))
+
+### ClickHouse release 19.11.5.28, 2019-08-05 {#clickhouse-release-19-11-5-28-2019-08-05}
+
+#### Bug fix {#bug-fix-19}
+
+- Fixed the possibility of hanging queries when server is overloaded. [\#6301](https://github.com/ClickHouse/ClickHouse/pull/6301) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Fix FPE in yandexConsistentHash function. This fixes [\#6304](https://github.com/ClickHouse/ClickHouse/issues/6304). [\#6126](https://github.com/ClickHouse/ClickHouse/pull/6126) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Fixed bug in conversion of `LowCardinality` types in `AggregateFunctionFactory`. This fixes [\#6257](https://github.com/ClickHouse/ClickHouse/issues/6257). [\#6281](https://github.com/ClickHouse/ClickHouse/pull/6281) ([Nikolai Kochetov](https://github.com/KochetovNicolai))
+- Fix parsing of `bool` settings from `true` and `false` strings in configuration files. [\#6278](https://github.com/ClickHouse/ClickHouse/pull/6278) ([alesapin](https://github.com/alesapin))
+- Fix rare bug with incompatible stream headers in queries to `Distributed` table over `MergeTree` table when part of `WHERE` moves to `PREWHERE`. [\#6236](https://github.com/ClickHouse/ClickHouse/pull/6236) ([alesapin](https://github.com/alesapin))
+- Fixed overflow in integer division of signed type to unsigned type. This fixes [\#6214](https://github.com/ClickHouse/ClickHouse/issues/6214). [\#6233](https://github.com/ClickHouse/ClickHouse/pull/6233) ([alexey-milovidov](https://github.com/alexey-milovidov))
+
+#### Backward Incompatible Change {#backward-incompatible-change-5}
+
+- `Kafka` still broken.
+
+### ClickHouse release 19.11.4.24, 2019-08-01 {#clickhouse-release-19-11-4-24-2019-08-01}
+
+#### Bug Fix {#bug-fix-20}
+
+- Fix bug with writing secondary indices marks with adaptive granularity. [\#6126](https://github.com/ClickHouse/ClickHouse/pull/6126) ([alesapin](https://github.com/alesapin))
+- Fix `WITH ROLLUP` and `WITH CUBE` modifiers of `GROUP BY` with two-level aggregation. [\#6225](https://github.com/ClickHouse/ClickHouse/pull/6225) ([Anton Popov](https://github.com/CurtizJ))
+- Fixed hang in `JSONExtractRaw` function. Fixed [\#6195](https://github.com/ClickHouse/ClickHouse/issues/6195) [\#6198](https://github.com/ClickHouse/ClickHouse/pull/6198) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Fix segfault in ExternalLoader::reloadOutdated(). [\#6082](https://github.com/ClickHouse/ClickHouse/pull/6082) ([Vitaly Baranov](https://github.com/vitlibar))
+- Fixed the case when server may close listening sockets but not shutdown and continue serving remaining queries. You may end up with two running clickhouse-server processes. Sometimes, the server may return an error `bad_function_call` for remaining queries. [\#6231](https://github.com/ClickHouse/ClickHouse/pull/6231) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Fixed useless and incorrect condition on update field for initial loading of external dictionaries via ODBC, MySQL, ClickHouse and HTTP. This fixes [\#6069](https://github.com/ClickHouse/ClickHouse/issues/6069) [\#6083](https://github.com/ClickHouse/ClickHouse/pull/6083) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Fixed irrelevant exception in cast of `LowCardinality(Nullable)` to not-Nullable column in case if it doesn’t contain Nulls (e.g. in query like `SELECT CAST(CAST('Hello' AS LowCardinality(Nullable(String))) AS String)`. [\#6094](https://github.com/ClickHouse/ClickHouse/issues/6094) [\#6119](https://github.com/ClickHouse/ClickHouse/pull/6119) ([Nikolai Kochetov](https://github.com/KochetovNicolai))
+- Fix non-deterministic result of “uniq” aggregate function in extreme rare cases. The bug was present in all ClickHouse versions. [\#6058](https://github.com/ClickHouse/ClickHouse/pull/6058) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Segfault when we set a little bit too high CIDR on the function `IPv6CIDRToRange`. [\#6068](https://github.com/ClickHouse/ClickHouse/pull/6068) ([Guillaume Tassery](https://github.com/YiuRULE))
+- Fixed small memory leak when server throw many exceptions from many different contexts. [\#6144](https://github.com/ClickHouse/ClickHouse/pull/6144) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Fix the situation when consumer got paused before subscription and not resumed afterwards. [\#6075](https://github.com/ClickHouse/ClickHouse/pull/6075) ([Ivan](https://github.com/abyss7)) Note that Kafka is broken in this version.
+- Clearing the Kafka data buffer from the previous read operation that was completed with an error [\#6026](https://github.com/ClickHouse/ClickHouse/pull/6026) ([Nikolay](https://github.com/bopohaa)) Note that Kafka is broken in this version.
+- Since `StorageMergeTree::background_task_handle` is initialized in `startup()` the `MergeTreeBlockOutputStream::write()` may try to use it before initialization. Just check if it is initialized. [\#6080](https://github.com/ClickHouse/ClickHouse/pull/6080) ([Ivan](https://github.com/abyss7))
+
+#### Build/Testing/Packaging Improvement {#buildtestingpackaging-improvement-6}
+
+- Added official `rpm` packages. [\#5740](https://github.com/ClickHouse/ClickHouse/pull/5740) ([proller](https://github.com/proller)) ([alesapin](https://github.com/alesapin))
+- Add an ability to build `.rpm` and `.tgz` packages with `packager` script. [\#5769](https://github.com/ClickHouse/ClickHouse/pull/5769) ([alesapin](https://github.com/alesapin))
+- Fixes for “Arcadia” build system. [\#6223](https://github.com/ClickHouse/ClickHouse/pull/6223) ([proller](https://github.com/proller))
+
+#### Backward Incompatible Change {#backward-incompatible-change-6}
+
+- `Kafka` is broken in this version.
+
+### ClickHouse release 19.11.3.11, 2019-07-18 {#clickhouse-release-19-11-3-11-2019-07-18}
+
+#### New Feature {#new-feature-6}
+
+- Added support for prepared statements. [\#5331](https://github.com/ClickHouse/ClickHouse/pull/5331/) ([Alexander](https://github.com/sanych73)) [\#5630](https://github.com/ClickHouse/ClickHouse/pull/5630) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- `DoubleDelta` and `Gorilla` column codecs [\#5600](https://github.com/ClickHouse/ClickHouse/pull/5600) ([Vasily Nemkov](https://github.com/Enmk))
+- Added `os_thread_priority` setting that allows to control the “nice” value of query processing threads that is used by OS to adjust dynamic scheduling priority. It requires `CAP_SYS_NICE` capabilities to work. This implements [\#5858](https://github.com/ClickHouse/ClickHouse/issues/5858) [\#5909](https://github.com/ClickHouse/ClickHouse/pull/5909) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Implement `_topic`, `_offset`, `_key` columns for Kafka engine [\#5382](https://github.com/ClickHouse/ClickHouse/pull/5382) ([Ivan](https://github.com/abyss7)) Note that Kafka is broken in this version.
+- Add aggregate function combinator `-Resample` [\#5590](https://github.com/ClickHouse/ClickHouse/pull/5590) ([hcz](https://github.com/hczhcz))
+- Aggregate functions `groupArrayMovingSum(win_size)(x)` and `groupArrayMovingAvg(win_size)(x)`, which calculate moving sum/avg with or without window-size limitation. [\#5595](https://github.com/ClickHouse/ClickHouse/pull/5595) ([inv2004](https://github.com/inv2004))
+- Add synonim `arrayFlatten` \<-\> `flatten` [\#5764](https://github.com/ClickHouse/ClickHouse/pull/5764) ([hcz](https://github.com/hczhcz))
+- Intergate H3 function `geoToH3` from Uber. [\#4724](https://github.com/ClickHouse/ClickHouse/pull/4724) ([Remen Ivan](https://github.com/BHYCHIK)) [\#5805](https://github.com/ClickHouse/ClickHouse/pull/5805) ([alexey-milovidov](https://github.com/alexey-milovidov))
+
+#### Bug Fix {#bug-fix-21}
+
+- Implement DNS cache with asynchronous update. Separate thread resolves all hosts and updates DNS cache with period (setting `dns_cache_update_period`). It should help, when ip of hosts changes frequently. [\#5857](https://github.com/ClickHouse/ClickHouse/pull/5857) ([Anton Popov](https://github.com/CurtizJ))
+- Fix segfault in `Delta` codec which affects columns with values less than 32 bits size. The bug led to random memory corruption. [\#5786](https://github.com/ClickHouse/ClickHouse/pull/5786) ([alesapin](https://github.com/alesapin))
+- Fix segfault in TTL merge with non-physical columns in block. [\#5819](https://github.com/ClickHouse/ClickHouse/pull/5819) ([Anton Popov](https://github.com/CurtizJ))
+- Fix rare bug in checking of part with `LowCardinality` column. Previously `checkDataPart` always fails for part with `LowCardinality` column. [\#5832](https://github.com/ClickHouse/ClickHouse/pull/5832) ([alesapin](https://github.com/alesapin))
+- Avoid hanging connections when server thread pool is full. It is important for connections from `remote` table function or connections to a shard without replicas when there is long connection timeout. This fixes [\#5878](https://github.com/ClickHouse/ClickHouse/issues/5878) [\#5881](https://github.com/ClickHouse/ClickHouse/pull/5881) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Support for constant arguments to `evalMLModel` function. This fixes [\#5817](https://github.com/ClickHouse/ClickHouse/issues/5817) [\#5820](https://github.com/ClickHouse/ClickHouse/pull/5820) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Fixed the issue when ClickHouse determines default time zone as `UCT` instead of `UTC`. This fixes [\#5804](https://github.com/ClickHouse/ClickHouse/issues/5804). [\#5828](https://github.com/ClickHouse/ClickHouse/pull/5828) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Fixed buffer underflow in `visitParamExtractRaw`. This fixes [\#5901](https://github.com/ClickHouse/ClickHouse/issues/5901) [\#5902](https://github.com/ClickHouse/ClickHouse/pull/5902) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Now distributed `DROP/ALTER/TRUNCATE/OPTIMIZE ON CLUSTER` queries will be executed directly on leader replica. [\#5757](https://github.com/ClickHouse/ClickHouse/pull/5757) ([alesapin](https://github.com/alesapin))
+- Fix `coalesce` for `ColumnConst` with `ColumnNullable` + related changes. [\#5755](https://github.com/ClickHouse/ClickHouse/pull/5755) ([Artem Zuikov](https://github.com/4ertus2))
+- Fix the `ReadBufferFromKafkaConsumer` so that it keeps reading new messages after `commit()` even if it was stalled before [\#5852](https://github.com/ClickHouse/ClickHouse/pull/5852) ([Ivan](https://github.com/abyss7))
+- Fix `FULL` and `RIGHT` JOIN results when joining on `Nullable` keys in right table. [\#5859](https://github.com/ClickHouse/ClickHouse/pull/5859) ([Artem Zuikov](https://github.com/4ertus2))
+- Possible fix of infinite sleeping of low-priority queries. [\#5842](https://github.com/ClickHouse/ClickHouse/pull/5842) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Fix race condition, which cause that some queries may not appear in query\_log after `SYSTEM FLUSH LOGS` query. [\#5456](https://github.com/ClickHouse/ClickHouse/issues/5456) [\#5685](https://github.com/ClickHouse/ClickHouse/pull/5685) ([Anton Popov](https://github.com/CurtizJ))
+- Fixed `heap-use-after-free` ASan warning in ClusterCopier caused by watch which try to use already removed copier object. [\#5871](https://github.com/ClickHouse/ClickHouse/pull/5871) ([Nikolai Kochetov](https://github.com/KochetovNicolai))
+- Fixed wrong `StringRef` pointer returned by some implementations of `IColumn::deserializeAndInsertFromArena`. This bug affected only unit-tests. [\#5973](https://github.com/ClickHouse/ClickHouse/pull/5973) ([Nikolai Kochetov](https://github.com/KochetovNicolai))
+- Prevent source and intermediate array join columns of masking same name columns. [\#5941](https://github.com/ClickHouse/ClickHouse/pull/5941) ([Artem Zuikov](https://github.com/4ertus2))
+- Fix insert and select query to MySQL engine with MySQL style identifier quoting. [\#5704](https://github.com/ClickHouse/ClickHouse/pull/5704) ([Winter Zhang](https://github.com/zhang2014))
+- Now `CHECK TABLE` query can work with MergeTree engine family. It returns check status and message if any for each part (or file in case of simplier engines). Also, fix bug in fetch of a broken part. [\#5865](https://github.com/ClickHouse/ClickHouse/pull/5865) ([alesapin](https://github.com/alesapin))
+- Fix SPLIT\_SHARED\_LIBRARIES runtime [\#5793](https://github.com/ClickHouse/ClickHouse/pull/5793) ([Danila Kutenin](https://github.com/danlark1))
+- Fixed time zone initialization when `/etc/localtime` is a relative symlink like `../usr/share/zoneinfo/Europe/Moscow` [\#5922](https://github.com/ClickHouse/ClickHouse/pull/5922) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- clickhouse-copier: Fix use-after free on shutdown [\#5752](https://github.com/ClickHouse/ClickHouse/pull/5752) ([proller](https://github.com/proller))
+- Updated `simdjson`. Fixed the issue that some invalid JSONs with zero bytes successfully parse. [\#5938](https://github.com/ClickHouse/ClickHouse/pull/5938) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Fix shutdown of SystemLogs [\#5802](https://github.com/ClickHouse/ClickHouse/pull/5802) ([Anton Popov](https://github.com/CurtizJ))
+- Fix hanging when condition in invalidate\_query depends on a dictionary. [\#6011](https://github.com/ClickHouse/ClickHouse/pull/6011) ([Vitaly Baranov](https://github.com/vitlibar))
+
+#### Improvement {#improvement-6}
+
+- Allow unresolvable addresses in cluster configuration. They will be considered unavailable and tried to resolve at every connection attempt. This is especially useful for Kubernetes. This fixes [\#5714](https://github.com/ClickHouse/ClickHouse/issues/5714) [\#5924](https://github.com/ClickHouse/ClickHouse/pull/5924) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Close idle TCP connections (with one hour timeout by default). This is especially important for large clusters with multiple distributed tables on every server, because every server can possibly keep a connection pool to every other server, and after peak query concurrency, connections will stall. This fixes [\#5879](https://github.com/ClickHouse/ClickHouse/issues/5879) [\#5880](https://github.com/ClickHouse/ClickHouse/pull/5880) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Better quality of `topK` function. Changed the SavingSpace set behavior to remove the last element if the new element have a bigger weight. [\#5833](https://github.com/ClickHouse/ClickHouse/issues/5833) [\#5850](https://github.com/ClickHouse/ClickHouse/pull/5850) ([Guillaume Tassery](https://github.com/YiuRULE))
+- URL functions to work with domains now can work for incomplete URLs without scheme [\#5725](https://github.com/ClickHouse/ClickHouse/pull/5725) ([alesapin](https://github.com/alesapin))
+- Checksums added to the `system.parts_columns` table. [\#5874](https://github.com/ClickHouse/ClickHouse/pull/5874) ([Nikita Mikhaylov](https://github.com/nikitamikhaylov))
+- Added `Enum` data type as a synonim for `Enum8` or `Enum16`. [\#5886](https://github.com/ClickHouse/ClickHouse/pull/5886) ([dimarub2000](https://github.com/dimarub2000))
+- Full bit transpose variant for `T64` codec. Could lead to better compression with `zstd`. [\#5742](https://github.com/ClickHouse/ClickHouse/pull/5742) ([Artem Zuikov](https://github.com/4ertus2))
+- Condition on `startsWith` function now can uses primary key. This fixes [\#5310](https://github.com/ClickHouse/ClickHouse/issues/5310) and [\#5882](https://github.com/ClickHouse/ClickHouse/issues/5882) [\#5919](https://github.com/ClickHouse/ClickHouse/pull/5919) ([dimarub2000](https://github.com/dimarub2000))
+- Allow to use `clickhouse-copier` with cross-replication cluster topology by permitting empty database name. [\#5745](https://github.com/ClickHouse/ClickHouse/pull/5745) ([nvartolomei](https://github.com/nvartolomei))
+- Use `UTC` as default timezone on a system without `tzdata` (e.g. bare Docker container). Before this patch, error message `Could not determine local time zone` was printed and server or client refused to start. [\#5827](https://github.com/ClickHouse/ClickHouse/pull/5827) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Returned back support for floating point argument in function `quantileTiming` for backward compatibility. [\#5911](https://github.com/ClickHouse/ClickHouse/pull/5911) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Show which table is missing column in error messages. [\#5768](https://github.com/ClickHouse/ClickHouse/pull/5768) ([Ivan](https://github.com/abyss7))
+- Disallow run query with same query\_id by various users [\#5430](https://github.com/ClickHouse/ClickHouse/pull/5430) ([proller](https://github.com/proller))
+- More robust code for sending metrics to Graphite. It will work even during long multiple `RENAME TABLE` operation. [\#5875](https://github.com/ClickHouse/ClickHouse/pull/5875) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- More informative error messages will be displayed when ThreadPool cannot schedule a task for execution. This fixes [\#5305](https://github.com/ClickHouse/ClickHouse/issues/5305) [\#5801](https://github.com/ClickHouse/ClickHouse/pull/5801) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Inverting ngramSearch to be more intuitive [\#5807](https://github.com/ClickHouse/ClickHouse/pull/5807) ([Danila Kutenin](https://github.com/danlark1))
+- Add user parsing in HDFS engine builder [\#5946](https://github.com/ClickHouse/ClickHouse/pull/5946) ([akonyaev90](https://github.com/akonyaev90))
+- Update default value of `max_ast_elements parameter` [\#5933](https://github.com/ClickHouse/ClickHouse/pull/5933) ([Artem Konovalov](https://github.com/izebit))
+- Added a notion of obsolete settings. The obsolete setting `allow_experimental_low_cardinality_type` can be used with no effect. [0f15c01c6802f7ce1a1494c12c846be8c98944cd](https://github.com/ClickHouse/ClickHouse/commit/0f15c01c6802f7ce1a1494c12c846be8c98944cd) [Alexey Milovidov](https://github.com/alexey-milovidov)
+
+#### Performance Improvement {#performance-improvement-4}
+
+- Increase number of streams to SELECT from Merge table for more uniform distribution of threads. Added setting `max_streams_multiplier_for_merge_tables`. This fixes [\#5797](https://github.com/ClickHouse/ClickHouse/issues/5797) [\#5915](https://github.com/ClickHouse/ClickHouse/pull/5915) ([alexey-milovidov](https://github.com/alexey-milovidov))
+
+#### Build/Testing/Packaging Improvement {#buildtestingpackaging-improvement-7}
+
+- Add a backward compatibility test for client-server interaction with different versions of clickhouse. [\#5868](https://github.com/ClickHouse/ClickHouse/pull/5868) ([alesapin](https://github.com/alesapin))
+- Test coverage information in every commit and pull request. [\#5896](https://github.com/ClickHouse/ClickHouse/pull/5896) ([alesapin](https://github.com/alesapin))
+- Cooperate with address sanitizer to support our custom allocators (`Arena` and `ArenaWithFreeLists`) for better debugging of “use-after-free” errors. [\#5728](https://github.com/ClickHouse/ClickHouse/pull/5728) ([akuzm](https://github.com/akuzm))
+- Switch to [LLVM libunwind implementation](https://github.com/llvm-mirror/libunwind) for C++ exception handling and for stack traces printing [\#4828](https://github.com/ClickHouse/ClickHouse/pull/4828) ([Nikita Lapkov](https://github.com/laplab))
+- Add two more warnings from -Weverything [\#5923](https://github.com/ClickHouse/ClickHouse/pull/5923) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Allow to build ClickHouse with Memory Sanitizer. [\#3949](https://github.com/ClickHouse/ClickHouse/pull/3949) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Fixed ubsan report about `bitTest` function in fuzz test. [\#5943](https://github.com/ClickHouse/ClickHouse/pull/5943) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Docker: added possibility to init a ClickHouse instance which requires authentication. [\#5727](https://github.com/ClickHouse/ClickHouse/pull/5727) ([Korviakov Andrey](https://github.com/shurshun))
+- Update librdkafka to version 1.1.0 [\#5872](https://github.com/ClickHouse/ClickHouse/pull/5872) ([Ivan](https://github.com/abyss7))
+- Add global timeout for integration tests and disable some of them in tests code. [\#5741](https://github.com/ClickHouse/ClickHouse/pull/5741) ([alesapin](https://github.com/alesapin))
+- Fix some ThreadSanitizer failures. [\#5854](https://github.com/ClickHouse/ClickHouse/pull/5854) ([akuzm](https://github.com/akuzm))
+- The `--no-undefined` option forces the linker to check all external names for existence while linking. It’s very useful to track real dependencies between libraries in the split build mode. [\#5855](https://github.com/ClickHouse/ClickHouse/pull/5855) ([Ivan](https://github.com/abyss7))
+- Added performance test for [\#5797](https://github.com/ClickHouse/ClickHouse/issues/5797) [\#5914](https://github.com/ClickHouse/ClickHouse/pull/5914) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Fixed compatibility with gcc-7. [\#5840](https://github.com/ClickHouse/ClickHouse/pull/5840) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Added support for gcc-9. This fixes [\#5717](https://github.com/ClickHouse/ClickHouse/issues/5717) [\#5774](https://github.com/ClickHouse/ClickHouse/pull/5774) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Fixed error when libunwind can be linked incorrectly. [\#5948](https://github.com/ClickHouse/ClickHouse/pull/5948) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Fixed a few warnings found by PVS-Studio. [\#5921](https://github.com/ClickHouse/ClickHouse/pull/5921) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Added initial support for `clang-tidy` static analyzer. [\#5806](https://github.com/ClickHouse/ClickHouse/pull/5806) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Convert BSD/Linux endian macros( ‘be64toh’ and ‘htobe64’) to the Mac OS X equivalents [\#5785](https://github.com/ClickHouse/ClickHouse/pull/5785) ([Fu Chen](https://github.com/fredchenbj))
+- Improved integration tests guide. [\#5796](https://github.com/ClickHouse/ClickHouse/pull/5796) ([Vladimir Chebotarev](https://github.com/excitoon))
+- Fixing build at macosx + gcc9 [\#5822](https://github.com/ClickHouse/ClickHouse/pull/5822) ([filimonov](https://github.com/filimonov))
+- Fix a hard-to-spot typo: aggreAGte -\> aggregate. [\#5753](https://github.com/ClickHouse/ClickHouse/pull/5753) ([akuzm](https://github.com/akuzm))
+- Fix freebsd build [\#5760](https://github.com/ClickHouse/ClickHouse/pull/5760) ([proller](https://github.com/proller))
+- Add link to experimental YouTube channel to website [\#5845](https://github.com/ClickHouse/ClickHouse/pull/5845) ([Ivan Blinkov](https://github.com/blinkov))
+- CMake: add option for coverage flags: WITH\_COVERAGE [\#5776](https://github.com/ClickHouse/ClickHouse/pull/5776) ([proller](https://github.com/proller))
+- Fix initial size of some inline PODArray’s. [\#5787](https://github.com/ClickHouse/ClickHouse/pull/5787) ([akuzm](https://github.com/akuzm))
+- clickhouse-server.postinst: fix os detection for centos 6 [\#5788](https://github.com/ClickHouse/ClickHouse/pull/5788) ([proller](https://github.com/proller))
+- Added Arch linux package generation. [\#5719](https://github.com/ClickHouse/ClickHouse/pull/5719) ([Vladimir Chebotarev](https://github.com/excitoon))
+- Split Common/config.h by libs (dbms) [\#5715](https://github.com/ClickHouse/ClickHouse/pull/5715) ([proller](https://github.com/proller))
+- Fixes for “Arcadia” build platform [\#5795](https://github.com/ClickHouse/ClickHouse/pull/5795) ([proller](https://github.com/proller))
+- Fixes for unconventional build (gcc9, no submodules) [\#5792](https://github.com/ClickHouse/ClickHouse/pull/5792) ([proller](https://github.com/proller))
+- Require explicit type in unalignedStore because it was proven to be bug-prone [\#5791](https://github.com/ClickHouse/ClickHouse/pull/5791) ([akuzm](https://github.com/akuzm))
+- Fixes MacOS build [\#5830](https://github.com/ClickHouse/ClickHouse/pull/5830) ([filimonov](https://github.com/filimonov))
+- Performance test concerning the new JIT feature with bigger dataset, as requested here [\#5263](https://github.com/ClickHouse/ClickHouse/issues/5263) [\#5887](https://github.com/ClickHouse/ClickHouse/pull/5887) ([Guillaume Tassery](https://github.com/YiuRULE))
+- Run stateful tests in stress test [12693e568722f11e19859742f56428455501fd2a](https://github.com/ClickHouse/ClickHouse/commit/12693e568722f11e19859742f56428455501fd2a) ([alesapin](https://github.com/alesapin))
+
+#### Backward Incompatible Change {#backward-incompatible-change-7}
+
+- `Kafka` is broken in this version.
+- Enable `adaptive_index_granularity` = 10MB by default for new `MergeTree` tables. If you created new MergeTree tables on version 19.11+, downgrade to versions prior to 19.6 will be impossible. [\#5628](https://github.com/ClickHouse/ClickHouse/pull/5628) ([alesapin](https://github.com/alesapin))
+- Removed obsolete undocumented embedded dictionaries that were used by Yandex.Metrica. The functions `OSIn`, `SEIn`, `OSToRoot`, `SEToRoot`, `OSHierarchy`, `SEHierarchy` are no longer available. If you are using these functions, write email to clickhouse-feedback@yandex-team.com. Note: at the last moment we decided to keep these functions for a while. [\#5780](https://github.com/ClickHouse/ClickHouse/pull/5780) ([alexey-milovidov](https://github.com/alexey-milovidov))
+
+## ClickHouse release 19.10 {#clickhouse-release-19-10}
+
+### ClickHouse release 19.10.1.5, 2019-07-12 {#clickhouse-release-19-10-1-5-2019-07-12}
+
+#### New Feature {#new-feature-7}
+
+- Add new column codec: `T64`. Made for (U)IntX/EnumX/Data(Time)/DecimalX columns. It should be good for columns with constant or small range values. Codec itself allows enlarge or shrink data type without re-compression. [\#5557](https://github.com/ClickHouse/ClickHouse/pull/5557) ([Artem Zuikov](https://github.com/4ertus2))
+- Add database engine `MySQL` that allow to view all the tables in remote MySQL server [\#5599](https://github.com/ClickHouse/ClickHouse/pull/5599) ([Winter Zhang](https://github.com/zhang2014))
+- `bitmapContains` implementation. It’s 2x faster than `bitmapHasAny` if the second bitmap contains one element. [\#5535](https://github.com/ClickHouse/ClickHouse/pull/5535) ([Zhichang Yu](https://github.com/yuzhichang))
+- Support for `crc32` function (with behaviour exactly as in MySQL or PHP). Do not use it if you need a hash function. [\#5661](https://github.com/ClickHouse/ClickHouse/pull/5661) ([Remen Ivan](https://github.com/BHYCHIK))
+- Implemented `SYSTEM START/STOP DISTRIBUTED SENDS` queries to control asynchronous inserts into `Distributed` tables. [\#4935](https://github.com/ClickHouse/ClickHouse/pull/4935) ([Winter Zhang](https://github.com/zhang2014))
+
+#### Bug Fix {#bug-fix-22}
+
+- Ignore query execution limits and max parts size for merge limits while executing mutations. [\#5659](https://github.com/ClickHouse/ClickHouse/pull/5659) ([Anton Popov](https://github.com/CurtizJ))
+- Fix bug which may lead to deduplication of normal blocks (extremely rare) and insertion of duplicate blocks (more often). [\#5549](https://github.com/ClickHouse/ClickHouse/pull/5549) ([alesapin](https://github.com/alesapin))
+- Fix of function `arrayEnumerateUniqRanked` for arguments with empty arrays [\#5559](https://github.com/ClickHouse/ClickHouse/pull/5559) ([proller](https://github.com/proller))
+- Don’t subscribe to Kafka topics without intent to poll any messages. [\#5698](https://github.com/ClickHouse/ClickHouse/pull/5698) ([Ivan](https://github.com/abyss7))
+- Make setting `join_use_nulls` get no effect for types that cannot be inside Nullable [\#5700](https://github.com/ClickHouse/ClickHouse/pull/5700) ([Olga Khvostikova](https://github.com/stavrolia))
+- Fixed `Incorrect size of index granularity` errors [\#5720](https://github.com/ClickHouse/ClickHouse/pull/5720) ([coraxster](https://github.com/coraxster))
+- Fix Float to Decimal convert overflow [\#5607](https://github.com/ClickHouse/ClickHouse/pull/5607) ([coraxster](https://github.com/coraxster))
+- Flush buffer when `WriteBufferFromHDFS`’s destructor is called. This fixes writing into `HDFS`. [\#5684](https://github.com/ClickHouse/ClickHouse/pull/5684) ([Xindong Peng](https://github.com/eejoin))
+
+#### Improvement {#improvement-7}
+
+- Treat empty cells in `CSV` as default values when the setting `input_format_defaults_for_omitted_fields` is enabled. [\#5625](https://github.com/ClickHouse/ClickHouse/pull/5625) ([akuzm](https://github.com/akuzm))
+- Non-blocking loading of external dictionaries. [\#5567](https://github.com/ClickHouse/ClickHouse/pull/5567) ([Vitaly Baranov](https://github.com/vitlibar))
+- Network timeouts can be dynamically changed for already established connections according to the settings. [\#4558](https://github.com/ClickHouse/ClickHouse/pull/4558) ([Konstantin Podshumok](https://github.com/podshumok))
+- Using “public\_suffix\_list” for functions `firstSignificantSubdomain`, `cutToFirstSignificantSubdomain`. It’s using a perfect hash table generated by `gperf` with a list generated from the file: https://publicsuffix.org/list/public\_suffix\_list.dat. (for example, now we recognize the domain `ac.uk` as non-significant). [\#5030](https://github.com/ClickHouse/ClickHouse/pull/5030) ([Guillaume Tassery](https://github.com/YiuRULE))
+- Adopted `IPv6` data type in system tables; unified client info columns in `system.processes` and `system.query_log` [\#5640](https://github.com/ClickHouse/ClickHouse/pull/5640) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Using sessions for connections with MySQL compatibility protocol. \#5476 [\#5646](https://github.com/ClickHouse/ClickHouse/pull/5646) ([Yuriy Baranov](https://github.com/yurriy))
+- Support more `ALTER` queries `ON CLUSTER`. [\#5593](https://github.com/ClickHouse/ClickHouse/pull/5593) [\#5613](https://github.com/ClickHouse/ClickHouse/pull/5613) ([sundyli](https://github.com/sundy-li))
+- Support `` section in `clickhouse-local` config file. [\#5540](https://github.com/ClickHouse/ClickHouse/pull/5540) ([proller](https://github.com/proller))
+- Allow run query with `remote` table function in `clickhouse-local` [\#5627](https://github.com/ClickHouse/ClickHouse/pull/5627) ([proller](https://github.com/proller))
+
+#### Performance Improvement {#performance-improvement-5}
+
+- Add the possibility to write the final mark at the end of MergeTree columns. It allows to avoid useless reads for keys that are out of table data range. It is enabled only if adaptive index granularity is in use. [\#5624](https://github.com/ClickHouse/ClickHouse/pull/5624) ([alesapin](https://github.com/alesapin))
+- Improved performance of MergeTree tables on very slow filesystems by reducing number of `stat` syscalls. [\#5648](https://github.com/ClickHouse/ClickHouse/pull/5648) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Fixed performance degradation in reading from MergeTree tables that was introduced in version 19.6. Fixes \#5631. [\#5633](https://github.com/ClickHouse/ClickHouse/pull/5633) ([alexey-milovidov](https://github.com/alexey-milovidov))
+
+#### Build/Testing/Packaging Improvement {#buildtestingpackaging-improvement-8}
+
+- Implemented `TestKeeper` as an implementation of ZooKeeper interface used for testing [\#5643](https://github.com/ClickHouse/ClickHouse/pull/5643) ([alexey-milovidov](https://github.com/alexey-milovidov)) ([levushkin aleksej](https://github.com/alexey-milovidov))
+- From now on `.sql` tests can be run isolated by server, in parallel, with random database. It allows to run them faster, add new tests with custom server configurations, and be sure that different tests doesn’t affect each other. [\#5554](https://github.com/ClickHouse/ClickHouse/pull/5554) ([Ivan](https://github.com/abyss7))
+- Remove `` and `` from performance tests [\#5672](https://github.com/ClickHouse/ClickHouse/pull/5672) ([Olga Khvostikova](https://github.com/stavrolia))
+- Fixed “select\_format” performance test for `Pretty` formats [\#5642](https://github.com/ClickHouse/ClickHouse/pull/5642) ([alexey-milovidov](https://github.com/alexey-milovidov))
+
+## ClickHouse release 19.9 {#clickhouse-release-19-9}
+
+### ClickHouse release 19.9.3.31, 2019-07-05 {#clickhouse-release-19-9-3-31-2019-07-05}
+
+#### Bug Fix {#bug-fix-23}
+
+- Fix segfault in Delta codec which affects columns with values less than 32 bits size. The bug led to random memory corruption. [\#5786](https://github.com/ClickHouse/ClickHouse/pull/5786) ([alesapin](https://github.com/alesapin))
+- Fix rare bug in checking of part with LowCardinality column. [\#5832](https://github.com/ClickHouse/ClickHouse/pull/5832) ([alesapin](https://github.com/alesapin))
+- Fix segfault in TTL merge with non-physical columns in block. [\#5819](https://github.com/ClickHouse/ClickHouse/pull/5819) ([Anton Popov](https://github.com/CurtizJ))
+- Fix potential infinite sleeping of low-priority queries. [\#5842](https://github.com/ClickHouse/ClickHouse/pull/5842) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Fix how ClickHouse determines default time zone as UCT instead of UTC. [\#5828](https://github.com/ClickHouse/ClickHouse/pull/5828) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Fix bug about executing distributed DROP/ALTER/TRUNCATE/OPTIMIZE ON CLUSTER queries on follower replica before leader replica. Now they will be executed directly on leader replica. [\#5757](https://github.com/ClickHouse/ClickHouse/pull/5757) ([alesapin](https://github.com/alesapin))
+- Fix race condition, which cause that some queries may not appear in query\_log instantly after SYSTEM FLUSH LOGS query. [\#5685](https://github.com/ClickHouse/ClickHouse/pull/5685) ([Anton Popov](https://github.com/CurtizJ))
+- Added missing support for constant arguments to `evalMLModel` function. [\#5820](https://github.com/ClickHouse/ClickHouse/pull/5820) ([alexey-milovidov](https://github.com/alexey-milovidov))
+
+### ClickHouse release 19.9.2.4, 2019-06-24 {#clickhouse-release-19-9-2-4-2019-06-24}
+
+#### New Feature {#new-feature-8}
+
+- Print information about frozen parts in `system.parts` table. [\#5471](https://github.com/ClickHouse/ClickHouse/pull/5471) ([proller](https://github.com/proller))
+- Ask client password on clickhouse-client start on tty if not set in arguments [\#5092](https://github.com/ClickHouse/ClickHouse/pull/5092) ([proller](https://github.com/proller))
+- Implement `dictGet` and `dictGetOrDefault` functions for Decimal types. [\#5394](https://github.com/ClickHouse/ClickHouse/pull/5394) ([Artem Zuikov](https://github.com/4ertus2))
+
+#### Improvement {#improvement-8}
+
+- Debian init: Add service stop timeout [\#5522](https://github.com/ClickHouse/ClickHouse/pull/5522) ([proller](https://github.com/proller))
+- Add setting forbidden by default to create table with suspicious types for LowCardinality [\#5448](https://github.com/ClickHouse/ClickHouse/pull/5448) ([Olga Khvostikova](https://github.com/stavrolia))
+- Regression functions return model weights when not used as State in function `evalMLMethod`. [\#5411](https://github.com/ClickHouse/ClickHouse/pull/5411) ([Quid37](https://github.com/Quid37))
+- Rename and improve regression methods. [\#5492](https://github.com/ClickHouse/ClickHouse/pull/5492) ([Quid37](https://github.com/Quid37))
+- Clearer interfaces of string searchers. [\#5586](https://github.com/ClickHouse/ClickHouse/pull/5586) ([Danila Kutenin](https://github.com/danlark1))
+
+#### Bug Fix {#bug-fix-24}
+
+- Fix potential data loss in Kafka [\#5445](https://github.com/ClickHouse/ClickHouse/pull/5445) ([Ivan](https://github.com/abyss7))
+- Fix potential infinite loop in `PrettySpace` format when called with zero columns [\#5560](https://github.com/ClickHouse/ClickHouse/pull/5560) ([Olga Khvostikova](https://github.com/stavrolia))
+- Fixed UInt32 overflow bug in linear models. Allow eval ML model for non-const model argument. [\#5516](https://github.com/ClickHouse/ClickHouse/pull/5516) ([Nikolai Kochetov](https://github.com/KochetovNicolai))
+- `ALTER TABLE ... DROP INDEX IF EXISTS ...` should not raise an exception if provided index does not exist [\#5524](https://github.com/ClickHouse/ClickHouse/pull/5524) ([Gleb Novikov](https://github.com/NanoBjorn))
+- Fix segfault with `bitmapHasAny` in scalar subquery [\#5528](https://github.com/ClickHouse/ClickHouse/pull/5528) ([Zhichang Yu](https://github.com/yuzhichang))
+- Fixed error when replication connection pool doesn’t retry to resolve host, even when DNS cache was dropped. [\#5534](https://github.com/ClickHouse/ClickHouse/pull/5534) ([alesapin](https://github.com/alesapin))
+- Fixed `ALTER ... MODIFY TTL` on ReplicatedMergeTree. [\#5539](https://github.com/ClickHouse/ClickHouse/pull/5539) ([Anton Popov](https://github.com/CurtizJ))
+- Fix INSERT into Distributed table with MATERIALIZED column [\#5429](https://github.com/ClickHouse/ClickHouse/pull/5429) ([Azat Khuzhin](https://github.com/azat))
+- Fix bad alloc when truncate Join storage [\#5437](https://github.com/ClickHouse/ClickHouse/pull/5437) ([TCeason](https://github.com/TCeason))
+- In recent versions of package tzdata some of files are symlinks now. The current mechanism for detecting default timezone gets broken and gives wrong names for some timezones. Now at least we force the timezone name to the contents of TZ if provided. [\#5443](https://github.com/ClickHouse/ClickHouse/pull/5443) ([Ivan](https://github.com/abyss7))
+- Fix some extremely rare cases with MultiVolnitsky searcher when the constant needles in sum are at least 16KB long. The algorithm missed or overwrote the previous results which can lead to the incorrect result of `multiSearchAny`. [\#5588](https://github.com/ClickHouse/ClickHouse/pull/5588) ([Danila Kutenin](https://github.com/danlark1))
+- Fix the issue when settings for ExternalData requests couldn’t use ClickHouse settings. Also, for now, settings `date_time_input_format` and `low_cardinality_allow_in_native_format` cannot be used because of the ambiguity of names (in external data it can be interpreted as table format and in the query it can be a setting). [\#5455](https://github.com/ClickHouse/ClickHouse/pull/5455) ([Danila Kutenin](https://github.com/danlark1))
+- Fix bug when parts were removed only from FS without dropping them from Zookeeper. [\#5520](https://github.com/ClickHouse/ClickHouse/pull/5520) ([alesapin](https://github.com/alesapin))
+- Remove debug logging from MySQL protocol [\#5478](https://github.com/ClickHouse/ClickHouse/pull/5478) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Skip ZNONODE during DDL query processing [\#5489](https://github.com/ClickHouse/ClickHouse/pull/5489) ([Azat Khuzhin](https://github.com/azat))
+- Fix mix `UNION ALL` result column type. There were cases with inconsistent data and column types of resulting columns. [\#5503](https://github.com/ClickHouse/ClickHouse/pull/5503) ([Artem Zuikov](https://github.com/4ertus2))
+- Throw an exception on wrong integers in `dictGetT` functions instead of crash. [\#5446](https://github.com/ClickHouse/ClickHouse/pull/5446) ([Artem Zuikov](https://github.com/4ertus2))
+- Fix wrong element\_count and load\_factor for hashed dictionary in `system.dictionaries` table. [\#5440](https://github.com/ClickHouse/ClickHouse/pull/5440) ([Azat Khuzhin](https://github.com/azat))
+
+#### Build/Testing/Packaging Improvement {#buildtestingpackaging-improvement-9}
+
+- Fixed build without `Brotli` HTTP compression support (`ENABLE_BROTLI=OFF` cmake variable). [\#5521](https://github.com/ClickHouse/ClickHouse/pull/5521) ([Anton Yuzhaninov](https://github.com/citrin))
+- Include roaring.h as roaring/roaring.h [\#5523](https://github.com/ClickHouse/ClickHouse/pull/5523) ([Orivej Desh](https://github.com/orivej))
+- Fix gcc9 warnings in hyperscan (\#line directive is evil!) [\#5546](https://github.com/ClickHouse/ClickHouse/pull/5546) ([Danila Kutenin](https://github.com/danlark1))
+- Fix all warnings when compiling with gcc-9. Fix some contrib issues. Fix gcc9 ICE and submit it to bugzilla. [\#5498](https://github.com/ClickHouse/ClickHouse/pull/5498) ([Danila Kutenin](https://github.com/danlark1))
+- Fixed linking with lld [\#5477](https://github.com/ClickHouse/ClickHouse/pull/5477) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Remove unused specializations in dictionaries [\#5452](https://github.com/ClickHouse/ClickHouse/pull/5452) ([Artem Zuikov](https://github.com/4ertus2))
+- Improvement performance tests for formatting and parsing tables for different types of files [\#5497](https://github.com/ClickHouse/ClickHouse/pull/5497) ([Olga Khvostikova](https://github.com/stavrolia))
+- Fixes for parallel test run [\#5506](https://github.com/ClickHouse/ClickHouse/pull/5506) ([proller](https://github.com/proller))
+- Docker: use configs from clickhouse-test [\#5531](https://github.com/ClickHouse/ClickHouse/pull/5531) ([proller](https://github.com/proller))
+- Fix compile for FreeBSD [\#5447](https://github.com/ClickHouse/ClickHouse/pull/5447) ([proller](https://github.com/proller))
+- Upgrade boost to 1.70 [\#5570](https://github.com/ClickHouse/ClickHouse/pull/5570) ([proller](https://github.com/proller))
+- Fix build clickhouse as submodule [\#5574](https://github.com/ClickHouse/ClickHouse/pull/5574) ([proller](https://github.com/proller))
+- Improve JSONExtract performance tests [\#5444](https://github.com/ClickHouse/ClickHouse/pull/5444) ([Vitaly Baranov](https://github.com/vitlibar))
+
+## ClickHouse release 19.8 {#clickhouse-release-19-8}
+
+### ClickHouse release 19.8.3.8, 2019-06-11 {#clickhouse-release-19-8-3-8-2019-06-11}
+
+#### New Features {#new-features}
+
+- Added functions to work with JSON [\#4686](https://github.com/ClickHouse/ClickHouse/pull/4686) ([hcz](https://github.com/hczhcz)) [\#5124](https://github.com/ClickHouse/ClickHouse/pull/5124). ([Vitaly Baranov](https://github.com/vitlibar))
+- Add a function basename, with a similar behaviour to a basename function, which exists in a lot of languages (`os.path.basename` in python, `basename` in PHP, etc…). Work with both an UNIX-like path or a Windows path. [\#5136](https://github.com/ClickHouse/ClickHouse/pull/5136) ([Guillaume Tassery](https://github.com/YiuRULE))
+- Added `LIMIT n, m BY` or `LIMIT m OFFSET n BY` syntax to set offset of n for LIMIT BY clause. [\#5138](https://github.com/ClickHouse/ClickHouse/pull/5138) ([Anton Popov](https://github.com/CurtizJ))
+- Added new data type `SimpleAggregateFunction`, which allows to have columns with light aggregation in an `AggregatingMergeTree`. This can only be used with simple functions like `any`, `anyLast`, `sum`, `min`, `max`. [\#4629](https://github.com/ClickHouse/ClickHouse/pull/4629) ([Boris Granveaud](https://github.com/bgranvea))
+- Added support for non-constant arguments in function `ngramDistance` [\#5198](https://github.com/ClickHouse/ClickHouse/pull/5198) ([Danila Kutenin](https://github.com/danlark1))
+- Added functions `skewPop`, `skewSamp`, `kurtPop` and `kurtSamp` to compute for sequence skewness, sample skewness, kurtosis and sample kurtosis respectively. [\#5200](https://github.com/ClickHouse/ClickHouse/pull/5200) ([hcz](https://github.com/hczhcz))
+- Support rename operation for `MaterializeView` storage. [\#5209](https://github.com/ClickHouse/ClickHouse/pull/5209) ([Guillaume Tassery](https://github.com/YiuRULE))
+- Added server which allows connecting to ClickHouse using MySQL client. [\#4715](https://github.com/ClickHouse/ClickHouse/pull/4715) ([Yuriy Baranov](https://github.com/yurriy))
+- Add `toDecimal*OrZero` and `toDecimal*OrNull` functions. [\#5291](https://github.com/ClickHouse/ClickHouse/pull/5291) ([Artem Zuikov](https://github.com/4ertus2))
+- Support Decimal types in functions: `quantile`, `quantiles`, `median`, `quantileExactWeighted`, `quantilesExactWeighted`, medianExactWeighted. [\#5304](https://github.com/ClickHouse/ClickHouse/pull/5304) ([Artem Zuikov](https://github.com/4ertus2))
+- Added `toValidUTF8` function, which replaces all invalid UTF-8 characters by replacement character � (U+FFFD). [\#5322](https://github.com/ClickHouse/ClickHouse/pull/5322) ([Danila Kutenin](https://github.com/danlark1))
+- Added `format` function. Formatting constant pattern (simplified Python format pattern) with the strings listed in the arguments. [\#5330](https://github.com/ClickHouse/ClickHouse/pull/5330) ([Danila Kutenin](https://github.com/danlark1))
+- Added `system.detached_parts` table containing information about detached parts of `MergeTree` tables. [\#5353](https://github.com/ClickHouse/ClickHouse/pull/5353) ([akuzm](https://github.com/akuzm))
+- Added `ngramSearch` function to calculate the non-symmetric difference between needle and haystack. [\#5418](https://github.com/ClickHouse/ClickHouse/pull/5418)[\#5422](https://github.com/ClickHouse/ClickHouse/pull/5422) ([Danila Kutenin](https://github.com/danlark1))
+- Implementation of basic machine learning methods (stochastic linear regression and logistic regression) using aggregate functions interface. Has different strategies for updating model weights (simple gradient descent, momentum method, Nesterov method). Also supports mini-batches of custom size. [\#4943](https://github.com/ClickHouse/ClickHouse/pull/4943) ([Quid37](https://github.com/Quid37))
+- Implementation of `geohashEncode` and `geohashDecode` functions. [\#5003](https://github.com/ClickHouse/ClickHouse/pull/5003) ([Vasily Nemkov](https://github.com/Enmk))
+- Added aggregate function `timeSeriesGroupSum`, which can aggregate different time series that sample timestamp not alignment. It will use linear interpolation between two sample timestamp and then sum time-series together. Added aggregate function `timeSeriesGroupRateSum`, which calculates the rate of time-series and then sum rates together. [\#4542](https://github.com/ClickHouse/ClickHouse/pull/4542) ([Yangkuan Liu](https://github.com/LiuYangkuan))
+- Added functions `IPv4CIDRtoIPv4Range` and `IPv6CIDRtoIPv6Range` to calculate the lower and higher bounds for an IP in the subnet using a CIDR. [\#5095](https://github.com/ClickHouse/ClickHouse/pull/5095) ([Guillaume Tassery](https://github.com/YiuRULE))
+- Add a X-ClickHouse-Summary header when we send a query using HTTP with enabled setting `send_progress_in_http_headers`. Return the usual information of X-ClickHouse-Progress, with additional information like how many rows and bytes were inserted in the query. [\#5116](https://github.com/ClickHouse/ClickHouse/pull/5116) ([Guillaume Tassery](https://github.com/YiuRULE))
+
+#### Improvements {#improvements}
+
+- Added `max_parts_in_total` setting for MergeTree family of tables (default: 100 000) that prevents unsafe specification of partition key \#5166. [\#5171](https://github.com/ClickHouse/ClickHouse/pull/5171) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- `clickhouse-obfuscator`: derive seed for individual columns by combining initial seed with column name, not column position. This is intended to transform datasets with multiple related tables, so that tables will remain JOINable after transformation. [\#5178](https://github.com/ClickHouse/ClickHouse/pull/5178) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Added functions `JSONExtractRaw`, `JSONExtractKeyAndValues`. Renamed functions `jsonExtract` to `JSONExtract`. When something goes wrong these functions return the correspondent values, not `NULL`. Modified function `JSONExtract`, now it gets the return type from its last parameter and doesn’t inject nullables. Implemented fallback to RapidJSON in case AVX2 instructions are not available. Simdjson library updated to a new version. [\#5235](https://github.com/ClickHouse/ClickHouse/pull/5235) ([Vitaly Baranov](https://github.com/vitlibar))
+- Now `if` and `multiIf` functions don’t rely on the condition’s `Nullable`, but rely on the branches for sql compatibility. [\#5238](https://github.com/ClickHouse/ClickHouse/pull/5238) ([Jian Wu](https://github.com/janplus))
+- `In` predicate now generates `Null` result from `Null` input like the `Equal` function. [\#5152](https://github.com/ClickHouse/ClickHouse/pull/5152) ([Jian Wu](https://github.com/janplus))
+- Check the time limit every (flush\_interval / poll\_timeout) number of rows from Kafka. This allows to break the reading from Kafka consumer more frequently and to check the time limits for the top-level streams [\#5249](https://github.com/ClickHouse/ClickHouse/pull/5249) ([Ivan](https://github.com/abyss7))
+- Link rdkafka with bundled SASL. It should allow to use SASL SCRAM authentication [\#5253](https://github.com/ClickHouse/ClickHouse/pull/5253) ([Ivan](https://github.com/abyss7))
+- Batched version of RowRefList for ALL JOINS. [\#5267](https://github.com/ClickHouse/ClickHouse/pull/5267) ([Artem Zuikov](https://github.com/4ertus2))
+- clickhouse-server: more informative listen error messages. [\#5268](https://github.com/ClickHouse/ClickHouse/pull/5268) ([proller](https://github.com/proller))
+- Support dictionaries in clickhouse-copier for functions in `` [\#5270](https://github.com/ClickHouse/ClickHouse/pull/5270) ([proller](https://github.com/proller))
+- Add new setting `kafka_commit_every_batch` to regulate Kafka committing policy.
+ It allows to set commit mode: after every batch of messages is handled, or after the whole block is written to the storage. It’s a trade-off between losing some messages or reading them twice in some extreme situations. [\#5308](https://github.com/ClickHouse/ClickHouse/pull/5308) ([Ivan](https://github.com/abyss7))
+- Make `windowFunnel` support other Unsigned Integer Types. [\#5320](https://github.com/ClickHouse/ClickHouse/pull/5320) ([sundyli](https://github.com/sundy-li))
+- Allow to shadow virtual column `_table` in Merge engine. [\#5325](https://github.com/ClickHouse/ClickHouse/pull/5325) ([Ivan](https://github.com/abyss7))
+- Make `sequenceMatch` aggregate functions support other unsigned Integer types [\#5339](https://github.com/ClickHouse/ClickHouse/pull/5339) ([sundyli](https://github.com/sundy-li))
+- Better error messages if checksum mismatch is most likely caused by hardware failures. [\#5355](https://github.com/ClickHouse/ClickHouse/pull/5355) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Check that underlying tables support sampling for `StorageMerge` [\#5366](https://github.com/ClickHouse/ClickHouse/pull/5366) ([Ivan](https://github.com/abyss7))
+- Сlose MySQL connections after their usage in external dictionaries. It is related to issue \#893. [\#5395](https://github.com/ClickHouse/ClickHouse/pull/5395) ([Clément Rodriguez](https://github.com/clemrodriguez))
+- Improvements of MySQL Wire Protocol. Changed name of format to MySQLWire. Using RAII for calling RSA\_free. Disabling SSL if context cannot be created. [\#5419](https://github.com/ClickHouse/ClickHouse/pull/5419) ([Yuriy Baranov](https://github.com/yurriy))
+- clickhouse-client: allow to run with unaccessable history file (read-only, no disk space, file is directory, …). [\#5431](https://github.com/ClickHouse/ClickHouse/pull/5431) ([proller](https://github.com/proller))
+- Respect query settings in asynchronous INSERTs into Distributed tables. [\#4936](https://github.com/ClickHouse/ClickHouse/pull/4936) ([TCeason](https://github.com/TCeason))
+- Renamed functions `leastSqr` to `simpleLinearRegression`, `LinearRegression` to `linearRegression`, `LogisticRegression` to `logisticRegression`. [\#5391](https://github.com/ClickHouse/ClickHouse/pull/5391) ([Nikolai Kochetov](https://github.com/KochetovNicolai))
+
+#### Performance Improvements {#performance-improvements}
+
+- Parallelize processing of parts of non-replicated MergeTree tables in ALTER MODIFY query. [\#4639](https://github.com/ClickHouse/ClickHouse/pull/4639) ([Ivan Kush](https://github.com/IvanKush))
+- Optimizations in regular expressions extraction. [\#5193](https://github.com/ClickHouse/ClickHouse/pull/5193) [\#5191](https://github.com/ClickHouse/ClickHouse/pull/5191) ([Danila Kutenin](https://github.com/danlark1))
+- Do not add right join key column to join result if it’s used only in join on section. [\#5260](https://github.com/ClickHouse/ClickHouse/pull/5260) ([Artem Zuikov](https://github.com/4ertus2))
+- Freeze the Kafka buffer after first empty response. It avoids multiple invokations of `ReadBuffer::next()` for empty result in some row-parsing streams. [\#5283](https://github.com/ClickHouse/ClickHouse/pull/5283) ([Ivan](https://github.com/abyss7))
+- `concat` function optimization for multiple arguments. [\#5357](https://github.com/ClickHouse/ClickHouse/pull/5357) ([Danila Kutenin](https://github.com/danlark1))
+- Query optimisation. Allow push down IN statement while rewriting commа/cross join into inner one. [\#5396](https://github.com/ClickHouse/ClickHouse/pull/5396) ([Artem Zuikov](https://github.com/4ertus2))
+- Upgrade our LZ4 implementation with reference one to have faster decompression. [\#5070](https://github.com/ClickHouse/ClickHouse/pull/5070) ([Danila Kutenin](https://github.com/danlark1))
+- Implemented MSD radix sort (based on kxsort), and partial sorting. [\#5129](https://github.com/ClickHouse/ClickHouse/pull/5129) ([Evgenii Pravda](https://github.com/kvinty))
+
+#### Bug Fixes {#bug-fixes}
+
+- Fix push require columns with join [\#5192](https://github.com/ClickHouse/ClickHouse/pull/5192) ([Winter Zhang](https://github.com/zhang2014))
+- Fixed bug, when ClickHouse is run by systemd, the command `sudo service clickhouse-server forcerestart` was not working as expected. [\#5204](https://github.com/ClickHouse/ClickHouse/pull/5204) ([proller](https://github.com/proller))
+- Fix http error codes in DataPartsExchange (interserver http server on 9009 port always returned code 200, even on errors). [\#5216](https://github.com/ClickHouse/ClickHouse/pull/5216) ([proller](https://github.com/proller))
+- Fix SimpleAggregateFunction for String longer than MAX\_SMALL\_STRING\_SIZE [\#5311](https://github.com/ClickHouse/ClickHouse/pull/5311) ([Azat Khuzhin](https://github.com/azat))
+- Fix error for `Decimal` to `Nullable(Decimal)` conversion in IN. Support other Decimal to Decimal conversions (including different scales). [\#5350](https://github.com/ClickHouse/ClickHouse/pull/5350) ([Artem Zuikov](https://github.com/4ertus2))
+- Fixed FPU clobbering in simdjson library that lead to wrong calculation of `uniqHLL` and `uniqCombined` aggregate function and math functions such as `log`. [\#5354](https://github.com/ClickHouse/ClickHouse/pull/5354) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Fixed handling mixed const/nonconst cases in JSON functions. [\#5435](https://github.com/ClickHouse/ClickHouse/pull/5435) ([Vitaly Baranov](https://github.com/vitlibar))
+- Fix `retention` function. Now all conditions that satisfy in a row of data are added to the data state. [\#5119](https://github.com/ClickHouse/ClickHouse/pull/5119) ([小路](https://github.com/nicelulu))
+- Fix result type for `quantileExact` with Decimals. [\#5304](https://github.com/ClickHouse/ClickHouse/pull/5304) ([Artem Zuikov](https://github.com/4ertus2))
+
+#### Documentation {#documentation}
+
+- Translate documentation for `CollapsingMergeTree` to chinese. [\#5168](https://github.com/ClickHouse/ClickHouse/pull/5168) ([张风啸](https://github.com/AlexZFX))
+- Translate some documentation about table engines to chinese.
+ [\#5134](https://github.com/ClickHouse/ClickHouse/pull/5134)
+ [\#5328](https://github.com/ClickHouse/ClickHouse/pull/5328)
+ ([never lee](https://github.com/neverlee))
+
+#### Build/Testing/Packaging Improvements {#buildtestingpackaging-improvements}
+
+- Fix some sanitizer reports that show probable use-after-free.[\#5139](https://github.com/ClickHouse/ClickHouse/pull/5139) [\#5143](https://github.com/ClickHouse/ClickHouse/pull/5143) [\#5393](https://github.com/ClickHouse/ClickHouse/pull/5393) ([Ivan](https://github.com/abyss7))
+- Move performance tests out of separate directories for convenience. [\#5158](https://github.com/ClickHouse/ClickHouse/pull/5158) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Fix incorrect performance tests. [\#5255](https://github.com/ClickHouse/ClickHouse/pull/5255) ([alesapin](https://github.com/alesapin))
+- Added a tool to calculate checksums caused by bit flips to debug hardware issues. [\#5334](https://github.com/ClickHouse/ClickHouse/pull/5334) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Make runner script more usable. [\#5340](https://github.com/ClickHouse/ClickHouse/pull/5340)[\#5360](https://github.com/ClickHouse/ClickHouse/pull/5360) ([filimonov](https://github.com/filimonov))
+- Add small instruction how to write performance tests. [\#5408](https://github.com/ClickHouse/ClickHouse/pull/5408) ([alesapin](https://github.com/alesapin))
+- Add ability to make substitutions in create, fill and drop query in performance tests [\#5367](https://github.com/ClickHouse/ClickHouse/pull/5367) ([Olga Khvostikova](https://github.com/stavrolia))
+
+## ClickHouse release 19.7 {#clickhouse-release-19-7}
+
+### ClickHouse release 19.7.5.29, 2019-07-05 {#clickhouse-release-19-7-5-29-2019-07-05}
+
+#### Bug Fix {#bug-fix-25}
+
+- Fix performance regression in some queries with JOIN. [\#5192](https://github.com/ClickHouse/ClickHouse/pull/5192) ([Winter Zhang](https://github.com/zhang2014))
+
+### ClickHouse release 19.7.5.27, 2019-06-09 {#clickhouse-release-19-7-5-27-2019-06-09}
+
+#### New features {#new-features-1}
+
+- Added bitmap related functions `bitmapHasAny` and `bitmapHasAll` analogous to `hasAny` and `hasAll` functions for arrays. [\#5279](https://github.com/ClickHouse/ClickHouse/pull/5279) ([Sergi Vladykin](https://github.com/svladykin))
+
+#### Bug Fixes {#bug-fixes-1}
+
+- Fix segfault on `minmax` INDEX with Null value. [\#5246](https://github.com/ClickHouse/ClickHouse/pull/5246) ([Nikita Vasilev](https://github.com/nikvas0))
+- Mark all input columns in LIMIT BY as required output. It fixes ‘Not found column’ error in some distributed queries. [\#5407](https://github.com/ClickHouse/ClickHouse/pull/5407) ([Constantin S. Pan](https://github.com/kvap))
+- Fix “Column ‘0’ already exists” error in `SELECT .. PREWHERE` on column with DEFAULT [\#5397](https://github.com/ClickHouse/ClickHouse/pull/5397) ([proller](https://github.com/proller))
+- Fix `ALTER MODIFY TTL` query on `ReplicatedMergeTree`. [\#5539](https://github.com/ClickHouse/ClickHouse/pull/5539/commits) ([Anton Popov](https://github.com/CurtizJ))
+- Don’t crash the server when Kafka consumers have failed to start. [\#5285](https://github.com/ClickHouse/ClickHouse/pull/5285) ([Ivan](https://github.com/abyss7))
+- Fixed bitmap functions produce wrong result. [\#5359](https://github.com/ClickHouse/ClickHouse/pull/5359) ([Andy Yang](https://github.com/andyyzh))
+- Fix element\_count for hashed dictionary (do not include duplicates) [\#5440](https://github.com/ClickHouse/ClickHouse/pull/5440) ([Azat Khuzhin](https://github.com/azat))
+- Use contents of environment variable TZ as the name for timezone. It helps to correctly detect default timezone in some cases.[\#5443](https://github.com/ClickHouse/ClickHouse/pull/5443) ([Ivan](https://github.com/abyss7))
+- Do not try to convert integers in `dictGetT` functions, because it doesn’t work correctly. Throw an exception instead. [\#5446](https://github.com/ClickHouse/ClickHouse/pull/5446) ([Artem Zuikov](https://github.com/4ertus2))
+- Fix settings in ExternalData HTTP request. [\#5455](https://github.com/ClickHouse/ClickHouse/pull/5455) ([Danila
+ Kutenin](https://github.com/danlark1))
+- Fix bug when parts were removed only from FS without dropping them from Zookeeper. [\#5520](https://github.com/ClickHouse/ClickHouse/pull/5520) ([alesapin](https://github.com/alesapin))
+- Fix segmentation fault in `bitmapHasAny` function. [\#5528](https://github.com/ClickHouse/ClickHouse/pull/5528) ([Zhichang Yu](https://github.com/yuzhichang))
+- Fixed error when replication connection pool doesn’t retry to resolve host, even when DNS cache was dropped. [\#5534](https://github.com/ClickHouse/ClickHouse/pull/5534) ([alesapin](https://github.com/alesapin))
+- Fixed `DROP INDEX IF EXISTS` query. Now `ALTER TABLE ... DROP INDEX IF EXISTS ...` query doesn’t raise an exception if provided index does not exist. [\#5524](https://github.com/ClickHouse/ClickHouse/pull/5524) ([Gleb Novikov](https://github.com/NanoBjorn))
+- Fix union all supertype column. There were cases with inconsistent data and column types of resulting columns. [\#5503](https://github.com/ClickHouse/ClickHouse/pull/5503) ([Artem Zuikov](https://github.com/4ertus2))
+- Skip ZNONODE during DDL query processing. Before if another node removes the znode in task queue, the one that
+ did not process it, but already get list of children, will terminate the DDLWorker thread. [\#5489](https://github.com/ClickHouse/ClickHouse/pull/5489) ([Azat Khuzhin](https://github.com/azat))
+- Fix INSERT into Distributed() table with MATERIALIZED column. [\#5429](https://github.com/ClickHouse/ClickHouse/pull/5429) ([Azat Khuzhin](https://github.com/azat))
+
+### ClickHouse release 19.7.3.9, 2019-05-30 {#clickhouse-release-19-7-3-9-2019-05-30}
+
+#### New Features {#new-features-2}
+
+- Allow to limit the range of a setting that can be specified by user.
+ These constraints can be set up in user settings profile.
+ [\#4931](https://github.com/ClickHouse/ClickHouse/pull/4931) ([Vitaly
+ Baranov](https://github.com/vitlibar))
+- Add a second version of the function `groupUniqArray` with an optional
+ `max_size` parameter that limits the size of the resulting array. This
+ behavior is similar to `groupArray(max_size)(x)` function.
+ [\#5026](https://github.com/ClickHouse/ClickHouse/pull/5026) ([Guillaume
+ Tassery](https://github.com/YiuRULE))
+- For TSVWithNames/CSVWithNames input file formats, column order can now be
+ determined from file header. This is controlled by
+ `input_format_with_names_use_header` parameter.
+ [\#5081](https://github.com/ClickHouse/ClickHouse/pull/5081)
+ ([Alexander](https://github.com/Akazz))
+
+#### Bug Fixes {#bug-fixes-2}
+
+- Crash with uncompressed\_cache + JOIN during merge (\#5197)
+ [\#5133](https://github.com/ClickHouse/ClickHouse/pull/5133) ([Danila
+ Kutenin](https://github.com/danlark1))
+- Segmentation fault on a clickhouse-client query to system tables. \#5066
+ [\#5127](https://github.com/ClickHouse/ClickHouse/pull/5127)
+ ([Ivan](https://github.com/abyss7))
+- Data loss on heavy load via KafkaEngine (\#4736)
+ [\#5080](https://github.com/ClickHouse/ClickHouse/pull/5080)
+ ([Ivan](https://github.com/abyss7))
+- Fixed very rare data race condition that could happen when executing a query with UNION ALL involving at least two SELECTs from system.columns, system.tables, system.parts, system.parts\_tables or tables of Merge family and performing ALTER of columns of the related tables concurrently. [\#5189](https://github.com/ClickHouse/ClickHouse/pull/5189) ([alexey-milovidov](https://github.com/alexey-milovidov))
+
+#### Performance Improvements {#performance-improvements-1}
+
+- Use radix sort for sorting by single numeric column in `ORDER BY` without
+ `LIMIT`. [\#5106](https://github.com/ClickHouse/ClickHouse/pull/5106),
+ [\#4439](https://github.com/ClickHouse/ClickHouse/pull/4439)
+ ([Evgenii Pravda](https://github.com/kvinty),
+ [alexey-milovidov](https://github.com/alexey-milovidov))
+
+#### Documentation {#documentation-1}
+
+- Translate documentation for some table engines to Chinese.
+ [\#5107](https://github.com/ClickHouse/ClickHouse/pull/5107),
+ [\#5094](https://github.com/ClickHouse/ClickHouse/pull/5094),
+ [\#5087](https://github.com/ClickHouse/ClickHouse/pull/5087)
+ ([张风啸](https://github.com/AlexZFX)),
+ [\#5068](https://github.com/ClickHouse/ClickHouse/pull/5068) ([never
+ lee](https://github.com/neverlee))
+
+#### Build/Testing/Packaging Improvements {#buildtestingpackaging-improvements-1}
+
+- Print UTF-8 characters properly in `clickhouse-test`.
+ [\#5084](https://github.com/ClickHouse/ClickHouse/pull/5084)
+ ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Add command line parameter for clickhouse-client to always load suggestion
+ data. [\#5102](https://github.com/ClickHouse/ClickHouse/pull/5102)
+ ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Resolve some of PVS-Studio warnings.
+ [\#5082](https://github.com/ClickHouse/ClickHouse/pull/5082)
+ ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Update LZ4 [\#5040](https://github.com/ClickHouse/ClickHouse/pull/5040) ([Danila
+ Kutenin](https://github.com/danlark1))
+- Add gperf to build requirements for upcoming pull request \#5030.
+ [\#5110](https://github.com/ClickHouse/ClickHouse/pull/5110)
+ ([proller](https://github.com/proller))
+
+## ClickHouse release 19.6 {#clickhouse-release-19-6}
+
+### ClickHouse release 19.6.3.18, 2019-06-13 {#clickhouse-release-19-6-3-18-2019-06-13}
+
+#### Bug Fixes {#bug-fixes-3}
+
+- Fixed IN condition pushdown for queries from table functions `mysql` and `odbc` and corresponding table engines. This fixes \#3540 and \#2384. [\#5313](https://github.com/ClickHouse/ClickHouse/pull/5313) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Fix deadlock in Zookeeper. [\#5297](https://github.com/ClickHouse/ClickHouse/pull/5297) ([github1youlc](https://github.com/github1youlc))
+- Allow quoted decimals in CSV. [\#5284](https://github.com/ClickHouse/ClickHouse/pull/5284) ([Artem Zuikov](https://github.com/4ertus2)
+- Disallow conversion from float Inf/NaN into Decimals (throw exception). [\#5282](https://github.com/ClickHouse/ClickHouse/pull/5282) ([Artem Zuikov](https://github.com/4ertus2))
+- Fix data race in rename query. [\#5247](https://github.com/ClickHouse/ClickHouse/pull/5247) ([Winter Zhang](https://github.com/zhang2014))
+- Temporarily disable LFAlloc. Usage of LFAlloc might lead to a lot of MAP\_FAILED in allocating UncompressedCache and in a result to crashes of queries at high loaded servers. [cfdba93](https://github.com/ClickHouse/ClickHouse/commit/cfdba938ce22f16efeec504f7f90206a515b1280)([Danila Kutenin](https://github.com/danlark1))
+
+### ClickHouse release 19.6.2.11, 2019-05-13 {#clickhouse-release-19-6-2-11-2019-05-13}
+
+#### New Features {#new-features-3}
+
+- TTL expressions for columns and tables. [\#4212](https://github.com/ClickHouse/ClickHouse/pull/4212) ([Anton Popov](https://github.com/CurtizJ))
+- Added support for `brotli` compression for HTTP responses (Accept-Encoding: br) [\#4388](https://github.com/ClickHouse/ClickHouse/pull/4388) ([Mikhail](https://github.com/fandyushin))
+- Added new function `isValidUTF8` for checking whether a set of bytes is correctly utf-8 encoded. [\#4934](https://github.com/ClickHouse/ClickHouse/pull/4934) ([Danila Kutenin](https://github.com/danlark1))
+- Add new load balancing policy `first_or_random` which sends queries to the first specified host and if it’s inaccessible send queries to random hosts of shard. Useful for cross-replication topology setups. [\#5012](https://github.com/ClickHouse/ClickHouse/pull/5012) ([nvartolomei](https://github.com/nvartolomei))
+
+#### Experimental Features {#experimental-features-1}
+
+- Add setting `index_granularity_bytes` (adaptive index granularity) for MergeTree\* tables family. [\#4826](https://github.com/ClickHouse/ClickHouse/pull/4826) ([alesapin](https://github.com/alesapin))
+
+#### Improvements {#improvements-1}
+
+- Added support for non-constant and negative size and length arguments for function `substringUTF8`. [\#4989](https://github.com/ClickHouse/ClickHouse/pull/4989) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Disable push-down to right table in left join, left table in right join, and both tables in full join. This fixes wrong JOIN results in some cases. [\#4846](https://github.com/ClickHouse/ClickHouse/pull/4846) ([Ivan](https://github.com/abyss7))
+- `clickhouse-copier`: auto upload task configuration from `--task-file` option [\#4876](https://github.com/ClickHouse/ClickHouse/pull/4876) ([proller](https://github.com/proller))
+- Added typos handler for storage factory and table functions factory. [\#4891](https://github.com/ClickHouse/ClickHouse/pull/4891) ([Danila Kutenin](https://github.com/danlark1))
+- Support asterisks and qualified asterisks for multiple joins without subqueries [\#4898](https://github.com/ClickHouse/ClickHouse/pull/4898) ([Artem Zuikov](https://github.com/4ertus2))
+- Make missing column error message more user friendly. [\#4915](https://github.com/ClickHouse/ClickHouse/pull/4915) ([Artem Zuikov](https://github.com/4ertus2))
+
+#### Performance Improvements {#performance-improvements-2}
+
+- Significant speedup of ASOF JOIN [\#4924](https://github.com/ClickHouse/ClickHouse/pull/4924) ([Martijn Bakker](https://github.com/Gladdy))
+
+#### Backward Incompatible Changes {#backward-incompatible-changes}
+
+- HTTP header `Query-Id` was renamed to `X-ClickHouse-Query-Id` for consistency. [\#4972](https://github.com/ClickHouse/ClickHouse/pull/4972) ([Mikhail](https://github.com/fandyushin))
+
+#### Bug Fixes {#bug-fixes-4}
+
+- Fixed potential null pointer dereference in `clickhouse-copier`. [\#4900](https://github.com/ClickHouse/ClickHouse/pull/4900) ([proller](https://github.com/proller))
+- Fixed error on query with JOIN + ARRAY JOIN [\#4938](https://github.com/ClickHouse/ClickHouse/pull/4938) ([Artem Zuikov](https://github.com/4ertus2))
+- Fixed hanging on start of the server when a dictionary depends on another dictionary via a database with engine=Dictionary. [\#4962](https://github.com/ClickHouse/ClickHouse/pull/4962) ([Vitaly Baranov](https://github.com/vitlibar))
+- Partially fix distributed\_product\_mode = local. It’s possible to allow columns of local tables in where/having/order by/… via table aliases. Throw exception if table does not have alias. There’s not possible to access to the columns without table aliases yet. [\#4986](https://github.com/ClickHouse/ClickHouse/pull/4986) ([Artem Zuikov](https://github.com/4ertus2))
+- Fix potentially wrong result for `SELECT DISTINCT` with `JOIN` [\#5001](https://github.com/ClickHouse/ClickHouse/pull/5001) ([Artem Zuikov](https://github.com/4ertus2))
+- Fixed very rare data race condition that could happen when executing a query with UNION ALL involving at least two SELECTs from system.columns, system.tables, system.parts, system.parts\_tables or tables of Merge family and performing ALTER of columns of the related tables concurrently. [\#5189](https://github.com/ClickHouse/ClickHouse/pull/5189) ([alexey-milovidov](https://github.com/alexey-milovidov))
+
+#### Build/Testing/Packaging Improvements {#buildtestingpackaging-improvements-2}
+
+- Fixed test failures when running clickhouse-server on different host [\#4713](https://github.com/ClickHouse/ClickHouse/pull/4713) ([Vasily Nemkov](https://github.com/Enmk))
+- clickhouse-test: Disable color control sequences in non tty environment. [\#4937](https://github.com/ClickHouse/ClickHouse/pull/4937) ([alesapin](https://github.com/alesapin))
+- clickhouse-test: Allow use any test database (remove `test.` qualification where it possible) [\#5008](https://github.com/ClickHouse/ClickHouse/pull/5008) ([proller](https://github.com/proller))
+- Fix ubsan errors [\#5037](https://github.com/ClickHouse/ClickHouse/pull/5037) ([Vitaly Baranov](https://github.com/vitlibar))
+- Yandex LFAlloc was added to ClickHouse to allocate MarkCache and UncompressedCache data in different ways to catch segfaults more reliable [\#4995](https://github.com/ClickHouse/ClickHouse/pull/4995) ([Danila Kutenin](https://github.com/danlark1))
+- Python util to help with backports and changelogs. [\#4949](https://github.com/ClickHouse/ClickHouse/pull/4949) ([Ivan](https://github.com/abyss7))
+
+## ClickHouse release 19.5 {#clickhouse-release-19-5}
+
+### ClickHouse release 19.5.4.22, 2019-05-13 {#clickhouse-release-19-5-4-22-2019-05-13}
+
+#### Bug fixes {#bug-fixes-5}
+
+- Fixed possible crash in bitmap\* functions [\#5220](https://github.com/ClickHouse/ClickHouse/pull/5220) [\#5228](https://github.com/ClickHouse/ClickHouse/pull/5228) ([Andy Yang](https://github.com/andyyzh))
+- Fixed very rare data race condition that could happen when executing a query with UNION ALL involving at least two SELECTs from system.columns, system.tables, system.parts, system.parts\_tables or tables of Merge family and performing ALTER of columns of the related tables concurrently. [\#5189](https://github.com/ClickHouse/ClickHouse/pull/5189) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Fixed error `Set for IN is not created yet in case of using single LowCardinality column in the left part of IN`. This error happened if LowCardinality column was the part of primary key. \#5031 [\#5154](https://github.com/ClickHouse/ClickHouse/pull/5154) ([Nikolai Kochetov](https://github.com/KochetovNicolai))
+- Modification of retention function: If a row satisfies both the first and NTH condition, only the first satisfied condition is added to the data state. Now all conditions that satisfy in a row of data are added to the data state. [\#5119](https://github.com/ClickHouse/ClickHouse/pull/5119) ([小路](https://github.com/nicelulu))
+
+### ClickHouse release 19.5.3.8, 2019-04-18 {#clickhouse-release-19-5-3-8-2019-04-18}
+
+#### Bug fixes {#bug-fixes-6}
+
+- Fixed type of setting `max_partitions_per_insert_block` from boolean to UInt64. [\#5028](https://github.com/ClickHouse/ClickHouse/pull/5028) ([Mohammad Hossein Sekhavat](https://github.com/mhsekhavat))
+
+### ClickHouse release 19.5.2.6, 2019-04-15 {#clickhouse-release-19-5-2-6-2019-04-15}
+
+#### New Features {#new-features-4}
+
+- [Hyperscan](https://github.com/intel/hyperscan) multiple regular expression matching was added (functions `multiMatchAny`, `multiMatchAnyIndex`, `multiFuzzyMatchAny`, `multiFuzzyMatchAnyIndex`). [\#4780](https://github.com/ClickHouse/ClickHouse/pull/4780), [\#4841](https://github.com/ClickHouse/ClickHouse/pull/4841) ([Danila Kutenin](https://github.com/danlark1))
+- `multiSearchFirstPosition` function was added. [\#4780](https://github.com/ClickHouse/ClickHouse/pull/4780) ([Danila Kutenin](https://github.com/danlark1))
+- Implement the predefined expression filter per row for tables. [\#4792](https://github.com/ClickHouse/ClickHouse/pull/4792) ([Ivan](https://github.com/abyss7))
+- A new type of data skipping indices based on bloom filters (can be used for `equal`, `in` and `like` functions). [\#4499](https://github.com/ClickHouse/ClickHouse/pull/4499) ([Nikita Vasilev](https://github.com/nikvas0))
+- Added `ASOF JOIN` which allows to run queries that join to the most recent value known. [\#4774](https://github.com/ClickHouse/ClickHouse/pull/4774) [\#4867](https://github.com/ClickHouse/ClickHouse/pull/4867) [\#4863](https://github.com/ClickHouse/ClickHouse/pull/4863) [\#4875](https://github.com/ClickHouse/ClickHouse/pull/4875) ([Martijn Bakker](https://github.com/Gladdy), [Artem Zuikov](https://github.com/4ertus2))
+- Rewrite multiple `COMMA JOIN` to `CROSS JOIN`. Then rewrite them to `INNER JOIN` if possible. [\#4661](https://github.com/ClickHouse/ClickHouse/pull/4661) ([Artem Zuikov](https://github.com/4ertus2))
+
+#### Improvement {#improvement-9}
+
+- `topK` and `topKWeighted` now supports custom `loadFactor` (fixes issue [\#4252](https://github.com/ClickHouse/ClickHouse/issues/4252)). [\#4634](https://github.com/ClickHouse/ClickHouse/pull/4634) ([Kirill Danshin](https://github.com/kirillDanshin))
+- Allow to use `parallel_replicas_count > 1` even for tables without sampling (the setting is simply ignored for them). In previous versions it was lead to exception. [\#4637](https://github.com/ClickHouse/ClickHouse/pull/4637) ([Alexey Elymanov](https://github.com/digitalist))
+- Support for `CREATE OR REPLACE VIEW`. Allow to create a view or set a new definition in a single statement. [\#4654](https://github.com/ClickHouse/ClickHouse/pull/4654) ([Boris Granveaud](https://github.com/bgranvea))
+- `Buffer` table engine now supports `PREWHERE`. [\#4671](https://github.com/ClickHouse/ClickHouse/pull/4671) ([Yangkuan Liu](https://github.com/LiuYangkuan))
+- Add ability to start replicated table without metadata in zookeeper in `readonly` mode. [\#4691](https://github.com/ClickHouse/ClickHouse/pull/4691) ([alesapin](https://github.com/alesapin))
+- Fixed flicker of progress bar in clickhouse-client. The issue was most noticeable when using `FORMAT Null` with streaming queries. [\#4811](https://github.com/ClickHouse/ClickHouse/pull/4811) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Allow to disable functions with `hyperscan` library on per user basis to limit potentially excessive and uncontrolled resource usage. [\#4816](https://github.com/ClickHouse/ClickHouse/pull/4816) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Add version number logging in all errors. [\#4824](https://github.com/ClickHouse/ClickHouse/pull/4824) ([proller](https://github.com/proller))
+- Added restriction to the `multiMatch` functions which requires string size to fit into `unsigned int`. Also added the number of arguments limit to the `multiSearch` functions. [\#4834](https://github.com/ClickHouse/ClickHouse/pull/4834) ([Danila Kutenin](https://github.com/danlark1))
+- Improved usage of scratch space and error handling in Hyperscan. [\#4866](https://github.com/ClickHouse/ClickHouse/pull/4866) ([Danila Kutenin](https://github.com/danlark1))
+- Fill `system.graphite_detentions` from a table config of `*GraphiteMergeTree` engine tables. [\#4584](https://github.com/ClickHouse/ClickHouse/pull/4584) ([Mikhail f. Shiryaev](https://github.com/Felixoid))
+- Rename `trigramDistance` function to `ngramDistance` and add more functions with `CaseInsensitive` and `UTF`. [\#4602](https://github.com/ClickHouse/ClickHouse/pull/4602) ([Danila Kutenin](https://github.com/danlark1))
+- Improved data skipping indices calculation. [\#4640](https://github.com/ClickHouse/ClickHouse/pull/4640) ([Nikita Vasilev](https://github.com/nikvas0))
+- Keep ordinary, `DEFAULT`, `MATERIALIZED` and `ALIAS` columns in a single list (fixes issue [\#2867](https://github.com/ClickHouse/ClickHouse/issues/2867)). [\#4707](https://github.com/ClickHouse/ClickHouse/pull/4707) ([Alex Zatelepin](https://github.com/ztlpn))
+
+#### Bug Fix {#bug-fix-26}
+
+- Avoid `std::terminate` in case of memory allocation failure. Now `std::bad_alloc` exception is thrown as expected. [\#4665](https://github.com/ClickHouse/ClickHouse/pull/4665) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Fixes capnproto reading from buffer. Sometimes files wasn’t loaded successfully by HTTP. [\#4674](https://github.com/ClickHouse/ClickHouse/pull/4674) ([Vladislav](https://github.com/smirnov-vs))
+- Fix error `Unknown log entry type: 0` after `OPTIMIZE TABLE FINAL` query. [\#4683](https://github.com/ClickHouse/ClickHouse/pull/4683) ([Amos Bird](https://github.com/amosbird))
+- Wrong arguments to `hasAny` or `hasAll` functions may lead to segfault. [\#4698](https://github.com/ClickHouse/ClickHouse/pull/4698) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Deadlock may happen while executing `DROP DATABASE dictionary` query. [\#4701](https://github.com/ClickHouse/ClickHouse/pull/4701) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Fix undefined behavior in `median` and `quantile` functions. [\#4702](https://github.com/ClickHouse/ClickHouse/pull/4702) ([hcz](https://github.com/hczhcz))
+- Fix compression level detection when `network_compression_method` in lowercase. Broken in v19.1. [\#4706](https://github.com/ClickHouse/ClickHouse/pull/4706) ([proller](https://github.com/proller))
+- Fixed ignorance of `UTC` setting (fixes issue [\#4658](https://github.com/ClickHouse/ClickHouse/issues/4658)). [\#4718](https://github.com/ClickHouse/ClickHouse/pull/4718) ([proller](https://github.com/proller))
+- Fix `histogram` function behaviour with `Distributed` tables. [\#4741](https://github.com/ClickHouse/ClickHouse/pull/4741) ([olegkv](https://github.com/olegkv))
+- Fixed tsan report `destroy of a locked mutex`. [\#4742](https://github.com/ClickHouse/ClickHouse/pull/4742) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Fixed TSan report on shutdown due to race condition in system logs usage. Fixed potential use-after-free on shutdown when part\_log is enabled. [\#4758](https://github.com/ClickHouse/ClickHouse/pull/4758) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Fix recheck parts in `ReplicatedMergeTreeAlterThread` in case of error. [\#4772](https://github.com/ClickHouse/ClickHouse/pull/4772) ([Nikolai Kochetov](https://github.com/KochetovNicolai))
+- Arithmetic operations on intermediate aggregate function states were not working for constant arguments (such as subquery results). [\#4776](https://github.com/ClickHouse/ClickHouse/pull/4776) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Always backquote column names in metadata. Otherwise it’s impossible to create a table with column named `index` (server won’t restart due to malformed `ATTACH` query in metadata). [\#4782](https://github.com/ClickHouse/ClickHouse/pull/4782) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Fix crash in `ALTER ... MODIFY ORDER BY` on `Distributed` table. [\#4790](https://github.com/ClickHouse/ClickHouse/pull/4790) ([TCeason](https://github.com/TCeason))
+- Fix segfault in `JOIN ON` with enabled `enable_optimize_predicate_expression`. [\#4794](https://github.com/ClickHouse/ClickHouse/pull/4794) ([Winter Zhang](https://github.com/zhang2014))
+- Fix bug with adding an extraneous row after consuming a protobuf message from Kafka. [\#4808](https://github.com/ClickHouse/ClickHouse/pull/4808) ([Vitaly Baranov](https://github.com/vitlibar))
+- Fix crash of `JOIN` on not-nullable vs nullable column. Fix `NULLs` in right keys in `ANY JOIN` + `join_use_nulls`. [\#4815](https://github.com/ClickHouse/ClickHouse/pull/4815) ([Artem Zuikov](https://github.com/4ertus2))
+- Fix segmentation fault in `clickhouse-copier`. [\#4835](https://github.com/ClickHouse/ClickHouse/pull/4835) ([proller](https://github.com/proller))
+- Fixed race condition in `SELECT` from `system.tables` if the table is renamed or altered concurrently. [\#4836](https://github.com/ClickHouse/ClickHouse/pull/4836) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Fixed data race when fetching data part that is already obsolete. [\#4839](https://github.com/ClickHouse/ClickHouse/pull/4839) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Fixed rare data race that can happen during `RENAME` table of MergeTree family. [\#4844](https://github.com/ClickHouse/ClickHouse/pull/4844) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Fixed segmentation fault in function `arrayIntersect`. Segmentation fault could happen if function was called with mixed constant and ordinary arguments. [\#4847](https://github.com/ClickHouse/ClickHouse/pull/4847) ([Lixiang Qian](https://github.com/fancyqlx))
+- Fixed reading from `Array(LowCardinality)` column in rare case when column contained a long sequence of empty arrays. [\#4850](https://github.com/ClickHouse/ClickHouse/pull/4850) ([Nikolai Kochetov](https://github.com/KochetovNicolai))
+- Fix crash in `FULL/RIGHT JOIN` when we joining on nullable vs not nullable. [\#4855](https://github.com/ClickHouse/ClickHouse/pull/4855) ([Artem Zuikov](https://github.com/4ertus2))
+- Fix `No message received` exception while fetching parts between replicas. [\#4856](https://github.com/ClickHouse/ClickHouse/pull/4856) ([alesapin](https://github.com/alesapin))
+- Fixed `arrayIntersect` function wrong result in case of several repeated values in single array. [\#4871](https://github.com/ClickHouse/ClickHouse/pull/4871) ([Nikolai Kochetov](https://github.com/KochetovNicolai))
+- Fix a race condition during concurrent `ALTER COLUMN` queries that could lead to a server crash (fixes issue [\#3421](https://github.com/ClickHouse/ClickHouse/issues/3421)). [\#4592](https://github.com/ClickHouse/ClickHouse/pull/4592) ([Alex Zatelepin](https://github.com/ztlpn))
+- Fix incorrect result in `FULL/RIGHT JOIN` with const column. [\#4723](https://github.com/ClickHouse/ClickHouse/pull/4723) ([Artem Zuikov](https://github.com/4ertus2))
+- Fix duplicates in `GLOBAL JOIN` with asterisk. [\#4705](https://github.com/ClickHouse/ClickHouse/pull/4705) ([Artem Zuikov](https://github.com/4ertus2))
+- Fix parameter deduction in `ALTER MODIFY` of column `CODEC` when column type is not specified. [\#4883](https://github.com/ClickHouse/ClickHouse/pull/4883) ([alesapin](https://github.com/alesapin))
+- Functions `cutQueryStringAndFragment()` and `queryStringAndFragment()` now works correctly when `URL` contains a fragment and no query. [\#4894](https://github.com/ClickHouse/ClickHouse/pull/4894) ([Vitaly Baranov](https://github.com/vitlibar))
+- Fix rare bug when setting `min_bytes_to_use_direct_io` is greater than zero, which occures when thread have to seek backward in column file. [\#4897](https://github.com/ClickHouse/ClickHouse/pull/4897) ([alesapin](https://github.com/alesapin))
+- Fix wrong argument types for aggregate functions with `LowCardinality` arguments (fixes issue [\#4919](https://github.com/ClickHouse/ClickHouse/issues/4919)). [\#4922](https://github.com/ClickHouse/ClickHouse/pull/4922) ([Nikolai Kochetov](https://github.com/KochetovNicolai))
+- Fix wrong name qualification in `GLOBAL JOIN`. [\#4969](https://github.com/ClickHouse/ClickHouse/pull/4969) ([Artem Zuikov](https://github.com/4ertus2))
+- Fix function `toISOWeek` result for year 1970. [\#4988](https://github.com/ClickHouse/ClickHouse/pull/4988) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Fix `DROP`, `TRUNCATE` and `OPTIMIZE` queries duplication, when executed on `ON CLUSTER` for `ReplicatedMergeTree*` tables family. [\#4991](https://github.com/ClickHouse/ClickHouse/pull/4991) ([alesapin](https://github.com/alesapin))
+
+#### Backward Incompatible Change {#backward-incompatible-change-8}
+
+- Rename setting `insert_sample_with_metadata` to setting `input_format_defaults_for_omitted_fields`. [\#4771](https://github.com/ClickHouse/ClickHouse/pull/4771) ([Artem Zuikov](https://github.com/4ertus2))
+- Added setting `max_partitions_per_insert_block` (with value 100 by default). If inserted block contains larger number of partitions, an exception is thrown. Set it to 0 if you want to remove the limit (not recommended). [\#4845](https://github.com/ClickHouse/ClickHouse/pull/4845) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Multi-search functions were renamed (`multiPosition` to `multiSearchAllPositions`, `multiSearch` to `multiSearchAny`, `firstMatch` to `multiSearchFirstIndex`). [\#4780](https://github.com/ClickHouse/ClickHouse/pull/4780) ([Danila Kutenin](https://github.com/danlark1))
+
+#### Performance Improvement {#performance-improvement-6}
+
+- Optimize Volnitsky searcher by inlining, giving about 5-10% search improvement for queries with many needles or many similar bigrams. [\#4862](https://github.com/ClickHouse/ClickHouse/pull/4862) ([Danila Kutenin](https://github.com/danlark1))
+- Fix performance issue when setting `use_uncompressed_cache` is greater than zero, which appeared when all read data contained in cache. [\#4913](https://github.com/ClickHouse/ClickHouse/pull/4913) ([alesapin](https://github.com/alesapin))
+
+#### Build/Testing/Packaging Improvement {#buildtestingpackaging-improvement-10}
+
+- Hardening debug build: more granular memory mappings and ASLR; add memory protection for mark cache and index. This allows to find more memory stomping bugs in case when ASan and MSan cannot do it. [\#4632](https://github.com/ClickHouse/ClickHouse/pull/4632) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Add support for cmake variables `ENABLE_PROTOBUF`, `ENABLE_PARQUET` and `ENABLE_BROTLI` which allows to enable/disable the above features (same as we can do for librdkafka, mysql, etc). [\#4669](https://github.com/ClickHouse/ClickHouse/pull/4669) ([Silviu Caragea](https://github.com/silviucpp))
+- Add ability to print process list and stacktraces of all threads if some queries are hung after test run. [\#4675](https://github.com/ClickHouse/ClickHouse/pull/4675) ([alesapin](https://github.com/alesapin))
+- Add retries on `Connection loss` error in `clickhouse-test`. [\#4682](https://github.com/ClickHouse/ClickHouse/pull/4682) ([alesapin](https://github.com/alesapin))
+- Add freebsd build with vagrant and build with thread sanitizer to packager script. [\#4712](https://github.com/ClickHouse/ClickHouse/pull/4712) [\#4748](https://github.com/ClickHouse/ClickHouse/pull/4748) ([alesapin](https://github.com/alesapin))
+- Now user asked for password for user `'default'` during installation. [\#4725](https://github.com/ClickHouse/ClickHouse/pull/4725) ([proller](https://github.com/proller))
+- Suppress warning in `rdkafka` library. [\#4740](https://github.com/ClickHouse/ClickHouse/pull/4740) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Allow ability to build without ssl. [\#4750](https://github.com/ClickHouse/ClickHouse/pull/4750) ([proller](https://github.com/proller))
+- Add a way to launch clickhouse-server image from a custom user. [\#4753](https://github.com/ClickHouse/ClickHouse/pull/4753) ([Mikhail f. Shiryaev](https://github.com/Felixoid))
+- Upgrade contrib boost to 1.69. [\#4793](https://github.com/ClickHouse/ClickHouse/pull/4793) ([proller](https://github.com/proller))
+- Disable usage of `mremap` when compiled with Thread Sanitizer. Surprisingly enough, TSan does not intercept `mremap` (though it does intercept `mmap`, `munmap`) that leads to false positives. Fixed TSan report in stateful tests. [\#4859](https://github.com/ClickHouse/ClickHouse/pull/4859) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Add test checking using format schema via HTTP interface. [\#4864](https://github.com/ClickHouse/ClickHouse/pull/4864) ([Vitaly Baranov](https://github.com/vitlibar))
+
+## ClickHouse release 19.4 {#clickhouse-release-19-4}
+
+### ClickHouse release 19.4.4.33, 2019-04-17 {#clickhouse-release-19-4-4-33-2019-04-17}
+
+#### Bug Fixes {#bug-fixes-7}
+
+- Avoid `std::terminate` in case of memory allocation failure. Now `std::bad_alloc` exception is thrown as expected. [\#4665](https://github.com/ClickHouse/ClickHouse/pull/4665) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Fixes capnproto reading from buffer. Sometimes files wasn’t loaded successfully by HTTP. [\#4674](https://github.com/ClickHouse/ClickHouse/pull/4674) ([Vladislav](https://github.com/smirnov-vs))
+- Fix error `Unknown log entry type: 0` after `OPTIMIZE TABLE FINAL` query. [\#4683](https://github.com/ClickHouse/ClickHouse/pull/4683) ([Amos Bird](https://github.com/amosbird))
+- Wrong arguments to `hasAny` or `hasAll` functions may lead to segfault. [\#4698](https://github.com/ClickHouse/ClickHouse/pull/4698) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Deadlock may happen while executing `DROP DATABASE dictionary` query. [\#4701](https://github.com/ClickHouse/ClickHouse/pull/4701) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Fix undefined behavior in `median` and `quantile` functions. [\#4702](https://github.com/ClickHouse/ClickHouse/pull/4702) ([hcz](https://github.com/hczhcz))
+- Fix compression level detection when `network_compression_method` in lowercase. Broken in v19.1. [\#4706](https://github.com/ClickHouse/ClickHouse/pull/4706) ([proller](https://github.com/proller))
+- Fixed ignorance of `UTC` setting (fixes issue [\#4658](https://github.com/ClickHouse/ClickHouse/issues/4658)). [\#4718](https://github.com/ClickHouse/ClickHouse/pull/4718) ([proller](https://github.com/proller))
+- Fix `histogram` function behaviour with `Distributed` tables. [\#4741](https://github.com/ClickHouse/ClickHouse/pull/4741) ([olegkv](https://github.com/olegkv))
+- Fixed tsan report `destroy of a locked mutex`. [\#4742](https://github.com/ClickHouse/ClickHouse/pull/4742) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Fixed TSan report on shutdown due to race condition in system logs usage. Fixed potential use-after-free on shutdown when part\_log is enabled. [\#4758](https://github.com/ClickHouse/ClickHouse/pull/4758) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Fix recheck parts in `ReplicatedMergeTreeAlterThread` in case of error. [\#4772](https://github.com/ClickHouse/ClickHouse/pull/4772) ([Nikolai Kochetov](https://github.com/KochetovNicolai))
+- Arithmetic operations on intermediate aggregate function states were not working for constant arguments (such as subquery results). [\#4776](https://github.com/ClickHouse/ClickHouse/pull/4776) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Always backquote column names in metadata. Otherwise it’s impossible to create a table with column named `index` (server won’t restart due to malformed `ATTACH` query in metadata). [\#4782](https://github.com/ClickHouse/ClickHouse/pull/4782) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Fix crash in `ALTER ... MODIFY ORDER BY` on `Distributed` table. [\#4790](https://github.com/ClickHouse/ClickHouse/pull/4790) ([TCeason](https://github.com/TCeason))
+- Fix segfault in `JOIN ON` with enabled `enable_optimize_predicate_expression`. [\#4794](https://github.com/ClickHouse/ClickHouse/pull/4794) ([Winter Zhang](https://github.com/zhang2014))
+- Fix bug with adding an extraneous row after consuming a protobuf message from Kafka. [\#4808](https://github.com/ClickHouse/ClickHouse/pull/4808) ([Vitaly Baranov](https://github.com/vitlibar))
+- Fix segmentation fault in `clickhouse-copier`. [\#4835](https://github.com/ClickHouse/ClickHouse/pull/4835) ([proller](https://github.com/proller))
+- Fixed race condition in `SELECT` from `system.tables` if the table is renamed or altered concurrently. [\#4836](https://github.com/ClickHouse/ClickHouse/pull/4836) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Fixed data race when fetching data part that is already obsolete. [\#4839](https://github.com/ClickHouse/ClickHouse/pull/4839) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Fixed rare data race that can happen during `RENAME` table of MergeTree family. [\#4844](https://github.com/ClickHouse/ClickHouse/pull/4844) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Fixed segmentation fault in function `arrayIntersect`. Segmentation fault could happen if function was called with mixed constant and ordinary arguments. [\#4847](https://github.com/ClickHouse/ClickHouse/pull/4847) ([Lixiang Qian](https://github.com/fancyqlx))
+- Fixed reading from `Array(LowCardinality)` column in rare case when column contained a long sequence of empty arrays. [\#4850](https://github.com/ClickHouse/ClickHouse/pull/4850) ([Nikolai Kochetov](https://github.com/KochetovNicolai))
+- Fix `No message received` exception while fetching parts between replicas. [\#4856](https://github.com/ClickHouse/ClickHouse/pull/4856) ([alesapin](https://github.com/alesapin))
+- Fixed `arrayIntersect` function wrong result in case of several repeated values in single array. [\#4871](https://github.com/ClickHouse/ClickHouse/pull/4871) ([Nikolai Kochetov](https://github.com/KochetovNicolai))
+- Fix a race condition during concurrent `ALTER COLUMN` queries that could lead to a server crash (fixes issue [\#3421](https://github.com/ClickHouse/ClickHouse/issues/3421)). [\#4592](https://github.com/ClickHouse/ClickHouse/pull/4592) ([Alex Zatelepin](https://github.com/ztlpn))
+- Fix parameter deduction in `ALTER MODIFY` of column `CODEC` when column type is not specified. [\#4883](https://github.com/ClickHouse/ClickHouse/pull/4883) ([alesapin](https://github.com/alesapin))
+- Functions `cutQueryStringAndFragment()` and `queryStringAndFragment()` now works correctly when `URL` contains a fragment and no query. [\#4894](https://github.com/ClickHouse/ClickHouse/pull/4894) ([Vitaly Baranov](https://github.com/vitlibar))
+- Fix rare bug when setting `min_bytes_to_use_direct_io` is greater than zero, which occures when thread have to seek backward in column file. [\#4897](https://github.com/ClickHouse/ClickHouse/pull/4897) ([alesapin](https://github.com/alesapin))
+- Fix wrong argument types for aggregate functions with `LowCardinality` arguments (fixes issue [\#4919](https://github.com/ClickHouse/ClickHouse/issues/4919)). [\#4922](https://github.com/ClickHouse/ClickHouse/pull/4922) ([Nikolai Kochetov](https://github.com/KochetovNicolai))
+- Fix function `toISOWeek` result for year 1970. [\#4988](https://github.com/ClickHouse/ClickHouse/pull/4988) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Fix `DROP`, `TRUNCATE` and `OPTIMIZE` queries duplication, when executed on `ON CLUSTER` for `ReplicatedMergeTree*` tables family. [\#4991](https://github.com/ClickHouse/ClickHouse/pull/4991) ([alesapin](https://github.com/alesapin))
+
+#### Improvements {#improvements-2}
+
+- Keep ordinary, `DEFAULT`, `MATERIALIZED` and `ALIAS` columns in a single list (fixes issue [\#2867](https://github.com/ClickHouse/ClickHouse/issues/2867)). [\#4707](https://github.com/ClickHouse/ClickHouse/pull/4707) ([Alex Zatelepin](https://github.com/ztlpn))
+
+### ClickHouse release 19.4.3.11, 2019-04-02 {#clickhouse-release-19-4-3-11-2019-04-02}
+
+#### Bug Fixes {#bug-fixes-8}
+
+- Fix crash in `FULL/RIGHT JOIN` when we joining on nullable vs not nullable. [\#4855](https://github.com/ClickHouse/ClickHouse/pull/4855) ([Artem Zuikov](https://github.com/4ertus2))
+- Fix segmentation fault in `clickhouse-copier`. [\#4835](https://github.com/ClickHouse/ClickHouse/pull/4835) ([proller](https://github.com/proller))
+
+#### Build/Testing/Packaging Improvement {#buildtestingpackaging-improvement-11}
+
+- Add a way to launch clickhouse-server image from a custom user. [\#4753](https://github.com/ClickHouse/ClickHouse/pull/4753) ([Mikhail f. Shiryaev](https://github.com/Felixoid))
+
+### ClickHouse release 19.4.2.7, 2019-03-30 {#clickhouse-release-19-4-2-7-2019-03-30}
+
+#### Bug Fixes {#bug-fixes-9}
+
+- Fixed reading from `Array(LowCardinality)` column in rare case when column contained a long sequence of empty arrays. [\#4850](https://github.com/ClickHouse/ClickHouse/pull/4850) ([Nikolai Kochetov](https://github.com/KochetovNicolai))
+
+### ClickHouse release 19.4.1.3, 2019-03-19 {#clickhouse-release-19-4-1-3-2019-03-19}
+
+#### Bug Fixes {#bug-fixes-10}
+
+- Fixed remote queries which contain both `LIMIT BY` and `LIMIT`. Previously, if `LIMIT BY` and `LIMIT` were used for remote query, `LIMIT` could happen before `LIMIT BY`, which led to too filtered result. [\#4708](https://github.com/ClickHouse/ClickHouse/pull/4708) ([Constantin S. Pan](https://github.com/kvap))
+
+### ClickHouse release 19.4.0.49, 2019-03-09 {#clickhouse-release-19-4-0-49-2019-03-09}
+
+#### New Features {#new-features-5}
+
+- Added full support for `Protobuf` format (input and output, nested data structures). [\#4174](https://github.com/ClickHouse/ClickHouse/pull/4174) [\#4493](https://github.com/ClickHouse/ClickHouse/pull/4493) ([Vitaly Baranov](https://github.com/vitlibar))
+- Added bitmap functions with Roaring Bitmaps. [\#4207](https://github.com/ClickHouse/ClickHouse/pull/4207) ([Andy Yang](https://github.com/andyyzh)) [\#4568](https://github.com/ClickHouse/ClickHouse/pull/4568) ([Vitaly Baranov](https://github.com/vitlibar))
+- Parquet format support. [\#4448](https://github.com/ClickHouse/ClickHouse/pull/4448) ([proller](https://github.com/proller))
+- N-gram distance was added for fuzzy string comparison. It is similar to q-gram metrics in R language. [\#4466](https://github.com/ClickHouse/ClickHouse/pull/4466) ([Danila Kutenin](https://github.com/danlark1))
+- Combine rules for graphite rollup from dedicated aggregation and retention patterns. [\#4426](https://github.com/ClickHouse/ClickHouse/pull/4426) ([Mikhail f. Shiryaev](https://github.com/Felixoid))
+- Added `max_execution_speed` and `max_execution_speed_bytes` to limit resource usage. Added `min_execution_speed_bytes` setting to complement the `min_execution_speed`. [\#4430](https://github.com/ClickHouse/ClickHouse/pull/4430) ([Winter Zhang](https://github.com/zhang2014))
+- Implemented function `flatten`. [\#4555](https://github.com/ClickHouse/ClickHouse/pull/4555) [\#4409](https://github.com/ClickHouse/ClickHouse/pull/4409) ([alexey-milovidov](https://github.com/alexey-milovidov), [kzon](https://github.com/kzon))
+- Added functions `arrayEnumerateDenseRanked` and `arrayEnumerateUniqRanked` (it’s like `arrayEnumerateUniq` but allows to fine tune array depth to look inside multidimensional arrays). [\#4475](https://github.com/ClickHouse/ClickHouse/pull/4475) ([proller](https://github.com/proller)) [\#4601](https://github.com/ClickHouse/ClickHouse/pull/4601) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Multiple JOINS with some restrictions: no asterisks, no complex aliases in ON/WHERE/GROUP BY/… [\#4462](https://github.com/ClickHouse/ClickHouse/pull/4462) ([Artem Zuikov](https://github.com/4ertus2))
+
+#### Bug Fixes {#bug-fixes-11}
+
+- This release also contains all bug fixes from 19.3 and 19.1.
+- Fixed bug in data skipping indices: order of granules after INSERT was incorrect. [\#4407](https://github.com/ClickHouse/ClickHouse/pull/4407) ([Nikita Vasilev](https://github.com/nikvas0))
+- Fixed `set` index for `Nullable` and `LowCardinality` columns. Before it, `set` index with `Nullable` or `LowCardinality` column led to error `Data type must be deserialized with multiple streams` while selecting. [\#4594](https://github.com/ClickHouse/ClickHouse/pull/4594) ([Nikolai Kochetov](https://github.com/KochetovNicolai))
+- Correctly set update\_time on full `executable` dictionary update. [\#4551](https://github.com/ClickHouse/ClickHouse/pull/4551) ([Tema Novikov](https://github.com/temoon))
+- Fix broken progress bar in 19.3. [\#4627](https://github.com/ClickHouse/ClickHouse/pull/4627) ([filimonov](https://github.com/filimonov))
+- Fixed inconsistent values of MemoryTracker when memory region was shrinked, in certain cases. [\#4619](https://github.com/ClickHouse/ClickHouse/pull/4619) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Fixed undefined behaviour in ThreadPool. [\#4612](https://github.com/ClickHouse/ClickHouse/pull/4612) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Fixed a very rare crash with the message `mutex lock failed: Invalid argument` that could happen when a MergeTree table was dropped concurrently with a SELECT. [\#4608](https://github.com/ClickHouse/ClickHouse/pull/4608) ([Alex Zatelepin](https://github.com/ztlpn))
+- ODBC driver compatibility with `LowCardinality` data type. [\#4381](https://github.com/ClickHouse/ClickHouse/pull/4381) ([proller](https://github.com/proller))
+- FreeBSD: Fixup for `AIOcontextPool: Found io_event with unknown id 0` error. [\#4438](https://github.com/ClickHouse/ClickHouse/pull/4438) ([urgordeadbeef](https://github.com/urgordeadbeef))
+- `system.part_log` table was created regardless to configuration. [\#4483](https://github.com/ClickHouse/ClickHouse/pull/4483) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Fix undefined behaviour in `dictIsIn` function for cache dictionaries. [\#4515](https://github.com/ClickHouse/ClickHouse/pull/4515) ([alesapin](https://github.com/alesapin))
+- Fixed a deadlock when a SELECT query locks the same table multiple times (e.g. from different threads or when executing multiple subqueries) and there is a concurrent DDL query. [\#4535](https://github.com/ClickHouse/ClickHouse/pull/4535) ([Alex Zatelepin](https://github.com/ztlpn))
+- Disable compile\_expressions by default until we get own `llvm` contrib and can test it with `clang` and `asan`. [\#4579](https://github.com/ClickHouse/ClickHouse/pull/4579) ([alesapin](https://github.com/alesapin))
+- Prevent `std::terminate` when `invalidate_query` for `clickhouse` external dictionary source has returned wrong resultset (empty or more than one row or more than one column). Fixed issue when the `invalidate_query` was performed every five seconds regardless to the `lifetime`. [\#4583](https://github.com/ClickHouse/ClickHouse/pull/4583) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Avoid deadlock when the `invalidate_query` for a dictionary with `clickhouse` source was involving `system.dictionaries` table or `Dictionaries` database (rare case). [\#4599](https://github.com/ClickHouse/ClickHouse/pull/4599) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Fixes for CROSS JOIN with empty WHERE. [\#4598](https://github.com/ClickHouse/ClickHouse/pull/4598) ([Artem Zuikov](https://github.com/4ertus2))
+- Fixed segfault in function “replicate” when constant argument is passed. [\#4603](https://github.com/ClickHouse/ClickHouse/pull/4603) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Fix lambda function with predicate optimizer. [\#4408](https://github.com/ClickHouse/ClickHouse/pull/4408) ([Winter Zhang](https://github.com/zhang2014))
+- Multiple JOINs multiple fixes. [\#4595](https://github.com/ClickHouse/ClickHouse/pull/4595) ([Artem Zuikov](https://github.com/4ertus2))
+
+#### Improvements {#improvements-3}
+
+- Support aliases in JOIN ON section for right table columns. [\#4412](https://github.com/ClickHouse/ClickHouse/pull/4412) ([Artem Zuikov](https://github.com/4ertus2))
+- Result of multiple JOINs need correct result names to be used in subselects. Replace flat aliases with source names in result. [\#4474](https://github.com/ClickHouse/ClickHouse/pull/4474) ([Artem Zuikov](https://github.com/4ertus2))
+- Improve push-down logic for joined statements. [\#4387](https://github.com/ClickHouse/ClickHouse/pull/4387) ([Ivan](https://github.com/abyss7))
+
+#### Performance Improvements {#performance-improvements-3}
+
+- Improved heuristics of “move to PREWHERE” optimization. [\#4405](https://github.com/ClickHouse/ClickHouse/pull/4405) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Use proper lookup tables that uses HashTable’s API for 8-bit and 16-bit keys. [\#4536](https://github.com/ClickHouse/ClickHouse/pull/4536) ([Amos Bird](https://github.com/amosbird))
+- Improved performance of string comparison. [\#4564](https://github.com/ClickHouse/ClickHouse/pull/4564) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Cleanup distributed DDL queue in a separate thread so that it doesn’t slow down the main loop that processes distributed DDL tasks. [\#4502](https://github.com/ClickHouse/ClickHouse/pull/4502) ([Alex Zatelepin](https://github.com/ztlpn))
+- When `min_bytes_to_use_direct_io` is set to 1, not every file was opened with O\_DIRECT mode because the data size to read was sometimes underestimated by the size of one compressed block. [\#4526](https://github.com/ClickHouse/ClickHouse/pull/4526) ([alexey-milovidov](https://github.com/alexey-milovidov))
+
+#### Build/Testing/Packaging Improvement {#buildtestingpackaging-improvement-12}
+
+- Added support for clang-9 [\#4604](https://github.com/ClickHouse/ClickHouse/pull/4604) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Fix wrong `__asm__` instructions (again) [\#4621](https://github.com/ClickHouse/ClickHouse/pull/4621) ([Konstantin Podshumok](https://github.com/podshumok))
+- Add ability to specify settings for `clickhouse-performance-test` from command line. [\#4437](https://github.com/ClickHouse/ClickHouse/pull/4437) ([alesapin](https://github.com/alesapin))
+- Add dictionaries tests to integration tests. [\#4477](https://github.com/ClickHouse/ClickHouse/pull/4477) ([alesapin](https://github.com/alesapin))
+- Added queries from the benchmark on the website to automated performance tests. [\#4496](https://github.com/ClickHouse/ClickHouse/pull/4496) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- `xxhash.h` does not exist in external lz4 because it is an implementation detail and its symbols are namespaced with `XXH_NAMESPACE` macro. When lz4 is external, xxHash has to be external too, and the dependents have to link to it. [\#4495](https://github.com/ClickHouse/ClickHouse/pull/4495) ([Orivej Desh](https://github.com/orivej))
+- Fixed a case when `quantileTiming` aggregate function can be called with negative or floating point argument (this fixes fuzz test with undefined behaviour sanitizer). [\#4506](https://github.com/ClickHouse/ClickHouse/pull/4506) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Spelling error correction. [\#4531](https://github.com/ClickHouse/ClickHouse/pull/4531) ([sdk2](https://github.com/sdk2))
+- Fix compilation on Mac. [\#4371](https://github.com/ClickHouse/ClickHouse/pull/4371) ([Vitaly Baranov](https://github.com/vitlibar))
+- Build fixes for FreeBSD and various unusual build configurations. [\#4444](https://github.com/ClickHouse/ClickHouse/pull/4444) ([proller](https://github.com/proller))
+
+## ClickHouse release 19.3 {#clickhouse-release-19-3}
+
+### ClickHouse release 19.3.9.1, 2019-04-02 {#clickhouse-release-19-3-9-1-2019-04-02}
+
+#### Bug Fixes {#bug-fixes-12}
+
+- Fix crash in `FULL/RIGHT JOIN` when we joining on nullable vs not nullable. [\#4855](https://github.com/ClickHouse/ClickHouse/pull/4855) ([Artem Zuikov](https://github.com/4ertus2))
+- Fix segmentation fault in `clickhouse-copier`. [\#4835](https://github.com/ClickHouse/ClickHouse/pull/4835) ([proller](https://github.com/proller))
+- Fixed reading from `Array(LowCardinality)` column in rare case when column contained a long sequence of empty arrays. [\#4850](https://github.com/ClickHouse/ClickHouse/pull/4850) ([Nikolai Kochetov](https://github.com/KochetovNicolai))
+
+#### Build/Testing/Packaging Improvement {#buildtestingpackaging-improvement-13}
+
+- Add a way to launch clickhouse-server image from a custom user [\#4753](https://github.com/ClickHouse/ClickHouse/pull/4753) ([Mikhail f. Shiryaev](https://github.com/Felixoid))
+
+### ClickHouse release 19.3.7, 2019-03-12 {#clickhouse-release-19-3-7-2019-03-12}
+
+#### Bug fixes {#bug-fixes-13}
+
+- Fixed error in \#3920. This error manifests itself as random cache corruption (messages `Unknown codec family code`, `Cannot seek through file`) and segfaults. This bug first appeared in version 19.1 and is present in versions up to 19.1.10 and 19.3.6. [\#4623](https://github.com/ClickHouse/ClickHouse/pull/4623) ([alexey-milovidov](https://github.com/alexey-milovidov))
+
+### ClickHouse release 19.3.6, 2019-03-02 {#clickhouse-release-19-3-6-2019-03-02}
+
+#### Bug fixes {#bug-fixes-14}
+
+- When there are more than 1000 threads in a thread pool, `std::terminate` may happen on thread exit. [Azat Khuzhin](https://github.com/azat) [\#4485](https://github.com/ClickHouse/ClickHouse/pull/4485) [\#4505](https://github.com/ClickHouse/ClickHouse/pull/4505) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Now it’s possible to create `ReplicatedMergeTree*` tables with comments on columns without defaults and tables with columns codecs without comments and defaults. Also fix comparison of codecs. [\#4523](https://github.com/ClickHouse/ClickHouse/pull/4523) ([alesapin](https://github.com/alesapin))
+- Fixed crash on JOIN with array or tuple. [\#4552](https://github.com/ClickHouse/ClickHouse/pull/4552) ([Artem Zuikov](https://github.com/4ertus2))
+- Fixed crash in clickhouse-copier with the message `ThreadStatus not created`. [\#4540](https://github.com/ClickHouse/ClickHouse/pull/4540) ([Artem Zuikov](https://github.com/4ertus2))
+- Fixed hangup on server shutdown if distributed DDLs were used. [\#4472](https://github.com/ClickHouse/ClickHouse/pull/4472) ([Alex Zatelepin](https://github.com/ztlpn))
+- Incorrect column numbers were printed in error message about text format parsing for columns with number greater than 10. [\#4484](https://github.com/ClickHouse/ClickHouse/pull/4484) ([alexey-milovidov](https://github.com/alexey-milovidov))
+
+#### Build/Testing/Packaging Improvements {#buildtestingpackaging-improvements-3}
+
+- Fixed build with AVX enabled. [\#4527](https://github.com/ClickHouse/ClickHouse/pull/4527) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Enable extended accounting and IO accounting based on good known version instead of kernel under which it is compiled. [\#4541](https://github.com/ClickHouse/ClickHouse/pull/4541) ([nvartolomei](https://github.com/nvartolomei))
+- Allow to skip setting of core\_dump.size\_limit, warning instead of throw if limit set fail. [\#4473](https://github.com/ClickHouse/ClickHouse/pull/4473) ([proller](https://github.com/proller))
+- Removed the `inline` tags of `void readBinary(...)` in `Field.cpp`. Also merged redundant `namespace DB` blocks. [\#4530](https://github.com/ClickHouse/ClickHouse/pull/4530) ([hcz](https://github.com/hczhcz))
+
+### ClickHouse release 19.3.5, 2019-02-21 {#clickhouse-release-19-3-5-2019-02-21}
+
+#### Bug fixes {#bug-fixes-15}
+
+- Fixed bug with large http insert queries processing. [\#4454](https://github.com/ClickHouse/ClickHouse/pull/4454) ([alesapin](https://github.com/alesapin))
+- Fixed backward incompatibility with old versions due to wrong implementation of `send_logs_level` setting. [\#4445](https://github.com/ClickHouse/ClickHouse/pull/4445) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Fixed backward incompatibility of table function `remote` introduced with column comments. [\#4446](https://github.com/ClickHouse/ClickHouse/pull/4446) ([alexey-milovidov](https://github.com/alexey-milovidov))
+
+### ClickHouse release 19.3.4, 2019-02-16 {#clickhouse-release-19-3-4-2019-02-16}
+
+#### Improvements {#improvements-4}
+
+- Table index size is not accounted for memory limits when doing `ATTACH TABLE` query. Avoided the possibility that a table cannot be attached after being detached. [\#4396](https://github.com/ClickHouse/ClickHouse/pull/4396) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Slightly raised up the limit on max string and array size received from ZooKeeper. It allows to continue to work with increased size of `CLIENT_JVMFLAGS=-Djute.maxbuffer=...` on ZooKeeper. [\#4398](https://github.com/ClickHouse/ClickHouse/pull/4398) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Allow to repair abandoned replica even if it already has huge number of nodes in its queue. [\#4399](https://github.com/ClickHouse/ClickHouse/pull/4399) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Add one required argument to `SET` index (max stored rows number). [\#4386](https://github.com/ClickHouse/ClickHouse/pull/4386) ([Nikita Vasilev](https://github.com/nikvas0))
+
+#### Bug Fixes {#bug-fixes-16}
+
+- Fixed `WITH ROLLUP` result for group by single `LowCardinality` key. [\#4384](https://github.com/ClickHouse/ClickHouse/pull/4384) ([Nikolai Kochetov](https://github.com/KochetovNicolai))
+- Fixed bug in the set index (dropping a granule if it contains more than `max_rows` rows). [\#4386](https://github.com/ClickHouse/ClickHouse/pull/4386) ([Nikita Vasilev](https://github.com/nikvas0))
+- A lot of FreeBSD build fixes. [\#4397](https://github.com/ClickHouse/ClickHouse/pull/4397) ([proller](https://github.com/proller))
+- Fixed aliases substitution in queries with subquery containing same alias (issue [\#4110](https://github.com/ClickHouse/ClickHouse/issues/4110)). [\#4351](https://github.com/ClickHouse/ClickHouse/pull/4351) ([Artem Zuikov](https://github.com/4ertus2))
+
+#### Build/Testing/Packaging Improvements {#buildtestingpackaging-improvements-4}
+
+- Add ability to run `clickhouse-server` for stateless tests in docker image. [\#4347](https://github.com/ClickHouse/ClickHouse/pull/4347) ([Vasily Nemkov](https://github.com/Enmk))
+
+### ClickHouse release 19.3.3, 2019-02-13 {#clickhouse-release-19-3-3-2019-02-13}
+
+#### New Features {#new-features-6}
+
+- Added the `KILL MUTATION` statement that allows removing mutations that are for some reasons stuck. Added `latest_failed_part`, `latest_fail_time`, `latest_fail_reason` fields to the `system.mutations` table for easier troubleshooting. [\#4287](https://github.com/ClickHouse/ClickHouse/pull/4287) ([Alex Zatelepin](https://github.com/ztlpn))
+- Added aggregate function `entropy` which computes Shannon entropy. [\#4238](https://github.com/ClickHouse/ClickHouse/pull/4238) ([Quid37](https://github.com/Quid37))
+- Added ability to send queries `INSERT INTO tbl VALUES (....` to server without splitting on `query` and `data` parts. [\#4301](https://github.com/ClickHouse/ClickHouse/pull/4301) ([alesapin](https://github.com/alesapin))
+- Generic implementation of `arrayWithConstant` function was added. [\#4322](https://github.com/ClickHouse/ClickHouse/pull/4322) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Implemented `NOT BETWEEN` comparison operator. [\#4228](https://github.com/ClickHouse/ClickHouse/pull/4228) ([Dmitry Naumov](https://github.com/nezed))
+- Implement `sumMapFiltered` in order to be able to limit the number of keys for which values will be summed by `sumMap`. [\#4129](https://github.com/ClickHouse/ClickHouse/pull/4129) ([Léo Ercolanelli](https://github.com/ercolanelli-leo))
+- Added support of `Nullable` types in `mysql` table function. [\#4198](https://github.com/ClickHouse/ClickHouse/pull/4198) ([Emmanuel Donin de Rosière](https://github.com/edonin))
+- Support for arbitrary constant expressions in `LIMIT` clause. [\#4246](https://github.com/ClickHouse/ClickHouse/pull/4246) ([k3box](https://github.com/k3box))
+- Added `topKWeighted` aggregate function that takes additional argument with (unsigned integer) weight. [\#4245](https://github.com/ClickHouse/ClickHouse/pull/4245) ([Andrew Golman](https://github.com/andrewgolman))
+- `StorageJoin` now supports `join_any_take_last_row` setting that allows overwriting existing values of the same key. [\#3973](https://github.com/ClickHouse/ClickHouse/pull/3973) ([Amos Bird](https://github.com/amosbird)
+- Added function `toStartOfInterval`. [\#4304](https://github.com/ClickHouse/ClickHouse/pull/4304) ([Vitaly Baranov](https://github.com/vitlibar))
+- Added `RowBinaryWithNamesAndTypes` format. [\#4200](https://github.com/ClickHouse/ClickHouse/pull/4200) ([Oleg V. Kozlyuk](https://github.com/DarkWanderer))
+- Added `IPv4` and `IPv6` data types. More effective implementations of `IPv*` functions. [\#3669](https://github.com/ClickHouse/ClickHouse/pull/3669) ([Vasily Nemkov](https://github.com/Enmk))
+- Added function `toStartOfTenMinutes()`. [\#4298](https://github.com/ClickHouse/ClickHouse/pull/4298) ([Vitaly Baranov](https://github.com/vitlibar))
+- Added `Protobuf` output format. [\#4005](https://github.com/ClickHouse/ClickHouse/pull/4005) [\#4158](https://github.com/ClickHouse/ClickHouse/pull/4158) ([Vitaly Baranov](https://github.com/vitlibar))
+- Added brotli support for HTTP interface for data import (INSERTs). [\#4235](https://github.com/ClickHouse/ClickHouse/pull/4235) ([Mikhail](https://github.com/fandyushin))
+- Added hints while user make typo in function name or type in command line client. [\#4239](https://github.com/ClickHouse/ClickHouse/pull/4239) ([Danila Kutenin](https://github.com/danlark1))
+- Added `Query-Id` to Server’s HTTP Response header. [\#4231](https://github.com/ClickHouse/ClickHouse/pull/4231) ([Mikhail](https://github.com/fandyushin))
+
+#### Experimental features {#experimental-features-2}
+
+- Added `minmax` and `set` data skipping indices for MergeTree table engines family. [\#4143](https://github.com/ClickHouse/ClickHouse/pull/4143) ([Nikita Vasilev](https://github.com/nikvas0))
+- Added conversion of `CROSS JOIN` to `INNER JOIN` if possible. [\#4221](https://github.com/ClickHouse/ClickHouse/pull/4221) [\#4266](https://github.com/ClickHouse/ClickHouse/pull/4266) ([Artem Zuikov](https://github.com/4ertus2))
+
+#### Bug Fixes {#bug-fixes-17}
+
+- Fixed `Not found column` for duplicate columns in `JOIN ON` section. [\#4279](https://github.com/ClickHouse/ClickHouse/pull/4279) ([Artem Zuikov](https://github.com/4ertus2))
+- Make `START REPLICATED SENDS` command start replicated sends. [\#4229](https://github.com/ClickHouse/ClickHouse/pull/4229) ([nvartolomei](https://github.com/nvartolomei))
+- Fixed aggregate functions execution with `Array(LowCardinality)` arguments. [\#4055](https://github.com/ClickHouse/ClickHouse/pull/4055) ([KochetovNicolai](https://github.com/KochetovNicolai))
+- Fixed wrong behaviour when doing `INSERT ... SELECT ... FROM file(...)` query and file has `CSVWithNames` or `TSVWIthNames` format and the first data row is missing. [\#4297](https://github.com/ClickHouse/ClickHouse/pull/4297) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Fixed crash on dictionary reload if dictionary not available. This bug was appeared in 19.1.6. [\#4188](https://github.com/ClickHouse/ClickHouse/pull/4188) ([proller](https://github.com/proller))
+- Fixed `ALL JOIN` with duplicates in right table. [\#4184](https://github.com/ClickHouse/ClickHouse/pull/4184) ([Artem Zuikov](https://github.com/4ertus2))
+- Fixed segmentation fault with `use_uncompressed_cache=1` and exception with wrong uncompressed size. This bug was appeared in 19.1.6. [\#4186](https://github.com/ClickHouse/ClickHouse/pull/4186) ([alesapin](https://github.com/alesapin))
+- Fixed `compile_expressions` bug with comparison of big (more than int16) dates. [\#4341](https://github.com/ClickHouse/ClickHouse/pull/4341) ([alesapin](https://github.com/alesapin))
+- Fixed infinite loop when selecting from table function `numbers(0)`. [\#4280](https://github.com/ClickHouse/ClickHouse/pull/4280) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Temporarily disable predicate optimization for `ORDER BY`. [\#3890](https://github.com/ClickHouse/ClickHouse/pull/3890) ([Winter Zhang](https://github.com/zhang2014))
+- Fixed `Illegal instruction` error when using base64 functions on old CPUs. This error has been reproduced only when ClickHouse was compiled with gcc-8. [\#4275](https://github.com/ClickHouse/ClickHouse/pull/4275) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Fixed `No message received` error when interacting with PostgreSQL ODBC Driver through TLS connection. Also fixes segfault when using MySQL ODBC Driver. [\#4170](https://github.com/ClickHouse/ClickHouse/pull/4170) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Fixed incorrect result when `Date` and `DateTime` arguments are used in branches of conditional operator (function `if`). Added generic case for function `if`. [\#4243](https://github.com/ClickHouse/ClickHouse/pull/4243) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- ClickHouse dictionaries now load within `clickhouse` process. [\#4166](https://github.com/ClickHouse/ClickHouse/pull/4166) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Fixed deadlock when `SELECT` from a table with `File` engine was retried after `No such file or directory` error. [\#4161](https://github.com/ClickHouse/ClickHouse/pull/4161) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Fixed race condition when selecting from `system.tables` may give `table doesn't exist` error. [\#4313](https://github.com/ClickHouse/ClickHouse/pull/4313) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- `clickhouse-client` can segfault on exit while loading data for command line suggestions if it was run in interactive mode. [\#4317](https://github.com/ClickHouse/ClickHouse/pull/4317) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Fixed a bug when the execution of mutations containing `IN` operators was producing incorrect results. [\#4099](https://github.com/ClickHouse/ClickHouse/pull/4099) ([Alex Zatelepin](https://github.com/ztlpn))
+- Fixed error: if there is a database with `Dictionary` engine, all dictionaries forced to load at server startup, and if there is a dictionary with ClickHouse source from localhost, the dictionary cannot load. [\#4255](https://github.com/ClickHouse/ClickHouse/pull/4255) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Fixed error when system logs are tried to create again at server shutdown. [\#4254](https://github.com/ClickHouse/ClickHouse/pull/4254) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Correctly return the right type and properly handle locks in `joinGet` function. [\#4153](https://github.com/ClickHouse/ClickHouse/pull/4153) ([Amos Bird](https://github.com/amosbird))
+- Added `sumMapWithOverflow` function. [\#4151](https://github.com/ClickHouse/ClickHouse/pull/4151) ([Léo Ercolanelli](https://github.com/ercolanelli-leo))
+- Fixed segfault with `allow_experimental_multiple_joins_emulation`. [52de2c](https://github.com/ClickHouse/ClickHouse/commit/52de2cd927f7b5257dd67e175f0a5560a48840d0) ([Artem Zuikov](https://github.com/4ertus2))
+- Fixed bug with incorrect `Date` and `DateTime` comparison. [\#4237](https://github.com/ClickHouse/ClickHouse/pull/4237) ([valexey](https://github.com/valexey))
+- Fixed fuzz test under undefined behavior sanitizer: added parameter type check for `quantile*Weighted` family of functions. [\#4145](https://github.com/ClickHouse/ClickHouse/pull/4145) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Fixed rare race condition when removing of old data parts can fail with `File not found` error. [\#4378](https://github.com/ClickHouse/ClickHouse/pull/4378) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Fix install package with missing /etc/clickhouse-server/config.xml. [\#4343](https://github.com/ClickHouse/ClickHouse/pull/4343) ([proller](https://github.com/proller))
+
+#### Build/Testing/Packaging Improvements {#buildtestingpackaging-improvements-5}
+
+- Debian package: correct /etc/clickhouse-server/preprocessed link according to config. [\#4205](https://github.com/ClickHouse/ClickHouse/pull/4205) ([proller](https://github.com/proller))
+- Various build fixes for FreeBSD. [\#4225](https://github.com/ClickHouse/ClickHouse/pull/4225) ([proller](https://github.com/proller))
+- Added ability to create, fill and drop tables in perftest. [\#4220](https://github.com/ClickHouse/ClickHouse/pull/4220) ([alesapin](https://github.com/alesapin))
+- Added a script to check for duplicate includes. [\#4326](https://github.com/ClickHouse/ClickHouse/pull/4326) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Added ability to run queries by index in performance test. [\#4264](https://github.com/ClickHouse/ClickHouse/pull/4264) ([alesapin](https://github.com/alesapin))
+- Package with debug symbols is suggested to be installed. [\#4274](https://github.com/ClickHouse/ClickHouse/pull/4274) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Refactoring of performance-test. Better logging and signals handling. [\#4171](https://github.com/ClickHouse/ClickHouse/pull/4171) ([alesapin](https://github.com/alesapin))
+- Added docs to anonymized Yandex.Metrika datasets. [\#4164](https://github.com/ClickHouse/ClickHouse/pull/4164) ([alesapin](https://github.com/alesapin))
+- Аdded tool for converting an old month-partitioned part to the custom-partitioned format. [\#4195](https://github.com/ClickHouse/ClickHouse/pull/4195) ([Alex Zatelepin](https://github.com/ztlpn))
+- Added docs about two datasets in s3. [\#4144](https://github.com/ClickHouse/ClickHouse/pull/4144) ([alesapin](https://github.com/alesapin))
+- Added script which creates changelog from pull requests description. [\#4169](https://github.com/ClickHouse/ClickHouse/pull/4169) [\#4173](https://github.com/ClickHouse/ClickHouse/pull/4173) ([KochetovNicolai](https://github.com/KochetovNicolai)) ([KochetovNicolai](https://github.com/KochetovNicolai))
+- Added puppet module for Clickhouse. [\#4182](https://github.com/ClickHouse/ClickHouse/pull/4182) ([Maxim Fedotov](https://github.com/MaxFedotov))
+- Added docs for a group of undocumented functions. [\#4168](https://github.com/ClickHouse/ClickHouse/pull/4168) ([Winter Zhang](https://github.com/zhang2014))
+- ARM build fixes. [\#4210](https://github.com/ClickHouse/ClickHouse/pull/4210)[\#4306](https://github.com/ClickHouse/ClickHouse/pull/4306) [\#4291](https://github.com/ClickHouse/ClickHouse/pull/4291) ([proller](https://github.com/proller)) ([proller](https://github.com/proller))
+- Dictionary tests now able to run from `ctest`. [\#4189](https://github.com/ClickHouse/ClickHouse/pull/4189) ([proller](https://github.com/proller))
+- Now `/etc/ssl` is used as default directory with SSL certificates. [\#4167](https://github.com/ClickHouse/ClickHouse/pull/4167) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Added checking SSE and AVX instruction at start. [\#4234](https://github.com/ClickHouse/ClickHouse/pull/4234) ([Igr](https://github.com/igron99))
+- Init script will wait server until start. [\#4281](https://github.com/ClickHouse/ClickHouse/pull/4281) ([proller](https://github.com/proller))
+
+#### Backward Incompatible Changes {#backward-incompatible-changes-1}
+
+- Removed `allow_experimental_low_cardinality_type` setting. `LowCardinality` data types are production ready. [\#4323](https://github.com/ClickHouse/ClickHouse/pull/4323) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Reduce mark cache size and uncompressed cache size accordingly to available memory amount. [\#4240](https://github.com/ClickHouse/ClickHouse/pull/4240) ([Lopatin Konstantin](https://github.com/k-lopatin)
+- Added keyword `INDEX` in `CREATE TABLE` query. A column with name `index` must be quoted with backticks or double quotes: `` `index` ``. [\#4143](https://github.com/ClickHouse/ClickHouse/pull/4143) ([Nikita Vasilev](https://github.com/nikvas0))
+- `sumMap` now promote result type instead of overflow. The old `sumMap` behavior can be obtained by using `sumMapWithOverflow` function. [\#4151](https://github.com/ClickHouse/ClickHouse/pull/4151) ([Léo Ercolanelli](https://github.com/ercolanelli-leo))
+
+#### Performance Improvements {#performance-improvements-4}
+
+- `std::sort` replaced by `pdqsort` for queries without `LIMIT`. [\#4236](https://github.com/ClickHouse/ClickHouse/pull/4236) ([Evgenii Pravda](https://github.com/kvinty))
+- Now server reuse threads from global thread pool. This affects performance in some corner cases. [\#4150](https://github.com/ClickHouse/ClickHouse/pull/4150) ([alexey-milovidov](https://github.com/alexey-milovidov))
+
+#### Improvements {#improvements-5}
+
+- Implemented AIO support for FreeBSD. [\#4305](https://github.com/ClickHouse/ClickHouse/pull/4305) ([urgordeadbeef](https://github.com/urgordeadbeef))
+- `SELECT * FROM a JOIN b USING a, b` now return `a` and `b` columns only from the left table. [\#4141](https://github.com/ClickHouse/ClickHouse/pull/4141) ([Artem Zuikov](https://github.com/4ertus2))
+- Allow `-C` option of client to work as `-c` option. [\#4232](https://github.com/ClickHouse/ClickHouse/pull/4232) ([syominsergey](https://github.com/syominsergey))
+- Now option `--password` used without value requires password from stdin. [\#4230](https://github.com/ClickHouse/ClickHouse/pull/4230) ([BSD\_Conqueror](https://github.com/bsd-conqueror))
+- Added highlighting of unescaped metacharacters in string literals that contain `LIKE` expressions or regexps. [\#4327](https://github.com/ClickHouse/ClickHouse/pull/4327) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Added cancelling of HTTP read only queries if client socket goes away. [\#4213](https://github.com/ClickHouse/ClickHouse/pull/4213) ([nvartolomei](https://github.com/nvartolomei))
+- Now server reports progress to keep client connections alive. [\#4215](https://github.com/ClickHouse/ClickHouse/pull/4215) ([Ivan](https://github.com/abyss7))
+- Slightly better message with reason for OPTIMIZE query with `optimize_throw_if_noop` setting enabled. [\#4294](https://github.com/ClickHouse/ClickHouse/pull/4294) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Added support of `--version` option for clickhouse server. [\#4251](https://github.com/ClickHouse/ClickHouse/pull/4251) ([Lopatin Konstantin](https://github.com/k-lopatin))
+- Added `--help/-h` option to `clickhouse-server`. [\#4233](https://github.com/ClickHouse/ClickHouse/pull/4233) ([Yuriy Baranov](https://github.com/yurriy))
+- Added support for scalar subqueries with aggregate function state result. [\#4348](https://github.com/ClickHouse/ClickHouse/pull/4348) ([Nikolai Kochetov](https://github.com/KochetovNicolai))
+- Improved server shutdown time and ALTERs waiting time. [\#4372](https://github.com/ClickHouse/ClickHouse/pull/4372) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Added info about the replicated\_can\_become\_leader setting to system.replicas and add logging if the replica won’t try to become leader. [\#4379](https://github.com/ClickHouse/ClickHouse/pull/4379) ([Alex Zatelepin](https://github.com/ztlpn))
+
+## ClickHouse release 19.1 {#clickhouse-release-19-1}
+
+### ClickHouse release 19.1.14, 2019-03-14 {#clickhouse-release-19-1-14-2019-03-14}
+
+- Fixed error `Column ... queried more than once` that may happen if the setting `asterisk_left_columns_only` is set to 1 in case of using `GLOBAL JOIN` with `SELECT *` (rare case). The issue does not exist in 19.3 and newer. [6bac7d8d](https://github.com/ClickHouse/ClickHouse/pull/4692/commits/6bac7d8d11a9b0d6de0b32b53c47eb2f6f8e7062) ([Artem Zuikov](https://github.com/4ertus2))
+
+### ClickHouse release 19.1.13, 2019-03-12 {#clickhouse-release-19-1-13-2019-03-12}
+
+This release contains exactly the same set of patches as 19.3.7.
+
+### ClickHouse release 19.1.10, 2019-03-03 {#clickhouse-release-19-1-10-2019-03-03}
+
+This release contains exactly the same set of patches as 19.3.6.
+
+## ClickHouse release 19.1 {#clickhouse-release-19-1-1}
+
+### ClickHouse release 19.1.9, 2019-02-21 {#clickhouse-release-19-1-9-2019-02-21}
+
+#### Bug fixes {#bug-fixes-18}
+
+- Fixed backward incompatibility with old versions due to wrong implementation of `send_logs_level` setting. [\#4445](https://github.com/ClickHouse/ClickHouse/pull/4445) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Fixed backward incompatibility of table function `remote` introduced with column comments. [\#4446](https://github.com/ClickHouse/ClickHouse/pull/4446) ([alexey-milovidov](https://github.com/alexey-milovidov))
+
+### ClickHouse release 19.1.8, 2019-02-16 {#clickhouse-release-19-1-8-2019-02-16}
+
+#### Bug Fixes {#bug-fixes-19}
+
+- Fix install package with missing /etc/clickhouse-server/config.xml. [\#4343](https://github.com/ClickHouse/ClickHouse/pull/4343) ([proller](https://github.com/proller))
+
+## ClickHouse release 19.1 {#clickhouse-release-19-1-2}
+
+### ClickHouse release 19.1.7, 2019-02-15 {#clickhouse-release-19-1-7-2019-02-15}
+
+#### Bug Fixes {#bug-fixes-20}
+
+- Correctly return the right type and properly handle locks in `joinGet` function. [\#4153](https://github.com/ClickHouse/ClickHouse/pull/4153) ([Amos Bird](https://github.com/amosbird))
+- Fixed error when system logs are tried to create again at server shutdown. [\#4254](https://github.com/ClickHouse/ClickHouse/pull/4254) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Fixed error: if there is a database with `Dictionary` engine, all dictionaries forced to load at server startup, and if there is a dictionary with ClickHouse source from localhost, the dictionary cannot load. [\#4255](https://github.com/ClickHouse/ClickHouse/pull/4255) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Fixed a bug when the execution of mutations containing `IN` operators was producing incorrect results. [\#4099](https://github.com/ClickHouse/ClickHouse/pull/4099) ([Alex Zatelepin](https://github.com/ztlpn))
+- `clickhouse-client` can segfault on exit while loading data for command line suggestions if it was run in interactive mode. [\#4317](https://github.com/ClickHouse/ClickHouse/pull/4317) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Fixed race condition when selecting from `system.tables` may give `table doesn't exist` error. [\#4313](https://github.com/ClickHouse/ClickHouse/pull/4313) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Fixed deadlock when `SELECT` from a table with `File` engine was retried after `No such file or directory` error. [\#4161](https://github.com/ClickHouse/ClickHouse/pull/4161) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Fixed an issue: local ClickHouse dictionaries are loaded via TCP, but should load within process. [\#4166](https://github.com/ClickHouse/ClickHouse/pull/4166) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Fixed `No message received` error when interacting with PostgreSQL ODBC Driver through TLS connection. Also fixes segfault when using MySQL ODBC Driver. [\#4170](https://github.com/ClickHouse/ClickHouse/pull/4170) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Temporarily disable predicate optimization for `ORDER BY`. [\#3890](https://github.com/ClickHouse/ClickHouse/pull/3890) ([Winter Zhang](https://github.com/zhang2014))
+- Fixed infinite loop when selecting from table function `numbers(0)`. [\#4280](https://github.com/ClickHouse/ClickHouse/pull/4280) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Fixed `compile_expressions` bug with comparison of big (more than int16) dates. [\#4341](https://github.com/ClickHouse/ClickHouse/pull/4341) ([alesapin](https://github.com/alesapin))
+- Fixed segmentation fault with `uncompressed_cache=1` and exception with wrong uncompressed size. [\#4186](https://github.com/ClickHouse/ClickHouse/pull/4186) ([alesapin](https://github.com/alesapin))
+- Fixed `ALL JOIN` with duplicates in right table. [\#4184](https://github.com/ClickHouse/ClickHouse/pull/4184) ([Artem Zuikov](https://github.com/4ertus2))
+- Fixed wrong behaviour when doing `INSERT ... SELECT ... FROM file(...)` query and file has `CSVWithNames` or `TSVWIthNames` format and the first data row is missing. [\#4297](https://github.com/ClickHouse/ClickHouse/pull/4297) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Fixed aggregate functions execution with `Array(LowCardinality)` arguments. [\#4055](https://github.com/ClickHouse/ClickHouse/pull/4055) ([KochetovNicolai](https://github.com/KochetovNicolai))
+- Debian package: correct /etc/clickhouse-server/preprocessed link according to config. [\#4205](https://github.com/ClickHouse/ClickHouse/pull/4205) ([proller](https://github.com/proller))
+- Fixed fuzz test under undefined behavior sanitizer: added parameter type check for `quantile*Weighted` family of functions. [\#4145](https://github.com/ClickHouse/ClickHouse/pull/4145) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Make `START REPLICATED SENDS` command start replicated sends. [\#4229](https://github.com/ClickHouse/ClickHouse/pull/4229) ([nvartolomei](https://github.com/nvartolomei))
+- Fixed `Not found column` for duplicate columns in JOIN ON section. [\#4279](https://github.com/ClickHouse/ClickHouse/pull/4279) ([Artem Zuikov](https://github.com/4ertus2))
+- Now `/etc/ssl` is used as default directory with SSL certificates. [\#4167](https://github.com/ClickHouse/ClickHouse/pull/4167) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Fixed crash on dictionary reload if dictionary not available. [\#4188](https://github.com/ClickHouse/ClickHouse/pull/4188) ([proller](https://github.com/proller))
+- Fixed bug with incorrect `Date` and `DateTime` comparison. [\#4237](https://github.com/ClickHouse/ClickHouse/pull/4237) ([valexey](https://github.com/valexey))
+- Fixed incorrect result when `Date` and `DateTime` arguments are used in branches of conditional operator (function `if`). Added generic case for function `if`. [\#4243](https://github.com/ClickHouse/ClickHouse/pull/4243) ([alexey-milovidov](https://github.com/alexey-milovidov))
+
+### ClickHouse release 19.1.6, 2019-01-24 {#clickhouse-release-19-1-6-2019-01-24}
+
+#### New Features {#new-features-7}
+
+- Custom per column compression codecs for tables. [\#3899](https://github.com/ClickHouse/ClickHouse/pull/3899) [\#4111](https://github.com/ClickHouse/ClickHouse/pull/4111) ([alesapin](https://github.com/alesapin), [Winter Zhang](https://github.com/zhang2014), [Anatoly](https://github.com/Sindbag))
+- Added compression codec `Delta`. [\#4052](https://github.com/ClickHouse/ClickHouse/pull/4052) ([alesapin](https://github.com/alesapin))
+- Allow to `ALTER` compression codecs. [\#4054](https://github.com/ClickHouse/ClickHouse/pull/4054) ([alesapin](https://github.com/alesapin))
+- Added functions `left`, `right`, `trim`, `ltrim`, `rtrim`, `timestampadd`, `timestampsub` for SQL standard compatibility. [\#3826](https://github.com/ClickHouse/ClickHouse/pull/3826) ([Ivan Blinkov](https://github.com/blinkov))
+- Support for write in `HDFS` tables and `hdfs` table function. [\#4084](https://github.com/ClickHouse/ClickHouse/pull/4084) ([alesapin](https://github.com/alesapin))
+- Added functions to search for multiple constant strings from big haystack: `multiPosition`, `multiSearch` ,`firstMatch` also with `-UTF8`, `-CaseInsensitive`, and `-CaseInsensitiveUTF8` variants. [\#4053](https://github.com/ClickHouse/ClickHouse/pull/4053) ([Danila Kutenin](https://github.com/danlark1))
+- Pruning of unused shards if `SELECT` query filters by sharding key (setting `optimize_skip_unused_shards`). [\#3851](https://github.com/ClickHouse/ClickHouse/pull/3851) ([Gleb Kanterov](https://github.com/kanterov), [Ivan](https://github.com/abyss7))
+- Allow `Kafka` engine to ignore some number of parsing errors per block. [\#4094](https://github.com/ClickHouse/ClickHouse/pull/4094) ([Ivan](https://github.com/abyss7))
+- Added support for `CatBoost` multiclass models evaluation. Function `modelEvaluate` returns tuple with per-class raw predictions for multiclass models. `libcatboostmodel.so` should be built with [\#607](https://github.com/catboost/catboost/pull/607). [\#3959](https://github.com/ClickHouse/ClickHouse/pull/3959) ([KochetovNicolai](https://github.com/KochetovNicolai))
+- Added functions `filesystemAvailable`, `filesystemFree`, `filesystemCapacity`. [\#4097](https://github.com/ClickHouse/ClickHouse/pull/4097) ([Boris Granveaud](https://github.com/bgranvea))
+- Added hashing functions `xxHash64` and `xxHash32`. [\#3905](https://github.com/ClickHouse/ClickHouse/pull/3905) ([filimonov](https://github.com/filimonov))
+- Added `gccMurmurHash` hashing function (GCC flavoured Murmur hash) which uses the same hash seed as [gcc](https://github.com/gcc-mirror/gcc/blob/41d6b10e96a1de98e90a7c0378437c3255814b16/libstdc%2B%2B-v3/include/bits/functional_hash.h#L191) [\#4000](https://github.com/ClickHouse/ClickHouse/pull/4000) ([sundyli](https://github.com/sundy-li))
+- Added hashing functions `javaHash`, `hiveHash`. [\#3811](https://github.com/ClickHouse/ClickHouse/pull/3811) ([shangshujie365](https://github.com/shangshujie365))
+- Added table function `remoteSecure`. Function works as `remote`, but uses secure connection. [\#4088](https://github.com/ClickHouse/ClickHouse/pull/4088) ([proller](https://github.com/proller))
+
+#### Experimental features {#experimental-features-3}
+
+- Added multiple JOINs emulation (`allow_experimental_multiple_joins_emulation` setting). [\#3946](https://github.com/ClickHouse/ClickHouse/pull/3946) ([Artem Zuikov](https://github.com/4ertus2))
+
+#### Bug Fixes {#bug-fixes-21}
+
+- Make `compiled_expression_cache_size` setting limited by default to lower memory consumption. [\#4041](https://github.com/ClickHouse/ClickHouse/pull/4041) ([alesapin](https://github.com/alesapin))
+- Fix a bug that led to hangups in threads that perform ALTERs of Replicated tables and in the thread that updates configuration from ZooKeeper. [\#2947](https://github.com/ClickHouse/ClickHouse/issues/2947) [\#3891](https://github.com/ClickHouse/ClickHouse/issues/3891) [\#3934](https://github.com/ClickHouse/ClickHouse/pull/3934) ([Alex Zatelepin](https://github.com/ztlpn))
+- Fixed a race condition when executing a distributed ALTER task. The race condition led to more than one replica trying to execute the task and all replicas except one failing with a ZooKeeper error. [\#3904](https://github.com/ClickHouse/ClickHouse/pull/3904) ([Alex Zatelepin](https://github.com/ztlpn))
+- Fix a bug when `from_zk` config elements weren’t refreshed after a request to ZooKeeper timed out. [\#2947](https://github.com/ClickHouse/ClickHouse/issues/2947) [\#3947](https://github.com/ClickHouse/ClickHouse/pull/3947) ([Alex Zatelepin](https://github.com/ztlpn))
+- Fix bug with wrong prefix for IPv4 subnet masks. [\#3945](https://github.com/ClickHouse/ClickHouse/pull/3945) ([alesapin](https://github.com/alesapin))
+- Fixed crash (`std::terminate`) in rare cases when a new thread cannot be created due to exhausted resources. [\#3956](https://github.com/ClickHouse/ClickHouse/pull/3956) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Fix bug when in `remote` table function execution when wrong restrictions were used for in `getStructureOfRemoteTable`. [\#4009](https://github.com/ClickHouse/ClickHouse/pull/4009) ([alesapin](https://github.com/alesapin))
+- Fix a leak of netlink sockets. They were placed in a pool where they were never deleted and new sockets were created at the start of a new thread when all current sockets were in use. [\#4017](https://github.com/ClickHouse/ClickHouse/pull/4017) ([Alex Zatelepin](https://github.com/ztlpn))
+- Fix bug with closing `/proc/self/fd` directory earlier than all fds were read from `/proc` after forking `odbc-bridge` subprocess. [\#4120](https://github.com/ClickHouse/ClickHouse/pull/4120) ([alesapin](https://github.com/alesapin))
+- Fixed String to UInt monotonic conversion in case of usage String in primary key. [\#3870](https://github.com/ClickHouse/ClickHouse/pull/3870) ([Winter Zhang](https://github.com/zhang2014))
+- Fixed error in calculation of integer conversion function monotonicity. [\#3921](https://github.com/ClickHouse/ClickHouse/pull/3921) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Fixed segfault in `arrayEnumerateUniq`, `arrayEnumerateDense` functions in case of some invalid arguments. [\#3909](https://github.com/ClickHouse/ClickHouse/pull/3909) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Fix UB in StorageMerge. [\#3910](https://github.com/ClickHouse/ClickHouse/pull/3910) ([Amos Bird](https://github.com/amosbird))
+- Fixed segfault in functions `addDays`, `subtractDays`. [\#3913](https://github.com/ClickHouse/ClickHouse/pull/3913) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Fixed error: functions `round`, `floor`, `trunc`, `ceil` may return bogus result when executed on integer argument and large negative scale. [\#3914](https://github.com/ClickHouse/ClickHouse/pull/3914) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Fixed a bug induced by ‘kill query sync’ which leads to a core dump. [\#3916](https://github.com/ClickHouse/ClickHouse/pull/3916) ([muVulDeePecker](https://github.com/fancyqlx))
+- Fix bug with long delay after empty replication queue. [\#3928](https://github.com/ClickHouse/ClickHouse/pull/3928) [\#3932](https://github.com/ClickHouse/ClickHouse/pull/3932) ([alesapin](https://github.com/alesapin))
+- Fixed excessive memory usage in case of inserting into table with `LowCardinality` primary key. [\#3955](https://github.com/ClickHouse/ClickHouse/pull/3955) ([KochetovNicolai](https://github.com/KochetovNicolai))
+- Fixed `LowCardinality` serialization for `Native` format in case of empty arrays. [\#3907](https://github.com/ClickHouse/ClickHouse/issues/3907) [\#4011](https://github.com/ClickHouse/ClickHouse/pull/4011) ([KochetovNicolai](https://github.com/KochetovNicolai))
+- Fixed incorrect result while using distinct by single LowCardinality numeric column. [\#3895](https://github.com/ClickHouse/ClickHouse/issues/3895) [\#4012](https://github.com/ClickHouse/ClickHouse/pull/4012) ([KochetovNicolai](https://github.com/KochetovNicolai))
+- Fixed specialized aggregation with LowCardinality key (in case when `compile` setting is enabled). [\#3886](https://github.com/ClickHouse/ClickHouse/pull/3886) ([KochetovNicolai](https://github.com/KochetovNicolai))
+- Fix user and password forwarding for replicated tables queries. [\#3957](https://github.com/ClickHouse/ClickHouse/pull/3957) ([alesapin](https://github.com/alesapin)) ([小路](https://github.com/nicelulu))
+- Fixed very rare race condition that can happen when listing tables in Dictionary database while reloading dictionaries. [\#3970](https://github.com/ClickHouse/ClickHouse/pull/3970) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Fixed incorrect result when HAVING was used with ROLLUP or CUBE. [\#3756](https://github.com/ClickHouse/ClickHouse/issues/3756) [\#3837](https://github.com/ClickHouse/ClickHouse/pull/3837) ([Sam Chou](https://github.com/reflection))
+- Fixed column aliases for query with `JOIN ON` syntax and distributed tables. [\#3980](https://github.com/ClickHouse/ClickHouse/pull/3980) ([Winter Zhang](https://github.com/zhang2014))
+- Fixed error in internal implementation of `quantileTDigest` (found by Artem Vakhrushev). This error never happens in ClickHouse and was relevant only for those who use ClickHouse codebase as a library directly. [\#3935](https://github.com/ClickHouse/ClickHouse/pull/3935) ([alexey-milovidov](https://github.com/alexey-milovidov))
+
+#### Improvements {#improvements-6}
+
+- Support for `IF NOT EXISTS` in `ALTER TABLE ADD COLUMN` statements along with `IF EXISTS` in `DROP/MODIFY/CLEAR/COMMENT COLUMN`. [\#3900](https://github.com/ClickHouse/ClickHouse/pull/3900) ([Boris Granveaud](https://github.com/bgranvea))
+- Function `parseDateTimeBestEffort`: support for formats `DD.MM.YYYY`, `DD.MM.YY`, `DD-MM-YYYY`, `DD-Mon-YYYY`, `DD/Month/YYYY` and similar. [\#3922](https://github.com/ClickHouse/ClickHouse/pull/3922) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- `CapnProtoInputStream` now support jagged structures. [\#4063](https://github.com/ClickHouse/ClickHouse/pull/4063) ([Odin Hultgren Van Der Horst](https://github.com/Miniwoffer))
+- Usability improvement: added a check that server process is started from the data directory’s owner. Do not allow to start server from root if the data belongs to non-root user. [\#3785](https://github.com/ClickHouse/ClickHouse/pull/3785) ([sergey-v-galtsev](https://github.com/sergey-v-galtsev))
+- Better logic of checking required columns during analysis of queries with JOINs. [\#3930](https://github.com/ClickHouse/ClickHouse/pull/3930) ([Artem Zuikov](https://github.com/4ertus2))
+- Decreased the number of connections in case of large number of Distributed tables in a single server. [\#3726](https://github.com/ClickHouse/ClickHouse/pull/3726) ([Winter Zhang](https://github.com/zhang2014))
+- Supported totals row for `WITH TOTALS` query for ODBC driver. [\#3836](https://github.com/ClickHouse/ClickHouse/pull/3836) ([Maksim Koritckiy](https://github.com/nightweb))
+- Allowed to use `Enum`s as integers inside if function. [\#3875](https://github.com/ClickHouse/ClickHouse/pull/3875) ([Ivan](https://github.com/abyss7))
+- Added `low_cardinality_allow_in_native_format` setting. If disabled, do not use `LowCadrinality` type in `Native` format. [\#3879](https://github.com/ClickHouse/ClickHouse/pull/3879) ([KochetovNicolai](https://github.com/KochetovNicolai))
+- Removed some redundant objects from compiled expressions cache to lower memory usage. [\#4042](https://github.com/ClickHouse/ClickHouse/pull/4042) ([alesapin](https://github.com/alesapin))
+- Add check that `SET send_logs_level = 'value'` query accept appropriate value. [\#3873](https://github.com/ClickHouse/ClickHouse/pull/3873) ([Sabyanin Maxim](https://github.com/s-mx))
+- Fixed data type check in type conversion functions. [\#3896](https://github.com/ClickHouse/ClickHouse/pull/3896) ([Winter Zhang](https://github.com/zhang2014))
+
+#### Performance Improvements {#performance-improvements-5}
+
+- Add a MergeTree setting `use_minimalistic_part_header_in_zookeeper`. If enabled, Replicated tables will store compact part metadata in a single part znode. This can dramatically reduce ZooKeeper snapshot size (especially if the tables have a lot of columns). Note that after enabling this setting you will not be able to downgrade to a version that doesn’t support it. [\#3960](https://github.com/ClickHouse/ClickHouse/pull/3960) ([Alex Zatelepin](https://github.com/ztlpn))
+- Add an DFA-based implementation for functions `sequenceMatch` and `sequenceCount` in case pattern doesn’t contain time. [\#4004](https://github.com/ClickHouse/ClickHouse/pull/4004) ([Léo Ercolanelli](https://github.com/ercolanelli-leo))
+- Performance improvement for integer numbers serialization. [\#3968](https://github.com/ClickHouse/ClickHouse/pull/3968) ([Amos Bird](https://github.com/amosbird))
+- Zero left padding PODArray so that -1 element is always valid and zeroed. It’s used for branchless calculation of offsets. [\#3920](https://github.com/ClickHouse/ClickHouse/pull/3920) ([Amos Bird](https://github.com/amosbird))
+- Reverted `jemalloc` version which lead to performance degradation. [\#4018](https://github.com/ClickHouse/ClickHouse/pull/4018) ([alexey-milovidov](https://github.com/alexey-milovidov))
+
+#### Backward Incompatible Changes {#backward-incompatible-changes-2}
+
+- Removed undocumented feature `ALTER MODIFY PRIMARY KEY` because it was superseded by the `ALTER MODIFY ORDER BY` command. [\#3887](https://github.com/ClickHouse/ClickHouse/pull/3887) ([Alex Zatelepin](https://github.com/ztlpn))
+- Removed function `shardByHash`. [\#3833](https://github.com/ClickHouse/ClickHouse/pull/3833) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Forbid using scalar subqueries with result of type `AggregateFunction`. [\#3865](https://github.com/ClickHouse/ClickHouse/pull/3865) ([Ivan](https://github.com/abyss7))
+
+#### Build/Testing/Packaging Improvements {#buildtestingpackaging-improvements-6}
+
+- Added support for PowerPC (`ppc64le`) build. [\#4132](https://github.com/ClickHouse/ClickHouse/pull/4132) ([Danila Kutenin](https://github.com/danlark1))
+- Stateful functional tests are run on public available dataset. [\#3969](https://github.com/ClickHouse/ClickHouse/pull/3969) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Fixed error when the server cannot start with the `bash: /usr/bin/clickhouse-extract-from-config: Operation not permitted` message within Docker or systemd-nspawn. [\#4136](https://github.com/ClickHouse/ClickHouse/pull/4136) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Updated `rdkafka` library to v1.0.0-RC5. Used cppkafka instead of raw C interface. [\#4025](https://github.com/ClickHouse/ClickHouse/pull/4025) ([Ivan](https://github.com/abyss7))
+- Updated `mariadb-client` library. Fixed one of issues found by UBSan. [\#3924](https://github.com/ClickHouse/ClickHouse/pull/3924) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Some fixes for UBSan builds. [\#3926](https://github.com/ClickHouse/ClickHouse/pull/3926) [\#3021](https://github.com/ClickHouse/ClickHouse/pull/3021) [\#3948](https://github.com/ClickHouse/ClickHouse/pull/3948) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Added per-commit runs of tests with UBSan build.
+- Added per-commit runs of PVS-Studio static analyzer.
+- Fixed bugs found by PVS-Studio. [\#4013](https://github.com/ClickHouse/ClickHouse/pull/4013) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Fixed glibc compatibility issues. [\#4100](https://github.com/ClickHouse/ClickHouse/pull/4100) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Move Docker images to 18.10 and add compatibility file for glibc \>= 2.28 [\#3965](https://github.com/ClickHouse/ClickHouse/pull/3965) ([alesapin](https://github.com/alesapin))
+- Add env variable if user don’t want to chown directories in server Docker image. [\#3967](https://github.com/ClickHouse/ClickHouse/pull/3967) ([alesapin](https://github.com/alesapin))
+- Enabled most of the warnings from `-Weverything` in clang. Enabled `-Wpedantic`. [\#3986](https://github.com/ClickHouse/ClickHouse/pull/3986) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Added a few more warnings that are available only in clang 8. [\#3993](https://github.com/ClickHouse/ClickHouse/pull/3993) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Link to `libLLVM` rather than to individual LLVM libs when using shared linking. [\#3989](https://github.com/ClickHouse/ClickHouse/pull/3989) ([Orivej Desh](https://github.com/orivej))
+- Added sanitizer variables for test images. [\#4072](https://github.com/ClickHouse/ClickHouse/pull/4072) ([alesapin](https://github.com/alesapin))
+- `clickhouse-server` debian package will recommend `libcap2-bin` package to use `setcap` tool for setting capabilities. This is optional. [\#4093](https://github.com/ClickHouse/ClickHouse/pull/4093) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Improved compilation time, fixed includes. [\#3898](https://github.com/ClickHouse/ClickHouse/pull/3898) ([proller](https://github.com/proller))
+- Added performance tests for hash functions. [\#3918](https://github.com/ClickHouse/ClickHouse/pull/3918) ([filimonov](https://github.com/filimonov))
+- Fixed cyclic library dependences. [\#3958](https://github.com/ClickHouse/ClickHouse/pull/3958) ([proller](https://github.com/proller))
+- Improved compilation with low available memory. [\#4030](https://github.com/ClickHouse/ClickHouse/pull/4030) ([proller](https://github.com/proller))
+- Added test script to reproduce performance degradation in `jemalloc`. [\#4036](https://github.com/ClickHouse/ClickHouse/pull/4036) ([alexey-milovidov](https://github.com/alexey-milovidov))
+- Fixed misspells in comments and string literals under `dbms`. [\#4122](https://github.com/ClickHouse/ClickHouse/pull/4122) ([maiha](https://github.com/maiha))
+- Fixed typos in comments. [\#4089](https://github.com/ClickHouse/ClickHouse/pull/4089) ([Evgenii Pravda](https://github.com/kvinty))
+
+## [Changelog for 2018](https://github.com/ClickHouse/ClickHouse/blob/master/docs/en/changelog/2018.md)
diff --git a/docs/fa/changelog/index.md b/docs/fa/changelog/index.md
deleted file mode 120000
index 79b747aee1b..00000000000
--- a/docs/fa/changelog/index.md
+++ /dev/null
@@ -1 +0,0 @@
-../../../CHANGELOG.md
\ No newline at end of file
diff --git a/docs/fa/changelog/index.md b/docs/fa/changelog/index.md
new file mode 100644
index 00000000000..1a89e03c333
--- /dev/null
+++ b/docs/fa/changelog/index.md
@@ -0,0 +1,5 @@
+---
+en_copy: true
+---
+
+../../../CHANGELOG.md
\ No newline at end of file
diff --git a/docs/fa/commercial/cloud.md b/docs/fa/commercial/cloud.md
deleted file mode 120000
index eb58e4a90be..00000000000
--- a/docs/fa/commercial/cloud.md
+++ /dev/null
@@ -1 +0,0 @@
-../../en/commercial/cloud.md
\ No newline at end of file
diff --git a/docs/fa/commercial/cloud.md b/docs/fa/commercial/cloud.md
new file mode 100644
index 00000000000..f096bdb92cf
--- /dev/null
+++ b/docs/fa/commercial/cloud.md
@@ -0,0 +1,20 @@
+---
+en_copy: true
+---
+
+# ClickHouse Cloud Service Providers {#clickhouse-cloud-service-providers}
+
+!!! info "Info"
+ If you have launched a public cloud with managed ClickHouse service, feel free to [open a pull-request](https://github.com/ClickHouse/ClickHouse/edit/master/docs/en/commercial/cloud.md) adding it to the following list.
+
+## Yandex Cloud {#yandex-cloud}
+
+[Yandex Managed Service for ClickHouse](https://cloud.yandex.com/services/managed-clickhouse?utm_source=referrals&utm_medium=clickhouseofficialsite&utm_campaign=link3) provides the following key features:
+
+- Fully managed ZooKeeper service for [ClickHouse replication](../operations/table_engines/replication.md)
+- Multiple storage type choices
+- Replicas in different availability zones
+- Encryption and isolation
+- Automated maintenance
+
+{## [Original article](https://clickhouse.tech/docs/en/commercial/cloud/) ##}
diff --git a/docs/fa/data_types/datetime64.md b/docs/fa/data_types/datetime64.md
deleted file mode 120000
index e59b41ea1ae..00000000000
--- a/docs/fa/data_types/datetime64.md
+++ /dev/null
@@ -1 +0,0 @@
-../../en/data_types/datetime64.md
\ No newline at end of file
diff --git a/docs/fa/data_types/datetime64.md b/docs/fa/data_types/datetime64.md
new file mode 100644
index 00000000000..e28390bbdd4
--- /dev/null
+++ b/docs/fa/data_types/datetime64.md
@@ -0,0 +1,101 @@
+---
+en_copy: true
+---
+
+# DateTime64 {#data_type-datetime64}
+
+Allows to store an instant in time, that can be expressed as a calendar date and a time of a day, with defined sub-second precision
+
+Tick size (precision): 10-precision seconds
+
+Syntax:
+
+``` sql
+DateTime64(precision, [timezone])
+```
+
+Internally, stores data as a number of ‘ticks’ since epoch start (1970-01-01 00:00:00 UTC) as Int64. The tick resolution is determined by the precision parameter. Additionally, the `DateTime64` type can store time zone that is the same for the entire column, that affects how the values of the `DateTime64` type values are displayed in text format and how the values specified as strings are parsed (‘2020-01-01 05:00:01.000’). The time zone is not stored in the rows of the table (or in resultset), but is stored in the column metadata. See details in [DateTime](datetime.md).
+
+## Examples {#examples}
+
+**1.** Creating a table with `DateTime64`-type column and inserting data into it:
+
+``` sql
+CREATE TABLE dt
+(
+ `timestamp` DateTime64(3, 'Europe/Moscow'),
+ `event_id` UInt8
+)
+ENGINE = TinyLog
+```
+
+``` sql
+INSERT INTO dt Values (1546300800000, 1), ('2019-01-01 00:00:00', 2)
+```
+
+``` sql
+SELECT * FROM dt
+```
+
+``` text
+┌───────────────timestamp─┬─event_id─┐
+│ 2019-01-01 03:00:00.000 │ 1 │
+│ 2019-01-01 00:00:00.000 │ 2 │
+└─────────────────────────┴──────────┘
+```
+
+- When inserting datetime as an integer, it is treated as an appropriately scaled Unix Timestamp (UTC). `1546300800000` (with precision 3) represents `'2019-01-01 00:00:00'` UTC. However, as `timestamp` column has `Europe/Moscow` (UTC+3) timezone specified, when outputting as a string the value will be shown as `'2019-01-01 03:00:00'`
+- When inserting string value as datetime, it is treated as being in column timezone. `'2019-01-01 00:00:00'` will be treated as being in `Europe/Moscow` timezone and stored as `1546290000000`.
+
+**2.** Filtering on `DateTime64` values
+
+``` sql
+SELECT * FROM dt WHERE timestamp = toDateTime64('2019-01-01 00:00:00', 3, 'Europe/Moscow')
+```
+
+``` text
+┌───────────────timestamp─┬─event_id─┐
+│ 2019-01-01 00:00:00.000 │ 2 │
+└─────────────────────────┴──────────┘
+```
+
+Unlike `DateTime`, `DateTime64` values are not converted from `String` automatically
+
+**3.** Getting a time zone for a `DateTime64`-type value:
+
+``` sql
+SELECT toDateTime64(now(), 3, 'Europe/Moscow') AS column, toTypeName(column) AS x
+```
+
+``` text
+┌──────────────────column─┬─x──────────────────────────────┐
+│ 2019-10-16 04:12:04.000 │ DateTime64(3, 'Europe/Moscow') │
+└─────────────────────────┴────────────────────────────────┘
+```
+
+**4.** Timezone conversion
+
+``` sql
+SELECT
+toDateTime64(timestamp, 3, 'Europe/London') as lon_time,
+toDateTime64(timestamp, 3, 'Europe/Moscow') as mos_time
+FROM dt
+```
+
+``` text
+┌───────────────lon_time──┬────────────────mos_time─┐
+│ 2019-01-01 00:00:00.000 │ 2019-01-01 03:00:00.000 │
+│ 2018-12-31 21:00:00.000 │ 2019-01-01 00:00:00.000 │
+└─────────────────────────┴─────────────────────────┘
+```
+
+## See Also {#see-also}
+
+- [Type conversion functions](../query_language/functions/type_conversion_functions.md)
+- [Functions for working with dates and times](../query_language/functions/date_time_functions.md)
+- [Functions for working with arrays](../query_language/functions/array_functions.md)
+- [The `date_time_input_format` setting](../operations/settings/settings.md#settings-date_time_input_format)
+- [The `timezone` server configuration parameter](../operations/server_settings/settings.md#server_settings-timezone)
+- [Operators for working with dates and times](../query_language/operators.md#operators-datetime)
+- [`Date` data type](date.md)
+- [`DateTime` data type](datetime.md)
diff --git a/docs/fa/data_types/decimal.md b/docs/fa/data_types/decimal.md
deleted file mode 120000
index ccea440adfa..00000000000
--- a/docs/fa/data_types/decimal.md
+++ /dev/null
@@ -1 +0,0 @@
-../../en/data_types/decimal.md
\ No newline at end of file
diff --git a/docs/fa/data_types/decimal.md b/docs/fa/data_types/decimal.md
new file mode 100644
index 00000000000..d960675a310
--- /dev/null
+++ b/docs/fa/data_types/decimal.md
@@ -0,0 +1,106 @@
+---
+en_copy: true
+---
+
+# Decimal(P, S), Decimal32(S), Decimal64(S), Decimal128(S) {#decimalp-s-decimal32s-decimal64s-decimal128s}
+
+Signed fixed-point numbers that keep precision during add, subtract and multiply operations. For division least significant digits are discarded (not rounded).
+
+## Parameters {#parameters}
+
+- P - precision. Valid range: \[ 1 : 38 \]. Determines how many decimal digits number can have (including fraction).
+- S - scale. Valid range: \[ 0 : P \]. Determines how many decimal digits fraction can have.
+
+Depending on P parameter value Decimal(P, S) is a synonym for:
+- P from \[ 1 : 9 \] - for Decimal32(S)
+- P from \[ 10 : 18 \] - for Decimal64(S)
+- P from \[ 19 : 38 \] - for Decimal128(S)
+
+## Decimal value ranges {#decimal-value-ranges}
+
+- Decimal32(S) - ( -1 \* 10^(9 - S), 1 \* 10^(9 - S) )
+- Decimal64(S) - ( -1 \* 10^(18 - S), 1 \* 10^(18 - S) )
+- Decimal128(S) - ( -1 \* 10^(38 - S), 1 \* 10^(38 - S) )
+
+For example, Decimal32(4) can contain numbers from -99999.9999 to 99999.9999 with 0.0001 step.
+
+## Internal representation {#internal-representation}
+
+Internally data is represented as normal signed integers with respective bit width. Real value ranges that can be stored in memory are a bit larger than specified above, which are checked only on conversion from a string.
+
+Because modern CPU’s do not support 128-bit integers natively, operations on Decimal128 are emulated. Because of this Decimal128 works significantly slower than Decimal32/Decimal64.
+
+## Operations and result type {#operations-and-result-type}
+
+Binary operations on Decimal result in wider result type (with any order of arguments).
+
+- Decimal64(S1) Decimal32(S2) -\> Decimal64(S)
+- Decimal128(S1) Decimal32(S2) -\> Decimal128(S)
+- Decimal128(S1) Decimal64(S2) -\> Decimal128(S)
+
+Rules for scale:
+
+- add, subtract: S = max(S1, S2).
+- multuply: S = S1 + S2.
+- divide: S = S1.
+
+For similar operations between Decimal and integers, the result is Decimal of the same size as an argument.
+
+Operations between Decimal and Float32/Float64 are not defined. If you need them, you can explicitly cast one of argument using toDecimal32, toDecimal64, toDecimal128 or toFloat32, toFloat64 builtins. Keep in mind that the result will lose precision and type conversion is a computationally expensive operation.
+
+Some functions on Decimal return result as Float64 (for example, var or stddev). Intermediate calculations might still be performed in Decimal, which might lead to different results between Float64 and Decimal inputs with the same values.
+
+## Overflow checks {#overflow-checks}
+
+During calculations on Decimal, integer overflows might happen. Excessive digits in a fraction are discarded (not rounded). Excessive digits in integer part will lead to an exception.
+
+``` sql
+SELECT toDecimal32(2, 4) AS x, x / 3
+```
+
+``` text
+┌──────x─┬─divide(toDecimal32(2, 4), 3)─┐
+│ 2.0000 │ 0.6666 │
+└────────┴──────────────────────────────┘
+```
+
+``` sql
+SELECT toDecimal32(4.2, 8) AS x, x * x
+```
+
+``` text
+DB::Exception: Scale is out of bounds.
+```
+
+``` sql
+SELECT toDecimal32(4.2, 8) AS x, 6 * x
+```
+
+``` text
+DB::Exception: Decimal math overflow.
+```
+
+Overflow checks lead to operations slowdown. If it is known that overflows are not possible, it makes sense to disable checks using `decimal_check_overflow` setting. When checks are disabled and overflow happens, the result will be incorrect:
+
+``` sql
+SET decimal_check_overflow = 0;
+SELECT toDecimal32(4.2, 8) AS x, 6 * x
+```
+
+``` text
+┌──────────x─┬─multiply(6, toDecimal32(4.2, 8))─┐
+│ 4.20000000 │ -17.74967296 │
+└────────────┴──────────────────────────────────┘
+```
+
+Overflow checks happen not only on arithmetic operations but also on value comparison:
+
+``` sql
+SELECT toDecimal32(1, 8) < 100
+```
+
+``` text
+DB::Exception: Can't compare.
+```
+
+[Original article](https://clickhouse.tech/docs/en/data_types/decimal/)
diff --git a/docs/fa/data_types/domains/ipv4.md b/docs/fa/data_types/domains/ipv4.md
deleted file mode 120000
index eb4cc7d57b5..00000000000
--- a/docs/fa/data_types/domains/ipv4.md
+++ /dev/null
@@ -1 +0,0 @@
-../../../en/data_types/domains/ipv4.md
\ No newline at end of file
diff --git a/docs/fa/data_types/domains/ipv4.md b/docs/fa/data_types/domains/ipv4.md
new file mode 100644
index 00000000000..55be7ab4439
--- /dev/null
+++ b/docs/fa/data_types/domains/ipv4.md
@@ -0,0 +1,81 @@
+---
+en_copy: true
+---
+
+## IPv4 {#ipv4}
+
+`IPv4` is a domain based on `UInt32` type and serves as a typed replacement for storing IPv4 values. It provides compact storage with the human-friendly input-output format and column type information on inspection.
+
+### Basic Usage {#basic-usage}
+
+``` sql
+CREATE TABLE hits (url String, from IPv4) ENGINE = MergeTree() ORDER BY url;
+
+DESCRIBE TABLE hits;
+```
+
+``` text
+┌─name─┬─type───┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┐
+│ url │ String │ │ │ │ │
+│ from │ IPv4 │ │ │ │ │
+└──────┴────────┴──────────────┴────────────────────┴─────────┴──────────────────┘
+```
+
+OR you can use IPv4 domain as a key:
+
+``` sql
+CREATE TABLE hits (url String, from IPv4) ENGINE = MergeTree() ORDER BY from;
+```
+
+`IPv4` domain supports custom input format as IPv4-strings:
+
+``` sql
+INSERT INTO hits (url, from) VALUES ('https://wikipedia.org', '116.253.40.133')('https://clickhouse.tech', '183.247.232.58')('https://clickhouse.yandex/docs/en/', '116.106.34.242');
+
+SELECT * FROM hits;
+```
+
+``` text
+┌─url────────────────────────────────┬───────────from─┐
+│ https://clickhouse.tech/docs/en/ │ 116.106.34.242 │
+│ https://wikipedia.org │ 116.253.40.133 │
+│ https://clickhouse.tech │ 183.247.232.58 │
+└────────────────────────────────────┴────────────────┘
+```
+
+Values are stored in compact binary form:
+
+``` sql
+SELECT toTypeName(from), hex(from) FROM hits LIMIT 1;
+```
+
+``` text
+┌─toTypeName(from)─┬─hex(from)─┐
+│ IPv4 │ B7F7E83A │
+└──────────────────┴───────────┘
+```
+
+Domain values are not implicitly convertible to types other than `UInt32`.
+If you want to convert `IPv4` value to a string, you have to do that explicitly with `IPv4NumToString()` function:
+
+``` sql
+SELECT toTypeName(s), IPv4NumToString(from) as s FROM hits LIMIT 1;
+```
+
+ ┌─toTypeName(IPv4NumToString(from))─┬─s──────────────┐
+ │ String │ 183.247.232.58 │
+ └───────────────────────────────────┴────────────────┘
+
+Or cast to a `UInt32` value:
+
+``` sql
+SELECT toTypeName(i), CAST(from as UInt32) as i FROM hits LIMIT 1;
+```
+
+``` text
+┌─toTypeName(CAST(from, 'UInt32'))─┬──────────i─┐
+│ UInt32 │ 3086477370 │
+└──────────────────────────────────┴────────────┘
+```
+
+[Original article](https://clickhouse.tech/docs/en/data_types/domains/ipv4)
diff --git a/docs/fa/data_types/domains/ipv6.md b/docs/fa/data_types/domains/ipv6.md
deleted file mode 120000
index cca37a22458..00000000000
--- a/docs/fa/data_types/domains/ipv6.md
+++ /dev/null
@@ -1 +0,0 @@
-../../../en/data_types/domains/ipv6.md
\ No newline at end of file
diff --git a/docs/fa/data_types/domains/ipv6.md b/docs/fa/data_types/domains/ipv6.md
new file mode 100644
index 00000000000..4b856230ccc
--- /dev/null
+++ b/docs/fa/data_types/domains/ipv6.md
@@ -0,0 +1,83 @@
+---
+en_copy: true
+---
+
+## IPv6 {#ipv6}
+
+`IPv6` is a domain based on `FixedString(16)` type and serves as a typed replacement for storing IPv6 values. It provides compact storage with the human-friendly input-output format and column type information on inspection.
+
+### Basic Usage {#basic-usage}
+
+``` sql
+CREATE TABLE hits (url String, from IPv6) ENGINE = MergeTree() ORDER BY url;
+
+DESCRIBE TABLE hits;
+```
+
+``` text
+┌─name─┬─type───┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┐
+│ url │ String │ │ │ │ │
+│ from │ IPv6 │ │ │ │ │
+└──────┴────────┴──────────────┴────────────────────┴─────────┴──────────────────┘
+```
+
+OR you can use `IPv6` domain as a key:
+
+``` sql
+CREATE TABLE hits (url String, from IPv6) ENGINE = MergeTree() ORDER BY from;
+```
+
+`IPv6` domain supports custom input as IPv6-strings:
+
+``` sql
+INSERT INTO hits (url, from) VALUES ('https://wikipedia.org', '2a02:aa08:e000:3100::2')('https://clickhouse.tech', '2001:44c8:129:2632:33:0:252:2')('https://clickhouse.yandex/docs/en/', '2a02:e980:1e::1');
+
+SELECT * FROM hits;
+```
+
+``` text
+┌─url────────────────────────────────┬─from──────────────────────────┐
+│ https://clickhouse.tech │ 2001:44c8:129:2632:33:0:252:2 │
+│ https://clickhouse.tech/docs/en/ │ 2a02:e980:1e::1 │
+│ https://wikipedia.org │ 2a02:aa08:e000:3100::2 │
+└────────────────────────────────────┴───────────────────────────────┘
+```
+
+Values are stored in compact binary form:
+
+``` sql
+SELECT toTypeName(from), hex(from) FROM hits LIMIT 1;
+```
+
+``` text
+┌─toTypeName(from)─┬─hex(from)────────────────────────┐
+│ IPv6 │ 200144C8012926320033000002520002 │
+└──────────────────┴──────────────────────────────────┘
+```
+
+Domain values are not implicitly convertible to types other than `FixedString(16)`.
+If you want to convert `IPv6` value to a string, you have to do that explicitly with `IPv6NumToString()` function:
+
+``` sql
+SELECT toTypeName(s), IPv6NumToString(from) as s FROM hits LIMIT 1;
+```
+
+``` text
+┌─toTypeName(IPv6NumToString(from))─┬─s─────────────────────────────┐
+│ String │ 2001:44c8:129:2632:33:0:252:2 │
+└───────────────────────────────────┴───────────────────────────────┘
+```
+
+Or cast to a `FixedString(16)` value:
+
+``` sql
+SELECT toTypeName(i), CAST(from as FixedString(16)) as i FROM hits LIMIT 1;
+```
+
+``` text
+┌─toTypeName(CAST(from, 'FixedString(16)'))─┬─i───────┐
+│ FixedString(16) │ ��� │
+└───────────────────────────────────────────┴─────────┘
+```
+
+[Original article](https://clickhouse.tech/docs/en/data_types/domains/ipv6)
diff --git a/docs/fa/data_types/domains/overview.md b/docs/fa/data_types/domains/overview.md
deleted file mode 120000
index 13465d655ee..00000000000
--- a/docs/fa/data_types/domains/overview.md
+++ /dev/null
@@ -1 +0,0 @@
-../../../en/data_types/domains/overview.md
\ No newline at end of file
diff --git a/docs/fa/data_types/domains/overview.md b/docs/fa/data_types/domains/overview.md
new file mode 100644
index 00000000000..a131f8f6f3e
--- /dev/null
+++ b/docs/fa/data_types/domains/overview.md
@@ -0,0 +1,29 @@
+---
+en_copy: true
+---
+
+# Domains {#domains}
+
+Domains are special-purpose types that add some extra features atop of existing base type, but leaving on-wire and on-disc format of the underlying data type intact. At the moment, ClickHouse does not support user-defined domains.
+
+You can use domains anywhere corresponding base type can be used, for example:
+
+- Create a column of a domain type
+- Read/write values from/to domain column
+- Use it as an index if a base type can be used as an index
+- Call functions with values of domain column
+
+### Extra Features of Domains {#extra-features-of-domains}
+
+- Explicit column type name in `SHOW CREATE TABLE` or `DESCRIBE TABLE`
+- Input from human-friendly format with `INSERT INTO domain_table(domain_column) VALUES(...)`
+- Output to human-friendly format for `SELECT domain_column FROM domain_table`
+- Loading data from an external source in the human-friendly format: `INSERT INTO domain_table FORMAT CSV ...`
+
+### Limitations {#limitations}
+
+- Can’t convert index column of base type to domain type via `ALTER TABLE`.
+- Can’t implicitly convert string values into domain values when inserting data from another column or table.
+- Domain adds no constrains on stored values.
+
+[Original article](https://clickhouse.tech/docs/en/data_types/domains/overview)
diff --git a/docs/fa/data_types/nullable.md b/docs/fa/data_types/nullable.md
deleted file mode 120000
index 0233f91d954..00000000000
--- a/docs/fa/data_types/nullable.md
+++ /dev/null
@@ -1 +0,0 @@
-../../en/data_types/nullable.md
\ No newline at end of file
diff --git a/docs/fa/data_types/nullable.md b/docs/fa/data_types/nullable.md
new file mode 100644
index 00000000000..60c4cfab81e
--- /dev/null
+++ b/docs/fa/data_types/nullable.md
@@ -0,0 +1,43 @@
+---
+en_copy: true
+---
+
+# Nullable(TypeName) {#data_type-nullable}
+
+Allows to store special marker ([NULL](../query_language/syntax.md)) that denotes “missing value” alongside normal values allowed by `TypeName`. For example, a `Nullable(Int8)` type column can store `Int8` type values, and the rows that don’t have a value will store `NULL`.
+
+For a `TypeName`, you can’t use composite data types [Array](array.md) and [Tuple](tuple.md). Composite data types can contain `Nullable` type values, such as `Array(Nullable(Int8))`.
+
+A `Nullable` type field can’t be included in table indexes.
+
+`NULL` is the default value for any `Nullable` type, unless specified otherwise in the ClickHouse server configuration.
+
+## Storage features {#storage-features}
+
+To store `Nullable` type values in a table column, ClickHouse uses a separate file with `NULL` masks in addition to normal file with values. Entries in masks file allow ClickHouse to distinguish between `NULL` and a default value of corresponding data type for each table row. Because of an additional file, `Nullable` column consumes additional storage space compared to a similar normal one.
+
+!!! info "Note"
+ Using `Nullable` almost always negatively affects performance, keep this in mind when designing your databases.
+
+## Usage example {#usage-example}
+
+``` sql
+CREATE TABLE t_null(x Int8, y Nullable(Int8)) ENGINE TinyLog
+```
+
+``` sql
+INSERT INTO t_null VALUES (1, NULL), (2, 3)
+```
+
+``` sql
+SELECT x + y FROM t_null
+```
+
+``` text
+┌─plus(x, y)─┐
+│ ᴺᵁᴸᴸ │
+│ 5 │
+└────────────┘
+```
+
+[Original article](https://clickhouse.tech/docs/en/data_types/nullable/)
diff --git a/docs/fa/data_types/special_data_types/interval.md b/docs/fa/data_types/special_data_types/interval.md
deleted file mode 120000
index 6829f5ced00..00000000000
--- a/docs/fa/data_types/special_data_types/interval.md
+++ /dev/null
@@ -1 +0,0 @@
-../../../en/data_types/special_data_types/interval.md
\ No newline at end of file
diff --git a/docs/fa/data_types/special_data_types/interval.md b/docs/fa/data_types/special_data_types/interval.md
new file mode 100644
index 00000000000..8a37476579c
--- /dev/null
+++ b/docs/fa/data_types/special_data_types/interval.md
@@ -0,0 +1,82 @@
+---
+en_copy: true
+---
+
+# Interval {#data-type-interval}
+
+The family of data types representing time and date intervals. The resulting types of the [INTERVAL](../../query_language/operators.md#operator-interval) operator.
+
+!!! warning "Warning"
+ `Interval` data type values can’t be stored in tables.
+
+Structure:
+
+- Time interval as an unsigned integer value.
+- Type of an interval.
+
+Supported interval types:
+
+- `SECOND`
+- `MINUTE`
+- `HOUR`
+- `DAY`
+- `WEEK`
+- `MONTH`
+- `QUARTER`
+- `YEAR`
+
+For each interval type, there is a separate data type. For example, the `DAY` interval corresponds to the `IntervalDay` data type:
+
+``` sql
+SELECT toTypeName(INTERVAL 4 DAY)
+```
+
+``` text
+┌─toTypeName(toIntervalDay(4))─┐
+│ IntervalDay │
+└──────────────────────────────┘
+```
+
+## Usage Remarks {#data-type-interval-usage-remarks}
+
+You can use `Interval`-type values in arithmetical operations with [Date](../../data_types/date.md) and [DateTime](../../data_types/datetime.md)-type values. For example, you can add 4 days to the current time:
+
+``` sql
+SELECT now() as current_date_time, current_date_time + INTERVAL 4 DAY
+```
+
+``` text
+┌───current_date_time─┬─plus(now(), toIntervalDay(4))─┐
+│ 2019-10-23 10:58:45 │ 2019-10-27 10:58:45 │
+└─────────────────────┴───────────────────────────────┘
+```
+
+Intervals with different types can’t be combined. You can’t use intervals like `4 DAY 1 HOUR`. Specify intervals in units that are smaller or equal to the smallest unit of the interval, for example, the interval `1 day and an hour` interval can be expressed as `25 HOUR` or `90000 SECOND`.
+
+You can’t perform arithmetical operations with `Interval`-type values, but you can add intervals of different types consequently to values in `Date` or `DateTime` data types. For example:
+
+``` sql
+SELECT now() AS current_date_time, current_date_time + INTERVAL 4 DAY + INTERVAL 3 HOUR
+```
+
+``` text
+┌───current_date_time─┬─plus(plus(now(), toIntervalDay(4)), toIntervalHour(3))─┐
+│ 2019-10-23 11:16:28 │ 2019-10-27 14:16:28 │
+└─────────────────────┴────────────────────────────────────────────────────────┘
+```
+
+The following query causes an exception:
+
+``` sql
+select now() AS current_date_time, current_date_time + (INTERVAL 4 DAY + INTERVAL 3 HOUR)
+```
+
+``` text
+Received exception from server (version 19.14.1):
+Code: 43. DB::Exception: Received from localhost:9000. DB::Exception: Wrong argument types for function plus: if one argument is Interval, then another must be Date or DateTime..
+```
+
+## See Also {#see-also}
+
+- [INTERVAL](../../query_language/operators.md#operator-interval) operator
+- [toInterval](../../query_language/functions/type_conversion_functions.md#function-tointerval) type convertion functions
diff --git a/docs/fa/data_types/special_data_types/nothing.md b/docs/fa/data_types/special_data_types/nothing.md
deleted file mode 120000
index 197a752ce9c..00000000000
--- a/docs/fa/data_types/special_data_types/nothing.md
+++ /dev/null
@@ -1 +0,0 @@
-../../../en/data_types/special_data_types/nothing.md
\ No newline at end of file
diff --git a/docs/fa/data_types/special_data_types/nothing.md b/docs/fa/data_types/special_data_types/nothing.md
new file mode 100644
index 00000000000..a7a7e5b57d5
--- /dev/null
+++ b/docs/fa/data_types/special_data_types/nothing.md
@@ -0,0 +1,23 @@
+---
+en_copy: true
+---
+
+# Nothing {#nothing}
+
+The only purpose of this data type is to represent cases where a value is not expected. So you can’t create a `Nothing` type value.
+
+For example, literal [NULL](../../query_language/syntax.md#null-literal) has type of `Nullable(Nothing)`. See more about [Nullable](../../data_types/nullable.md).
+
+The `Nothing` type can also used to denote empty arrays:
+
+``` sql
+SELECT toTypeName(array())
+```
+
+``` text
+┌─toTypeName(array())─┐
+│ Array(Nothing) │
+└─────────────────────┘
+```
+
+[Original article](https://clickhouse.tech/docs/en/data_types/special_data_types/nothing/)
diff --git a/docs/fa/data_types/uuid.md b/docs/fa/data_types/uuid.md
deleted file mode 120000
index aba05e889ac..00000000000
--- a/docs/fa/data_types/uuid.md
+++ /dev/null
@@ -1 +0,0 @@
-../../en/data_types/uuid.md
\ No newline at end of file
diff --git a/docs/fa/data_types/uuid.md b/docs/fa/data_types/uuid.md
new file mode 100644
index 00000000000..4546be19371
--- /dev/null
+++ b/docs/fa/data_types/uuid.md
@@ -0,0 +1,74 @@
+---
+en_copy: true
+---
+
+# UUID {#uuid-data-type}
+
+A universally unique identifier (UUID) is a 16-byte number used to identify records. For detailed information about the UUID, see [Wikipedia](https://en.wikipedia.org/wiki/Universally_unique_identifier).
+
+The example of UUID type value is represented below:
+
+``` text
+61f0c404-5cb3-11e7-907b-a6006ad3dba0
+```
+
+If you do not specify the UUID column value when inserting a new record, the UUID value is filled with zero:
+
+``` text
+00000000-0000-0000-0000-000000000000
+```
+
+## How to generate {#how-to-generate}
+
+To generate the UUID value, ClickHouse provides the [generateUUIDv4](../query_language/functions/uuid_functions.md) function.
+
+## Usage example {#usage-example}
+
+**Example 1**
+
+This example demonstrates creating a table with the UUID type column and inserting a value into the table.
+
+``` sql
+CREATE TABLE t_uuid (x UUID, y String) ENGINE=TinyLog
+```
+
+``` sql
+INSERT INTO t_uuid SELECT generateUUIDv4(), 'Example 1'
+```
+
+``` sql
+SELECT * FROM t_uuid
+```
+
+``` text
+┌────────────────────────────────────x─┬─y─────────┐
+│ 417ddc5d-e556-4d27-95dd-a34d84e46a50 │ Example 1 │
+└──────────────────────────────────────┴───────────┘
+```
+
+**Example 2**
+
+In this example, the UUID column value is not specified when inserting a new record.
+
+``` sql
+INSERT INTO t_uuid (y) VALUES ('Example 2')
+```
+
+``` sql
+SELECT * FROM t_uuid
+```
+
+``` text
+┌────────────────────────────────────x─┬─y─────────┐
+│ 417ddc5d-e556-4d27-95dd-a34d84e46a50 │ Example 1 │
+│ 00000000-0000-0000-0000-000000000000 │ Example 2 │
+└──────────────────────────────────────┴───────────┘
+```
+
+## Restrictions {#restrictions}
+
+The UUID data type only supports functions which [String](string.md) data type also supports (for example, [min](../query_language/agg_functions/reference.md#agg_function-min), [max](../query_language/agg_functions/reference.md#agg_function-max), and [count](../query_language/agg_functions/reference.md#agg_function-count)).
+
+The UUID data type is not supported by arithmetic operations (for example, [abs](../query_language/functions/arithmetic_functions.md#arithm_func-abs)) or aggregate functions, such as [sum](../query_language/agg_functions/reference.md#agg_function-sum) and [avg](../query_language/agg_functions/reference.md#agg_function-avg).
+
+[Original article](https://clickhouse.tech/docs/en/data_types/uuid/)
diff --git a/docs/fa/database_engines/index.md b/docs/fa/database_engines/index.md
deleted file mode 120000
index bbdb762a4ad..00000000000
--- a/docs/fa/database_engines/index.md
+++ /dev/null
@@ -1 +0,0 @@
-../../en/database_engines/index.md
\ No newline at end of file
diff --git a/docs/fa/database_engines/index.md b/docs/fa/database_engines/index.md
new file mode 100644
index 00000000000..2092982b496
--- /dev/null
+++ b/docs/fa/database_engines/index.md
@@ -0,0 +1,17 @@
+---
+en_copy: true
+---
+
+# Database Engines {#database-engines}
+
+Database engines allow you to work with tables.
+
+By default, ClickHouse uses its native database engine, which provides configurable [table engines](../operations/table_engines/index.md) and an [SQL dialect](../query_language/syntax.md).
+
+You can also use the following database engines:
+
+- [MySQL](mysql.md)
+
+- [Lazy](lazy.md)
+
+[Original article](https://clickhouse.tech/docs/en/database_engines/)
diff --git a/docs/fa/database_engines/lazy.md b/docs/fa/database_engines/lazy.md
deleted file mode 120000
index 66830dcdb2f..00000000000
--- a/docs/fa/database_engines/lazy.md
+++ /dev/null
@@ -1 +0,0 @@
-../../en/database_engines/lazy.md
\ No newline at end of file
diff --git a/docs/fa/database_engines/lazy.md b/docs/fa/database_engines/lazy.md
new file mode 100644
index 00000000000..45c5fd602d7
--- /dev/null
+++ b/docs/fa/database_engines/lazy.md
@@ -0,0 +1,15 @@
+---
+en_copy: true
+---
+
+# Lazy {#lazy}
+
+Keeps tables in RAM only `expiration_time_in_seconds` seconds after last access. Can be used only with \*Log tables.
+
+It’s optimized for storing many small \*Log tables, for which there is a long time interval between accesses.
+
+## Creating a Database {#creating-a-database}
+
+ CREATE DATABASE testlazy ENGINE = Lazy(expiration_time_in_seconds);
+
+[Original article](https://clickhouse.tech/docs/en/database_engines/lazy/)
diff --git a/docs/fa/database_engines/mysql.md b/docs/fa/database_engines/mysql.md
deleted file mode 120000
index 51ac4126e2d..00000000000
--- a/docs/fa/database_engines/mysql.md
+++ /dev/null
@@ -1 +0,0 @@
-../../en/database_engines/mysql.md
\ No newline at end of file
diff --git a/docs/fa/database_engines/mysql.md b/docs/fa/database_engines/mysql.md
new file mode 100644
index 00000000000..0c82f5860eb
--- /dev/null
+++ b/docs/fa/database_engines/mysql.md
@@ -0,0 +1,132 @@
+---
+en_copy: true
+---
+
+# MySQL {#mysql}
+
+Allows to connect to databases on a remote MySQL server and perform `INSERT` and `SELECT` queries to exchange data between ClickHouse and MySQL.
+
+The `MySQL` database engine translate queries to the MySQL server so you can perform operations such as `SHOW TABLES` or `SHOW CREATE TABLE`.
+
+You cannot perform the following queries:
+
+- `RENAME`
+- `CREATE TABLE`
+- `ALTER`
+
+## Creating a Database {#creating-a-database}
+
+``` sql
+CREATE DATABASE [IF NOT EXISTS] db_name [ON CLUSTER cluster]
+ENGINE = MySQL('host:port', 'database', 'user', 'password')
+```
+
+**Engine Parameters**
+
+- `host:port` — MySQL server address.
+- `database` — Remote database name.
+- `user` — MySQL user.
+- `password` — User password.
+
+## Data Types Support {#data_types-support}
+
+| MySQL | ClickHouse |
+|----------------------------------|---------------------------------------------|
+| UNSIGNED TINYINT | [UInt8](../data_types/int_uint.md) |
+| TINYINT | [Int8](../data_types/int_uint.md) |
+| UNSIGNED SMALLINT | [UInt16](../data_types/int_uint.md) |
+| SMALLINT | [Int16](../data_types/int_uint.md) |
+| UNSIGNED INT, UNSIGNED MEDIUMINT | [UInt32](../data_types/int_uint.md) |
+| INT, MEDIUMINT | [Int32](../data_types/int_uint.md) |
+| UNSIGNED BIGINT | [UInt64](../data_types/int_uint.md) |
+| BIGINT | [Int64](../data_types/int_uint.md) |
+| FLOAT | [Float32](../data_types/float.md) |
+| DOUBLE | [Float64](../data_types/float.md) |
+| DATE | [Date](../data_types/date.md) |
+| DATETIME, TIMESTAMP | [DateTime](../data_types/datetime.md) |
+| BINARY | [FixedString](../data_types/fixedstring.md) |
+
+All other MySQL data types are converted into [String](../data_types/string.md).
+
+[Nullable](../data_types/nullable.md) is supported.
+
+## Examples of Use {#examples-of-use}
+
+Table in MySQL:
+
+``` text
+mysql> USE test;
+Database changed
+
+mysql> CREATE TABLE `mysql_table` (
+ -> `int_id` INT NOT NULL AUTO_INCREMENT,
+ -> `float` FLOAT NOT NULL,
+ -> PRIMARY KEY (`int_id`));
+Query OK, 0 rows affected (0,09 sec)
+
+mysql> insert into mysql_table (`int_id`, `float`) VALUES (1,2);
+Query OK, 1 row affected (0,00 sec)
+
+mysql> select * from mysql_table;
++--------+-------+
+| int_id | value |
++--------+-------+
+| 1 | 2 |
++--------+-------+
+1 row in set (0,00 sec)
+```
+
+Database in ClickHouse, exchanging data with the MySQL server:
+
+``` sql
+CREATE DATABASE mysql_db ENGINE = MySQL('localhost:3306', 'test', 'my_user', 'user_password')
+```
+
+``` sql
+SHOW DATABASES
+```
+
+``` text
+┌─name─────┐
+│ default │
+│ mysql_db │
+│ system │
+└──────────┘
+```
+
+``` sql
+SHOW TABLES FROM mysql_db
+```
+
+``` text
+┌─name─────────┐
+│ mysql_table │
+└──────────────┘
+```
+
+``` sql
+SELECT * FROM mysql_db.mysql_table
+```
+
+``` text
+┌─int_id─┬─value─┐
+│ 1 │ 2 │
+└────────┴───────┘
+```
+
+``` sql
+INSERT INTO mysql_db.mysql_table VALUES (3,4)
+```
+
+``` sql
+SELECT * FROM mysql_db.mysql_table
+```
+
+``` text
+┌─int_id─┬─value─┐
+│ 1 │ 2 │
+│ 3 │ 4 │
+└────────┴───────┘
+```
+
+[Original article](https://clickhouse.tech/docs/en/database_engines/mysql/)
diff --git a/docs/fa/development/architecture.md b/docs/fa/development/architecture.md
deleted file mode 120000
index abda4dd48a8..00000000000
--- a/docs/fa/development/architecture.md
+++ /dev/null
@@ -1 +0,0 @@
-../../en/development/architecture.md
\ No newline at end of file
diff --git a/docs/fa/development/architecture.md b/docs/fa/development/architecture.md
new file mode 100644
index 00000000000..dc92d425d37
--- /dev/null
+++ b/docs/fa/development/architecture.md
@@ -0,0 +1,200 @@
+---
+en_copy: true
+---
+
+# Overview of ClickHouse Architecture {#overview-of-clickhouse-architecture}
+
+ClickHouse is a true column-oriented DBMS. Data is stored by columns and during the execution of arrays (vectors or chunks of columns). Whenever possible, operations are dispatched on arrays, rather than on individual values. It is called “vectorized query execution,” and it helps lower the cost of actual data processing.
+
+> This idea is nothing new. It dates back to the `APL` programming language and its descendants: `A +`, `J`, `K`, and `Q`. Array programming is used in scientific data processing. Neither is this idea something new in relational databases: for example, it is used in the `Vectorwise` system.
+
+There are two different approaches for speeding up query processing: vectorized query execution and runtime code generation. The latter removes all indirection and dynamic dispatch. Neither of these approaches is strictly better than the other. Runtime code generation can be better when it fuses many operations, thus fully utilizing CPU execution units and the pipeline. Vectorized query execution can be less practical because it involves temporary vectors that must be written to the cache and read back. If the temporary data does not fit in the L2 cache, this becomes an issue. But vectorized query execution more easily utilizes the SIMD capabilities of the CPU. A [research paper](http://15721.courses.cs.cmu.edu/spring2016/papers/p5-sompolski.pdf) written by our friends shows that it is better to combine both approaches. ClickHouse uses vectorized query execution and has limited initial support for runtime code generation.
+
+## Columns {#columns}
+
+`IColumn` interface is used to represent columns in memory (actually, chunks of columns). This interface provides helper methods for the implementation of various relational operators. Almost all operations are immutable: they do not modify the original column, but create a new modified one. For example, the `IColumn :: filter` method accepts a filter byte mask. It is used for the `WHERE` and `HAVING` relational operators. Additional examples: the `IColumn :: permute` method to support `ORDER BY`, the `IColumn :: cut` method to support `LIMIT`.
+
+Various `IColumn` implementations (`ColumnUInt8`, `ColumnString`, and so on) are responsible for the memory layout of columns. The memory layout is usually a contiguous array. For the integer type of columns, it is just one contiguous array, like `std :: vector`. For `String` and `Array` columns, it is two vectors: one for all array elements, placed contiguously, and a second one for offsets to the beginning of each array. There is also `ColumnConst` that stores just one value in memory, but looks like a column.
+
+## Field {#field}
+
+Nevertheless, it is possible to work with individual values as well. To represent an individual value, the `Field` is used. `Field` is just a discriminated union of `UInt64`, `Int64`, `Float64`, `String` and `Array`. `IColumn` has the `operator[]` method to get the n-th value as a `Field` and the `insert` method to append a `Field` to the end of a column. These methods are not very efficient, because they require dealing with temporary `Field` objects representing an individual value. There are more efficient methods, such as `insertFrom`, `insertRangeFrom`, and so on.
+
+`Field` doesn’t have enough information about a specific data type for a table. For example, `UInt8`, `UInt16`, `UInt32`, and `UInt64` are all represented as `UInt64` in a `Field`.
+
+## Leaky Abstractions {#leaky-abstractions}
+
+`IColumn` has methods for common relational transformations of data, but they don’t meet all needs. For example, `ColumnUInt64` doesn’t have a method to calculate the sum of two columns, and `ColumnString` doesn’t have a method to run a substring search. These countless routines are implemented outside of `IColumn`.
+
+Various functions on columns can be implemented in a generic, non-efficient way using `IColumn` methods to extract `Field` values, or in a specialized way using knowledge of inner memory layout of data in a specific `IColumn` implementation. It is implemented by casting functions to a specific `IColumn` type and deal with internal representation directly. For example, `ColumnUInt64` has the `getData` method that returns a reference to an internal array, then a separate routine reads or fills that array directly. We have “leaky abstractions” to allow efficient specializations of various routines.
+
+## Data Types {#data_types}
+
+`IDataType` is responsible for serialization and deserialization: for reading and writing chunks of columns or individual values in binary or text form. `IDataType` directly corresponds to data types in tables. For example, there are `DataTypeUInt32`, `DataTypeDateTime`, `DataTypeString` and so on.
+
+`IDataType` and `IColumn` are only loosely related to each other. Different data types can be represented in memory by the same `IColumn` implementations. For example, `DataTypeUInt32` and `DataTypeDateTime` are both represented by `ColumnUInt32` or `ColumnConstUInt32`. In addition, the same data type can be represented by different `IColumn` implementations. For example, `DataTypeUInt8` can be represented by `ColumnUInt8` or `ColumnConstUInt8`.
+
+`IDataType` only stores metadata. For instance, `DataTypeUInt8` doesn’t store anything at all (except vptr) and `DataTypeFixedString` stores just `N` (the size of fixed-size strings).
+
+`IDataType` has helper methods for various data formats. Examples are methods to serialize a value with possible quoting, to serialize a value for JSON, and to serialize a value as part of the XML format. There is no direct correspondence to data formats. For example, the different data formats `Pretty` and `TabSeparated` can use the same `serializeTextEscaped` helper method from the `IDataType` interface.
+
+## Block {#block}
+
+A `Block` is a container that represents a subset (chunk) of a table in memory. It is just a set of triples: `(IColumn, IDataType, column name)`. During query execution, data is processed by `Block`s. If we have a `Block`, we have data (in the `IColumn` object), we have information about its type (in `IDataType`) that tells us how to deal with that column, and we have the column name. It could be either the original column name from the table or some artificial name assigned for getting temporary results of calculations.
+
+When we calculate some function over columns in a block, we add another column with its result to the block, and we don’t touch columns for arguments of the function because operations are immutable. Later, unneeded columns can be removed from the block, but not modified. It is convenient for the elimination of common subexpressions.
+
+Blocks are created for every processed chunk of data. Note that for the same type of calculation, the column names and types remain the same for different blocks, and only column data changes. It is better to split block data from the block header because small block sizes have a high overhead of temporary strings for copying shared\_ptrs and column names.
+
+## Block Streams {#block-streams}
+
+Block streams are for processing data. We use streams of blocks to read data from somewhere, perform data transformations, or write data to somewhere. `IBlockInputStream` has the `read` method to fetch the next block while available. `IBlockOutputStream` has the `write` method to push the block somewhere.
+
+Streams are responsible for:
+
+1. Reading or writing to a table. The table just returns a stream for reading or writing blocks.
+2. Implementing data formats. For example, if you want to output data to a terminal in `Pretty` format, you create a block output stream where you push blocks, and it formats them.
+3. Performing data transformations. Let’s say you have `IBlockInputStream` and want to create a filtered stream. You create `FilterBlockInputStream` and initialize it with your stream. Then when you pull a block from `FilterBlockInputStream`, it pulls a block from your stream, filters it, and returns the filtered block to you. Query execution pipelines are represented this way.
+
+There are more sophisticated transformations. For example, when you pull from `AggregatingBlockInputStream`, it reads all data from its source, aggregates it, and then returns a stream of aggregated data for you. Another example: `UnionBlockInputStream` accepts many input sources in the constructor and also a number of threads. It launches multiple threads and reads from multiple sources in parallel.
+
+> Block streams use the “pull” approach to control flow: when you pull a block from the first stream, it consequently pulls the required blocks from nested streams, and the entire execution pipeline will work. Neither “pull” nor “push” is the best solution, because control flow is implicit, and that limits the implementation of various features like simultaneous execution of multiple queries (merging many pipelines together). This limitation could be overcome with coroutines or just running extra threads that wait for each other. We may have more possibilities if we make control flow explicit: if we locate the logic for passing data from one calculation unit to another outside of those calculation units. Read this [article](http://journal.stuffwithstuff.com/2013/01/13/iteration-inside-and-out/) for more thoughts.
+
+We should note that the query execution pipeline creates temporary data at each step. We try to keep block size small enough so that temporary data fits in the CPU cache. With that assumption, writing and reading temporary data is almost free in comparison with other calculations. We could consider an alternative, which is to fuse many operations in the pipeline together. It could make the pipeline as short as possible and remove much of the temporary data, which could be an advantage, but it also has drawbacks. For example, a split pipeline makes it easy to implement caching intermediate data, stealing intermediate data from similar queries running at the same time, and merging pipelines for similar queries.
+
+## Formats {#formats}
+
+Data formats are implemented with block streams. There are “presentational” formats only suitable for the output of data to the client, such as `Pretty` format, which provides only `IBlockOutputStream`. And there are input/output formats, such as `TabSeparated` or `JSONEachRow`.
+
+There are also row streams: `IRowInputStream` and `IRowOutputStream`. They allow you to pull/push data by individual rows, not by blocks. And they are only needed to simplify the implementation of row-oriented formats. The wrappers `BlockInputStreamFromRowInputStream` and `BlockOutputStreamFromRowOutputStream` allow you to convert row-oriented streams to regular block-oriented streams.
+
+## I/O {#io}
+
+For byte-oriented input/output, there are `ReadBuffer` and `WriteBuffer` abstract classes. They are used instead of C++ `iostream`s. Don’t worry: every mature C++ project is using something other than `iostream`s for good reasons.
+
+`ReadBuffer` and `WriteBuffer` are just a contiguous buffer and a cursor pointing to the position in that buffer. Implementations may own or not own the memory for the buffer. There is a virtual method to fill the buffer with the following data (for `ReadBuffer`) or to flush the buffer somewhere (for `WriteBuffer`). The virtual methods are rarely called.
+
+Implementations of `ReadBuffer`/`WriteBuffer` are used for working with files and file descriptors and network sockets, for implementing compression (`CompressedWriteBuffer` is initialized with another WriteBuffer and performs compression before writing data to it), and for other purposes – the names `ConcatReadBuffer`, `LimitReadBuffer`, and `HashingWriteBuffer` speak for themselves.
+
+Read/WriteBuffers only deal with bytes. There are functions from `ReadHelpers` and `WriteHelpers` header files to help with formatting input/output. For example, there are helpers to write a number in decimal format.
+
+Let’s look at what happens when you want to write a result set in `JSON` format to stdout. You have a result set ready to be fetched from `IBlockInputStream`. You create `WriteBufferFromFileDescriptor(STDOUT_FILENO)` to write bytes to stdout. You create `JSONRowOutputStream`, initialized with that `WriteBuffer`, to write rows in `JSON` to stdout. You create `BlockOutputStreamFromRowOutputStream` on top of it, to represent it as `IBlockOutputStream`. Then you call `copyData` to transfer data from `IBlockInputStream` to `IBlockOutputStream`, and everything works. Internally, `JSONRowOutputStream` will write various JSON delimiters and call the `IDataType::serializeTextJSON` method with a reference to `IColumn` and the row number as arguments. Consequently, `IDataType::serializeTextJSON` will call a method from `WriteHelpers.h`: for example, `writeText` for numeric types and `writeJSONString` for `DataTypeString`.
+
+## Tables {#tables}
+
+The `IStorage` interface represents tables. Different implementations of that interface are different table engines. Examples are `StorageMergeTree`, `StorageMemory`, and so on. Instances of these classes are just tables.
+
+The key `IStorage` methods are `read` and `write`. There are also `alter`, `rename`, `drop`, and so on. The `read` method accepts the following arguments: the set of columns to read from a table, the `AST` query to consider, and the desired number of streams to return. It returns one or multiple `IBlockInputStream` objects and information about the stage of data processing that was completed inside a table engine during query execution.
+
+In most cases, the read method is only responsible for reading the specified columns from a table, not for any further data processing. All further data processing is done by the query interpreter and is outside the responsibility of `IStorage`.
+
+But there are notable exceptions:
+
+- The AST query is passed to the `read` method, and the table engine can use it to derive index usage and to read fewer data from a table.
+- Sometimes the table engine can process data itself to a specific stage. For example, `StorageDistributed` can send a query to remote servers, ask them to process data to a stage where data from different remote servers can be merged, and return that preprocessed data. The query interpreter then finishes processing the data.
+
+The table’s `read` method can return multiple `IBlockInputStream` objects to allow parallel data processing. These multiple block input streams can read from a table in parallel. Then you can wrap these streams with various transformations (such as expression evaluation or filtering) that can be calculated independently and create a `UnionBlockInputStream` on top of them, to read from multiple streams in parallel.
+
+There are also `TableFunction`s. These are functions that return a temporary `IStorage` object to use in the `FROM` clause of a query.
+
+To get a quick idea of how to implement your table engine, look at something simple, like `StorageMemory` or `StorageTinyLog`.
+
+> As the result of the `read` method, `IStorage` returns `QueryProcessingStage` – information about what parts of the query were already calculated inside storage.
+
+## Parsers {#parsers}
+
+A hand-written recursive descent parser parses a query. For example, `ParserSelectQuery` just recursively calls the underlying parsers for various parts of the query. Parsers create an `AST`. The `AST` is represented by nodes, which are instances of `IAST`.
+
+> Parser generators are not used for historical reasons.
+
+## Interpreters {#interpreters}
+
+Interpreters are responsible for creating the query execution pipeline from an `AST`. There are simple interpreters, such as `InterpreterExistsQuery` and `InterpreterDropQuery`, or the more sophisticated `InterpreterSelectQuery`. The query execution pipeline is a combination of block input or output streams. For example, the result of interpreting the `SELECT` query is the `IBlockInputStream` to read the result set from; the result of the INSERT query is the `IBlockOutputStream` to write data for insertion to, and the result of interpreting the `INSERT SELECT` query is the `IBlockInputStream` that returns an empty result set on the first read, but that copies data from `SELECT` to `INSERT` at the same time.
+
+`InterpreterSelectQuery` uses `ExpressionAnalyzer` and `ExpressionActions` machinery for query analysis and transformations. This is where most rule-based query optimizations are done. `ExpressionAnalyzer` is quite messy and should be rewritten: various query transformations and optimizations should be extracted to separate classes to allow modular transformations or query.
+
+## Functions {#functions}
+
+There are ordinary functions and aggregate functions. For aggregate functions, see the next section.
+
+Ordinary functions don’t change the number of rows – they work as if they are processing each row independently. In fact, functions are not called for individual rows, but for `Block`’s of data to implement vectorized query execution.
+
+There are some miscellaneous functions, like [blockSize](../query_language/functions/other_functions.md#function-blocksize), [rowNumberInBlock](../query_language/functions/other_functions.md#function-rownumberinblock), and [runningAccumulate](../query_language/functions/other_functions.md#function-runningaccumulate), that exploit block processing and violate the independence of rows.
+
+ClickHouse has strong typing, so there’s no implicit type conversion. If a function doesn’t support a specific combination of types, it throws an exception. But functions can work (be overloaded) for many different combinations of types. For example, the `plus` function (to implement the `+` operator) works for any combination of numeric types: `UInt8` + `Float32`, `UInt16` + `Int8`, and so on. Also, some variadic functions can accept any number of arguments, such as the `concat` function.
+
+Implementing a function may be slightly inconvenient because a function explicitly dispatches supported data types and supported `IColumns`. For example, the `plus` function has code generated by instantiation of a C++ template for each combination of numeric types, and constant or non-constant left and right arguments.
+
+It is an excellent place to implement runtime code generation to avoid template code bloat. Also, it makes it possible to add fused functions like fused multiply-add or to make multiple comparisons in one loop iteration.
+
+Due to vectorized query execution, functions are not short-circuited. For example, if you write `WHERE f(x) AND g(y)`, both sides are calculated, even for rows, when `f(x)` is zero (except when `f(x)` is a zero constant expression). But if the selectivity of the `f(x)` condition is high, and calculation of `f(x)` is much cheaper than `g(y)`, it’s better to implement multi-pass calculation. It would first calculate `f(x)`, then filter columns by the result, and then calculate `g(y)` only for smaller, filtered chunks of data.
+
+## Aggregate Functions {#aggregate-functions}
+
+Aggregate functions are stateful functions. They accumulate passed values into some state and allow you to get results from that state. They are managed with the `IAggregateFunction` interface. States can be rather simple (the state for `AggregateFunctionCount` is just a single `UInt64` value) or quite complex (the state of `AggregateFunctionUniqCombined` is a combination of a linear array, a hash table, and a `HyperLogLog` probabilistic data structure).
+
+States are allocated in `Arena` (a memory pool) to deal with multiple states while executing a high-cardinality `GROUP BY` query. States can have a non-trivial constructor and destructor: for example, complicated aggregation states can allocate additional memory themselves. It requires some attention to creating and destroying states and properly passing their ownership and destruction order.
+
+Aggregation states can be serialized and deserialized to pass over the network during distributed query execution or to write them on the disk where there is not enough RAM. They can even be stored in a table with the `DataTypeAggregateFunction` to allow incremental aggregation of data.
+
+> The serialized data format for aggregate function states is not versioned right now. It is ok if aggregate states are only stored temporarily. But we have the `AggregatingMergeTree` table engine for incremental aggregation, and people are already using it in production. It is the reason why backward compatibility is required when changing the serialized format for any aggregate function in the future.
+
+## Server {#server}
+
+The server implements several different interfaces:
+
+- An HTTP interface for any foreign clients.
+- A TCP interface for the native ClickHouse client and for cross-server communication during distributed query execution.
+- An interface for transferring data for replication.
+
+Internally, it is just a primitive multithreaded server without coroutines or fibers. Since the server is not designed to process a high rate of simple queries but to process a relatively low rate of complex queries, each of them can process a vast amount of data for analytics.
+
+The server initializes the `Context` class with the necessary environment for query execution: the list of available databases, users and access rights, settings, clusters, the process list, the query log, and so on. Interpreters use this environment.
+
+We maintain full backward and forward compatibility for the server TCP protocol: old clients can talk to new servers, and new clients can talk to old servers. But we don’t want to maintain it eternally, and we are removing support for old versions after about one year.
+
+!!! note "Note"
+ For most external applications, we recommend using the HTTP interface because it is simple and easy to use. The TCP protocol is more tightly linked to internal data structures: it uses an internal format for passing blocks of data, and it uses custom framing for compressed data. We haven’t released a C library for that protocol because it requires linking most of the ClickHouse codebase, which is not practical.
+
+## Distributed Query Execution {#distributed-query-execution}
+
+Servers in a cluster setup are mostly independent. You can create a `Distributed` table on one or all servers in a cluster. The `Distributed` table does not store data itself – it only provides a “view” to all local tables on multiple nodes of a cluster. When you SELECT from a `Distributed` table, it rewrites that query, chooses remote nodes according to load balancing settings, and sends the query to them. The `Distributed` table requests remote servers to process a query just up to a stage where intermediate results from different servers can be merged. Then it receives the intermediate results and merges them. The distributed table tries to distribute as much work as possible to remote servers and does not send much intermediate data over the network.
+
+Things become more complicated when you have subqueries in IN or JOIN clauses, and each of them uses a `Distributed` table. We have different strategies for the execution of these queries.
+
+There is no global query plan for distributed query execution. Each node has its local query plan for its part of the job. We only have simple one-pass distributed query execution: we send queries for remote nodes and then merge the results. But this is not feasible for complicated queries with high cardinality GROUP BYs or with a large amount of temporary data for JOIN. In such cases, we need to “reshuffle” data between servers, which requires additional coordination. ClickHouse does not support that kind of query execution, and we need to work on it.
+
+## Merge Tree {#merge-tree}
+
+`MergeTree` is a family of storage engines that supports indexing by primary key. The primary key can be an arbitrary tuple of columns or expressions. Data in a `MergeTree` table is stored in “parts”. Each part stores data in the primary key order, so data is ordered lexicographically by the primary key tuple. All the table columns are stored in separate `column.bin` files in these parts. The files consist of compressed blocks. Each block is usually from 64 KB to 1 MB of uncompressed data, depending on the average value size. The blocks consist of column values placed contiguously one after the other. Column values are in the same order for each column (the primary key defines the order), so when you iterate by many columns, you get values for the corresponding rows.
+
+The primary key itself is “sparse”. It doesn’t address every single row, but only some ranges of data. A separate `primary.idx` file has the value of the primary key for each N-th row, where N is called `index_granularity` (usually, N = 8192). Also, for each column, we have `column.mrk` files with “marks,” which are offsets to each N-th row in the data file. Each mark is a pair: the offset in the file to the beginning of the compressed block, and the offset in the decompressed block to the beginning of data. Usually, compressed blocks are aligned by marks, and the offset in the decompressed block is zero. Data for `primary.idx` always resides in memory, and data for `column.mrk` files is cached.
+
+When we are going to read something from a part in `MergeTree`, we look at `primary.idx` data and locate ranges that could contain requested data, then look at `column.mrk` data and calculate offsets for where to start reading those ranges. Because of sparseness, excess data may be read. ClickHouse is not suitable for a high load of simple point queries, because the entire range with `index_granularity` rows must be read for each key, and the entire compressed block must be decompressed for each column. We made the index sparse because we must be able to maintain trillions of rows per single server without noticeable memory consumption for the index. Also, because the primary key is sparse, it is not unique: it cannot check the existence of the key in the table at INSERT time. You could have many rows with the same key in a table.
+
+When you `INSERT` a bunch of data into `MergeTree`, that bunch is sorted by primary key order and forms a new part. There are background threads that periodically select some parts and merge them into a single sorted part to keep the number of parts relatively low. That’s why it is called `MergeTree`. Of course, merging leads to “write amplification”. All parts are immutable: they are only created and deleted, but not modified. When SELECT is executed, it holds a snapshot of the table (a set of parts). After merging, we also keep old parts for some time to make a recovery after failure easier, so if we see that some merged part is probably broken, we can replace it with its source parts.
+
+`MergeTree` is not an LSM tree because it doesn’t contain “memtable” and “log”: inserted data is written directly to the filesystem. This makes it suitable only to INSERT data in batches, not by individual row and not very frequently – about once per second is ok, but a thousand times a second is not. We did it this way for simplicity’s sake, and because we are already inserting data in batches in our applications.
+
+> MergeTree tables can only have one (primary) index: there aren’t any secondary indices. It would be nice to allow multiple physical representations under one logical table, for example, to store data in more than one physical order or even to allow representations with pre-aggregated data along with original data.
+
+There are MergeTree engines that are doing additional work during background merges. Examples are `CollapsingMergeTree` and `AggregatingMergeTree`. This could be treated as special support for updates. Keep in mind that these are not real updates because users usually have no control over the time when background merges are executed, and data in a `MergeTree` table is almost always stored in more than one part, not in completely merged form.
+
+## Replication {#replication}
+
+Replication in ClickHouse can be configured on a per-table basis. You could have some replicated and some non-replicated tables on the same server. You could also have tables replicated in different ways, such as one table with two-factor replication and another with three-factor.
+
+Replication is implemented in the `ReplicatedMergeTree` storage engine. The path in `ZooKeeper` is specified as a parameter for the storage engine. All tables with the same path in `ZooKeeper` become replicas of each other: they synchronize their data and maintain consistency. Replicas can be added and removed dynamically simply by creating or dropping a table.
+
+Replication uses an asynchronous multi-master scheme. You can insert data into any replica that has a session with `ZooKeeper`, and data is replicated to all other replicas asynchronously. Because ClickHouse doesn’t support UPDATEs, replication is conflict-free. As there is no quorum acknowledgment of inserts, just-inserted data might be lost if one node fails.
+
+Metadata for replication is stored in ZooKeeper. There is a replication log that lists what actions to do. Actions are: get part; merge parts; drop a partition, and so on. Each replica copies the replication log to its queue and then executes the actions from the queue. For example, on insertion, the “get the part” action is created in the log, and every replica downloads that part. Merges are coordinated between replicas to get byte-identical results. All parts are merged in the same way on all replicas. It is achieved by electing one replica as the leader, and that replica initiates merges and writes “merge parts” actions to the log.
+
+Replication is physical: only compressed parts are transferred between nodes, not queries. Merges are processed on each replica independently in most cases to lower the network costs by avoiding network amplification. Large merged parts are sent over the network only in cases of significant replication lag.
+
+Besides, each replica stores its state in ZooKeeper as the set of parts and its checksums. When the state on the local filesystem diverges from the reference state in ZooKeeper, the replica restores its consistency by downloading missing and broken parts from other replicas. When there is some unexpected or broken data in the local filesystem, ClickHouse does not remove it, but moves it to a separate directory and forgets it.
+
+!!! note "Note"
+ The ClickHouse cluster consists of independent shards, and each shard consists of replicas. The cluster is **not elastic**, so after adding a new shard, data is not rebalanced between shards automatically. Instead, the cluster load is supposed to be adjusted to be uneven. This implementation gives you more control, and it is ok for relatively small clusters, such as tens of nodes. But for clusters with hundreds of nodes that we are using in production, this approach becomes a significant drawback. We should implement a table engine that spans across the cluster with dynamically replicated regions that could be split and balanced between clusters automatically.
+
+{## [Original article](https://clickhouse.tech/docs/en/development/architecture/) ##}
diff --git a/docs/fa/development/browse_code.md b/docs/fa/development/browse_code.md
deleted file mode 120000
index 8c08c622129..00000000000
--- a/docs/fa/development/browse_code.md
+++ /dev/null
@@ -1 +0,0 @@
-../../en/development/browse_code.md
\ No newline at end of file
diff --git a/docs/fa/development/browse_code.md b/docs/fa/development/browse_code.md
new file mode 100644
index 00000000000..c3016d5e1dc
--- /dev/null
+++ b/docs/fa/development/browse_code.md
@@ -0,0 +1,11 @@
+---
+en_copy: true
+---
+
+# Browse ClickHouse Source Code {#browse-clickhouse-source-code}
+
+You can use **Woboq** online code browser available [here](https://clickhouse-test-reports.s3.yandex.net/codebrowser/html_report///ClickHouse/dbms/index.html). It provides code navigation and semantic highlighting, search and indexing. The code snapshot is updated daily.
+
+Also, you can browse sources on [GitHub](https://github.com/ClickHouse/ClickHouse) as usual.
+
+If you’re interested what IDE to use, we recommend CLion, QT Creator, VS Code and KDevelop (with caveats). You can use any favourite IDE. Vim and Emacs also count.
diff --git a/docs/fa/development/build.md b/docs/fa/development/build.md
deleted file mode 120000
index 480dbc2e9f5..00000000000
--- a/docs/fa/development/build.md
+++ /dev/null
@@ -1 +0,0 @@
-../../en/development/build.md
\ No newline at end of file
diff --git a/docs/fa/development/build.md b/docs/fa/development/build.md
new file mode 100644
index 00000000000..32042a4128e
--- /dev/null
+++ b/docs/fa/development/build.md
@@ -0,0 +1,138 @@
+---
+en_copy: true
+---
+
+# How to Build ClickHouse for Development {#how-to-build-clickhouse-for-development}
+
+The following tutorial is based on the Ubuntu Linux system.
+With appropriate changes, it should also work on any other Linux distribution.
+Supported platforms: x86\_64 and AArch64. Support for Power9 is experimental.
+
+## Install Git, CMake, Python and Ninja {#install-git-cmake-python-and-ninja}
+
+``` bash
+$ sudo apt-get install git cmake python ninja-build
+```
+
+Or cmake3 instead of cmake on older systems.
+
+## Install GCC 9 {#install-gcc-9}
+
+There are several ways to do this.
+
+### Install from a PPA Package {#install-from-a-ppa-package}
+
+``` bash
+$ sudo apt-get install software-properties-common
+$ sudo apt-add-repository ppa:ubuntu-toolchain-r/test
+$ sudo apt-get update
+$ sudo apt-get install gcc-9 g++-9
+```
+
+### Install from Sources {#install-from-sources}
+
+Look at [utils/ci/build-gcc-from-sources.sh](https://github.com/ClickHouse/ClickHouse/blob/master/utils/ci/build-gcc-from-sources.sh)
+
+## Use GCC 9 for Builds {#use-gcc-9-for-builds}
+
+``` bash
+$ export CC=gcc-9
+$ export CXX=g++-9
+```
+
+## Checkout ClickHouse Sources {#checkout-clickhouse-sources}
+
+``` bash
+$ git clone --recursive git@github.com:ClickHouse/ClickHouse.git
+```
+
+or
+
+``` bash
+$ git clone --recursive https://github.com/ClickHouse/ClickHouse.git
+```
+
+## Build ClickHouse {#build-clickhouse}
+
+``` bash
+$ cd ClickHouse
+$ mkdir build
+$ cd build
+$ cmake ..
+$ ninja
+$ cd ..
+```
+
+To create an executable, run `ninja clickhouse`.
+This will create the `programs/clickhouse` executable, which can be used with `client` or `server` arguments.
+
+# How to Build ClickHouse on Any Linux {#how-to-build-clickhouse-on-any-linux}
+
+The build requires the following components:
+
+- Git (is used only to checkout the sources, it’s not needed for the build)
+- CMake 3.10 or newer
+- Ninja (recommended) or Make
+- C++ compiler: gcc 9 or clang 8 or newer
+- Linker: lld or gold (the classic GNU ld won’t work)
+- Python (is only used inside LLVM build and it is optional)
+
+If all the components are installed, you may build in the same way as the steps above.
+
+Example for Ubuntu Eoan:
+
+ sudo apt update
+ sudo apt install git cmake ninja-build g++ python
+ git clone --recursive https://github.com/ClickHouse/ClickHouse.git
+ mkdir build && cd build
+ cmake ../ClickHouse
+ ninja
+
+Example for OpenSUSE Tumbleweed:
+
+ sudo zypper install git cmake ninja gcc-c++ python lld
+ git clone --recursive https://github.com/ClickHouse/ClickHouse.git
+ mkdir build && cd build
+ cmake ../ClickHouse
+ ninja
+
+Example for Fedora Rawhide:
+
+ sudo yum update
+ yum --nogpg install git cmake make gcc-c++ python2
+ git clone --recursive https://github.com/ClickHouse/ClickHouse.git
+ mkdir build && cd build
+ cmake ../ClickHouse
+ make -j $(nproc)
+
+# You Don’t Have to Build ClickHouse {#you-dont-have-to-build-clickhouse}
+
+ClickHouse is available in pre-built binaries and packages. Binaries are portable and can be run on any Linux flavour.
+
+They are built for stable, prestable and testing releases as long as for every commit to master and for every pull request.
+
+To find the freshest build from `master`, go to [commits page](https://github.com/ClickHouse/ClickHouse/commits/master), click on the first green checkmark or red cross near commit, and click to the “Details” link right after “ClickHouse Build Check”.
+
+# How to Build ClickHouse Debian Package {#how-to-build-clickhouse-debian-package}
+
+## Install Git and Pbuilder {#install-git-and-pbuilder}
+
+``` bash
+$ sudo apt-get update
+$ sudo apt-get install git python pbuilder debhelper lsb-release fakeroot sudo debian-archive-keyring debian-keyring
+```
+
+## Checkout ClickHouse Sources {#checkout-clickhouse-sources-1}
+
+``` bash
+$ git clone --recursive --branch master https://github.com/ClickHouse/ClickHouse.git
+$ cd ClickHouse
+```
+
+## Run Release Script {#run-release-script}
+
+``` bash
+$ ./release
+```
+
+[Original article](https://clickhouse.tech/docs/en/development/build/)
diff --git a/docs/fa/development/build_cross_arm.md b/docs/fa/development/build_cross_arm.md
deleted file mode 120000
index 983a9872dc1..00000000000
--- a/docs/fa/development/build_cross_arm.md
+++ /dev/null
@@ -1 +0,0 @@
-../../en/development/build_cross_arm.md
\ No newline at end of file
diff --git a/docs/fa/development/build_cross_arm.md b/docs/fa/development/build_cross_arm.md
new file mode 100644
index 00000000000..0936a3133b2
--- /dev/null
+++ b/docs/fa/development/build_cross_arm.md
@@ -0,0 +1,40 @@
+---
+en_copy: true
+---
+
+# How to Build ClickHouse on Linux for AARCH64 (ARM64) architecture {#how-to-build-clickhouse-on-linux-for-aarch64-arm64-architecture}
+
+This is for the case when you have Linux machine and want to use it to build `clickhouse` binary that will run on another Linux machine with AARCH64 CPU architecture. This is intended for continuous integration checks that run on Linux servers.
+
+The cross-build for AARCH64 is based on the [Build instructions](build.md), follow them first.
+
+# Install Clang-8 {#install-clang-8}
+
+Follow the instructions from https://apt.llvm.org/ for your Ubuntu or Debian setup.
+For example, in Ubuntu Bionic you can use the following commands:
+
+``` bash
+echo "deb [trusted=yes] http://apt.llvm.org/bionic/ llvm-toolchain-bionic-8 main" | sudo tee /etc/apt/sources.list.d/llvm.list
+sudo apt-get update
+sudo apt-get install clang-8
+```
+
+# Install Cross-Compilation Toolset {#install-cross-compilation-toolset}
+
+``` bash
+cd ClickHouse
+mkdir -p build-aarch64/cmake/toolchain/linux-aarch64
+wget 'https://developer.arm.com/-/media/Files/downloads/gnu-a/8.3-2019.03/binrel/gcc-arm-8.3-2019.03-x86_64-aarch64-linux-gnu.tar.xz?revision=2e88a73f-d233-4f96-b1f4-d8b36e9bb0b9&la=en' -O gcc-arm-8.3-2019.03-x86_64-aarch64-linux-gnu.tar.xz
+tar xJf gcc-arm-8.3-2019.03-x86_64-aarch64-linux-gnu.tar.xz -C build-aarch64/cmake/toolchain/linux-aarch64 --strip-components=1
+```
+
+# Build ClickHouse {#build-clickhouse}
+
+``` bash
+cd ClickHouse
+mkdir build-arm64
+CC=clang-8 CXX=clang++-8 cmake . -Bbuild-arm64 -DCMAKE_TOOLCHAIN_FILE=cmake/linux/toolchain-aarch64.cmake
+ninja -C build-arm64
+```
+
+The resulting binary will run only on Linux with the AARCH64 CPU architecture.
diff --git a/docs/fa/development/build_cross_osx.md b/docs/fa/development/build_cross_osx.md
deleted file mode 120000
index 72e64e8631f..00000000000
--- a/docs/fa/development/build_cross_osx.md
+++ /dev/null
@@ -1 +0,0 @@
-../../en/development/build_cross_osx.md
\ No newline at end of file
diff --git a/docs/fa/development/build_cross_osx.md b/docs/fa/development/build_cross_osx.md
new file mode 100644
index 00000000000..a708dc4d4f3
--- /dev/null
+++ b/docs/fa/development/build_cross_osx.md
@@ -0,0 +1,61 @@
+---
+en_copy: true
+---
+
+# How to Build ClickHouse on Linux for Mac OS X {#how-to-build-clickhouse-on-linux-for-mac-os-x}
+
+This is for the case when you have Linux machine and want to use it to build `clickhouse` binary that will run on OS X. This is intended for continuous integration checks that run on Linux servers. If you want to build ClickHouse directly on Mac OS X, then proceed with [another instruction](build_osx.md).
+
+The cross-build for Mac OS X is based on the [Build instructions](build.md), follow them first.
+
+# Install Clang-8 {#install-clang-8}
+
+Follow the instructions from https://apt.llvm.org/ for your Ubuntu or Debian setup.
+For example the commands for Bionic are like:
+
+``` bash
+sudo echo "deb [trusted=yes] http://apt.llvm.org/bionic/ llvm-toolchain-bionic-8 main" >> /etc/apt/sources.list
+sudo apt-get install clang-8
+```
+
+# Install Cross-Compilation Toolset {#install-cross-compilation-toolset}
+
+Let’s remember the path where we install `cctools` as ${CCTOOLS}
+
+``` bash
+mkdir ${CCTOOLS}
+
+git clone https://github.com/tpoechtrager/apple-libtapi.git
+cd apple-libtapi
+INSTALLPREFIX=${CCTOOLS} ./build.sh
+./install.sh
+cd ..
+
+git clone https://github.com/tpoechtrager/cctools-port.git
+cd cctools-port/cctools
+./configure --prefix=${CCTOOLS} --with-libtapi=${CCTOOLS} --target=x86_64-apple-darwin
+make install
+```
+
+Also, we need to download macOS X SDK into the working tree.
+
+``` bash
+cd ClickHouse
+wget 'https://github.com/phracker/MacOSX-SDKs/releases/download/10.14-beta4/MacOSX10.14.sdk.tar.xz'
+mkdir -p build-darwin/cmake/toolchain/darwin-x86_64
+tar xJf MacOSX10.14.sdk.tar.xz -C build-darwin/cmake/toolchain/darwin-x86_64 --strip-components=1
+```
+
+# Build ClickHouse {#build-clickhouse}
+
+``` bash
+cd ClickHouse
+mkdir build-osx
+CC=clang-8 CXX=clang++-8 cmake . -Bbuild-osx -DCMAKE_TOOLCHAIN_FILE=cmake/darwin/toolchain-x86_64.cmake \
+ -DCMAKE_AR:FILEPATH=${CCTOOLS}/bin/x86_64-apple-darwin-ar \
+ -DCMAKE_RANLIB:FILEPATH=${CCTOOLS}/bin/x86_64-apple-darwin-ranlib \
+ -DLINKER_NAME=${CCTOOLS}/bin/x86_64-apple-darwin-ld
+ninja -C build-osx
+```
+
+The resulting binary will have a Mach-O executable format and can’t be run on Linux.
diff --git a/docs/fa/development/build_osx.md b/docs/fa/development/build_osx.md
deleted file mode 120000
index f9adaf24584..00000000000
--- a/docs/fa/development/build_osx.md
+++ /dev/null
@@ -1 +0,0 @@
-../../en/development/build_osx.md
\ No newline at end of file
diff --git a/docs/fa/development/build_osx.md b/docs/fa/development/build_osx.md
new file mode 100644
index 00000000000..6b1839aaf7f
--- /dev/null
+++ b/docs/fa/development/build_osx.md
@@ -0,0 +1,90 @@
+---
+en_copy: true
+---
+
+# How to Build ClickHouse on Mac OS X {#how-to-build-clickhouse-on-mac-os-x}
+
+Build should work on Mac OS X 10.15 (Catalina)
+
+## Install Homebrew {#install-homebrew}
+
+``` bash
+$ /usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
+```
+
+## Install Required Compilers, Tools, and Libraries {#install-required-compilers-tools-and-libraries}
+
+``` bash
+$ brew install cmake ninja libtool gettext
+```
+
+## Checkout ClickHouse Sources {#checkout-clickhouse-sources}
+
+``` bash
+$ git clone --recursive git@github.com:ClickHouse/ClickHouse.git
+```
+
+or
+
+``` bash
+$ git clone --recursive https://github.com/ClickHouse/ClickHouse.git
+
+$ cd ClickHouse
+```
+
+## Build ClickHouse {#build-clickhouse}
+
+``` bash
+$ mkdir build
+$ cd build
+$ cmake .. -DCMAKE_CXX_COMPILER=`which clang++` -DCMAKE_C_COMPILER=`which clang`
+$ ninja
+$ cd ..
+```
+
+## Caveats {#caveats}
+
+If you intend to run clickhouse-server, make sure to increase the system’s maxfiles variable.
+
+!!! info "Note"
+ You’ll need to use sudo.
+
+To do so, create the following file:
+
+/Library/LaunchDaemons/limit.maxfiles.plist:
+
+``` xml
+
+
+
+
+ Label
+ limit.maxfiles
+ ProgramArguments
+
+ launchctl
+ limit
+ maxfiles
+ 524288
+ 524288
+
+ RunAtLoad
+
+ ServiceIPC
+
+
+
+```
+
+Execute the following command:
+
+``` bash
+$ sudo chown root:wheel /Library/LaunchDaemons/limit.maxfiles.plist
+```
+
+Reboot.
+
+To check if it’s working, you can use `ulimit -n` command.
+
+[Original article](https://clickhouse.tech/docs/en/development/build_osx/)
diff --git a/docs/fa/development/contrib.md b/docs/fa/development/contrib.md
deleted file mode 120000
index 4749f95f9ef..00000000000
--- a/docs/fa/development/contrib.md
+++ /dev/null
@@ -1 +0,0 @@
-../../en/development/contrib.md
\ No newline at end of file
diff --git a/docs/fa/development/contrib.md b/docs/fa/development/contrib.md
new file mode 100644
index 00000000000..2404509187e
--- /dev/null
+++ b/docs/fa/development/contrib.md
@@ -0,0 +1,39 @@
+---
+en_copy: true
+---
+
+# Third-Party Libraries Used {#third-party-libraries-used}
+
+| Library | License |
+|---------------------|----------------------------------------------------------------------------------------------------------------------------------------------|
+| base64 | [BSD 2-Clause License](https://github.com/aklomp/base64/blob/a27c565d1b6c676beaf297fe503c4518185666f7/LICENSE) |
+| boost | [Boost Software License 1.0](https://github.com/ClickHouse-Extras/boost-extra/blob/6883b40449f378019aec792f9983ce3afc7ff16e/LICENSE_1_0.txt) |
+| brotli | [MIT](https://github.com/google/brotli/blob/master/LICENSE) |
+| capnproto | [MIT](https://github.com/capnproto/capnproto/blob/master/LICENSE) |
+| cctz | [Apache License 2.0](https://github.com/google/cctz/blob/4f9776a310f4952454636363def82c2bf6641d5f/LICENSE.txt) |
+| double-conversion | [BSD 3-Clause License](https://github.com/google/double-conversion/blob/cf2f0f3d547dc73b4612028a155b80536902ba02/LICENSE) |
+| FastMemcpy | [MIT](https://github.com/ClickHouse/ClickHouse/blob/master/libs/libmemcpy/impl/LICENSE) |
+| googletest | [BSD 3-Clause License](https://github.com/google/googletest/blob/master/LICENSE) |
+| h3 | [Apache License 2.0](https://github.com/uber/h3/blob/master/LICENSE) |
+| hyperscan | [BSD 3-Clause License](https://github.com/intel/hyperscan/blob/master/LICENSE) |
+| libbtrie | [BSD 2-Clause License](https://github.com/ClickHouse/ClickHouse/blob/master/contrib/libbtrie/LICENSE) |
+| libcxxabi | [BSD + MIT](https://github.com/ClickHouse/ClickHouse/blob/master/libs/libglibc-compatibility/libcxxabi/LICENSE.TXT) |
+| libdivide | [Zlib License](https://github.com/ClickHouse/ClickHouse/blob/master/contrib/libdivide/LICENSE.txt) |
+| libgsasl | [LGPL v2.1](https://github.com/ClickHouse-Extras/libgsasl/blob/3b8948a4042e34fb00b4fb987535dc9e02e39040/LICENSE) |
+| libhdfs3 | [Apache License 2.0](https://github.com/ClickHouse-Extras/libhdfs3/blob/bd6505cbb0c130b0db695305b9a38546fa880e5a/LICENSE.txt) |
+| libmetrohash | [Apache License 2.0](https://github.com/ClickHouse/ClickHouse/blob/master/contrib/libmetrohash/LICENSE) |
+| libpcg-random | [Apache License 2.0](https://github.com/ClickHouse/ClickHouse/blob/master/contrib/libpcg-random/LICENSE-APACHE.txt) |
+| libressl | [OpenSSL License](https://github.com/ClickHouse-Extras/ssl/blob/master/COPYING) |
+| librdkafka | [BSD 2-Clause License](https://github.com/edenhill/librdkafka/blob/363dcad5a23dc29381cc626620e68ae418b3af19/LICENSE) |
+| libwidechar\_width | [CC0 1.0 Universal](https://github.com/ClickHouse/ClickHouse/blob/master/libs/libwidechar_width/LICENSE) |
+| llvm | [BSD 3-Clause License](https://github.com/ClickHouse-Extras/llvm/blob/163def217817c90fb982a6daf384744d8472b92b/llvm/LICENSE.TXT) |
+| lz4 | [BSD 2-Clause License](https://github.com/lz4/lz4/blob/c10863b98e1503af90616ae99725ecd120265dfb/LICENSE) |
+| mariadb-connector-c | [LGPL v2.1](https://github.com/ClickHouse-Extras/mariadb-connector-c/blob/3.1/COPYING.LIB) |
+| murmurhash | [Public Domain](https://github.com/ClickHouse/ClickHouse/blob/master/contrib/murmurhash/LICENSE) |
+| pdqsort | [Zlib License](https://github.com/ClickHouse/ClickHouse/blob/master/contrib/pdqsort/license.txt) |
+| poco | [Boost Software License - Version 1.0](https://github.com/ClickHouse-Extras/poco/blob/fe5505e56c27b6ecb0dcbc40c49dc2caf4e9637f/LICENSE) |
+| protobuf | [BSD 3-Clause License](https://github.com/ClickHouse-Extras/protobuf/blob/12735370922a35f03999afff478e1c6d7aa917a4/LICENSE) |
+| re2 | [BSD 3-Clause License](https://github.com/google/re2/blob/7cf8b88e8f70f97fd4926b56aa87e7f53b2717e0/LICENSE) |
+| UnixODBC | [LGPL v2.1](https://github.com/ClickHouse-Extras/UnixODBC/tree/b0ad30f7f6289c12b76f04bfb9d466374bb32168) |
+| zlib-ng | [Zlib License](https://github.com/ClickHouse-Extras/zlib-ng/blob/develop/LICENSE.md) |
+| zstd | [BSD 3-Clause License](https://github.com/facebook/zstd/blob/dev/LICENSE) |
diff --git a/docs/fa/development/developer_instruction.md b/docs/fa/development/developer_instruction.md
deleted file mode 120000
index bdfa9047aa2..00000000000
--- a/docs/fa/development/developer_instruction.md
+++ /dev/null
@@ -1 +0,0 @@
-../../en/development/developer_instruction.md
\ No newline at end of file
diff --git a/docs/fa/development/developer_instruction.md b/docs/fa/development/developer_instruction.md
new file mode 100644
index 00000000000..8c5c0214c3e
--- /dev/null
+++ b/docs/fa/development/developer_instruction.md
@@ -0,0 +1,282 @@
+---
+en_copy: true
+---
+
+Building of ClickHouse is supported on Linux, FreeBSD and Mac OS X.
+
+# If you use Windows {#if-you-use-windows}
+
+If you use Windows, you need to create a virtual machine with Ubuntu. To start working with a virtual machine please install VirtualBox. You can download Ubuntu from the website: https://www.ubuntu.com/\#download. Please create a virtual machine from the downloaded image (you should reserve at least 4GB of RAM for it). To run a command-line terminal in Ubuntu, please locate a program containing the word “terminal” in its name (gnome-terminal, konsole etc.) or just press Ctrl+Alt+T.
+
+# If you use a 32-bit system {#if-you-use-a-32-bit-system}
+
+ClickHouse cannot work or build on a 32-bit system. You should acquire access to a 64-bit system and you can continue reading.
+
+# Creating a repository on GitHub {#creating-a-repository-on-github}
+
+To start working with ClickHouse repository you will need a GitHub account.
+
+You probably already have one, but if you don’t, please register at https://github.com. In case you do not have SSH keys, you should generate them and then upload them on GitHub. It is required for sending over your patches. It is also possible to use the same SSH keys that you use with any other SSH servers - probably you already have those.
+
+Create a fork of ClickHouse repository. To do that please click on the “fork” button in the upper right corner at https://github.com/ClickHouse/ClickHouse. It will fork your own copy of ClickHouse/ClickHouse to your account.
+
+The development process consists of first committing the intended changes into your fork of ClickHouse and then creating a “pull request” for these changes to be accepted into the main repository (ClickHouse/ClickHouse).
+
+To work with git repositories, please install `git`.
+
+To do that in Ubuntu you would run in the command line terminal:
+
+ sudo apt update
+ sudo apt install git
+
+A brief manual on using Git can be found here: https://services.github.com/on-demand/downloads/github-git-cheat-sheet.pdf.
+For a detailed manual on Git see https://git-scm.com/book/en/v2.
+
+# Cloning a repository to your development machine {#cloning-a-repository-to-your-development-machine}
+
+Next, you need to download the source files onto your working machine. This is called “to clone a repository” because it creates a local copy of the repository on your working machine.
+
+In the command line terminal run:
+
+ git clone --recursive git@guthub.com:your_github_username/ClickHouse.git
+ cd ClickHouse
+
+Note: please, substitute *your\_github\_username* with what is appropriate!
+
+This command will create a directory `ClickHouse` containing the working copy of the project.
+
+It is important that the path to the working directory contains no whitespaces as it may lead to problems with running the build system.
+
+Please note that ClickHouse repository uses `submodules`. That is what the references to additional repositories are called (i.e. external libraries on which the project depends). It means that when cloning the repository you need to specify the `--recursive` flag as in the example above. If the repository has been cloned without submodules, to download them you need to run the following:
+
+ git submodule init
+ git submodule update
+
+You can check the status with the command: `git submodule status`.
+
+If you get the following error message:
+
+ Permission denied (publickey).
+ fatal: Could not read from remote repository.
+
+ Please make sure you have the correct access rights
+ and the repository exists.
+
+It generally means that the SSH keys for connecting to GitHub are missing. These keys are normally located in `~/.ssh`. For SSH keys to be accepted you need to upload them in the settings section of GitHub UI.
+
+You can also clone the repository via https protocol:
+
+ git clone https://github.com/ClickHouse/ClickHouse.git
+
+This, however, will not let you send your changes to the server. You can still use it temporarily and add the SSH keys later replacing the remote address of the repository with `git remote` command.
+
+You can also add original ClickHouse repo’s address to your local repository to pull updates from there:
+
+ git remote add upstream git@github.com:ClickHouse/ClickHouse.git
+
+After successfully running this command you will be able to pull updates from the main ClickHouse repo by running `git pull upstream master`.
+
+## Working with submodules {#working-with-submodules}
+
+Working with submodules in git could be painful. Next commands will help to manage it:
+
+ # ! each command accepts --recursive
+ # Update remote URLs for submodules. Barely rare case
+ git submodule sync
+ # Add new submodules
+ git submodule init
+ # Update existing submodules to the current state
+ git submodule update
+ # Two last commands could be merged together
+ git submodule update --init
+
+The next commands would help you to reset all submodules to the initial state (!WARNING! - any changes inside will be deleted):
+
+ # Synchronizes submodules' remote URL with .gitmodules
+ git submodule sync --recursive
+ # Update the registered submodules with initialize not yet initialized
+ git submodule update --init --recursive
+ # Reset all changes done after HEAD
+ git submodule foreach git reset --hard
+ # Clean files from .gitignore
+ git submodule foreach git clean -xfd
+ # Repeat last 4 commands for all submodule
+ git submodule foreach git submodule sync --recursive
+ git submodule foreach git submodule update --init --recursive
+ git submodule foreach git submodule foreach git reset --hard
+ git submodule foreach git submodule foreach git clean -xfd
+
+# Build System {#build-system}
+
+ClickHouse uses CMake and Ninja for building.
+
+CMake - a meta-build system that can generate Ninja files (build tasks).
+Ninja - a smaller build system with a focus on the speed used to execute those cmake generated tasks.
+
+To install on Ubuntu, Debian or Mint run `sudo apt install cmake ninja-build`.
+
+On CentOS, RedHat run `sudo yum install cmake ninja-build`.
+
+If you use Arch or Gentoo, you probably know it yourself how to install CMake.
+
+For installing CMake and Ninja on Mac OS X first install Homebrew and then install everything else via brew:
+
+ /usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
+ brew install cmake ninja
+
+Next, check the version of CMake: `cmake --version`. If it is below 3.3, you should install a newer version from the website: https://cmake.org/download/.
+
+# Optional External Libraries {#optional-external-libraries}
+
+ClickHouse uses several external libraries for building. All of them do not need to be installed separately as they are built together with ClickHouse from the sources located in the submodules. You can check the list in `contrib`.
+
+# C++ Compiler {#c-compiler}
+
+Compilers GCC starting from version 9 and Clang version 8 or above are supported for building ClickHouse.
+
+Official Yandex builds currently use GCC because it generates machine code of slightly better performance (yielding a difference of up to several percent according to our benchmarks). And Clang is more convenient for development usually. Though, our continuous integration (CI) platform runs checks for about a dozen of build combinations.
+
+To install GCC on Ubuntu run: `sudo apt install gcc g++`
+
+Check the version of gcc: `gcc --version`. If it is below 9, then follow the instruction here: https://clickhouse.tech/docs/en/development/build/\#install-gcc-9.
+
+Mac OS X build is supported only for Clang. Just run `brew install llvm`
+
+If you decide to use Clang, you can also install `libc++` and `lld`, if you know what it is. Using `ccache` is also recommended.
+
+# The Building process {#the-building-process}
+
+Now that you are ready to build ClickHouse we recommend you to create a separate directory `build` inside `ClickHouse` that will contain all of the build artefacts:
+
+ mkdir build
+ cd build
+
+You can have several different directories (build\_release, build\_debug, etc.) for different types of build.
+
+While inside the `build` directory, configure your build by running CMake. Before the first run, you need to define environment variables that specify compiler (version 9 gcc compiler in this example).
+
+Linux:
+
+ export CC=gcc-9 CXX=g++-9
+ cmake ..
+
+Mac OS X:
+
+ export CC=clang CXX=clang++
+ cmake ..
+
+The `CC` variable specifies the compiler for C (short for C Compiler), and `CXX` variable instructs which C++ compiler is to be used for building.
+
+For a faster build, you can resort to the `debug` build type - a build with no optimizations. For that supply the following parameter `-D CMAKE_BUILD_TYPE=Debug`:
+
+ cmake -D CMAKE_BUILD_TYPE=Debug ..
+
+You can change the type of build by running this command in the `build` directory.
+
+Run ninja to build:
+
+ ninja clickhouse-server clickhouse-client
+
+Only the required binaries are going to be built in this example.
+
+If you require to build all the binaries (utilities and tests), you should run ninja with no parameters:
+
+ ninja
+
+Full build requires about 30GB of free disk space or 15GB to build the main binaries.
+
+When a large amount of RAM is available on build machine you should limit the number of build tasks run in parallel with `-j` param:
+
+ ninja -j 1 clickhouse-server clickhouse-client
+
+On machines with 4GB of RAM, it is recommended to specify 1, for 8GB of RAM `-j 2` is recommended.
+
+If you get the message: `ninja: error: loading 'build.ninja': No such file or directory`, it means that generating a build configuration has failed and you need to inspect the message above.
+
+Upon the successful start of the building process, you’ll see the build progress - the number of processed tasks and the total number of tasks.
+
+While building messages about protobuf files in libhdfs2 library like `libprotobuf WARNING` may show up. They affect nothing and are safe to be ignored.
+
+Upon successful build you get an executable file `ClickHouse//programs/clickhouse`:
+
+ ls -l programs/clickhouse
+
+# Running the built executable of ClickHouse {#running-the-built-executable-of-clickhouse}
+
+To run the server under the current user you need to navigate to `ClickHouse/programs/server/` (located outside of `build`) and run:
+
+ ../../../build/programs/clickhouse server
+
+In this case, ClickHouse will use config files located in the current directory. You can run `clickhouse server` from any directory specifying the path to a config file as a command-line parameter `--config-file`.
+
+To connect to ClickHouse with clickhouse-client in another terminal navigate to `ClickHouse/build/programs/` and run `clickhouse client`.
+
+If you get `Connection refused` message on Mac OS X or FreeBSD, try specifying host address 127.0.0.1:
+
+ clickhouse client --host 127.0.0.1
+
+You can replace the production version of ClickHouse binary installed in your system with your custom-built ClickHouse binary. To do that install ClickHouse on your machine following the instructions from the official website. Next, run the following:
+
+ sudo service clickhouse-server stop
+ sudo cp ClickHouse/build/programs/clickhouse /usr/bin/
+ sudo service clickhouse-server start
+
+Note that `clickhouse-client`, `clickhouse-server` and others are symlinks to the commonly shared `clickhouse` binary.
+
+You can also run your custom-built ClickHouse binary with the config file from the ClickHouse package installed on your system:
+
+ sudo service clickhouse-server stop
+ sudo -u clickhouse ClickHouse/build/programs/clickhouse server --config-file /etc/clickhouse-server/config.xml
+
+# IDE (Integrated Development Environment) {#ide-integrated-development-environment}
+
+If you do not know which IDE to use, we recommend that you use CLion. CLion is commercial software, but it offers 30 days free trial period. It is also free of charge for students. CLion can be used both on Linux and on Mac OS X.
+
+KDevelop and QTCreator are other great alternatives of an IDE for developing ClickHouse. KDevelop comes in as a very handy IDE although unstable. If KDevelop crashes after a while upon opening project, you should click “Stop All” button as soon as it has opened the list of project’s files. After doing so KDevelop should be fine to work with.
+
+As simple code editors, you can use Sublime Text or Visual Studio Code, or Kate (all of which are available on Linux).
+
+Just in case, it is worth mentioning that CLion creates `build` path on its own, it also on its own selects `debug` for build type, for configuration it uses a version of CMake that is defined in CLion and not the one installed by you, and finally, CLion will use `make` to run build tasks instead of `ninja`. This is normal behaviour, just keep that in mind to avoid confusion.
+
+# Writing Code {#writing-code}
+
+The description of ClickHouse architecture can be found here: https://clickhouse.tech/docs/en/development/architecture/
+
+The Code Style Guide: https://clickhouse.tech/docs/en/development/style/
+
+Writing tests: https://clickhouse.tech/docs/en/development/tests/
+
+List of tasks: https://github.com/ClickHouse/ClickHouse/blob/master/testsructions/easy\_tasks\_sorted\_en.md
+
+# Test Data {#test-data}
+
+Developing ClickHouse often requires loading realistic datasets. It is particularly important for performance testing. We have a specially prepared set of anonymized data from Yandex.Metrica. It requires additionally some 3GB of free disk space. Note that this data is not required to accomplish most of the development tasks.
+
+ sudo apt install wget xz-utils
+
+ wget https://clickhouse-datasets.s3.yandex.net/hits/tsv/hits_v1.tsv.xz
+ wget https://clickhouse-datasets.s3.yandex.net/visits/tsv/visits_v1.tsv.xz
+
+ xz -v -d hits_v1.tsv.xz
+ xz -v -d visits_v1.tsv.xz
+
+ clickhouse-client
+
+ CREATE TABLE test.hits ( WatchID UInt64, JavaEnable UInt8, Title String, GoodEvent Int16, EventTime DateTime, EventDate Date, CounterID UInt32, ClientIP UInt32, ClientIP6 FixedString(16), RegionID UInt32, UserID UInt64, CounterClass Int8, OS UInt8, UserAgent UInt8, URL String, Referer String, URLDomain String, RefererDomain String, Refresh UInt8, IsRobot UInt8, RefererCategories Array(UInt16), URLCategories Array(UInt16), URLRegions Array(UInt32), RefererRegions Array(UInt32), ResolutionWidth UInt16, ResolutionHeight UInt16, ResolutionDepth UInt8, FlashMajor UInt8, FlashMinor UInt8, FlashMinor2 String, NetMajor UInt8, NetMinor UInt8, UserAgentMajor UInt16, UserAgentMinor FixedString(2), CookieEnable UInt8, JavascriptEnable UInt8, IsMobile UInt8, MobilePhone UInt8, MobilePhoneModel String, Params String, IPNetworkID UInt32, TraficSourceID Int8, SearchEngineID UInt16, SearchPhrase String, AdvEngineID UInt8, IsArtifical UInt8, WindowClientWidth UInt16, WindowClientHeight UInt16, ClientTimeZone Int16, ClientEventTime DateTime, SilverlightVersion1 UInt8, SilverlightVersion2 UInt8, SilverlightVersion3 UInt32, SilverlightVersion4 UInt16, PageCharset String, CodeVersion UInt32, IsLink UInt8, IsDownload UInt8, IsNotBounce UInt8, FUniqID UInt64, HID UInt32, IsOldCounter UInt8, IsEvent UInt8, IsParameter UInt8, DontCountHits UInt8, WithHash UInt8, HitColor FixedString(1), UTCEventTime DateTime, Age UInt8, Sex UInt8, Income UInt8, Interests UInt16, Robotness UInt8, GeneralInterests Array(UInt16), RemoteIP UInt32, RemoteIP6 FixedString(16), WindowName Int32, OpenerName Int32, HistoryLength Int16, BrowserLanguage FixedString(2), BrowserCountry FixedString(2), SocialNetwork String, SocialAction String, HTTPError UInt16, SendTiming Int32, DNSTiming Int32, ConnectTiming Int32, ResponseStartTiming Int32, ResponseEndTiming Int32, FetchTiming Int32, RedirectTiming Int32, DOMInteractiveTiming Int32, DOMContentLoadedTiming Int32, DOMCompleteTiming Int32, LoadEventStartTiming Int32, LoadEventEndTiming Int32, NSToDOMContentLoadedTiming Int32, FirstPaintTiming Int32, RedirectCount Int8, SocialSourceNetworkID UInt8, SocialSourcePage String, ParamPrice Int64, ParamOrderID String, ParamCurrency FixedString(3), ParamCurrencyID UInt16, GoalsReached Array(UInt32), OpenstatServiceName String, OpenstatCampaignID String, OpenstatAdID String, OpenstatSourceID String, UTMSource String, UTMMedium String, UTMCampaign String, UTMContent String, UTMTerm String, FromTag String, HasGCLID UInt8, RefererHash UInt64, URLHash UInt64, CLID UInt32, YCLID UInt64, ShareService String, ShareURL String, ShareTitle String, `ParsedParams.Key1` Array(String), `ParsedParams.Key2` Array(String), `ParsedParams.Key3` Array(String), `ParsedParams.Key4` Array(String), `ParsedParams.Key5` Array(String), `ParsedParams.ValueDouble` Array(Float64), IslandID FixedString(16), RequestNum UInt32, RequestTry UInt8) ENGINE = MergeTree PARTITION BY toYYYYMM(EventDate) SAMPLE BY intHash32(UserID) ORDER BY (CounterID, EventDate, intHash32(UserID), EventTime);
+
+ CREATE TABLE test.visits ( CounterID UInt32, StartDate Date, Sign Int8, IsNew UInt8, VisitID UInt64, UserID UInt64, StartTime DateTime, Duration UInt32, UTCStartTime DateTime, PageViews Int32, Hits Int32, IsBounce UInt8, Referer String, StartURL String, RefererDomain String, StartURLDomain String, EndURL String, LinkURL String, IsDownload UInt8, TraficSourceID Int8, SearchEngineID UInt16, SearchPhrase String, AdvEngineID UInt8, PlaceID Int32, RefererCategories Array(UInt16), URLCategories Array(UInt16), URLRegions Array(UInt32), RefererRegions Array(UInt32), IsYandex UInt8, GoalReachesDepth Int32, GoalReachesURL Int32, GoalReachesAny Int32, SocialSourceNetworkID UInt8, SocialSourcePage String, MobilePhoneModel String, ClientEventTime DateTime, RegionID UInt32, ClientIP UInt32, ClientIP6 FixedString(16), RemoteIP UInt32, RemoteIP6 FixedString(16), IPNetworkID UInt32, SilverlightVersion3 UInt32, CodeVersion UInt32, ResolutionWidth UInt16, ResolutionHeight UInt16, UserAgentMajor UInt16, UserAgentMinor UInt16, WindowClientWidth UInt16, WindowClientHeight UInt16, SilverlightVersion2 UInt8, SilverlightVersion4 UInt16, FlashVersion3 UInt16, FlashVersion4 UInt16, ClientTimeZone Int16, OS UInt8, UserAgent UInt8, ResolutionDepth UInt8, FlashMajor UInt8, FlashMinor UInt8, NetMajor UInt8, NetMinor UInt8, MobilePhone UInt8, SilverlightVersion1 UInt8, Age UInt8, Sex UInt8, Income UInt8, JavaEnable UInt8, CookieEnable UInt8, JavascriptEnable UInt8, IsMobile UInt8, BrowserLanguage UInt16, BrowserCountry UInt16, Interests UInt16, Robotness UInt8, GeneralInterests Array(UInt16), Params Array(String), `Goals.ID` Array(UInt32), `Goals.Serial` Array(UInt32), `Goals.EventTime` Array(DateTime), `Goals.Price` Array(Int64), `Goals.OrderID` Array(String), `Goals.CurrencyID` Array(UInt32), WatchIDs Array(UInt64), ParamSumPrice Int64, ParamCurrency FixedString(3), ParamCurrencyID UInt16, ClickLogID UInt64, ClickEventID Int32, ClickGoodEvent Int32, ClickEventTime DateTime, ClickPriorityID Int32, ClickPhraseID Int32, ClickPageID Int32, ClickPlaceID Int32, ClickTypeID Int32, ClickResourceID Int32, ClickCost UInt32, ClickClientIP UInt32, ClickDomainID UInt32, ClickURL String, ClickAttempt UInt8, ClickOrderID UInt32, ClickBannerID UInt32, ClickMarketCategoryID UInt32, ClickMarketPP UInt32, ClickMarketCategoryName String, ClickMarketPPName String, ClickAWAPSCampaignName String, ClickPageName String, ClickTargetType UInt16, ClickTargetPhraseID UInt64, ClickContextType UInt8, ClickSelectType Int8, ClickOptions String, ClickGroupBannerID Int32, OpenstatServiceName String, OpenstatCampaignID String, OpenstatAdID String, OpenstatSourceID String, UTMSource String, UTMMedium String, UTMCampaign String, UTMContent String, UTMTerm String, FromTag String, HasGCLID UInt8, FirstVisit DateTime, PredLastVisit Date, LastVisit Date, TotalVisits UInt32, `TraficSource.ID` Array(Int8), `TraficSource.SearchEngineID` Array(UInt16), `TraficSource.AdvEngineID` Array(UInt8), `TraficSource.PlaceID` Array(UInt16), `TraficSource.SocialSourceNetworkID` Array(UInt8), `TraficSource.Domain` Array(String), `TraficSource.SearchPhrase` Array(String), `TraficSource.SocialSourcePage` Array(String), Attendance FixedString(16), CLID UInt32, YCLID UInt64, NormalizedRefererHash UInt64, SearchPhraseHash UInt64, RefererDomainHash UInt64, NormalizedStartURLHash UInt64, StartURLDomainHash UInt64, NormalizedEndURLHash UInt64, TopLevelDomain UInt64, URLScheme UInt64, OpenstatServiceNameHash UInt64, OpenstatCampaignIDHash UInt64, OpenstatAdIDHash UInt64, OpenstatSourceIDHash UInt64, UTMSourceHash UInt64, UTMMediumHash UInt64, UTMCampaignHash UInt64, UTMContentHash UInt64, UTMTermHash UInt64, FromHash UInt64, WebVisorEnabled UInt8, WebVisorActivity UInt32, `ParsedParams.Key1` Array(String), `ParsedParams.Key2` Array(String), `ParsedParams.Key3` Array(String), `ParsedParams.Key4` Array(String), `ParsedParams.Key5` Array(String), `ParsedParams.ValueDouble` Array(Float64), `Market.Type` Array(UInt8), `Market.GoalID` Array(UInt32), `Market.OrderID` Array(String), `Market.OrderPrice` Array(Int64), `Market.PP` Array(UInt32), `Market.DirectPlaceID` Array(UInt32), `Market.DirectOrderID` Array(UInt32), `Market.DirectBannerID` Array(UInt32), `Market.GoodID` Array(String), `Market.GoodName` Array(String), `Market.GoodQuantity` Array(Int32), `Market.GoodPrice` Array(Int64), IslandID FixedString(16)) ENGINE = CollapsingMergeTree(Sign) PARTITION BY toYYYYMM(StartDate) SAMPLE BY intHash32(UserID) ORDER BY (CounterID, StartDate, intHash32(UserID), VisitID);
+
+ clickhouse-client --max_insert_block_size 100000 --query "INSERT INTO test.hits FORMAT TSV" < hits_v1.tsv
+ clickhouse-client --max_insert_block_size 100000 --query "INSERT INTO test.visits FORMAT TSV" < visits_v1.tsv
+
+# Creating Pull Request {#creating-pull-request}
+
+Navigate to your fork repository in GitHub’s UI. If you have been developing in a branch, you need to select that branch. There will be a “Pull request” button located on the screen. In essence, this means “create a request for accepting my changes into the main repository”.
+
+A pull request can be created even if the work is not completed yet. In this case please put the word “WIP” (work in progress) at the beginning of the title, it can be changed later. This is useful for cooperative reviewing and discussion of changes as well as for running all of the available tests. It is important that you provide a brief description of your changes, it will later be used for generating release changelogs.
+
+Testing will commence as soon as Yandex employees label your PR with a tag “can be tested”. The results of some first checks (e.g. code style) will come in within several minutes. Build check results will arrive within half an hour. And the main set of tests will report itself within an hour.
+
+The system will prepare ClickHouse binary builds for your pull request individually. To retrieve these builds click the “Details” link next to “ClickHouse build check” entry in the list of checks. There you will find direct links to the built .deb packages of ClickHouse which you can deploy even on your production servers (if you have no fear).
+
+Most probably some of the builds will fail at first times. This is due to the fact that we check builds both with gcc as well as with clang, with almost all of existing warnings (always with the `-Werror` flag) enabled for clang. On that same page, you can find all of the build logs so that you do not have to build ClickHouse in all of the possible ways.
diff --git a/docs/fa/development/index.md b/docs/fa/development/index.md
deleted file mode 120000
index 1e2ad97dcc5..00000000000
--- a/docs/fa/development/index.md
+++ /dev/null
@@ -1 +0,0 @@
-../../en/development/index.md
\ No newline at end of file
diff --git a/docs/fa/development/index.md b/docs/fa/development/index.md
new file mode 100644
index 00000000000..727e89ca891
--- /dev/null
+++ b/docs/fa/development/index.md
@@ -0,0 +1,7 @@
+---
+en_copy: true
+---
+
+# ClickHouse Development {#clickhouse-development}
+
+[Original article](https://clickhouse.tech/docs/en/development/)
diff --git a/docs/fa/development/style.md b/docs/fa/development/style.md
deleted file mode 120000
index c1bbf11f421..00000000000
--- a/docs/fa/development/style.md
+++ /dev/null
@@ -1 +0,0 @@
-../../en/development/style.md
\ No newline at end of file
diff --git a/docs/fa/development/style.md b/docs/fa/development/style.md
new file mode 100644
index 00000000000..5df099ecd52
--- /dev/null
+++ b/docs/fa/development/style.md
@@ -0,0 +1,838 @@
+---
+en_copy: true
+---
+
+# How to Write C++ Code {#how-to-write-c-code}
+
+## General Recommendations {#general-recommendations}
+
+**1.** The following are recommendations, not requirements.
+
+**2.** If you are editing code, it makes sense to follow the formatting of the existing code.
+
+**3.** Code style is needed for consistency. Consistency makes it easier to read the code, and it also makes it easier to search the code.
+
+**4.** Many of the rules do not have logical reasons; they are dictated by established practices.
+
+## Formatting {#formatting}
+
+**1.** Most of the formatting will be done automatically by `clang-format`.
+
+**2.** Indents are 4 spaces. Configure your development environment so that a tab adds four spaces.
+
+**3.** Opening and closing curly brackets must be on a separate line.
+
+``` cpp
+inline void readBoolText(bool & x, ReadBuffer & buf)
+{
+ char tmp = '0';
+ readChar(tmp, buf);
+ x = tmp != '0';
+}
+```
+
+**4.** If the entire function body is a single `statement`, it can be placed on a single line. Place spaces around curly braces (besides the space at the end of the line).
+
+``` cpp
+inline size_t mask() const { return buf_size() - 1; }
+inline size_t place(HashValue x) const { return x & mask(); }
+```
+
+**5.** For functions. Don’t put spaces around brackets.
+
+``` cpp
+void reinsert(const Value & x)
+```
+
+``` cpp
+memcpy(&buf[place_value], &x, sizeof(x));
+```
+
+**6.** In `if`, `for`, `while` and other expressions, a space is inserted in front of the opening bracket (as opposed to function calls).
+
+``` cpp
+for (size_t i = 0; i < rows; i += storage.index_granularity)
+```
+
+**7.** Add spaces around binary operators (`+`, `-`, `*`, `/`, `%`, …) and the ternary operator `?:`.
+
+``` cpp
+UInt16 year = (s[0] - '0') * 1000 + (s[1] - '0') * 100 + (s[2] - '0') * 10 + (s[3] - '0');
+UInt8 month = (s[5] - '0') * 10 + (s[6] - '0');
+UInt8 day = (s[8] - '0') * 10 + (s[9] - '0');
+```
+
+**8.** If a line feed is entered, put the operator on a new line and increase the indent before it.
+
+``` cpp
+if (elapsed_ns)
+ message << " ("
+ << rows_read_on_server * 1000000000 / elapsed_ns << " rows/s., "
+ << bytes_read_on_server * 1000.0 / elapsed_ns << " MB/s.) ";
+```
+
+**9.** You can use spaces for alignment within a line, if desired.
+
+``` cpp
+dst.ClickLogID = click.LogID;
+dst.ClickEventID = click.EventID;
+dst.ClickGoodEvent = click.GoodEvent;
+```
+
+**10.** Don’t use spaces around the operators `.`, `->`.
+
+If necessary, the operator can be wrapped to the next line. In this case, the offset in front of it is increased.
+
+**11.** Do not use a space to separate unary operators (`--`, `++`, `*`, `&`, …) from the argument.
+
+**12.** Put a space after a comma, but not before it. The same rule goes for a semicolon inside a `for` expression.
+
+**13.** Do not use spaces to separate the `[]` operator.
+
+**14.** In a `template <...>` expression, use a space between `template` and `<`; no spaces after `<` or before `>`.
+
+``` cpp
+template
+struct AggregatedStatElement
+{}
+```
+
+**15.** In classes and structures, write `public`, `private`, and `protected` on the same level as `class/struct`, and indent the rest of the code.
+
+``` cpp
+template
+class MultiVersion
+{
+public:
+ /// Version of object for usage. shared_ptr manage lifetime of version.
+ using Version = std::shared_ptr;
+ ...
+}
+```
+
+**16.** If the same `namespace` is used for the entire file, and there isn’t anything else significant, an offset is not necessary inside `namespace`.
+
+**17.** If the block for an `if`, `for`, `while`, or other expression consists of a single `statement`, the curly brackets are optional. Place the `statement` on a separate line, instead. This rule is also valid for nested `if`, `for`, `while`, …
+
+But if the inner `statement` contains curly brackets or `else`, the external block should be written in curly brackets.
+
+``` cpp
+/// Finish write.
+for (auto & stream : streams)
+ stream.second->finalize();
+```
+
+**18.** There shouldn’t be any spaces at the ends of lines.
+
+**19.** Source files are UTF-8 encoded.
+
+**20.** Non-ASCII characters can be used in string literals.
+
+``` cpp
+<< ", " << (timer.elapsed() / chunks_stats.hits) << " μsec/hit.";
+```
+
+**21.** Do not write multiple expressions in a single line.
+
+**22.** Group sections of code inside functions and separate them with no more than one empty line.
+
+**23.** Separate functions, classes, and so on with one or two empty lines.
+
+**24.** `A const` (related to a value) must be written before the type name.
+
+``` cpp
+//correct
+const char * pos
+const std::string & s
+//incorrect
+char const * pos
+```
+
+**25.** When declaring a pointer or reference, the `*` and `&` symbols should be separated by spaces on both sides.
+
+``` cpp
+//correct
+const char * pos
+//incorrect
+const char* pos
+const char *pos
+```
+
+**26.** When using template types, alias them with the `using` keyword (except in the simplest cases).
+
+In other words, the template parameters are specified only in `using` and aren’t repeated in the code.
+
+`using` can be declared locally, such as inside a function.
+
+``` cpp
+//correct
+using FileStreams = std::map>;
+FileStreams streams;
+//incorrect
+std::map> streams;
+```
+
+**27.** Do not declare several variables of different types in one statement.
+
+``` cpp
+//incorrect
+int x, *y;
+```
+
+**28.** Do not use C-style casts.
+
+``` cpp
+//incorrect
+std::cerr << (int)c <<; std::endl;
+//correct
+std::cerr << static_cast(c) << std::endl;
+```
+
+**29.** In classes and structs, group members and functions separately inside each visibility scope.
+
+**30.** For small classes and structs, it is not necessary to separate the method declaration from the implementation.
+
+The same is true for small methods in any classes or structs.
+
+For templated classes and structs, don’t separate the method declarations from the implementation (because otherwise they must be defined in the same translation unit).
+
+**31.** You can wrap lines at 140 characters, instead of 80.
+
+**32.** Always use the prefix increment/decrement operators if postfix is not required.
+
+``` cpp
+for (Names::const_iterator it = column_names.begin(); it != column_names.end(); ++it)
+```
+
+## Comments {#comments}
+
+**1.** Be sure to add comments for all non-trivial parts of code.
+
+This is very important. Writing the comment might help you realize that the code isn’t necessary, or that it is designed wrong.
+
+``` cpp
+/** Part of piece of memory, that can be used.
+ * For example, if internal_buffer is 1MB, and there was only 10 bytes loaded to buffer from file for reading,
+ * then working_buffer will have size of only 10 bytes
+ * (working_buffer.end() will point to position right after those 10 bytes available for read).
+ */
+```
+
+**2.** Comments can be as detailed as necessary.
+
+**3.** Place comments before the code they describe. In rare cases, comments can come after the code, on the same line.
+
+``` cpp
+/** Parses and executes the query.
+*/
+void executeQuery(
+ ReadBuffer & istr, /// Where to read the query from (and data for INSERT, if applicable)
+ WriteBuffer & ostr, /// Where to write the result
+ Context & context, /// DB, tables, data types, engines, functions, aggregate functions...
+ BlockInputStreamPtr & query_plan, /// Here could be written the description on how query was executed
+ QueryProcessingStage::Enum stage = QueryProcessingStage::Complete /// Up to which stage process the SELECT query
+ )
+```
+
+**4.** Comments should be written in English only.
+
+**5.** If you are writing a library, include detailed comments explaining it in the main header file.
+
+**6.** Do not add comments that do not provide additional information. In particular, do not leave empty comments like this:
+
+``` cpp
+/*
+* Procedure Name:
+* Original procedure name:
+* Author:
+* Date of creation:
+* Dates of modification:
+* Modification authors:
+* Original file name:
+* Purpose:
+* Intent:
+* Designation:
+* Classes used:
+* Constants:
+* Local variables:
+* Parameters:
+* Date of creation:
+* Purpose:
+*/
+```
+
+The example is borrowed from the resource http://home.tamk.fi/~jaalto/course/coding-style/doc/unmaintainable-code/.
+
+**7.** Do not write garbage comments (author, creation date ..) at the beginning of each file.
+
+**8.** Single-line comments begin with three slashes: `///` and multi-line comments begin with `/**`. These comments are considered “documentation”.
+
+Note: You can use Doxygen to generate documentation from these comments. But Doxygen is not generally used because it is more convenient to navigate the code in the IDE.
+
+**9.** Multi-line comments must not have empty lines at the beginning and end (except the line that closes a multi-line comment).
+
+**10.** For commenting out code, use basic comments, not “documenting” comments.
+
+**11.** Delete the commented out parts of the code before committing.
+
+**12.** Do not use profanity in comments or code.
+
+**13.** Do not use uppercase letters. Do not use excessive punctuation.
+
+``` cpp
+/// WHAT THE FAIL???
+```
+
+**14.** Do not use comments to make delimeters.
+
+``` cpp
+///******************************************************
+```
+
+**15.** Do not start discussions in comments.
+
+``` cpp
+/// Why did you do this stuff?
+```
+
+**16.** There’s no need to write a comment at the end of a block describing what it was about.
+
+``` cpp
+/// for
+```
+
+## Names {#names}
+
+**1.** Use lowercase letters with underscores in the names of variables and class members.
+
+``` cpp
+size_t max_block_size;
+```
+
+**2.** For the names of functions (methods), use camelCase beginning with a lowercase letter.
+
+``` cpp
+std::string getName() const override { return "Memory"; }
+```
+
+**3.** For the names of classes (structs), use CamelCase beginning with an uppercase letter. Prefixes other than I are not used for interfaces.
+
+``` cpp
+class StorageMemory : public IStorage
+```
+
+**4.** `using` are named the same way as classes, or with `_t` on the end.
+
+**5.** Names of template type arguments: in simple cases, use `T`; `T`, `U`; `T1`, `T2`.
+
+For more complex cases, either follow the rules for class names, or add the prefix `T`.
+
+``` cpp
+template
+struct AggregatedStatElement
+```
+
+**6.** Names of template constant arguments: either follow the rules for variable names, or use `N` in simple cases.
+
+``` cpp
+template
+struct ExtractDomain
+```
+
+**7.** For abstract classes (interfaces) you can add the `I` prefix.
+
+``` cpp
+class IBlockInputStream
+```
+
+**8.** If you use a variable locally, you can use the short name.
+
+In all other cases, use a name that describes the meaning.
+
+``` cpp
+bool info_successfully_loaded = false;
+```
+
+**9.** Names of `define`s and global constants use ALL\_CAPS with underscores.
+
+``` cpp
+#define MAX_SRC_TABLE_NAMES_TO_STORE 1000
+```
+
+**10.** File names should use the same style as their contents.
+
+If a file contains a single class, name the file the same way as the class (CamelCase).
+
+If the file contains a single function, name the file the same way as the function (camelCase).
+
+**11.** If the name contains an abbreviation, then:
+
+- For variable names, the abbreviation should use lowercase letters `mysql_connection` (not `mySQL_connection`).
+- For names of classes and functions, keep the uppercase letters in the abbreviation`MySQLConnection` (not `MySqlConnection`).
+
+**12.** Constructor arguments that are used just to initialize the class members should be named the same way as the class members, but with an underscore at the end.
+
+``` cpp
+FileQueueProcessor(
+ const std::string & path_,
+ const std::string & prefix_,
+ std::shared_ptr handler_)
+ : path(path_),
+ prefix(prefix_),
+ handler(handler_),
+ log(&Logger::get("FileQueueProcessor"))
+{
+}
+```
+
+The underscore suffix can be omitted if the argument is not used in the constructor body.
+
+**13.** There is no difference in the names of local variables and class members (no prefixes required).
+
+``` cpp
+timer (not m_timer)
+```
+
+**14.** For the constants in an `enum`, use CamelCase with a capital letter. ALL\_CAPS is also acceptable. If the `enum` is non-local, use an `enum class`.
+
+``` cpp
+enum class CompressionMethod
+{
+ QuickLZ = 0,
+ LZ4 = 1,
+};
+```
+
+**15.** All names must be in English. Transliteration of Russian words is not allowed.
+
+ not Stroka
+
+**16.** Abbreviations are acceptable if they are well known (when you can easily find the meaning of the abbreviation in Wikipedia or in a search engine).
+
+ `AST`, `SQL`.
+
+ Not `NVDH` (some random letters)
+
+Incomplete words are acceptable if the shortened version is common use.
+
+You can also use an abbreviation if the full name is included next to it in the comments.
+
+**17.** File names with C++ source code must have the `.cpp` extension. Header files must have the `.h` extension.
+
+## How to Write Code {#how-to-write-code}
+
+**1.** Memory management.
+
+Manual memory deallocation (`delete`) can only be used in library code.
+
+In library code, the `delete` operator can only be used in destructors.
+
+In application code, memory must be freed by the object that owns it.
+
+Examples:
+
+- The easiest way is to place an object on the stack, or make it a member of another class.
+- For a large number of small objects, use containers.
+- For automatic deallocation of a small number of objects that reside in the heap, use `shared_ptr/unique_ptr`.
+
+**2.** Resource management.
+
+Use `RAII` and see above.
+
+**3.** Error handling.
+
+Use exceptions. In most cases, you only need to throw an exception, and don’t need to catch it (because of `RAII`).
+
+In offline data processing applications, it’s often acceptable to not catch exceptions.
+
+In servers that handle user requests, it’s usually enough to catch exceptions at the top level of the connection handler.
+
+In thread functions, you should catch and keep all exceptions to rethrow them in the main thread after `join`.
+
+``` cpp
+/// If there weren't any calculations yet, calculate the first block synchronously
+if (!started)
+{
+ calculate();
+ started = true;
+}
+else /// If calculations are already in progress, wait for the result
+ pool.wait();
+
+if (exception)
+ exception->rethrow();
+```
+
+Never hide exceptions without handling. Never just blindly put all exceptions to log.
+
+``` cpp
+//Not correct
+catch (...) {}
+```
+
+If you need to ignore some exceptions, do so only for specific ones and rethrow the rest.
+
+``` cpp
+catch (const DB::Exception & e)
+{
+ if (e.code() == ErrorCodes::UNKNOWN_AGGREGATE_FUNCTION)
+ return nullptr;
+ else
+ throw;
+}
+```
+
+When using functions with response codes or `errno`, always check the result and throw an exception in case of error.
+
+``` cpp
+if (0 != close(fd))
+ throwFromErrno("Cannot close file " + file_name, ErrorCodes::CANNOT_CLOSE_FILE);
+```
+
+`Do not use assert`.
+
+**4.** Exception types.
+
+There is no need to use complex exception hierarchy in application code. The exception text should be understandable to a system administrator.
+
+**5.** Throwing exceptions from destructors.
+
+This is not recommended, but it is allowed.
+
+Use the following options:
+
+- Create a function (`done()` or `finalize()`) that will do all the work in advance that might lead to an exception. If that function was called, there should be no exceptions in the destructor later.
+- Tasks that are too complex (such as sending messages over the network) can be put in separate method that the class user will have to call before destruction.
+- If there is an exception in the destructor, it’s better to log it than to hide it (if the logger is available).
+- In simple applications, it is acceptable to rely on `std::terminate` (for cases of `noexcept` by default in C++11) to handle exceptions.
+
+**6.** Anonymous code blocks.
+
+You can create a separate code block inside a single function in order to make certain variables local, so that the destructors are called when exiting the block.
+
+``` cpp
+Block block = data.in->read();
+
+{
+ std::lock_guard lock(mutex);
+ data.ready = true;
+ data.block = block;
+}
+
+ready_any.set();
+```
+
+**7.** Multithreading.
+
+In offline data processing programs:
+
+- Try to get the best possible performance on a single CPU core. You can then parallelize your code if necessary.
+
+In server applications:
+
+- Use the thread pool to process requests. At this point, we haven’t had any tasks that required userspace context switching.
+
+Fork is not used for parallelization.
+
+**8.** Syncing threads.
+
+Often it is possible to make different threads use different memory cells (even better: different cache lines,) and to not use any thread synchronization (except `joinAll`).
+
+If synchronization is required, in most cases, it is sufficient to use mutex under `lock_guard`.
+
+In other cases use system synchronization primitives. Do not use busy wait.
+
+Atomic operations should be used only in the simplest cases.
+
+Do not try to implement lock-free data structures unless it is your primary area of expertise.
+
+**9.** Pointers vs references.
+
+In most cases, prefer references.
+
+**10.** const.
+
+Use constant references, pointers to constants, `const_iterator`, and const methods.
+
+Consider `const` to be default and use non-`const` only when necessary.
+
+When passing variables by value, using `const` usually does not make sense.
+
+**11.** unsigned.
+
+Use `unsigned` if necessary.
+
+**12.** Numeric types.
+
+Use the types `UInt8`, `UInt16`, `UInt32`, `UInt64`, `Int8`, `Int16`, `Int32`, and `Int64`, as well as `size_t`, `ssize_t`, and `ptrdiff_t`.
+
+Don’t use these types for numbers: `signed/unsigned long`, `long long`, `short`, `signed/unsigned char`, `char`.
+
+**13.** Passing arguments.
+
+Pass complex values by reference (including `std::string`).
+
+If a function captures ownership of an object created in the heap, make the argument type `shared_ptr` or `unique_ptr`.
+
+**14.** Return values.
+
+In most cases, just use `return`. Do not write `[return std::move(res)]{.strike}`.
+
+If the function allocates an object on heap and returns it, use `shared_ptr` or `unique_ptr`.
+
+In rare cases you might need to return the value via an argument. In this case, the argument should be a reference.
+
+``` cpp
+using AggregateFunctionPtr = std::shared_ptr;
+
+/** Allows creating an aggregate function by its name.
+ */
+class AggregateFunctionFactory
+{
+public:
+ AggregateFunctionFactory();
+ AggregateFunctionPtr get(const String & name, const DataTypes & argument_types) const;
+```
+
+**15.** namespace.
+
+There is no need to use a separate `namespace` for application code.
+
+Small libraries don’t need this, either.
+
+For medium to large libraries, put everything in a `namespace`.
+
+In the library’s `.h` file, you can use `namespace detail` to hide implementation details not needed for the application code.
+
+In a `.cpp` file, you can use a `static` or anonymous namespace to hide symbols.
+
+Also, a `namespace` can be used for an `enum` to prevent the corresponding names from falling into an external `namespace` (but it’s better to use an `enum class`).
+
+**16.** Deferred initialization.
+
+If arguments are required for initialization, then you normally shouldn’t write a default constructor.
+
+If later you’ll need to delay initialization, you can add a default constructor that will create an invalid object. Or, for a small number of objects, you can use `shared_ptr/unique_ptr`.
+
+``` cpp
+Loader(DB::Connection * connection_, const std::string & query, size_t max_block_size_);
+
+/// For deferred initialization
+Loader() {}
+```
+
+**17.** Virtual functions.
+
+If the class is not intended for polymorphic use, you do not need to make functions virtual. This also applies to the destructor.
+
+**18.** Encodings.
+
+Use UTF-8 everywhere. Use `std::string`and`char *`. Do not use `std::wstring`and`wchar_t`.
+
+**19.** Logging.
+
+See the examples everywhere in the code.
+
+Before committing, delete all meaningless and debug logging, and any other types of debug output.
+
+Logging in cycles should be avoided, even on the Trace level.
+
+Logs must be readable at any logging level.
+
+Logging should only be used in application code, for the most part.
+
+Log messages must be written in English.
+
+The log should preferably be understandable for the system administrator.
+
+Do not use profanity in the log.
+
+Use UTF-8 encoding in the log. In rare cases you can use non-ASCII characters in the log.
+
+**20.** Input-output.
+
+Don’t use `iostreams` in internal cycles that are critical for application performance (and never use `stringstream`).
+
+Use the `DB/IO` library instead.
+
+**21.** Date and time.
+
+See the `DateLUT` library.
+
+**22.** include.
+
+Always use `#pragma once` instead of include guards.
+
+**23.** using.
+
+`using namespace` is not used. You can use `using` with something specific. But make it local inside a class or function.
+
+**24.** Do not use `trailing return type` for functions unless necessary.
+
+``` cpp
+[auto f() -> void;]{.strike}
+```
+
+**25.** Declaration and initialization of variables.
+
+``` cpp
+//right way
+std::string s = "Hello";
+std::string s{"Hello"};
+
+//wrong way
+auto s = std::string{"Hello"};
+```
+
+**26.** For virtual functions, write `virtual` in the base class, but write `override` instead of `virtual` in descendent classes.
+
+## Unused Features of C++ {#unused-features-of-c}
+
+**1.** Virtual inheritance is not used.
+
+**2.** Exception specifiers from C++03 are not used.
+
+## Platform {#platform}
+
+**1.** We write code for a specific platform.
+
+But other things being equal, cross-platform or portable code is preferred.
+
+**2.** Language: C++17.
+
+**3.** Compiler: `gcc`. At this time (December 2017), the code is compiled using version 7.2. (It can also be compiled using `clang 4`.)
+
+The standard library is used (`libstdc++` or `libc++`).
+
+**4.**OS: Linux Ubuntu, not older than Precise.
+
+**5.**Code is written for x86\_64 CPU architecture.
+
+The CPU instruction set is the minimum supported set among our servers. Currently, it is SSE 4.2.
+
+**6.** Use `-Wall -Wextra -Werror` compilation flags.
+
+**7.** Use static linking with all libraries except those that are difficult to connect to statically (see the output of the `ldd` command).
+
+**8.** Code is developed and debugged with release settings.
+
+## Tools {#tools}
+
+**1.** KDevelop is a good IDE.
+
+**2.** For debugging, use `gdb`, `valgrind` (`memcheck`), `strace`, `-fsanitize=...`, or `tcmalloc_minimal_debug`.
+
+**3.** For profiling, use `Linux Perf`, `valgrind` (`callgrind`), or `strace -cf`.
+
+**4.** Sources are in Git.
+
+**5.** Assembly uses `CMake`.
+
+**6.** Programs are released using `deb` packages.
+
+**7.** Commits to master must not break the build.
+
+Though only selected revisions are considered workable.
+
+**8.** Make commits as often as possible, even if the code is only partially ready.
+
+Use branches for this purpose.
+
+If your code in the `master` branch is not buildable yet, exclude it from the build before the `push`. You’ll need to finish it or remove it within a few days.
+
+**9.** For non-trivial changes, use branches and publish them on the server.
+
+**10.** Unused code is removed from the repository.
+
+## Libraries {#libraries}
+
+**1.** The C++14 standard library is used (experimental extensions are allowed), as well as `boost` and `Poco` frameworks.
+
+**2.** If necessary, you can use any well-known libraries available in the OS package.
+
+If there is a good solution already available, then use it, even if it means you have to install another library.
+
+(But be prepared to remove bad libraries from code.)
+
+**3.** You can install a library that isn’t in the packages, if the packages don’t have what you need or have an outdated version or the wrong type of compilation.
+
+**4.** If the library is small and doesn’t have its own complex build system, put the source files in the `contrib` folder.
+
+**5.** Preference is always given to libraries that are already in use.
+
+## General Recommendations {#general-recommendations-1}
+
+**1.** Write as little code as possible.
+
+**2.** Try the simplest solution.
+
+**3.** Don’t write code until you know how it’s going to work and how the inner loop will function.
+
+**4.** In the simplest cases, use `using` instead of classes or structs.
+
+**5.** If possible, do not write copy constructors, assignment operators, destructors (other than a virtual one, if the class contains at least one virtual function), move constructors or move assignment operators. In other words, the compiler-generated functions must work correctly. You can use `default`.
+
+**6.** Code simplification is encouraged. Reduce the size of your code where possible.
+
+## Additional Recommendations {#additional-recommendations}
+
+**1.** Explicitly specifying `std::` for types from `stddef.h`
+
+is not recommended. In other words, we recommend writing `size_t` instead `std::size_t`, because it’s shorter.
+
+It is acceptable to add `std::`.
+
+**2.** Explicitly specifying `std::` for functions from the standard C library
+
+is not recommended. In other words, write `memcpy` instead of `std::memcpy`.
+
+The reason is that there are similar non-standard functions, such as `memmem`. We do use these functions on occasion. These functions do not exist in `namespace std`.
+
+If you write `std::memcpy` instead of `memcpy` everywhere, then `memmem` without `std::` will look strange.
+
+Nevertheless, you can still use `std::` if you prefer it.
+
+**3.** Using functions from C when the same ones are available in the standard C++ library.
+
+This is acceptable if it is more efficient.
+
+For example, use `memcpy` instead of `std::copy` for copying large chunks of memory.
+
+**4.** Multiline function arguments.
+
+Any of the following wrapping styles are allowed:
+
+``` cpp
+function(
+ T1 x1,
+ T2 x2)
+```
+
+``` cpp
+function(
+ size_t left, size_t right,
+ const & RangesInDataParts ranges,
+ size_t limit)
+```
+
+``` cpp
+function(size_t left, size_t right,
+ const & RangesInDataParts ranges,
+ size_t limit)
+```
+
+``` cpp
+function(size_t left, size_t right,
+ const & RangesInDataParts ranges,
+ size_t limit)
+```
+
+``` cpp
+function(
+ size_t left,
+ size_t right,
+ const & RangesInDataParts ranges,
+ size_t limit)
+```
+
+[Original article](https://clickhouse.tech/docs/en/development/style/)
diff --git a/docs/fa/development/tests.md b/docs/fa/development/tests.md
deleted file mode 120000
index c03d36c3916..00000000000
--- a/docs/fa/development/tests.md
+++ /dev/null
@@ -1 +0,0 @@
-../../en/development/tests.md
\ No newline at end of file
diff --git a/docs/fa/development/tests.md b/docs/fa/development/tests.md
new file mode 100644
index 00000000000..c9181349a2b
--- /dev/null
+++ b/docs/fa/development/tests.md
@@ -0,0 +1,249 @@
+---
+en_copy: true
+---
+
+# ClickHouse Testing {#clickhouse-testing}
+
+## Functional Tests {#functional-tests}
+
+Functional tests are the most simple and convenient to use. Most of ClickHouse features can be tested with functional tests and they are mandatory to use for every change in ClickHouse code that can be tested that way.
+
+Each functional test sends one or multiple queries to the running ClickHouse server and compares the result with reference.
+
+Tests are located in `testsies` directory. There are two subdirectories: `stateless` and `stateful`. Stateless tests run queries without any preloaded test data - they often create small synthetic datasets on the fly, within the test itself. Stateful tests require preloaded test data from Yandex.Metrica and not available to general public. We tend to use only `stateless` tests and avoid adding new `stateful` tests.
+
+Each test can be one of two types: `.sql` and `.sh`. `.sql` test is the simple SQL script that is piped to `clickhouse-client --multiquery --testmode`. `.sh` test is a script that is run by itself.
+
+To run all tests, use `testskhouse-test` tool. Look `--help` for the list of possible options. You can simply run all tests or run subset of tests filtered by substring in test name: `./clickhouse-test substring`.
+
+The most simple way to invoke functional tests is to copy `clickhouse-client` to `/usr/bin/`, run `clickhouse-server` and then run `./clickhouse-test` from its own directory.
+
+To add new test, create a `.sql` or `.sh` file in `testsies/0_stateless` directory, check it manually and then generate `.reference` file in the following way: `clickhouse-client -n --testmode < 00000_test.sql > 00000_test.reference` or `./00000_test.sh > ./00000_test.reference`.
+
+Tests should use (create, drop, etc) only tables in `test` database that is assumed to be created beforehand; also tests can use temporary tables.
+
+If you want to use distributed queries in functional tests, you can leverage `remote` table function with `127.0.0.{1..2}` addresses for the server to query itself; or you can use predefined test clusters in server configuration file like `test_shard_localhost`.
+
+Some tests are marked with `zookeeper`, `shard` or `long` in their names.
+`zookeeper` is for tests that are using ZooKeeper. `shard` is for tests that
+requires server to listen `127.0.0.*`; `distributed` or `global` have the same
+meaning. `long` is for tests that run slightly longer that one second. You can
+disable these groups of tests using `--no-zookeeper`, `--no-shard` and
+`--no-long` options, respectively.
+
+## Known bugs {#known-bugs}
+
+If we know some bugs that can be easily reproduced by functional tests, we place prepared functional tests in `testsies/bugs` directory. These tests will be moved to `teststests_stateless` when bugs are fixed.
+
+## Integration Tests {#integration-tests}
+
+Integration tests allow to test ClickHouse in clustered configuration and ClickHouse interaction with other servers like MySQL, Postgres, MongoDB. They are useful to emulate network splits, packet drops, etc. These tests are run under Docker and create multiple containers with various software.
+
+See `testsgration/README.md` on how to run these tests.
+
+Note that integration of ClickHouse with third-party drivers is not tested. Also we currently don’t have integration tests with our JDBC and ODBC drivers.
+
+## Unit Tests {#unit-tests}
+
+Unit tests are useful when you want to test not the ClickHouse as a whole, but a single isolated library or class. You can enable or disable build of tests with `ENABLE_TESTS` CMake option. Unit tests (and other test programs) are located in `tests` subdirectories across the code. To run unit tests, type `ninja test`. Some tests use `gtest`, but some are just programs that return non-zero exit code on test failure.
+
+It’s not necessarily to have unit tests if the code is already covered by functional tests (and functional tests are usually much more simple to use).
+
+## Performance Tests {#performance-tests}
+
+Performance tests allow to measure and compare performance of some isolated part of ClickHouse on synthetic queries. Tests are located at `tests/performance`. Each test is represented by `.xml` file with description of test case. Tests are run with `clickhouse performance-test` tool (that is embedded in `clickhouse` binary). See `--help` for invocation.
+
+Each test run one or miltiple queries (possibly with combinations of parameters) in a loop with some conditions for stop (like “maximum execution speed is not changing in three seconds”) and measure some metrics about query performance (like “maximum execution speed”). Some tests can contain preconditions on preloaded test dataset.
+
+If you want to improve performance of ClickHouse in some scenario, and if improvements can be observed on simple queries, it is highly recommended to write a performance test. It always makes sense to use `perf top` or other perf tools during your tests.
+
+## Test Tools And Scripts {#test-tools-and-scripts}
+
+Some programs in `tests` directory are not prepared tests, but are test tools. For example, for `Lexer` there is a tool `dbms/Parsers/tests/lexer` that just do tokenization of stdin and writes colorized result to stdout. You can use these kind of tools as a code examples and for exploration and manual testing.
+
+You can also place pair of files `.sh` and `.reference` along with the tool to run it on some predefined input - then script result can be compared to `.reference` file. These kind of tests are not automated.
+
+## Miscellanous Tests {#miscellanous-tests}
+
+There are tests for external dictionaries located at `tests/external_dictionaries` and for machine learned models in `tests/external_models`. These tests are not updated and must be transferred to integration tests.
+
+There is separate test for quorum inserts. This test run ClickHouse cluster on separate servers and emulate various failure cases: network split, packet drop (between ClickHouse nodes, between ClickHouse and ZooKeeper, between ClickHouse server and client, etc.), `kill -9`, `kill -STOP` and `kill -CONT` , like [Jepsen](https://aphyr.com/tags/Jepsen). Then the test checks that all acknowledged inserts was written and all rejected inserts was not.
+
+Quorum test was written by separate team before ClickHouse was open-sourced. This team no longer work with ClickHouse. Test was accidentially written in Java. For these reasons, quorum test must be rewritten and moved to integration tests.
+
+## Manual Testing {#manual-testing}
+
+When you develop a new feature, it is reasonable to also test it manually. You can do it with the following steps:
+
+Build ClickHouse. Run ClickHouse from the terminal: change directory to `programs/clickhouse-server` and run it with `./clickhouse-server`. It will use configuration (`config.xml`, `users.xml` and files within `config.d` and `users.d` directories) from the current directory by default. To connect to ClickHouse server, run `programs/clickhouse-client/clickhouse-client`.
+
+Note that all clickhouse tools (server, client, etc) are just symlinks to a single binary named `clickhouse`. You can find this binary at `programs/clickhouse`. All tools can also be invoked as `clickhouse tool` instead of `clickhouse-tool`.
+
+Alternatively you can install ClickHouse package: either stable release from Yandex repository or you can build package for yourself with `./release` in ClickHouse sources root. Then start the server with `sudo service clickhouse-server start` (or stop to stop the server). Look for logs at `/etc/clickhouse-server/clickhouse-server.log`.
+
+When ClickHouse is already installed on your system, you can build a new `clickhouse` binary and replace the existing binary:
+
+``` bash
+$ sudo service clickhouse-server stop
+$ sudo cp ./clickhouse /usr/bin/
+$ sudo service clickhouse-server start
+```
+
+Also you can stop system clickhouse-server and run your own with the same configuration but with logging to terminal:
+
+``` bash
+$ sudo service clickhouse-server stop
+$ sudo -u clickhouse /usr/bin/clickhouse server --config-file /etc/clickhouse-server/config.xml
+```
+
+Example with gdb:
+
+``` bash
+$ sudo -u clickhouse gdb --args /usr/bin/clickhouse server --config-file /etc/clickhouse-server/config.xml
+```
+
+If the system clickhouse-server is already running and you don’t want to stop it, you can change port numbers in your `config.xml` (or override them in a file in `config.d` directory), provide appropriate data path, and run it.
+
+`clickhouse` binary has almost no dependencies and works across wide range of Linux distributions. To quick and dirty test your changes on a server, you can simply `scp` your fresh built `clickhouse` binary to your server and then run it as in examples above.
+
+## Testing Environment {#testing-environment}
+
+Before publishing release as stable we deploy it on testing environment. Testing environment is a cluster that process 1/39 part of [Yandex.Metrica](https://metrica.yandex.com/) data. We share our testing environment with Yandex.Metrica team. ClickHouse is upgraded without downtime on top of existing data. We look at first that data is processed successfully without lagging from realtime, the replication continue to work and there is no issues visible to Yandex.Metrica team. First check can be done in the following way:
+
+``` sql
+SELECT hostName() AS h, any(version()), any(uptime()), max(UTCEventTime), count() FROM remote('example01-01-{1..3}t', merge, hits) WHERE EventDate >= today() - 2 GROUP BY h ORDER BY h;
+```
+
+In some cases we also deploy to testing environment of our friend teams in Yandex: Market, Cloud, etc. Also we have some hardware servers that are used for development purposes.
+
+## Load Testing {#load-testing}
+
+After deploying to testing environment we run load testing with queries from production cluster. This is done manually.
+
+Make sure you have enabled `query_log` on your production cluster.
+
+Collect query log for a day or more:
+
+``` bash
+$ clickhouse-client --query="SELECT DISTINCT query FROM system.query_log WHERE event_date = today() AND query LIKE '%ym:%' AND query NOT LIKE '%system.query_log%' AND type = 2 AND is_initial_query" > queries.tsv
+```
+
+This is a way complicated example. `type = 2` will filter queries that are executed successfully. `query LIKE '%ym:%'` is to select relevant queries from Yandex.Metrica. `is_initial_query` is to select only queries that are initiated by client, not by ClickHouse itself (as parts of distributed query processing).
+
+`scp` this log to your testing cluster and run it as following:
+
+``` bash
+$ clickhouse benchmark --concurrency 16 < queries.tsv
+```
+
+(probably you also want to specify a `--user`)
+
+Then leave it for a night or weekend and go take a rest.
+
+You should check that `clickhouse-server` doesn’t crash, memory footprint is bounded and performance not degrading over time.
+
+Precise query execution timings are not recorded and not compared due to high variability of queries and environment.
+
+## Build Tests {#build-tests}
+
+Build tests allow to check that build is not broken on various alternative configurations and on some foreign systems. Tests are located at `ci` directory. They run build from source inside Docker, Vagrant, and sometimes with `qemu-user-static` inside Docker. These tests are under development and test runs are not automated.
+
+Motivation:
+
+Normally we release and run all tests on a single variant of ClickHouse build. But there are alternative build variants that are not thoroughly tested. Examples:
+
+- build on FreeBSD;
+- build on Debian with libraries from system packages;
+- build with shared linking of libraries;
+- build on AArch64 platform;
+- build on PowerPc platform.
+
+For example, build with system packages is bad practice, because we cannot guarantee what exact version of packages a system will have. But this is really needed by Debian maintainers. For this reason we at least have to support this variant of build. Another example: shared linking is a common source of trouble, but it is needed for some enthusiasts.
+
+Though we cannot run all tests on all variant of builds, we want to check at least that various build variants are not broken. For this purpose we use build tests.
+
+## Testing For Protocol Compatibility {#testing-for-protocol-compatibility}
+
+When we extend ClickHouse network protocol, we test manually that old clickhouse-client works with new clickhouse-server and new clickhouse-client works with old clickhouse-server (simply by running binaries from corresponding packages).
+
+## Help From The Compiler {#help-from-the-compiler}
+
+Main ClickHouse code (that is located in `dbms` directory) is built with `-Wall -Wextra -Werror` and with some additional enabled warnings. Although these options are not enabled for third-party libraries.
+
+Clang has even more useful warnings - you can look for them with `-Weverything` and pick something to default build.
+
+For production builds, gcc is used (it still generates slightly more efficient code than clang). For development, clang is usually more convenient to use. You can build on your own machine with debug mode (to save battery of your laptop), but please note that compiler is able to generate more warnings with `-O3` due to better control flow and inter-procedure analysis. When building with clang, `libc++` is used instead of `libstdc++` and when building with debug mode, debug version of `libc++` is used that allows to catch more errors at runtime.
+
+## Sanitizers {#sanitizers}
+
+**Address sanitizer**.
+We run functional and integration tests under ASan on per-commit basis.
+
+**Valgrind (Memcheck)**.
+We run functional tests under Valgrind overnight. It takes multiple hours. Currently there is one known false positive in `re2` library, see [this article](https://research.swtch.com/sparse).
+
+**Undefined behaviour sanitizer.**
+We run functional and integration tests under ASan on per-commit basis.
+
+**Thread sanitizer**.
+We run functional tests under TSan on per-commit basis. We still don’t run integration tests under TSan on per-commit basis.
+
+**Memory sanitizer**.
+Currently we still don’t use MSan.
+
+**Debug allocator.**
+Debug version of `jemalloc` is used for debug build.
+
+## Fuzzing {#fuzzing}
+
+We use simple fuzz test to generate random SQL queries and to check that the server doesn’t die. Fuzz testing is performed with Address sanitizer. You can find it in `00746_sql_fuzzy.pl`. This test should be run continuously (overnight and longer).
+
+As of December 2018, we still don’t use isolated fuzz testing of library code.
+
+## Security Audit {#security-audit}
+
+People from Yandex Cloud department do some basic overview of ClickHouse capabilities from the security standpoint.
+
+## Static Analyzers {#static-analyzers}
+
+We run `PVS-Studio` on per-commit basis. We have evaluated `clang-tidy`, `Coverity`, `cppcheck`, `PVS-Studio`, `tscancode`. You will find instructions for usage in `tests/instructions/` directory. Also you can read [the article in russian](https://habr.com/company/yandex/blog/342018/).
+
+If you use `CLion` as an IDE, you can leverage some `clang-tidy` checks out of the box.
+
+## Hardening {#hardening}
+
+`FORTIFY_SOURCE` is used by default. It is almost useless, but still makes sense in rare cases and we don’t disable it.
+
+## Code Style {#code-style}
+
+Code style rules are described [here](https://clickhouse.tech/docs/en/development/style/).
+
+To check for some common style violations, you can use `utils/check-style` script.
+
+To force proper style of your code, you can use `clang-format`. File `.clang-format` is located at the sources root. It mostly corresponding with our actual code style. But it’s not recommended to apply `clang-format` to existing files because it makes formatting worse. You can use `clang-format-diff` tool that you can find in clang source repository.
+
+Alternatively you can try `uncrustify` tool to reformat your code. Configuration is in `uncrustify.cfg` in the sources root. It is less tested than `clang-format`.
+
+`CLion` has its own code formatter that has to be tuned for our code style.
+
+## Metrica B2B Tests {#metrica-b2b-tests}
+
+Each ClickHouse release is tested with Yandex Metrica and AppMetrica engines. Testing and stable versions of ClickHouse are deployed on VMs and run with a small copy of Metrica engine that is processing fixed sample of input data. Then results of two instances of Metrica engine are compared together.
+
+These tests are automated by separate team. Due to high number of moving parts, tests are fail most of the time by completely unrelated reasons, that are very difficult to figure out. Most likely these tests have negative value for us. Nevertheless these tests was proved to be useful in about one or two times out of hundreds.
+
+## Test Coverage {#test-coverage}
+
+As of July 2018 we don’t track test coverage.
+
+## Test Automation {#test-automation}
+
+We run tests with Yandex internal CI and job automation system named “Sandbox”.
+
+Build jobs and tests are run in Sandbox on per commit basis. Resulting packages and test results are published in GitHub and can be downloaded by direct links. Artifacts are stored eternally. When you send a pull request on GitHub, we tag it as “can be tested” and our CI system will build ClickHouse packages (release, debug, with address sanitizer, etc) for you.
+
+We don’t use Travis CI due to the limit on time and computational power.
+We don’t use Jenkins. It was used before and now we are happy we are not using Jenkins.
+
+[Original article](https://clickhouse.tech/docs/en/development/tests/)
+velopment/tests/)
diff --git a/docs/fa/faq/general.md b/docs/fa/faq/general.md
deleted file mode 120000
index bc267395b1b..00000000000
--- a/docs/fa/faq/general.md
+++ /dev/null
@@ -1 +0,0 @@
-../../en/faq/general.md
\ No newline at end of file
diff --git a/docs/fa/faq/general.md b/docs/fa/faq/general.md
new file mode 100644
index 00000000000..758382d7123
--- /dev/null
+++ b/docs/fa/faq/general.md
@@ -0,0 +1,57 @@
+---
+en_copy: true
+---
+
+# General Questions {#general-questions}
+
+## Why Not Use Something Like MapReduce? {#why-not-use-something-like-mapreduce}
+
+We can refer to systems like MapReduce as distributed computing systems in which the reduce operation is based on distributed sorting. The most common open-source solution in this class is [Apache Hadoop](http://hadoop.apache.org). Yandex uses its in-house solution, YT.
+
+These systems aren’t appropriate for online queries due to their high latency. In other words, they can’t be used as the back-end for a web interface. These types of systems aren’t useful for real-time data updates. Distributed sorting isn’t the best way to perform reduce operations if the result of the operation and all the intermediate results (if there are any) are located in the RAM of a single server, which is usually the case for online queries. In such a case, a hash table is an optimal way to perform reduce operations. A common approach to optimizing map-reduce tasks is pre-aggregation (partial reduce) using a hash table in RAM. The user performs this optimization manually. Distributed sorting is one of the main causes of reduced performance when running simple map-reduce tasks.
+
+Most MapReduce implementations allow you to execute arbitrary code on a cluster. But a declarative query language is better suited to OLAP to run experiments quickly. For example, Hadoop has Hive and Pig. Also consider Cloudera Impala or Shark (outdated) for Spark, as well as Spark SQL, Presto, and Apache Drill. Performance when running such tasks is highly sub-optimal compared to specialized systems, but relatively high latency makes it unrealistic to use these systems as the backend for a web interface.
+
+## What If I Have a Problem with Encodings When Using Oracle Through ODBC? {#oracle-odbc-encodings}
+
+If you use Oracle through the ODBC driver as a source of external dictionaries, you need to set the correct value for the `NLS_LANG` environment variable in `/etc/default/clickhouse`. For more information, see the [Oracle NLS\_LANG FAQ](https://www.oracle.com/technetwork/products/globalization/nls-lang-099431.html).
+
+**Example**
+
+``` sql
+NLS_LANG=RUSSIAN_RUSSIA.UTF8
+```
+
+## How Do I Export Data from ClickHouse to a File? {#how-to-export-to-file}
+
+### Using INTO OUTFILE Clause {#using-into-outfile-clause}
+
+Add an [INTO OUTFILE](../query_language/select/#into-outfile-clause) clause to your query.
+
+For example:
+
+``` sql
+SELECT * FROM table INTO OUTFILE 'file'
+```
+
+By default, ClickHouse uses the [TabSeparated](../interfaces/formats.md#tabseparated) format for output data. To select the [data format](../interfaces/formats.md), use the [FORMAT clause](../query_language/select/#format-clause).
+
+For example:
+
+``` sql
+SELECT * FROM table INTO OUTFILE 'file' FORMAT CSV
+```
+
+### Using a File-Engine Table {#using-a-file-engine-table}
+
+See [File](../operations/table_engines/file.md).
+
+### Using Command-Line Redirection {#using-command-line-redirection}
+
+``` sql
+$ clickhouse-client --query "SELECT * from table" --format FormatName > result.txt
+```
+
+See [clickhouse-client](../interfaces/cli.md).
+
+{## [Original article](https://clickhouse.tech/docs/en/faq/general/) ##}
diff --git a/docs/fa/getting_started/example_datasets/index.md b/docs/fa/getting_started/example_datasets/index.md
deleted file mode 120000
index c891314f915..00000000000
--- a/docs/fa/getting_started/example_datasets/index.md
+++ /dev/null
@@ -1 +0,0 @@
-../../../en/getting_started/example_datasets/index.md
\ No newline at end of file
diff --git a/docs/fa/getting_started/example_datasets/index.md b/docs/fa/getting_started/example_datasets/index.md
new file mode 100644
index 00000000000..a07ff8b0010
--- /dev/null
+++ b/docs/fa/getting_started/example_datasets/index.md
@@ -0,0 +1,18 @@
+---
+en_copy: true
+---
+
+# Example Datasets
+
+This section describes how to obtain example datasets and import them into ClickHouse.
+For some datasets example queries are also available.
+
+* [Anonymized Yandex.Metrica Dataset](metrica.md)
+* [Star Schema Benchmark](star_schema.md)
+* [WikiStat](wikistat.md)
+* [Terabyte of Click Logs from Criteo](criteo.md)
+* [AMPLab Big Data Benchmark](amplab_benchmark.md)
+* [New York Taxi Data](nyc_taxi.md)
+* [OnTime](ontime.md)
+
+[Original article](https://clickhouse.tech/docs/en/getting_started/example_datasets)
diff --git a/docs/fa/getting_started/playground.md b/docs/fa/getting_started/playground.md
deleted file mode 120000
index de5b41f453e..00000000000
--- a/docs/fa/getting_started/playground.md
+++ /dev/null
@@ -1 +0,0 @@
-../../en/getting_started/playground.md
\ No newline at end of file
diff --git a/docs/fa/getting_started/playground.md b/docs/fa/getting_started/playground.md
new file mode 100644
index 00000000000..186cb9030c2
--- /dev/null
+++ b/docs/fa/getting_started/playground.md
@@ -0,0 +1,45 @@
+---
+en_copy: true
+---
+
+# ClickHouse Playground {#clickhouse-playground}
+
+[ClickHouse Playground](https://play.clickhouse.tech?file=welcome) allows people to experiment with ClickHouse by running queries instantly, without setting up their server or cluster.
+Several example datasets are available in the Playground as well as sample queries that show ClickHouse features.
+
+The queries are executed as a read-only user. It implies some limitations:
+
+- DDL queries are not allowed
+- INSERT queries are not allowed
+
+The following settings are also enforced:
+- [`max_result_bytes=10485760`](../operations/settings/query_complexity/#max-result-bytes)
+- [`max_result_rows=2000`](../operations/settings/query_complexity/#setting-max_result_rows)
+- [`result_overflow_mode=break`](../operations/settings/query_complexity/#result-overflow-mode)
+- [`max_execution_time=60000`](../operations/settings/query_complexity/#max-execution-time)
+
+ClickHouse Playground gives the experience of m2.small
+[Managed Service for ClickHouse](https://cloud.yandex.com/services/managed-clickhouse)
+instance hosted in [Yandex.Cloud](https://cloud.yandex.com/).
+More information about [cloud providers](../commercial/cloud.md).
+
+ClickHouse Playground web interface makes requests via ClickHouse [HTTP API](../interfaces/http.md).
+The Playground backend is just a ClickHouse cluster without any additional server-side application.
+ClickHouse HTTPS endpoint is also available as a part of the Playground.
+
+You can make queries to playground using any HTTP client, for example [curl](https://curl.haxx.se) or [wget](https://www.gnu.org/software/wget/), or set up a connection using [JDBC](../interfaces/jdbc.md) or [ODBC](../interfaces/odbc.md) drivers.
+More information about software products that support ClickHouse is available [here](../interfaces/index.md).
+
+| Parameter | Value |
+|:----------|:--------------------------------------|
+| Endpoint | https://play-api.clickhouse.tech:8443 |
+| User | `playground` |
+| Password | `clickhouse` |
+
+Note that this endpoint requires a secure connection.
+
+Example:
+
+``` bash
+curl "https://play-api.clickhouse.tech:8443/?query=SELECT+'Play+ClickHouse!';&user=playground&password=clickhouse&database=datasets"
+```
diff --git a/docs/fa/guides/apply_catboost_model.md b/docs/fa/guides/apply_catboost_model.md
deleted file mode 120000
index dd36e885974..00000000000
--- a/docs/fa/guides/apply_catboost_model.md
+++ /dev/null
@@ -1 +0,0 @@
-../../en/guides/apply_catboost_model.md
\ No newline at end of file
diff --git a/docs/fa/guides/apply_catboost_model.md b/docs/fa/guides/apply_catboost_model.md
new file mode 100644
index 00000000000..62eb386147f
--- /dev/null
+++ b/docs/fa/guides/apply_catboost_model.md
@@ -0,0 +1,236 @@
+---
+en_copy: true
+---
+
+# Applying a Catboost Model in ClickHouse {#applying-catboost-model-in-clickhouse}
+
+[CatBoost](https://catboost.ai) is a free and open-source gradient boosting library developed at [Yandex](https://yandex.com/company/) for machine learning.
+
+With this instruction, you will learn to apply pre-trained models in ClickHouse by running model inference from SQL.
+
+To apply a CatBoost model in ClickHouse:
+
+1. [Create a Table](#create-table).
+2. [Insert the Data to the Table](#insert-data-to-table).
+3. [Integrate CatBoost into ClickHouse](#integrate-catboost-into-clickhouse) (Optional step).
+4. [Run the Model Inference from SQL](#run-model-inference).
+
+For more information about training CatBoost models, see [Training and applying models](https://catboost.ai/docs/features/training.html#training).
+
+## Prerequisites {#prerequisites}
+
+If you don’t have the [Docker](https://docs.docker.com/install/) yet, install it.
+
+!!! note "Note"
+ [Docker](https://www.docker.com) is a software platform that allows you to create containers that isolate a CatBoost and ClickHouse installation from the rest of the system.
+
+Before applying a CatBoost model:
+
+**1.** Pull the [Docker image](https://hub.docker.com/r/yandex/tutorial-catboost-clickhouse) from the registry:
+
+``` bash
+$ docker pull yandex/tutorial-catboost-clickhouse
+```
+
+This Docker image contains everything you need to run CatBoost and ClickHouse: code, runtime, libraries, environment variables, and configuration files.
+
+**2.** Make sure the Docker image has been successfully pulled:
+
+``` bash
+$ docker image ls
+REPOSITORY TAG IMAGE ID CREATED SIZE
+yandex/tutorial-catboost-clickhouse latest 622e4d17945b 22 hours ago 1.37GB
+```
+
+**3.** Start a Docker container based on this image:
+
+``` bash
+$ docker run -it -p 8888:8888 yandex/tutorial-catboost-clickhouse
+```
+
+## 1. Create a Table {#create-table}
+
+To create a ClickHouse table for the training sample:
+
+**1.** Start ClickHouse console client in the interactive mode:
+
+``` bash
+$ clickhouse client
+```
+
+!!! note "Note"
+ The ClickHouse server is already running inside the Docker container.
+
+**2.** Create the table using the command:
+
+``` sql
+:) CREATE TABLE amazon_train
+(
+ date Date MATERIALIZED today(),
+ ACTION UInt8,
+ RESOURCE UInt32,
+ MGR_ID UInt32,
+ ROLE_ROLLUP_1 UInt32,
+ ROLE_ROLLUP_2 UInt32,
+ ROLE_DEPTNAME UInt32,
+ ROLE_TITLE UInt32,
+ ROLE_FAMILY_DESC UInt32,
+ ROLE_FAMILY UInt32,
+ ROLE_CODE UInt32
+)
+ENGINE = MergeTree ORDER BY date
+```
+
+**3.** Exit from ClickHouse console client:
+
+``` sql
+:) exit
+```
+
+## 2. Insert the Data to the Table {#insert-data-to-table}
+
+To insert the data:
+
+**1.** Run the following command:
+
+``` bash
+$ clickhouse client --host 127.0.0.1 --query 'INSERT INTO amazon_train FORMAT CSVWithNames' < ~/amazon/train.csv
+```
+
+**2.** Start ClickHouse console client in the interactive mode:
+
+``` bash
+$ clickhouse client
+```
+
+**3.** Make sure the data has been uploaded:
+
+``` sql
+:) SELECT count() FROM amazon_train
+
+SELECT count()
+FROM amazon_train
+
++-count()-+
+| 65538 |
++---------+
+```
+
+## 3. Integrate CatBoost into ClickHouse {#integrate-catboost-into-clickhouse}
+
+!!! note "Note"
+ **Optional step.** The Docker image contains everything you need to run CatBoost and ClickHouse.
+
+To integrate CatBoost into ClickHouse:
+
+**1.** Build the evaluation library.
+
+The fastest way to evaluate a CatBoost model is compile `libcatboostmodel.` library. For more information about how to build the library, see [CatBoost documentation](https://catboost.ai/docs/concepts/c-plus-plus-api_dynamic-c-pluplus-wrapper.html).
+
+**2.** Create a new directory anywhere and with any name, for example, `data` and put the created library in it. The Docker image already contains the library `data/libcatboostmodel.so`.
+
+**3.** Create a new directory for config model anywhere and with any name, for example, `models`.
+
+**4.** Create a model configuration file with any name, for example, `models/amazon_model.xml`.
+
+**5.** Describe the model configuration:
+
+``` xml
+
+
+
+ catboost
+
+ amazon
+
+ /home/catboost/tutorial/catboost_model.bin
+
+ 0
+
+
+```
+
+**6.** Add the path to CatBoost and the model configuration to the ClickHouse configuration:
+
+``` xml
+
+/home/catboost/data/libcatboostmodel.so
+/home/catboost/models/*_model.xml
+```
+
+## 4. Run the Model Inference from SQL {#run-model-inference}
+
+For test model run the ClickHouse client `$ clickhouse client`.
+
+Let’s make sure that the model is working:
+
+``` sql
+:) SELECT
+ modelEvaluate('amazon',
+ RESOURCE,
+ MGR_ID,
+ ROLE_ROLLUP_1,
+ ROLE_ROLLUP_2,
+ ROLE_DEPTNAME,
+ ROLE_TITLE,
+ ROLE_FAMILY_DESC,
+ ROLE_FAMILY,
+ ROLE_CODE) > 0 AS prediction,
+ ACTION AS target
+FROM amazon_train
+LIMIT 10
+```
+
+!!! note "Note"
+ Function [modelEvaluate](../query_language/functions/other_functions.md#function-modelevaluate) returns tuple with per-class raw predictions for multiclass models.
+
+Let’s predict the probability:
+
+``` sql
+:) SELECT
+ modelEvaluate('amazon',
+ RESOURCE,
+ MGR_ID,
+ ROLE_ROLLUP_1,
+ ROLE_ROLLUP_2,
+ ROLE_DEPTNAME,
+ ROLE_TITLE,
+ ROLE_FAMILY_DESC,
+ ROLE_FAMILY,
+ ROLE_CODE) AS prediction,
+ 1. / (1 + exp(-prediction)) AS probability,
+ ACTION AS target
+FROM amazon_train
+LIMIT 10
+```
+
+!!! note "Note"
+ More info about [exp()](../query_language/functions/math_functions.md) function.
+
+Let’s calculate LogLoss on the sample:
+
+``` sql
+:) SELECT -avg(tg * log(prob) + (1 - tg) * log(1 - prob)) AS logloss
+FROM
+(
+ SELECT
+ modelEvaluate('amazon',
+ RESOURCE,
+ MGR_ID,
+ ROLE_ROLLUP_1,
+ ROLE_ROLLUP_2,
+ ROLE_DEPTNAME,
+ ROLE_TITLE,
+ ROLE_FAMILY_DESC,
+ ROLE_FAMILY,
+ ROLE_CODE) AS prediction,
+ 1. / (1. + exp(-prediction)) AS prob,
+ ACTION AS tg
+ FROM amazon_train
+)
+```
+
+!!! note "Note"
+ More info about [avg()](../query_language/agg_functions/reference.md#agg_function-avg) and [log()](../query_language/functions/math_functions.md) functions.
+
+[Original article](https://clickhouse.tech/docs/en/guides/apply_catboost_model/)
diff --git a/docs/fa/guides/index.md b/docs/fa/guides/index.md
deleted file mode 120000
index 162dcbc3b8f..00000000000
--- a/docs/fa/guides/index.md
+++ /dev/null
@@ -1 +0,0 @@
-../../en/guides/index.md
\ No newline at end of file
diff --git a/docs/fa/guides/index.md b/docs/fa/guides/index.md
new file mode 100644
index 00000000000..c1968730961
--- /dev/null
+++ b/docs/fa/guides/index.md
@@ -0,0 +1,12 @@
+---
+en_copy: true
+---
+
+# ClickHouse Guides {#clickhouse-guides}
+
+List of detailed step-by-step instructions that help to solve various tasks using ClickHouse:
+
+- [Tutorial on simple cluster set-up](../getting_started/tutorial.md)
+- [Applying a CatBoost model in ClickHouse](apply_catboost_model.md)
+
+[Original article](https://clickhouse.tech/docs/en/guides/)
diff --git a/docs/fa/interfaces/mysql.md b/docs/fa/interfaces/mysql.md
deleted file mode 120000
index df728b35f80..00000000000
--- a/docs/fa/interfaces/mysql.md
+++ /dev/null
@@ -1 +0,0 @@
-../../en/interfaces/mysql.md
\ No newline at end of file
diff --git a/docs/fa/interfaces/mysql.md b/docs/fa/interfaces/mysql.md
new file mode 100644
index 00000000000..668c0b7b9c3
--- /dev/null
+++ b/docs/fa/interfaces/mysql.md
@@ -0,0 +1,46 @@
+---
+en_copy: true
+---
+
+# MySQL interface {#mysql-interface}
+
+ClickHouse supports MySQL wire protocol. It can be enabled by [mysql\_port](../operations/server_settings/settings.md#server_settings-mysql_port) setting in configuration file:
+
+``` xml
+9004
+```
+
+Example of connecting using command-line tool `mysql`:
+
+``` bash
+$ mysql --protocol tcp -u default -P 9004
+```
+
+Output if a connection succeeded:
+
+``` text
+Welcome to the MySQL monitor. Commands end with ; or \g.
+Your MySQL connection id is 4
+Server version: 20.2.1.1-ClickHouse
+
+Copyright (c) 2000, 2019, Oracle and/or its affiliates. All rights reserved.
+
+Oracle is a registered trademark of Oracle Corporation and/or its
+affiliates. Other names may be trademarks of their respective
+owners.
+
+Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
+
+mysql>
+```
+
+For compatibility with all MySQL clients, it is recommended to specify user password with [double SHA1](../operations/settings/settings_users.md#password_double_sha1_hex) in configuration file.
+If user password is specified using [SHA256](../operations/settings/settings_users.md#password_sha256_hex), some clients won’t be able to authenticate (mysqljs and old versions of command-line tool mysql).
+
+Restrictions:
+
+- prepared queries are not supported
+
+- some data types are sent as strings
+
+[Original article](https://clickhouse.tech/docs/en/interfaces/mysql/)
diff --git a/docs/fa/introduction/adopters.md b/docs/fa/introduction/adopters.md
deleted file mode 120000
index 659153d5f6c..00000000000
--- a/docs/fa/introduction/adopters.md
+++ /dev/null
@@ -1 +0,0 @@
-../../en/introduction/adopters.md
\ No newline at end of file
diff --git a/docs/fa/introduction/adopters.md b/docs/fa/introduction/adopters.md
new file mode 100644
index 00000000000..ef841b2fa05
--- /dev/null
+++ b/docs/fa/introduction/adopters.md
@@ -0,0 +1,79 @@
+---
+en_copy: true
+---
+
+# ClickHouse Adopters {#clickhouse-adopters}
+
+!!! warning "Disclaimer"
+ The following list of companies using ClickHouse and their success stories is assembled from public sources, thus might differ from current reality. We’d appreciate it if you share the story of adopting ClickHouse in your company and [add it to the list](https://github.com/ClickHouse/ClickHouse/edit/master/docs/en/introduction/adopters.md), but please make sure you won’t have any NDA issues by doing so. Providing updates with publications from other companies is also useful.
+
+| Company | Industry | Usecase | Cluster Size | (Un)Compressed Data Size\* | Reference |
+|-----------------------------------------------------------------------------|---------------------------------|-----------------------|------------------------------------------------------------|------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| [2gis](https://2gis.ru) | Maps | Monitoring | — | — | [Talk in Russian, July 2019](https://youtu.be/58sPkXfq6nw) |
+| [Aloha Browser](https://alohabrowser.com/) | Mobile App | Browser backend | — | — | [Slides in Russian, May 2019](https://github.com/yandex/clickhouse-presentations/blob/master/meetup22/aloha.pdf) |
+| [Amadeus](https://amadeus.com/) | Travel | Analytics | — | — | [Press Release, April 2018](https://www.altinity.com/blog/2018/4/5/amadeus-technologies-launches-investment-and-insights-tool-based-on-machine-learning-and-strategy-algorithms) |
+| [Appsflyer](https://www.appsflyer.com) | Mobile analytics | Main product | — | — | [Talk in Russian, July 2019](https://www.youtube.com/watch?v=M3wbRlcpBbY) |
+| [ArenaData](https://arenadata.tech/) | Data Platform | Main product | — | — | [Slides in Russian, December 2019](https://github.com/ClickHouse/clickhouse-presentations/blob/master/meetup38/indexes.pdf) |
+| [Badoo](https://badoo.com) | Dating | Timeseries | — | — | [Slides in Russian, December 2019](https://presentations.clickhouse.tech/meetup38/forecast.pdf) |
+| [Benocs](https://www.benocs.com/) | Network Telemetry and Analytics | Main Product | — | — | [Slides in English, October 2017](https://github.com/ClickHouse/clickhouse-presentations/blob/master/meetup9/lpm.pdf) |
+| [Bloomberg](https://www.bloomberg.com/) | Finance, Media | Monitoring | 102 servers | — | [Slides, May 2018](https://www.slideshare.net/Altinity/http-analytics-for-6m-requests-per-second-using-clickhouse-by-alexander-bocharov) |
+| [Bloxy](https://bloxy.info) | Blockchain | Analytics | — | — | [Slides in Russian, August 2018](https://github.com/ClickHouse/clickhouse-presentations/blob/master/meetup17/4_bloxy.pptx) |
+| `Dataliance/UltraPower` | Telecom | Analytics | — | — | [Slides in Chinese, January 2018](https://github.com/ClickHouse/clickhouse-presentations/blob/master/meetup12/telecom.pdf) |
+| [CARTO](https://carto.com/) | Business Intelligence | Geo analytics | — | — | [Geospatial processing with Clickhouse](https://carto.com/blog/geospatial-processing-with-clickhouse/) |
+| [CERN](http://public.web.cern.ch/public/) | Research | Experiment | — | — | [Press release, April 2012](https://www.yandex.com/company/press_center/press_releases/2012/2012-04-10/) |
+| [Cisco](http://cisco.com/) | Networking | Traffic analysis | — | — | [Lightning talk, October 2019](https://youtu.be/-hI1vDR2oPY?t=5057) |
+| [Citadel Securities](https://www.citadelsecurities.com/) | Finance | — | — | — | [Contribution, March 2019](https://github.com/ClickHouse/ClickHouse/pull/4774) |
+| [Citymobil](https://city-mobil.ru) | Taxi | Analytics | — | — | [Blog Post in Russian, March 2020](https://habr.com/en/company/citymobil/blog/490660/) |
+| [ContentSquare](https://contentsquare.com) | Web analytics | Main product | — | — | [Blog post in French, November 2018](http://souslecapot.net/2018/11/21/patrick-chatain-vp-engineering-chez-contentsquare-penser-davantage-amelioration-continue-que-revolution-constante/) |
+| [Cloudflare](https://cloudflare.com) | CDN | Traffic analysis | 36 servers | — | [Blog post, May 2017](https://blog.cloudflare.com/how-cloudflare-analyzes-1m-dns-queries-per-second/), [Blog post, March 2018](https://blog.cloudflare.com/http-analytics-for-6m-requests-per-second-using-clickhouse/) |
+| [Corunet](https://coru.net/) | Analytics | Main product | — | — | [Slides in English, April 2019](https://github.com/ClickHouse/clickhouse-presentations/blob/master/meetup21/predictive_models.pdf) |
+| [CraiditX 氪信](https://creditx.com) | Finance AI | Analysis | — | — | [Slides in English, November 2019](https://github.com/ClickHouse/clickhouse-presentations/blob/master/meetup33/udf.pptx) |
+| [Criteo/Storetail](https://www.criteo.com/) | Retail | Main product | — | — | [Slides in English, October 2018](https://github.com/ClickHouse/clickhouse-presentations/blob/master/meetup18/3_storetail.pptx) |
+| [Deutsche Bank](https://db.com) | Finance | BI Analytics | — | — | [Slides in English, October 2019](https://bigdatadays.ru/wp-content/uploads/2019/10/D2-H3-3_Yakunin-Goihburg.pdf) |
+| [Diva-e](https://www.diva-e.com) | Digital consulting | Main Product | — | — | [Slides in English, September 2019](https://github.com/ClickHouse/clickhouse-presentations/blob/master/meetup29/ClickHouse-MeetUp-Unusual-Applications-sd-2019-09-17.pdf) |
+| [Exness](https://www.exness.com) | Trading | Metrics, Logging | — | — | [Talk in Russian, May 2019](https://youtu.be/_rpU-TvSfZ8?t=3215) |
+| [Geniee](https://geniee.co.jp) | Ad network | Main product | — | — | [Blog post in Japanese, July 2017](https://tech.geniee.co.jp/entry/2017/07/20/160100) |
+| [HUYA](https://www.huya.com/) | Video Streaming | Analytics | — | — | [Slides in Chinese, October 2018](https://github.com/ClickHouse/clickhouse-presentations/blob/master/meetup19/7.%20ClickHouse万亿数据分析实践%20李本旺(sundy-li)%20虎牙.pdf) |
+| [Idealista](https://www.idealista.com) | Real Estate | Analytics | — | — | [Blog Post in English, April 2019](https://clickhouse.yandex/blog/en/clickhouse-meetup-in-madrid-on-april-2-2019) |
+| [Infovista](https://www.infovista.com/) | Networks | Analytics | — | — | [Slides in English, October 2019](https://github.com/ClickHouse/clickhouse-presentations/blob/master/meetup30/infovista.pdf) |
+| [InnoGames](https://www.innogames.com) | Games | Metrics, Logging | — | — | [Slides in Russian, September 2019](https://github.com/ClickHouse/clickhouse-presentations/blob/master/meetup28/graphite_and_clickHouse.pdf) |
+| [Integros](https://integros.com) | Platform for video services | Analytics | — | — | [Slides in Russian, May 2019](https://github.com/ClickHouse/clickhouse-presentations/blob/master/meetup22/strategies.pdf) |
+| [Kodiak Data](https://www.kodiakdata.com/) | Clouds | Main product | — | — | [Slides in Engish, April 2018](https://github.com/ClickHouse/clickhouse-presentations/blob/master/meetup13/kodiak_data.pdf) |
+| [Kontur](https://kontur.ru) | Software Development | Metrics | — | — | [Talk in Russian, November 2018](https://www.youtube.com/watch?v=U4u4Bd0FtrY) |
+| [LifeStreet](https://lifestreet.com/) | Ad network | Main product | 75 servers (3 replicas) | 5.27 PiB | [Blog post in Russian, February 2017](https://habr.com/en/post/322620/) |
+| [Mail.ru Cloud Solutions](https://mcs.mail.ru/) | Cloud services | Main product | — | — | [Running ClickHouse Instance, in Russian](https://mcs.mail.ru/help/db-create/clickhouse#) |
+| [MessageBird](https://www.messagebird.com) | Telecommunications | Statistics | — | — | [Slides in English, November 2018](https://github.com/ClickHouse/clickhouse-presentations/blob/master/meetup20/messagebird.pdf) |
+| [MGID](https://www.mgid.com/) | Ad network | Web-analytics | — | — | [Our experience in implementing analytical DBMS ClickHouse, in Russian](http://gs-studio.com/news-about-it/32777----clickhouse---c) |
+| [OneAPM](https://www.oneapm.com/) | Monitorings and Data Analysis | Main product | — | — | [Slides in Chinese, October 2018](https://github.com/ClickHouse/clickhouse-presentations/blob/master/meetup19/8.%20clickhouse在OneAPM的应用%20杜龙.pdf) |
+| [Pragma Innovation](http://www.pragma-innovation.fr/) | Telemetry and Big Data Analysis | Main product | — | — | [Slides in English, October 2018](https://github.com/ClickHouse/clickhouse-presentations/blob/master/meetup18/4_pragma_innovation.pdf) |
+| [QINGCLOUD](https://www.qingcloud.com/) | Cloud services | Main product | — | — | [Slides in Chinese, October 2018](https://github.com/ClickHouse/clickhouse-presentations/blob/master/meetup19/4.%20Cloud%20%2B%20TSDB%20for%20ClickHouse%20张健%20QingCloud.pdf) |
+| [Qrator](https://qrator.net) | DDoS protection | Main product | — | — | [Blog Post, March 2019](https://blog.qrator.net/en/clickhouse-ddos-mitigation_37/) |
+| [Beijing PERCENT Information Technology Co., Ltd.](https://www.percent.cn/) | Analytics | Main Product | — | — | [Slides in Chinese, June 2019](https://github.com/ClickHouse/clickhouse-presentations/blob/master/meetup24/4.%20ClickHouse万亿数据双中心的设计与实践%20.pdf) |
+| [Rambler](https://rambler.ru) | Internet services | Analytics | — | — | [Talk in Russian, April 2018](https://medium.com/@ramblertop/разработка-api-clickhouse-для-рамблер-топ-100-f4c7e56f3141) |
+| [Tencent](https://www.tencent.com) | Messaging | Logging | — | — | [Talk in Chinese, November 2019](https://youtu.be/T-iVQRuw-QY?t=5050) |
+| [Traffic Stars](https://trafficstars.com/) | AD network | — | — | — | [Slides in Russian, May 2018](https://github.com/ClickHouse/clickhouse-presentations/blob/master/meetup15/lightning/ninja.pdf) |
+| [S7 Airlines](https://www.s7.ru) | Airlines | Metrics, Logging | — | — | [Talk in Russian, March 2019](https://www.youtube.com/watch?v=nwG68klRpPg&t=15s) |
+| [SEMrush](https://www.semrush.com/) | Marketing | Main product | — | — | [Slides in Russian, August 2018](https://github.com/ClickHouse/clickhouse-presentations/blob/master/meetup17/5_semrush.pdf) |
+| [scireum GmbH](https://www.scireum.de/) | e-Commerce | Main product | — | — | [Talk in German, February 2020](https://www.youtube.com/watch?v=7QWAn5RbyR4) |
+| [Sentry](https://sentry.io/) | Software developer | Backend for product | — | — | [Blog Post in English, May 2019](https://blog.sentry.io/2019/05/16/introducing-snuba-sentrys-new-search-infrastructure) |
+| [SGK](http://www.sgk.gov.tr/wps/portal/sgk/tr) | Goverment Social Security | Analytics | — | — | [Slides in English, November 2019](https://github.com/ClickHouse/clickhouse-presentations/blob/master/meetup35/ClickHouse%20Meetup-Ramazan%20POLAT.pdf) |
+| [seo.do](https://seo.do/) | Analytics | Main product | — | — | [Slides in English, November 2019](https://github.com/ClickHouse/clickhouse-presentations/blob/master/meetup35/CH%20Presentation-%20Metehan%20Çetinkaya.pdf) |
+| [Sina](http://english.sina.com/index.html) | News | — | — | — | [Slides in Chinese, October 2018](https://github.com/ClickHouse/clickhouse-presentations/blob/master/meetup19/6.%20ClickHouse最佳实践%20高鹏_新浪.pdf) |
+| [SMI2](https://smi2.ru/) | News | Analytics | — | — | [Blog Post in Russian, November 2017](https://habr.com/ru/company/smi2/blog/314558/) |
+| [Splunk](https://www.splunk.com/) | Business Analytics | Main product | — | — | [Slides in English, January 2018](https://github.com/ClickHouse/clickhouse-presentations/blob/master/meetup12/splunk.pdf) |
+| [Spotify](https://www.spotify.com) | Music | Experimentation | — | — | [Slides, July 2018](https://www.slideshare.net/glebus/using-clickhouse-for-experimentation-104247173) |
+| [Tencent](https://www.tencent.com) | Big Data | Data processing | — | — | [Slides in Chinese, October 2018](https://github.com/ClickHouse/clickhouse-presentations/blob/master/meetup19/5.%20ClickHouse大数据集群应用_李俊飞腾讯网媒事业部.pdf) |
+| [Uber](https://www.uber.com) | Taxi | Logging | — | — | [Slides, February 2020](https://presentations.clickhouse.tech/meetup40/uber.pdf) |
+| [VKontakte](https://vk.com) | Social Network | Statistics, Logging | — | — | [Slides in Russian, August 2018](https://github.com/ClickHouse/clickhouse-presentations/blob/master/meetup17/3_vk.pdf) |
+| [Wisebits](https://wisebits.com/) | IT Solutions | Analytics | — | — | [Slides in Russian, May 2019](https://github.com/ClickHouse/clickhouse-presentations/blob/master/meetup22/strategies.pdf) |
+| [Xiaoxin Tech.](https://www.xiaoheiban.cn/) | Education | Common purpose | — | — | [Slides in English, November 2019](https://github.com/ClickHouse/clickhouse-presentations/blob/master/meetup33/sync-clickhouse-with-mysql-mongodb.pptx) |
+| [Ximalaya](https://www.ximalaya.com/) | Audio sharing | OLAP | — | — | [Slides in English, November 2019](https://github.com/ClickHouse/clickhouse-presentations/blob/master/meetup33/ximalaya.pdf) |
+| [Yandex Cloud](https://cloud.yandex.ru/services/managed-clickhouse) | Public Cloud | Main product | — | — | [Talk in Russian, December 2019](https://www.youtube.com/watch?v=pgnak9e_E0o) |
+| [Yandex DataLens](https://cloud.yandex.ru/services/datalens) | Business Intelligence | Main product | — | — | [Slides in Russian, December 2019](https://presentations.clickhouse.tech/meetup38/datalens.pdf) |
+| [Yandex Market](https://market.yandex.ru/) | e-Commerce | Metrics, Logging | — | — | [Talk in Russian, January 2019](https://youtu.be/_l1qP0DyBcA?t=478) |
+| [Yandex Metrica](https://metrica.yandex.com) | Web analytics | Main product | 360 servers in one cluster, 1862 servers in one department | 66.41 PiB / 5.68 PiB | [Slides, February 2020](https://presentations.clickhouse.tech/meetup40/introduction/#13) |
+| [ЦВТ](https://htc-cs.ru/) | Software Development | Metrics, Logging | — | — | [Blog Post, March 2019, in Russian](https://vc.ru/dev/62715-kak-my-stroili-monitoring-na-prometheus-clickhouse-i-elk) |
+| [МКБ](https://mkb.ru/) | Bank | Web-system monitoring | — | — | [Slides in Russian, September 2019](https://github.com/ClickHouse/clickhouse-presentations/blob/master/meetup28/mkb.pdf) |
+| [金数据](https://jinshuju.net) | BI Analytics | Main product | — | — | [Slides in Chinese, October 2019](https://github.com/ClickHouse/clickhouse-presentations/blob/master/meetup24/3.%20金数据数据架构调整方案Public.pdf) |
+
+[Original article](https://clickhouse.tech/docs/en/introduction/adopters/)
diff --git a/docs/fa/operations/access_rights.md b/docs/fa/operations/access_rights.md
deleted file mode 120000
index 73463029569..00000000000
--- a/docs/fa/operations/access_rights.md
+++ /dev/null
@@ -1 +0,0 @@
-../../en/operations/access_rights.md
\ No newline at end of file
diff --git a/docs/fa/operations/access_rights.md b/docs/fa/operations/access_rights.md
new file mode 100644
index 00000000000..d5f7da9161e
--- /dev/null
+++ b/docs/fa/operations/access_rights.md
@@ -0,0 +1,110 @@
+---
+en_copy: true
+---
+
+# Access Rights {#access-rights}
+
+Users and access rights are set up in the user config. This is usually `users.xml`.
+
+Users are recorded in the `users` section. Here is a fragment of the `users.xml` file:
+
+``` xml
+
+
+
+
+
+
+
+
+
+
+
+ default
+
+
+ default
+
+
+
+
+
+
+ web
+ default
+
+ test
+
+
+ test
+
+
+
+```
+
+You can see a declaration from two users: `default`and`web`. We added the `web` user separately.
+
+The `default` user is chosen in cases when the username is not passed. The `default` user is also used for distributed query processing, if the configuration of the server or cluster doesn’t specify the `user` and `password` (see the section on the [Distributed](../operations/table_engines/distributed.md) engine).
+
+The user that is used for exchanging information between servers combined in a cluster must not have substantial restrictions or quotas – otherwise, distributed queries will fail.
+
+The password is specified in clear text (not recommended) or in SHA-256. The hash isn’t salted. In this regard, you should not consider these passwords as providing security against potential malicious attacks. Rather, they are necessary for protection from employees.
+
+A list of networks is specified that access is allowed from. In this example, the list of networks for both users is loaded from a separate file (`/etc/metrika.xml`) containing the `networks` substitution. Here is a fragment of it:
+
+``` xml
+
+ ...
+
+ ::/64
+ 203.0.113.0/24
+ 2001:DB8::/32
+ ...
+
+
+```
+
+You could define this list of networks directly in `users.xml`, or in a file in the `users.d` directory (for more information, see the section “[Configuration files](configuration_files.md#configuration_files)”).
+
+The config includes comments explaining how to open access from everywhere.
+
+For use in production, only specify `ip` elements (IP addresses and their masks), since using `host` and `hoost_regexp` might cause extra latency.
+
+Next the user settings profile is specified (see the section “[Settings profiles](settings/settings_profiles.md)”. You can specify the default profile, `default'`. The profile can have any name. You can specify the same profile for different users. The most important thing you can write in the settings profile is `readonly=1`, which ensures read-only access. Then specify the quota to be used (see the section “[Quotas](quotas.md#quotas)”). You can specify the default quota: `default`. It is set in the config by default to only count resource usage, without restricting it. The quota can have any name. You can specify the same quota for different users – in this case, resource usage is calculated for each user individually.
+
+In the optional `` section, you can also specify a list of databases that the user can access. By default, all databases are available to the user. You can specify the `default` database. In this case, the user will receive access to the database by default.
+
+In the optional `` section, you can also specify a list of dictionaries that the user can access. By default, all dictionaries are available to the user.
+
+Access to the `system` database is always allowed (since this database is used for processing queries).
+
+The user can get a list of all databases and tables in them by using `SHOW` queries or system tables, even if access to individual databases isn’t allowed.
+
+Database access is not related to the [readonly](settings/permissions_for_queries.md#settings_readonly) setting. You can’t grant full access to one database and `readonly` access to another one.
+
+[Original article](https://clickhouse.tech/docs/en/operations/access_rights/)
diff --git a/docs/fa/operations/backup.md b/docs/fa/operations/backup.md
deleted file mode 120000
index 1003fb30e61..00000000000
--- a/docs/fa/operations/backup.md
+++ /dev/null
@@ -1 +0,0 @@
-../../en/operations/backup.md
\ No newline at end of file
diff --git a/docs/fa/operations/backup.md b/docs/fa/operations/backup.md
new file mode 100644
index 00000000000..90efb613098
--- /dev/null
+++ b/docs/fa/operations/backup.md
@@ -0,0 +1,38 @@
+---
+en_copy: true
+---
+
+# Data Backup {#data-backup}
+
+While [replication](table_engines/replication.md) provides protection from hardware failures, it does not protect against human errors: accidental deletion of data, deletion of the wrong table or a table on the wrong cluster, and software bugs that result in incorrect data processing or data corruption. In many cases mistakes like these will affect all replicas. ClickHouse has built-in safeguards to prevent some types of mistakes — for example, by default [you can’t just drop tables with a MergeTree-like engine containing more than 50 Gb of data](https://github.com/ClickHouse/ClickHouse/blob/v18.14.18-stable/programs/server/config.xml#L322-L330). However, these safeguards don’t cover all possible cases and can be circumvented.
+
+In order to effectively mitigate possible human errors, you should carefully prepare a strategy for backing up and restoring your data **in advance**.
+
+Each company has different resources available and business requirements, so there’s no universal solution for ClickHouse backups and restores that will fit every situation. What works for one gigabyte of data likely won’t work for tens of petabytes. There are a variety of possible approaches with their own pros and cons, which will be discussed below. It is a good idea to use several approaches instead of just one in order to compensate for their various shortcomings.
+
+!!! note "Note"
+ Keep in mind that if you backed something up and never tried to restore it, chances are that restore will not work properly when you actually need it (or at least it will take longer than business can tolerate). So whatever backup approach you choose, make sure to automate the restore process as well, and practice it on a spare ClickHouse cluster regularly.
+
+## Duplicating Source Data Somewhere Else {#duplicating-source-data-somewhere-else}
+
+Often data that is ingested into ClickHouse is delivered through some sort of persistent queue, such as [Apache Kafka](https://kafka.apache.org). In this case it is possible to configure an additional set of subscribers that will read the same data stream while it is being written to ClickHouse and store it in cold storage somewhere. Most companies already have some default recommended cold storage, which could be an object store or a distributed filesystem like [HDFS](https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-hdfs/HdfsDesign.html).
+
+## Filesystem Snapshots {#filesystem-snapshots}
+
+Some local filesystems provide snapshot functionality (for example, [ZFS](https://en.wikipedia.org/wiki/ZFS)), but they might not be the best choice for serving live queries. A possible solution is to create additional replicas with this kind of filesystem and exclude them from the [Distributed](table_engines/distributed.md) tables that are used for `SELECT` queries. Snapshots on such replicas will be out of reach of any queries that modify data. As a bonus, these replicas might have special hardware configurations with more disks attached per server, which would be cost-effective.
+
+## clickhouse-copier {#clickhouse-copier}
+
+[clickhouse-copier](utils/clickhouse-copier.md) is a versatile tool that was initially created to re-shard petabyte-sized tables. It can also be used for backup and restore purposes because it reliably copies data between ClickHouse tables and clusters.
+
+For smaller volumes of data, a simple `INSERT INTO ... SELECT ...` to remote tables might work as well.
+
+## Manipulations with Parts {#manipulations-with-parts}
+
+ClickHouse allows using the `ALTER TABLE ... FREEZE PARTITION ...` query to create a local copy of table partitions. This is implemented using hardlinks to the `/var/lib/clickhouse/shadow/` folder, so it usually does not consume extra disk space for old data. The created copies of files are not handled by ClickHouse server, so you can just leave them there: you will have a simple backup that doesn’t require any additional external system, but it will still be prone to hardware issues. For this reason, it’s better to remotely copy them to another location and then remove the local copies. Distributed filesystems and object stores are still a good options for this, but normal attached file servers with a large enough capacity might work as well (in this case the transfer will occur via the network filesystem or maybe [rsync](https://en.wikipedia.org/wiki/Rsync)).
+
+For more information about queries related to partition manipulations, see the [ALTER documentation](../query_language/alter.md#alter_manipulations-with-partitions).
+
+A third-party tool is available to automate this approach: [clickhouse-backup](https://github.com/AlexAkulov/clickhouse-backup).
+
+[Original article](https://clickhouse.tech/docs/en/operations/backup/)
diff --git a/docs/fa/operations/configuration_files.md b/docs/fa/operations/configuration_files.md
deleted file mode 120000
index a2d73dbaa25..00000000000
--- a/docs/fa/operations/configuration_files.md
+++ /dev/null
@@ -1 +0,0 @@
-../../en/operations/configuration_files.md
\ No newline at end of file
diff --git a/docs/fa/operations/configuration_files.md b/docs/fa/operations/configuration_files.md
new file mode 100644
index 00000000000..60e6e9a8bde
--- /dev/null
+++ b/docs/fa/operations/configuration_files.md
@@ -0,0 +1,54 @@
+---
+en_copy: true
+---
+
+# Configuration Files {#configuration_files}
+
+ClickHouse supports multi-file configuration management. The main server configuration file is `/etc/clickhouse-server/config.xml`. Other files must be in the `/etc/clickhouse-server/config.d` directory.
+
+!!! note "Note"
+ All the configuration files should be in XML format. Also, they should have the same root element, usually ``.
+
+Some settings specified in the main configuration file can be overridden in other configuration files. The `replace` or `remove` attributes can be specified for the elements of these configuration files.
+
+If neither is specified, it combines the contents of elements recursively, replacing values of duplicate children.
+
+If `replace` is specified, it replaces the entire element with the specified one.
+
+If `remove` is specified, it deletes the element.
+
+The config can also define “substitutions”. If an element has the `incl` attribute, the corresponding substitution from the file will be used as the value. By default, the path to the file with substitutions is `/etc/metrika.xml`. This can be changed in the [include\_from](server_settings/settings.md#server_settings-include_from) element in the server config. The substitution values are specified in `/yandex/substitution_name` elements in this file. If a substitution specified in `incl` does not exist, it is recorded in the log. To prevent ClickHouse from logging missing substitutions, specify the `optional="true"` attribute (for example, settings for [macros](server_settings/settings.md)).
+
+Substitutions can also be performed from ZooKeeper. To do this, specify the attribute `from_zk = "/path/to/node"`. The element value is replaced with the contents of the node at `/path/to/node` in ZooKeeper. You can also put an entire XML subtree on the ZooKeeper node and it will be fully inserted into the source element.
+
+The `config.xml` file can specify a separate config with user settings, profiles, and quotas. The relative path to this config is set in the `users_config` element. By default, it is `users.xml`. If `users_config` is omitted, the user settings, profiles, and quotas are specified directly in `config.xml`.
+
+Users configuration can be splitted into separate files similar to `config.xml` and `config.d/`.
+Directory name is defined as `users_config` setting without `.xml` postfix concatenated with `.d`.
+Directory `users.d` is used by default, as `users_config` defaults to `users.xml`.
+For example, you can have separate config file for each user like this:
+
+``` bash
+$ cat /etc/clickhouse-server/users.d/alice.xml
+```
+
+``` xml
+
+
+
+ analytics
+
+ ::/0
+
+ ...
+ analytics
+
+
+
+```
+
+For each config file, the server also generates `file-preprocessed.xml` files when starting. These files contain all the completed substitutions and overrides, and they are intended for informational use. If ZooKeeper substitutions were used in the config files but ZooKeeper is not available on the server start, the server loads the configuration from the preprocessed file.
+
+The server tracks changes in config files, as well as files and ZooKeeper nodes that were used when performing substitutions and overrides, and reloads the settings for users and clusters on the fly. This means that you can modify the cluster, users, and their settings without restarting the server.
+
+[Original article](https://clickhouse.tech/docs/en/operations/configuration_files/)
diff --git a/docs/fa/operations/index.md b/docs/fa/operations/index.md
deleted file mode 120000
index f7012cd713b..00000000000
--- a/docs/fa/operations/index.md
+++ /dev/null
@@ -1 +0,0 @@
-../../en/operations/settings/index.md
\ No newline at end of file
diff --git a/docs/fa/operations/index.md b/docs/fa/operations/index.md
new file mode 100644
index 00000000000..921307be179
--- /dev/null
+++ b/docs/fa/operations/index.md
@@ -0,0 +1,24 @@
+---
+en_copy: true
+---
+
+# Operations {#operations}
+
+ClickHouse operations manual consists of the following major sections:
+
+- [Requirements](requirements.md)
+- [Monitoring](monitoring.md)
+- [Troubleshooting](troubleshooting.md)
+- [Usage Recommendations](tips.md)
+- [Update Procedure](update.md)
+- [Access Rights](access_rights.md)
+- [Data Backup](backup.md)
+- [Configuration Files](configuration_files.md)
+- [Quotas](quotas.md)
+- [System Tables](system_tables.md)
+- [Server Configuration Parameters](server_settings/index.md)
+- [How To Test Your Hardware With ClickHouse](performance_test.md)
+- [Settings](settings/index.md)
+- [Utilities](utils/index.md)
+
+[Original article](https://clickhouse.tech/docs/en/operations/)
diff --git a/docs/fa/operations/monitoring.md b/docs/fa/operations/monitoring.md
deleted file mode 120000
index 515ae8b4fff..00000000000
--- a/docs/fa/operations/monitoring.md
+++ /dev/null
@@ -1 +0,0 @@
-../../en/operations/monitoring.md
\ No newline at end of file
diff --git a/docs/fa/operations/monitoring.md b/docs/fa/operations/monitoring.md
new file mode 100644
index 00000000000..b8d72b07b61
--- /dev/null
+++ b/docs/fa/operations/monitoring.md
@@ -0,0 +1,41 @@
+---
+en_copy: true
+---
+
+# Monitoring {#monitoring}
+
+You can monitor:
+
+- Utilization of hardware resources.
+- ClickHouse server metrics.
+
+## Resource Utilization {#resource-utilization}
+
+ClickHouse does not monitor the state of hardware resources by itself.
+
+It is highly recommended to set up monitoring for:
+
+- Load and temperature on processors.
+
+ You can use [dmesg](https://en.wikipedia.org/wiki/Dmesg), [turbostat](https://www.linux.org/docs/man8/turbostat.html) or other instruments.
+
+- Utilization of storage system, RAM and network.
+
+## ClickHouse Server Metrics {#clickhouse-server-metrics}
+
+ClickHouse server has embedded instruments for self-state monitoring.
+
+To track server events use server logs. See the [logger](server_settings/settings.md#server_settings-logger) section of the configuration file.
+
+ClickHouse collects:
+
+- Different metrics of how the server uses computational resources.
+- Common statistics on query processing.
+
+You can find metrics in the [system.metrics](system_tables.md#system_tables-metrics), [system.events](system_tables.md#system_tables-events), and [system.asynchronous\_metrics](system_tables.md#system_tables-asynchronous_metrics) tables.
+
+You can configure ClickHouse to export metrics to [Graphite](https://github.com/graphite-project). See the [Graphite section](server_settings/settings.md#server_settings-graphite) in the ClickHouse server configuration file. Before configuring export of metrics, you should set up Graphite by following their official [guide](https://graphite.readthedocs.io/en/latest/install.html).
+
+Additionally, you can monitor server availability through the HTTP API. Send the `HTTP GET` request to `/ping`. If the server is available, it responds with `200 OK`.
+
+To monitor servers in a cluster configuration, you should set the [max\_replica\_delay\_for\_distributed\_queries](settings/settings.md#settings-max_replica_delay_for_distributed_queries) parameter and use the HTTP resource `/replicas_status`. A request to `/replicas_status` returns `200 OK` if the replica is available and is not delayed behind the other replicas. If a replica is delayed, it returns `503 HTTP_SERVICE_UNAVAILABLE` with information about the gap.
diff --git a/docs/fa/operations/performance/sampling_query_profiler.md b/docs/fa/operations/performance/sampling_query_profiler.md
deleted file mode 120000
index c55c58684ba..00000000000
--- a/docs/fa/operations/performance/sampling_query_profiler.md
+++ /dev/null
@@ -1 +0,0 @@
-../../../en/operations/performance/sampling_query_profiler.md
\ No newline at end of file
diff --git a/docs/fa/operations/performance/sampling_query_profiler.md b/docs/fa/operations/performance/sampling_query_profiler.md
new file mode 100644
index 00000000000..25368fcd883
--- /dev/null
+++ b/docs/fa/operations/performance/sampling_query_profiler.md
@@ -0,0 +1,61 @@
+---
+en_copy: true
+---
+
+# Sampling Query Profiler {#sampling-query-profiler}
+
+ClickHouse runs sampling profiler that allows analyzing query execution. Using profiler you can find source code routines that used the most frequently during query execution. You can trace CPU time and wall-clock time spent including idle time.
+
+To use profiler:
+
+- Setup the [trace\_log](../server_settings/settings.md#server_settings-trace_log) section of the server configuration.
+
+ This section configures the [trace\_log](../system_tables.md#system_tables-trace_log) system table containing the results of the profiler functioning. It is configured by default. Remember that data in this table is valid only for a running server. After the server restart, ClickHouse doesn’t clean up the table and all the stored virtual memory address may become invalid.
+
+- Setup the [query\_profiler\_cpu\_time\_period\_ns](../settings/settings.md#query_profiler_cpu_time_period_ns) or [query\_profiler\_real\_time\_period\_ns](../settings/settings.md#query_profiler_real_time_period_ns) settings. Both settings can be used simultaneously.
+
+ These settings allow you to configure profiler timers. As these are the session settings, you can get different sampling frequency for the whole server, individual users or user profiles, for your interactive session, and for each individual query.
+
+The default sampling frequency is one sample per second and both CPU and real timers are enabled. This frequency allows collecting enough information about ClickHouse cluster. At the same time, working with this frequency, profiler doesn’t affect ClickHouse server’s performance. If you need to profile each individual query try to use higher sampling frequency.
+
+To analyze the `trace_log` system table:
+
+- Install the `clickhouse-common-static-dbg` package. See [Install from DEB Packages](../../getting_started/install.md#install-from-deb-packages).
+
+- Allow introspection functions by the [allow\_introspection\_functions](../settings/settings.md#settings-allow_introspection_functions) setting.
+
+ For security reasons, introspection functions are disabled by default.
+
+- Use the `addressToLine`, `addressToSymbol` and `demangle` [introspection functions](../../query_language/functions/introspection.md) to get function names and their positions in ClickHouse code. To get a profile for some query, you need to aggregate data from the `trace_log` table. You can aggregate data by individual functions or by the whole stack traces.
+
+If you need to visualize `trace_log` info, try [flamegraph](../../interfaces/third-party/gui/#clickhouse-flamegraph) and [speedscope](https://github.com/laplab/clickhouse-speedscope).
+
+## Example {#example}
+
+In this example we:
+
+- Filtering `trace_log` data by a query identifier and the current date.
+
+- Aggregating by stack trace.
+
+- Using introspection functions, we will get a report of:
+
+ - Names of symbols and corresponding source code functions.
+ - Source code locations of these functions.
+
+
+
+``` sql
+SELECT
+ count(),
+ arrayStringConcat(arrayMap(x -> concat(demangle(addressToSymbol(x)), '\n ', addressToLine(x)), trace), '\n') AS sym
+FROM system.trace_log
+WHERE (query_id = 'ebca3574-ad0a-400a-9cbc-dca382f5998c') AND (event_date = today())
+GROUP BY trace
+ORDER BY count() DESC
+LIMIT 10
+```
+
+``` text
+{% include "operations/performance/sampling_query_profiler_example_result.txt" %}
+```
diff --git a/docs/fa/operations/performance/sampling_query_profiler_example_result.txt b/docs/fa/operations/performance/sampling_query_profiler_example_result.txt
deleted file mode 120000
index 58c5abe7122..00000000000
--- a/docs/fa/operations/performance/sampling_query_profiler_example_result.txt
+++ /dev/null
@@ -1 +0,0 @@
-../../../en/operations/performance/sampling_query_profiler_example_result.txt
\ No newline at end of file
diff --git a/docs/fa/operations/performance/sampling_query_profiler_example_result.txt b/docs/fa/operations/performance/sampling_query_profiler_example_result.txt
new file mode 100644
index 00000000000..a5f6d71ca95
--- /dev/null
+++ b/docs/fa/operations/performance/sampling_query_profiler_example_result.txt
@@ -0,0 +1,560 @@
+---
+en_copy: true
+---
+
+Row 1:
+──────
+count(): 6344
+sym: StackTrace::StackTrace(ucontext_t const&)
+ /home/milovidov/ClickHouse/build_gcc9/../dbms/Common/StackTrace.cpp:208
+DB::(anonymous namespace)::writeTraceInfo(DB::TimerType, int, siginfo_t*, void*) [clone .isra.0]
+ /home/milovidov/ClickHouse/build_gcc9/../dbms/IO/BufferBase.h:99
+
+
+read
+
+DB::ReadBufferFromFileDescriptor::nextImpl()
+ /home/milovidov/ClickHouse/build_gcc9/../dbms/IO/ReadBufferFromFileDescriptor.cpp:56
+DB::CompressedReadBufferBase::readCompressedData(unsigned long&, unsigned long&)
+ /home/milovidov/ClickHouse/build_gcc9/../dbms/IO/ReadBuffer.h:54
+DB::CompressedReadBufferFromFile::nextImpl()
+ /home/milovidov/ClickHouse/build_gcc9/../dbms/Compression/CompressedReadBufferFromFile.cpp:22
+DB::CompressedReadBufferFromFile::seek(unsigned long, unsigned long)
+ /home/milovidov/ClickHouse/build_gcc9/../dbms/Compression/CompressedReadBufferFromFile.cpp:63
+DB::MergeTreeReaderStream::seekToMark(unsigned long)
+ /home/milovidov/ClickHouse/build_gcc9/../dbms/Storages/MergeTree/MergeTreeReaderStream.cpp:200
+std::_Function_handler > const&), DB::MergeTreeReader::readData(std::__cxx11::basic_string, std::allocator > const&, DB::IDataType const&, DB::IColumn&, unsigned long, bool, unsigned long, bool)::{lambda(bool)#1}::operator()(bool) const::{lambda(std::vector > const&)#1}>::_M_invoke(std::_Any_data const&, std::vector > const&)
+ /home/milovidov/ClickHouse/build_gcc9/../dbms/Storages/MergeTree/MergeTreeReader.cpp:212
+DB::IDataType::deserializeBinaryBulkWithMultipleStreams(DB::IColumn&, unsigned long, DB::IDataType::DeserializeBinaryBulkSettings&, std::shared_ptr&) const
+ /usr/local/include/c++/9.1.0/bits/std_function.h:690
+DB::MergeTreeReader::readData(std::__cxx11::basic_string, std::allocator > const&, DB::IDataType const&, DB::IColumn&, unsigned long, bool, unsigned long, bool)
+ /home/milovidov/ClickHouse/build_gcc9/../dbms/Storages/MergeTree/MergeTreeReader.cpp:232
+DB::MergeTreeReader::readRows(unsigned long, bool, unsigned long, DB::Block&)
+ /home/milovidov/ClickHouse/build_gcc9/../dbms/Storages/MergeTree/MergeTreeReader.cpp:111
+DB::MergeTreeRangeReader::DelayedStream::finalize(DB::Block&)
+ /home/milovidov/ClickHouse/build_gcc9/../dbms/Storages/MergeTree/MergeTreeRangeReader.cpp:35
+DB::MergeTreeRangeReader::continueReadingChain(DB::MergeTreeRangeReader::ReadResult&)
+ /home/milovidov/ClickHouse/build_gcc9/../dbms/Storages/MergeTree/MergeTreeRangeReader.cpp:219
+DB::MergeTreeRangeReader::read(unsigned long, std::vector >&)
+ /home/milovidov/ClickHouse/build_gcc9/../dbms/Storages/MergeTree/MergeTreeRangeReader.cpp:487
+DB::MergeTreeBaseSelectBlockInputStream::readFromPartImpl()
+ /home/milovidov/ClickHouse/build_gcc9/../dbms/Storages/MergeTree/MergeTreeBaseSelectBlockInputStream.cpp:158
+DB::MergeTreeBaseSelectBlockInputStream::readImpl()
+ /usr/local/include/c++/9.1.0/bits/stl_vector.h:108
+DB::IBlockInputStream::read()
+ /usr/local/include/c++/9.1.0/bits/stl_vector.h:108
+DB::ExpressionBlockInputStream::readImpl()
+ /home/milovidov/ClickHouse/build_gcc9/../dbms/DataStreams/ExpressionBlockInputStream.cpp:34
+DB::IBlockInputStream::read()
+ /usr/local/include/c++/9.1.0/bits/stl_vector.h:108
+DB::PartialSortingBlockInputStream::readImpl()
+ /home/milovidov/ClickHouse/build_gcc9/../dbms/DataStreams/PartialSortingBlockInputStream.cpp:13
+DB::IBlockInputStream::read()
+ /usr/local/include/c++/9.1.0/bits/stl_vector.h:108
+DB::ParallelInputsProcessor::loop(unsigned long)
+ /usr/local/include/c++/9.1.0/bits/atomic_base.h:419
+DB::ParallelInputsProcessor::thread(std::shared_ptr, unsigned long)
+ /home/milovidov/ClickHouse/build_gcc9/../dbms/DataStreams/ParallelInputsProcessor.h:215
+ThreadFromGlobalPool::ThreadFromGlobalPool::*)(std::shared_ptr, unsigned long), DB::ParallelInputsProcessor*, std::shared_ptr, unsigned long&>(void (DB::ParallelInputsProcessor::*&&)(std::shared_ptr, unsigned long), DB::ParallelInputsProcessor*&&, std::shared_ptr&&, unsigned long&)::{lambda()#1}::operator()() const
+ /usr/local/include/c++/9.1.0/bits/shared_ptr_base.h:729
+ThreadPoolImpl::worker(std::_List_iterator)
+ /usr/local/include/c++/9.1.0/bits/unique_lock.h:69
+execute_native_thread_routine
+ /home/milovidov/ClickHouse/ci/workspace/gcc/gcc-build/x86_64-pc-linux-gnu/libstdc++-v3/include/bits/unique_ptr.h:81
+start_thread
+
+__clone
+
+
+Row 2:
+──────
+count(): 3295
+sym: StackTrace::StackTrace(ucontext_t const&)
+ /home/milovidov/ClickHouse/build_gcc9/../dbms/Common/StackTrace.cpp:208
+DB::(anonymous namespace)::writeTraceInfo(DB::TimerType, int, siginfo_t*, void*) [clone .isra.0]
+ /home/milovidov/ClickHouse/build_gcc9/../dbms/IO/BufferBase.h:99
+
+
+__pthread_cond_wait
+
+std::condition_variable::wait(std::unique_lock&)
+ /home/milovidov/ClickHouse/ci/workspace/gcc/gcc-build/x86_64-pc-linux-gnu/libstdc++-v3/src/c++11/../../../../../gcc-9.1.0/libstdc++-v3/src/c++11/condition_variable.cc:55
+Poco::Semaphore::wait()
+ /home/milovidov/ClickHouse/build_gcc9/../contrib/poco/Foundation/src/Semaphore.cpp:61
+DB::UnionBlockInputStream::readImpl()
+ /usr/local/include/c++/9.1.0/x86_64-pc-linux-gnu/bits/gthr-default.h:748
+DB::IBlockInputStream::read()
+ /usr/local/include/c++/9.1.0/bits/stl_vector.h:108
+DB::MergeSortingBlockInputStream::readImpl()
+ /home/milovidov/ClickHouse/build_gcc9/../dbms/Core/Block.h:90
+DB::IBlockInputStream::read()
+ /usr/local/include/c++/9.1.0/bits/stl_vector.h:108
+DB::ExpressionBlockInputStream::readImpl()
+ /home/milovidov/ClickHouse/build_gcc9/../dbms/DataStreams/ExpressionBlockInputStream.cpp:34
+DB::IBlockInputStream::read()
+ /usr/local/include/c++/9.1.0/bits/stl_vector.h:108
+DB::LimitBlockInputStream::readImpl()
+ /usr/local/include/c++/9.1.0/bits/stl_vector.h:108
+DB::IBlockInputStream::read()
+ /usr/local/include/c++/9.1.0/bits/stl_vector.h:108
+DB::AsynchronousBlockInputStream::calculate()
+ /usr/local/include/c++/9.1.0/bits/stl_vector.h:108
+std::_Function_handler::_M_invoke(std::_Any_data const&)
+ /usr/local/include/c++/9.1.0/bits/atomic_base.h:551
+ThreadPoolImpl::worker(std::_List_iterator)
+ /usr/local/include/c++/9.1.0/x86_64-pc-linux-gnu/bits/gthr-default.h:748
+ThreadFromGlobalPool::ThreadFromGlobalPool::scheduleImpl(std::function, int, std::optional)::{lambda()#3}>(ThreadPoolImpl::scheduleImpl(std::function, int, std::optional)::{lambda()#3}&&)::{lambda()#1}::operator()() const
+ /home/milovidov/ClickHouse/build_gcc9/../dbms/Common/ThreadPool.h:146
+ThreadPoolImpl::worker(std::_List_iterator)
+ /usr/local/include/c++/9.1.0/bits/unique_lock.h:69
+execute_native_thread_routine
+ /home/milovidov/ClickHouse/ci/workspace/gcc/gcc-build/x86_64-pc-linux-gnu/libstdc++-v3/include/bits/unique_ptr.h:81
+start_thread
+
+__clone
+
+
+Row 3:
+──────
+count(): 1978
+sym: StackTrace::StackTrace(ucontext_t const&)
+ /home/milovidov/ClickHouse/build_gcc9/../dbms/Common/StackTrace.cpp:208
+DB::(anonymous namespace)::writeTraceInfo(DB::TimerType, int, siginfo_t*, void*) [clone .isra.0]
+ /home/milovidov/ClickHouse/build_gcc9/../dbms/IO/BufferBase.h:99
+
+
+DB::VolnitskyBase >::search(unsigned char const*, unsigned long) const
+ /opt/milovidov/ClickHouse/build_gcc9/programs/clickhouse
+DB::MatchImpl::vector_constant(DB::PODArray, 15ul, 16ul> const&, DB::PODArray, 15ul, 16ul> const&, std::__cxx11::basic_string, std::allocator > const&, DB::PODArray, 15ul, 16ul>&)
+ /opt/milovidov/ClickHouse/build_gcc9/programs/clickhouse
+DB::FunctionsStringSearch, DB::NameLike>::executeImpl(DB::Block&, std::vector > const&, unsigned long, unsigned long)
+ /opt/milovidov/ClickHouse/build_gcc9/programs/clickhouse
+DB::PreparedFunctionImpl::execute(DB::Block&, std::vector > const&, unsigned long, unsigned long, bool)
+ /home/milovidov/ClickHouse/build_gcc9/../dbms/Functions/IFunction.cpp:464
+DB::ExpressionAction::execute(DB::Block&, bool) const
+ /usr/local/include/c++/9.1.0/bits/stl_vector.h:677
+DB::ExpressionActions::execute(DB::Block&, bool) const
+ /home/milovidov/ClickHouse/build_gcc9/../dbms/Interpreters/ExpressionActions.cpp:739
+DB::MergeTreeRangeReader::executePrewhereActionsAndFilterColumns(DB::MergeTreeRangeReader::ReadResult&)
+ /home/milovidov/ClickHouse/build_gcc9/../dbms/Storages/MergeTree/MergeTreeRangeReader.cpp:660
+DB::MergeTreeRangeReader::read(unsigned long, std::vector >&)
+ /home/milovidov/ClickHouse/build_gcc9/../dbms/Storages/MergeTree/MergeTreeRangeReader.cpp:546
+DB::MergeTreeRangeReader::read(unsigned long, std::vector >&)
+ /usr/local/include/c++/9.1.0/bits/stl_vector.h:108
+DB::MergeTreeBaseSelectBlockInputStream::readFromPartImpl()
+ /home/milovidov/ClickHouse/build_gcc9/../dbms/Storages/MergeTree/MergeTreeBaseSelectBlockInputStream.cpp:158
+DB::MergeTreeBaseSelectBlockInputStream::readImpl()
+ /usr/local/include/c++/9.1.0/bits/stl_vector.h:108
+DB::IBlockInputStream::read()
+ /usr/local/include/c++/9.1.0/bits/stl_vector.h:108
+DB::ExpressionBlockInputStream::readImpl()
+ /home/milovidov/ClickHouse/build_gcc9/../dbms/DataStreams/ExpressionBlockInputStream.cpp:34
+DB::IBlockInputStream::read()
+ /usr/local/include/c++/9.1.0/bits/stl_vector.h:108
+DB::PartialSortingBlockInputStream::readImpl()
+ /home/milovidov/ClickHouse/build_gcc9/../dbms/DataStreams/PartialSortingBlockInputStream.cpp:13
+DB::IBlockInputStream::read()
+ /usr/local/include/c++/9.1.0/bits/stl_vector.h:108
+DB::ParallelInputsProcessor::loop(unsigned long)
+ /usr/local/include/c++/9.1.0/bits/atomic_base.h:419
+DB::ParallelInputsProcessor::thread(std::shared_ptr, unsigned long)
+ /home/milovidov/ClickHouse/build_gcc9/../dbms/DataStreams/ParallelInputsProcessor.h:215
+ThreadFromGlobalPool::ThreadFromGlobalPool::*)(std::shared_ptr, unsigned long), DB::ParallelInputsProcessor*, std::shared_ptr, unsigned long&>(void (DB::ParallelInputsProcessor::*&&)(std::shared_ptr, unsigned long), DB::ParallelInputsProcessor*&&, std::shared_ptr&&, unsigned long&)::{lambda()#1}::operator()() const
+ /usr/local/include/c++/9.1.0/bits/shared_ptr_base.h:729
+ThreadPoolImpl::worker(std::_List_iterator)
+ /usr/local/include/c++/9.1.0/bits/unique_lock.h:69
+execute_native_thread_routine
+ /home/milovidov/ClickHouse/ci/workspace/gcc/gcc-build/x86_64-pc-linux-gnu/libstdc++-v3/include/bits/unique_ptr.h:81
+start_thread
+
+__clone
+
+
+Row 4:
+──────
+count(): 1913
+sym: StackTrace::StackTrace(ucontext_t const&)
+ /home/milovidov/ClickHouse/build_gcc9/../dbms/Common/StackTrace.cpp:208
+DB::(anonymous namespace)::writeTraceInfo(DB::TimerType, int, siginfo_t*, void*) [clone .isra.0]
+ /home/milovidov/ClickHouse/build_gcc9/../dbms/IO/BufferBase.h:99
+
+
+DB::VolnitskyBase >::search(unsigned char const*, unsigned long) const
+ /opt/milovidov/ClickHouse/build_gcc9/programs/clickhouse
+DB::MatchImpl::vector_constant(DB::PODArray, 15ul, 16ul> const&, DB::PODArray, 15ul, 16ul> const&, std::__cxx11::basic_string, std::allocator > const&, DB::PODArray, 15ul, 16ul>&)
+ /opt/milovidov/ClickHouse/build_gcc9/programs/clickhouse
+DB::FunctionsStringSearch, DB::NameLike>::executeImpl(DB::Block&, std::vector > const&, unsigned long, unsigned long)
+ /opt/milovidov/ClickHouse/build_gcc9/programs/clickhouse
+DB::PreparedFunctionImpl::execute(DB::Block&, std::vector > const&, unsigned long, unsigned long, bool)
+ /home/milovidov/ClickHouse/build_gcc9/../dbms/Functions/IFunction.cpp:464
+DB::ExpressionAction::execute(DB::Block&, bool) const
+ /usr/local/include/c++/9.1.0/bits/stl_vector.h:677
+DB::ExpressionActions::execute(DB::Block&, bool) const
+ /home/milovidov/ClickHouse/build_gcc9/../dbms/Interpreters/ExpressionActions.cpp:739
+DB::MergeTreeRangeReader::executePrewhereActionsAndFilterColumns(DB::MergeTreeRangeReader::ReadResult&)
+ /home/milovidov/ClickHouse/build_gcc9/../dbms/Storages/MergeTree/MergeTreeRangeReader.cpp:660
+DB::MergeTreeRangeReader::read(unsigned long, std::vector >&)
+ /home/milovidov/ClickHouse/build_gcc9/../dbms/Storages/MergeTree/MergeTreeRangeReader.cpp:546
+DB::MergeTreeRangeReader::read(unsigned long, std::vector >&)
+ /usr/local/include/c++/9.1.0/bits/stl_vector.h:108
+DB::MergeTreeBaseSelectBlockInputStream::readFromPartImpl()
+ /home/milovidov/ClickHouse/build_gcc9/../dbms/Storages/MergeTree/MergeTreeBaseSelectBlockInputStream.cpp:158
+DB::MergeTreeBaseSelectBlockInputStream::readImpl()
+ /usr/local/include/c++/9.1.0/bits/stl_vector.h:108
+DB::IBlockInputStream::read()
+ /usr/local/include/c++/9.1.0/bits/stl_vector.h:108
+DB::ExpressionBlockInputStream::readImpl()
+ /home/milovidov/ClickHouse/build_gcc9/../dbms/DataStreams/ExpressionBlockInputStream.cpp:34
+DB::IBlockInputStream::read()
+ /usr/local/include/c++/9.1.0/bits/stl_vector.h:108
+DB::PartialSortingBlockInputStream::readImpl()
+ /home/milovidov/ClickHouse/build_gcc9/../dbms/DataStreams/PartialSortingBlockInputStream.cpp:13
+DB::IBlockInputStream::read()
+ /usr/local/include/c++/9.1.0/bits/stl_vector.h:108
+DB::ParallelInputsProcessor::loop(unsigned long)
+ /usr/local/include/c++/9.1.0/bits/atomic_base.h:419
+DB::ParallelInputsProcessor::thread(std::shared_ptr, unsigned long)
+ /home/milovidov/ClickHouse/build_gcc9/../dbms/DataStreams/ParallelInputsProcessor.h:215
+ThreadFromGlobalPool::ThreadFromGlobalPool::*)(std::shared_ptr, unsigned long), DB::ParallelInputsProcessor*, std::shared_ptr, unsigned long&>(void (DB::ParallelInputsProcessor::*&&)(std::shared_ptr, unsigned long), DB::ParallelInputsProcessor*&&, std::shared_ptr&&, unsigned long&)::{lambda()#1}::operator()() const
+ /usr/local/include/c++/9.1.0/bits/shared_ptr_base.h:729
+ThreadPoolImpl::worker(std::_List_iterator)
+ /usr/local/include/c++/9.1.0/bits/unique_lock.h:69
+execute_native_thread_routine
+ /home/milovidov/ClickHouse/ci/workspace/gcc/gcc-build/x86_64-pc-linux-gnu/libstdc++-v3/include/bits/unique_ptr.h:81
+start_thread
+
+__clone
+
+
+Row 5:
+──────
+count(): 1672
+sym: StackTrace::StackTrace(ucontext_t const&)
+ /home/milovidov/ClickHouse/build_gcc9/../dbms/Common/StackTrace.cpp:208
+DB::(anonymous namespace)::writeTraceInfo(DB::TimerType, int, siginfo_t*, void*) [clone .isra.0]
+ /home/milovidov/ClickHouse/build_gcc9/../dbms/IO/BufferBase.h:99
+
+
+DB::VolnitskyBase >::search(unsigned char const*, unsigned long) const
+ /opt/milovidov/ClickHouse/build_gcc9/programs/clickhouse
+DB::MatchImpl::vector_constant(DB::PODArray, 15ul, 16ul> const&, DB::PODArray, 15ul, 16ul> const&, std::__cxx11::basic_string, std::allocator > const&, DB::PODArray, 15ul, 16ul>&)
+ /opt/milovidov/ClickHouse/build_gcc9/programs/clickhouse
+DB::FunctionsStringSearch, DB::NameLike>::executeImpl(DB::Block&, std::vector > const&, unsigned long, unsigned long)
+ /opt/milovidov/ClickHouse/build_gcc9/programs/clickhouse
+DB::PreparedFunctionImpl::execute(DB::Block&, std::vector > const&, unsigned long, unsigned long, bool)
+ /home/milovidov/ClickHouse/build_gcc9/../dbms/Functions/IFunction.cpp:464
+DB::ExpressionAction::execute(DB::Block&, bool) const
+ /usr/local/include/c++/9.1.0/bits/stl_vector.h:677
+DB::ExpressionActions::execute(DB::Block&, bool) const
+ /home/milovidov/ClickHouse/build_gcc9/../dbms/Interpreters/ExpressionActions.cpp:739
+DB::MergeTreeRangeReader::executePrewhereActionsAndFilterColumns(DB::MergeTreeRangeReader::ReadResult&)
+ /home/milovidov/ClickHouse/build_gcc9/../dbms/Storages/MergeTree/MergeTreeRangeReader.cpp:660
+DB::MergeTreeRangeReader::read(unsigned long, std::vector >&)
+ /home/milovidov/ClickHouse/build_gcc9/../dbms/Storages/MergeTree/MergeTreeRangeReader.cpp:546
+DB::MergeTreeRangeReader::read(unsigned long, std::vector >&)
+ /usr/local/include/c++/9.1.0/bits/stl_vector.h:108
+DB::MergeTreeBaseSelectBlockInputStream::readFromPartImpl()
+ /home/milovidov/ClickHouse/build_gcc9/../dbms/Storages/MergeTree/MergeTreeBaseSelectBlockInputStream.cpp:158
+DB::MergeTreeBaseSelectBlockInputStream::readImpl()
+ /usr/local/include/c++/9.1.0/bits/stl_vector.h:108
+DB::IBlockInputStream::read()
+ /usr/local/include/c++/9.1.0/bits/stl_vector.h:108
+DB::ExpressionBlockInputStream::readImpl()
+ /home/milovidov/ClickHouse/build_gcc9/../dbms/DataStreams/ExpressionBlockInputStream.cpp:34
+DB::IBlockInputStream::read()
+ /usr/local/include/c++/9.1.0/bits/stl_vector.h:108
+DB::PartialSortingBlockInputStream::readImpl()
+ /home/milovidov/ClickHouse/build_gcc9/../dbms/DataStreams/PartialSortingBlockInputStream.cpp:13
+DB::IBlockInputStream::read()
+ /usr/local/include/c++/9.1.0/bits/stl_vector.h:108
+DB::ParallelInputsProcessor::loop(unsigned long)
+ /usr/local/include/c++/9.1.0/bits/atomic_base.h:419
+DB::ParallelInputsProcessor::thread(std::shared_ptr, unsigned long)
+ /home/milovidov/ClickHouse/build_gcc9/../dbms/DataStreams/ParallelInputsProcessor.h:215
+ThreadFromGlobalPool::ThreadFromGlobalPool::*)(std::shared_ptr, unsigned long), DB::ParallelInputsProcessor*, std::shared_ptr, unsigned long&>(void (DB::ParallelInputsProcessor::*&&)(std::shared_ptr, unsigned long), DB::ParallelInputsProcessor*&&, std::shared_ptr&&, unsigned long&)::{lambda()#1}::operator()() const
+ /usr/local/include/c++/9.1.0/bits/shared_ptr_base.h:729
+ThreadPoolImpl::worker(std::_List_iterator)
+ /usr/local/include/c++/9.1.0/bits/unique_lock.h:69
+execute_native_thread_routine
+ /home/milovidov/ClickHouse/ci/workspace/gcc/gcc-build/x86_64-pc-linux-gnu/libstdc++-v3/include/bits/unique_ptr.h:81
+start_thread
+
+__clone
+
+
+Row 6:
+──────
+count(): 1531
+sym: StackTrace::StackTrace(ucontext_t const&)
+ /home/milovidov/ClickHouse/build_gcc9/../dbms/Common/StackTrace.cpp:208
+DB::(anonymous namespace)::writeTraceInfo(DB::TimerType, int, siginfo_t*, void*) [clone .isra.0]
+ /home/milovidov/ClickHouse/build_gcc9/../dbms/IO/BufferBase.h:99
+
+
+read
+
+DB::ReadBufferFromFileDescriptor::nextImpl()
+ /home/milovidov/ClickHouse/build_gcc9/../dbms/IO/ReadBufferFromFileDescriptor.cpp:56
+DB::CompressedReadBufferBase::readCompressedData(unsigned long&, unsigned long&)
+ /home/milovidov/ClickHouse/build_gcc9/../dbms/IO/ReadBuffer.h:54
+DB::CompressedReadBufferFromFile::nextImpl()
+ /home/milovidov/ClickHouse/build_gcc9/../dbms/Compression/CompressedReadBufferFromFile.cpp:22
+void DB::deserializeBinarySSE2<4>(DB::PODArray, 15ul, 16ul>&, DB::PODArray, 15ul, 16ul>&, DB::ReadBuffer&, unsigned long)
+ /home/milovidov/ClickHouse/build_gcc9/../dbms/IO/ReadBuffer.h:53
+DB::DataTypeString::deserializeBinaryBulk(DB::IColumn&, DB::ReadBuffer&, unsigned long, double) const
+ /home/milovidov/ClickHouse/build_gcc9/../dbms/DataTypes/DataTypeString.cpp:202
+DB::MergeTreeReader::readData(std::__cxx11::basic_string, std::allocator > const&, DB::IDataType const&, DB::IColumn&, unsigned long, bool, unsigned long, bool)
+ /home/milovidov/ClickHouse/build_gcc9/../dbms/Storages/MergeTree/MergeTreeReader.cpp:232
+DB::MergeTreeReader::readRows(unsigned long, bool, unsigned long, DB::Block&)
+ /home/milovidov/ClickHouse/build_gcc9/../dbms/Storages/MergeTree/MergeTreeReader.cpp:111
+DB::MergeTreeRangeReader::DelayedStream::finalize(DB::Block&)
+ /home/milovidov/ClickHouse/build_gcc9/../dbms/Storages/MergeTree/MergeTreeRangeReader.cpp:35
+DB::MergeTreeRangeReader::startReadingChain(unsigned long, std::vector >&)
+ /home/milovidov/ClickHouse/build_gcc9/../dbms/Storages/MergeTree/MergeTreeRangeReader.cpp:219
+DB::MergeTreeRangeReader::read(unsigned long, std::vector >&)
+ /usr/local/include/c++/9.1.0/bits/stl_vector.h:108
+DB::MergeTreeRangeReader::read(unsigned long, std::vector >&)
+ /usr/local/include/c++/9.1.0/bits/stl_vector.h:108
+DB::MergeTreeBaseSelectBlockInputStream::readFromPartImpl()
+ /home/milovidov/ClickHouse/build_gcc9/../dbms/Storages/MergeTree/MergeTreeBaseSelectBlockInputStream.cpp:158
+DB::MergeTreeBaseSelectBlockInputStream::readImpl()
+ /usr/local/include/c++/9.1.0/bits/stl_vector.h:108
+DB::IBlockInputStream::read()
+ /usr/local/include/c++/9.1.0/bits/stl_vector.h:108
+DB::ExpressionBlockInputStream::readImpl()
+ /home/milovidov/ClickHouse/build_gcc9/../dbms/DataStreams/ExpressionBlockInputStream.cpp:34
+DB::IBlockInputStream::read()
+ /usr/local/include/c++/9.1.0/bits/stl_vector.h:108
+DB::PartialSortingBlockInputStream::readImpl()
+ /home/milovidov/ClickHouse/build_gcc9/../dbms/DataStreams/PartialSortingBlockInputStream.cpp:13
+DB::IBlockInputStream::read()
+ /usr/local/include/c++/9.1.0/bits/stl_vector.h:108
+DB::ParallelInputsProcessor::loop(unsigned long)
+ /usr/local/include/c++/9.1.0/bits/atomic_base.h:419
+DB::ParallelInputsProcessor::thread(std::shared_ptr, unsigned long)
+ /home/milovidov/ClickHouse/build_gcc9/../dbms/DataStreams/ParallelInputsProcessor.h:215
+ThreadFromGlobalPool::ThreadFromGlobalPool::*)(std::shared_ptr, unsigned long), DB::ParallelInputsProcessor*, std::shared_ptr, unsigned long&>(void (DB::ParallelInputsProcessor::*&&)(std::shared_ptr, unsigned long), DB::ParallelInputsProcessor*&&, std::shared_ptr&&, unsigned long&)::{lambda()#1}::operator()() const
+ /usr/local/include/c++/9.1.0/bits/shared_ptr_base.h:729
+ThreadPoolImpl::worker(std::_List_iterator)
+ /usr/local/include/c++/9.1.0/bits/unique_lock.h:69
+execute_native_thread_routine
+ /home/milovidov/ClickHouse/ci/workspace/gcc/gcc-build/x86_64-pc-linux-gnu/libstdc++-v3/include/bits/unique_ptr.h:81
+start_thread
+
+__clone
+
+
+Row 7:
+──────
+count(): 1034
+sym: StackTrace::StackTrace(ucontext_t const&)
+ /home/milovidov/ClickHouse/build_gcc9/../dbms/Common/StackTrace.cpp:208
+DB::(anonymous namespace)::writeTraceInfo(DB::TimerType, int, siginfo_t*, void*) [clone .isra.0]
+ /home/milovidov/ClickHouse/build_gcc9/../dbms/IO/BufferBase.h:99
+
+
+DB::VolnitskyBase >::search(unsigned char const*, unsigned long) const
+ /opt/milovidov/ClickHouse/build_gcc9/programs/clickhouse
+DB::MatchImpl::vector_constant(DB::PODArray, 15ul, 16ul> const&, DB::PODArray, 15ul, 16ul> const&, std::__cxx11::basic_string, std::allocator > const&, DB::PODArray, 15ul, 16ul>&)
+ /opt/milovidov/ClickHouse/build_gcc9/programs/clickhouse
+DB::FunctionsStringSearch, DB::NameLike>::executeImpl(DB::Block&, std::vector > const&, unsigned long, unsigned long)
+ /opt/milovidov/ClickHouse/build_gcc9/programs/clickhouse
+DB::PreparedFunctionImpl::execute(DB::Block&, std::vector > const&, unsigned long, unsigned long, bool)
+ /home/milovidov/ClickHouse/build_gcc9/../dbms/Functions/IFunction.cpp:464
+DB::ExpressionAction::execute(DB::Block&, bool) const
+ /usr/local/include/c++/9.1.0/bits/stl_vector.h:677
+DB::ExpressionActions::execute(DB::Block&, bool) const
+ /home/milovidov/ClickHouse/build_gcc9/../dbms/Interpreters/ExpressionActions.cpp:739
+DB::MergeTreeRangeReader::executePrewhereActionsAndFilterColumns(DB::MergeTreeRangeReader::ReadResult&)
+ /home/milovidov/ClickHouse/build_gcc9/../dbms/Storages/MergeTree/MergeTreeRangeReader.cpp:660
+DB::MergeTreeRangeReader::read(unsigned long, std::vector >&)
+ /home/milovidov/ClickHouse/build_gcc9/../dbms/Storages/MergeTree/MergeTreeRangeReader.cpp:546
+DB::MergeTreeRangeReader::read(unsigned long, std::vector >&)
+ /usr/local/include/c++/9.1.0/bits/stl_vector.h:108
+DB::MergeTreeBaseSelectBlockInputStream::readFromPartImpl()
+ /home/milovidov/ClickHouse/build_gcc9/../dbms/Storages/MergeTree/MergeTreeBaseSelectBlockInputStream.cpp:158
+DB::MergeTreeBaseSelectBlockInputStream::readImpl()
+ /usr/local/include/c++/9.1.0/bits/stl_vector.h:108
+DB::IBlockInputStream::read()
+ /usr/local/include/c++/9.1.0/bits/stl_vector.h:108
+DB::ExpressionBlockInputStream::readImpl()
+ /home/milovidov/ClickHouse/build_gcc9/../dbms/DataStreams/ExpressionBlockInputStream.cpp:34
+DB::IBlockInputStream::read()
+ /usr/local/include/c++/9.1.0/bits/stl_vector.h:108
+DB::PartialSortingBlockInputStream::readImpl()
+ /home/milovidov/ClickHouse/build_gcc9/../dbms/DataStreams/PartialSortingBlockInputStream.cpp:13
+DB::IBlockInputStream::read()
+ /usr/local/include/c++/9.1.0/bits/stl_vector.h:108
+DB::ParallelInputsProcessor::loop(unsigned long)
+ /usr/local/include/c++/9.1.0/bits/atomic_base.h:419
+DB::ParallelInputsProcessor::thread(std::shared_ptr, unsigned long)
+ /home/milovidov/ClickHouse/build_gcc9/../dbms/DataStreams/ParallelInputsProcessor.h:215
+ThreadFromGlobalPool::ThreadFromGlobalPool::*)(std::shared_ptr, unsigned long), DB::ParallelInputsProcessor*, std::shared_ptr, unsigned long&>(void (DB::ParallelInputsProcessor::*&&)(std::shared_ptr, unsigned long), DB::ParallelInputsProcessor*&&, std::shared_ptr&&, unsigned long&)::{lambda()#1}::operator()() const
+ /usr/local/include/c++/9.1.0/bits/shared_ptr_base.h:729
+ThreadPoolImpl::worker(std::_List_iterator)
+ /usr/local/include/c++/9.1.0/bits/unique_lock.h:69
+execute_native_thread_routine
+ /home/milovidov/ClickHouse/ci/workspace/gcc/gcc-build/x86_64-pc-linux-gnu/libstdc++-v3/include/bits/unique_ptr.h:81
+start_thread
+
+__clone
+
+
+Row 8:
+──────
+count(): 989
+sym: StackTrace::StackTrace(ucontext_t const&)
+ /home/milovidov/ClickHouse/build_gcc9/../dbms/Common/StackTrace.cpp:208
+DB::(anonymous namespace)::writeTraceInfo(DB::TimerType, int, siginfo_t*, void*) [clone .isra.0]
+ /home/milovidov/ClickHouse/build_gcc9/../dbms/IO/BufferBase.h:99
+
+
+__lll_lock_wait
+
+pthread_mutex_lock
+
+DB::MergeTreeReaderStream::loadMarks()
+ /usr/local/include/c++/9.1.0/bits/std_mutex.h:103
+DB::MergeTreeReaderStream::MergeTreeReaderStream(std::__cxx11::basic_string, std::allocator > const&, std::__cxx11::basic_string, std::allocator > const&, unsigned long, std::vector > const&, DB::MarkCache*, bool, DB::UncompressedCache*, unsigned long, unsigned long, unsigned long, DB::MergeTreeIndexGranularityInfo const*, std::function const&, int)
+ /home/milovidov/ClickHouse/build_gcc9/../dbms/Storages/MergeTree/MergeTreeReaderStream.cpp:107
+std::_Function_handler > const&), DB::MergeTreeReader::addStreams(std::__cxx11::basic_string