mirror of
https://github.com/ClickHouse/ClickHouse.git
synced 2024-09-20 08:40:50 +00:00
Merge branch 'master' into yandex-to-clickhouse-in-configs
This commit is contained in:
mergify[bot]
GitHub
commit
fd39fc58d7
2
AUTHORS
2
AUTHORS
@ -1,2 +1,2 @@
|
||||
To see the list of authors who created the source code of ClickHouse, published and distributed by YANDEX LLC as the owner,
|
||||
To see the list of authors who created the source code of ClickHouse, published and distributed by ClickHouse, Inc. as the owner,
|
||||
run "SELECT * FROM system.contributors;" query on any ClickHouse server.
|
||||
|
||||
@ -3604,6 +3604,18 @@ Possible values:
|
||||
|
||||
Default value: `1000`.
|
||||
|
||||
## short_circuit_function_evaluation {#short-circuit-function-evaluation}
|
||||
|
||||
Allows calculating the [if](../../sql-reference/functions/conditional-functions.md#if), [multiIf](../../sql-reference/functions/conditional-functions.md#multiif), [and](../../sql-reference/functions/logical-functions.md#logical-and-function), and [or](../../sql-reference/functions/logical-functions.md#logical-or-function) functions according to a [short scheme](https://en.wikipedia.org/wiki/Short-circuit_evaluation). This helps optimize the execution of complex expressions in these functions and prevent possible exceptions (such as division by zero when it is not expected).
|
||||
|
||||
Possible values:
|
||||
|
||||
- `enable` — Enables short-circuit function evaluation for functions that are suitable for it (can throw an exception or computationally heavy).
|
||||
- `force_enable` — Enables short-circuit function evaluation for all functions.
|
||||
- `disable` — Disables short-circuit function evaluation.
|
||||
|
||||
Default value: `enable`.
|
||||
|
||||
## max_hyperscan_regexp_length {#max-hyperscan-regexp-length}
|
||||
|
||||
Defines the maximum length for each regular expression in the [hyperscan multi-match functions](../../sql-reference/functions/string-search-functions.md#multimatchanyhaystack-pattern1-pattern2-patternn).
|
||||
@ -3629,7 +3641,6 @@ Result:
|
||||
┌─multiMatchAny('abcd', ['ab', 'bcd', 'c', 'd'])─┐
|
||||
│ 1 │
|
||||
└────────────────────────────────────────────────┘
|
||||
|
||||
```
|
||||
|
||||
Query:
|
||||
@ -3648,7 +3659,6 @@ Exception: Regexp length too large.
|
||||
|
||||
- [max_hyperscan_regexp_total_length](#max-hyperscan-regexp-total-length)
|
||||
|
||||
|
||||
## max_hyperscan_regexp_total_length {#max-hyperscan-regexp-total-length}
|
||||
|
||||
Sets the maximum length total of all regular expressions in each [hyperscan multi-match function](../../sql-reference/functions/string-search-functions.md#multimatchanyhaystack-pattern1-pattern2-patternn).
|
||||
|
||||
@ -12,11 +12,13 @@ Controls conditional branching. Unlike most systems, ClickHouse always evaluate
|
||||
**Syntax**
|
||||
|
||||
``` sql
|
||||
SELECT if(cond, then, else)
|
||||
if(cond, then, else)
|
||||
```
|
||||
|
||||
If the condition `cond` evaluates to a non-zero value, returns the result of the expression `then`, and the result of the expression `else`, if present, is skipped. If the `cond` is zero or `NULL`, then the result of the `then` expression is skipped and the result of the `else` expression, if present, is returned.
|
||||
|
||||
You can use the [short_circuit_function_evaluation](../../operations/settings/settings.md#short-circuit-function-evaluation) setting to calculate the `if` function according to a short scheme. If this setting is enabled, `then` expression is evaluated only on rows where `cond` is true, `else` expression – where `cond` is false. For example, an exception about division by zero is not thrown when executing the query `SELECT if(number = 0, 0, intDiv(42, number)) FROM numbers(10)`, because `intDiv(42, number)` will be evaluated only for numbers that doesn't satisfy condition `number = 0`.
|
||||
|
||||
**Arguments**
|
||||
|
||||
- `cond` – The condition for evaluation that can be zero or not. The type is UInt8, Nullable(UInt8) or NULL.
|
||||
@ -115,9 +117,15 @@ Returns `then` if the `cond` evaluates to be true (greater than zero), otherwise
|
||||
|
||||
Allows you to write the [CASE](../../sql-reference/operators/index.md#operator_case) operator more compactly in the query.
|
||||
|
||||
Syntax: `multiIf(cond_1, then_1, cond_2, then_2, ..., else)`
|
||||
**Syntax**
|
||||
|
||||
**Arguments:**
|
||||
``` sql
|
||||
multiIf(cond_1, then_1, cond_2, then_2, ..., else)
|
||||
```
|
||||
|
||||
You can use the [short_circuit_function_evaluation](../../operations/settings/settings.md#short-circuit-function-evaluation) setting to calculate the `multiIf` function according to a short scheme. If this setting is enabled, `then_i` expression is evaluated only on rows where `((NOT cond_1) AND (NOT cond_2) AND ... AND (NOT cond_{i-1}) AND cond_i)` is true, `cond_i` will be evaluated only on rows where `((NOT cond_1) AND (NOT cond_2) AND ... AND (NOT cond_{i-1}))` is true. For example, an exception about division by zero is not thrown when executing the query `SELECT multiIf(number = 2, intDiv(1, number), number = 5) FROM numbers(10)`.
|
||||
|
||||
**Arguments**
|
||||
|
||||
- `cond_N` — The condition for the function to return `then_N`.
|
||||
- `then_N` — The result of the function when executed.
|
||||
@ -201,4 +209,3 @@ FROM LEFT_RIGHT
|
||||
│ 4 │ ᴺᵁᴸᴸ │ Both equal │
|
||||
└──────┴───────┴──────────────────┘
|
||||
```
|
||||
|
||||
|
||||
@ -19,6 +19,8 @@ Calculates the result of the logical conjunction between two or more values. Cor
|
||||
and(val1, val2...)
|
||||
```
|
||||
|
||||
You can use the [short_circuit_function_evaluation](../../operations/settings/settings.md#short-circuit-function-evaluation) setting to calculate the `and` function according to a short scheme. If this setting is enabled, `vali` is evaluated only on rows where `(val1 AND val2 AND ... AND val{i-1})` is true. For example, an exception about division by zero is not thrown when executing the query `SELECT and(number = 2, intDiv(1, number)) FROM numbers(10)`.
|
||||
|
||||
**Arguments**
|
||||
|
||||
- `val1, val2, ...` — List of at least two values. [Int](../../sql-reference/data-types/int-uint.md), [UInt](../../sql-reference/data-types/int-uint.md), [Float](../../sql-reference/data-types/float.md) or [Nullable](../../sql-reference/data-types/nullable.md).
|
||||
@ -68,9 +70,11 @@ Calculates the result of the logical disjunction between two or more values. Cor
|
||||
**Syntax**
|
||||
|
||||
``` sql
|
||||
and(val1, val2...)
|
||||
or(val1, val2...)
|
||||
```
|
||||
|
||||
You can use the [short_circuit_function_evaluation](../../operations/settings/settings.md#short-circuit-function-evaluation) setting to calculate the `or` function according to a short scheme. If this setting is enabled, `vali` is evaluated only on rows where `((NOT val1) AND (NOT val2) AND ... AND (NOT val{i-1}))` is true. For example, an exception about division by zero is not thrown when executing the query `SELECT or(number = 0, intDiv(1, number) != 0) FROM numbers(10)`.
|
||||
|
||||
**Arguments**
|
||||
|
||||
- `val1, val2, ...` — List of at least two values. [Int](../../sql-reference/data-types/int-uint.md), [UInt](../../sql-reference/data-types/int-uint.md), [Float](../../sql-reference/data-types/float.md) or [Nullable](../../sql-reference/data-types/nullable.md).
|
||||
|
||||
@ -176,7 +176,7 @@ roundBankers(4.5) = 4
|
||||
roundBankers(3.55, 1) = 3.6
|
||||
roundBankers(3.65, 1) = 3.6
|
||||
roundBankers(10.35, 1) = 10.4
|
||||
roundBankers(10.755, 2) = 11,76
|
||||
roundBankers(10.755, 2) = 10.76
|
||||
```
|
||||
|
||||
**See Also**
|
||||
|
||||
@ -104,6 +104,28 @@ There are many nuances to processing `NULL`. For example, if at least one of the
|
||||
|
||||
In queries, you can check `NULL` using the [IS NULL](../sql-reference/operators/index.md#operator-is-null) and [IS NOT NULL](../sql-reference/operators/index.md) operators and the related functions `isNull` and `isNotNull`.
|
||||
|
||||
### Heredoc {#heredeoc}
|
||||
|
||||
A [heredoc](https://en.wikipedia.org/wiki/Here_document) is a way to define a string (often multiline), while maintaining the original formatting. A heredoc is defined as a custom string literal, placed between two `$` symbols, for example `$heredoc$`. A value between two heredocs is processed "as-is".
|
||||
|
||||
You can use a heredoc to embed snippets of SQL, HTML, or XML code, etc.
|
||||
|
||||
**Example**
|
||||
|
||||
Query:
|
||||
|
||||
```sql
|
||||
SELECT $smth$SHOW CREATE VIEW my_view$smth$;
|
||||
```
|
||||
|
||||
Result:
|
||||
|
||||
```text
|
||||
┌─'SHOW CREATE VIEW my_view'─┐
|
||||
│ SHOW CREATE VIEW my_view │
|
||||
└────────────────────────────┘
|
||||
```
|
||||
|
||||
## Functions {#functions}
|
||||
|
||||
Function calls are written like an identifier with a list of arguments (possibly empty) in round brackets. In contrast to standard SQL, the brackets are required, even for an empty argument list. Example: `now()`.
|
||||
|
||||
@ -3376,6 +3376,18 @@ SETTINGS index_granularity = 8192 │
|
||||
|
||||
Значение по умолчанию: `1000`.
|
||||
|
||||
## short_circuit_function_evaluation {#short-circuit-function-evaluation}
|
||||
|
||||
Позволяет вычислять функции [if](../../sql-reference/functions/conditional-functions.md#if), [multiIf](../../sql-reference/functions/conditional-functions.md#multiif), [and](../../sql-reference/functions/logical-functions.md#logical-and-function) и [or](../../sql-reference/functions/logical-functions.md#logical-or-function) по [короткой схеме](https://ru-wikipedia-org.turbopages.org/ru.wikipedia.org/s/wiki/Вычисления_по_короткой_схеме). Это помогает оптимизировать выполнение сложных выражений в этих функциях и предотвратить возможные исключения (например, деление на ноль, когда оно не ожидается).
|
||||
|
||||
Возможные значения:
|
||||
|
||||
- `enable` — по короткой схеме вычисляются функции, которые подходят для этого (могут сгенерировать исключение или требуют сложных вычислений).
|
||||
- `force_enable` — все функции вычисляются по короткой схеме.
|
||||
- `disable` — вычисление функций по короткой схеме отключено.
|
||||
|
||||
Значение по умолчанию: `enable`.
|
||||
|
||||
## max_hyperscan_regexp_length {#max-hyperscan-regexp-length}
|
||||
|
||||
Задает максимальную длину каждого регулярного выражения в [hyperscan-функциях](../../sql-reference/functions/string-search-functions.md#multimatchanyhaystack-pattern1-pattern2-patternn) поиска множественных совпадений в строке.
|
||||
|
||||
@ -12,11 +12,13 @@ toc_title: "Условные функции"
|
||||
**Синтаксис**
|
||||
|
||||
``` sql
|
||||
SELECT if(cond, then, else)
|
||||
if(cond, then, else)
|
||||
```
|
||||
|
||||
Если условие `cond` не равно нулю, то возвращается результат выражения `then`. Если условие `cond` равно нулю или является NULL, то результат выражения `then` пропускается и возвращается результат выражения `else`.
|
||||
|
||||
Чтобы вычислять функцию `if` по короткой схеме, используйте настройку [short_circuit_function_evaluation](../../operations/settings/settings.md#short-circuit-function-evaluation). Если настройка включена, то выражение `then` вычисляется только для строк, где условие `cond` верно, а выражение `else` – для строк, где условие `cond` неверно. Например, при выполнении запроса `SELECT if(number = 0, 0, intDiv(42, number)) FROM numbers(10)` не будет сгенерировано исключение из-за деления на ноль, так как `intDiv(42, number)` будет вычислено только для чисел, которые не удовлетворяют условию `number = 0`.
|
||||
|
||||
**Аргументы**
|
||||
|
||||
- `cond` – проверяемое условие. Может быть [UInt8](../../sql-reference/functions/conditional-functions.md) или `NULL`.
|
||||
@ -77,7 +79,13 @@ SELECT if(0, plus(2, 2), plus(2, 6));
|
||||
|
||||
Позволяет более компактно записать оператор [CASE](../operators/index.md#operator_case) в запросе.
|
||||
|
||||
multiIf(cond_1, then_1, cond_2, then_2...else)
|
||||
**Синтаксис**
|
||||
|
||||
``` sql
|
||||
multiIf(cond_1, then_1, cond_2, then_2, ..., else)
|
||||
```
|
||||
|
||||
Чтобы вычислять функцию `multiIf` по короткой схеме, используйте настройку [short_circuit_function_evaluation](../../operations/settings/settings.md#short-circuit-function-evaluation). Если настройка включена, то выражение `then_i` вычисляется только для строк, где условие `((NOT cond_1) AND (NOT cond_2) AND ... AND (NOT cond_{i-1}) AND cond_i)` верно, `cond_i` вычисляется только для строк, где условие `((NOT cond_1) AND (NOT cond_2) AND ... AND (NOT cond_{i-1}))` верно. Например, при выполнении запроса `SELECT multiIf(number = 2, intDiv(1, number), number = 5) FROM numbers(10)` не будет сгенерировано исключение из-за деления на ноль.
|
||||
|
||||
**Аргументы**
|
||||
|
||||
@ -110,4 +118,3 @@ SELECT if(0, plus(2, 2), plus(2, 6));
|
||||
│ ᴺᵁᴸᴸ │
|
||||
└────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
|
||||
@ -19,6 +19,8 @@ toc_title: "Логические функции"
|
||||
and(val1, val2...)
|
||||
```
|
||||
|
||||
Чтобы вычислять функцию `and` по короткой схеме, используйте настройку [short_circuit_function_evaluation](../../operations/settings/settings.md#short-circuit-function-evaluation). Если настройка включена, то выражение `vali` вычисляется только для строк, где условие `(val1 AND val2 AND ... AND val{i-1})` верно. Например, при выполнении запроса `SELECT and(number = 2, intDiv(1, number)) FROM numbers(10)` не будет сгенерировано исключение из-за деления на ноль.
|
||||
|
||||
**Аргументы**
|
||||
|
||||
- `val1, val2, ...` — список из как минимум двух значений. [Int](../../sql-reference/data-types/int-uint.md), [UInt](../../sql-reference/data-types/int-uint.md), [Float](../../sql-reference/data-types/float.md) или [Nullable](../../sql-reference/data-types/nullable.md).
|
||||
@ -71,6 +73,8 @@ SELECT and(NULL, 1, 10, -2);
|
||||
and(val1, val2...)
|
||||
```
|
||||
|
||||
Чтобы вычислять функцию `or` по короткой схеме, используйте настройку [short_circuit_function_evaluation](../../operations/settings/settings.md#short-circuit-function-evaluation). Если настройка включена, то выражение `vali` вычисляется только для строк, где условие `((NOT val1) AND (NOT val2) AND ... AND (NOT val{i-1}))` верно. Например, при выполнении запроса `SELECT or(number = 0, intDiv(1, number) != 0) FROM numbers(10)` не будет сгенерировано исключение из-за деления на ноль.
|
||||
|
||||
**Аргументы**
|
||||
|
||||
- `val1, val2, ...` — список из как минимум двух значений. [Int](../../sql-reference/data-types/int-uint.md), [UInt](../../sql-reference/data-types/int-uint.md), [Float](../../sql-reference/data-types/float.md) или [Nullable](../../sql-reference/data-types/nullable.md).
|
||||
|
||||
@ -3,7 +3,7 @@ toc_priority: 31
|
||||
toc_title: "Синтаксис"
|
||||
---
|
||||
|
||||
# Синтаксис {#sintaksis}
|
||||
# Синтаксис {#syntax}
|
||||
|
||||
В системе есть два вида парсеров: полноценный парсер SQL (recursive descent parser) и парсер форматов данных (быстрый потоковый парсер).
|
||||
Во всех случаях кроме запроса INSERT, используется только полноценный парсер SQL.
|
||||
@ -21,11 +21,11 @@ INSERT INTO t VALUES (1, 'Hello, world'), (2, 'abc'), (3, 'def')
|
||||
|
||||
Далее пойдёт речь о полноценном парсере. О парсерах форматов, смотри раздел «Форматы».
|
||||
|
||||
## Пробелы {#probely}
|
||||
## Пробелы {#spaces}
|
||||
|
||||
Между синтаксическими конструкциями (в том числе, в начале и конце запроса) может быть расположено произвольное количество пробельных символов. К пробельным символам относятся пробел, таб, перевод строки, CR, form feed.
|
||||
|
||||
## Комментарии {#kommentarii}
|
||||
## Комментарии {#comments}
|
||||
|
||||
Поддерживаются комментарии в SQL-стиле и C-стиле.
|
||||
Комментарии в SQL-стиле: от `--` до конца строки. Пробел после `--` может не ставиться.
|
||||
@ -63,7 +63,7 @@ INSERT INTO t VALUES (1, 'Hello, world'), (2, 'abc'), (3, 'def')
|
||||
|
||||
Существуют: числовые, строковые, составные литералы и `NULL`.
|
||||
|
||||
### Числовые {#chislovye}
|
||||
### Числовые {#numeric}
|
||||
|
||||
Числовой литерал пытается распарситься:
|
||||
|
||||
@ -83,7 +83,7 @@ INSERT INTO t VALUES (1, 'Hello, world'), (2, 'abc'), (3, 'def')
|
||||
|
||||
Минимальный набор символов, которых вам необходимо экранировать в строковых литералах: `'` и `\`. Одинарная кавычка может быть экранирована одинарной кавычкой, литералы `'It\'s'` и `'It''s'` эквивалентны.
|
||||
|
||||
### Составные {#sostavnye}
|
||||
### Составные {#compound}
|
||||
|
||||
Поддерживаются конструкции для массивов: `[1, 2, 3]` и кортежей: `(1, 'Hello, world!', 2)`.
|
||||
На самом деле, это вовсе не литералы, а выражение с оператором создания массива и оператором создания кортежа, соответственно.
|
||||
@ -102,17 +102,39 @@ INSERT INTO t VALUES (1, 'Hello, world'), (2, 'abc'), (3, 'def')
|
||||
|
||||
В запросах можно проверить `NULL` с помощью операторов [IS NULL](operators/index.md#operator-is-null) и [IS NOT NULL](operators/index.md), а также соответствующих функций `isNull` и `isNotNull`.
|
||||
|
||||
## Функции {#funktsii}
|
||||
### Heredoc {#heredeoc}
|
||||
|
||||
Синтаксис [heredoc](https://ru.wikipedia.org/wiki/Heredoc-синтаксис) — это способ определения строк с сохранением исходного формата (часто с переносом строки). `Heredoc` задается как произвольный строковый литерал между двумя символами `$`, например `$heredoc$`. Значение между двумя `heredoc` обрабатывается "как есть".
|
||||
|
||||
Синтаксис `heredoc` часто используют для вставки кусков кода SQL, HTML, XML и т.п.
|
||||
|
||||
**Пример**
|
||||
|
||||
Запрос:
|
||||
|
||||
```sql
|
||||
SELECT $smth$SHOW CREATE VIEW my_view$smth$;
|
||||
```
|
||||
|
||||
Результат:
|
||||
|
||||
```text
|
||||
┌─'SHOW CREATE VIEW my_view'─┐
|
||||
│ SHOW CREATE VIEW my_view │
|
||||
└────────────────────────────┘
|
||||
```
|
||||
|
||||
## Функции {#functions}
|
||||
|
||||
Функции записываются как идентификатор со списком аргументов (возможно, пустым) в скобках. В отличие от стандартного SQL, даже в случае пустого списка аргументов, скобки обязательны. Пример: `now()`.
|
||||
Бывают обычные и агрегатные функции (смотрите раздел «Агрегатные функции»). Некоторые агрегатные функции могут содержать два списка аргументов в круглых скобках. Пример: `quantile(0.9)(x)`. Такие агрегатные функции называются «параметрическими», а первый список аргументов называется «параметрами». Синтаксис агрегатных функций без параметров ничем не отличается от обычных функций.
|
||||
|
||||
## Операторы {#operatory}
|
||||
## Операторы {#operators}
|
||||
|
||||
Операторы преобразуются в соответствующие им функции во время парсинга запроса, с учётом их приоритета и ассоциативности.
|
||||
Например, выражение `1 + 2 * 3 + 4` преобразуется в `plus(plus(1, multiply(2, 3)), 4)`.
|
||||
|
||||
## Типы данных и движки таблиц {#tipy-dannykh-i-dvizhki-tablits}
|
||||
## Типы данных и движки таблиц {#data_types-and-database-table-engines}
|
||||
|
||||
Типы данных и движки таблиц в запросе `CREATE` записываются также, как идентификаторы или также как функции. То есть, могут содержать или не содержать список аргументов в круглых скобках. Подробнее смотрите разделы «Типы данных», «Движки таблиц», «CREATE».
|
||||
|
||||
|
||||
@ -5,4 +5,4 @@ toc_priority: 82
|
||||
|
||||
# Что нового в ClickHouse?
|
||||
|
||||
Планы развития вкратце изложены [здесь](extended-roadmap.md), а новости по предыдущим релизам подробно описаны в [журнале изменений](changelog/index.md).
|
||||
Планы развития вкратце изложены [здесь](https://github.com/ClickHouse/ClickHouse/issues/17623), а новости по предыдущим релизам подробно описаны в [журнале изменений](changelog/index.md).
|
||||
|
||||
@ -8,6 +8,9 @@ import subprocess
|
||||
|
||||
|
||||
def test_single_page(input_path, lang):
|
||||
if not (lang == 'en' or lang == 'ru'):
|
||||
return
|
||||
|
||||
with open(input_path) as f:
|
||||
soup = bs4.BeautifulSoup(
|
||||
f,
|
||||
@ -33,11 +36,8 @@ def test_single_page(input_path, lang):
|
||||
logging.info('Link to nowhere: %s' % href)
|
||||
|
||||
if links_to_nowhere:
|
||||
if lang == 'en' or lang == 'ru':
|
||||
logging.error(f'Found {links_to_nowhere} links to nowhere in {lang}')
|
||||
# TODO: restore sys.exit(1) here
|
||||
else:
|
||||
logging.warning(f'Found {links_to_nowhere} links to nowhere in {lang}')
|
||||
logging.error(f'Found {links_to_nowhere} links to nowhere in {lang}')
|
||||
sys.exit(1)
|
||||
|
||||
if len(anchor_points) <= 10:
|
||||
logging.error('Html parsing is probably broken')
|
||||
|
||||
@ -102,7 +102,7 @@ namespace
|
||||
// This range is hardcoded below
|
||||
if (precision_param > 20 || precision_param < 12)
|
||||
throw Exception(
|
||||
"Parameter for aggregate function " + name + " is out or range: [12, 20].", ErrorCodes::ARGUMENT_OUT_OF_BOUND);
|
||||
"Parameter for aggregate function " + name + " is out of range: [12, 20].", ErrorCodes::ARGUMENT_OUT_OF_BOUND);
|
||||
precision = precision_param;
|
||||
}
|
||||
|
||||
|
||||
@ -515,8 +515,8 @@ class IColumn;
|
||||
M(Milliseconds, async_insert_busy_timeout_ms, 200, "Maximum time to wait before dumping collected data per query since the first data appeared", 0) \
|
||||
M(Milliseconds, async_insert_stale_timeout_ms, 0, "Maximum time to wait before dumping collected data per query since the last data appeared. Zero means no timeout at all", 0) \
|
||||
\
|
||||
M(Int64, remote_disk_read_backoff_threashold, 10000, "Max wait time when trying to read data for remote disk", 0) \
|
||||
M(Int64, remote_disk_read_backoff_max_tries, 5, "Max attempts to read with backoff", 0) \
|
||||
M(Int64, remote_fs_read_backoff_threshold, 10000, "Max wait time when trying to read data for remote disk", 0) \
|
||||
M(Int64, remote_fs_read_backoff_max_tries, 5, "Max attempts to read with backoff", 0) \
|
||||
\
|
||||
/** Experimental functions */ \
|
||||
M(Bool, allow_experimental_funnel_functions, false, "Enable experimental functions for funnel analysis.", 0) \
|
||||
|
||||
@ -112,23 +112,29 @@ public:
|
||||
const String & uri_,
|
||||
RemoteMetadata metadata_,
|
||||
ContextPtr context_,
|
||||
size_t buf_size_)
|
||||
size_t buf_size_,
|
||||
size_t backoff_threshold_,
|
||||
size_t max_tries_)
|
||||
: ReadIndirectBufferFromRemoteFS<ReadIndirectBufferFromWebServer>(metadata_)
|
||||
, uri(uri_)
|
||||
, context(context_)
|
||||
, buf_size(buf_size_)
|
||||
, backoff_threshold(backoff_threshold_)
|
||||
, max_tries(max_tries_)
|
||||
{
|
||||
}
|
||||
|
||||
std::unique_ptr<ReadIndirectBufferFromWebServer> createReadBuffer(const String & path) override
|
||||
{
|
||||
return std::make_unique<ReadIndirectBufferFromWebServer>(fs::path(uri) / path, context, buf_size);
|
||||
return std::make_unique<ReadIndirectBufferFromWebServer>(fs::path(uri) / path, context, buf_size, backoff_threshold, max_tries);
|
||||
}
|
||||
|
||||
private:
|
||||
String uri;
|
||||
ContextPtr context;
|
||||
size_t buf_size;
|
||||
size_t backoff_threshold;
|
||||
size_t max_tries;
|
||||
};
|
||||
|
||||
|
||||
@ -190,7 +196,8 @@ std::unique_ptr<ReadBufferFromFileBase> DiskWebServer::readFile(const String & p
|
||||
RemoteMetadata meta(path, remote_path);
|
||||
meta.remote_fs_objects.emplace_back(std::make_pair(remote_path, iter->second.size));
|
||||
|
||||
auto reader = std::make_unique<ReadBufferFromWebServer>(url, meta, getContext(), read_settings.remote_fs_buffer_size);
|
||||
auto reader = std::make_unique<ReadBufferFromWebServer>(url, meta, getContext(),
|
||||
read_settings.remote_fs_buffer_size, read_settings.remote_fs_backoff_threshold, read_settings.remote_fs_backoff_max_tries);
|
||||
return std::make_unique<SeekAvoidingReadBuffer>(std::move(reader), min_bytes_for_seek);
|
||||
}
|
||||
|
||||
|
||||
@ -22,18 +22,17 @@ namespace ErrorCodes
|
||||
|
||||
static const auto WAIT_MS = 10;
|
||||
|
||||
ReadIndirectBufferFromWebServer::ReadIndirectBufferFromWebServer(const String & url_,
|
||||
ContextPtr context_,
|
||||
size_t buf_size_)
|
||||
|
||||
ReadIndirectBufferFromWebServer::ReadIndirectBufferFromWebServer(
|
||||
const String & url_, ContextPtr context_, size_t buf_size_, size_t backoff_threshold_, size_t max_tries_)
|
||||
: BufferWithOwnMemory<SeekableReadBuffer>(buf_size_)
|
||||
, log(&Poco::Logger::get("ReadIndirectBufferFromWebServer"))
|
||||
, context(context_)
|
||||
, url(url_)
|
||||
, buf_size(buf_size_)
|
||||
, backoff_threshold_ms(backoff_threshold_)
|
||||
, max_tries(max_tries_)
|
||||
{
|
||||
const auto & settings = context->getSettingsRef();
|
||||
wait_threshold_ms = settings.remote_disk_read_backoff_threashold;
|
||||
max_tries = settings.remote_disk_read_backoff_max_tries;
|
||||
}
|
||||
|
||||
|
||||
@ -79,7 +78,7 @@ bool ReadIndirectBufferFromWebServer::nextImpl()
|
||||
WriteBufferFromOwnString error_msg;
|
||||
for (size_t i = 0; (i < max_tries) && !successful_read && !next_result; ++i)
|
||||
{
|
||||
while (milliseconds_to_wait < wait_threshold_ms)
|
||||
while (milliseconds_to_wait < backoff_threshold_ms)
|
||||
{
|
||||
try
|
||||
{
|
||||
|
||||
@ -18,7 +18,8 @@ class ReadIndirectBufferFromWebServer : public BufferWithOwnMemory<SeekableReadB
|
||||
public:
|
||||
explicit ReadIndirectBufferFromWebServer(const String & url_,
|
||||
ContextPtr context_,
|
||||
size_t buf_size_ = DBMS_DEFAULT_BUFFER_SIZE);
|
||||
size_t buf_size_ = DBMS_DEFAULT_BUFFER_SIZE,
|
||||
size_t backoff_threshold_ = 10000, size_t max_tries_ = 4);
|
||||
|
||||
bool nextImpl() override;
|
||||
|
||||
@ -39,7 +40,7 @@ private:
|
||||
|
||||
off_t offset = 0;
|
||||
|
||||
size_t wait_threshold_ms;
|
||||
size_t backoff_threshold_ms;
|
||||
size_t max_tries;
|
||||
};
|
||||
|
||||
|
||||
@ -66,6 +66,9 @@ struct ReadSettings
|
||||
/// For 'pread_threadpool' method. Lower is more priority.
|
||||
size_t priority = 0;
|
||||
|
||||
size_t remote_fs_backoff_threshold = 10000;
|
||||
size_t remote_fs_backoff_max_tries = 4;
|
||||
|
||||
ReadSettings adjustBufferSize(size_t file_size) const
|
||||
{
|
||||
ReadSettings res = *this;
|
||||
|
||||
@ -2844,6 +2844,9 @@ ReadSettings Context::getReadSettings() const
|
||||
res.local_fs_prefetch = settings.local_filesystem_read_prefetch;
|
||||
res.remote_fs_prefetch = settings.remote_filesystem_read_prefetch;
|
||||
|
||||
res.remote_fs_backoff_threshold = settings.remote_fs_read_backoff_threshold;
|
||||
res.remote_fs_backoff_max_tries = settings.remote_fs_read_backoff_max_tries;
|
||||
|
||||
res.local_fs_buffer_size = settings.max_read_buffer_size;
|
||||
res.direct_io_threshold = settings.min_bytes_to_use_direct_io;
|
||||
res.mmap_threshold = settings.min_bytes_to_use_mmap_io;
|
||||
|
||||
@ -961,7 +961,9 @@ void StorageReplicatedMergeTree::checkTableStructure(const String & zookeeper_pr
|
||||
const ColumnsDescription & old_columns = metadata_snapshot->getColumns();
|
||||
if (columns_from_zk != old_columns)
|
||||
{
|
||||
throw Exception("Table columns structure in ZooKeeper is different from local table structure", ErrorCodes::INCOMPATIBLE_COLUMNS);
|
||||
throw Exception(ErrorCodes::INCOMPATIBLE_COLUMNS,
|
||||
"Table columns structure in ZooKeeper is different from local table structure. Local columns:\n"
|
||||
"{}\nZookeeper columns:\n{}", old_columns.toString(), columns_from_zk.toString());
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
@ -6,7 +6,7 @@ author: '[Alexey Milovidov](https://github.com/alexey-milovidov)'
|
||||
tags: ['company', 'incorporation', 'yandex', 'community']
|
||||
---
|
||||
|
||||
Today I’m happy to announce **ClickHouse Inc.**, the new home of ClickHouse. The development team has moved from Yandex and joined ClickHouse Inc. to continue building the fastest (and the greatest) analytical database management system. The company has received nearly $50M in Series A funding led by Index Ventures and Benchmark with participation by Yandex N.V. and others. I created ClickHouse, Inc. with two co-founders, [Yury Izrailevsky](https://www.linkedin.com/in/yuryizrailevsky/) and [Aaron Katz](https://www.linkedin.com/in/aaron-k-5762094/). I will continue to lead the development of ClickHouse as Chief Technology Officer (CTO), Yury will run product and engineering, and Aaron will be CEO.
|
||||
Today I’m happy to announce **ClickHouse Inc.**, the new home of ClickHouse. The development team has moved from Yandex and joined ClickHouse Inc. to continue building the fastest (and the greatest) analytical database management system. The company has received nearly $50M in Series A funding led by Index Ventures and Benchmark with participation by Yandex N.V. and others. I created ClickHouse, Inc. with two co-founders, [Yury Izrailevsky](https://www.linkedin.com/in/yuryizrailevsky/) and [Aaron Katz](https://www.linkedin.com/in/aaron-katz-5762094/). I will continue to lead the development of ClickHouse as Chief Technology Officer (CTO), Yury will run product and engineering, and Aaron will be CEO.
|
||||
|
||||
## History of ClickHouse
|
||||
|
||||
@ -39,7 +39,7 @@ Lastly, ClickHouse was purpose-built from the beginning to:
|
||||
|
||||
Yandex N.V. is the largest internet company in Europe and employs over 14,000 people. They develop search, advertisement, and e-commerce services, ride tech and food tech solutions, self-driving cars... and also ClickHouse with a team of 15 engineers. It is hard to believe that we have managed to build a world-class leading analytical DBMS with such a small team while leveraging the global community. While this was barely enough to keep up with the development of the open-source product, everyone understands that the potential of ClickHouse technology highly outgrows such a small team.
|
||||
|
||||
We decided to unite the resources: take the team of core ClickHouse developers, bring in a world-class business team led by [Aaron Katz](https://www.linkedin.com/in/aaron-k-5762094/) and a cloud engineering team led by [Yury Izrailevsky](https://www.linkedin.com/in/yuryizrailevsky/), keep the power of open source, add the investment from the leading VC funds, and make an international company 100% focused on ClickHouse. I’m thrilled to announce ClickHouse, Inc.
|
||||
We decided to unite the resources: take the team of core ClickHouse developers, bring in a world-class business team led by [Aaron Katz](https://www.linkedin.com/in/aaron-katz-5762094/) and a cloud engineering team led by [Yury Izrailevsky](https://www.linkedin.com/in/yuryizrailevsky/), keep the power of open source, add the investment from the leading VC funds, and make an international company 100% focused on ClickHouse. I’m thrilled to announce ClickHouse, Inc.
|
||||
|
||||
## What’s Next?
|
||||
|
||||
|
||||
@ -25,7 +25,7 @@
|
||||
</div>
|
||||
<div class="col-lg-4 mb-3 mb-lg-0 text-center">
|
||||
|
||||
<a href=" https://www.linkedin.com/in/aaron-k-5762094/" target="_blank" class="photo-frame mx-auto" rel="external nofollow">
|
||||
<a href=" https://www.linkedin.com/in/aaron-katz-5762094/" target="_blank" class="photo-frame mx-auto" rel="external nofollow">
|
||||
<img src="/images/photos/aaron-katz.jpg">
|
||||
</a>
|
||||
<h3 class="display-4">
|
||||
|
||||
@ -41,7 +41,7 @@
|
||||
{% for code, name in config.extra.languages.items() %}
|
||||
<a class="dropdown-item{% if language == code %} disabled{% endif %}"
|
||||
href="/docs/{{ code }}/{{ page.url }}">
|
||||
<img src="/images/flags/{{ code }}.svg" alt="" title="" width="60" class="d-inline-block mr-2" />{{ name }}
|
||||
<img src="/images/flags/{{ code }}.svg" alt="" title="" width="32" class="d-inline-block mr-2" />{{ name }}
|
||||
</a>
|
||||
{% endfor %}
|
||||
</div>
|
||||
|
||||
Loading…
Reference in New Issue
Block a user