Commit Graph

467 Commits

Author SHA1 Message Date
Igor Nikonov
7fd851b4e9 Merge remote-tracking branch 'origin/master' into pr-custom-key-failover 2024-01-08 20:43:20 +00:00
Nikolai Kochetov
1b20ce5162 Cleanup 2024-01-02 17:50:06 +00:00
Nikolai Kochetov
8936c8376a Use predicate in getTaskIteratorExtension. 2024-01-02 17:14:16 +00:00
Igor Nikonov
bee15325fc Merge remote-tracking branch 'origin/master' into pr-custom-key-failover 2023-12-26 21:56:46 +00:00
Raúl Marín
e87b9751bd Cleanup some known short messages 2023-12-26 12:58:50 +01:00
Igor Nikonov
52db759c92 Merge remote-tracking branch 'origin/master' into pr-custom-key-failover 2023-12-18 11:59:13 +00:00
Alexey Milovidov
58396c5546
Merge pull request #57218 from tntnatbry/issue-43666
Issue 43666: Add skip_unavailable_shards as a setting for Distributed table.
2023-12-18 04:48:57 +01:00
Igor Nikonov
f7e6a17052 Merge remote-tracking branch 'origin/master' into pr-custom-key-failover 2023-12-10 20:21:47 +00:00
Gagan Goel
e547db0a8c Issue 43666: Add skip_unavailable_shards as a setting for Distributed table.
This setting, when enabled (disabled by default), allows ClickHouse to
silently skip unavailable shards of a Distributed table during a query
execution, instead of throwing an exception to the client.
2023-12-08 15:43:59 -05:00
zhongyuankai
7b0f8d44e8
Make DirectoryMonitor handle cluster node list change (#42826) 2023-12-08 14:41:51 +01:00
Igor Nikonov
87a4c0bde4 Simple cleanup in distributed (while dealing with parallel replicas) 2023-12-05 12:25:02 +00:00
Igor Nikonov
a763c55c2e Fix 2023-11-30 22:42:57 +00:00
Igor Nikonov
45ebe78d32 Cleanup 2023-11-30 21:40:45 +00:00
Igor Nikonov
8ba5c32f1c Remove commented code 2023-11-30 18:42:45 +00:00
Igor Nikonov
59daabe95a Some refactoring and cleanup
+ change load balancing for custom key w/o hedged connections
  (need the test though)
2023-11-30 14:08:01 +00:00
Igor Nikonov
e823046646 Fixes 2023-11-27 15:27:04 +00:00
Igor Nikonov
63adf4d20c Remove commented code 2023-11-26 22:07:57 +00:00
Igor Nikonov
774347d231 Parallel replicas custom key: skip unavailable replicas 2023-11-26 21:38:49 +00:00
Igor Nikonov
63a1625b77 Merge remote-tracking branch 'origin/master' into pr-cleanup-narrow-dependency 2023-11-21 16:05:48 +00:00
Igor Nikonov
66f6a6575f Cleanup iteration: settings usage 2023-11-21 13:29:04 +00:00
Igor Nikonov
83c6ac4f6c Remove storage as parameter dependency, only columns are needed 2023-11-21 12:14:12 +00:00
Alexey Milovidov
d56cbda185 Add metrics for the number of queued jobs, which is useful for the IO thread pool 2023-11-18 19:07:59 +01:00
Nikolai Kochetov
6ef5c4cafe
Merge pull request #56522 from kitaisreal/analyzer-support-explain-estimate
Analyzer support EXPLAIN ESTIMATE
2023-11-15 13:17:31 +01:00
Sema Checherinda
b9cc1580e0
Merge pull request #56367 from canhld94/ch_table_reinit_new_disk
Adding new disk to storage configuration without restart
2023-11-14 15:54:22 +01:00
Maksim Kita
6fb7d44b62 Analyzer support EXPLAIN ESTIMATE 2023-11-09 19:43:14 +03:00
kssenii
f2c0434c4d Merge remote-tracking branch 'origin/master' into minor-improvements-for-s3-queue 2023-11-06 15:51:11 +01:00
kssenii
da21413354 Better shutdown 2023-11-06 15:47:57 +01:00
Duc Canh Le
4c21ba7b6f tables auto initialize new disks without restart
Signed-off-by: Duc Canh Le <duccanh.le@ahrefs.com>
2023-11-06 10:26:48 +00:00
Azat Khuzhin
c25d6cd624
Rename directory monitor concept into background INSERT (#55978)
* Limit log frequence for "Skipping send data over distributed table" message

After SYSTEM STOP DISTRIBUTED SENDS it will constantly print this
message.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>

* Rename directory monitor concept into async INSERT

Rename the following query settings (with preserving backward
compatiblity, by keeping old name as an alias):
- distributed_directory_monitor_sleep_time_ms -> distributed_async_insert_sleep_time_ms
- distributed_directory_monitor_max_sleep_time_ms -> distributed_async_insert_max_sleep_time_ms
- distributed_directory_monitor_batch -> distributed_async_insert_batch_inserts
- distributed_directory_monitor_split_batch_on_failure -> distributed_async_insert_split_batch_on_failure

Rename the following table settings (with preserving backward
compatiblity, by keeping old name as an alias):
- monitor_batch_inserts -> async_insert_batch
- monitor_split_batch_on_failure -> async_insert_split_batch_on_failure
- directory_monitor_sleep_time_ms -> async_insert_sleep_time_ms
- directory_monitor_max_sleep_time_ms -> async_insert_max_sleep_time_ms

And also update all the references:

    $ gg -e directory_monitor_ -e monitor_ tests docs | cut -d: -f1 | sort -u | xargs sed -e 's/distributed_directory_monitor_sleep_time_ms/distributed_async_insert_sleep_time_ms/g' -e 's/distributed_directory_monitor_max_sleep_time_ms/distributed_async_insert_max_sleep_time_ms/g' -e 's/distributed_directory_monitor_batch_inserts/distributed_async_insert_batch/g' -e 's/distributed_directory_monitor_split_batch_on_failure/distributed_async_insert_split_batch_on_failure/g' -e 's/monitor_batch_inserts/async_insert_batch/g' -e 's/monitor_split_batch_on_failure/async_insert_split_batch_on_failure/g' -e 's/monitor_sleep_time_ms/async_insert_sleep_time_ms/g' -e 's/monitor_max_sleep_time_ms/async_insert_max_sleep_time_ms/g' -i

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>

* Rename async_insert for Distributed into background_insert

This will avoid amigibuity between general async INSERT's and INSERT
into Distributed, which are indeed background, so new term express it
even better.

Mostly done with:

    $ git di HEAD^ --name-only | xargs sed -i -e 's/distributed_async_insert/distributed_background_insert/g' -e 's/async_insert_batch/background_insert_batch/g' -e 's/async_insert_split_batch_on_failure/background_insert_split_batch_on_failure/g' -e 's/async_insert_sleep_time_ms/background_insert_sleep_time_ms/g' -e 's/async_insert_max_sleep_time_ms/background_insert_max_sleep_time_ms/g'

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>

* Mark 02417_opentelemetry_insert_on_distributed_table as long

CI: https://s3.amazonaws.com/clickhouse-test-reports/55978/7a6abb03a0b507e29e999cb7e04f246a119c6f28/stateless_tests_flaky_check__asan_.html
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>

---------

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-11-01 15:09:39 +01:00
Nikolai Kochetov
cc3c038394 Fixing test. 2023-10-30 16:29:22 +00:00
Nikolai Kochetov
c5100cd9b1 Fixing build 2023-10-04 09:01:29 +00:00
Nikolai Kochetov
d944b59902 Merge branch 'master' into planner-prepare-filters-for-analysis-2 2023-10-03 14:28:16 +00:00
Nikolai Kochetov
3b757d60a2 Remove commented code. 2023-10-03 14:22:20 +00:00
George Gamezardashvili
0ce30ab6d5
SSH keys authentication (#41109)
Added new type of authentication based on SSH keys. It works only for Native TCP protocol.

Co-authored-by: Nikita Mikhaylov <nikitamikhaylov@clickhouse.com>
Co-authored-by: Robert Schulze <robert@clickhouse.com>
2023-09-26 17:50:19 +02:00
SmitaRKulkarni
d8adf05de2
Added a new column _block_number (#47532)
Added a new virtual column _block_number which is persisted on merges when allow_experimental_block_number_column is enabled
2023-09-20 11:31:12 +02:00
vdimir
1aa18e0eb6
Analyzer: Remove constants from header in StorageDistributed 2023-09-14 16:44:18 +00:00
Nikolai Kochetov
903c966cc8
Merge branch 'master' into planner-prepare-filters-for-analysis-2 2023-09-11 16:14:03 +02:00
Igor Nikonov
2fdc700da2 Style 2023-09-09 13:33:18 +00:00
Igor Nikonov
5470a5b60f Fix style 2023-09-09 06:58:18 +00:00
Igor Nikonov
7a396139df Cleanup: unnecessary SelectQueryInfo usage around distributed 2023-09-08 21:53:38 +00:00
Alexander Tokmakov
d41eca1dcc rename new method 2023-08-28 16:01:00 +02:00
Alexander Tokmakov
9ab545e28c do not wait for flush on shutdown 2023-08-25 19:09:10 +02:00
Nikolai Kochetov
4e4620250d Fixing clang tidy. 2023-08-25 09:51:52 +00:00
Nikolai Kochetov
506c5667a7 Remove comment. 2023-08-24 18:21:41 +00:00
Nikolai Kochetov
9737dbca75 Fix window functions. 2023-08-24 16:37:27 +00:00
Nikolai Kochetov
cb851fcee0 Cleanup. 2023-08-24 11:07:17 +00:00
Nikolai Kochetov
33b8b93d1b Re-implement getOptimizedQueryProcessingStage with analyzer. 2023-08-24 11:07:17 +00:00
Nikolai Kochetov
fc90a1a0bd Fix some skip_unused_shards tests. 2023-08-24 11:07:17 +00:00
Nikolai Kochetov
26e0ad8d72 Re-impl evaluateExpressionOverConstantCondition [part 1] 2023-08-24 11:07:17 +00:00
Azat Khuzhin
17ca2661a1 Add ability to turn off flush of Distributed on DETACH/DROP/server shutdown
Sometimes you can have tons of data there, i.e. few TiBs, and sending
them on server shutdown does not looks sane (maybe there is a bug and
you need to update/restart to fix flushing).

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-08-17 08:58:06 +02:00