Commit Graph

244 Commits

Author SHA1 Message Date
Azat Khuzhin
c25d6cd624
Rename directory monitor concept into background INSERT (#55978)
* Limit log frequence for "Skipping send data over distributed table" message

After SYSTEM STOP DISTRIBUTED SENDS it will constantly print this
message.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>

* Rename directory monitor concept into async INSERT

Rename the following query settings (with preserving backward
compatiblity, by keeping old name as an alias):
- distributed_directory_monitor_sleep_time_ms -> distributed_async_insert_sleep_time_ms
- distributed_directory_monitor_max_sleep_time_ms -> distributed_async_insert_max_sleep_time_ms
- distributed_directory_monitor_batch -> distributed_async_insert_batch_inserts
- distributed_directory_monitor_split_batch_on_failure -> distributed_async_insert_split_batch_on_failure

Rename the following table settings (with preserving backward
compatiblity, by keeping old name as an alias):
- monitor_batch_inserts -> async_insert_batch
- monitor_split_batch_on_failure -> async_insert_split_batch_on_failure
- directory_monitor_sleep_time_ms -> async_insert_sleep_time_ms
- directory_monitor_max_sleep_time_ms -> async_insert_max_sleep_time_ms

And also update all the references:

    $ gg -e directory_monitor_ -e monitor_ tests docs | cut -d: -f1 | sort -u | xargs sed -e 's/distributed_directory_monitor_sleep_time_ms/distributed_async_insert_sleep_time_ms/g' -e 's/distributed_directory_monitor_max_sleep_time_ms/distributed_async_insert_max_sleep_time_ms/g' -e 's/distributed_directory_monitor_batch_inserts/distributed_async_insert_batch/g' -e 's/distributed_directory_monitor_split_batch_on_failure/distributed_async_insert_split_batch_on_failure/g' -e 's/monitor_batch_inserts/async_insert_batch/g' -e 's/monitor_split_batch_on_failure/async_insert_split_batch_on_failure/g' -e 's/monitor_sleep_time_ms/async_insert_sleep_time_ms/g' -e 's/monitor_max_sleep_time_ms/async_insert_max_sleep_time_ms/g' -i

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>

* Rename async_insert for Distributed into background_insert

This will avoid amigibuity between general async INSERT's and INSERT
into Distributed, which are indeed background, so new term express it
even better.

Mostly done with:

    $ git di HEAD^ --name-only | xargs sed -i -e 's/distributed_async_insert/distributed_background_insert/g' -e 's/async_insert_batch/background_insert_batch/g' -e 's/async_insert_split_batch_on_failure/background_insert_split_batch_on_failure/g' -e 's/async_insert_sleep_time_ms/background_insert_sleep_time_ms/g' -e 's/async_insert_max_sleep_time_ms/background_insert_max_sleep_time_ms/g'

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>

* Mark 02417_opentelemetry_insert_on_distributed_table as long

CI: https://s3.amazonaws.com/clickhouse-test-reports/55978/7a6abb03a0b507e29e999cb7e04f246a119c6f28/stateless_tests_flaky_check__asan_.html
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>

---------

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-11-01 15:09:39 +01:00
avogar
7cfdd88ccf Address comments 2023-09-15 09:32:41 +00:00
avogar
b9c28ef1f7 Fix for analyzer 2023-09-14 20:10:07 +00:00
avogar
93c49dfeaa Fix using structure from insertion tables in case of defaults and explicit insert columns 2023-09-14 19:31:12 +00:00
Alexey Milovidov
2c96580a77
Merge branch 'master' into concurrency-control-controllable 2023-07-04 23:16:04 +03:00
Antonio Andelic
b11f744252
Correctly disable async insert with deduplication when it's not needed (#50663)
* Correctly disable async insert when it's not used

* Better

* Add comment

* Better

* Fix tests

---------

Co-authored-by: Nikita Mikhaylov <mikhaylovnikitka@gmail.com>
2023-06-07 20:33:08 +02:00
Azat Khuzhin
6b5933a3db Add new profile events for queries with subqueries
Differs from SelectQuery/InsertQuery that it could be > 1 (in case of
MATERIALIZED VIEW attached to the table), since each local SELECT/INSERT
query is accounted, not only initial.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-05-26 18:28:40 +02:00
Alexey Milovidov
6fddb5bad3 Simplification 2023-05-07 06:31:00 +02:00
Alexey Milovidov
a695d6227d Make concurrency control controllable 2023-05-07 06:16:30 +02:00
Michael Kolupaev
0fa311ee38 Make INSERT do more things in parallel to avoid getting bottlenecked on one thread 2023-05-05 00:49:27 +00:00
pufit
d717372c6e
Revert "Make Schema inference works for CREATE AS SELECT" 2023-04-13 13:35:32 -04:00
pufit
a8c892f925
Merge pull request #48679 from ucasfl/scheme-infer
Make Schema inference works for CREATE AS SELECT
2023-04-13 13:02:04 -04:00
flynn
ea9187511e Make Schema inference works for CREATE AS SELECT 2023-04-12 04:21:03 +00:00
Azat Khuzhin
44ae8485f1 Use one ThreadGroup while pushing to materialized views
Before this patch only the case when ThreadStatus (current_thread)
already has ThreadGroup works that way, after they will be identical in
this aspect.

But this should not affect anything, but it just make sense.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-04-07 15:37:35 +02:00
Anton Popov
d5864fa88e allow to fallback from async insert in case of large amount of data 2023-02-22 21:59:24 +00:00
avogar
c3e8dd8984 Fix low cardinality case 2023-02-08 19:14:28 +00:00
avogar
738311c7fa Use default of columnt type in insert_null_as_default if column DEFAULT values is not specified 2023-02-08 15:48:18 +00:00
Alexey Milovidov
62bfdb349a Merge branch 'master' into fix-trash-query-kind 2023-02-02 04:26:49 +01:00
Alexey Milovidov
9b908a0be0 Fix trash around query kind, part 2 2023-02-01 03:31:31 +01:00
Alexey Milovidov
5c4db119fa Fix trash around query kind, part 1 2023-02-01 03:11:54 +01:00
Raúl Marín
7c31cb7adc Proper includes for ConnectionTimeoutsContext.h 2023-01-31 16:11:32 +01:00
Alexander Tokmakov
70d1adfe4b
Better formatting for exception messages (#45449)
* save format string for NetException

* format exceptions

* format exceptions 2

* format exceptions 3

* format exceptions 4

* format exceptions 5

* format exceptions 6

* fix

* format exceptions 7

* format exceptions 8

* Update MergeTreeIndexGin.cpp

* Update AggregateFunctionMap.cpp

* Update AggregateFunctionMap.cpp

* fix
2023-01-24 00:13:58 +03:00
Maksim Kita
5146087db1 Fix style 2023-01-21 12:19:09 +01:00
Maksim Kita
e067a55b78 Fixed tests 2023-01-20 12:19:16 +01:00
Maksim Kita
21b288c620 Fixed build 2023-01-18 10:44:40 +01:00
Maksim Kita
0c7e0be0b6 Analyzer support INSERT SELECT 2023-01-16 12:17:14 +01:00
Nikita Mikhaylov
857799fbca
Parallel distributed insert select with s3Cluster [3] (#44955)
* Revert "Revert "Resurrect parallel distributed insert select with s3Cluster (#41535)""

This reverts commit b8d9066004.

* Fix build

* Better

* Fix test

* Automatic style fix

Co-authored-by: robot-clickhouse <robot-clickhouse@users.noreply.github.com>
2023-01-09 13:30:32 +01:00
SmitaRKulkarni
16fca48f5b
Disable functions in readonly for inserts (#44290)
Implementation:
* Added a new parameter to ITableFunction::execute to check if query is insert query.
* Used the new parameter in InterpreterInsertQuery::getTable function when calling ITableFunction::execute .
Testing:
* Added test case for INSERT in 02473_functions_in_readonly_mode.sh.

Co-authored-by: Nikita Mikhaylov <mikhaylovnikitka@gmail.com>
2023-01-03 15:25:04 +01:00
Han Fei
11f93dec3f tmp commit 2022-11-14 15:53:45 +01:00
Azat Khuzhin
4e76629aaf Fixes for -Wshorten-64-to-32
- lots of static_cast
- add safe_cast
- types adjustments
  - config
  - IStorage::read/watch
  - ...
- some TODO's (to convert types in future)

P.S. That was quite a journey...

v2: fixes after rebase
v3: fix conflicts after #42308 merged
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-10-21 13:25:19 +02:00
Azat Khuzhin
5166a1d58e Fix written_bytes/written_rows in INSERT SELECT intermediate progress packets
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-10-05 19:12:18 +02:00
Nikolai Kochetov
00965ce17a Fixing race. 2022-09-22 17:57:04 +00:00
Nikolai Kochetov
446453bdf5 Fix a destruction order for views ThreadStatus 2022-09-21 18:37:40 +00:00
Duc Canh Le
8590cc46c4 implement intersect + except distinct 2022-08-30 18:09:01 +08:00
Alexander Tokmakov
f9f85a0e8b Revert "Parallel distributed insert select from *Cluster table functions (#39107)"
This reverts commit d3cc234986.
2022-08-24 15:17:15 +03:00
Nikita Mikhaylov
d3cc234986
Parallel distributed insert select from *Cluster table functions (#39107) 2022-08-15 12:41:17 +02:00
Azat Khuzhin
dd3515da98 Fix parallel_view_processing with optimize_trivial_insert_select=1 2022-07-03 15:14:05 +03:00
Azat Khuzhin
4ae7db8369 Fix max_insert_threads while pushing to views
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-07-03 15:14:05 +03:00
Dmitry Novik
376412e417 Small refactoring 2022-06-16 15:41:04 +00:00
Nikolai Kochetov
85f4295370 Cleanup 2022-06-02 10:34:40 +00:00
Nikolai Kochetov
147a819221 Refactor a little bit more. 2022-05-31 14:43:38 +00:00
Nikolai Kochetov
b80b1940ce Fix some tests. 2022-05-27 20:47:35 +00:00
Nikolai Kochetov
1b85f2c1d6 Merge branch 'master' into refactor-read-metrics-and-callbacks 2022-05-25 16:27:40 +02:00
Nikolai Kochetov
3d84aae0ab Better. 2022-05-24 20:06:08 +00:00
Nikolai Kochetov
9756b759c6 Move some resources 2022-05-23 13:46:57 +00:00
Nikolai Kochetov
56feef01e7 Move some resources 2022-05-20 19:49:31 +00:00
Robert Schulze
b475fbc9a7
Merge pull request #37300 from ClickHouse/cmake-cleanup-pt3
Various cmake cleanups
2022-05-20 10:02:36 +02:00
Robert Schulze
43945cea1b
Fixing some warnings 2022-05-16 20:59:27 +02:00
Vxider
f2ef27e5d6 rename windowview header 2022-05-14 09:21:54 +00:00
mergify[bot]
7ffad69019
Merge branch 'master' into wv-populate 2022-05-12 11:52:04 +00:00