thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-10 08:22:00 +00:00
Code Issues Packages Projects Releases Wiki Activity
114,497 Commits 9,221 Branches 1,782 Tags 2 GiB
0c0ea7bfc0
Commit Graph

3363 Commits

Author SHA1 Message Date
Alexander Gololobov
ba5ca15c40
Merge pull request #46835 from ClickHouse/reduce_mem_in_mutation_with_subquery 
Reduce memory consumption by mutations with big subqueries used with IN
 2023-04-17 23:55:08 +02:00
Alexander Gololobov
fc4fd3e300 Clarify setting description 
Co-authored-by: Vladimir C <vdimir@clickhouse.com>
 2023-04-17 18:23:52 +02:00
kssenii
86ca6c4f44 Commit forgotten file 2023-04-17 16:44:49 +02:00
凌涛
7ed4c413e3 add settings allow_suspicious_indices 2023-04-17 20:23:19 +08:00
Michael Kolupaev
87be78e6de Better 2023-04-17 04:58:32 +00:00
Michael Kolupaev
dc6e34075e Read less unnecessary data from Parquet files 2023-04-17 04:58:32 +00:00
Alexander Gololobov
7a381e9232 Added enable_sharing_sets_for_mutations setting 2023-04-14 16:12:18 +02:00
Alexey Milovidov
84cb2ff879 Added setting "use_index_for_in_with_subqueries_max_values" #2571 2023-04-14 16:12:18 +02:00
Dmitry Novik
352a240a20
Merge pull request #47984 from ClickHouse/optimize-compilation 
Optimize compilation
 2023-04-14 15:34:46 +02:00
Kseniia Sumarokova
54518bf928
Merge branch 'master' into better-tests-for-data-lakes 2023-04-13 23:45:12 +02:00
Alexander Tokmakov
af1bf08663
Revert "Add MemoryTracker for the background tasks" 2023-04-13 21:05:02 +03:00
Dmitry Novik
5cc9b46f78 Merge remote-tracking branch 'origin/master' into optimize-compilation 2023-04-13 16:04:09 +02:00
kssenii
6f53784f22 Merge remote-tracking branch 'upstream/master' into better-tests-for-data-lakes 2023-04-13 15:56:40 +02:00
kssenii
3ac7f999bc Review fixes 2023-04-13 13:38:16 +02:00
Dmitry Novik
467ecf45e1
Merge pull request #46089 from ClickHouse/background-memory-tracker 
Add MemoryTracker for the background tasks
 2023-04-13 13:29:27 +02:00
Robert Schulze
f41354ccd6
Merge pull request #48671 from ClickHouse/rs/gcc-removal 
Remove GCC remainders
 2023-04-13 10:15:35 +02:00
Sergei Trifonov
86d37916a0
Merge pull request #48146 from azat/threadpool-pipeline-executor 
Use ThreadPool in PipelineExecutor
 2023-04-13 09:45:15 +02:00
Alexander Tokmakov
75f18b1198
Revert "Check simple dictionary key is native unsigned integer" 2023-04-13 01:32:19 +03:00
Robert Schulze
3f7ce60e03
Merge branch 'master' into rs/gcc-removal 2023-04-12 22:17:04 +02:00
Anton Popov
1520f3e924
Merge pull request #48335 from lzydmxy/check_sample_dict_key_is_correct 
Check simple dictionary key is native unsigned integer
 2023-04-12 14:27:39 +02:00
Azat Khuzhin
1d9a785b9b Add missing includes (due to removing ThreadPool.h from PipelineExecutor.h) 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-04-11 20:45:46 +02:00
Robert Schulze
05606a8835
Clean up GCC warning pragmas 2023-04-11 18:21:08 +00:00
Sergei Trifonov
7c309a0134
Merge branch 'master' into mburdukov_multi_settings 2023-04-11 17:53:43 +02:00
Dmitry Novik
06e6794fc0 Merge remote-tracking branch 'origin/master' into background-memory-tracker 2023-04-11 15:29:35 +00:00
Robert Schulze
b7d5df7aa1
Merge branch 'master' into rs/better-f-in-formatdatetime 2023-04-11 08:11:04 +02:00
Azat Khuzhin
79b83c4fd2 Remove superfluous includes of logger_userful.h from headers 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-04-10 17:59:30 +02:00
MikhailBurdukov
54d7a412ec Fixed unit tests 2023-04-07 11:04:00 +00:00
Azat Khuzhin
011480924a Use forward declaration of ThreadPool 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-04-07 11:25:35 +02:00
MikhailBurdukov
5282a20aba Test and fix 2023-04-06 15:50:28 +00:00
Alexey Milovidov
3066d5df52
Merge pull request #48195 from ClickHouse/disable-compile-aggregations 
Remove a feature
 2023-04-06 18:09:48 +03:00
kssenii
0f02349473 Fix 2023-04-06 14:14:22 +02:00
Robert Schulze
5eb31bba87
MySQL compat: Align behavior of %f in formatDateTime() with parseDateTime() 2023-04-06 08:13:04 +00:00
Robert Schulze
ce2692ba79
Merge pull request #47246 from ClickHouse/rs/m 
Change the behavior of formatter %M in function formatDateTime() from minutes to month name
 2023-04-06 09:19:50 +02:00
Robert Schulze
809a7fbb7a
Incorporate more review feedback 2023-04-05 19:43:30 +00:00
robot-clickhouse
3d2c279859
Merge pull request #48242 from azat/better-throttling 
Per-query/per-server throttling for remote IO/local IO/BACKUPs
 2023-04-05 19:32:37 +02:00
Alexey Milovidov
5477d67acd
Merge branch 'master' into disable-compile-aggregations 2023-04-05 18:17:17 +03:00
Anton Popov
027f8a02c8
Merge branch 'master' into check_sample_dict_key_is_correct 2023-04-05 16:31:57 +02:00
Nikita Mikhaylov
7b526d3df9
Always communicate with local replica through loopback interface for parallel replicas (#48246) 2023-04-05 13:32:46 +02:00
Azat Khuzhin
4008155a6e Add max_ prefix for backup_bandwidth settings 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-04-05 10:54:25 +02:00
Azat Khuzhin
61405b827d Use MAKE_DEPRECATED_BY_SERVER_CONFIG() for deprecated throttle settings 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-04-05 09:40:18 +02:00
Azat Khuzhin
c332d290d8 Keep only one throttler for BACKUPs IO (instead of separate read/write) 
There is no need in separate read/write throttling, because you cannot
write faster then read anyway, and plus this makes the code less cleaner

(and also it will allow avoid implementing throttling backups to S3,
since it does not use common S3 writer).

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-04-05 09:40:17 +02:00
Azat Khuzhin
218b1f9c29 Add ability to throttle BACKUPs on per-server/backup basis 
Server settings:
- backup_read_bandwidth_for_server
- backup_write_bandwidth_for_server

Query settings:
- backup_read_bandwidth
- backup_write_bandwidth

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-04-05 09:39:48 +02:00
Azat Khuzhin
a25dd1d348 Add ability to throttle local IO on per-query/server basis 
Server settings:
- max_local_read_bandwidth_for_server
- max_local_write_bandwidth_for_server

Query settings:
- max_local_read_bandwidth
- max_local_write_bandwidth

This is the preparation for adding ability to throttle BACKUPs

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-04-05 09:39:15 +02:00
Azat Khuzhin
b3406beeb7 Add per-query network throttling 
Controlled with:
- max_remote_read_network_bandwidth
- max_remote_write_network_bandwidth

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-04-05 09:39:15 +02:00
Azat Khuzhin
40cb87c2ac Make remote bandwidth settings server settings 
Right now those two settings are applied only at start:
- max_remote_read_network_bandwidth_for_server
- max_remote_write_network_bandwidth_for_server

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-04-05 09:39:15 +02:00
Nikita Mikhaylov
fa5b2bd4a5
Added Keeper retries for backups operations (#47224) 2023-04-04 18:12:08 +02:00
lzydmxy
529e1466df use check_dictionary_primary_key instead of check_sample_dict_key_is_correct 2023-04-04 12:04:17 +08:00
Antonio Andelic
950a7d9948 Merge branch 'master' into keeper-map-strict-mode 2023-04-03 14:26:14 +00:00
Kruglov Pavel
9e595c82fa
Merge pull request #47494 from KevinyhZou/enable_return_null_element_not_exist 
Enable return null and complext type for JSON_VALUE function
 2023-04-03 14:05:18 +02:00
Robert Schulze
3e124f09d0
Merge remote-tracking branch 'origin/master' into qc-compression 2023-04-03 11:28:17 +00:00
lzydmxy
368c120f42 check sample dictionary key is native unsigned integer 2023-04-03 15:48:40 +08:00
KevinyhZou
c2687c0544 review fix 2023-04-02 14:14:35 +08:00
KevinyhZou
ce5f4cf4ab code review fix 2023-04-02 12:37:18 +08:00
KevinyhZou
e0a10dc62f enable nullable return type 2023-04-02 12:30:59 +08:00
Antonio Andelic
1e3abc9e84 Add strict mode for KeeperMap 2023-03-31 14:24:32 +00:00
Nikolai Kochetov
f06bdc85eb
Merge pull request #48226 from ClickHouse/rename-setting-stop_reading_on_first_cancel 
Rename setting stop_reading_on_first_cancel
 2023-03-30 18:03:23 +02:00
filimonov
fbb22348ea
Refactor reading the pool setting & from server config. (#48055) 
After #36425 there was a lot of confusions/problems with configuring pools - when the message was confusing, and settings need to be ajusted in several places.
See some examples in #44251, #43351, #47900, #46515.

The commit includes the following changes:
1) Introduced a unified mechanism for reading pool sizes from the configuration file(s). Previously, pool sizes were read from the Context.cpp with fallbacks to profiles, whereas main_config_reloader in Server.cpp read them directly without fallbacks.
2) Corrected the data type for background_merges_mutations_concurrency_ratio. It should be float instead of int.
3) Refactored the default values for settings. Previously, they were defined in multiple places throughout the codebase, but they are now defined in one place (or two, to be exact: Settings.h and ServerSettings.h).
4) Improved documentation, including the correct message in system.settings.

Additionally make the code more conform with #46550.
 2023-03-30 16:44:11 +02:00
Nikolai Kochetov
a8a28bdf3d Merge branch 'master' into rename-setting-stop_reading_on_first_cancel 2023-03-30 16:07:14 +02:00
Nikolai Kochetov
f870e6f7a9 Rename setting stop_reading_on_first_cancel to partial_result_on_first_cancel. 2023-03-30 14:00:44 +00:00
Alexey Milovidov
103972497b Remove unused setting 2023-03-30 12:07:56 +02:00
Robert Schulze
b0473098c6
Merge branch 'master' into qc-compression 2023-03-30 10:55:42 +02:00
Alexey Milovidov
50bdea3c76 Fix bug in aggregate functions JIT 2023-03-30 09:57:17 +02:00
Alexey Milovidov
cc345853e7
Merge pull request #47594 from ClickHouse/decimal256-various 
Add specializations for Decimal256 in various places
 2023-03-29 17:33:20 +03:00
Alexander Tokmakov
c5b6ed410d
Merge pull request #48130 from ClickHouse/support_undrop_table 
Merging "Support undrop table"
 2023-03-29 16:49:58 +03:00
Vitaly Baranov
481a7a76ac
Simplify backup coordination for file infos (#48095) 
* Remove obsolete code for archive suffixes.

* Simplify backup coordination, stop using it for restoring.

* Build all file infos before writing to backup. Decrease number of znodes.

* Split long values before writing to ZooKeeper.

* Use separate mutexes for unrelated activities.

* Make test test_disallow_concurrency less flaky.

* Add comments and test for backup_keeper_value_max_size.
 2023-03-29 15:19:40 +02:00
Robert Schulze
b324d0ed5e
Merge branch 'master' into qc-compression 2023-03-29 15:09:22 +02:00
Alexander Tokmakov
a1076005c7 Merge branch 'master' into support_undrop_table 2023-03-28 22:03:31 +02:00
Igor Nikonov
259df04415 Merge remote-tracking branch 'origin/master' into decimal256-various 2023-03-28 13:36:53 +00:00
Vladimir C
d32c285d17
Merge branch 'master' into vdimir/direct-dict-async-read 2023-03-28 12:41:20 +02:00
Vladimir C
5fa519d043
Merge pull request #48047 from ClickHouse/vdimir/max_number_of_mutatuins_for_replica 2023-03-28 10:50:35 +02:00
xiedeyantu
5a18e0d5a5 support undrop table 2023-03-27 22:16:36 +08:00
Han Fei
e3afa5090f
Merge pull request #47218 from hanfei1991/hanfei/optimize-regexp-tree-1 
Refine OptimizeRegularExpression Function and RegexpTreeDict
 2023-03-27 15:23:01 +02:00
vdimir
08fa1846b1
Add merge tree setting max_number_of_mutations_for_replica 2023-03-27 11:56:48 +00:00
Kruglov Pavel
bc92e3184e
Merge branch 'master' into non-blocking-connect 2023-03-27 13:50:50 +02:00
vdimir
f6de216041
PullingAsyncPipelineExecutor for Direct dictionary with ClickHouse source 2023-03-27 09:52:26 +00:00
Nikolai Kochetov
73e98de46d Merge branch 'master' into aggregate-projections-analysis-query-plan 2023-03-23 21:28:36 +01:00
Kruglov Pavel
3ee12e21fb
Merge branch 'master' into non-blocking-connect 2023-03-23 20:53:44 +01:00
Nikolai Kochetov
0f01725d8b
Merge pull request #45689 from alexX512/master 
Add an option to show the partial result on query cancellation
 2023-03-23 17:34:15 +01:00
Azat Khuzhin
2b439f079e
Make backup_threads/restore_threads server settings (#47881) 
Those settings can be applied only at start, so it is better to have
them in configuration file instead of user profile settings.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-03-23 15:44:15 +01:00
Nikita Taranov
4b635afe49
Set max_final_threads to number of cores by default (#47915) 2023-03-23 15:18:37 +01:00
Nikita Mikhaylov
301e26b23d
Parallel replicas. Part 2.5 (#47858) 2023-03-22 13:05:38 +01:00
Dmitry Novik
8c3f3f437f Atempt to avoid including Decimal.h all over the code 2023-03-22 02:33:10 +01:00
avogar
38e44861ae Fix possible race conditions 2023-03-21 16:01:54 +00:00
Robert Schulze
d8493780b2
Query cache: Enable compression and squashing of result blocks 
ClickHouse reads table data in blocks of 'max_block_size' rows. Due to
filtering, aggregation, etc., result blocks are typically much smaller
than 'max_block_size' but there are also cases where they are much
bigger. Setting 'query_cache_squash_partial_results' (enabled by
default) now controls if result blocks are squashed (if they are tiny)
or split (if they are large) into blocks of 'max_block_size' size before
insertion into the query result cache. This reduces performance of
writes into the query cache but improves compressability of cache
entries and provides more natural block granularity when query results
are later served from the query cache.

Entries in the query cache are now also compressed by default. This
reduces the overall memory consumption at the cost of slower writes into
/ reads from the query cache. To disable compression, use setting
'query_cache_compress_entries'.
 2023-03-20 20:11:41 +00:00
Sergei Trifonov
273770519b
Merge branch 'master' into master 2023-03-20 18:28:10 +01:00
Alexey Milovidov
4a44dafdee Merge branch 'master' into decimal256-various 2023-03-20 03:30:14 +01:00
Alexey Milovidov
ddf29b69f9
Merge branch 'master' into fuzzer-of-data-formats 2023-03-20 03:26:51 +03:00
Robert Schulze
0e01e912fe
Preparation for libcxx(abi), llvm, clang-tidy 16 2023-03-19 17:25:41 +00:00
Alexey Milovidov
26d17fe560 Merge branch 'master' of github.com:ClickHouse/ClickHouse into decimal256-various 2023-03-17 22:15:16 +01:00
Alexey Milovidov
c11915d35b Merge branch 'fix-bson-2' of github.com:Avogar/ClickHouse into fuzzer-of-data-formats 2023-03-17 22:05:46 +01:00
alexX512
023c0ba46e Fix comments 2023-03-17 20:05:10 +00:00
Alexey Perevyshin
2c6534f5c8
Merge branch 'master' into master 2023-03-16 23:57:36 +04:00
Alexey Milovidov
704cb54afe Merge branch 'master' of github.com:ClickHouse/ClickHouse into fuzzer-of-data-formats 2023-03-16 19:50:33 +01:00
serxa
3ad7194e31 add pool size metrics 2023-03-16 17:31:41 +00:00
robot-clickhouse-ci-1
1223b8959b
Merge pull request #47626 from ClickHouse/rs/follow-up-to-47614 
Follow-up to #47614
 2023-03-16 00:31:51 +01:00
Alexey Milovidov
92a5b81222 Merge branch 'master' of github.com:ClickHouse/ClickHouse into fuzzer-of-data-formats 2023-03-15 23:59:16 +01:00
pufit
f4d9b4a888
Merge pull request #47086 from pkit/pkit/insert_http_body 
use http request body in predefined http handlers
 2023-03-15 17:41:05 -04:00
Alexey Perevyshin
1e9ef12f51
Merge branch 'master' into master 2023-03-15 22:21:02 +04:00
Robert Schulze
785ade7c39
Follow-up to #47614 2023-03-15 18:05:53 +00:00
Han Fei
a532503466
Merge branch 'master' into hanfei/optimize-regexp-tree-1 2023-03-15 17:56:01 +01:00