thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-15 19:02:04 +00:00
Code Issues Packages Projects Releases Wiki Activity
101,475 Commits 9,326 Branches 1,783 Tags 2.1 GiB
252575ee23
Commit Graph

7895 Commits

Author SHA1 Message Date
Duc Canh Le
2b82205590
Merge branch 'master' into ch_canh_fix_aggregator_lc 2022-10-16 16:59:12 +08:00
Duc Canh Le
9e9e967f1f choose correct aggregation method for lc128 and lc512 2022-10-16 16:57:15 +08:00
Vitaly Baranov
91c438eebe
Merge pull request #42263 from vitlibar/refactor-udf 
Refactor the implementation of user-defined functions
 2022-10-15 12:33:10 +02:00
avogar
52427e6028 Remove code duplication 2022-10-14 18:07:02 +00:00
Kruglov Pavel
b4e327d9e0
Merge pull request #42313 from ClickHouse/Avogar-patch-2 
Remove forgotten debug logging
 2022-10-14 19:23:06 +02:00
Kruglov Pavel
11a9b5977a
Fix typos 2022-10-14 19:22:05 +02:00
nvartolomei
7d9f097d3b
Fix bad copy & paste when loading background_merges_mutations_concurrency_ratio setting (#42315) 2022-10-14 19:19:14 +02:00
Nikolai Kochetov
b98b1759c8 Fix additional_table_filters for Distributed. 2022-10-14 17:04:42 +00:00
Kruglov Pavel
ff11904850
Merge branch 'master' into improve-combinators 2022-10-14 17:19:31 +02:00
Kruglov Pavel
f1dd247181
Merge branch 'master' into improve-schema-inference-insert-select 2022-10-14 17:16:58 +02:00
avogar
10335aec13 Improve using structure from insertion table in table functions 2022-10-14 15:09:35 +00:00
Kruglov Pavel
d43b5bbcd0
Remove forgotten debug logging 2022-10-14 15:47:46 +02:00
Alexander Tokmakov
9b7edd4f13
Merge pull request #42106 from ClickHouse/fix_loading_dependencies 
Fixes for loading dependencies
 2022-10-14 14:31:31 +03:00
Vladimir C
cf78c70bdd
Merge pull request #41544 from lhuang09287750/master 2022-10-13 17:47:41 +02:00
Vitaly Baranov
2e59d671fc Split UserDefinedSQLObjectsLoader to interface and implementation. 2022-10-13 10:32:32 +02:00
Nikolay Degterinsky
87b77094f0
Merge pull request #42187 from FArthur-cmd/add_default_support_for_settings 
Add support for keyword DEFAULT in the set query
 2022-10-12 15:30:30 +02:00
lhuang09287750
215acf5e5b check the result of conversion for decimal IN float 2022-10-12 15:28:27 +02:00
lhuang09287750
4b0ac268e8 delete an unusable include 2022-10-12 15:28:27 +02:00
lhuang09287750
cab0291873 for checkstyle:use a internal function to get field string ,instead of std::stringstream 2022-10-12 15:28:27 +02:00
lhuang09287750
338f41913c add a function to process floating point literal comparing with decimal type in IN opertor 2022-10-12 15:28:27 +02:00
vdimir
0178307c27 Followup for TemporaryDataOnDisk 2022-10-12 15:25:23 +02:00
Nikolay Degterinsky
896e3c47f7
Merge pull request #42135 from ClibMouse/add-null-pointer-checks 
Add null pointer checks
 2022-10-12 01:22:55 +02:00
FArthur-cmd
b6304c46d4 move default settings to different part of ast 2022-10-11 18:25:28 +00:00
ltrk2
15aa8a88dc Throw an exception in case of inconsistent state 2022-10-11 05:04:47 -07:00
Kseniia Sumarokova
3819a21033
Merge pull request #42078 from azat/insert-progress 
Fix progress for INSERT SELECT
 2022-10-11 10:56:20 +02:00
Alexander Tokmakov
4b371bd20c fix 2022-10-10 16:38:35 +02:00
Alexander Tokmakov
fa1134f299 Merge branch 'master' into fix_loading_dependencies 2022-10-10 16:30:52 +02:00
Yakov Olkhovskiy
b5808a3f00
Merge branch 'master' into composable-protocol 2022-10-10 08:07:48 -04:00
Maksim Kita
2f5b2d782c
Merge pull request #42063 from kitaisreal/function-name-normalization-header-functions 
Function name normalization fix functions header
 2022-10-09 14:19:33 +03:00
Yakov Olkhovskiy
6c8e52eb67
Merge branch 'master' into composable-protocol 2022-10-08 20:33:32 -04:00
Alexey Milovidov
9ee6896916
Merge pull request #42002 from ClickHouse/relax-too-many-parts-threshold 
Relax "too many parts" threshold
 2022-10-08 20:17:01 +03:00
Alexey Milovidov
a9cbdeec24
Merge pull request #42180 from ClickHouse/unique-thread-ids 
Make thread ids in the process list and query_log unique to avoid waste
 2022-10-08 20:06:02 +03:00
Alexander Tokmakov
bd10a9d2d4
Merge pull request #42168 from ClickHouse/unreachable_macro 
Abort instead of `__builtin_unreachable` in debug builds
 2022-10-08 19:05:57 +03:00
Alexey Milovidov
9bf5fe728b
Merge pull request #42173 from ClickHouse/remove-temporary-live-view 
Remove TEMPORARY LIVE VIEW
 2022-10-08 19:05:33 +03:00
Alexey Milovidov
2f83d87905 Make thread_ids unique 2022-10-08 02:42:41 +02:00
Alexey Milovidov
ec5a32f534 Make thread_ids unique 2022-10-08 02:05:36 +02:00
Alexey Milovidov
169c505603 Relax "too many parts" thresholds 2022-10-08 00:24:51 +02:00
Alexey Milovidov
b9159db82f Remove temporary live views 2022-10-08 00:16:36 +02:00
Alexander Tokmakov
4175f8cde6 abort instead of __builtin_unreachable in debug builds 2022-10-07 21:49:08 +02:00
Alexander Tokmakov
a1ff835894 fix 2022-10-07 19:52:34 +02:00
Alexander Tokmakov
014784a9ca Merge branch 'master' into fix_loading_dependencies 2022-10-07 18:58:11 +02:00
Maksim Kita
930bda9c72
Merge branch 'master' into jit-compilation-migration-to-llvm-15 2022-10-07 18:45:53 +03:00
Alexander Tokmakov
ed76584876 fix 2022-10-07 16:52:04 +02:00
Alexander Tokmakov
b8d9066004 Revert "Resurrect parallel distributed insert select with s3Cluster (#41535)" 
This reverts commit 860e34e760.
 2022-10-07 15:53:30 +02:00
Maksim Kita
76d6204f99 JIT compilation migration to LLVM 15 2022-10-07 10:53:38 +02:00
Yakov Olkhovskiy
2229087896
Merge branch 'master' into composable-protocol 2022-10-07 02:23:26 -04:00
ltrk2
696a294ef3 Add null pointer checks 2022-10-06 05:48:04 -07:00
Robert Schulze
cc24fcd6d5
Merge pull request #41873 from ClickHouse/generated-file-cleanup 
Consolidate CMake-generated config headers
 2022-10-06 14:31:03 +02:00
Raúl Marín
adbaaca2f5
QOL log improvements (#41947) 
* Uniformize disk reservation logs

* Remove log about destroying stuff that appears all the time

* More tweaks on disk reservation logs

* Reorder logs in hash join

* Remove log that provides little information

* Collapse part removal logs

Co-authored-by: Sergei Trifonov <sergei@clickhouse.com>
 2022-10-06 14:22:44 +02:00
Robert Schulze
78be400ac0
Merge remote-tracking branch 'origin/master' into generated-file-cleanup 2022-10-06 12:22:43 +00:00
Nikita Mikhaylov
860e34e760
Resurrect parallel distributed insert select with s3Cluster (#41535) 2022-10-06 13:47:32 +02:00
Maksim Kita
6ad435d034
Merge branch 'master' into function-name-normalization-header-functions 2022-10-06 13:29:21 +03:00
Robert Schulze
da5a2e2db0
Merge remote-tracking branch 'origin/master' into generated-file-cleanup 
Physical merge conflicts:
- src/Common/ZooKeeper/ZooKeeperImpl.cpp
- src/Core/config_core.h.in
- src/Functions/FunctionsAES.h
- src/Functions/config_functions.h.in
- src/configure_config.cmake

Logical merge conflicts:
- Functions/tryDecrypt.cpp
 2022-10-06 08:43:25 +00:00
Anton Popov
190e1f6830
Merge branch 'master' into async-inserts-log 2022-10-06 01:59:27 +02:00
Alexander Tokmakov
d682c2ccdb fix 2022-10-05 22:13:16 +02:00
Alexander Tokmakov
690ec74bf2 better handling for expressions in dictGet 2022-10-05 20:58:27 +02:00
Alexander Tokmakov
25df3efca5 check dependencies before table shutdown 2022-10-05 20:57:55 +02:00
Azat Khuzhin
5166a1d58e Fix written_bytes/written_rows in INSERT SELECT intermediate progress packets 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-10-05 19:12:18 +02:00
Anton Popov
f61e98f0e8
Merge pull request #40668 from ClickHouse/async-insert-max-data-size-fix 
Use settings from query context for async INSERTs
 2022-10-05 15:15:19 +02:00
Anton Popov
e9928fd703 fix clang-tidy 2022-10-04 23:50:14 +00:00
Maksim Kita
f4101861e7 Function name normalization fix functions header 2022-10-04 17:29:33 +02:00
Anton Popov
121ef74b17 slightly better 2022-10-04 14:19:05 +00:00
Anton Popov
13b4d5f7dc fix build 2022-10-04 12:54:27 +00:00
Anton Popov
148995894f add system table asynchronous_inserts_log 2022-10-03 21:35:55 +00:00
Anton Popov
6e61cf92f5 Merge remote-tracking branch 'upstream/master' into HEAD 2022-10-03 13:16:57 +00:00
Alexander Tokmakov
00914a174d
Merge pull request #41410 from ClickHouse/keeper-multiread 
Add MultiRead support in Keeper and internal ZK client
 2022-10-03 14:17:57 +03:00
Robert Schulze
aa9373c9c2
Merge pull request #41951 from ClickHouse/llvm-14 
Bump LLVM from 13 to 14
 2022-10-03 12:34:31 +02:00
Alexander Tokmakov
7b5b7e4963
Merge pull request #41945 from ClickHouse/enable_more_s3_tests 
Enable some disabled S3 tests
 2022-10-03 13:25:50 +03:00
Robert Schulze
db5ef7b3cb
Merge branch 'master' into generated-file-cleanup 2022-10-02 23:13:18 +02:00
Robert Schulze
5b1a7755d8
Fix typos 2022-10-02 21:38:46 +02:00
Vitaly Baranov
baf48ec400
Merge pull request #41562 from vitlibar/refactor-wiping-info-from-logs 
Refactor wiping sensitive information from logs.
 2022-10-02 12:58:58 +02:00
Alexey Milovidov
daab7f7f61
Merge pull request #41987 from ClickHouse/musl-4 
Preparation for Musl build, part 4
 2022-10-02 02:35:01 +03:00
Alexander Tokmakov
e40917a9c5 fix 2022-10-02 00:44:46 +02:00
Alexey Milovidov
bd277348ca
Merge pull request #41984 from ClickHouse/vdimir/followup-40893 
Rename max_temp_data_on_disk -> max_temporary_data_on_disk
 2022-10-01 23:45:49 +03:00
Alexey Milovidov
0e3b9bf3ff Add more checkStackSize calls 2022-10-01 17:48:56 +02:00
Vitaly Baranov
a5b2a49ab1 Refactor wiping sensitive information from logs. 2022-10-01 17:17:45 +02:00
Alexey Milovidov
25b75a6982
Merge pull request #41983 from amosbird/issue_9338 
Support countState(*)
 2022-10-01 17:34:42 +03:00
vdimir
498ba33160
Rename max_temp_data_on_disk -> max_temporary_data_on_disk 2022-10-01 08:51:33 +00:00
Amos Bird
98fc22b08e
Support countState(*) 2022-10-01 14:59:26 +08:00
Vladimir C
895afdec45
Merge pull request #40893 from ClickHouse/vdimir/track-tmp-disk 2022-09-30 11:27:24 +02:00
Robert Schulze
4ac308a7a6
Update submodule + fix compile error 2022-09-30 09:23:41 +00:00
Robert Schulze
cc92a2d174
Merge branch 'master' into generated-file-cleanup 2022-09-30 09:56:31 +02:00
Robert Schulze
f38d71aac9
Fix includes 2022-09-29 20:34:21 +00:00
Robert Schulze
5dcdf60f68
Merge pull request #41762 from ClickHouse/bump-llvm-to-13 
Bump LLVM from 12 to 13
 2022-09-29 19:37:46 +02:00
Nikolai Kochetov
8e23bee03c
Merge pull request #41650 from ClickHouse/keep-destruction-order-of-view-thread-statuses 
Fix a destruction order for views ThreadStatus
 2022-09-29 14:28:14 +02:00
vdimir
f495361e28
fixes for TemporaryDataOnDisk 2022-09-29 10:09:29 +00:00
vdimir
14cd2b632c
fix Context::shutdown 2022-09-29 09:51:49 +00:00
vdimir
1234b70118
Handle decorator checking tmp disk type 2022-09-29 09:51:49 +00:00
vdimir
0605f6ed7f
fix after rebase 2022-09-29 09:51:48 +00:00
vdimir
cb4397d1ed
Create reader only after the TemporaryDataOnDisk::read 2022-09-29 09:51:48 +00:00
vdimir
6f8e8b979d
Revert "wip" 
This reverts commit 46e4f0236df9a6f7b03d40278e583bc93b96559a.
 2022-09-29 09:51:47 +00:00
vdimir
74d45325b3
wip 2022-09-29 09:51:46 +00:00
vdimir
0f1a7c252d
better TemporaryDataOnDisk 2022-09-29 09:51:46 +00:00
Vladimir C
dd20fa37f5
Fix typos 2022-09-29 09:51:45 +00:00
vdimir
9f3f34548c
Allow to create temporaty streams on leaf TemporaryDataOnDisk 2022-09-29 09:51:45 +00:00
vdimir
858769584f
Fix MergeSortingTransform 2022-09-29 09:51:43 +00:00
vdimir
0093752271
Upd log message 2022-09-29 09:51:43 +00:00
vdimir
a56a10f089
Do not require tmp_data in Aggregator 2022-09-29 09:51:42 +00:00
vdimir
15c7a3be34
Temp data on disk: build 2022-09-29 09:51:41 +00:00
vdimir
c0898ce289
Use abstraction for temporary data on disk in Sort and Aggregation 2022-09-29 09:51:41 +00:00
vdimir
ac39bbb3f1
[wip] Common interface for temporary data on disk 2022-09-29 09:51:40 +00:00
vdimir
367c77f0fb
Check if tmp volume contains only local disks 2022-09-29 09:51:39 +00:00
Antonio Andelic
fcc54104a4 Define methods for multi read requests 2022-09-29 09:19:47 +00:00
Antonio Andelic
3109ce51c6 Merge branch 'master' into keeper-multiread 2022-09-29 06:50:49 +00:00
Robert Schulze
fd86829824
Consolidate config_core.h into config.h 
Less duplication, less confusion ...
 2022-09-28 13:31:57 +00:00
Arthur Passos
da9393ac6a
Merge branch 'master' into add_select_final_mt_setting 2022-09-28 09:56:29 -03:00
Kruglov Pavel
6340369c2a
Merge branch 'master' into improve-combinators 2022-09-28 14:55:30 +02:00
Robert Schulze
6d70b4a1f6
Generate config_version.h into ${CONFIG_INCLUDE_PATH} 
This makes the target location consistent with other auto-generated
files like config_formats.h, config_core.h, and config_functions.h and
simplifies the build of clickhouse_common.
 2022-09-28 12:48:26 +00:00
Nikolai Kochetov
da375f27c5
Merge pull request #41740 from ClickHouse/fix-41269 
Fix possible crash for SELECT from Merge table
 2022-09-28 14:45:02 +02:00
Nikolai Kochetov
9564b8b3fa Add comment. 2022-09-28 11:30:09 +00:00
Yakov Olkhovskiy
7c94f98f6a
Merge pull request #37961 from ClibMouse/Kusto-phase1 
Kusto phase1
 2022-09-27 12:42:15 -04:00
Antonio Andelic
265c8b3d26 Merge branch 'master' into keeper-multiread 2022-09-27 12:37:22 +00:00
Antonio Andelic
7a9afc4bdd Fix zookeeper_log 2022-09-27 12:36:59 +00:00
Alexander Tokmakov
3d4a5a493b
Merge pull request #41484 from FrankChen021/on_cluster_dll 
Add OpenTelemetry support to ON CLUSTER DDL
 2022-09-27 15:15:52 +03:00
Alexander Gololobov
1719e000ea
Merge pull request #41763 from zhangjmruc/fix/disable_mergetree_table_with_lwd_virtual_column 
Disable merge tree table with lightweight delete virtual column name _row_exists
 2022-09-27 12:20:19 +02:00
Kseniia Sumarokova
caf6a99f41
Merge pull request #41011 from kssenii/fix-thread-status 
Fix incorrect attach query in threadpool readers, get rid of static threadpools for reads/writes, make threadpool size for reads/writes configurable
 2022-09-27 11:35:07 +02:00
Alexander Gololobov
f21f933da7
Merge branch 'master' into fix/disable_mergetree_table_with_lwd_virtual_column 2022-09-26 23:44:45 +02:00
Robert Schulze
3efd42ae2e
Fix tests (hopefully) 
- LLVM migrates to opaque pointers
  (https://llvm.org/docs/OpaquePointers.html). In LLVM 13, opaque
  pointers are WIP and not recommended for use.

- But as preparation, they already changed some API, specifically
  they expect that IRBuilder::CreateConstGEP*() no longer get nullptr as
  first argument.

- We are now passing explicitly the value which was in LLVM 12 figured
  out automatically under the hood.

- Since the passed pointer is still non-opaque, I don't think this will
  work in LLVM 15 but it could be okay for a migration.
 2022-09-26 19:43:28 +00:00
Nikita Mikhaylov
c993051265 Fix deadlock and flaky test 2022-09-26 15:17:34 +00:00
Sergei Trifonov
060402cab7
Merge pull request #41675 from ZhiguoZh/20220922-lock-opt 
Optimize the lock contentions for ThreadGroupStatus::mutex
 2022-09-26 14:33:47 +02:00
Kruglov Pavel
3dc54272ed
Merge branch 'master' into improve-combinators 2022-09-26 13:03:32 +02:00
Nikita Mikhaylov
4d1d87da2c Delete garbage 2022-09-26 10:24:39 +00:00
Nikita Mikhaylov
852d084950 Save 2022-09-26 10:24:39 +00:00
Nikita Mikhaylov
fa587ce057 Better 2022-09-26 10:24:39 +00:00
Nikita Mikhaylov
a5eacc2e02 Even better 2022-09-26 10:24:39 +00:00
Nikita Mikhaylov
636cedf488 Better 2022-09-26 10:24:39 +00:00
Nikita Mikhaylov
9b35d54e8f First try 2022-09-26 10:24:39 +00:00
Antonio Andelic
97385ca861 Format 2022-09-26 09:33:50 +00:00
Antonio Andelic
56cc3c7137 Better TransactionLog with multiread 2022-09-26 09:31:27 +00:00
Antonio Andelic
cc3719e463 Use multiread 2022-09-26 09:07:34 +00:00
jianmei zhang
cf40c57562 Disable mergetree table with lightweight delete column name 2022-09-26 15:33:43 +08:00
Robert Schulze
9c62797d28
Bump LLVM from 12 to 13 2022-09-26 07:13:46 +00:00
Kseniia Sumarokova
50f7ce6107
Merge branch 'master' into fix-thread-status 2022-09-24 17:22:45 +02:00
Kseniia Sumarokova
21e09f3e1f
Merge pull request #41733 from kssenii/cache-logging-level-reduce 
Change logging levels in cache
 2022-09-24 17:20:46 +02:00
Nikolai Kochetov
1c67437aa4 Fix possible crash for SELECT from Merge table with optimize_monotonous_functions_in_order_by eanbled. 2022-09-23 18:37:25 +00:00
kssenii
0a801dad2a Merge remote-tracking branch 'upstream/master' into fix-thread-status 2022-09-23 19:39:07 +02:00
kssenii
30726721ad Fix threadpool reader (for local fs) 2022-09-23 19:35:16 +02:00
Kseniia Sumarokova
b205824670
Merge branch 'master' into cache-config-allow-readable-format-settings 2022-09-23 15:40:07 +02:00
kssenii
c122e4dd1f Refactor log levels 2022-09-23 15:32:05 +02:00
kssenii
ab702e43fc Merge remote-tracking branch 'upstream/master' into fix-thread-status 2022-09-23 15:21:33 +02:00
Nikita Mikhaylov
dbcba1490c
Try to make MergeTreeRestartingThread more reactive in case of expired sessions (#41092) 2022-09-23 14:40:04 +02:00
Frank Chen
34bcb6a82b Fix style 
Signed-off-by: Frank Chen <frank.chen021@outlook.com>
 2022-09-23 11:48:50 +08:00
Frank Chen
45e3d7d788 Update name/comments 2022-09-23 11:36:06 +08:00
Frank Chen
2344e0738e Keep compatibility during upgrading 2022-09-23 11:32:21 +08:00
Frank Chen
40f9e0b69a Address review comments 
Signed-off-by: Frank Chen <frank.chen021@outlook.com>
 2022-09-23 10:08:42 +08:00
Nikolai Kochetov
00965ce17a Fixing race. 2022-09-22 17:57:04 +00:00
Nikolay Degterinsky
a7f3c7086c
Merge pull request #41341 from evillique/mandatory-identification 
Add a setting requiring no_password to be explicitly specified when creating a user
 2022-09-22 15:19:43 +02:00
kssenii
f038c7ea00 Better config for cache 2022-09-22 14:50:29 +02:00
Nikolai Kochetov
63e757a659
Merge pull request #36379 from amosbird/projection-compatibility-fix 
Make minmax_count projection backward compatible
 2022-09-22 14:15:35 +02:00
Kseniia Sumarokova
b11aa0dd0e
Merge pull request #41680 from ClickHouse/kssenii-patch-5 
Fix error message
 2022-09-22 13:26:42 +02:00
Kseniia Sumarokova
5caac39328
Merge pull request #41508 from kssenii/rename-some-cache-commands 
Rename: `show caches` -> `show filesystem caches`, `describe cache` -> `describe filesystem cache`
 2022-09-22 13:09:14 +02:00
Alexander Tokmakov
021b882455 Revert "Disable parallel s3 multipart upload for part moves. (#41268)" 
This reverts commit 0414d95878.
 2022-09-22 13:59:25 +03:00
Kseniia Sumarokova
73ccf94778
Update FileCacheSettings.cpp 2022-09-22 12:53:26 +02:00
Zhiguo Zhou
223c1230b6 Optimize the lock contentions for ThreadGroupStatus::mutex 
The release of ThreadGroupStatus::finished_threads_counters_memory
via the getProfileEventsCountersAndMemoryForThreads method brings
lots of lock contentions for ThreadGroupStatus::mutex and lowers
the overall performance. This commit optimizes this performance
issue by replacing the method call with an equivalent but more
lightweight code block.
 2022-09-22 16:23:16 +08:00
Alexey Milovidov
e16d583aaf
Merge pull request #41280 from ClickHouse/another_fix_for_an_ast_optimization 
Fix WHERE pushdown when subquery has OFFSET
 2022-09-22 07:33:44 +03:00
Alexey Milovidov
45afacdae4
Merge pull request #41186 from ClickHouse/fix-three-fourth-of-trash 
Fix more than half of the trash
 2022-09-22 07:28:26 +03:00
Alexey Milovidov
5c508909c0
Merge pull request #41605 from ClickHouse/fix-amusing-error 
Fix amusing error in conversion from Ordinary to Atomic
 2022-09-22 07:25:41 +03:00
Frank Chen
99c74d182c
Merge branch 'master' into on_cluster_dll 2022-09-22 09:58:10 +08:00
Nikolai Kochetov
446453bdf5 Fix a destruction order for views ThreadStatus 2022-09-21 18:37:40 +00:00
Nikolay Degterinsky
49e0a87b8f
Merge pull request #41495 from canhld94/ch_fixedstring_len 
Add settings allow_suspicious_fixed_string_types
 2022-09-21 19:35:34 +02:00
Nikolai Kochetov
0414d95878
Disable parallel s3 multipart upload for part moves. (#41268) 
* Disable parallel s3 multipart upload for part moves.

* Add setting s3_allow_parallel_part_upload
 2022-09-21 19:10:32 +02:00
Nikita Taranov
100c055510
Prefetching in aggregation (#39304) 
* impl

* stash

* clean up

* do not apply when HT is small

* make branch static

* also in merge

* do not hardcode look ahead value

* fix

* apply to methods with cheap key calculation

* more tests

* silence tidy

* fix build

* support HashMethodKeysFixed

* apply during merge only for cheap

* stash

* fixes

* rename method

* add feature flag

* cache prefetch threshold value

* fix

* fix

* Update HashMap.h

* fix typo

* 256KB as default l2 size

Co-authored-by: Alexey Milovidov <milovidov@clickhouse.com>
 2022-09-21 18:59:07 +02:00
Alexander Tokmakov
fa6a4c5b78
Merge branch 'master' into another_fix_for_an_ast_optimization 2022-09-21 19:05:08 +03:00
Kruglov Pavel
22e11aef2d
Merge pull request #40910 from Avogar/new-json-formats 
Add new JSON formats, add improvements and refactoring
 2022-09-21 14:19:08 +02:00
Robert Schulze
186d4e4140
Merge pull request #40818 from Vector-Similarity-Search-for-ClickHouse/annoy 
Add annoy index
 2022-09-21 13:42:43 +02:00
Nikolay Degterinsky
7292d47923
Merge branch 'master' into mandatory-identification 2022-09-21 12:18:17 +02:00
Nikolay Degterinsky
f0ca365fa0 Better 2022-09-21 10:16:49 +00:00
Alexey Milovidov
f5f08f973e Fix amusing error in conversion from Ordinary to Atomic 2022-09-21 07:10:33 +02:00
Nikolai Kochetov
9e1bb19559
Purge jemalloc arenas in case of high memory usage. (#40277) 
* Purge jemalloc arenas in case of high memory usage.

* Purge jemalloc arenas in case of high memory usage.

* Get RSS before jemalloc counters. Try to avoid negative RSS.

* Try to avoid negative RSS.

* muzzy -> dirty

* Another fix.

* Update MemoryTracker.cpp

* Wait for purged memory.

* Revert "Wait for purged memory."

This reverts commit 53a2621a2d.
 2022-09-20 23:56:10 +02:00
Alexey Milovidov
45bd3cfc30 Merge branch 'master' into fix-three-fourth-of-trash 2022-09-20 21:27:41 +02:00
Alexander Tokmakov
f9e7451d88
Merge pull request #41343 from azat/deadlock-fix 
Fix possible deadlock with async_socket_for_remote/use_hedged_requests and parallel KILL
 2022-09-20 16:59:31 +03:00
Filatenkov Artur
2139d5be17
Merge branch 'master' into annoy 2022-09-20 15:28:08 +03:00
Duc Canh Le
c3970afc45
Merge branch 'master' into ch_fixedstring_len 2022-09-20 20:22:14 +08:00
kssenii
b8079e4577 Merge remote-tracking branch 'upstream/master' into rename-some-cache-commands 2022-09-20 13:02:18 +02:00
Alexander Tokmakov
c8f2820bf1
Merge pull request #41467 from azat/kill-query/lock-fix 
Fix possible hung/deadlock on query cancellation
 2022-09-20 13:59:55 +03:00
Vitaly Baranov
f58f8104b5
Merge pull request #41474 from vitlibar/mask-some-info-in-logs 
Mask some information in logs.
 2022-09-20 12:11:02 +02:00
Kseniia Sumarokova
3c7bbad0b0
Merge pull request #41437 from kssenii/fix-logical-error-write-through-cache 
Fix assertion in write-through cache
 2022-09-20 11:51:19 +02:00
Vladimir C
4935bf60b7
Merge pull request #41130 from ClickHouse/vdimir/decimal-in-41125 2022-09-20 10:07:50 +02:00
Frank Chen
a89140ae98 Fix style 
Signed-off-by: Frank Chen <frank.chen021@outlook.com>
 2022-09-20 10:37:54 +08:00
Azat Khuzhin
f76f14a99c Fix possible hung/deadlock on query cancellation 
Right due to graceful cancellation of the query, it is possible to hung
accepting new queries or even have a deadlock, this is because
cancellation is done while acquiring ProcessListBase::mutex.

So this patch makes query cancellation lock-free, and now the lock will
be acquired only for preparing the query and after cancel is done.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-09-19 17:31:21 +02:00
Amos Bird
652c7cb519
Add comments to FunctionNameNormalizer. 2022-09-19 22:07:04 +08:00
Vitaly Baranov
052ecaa211
Merge branch 'master' into improve-recovery-of-replicated-access-storage 2022-09-19 15:26:21 +02:00
Vitaly Baranov
f75dd93965 Mask some information in logs. 2022-09-19 15:00:06 +02:00
Kruglov Pavel
47f6f09ce0
Merge branch 'master' into improve-combinators 2022-09-19 14:31:12 +02:00
Kruglov Pavel
712aebc077
Merge pull request #41083 from Avogar/fix-map-combinator 
Fix possible segfaults, use-heap-after-free and memory leak in aggregate function combinators
 2022-09-19 14:29:17 +02:00
Alexander Tokmakov
c6efb1773e
Merge branch 'master' into fix_drop_partition_on_cluster 2022-09-19 15:18:05 +03:00
Yakov Olkhovskiy
2f2080f5a4
Merge branch 'master' into composable-protocol 2022-09-19 08:15:07 -04:00
kssenii
e3cd3686af Rename 2022-09-19 14:02:51 +02:00
Kseniia Sumarokova
631aad1671
Merge branch 'master' into fix-logical-error-write-through-cache 2022-09-19 12:29:55 +02:00
vdimir
a6dc99983d
Fix tidy build 2022-09-19 08:42:23 +00:00
vdimir
5c9bfbc116
Fix incorrect result in case of decimal precision loss in IN operator 2022-09-19 08:42:22 +00:00
Vladimir C
b6b5932efc
Merge pull request #41223 from ClickHouse/vdimir/exponential_rewrite_21557 2022-09-19 10:07:24 +02:00
Duc Canh Le
61d6fae14b add settings allow_suspicious_fixed_string_types 2022-09-19 15:00:04 +08:00
Alexey Milovidov
730655d4fd Fix 8/9 of trash 2022-09-19 08:53:20 +02:00
Alexey Milovidov
ab4db2d0c4 Fix 5/6 of trash 2022-09-19 08:50:53 +02:00
Alexey Milovidov
3c0961c48b
Merge branch 'master' into fix-map-combinator 2022-09-19 09:32:16 +03:00
Alexey Milovidov
b01f5bdca8
Merge pull request #39354 from ClickHouse/window-order-by 
Respect ORDER BY clause in window functions
 2022-09-19 09:22:37 +03:00
Frank Chen
b056bc1021 Add span for executeDDLQueryOnCluster 
Signed-off-by: Frank Chen <frank.chen021@outlook.com>
 2022-09-19 13:43:40 +08:00
Yong Wang
37ae7a8cca Kusto-phase1 : apply parser comments to kusto, remove unused variable 2022-09-18 20:25:27 -07:00
Frank Chen
06ae2fb2b5 Remove assertEOF to improve the compability 
Signed-off-by: Frank Chen <frank.chen021@outlook.com>
 2022-09-19 11:20:58 +08:00
Yong Wang
4e4a04c08b
Merge branch 'ClickHouse:master' into Kusto-phase1 2022-09-18 21:40:30 -04:00
Vitaly Baranov
5365b105cc Add SYSTEM RELOAD USERS command. 2022-09-18 12:44:00 +02:00
Yakov Olkhovskiy
c66f412300 pass session certificate for showCertificate() 2022-09-18 07:11:52 +00:00
Alexey Milovidov
c41e37fa8e Code simplification 2022-09-18 08:58:32 +02:00
Alexey Milovidov
3c43d6516b Fix style 2022-09-18 08:56:52 +02:00
Alexey Milovidov
791de6592b Remove trash from Field 2022-09-18 05:16:08 +02:00
kssenii
51745dcb43 Fix assertion in write-through cache 2022-09-17 20:20:32 +02:00
Anton Popov
07cfb0cba4
Merge branch 'master' into dynamic-columns-21 2022-09-17 19:02:58 +02:00
Igor Nikonov
475443c10c
Merge pull request #41257 from ClickHouse/igor/mixed_improvements_along_the_way 
Tiny improvements along the code
 2022-09-17 10:43:46 +02:00
Alexey Milovidov
c09604bef2
Merge pull request #41363 from kitaisreal/small-style-fixes 
Small style fixes
 2022-09-17 00:19:00 +03:00
avogar
0101cc2e56 Support complex combinators in window transform, arrayReduce*, initializeAggregation and Aggregate functons versionning 2022-09-16 19:07:36 +00:00
avogar
c9a9ef5b7e Fix comments 2022-09-16 16:12:30 +00:00
Alexander Tokmakov
65474830ac make tryEnqueueReplicatedDDL virtual 2022-09-16 16:25:32 +02:00
Robert Schulze
b32b02d844
Merge pull request #40897 from ClickHouse/catboost-bridge-resurrected 
Move CatBoost evaluation into clickhouse-library-bridge
 2022-09-16 13:12:09 +02:00
Alexey Milovidov
da01982652
Merge pull request #41046 from azat/build/llvm-15 
Switch to llvm/clang 15
 2022-09-16 07:31:06 +03:00
Alexander Tokmakov
03c193ccca fix ON CLUSTER with Replicated database cluster 2022-09-15 21:15:57 +02:00
Alexander Tokmakov
32391f737a Merge branch 'master' into fix_41209 2022-09-15 20:27:51 +02:00
Alexander Tokmakov
b26cc5a1ba
Merge pull request #41296 from young-scott/fix_issuse_41096 
fix issue #41096
 2022-09-15 15:44:05 +03:00
Alexander Tokmakov
77c4ea9647 fix 2022-09-15 14:34:31 +02:00
Azat Khuzhin
5cc5ca22de Fix possible deadlock with async_socket_for_remote/use_hedged_requests and parallel KILL 
Right now it is possible to call QueryStatus::addPipelineExecutor() when
the executors_mutex already acquired, it is possible when the query was
cancelled via KILL QUERY.

Here I will show some traces from debugger from a real example, where
tons of ProcessList::insert() got deadlocked.

Let's look at the lock owner for one of the threads that was deadlocked
in ProcessList::insert():

    (gdb) p *mutex
    $2 = {
      __data = {
        __owner = 46899,
      },
    }

And now let's see the stack trace of the 46899:

    #0  __lll_lock_wait () at ../sysdeps/unix/sysv/linux/x86_64/lowlevellock.S:103
    #1  0x00007fb65569b714 in __GI___pthread_mutex_lock (mutex=0x7fb4a9d15298) at ../nptl/pthread_mutex_lock.c:80
    #2  0x000000001b6edd91 in pthread_mutex_lock (arg=0x7fb4a9d15298) at ../src/Common/ThreadFuzzer.cpp:317
    #3  std::__1::__libcpp_mutex_lock (__m=0x7fb4a9d15298) at ../contrib/libcxx/include/__threading_support:303
    #4  std::__1::mutex::lock (this=0x7fb4a9d15298) at ../contrib/libcxx/src/mutex.cpp:33
    #5  0x0000000014c7ae63 in std::__1::lock_guard<std::__1::mutex>::lock_guard (__m=..., this=<optimized out>) at ../contrib/libcxx/include/__mutex_base:91
    #6  DB::QueryStatus::addPipelineExecutor (this=0x7fb4a9d14f90, e=0x80) at ../src/Interpreters/ProcessList.cpp:372
    #7  0x0000000015bee4a7 in DB::PipelineExecutor::PipelineExecutor (this=0x7fb4b1e53618, processors=..., elem=<optimized out>) at ../src/Processors/Executors/PipelineExecutor.cpp:54
    #12 std::__1::make_shared<DB::PipelineExecutor, std::__1::vector<std::__1::shared_ptr<DB::IProcessor>, std::__1::allocator<std::__1::shared_ptr<DB::IProcessor> > >&, DB::QueryStatus*&, void> (__args=@0x7fb63095b9b0: 0x7fb4a9d14f90, __args=@0x7fb63095b9b0: 0x7fb4a9d14f90) at ../contrib/libcxx/include/__memory/shared_ptr.h:963
    #13 DB::QueryPipelineBuilder::execute (this=0x7fb63095b8b0) at ../src/QueryPipeline/QueryPipelineBuilder.cpp:552
    #14 0x00000000158c6c27 in DB::Connection::sendExternalTablesData (this=0x7fb6545e9d98, data=...) at ../src/Client/Connection.cpp:797
    #27 0x0000000014043a81 in DB::RemoteQueryExecutorRoutine::operator() (this=0x7fb63095bf20, sink=...) at ../src/QueryPipeline/RemoteQueryExecutorReadContext.cpp:46
    #32 0x000000000a16dd4f in make_fcontext () at ../contrib/boost/libs/context/src/asm/make_x86_64_sysv_elf_gas.S:71

And also in the logs you can see very strange things for this thread:

    2022.09.13 14:14:51.228979 [ 51145 ] {1712D4E914EC7C99} <Debug> Connection (localhost:9000): Sent data for 1 external tables, total 11 rows in 0.00046389 sec., 23688 rows/sec., 3.84 KiB (8.07 MiB/sec.), compressed 1.1070121092649958 times to 3.47 KiB (7.29 MiB/sec.)
    ...
    2022.09.13 14:14:51.719402 [ 46899 ] {7c90ffa4-1dc8-42fd-938c-4e307c244394} <Debug> executeQuery: (from 10.101.15.181:42478) KILL QUERY WHERE query_id = '1712D4E914EC7C99' (stage: Complete)
    2022.09.13 14:14:51.719488 [ 46899 ] {7c90ffa4-1dc8-42fd-938c-4e307c244394} <Debug> executeQuery: (internal) SELECT query_id, user, query FROM system.processes WHERE query_id = '1712D4E914EC7C99' (stage: Complete)
    2022.09.13 14:14:51.719754 [ 46899 ] {7c90ffa4-1dc8-42fd-938c-4e307c244394} <Trace> ContextAccess (default): Access granted: SELECT(user, query_id, query) ON system.processes
    2022.09.13 14:14:51.720544 [ 46899 ] {7c90ffa4-1dc8-42fd-938c-4e307c244394} <Trace> InterpreterSelectQuery: FetchColumns -> Complete
    2022.09.13 14:14:53.228964 [ 46899 ] {7c90ffa4-1dc8-42fd-938c-4e307c244394} <Debug> Connection (localhost:9000): Sent data for 2 scalars, total 2 rows in 2.6838e-05 sec., 73461 rows/sec., 68.00 B (2.38 MiB/sec.), compressed 0.4594594594594595 times to 148.00 B (5.16 MiB/sec.)

How is this possible? The answer is fibers and query cancellation
routine. During cancellation of async queries it going into fibers again
and try to do this gracefully. However because of this during canceling
query it may call QueryStatus::addPipelineExecutor() from
QueryStatus::cancelQuery().

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-09-15 14:41:27 +03:00
Maksim Kita
276d3628e9 Small style fixes 2022-09-15 13:05:18 +02:00
Frank Chen
f576d8191c Merge branch 'master' into on_cluster_dll 2022-09-15 18:05:36 +08:00
Frank Chen
ac848727e5 Read tracing context in right pos 2022-09-15 18:00:08 +08:00
Nikolay Degterinsky
72d8a04ee6 Add setting 'allow_implicit_no_password' 2022-09-15 08:07:42 +00:00
Robert Schulze
b64af5989e
Merge branch 'master' into opentelemetry_stress 2022-09-15 09:43:11 +02:00
Frank Chen
a4ef0c0281 Set up tracing context for DDLWorker 2022-09-15 14:57:00 +08:00
Frank Chen
490089cc4f Copy tracing context from current thread to DDLLogEntry as parent context 2022-09-15 14:48:24 +08:00
Frank Chen
52224875e2 Serialize tracing context to DDL log entry 2022-09-15 14:47:43 +08:00
Anton Popov
66667737f0 fix writing of empty columns of type Object 2022-09-15 03:38:22 +00:00
avogar
f1a0501eb2 Fix memory leaks and segfaults in combinators 2022-09-14 18:01:49 +00:00
Alexander Tokmakov
fc73134145 fix race between DROP and short form of ATTACH 2022-09-14 18:03:36 +02:00
Kruglov Pavel
3396ff6c3a
Merge pull request #40516 from zjial/record_errors_for_import_by_csv 
Record errors while reading text formats (CSV, TSV).
 2022-09-14 12:52:32 +02:00
young scott
b6888022cd
Merge branch 'master' into fix_issuse_41096 2022-09-14 17:38:22 +08:00
young scott
bc111b5680 fix issuse#41096 2022-09-14 09:15:40 +00:00
Alexander Tokmakov
cf7d928554
Update PredicateRewriteVisitor.cpp 2022-09-13 22:38:55 +03:00
Alexander Tokmakov
0175f3d0d7 fix WHERE pushdown when subquery has OFFSET 2022-09-13 20:44:45 +02:00
Vladimir C
f0e4088c84
Merge pull request #41154 from azat/fix-sorted-block-writer 2022-09-13 19:14:45 +02:00
Dmitry Novik
7e3fb0a681
Merge pull request #40762 from ClickHouse/grouping-comp 
Fix GROUPING function SQL compatibility
 2022-09-13 17:03:06 +02:00
kssenii
52ef3758c4 Merge remote-tracking branch 'upstream/master' into fix-thread-status 2022-09-13 16:34:31 +02:00
Kruglov Pavel
faa62f96ae
Merge branch 'master' into new-json-formats 2022-09-13 14:41:13 +02:00
Antonio Andelic
baf7255cff
Merge pull request #39976 from ClickHouse/keeper-storage 
KeeperMap storage engine
 2022-09-13 14:15:09 +02:00
Anton Popov
1989380485
Merge pull request #41122 from lthaooo/optimize_compact_part_mutation 
optimize compact part mutation
 2022-09-13 14:10:21 +02:00
Igor Nikonov
d8acb0e7ad Tiny improvements along the code 
- less allocations (potentially)
 2022-09-13 12:06:56 +00:00
Alexander Tokmakov
4d146b05a9
Merge pull request #38262 from PolyProgrammist/fix-ordinary-system-unfreeze 
Fix SYSTEM UNFREEZE for ordinary database
 2022-09-13 14:55:11 +03:00
Alexander Tokmakov
2845494624
Merge pull request #41120 from azat/async-insert-queue-term 
Terminate AsynchronousInsertQueue w/o waiting for timeout
 2022-09-13 14:52:50 +03:00
Kseniia Sumarokova
b2c9c04c7b
Merge pull request #40821 from kssenii/improve-marks-cache-loading 
Allow to load marks in threadpool in advance
 2022-09-13 12:31:58 +02:00
Frank Chen
d3265150c0 Make sure span is finished in the onFinish callback 
Signed-off-by: Frank Chen <frank.chen021@outlook.com>
 2022-09-13 18:28:21 +08:00
Kseniia Sumarokova
3dd28bb79a
Merge pull request #41042 from kssenii/refactor-cache-for-async-download 
Cache refactoring for async download
 2022-09-13 12:26:35 +02:00
Robert Schulze
fac1be9700
chore: restore SYSTEM RELOAD MODEL(S) and moniting view SYSTEM.MODELS 
- This commit restores statements "SYSTEM RELOAD MODEL(S)" which provide
  a mechanism to update a model explicitly. It also saves potentially
  unnecessary reloads of a model from disk after it's initial load.

  To keep the complexity low, the semantics of "SYSTEM RELOAD MODEL(S)
  was changed from eager to lazy. This means that both statements
  previously immedately reloaded the specified/all models, whereas now
  the statements only trigger an unload and the first call to
  catboostEvaluate() does the actual load.

- Monitoring view SYSTEM.MODELS is also restored but with some obsolete
  fields removed. The view was not documented in the past and for now it
  remains undocumented. The commit is thus not considered a breach of
  ClickHouse's public interface.
 2022-09-12 19:33:02 +00:00
Kseniia Sumarokova
5d1e9822f8
Merge branch 'master' into fix-filesystem_cache_log 2022-09-12 20:32:42 +02:00