thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-14 02:12:21 +00:00
Code Issues Packages Projects Releases Wiki Activity
119,908 Commits 9,330 Branches 1,783 Tags 2.1 GiB
0db9c79886
Commit Graph

10328 Commits

Author SHA1 Message Date
Nikolai Kochetov
f226397fa3 Fix check. 2023-06-22 19:21:08 +00:00
Alexey Milovidov
369ad0aea3 Remove ALTER of LIVE VIEW 2023-06-22 20:32:29 +02:00
Alexander Tokmakov
3afb0933a5 Merge branch 'master' into fix_trash_in_disks 2023-06-22 20:27:57 +02:00
Nikolai Kochetov
75391afbd8 Fix typo. 2023-06-22 18:16:48 +00:00
Nikolai Kochetov
ba4b5c3350 Fix stupid bug. 2023-06-22 18:15:16 +00:00
kssenii
dd49f4a565 Remove key 2023-06-22 17:07:52 +02:00
Nikolai Kochetov
22e49748b5 Cleanup. 2023-06-22 14:23:04 +00:00
Smita Kulkarni
3c7d749d2c Added check of StorageView before checking parameterized view to fix seg fault when materialized view is used with brackets 2023-06-22 14:47:04 +02:00
Robert Schulze
a9f1393862
Make the fix more minimal 2023-06-22 11:46:27 +00:00
Robert Schulze
a1c131c0da
Query Cache: Try to fix bad cast from ColumnConst to ColumnVector<char8_t> 
Resolves #49445

The query cache buffers query result chunks and eventually squashes
them before insertion into the cache. Here, squashing failed because not
all chunks were of the same type. Looks like chunks of the same
underlying type (e.g. UInt8) in a query result be of mixed const, sparse
or low-cardinality type. Fix this by always materializing the data
regardless of the compression. Strange thing is that the failing query
in the stress test (*) isn't able to reproduce the bug, and I haven't
managed to trigger the issue otherwise, so no test case is added.

(*) SELECT 1 UNION ALL SELECT 1 INTERSECT SELECT 1
    E.g. here: https://s3.amazonaws.com/clickhouse-test-reports/0/18817517ed6f8849e3d979e10fbb273e0edf0eaa/stress_test__debug_/fatal_messages.txt
 2023-06-22 11:46:22 +00:00
kssenii
8ff8125ebd Better 2023-06-22 13:25:47 +02:00
kssenii
4fbf52f2de Less loggers 2023-06-22 13:08:53 +02:00
Kseniia Sumarokova
b7ef782335
Merge pull request #51242 from kssenii/fix-possible-incorrect-elements-limit 
Fix sometimes incorrect current_elements count in fs cache
 2023-06-22 12:51:43 +02:00
Nikolai Kochetov
a940031878 Merge branch 'master' into refactor-subqueries-for-in 2023-06-22 12:18:48 +02:00
Alexey Milovidov
a050b9e9d9
Update FileCache.cpp 2023-06-22 01:28:52 +03:00
Igor Nikonov
e3b43c2b88 Move instead of copy 2023-06-21 21:54:44 +00:00
Igor Nikonov
ad0677147f Trying slightly improve eviction path 
+ less copying of shared pointers
+ less indirection in overflow check
 2023-06-21 21:48:58 +00:00
Nikolai Kochetov
0af028ab3f Refactor a bit more. 2023-06-21 20:55:27 +00:00
Alexander Tokmakov
b7dd334d55
Merge branch 'master' into fix_trash_in_disks 2023-06-21 22:23:14 +03:00
Sema Checherinda
01de36f1fa
Merge pull request #50395 from CheSema/better-log 
require `finalize()` call before d-tor for all writes buffers
 2023-06-21 21:12:02 +02:00
kssenii
78ea4feca0 Merge remote-tracking branch 'upstream/master' into delete-non-evictable-feature 2023-06-21 20:52:08 +02:00
kssenii
9aa4647f4f Delete feature 2023-06-21 20:48:12 +02:00
Kseniia Sumarokova
45d7f39566
Merge pull request #51222 from ClickHouse/kssenii-patch-7 
Disable cache setting do_not_evict_index_and_mark_files
 2023-06-21 20:17:33 +02:00
Anton Popov
022b85ba0b
Merge pull request #51164 from CurtizJ/better-check-part-with-cache 
Better checking of data parts on disks with filesystem cache
 2023-06-21 20:08:53 +02:00
kssenii
38466169c7 Better 2023-06-21 20:07:45 +02:00
kssenii
3ce17fd570 Fix log message 2023-06-21 18:30:46 +02:00
kssenii
509a50f651 Merge remote-tracking branch 'origin/fix-possible-incorrect-elements-limit' into fix-possible-incorrect-elements-limit 2023-06-21 18:29:16 +02:00
kssenii
b007c9f194 Review comments 2023-06-21 18:28:14 +02:00
Kseniia Sumarokova
d18da7098a
Update LRUFileCachePriority.cpp 2023-06-21 17:25:09 +02:00
kssenii
b723f9e7bf Fix sometimes not correct current_elements_num 2023-06-21 16:45:52 +02:00
Kseniia Sumarokova
5c38231a14
One more profile event for fs cache (#51223) 2023-06-21 15:18:25 +02:00
Alexander Tokmakov
318ac807c7 Merge branch 'master' into fix_trash_in_disks 2023-06-21 14:04:01 +02:00
Robert Schulze
dcfcbd8345
Merge remote-tracking branch 'rschu1ze/master' into inefficient-query-cache 2023-06-21 09:23:31 +00:00
Kseniia Sumarokova
b60520ec55
Disable cache setting do_not_evict_index_and_mark_files 2023-06-21 11:15:55 +02:00
Robert Schulze
06e8590c8e
Merge pull request #50775 from rschu1ze/non-experimental-qpl-deflate 
Mark QPL_DEFLATE non-experimental but default: off-by-default
 2023-06-21 10:43:16 +02:00
Kseniia Sumarokova
e2a4526108
Revert bad change 2023-06-21 00:35:01 +02:00
kssenii
0efaecab51 Revert "Merge pull request #51149 from ClickHouse/revert-48821-localfilefunction" 
This reverts commit a09e6bbb8e, reversing
changes made to ce38d64c5a.
 2023-06-21 00:29:46 +02:00
kssenii
b8085dfa0e Fix bad merge 2023-06-21 00:20:40 +02:00
Nikolai Kochetov
266e81702a Fixing clang-tidy 2023-06-20 19:48:39 +00:00
Nikolai Kochetov
c0b1d7f4f2 Fixing 01774_tuple_null_in 2023-06-20 19:12:22 +00:00
Sema Checherinda
9b0c3359cf
Merge branch 'master' into better-log 2023-06-20 20:37:36 +02:00
Alexey Milovidov
c097e6e53d
Merge pull request #50531 from kitaisreal/jit-infrastructure-refactoring 
JIT infrastructure refactoring
 2023-06-20 21:13:44 +03:00
Robert Schulze
bc7df2baca
Merge branch 'master' into non-experimental-qpl-deflate 2023-06-20 20:06:01 +02:00
Nikolai Kochetov
5a1f819e7b Fix more tests. 2023-06-20 17:33:09 +00:00
Nikolai Kochetov
518ba6cbf6 Fix more tests. 2023-06-20 16:43:22 +00:00
Nikolai Kochetov
b4273a1963 Fixing some tests. 2023-06-20 14:53:13 +00:00
kssenii
44377b0180 Rename 2023-06-20 13:36:11 +02:00
kssenii
7df815d280 Fix 2023-06-20 13:33:16 +02:00
kssenii
5c4c2a3982 Merge remote-tracking branch 'upstream/master' into min-file-segment-size 2023-06-20 13:33:08 +02:00
Kseniia Sumarokova
c3227fd44f
Merge pull request #51161 from kssenii/more-metrics-in-cache 
More profile events for fs cache
 2023-06-20 12:44:00 +02:00
Raúl Marín
b1cedf0e42 Simplify handling of AsyncInsertFlush 2023-06-20 11:38:19 +02:00
Raúl Marín
9b899b9351 Fix async insert logs with setting on mergetree 2023-06-20 11:38:19 +02:00
Raúl Marín
5ee813b8d0 Fix style 2023-06-20 11:38:19 +02:00
Raúl Marín
5ac7d95baf Cleanup 2023-06-20 11:38:19 +02:00
Raúl Marín
cf74aee6c9 Log async query flushes to query_log 2023-06-20 11:38:19 +02:00
Raúl Marín
7ab660acfb Extract logging functions outside of executeQueryImpl 2023-06-20 11:38:19 +02:00
Maksim Kita
7e5017dd31 Fixed tests 2023-06-20 11:42:22 +03:00
Maksim Kita
6e26fde707 Updated tests 2023-06-20 11:42:22 +03:00
Maksim Kita
129b97c727 JIT infrastructure refactoring 2023-06-20 11:42:22 +03:00
Nikolai Kochetov
7280078bc6 Refactoring a bit more. 2023-06-19 19:01:56 +00:00
Alexander Tokmakov
a2833b2060 remove AsyncTrashPoolExecutor 2023-06-19 19:58:23 +02:00
Han Fei
959f66eb8a
Merge pull request #50697 from kitaisreal/sort-block-improvements 
Sort block improvements
 2023-06-19 19:02:03 +02:00
Anton Popov
1ef19dbe4d drop cache on check of broken part 2023-06-19 16:32:51 +00:00
kssenii
ec86028fd8 Change to microseconds 2023-06-19 17:48:56 +02:00
kssenii
0ca2916163 More metrics in cache 2023-06-19 17:22:41 +02:00
Sergei Trifonov
5867f39fc7
Merge pull request #50675 from ClickHouse/unify-priorities-pools 
Unify priorities for connection pools
 2023-06-19 17:16:15 +02:00
Nikolai Kochetov
8e7d06e0a4 Remove isReady from FutureSet iface. 2023-06-19 12:56:24 +00:00
Nikolai Kochetov
880c9aea09 Fixing style. 2023-06-19 10:47:43 +00:00
Alexander Tokmakov
caceb7c862
Revert "Added ability to implicitly use file/hdfs/s3 table functions in clickhouse-local" 2023-06-19 13:21:09 +03:00
Maksim Kita
4c72995537 Fixed build 2023-06-19 13:02:35 +03:00
Sema Checherinda
2bbfa279ec fix gtest_lru_file_cache, fix remove from pending 2023-06-18 21:05:00 +02:00
Sema Checherinda
ce7cd67981 finalize CascadeWriteBuffer 2023-06-18 17:35:12 +02:00
Maksim Kita
977cd65cfb Fixed build 2023-06-18 12:52:08 +03:00
Maksim Kita
2f3dce5fe6 Fixed tests 2023-06-18 12:52:08 +03:00
Maksim Kita
7d17a70d17 Fixed tests 2023-06-18 12:52:08 +03:00
Maksim Kita
8d3212363e Fixed tests 2023-06-18 12:52:08 +03:00
Maksim Kita
ca298d66cb Sort block improvements 2023-06-18 12:52:08 +03:00
Robert Schulze
5311ebe1d2
Make query cache work effective with nested queries. 2023-06-17 20:27:25 +00:00
Robert Schulze
24717ed908
Factorize check if query cache exists 2023-06-17 18:24:20 +00:00
Robert Schulze
69e7c1cc82
Introduce convenience ctor for Key construction during read 2023-06-17 18:22:15 +00:00
Robert Schulze
6e8af09289
Provide query cache reader with dummy block 2023-06-17 12:48:29 +00:00
Robert Schulze
0adc98865d
Add variable for access to found key components 2023-06-17 12:43:14 +00:00
Robert Schulze
840e73eb74
Update #includes 2023-06-17 12:28:18 +00:00
Robert Schulze
83e2196a17
Make is_shared const 2023-06-17 12:26:53 +00:00
Alexey Milovidov
df9abccfaa
Merge branch 'master' into localfilefunction 2023-06-17 00:45:54 +03:00
Nikolai Kochetov
afa74f697c Refactor a bit. 2023-06-16 19:38:50 +00:00
Nikolai Kochetov
1cb7ba4895 Revert "Trying to fix a test." 
This reverts commit d0d39c8f61.
 2023-06-15 16:42:38 +00:00
Nikolai Kochetov
d34b6e5f35 Try to fix analyzer tests. 2023-06-15 16:27:51 +00:00
Zhiguo Zhou
ff6629d1d1 Enhance safety of function generateOptimizedDateFilterAST 
This commit checks the corner case where the comparator is none
of equals, notEquals, less, lessOrEquals, greater, greaterOrEquals,
and throws LOGICAL_ERROR exception if so.
 2023-06-15 22:04:13 +08:00
Nikolay Degterinsky
9a25958be8 Add HTTP header filtering 2023-06-15 13:49:49 +00:00
Zhiguo Zhou
d14299eb09 The general optimization of predicates with date/datetime converters 
As is suggested in issue #15257, the function preimage is a general
solution to the optimization problem with predicates containing the
date and datetime converters. This commit implements the idea by
adding the new methods, hasInformationAboutPreimage and getPreimage,
to IFunction/IFunctionBase, and having the specific convert functions
define their own preimage. Moreover, we added a new pass in the
TreeOptimizer and a new AST visitor for in-place rewriting the AST
with the converters' preimage.

Specifically, the optimization is applied to toYear and toYYYYMM.
 2023-06-15 21:48:02 +08:00
kssenii
b8a4a784de Fix unit test, add check 2023-06-15 14:22:13 +02:00
Kseniia Sumarokova
31e08635bf
Merge branch 'master' into unify-priorities-pools 2023-06-15 12:51:45 +02:00
kssenii
0023bf69cb Fix 2023-06-15 12:22:50 +02:00
kssenii
8c610275c2 Review fixes 2023-06-14 23:35:09 +02:00
kssenii
0aa363f525 Fix style check 2023-06-14 23:13:09 +02:00
Nikolai Kochetov
d0d39c8f61 Trying to fix a test. 2023-06-14 18:10:30 +00:00
kssenii
96fb7f04cb Finish download of partially downloaded file segments in the background instead of resizing 2023-06-14 19:06:13 +02:00
Dmitry Novik
1d88b16830
Merge pull request #50584 from ClickHouse/analyzer-optimizations-on-shards 
Analyzer: Do not apply Query Tree optimizations on shards
 2023-06-14 14:31:46 +02:00
kssenii
739dc6739f Progress 2023-06-14 13:51:13 +02:00
Antonio Andelic
4825286b0c
Merge branch 'master' into keeper-with-disks 2023-06-14 10:48:53 +02:00
Nikita Mikhaylov
52a460df67
Tests with parallel replicas are no more "always green" (#50896) 2023-06-13 16:43:35 +02:00
Robert Schulze
3617316026
Merge branch 'master' into non-experimental-qpl-deflate 2023-06-12 16:01:45 +02:00
Dmitry Novik
26c9bda144 Add a comment 2023-06-12 13:54:45 +00:00
Antonio Andelic
cb3ac81110
Merge branch 'master' into keeper-with-disks 2023-06-12 15:14:24 +02:00
Anton Popov
32caf87163
Merge pull request #50104 from amosbird/fix_43107_47549 
Proper mutation of skip indices and projections
 2023-06-12 15:09:18 +02:00
Robert Schulze
8222861f2e
Merge remote-tracking branch 'rschu1ze/master' into non-experimental-qpl-deflate 2023-06-11 14:39:30 +00:00
Alexey Milovidov
2a126aaa76
Merge branch 'master' into fix-attach-table-function-name-normalizer 2023-06-10 13:14:17 +03:00
Robert Schulze
43d4544c97
Merge pull request #50810 from rschu1ze/desctructing 
Desctructing --> Destructing
 2023-06-09 21:59:26 +02:00
Nikolai Kochetov
6ce8329bda Merge branch 'master' into refactor-subqueries-for-in 2023-06-09 20:04:27 +02:00
Nikolai Kochetov
9a4043a4b4 Fixing more tests. 2023-06-09 17:51:59 +00:00
Robert Schulze
f8dc408ccb
Desctructing --> Destructing 2023-06-09 16:23:22 +00:00
Nikolai Kochetov
e0277b071b
Merge pull request #50699 from ClickHouse/vdimir/revert_grace_join_changed 
Revert recent grace hash join changes
 2023-06-09 17:56:52 +02:00
Robert Schulze
1aa158909e
enable_qpl_deflate_codec --> enable_deflate_qpl_codec 2023-06-09 12:43:33 +00:00
jinjunzh
f1192d59af
refine patch according to comments 2023-06-09 12:43:15 +00:00
jinjunzh
056ca4f555
Add extensive testing cases for deflate qpl codec 2023-06-09 12:42:59 +00:00
Nikolai Kochetov
c73a346939
Merge pull request #50721 from ClickHouse/global-join-too-many-columns 
Do not read all the columns from right GLOBAL JOIN table.
 2023-06-09 14:33:07 +02:00
Alexander Tokmakov
8f9c74debb
Support OPTIMIZE for temporary tables (#50710) 
* Update InterpreterOptimizeQuery.cpp

* Update 02525_different_engines_in_temporary_tables.sql

* Update MergeTreeData.cpp
 2023-06-09 14:31:49 +03:00
alesapin
2fec796e73
Merge pull request #50604 from ClickHouse/azure_table_function 
Azure table function
 2023-06-09 11:44:04 +02:00
Kseniia Sumarokova
92de3821b5
Merge pull request #50682 from ClickHouse/kssenii-patch-7 
Update default max_elements limit in fs cache
 2023-06-09 11:18:20 +02:00
Kseniia Sumarokova
029a4fd201
Merge branch 'master' into fix-flaky-unit-test 2023-06-09 11:11:07 +02:00
Kseniia Sumarokova
62fc6cf007
Merge branch 'master' into fix-error-in-stress-test 2023-06-09 11:07:10 +02:00
Nikolai Kochetov
350becba5d Fixing build. 2023-06-08 17:56:35 +00:00
Kseniia Sumarokova
2ea2782447
Merge branch 'master' into kssenii-patch-7 2023-06-08 19:50:57 +02:00
alesapin
37b5be60b5
Merge branch 'master' into fix-flaky-unit-test 2023-06-08 18:37:12 +02:00
Nikolai Kochetov
b6e32cd5e3 Do not read all the columns from right GLOBAL JOIN table. 2023-06-08 15:30:23 +00:00
alesapin
79edd06ee7 Merge remote-tracking branch 'origin/master' into azure_table_function 2023-06-08 17:05:19 +02:00
kssenii
9b04e85137 Fix 2023-06-08 16:43:27 +02:00
kssenii
964e0d042c Fix 2023-06-08 16:25:24 +02:00
Nikolai Kochetov
471b0d8ad6
Merge pull request #50638 from ClickHouse/vdimir/semi_join_use_nulls_50540 
Fix crash in anti/semi join
 2023-06-08 12:43:34 +02:00
Dmitry Novik
a92ce7f696
Merge branch 'master' into analyzer-optimizations-on-shards 2023-06-08 11:55:58 +02:00
vdimir
f0a1c8afa2
Revert "Merge pull request #49483 from bigo-sg/grace_hash_full_join" 
This reverts commit fa93c388b1, reversing
changes made to a2c0a65344.
 2023-06-08 09:40:41 +00:00
vdimir
15b6651df6
Revert "Merge pull request #49816 from bigo-sg/grace_hash_reserve_hash_table" 
This reverts commit 3f892ceb12, reversing
changes made to 32ffa2ae0b.
 2023-06-08 09:40:11 +00:00
Kseniia Sumarokova
9a7852c908
Merge branch 'master' into fix-cache-slow-with-insert 2023-06-08 10:49:46 +02:00
Nikolay Degterinsky
188bac414e
Merge pull request #50280 from lucas-tubi/fix-datetime64-index-scaning 
Fixed type conversion from Date/Date32 to DateTime64 when querying with DateTime64 index
 2023-06-08 07:10:55 +02:00
Kseniia Sumarokova
c2fc0713f2
Update FileCache_fwd.h 2023-06-07 21:08:48 +02:00
kssenii
129473ae74 Fix 2023-06-07 20:48:03 +02:00
Antonio Andelic
b11f744252
Correctly disable async insert with deduplication when it's not needed (#50663) 
* Correctly disable async insert when it's not used

* Better

* Add comment

* Better

* Fix tests

---------

Co-authored-by: Nikita Mikhaylov <mikhaylovnikitka@gmail.com>
 2023-06-07 20:33:08 +02:00
vdimir
a268250aff Fixes for LowCardinality Nullable in HashJoin 2023-06-07 20:16:36 +02:00
vdimir
f3b5a87a66 Fixing crash in anti/semi join 2023-06-07 20:16:36 +02:00
serxa
2df0e0c669 Unify priorities for connections 2023-06-07 16:25:52 +00:00
Alexander Sapin
5d52c41689 Merge branch 'master' into azure_table_function 2023-06-07 12:33:26 +02:00
Kseniia Sumarokova
d25ea9b0cf
Merge pull request #50470 from kssenii/add-some-assertions-2 
Add some assertions
 2023-06-07 12:28:28 +02:00
Alexey Milovidov
9b49469e54
Merge pull request #50637 from Avogar/values-lc-null 
Fix converting Null to LowCardinality(Nullable) in values table function
 2023-06-07 07:38:11 +03:00
Han Fei
4130e1e9ac
Merge branch 'master' into revert_incorrect_optimizations 2023-06-06 21:44:39 +02:00
avogar
e9c267ed69 Fix converting Null to LowCardinality(Nullable) in values table function 2023-06-06 17:51:53 +00:00
Raúl Marín
473743b49f
Disable pure parallel replicas if trivial count optimization is possible (#50594) 2023-06-06 18:38:32 +02:00
Nikolay Degterinsky
38edd6c3e7
Update src/Interpreters/tests/gtest_convertFieldToType.cpp 2023-06-06 18:28:34 +02:00
Nikolay Degterinsky
6c82ee45e2
Fix build 2023-06-06 18:27:16 +02:00
Antonio Andelic
42798e446b Merge branch 'master' into keeper-with-disks 2023-06-06 14:55:47 +00:00
Raúl Marín
8d2b868312 Revert "Merge pull request #50062 from ZhiguoZh/20230511-toyear" 
This reverts commit 55c2dbcc2d, reversing
changes made to 23f894b995.
 2023-06-06 16:10:22 +02:00
Raúl Marín
68a9ea72de Revert "Merge pull request #50307 from ZhiguoZh/20230527-toyyyymm" 
This reverts commit 17261e52da, reversing
changes made to d302eae85e.
 2023-06-06 16:10:01 +02:00
kssenii
c84881ac68 Merge remote-tracking branch 'origin/add-some-assertions-2' into add-some-assertions-2 2023-06-06 15:16:42 +02:00
Nikolay Degterinsky
9ae4d929ea
Update src/Interpreters/tests/gtest_convertFieldToType.cpp 2023-06-06 15:16:18 +02:00
kssenii
a96c1ea86f Fix 2023-06-06 15:15:51 +02:00
Nikita Mikhaylov
e87348010d
Rework loading and removing of data parts for MergeTree tables. (#49474) 
Co-authored-by: Sergei Trifonov <sergei@clickhouse.com>
 2023-06-06 14:42:56 +02:00
Dmitry Novik
0e518d7de2
Merge branch 'master' into analyzer-optimizations-on-shards 2023-06-06 14:02:25 +02:00
Kseniia Sumarokova
175515b9ec
Merge pull request #50583 from kssenii/fix-logical-error-in-try-reserve-2 
Fix logical error in stress test (Not enough space to add ...)
 2023-06-06 12:14:46 +02:00
Michael Kolupaev
9f80900d6f Changes related to an internal feature 2023-06-05 18:20:38 -07:00
Smita Kulkarni
dedb9067ce WIP : Azure Table Function, added read and StorageAzureSource 2023-06-05 20:36:17 +02:00
Kseniia Sumarokova
0832cb2d7a
Update Metadata.h 2023-06-05 17:51:12 +02:00
Kseniia Sumarokova
2b3db1d33c
Update Metadata.cpp 2023-06-05 17:05:06 +02:00
Dmitry Novik
fe1354f221 Analyzer: Do not apply Query Tree optimizations on shards 2023-06-05 12:32:08 +00:00
kssenii
58932b81cc Merge remote-tracking branch 'upstream/master' into fix-logical-error-in-try-reserve-2 2023-06-05 13:07:12 +02:00
kssenii
f3a8517a44 Fix 2023-06-05 13:07:07 +02:00
Kseniia Sumarokova
22110ac742
Fix exception message 2023-06-05 11:59:49 +02:00
Antonio Andelic
c4873027c3 Merge branch 'master' into keeper-with-disks 2023-06-05 07:08:33 +00:00
Alexey Milovidov
4ef0196a1b
Merge branch 'master' into fix-datetime64-index-scaning 2023-06-05 06:42:35 +03:00
Alexey Milovidov
4b1bb75a07
Merge pull request #50383 from bigo-sg/grace_hash_join_mismatch_right_block 
Without applying `prepareRightBlock` will cause mismatch block structrue
 2023-06-05 05:57:16 +03:00
Amos Bird
a66f68e5df
fix again 2023-06-05 01:48:13 +08:00
Amos Bird
e24c9267bc
fix 2023-06-04 20:06:27 +08:00
kssenii
24e015b961 Fix 2023-06-04 13:34:20 +02:00
kssenii
411cf84db2 Merge remote-tracking branch 'upstream/master' into add-some-assertions-2 2023-06-04 13:32:57 +02:00
Amos Bird
92b2200c55
mutation stages can be empty 2023-06-04 18:32:12 +08:00
Amos Bird
97bd3f0483
Proper mutation of skip indices and projections 2023-06-04 18:32:08 +08:00
kssenii
009fe3d25e Add profile events for eviction 2023-06-03 21:35:27 +02:00
kssenii
05fbebbdac Merge remote-tracking branch 'upstream/master' into add-some-assertions-2 2023-06-03 20:19:14 +02:00
Alexey Milovidov
9e938d7eec
Merge pull request #49287 from nickitat/optimize_reading2 
Optimise hot reading from remote tables
 2023-06-02 23:51:57 +03:00
Nikolai Kochetov
975e58c56d Fix style. 2023-06-02 15:15:44 +00:00
Antonio Andelic
5f625dcdd9 Merge branch 'master' into keeper-with-disks 2023-06-02 13:35:18 +00:00
vdimir
b1f58d765a
Merge pull request #48633 from ClickHouse/vdimir/any_join_single_row 2023-06-02 15:05:13 +02:00
kssenii
8884be2439 Merge remote-tracking branch 'upstream/master' into add-some-assertions-2 2023-06-02 14:58:34 +02:00
kssenii
5cb4363e58 Remove assertion 2023-06-02 14:50:00 +02:00
Nikita Mikhaylov
73db383727
Limit the number of in-flight tasks for loading outdated parts (#50450) 
* Done

* Update programs/local/LocalServer.cpp

Co-authored-by: Alexander Tokmakov <tavplubix@clickhouse.com>

* Bump

---------

Co-authored-by: Alexander Tokmakov <tavplubix@clickhouse.com>
 2023-06-02 14:10:26 +02:00
kssenii
79c14c89ee Add some assertions 2023-06-02 12:53:25 +02:00
Nikolai Kochetov
92859ebb3b Fixing more tests. 2023-06-01 21:15:15 +00:00
vdimir
bdb192cf27
Test right join in 02725_any_join_single_row, style code changes 2023-06-01 15:43:37 +00:00
Antonio Andelic
514beb783b Merge branch 'master' into keeper-with-disks 2023-06-01 12:19:43 +00:00
vdimir
62f950ddaf
Keep blocks with nulls for right and full join 2023-06-01 12:03:10 +00:00
vdimir
68d46c81b9
Do not store blocks in hash join if nothing is inseted 2023-06-01 12:03:09 +00:00
Zhiguo Zhou
3d0f336643 Optimize predicate with toYYYYMM converter 
Like toYear(PR #50062), the predicate with toYYYYMM could also be
optimized with its preimage. For example, we could transform the
predicate "toYYYYMM(c) = 202305" to "c >= '2023-05-01' AND c <=
'2023-05-31'", so that the invocation of toYYYYMM is avoided. The
similar transformation also applies to other comparisons.

This commit extends OptimizeDateFilterInPlaceVisitor by having it
recognize the toYYYYMM converter in the WHERE/PREWHERE clauses and
replace it with the above tranformed predicate in-place at the AST
level.
 2023-06-01 10:20:45 +08:00
lgbo-ustc
5dbce62ec3 fixed. without apply prepareRightBlock will cause mismatch block structure 2023-06-01 08:10:28 +08:00
Nikita Mikhaylov
3543d95980
Parallel replicas always skip unavailable ones (#50293) 2023-05-31 22:10:33 +02:00
kssenii
f15769ee6c Fix 2023-05-30 21:29:08 +02:00
Nikolai Kochetov
c740328426 Fix more tests. 2023-05-30 18:51:18 +00:00
Nikolai Kochetov
8cec00dd6e Merge branch 'master' into refactor-subqueries-for-in 2023-05-30 18:08:37 +02:00
Nikolai Kochetov
e91934bcea Fixing style. 2023-05-30 16:02:14 +00:00
Vasily Nemkov
93cf34320c Fixed convertFieldToType case of converting Date and Date32 to DateTime64 Field 
Also added a unit-test that clarifies implementation
 2023-05-30 16:10:52 +02:00
Sema Checherinda
84d6bfe5f6
Merge pull request #50204 from azat/InsertSelectQueries-pe 
Add new profile events for queries with subqueries
 2023-05-30 15:19:04 +02:00
Antonio Andelic
bb2acb50d2
Merge pull request #50186 from LiuYangkuan/support_fqdn_in_cluster_discovery 
support passing fqdn to register cluster node in keeper
 2023-05-30 11:54:09 +02:00
Antonio Andelic
67746bfff2 Merge branch 'master' into keeper-with-disks 2023-05-30 09:50:12 +00:00
SmitaRKulkarni
f2172ab8dd
Merge pull request #46661 from ClickHouse/42192_Lower_parallel_parsing_threads_with_globs 
When reading from multiple files displace parallel parsing
 2023-05-30 08:53:08 +02:00
Lucas Chang
954e76cb7a use getScaleMultiplier 2023-05-30 09:38:50 +08:00
Nikita Taranov
a3652f1727 Merge branch 'master' into optimize_reading2 2023-05-29 15:26:18 +02:00
Robert Schulze
47daaad172
Merge pull request #49778 from helifu/master2 
Support custom 'order by' expression for system log tables
 2023-05-29 10:05:14 +02:00
Alexey Milovidov
9647cfa33d
Merge pull request #50291 from kitaisreal/jit-aggregation-nullable-key-fix 
JIT aggregation nullable key fix
 2023-05-29 02:08:02 +03:00
alesapin
3a3cee586a
Merge pull request #50257 from CurtizJ/useless-refactoring 
Yet another refactoring
 2023-05-28 21:42:16 +02:00
Maksim Kita
e9840bc6e1 JIT aggregation nullable key fix 2023-05-28 21:05:17 +03:00
Nikita Taranov
42a9072888 Merge branch 'master' into optimize_reading2 2023-05-28 18:04:44 +02:00
helifu
6e8136e6f4 Add 'settings' expression for system log tables 2023-05-28 12:59:34 +08:00
helifu
59a815e4e6 Update according to review comments 2023-05-28 12:59:34 +08:00
helifu
57e5a4aa63 Add the 'order_by' description for system log tables 2023-05-28 12:59:34 +08:00
helifu
03e11b6e16 Remove the duplicate pair of parentheses 2023-05-28 12:59:34 +08:00
helifu
90a6437716 Support custom 'order by' expression for system log tables 2023-05-28 12:59:34 +08:00
Lucas Chang
0d873d50a3 format 2023-05-28 10:49:36 +08:00
Lucas Chang
93415789d7 format 2023-05-28 10:42:42 +08:00
Lucas Chang
864ba06029
Merge branch 'master' into fix-datetime64-index-scaning 2023-05-27 20:44:29 +08:00
Lucas Chang
41c599b862 fix date32 2023-05-27 17:59:41 +08:00
Lucas Chang
806b7fc18b fix datetime64 index querying 2023-05-27 17:39:21 +08:00
Nikolai Kochetov
6ed7a3b73f Fixing more tests. 2023-05-26 19:25:33 +00:00
Azat Khuzhin
6b5933a3db Add new profile events for queries with subqueries 
Differs from SelectQuery/InsertQuery that it could be > 1 (in case of
MATERIALIZED VIEW attached to the table), since each local SELECT/INSERT
query is accounted, not only initial.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-05-26 18:28:40 +02:00
Sergei Trifonov
0d1f2e297b
Unify priorities: lower value means higher priority (#50205) 2023-05-26 15:55:30 +02:00
Anton Popov
f9c7fcbe0a
Merge branch 'master' into useless-refactoring 2023-05-26 13:45:58 +02:00
Antonio Andelic
8a2a63a7bd Merge branch 'master' into keeper-with-disks 2023-05-26 10:48:41 +00:00
Raúl Marín
b3a96de533
Pure parallel replicas: JOIN support (#49544) 2023-05-26 01:04:24 +02:00
Anton Popov
612173e734 refactoring near alter conversions 2023-05-25 22:54:54 +00:00
Nikolai Kochetov
b5b261b22c Merge branch 'master' into refactor-subqueries-for-in 2023-05-25 21:22:06 +02:00
Nikolai Kochetov
7ebe19f5fb Fixing more tests 2023-05-25 19:18:11 +00:00
Nikolai Kochetov
30ff5113d9 Fixing more tests. 2023-05-25 13:33:52 +00:00
Kseniia Sumarokova
f1a3c9cfd5
Merge pull request #50109 from kssenii/abstract-async-prefetched-buffer 
Make async prefetched buffer work with arbitrary impl
 2023-05-25 15:06:44 +02:00
Nikita Mikhaylov
1c3b6738f4
Fixes for parallel replicas (#50195) 2023-05-25 14:41:04 +02:00
Han Fei
55c2dbcc2d
Merge pull request #50062 from ZhiguoZh/20230511-toyear 
Optimize predicate with toYear converter
 2023-05-25 13:25:55 +02:00
vdimir
2df4130d82
Update src/Interpreters/ClusterDiscovery.cpp 2023-05-25 13:06:40 +02:00
Sema Checherinda
23f894b995
Merge pull request #49777 from helifu/master1 
Add 'initial_query_id' field for system.processors_profile_log
 2023-05-25 12:55:32 +02:00
Sema Checherinda
3329a8428d
Merge pull request #49779 from helifu/master3 
Add 'partitions' field for system.query_log
 2023-05-25 12:51:40 +02:00
Igor Nikonov
1c0b02c3c4
Merge pull request #49503 from ClickHouse/fill_with_by_sorting_prefix_2 
WITH FILL by sorting prefix
 2023-05-25 12:37:40 +02:00
何李夫
e4c8c4cecf
Add zookeeper name in endpoint id (#49780) 
* Add zookeeper name in endpoint id

When we migrate a replicated table from one zookeeper cluster to
another (the reason why we migration is that zookeeper's load is
too high), we will create a new table with the same zpath, but it
will fail and the old table will be in trouble.

Here is some infomation:
1.old table:
  CREATE TABLE a1 (`id` UInt64)
  ENGINE = ReplicatedMergeTree('/clickhouse/tables/default/a1/{shard}', '{replica}')
  ORDER BY (id);
2.new table:
  CREATE TABLE a2 (`id` UInt64)
  ENGINE = ReplicatedMergeTree('aux1:/clickhouse/tables/default/a1/{shard}', '{replica}')
  ORDER BY (id);
3.error info:
  <Error> executeQuery: Code: 220. DB::Exception: Duplicate interserver IO endpoint:
          DataPartsExchange:/clickhouse/tables/default/a1/01/replicas/02.
          (DUPLICATE_INTERSERVER_IO_ENDPOINT)
  <Error> InterserverIOHTTPHandler: Code: 221. DB::Exception: No interserver IO endpoint
          named DataPartsExchange:/clickhouse/tables/default/a1/01/replicas/02.
          (NO_SUCH_INTERSERVER_IO_ENDPOINT)

* Revert "Add zookeeper name in endpoint id"

This reverts commit 9deb75b249619b7abdd38e3949ca8b3a76c9df8e.

* Add zookeeper name in endpoint id

When we migrate a replicated table from one zookeeper cluster to
another (the reason why we migration is that zookeeper's load is
too high), we will create a new table with the same zpath, but it
will fail and the old table will be in trouble.

* Fix incompatible with a new setting

* add a test, fix other issues

* Update 02442_auxiliary_zookeeper_endpoint_id.sql

* Update 02735_system_zookeeper_connection.reference

* Update 02735_system_zookeeper_connection.sql

* Update run.sh

* Remove the 'no-fasttest' tag

* Update 02442_auxiliary_zookeeper_endpoint_id.sql

---------

Co-authored-by: Alexander Tokmakov <tavplubix@clickhouse.com>
Co-authored-by: Alexander Tokmakov <tavplubix@gmail.com>
 2023-05-25 12:50:14 +03:00
Antonio Andelic
ae84716eba Merge branch 'master' into keeper-with-disks 2023-05-25 08:45:19 +00:00
Zhiguo Zhou
1bc4eb1a6c OptimizeDateFilterVisitor: Revise variable names for clarity 2023-05-25 13:47:03 +08:00
helifu
802b63f2ab Add 'initial_query_id' field for system.processors_profile_log 
Facilitate profile data association and aggregation for the same query
 2023-05-25 09:37:02 +08:00
Zhiguo Zhou
773a5bbbaa Optimize predicate with toYear converter 
The date converters, such as toYear, are widely used in the where
clauses of the SQL queries, however, these conversions are often
expensive due to the complexity of the calendar system.

The function preimage is found an optimization for the predicates
with the converters. Given a predicate, toYear(c) = y, we could
convert it to its equivalent form: c >= b AND c <= e, where b is
"y-01-01" and e is "y-12-31". The similar transformation applies
to other comparisons (<>, <, >, <=, <=).

This commit implemented the above transformation at the AST level
by adding a new pass in the TreeOptimizer and a new AST visitor
for in-place replacing the predicates of toYear with the converted
ones.
 2023-05-25 09:11:51 +08:00
Nikolai Kochetov
d8f39b8df1 Fixing more tests. 2023-05-24 17:53:37 +00:00
helifu
2255b0287a Add 'partitions' field for system.query_log 2023-05-24 20:42:31 +08:00
Igor Nikonov
2f5ed81e0d
Merge branch 'master' into fill_with_by_sorting_prefix_2 2023-05-24 14:40:44 +02:00
vdimir
3f892ceb12
Merge pull request #49816 from bigo-sg/grace_hash_reserve_hash_table 2023-05-24 11:48:19 +02:00
LiuYangkuan
0df4164180 Merge remote-tracking branch 'origin/master' into cluster_discovery 2023-05-24 17:37:01 +08:00
Antonio Andelic
fa3fb49d33 Merge branch 'master' into keeper-with-disks 2023-05-24 09:04:47 +00:00
Antonio Andelic
161afea266 Add support for changelog 2023-05-24 07:43:57 +00:00
alekseygolub
34fe8ba5cd
Merge branch 'ClickHouse:master' into localfilefunction 2023-05-24 09:49:03 +03:00
Igor Nikonov
e9c86527b0
Merge branch 'master' into fill_with_by_sorting_prefix_2 2023-05-23 22:58:21 +02:00