thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-14 18:32:29 +00:00
Code Issues Packages Projects Releases Wiki Activity
99,384 Commits 9,328 Branches 1,783 Tags 2.1 GiB
9f39a6a049
Commit Graph

3508 Commits

Author SHA1 Message Date
Nikita Taranov
e5e0a24ab3 return chunks from prepareBlockAndFillWithoutKey 2022-08-16 18:56:22 +02:00
Igor Nikonov
d4367de7bb Rename setting to optimize_sorting_by_input_stream_properties 2022-08-16 16:27:41 +00:00
Vladimir Chebotaryov
3cc03b141e Fixed tests on Debug build type. 2022-08-16 15:43:37 +02:00
Vladimir Chebotaryov
66f9bfca61 Fixed point of origin for exponential decay window functions to the last value in window. 2022-08-16 15:43:37 +02:00
avogar
99d8727335 Fix tests 2022-08-16 12:56:51 +00:00
avogar
936c457734 Remove unnended field 2022-08-16 09:51:52 +00:00
avogar
e1ff996ec3 Allow to specify structure hints in schema inference 2022-08-16 09:46:57 +00:00
Maksim Kita
110470809b
Merge pull request #40121 from amosbird/profile-processor-1 
Extend processors_profile_log with more info
 2022-08-16 09:49:12 +02:00
Igor Nikonov
aba00952f5 Fix: don't set sort mode in ReadFromMergeTree if sort description empty 2022-08-15 20:58:20 +00:00
Kruglov Pavel
2c5c0d6d47
Fix typo 2022-08-15 19:55:28 +02:00
avogar
ca0d883c0f Fix possible segfault in CapnProto input format 2022-08-15 15:36:18 +00:00
Igor Nikonov
ea10fd65b8 Sorting properties in EXPLAIN PLAN 
~ change formatting for sorting
~ rename sortmode option -> sorting
 2022-08-15 15:14:59 +00:00
avogar
c160033837 Fix 2022-08-15 11:38:28 +00:00
Igor Nikonov
d83bea626c Merge remote-tracking branch 'origin/master' into skipping_sorting_step 2022-08-13 21:46:34 +00:00
Igor Nikonov
f33a0d8c85 More simple way to check if sorting order is preserved 
- there is a case where it's done wrong
 2022-08-12 23:42:37 +00:00
avogar
78e197063c Better example 2022-08-12 19:08:36 +00:00
avogar
763f84b623 Remove bad comment 2022-08-12 19:05:57 +00:00
avogar
9addded80e Remove logging 2022-08-12 19:01:02 +00:00
avogar
000336622a Remove logging 2022-08-12 18:59:52 +00:00
avogar
398576e9c9 Improve and fix dictionaries in Arrow format 2022-08-12 18:56:21 +00:00
Kseniia Sumarokova
a6cfc7bc3b
Merge pull request #34651 from alexX512/master 
New caching strategies
 2022-08-12 17:23:37 +02:00
Anton Popov
4bd50bb06c
Merge branch 'master' into distinct_sorted_simplify 2022-08-12 17:11:18 +02:00
Kruglov Pavel
4c7222d938
Merge pull request #40020 from canhld94/ch_canh_fix_hash 
fix HashMethodOneNumber with const column
 2022-08-12 14:40:24 +02:00
Amos Bird
99a38e41aa
processor profile 2022-08-11 21:03:34 +08:00
Igor Nikonov
75f6fcfa70 Merge remote-tracking branch 'origin/master' into skipping_sorting_step 2022-08-11 12:35:55 +00:00
Amos Bird
fa8fab2e8f
Fix KeyCondition with other filters 2022-08-11 19:20:44 +08:00
Maksim Kita
6bec0f5854
Merge pull request #38956 from vdimir/dict-join-refactoring 
Join with dictionary refactoring
 2022-08-11 11:54:11 +02:00
Vladimir C
2d44e6c458
Merge pull request #39343 from vdimir/refactor-prepared-sets 
Refactor PreparedSets/SubqueryForSet
 2022-08-11 11:19:18 +02:00
Vladimir Chebotaryov
748979a9c0
Merge branch 'master' into betterorderbyoptimization 2022-08-11 11:09:52 +03:00
Duc Canh Le
84cd867aa8 materialize column instead of handling column in hash method 2022-08-11 10:46:06 +08:00
Anton Popov
3fdf428834
Merge pull request #39186 from Avogar/numbers-schema-inference 
Add new features in schema inference
 2022-08-11 00:53:54 +02:00
vdimir
ad91c16ba0
Rename join_common -> JoinUtils 2022-08-10 14:20:28 +00:00
vdimir
b7c5c54181
Fix build 2022-08-10 13:43:55 +00:00
vdimir
5eb4cd39e0
Merge branch 'master' into refactor-prepared-sets 2022-08-10 11:47:49 +00:00
Maksim Kita
aff8149f5c
Merge pull request #39998 from kitaisreal/actions-dag-refactoring 
ActionsDAG rename index to outputs
 2022-08-10 11:44:18 +02:00
Igor Nikonov
754a9fb096 Merge remote-tracking branch 'origin/master' into skipping_sorting_step 2022-08-09 22:20:17 +00:00
Arthur Passos
c4d8ad2222 Add docs 2022-08-09 15:58:46 -03:00
Arthur Passos
e724e7bef6 Update arrow dict to lc comment 2022-08-09 15:52:37 -03:00
Arthur Passos
6eb89fd780 Fix both arrow dict de-serialization and dict of nullable de-serialization 2022-08-09 15:06:22 -03:00
Arthur Passos
be1e32c3f1
Merge branch 'ClickHouse:master' into fix_arrow_column_dictionary_to_ch_lc 2022-08-09 15:04:06 -03:00
Maksim Kita
acbfcf440b
Merge branch 'master' into actions-dag-refactoring 2022-08-09 18:52:08 +02:00
Igor Nikonov
70b52f7cb9 Fix test, review comments 2022-08-09 16:29:56 +00:00
Maksim Kita
a576a55375 Fixed build 2022-08-09 15:03:59 +02:00
Kruglov Pavel
088e8cf9bd
Merge branch 'master' into numbers-schema-inference 2022-08-09 14:00:36 +02:00
Kruglov Pavel
99b9e85a8f
Merge pull request #39646 from Avogar/more-formats 
Add more Pretty formats
 2022-08-09 13:59:47 +02:00
Igor Nikonov
366ead3828 Consider aliases when checking if sorting order is preserved by 
expression
 2022-08-09 11:27:17 +00:00
Igor Nikonov
1439664df6 EXPLAIN tests 2022-08-08 20:46:43 +00:00
Maksim Kita
c030fd05e7 ActionsDAG rename index to outputs 2022-08-08 18:01:32 +02:00
Igor Nikonov
15bdeba074 Fix review comments 2022-08-08 14:43:03 +00:00
avogar
1304e3487c Add comments, remove unneded stuff 2022-08-08 13:43:14 +00:00
Igor Nikonov
86c5280db3 Merge remote-tracking branch 'origin/master' into skipping_sorting_step 2022-08-08 12:43:06 +00:00
Igor Nikonov
6572a447fa Check if sorting order is preserved for FilterStep 2022-08-08 12:42:10 +00:00
avogar
2f95726b06 Fix comments 2022-08-08 12:41:00 +00:00
vdimir
708747ca0b
Merge branch 'master' into refactor-prepared-sets 2022-08-08 14:27:18 +02:00
Vladimir C
6bd4821656
Merge pull request #39575 from vdimir/join_pushdown_column_not_found 
Fix column not found for push down with join
 2022-08-08 14:04:42 +02:00
alexX512
6bf29cb610 Change class LRUCache to class CachBase. Check running CacheBase with default pcahce policy SLRU 2022-08-07 19:59:30 +00:00
Igor Nikonov
254d14b1d3 Remove debug trace from DistinctStep 
+ this information can be seen later via EXPLAIN PLAN
 2022-08-07 13:12:11 +00:00
Igor Nikonov
f52720fedc Try to fix style errors 2022-08-06 23:33:19 +00:00
Igor Nikonov
988ce29e9e Check if expression step violates sorting correctly 2022-08-06 22:37:14 +00:00
Igor Nikonov
8278da6475 Fix: read row counts before move columns out of chunk 2022-08-05 21:29:57 +00:00
Igor Nikonov
9fddf6efde Merge remote-tracking branch 'origin/master' into ordinary_distinct_small_refact 2022-08-05 19:23:44 +00:00
avogar
9b1a267203 Refactor, remove TTL, add size limit, add system table and system query 2022-08-05 16:20:15 +00:00
Igor Nikonov
f5f6ada69b ActionsDAG check is not dependent on sort description 2022-08-05 11:00:18 +00:00
Nikolai Kochetov
cf34232675 Output header is now empty for every MV chain. 
Instead of checking that number of processors different for different
threads, simply always return empty header from buildChainImpl(), by
adding explicit conversion.

v2: ignore UNKNOWN_TABLE errors in test
 2022-08-05 13:16:32 +03:00
Igor Nikonov
246f7b71b6
Merge branch 'master' into skipping_sorting_step 2022-08-05 00:03:54 +02:00
Igor Nikonov
a70c47f780 EXPLAIN PLAN sortmode options 
+ ActionsDAG fixes
 2022-08-04 22:02:53 +00:00
Igor Nikonov
0a659f5ab8 Analyze ActionsDAG in ExpressionStep 
+ check if sorting order can be kept from prevous step
+ EXPLAIN PLAN header=1 contains Sort Mode and Sort Description for each step
+ some tests
+ fix some review comments
 2022-08-04 19:30:25 +00:00
Arthur Passos
62d48053c0 Use insertDefault instead of insert(0) 2022-08-04 15:53:44 -03:00
Arthur Passos
c307e9a228 Fix ArrowColumn dictionary to CH low cardinality conversion 2022-08-04 15:34:44 -03:00
Nikolai Kochetov
658a269d56
Merge branch 'master' into use-dag-in-key-condition 2022-08-04 16:18:40 +02:00
Vladimir C
a627b00c43
Merge branch 'master' into refactor-prepared-sets 2022-08-04 13:27:38 +02:00
Kruglov Pavel
235649cb98
Merge pull request #39458 from Avogar/fix-cancel-insert-into-function 
Fix WriteBuffer finalize when cancel insert into function
 2022-08-04 13:02:08 +02:00
vdimir
298fb2431d
Fix column not found for push down with join 2022-08-04 10:58:21 +00:00
Nikita Taranov
4943202921
Improve memory usage during memory efficient merging of aggregation results (#39429) 2022-08-03 17:56:59 +02:00
Vladimir Chebotaryov
484d8f7028
Merge branch 'master' into betterorderbyoptimization 2022-08-03 08:14:50 +03:00
Igor Nikonov
8eafca65c9 Fix style 2022-08-02 17:58:31 +00:00
Igor Nikonov
30782a2b05 Test: distinct sorted is not used on const column 2022-08-02 17:44:43 +00:00
Kruglov Pavel
6b2186bfeb
Merge branch 'master' into numbers-schema-inference 2022-08-02 19:34:53 +02:00
Igor Nikonov
56697125d3 Merge remote-tracking branch 'origin/master' into distinct_sorted_simplify 2022-08-02 11:18:46 +00:00
Igor Nikonov
4bf42ce867 Merge remote-tracking branch 'origin/master' into skipping_sorting_step 2022-08-02 10:50:25 +00:00
Igor Nikonov
0cb59aa38c ExpressionStep: simple check if expression changes sorting order 2022-08-02 10:45:05 +00:00
Kruglov Pavel
42136b7630
Merge pull request #39647 from Avogar/fix-arrow-strings 
Fix strings in dictionary in Arrow format
 2022-08-01 12:46:07 +02:00
Vladimir Chebotaryov
aaa6049678
Merge branch 'master' into betterorderbyoptimization 2022-08-01 11:42:48 +03:00
Vladimir Chebotaryov
9fc3addea9 Fixed using column_after_join for handling WINDOW expressions in ExpressionAnalyzer, shall be aggregated_columns. 2022-07-31 23:36:20 +03:00
Nikolai Kochetov
22fbfe19a4 Merge branch 'master' into use-dag-in-key-condition 2022-07-31 21:54:12 +02:00
Igor Nikonov
7f0adb5eb0 Merge remote-tracking branch 'origin/master' into skipping_sorting_step 2022-07-31 07:07:36 +00:00
Igor Nikonov
83e1dd1172
Merge branch 'master' into ordinary_distinct_small_refact 2022-07-31 00:23:21 +02:00
Alexey Milovidov
4828be7fc4 Fix double escaping in the metadata of FORMAT JSON 2022-07-30 23:56:41 +02:00
Igor Nikonov
7245ddcc20 Simple refactoring: ordinary DISTINCT implementation 2022-07-30 20:25:56 +00:00
Igor Nikonov
a7cfad105e
Merge branch 'master' into distinct_sorted_simplify 2022-07-30 21:57:53 +02:00
Igor Nikonov
3be51a6dea Construct DistinctSortedTransform only when it makes sense 
otherwise fallback to DistinctTransform (i.e. ordinary distinct)
 2022-07-30 19:41:03 +00:00
Maksim Kita
acb0137dbb
Merge pull request #39718 from kitaisreal/join-enums-refactoring 
Join enums refactoring
 2022-07-30 13:53:08 +02:00
Igor Nikonov
d951154ef4 Proved NULLs direction when compare rows 2022-07-29 22:12:03 +00:00
Igor Nikonov
13dc1697fb Remove unnecessary initialization 2022-07-29 20:34:23 +00:00
Igor Nikonov
b44373ba8f Merge remote-tracking branch 'origin/master' into distinct_sorted_simplify 2022-07-29 20:33:26 +00:00
Igor Nikonov
7b0b38e997 DistinctSortedTransform works only if columns contains sort prefix of 
sort description
 2022-07-29 20:01:07 +00:00
Kruglov Pavel
ccd1e1bdb8
Merge branch 'master' into fix-cancel-insert-into-function 2022-07-29 20:27:32 +02:00
Maksim Kita
8fc6bad4f4 Join enums refactoring 2022-07-29 18:35:05 +02:00
Nikolai Kochetov
59a11b32ad
Merge branch 'master' into use-dag-in-key-condition 2022-07-29 17:01:33 +02:00
Igor Nikonov
fb5799fe7f More generic getInputOrderInfo() in query info 2022-07-29 14:36:36 +00:00
Igor Nikonov
4af435bdda Fix: handle all const columns case correctly 2022-07-28 21:22:06 +00:00
Igor Nikonov
f414af2348
Merge pull request #39538 from ClickHouse/fix_distinct_sorted 
Fix: DistinctSortedTransform doesn't take advantage of sorting
 2022-07-28 21:47:10 +02:00
Sergei Trifonov
edf6b02723
Merge pull request #37558 from ClickHouse/concurrency-control 
add concurrency control for pipeline executors
 2022-07-28 08:56:24 +02:00
Vladimir C
115506356c
Merge branch 'master' into refactor-prepared-sets 2022-07-27 19:57:23 +02:00
Sergei Trifonov
05e4bd6c8b review fix 2022-07-27 15:36:50 +02:00
Igor Nikonov
377c04fbf1 Merge remote-tracking branch 'origin/master' into fix_distinct_sorted 2022-07-27 13:01:18 +00:00
Anton Popov
1547c010b9
Merge pull request #39432 from ClickHouse/distinct_sorted_chunk_perf_impr 
DISTINCT in order: perf improvement
 2022-07-27 14:17:58 +02:00
avogar
01a309d4e3 Fix strings in dictionary in Arrow format 2022-07-27 12:02:27 +00:00
avogar
f925046dc4 Add more Pretty formats 2022-07-27 11:37:02 +00:00
vdimir
8eecb9ef82
upd PreparedSets: rename/change signature of methods, add comments 2022-07-27 11:22:16 +00:00
Nikolai Kochetov
873432fb53
Merge pull request #37849 from ClickHouse/bug-with-fill-date 
Enforce equality of WITH FILL type with ORDER BY column's type for date/time types.
 2022-07-27 12:27:53 +02:00
Kruglov Pavel
381ea139c2
Merge branch 'master' into schema-inference-cache 2022-07-27 11:35:36 +02:00
Igor Nikonov
64e51e56e7 Allocate memory for column arrays once 2022-07-27 08:22:07 +00:00
Igor Nikonov
12a7567402 Some polishing 2022-07-27 07:58:54 +00:00
Igor Nikonov
589104fa6e Make building column arrays for chunk processing straightforward 2022-07-27 07:44:42 +00:00
Alexander Gololobov
a64aa00869
Merge pull request #37893 from zhangjmruc/feature/sql-standard-delete 
Support SQL standard "delete from ... where ..." syntax and lightweight implementation on merge tree tables
 2022-07-26 23:39:07 +02:00
Igor Nikonov
cac4d77d0b Merge remote-tracking branch 'origin/master' into distinct_sorted_chunk_perf_impr 2022-07-26 20:36:38 +00:00
Yakov Olkhovskiy
0055c9307d
style fix 2022-07-26 16:08:03 -04:00
Igor Nikonov
24f3a6905f
Merge branch 'master' into fix_distinct_sorted 2022-07-26 21:57:44 +02:00
Igor Nikonov
d196ab24d4 Calculate DISTINCT columns positions which form sort prefix in sort 
description once
 2022-07-26 19:55:29 +00:00
Yakov Olkhovskiy
d93c67e303 comment and test added 2022-07-26 15:28:11 -04:00
vdimir
d9928ac93d
Add methods to SubqueryForSet, do not use refernce to SetPtr 2022-07-26 18:39:09 +00:00
vdimir
1e3fa2e01f
Refactor PreparedSets/SubqueryForSet 2022-07-26 18:39:02 +00:00
Nikolai Kochetov
d7cc3831ea Push predicate over StorageMerge. 2022-07-26 15:01:39 +00:00
Kruglov Pavel
c362551642
Merge pull request #39357 from Avogar/fix-protobuf-capnp-empty-message 
Fix schema inference in case of empty messages in Protobuf/CapnProto formats
 2022-07-26 12:33:28 +02:00
Kruglov Pavel
53159db782
Merge branch 'master' into numbers-schema-inference 2022-07-26 12:32:49 +02:00
Sergei Trifonov
24ab5fbb86 fix finish() condition to account only active threads 2022-07-26 11:32:45 +02:00
Alexander Gololobov
0666ec2e1f
Merge branch 'master' into feature/sql-standard-delete 2022-07-26 10:42:39 +02:00
Vladimir Chebotaryov
f32d9c5539
Uppercase ROWS, GROUPS, RANGE in queries with windows. (#39410) 2022-07-25 22:53:53 +02:00
Alexander Gololobov
25deba2c1b
Merge branch 'master' into feature/sql-standard-delete 2022-07-25 22:13:20 +02:00
Nikolai Kochetov
b70be40804
Merge branch 'master' into use-dag-in-key-condition 2022-07-25 14:30:22 +02:00
Igor Nikonov
572dd864fe Merge remote-tracking branch 'origin/master' into skipping_sorting_step 2022-07-25 12:19:14 +00:00
Sergei Trifonov
d9d1381c3f Merge branch 'master' into concurrency-control 2022-07-25 14:16:46 +02:00
Kruglov Pavel
83c7da6e88
Merge branch 'master' into fix-protobuf-capnp-empty-message 2022-07-25 13:02:41 +02:00
Igor Nikonov
dc96a6270d Remove optimize_memory_usage flag 
Memory usage with DistinctSortedTransform fixed in #39538
 2022-07-25 10:27:33 +00:00
Alexey Milovidov
388d06fda1
Merge pull request #39535 from ClickHouse/stringref 
Less usage of StringRef
 2022-07-25 04:06:11 +03:00
Igor Nikonov
41e72aac83 Fix: DistinctSortedTransform doesn't take advantage of sorting 
clearing_columns are set incorrectly, so we never clear HashSet
 2022-07-24 21:35:36 +00:00
Alexander Gololobov
460950ecdc
Merge branch 'master' into feature/sql-standard-delete 2022-07-24 21:27:22 +02:00
Robert Schulze
4333750985
Less usage of StringRef 
... replaced by std::string_view, see #39262
 2022-07-24 18:33:52 +00:00
Igor Nikonov
95511428b3 Couple optimizations 
+ do not apply filter to chunk if there is no data for output
+ checking clear_data flag at compile time
 2022-07-23 00:03:26 +00:00
Igor Nikonov
739ff34c6e Add some tests, still not sure about optimize_memory_usage option 2022-07-22 22:48:26 +00:00
Igor Nikonov
329adc3419
Merge branch 'master' into distinct_sorted_chunk_perf_impr 2022-07-22 12:58:59 +02:00
Igor Nikonov
51f63c3a0c Add setting optimize_distinct_in_order_memory_usage 2022-07-22 10:57:57 +00:00
Alexander Tokmakov
bed2206ae9
Merge pull request #39460 from ClickHouse/remove_some_dead_and_commented_code 
Remove some dead and commented code
 2022-07-22 13:24:34 +03:00
Igor Nikonov
7db5d54820 Adopt to the case when not all columns in distinct are part of sorting 
description
 2022-07-21 21:04:58 +00:00
avogar
794aa691bc Merge branch 'master' of github.com:ClickHouse/ClickHouse into fix-protobuf-capnp-empty-message 2022-07-21 17:04:37 +00:00
Kruglov Pavel
9252f42b4c
Merge branch 'master' into schema-inference-cache 2022-07-21 18:59:14 +02:00
Robert Schulze
ea0a3bf600
Merge branch 'master' into stringref-to-string_view 2022-07-21 18:33:06 +02:00
avogar
fd534aa3fa wqMerge branch 'master' of github.com:ClickHouse/ClickHouse into numbers-schema-inference 2022-07-21 15:43:17 +00:00
Igor Nikonov
122a1123b2 - disable the worst case for distinct in order in perf test for now 
+ functional test for query with the worst perfomance
+ debug logging in DistinctStep
 2022-07-21 15:03:19 +00:00
Alexander Tokmakov
a8da5d96fc remove some dead and commented code 2022-07-21 15:05:48 +02:00
avogar
6b541aa98f Fix WriteBuffer finalize when cancel insert into function 2022-07-21 12:18:37 +00:00
Alexander Gololobov
1ea9f143ff Leave only _row_exists-based implementation of lightweight delete 2022-07-21 11:26:13 +02:00
Nikolai Kochetov
e15967e9db
Merge pull request #38475 from ClickHouse/additional-filters 
Additional filters for a table (from setting)
 2022-07-21 07:52:04 +02:00
Alexey Milovidov
844042fc18
Merge pull request #39433 from ClickHouse/revert-39396-try-fix-write-buffer-terminate 
Revert "Fix WriteBuffer finalize in destructor when cacnel query"
 2022-07-21 07:04:07 +03:00
Alexey Milovidov
dcda9d3bd1
Merge pull request #39365 from Avogar/fix-capnproto-abort 
Avoid possible abort() in CapnProto on exception descruction
 2022-07-21 05:20:45 +03:00
Kruglov Pavel
92995a832b
Revert "Fix WriteBuffer finalize in destructor when cacnel query" 2022-07-21 01:45:16 +02:00
Igor Nikonov
635a566bec Comment change 2022-07-20 21:37:46 +00:00
Igor Nikonov
e50aebb5f0
Merge branch 'master' into distinct_sorted_chunk_perf_impr 2022-07-20 23:17:11 +02:00
Igor Nikonov
965f96bd84 DISTINCT in order: perf improvement 
+ reduce allocations in DistinctSortedChunkTransform
+ use it for final distinct as well
 2022-07-20 20:44:47 +00:00
Nikolai Kochetov
91043351aa Fixing build. 2022-07-20 20:30:16 +00:00
Nikolai Kochetov
4e8cd70b1d
Merge branch 'master' into use-dag-in-key-condition 2022-07-20 17:38:33 +02:00
Yakov Olkhovskiy
c4d040e02c
Merge branch 'master' into bug-with-fill-date 2022-07-20 09:10:45 -04:00
Dmitry Novik
a09422de7b
Merge pull request #38642 from ClickHouse/group-by-use-nulls 
Add group_by_use_nulls setting
 2022-07-20 14:42:42 +02:00
Kruglov Pavel
46da17ca8c
Merge branch 'master' into numbers-schema-inference 2022-07-20 13:32:39 +02:00
Kruglov Pavel
3046cd6d29
Merge branch 'master' into schema-inference-cache 2022-07-20 13:30:42 +02:00
avogar
784ee11594 Add settings to skip fields with unsupported types in Protobuf/CapnProto schema inference 2022-07-20 11:16:25 +00:00
Kruglov Pavel
a1b63b4a02
Fix style 2022-07-20 12:07:22 +02:00
Kruglov Pavel
7722b647b7
Merge pull request #39396 from Avogar/try-fix-write-buffer-terminate 
Fix WriteBuffer finalize in destructor when cacnel query
 2022-07-20 12:06:20 +02:00
Nikolai Kochetov
f570cde815 Fixing build. 2022-07-19 20:19:57 +00:00
avogar
5c16d6b553 Fix WriteBuffer finalize in destructor when cacnel query 2022-07-19 19:21:30 +00:00
Igor Nikonov
c74600d282
Merge branch 'master' into skipping_sorting_step 2022-07-19 18:59:36 +02:00
Igor Nikonov
1fe83cc8d8 optimize_sorting_for_input_stream setting and perf tests 2022-07-19 16:58:15 +00:00
Nikolai Kochetov
eaeb30a71a Merge branch 'master' into use-dag-in-key-condition 2022-07-19 18:39:52 +02:00
Dmitry Novik
50989bdb68
Merge branch 'master' into group-by-use-nulls 2022-07-19 14:58:01 +02:00
avogar
4f020654be Get rid of unneded ifdefs 2022-07-19 12:12:40 +00:00
Robert Schulze
81ef1099cc
Even less usage of StringRef 
--> see #39300
 2022-07-19 07:01:06 +00:00
avogar
6eb234a1cc Avoid abort() in capnproto on exception descruction 2022-07-18 19:53:24 +00:00
Alexander Gololobov
9de72d995a POC lightweight delete using __row_exists virtual column and prewhere-like filtering 2022-07-18 20:06:42 +02:00
Robert Schulze
32637cb1b9
Fix build 2022-07-18 07:58:59 +00:00
Robert Schulze
13482af4ee
First try at reducing the use of StringRef 
- to be replaced by std::string_view
- suggested in #39262
 2022-07-17 17:26:02 +00:00
Robert Schulze
fffeab2e86
Merge pull request #39262 from ClickHouse/stringref-by-value 
Pass const StringRef by value, not by reference
 2022-07-17 16:53:15 +02:00
Igor Nikonov
508fd5616d Merge remote-tracking branch 'origin/master' into skipping_sorting_step 2022-07-16 00:24:20 +00:00
Igor Nikonov
ef0ef9e03b
Merge pull request #39191 from ClickHouse/sort_transform_cleanup 
Cleanup: done during #38719 (SortingStep: deduce way to sort based on input stream sort)
 2022-07-16 01:53:58 +02:00
Igor Nikonov
e607d103cc Just to trigger CI 2022-07-15 23:16:02 +00:00
Igor Nikonov
6f224b026a Perf test. Code polishing 2022-07-15 21:54:57 +00:00
Igor Nikonov
d9b312f955 Self-review + revert test 2022-07-15 17:00:25 +00:00
Robert Schulze
deda29b46b
Pass const StringRef by value, not by reference 
See #39224
 2022-07-15 11:34:56 +00:00
Igor Nikonov
a3a1ccc520 Fix: SortMode::Chunk 2022-07-15 10:26:13 +00:00
jianmei zhang
9d27af7ee2 For some columns mutations, skip to apply deleted mask when read some columns. Also add unit test case 2022-07-15 12:32:41 +08:00
Igor Nikonov
8170f4e33a
Merge branch 'master' into skipping_sorting_step 2022-07-14 23:05:45 +02:00
Igor Nikonov
1efdb4e3e5 Disable finish sort with sorted chunks 2022-07-14 21:02:44 +00:00
Igor Nikonov
7cd12393c2 If sorting type is specified then use it. Otherwise rely on sort description 2022-07-14 16:26:25 +00:00
Igor Nikonov
1d49adad20 Introduce Auto mode for sorting step (replace others for now) 2022-07-14 13:29:39 +00:00
Kruglov Pavel
b38241b08a
Merge branch 'master' into schema-inference-cache 2022-07-14 12:29:54 +02:00
Ilya Yatsishin
5c8b9e4a14
Merge pull request #39170 from qoega/otel-trace-processors-setting 
Add setting opentelemetry_trace_processors
 2022-07-14 11:27:05 +02:00
Vladimir C
66f2ea5ebb
Merge pull request #39123 from vdimir/fjs-pushdown 
Pushdown filter to the right side of sorting join
 2022-07-14 09:32:19 +02:00
Igor Nikonov
b7f46d954e Cleanup: related to #38719 (SortingStep: deduce way to sort based on input stream sort) 2022-07-13 17:57:37 +00:00
Igor Nikonov
b73aca2a3b
Merge branch 'master' into skipping_sorting_step 2022-07-13 19:06:41 +02:00
Igor Nikonov
159c9428bd clean up 2022-07-13 17:05:54 +00:00
Igor Nikonov
f0d547993a Fix: 01655_plan_optimizations_optimize_read_in_window_order_long 
basically I returned code. Both plans with Finish sorting, need to check
sorting prefix
 2022-07-13 16:51:51 +00:00
avogar
7cde9d3b40 Add new features in schema inference 2022-07-13 15:57:55 +00:00
Vladimir C
27ec36ca91
Merge pull request #37648 from vdimir/enable_parallel_hash 
Bugfixes and performance improvements for parallel_hash
 2022-07-13 14:52:57 +02:00
vdimir
4124dc9ac4
Rewrite tryPushDownFilter for join with lambda 2022-07-13 12:06:29 +00:00
vdimir
549a85fee9
Throw logical error on child idx mismatch in tryAddNewFilterStep 2022-07-13 11:53:46 +00:00
vdimir
fa59133463
Do not spam log in MergeJoinAlgorithm 2022-07-13 11:51:11 +00:00
Nikolai Kochetov
4f8cc871eb
Merge pull request #39132 from ClickHouse/fix-data-race-in-completed-pipeline-executor 
Fix data race in CompletedPipelineExecutor.
 2022-07-13 13:39:44 +02:00
vdimir
bddf6c1b32
Pushdown filter to the right side of sorting join 2022-07-13 11:36:25 +00:00
Yatsishin Ilya
d886aea351 Add setting opentelemetry_trace_processors 2022-07-13 11:31:24 +00:00
Vladimir C
d1d1e4d8a1
Merge pull request #38943 from amosbird/better-join-plan1 
Avoid redundant join block transformation during planning.
 2022-07-13 12:39:45 +02:00
Igor Nikonov
1d6f699a12 Use sort mode Port for reading in order 2022-07-12 21:56:00 +00:00
Dmitry Novik
5f65b45269
Merge branch 'master' into group-by-use-nulls 2022-07-12 22:36:04 +02:00
vdimir
63aebd17b2 Remove TabSeparatedSorted 2022-07-12 20:22:35 +02:00
vdimir
46df417c2e Fix empty line sorting in TabSeparatedSorted 2022-07-12 20:22:35 +02:00
vdimir
f51b25b262 clickhouse test ignore order via special format 2022-07-12 20:22:35 +02:00
Nikolai Kochetov
ec24f730b1
Merge pull request #39125 from ClickHouse/fix-insert-into-mv-with-extremes-enabled 
Fix insert into MV with enabled extremes.
 2022-07-12 20:19:41 +02:00
Nikolai Kochetov
93bf22f585 Fix data race in CompletedPipelineExecutor. 2022-07-12 17:59:31 +00:00
Amos Bird
982e1a73d3
Better 2022-07-12 22:21:46 +08:00
Amos Bird
d3709c6c26
Avoid redundant join block transformation. 2022-07-12 22:20:10 +08:00
Amos Bird
b9d9ca5194
style fix 2022-07-12 22:20:08 +08:00
Nikolai Kochetov
75c3926cbb Fix insert into MV with enabled extremes. 2022-07-12 13:57:36 +00:00
Dmitry Novik
aabf5123d6 Fixup 2022-07-12 13:46:06 +00:00
Igor Nikonov
a1f36d5536
Merge branch 'master' into skipping_sorting_step 2022-07-12 15:15:14 +02:00
Igor Nikonov
2c8d9080bd Fix: consider collation in column sort description comparison 2022-07-12 13:14:10 +00:00
Dmitry Novik
cfca3db884 Fix crash with totals 2022-07-12 12:15:43 +00:00
mergify[bot]
48688da835
Merge branch 'master' into concurrency-control 2022-07-12 10:57:13 +00:00
Igor Nikonov
ea5e7793b2 Fix: self-review comments 2022-07-11 21:26:39 +00:00
Igor Nikonov
e0776b1c82 Fix: test for optimize read in window order 
+ code polishing
 2022-07-11 20:59:38 +00:00
Igor Nikonov
0ca8166ab2 Fix: forgot to return sorting type in constructors 2022-07-11 20:59:38 +00:00
Igor Nikonov
47bed7e318 Try to choose sorting transform based on sort description with fallback 2022-07-11 20:59:38 +00:00
Igor Nikonov
53272b2f08 Code polishing 2022-07-11 20:59:38 +00:00
Igor Nikonov
2a7e3bd741 Fix + SortMode::None as default value 2022-07-11 20:59:38 +00:00
Igor Nikonov
16d2319a8d SortingStep: type of sorting is deduced based on input stream sorting description in during transformation 
+ perf test
 2022-07-11 20:59:38 +00:00
Igor Nikonov
7d4d92bd61 In case full sort was wrong choise during plan interpretation 2022-07-11 20:59:38 +00:00
Igor Nikonov
67ce421e38 Skip sorting step if input stream is globally sorted 2022-07-11 20:59:38 +00:00
Robert Schulze
1a7727a254
Prefix overridden add_executable() command with "clickhouse_" 
A simple HelloWorld program with zero includes except iostream triggers
a build of ca. 2000 source files. The reason is that ClickHouse's
top-level CMakeLists.txt overrides "add_executable()" to link all
binaries against "clickhouse_new_delete". This links against
"clickhouse_common_io", which in turn has lots of 3rd party library
dependencies ... Without linking "clickhouse_new_delete", the number of
compiled files for "HelloWorld" goes down to ca. 70.

As an example, the self-extracting-executable needs none of its current
dependencies but other programs may also benefit.

In order to restore access to the original "add_executable()", the
overriding version is now prefixed. There is precedence for a
"clickhouse_" prefix (as opposed to "ch_"), for example
"clickhouse_split_debug_symbols". In general prefixing makes sense also
because overriding CMake commands relies on undocumented behavior and is
considered not-so-great practice (*).

(*) https://crascit.com/2018/09/14/do-not-redefine-cmake-commands/
 2022-07-11 19:36:18 +02:00
vdimir
da523f3288
Fix assertion in full soring merge join 2022-07-08 11:31:15 +00:00
Dmitry Novik
d1df66687b
Merge branch 'master' into group-by-use-nulls 2022-07-07 20:54:38 +02:00
Dmitry Novik
1587385f7a Cleanup code 2022-07-07 18:53:20 +00:00
Robert Schulze
f15d9ca59c
Merge pull request #38774 from zvonand/zvonand-nnd 
Reintroduce nonNegativeDerivative()
 2022-07-07 20:39:13 +02:00
vdimir
7c586a9e7c
Minor updates for full soring merge join 2022-07-06 14:28:05 +00:00
vdimir
881d352e05
upd full sorting join 2022-07-06 14:28:05 +00:00
vdimir
aff6654d52
minor changes in full sort join 2022-07-06 14:27:33 +00:00
vdimir
f8e66601a7
Fix column remap in MergeJoinTransform 2022-07-06 14:27:32 +00:00
vdimir
0b994bb258
fix build 2022-07-06 14:27:32 +00:00
vdimir
753a567da8
full sorting join with using 2022-07-06 14:27:29 +00:00
vdimir
a90ac59ee5
MergeJoinAlgorithm::createBlockWithDefaults 2022-07-06 14:26:19 +00:00
vdimir
d184e184b4
full sort join: check key types, more tests 2022-07-06 14:26:19 +00:00
vdimir
a2a7abc2e9
add not implemented checks, add using testcase to full sort join 2022-07-06 14:26:18 +00:00
vdimir
92ff43eb7c
tests full sort join 2022-07-06 14:26:18 +00:00
vdimir
a0144e115d
full sorting all join 2022-07-06 14:26:18 +00:00
vdimir
4e88e8f5ec
full sort join: move block list to all join state 2022-07-06 14:26:17 +00:00
vdimir
94192a23fc
enable total compare in MergeJoinAlgorithm 2022-07-06 14:26:16 +00:00
vdimir
a92c60ba06
fix nulls comparsion in full sorting join 2022-07-06 14:26:15 +00:00
vdimir
7c5a5f4b64
full sorted any join tests passed 2022-07-06 14:26:15 +00:00
vdimir
26d812ec72
wip any full sorting merge, rewrite cursor 2022-07-06 14:26:14 +00:00
vdimir
a2d190edb8
wip MergeJoinTransform 2022-07-06 14:25:12 +00:00
vdimir
0b9d4ee640
wip sort join same rows 2022-07-06 14:25:12 +00:00
vdimir
6d198ff3d7
fix style 2022-07-06 14:25:11 +00:00
vdimir
88d8dc5be2
wip full sort any join 2022-07-06 14:25:11 +00:00
vdimir
ba787db0bb
Fix build, small changes 2022-07-06 14:25:10 +00:00
vdimir
d34a66c915
wip sorting merge 2022-07-06 14:25:09 +00:00
vdimir
1b429fc1af
wip: any left/right sorting join 2022-07-06 14:23:46 +00:00
vdimir
8dce97123c
wip: any inner full sorting join 2022-07-06 14:23:46 +00:00
vdimir
4a16195964
Calculate output header for full sorting merge join 2022-07-06 14:23:45 +00:00
vdimir
fa8eb35599
Pipeline for full sorting merge join 2022-07-06 14:23:44 +00:00
Maksim Kita
b94489d52c
Merge pull request #38859 from kitaisreal/merge-tree-merge-disable-batch-optimization 
MergeTree merge disable batch optimization
 2022-07-06 15:59:40 +02:00
Nikolai Kochetov
7de2f229ab
Merge pull request #38584 from ClickHouse/filimonov-AggregatingTransform-expandPipeline 
Add check for empty proccessors in AggregatingTransform::expandPipeline
 2022-07-06 14:38:40 +02:00
Andrey Zvonov
7de39d9b15 Merge branch 'master' of github.com:ClickHouse/ClickHouse into zvonand-nnd 2022-07-06 10:59:35 +03:00
Maksim Kita
bdc21737d5 MergeTree merge disable batch optimization 2022-07-05 16:15:00 +02:00
zvonand
8a270c01e9 fix floating point in intervals 2022-07-04 20:45:05 +03:00
Dmitry Novik
864ab20582 Use correct intermediate header for ROLLUP and CUBE 2022-07-04 16:17:58 +00:00
Kruglov Pavel
4080f055b6
Merge pull request #38477 from Avogar/sql-insert-format 
Add SQLInsert output format
 2022-07-04 15:06:33 +02:00
zvonand
f814985adf minor improvements 2022-07-04 16:03:59 +03:00
zvonand
eac84351f6 fix behavior 2022-07-04 01:26:07 +03:00
Igor Nikonov
2e2ef08712
Merge pull request #37803 from ClickHouse/dictinct_in_order_optimization 
DISTINCT in order optimization
 2022-07-03 21:59:04 +02:00
Igor Nikonov
2577b59f4c
Merge pull request #38721 from ClickHouse/sort_description_4_read_from_merge_tree 
Provide sort description for output stream in ReadFromMergeTree step
 2022-07-03 18:06:25 +02:00
zvonand
8e99ea84a8 fix LOGICAL_ERROR 2022-07-02 14:09:51 +03:00
mergify[bot]
12f5250e86
Merge branch 'master' into dictinct_in_order_optimization 2022-07-01 22:51:35 +00:00
Igor Nikonov
9ef8ff5a31 Addressing review comments 2022-07-01 22:50:00 +00:00
Igor Nikonov
1d665e9576 Provide sort description for output stream in ReadFromMergeTree step 2022-07-01 21:28:01 +00:00
Anton Popov
ef87e1207c better support of read_in_order in case of fixed prefix of sorting key 2022-07-01 16:45:01 +00:00
Dmitry Novik
81dd90893e Merge remote-tracking branch 'origin/master' into group-by-use-nulls 2022-07-01 16:24:05 +00:00
Nikita Taranov
8ba3d405de impl 2022-07-01 16:05:32 +02:00
zvonand
3b5332d15e Revert "Revert "Non Negative Derivative window function"" 
This reverts commit dea3b5bfce.
 2022-07-01 18:59:07 +05:00
Alexey Milovidov
20841f0e1e
Merge pull request #38551 from ClickHouse/revert-37628-non-neg-deriv 
Revert "Non Negative Derivative window function"
 2022-07-01 02:46:28 +03:00
avogar
5b0fd31c64 Put column names in quotes 2022-06-30 16:14:30 +00:00
Dmitry Novik
33f601ec0a Commit support use_nulls for GS 2022-06-30 15:14:26 +00:00
Igor Nikonov
488ee75fc4 + use DistinctSorted for final distinct step 
+ fix performance tests
 2022-06-30 13:03:39 +00:00
Maksim Kita
0de66a2712
Merge pull request #38449 from ClickHouse/revert-38361-revert-38324-fix-partial-sort 
Revert "Revert "Fix optimization in PartialSortingTransform (SIGSEGV and possible incorrect result)""
 2022-06-30 13:02:38 +02:00
Dmitry Novik
98e9bc84d5 Refector ROLLUP and CUBE 2022-06-30 10:13:58 +00:00
Antonio Andelic
de264117fd
Merge pull request #38118 from bigo-sg/storagehive_struct_type 
Add struct type support in `StorageHive`
 2022-06-30 09:11:13 +02:00
mergify[bot]
4cbbfb431d
Merge branch 'master' into dictinct_in_order_optimization 2022-06-29 23:32:17 +00:00
Igor Nikonov
d435532c68 Adapt range search algorithm to high cardinality case 
+ range search done in steps of some number of rows.
  Controled by new
  setting `distinct_in_order_range_search_step`. By default 0, i.e.
  whole chunk
+ before start binary search, linear probing is done on each step (32
  rows currently)
 2022-06-29 23:30:35 +00:00
mergify[bot]
36139eacd7
Merge branch 'master' into dictinct_in_order_optimization 2022-06-29 13:37:16 +00:00
filimonov
e0acb6e337
Add check for empty proccessors in AggregatingTransform::expandPipeline 2022-06-29 15:23:53 +02:00
Igor Nikonov
3627c6ff36 Perf tests with high cardinality 2022-06-29 13:13:39 +00:00
mergify[bot]
9482c99ab8
Merge branch 'master' into sql-insert-format 2022-06-29 11:03:07 +00:00
mergify[bot]
26258959b1
Merge branch 'master' into distinct_sorted_small_refact 2022-06-29 09:38:34 +00:00
Alexey Milovidov
4cda5491f6
Merge pull request #38542 from ClickHouse/unique_lock-to-lock_guard 
Don't use std::unique_lock unless we need to
 2022-06-29 09:40:56 +03:00