Alexey Milovidov
ab91c99495
Merge branch 'master' into control_block_size_in_aggregator
2022-08-20 21:28:27 +03:00
Alexey Milovidov
74e1f4dc61
Fix clang-tidy
2022-08-20 17:09:20 +02:00
avogar
612ffaffde
Make schema inference cache better, respect format settings that can change the schema
2022-08-19 16:39:13 +00:00
Nikita Taranov
1b6e7b9ca2
Merge branch 'master' into sort_mode_rename
2022-08-19 12:31:59 +02:00
Kruglov Pavel
b67cb9e378
Merge pull request #40173 from Avogar/arrow-dict
...
Improve and fix dictionaries in Arrow format
2022-08-18 20:54:55 +02:00
Kruglov Pavel
09a2ff8843
Merge pull request #40293 from joshuataylor/feature/arrow-large-binary-string
...
Add support for LARGE_BINARY/LARGE_STRING with Arrow
2022-08-18 14:01:58 +02:00
avogar
a6318cecd5
Fix hive test
2022-08-18 11:32:42 +00:00
Nikolai Kochetov
5a85531ef7
Merge pull request #38286 from Avogar/schema-inference-cache
...
Add schema inference cache for s3/hdfs/file/url
2022-08-18 13:07:50 +02:00
Kruglov Pavel
d7056376eb
Merge pull request #40068 from Avogar/schema-inference-hints
...
Allow to specify structure hints in schema inference
2022-08-18 12:19:45 +02:00
Igor Nikonov
6fe8b61345
Merge branch 'master' into sort_mode_rename
2022-08-17 19:19:29 +02:00
Yakov Olkhovskiy
40fd6e189a
call readColumnWithStringData
2022-08-17 09:54:01 -04:00
Kruglov Pavel
19af748737
Fix typo
2022-08-17 14:29:09 +02:00
Kruglov Pavel
00d04456ff
Try reduce code duplication
2022-08-17 14:28:15 +02:00
Vladimir C
b876cc17c9
Merge pull request #39593 from quickhouse/fixexponentialdecaywindowfunctions
...
Fixed point of origin for exponential decay window functions to the last value in window
2022-08-17 14:19:59 +02:00
Igor Nikonov
5ceaeb9e12
Sorting mode renaming
...
+ sort mode -> sort scope
+ Stream -> Global
+ Port -> Stream
2022-08-17 12:19:36 +00:00
avogar
8dd54c043d
Merge branch 'master' of github.com:ClickHouse/ClickHouse into schema-inference-cache
2022-08-17 11:47:40 +00:00
Igor Nikonov
46ed4f6cdf
Merge pull request #38719 from ClickHouse/skipping_sorting_step
...
SortingStep: deduce way to sort based on input stream sort description
2022-08-17 12:58:11 +02:00
Josh Taylor
628d2bbff5
Add support for LARGE_BINARY/LARGE_STRING with Arrow
2022-08-17 10:25:06 +08:00
Nikita Taranov
6bdbaccc37
use max_block_size from settings
2022-08-16 18:56:22 +02:00
Nikita Taranov
63bc894a42
more parallelism
2022-08-16 18:56:22 +02:00
Nikita Taranov
f650b23ee3
generate many blocks
2022-08-16 18:56:22 +02:00
Nikita Taranov
db0110fd7a
more accurate crutch
2022-08-16 18:56:22 +02:00
Nikita Taranov
e5e0a24ab3
return chunks from prepareBlockAndFillWithoutKey
2022-08-16 18:56:22 +02:00
Igor Nikonov
d4367de7bb
Rename setting to optimize_sorting_by_input_stream_properties
2022-08-16 16:27:41 +00:00
Vladimir Chebotaryov
3cc03b141e
Fixed tests on Debug build type.
2022-08-16 15:43:37 +02:00
Vladimir Chebotaryov
66f9bfca61
Fixed point of origin for exponential decay window functions to the last value in window.
2022-08-16 15:43:37 +02:00
avogar
99d8727335
Fix tests
2022-08-16 12:56:51 +00:00
avogar
936c457734
Remove unnended field
2022-08-16 09:51:52 +00:00
avogar
e1ff996ec3
Allow to specify structure hints in schema inference
2022-08-16 09:46:57 +00:00
Maksim Kita
110470809b
Merge pull request #40121 from amosbird/profile-processor-1
...
Extend processors_profile_log with more info
2022-08-16 09:49:12 +02:00
Igor Nikonov
aba00952f5
Fix: don't set sort mode in ReadFromMergeTree if sort description empty
2022-08-15 20:58:20 +00:00
Kruglov Pavel
2c5c0d6d47
Fix typo
2022-08-15 19:55:28 +02:00
avogar
ca0d883c0f
Fix possible segfault in CapnProto input format
2022-08-15 15:36:18 +00:00
Igor Nikonov
ea10fd65b8
Sorting properties in EXPLAIN PLAN
...
~ change formatting for sorting
~ rename sortmode option -> sorting
2022-08-15 15:14:59 +00:00
avogar
c160033837
Fix
2022-08-15 11:38:28 +00:00
Igor Nikonov
d83bea626c
Merge remote-tracking branch 'origin/master' into skipping_sorting_step
2022-08-13 21:46:34 +00:00
Igor Nikonov
f33a0d8c85
More simple way to check if sorting order is preserved
...
- there is a case where it's done wrong
2022-08-12 23:42:37 +00:00
avogar
78e197063c
Better example
2022-08-12 19:08:36 +00:00
avogar
763f84b623
Remove bad comment
2022-08-12 19:05:57 +00:00
avogar
9addded80e
Remove logging
2022-08-12 19:01:02 +00:00
avogar
000336622a
Remove logging
2022-08-12 18:59:52 +00:00
avogar
398576e9c9
Improve and fix dictionaries in Arrow format
2022-08-12 18:56:21 +00:00
Kseniia Sumarokova
a6cfc7bc3b
Merge pull request #34651 from alexX512/master
...
New caching strategies
2022-08-12 17:23:37 +02:00
Anton Popov
4bd50bb06c
Merge branch 'master' into distinct_sorted_simplify
2022-08-12 17:11:18 +02:00
Kruglov Pavel
4c7222d938
Merge pull request #40020 from canhld94/ch_canh_fix_hash
...
fix HashMethodOneNumber with const column
2022-08-12 14:40:24 +02:00
Amos Bird
99a38e41aa
processor profile
2022-08-11 21:03:34 +08:00
Igor Nikonov
75f6fcfa70
Merge remote-tracking branch 'origin/master' into skipping_sorting_step
2022-08-11 12:35:55 +00:00
Amos Bird
fa8fab2e8f
Fix KeyCondition with other filters
2022-08-11 19:20:44 +08:00
Maksim Kita
6bec0f5854
Merge pull request #38956 from vdimir/dict-join-refactoring
...
Join with dictionary refactoring
2022-08-11 11:54:11 +02:00
Vladimir C
2d44e6c458
Merge pull request #39343 from vdimir/refactor-prepared-sets
...
Refactor PreparedSets/SubqueryForSet
2022-08-11 11:19:18 +02:00
Vladimir Chebotaryov
748979a9c0
Merge branch 'master' into betterorderbyoptimization
2022-08-11 11:09:52 +03:00
Duc Canh Le
84cd867aa8
materialize column instead of handling column in hash method
2022-08-11 10:46:06 +08:00
Anton Popov
3fdf428834
Merge pull request #39186 from Avogar/numbers-schema-inference
...
Add new features in schema inference
2022-08-11 00:53:54 +02:00
vdimir
ad91c16ba0
Rename join_common -> JoinUtils
2022-08-10 14:20:28 +00:00
vdimir
b7c5c54181
Fix build
2022-08-10 13:43:55 +00:00
vdimir
5eb4cd39e0
Merge branch 'master' into refactor-prepared-sets
2022-08-10 11:47:49 +00:00
Maksim Kita
aff8149f5c
Merge pull request #39998 from kitaisreal/actions-dag-refactoring
...
ActionsDAG rename index to outputs
2022-08-10 11:44:18 +02:00
Igor Nikonov
754a9fb096
Merge remote-tracking branch 'origin/master' into skipping_sorting_step
2022-08-09 22:20:17 +00:00
Arthur Passos
c4d8ad2222
Add docs
2022-08-09 15:58:46 -03:00
Arthur Passos
e724e7bef6
Update arrow dict to lc comment
2022-08-09 15:52:37 -03:00
Arthur Passos
6eb89fd780
Fix both arrow dict de-serialization and dict of nullable de-serialization
2022-08-09 15:06:22 -03:00
Arthur Passos
be1e32c3f1
Merge branch 'ClickHouse:master' into fix_arrow_column_dictionary_to_ch_lc
2022-08-09 15:04:06 -03:00
Maksim Kita
acbfcf440b
Merge branch 'master' into actions-dag-refactoring
2022-08-09 18:52:08 +02:00
Igor Nikonov
70b52f7cb9
Fix test, review comments
2022-08-09 16:29:56 +00:00
Maksim Kita
a576a55375
Fixed build
2022-08-09 15:03:59 +02:00
Kruglov Pavel
088e8cf9bd
Merge branch 'master' into numbers-schema-inference
2022-08-09 14:00:36 +02:00
Kruglov Pavel
99b9e85a8f
Merge pull request #39646 from Avogar/more-formats
...
Add more Pretty formats
2022-08-09 13:59:47 +02:00
Igor Nikonov
366ead3828
Consider aliases when checking if sorting order is preserved by
...
expression
2022-08-09 11:27:17 +00:00
Igor Nikonov
1439664df6
EXPLAIN tests
2022-08-08 20:46:43 +00:00
Maksim Kita
c030fd05e7
ActionsDAG rename index to outputs
2022-08-08 18:01:32 +02:00
Igor Nikonov
15bdeba074
Fix review comments
2022-08-08 14:43:03 +00:00
avogar
1304e3487c
Add comments, remove unneded stuff
2022-08-08 13:43:14 +00:00
Igor Nikonov
86c5280db3
Merge remote-tracking branch 'origin/master' into skipping_sorting_step
2022-08-08 12:43:06 +00:00
Igor Nikonov
6572a447fa
Check if sorting order is preserved for FilterStep
2022-08-08 12:42:10 +00:00
avogar
2f95726b06
Fix comments
2022-08-08 12:41:00 +00:00
vdimir
708747ca0b
Merge branch 'master' into refactor-prepared-sets
2022-08-08 14:27:18 +02:00
Vladimir C
6bd4821656
Merge pull request #39575 from vdimir/join_pushdown_column_not_found
...
Fix column not found for push down with join
2022-08-08 14:04:42 +02:00
alexX512
6bf29cb610
Change class LRUCache to class CachBase. Check running CacheBase with default pcahce policy SLRU
2022-08-07 19:59:30 +00:00
Igor Nikonov
254d14b1d3
Remove debug trace from DistinctStep
...
+ this information can be seen later via EXPLAIN PLAN
2022-08-07 13:12:11 +00:00
Igor Nikonov
f52720fedc
Try to fix style errors
2022-08-06 23:33:19 +00:00
Igor Nikonov
988ce29e9e
Check if expression step violates sorting correctly
2022-08-06 22:37:14 +00:00
Igor Nikonov
8278da6475
Fix: read row counts before move columns out of chunk
2022-08-05 21:29:57 +00:00
Igor Nikonov
9fddf6efde
Merge remote-tracking branch 'origin/master' into ordinary_distinct_small_refact
2022-08-05 19:23:44 +00:00
avogar
9b1a267203
Refactor, remove TTL, add size limit, add system table and system query
2022-08-05 16:20:15 +00:00
Igor Nikonov
f5f6ada69b
ActionsDAG check is not dependent on sort description
2022-08-05 11:00:18 +00:00
Nikolai Kochetov
cf34232675
Output header is now empty for every MV chain.
...
Instead of checking that number of processors different for different
threads, simply always return empty header from buildChainImpl(), by
adding explicit conversion.
v2: ignore UNKNOWN_TABLE errors in test
2022-08-05 13:16:32 +03:00
Igor Nikonov
246f7b71b6
Merge branch 'master' into skipping_sorting_step
2022-08-05 00:03:54 +02:00
Igor Nikonov
a70c47f780
EXPLAIN PLAN sortmode options
...
+ ActionsDAG fixes
2022-08-04 22:02:53 +00:00
Igor Nikonov
0a659f5ab8
Analyze ActionsDAG in ExpressionStep
...
+ check if sorting order can be kept from prevous step
+ EXPLAIN PLAN header=1 contains Sort Mode and Sort Description for each step
+ some tests
+ fix some review comments
2022-08-04 19:30:25 +00:00
Arthur Passos
62d48053c0
Use insertDefault instead of insert(0)
2022-08-04 15:53:44 -03:00
Arthur Passos
c307e9a228
Fix ArrowColumn dictionary to CH low cardinality conversion
2022-08-04 15:34:44 -03:00
Nikolai Kochetov
658a269d56
Merge branch 'master' into use-dag-in-key-condition
2022-08-04 16:18:40 +02:00
Vladimir C
a627b00c43
Merge branch 'master' into refactor-prepared-sets
2022-08-04 13:27:38 +02:00
Kruglov Pavel
235649cb98
Merge pull request #39458 from Avogar/fix-cancel-insert-into-function
...
Fix WriteBuffer finalize when cancel insert into function
2022-08-04 13:02:08 +02:00
vdimir
298fb2431d
Fix column not found for push down with join
2022-08-04 10:58:21 +00:00
Nikita Taranov
4943202921
Improve memory usage during memory efficient merging of aggregation results ( #39429 )
2022-08-03 17:56:59 +02:00
Vladimir Chebotaryov
484d8f7028
Merge branch 'master' into betterorderbyoptimization
2022-08-03 08:14:50 +03:00
Igor Nikonov
8eafca65c9
Fix style
2022-08-02 17:58:31 +00:00
Igor Nikonov
30782a2b05
Test: distinct sorted is not used on const column
2022-08-02 17:44:43 +00:00
Kruglov Pavel
6b2186bfeb
Merge branch 'master' into numbers-schema-inference
2022-08-02 19:34:53 +02:00
Igor Nikonov
56697125d3
Merge remote-tracking branch 'origin/master' into distinct_sorted_simplify
2022-08-02 11:18:46 +00:00
Igor Nikonov
4bf42ce867
Merge remote-tracking branch 'origin/master' into skipping_sorting_step
2022-08-02 10:50:25 +00:00
Igor Nikonov
0cb59aa38c
ExpressionStep: simple check if expression changes sorting order
2022-08-02 10:45:05 +00:00
Kruglov Pavel
42136b7630
Merge pull request #39647 from Avogar/fix-arrow-strings
...
Fix strings in dictionary in Arrow format
2022-08-01 12:46:07 +02:00
Vladimir Chebotaryov
aaa6049678
Merge branch 'master' into betterorderbyoptimization
2022-08-01 11:42:48 +03:00
Vladimir Chebotaryov
9fc3addea9
Fixed using column_after_join for handling WINDOW expressions in ExpressionAnalyzer, shall be aggregated_columns.
2022-07-31 23:36:20 +03:00
Nikolai Kochetov
22fbfe19a4
Merge branch 'master' into use-dag-in-key-condition
2022-07-31 21:54:12 +02:00
Igor Nikonov
7f0adb5eb0
Merge remote-tracking branch 'origin/master' into skipping_sorting_step
2022-07-31 07:07:36 +00:00
Igor Nikonov
83e1dd1172
Merge branch 'master' into ordinary_distinct_small_refact
2022-07-31 00:23:21 +02:00
Alexey Milovidov
4828be7fc4
Fix double escaping in the metadata of FORMAT JSON
2022-07-30 23:56:41 +02:00
Igor Nikonov
7245ddcc20
Simple refactoring: ordinary DISTINCT implementation
2022-07-30 20:25:56 +00:00
Igor Nikonov
a7cfad105e
Merge branch 'master' into distinct_sorted_simplify
2022-07-30 21:57:53 +02:00
Igor Nikonov
3be51a6dea
Construct DistinctSortedTransform only when it makes sense
...
otherwise fallback to DistinctTransform (i.e. ordinary distinct)
2022-07-30 19:41:03 +00:00
Maksim Kita
acb0137dbb
Merge pull request #39718 from kitaisreal/join-enums-refactoring
...
Join enums refactoring
2022-07-30 13:53:08 +02:00
Igor Nikonov
d951154ef4
Proved NULLs direction when compare rows
2022-07-29 22:12:03 +00:00
Igor Nikonov
13dc1697fb
Remove unnecessary initialization
2022-07-29 20:34:23 +00:00
Igor Nikonov
b44373ba8f
Merge remote-tracking branch 'origin/master' into distinct_sorted_simplify
2022-07-29 20:33:26 +00:00
Igor Nikonov
7b0b38e997
DistinctSortedTransform works only if columns contains sort prefix of
...
sort description
2022-07-29 20:01:07 +00:00
Kruglov Pavel
ccd1e1bdb8
Merge branch 'master' into fix-cancel-insert-into-function
2022-07-29 20:27:32 +02:00
Maksim Kita
8fc6bad4f4
Join enums refactoring
2022-07-29 18:35:05 +02:00
Nikolai Kochetov
59a11b32ad
Merge branch 'master' into use-dag-in-key-condition
2022-07-29 17:01:33 +02:00
Igor Nikonov
fb5799fe7f
More generic getInputOrderInfo() in query info
2022-07-29 14:36:36 +00:00
Igor Nikonov
4af435bdda
Fix: handle all const columns case correctly
2022-07-28 21:22:06 +00:00
Igor Nikonov
f414af2348
Merge pull request #39538 from ClickHouse/fix_distinct_sorted
...
Fix: DistinctSortedTransform doesn't take advantage of sorting
2022-07-28 21:47:10 +02:00
Sergei Trifonov
edf6b02723
Merge pull request #37558 from ClickHouse/concurrency-control
...
add concurrency control for pipeline executors
2022-07-28 08:56:24 +02:00
Vladimir C
115506356c
Merge branch 'master' into refactor-prepared-sets
2022-07-27 19:57:23 +02:00
Sergei Trifonov
05e4bd6c8b
review fix
2022-07-27 15:36:50 +02:00
Igor Nikonov
377c04fbf1
Merge remote-tracking branch 'origin/master' into fix_distinct_sorted
2022-07-27 13:01:18 +00:00
Anton Popov
1547c010b9
Merge pull request #39432 from ClickHouse/distinct_sorted_chunk_perf_impr
...
DISTINCT in order: perf improvement
2022-07-27 14:17:58 +02:00
avogar
01a309d4e3
Fix strings in dictionary in Arrow format
2022-07-27 12:02:27 +00:00
avogar
f925046dc4
Add more Pretty formats
2022-07-27 11:37:02 +00:00
vdimir
8eecb9ef82
upd PreparedSets: rename/change signature of methods, add comments
2022-07-27 11:22:16 +00:00
Nikolai Kochetov
873432fb53
Merge pull request #37849 from ClickHouse/bug-with-fill-date
...
Enforce equality of WITH FILL type with ORDER BY column's type for date/time types.
2022-07-27 12:27:53 +02:00
Kruglov Pavel
381ea139c2
Merge branch 'master' into schema-inference-cache
2022-07-27 11:35:36 +02:00
Igor Nikonov
64e51e56e7
Allocate memory for column arrays once
2022-07-27 08:22:07 +00:00
Igor Nikonov
12a7567402
Some polishing
2022-07-27 07:58:54 +00:00
Igor Nikonov
589104fa6e
Make building column arrays for chunk processing straightforward
2022-07-27 07:44:42 +00:00
Alexander Gololobov
a64aa00869
Merge pull request #37893 from zhangjmruc/feature/sql-standard-delete
...
Support SQL standard "delete from ... where ..." syntax and lightweight implementation on merge tree tables
2022-07-26 23:39:07 +02:00
Igor Nikonov
cac4d77d0b
Merge remote-tracking branch 'origin/master' into distinct_sorted_chunk_perf_impr
2022-07-26 20:36:38 +00:00
Yakov Olkhovskiy
0055c9307d
style fix
2022-07-26 16:08:03 -04:00
Igor Nikonov
24f3a6905f
Merge branch 'master' into fix_distinct_sorted
2022-07-26 21:57:44 +02:00
Igor Nikonov
d196ab24d4
Calculate DISTINCT columns positions which form sort prefix in sort
...
description once
2022-07-26 19:55:29 +00:00
Yakov Olkhovskiy
d93c67e303
comment and test added
2022-07-26 15:28:11 -04:00
vdimir
d9928ac93d
Add methods to SubqueryForSet, do not use refernce to SetPtr
2022-07-26 18:39:09 +00:00
vdimir
1e3fa2e01f
Refactor PreparedSets/SubqueryForSet
2022-07-26 18:39:02 +00:00
Nikolai Kochetov
d7cc3831ea
Push predicate over StorageMerge.
2022-07-26 15:01:39 +00:00
Kruglov Pavel
c362551642
Merge pull request #39357 from Avogar/fix-protobuf-capnp-empty-message
...
Fix schema inference in case of empty messages in Protobuf/CapnProto formats
2022-07-26 12:33:28 +02:00
Kruglov Pavel
53159db782
Merge branch 'master' into numbers-schema-inference
2022-07-26 12:32:49 +02:00
Sergei Trifonov
24ab5fbb86
fix finish() condition to account only active threads
2022-07-26 11:32:45 +02:00
Alexander Gololobov
0666ec2e1f
Merge branch 'master' into feature/sql-standard-delete
2022-07-26 10:42:39 +02:00
Vladimir Chebotaryov
f32d9c5539
Uppercase ROWS, GROUPS, RANGE in queries with windows. ( #39410 )
2022-07-25 22:53:53 +02:00
Alexander Gololobov
25deba2c1b
Merge branch 'master' into feature/sql-standard-delete
2022-07-25 22:13:20 +02:00
Nikolai Kochetov
b70be40804
Merge branch 'master' into use-dag-in-key-condition
2022-07-25 14:30:22 +02:00
Igor Nikonov
572dd864fe
Merge remote-tracking branch 'origin/master' into skipping_sorting_step
2022-07-25 12:19:14 +00:00
Sergei Trifonov
d9d1381c3f
Merge branch 'master' into concurrency-control
2022-07-25 14:16:46 +02:00
Kruglov Pavel
83c7da6e88
Merge branch 'master' into fix-protobuf-capnp-empty-message
2022-07-25 13:02:41 +02:00
Igor Nikonov
dc96a6270d
Remove optimize_memory_usage flag
...
Memory usage with DistinctSortedTransform fixed in #39538
2022-07-25 10:27:33 +00:00
Alexey Milovidov
388d06fda1
Merge pull request #39535 from ClickHouse/stringref
...
Less usage of StringRef
2022-07-25 04:06:11 +03:00
Igor Nikonov
41e72aac83
Fix: DistinctSortedTransform doesn't take advantage of sorting
...
clearing_columns are set incorrectly, so we never clear HashSet
2022-07-24 21:35:36 +00:00
Alexander Gololobov
460950ecdc
Merge branch 'master' into feature/sql-standard-delete
2022-07-24 21:27:22 +02:00
Robert Schulze
4333750985
Less usage of StringRef
...
... replaced by std::string_view, see #39262
2022-07-24 18:33:52 +00:00
Igor Nikonov
95511428b3
Couple optimizations
...
+ do not apply filter to chunk if there is no data for output
+ checking clear_data flag at compile time
2022-07-23 00:03:26 +00:00
Igor Nikonov
739ff34c6e
Add some tests, still not sure about optimize_memory_usage option
2022-07-22 22:48:26 +00:00
Igor Nikonov
329adc3419
Merge branch 'master' into distinct_sorted_chunk_perf_impr
2022-07-22 12:58:59 +02:00
Igor Nikonov
51f63c3a0c
Add setting optimize_distinct_in_order_memory_usage
2022-07-22 10:57:57 +00:00
Alexander Tokmakov
bed2206ae9
Merge pull request #39460 from ClickHouse/remove_some_dead_and_commented_code
...
Remove some dead and commented code
2022-07-22 13:24:34 +03:00
Igor Nikonov
7db5d54820
Adopt to the case when not all columns in distinct are part of sorting
...
description
2022-07-21 21:04:58 +00:00
avogar
794aa691bc
Merge branch 'master' of github.com:ClickHouse/ClickHouse into fix-protobuf-capnp-empty-message
2022-07-21 17:04:37 +00:00
Kruglov Pavel
9252f42b4c
Merge branch 'master' into schema-inference-cache
2022-07-21 18:59:14 +02:00
Robert Schulze
ea0a3bf600
Merge branch 'master' into stringref-to-string_view
2022-07-21 18:33:06 +02:00
avogar
fd534aa3fa
wqMerge branch 'master' of github.com:ClickHouse/ClickHouse into numbers-schema-inference
2022-07-21 15:43:17 +00:00
Igor Nikonov
122a1123b2
- disable the worst case for distinct in order in perf test for now
...
+ functional test for query with the worst perfomance
+ debug logging in DistinctStep
2022-07-21 15:03:19 +00:00
Alexander Tokmakov
a8da5d96fc
remove some dead and commented code
2022-07-21 15:05:48 +02:00
avogar
6b541aa98f
Fix WriteBuffer finalize when cancel insert into function
2022-07-21 12:18:37 +00:00
Alexander Gololobov
1ea9f143ff
Leave only _row_exists-based implementation of lightweight delete
2022-07-21 11:26:13 +02:00
Nikolai Kochetov
e15967e9db
Merge pull request #38475 from ClickHouse/additional-filters
...
Additional filters for a table (from setting)
2022-07-21 07:52:04 +02:00
Alexey Milovidov
844042fc18
Merge pull request #39433 from ClickHouse/revert-39396-try-fix-write-buffer-terminate
...
Revert "Fix WriteBuffer finalize in destructor when cacnel query"
2022-07-21 07:04:07 +03:00
Alexey Milovidov
dcda9d3bd1
Merge pull request #39365 from Avogar/fix-capnproto-abort
...
Avoid possible abort() in CapnProto on exception descruction
2022-07-21 05:20:45 +03:00
Kruglov Pavel
92995a832b
Revert "Fix WriteBuffer finalize in destructor when cacnel query"
2022-07-21 01:45:16 +02:00
Igor Nikonov
635a566bec
Comment change
2022-07-20 21:37:46 +00:00
Igor Nikonov
e50aebb5f0
Merge branch 'master' into distinct_sorted_chunk_perf_impr
2022-07-20 23:17:11 +02:00
Igor Nikonov
965f96bd84
DISTINCT in order: perf improvement
...
+ reduce allocations in DistinctSortedChunkTransform
+ use it for final distinct as well
2022-07-20 20:44:47 +00:00
Nikolai Kochetov
91043351aa
Fixing build.
2022-07-20 20:30:16 +00:00
Nikolai Kochetov
4e8cd70b1d
Merge branch 'master' into use-dag-in-key-condition
2022-07-20 17:38:33 +02:00
Yakov Olkhovskiy
c4d040e02c
Merge branch 'master' into bug-with-fill-date
2022-07-20 09:10:45 -04:00
Dmitry Novik
a09422de7b
Merge pull request #38642 from ClickHouse/group-by-use-nulls
...
Add group_by_use_nulls setting
2022-07-20 14:42:42 +02:00
Kruglov Pavel
46da17ca8c
Merge branch 'master' into numbers-schema-inference
2022-07-20 13:32:39 +02:00
Kruglov Pavel
3046cd6d29
Merge branch 'master' into schema-inference-cache
2022-07-20 13:30:42 +02:00
avogar
784ee11594
Add settings to skip fields with unsupported types in Protobuf/CapnProto schema inference
2022-07-20 11:16:25 +00:00
Kruglov Pavel
a1b63b4a02
Fix style
2022-07-20 12:07:22 +02:00
Kruglov Pavel
7722b647b7
Merge pull request #39396 from Avogar/try-fix-write-buffer-terminate
...
Fix WriteBuffer finalize in destructor when cacnel query
2022-07-20 12:06:20 +02:00
Nikolai Kochetov
f570cde815
Fixing build.
2022-07-19 20:19:57 +00:00
avogar
5c16d6b553
Fix WriteBuffer finalize in destructor when cacnel query
2022-07-19 19:21:30 +00:00
Igor Nikonov
c74600d282
Merge branch 'master' into skipping_sorting_step
2022-07-19 18:59:36 +02:00
Igor Nikonov
1fe83cc8d8
optimize_sorting_for_input_stream setting and perf tests
2022-07-19 16:58:15 +00:00
Nikolai Kochetov
eaeb30a71a
Merge branch 'master' into use-dag-in-key-condition
2022-07-19 18:39:52 +02:00
Dmitry Novik
50989bdb68
Merge branch 'master' into group-by-use-nulls
2022-07-19 14:58:01 +02:00
avogar
4f020654be
Get rid of unneded ifdefs
2022-07-19 12:12:40 +00:00
Robert Schulze
81ef1099cc
Even less usage of StringRef
...
--> see #39300
2022-07-19 07:01:06 +00:00
avogar
6eb234a1cc
Avoid abort() in capnproto on exception descruction
2022-07-18 19:53:24 +00:00
Alexander Gololobov
9de72d995a
POC lightweight delete using __row_exists virtual column and prewhere-like filtering
2022-07-18 20:06:42 +02:00
Robert Schulze
32637cb1b9
Fix build
2022-07-18 07:58:59 +00:00
Robert Schulze
13482af4ee
First try at reducing the use of StringRef
...
- to be replaced by std::string_view
- suggested in #39262
2022-07-17 17:26:02 +00:00
Robert Schulze
fffeab2e86
Merge pull request #39262 from ClickHouse/stringref-by-value
...
Pass const StringRef by value, not by reference
2022-07-17 16:53:15 +02:00
Igor Nikonov
508fd5616d
Merge remote-tracking branch 'origin/master' into skipping_sorting_step
2022-07-16 00:24:20 +00:00
Igor Nikonov
ef0ef9e03b
Merge pull request #39191 from ClickHouse/sort_transform_cleanup
...
Cleanup: done during #38719 (SortingStep: deduce way to sort based on input stream sort)
2022-07-16 01:53:58 +02:00
Igor Nikonov
e607d103cc
Just to trigger CI
2022-07-15 23:16:02 +00:00
Igor Nikonov
6f224b026a
Perf test. Code polishing
2022-07-15 21:54:57 +00:00
Igor Nikonov
d9b312f955
Self-review + revert test
2022-07-15 17:00:25 +00:00
Robert Schulze
deda29b46b
Pass const StringRef by value, not by reference
...
See #39224
2022-07-15 11:34:56 +00:00
Igor Nikonov
a3a1ccc520
Fix: SortMode::Chunk
2022-07-15 10:26:13 +00:00
jianmei zhang
9d27af7ee2
For some columns mutations, skip to apply deleted mask when read some columns. Also add unit test case
2022-07-15 12:32:41 +08:00
Igor Nikonov
8170f4e33a
Merge branch 'master' into skipping_sorting_step
2022-07-14 23:05:45 +02:00
Igor Nikonov
1efdb4e3e5
Disable finish sort with sorted chunks
2022-07-14 21:02:44 +00:00
Igor Nikonov
7cd12393c2
If sorting type is specified then use it. Otherwise rely on sort description
2022-07-14 16:26:25 +00:00
Igor Nikonov
1d49adad20
Introduce Auto mode for sorting step (replace others for now)
2022-07-14 13:29:39 +00:00
Kruglov Pavel
b38241b08a
Merge branch 'master' into schema-inference-cache
2022-07-14 12:29:54 +02:00
Ilya Yatsishin
5c8b9e4a14
Merge pull request #39170 from qoega/otel-trace-processors-setting
...
Add setting opentelemetry_trace_processors
2022-07-14 11:27:05 +02:00
Vladimir C
66f2ea5ebb
Merge pull request #39123 from vdimir/fjs-pushdown
...
Pushdown filter to the right side of sorting join
2022-07-14 09:32:19 +02:00
Igor Nikonov
b7f46d954e
Cleanup: related to #38719 (SortingStep: deduce way to sort based on input stream sort)
2022-07-13 17:57:37 +00:00
Igor Nikonov
b73aca2a3b
Merge branch 'master' into skipping_sorting_step
2022-07-13 19:06:41 +02:00
Igor Nikonov
159c9428bd
clean up
2022-07-13 17:05:54 +00:00
Igor Nikonov
f0d547993a
Fix: 01655_plan_optimizations_optimize_read_in_window_order_long
...
basically I returned code. Both plans with Finish sorting, need to check
sorting prefix
2022-07-13 16:51:51 +00:00
avogar
7cde9d3b40
Add new features in schema inference
2022-07-13 15:57:55 +00:00
Vladimir C
27ec36ca91
Merge pull request #37648 from vdimir/enable_parallel_hash
...
Bugfixes and performance improvements for parallel_hash
2022-07-13 14:52:57 +02:00
vdimir
4124dc9ac4
Rewrite tryPushDownFilter for join with lambda
2022-07-13 12:06:29 +00:00
vdimir
549a85fee9
Throw logical error on child idx mismatch in tryAddNewFilterStep
2022-07-13 11:53:46 +00:00
vdimir
fa59133463
Do not spam log in MergeJoinAlgorithm
2022-07-13 11:51:11 +00:00
Nikolai Kochetov
4f8cc871eb
Merge pull request #39132 from ClickHouse/fix-data-race-in-completed-pipeline-executor
...
Fix data race in CompletedPipelineExecutor.
2022-07-13 13:39:44 +02:00
vdimir
bddf6c1b32
Pushdown filter to the right side of sorting join
2022-07-13 11:36:25 +00:00
Yatsishin Ilya
d886aea351
Add setting opentelemetry_trace_processors
2022-07-13 11:31:24 +00:00
Vladimir C
d1d1e4d8a1
Merge pull request #38943 from amosbird/better-join-plan1
...
Avoid redundant join block transformation during planning.
2022-07-13 12:39:45 +02:00
Igor Nikonov
1d6f699a12
Use sort mode Port for reading in order
2022-07-12 21:56:00 +00:00
Dmitry Novik
5f65b45269
Merge branch 'master' into group-by-use-nulls
2022-07-12 22:36:04 +02:00
vdimir
63aebd17b2
Remove TabSeparatedSorted
2022-07-12 20:22:35 +02:00
vdimir
46df417c2e
Fix empty line sorting in TabSeparatedSorted
2022-07-12 20:22:35 +02:00
vdimir
f51b25b262
clickhouse test ignore order via special format
2022-07-12 20:22:35 +02:00
Nikolai Kochetov
ec24f730b1
Merge pull request #39125 from ClickHouse/fix-insert-into-mv-with-extremes-enabled
...
Fix insert into MV with enabled extremes.
2022-07-12 20:19:41 +02:00
Nikolai Kochetov
93bf22f585
Fix data race in CompletedPipelineExecutor.
2022-07-12 17:59:31 +00:00
Amos Bird
982e1a73d3
Better
2022-07-12 22:21:46 +08:00
Amos Bird
d3709c6c26
Avoid redundant join block transformation.
2022-07-12 22:20:10 +08:00
Amos Bird
b9d9ca5194
style fix
2022-07-12 22:20:08 +08:00
Nikolai Kochetov
75c3926cbb
Fix insert into MV with enabled extremes.
2022-07-12 13:57:36 +00:00
Dmitry Novik
aabf5123d6
Fixup
2022-07-12 13:46:06 +00:00
Igor Nikonov
a1f36d5536
Merge branch 'master' into skipping_sorting_step
2022-07-12 15:15:14 +02:00
Igor Nikonov
2c8d9080bd
Fix: consider collation in column sort description comparison
2022-07-12 13:14:10 +00:00
Dmitry Novik
cfca3db884
Fix crash with totals
2022-07-12 12:15:43 +00:00
mergify[bot]
48688da835
Merge branch 'master' into concurrency-control
2022-07-12 10:57:13 +00:00
Igor Nikonov
ea5e7793b2
Fix: self-review comments
2022-07-11 21:26:39 +00:00
Igor Nikonov
e0776b1c82
Fix: test for optimize read in window order
...
+ code polishing
2022-07-11 20:59:38 +00:00
Igor Nikonov
0ca8166ab2
Fix: forgot to return sorting type in constructors
2022-07-11 20:59:38 +00:00
Igor Nikonov
47bed7e318
Try to choose sorting transform based on sort description with fallback
2022-07-11 20:59:38 +00:00
Igor Nikonov
53272b2f08
Code polishing
2022-07-11 20:59:38 +00:00
Igor Nikonov
2a7e3bd741
Fix + SortMode::None as default value
2022-07-11 20:59:38 +00:00
Igor Nikonov
16d2319a8d
SortingStep: type of sorting is deduced based on input stream sorting description in during transformation
...
+ perf test
2022-07-11 20:59:38 +00:00
Igor Nikonov
7d4d92bd61
In case full sort was wrong choise during plan interpretation
2022-07-11 20:59:38 +00:00
Igor Nikonov
67ce421e38
Skip sorting step if input stream is globally sorted
2022-07-11 20:59:38 +00:00
Robert Schulze
1a7727a254
Prefix overridden add_executable() command with "clickhouse_"
...
A simple HelloWorld program with zero includes except iostream triggers
a build of ca. 2000 source files. The reason is that ClickHouse's
top-level CMakeLists.txt overrides "add_executable()" to link all
binaries against "clickhouse_new_delete". This links against
"clickhouse_common_io", which in turn has lots of 3rd party library
dependencies ... Without linking "clickhouse_new_delete", the number of
compiled files for "HelloWorld" goes down to ca. 70.
As an example, the self-extracting-executable needs none of its current
dependencies but other programs may also benefit.
In order to restore access to the original "add_executable()", the
overriding version is now prefixed. There is precedence for a
"clickhouse_" prefix (as opposed to "ch_"), for example
"clickhouse_split_debug_symbols". In general prefixing makes sense also
because overriding CMake commands relies on undocumented behavior and is
considered not-so-great practice (*).
(*) https://crascit.com/2018/09/14/do-not-redefine-cmake-commands/
2022-07-11 19:36:18 +02:00
vdimir
da523f3288
Fix assertion in full soring merge join
2022-07-08 11:31:15 +00:00
Dmitry Novik
d1df66687b
Merge branch 'master' into group-by-use-nulls
2022-07-07 20:54:38 +02:00
Dmitry Novik
1587385f7a
Cleanup code
2022-07-07 18:53:20 +00:00
Robert Schulze
f15d9ca59c
Merge pull request #38774 from zvonand/zvonand-nnd
...
Reintroduce nonNegativeDerivative()
2022-07-07 20:39:13 +02:00
vdimir
7c586a9e7c
Minor updates for full soring merge join
2022-07-06 14:28:05 +00:00
vdimir
881d352e05
upd full sorting join
2022-07-06 14:28:05 +00:00
vdimir
aff6654d52
minor changes in full sort join
2022-07-06 14:27:33 +00:00
vdimir
f8e66601a7
Fix column remap in MergeJoinTransform
2022-07-06 14:27:32 +00:00
vdimir
0b994bb258
fix build
2022-07-06 14:27:32 +00:00
vdimir
753a567da8
full sorting join with using
2022-07-06 14:27:29 +00:00
vdimir
a90ac59ee5
MergeJoinAlgorithm::createBlockWithDefaults
2022-07-06 14:26:19 +00:00
vdimir
d184e184b4
full sort join: check key types, more tests
2022-07-06 14:26:19 +00:00
vdimir
a2a7abc2e9
add not implemented checks, add using testcase to full sort join
2022-07-06 14:26:18 +00:00
vdimir
92ff43eb7c
tests full sort join
2022-07-06 14:26:18 +00:00
vdimir
a0144e115d
full sorting all join
2022-07-06 14:26:18 +00:00
vdimir
4e88e8f5ec
full sort join: move block list to all join state
2022-07-06 14:26:17 +00:00
vdimir
94192a23fc
enable total compare in MergeJoinAlgorithm
2022-07-06 14:26:16 +00:00
vdimir
a92c60ba06
fix nulls comparsion in full sorting join
2022-07-06 14:26:15 +00:00
vdimir
7c5a5f4b64
full sorted any join tests passed
2022-07-06 14:26:15 +00:00
vdimir
26d812ec72
wip any full sorting merge, rewrite cursor
2022-07-06 14:26:14 +00:00
vdimir
a2d190edb8
wip MergeJoinTransform
2022-07-06 14:25:12 +00:00
vdimir
0b9d4ee640
wip sort join same rows
2022-07-06 14:25:12 +00:00
vdimir
6d198ff3d7
fix style
2022-07-06 14:25:11 +00:00
vdimir
88d8dc5be2
wip full sort any join
2022-07-06 14:25:11 +00:00
vdimir
ba787db0bb
Fix build, small changes
2022-07-06 14:25:10 +00:00
vdimir
d34a66c915
wip sorting merge
2022-07-06 14:25:09 +00:00
vdimir
1b429fc1af
wip: any left/right sorting join
2022-07-06 14:23:46 +00:00
vdimir
8dce97123c
wip: any inner full sorting join
2022-07-06 14:23:46 +00:00
vdimir
4a16195964
Calculate output header for full sorting merge join
2022-07-06 14:23:45 +00:00
vdimir
fa8eb35599
Pipeline for full sorting merge join
2022-07-06 14:23:44 +00:00
Maksim Kita
b94489d52c
Merge pull request #38859 from kitaisreal/merge-tree-merge-disable-batch-optimization
...
MergeTree merge disable batch optimization
2022-07-06 15:59:40 +02:00
Nikolai Kochetov
7de2f229ab
Merge pull request #38584 from ClickHouse/filimonov-AggregatingTransform-expandPipeline
...
Add check for empty proccessors in AggregatingTransform::expandPipeline
2022-07-06 14:38:40 +02:00
Andrey Zvonov
7de39d9b15
Merge branch 'master' of github.com:ClickHouse/ClickHouse into zvonand-nnd
2022-07-06 10:59:35 +03:00
Maksim Kita
bdc21737d5
MergeTree merge disable batch optimization
2022-07-05 16:15:00 +02:00
zvonand
8a270c01e9
fix floating point in intervals
2022-07-04 20:45:05 +03:00
Dmitry Novik
864ab20582
Use correct intermediate header for ROLLUP and CUBE
2022-07-04 16:17:58 +00:00
Kruglov Pavel
4080f055b6
Merge pull request #38477 from Avogar/sql-insert-format
...
Add SQLInsert output format
2022-07-04 15:06:33 +02:00
zvonand
f814985adf
minor improvements
2022-07-04 16:03:59 +03:00
zvonand
eac84351f6
fix behavior
2022-07-04 01:26:07 +03:00
Igor Nikonov
2e2ef08712
Merge pull request #37803 from ClickHouse/dictinct_in_order_optimization
...
DISTINCT in order optimization
2022-07-03 21:59:04 +02:00
Igor Nikonov
2577b59f4c
Merge pull request #38721 from ClickHouse/sort_description_4_read_from_merge_tree
...
Provide sort description for output stream in ReadFromMergeTree step
2022-07-03 18:06:25 +02:00
zvonand
8e99ea84a8
fix LOGICAL_ERROR
2022-07-02 14:09:51 +03:00
