Commit Graph

3349 Commits

Author SHA1 Message Date
Alexander Gololobov
6a69e08799
Merge pull request #40559 from ClickHouse/lwd_vertical_merge_fix
Fix vertical merge of parts with lightweight deleted rows
2022-08-25 20:47:44 +02:00
Frank Chen
bb00dcc19b Remove using namespace from header
Signed-off-by: Frank Chen <frank.chen021@outlook.com>
2022-08-25 20:20:13 +08:00
Frank Chen
99c37ce6c6
Merge branch 'master' into tracing_context_propagation 2022-08-25 10:07:16 +08:00
Nikita Taranov
ac34a17551
Merge branch 'master' into control_block_size_in_aggregator 2022-08-24 20:25:28 +02:00
vdimir
91788f29e8
Upd TemporaryFileOnDisk 2022-08-24 16:15:54 +00:00
vdimir
7194df1184
Move back TemporaryFile -> TemporaryFileOnDisk 2022-08-24 16:14:11 +00:00
vdimir
0349c85017
Use getCompressedBytes in BufferingToFileTransform and TemporaryFileStream 2022-08-24 16:14:10 +00:00
vdimir
51c44424cc
More metrics for temp files 2022-08-24 16:14:09 +00:00
vdimir
1321ac87b5
Minor fixes 2022-08-24 16:14:07 +00:00
vdimir
7e0c9062c7
Add ProfileEvents::ExternalSort(Un)CompressedBytes 2022-08-24 16:14:07 +00:00
Kruglov Pavel
e6e7f5db93
Merge pull request #40491 from mini4/fix-settings-input_format_tsv_skip_first_lines
Fix bug in settings input_format_tsv_skip_first_lines of format TSV
2022-08-24 15:57:45 +02:00
Alexander Gololobov
1c2dd50ca5 Fix vertical merge of parts with lightweight deleted rows 2022-08-24 15:18:33 +02:00
Kruglov Pavel
0781e8b4f7
Merge pull request #40534 from Avogar/nested-in-avro
Support reading Array(Record) into flatten nested table in Avro
2022-08-24 13:33:12 +02:00
Frank Chen
cd19366b44 Move classes into DB::OpenTelemetry namespace 2022-08-24 16:41:40 +08:00
kgurjev
f62c2c3221 Fix bug in settings input_format_tsv_skip_first_lines of format TSV 2022-08-24 10:02:57 +03:00
Kruglov Pavel
72f02bd6eb
Merge pull request #40414 from Avogar/improve-schema-inference-cache
Improve schema inference cache, respect format settings that can change the schema
2022-08-23 17:04:58 +02:00
avogar
29a887578b Fix 2022-08-23 11:42:57 +00:00
avogar
581e569d04 Support reading Array(Record) into flatten nested table in Avro 2022-08-23 11:05:02 +00:00
Arthur Passos
f8e2ab0a20 Use FileReader::GetRecordBatchReader instead of FileReader::ReadRowGroup to parse Parquet 2022-08-22 08:21:32 -03:00
Alexey Milovidov
ab91c99495
Merge branch 'master' into control_block_size_in_aggregator 2022-08-20 21:28:27 +03:00
Alexey Milovidov
74e1f4dc61 Fix clang-tidy 2022-08-20 17:09:20 +02:00
avogar
612ffaffde Make schema inference cache better, respect format settings that can change the schema 2022-08-19 16:39:13 +00:00
Nikita Taranov
1b6e7b9ca2
Merge branch 'master' into sort_mode_rename 2022-08-19 12:31:59 +02:00
Kruglov Pavel
b67cb9e378
Merge pull request #40173 from Avogar/arrow-dict
Improve and fix dictionaries in Arrow format
2022-08-18 20:54:55 +02:00
Kruglov Pavel
09a2ff8843
Merge pull request #40293 from joshuataylor/feature/arrow-large-binary-string
Add support for LARGE_BINARY/LARGE_STRING with Arrow
2022-08-18 14:01:58 +02:00
avogar
a6318cecd5 Fix hive test 2022-08-18 11:32:42 +00:00
Nikolai Kochetov
5a85531ef7
Merge pull request #38286 from Avogar/schema-inference-cache
Add schema inference cache for s3/hdfs/file/url
2022-08-18 13:07:50 +02:00
Kruglov Pavel
d7056376eb
Merge pull request #40068 from Avogar/schema-inference-hints
Allow to specify structure hints in schema inference
2022-08-18 12:19:45 +02:00
Igor Nikonov
6fe8b61345
Merge branch 'master' into sort_mode_rename 2022-08-17 19:19:29 +02:00
Yakov Olkhovskiy
40fd6e189a
call readColumnWithStringData 2022-08-17 09:54:01 -04:00
Kruglov Pavel
19af748737
Fix typo 2022-08-17 14:29:09 +02:00
Kruglov Pavel
00d04456ff
Try reduce code duplication 2022-08-17 14:28:15 +02:00
Vladimir C
b876cc17c9
Merge pull request #39593 from quickhouse/fixexponentialdecaywindowfunctions
Fixed point of origin for exponential decay window functions to the last value in window
2022-08-17 14:19:59 +02:00
Igor Nikonov
5ceaeb9e12 Sorting mode renaming
+ sort mode -> sort scope
+ Stream -> Global
+ Port -> Stream
2022-08-17 12:19:36 +00:00
avogar
8dd54c043d Merge branch 'master' of github.com:ClickHouse/ClickHouse into schema-inference-cache 2022-08-17 11:47:40 +00:00
Igor Nikonov
46ed4f6cdf
Merge pull request #38719 from ClickHouse/skipping_sorting_step
SortingStep: deduce way to sort based on input stream sort description
2022-08-17 12:58:11 +02:00
Josh Taylor
628d2bbff5 Add support for LARGE_BINARY/LARGE_STRING with Arrow 2022-08-17 10:25:06 +08:00
Nikita Taranov
6bdbaccc37 use max_block_size from settings 2022-08-16 18:56:22 +02:00
Nikita Taranov
63bc894a42 more parallelism 2022-08-16 18:56:22 +02:00
Nikita Taranov
f650b23ee3 generate many blocks 2022-08-16 18:56:22 +02:00
Nikita Taranov
db0110fd7a more accurate crutch 2022-08-16 18:56:22 +02:00
Nikita Taranov
e5e0a24ab3 return chunks from prepareBlockAndFillWithoutKey 2022-08-16 18:56:22 +02:00
Igor Nikonov
d4367de7bb Rename setting to optimize_sorting_by_input_stream_properties 2022-08-16 16:27:41 +00:00
Vladimir Chebotaryov
3cc03b141e Fixed tests on Debug build type. 2022-08-16 15:43:37 +02:00
Vladimir Chebotaryov
66f9bfca61 Fixed point of origin for exponential decay window functions to the last value in window. 2022-08-16 15:43:37 +02:00
avogar
99d8727335 Fix tests 2022-08-16 12:56:51 +00:00
avogar
936c457734 Remove unnended field 2022-08-16 09:51:52 +00:00
avogar
e1ff996ec3 Allow to specify structure hints in schema inference 2022-08-16 09:46:57 +00:00
Maksim Kita
110470809b
Merge pull request #40121 from amosbird/profile-processor-1
Extend processors_profile_log with more info
2022-08-16 09:49:12 +02:00
Igor Nikonov
aba00952f5 Fix: don't set sort mode in ReadFromMergeTree if sort description empty 2022-08-15 20:58:20 +00:00
Kruglov Pavel
2c5c0d6d47
Fix typo 2022-08-15 19:55:28 +02:00
avogar
ca0d883c0f Fix possible segfault in CapnProto input format 2022-08-15 15:36:18 +00:00
Igor Nikonov
ea10fd65b8 Sorting properties in EXPLAIN PLAN
~ change formatting for sorting
~ rename sortmode option -> sorting
2022-08-15 15:14:59 +00:00
avogar
c160033837 Fix 2022-08-15 11:38:28 +00:00
Igor Nikonov
d83bea626c Merge remote-tracking branch 'origin/master' into skipping_sorting_step 2022-08-13 21:46:34 +00:00
Igor Nikonov
f33a0d8c85 More simple way to check if sorting order is preserved
- there is a case where it's done wrong
2022-08-12 23:42:37 +00:00
avogar
78e197063c Better example 2022-08-12 19:08:36 +00:00
avogar
763f84b623 Remove bad comment 2022-08-12 19:05:57 +00:00
avogar
9addded80e Remove logging 2022-08-12 19:01:02 +00:00
avogar
000336622a Remove logging 2022-08-12 18:59:52 +00:00
avogar
398576e9c9 Improve and fix dictionaries in Arrow format 2022-08-12 18:56:21 +00:00
Kseniia Sumarokova
a6cfc7bc3b
Merge pull request #34651 from alexX512/master
New caching strategies
2022-08-12 17:23:37 +02:00
Anton Popov
4bd50bb06c
Merge branch 'master' into distinct_sorted_simplify 2022-08-12 17:11:18 +02:00
Kruglov Pavel
4c7222d938
Merge pull request #40020 from canhld94/ch_canh_fix_hash
fix HashMethodOneNumber with const column
2022-08-12 14:40:24 +02:00
Amos Bird
99a38e41aa
processor profile 2022-08-11 21:03:34 +08:00
Igor Nikonov
75f6fcfa70 Merge remote-tracking branch 'origin/master' into skipping_sorting_step 2022-08-11 12:35:55 +00:00
Amos Bird
fa8fab2e8f
Fix KeyCondition with other filters 2022-08-11 19:20:44 +08:00
Maksim Kita
6bec0f5854
Merge pull request #38956 from vdimir/dict-join-refactoring
Join with dictionary refactoring
2022-08-11 11:54:11 +02:00
Vladimir C
2d44e6c458
Merge pull request #39343 from vdimir/refactor-prepared-sets
Refactor PreparedSets/SubqueryForSet
2022-08-11 11:19:18 +02:00
Vladimir Chebotaryov
748979a9c0
Merge branch 'master' into betterorderbyoptimization 2022-08-11 11:09:52 +03:00
Duc Canh Le
84cd867aa8 materialize column instead of handling column in hash method 2022-08-11 10:46:06 +08:00
Anton Popov
3fdf428834
Merge pull request #39186 from Avogar/numbers-schema-inference
Add new features in schema inference
2022-08-11 00:53:54 +02:00
vdimir
ad91c16ba0
Rename join_common -> JoinUtils 2022-08-10 14:20:28 +00:00
vdimir
b7c5c54181
Fix build 2022-08-10 13:43:55 +00:00
vdimir
5eb4cd39e0
Merge branch 'master' into refactor-prepared-sets 2022-08-10 11:47:49 +00:00
Maksim Kita
aff8149f5c
Merge pull request #39998 from kitaisreal/actions-dag-refactoring
ActionsDAG rename index to outputs
2022-08-10 11:44:18 +02:00
Igor Nikonov
754a9fb096 Merge remote-tracking branch 'origin/master' into skipping_sorting_step 2022-08-09 22:20:17 +00:00
Arthur Passos
c4d8ad2222 Add docs 2022-08-09 15:58:46 -03:00
Arthur Passos
e724e7bef6 Update arrow dict to lc comment 2022-08-09 15:52:37 -03:00
Arthur Passos
6eb89fd780 Fix both arrow dict de-serialization and dict of nullable de-serialization 2022-08-09 15:06:22 -03:00
Arthur Passos
be1e32c3f1
Merge branch 'ClickHouse:master' into fix_arrow_column_dictionary_to_ch_lc 2022-08-09 15:04:06 -03:00
Maksim Kita
acbfcf440b
Merge branch 'master' into actions-dag-refactoring 2022-08-09 18:52:08 +02:00
Igor Nikonov
70b52f7cb9 Fix test, review comments 2022-08-09 16:29:56 +00:00
Maksim Kita
a576a55375 Fixed build 2022-08-09 15:03:59 +02:00
Kruglov Pavel
088e8cf9bd
Merge branch 'master' into numbers-schema-inference 2022-08-09 14:00:36 +02:00
Kruglov Pavel
99b9e85a8f
Merge pull request #39646 from Avogar/more-formats
Add more Pretty formats
2022-08-09 13:59:47 +02:00
Igor Nikonov
366ead3828 Consider aliases when checking if sorting order is preserved by
expression
2022-08-09 11:27:17 +00:00
Igor Nikonov
1439664df6 EXPLAIN tests 2022-08-08 20:46:43 +00:00
Maksim Kita
c030fd05e7 ActionsDAG rename index to outputs 2022-08-08 18:01:32 +02:00
Igor Nikonov
15bdeba074 Fix review comments 2022-08-08 14:43:03 +00:00
avogar
1304e3487c Add comments, remove unneded stuff 2022-08-08 13:43:14 +00:00
Igor Nikonov
86c5280db3 Merge remote-tracking branch 'origin/master' into skipping_sorting_step 2022-08-08 12:43:06 +00:00
Igor Nikonov
6572a447fa Check if sorting order is preserved for FilterStep 2022-08-08 12:42:10 +00:00
avogar
2f95726b06 Fix comments 2022-08-08 12:41:00 +00:00
vdimir
708747ca0b
Merge branch 'master' into refactor-prepared-sets 2022-08-08 14:27:18 +02:00
Vladimir C
6bd4821656
Merge pull request #39575 from vdimir/join_pushdown_column_not_found
Fix column not found for push down with join
2022-08-08 14:04:42 +02:00
alexX512
6bf29cb610 Change class LRUCache to class CachBase. Check running CacheBase with default pcahce policy SLRU 2022-08-07 19:59:30 +00:00
Igor Nikonov
254d14b1d3 Remove debug trace from DistinctStep
+ this information can be seen later via EXPLAIN PLAN
2022-08-07 13:12:11 +00:00
Igor Nikonov
f52720fedc Try to fix style errors 2022-08-06 23:33:19 +00:00
Igor Nikonov
988ce29e9e Check if expression step violates sorting correctly 2022-08-06 22:37:14 +00:00