Commit Graph

4597 Commits

Author SHA1 Message Date
avogar
5312763425 Clean 2023-03-20 19:07:06 +00:00
avogar
ba389c2a3d Fix 2023-03-20 19:06:02 +00:00
Sergei Trifonov
273770519b
Merge branch 'master' into master 2023-03-20 18:28:10 +01:00
Kruglov Pavel
1d12e85d55
Merge branch 'master' into non-blocking-connect 2023-03-20 13:23:55 +01:00
Alexey Milovidov
ddf29b69f9
Merge branch 'master' into fuzzer-of-data-formats 2023-03-20 03:26:51 +03:00
Alexey Milovidov
e788bf0712
Merge pull request #47721 from ClickHouse/revert-47622-analyzer-planner-fixes-before-enable-by-default
Revert "Analyzer planner fixes before enable by default"
2023-03-20 03:24:03 +03:00
Konstantin Morozov
dec5972194 add optimization for MemoryStorageStep 2023-03-19 20:35:24 +01:00
Robert Schulze
0e01e912fe
Preparation for libcxx(abi), llvm, clang-tidy 16 2023-03-19 17:25:41 +00:00
Alexey Milovidov
ef5e8462da
Revert "Analyzer planner fixes before enable by default" 2023-03-19 19:25:37 +03:00
Alexey Milovidov
2a077f11f6 Merge branch 'master' into fuzzer-of-data-formats 2023-03-19 01:07:31 +01:00
Alexey Milovidov
5f5e6d28c7
Merge pull request #47690 from Avogar/bson-fix-3
Fix bug in tuple as array serialization in BSONEachRow format
2023-03-18 22:18:40 +03:00
Alexey Milovidov
93a3962f45
Merge pull request #47609 from Avogar/fix-bson-2
Check for duplicate column in BSONEachRow format
2023-03-18 22:17:27 +03:00
Alexey Milovidov
26c17b61fb
Merge pull request #47434 from Avogar/avro-improvements
Support Decimals and Date32 in Avro format
2023-03-18 22:16:34 +03:00
Maksim Kita
5187138510
Merge pull request #47622 from kitaisreal/analyzer-planner-fixes-before-enable-by-default
Analyzer planner fixes before enable by default
2023-03-18 13:11:40 +03:00
Sema Checherinda
0b641fcead
Merge pull request #47564 from CheSema/remove-redundant
remove counters for threads, fix negative counters
2023-03-18 02:23:52 +01:00
Han Fei
722e6bb2e8
Merge pull request #47680 from ClickHouse/rs/uncruftify-build
CMake: Remove further cruft from build
2023-03-17 22:58:46 +01:00
Alexey Milovidov
c11915d35b Merge branch 'fix-bson-2' of github.com:Avogar/ClickHouse into fuzzer-of-data-formats 2023-03-17 22:05:46 +01:00
alexX512
023c0ba46e Fix comments 2023-03-17 20:05:10 +00:00
Nikolai Kochetov
6509d9e525 Refactir a bit more. 2023-03-17 18:36:49 +00:00
Kruglov Pavel
d72bf0af71
Fix style 2023-03-17 19:35:20 +01:00
Kruglov Pavel
a5a7df85c7
Merge branch 'master' into bson-fix-3 2023-03-17 19:30:28 +01:00
avogar
facb978830 Fix bug in tuple as array serialization in BSONEachRow format 2023-03-17 17:59:29 +00:00
Kruglov Pavel
c376712007
Fix 2023-03-17 16:56:24 +01:00
Kruglov Pavel
5d0d7695aa
Fix 2023-03-17 16:55:13 +01:00
Robert Schulze
f72a337074
Remove cruft from build
No need to check compiler flags, clang >= 15 supports all of them.
2023-03-17 13:44:04 +00:00
Maksim Kita
2d2696bc20 Fixed code review issues 2023-03-17 14:34:23 +01:00
Kruglov Pavel
4522b74fc6 Fix typo 2023-03-17 13:05:41 +00:00
avogar
f8dab850a9 Check for duplicate column in BSONEachRow format 2023-03-17 13:05:41 +00:00
avogar
4b46b5b748 Fix, make better 2023-03-17 13:02:20 +00:00
Sema Checherinda
49feba9272
Merge pull request #47632 from ClickHouse/filling-transform-untangle
FillingTransform: remove unnecessary indirection when accessing columns
2023-03-17 12:40:53 +01:00
Alexander Tokmakov
7719acad72
Revert "Fix BSONEachRow parallel parsing when document size is invalid" 2023-03-17 14:00:19 +03:00
Alexey Milovidov
651aba0d1b Slightly better 2023-03-17 00:19:49 +01:00
Alexey Milovidov
31b46df85c Fix an error 2023-03-17 00:13:52 +01:00
Igor Nikonov
dbbf20909a Merge remote-tracking branch 'origin/master' into query-plan-update-sort-description 2023-03-16 20:50:57 +00:00
Igor Nikonov
4551233339 Remove leftovers 2023-03-16 20:50:21 +00:00
Alexey Perevyshin
2c6534f5c8
Merge branch 'master' into master 2023-03-16 23:57:36 +04:00
Igor Nikonov
34e9b07854 - remove debug logging
+ init columns method
2023-03-16 19:33:53 +00:00
Sema Checherinda
3c6deddd1d work with comments on PR 2023-03-16 19:55:58 +01:00
Alexey Milovidov
704cb54afe Merge branch 'master' of github.com:ClickHouse/ClickHouse into fuzzer-of-data-formats 2023-03-16 19:50:33 +01:00
Igor Nikonov
f7da3821b8 Make lambdas ordinary functions 2023-03-16 18:46:42 +00:00
Igor Nikonov
288cc0843a some polishing 2023-03-16 18:09:08 +00:00
Igor Nikonov
4904a9fb7a Create result columns as MutableColumns at the begining
Use IColumn* to manipulate columns on specific positions in result columns
2023-03-16 17:44:29 +00:00
Igor Nikonov
1d735b3773 Fix build 2023-03-16 15:47:27 +00:00
Kruglov Pavel
ffa8af2861
Merge pull request #47540 from Avogar/fix-bson-bug
Fix BSONEachRow parallel parsing when document size is invalid
2023-03-16 16:25:25 +01:00
Igor Nikonov
f5c8250f58 Merge remote-tracking branch 'origin/master' into query-plan-update-sort-description 2023-03-16 14:53:52 +00:00
alexX512
29dd2e56f9 Fix pulling pipeline 2023-03-16 12:41:17 +00:00
Igor Nikonov
3a3cb703d4
Merge pull request #47399 from ClickHouse/improve-remove-redundant-distinct
Remove distinct on top of MergingAggregatedStep
2023-03-16 10:36:32 +01:00
Maksim Kita
558aa6c164
Merge pull request #47583 from kitaisreal/merge-tree-explain-prewhere-and-row-policy
ReadFromMergeTree explain prewhere and row policy actions
2023-03-16 12:31:34 +03:00
Alexey Milovidov
92a5b81222 Merge branch 'master' of github.com:ClickHouse/ClickHouse into fuzzer-of-data-formats 2023-03-15 23:59:16 +01:00
Igor Nikonov
b369366b19 Some refactoring
+ result_columns
+ move interpolate to method from lambda
2023-03-15 22:13:16 +00:00
Sema Checherinda
67ac858a52 remove counters for threads, fix negative counters 2023-03-15 22:22:22 +01:00
Igor Nikonov
7d8e5533f3
Merge pull request #47385 from ClickHouse/fix-distinct-in-order-analyzer
Fix: apply reading in order for distinct
2023-03-15 21:48:33 +01:00
Maksim Kita
87f8138c55 Analyzer support row policy 2023-03-15 15:21:49 +01:00
Maksim Kita
6c2084052c Fixed tests 2023-03-15 15:21:49 +01:00
Maksim Kita
c102adb728 Analyzer added PREWHERE support 2023-03-15 15:21:49 +01:00
alexX512
0088c18192 Fix fasttest error 2023-03-15 13:37:47 +00:00
avogar
c4ac4c596f Fix comments 2023-03-15 13:17:13 +00:00
alexX512
cbbdb5aabc Fix typos 2023-03-15 13:05:38 +00:00
Igor Nikonov
d69fb0c396 Merge remote-tracking branch 'origin/master' into improve-remove-redundant-distinct 2023-03-15 12:45:20 +00:00
avogar
17f5356178 Improve async reading from socket 2023-03-15 12:18:08 +00:00
Alexey Milovidov
bb6b775884 Merge branch 'master' into fuzzer-of-data-formats 2023-03-15 12:42:00 +01:00
Alexey Milovidov
0631381161 Merge branch 'fix-bson-2' of github.com:Avogar/ClickHouse into fuzzer-of-data-formats 2023-03-15 12:41:54 +01:00
avogar
26e15b288b Check for duplicate column in BSONEachRow format 2023-03-15 11:37:25 +00:00
alexX512
d47e10484f Merge branch 'master' of github.com:ClickHouse/ClickHouse 2023-03-15 11:14:34 +00:00
Robert Schulze
5cee479f0c
Merge pull request #47424 from ClickHouse/rs/compile-with-cpp23
Compile with C++23
2023-03-15 10:55:42 +01:00
alexX512
14a7c5a019 Add stop_reading_on_first_cancel setting 2023-03-15 06:06:55 +00:00
Alexey Milovidov
e443c4e682
Merge pull request #47538 from Avogar/proper-parquet-fix
Proper fix for bug in parquet, revert reverted #45878
2023-03-14 22:29:39 +03:00
Maksim Kita
26ba62bb9e ReadFromMergeTree explain prewhere and row policy actions 2023-03-14 18:02:56 +01:00
Igor Nikonov
0ecce85c48 Crutch for 02404_memory_bound_merging 2023-03-14 16:37:14 +00:00
Igor Nikonov
e0e06ceeef Fix: 02343_group_by_use_nulls 2023-03-14 12:58:12 +00:00
Igor Nikonov
e82f7b9622 Polishing 2023-03-13 21:47:21 +00:00
Igor Nikonov
4239bfb586 FillingNoopTransform for totals stream 2023-03-13 21:47:21 +00:00
Igor Nikonov
85b0c5a1a4 Remove TOTALS handling in FillTransform 2023-03-13 21:47:21 +00:00
Nikolai Kochetov
669a92bae0 Merge branch 'master' into aggregate-projections-analysis-query-plan 2023-03-13 19:55:49 +01:00
avogar
637d8d0f09 Fix BSONEachRow parallel parsing when document size is invalid 2023-03-13 18:52:29 +00:00
Alexey Milovidov
a411ae10b5
Merge pull request #47486 from ClickHouse/revert_25674
Remove "asynchronous connection drain"
2023-03-13 21:40:39 +03:00
avogar
278c76f85a Fix 2023-03-13 18:23:51 +00:00
avogar
4213ec609f Proper fix for bug in parquet, revert reverted #45878 2023-03-13 18:22:09 +00:00
Igor Nikonov
14498c1d2b Merge remote-tracking branch 'origin/master' into query-plan-update-sort-description 2023-03-13 16:33:42 +00:00
Alexey Milovidov
278b8f74c2
Merge pull request #47492 from ClickHouse/revert-45878
Revert #45878 and add a test
2023-03-12 23:43:06 +03:00
Alexey Milovidov
f445649845
Merge branch 'master' into rs/compile-with-cpp23 2023-03-12 21:33:20 +03:00
avogar
5a18acde90 Revert #45878 and add a test 2023-03-11 21:15:14 +00:00
flynn
e56eeac05f fix 2023-03-11 18:48:36 +00:00
Alexander Tokmakov
7b1b238d0b Revert "Merge pull request #25674 from amosbird/distributedreturnconnection"
This reverts commit 5ffd99dfd4, reversing
changes made to 2796aa333f.
2023-03-11 19:09:47 +01:00
flynn
2682844951 fix 2023-03-11 18:06:50 +00:00
flynn
3190b89f9a Fix 2023-03-11 18:04:19 +00:00
flynn
0084a61c76 Merge branch 'master' of github.com:ClickHouse/ClickHouse into explain-graph 2023-03-11 15:35:47 +00:00
flynn
3e554c2925 fix 2023-03-11 15:24:19 +00:00
flynn
3687698b15 Fix explain graph with projection 2023-03-11 15:03:10 +00:00
Igor Nikonov
73d3cc6f0c some polishing 2023-03-11 12:40:44 +00:00
Igor Nikonov
89580429f2 Merge remote-tracking branch 'origin/master' into fix-distinct-in-order-analyzer 2023-03-11 12:04:16 +00:00
Igor Nikonov
15134cb8c5 Remove debug log 2023-03-11 12:03:34 +00:00
Igor Nikonov
c45c04a1bb Merge remote-tracking branch 'origin/master' into improve-remove-redundant-distinct 2023-03-11 09:30:38 +00:00
Igor Nikonov
04b1a0cb08 Fix style 2023-03-11 09:29:50 +00:00
Alexey Milovidov
6ba54ab421
Merge branch 'master' into rs/compile-with-cpp23 2023-03-11 06:33:14 +03:00
Igor Nikonov
4ae9030405 Check if aggregation keys form prefix in distinct columns 2023-03-10 22:03:53 +00:00
Igor Nikonov
73119b920b Remove distinct_columns 2023-03-10 20:32:09 +00:00
Nikolai Kochetov
91a2f39c0c
Merge branch 'master' into fix-optimize-aggregate-in-order-with-aliases 2023-03-10 21:17:02 +01:00
Nikolai Kochetov
2fc5864a11 Fix aggregation-in-order with aliases. 2023-03-10 19:37:42 +00:00
Robert Schulze
348a40aa5d
Compile with C++23 2023-03-10 19:11:58 +00:00
avogar
b6c91d9ace Support Decimals and Date32 in Avro format 2023-03-10 15:48:18 +00:00
Igor Nikonov
acffc98bd1 Fix build 2023-03-09 12:22:32 +00:00
Kruglov Pavel
fe973f3d6f
Merge branch 'master' into native-types-conversions 2023-03-09 13:03:25 +01:00
Kruglov Pavel
71b6d6c6ae
Merge pull request #47114 from Avogar/parquet-compression
Improve working with compression methods in Parquet/ORC/Arrow formats
2023-03-09 13:02:18 +01:00
Igor Nikonov
2ea3faf90c Remove distinct on top of MergingAggregatedStep as well 2023-03-09 11:44:38 +00:00
Igor Nikonov
feefce7de5 Test + remove debug logging 2023-03-09 09:24:50 +00:00
Azat Khuzhin
d7b5e36218
Merge branch 'master' into mv-ignore-errors 2023-03-08 22:13:05 +01:00
Igor Nikonov
c952886e66 Merge remote-tracking branch 'origin/master' into query-plan-update-sort-description 2023-03-08 18:09:37 +00:00
Igor Nikonov
cfa7119fa7 Fix: apply reading in order for distinct
+ respect aliases
2023-03-08 17:56:33 +00:00
avogar
79ef95bc0c Fix tests and builds 2023-03-08 12:54:22 +00:00
Maksim Kita
b6f3b81403 Query plan filter push down optimization filled JOIN fix 2023-03-08 11:16:06 +01:00
Vladimir C
4f048ce535
Merge pull request #47153 from ClickHouse/vdimir/grace_hash_limit_race_45986 2023-03-08 10:18:32 +01:00
Igor Nikonov
3f7dd78c9b Merge remote-tracking branch 'origin/master' into query-plan-update-sort-description 2023-03-07 22:45:31 +00:00
Igor Nikonov
79d4d029b8 Fix Distinct properties 2023-03-07 22:40:11 +00:00
Igor Nikonov
6f2851ef70 Update FilterStep 2023-03-07 22:16:22 +00:00
Igor Nikonov
e88912009a Update data streams for entire plan 2023-03-07 20:49:55 +00:00
Alexander Tokmakov
ffd55d951d
Merge branch 'master' into followup_46681 2023-03-07 23:38:06 +03:00
Kruglov Pavel
69a1309ade
Merge branch 'master' into native-types-conversions 2023-03-07 20:06:17 +01:00
Kruglov Pavel
ba0cc95441
Fix typo 2023-03-07 20:04:52 +01:00
Igor Nikonov
5f5e15f8c1 Update sort description to respect aliases in header 2023-03-07 18:28:10 +00:00
Nikolai Kochetov
8186473cc2 Remove some commented code. 2023-03-07 14:28:28 +00:00
Alexander Tokmakov
7ce20f5cd2 fix tests 2023-03-07 14:25:28 +01:00
Alexey Perevyshin
e676c1de0b
Merge branch 'master' into master 2023-03-07 09:48:49 +03:00
Kruglov Pavel
479cd9b90b
Merge pull request #46972 from Avogar/json-date-int-inference
Fix date and int inference from string in JSON
2023-03-06 20:40:38 +01:00
Kruglov Pavel
3de905bb7c
Merge pull request #46616 from Avogar/fix-ipv4-ipv6-formats
Fix IPv4/IPv6 serialization/deserialization in binary formats
2023-03-06 19:40:29 +01:00
Kruglov Pavel
fed645a494
Don't use switch 2023-03-06 18:03:54 +01:00
Kruglov Pavel
df34ab240d
Don't use switch 2023-03-06 18:03:17 +01:00
Vladimir C
3076e92980
Apply suggestions from code review
Co-authored-by: Igor Nikonov <954088+devcrafter@users.noreply.github.com>
2023-03-06 16:56:02 +01:00
avogar
d8e5fb5195 Use switch, use lz4 by default 2023-03-06 15:12:15 +00:00
Vladimir C
48f644a958
Fix style in JoiningTransform.cpp 2023-03-06 10:38:21 +01:00
vdimir
a0f5756751
Fix DelayedJoinedBlocksWorkerTransform for grace hash join 2023-03-03 16:30:01 +00:00
Alexander Tokmakov
cadaf06829
Merge pull request #46681 from ClickHouse/fix_insert_cancellation_in_native_protocol
Fix queries cancellation when a client dies
2023-03-03 16:27:20 +03:00
Maksim Kita
3cd88003dd
Merge pull request #47101 from kitaisreal/analyzer-planner-fixes-before-enable-by-default
Analyzer Planner fixes before enable by default
2023-03-02 22:16:39 +03:00
Alexander Tokmakov
c518f3690e Merge branch 'master' into fix_insert_cancellation_in_native_protocol 2023-03-02 18:24:08 +01:00
Kruglov Pavel
3b034e7c35
Merge branch 'master' into parquet-compression 2023-03-02 15:55:48 +01:00
Robert Schulze
740aeaba1f
Apply some CTAD 2023-03-02 13:36:47 +00:00
Kruglov Pavel
ce8e49a9a0
Merge pull request #45693 from Avogar/fix-kafka-protobuf
Fix formats parser resetting, test processing bad messages in kafka
2023-03-02 11:50:45 +01:00
avogar
a6cf2cdab8 Fix style, add docs 2023-03-02 10:36:07 +00:00
Alexey Milovidov
57ac42f48b
Merge pull request #46804 from nickitat/async_reading_small_limit
Fix async reading pipeline when small limit is present
2023-03-02 02:30:19 +03:00
avogar
5ab5902f38 Allow control compression in Parquet/ORC/Arrow output formats, support more compression for input formats 2023-03-01 21:27:46 +00:00
Maksim Kita
51ee007e01 Fixed tests 2023-03-01 18:05:07 +01:00
Alexander Tokmakov
a97e15e36f Merge branch 'master' into fix_insert_cancellation_in_native_protocol 2023-03-01 14:26:41 +01:00
Kseniia Sumarokova
fc619d06ba
Merge pull request #40807 from maks-buren630501/master
Fix create materialized view with MaterializedPostgreSQL
2023-02-28 15:10:14 +01:00
Kruglov Pavel
65f06fc9b1
Merge branch 'master' into json-date-int-inference 2023-02-28 14:31:57 +01:00
Kruglov Pavel
545e5c0f5b
Merge pull request #46617 from Avogar/use-parquet-2
Change default parquet format version to v2_latest, add a setting to controle the version
2023-02-28 14:30:59 +01:00
kssenii
24ce30c41b Merge remote-tracking branch 'upstream/master' into maks-buren630501-master 2023-02-27 22:06:11 +01:00
avogar
ab899bf2f3 Allow types conversion in Native input format 2023-02-27 19:28:19 +00:00
Nikolai Kochetov
1e0ea2446e Refactor a bit. 2023-02-27 16:59:32 +00:00
Nikolai Kochetov
9bf828cc98 Refactor a bit. 2023-02-27 16:43:54 +00:00
avogar
2e921e3d6b Fix date and int inference from string in JSON 2023-02-27 16:00:19 +00:00
Nikolai Kochetov
0400677483 makeMaterializingDAG 2023-02-27 15:41:50 +00:00
avogar
12a9ff36b2 Better exception messages when schema_inference_hints is ill-formatted 2023-02-27 15:20:56 +00:00
Kruglov Pavel
443dedddca
Merge branch 'master' into use-parquet-2 2023-02-27 14:31:43 +01:00
Nikolai Kochetov
3392b96cc3
Merge branch 'master' into aggregate-projections-analysis-query-plan 2023-02-27 14:07:04 +01:00
Nikita Taranov
70d5a41c51 impl 2023-02-27 13:57:29 +01:00
Yakov Olkhovskiy
7b8d2d716e bypass empty blocks 2023-02-25 23:26:38 +00:00
Nikolai Kochetov
7045ed539f Refactor a bit more. 2023-02-24 20:04:12 +00:00
Nikolai Kochetov
53b006dd5c Refactor a bit more. 2023-02-24 19:59:13 +00:00
Nikolai Kochetov
030a0ba7fb Refactor a bit more. 2023-02-24 18:39:27 +00:00
avogar
12b38ece1d Fix build 2023-02-24 13:07:09 +00:00
Kruglov Pavel
2150b836b1
Merge branch 'master' into fix-kafka-protobuf 2023-02-24 12:48:07 +01:00
Alexander Tokmakov
fe92fd8a61 Merge branch 'master' into fix_insert_cancellation_in_native_protocol 2023-02-23 22:35:31 +01:00
Nikolai Kochetov
c63bbbb374 Refactor a bit more. 2023-02-23 20:22:12 +00:00
Alexander Tokmakov
d21a6a3ba4 fix 2023-02-23 20:52:39 +01:00
Kruglov Pavel
47f9ca2166
Merge branch 'master' into fix-ipv4-ipv6-formats 2023-02-23 20:32:43 +01:00
avogar
54622566df Add setting to change parquet version 2023-02-23 16:14:10 +00:00
Anton Popov
955b54a6f8
Merge pull request #46753 from CurtizJ/fallback-from-async-insert
Fallback from async insert in case of large amount of data
2023-02-23 16:21:44 +01:00
Kruglov Pavel
35887911e3
Merge pull request #46284 from Avogar/null-as-default-all-formats
Extend setting `input_format_null_as_default` for more formats
2023-02-23 14:48:22 +01:00
Igor Nikonov
271b72abf4
Merge pull request #46642 from ClickHouse/remove_redundant_sorting_fix
Fix: remove redundant sorting optimization
2023-02-22 23:33:15 +01:00
Anton Popov
d5864fa88e allow to fallback from async insert in case of large amount of data 2023-02-22 21:59:24 +00:00
Nikolai Kochetov
c8ce1f2b50 Refactor a bit. 2023-02-22 17:27:21 +00:00
Nikolai Kochetov
67db9123d8 Refactor a bit. 2023-02-22 16:40:40 +00:00
flynn
678e4250cd
Fix incorrect predicate push down with grouping sets (#46151) 2023-02-22 11:54:19 +01:00
Azat Khuzhin
d0b50adffa Remove materialized_views_ignore_errors from FinalizingViewsTransform
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-02-22 11:00:14 +01:00
Alexander Tokmakov
f5c232707d cancel insert queries correctly 2023-02-21 22:04:39 +01:00
Nikolai Kochetov
a0b88f0a2d Fix typos. 2023-02-21 20:01:59 +00:00
Nikolai Kochetov
a47860af17 Fixing more tests. 2023-02-21 19:50:17 +00:00
Igor Nikonov
807a49021c Merge remote-tracking branch 'origin/master' into remove_redundant_sorting_fix 2023-02-21 16:59:16 +00:00
Igor Nikonov
45b1b66fd8 Remove unnecessary comment 2023-02-21 16:58:45 +00:00
Azat Khuzhin
81435282ec Add an entry to system.query_views_log regardless of materialized_views_ignore_errors
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-02-21 17:23:19 +01:00
Kruglov Pavel
a25255cabc
Update StreamingFormatExecutor.h 2023-02-21 16:57:44 +01:00
Kruglov Pavel
ef0d6becba
Merge branch 'master' into null-as-default-all-formats 2023-02-21 16:52:39 +01:00
Nikolai Kochetov
c5f93eb108 Fix more tests. 2023-02-21 15:44:50 +00:00
Azat Khuzhin
a110e0f022 Allow to ignore errors while pushing to MATERILIZED VIEW
This can be useful in the following scenarious:

- you want to duplicate the data to another table and you don't care
  about the errors
- you want to duplicate system.*_log to another server, you are adding
  materialized view that will push to Distributed table, but you don't
  want to miss original blocks in the local system.*_log
- you want to push some data to a 3d party service, using i.e. URL
  engine.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-02-21 16:28:38 +01:00
Alexander Gololobov
5354b0d069
Merge pull request #46140 from ClickHouse/split_prewhere_dag
Split PREWHERE DAG into multiple read steps
2023-02-21 14:37:09 +01:00
Kseniia Sumarokova
1f3be929f7
Update ReadFromMergeTree.cpp 2023-02-21 12:50:26 +01:00
Igor Nikonov
2dfc0d008b Fix: remove redundant sorting optimization
+ incorrect sorting step removal in case of parent step has more than 1
  children
2023-02-21 11:48:28 +00:00
Nikita Taranov
593de3416b
Fix typo in read prefetch 2023-02-21 12:27:08 +01:00
Kruglov Pavel
828a06b511
Merge branch 'master' into fix-kafka-protobuf 2023-02-20 21:45:43 +01:00
Kruglov Pavel
b0424c1021
Merge pull request #46171 from Avogar/insert-null-as-default
Use default of column type in `insert_null_as_default` if column DEFAULT values is not specified
2023-02-20 21:45:02 +01:00
avogar
a568704d63 Fix avro 2023-02-20 20:43:28 +00:00
Kruglov Pavel
9866ecfe8b
Merge branch 'master' into null-as-default-all-formats 2023-02-20 20:49:30 +01:00
Nikolai Kochetov
ee597b939f Merge branch 'master' into aggregate-projections-analysis-query-plan 2023-02-20 19:46:52 +00:00
Maksim Kita
4d23f74a53
Merge pull request #46494 from kitaisreal/query-plan-filter-push-down-sorting-fix
Planner filter push down optimization fix
2023-02-20 21:36:34 +03:00
Nikolai Kochetov
37a5e10b75 Fix more tests. 2023-02-20 17:55:10 +00:00
avogar
42ae0582d3 Try to use parquet v2 instead of v1 in output format 2023-02-20 17:46:54 +00:00
avogar
8da3594cd8 Fix IPv4/IPv6 serialization/deserialization in binary formats 2023-02-20 17:42:56 +00:00
Nikolai Kochetov
84e442e620 Fixing build and style. 2023-02-20 14:21:20 +00:00
Nikolai Kochetov
2a8460d962 Merge branch 'master' into aggregate-projections-analysis-query-plan 2023-02-20 13:54:05 +01:00
Alexey Milovidov
d8cda3dbb8 Remove PVS-Studio 2023-02-19 23:30:05 +01:00
Alexey Milovidov
17992b178a
Merge pull request #45364 from nickitat/aggr_partitions_independently
Add option to aggregate partitions independently
2023-02-19 17:44:18 +03:00
Maksim Kita
72719584ea Planner filter push down optimization fix 2023-02-19 12:02:59 +01:00
Maksim Kita
6dffe52e68
Merge pull request #46471 from kitaisreal/analyzer-planner-fixes-before-enable-by-default
Analyzer planner fixes before enable by default
2023-02-19 13:49:18 +03:00
Maksim Kita
cbd961de98 Fixed code review issues 2023-02-18 17:06:00 +01:00
Nikita Taranov
eee39b7bd1 bring back lost lines 2023-02-18 14:30:49 +00:00
Nikita Taranov
ef61c22167 another review fixes 2023-02-18 14:26:24 +00:00
Nikita Taranov
f70044f34b Merge branch 'master' into aggr_partitions_independently 2023-02-18 13:19:05 +00:00
Kseniia Sumarokova
bda4c94f66
Merge pull request #45384 from ucasfl/iceberg 2023-02-17 23:38:24 +01:00
Igor Nikonov
0bad474fb5
Merge pull request #44176 from ClickHouse/igor/remove_redundant_distinct
Remove redundant DISTINCT
2023-02-17 22:07:03 +01:00
Nikolai Kochetov
6de781afcc Fix 01710 tests. 2023-02-17 19:18:46 +00:00
Igor Nikonov
486501296c Merge remote-tracking branch 'origin/master' into igor/remove_redundant_distinct 2023-02-17 14:40:02 +00:00
Igor Nikonov
64dfc9f8ce Fix clang-analyzer warning 2023-02-17 14:39:20 +00:00
flynn
ebd88aab41
Merge branch 'master' into iceberg 2023-02-17 21:53:29 +08:00
Igor Nikonov
3871959605
Merge branch 'master' into fix-crash-with-distinct 2023-02-17 11:43:43 +01:00
Alexey Milovidov
993a39e2a1
Merge pull request #46360 from genzgd/http_client_version
Allow to provide the protocol version by HTTP clients to tune the compatibility of the Native output format.
2023-02-16 22:17:04 +03:00
Nikolai Kochetov
9cca571777 Fix constants in the result of MergingSortedAlgorithm. 2023-02-16 17:58:35 +00:00
Alexander Gololobov
a6061e022e Merge branch 'master' into split_prewhere_dag 2023-02-16 14:43:58 +01:00
youennL-cs
6526c2a8ab
[RFC] Replacing merge tree new engine (#41005)
* Add new engine to ReplacingMergeTree corresponding to the ReplacingCollapsingMergeTree

* Add new test for the new ReplacingMergeTree engine

* Limit sign value to -1/1

* Add new engine to ReplacingMergeTree corresponding to the ReplacingCollapsingMergeTree

* Add new test for the new ReplacingMergeTree engine

* Limit sign value to -1/1

* Replace sign column(Int8) by is_deleted(UInt8)

* Add new engine to ReplacingMergeTree corresponding to the ReplacingCollapsingMergeTree

* Add new test for the new ReplacingMergeTree engine

* Limit sign value to -1/1

* Replace sign column(Int8) by is_deleted(UInt8)

* Add new engine to ReplacingMergeTree corresponding to the ReplacingCollapsingMergeTree

* Add new test for the new ReplacingMergeTree engine

* Limit sign value to -1/1

* Replace sign column(Int8) by is_deleted(UInt8)

* Add keyword 'CLEANUP' when OPTIMIZE

* Cleanup uniquely when it's a replacingMergeTree

* Propagate CLEANUP information and change from 'with_cleanup' to 'cleanup'

* Cleanup data flagged as 'is_deleted'

* Fix merge when optimize and add a test

* Fix OPTIMIZE and INSERT + add tests

* New fix for cleanup at the merge

* Cleanup debug logs

* Add the SETTINGS option 'clean_deleted_rows' that can be 'never' or 'always'

* Fix regression bug; Now REplicatedMergeTree can be called as before without 'is_deleted'

* Add Replicated tests

* Disable tag 'long' for our test and cleanup some white spaces

* Update tests

* Fix tests and remove additional useless whitespace

* Fix replica test

* Style clean && add condition check for is_deleted values

* clean_deleted_rows settings is nom an enum

* Add valid default value to the clean_deleted_rows settings

* Update cleanup checkers to use the enum and fix typos in the test

* Fix submodule contrib/AMQP-CPP pointer

* Add missing messages in test reference and remove a print with non derterministic order

* fix replica test reference

* Fix edge case

* Fix a typo for the spell checker

* Fix reference

* Fix a condition to raise an error if is_deleted differ from 0/1 and cleanup

* Change tests file name and update number

* This should fix the ReplacingMergeTree parameter set

* Fix replicated parameters

* Disable allow_deprecated_syntax_for_merge_tree for our new column

* Fix a test

* Remove non deterministic order print in the test

* Test on replicas

* Remove a condition, when checking optional parameters, that should not be sueful since we disabled the deprected_syntaxe

* Revert "Remove a condition, when checking optional parameters, that should not be useful since we disabled the deprected_syntaxe"

This reverts commit b65d64c05e.

* Fix replica management and limit the number of argument to two maximum, due to the possiblity of deprecated table create/attach failing otherwise

* Test a fix for replicated log information error

* Try to add sync to have consistent results

* Change path of replicas that should cause one issue and add few prints in case it's not that

* Get cleanup info on replicas only if information found

* Fix style issues

* Try to avoid replication error 'cannot select parts...' and and replica read/write field order

* Cleanup according to PR reviews
 and add tests on error raised.

* Update src/Storages/MergeTree/registerStorageMergeTree.cpp

Co-authored-by: Alexander Tokmakov <tavplubix@gmail.com>

* Select ... FINAL don't show rows with is_deleted = true

* Update and fix SELECT ... FINAL merge parameter

* Remove is_deleted rows only on the version inserted when merge

* Fix (master) updates issues

* Revert changes that should not be commited

* Add changes according to review

* Revert changes that should not be commited - part 2

---------

Co-authored-by: Alexander Tokmakov <tavplubix@gmail.com>
2023-02-16 16:03:16 +03:00
Maksim Kita
b72ea982b0 Remove unnecessary includes of InterpreterSelectQuery header 2023-02-16 12:17:03 +01:00
Maksim Kita
4ddf1b0f48 Planner filter push down optimization fix 2023-02-16 12:17:03 +01:00
Maksim Kita
80af0666ea Analyzer support Materialized View 2023-02-16 12:17:03 +01:00
Maksim Kita
077555bd5d Analyzer support build pushing to views chain 2023-02-16 12:17:03 +01:00
Maksim Kita
f8442b2a8d Analyzer support LiveView 2023-02-16 12:17:03 +01:00
Kseniia Sumarokova
947753bd4c
Merge pull request #46409 from kssenii/follow-up-to-46168
Follow-up to #46168
2023-02-16 11:26:54 +01:00
Nikolay Degterinsky
ebec5b8050
Merge pull request #46422 from evillique/move-sources
Move MongoDB and PostgreSQL sources to Sources folder
2023-02-16 10:51:45 +01:00
flynn
ecc39978d7 fix conflict 2023-02-16 02:23:55 +00:00
Alexander Tokmakov
53e32b19e6
Update AggregatingTransform.cpp 2023-02-16 02:13:18 +03:00
Igor Nikonov
cb89fccac6 Disable debug output 2023-02-15 21:26:00 +00:00
Igor Nikonov
766f04b04c Merge remote-tracking branch 'origin/master' into igor/remove_redundant_distinct 2023-02-15 21:24:25 +00:00
Nikolai Kochetov
f524dae924 Fix more tests. 2023-02-15 19:17:24 +00:00
alexX512
26fd12e0c7 Add checking for source processors in executing graph 2023-02-15 15:40:42 +00:00
Vladimir C
3e77463125
Merge pull request #46256 from bigo-sg/window_functions 2023-02-15 15:36:41 +01:00
Kruglov Pavel
e56dead5aa
Merge branch 'master' into fix-kafka-protobuf 2023-02-15 15:16:25 +01:00
Vladimir C
645fee3d94
Merge pull request #46009 from ClickHouse/vdimir/fix_read_in_order_final_desc 2023-02-15 11:23:07 +01:00
Nikolay Degterinsky
bcc4adf73d
Merge branch 'master' into move-sources 2023-02-15 08:46:09 +01:00
alexX512
5aabfe8644 Hard cancel for pipeline executor 2023-02-15 06:50:33 +00:00
lgbo-ustc
bfbd880475 fixed binary_tidy build failure 2023-02-15 08:44:10 +08:00
Nikolay Degterinsky
6e4b660033 Move MongoDB and PostgreSQL sources to Sources folder 2023-02-14 22:35:10 +00:00
Igor Nikonov
3eef3bf643 Simplify aggregation pass
always checking AggregatingStep and skipping Rollup, Cube, TotalsHaving
2023-02-14 21:49:56 +00:00
alexX512
e424ad9ba8 Delete unimportant logs 2023-02-14 21:24:33 +00:00
alexX512
b466855e04 Add hard_cancel mod 2023-02-14 21:20:01 +00:00
Igor Nikonov
425b99afca GROUP BY WITH TOTALS 2023-02-14 21:14:41 +00:00
Nikolai Kochetov
62bce41d3d Fix more tests. 2023-02-14 19:31:20 +00:00
Kseniia Sumarokova
fe155bb210
Merge branch 'master' into follow-up-to-46168 2023-02-14 18:28:46 +01:00
Geoff Genz
f3bcf26959
Merge branch 'ClickHouse:master' into http_client_version 2023-02-14 09:31:29 -07:00
kssenii
5bda358fb7 Follow-up to #46168 2023-02-14 15:45:38 +01:00
flynn
e05b4e9745 fix conflict 2023-02-14 13:49:28 +00:00
Nikita Taranov
cbd10c4493
Fix mbm with aio on top of query plan (#45892) 2023-02-14 14:02:54 +01:00
Igor Nikonov
49763f7dc2 Merge remote-tracking branch 'origin/master' into igor/remove_redundant_distinct 2023-02-14 12:13:22 +00:00
Igor Nikonov
7e24b5f059 Fix tests & review comment 2023-02-14 12:10:02 +00:00
Geoff Genz
2857b16562
Merge branch 'ClickHouse:master' into http_client_version 2023-02-14 03:41:17 -07:00
robot-ch-test-poll4
7c420f0982
Merge pull request #46350 from CurtizJ/fix-format-values
Fix `LOGICAL_ERROR` in async inserts with invalid data in format `VALUES`
2023-02-14 10:12:08 +01:00
Alexey Milovidov
307551f857
Update NativeFormat.cpp 2023-02-14 11:08:33 +03:00
lgbo-ustc
74d6ccad4f fixed
change window specification as (partition by a order by b rows between unbounded preceding and unbounded following)
2023-02-14 15:49:12 +08:00
lgbo-ustc
0b32a655a8 fixed 2023-02-14 15:49:12 +08:00
lgbo-ustc
9a496aea6a add new window function, ntile 2023-02-14 15:49:11 +08:00
Nikolai Kochetov
522a39f93f Normal Projections analysis using query plan [In progress] 2023-02-13 19:56:31 +00:00
Nikolai Kochetov
e6e486c4a1 Not crash with minmax projection. 2023-02-13 17:10:10 +00:00
Nikolai Kochetov
d6ea566b20 Mostly support minmax projection. 2023-02-13 16:52:21 +00:00
Geoff Genz
be8bf3a6a3
Merge branch 'master' into http_client_version 2023-02-13 08:43:59 -07:00
Anton Popov
7f31196606 fix LOGICAL_ERROR in async inserts 2023-02-13 14:59:14 +00:00
Kruglov Pavel
e139070043
Merge branch 'master' into fix-kafka-protobuf 2023-02-13 12:38:18 +01:00
Vladimir C
89cb55aa1f
whitespace 2023-02-13 12:19:31 +01:00
Vladimir C
1a89ecd386 Apply suggestions from code review
Co-authored-by: Nikita Taranov <nikita.taranov@clickhouse.com>
2023-02-13 12:17:56 +01:00
vdimir
e4c0afcc3e Do not build filtering set for sorting join with const columns 2023-02-13 12:17:56 +01:00
Alexander Gololobov
8fc08c74f0 Pass action_settings to getReadTaskColumns() 2023-02-12 21:55:53 +01:00
Alexander Gololobov
7634b9f56e Added settings to enable the new logic 2023-02-12 21:54:23 +01:00
kssenii
3067c1d723 Merge remote-tracking branch 'upstream/master' into resubmit-prefetches 2023-02-11 11:36:23 +01:00
Alexander Gololobov
f2a975f548
Merge pull request #46270 from Avogar/fix-arrow-lc
Fix possible out of bounds error while reading LowCardinality(Nullable) in Arrow format
2023-02-11 10:03:00 +01:00
Vladimir C
ea670acb72
Add comment to requestReadingInOrder 2023-02-10 20:05:18 +01:00
Kruglov Pavel
8bbf62508f
Merge branch 'master' into fix-kafka-protobuf 2023-02-10 18:32:25 +01:00
Kruglov Pavel
2a3cb8b4ee
Merge pull request #45340 from Avogar/parquet-fixed-binary
Support FixedSizeBinary type in Parquet/Arrow
2023-02-10 18:31:20 +01:00
avogar
04cf144edc Fix TSKV, update docs 2023-02-10 17:20:51 +00:00
Alexey Milovidov
d5d87bc6b7
Merge pull request #46152 from ClickHouse/with-fill-bigint
More convenient usage of big integers and ORDER BY WITH FILL.
2023-02-10 20:05:32 +03:00
avogar
d1efd02480 Extend setting input_format_null_as_default for more formats 2023-02-10 16:41:09 +00:00
avogar
deda940d87 Fix possible out of bounds error while reading LowCardinality(Nullable) in Arrow format 2023-02-10 12:57:47 +00:00
Geoff Genz
99c3ff53c5 Merge remote-tracking branch 'origin/master' into http_client_version
# Conflicts:
#	src/Interpreters/Context.cpp
#	src/Interpreters/Context.h
2023-02-10 04:35:53 -07:00
Geoff Genz
db982695fa Simplify NativeOutputFormat constructors 2023-02-10 04:30:08 -07:00
Geoff Genz
7ed8ed0284 Add support for client_protocol_version sent with HTTP 2023-02-10 03:47:06 -07:00
flynn
37a850bd36 Merge branch 'master' of github.com:ClickHouse/ClickHouse into iceberg 2023-02-10 08:47:14 +00:00
flynn
db15634a01 fix conflict 2023-02-10 08:41:04 +00:00
avogar
ba761e04d7 Handle unknown exceptions 2023-02-09 12:31:01 +00:00
Kruglov Pavel
c3c0356fdd
Merge branch 'master' into fix-kafka-protobuf 2023-02-09 13:04:55 +01:00
Igor Nikonov
df23731061 Removing DISTINCT on top of aggregation if possible 2023-02-09 11:37:15 +00:00
vdimir
45f7ef6c8f
Do not apply read in reverse order for queries with final 2023-02-09 11:00:22 +00:00
vdimir
a179528ea4
Revert "Fix read in order optimization for DESC sorting with FINAL"
This reverts commit cf762347d5.
2023-02-09 10:47:45 +00:00
Robert Schulze
e490ec91d9
Merge branch 'master' into rs/fix-fragile-linking 2023-02-09 11:33:59 +01:00
Kruglov Pavel
b5f90c608a
Merge branch 'master' into parquet-fixed-binary 2023-02-09 11:26:38 +01:00
vdimir
cf762347d5
Fix read in order optimization for DESC sorting with FINAL 2023-02-09 10:26:37 +00:00
Vladimir C
0042c20770
Merge pull request #46054 from alexX512/bug_fix_pulling_pipeline 2023-02-09 10:50:46 +01:00
Alexey Milovidov
3a75ede60a Implement #16733 2023-02-09 08:41:47 +01:00
Alexey Milovidov
cb847172ea Fix something 2023-02-09 08:39:46 +01:00
Alexey Milovidov
a40ef2bf18 Make it better 2023-02-09 08:09:19 +01:00
Alexey Milovidov
292a5abbc9 Merge remote-tracking branch 'origin/master' into with-fill-bigint 2023-02-09 01:47:51 +01:00
Alexander Tokmakov
8101b044fa
Merge pull request #46091 from azat/sanity-assertions
Sanity assertions for closing file descriptors
2023-02-09 01:02:03 +03:00
kssenii
b0b865c32e Resubmit prefetches 2023-02-08 21:26:24 +01:00
avogar
c3e8dd8984 Fix low cardinality case 2023-02-08 19:14:28 +00:00
Nikolai Kochetov
e7dba2a85b Partly support minmax projection. 2023-02-08 18:34:22 +00:00
Nikolai Kochetov
d04b3a30cc Fix 01710_projections_partial_optimize_aggregation_in_order 2023-02-08 14:26:20 +00:00
Nikolai Kochetov
f30d1efe2f Fix test 01710_projections. 2023-02-08 13:32:34 +00:00