Commit Graph

37952 Commits

Author SHA1 Message Date
Igor Nikonov
bf05082984
Merge branch 'master' into keeper-retries-by-default 2023-02-21 17:59:55 +01:00
Igor Nikonov
807a49021c Merge remote-tracking branch 'origin/master' into remove_redundant_sorting_fix 2023-02-21 16:59:16 +00:00
Alexey Milovidov
848286eca7
Merge pull request #40945 from arthurpassos/add_select_final_mt_setting
Add user setting to force select final at query level
2023-02-21 19:58:45 +03:00
Igor Nikonov
45b1b66fd8 Remove unnecessary comment 2023-02-21 16:58:45 +00:00
Kruglov Pavel
ef0d6becba
Merge branch 'master' into null-as-default-all-formats 2023-02-21 16:52:39 +01:00
Kseniia Sumarokova
655e7f29f7
Merge pull request #46595 from kssenii/update-rabbit-producer
Fix rabbitmq test
2023-02-21 16:18:55 +01:00
Salvatore Mesoraca
c83583e46b
ActionsDAG: do not change result of and() during optimization
Previously `and` results were inconsistent.
Some optimizations can try to simplify and completely remove a
single-argument `and`, replacing it with a cast to UInt8.
This works well for all numeric types except floats.
A float passed to `and` is normally interpreted as `bool` and
values between 0 and 1 are treated as 1.
The same value cast to an integer type will be truncated to 0.
2023-02-21 15:39:44 +01:00
Alexander Gololobov
5354b0d069
Merge pull request #46140 from ClickHouse/split_prewhere_dag
Split PREWHERE DAG into multiple read steps
2023-02-21 14:37:09 +01:00
Alexander Tokmakov
b5e6d74d48
More interesting settings for Stress Tests (#41534)
* Update stress

* fix

---------

Co-authored-by: Alexey Milovidov <milovidov@clickhouse.com>
2023-02-21 15:36:23 +03:00
Kseniia Sumarokova
843092e655
Merge pull request #46640 from ClickHouse/nickitat-patch-10
Fix typo in read prefetch
2023-02-21 13:32:49 +01:00
Kruglov Pavel
36870f0801
Merge pull request #46569 from ClickHouse/add-bug
Fix processing Const(LowCardinality) in arrayMap
2023-02-21 13:11:06 +01:00
Kruglov Pavel
2924e8e802
Merge pull request #46631 from bigo-sg/json_array_length
Add new function JSONArrayLength
2023-02-21 12:50:27 +01:00
Kseniia Sumarokova
1f3be929f7
Update ReadFromMergeTree.cpp 2023-02-21 12:50:26 +01:00
Igor Nikonov
2dfc0d008b Fix: remove redundant sorting optimization
+ incorrect sorting step removal in case of parent step has more than 1
  children
2023-02-21 11:48:28 +00:00
Nikita Taranov
593de3416b
Fix typo in read prefetch 2023-02-21 12:27:08 +01:00
Kseniia Sumarokova
09a979f106
Merge branch 'master' into update-rabbit-producer 2023-02-21 11:50:30 +01:00
Vladimir C
a791250a4a
Merge pull request #38614 from bharatnc/ncb/profile-events-partlog 2023-02-21 11:06:44 +01:00
Yakov Olkhovskiy
ce2b044532 analyzer - apply limit and offset settings 2023-02-21 03:29:14 +00:00
taiyang-li
fcd3e44297 finish dev 2023-02-21 10:55:13 +08:00
Alexey Milovidov
67d808f49a Add settings changes history 2023-02-21 00:30:38 +01:00
Alexey Milovidov
4d0ba39e7f Merge branch 'master' into keeper-retries-by-default 2023-02-21 00:27:37 +01:00
Kseniia Sumarokova
24d0490ef3
Merge branch 'master' into delay-loading-of-named-collections 2023-02-21 00:24:18 +01:00
Kruglov Pavel
b0424c1021
Merge pull request #46171 from Avogar/insert-null-as-default
Use default of column type in `insert_null_as_default` if column DEFAULT values is not specified
2023-02-20 21:45:02 +01:00
avogar
a568704d63 Fix avro 2023-02-20 20:43:28 +00:00
Kruglov Pavel
9866ecfe8b
Merge branch 'master' into null-as-default-all-formats 2023-02-20 20:49:30 +01:00
Arthur Passos
4f2fcf763b
Merge branch 'master' into add_select_final_mt_setting 2023-02-20 16:22:14 -03:00
Dmitry Novik
3c4c527bce Fix MemoryTracker counters for async inserts 2023-02-20 18:40:57 +00:00
Maksim Kita
4d23f74a53
Merge pull request #46494 from kitaisreal/query-plan-filter-push-down-sorting-fix
Planner filter push down optimization fix
2023-02-20 21:36:34 +03:00
Raúl Marín
87a4fb0252 Update aliases when clone happens 2023-02-20 19:08:54 +01:00
Kseniia Sumarokova
47604ad77e
Merge pull request #46106 from kssenii/return-materialized-postrgesql-tests-back
Enable disabled MaterializedPostgreSQL tests back in CI
2023-02-20 19:04:58 +01:00
avogar
42ae0582d3 Try to use parquet v2 instead of v1 in output format 2023-02-20 17:46:54 +00:00
SmitaRKulkarni
8ebbd03553
Merge pull request #46583 from ClickHouse/Support_parameterized_view_with_in_clause
Support for IN clause in parameterized views
2023-02-20 18:26:31 +01:00
Alexander Gololobov
c84a64d22f Return chunks with 0 rows from MergeTreeSource to report progress when rows are filtered in PREWHERE 2023-02-20 17:35:01 +01:00
Raúl Marín
2273acaa30 Fix incorrect alias recursion in QueryNormalizer 2023-02-20 16:51:52 +01:00
kssenii
9d16205c8a Load named collections on first access 2023-02-20 16:33:45 +01:00
Alexey Milovidov
5ef159fa10
Merge pull request #46565 from ClickHouse/remove-pvs-studio
Remove PVS-Studio
2023-02-20 17:10:53 +03:00
avogar
1539f812b3 Fix processing Const(LowCardinality) in arrayMap 2023-02-20 13:48:52 +00:00
kssenii
80a0aa1327 Merge remote-tracking branch 'upstream/master' into return-materialized-postrgesql-tests-back 2023-02-20 14:37:39 +01:00
kssenii
361678ad73 rabbitmq-test-fix 2023-02-20 14:16:40 +01:00
vdimir
78c35ffc45
finally fix attachProfileCountersScope 2023-02-20 12:28:44 +00:00
SmitaRKulkarni
90834d4aa5
Merge pull request #46487 from ClickHouse/45185_Inner_join_with_where
Updated to not clear on_expression from table_join
2023-02-20 11:38:31 +01:00
Robert Schulze
135170b27d
Merge pull request #44973 from ClickHouse/remove-smp-limitation
Slightly modify the detection of the number of CPU cores
2023-02-20 11:22:39 +01:00
Smita Kulkarni
e5e1027637 Support for IN clause in parameterized views
Implementation:
* In case of parameterized views, the IN clause cannot be evaluated as constant expression during CREATE VIEW, added a check to ignore this step in case of parameterized view.
* If parmeters are not in IN clause, we continue to evaluate it as constant expression.
2023-02-20 09:48:55 +01:00
Yakov Olkhovskiy
d9bc4bcccb
Merge pull request #46296 from jh0x/feature-array-partial-sort
Add arrayPartialSort and arrayPartialReverseSort
2023-02-20 02:49:38 -05:00
Nikolay Degterinsky
f9dac6d4cf
Fix ASTAsterisk::clone() 2023-02-20 00:26:14 +01:00
Alexey Milovidov
d8cda3dbb8 Remove PVS-Studio 2023-02-19 23:30:05 +01:00
pufit
ec49fbec23
Update src/Core/ServerSettings.h
Co-authored-by: Sergei Trifonov <sergei@clickhouse.com>
2023-02-19 16:49:01 -05:00
Arthur Passos
3d9b23e5f3
Merge branch 'master' into add_select_final_mt_setting 2023-02-19 15:01:41 -03:00
Han Fei
06e679d6cd
Merge pull request #46549 from hanfei1991/hanfei/fix-async-insert-query-num-setting
enable async-insert-max-query-number only if async_insert_deduplicate
2023-02-19 17:53:16 +01:00
Alexey Milovidov
17992b178a
Merge pull request #45364 from nickitat/aggr_partitions_independently
Add option to aggregate partitions independently
2023-02-19 17:44:18 +03:00
Alexey Milovidov
fa7f7e1d3a
Merge pull request #37870 from zzachimed/Aggregate_Sum_AVX512
Add avx512 support for Aggregate Sum, function unary arithmetic, func…
2023-02-19 17:21:16 +03:00
Alexey Milovidov
1adfbac19f
Merge pull request #37894 from yaqi-zhao/lower_upper
add avx512 support for Lower/Upper function
2023-02-19 17:18:10 +03:00
Yakov Olkhovskiy
2b9ce319b8
remove unnecessary include 2023-02-19 08:37:51 -05:00
Alexey Milovidov
5359dcc014
Merge pull request #46554 from ucasfl/iceberg-fix
fix build without avro
2023-02-19 14:35:29 +03:00
Alexander Gololobov
520b381675 Fixes for executing actions on block with unknown row count 2023-02-19 12:14:23 +01:00
Maksim Kita
72719584ea Planner filter push down optimization fix 2023-02-19 12:02:59 +01:00
Maksim Kita
6dffe52e68
Merge pull request #46471 from kitaisreal/analyzer-planner-fixes-before-enable-by-default
Analyzer planner fixes before enable by default
2023-02-19 13:49:18 +03:00
Alexander Gololobov
1b8cae8ee5
Merge pull request #46529 from ClickHouse/regexp_extract
Merge #46469
2023-02-19 10:28:35 +01:00
flynn
7b4fbf33b3 fix 2023-02-19 03:47:14 +00:00
Alexey Milovidov
417158f59f
Merge branch 'master' into lower_upper 2023-02-19 04:05:10 +03:00
Alexey Milovidov
a22f0eee2a
Merge branch 'master' into Aggregate_Sum_AVX512 2023-02-19 04:03:45 +03:00
Alexander Gololobov
1480fce688 Removed extra identation 2023-02-18 21:41:30 +01:00
Alexander Gololobov
91b15caa8b Simplify arguments validation 2023-02-18 21:41:18 +01:00
Maksim Kita
7cc10008e3 Fixed build 2023-02-18 20:34:12 +01:00
Alexander Gololobov
0ed0b191f8 Fix for Nullable step filter column 2023-02-18 19:12:32 +01:00
flynn
177261a5ac fix typo 2023-02-18 17:34:37 +00:00
flynn
830398f0bd fix build without avro
fix
2023-02-18 17:32:35 +00:00
Alexander Gololobov
88fb6c3c25 Fix for incorrect combined PREWHERE column 2023-02-18 18:29:16 +01:00
Joanna Hulboj
cae064822c FIXUP: FunctionArrayMapped - Simpler arg passing for check/exec 2023-02-18 17:20:57 +00:00
Maksim Kita
cbd961de98 Fixed code review issues 2023-02-18 17:06:00 +01:00
Nikolay Degterinsky
f93aae80b8
Merge pull request #46407 from evillique/parameters-rename-query
Add parameterized RENAME queries
2023-02-18 16:04:55 +01:00
Alexey Milovidov
f073cd470f
Merge pull request #46478 from ClickHouse/fix_test_store_cleanup
Fix some flaky integration tests
2023-02-18 17:33:02 +03:00
Nikita Taranov
eee39b7bd1 bring back lost lines 2023-02-18 14:30:49 +00:00
Nikita Taranov
ef61c22167 another review fixes 2023-02-18 14:26:24 +00:00
Alexey Milovidov
46e23d7eee
Merge pull request #46552 from azat/state-reset-nothrow
Remove extra try/catch for QueryState/LocalQueryState reset
2023-02-18 17:20:03 +03:00
Nikita Taranov
f70044f34b Merge branch 'master' into aggr_partitions_independently 2023-02-18 13:19:05 +00:00
Joanna Hulboj
c2f5331a2d Revert "FIXUP: Into FunctionArrayMapped: Simplify the dispatch in case of positional params"
This reverts commit d48bdcc754.
2023-02-18 12:54:52 +00:00
Kseniia Sumarokova
b0ea047e76
Merge pull request #46533 from kssenii/fix-disk-setting
Allow `disk` merge tree setting in query to override config merge tree setting `storage_policy`
2023-02-18 12:36:54 +01:00
Azat Khuzhin
6f7b6e9206 Remove extra try/catch for QueryState reset
It was possible back when destuctors may throw, in a pre C++11.

I've found one place where it may throw from 2012:

    ~UnionBlockInputStream()
    {
        ...
        if (exception && !std::uncaught_exception())
            exception->rethrow();
        ...
    }

Refs: 8a053aba54
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-02-18 10:55:15 +01:00
Azat Khuzhin
f9cd66db36 Remove extra try/catch for LocalQueryState reset
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-02-18 10:55:09 +01:00
pufit
4496db66b2 Implement system.server_settings 2023-02-17 23:46:09 -05:00
Han Fei
b76df1401e enable async-insert-max-query-number only if async_insert_deduplicate is true 2023-02-18 01:08:28 +01:00
Kseniia Sumarokova
bda4c94f66
Merge pull request #45384 from ucasfl/iceberg 2023-02-17 23:38:24 +01:00
Igor Nikonov
0bad474fb5
Merge pull request #44176 from ClickHouse/igor/remove_redundant_distinct
Remove redundant DISTINCT
2023-02-17 22:07:03 +01:00
Igor Nikonov
eca2f2866e
Merge pull request #46493 from ClickHouse/fix-crash-with-distinct
Fix constants in the result of MergingSortedAlgorithm.
2023-02-17 22:05:24 +01:00
Kseniia Sumarokova
a0b28edd46
Merge pull request #46521 from kssenii/fix-stress-test-logical-errors
Fix stress test
2023-02-17 21:01:20 +01:00
Dmitry Novik
197116dfeb
Merge pull request #46451 from ClickHouse/copy-s3-file-fix
Wait for tasks finish when scheduler failed
2023-02-17 20:11:23 +01:00
Alexander Gololobov
3d9b3c1b76 Fix for read buffer overrun 2023-02-17 19:38:43 +01:00
Alexander Tokmakov
53b6fd79fb
Update DatabaseCatalog.cpp 2023-02-17 21:18:49 +03:00
Alexander Tokmakov
3457b68dcc fix 2023-02-17 17:57:49 +01:00
Nikolay Degterinsky
5104b2e8bf
Merge branch 'master' into parameters-rename-query 2023-02-17 17:30:37 +01:00
kssenii
afb36dd120 Allow disk setting to override storage_policy from config merge tree settings insteaf of exception 2023-02-17 17:11:29 +01:00
Maksim Kita
6c1436f78e Fixed tests 2023-02-17 16:50:46 +01:00
Kseniia Sumarokova
bf6e6a88f7
Merge branch 'master' into iceberg 2023-02-17 16:36:28 +01:00
Alexey Milovidov
45b1eabd36
Merge pull request #46420 from evillique/replace-transformer-fix-parameters
Fix parameterized queries with REPLACE transformer
2023-02-17 18:25:57 +03:00
Igor Nikonov
486501296c Merge remote-tracking branch 'origin/master' into igor/remove_redundant_distinct 2023-02-17 14:40:02 +00:00
Igor Nikonov
64dfc9f8ce Fix clang-analyzer warning 2023-02-17 14:39:20 +00:00
flynn
ebd88aab41
Merge branch 'master' into iceberg 2023-02-17 21:53:29 +08:00
Alexander Gololobov
bcb196a11f Extracted common logic into saveMatch() 2023-02-17 14:42:37 +01:00
Nikolai Kochetov
b7a96eaead
Merge branch 'master' into fix-crash-with-distinct 2023-02-17 14:25:47 +01:00
Vladimir C
0f182f6164
Merge pull request #46454 from azat/merge-alias-prewhere 2023-02-17 14:10:52 +01:00
Alexander Tokmakov
fb3574d2cf Merge branch 'master' into fix_test_store_cleanup 2023-02-17 13:59:22 +01:00
flynn
b3a9468661 fix 2023-02-17 12:42:24 +00:00
flynn
4b1d997b82 fix 2023-02-17 12:27:53 +00:00
kssenii
bae85c50e9 Fix stress test 2023-02-17 12:40:17 +01:00
Alexander Tokmakov
9a664efeb6
Merge branch 'master' into fixes_for_993 2023-02-17 14:17:08 +03:00
Igor Nikonov
3871959605
Merge branch 'master' into fix-crash-with-distinct 2023-02-17 11:43:43 +01:00
Sema Checherinda
b551b9fb3a
Merge pull request #46452 from CheSema/resolve-race-get-csn
resolve race in getCSNAndAssert
2023-02-17 11:31:58 +01:00
flynn
1d5b7ebc73 fix 2023-02-17 09:01:13 +00:00
flynn
561b575111 fix style 2023-02-17 08:35:38 +00:00
flynn
a39f6f419b refactor 2023-02-17 08:27:52 +00:00
Joanna Hulboj
3771d97936 FIXUP: Unused error code 2023-02-17 07:40:54 +00:00
Nikolay Degterinsky
3b19823960
Merge branch 'master' into replace-transformer-fix-parameters 2023-02-17 02:02:54 +01:00
Joanna Hulboj
d48bdcc754 FIXUP: Into FunctionArrayMapped: Simplify the dispatch in case of positional params 2023-02-16 20:17:49 +00:00
Joanna Hulboj
8747e23dbb FIXUP: PR comments 2023-02-16 20:17:49 +00:00
Azat Khuzhin
5af2967f65 Revert code in TreeRewriter for proper column order for UNION
It is not required after #44920, which has a proper fix that does this
in ExpressionAnalyzer.

This reverts commit d98336ad83.
Reverts: #37887

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-02-16 20:46:39 +01:00
Alexey Milovidov
993a39e2a1
Merge pull request #46360 from genzgd/http_client_version
Allow to provide the protocol version by HTTP clients to tune the compatibility of the Native output format.
2023-02-16 22:17:04 +03:00
HarryLeeIBM
3a635e428a Fix xxhash endian issue for s390x 2023-02-16 11:03:41 -08:00
Arthur Passos
ba6b7b1610 fix test_alias_column 2023-02-16 16:01:25 -03:00
Maksim Kita
11965e59e2 Analyzer Planner disable by default 2023-02-16 19:46:24 +01:00
Alexander Gololobov
e0cc1b9ae4 Fix assertion when index argument is not passed 2023-02-16 19:29:26 +01:00
Alexander Gololobov
7b4219e23a Added documentation field 2023-02-16 19:26:10 +01:00
Sema Checherinda
73a506b996
Update src/Interpreters/TransactionLog.cpp
Co-authored-by: Alexander Tokmakov <tavplubix@gmail.com>
2023-02-16 19:01:02 +01:00
Sema Checherinda
0511f5e635
Update src/Interpreters/TransactionLog.cpp
Co-authored-by: Alexander Tokmakov <tavplubix@gmail.com>
2023-02-16 19:00:55 +01:00
Sema Checherinda
bc661442c3
Update src/Interpreters/TransactionLog.cpp
Co-authored-by: Alexander Tokmakov <tavplubix@gmail.com>
2023-02-16 19:00:46 +01:00
Joanna Hulboj
5e32e20abb FIXUP: arrayPartialSort arg check and few tests 2023-02-16 17:59:12 +00:00
Joanna Hulboj
221e4942b2 FIXUP: check in FunctionArrayMapped refactor 2023-02-16 17:59:12 +00:00
Joanna Hulboj
4ee665b158 FIXUP: Missing ErrorCodes 2023-02-16 17:59:12 +00:00
Joanna Hulboj
7bb7ea6913 Added arrayPartialSort and arraySort functions 2023-02-16 17:59:12 +00:00
Joanna Hulboj
23d173a53c Refactor FunctionArrayMapped to allow for a fixed number of extra positional arguments 2023-02-16 17:59:12 +00:00
Nikolai Kochetov
9cca571777 Fix constants in the result of MergingSortedAlgorithm. 2023-02-16 17:58:35 +00:00
kssenii
417052e4b2 Fix remaining tests 2023-02-16 18:56:19 +01:00
Alexander Tokmakov
00f9a94ddb
Merge branch 'master' into apply_move_partition_to_all_replicas 2023-02-16 20:22:35 +03:00
Robert Schulze
0ff404da9c
Merge pull request #46302 from JakeBamrah/master
Expand date formatters and add `DATE_FORMAT` function for better MySQL compatibility
2023-02-16 17:54:12 +01:00
Azat Khuzhin
beff5ac795 Fix PREWHERE for Merge with different default types
In case of underlying table has an ALIAS for this column, while in Merge
table it is not marked as an alias, there will NOT_FOUND_COLUMN_IN_BLOCK
error.

Further more, when underlying tables has different default type for the
column, i.e. one has ALIAS and another has real column, then you will
also get NOT_FOUND_COLUMN_IN_BLOCK, because Merge engine should take
care of this.

Also this patch reworks how PREWHERE is handled for Merge table, and now
if you use PREWHERE on the column that has the same type and default
type (ALIAS, ...) then it will be possible, and only if the type
differs, it will be prohibited and throw ILLEGAL_PREWHERE error.

And last, but not least, also respect this restrictions for
optimize_move_to_prewhere.

v2: introduce IStorage::supportedPrewhereColumns()
v3: Remove excessive condition for PREWHERE in StorageMerge::read()
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-02-16 17:21:53 +01:00
kssenii
75d62ee24a Remove support of feature which does not work reliably 2023-02-16 16:41:31 +01:00
SmitaRKulkarni
f3f459fe34
Merge branch 'master' into 45185_Inner_join_with_where 2023-02-16 16:20:20 +01:00
Smita Kulkarni
6763a9d1ff Updated to not clear on_expression from table_join as its used by future analyze runs 2023-02-16 16:15:18 +01:00
Alexander Gololobov
a6061e022e Merge branch 'master' into split_prewhere_dag 2023-02-16 14:43:58 +01:00
flynn
eae73a1511 fix 2023-02-16 13:42:19 +00:00
Alexander Tokmakov
9feb448a5e fix a race condition 2023-02-16 14:41:18 +01:00
Alexander Gololobov
d0f402fa5b
Merge pull request #46391 from ClickHouse/debug-docker
Easy debugging of clickhouse-server in Docker
2023-02-16 14:09:43 +01:00
youennL-cs
6526c2a8ab
[RFC] Replacing merge tree new engine (#41005)
* Add new engine to ReplacingMergeTree corresponding to the ReplacingCollapsingMergeTree

* Add new test for the new ReplacingMergeTree engine

* Limit sign value to -1/1

* Add new engine to ReplacingMergeTree corresponding to the ReplacingCollapsingMergeTree

* Add new test for the new ReplacingMergeTree engine

* Limit sign value to -1/1

* Replace sign column(Int8) by is_deleted(UInt8)

* Add new engine to ReplacingMergeTree corresponding to the ReplacingCollapsingMergeTree

* Add new test for the new ReplacingMergeTree engine

* Limit sign value to -1/1

* Replace sign column(Int8) by is_deleted(UInt8)

* Add new engine to ReplacingMergeTree corresponding to the ReplacingCollapsingMergeTree

* Add new test for the new ReplacingMergeTree engine

* Limit sign value to -1/1

* Replace sign column(Int8) by is_deleted(UInt8)

* Add keyword 'CLEANUP' when OPTIMIZE

* Cleanup uniquely when it's a replacingMergeTree

* Propagate CLEANUP information and change from 'with_cleanup' to 'cleanup'

* Cleanup data flagged as 'is_deleted'

* Fix merge when optimize and add a test

* Fix OPTIMIZE and INSERT + add tests

* New fix for cleanup at the merge

* Cleanup debug logs

* Add the SETTINGS option 'clean_deleted_rows' that can be 'never' or 'always'

* Fix regression bug; Now REplicatedMergeTree can be called as before without 'is_deleted'

* Add Replicated tests

* Disable tag 'long' for our test and cleanup some white spaces

* Update tests

* Fix tests and remove additional useless whitespace

* Fix replica test

* Style clean && add condition check for is_deleted values

* clean_deleted_rows settings is nom an enum

* Add valid default value to the clean_deleted_rows settings

* Update cleanup checkers to use the enum and fix typos in the test

* Fix submodule contrib/AMQP-CPP pointer

* Add missing messages in test reference and remove a print with non derterministic order

* fix replica test reference

* Fix edge case

* Fix a typo for the spell checker

* Fix reference

* Fix a condition to raise an error if is_deleted differ from 0/1 and cleanup

* Change tests file name and update number

* This should fix the ReplacingMergeTree parameter set

* Fix replicated parameters

* Disable allow_deprecated_syntax_for_merge_tree for our new column

* Fix a test

* Remove non deterministic order print in the test

* Test on replicas

* Remove a condition, when checking optional parameters, that should not be sueful since we disabled the deprected_syntaxe

* Revert "Remove a condition, when checking optional parameters, that should not be useful since we disabled the deprected_syntaxe"

This reverts commit b65d64c05e.

* Fix replica management and limit the number of argument to two maximum, due to the possiblity of deprecated table create/attach failing otherwise

* Test a fix for replicated log information error

* Try to add sync to have consistent results

* Change path of replicas that should cause one issue and add few prints in case it's not that

* Get cleanup info on replicas only if information found

* Fix style issues

* Try to avoid replication error 'cannot select parts...' and and replica read/write field order

* Cleanup according to PR reviews
 and add tests on error raised.

* Update src/Storages/MergeTree/registerStorageMergeTree.cpp

Co-authored-by: Alexander Tokmakov <tavplubix@gmail.com>

* Select ... FINAL don't show rows with is_deleted = true

* Update and fix SELECT ... FINAL merge parameter

* Remove is_deleted rows only on the version inserted when merge

* Fix (master) updates issues

* Revert changes that should not be commited

* Add changes according to review

* Revert changes that should not be commited - part 2

---------

Co-authored-by: Alexander Tokmakov <tavplubix@gmail.com>
2023-02-16 16:03:16 +03:00
flynn
7f4c23ec8a fix 2023-02-16 12:48:22 +00:00
Maksim Kita
05baf271f0 Analyzer fix table functions with invalid arguments analysis 2023-02-16 12:17:04 +01:00
Maksim Kita
57b5d9852f Fixed build 2023-02-16 12:17:03 +01:00
Maksim Kita
b72ea982b0 Remove unnecessary includes of InterpreterSelectQuery header 2023-02-16 12:17:03 +01:00
Maksim Kita
4ddf1b0f48 Planner filter push down optimization fix 2023-02-16 12:17:03 +01:00
Maksim Kita
6b2adc1ec2 Analyzer storage Merge fixes 2023-02-16 12:17:03 +01:00
Maksim Kita
b1ab2af7ad Analyzer support storage Merge 2023-02-16 12:17:03 +01:00
Maksim Kita
469ad2c51b Fixed build 2023-02-16 12:17:03 +01:00
Maksim Kita
80af0666ea Analyzer support Materialized View 2023-02-16 12:17:03 +01:00
Maksim Kita
be9406ec0e Fixed tests 2023-02-16 12:17:03 +01:00
Maksim Kita
b2cc71f413 Fixed tests 2023-02-16 12:17:03 +01:00
Maksim Kita
c29f3c0f99 Analyzer fix GROUP BY columns validation 2023-02-16 12:17:03 +01:00
Maksim Kita
84065fb13f Analyzer added distributed table functions support 2023-02-16 12:17:03 +01:00
Maksim Kita
9b7cd64093 Analyzer join_use_nulls fix 2023-02-16 12:17:03 +01:00
Maksim Kita
0cba5848ac Fixed tests 2023-02-16 12:17:03 +01:00
Maksim Kita
c0e513efeb Fixed style check 2023-02-16 12:17:03 +01:00
Maksim Kita
afb3eb3c31 Planner JOIN planning fix 2023-02-16 12:17:03 +01:00
Maksim Kita
a090a8449d Analyzer distributed read fix 2023-02-16 12:17:03 +01:00
Maksim Kita
3a624e6bef Analyzer ColumnNode AST conversion fix 2023-02-16 12:17:03 +01:00
Maksim Kita
f21486c95b Analyzer validate that there are no window functions in GROUP BY 2023-02-16 12:17:03 +01:00
Maksim Kita
af38660cf5 Analyzer support ARRAY JOIN with Nested 2023-02-16 12:17:03 +01:00
Maksim Kita
077555bd5d Analyzer support build pushing to views chain 2023-02-16 12:17:03 +01:00
Maksim Kita
f73c49b082 Fixed style check 2023-02-16 12:17:03 +01:00
Maksim Kita
f8442b2a8d Analyzer support LiveView 2023-02-16 12:17:03 +01:00
Maksim Kita
5cc56eb108 Analyzer better support for single_join_prefer_left_table setting 2023-02-16 12:17:03 +01:00
Maksim Kita
7e58e23d3a Analyzer subquery resolve fix 2023-02-16 12:17:03 +01:00
Maksim Kita
70eb41c6a8 Analyzer add single_join_prefer_left_table setting 2023-02-16 12:17:03 +01:00
Maksim Kita
caaff8f387 GroupingFunctionsResolvePass crash fix 2023-02-16 12:17:03 +01:00
Maksim Kita
387f035560 QueryTreePassManager add ComparisonTupleEliminationPass 2023-02-16 12:17:03 +01:00
Maksim Kita
a12aea45f6 Analyzer matcher resolve fix 2023-02-16 12:17:02 +01:00
Maksim Kita
8497eba49f Fixed tests 2023-02-16 12:17:02 +01:00
Maksim Kita
cd3e8482ae Analyzer TableFunctionExplain fix 2023-02-16 12:17:02 +01:00
Maksim Kita
2830877e22 Analyzer support table functions 2023-02-16 12:17:02 +01:00
Maksim Kita
8374133123 Analyzer evaluate scalar subquery fix 2023-02-16 12:17:02 +01:00
Maksim Kita
484fc40e64 Analyzer identifier resolution for subcolumns fix 2023-02-16 12:17:02 +01:00
Maksim Kita
25da9dcef7 StorageDistributed Planner initialization fix 2023-02-16 12:17:02 +01:00
Maksim Kita
d3b604de5a FunctionToSubcolumnsPass mapContains map argument result type fix 2023-02-16 12:17:02 +01:00
Maksim Kita
c469e10092 Dictionaries DictionaryStorageFetchRequest fix 2023-02-16 12:17:02 +01:00
Maksim Kita
5b7fb9c458 Analyzer support enable_global_with_statement setting 2023-02-16 12:17:02 +01:00
Maksim Kita
d703c65dbd Analyzer lambda resolve unknown identifier crash fix 2023-02-16 12:17:02 +01:00
Maksim Kita
75876200af Planner disable names projection on shards 2023-02-16 12:17:02 +01:00
Maksim Kita
480619af63 Analyzer ARRAY JOIN disable alias validation for subquery or table function 2023-02-16 12:17:02 +01:00
Maksim Kita
b9cb067153 Analyzer table function arguments resolution fix 2023-02-16 12:17:02 +01:00
Maksim Kita
86ee97aed5 AggregateFunctionsArithmeticOperationsPass disable optimization if both arguments are non constant 2023-02-16 12:17:02 +01:00
Maksim Kita
301d9e22c2 Analyzer change countDistinct function implementation using count_distinct_implementation setting 2023-02-16 12:17:02 +01:00
Maksim Kita
75cbb0e0d0 Analyzer scalar subqueries execution fix 2023-02-16 12:17:02 +01:00
Maksim Kita
fa0ef340b6 Analyzer IN function invalid arguments size error fix 2023-02-16 12:17:02 +01:00
Maksim Kita
0fe3ba36c4 Analyzer added ParitionPruner support 2023-02-16 12:17:02 +01:00
Maksim Kita
45f45b290c Analyzer recursive SQL user defined functions fix 2023-02-16 12:17:02 +01:00
Maksim Kita
9b7710ba40 Analyzer count function handling with unqualified matcher fix 2023-02-16 12:17:02 +01:00
Maksim Kita
8eb93b216b Planner JOIN TREE unused columns drop fix 2023-02-16 12:17:02 +01:00
Maksim Kita
84f3f8e18b Planner UNION different number of columns exception fix 2023-02-16 12:17:02 +01:00
Maksim Kita
bb56af0e2b MergeTreeIndexGin crash fix 2023-02-16 12:17:02 +01:00
Maksim Kita
c7c5e29ff7 Analyzer resolve identifier as table expression crash fix 2023-02-16 12:17:02 +01:00
Maksim Kita
cbcfd404c8 AggregateFunctionSequenceNextNode fix argument types 2023-02-16 12:17:02 +01:00
Maksim Kita
7ffee1dafe QueryTreePassManager disable validation for aggregate functions with LowCardinality arguments 2023-02-16 12:17:02 +01:00
Maksim Kita
945ce7ca63 Planner support indexHint function 2023-02-16 12:17:02 +01:00
Maksim Kita
2aef331846 Analyzer JOIN disable USING for non identifier nodes 2023-02-16 12:17:02 +01:00
Maksim Kita
440f2f5834 Function fromModifiedJulianDay argument types fix 2023-02-16 12:17:02 +01:00
Maksim Kita
03f14f7c50 Analyzer fix MutationsInterpreter 2023-02-16 12:17:02 +01:00
Maksim Kita
c4cb3bb03f Analyzer fix combinators with NULL argument 2023-02-16 12:17:02 +01:00
Maksim Kita
77fee97705 MergeTreeIndexFullText fix prepared set index analysis 2023-02-16 12:17:02 +01:00
Maksim Kita
96fb630231 Analyzer Planner enable by default 2023-02-16 12:17:01 +01:00
Alexey Milovidov
a9b4f2ef3a
Merge pull request #46435 from ClickHouse/rs/consistent-db-metric
Exclude internal databases from async metric "NumberOfDatabases"
2023-02-16 14:09:59 +03:00
Kseniia Sumarokova
187f7482e1
Merge pull request #46044 from ucasfl/log-engine
Storage Log family support settings storage_policy
2023-02-16 12:09:47 +01:00
Nikolay Degterinsky
d9ffbeb4a2 Small improvements 2023-02-16 10:51:13 +00:00
Nikolay Degterinsky
d8c5c7c503
Merge branch 'master' into parameters-rename-query 2023-02-16 11:30:20 +01:00
Kseniia Sumarokova
4303ee9114
Merge pull request #46404 from kssenii/fix-rabbit-test-4
Fix test test_rabbitmq_overloaded_insert
2023-02-16 11:28:18 +01:00
Kseniia Sumarokova
947753bd4c
Merge pull request #46409 from kssenii/follow-up-to-46168
Follow-up to #46168
2023-02-16 11:26:54 +01:00
Nikolay Degterinsky
e61947bf47
Merge pull request #44662 from evillique/ulid
Add generateULID function
2023-02-16 11:21:23 +01:00
flynn
2968cdc8f6 fix 2023-02-16 10:18:22 +00:00
Jake Bamrah
4ea814a843 add DATE_FORMAT alias and remaining mySQL date formatters 2023-02-16 09:52:19 +00:00
Nikolay Degterinsky
ebec5b8050
Merge pull request #46422 from evillique/move-sources
Move MongoDB and PostgreSQL sources to Sources folder
2023-02-16 10:51:45 +01:00
taiyang-li
4a7dd63f41 add new function regexp_extract, like in spark 2023-02-16 17:12:58 +08:00
Smita Kulkarni
0506d9289c Updated Backup/Restore Coordination construction and removed coordination_path and added uuid in settings - Use cluster state data to check concurrent backup/restore 2023-02-16 09:30:27 +01:00
Sergei Trifonov
3ad41b29ec
Merge pull request #46247 from ClickHouse/round-robin-merge-scheduler 2023-02-16 08:03:22 +01:00
flynn
ecc39978d7 fix conflict 2023-02-16 02:23:55 +00:00
Dmitry Novik
659e680c76 Add a comment 2023-02-16 01:24:31 +00:00
Alexander Tokmakov
53e32b19e6
Update AggregatingTransform.cpp 2023-02-16 02:13:18 +03:00
Alexander Gololobov
cdf3828575
Removed unneeded default value for enable_multiple_prewhere_read_steps 2023-02-15 22:41:55 +01:00
Alexander Gololobov
6936471150
fix include
Co-authored-by: Nikita Taranov <nikita.taranov@clickhouse.com>
2023-02-15 22:37:40 +01:00
Alexander Gololobov
8b528739c1
fix typo
Co-authored-by: Nikita Taranov <nikita.taranov@clickhouse.com>
2023-02-15 22:37:18 +01:00
Alexander Gololobov
3ff0c7edcc
fix typo
Co-authored-by: Nikita Taranov <nikita.taranov@clickhouse.com>
2023-02-15 22:37:04 +01:00
Igor Nikonov
cb89fccac6 Disable debug output 2023-02-15 21:26:00 +00:00
Igor Nikonov
766f04b04c Merge remote-tracking branch 'origin/master' into igor/remove_redundant_distinct 2023-02-15 21:24:25 +00:00
Antonio Andelic
17494d79c2
Merge pull request #46070 from ClickHouse/s3-client-polish
Polish S3 client
2023-02-15 21:22:47 +01:00
Sema Checherinda
dd1b509637 resolve race in getCSNAndAssert 2023-02-15 18:16:39 +01:00
Han Fei
5458d4216f
Merge branch 'master' into dev-prewhere 2023-02-15 17:59:44 +01:00
Alexander Gololobov
a525075614
Merge branch 'master' into debug-docker 2023-02-15 16:50:25 +01:00
Dmitry Novik
cfb832b205 Wait for tasks finish when scheduler failed 2023-02-15 15:50:06 +00:00
Alexander Tokmakov
885f14311c fix 2023-02-15 16:37:44 +01:00
Vladimir C
3e77463125
Merge pull request #46256 from bigo-sg/window_functions 2023-02-15 15:36:41 +01:00
Kruglov Pavel
4f380370a9
Fix s3Cluster schema inference in parallel distributed insert select (#46381)
* Fix s3Cluster schema inference in parallel distributed insert select
* Try fix flaky test
* Try SYSTEM SYNC REPLICA to avoid test flakiness
2023-02-15 15:30:43 +01:00
Maksim Kita
88c32002e5
Merge pull request #46306 from ClickHouse/analyzer-limits
Analyzer - storage limits - propagate limits into subquery
2023-02-15 17:15:33 +03:00
Kruglov Pavel
9fd2226c4c
Update NativeReader.h 2023-02-15 15:13:04 +01:00
flynn
de11a05d89 remove redundant header 2023-02-15 13:27:25 +00:00
Alexey Milovidov
c2660646fe
Merge branch 'master' into debug-docker 2023-02-15 15:59:49 +03:00
Robert Schulze
44815240f5
Exclude internal databases from async metric "NumberOfDatabases" 2023-02-15 12:58:00 +00:00
Alexey Milovidov
77f60311da
Merge branch 'master' into add_select_final_mt_setting 2023-02-15 15:35:37 +03:00
vdimir
0d6c6a94de
finally fix attachProfileCountersScope 2023-02-15 12:18:17 +00:00
flynn
289c5c60d3 fix 2023-02-15 11:24:06 +00:00
Vladimir C
645fee3d94
Merge pull request #46009 from ClickHouse/vdimir/fix_read_in_order_final_desc 2023-02-15 11:23:07 +01:00
kssenii
f75d69a954 Fix 2023-02-15 11:14:50 +01:00
Nikolay Degterinsky
24bbf885db
Merge branch 'master' into parameters-rename-query 2023-02-15 10:53:44 +01:00
Robert Schulze
ec33204c05
Merge pull request #46344 from ClickHouse/rs/stabilize-full_text_seaerch_test
Try to stabilize test 02346_full_text_search.sql
2023-02-15 10:47:00 +01:00
Antonio Andelic
f76ef72ac0
Merge branch 'master' into s3-client-polish 2023-02-15 09:20:06 +01:00
Nikolay Degterinsky
bcc4adf73d
Merge branch 'master' into move-sources 2023-02-15 08:46:09 +01:00