Commit Graph

9125 Commits

Author SHA1 Message Date
Maksim Kita
6dffe52e68
Merge pull request #46471 from kitaisreal/analyzer-planner-fixes-before-enable-by-default
Analyzer planner fixes before enable by default
2023-02-19 13:49:18 +03:00
Maksim Kita
cbd961de98 Fixed code review issues 2023-02-18 17:06:00 +01:00
Nikolay Degterinsky
f93aae80b8
Merge pull request #46407 from evillique/parameters-rename-query
Add parameterized RENAME queries
2023-02-18 16:04:55 +01:00
Alexey Milovidov
f073cd470f
Merge pull request #46478 from ClickHouse/fix_test_store_cleanup
Fix some flaky integration tests
2023-02-18 17:33:02 +03:00
Nikita Taranov
f70044f34b Merge branch 'master' into aggr_partitions_independently 2023-02-18 13:19:05 +00:00
Han Fei
b76df1401e enable async-insert-max-query-number only if async_insert_deduplicate is true 2023-02-18 01:08:28 +01:00
Alexander Tokmakov
53b6fd79fb
Update DatabaseCatalog.cpp 2023-02-17 21:18:49 +03:00
Roman Vasin
3e2b56fabc
Merge branch 'master' into ADQM-639 2023-02-17 21:14:26 +03:00
Alexander Tokmakov
3457b68dcc fix 2023-02-17 17:57:49 +01:00
Nikolay Degterinsky
5104b2e8bf
Merge branch 'master' into parameters-rename-query 2023-02-17 17:30:37 +01:00
Alexey Milovidov
45b1eabd36
Merge pull request #46420 from evillique/replace-transformer-fix-parameters
Fix parameterized queries with REPLACE transformer
2023-02-17 18:25:57 +03:00
Antonio Andelic
ab51c1d975
Merge branch 'master' into custom-key-parallel-replicas 2023-02-17 15:06:41 +01:00
Vladimir C
0f182f6164
Merge pull request #46454 from azat/merge-alias-prewhere 2023-02-17 14:10:52 +01:00
Alexander Tokmakov
fb3574d2cf Merge branch 'master' into fix_test_store_cleanup 2023-02-17 13:59:22 +01:00
kssenii
d051ee9a1e Fix build 2023-02-17 13:46:30 +01:00
Alexander Tokmakov
9a664efeb6
Merge branch 'master' into fixes_for_993 2023-02-17 14:17:08 +03:00
Sema Checherinda
b551b9fb3a
Merge pull request #46452 from CheSema/resolve-race-get-csn
resolve race in getCSNAndAssert
2023-02-17 11:31:58 +01:00
Roman Vasin
fde8e7e858 Add try/catch in TemporaryTableHolder destructor 2023-02-17 07:47:23 +00:00
Nikolay Degterinsky
3b19823960
Merge branch 'master' into replace-transformer-fix-parameters 2023-02-17 02:02:54 +01:00
kssenii
0bceb5957a Fix race 2023-02-16 21:23:03 +01:00
Azat Khuzhin
5af2967f65 Revert code in TreeRewriter for proper column order for UNION
It is not required after #44920, which has a proper fix that does this
in ExpressionAnalyzer.

This reverts commit d98336ad83.
Reverts: #37887

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-02-16 20:46:39 +01:00
Alexey Milovidov
993a39e2a1
Merge pull request #46360 from genzgd/http_client_version
Allow to provide the protocol version by HTTP clients to tune the compatibility of the Native output format.
2023-02-16 22:17:04 +03:00
Sema Checherinda
73a506b996
Update src/Interpreters/TransactionLog.cpp
Co-authored-by: Alexander Tokmakov <tavplubix@gmail.com>
2023-02-16 19:01:02 +01:00
Sema Checherinda
0511f5e635
Update src/Interpreters/TransactionLog.cpp
Co-authored-by: Alexander Tokmakov <tavplubix@gmail.com>
2023-02-16 19:00:55 +01:00
Sema Checherinda
bc661442c3
Update src/Interpreters/TransactionLog.cpp
Co-authored-by: Alexander Tokmakov <tavplubix@gmail.com>
2023-02-16 19:00:46 +01:00
Alexander Tokmakov
00f9a94ddb
Merge branch 'master' into apply_move_partition_to_all_replicas 2023-02-16 20:22:35 +03:00
Azat Khuzhin
beff5ac795 Fix PREWHERE for Merge with different default types
In case of underlying table has an ALIAS for this column, while in Merge
table it is not marked as an alias, there will NOT_FOUND_COLUMN_IN_BLOCK
error.

Further more, when underlying tables has different default type for the
column, i.e. one has ALIAS and another has real column, then you will
also get NOT_FOUND_COLUMN_IN_BLOCK, because Merge engine should take
care of this.

Also this patch reworks how PREWHERE is handled for Merge table, and now
if you use PREWHERE on the column that has the same type and default
type (ALIAS, ...) then it will be possible, and only if the type
differs, it will be prohibited and throw ILLEGAL_PREWHERE error.

And last, but not least, also respect this restrictions for
optimize_move_to_prewhere.

v2: introduce IStorage::supportedPrewhereColumns()
v3: Remove excessive condition for PREWHERE in StorageMerge::read()
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-02-16 17:21:53 +01:00
SmitaRKulkarni
f3f459fe34
Merge branch 'master' into 45185_Inner_join_with_where 2023-02-16 16:20:20 +01:00
Smita Kulkarni
6763a9d1ff Updated to not clear on_expression from table_join as its used by future analyze runs 2023-02-16 16:15:18 +01:00
kssenii
730ac4de93 Try get rid of mutex aroung reserved_size 2023-02-16 14:46:16 +01:00
Alexander Gololobov
a6061e022e Merge branch 'master' into split_prewhere_dag 2023-02-16 14:43:58 +01:00
Alexander Tokmakov
9feb448a5e fix a race condition 2023-02-16 14:41:18 +01:00
kssenii
d14f86cfe8 Get rid of download_mutex 2023-02-16 14:33:23 +01:00
kssenii
e182f163f8 Review fixes 2023-02-16 14:19:30 +01:00
youennL-cs
6526c2a8ab
[RFC] Replacing merge tree new engine (#41005)
* Add new engine to ReplacingMergeTree corresponding to the ReplacingCollapsingMergeTree

* Add new test for the new ReplacingMergeTree engine

* Limit sign value to -1/1

* Add new engine to ReplacingMergeTree corresponding to the ReplacingCollapsingMergeTree

* Add new test for the new ReplacingMergeTree engine

* Limit sign value to -1/1

* Replace sign column(Int8) by is_deleted(UInt8)

* Add new engine to ReplacingMergeTree corresponding to the ReplacingCollapsingMergeTree

* Add new test for the new ReplacingMergeTree engine

* Limit sign value to -1/1

* Replace sign column(Int8) by is_deleted(UInt8)

* Add new engine to ReplacingMergeTree corresponding to the ReplacingCollapsingMergeTree

* Add new test for the new ReplacingMergeTree engine

* Limit sign value to -1/1

* Replace sign column(Int8) by is_deleted(UInt8)

* Add keyword 'CLEANUP' when OPTIMIZE

* Cleanup uniquely when it's a replacingMergeTree

* Propagate CLEANUP information and change from 'with_cleanup' to 'cleanup'

* Cleanup data flagged as 'is_deleted'

* Fix merge when optimize and add a test

* Fix OPTIMIZE and INSERT + add tests

* New fix for cleanup at the merge

* Cleanup debug logs

* Add the SETTINGS option 'clean_deleted_rows' that can be 'never' or 'always'

* Fix regression bug; Now REplicatedMergeTree can be called as before without 'is_deleted'

* Add Replicated tests

* Disable tag 'long' for our test and cleanup some white spaces

* Update tests

* Fix tests and remove additional useless whitespace

* Fix replica test

* Style clean && add condition check for is_deleted values

* clean_deleted_rows settings is nom an enum

* Add valid default value to the clean_deleted_rows settings

* Update cleanup checkers to use the enum and fix typos in the test

* Fix submodule contrib/AMQP-CPP pointer

* Add missing messages in test reference and remove a print with non derterministic order

* fix replica test reference

* Fix edge case

* Fix a typo for the spell checker

* Fix reference

* Fix a condition to raise an error if is_deleted differ from 0/1 and cleanup

* Change tests file name and update number

* This should fix the ReplacingMergeTree parameter set

* Fix replicated parameters

* Disable allow_deprecated_syntax_for_merge_tree for our new column

* Fix a test

* Remove non deterministic order print in the test

* Test on replicas

* Remove a condition, when checking optional parameters, that should not be sueful since we disabled the deprected_syntaxe

* Revert "Remove a condition, when checking optional parameters, that should not be useful since we disabled the deprected_syntaxe"

This reverts commit b65d64c05e.

* Fix replica management and limit the number of argument to two maximum, due to the possiblity of deprecated table create/attach failing otherwise

* Test a fix for replicated log information error

* Try to add sync to have consistent results

* Change path of replicas that should cause one issue and add few prints in case it's not that

* Get cleanup info on replicas only if information found

* Fix style issues

* Try to avoid replication error 'cannot select parts...' and and replica read/write field order

* Cleanup according to PR reviews
 and add tests on error raised.

* Update src/Storages/MergeTree/registerStorageMergeTree.cpp

Co-authored-by: Alexander Tokmakov <tavplubix@gmail.com>

* Select ... FINAL don't show rows with is_deleted = true

* Update and fix SELECT ... FINAL merge parameter

* Remove is_deleted rows only on the version inserted when merge

* Fix (master) updates issues

* Revert changes that should not be commited

* Add changes according to review

* Revert changes that should not be commited - part 2

---------

Co-authored-by: Alexander Tokmakov <tavplubix@gmail.com>
2023-02-16 16:03:16 +03:00
kssenii
b6c3694369 Release the lock earlier 2023-02-16 13:49:12 +01:00
kssenii
bf1174d5af Cleanup: fix variables names 2023-02-16 13:26:08 +01:00
kssenii
dbbab26167 Merge remote-tracking branch 'upstream/master' into cache-better-locks 2023-02-16 12:29:38 +01:00
Maksim Kita
b72ea982b0 Remove unnecessary includes of InterpreterSelectQuery header 2023-02-16 12:17:03 +01:00
Maksim Kita
6b2adc1ec2 Analyzer storage Merge fixes 2023-02-16 12:17:03 +01:00
Maksim Kita
b1ab2af7ad Analyzer support storage Merge 2023-02-16 12:17:03 +01:00
Maksim Kita
80af0666ea Analyzer support Materialized View 2023-02-16 12:17:03 +01:00
Maksim Kita
b2cc71f413 Fixed tests 2023-02-16 12:17:03 +01:00
Maksim Kita
84065fb13f Analyzer added distributed table functions support 2023-02-16 12:17:03 +01:00
Maksim Kita
f8442b2a8d Analyzer support LiveView 2023-02-16 12:17:03 +01:00
Maksim Kita
70eb41c6a8 Analyzer add single_join_prefer_left_table setting 2023-02-16 12:17:03 +01:00
Maksim Kita
2830877e22 Analyzer support table functions 2023-02-16 12:17:02 +01:00
Maksim Kita
03f14f7c50 Analyzer fix MutationsInterpreter 2023-02-16 12:17:02 +01:00
Alexey Milovidov
a9b4f2ef3a
Merge pull request #46435 from ClickHouse/rs/consistent-db-metric
Exclude internal databases from async metric "NumberOfDatabases"
2023-02-16 14:09:59 +03:00
Nikolay Degterinsky
d9ffbeb4a2 Small improvements 2023-02-16 10:51:13 +00:00
Nikolay Degterinsky
d8c5c7c503
Merge branch 'master' into parameters-rename-query 2023-02-16 11:30:20 +01:00
Anton Popov
3730ea388f fix issues with sparse columns 2023-02-15 21:46:26 +00:00
Sema Checherinda
dd1b509637 resolve race in getCSNAndAssert 2023-02-15 18:16:39 +01:00
Dmitry Novik
0e5975c50d
Merge branch 'master' into analyzer-group-by-use-nulls 2023-02-15 17:01:30 +01:00
Robert Schulze
44815240f5
Exclude internal databases from async metric "NumberOfDatabases" 2023-02-15 12:58:00 +00:00
Alexey Milovidov
77f60311da
Merge branch 'master' into add_select_final_mt_setting 2023-02-15 15:35:37 +03:00
vdimir
0d6c6a94de
finally fix attachProfileCountersScope 2023-02-15 12:18:17 +00:00
Nikolay Degterinsky
24bbf885db
Merge branch 'master' into parameters-rename-query 2023-02-15 10:53:44 +01:00
Nikolay Degterinsky
eb3a7f55b9 Fix parameterized queries with REPLACE transformer 2023-02-14 22:02:54 +00:00
Sergei Trifonov
b7a4fa8bd9
Merge branch 'master' into round-robin-merge-scheduler 2023-02-14 20:39:02 +01:00
serxa
a7bbf02bac fix possible deadlock 2023-02-14 19:37:09 +00:00
Roman Vasin
f410834bfb Remove dropTable of temp table from InterpreterDropQuery. 2023-02-14 17:03:36 +00:00
Geoff Genz
f3bcf26959
Merge branch 'ClickHouse:master' into http_client_version 2023-02-14 09:31:29 -07:00
Alexander Gololobov
7ee94d6a90 Fix for 'Result column is not empty' bug found by fuzzer 2023-02-14 15:34:04 +01:00
vdimir
1b715a3388
Allow scope override in ThreadStatus::attachProfileCountersScope 2023-02-14 14:11:13 +00:00
vdimir
abd9119583
Remove unnecessary ProfileEventsScope 2023-02-14 14:11:12 +00:00
vdimir
83dea52aff
add sanity check to ThreadStatus::attachProfileCountersScope 2023-02-14 14:11:12 +00:00
vdimir
4c1aeb7972
Use overload of PartLog::addNewPart with profile_events 2023-02-14 14:11:12 +00:00
vdimir
7a66c0105a
Remove unused ProfileEventsExt::dumpToString 2023-02-14 14:11:11 +00:00
vdimir
fa08f72bba
Move subthread_profile_events to ThreadStatus 2023-02-14 14:11:11 +00:00
vdimir
a228f7f419
[wip] ProfileCounters for each part 2023-02-14 14:11:09 +00:00
Alexey Milovidov
f554ff9d99
Remove trash 2023-02-14 14:11:08 +00:00
John Skopis
f7604cc686
[feat] Add ProfileEvents map to PartLog
closes #10316
2023-02-14 14:11:02 +00:00
Roman Vasin
b0ced21c30 Move temp database clean up into new function removeDataPath() 2023-02-14 13:22:15 +00:00
Antonio Andelic
adde580756
Merge branch 'master' into custom-key-parallel-replicas 2023-02-14 14:09:12 +01:00
Nikita Taranov
cbd10c4493
Fix mbm with aio on top of query plan (#45892) 2023-02-14 14:02:54 +01:00
Nikolay Degterinsky
b9d3160107 Add parameterized RENAME queries 2023-02-14 13:01:06 +00:00
Roman Vasin
42f2c537e2 Remove setting database for create query 2023-02-14 12:10:44 +00:00
lzydmxy
33aff3f34c Merge branch 'master' into apply_move_partition_to_all_replicas 2023-02-14 18:07:58 +08:00
lzydmxy
bbf94a2664 Apply ALTER TABLE table_name ON CLUSTER cluster MOVE PARTITION|PART partition_expr TO DISK|VOLUME 'disk_name' to all replicas.
Because `ALTER TABLE t MOVE` is not replicated.
2023-02-14 17:29:44 +08:00
Alexey Milovidov
d102c4eafe
Update Context.h 2023-02-14 11:12:40 +03:00
Alexander Tokmakov
e712fbecb2 fix race between drop and create 2023-02-14 00:45:28 +01:00
Anton Popov
e9e6d735ef add query_id header in all queries 2023-02-13 16:44:54 +00:00
Roman Vasin
769ff6d273 Merge branch 'ADQM-639' of github.com:arenadata/ClickHouse into ADQM-639 2023-02-13 16:23:15 +00:00
Roman Vasin
8cc768fde2 Add KeeperMap as disallowed engine for temporary tables 2023-02-13 16:23:04 +00:00
Geoff Genz
be8bf3a6a3
Merge branch 'master' into http_client_version 2023-02-13 08:43:59 -07:00
Alexander Gololobov
39e4ca9850 Cleanup 2023-02-13 14:40:54 +01:00
Vladimir C
1a89ecd386 Apply suggestions from code review
Co-authored-by: Nikita Taranov <nikita.taranov@clickhouse.com>
2023-02-13 12:17:56 +01:00
vdimir
e4c0afcc3e Do not build filtering set for sorting join with const columns 2023-02-13 12:17:56 +01:00
Alexander Gololobov
45ef2f6d60 Split prewhere actions into separate conjuctive steps 2023-02-12 21:54:23 +01:00
Alexey Milovidov
5f00bc49f8
Merge branch 'master' into addParts 2023-02-12 05:55:18 +03:00
Alexey Milovidov
6e0dab71ed
Merge pull request #46188 from bigo-sg/rewrite_array_exists
Rewrite array exists to has
2023-02-12 05:53:22 +03:00
Sergei Trifonov
297587531b
Merge branch 'master' into round-robin-merge-scheduler 2023-02-11 17:19:27 +01:00
serxa
c58b165b0f add config option to select scheduling policy 2023-02-11 16:18:42 +00:00
kssenii
3067c1d723 Merge remote-tracking branch 'upstream/master' into resubmit-prefetches 2023-02-11 11:36:23 +01:00
Sergei Trifonov
fbedf4d520
Merge branch 'master' into round-robin-merge-scheduler 2023-02-11 10:30:26 +01:00
Kseniia Sumarokova
3be36c7979
Merge pull request #41976 from kssenii/allow-single-disk-instead-of-storage-policy
Allow configuring storage as `SETTINGS disk='<disk_name>'` (instead of `storage_policy`) and explicit disk creation `SETTINGS disk=disk(<disk_configuration>)`
2023-02-10 21:38:50 +01:00
Alexey Milovidov
d5d87bc6b7
Merge pull request #46152 from ClickHouse/with-fill-bigint
More convenient usage of big integers and ORDER BY WITH FILL.
2023-02-10 20:05:32 +03:00
Roman Vasin
bb465ce7dd
Merge branch 'master' into ADQM-639 2023-02-10 17:04:14 +03:00
Alexander Tokmakov
8eae498ec6
Merge pull request #45936 from vitlibar/add-setting-check-referential-deps-on-drop
Check referential dependencies on DROP TABLE
2023-02-10 16:21:00 +03:00
Geoff Genz
99c3ff53c5 Merge remote-tracking branch 'origin/master' into http_client_version
# Conflicts:
#	src/Interpreters/Context.cpp
#	src/Interpreters/Context.h
2023-02-10 04:35:53 -07:00
Geoff Genz
7ed8ed0284 Add support for client_protocol_version sent with HTTP 2023-02-10 03:47:06 -07:00
KitKatKKK
ea300ce13b
Merge branch 'master' into addParts 2023-02-10 16:48:04 +08:00
attack204
93e92dab3b rename API 2023-02-10 16:05:50 +08:00
taiyang-li
5c1425b322 Merge branch 'rewrite_array_exists' of https://github.com/bigo-sg/ClickHouse into rewrite_array_exists 2023-02-10 11:32:56 +08:00
taiyang-li
58c348fecd fix build error 2023-02-10 11:27:10 +08:00
serxa
a4006ec5a1 add explanation 2023-02-09 19:49:11 +00:00
serxa
bf299b6696 Merge branch 'round-robin-merge-scheduler' of github.com:ClickHouse/ClickHouse into round-robin-merge-scheduler 2023-02-09 19:33:17 +00:00
serxa
cc3c76944b remove redundant code 2023-02-09 19:31:54 +00:00
Sergei Trifonov
e3727d9b0e
Merge branch 'master' into round-robin-merge-scheduler 2023-02-09 20:04:59 +01:00
serxa
f61f95fcec Replace background executor scheduler merges and mutations with round-robin 2023-02-09 18:54:15 +00:00
Robert Schulze
566fb5b9da
Merge pull request #46006 from ClickHouse/rs/fix-fragile-linking
Fix terribly broken, fragile and potentially cyclic linking
2023-02-09 18:19:44 +01:00
李扬
b9c942e2e7
Merge branch 'master' into rewrite_array_exists 2023-02-09 21:47:38 +08:00
Vitaly Baranov
39bacfb9dc
Merge branch 'master' into allow-single-disk-instead-of-storage-policy 2023-02-09 13:16:55 +01:00
avogar
01c42dd6f3 Fix 2023-02-09 12:03:56 +00:00
alesapin
859f528fe1
Merge pull request #45648 from ClickHouse/45508_Update_strategy_for_system_sync_replica
Updated checking of SYSTEM SYNC REPLICA
2023-02-09 11:51:05 +01:00
Robert Schulze
e490ec91d9
Merge branch 'master' into rs/fix-fragile-linking 2023-02-09 11:33:59 +01:00
taiyang-li
e1fed0352e remove useless code 2023-02-09 16:32:11 +08:00
taiyang-li
19ca0ec4af add passes for rewriting arrayexists 2023-02-09 16:30:53 +08:00
taiyang-li
af7a6abf80 fix bugs 2023-02-09 15:15:08 +08:00
taiyang-li
1ce12c8ab5 add visitor files 2023-02-09 12:26:32 +08:00
taiyang-li
48f0fb0d52 rewrite array exists to has 2023-02-09 12:06:42 +08:00
Alexey Milovidov
292a5abbc9 Merge remote-tracking branch 'origin/master' into with-fill-bigint 2023-02-09 01:47:51 +01:00
Vitaly Baranov
4869d3806c Add setting check_referential_table_dependencies to check referential dependencies on DROP TABLE. 2023-02-08 23:56:59 +01:00
kssenii
b0b865c32e Resubmit prefetches 2023-02-08 21:26:24 +01:00
avogar
85dd330374 Add test, fix low cardinality again 2023-02-08 19:24:04 +00:00
avogar
c3e8dd8984 Fix low cardinality case 2023-02-08 19:14:28 +00:00
avogar
738311c7fa Use default of columnt type in insert_null_as_default if column DEFAULT values is not specified 2023-02-08 15:48:18 +00:00
Alexander Gololobov
a7c30eecf4
Merge pull request #46141 from ClickHouse/fix-assertion-in-statistics
Fix assertion in statistical functions
2023-02-08 13:42:02 +01:00
Robert Schulze
6ff232d782
Merge branch 'master' into rs/fix-fragile-linking 2023-02-08 12:51:12 +01:00
Kruglov Pavel
8bdb1c3453
Merge pull request #44728 from bigo-sg/improve_count_if
Improve count if rewrite
2023-02-08 12:21:29 +01:00
Alexey Milovidov
a9ec73a209 Fix strange trash 2023-02-08 08:26:27 +01:00
Alexey Milovidov
55c3bbb739 Fix assertion in statistical functions 2023-02-08 00:09:41 +01:00
kssenii
5c201698f4 Fix changes in commit c4be868222 2023-02-07 16:50:55 +01:00
Roman Vasin
2b300e06d0 Fix setEngine() for temporary tables 2023-02-07 14:24:47 +00:00
Yakov Olkhovskiy
4e7dffafe4
Merge pull request #46087 from ClickHouse/fix-ip-bloom
Allow bloom filter for IPv4 and IPv6
2023-02-07 08:29:28 -05:00
Yakov Olkhovskiy
909a772701
Merge pull request #45375 from ClibMouse/window_view_fix
Window view functional test failed on platform s390x
2023-02-07 08:23:50 -05:00
Robert Schulze
08c1f8346e
Merge remote-tracking branch 'origin/master' into rs/fix-fragile-linking 2023-02-07 11:22:00 +00:00
Vitaly Baranov
f021c0eab6
Merge pull request #46058 from vitlibar/remove-unnecessary-execute-for-const-expr
Remove unnecessary execute() while evaluating a constant expression.
2023-02-07 12:09:37 +01:00
Roman Vasin
9488bd834f
Merge branch 'master' into ADQM-639 2023-02-07 09:45:10 +03:00
Yakov Olkhovskiy
e433c26cf9 allow bloom filter for IPv4 and IPv6 2023-02-06 16:30:18 +00:00
Suzy Wang
6edbffa3cc
Merge branch 'master' into window_view_fix 2023-02-06 10:06:04 -05:00
kssenii
ae9922abe5 Fix clang tidy 2023-02-06 13:17:21 +01:00
kssenii
fb348f8222 Merge remote-tracking branch 'origin/cache-better-locks' into cache-better-locks 2023-02-06 13:15:26 +01:00
kssenii
c4be868222 Fix delayed cleanup 2023-02-06 13:15:02 +01:00
alesapin
b94f9f8006
Merge branch 'master' into 45508_Update_strategy_for_system_sync_replica 2023-02-06 12:38:39 +01:00
Antonio Andelic
f67e5505ab Merge branch 'master' into custom-key-parallel-replicas 2023-02-06 11:12:39 +00:00
Robert Schulze
84b9ff450f
Fix terribly broken, fragile and potentially cyclic linking
Sorry for the clickbaity title. This is about static method
ConnectionTimeouts::getHTTPTimeouts(). It was be declared in header
IO/ConnectionTimeouts.h, and defined in header
IO/ConnectionTimeoutsContext.h (!). This is weird and caused issues with
linking on s390x (##45520). There was an attempt to fix some
inconsistencies (#45848) but neither did @Algunenano nor me at first
really understand why the definition is in the header.

Turns out that ConnectionTimeoutsContext.h is only #include'd from
source files which are part of the normal server build BUT NOT part of
the keeper standalone build (which must be enabled via CMake
-DBUILD_STANDALONE_KEEPER=1). This dependency was not documented and as
a result, some misguided workarounds were introduced earlier, e.g.
0341c6c54b

The deeper cause was that getHTTPTimeouts() is passed a "Context". This
class is part of the "dbms" libary which is deliberately not linked by
the standalone build of clickhouse-keeper. The context is only used to
read the settings and the "Settings" class is part of the
clickhouse_common library which is linked by clickhouse-keeper already.

To resolve this mess, this PR

- creates source file IO/ConnectionTimeouts.cpp and moves all
  ConnectionTimeouts definitions into it, including getHTTPTimeouts().

- breaks the wrong dependency by passing "Settings" instead of "Context"
  into getHTTPTimeouts().

- resolves the previous hacks
2023-02-05 20:49:34 +00:00
Vitaly Baranov
a3423ce196 Remove unnecessary execute() while evaluating a constant expression. 2023-02-05 20:26:34 +01:00
Robert Schulze
9b841d67e8
Merge remote-tracking branch 'origin/master' into qc-max_cache_size 2023-02-05 16:47:02 +00:00
Kseniia Sumarokova
80c96f7d3d
Merge branch 'master' into cache-better-locks 2023-02-04 22:31:59 +01:00
kssenii
5abbc7cad6 Cleanup 2023-02-04 19:41:41 +01:00
kssenii
1cbff88a7c Fix integration test 2023-02-04 19:31:19 +01:00
Nikita Taranov
b983b363f8 Merge branch 'master' into aggr_partitions_independently 2023-02-04 18:24:31 +00:00
kssenii
5f124263a6 Revert changes with complex SettingsChanges 2023-02-03 21:04:45 +01:00
Nikita Mikhaylov
33877b5e00
Parallel replicas. Part [2] (#43772) 2023-02-03 14:34:18 +01:00
Antonio Andelic
d5117f2aa6
Define S3 client with bucket and endpoint resolution (#45783)
* Update aws

* Define S3 client with bucket and endpoint resolution

* Add defines for ErrorCodes

* Use S3Client everywhere

* Remove unused errorcode

* Add DROP S3 CLIENT CACHE query

* Add a comment

* Fix style

* Update aws

* Update reference files

* Add missing include

* Fix unit test

* Remove unneeded declarations

* Correctly use RetryStrategy

* Rename S3Client to Client

* Fix retry count

* fix clang-tidy warnings
2023-02-03 14:30:52 +01:00
Arthur Passos
0f2b9612e3 revert old architecture visitor until I get it working if needed 2023-02-03 10:03:39 -03:00
Robert Schulze
f72286c8f3
Merge branch 'master' into qc-max_cache_size 2023-02-03 10:54:35 +01:00
Alexey Milovidov
108d1c3442
Merge pull request #45872 from ClickHouse/fix-trash-query-kind
Remove trash, add a feature and fix a bug
2023-02-03 09:58:43 +03:00
Alexey Milovidov
ebbc22bcf1 Fix error 2023-02-03 02:16:30 +01:00
kssenii
45a65e5a7b Merge remote-tracking branch 'origin/cache-better-locks' into cache-better-locks 2023-02-02 20:51:38 +01:00
kssenii
34be401ce2 Rename some variables 2023-02-02 20:50:37 +01:00
kssenii
a391de18a1 Merge remote-tracking branch 'upstream/master' into allow-single-disk-instead-of-storage-policy 2023-02-02 20:49:41 +01:00
Robert Schulze
b1889b0284
Make the naming a little less confusing 2023-02-02 19:18:39 +00:00
Robert Schulze
0e5a10b856
Merge pull request #45751 from ClickHouse/qc-better-logging
Improve logging in Query Cache
2023-02-02 20:11:18 +01:00
Suzy Wang
1619971b95
Merge branch 'master' into window_view_fix 2023-02-02 11:04:38 -05:00
Alexander Gololobov
ec3bb0c04e
Merge pull request #45794 from ClickHouse/parallelize_is_storage_touched_by_mutations
Do not restrict count() query to 1 thread in isStorageTouchedByMutations()
2023-02-02 16:44:11 +01:00
Kseniia Sumarokova
24da7f2b54
Merge branch 'master' into cache-better-locks 2023-02-02 16:07:13 +01:00
Robert Schulze
8055b7ed07
Merge branch 'master' into qc-better-logging 2023-02-02 11:45:37 +01:00
Robert Schulze
53e1c9cd08
Merge branch 'master' into qc-max_cache_size 2023-02-02 11:45:29 +01:00
kssenii
d9129a64f2 Remove duplicate file 2023-02-02 11:18:52 +01:00
Alexey Milovidov
62bfdb349a Merge branch 'master' into fix-trash-query-kind 2023-02-02 04:26:49 +01:00
Suzy Wang
5b3054492d
Update src/Interpreters/Aggregator.h
Co-authored-by: Yakov Olkhovskiy <99031427+yakov-olkhovskiy@users.noreply.github.com>
2023-02-01 15:21:34 -05:00
Arthur Passos
7e7b1829be temp 2023-02-01 14:29:59 -03:00
kssenii
a083178eec Fix style 2023-02-01 18:16:42 +01:00
kssenii
3075a496cb Refactoring 2023-02-01 17:12:14 +01:00
Dmitry Novik
eee7df8149 WIP on group_by_use_nulls 2023-02-01 13:33:32 +00:00
kssenii
6320a1c4e1 Fix stress tests 2023-02-01 14:28:47 +01:00
Vladimir C
0d48eed68f
Merge pull request #45848 from Algunenano/fix_connection_timeouts_context_include 2023-02-01 12:53:50 +01:00
Robert Schulze
216636f984
Make query cache global settings re-configurable 2023-02-01 10:35:19 +00:00
Robert Schulze
8a9493aeb9
Update QueryCache.cpp 2023-02-01 11:28:00 +01:00
Robert Schulze
f6259fbd65
Query cache: Improve logging 2023-02-01 10:24:17 +00:00
Robert Schulze
b512316586
Merge pull request #45682 from ClickHouse/rename-qrc-to-qc
Rename "Query Result Cache" to "Query Cache"
2023-02-01 11:23:29 +01:00
Alexander Gololobov
e705f11ad1
Merge pull request #45786 from ClickHouse/delete_on_cluster
Support DELETE ON CLUSTER
2023-02-01 10:52:01 +01:00
robot-ch-test-poll2
202fa9fa37
Merge pull request #45859 from FArthur-cmd/show_engines_support
Add support for SHOW ENGINES query
2023-02-01 07:32:56 +01:00
Alexey Milovidov
9b908a0be0 Fix trash around query kind, part 2 2023-02-01 03:31:31 +01:00
Alexey Milovidov
5c4db119fa Fix trash around query kind, part 1 2023-02-01 03:11:54 +01:00
Alexey Milovidov
c7f9cf6587
Update InterpreterShowEngineQuery.h 2023-02-01 04:57:58 +03:00
Arthur Passos
2c15ede37c Change setting name from force_select_final to final 2023-01-31 15:21:00 -03:00
kssenii
ad34ff24dc Fix memory leak 2023-01-31 16:41:43 +01:00
FArthur-cmd
a8e38738e9 add support for show engines query 2023-01-31 18:35:10 +03:00
Raúl Marín
7c31cb7adc Proper includes for ConnectionTimeoutsContext.h 2023-01-31 16:11:32 +01:00
Alexander Gololobov
96cb23a6a1 Break on first non-empty block 2023-01-31 15:49:56 +01:00
kssenii
f497e3bcbe Better 2023-01-31 14:48:51 +01:00
Arthur Passos
f194b452e7 auto -> bool 2023-01-31 09:13:57 -03:00
Antonio Andelic
c99efa75b7
Merge branch 'master' into custom-key-parallel-replicas 2023-01-31 11:58:30 +01:00
Robert Schulze
325c6bdf3d
Renaming: "Query Result Cache" --> "Query Cache"
Reasons:

- The cache will at some point store intermediate results as opposed to
  only query results. We should change the terminology now without
  having to worry about backward compat.

- Equivalent caches in MySQL (1) and Starrocks (2) are called "query
  cache".

- The new name is ca. 13.8% more catchy.

(1) https://dev.mysql.com/doc/refman/5.6/en/query-cache.html
(2) https://docs.starrocks.io/en-us/2.5/using_starrocks/query_cache
2023-01-31 09:54:34 +00:00
Robert Schulze
659a64a1d9
Merge pull request #45679 from ClickHouse/untangle-query-result-cache-settings
Untangle configuration of query result cache
2023-01-31 10:44:14 +01:00
Alexander Gololobov
93b82a2c44 Use multithreaded executor in isStorageTouchedByMutations() 2023-01-30 23:48:52 +01:00
Alexander Gololobov
4eea3de654 Do not restrict count() query to 1 thread in isStorageTouchedByMutations() 2023-01-30 22:29:28 +01:00
Alexander Gololobov
863548114d force_wait flag is no longer needed 2023-01-30 18:38:28 +01:00
Nikita Taranov
3e3d1d15a8 properly handle case when there is nondeterministic functions 2023-01-30 17:11:56 +00:00
Nikita Taranov
581f31ad3d better 2023-01-30 17:11:56 +00:00
Nikita Taranov
a18343773f improve perf 2023-01-30 17:11:56 +00:00
Nikita Taranov
a2c9aeb7c9 stash 2023-01-30 17:10:45 +00:00
Arthur Passos
48e5211454
Merge branch 'ClickHouse:master' into add_select_final_mt_setting 2023-01-30 13:01:36 -03:00
Alexander Gololobov
de841b29c7 Support delete ON CLUSTER 2023-01-30 16:42:52 +01:00
Alexander Gololobov
386ae9266d Rewrite DELETE into ALTER UPDATE _row_exists=0 2023-01-30 15:46:17 +01:00
Robert Schulze
27b978381b
Use more speaking setting names 2023-01-30 14:20:43 +00:00
kssenii
0378edc3fe Better test 2023-01-30 15:19:09 +01:00
Roman Vasin
df4a9f111c Disallow creation of temporary tables with ON CLUSTER and with Repliacted table engines 2023-01-30 08:08:20 +00:00
SmitaRKulkarni
809354dc30
Merge branch 'master' into 45508_Update_strategy_for_system_sync_replica 2023-01-29 11:28:21 +01:00
taiyang-li
3ee792f4e2 Merge branch 'master' into improve_count_if 2023-01-29 15:46:23 +08:00
Alexey Milovidov
f4eb19bd69
Update InterpreterSystemQuery.cpp 2023-01-29 04:43:24 +03:00
freedomDR
379850b6c4 add system sync file cache 2023-01-28 04:27:10 +00:00
kssenii
a05054e10e Fix 2023-01-27 17:05:01 +01:00
SmitaRKulkarni
b22fd40d92
Merge branch 'master' into 45508_Update_strategy_for_system_sync_replica 2023-01-27 15:00:47 +01:00
kssenii
dc3cff9f06 Merge remote-tracking branch 'upstream/master' into cache-better-locks 2023-01-27 12:39:23 +01:00
Robert Schulze
e30c38cab3
Fix up some leftovers 2023-01-27 10:43:43 +00:00
Robert Schulze
7d97fab2dc
Untangle query result cache settings
This PR modularizes the configuration options of the query result cache
(QRC).

1. Add a feature toggle "allow_experimental_query_result_cache = false"
   which controls whether the experimental QRC can be used at all. This
   configuration option will be removed once the QRC becomes
   non-experimental.

2. Modularize the existing two configuration settings
   "enable_experimental_query_result_cache" and
   "enable_experimental_query_result_cache_passive_usage" into three
   orthogonal settings:

   - "use_query_result_cache = false" controls whether a query (or all
     queries of the session) utilize the query result cache

   - settings "use_query_result_cache_passive_usage = true" and
     "use_query_result_cache_active_usage = true" control the way a
     query (or all queries of the session) utilize the query result
     cache: "passive" use is reading, "active" use is writing.
2023-01-26 21:17:48 +00:00
Alexander Tokmakov
067b1f5f13 Merge branch 'master' into exception_message_patterns4 2023-01-26 15:20:58 +01:00
Roman Vasin
c09d8e744f Make MergeTree and Log table engines DROP and clean data 2023-01-26 14:15:09 +00:00
Antonio Andelic
07b2c5ce0f
Merge branch 'master' into custom-key-parallel-replicas 2023-01-26 12:42:04 +01:00
Smita Kulkarni
8c885ffeae Updated checking of SYSTEM SYNC REPLICA
Implementation:
* Updated to wait for current last entry to be processed (after pulling shared log) instead of queue size becoming 0.
* Updated Subscriber to notify both queue size and removed log_entry_id.
2023-01-26 11:40:09 +01:00
Alexander Tokmakov
a584ad0eb1 forbid runtime strings 2023-01-26 10:52:47 +01:00
Kseniia Sumarokova
297516a084
Merge pull request #45614 from kssenii/rename-some-setting
Rename setting `max_query_cache_size`
2023-01-26 10:46:03 +01:00
Alexander Tokmakov
9b670946db Merge branch 'master' into exception_message_patterns5 2023-01-26 00:41:32 +01:00
Alexander Tokmakov
3744fa2c63 format more messages 2023-01-25 21:16:42 +01:00
kssenii
29de87e145 Rename setting 2023-01-25 16:36:36 +01:00
Alexander Tokmakov
b9e5c586e6
Merge pull request #45566 from ClickHouse/resubmit_45493
Resubmit "Fix possible in-use table after DETACH"
2023-01-25 16:39:55 +03:00
Alexander Tokmakov
6eb557b2ba Merge branch 'master' into exception_message_patterns4 2023-01-25 13:49:17 +01:00
Robert Schulze
59528cfca0
Merge pull request #45460 from ClickHouse/inv-index-cleanup
Cleanup of inverted index
2023-01-25 13:23:38 +01:00
Vladimir C
cfefaebe8f
Merge pull request #45558 from ClickHouse/vdimir/remove-ast-fuse 2023-01-25 11:25:36 +01:00
Sergei Trifonov
0d1ea05ff6
Merge pull request #45007 from ClickHouse/cancellable-mutex-integration
Fast shared mutex integration
2023-01-25 11:15:46 +01:00
Alexander Tokmakov
6621834e37 fix 2023-01-24 17:10:54 +01:00
vdimir
0e09d0a00c
Remove FuseSumCountAggregatesVisitor 2023-01-24 14:58:26 +00:00
Alexander Tokmakov
31e16c4b4d fix 2023-01-24 15:29:19 +01:00
Antonio Andelic
ffa3d1d603 Run query on 1 replica if no custom_key 2023-01-24 13:58:42 +00:00
Alexander Tokmakov
6ecae8388e Merge branch 'master' into exception_message_patterns4 2023-01-24 14:42:36 +01:00
Alexander Tokmakov
42a976fe3d Revert "Revert "Merge pull request #45493 from azat/fix-detach""
This reverts commit 9dc4f02d15.
2023-01-24 14:15:51 +01:00
SmitaRKulkarni
d387835774
Merge branch 'master' into 45486_Fix_flaky_test_for_disallowing_concurrent_backups_restores 2023-01-24 13:49:32 +01:00
Vladimir C
f818e3070d
Merge pull request #45391 from ClickHouse/vdimir/join_prewhere_bug_44062 2023-01-24 13:23:31 +01:00
Alexander Tokmakov
c25e5ff8de
Merge pull request #45545 from ClickHouse/revert_45439
Revert "Merge pull request #45493 from azat/fix-detach"
2023-01-24 15:21:33 +03:00
Vladimir C
0403f801a8
Merge pull request #44238 from ClickHouse/vdimir/grace_join_mem 2023-01-24 13:02:11 +01:00
Antonio Andelic
7df545fbe3 Remove unused define 2023-01-24 11:21:09 +00:00
Alexander Tokmakov
9dc4f02d15 Revert "Merge pull request #45493 from azat/fix-detach"
This reverts commit a182a6b851, reversing
changes made to c47a29a089.
2023-01-24 12:20:37 +01:00
Antonio Andelic
95853af459 Merge branch 'master' into custom-key-parallel-replicas 2023-01-24 10:49:40 +00:00
Antonio Andelic
37b62b3a58 Use Map for custom_key 2023-01-24 10:46:47 +00:00
Robert Schulze
9ff2bfcbf5
Merge remote-tracking branch 'origin/master' into inv-index-cleanup
src/Interpreters/GinFilter.cpp
	src/Interpreters/InterpreterCreateQuery.cpp
	src/Storages/MergeTree/MergeTreeData.cpp
	src/Storages/MergeTree/MergeTreeDataPartWriterOnDisk.cpp
	src/Storages/MergeTree/MergeTreeIndexInverted.cpp
2023-01-24 10:09:42 +00:00