thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-16 11:22:12 +00:00
Code Issues Packages Projects Releases Wiki Activity
111,211 Commits 9,338 Branches 1,783 Tags 2.1 GiB
13f29a7242
Commit Graph

12060 Commits

Author SHA1 Message Date
Maksim Kita
6dffe52e68
Merge pull request #46471 from kitaisreal/analyzer-planner-fixes-before-enable-by-default 
Analyzer planner fixes before enable by default
 2023-02-19 13:49:18 +03:00
flynn
7b4fbf33b3 fix 2023-02-19 03:47:14 +00:00
Alexander Gololobov
0ed0b191f8 Fix for Nullable step filter column 2023-02-18 19:12:32 +01:00
flynn
177261a5ac fix typo 2023-02-18 17:34:37 +00:00
flynn
830398f0bd fix build without avro 
fix
 2023-02-18 17:32:35 +00:00
Alexander Gololobov
88fb6c3c25 Fix for incorrect combined PREWHERE column 2023-02-18 18:29:16 +01:00
Maksim Kita
cbd961de98 Fixed code review issues 2023-02-18 17:06:00 +01:00
Nikolay Degterinsky
f93aae80b8
Merge pull request #46407 from evillique/parameters-rename-query 
Add parameterized RENAME queries
 2023-02-18 16:04:55 +01:00
Kseniia Sumarokova
b0ea047e76
Merge pull request #46533 from kssenii/fix-disk-setting 
Allow `disk` merge tree setting in query to override config merge tree setting `storage_policy`
 2023-02-18 12:36:54 +01:00
pufit
4496db66b2 Implement system.server_settings 2023-02-17 23:46:09 -05:00
Kseniia Sumarokova
bda4c94f66
Merge pull request #45384 from ucasfl/iceberg 2023-02-17 23:38:24 +01:00
Kseniia Sumarokova
a0b28edd46
Merge pull request #46521 from kssenii/fix-stress-test-logical-errors 
Fix stress test
 2023-02-17 21:01:20 +01:00
Nikolay Degterinsky
5104b2e8bf
Merge branch 'master' into parameters-rename-query 2023-02-17 17:30:37 +01:00
kssenii
afb36dd120 Allow disk setting to override storage_policy from config merge tree settings insteaf of exception 2023-02-17 17:11:29 +01:00
Maksim Kita
6c1436f78e Fixed tests 2023-02-17 16:50:46 +01:00
Antonio Andelic
ab51c1d975
Merge branch 'master' into custom-key-parallel-replicas 2023-02-17 15:06:41 +01:00
flynn
ebd88aab41
Merge branch 'master' into iceberg 2023-02-17 21:53:29 +08:00
Vladimir C
0f182f6164
Merge pull request #46454 from azat/merge-alias-prewhere 2023-02-17 14:10:52 +01:00
flynn
b3a9468661 fix 2023-02-17 12:42:24 +00:00
flynn
4b1d997b82 fix 2023-02-17 12:27:53 +00:00
kssenii
bae85c50e9 Fix stress test 2023-02-17 12:40:17 +01:00
Alexander Tokmakov
9a664efeb6
Merge branch 'master' into fixes_for_993 2023-02-17 14:17:08 +03:00
Sema Checherinda
b551b9fb3a
Merge pull request #46452 from CheSema/resolve-race-get-csn 
resolve race in getCSNAndAssert
 2023-02-17 11:31:58 +01:00
flynn
1d5b7ebc73 fix 2023-02-17 09:01:13 +00:00
flynn
561b575111 fix style 2023-02-17 08:35:38 +00:00
flynn
a39f6f419b refactor 2023-02-17 08:27:52 +00:00
Arthur Passos
ba6b7b1610 fix test_alias_column 2023-02-16 16:01:25 -03:00
kssenii
417052e4b2 Fix remaining tests 2023-02-16 18:56:19 +01:00
Azat Khuzhin
beff5ac795 Fix PREWHERE for Merge with different default types 
In case of underlying table has an ALIAS for this column, while in Merge
table it is not marked as an alias, there will NOT_FOUND_COLUMN_IN_BLOCK
error.

Further more, when underlying tables has different default type for the
column, i.e. one has ALIAS and another has real column, then you will
also get NOT_FOUND_COLUMN_IN_BLOCK, because Merge engine should take
care of this.

Also this patch reworks how PREWHERE is handled for Merge table, and now
if you use PREWHERE on the column that has the same type and default
type (ALIAS, ...) then it will be possible, and only if the type
differs, it will be prohibited and throw ILLEGAL_PREWHERE error.

And last, but not least, also respect this restrictions for
optimize_move_to_prewhere.

v2: introduce IStorage::supportedPrewhereColumns()
v3: Remove excessive condition for PREWHERE in StorageMerge::read()
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-02-16 17:21:53 +01:00
kssenii
75d62ee24a Remove support of feature which does not work reliably 2023-02-16 16:41:31 +01:00
Alexander Gololobov
a6061e022e Merge branch 'master' into split_prewhere_dag 2023-02-16 14:43:58 +01:00
flynn
eae73a1511 fix 2023-02-16 13:42:19 +00:00
youennL-cs
6526c2a8ab
[RFC] Replacing merge tree new engine (#41005) 
* Add new engine to ReplacingMergeTree corresponding to the ReplacingCollapsingMergeTree

* Add new test for the new ReplacingMergeTree engine

* Limit sign value to -1/1

* Add new engine to ReplacingMergeTree corresponding to the ReplacingCollapsingMergeTree

* Add new test for the new ReplacingMergeTree engine

* Limit sign value to -1/1

* Replace sign column(Int8) by is_deleted(UInt8)

* Add new engine to ReplacingMergeTree corresponding to the ReplacingCollapsingMergeTree

* Add new test for the new ReplacingMergeTree engine

* Limit sign value to -1/1

* Replace sign column(Int8) by is_deleted(UInt8)

* Add new engine to ReplacingMergeTree corresponding to the ReplacingCollapsingMergeTree

* Add new test for the new ReplacingMergeTree engine

* Limit sign value to -1/1

* Replace sign column(Int8) by is_deleted(UInt8)

* Add keyword 'CLEANUP' when OPTIMIZE

* Cleanup uniquely when it's a replacingMergeTree

* Propagate CLEANUP information and change from 'with_cleanup' to 'cleanup'

* Cleanup data flagged as 'is_deleted'

* Fix merge when optimize and add a test

* Fix OPTIMIZE and INSERT + add tests

* New fix for cleanup at the merge

* Cleanup debug logs

* Add the SETTINGS option 'clean_deleted_rows' that can be 'never' or 'always'

* Fix regression bug; Now REplicatedMergeTree can be called as before without 'is_deleted'

* Add Replicated tests

* Disable tag 'long' for our test and cleanup some white spaces

* Update tests

* Fix tests and remove additional useless whitespace

* Fix replica test

* Style clean && add condition check for is_deleted values

* clean_deleted_rows settings is nom an enum

* Add valid default value to the clean_deleted_rows settings

* Update cleanup checkers to use the enum and fix typos in the test

* Fix submodule contrib/AMQP-CPP pointer

* Add missing messages in test reference and remove a print with non derterministic order

* fix replica test reference

* Fix edge case

* Fix a typo for the spell checker

* Fix reference

* Fix a condition to raise an error if is_deleted differ from 0/1 and cleanup

* Change tests file name and update number

* This should fix the ReplacingMergeTree parameter set

* Fix replicated parameters

* Disable allow_deprecated_syntax_for_merge_tree for our new column

* Fix a test

* Remove non deterministic order print in the test

* Test on replicas

* Remove a condition, when checking optional parameters, that should not be sueful since we disabled the deprected_syntaxe

* Revert "Remove a condition, when checking optional parameters, that should not be useful since we disabled the deprected_syntaxe"

This reverts commit b65d64c05e.

* Fix replica management and limit the number of argument to two maximum, due to the possiblity of deprecated table create/attach failing otherwise

* Test a fix for replicated log information error

* Try to add sync to have consistent results

* Change path of replicas that should cause one issue and add few prints in case it's not that

* Get cleanup info on replicas only if information found

* Fix style issues

* Try to avoid replication error 'cannot select parts...' and and replica read/write field order

* Cleanup according to PR reviews
 and add tests on error raised.

* Update src/Storages/MergeTree/registerStorageMergeTree.cpp

Co-authored-by: Alexander Tokmakov <tavplubix@gmail.com>

* Select ... FINAL don't show rows with is_deleted = true

* Update and fix SELECT ... FINAL merge parameter

* Remove is_deleted rows only on the version inserted when merge

* Fix (master) updates issues

* Revert changes that should not be commited

* Add changes according to review

* Revert changes that should not be commited - part 2

---------

Co-authored-by: Alexander Tokmakov <tavplubix@gmail.com>
 2023-02-16 16:03:16 +03:00
flynn
7f4c23ec8a fix 2023-02-16 12:48:22 +00:00
Maksim Kita
05baf271f0 Analyzer fix table functions with invalid arguments analysis 2023-02-16 12:17:04 +01:00
Maksim Kita
57b5d9852f Fixed build 2023-02-16 12:17:03 +01:00
Maksim Kita
b72ea982b0 Remove unnecessary includes of InterpreterSelectQuery header 2023-02-16 12:17:03 +01:00
Maksim Kita
6b2adc1ec2 Analyzer storage Merge fixes 2023-02-16 12:17:03 +01:00
Maksim Kita
b1ab2af7ad Analyzer support storage Merge 2023-02-16 12:17:03 +01:00
Maksim Kita
84065fb13f Analyzer added distributed table functions support 2023-02-16 12:17:03 +01:00
Maksim Kita
a090a8449d Analyzer distributed read fix 2023-02-16 12:17:03 +01:00
Maksim Kita
f73c49b082 Fixed style check 2023-02-16 12:17:03 +01:00
Maksim Kita
f8442b2a8d Analyzer support LiveView 2023-02-16 12:17:03 +01:00
Maksim Kita
a12aea45f6 Analyzer matcher resolve fix 2023-02-16 12:17:02 +01:00
Maksim Kita
25da9dcef7 StorageDistributed Planner initialization fix 2023-02-16 12:17:02 +01:00
Maksim Kita
0fe3ba36c4 Analyzer added ParitionPruner support 2023-02-16 12:17:02 +01:00
Maksim Kita
bb56af0e2b MergeTreeIndexGin crash fix 2023-02-16 12:17:02 +01:00
Maksim Kita
77fee97705 MergeTreeIndexFullText fix prepared set index analysis 2023-02-16 12:17:02 +01:00
Alexey Milovidov
a9b4f2ef3a
Merge pull request #46435 from ClickHouse/rs/consistent-db-metric 
Exclude internal databases from async metric "NumberOfDatabases"
 2023-02-16 14:09:59 +03:00
Kseniia Sumarokova
187f7482e1
Merge pull request #46044 from ucasfl/log-engine 
Storage Log family support settings storage_policy
 2023-02-16 12:09:47 +01:00
Nikolay Degterinsky
d8c5c7c503
Merge branch 'master' into parameters-rename-query 2023-02-16 11:30:20 +01:00
Kseniia Sumarokova
4303ee9114
Merge pull request #46404 from kssenii/fix-rabbit-test-4 
Fix test test_rabbitmq_overloaded_insert
 2023-02-16 11:28:18 +01:00
Kseniia Sumarokova
947753bd4c
Merge pull request #46409 from kssenii/follow-up-to-46168 
Follow-up to #46168
 2023-02-16 11:26:54 +01:00
flynn
2968cdc8f6 fix 2023-02-16 10:18:22 +00:00
Nikolay Degterinsky
ebec5b8050
Merge pull request #46422 from evillique/move-sources 
Move MongoDB and PostgreSQL sources to Sources folder
 2023-02-16 10:51:45 +01:00
Sergei Trifonov
3ad41b29ec
Merge pull request #46247 from ClickHouse/round-robin-merge-scheduler 2023-02-16 08:03:22 +01:00
flynn
ecc39978d7 fix conflict 2023-02-16 02:23:55 +00:00
Alexander Gololobov
cdf3828575
Removed unneeded default value for enable_multiple_prewhere_read_steps 2023-02-15 22:41:55 +01:00
Alexander Gololobov
6936471150
fix include 
Co-authored-by: Nikita Taranov <nikita.taranov@clickhouse.com>
 2023-02-15 22:37:40 +01:00
Alexander Gololobov
8b528739c1
fix typo 
Co-authored-by: Nikita Taranov <nikita.taranov@clickhouse.com>
 2023-02-15 22:37:18 +01:00
Alexander Gololobov
3ff0c7edcc
fix typo 
Co-authored-by: Nikita Taranov <nikita.taranov@clickhouse.com>
 2023-02-15 22:37:04 +01:00
Antonio Andelic
17494d79c2
Merge pull request #46070 from ClickHouse/s3-client-polish 
Polish S3 client
 2023-02-15 21:22:47 +01:00
Nikolai Kochetov
f524dae924 Fix more tests. 2023-02-15 19:17:24 +00:00
Sema Checherinda
dd1b509637 resolve race in getCSNAndAssert 2023-02-15 18:16:39 +01:00
Han Fei
5458d4216f
Merge branch 'master' into dev-prewhere 2023-02-15 17:59:44 +01:00
Alexander Tokmakov
885f14311c fix 2023-02-15 16:37:44 +01:00
Kruglov Pavel
4f380370a9
Fix s3Cluster schema inference in parallel distributed insert select (#46381) 
* Fix s3Cluster schema inference in parallel distributed insert select
* Try fix flaky test
* Try SYSTEM SYNC REPLICA to avoid test flakiness
 2023-02-15 15:30:43 +01:00
Kruglov Pavel
e56dead5aa
Merge branch 'master' into fix-kafka-protobuf 2023-02-15 15:16:25 +01:00
flynn
de11a05d89 remove redundant header 2023-02-15 13:27:25 +00:00
Robert Schulze
44815240f5
Exclude internal databases from async metric "NumberOfDatabases" 2023-02-15 12:58:00 +00:00
Alexey Milovidov
77f60311da
Merge branch 'master' into add_select_final_mt_setting 2023-02-15 15:35:37 +03:00
flynn
289c5c60d3 fix 2023-02-15 11:24:06 +00:00
Vladimir C
645fee3d94
Merge pull request #46009 from ClickHouse/vdimir/fix_read_in_order_final_desc 2023-02-15 11:23:07 +01:00
kssenii
f75d69a954 Fix 2023-02-15 11:14:50 +01:00
Antonio Andelic
4dd4aac125
Merge branch 'master' into custom-key-parallel-replicas 2023-02-15 11:02:37 +01:00
Nikolay Degterinsky
24bbf885db
Merge branch 'master' into parameters-rename-query 2023-02-15 10:53:44 +01:00
Robert Schulze
ec33204c05
Merge pull request #46344 from ClickHouse/rs/stabilize-full_text_seaerch_test 
Try to stabilize test 02346_full_text_search.sql
 2023-02-15 10:47:00 +01:00
Antonio Andelic
f76ef72ac0
Merge branch 'master' into s3-client-polish 2023-02-15 09:20:06 +01:00
Nikolay Degterinsky
bcc4adf73d
Merge branch 'master' into move-sources 2023-02-15 08:46:09 +01:00
flynn
18cf72147e remove more redundant header files 2023-02-15 07:02:44 +00:00
flynn
50c0693afd fix 2023-02-15 06:49:53 +00:00
flynn
1468e9b9d9 fix 2023-02-15 06:37:05 +00:00
flynn
e4e3adf160 fix 2023-02-15 05:05:46 +00:00
flynn
22a3439eec Merge branch 'master' of github.com:ClickHouse/ClickHouse into log-engine 2023-02-15 03:57:11 +00:00
flynn
f31451822e fix 2023-02-15 03:56:01 +00:00
Nikolay Degterinsky
6e4b660033 Move MongoDB and PostgreSQL sources to Sources folder 2023-02-14 22:35:10 +00:00
Alexander Tokmakov
e96340e183
Merge pull request #46410 from ClickHouse/fix_noisy_log_messages 
Fix noisy log messages
 2023-02-15 01:22:42 +03:00
Nikolay Degterinsky
1fc28c883f Fix tests 2023-02-14 22:19:46 +00:00
Alexander Tokmakov
8ddbc85919
Merge branch 'master' into followup_revert_39737 2023-02-15 01:15:17 +03:00
Sergei Trifonov
b7a4fa8bd9
Merge branch 'master' into round-robin-merge-scheduler 2023-02-14 20:39:02 +01:00
serxa
a7bbf02bac fix possible deadlock 2023-02-14 19:37:09 +00:00
pufit
e914e7c98d
Merge pull request #45148 from mateng0915/replicas_status_api_optimize 
Feature: Optimize the replica delay api logic
 2023-02-14 13:44:52 -05:00
Han Fei
937fade982 clean up 2023-02-14 18:54:19 +01:00
Han Fei
86fda9bd22 some clean up 2023-02-14 18:52:09 +01:00
Han Fei
32050acde5 Merge branch 'master' into dev-prewhere 2023-02-14 18:39:13 +01:00
Kseniia Sumarokova
fe155bb210
Merge branch 'master' into follow-up-to-46168 2023-02-14 18:28:46 +01:00
Anton Popov
ecc6ff707b
Merge pull request #46387 from CurtizJ/fix-update-from-empty-nested 
Fix `ALTER UPDATE` of empty `Nested` columns
 2023-02-14 16:28:32 +01:00
kssenii
5bda358fb7 Follow-up to #46168 2023-02-14 15:45:38 +01:00
Alexander Tokmakov
1bf3735ca9 fix noisy log messages 2023-02-14 15:33:27 +01:00
vdimir
abd9119583
Remove unnecessary ProfileEventsScope 2023-02-14 14:11:12 +00:00
vdimir
4c1aeb7972
Use overload of PartLog::addNewPart with profile_events 2023-02-14 14:11:12 +00:00
vdimir
07d7478bc7
wip 2023-02-14 14:11:10 +00:00
vdimir
994dcb93f9
Rename ScopedProfileEvents -> ProfileEventsScope 2023-02-14 14:11:10 +00:00
vdimir
a228f7f419
[wip] ProfileCounters for each part 2023-02-14 14:11:09 +00:00
Alexey Milovidov
f554ff9d99
Remove trash 2023-02-14 14:11:08 +00:00
Nicolae Vartolomei
96e4411694
Fix style 2023-02-14 14:11:07 +00:00
John Skopis
f7604cc686
[feat] Add ProfileEvents map to PartLog 
closes #10316
 2023-02-14 14:11:02 +00:00
flynn
e05b4e9745 fix conflict 2023-02-14 13:49:28 +00:00
Antonio Andelic
f703b4c25b Merge branch 'master' into s3-client-polish 2023-02-14 13:33:45 +00:00
Antonio Andelic
15df1fcd72
Merge pull request #46339 from ClickHouse/refactor-s3 
Extract common logic for S3
 2023-02-14 14:30:43 +01:00
Antonio Andelic
adde580756
Merge branch 'master' into custom-key-parallel-replicas 2023-02-14 14:09:12 +01:00
Antonio Andelic
1826c29c82 Merge branch 'master' into s3-client-polish 2023-02-14 13:08:27 +00:00
Nikita Taranov
cbd10c4493
Fix mbm with aio on top of query plan (#45892) 2023-02-14 14:02:54 +01:00
Nikolay Degterinsky
b9d3160107 Add parameterized RENAME queries 2023-02-14 13:01:06 +00:00
Robert Schulze
95c50b8653
Address the actual problem (not an optimal approach yet ...) 2023-02-14 12:32:09 +00:00
kssenii
bead024ecb Fix test 2023-02-14 12:45:57 +01:00
Kseniia Sumarokova
0b55021c68
Merge pull request #46337 from kssenii/rabbitmq-tests-fix-2 
Fix flaky test_storage_rabbitmq::test_rabbitmq_address
 2023-02-14 11:44:34 +01:00
Sema Checherinda
fba7ee6332
Merge pull request #46347 from CheSema/fix-select-candite-to-check 
fix candidate selection
 2023-02-14 11:18:42 +01:00
mateng915
1faf09ec21
Merge branch 'master' into replicas_status_api_optimize 2023-02-14 17:55:23 +08:00
Robert Schulze
61e315f30d
Merge pull request #45520 from ClibMouse/z-build-0120 
s390x build support
 2023-02-14 10:01:38 +01:00
Antonio Andelic
5ab24285fc Fix arg parsing 2023-02-14 08:33:59 +00:00
Antonio Andelic
005888b0a4
Merge pull request #46330 from ClickHouse/delete-update-keepermap 
Add support for DELETE/UPDATE for KeeperMap
 2023-02-14 09:10:41 +01:00
Anton Popov
1ab37ad5ed fix update of empty Nested columns 2023-02-14 02:31:42 +00:00
mateng915
d28792528a
Merge branch 'master' into replicas_status_api_optimize 2023-02-14 10:03:07 +08:00
Alexander Tokmakov
e712fbecb2 fix race between drop and create 2023-02-14 00:45:28 +01:00
Alexander Gololobov
d76e4f60a8 Fix for combined node for condition group not found 2023-02-13 23:14:18 +01:00
Nikolai Kochetov
1c0c662fe1
Merge pull request #46275 from ClickHouse/less-mem-usage-for-vertical-merge 
Less memory usage for vertical merge (non-remote disk)
 2023-02-13 21:10:43 +01:00
Alexander Tokmakov
a4d9688775 fix 'Directory not empty after drop' with zero copy replication 2023-02-13 19:35:07 +01:00
Alexander Gololobov
0fe080cc6f Logger instead of debug prints 2023-02-13 17:43:41 +01:00
Alexander Gololobov
18a29e81fc Better formatting in PrewhereExprInfo::dump() 2023-02-13 17:42:27 +01:00
flynn
68748d6aef fix 2023-02-13 16:06:00 +00:00
Antonio Andelic
cdf570a64f Better 2023-02-13 15:23:48 +00:00
kssenii
614d57d245 Fix race 2023-02-13 16:02:41 +01:00
Alexander Gololobov
f0011862a9 Move prewhere splitting logic into a separate file 2023-02-13 16:00:03 +01:00
Antonio Andelic
95db6fb1a6
Merge pull request #45659 from ClickHouse/add-support-for-settings-alias 
Add support for settings aliases/Rename `replication_alter_partitions_sync` to `alter_sync`
 2023-02-13 15:31:12 +01:00
Suzy Wang
e140bed3ea
Merge branch 'master' into z-build-0120 2023-02-13 09:06:48 -05:00
Sema Checherinda
a48b6ec612 fix candidate selection 2023-02-13 14:52:55 +01:00
Alexander Tokmakov
69f579b8ec remove an unused argument 2023-02-13 14:46:46 +01:00
Alexander Gololobov
39e4ca9850 Cleanup 2023-02-13 14:40:54 +01:00
Alexander Gololobov
c39cfee954 Properly set "remove" flag for step prewhere_columns when they are used in futher steps 2023-02-13 14:11:18 +01:00
flynn
c49a293a9f refactor and get rid of s3 2023-02-13 12:39:54 +00:00
Antonio Andelic
3a6ea861d8 Extract common argument parsing logic 2023-02-13 12:27:49 +00:00
Alexander Gololobov
3060bee18a Properly handle constants in multiple steps 2023-02-13 12:58:33 +01:00
Alexey Milovidov
8265db80ff
Merge pull request #46309 from ClickHouse/revert-39737 
[RFC] Revert #39737
 2023-02-13 14:41:02 +03:00
Kruglov Pavel
e139070043
Merge branch 'master' into fix-kafka-protobuf 2023-02-13 12:38:18 +01:00
Antonio Andelic
454705b261 Add support for DELETE/UPDATE for KeeperMap 2023-02-13 11:22:43 +00:00
Vladimir C
89cb55aa1f
whitespace 2023-02-13 12:19:31 +01:00
Alexey Milovidov
9be1bba92f
Merge pull request #46310 from ClickHouse/replicas_status_remove_mutex 
Remove mutex from replicas_status
 2023-02-13 13:36:11 +03:00
flynn
d3dd9421da refactor and get rid of s3 2023-02-13 08:29:22 +00:00
Alexey Milovidov
1b912c07cd
Merge pull request #42391 from ClickHouse/filimonov-kafka-Local-No-offset-stored 
Attempt to fix 'Local: No offset stored message' from Kafka
 2023-02-13 10:49:49 +03:00
Antonio Andelic
8d16fe5793
Merge branch 'master' into add-support-for-settings-alias 2023-02-13 08:46:00 +01:00
mateng915
c876ad901b
Merge branch 'master' into replicas_status_api_optimize 2023-02-13 11:38:55 +08:00
Alexander Gololobov
a89465d1cb Fix build after merge 2023-02-12 22:48:28 +01:00
Alexander Gololobov
3ad0683de9 Refactor DAG splitting into steps 2023-02-12 21:55:53 +01:00
Alexander Gololobov
12e576cddb Slight refactoring 2023-02-12 21:55:53 +01:00
Alexander Gololobov
8fc08c74f0 Pass action_settings to getReadTaskColumns() 2023-02-12 21:55:53 +01:00
Alexander Gololobov
e460d53619 Use different name when filter column is CASTed to UInt8 2023-02-12 21:55:41 +01:00
Alexander Gololobov
7634b9f56e Added settings to enable the new logic 2023-02-12 21:54:23 +01:00
Alexander Gololobov
120b112743 Cleanup 2023-02-12 21:54:23 +01:00
Alexander Gololobov
31f5dbe2a3 Cast intermediate step result to Bool if needed 2023-02-12 21:54:23 +01:00
Alexander Gololobov
1b207d9fa0 Fix for overwritten row level filter step 2023-02-12 21:54:23 +01:00
Alexander Gololobov
249e670c9f Properly preserve columns that are required by next steps 2023-02-12 21:54:23 +01:00
Alexander Gololobov
45ef2f6d60 Split prewhere actions into separate conjuctive steps 2023-02-12 21:54:23 +01:00
alesapin
1683cd5d6f
Merge branch 'master' into mutations_rename_hang 2023-02-12 13:06:48 +01:00
Alexey Milovidov
e6b44756dc Fixed build 2023-02-12 07:45:28 +01:00
Alexey Milovidov
b8cbc947ce Remove mutex from replicas_status 2023-02-12 07:39:16 +01:00
Alexey Milovidov
e2079b78a3 [RFC] Revert #39737 2023-02-12 07:24:42 +01:00
Alexey Milovidov
384ae944fe Merge branch 'master' into filimonov-kafka-Local-No-offset-stored 2023-02-12 04:01:07 +01:00
Alexey Milovidov
5f00bc49f8
Merge branch 'master' into addParts 2023-02-12 05:55:18 +03:00
Alexey Milovidov
a97f9b2caf
Update MergeTreeData.cpp 2023-02-12 05:55:07 +03:00
Alexey Milovidov
395d6b6bd5
Merge pull request #46282 from CurtizJ/return-back-vertical-merges 
Allow vertical merges from compact to wide parts [2]
 2023-02-12 05:51:12 +03:00
serxa
df6b4c4372 fix typo 2023-02-11 20:18:19 +00:00
serxa
9a7f70f595 fix comment 2023-02-11 16:54:57 +00:00
serxa
eb8cf6f816 Merge branch 'round-robin-merge-scheduler' of github.com:ClickHouse/ClickHouse into round-robin-merge-scheduler 2023-02-11 16:45:55 +00:00
serxa
d2f8377c47 fix check 2023-02-11 16:45:45 +00:00
Sergei Trifonov
297587531b
Merge branch 'master' into round-robin-merge-scheduler 2023-02-11 17:19:27 +01:00
serxa
c58b165b0f add config option to select scheduling policy 2023-02-11 16:18:42 +00:00
Kseniia Sumarokova
ed6f623f3a
Merge pull request #46168 from kssenii/resubmit-prefetches 
Allow to increase prefetching of data for reads
 2023-02-11 16:22:47 +01:00
flynn
50dace9571 fix 2023-02-11 13:58:19 +00:00
flynn
d3255fa420 Merge branch 'master' of github.com:ClickHouse/ClickHouse into log-engine 2023-02-11 12:26:16 +00:00
alesapin
5c83ddabf9 Merge branch 'master' into mutations_rename_hang 2023-02-11 12:18:58 +01:00
kssenii
3067c1d723 Merge remote-tracking branch 'upstream/master' into resubmit-prefetches 2023-02-11 11:36:23 +01:00
Kseniia Sumarokova
2c397551a0
Update RabbitMQProducer.cpp 2023-02-11 11:23:29 +01:00
Kseniia Sumarokova
0e48ddc4c7
Merge pull request #46283 from kssenii/add-logs-for-rabbit 
Add more logging to RabbitMQ (to help debug tests)
 2023-02-11 11:21:13 +01:00
Sergei Trifonov
fbedf4d520
Merge branch 'master' into round-robin-merge-scheduler 2023-02-11 10:30:26 +01:00
Sergei Trifonov
2931c3bbe1
Merge pull request #45711 from ClickHouse/io-scheduler-integration 
Integrate IO scheduler with buffers for remote reads and writes
 2023-02-11 10:26:58 +01:00
robot-ch-test-poll2
4e9d1275c7
Merge pull request #46278 from CheSema/race-check-table 
fix data race between check table request and background checker
 2023-02-11 09:50:48 +01:00
kssenii
3545568440 Fix build 2023-02-10 22:15:44 +01:00
Kseniia Sumarokova
3be36c7979
Merge pull request #41976 from kssenii/allow-single-disk-instead-of-storage-policy 
Allow configuring storage as `SETTINGS disk='<disk_name>'` (instead of `storage_policy`) and explicit disk creation `SETTINGS disk=disk(<disk_configuration>)`
 2023-02-10 21:38:50 +01:00
Vladimir C
ea670acb72
Add comment to requestReadingInOrder 2023-02-10 20:05:18 +01:00
serxa
8c3f85e58d more review fixes 2023-02-10 17:39:28 +00:00
Kruglov Pavel
8bbf62508f
Merge branch 'master' into fix-kafka-protobuf 2023-02-10 18:32:25 +01:00
kssenii
96e745be09 Add logging 2023-02-10 18:14:49 +01:00
Anton Popov
1205932a19 add setting to disable vertical merges from compact parts 2023-02-10 16:33:46 +00:00
kssenii
468d5216da Remove logging because of test 00002_log_and_exception_messages_formatting 2023-02-10 17:25:26 +01:00
Sema Checherinda
ef133ee30d fix data race between check table query and background part checker thread 2023-02-10 16:22:35 +01:00
Suzy Wang
1c2a52c57b
Update src/Storages/StorageS3Cluster.cpp 2023-02-10 10:09:24 -05:00
Anton Popov
a252d3a139 Revert "Merge pull request #46236 from ClickHouse/revert-45681-compact-parts-vertical-merge" 
This reverts commit 27bc317ef9, reversing
changes made to ccf7b6f4c1.
 2023-02-10 14:13:29 +00:00
Anton Popov
27bc317ef9
Merge pull request #46236 from ClickHouse/revert-45681-compact-parts-vertical-merge 
Revert "Allow vertical merges from compact to wide parts"
 2023-02-10 14:31:33 +01:00
alesapin
99dc7fbde8
Update src/Storages/MutationCommands.h 2023-02-10 14:27:59 +01:00
alesapin
808a939ad2 Small rehacktoing 2023-02-10 14:25:19 +01:00
Alexander Tokmakov
8eae498ec6
Merge pull request #45936 from vitlibar/add-setting-check-referential-deps-on-drop 
Check referential dependencies on DROP TABLE
 2023-02-10 16:21:00 +03:00
KitKatKKK
d31457de44
Update src/Storages/MergeTree/MergeTreeData.cpp 
Co-authored-by: flynn <fenglv15@mails.ucas.ac.cn>
 2023-02-10 21:14:42 +08:00
KitKatKKK
7664114349
Update src/Storages/MergeTree/MergeTreeData.cpp 
Co-authored-by: flynn <fenglv15@mails.ucas.ac.cn>
 2023-02-10 21:14:33 +08:00
kssenii
571aabdb69 Fix clang-tidy 2023-02-10 12:39:05 +01:00
Nikolai Kochetov
8c22c740a4
Merge pull request #46205 from ClickHouse/fix-write-buffer-destruction-order-for-vertical-merge 
Fix write buffer destruction order for vertical merge.
 2023-02-10 11:47:23 +01:00
flynn
9743a05cde fix style 2023-02-10 10:30:43 +00:00
flynn
09d7ca26e6 fix 2023-02-10 10:22:55 +00:00
KitKatKKK
ea300ce13b
Merge branch 'master' into addParts 2023-02-10 16:48:04 +08:00
flynn
37a850bd36 Merge branch 'master' of github.com:ClickHouse/ClickHouse into iceberg 2023-02-10 08:47:14 +00:00
KitKatKKK
fa73b00a66
Update src/Storages/MergeTree/MergeTreeData.h 
Co-authored-by: flynn <fenglv15@mails.ucas.ac.cn>
 2023-02-10 16:46:57 +08:00
flynn
db15634a01 fix conflict 2023-02-10 08:41:04 +00:00
attack204
7f41173a1f rename API 2023-02-10 16:24:10 +08:00
attack204
93e92dab3b rename API 2023-02-10 16:05:50 +08:00
attack204
52960d4e7d add API to MergeTreeData 2023-02-10 11:00:11 +08:00
serxa
acabee8554 review fixes 2023-02-09 20:17:46 +00:00
Nikolai Kochetov
4aa6e4d683 Revert "Add return flag for preFinalize." 
This reverts commit 3261378ef8.
 2023-02-09 19:51:09 +00:00
Nikolai Kochetov
3261378ef8 Add return flag for preFinalize. 2023-02-09 19:50:54 +00:00
Suzy Wang
64313d7020
Merge branch 'master' into z-build-0120 2023-02-09 14:49:22 -05:00
Nikolai Kochetov
98765cc20b Disable delayed streams for Vertical merges for non-remote disks. 2023-02-09 19:39:34 +00:00
serxa
bf299b6696 Merge branch 'round-robin-merge-scheduler' of github.com:ClickHouse/ClickHouse into round-robin-merge-scheduler 2023-02-09 19:33:17 +00:00
serxa
cc3c76944b remove redundant code 2023-02-09 19:31:54 +00:00
Sergei Trifonov
e3727d9b0e
Merge branch 'master' into round-robin-merge-scheduler 2023-02-09 20:04:59 +01:00
serxa
f61f95fcec Replace background executor scheduler merges and mutations with round-robin 2023-02-09 18:54:15 +00:00
Robert Schulze
566fb5b9da
Merge pull request #46006 from ClickHouse/rs/fix-fragile-linking 
Fix terribly broken, fragile and potentially cyclic linking
 2023-02-09 18:19:44 +01:00
kssenii
214ffe0bb0 Remove unused code 2023-02-09 17:44:26 +01:00
kssenii
9e04e57949 Allow to give access only to certain named collections 2023-02-09 17:22:57 +01:00
Antonio Andelic
f96d480563
Merge branch 'master' into add-support-for-settings-alias 2023-02-09 16:07:45 +01:00
Anton Popov
612fe0bd00
Revert "Allow vertical merges from compact to wide parts" 2023-02-09 15:42:46 +01:00
alesapin
9b23068d5e
Merge pull request #45821 from ClickHouse/explicit_drop_part_entry_type 
Make separate DROP_PART log entry type
 2023-02-09 15:11:13 +01:00
alesapin
fd52979991 Fix snapshot for ordinary merge tree 2023-02-09 14:56:13 +01:00
attack204
ac81ef0fd8 beauty 2023-02-09 20:36:08 +08:00
attack204
07ea30e6e0 beauty 2023-02-09 20:35:28 +08:00
Vitaly Baranov
39bacfb9dc
Merge branch 'master' into allow-single-disk-instead-of-storage-policy 2023-02-09 13:16:55 +01:00
Kruglov Pavel
c3c0356fdd
Merge branch 'master' into fix-kafka-protobuf 2023-02-09 13:04:55 +01:00
kssenii
2a48a96f83 Fix tests 2023-02-09 12:26:15 +01:00
vdimir
45f7ef6c8f
Do not apply read in reverse order for queries with final 2023-02-09 11:00:22 +00:00
Nikolai Kochetov
65ec9bd7d0 Fix write buffer destruction order for vertical merge. 2023-02-09 10:58:49 +00:00
alesapin
859f528fe1
Merge pull request #45648 from ClickHouse/45508_Update_strategy_for_system_sync_replica 
Updated checking of SYSTEM SYNC REPLICA
 2023-02-09 11:51:05 +01:00
Robert Schulze
e490ec91d9
Merge branch 'master' into rs/fix-fragile-linking 2023-02-09 11:33:59 +01:00
KitKatKKK
54513d1db1
Merge branch 'master' into addParts 2023-02-09 15:41:49 +08:00
attack204
a5d7f002ec beauty 2023-02-09 11:03:04 +08:00
attack204
9ce15279ae reformat 2023-02-09 10:54:18 +08:00
attack204
f65756c0c8 fix style 2023-02-09 09:34:11 +08:00
Vitaly Baranov
4869d3806c Add setting check_referential_table_dependencies to check referential dependencies on DROP TABLE. 2023-02-08 23:56:59 +01:00
Alexander Tokmakov
8101b044fa
Merge pull request #46091 from azat/sanity-assertions 
Sanity assertions for closing file descriptors
 2023-02-09 01:02:03 +03:00
Anton Popov
31adebdca8
Merge pull request #46045 from azat/compact-nested-fix 
Fix reading of non existing nested columns with multiple level in compact parts
 2023-02-08 22:54:11 +01:00
kssenii
b0b865c32e Resubmit prefetches 2023-02-08 21:26:24 +01:00
avogar
c3e8dd8984 Fix low cardinality case 2023-02-08 19:14:28 +00:00
Sergei Trifonov
b0fc551c27
Merge branch 'master' into io-scheduler-integration 2023-02-08 18:16:00 +01:00
attack204
ecaa97c5b1 fix style 2023-02-08 23:45:39 +08:00
attack204
1328725054 fix style 2023-02-08 22:58:31 +08:00
attack204
01cac6c49b fix style 2023-02-08 22:49:10 +08:00
attack204
752d1f81b0 fix style 2023-02-08 22:36:07 +08:00
attack204
1fdf3659da fix style 2023-02-08 22:29:23 +08:00
attack204
41daf88d64 fix style 2023-02-08 22:20:47 +08:00
attack204
820e5bd9c0 fix style 2023-02-08 22:15:05 +08:00
attack204
7c6e30ebe2 fix pointer bug 2023-02-08 22:02:58 +08:00
attack204
b1fc12c162 delete extra API and use dynamic_cast to compute 2023-02-08 21:52:13 +08:00
alesapin
a28d10a810 Fxi 2023-02-08 14:50:30 +01:00
Smita Kulkarni
87ec32f235 Updated to use znode_name instead of log_entry_id - Updated checking of SYSTEM SYNC REPLICA 2023-02-08 14:07:06 +01:00
attack204
b3e2f99f4f
Merge branch 'master' into addParts 2023-02-08 20:27:54 +08:00
attack204
9e1081b79a add parts, active_parts total_marks to system.tables 2023-02-08 20:23:06 +08:00
Robert Schulze
6ff232d782
Merge branch 'master' into rs/fix-fragile-linking 2023-02-08 12:51:12 +01:00
alesapin
2f69df1c98 Merge remote-tracking branch 'origin' into mutations_rename_hang 2023-02-08 12:34:19 +01:00
Antonio Andelic
98b6944d37 Merge branch 'master' into s3-client-polish 2023-02-08 10:07:00 +00:00
Azat Khuzhin
151ba92a2e Fix reading of non existing nested columns with multiple level in compact parts 
Consider the following example:

    CREATE TABLE data (root.array_str Array(UInt8)) ENGINE = MergeTree() ORDER BY tuple();
    INSERT INTO data VALUES ([]);
    ALTER TABLE data ADD COLUMN root.nested_array Array(Array(UInt8));

In this case the first part will not have data for root.nested_array,
and thanks to #37152 it will simply read offsets column from
root.array_str, however since root.nested_array is a nested array, it
will try to read elements from the same offsets stream and if you are
lucky enough you will get one of the following errors:

- Cannot read all data. Bytes read: 1. Bytes expected: 8.: (while reading column root.nested_array): While executing MergeTreeInOrder. (CANNOT_READ_ALL_DATA)
- DB::Exception: Array size is too large: 8233460228287709730: (while reading column serp.serp_features): While executing MergeTreeInOrder.

So to address this, findColumnForOffsets() had been changed to return
the level of the column too, to allow to read only up to this level.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-02-08 10:21:40 +01:00
alesapin
bbf044f477 Fxi 2023-02-07 22:30:38 +01:00
Smita Kulkarni
8c9d994fd7 Extended scope of state_mutex while adding subscriber - Updated checking of SYSTEM SYNC REPLICA 2023-02-07 22:04:17 +01:00
alesapin
bc58fd156b Merge remote-tracking branch 'origin' into mutations_rename_hang 2023-02-07 20:53:04 +01:00
alesapin
36f41da6e0 One more bugfix 2023-02-07 18:53:47 +01:00
Kseniia Sumarokova
41dbdcff67
Merge branch 'master' into simplify-storage-s3-configuration 2023-02-07 18:09:34 +01:00
Smita Kulkarni
ecea28a832 Fixed style check - Updated checking of SYSTEM SYNC REPLICA 2023-02-07 17:27:34 +01:00
Nikolai Kochetov
3912f5a333
Merge pull request #45681 from CurtizJ/compact-parts-vertical-merge 
Allow vertical merges from compact to wide parts
 2023-02-07 17:09:17 +01:00
Smita Kulkarni
34341ddabe Updated locks and removed getLogEntryIds function - Updated checking of SYSTEM SYNC REPLICA 2023-02-07 16:57:58 +01:00
kssenii
40deaf1661 Fix clang-tidy 2023-02-07 16:30:04 +01:00
Sema Checherinda
52d4e78b7f
Merge pull request #45907 from CheSema/forbid-wal-on-s3 
do not run wal on remote disks
 2023-02-07 14:48:00 +01:00
Yakov Olkhovskiy
4e7dffafe4
Merge pull request #46087 from ClickHouse/fix-ip-bloom 
Allow bloom filter for IPv4 and IPv6
 2023-02-07 08:29:28 -05:00
Robert Schulze
10af0b3e49
Reduce redundancies 2023-02-07 12:27:23 +00:00
alesapin
1fd9baa3f3 Fix fetch: 2023-02-07 12:49:51 +01:00
kssenii
9485873a2f Fix integration test 2023-02-07 12:45:23 +01:00
Robert Schulze
08c1f8346e
Merge remote-tracking branch 'origin/master' into rs/fix-fragile-linking 2023-02-07 11:22:00 +00:00
Smita Kulkarni
fb765696b6 Removed state_mutex lock from addSubscriber - Updated checking of SYSTEM SYNC REPLICA 2023-02-07 12:19:58 +01:00
Robert Schulze
690d873126
Merge pull request #46088 from ClickHouse/rs/fix-liburing-build 
Fix build with `-DENABLE_LIBURING=0` (or `-DENABLE_LIBRARIES=0`)
 2023-02-07 12:16:14 +01:00
Sergei Trifonov
950c04cb0e
Merge pull request #44907 from azat/dist/async-INSERT-error-message-fix 
Fix error message for a broken distributed batches ("While sending batch")
 2023-02-07 12:14:52 +01:00
Vitaly Baranov
f021c0eab6
Merge pull request #46058 from vitlibar/remove-unnecessary-execute-for-const-expr 
Remove unnecessary execute() while evaluating a constant expression.
 2023-02-07 12:09:37 +01:00
Azat Khuzhin
8cc41b7f41 Check return value of ::close() 
Note, that according close(2), EINTR should not be retriable for close()

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-02-07 11:28:22 +01:00
Smita Kulkarni
135615cda6 Removed unwanted check from before callback - Updated checking of SYSTEM SYNC REPLICA 2023-02-07 10:27:45 +01:00
Smita Kulkarni
ef29b61786 Updated callback to return log entry ids - Updated checking of SYSTEM SYNC REPLICA 2023-02-07 10:02:38 +01:00
Smita Kulkarni
f34ef86332 Moved background task trigger before getting log entries to wait - Updated checking of SYSTEM SYNC REPLICA 2023-02-07 08:49:41 +01:00
mateng915
708111b62f
Merge branch 'master' into replicas_status_api_optimize 2023-02-07 14:51:27 +08:00
Anton Popov
7dfea59267 remove an assertion 2023-02-07 00:50:41 +00:00
Alexander Tokmakov
13da43ead8
Merge branch 'master' into fix_test_00002 2023-02-07 03:25:42 +03:00
alesapin
dd3a98e88d Fixes 2023-02-06 22:30:31 +01:00
alesapin
aac5da77e8 fix stupid bug 2023-02-06 21:32:40 +01:00
Nikolai Kochetov
b2cc2332f5 Fix 01710_force_use_projection. 2023-02-06 19:20:03 +00:00
kssenii
6cea67d58b Cleanup 2023-02-06 18:11:33 +01:00
alesapin
2a0484ca41 Add commnents, fix bugs 2023-02-06 18:06:14 +01:00
Robert Schulze
8840aa9242
Fix build with -DENABLE_LIBURING=0 (or -DENABLE_LIBRARIES=0) 
Follow-up to #38456

The problem was that code which uses liburing was conditionally build
depending on define OS_LINUX and not based on whether liburing was
enabled/disabled at build time.
 2023-02-06 16:57:42 +00:00
Yakov Olkhovskiy
e433c26cf9 allow bloom filter for IPv4 and IPv6 2023-02-06 16:30:18 +00:00
Alexander Tokmakov
af655eb7dc
Merge pull request #45889 from CheSema/release-merge-blocker-drop-part 
do not block merges when old parts are droping in drop queries
 2023-02-06 19:09:09 +03:00
kssenii
ab0dedf0c8 Simplify code around storage s3 configuration 2023-02-06 16:23:17 +01:00
Alexander Tokmakov
98a4e1ab2c better comment 2023-02-06 15:08:30 +01:00
alesapin
30a7198411
Update src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp 2023-02-06 15:06:28 +01:00
alesapin
bd4a3ce2d5
Update src/Storages/MergeTree/DataPartsExchange.cpp 2023-02-06 15:05:13 +01:00
Alexander Tokmakov
86feaae253 Merge branch 'master' into explicit_drop_part_entry_type 2023-02-06 14:59:59 +01:00
alesapin
a02e6f3d5b fix bug ordinary merge tree 2023-02-06 14:57:21 +01:00
alesapin
e51e385017 fix style 2023-02-06 14:16:17 +01:00
Alexander Tokmakov
e9f770d89d fix 2023-02-06 14:03:14 +01:00
alesapin
b94f9f8006
Merge branch 'master' into 45508_Update_strategy_for_system_sync_replica 2023-02-06 12:38:39 +01:00
Kseniia Sumarokova
38c001ca42
Merge pull request #45957 from xiedeyantu/s3_file_not_found 
Throw an error on no files satisfying S3 wildcard
 2023-02-06 12:32:12 +01:00
Alexander Gololobov
d18364476d
Merge pull request #46047 from azat/system.processes-elapsed-fix 
Fix elapsed column in system.processes (10x error)
 2023-02-06 12:26:05 +01:00
alesapin
b265300a19 Merge branch 'master' into mutations_rename_hang 2023-02-06 12:25:29 +01:00
Antonio Andelic
f67e5505ab Merge branch 'master' into custom-key-parallel-replicas 2023-02-06 11:12:39 +00:00
Sema Checherinda
734d3a318b
const auto ref in the loop 2023-02-06 12:09:33 +01:00
Antonio Andelic
4cd361747d Address PR comment 2023-02-06 10:11:48 +00:00
flynn
0dd8a61a8e fix conflict 2023-02-06 03:25:12 +00:00
mateng915
3e5d3914ba
Merge branch 'master' into replicas_status_api_optimize 2023-02-06 09:58:31 +08:00
Robert Schulze
84b9ff450f
Fix terribly broken, fragile and potentially cyclic linking 
Sorry for the clickbaity title. This is about static method
ConnectionTimeouts::getHTTPTimeouts(). It was be declared in header
IO/ConnectionTimeouts.h, and defined in header
IO/ConnectionTimeoutsContext.h (!). This is weird and caused issues with
linking on s390x (##45520). There was an attempt to fix some
inconsistencies (#45848) but neither did @Algunenano nor me at first
really understand why the definition is in the header.

Turns out that ConnectionTimeoutsContext.h is only #include'd from
source files which are part of the normal server build BUT NOT part of
the keeper standalone build (which must be enabled via CMake
-DBUILD_STANDALONE_KEEPER=1). This dependency was not documented and as
a result, some misguided workarounds were introduced earlier, e.g.
0341c6c54b

The deeper cause was that getHTTPTimeouts() is passed a "Context". This
class is part of the "dbms" libary which is deliberately not linked by
the standalone build of clickhouse-keeper. The context is only used to
read the settings and the "Settings" class is part of the
clickhouse_common library which is linked by clickhouse-keeper already.

To resolve this mess, this PR

- creates source file IO/ConnectionTimeouts.cpp and moves all
  ConnectionTimeouts definitions into it, including getHTTPTimeouts().

- breaks the wrong dependency by passing "Settings" instead of "Context"
  into getHTTPTimeouts().

- resolves the previous hacks
 2023-02-05 20:49:34 +00:00
Vitaly Baranov
a3423ce196 Remove unnecessary execute() while evaluating a constant expression. 2023-02-05 20:26:34 +01:00
Han Fei
532b341de9
Merge pull request #45975 from ucasfl/_part 
use LowCardnality for _part and _partition_id virtual column
 2023-02-05 18:00:46 +01:00
Antonio Andelic
12569da984 Merge branch 'master' into add-support-for-settings-alias 2023-02-05 16:08:57 +00:00
kssenii
a7eb1b1419 Fix integration test 2023-02-04 22:57:30 +01:00
Azat Khuzhin
1307e6d323 Fix elapsed column in system.processes (10x error) 
Fixes: #43455 (cc @Algunenano)
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-02-04 22:37:11 +01:00
kssenii
5abbc7cad6 Cleanup 2023-02-04 19:41:41 +01:00
kssenii
1cbff88a7c Fix integration test 2023-02-04 19:31:19 +01:00
flynn
a10f020b2d Storage Log faminy support settings storage policy 2023-02-04 14:28:31 +00:00
xiedeyantu
f13eedd644 change settings name 2023-02-04 22:11:14 +08:00
Alexey Milovidov
22bd0b6f69
Merge pull request #38983 from CurtizJ/randomize-mt-settings 
Allow to randomize merge tree settings in tests
 2023-02-04 02:59:52 +01:00
alesapin
fd4ae0990b Tiny fix 2023-02-03 21:07:34 +01:00
kssenii
5f124263a6 Revert changes with complex SettingsChanges 2023-02-03 21:04:45 +01:00
alesapin
51b6154d68 Fix stupid bug 2023-02-03 20:56:09 +01:00
alesapin
9416401406 Fix 2023-02-03 20:52:10 +01:00
alesapin
f91c10d09d Initial support for version 2023-02-03 20:49:07 +01:00
Antonio Andelic
60e5ee84c2 Extract common code 2023-02-03 18:44:56 +00:00
Nikolai Kochetov
310a4c69af Aggregate Projections analysis using query plan [In progress] 2023-02-03 18:30:06 +00:00
Antonio Andelic
3a8ba689f2 Update tests and system.settings 2023-02-03 18:26:31 +00:00
alesapin
7a0d0f0c53 Some code which doesn't work 2023-02-03 19:14:49 +01:00
Antonio Andelic
8929838fcc Merge branch 'master' into add-support-for-settings-alias 2023-02-03 17:47:00 +00:00
Alexander Tokmakov
3f11948bb0
Merge branch 'master' into stack_trace_in_part_log 2023-02-03 20:05:00 +03:00
Alexander Tokmakov
7e6f7c79f2
Merge pull request #45457 from FrankChen021/exception_time 
Add last_exception_time to replication_queue
 2023-02-03 20:00:15 +03:00
Nikita Mikhaylov
33877b5e00
Parallel replicas. Part [2] (#43772) 2023-02-03 14:34:18 +01:00
Antonio Andelic
d5117f2aa6
Define S3 client with bucket and endpoint resolution (#45783) 
* Update aws

* Define S3 client with bucket and endpoint resolution

* Add defines for ErrorCodes

* Use S3Client everywhere

* Remove unused errorcode

* Add DROP S3 CLIENT CACHE query

* Add a comment

* Fix style

* Update aws

* Update reference files

* Add missing include

* Fix unit test

* Remove unneeded declarations

* Correctly use RetryStrategy

* Rename S3Client to Client

* Fix retry count

* fix clang-tidy warnings
 2023-02-03 14:30:52 +01:00
Vitaly Baranov
45d2d678ab
Merge pull request #45800 from vitlibar/rename-new-columns-in-system-backups 
Rename new columns in system.backups
 2023-02-03 14:00:16 +01:00
Azat Khuzhin
a196f995b1 Fix error message for a broken distributed batches ("While sending batch") 
There was an error from the begginning that does not respect
file_indices, and iterate only over file_index_to_path, while this is
not correct, since there can be less files then in file_index_to_path,
and this is what file_indices for.

Note, that only an error message was wrong, logic was fine. You can
verify this by the logs:

    2022.12.07 11:55:50.951976 [ 39217 ] {} <Debug> default.dist.DirectoryMonitor: Sending a batch of 10 files to localhost:9000 (128.42 thousand rows, 36.32 MiB bytes).
    2022.12.07 11:55:50.953762 [ 39217 ] {} <Error> default.dist.DirectoryMonitor: Code: 516. DB::Exception: Received from localhost:9000. DB::Exception: Interserver authentication failed. Stack trace:
    ...
    : While sending batch, nums: 62, files: /work6/clickhouse/data/default/dist/shard1_replica1/66827258.bin

As you can see "Sending a batch of 10 files" but "nums: 62"

Fixes: #23856
Refs: #41813
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-02-03 13:54:40 +01:00
Alexey Milovidov
3e3df376c0
Merge pull request #45995 from CurtizJ/check-dynamic-columns 
Check dynamic columns of part before its commit
 2023-02-03 15:39:54 +03:00
Sema Checherinda
e0db071563
Merge branch 'master' into release-merge-blocker-drop-part 2023-02-03 13:35:51 +01:00
Sema Checherinda
fe0b13e9f4
Merge pull request #45458 from FrankChen021/better_postpone_reason 
Show the name of replica that is executing a merge in the postpone_reason
 2023-02-03 13:15:10 +01:00
alesapin
4a3277b1fd Merge branch 'master' into mutations_rename_hang 2023-02-03 12:35:37 +01:00
alesapin
15ddcdc4a6 Fix clang tidy 2023-02-03 12:34:47 +01:00
Kseniia Sumarokova
ef7acb9b66
Merge pull request #44404 from kssenii/storage-rabbitmq-fix-flush 
RabbitMQ - fix writing many small blocks by flushing data only exactly by flush_interval_ms or by max_block_size
 2023-02-03 11:50:12 +01:00
Smita Kulkarni
123f3cfe1f Used unordered_set for log entries and fixed the callback function - Updated checking of SYSTEM SYNC REPLICA 2023-02-03 08:48:16 +01:00
Frank Chen
d38adfab30 Merge two overridden functions as one 2023-02-03 15:27:45 +08:00
flynn
d119367d7d fix 2023-02-03 07:06:58 +00:00
Alexey Milovidov
108d1c3442
Merge pull request #45872 from ClickHouse/fix-trash-query-kind 
Remove trash, add a feature and fix a bug
 2023-02-03 09:58:43 +03:00
chen
07a6de1713
Merge branch 'master' into s3_file_not_found 2023-02-03 12:29:17 +08:00
xiedeyantu
562642ab7f add settings s3_allow_throw_if_mismatch_files 2023-02-03 12:27:13 +08:00
Frank Chen
d3a05a11da Merge remote-tracking branch 'remotes/github/master' into stack_trace_in_part_log 2023-02-03 11:39:43 +08:00
Frank Chen
7af5ae7f2e
Merge branch 'master' into better_postpone_reason 2023-02-03 11:20:37 +08:00
Anton Popov
8ca602a148 check dynamic columns of part before its commit 2023-02-03 02:58:28 +00:00
alesapin
b8b0f1c3d3 Make RENAME COLUMN barrier 2023-02-02 21:11:19 +01:00
kssenii
a391de18a1 Merge remote-tracking branch 'upstream/master' into allow-single-disk-instead-of-storage-policy 2023-02-02 20:49:41 +01:00
alesapin
fc6c62e2ff Add comment 2023-02-02 17:41:36 +01:00
flynn
2d1dd694c6 make _table LowCardinality 2023-02-02 16:33:31 +00:00
alesapin
bdc530dead Fix style 2023-02-02 17:30:38 +01:00
flynn
f88a8bac19 fix 2023-02-02 16:22:09 +00:00
flynn
bc38ebaf52 use LowCardnality for _part and _partition_id virtual column 
fix
 2023-02-02 16:20:29 +00:00
alesapin
dea46e58fb Fixes for ordinary merge tree 2023-02-02 17:03:24 +01:00
alesapin
174bdba912 Fix bugs in alter rename column 2023-02-02 16:57:46 +01:00
Alexander Gololobov
ec3bb0c04e
Merge pull request #45794 from ClickHouse/parallelize_is_storage_touched_by_mutations 
Do not restrict count() query to 1 thread in isStorageTouchedByMutations()
 2023-02-02 16:44:11 +01:00
Kseniia Sumarokova
6705fafef9
Merge branch 'master' into storage-rabbitmq-fix-flush 2023-02-02 16:06:51 +01:00
Sema Checherinda
44be0f41dd
Merge branch 'master' into release-merge-blocker-drop-part 2023-02-02 14:12:40 +01:00
Antonio Andelic
f613bfd8d2 Merge branch 'master' into add-support-for-settings-alias 2023-02-02 11:15:39 +00:00
xiedeyantu
e22cc0eb98 Throw an error on no files satisfying S3 wildcard 2023-02-02 19:13:34 +08:00
Alexey Milovidov
62bfdb349a Merge branch 'master' into fix-trash-query-kind 2023-02-02 04:26:49 +01:00
Sema Checherinda
7c416477ac do not run wal on remote disks 2023-02-01 20:03:45 +01:00
Nikolai Kochetov
ae420b30e5 Aggregate Projections analysis using query plan [In progress] 2023-02-01 17:55:06 +00:00
kssenii
bbf060bf89 Fix test 2023-02-01 18:22:05 +01:00
Sema Checherinda
3e1b17e4cf
fix build 2023-02-01 17:09:43 +01:00
Anton Popov
9f26e16963 loose the assertion 2023-02-01 15:31:06 +00:00
Anton Popov
8dd6dbc452
Merge pull request #45857 from CurtizJ/fix-minor-mistake 
Fix minor mistake after refactoring
 2023-02-01 15:33:19 +01:00
Sema Checherinda
41a1dddc7a
Merge branch 'master' into better_postpone_reason 2023-02-01 15:16:34 +01:00
Antonio Andelic
85cfee4bb9 Better alias definition 2023-02-01 13:54:03 +00:00
Alexander Tokmakov
aba4a00a17 fix 2023-02-01 14:47:57 +01:00
Sema Checherinda
db2aa48c91 do not block merges when old parts are droping in drop queries 2023-02-01 13:30:56 +01:00
Vladimir C
0d48eed68f
Merge pull request #45848 from Algunenano/fix_connection_timeouts_context_include 2023-02-01 12:53:50 +01:00
Robert Schulze
b512316586
Merge pull request #45682 from ClickHouse/rename-qrc-to-qc 
Rename "Query Result Cache" to "Query Cache"
 2023-02-01 11:23:29 +01:00
Alexander Gololobov
e705f11ad1
Merge pull request #45786 from ClickHouse/delete_on_cluster 
Support DELETE ON CLUSTER
 2023-02-01 10:52:01 +01:00
Antonio Andelic
ec04cf6876 Merge branch 'master' into add-support-for-settings-alias 2023-02-01 09:01:01 +00:00
Alexey Milovidov
9b908a0be0 Fix trash around query kind, part 2 2023-02-01 03:31:31 +01:00
Alexey Milovidov
5c4db119fa Fix trash around query kind, part 1 2023-02-01 03:11:54 +01:00
Vitaly Baranov
96b140cc95 Rename columns in system.backups: num_files, num_processed_files, processed_files_size 
num_processed_files -> num_files (BACKUP) / files_read (RESTORE)
processed_files_size -> total_size (BACKUP) / bytes_read (RESTORE)
 2023-01-31 22:45:41 +01:00
Nikolai Kochetov
e2c32ccbca Aggregate Projections analysis using query plan [In progress] 2023-01-31 20:33:01 +00:00
kssenii
4a1bdcf24c Merge remote-tracking branch 'upstream/master' into storage-rabbitmq-fix-flush 2023-01-31 18:11:26 +01:00
Anton Popov
1c4174750d fix mistake after refactoring 2023-01-31 15:48:14 +00:00
Raúl Marín
7c31cb7adc Proper includes for ConnectionTimeoutsContext.h 2023-01-31 16:11:32 +01:00
flynn
fd1ee98183 fix style 2023-01-31 14:40:48 +00:00
Alexander Tokmakov
7862990721 Merge branch 'master' into explicit_drop_part_entry_type 2023-01-31 13:49:04 +01:00
Alexander Tokmakov
2e8100b6e4 make separate DROP_PART log entry type 2023-01-31 13:37:56 +01:00
Alexander Tokmakov
51b0f66e6b
Merge pull request #45793 from ClickHouse/fix_part_check_cancellation 
Fix race condition on a part check cancellation
 2023-01-31 14:15:17 +03:00