Commit Graph

413 Commits

Author SHA1 Message Date
alesapin
3a24f0b6c6 fxi 2023-04-25 15:33:41 +02:00
Michael Lex
51c9ae05fb Fix unexpected part name error when trying to drop a ignored detache partition with zero copy replication
Before this fix, the integration test failed with this error

E           Code: 233. DB::Exception: Received from 172.22.0.9:9000. DB::Exception: Unexpected part name: ignored_all_0_0_0 for format version: 1. Stack trace:
E
E           0. /home/ubuntu/clickhouse/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x19bc7d02 in /usr/bin/clickhouse
E           1. /home/ubuntu/clickhouse/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x19bc7ccd in /usr/bin/clickhouse
E           2. /home/ubuntu/clickhouse/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x3194fae0 in /usr/bin/clickhouse
E           3. /home/ubuntu/clickhouse/src/Common/Exception.cpp:89: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x2268ae6e in /usr/bin/clickhouse
E           4. /home/ubuntu/clickhouse/src/Common/Exception.h:54: DB::Exception::Exception(String&&, int, bool) @ 0x19bbb00a in /usr/bin/clickhouse
E           5. /home/ubuntu/clickhouse/src/Common/Exception.h:81: DB::Exception::Exception<String const&, StrongTypedef<unsigned int, DB::MergeTreeDataFormatVersionTag>&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<StrongTypedef<unsigned int, DB::MergeTreeDataFormatVersionTag>&>::type>, String const&, StrongTypedef<unsigned int, DB::MergeTreeDataFormatVersionTag>&) @ 0x2c4f318c in /usr/bin/clickhouse
E           6. /home/ubuntu/clickhouse/src/Storages/MergeTree/MergeTreePartInfo.cpp:23: DB::MergeTreePartInfo::fromPartName(String const&, StrongTypedef<unsigned int, DB::MergeTreeDataFormatVersionTag>) @ 0x2c4f0867 in /usr/bin/clickhouse
E           7. /home/ubuntu/clickhouse/src/Storages/StorageReplicatedMergeTree.cpp:8305: DB::(anonymous namespace)::getParentLockedBlobs(std::shared_ptr<DB::ZooKeeperWithFaultInjection> const&, String const&, String const&, StrongTypedef<unsigned int, DB::MergeTreeDataFormatVersionTag>, Poco::Logger*) @ 0x2bca90f7 in /usr/bin/clickhouse
2023-04-17 15:57:41 +00:00
alesapin
a72ee0b85e Merge remote-tracking branch 'origin/master' into revert-46909-revert-45911-mutations_rename_hang 2023-04-11 20:14:43 +02:00
Alexander Tokmakov
b5fe1ada0b more options for sync replica 2023-03-28 01:39:36 +02:00
Nikolai Kochetov
73e98de46d Merge branch 'master' into aggregate-projections-analysis-query-plan 2023-03-23 21:28:36 +01:00
Alexander Tokmakov
d11aee2b0c
Merge branch 'master' into fix_993 2023-03-17 21:28:23 +03:00
alesapin
1022c47421 Review fixes 2023-03-17 17:45:02 +01:00
alesapin
e44fca8bb9 Add test 2023-03-17 12:56:31 +01:00
alesapin
4a80197f16 Merge branch 'master' into fix_zero_copy_lock_check 2023-03-17 11:18:51 +01:00
alesapin
3731567b6a Bad code 2023-03-16 21:48:47 +01:00
Azat Khuzhin
8495deb7e3 Remove unused removePartFromZooKeeper()
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
(cherry picked from commit dbdb46e906)
2023-03-16 16:43:23 +01:00
Alexander Tokmakov
8e36f883d3 add 'strict' mode for sync replica, fix 00993 2023-03-16 16:04:42 +01:00
alesapin
ef7508469c Fix wait for zero-copy replication move during manual move 2023-03-15 22:29:24 +01:00
Nikolai Kochetov
3392b96cc3
Merge branch 'master' into aggregate-projections-analysis-query-plan 2023-02-27 14:07:04 +01:00
alesapin
e3961c118a
Revert "Revert "Fix several RENAME COLUMN bugs."" 2023-02-27 12:27:57 +01:00
Alexey Milovidov
8f40bbd5b4
Revert "Fix several RENAME COLUMN bugs." 2023-02-26 05:47:05 +03:00
Alexey Milovidov
eb89ab471f
Merge pull request #45911 from ClickHouse/mutations_rename_hang
Fix several `RENAME COLUMN` bugs.
2023-02-26 01:56:40 +03:00
Nikolai Kochetov
2a8460d962 Merge branch 'master' into aggregate-projections-analysis-query-plan 2023-02-20 13:54:05 +01:00
Alexander Tokmakov
9a664efeb6
Merge branch 'master' into fixes_for_993 2023-02-17 14:17:08 +03:00
youennL-cs
6526c2a8ab
[RFC] Replacing merge tree new engine (#41005)
* Add new engine to ReplacingMergeTree corresponding to the ReplacingCollapsingMergeTree

* Add new test for the new ReplacingMergeTree engine

* Limit sign value to -1/1

* Add new engine to ReplacingMergeTree corresponding to the ReplacingCollapsingMergeTree

* Add new test for the new ReplacingMergeTree engine

* Limit sign value to -1/1

* Replace sign column(Int8) by is_deleted(UInt8)

* Add new engine to ReplacingMergeTree corresponding to the ReplacingCollapsingMergeTree

* Add new test for the new ReplacingMergeTree engine

* Limit sign value to -1/1

* Replace sign column(Int8) by is_deleted(UInt8)

* Add new engine to ReplacingMergeTree corresponding to the ReplacingCollapsingMergeTree

* Add new test for the new ReplacingMergeTree engine

* Limit sign value to -1/1

* Replace sign column(Int8) by is_deleted(UInt8)

* Add keyword 'CLEANUP' when OPTIMIZE

* Cleanup uniquely when it's a replacingMergeTree

* Propagate CLEANUP information and change from 'with_cleanup' to 'cleanup'

* Cleanup data flagged as 'is_deleted'

* Fix merge when optimize and add a test

* Fix OPTIMIZE and INSERT + add tests

* New fix for cleanup at the merge

* Cleanup debug logs

* Add the SETTINGS option 'clean_deleted_rows' that can be 'never' or 'always'

* Fix regression bug; Now REplicatedMergeTree can be called as before without 'is_deleted'

* Add Replicated tests

* Disable tag 'long' for our test and cleanup some white spaces

* Update tests

* Fix tests and remove additional useless whitespace

* Fix replica test

* Style clean && add condition check for is_deleted values

* clean_deleted_rows settings is nom an enum

* Add valid default value to the clean_deleted_rows settings

* Update cleanup checkers to use the enum and fix typos in the test

* Fix submodule contrib/AMQP-CPP pointer

* Add missing messages in test reference and remove a print with non derterministic order

* fix replica test reference

* Fix edge case

* Fix a typo for the spell checker

* Fix reference

* Fix a condition to raise an error if is_deleted differ from 0/1 and cleanup

* Change tests file name and update number

* This should fix the ReplacingMergeTree parameter set

* Fix replicated parameters

* Disable allow_deprecated_syntax_for_merge_tree for our new column

* Fix a test

* Remove non deterministic order print in the test

* Test on replicas

* Remove a condition, when checking optional parameters, that should not be sueful since we disabled the deprected_syntaxe

* Revert "Remove a condition, when checking optional parameters, that should not be useful since we disabled the deprected_syntaxe"

This reverts commit b65d64c05e.

* Fix replica management and limit the number of argument to two maximum, due to the possiblity of deprecated table create/attach failing otherwise

* Test a fix for replicated log information error

* Try to add sync to have consistent results

* Change path of replicas that should cause one issue and add few prints in case it's not that

* Get cleanup info on replicas only if information found

* Fix style issues

* Try to avoid replication error 'cannot select parts...' and and replica read/write field order

* Cleanup according to PR reviews
 and add tests on error raised.

* Update src/Storages/MergeTree/registerStorageMergeTree.cpp

Co-authored-by: Alexander Tokmakov <tavplubix@gmail.com>

* Select ... FINAL don't show rows with is_deleted = true

* Update and fix SELECT ... FINAL merge parameter

* Remove is_deleted rows only on the version inserted when merge

* Fix (master) updates issues

* Revert changes that should not be commited

* Add changes according to review

* Revert changes that should not be commited - part 2

---------

Co-authored-by: Alexander Tokmakov <tavplubix@gmail.com>
2023-02-16 16:03:16 +03:00
mateng915
d28792528a
Merge branch 'master' into replicas_status_api_optimize 2023-02-14 10:03:07 +08:00
Alexander Tokmakov
e712fbecb2 fix race between drop and create 2023-02-14 00:45:28 +01:00
mateng915
c876ad901b
Merge branch 'master' into replicas_status_api_optimize 2023-02-13 11:38:55 +08:00
Alexey Milovidov
e6b44756dc Fixed build 2023-02-12 07:45:28 +01:00
alesapin
5c83ddabf9 Merge branch 'master' into mutations_rename_hang 2023-02-11 12:18:58 +01:00
alesapin
b94f9f8006
Merge branch 'master' into 45508_Update_strategy_for_system_sync_replica 2023-02-06 12:38:39 +01:00
alesapin
b265300a19 Merge branch 'master' into mutations_rename_hang 2023-02-06 12:25:29 +01:00
mateng915
3e5d3914ba
Merge branch 'master' into replicas_status_api_optimize 2023-02-06 09:58:31 +08:00
alesapin
f91c10d09d Initial support for version 2023-02-03 20:49:07 +01:00
alesapin
7a0d0f0c53 Some code which doesn't work 2023-02-03 19:14:49 +01:00
alesapin
bdc530dead Fix style 2023-02-02 17:30:38 +01:00
Sema Checherinda
41a1dddc7a
Merge branch 'master' into better_postpone_reason 2023-02-01 15:16:34 +01:00
Nikolai Kochetov
e2c32ccbca Aggregate Projections analysis using query plan [In progress] 2023-01-31 20:33:01 +00:00
Alexander Gololobov
863548114d force_wait flag is no longer needed 2023-01-30 18:38:28 +01:00
SmitaRKulkarni
b22fd40d92
Merge branch 'master' into 45508_Update_strategy_for_system_sync_replica 2023-01-27 15:00:47 +01:00
Smita Kulkarni
8c885ffeae Updated checking of SYSTEM SYNC REPLICA
Implementation:
* Updated to wait for current last entry to be processed (after pulling shared log) instead of queue size becoming 0.
* Updated Subscriber to notify both queue size and removed log_entry_id.
2023-01-26 11:40:09 +01:00
Anton Popov
8e3698c91f refactoring of code near merge tree parts 2023-01-25 17:34:09 +00:00
Frank Chen
1fe9bde594 Set replica name to zero copy exclusive lock 2023-01-20 15:48:33 +08:00
mateng915
5dc81fac2b
Merge branch 'master' into replicas_status_api_optimize 2023-01-20 10:19:40 +08:00
Han Fei
3481b4d50a fix style 2023-01-16 10:41:35 +01:00
Han Fei
a258a39eb1 Merge branch 'master' into hanfei/async-cache 2023-01-13 20:17:58 +01:00
mateng0915
eae0f9957c Feature: Optimize the replica delay api logic
Description:
============
Currently if in the Whole CK instance has one table is read only status
then the API  /replicas_status will throw error, xxx table is read only

For make this monitor can work in prod env, we can catch the read only
status instead of directly throw error

Solution:
=========
Return other normal table's delay value even if the CK instance has readonly Replicatedxxx table

Please enter the commit message for your changes. Lines starting
2023-01-13 14:44:27 +08:00
Raúl Marín
9f121d5d9c
Fix background_fetches_pool_size config reload (#45189) 2023-01-12 23:41:25 +01:00
Han Fei
bef6168c05 support cache for async inserts block ids 2023-01-10 13:19:12 +01:00
alesapin
616e37ba43
Merge pull request #44847 from ClickHouse/fix_rare_race
Fix rare race which can lead to queue hang
2023-01-10 12:09:10 +01:00
Alexander Gololobov
742887ce53
Merge pull request #44718 from ClickHouse/lwd_mutation_always_sync
Make lightweight deletes always synchronous
2023-01-09 15:58:02 +01:00
Nikita Mikhaylov
857799fbca
Parallel distributed insert select with s3Cluster [3] (#44955)
* Revert "Revert "Resurrect parallel distributed insert select with s3Cluster (#41535)""

This reverts commit b8d9066004.

* Fix build

* Better

* Fix test

* Automatic style fix

Co-authored-by: robot-clickhouse <robot-clickhouse@users.noreply.github.com>
2023-01-09 13:30:32 +01:00
alesapin
da83e97b4c Fix startup issue 2023-01-06 15:36:53 +01:00
alesapin
2a5b4067e4 Review fixes 2023-01-04 15:15:14 +01:00
Alexander Gololobov
2c64f0042a Added force_wait parameter to IStorage::mutate() 2022-12-30 16:23:03 +01:00