Commit Graph

614 Commits

Author SHA1 Message Date
alesapin
c977c33d6d Fix bug in execution of TTL GROUP BY 2021-06-27 19:18:15 +03:00
alesapin
4be4bc21e2 Fix for fix 2021-06-23 23:57:49 +03:00
alesapin
81c74435a3 Fix drop part bug 2021-06-23 22:25:30 +03:00
Mike Kot
4c391f8e99
SYSTEM RESTORE REPLICA replica [ON CLUSTER cluster] (#13652)
* initial commit: add setting and stub

* typo

* added test stub

* fix

* wip merging new integration test and code proto

* adding steps interpreters

* adding firstly proposed solution (moving parts etc)

* added checking zookeeper path existence

* fixing the include

* fixing and sorting includes

* fixing outdated struct

* fix the name

* added ast ptr as level of indirection

* fix ref

* updating the changes

* working on test stub

* fix iterator -> reference

* revert rocksdb submodule update

* fixed show privileges test

* updated the test stub

* replaced rand() with thread_local_rng(), updated the tests

updated the test

fixed test config path

test fix

removed error messages

fixed the test

updated the test

fixed string literal

fixed literal

typo: =

* fixed the empty replica error message

* updated the test and the code with logs

* updated the possible test cases, updated

* added the code/test milestone comments

* updated the test (added more testcases)

* replaced native assert with CH one

* individual replicas recursive delete fix

* updated the AS db.name AST

* two small logging fixes

* manually generated AST fixes

* Updated the test, added the possible algo change

* Some thoughts about optimizing the solution:

ALTER MOVE PARTITION .. TO TABLE -> move to detached/ + ALTER ... ATTACH

* fix

* Removed the replica sync in test as it's invalid

* Some test tweaks

* tmp

* Rewrote the algo by using the executeQuery instead of

hand-crafting the ASTPtr.

Two questions still active.

* tr: logging active parts

* Extracted the parts moving algo into a separate helper function

* Fixed the test data and the queries slightly

* Replaced query to system.parts to direct invocation,

started building the test that breaks on various parts.

* Added the case for tables when at least one replica is alive

* Updated the test to test replicas restoration by detaching/attaching

* Altered the test to check restoration without replica restart

* Added the tables swap in the start if the server failed last time

* Hotfix when only /replicas/replica... path was deleted

* Restore ZK paths while creating a replicated MergeTree table

* Updated the docs, fixed the algo for individual replicas restoration case

* Initial parts table storage fix, tests sync fix

* Reverted individual replica restoration to general algo

* Slightly optimised getDataParts

* Trying another solution with parts detaching

* Rewrote algo without any steps, added ON CLUSTER support

* Attaching parts from other replica on restoration

* Getting part checksums from ZK

* Removed ON CLUSTER, finished working solution

* Multiple small changes after review

* Fixing parallel test

* Supporting rewritten form on cluster

* Test fix

* Moar logging

* Using source replica as checksum provider

* improve test, remove some code from parser

* Trying solution with move to detached + forget

* Moving all parts (not only Committed) to detached

* Edited docs for RESTORE REPLICA

* Re-merging

* minor fixes

Co-authored-by: Alexander Tokmakov <avtokmakov@yandex-team.ru>
2021-06-20 11:24:43 +03:00
Maksim Kita
67e9b85951 Merge ext into common 2021-06-16 23:28:41 +03:00
tavplubix
e2ecc51a1f
Merge pull request #25087 from ClickHouse/srmt_remove_copypaste
Remove copypaste from StorageReplicatedMergeTree
2021-06-15 12:43:55 +03:00
alexey-milovidov
abe206f4fc
Merge pull request #25037 from nvartolomei/nv/queue-entry-wait-dead-code
Update waitForTableReplicaToProcessLogEntry comments
2021-06-12 03:11:13 +03:00
Alexander Tokmakov
3ade38df82 remove copypaste 2021-06-08 22:17:45 +03:00
Azat Khuzhin
1062d0ec91 Distinguish KILL MUTATION for different tables.
Before this patch KILL MUTATION marks mutation as canceled just by name
(and part numbers) so if you have multiple tables with the same part
name, then killing mutation for one table, will mark it as killed for
another too.

Fix this by comparing StorageID too (it is better to use StorageID over
database/table to avoid ambiguity by using UUIDs for comparing).

Here is a failure of the 01414_freeze_does_not_prevent_alters on CI [1].

  [1]: https://clickhouse-test-reports.s3.yandex.net/24069/9fb69dcf98c71a939d200cad3c8491bf43a44622/functional_stateless_tests_(ubsan).html#fail1
2021-06-08 10:51:22 +03:00
Nicolae Vartolomei
19f64b3f25 Update waitForTableReplicaToProcessLogEntry comments 2021-06-07 11:01:57 +01:00
alesapin
2ea9d998e8
Merge pull request #24960 from nvartolomei/nv/queue-entry-wait-dead-code
Delete support for waiting on queue- entries, is this dead code?
2021-06-07 12:40:34 +03:00
Nicolae Vartolomei
a9d108fc5f Delete support for waiting on queue- entries, is this dead code? 2021-06-04 12:43:46 +01:00
alesapin
4127bbbb9c Remove compatibility code with 18.12 2021-06-04 13:20:15 +03:00
alesapin
4a3fd34a9d Fix typo 2021-06-04 11:33:07 +03:00
alesapin
f9c2f6925d Don't try if source replica was dropped 2021-06-04 11:32:33 +03:00
alesapin
1a421260b3 Fix endless wait in replica clone 2021-06-04 11:22:24 +03:00
tavplubix
1aff716f18
Merge pull request #24777 from ClickHouse/fix_intersection_with_lost_part
Do not crash on intersecting virtual parts
2021-06-03 15:42:46 +03:00
alesapin
4a1bf51ad1
Merge branch 'master' into limited_fetches 2021-06-02 23:01:47 +03:00
Nikolai Kochetov
134ad18249
Merge pull request #24574 from ClickHouse/refactor-MergeTreeDataSelectExecutor
Refactor MergeTreeDataSelectExecutor
2021-06-02 13:17:54 +03:00
alesapin
1c0973162d
Merge pull request #24803 from ClickHouse/followupfix_zookeeper_timeouts
Don't capture temporary references in ZooKeeper client callbacks.
2021-06-02 01:08:17 +03:00
Alexander Tokmakov
7d7c5638a5 Merge branch 'master' into fix_intersection_with_lost_part 2021-06-01 16:32:59 +03:00
Alexander Tokmakov
5969891611 do not crash on intersecting parts 2021-06-01 16:25:23 +03:00
Nikolai Kochetov
e615858b10 Merge branch 'master' into refactor-MergeTreeDataSelectExecutor 2021-06-01 16:25:04 +03:00
alexey-milovidov
c7bd3b8674
Merge pull request #23315 from CurtizJ/empty-parts-fix
Fix waiting while dropping empty parts
2021-06-01 13:52:49 +03:00
alesapin
c3321bef30 Merge branch 'master' into limited_fetches 2021-06-01 13:09:02 +03:00
Nikolai Kochetov
b54ff696e5 Merge branch 'master' into refactor-MergeTreeDataSelectExecutor 2021-06-01 12:08:37 +03:00
alesapin
b7c4f7dff8
Merge pull request #24809 from ClickHouse/fix_kill_mutation_wait
Fix mutation wait on mutation kill
2021-06-01 10:37:24 +03:00
alesapin
4e038b2a61 Remove copy-paste and rename (review fixes) 2021-06-01 10:30:06 +03:00
tavplubix
e9ff0b6d70
Merge pull request #23657 from kssenii/poco-file-to-std-fs
Poco::File to std::filesystem
2021-05-31 23:17:02 +03:00
alesapin
18988a60ad Fix mutation wait 2021-05-31 18:03:45 +03:00
Nikolai Kochetov
afc1fe7f3d Make ContextPtr const by default. 2021-05-31 17:49:02 +03:00
alesapin
2fe75e33ee Replace Poco::Event with std future in synchronous calls in ZooKeeper client 2021-05-31 16:03:30 +03:00
Alexander Tokmakov
16647fe8ce some unrelated fixes 2021-05-31 00:29:37 +03:00
Nikolai Kochetov
a6ef5778cc
Merge pull request #24692 from amosbird/projection-fix6
Fix alter table drop projection if exists
2021-05-30 12:16:52 +03:00
Amos Bird
edd23d06ad
Fix alter table drop projection if exists 2021-05-28 21:27:36 +08:00
Anton Popov
179f172c4c fix build 2021-05-28 02:10:44 +03:00
Anton Popov
336abfa475 Merge remote-tracking branch 'upstream/master' into HEAD 2021-05-28 01:58:30 +03:00
Nikolai Kochetov
936bd8a011 Merge branch 'master' into refactor-MergeTreeDataSelectExecutor 2021-05-27 20:47:39 +03:00
Nikolai Kochetov
1aeb705b20 Fix some tests. 2021-05-27 19:53:58 +03:00
alesapin
2985def91d Add tests 2021-05-27 15:54:47 +03:00
alesapin
42b2383e01 Basic throttler implementation 2021-05-26 23:37:44 +03:00
kssenii
73f16ee9ee Merge branch 'master' of github.com:ClickHouse/ClickHouse into poco-file-to-std-fs 2021-05-26 23:08:08 +03:00
alesapin
069d45790f
Merge pull request #24157 from kssenii/modulo-partition-key
Adjust modulo function when used in partition key
2021-05-26 10:30:54 +03:00
tavplubix
84f9d38620
Merge pull request #24321 from amosbird/fixdroppartition
Fix drop partition in rare case
2021-05-26 01:34:22 +03:00
Anton Popov
7d1431f6b6 better code near dropPartition 2021-05-25 20:25:00 +03:00
tavplubix
6b228f2630
Update StorageReplicatedMergeTree.cpp 2021-05-24 15:13:42 +03:00
tavplubix
e0f214d0e9
Update StorageReplicatedMergeTree.cpp 2021-05-24 14:23:23 +03:00
kssenii
659e57e60f Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into modulo-partition-key 2021-05-23 22:46:00 +00:00
kssenii
bbf56e35ac Merge branch 'master' of github.com:ClickHouse/ClickHouse into poco-file-to-std-fs 2021-05-23 10:54:48 +03:00
kssenii
096d117f68 Review fixes 2021-05-22 00:12:46 +03:00