Commit Graph

9590 Commits

Author SHA1 Message Date
Mike Kot
4c391f8e99
SYSTEM RESTORE REPLICA replica [ON CLUSTER cluster] (#13652)
* initial commit: add setting and stub

* typo

* added test stub

* fix

* wip merging new integration test and code proto

* adding steps interpreters

* adding firstly proposed solution (moving parts etc)

* added checking zookeeper path existence

* fixing the include

* fixing and sorting includes

* fixing outdated struct

* fix the name

* added ast ptr as level of indirection

* fix ref

* updating the changes

* working on test stub

* fix iterator -> reference

* revert rocksdb submodule update

* fixed show privileges test

* updated the test stub

* replaced rand() with thread_local_rng(), updated the tests

updated the test

fixed test config path

test fix

removed error messages

fixed the test

updated the test

fixed string literal

fixed literal

typo: =

* fixed the empty replica error message

* updated the test and the code with logs

* updated the possible test cases, updated

* added the code/test milestone comments

* updated the test (added more testcases)

* replaced native assert with CH one

* individual replicas recursive delete fix

* updated the AS db.name AST

* two small logging fixes

* manually generated AST fixes

* Updated the test, added the possible algo change

* Some thoughts about optimizing the solution:

ALTER MOVE PARTITION .. TO TABLE -> move to detached/ + ALTER ... ATTACH

* fix

* Removed the replica sync in test as it's invalid

* Some test tweaks

* tmp

* Rewrote the algo by using the executeQuery instead of

hand-crafting the ASTPtr.

Two questions still active.

* tr: logging active parts

* Extracted the parts moving algo into a separate helper function

* Fixed the test data and the queries slightly

* Replaced query to system.parts to direct invocation,

started building the test that breaks on various parts.

* Added the case for tables when at least one replica is alive

* Updated the test to test replicas restoration by detaching/attaching

* Altered the test to check restoration without replica restart

* Added the tables swap in the start if the server failed last time

* Hotfix when only /replicas/replica... path was deleted

* Restore ZK paths while creating a replicated MergeTree table

* Updated the docs, fixed the algo for individual replicas restoration case

* Initial parts table storage fix, tests sync fix

* Reverted individual replica restoration to general algo

* Slightly optimised getDataParts

* Trying another solution with parts detaching

* Rewrote algo without any steps, added ON CLUSTER support

* Attaching parts from other replica on restoration

* Getting part checksums from ZK

* Removed ON CLUSTER, finished working solution

* Multiple small changes after review

* Fixing parallel test

* Supporting rewritten form on cluster

* Test fix

* Moar logging

* Using source replica as checksum provider

* improve test, remove some code from parser

* Trying solution with move to detached + forget

* Moving all parts (not only Committed) to detached

* Edited docs for RESTORE REPLICA

* Re-merging

* minor fixes

Co-authored-by: Alexander Tokmakov <avtokmakov@yandex-team.ru>
2021-06-20 11:24:43 +03:00
Azat Khuzhin
c6339b4c08 Stable NOT chain formatting 2021-06-19 08:03:36 +03:00
Alexey Milovidov
29b80af6cc Skip RocksDB tests for Ordinary database 2021-06-19 00:09:50 +03:00
Kseniia Sumarokova
4fe722d1a4
Merge pull request #25431 from vitlibar/no-underscore_before-preprocessed_configs
Fix underscore before preprocessed configs.
2021-06-18 17:53:03 +03:00
alexey-milovidov
e4d80f53fa
Merge pull request #25311 from ClickHouse/fix-bad-test-client_print_query_id
Fix flaky test 01520_client_print_query_id and others.
2021-06-18 14:41:29 +03:00
Maksim Kita
822f77be74
Merge pull request #25458 from kitaisreal/low-cardinality-interval-type
DataTypeLowCardinality support DataTypeInterval tests
2021-06-18 13:16:42 +03:00
Maksim Kita
180f0aa1e0 Added tests for DataTypeLowCardinality support DataTypeInterval 2021-06-18 11:49:40 +03:00
Vladimir
6af272891a
Merge pull request #24870 from azat/mv-dist-join-fix 2021-06-18 11:18:43 +03:00
Vladimir
fab7c9c7f6
Merge pull request #24061 from vdimir/issue-23351 2021-06-18 11:07:28 +03:00
Vladimir
51e5a019ed
Merge pull request #25410 from vdimir/issue-21730 2021-06-18 11:00:26 +03:00
Maksim Kita
1b53719d0f
Merge pull request #25446 from ClickHouse/add-test-8417
Add a test for #8417
2021-06-18 10:20:09 +03:00
Nikolai Kochetov
b2620e1def
Merge pull request #24405 from amosbird/replacemorealias
Rewrite more alias columns
2021-06-18 09:10:31 +03:00
Vitaly Baranov
f766d74fb6 Fix testflows LDAP tests. 2021-06-18 06:18:40 +03:00
alexey-milovidov
85c64346d9
Merge pull request #25409 from vdimir/issue-20624
Add test from issue #20624
2021-06-18 01:52:12 +03:00
Alexey Milovidov
16da9d3002 Add a test for #8417 2021-06-18 01:45:27 +03:00
Maksim Kita
b34b66c55d
Merge pull request #25394 from ClickHouse/function-bit-position-to-array-merge
Merging #23843
2021-06-18 01:31:33 +03:00
Nikolai Kochetov
669b8a8b96
Merge pull request #25370 from ClickHouse/disable-query_plan_filter_push_down-because-of-25368
Fix lost WHERE condition in expression-push-down optimization of query plan
2021-06-17 23:07:32 +03:00
Maksim Kita
1de01ae722
Merge pull request #25417 from Avogar/turn-off-coverage
Turn off WITH_COVERAGE in build with clang-tidy
2021-06-17 22:35:58 +03:00
Maksim Kita
ca672b0a0d Function bitPositionsToArray added support for big integers 2021-06-17 22:32:39 +03:00
Vladimir
9e5ef95677
Merge pull request #25288 from amosbird/fixjoingetornull 2021-06-17 21:54:21 +03:00
alexey-milovidov
35411d1085
Merge pull request #24427 from azat/formatting-fixes
Fix formatting of negative values
2021-06-17 21:21:47 +03:00
mergify[bot]
1740e1d00d
Merge branch 'master' into fix-bad-test-client_print_query_id 2021-06-17 18:03:31 +00:00
alexey-milovidov
3a049a2bcd
Update 01676_long_clickhouse_client_autocomplete.sh 2021-06-17 20:58:37 +03:00
alexey-milovidov
66c0df99d4
Merge pull request #25323 from ClickHouse/try-to-change-signal-for-integration-rests
Use signal 15 in restart_with_latest_version
2021-06-17 20:56:00 +03:00
alexey-milovidov
496aff2118
Merge pull request #25300 from ClickHouse/simpla-agg-func-for-summing-mt
Support simple aggregate functions for SummingMergeTree.
2021-06-17 20:18:51 +03:00
alexey-milovidov
d8174cb9fd
Merge pull request #25390 from ClickHouse/add-test-23163
Add a test for #23163
2021-06-17 20:08:35 +03:00
alexey-milovidov
c48402d997
Merge pull request #23434 from amosbird/preferglobal
Add prefer_global_in_and_join setting
2021-06-17 19:48:28 +03:00
Pavel Kruglov
7cf88594da Turn off WITH_COVERAGE in build with clang-tidy 2021-06-17 17:35:36 +03:00
Kseniia Sumarokova
f9c0bfb769
Merge pull request #25199 from sand6255/MySQL-column-comments-support
MaterializeMySQL Engine column comments support
2021-06-17 17:08:53 +03:00
vdimir
c95ff5d38d
Upd 01049_join_low_card_bug_long 2021-06-17 15:55:41 +03:00
vdimir
b7c421854d
Upd 01049_join_low_card_bug_long 2021-06-17 15:44:58 +03:00
vdimir
23fb229dec
Merge branch 'update_01049_join_low_card_bug' into issue-23351 2021-06-17 15:38:40 +03:00
vdimir
e86592d0b5
Add test 01916_lowcard_dict_type 2021-06-17 15:30:22 +03:00
vdimir
8d4ccb40de
Add test from issue #20624 2021-06-17 14:20:48 +03:00
vdimir
3895f752eb
Revert "Replace engine Memory to TinyLog in 01049_join_low_card_bug.sql"
This reverts commit 64e8f8222e.
2021-06-17 13:16:26 +03:00
vdimir
ae450fb2a2
Merge branch 'master' into update_01049_join_low_card_bug 2021-06-17 13:15:15 +03:00
Nikolai Kochetov
4a496f1374 Fix lost considion in predicate push down. 2021-06-17 12:47:12 +03:00
Maksim Kita
bc86b2b738 Updated function bitPositionsToArray before merge 2021-06-17 12:20:53 +03:00
vdimir
64e8f8222e
Replace engine Memory to TinyLog in 01049_join_low_card_bug.sql 2021-06-17 12:00:33 +03:00
vdimir
8f6614bc8f
Replace engine Memory to TinyLog in 01049_join_low_card_bug_long.sql 2021-06-17 11:59:32 +03:00
Vladimir
7a56ab8b1d
Merge pull request #25299 from mga-chka/equals_op_in_sequence_functions 2021-06-17 11:45:04 +03:00
mergify[bot]
13fdefebf5
Merge branch 'master' into fixjoingetornull 2021-06-17 08:26:13 +00:00
Maksim Kita
5b2380d541
Merge branch 'function-bit-position-to-array-merge' into feature-function-bitpositionToArray 2021-06-17 10:23:41 +03:00
Maksim Kita
37770f20fc
Merge pull request #25372 from ClickHouse/add-date-name-function-merge
Merging #23085
2021-06-17 10:16:32 +03:00
Azat Khuzhin
70df1536a2 Update 01566_negate_formatting expectations 2021-06-17 10:07:05 +03:00
Azat Khuzhin
9242654231 negate: replace special case with parens
v0: negate: add parens for negative values
    Before this patch -(-1) was formatted to --1, but this is not valid
    syntax.
v2: replace special case with parens, otherwise formatting of -(-(-1))
    will be unstable, in v0 it was formatted to -(1) but -(1) is
    formatted to -1 [1].

  [1]: https://clickhouse-test-reports.s3.yandex.net/24427/4512460aeb02fec557c558eee3f8291a6297efb2/fuzzer_asan/fuzzer.log
2021-06-17 10:07:03 +03:00
Amos Bird
c8ea6527cb
Add prefer_global_in_and_join setting 2021-06-17 14:27:29 +08:00
mergify[bot]
04adc58ba1
Merge branch 'master' into try-to-change-signal-for-integration-rests 2021-06-17 06:20:40 +00:00
alexey-milovidov
5e63d28782
Update 01676_long_clickhouse_client_autocomplete.sh 2021-06-17 09:20:37 +03:00
alexey-milovidov
fffd69b023
Merge pull request #25283 from ClickHouse/if-support-int-decimal
Remove code and fix #20549
2021-06-17 09:17:42 +03:00