Commit Graph

505 Commits

Author SHA1 Message Date
zxc111
cd97478c06 check sampling condition when create table 2021-07-12 00:38:20 +08:00
Anton Popov
9f1ffa777f remove unused code 2021-07-08 15:06:33 +03:00
Anton Popov
9071ecd428 fix alter of settings in MergeTree 2021-07-05 15:44:58 +03:00
alesapin
cf2fc94d9d
Merge pull request #25820 from ClickHouse/better_remove_empty_parts
Replace lost parts with empty parts instead of hacking replication queue
2021-07-04 18:55:49 +03:00
Aleksei Semiglazov
9a5365fc41
CLICKHOUSE-784: reset merge tree setting
Add an ability to reset custom setting to default and remove it from table's metadata.
This will allow to rollback the change without knowing the system/config's default.

Signed-off-by: Aleksei Semiglazov <asemiglazov@cloudflare.com>
2021-07-01 20:09:54 +01:00
mergify[bot]
98fa9f7951
Merge branch 'master' into better_remove_empty_parts 2021-07-01 11:32:43 +00:00
alesapin
94d4d32e4c
Merge pull request #25783 from ClickHouse/followup_drop_part_fix
Relax check during DROP PART
2021-06-30 12:25:42 +03:00
mergify[bot]
8182b6e890
Merge branch 'master' into better_remove_empty_parts 2021-06-30 08:42:26 +00:00
alesapin
0af53ace55 Fix comment 2021-06-29 22:49:41 +03:00
alesapin
ca1d21b663
Update src/Storages/MergeTree/MergeTreeData.cpp
Co-authored-by: tavplubix <avtokmakov@yandex-team.ru>
2021-06-29 22:48:46 +03:00
alesapin
a6834213a1 Add tests 2021-06-29 22:47:54 +03:00
Nikolai Kochetov
d5bd2b1fa9
Merge pull request #25719 from ClickHouse/remove-PrewhereDAGInfo
Remove PrewhereDAGInfo.
2021-06-29 18:58:03 +03:00
alesapin
7680eab0e4 one more fix 2021-06-29 13:03:00 +03:00
alesapin
941a6e539f Fix 2021-06-29 10:36:20 +03:00
alesapin
d20feeb42c Relax check during DROP PART 2021-06-28 15:05:22 +03:00
alesapin
4c213b639f
Merge pull request #25548 from nikitamikhaylov/background-processing
A little improvement in BackgroundJobsExecutor
2021-06-28 13:24:21 +03:00
Nikolai Kochetov
3544b059ce Merge branch 'master' into remove-PrewhereDAGInfo 2021-06-25 17:52:03 +03:00
Nikolai Kochetov
6bc0a628cd Remove PrewhereDAGInfo. 2021-06-25 17:49:28 +03:00
alesapin
e6da53dc46 better 2021-06-24 00:09:06 +03:00
alesapin
24876291d1 Followup fix 2021-06-24 00:01:07 +03:00
alesapin
4be4bc21e2 Fix for fix 2021-06-23 23:57:49 +03:00
alesapin
81c74435a3 Fix drop part bug 2021-06-23 22:25:30 +03:00
Nikita Mikhaylov
c66a3b22b5 done 2021-06-22 23:24:47 +00:00
Mike Kot
4c391f8e99
SYSTEM RESTORE REPLICA replica [ON CLUSTER cluster] (#13652)
* initial commit: add setting and stub

* typo

* added test stub

* fix

* wip merging new integration test and code proto

* adding steps interpreters

* adding firstly proposed solution (moving parts etc)

* added checking zookeeper path existence

* fixing the include

* fixing and sorting includes

* fixing outdated struct

* fix the name

* added ast ptr as level of indirection

* fix ref

* updating the changes

* working on test stub

* fix iterator -> reference

* revert rocksdb submodule update

* fixed show privileges test

* updated the test stub

* replaced rand() with thread_local_rng(), updated the tests

updated the test

fixed test config path

test fix

removed error messages

fixed the test

updated the test

fixed string literal

fixed literal

typo: =

* fixed the empty replica error message

* updated the test and the code with logs

* updated the possible test cases, updated

* added the code/test milestone comments

* updated the test (added more testcases)

* replaced native assert with CH one

* individual replicas recursive delete fix

* updated the AS db.name AST

* two small logging fixes

* manually generated AST fixes

* Updated the test, added the possible algo change

* Some thoughts about optimizing the solution:

ALTER MOVE PARTITION .. TO TABLE -> move to detached/ + ALTER ... ATTACH

* fix

* Removed the replica sync in test as it's invalid

* Some test tweaks

* tmp

* Rewrote the algo by using the executeQuery instead of

hand-crafting the ASTPtr.

Two questions still active.

* tr: logging active parts

* Extracted the parts moving algo into a separate helper function

* Fixed the test data and the queries slightly

* Replaced query to system.parts to direct invocation,

started building the test that breaks on various parts.

* Added the case for tables when at least one replica is alive

* Updated the test to test replicas restoration by detaching/attaching

* Altered the test to check restoration without replica restart

* Added the tables swap in the start if the server failed last time

* Hotfix when only /replicas/replica... path was deleted

* Restore ZK paths while creating a replicated MergeTree table

* Updated the docs, fixed the algo for individual replicas restoration case

* Initial parts table storage fix, tests sync fix

* Reverted individual replica restoration to general algo

* Slightly optimised getDataParts

* Trying another solution with parts detaching

* Rewrote algo without any steps, added ON CLUSTER support

* Attaching parts from other replica on restoration

* Getting part checksums from ZK

* Removed ON CLUSTER, finished working solution

* Multiple small changes after review

* Fixing parallel test

* Supporting rewritten form on cluster

* Test fix

* Moar logging

* Using source replica as checksum provider

* improve test, remove some code from parser

* Trying solution with move to detached + forget

* Moving all parts (not only Committed) to detached

* Edited docs for RESTORE REPLICA

* Re-merging

* minor fixes

Co-authored-by: Alexander Tokmakov <avtokmakov@yandex-team.ru>
2021-06-20 11:24:43 +03:00
Maksim Kita
67e9b85951 Merge ext into common 2021-06-16 23:28:41 +03:00
Kseniia Sumarokova
d6240f2bd4
Merge pull request #25309 from azat/fs-fix
Catch ErrnoException during parts cleaning
2021-06-16 22:03:54 +03:00
Nikolai Kochetov
8bf55ae2b5
Merge pull request #25166 from ClickHouse/always-detach-parts-with-wrong-partition-id
Always detach parts with wrong partition ID
2021-06-16 11:06:07 +03:00
Azat Khuzhin
de9689379f Catch ErrnoException during parts cleaning
Avoids noisy messages like:

    2021.06.15 09:27:35.546982 [ 58176 ] {} <Error> auto DB::IBackgroundJobExecutor::jobExecutingTask()::(anonymous class)::operator()() const: Code: 481, e.displayText() = DB::ErrnoException: Cannot check modification time for file: /var/lib/clickhouse/store/c52/c52619f8-99bd-435e-8526-19f899bdb35e/tmp_insert_202106_7010_7010_0/, errno: 2, strerror: No such file or directory, Stack trace (when copying this message, always include the lines below):

    0. DB::Exception::Exception() @ 0x903209a in /usr/lib/debug/.build-id/be/ea088a1830cb2f.debug
    1. DB::throwFromErrnoWithPath() @ 0x9033089 in /usr/lib/debug/.build-id/be/ea088a1830cb2f.debug
    2. FS::getModificationTime() @ 0x907b166 in /usr/lib/debug/.build-id/be/ea088a1830cb2f.debug
    3. DB::DiskLocal::getLastModified() @ 0xf9a6669 in /usr/lib/debug/.build-id/be/ea088a1830cb2f.debug
    4. DB::MergeTreeData::clearOldTemporaryDirectories(long) @ 0x1069a6de in /usr/lib/debug/.build-id/be/ea088a1830cb2f.debug
2021-06-16 06:56:55 +03:00
tavplubix
e2ecc51a1f
Merge pull request #25087 from ClickHouse/srmt_remove_copypaste
Remove copypaste from StorageReplicatedMergeTree
2021-06-15 12:43:55 +03:00
alexey-milovidov
30f0124e77
Merge branch 'master' into always-detach-parts-with-wrong-partition-id 2021-06-12 02:55:55 +03:00
Nikolai Kochetov
79c69ea316 Fix build. 2021-06-11 11:29:39 +03:00
Nikolai Kochetov
6222fcffe0 Never remove parts on start at all. Always move. 2021-06-10 15:47:55 +03:00
Nikolai Kochetov
aa90309abf Never remove parts on start at all. Always move. 2021-06-10 15:46:03 +03:00
Nikolai Kochetov
847dfb9914 Merge remote-tracking branch 'origin' into always-detach-parts-with-wrong-partition-id 2021-06-10 12:37:46 +03:00
Nikolai Kochetov
d52b87d47d Always detach proken parts with wrong partition id. 2021-06-10 12:37:33 +03:00
Amos Bird
4e358b28f1
Disable projection with final is used 2021-06-10 17:11:17 +08:00
Alexander Tokmakov
cef22688ff make code less ugly 2021-06-09 15:36:47 +03:00
Alexander Tokmakov
3ade38df82 remove copypaste 2021-06-08 22:17:45 +03:00
tavplubix
1aff716f18
Merge pull request #24777 from ClickHouse/fix_intersection_with_lost_part
Do not crash on intersecting virtual parts
2021-06-03 15:42:46 +03:00
Alexander Tokmakov
7d7c5638a5 Merge branch 'master' into fix_intersection_with_lost_part 2021-06-01 16:32:59 +03:00
Alexander Tokmakov
5969891611 do not crash on intersecting parts 2021-06-01 16:25:23 +03:00
Nikolai Kochetov
e615858b10 Merge branch 'master' into refactor-MergeTreeDataSelectExecutor 2021-06-01 16:25:04 +03:00
alexey-milovidov
c7bd3b8674
Merge pull request #23315 from CurtizJ/empty-parts-fix
Fix waiting while dropping empty parts
2021-06-01 13:52:49 +03:00
Nikolai Kochetov
b54ff696e5 Merge branch 'master' into refactor-MergeTreeDataSelectExecutor 2021-06-01 12:08:37 +03:00
tavplubix
e9ff0b6d70
Merge pull request #23657 from kssenii/poco-file-to-std-fs
Poco::File to std::filesystem
2021-05-31 23:17:02 +03:00
Nikolai Kochetov
b9aa404508 Review fixes. 2021-05-31 21:25:43 +03:00
Nikolai Kochetov
afc1fe7f3d Make ContextPtr const by default. 2021-05-31 17:49:02 +03:00
Alexander Tokmakov
16647fe8ce some unrelated fixes 2021-05-31 00:29:37 +03:00
kssenii
2a631aaf08 Final fixes 2021-05-29 00:34:44 +03:00
Anton Popov
336abfa475 Merge remote-tracking branch 'upstream/master' into HEAD 2021-05-28 01:58:30 +03:00