Alexey Zatelepin
66d49f0abc
start and stop mutations_finalizing_task during replica restart [#CLICKHOUSE-3747]
2018-08-22 16:57:30 +03:00
Alexey Zatelepin
b6aa7542d0
add BackgroundSchedulePool::TaskInfo::activateAndSchedule() method
...
Almost always it (and not activate()) is precisely what is needed.
2018-08-22 16:43:27 +03:00
Alexey Zatelepin
55d82dfc84
fix race in initialization of ReplicatedMergeTreeRestartingThread [#CLICKHOUSE-3899]
2018-08-21 18:22:55 +03:00
Alexey Milovidov
36db216abf
Preparation for extra warnings [#CLICKHOUSE-2]
2018-08-10 07:02:56 +03:00
Alexey Zatelepin
4863c5c178
fix potential segfault in MergeTreeData::delayInsertOrThrowIfNeeded [#CLICKHOUSE-3862]
2018-07-30 21:47:24 +03:00
Alexey Zatelepin
cc7950a733
don't destroy Cleanup- and AlterThread objects, just deactivate the tasks [#CLICKHOUSE-3862]
...
Otherwise segfaults are possible when the objects are recreated in the middle of the queue task execution.
2018-07-30 21:47:24 +03:00
Alexey Zatelepin
d89221c458
easier init and deactivation of BackgroundSchedulePool tasks
2018-05-31 16:18:52 +03:00
Alexey Zatelepin
f6c72f8e56
Merge branch 'master' into replicated-mergetree-mutations
...
Conflicts:
dbms/src/Common/ActionBlocker.h
dbms/src/Storages/MergeTree/ActiveDataPartSet.cpp
dbms/src/Storages/MergeTree/ActiveDataPartSet.h
dbms/src/Storages/MergeTree/MergeTreeData.h
dbms/src/Storages/MergeTree/MergeTreeDataPart.cpp
dbms/src/Storages/MergeTree/ReplicatedMergeTreeAlterThread.cpp
dbms/src/Storages/MergeTree/ReplicatedMergeTreeLogEntry.cpp
dbms/src/Storages/MergeTree/ReplicatedMergeTreeLogEntry.h
dbms/src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp
dbms/src/Storages/MergeTree/ReplicatedMergeTreeQueue.h
dbms/src/Storages/MergeTree/ReplicatedMergeTreeRestartingThread.cpp
dbms/src/Storages/StorageMergeTree.cpp
dbms/src/Storages/StorageReplicatedMergeTree.cpp
dbms/src/Storages/StorageReplicatedMergeTree.h
2018-05-28 18:37:30 +03:00
Alexey Zatelepin
1147e052e8
select and write part mutations to queue [#CLICKHOUSE-3747]
2018-05-25 23:13:00 +03:00
Alexey Zatelepin
42850f09b7
load mutations from ZK [#CLICKHOUSE-3747]
2018-05-25 23:06:15 +03:00
Alexey Zatelepin
17437a95bf
merge selection based on checking current inserts [#CLICKHOUSE-3732]
2018-05-25 23:03:56 +03:00
Vitaliy Lyudvichenko
c888903488
Add query ALTER TABLE t1 [REPLACE|ATTACH] PARTITION FROM t2
...
* Add ATTACH PARTITION FROM table for MergeTree. [#CLICKHOUSE-3546]
* Implemented replicated case on non-leader replica. [#CLICKHOUSE-3546]
* Disable merges in the dropping range. [#CLICKHOUSE-3546]
* DROP PARTITION is atomic and simpler now. [#CLICKHOUSE-3546]
* Implemented more SYSTEM queries. [#CLICKHOUSE-2931] [#CLICKHOUSE-3546]
SYSTEM queries:
RESTART REPLICAS
SYNC REPLICA db.name
STOP MERGES [db.name]
START MERGES [db.name]
STOP FETCHES [db.name]
START FETCHES [db.name]
STOP REPLICATED SENDS [db.name]
START REPLICATED SENDS [db.name]
STOP REPLICATION QUEUES [db.name]
START REPLICATION QUEUES [db.name]
* Fixed a bunch of bugs in REPLACE PARTITION. [#CLICKHOUSE-3546]
* Add tests for REPLACE PARTITION and SYSTEM. [#CLICKHOUSE-3546]
* Add system.part_log logging. [#CLICKHOUSE-3546]
* Fixed long wait in SYNC REPLICA. [#CLICKHOUSE-3546]
* Add requested changes. [#CLICKHOUSE-3546]
Fixed clickhouse-client bad return code.
* Add requested chenges. [#CLICKHOUSE-3546]
* Add requested chenges. [#CLICKHOUSE-3546]
2018-05-21 16:49:54 +03:00
alexey-milovidov
517318334a
Merge pull request #1722 from silviucpp/background-schedule-pool-fix
...
Background schedule pool fixed
2018-05-19 19:00:43 +03:00
Alex Zatelepin
7244dfe1f5
better detached part name
2018-05-08 15:47:06 +03:00
Alexey Zatelepin
6047db7ea2
fix removing failed quorum parts at startup (tryGetChilden actually returns error code, not success flag)
2018-05-07 23:31:28 +03:00
Silviu Caragea
32dd4554da
Merge remote-tracking branch 'github/master' into background-schedule-pool-fix
...
# Conflicts:
# dbms/src/Storages/StorageReplicatedMergeTree.cpp
2018-04-19 09:29:49 +03:00
Vitaliy Lyudvichenko
bf832b3ea6
Fixed OPTIMIZE after RENAME of replicated table. [#CLICKHOUSE-2]
2018-04-17 20:59:42 +03:00
Silviu Caragea
c547c5a7b0
Merge remote-tracking branch 'origin/background-schedule-pool-fix'
...
# Conflicts:
# dbms/src/Common/ZooKeeper/LeaderElection.h
# dbms/src/Common/ZooKeeper/ZooKeeper.cpp
# dbms/src/Storages/MergeTree/ReplicatedMergeTreeAlterThread.cpp
# dbms/src/Storages/MergeTree/ReplicatedMergeTreeCleanupThread.cpp
# dbms/src/Storages/MergeTree/ReplicatedMergeTreeCleanupThread.h
# dbms/src/Storages/MergeTree/ReplicatedMergeTreePartCheckThread.cpp
# dbms/src/Storages/MergeTree/ReplicatedMergeTreeRestartingThread.cpp
# dbms/src/Storages/StorageReplicatedMergeTree.cpp
2018-04-10 16:20:14 +03:00
Alexey Milovidov
0a4b349f73
Merge branch 'master' of github.com:yandex/ClickHouse
2018-04-07 00:47:18 +03:00
Alexey Milovidov
4e5e0fa664
ReplicatedMergeTree: Better diagnostics [#CLICKHOUSE-2]
2018-04-07 00:46:57 +03:00
Alexey Zatelepin
3a8ca70d1f
fix races in leader election [#CLICKHOUSE-3533]
2018-04-06 19:10:23 +03:00
Alexey Milovidov
2310bd7947
Miscellaneous [#CLICKHOUSE-2]
2018-04-03 20:35:48 +03:00
Silviu Caragea
1418e339e4
Rename merge_selecting_handle with merge_selecting_task_handle for consistency
2018-03-26 23:18:19 +03:00
Alexey Milovidov
522bf01db9
Using different ZooKeeper library (development) [#CLICKHOUSE-2]
2018-03-25 03:15:52 +03:00
Alexey Milovidov
58f32f98d7
Using different ZooKeeper library (development) [#CLICKHOUSE-2]
2018-03-24 23:11:46 +03:00
Alexey Milovidov
10be6ca8f8
Rewriting ZooKeeper library [#CLICKHOUSE-2]
2018-03-24 03:45:04 +03:00
Vitaliy Lyudvichenko
883de7a2a6
startup() waits for initialization of ReplicatedMergeTreeRestartingThread. [#CLICKHOUSE-2]
2018-03-23 00:26:46 +03:00
Silviu Caragea
6629b03af9
Fixed few of the observation of the code review
2018-03-22 17:09:30 +02:00
Silviu Caragea
b24a4b2111
Add back the buggy BackgroundSchedulePool
2018-03-22 17:07:12 +02:00
Vitaliy Lyudvichenko
8028ba0de9
Refactoring of zkutil::ZooKeeper::multi* functions. [#CLICKHOUSE-2]
2018-03-14 16:29:03 +03:00
Alexey Zatelepin
6d5ce3fa9a
revert BackgroundSchedulePool usage in Replicated tables
2017-12-22 21:25:54 +03:00
Alexey Zatelepin
2340e55d0e
revert to old behaviour of getPartIfExists() [#CLICKHOUSE-3508]
2017-12-20 20:04:14 +03:00
Alexey Milovidov
6c8b5e29d8
Additions after merge [#CLICKHOUSE-2].
2017-11-20 00:17:58 +03:00
alexey-milovidov
fc0a51f1b9
Merge branch 'master' into fix4
2017-11-19 23:57:44 +03:00
Alexey Milovidov
94d3b49e99
BackgroundSchedulePool: miscellaneous [#CLICKHOUSE-2].
2017-11-19 03:27:43 +03:00
Alexey Milovidov
9b92c0fecd
BackgroundSchedulePool: miscellaneous [#CLICKHOUSE-2].
2017-11-19 03:22:13 +03:00
Silviu Caragea
dfecd0b2f2
Merge with master
2017-11-18 10:16:06 +02:00
Alexey Milovidov
729b114c15
Removed unsupported functionality [#CLICKHOUSE-2].
2017-11-18 05:34:00 +03:00
Alexey Milovidov
076e0ae014
More clear [#CLICKHOUSE-2].
2017-11-18 00:20:28 +03:00
Alexey Milovidov
7c9ba15ba1
Style [#CLICKHOUSE-2].
2017-11-17 23:42:03 +03:00
Silviu Caragea
64619c614f
Basic idea is :
...
- We add BackgroundSchedulePool which can execute a function at a specific point in time. Basically all tasks are added in a queue and precessed by worker threads.
- The most important difference between this and BackgroundProcessingPool is that we have the guarantee that the same function is not executed from many workers in the same time
- Each of the following classes instead starting a thread will register a task in BackgroundSchedulePool and when they need to run will call schedule or scheduleAfter(duration) functions
This commit is moving all threads created by ReplicatedMergeTree to BackgroundSchedulePool tasks
NOTE: I did a minimum number of changes to be much simple to review the code
2017-11-17 10:58:35 +02:00
proller
3b58e9cd67
Fix atomic usage
2017-11-01 20:15:05 +03:00
proller
c5dcde0c78
fix atomic usage
2017-11-01 16:58:02 +03:00
Vitaliy Lyudvichenko
b5c7eaa80e
Force data parts cleanup before repeat fetch. [#CLICKHOUSE-3178]
2017-10-26 17:16:06 +03:00
Alexey Zatelepin
c61d4106e8
allow loading all MergeTree settings from a query [#CLICKHOUSE-3000]
2017-10-20 21:28:45 +03:00
Vitaliy Lyudvichenko
62ea1133dd
Disable part sendings and fetches before ALTER. [#CLICKHOUSE-3343]
2017-10-12 23:26:54 +03:00
Alexey Zatelepin
df58949a68
make exit out of the loop possible when ZooKeeper is unavailable [#CLICKHOUSE-3259]
2017-09-20 06:05:20 +03:00
Vitaliy Lyudvichenko
10c14cfccb
Small code enhancements according to clang-tidy. [#CLICKHOUSE-2931]
2017-08-04 17:35:36 +03:00
Alexey Zatelepin
4a98d83a3b
introduce replicated_can_become_leader merge_tree setting [#CLICKHOUSE-3109]
2017-07-11 20:44:35 +03:00
Alexey Milovidov
061c214c16
Fixed error [#CLICKHOUSE-3074].
2017-06-23 09:57:55 +03:00