Alexey Zatelepin
2e77c508ad
add comments [#CLICKHOUSE-3747]
2018-06-04 14:43:09 +03:00
alexey-milovidov
02f7277dc7
Update ReplicatedMergeTreeQueue.h
2018-06-01 21:06:43 +03:00
Alexey Zatelepin
c9bb986330
Merge branch 'master' into replicated-mergetree-mutations
...
Conflicts:
dbms/src/Storages/MergeTree/AbandonableLockInZooKeeper.h
dbms/src/Storages/MergeTree/ReplicatedMergeTreePartCheckThread.cpp
dbms/src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp
dbms/src/Storages/MergeTree/ReplicatedMergeTreeQueue.h
dbms/src/Storages/StorageReplicatedMergeTree.cpp
2018-05-31 18:52:42 +03:00
Vitaliy Lyudvichenko
dc5de3bbec
More optimal conflicts check in future_parts. [#CLICKHOUSE-2]
2018-05-31 17:14:50 +03:00
Alexey Zatelepin
d89221c458
easier init and deactivation of BackgroundSchedulePool tasks
2018-05-31 16:18:52 +03:00
Alexey Zatelepin
f6c72f8e56
Merge branch 'master' into replicated-mergetree-mutations
...
Conflicts:
dbms/src/Common/ActionBlocker.h
dbms/src/Storages/MergeTree/ActiveDataPartSet.cpp
dbms/src/Storages/MergeTree/ActiveDataPartSet.h
dbms/src/Storages/MergeTree/MergeTreeData.h
dbms/src/Storages/MergeTree/MergeTreeDataPart.cpp
dbms/src/Storages/MergeTree/ReplicatedMergeTreeAlterThread.cpp
dbms/src/Storages/MergeTree/ReplicatedMergeTreeLogEntry.cpp
dbms/src/Storages/MergeTree/ReplicatedMergeTreeLogEntry.h
dbms/src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp
dbms/src/Storages/MergeTree/ReplicatedMergeTreeQueue.h
dbms/src/Storages/MergeTree/ReplicatedMergeTreeRestartingThread.cpp
dbms/src/Storages/StorageMergeTree.cpp
dbms/src/Storages/StorageReplicatedMergeTree.cpp
dbms/src/Storages/StorageReplicatedMergeTree.h
2018-05-28 18:37:30 +03:00
Alexey Zatelepin
2b80fbf972
DROP PARTITION now works with mutated parts [#CLICKHOUSE-3747]
2018-05-25 23:13:00 +03:00
Alexey Zatelepin
fdb33d8f3c
execute mutation log entries [#CLICKHOUSE-3747]
2018-05-25 23:13:00 +03:00
Alexey Zatelepin
1147e052e8
select and write part mutations to queue [#CLICKHOUSE-3747]
2018-05-25 23:13:00 +03:00
Alexey Zatelepin
2721daf782
don't select merges of parts with different mutation versions [#CLICKHOUSE-3747]
2018-05-25 23:06:28 +03:00
Alexey Zatelepin
4c661df146
notify relevant events directly from queue [#CLICKHOUSE-3747]
2018-05-25 23:06:23 +03:00
Alexey Zatelepin
42850f09b7
load mutations from ZK [#CLICKHOUSE-3747]
2018-05-25 23:06:15 +03:00
Alexey Zatelepin
17437a95bf
merge selection based on checking current inserts [#CLICKHOUSE-3732]
2018-05-25 23:03:56 +03:00
Vitaliy Lyudvichenko
c888903488
Add query ALTER TABLE t1 [REPLACE|ATTACH] PARTITION FROM t2
...
* Add ATTACH PARTITION FROM table for MergeTree. [#CLICKHOUSE-3546]
* Implemented replicated case on non-leader replica. [#CLICKHOUSE-3546]
* Disable merges in the dropping range. [#CLICKHOUSE-3546]
* DROP PARTITION is atomic and simpler now. [#CLICKHOUSE-3546]
* Implemented more SYSTEM queries. [#CLICKHOUSE-2931] [#CLICKHOUSE-3546]
SYSTEM queries:
RESTART REPLICAS
SYNC REPLICA db.name
STOP MERGES [db.name]
START MERGES [db.name]
STOP FETCHES [db.name]
START FETCHES [db.name]
STOP REPLICATED SENDS [db.name]
START REPLICATED SENDS [db.name]
STOP REPLICATION QUEUES [db.name]
START REPLICATION QUEUES [db.name]
* Fixed a bunch of bugs in REPLACE PARTITION. [#CLICKHOUSE-3546]
* Add tests for REPLACE PARTITION and SYSTEM. [#CLICKHOUSE-3546]
* Add system.part_log logging. [#CLICKHOUSE-3546]
* Fixed long wait in SYNC REPLICA. [#CLICKHOUSE-3546]
* Add requested changes. [#CLICKHOUSE-3546]
Fixed clickhouse-client bad return code.
* Add requested chenges. [#CLICKHOUSE-3546]
* Add requested chenges. [#CLICKHOUSE-3546]
2018-05-21 16:49:54 +03:00
Silviu Caragea
c547c5a7b0
Merge remote-tracking branch 'origin/background-schedule-pool-fix'
...
# Conflicts:
# dbms/src/Common/ZooKeeper/LeaderElection.h
# dbms/src/Common/ZooKeeper/ZooKeeper.cpp
# dbms/src/Storages/MergeTree/ReplicatedMergeTreeAlterThread.cpp
# dbms/src/Storages/MergeTree/ReplicatedMergeTreeCleanupThread.cpp
# dbms/src/Storages/MergeTree/ReplicatedMergeTreeCleanupThread.h
# dbms/src/Storages/MergeTree/ReplicatedMergeTreePartCheckThread.cpp
# dbms/src/Storages/MergeTree/ReplicatedMergeTreeRestartingThread.cpp
# dbms/src/Storages/StorageReplicatedMergeTree.cpp
2018-04-10 16:20:14 +03:00
Silviu Caragea
0aa9b9efbe
Use consistent names for the task variables (change from next_update_task_handle to merge_selecting_handle) (reverted from commit 50992843c6)
2018-03-26 23:12:17 +03:00
Silviu Caragea
50992843c6
Use consistent names for the task variables (change from next_update_task_handle to merge_selecting_handle)
2018-03-26 22:59:58 +03:00
Silviu Caragea
31874ed172
Use consistent names for the task variables
2018-03-26 22:37:47 +03:00
Silviu Caragea
b24a4b2111
Add back the buggy BackgroundSchedulePool
2018-03-22 17:07:12 +02:00
Alexey Milovidov
d690c24198
Using different ZooKeeper library (development) [#CLICKHOUSE-2]
2018-03-22 00:40:53 +03:00
Alexey Milovidov
2e371822ad
Changed low level data race to high level data race [#CLICKHOUSE-2]
2018-03-03 19:46:32 +03:00
Alexey Milovidov
3b0d713120
Cleanup semantics [#CLICKHOUSE-2]
2018-03-03 19:26:06 +03:00
Alexey Zatelepin
6d5ce3fa9a
revert BackgroundSchedulePool usage in Replicated tables
2017-12-22 21:25:54 +03:00
Alexey Zatelepin
01d42242b1
mark queue as dirty if status of ZK operation is unknown [#CLICKHOUSE-3405]
2017-11-28 17:09:53 +03:00
Alexey Zatelepin
78aaa42ddd
add idempotent retries of updating the replica queue in ZooKeeper [#CLICKHOUSE-3405]
2017-11-23 16:19:29 +03:00
Silviu Caragea
64619c614f
Basic idea is :
...
- We add BackgroundSchedulePool which can execute a function at a specific point in time. Basically all tasks are added in a queue and precessed by worker threads.
- The most important difference between this and BackgroundProcessingPool is that we have the guarantee that the same function is not executed from many workers in the same time
- Each of the following classes instead starting a thread will register a task in BackgroundSchedulePool and when they need to run will call schedule or scheduleAfter(duration) functions
This commit is moving all threads created by ReplicatedMergeTree to BackgroundSchedulePool tasks
NOTE: I did a minimum number of changes to be much simple to review the code
2017-11-17 10:58:35 +02:00
Alexey Zatelepin
2f80c150db
save MergeTree data format version and parse part names depending on it [#CLICKHOUSE-3000]
2017-09-26 20:40:02 +03:00
Alexey Milovidov
2af8747164
Removed obsolete code [#CLICKHOUSE-3270].
2017-09-01 21:21:01 +03:00
Vitaliy Lyudvichenko
53adb891b3
Add replicated DROP COLUMN FROM partition for optimistic case. [#CLICKHOUSE-2896]
2017-06-22 00:56:38 +03:00
Alexey Milovidov
05accbdcbe
Removed cyclic dependency [#DEVTOOLS-3381].
2017-06-19 23:06:35 +03:00
Vitaliy Lyudvichenko
3a2c1a195d
Add ZooKeeper session recovering and trashy integration test. [#CLICKHOUSE-5]
2017-06-01 13:49:20 +03:00
Vitaliy Lyudvichenko
1d00f45dd3
Detect race condition between fetches and merges of the same part. [#CLICKHOUSE-2989]
2017-05-17 21:44:16 +04:00
Alexey Zatelepin
2bd089d589
start counting absolute replica delay when there are problems with queue update [#CLICKHOUSE-2141]
2017-04-17 18:06:12 +03:00
f1yegor
0b243a8b01
translate comments
2017-04-16 23:17:32 +03:00
Alexey Milovidov
310736cd7e
Moved headers and sources to same place [#CLICKHOUSE-3].
2017-04-01 12:22:42 +03:00
