thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-10-06 08:30:54 +00:00
Code Issues Packages Projects Releases Wiki Activity
17,723 Commits 8,977 Branches 1,768 Tags 2 GiB
b6aa7542d0
Commit Graph

107 Commits

Author SHA1 Message Date
Alexey Zatelepin
b6aa7542d0 add BackgroundSchedulePool::TaskInfo::activateAndSchedule() method 
Almost always it (and not activate()) is precisely what is needed.
 2018-08-22 16:43:27 +03:00
Alexey Zatelepin
55d82dfc84 fix race in initialization of ReplicatedMergeTreeRestartingThread [#CLICKHOUSE-3899] 2018-08-21 18:22:55 +03:00
Alexey Milovidov
36db216abf Preparation for extra warnings [#CLICKHOUSE-2] 2018-08-10 07:02:56 +03:00
Alexey Zatelepin
4863c5c178 fix potential segfault in MergeTreeData::delayInsertOrThrowIfNeeded [#CLICKHOUSE-3862] 2018-07-30 21:47:24 +03:00
Alexey Zatelepin
cc7950a733 don't destroy Cleanup- and AlterThread objects, just deactivate the tasks [#CLICKHOUSE-3862] 
Otherwise segfaults are possible when the objects are recreated in the middle of the queue task execution.
 2018-07-30 21:47:24 +03:00
Alexey Zatelepin
d89221c458 easier init and deactivation of BackgroundSchedulePool tasks 2018-05-31 16:18:52 +03:00
Alexey Zatelepin
f6c72f8e56 Merge branch 'master' into replicated-mergetree-mutations 
Conflicts:
	dbms/src/Common/ActionBlocker.h
	dbms/src/Storages/MergeTree/ActiveDataPartSet.cpp
	dbms/src/Storages/MergeTree/ActiveDataPartSet.h
	dbms/src/Storages/MergeTree/MergeTreeData.h
	dbms/src/Storages/MergeTree/MergeTreeDataPart.cpp
	dbms/src/Storages/MergeTree/ReplicatedMergeTreeAlterThread.cpp
	dbms/src/Storages/MergeTree/ReplicatedMergeTreeLogEntry.cpp
	dbms/src/Storages/MergeTree/ReplicatedMergeTreeLogEntry.h
	dbms/src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp
	dbms/src/Storages/MergeTree/ReplicatedMergeTreeQueue.h
	dbms/src/Storages/MergeTree/ReplicatedMergeTreeRestartingThread.cpp
	dbms/src/Storages/StorageMergeTree.cpp
	dbms/src/Storages/StorageReplicatedMergeTree.cpp
	dbms/src/Storages/StorageReplicatedMergeTree.h
 2018-05-28 18:37:30 +03:00
Alexey Zatelepin
1147e052e8 select and write part mutations to queue [#CLICKHOUSE-3747] 2018-05-25 23:13:00 +03:00
Alexey Zatelepin
42850f09b7 load mutations from ZK [#CLICKHOUSE-3747] 2018-05-25 23:06:15 +03:00
Alexey Zatelepin
17437a95bf merge selection based on checking current inserts [#CLICKHOUSE-3732] 2018-05-25 23:03:56 +03:00
Vitaliy Lyudvichenko
c888903488
Add query ALTER TABLE t1 [REPLACE|ATTACH] PARTITION FROM t2 
* Add ATTACH PARTITION FROM table for MergeTree. [#CLICKHOUSE-3546]

* Implemented replicated case on non-leader replica. [#CLICKHOUSE-3546]

* Disable merges in the dropping range. [#CLICKHOUSE-3546]

* DROP PARTITION is atomic and simpler now. [#CLICKHOUSE-3546]

* Implemented more SYSTEM queries. [#CLICKHOUSE-2931] [#CLICKHOUSE-3546]

SYSTEM queries:
RESTART REPLICAS
SYNC REPLICA db.name
STOP MERGES [db.name]
START MERGES [db.name]
STOP FETCHES [db.name]
START FETCHES [db.name]
STOP REPLICATED SENDS [db.name]
START REPLICATED SENDS [db.name]
STOP REPLICATION QUEUES [db.name]
START REPLICATION QUEUES [db.name]

* Fixed a bunch of bugs in REPLACE PARTITION. [#CLICKHOUSE-3546]

* Add tests for REPLACE PARTITION and SYSTEM. [#CLICKHOUSE-3546]

* Add system.part_log logging. [#CLICKHOUSE-3546]

* Fixed long wait in SYNC REPLICA. [#CLICKHOUSE-3546]

* Add requested changes. [#CLICKHOUSE-3546]

Fixed clickhouse-client bad return code.

* Add requested chenges. [#CLICKHOUSE-3546]

* Add requested chenges. [#CLICKHOUSE-3546]
 2018-05-21 16:49:54 +03:00
alexey-milovidov
517318334a
Merge pull request #1722 from silviucpp/background-schedule-pool-fix 
Background schedule pool fixed
 2018-05-19 19:00:43 +03:00
Alex Zatelepin
7244dfe1f5
better detached part name 2018-05-08 15:47:06 +03:00
Alexey Zatelepin
6047db7ea2 fix removing failed quorum parts at startup (tryGetChilden actually returns error code, not success flag) 2018-05-07 23:31:28 +03:00
Silviu Caragea
32dd4554da Merge remote-tracking branch 'github/master' into background-schedule-pool-fix 
# Conflicts:
#	dbms/src/Storages/StorageReplicatedMergeTree.cpp
 2018-04-19 09:29:49 +03:00
Vitaliy Lyudvichenko
bf832b3ea6 Fixed OPTIMIZE after RENAME of replicated table. [#CLICKHOUSE-2] 2018-04-17 20:59:42 +03:00
Silviu Caragea
c547c5a7b0 Merge remote-tracking branch 'origin/background-schedule-pool-fix' 
# Conflicts:
#	dbms/src/Common/ZooKeeper/LeaderElection.h
#	dbms/src/Common/ZooKeeper/ZooKeeper.cpp
#	dbms/src/Storages/MergeTree/ReplicatedMergeTreeAlterThread.cpp
#	dbms/src/Storages/MergeTree/ReplicatedMergeTreeCleanupThread.cpp
#	dbms/src/Storages/MergeTree/ReplicatedMergeTreeCleanupThread.h
#	dbms/src/Storages/MergeTree/ReplicatedMergeTreePartCheckThread.cpp
#	dbms/src/Storages/MergeTree/ReplicatedMergeTreeRestartingThread.cpp
#	dbms/src/Storages/StorageReplicatedMergeTree.cpp
 2018-04-10 16:20:14 +03:00
Alexey Milovidov
0a4b349f73 Merge branch 'master' of github.com:yandex/ClickHouse 2018-04-07 00:47:18 +03:00
Alexey Milovidov
4e5e0fa664 ReplicatedMergeTree: Better diagnostics [#CLICKHOUSE-2] 2018-04-07 00:46:57 +03:00
Alexey Zatelepin
3a8ca70d1f fix races in leader election [#CLICKHOUSE-3533] 2018-04-06 19:10:23 +03:00
Alexey Milovidov
2310bd7947 Miscellaneous [#CLICKHOUSE-2] 2018-04-03 20:35:48 +03:00
Silviu Caragea
1418e339e4 Rename merge_selecting_handle with merge_selecting_task_handle for consistency 2018-03-26 23:18:19 +03:00
Alexey Milovidov
522bf01db9 Using different ZooKeeper library (development) [#CLICKHOUSE-2] 2018-03-25 03:15:52 +03:00
Alexey Milovidov
58f32f98d7 Using different ZooKeeper library (development) [#CLICKHOUSE-2] 2018-03-24 23:11:46 +03:00
Alexey Milovidov
10be6ca8f8 Rewriting ZooKeeper library [#CLICKHOUSE-2] 2018-03-24 03:45:04 +03:00
Vitaliy Lyudvichenko
883de7a2a6 startup() waits for initialization of ReplicatedMergeTreeRestartingThread. [#CLICKHOUSE-2] 2018-03-23 00:26:46 +03:00
Silviu Caragea
6629b03af9 Fixed few of the observation of the code review 2018-03-22 17:09:30 +02:00
Silviu Caragea
b24a4b2111 Add back the buggy BackgroundSchedulePool 2018-03-22 17:07:12 +02:00
Vitaliy Lyudvichenko
8028ba0de9 Refactoring of zkutil::ZooKeeper::multi* functions. [#CLICKHOUSE-2] 2018-03-14 16:29:03 +03:00
Alexey Zatelepin
6d5ce3fa9a revert BackgroundSchedulePool usage in Replicated tables 2017-12-22 21:25:54 +03:00
Alexey Zatelepin
2340e55d0e revert to old behaviour of getPartIfExists() [#CLICKHOUSE-3508] 2017-12-20 20:04:14 +03:00
Alexey Milovidov
6c8b5e29d8 Additions after merge [#CLICKHOUSE-2]. 2017-11-20 00:17:58 +03:00
alexey-milovidov
fc0a51f1b9
Merge branch 'master' into fix4 2017-11-19 23:57:44 +03:00
Alexey Milovidov
94d3b49e99 BackgroundSchedulePool: miscellaneous [#CLICKHOUSE-2]. 2017-11-19 03:27:43 +03:00
Alexey Milovidov
9b92c0fecd BackgroundSchedulePool: miscellaneous [#CLICKHOUSE-2]. 2017-11-19 03:22:13 +03:00
Silviu Caragea
dfecd0b2f2 Merge with master 2017-11-18 10:16:06 +02:00
Alexey Milovidov
729b114c15 Removed unsupported functionality [#CLICKHOUSE-2]. 2017-11-18 05:34:00 +03:00
Alexey Milovidov
076e0ae014 More clear [#CLICKHOUSE-2]. 2017-11-18 00:20:28 +03:00
Alexey Milovidov
7c9ba15ba1 Style [#CLICKHOUSE-2]. 2017-11-17 23:42:03 +03:00
Silviu Caragea
64619c614f Basic idea is : 
- We add BackgroundSchedulePool which can execute a function at a specific point in time. Basically all tasks are added in a queue and precessed by worker threads.
- The most important difference between this and BackgroundProcessingPool is that we have the guarantee that the same function is not executed from many workers in the same time
- Each of the following classes instead starting a thread will register a task in BackgroundSchedulePool and when they need to run will call schedule or scheduleAfter(duration) functions

This commit is moving all threads created by ReplicatedMergeTree to BackgroundSchedulePool tasks

NOTE: I did a minimum number of changes to be much simple to review the code
 2017-11-17 10:58:35 +02:00
proller
3b58e9cd67 Fix atomic usage 2017-11-01 20:15:05 +03:00
proller
c5dcde0c78 fix atomic usage 2017-11-01 16:58:02 +03:00
Vitaliy Lyudvichenko
b5c7eaa80e Force data parts cleanup before repeat fetch. [#CLICKHOUSE-3178] 2017-10-26 17:16:06 +03:00
Alexey Zatelepin
c61d4106e8 allow loading all MergeTree settings from a query [#CLICKHOUSE-3000] 2017-10-20 21:28:45 +03:00
Vitaliy Lyudvichenko
62ea1133dd Disable part sendings and fetches before ALTER. [#CLICKHOUSE-3343] 2017-10-12 23:26:54 +03:00
Alexey Zatelepin
df58949a68 make exit out of the loop possible when ZooKeeper is unavailable [#CLICKHOUSE-3259] 2017-09-20 06:05:20 +03:00
Vitaliy Lyudvichenko
10c14cfccb Small code enhancements according to clang-tidy. [#CLICKHOUSE-2931] 2017-08-04 17:35:36 +03:00
Alexey Zatelepin
4a98d83a3b introduce replicated_can_become_leader merge_tree setting [#CLICKHOUSE-3109] 2017-07-11 20:44:35 +03:00
Alexey Milovidov
061c214c16 Fixed error [#CLICKHOUSE-3074]. 2017-06-23 09:57:55 +03:00
alexey-milovidov
72822714c3 Merge branch 'master' into CLICKHOUSE-2989 2017-05-25 21:04:37 +04:00