thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-10-06 08:30:54 +00:00
Code Issues Packages Projects Releases Wiki Activity
30,567 Commits 8,977 Branches 1,768 Tags 2 GiB
5c73a75843
Commit Graph

1019 Commits

Author SHA1 Message Date
Vadim
c1274f9776
Update StorageReplicatedMergeTree.cpp 2018-08-23 17:13:15 +03:00
VadimPE
9beb841406 CLICKHOUSE-3847 fix throw and del log 2018-08-23 16:55:59 +03:00
Vadim
22b7c436e5
Update StorageReplicatedMergeTree.cpp 2018-08-22 17:26:37 +03:00
Vadim
62b9b039cd
change exception 2018-08-22 17:25:30 +03:00
Vadim
623c6e0553
Update StorageReplicatedMergeTree.cpp 2018-08-22 17:19:48 +03:00
VadimPE
d81d4dfc70 CLICKHOUSE-3847 change createReplica and change check in cloneReolica() 2018-08-22 17:01:54 +03:00
Alexey Zatelepin
b6aa7542d0 add BackgroundSchedulePool::TaskInfo::activateAndSchedule() method 
Almost always it (and not activate()) is precisely what is needed.
 2018-08-22 16:43:27 +03:00
Alexey Zatelepin
55d82dfc84 fix race in initialization of ReplicatedMergeTreeRestartingThread [#CLICKHOUSE-3899] 2018-08-21 18:22:55 +03:00
Vadim
854192e666
Update StorageReplicatedMergeTree.cpp 2018-08-21 17:26:20 +03:00
Vadim
3598f39f82
Merge branch 'master' into ISSUES-2581 2018-08-21 17:24:03 +03:00
Vadim
8f3350216f
Merge branch 'master' into CLICKHOUSE-3847 2018-08-21 13:27:18 +03:00
VadimPE
99a26f17f6 CLICKHOUSE-3847 fix settings 2018-08-21 13:23:48 +03:00
Alexey Milovidov
3b39f6a4c3 Split ZooKeeperExceptions to three different metrics #2316 2018-08-21 04:27:22 +03:00
VadimPE
a49cc41279 CLICKHOUSE-3847 add support old version of CH server 2018-08-21 02:08:45 +03:00
VadimPE
4064ec0bc2 CLICKHOUSE-3847 add value '1' and '2' in is_lost 2018-08-20 20:15:04 +03:00
VadimPE
0c119d7667 CLICKHOUSE-3847 fix bugs 2018-08-20 16:31:24 +03:00
Nikolai Kochetov
391b4b5927 Merged with master. 2018-08-20 15:25:06 +03:00
proller
62b6b6d73e Rare segfault fix 2018-08-17 21:23:10 +03:00
VadimPE
c8e2506b5f ISSUES-2581 add check when replica do not have part added with quorum 2018-08-14 19:40:51 +03:00
VadimPE
a37ad50329 ISSUES-2581 fix build 2018-08-14 13:24:39 +03:00
VadimPE
3c0f6ce87b ISSUES-2581 fix SELECT with select_sequential_consistency 2018-08-14 12:57:18 +03:00
Alexey Milovidov
36db216abf Preparation for extra warnings [#CLICKHOUSE-2] 2018-08-10 07:02:56 +03:00
Alexey Milovidov
d8ca4345b6 Fixed style violations [#CLICKHOUSE-2] 2018-08-10 04:41:54 +03:00
Vadim
44cb046eb9
Update StorageReplicatedMergeTree.cpp 2018-08-09 19:25:19 +03:00
Vadim
434017d193
Update StorageReplicatedMergeTree.cpp 2018-08-09 18:09:05 +03:00
VadimPE
75900f494f CLICKHOUSE-3847 add "is_lost". load() in Queue can check duplicate records 2018-08-09 18:06:39 +03:00
VadimPE
d5a6967fc4 CLICKHOUSE-3847 fix 2018-08-09 11:44:49 +03:00
alesapin
402815977f Add new error in fetchPartition op 2018-08-08 21:01:25 +03:00
VadimPE
c16d876867 CLICKHOUSE-3847 fix race in set log_pointer in cloneReplica() 2018-08-08 18:25:07 +03:00
VadimPE
4c0bb93345 CLICKHOUSE-3847 fix UInt - UInt. del clearQueue() 2018-08-08 17:07:39 +03:00
VadimPE
8d28b9f6b4 CLICKHOUSE-3847 we can do pullToLog() after we mark replica is_active and before we calc delay. 2018-08-08 16:53:06 +03:00
VadimPE
32b6965cf2 CLICKHOUSE-3847 Update. Add new settings. Del is_lost 2018-08-07 18:21:42 +03:00
alexey-milovidov
0ef2a94d46
Merge pull request #2782 from VadimPE/CLICKHOUSE-3714 
CLICKHOUSE-3714 add max_partition_size_to_drop
 2018-08-06 20:20:59 +03:00
alesapin
425918afe1 CLICKHOUSE-3880: Fix replace partition logic inside replicated merge tree 2018-08-06 15:59:51 +03:00
VadimPE
490efebe62 CLICKHOUSE-3714 add size pf all part in partition 2018-08-06 11:27:59 +03:00
Alexey Milovidov
2016374d20 Merging #2226 2018-08-05 10:05:43 +03:00
Alexey Zatelepin
b6b54eea52 old mutations cleaner for ReplicatedMergeTree engines [#CLICKHOUSE-3828] 2018-08-03 22:21:44 +03:00
VadimPE
b97fa2eb4d CLICKHOUSE-3714 bool ckeck_CanBeDropped -> void ... 2018-08-03 12:39:01 +03:00
VadimPE
6142d9ae6a CLICKHOUSE-3714 fix function names and get partition size 2018-08-03 11:33:57 +03:00
VadimPE
98c33de4e3 CLICKHOUSE-3714 2018-08-02 12:03:38 +03:00
VadimPE
08fe108ba8 CLICKHOUSE-3714 fix methods name 2018-08-02 11:48:57 +03:00
Vadim
034a88a504
Merge branch 'master' into CLICKHOUSE-3714 2018-08-01 20:42:56 +03:00
VadimPE
66d9ba3eb7 CLICKHOUSE-3714 add max_partition_size_to_drop 2018-08-01 20:41:18 +03:00
alesapin
e81108b954 CLICKHOUSE-3863: Add more convinient exception message 2018-08-01 00:29:47 +03:00
alesapin
2db2d208e1 CLICKHOUSE-3863: Add HTTPS support for interserver replication 2018-08-01 00:29:47 +03:00
Alexey Zatelepin
4863c5c178 fix potential segfault in MergeTreeData::delayInsertOrThrowIfNeeded [#CLICKHOUSE-3862] 2018-07-30 21:47:24 +03:00
Alexey Zatelepin
cc7950a733 don't destroy Cleanup- and AlterThread objects, just deactivate the tasks [#CLICKHOUSE-3862] 
Otherwise segfaults are possible when the objects are recreated in the middle of the queue task execution.
 2018-07-30 21:47:24 +03:00
alesapin
6abbfbce87 CLICKHOUSE-3832: Add HTTP Basic authentification in replication protocol 2018-07-26 18:10:57 +03:00
alexey-milovidov
68b57e605a
Merge branch 'master' into background-processing-pool-backoff 2018-07-16 06:27:43 +03:00
Alexey Zatelepin
78b83d78f0 execute part mutations in a background pool [#CLICKHOUSE-3748] 2018-07-16 06:22:22 +03:00
Alexey Zatelepin
ce17868a5c execute mutations synchronously for StorageMergeTree [#CLICKHOUSE-3748] 2018-07-16 06:22:22 +03:00
Amos Bird
ea01a3b22e add OPTIMIZE FINAL support for ReplicatedMergeTree. 2018-07-09 18:03:23 +03:00
Alexey Milovidov
7dc1276afe Little better #2281 2018-07-08 08:26:51 +03:00
Alexey Milovidov
0a188389c0 Fixed error #2599 2018-07-05 21:56:33 +03:00
Alexey Zatelepin
a41ee1f0a7 switch to ephemeral nodes for block number locks [#CLICKHOUSE-3802] 2018-07-05 20:06:33 +03:00
Alexey Zatelepin
0a46d231ed speedup merge selection: async exists check for selected parts [#CLICKHOUSE-3802] 2018-07-05 20:06:33 +03:00
Alexey Zatelepin
116ed0a59e add RestartingThread wakeup for faster session recovery 2018-07-05 19:30:52 +03:00
Nikolai Kochetov
ba4cf6faf0 Merged with master. 2018-06-25 15:08:21 +03:00
Alexey Zatelepin
a6c61848f1 mutations finalizing task [#CLICKHOUSE-3747] 2018-06-21 22:47:44 +03:00
alexey-milovidov
08ec751a8c
Merge pull request #2504 from yandex/system-mutations-table 
[WIP] system.mutations table skeleton
 2018-06-13 23:00:30 +03:00
Alexey Zatelepin
4ee581117b system.mutations table skeleton [#CLICKHOUSE-3747] 2018-06-13 17:15:36 +03:00
Alexey Zatelepin
c5c601f662 save serialized commands in ReplicatedMergeTreeMutationEntry; split InterpreterAlterQuery::parseAlter() [#CLICKHOUSE-3747] 2018-06-13 17:14:08 +03:00
Alexey Milovidov
d864f39ec7 Merge branch 'master' into zhang2014-feature/support_truncate 2018-06-09 19:10:06 +03:00
Alexey Milovidov
d56c199f07 Merging: Fixed half of problems #2260 2018-06-09 18:48:22 +03:00
Alexey Milovidov
0bc678474e Merge branch 'feature/support_truncate' of https://github.com/zhang2014/ClickHouse into zhang2014-feature/support_truncate 2018-06-09 17:24:50 +03:00
Alexey Zatelepin
3947f25365 add general comment about the mutation algorithm [#CLICKHOUSE-3747] 2018-06-05 17:55:35 +03:00
Alexey Milovidov
0f96a5c1c5 Suppressions for PVS-Studio [#CLICKHOUSE-3] 2018-06-04 22:46:47 +03:00
Alexey Zatelepin
e55097b866 introduce constants for max level and max block number 2018-06-04 14:23:47 +03:00
Alexey Zatelepin
f63d2dc3c8 cannot take a number of unlocked abandonable lock 2018-05-31 20:28:17 +03:00
Alexey Zatelepin
c9bb986330 Merge branch 'master' into replicated-mergetree-mutations 
Conflicts:
	dbms/src/Storages/MergeTree/AbandonableLockInZooKeeper.h
	dbms/src/Storages/MergeTree/ReplicatedMergeTreePartCheckThread.cpp
	dbms/src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp
	dbms/src/Storages/MergeTree/ReplicatedMergeTreeQueue.h
	dbms/src/Storages/StorageReplicatedMergeTree.cpp
 2018-05-31 18:52:42 +03:00
Vitaliy Lyudvichenko
dc5de3bbec More optimal conflicts check in future_parts. [#CLICKHOUSE-2] 2018-05-31 17:14:50 +03:00
Vitaliy Lyudvichenko
888f5f4a24 Allow merges after CLEAR COLUMN. [#CLICKHOUSE-3] 
Resolves #2315
 2018-05-31 17:14:50 +03:00
Alexey Zatelepin
d89221c458 easier init and deactivation of BackgroundSchedulePool tasks 2018-05-31 16:18:52 +03:00
Alexey Zatelepin
f6c72f8e56 Merge branch 'master' into replicated-mergetree-mutations 
Conflicts:
	dbms/src/Common/ActionBlocker.h
	dbms/src/Storages/MergeTree/ActiveDataPartSet.cpp
	dbms/src/Storages/MergeTree/ActiveDataPartSet.h
	dbms/src/Storages/MergeTree/MergeTreeData.h
	dbms/src/Storages/MergeTree/MergeTreeDataPart.cpp
	dbms/src/Storages/MergeTree/ReplicatedMergeTreeAlterThread.cpp
	dbms/src/Storages/MergeTree/ReplicatedMergeTreeLogEntry.cpp
	dbms/src/Storages/MergeTree/ReplicatedMergeTreeLogEntry.h
	dbms/src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp
	dbms/src/Storages/MergeTree/ReplicatedMergeTreeQueue.h
	dbms/src/Storages/MergeTree/ReplicatedMergeTreeRestartingThread.cpp
	dbms/src/Storages/StorageMergeTree.cpp
	dbms/src/Storages/StorageReplicatedMergeTree.cpp
	dbms/src/Storages/StorageReplicatedMergeTree.h
 2018-05-28 18:37:30 +03:00
Alexey Zatelepin
59fe12ed15 part log and metrics [#CLICKHOUSE-3747] 2018-05-25 23:13:00 +03:00
Alexey Zatelepin
aff6ad42ed support empty parts [#CLICKHOUSE-3747] 2018-05-25 23:13:00 +03:00
Alexey Zatelepin
2b80fbf972 DROP PARTITION now works with mutated parts [#CLICKHOUSE-3747] 2018-05-25 23:13:00 +03:00
Alexey Zatelepin
fdb33d8f3c execute mutation log entries [#CLICKHOUSE-3747] 2018-05-25 23:13:00 +03:00
Alexey Zatelepin
1147e052e8 select and write part mutations to queue [#CLICKHOUSE-3747] 2018-05-25 23:13:00 +03:00
Alexey Zatelepin
2721daf782 don't select merges of parts with different mutation versions [#CLICKHOUSE-3747] 2018-05-25 23:06:28 +03:00
Alexey Zatelepin
4c661df146 notify relevant events directly from queue [#CLICKHOUSE-3747] 2018-05-25 23:06:23 +03:00
Alexey Zatelepin
42850f09b7 load mutations from ZK [#CLICKHOUSE-3747] 2018-05-25 23:06:15 +03:00
Alexey Zatelepin
49439aa2d2 write mutations to ZK [#CLICKHOUSE-3747] 2018-05-25 23:06:08 +03:00
Alexey Zatelepin
4ca3bf65f3 test getting abandonable lock in all partitions [#CLICKHOUSE-3747] 2018-05-25 23:05:10 +03:00
Alexey Zatelepin
17437a95bf merge selection based on checking current inserts [#CLICKHOUSE-3732] 2018-05-25 23:03:56 +03:00
Alexey Milovidov
13762471da Changed DayNum_t to DayNum as Amos Bird suggested [#CLICKHOUSE-3] 2018-05-25 16:29:15 +03:00
Vitaliy Lyudvichenko
c888903488
Add query ALTER TABLE t1 [REPLACE|ATTACH] PARTITION FROM t2 
* Add ATTACH PARTITION FROM table for MergeTree. [#CLICKHOUSE-3546]

* Implemented replicated case on non-leader replica. [#CLICKHOUSE-3546]

* Disable merges in the dropping range. [#CLICKHOUSE-3546]

* DROP PARTITION is atomic and simpler now. [#CLICKHOUSE-3546]

* Implemented more SYSTEM queries. [#CLICKHOUSE-2931] [#CLICKHOUSE-3546]

SYSTEM queries:
RESTART REPLICAS
SYNC REPLICA db.name
STOP MERGES [db.name]
START MERGES [db.name]
STOP FETCHES [db.name]
START FETCHES [db.name]
STOP REPLICATED SENDS [db.name]
START REPLICATED SENDS [db.name]
STOP REPLICATION QUEUES [db.name]
START REPLICATION QUEUES [db.name]

* Fixed a bunch of bugs in REPLACE PARTITION. [#CLICKHOUSE-3546]

* Add tests for REPLACE PARTITION and SYSTEM. [#CLICKHOUSE-3546]

* Add system.part_log logging. [#CLICKHOUSE-3546]

* Fixed long wait in SYNC REPLICA. [#CLICKHOUSE-3546]

* Add requested changes. [#CLICKHOUSE-3546]

Fixed clickhouse-client bad return code.

* Add requested chenges. [#CLICKHOUSE-3546]

* Add requested chenges. [#CLICKHOUSE-3546]
 2018-05-21 16:49:54 +03:00
zhang2014
fa865636ce ISSUES-2259 add comment & move delete database metadata implementation 2018-05-21 11:38:46 +08:00
zhang2014
6613e56784 ISSUES-2259 support truncate syntax 2018-05-21 11:38:46 +08:00
alexey-milovidov
517318334a
Merge pull request #1722 from silviucpp/background-schedule-pool-fix 
Background schedule pool fixed
 2018-05-19 19:00:43 +03:00
Vitaliy Lyudvichenko
4b093e3e0a Don't make extra checksums checks when commit a part. [#CLICKHOUSE-2] 
This patch is very crucial in case of many replicas.
 2018-05-18 02:01:36 +03:00
Vitaliy Lyudvichenko
e152f223c9 Fix nodes leak in case of session expiration. [#CLICKHOUSE-2] 2018-04-26 22:46:45 +03:00
Silviu Caragea
1dd5a704fc Merge remote-tracking branch 'origin/master' into background-schedule-pool-fix 
# Conflicts:
#	dbms/src/Common/ZooKeeper/LeaderElection.h
#	dbms/src/Storages/MergeTree/ReplicatedMergeTreeAlterThread.cpp
#	dbms/src/Storages/MergeTree/ReplicatedMergeTreeCleanupThread.cpp
#	dbms/src/Storages/MergeTree/ReplicatedMergeTreePartCheckThread.cpp
#	dbms/src/Storages/StorageReplicatedMergeTree.cpp
 2018-04-24 20:11:59 +03:00
Alexey Milovidov
499b67642f Allow to startup with replicated tables in readonly mode when there is no ZooKeeper configured [#CLICKHOUSE-2] 2018-04-21 21:41:06 +03:00
Nikolai Kochetov
a6375d6155 merged with master 2018-04-20 17:15:03 +03:00
Alexey Milovidov
da94c7dd91 Exit from queueUpdatingThread when ZooKeeper session is expired (non-significant change) [#CLICKHOUSE-2] 2018-04-19 21:01:50 +03:00
Nikolai Kochetov
dd10c2073c merged with master 2018-04-19 18:10:44 +03:00
Nikolai Kochetov
a3308bf675 added IStorage::getQueryProcessingStage 2018-04-19 17:47:09 +03:00
Silviu Caragea
32dd4554da Merge remote-tracking branch 'github/master' into background-schedule-pool-fix 
# Conflicts:
#	dbms/src/Storages/StorageReplicatedMergeTree.cpp
 2018-04-19 09:29:49 +03:00
alexey-milovidov
089e71d9a5
Merge pull request #2235 from yandex/small-enhancements 
Small enhancements
 2018-04-17 22:45:01 +03:00
Vitaliy Lyudvichenko
bf832b3ea6 Fixed OPTIMIZE after RENAME of replicated table. [#CLICKHOUSE-2] 2018-04-17 20:59:42 +03:00
proller
27b4af9dd7 Change obsolete comment 2018-04-16 23:25:07 +03:00
proller
157067b127 CLICKHOUSE-3700: Fix empty password on replica query 2018-04-10 17:49:52 +03:00
Silviu Caragea
f1b8d4bc39 Merge remote-tracking branch 'github/master' into background-schedule-pool-fix 2018-04-10 16:27:17 +03:00
Silviu Caragea
c547c5a7b0 Merge remote-tracking branch 'origin/background-schedule-pool-fix' 
# Conflicts:
#	dbms/src/Common/ZooKeeper/LeaderElection.h
#	dbms/src/Common/ZooKeeper/ZooKeeper.cpp
#	dbms/src/Storages/MergeTree/ReplicatedMergeTreeAlterThread.cpp
#	dbms/src/Storages/MergeTree/ReplicatedMergeTreeCleanupThread.cpp
#	dbms/src/Storages/MergeTree/ReplicatedMergeTreeCleanupThread.h
#	dbms/src/Storages/MergeTree/ReplicatedMergeTreePartCheckThread.cpp
#	dbms/src/Storages/MergeTree/ReplicatedMergeTreeRestartingThread.cpp
#	dbms/src/Storages/StorageReplicatedMergeTree.cpp
 2018-04-10 16:20:14 +03:00
Alexey Milovidov
d3cf85057b Removed useless code [#CLICKHOUSE-2] 2018-04-09 18:49:12 +03:00
Alexey Milovidov
0a4b349f73 Merge branch 'master' of github.com:yandex/ClickHouse 2018-04-07 00:47:18 +03:00
Alexey Milovidov
4e5e0fa664 ReplicatedMergeTree: Better diagnostics [#CLICKHOUSE-2] 2018-04-07 00:46:57 +03:00
alexey-milovidov
84726e17e8
Merge pull request #2185 from yandex/fix-leader-election-race 
Fix races in leader election.
 2018-04-07 00:18:26 +03:00
Alexey Milovidov
0d8d99e7a4 Fixed typo [#CLICKHOUSE-2] 2018-04-06 22:48:54 +03:00
Alexey Zatelepin
3a8ca70d1f fix races in leader election [#CLICKHOUSE-3533] 2018-04-06 19:10:23 +03:00
Silviu Caragea
f811da7ed4 Move merge selection state into ReplicatedMergeTreeMergeSelectingThread 2018-04-02 15:45:55 +03:00
alexey-milovidov
428ee63a04
Merge branch 'master' into libzookeeper-rewrite 2018-04-01 17:31:20 +03:00
Silviu Caragea
1418e339e4 Rename merge_selecting_handle with merge_selecting_task_handle for consistency 2018-03-26 23:18:19 +03:00
Silviu Caragea
0aa9b9efbe Use consistent names for the task variables (change from next_update_task_handle to merge_selecting_handle) (reverted from commit 50992843c6) 2018-03-26 23:12:17 +03:00
Silviu Caragea
50992843c6 Use consistent names for the task variables (change from next_update_task_handle to merge_selecting_handle) 2018-03-26 22:59:58 +03:00
Silviu Caragea
31874ed172 Use consistent names for the task variables 2018-03-26 22:37:47 +03:00
Alex Zatelepin
c5b04f7f17 Add info about total uncompressed part size to system tables (#2052) 
* column size calculating routines now take into account multiple streams [#CLICKHOUSE-2831]

* rename for clarity

* use more appropriate method for calculating table size

* add info about total uncompressed part size to system.parts and system.parts_columns [#CLICKHOUSE-2831]

* rename columns for clarity [#CLICKHOUSE-2831]

* count shared substreams only once [#CLICKHOUSE-2831]
 2018-03-26 17:18:04 +03:00
Alexey Milovidov
522bf01db9 Using different ZooKeeper library (development) [#CLICKHOUSE-2] 2018-03-25 03:15:52 +03:00
Alexey Milovidov
ae9633c1e5 Using different ZooKeeper library (development) [#CLICKHOUSE-2] 2018-03-24 23:00:16 +03:00
Alexey Milovidov
26584844b6 Rewriting ZooKeeper library [#CLICKHOUSE-2] 2018-03-24 04:00:12 +03:00
Alexey Milovidov
10be6ca8f8 Rewriting ZooKeeper library [#CLICKHOUSE-2] 2018-03-24 03:45:04 +03:00
Vitaliy Lyudvichenko
883de7a2a6 startup() waits for initialization of ReplicatedMergeTreeRestartingThread. [#CLICKHOUSE-2] 2018-03-23 00:26:46 +03:00
Vitaliy Lyudvichenko
fe8dfd3038 Add minimalistic checksums. [#CLICKHOUSE-3660] 2018-03-23 00:26:46 +03:00
Silviu Caragea
6629b03af9 Fixed few of the observation of the code review 2018-03-22 17:09:30 +02:00
Silviu Caragea
b24a4b2111 Add back the buggy BackgroundSchedulePool 2018-03-22 17:07:12 +02:00
Alexey Milovidov
3c16faaff0 Merge branch 'master' of github.com:yandex/ClickHouse 2018-03-14 21:57:15 +03:00
Alexey Milovidov
57279a87c7 Whitespaces [#CLICKHOUSE-2]. 2018-03-14 19:40:08 +03:00
Vitaliy Lyudvichenko
8028ba0de9 Refactoring of zkutil::ZooKeeper::multi* functions. [#CLICKHOUSE-2] 2018-03-14 16:29:03 +03:00
alexey-milovidov
9ac2569c65
Merge branch 'master' into refactor-table-declaration 2018-03-14 05:35:50 +03:00
alexey-milovidov
04cc8a52f7
Merge pull request #2032 from yandex/system-table-macro 
System table macro
 2018-03-14 02:10:30 +03:00
Alexey Zatelepin
5906c9a988 rename for clarity 2018-03-13 18:00:28 +03:00
Alexey Zatelepin
fdffbe53ac encapsulate ColumnsDeclaration [#CLICKHOUSE-3241] 2018-03-13 17:41:25 +03:00
Alexey Milovidov
b7475b1d2c Fixed error: wrong timeouts passed to ConnectionPoolWithFailover [#CLICKHOUSE-3641] 2018-03-12 23:25:18 +03:00
Alexey Zatelepin
78d92f6794 add some useful methods, rename for clarity [#CLICKHOUSE-3241] 2018-03-12 22:10:13 +03:00
Alexey Zatelepin
71642b0880 use ColumnsDescription everywhere [#CLICKHOUSE-3241] 2018-03-12 22:07:48 +03:00
Nikolai Kochetov
4dd7f47ad5 reload macros with config [#CLICKHOUSE-3591] 2018-03-12 21:16:51 +03:00
Alexey Milovidov
afbbd780e3 Fixed error with initialization of PartLog [#CLICKHOUSE-3354] 2018-03-10 22:57:13 +03:00
Alexey Milovidov
3f8ef38b41 Fixed wording #245 2018-03-10 02:23:15 +03:00
proller
d73fea5ab0
Fix macos compile (#2006) 
* Fix macos compile

* rename cluster-copier -> copier
 2018-03-07 21:17:17 +03:00
Alexey Milovidov
8c37547bfe Fixed harmless data race [#CLICKHOUSE-2] 2018-03-03 18:35:24 +03:00
Alexey Milovidov
6e93635275 Better UNION ALL: development #1947 2018-02-25 09:35:24 +03:00
alexey-milovidov
82fa3883f8
Merge pull request #1930 from yandex/nonblocking-precommitted 
Allow PreCommitted parts to contain each other
 2018-02-22 23:35:16 +03:00
alexey-milovidov
ee41f7000f
Merge pull request #1940 from yandex/CLICKHOUSE-3571 
Try to reduce false positive of sanity check
 2018-02-22 01:59:33 +03:00
Vitaliy Lyudvichenko
47becc0b92 Try to reduce false positive of sanity check. More detail dignostics. [#CLICKHOUSE-3571] 2018-02-21 20:06:29 +03:00
Alexey Zatelepin
5bc6bd55b1 allow PreCommitted parts to contain each other, clean up code 2018-02-20 16:33:28 +03:00
Alexey Milovidov
fa50fe80a0 Added method "getHeader" in IBlockOutputStream: development [#CLICKHOUSE-2] 2018-02-19 03:45:32 +03:00
Alexey Milovidov
30b657a0da Header in every stream: development [#CLICKHOUSE-2] 2018-02-15 21:54:12 +03:00
Nikolai Kochetov
64b3a8baa7 added secondary sort expression [#CLICKHOUSE-3479] 2018-02-13 00:13:50 +03:00
Vitaliy Lyudvichenko
94f38d744e
Merge pull request #1750 from yandex/CLICKHOUSE-3346 
Fault-tolerant cluster copier util
 2018-01-31 13:21:51 +03:00
Vitaliy Lyudvichenko
e2fb6d8c80 Merge remote-tracking branch 'origin/master' into CLICKHOUSE-3346 2018-01-25 22:00:08 +03:00
Vitaliy Lyudvichenko
61233b886a Better system.part_log. [#CLICKHOUSE-3342] 2018-01-25 21:40:02 +03:00
Vitaliy Lyudvichenko
26e58271e9 Removed repeatable columns field from Storages declarations. [#CLICKHOUSE-2] 2018-01-25 17:42:39 +03:00
Vitaliy Lyudvichenko
b4ab7d4b8e Make early deduplication with block number creation. [#CLICKHOUSE-3545] 2018-01-23 18:50:29 +03:00
Vitaliy Lyudvichenko
1000328571 Merge remote-tracking branch 'origin/master' into CLICKHOUSE-3346 2018-01-22 20:31:08 +03:00
alexey-milovidov
1b462b0b94
Merge pull request #1759 from yandex/CLICKHOUSE-3432 
Fixed 'intersects previous part' due to blind OPTIMIZE
 2018-01-19 21:48:41 +03:00
alexey-milovidov
6198fe4a1a
Update StorageReplicatedMergeTree.cpp 2018-01-19 21:46:22 +03:00
alexey-milovidov
e8ebf3b56a
Update StorageReplicatedMergeTree.cpp 2018-01-19 21:45:28 +03:00
alexey-milovidov
7a25b441f4
Merge pull request #1773 from yandex/small-enhancements 
Small enhancements
 2018-01-16 22:07:43 +03:00
Vitaliy Lyudvichenko
b68b60fb38 More detailed test. [#CLICKHOUSE-3] 
Resolves #1553.
 2018-01-16 15:05:01 +03:00
proller
d97b2edc7c Make separate libs Common/ZooKeeper and Common/ConfigProcessor (#1749) 
* Make separate libs Common/ZooKeeper and Common/ConfigProcessor
renamed:    dbms/src/Common/StringUtils.h -> dbms/src/Common/StringUtils/StringUtils.h

* Fix space

* Fix data in system.build_options

* merge

* merge
 2018-01-15 22:07:47 +03:00
Vitaliy Lyudvichenko
554ed52920 Avoid superflous pullLogsToQueue() calls. [#CLICKHOUSE-3432] 2018-01-15 13:30:31 +03:00
Vitaliy Lyudvichenko
03bca2144d OPTIMIZE doesn't produce intersecting merges now, add optimize_throw_if_noop setting. [#CLICKHOUSE-3432] [#CLICKHOUSE-3420] 
Resolves #1753
 2018-01-15 13:13:18 +03:00
Vitaliy Lyudvichenko
388d47bbbc Merge branch 'CLICKHOUSE-3346-v310' into CLICKHOUSE-3346 2018-01-11 23:51:30 +03:00
Vitaliy Lyudvichenko
c818beedcb Make data_path creation for Databases optional. [#CLICKHOUSE-2] 2018-01-11 21:55:43 +03:00
Nikolai Kochetov
cbeeb84999 added http_*_timeout settings [#CLICKHOUSE-3440] 2018-01-09 21:41:01 +03:00
Chen Yufei
88ef42c222 Aggressive merge for ReplicatedMergeTree optimize. 2018-01-09 20:44:07 +03:00
Alexey Milovidov
fcdd5de47d Revert "Removed questionable code (2) [#CLICKHOUSE-2]." 
This reverts commit fcabe8b888.
 2017-12-28 07:39:19 +03:00
Alexey Milovidov
798299ba89 Merge branch 'aavdonkin-master' 2017-12-28 07:33:55 +03:00
Alexey Milovidov
b2fcf06353 Merging [#CLICKHOUSE-3114] 2017-12-27 00:34:06 +03:00
Alexey Milovidov
c89041ce63 Removed questionable code (2) [#CLICKHOUSE-2]. 2017-12-26 01:51:04 +03:00
Alexey Milovidov
ec71908b00 Revert "Removed questionable code (2) [#CLICKHOUSE-2]." 
This reverts commit fcabe8b888.
 2017-12-26 00:57:29 +03:00
Alexey Milovidov
fcabe8b888 Removed questionable code (2) [#CLICKHOUSE-2]. 2017-12-26 00:32:33 +03:00
Alexey Milovidov
a85b2e5a95 Removed questionable code [#CLICKHOUSE-2]. 2017-12-26 00:10:46 +03:00
Alexey Zatelepin
c3348e27a4 shorter [#CLICKHOUSE-2] 2017-12-25 20:31:14 +03:00
Alexey Zatelepin
6d5ce3fa9a revert BackgroundSchedulePool usage in Replicated tables 2017-12-22 21:25:54 +03:00
Alexey Zatelepin
835b957e95 add comments [#CLICKHOUSE-3508] 2017-12-20 20:40:36 +03:00
Alexey Zatelepin
d35baafde8 check PreCommitted parts when executing log entries and in PartChecker [#CLICKHOUSE-3508] 2017-12-20 20:04:14 +03:00
Alexey Zatelepin
2340e55d0e revert to old behaviour of getPartIfExists() [#CLICKHOUSE-3508] 2017-12-20 20:04:14 +03:00
Alexey Milovidov
1aa99092c3 Unification of serde of data types: development [#CLICKHOUSE-2838]. 2017-12-03 05:15:35 +03:00
Alexey Milovidov
54e601019c Better [#CLICKHOUSE-2]. 2017-12-02 00:40:58 +03:00
Alexey Milovidov
acd78e4891 Better [#CLICKHOUSE-2]. 2017-12-02 00:13:25 +03:00
Alexey Milovidov
1a6cb2989b Better [#CLICKHOUSE-2]. 2017-12-01 21:36:55 +03:00
Vitaliy Lyudvichenko
b738d1baed Add multi index for data_parts storage. And fixed bugs. [#CLICKHOUSE-3452] 
Fixed handling of obsolete parts.
Fixed conflict resolution between simultaneous PreCommitted covering parts.
Fixed memory leak caused by ordinary MergeTree parts stucked in Deleting state.
Added hidden _state column into system.parts.
 2017-11-29 14:25:11 +03:00
proller
0d3c4f4d67 Core/FieldVisitors.h -> Common/FieldVisitors.h 
Core/Progress.h -> IO/Progress.h
    tests: sudo --non-interactive
    Fix freebsd link
 2017-11-24 16:55:31 +03:00
Alexey Milovidov
f6f77f5f0b Fixed linking [#CLICKHOUSE-2]. 2017-11-20 08:22:07 +03:00
Alexey Milovidov
6c8b5e29d8 Additions after merge [#CLICKHOUSE-2]. 2017-11-20 00:17:58 +03:00
Alexey Milovidov
9b92c0fecd BackgroundSchedulePool: miscellaneous [#CLICKHOUSE-2]. 2017-11-19 03:22:13 +03:00
Silviu Caragea
dfecd0b2f2 Merge with master 2017-11-18 10:16:06 +02:00
Alexey Milovidov
729b114c15 Removed unsupported functionality [#CLICKHOUSE-2]. 2017-11-18 05:34:00 +03:00
Alexey Milovidov
076e0ae014 More clear [#CLICKHOUSE-2]. 2017-11-18 00:20:28 +03:00
Alexey Milovidov
7c9ba15ba1 Style [#CLICKHOUSE-2]. 2017-11-17 23:42:03 +03:00
Silviu Caragea
64619c614f Basic idea is : 
- We add BackgroundSchedulePool which can execute a function at a specific point in time. Basically all tasks are added in a queue and precessed by worker threads.
- The most important difference between this and BackgroundProcessingPool is that we have the guarantee that the same function is not executed from many workers in the same time
- Each of the following classes instead starting a thread will register a task in BackgroundSchedulePool and when they need to run will call schedule or scheduleAfter(duration) functions

This commit is moving all threads created by ReplicatedMergeTree to BackgroundSchedulePool tasks

NOTE: I did a minimum number of changes to be much simple to review the code
 2017-11-17 10:58:35 +02:00
Alexey Zatelepin
99b2d135df clear deduplication blocks when doing DROP PARTITION [#CLICKHOUSE-3208] 2017-11-15 23:13:10 +03:00
Alexey Milovidov
4e04c4da5f Attempt to change ext::shared_ptr_helper (incomplete) [#CLICKHOUSE-2]. 2017-11-04 06:56:31 +03:00
KochetovNicolai
bfb4e8f3fe zookeeper chroot fix (#1430) 
* added root and identity section to zookeeper configuration [#CLICKHOUSE-3031]

* added root and identity section to zookeeper configuration [#CLICKHOUSE-3031]

* moved root section in zookeeper configuration [#CLICKHOUSE-3031]

* fixed root section in zookeeper configuration [#CLICKHOUSE-3031]

* added tests [#CLICKHOUSE-3031]

* replaced tabs to spaces in config files [#CLICKHOUSE-3031]

* fix build [#CLICKHOUSE-3031]

* added chech that root starts with '/' in zookeeper config file [#CLICKHOUSE-3031]

* added comments [#CLICKHOUSE-3031]

* Check if zookeeper chroot exists before start. Remove tailing slash form chroot. [#CLICKHOUSE-3400]

* added starting '/' to zookeeper path in ReplicatedMergeTree
 2017-11-02 01:59:27 +03:00
alexey-milovidov
3b99b723d7
Merge pull request #1245 from yandex/CLICKHOUSE-3178 
Atomic parts addition and removing
 2017-10-27 23:03:33 +03:00
Vitaliy Lyudvichenko
a0d06c5ee0 Don't deduplicate a block inserted from MV if the source block is Ok. [#CLICKHOUSE-3381] 2017-10-26 17:58:00 +03:00
Vitaliy Lyudvichenko
397b8bcf92 Smarter parts deletion. [#CLICKHOUSE-3178] 2017-10-26 17:16:06 +03:00
Vitaliy Lyudvichenko
b5c7eaa80e Force data parts cleanup before repeat fetch. [#CLICKHOUSE-3178] 2017-10-26 17:16:06 +03:00
Vitaliy Lyudvichenko
71833a7f0f More correct procedure of parts deleting. [#CLICKHOUSE-3178] 2017-10-26 17:16:06 +03:00
Vitaliy Lyudvichenko
db3a67a421 Add clearer RangeFiltered implementation. [#CLICKHOUSE-3178] 2017-10-26 17:16:06 +03:00
Vitaliy Lyudvichenko
e2a12d1088 Introduced logic for MergeTree parts states. [#CLICKHOUSE-3178] 2017-10-26 17:16:06 +03:00
Alexey Milovidov
65982a1642 Fixed exception messages [#CLICKHOUSE-2]. 2017-10-25 22:17:37 +03:00
Alexey Zatelepin
c61d4106e8 allow loading all MergeTree settings from a query [#CLICKHOUSE-3000] 2017-10-20 21:28:45 +03:00
Alexey Zatelepin
a7c85e0c89 save index_granularity to ATTACH query if it wasn't set explicitly [#CLICKHOUSE-3000] 2017-10-20 21:28:44 +03:00
Alexey Zatelepin
e51f95872f fix backward compatibility for new part format: [#CLICKHOUSE-3000] 
* do not write out partition and minmax index files for old MergeTreeData format
* download the part from another replica if there are missing or extra files
 2017-10-19 21:24:30 +03:00
Alexey Milovidov
d3d976a6fb Misc [#CLICKHOUSE-2]. 2017-10-12 23:45:55 +03:00
proller
b993667156 Try fix strange terminate (#1329) 
* Tests: External: rename --use_http  => --no_http and fix

* Try fix strange terminate
 2017-10-12 23:42:58 +03:00
Vitaliy Lyudvichenko
62ea1133dd Disable part sendings and fetches before ALTER. [#CLICKHOUSE-3343] 2017-10-12 23:26:54 +03:00
Vitaliy Lyudvichenko
cabf6d7799 Add better logging if OPTIMIZE cannot be executed. [#CLICKHOUSE-2] 2017-10-12 22:28:24 +03:00
Alexey Zatelepin
728a23f60e allow creation of custom-partitioned tables with experimental syntax [#CLICKHOUSE-3000] 2017-09-26 20:40:02 +03:00
Alexey Zatelepin
a10ba0cd91 serialize partition key in Replicated tables metadata [#CLICKHOUSE-3000] 2017-09-26 20:40:02 +03:00
Alexey Zatelepin
909b46de0f add MergeTreeDataFormatVersion constant [#CLICKHOUSE-3000] 2017-09-26 20:40:02 +03:00
Alexey Zatelepin
68cb592277 allow arbitrary partition literals in ALTER PARTITION queries [#CLICKHOUSE-3000] 2017-09-26 20:40:02 +03:00
Alexey Zatelepin
2f80c150db save MergeTree data format version and parse part names depending on it [#CLICKHOUSE-3000] 2017-09-26 20:40:02 +03:00
Alexey Milovidov
94b975e7f9 Removed function deprecated in C++ [#CLICKHOUSE-3174]. 2017-09-09 02:31:18 +03:00
Alexey Milovidov
8b313ab99e Fixed warnings found by cppcheck [#CLICKHOUSE-3300]. 2017-09-08 00:04:48 +03:00
Alexey Milovidov
0dcee75180 Supporessed useless false positive errors found by cppcheck [#CLICKHOUSE-3300]. 2017-09-07 22:13:37 +03:00
Vitaliy Lyudvichenko
d55d5eca46 Remove extra Context reference and add required ones. Part 2. [#CLICKHOUSE-3117] 2017-09-06 04:18:07 +03:00
Vitaliy Lyudvichenko
566160c048 Use FIFO lock in IStorage. [#CLICKHOUSE-3246] 2017-09-01 20:45:05 +03:00
Alexey Zatelepin
d2847f22af PR fixes [#CLICKHOUSE-3000] 2017-08-31 23:32:03 +03:00
Alexey Zatelepin
14ba0a9acf rename partition -> partition_id for clarity [#CLICKHOUSE-3000] 2017-08-31 23:32:03 +03:00
Alexey Zatelepin
645e4b7a53 generalize MinMax index [#CLICKHOUSE-3000] 2017-08-31 23:32:03 +03:00
Alexey Zatelepin
5471979f46 generic partitioning of inserted blocks [#CLICKHOUSE-3000] 2017-08-31 23:32:03 +03:00
Alexey Zatelepin
99969ab77a Introduce MergeTreePartitionIndex. Isolate its loading code. [#CLICKHOUSE-3000] 2017-08-31 23:32:03 +03:00
Alexey Milovidov
495daffa16 Miscellaneous [#CLICKHOUSE-2]. 2017-08-30 21:13:32 +03:00
Vitaliy Lyudvichenko
5916d823ef Fix deadlock in case of using Buffer table with CLEAR COLUMN IN PARTITION. [#CLICKHOUSE-3238] 2017-08-30 17:34:40 +03:00
Vitaliy Lyudvichenko
84660029ec Don't leave trash part nodes in ZK in case of deleting non-existing duplicated parts. [#CLICKHOUSE-3252] 2017-08-29 20:49:37 +03:00
Vitaliy Lyudvichenko
fd63277b15 Unified spelling. [#CLICKHOUSE-2] 2017-08-28 18:27:25 +03:00
Vitaliy Lyudvichenko
37820f7f64 Don't leave trash parts in ZK in case of temporary errors. [#CLICKHOUSE-3252] 2017-08-28 18:27:25 +03:00
Alexey Zatelepin
f25f0cd759 reduce number of places where partitioning by month is assumed [#CLICKHOUSE-3000] 2017-08-15 22:30:52 +03:00
Vitaliy Lyudvichenko
8a14cc49ff asyncMulti() -> multi(). [#CLICKHOUSE-3221] 2017-08-15 19:18:02 +03:00
Alexey Zatelepin
3f3b7f61e2 always send changed settings for remote connections [#CLICKHOUSE-3151] 2017-08-13 12:17:09 +03:00
Vitaliy Lyudvichenko
351a0905d5 Implemented fast block and parts cleaning. [#CLICKHOUSE-3207] 2017-08-10 23:09:05 +03:00
Vitaliy Lyudvichenko
10c14cfccb Small code enhancements according to clang-tidy. [#CLICKHOUSE-2931] 2017-08-04 17:35:36 +03:00
Alexey Milovidov
6578fb2645 Fixed error [#CLICKHOUSE-3191]. 2017-08-01 00:39:24 +03:00
Alexey Milovidov
8fb452e99a Fixed wrong log message when removing temporary part after merge, that is not byte-identical than same part on replicas; added profile event; added fail-safe logic [#MTRSADMIN-3175]. 2017-07-29 00:25:24 +03:00
Vitaliy Lyudvichenko
bc9ee54985 Don't throw exception in case of clearing empty part. [#CLICKHOUSE-2896] 2017-07-24 18:04:17 +03:00
alexey-milovidov
d4d266fa18 ColumnConst unification (#1011) 
* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].

* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].

* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].

* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].

* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].

* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].

* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].

* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].

* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].

* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].

* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].

* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].

* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].

* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].

* Fixed error in ColumnArray::replicateGeneric [#CLICKHOUSE-3150].

* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].

* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].

* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].

* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].

* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].

* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].

* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].

* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].

* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].

* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].

* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].

* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].

* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].
 2017-07-21 09:35:58 +03:00
Alexey Milovidov
9406b6a5aa Get rid of ASTSet, because it is not really an AST and it lead to bugs [#CLICKHOUSE-3148]. 2017-07-15 06:48:36 +03:00
Nikolai Kochetov
283810e8ab merged with master 2017-07-13 23:59:34 +03:00
Nikolai Kochetov
75be0b81d8 removed typeid_cast from IColumn.h 2017-07-13 23:58:19 +03:00
Alexey Milovidov
5b5795633e Using lexer (incomplete) [#CLICKHOUSE-2]. 2017-07-13 08:59:01 +03:00
Alexey Zatelepin
bdc529645e fix cyclical yielding of leadership at startup [#CLICKHOUSE-3074] 2017-07-12 20:09:44 +03:00
Vitaliy Lyudvichenko
97f7b46ebd Fixed CLEAR COLUMN test in case of deduplication rollback. [#CLICKHOUSE-2] 2017-06-26 19:34:38 +03:00
Vitaliy Lyudvichenko
9c0da6e698 Addition to previous commit. Fix error code, rename test tables. [#CLICKHOUSE-2] 2017-06-26 17:35:26 +03:00
Vitaliy Lyudvichenko
49cbc33a6b Fix ZK exception in case of concurrent INSERTs in new partition. [#CLICKHOUSE-2] 2017-06-26 17:21:25 +03:00
Vitaliy Lyudvichenko
b53d6976b3 Fix build after merge. [#CLICKHOUSE-2] 2017-06-26 15:47:15 +03:00
Alexey Milovidov
a48a21f593 Merge branch 'master' into CLICKHOUSE-2896 2017-06-26 11:54:58 +03:00
Alexey Milovidov
275a1d4651 Improvement [#CLICKHOUSE-3091]. 2017-06-25 05:48:38 +03:00
Alexey Milovidov
516b095423 Fixed error in prev. revision [#CLICKHOUSE-3091]. 2017-06-25 04:54:25 +03:00
Alexey Milovidov
8228acd3ea Removed obsolete code that was relevant only in extremely rare case of transforming CollapsingMergeTree to ReplicatedCollapsingMergeTree through ATTACHing data parts and while some considerations about data order in table is important [#CLICKHOUSE-3091]. 2017-06-25 03:51:51 +03:00
Alexey Milovidov
3c660fd65b Fixed translation errors [#CLICKHOUSE-3091]. 2017-06-25 03:17:08 +03:00
Alexey Milovidov
28e9e06992 Preparation: locking block number only after write to filesystem; removed obsolete code [#CLICKHOUSE-3091]. 2017-06-25 03:01:10 +03:00
Alexey Milovidov
c7f35997f5 Removed unused variable [#CLICKHOUSE-2]. 2017-06-25 01:36:43 +03:00
proller
4db8d09de9 Reorganize includes. part 1 (#921) 
* Make libunwind optional. Allow use custom libcctz

* fix

* Fix

* fix

* Update BaseDaemon.cpp

* Update CMakeLists.txt

* Reorganize includes. part 1

* Update dbms_include.cmake

* Reorganize includes. part 2

* Reorganize includes. part 3

* dbms/src/Common/ThreadPool -> libs/libcommon

* Reorganize includes. part 4

* Fix print_include_directories

* Update thread_creation_latency.cpp

* Update StringRef.h
 2017-06-23 23:22:35 +03:00
Alexey Milovidov
f409cad0ac Miscellaneous [#CLICKHOUSE-3091]. 2017-06-23 09:18:00 +03:00
Alexey Milovidov
7d732343eb Fixed error in prev. revision [#CLICKHOUSE-3091]. 2017-06-22 22:52:06 +03:00
Alexey Milovidov
22e7070fb0 Merge branch 'master' of github.com:yandex/ClickHouse 2017-06-22 19:07:57 +03:00
Alexey Milovidov
bb7fc04e2b Whitespaces [#CLICKHOUSE-2]. 2017-06-22 19:07:52 +03:00
Vitaliy Lyudvichenko
d7db235269 Better names and log messages. [#CLICKHOUSE-2896] 2017-06-22 18:32:09 +03:00
Vitaliy Lyudvichenko
99f502d5e0 Support OPTIMIZE query on non-leader replica. [#CLICKHOUSE-2] 2017-06-22 18:01:08 +03:00
proller
51b41aa0e3 Fix build 2017-06-22 17:14:13 +03:00
Vitaliy Lyudvichenko
eff3588f28 Rename DROP COLUMN FROM PARTITION -> CLEAR COLUMN IN PARTITION. [#CLICKHOUSE-2896] 2017-06-22 16:18:19 +03:00
Alexey Milovidov
db546dc827 Caching parts that cannot be merged for some time [#CLICKHOUSE-3091]. 2017-06-22 03:28:44 +03:00
Vitaliy Lyudvichenko
53adb891b3 Add replicated DROP COLUMN FROM partition for optimistic case. [#CLICKHOUSE-2896] 2017-06-22 00:56:38 +03:00
Alexey Milovidov
5c322f5756 Loosened threshold on amount of replica inconsistency to refuse server startup [#CLICKHOUSE-3076]. 2017-06-21 22:07:08 +03:00
Vitaliy Lyudvichenko
a098fb170a Avoid creating multiple global Context. [#CLICKHOUSE-3083] 2017-06-20 10:49:18 +03:00
Alexey Milovidov
05accbdcbe Removed cyclic dependency [#DEVTOOLS-3381]. 2017-06-19 23:06:35 +03:00
Alexey Milovidov
43fedfecea Get rid of INSERT ID [#CLICKHOUSE-31]. 2017-06-13 07:13:20 +03:00
Alexey Milovidov
38885d00a3 Merge branch 'storage-startup-method' 2017-06-07 02:27:16 +03:00
Alexey Milovidov
9c5ddce06d Get rid of .hpp files [#CLICKHOUSE-2]. 2017-06-06 20:18:32 +03:00
Alexey Milovidov
9ebf0b633f Added IStorage::startup method to avoid starting replication and merging threads too early during server startup [#CLICKHOUSE-2]. 2017-06-06 20:06:14 +03:00
Vitaliy Lyudvichenko
1e4906816c Avoid SIGSEGV in SystemLog destructors. [#CLICKHOUSE-3023] 2017-06-05 17:25:10 +03:00
alexey-milovidov
147b2f62b2 Update StorageReplicatedMergeTree.cpp 2017-06-02 20:02:59 +04:00
Vitaliy Lyudvichenko
6bddd4c1c5 Clean incomplete part nodes in ZooKeeper. [#CLICKHOUSE-3040] 2017-06-02 20:02:59 +04:00
Alexey Milovidov
ba499585cd Merge branch 'master' of github.com:yandex/ClickHouse 2017-06-02 18:54:43 +03:00
Alexey Milovidov
80080f1ad6 Small refinements [#CLICKHOUSE-2]. 2017-06-02 18:54:39 +03:00
Vitaliy Lyudvichenko
1a4b0eb355 Remove tmp part directories on server start. [#CLICKHOUSE-3044] 2017-05-31 18:01:25 +03:00
alexey-milovidov
72822714c3 Merge branch 'master' into CLICKHOUSE-2989 2017-05-25 21:04:37 +04:00
Vitaliy Lyudvichenko
1e29795cf9 Fix race condition between cleanup and fetch thread. [#CLICKHOUSE-2989] 
It was happening during ALTERs.
TODO: stop fetches during ALTER.
 2017-05-25 13:08:52 +03:00
Alexey Milovidov
a77369d082 Preparation: Storages do not modify query [#CLICKHOUSE-31]. 2017-05-25 04:12:41 +03:00
Alexey Milovidov
557c3dc7f4 Preparation: removed support for unreplicated data in Replicated tables [#CLICKHOUSE-31]. 2017-05-25 00:38:56 +03:00
Alexey Milovidov
48f65e9097 Preparation: untangling [#CLICKHOUSE-31]. 2017-05-25 00:06:29 +03:00
Vitaliy Lyudvichenko
d9637c2980 MergeTreeDataPart refactoring. [#CLICKHOUSE-2989] 
Use relative_path to store actual part path.
Use renameTo() to move parts.
 2017-05-24 23:23:58 +03:00
Alexey Milovidov
0f252046ca Preparations [#CLICKHOUSE-2] 2017-05-22 01:25:25 +03:00
Alexey Zatelepin
9c6829e982 Fix replica delay counting in case of ZooKeeper communication problems [#CLICKHOUSE-2141] 2017-05-20 17:32:00 +04:00
Vitaliy Lyudvichenko
1d00f45dd3 Detect race condition between fetches and merges of the same part. [#CLICKHOUSE-2989] 2017-05-17 21:44:16 +04:00
Vitaliy Lyudvichenko
6262fc8650 Fix inccorect columns size in MergeTreeData after ALTER. [#CLICKHOUSE-2] 2017-05-15 10:34:40 +04:00
Alexey Milovidov
ee6a7ee204 Allow to disable replicated max parallel fetches/sends [#CLICKHOUSE-2960]. 2017-04-20 21:37:01 +03:00
alexey-milovidov
512d86ceaf Merge pull request #706 from yandex/lagging-replicas-failover 
Check for stale replicas when performing distributed queries
 2017-04-20 21:30:02 +03:00
Alexey Milovidov
2d3e09af82 Fixed error [#CLICKHOUSE-2960]. 2017-04-20 05:22:17 +03:00
Alexey Zatelepin
ba254a082e Merge branch 'master' into lagging-replicas-failover 
Conflicts:
	dbms/src/Core/ErrorCodes.cpp
 2017-04-19 21:31:29 +03:00
Alexey Milovidov
47d2ca4136 Fixed error with fetches and sends limit when queue is traversed without pause [#CLICKHOUSE-2960]. 2017-04-18 23:36:19 +03:00
Yuri Dyachenko
8fb2b7ee3d fix zookeeper format version 4 2017-04-17 21:15:14 +03:00
alexey-milovidov
5a8344382b Update StorageReplicatedMergeTree.cpp 2017-04-17 21:15:14 +03:00
Yuri Dyachenko
8792c4f3ca add 'deduplicate' flag into zookeeper 'merge' Entry 2017-04-17 21:15:14 +03:00
Yuri Dyachenko
10082f3ef4 support DEDUPLICATE option in OPTIMIZE query 2017-04-17 21:15:14 +03:00
Alexey Zatelepin
2bd089d589 start counting absolute replica delay when there are problems with queue update [#CLICKHOUSE-2141] 2017-04-17 18:06:12 +03:00
f1yegor
0b243a8b01 translate comments 2017-04-16 23:17:32 +03:00
proller
1bc311eefd ReplicatedMergeTree: Max streams to send data [#CLICKHOUSE-2878] (#656) 
* ReplicatedMergeTree: Max streams to send data [#CLICKHOUSE-2878]

* fix

* better messages on client

* change code to 429 TOO_MANY_REQUESTS

* wip

* better message

* Update InterserverIOHandler.h

* Update InterserverIOHTTPHandler.cpp

* Update StorageReplicatedMergeTree.cpp
 2017-04-06 16:03:23 +03:00
Alexey Milovidov
310736cd7e Moved headers and sources to same place [#CLICKHOUSE-3]. 2017-04-01 12:22:42 +03:00
Alexey Milovidov
137ad95929 Changed tabs to spaces in code [#CLICKHOUSE-3]. 2017-04-01 11:35:09 +03:00
Alexey Milovidov
4423420c20 Merge branch 'master' of github.com:yandex/ClickHouse 2017-03-31 19:00:42 +03:00
Alexey Milovidov
4f44445ff0 Removed all trailing whitespaces [#CLICKHOUSE-2]. find . -name '*.h' -or -name '*.cpp' -or -name '*.cmake' -or -name 'CMakeLists.txt' -or -name '*.html' -or -name '*.xml' | grep -v contrib | xargs sed -i -r -e 's/\s+$//' 2017-03-31 19:00:30 +03:00
proller
fbdcab343f Limit fetching threads [#CLICKHOUSE-2878] (#648) 
* Max parallel fetches limit

* use MergeTreeSettings.h

* clean
 2017-03-31 17:46:48 +03:00
Ravengg
a973e3898d Add remove for part log (#638) 
* Add REMOVE_PART

* fix

* fix

* fix codestyle
 2017-03-28 20:18:49 +03:00
Alexey Milovidov
d3ae1267eb Fixed error when parts of unlimited total size could be selected for merge in case of low free disk space [#CLICKHOUSE-2895]. 2017-03-20 23:29:12 +03:00
Bogdan Voronin
a3ff193fe3 Add log merge for replicas 2017-03-16 00:23:02 +04:00
Ravengg
b47cc72701 Fix part log (#591) 
* fix_part_log

* fixed language back

* add reset part_log back

* reset part_log when don't need more

* Fix header

* fix
 2017-03-14 19:58:34 +04:00
Alexey Milovidov
26788e3938 Formatting [#CLICKHOUSE-2]. 2017-03-13 21:01:46 +03:00
f1yegor
401425bb89 translate comments 2017-03-12 20:18:07 +01:00
Ravengg
b079dacfd1 Added part_log with statistics scripts (#549) 
* Added part_log

* first_test

* filter and hits_res

* Add renamer and drawer

* Add columns database and table into PartLog

* Add normal way to get table_name and database_name from part

* improve drawer

* add stats for random size parts

* Merge converter and drawer

* make drawer more informative

* add new data

* add new data

* new data

* add long range stats

* for checking best way

* Add add_parts script

* Good style for global merge

* delete commented code

* Fixed spaces to tabs

* Note that Stopwatch is started automatically.

* Style

* Update StorageMergeTree.cpp

* Update StorageReplicatedMergeTree.cpp

* Switch act_time_ms to duration_ms

* Added ability to disable part_log

* fixed getPartLog

* fix usage getPartLog

* fix
 2017-03-07 21:13:54 +04:00
proller
05e87ad7a2 fix scope_exit formatting 2017-03-04 00:10:41 +03:00
Alexey Milovidov
c9b4a163be Added info about uncompressed size to system.columns table [#CLICKHOUSE-2831]. 2017-02-12 01:31:22 +04:00
Alexey Milovidov
d065087506 Fixed timeouts of ALTER [#CLICKHOUSE-2836]. 2017-02-10 21:09:49 +03:00
Alexey Zatelepin
aace171ef6 forbid multidimensional array columns in MergeTree tables [#CLICKHOUSE-2803] 2017-02-10 09:05:33 +04:00
Vitaliy Lyudvichenko
c9e52a5cf9 Add requested PR changes. [#CLICKHOUSE-2145] [issue #325] 2017-01-24 22:26:29 +04:00
Vitaliy Lyudvichenko
20a5ae4bc4 Add protection from accidental DROP queries. [#CLICKHOUSE-2184] 2017-01-24 22:26:29 +04:00
Vitaliy Lyudvichenko
1c38a89785 Add source_part_names column into system.merges. [#CLICKHOUSE-2145] 2017-01-24 22:26:29 +04:00
Alexey Milovidov
58e5dad1a1 Squashed commit of the following: 
commit e712f469a5
Author: Alexey Milovidov <milovidov@yandex-team.ru>
Date:   Sat Jan 14 11:59:13 2017 +0300

    Less dependencies [#CLICKHOUSE-2]

commit 2a00282308
Author: Alexey Milovidov <milovidov@yandex-team.ru>
Date:   Sat Jan 14 11:58:30 2017 +0300

    Less dependencies [#CLICKHOUSE-2]

commit 9e06f407c8
Author: Alexey Milovidov <milovidov@yandex-team.ru>
Date:   Sat Jan 14 11:55:14 2017 +0300

    Less dependencies [#CLICKHOUSE-2]

commit 9581620f1e
Author: Alexey Milovidov <milovidov@yandex-team.ru>
Date:   Sat Jan 14 11:54:22 2017 +0300

    Less dependencies [#CLICKHOUSE-2]

commit 2a8564c68c
Author: Alexey Milovidov <milovidov@yandex-team.ru>
Date:   Sat Jan 14 11:47:34 2017 +0300

    Less dependencies [#CLICKHOUSE-2]

commit cf60632d78
Author: Alexey Milovidov <milovidov@yandex-team.ru>
Date:   Sat Jan 14 11:40:09 2017 +0300

    Less dependencies [#CLICKHOUSE-2]

commit ee3d1dc6e0
Author: Alexey Milovidov <milovidov@yandex-team.ru>
Date:   Sat Jan 14 11:22:49 2017 +0300

    Less dependencies [#CLICKHOUSE-2]

commit 65592ef711
Author: Alexey Milovidov <milovidov@yandex-team.ru>
Date:   Sat Jan 14 11:18:17 2017 +0300

    Less dependencies [#CLICKHOUSE-2]

commit 37972c2573
Author: Alexey Milovidov <milovidov@yandex-team.ru>
Date:   Sat Jan 14 11:17:06 2017 +0300

    Less dependencies [#CLICKHOUSE-2]

commit dd909d1499
Author: Alexey Milovidov <milovidov@yandex-team.ru>
Date:   Sat Jan 14 11:16:28 2017 +0300

    Less dependencies [#CLICKHOUSE-2]

commit 3cf43266ca
Author: Alexey Milovidov <milovidov@yandex-team.ru>
Date:   Sat Jan 14 11:15:42 2017 +0300

    Less dependencies [#CLICKHOUSE-2]

commit 6731a3df96
Author: Alexey Milovidov <milovidov@yandex-team.ru>
Date:   Sat Jan 14 11:13:35 2017 +0300

    Less dependencies [#CLICKHOUSE-2]

commit 1b5727e0d5
Author: Alexey Milovidov <milovidov@yandex-team.ru>
Date:   Sat Jan 14 11:11:18 2017 +0300

    Less dependencies [#CLICKHOUSE-2]

commit bbcf726a55
Author: Alexey Milovidov <milovidov@yandex-team.ru>
Date:   Sat Jan 14 11:09:04 2017 +0300

    Less dependencies [#CLICKHOUSE-2]

commit c03b477d5e
Author: Alexey Milovidov <milovidov@yandex-team.ru>
Date:   Sat Jan 14 11:06:30 2017 +0300

    Less dependencies [#CLICKHOUSE-2]

commit 2986e2fb04
Author: Alexey Milovidov <milovidov@yandex-team.ru>
Date:   Sat Jan 14 11:05:44 2017 +0300

    Less dependencies [#CLICKHOUSE-2]

commit 5d6cdef13d
Author: Alexey Milovidov <milovidov@yandex-team.ru>
Date:   Sat Jan 14 11:04:53 2017 +0300

    Less dependencies [#CLICKHOUSE-2]

commit f2b819b25c
Author: Alexey Milovidov <milovidov@yandex-team.ru>
Date:   Sat Jan 14 11:01:47 2017 +0300

    Less dependencies [#CLICKHOUSE-2]
 2017-01-14 12:00:19 +03:00
Alexey Milovidov
d89ee33ce2 Squashed commit of the following: 
commit c567d4e1fe
Author: Alexey Milovidov <milovidov@yandex-team.ru>
Date:   Fri Jan 6 20:35:01 2017 +0300

    Style [#METR-2944].

commit 26bf3e1228
Author: Alexey Milovidov <milovidov@yandex-team.ru>
Date:   Fri Jan 6 20:33:11 2017 +0300

    Miscellaneous [#METR-2944].

commit eb946f4c6f
Author: Alexey Milovidov <milovidov@yandex-team.ru>
Date:   Fri Jan 6 20:30:19 2017 +0300

    Miscellaneous [#METR-2944].

commit 78c867a147
Author: Alexey Milovidov <milovidov@yandex-team.ru>
Date:   Fri Jan 6 20:11:41 2017 +0300

    Miscellaneous [#METR-2944].

commit 6604c5c83c
Author: Alexey Milovidov <milovidov@yandex-team.ru>
Date:   Fri Jan 6 19:56:15 2017 +0300

    Miscellaneous [#METR-2944].

commit 23fbf05c1d
Author: Alexey Milovidov <milovidov@yandex-team.ru>
Date:   Fri Jan 6 19:47:52 2017 +0300

    Miscellaneous [#METR-2944].

commit 98772faf11
Author: Alexey Milovidov <milovidov@yandex-team.ru>
Date:   Fri Jan 6 19:46:05 2017 +0300

    Miscellaneous [#METR-2944].

commit 3dc636ab9f
Author: Alexey Milovidov <milovidov@yandex-team.ru>
Date:   Fri Jan 6 19:39:46 2017 +0300

    Miscellaneous [#METR-2944].

commit 3e16aee954
Author: Alexey Milovidov <milovidov@yandex-team.ru>
Date:   Fri Jan 6 19:38:03 2017 +0300

    Miscellaneous [#METR-2944].

commit ae7e7e90eb
Author: Alexey Milovidov <milovidov@yandex-team.ru>
Date:   Fri Jan 6 19:34:15 2017 +0300

    Miscellaneous [#METR-2944].
 2017-01-06 20:41:19 +03:00
Vitaliy Lyudvichenko
07d720565c Added profile events for INSERTs. [#METR-23930] 
Added new column and CurrentMetrics data into system.events table.
Removed unnecessary code from Context and Merge list. [#METR-23911]
 2016-12-31 02:39:53 +03:00
Vitaliy Lyudvichenko
108bfac17e Added memory tracking events and columns for merges. [#METR-23911] 2016-12-31 02:39:53 +03:00
Vladimir Chebotarev
d172cb1fc2 StorageReplicatedMergeTree: Correct taking /quorum/last_part into account when merging. [#METR-23826] 2016-12-15 17:32:47 +03:00
Alexey Milovidov
54af522de1 Preparation [#METR-2944]. 2016-12-12 10:24:56 +03:00
Alexey Milovidov
d5a002f441 Using internal boost library [#METR-2944]. 2016-12-12 07:57:55 +03:00
Alexey Milovidov
d605bfdc9d Using internal boost library [#METR-2944]. 2016-12-12 07:57:09 +03:00
Alexey Milovidov
e97ecb6cf7 Merge branch 'master' into METR-19266 
Conflicts:
	dbms/CMakeLists.txt
	dbms/include/DB/Functions/FunctionsArray.h
	dbms/include/DB/Functions/FunctionsString.h
	dbms/include/DB/Parsers/ExpressionListParsers.h
	dbms/include/DB/Storages/MergeTree/MergeTreeBlockInputStream.h
	dbms/include/DB/Storages/MergeTree/MergeTreeReader.h
	dbms/include/DB/Storages/MergeTree/MergeTreeThreadBlockInputStream.h
	dbms/include/DB/Storages/MergeTree/MergedBlockOutputStream.h
	dbms/src/Core/ErrorCodes.cpp
	dbms/src/Parsers/ASTFunction.cpp
	dbms/src/Storages/MergeTree/MergeTreeData.cpp
	dbms/src/Storages/MergeTree/MergeTreeDataPart.cpp
	dbms/src/Storages/MergeTree/MergeTreeReader.cpp
	dbms/src/Storages/StorageLog.cpp
 2016-12-10 07:51:36 +03:00
proller
26fe10a7b4 Merge remote-tracking branch 'upstream/master' into METR-23466 2016-11-25 03:22:30 +03:00
Alexey Milovidov
7d280a55a3 Fixed error that server could not shutdown while fetching parts [#METR-23616]. 2016-11-25 02:18:24 +03:00
Alexey Milovidov
167782c230 Addition to prev. revision [#METR-23615]. 2016-11-25 01:43:49 +03:00
Alexey Milovidov
8e5889cf0e Improved merge selecting algorithm for Replicated tables [#METR-23615]. 2016-11-25 01:41:27 +03:00
proller
eeeacd0805 merge upstream 2016-11-23 01:33:02 +03:00
Alexey Milovidov
bba5dcbcfa Fixed deadlock [#METR-23557]. 2016-11-22 22:27:45 +03:00
Alexey Milovidov
3b635bf641 Merge branch 'master' of github.com:yandex/ClickHouse 2016-11-22 20:42:21 +03:00
Alexey Milovidov
a464f2303d Better logic for ALTER in ReplicatedMergeTree [#METR-22214]. 2016-11-22 20:41:45 +03:00
proller
e280569453 Everything [maybe] works! 2016-11-22 18:03:54 +03:00
Alexey Milovidov
9018906e1a Better [#METR-2944]. 2016-11-20 15:43:20 +03:00
Alexey Milovidov
866e64a65e Fixed deadlock in test when many replicas on single server try to execute multiple attaches and wait for each other [#METR-23507]. 2016-11-16 09:08:29 +03:00
Alexey Milovidov
5b72f0956a Setting level to zero after ATTACH [#METR-23438]. 2016-11-11 20:33:43 +03:00
Alexey Milovidov
bee00bfcfc MergeSelector: development [#METR-21841]. 2016-10-30 14:05:45 +03:00
Alexey Milovidov
74c3b30622 dbms: Simplified logic of selecting merges [#METR-21841]. 2016-10-28 01:50:02 +03:00
Alexey Milovidov
2dd43be6e6 Don't hold shared_ptr to zookeeper for long time, because it makes very difficult to remove ephemeral nodes in case of temporary communication errors and thus breaks leader election [#METR-23272]. 2016-10-24 15:34:08 +03:00
Alexey Milovidov
9af5ce1e3b Separate compilation of ProfileEvents; comments [#METR-23237]. 2016-10-24 05:02:37 +03:00
Alexey Arno
924402f33e dbms: Server: merged from master [#METR-19266] 2016-10-19 18:00:56 +03:00
Vitaliy Lyudvichenko
669e2982a5 Made merge_selecting_thread creation & deletion atomic. [#METR-22471] 2016-10-14 14:47:11 +03:00
Vitaliy Lyudvichenko
aa863ea7bb Fixed std::terminate on DROP of StorageReplicatedMergeTree. [#METR-22471] 2016-10-14 13:17:05 +03:00
Alexey Milovidov
6b2755bf2a Non significant change [#METR-2944]. 2016-09-23 23:33:06 +03:00