Vadim
8f3350216f
Merge branch 'master' into CLICKHOUSE-3847
2018-08-21 13:27:18 +03:00
VadimPE
99a26f17f6
CLICKHOUSE-3847 fix settings
2018-08-21 13:23:48 +03:00
Alexey Milovidov
3b39f6a4c3
Split ZooKeeperExceptions to three different metrics #2316
2018-08-21 04:27:22 +03:00
VadimPE
a49cc41279
CLICKHOUSE-3847 add support old version of CH server
2018-08-21 02:08:45 +03:00
VadimPE
4064ec0bc2
CLICKHOUSE-3847 add value '1' and '2' in is_lost
2018-08-20 20:15:04 +03:00
VadimPE
0c119d7667
CLICKHOUSE-3847 fix bugs
2018-08-20 16:31:24 +03:00
Nikolai Kochetov
391b4b5927
Merged with master.
2018-08-20 15:25:06 +03:00
proller
62b6b6d73e
Rare segfault fix
2018-08-17 21:23:10 +03:00
VadimPE
c8e2506b5f
ISSUES-2581 add check when replica do not have part added with quorum
2018-08-14 19:40:51 +03:00
VadimPE
a37ad50329
ISSUES-2581 fix build
2018-08-14 13:24:39 +03:00
VadimPE
3c0f6ce87b
ISSUES-2581 fix SELECT with select_sequential_consistency
2018-08-14 12:57:18 +03:00
Alexey Milovidov
36db216abf
Preparation for extra warnings [#CLICKHOUSE-2]
2018-08-10 07:02:56 +03:00
Alexey Milovidov
d8ca4345b6
Fixed style violations [#CLICKHOUSE-2]
2018-08-10 04:41:54 +03:00
Vadim
44cb046eb9
Update StorageReplicatedMergeTree.cpp
2018-08-09 19:25:19 +03:00
Vadim
434017d193
Update StorageReplicatedMergeTree.cpp
2018-08-09 18:09:05 +03:00
VadimPE
75900f494f
CLICKHOUSE-3847 add "is_lost". load() in Queue can check duplicate records
2018-08-09 18:06:39 +03:00
VadimPE
d5a6967fc4
CLICKHOUSE-3847 fix
2018-08-09 11:44:49 +03:00
alesapin
402815977f
Add new error in fetchPartition op
2018-08-08 21:01:25 +03:00
VadimPE
c16d876867
CLICKHOUSE-3847 fix race in set log_pointer in cloneReplica()
2018-08-08 18:25:07 +03:00
VadimPE
4c0bb93345
CLICKHOUSE-3847 fix UInt - UInt. del clearQueue()
2018-08-08 17:07:39 +03:00
VadimPE
8d28b9f6b4
CLICKHOUSE-3847 we can do pullToLog() after we mark replica is_active and before we calc delay.
2018-08-08 16:53:06 +03:00
VadimPE
32b6965cf2
CLICKHOUSE-3847 Update. Add new settings. Del is_lost
2018-08-07 18:21:42 +03:00
alexey-milovidov
0ef2a94d46
Merge pull request #2782 from VadimPE/CLICKHOUSE-3714
...
CLICKHOUSE-3714 add max_partition_size_to_drop
2018-08-06 20:20:59 +03:00
alesapin
425918afe1
CLICKHOUSE-3880: Fix replace partition logic inside replicated merge tree
2018-08-06 15:59:51 +03:00
VadimPE
490efebe62
CLICKHOUSE-3714 add size pf all part in partition
2018-08-06 11:27:59 +03:00
Alexey Milovidov
2016374d20
Merging #2226
2018-08-05 10:05:43 +03:00
Alexey Zatelepin
b6b54eea52
old mutations cleaner for ReplicatedMergeTree engines [#CLICKHOUSE-3828]
2018-08-03 22:21:44 +03:00
VadimPE
b97fa2eb4d
CLICKHOUSE-3714 bool ckeck_CanBeDropped -> void ...
2018-08-03 12:39:01 +03:00
VadimPE
6142d9ae6a
CLICKHOUSE-3714 fix function names and get partition size
2018-08-03 11:33:57 +03:00
VadimPE
98c33de4e3
CLICKHOUSE-3714
2018-08-02 12:03:38 +03:00
VadimPE
08fe108ba8
CLICKHOUSE-3714 fix methods name
2018-08-02 11:48:57 +03:00
Vadim
034a88a504
Merge branch 'master' into CLICKHOUSE-3714
2018-08-01 20:42:56 +03:00
VadimPE
66d9ba3eb7
CLICKHOUSE-3714 add max_partition_size_to_drop
2018-08-01 20:41:18 +03:00
alesapin
e81108b954
CLICKHOUSE-3863: Add more convinient exception message
2018-08-01 00:29:47 +03:00
alesapin
2db2d208e1
CLICKHOUSE-3863: Add HTTPS support for interserver replication
2018-08-01 00:29:47 +03:00
Alexey Zatelepin
4863c5c178
fix potential segfault in MergeTreeData::delayInsertOrThrowIfNeeded [#CLICKHOUSE-3862]
2018-07-30 21:47:24 +03:00
Alexey Zatelepin
cc7950a733
don't destroy Cleanup- and AlterThread objects, just deactivate the tasks [#CLICKHOUSE-3862]
...
Otherwise segfaults are possible when the objects are recreated in the middle of the queue task execution.
2018-07-30 21:47:24 +03:00
alesapin
6abbfbce87
CLICKHOUSE-3832: Add HTTP Basic authentification in replication protocol
2018-07-26 18:10:57 +03:00
alexey-milovidov
68b57e605a
Merge branch 'master' into background-processing-pool-backoff
2018-07-16 06:27:43 +03:00
Alexey Zatelepin
78b83d78f0
execute part mutations in a background pool [#CLICKHOUSE-3748]
2018-07-16 06:22:22 +03:00
Alexey Zatelepin
ce17868a5c
execute mutations synchronously for StorageMergeTree [#CLICKHOUSE-3748]
2018-07-16 06:22:22 +03:00
Amos Bird
ea01a3b22e
add OPTIMIZE FINAL support for ReplicatedMergeTree.
2018-07-09 18:03:23 +03:00
Alexey Milovidov
7dc1276afe
Little better #2281
2018-07-08 08:26:51 +03:00
Alexey Milovidov
0a188389c0
Fixed error #2599
2018-07-05 21:56:33 +03:00
Alexey Zatelepin
a41ee1f0a7
switch to ephemeral nodes for block number locks [#CLICKHOUSE-3802]
2018-07-05 20:06:33 +03:00
Alexey Zatelepin
0a46d231ed
speedup merge selection: async exists check for selected parts [#CLICKHOUSE-3802]
2018-07-05 20:06:33 +03:00
Alexey Zatelepin
116ed0a59e
add RestartingThread wakeup for faster session recovery
2018-07-05 19:30:52 +03:00
Nikolai Kochetov
ba4cf6faf0
Merged with master.
2018-06-25 15:08:21 +03:00
Alexey Zatelepin
a6c61848f1
mutations finalizing task [#CLICKHOUSE-3747]
2018-06-21 22:47:44 +03:00
alexey-milovidov
08ec751a8c
Merge pull request #2504 from yandex/system-mutations-table
...
[WIP] system.mutations table skeleton
2018-06-13 23:00:30 +03:00
Alexey Zatelepin
4ee581117b
system.mutations table skeleton [#CLICKHOUSE-3747]
2018-06-13 17:15:36 +03:00
Alexey Zatelepin
c5c601f662
save serialized commands in ReplicatedMergeTreeMutationEntry; split InterpreterAlterQuery::parseAlter() [#CLICKHOUSE-3747]
2018-06-13 17:14:08 +03:00
Alexey Milovidov
d864f39ec7
Merge branch 'master' into zhang2014-feature/support_truncate
2018-06-09 19:10:06 +03:00
Alexey Milovidov
d56c199f07
Merging: Fixed half of problems #2260
2018-06-09 18:48:22 +03:00
Alexey Milovidov
0bc678474e
Merge branch 'feature/support_truncate' of https://github.com/zhang2014/ClickHouse into zhang2014-feature/support_truncate
2018-06-09 17:24:50 +03:00
Alexey Zatelepin
3947f25365
add general comment about the mutation algorithm [#CLICKHOUSE-3747]
2018-06-05 17:55:35 +03:00
Alexey Milovidov
0f96a5c1c5
Suppressions for PVS-Studio [#CLICKHOUSE-3]
2018-06-04 22:46:47 +03:00
Alexey Zatelepin
e55097b866
introduce constants for max level and max block number
2018-06-04 14:23:47 +03:00
Alexey Zatelepin
f63d2dc3c8
cannot take a number of unlocked abandonable lock
2018-05-31 20:28:17 +03:00
Alexey Zatelepin
c9bb986330
Merge branch 'master' into replicated-mergetree-mutations
...
Conflicts:
dbms/src/Storages/MergeTree/AbandonableLockInZooKeeper.h
dbms/src/Storages/MergeTree/ReplicatedMergeTreePartCheckThread.cpp
dbms/src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp
dbms/src/Storages/MergeTree/ReplicatedMergeTreeQueue.h
dbms/src/Storages/StorageReplicatedMergeTree.cpp
2018-05-31 18:52:42 +03:00
Vitaliy Lyudvichenko
dc5de3bbec
More optimal conflicts check in future_parts. [#CLICKHOUSE-2]
2018-05-31 17:14:50 +03:00
Vitaliy Lyudvichenko
888f5f4a24
Allow merges after CLEAR COLUMN. [#CLICKHOUSE-3]
...
Resolves #2315
2018-05-31 17:14:50 +03:00
Alexey Zatelepin
d89221c458
easier init and deactivation of BackgroundSchedulePool tasks
2018-05-31 16:18:52 +03:00
Alexey Zatelepin
f6c72f8e56
Merge branch 'master' into replicated-mergetree-mutations
...
Conflicts:
dbms/src/Common/ActionBlocker.h
dbms/src/Storages/MergeTree/ActiveDataPartSet.cpp
dbms/src/Storages/MergeTree/ActiveDataPartSet.h
dbms/src/Storages/MergeTree/MergeTreeData.h
dbms/src/Storages/MergeTree/MergeTreeDataPart.cpp
dbms/src/Storages/MergeTree/ReplicatedMergeTreeAlterThread.cpp
dbms/src/Storages/MergeTree/ReplicatedMergeTreeLogEntry.cpp
dbms/src/Storages/MergeTree/ReplicatedMergeTreeLogEntry.h
dbms/src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp
dbms/src/Storages/MergeTree/ReplicatedMergeTreeQueue.h
dbms/src/Storages/MergeTree/ReplicatedMergeTreeRestartingThread.cpp
dbms/src/Storages/StorageMergeTree.cpp
dbms/src/Storages/StorageReplicatedMergeTree.cpp
dbms/src/Storages/StorageReplicatedMergeTree.h
2018-05-28 18:37:30 +03:00
Alexey Zatelepin
59fe12ed15
part log and metrics [#CLICKHOUSE-3747]
2018-05-25 23:13:00 +03:00
Alexey Zatelepin
aff6ad42ed
support empty parts [#CLICKHOUSE-3747]
2018-05-25 23:13:00 +03:00
Alexey Zatelepin
2b80fbf972
DROP PARTITION now works with mutated parts [#CLICKHOUSE-3747]
2018-05-25 23:13:00 +03:00
Alexey Zatelepin
fdb33d8f3c
execute mutation log entries [#CLICKHOUSE-3747]
2018-05-25 23:13:00 +03:00
Alexey Zatelepin
1147e052e8
select and write part mutations to queue [#CLICKHOUSE-3747]
2018-05-25 23:13:00 +03:00
Alexey Zatelepin
2721daf782
don't select merges of parts with different mutation versions [#CLICKHOUSE-3747]
2018-05-25 23:06:28 +03:00
Alexey Zatelepin
4c661df146
notify relevant events directly from queue [#CLICKHOUSE-3747]
2018-05-25 23:06:23 +03:00
Alexey Zatelepin
42850f09b7
load mutations from ZK [#CLICKHOUSE-3747]
2018-05-25 23:06:15 +03:00
Alexey Zatelepin
49439aa2d2
write mutations to ZK [#CLICKHOUSE-3747]
2018-05-25 23:06:08 +03:00
Alexey Zatelepin
4ca3bf65f3
test getting abandonable lock in all partitions [#CLICKHOUSE-3747]
2018-05-25 23:05:10 +03:00
Alexey Zatelepin
17437a95bf
merge selection based on checking current inserts [#CLICKHOUSE-3732]
2018-05-25 23:03:56 +03:00
Alexey Milovidov
13762471da
Changed DayNum_t to DayNum as Amos Bird suggested [#CLICKHOUSE-3]
2018-05-25 16:29:15 +03:00
Vitaliy Lyudvichenko
c888903488
Add query ALTER TABLE t1 [REPLACE|ATTACH] PARTITION FROM t2
...
* Add ATTACH PARTITION FROM table for MergeTree. [#CLICKHOUSE-3546]
* Implemented replicated case on non-leader replica. [#CLICKHOUSE-3546]
* Disable merges in the dropping range. [#CLICKHOUSE-3546]
* DROP PARTITION is atomic and simpler now. [#CLICKHOUSE-3546]
* Implemented more SYSTEM queries. [#CLICKHOUSE-2931] [#CLICKHOUSE-3546]
SYSTEM queries:
RESTART REPLICAS
SYNC REPLICA db.name
STOP MERGES [db.name]
START MERGES [db.name]
STOP FETCHES [db.name]
START FETCHES [db.name]
STOP REPLICATED SENDS [db.name]
START REPLICATED SENDS [db.name]
STOP REPLICATION QUEUES [db.name]
START REPLICATION QUEUES [db.name]
* Fixed a bunch of bugs in REPLACE PARTITION. [#CLICKHOUSE-3546]
* Add tests for REPLACE PARTITION and SYSTEM. [#CLICKHOUSE-3546]
* Add system.part_log logging. [#CLICKHOUSE-3546]
* Fixed long wait in SYNC REPLICA. [#CLICKHOUSE-3546]
* Add requested changes. [#CLICKHOUSE-3546]
Fixed clickhouse-client bad return code.
* Add requested chenges. [#CLICKHOUSE-3546]
* Add requested chenges. [#CLICKHOUSE-3546]
2018-05-21 16:49:54 +03:00
zhang2014
fa865636ce
ISSUES-2259 add comment & move delete database metadata implementation
2018-05-21 11:38:46 +08:00
zhang2014
6613e56784
ISSUES-2259 support truncate syntax
2018-05-21 11:38:46 +08:00
alexey-milovidov
517318334a
Merge pull request #1722 from silviucpp/background-schedule-pool-fix
...
Background schedule pool fixed
2018-05-19 19:00:43 +03:00
Vitaliy Lyudvichenko
4b093e3e0a
Don't make extra checksums checks when commit a part. [#CLICKHOUSE-2]
...
This patch is very crucial in case of many replicas.
2018-05-18 02:01:36 +03:00
Vitaliy Lyudvichenko
e152f223c9
Fix nodes leak in case of session expiration. [#CLICKHOUSE-2]
2018-04-26 22:46:45 +03:00
Silviu Caragea
1dd5a704fc
Merge remote-tracking branch 'origin/master' into background-schedule-pool-fix
...
# Conflicts:
# dbms/src/Common/ZooKeeper/LeaderElection.h
# dbms/src/Storages/MergeTree/ReplicatedMergeTreeAlterThread.cpp
# dbms/src/Storages/MergeTree/ReplicatedMergeTreeCleanupThread.cpp
# dbms/src/Storages/MergeTree/ReplicatedMergeTreePartCheckThread.cpp
# dbms/src/Storages/StorageReplicatedMergeTree.cpp
2018-04-24 20:11:59 +03:00
Alexey Milovidov
499b67642f
Allow to startup with replicated tables in readonly mode when there is no ZooKeeper configured [#CLICKHOUSE-2]
2018-04-21 21:41:06 +03:00
Nikolai Kochetov
a6375d6155
merged with master
2018-04-20 17:15:03 +03:00
Alexey Milovidov
da94c7dd91
Exit from queueUpdatingThread when ZooKeeper session is expired (non-significant change) [#CLICKHOUSE-2]
2018-04-19 21:01:50 +03:00
Nikolai Kochetov
dd10c2073c
merged with master
2018-04-19 18:10:44 +03:00
Nikolai Kochetov
a3308bf675
added IStorage::getQueryProcessingStage
2018-04-19 17:47:09 +03:00
Silviu Caragea
32dd4554da
Merge remote-tracking branch 'github/master' into background-schedule-pool-fix
...
# Conflicts:
# dbms/src/Storages/StorageReplicatedMergeTree.cpp
2018-04-19 09:29:49 +03:00
alexey-milovidov
089e71d9a5
Merge pull request #2235 from yandex/small-enhancements
...
Small enhancements
2018-04-17 22:45:01 +03:00
Vitaliy Lyudvichenko
bf832b3ea6
Fixed OPTIMIZE after RENAME of replicated table. [#CLICKHOUSE-2]
2018-04-17 20:59:42 +03:00
proller
27b4af9dd7
Change obsolete comment
2018-04-16 23:25:07 +03:00
proller
157067b127
CLICKHOUSE-3700: Fix empty password on replica query
2018-04-10 17:49:52 +03:00
Silviu Caragea
f1b8d4bc39
Merge remote-tracking branch 'github/master' into background-schedule-pool-fix
2018-04-10 16:27:17 +03:00
Silviu Caragea
c547c5a7b0
Merge remote-tracking branch 'origin/background-schedule-pool-fix'
...
# Conflicts:
# dbms/src/Common/ZooKeeper/LeaderElection.h
# dbms/src/Common/ZooKeeper/ZooKeeper.cpp
# dbms/src/Storages/MergeTree/ReplicatedMergeTreeAlterThread.cpp
# dbms/src/Storages/MergeTree/ReplicatedMergeTreeCleanupThread.cpp
# dbms/src/Storages/MergeTree/ReplicatedMergeTreeCleanupThread.h
# dbms/src/Storages/MergeTree/ReplicatedMergeTreePartCheckThread.cpp
# dbms/src/Storages/MergeTree/ReplicatedMergeTreeRestartingThread.cpp
# dbms/src/Storages/StorageReplicatedMergeTree.cpp
2018-04-10 16:20:14 +03:00
Alexey Milovidov
d3cf85057b
Removed useless code [#CLICKHOUSE-2]
2018-04-09 18:49:12 +03:00
Alexey Milovidov
0a4b349f73
Merge branch 'master' of github.com:yandex/ClickHouse
2018-04-07 00:47:18 +03:00
Alexey Milovidov
4e5e0fa664
ReplicatedMergeTree: Better diagnostics [#CLICKHOUSE-2]
2018-04-07 00:46:57 +03:00
alexey-milovidov
84726e17e8
Merge pull request #2185 from yandex/fix-leader-election-race
...
Fix races in leader election.
2018-04-07 00:18:26 +03:00
Alexey Milovidov
0d8d99e7a4
Fixed typo [#CLICKHOUSE-2]
2018-04-06 22:48:54 +03:00
Alexey Zatelepin
3a8ca70d1f
fix races in leader election [#CLICKHOUSE-3533]
2018-04-06 19:10:23 +03:00
Silviu Caragea
f811da7ed4
Move merge selection state into ReplicatedMergeTreeMergeSelectingThread
2018-04-02 15:45:55 +03:00
alexey-milovidov
428ee63a04
Merge branch 'master' into libzookeeper-rewrite
2018-04-01 17:31:20 +03:00
Silviu Caragea
1418e339e4
Rename merge_selecting_handle with merge_selecting_task_handle for consistency
2018-03-26 23:18:19 +03:00
Silviu Caragea
0aa9b9efbe
Use consistent names for the task variables (change from next_update_task_handle to merge_selecting_handle) (reverted from commit 50992843c6
)
2018-03-26 23:12:17 +03:00
Silviu Caragea
50992843c6
Use consistent names for the task variables (change from next_update_task_handle to merge_selecting_handle)
2018-03-26 22:59:58 +03:00
Silviu Caragea
31874ed172
Use consistent names for the task variables
2018-03-26 22:37:47 +03:00
Alex Zatelepin
c5b04f7f17
Add info about total uncompressed part size to system tables ( #2052 )
...
* column size calculating routines now take into account multiple streams [#CLICKHOUSE-2831]
* rename for clarity
* use more appropriate method for calculating table size
* add info about total uncompressed part size to system.parts and system.parts_columns [#CLICKHOUSE-2831]
* rename columns for clarity [#CLICKHOUSE-2831]
* count shared substreams only once [#CLICKHOUSE-2831]
2018-03-26 17:18:04 +03:00
Alexey Milovidov
522bf01db9
Using different ZooKeeper library (development) [#CLICKHOUSE-2]
2018-03-25 03:15:52 +03:00
Alexey Milovidov
ae9633c1e5
Using different ZooKeeper library (development) [#CLICKHOUSE-2]
2018-03-24 23:00:16 +03:00
Alexey Milovidov
26584844b6
Rewriting ZooKeeper library [#CLICKHOUSE-2]
2018-03-24 04:00:12 +03:00
Alexey Milovidov
10be6ca8f8
Rewriting ZooKeeper library [#CLICKHOUSE-2]
2018-03-24 03:45:04 +03:00
Vitaliy Lyudvichenko
883de7a2a6
startup() waits for initialization of ReplicatedMergeTreeRestartingThread. [#CLICKHOUSE-2]
2018-03-23 00:26:46 +03:00
Vitaliy Lyudvichenko
fe8dfd3038
Add minimalistic checksums. [#CLICKHOUSE-3660]
2018-03-23 00:26:46 +03:00
Silviu Caragea
6629b03af9
Fixed few of the observation of the code review
2018-03-22 17:09:30 +02:00
Silviu Caragea
b24a4b2111
Add back the buggy BackgroundSchedulePool
2018-03-22 17:07:12 +02:00
Alexey Milovidov
3c16faaff0
Merge branch 'master' of github.com:yandex/ClickHouse
2018-03-14 21:57:15 +03:00
Alexey Milovidov
57279a87c7
Whitespaces [#CLICKHOUSE-2].
2018-03-14 19:40:08 +03:00
Vitaliy Lyudvichenko
8028ba0de9
Refactoring of zkutil::ZooKeeper::multi* functions. [#CLICKHOUSE-2]
2018-03-14 16:29:03 +03:00
alexey-milovidov
9ac2569c65
Merge branch 'master' into refactor-table-declaration
2018-03-14 05:35:50 +03:00
alexey-milovidov
04cc8a52f7
Merge pull request #2032 from yandex/system-table-macro
...
System table macro
2018-03-14 02:10:30 +03:00
Alexey Zatelepin
5906c9a988
rename for clarity
2018-03-13 18:00:28 +03:00
Alexey Zatelepin
fdffbe53ac
encapsulate ColumnsDeclaration [#CLICKHOUSE-3241]
2018-03-13 17:41:25 +03:00
Alexey Milovidov
b7475b1d2c
Fixed error: wrong timeouts passed to ConnectionPoolWithFailover [#CLICKHOUSE-3641]
2018-03-12 23:25:18 +03:00
Alexey Zatelepin
78d92f6794
add some useful methods, rename for clarity [#CLICKHOUSE-3241]
2018-03-12 22:10:13 +03:00
Alexey Zatelepin
71642b0880
use ColumnsDescription everywhere [#CLICKHOUSE-3241]
2018-03-12 22:07:48 +03:00
Nikolai Kochetov
4dd7f47ad5
reload macros with config [#CLICKHOUSE-3591]
2018-03-12 21:16:51 +03:00
Alexey Milovidov
afbbd780e3
Fixed error with initialization of PartLog [#CLICKHOUSE-3354]
2018-03-10 22:57:13 +03:00
Alexey Milovidov
3f8ef38b41
Fixed wording #245
2018-03-10 02:23:15 +03:00
proller
d73fea5ab0
Fix macos compile ( #2006 )
...
* Fix macos compile
* rename cluster-copier -> copier
2018-03-07 21:17:17 +03:00
Alexey Milovidov
8c37547bfe
Fixed harmless data race [#CLICKHOUSE-2]
2018-03-03 18:35:24 +03:00
Alexey Milovidov
6e93635275
Better UNION ALL: development #1947
2018-02-25 09:35:24 +03:00
alexey-milovidov
82fa3883f8
Merge pull request #1930 from yandex/nonblocking-precommitted
...
Allow PreCommitted parts to contain each other
2018-02-22 23:35:16 +03:00
alexey-milovidov
ee41f7000f
Merge pull request #1940 from yandex/CLICKHOUSE-3571
...
Try to reduce false positive of sanity check
2018-02-22 01:59:33 +03:00
Vitaliy Lyudvichenko
47becc0b92
Try to reduce false positive of sanity check. More detail dignostics. [#CLICKHOUSE-3571]
2018-02-21 20:06:29 +03:00
Alexey Zatelepin
5bc6bd55b1
allow PreCommitted parts to contain each other, clean up code
2018-02-20 16:33:28 +03:00
Alexey Milovidov
fa50fe80a0
Added method "getHeader" in IBlockOutputStream: development [#CLICKHOUSE-2]
2018-02-19 03:45:32 +03:00
Alexey Milovidov
30b657a0da
Header in every stream: development [#CLICKHOUSE-2]
2018-02-15 21:54:12 +03:00
Nikolai Kochetov
64b3a8baa7
added secondary sort expression [#CLICKHOUSE-3479]
2018-02-13 00:13:50 +03:00
Vitaliy Lyudvichenko
94f38d744e
Merge pull request #1750 from yandex/CLICKHOUSE-3346
...
Fault-tolerant cluster copier util
2018-01-31 13:21:51 +03:00
Vitaliy Lyudvichenko
e2fb6d8c80
Merge remote-tracking branch 'origin/master' into CLICKHOUSE-3346
2018-01-25 22:00:08 +03:00
Vitaliy Lyudvichenko
61233b886a
Better system.part_log. [#CLICKHOUSE-3342]
2018-01-25 21:40:02 +03:00
Vitaliy Lyudvichenko
26e58271e9
Removed repeatable columns field from Storages declarations. [#CLICKHOUSE-2]
2018-01-25 17:42:39 +03:00
Vitaliy Lyudvichenko
b4ab7d4b8e
Make early deduplication with block number creation. [#CLICKHOUSE-3545]
2018-01-23 18:50:29 +03:00
Vitaliy Lyudvichenko
1000328571
Merge remote-tracking branch 'origin/master' into CLICKHOUSE-3346
2018-01-22 20:31:08 +03:00
alexey-milovidov
1b462b0b94
Merge pull request #1759 from yandex/CLICKHOUSE-3432
...
Fixed 'intersects previous part' due to blind OPTIMIZE
2018-01-19 21:48:41 +03:00
alexey-milovidov
6198fe4a1a
Update StorageReplicatedMergeTree.cpp
2018-01-19 21:46:22 +03:00
alexey-milovidov
e8ebf3b56a
Update StorageReplicatedMergeTree.cpp
2018-01-19 21:45:28 +03:00
alexey-milovidov
7a25b441f4
Merge pull request #1773 from yandex/small-enhancements
...
Small enhancements
2018-01-16 22:07:43 +03:00
Vitaliy Lyudvichenko
b68b60fb38
More detailed test. [#CLICKHOUSE-3]
...
Resolves #1553 .
2018-01-16 15:05:01 +03:00
proller
d97b2edc7c
Make separate libs Common/ZooKeeper and Common/ConfigProcessor ( #1749 )
...
* Make separate libs Common/ZooKeeper and Common/ConfigProcessor
renamed: dbms/src/Common/StringUtils.h -> dbms/src/Common/StringUtils/StringUtils.h
* Fix space
* Fix data in system.build_options
* merge
* merge
2018-01-15 22:07:47 +03:00
Vitaliy Lyudvichenko
554ed52920
Avoid superflous pullLogsToQueue() calls. [#CLICKHOUSE-3432]
2018-01-15 13:30:31 +03:00
Vitaliy Lyudvichenko
03bca2144d
OPTIMIZE doesn't produce intersecting merges now, add optimize_throw_if_noop setting. [#CLICKHOUSE-3432] [#CLICKHOUSE-3420]
...
Resolves #1753
2018-01-15 13:13:18 +03:00
Vitaliy Lyudvichenko
388d47bbbc
Merge branch 'CLICKHOUSE-3346-v310' into CLICKHOUSE-3346
2018-01-11 23:51:30 +03:00
Vitaliy Lyudvichenko
c818beedcb
Make data_path creation for Databases optional. [#CLICKHOUSE-2]
2018-01-11 21:55:43 +03:00
Nikolai Kochetov
cbeeb84999
added http_*_timeout settings [#CLICKHOUSE-3440]
2018-01-09 21:41:01 +03:00
Chen Yufei
88ef42c222
Aggressive merge for ReplicatedMergeTree optimize.
2018-01-09 20:44:07 +03:00
Alexey Milovidov
fcdd5de47d
Revert "Removed questionable code (2) [#CLICKHOUSE-2]."
...
This reverts commit fcabe8b888
.
2017-12-28 07:39:19 +03:00
Alexey Milovidov
798299ba89
Merge branch 'aavdonkin-master'
2017-12-28 07:33:55 +03:00
Alexey Milovidov
b2fcf06353
Merging [#CLICKHOUSE-3114]
2017-12-27 00:34:06 +03:00
Alexey Milovidov
c89041ce63
Removed questionable code (2) [#CLICKHOUSE-2].
2017-12-26 01:51:04 +03:00
Alexey Milovidov
ec71908b00
Revert "Removed questionable code (2) [#CLICKHOUSE-2]."
...
This reverts commit fcabe8b888
.
2017-12-26 00:57:29 +03:00
Alexey Milovidov
fcabe8b888
Removed questionable code (2) [#CLICKHOUSE-2].
2017-12-26 00:32:33 +03:00
Alexey Milovidov
a85b2e5a95
Removed questionable code [#CLICKHOUSE-2].
2017-12-26 00:10:46 +03:00
Alexey Zatelepin
c3348e27a4
shorter [#CLICKHOUSE-2]
2017-12-25 20:31:14 +03:00
Alexey Zatelepin
6d5ce3fa9a
revert BackgroundSchedulePool usage in Replicated tables
2017-12-22 21:25:54 +03:00
Alexey Zatelepin
835b957e95
add comments [#CLICKHOUSE-3508]
2017-12-20 20:40:36 +03:00
Alexey Zatelepin
d35baafde8
check PreCommitted parts when executing log entries and in PartChecker [#CLICKHOUSE-3508]
2017-12-20 20:04:14 +03:00
Alexey Zatelepin
2340e55d0e
revert to old behaviour of getPartIfExists() [#CLICKHOUSE-3508]
2017-12-20 20:04:14 +03:00
Alexey Milovidov
1aa99092c3
Unification of serde of data types: development [#CLICKHOUSE-2838].
2017-12-03 05:15:35 +03:00
Alexey Milovidov
54e601019c
Better [#CLICKHOUSE-2].
2017-12-02 00:40:58 +03:00
Alexey Milovidov
acd78e4891
Better [#CLICKHOUSE-2].
2017-12-02 00:13:25 +03:00
Alexey Milovidov
1a6cb2989b
Better [#CLICKHOUSE-2].
2017-12-01 21:36:55 +03:00
Vitaliy Lyudvichenko
b738d1baed
Add multi index for data_parts storage. And fixed bugs. [#CLICKHOUSE-3452]
...
Fixed handling of obsolete parts.
Fixed conflict resolution between simultaneous PreCommitted covering parts.
Fixed memory leak caused by ordinary MergeTree parts stucked in Deleting state.
Added hidden _state column into system.parts.
2017-11-29 14:25:11 +03:00
proller
0d3c4f4d67
Core/FieldVisitors.h -> Common/FieldVisitors.h
...
Core/Progress.h -> IO/Progress.h
tests: sudo --non-interactive
Fix freebsd link
2017-11-24 16:55:31 +03:00
Alexey Milovidov
f6f77f5f0b
Fixed linking [#CLICKHOUSE-2].
2017-11-20 08:22:07 +03:00
Alexey Milovidov
6c8b5e29d8
Additions after merge [#CLICKHOUSE-2].
2017-11-20 00:17:58 +03:00
Alexey Milovidov
9b92c0fecd
BackgroundSchedulePool: miscellaneous [#CLICKHOUSE-2].
2017-11-19 03:22:13 +03:00
Silviu Caragea
dfecd0b2f2
Merge with master
2017-11-18 10:16:06 +02:00
Alexey Milovidov
729b114c15
Removed unsupported functionality [#CLICKHOUSE-2].
2017-11-18 05:34:00 +03:00
Alexey Milovidov
076e0ae014
More clear [#CLICKHOUSE-2].
2017-11-18 00:20:28 +03:00
Alexey Milovidov
7c9ba15ba1
Style [#CLICKHOUSE-2].
2017-11-17 23:42:03 +03:00
Silviu Caragea
64619c614f
Basic idea is :
...
- We add BackgroundSchedulePool which can execute a function at a specific point in time. Basically all tasks are added in a queue and precessed by worker threads.
- The most important difference between this and BackgroundProcessingPool is that we have the guarantee that the same function is not executed from many workers in the same time
- Each of the following classes instead starting a thread will register a task in BackgroundSchedulePool and when they need to run will call schedule or scheduleAfter(duration) functions
This commit is moving all threads created by ReplicatedMergeTree to BackgroundSchedulePool tasks
NOTE: I did a minimum number of changes to be much simple to review the code
2017-11-17 10:58:35 +02:00
Alexey Zatelepin
99b2d135df
clear deduplication blocks when doing DROP PARTITION [#CLICKHOUSE-3208]
2017-11-15 23:13:10 +03:00
Alexey Milovidov
4e04c4da5f
Attempt to change ext::shared_ptr_helper (incomplete) [#CLICKHOUSE-2].
2017-11-04 06:56:31 +03:00
KochetovNicolai
bfb4e8f3fe
zookeeper chroot fix ( #1430 )
...
* added root and identity section to zookeeper configuration [#CLICKHOUSE-3031]
* added root and identity section to zookeeper configuration [#CLICKHOUSE-3031]
* moved root section in zookeeper configuration [#CLICKHOUSE-3031]
* fixed root section in zookeeper configuration [#CLICKHOUSE-3031]
* added tests [#CLICKHOUSE-3031]
* replaced tabs to spaces in config files [#CLICKHOUSE-3031]
* fix build [#CLICKHOUSE-3031]
* added chech that root starts with '/' in zookeeper config file [#CLICKHOUSE-3031]
* added comments [#CLICKHOUSE-3031]
* Check if zookeeper chroot exists before start. Remove tailing slash form chroot. [#CLICKHOUSE-3400]
* added starting '/' to zookeeper path in ReplicatedMergeTree
2017-11-02 01:59:27 +03:00
alexey-milovidov
3b99b723d7
Merge pull request #1245 from yandex/CLICKHOUSE-3178
...
Atomic parts addition and removing
2017-10-27 23:03:33 +03:00
Vitaliy Lyudvichenko
a0d06c5ee0
Don't deduplicate a block inserted from MV if the source block is Ok. [#CLICKHOUSE-3381]
2017-10-26 17:58:00 +03:00
Vitaliy Lyudvichenko
397b8bcf92
Smarter parts deletion. [#CLICKHOUSE-3178]
2017-10-26 17:16:06 +03:00
Vitaliy Lyudvichenko
b5c7eaa80e
Force data parts cleanup before repeat fetch. [#CLICKHOUSE-3178]
2017-10-26 17:16:06 +03:00
Vitaliy Lyudvichenko
71833a7f0f
More correct procedure of parts deleting. [#CLICKHOUSE-3178]
2017-10-26 17:16:06 +03:00
Vitaliy Lyudvichenko
db3a67a421
Add clearer RangeFiltered implementation. [#CLICKHOUSE-3178]
2017-10-26 17:16:06 +03:00
Vitaliy Lyudvichenko
e2a12d1088
Introduced logic for MergeTree parts states. [#CLICKHOUSE-3178]
2017-10-26 17:16:06 +03:00
Alexey Milovidov
65982a1642
Fixed exception messages [#CLICKHOUSE-2].
2017-10-25 22:17:37 +03:00
Alexey Zatelepin
c61d4106e8
allow loading all MergeTree settings from a query [#CLICKHOUSE-3000]
2017-10-20 21:28:45 +03:00
Alexey Zatelepin
a7c85e0c89
save index_granularity to ATTACH query if it wasn't set explicitly [#CLICKHOUSE-3000]
2017-10-20 21:28:44 +03:00
Alexey Zatelepin
e51f95872f
fix backward compatibility for new part format: [#CLICKHOUSE-3000]
...
* do not write out partition and minmax index files for old MergeTreeData format
* download the part from another replica if there are missing or extra files
2017-10-19 21:24:30 +03:00
Alexey Milovidov
d3d976a6fb
Misc [#CLICKHOUSE-2].
2017-10-12 23:45:55 +03:00
proller
b993667156
Try fix strange terminate ( #1329 )
...
* Tests: External: rename --use_http => --no_http and fix
* Try fix strange terminate
2017-10-12 23:42:58 +03:00
Vitaliy Lyudvichenko
62ea1133dd
Disable part sendings and fetches before ALTER. [#CLICKHOUSE-3343]
2017-10-12 23:26:54 +03:00
Vitaliy Lyudvichenko
cabf6d7799
Add better logging if OPTIMIZE cannot be executed. [#CLICKHOUSE-2]
2017-10-12 22:28:24 +03:00
Alexey Zatelepin
728a23f60e
allow creation of custom-partitioned tables with experimental syntax [#CLICKHOUSE-3000]
2017-09-26 20:40:02 +03:00
Alexey Zatelepin
a10ba0cd91
serialize partition key in Replicated tables metadata [#CLICKHOUSE-3000]
2017-09-26 20:40:02 +03:00
Alexey Zatelepin
909b46de0f
add MergeTreeDataFormatVersion constant [#CLICKHOUSE-3000]
2017-09-26 20:40:02 +03:00
Alexey Zatelepin
68cb592277
allow arbitrary partition literals in ALTER PARTITION queries [#CLICKHOUSE-3000]
2017-09-26 20:40:02 +03:00
Alexey Zatelepin
2f80c150db
save MergeTree data format version and parse part names depending on it [#CLICKHOUSE-3000]
2017-09-26 20:40:02 +03:00
Alexey Milovidov
94b975e7f9
Removed function deprecated in C++ [#CLICKHOUSE-3174].
2017-09-09 02:31:18 +03:00
Alexey Milovidov
8b313ab99e
Fixed warnings found by cppcheck [#CLICKHOUSE-3300].
2017-09-08 00:04:48 +03:00
Alexey Milovidov
0dcee75180
Supporessed useless false positive errors found by cppcheck [#CLICKHOUSE-3300].
2017-09-07 22:13:37 +03:00
Vitaliy Lyudvichenko
d55d5eca46
Remove extra Context reference and add required ones. Part 2. [#CLICKHOUSE-3117]
2017-09-06 04:18:07 +03:00
Vitaliy Lyudvichenko
566160c048
Use FIFO lock in IStorage. [#CLICKHOUSE-3246]
2017-09-01 20:45:05 +03:00
Alexey Zatelepin
d2847f22af
PR fixes [#CLICKHOUSE-3000]
2017-08-31 23:32:03 +03:00
Alexey Zatelepin
14ba0a9acf
rename partition -> partition_id for clarity [#CLICKHOUSE-3000]
2017-08-31 23:32:03 +03:00
Alexey Zatelepin
645e4b7a53
generalize MinMax index [#CLICKHOUSE-3000]
2017-08-31 23:32:03 +03:00
Alexey Zatelepin
5471979f46
generic partitioning of inserted blocks [#CLICKHOUSE-3000]
2017-08-31 23:32:03 +03:00
Alexey Zatelepin
99969ab77a
Introduce MergeTreePartitionIndex. Isolate its loading code. [#CLICKHOUSE-3000]
2017-08-31 23:32:03 +03:00
Alexey Milovidov
495daffa16
Miscellaneous [#CLICKHOUSE-2].
2017-08-30 21:13:32 +03:00
Vitaliy Lyudvichenko
5916d823ef
Fix deadlock in case of using Buffer table with CLEAR COLUMN IN PARTITION. [#CLICKHOUSE-3238]
2017-08-30 17:34:40 +03:00
Vitaliy Lyudvichenko
84660029ec
Don't leave trash part nodes in ZK in case of deleting non-existing duplicated parts. [#CLICKHOUSE-3252]
2017-08-29 20:49:37 +03:00
Vitaliy Lyudvichenko
fd63277b15
Unified spelling. [#CLICKHOUSE-2]
2017-08-28 18:27:25 +03:00
Vitaliy Lyudvichenko
37820f7f64
Don't leave trash parts in ZK in case of temporary errors. [#CLICKHOUSE-3252]
2017-08-28 18:27:25 +03:00
Alexey Zatelepin
f25f0cd759
reduce number of places where partitioning by month is assumed [#CLICKHOUSE-3000]
2017-08-15 22:30:52 +03:00
Vitaliy Lyudvichenko
8a14cc49ff
asyncMulti() -> multi(). [#CLICKHOUSE-3221]
2017-08-15 19:18:02 +03:00
Alexey Zatelepin
3f3b7f61e2
always send changed settings for remote connections [#CLICKHOUSE-3151]
2017-08-13 12:17:09 +03:00
Vitaliy Lyudvichenko
351a0905d5
Implemented fast block and parts cleaning. [#CLICKHOUSE-3207]
2017-08-10 23:09:05 +03:00
Vitaliy Lyudvichenko
10c14cfccb
Small code enhancements according to clang-tidy. [#CLICKHOUSE-2931]
2017-08-04 17:35:36 +03:00
Alexey Milovidov
6578fb2645
Fixed error [#CLICKHOUSE-3191].
2017-08-01 00:39:24 +03:00
Alexey Milovidov
8fb452e99a
Fixed wrong log message when removing temporary part after merge, that is not byte-identical than same part on replicas; added profile event; added fail-safe logic [#MTRSADMIN-3175].
2017-07-29 00:25:24 +03:00
Vitaliy Lyudvichenko
bc9ee54985
Don't throw exception in case of clearing empty part. [#CLICKHOUSE-2896]
2017-07-24 18:04:17 +03:00
alexey-milovidov
d4d266fa18
ColumnConst unification ( #1011 )
...
* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].
* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].
* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].
* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].
* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].
* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].
* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].
* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].
* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].
* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].
* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].
* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].
* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].
* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].
* Fixed error in ColumnArray::replicateGeneric [#CLICKHOUSE-3150].
* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].
* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].
* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].
* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].
* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].
* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].
* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].
* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].
* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].
* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].
* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].
* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].
* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].
2017-07-21 09:35:58 +03:00
Alexey Milovidov
9406b6a5aa
Get rid of ASTSet, because it is not really an AST and it lead to bugs [#CLICKHOUSE-3148].
2017-07-15 06:48:36 +03:00
Nikolai Kochetov
283810e8ab
merged with master
2017-07-13 23:59:34 +03:00
Nikolai Kochetov
75be0b81d8
removed typeid_cast from IColumn.h
2017-07-13 23:58:19 +03:00
Alexey Milovidov
5b5795633e
Using lexer (incomplete) [#CLICKHOUSE-2].
2017-07-13 08:59:01 +03:00
Alexey Zatelepin
bdc529645e
fix cyclical yielding of leadership at startup [#CLICKHOUSE-3074]
2017-07-12 20:09:44 +03:00
Vitaliy Lyudvichenko
97f7b46ebd
Fixed CLEAR COLUMN test in case of deduplication rollback. [#CLICKHOUSE-2]
2017-06-26 19:34:38 +03:00
Vitaliy Lyudvichenko
9c0da6e698
Addition to previous commit. Fix error code, rename test tables. [#CLICKHOUSE-2]
2017-06-26 17:35:26 +03:00
Vitaliy Lyudvichenko
49cbc33a6b
Fix ZK exception in case of concurrent INSERTs in new partition. [#CLICKHOUSE-2]
2017-06-26 17:21:25 +03:00
Vitaliy Lyudvichenko
b53d6976b3
Fix build after merge. [#CLICKHOUSE-2]
2017-06-26 15:47:15 +03:00
Alexey Milovidov
a48a21f593
Merge branch 'master' into CLICKHOUSE-2896
2017-06-26 11:54:58 +03:00
Alexey Milovidov
275a1d4651
Improvement [#CLICKHOUSE-3091].
2017-06-25 05:48:38 +03:00
Alexey Milovidov
516b095423
Fixed error in prev. revision [#CLICKHOUSE-3091].
2017-06-25 04:54:25 +03:00
Alexey Milovidov
8228acd3ea
Removed obsolete code that was relevant only in extremely rare case of transforming CollapsingMergeTree to ReplicatedCollapsingMergeTree through ATTACHing data parts and while some considerations about data order in table is important [#CLICKHOUSE-3091].
2017-06-25 03:51:51 +03:00
Alexey Milovidov
3c660fd65b
Fixed translation errors [#CLICKHOUSE-3091].
2017-06-25 03:17:08 +03:00
Alexey Milovidov
28e9e06992
Preparation: locking block number only after write to filesystem; removed obsolete code [#CLICKHOUSE-3091].
2017-06-25 03:01:10 +03:00
Alexey Milovidov
c7f35997f5
Removed unused variable [#CLICKHOUSE-2].
2017-06-25 01:36:43 +03:00
proller
4db8d09de9
Reorganize includes. part 1 ( #921 )
...
* Make libunwind optional. Allow use custom libcctz
* fix
* Fix
* fix
* Update BaseDaemon.cpp
* Update CMakeLists.txt
* Reorganize includes. part 1
* Update dbms_include.cmake
* Reorganize includes. part 2
* Reorganize includes. part 3
* dbms/src/Common/ThreadPool -> libs/libcommon
* Reorganize includes. part 4
* Fix print_include_directories
* Update thread_creation_latency.cpp
* Update StringRef.h
2017-06-23 23:22:35 +03:00
Alexey Milovidov
f409cad0ac
Miscellaneous [#CLICKHOUSE-3091].
2017-06-23 09:18:00 +03:00
Alexey Milovidov
7d732343eb
Fixed error in prev. revision [#CLICKHOUSE-3091].
2017-06-22 22:52:06 +03:00
Alexey Milovidov
22e7070fb0
Merge branch 'master' of github.com:yandex/ClickHouse
2017-06-22 19:07:57 +03:00