Commit Graph

296 Commits

Author SHA1 Message Date
Alexander Tokmakov
c6717e0d3f refactor attachPartition 2019-07-30 22:11:15 +03:00
Alexander Tokmakov
2f33df1b2e rename all parts before attaching 2019-07-30 20:24:40 +03:00
Igr Mineev
31c36efa25 Merge branch 'master' of github.com:yandex/ClickHouse into feature_multiple_disks 2019-07-30 19:13:58 +03:00
Alexander Tokmakov
3c15b87003 Merge branch 'master' into alter_table_drop_detached_part 2019-07-29 14:29:23 +03:00
Alexander Tokmakov
0fc47fbbe4 fixes 2019-07-29 13:27:37 +03:00
alesapin
d9a2ac123a Fix small typo and merge 2019-07-29 12:17:37 +03:00
Guillaume Tassery
372b9026d4 cosmetic changes 2019-07-26 12:09:36 +02:00
Guillaume Tassery
f52fed5cea Let the user able to move a partition to an another table 2019-07-26 10:42:17 +02:00
Alexander Tokmakov
1805ab5736 attach 2019-07-25 13:46:07 +03:00
alesapin
8a00ce5ff1 Intermediate step 2019-07-24 15:56:39 +03:00
Igr
3e017ec69c
Merge branch 'master' into feature_multiple_disks 2019-07-23 16:38:20 +03:00
Igr Mineev
e967c23adb Add Space interface for reservation. Add move to volume command 2019-07-23 16:34:17 +03:00
CurtizJ
b9dc472071 Merge remote-tracking branch 'upstream/master' into order-by-efficient 2019-07-19 01:09:59 +03:00
Igr Mineev
52bca488dc Add alter move part command 2019-07-18 18:19:03 +03:00
Amos Bird
9fa955403b Optimize count()
Choose the smallest column to count if possible.
2019-07-17 02:29:17 +08:00
Igr Mineev
21e0bec3d0 Merge remote-tracking branch 'upstream/master' into feature_multiple_disks 2019-07-16 13:45:07 +03:00
Gleb Novikov
c1d91222d5 Ranamed setPrimaryKeyIndicesAndColumns to setProperties 2019-07-13 13:42:52 +03:00
CurtizJ
410b62f7fa Merge branch 'clickhouse-4013' of https://github.com/anrodigina/ClickHouse into order-by-efficient 2019-07-05 13:48:04 +03:00
Gleb Novikov
71c2ca0fe9 Merge branch 'master' into table-constraints 2019-07-04 22:40:00 +03:00
alesapin
dd37370aab Fix test and add one 2019-06-19 19:16:13 +03:00
alesapin
fefce49c8f Mixed parts by setting 2019-06-19 17:46:06 +03:00
alesapin
b544b62e44 A log of bugs 2019-06-19 13:07:56 +03:00
alesapin
fb22200890 Fix name hidding and stupid bug 2019-06-17 18:35:53 +03:00
alesapin
3a4dcfbacd Fix inconsistency in zookeeper metadata and disable adaptive granularity for old format 2019-06-17 13:34:25 +03:00
alesapin
22026e0333 Enable adaptive index granularity by default 2019-06-14 18:39:21 +03:00
Igr Mineev
842c72bb6d Merge branch 'master' into feature_multiple_disks 2019-06-09 15:38:54 +03:00
Igr Mineev
0eba56c4a7 Added move command. 2019-06-07 22:16:42 +03:00
Gleb Novikov
e7293486bd Added ALTER TABLE support to constraints (ADD CONSTRAINT, DROP CONSTRAINT) 2019-06-02 17:41:12 +03:00
Gleb Novikov
337897c4b8 Merge branch 'master' into table-constraints 2019-05-26 12:36:02 +03:00
Igor Mineev
15ee6207ab Chg 'Schema' to 'StoragePolicy'. Add system.disks. Add system.storage_politics. Add disk to system.parts. Add policy to system.tables. 2019-05-24 22:03:07 +03:00
Igor Mineev
4f61c9d30f Const Schema fix 2019-05-22 22:20:10 +03:00
Alexander Kuzmenkov
43655a8db7 Add system.detached_parts table.
https://github.com/yandex/ClickHouse/issues/5164
2019-05-21 15:06:36 +03:00
Anastasiya Rodigina
6132b57beb Fix style issues 2019-05-18 16:22:49 +03:00
anrodigina
d7ee2cb907
Merge branch 'master' into clickhouse-4013 2019-05-18 15:34:23 +03:00
Gleb Novikov
ce93896608 Added constraints description to MergeTree and related storages 2019-05-18 11:05:52 +03:00
Igor Mineev
6596c3f6e0 DataPartsExchange new protocol with data size. StorageReplicatedMergeTree::fetchPartition multipath fix. Choosing on mutate any disk to write mutation file. 2019-05-12 17:57:23 +03:00
Igor Mineev
b4b000bf84 Merge branch 'master' into feature_multiple_disks 2019-05-11 21:50:29 +03:00
ivan-kush
ab6558b8b4 add spaces amp 2019-05-10 17:34:02 +03:00
Ivan Kush
2a356791ab Merge branch 'master' into feat/parallel-alter-modify 2019-05-07 23:53:14 +03:00
alexey-milovidov
63b3d7dbdb
Update MergeTreeData.h 2019-05-06 01:13:07 +03:00
alexey-milovidov
c04c614bc0
Merge branch 'master' into clickhouse-4013 2019-05-05 23:22:45 +03:00
Alexey Milovidov
b2443f6aa6 Removed huge chunk of bad code 2019-05-04 06:45:58 +03:00
Alexey Milovidov
24dc2c858a Removed huge chunk of bad code 2019-05-03 21:07:59 +03:00
Alexey Milovidov
a6ca9f266f Removed huge chunk of bad code 2019-05-03 05:00:57 +03:00
Alexey Milovidov
41a32d8693 Removed bad code 2019-05-03 03:27:11 +03:00
ivan-kush
9ce742e040 Merge remote-tracking branch 'upstream/master' into feat/parallel-alter-modify 2019-05-03 00:25:56 +03:00
Alexey Milovidov
6a271adad3 Added max_parts_in_total threshold to MergeTree tables 2019-05-02 17:48:54 +03:00
ivan-kush
3ecb66d965 debug stress address 2019-05-02 16:42:09 +03:00
ivan-kush
4a3a1fc6d9 return transactions cleaning 2019-05-01 15:43:43 +03:00
Ivan Kushnarenko
e38f6e08d8 debug tsan refactor trash 2019-05-01 13:26:49 +03:00
Igor Mineev
8edcae5530 Merge remote-tracking branch 'upstream/master' into feature_multiple_disks 2019-04-24 15:35:08 +03:00
Igor Mineev
79abe85328 New data path: disk_path + db_name + table_name. Removed default path from MergeTree Constructor 2019-04-21 21:38:44 +03:00
Anastasiya Rodigina
0c1735f1b1 ReverseBlockInputStream, optimization of group by, tests 2019-04-18 00:20:51 +03:00
alesapin
faeb95cfd7
Merge branch 'master' into adaptive_index_granularity 2019-04-15 19:36:36 +03:00
Anton Popov
7f93f9ad2f TTL for columns and tables (#4212)
Add TTL for columns and tables.
2019-04-15 12:30:45 +03:00
Igor Mineev
563ab2a5ec Merge remote-tracking branch 'upstream/master' into feature_multiple_disks 2019-04-05 23:07:24 +03:00
Igor Mineev
3a38b16da1 Add rename MergeTreeData function. Add shared_ptr to disks in Volumes. 2019-04-05 22:45:59 +03:00
Igor Mineev
968b7286e2 Stylefixes. 2019-04-05 20:37:27 +03:00
Igor Mineev
0b1a33c722 Format file moved to default. Add default Storage configuration 2019-04-05 15:30:26 +03:00
Igor Mineev
7824aa528f Add Schemes to config. Parse scheme as default on MergeTreeData init 2019-04-04 20:19:11 +03:00
Igor Mineev
341e3f1c5b Add space reservation for each new MargeTreeDataPart. 2019-04-03 15:52:09 +03:00
Igor Mineev
8047d16a96 Add Path chooser. Add DiskMonitort multiple disks support 2019-04-01 21:41:19 +03:00
alesapin
7b67200258 Merge remote-tracking branch 'origin' into adaptive_index_granularity 2019-04-01 11:54:19 +03:00
Alexey Milovidov
8d3ef5df19 Fixed data race in clearOldTemporaryDirectories 2019-03-29 12:28:16 +03:00
Alexey Milovidov
34c0682353 Removed useless method 2019-03-29 11:37:05 +03:00
Alexey Milovidov
350a0fe129 Miscellaneous 2019-03-28 22:58:41 +03:00
alesapin
5528edea93 Fix build warnings 2019-03-27 19:23:38 +03:00
alesapin
aba51a11ba Large refactoring (only compilable) 2019-03-25 16:55:24 +03:00
alesapin
673a72eac6 Merge remote-tracking branch 'origin' into adaptive_index_granularity 2019-03-25 11:41:38 +03:00
Alexey Milovidov
5a7237cc13 Miscellaneous 2019-03-23 04:27:08 +03:00
alesapin
ce03636f05 Merge master 2019-03-18 15:02:33 +03:00
Nikita Vasilev
5279c7ff81 setting 2019-02-12 20:54:52 +03:00
Nikita Vasilev
258367c1dd ASTs 2019-02-06 10:51:35 +03:00
Nikita Vasilev
69daa33462 removed unused data 2019-01-30 13:58:10 +03:00
Nikita Vasilev
160c8c0807 fixed bug with duplicate column 2019-01-30 13:18:59 +03:00
Nikita Vasilev
f0a0dbbf5a fixed bug with replMT indices storing in zk 2019-01-21 17:47:47 +03:00
Nikita Vasilev
c32d1983a0 go to indicesDescr 2019-01-21 12:47:35 +03:00
Nikita Vasilev
977b1e20d0 moved indices 2019-01-20 23:03:17 +03:00
Nikita Vasilev
3f5447f471 go to IndicesDescr 2019-01-20 21:43:49 +03:00
Nikita Vasilev
4a19523698 Merge remote-tracking branch 'upstream/master' into nikvas0/index 2019-01-20 10:14:06 +03:00
Nikita Vasilev
0e6f5b4356 fix 2019-01-17 21:22:45 +03:00
Nikita Vasilev
c4a725a496 indices 2019-01-17 15:11:36 +03:00
Nikita Vasilev
2de2b6f32e fixed alter 2019-01-16 19:53:38 +03:00
Alexey Zatelepin
0f8e5f8522 Merge branch 'master' into zk-columns-compact-storage
Conflicts:
	dbms/src/Storages/StorageReplicatedMergeTree.cpp
2019-01-16 17:45:16 +03:00
Nikita Vasilev
8c35ccdd1d fix alters 2019-01-16 15:29:26 +03:00
Nikita Vasilev
d310d1a5ec fixed setSkipIndexes 2019-01-10 19:51:49 +03:00
Nikita Vasilev
701627ec60 fixed includes 2019-01-10 16:50:41 +03:00
Nikita Vasilev
d1e2ab85e3 asts -> exprlist 2019-01-08 14:04:25 +03:00
Nikita Vasilev
e95376e550 Merge branch 'master' into nikvas0/index 2019-01-06 16:21:00 +03:00
Nikita Vasilev
82cc39d441 added setSkipIndexes 2019-01-05 15:35:13 +03:00
Alexey Milovidov
1d4701b0bc Added some warnings from clang's -Weverything 2019-01-04 15:10:00 +03:00
Alexey Milovidov
e33e5150b7 Miscellaneous [#CLICKHOUSE-2] 2019-01-02 09:44:36 +03:00
Alexey Zatelepin
9bd09b44ed introduce ReplicatedMergeTreePartHeader [#CLICKHOUSE-4024]
Conflicts:
	dbms/src/Common/ErrorCodes.cpp
2018-12-29 17:11:16 +03:00
Nikita Vasilev
c89df916ab ptrs 2018-12-26 20:34:44 +03:00
Nikita Vasilev
04a8ea85af added index interface and factory 2018-12-26 15:19:24 +03:00
Alexey Milovidov
4c0b95ac02 Less dependencies [#CLICKHOUSE-2] 2018-12-26 02:11:36 +03:00
Alexey Zatelepin
b7afe46dfd remove ALTER MODIFY PRIMARY KEY command [#CLICKHOUSE-4214] 2018-12-20 21:57:20 +03:00
Emmanuel Donin de Rosière
982cd749be Better management of DateTime partitionning #3786
Adding min_time and max_date in system.parts table
2018-12-10 11:20:19 +01:00
alesapin
bf338b957f Merge master 2018-12-03 12:10:06 +03:00
alesapin
497a09c41b Return to adaptive granularity 2018-11-29 16:12:20 +03:00
Alexey Zatelepin
e085bc9816 adapt code of #3609 [#CLICKHOUSE-3859] 2018-11-27 20:07:10 +03:00
Alexey Zatelepin
d3d9242462 Merge branch 'master' into alter-order-by
Conflicts:
	dbms/src/Parsers/ASTAlterQuery.h
	dbms/src/Parsers/ParserAlterQuery.cpp
	dbms/src/Storages/AlterCommands.cpp
	dbms/src/Storages/AlterCommands.h
2018-11-26 16:01:48 +03:00
Alexey Milovidov
dbbacf4f87 Added more style checks [#CLICKHOUSE-3] 2018-11-23 21:52:00 +03:00
Ivan Lezhankin
1e070a7c51 Move FREEZE_ALL command source code inside |PartitionCommand| 2018-11-20 20:13:59 +03:00
alesapin
6123993ec0 Load granularity to memroy 2018-11-15 17:06:54 +03:00
Alexey Zatelepin
b071547b2d add checks for ALTER MODIFY ORDER BY [#CLICKHOUSE-3859] 2018-11-14 20:12:32 +03:00
alesapin
b2b29815dd Change fixed index granularity to adaptive 2018-11-14 10:49:20 +03:00
Alexey Zatelepin
29b34365e6 Merge branch 'master' into alter-order-by
Conflicts:
	dbms/src/Interpreters/ExpressionAnalyzer.cpp
	dbms/src/Storages/MergeTree/MergeTreeData.cpp
	dbms/src/Storages/MergeTree/MergeTreeDataSelectExecutor.cpp
2018-11-13 16:27:19 +03:00
Alexey Zatelepin
2cc0aae82c rearrange AST members in MergeTreeData [#CLICKHOUSE-3859] 2018-11-13 16:15:17 +03:00
Alexey Zatelepin
7167bfd7b3 fix prewhere with FINAL for VersionedCollapsingMergeTree [#CLICKHOUSE-3859] 2018-11-13 16:15:12 +03:00
Alexey Zatelepin
8b85822192 store original key ASTs, and not extracted ASTExpressionLists [#CLICKHOUSE-3859] 2018-11-13 16:14:44 +03:00
Alexey Zatelepin
0c0c4fd28d add sorting_and_primary_keys_independent [#CLICKHOUSE-3859] 2018-11-13 16:13:19 +03:00
Alexey Zatelepin
c4bf38ef07 comments [#CLICKHOUSE-3859] 2018-11-13 16:13:06 +03:00
Alexey Zatelepin
f4c26e78d2 renames [#CLICKHOUSE-3859] 2018-11-13 16:13:01 +03:00
Alexey Zatelepin
0fa86162d9 separate sorting key and primary key, allow setting primary key separately [#CLICKHOUSE-3859] 2018-11-13 16:12:55 +03:00
Alexey Zatelepin
9a92c0a55c sort key is distinct from primary key [#CLICKHOUSE-3859] 2018-11-13 16:12:49 +03:00
alesapin
6f5fef5344 CLICKHOUSE-3971: Write data with adaptive index granularity 2018-11-12 19:14:37 +03:00
Ivan Lezhankin
5d3da110a8 Refactor |freezePartition()| and |freezeAll()| 2018-11-12 15:26:15 +03:00
Ivan Lezhankin
935615a647 Reimplement FREEZE command. 2018-11-12 15:26:14 +03:00
Ivan Lezhankin
03e6a29eb2 Implement 'ALTER … FREEZE' command.
Need some tests!
2018-11-12 15:26:14 +03:00
Alexey Milovidov
99a64d3b92 Presumably better #2278 2018-10-17 06:13:00 +03:00
Alexey Zatelepin
c2c12f22f7 try to immediately delete the part after checksums mismatch [#CLICKHOUSE-2837] 2018-09-21 19:07:21 +03:00
Alexey Zatelepin
6da58aedd8 don't fetch mutated part if we have local part with the same checksums [#CLICKHOUSE-3937] 2018-09-11 19:41:01 +03:00
VadimPE
10ab9587a2 CLICKHOUSE-3839 add virtual column _partition_id 2018-09-10 12:53:13 +03:00
Alexey Zatelepin
a0730d7696 old mutations cleaner for non-replicated MergeTree engines [#CLICKHOUSE-3828] 2018-08-03 22:21:44 +03:00
Alexey Milovidov
b5b48d59fa Compatibility with obsolete version of Poco for alternative build variants #2759 2018-07-30 23:21:45 +03:00
Alexey Milovidov
e86f73e465 Fixed half of bad code #2571 2018-07-04 20:28:52 +03:00
Alexey Milovidov
f5adbceed2 Non significant changes according to clang's -Weverything, part 3 [#CLICKHOUSE-2] 2018-06-03 23:39:06 +03:00
Alexey Zatelepin
f6c72f8e56 Merge branch 'master' into replicated-mergetree-mutations
Conflicts:
	dbms/src/Common/ActionBlocker.h
	dbms/src/Storages/MergeTree/ActiveDataPartSet.cpp
	dbms/src/Storages/MergeTree/ActiveDataPartSet.h
	dbms/src/Storages/MergeTree/MergeTreeData.h
	dbms/src/Storages/MergeTree/MergeTreeDataPart.cpp
	dbms/src/Storages/MergeTree/ReplicatedMergeTreeAlterThread.cpp
	dbms/src/Storages/MergeTree/ReplicatedMergeTreeLogEntry.cpp
	dbms/src/Storages/MergeTree/ReplicatedMergeTreeLogEntry.h
	dbms/src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp
	dbms/src/Storages/MergeTree/ReplicatedMergeTreeQueue.h
	dbms/src/Storages/MergeTree/ReplicatedMergeTreeRestartingThread.cpp
	dbms/src/Storages/StorageMergeTree.cpp
	dbms/src/Storages/StorageReplicatedMergeTree.cpp
	dbms/src/Storages/StorageReplicatedMergeTree.h
2018-05-28 18:37:30 +03:00
Alexey Zatelepin
1147e052e8 select and write part mutations to queue [#CLICKHOUSE-3747] 2018-05-25 23:13:00 +03:00
Alexey Zatelepin
9a32c76e76 parts versioning [#CLICKHOUSE-3687] 2018-05-25 23:04:07 +03:00
Vitaliy Lyudvichenko
87ad1df259 More atomic insert to MATERIALIZED VIEW. [#CLICKHOUSE-3743] 2018-05-23 19:18:04 +03:00
Vitaliy Lyudvichenko
c888903488
Add query ALTER TABLE t1 [REPLACE|ATTACH] PARTITION FROM t2
* Add ATTACH PARTITION FROM table for MergeTree. [#CLICKHOUSE-3546]

* Implemented replicated case on non-leader replica. [#CLICKHOUSE-3546]

* Disable merges in the dropping range. [#CLICKHOUSE-3546]

* DROP PARTITION is atomic and simpler now. [#CLICKHOUSE-3546]

* Implemented more SYSTEM queries. [#CLICKHOUSE-2931] [#CLICKHOUSE-3546]

SYSTEM queries:
RESTART REPLICAS
SYNC REPLICA db.name
STOP MERGES [db.name]
START MERGES [db.name]
STOP FETCHES [db.name]
START FETCHES [db.name]
STOP REPLICATED SENDS [db.name]
START REPLICATED SENDS [db.name]
STOP REPLICATION QUEUES [db.name]
START REPLICATION QUEUES [db.name]

* Fixed a bunch of bugs in REPLACE PARTITION. [#CLICKHOUSE-3546]

* Add tests for REPLACE PARTITION and SYSTEM. [#CLICKHOUSE-3546]

* Add system.part_log logging. [#CLICKHOUSE-3546]

* Fixed long wait in SYNC REPLICA. [#CLICKHOUSE-3546]

* Add requested changes. [#CLICKHOUSE-3546]

Fixed clickhouse-client bad return code.

* Add requested chenges. [#CLICKHOUSE-3546]

* Add requested chenges. [#CLICKHOUSE-3546]
2018-05-21 16:49:54 +03:00
Alexey Milovidov
d71b3a95ef Renamed PK to Key where it's appropriate #2246 2018-04-20 03:20:50 +03:00
Alexey Milovidov
c0978919e3 Fixed error with partition key IN, part 1 #2246 2018-04-19 23:34:02 +03:00
Nikolai Kochetov
b96039bafe added check for partition column in mayBenefitFromIndexForIn #2170 2018-04-05 00:03:16 +03:00
Alex Zatelepin
c5b04f7f17 Add info about total uncompressed part size to system tables (#2052)
* column size calculating routines now take into account multiple streams [#CLICKHOUSE-2831]

* rename for clarity

* use more appropriate method for calculating table size

* add info about total uncompressed part size to system.parts and system.parts_columns [#CLICKHOUSE-2831]

* rename columns for clarity [#CLICKHOUSE-2831]

* count shared substreams only once [#CLICKHOUSE-2831]
2018-03-26 17:18:04 +03:00
Alexey Milovidov
02c67e2b13 Fixed error [#CLICKHOUSE-2] 2018-03-16 09:51:37 +03:00
Alexey Zatelepin
5906c9a988 rename for clarity 2018-03-13 18:00:28 +03:00
Alexey Zatelepin
fdffbe53ac encapsulate ColumnsDeclaration [#CLICKHOUSE-3241] 2018-03-13 17:41:25 +03:00
Alexey Zatelepin
71642b0880 use ColumnsDescription everywhere [#CLICKHOUSE-3241] 2018-03-12 22:07:48 +03:00
Alexey Milovidov
3f8ef38b41 Fixed wording #245 2018-03-10 02:23:15 +03:00
alexey-milovidov
82fa3883f8
Merge pull request #1930 from yandex/nonblocking-precommitted
Allow PreCommitted parts to contain each other
2018-02-22 23:35:16 +03:00
Alexey Zatelepin
610f70fbeb forbid non-deterministic functions in primary and partition keys [#CLICKHOUSE-3455] 2018-02-21 20:54:16 +03:00
Alexey Zatelepin
53b5c5f54c add check for intersecting parts when adding new parts 2018-02-20 16:58:41 +03:00
Alexey Zatelepin
5bc6bd55b1 allow PreCommitted parts to contain each other, clean up code 2018-02-20 16:33:28 +03:00
Alexey Milovidov
d0bfa15525 Removed UnsortedMergeTree, part 2 [#CLICKHOUSE-2] 2018-02-19 20:31:49 +03:00
Nikolai Kochetov
90d67d4e8a added test for moving partition from collapsing to versioned collapsing merge tree [#CLICKHOUSE-3479] 2018-02-13 00:13:50 +03:00
Nikolai Kochetov
64b3a8baa7 added secondary sort expression [#CLICKHOUSE-3479] 2018-02-13 00:13:50 +03:00
Nikolai Kochetov
32bdf7c38c renamed MultiversionMergeTree to VersionedCollapsingMergeTree [#CLICKHOUSE-3479] 2018-02-13 00:13:50 +03:00