thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-10-07 00:50:56 +00:00
Code Issues Packages Projects Releases Wiki Activity
34,229 Commits 8,977 Branches 1,768 Tags 2 GiB
9fe94ca5f7
Commit Graph

753 Commits

Author SHA1 Message Date
alexey-milovidov
88b34999fe
Merge pull request #5247 from zhang2014/fix/rename_data_race 
fix data race in rename query
 2019-05-14 22:07:52 +03:00
Igor Mineev
74466de6d8 Minor fixes. 2019-05-13 23:58:22 +03:00
zhang2014
2206ba464f fix data race in rename query 2019-05-13 15:54:38 +08:00
Igor Mineev
6596c3f6e0 DataPartsExchange new protocol with data size. StorageReplicatedMergeTree::fetchPartition multipath fix. Choosing on mutate any disk to write mutation file. 2019-05-12 17:57:23 +03:00
Igor Mineev
af300c667b Bugfix 2019-05-11 23:44:43 +03:00
Igor Mineev
b4b000bf84 Merge branch 'master' into feature_multiple_disks 2019-05-11 21:50:29 +03:00
ivan-kush
ab6558b8b4 add spaces amp 2019-05-10 17:34:02 +03:00
Nikita Vasilev
ad07dd2e1f fix 2019-05-09 20:04:05 +03:00
Nikita Vasilev
8d2b32fda8 clear index 2019-05-09 17:25:18 +03:00
Ivan Kush
2a356791ab Merge branch 'master' into feat/parallel-alter-modify 2019-05-07 23:53:14 +03:00
ivan-kush
5bde1c69ab last try? 2019-05-07 23:41:15 +03:00
Alexey Milovidov
a6ca9f266f Removed huge chunk of bad code 2019-05-03 05:00:57 +03:00
Alexey Milovidov
0a8e888671 Fixed bad code 2019-05-03 03:26:46 +03:00
ivan-kush
abfb681a04 hz 2019-05-03 00:43:24 +03:00
ivan-kush
d555a165a4 debug stress address 2019-05-02 21:48:38 +03:00
Alexey Milovidov
270a31abb5 Rename method for consistency 2019-05-02 19:07:23 +03:00
ivan-kush
ea82760463 add reset 2019-05-02 18:12:57 +03:00
ivan-kush
591325468b make_unique 2019-05-02 14:54:34 +03:00
ivan-kush
4a3a1fc6d9 return transactions cleaning 2019-05-01 15:43:43 +03:00
Ivan Kushnarenko
e38f6e08d8 debug tsan refactor trash 2019-05-01 13:26:49 +03:00
Ivan Kushnarenko
bdbe97276c remove groups 2019-04-27 23:15:10 +03:00
ivan-kush
26b3910b7d tsan debug 2019-04-27 20:01:30 +03:00
Ivan Kushnarenko
cb51558bdf check build clang7 splitted 2019-04-27 11:24:50 +03:00
ivan-kush
ba61e16093 style 2019-04-26 00:37:57 +03:00
ivan-kush
86af5fc0c2 Merge remote-tracking branch 'upstream/master' into feat/parallel-alter-modify 2019-04-25 22:48:44 +03:00
ivan-kush
af02fe4a96 Merge remote-tracking branch 'origin/feat/parallel-alter-modify' into feat/parallel-alter-modify 2019-04-25 22:33:09 +03:00
ivan-kush
33201fa9a8 rev 2019-04-25 21:57:43 +03:00
Igor Mineev
8edcae5530 Merge remote-tracking branch 'upstream/master' into feature_multiple_disks 2019-04-24 15:35:08 +03:00
Igor Mineev
79abe85328 New data path: disk_path + db_name + table_name. Removed default path from MergeTree Constructor 2019-04-21 21:38:44 +03:00
Anton Popov
7f93f9ad2f TTL for columns and tables (#4212) 
Add TTL for columns and tables.
 2019-04-15 12:30:45 +03:00
Igor Mineev
563ab2a5ec Merge remote-tracking branch 'upstream/master' into feature_multiple_disks 2019-04-05 23:07:24 +03:00
Igor Mineev
3a38b16da1 Add rename MergeTreeData function. Add shared_ptr to disks in Volumes. 2019-04-05 22:45:59 +03:00
Igor Mineev
968b7286e2 Stylefixes. 2019-04-05 20:37:27 +03:00
Igor Mineev
0b1a33c722 Format file moved to default. Add default Storage configuration 2019-04-05 15:30:26 +03:00
Igor Mineev
7824aa528f Add Schemes to config. Parse scheme as default on MergeTreeData init 2019-04-04 20:19:11 +03:00
Igor Mineev
0441480502 Storage getDataPath -> getDataPaths 2019-04-04 16:13:59 +03:00
Igor Mineev
341e3f1c5b Add space reservation for each new MargeTreeDataPart. 2019-04-03 15:52:09 +03:00
Igor Mineev
8047d16a96 Add Path chooser. Add DiskMonitort multiple disks support 2019-04-01 21:41:19 +03:00
alexey-milovidov
d10b256a8a
Merge pull request #4845 from yandex/max_partitions_per_insert_block 
Implement "max_partitions_per_insert_block" setting
 2019-03-29 16:20:46 +03:00
Alexey Milovidov
8cb1619607 Implement "max_partitions_per_insert_block" setting #4700 2019-03-29 12:34:07 +03:00
Alexey Milovidov
8d3ef5df19 Fixed data race in clearOldTemporaryDirectories 2019-03-29 12:28:16 +03:00
Ivan Lezhankin
61954d855d More reverts 2019-03-15 20:09:14 +03:00
alexey-milovidov
fc3cb57881
Update StorageMergeTree.cpp 2019-03-12 16:26:26 +03:00
alexey-milovidov
72f785bd1d
Update StorageMergeTree.cpp 2019-03-12 16:24:56 +03:00
alexey-milovidov
6daee801fa
Update StorageMergeTree.cpp 2019-03-12 16:24:03 +03:00
ivan-kush
3f01882310 format2 2019-03-12 08:56:02 +03:00
ivan-kush
6acb39032f format 2019-03-11 23:17:50 +03:00
Ivan Lezhankin
d1ead22073 Rename As<…> → as<…> 2019-03-11 16:22:51 +03:00
Ivan Lezhankin
1e9f7847dc Merge remote-tracking branch 'upstream/master' into iast-as 2019-03-11 16:11:22 +03:00
Ivan Lezhankin
a7e838e328 Replace all other casts to method IAST::As<Derived> 
Also remove some redundant methods like IsIdentifier() et. al.
 2019-03-11 15:49:39 +03:00
ivan-kush
c3182d4ae6 fix erase 2019-03-10 23:29:58 +03:00
ivan-kush
521e897a92 init 2019-03-10 00:36:00 +03:00
Ivan Lezhankin
0e325c7a4a Replace const typeid_casts with As<Derived>() method invocations 2019-03-08 12:48:20 +03:00
Alex Zatelepin
896b8b15f6 fix build 2019-03-07 23:52:25 +03:00
Alexey Zatelepin
3344955154 separate Read and Write lock holders, better method names 2019-03-07 21:10:21 +03:00
Alexey Zatelepin
ab025805a3 alter intention lock 2019-03-07 20:21:18 +03:00
Alexey Zatelepin
c5dde2c9fd get rid of default query_id value in RWLock::getLock [#CLICKHOUSE-3789] 2019-02-28 12:37:05 +03:00
Alexey Zatelepin
0d7ab9e3ea allow acquiring read lock to a table out of order if it was already locked by the same query [#CLICKHOUSE-3789] 
this is important in preventing ALTER deadlocks
 2019-02-27 21:34:38 +03:00
Vitaly Baranov
309df832ec IStorage::read() function, "max_block_size" parameter: change type UInt64 -> size_t. 2019-02-19 14:56:08 +03:00
alexey-milovidov
48280074c4
Merge pull request #4369 from yandex/fix-race-condition-merge-tree-clear-old-temporary-directories 
Fixed rare race condition on startup of non-replicated MergeTree tables: concurrent attempt to remove a temporary directory [#CLICKHOUSE-4296]
 2019-02-12 21:37:48 +03:00
Nikita Vasilev
5279c7ff81 setting 2019-02-12 20:54:52 +03:00
Alexey Milovidov
7d1e755a7c Fixed rare race condition on startup of non-replicated MergeTree tables: concurrent attempt to remove a temporary directory [#CLICKHOUSE-4296] 2019-02-12 19:41:06 +03:00
Alexey Milovidov
58a6accee5 Updated test #4246 2019-02-10 19:55:12 +03:00
alexey-milovidov
2777e54a57
Merge pull request #4287 from yandex/mutations-introspection 
KILL MUTATION
 2019-02-10 00:51:30 +03:00
Alexey Milovidov
2eb1876a7a Slightly better message with reason for OPTIMIZE query with "optimize_throw_if_noop" 2019-02-06 18:44:27 +03:00
Alexey Zatelepin
3ec6f508f1 Merge branch 'master' into mutations-introspection 
Conflicts:
	dbms/src/Interpreters/InterpreterKillQueryQuery.cpp
 2019-02-06 15:22:48 +03:00
Alexey Zatelepin
8e437b191e try executing other mutations immediately after KILL MUTATION [#CLICKHOUSE-3912] 2019-02-05 21:09:16 +03:00
Alexey Zatelepin
d482f3e5d0 return CancellationCode from killMutation [#CLICKHOUSE-3912] 2019-02-05 21:09:15 +03:00
Alexey Zatelepin
e32f153328 KILL MUTATION for StorageReplicatedMergeTree [#CLICKHOUSE-3912] 2019-02-05 21:09:15 +03:00
Alexey Zatelepin
059c1b3589 write part_log.partition_id everywhere 2019-02-05 21:09:15 +03:00
Alexey Zatelepin
ca93fd483a kill ordinary MergeTree mutations [#CLICKHOUSE-3912] 2019-02-05 21:09:10 +03:00
Alexey Zatelepin
5832e474a4 is_mutation flag in system.merges [#CLICKHOUSE-3912] 2019-02-05 21:09:05 +03:00
Alexey Zatelepin
92a4342c31 system.merges element for mutations [#CLICKHOUSE-3912] 2019-02-05 21:09:00 +03:00
Alexey Zatelepin
51017959f2 fix typo in calculation of part_log.rows_read 2019-02-05 21:09:00 +03:00
Alexey Zatelepin
67be566325 KILL MUTATION skeleton [#CLICKHOUSE-3912] 2019-02-05 21:08:41 +03:00
Nikita Vasilev
a1b0dede07 Data Skipping Indices (#4143) 
* made index parser

* added index parsing

* some fixes

* added index interface and factory

* fixed compilation

* ptrs

* added indexParts

* indextypes

* index condition

* IndexCondition

* added indexes in selectexecutor

* fix

* changed comment

* fix

* added granularity

* comments

* fix

* fix

* added writing indexes

* removed indexpart class

* fix

* added setSkipIndexes

* add rw for MergeTreeIndexes

* fixes

* upd error

* fix

* fix

* reading

* test index

* fixed nullptr error

* fixed

* fix

* unique names

* asts -> exprlist

* minmax index

* fix

* fixed select

* fixed merging

* fixed mutation

* working minmax

* removed test index

* fixed style

* added indexes to checkDataPart

* added tests for minmax index

* fixed constructor

* fix style

* fixed includes

* fixed setSkipIndexes

* added indexes meta to zookeeper

* added parsing

* removed throw

* alter cmds parse

* fix

* added alter

* fix

* alters fix

* fix alters

* fix "after"

* fixed alter

* alter fix + test

* fixes

* upd setSkipIndexes

* fixed alter bug with drop all indices

* fix metadata editing

* new test and repl fix

* rm test files

* fixed repl alter

* fix

* fix

* indices

* MTReadStream

* upd test for bug

* fix

* added useful parsers and ast classes

* fix

* fix comments

* replaced columns

* fix

* fixed parsing

* fixed printing

* fix err

* basic IndicesDescription

* go to IndicesDescr

* moved indices

* go to indicesDescr

* fix test minmax_index*

* fixed MT alter

* fixed bug with replMT indices storing in zk

* rename

* refactoring

* docs ru

* docs ru

* docs en

* refactor

* rename tests

* fix docs

* refactoring

* fix

* fix

* fix

* fixed style

* unique idx

* unique

* fix

* better minmax calculation

* upd

* added getBlock

* unique_condition

* added termForAST

* unique

* fixed not

* uniqueCondition::mayBeTrueOnGranule

* fix

* fixed bug with double column

* is always true

* fix

* key set

* spaces

* test

* tests

* fix

* unique

* fix

* fix

* fixed bug with duplicate column

* removed unused data

* fix

* fixes

* __bitSwapLastTwo

* fix
 2019-02-05 17:50:25 +03:00
Alexey Zatelepin
b8ee63fa3a save information about failed parts for ordinary MergeTree mutations [#CLICKHOUSE-3192] 2019-02-05 16:04:58 +03:00
Alexey Milovidov
1d4701b0bc Added some warnings from clang's -Weverything 2019-01-04 15:10:00 +03:00
Alexey Milovidov
e33e5150b7 Miscellaneous [#CLICKHOUSE-2] 2019-01-02 09:44:36 +03:00
alesapin
8bc00f4632 CLICKHOUSE-4229: Fix user forwarding for replicated queries which actually send queries to master node 2018-12-28 19:21:32 +03:00
Alexey Milovidov
7509db544c Merge branch 'master' of github.com:yandex/ClickHouse into ubsan-fixes-3 2018-12-26 22:38:11 +03:00
alesapin
9342700ce1 Move processing pool to ternary logic 2018-12-26 20:03:29 +03:00
Alexey Milovidov
bbbefe2c03 Less dependencies [#CLICKHOUSE-2] 2018-12-26 02:18:07 +03:00
Alexey Milovidov
3be55436bf Less dependencies [#CLICKHOUSE-2] 2018-12-26 02:13:30 +03:00
Alexey Zatelepin
b7afe46dfd remove ALTER MODIFY PRIMARY KEY command [#CLICKHOUSE-4214] 2018-12-20 21:57:20 +03:00
proller
b213ee5e5d
Fix macos build (#3863) 2018-12-18 20:41:03 +03:00
alexey-milovidov
7cc867de0a
Merge branch 'master' into background-processing-pool-backoff 2018-12-14 17:41:55 +03:00
alexey-milovidov
48b30edcf5
Merge branch 'master' into alter-order-by 2018-12-02 21:06:38 +03:00
Ivan Lezhankin
8b5a05947c Remove redundant information from RWLock. 
And fix broken exception in some race conditions
 2018-11-28 18:50:52 +03:00
Alexey Zatelepin
1b27fb046d Merge branch 'master' into alter-order-by 
Conflicts:
	dbms/programs/server/Server.cpp
 2018-11-28 16:47:07 +03:00
Alexey Zatelepin
9ae79c60e8 get rid of (ASTPtr *) in AlterCommands::apply [#CLICKHOUSE-3859] 2018-11-28 16:34:57 +03:00
Alexey Zatelepin
e085bc9816 adapt code of #3609 [#CLICKHOUSE-3859] 2018-11-27 20:07:10 +03:00
Ivan Lezhankin
73aa7f2027 Rename method partition() → alterPartition() 
Also fix build with internal shared libraries.
 2018-11-26 17:43:40 +03:00
Alexey Zatelepin
d3d9242462 Merge branch 'master' into alter-order-by 
Conflicts:
	dbms/src/Parsers/ASTAlterQuery.h
	dbms/src/Parsers/ParserAlterQuery.cpp
	dbms/src/Storages/AlterCommands.cpp
	dbms/src/Storages/AlterCommands.h
 2018-11-26 16:01:48 +03:00
Sabyanin Maxim
3a67b505c5 Merge branch 'master' of https://github.com/yandex/ClickHouse into CLICKHOUSE-4032 2018-11-22 16:54:10 +03:00
alexey-milovidov
5038f13b08
Merge pull request #3514 from abyss7/CLICKHOUSE-3766 
Implement 'ALTER … FREEZE' command
 2018-11-21 07:17:21 +03:00
Ivan Lezhankin
1e070a7c51 Move FREEZE_ALL command source code inside |PartitionCommand| 2018-11-20 20:13:59 +03:00
Ivan Lezhankin
74fe350451 Refactor partition commands handling into separate interface method. 2018-11-20 17:59:34 +03:00
Sabyanin Maxim
e3fc18fcb8 add getters of expressions names in StorageReplicatedMergeTree 2018-11-20 15:19:12 +03:00
Sabyanin Maxim
d9e6789f78 make MergeTree work with comments 2018-11-15 16:12:27 +03:00
Alexey Zatelepin
b071547b2d add checks for ALTER MODIFY ORDER BY [#CLICKHOUSE-3859] 2018-11-14 20:12:32 +03:00
Alexey Zatelepin
29b34365e6 Merge branch 'master' into alter-order-by 
Conflicts:
	dbms/src/Interpreters/ExpressionAnalyzer.cpp
	dbms/src/Storages/MergeTree/MergeTreeData.cpp
	dbms/src/Storages/MergeTree/MergeTreeDataSelectExecutor.cpp
 2018-11-13 16:27:19 +03:00
Alexey Zatelepin
2cc0aae82c rearrange AST members in MergeTreeData [#CLICKHOUSE-3859] 2018-11-13 16:15:17 +03:00
Alexey Zatelepin
8b85822192 store original key ASTs, and not extracted ASTExpressionLists [#CLICKHOUSE-3859] 2018-11-13 16:14:44 +03:00
Alexey Zatelepin
0c0c4fd28d add sorting_and_primary_keys_independent [#CLICKHOUSE-3859] 2018-11-13 16:13:19 +03:00
Alexey Zatelepin
07a22100cb ALTER ORDER BY for ordinary MergeTree [#CLICKHOUSE-3859] 2018-11-13 16:13:14 +03:00
Alexey Zatelepin
f4c26e78d2 renames [#CLICKHOUSE-3859] 2018-11-13 16:13:01 +03:00
Alexey Zatelepin
0fa86162d9 separate sorting key and primary key, allow setting primary key separately [#CLICKHOUSE-3859] 2018-11-13 16:12:55 +03:00
Alexey Zatelepin
9a92c0a55c sort key is distinct from primary key [#CLICKHOUSE-3859] 2018-11-13 16:12:49 +03:00
Ivan Lezhankin
4eb0f33ec2 Add some locks on freeze and support |freezeAll()| for replicated merge-tree. 2018-11-12 15:26:15 +03:00
Ivan Lezhankin
935615a647 Reimplement FREEZE command. 2018-11-12 15:26:14 +03:00
Ivan Lezhankin
03e6a29eb2 Implement 'ALTER … FREEZE' command. 
Need some tests!
 2018-11-12 15:26:14 +03:00
alexey-milovidov
98c2ed582c
Merge pull request #2863 from VadimPE/ISSUES-2581 
ISSUES-2581 fix SELECT with sequential consistency
 2018-10-22 23:55:33 +03:00
VadimPE
28102bc3b5 ISSUES-2581 get pointer to max_blocls 2018-09-24 12:53:28 +03:00
Alexey Zatelepin
c2c12f22f7 try to immediately delete the part after checksums mismatch [#CLICKHOUSE-2837] 2018-09-21 19:07:21 +03:00
Vadim
204e1d437f
Merge branch 'master' into ISSUES-2581 2018-09-17 15:30:08 +03:00
zhang2014
7a87da78d7 ISSUES-3110 fix merge and distributed engine query stage 2018-09-14 18:16:27 +08:00
alexey-milovidov
0223225a05
Merge pull request #3035 from yandex/alter-update 
ALTER UPDATE
 2018-09-13 06:03:35 +03:00
alesapin
ca2334f992 Fix build 2018-09-12 22:57:14 +03:00
alesapin
1664bec008 CLICKHOUSE-3970: Add setting for O_DIRECT for big merges 2018-09-12 20:19:52 +03:00
Alexey Zatelepin
d77037fec7 fix use after move 2018-09-07 18:54:34 +03:00
Alexey Zatelepin
1064d1d26b update MATERIALIZED columns when their dependencies are updated [#CLICKHOUSE-13] 2018-09-07 18:54:18 +03:00
Alexey Zatelepin
816b91e2e3 additional checks for UPDATEs of key columns [#CLICKHOUSE-13] 2018-09-04 17:34:50 +03:00
VadimPE
6e78f576d7 ISSUES-2581 add comments, and fix 2018-09-03 11:25:35 +03:00
Vadim
3598f39f82
Merge branch 'master' into ISSUES-2581 2018-08-21 17:24:03 +03:00
Nikolai Kochetov
391b4b5927 Merged with master. 2018-08-20 15:25:06 +03:00
VadimPE
3c0f6ce87b ISSUES-2581 fix SELECT with select_sequential_consistency 2018-08-14 12:57:18 +03:00
Alexey Milovidov
36db216abf Preparation for extra warnings [#CLICKHOUSE-2] 2018-08-10 07:02:56 +03:00
Alexey Milovidov
d8ca4345b6 Fixed style violations [#CLICKHOUSE-2] 2018-08-10 04:41:54 +03:00
alexey-milovidov
0ef2a94d46
Merge pull request #2782 from VadimPE/CLICKHOUSE-3714 
CLICKHOUSE-3714 add max_partition_size_to_drop
 2018-08-06 20:20:59 +03:00
VadimPE
c15ba5e7d9 CLICKHOUSE-3714 2018-08-06 11:30:54 +03:00
VadimPE
490efebe62 CLICKHOUSE-3714 add size pf all part in partition 2018-08-06 11:27:59 +03:00
Alexey Milovidov
2016374d20 Merging #2226 2018-08-05 10:05:43 +03:00
Alexey Zatelepin
a0730d7696 old mutations cleaner for non-replicated MergeTree engines [#CLICKHOUSE-3828] 2018-08-03 22:21:44 +03:00
Vadim
d821f8a4cc
Update StorageMergeTree.cpp 2018-08-03 16:17:32 +03:00
VadimPE
b97fa2eb4d CLICKHOUSE-3714 bool ckeck_CanBeDropped -> void ... 2018-08-03 12:39:01 +03:00
VadimPE
6142d9ae6a CLICKHOUSE-3714 fix function names and get partition size 2018-08-03 11:33:57 +03:00
VadimPE
08fe108ba8 CLICKHOUSE-3714 fix methods name 2018-08-02 11:48:57 +03:00
Vadim
034a88a504
Merge branch 'master' into CLICKHOUSE-3714 2018-08-01 20:42:56 +03:00
VadimPE
66d9ba3eb7 CLICKHOUSE-3714 add max_partition_size_to_drop 2018-08-01 20:41:18 +03:00
Alexey Zatelepin
c793277bcc add forgotten sort [#CLICKHOUSE-3748] 2018-07-23 21:04:13 +03:00
alexey-milovidov
68b57e605a
Merge branch 'master' into background-processing-pool-backoff 2018-07-16 06:27:43 +03:00
Alexey Zatelepin
88aa790a2d save mutations to disk and load them on startup [#CLICKHOUSE-3748] 2018-07-16 06:22:22 +03:00
Alexey Zatelepin
78b83d78f0 execute part mutations in a background pool [#CLICKHOUSE-3748] 2018-07-16 06:22:22 +03:00
Alexey Zatelepin
88300258a7 info in system.mutations for nonreplicated MergeTree [#CLICKHOUSE-3748] 2018-07-16 06:22:22 +03:00
Alexey Zatelepin
ce17868a5c execute mutations synchronously for StorageMergeTree [#CLICKHOUSE-3748] 2018-07-16 06:22:22 +03:00
Alexey Milovidov
7dc1276afe Little better #2281 2018-07-08 08:26:51 +03:00
Alexey Milovidov
4a35b57cea Little better #2599 2018-07-05 21:45:18 +03:00
Amos Bird
78beacb3a5 Make OPTIMIZE be able to merge all parts with FINAL. 2018-07-05 21:32:54 +03:00
Nikolai Kochetov
ba4cf6faf0 Merged with master. 2018-06-25 15:08:21 +03:00
Alexey Milovidov
72af2b7b0f Removed wrong modification #2260 2018-06-10 21:26:02 +03:00
Alexey Milovidov
6475ea98c7 Merging: Fixed most of problems, actualized test #2260 2018-06-09 21:17:27 +03:00
Alexey Milovidov
d56c199f07 Merging: Fixed half of problems #2260 2018-06-09 18:48:22 +03:00
Alexey Milovidov
0bc678474e Merge branch 'feature/support_truncate' of https://github.com/zhang2014/ClickHouse into zhang2014-feature/support_truncate 2018-06-09 17:24:50 +03:00
Alexey Zatelepin
f6c72f8e56 Merge branch 'master' into replicated-mergetree-mutations 
Conflicts:
	dbms/src/Common/ActionBlocker.h
	dbms/src/Storages/MergeTree/ActiveDataPartSet.cpp
	dbms/src/Storages/MergeTree/ActiveDataPartSet.h
	dbms/src/Storages/MergeTree/MergeTreeData.h
	dbms/src/Storages/MergeTree/MergeTreeDataPart.cpp
	dbms/src/Storages/MergeTree/ReplicatedMergeTreeAlterThread.cpp
	dbms/src/Storages/MergeTree/ReplicatedMergeTreeLogEntry.cpp
	dbms/src/Storages/MergeTree/ReplicatedMergeTreeLogEntry.h
	dbms/src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp
	dbms/src/Storages/MergeTree/ReplicatedMergeTreeQueue.h
	dbms/src/Storages/MergeTree/ReplicatedMergeTreeRestartingThread.cpp
	dbms/src/Storages/StorageMergeTree.cpp
	dbms/src/Storages/StorageReplicatedMergeTree.cpp
	dbms/src/Storages/StorageReplicatedMergeTree.h
 2018-05-28 18:37:30 +03:00
Alexey Zatelepin
1147e052e8 select and write part mutations to queue [#CLICKHOUSE-3747] 2018-05-25 23:13:00 +03:00
Alexey Zatelepin
9a32c76e76 parts versioning [#CLICKHOUSE-3687] 2018-05-25 23:04:07 +03:00
Vitaliy Lyudvichenko
c888903488
Add query ALTER TABLE t1 [REPLACE|ATTACH] PARTITION FROM t2 
* Add ATTACH PARTITION FROM table for MergeTree. [#CLICKHOUSE-3546]

* Implemented replicated case on non-leader replica. [#CLICKHOUSE-3546]

* Disable merges in the dropping range. [#CLICKHOUSE-3546]

* DROP PARTITION is atomic and simpler now. [#CLICKHOUSE-3546]

* Implemented more SYSTEM queries. [#CLICKHOUSE-2931] [#CLICKHOUSE-3546]

SYSTEM queries:
RESTART REPLICAS
SYNC REPLICA db.name
STOP MERGES [db.name]
START MERGES [db.name]
STOP FETCHES [db.name]
START FETCHES [db.name]
STOP REPLICATED SENDS [db.name]
START REPLICATED SENDS [db.name]
STOP REPLICATION QUEUES [db.name]
START REPLICATION QUEUES [db.name]

* Fixed a bunch of bugs in REPLACE PARTITION. [#CLICKHOUSE-3546]

* Add tests for REPLACE PARTITION and SYSTEM. [#CLICKHOUSE-3546]

* Add system.part_log logging. [#CLICKHOUSE-3546]

* Fixed long wait in SYNC REPLICA. [#CLICKHOUSE-3546]

* Add requested changes. [#CLICKHOUSE-3546]

Fixed clickhouse-client bad return code.

* Add requested chenges. [#CLICKHOUSE-3546]

* Add requested chenges. [#CLICKHOUSE-3546]
 2018-05-21 16:49:54 +03:00
zhang2014
6613e56784 ISSUES-2259 support truncate syntax 2018-05-21 11:38:46 +08:00
Nikolai Kochetov
a3308bf675 added IStorage::getQueryProcessingStage 2018-04-19 17:47:09 +03:00
Alex Zatelepin
c5b04f7f17 Add info about total uncompressed part size to system tables (#2052) 
* column size calculating routines now take into account multiple streams [#CLICKHOUSE-2831]

* rename for clarity

* use more appropriate method for calculating table size

* add info about total uncompressed part size to system.parts and system.parts_columns [#CLICKHOUSE-2831]

* rename columns for clarity [#CLICKHOUSE-2831]

* count shared substreams only once [#CLICKHOUSE-2831]
 2018-03-26 17:18:04 +03:00
Alexey Zatelepin
5906c9a988 rename for clarity 2018-03-13 18:00:28 +03:00
Alexey Zatelepin
fdffbe53ac encapsulate ColumnsDeclaration [#CLICKHOUSE-3241] 2018-03-13 17:41:25 +03:00
Alexey Zatelepin
78d92f6794 add some useful methods, rename for clarity [#CLICKHOUSE-3241] 2018-03-12 22:10:13 +03:00
Alexey Zatelepin
71642b0880 use ColumnsDescription everywhere [#CLICKHOUSE-3241] 2018-03-12 22:07:48 +03:00
Alexey Milovidov
afbbd780e3 Fixed error with initialization of PartLog [#CLICKHOUSE-3354] 2018-03-10 22:57:13 +03:00
Alexey Milovidov
601ff1eee4 Simplification of AST #1947 2018-02-26 06:37:08 +03:00
Alexey Milovidov
6e93635275 Better UNION ALL: development #1947 2018-02-25 09:35:24 +03:00
Alexey Milovidov
722b6287ec Miscellaneous [#CLICKHOUSE-2] 2018-02-25 05:43:27 +03:00
Nikolai Kochetov
64b3a8baa7 added secondary sort expression [#CLICKHOUSE-3479] 2018-02-13 00:13:50 +03:00
Vitaliy Lyudvichenko
94f38d744e
Merge pull request #1750 from yandex/CLICKHOUSE-3346 
Fault-tolerant cluster copier util
 2018-01-31 13:21:51 +03:00
Vitaliy Lyudvichenko
61233b886a Better system.part_log. [#CLICKHOUSE-3342] 2018-01-25 21:40:02 +03:00
Vitaliy Lyudvichenko
26e58271e9 Removed repeatable columns field from Storages declarations. [#CLICKHOUSE-2] 2018-01-25 17:42:39 +03:00
Vitaliy Lyudvichenko
1000328571 Merge remote-tracking branch 'origin/master' into CLICKHOUSE-3346 2018-01-22 20:31:08 +03:00
alexey-milovidov
1b462b0b94
Merge pull request #1759 from yandex/CLICKHOUSE-3432 
Fixed 'intersects previous part' due to blind OPTIMIZE
 2018-01-19 21:48:41 +03:00
alexey-milovidov
7a25b441f4
Merge pull request #1773 from yandex/small-enhancements 
Small enhancements
 2018-01-16 22:07:43 +03:00
Vitaliy Lyudvichenko
b68b60fb38 More detailed test. [#CLICKHOUSE-3] 
Resolves #1553.
 2018-01-16 15:05:01 +03:00
proller
d97b2edc7c Make separate libs Common/ZooKeeper and Common/ConfigProcessor (#1749) 
* Make separate libs Common/ZooKeeper and Common/ConfigProcessor
renamed:    dbms/src/Common/StringUtils.h -> dbms/src/Common/StringUtils/StringUtils.h

* Fix space

* Fix data in system.build_options

* merge

* merge
 2018-01-15 22:07:47 +03:00
Vitaliy Lyudvichenko
03bca2144d OPTIMIZE doesn't produce intersecting merges now, add optimize_throw_if_noop setting. [#CLICKHOUSE-3432] [#CLICKHOUSE-3420] 
Resolves #1753
 2018-01-15 13:13:18 +03:00
Vitaliy Lyudvichenko
388d47bbbc Merge branch 'CLICKHOUSE-3346-v310' into CLICKHOUSE-3346 2018-01-11 23:51:30 +03:00
Vitaliy Lyudvichenko
c818beedcb Make data_path creation for Databases optional. [#CLICKHOUSE-2] 2018-01-11 21:55:43 +03:00
Alexey Milovidov
fcdd5de47d Revert "Removed questionable code (2) [#CLICKHOUSE-2]." 
This reverts commit fcabe8b888.
 2017-12-28 07:39:19 +03:00
Alexey Milovidov
c89041ce63 Removed questionable code (2) [#CLICKHOUSE-2]. 2017-12-26 01:51:04 +03:00
Alexey Milovidov
ec71908b00 Revert "Removed questionable code (2) [#CLICKHOUSE-2]." 
This reverts commit fcabe8b888.
 2017-12-26 00:57:29 +03:00
Alexey Milovidov
fcabe8b888 Removed questionable code (2) [#CLICKHOUSE-2]. 2017-12-26 00:32:33 +03:00
Alexey Milovidov
a85b2e5a95 Removed questionable code [#CLICKHOUSE-2]. 2017-12-26 00:10:46 +03:00
Alexey Zatelepin
c3348e27a4 shorter [#CLICKHOUSE-2] 2017-12-25 20:31:14 +03:00
Alexey Milovidov
1aa99092c3 Unification of serde of data types: development [#CLICKHOUSE-2838]. 2017-12-03 05:15:35 +03:00
Alexey Milovidov
acd78e4891 Better [#CLICKHOUSE-2]. 2017-12-02 00:13:25 +03:00
Vitaliy Lyudvichenko
b738d1baed Add multi index for data_parts storage. And fixed bugs. [#CLICKHOUSE-3452] 
Fixed handling of obsolete parts.
Fixed conflict resolution between simultaneous PreCommitted covering parts.
Fixed memory leak caused by ordinary MergeTree parts stucked in Deleting state.
Added hidden _state column into system.parts.
 2017-11-29 14:25:11 +03:00
proller
0d3c4f4d67 Core/FieldVisitors.h -> Common/FieldVisitors.h 
Core/Progress.h -> IO/Progress.h
    tests: sudo --non-interactive
    Fix freebsd link
 2017-11-24 16:55:31 +03:00
Alexey Milovidov
01e8adbfff Removed "experimental" namespace [#CLICKHOUSE-2]. 2017-11-20 07:15:43 +03:00
Vitaliy Lyudvichenko
e2a12d1088 Introduced logic for MergeTree parts states. [#CLICKHOUSE-3178] 2017-10-26 17:16:06 +03:00
Alexey Zatelepin
a7c85e0c89 save index_granularity to ATTACH query if it wasn't set explicitly [#CLICKHOUSE-3000] 2017-10-20 21:28:44 +03:00
Alexey Zatelepin
48dea685d0 use more typesafe AST objects in ASTCreateQuery [#CLICKHOUSE-3000] 2017-10-20 21:28:41 +03:00
Vitaliy Lyudvichenko
62ea1133dd Disable part sendings and fetches before ALTER. [#CLICKHOUSE-3343] 2017-10-12 23:26:54 +03:00
Vitaliy Lyudvichenko
cabf6d7799 Add better logging if OPTIMIZE cannot be executed. [#CLICKHOUSE-2] 2017-10-12 22:28:24 +03:00
Alexey Zatelepin
728a23f60e allow creation of custom-partitioned tables with experimental syntax [#CLICKHOUSE-3000] 2017-09-26 20:40:02 +03:00
Alexey Zatelepin
a10ba0cd91 serialize partition key in Replicated tables metadata [#CLICKHOUSE-3000] 2017-09-26 20:40:02 +03:00
Alexey Zatelepin
68cb592277 allow arbitrary partition literals in ALTER PARTITION queries [#CLICKHOUSE-3000] 2017-09-26 20:40:02 +03:00
Alexey Zatelepin
2f80c150db save MergeTree data format version and parse part names depending on it [#CLICKHOUSE-3000] 2017-09-26 20:40:02 +03:00
Alexey Milovidov
dc4976f562 Implemented some suggestions from clang-tidy (part 8) [#CLICKHOUSE-3301]. 2017-09-08 07:58:57 +03:00
alexey-milovidov
775d95888a Merge pull request #1163 from yandex/CLICKHOUSE-2931 
Introduce SYSTEM queries
 2017-09-01 20:45:29 +03:00
Vitaliy Lyudvichenko
566160c048 Use FIFO lock in IStorage. [#CLICKHOUSE-3246] 2017-09-01 20:45:05 +03:00
Alexey Zatelepin
14ba0a9acf rename partition -> partition_id for clarity [#CLICKHOUSE-3000] 2017-08-31 23:32:03 +03:00
Alexey Zatelepin
5471979f46 generic partitioning of inserted blocks [#CLICKHOUSE-3000] 2017-08-31 23:32:03 +03:00
Alexey Zatelepin
99969ab77a Introduce MergeTreePartitionIndex. Isolate its loading code. [#CLICKHOUSE-3000] 2017-08-31 23:32:03 +03:00
Vitaliy Lyudvichenko
48f5d8f1a3 Add SYSTEM DROP DNS CACHE. [#CLICKHOUSE-2931] 2017-08-28 20:27:16 +03:00
Alexey Zatelepin
f25f0cd759 reduce number of places where partitioning by month is assumed [#CLICKHOUSE-3000] 2017-08-15 22:30:52 +03:00
Vitaliy Lyudvichenko
10c14cfccb Small code enhancements according to clang-tidy. [#CLICKHOUSE-2931] 2017-08-04 17:35:36 +03:00
Alexey Milovidov
9406b6a5aa Get rid of ASTSet, because it is not really an AST and it lead to bugs [#CLICKHOUSE-3148]. 2017-07-15 06:48:36 +03:00
Nikolai Kochetov
75be0b81d8 removed typeid_cast from IColumn.h 2017-07-13 23:58:19 +03:00
Alexey Milovidov
a48a21f593 Merge branch 'master' into CLICKHOUSE-2896 2017-06-26 11:54:58 +03:00
Alexey Milovidov
28e9e06992 Preparation: locking block number only after write to filesystem; removed obsolete code [#CLICKHOUSE-3091]. 2017-06-25 03:01:10 +03:00
Vitaliy Lyudvichenko
d7db235269 Better names and log messages. [#CLICKHOUSE-2896] 2017-06-22 18:32:09 +03:00
Vitaliy Lyudvichenko
99f502d5e0 Support OPTIMIZE query on non-leader replica. [#CLICKHOUSE-2] 2017-06-22 18:01:08 +03:00
Vitaliy Lyudvichenko
53adb891b3 Add replicated DROP COLUMN FROM partition for optimistic case. [#CLICKHOUSE-2896] 2017-06-22 00:56:38 +03:00
Vitaliy Lyudvichenko
534b766250 CREATE of MergeTree checks that data dir is empty. [#CLICKHOUSE-3083] 2017-06-20 10:49:18 +03:00
Alexey Milovidov
38885d00a3 Merge branch 'storage-startup-method' 2017-06-07 02:27:16 +03:00
Alexey Milovidov
0d7d379b44 Removed duplicate code [#CLICKHOUSE-2]. 2017-06-06 21:36:13 +03:00
Alexey Milovidov
9ebf0b633f Added IStorage::startup method to avoid starting replication and merging threads too early during server startup [#CLICKHOUSE-2]. 2017-06-06 20:06:14 +03:00
Vitaliy Lyudvichenko
1e4906816c Avoid SIGSEGV in SystemLog destructors. [#CLICKHOUSE-3023] 2017-06-05 17:25:10 +03:00
Alexey Milovidov
ba499585cd Merge branch 'master' of github.com:yandex/ClickHouse 2017-06-02 18:54:43 +03:00
Alexey Milovidov
80080f1ad6 Small refinements [#CLICKHOUSE-2]. 2017-06-02 18:54:39 +03:00
Vitaliy Lyudvichenko
1a4b0eb355 Remove tmp part directories on server start. [#CLICKHOUSE-3044] 2017-05-31 18:01:25 +03:00
Alexey Milovidov
a77369d082 Preparation: Storages do not modify query [#CLICKHOUSE-31]. 2017-05-25 04:12:41 +03:00
Alexey Milovidov
557c3dc7f4 Preparation: removed support for unreplicated data in Replicated tables [#CLICKHOUSE-31]. 2017-05-25 00:38:56 +03:00
Alexey Milovidov
48f65e9097 Preparation: untangling [#CLICKHOUSE-31]. 2017-05-25 00:06:29 +03:00
Alexey Milovidov
0dd71e32e0 Preparations, part 8 [#CLICKHOUSE-31]. 2017-05-23 23:04:42 +03:00
Alexey Milovidov
0f252046ca Preparations [#CLICKHOUSE-2] 2017-05-22 01:25:25 +03:00
Vitaliy Lyudvichenko
6262fc8650 Fix inccorect columns size in MergeTreeData after ALTER. [#CLICKHOUSE-2] 2017-05-15 10:34:40 +04:00
Yuri Dyachenko
51d2ffc654 fix TODO 2017-04-17 21:15:14 +03:00
Yuri Dyachenko
10082f3ef4 support DEDUPLICATE option in OPTIMIZE query 2017-04-17 21:15:14 +03:00
f1yegor
0b243a8b01 translate comments 2017-04-16 23:17:32 +03:00
artpaul
050f1d4b0a drop column from partition for MergeTree 2017-04-15 06:52:04 +03:00
Alexey Milovidov
310736cd7e Moved headers and sources to same place [#CLICKHOUSE-3]. 2017-04-01 12:22:42 +03:00
Alexey Milovidov
09e3134bf4 Revert "Using C++17 [#CLICKHOUSE-3]." 
This reverts commit 431a735efe.
 2017-04-01 11:51:30 +03:00
Alexey Milovidov
ecb0b87bd8 Using C++17 [#CLICKHOUSE-3]. 2017-04-01 11:51:30 +03:00
Alexey Milovidov
137ad95929 Changed tabs to spaces in code [#CLICKHOUSE-3]. 2017-04-01 11:35:09 +03:00
Alexey Milovidov
4f44445ff0 Removed all trailing whitespaces [#CLICKHOUSE-2]. find . -name '*.h' -or -name '*.cpp' -or -name '*.cmake' -or -name 'CMakeLists.txt' -or -name '*.html' -or -name '*.xml' | grep -v contrib | xargs sed -i -r -e 's/\s+$//' 2017-03-31 19:00:30 +03:00
Ravengg
a973e3898d Add remove for part log (#638) 
* Add REMOVE_PART

* fix

* fix

* fix codestyle
 2017-03-28 20:18:49 +03:00
Alexey Milovidov
d3ae1267eb Fixed error when parts of unlimited total size could be selected for merge in case of low free disk space [#CLICKHOUSE-2895]. 2017-03-20 23:29:12 +03:00
Ravengg
b47cc72701 Fix part log (#591) 
* fix_part_log

* fixed language back

* add reset part_log back

* reset part_log when don't need more

* Fix header

* fix
 2017-03-14 19:58:34 +04:00
Alexey Milovidov
26788e3938 Formatting [#CLICKHOUSE-2]. 2017-03-13 21:01:46 +03:00
f1yegor
401425bb89 translate comments 2017-03-12 20:18:07 +01:00
Ravengg
b079dacfd1 Added part_log with statistics scripts (#549) 
* Added part_log

* first_test

* filter and hits_res

* Add renamer and drawer

* Add columns database and table into PartLog

* Add normal way to get table_name and database_name from part

* improve drawer

* add stats for random size parts

* Merge converter and drawer

* make drawer more informative

* add new data

* add new data

* new data

* add long range stats

* for checking best way

* Add add_parts script

* Good style for global merge

* delete commented code

* Fixed spaces to tabs

* Note that Stopwatch is started automatically.

* Style

* Update StorageMergeTree.cpp

* Update StorageReplicatedMergeTree.cpp

* Switch act_time_ms to duration_ms

* Added ability to disable part_log

* fixed getPartLog

* fix usage getPartLog

* fix
 2017-03-07 21:13:54 +04:00
Alexey Zatelepin
7090d0861a reinitialize primary key after ALTER [#CLICKHOUSE-2853] 2017-02-21 21:32:04 +04:00
Alexey Milovidov
c9b4a163be Added info about uncompressed size to system.columns table [#CLICKHOUSE-2831]. 2017-02-12 01:31:22 +04:00
Alexey Zatelepin
aace171ef6 forbid multidimensional array columns in MergeTree tables [#CLICKHOUSE-2803] 2017-02-10 09:05:33 +04:00
Vitaliy Lyudvichenko
c9e52a5cf9 Add requested PR changes. [#CLICKHOUSE-2145] [issue #325] 2017-01-24 22:26:29 +04:00