Commit Graph

1096 Commits

Author SHA1 Message Date
Alexey Milovidov
f4815364f6 Merging [#METR-19266]. 2016-12-10 08:03:12 +03:00
Alexey Milovidov
e97ecb6cf7 Merge branch 'master' into METR-19266
Conflicts:
	dbms/CMakeLists.txt
	dbms/include/DB/Functions/FunctionsArray.h
	dbms/include/DB/Functions/FunctionsString.h
	dbms/include/DB/Parsers/ExpressionListParsers.h
	dbms/include/DB/Storages/MergeTree/MergeTreeBlockInputStream.h
	dbms/include/DB/Storages/MergeTree/MergeTreeReader.h
	dbms/include/DB/Storages/MergeTree/MergeTreeThreadBlockInputStream.h
	dbms/include/DB/Storages/MergeTree/MergedBlockOutputStream.h
	dbms/src/Core/ErrorCodes.cpp
	dbms/src/Parsers/ASTFunction.cpp
	dbms/src/Storages/MergeTree/MergeTreeData.cpp
	dbms/src/Storages/MergeTree/MergeTreeDataPart.cpp
	dbms/src/Storages/MergeTree/MergeTreeReader.cpp
	dbms/src/Storages/StorageLog.cpp
2016-12-10 07:51:36 +03:00
Vitaliy Lyudvichenko
855dc56598 Added treshold setting for Vertical merge algorithm. [#METR-23305] 2016-12-09 13:13:16 +03:00
Vitaliy Lyudvichenko
f8eb9028fd Refined query_log, processes and merges metrics.
Also, dbms/CMakeLists.txt was rewritten.
2016-12-09 13:11:25 +03:00
Alexey Milovidov
514715589b Preparation [#METR-23747]. 2016-12-08 05:49:04 +03:00
proller
76dc51aae5 Merge remote-tracking branch 'upstream/master' into METR-23466 2016-12-07 22:08:09 +03:00
Alexey Milovidov
072f2fc7b8 Merges: start to lower maximum size of merge only when number of free entries in pool/queue is lower than specified threshold [#METR-21504]. 2016-12-06 06:09:07 +03:00
Alexey Milovidov
0717bfa4d2 Simplified LevelMergeSelector [#METR-21504]. 2016-12-06 00:34:46 +03:00
Vitaliy Lyudvichenko
202ba912ce Vertical merging algorithm for MergeTree engines. [#METR-23305] 2016-11-29 22:41:31 +03:00
proller
b529f29bb7 Merge remote-tracking branch 'upstream/master' into METR-23466 2016-11-29 19:57:01 +03:00
Alexey Milovidov
0a2d1ebbb3 Attempt to improve merge selecting algorithm [#METR-23632]. 2016-11-28 06:46:08 +03:00
Alexey Milovidov
cac8f9216f MergeSelector: attempt to do something that has sense [#METR-23632]. 2016-11-28 06:17:31 +03:00
proller
26fe10a7b4 Merge remote-tracking branch 'upstream/master' into METR-23466 2016-11-25 03:22:30 +03:00
Alexey Milovidov
8e5889cf0e Improved merge selecting algorithm for Replicated tables [#METR-23615]. 2016-11-25 01:41:27 +03:00
proller
3b335b5546 Merge remote-tracking branch 'upstream/master' into METR-23466 2016-11-24 22:58:40 +03:00
alexey-milovidov
8c2e228856 Revert "New "vertical" algorithm for MergeTrees' parts merges" 2016-11-24 23:08:54 +04:00
proller
e855c16f4a Merge remote-tracking branch 'upstream/master' into METR-23466 2016-11-24 21:25:56 +03:00
alexey-milovidov
8544a8b852 Merge pull request #171 from ludv1x/METR-23305
New "vertical" algorithm for MergeTrees' parts merges
2016-11-24 22:07:41 +04:00
proller
8c07957e38 Merge remote-tracking branch 'upstream/master' into METR-23466 2016-11-24 04:01:18 +03:00
alexey-milovidov
fdabcd4e38 Merge pull request #192 from ludv1x/enums-fix
Fixed Enum usage in PK expressions.
2016-11-24 03:13:16 +04:00
proller
c919c02ed4 Merge remote-tracking branch 'upstream/master' into METR-23466 2016-11-24 00:04:38 +03:00
Alexey Milovidov
a2e9359b18 Fixed rare deadlock after startup when structure of query_log was changed [#METR-23594]. 2016-11-23 23:41:39 +03:00
Vitaliy Lyudvichenko
4f9064ca19 Fixed case when pk column is wrapped by functions. 2016-11-23 22:58:33 +03:00
Vitaliy Lyudvichenko
d27951b4a7 Added requested changes. [#METR-23473] 2016-11-23 16:33:05 +03:00
Vitaliy Lyudvichenko
e07fc7cea5 Fixed Enum usage in PK expressions. [#METR-23473]
Added check for duplicates of column names in CREATE query.
2016-11-23 16:19:15 +03:00
Vitaliy Lyudvichenko
ef593d1b01 Added merge_tree.enable_vertical_merge_algorithm setting. 2016-11-23 15:59:25 +03:00
Vitaliy Lyudvichenko
588add5a49 Added requested changes.
Also fixed single thread read from empty storage Log.
2016-11-23 15:52:12 +03:00
Vitaliy Lyudvichenko
2cd5b8b604 Updated MergeInfo and progress callbacks with respect to vertical merge alg. 2016-11-23 15:52:12 +03:00
Vitaliy Lyudvichenko
ec9bcd8054 Vertical merging algorithm for ordinary MergeTreee. 2016-11-23 15:47:19 +03:00
proller
eeeacd0805 merge upstream 2016-11-23 01:33:02 +03:00
Alexey Milovidov
bba5dcbcfa Fixed deadlock [#METR-23557]. 2016-11-22 22:27:45 +03:00
Alexey Milovidov
94e344ee38 Part checks don't lock ALTERs [#METR-22214]. 2016-11-22 21:49:17 +03:00
proller
0b40a13ea7 Merge remote-tracking branch 'upstream/master' into METR-23466 2016-11-22 14:56:57 +03:00
Alexey Milovidov
9018906e1a Better [#METR-2944]. 2016-11-20 15:43:20 +03:00
proller
61e6ee9100 Merge remote-tracking branch 'upstream/master' into METR-23466 2016-11-18 15:14:34 +03:00
Alexey Milovidov
a44e5193b0 Better backpressure for INSERT SELECT into MergeTree [#METR-22327]. 2016-11-18 05:05:25 +03:00
proller
881319c4ec Merge remote-tracking branch 'upstream/master' into METR-23466 2016-11-18 04:48:21 +03:00
Alexey Milovidov
b19e61757e Little improvement for large number of small inserts [#METR-22327]. 2016-11-18 04:03:57 +03:00
proller
8fec8e2ac4 simple get works 2016-11-16 02:55:45 +03:00
Alexey Milovidov
e7fccc4044 Modified backpressure method for MergeTree [#METR-23488]. 2016-11-14 21:19:08 +03:00
Vitaliy Lyudvichenko
107f7b34c7 Added requested PR changes, 2nd iteration. 2016-11-14 02:57:18 +03:00
Vitaliy Lyudvichenko
bf924009b9 Added support of seekable files for StorageFile, more options for clickhouse-local. [#METR-23101] 2016-11-09 22:05:35 +03:00
Vitaliy Lyudvichenko
2242cfd6b0 Put DatabaseMemory class into separated file. 2016-11-09 22:05:35 +03:00
Vitaliy Lyudvichenko
0e2eb5436b Added StorageFile and template for clickhouse-local app. 2016-11-09 22:03:47 +03:00
Alexey Milovidov
34a9af27bf Inreased default ALTERs safety threshold [#METR-2944]. 2016-11-09 17:14:39 +03:00
Alexey Milovidov
efa056fc77 Attempt to improve merge selecting algorithm [#METR-21840]. 2016-11-05 04:11:51 +03:00
Alexey Milovidov
aaec9d1db6 Attempt for improvement [#METR-21840]. 2016-11-01 22:20:24 +03:00
Vladimir Smirnov
d36f52502e Make it compilable on OS X
It's still hackish and dirty, but server and client compies.

Server starts, but throwes meaningless exception on any query.

Client seems to be working fine.

Linux compilation might (but shouldn't) be broken (not tested).
2016-11-01 17:59:21 +01:00
Alexey Milovidov
a3a8b49e80 MergeSelector: development [#METR-21841]. 2016-11-01 14:26:36 +03:00
Alexey Milovidov
91dbf74ac0 MergeSelector: development [#METR-21841]. 2016-11-01 11:38:55 +03:00
Alexey Milovidov
573a84c89a MergeSelector: development [#METR-21841]. 2016-10-31 22:45:22 +03:00
Alexey Milovidov
49602c44a9 MergeSelector: development [#METR-21841]. 2016-10-31 22:32:08 +03:00
Alexey Milovidov
bee00bfcfc MergeSelector: development [#METR-21841]. 2016-10-30 14:05:45 +03:00
Alexey Milovidov
b7f635d8df Merge selector: development [#METR-21840]. 2016-10-30 12:13:30 +03:00
Alexey Milovidov
ba48688f25 MergeSelector: development [#METR-21840]. 2016-10-30 11:15:55 +03:00
Alexey Milovidov
c5c63bdf5d MergeSelector: development [#METR-21840]. 2016-10-30 06:39:28 +03:00
Alexey Milovidov
3d48f9d34b MergeSelector: development [#METR-21840]. 2016-10-30 06:12:25 +03:00
Alexey Milovidov
b5156a115d Additions [#METR-21504]. 2016-10-28 02:28:35 +03:00
Alexey Milovidov
74c3b30622 dbms: Simplified logic of selecting merges [#METR-21841]. 2016-10-28 01:50:02 +03:00
Alexey Milovidov
f6f92cb7bc Fixed error with falsely reported large memory consumption of long running INSERT queries [#METR-23288]. 2016-10-25 17:22:10 +03:00
Alexey Milovidov
8be993bd1e Preparation [#METR-2944]. 2016-10-25 09:49:24 +03:00
Alexey Milovidov
15b369bd08 More instrumentation [#METR-23272]. 2016-10-24 16:47:15 +03:00
Alexey Milovidov
2dd43be6e6 Don't hold shared_ptr to zookeeper for long time, because it makes very difficult to remove ephemeral nodes in case of temporary communication errors and thus breaks leader election [#METR-23272]. 2016-10-24 15:34:08 +03:00
Alexey Milovidov
ea852bb9d8 Separate compilation of CurrentMetrics; Separated AsynchronousMetrics from CurrentMetrics; Comments [#METR-23237]. 2016-10-24 07:06:27 +03:00
Alexey Milovidov
cac493d0a0 Addition to prev. revision [#METR-23237]. 2016-10-24 05:06:52 +03:00
Alexey Milovidov
32eeb84992 Merge branch 'master' of github.com:yandex/ClickHouse 2016-10-24 05:03:22 +03:00
Alexey Milovidov
9af5ce1e3b Separate compilation of ProfileEvents; comments [#METR-23237]. 2016-10-24 05:02:37 +03:00
f1yegor
af01f6fdcc translate config.xml and MergeTreeSettings.h 2016-10-23 22:13:10 +02:00
Alexey Milovidov
f05fa7695a Removed useless code [#METR-2944]. 2016-10-23 08:37:45 +03:00
Alexey Milovidov
afcd933237 Probably better [#METR-23198]. 2016-10-23 06:46:51 +03:00
Alexey Arno
a75cfb6df5 Merge branch 'master' into METR-19266 [#METR-19266] 2016-10-21 19:53:35 +03:00
Alexey Arno
3792a9f302 dbms: Server: fixes [#METR-19266] 2016-10-20 13:13:07 +03:00
Yuri Dyachenko
297a1547c5 more usable settings for delayed insert. issue: #METR-22563 2016-10-19 20:58:27 +03:00
Alexey Arno
924402f33e dbms: Server: merged from master [#METR-19266] 2016-10-19 18:00:56 +03:00
Alexey Arno
b5ff08bce6 dbms: Server: semantic cleanup [#METR-19266] 2016-10-19 10:15:26 +03:00
alexey-milovidov
8d7ba10835 Merge pull request #132 from ludv1x/METR-22802
Automatic reload of remote_servers configuration.
2016-10-14 23:32:27 +04:00
alexey-milovidov
fad9272161 Merge pull request #137 from ludv1x/METR-22471
Fixed std::terminate on DROP of StorageReplicatedMergeTree
2016-10-14 23:28:39 +04:00
Vitaliy Lyudvichenko
a0760a4821 Added requested PR changes. [#METR-22802] 2016-10-14 20:54:18 +03:00
Vitaliy Lyudvichenko
669e2982a5 Made merge_selecting_thread creation & deletion atomic. [#METR-22471] 2016-10-14 14:47:11 +03:00
Vitaliy Lyudvichenko
aa863ea7bb Fixed std::terminate on DROP of StorageReplicatedMergeTree. [#METR-22471] 2016-10-14 13:17:05 +03:00
Alexey Milovidov
dce56bfeba Implemented TODO [#METR-23132]. 2016-10-14 05:51:03 +03:00
Vitaliy Lyudvichenko
ebbc9b9877 Added abilty to automatically update clusters configuration. [#METR-22802] 2016-10-11 17:17:49 +03:00
Alexey Milovidov
6b2755bf2a Non significant change [#METR-2944]. 2016-09-23 23:33:06 +03:00
Alexey Arno
b7ca838dc0 dbms: Server: cleanups [#METR-19266] 2016-09-11 13:06:34 +03:00
Alexey Milovidov
469eebe4f0 Investigation [#METR-2944]. 2016-09-04 20:12:07 +03:00
Alexey Milovidov
322e5031bd Investigation [#METR-22327]. 2016-09-02 07:03:40 +03:00
Alexey Arno
6c064ab55b dbms: Better comment [#METR-18510] 2016-09-01 13:24:06 +03:00
Alexey Milovidov
0db84044e5 Better [#METR-22608]. 2016-09-01 05:26:45 +03:00
alexey-milovidov
012f34a799 Merge pull request #96 from yurial/yurial-IStorage-enable_shared_from_this
use std::enable_shared_from_this for IStorage
2016-08-31 00:00:23 +04:00
Yuri Dyachenko
94a2186dde rename ext::share_ptr_helper -> ext::shared_ptr_helper 2016-08-30 22:27:15 +03:00
Yuri Dyachenko
e5825f790f use std::enable_shared_from_this for IStorage 2016-08-29 16:28:34 +03:00
Alexey Milovidov
867b73b6b0 Fixed error with cancelling merges on ALTERs [#METR-22524]. 2016-08-26 22:50:04 +03:00
Alexey Arno
cd0541134a dbms: post-testing fixes [#METR-19266] 2016-08-24 03:39:38 +03:00
Alexey Milovidov
e0f3cbb432 Added optional 'default_database' parameter for Cluster (allow Distributed tables to connect to different databases on different shards/replicas) [#METR-22462]. 2016-08-22 23:34:21 +03:00
Alexey Milovidov
b3b2cd110a Tiny modification [#METR-22327]. 2016-08-13 04:59:09 +03:00
Alexey Arno
c74ee4e478 dbms: Code cleanup [#METR-19266] 2016-08-12 18:22:28 +03:00
Alexey Milovidov
6cc8dadd6d Fixed error [#METR-21993]. 2016-08-11 00:20:46 +03:00
Alexey Arno
f5d9d0ebc9 Merge branch 'master' into METR-19266 [#METR-19266] 2016-08-10 22:13:15 +03:00
Alexey Arno
e2b6e10ec1 dbms: Various fixes + more comments [#METR-19266] 2016-08-10 22:12:29 +03:00
Alexey Milovidov
d8eb52c189 Improvement [#METR-22352]. 2016-08-10 10:20:21 +03:00