Vitaliy Lyudvichenko
47becc0b92
Try to reduce false positive of sanity check. More detail dignostics. [#CLICKHOUSE-3571]
2018-02-21 20:06:29 +03:00
Alexey Zatelepin
5bc6bd55b1
allow PreCommitted parts to contain each other, clean up code
2018-02-20 16:33:28 +03:00
Alexey Milovidov
fa50fe80a0
Added method "getHeader" in IBlockOutputStream: development [#CLICKHOUSE-2]
2018-02-19 03:45:32 +03:00
Alexey Milovidov
30b657a0da
Header in every stream: development [#CLICKHOUSE-2]
2018-02-15 21:54:12 +03:00
Nikolai Kochetov
64b3a8baa7
added secondary sort expression [#CLICKHOUSE-3479]
2018-02-13 00:13:50 +03:00
Vitaliy Lyudvichenko
94f38d744e
Merge pull request #1750 from yandex/CLICKHOUSE-3346
...
Fault-tolerant cluster copier util
2018-01-31 13:21:51 +03:00
Vitaliy Lyudvichenko
e2fb6d8c80
Merge remote-tracking branch 'origin/master' into CLICKHOUSE-3346
2018-01-25 22:00:08 +03:00
Vitaliy Lyudvichenko
61233b886a
Better system.part_log. [#CLICKHOUSE-3342]
2018-01-25 21:40:02 +03:00
Vitaliy Lyudvichenko
26e58271e9
Removed repeatable columns field from Storages declarations. [#CLICKHOUSE-2]
2018-01-25 17:42:39 +03:00
Vitaliy Lyudvichenko
b4ab7d4b8e
Make early deduplication with block number creation. [#CLICKHOUSE-3545]
2018-01-23 18:50:29 +03:00
Vitaliy Lyudvichenko
1000328571
Merge remote-tracking branch 'origin/master' into CLICKHOUSE-3346
2018-01-22 20:31:08 +03:00
alexey-milovidov
1b462b0b94
Merge pull request #1759 from yandex/CLICKHOUSE-3432
...
Fixed 'intersects previous part' due to blind OPTIMIZE
2018-01-19 21:48:41 +03:00
alexey-milovidov
6198fe4a1a
Update StorageReplicatedMergeTree.cpp
2018-01-19 21:46:22 +03:00
alexey-milovidov
e8ebf3b56a
Update StorageReplicatedMergeTree.cpp
2018-01-19 21:45:28 +03:00
alexey-milovidov
7a25b441f4
Merge pull request #1773 from yandex/small-enhancements
...
Small enhancements
2018-01-16 22:07:43 +03:00
Vitaliy Lyudvichenko
b68b60fb38
More detailed test. [#CLICKHOUSE-3]
...
Resolves #1553 .
2018-01-16 15:05:01 +03:00
proller
d97b2edc7c
Make separate libs Common/ZooKeeper and Common/ConfigProcessor ( #1749 )
...
* Make separate libs Common/ZooKeeper and Common/ConfigProcessor
renamed: dbms/src/Common/StringUtils.h -> dbms/src/Common/StringUtils/StringUtils.h
* Fix space
* Fix data in system.build_options
* merge
* merge
2018-01-15 22:07:47 +03:00
Vitaliy Lyudvichenko
554ed52920
Avoid superflous pullLogsToQueue() calls. [#CLICKHOUSE-3432]
2018-01-15 13:30:31 +03:00
Vitaliy Lyudvichenko
03bca2144d
OPTIMIZE doesn't produce intersecting merges now, add optimize_throw_if_noop setting. [#CLICKHOUSE-3432] [#CLICKHOUSE-3420]
...
Resolves #1753
2018-01-15 13:13:18 +03:00
Vitaliy Lyudvichenko
388d47bbbc
Merge branch 'CLICKHOUSE-3346-v310' into CLICKHOUSE-3346
2018-01-11 23:51:30 +03:00
Vitaliy Lyudvichenko
c818beedcb
Make data_path creation for Databases optional. [#CLICKHOUSE-2]
2018-01-11 21:55:43 +03:00
Nikolai Kochetov
cbeeb84999
added http_*_timeout settings [#CLICKHOUSE-3440]
2018-01-09 21:41:01 +03:00
Chen Yufei
88ef42c222
Aggressive merge for ReplicatedMergeTree optimize.
2018-01-09 20:44:07 +03:00
Alexey Milovidov
fcdd5de47d
Revert "Removed questionable code (2) [#CLICKHOUSE-2]."
...
This reverts commit fcabe8b888
.
2017-12-28 07:39:19 +03:00
Alexey Milovidov
798299ba89
Merge branch 'aavdonkin-master'
2017-12-28 07:33:55 +03:00
Alexey Milovidov
b2fcf06353
Merging [#CLICKHOUSE-3114]
2017-12-27 00:34:06 +03:00
Alexey Milovidov
c89041ce63
Removed questionable code (2) [#CLICKHOUSE-2].
2017-12-26 01:51:04 +03:00
Alexey Milovidov
ec71908b00
Revert "Removed questionable code (2) [#CLICKHOUSE-2]."
...
This reverts commit fcabe8b888
.
2017-12-26 00:57:29 +03:00
Alexey Milovidov
fcabe8b888
Removed questionable code (2) [#CLICKHOUSE-2].
2017-12-26 00:32:33 +03:00
Alexey Milovidov
a85b2e5a95
Removed questionable code [#CLICKHOUSE-2].
2017-12-26 00:10:46 +03:00
Alexey Zatelepin
c3348e27a4
shorter [#CLICKHOUSE-2]
2017-12-25 20:31:14 +03:00
Alexey Zatelepin
6d5ce3fa9a
revert BackgroundSchedulePool usage in Replicated tables
2017-12-22 21:25:54 +03:00
Alexey Zatelepin
835b957e95
add comments [#CLICKHOUSE-3508]
2017-12-20 20:40:36 +03:00
Alexey Zatelepin
d35baafde8
check PreCommitted parts when executing log entries and in PartChecker [#CLICKHOUSE-3508]
2017-12-20 20:04:14 +03:00
Alexey Zatelepin
2340e55d0e
revert to old behaviour of getPartIfExists() [#CLICKHOUSE-3508]
2017-12-20 20:04:14 +03:00
Alexey Milovidov
1aa99092c3
Unification of serde of data types: development [#CLICKHOUSE-2838].
2017-12-03 05:15:35 +03:00
Alexey Milovidov
54e601019c
Better [#CLICKHOUSE-2].
2017-12-02 00:40:58 +03:00
Alexey Milovidov
acd78e4891
Better [#CLICKHOUSE-2].
2017-12-02 00:13:25 +03:00
Alexey Milovidov
1a6cb2989b
Better [#CLICKHOUSE-2].
2017-12-01 21:36:55 +03:00
Vitaliy Lyudvichenko
b738d1baed
Add multi index for data_parts storage. And fixed bugs. [#CLICKHOUSE-3452]
...
Fixed handling of obsolete parts.
Fixed conflict resolution between simultaneous PreCommitted covering parts.
Fixed memory leak caused by ordinary MergeTree parts stucked in Deleting state.
Added hidden _state column into system.parts.
2017-11-29 14:25:11 +03:00
proller
0d3c4f4d67
Core/FieldVisitors.h -> Common/FieldVisitors.h
...
Core/Progress.h -> IO/Progress.h
tests: sudo --non-interactive
Fix freebsd link
2017-11-24 16:55:31 +03:00
Alexey Milovidov
f6f77f5f0b
Fixed linking [#CLICKHOUSE-2].
2017-11-20 08:22:07 +03:00
Alexey Milovidov
6c8b5e29d8
Additions after merge [#CLICKHOUSE-2].
2017-11-20 00:17:58 +03:00
Alexey Milovidov
9b92c0fecd
BackgroundSchedulePool: miscellaneous [#CLICKHOUSE-2].
2017-11-19 03:22:13 +03:00
Silviu Caragea
dfecd0b2f2
Merge with master
2017-11-18 10:16:06 +02:00
Alexey Milovidov
729b114c15
Removed unsupported functionality [#CLICKHOUSE-2].
2017-11-18 05:34:00 +03:00
Alexey Milovidov
076e0ae014
More clear [#CLICKHOUSE-2].
2017-11-18 00:20:28 +03:00
Alexey Milovidov
7c9ba15ba1
Style [#CLICKHOUSE-2].
2017-11-17 23:42:03 +03:00
Silviu Caragea
64619c614f
Basic idea is :
...
- We add BackgroundSchedulePool which can execute a function at a specific point in time. Basically all tasks are added in a queue and precessed by worker threads.
- The most important difference between this and BackgroundProcessingPool is that we have the guarantee that the same function is not executed from many workers in the same time
- Each of the following classes instead starting a thread will register a task in BackgroundSchedulePool and when they need to run will call schedule or scheduleAfter(duration) functions
This commit is moving all threads created by ReplicatedMergeTree to BackgroundSchedulePool tasks
NOTE: I did a minimum number of changes to be much simple to review the code
2017-11-17 10:58:35 +02:00
Alexey Zatelepin
99b2d135df
clear deduplication blocks when doing DROP PARTITION [#CLICKHOUSE-3208]
2017-11-15 23:13:10 +03:00
Alexey Milovidov
4e04c4da5f
Attempt to change ext::shared_ptr_helper (incomplete) [#CLICKHOUSE-2].
2017-11-04 06:56:31 +03:00
KochetovNicolai
bfb4e8f3fe
zookeeper chroot fix ( #1430 )
...
* added root and identity section to zookeeper configuration [#CLICKHOUSE-3031]
* added root and identity section to zookeeper configuration [#CLICKHOUSE-3031]
* moved root section in zookeeper configuration [#CLICKHOUSE-3031]
* fixed root section in zookeeper configuration [#CLICKHOUSE-3031]
* added tests [#CLICKHOUSE-3031]
* replaced tabs to spaces in config files [#CLICKHOUSE-3031]
* fix build [#CLICKHOUSE-3031]
* added chech that root starts with '/' in zookeeper config file [#CLICKHOUSE-3031]
* added comments [#CLICKHOUSE-3031]
* Check if zookeeper chroot exists before start. Remove tailing slash form chroot. [#CLICKHOUSE-3400]
* added starting '/' to zookeeper path in ReplicatedMergeTree
2017-11-02 01:59:27 +03:00
alexey-milovidov
3b99b723d7
Merge pull request #1245 from yandex/CLICKHOUSE-3178
...
Atomic parts addition and removing
2017-10-27 23:03:33 +03:00
Vitaliy Lyudvichenko
a0d06c5ee0
Don't deduplicate a block inserted from MV if the source block is Ok. [#CLICKHOUSE-3381]
2017-10-26 17:58:00 +03:00
Vitaliy Lyudvichenko
397b8bcf92
Smarter parts deletion. [#CLICKHOUSE-3178]
2017-10-26 17:16:06 +03:00
Vitaliy Lyudvichenko
b5c7eaa80e
Force data parts cleanup before repeat fetch. [#CLICKHOUSE-3178]
2017-10-26 17:16:06 +03:00
Vitaliy Lyudvichenko
71833a7f0f
More correct procedure of parts deleting. [#CLICKHOUSE-3178]
2017-10-26 17:16:06 +03:00
Vitaliy Lyudvichenko
db3a67a421
Add clearer RangeFiltered implementation. [#CLICKHOUSE-3178]
2017-10-26 17:16:06 +03:00
Vitaliy Lyudvichenko
e2a12d1088
Introduced logic for MergeTree parts states. [#CLICKHOUSE-3178]
2017-10-26 17:16:06 +03:00
Alexey Milovidov
65982a1642
Fixed exception messages [#CLICKHOUSE-2].
2017-10-25 22:17:37 +03:00
Alexey Zatelepin
c61d4106e8
allow loading all MergeTree settings from a query [#CLICKHOUSE-3000]
2017-10-20 21:28:45 +03:00
Alexey Zatelepin
a7c85e0c89
save index_granularity to ATTACH query if it wasn't set explicitly [#CLICKHOUSE-3000]
2017-10-20 21:28:44 +03:00
Alexey Zatelepin
e51f95872f
fix backward compatibility for new part format: [#CLICKHOUSE-3000]
...
* do not write out partition and minmax index files for old MergeTreeData format
* download the part from another replica if there are missing or extra files
2017-10-19 21:24:30 +03:00
Alexey Milovidov
d3d976a6fb
Misc [#CLICKHOUSE-2].
2017-10-12 23:45:55 +03:00
proller
b993667156
Try fix strange terminate ( #1329 )
...
* Tests: External: rename --use_http => --no_http and fix
* Try fix strange terminate
2017-10-12 23:42:58 +03:00
Vitaliy Lyudvichenko
62ea1133dd
Disable part sendings and fetches before ALTER. [#CLICKHOUSE-3343]
2017-10-12 23:26:54 +03:00
Vitaliy Lyudvichenko
cabf6d7799
Add better logging if OPTIMIZE cannot be executed. [#CLICKHOUSE-2]
2017-10-12 22:28:24 +03:00
Alexey Zatelepin
728a23f60e
allow creation of custom-partitioned tables with experimental syntax [#CLICKHOUSE-3000]
2017-09-26 20:40:02 +03:00
Alexey Zatelepin
a10ba0cd91
serialize partition key in Replicated tables metadata [#CLICKHOUSE-3000]
2017-09-26 20:40:02 +03:00
Alexey Zatelepin
909b46de0f
add MergeTreeDataFormatVersion constant [#CLICKHOUSE-3000]
2017-09-26 20:40:02 +03:00
Alexey Zatelepin
68cb592277
allow arbitrary partition literals in ALTER PARTITION queries [#CLICKHOUSE-3000]
2017-09-26 20:40:02 +03:00
Alexey Zatelepin
2f80c150db
save MergeTree data format version and parse part names depending on it [#CLICKHOUSE-3000]
2017-09-26 20:40:02 +03:00
Alexey Milovidov
94b975e7f9
Removed function deprecated in C++ [#CLICKHOUSE-3174].
2017-09-09 02:31:18 +03:00
Alexey Milovidov
8b313ab99e
Fixed warnings found by cppcheck [#CLICKHOUSE-3300].
2017-09-08 00:04:48 +03:00
Alexey Milovidov
0dcee75180
Supporessed useless false positive errors found by cppcheck [#CLICKHOUSE-3300].
2017-09-07 22:13:37 +03:00
Vitaliy Lyudvichenko
d55d5eca46
Remove extra Context reference and add required ones. Part 2. [#CLICKHOUSE-3117]
2017-09-06 04:18:07 +03:00
Vitaliy Lyudvichenko
566160c048
Use FIFO lock in IStorage. [#CLICKHOUSE-3246]
2017-09-01 20:45:05 +03:00
Alexey Zatelepin
d2847f22af
PR fixes [#CLICKHOUSE-3000]
2017-08-31 23:32:03 +03:00
Alexey Zatelepin
14ba0a9acf
rename partition -> partition_id for clarity [#CLICKHOUSE-3000]
2017-08-31 23:32:03 +03:00
Alexey Zatelepin
645e4b7a53
generalize MinMax index [#CLICKHOUSE-3000]
2017-08-31 23:32:03 +03:00
Alexey Zatelepin
5471979f46
generic partitioning of inserted blocks [#CLICKHOUSE-3000]
2017-08-31 23:32:03 +03:00
Alexey Zatelepin
99969ab77a
Introduce MergeTreePartitionIndex. Isolate its loading code. [#CLICKHOUSE-3000]
2017-08-31 23:32:03 +03:00
Alexey Milovidov
495daffa16
Miscellaneous [#CLICKHOUSE-2].
2017-08-30 21:13:32 +03:00
Vitaliy Lyudvichenko
5916d823ef
Fix deadlock in case of using Buffer table with CLEAR COLUMN IN PARTITION. [#CLICKHOUSE-3238]
2017-08-30 17:34:40 +03:00
Vitaliy Lyudvichenko
84660029ec
Don't leave trash part nodes in ZK in case of deleting non-existing duplicated parts. [#CLICKHOUSE-3252]
2017-08-29 20:49:37 +03:00
Vitaliy Lyudvichenko
fd63277b15
Unified spelling. [#CLICKHOUSE-2]
2017-08-28 18:27:25 +03:00
Vitaliy Lyudvichenko
37820f7f64
Don't leave trash parts in ZK in case of temporary errors. [#CLICKHOUSE-3252]
2017-08-28 18:27:25 +03:00
Alexey Zatelepin
f25f0cd759
reduce number of places where partitioning by month is assumed [#CLICKHOUSE-3000]
2017-08-15 22:30:52 +03:00
Vitaliy Lyudvichenko
8a14cc49ff
asyncMulti() -> multi(). [#CLICKHOUSE-3221]
2017-08-15 19:18:02 +03:00
Alexey Zatelepin
3f3b7f61e2
always send changed settings for remote connections [#CLICKHOUSE-3151]
2017-08-13 12:17:09 +03:00
Vitaliy Lyudvichenko
351a0905d5
Implemented fast block and parts cleaning. [#CLICKHOUSE-3207]
2017-08-10 23:09:05 +03:00
Vitaliy Lyudvichenko
10c14cfccb
Small code enhancements according to clang-tidy. [#CLICKHOUSE-2931]
2017-08-04 17:35:36 +03:00
Alexey Milovidov
6578fb2645
Fixed error [#CLICKHOUSE-3191].
2017-08-01 00:39:24 +03:00
Alexey Milovidov
8fb452e99a
Fixed wrong log message when removing temporary part after merge, that is not byte-identical than same part on replicas; added profile event; added fail-safe logic [#MTRSADMIN-3175].
2017-07-29 00:25:24 +03:00
Vitaliy Lyudvichenko
bc9ee54985
Don't throw exception in case of clearing empty part. [#CLICKHOUSE-2896]
2017-07-24 18:04:17 +03:00
alexey-milovidov
d4d266fa18
ColumnConst unification ( #1011 )
...
* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].
* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].
* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].
* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].
* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].
* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].
* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].
* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].
* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].
* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].
* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].
* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].
* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].
* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].
* Fixed error in ColumnArray::replicateGeneric [#CLICKHOUSE-3150].
* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].
* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].
* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].
* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].
* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].
* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].
* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].
* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].
* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].
* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].
* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].
* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].
* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].
2017-07-21 09:35:58 +03:00
Alexey Milovidov
9406b6a5aa
Get rid of ASTSet, because it is not really an AST and it lead to bugs [#CLICKHOUSE-3148].
2017-07-15 06:48:36 +03:00
Nikolai Kochetov
283810e8ab
merged with master
2017-07-13 23:59:34 +03:00
Nikolai Kochetov
75be0b81d8
removed typeid_cast from IColumn.h
2017-07-13 23:58:19 +03:00
Alexey Milovidov
5b5795633e
Using lexer (incomplete) [#CLICKHOUSE-2].
2017-07-13 08:59:01 +03:00