alexey-milovidov
d6750880e0
Update ReplicatedMergeTreeCleanupThread.cpp
2017-07-27 11:19:20 +03:00
Vitaliy Lyudvichenko
cd5bb4d921
Add replicated_deduplication_window_seconds merge_tree parameter. [#CLICKHOUSE-3173]
2017-07-27 11:19:20 +03:00
Vitaliy Lyudvichenko
dcc4d3bff0
Avoid zxid overflow problem when cleaning old blocks used in the deduplication. [#CLICKHOUSE-3173]
2017-07-27 11:19:20 +03:00
Nikolai Kochetov
01a004050d
fixed MergeTreeBlockSizePredictor updating from empy block while reading
2017-07-26 20:03:09 +03:00
Alexey Milovidov
4a4e863f9d
Fixed warning on clang 6 [#CLICKHOUSE-2].
2017-07-25 01:44:13 +03:00
Nikolai Kochetov
fcd50e3629
fixed comment
2017-07-24 18:06:32 +03:00
Nikolai Kochetov
b55dcc9f93
cnanged comments; fixed 00443_preferred_block_size_bytes test
2017-07-24 16:59:03 +03:00
Nikolai Kochetov
bc0921a64e
added comments
2017-07-21 21:02:02 +03:00
Nikolai Kochetov
c64b65ee29
ignore preferred_max_column_in_block_size_bytes if zero [#CLICKHOUSE-3170]
2017-07-21 20:45:51 +03:00
Nikolai Kochetov
755d16b22d
set min block size as index granularity for max_block_size limit during reading [#CLICKHOUSE-3170]
2017-07-21 14:55:33 +03:00
Alexey Milovidov
c7b1e92522
Fixed error after merge [#CLICKHOUSE-3150].
2017-07-21 09:40:05 +03:00
alexey-milovidov
d4d266fa18
ColumnConst unification ( #1011 )
...
* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].
* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].
* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].
* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].
* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].
* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].
* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].
* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].
* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].
* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].
* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].
* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].
* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].
* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].
* Fixed error in ColumnArray::replicateGeneric [#CLICKHOUSE-3150].
* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].
* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].
* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].
* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].
* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].
* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].
* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].
* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].
* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].
* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].
* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].
* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].
* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].
2017-07-21 09:35:58 +03:00
alexey-milovidov
1a1112d8bb
Update MergeTreeBlockReadUtils.h
2017-07-20 23:45:23 +03:00
Nikolai Kochetov
0b7439469d
fixed MergeTreeBlockSizePredictor initialization for sample block
2017-07-20 23:45:23 +03:00
Nikolai Kochetov
00ae32c946
renamed approximate_number_of_rows_in_part to number_of_rows_in_part
2017-07-20 23:45:23 +03:00
Nikolai Kochetov
e612152584
changed initialization in MergeTreeBlockSizePredictor
2017-07-20 23:45:23 +03:00
Nikolai Kochetov
2afd84c40c
fixed is_reading_finished status assignment in MergeTreeRangeReader [#CLICKHOUSE-3161]
2017-07-19 19:39:18 +03:00
Nikolai Kochetov
b5e94ede62
fixed is_reading_finished status assignment in MergeTreeRangeReader [#CLICKHOUSE-3161]
2017-07-19 16:44:46 +03:00
Alexey Milovidov
6e6e9f9d38
Removed useless variable [#CLICKHOUSE-3161].
2017-07-19 04:05:39 +03:00
Nikolai Kochetov
7e293df9af
merged with master
2017-07-17 22:48:22 +03:00
Nikolai Kochetov
eae94edd6b
fixed vertical merge for ReplacingSortedBlockInputStream [#CLICKHOUSE-3131]
2017-07-17 22:40:30 +03:00
Alexey Milovidov
9406b6a5aa
Get rid of ASTSet, because it is not really an AST and it lead to bugs [#CLICKHOUSE-3148].
2017-07-15 06:48:36 +03:00
Nikolai Kochetov
a13423e552
added vertical merge for ReplacingSortedBlockInputStream
2017-07-14 21:17:23 +03:00
Alexey Milovidov
1a0f5dd7cf
Getting rid of ASTSet (incomplete) [#CLICKHOUSE-2]
2017-07-14 03:33:37 +03:00
Nikolai Kochetov
283810e8ab
merged with master
2017-07-13 23:59:34 +03:00
Nikolai Kochetov
75be0b81d8
removed typeid_cast from IColumn.h
2017-07-13 23:58:19 +03:00
Nikolai Kochetov
210f14aaba
renamed IDataType::updateAvgValueSizeHints to IDataType::updateAvgValueSizeHint
2017-07-13 21:54:17 +03:00
Nikolai Kochetov
f3f217b40a
fixed tests
2017-07-13 21:45:09 +03:00
Nikolai Kochetov
546012079c
fixed reading with zero max block size setting
2017-07-13 21:45:09 +03:00
Nikolai Kochetov
e2d032b883
fixed reading with zero max block size setting
2017-07-13 21:45:09 +03:00
Nikolai Kochetov
8ea812e669
better comment for MergeTreeRangeReader
2017-07-13 21:45:09 +03:00
Nikolai Kochetov
f232afd2f8
round number of rows to nearest mark in MergeTreeBaseBlockInputStream
2017-07-13 21:45:09 +03:00
Nikolai Kochetov
f1775af3f0
deleted commented code
2017-07-13 21:45:09 +03:00
Nikolai Kochetov
30ed774bf7
refactoring and more comments in MergeTreeBaseBlockInputStream [#CLICKHOUSE-3065]
2017-07-13 21:45:09 +03:00
Nikolai Kochetov
89386394aa
refactoring and more comments in MergeTreeBaseBlockInputStream [#CLICKHOUSE-3065]
2017-07-13 21:45:09 +03:00
Nikolai Kochetov
8c65d6013e
added preferred_max_column_in_block_size_bytes setting, added filtration ratio prediction [#CLICKHOUSE-3065]
2017-07-13 21:45:09 +03:00
Nikolai Kochetov
4470afc9a9
added estimateNumRowsMax to MergeTreeBlockSizePredictor based on max column size
2017-07-13 21:45:09 +03:00
Nikolai Kochetov
6b29b3bd47
round number of rows to next mark in MergeTreeBaseBlockInputStream
2017-07-13 21:45:09 +03:00
Nikolai Kochetov
d14f64af7b
delected commented logging code
2017-07-13 21:45:09 +03:00
Nikolai Kochetov
586dff9126
delay reading for granulas if prewhere column is false during partial granula reading
2017-07-13 21:45:09 +03:00
Nikolai Kochetov
86680f5513
fixed partial granula reading for const columns in prewhere
2017-07-13 21:45:09 +03:00
Nikolai Kochetov
609711b20c
partial part reading with prewhere
2017-07-13 21:45:09 +03:00
Nikolai Kochetov
c6d5ef6d30
added MergeTreeRangeReader
2017-07-13 21:45:09 +03:00
Nikolai Kochetov
985e449bd3
added IDataType::updateAvgValueSizeHints; renamed PODArray::allocated_size to PODArray::allocated_bytes; renamed renamed IColumn::allocatedSize to IColumn::allocatedBytes; removed IColumn.h from IDataType.h
2017-07-13 19:49:09 +03:00
Alexey Milovidov
de75fa000b
Using lexer (incomplete) [#CLICKHOUSE-2].
2017-07-13 08:59:01 +03:00
Alexey Milovidov
83c8043193
Using lexer (incomplete) [#CLICKHOUSE-2].
2017-07-13 08:59:01 +03:00
Alexey Milovidov
4b1eeee1a3
Using lexer (incomplete) [#CLICKHOUSE-2].
2017-07-13 08:59:01 +03:00
Alexey Zatelepin
58d541b740
slightly more robust ReplicatedMergeTreePartCheckThread::TemporarilyStop [#CLICKHOUSE-3074]
2017-07-12 20:09:44 +03:00
Vitaliy Lyudvichenko
b843ea5b3f
Resolves #964 . Fixed input_format_skip_unknown_fields for negative numbers. [#CLICKHOUSE-3]
2017-07-11 21:36:51 +03:00
Alexey Zatelepin
4a98d83a3b
introduce replicated_can_become_leader merge_tree setting [#CLICKHOUSE-3109]
2017-07-11 20:44:35 +03:00
Alexey Milovidov
2bd84600d1
Miscellaneous [#CLICKHOUSE-2].
2017-07-11 04:20:40 +03:00
alexey-milovidov
b64b243a73
Merge pull request #945 from YiuRULE/master
...
Implement natively uuid type in Clickhouse
2017-07-10 07:00:45 +03:00
Alexey Zatelepin
89939a685a
Stream row sources from disk for vertical merge [#CLICKHOUSE-3118]
2017-07-09 21:57:06 +03:00
Marek Vavruša
ff73394124
PKCondition: infer index use with pk subexpression
...
By default only constraints explicitly matching
primary key expression (or expression wrapped in
a monotonic function) are eligible for part and
range selection. So for example, if index is:
(toStartOfHour(dt), UserID)
Then a query such as this resorts to full scan:
SELECT count() FROM t WHERE dt = now()
Intuitively, only parts with toStartOfHour(now())
could be selected, but it is less trivial to prove.
The primary key currently can be wrapped in a chain
of monotonic functions, so following would work:
toStartOfHour(dt) = toStartOfHour(now()) AND dt = now()
It must be however explicitly stated, if we wanted
to infer that we’d have to know the inverse function,
and prove that the inverse function is monotonic
on given interval. This is not practical as
there is no inverse function that for example undos
rounding, it isn’t strictly monotonic.
There are however functions that don’t transform
output range and preserve monotonicity on the
complete input range, such as rounding or casts
to a same or wider numeric type. This eliminates
the need to find inverse function, as no check for monotonicity over arbitrary interval is needed,
and thus makes this optimisation possible.
2017-07-09 17:13:51 +03:00
Alexey Milovidov
032d12d431
Fixed translation error [#CLICKHOUSE-2].
2017-07-04 15:46:31 +03:00
Guillaume Tassery
cba9815981
Change Uuid to UUID datatype and make UInt128 a generic type
2017-07-04 12:42:53 +02:00
Guillaume Tassery
525851ec68
sync master
2017-06-30 22:21:42 +02:00
Guillaume Tassery
1231952fca
Implement UUID type and toUuid function
2017-06-30 22:01:27 +02:00
Alexey Zatelepin
9de4d8facb
allow loading missing setting from config [#CLICKHOUSE-3091]
2017-06-28 19:56:35 +03:00
Alexey Zatelepin
7cf413dea8
Revert "Raised up limit on max size of parts to merge [#CLICKHOUSE-3091]."
...
This reverts commit 425156e32c
2017-06-28 15:32:52 +03:00
Alexey Milovidov
a48a21f593
Merge branch 'master' into CLICKHOUSE-2896
2017-06-26 11:54:58 +03:00
Alexey Milovidov
8f145da1e8
Preparation [#CLICKHOUSE-2].
2017-06-25 06:43:59 +03:00
Alexey Milovidov
275a1d4651
Improvement [#CLICKHOUSE-3091].
2017-06-25 05:48:38 +03:00
Alexey Milovidov
90a1ae1062
Fixed error in prev. revision [#CLICKHOUSE-3091].
2017-06-25 05:22:10 +03:00
Alexey Milovidov
8228acd3ea
Removed obsolete code that was relevant only in extremely rare case of transforming CollapsingMergeTree to ReplicatedCollapsingMergeTree through ATTACHing data parts and while some considerations about data order in table is important [#CLICKHOUSE-3091].
2017-06-25 03:51:51 +03:00
Alexey Milovidov
3c660fd65b
Fixed translation errors [#CLICKHOUSE-3091].
2017-06-25 03:17:08 +03:00
Alexey Milovidov
28e9e06992
Preparation: locking block number only after write to filesystem; removed obsolete code [#CLICKHOUSE-3091].
2017-06-25 03:01:10 +03:00
Alexey Milovidov
c7f35997f5
Removed unused variable [#CLICKHOUSE-2].
2017-06-25 01:36:43 +03:00
proller
4db8d09de9
Reorganize includes. part 1 ( #921 )
...
* Make libunwind optional. Allow use custom libcctz
* fix
* Fix
* fix
* Update BaseDaemon.cpp
* Update CMakeLists.txt
* Reorganize includes. part 1
* Update dbms_include.cmake
* Reorganize includes. part 2
* Reorganize includes. part 3
* dbms/src/Common/ThreadPool -> libs/libcommon
* Reorganize includes. part 4
* Fix print_include_directories
* Update thread_creation_latency.cpp
* Update StringRef.h
2017-06-23 23:22:35 +03:00
Alexey Milovidov
061c214c16
Fixed error [#CLICKHOUSE-3074].
2017-06-23 09:57:55 +03:00
Alexey Milovidov
425156e32c
Raised up limit on max size of parts to merge [#CLICKHOUSE-3091].
2017-06-22 21:29:40 +03:00
Alexey Milovidov
6eb06774d3
Addition to prev. revision [#CLICKHOUSE-3091].
2017-06-22 19:29:15 +03:00
Alexey Milovidov
c96719685c
Lowered max_delay_to_insert to one second, because higher values are non practical and could be harmful for usual failover logic in client apps [#CLICKHOUSE-3091].
2017-06-22 19:17:01 +03:00
Vitaliy Lyudvichenko
eff3588f28
Rename DROP COLUMN FROM PARTITION -> CLEAR COLUMN IN PARTITION. [#CLICKHOUSE-2896]
2017-06-22 16:18:19 +03:00
Vitaliy Lyudvichenko
53adb891b3
Add replicated DROP COLUMN FROM partition for optimistic case. [#CLICKHOUSE-2896]
2017-06-22 00:56:38 +03:00
Alexey Milovidov
7bdc87adf0
Loosened threshold on broken parts in filesystem [#CLICKHOUSE-3076].
2017-06-21 22:11:11 +03:00
Alexey Milovidov
5c322f5756
Loosened threshold on amount of replica inconsistency to refuse server startup [#CLICKHOUSE-3076].
2017-06-21 22:07:08 +03:00
Alexey Milovidov
6df5d556e5
Changed namespace [#DEVTOOLS-3381].
2017-06-21 11:35:38 +03:00
Alexey Milovidov
604de2bfeb
Better [#DEVTOOLS-3381].
2017-06-21 04:24:05 +03:00
Alexey Milovidov
05accbdcbe
Removed cyclic dependency [#DEVTOOLS-3381].
2017-06-19 23:06:35 +03:00
Vitaliy Lyudvichenko
63ac609e8b
Avoid overestimation of avg. column element size. [#CLICKHOUSE-3058]
2017-06-15 20:33:03 +03:00
alexey-milovidov
a669136ac2
Merge branch 'master' into CLICKHOUSE-5
2017-06-13 07:30:36 +03:00
Vitaliy Lyudvichenko
49eab81d9a
Fix: ReshardingWorker doesn't finish if ZooKeeper session expired. [#CLICKHOUSE-2]
2017-06-13 07:15:25 +03:00
Alexey Milovidov
43fedfecea
Get rid of INSERT ID [#CLICKHOUSE-31].
2017-06-13 07:13:20 +03:00
Alexey Milovidov
38885d00a3
Merge branch 'storage-startup-method'
2017-06-07 02:27:16 +03:00
Alexey Milovidov
9c5ddce06d
Get rid of .hpp files [#CLICKHOUSE-2].
2017-06-06 20:18:32 +03:00
Alexey Milovidov
9ebf0b633f
Added IStorage::startup method to avoid starting replication and merging threads too early during server startup [#CLICKHOUSE-2].
2017-06-06 20:06:14 +03:00
Alexey Milovidov
153693cd4c
Non significant modifications [#CLICKHOUSE-3].
2017-06-05 23:43:23 +03:00
Vitaliy Lyudvichenko
1e4906816c
Avoid SIGSEGV in SystemLog destructors. [#CLICKHOUSE-3023]
2017-06-05 17:25:10 +03:00
Alexey Milovidov
ba499585cd
Merge branch 'master' of github.com:yandex/ClickHouse
2017-06-02 18:54:43 +03:00
Alexey Milovidov
80080f1ad6
Small refinements [#CLICKHOUSE-2].
2017-06-02 18:54:39 +03:00
Vitaliy Lyudvichenko
3a2c1a195d
Add ZooKeeper session recovering and trashy integration test. [#CLICKHOUSE-5]
2017-06-01 13:49:20 +03:00
Vitaliy Lyudvichenko
209015574f
Workable version for CREATE DROP w/o ZK tests. [#CLICKHOUSE-5]
2017-06-01 13:49:20 +03:00
Vitaliy Lyudvichenko
1a4b0eb355
Remove tmp part directories on server start. [#CLICKHOUSE-3044]
2017-05-31 18:01:25 +03:00
Alexey Milovidov
fa64d14f26
More readable temporary part names [#CLICKHOUSE-2].
2017-05-26 03:47:06 +03:00
Alexey Milovidov
ed44304c49
Removed obsolete commented code [#CLICKHOUSE-2].
2017-05-26 03:43:35 +03:00
alexey-milovidov
c79ebc586d
Merge pull request #825 from yandex/CLICKHOUSE-2989
...
Separate dirs for different part sources, fix RC during replicated alters
2017-05-26 04:42:03 +04:00
Alexey Milovidov
d35a586af2
Fixed deadlock when using executable dictionary source with cache or complex_key_cache layout [#CLICKHOUSE-3039].
2017-05-25 23:59:14 +03:00
alexey-milovidov
72822714c3
Merge branch 'master' into CLICKHOUSE-2989
2017-05-25 21:04:37 +04:00
Vitaliy Lyudvichenko
1e29795cf9
Fix race condition between cleanup and fetch thread. [#CLICKHOUSE-2989]
...
It was happening during ALTERs.
TODO: stop fetches during ALTER.
2017-05-25 13:08:52 +03:00
