Alexey Milovidov
e86d0b4a74
Merge branch 'master' of github.com:yandex/ClickHouse
2017-08-10 00:07:10 +03:00
Alexey Milovidov
56a5bb53fa
Miscellaneous [#CLICKHOUSE-2].
2017-08-10 00:07:01 +03:00
Vitaliy Lyudvichenko
27174d9dd2
Avoid many checks of sending parts if sendings fail due to network error. [#CLICKHOUSE-2]
2017-08-09 23:57:42 +03:00
Vitaliy Lyudvichenko
00ce8c9603
Add ctime caching in MergeTree's cleaner. [#CLICKHOUSE-3173]
2017-08-08 03:28:43 +03:00
Vitaliy Lyudvichenko
10c14cfccb
Small code enhancements according to clang-tidy. [#CLICKHOUSE-2931]
2017-08-04 17:35:36 +03:00
Nikolai Kochetov
0c15b2c6fc
fixed number of rows to read from first granula in mergetree
2017-08-03 15:58:04 +03:00
alexey-milovidov
ae8783aee3
Merge pull request #1045 from prog8/nocompression
...
Added compression NONE
2017-08-01 23:07:52 +03:00
alexey-milovidov
2ae6f1ebdb
Update MergeTreeDataWriter.cpp
2017-08-01 23:07:16 +03:00
Pawel Rog
d6a2056851
Applied changes requested by Alexey
2017-08-01 16:48:24 +02:00
Pawel Rog
9f799820d9
Applied changes requested by Alexey
2017-08-01 10:12:15 +02:00
Alexey Milovidov
6578fb2645
Fixed error [#CLICKHOUSE-3191].
2017-08-01 00:39:24 +03:00
alexey-milovidov
d174ebc5c9
Update MergeTreeDataWriter.cpp
2017-07-31 20:44:58 +03:00
alexey-milovidov
4877aa394c
Update MergeTreeData.cpp
2017-07-31 20:44:41 +03:00
Pawel Rog
3a6c444996
Merge branch 'master' into nocompression
2017-07-31 13:36:58 +02:00
Pawel Rog
e1ab721d3d
Use compression method from configs
2017-07-31 13:12:28 +02:00
Pawel Rog
a0cc54457e
Added compression NONE
2017-07-31 12:44:45 +02:00
Nikolai Kochetov
78991b25ee
fixed preferred_max_column_in_block_size_bytes disabling
2017-07-30 01:24:57 +03:00
Alexey Milovidov
8fb452e99a
Fixed wrong log message when removing temporary part after merge, that is not byte-identical than same part on replicas; added profile event; added fail-safe logic [#MTRSADMIN-3175].
2017-07-29 00:25:24 +03:00
proller
700d4e0904
Fix boost 1.64 and gcc7+ compile errors ( #1040 )
...
* PerformanceTest: use getMultiple*FromConfig, fix debug helpers
* Missing file
* clnag-format of Split ComplexKeyCacheDictionary
* Requested changes
* Fix boost 1.64 and gcc7+ compile errors
* More fixes
2017-07-28 23:41:51 +03:00
alexey-milovidov
6917eb3153
Update MergeTreeBaseBlockInputStream.cpp
2017-07-28 22:29:40 +03:00
Nikolai Kochetov
b714d2b514
fixed column size calculation for Array column in MergeTreeReader::readRows
2017-07-28 22:29:40 +03:00
Nikolai Kochetov
d9f4f9b0f9
changed skipping rows for Array columns in MergeTreeBaseBlockInputStream
2017-07-28 22:29:40 +03:00
Alexey Milovidov
dd1f54bce2
Changed Poco::RWLock to std::shared_lock [#APRILDEV-91].
2017-07-28 20:34:02 +03:00
Vitaliy Lyudvichenko
515271544d
Add detail comment. [#CLICKHOUSE-3173]
2017-07-27 11:19:20 +03:00
Vitaliy Lyudvichenko
445608afec
Fix an iterator pointing after std::end(). [#CLICKHOUSE-3173]
2017-07-27 11:19:20 +03:00
alexey-milovidov
73c2a1483d
Update ReplicatedMergeTreeCleanupThread.cpp
2017-07-27 11:19:20 +03:00
alexey-milovidov
dcd7354fb2
Update ReplicatedMergeTreeCleanupThread.cpp
2017-07-27 11:19:20 +03:00
alexey-milovidov
d6750880e0
Update ReplicatedMergeTreeCleanupThread.cpp
2017-07-27 11:19:20 +03:00
Vitaliy Lyudvichenko
cd5bb4d921
Add replicated_deduplication_window_seconds merge_tree parameter. [#CLICKHOUSE-3173]
2017-07-27 11:19:20 +03:00
Vitaliy Lyudvichenko
dcc4d3bff0
Avoid zxid overflow problem when cleaning old blocks used in the deduplication. [#CLICKHOUSE-3173]
2017-07-27 11:19:20 +03:00
Nikolai Kochetov
01a004050d
fixed MergeTreeBlockSizePredictor updating from empy block while reading
2017-07-26 20:03:09 +03:00
Alexey Milovidov
4a4e863f9d
Fixed warning on clang 6 [#CLICKHOUSE-2].
2017-07-25 01:44:13 +03:00
Nikolai Kochetov
fcd50e3629
fixed comment
2017-07-24 18:06:32 +03:00
Nikolai Kochetov
b55dcc9f93
cnanged comments; fixed 00443_preferred_block_size_bytes test
2017-07-24 16:59:03 +03:00
Nikolai Kochetov
bc0921a64e
added comments
2017-07-21 21:02:02 +03:00
Nikolai Kochetov
c64b65ee29
ignore preferred_max_column_in_block_size_bytes if zero [#CLICKHOUSE-3170]
2017-07-21 20:45:51 +03:00
Nikolai Kochetov
755d16b22d
set min block size as index granularity for max_block_size limit during reading [#CLICKHOUSE-3170]
2017-07-21 14:55:33 +03:00
Alexey Milovidov
c7b1e92522
Fixed error after merge [#CLICKHOUSE-3150].
2017-07-21 09:40:05 +03:00
alexey-milovidov
d4d266fa18
ColumnConst unification ( #1011 )
...
* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].
* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].
* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].
* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].
* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].
* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].
* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].
* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].
* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].
* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].
* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].
* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].
* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].
* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].
* Fixed error in ColumnArray::replicateGeneric [#CLICKHOUSE-3150].
* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].
* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].
* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].
* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].
* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].
* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].
* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].
* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].
* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].
* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].
* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].
* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].
* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].
2017-07-21 09:35:58 +03:00
alexey-milovidov
1a1112d8bb
Update MergeTreeBlockReadUtils.h
2017-07-20 23:45:23 +03:00
Nikolai Kochetov
0b7439469d
fixed MergeTreeBlockSizePredictor initialization for sample block
2017-07-20 23:45:23 +03:00
Nikolai Kochetov
00ae32c946
renamed approximate_number_of_rows_in_part to number_of_rows_in_part
2017-07-20 23:45:23 +03:00
Nikolai Kochetov
e612152584
changed initialization in MergeTreeBlockSizePredictor
2017-07-20 23:45:23 +03:00
Nikolai Kochetov
2afd84c40c
fixed is_reading_finished status assignment in MergeTreeRangeReader [#CLICKHOUSE-3161]
2017-07-19 19:39:18 +03:00
Nikolai Kochetov
b5e94ede62
fixed is_reading_finished status assignment in MergeTreeRangeReader [#CLICKHOUSE-3161]
2017-07-19 16:44:46 +03:00
Alexey Milovidov
6e6e9f9d38
Removed useless variable [#CLICKHOUSE-3161].
2017-07-19 04:05:39 +03:00
Nikolai Kochetov
7e293df9af
merged with master
2017-07-17 22:48:22 +03:00
Nikolai Kochetov
eae94edd6b
fixed vertical merge for ReplacingSortedBlockInputStream [#CLICKHOUSE-3131]
2017-07-17 22:40:30 +03:00
Alexey Milovidov
9406b6a5aa
Get rid of ASTSet, because it is not really an AST and it lead to bugs [#CLICKHOUSE-3148].
2017-07-15 06:48:36 +03:00
Nikolai Kochetov
a13423e552
added vertical merge for ReplacingSortedBlockInputStream
2017-07-14 21:17:23 +03:00
Alexey Milovidov
1a0f5dd7cf
Getting rid of ASTSet (incomplete) [#CLICKHOUSE-2]
2017-07-14 03:33:37 +03:00
Nikolai Kochetov
283810e8ab
merged with master
2017-07-13 23:59:34 +03:00
Nikolai Kochetov
75be0b81d8
removed typeid_cast from IColumn.h
2017-07-13 23:58:19 +03:00
Nikolai Kochetov
210f14aaba
renamed IDataType::updateAvgValueSizeHints to IDataType::updateAvgValueSizeHint
2017-07-13 21:54:17 +03:00
Nikolai Kochetov
f3f217b40a
fixed tests
2017-07-13 21:45:09 +03:00
Nikolai Kochetov
546012079c
fixed reading with zero max block size setting
2017-07-13 21:45:09 +03:00
Nikolai Kochetov
e2d032b883
fixed reading with zero max block size setting
2017-07-13 21:45:09 +03:00
Nikolai Kochetov
8ea812e669
better comment for MergeTreeRangeReader
2017-07-13 21:45:09 +03:00
Nikolai Kochetov
f232afd2f8
round number of rows to nearest mark in MergeTreeBaseBlockInputStream
2017-07-13 21:45:09 +03:00
Nikolai Kochetov
f1775af3f0
deleted commented code
2017-07-13 21:45:09 +03:00
Nikolai Kochetov
30ed774bf7
refactoring and more comments in MergeTreeBaseBlockInputStream [#CLICKHOUSE-3065]
2017-07-13 21:45:09 +03:00
Nikolai Kochetov
89386394aa
refactoring and more comments in MergeTreeBaseBlockInputStream [#CLICKHOUSE-3065]
2017-07-13 21:45:09 +03:00
Nikolai Kochetov
8c65d6013e
added preferred_max_column_in_block_size_bytes setting, added filtration ratio prediction [#CLICKHOUSE-3065]
2017-07-13 21:45:09 +03:00
Nikolai Kochetov
4470afc9a9
added estimateNumRowsMax to MergeTreeBlockSizePredictor based on max column size
2017-07-13 21:45:09 +03:00
Nikolai Kochetov
6b29b3bd47
round number of rows to next mark in MergeTreeBaseBlockInputStream
2017-07-13 21:45:09 +03:00
Nikolai Kochetov
d14f64af7b
delected commented logging code
2017-07-13 21:45:09 +03:00
Nikolai Kochetov
586dff9126
delay reading for granulas if prewhere column is false during partial granula reading
2017-07-13 21:45:09 +03:00
Nikolai Kochetov
86680f5513
fixed partial granula reading for const columns in prewhere
2017-07-13 21:45:09 +03:00
Nikolai Kochetov
609711b20c
partial part reading with prewhere
2017-07-13 21:45:09 +03:00
Nikolai Kochetov
c6d5ef6d30
added MergeTreeRangeReader
2017-07-13 21:45:09 +03:00
Nikolai Kochetov
985e449bd3
added IDataType::updateAvgValueSizeHints; renamed PODArray::allocated_size to PODArray::allocated_bytes; renamed renamed IColumn::allocatedSize to IColumn::allocatedBytes; removed IColumn.h from IDataType.h
2017-07-13 19:49:09 +03:00
Alexey Milovidov
de75fa000b
Using lexer (incomplete) [#CLICKHOUSE-2].
2017-07-13 08:59:01 +03:00
Alexey Milovidov
83c8043193
Using lexer (incomplete) [#CLICKHOUSE-2].
2017-07-13 08:59:01 +03:00
Alexey Milovidov
4b1eeee1a3
Using lexer (incomplete) [#CLICKHOUSE-2].
2017-07-13 08:59:01 +03:00
Alexey Zatelepin
58d541b740
slightly more robust ReplicatedMergeTreePartCheckThread::TemporarilyStop [#CLICKHOUSE-3074]
2017-07-12 20:09:44 +03:00
Vitaliy Lyudvichenko
b843ea5b3f
Resolves #964 . Fixed input_format_skip_unknown_fields for negative numbers. [#CLICKHOUSE-3]
2017-07-11 21:36:51 +03:00
Alexey Zatelepin
4a98d83a3b
introduce replicated_can_become_leader merge_tree setting [#CLICKHOUSE-3109]
2017-07-11 20:44:35 +03:00
Alexey Milovidov
2bd84600d1
Miscellaneous [#CLICKHOUSE-2].
2017-07-11 04:20:40 +03:00
alexey-milovidov
b64b243a73
Merge pull request #945 from YiuRULE/master
...
Implement natively uuid type in Clickhouse
2017-07-10 07:00:45 +03:00
Alexey Zatelepin
89939a685a
Stream row sources from disk for vertical merge [#CLICKHOUSE-3118]
2017-07-09 21:57:06 +03:00
Marek Vavruša
ff73394124
PKCondition: infer index use with pk subexpression
...
By default only constraints explicitly matching
primary key expression (or expression wrapped in
a monotonic function) are eligible for part and
range selection. So for example, if index is:
(toStartOfHour(dt), UserID)
Then a query such as this resorts to full scan:
SELECT count() FROM t WHERE dt = now()
Intuitively, only parts with toStartOfHour(now())
could be selected, but it is less trivial to prove.
The primary key currently can be wrapped in a chain
of monotonic functions, so following would work:
toStartOfHour(dt) = toStartOfHour(now()) AND dt = now()
It must be however explicitly stated, if we wanted
to infer that we’d have to know the inverse function,
and prove that the inverse function is monotonic
on given interval. This is not practical as
there is no inverse function that for example undos
rounding, it isn’t strictly monotonic.
There are however functions that don’t transform
output range and preserve monotonicity on the
complete input range, such as rounding or casts
to a same or wider numeric type. This eliminates
the need to find inverse function, as no check for monotonicity over arbitrary interval is needed,
and thus makes this optimisation possible.
2017-07-09 17:13:51 +03:00
Alexey Milovidov
032d12d431
Fixed translation error [#CLICKHOUSE-2].
2017-07-04 15:46:31 +03:00
Guillaume Tassery
cba9815981
Change Uuid to UUID datatype and make UInt128 a generic type
2017-07-04 12:42:53 +02:00
Guillaume Tassery
525851ec68
sync master
2017-06-30 22:21:42 +02:00
Guillaume Tassery
1231952fca
Implement UUID type and toUuid function
2017-06-30 22:01:27 +02:00
Alexey Zatelepin
9de4d8facb
allow loading missing setting from config [#CLICKHOUSE-3091]
2017-06-28 19:56:35 +03:00
Alexey Zatelepin
7cf413dea8
Revert "Raised up limit on max size of parts to merge [#CLICKHOUSE-3091]."
...
This reverts commit 425156e32c
2017-06-28 15:32:52 +03:00
Alexey Milovidov
a48a21f593
Merge branch 'master' into CLICKHOUSE-2896
2017-06-26 11:54:58 +03:00
Alexey Milovidov
8f145da1e8
Preparation [#CLICKHOUSE-2].
2017-06-25 06:43:59 +03:00
Alexey Milovidov
275a1d4651
Improvement [#CLICKHOUSE-3091].
2017-06-25 05:48:38 +03:00
Alexey Milovidov
90a1ae1062
Fixed error in prev. revision [#CLICKHOUSE-3091].
2017-06-25 05:22:10 +03:00
Alexey Milovidov
8228acd3ea
Removed obsolete code that was relevant only in extremely rare case of transforming CollapsingMergeTree to ReplicatedCollapsingMergeTree through ATTACHing data parts and while some considerations about data order in table is important [#CLICKHOUSE-3091].
2017-06-25 03:51:51 +03:00
Alexey Milovidov
3c660fd65b
Fixed translation errors [#CLICKHOUSE-3091].
2017-06-25 03:17:08 +03:00
Alexey Milovidov
28e9e06992
Preparation: locking block number only after write to filesystem; removed obsolete code [#CLICKHOUSE-3091].
2017-06-25 03:01:10 +03:00
Alexey Milovidov
c7f35997f5
Removed unused variable [#CLICKHOUSE-2].
2017-06-25 01:36:43 +03:00
proller
4db8d09de9
Reorganize includes. part 1 ( #921 )
...
* Make libunwind optional. Allow use custom libcctz
* fix
* Fix
* fix
* Update BaseDaemon.cpp
* Update CMakeLists.txt
* Reorganize includes. part 1
* Update dbms_include.cmake
* Reorganize includes. part 2
* Reorganize includes. part 3
* dbms/src/Common/ThreadPool -> libs/libcommon
* Reorganize includes. part 4
* Fix print_include_directories
* Update thread_creation_latency.cpp
* Update StringRef.h
2017-06-23 23:22:35 +03:00
Alexey Milovidov
061c214c16
Fixed error [#CLICKHOUSE-3074].
2017-06-23 09:57:55 +03:00
Alexey Milovidov
425156e32c
Raised up limit on max size of parts to merge [#CLICKHOUSE-3091].
2017-06-22 21:29:40 +03:00
Alexey Milovidov
6eb06774d3
Addition to prev. revision [#CLICKHOUSE-3091].
2017-06-22 19:29:15 +03:00
Alexey Milovidov
c96719685c
Lowered max_delay_to_insert to one second, because higher values are non practical and could be harmful for usual failover logic in client apps [#CLICKHOUSE-3091].
2017-06-22 19:17:01 +03:00
