Alexey Milovidov
6578fb2645
Fixed error [#CLICKHOUSE-3191].
2017-08-01 00:39:24 +03:00
alexey-milovidov
d174ebc5c9
Update MergeTreeDataWriter.cpp
2017-07-31 20:44:58 +03:00
alexey-milovidov
4877aa394c
Update MergeTreeData.cpp
2017-07-31 20:44:41 +03:00
alexey-milovidov
fdb7e080be
Update CompressedWriteBuffer.cpp
2017-07-31 20:43:33 +03:00
alexey-milovidov
50244880cc
Update CompressedReadBufferBase.cpp
2017-07-31 20:42:23 +03:00
alexey-milovidov
3d2ec763ed
Update CompressedReadBufferBase.cpp
2017-07-31 20:40:56 +03:00
Pawel Rog
3a6c444996
Merge branch 'master' into nocompression
2017-07-31 13:36:58 +02:00
Pawel Rog
e1ab721d3d
Use compression method from configs
2017-07-31 13:12:28 +02:00
Pawel Rog
a0cc54457e
Added compression NONE
2017-07-31 12:44:45 +02:00
Alexey Milovidov
4e76c96381
Temporarily disabled "preferred_max_column_in_block_size_bytes" option [#CLICKHOUSE-3188].
2017-07-31 03:50:06 +03:00
Alexey Milovidov
306070d2cc
Fixed error in one of prev. revisions [#CLICKHOUSE-3150].
2017-07-30 22:47:32 +03:00
proller
460d66c68d
Split FunctionsArray.cpp register ( #1043 )
...
* PerformanceTest: use getMultiple*FromConfig, fix debug helpers
* Missing file
* gcc7 fixes
* clnag-format of Split ComplexKeyCacheDictionary
* Requested changes
* Fix boost 1.64 and gcc7+ compile errors
* More fixes
* Fix arm build
* Fix nothrow
* Better arm detection
* Split FunctionsArray.cpp register
2017-07-30 15:31:58 +03:00
Nikolai Kochetov
78991b25ee
fixed preferred_max_column_in_block_size_bytes disabling
2017-07-30 01:24:57 +03:00
Alexey Milovidov
8fb452e99a
Fixed wrong log message when removing temporary part after merge, that is not byte-identical than same part on replicas; added profile event; added fail-safe logic [#MTRSADMIN-3175].
2017-07-29 00:25:24 +03:00
proller
700d4e0904
Fix boost 1.64 and gcc7+ compile errors ( #1040 )
...
* PerformanceTest: use getMultiple*FromConfig, fix debug helpers
* Missing file
* clnag-format of Split ComplexKeyCacheDictionary
* Requested changes
* Fix boost 1.64 and gcc7+ compile errors
* More fixes
2017-07-28 23:41:51 +03:00
alexey-milovidov
6917eb3153
Update MergeTreeBaseBlockInputStream.cpp
2017-07-28 22:29:40 +03:00
Nikolai Kochetov
b714d2b514
fixed column size calculation for Array column in MergeTreeReader::readRows
2017-07-28 22:29:40 +03:00
Nikolai Kochetov
d9f4f9b0f9
changed skipping rows for Array columns in MergeTreeBaseBlockInputStream
2017-07-28 22:29:40 +03:00
Alexey Milovidov
4c89570988
Don't limit the size of internal queries (this is for cache dictionaries with ClickHouse source) [#CLICKHOUSE-2].
2017-07-28 21:43:40 +03:00
Alexey Milovidov
dd1f54bce2
Changed Poco::RWLock to std::shared_lock [#APRILDEV-91].
2017-07-28 20:34:02 +03:00
Alexey Milovidov
3502d36d7d
Fixed build [#CLICKHOUSE-2].
2017-07-28 02:23:13 +03:00
Alexey Milovidov
e4af70149f
Fixed build [#CLICKHOUSE-2].
2017-07-28 00:27:10 +03:00
Alexey Milovidov
acf9c2fbd5
Miscellaneous [#CLICKHOUSE-2].
2017-07-28 00:05:33 +03:00
proller
9eb6e5b19c
clang-format of Split ComplexKeyCacheDictionary ( #1035 )
...
* clnag-format of Split ComplexKeyCacheDictionary
* Requested changes
2017-07-27 23:39:40 +03:00
Alexey Milovidov
2c815356c1
Fixed regression in parsing of CAST [#CLICKHOUSE-3180].
2017-07-27 22:25:52 +03:00
proller
3dbde50daa
Split ComplexKeyCacheDictionary to use less then 1.5G ram when compile ( #1034 )
...
* Split FunctionsArithmetic.cpp to generated functions
* Use ccache if found
* Do not use ccache if ccache defined in CMAKE_CXX_COMPILER_LAUNCHER
* check_include.sh: print memory usage
* Try split CacheDictionary source
* Split ok
* wip
* wip
* wip
* wip
* wip
* wip
2017-07-27 22:05:55 +03:00
Nikolai Kochetov
c88e8a79cc
merged with master
2017-07-27 18:43:46 +03:00
Nikolai Kochetov
2f8f199d08
separated connection pool creation from StorageDistributedDirectoryMonitor in StorageDistributed; fixed bugs from review [#CLICKHOUSE-3033]
2017-07-27 18:24:39 +03:00
Vitaliy Lyudvichenko
515271544d
Add detail comment. [#CLICKHOUSE-3173]
2017-07-27 11:19:20 +03:00
Vitaliy Lyudvichenko
445608afec
Fix an iterator pointing after std::end(). [#CLICKHOUSE-3173]
2017-07-27 11:19:20 +03:00
alexey-milovidov
73c2a1483d
Update ReplicatedMergeTreeCleanupThread.cpp
2017-07-27 11:19:20 +03:00
alexey-milovidov
dcd7354fb2
Update ReplicatedMergeTreeCleanupThread.cpp
2017-07-27 11:19:20 +03:00
alexey-milovidov
d6750880e0
Update ReplicatedMergeTreeCleanupThread.cpp
2017-07-27 11:19:20 +03:00
Vitaliy Lyudvichenko
cd5bb4d921
Add replicated_deduplication_window_seconds merge_tree parameter. [#CLICKHOUSE-3173]
2017-07-27 11:19:20 +03:00
Vitaliy Lyudvichenko
dcc4d3bff0
Avoid zxid overflow problem when cleaning old blocks used in the deduplication. [#CLICKHOUSE-3173]
2017-07-27 11:19:20 +03:00
proller
ff7f3af238
Split FunctionsArithmetic.cpp to generated functions
2017-07-26 21:32:35 +03:00
Nikolai Kochetov
01a004050d
fixed MergeTreeBlockSizePredictor updating from empy block while reading
2017-07-26 20:03:09 +03:00
Alexey Milovidov
cd4fef7767
Miscellaneous [#CLICKHOUSE-2].
2017-07-26 08:55:59 +03:00
Alexey Milovidov
3dcb2e6aed
Fixed test [#CLICKHOUSE-2].
2017-07-26 04:21:19 +03:00
Alexey Milovidov
de1f4f4c5b
Merge branch 'master' of github.com:yandex/ClickHouse
2017-07-26 00:07:13 +03:00
Alexey Milovidov
d57f5b81a4
Removed MaterializedView inheritance of View, as they are not related. Fixed error with Views [#CLICKHOUSE-3175].
2017-07-26 00:07:05 +03:00
Alexey Zatelepin
64fa3ebc58
correctly choose number of merge threads for external aggregation [#CLICKHOUSE-2]
2017-07-25 22:43:23 +03:00
Nikolai Kochetov
b3157aebb2
added synchronous insert into distributed table [#CLICKHOUSE-3033]
2017-07-25 22:42:36 +03:00
Alexey Milovidov
ec676baee7
Fixed error after merge [#CLICKHOUSE-2].
2017-07-25 22:07:07 +03:00
Alexey Milovidov
6851a7754a
Fixed linkage [#CLICKHOUSE-2].
2017-07-25 21:32:52 +03:00
Alexey Milovidov
d20c411b0d
Fixed linkage [#CLICKHOUSE-2].
2017-07-25 21:19:22 +03:00
Alexey Zatelepin
2230bef68a
reset resulting bucket_num if source blocks are from different buckets [#CLICKHOUSE-3133]
2017-07-25 20:56:09 +03:00
alexey-milovidov
b831d88162
Update ArenaAllocator.h
2017-07-25 20:06:24 +03:00
Vitaliy Lyudvichenko
870b1aefcb
Get rid of of PODArray copypaste. [#CLICKHOUSE-3084]
2017-07-25 20:04:10 +03:00
Vitaliy Lyudvichenko
ae3b1d8d3a
Fix clang compilation. [#CLICKHOUSE-3084]
2017-07-25 20:04:10 +03:00
Vitaliy Lyudvichenko
fefb4dad67
Get rid of old groupArray() implementations. [#CLICKHOUSE-3084]
2017-07-25 20:04:10 +03:00
Vitaliy Lyudvichenko
792faaa2db
Add adaptive allocator. [#CLICKHOUSE-3084]
2017-07-25 20:04:10 +03:00
Vitaliy Lyudvichenko
03e5bf9471
Add support of parameters in aggregate function constructors. [#CLICKHOUSE-3084]
2017-07-25 20:04:10 +03:00
Vitaliy Lyudvichenko
c3a15da3f8
Add support of limit parameter into groupArray. [#CLICKHOUSE-3084]
2017-07-25 20:04:10 +03:00
Vitaliy Lyudvichenko
c2cc8d6147
Add several groupArray implementations. [#CLICKHOUSE-3084]
2017-07-25 20:04:10 +03:00
Guillaume Tassery
03bbe9938a
Function for generate an uuid v4 ( #1008 )
...
* Fix compilation error when include FunctionsRandom.h twice
* Implement function for generate UUID V4
* remove isDeterministicInScopeOfQuery for generateUUIDV4
* Uniformize RandX Implementation
* rename generateUUIDV4 into generateUUIDv4
* Modify random function for making it more generic
2017-07-25 20:00:38 +03:00
Alexey Zatelepin
18672204e5
fix handling of overflow data during external aggregation [#CLICKHOUSE-3133]
2017-07-25 19:56:48 +03:00
Alexey Zatelepin
cd5dc7ca29
fix segfault when handling overflow blocks in MergingAggregatedMemoryEfficientBlockInputStream [#CLICKHOUSE-3133]
2017-07-25 19:56:48 +03:00
Alexey Milovidov
f153d5f5db
Additions to prev. revision [#CLICKHOUSE-2].
2017-07-25 01:53:44 +03:00
Alexey Milovidov
b4c610a8e7
Fixed errors [#CLICKHOUSE-2].
2017-07-25 01:44:13 +03:00
Alexey Milovidov
188ee349bd
Fixed errors [#CLICKHOUSE-2].
2017-07-25 01:44:13 +03:00
Alexey Milovidov
a6df2e7151
Added support for non-constant and negative offset and size for substring function (continued) [#CLICKHOUSE-2].
2017-07-25 01:44:13 +03:00
Alexey Milovidov
487278e2f5
Added support for non-constant and negative offset and size for substring function (continued) [#CLICKHOUSE-2].
2017-07-25 01:44:13 +03:00
Alexey Milovidov
3144cbc9c1
Added support for non-constant and negative offset and size for substring function (continued) [#CLICKHOUSE-2].
2017-07-25 01:44:13 +03:00
Alexey Milovidov
94768fe9bf
Added support for non-constant and negative offset and size for substring function (continued) [#CLICKHOUSE-2].
2017-07-25 01:44:13 +03:00
Alexey Milovidov
52b5bae62f
Added support for non-constant and negative offset and size for substring function (continued) [#CLICKHOUSE-2].
2017-07-25 01:44:13 +03:00
Alexey Milovidov
30ff4a78a6
Added support for non-constant and negative offset and size for substring function (continued) [#CLICKHOUSE-2].
2017-07-25 01:44:13 +03:00
Alexey Milovidov
14de659c99
Added support for non-constant offset and size for substring function [#CLICKHOUSE-2].
2017-07-25 01:44:13 +03:00
Alexey Milovidov
4a4e863f9d
Fixed warning on clang 6 [#CLICKHOUSE-2].
2017-07-25 01:44:13 +03:00
Alexey Milovidov
0c8e082954
Using GatherUtils (experimental) [#CLICKHOUSE-2].
2017-07-25 01:44:13 +03:00
Alexey Milovidov
366ad1595e
Fixed warning on clang 6 [#CLICKHOUSE-2].
2017-07-25 01:44:13 +03:00
Alexey Milovidov
1557fa2e8d
Removed "dispatchFor" templates, because gcc 6 cannot compile them (only clang is capable) [#CLICKHOUSE-2].
2017-07-25 01:44:13 +03:00
Alexey Milovidov
ffbd2f46f9
GatherUtils (experimental) [#CLICKHOUSE-2].
2017-07-25 01:44:13 +03:00
Alexey Milovidov
9540a2da90
GatherUtils (experimental) [#CLICKHOUSE-2].
2017-07-25 01:44:13 +03:00
Alexey Milovidov
bfdc108a08
Cleaning old code [#CLICKHOUSE-2].
2017-07-25 01:44:13 +03:00
Alexey Milovidov
ee81d53bd5
Functions simplification [#CLICKHOUSE-2].
2017-07-25 01:44:13 +03:00
alexey-milovidov
bbe70e8cfa
Update FunctionsStringSearch.cpp
2017-07-24 20:36:16 +03:00
alexey-milovidov
d3159ed2c5
Merge pull request #1015 from yandex/fix-performance-after-changing-reading
...
Fix performance after changing reading
2017-07-24 18:07:57 +03:00
Nikolai Kochetov
fcd50e3629
fixed comment
2017-07-24 18:06:32 +03:00
proller
eba5ad73ec
clean
2017-07-24 18:04:49 +03:00
proller
57b9bcd354
Fix missing includes
2017-07-24 18:04:49 +03:00
Vitaliy Lyudvichenko
bc9ee54985
Don't throw exception in case of clearing empty part. [#CLICKHOUSE-2896]
2017-07-24 18:04:17 +03:00
Nikolai Kochetov
b55dcc9f93
cnanged comments; fixed 00443_preferred_block_size_bytes test
2017-07-24 16:59:03 +03:00
Alexey Milovidov
efaa44bea8
Attempt to improve performance [#CLICKHOUSE-3].
2017-07-23 09:53:28 +03:00
Alexey Milovidov
a76061e1a9
Fixed MAC address related functions; added test [#CLICKHOUSE-3].
2017-07-23 09:13:57 +03:00
Alexey Milovidov
1e8547b3f6
Fixed regression [#CLICKHOUSE-3148].
2017-07-21 23:59:01 +03:00
Alexey Milovidov
31d8ba3b34
Fixed error after merge [#CLICKHOUSE-3150].
2017-07-21 23:20:22 +03:00
Nikolai Kochetov
10bbededb0
merged with master
2017-07-21 21:10:14 +03:00
Nikolai Kochetov
bc0921a64e
added comments
2017-07-21 21:02:02 +03:00
alexey-milovidov
6d47981e86
Update Collator.cpp
2017-07-21 20:59:39 +03:00
Vladimir Kolobaev
1b8a062e7b
Issue #1000 - Returned some remote code in Collator.app
2017-07-21 20:59:39 +03:00
Vladimir Kolobaev
e85cd44e48
Issue #1000 - remove 'diagnostic ignored -Wunused-private-field' in Collator.app
2017-07-21 20:59:39 +03:00
Alexey Zatelepin
c939d2f96e
put Distributed inserts with different block structure into different batches [#CLICKHOUSE-3163]
2017-07-21 20:55:38 +03:00
Nikolai Kochetov
ca1957ab82
ignore preferred_max_column_in_block_size_bytes if zero [#CLICKHOUSE-3170]
2017-07-21 20:54:09 +03:00
Nikolai Kochetov
cc349b410e
set min block size as index granularity for max_block_size limit during reading [#CLICKHOUSE-3170]
2017-07-21 20:54:09 +03:00
Nikolai Kochetov
5477e7c1f9
added date and time to number functions [#CLICKHOUSE-3132]
2017-07-21 20:51:08 +03:00
Nikolai Kochetov
c64b65ee29
ignore preferred_max_column_in_block_size_bytes if zero [#CLICKHOUSE-3170]
2017-07-21 20:45:51 +03:00
Nikolai Kochetov
755d16b22d
set min block size as index granularity for max_block_size limit during reading [#CLICKHOUSE-3170]
2017-07-21 14:55:33 +03:00
Alexey Milovidov
3a0bf8e642
Fixed error after merge [#CLICKHOUSE-3150].
2017-07-21 09:55:33 +03:00
Alexey Milovidov
e1cc8289b2
Fixed error after merge [#CLICKHOUSE-3150].
2017-07-21 09:46:50 +03:00
Alexey Milovidov
c7b1e92522
Fixed error after merge [#CLICKHOUSE-3150].
2017-07-21 09:40:05 +03:00
alexey-milovidov
d4d266fa18
ColumnConst unification ( #1011 )
...
* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].
* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].
* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].
* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].
* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].
* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].
* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].
* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].
* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].
* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].
* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].
* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].
* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].
* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].
* Fixed error in ColumnArray::replicateGeneric [#CLICKHOUSE-3150].
* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].
* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].
* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].
* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].
* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].
* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].
* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].
* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].
* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].
* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].
* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].
* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].
* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].
2017-07-21 09:35:58 +03:00
alexey-milovidov
1a1112d8bb
Update MergeTreeBlockReadUtils.h
2017-07-20 23:45:23 +03:00
Nikolai Kochetov
0b7439469d
fixed MergeTreeBlockSizePredictor initialization for sample block
2017-07-20 23:45:23 +03:00
Nikolai Kochetov
00ae32c946
renamed approximate_number_of_rows_in_part to number_of_rows_in_part
2017-07-20 23:45:23 +03:00
Nikolai Kochetov
e612152584
changed initialization in MergeTreeBlockSizePredictor
2017-07-20 23:45:23 +03:00
Vladimir Chebotarev
7254478fde
functions: Remote mode for hasColumnInTable. [#CHEBOTAREV-9]
2017-07-20 17:08:42 +03:00
Nikolai Kochetov
2afd84c40c
fixed is_reading_finished status assignment in MergeTreeRangeReader [#CLICKHOUSE-3161]
2017-07-19 19:39:18 +03:00
Guillaume Tassery
57b4066da8
Improve performance for deserialization/serialization of UUID type
2017-07-19 19:26:37 +03:00
proller
3f9cdfe464
PerformanceTest: Add memory and threads info ( #1004 )
...
* Cmake: clear test variables with CMakePushCheckState (it will fix macos build issue)
* Add some perf tests
* Fix PerformanceTest, fix tests
* Add nyc-taxi queries
* Update PerformanceTest.cpp
* PerformanceTest: allow recursive .xml search
* Tests tune
* Move tests to dirs
* --recursive
* Add space
* PerformanceTest: Add memory and threads info
2017-07-19 18:18:47 +03:00
Nikolai Kochetov
b5e94ede62
fixed is_reading_finished status assignment in MergeTreeRangeReader [#CLICKHOUSE-3161]
2017-07-19 16:44:46 +03:00
Alexey Milovidov
6e6e9f9d38
Removed useless variable [#CLICKHOUSE-3161].
2017-07-19 04:05:39 +03:00
proller
ae626281b6
PerformanceTest: add --recursive option. add more threaded/not threaded tests ( #997 )
...
* Cmake: clear test variables with CMakePushCheckState (it will fix macos build issue)
* Add some perf tests
* Fix PerformanceTest, fix tests
* Add nyc-taxi queries
* Update PerformanceTest.cpp
* PerformanceTest: allow recursive .xml search
* Tests tune
* Move tests to dirs
* --recursive
* Add space
2017-07-18 23:41:24 +03:00
Alexey Zatelepin
4866a37a2b
fix selecting max_threads for MergingAggregatedBlockInputStream [#CLICKHOUSE-3156]
2017-07-18 23:40:23 +03:00
alexey-milovidov
25288ac4d5
Update PerformanceTest.cpp
2017-07-18 19:35:01 +03:00
proller
cd54ff0f82
Fix PerformanceTest, fix tests
2017-07-18 19:35:01 +03:00
Nikolai Kochetov
7e293df9af
merged with master
2017-07-17 22:48:22 +03:00
Nikolai Kochetov
39cb5e0e3e
fixed ColumnGathererStream finish reading condition
2017-07-17 22:43:09 +03:00
Nikolai Kochetov
eae94edd6b
fixed vertical merge for ReplacingSortedBlockInputStream [#CLICKHOUSE-3131]
2017-07-17 22:40:30 +03:00
Alexey Milovidov
295459e89e
Miscellaneous [#CLICKHOUSE-2].
2017-07-17 01:55:40 +03:00
Alexey Milovidov
bfc099ea5f
Miscellaneous [#CLICKHOUSE-2].
2017-07-17 01:41:15 +03:00
Alexey Milovidov
930c373309
Added synonims for few data types for compatibility with Tableau [#CLICKHOUSE-3141].
2017-07-16 08:09:47 +03:00
Alexey Milovidov
5f98f4b862
Addition to prev. revision [#CLICKHOUSE-3149].
2017-07-16 07:11:11 +03:00
Alexey Milovidov
5c3ca0c16e
Inverted dependency in DataTypeFactory [#CLICKHOUSE-3149].
2017-07-16 07:11:11 +03:00
Alexey Milovidov
17786c0e6d
Added missing file [#CLICKHOUSE-3148].
2017-07-15 07:06:51 +03:00
Alexey Milovidov
9406b6a5aa
Get rid of ASTSet, because it is not really an AST and it lead to bugs [#CLICKHOUSE-3148].
2017-07-15 06:48:36 +03:00
Alexey Milovidov
200a1e27e3
Merge branch 'master' into get-rid-of-ast-set
2017-07-15 03:23:20 +03:00
Vitaliy Lyudvichenko
c6eb11aad5
Use more suitable method hasRealColumn. [#CLICKHOUSE-2896]
2017-07-15 03:21:19 +03:00
Vitaliy Lyudvichenko
9ce4ff4c5b
Add check for unexisting column in CLEAR COLUMN. [#CLICKHOUSE-2896]
2017-07-15 03:21:19 +03:00
Alexey Zatelepin
b10e9d1f18
Vertical merge fix: don't forget to output last source_to_fully_copy [#CLICKHOUSE-3118]
2017-07-14 22:06:40 +03:00
Nikolai Kochetov
a13423e552
added vertical merge for ReplacingSortedBlockInputStream
2017-07-14 21:17:23 +03:00
Alexey Milovidov
9bb902c3ef
Allow to use database.table in right hand side of IN, JOIN [#CLICKHOUSE-3144].
2017-07-14 04:00:29 +03:00
Alexey Milovidov
4a5fb9eb7b
Miscellaneous [#CLICKHOUSE-2].
2017-07-14 03:34:00 +03:00
Alexey Milovidov
1a0f5dd7cf
Getting rid of ASTSet (incomplete) [#CLICKHOUSE-2]
2017-07-14 03:33:37 +03:00
alexey-milovidov
37985d3cb8
Merge pull request #981 from yandex/reduce-memory-consumprion-during-reading-column-string-from-native-block
...
Reduce memory consumprion during reading column string from native block
2017-07-14 00:01:50 +03:00
Nikolai Kochetov
283810e8ab
merged with master
2017-07-13 23:59:34 +03:00
Nikolai Kochetov
75be0b81d8
removed typeid_cast from IColumn.h
2017-07-13 23:58:19 +03:00
Maxim Fridental
82b53b712a
findClusterIndex: fix virtual method, improve formatting
2017-07-13 22:39:37 +03:00
Maxim Fridental
ff4df27cad
Improve implementation of findClusterIndex and findClusterValue, add test
2017-07-13 22:39:37 +03:00
Nikolai Kochetov
210f14aaba
renamed IDataType::updateAvgValueSizeHints to IDataType::updateAvgValueSizeHint
2017-07-13 21:54:17 +03:00
Nikolai Kochetov
f3f217b40a
fixed tests
2017-07-13 21:45:09 +03:00
Nikolai Kochetov
546012079c
fixed reading with zero max block size setting
2017-07-13 21:45:09 +03:00
Nikolai Kochetov
e2d032b883
fixed reading with zero max block size setting
2017-07-13 21:45:09 +03:00
Nikolai Kochetov
8ea812e669
better comment for MergeTreeRangeReader
2017-07-13 21:45:09 +03:00
Nikolai Kochetov
f232afd2f8
round number of rows to nearest mark in MergeTreeBaseBlockInputStream
2017-07-13 21:45:09 +03:00
Nikolai Kochetov
f1775af3f0
deleted commented code
2017-07-13 21:45:09 +03:00
Nikolai Kochetov
30ed774bf7
refactoring and more comments in MergeTreeBaseBlockInputStream [#CLICKHOUSE-3065]
2017-07-13 21:45:09 +03:00
Nikolai Kochetov
89386394aa
refactoring and more comments in MergeTreeBaseBlockInputStream [#CLICKHOUSE-3065]
2017-07-13 21:45:09 +03:00
Nikolai Kochetov
8c65d6013e
added preferred_max_column_in_block_size_bytes setting, added filtration ratio prediction [#CLICKHOUSE-3065]
2017-07-13 21:45:09 +03:00
Nikolai Kochetov
4470afc9a9
added estimateNumRowsMax to MergeTreeBlockSizePredictor based on max column size
2017-07-13 21:45:09 +03:00
Nikolai Kochetov
6b29b3bd47
round number of rows to next mark in MergeTreeBaseBlockInputStream
2017-07-13 21:45:09 +03:00
Nikolai Kochetov
d14f64af7b
delected commented logging code
2017-07-13 21:45:09 +03:00
Nikolai Kochetov
586dff9126
delay reading for granulas if prewhere column is false during partial granula reading
2017-07-13 21:45:09 +03:00
Nikolai Kochetov
86680f5513
fixed partial granula reading for const columns in prewhere
2017-07-13 21:45:09 +03:00
Nikolai Kochetov
609711b20c
partial part reading with prewhere
2017-07-13 21:45:09 +03:00
Nikolai Kochetov
c6d5ef6d30
added MergeTreeRangeReader
2017-07-13 21:45:09 +03:00
Nikolai Kochetov
5d1f4355a1
fixed allocated size calculation for ColumnArray
2017-07-13 19:53:34 +03:00
Nikolai Kochetov
985e449bd3
added IDataType::updateAvgValueSizeHints; renamed PODArray::allocated_size to PODArray::allocated_bytes; renamed renamed IColumn::allocatedSize to IColumn::allocatedBytes; removed IColumn.h from IDataType.h
2017-07-13 19:49:09 +03:00
Nikolai Kochetov
7c450aa5a5
added Block::allocatedSize()
2017-07-13 17:47:46 +03:00
Nikolai Kochetov
d67a6c08cd
fixed allocated size calculation for ColumnString and ColumnVector
2017-07-13 17:41:55 +03:00
Alexey Milovidov
8a8596c58f
Using lexer: additions [#CLICKHOUSE-2].
2017-07-13 09:10:56 +03:00
Alexey Milovidov
c4ea197232
Using lexer: fixed error [#CLICKHOUSE-2].
2017-07-13 09:05:38 +03:00
Alexey Milovidov
aa7ab3e23e
Using lexer: added missing file [#CLICKHOUSE-2].
2017-07-13 08:59:01 +03:00
Alexey Milovidov
00486d0de4
Using lexer: better error messages [#CLICKHOUSE-2].
2017-07-13 08:59:01 +03:00
Alexey Milovidov
cedf9b34c9
Using lexer: better error messages [#CLICKHOUSE-2].
2017-07-13 08:59:01 +03:00
Alexey Milovidov
3659f653fb
Using lexer: better error messages [#CLICKHOUSE-2].
2017-07-13 08:59:01 +03:00
Alexey Milovidov
6f4f95412f
Using lexer: better error messages [#CLICKHOUSE-2].
2017-07-13 08:59:01 +03:00
Alexey Milovidov
708b69abed
Using lexer: better error messages [#CLICKHOUSE-2].
2017-07-13 08:59:01 +03:00
Alexey Milovidov
47d9083c08
Using lexer: better error messages [#CLICKHOUSE-2].
2017-07-13 08:59:01 +03:00
Alexey Milovidov
96468884f3
Using lexer: better error messages [#CLICKHOUSE-2].
2017-07-13 08:59:01 +03:00
Alexey Milovidov
0614ba5684
Using lexer: better error messages [#CLICKHOUSE-2].
2017-07-13 08:59:01 +03:00
Alexey Milovidov
be1710ecef
Using lexer (development) [#CLICKHOUSE-2].
2017-07-13 08:59:01 +03:00
Alexey Milovidov
d4974b0724
Using lexer (development) [#CLICKHOUSE-2].
2017-07-13 08:59:01 +03:00
Alexey Milovidov
f6ff1f7e72
Using lexer (development) [#CLICKHOUSE-2].
2017-07-13 08:59:01 +03:00
Alexey Milovidov
2e6ceeba1c
Using lexer (development) [#CLICKHOUSE-2].
2017-07-13 08:59:01 +03:00
Alexey Milovidov
a8b6920eab
Using lexer (development) [#CLICKHOUSE-2].
2017-07-13 08:59:01 +03:00
Alexey Milovidov
cd8d8bf972
Using lexer (development) [#CLICKHOUSE-2].
2017-07-13 08:59:01 +03:00
Alexey Milovidov
57d2f5d0e6
Using lexer (development) [#CLICKHOUSE-2].
2017-07-13 08:59:01 +03:00
Alexey Milovidov
4562785c2a
Using lexer (development) [#CLICKHOUSE-2].
2017-07-13 08:59:01 +03:00
Alexey Milovidov
4255b81f3f
Using lexer (development) [#CLICKHOUSE-2].
2017-07-13 08:59:01 +03:00
Alexey Milovidov
65955d50e7
Using lexer (incomplete) [#CLICKHOUSE-2].
2017-07-13 08:59:01 +03:00
Alexey Milovidov
bec6aac3bb
Using lexer (incomplete) [#CLICKHOUSE-2].
2017-07-13 08:59:01 +03:00
Alexey Milovidov
e4d410fe13
Using lexer (incomplete) [#CLICKHOUSE-2].
2017-07-13 08:59:01 +03:00
Alexey Milovidov
d5c9301776
Using lexer (incomplete) [#CLICKHOUSE-2].
2017-07-13 08:59:01 +03:00
Alexey Milovidov
02274ed511
Using lexer (incomplete) [#CLICKHOUSE-2].
2017-07-13 08:59:01 +03:00
Alexey Milovidov
1cd998a9bd
Using lexer (incomplete) [#CLICKHOUSE-2].
2017-07-13 08:59:01 +03:00
Alexey Milovidov
7068d1ff81
Using lexer (incomplete) [#CLICKHOUSE-2].
2017-07-13 08:59:01 +03:00
Alexey Milovidov
de75fa000b
Using lexer (incomplete) [#CLICKHOUSE-2].
2017-07-13 08:59:01 +03:00
Alexey Milovidov
2073fef375
Using lexer (incomplete) [#CLICKHOUSE-2].
2017-07-13 08:59:01 +03:00
Alexey Milovidov
83c8043193
Using lexer (incomplete) [#CLICKHOUSE-2].
2017-07-13 08:59:01 +03:00
Alexey Milovidov
1bab7fc344
Using lexer (incomplete) [#CLICKHOUSE-2].
2017-07-13 08:59:01 +03:00
Alexey Milovidov
d815b766fa
Using lexer (incomplete) [#CLICKHOUSE-2].
2017-07-13 08:59:01 +03:00
Alexey Milovidov
4b1eeee1a3
Using lexer (incomplete) [#CLICKHOUSE-2].
2017-07-13 08:59:01 +03:00
Alexey Milovidov
5b5795633e
Using lexer (incomplete) [#CLICKHOUSE-2].
2017-07-13 08:59:01 +03:00
Alexey Milovidov
ad598d5dfc
Added lexer [#CLICKHOUSE-2].
2017-07-13 08:59:01 +03:00
Alexey Milovidov
e8170fde2a
Fixed error in prev. revisions [#CLICKHOUSE-2].
2017-07-13 03:04:51 +03:00
Nikolai Kochetov
1cf669cc0a
made collectIdentifiersInFirstLevelOfSelectQuery static
2017-07-12 21:51:51 +03:00
Nikolai Kochetov
5ed5e8c75d
moved getRequestedColumns to StorageMerge::collectIdentifiersInFirstLevelOfSelectQuery
2017-07-12 21:51:51 +03:00
Nikolai Kochetov
7c705b1a24
added RemoveColumnsBlockInputStream
2017-07-12 21:51:51 +03:00
Nikolai Kochetov
5924b13f4c
fixed unnecessary columns removing for merge table on distibuted
2017-07-12 21:51:51 +03:00
Nikolai Kochetov
489af678d5
added avg_value_size_hint for NativeBlockInputStream [#CLICKHOUSE-3066]
2017-07-12 21:41:08 +03:00
Alexey Zatelepin
58d541b740
slightly more robust ReplicatedMergeTreePartCheckThread::TemporarilyStop [#CLICKHOUSE-3074]
2017-07-12 20:09:44 +03:00
Alexey Zatelepin
bdc529645e
fix cyclical yielding of leadership at startup [#CLICKHOUSE-3074]
2017-07-12 20:09:44 +03:00
Alexey Milovidov
1908b2b5a5
Fixed error in prev. revision [#CLICKHOUSE-2].
2017-07-12 19:51:39 +03:00
Alexey Zatelepin
5e61a56743
always initialize trie ptr (just in case) [#CLICKHOUSE-3087]
2017-07-12 16:35:05 +03:00
Alexey Zatelepin
4d4387f1c8
fix reloading of TrieDictionary [#CLICKHOUSE-3087]
2017-07-12 16:35:05 +03:00
Alexey Milovidov
bb38ec2c21
Fixed discrepancy in file names [#CLICKHOUSE-2].
2017-07-12 04:16:01 +03:00
Alexey Milovidov
1048fccf6b
Misc [#CLICKHOUSE-2].
2017-07-12 03:49:30 +03:00
Alexey Milovidov
24331a18f5
Removing dependency on functions from client: bug fixes [#CLICKHOUSE-2].
2017-07-12 03:42:50 +03:00
Alexey Milovidov
976e47e1af
Removing dependency on functions from client: preparation [#CLICKHOUSE-2].
2017-07-12 03:42:50 +03:00
Alexey Milovidov
8c1a6f5736
Removing dependency on functions from client: preparation [#CLICKHOUSE-2].
2017-07-12 03:42:50 +03:00
Alexey Milovidov
33c6e0b804
Removing dependency on functions from client: preparation [#CLICKHOUSE-2].
2017-07-12 03:42:50 +03:00
Alexey Milovidov
3e3af1cff4
Misc [#CLICKHOUSE-2].
2017-07-11 23:12:15 +03:00
Alexey Milovidov
fb8d8ec768
Fixed error (tnx Vladimir Chebotarev) [#CLICKHOUSE-2].
2017-07-11 21:59:01 +03:00
Vitaliy Lyudvichenko
b843ea5b3f
Resolves #964 . Fixed input_format_skip_unknown_fields for negative numbers. [#CLICKHOUSE-3]
2017-07-11 21:36:51 +03:00
Alexey Zatelepin
7892129481
peformance-test: fix rows_read, bytes_read calculation [#CLICKHOUSE-3086]
2017-07-11 21:26:05 +03:00
Nikolai Kochetov
ee66169d5e
added throw_if_column_not_found to FilterColumnsBlockInputStream [#CLICKHOUSE-3111]
2017-07-11 20:58:49 +03:00
Alexey Zatelepin
57ba50f585
allow batching Distributed inserts [#CLICKHOUSE-3126]
2017-07-11 20:56:17 +03:00
Alexey Zatelepin
4a98d83a3b
introduce replicated_can_become_leader merge_tree setting [#CLICKHOUSE-3109]
2017-07-11 20:44:35 +03:00
Alexey Milovidov
cefc1a58b6
Merge branch 'master' of github.com:yandex/ClickHouse
2017-07-11 04:20:48 +03:00
Alexey Milovidov
2bd84600d1
Miscellaneous [#CLICKHOUSE-2].
2017-07-11 04:20:40 +03:00
Nikolai Kochetov
524507b9b8
fixed setting comparison
2017-07-10 21:15:56 +03:00
Nikolai Kochetov
c06c234c63
added deleted comment after merge
2017-07-10 17:33:52 +03:00
Nikolai Kochetov
7d5590d66d
Merge branch 'master' into output-format-json-quote-denormals
2017-07-10 17:29:17 +03:00
Nikolai Kochetov
39797813cc
disabled check for settings with same value in readonly mode in InterpreterSetQuery [#CLICKHOUSE-3041]; added tests for output_format_json_quote_denormals setting
2017-07-10 17:03:34 +03:00
Alexey Milovidov
86eb4d2d06
Renamed Singleton to avoid conflicts with Arcadia* codebase [#DEVTOOLS-3343].
...
* - "Arcadia" is the name of Yandex's monorepo.
2017-07-10 07:34:14 +03:00
Alexey Milovidov
16a6eee397
Fixed build after merge [#CLICKHOUSE-3].
2017-07-10 07:22:36 +03:00
Alexey Milovidov
1040abfd18
Fixed build after merge [#CLICKHOUSE-3].
2017-07-10 07:16:11 +03:00
Alexey Milovidov
a742ac3b6c
Fixed build after merge [#CLICKHOUSE-3].
2017-07-10 07:10:39 +03:00
Alexey Milovidov
78e905aa3a
Fixed build after merge [#CLICKHOUSE-3].
2017-07-10 07:10:25 +03:00
alexey-milovidov
b64b243a73
Merge pull request #945 from YiuRULE/master
...
Implement natively uuid type in Clickhouse
2017-07-10 07:00:45 +03:00
Alexey Milovidov
119961133c
Removed obsolete test [#CLICKHOUSE-3].
2017-07-10 06:47:20 +03:00
Alexey Milovidov
bf58b8cdb1
Fixed test [#CLICKHOUSE-3].
2017-07-10 06:45:04 +03:00
Alexey Milovidov
db3d56b1f5
Removed obsolete test binary [#CLICKHOUSE-3].
2017-07-10 06:43:25 +03:00
Alexey Zatelepin
89939a685a
Stream row sources from disk for vertical merge [#CLICKHOUSE-3118]
2017-07-09 21:57:06 +03:00
Alexey Zatelepin
1cd936579d
introduce IColumn::gather() [#CLICKHOUSE-3118]
2017-07-09 21:57:06 +03:00
Nikolai Kochetov
d804855460
added FormatSettingsJSON.h
2017-07-09 20:16:26 +03:00
Nikolai Kochetov
25ac546e78
disabled exception during overriding settings with same value in readonly mode [#CLICKHOUSE-3041]
2017-07-09 20:16:26 +03:00
Nikolai Kochetov
63119eda7d
updated FormatSettingsJSON
2017-07-09 20:16:26 +03:00
Nikolai Kochetov
0ead0ee2f1
added FormatSettingsJSON [#CLICKHOUSE-3060]
2017-07-09 20:16:26 +03:00
alexey-milovidov
894d09f797
Update IFunction.h
2017-07-09 17:13:51 +03:00
Marek Vavruša
ff73394124
PKCondition: infer index use with pk subexpression
...
By default only constraints explicitly matching
primary key expression (or expression wrapped in
a monotonic function) are eligible for part and
range selection. So for example, if index is:
(toStartOfHour(dt), UserID)
Then a query such as this resorts to full scan:
SELECT count() FROM t WHERE dt = now()
Intuitively, only parts with toStartOfHour(now())
could be selected, but it is less trivial to prove.
The primary key currently can be wrapped in a chain
of monotonic functions, so following would work:
toStartOfHour(dt) = toStartOfHour(now()) AND dt = now()
It must be however explicitly stated, if we wanted
to infer that we’d have to know the inverse function,
and prove that the inverse function is monotonic
on given interval. This is not practical as
there is no inverse function that for example undos
rounding, it isn’t strictly monotonic.
There are however functions that don’t transform
output range and preserve monotonicity on the
complete input range, such as rounding or casts
to a same or wider numeric type. This eliminates
the need to find inverse function, as no check for monotonicity over arbitrary interval is needed,
and thus makes this optimisation possible.
2017-07-09 17:13:51 +03:00
Vitaliy Lyudvichenko
b46a2a6eab
Fixed infinite sleeping of low-priority queries. [#CLICKHOUSE-2]
2017-07-09 03:53:13 +03:00
Vitaliy Lyudvichenko
7150e9447a
Fixed multiIf in case of empty arguments. [#CLICKHOUSE-3119]
2017-07-07 17:42:37 +03:00
Nikolai Kochetov
2f08ba9e94
added FormatSettingsJSON.h
2017-07-06 18:11:12 +03:00
Guillaume Tassery
2ec96fe822
Make UUID as a strong type
2017-07-06 16:50:29 +02:00
Nikolai Kochetov
d6001618f0
disabled exception during overriding settings with same value in readonly mode [#CLICKHOUSE-3041]
2017-07-06 17:42:31 +03:00
Liu Cong
08e43b149a
Fix a typo of 'Columns/ColumnVector.h': metod => method
2017-07-05 20:46:37 +03:00
Nikolai Kochetov
c99a9b1c55
moved FilterColumnsBlockInputStream implementation to .cpp [#CLICKHOUSE-3111]
2017-07-05 20:14:17 +03:00
alexey-milovidov
85bdb472ea
Update FilterColumnsBlockInputStream.h
2017-07-05 20:14:17 +03:00