Commit Graph

5998 Commits

Author SHA1 Message Date
Alexey Milovidov
6578fb2645 Fixed error [#CLICKHOUSE-3191]. 2017-08-01 00:39:24 +03:00
alexey-milovidov
d174ebc5c9 Update MergeTreeDataWriter.cpp 2017-07-31 20:44:58 +03:00
alexey-milovidov
4877aa394c Update MergeTreeData.cpp 2017-07-31 20:44:41 +03:00
alexey-milovidov
fdb7e080be Update CompressedWriteBuffer.cpp 2017-07-31 20:43:33 +03:00
alexey-milovidov
50244880cc Update CompressedReadBufferBase.cpp 2017-07-31 20:42:23 +03:00
alexey-milovidov
3d2ec763ed Update CompressedReadBufferBase.cpp 2017-07-31 20:40:56 +03:00
Pawel Rog
3a6c444996 Merge branch 'master' into nocompression 2017-07-31 13:36:58 +02:00
Pawel Rog
e1ab721d3d Use compression method from configs 2017-07-31 13:12:28 +02:00
Pawel Rog
a0cc54457e Added compression NONE 2017-07-31 12:44:45 +02:00
Alexey Milovidov
4e76c96381 Temporarily disabled "preferred_max_column_in_block_size_bytes" option [#CLICKHOUSE-3188]. 2017-07-31 03:50:06 +03:00
Alexey Milovidov
306070d2cc Fixed error in one of prev. revisions [#CLICKHOUSE-3150]. 2017-07-30 22:47:32 +03:00
proller
460d66c68d Split FunctionsArray.cpp register (#1043)
* PerformanceTest: use getMultiple*FromConfig, fix debug helpers

* Missing file

* gcc7 fixes

* clnag-format of Split ComplexKeyCacheDictionary

* Requested changes

* Fix boost 1.64 and gcc7+ compile errors

* More fixes

* Fix arm build

* Fix nothrow

* Better arm detection

* Split FunctionsArray.cpp register
2017-07-30 15:31:58 +03:00
Nikolai Kochetov
78991b25ee fixed preferred_max_column_in_block_size_bytes disabling 2017-07-30 01:24:57 +03:00
Alexey Milovidov
8fb452e99a Fixed wrong log message when removing temporary part after merge, that is not byte-identical than same part on replicas; added profile event; added fail-safe logic [#MTRSADMIN-3175]. 2017-07-29 00:25:24 +03:00
proller
700d4e0904 Fix boost 1.64 and gcc7+ compile errors (#1040)
* PerformanceTest: use getMultiple*FromConfig, fix debug helpers

* Missing file

* clnag-format of Split ComplexKeyCacheDictionary

* Requested changes

* Fix boost 1.64 and gcc7+ compile errors

* More fixes
2017-07-28 23:41:51 +03:00
alexey-milovidov
6917eb3153 Update MergeTreeBaseBlockInputStream.cpp 2017-07-28 22:29:40 +03:00
Nikolai Kochetov
b714d2b514 fixed column size calculation for Array column in MergeTreeReader::readRows 2017-07-28 22:29:40 +03:00
Nikolai Kochetov
d9f4f9b0f9 changed skipping rows for Array columns in MergeTreeBaseBlockInputStream 2017-07-28 22:29:40 +03:00
Alexey Milovidov
4c89570988 Don't limit the size of internal queries (this is for cache dictionaries with ClickHouse source) [#CLICKHOUSE-2]. 2017-07-28 21:43:40 +03:00
Alexey Milovidov
dd1f54bce2 Changed Poco::RWLock to std::shared_lock [#APRILDEV-91]. 2017-07-28 20:34:02 +03:00
Alexey Milovidov
3502d36d7d Fixed build [#CLICKHOUSE-2]. 2017-07-28 02:23:13 +03:00
Alexey Milovidov
e4af70149f Fixed build [#CLICKHOUSE-2]. 2017-07-28 00:27:10 +03:00
Alexey Milovidov
acf9c2fbd5 Miscellaneous [#CLICKHOUSE-2]. 2017-07-28 00:05:33 +03:00
proller
9eb6e5b19c clang-format of Split ComplexKeyCacheDictionary (#1035)
* clnag-format of Split ComplexKeyCacheDictionary

* Requested changes
2017-07-27 23:39:40 +03:00
Alexey Milovidov
2c815356c1 Fixed regression in parsing of CAST [#CLICKHOUSE-3180]. 2017-07-27 22:25:52 +03:00
proller
3dbde50daa Split ComplexKeyCacheDictionary to use less then 1.5G ram when compile (#1034)
* Split FunctionsArithmetic.cpp to generated functions

* Use ccache if found

* Do not use ccache if ccache defined in CMAKE_CXX_COMPILER_LAUNCHER

* check_include.sh: print memory usage

* Try split CacheDictionary source

* Split ok

* wip

* wip

* wip

* wip

* wip

* wip
2017-07-27 22:05:55 +03:00
Nikolai Kochetov
c88e8a79cc merged with master 2017-07-27 18:43:46 +03:00
Nikolai Kochetov
2f8f199d08 separated connection pool creation from StorageDistributedDirectoryMonitor in StorageDistributed; fixed bugs from review [#CLICKHOUSE-3033] 2017-07-27 18:24:39 +03:00
Vitaliy Lyudvichenko
515271544d Add detail comment. [#CLICKHOUSE-3173] 2017-07-27 11:19:20 +03:00
Vitaliy Lyudvichenko
445608afec Fix an iterator pointing after std::end(). [#CLICKHOUSE-3173] 2017-07-27 11:19:20 +03:00
alexey-milovidov
73c2a1483d Update ReplicatedMergeTreeCleanupThread.cpp 2017-07-27 11:19:20 +03:00
alexey-milovidov
dcd7354fb2 Update ReplicatedMergeTreeCleanupThread.cpp 2017-07-27 11:19:20 +03:00
alexey-milovidov
d6750880e0 Update ReplicatedMergeTreeCleanupThread.cpp 2017-07-27 11:19:20 +03:00
Vitaliy Lyudvichenko
cd5bb4d921 Add replicated_deduplication_window_seconds merge_tree parameter. [#CLICKHOUSE-3173] 2017-07-27 11:19:20 +03:00
Vitaliy Lyudvichenko
dcc4d3bff0 Avoid zxid overflow problem when cleaning old blocks used in the deduplication. [#CLICKHOUSE-3173] 2017-07-27 11:19:20 +03:00
proller
ff7f3af238 Split FunctionsArithmetic.cpp to generated functions 2017-07-26 21:32:35 +03:00
Nikolai Kochetov
01a004050d fixed MergeTreeBlockSizePredictor updating from empy block while reading 2017-07-26 20:03:09 +03:00
Alexey Milovidov
cd4fef7767 Miscellaneous [#CLICKHOUSE-2]. 2017-07-26 08:55:59 +03:00
Alexey Milovidov
3dcb2e6aed Fixed test [#CLICKHOUSE-2]. 2017-07-26 04:21:19 +03:00
Alexey Milovidov
de1f4f4c5b Merge branch 'master' of github.com:yandex/ClickHouse 2017-07-26 00:07:13 +03:00
Alexey Milovidov
d57f5b81a4 Removed MaterializedView inheritance of View, as they are not related. Fixed error with Views [#CLICKHOUSE-3175]. 2017-07-26 00:07:05 +03:00
Alexey Zatelepin
64fa3ebc58 correctly choose number of merge threads for external aggregation [#CLICKHOUSE-2] 2017-07-25 22:43:23 +03:00
Nikolai Kochetov
b3157aebb2 added synchronous insert into distributed table [#CLICKHOUSE-3033] 2017-07-25 22:42:36 +03:00
Alexey Milovidov
ec676baee7 Fixed error after merge [#CLICKHOUSE-2]. 2017-07-25 22:07:07 +03:00
Alexey Milovidov
6851a7754a Fixed linkage [#CLICKHOUSE-2]. 2017-07-25 21:32:52 +03:00
Alexey Milovidov
d20c411b0d Fixed linkage [#CLICKHOUSE-2]. 2017-07-25 21:19:22 +03:00
Alexey Zatelepin
2230bef68a reset resulting bucket_num if source blocks are from different buckets [#CLICKHOUSE-3133] 2017-07-25 20:56:09 +03:00
alexey-milovidov
b831d88162 Update ArenaAllocator.h 2017-07-25 20:06:24 +03:00
Vitaliy Lyudvichenko
870b1aefcb Get rid of of PODArray copypaste. [#CLICKHOUSE-3084] 2017-07-25 20:04:10 +03:00
Vitaliy Lyudvichenko
ae3b1d8d3a Fix clang compilation. [#CLICKHOUSE-3084] 2017-07-25 20:04:10 +03:00
Vitaliy Lyudvichenko
fefb4dad67 Get rid of old groupArray() implementations. [#CLICKHOUSE-3084] 2017-07-25 20:04:10 +03:00
Vitaliy Lyudvichenko
792faaa2db Add adaptive allocator. [#CLICKHOUSE-3084] 2017-07-25 20:04:10 +03:00
Vitaliy Lyudvichenko
03e5bf9471 Add support of parameters in aggregate function constructors. [#CLICKHOUSE-3084] 2017-07-25 20:04:10 +03:00
Vitaliy Lyudvichenko
c3a15da3f8 Add support of limit parameter into groupArray. [#CLICKHOUSE-3084] 2017-07-25 20:04:10 +03:00
Vitaliy Lyudvichenko
c2cc8d6147 Add several groupArray implementations. [#CLICKHOUSE-3084] 2017-07-25 20:04:10 +03:00
Guillaume Tassery
03bbe9938a Function for generate an uuid v4 (#1008)
* Fix compilation error when include FunctionsRandom.h twice

* Implement function for generate UUID V4

* remove isDeterministicInScopeOfQuery for generateUUIDV4

* Uniformize RandX Implementation

* rename generateUUIDV4 into generateUUIDv4

* Modify random function for making it more generic
2017-07-25 20:00:38 +03:00
Alexey Zatelepin
18672204e5 fix handling of overflow data during external aggregation [#CLICKHOUSE-3133] 2017-07-25 19:56:48 +03:00
Alexey Zatelepin
cd5dc7ca29 fix segfault when handling overflow blocks in MergingAggregatedMemoryEfficientBlockInputStream [#CLICKHOUSE-3133] 2017-07-25 19:56:48 +03:00
Alexey Milovidov
f153d5f5db Additions to prev. revision [#CLICKHOUSE-2]. 2017-07-25 01:53:44 +03:00
Alexey Milovidov
b4c610a8e7 Fixed errors [#CLICKHOUSE-2]. 2017-07-25 01:44:13 +03:00
Alexey Milovidov
188ee349bd Fixed errors [#CLICKHOUSE-2]. 2017-07-25 01:44:13 +03:00
Alexey Milovidov
a6df2e7151 Added support for non-constant and negative offset and size for substring function (continued) [#CLICKHOUSE-2]. 2017-07-25 01:44:13 +03:00
Alexey Milovidov
487278e2f5 Added support for non-constant and negative offset and size for substring function (continued) [#CLICKHOUSE-2]. 2017-07-25 01:44:13 +03:00
Alexey Milovidov
3144cbc9c1 Added support for non-constant and negative offset and size for substring function (continued) [#CLICKHOUSE-2]. 2017-07-25 01:44:13 +03:00
Alexey Milovidov
94768fe9bf Added support for non-constant and negative offset and size for substring function (continued) [#CLICKHOUSE-2]. 2017-07-25 01:44:13 +03:00
Alexey Milovidov
52b5bae62f Added support for non-constant and negative offset and size for substring function (continued) [#CLICKHOUSE-2]. 2017-07-25 01:44:13 +03:00
Alexey Milovidov
30ff4a78a6 Added support for non-constant and negative offset and size for substring function (continued) [#CLICKHOUSE-2]. 2017-07-25 01:44:13 +03:00
Alexey Milovidov
14de659c99 Added support for non-constant offset and size for substring function [#CLICKHOUSE-2]. 2017-07-25 01:44:13 +03:00
Alexey Milovidov
4a4e863f9d Fixed warning on clang 6 [#CLICKHOUSE-2]. 2017-07-25 01:44:13 +03:00
Alexey Milovidov
0c8e082954 Using GatherUtils (experimental) [#CLICKHOUSE-2]. 2017-07-25 01:44:13 +03:00
Alexey Milovidov
366ad1595e Fixed warning on clang 6 [#CLICKHOUSE-2]. 2017-07-25 01:44:13 +03:00
Alexey Milovidov
1557fa2e8d Removed "dispatchFor" templates, because gcc 6 cannot compile them (only clang is capable) [#CLICKHOUSE-2]. 2017-07-25 01:44:13 +03:00
Alexey Milovidov
ffbd2f46f9 GatherUtils (experimental) [#CLICKHOUSE-2]. 2017-07-25 01:44:13 +03:00
Alexey Milovidov
9540a2da90 GatherUtils (experimental) [#CLICKHOUSE-2]. 2017-07-25 01:44:13 +03:00
Alexey Milovidov
bfdc108a08 Cleaning old code [#CLICKHOUSE-2]. 2017-07-25 01:44:13 +03:00
Alexey Milovidov
ee81d53bd5 Functions simplification [#CLICKHOUSE-2]. 2017-07-25 01:44:13 +03:00
alexey-milovidov
bbe70e8cfa Update FunctionsStringSearch.cpp 2017-07-24 20:36:16 +03:00
alexey-milovidov
d3159ed2c5 Merge pull request #1015 from yandex/fix-performance-after-changing-reading
Fix performance after changing reading
2017-07-24 18:07:57 +03:00
Nikolai Kochetov
fcd50e3629 fixed comment 2017-07-24 18:06:32 +03:00
proller
eba5ad73ec clean 2017-07-24 18:04:49 +03:00
proller
57b9bcd354 Fix missing includes 2017-07-24 18:04:49 +03:00
Vitaliy Lyudvichenko
bc9ee54985 Don't throw exception in case of clearing empty part. [#CLICKHOUSE-2896] 2017-07-24 18:04:17 +03:00
Nikolai Kochetov
b55dcc9f93 cnanged comments; fixed 00443_preferred_block_size_bytes test 2017-07-24 16:59:03 +03:00
Alexey Milovidov
efaa44bea8 Attempt to improve performance [#CLICKHOUSE-3]. 2017-07-23 09:53:28 +03:00
Alexey Milovidov
a76061e1a9 Fixed MAC address related functions; added test [#CLICKHOUSE-3]. 2017-07-23 09:13:57 +03:00
Alexey Milovidov
1e8547b3f6 Fixed regression [#CLICKHOUSE-3148]. 2017-07-21 23:59:01 +03:00
Alexey Milovidov
31d8ba3b34 Fixed error after merge [#CLICKHOUSE-3150]. 2017-07-21 23:20:22 +03:00
Nikolai Kochetov
10bbededb0 merged with master 2017-07-21 21:10:14 +03:00
Nikolai Kochetov
bc0921a64e added comments 2017-07-21 21:02:02 +03:00
alexey-milovidov
6d47981e86 Update Collator.cpp 2017-07-21 20:59:39 +03:00
Vladimir Kolobaev
1b8a062e7b Issue #1000 - Returned some remote code in Collator.app 2017-07-21 20:59:39 +03:00
Vladimir Kolobaev
e85cd44e48 Issue #1000 - remove 'diagnostic ignored -Wunused-private-field' in Collator.app 2017-07-21 20:59:39 +03:00
Alexey Zatelepin
c939d2f96e put Distributed inserts with different block structure into different batches [#CLICKHOUSE-3163] 2017-07-21 20:55:38 +03:00
Nikolai Kochetov
ca1957ab82 ignore preferred_max_column_in_block_size_bytes if zero [#CLICKHOUSE-3170] 2017-07-21 20:54:09 +03:00
Nikolai Kochetov
cc349b410e set min block size as index granularity for max_block_size limit during reading [#CLICKHOUSE-3170] 2017-07-21 20:54:09 +03:00
Nikolai Kochetov
5477e7c1f9 added date and time to number functions [#CLICKHOUSE-3132] 2017-07-21 20:51:08 +03:00
Nikolai Kochetov
c64b65ee29 ignore preferred_max_column_in_block_size_bytes if zero [#CLICKHOUSE-3170] 2017-07-21 20:45:51 +03:00
Nikolai Kochetov
755d16b22d set min block size as index granularity for max_block_size limit during reading [#CLICKHOUSE-3170] 2017-07-21 14:55:33 +03:00
Alexey Milovidov
3a0bf8e642 Fixed error after merge [#CLICKHOUSE-3150]. 2017-07-21 09:55:33 +03:00
Alexey Milovidov
e1cc8289b2 Fixed error after merge [#CLICKHOUSE-3150]. 2017-07-21 09:46:50 +03:00
Alexey Milovidov
c7b1e92522 Fixed error after merge [#CLICKHOUSE-3150]. 2017-07-21 09:40:05 +03:00
alexey-milovidov
d4d266fa18 ColumnConst unification (#1011)
* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].

* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].

* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].

* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].

* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].

* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].

* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].

* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].

* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].

* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].

* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].

* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].

* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].

* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].

* Fixed error in ColumnArray::replicateGeneric [#CLICKHOUSE-3150].

* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].

* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].

* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].

* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].

* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].

* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].

* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].

* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].

* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].

* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].

* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].

* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].

* ColumnConst: unification (incomplete) [#CLICKHOUSE-3150].
2017-07-21 09:35:58 +03:00
alexey-milovidov
1a1112d8bb Update MergeTreeBlockReadUtils.h 2017-07-20 23:45:23 +03:00
Nikolai Kochetov
0b7439469d fixed MergeTreeBlockSizePredictor initialization for sample block 2017-07-20 23:45:23 +03:00
Nikolai Kochetov
00ae32c946 renamed approximate_number_of_rows_in_part to number_of_rows_in_part 2017-07-20 23:45:23 +03:00
Nikolai Kochetov
e612152584 changed initialization in MergeTreeBlockSizePredictor 2017-07-20 23:45:23 +03:00
Vladimir Chebotarev
7254478fde functions: Remote mode for hasColumnInTable. [#CHEBOTAREV-9] 2017-07-20 17:08:42 +03:00
Nikolai Kochetov
2afd84c40c fixed is_reading_finished status assignment in MergeTreeRangeReader [#CLICKHOUSE-3161] 2017-07-19 19:39:18 +03:00
Guillaume Tassery
57b4066da8 Improve performance for deserialization/serialization of UUID type 2017-07-19 19:26:37 +03:00
proller
3f9cdfe464 PerformanceTest: Add memory and threads info (#1004)
* Cmake: clear test variables with CMakePushCheckState (it will fix macos build issue)

* Add some perf tests

* Fix PerformanceTest, fix tests

* Add nyc-taxi queries

* Update PerformanceTest.cpp

* PerformanceTest: allow recursive .xml search

* Tests tune

* Move tests to dirs

* --recursive

* Add space

* PerformanceTest: Add memory and threads info
2017-07-19 18:18:47 +03:00
Nikolai Kochetov
b5e94ede62 fixed is_reading_finished status assignment in MergeTreeRangeReader [#CLICKHOUSE-3161] 2017-07-19 16:44:46 +03:00
Alexey Milovidov
6e6e9f9d38 Removed useless variable [#CLICKHOUSE-3161]. 2017-07-19 04:05:39 +03:00
proller
ae626281b6 PerformanceTest: add --recursive option. add more threaded/not threaded tests (#997)
* Cmake: clear test variables with CMakePushCheckState (it will fix macos build issue)

* Add some perf tests

* Fix PerformanceTest, fix tests

* Add nyc-taxi queries

* Update PerformanceTest.cpp

* PerformanceTest: allow recursive .xml search

* Tests tune

* Move tests to dirs

* --recursive

* Add space
2017-07-18 23:41:24 +03:00
Alexey Zatelepin
4866a37a2b fix selecting max_threads for MergingAggregatedBlockInputStream [#CLICKHOUSE-3156] 2017-07-18 23:40:23 +03:00
alexey-milovidov
25288ac4d5 Update PerformanceTest.cpp 2017-07-18 19:35:01 +03:00
proller
cd54ff0f82 Fix PerformanceTest, fix tests 2017-07-18 19:35:01 +03:00
Nikolai Kochetov
7e293df9af merged with master 2017-07-17 22:48:22 +03:00
Nikolai Kochetov
39cb5e0e3e fixed ColumnGathererStream finish reading condition 2017-07-17 22:43:09 +03:00
Nikolai Kochetov
eae94edd6b fixed vertical merge for ReplacingSortedBlockInputStream [#CLICKHOUSE-3131] 2017-07-17 22:40:30 +03:00
Alexey Milovidov
295459e89e Miscellaneous [#CLICKHOUSE-2]. 2017-07-17 01:55:40 +03:00
Alexey Milovidov
bfc099ea5f Miscellaneous [#CLICKHOUSE-2]. 2017-07-17 01:41:15 +03:00
Alexey Milovidov
930c373309 Added synonims for few data types for compatibility with Tableau [#CLICKHOUSE-3141]. 2017-07-16 08:09:47 +03:00
Alexey Milovidov
5f98f4b862 Addition to prev. revision [#CLICKHOUSE-3149]. 2017-07-16 07:11:11 +03:00
Alexey Milovidov
5c3ca0c16e Inverted dependency in DataTypeFactory [#CLICKHOUSE-3149]. 2017-07-16 07:11:11 +03:00
Alexey Milovidov
17786c0e6d Added missing file [#CLICKHOUSE-3148]. 2017-07-15 07:06:51 +03:00
Alexey Milovidov
9406b6a5aa Get rid of ASTSet, because it is not really an AST and it lead to bugs [#CLICKHOUSE-3148]. 2017-07-15 06:48:36 +03:00
Alexey Milovidov
200a1e27e3 Merge branch 'master' into get-rid-of-ast-set 2017-07-15 03:23:20 +03:00
Vitaliy Lyudvichenko
c6eb11aad5 Use more suitable method hasRealColumn. [#CLICKHOUSE-2896] 2017-07-15 03:21:19 +03:00
Vitaliy Lyudvichenko
9ce4ff4c5b Add check for unexisting column in CLEAR COLUMN. [#CLICKHOUSE-2896] 2017-07-15 03:21:19 +03:00
Alexey Zatelepin
b10e9d1f18 Vertical merge fix: don't forget to output last source_to_fully_copy [#CLICKHOUSE-3118] 2017-07-14 22:06:40 +03:00
Nikolai Kochetov
a13423e552 added vertical merge for ReplacingSortedBlockInputStream 2017-07-14 21:17:23 +03:00
Alexey Milovidov
9bb902c3ef Allow to use database.table in right hand side of IN, JOIN [#CLICKHOUSE-3144]. 2017-07-14 04:00:29 +03:00
Alexey Milovidov
4a5fb9eb7b Miscellaneous [#CLICKHOUSE-2]. 2017-07-14 03:34:00 +03:00
Alexey Milovidov
1a0f5dd7cf Getting rid of ASTSet (incomplete) [#CLICKHOUSE-2] 2017-07-14 03:33:37 +03:00
alexey-milovidov
37985d3cb8 Merge pull request #981 from yandex/reduce-memory-consumprion-during-reading-column-string-from-native-block
Reduce memory consumprion during reading column string from native block
2017-07-14 00:01:50 +03:00
Nikolai Kochetov
283810e8ab merged with master 2017-07-13 23:59:34 +03:00
Nikolai Kochetov
75be0b81d8 removed typeid_cast from IColumn.h 2017-07-13 23:58:19 +03:00
Maxim Fridental
82b53b712a findClusterIndex: fix virtual method, improve formatting 2017-07-13 22:39:37 +03:00
Maxim Fridental
ff4df27cad Improve implementation of findClusterIndex and findClusterValue, add test 2017-07-13 22:39:37 +03:00
Nikolai Kochetov
210f14aaba renamed IDataType::updateAvgValueSizeHints to IDataType::updateAvgValueSizeHint 2017-07-13 21:54:17 +03:00
Nikolai Kochetov
f3f217b40a fixed tests 2017-07-13 21:45:09 +03:00
Nikolai Kochetov
546012079c fixed reading with zero max block size setting 2017-07-13 21:45:09 +03:00
Nikolai Kochetov
e2d032b883 fixed reading with zero max block size setting 2017-07-13 21:45:09 +03:00
Nikolai Kochetov
8ea812e669 better comment for MergeTreeRangeReader 2017-07-13 21:45:09 +03:00
Nikolai Kochetov
f232afd2f8 round number of rows to nearest mark in MergeTreeBaseBlockInputStream 2017-07-13 21:45:09 +03:00
Nikolai Kochetov
f1775af3f0 deleted commented code 2017-07-13 21:45:09 +03:00
Nikolai Kochetov
30ed774bf7 refactoring and more comments in MergeTreeBaseBlockInputStream [#CLICKHOUSE-3065] 2017-07-13 21:45:09 +03:00
Nikolai Kochetov
89386394aa refactoring and more comments in MergeTreeBaseBlockInputStream [#CLICKHOUSE-3065] 2017-07-13 21:45:09 +03:00
Nikolai Kochetov
8c65d6013e added preferred_max_column_in_block_size_bytes setting, added filtration ratio prediction [#CLICKHOUSE-3065] 2017-07-13 21:45:09 +03:00
Nikolai Kochetov
4470afc9a9 added estimateNumRowsMax to MergeTreeBlockSizePredictor based on max column size 2017-07-13 21:45:09 +03:00
Nikolai Kochetov
6b29b3bd47 round number of rows to next mark in MergeTreeBaseBlockInputStream 2017-07-13 21:45:09 +03:00
Nikolai Kochetov
d14f64af7b delected commented logging code 2017-07-13 21:45:09 +03:00
Nikolai Kochetov
586dff9126 delay reading for granulas if prewhere column is false during partial granula reading 2017-07-13 21:45:09 +03:00
Nikolai Kochetov
86680f5513 fixed partial granula reading for const columns in prewhere 2017-07-13 21:45:09 +03:00
Nikolai Kochetov
609711b20c partial part reading with prewhere 2017-07-13 21:45:09 +03:00
Nikolai Kochetov
c6d5ef6d30 added MergeTreeRangeReader 2017-07-13 21:45:09 +03:00
Nikolai Kochetov
5d1f4355a1 fixed allocated size calculation for ColumnArray 2017-07-13 19:53:34 +03:00
Nikolai Kochetov
985e449bd3 added IDataType::updateAvgValueSizeHints; renamed PODArray::allocated_size to PODArray::allocated_bytes; renamed renamed IColumn::allocatedSize to IColumn::allocatedBytes; removed IColumn.h from IDataType.h 2017-07-13 19:49:09 +03:00
Nikolai Kochetov
7c450aa5a5 added Block::allocatedSize() 2017-07-13 17:47:46 +03:00
Nikolai Kochetov
d67a6c08cd fixed allocated size calculation for ColumnString and ColumnVector 2017-07-13 17:41:55 +03:00
Alexey Milovidov
8a8596c58f Using lexer: additions [#CLICKHOUSE-2]. 2017-07-13 09:10:56 +03:00
Alexey Milovidov
c4ea197232 Using lexer: fixed error [#CLICKHOUSE-2]. 2017-07-13 09:05:38 +03:00
Alexey Milovidov
aa7ab3e23e Using lexer: added missing file [#CLICKHOUSE-2]. 2017-07-13 08:59:01 +03:00
Alexey Milovidov
00486d0de4 Using lexer: better error messages [#CLICKHOUSE-2]. 2017-07-13 08:59:01 +03:00
Alexey Milovidov
cedf9b34c9 Using lexer: better error messages [#CLICKHOUSE-2]. 2017-07-13 08:59:01 +03:00
Alexey Milovidov
3659f653fb Using lexer: better error messages [#CLICKHOUSE-2]. 2017-07-13 08:59:01 +03:00
Alexey Milovidov
6f4f95412f Using lexer: better error messages [#CLICKHOUSE-2]. 2017-07-13 08:59:01 +03:00
Alexey Milovidov
708b69abed Using lexer: better error messages [#CLICKHOUSE-2]. 2017-07-13 08:59:01 +03:00
Alexey Milovidov
47d9083c08 Using lexer: better error messages [#CLICKHOUSE-2]. 2017-07-13 08:59:01 +03:00
Alexey Milovidov
96468884f3 Using lexer: better error messages [#CLICKHOUSE-2]. 2017-07-13 08:59:01 +03:00
Alexey Milovidov
0614ba5684 Using lexer: better error messages [#CLICKHOUSE-2]. 2017-07-13 08:59:01 +03:00
Alexey Milovidov
be1710ecef Using lexer (development) [#CLICKHOUSE-2]. 2017-07-13 08:59:01 +03:00
Alexey Milovidov
d4974b0724 Using lexer (development) [#CLICKHOUSE-2]. 2017-07-13 08:59:01 +03:00
Alexey Milovidov
f6ff1f7e72 Using lexer (development) [#CLICKHOUSE-2]. 2017-07-13 08:59:01 +03:00
Alexey Milovidov
2e6ceeba1c Using lexer (development) [#CLICKHOUSE-2]. 2017-07-13 08:59:01 +03:00
Alexey Milovidov
a8b6920eab Using lexer (development) [#CLICKHOUSE-2]. 2017-07-13 08:59:01 +03:00
Alexey Milovidov
cd8d8bf972 Using lexer (development) [#CLICKHOUSE-2]. 2017-07-13 08:59:01 +03:00
Alexey Milovidov
57d2f5d0e6 Using lexer (development) [#CLICKHOUSE-2]. 2017-07-13 08:59:01 +03:00
Alexey Milovidov
4562785c2a Using lexer (development) [#CLICKHOUSE-2]. 2017-07-13 08:59:01 +03:00
Alexey Milovidov
4255b81f3f Using lexer (development) [#CLICKHOUSE-2]. 2017-07-13 08:59:01 +03:00
Alexey Milovidov
65955d50e7 Using lexer (incomplete) [#CLICKHOUSE-2]. 2017-07-13 08:59:01 +03:00
Alexey Milovidov
bec6aac3bb Using lexer (incomplete) [#CLICKHOUSE-2]. 2017-07-13 08:59:01 +03:00
Alexey Milovidov
e4d410fe13 Using lexer (incomplete) [#CLICKHOUSE-2]. 2017-07-13 08:59:01 +03:00
Alexey Milovidov
d5c9301776 Using lexer (incomplete) [#CLICKHOUSE-2]. 2017-07-13 08:59:01 +03:00
Alexey Milovidov
02274ed511 Using lexer (incomplete) [#CLICKHOUSE-2]. 2017-07-13 08:59:01 +03:00
Alexey Milovidov
1cd998a9bd Using lexer (incomplete) [#CLICKHOUSE-2]. 2017-07-13 08:59:01 +03:00
Alexey Milovidov
7068d1ff81 Using lexer (incomplete) [#CLICKHOUSE-2]. 2017-07-13 08:59:01 +03:00
Alexey Milovidov
de75fa000b Using lexer (incomplete) [#CLICKHOUSE-2]. 2017-07-13 08:59:01 +03:00
Alexey Milovidov
2073fef375 Using lexer (incomplete) [#CLICKHOUSE-2]. 2017-07-13 08:59:01 +03:00
Alexey Milovidov
83c8043193 Using lexer (incomplete) [#CLICKHOUSE-2]. 2017-07-13 08:59:01 +03:00
Alexey Milovidov
1bab7fc344 Using lexer (incomplete) [#CLICKHOUSE-2]. 2017-07-13 08:59:01 +03:00
Alexey Milovidov
d815b766fa Using lexer (incomplete) [#CLICKHOUSE-2]. 2017-07-13 08:59:01 +03:00
Alexey Milovidov
4b1eeee1a3 Using lexer (incomplete) [#CLICKHOUSE-2]. 2017-07-13 08:59:01 +03:00
Alexey Milovidov
5b5795633e Using lexer (incomplete) [#CLICKHOUSE-2]. 2017-07-13 08:59:01 +03:00
Alexey Milovidov
ad598d5dfc Added lexer [#CLICKHOUSE-2]. 2017-07-13 08:59:01 +03:00
Alexey Milovidov
e8170fde2a Fixed error in prev. revisions [#CLICKHOUSE-2]. 2017-07-13 03:04:51 +03:00
Nikolai Kochetov
1cf669cc0a made collectIdentifiersInFirstLevelOfSelectQuery static 2017-07-12 21:51:51 +03:00
Nikolai Kochetov
5ed5e8c75d moved getRequestedColumns to StorageMerge::collectIdentifiersInFirstLevelOfSelectQuery 2017-07-12 21:51:51 +03:00
Nikolai Kochetov
7c705b1a24 added RemoveColumnsBlockInputStream 2017-07-12 21:51:51 +03:00
Nikolai Kochetov
5924b13f4c fixed unnecessary columns removing for merge table on distibuted 2017-07-12 21:51:51 +03:00
Nikolai Kochetov
489af678d5 added avg_value_size_hint for NativeBlockInputStream [#CLICKHOUSE-3066] 2017-07-12 21:41:08 +03:00
Alexey Zatelepin
58d541b740 slightly more robust ReplicatedMergeTreePartCheckThread::TemporarilyStop [#CLICKHOUSE-3074] 2017-07-12 20:09:44 +03:00
Alexey Zatelepin
bdc529645e fix cyclical yielding of leadership at startup [#CLICKHOUSE-3074] 2017-07-12 20:09:44 +03:00
Alexey Milovidov
1908b2b5a5 Fixed error in prev. revision [#CLICKHOUSE-2]. 2017-07-12 19:51:39 +03:00
Alexey Zatelepin
5e61a56743 always initialize trie ptr (just in case) [#CLICKHOUSE-3087] 2017-07-12 16:35:05 +03:00
Alexey Zatelepin
4d4387f1c8 fix reloading of TrieDictionary [#CLICKHOUSE-3087] 2017-07-12 16:35:05 +03:00
Alexey Milovidov
bb38ec2c21 Fixed discrepancy in file names [#CLICKHOUSE-2]. 2017-07-12 04:16:01 +03:00
Alexey Milovidov
1048fccf6b Misc [#CLICKHOUSE-2]. 2017-07-12 03:49:30 +03:00
Alexey Milovidov
24331a18f5 Removing dependency on functions from client: bug fixes [#CLICKHOUSE-2]. 2017-07-12 03:42:50 +03:00
Alexey Milovidov
976e47e1af Removing dependency on functions from client: preparation [#CLICKHOUSE-2]. 2017-07-12 03:42:50 +03:00
Alexey Milovidov
8c1a6f5736 Removing dependency on functions from client: preparation [#CLICKHOUSE-2]. 2017-07-12 03:42:50 +03:00
Alexey Milovidov
33c6e0b804 Removing dependency on functions from client: preparation [#CLICKHOUSE-2]. 2017-07-12 03:42:50 +03:00
Alexey Milovidov
3e3af1cff4 Misc [#CLICKHOUSE-2]. 2017-07-11 23:12:15 +03:00
Alexey Milovidov
fb8d8ec768 Fixed error (tnx Vladimir Chebotarev) [#CLICKHOUSE-2]. 2017-07-11 21:59:01 +03:00
Vitaliy Lyudvichenko
b843ea5b3f Resolves #964. Fixed input_format_skip_unknown_fields for negative numbers. [#CLICKHOUSE-3] 2017-07-11 21:36:51 +03:00
Alexey Zatelepin
7892129481 peformance-test: fix rows_read, bytes_read calculation [#CLICKHOUSE-3086] 2017-07-11 21:26:05 +03:00
Nikolai Kochetov
ee66169d5e added throw_if_column_not_found to FilterColumnsBlockInputStream [#CLICKHOUSE-3111] 2017-07-11 20:58:49 +03:00
Alexey Zatelepin
57ba50f585 allow batching Distributed inserts [#CLICKHOUSE-3126] 2017-07-11 20:56:17 +03:00
Alexey Zatelepin
4a98d83a3b introduce replicated_can_become_leader merge_tree setting [#CLICKHOUSE-3109] 2017-07-11 20:44:35 +03:00
Alexey Milovidov
cefc1a58b6 Merge branch 'master' of github.com:yandex/ClickHouse 2017-07-11 04:20:48 +03:00
Alexey Milovidov
2bd84600d1 Miscellaneous [#CLICKHOUSE-2]. 2017-07-11 04:20:40 +03:00
Nikolai Kochetov
524507b9b8 fixed setting comparison 2017-07-10 21:15:56 +03:00
Nikolai Kochetov
c06c234c63 added deleted comment after merge 2017-07-10 17:33:52 +03:00
Nikolai Kochetov
7d5590d66d Merge branch 'master' into output-format-json-quote-denormals 2017-07-10 17:29:17 +03:00
Nikolai Kochetov
39797813cc disabled check for settings with same value in readonly mode in InterpreterSetQuery [#CLICKHOUSE-3041]; added tests for output_format_json_quote_denormals setting 2017-07-10 17:03:34 +03:00
Alexey Milovidov
86eb4d2d06 Renamed Singleton to avoid conflicts with Arcadia* codebase [#DEVTOOLS-3343].
* - "Arcadia" is the name of Yandex's monorepo.
2017-07-10 07:34:14 +03:00
Alexey Milovidov
16a6eee397 Fixed build after merge [#CLICKHOUSE-3]. 2017-07-10 07:22:36 +03:00
Alexey Milovidov
1040abfd18 Fixed build after merge [#CLICKHOUSE-3]. 2017-07-10 07:16:11 +03:00
Alexey Milovidov
a742ac3b6c Fixed build after merge [#CLICKHOUSE-3]. 2017-07-10 07:10:39 +03:00
Alexey Milovidov
78e905aa3a Fixed build after merge [#CLICKHOUSE-3]. 2017-07-10 07:10:25 +03:00
alexey-milovidov
b64b243a73 Merge pull request #945 from YiuRULE/master
Implement natively uuid type in Clickhouse
2017-07-10 07:00:45 +03:00
Alexey Milovidov
119961133c Removed obsolete test [#CLICKHOUSE-3]. 2017-07-10 06:47:20 +03:00
Alexey Milovidov
bf58b8cdb1 Fixed test [#CLICKHOUSE-3]. 2017-07-10 06:45:04 +03:00
Alexey Milovidov
db3d56b1f5 Removed obsolete test binary [#CLICKHOUSE-3]. 2017-07-10 06:43:25 +03:00
Alexey Zatelepin
89939a685a Stream row sources from disk for vertical merge [#CLICKHOUSE-3118] 2017-07-09 21:57:06 +03:00
Alexey Zatelepin
1cd936579d introduce IColumn::gather() [#CLICKHOUSE-3118] 2017-07-09 21:57:06 +03:00
Nikolai Kochetov
d804855460 added FormatSettingsJSON.h 2017-07-09 20:16:26 +03:00
Nikolai Kochetov
25ac546e78 disabled exception during overriding settings with same value in readonly mode [#CLICKHOUSE-3041] 2017-07-09 20:16:26 +03:00
Nikolai Kochetov
63119eda7d updated FormatSettingsJSON 2017-07-09 20:16:26 +03:00
Nikolai Kochetov
0ead0ee2f1 added FormatSettingsJSON [#CLICKHOUSE-3060] 2017-07-09 20:16:26 +03:00
alexey-milovidov
894d09f797 Update IFunction.h 2017-07-09 17:13:51 +03:00
Marek Vavruša
ff73394124 PKCondition: infer index use with pk subexpression
By default only constraints explicitly matching
primary key expression (or expression wrapped in
a monotonic function) are eligible for part and
range selection. So for example, if index is:

(toStartOfHour(dt), UserID)

Then a query such as this resorts to full scan:

SELECT count() FROM t WHERE dt = now()

Intuitively, only parts with toStartOfHour(now())
could be selected, but it is less trivial to prove.
The primary key currently can be wrapped in a chain
of monotonic functions, so following would work:

toStartOfHour(dt) = toStartOfHour(now()) AND dt = now()

It must be however explicitly stated, if we wanted
to infer that we’d have to know the inverse function,
and prove that the inverse function is monotonic
on given interval. This is not practical as
there is no inverse function that for example undos
rounding, it isn’t strictly monotonic.

There are however functions that don’t transform
output range and preserve monotonicity on the
complete input range, such as rounding or casts
to a same or wider numeric type. This eliminates
the need to find inverse function, as no check for monotonicity over arbitrary interval is needed,
and thus makes this optimisation possible.
2017-07-09 17:13:51 +03:00
Vitaliy Lyudvichenko
b46a2a6eab Fixed infinite sleeping of low-priority queries. [#CLICKHOUSE-2] 2017-07-09 03:53:13 +03:00
Vitaliy Lyudvichenko
7150e9447a Fixed multiIf in case of empty arguments. [#CLICKHOUSE-3119] 2017-07-07 17:42:37 +03:00
Nikolai Kochetov
2f08ba9e94 added FormatSettingsJSON.h 2017-07-06 18:11:12 +03:00
Guillaume Tassery
2ec96fe822 Make UUID as a strong type 2017-07-06 16:50:29 +02:00
Nikolai Kochetov
d6001618f0 disabled exception during overriding settings with same value in readonly mode [#CLICKHOUSE-3041] 2017-07-06 17:42:31 +03:00
Liu Cong
08e43b149a Fix a typo of 'Columns/ColumnVector.h': metod => method 2017-07-05 20:46:37 +03:00
Nikolai Kochetov
c99a9b1c55 moved FilterColumnsBlockInputStream implementation to .cpp [#CLICKHOUSE-3111] 2017-07-05 20:14:17 +03:00
alexey-milovidov
85bdb472ea Update FilterColumnsBlockInputStream.h 2017-07-05 20:14:17 +03:00