chertus
3b0d870804
rename function
2019-10-02 15:06:34 +03:00
chertus
c46cc3ff1a
refactoring: remove copy-paste
2019-10-01 21:51:33 +03:00
Alexander Kuzmenkov
f3bde19b74
Do not use iterators in find() and emplace() methods of hash tables.
...
Instead, these methods return a pointer to the required data as they are
stored inside the hash table. The caller uses overloaded functions to
get the key and "mapped" values from this pointer. Such an interface
avoids the need for constructing iterator-like wrapper objects, which is
especially important for compound hash tables such as the future
StringHashMap.
2019-09-24 17:44:35 +03:00
Nikolai Kochetov
1319168a3a
Merge branch 'master' into processors-2
2019-09-12 16:28:51 +03:00
Amos Bird
8f28a36c17
Use internal iteration over hash tables in Aggregator.
...
For compound hash tables such as the future StringHashMap, an
iterator-based API might be inefficient for iterating over a table or
for merging two tables, because:
1) the key has to be converted to a general format from a from a
component-specific format, which may differ between the components;
2) the information about the component of the compound hash table to
which the value belongs is lost, and has to be recalculated if the
value is reinserted.
A more efficient approach is to use internal iteration, that is,
map-like functions, which avoids unnecessary conversions when iterating,
and allows to use an efficient component-wise approach when merging.
2019-09-10 21:52:53 +03:00
Alexander Kuzmenkov
160e0faf77
Do not use iterator::getValue() in Aggregator.
...
Use separate key and "mapped" value references instead. This is
important for hash tables that do not store the key/"mapped" pair
directly, and cannot provide this interface without some runtime
overhead.
2019-09-10 20:12:30 +03:00
Nikolai Kochetov
52305cefbc
Merge branch 'merging-aggregated-processors' into processors-2
2019-09-06 15:21:17 +03:00
Nikolai Kochetov
d10b7d7a2b
Added ConvertingAggregatedToChunksTransform.
2019-09-06 15:19:59 +03:00
Nikolai Kochetov
bf0c0ed4f1
Merge branch 'master' into processors-2
2019-09-02 16:02:09 +03:00
Nikolai Kochetov
300dc80194
Update Aggregator interface.
2019-08-31 11:58:16 +03:00
Alexey Milovidov
b21fdff77e
Merging #6678
2019-08-27 21:59:21 +03:00
Weiqing Xu
2fbcd607f3
check free space when use external sort/aggerator
2019-08-27 08:23:07 +08:00
Alexey Milovidov
cdd6dca517
Remove Compiler
2019-08-24 00:10:26 +03:00
Alexey Milovidov
8966520801
Added "assert_cast" function as a replacement for static_cast for columns
2019-08-21 05:28:04 +03:00
Alexey Milovidov
2570907f44
Removed useless parameter
2019-08-12 00:45:18 +03:00
Alexey Milovidov
f00fa640ba
Removed old optimization that is now dominated by the new optimization
2019-08-11 02:40:15 +03:00
Alexey Milovidov
011e50cae6
Removed useless function arguments
2019-08-11 02:27:25 +03:00
Alexey Milovidov
3885cc20b8
Improved performance of aggregation without key
2019-08-11 02:22:23 +03:00
Alexey Milovidov
efa51a6cd9
Batch aggregation (experimental)
2019-08-11 01:36:55 +03:00
kreuzerkrieg
112fc71276
adding -Wshadow for GCC
2019-08-09 23:58:16 +03:00
Nikolai Kochetov
209793c981
Merged with master.
2019-07-08 16:00:54 +03:00
chertus
f19dee706a
move IColumn.isColumnConst() outside of interface
2019-06-27 22:28:52 +03:00
Nikolai Kochetov
33b176f2da
Fix style.
2019-04-05 14:34:11 +03:00
Nikolai Kochetov
5e39ec610c
Merged with master
2019-04-03 14:21:38 +03:00
Nikolai Kochetov
18133a4948
Added processors_test_aggregation.
2019-04-03 13:13:25 +03:00
Nikolai Kochetov
6976b65b2d
Added MergingAggregatedTransform.
2019-04-03 13:13:25 +03:00
proller
1a33f4da40
Optional current_thread and memory_tracker ( #4664 )
2019-03-14 21:03:35 +03:00
Amos Bird
26ab5dd7a7
A Proper lookup table that uses HashTable's API
...
This is the first step of allowing heterogeneous cells in hash tables.
performance test results are
```
1. HashMap<UInt16, UInt8, TrivialHash, HashTableFixedGrower<16>>;
2. NewLookupMap<UInt16, UInt8>
ResolutionWidth 30000 1 .................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................223550276.46
ResolutionWidth 30000 2 .................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................248772721.24
Best: 2 - 24877272124
ResolutionWidth 100000 1 ..........................................................................................................................................................................................................................................................238498413.99
ResolutionWidth 100000 2 ..........................................................................................................................................................................................................................................................261808889.98
Best: 2 - 26180888998
ResolutionWidth 300000 1 ...................................................................................239307348.81
ResolutionWidth 300000 2 ...................................................................................257592761.30
Best: 2 - 25759276130
ResolutionWidth 1000000 1 .........................240144759.26
ResolutionWidth 1000000 2 .........................257093531.91
Best: 2 - 25709353191
ResolutionWidth 5000000 1 .....241573260.35
ResolutionWidth 5000000 2 .....259314162.79
Best: 2 - 25931416279
ResolutionDepth 30000 1 .................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................217108119.84
ResolutionDepth 30000 2 .................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................249459504.41
Best: 2 - 24945950441
ResolutionDepth 100000 1 ..........................................................................................................................................................................................................................................................229065162.17
ResolutionDepth 100000 2 ..........................................................................................................................................................................................................................................................253769105.64
Best: 2 - 25376910564
ResolutionDepth 300000 1 ...................................................................................233079225.18
ResolutionDepth 300000 2 ...................................................................................256316273.78
Best: 2 - 25631627378
ResolutionDepth 1000000 1 .........................234184633.51
ResolutionDepth 1000000 2 .........................261100491.57
Best: 2 - 26110049157
ResolutionDepth 5000000 1 .....233118795.66
ResolutionDepth 5000000 2 .....252436160.41
Best: 2 - 25243616041
```
2019-03-01 16:47:13 +08:00
Alexey Milovidov
277484297a
Aggregator: miscellaneous
2019-02-08 19:54:04 +03:00
Nikolai Kochetov
6a729e59ba
Merged with master.
2019-02-01 11:36:57 +03:00
alexey-milovidov
fd5ab9ed2e
Merge pull request #4055 from yandex/fix-group-by-array-low-cardibality-arguments
...
Fix group by array low cardinality arguments
2019-01-29 21:32:45 +03:00
Nikolai Kochetov
21242ad658
Added tests. Restored recursiveRemoveLowCardinality interface.
2019-01-29 18:44:02 +03:00
Nikolai Kochetov
f7b9059dad
Merged with master
2019-01-24 18:06:11 +03:00
Ivan Lezhankin
05ab15cc84
Merge IBlockInputStream and IProfilingBlockInputStream into one
2019-01-23 17:48:50 +03:00
Nikolai Kochetov
6fce028b56
Refactor ColumnsHashing.
2019-01-21 13:39:53 +03:00
Nikolai Kochetov
d207498573
Added ColumnsHashing
2019-01-21 13:39:24 +03:00
Nikolai Kochetov
5a50a4fe21
Fix aggregate function low cardinality array argument. #4038
2019-01-14 21:17:55 +03:00
Alexey Milovidov
1d4701b0bc
Added some warnings from clang's -Weverything
2019-01-04 15:10:00 +03:00
Alexey Milovidov
e33e5150b7
Miscellaneous [#CLICKHOUSE-2]
2019-01-02 09:44:36 +03:00
proller
300bbdfb43
Fix cyclic lib depend ppart 2: clean
2018-12-28 21:15:26 +03:00
Alexey Milovidov
a8f09809f5
Simplified logic with "IColumn::convertToFullColumnIfConst" (suggested by Amos Bird) [#CLICKHOUSE-2]
2018-12-21 19:00:07 +03:00
Nikolai Kochetov
4e58b62b0c
Added LowCardinality support for specialized aggregation.
2018-12-20 16:28:20 +03:00
Nikolai Kochetov
b87d81512f
Fix merging aggregate data for NULL key.
2018-12-04 13:31:15 +03:00
Nikolai Kochetov
730796dc38
Fix merging aggregate data for NULL key.
2018-12-04 13:29:27 +03:00
Nikolai Kochetov
de524a0158
Fix merging aggregate data for NULL key.
2018-12-04 13:27:44 +03:00
Nikolai Kochetov
98499b1140
LowCardinality optimization for single nullable column.
2018-12-03 21:00:20 +03:00
Marek Vavruša
8fad726d95
Added null guards for all CurrentThread::attachToIfDetached()
...
refs #3622
2018-11-20 09:10:34 -08:00
Alexey Milovidov
c5173782dc
Potentially better #3490
2018-11-05 17:29:38 +03:00
alexey-milovidov
00fa7d9fad
Update Aggregator.cpp
2018-11-01 15:42:57 +03:00
igor.lapko
2d565f1099
code style fix
2018-11-01 13:42:15 +02:00
