Amos Bird
fbfbe161ec
Unify hash tables interface.
2019-11-06 00:08:04 +08:00
Alexander Kuzmenkov
f3bde19b74
Do not use iterators in find() and emplace() methods of hash tables.
...
Instead, these methods return a pointer to the required data as they are
stored inside the hash table. The caller uses overloaded functions to
get the key and "mapped" values from this pointer. Such an interface
avoids the need for constructing iterator-like wrapper objects, which is
especially important for compound hash tables such as the future
StringHashMap.
2019-09-24 17:44:35 +03:00
Alexander Kuzmenkov
160e0faf77
Do not use iterator::getValue() in Aggregator.
...
Use separate key and "mapped" value references instead. This is
important for hash tables that do not store the key/"mapped" pair
directly, and cannot provide this interface without some runtime
overhead.
2019-09-10 20:12:30 +03:00
Alexander Kuzmenkov
cacf6d3282
Key memory management for compound hash tables.
...
Some aggregation methods initially emplace a temporary StringRef key
into a hash table. Then, if the key was not seen before, they make a
persistent copy of the key and update the hash table with it. This
approach is not suitable for compound hash tables, because the logic of
when the persistent key is needed is more complex, and is contained
within the hash table itself.
In this commit, we switch to managing key memory with callbacks passed
to the hash table, that allow it to request a persistent copy of the key
if it is needed. This should be more appropriate for compound hash
tables.
This commit prepares for StringHashMap PR #5417 .
2019-09-06 18:05:28 +03:00
Alexey Milovidov
8966520801
Added "assert_cast" function as a replacement for static_cast for columns
2019-08-21 05:28:04 +03:00
kreuzerkrieg
112fc71276
adding -Wshadow for GCC
2019-08-09 23:58:16 +03:00
Alexander Kuzmenkov
011150fa8a
Make PairNoInit a simple struct.
2019-08-01 19:54:33 +03:00
Martijn Bakker
b0fd915724
remove the ASOFSplit struct
2019-03-28 18:35:50 +00:00
Amos Bird
26ab5dd7a7
A Proper lookup table that uses HashTable's API
...
This is the first step of allowing heterogeneous cells in hash tables.
performance test results are
```
1. HashMap<UInt16, UInt8, TrivialHash, HashTableFixedGrower<16>>;
2. NewLookupMap<UInt16, UInt8>
ResolutionWidth 30000 1 .................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................223550276.46
ResolutionWidth 30000 2 .................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................248772721.24
Best: 2 - 24877272124
ResolutionWidth 100000 1 ..........................................................................................................................................................................................................................................................238498413.99
ResolutionWidth 100000 2 ..........................................................................................................................................................................................................................................................261808889.98
Best: 2 - 26180888998
ResolutionWidth 300000 1 ...................................................................................239307348.81
ResolutionWidth 300000 2 ...................................................................................257592761.30
Best: 2 - 25759276130
ResolutionWidth 1000000 1 .........................240144759.26
ResolutionWidth 1000000 2 .........................257093531.91
Best: 2 - 25709353191
ResolutionWidth 5000000 1 .....241573260.35
ResolutionWidth 5000000 2 .....259314162.79
Best: 2 - 25931416279
ResolutionDepth 30000 1 .................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................217108119.84
ResolutionDepth 30000 2 .................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................249459504.41
Best: 2 - 24945950441
ResolutionDepth 100000 1 ..........................................................................................................................................................................................................................................................229065162.17
ResolutionDepth 100000 2 ..........................................................................................................................................................................................................................................................253769105.64
Best: 2 - 25376910564
ResolutionDepth 300000 1 ...................................................................................233079225.18
ResolutionDepth 300000 2 ...................................................................................256316273.78
Best: 2 - 25631627378
ResolutionDepth 1000000 1 .........................234184633.51
ResolutionDepth 1000000 2 .........................261100491.57
Best: 2 - 26110049157
ResolutionDepth 5000000 1 .....233118795.66
ResolutionDepth 5000000 2 .....252436160.41
Best: 2 - 25243616041
```
2019-03-01 16:47:13 +08:00
Nikolai Kochetov
a73f29ca2e
Fix LowCardinality cache.
2019-02-12 17:38:29 +03:00
alexey-milovidov
ba01f7fd77
Update ColumnsHashing.h
2019-02-06 19:58:27 +03:00
Nikolai Kochetov
a249fc7788
Do nut use Arena in ColumnsHashing for string in functions arrayUniq and arrayEnumerate.
2019-02-05 14:14:09 +03:00
Nikolai Kochetov
daa051b1f8
Fix tests.
2019-02-05 13:55:34 +03:00
Nikolai Kochetov
8db4116464
Update HashMethodHashed.
2019-02-05 13:55:08 +03:00
Nikolai Kochetov
d143a78419
Refactor ColumnsHashing
2019-02-01 11:23:38 +03:00
Nikolai Kochetov
5f1007a8df
Refactor Join.
2019-01-24 17:56:04 +03:00
Nikolai Kochetov
6fce028b56
Refactor ColumnsHashing.
2019-01-21 13:39:53 +03:00
Nikolai Kochetov
d207498573
Added ColumnsHashing
2019-01-21 13:39:24 +03:00
