thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-11-18 13:42:02 +00:00
Code Issues Packages Projects Releases Wiki Activity
30,046 Commits 9,138 Branches 1,774 Tags 2 GiB
b9a5952921
Commit Graph

520 Commits

Author SHA1 Message Date
Alexey Milovidov
ff07ffbbae Merge branch 'merging_redis' of https://github.com/CurtizJ/ClickHouse into CurtizJ-merging_redis 2019-09-25 02:27:05 +03:00
Alexander Kuzmenkov
f3bde19b74 Do not use iterators in find() and emplace() methods of hash tables. 
Instead, these methods return a pointer to the required data as they are
stored inside the hash table. The caller uses overloaded functions to
get the key and "mapped" values from this pointer. Such an interface
avoids the need for constructing iterator-like wrapper objects, which is
especially important for compound hash tables such as the future
StringHashMap.
 2019-09-24 17:44:35 +03:00
Azat Khuzhin
52f14c1ead
Update HashedDictionary.cpp (fixed rename in return type for SFINAE) 2019-09-22 10:39:50 +03:00
alexey-milovidov
a12cffee91
Update HashedDictionary.h 2019-09-22 05:11:08 +03:00
alexey-milovidov
ab9a7be45a
Comments are the must. 2019-09-22 05:09:40 +03:00
alexey-milovidov
9afea6f0b3
Update HashedDictionary.cpp 2019-09-22 05:04:49 +03:00
Azat Khuzhin
420089c301
Add new dictionary layout (sparse_hashed) that is more memory efficient 
With this new layout, sparsehash will be used over default HashMap,
sparsehash is more memory efficient but it is also slower.

So in a nutshell:
- HashMap uses ~2x more memory then sparse_hash_map
- HashMap ~2-2.5x faster then sparse_hash_map
(tested on lots of input, and the most close to production was
dictionary with 600KK hashes and UInt16 as value)

TODO:
- fix allocated memory calculation
- getBufferSizeInBytes/getBufferSizeInCells interface
- benchmarks

v0: replace HashMap with google::sparse_hash_map
v2: use google::sparse_hash_map only when <sparse> isset to true
v3: replace attributes with different layout
v4: use ch hash over std::hash
 2019-09-21 02:22:40 +03:00
CurtizJ
79a08ef8ea Merge remote-tracking branch 'upstream/master' into merging_redis 2019-09-18 12:58:22 +03:00
CurtizJ
7ccf04440a better code in Redis external dictionary 2019-09-17 20:57:48 +03:00
CurtizJ
df82e4bde8 fix build 2019-09-17 17:55:09 +03:00
CurtizJ
4480e97f9f fix build 2019-09-17 17:16:07 +03:00
CurtizJ
4df1f1bb9a better integration test for redis dictionary (but still bad) 2019-09-17 16:35:19 +03:00
CurtizJ
e9336c9166 improvements of redis external dictionary 2019-09-16 19:17:56 +03:00
CurtizJ
ad986f285e fix redis with mixed keys 2019-09-13 20:38:56 +03:00
CurtizJ
fa416dc941 fix Redis dictionary 2019-09-12 17:48:28 +03:00
Amos Bird
a1310ffbe6 Split libdbms.so using object library 
Now the linking time of incremental builds are around 1-2 seconds
 2019-09-11 01:56:01 +08:00
CurtizJ
3b3657ff1e Merge branch 'add-ext-dict-redis' of https://github.com/comunodi/ClickHouse into merging_redis 2019-09-09 13:51:50 +03:00
Vitaly Baranov
da8f67123f Improve error handling in cache dictionaries: 
allow using expired values while the source of a cache dictionary doesn't respond;
clone the source after an error to reset connections;
don't ask the source for a little time after error;
show the last exception in system.dictionaries for a cache dictionary too.
 2019-08-30 12:50:38 +03:00
Ivan
b4339f266d
Make a better build scheme (#6500) 
* Fix shared build
* Major default libs refactor
* Fix build with gcc_eh
* Link all libraries as a big group.
* Use global interface library as a group
* Build capnproto using our cmake
* Use only internal libunwind
 2019-08-28 23:49:37 +03:00
Alexey Milovidov
8966520801 Added "assert_cast" function as a replacement for static_cast for columns 2019-08-21 05:28:04 +03:00
Alexey Milovidov
09ecd865fc Allow to use library dictionary source with ASan. May impose troubles. 2019-08-14 03:26:38 +03:00
alesapin
2ca6c0b1bf Fix bug with memory allocation for string fields in complex key cache dictionary 2019-08-12 12:37:48 +03:00
kreuzerkrieg
112fc71276 adding -Wshadow for GCC 2019-08-09 23:58:16 +03:00
Alexey Milovidov
389cf9fcee Better fix: removed old and bad code #6069 2019-07-20 02:10:55 +03:00
Alexey Milovidov
21165edcb7 Quick fix for initial load of external dictionaries via ODBC 2019-07-20 02:06:15 +03:00
Vitaly Baranov
6d563c51e6 Remove unnecessary try-catch in the constructors of classes derived from IExternalLoadable. 2019-07-17 11:39:36 +03:00
Nikolai Kochetov
e9f3d4d6b8
Merge pull request #4914 from yandex/processors 
Processors
 2019-07-11 19:22:02 +03:00
Nikolai Kochetov
209793c981 Merged with master. 2019-07-08 16:00:54 +03:00
Alexey Milovidov
553e6a273e Two more fixes for PVS-Studio 2019-07-08 03:51:43 +03:00
Alexey Milovidov
3c4784a466 Removed obsolete undocumented embedded dictionaries that were used by Yandex.Metrica 2019-06-27 22:18:28 +03:00
Nikolai Kochetov
f9b29bfdd2 Merged with master. 2019-06-25 20:00:54 +03:00
Nikolai Kochetov
93b9f3e06a Disable processors for ClickHouseDictionarySource. 2019-06-25 19:41:06 +03:00
Alexey Milovidov
8f77d898f3 Renamed functions for consistency #5704 2019-06-24 17:51:37 +03:00
alexey-milovidov
8e41d89126
Merge pull request #5704 from zhang2014/fix/ISSUES-5697 
ISSUES-5697 fix insert and select query with mysql style identifier
 2019-06-24 17:40:28 +03:00
proller
1ac09fa721
Split Common/config.h by libs (dbms) (#5715) 2019-06-24 14:17:15 +03:00
zhang2014
1786407621 ISSUES-5697 fix insert and select query with mysql style identifier 2019-06-21 13:51:27 +08:00
alesapin
e45d727e9d
Merge pull request #4558 from podshumok/distributred-timeout 
Distributed and remote timeouts
 2019-06-17 13:58:17 +03:00
alexey-milovidov
39c6fbc324
Merge pull request #5567 from vitlibar/really-lazy-dictionary-load 
Non-blocking loading of external dictionaries
 2019-06-15 22:15:53 +03:00
Alexey Milovidov
b2100c7bcf Removed strange quoting style like `this'. See https://english.stackexchange.com/questions/17695/any-reference-on-the-usage-of-a-backtick-and-single-quotation-mark-like-this 
Added backQuote and backQuoteIfNeed when required to correctly quote identifiers with proper escaping.
 2019-06-15 15:06:22 +03:00
Vitaly Baranov
b4384ce2a9 Dictionaries can be loaded in parallel. 
A bad dictionary cannot block all the loading anymore.
Implemented really lazy loading of external dictionaries.
Provided more detailed information about the loading of each dictionary
to make diagnostics easier.
 2019-06-14 22:18:47 +03:00
Alexey Milovidov
c846d0f6af Namespace comments are unneeded according to the code style. 
find dbms -name '*.h' -or -name '*.cpp' | xargs grep -l -P '}\s*//+\s*namespace\s*' | xargs sed -i -r -e 's/}\s*\/\/+\s*namespace\s*.*$/}/'
 2019-06-13 13:43:37 +03:00
Konstantin Podshumok
e60a2fd1ad clickhouse dictionary source: fix timeouts for remotes 2019-06-02 12:43:20 +03:00
comunodi
08c2f183dd Fix complex dict with two keys 2019-06-02 04:22:06 +03:00
comunodi
a964af386c Optimize includes 2019-05-31 00:16:12 +03:00
comunodi
c58effc2af Fix inserting keys with hash map 2019-05-31 00:07:34 +03:00
chertus
3f2e09ac93 Merge branch 'master' into decimal 2019-05-29 13:58:33 +03:00
comunodi
559628575a Merge branch 'master' into add-ext-dict-redis 
# Conflicts:
#	dbms/tests/integration/image/Dockerfile
 2019-05-28 23:20:41 +03:00
comunodi
a8ce7530c9 Put keys in result block only if value exists 2019-05-28 23:06:06 +03:00
chertus
49e12eaa60 remove unused specializations in dictionaries 2019-05-28 22:30:01 +03:00
alexey-milovidov
c5d18dfde1
Merge pull request #5446 from 4ertus2/some 
Disable implicit type conversions in dictGet functions (it's not working)
 2019-05-28 20:34:13 +03:00