achulkov2
8160b2af6a
Compilation errors
2019-12-23 16:43:12 +03:00
achulkov2
c339acddf9
Compilation errors
2019-12-23 16:41:35 +03:00
achulkov2
bd13a888f5
Compilation errors
2019-12-23 16:40:42 +03:00
achulkov2
fcfe2fb234
Compilation errors
2019-12-23 16:39:17 +03:00
achulkov2
a8fec15b50
First implementation of dictGet
2019-12-23 16:23:11 +03:00
Alexander Burmak
0894ed9f33
Integration of log storages with IDisk interface
2019-12-23 16:11:56 +03:00
Nikita Mikhaylov
534d47e0d4
Merge branch 'master' of github.com:ClickHouse/ClickHouse into cache-dictionary
2019-12-23 15:24:33 +03:00
Nikita Mikhaylov
d93b0d5030
more
2019-12-23 14:47:20 +03:00
Nikolai Kochetov
68305165d1
Create query context for clickhouse dictionary source.
2019-12-23 12:25:58 +03:00
Alexey Milovidov
41ed1ed471
Added quoting of db and table names where appropriate
2019-12-20 19:53:37 +03:00
Nikita Mikhaylov
be5729e113
mvc
2019-12-19 21:22:04 +03:00
alesapin
e6cc87fc20
Fix broken unit test
2019-12-19 15:31:10 +03:00
alesapin
742caced3f
Add ability to work with different types besides Date in RangeHashed external dictionary created from DDL query.
2019-12-18 19:26:46 +03:00
alexey-milovidov
a7384b4d07
Merge branch 'master' into MoreGCCWarnings
2019-12-18 02:01:39 +03:00
kreuzerkrieg
9612bb0e79
More GCC warnings
2019-12-16 23:19:50 +02:00
alesapin
a2180b0bb3
Merge branch 'master' into restrictions_for_external_dictionaries_from_ddl
2019-12-16 23:07:16 +03:00
achulkov2
76ff03973c
More compilation errors
2019-12-16 18:46:51 +03:00
achulkov2
e9d2a1a8ad
More compilation errors
2019-12-16 18:34:46 +03:00
achulkov2
3f7fba6a78
Fix compilation errors
2019-12-16 18:24:26 +03:00
achulkov2
331e45a970
Fixes
2019-12-16 18:11:16 +03:00
achulkov2
66341cc40f
Temporary: will be renamed later
2019-12-16 17:31:35 +03:00
alexey-milovidov
575984b4a0
Merge pull request #8189 from ClickHouse/regions-more-languages
...
More languages for regions
2019-12-16 02:47:20 +03:00
Alexey Milovidov
df5f1f7433
More languages for regions
2019-12-12 22:34:58 +03:00
Vitaly Baranov
4c157007f5
Refactoring of using ExternalLoader in dictionary DDL:
...
Instead of using ExternalLoader::reload() now it's used reloadConfig() which reloads only what necessary.
Functions attachDictionary() and detachDictionary() are simplified and have lesser number of parameters.
Instead of injecting into LoadablesConfigReader's internals for creating dictionary a temp repository is used.
2019-12-12 21:45:58 +03:00
alesapin
6842ff02c9
Fix shared build
2019-12-10 23:39:36 +03:00
alesapin
ea8994b9e4
Add restrections for several sources of external dictionaries
2019-12-10 20:27:29 +03:00
alesapin
85aac386aa
Missed commit
2019-12-09 17:06:28 +03:00
alesapin
642ce2d751
Fix escaping and add string expressions support
2019-12-09 17:05:58 +03:00
alexey-milovidov
d051a03794
Merge pull request #8042 from excitoon-favorites/gmtime
...
Removed `localtime` from `HTTPDictionarySource::getUpdateFieldAndDate…
2019-12-09 04:57:18 +03:00
Vitaly Baranov
c733e5b50b
Add functions for writing std::chrono to string.
2019-12-05 21:03:57 +03:00
Vladimir Chebotarev
3c35b3db60
Removed localtime
from HTTPDictionarySource::getUpdateFieldAndDate()
.
2019-12-05 12:18:51 +03:00
achulkov2
6c6db598b1
Very small refactoring for external dictionaries
2019-12-02 18:59:45 +03:00
tavplubix
b75b6c294d
Merge pull request #7762 from ClickHouse/possible_segfaults_in_table_functions
...
Fix Storage lifetime in table functions
2019-11-22 02:29:06 +03:00
Alexander Tokmakov
2173bf5f4a
Merge branch 'master' into possible_segfaults_in_table_functions
2019-11-17 16:48:13 +03:00
Vitaly Baranov
ff6cba192b
Replace pointer with reference in SettingsCollection<T>::MemberInfos::findStrict().
...
Rename SettingsCommon.h -> SettingsCollection.h for consistency.
2019-11-17 03:51:51 +03:00
Alexander Tokmakov
55f60a6bad
hold shared pointers to table function Storage and Context from InterpreterSelectQuery
2019-11-15 21:42:32 +03:00
Alexey Milovidov
f9362a241f
Minor changes in RegionsNames
2019-11-11 23:54:57 +03:00
Amos Bird
fbfbe161ec
Unify hash tables interface.
2019-11-06 00:08:04 +08:00
alesapin
c12014ca15
Fix shared build
2019-10-22 19:47:11 +03:00
alesapin
4c5d150d7a
Review fixes
2019-10-22 13:47:43 +03:00
alesapin
83dbfe9634
New tests and fixes for them
2019-10-21 20:09:41 +03:00
alesapin
10f69eef03
Add tests for non standart dictionaries and fix bugs
2019-10-21 19:05:45 +03:00
alesapin
f7043c38bd
Less diff
2019-10-21 17:20:42 +03:00
alesapin
17c2f301e2
Remove accident cerr
2019-10-21 17:00:18 +03:00
alesapin
3987a45ea2
Fix build and style
2019-10-17 12:05:48 +03:00
alesapin
b4e0ded048
Almost working
2019-10-16 17:59:52 +03:00
alesapin
e690a3ca32
Half working code
2019-10-15 21:04:17 +03:00
alesapin
3e068b81de
Add loading of external dictionaries
2019-10-15 17:09:57 +03:00
alesapin
f850408ae4
Convert AST to Abstract configuration
2019-10-15 15:16:19 +03:00
alesapin
1ce0eb6e63
Appropriate error codes numbers
2019-10-14 13:04:10 +03:00
alesapin
2a5713e6e4
Compilable code
2019-10-10 23:47:47 +03:00
alesapin
67fc3347c2
Preparing databases for dictionaries
2019-10-10 20:33:01 +03:00
alexey-milovidov
e3861e0e88
Merge pull request #7242 from azat/sparse_hashed-name
...
Return SparseHashed name (system.dictionaries:type) for the sparse_hashed layout
2019-10-09 20:12:24 +03:00
Azat Khuzhin
0ff823b574
Return SparseHashed name (system.dictionaries:type) for the sparse_hashed layout
...
Due to tons of rebasing this bit had been forgotten.
Refs: 420089c301
("Add new dictionary layout (sparse_hashed) that is more memory efficient")
2019-10-09 11:11:41 +03:00
Alexander Kuzmenkov
2b30f2696c
Include Field.h in less files.
2019-10-08 16:23:24 +03:00
alesapin
cf094cd62b
Merge pull request #7155 from ClickHouse/dictionaries_ddl
...
Preparation of ExternalLoader for Dictionaries DDL
2019-10-03 20:12:19 +03:00
Alexander Kuzmenkov
8e245d4e8c
Merge pull request #7135 from ClickHouse/aku/msan-field
...
Fix some MemorySanitizer warnings for Fields.
2019-10-03 10:33:44 +03:00
alesapin
08681ac210
Preparation for AST configurations for external dictionaries
2019-09-30 19:12:08 +03:00
Alexander Kuzmenkov
ec2d9a083c
Remove excessive includes.
2019-09-30 15:31:14 +03:00
alexey-milovidov
42c9ea9aa3
Merge pull request #7092 from PerformanceVision/acl_file_storage
...
Let the choice to write credential for HTTP source on external dictionaries
2019-09-30 13:20:33 +03:00
alesapin
0e28dc4e51
Remove redundant classes
2019-09-26 19:12:15 +03:00
alesapin
903f826640
Remove runtime factory and remove one redundant object
2019-09-26 13:08:38 +03:00
Guillaume Tassery
647568a6f6
cosmetic
2019-09-26 05:34:22 +02:00
Guillaume Tassery
367a0dcdb4
cosmetic
2019-09-25 11:46:48 +02:00
Guillaume Tassery
8b4e789847
Rename key to name
2019-09-25 11:42:08 +02:00
Guillaume Tassery
77f2dfee6a
Add the positibility to add an header on an HTTP source
2019-09-25 10:08:46 +02:00
Guillaume Tassery
2435e9a12c
Add simple HTTP right credentials
2019-09-25 06:33:54 +02:00
Alexey Milovidov
ff07ffbbae
Merge branch 'merging_redis' of https://github.com/CurtizJ/ClickHouse into CurtizJ-merging_redis
2019-09-25 02:27:05 +03:00
Alexander Kuzmenkov
f3bde19b74
Do not use iterators in find() and emplace() methods of hash tables.
...
Instead, these methods return a pointer to the required data as they are
stored inside the hash table. The caller uses overloaded functions to
get the key and "mapped" values from this pointer. Such an interface
avoids the need for constructing iterator-like wrapper objects, which is
especially important for compound hash tables such as the future
StringHashMap.
2019-09-24 17:44:35 +03:00
Azat Khuzhin
52f14c1ead
Update HashedDictionary.cpp (fixed rename in return type for SFINAE)
2019-09-22 10:39:50 +03:00
alexey-milovidov
a12cffee91
Update HashedDictionary.h
2019-09-22 05:11:08 +03:00
alexey-milovidov
ab9a7be45a
Comments are the must.
2019-09-22 05:09:40 +03:00
alexey-milovidov
9afea6f0b3
Update HashedDictionary.cpp
2019-09-22 05:04:49 +03:00
Azat Khuzhin
420089c301
Add new dictionary layout (sparse_hashed) that is more memory efficient
...
With this new layout, sparsehash will be used over default HashMap,
sparsehash is more memory efficient but it is also slower.
So in a nutshell:
- HashMap uses ~2x more memory then sparse_hash_map
- HashMap ~2-2.5x faster then sparse_hash_map
(tested on lots of input, and the most close to production was
dictionary with 600KK hashes and UInt16 as value)
TODO:
- fix allocated memory calculation
- getBufferSizeInBytes/getBufferSizeInCells interface
- benchmarks
v0: replace HashMap with google::sparse_hash_map
v2: use google::sparse_hash_map only when <sparse> isset to true
v3: replace attributes with different layout
v4: use ch hash over std::hash
2019-09-21 02:22:40 +03:00
CurtizJ
79a08ef8ea
Merge remote-tracking branch 'upstream/master' into merging_redis
2019-09-18 12:58:22 +03:00
CurtizJ
7ccf04440a
better code in Redis external dictionary
2019-09-17 20:57:48 +03:00
CurtizJ
df82e4bde8
fix build
2019-09-17 17:55:09 +03:00
CurtizJ
4480e97f9f
fix build
2019-09-17 17:16:07 +03:00
CurtizJ
4df1f1bb9a
better integration test for redis dictionary (but still bad)
2019-09-17 16:35:19 +03:00
CurtizJ
e9336c9166
improvements of redis external dictionary
2019-09-16 19:17:56 +03:00
CurtizJ
ad986f285e
fix redis with mixed keys
2019-09-13 20:38:56 +03:00
CurtizJ
fa416dc941
fix Redis dictionary
2019-09-12 17:48:28 +03:00
Amos Bird
a1310ffbe6
Split libdbms.so using object library
...
Now the linking time of incremental builds are around 1-2 seconds
2019-09-11 01:56:01 +08:00
CurtizJ
3b3657ff1e
Merge branch 'add-ext-dict-redis' of https://github.com/comunodi/ClickHouse into merging_redis
2019-09-09 13:51:50 +03:00
Vitaly Baranov
da8f67123f
Improve error handling in cache dictionaries:
...
allow using expired values while the source of a cache dictionary doesn't respond;
clone the source after an error to reset connections;
don't ask the source for a little time after error;
show the last exception in system.dictionaries for a cache dictionary too.
2019-08-30 12:50:38 +03:00
Ivan
b4339f266d
Make a better build scheme ( #6500 )
...
* Fix shared build
* Major default libs refactor
* Fix build with gcc_eh
* Link all libraries as a big group.
* Use global interface library as a group
* Build capnproto using our cmake
* Use only internal libunwind
2019-08-28 23:49:37 +03:00
Alexey Milovidov
8966520801
Added "assert_cast" function as a replacement for static_cast for columns
2019-08-21 05:28:04 +03:00
Alexey Milovidov
09ecd865fc
Allow to use library dictionary source with ASan. May impose troubles.
2019-08-14 03:26:38 +03:00
alesapin
2ca6c0b1bf
Fix bug with memory allocation for string fields in complex key cache dictionary
2019-08-12 12:37:48 +03:00
kreuzerkrieg
112fc71276
adding -Wshadow for GCC
2019-08-09 23:58:16 +03:00
Alexey Milovidov
389cf9fcee
Better fix: removed old and bad code #6069
2019-07-20 02:10:55 +03:00
Alexey Milovidov
21165edcb7
Quick fix for initial load of external dictionaries via ODBC
2019-07-20 02:06:15 +03:00
Vitaly Baranov
6d563c51e6
Remove unnecessary try-catch in the constructors of classes derived from IExternalLoadable.
2019-07-17 11:39:36 +03:00
Nikolai Kochetov
e9f3d4d6b8
Merge pull request #4914 from yandex/processors
...
Processors
2019-07-11 19:22:02 +03:00
Nikolai Kochetov
209793c981
Merged with master.
2019-07-08 16:00:54 +03:00
Alexey Milovidov
553e6a273e
Two more fixes for PVS-Studio
2019-07-08 03:51:43 +03:00
Alexey Milovidov
3c4784a466
Removed obsolete undocumented embedded dictionaries that were used by Yandex.Metrica
2019-06-27 22:18:28 +03:00
Nikolai Kochetov
f9b29bfdd2
Merged with master.
2019-06-25 20:00:54 +03:00
Nikolai Kochetov
93b9f3e06a
Disable processors for ClickHouseDictionarySource.
2019-06-25 19:41:06 +03:00
Alexey Milovidov
8f77d898f3
Renamed functions for consistency #5704
2019-06-24 17:51:37 +03:00
alexey-milovidov
8e41d89126
Merge pull request #5704 from zhang2014/fix/ISSUES-5697
...
ISSUES-5697 fix insert and select query with mysql style identifier
2019-06-24 17:40:28 +03:00
proller
1ac09fa721
Split Common/config.h by libs (dbms) ( #5715 )
2019-06-24 14:17:15 +03:00
zhang2014
1786407621
ISSUES-5697 fix insert and select query with mysql style identifier
2019-06-21 13:51:27 +08:00
alesapin
e45d727e9d
Merge pull request #4558 from podshumok/distributred-timeout
...
Distributed and remote timeouts
2019-06-17 13:58:17 +03:00
alexey-milovidov
39c6fbc324
Merge pull request #5567 from vitlibar/really-lazy-dictionary-load
...
Non-blocking loading of external dictionaries
2019-06-15 22:15:53 +03:00
Alexey Milovidov
b2100c7bcf
Removed strange quoting style like `this'. See https://english.stackexchange.com/questions/17695/any-reference-on-the-usage-of-a-backtick-and-single-quotation-mark-like-this
...
Added backQuote and backQuoteIfNeed when required to correctly quote identifiers with proper escaping.
2019-06-15 15:06:22 +03:00
Vitaly Baranov
b4384ce2a9
Dictionaries can be loaded in parallel.
...
A bad dictionary cannot block all the loading anymore.
Implemented really lazy loading of external dictionaries.
Provided more detailed information about the loading of each dictionary
to make diagnostics easier.
2019-06-14 22:18:47 +03:00
Alexey Milovidov
c846d0f6af
Namespace comments are unneeded according to the code style.
...
find dbms -name '*.h' -or -name '*.cpp' | xargs grep -l -P '}\s*//+\s*namespace\s*' | xargs sed -i -r -e 's/}\s*\/\/+\s*namespace\s*.*$/}/'
2019-06-13 13:43:37 +03:00
Konstantin Podshumok
e60a2fd1ad
clickhouse dictionary source: fix timeouts for remotes
2019-06-02 12:43:20 +03:00
comunodi
08c2f183dd
Fix complex dict with two keys
2019-06-02 04:22:06 +03:00
comunodi
a964af386c
Optimize includes
2019-05-31 00:16:12 +03:00
comunodi
c58effc2af
Fix inserting keys with hash map
2019-05-31 00:07:34 +03:00
chertus
3f2e09ac93
Merge branch 'master' into decimal
2019-05-29 13:58:33 +03:00
comunodi
559628575a
Merge branch 'master' into add-ext-dict-redis
...
# Conflicts:
# dbms/tests/integration/image/Dockerfile
2019-05-28 23:20:41 +03:00
comunodi
a8ce7530c9
Put keys in result block only if value exists
2019-05-28 23:06:06 +03:00
chertus
49e12eaa60
remove unused specializations in dictionaries
2019-05-28 22:30:01 +03:00
alexey-milovidov
c5d18dfde1
Merge pull request #5446 from 4ertus2/some
...
Disable implicit type conversions in dictGet functions (it's not working)
2019-05-28 20:34:13 +03:00
chertus
0d77a96b3c
disable implicit type conversions in dictGet functions
2019-05-28 17:04:00 +03:00
Azat Khuzhin
0275d62523
Fix element_count for hashed dictionary (do not include duplicates)
2019-05-28 11:25:31 +03:00
comunodi
b3d8ec3e04
Handle Null keys
2019-05-26 18:55:09 +03:00
comunodi
179ad92874
Fix diff with master
2019-05-26 15:58:40 +03:00
comunodi
5ea9bb228c
Merge branch 'master' into add-ext-dict-redis
2019-05-26 15:49:35 +03:00
comunodi
ba879d95f7
Unify keys handling
2019-05-26 01:54:56 +03:00
comunodi
1f0afdcf68
Parse all args as strings
2019-05-25 03:28:09 +03:00
comunodi
ccf89f4be6
Change LOG_ERROR to LOG_INFO
2019-05-24 02:43:50 +03:00
chertus
f82ea1c1d5
dictGet() dictGetOrDefault for Decimals
2019-05-23 17:04:33 +03:00
Clément Rodriguez
c4c99c80a1
Make the connection closure parameterized in dictionary.xml
2019-05-23 15:55:00 +02:00
Clément Rodriguez
7b40d37212
Close MySQL connections after usage in MySQLDictionarySource
2019-05-23 15:09:07 +02:00
Ivan Lezhankin
0821d62516
Merge ITableDeclaration into IStorage.
...
Also a couple of massive include-refactorings with better forwarding
2019-05-17 17:34:25 +03:00
proller
c6d0e095e0
Prepare cmake for new contrib/poco ( #5062 )
2019-04-25 00:33:04 +03:00
proller
ba4687e110
Cmake: dont use LINK_MODE (but use BUILD_SHARED_LIBS) ( #5091 )
2019-04-24 12:06:28 +03:00
Alexey Milovidov
58d7d1e451
Resolved some of PVS-Studio warnings
2019-04-22 19:07:09 +03:00
proller
525726a5e2
Remove pocoext ( #5036 )
2019-04-17 20:36:58 +03:00
comunodi
b05113188c
Style fix
2019-04-17 13:14:07 +03:00
comunodi
5bc446befe
Parse date and datetime from Int64
2019-04-17 04:11:40 +03:00
comunodi
2023575344
Fix build
2019-04-17 02:26:57 +03:00
comunodi
d2427227dd
Support complex key with 1 or 2 parts
2019-04-17 02:13:07 +03:00
comunodi
9778f7c2f3
More logs
2019-04-15 04:34:10 +03:00
Gleb-Tretyakov
27d138818d
fix invalid memory dereference
2019-04-15 00:21:11 +03:00
comunodi
5849d66975
Use existing ErrorCode to indicate SELECT failure
2019-04-14 20:50:05 +03:00
comunodi
80827b5a9f
Build fix
2019-04-14 20:44:44 +03:00
comunodi
f3ead9fe5b
Style fix
2019-04-14 20:09:33 +03:00
comunodi
562f48ea96
Optional select db before usage. Use only one column in tests
2019-04-14 20:05:50 +03:00
comunodi
d7771b8a07
Throw exception instead if number of columns mismatch
2019-04-07 12:51:53 +03:00
comunodi
8abffd4f60
Fix build
2019-03-31 00:42:13 +03:00
FawnD2
dfed1579ac
Merge branch 'master' into add-ext-dict-redis
...
# Conflicts:
# ci/jobs/quick-build/run.sh
2019-03-30 16:42:42 +03:00
alesapin
2764401d0f
Fix exception 'No message received'
2019-03-29 21:10:03 +03:00
FawnD2
1e014060c9
Merge branch 'master' into add-ext-dict-redis
...
# Conflicts:
# dbms/tests/external_dictionaries/generate_and_test.py
# dbms/tests/external_dictionaries/run.sh
2019-03-16 17:14:27 +03:00
Alexey Milovidov
aea3bd2cc9
Fixed potential deadlock in ExternalDictionaries
2019-03-15 03:29:53 +03:00
Alexey Milovidov
9d89cc0fc1
Debugging PODArray
2019-03-08 06:23:02 +03:00