thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-14 02:12:21 +00:00
Code Issues Packages Projects Releases Wiki Activity
58,563 Commits 9,330 Branches 1,783 Tags 2.1 GiB
a22ec65e8a
Commit Graph

465 Commits

Author SHA1 Message Date
alexey-milovidov
30e3900235
Merge pull request #18435 from vdimir/ip-dict-minor-fix 
Raise an error if more than one key is provided to ip_dictionary
 2020-12-24 20:16:07 +03:00
Yatsishin Ilya
ce1e2b18a3 Fix codespell warnings. Split style checks. Update style checks docker 2020-12-24 13:17:52 +03:00
vdimir
54448ad4ce Remove unnecessary IPAddressDictionary::has overload 2020-12-24 07:38:10 +00:00
vdimir
05d45fcf61 Raise an error if more than one key is provided to ip_dictionary 2020-12-23 16:58:54 +00:00
Nikita Mikhailov
dcfbe782c6 Merge branch 'master' of github.com:ClickHouse/ClickHouse into parallel-parsing-input-format 2020-12-23 05:20:22 +03:00
Denis Glazachev
da7aa25e01 Store copies instead of refs in ExternalQueryBuilder instances 2020-12-21 03:27:35 +04:00
Nikita Mikhaylov
e0addac6fc save changes 2020-12-15 00:56:46 +03:00
Azat Khuzhin
5b3ab48861 More forward declaration for generic headers 
The following headers are pretty generic, so use forward declaration as
much as possible:
- Context.h
- Settings.h
- ConnectionTimeouts.h
(Also this shows that some missing some includes -- this has been fixed)

And split ConnectionTimeouts.h into ConnectionTimeoutsContext.h (since
module part cannot be added for it, due to recursive build dependencies
that will be introduced)

Also remove Settings from the RemoteBlockInputStream/RemoteQueryExecutor
and just pass the context, since settings was passed only in speicifc
places, that can allow making a copy of Context (i.e. Copier).

Approx results (How much units will be recompiled after changing file X?):

- ConnectionTimeouts.h
  - mainline: 100

- Context.h:
  - mainline: ~800
  - patched:  415

- Settings.h:
  - mainline: 900-1K
  - patched:  440 (most of them because of the Context.h)
 2020-12-12 17:43:10 +03:00
Alexey Milovidov
4ecfe8fef4 Remove unused headers 2020-12-08 02:10:22 +03:00
alexey-milovidov
4df1983d64
Merge pull request #17571 from vdimir/ip-dict-accept-alias 
IP dictionary works with aliased types `IPv4` and `IPv6` (close #6804)
 2020-12-07 02:19:32 +03:00
Ivan
315ff4f0d9
ANTLR4 Grammar for ClickHouse and new parser (#11298) 2020-12-04 05:15:44 +03:00
vdimir
5f330a9366
IP dictionary works with aliased types IPv4 and IPv6 (close #6804) 2020-11-28 23:12:19 +03:00
Nikolai Kochetov
729272391f
Merge branch 'master' into ip-dict-no-trie 2020-11-25 23:07:19 +03:00
Nikolai Kochetov
eec131df86
Update registerDictionaries.cpp 2020-11-25 16:27:10 +03:00
Nikita Mikhaylov
6787f98476
Merge pull request #14958 from nikitamikhaylov/refactor_cache_dictionary 
Refactor cache dictionary
 2020-11-24 14:55:02 +03:00
vdimir
52bc290616
Regenerate ya.make, add format null to ip_trie.xml 2020-11-24 11:20:11 +03:00
tavplubix
5cc9cb01cd
Merge pull request #16751 from amosbird/globalcontext 
Make global_context consistent.
 2020-11-22 18:46:17 +03:00
vdimir
a277a5bb16
Rename TrieDictionary -> IPAddressDictionary 2020-11-21 21:38:10 +03:00
vdimir
e2fac19686
Remove libbtrie 2020-11-21 21:31:15 +03:00
vdimir
7adbc5a0c6
Fix ip dict build 2020-11-21 20:39:16 +03:00
vdimir
77ffd25cc0
Fixes for IP dictionary 
More meaningful parse errors

Code style fixes, more comments

Fix bytes_allocated calculation
 2020-11-21 17:56:58 +03:00
Nikolai Kochetov
bac1def5f9
Merge pull request #17134 from abyss7/tcp-port 
Implement tcpPort() function for tests
 2020-11-20 20:32:55 +03:00
Amos Bird
1d9d586e20
Make global_context consistent. 2020-11-20 18:23:14 +08:00
vdimir
36544a45b7
Merge remote-tracking branch 'upstream/master' into ip-dict-no-trie 2020-11-19 18:56:24 +03:00
alexey-milovidov
29bfb5a518
Merge pull request #16997 from vdimir/source-clickhouse-usability 
source(clickhouse(...)) usability
 2020-11-19 07:37:00 +03:00
Ivan Lezhankin
17977e4c4b Use global context for gtests 2020-11-18 21:24:58 +03:00
Ivan Lezhankin
f43f61a10b Another gtest fix 2020-11-18 19:19:39 +03:00
Ivan Lezhankin
ca0740d333 Fix gtest for dictionaries 2020-11-18 17:58:55 +03:00
nikitamikhaylov
36a00ffb1e fix build 2020-11-18 17:28:38 +03:00
nikitamikhaylov
9a0fd70edc better code 2020-11-18 16:58:28 +03:00
nikitamikhaylov
13e711b27e rename back config 2020-11-18 16:38:14 +03:00
Ivan Lezhankin
fdd1c182ab Merge remote-tracking branch 'upstream/master' into tcp-port 2020-11-18 16:22:00 +03:00
nikitamikhaylov
9c6b896928 style + flaky test fix 2020-11-17 17:36:04 +03:00
nikitamikhaylov
4345f2987d better 2020-11-17 16:59:13 +03:00
nikitamikhaylov
20ae82b24b remote nolint 2020-11-17 16:59:13 +03:00
nikitamikhaylov
e604d37067 better 2020-11-17 16:59:13 +03:00
nikitamikhaylov
f596d95b4b better states for cells 2020-11-17 16:59:13 +03:00
nikitamikhaylov
80e77b6a61 better 2020-11-17 16:59:13 +03:00
nikitamikhaylov
31b1f644c1 better 2020-11-17 16:59:13 +03:00
nikitamikhaylov
0f5e980d85 style 2020-11-17 16:59:12 +03:00
nikitamikhaylov
321fb090f1 return back to handwritten cache 2020-11-17 16:59:12 +03:00
nikitamikhaylov
fbd0d14dd6 save 2020-11-17 16:59:12 +03:00
nikitamikhaylov
10adac00f4 avoid race on read/write to the cache of defaults 2020-11-17 16:59:12 +03:00
nikitamikhaylov
9b46b53fa3 better defaults 2020-11-17 16:59:12 +03:00
nikitamikhaylov
028fc00166 fix tests 2020-11-17 16:59:12 +03:00
nikitamikhaylov
d52e72d66c type_id cast 2020-11-17 16:59:12 +03:00
nikitamikhaylov
3df6e457f7 style 2020-11-17 16:59:12 +03:00
nikitamikhaylov
720d3a411f skip unused keys 2020-11-17 16:59:12 +03:00
nikitamikhaylov
82c6467a5d better 2020-11-17 16:59:12 +03:00
nikitamikhaylov
02b2b38716 add normal parent 2020-11-17 16:59:12 +03:00
nikitamikhaylov
0d1a8ea2a1 first try 2020-11-17 16:59:12 +03:00
Ivan Lezhankin
f897f7c93f Refactor IFunction to execute with const arguments 2020-11-17 16:24:45 +03:00
Ivan Lezhankin
1787cd89a7 Implement tcpPort() function literal 2020-11-17 16:24:13 +03:00
Alexey Milovidov
24f4fa6edf Follow Arcadia ya.make rules 2020-11-17 00:16:50 +03:00
vdimir
428f42053b
Merge remote-tracking branch 'upstream/master' into source-clickhouse-usability 2020-11-16 22:34:28 +03:00
Alexey Milovidov
3df04ce0c2 Follow Arcadia ya.make rules 2020-11-16 21:24:58 +03:00
vdimir
6dcb38db3f
Minor changes in IP dictionary 2020-11-16 21:08:31 +03:00
vdimir
608adc77ca
Remove non-unique entries in ip_dict 2020-11-16 10:43:55 +03:00
vdimir
9eb3a0140d
Add 96 bits to mapped IPv4 in ip_dict loading procedure 2020-11-15 20:09:28 +03:00
vdimir
5e0e22301b
Slightly improve ip_dict loading performance, handle v4 to v6 masks in prepossessing, add more tests 2020-11-15 18:55:34 +03:00
vdimir
8b91e0984c
SSE version of matchIPv6Subnet for ip_dict 2020-11-14 23:38:34 +03:00
vdimir
3c6104855d
Fix mask comparison for ip dict, add tests 2020-11-14 23:29:58 +03:00
vdimir
ca8e4b1607
Minor fixes of ip dict 2020-11-14 20:03:13 +03:00
vdimir
cd3cc329a1
Remove redundant anon namespace in ClickHouseDictionarySource.cpp 2020-11-14 18:23:48 +03:00
vdimir
1dce20e5da
Default host, port and user parameters for source(clickhouse(...)) 2020-11-14 16:44:45 +03:00
vdimir
7c19ad5ac7
Fix mappedIPv4ToBinary, add test reference for ip_dict 2020-11-12 10:37:27 +03:00
vdimir
dbec289c9a
[wip] rewrite ip_dict data struct, fix bugs, add tests 2020-11-12 00:58:30 +03:00
Alexander Tokmakov
5cdfcfb307 remove other stringstreams 2020-11-09 22:12:44 +03:00
vdimir
13685e83df
Exact ip records comparison for getBlockInputStream 2020-11-08 23:50:49 +03:00
vdimir
c306902fdf
Fix ip subnet comparison 2020-11-08 23:21:13 +03:00
vdimir
a67f5b780f
Use sorted ip array instead of trie in TrieDictionary 2020-11-08 19:19:04 +03:00
Alexey Milovidov
dba57529f8 Merge branch 'master' of github.com:yandex/ClickHouse into fix-stringstream 2020-11-07 22:03:38 +03:00
Alexey Milovidov
fd84d16387 Fix "server failed to start" error 2020-11-07 03:14:53 +03:00
Alexey Milovidov
c4398952da Addition to prev. revision 2020-11-06 20:40:11 +03:00
Alexey Milovidov
b56486510f Improve Arcadia 2020-11-05 12:55:01 +03:00
alesapin
94ae5aed73
Merge pull request #16550 from ClickHouse/remove_wrong_assertion 
Remove wrong assertion from SSDComplexKeyCacheDictionary
 2020-10-30 22:45:48 +03:00
alesapin
182f9fbdca Remove wrong assertion from SSDComplexKeyCacheDictionary 2020-10-30 12:36:46 +03:00
Maxim Akhmedov
3627fabfb9 Remove -g0 form Arcadia build settings. 2020-10-29 17:37:23 +03:00
Ivan
1d170f5745
ASTTableIdentifier Part #1: improve internal representation of ASTIdentifier name (#16149) 
* Use only |name_parts| as primary name source

* Restore legacy logic for table restoration

* Fix build

* Fix tests

* Add pytest server config

* Fix tests

* Fixes due to review
 2020-10-24 21:46:10 +03:00
alesapin
76a501d7c5 Merge branch 'master' into simplify-init-script 2020-10-19 10:56:40 +03:00
alesapin
07c12116e2
Merge pull request #16007 from CurtizJ/fix-complex-key-direct 
Allow to use direct layout for dictionaries with complex keys
 2020-10-16 10:03:10 +03:00
Nikita Mikhaylov
6512d84b31
Merge pull request #15985 from CurtizJ/redis-cached-dict 
Add support of cache layout for Redis dictionaries with complex key
 2020-10-15 22:35:36 +03:00
Anton Popov
9c9ad2cc85
Merge branch 'master' into fix-complex-key-direct 2020-10-15 18:53:59 +03:00
Nikolai Kochetov
963ce315b2
Merge pull request #15454 from azat/dict-prealloc 
Speedup hashed/sparse_hashed dictionary loading by preallocating the hash table
 2020-10-15 15:03:53 +03:00
Anton Popov
424dd3147c allow to use direct layout for dictionaries with complex keys 2020-10-15 14:59:41 +03:00
Anton Popov
3050b2e67b add support of cache layout for redis dictionaries with complex key 2020-10-14 22:57:07 +03:00
Alexey Milovidov
269b6383f5 Check for #pragma once in headers 2020-10-10 21:37:02 +03:00
Alexey Milovidov
bd805300bb Whitespace 2020-10-09 22:33:23 +03:00
Azat Khuzhin
064f901ea8 Add ability to preallocate hashtables for hashed/sparsehashed dictionaries 
preallocation can be used only when we know number of rows, and for this
we need:
- source clickhouse
- no filtering (i.e. lack of <where>), since filtering can filter
  too much rows and eventually it may allocate memory that will
  never be used.

For sparse_hash the difference is quite significant, preallocated
sparse_hash hashtable allocates ~33% faster (7.5 seconds vs 5 seconds
for insert, and the difference is more significant for higher number of
elements):

    $ ninja bench-sparse_hash-run
    [1/1] cd /src/ch/hashtable-bench/.cmake && ...ch/hashtable-bench/.cmake/bench-sparse_hash
    sparse_hash/insert: 7.574 <!--
    sparse_hash/find  : 2.14426
    sparse_hash/maxrss: 174MiB
    sparse_hash/time:   9710.51 msec (user+sys)

    $ time ninja bench-sparse_hash-preallocate-run
    [1/1] cd /src/ch/hashtable-bench/.cmake && ...-bench/.cmake/bench-sparse_hash-preallocate
    sparse_hash/insert: 5.0522 <!--
    sparse_hash/find  : 2.14024
    sparse_hash/maxrss: 174MiB
    sparse_hash/time:   7192.06 msec (user+sys)

P.S. the difference for sparse_hashed dictionary with 4e9 elements
(uint64, uint16) is ~18% (4975.905 vs 4103.569 sec)

v2: do not reallocate the dictionary from the progress callback
    Since this will access hashtable in parallel.
v3: drop PREALLOCATE() and do this only for source=clickhouse and empty
    <where>
 2020-10-09 22:28:14 +03:00
Nikita Mikhaylov
788b62b931
Merge pull request #14879 from nikitamikhaylov/yet_another_dictionary_bugfix 
Segfault in CacheDictionary
 2020-09-24 20:23:42 +03:00
Alexander Kuzmenkov
0752fa0137 Rethrow copy of exception in SSD dictionaries 
A follow-up to https://github.com/ClickHouse/ClickHouse/pull/14689 ,
that should fix asan reports in stress test in master.
 2020-09-22 18:54:11 +03:00
Nikita Mikhaylov
123f389f1a
Merge pull request #14689 from ClickHouse/aku/broken-perf 
Add context to errors in ExpressionAction
 2020-09-21 23:33:42 +03:00
nikitamikhaylov
2530b71aa6 style 2020-09-18 17:48:05 +03:00
nikitamikhaylov
822a78fa58 call under mutex 2020-09-18 17:25:45 +03:00
Vitaly Baranov
3356d75b23
Merge pull request #13156 from azat/cluster-secure 
Secure inter-cluster query execution (with initial_user as current query user) [v3]
 2020-09-17 17:11:00 +03:00
nikitamikhaylov
b33a5e0d14 done 2020-09-16 15:32:33 +03:00
Alexander Kuzmenkov
da2a3fffe8 fixup 2020-09-16 13:00:15 +03:00
Artem Zuikov
51ba12c2c3
Try speedup build (#14809) 2020-09-15 12:55:57 +03:00
Azat Khuzhin
0159c74f21 Secure inter-cluster query execution (with initial_user as current query user) [v3] 
Add inter-server cluster secret, it is used for Distributed queries
inside cluster, you can configure in the configuration file:

  <remote_servers>
      <logs>
          <shard>
              <secret>foobar</secret> <!-- empty -- works as before -->
              ...
          </shard>
      </logs>
  </remote_servers>

And this will allow clickhouse to make sure that the query was not
faked, and was issued from the node that knows the secret. And since
trust appeared it can use initial_user for query execution, this will
apply correct *_for_user (since with inter-server secret enabled, the
query will be executed from the same user on the shards as on initator,
unlike "default" user w/o it).

v2: Change user to the initial_user for Distributed queries if secret match
v3: Add Protocol::Cluster package
v4: Drop Protocol::Cluster and use plain Protocol::Hello + user marker
v5: Do not use user from Hello for cluster-secure (superfluous)
 2020-09-15 01:36:28 +03:00
Nikita Mikhaylov
44726c37c3
Merge branch 'master' into fix-executable-dictionary-source-hangup 2020-09-09 23:01:41 +04:00
Vasily Nemkov
3973a17530
MySql datatypes dateTime64 and decimal (#11512) 2020-09-09 15:18:02 +03:00
Ilya Yatsishin
11a247d2f4
Merge pull request #14556 from ClickHouse/fix-arcadia-2 
Lower binary size in "Arcadia" build
 2020-09-08 12:08:35 +03:00
Alexey Milovidov
e3924b8057 Fix "Arcadia" 2020-09-08 01:14:13 +03:00
alexey-milovidov
3903794386
Update ExecutableDictionarySource.cpp 2020-09-07 09:45:01 +03:00
alexey-milovidov
1f0d2be17a
Update ExecutableDictionarySource.h 2020-09-07 09:44:32 +03:00
Alexey Milovidov
8dd98f74a5 Remove debug output 2020-09-07 09:43:28 +03:00
Alexey Milovidov
04a6965006 Fix error with executable dictionary source 2020-09-07 09:40:01 +03:00
Alexey Milovidov
4a336e3814 More consistent invocation of skipBOMIfExists 2020-09-07 07:21:55 +03:00
alexey-milovidov
4b1b744644
Revert "Less number of threads in builder" 2020-09-03 01:00:41 +03:00
alexey-milovidov
7bd31fb3d3
Merge pull request #14313 from ClickHouse/fix-fuzz-test6 
Better check for tuple size in SSD cache complex key external dictionaries
 2020-09-03 00:50:24 +03:00
Alexey Milovidov
deb483eee5 Merge branch 'master' into fix-fuzz-test6 2020-09-02 04:40:42 +03:00
Alexey Milovidov
56bbac1569 Trigger CI 2020-09-02 04:28:52 +03:00
Anton Popov
7c3712c121
Merge pull request #13688 from it1804/master 
Redis dictionary support for simple Redis requirepass authorization
 2020-09-02 01:43:02 +03:00
Alexey Milovidov
142a5bcede Added validation of key types to SSD Cache dictionary 2020-09-01 02:10:04 +03:00
Alexey Milovidov
8fa61f785f Better check for tuple size in complex key external dictionaries 2020-09-01 01:55:52 +03:00
alexey-milovidov
bce8166420
Update FileDictionarySource.cpp 2020-08-27 21:51:19 +03:00
Alexey Milovidov
a4a6128431 Trigger CI 2020-08-26 23:45:24 +03:00
Nikita Mikhaylov
d05230c55a
Update CacheDictionary.cpp 2020-08-26 17:04:05 +03:00
Nikita Mikhaylov
da16f234cc
Update CacheDictionary.cpp 2020-08-26 14:06:32 +03:00
alexey-milovidov
f40b80ed45
Merge pull request #12756 from sundy-li/hotfix/dict-access 
Fix ClickHouseDictionarySource wrong access check
 2020-08-26 03:33:42 +03:00
Nikita Mikhaylov
21a50aab30
Merge branch 'master' into cache-dictionary-bugfix 2020-08-19 15:53:23 +04:00
Artem Zuikov
becc186c91
Add support for extended precision integers and decimals (#13097) 2020-08-19 14:52:17 +03:00
Alexey Milovidov
eeb769d2d4 Remove useless code around zkutil 2020-08-18 22:02:07 +03:00
Alexey Milovidov
0c75ebcba3 Fix typo 2020-08-15 06:28:48 +03:00
Alexey Milovidov
c260ecb58e Remove debug output 2020-08-15 06:27:36 +03:00
Alexey Milovidov
52c3857225 Fix server restart after stress test 2020-08-15 06:10:57 +03:00
it1804
a1c0c52c5b
Allow authenticate Redis with requirepass option 2020-08-14 02:01:25 +05:00
it1804
a79c3175a7
Allow authenticate Redis with requirepass option 2020-08-14 02:00:12 +05:00
Nikita Mikhaylov
9765588b9f better 2020-08-13 14:12:16 +03:00
Nikita Mikhaylov
7082baa96c add locks to fix datarace 2020-08-13 14:12:16 +03:00
Nikita Mikhaylov
4c89fb8f84 better 2020-08-11 22:50:48 +03:00
Nikita Mikhaylov
519c5c500e first attempt 2020-08-11 22:39:48 +03:00
alexey-milovidov
8912a1cda3
Merge pull request #13082 from CurtizJ/fix-redis 
Fix access to redis dictionary after connection was dropped once
 2020-08-10 03:29:17 +03:00
Alexey Milovidov
edd89a8610 Fix half of typos 2020-08-08 03:47:03 +03:00
Alexander Kuzmenkov
45b57f2df5 fixes 2020-08-06 00:53:35 +03:00
Alexander Kuzmenkov
d205b20c04 wip 2020-08-05 05:21:33 +03:00
Alexander Tokmakov
5b92f43f12 Merge branch 'master' into database_atomic_improvements 2020-08-03 14:35:40 +03:00
alexey-milovidov
56a974049e
Merge pull request #13158 from ClickHouse/amosbird-fm 
Merge #12574
 2020-08-02 01:48:57 +03:00
Alexey Milovidov
07f3662f6d Merge remote-tracking branch 'origin/master' into deprecate-special-zero-datetime 2020-07-31 23:49:06 +03:00
Alexey Milovidov
6d5590a4b4 Merge with master 2020-07-31 15:25:17 +03:00
alexey-milovidov
9c9ef0db9b
Merge branch 'master' into database_atomic_improvements 2020-07-31 14:56:49 +03:00
Alexey Milovidov
bf4e74375e generate ya.make 2020-07-31 02:43:55 +03:00
alexey-milovidov
74edcfb87e
Merge pull request #13073 from nikitamikhaylov/rewrite-tests 
BayesAB test
 2020-07-31 02:30:07 +03:00
Anton Popov
37499f2fe1 bump CI 2020-07-31 02:06:01 +03:00
Alexey Milovidov
83c9d9ef45 Merge branch 'master' into deprecate-special-zero-datetime 2020-07-31 00:17:04 +03:00
Nikita Mikhaylov
4d49d2c671 another removes 2020-07-30 13:31:14 +03:00
Nikita Mikhaylov
1d6c2ba689 rewrite other test + remove useless includes 2020-07-29 20:37:57 +03:00
Anton Popov
2bdf276687 fix access to redis dictionary after connection was dropped once 2020-07-29 19:20:17 +03:00
Anton Popov
48e6253442 minor fixes 2020-07-29 04:32:23 +03:00
Anton Popov
6d58279535 Merge remote-tracking branch 'upstream/master' into HEAD 2020-07-29 02:15:30 +03:00