thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-11-30 19:42:00 +00:00
Code Issues Packages Projects Releases Wiki Activity
90,281 Commits 9,190 Branches 1,782 Tags 2.2 GiB
ad1402d1eb
Commit Graph

26965 Commits

Author SHA1 Message Date
Alexey Milovidov
d62c57be3f Fix clang-tidy-14, part 1 2022-05-27 23:02:25 +02:00
Alexey Milovidov
8e9d771237 Fix clang-tidy-14, part 1 2022-05-27 23:02:05 +02:00
Alexey Milovidov
6c2699a991 Fix clang-tidy-14, part 1 2022-05-27 23:00:45 +02:00
Alexey Milovidov
3086c19341 Fix clang-tidy-14, part 1 2022-05-27 23:00:23 +02:00
Alexey Milovidov
c50791dd3b Fix clang-tidy-14, part 1 2022-05-27 22:52:14 +02:00
Alexey Milovidov
d2c6fd90cb Fix clang-tidy-14, part 1 2022-05-27 22:51:37 +02:00
Kseniia Sumarokova
10c9716467
Fix clang-tidy 2022-05-27 22:48:07 +02:00
Nikolai Kochetov
b80b1940ce Fix some tests. 2022-05-27 20:47:35 +00:00
HeenaBansal2009
a061acadbe Remove std::move from trivially-copyable object 2022-05-27 11:04:29 -07:00
Yakov Olkhovskiy
41ef0044f0 endpoint is added 2022-05-27 13:43:34 -04:00
mergify[bot]
f5ee337bab
Merge branch 'master' into revert-37545-revert-37424-fix_fetching_part_deadlock 2022-05-27 16:52:00 +00:00
mergify[bot]
923ad2e905
Merge branch 'master' into turn_on_s3_tests 2022-05-27 16:31:43 +00:00
Dmitry Novik
60b9d81773 Remove global_memory_usage_overcommit_max_wait_microseconds 2022-05-27 16:30:29 +00:00
alesapin
f63fa9bcc6
Merge pull request #37416 from Enmk/fix_ReplicatedMergeTree_comments 
Implemented changing comment to a ReplicatedMergeTree table
 2022-05-27 18:29:34 +02:00
Alexander Gololobov
9b1b30855c Fixed check for HUGE_VAL 2022-05-27 18:25:11 +02:00
Alexander Gololobov
6361c5f38c Fix for failed style check 2022-05-27 18:22:16 +02:00
Kseniia Sumarokova
8099361cbc
Update FileCache.cpp 2022-05-27 17:48:14 +02:00
Alexander Gololobov
540353566c Added LpNorm and LpDistance functions for arrays 2022-05-27 17:17:08 +02:00
Azat Khuzhin
8a224239c1 Prohibit optimize_aggregation_in_order with GROUPING SETS 
AggregatingStep ignores it anyway, and it leads to the following error
in getSortDescriptionFromGroupBy(), like in [1]:

    2022.05.24 04:29:29.279431 [ 3395 ] {26543564-8bc8-4a3a-b984-70a2adf0245d} <Fatal> : Logical error: 'Trying to get name of not a column: ExpressionList'.

  [1]: https://s3.amazonaws.com/clickhouse-test-reports/36914/67d3ac72d26ab74d69f03c03422349d4faae9e19/stateless_tests__ubsan__actions_.html

v2: revert change to getSortDescriptionFromGroupBy() after
    GroupingSetsRewriterVisitor had been introduced
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-05-27 17:44:57 +03:00
Azat Khuzhin
1f29b0a901 Rewrite queries GROUPING SETS (foo, bar) to GROUP BY foo, bar 
This is better then introducing separate
SelectQueryExpressionAnalyzer::useGroupingSetKey(), since for
optimize_aggregation_in_order that method will not be enough, because
size of ManyExpressionActions will not match size of SortDescription, in
ReadInOrderOptimizer::ReadInOrderOptimizer()

And plus it is cleaner.

v2: fix clang-tidy
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-05-27 17:44:51 +03:00
alesapin
5a296aec01 Fix build 2022-05-27 16:34:16 +02:00
alesapin
be1c3c132b Fix some trash 2022-05-27 16:08:49 +02:00
Anton Popov
abc90fad8d fix WITH FILL with negative itervals 2022-05-27 12:42:51 +00:00
alesapin
6d6779f17a
Merge pull request #37139 from ClickHouse/i_object_storage 
Separate object storage operations from disks
 2022-05-27 13:59:50 +02:00
alesapin
c79600c4c8 Fix build 2022-05-27 13:44:29 +02:00
alesapin
841858ec30
Revert "Revert "(only with zero-copy replication, non-production experimental feature not recommended to use) fix possible deadlock during fetching part"" 2022-05-27 13:13:36 +02:00
Azat Khuzhin
2613149f6b Fix converting types for UNION queries (may produce LOGICAL_ERROR) 
CI founds [1]:

    2022.02.20 15:14:23.969247 [ 492 ] {} <Fatal> BaseDaemon: (version 22.3.1.1, build id: 6082C357CFA6FF99) (from thread 472) (query_id: a5187ff9-962a-4e7c-86f6-8d48850a47d6) (query: SELECT 0., round(avgWeighted(x, y)) FROM (SELECT toDate(toDate('214748364.8', '-922337203.6854775808', '-0.1', NULL) - NULL, 10.000100135803223, '-2147483647'), 255 AS x, -2147483647 AS y UNION ALL SELECT y, NULL AS x, 2147483646 AS y)) Received signal Aborted (6)

  [1]: https://s3.amazonaws.com/clickhouse-test-reports/0/26d0e5438c86e52a145aaaf4cb523c399989a878/fuzzer_astfuzzerdebug,actions//report.html

The problem is that subqueries returns different headers:
- first query  -- x, y
- second query -- y, x

v2: Make order of columns strict only for UNION
    https://s3.amazonaws.com/clickhouse-test-reports/34775/9cc8c01a463d18c471853568b2f0af659a4e643f/stateless_tests__address__actions__[2/2].html
    Fixes: 00597_push_down_predicate_long
v3: add no-backward-compatibility-check for the test
Fixes: #37569
Resubmit: #34775
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
(cherry picked from commit a813f5996e)
 2022-05-27 14:11:57 +03:00
Han Fei
2ea027ffcb Support insert into system.zookeeper 2022-05-27 18:53:12 +08:00
Kseniia Sumarokova
141334448e
Merge pull request #37566 from kssenii/fix-assertion 
Fix failed assertion in cache
 2022-05-27 11:59:03 +02:00
Kseniia Sumarokova
2943d44bf1
Merge pull request #37554 from msaf1980/cleanup_hdfs 
Cleanup StorageHDFS (unused variables prevent build with clang 12)
 2022-05-27 11:57:18 +02:00
Kseniia Sumarokova
f5d69506b4
Merge pull request #37516 from KinderRiven/improve_local_cache 
Control cache downloads to avoid negative optimization of local caches
 2022-05-27 11:53:17 +02:00
zhanglistar
ca67e67a74 Fix a typo 2022-05-27 15:52:04 +08:00
Robert Schulze
80061aa3e2
Merge remote-tracking branch 'origin/master' into cached_patterns 2022-05-27 09:21:01 +02:00
Vxider
54d6f98122 flush and shutdown temporary table before drop 2022-05-27 04:50:36 +00:00
Dmitry Novik
3a9239b79f
Revert "RFC: Fix converting types for UNION queries (may produce LOGICAL_ERROR)" 2022-05-27 04:05:32 +02:00
Yakov Olkhovskiy
25884c68f1 http named collection source implemented for dictionary 2022-05-26 20:46:26 -04:00
Alexey Milovidov
8ba865bb60
Merge pull request #37344 from excitoon-favorites/fixs3colonandequalssign 
Fixed error with symbols in key name in S3
 2022-05-27 00:58:35 +03:00
Alexey Milovidov
86afa3a245
Merge pull request #37502 from ClickHouse/array_norm_dist_fixes 
Renamed arrayXXNorm/arrayXXDistance functions to XXNorm/XXDistance and fixed some overflow cases
 2022-05-27 00:56:29 +03:00
Alexander Gololobov
e655863d53
Merge pull request #37528 from kitaisreal/normalize-utf8-performance-tests-fix 
Functions normalizeUTF8 unstable performance tests fix
 2022-05-26 20:49:10 +02:00
Azat Khuzhin
c6c60364ae Remove unused MergeTreeDataMergerMutator::chooseMergeAlgorithm() 
In favor of
MergeTask::ExecuteAndFinalizeHorizontalPart::chooseMergeAlgorithm()

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-05-26 21:20:50 +03:00
Alexander Tokmakov
eb71dd4c78
Merge pull request #37547 from ClickHouse/followup_37398 
Follow-up to #37398
 2022-05-26 20:29:41 +03:00
Nikolai Kochetov
ec4e8d71b2 Fixing build 2022-05-26 15:33:21 +00:00
Dmitry Novik
673a521d0b
Merge pull request #34775 from azat/union-type-cast 
RFC: Fix converting types for UNION queries (may produce LOGICAL_ERROR)
 2022-05-26 17:28:23 +02:00
kssenii
36af6b1fa8 Fix assertion 2022-05-26 16:15:02 +02:00
alesapin
c862f89b8d Fix tidy 2022-05-26 15:43:21 +02:00
Alexander Tokmakov
e8f33fb0d9 fix flaky tests 2022-05-26 14:17:05 +02:00
Azat Khuzhin
dc9ca3d70c
Fix LOGICAL_ERROR in getMaxSourcePartsSizeForMerge during merges (#37413) 2022-05-26 14:14:58 +02:00
Nikolai Kochetov
bf95541531 Fixing style. 2022-05-26 11:09:36 +00:00
Nikolai Kochetov
84f97b53de Merge branch 'master' into refactor-read-metrics-and-callbacks 2022-05-26 11:07:45 +00:00
Nikolai Kochetov
fea2401f1f
Merge pull request #37532 from ClickHouse/add-separate-mutex-for-factories-info 
Use a separate mutex for query_factories_info in Context.
 2022-05-26 13:03:28 +02:00
mergify[bot]
a7629f900f
Merge branch 'master' into normalize-utf8-performance-tests-fix 2022-05-26 10:29:55 +00:00
Maksim Kita
3a92e61827
Merge pull request #37148 from kitaisreal/dictionary-get-descendants-performance-improvement 
Dictionary getDescendants performance improvement
 2022-05-26 12:29:17 +02:00
KinderRiven
824628c0da fix style 2022-05-26 16:51:16 +08:00
KinderRiven
822ecd982f better & support clean stash 2022-05-26 16:36:05 +08:00
Vasily Nemkov
abe6b5d013
Reverted unnecessary modification 2022-05-26 10:09:27 +03:00
Antonio Andelic
fe236c98d5
Merge pull request #37534 from ClickHouse/revert-37036-keeper-preprocess-operations 
Revert "Add support for preprocessing ZooKeeper operations in `clickhouse-keeper`"
 2022-05-26 08:14:46 +02:00
Yakov Olkhovskiy
2dc160a4c3 style fix 2022-05-25 20:56:36 -04:00
Dmitry Novik
5c3c994d2a
Merge pull request #37493 from ClickHouse/grouping-sets-optimization-fix 
Fix ORDER BY optimization in case of GROUPING SETS
 2022-05-26 02:25:02 +02:00
Anton Popov
f488efd27e fix tests 2022-05-26 00:03:31 +00:00
Dmitry Novik
16c6b60703 Introduce AggregationKeysInfo 2022-05-25 23:22:29 +00:00
alesapin
8f1aac0ce4 Fix merge with master 2022-05-26 00:44:45 +02:00
Alexey Milovidov
f321925032
Merge pull request #36341 from ClickHouse/allow-setuid-inside-clickhouse 
Allow to drop privileges at startup
 2022-05-26 01:07:04 +03:00
Dmitry Novik
7cd7782e4f Process columns more efficiently in GROUPING() 2022-05-25 21:55:41 +00:00
Dmitry Novik
3c1b6609ae Add comments and make tests more verbose 2022-05-25 21:23:35 +00:00
mergify[bot]
49cce189e3
Merge branch 'master' into floating_seconds 2022-05-25 21:08:55 +00:00
alesapin
1db9cf480b Merge remote-tracking branch 'origin/master' into i_object_storage 2022-05-25 22:50:22 +02:00
Maksim Kita
58cd1bd3ec
Merge pull request #36843 from bharatnc/ncb/h3-unidirectionaledges-funcs 
add h3 unidirectional edge functions
 2022-05-25 22:46:40 +02:00
Maksim Kita
bee3c30f66
Merge pull request #37524 from kitaisreal/geo-distance-functions-improve-performance 
Geo distance functions improve performance
 2022-05-25 22:40:40 +02:00
Maksim Kita
b12b363158 Fixed build of hierarchical index for HashedArrayDictionary 2022-05-25 22:40:19 +02:00
Alexander Gololobov
168b47d0ad Use same norm and distance function names for tuples and arrays 2022-05-25 22:39:59 +02:00
Alexander Gololobov
b065839f44 always return Float64 2022-05-25 22:27:00 +02:00
Alexander Gololobov
5df14cd956 Cast arguments to result type to avoid int overflow 2022-05-25 22:27:00 +02:00
Alexander Tokmakov
47820c216d
Revert "(only with zero-copy replication, non-production experimental feature not recommended to use) fix possible deadlock during fetching part" 2022-05-25 23:10:33 +03:00
Robert Schulze
49934a3dc8
Cache compiled regexps when evaluating non-const needles 
Needles in a (non-const) needle column may repeat and this commit allows
to skip compilation for known needles. Out of the different design
alternatives (see below, if someone is interested), we now maintain
- one global pattern cache,
- with a fixed size of 42k elements currently,
- and use LRU as eviction strategy.

------------------------------------------------------------------------

(sorry for the wall of text, dumping it here not for reading but just
for reference)

Write-up about considered design alternatives:

1. Keep the current global cache of const needles. For non-const
   needles, probe the cache but don't store values in it.
   Pros: need to maintain just a single cache, no problem with cache
         pollution assuming there are few distinct constant needles
   Cons: only useful if a non-const needle occurred as already as a
         const needle
   --> overall too simplistic

2. Keep the current global cache for const needles. For non-const
   needles, create a local (e.g. per-query) cache
   Pros: unlike (1.), non-const needles can be skipped even if they
         did not occur yet, no pollution of the const pattern cache when
         there are very many non-const needles (e.g. large / highly
         distinct needle columns).
   Cons: caches may explode "horizontally", i.e. we'll end up with the
         const cache + caches for Q1, Q2, ... QN, this makes it harder
         to control the overall space consumption, also patterns
         residing in different caches cannot be reused between queries,
         another difficulty is that the concept of "query" does not
         really exist at matching level - there are only column chunks
         and we'd potentially end up with 1 cache / chunk

3. Queries with const and non-const needles insert into the same global
   cache.
   Pros: the advantages of (2.) + allows to reuse compiled patterns
         accross parallel queries
   Cons: needs an eviction strategy to control cache size and pollution
         (and btw. (2.) also needs eviction strategies for the
         individual caches)

4. Queries with const needle use global cache, queries with non-const
   needle use a different global cache
   --> Overall similar to (3) but ignores the (likely) edge case that
       const and non-const needles overlap.

In sum, (3.) seems the simplest and most beneficial approach.

Eviction strategies:

0. Don't ever evict --> cache may grow infinitely and eventually make
   the system unusable (may even pose a DoS risk)

1. Flush the cache after a certain threshold is exceeded --> very
   simple but may lead to peridic performance drops

2. Use LRU --> more graceful performance degradation at threshold but
   comes with a (constant) performance overhead to maintain the LRU
   queue

In sum, given that the pattern compilation in RE2 should be quite costly
(pattern-to-DFA/NFA), LRU may be acceptable.
 2022-05-25 22:04:06 +02:00
Robert Schulze
ea60a614d2
Decrease namespace indent 2022-05-25 21:56:35 +02:00
alesapin
c7b16065e1 Merge with master 2022-05-25 21:47:05 +02:00
Nikolai Kochetov
6d4a26afac Update ReadProgressCallback. 2022-05-25 19:45:48 +00:00
Alexey Milovidov
abf2558fba
Merge pull request #37491 from ClickHouse/match_refactoring 
Refactorings of LIKE/MATCH code
 2022-05-25 22:05:38 +03:00
Alexey Milovidov
4482da9eb6
Update greatCircleDistance.cpp 2022-05-25 21:59:31 +03:00
alesapin
6f5c86e55e Merge branch 'master' into i_object_storage 2022-05-25 20:49:01 +02:00
Nikolai Kochetov
54d7e4139f Fix build. 2022-05-25 18:16:48 +00:00
alesapin
51868a9a4f
Merge pull request #37424 from metahys/fix_fetching_part_deadlock 
(only with zero-copy replication, non-production experimental feature not recommended to use) fix possible deadlock during fetching part
 2022-05-25 20:15:41 +02:00
Azat Khuzhin
a813f5996e Fix converting types for UNION queries (may produce LOGICAL_ERROR) 
CI founds [1]:

    2022.02.20 15:14:23.969247 [ 492 ] {} <Fatal> BaseDaemon: (version 22.3.1.1, build id: 6082C357CFA6FF99) (from thread 472) (query_id: a5187ff9-962a-4e7c-86f6-8d48850a47d6) (query: SELECT 0., round(avgWeighted(x, y)) FROM (SELECT toDate(toDate('214748364.8', '-922337203.6854775808', '-0.1', NULL) - NULL, 10.000100135803223, '-2147483647'), 255 AS x, -2147483647 AS y UNION ALL SELECT y, NULL AS x, 2147483646 AS y)) Received signal Aborted (6)

  [1]: https://s3.amazonaws.com/clickhouse-test-reports/0/26d0e5438c86e52a145aaaf4cb523c399989a878/fuzzer_astfuzzerdebug,actions//report.html

The problem is that subqueries returns different headers:
- first query  -- x, y
- second query -- y, x

v2: Make order of columns strict only for UNION
    https://s3.amazonaws.com/clickhouse-test-reports/34775/9cc8c01a463d18c471853568b2f0af659a4e643f/stateless_tests__address__actions__[2/2].html
    Fixes: 00597_push_down_predicate_long
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-05-25 20:31:47 +03:00
Nikolai Kochetov
ff98c24d44
Merge pull request #37048 from Avogar/fix-array-map-nothing 
Add default implementation for Nothing in functions
 2022-05-25 19:10:40 +02:00
Nikolai Kochetov
1f9b1cf726 Fixing build. 2022-05-25 18:59:46 +02:00
alesapin
0a3597da72
Merge pull request #34915 from ianton-ru/MDB-16962 
Fix collision of S3 operation log revision
 2022-05-25 18:15:31 +02:00
Yakov Olkhovskiy
6692b9c2ed showCertificate function implementation 2022-05-25 12:11:44 -04:00
Alexey Milovidov
cb92482ca5
Merge pull request #37484 from kitaisreal/function-has-all-avx2-dynamic-dispatch 
Function hasAll added dynamic dispatch
 2022-05-25 19:05:32 +03:00
Nikolai Kochetov
7b681fa8ac Fixing build. 2022-05-25 17:15:23 +02:00
avogar
ede6e2f433 Add docs for settings 2022-05-25 15:10:20 +00:00
avogar
4c9812d4c1 Allow to skip some of the first rows in CSV/TSV formats 2022-05-25 15:00:11 +00:00
KinderRiven
a33c7ce648 fix 2022-05-25 22:58:47 +08:00
Anton Popov
16e839ac71 add profile events for introspection of part types 2022-05-25 14:54:49 +00:00
Antonio Andelic
6a962549d5
Revert "Add support for preprocessing ZooKeeper operations in clickhouse-keeper" 2022-05-25 16:45:32 +02:00
Nikolai Kochetov
1b85f2c1d6 Merge branch 'master' into refactor-read-metrics-and-callbacks 2022-05-25 16:27:40 +02:00
msaf1980
fda6ddeffa cleanup StorageHDFS (unused variables) 2022-05-25 19:23:05 +05:00
mergify[bot]
73662b4436
Merge branch 'master' into fix_fetching_part_deadlock 2022-05-25 14:22:35 +00:00
Maksim Kita
28355114c0 Fixed tests 2022-05-25 16:19:29 +02:00
Nikolai Kochetov
6370c29049 Use a separate mutex for query_factories_info in Context. 2022-05-25 14:16:59 +00:00
Maksim Kita
e67b3537f7 Functions normalizeUTF8 unstable performance tests fix 2022-05-25 15:54:52 +02:00
KinderRiven
875557abc2 fix 2022-05-25 21:53:28 +08:00
KinderRiven
adbb821176 fix 2022-05-25 21:05:15 +08:00
mergify[bot]
f49552d48e
Merge branch 'master' into grouping-sets-optimization-fix 2022-05-25 13:03:54 +00:00
Maksim Kita
45da28ecae Improve performance of geo distance functions 2022-05-25 14:22:22 +02:00
KinderRiven
2211c1ddb8 fix 2022-05-25 20:15:43 +08:00
Maksim Kita
83554d1f2d Fixed style 2022-05-25 13:05:39 +02:00
Maksim Kita
fbec38ddb9 Fixed performance tests 2022-05-25 12:51:21 +02:00
KinderRiven
d0fcffec66 fix style 2022-05-25 17:51:03 +08:00
Maksim Kita
c372c3d6aa Fix performance tests 2022-05-25 11:49:59 +02:00
Maksim Kita
9a9df26eec Fixed tests 2022-05-25 11:44:37 +02:00
Maksim Kita
6c033f340b Fixed tests 2022-05-25 11:44:37 +02:00
Maksim Kita
0e5f13e53e MergingSortedAlgorithm single column specialization 2022-05-25 11:44:37 +02:00
KinderRiven
1ce219bae2 fix 2022-05-25 17:24:38 +08:00
Kseniia Sumarokova
b50d4549c9
Merge pull request #37356 from amosbird/partition-prune-for-s3 
"Partition pruning" for s3
 2022-05-25 11:03:07 +02:00
KinderRiven
e3f76cab55 impl improve remote fs cache 2022-05-25 16:54:28 +08:00
avogar
f782fa31c6 Merge branch 'master' of github.com:ClickHouse/ClickHouse into check-format-on-storage-creation 2022-05-25 08:42:54 +00:00
Robert Schulze
05e4fa7df1
Fix special case of trivial regexp 
Previously, we would alsays set 1 in case of a trivial regex (which is
correct). If someone in future builds a negated operator, then this
will produce wrong results. Right now, negation of regexp (SQL: NOT
MATCH) is implemented at a higher level, so we are safe and this is more
a preventive fix.
 2022-05-25 10:05:55 +02:00
Robert Schulze
01ab7b9bad
Pass strings in some places as string_view 
The original goal was to get change

  const auto & needle = String(
        reinterpret_cast<const char *>(cur_needle_data),
        cur_needle_length);

in Functions/MatchImpl.h into a std::string_view to save an allocation +
copy. The needle is eventually passed as search pattern into the re2
library. Re2 has an alternative constructor taking a const char * i.e. a
NULL-terminated string. Here, the needle is NULL-terminated but
1. this is only because it is passed inside a ColumnString yet this is
   not always the case (e.g. fixed string columns has a dense layout w/o
   NULL terminator).
2. assuming NULL termination for users != MatchImpl of the regex code is
   too dangerous.

So, for now we'll stay with copying to be on the safe side. One fine day
when re2 has a ptr/size ctor, we can use std::string_view.

Just changing a few other places from std::string to std::string_view
but this will not help with performance.
 2022-05-25 10:05:51 +02:00
Robert Schulze
e8c96777f6
Make OptimizedRegularExpression::analyze() private 2022-05-25 10:05:45 +02:00
Robert Schulze
040fbf3686
Tighter sanity checks in matching code 2022-05-25 10:05:06 +02:00
Robert Schulze
35bef17302
Introduce variables to hold the match result 
--> nicer when debugging
 2022-05-25 10:04:47 +02:00
Robert Schulze
b044d44fef
Refactoring: Make template instantiation easier to read 
- introduced class MatchTraits with enums that replace bool template
  parameters

- (minor: made negation the last template parameters because negation
  executes last during evaluation)
 2022-05-25 10:03:58 +02:00
Bharat Nallan Chakravarthy
57cfc0bd04 check for validity of h3 index 2022-05-25 06:17:15 +05:30
Alexey Milovidov
516fba27dc Merge branch 'master' into allow-setuid-inside-clickhouse 2022-05-24 23:31:14 +02:00
Nikolai Kochetov
3d84aae0ab Better. 2022-05-24 20:06:08 +00:00
Maksim Kita
c1777aec1e
Merge pull request #37481 from kitaisreal/partial-sorting-transform-optimization-fix 
Column compareImpl devirtualize compare call
 2022-05-24 22:05:41 +02:00
Nikolai Kochetov
333fd09dbf Fixing build. 2022-05-24 19:29:00 +00:00
Igor Nikonov
04e2737a57
Merge pull request #37337 from ClickHouse/column_decl_null_before_default_value 
Column declaration: [NOT] NULL right after type 

+ fixed: data_type_default_nullable=true, it didn't make columns nullable
         if the column declaration contains default expression w/o type

Issue #37229
 2022-05-24 21:16:25 +02:00
Alexander Gololobov
2ff747785e
Merge pull request #37394 from ClickHouse/array_norm_dist_fixes 
Do computations on the raw input data without copying to Eigen::Matrix
 2022-05-24 20:59:04 +02:00
Dmitry Novik
c4dc0f7cda Fix ORDER BY optimization in case of GROUPING SETS 2022-05-24 18:56:22 +00:00
Maksim Kita
96833b8696 ColumnImpl compareImpl added assert for compare result 2022-05-24 20:41:48 +02:00
Robert Schulze
7348a0eb28
Merge pull request #37251 from ClickHouse/non_const_like 
Support non-constant SQL functions (NOT) (I)LIKE and MATCH
 2022-05-24 20:28:31 +02:00
Robert Schulze
028f15c4fa
Review comment: Throw LOGICAL_ERROR for different sizes of haystack / needles 2022-05-24 20:19:13 +02:00
Vasily Nemkov
59b4d4a643 ALTER COMMENT is now local-only operation and immediately observable 2022-05-24 21:08:30 +03:00
Kruglov Pavel
0c3bcfa122
Merge pull request #36884 from Algunenano/http_proper_summary_and_exception 
HTTP: Always return summary data and exception (when possible)
 2022-05-24 18:33:24 +02:00
Maksim Kita
3c0c322d7c
Merge pull request #37480 from kitaisreal/dynamic-dispatch-infrastructure-improvements 
Dynamic dispatch infrastructure style fixes
 2022-05-24 18:13:53 +02:00
Vitaly Baranov
497c70d786
Merge pull request #37269 from vitlibar/rework-access-control-notifications 
Rework AccessControl's notifications.
 2022-05-24 17:47:45 +02:00
Raúl Marín
2f37ad7fb8 Improve comment 2022-05-24 17:11:15 +02:00
Raúl Marín
a86aa43baa Merge remote-tracking branch 'blessed/master' into floating_seconds_2 2022-05-24 17:10:45 +02:00
Maksim Kita
6fb51e8bd3 Function hasAll added dynamic dispatch 2022-05-24 17:06:06 +02:00
Vladimir C
0eaf22b370
Merge pull request #37453 from vdimir/join_auto_lc_nullable_bug 2022-05-24 16:28:23 +02:00
Vladimir C
bec4ae87c9
Merge pull request #37472 from amosbird/joinpushdown 2022-05-24 16:08:26 +02:00
Maksim Kita
86180614e7 Fixed tests 2022-05-24 15:33:03 +02:00
Anton Popov
e96af9fd75 better binary serialization of ColumnObject 2022-05-24 13:16:11 +00:00
alesapin
9a19309e69 Slightly better fix 2022-05-24 14:46:29 +02:00
Maksim Kita
bdc537ead3 Column compareImpl devirtualize compare call 2022-05-24 14:28:33 +02:00
Alexander Tokmakov
229d35408b
Merge pull request #37398 from ClickHouse/fixes_for_transactions 
Fixes for transactions
 2022-05-24 15:28:01 +03:00
alesapin
3ca7a8831b Revert "fix deadlock during fetching part" 
This reverts commit 6ae8a26fae.
 2022-05-24 14:26:06 +02:00
Maksim Kita
e6e4b2826d Dynamic dispatch infrastructure style fixes 2022-05-24 14:25:29 +02:00
alesapin
a5ba6bca95 Merge branch 'master' into metahys-fix_fetching_part_deadlock 2022-05-24 14:07:47 +02:00
Kruglov Pavel
6c9a524f6b
Merge pull request #37192 from Avogar/formats-with-names 
Improve performance and memory usage for select of subset of columns for some formats
 2022-05-24 13:28:14 +02:00
Kruglov Pavel
9c87424cd7
Merge pull request #37458 from Avogar/fix-parquet-nested 
Fix named tuples output in ORC/Arrow/Parquet formats
 2022-05-24 13:24:05 +02:00
alesapin
ea19f1f18c
Merge pull request #37440 from ClickHouse/fix-keeper-force-recovery-one-node 
Fix `clickhouse-keeper` force recovery for single node cluster
 2022-05-24 13:03:56 +02:00
Amos Bird
c25ef92139
Fix tests 2022-05-24 18:57:55 +08:00
Amos Bird
093d315756
partition pruning for s3 2022-05-24 18:57:55 +08:00
alesapin
164f8227f9 Fix memory sanitizer 2022-05-24 12:38:11 +02:00
Maksim Kita
712b000f2a
Merge pull request #37443 from kitaisreal/functions-normalize-utf8-fix 
Functions normalize utf8 fix
 2022-05-24 11:11:15 +02:00
李扬
e33cfc889c
Fix bug of datetime64 parsed from string '1969-12-31 23:59:59.123' (#37039) 2022-05-24 10:47:17 +02:00
Alexander Gololobov
7d0ed7e51a Remove eigen library 2022-05-24 10:24:50 +02:00
Alexander Gololobov
caad1435d5 Optimized the case when one the argumnets is Const 2022-05-24 10:24:50 +02:00
Alexander Gololobov
65fbda436a Do computations on the raw input data without copying to Eigen::Matrix 2022-05-24 10:24:50 +02:00
Bharat Nallan Chakravarthy
6e49b76cfd try suppress h3 asan errors 2022-05-24 10:22:46 +05:30
Amos Bird
76ddb39d02
refactor format 2022-05-24 12:09:00 +08:00
Amos Bird
983e52cd3f
Aggresive filter pushdown for join 2022-05-24 12:08:42 +08:00
taiyang-li
c7e68d664d fix building 2022-05-24 10:58:45 +08:00
Igor Nikonov
838a6c6f61 Column declaration: [NOT] NULL right after type 
+ fixed: data_type_default_nullable=true, it didn't make columns nullable
         if the column declaration contains default expression w/o type

Issue #37229
 2022-05-23 22:03:45 +00:00
Maksim Kita
168814ed78
Merge pull request #37442 from kitaisreal/compressed-write-buffer-added-comment 
CompressedWriteBuffer added comment
 2022-05-23 23:44:33 +02:00
mergify[bot]
51ff49a0ee
Merge branch 'master' into i_object_storage 2022-05-23 20:29:49 +00:00
Nikolai Kochetov
fd97a9d885 Move some resources 2022-05-23 19:47:32 +00:00
Alexander Tokmakov
fa21121f77 fix 2022-05-23 21:17:52 +02:00
Alexander Tokmakov
a262492cc1 slightly better 2022-05-23 20:53:33 +02:00
avogar
3651ef93fe Fix performance test 2022-05-23 17:42:13 +00:00
Alexander Tokmakov
6bc68c0cbc Merge branch 'master' into fixes_for_transactions 2022-05-23 18:49:21 +02:00
Kruglov Pavel
39b37a3240
Merge pull request #37455 from ClickHouse/Avogar-patch-5 
Fix failed thread stateless tests
 2022-05-23 18:38:38 +02:00
Antonio Andelic
e91e7fdba7 Fix style 2022-05-23 15:33:29 +00:00
avogar
034c7122be Mark JSONColumns supports subset of columns 2022-05-23 15:26:01 +00:00
Vladimir C
14a820c2a9
Merge pull request #37336 from vdimir/fix_clash_const_aggegate_join 2022-05-23 17:09:30 +02:00
Maksim Kita
996241493f
Merge pull request #37447 from kitaisreal/binary-function-vectorized-remove-macro 
BinaryFunctionVectorized remove macro
 2022-05-23 16:50:12 +02:00
Dmitry Novik
e9187ec4b7 Overcommit: update defaults, exception message and add ProfileEvent 2022-05-23 14:35:09 +00:00
alesapin
1e1e6d4fa0 Review fixes 2022-05-23 16:24:06 +02:00
avogar
ce4adb447f Fix named tuples output in ORC/Arrow/Parquet formats 2022-05-23 14:21:08 +00:00
alesapin
2064934e59
Update src/Disks/ObjectStorages/DiskObjectStorage.cpp 
Co-authored-by: Kseniia Sumarokova <54203879+kssenii@users.noreply.github.com>
 2022-05-23 16:02:29 +02:00
alesapin
1a78ea75bb
Update src/Disks/ObjectStorages/DiskObjectStorageMetadataHelper.h 
Co-authored-by: Kseniia Sumarokova <54203879+kssenii@users.noreply.github.com>
 2022-05-23 16:01:51 +02:00
alesapin
1b17086266
Update src/Disks/ObjectStorages/DiskObjectStorageMetadataHelper.h 
Co-authored-by: Kseniia Sumarokova <54203879+kssenii@users.noreply.github.com>
 2022-05-23 16:01:44 +02:00
Nikolai Kochetov
9756b759c6 Move some resources 2022-05-23 13:46:57 +00:00
Kruglov Pavel
a0aa841909
Fix failed thread stateless tests 2022-05-23 15:15:45 +02:00
vdimir
dabb150a95
Fix cast lowcard of nullable in JoinSwitcher 2022-05-23 13:05:59 +00:00
avogar
37b66c8a9e Check format name on storage creation 2022-05-23 12:48:48 +00:00
alesapin
cef3d96cfe
Update src/Disks/ObjectStorages/S3/S3ObjectStorage.h 
Co-authored-by: Kseniia Sumarokova <54203879+kssenii@users.noreply.github.com>
 2022-05-23 14:46:25 +02:00
alesapin
e76597e5d3
Update src/Core/Settings.h 
Co-authored-by: Kseniia Sumarokova <54203879+kssenii@users.noreply.github.com>
 2022-05-23 14:46:21 +02:00
Maksim Kita
fe21b4ca9e Fixed style check 2022-05-23 14:41:07 +02:00
metahys
6ae8a26fae fix deadlock during fetching part 2022-05-23 20:08:04 +08:00
Maksim Kita
008de5c779
Merge pull request #37438 from kitaisreal/function-binary-representation-style-fixes 
FunctionBinaryRepresentation style fixes
 2022-05-23 13:54:15 +02:00
alesapin
216184dfd3 Rename one more file 2022-05-23 13:20:32 +02:00
Maksim Kita
e550843d56 BinaryFunctionVectorized remove macro 2022-05-23 12:45:16 +02:00
Maksim Kita
585b86446e Added hierarchical_index_bytes_allocated column in system.dictionaries 2022-05-23 12:42:00 +02:00
Maksim Kita
f76e3801de Fixed tests 2022-05-23 12:42:00 +02:00
Maksim Kita
7e4c950bd9 Fixed style check 2022-05-23 12:42:00 +02:00
Maksim Kita
be9c3d9bd4 Fixed build 2022-05-23 12:42:00 +02:00
Maksim Kita
25d6bd1f34 Dictionaries optimize hierarchical index structure 2022-05-23 12:42:00 +02:00
Maksim Kita
1142e05683 Dictionaries allow to specify bidirectional for hierarhical attribute 2022-05-23 12:42:00 +02:00
Maksim Kita
100afa8bcf Dictionary getDescendants performance improvement 2022-05-23 12:42:00 +02:00
Sergei Trifonov
b6bf283f4d
Merge pull request #37243 from Vxider/mv-fix-dependency 
Fix adding dependency when MaterializedView is dropped
 2022-05-23 12:33:54 +02:00
Antonio Andelic
87d445295f
Merge pull request #37342 from azat/projections-optimize_aggregation_in_order-fix 
Fix projections with GROUP/ORDER BY in query and optimize_aggregation_in_order
 2022-05-23 12:29:16 +02:00
Maksim Kita
78782de887 Functions normalizeUTF8 logical error fix 2022-05-23 12:19:14 +02:00
Kruglov Pavel
4067d646df
Delete redundant file 2022-05-23 12:14:55 +02:00
Kruglov Pavel
f539fb835d
Merge branch 'master' into formats-with-names 2022-05-23 12:14:20 +02:00
Kruglov Pavel
ce48e8e102
Merge pull request #36975 from Avogar/json-columns-formats 
Add columnar JSON formats
 2022-05-23 12:11:28 +02:00
Kruglov Pavel
754e675ec3
Merge pull request #37253 from Avogar/fix-defaults 
Fix inserting defaults for missing values in columnar formats
 2022-05-23 12:10:14 +02:00
Maksim Kita
3a22d8ef2c CompressedWriteBuffer added comment 2022-05-23 12:09:44 +02:00
Kruglov Pavel
9bc74439c1
Merge pull request #37327 from Avogar/arrow-strings 
Allow to use String type instead of Binary in Arrow/Parquet/ORC formats
 2022-05-23 12:05:33 +02:00
Antonio Andelic
c268296fc6 Fix single node force recovery and add tests 2022-05-23 09:53:46 +00:00
Kseniia Sumarokova
ca3feddf73
Merge pull request #37426 from Vxider/wv-simplify 
Remove unused code in WindowView
 2022-05-23 11:43:51 +02:00
taiyang-li
c120b19802 fix building 2022-05-23 17:42:20 +08:00
taiyang-li
f13dcad58c Merge branch 'master' into fix_settings 2022-05-23 17:36:45 +08:00
Maksim Kita
98bb34f2f2 FunctionBinaryRepresentation style fixes 2022-05-23 10:59:33 +02:00
Robert Schulze
e25ca139cd
Implement SQL functions (NOT) (I)LIKE() + MATCH() with non-const needles 
With this commit, SQL functions LIKE and MATCH and their variants can
work with non-const needle arguments. E.g.

  create table tab
    (id UInt32, haystack String, needle String)
    engine = MergeTree()
    order by id;

  insert into tab values
  (1, 'Hello', '%ell%')
  (2, 'World', '%orl%')

  select id, haystack, needle, like(haystack, needle)
  from tab;

For that, methods vectorVector() and vectorFixedVector() were added to
MatchImpl. The existing code for const needles has an optimization where
the compiled regexp is cached. The new code expects a different needle
per row and consequently does not cache the regexp.
 2022-05-23 09:41:28 +02:00
Alexey Milovidov
c541d30a38 Fix error 2022-05-23 02:48:05 +02:00
Alexey Milovidov
cc985d9b92 Fix error 2022-05-23 02:30:33 +02:00
Alexey Milovidov
698e5e5352
Merge pull request #37415 from Joeywzr/gen_uuid 
Generate multiple columns with UUID
 2022-05-23 00:29:42 +03:00
alesapin
0c61579859 More comments 2022-05-22 23:15:42 +02:00
Alexey Milovidov
d32d45e12e
Merge pull request #37400 from ndchikin/window_frame_fix 
WindowTransform::moveRowNumber fix
 2022-05-22 20:35:00 +03:00
Robert Schulze
4829ae8380
Replace overly clever const argument logic by something simpler 
The previous logic was smart but too inflexible to support the next
commits. Replace by a simple pushdown logic where string search
implementations return their const arguments instead of having the
common class figure these out based on properties/traits.
 2022-05-22 17:50:38 +02:00
Robert Schulze
0299cc87e4
Improve naming consistency of string search code 
Just renamings, nothing major ...
 2022-05-22 17:50:38 +02:00
alesapin
06c3dd69c0 Move directories 2022-05-22 13:51:48 +02:00
Vxider
fcd317a642 update windowview create syntax 2022-05-22 10:18:34 +00:00
alesapin
eb69d963e2 Missed change 2022-05-22 12:16:50 +02:00
alesapin
c8d92b87c8 Merge branch 'master' into i_object_storage 2022-05-22 12:16:10 +02:00
Vxider
5f84f06d6d simplify windowview 2022-05-22 10:13:16 +00:00
Robert Schulze
19d53c14fa
Merge pull request #37382 from ClickHouse/wc++98-compat-extra-semi 
Enable -Wc++98-compat-extra-semi
 2022-05-22 09:40:45 +02:00
Memo
15a76d012f add NUMBER_OF_ARGUMENTS_DOESNT_MATCH defination 2022-05-22 13:38:47 +08:00
alesapin
b3bc0a18a0 fix test 2022-05-22 00:13:01 +02:00
Alexey Milovidov
4bfbb0b7ba
Merge pull request #37388 from CurtizJ/explicit-bool-operator 
Mark all `operator bool()` as explicit
 2022-05-22 00:18:19 +03:00
Alexey Milovidov
790f442362
Merge pull request #37407 from ClickHouse/certificates-mem-leak 
Fix possible memory leaks in system.certificates implementation
 2022-05-22 00:15:30 +03:00
Yakov Olkhovskiy
d878f193d8
Merge pull request #37013 from mnutt/hashid 
Add hashid support
 2022-05-21 17:14:54 -04:00
Kseniia Sumarokova
ec2a1c8ba5
Merge pull request #37188 from Vxider/wv-alter 
ALTER TABLE MODIFY QUERY support for WindowView
 2022-05-21 22:59:32 +02:00
Alexey Milovidov
6322d064b8
Merge pull request #37384 from Avogar/short-circuit-process 
Try to improve short circuit functions processing
 2022-05-21 23:43:56 +03:00
Alexey Milovidov
d1678c9662
Update ExpressionActions.cpp 2022-05-21 23:43:26 +03:00
Yakov Olkhovskiy
ce1df15e1c
Merge pull request #37377 from ClickHouse/sha256_hash-with-salt 
Allow SALT in CREATE USER user1 IDENTIFIED WITH sha256_hash
 2022-05-21 14:36:22 -04:00
Vasily Nemkov
c12f826d22 Implemented changing comment to a ReplicatedMergeTree table 2022-05-21 20:08:30 +03:00
Memo
942af133e5 init 2022-05-21 23:54:12 +08:00
Maksim Kita
0d69f35b6a Fixed style check 2022-05-21 14:54:45 +02:00
Vitaly Baranov
9ec3b35cf2 Use AccessExpireCache instead of ExpireCache. 2022-05-21 10:15:44 +02:00
Vitaly Baranov
58f4a86ec7 Rework notifications used in access management. 2022-05-21 10:15:39 +02:00
Alexander Tokmakov
1b453f517d fix 2022-05-21 02:32:35 +02:00
Yakov Olkhovskiy
4c13b52b6a scope guard resources 2022-05-20 19:59:17 -04:00
Yakov Olkhovskiy
6e3f741a25 allow SALT only for SHA256_HASH 2022-05-20 17:21:04 -04:00
ndchikin
4859108feb WindowTransform::moveRowNumber fix 2022-05-20 20:49:10 +00:00
Maksim Kita
42439aeb3c Improve performance of number comparison functions 2022-05-20 22:42:48 +02:00
Alexander Tokmakov
44f2d4529a better waiting, add fault injection 2022-05-20 22:10:01 +02:00