Dmitry Novik
3a9239b79f
Revert "RFC: Fix converting types for UNION queries (may produce LOGICAL_ERROR)"
2022-05-27 04:05:32 +02:00
Azat Khuzhin
a813f5996e
Fix converting types for UNION queries (may produce LOGICAL_ERROR)
...
CI founds [1]:
2022.02.20 15:14:23.969247 [ 492 ] {} <Fatal> BaseDaemon: (version 22.3.1.1, build id: 6082C357CFA6FF99) (from thread 472) (query_id: a5187ff9-962a-4e7c-86f6-8d48850a47d6) (query: SELECT 0., round(avgWeighted(x, y)) FROM (SELECT toDate(toDate('214748364.8', '-922337203.6854775808', '-0.1', NULL) - NULL, 10.000100135803223, '-2147483647'), 255 AS x, -2147483647 AS y UNION ALL SELECT y, NULL AS x, 2147483646 AS y)) Received signal Aborted (6)
[1]: https://s3.amazonaws.com/clickhouse-test-reports/0/26d0e5438c86e52a145aaaf4cb523c399989a878/fuzzer_astfuzzerdebug,actions//report.html
The problem is that subqueries returns different headers:
- first query -- x, y
- second query -- y, x
v2: Make order of columns strict only for UNION
https://s3.amazonaws.com/clickhouse-test-reports/34775/9cc8c01a463d18c471853568b2f0af659a4e643f/stateless_tests__address__actions__[2/2].html
Fixes: 00597_push_down_predicate_long
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-05-25 20:31:47 +03:00
taiyang-li
c7e68d664d
fix building
2022-05-24 10:58:45 +08:00
taiyang-li
c120b19802
fix building
2022-05-23 17:42:20 +08:00
taiyang-li
9f3dadf17c
Merge branch 'master' into fix_settings
2022-05-17 16:19:29 +08:00
vdimir
9b24e0d260
Apply optimizeCountConstantAndSumOne before JOINs
2022-05-10 14:30:33 +00:00
taiyang-li
fd878f7e7b
change as requested
2022-05-10 12:06:50 +08:00
Maksim Kita
57444fc7d3
Merge pull request #36444 from rschu1ze/clang-tidy-fixes
...
Clang tidy fixes
2022-04-21 16:11:27 +02:00
Robert Schulze
b24ca8de52
Fix various clang-tidy warnings
...
When I tried to add cool new clang-tidy 14 warnings, I noticed that the
current clang-tidy settings already produce a ton of warnings. This
commit addresses many of these. Almost all of them were non-critical,
i.e. C vs. C++ style casts.
2022-04-20 10:29:05 +02:00
Robert Schulze
118e94523c
Activate clang-tidy warning "readability-container-contains"
...
This check suggests replacing <Container>.count() by
<Container>.contains() which is more speaking and in case of
multimaps/multisets also faster.
2022-04-18 23:53:11 +02:00
Yakov Olkhovskiy
7dbe8bc2dc
major bugs fixed, tests added, docs updated
2022-04-07 01:21:24 -04:00
vdimir
1122db89db
Use float devision for avg after optimize_fuse_sum_count_avg
2022-03-28 11:26:42 +00:00
Anton Popov
fcdebea925
Merge remote-tracking branch 'upstream/master' into HEAD
2022-02-25 13:41:30 +03:00
mergify[bot]
dd947f964c
Merge branch 'master' into mv_cacheable_scalars
2022-02-07 10:07:26 +00:00
Anton Popov
78b9f15abb
Merge remote-tracking branch 'upstream/master' into HEAD
2022-01-30 03:24:37 +03:00
Raúl Marín
4b5ab80e3b
Better scalar cache handling
...
- Fixes global CTE scalar cache.
- Adds MVs back (views dependent on the source are cached locally and others globally
2022-01-26 17:36:45 +01:00
Anton Popov
e8ce091e68
Merge remote-tracking branch 'upstream/master' into HEAD
2022-01-21 20:11:18 +03:00
Azat Khuzhin
97acf190ed
Replace old _shard_num implementation with shardNum() function
...
_shard_num via constant identifier (from #7624 ) has too much issues,
take a look at #16947 for example.
shardNum() function (from #27020 ) should works better.
This changes the behaviour of _shard_num slightly, now it returns nested
shard number in case of nested distributed tables (Distributed over
Distributed), but this should be minor.
v2: Rewrite _shard_num to shardNum() in TreeRewriter
2022-01-10 21:21:24 +03:00
Maksim Kita
ebff389701
Merge pull request #32972 from kitaisreal/containers-iteration-fix-erase
...
Containers iteration fix erase
2021-12-20 16:47:19 +03:00
Nikolai Kochetov
2e62f086a1
Merge pull request #32751 from ClickHouse/fix-32668
...
Apply some more optimizations to NO_QUERY ast.
2021-12-20 15:47:25 +03:00
Maksim Kita
3feab5a975
Containers iteration fix erase
2021-12-20 13:42:31 +03:00
Nikolai Kochetov
93a33d52dc
Merge branch 'fix-32668' of github.com:yandex/ClickHouse into fix-32668
2021-12-17 20:37:29 +03:00
Nikolai Kochetov
c8a92c046f
Another try
2021-12-17 20:36:37 +03:00
Anton Popov
99ebabd822
Merge remote-tracking branch 'upstream/master' into HEAD
2021-12-17 19:02:29 +03:00
Nikolai Kochetov
20b88d9b4c
Always apply const if optimixation.
2021-12-17 15:31:30 +03:00
mergify[bot]
3a7179756f
Merge branch 'master' into fix-32668
2021-12-15 08:31:10 +00:00
Nikolai Kochetov
d394f0e753
Allpy some more optimizations to NO_QUERY ast.
2021-12-14 17:19:18 +03:00
dongyifeng
9df664e1c9
fix bug when remove unneeded columns in subquery ( #32289 )
2021-12-07 11:09:39 +03:00
Anton Popov
6f4d9a53b2
Merge remote-tracking branch 'origin/sparse-serialization' into HEAD
2021-12-01 15:54:33 +03:00
Raúl Marín
7781fc12ed
Reduce dependencies on ASTSelectWithUnionQuery.h
...
521 -> 77 files requiring changes
2021-11-26 19:27:16 +01:00
Anton Popov
a20922b2d3
Merge remote-tracking branch 'origin/sparse-serialization' into HEAD
2021-11-09 15:36:25 +03:00
vdimir
56bc802ee2
Support join on constant
2021-11-08 15:44:13 +03:00
vdimir
08545795b5
Separate option for enabling fuse syntax for sum, avg, count
2021-11-03 15:13:09 +03:00
vdimir
039af96b61
Set execute_scalar_subqueries in MutationsInterpreter only for MergeTree and prepareInterpreterSelectQuery
2021-10-21 18:05:46 +03:00
vdimir
ed0bb2cec2
Fix deadlock on ALTER with scalar subquery to the same table
2021-10-21 14:02:51 +03:00
Amos Bird
635783fb66
Only do TreeOptimizer for initial queries
2021-10-11 20:47:35 +08:00
Kseniia Sumarokova
438a981e76
Update TreeRewriter.cpp
2021-10-05 11:25:50 +03:00
kssenii
f1ea186bff
Exists
2021-10-04 13:12:30 +00:00
Ilya Golshtein
7ebc16c1b3
get rid of DNF and related features in ORs in JOIN
2021-09-28 14:11:33 +03:00
Ilya Golshtein
626bfdf23c
compatible filter conditions, fixes and new tests in ORs in JOIN
2021-09-28 14:11:33 +03:00
Ilya Golshtein
bbd548e81d
bypass filer conditions in DNF in ORs in JOIN (part 2)
2021-09-28 14:11:33 +03:00
Ilya Golshtein
336b2a4c68
bypass filer conditions in DNF in ORs in JOIN (part 1)
2021-09-28 14:11:33 +03:00
Ilya Golshtein
17e6cfbefb
DNF bugfix in ORs in JOIN
2021-09-28 14:11:32 +03:00
Ilya Golshtein
aa4751a632
checkStackSize moved to the top of DNF::distributed in ORs in JOIN
2021-09-28 14:11:32 +03:00
Ilya Golshtein
78ad6bf529
MAX_ORS, checkStackSize and beautification per review in ORs in JOIN
2021-09-28 14:11:32 +03:00
Ilya Golshtein
637ff19f79
optimizeDisjuncts in ORs in JOIN
2021-09-28 14:11:31 +03:00
vdimir
3b35ab6e8c
Not implemented for asof and auto join with multiple ORs
2021-09-28 14:11:31 +03:00
vdimir
71b6c9414c
Minor changes related to JOIN ON ORs
2021-09-28 14:11:31 +03:00
vdimir
0a9a028c6f
fix
2021-09-28 14:11:31 +03:00
vdimir
46187a73ee
wip
2021-09-28 14:11:31 +03:00
Ilya Golshtein
db50015eed
review changes 1 - ASTPtr, some comments
2021-09-28 14:11:29 +03:00
Ilya Golshtein
3766d47f31
ORs in JOINs
2021-09-28 14:11:28 +03:00
Vladimir C
bc7c41efff
Merge pull request #26657 from hexiaoting/fuse_quantile
2021-09-21 12:33:07 +03:00
Maksim Kita
de20e04dfd
Added executable user defined functions
2021-09-17 18:42:59 +03:00
vdimir
a3304a87a4
Rename optimize_syntax_fuse_aggregate -> optimize_syntax_fuse_functions
2021-09-14 15:27:12 +03:00
vdimir
a1463d5719
Merge remote-tracking branch 'origin/master' into fuse_quantile
2021-09-14 15:23:06 +03:00
vdimir
5a1aeeb044
Use optimize_syntax_fuse_aggregate instead of optimize_fuse_sum_count_avg and optimize_fuse_quantile
2021-09-14 15:21:16 +03:00
vdimir
a3db56f056
Revert "fuse multi setting options into one when Optimize"
...
This reverts commit bbd7799375
2021-09-14 15:12:21 +03:00
vdimir
3d2d994c5d
Skip JOIN OR/USING sections in replaceAliasColumnsInQuery
2021-09-14 14:13:19 +03:00
hexiaoting
bbd7799375
fuse multi setting options into one when Optimize
2021-09-14 18:51:13 +08:00
vdimir
a7f770ecb7
Do not replaceAliasColumnsInQuery for JOIN OR/USING sections
2021-09-14 13:08:20 +03:00
Anton Popov
4c388e3d84
Merge remote-tracking branch 'origin/sparse-serialization' into HEAD
2021-09-09 14:10:16 +03:00
vdimir
c4ffc2879f
Perform join type inference for USING at ExpressionAnalyzer instead of TreeRewriter
2021-09-01 15:05:47 +03:00
vdimir
e8e26463bf
Change signature of ASTSelectQuery::arrayJoinExpressionList
2021-08-31 15:27:44 +03:00
vdimir
3443b3b5c4
Separate functoin buildJoinedPlan, minor refactoring
2021-08-31 15:27:44 +03:00
vdimir
30e112d6a6
Do not convert join key columns for storage/dict join
2021-08-31 15:27:43 +03:00
Anton Popov
61239343e3
Merge remote-tracking branch 'origin/sparse-serialization' into HEAD
2021-08-20 16:33:30 +03:00
Maksim Kita
01682a86b3
Updated user defined functions implementation
2021-08-19 00:54:55 +03:00
vdimir
910787c5b6
Improve aliased_column_distributed_bug, fix check
2021-08-16 18:04:20 +03:00
vdimir
db187c6b2d
Fix aliased_column_distributed_bug
2021-08-16 17:26:14 +03:00
Anton Popov
9a9aebc644
fix tests
2021-08-04 20:38:10 +03:00
Anton Popov
699a3d9031
implement legacy_column_name_of_tuple_literal in less intrusive way
2021-08-03 21:03:24 +03:00
Anton Popov
e36736b50c
Merge remote-tracking branch 'origin/sparse-serialization' into HEAD
2021-08-02 22:52:02 +03:00
Vladimir
576b407804
Support conditions in JOIN ON section ( #24420 )
...
* Try to enforce table identification in CollectJoinOnKeysMatcher
* Support filtering conditions in JOIN ON for HashJoin
* Correct handle non equi join
* Update test 00878_join_unexpected_results
* Join on filters calculated as one row before join
* Do not lookup key in hash join if condition for row is not hold
* better
* Support filtering conditions in JOIN ON for MergeJoin
* Support Nullable mask in JOIN ON section
* Fix style in Interpreters/TableJoin.cpp
* Change return type of getColumnAsMask in join_common to ColumnPtr
* Handle Nullable(Nothing) type in JOIN ON section, add test cases
* Fix type cast JoinCommon::getColumnAsMask
* Check type if conditions in JOIN ON section, support functions
* Update tests with JOIN ON
* Style changes, add comments for conditions in JOIN ON section
* Add test cases for join on condtions
* JOIN ON key1 = key2 AND (cond1 OR cond2)
* Remove CollectJoinOnKeysVisitor has_join_keys
* Add test cases for join on nullable/lc conditions
* Fix style
* Change error code 48 to 403 in join on tests
* Fix whitespace
2021-07-21 20:03:33 +03:00
Anton Popov
f99374cca6
Merge remote-tracking branch 'origin/sparse-serialization' into HEAD
2021-07-20 18:20:21 +03:00
Amos Bird
d74aa64f6b
Fix column alias rewriting
2021-07-18 23:27:19 +08:00
Anton Popov
3ed7f5a6cc
dynamic subcolumns: add snapshot for storage
2021-07-09 06:15:41 +03:00
vdimir
5d7fc61bd5
Merge branch 'master' into join-materialized-columns
2021-07-08 12:30:00 +03:00
Anton Popov
072e65b728
Merge remote-tracking branch 'origin/sparse-serialization' into HEAD
2021-07-07 17:20:38 +03:00
Azat Khuzhin
15a14d6da2
Fix "No available columns" for Merge() storage
2021-06-29 00:39:38 +03:00
vdimir
2949cd1e6f
Support ALIASed columns in JOIN ON expression
2021-06-25 14:50:02 +03:00
vdimir
241b64d02c
Support ALIASed columns for right joined table
2021-06-24 17:57:21 +03:00
vdimir
84e02911cf
Materialized columns for joined table, don't add to asterisk select
2021-06-24 17:14:36 +03:00
vdimir
b00efaf3d1
Add materialized columns to joined columns
2021-06-23 17:03:39 +03:00
Amos Bird
0f7728cd5e
Fix RBAC test again
2021-06-17 23:00:03 +08:00
Amos Bird
aaac231c6a
Record expanded aliases for RBAC
2021-06-15 18:28:13 +08:00
Amos Bird
e3ae2f6e7a
Rewrite more alias columns
2021-06-15 17:54:28 +08:00
alexey-milovidov
05d1af153c
Merge branch 'master' into rename-const-context-ptr
2021-06-12 03:25:09 +03:00
Anton Popov
bb6f0dfbb8
Merge pull request #24406 from CurtizJ/rewrite-functions-to-subcolumns
...
Optimize some functions to subcolumns
2021-06-09 11:07:09 +03:00
alesapin
11a1606c15
Less strict
2021-06-08 12:54:00 +03:00
Anton Popov
6a5daca135
dynamic subcolumns: new format and several fixes
2021-06-08 12:33:04 +03:00
alesapin
5284f192ee
Ban loop aliases in table definitions
2021-06-07 23:59:38 +03:00
Nikolai Kochetov
dbaa6ffc62
Rename ContextConstPtr to ContextPtr.
2021-06-01 15:20:52 +03:00
Anton Popov
9f52362b81
Merge remote-tracking branch 'origin/sparse-serialization' into HEAD
2021-05-31 23:08:45 +03:00
Anton Popov
b24f65e1ad
Merge remote-tracking branch 'upstream/master' into HEAD
2021-05-31 23:04:52 +03:00
Alexander Kuzmenkov
3f57fc085b
remove mutable context references from functions interface
...
Also remove it from some visitors.
2021-05-28 19:45:37 +03:00
Anton Popov
0bdf9d207c
Merge remote-tracking branch 'origin/sparse-serialization' into HEAD
2021-05-26 00:25:47 +03:00
Anton Popov
13cfedd188
optimize some functions to subcolumns
2021-05-21 21:48:19 +03:00
Amos Bird
ebaf42a448
Reformat and fix some tests
2021-05-11 18:12:27 +08:00
Amos Bird
264cff6415
Projections
...
TODO (suggested by Nikolai)
1. Build query plan fro current query (inside storage::read) up to WithMergableState
2. Check, that plan is simple enough: Aggregating - Expression - Filter - ReadFromStorage (or simplier)
3. Check, that filter is the same as filter in projection, and also expression calculates the same aggregation keys as in projection
4. Return WithMergableState if projection applies
3 will be easier to do with ActionsDAG, cause it sees all functions, and dependencies are direct (but it is possible with ExpressionActions also)
Also need to figure out how prewhere works for projections, and
row_filter_policies.
wip
2021-05-11 18:12:23 +08:00
Anton Popov
e44706911e
dynamic columns: better getting of sample block
2021-05-05 02:02:54 +03:00
Amos Bird
8a3b5c1fab
Add _partition_value virtual column
2021-04-27 16:15:59 +08:00
Anton Popov
644df6be7d
dynamic subcolumns: wip
2021-04-24 07:09:01 +03:00
Alexander Kuzmenkov
5e0fb440a5
Update TreeRewriter.cpp
2021-04-16 13:35:02 +03:00
Alexander Kuzmenkov
2489b6af96
cleanup
2021-04-15 19:40:49 +03:00
Alexander Kuzmenkov
3b95b637a5
Merge remote-tracking branch 'origin/master' into HEAD
2021-04-15 18:19:53 +03:00
Vladimir
8c7ffda676
Merge pull request #22753 from ClickHouse/revert-19685-dev_joinon
2021-04-15 12:42:50 +03:00
Nikolai Kochetov
e731dfe650
Merge pull request #22991 from ClickHouse/untuple-and-subquery
...
Fix subquery with untuple.
2021-04-13 09:53:13 +03:00
hexiaoting
77c460e8d1
Merge remote-tracking branch 'origin/master' into dev-sumcount
2021-04-13 11:14:14 +08:00
Nikolai Kochetov
0448e1415f
Fix subquery with untuple.
2021-04-12 15:15:55 +03:00
Ivan
495c6e03aa
Replace all Context references with std::weak_ptr ( #22297 )
...
* Replace all Context references with std::weak_ptr
* Fix shared context captured by value
* Fix build
* Fix Context with named sessions
* Fix copy context
* Fix gcc build
* Merge with master and fix build
* Fix gcc-9 build
2021-04-11 02:33:54 +03:00
Vladimir
9fe20c1628
Revert "Move conditions from JOIN ON to WHERE"
2021-04-07 14:57:20 +03:00
vdimir
3f464595eb
Accurate removing 'join' part for queries to 'Merge' engine
2021-04-01 14:31:57 +03:00
alexey-milovidov
5c15b8a896
Merge branch 'master' into virtualutil
2021-03-30 23:59:44 +03:00
Amos Bird
69204e1d21
Add prefer_column_name_to_alias settings ( #22044 )
2021-03-30 16:51:45 +03:00
Alexey Milovidov
230b2930b7
Merge branch 'master' into virtualutil
2021-03-30 00:43:00 +03:00
hexiaoting
ab2aaa7fe7
Rewriter visitor and add performance test
2021-03-12 15:29:38 +08:00
hexiaoting
055073931a
Fix build error
2021-03-09 17:03:24 +08:00
hexiaoting
9dab51698f
Rewriter fuse logical
2021-03-08 11:58:18 +08:00
vdimir
b0f6f30298
Revert "Refactor join: make IJoin implementations independent from TableJoin"
...
This reverts commit e6406c3f4c
2021-03-05 17:34:43 +03:00
vdimir
e6406c3f4c
Refactor join: make IJoin implementations independent from TableJoin
2021-03-05 16:38:49 +03:00
Amos Bird
93b661ad5a
partition id pruning
2021-03-04 19:43:03 +08:00
hexiaoting
4fe75ad168
fix error
2021-03-03 17:18:53 +08:00
hexiaoting
e116346cbb
Merge remote-tracking branch 'origin/master' into dev-sumcount
2021-03-02 10:11:50 +08:00
hexiaoting
16333e9f3a
Suppport fuse aggregate functions:sum/avg/count to sumCount
2021-03-01 18:04:34 +08:00
vdimir
b7c7c97d10
Merge branch 'master' into join-cast-types-v2
2021-03-01 11:42:21 +03:00
alexey-milovidov
0067a3beac
Merge pull request #19801 from ucasFL/ms
...
continue of #19487
2021-02-25 19:12:47 +03:00
vdimir
4c6cde4d53
Merge remote-tracking branch 'upstream/master' into join-cast-types-v2
2021-02-25 16:24:28 +03:00
vdimir
1e37d7c84f
Add comments to TableJoin::inferJoinKeyCommonType
2021-02-19 14:28:41 +03:00
vdimir
3a7eddcf3a
Remove addRequiredLeftColumn, some tests for join on different types
2021-02-19 14:28:40 +03:00
vdimir
a378bd08aa
Perform implicit type conversion for JOIN ON keys
2021-02-19 14:28:40 +03:00
vdimir
435f63f42b
Calculate common type for join using in TreeRewriter
...
Split before_join and converting_join_columns dags
Add more detailed comments for column type conversion for join using
2021-02-19 14:28:32 +03:00
Amos Bird
2c4bc43014
Backward compatible
2021-02-18 20:20:29 +08:00
Amos Bird
2dc7ba160a
Better
2021-02-18 20:20:28 +08:00
Amos Bird
77fd060665
Normalize function names
2021-02-18 20:20:28 +08:00
vdimir
3e2c9ad011
Merge remote-tracking branch 'origin/master' into dev_joinon
2021-02-16 17:08:10 +03:00
vdimir
e39215e38b
Fix has_some condition on CollectJoinOnKeysVisitor
2021-02-16 11:03:02 +03:00
vdimir
5273242f86
Minor changes move ON to WHERE for INNER JOIN
2021-02-15 23:26:29 +03:00
hexiaoting
28b981a76b
Fix style error and test cases error
2021-02-09 18:08:55 +08:00
vdimir
241769ee7c
Handle syntax error for ARRAY JOIN with no args
2021-02-09 11:21:26 +03:00
hexiaoting
2c278f1e02
Restrict move JOINON to WHERE optimizer only to inner join
2021-02-08 13:44:50 +08:00
hexiaoting
45aee71fff
Modified some implementation
2021-02-03 18:52:20 +08:00
hexiaoting
56de38ea6a
Merge remote-tracking branch 'origin/master' into dev_joinon
2021-02-03 17:20:24 +08:00
feng lv
45cb78a67b
continue of #19487
...
fix
2021-01-29 08:22:03 +00:00
hexiaoting
4a17f5c73a
Move condistions from JOIN ON to WHERE
2021-01-27 11:24:17 +08:00
feng lv
9fdae60497
remove part of misleading exception message
2021-01-23 05:45:36 +00:00
Alexander Kuzmenkov
d0922e2985
Merge remote-tracking branch 'origin/master' into tmp
2021-01-18 23:23:49 +03:00
alexey-milovidov
15f4ae26c2
Merge pull request #17310 from CurtizJ/multiple-nested
...
Allow nested with multiple nesting and subcolumns of complex types
2021-01-17 15:00:26 +03:00
alexey-milovidov
d2366b5c6c
Merge pull request #18233 from vitlibar/fix-access-rights-for-select-count
...
Fix access rights for SELECT count()
2021-01-17 00:12:08 +03:00
alexey-milovidov
78fff6bc39
Merge branch 'master' into multiple-nested
2021-01-15 14:54:27 +03:00
