Commit Graph

506 Commits

Author SHA1 Message Date
Nikolai Kochetov
9528f7e753
Merge branch 'master' into bug-lost-column 2022-07-13 23:06:38 +02:00
Nikolai Kochetov
ba863b11a4 A little fix. Added test. 2022-07-13 21:06:15 +00:00
Yakov Olkhovskiy
3fd367ea58 check metadata_snapshot is not empty 2022-07-13 10:08:24 -04:00
Yakov Olkhovskiy
1f57f92bf7 prewhere action should preserve sorting columns 2022-07-13 03:01:23 -04:00
Dmitry Novik
d046132265 Allow to use higher-order functions in window expressions 2022-07-11 23:47:46 +00:00
Dmitry Novik
fd9d897df2 Fix ActionsDAG for window expressions 2022-07-11 23:18:44 +00:00
Dmitry Novik
29cfe33ed7 Small fix 2022-07-11 18:14:49 +00:00
Dmitry Novik
d1df66687b
Merge branch 'master' into group-by-use-nulls 2022-07-07 20:54:38 +02:00
Dmitry Novik
1587385f7a Cleanup code 2022-07-07 18:53:20 +00:00
vdimir
b27d3ec5ce
Fix build 2022-07-06 15:11:23 +00:00
vdimir
881d352e05
upd full sorting join 2022-07-06 14:28:05 +00:00
vdimir
d34a66c915
wip sorting merge 2022-07-06 14:25:09 +00:00
vdimir
1b429fc1af
wip: any left/right sorting join 2022-07-06 14:23:46 +00:00
vdimir
4a16195964
Calculate output header for full sorting merge join 2022-07-06 14:23:45 +00:00
vdimir
fa8eb35599
Pipeline for full sorting merge join 2022-07-06 14:23:44 +00:00
Vladimir C
3c7fdb604d
Merge pull request #35363 from vdimir/kv-join 2022-07-06 15:57:30 +02:00
Igor Nikonov
99de1ee109 Simple cleanup: interpreters and parsers 2022-07-05 22:13:39 +00:00
vdimir
17903117d2
better 2022-07-05 10:46:08 +00:00
vdimir
e40f799f96
handle right rey rename in direct join 2022-07-04 17:28:13 +00:00
vdimir
a1388813e4
fix src/Interpreters/ExpressionAnalyzer.cpp 2022-07-04 17:28:12 +00:00
vdimir
8359753a85
Move DirectJoin to separate file 2022-07-04 17:28:11 +00:00
vdimir
51a6c347ad
make join_alorithm settings with multiple choices 2022-07-04 17:28:11 +00:00
vdimir
0dfd400f51
check conditions before using direct join 2022-07-04 17:28:10 +00:00
vdimir
980000d04c
Change getByKeys, add StorageEmbeddedRocksDB::getByKeysImpl 2022-07-04 17:28:09 +00:00
vdimir
744c692be3
wip key value join 2022-07-04 17:28:09 +00:00
Dmitry Novik
e179449381 Fix some tests 2022-07-04 15:56:42 +00:00
Dmitry Novik
81dd90893e Merge remote-tracking branch 'origin/master' into group-by-use-nulls 2022-07-01 16:24:05 +00:00
Dmitry Novik
1d15d72211 Support NULLs in ROLLUP 2022-06-27 18:42:26 +00:00
Nikolai Kochetov
1d5ba77c87 Fix test. 2022-06-27 17:41:55 +00:00
Sergey Skvortsov
bb0e6fd835
fix: Cleanup 2022-06-23 08:41:45 +03:00
Sergey Skvortsov
14b82eceaa
feat: Add joined blocks to the GraceHashJoin in parallel 2022-06-23 04:41:21 +03:00
Dmitry Novik
f6692c34e6
Merge pull request #34632 from excitoon-favorites/optimizedprocessing
Optimized processing of ORDER BY in window functions
2022-06-20 20:03:26 +02:00
Vladimir Chebotarev
b81df2be83 Update src/Interpreters/ExpressionAnalyzer.cpp
Co-authored-by: Dmitry Novik <d1mas1k4@yandex.ru>
2022-06-20 05:15:08 +03:00
Vladimir Chebotarev
7f9557f8a3 Added optimize_read_in_window_order setting. 2022-06-20 05:15:07 +03:00
Vladimir Chebotarev
ec22f6d539 Draft. 2022-06-20 05:15:07 +03:00
Sergey Skvortsov
6696ccccb6
fix: Check for MergeJoin before GraceHashJoin 2022-06-18 21:14:07 +03:00
Sergey Skvortsov
d9e305a833 feat: Enable grace hash join conditionally 2022-06-18 02:31:56 +03:00
Sergey Skvortsov
29b6be9a00 feat: Temporarily enable grace hash join for all supported queries 2022-06-18 02:31:56 +03:00
Sergey Skvortsov
3386386a22 feat: Support grace hash join 2022-06-18 02:31:56 +03:00
Dmitry Novik
8479d880f3 Remove redundant code 2022-06-16 15:49:49 +00:00
Dmitry Novik
376412e417 Small refactoring 2022-06-16 15:41:04 +00:00
Dmitry Novik
0663f07e67 Rework expressions with window functions 2022-06-16 13:29:56 +00:00
Dmitry Novik
0c6ed20b42 Fix window expression evaluation 2022-06-13 23:11:53 +00:00
mergify[bot]
9e0d16f21d
Merge branch 'master' into window-function-expression 2022-06-10 15:31:53 +00:00
Dmitry Novik
58d79f8eed Support expressions with window functions 2022-06-04 02:01:49 +00:00
Nikolai Kochetov
fc726c236f
Merge pull request #37661 from ClickHouse/no_alias_actions
Trying to remove prewhere_info.alias_actions as they seem to always be included in prewhere_actions
2022-06-02 12:31:04 +02:00
Alexander Gololobov
e2dd6f6249 Removed prewhere_info.alias_actions 2022-05-30 19:58:23 +02:00
Dmitry Novik
16c6b60703 Introduce AggregationKeysInfo 2022-05-25 23:22:29 +00:00
Dmitry Novik
b3ccf96c81 Merge remote-tracking branch 'origin/master' into grouping-function 2022-05-19 17:58:33 +00:00
Dmitry Novik
e5b395e054 Support ROLLUP and CUBE in GROUPING function 2022-05-16 17:33:38 +00:00
Dmitry Novik
6fc7dfea80 Support ordinary GROUP BY 2022-05-13 23:04:12 +00:00
Azat Khuzhin
b29a0f3c84 Fix "Cannot create column of type Set" for distributed queries with LIMIT BY
The problem was that after #36454 it requires too much columns.

Fixes: #36454
Fixes: #37045
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-05-13 20:03:19 +03:00
Azat Khuzhin
751a85521e Fill possible WITH FILL error in SelectQueryExpressionAnalyzer::appendOrderBy()
If order_by_keys was filled by the code above then WITH FILL will
incorrectly handle constant columns.

Fixes: #16618
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-05-13 20:03:19 +03:00
Dmitry Novik
ae81268d4d Try to compute helper column lazy 2022-05-13 14:55:50 +00:00
Dmitry Novik
c5b40a9c91 WIP on GROUPING function 2022-05-12 16:40:26 +00:00
Vladimir C
4cb6ca2311
Merge pull request #36970 from ClickHouse/concurrent_join
Follow up for ConcurrentHashJoin
2022-05-12 15:09:29 +02:00
Maksim Kita
b1c154ccb4 Fixed style check 2022-05-11 21:59:51 +02:00
Maksim Kita
4e7d10297b Fixed style 2022-05-11 21:59:51 +02:00
Nikolai Kochetov
2d99f0ce13 Simplify code a little bit. 2022-05-11 12:16:15 +00:00
Nikolai Kochetov
a02e1d2f4a Simplify code a little bit. 2022-05-10 16:00:00 +00:00
mergify[bot]
55a6d22ad3
Merge branch 'master' into grouping-sets-fix 2022-05-09 14:02:10 +00:00
vdimir
d712985575
Minor style changes for ConcurrentHashJoin 2022-05-06 15:17:46 +00:00
Dmitry Novik
9a251e0028 Cleanup code 2022-05-05 18:13:00 +00:00
Dmitry Novik
4cc26aa38b Merge remote-tracking branch 'origin/master' into grouping-sets-fix
And fix execution of the query with only one grouping set
2022-05-05 17:14:52 +00:00
Dmitry Novik
161f52292b Support distributed queries 2022-05-05 13:56:16 +00:00
lgbo-ustc
595ba6bd9f udpate dispatchBlock() 2022-05-05 12:04:11 +08:00
lgbo-ustc
29a8db1345 fixed fast test 2022-05-05 10:32:19 +08:00
lgbo-ustc
981d560553 Merge remote-tracking branch 'ck/master' into concurrent_join 2022-04-25 13:00:04 +08:00
Dmitry Novik
ea6a25b8e7 Fix nullptr dereference 2022-04-21 11:14:22 +00:00
lgbo-ustc
bfd1a0e33a update codes 2022-04-21 12:14:36 +08:00
Dmitry Novik
61deae7105
Merge branch 'master' into grouping-sets-fix 2022-04-21 03:34:42 +02:00
Dmitry Novik
6e73cd6929 Implement parallel grouping sets processing 2022-04-21 01:18:40 +00:00
Azat Khuzhin
c6ffec1fec Fix "Cannot find column" error for distributed queries with LIMIT BY
Consider the following example:

    SELECT k FROM (SELECT materialize('foo') AS k, -1 AS v) ORDER BY abs(v) AS _v ASC LIMIT 1 BY k

The problem here is that in case of query has LIMIT BY for
WithMergeableState* (which is the final state on the remote shard for
distributd queries) it returns the following columns:
- k
- v

While it should return:
- k
- abs(v)

So as query w/o LIMIT BY, so that initiator will be able to do the sorting

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-04-20 15:23:24 +03:00
lgbo-ustc
f461d18de5 update method name 2022-04-20 19:51:35 +08:00
Antonio Andelic
bbb0be6a44
Merge pull request #36402 from rschu1ze/clang-tidy-contains
Activate clang-tidy warning "readability-container-contains"
2022-04-20 08:15:46 +02:00
Kruglov Pavel
894b1b163e
Merge pull request #36055 from helifu/master2
Don't materialize external tables if it's explain statement
2022-04-19 20:26:27 +02:00
lgbo-ustc
3d7338581b Improve join
now adding joined blocks from right table can be run parallelly, speedup the join process
2022-04-19 16:07:30 +08:00
Robert Schulze
118e94523c
Activate clang-tidy warning "readability-container-contains"
This check suggests replacing <Container>.count() by
<Container>.contains() which is more speaking and in case of
multimaps/multisets also faster.
2022-04-18 23:53:11 +02:00
Alexey Milovidov
242919eddd Remove abbreviation 2022-04-18 01:02:49 +02:00
Dmitry Novik
a16710c750 Merge remote-tracking branch 'origin/master' into grouping-sets-fix 2022-04-14 17:29:51 +00:00
mergify[bot]
ba22cb76bb
Merge branch 'master' into master2 2022-04-13 11:51:43 +00:00
Yakov Olkhovskiy
2588f8011f comment fix 2022-04-11 11:13:29 -04:00
Yakov Olkhovskiy
7293e01647 some comments added 2022-04-11 09:47:27 -04:00
helifu
1196dc3dbd Don't materialize external tables if it's explain statement
The explain statement of GLOBAL JOIN two distributed tables can speed up 100x:
  explain plan select ... from t1_dist global join t2_dist on ...
  explain pipeline select ... from t1_dist global join t2_dist on ...
2022-04-08 09:00:55 +08:00
Yakov Olkhovskiy
7dbe8bc2dc major bugs fixed, tests added, docs updated 2022-04-07 01:21:24 -04:00
Yakov Olkhovskiy
ac441b9dcf compiler suggestions 2022-04-06 00:05:06 -04:00
Yakov Olkhovskiy
90c4cd3de7
Merge branch 'master' into interpolate-feature 2022-04-05 14:39:07 -04:00
Yakov Olkhovskiy
e0d6033c39 all columns can participate in interpolate expression despite if they are selected or not, some optimization on expressionless INTERPOLATE 2022-04-05 14:26:49 -04:00
kssenii
d8960cde62 Fix positional order/etc by literal 2022-03-25 20:03:12 +01:00
kssenii
5103aefd34 Fix 2022-03-25 14:33:54 +01:00
Amos Bird
243de091bb
Validate some thoughts over making sets 2022-03-21 10:58:44 +08:00
Maksim Kita
b1a956c5f1 clang-tidy check performance-move-const-arg fix 2022-03-02 18:15:27 +00:00
Dmitry Novik
67df01d025
Merge branch 'master' into grouping-sets-fix 2022-02-22 04:18:03 -08:00
vdimir
99ca89c0ca
Fix StorageJoin and Asof or join_use_nulls in pipeline 2022-02-14 14:14:27 +00:00
vdimir
ee09ec4dd1
Apply join_use_nulls on types before join 2022-02-11 15:56:45 +00:00
Amos Bird
7674bc986e
Disable projection when there is JOIN or SAMPLE 2022-02-06 16:46:09 +08:00
Dmitry Novik
9ae9048184 Fix crash in GROUPING SETS 2022-01-14 13:18:05 +00:00
Dmitry Novik
b27af90202 Fix removing constant columns 2022-01-11 11:26:13 +00:00
Dmitry Novik
1095814bbf Cleanup code 2022-01-11 11:26:13 +00:00
fanzhou
6c6d3015de 1. refactor ExpressionAnalyzer
2. Update formatAST to support grouping sets query with distributed table
3. modify astExpression to support function in grouping sets
2022-01-11 11:26:12 +00:00
fanzhou
83e9e5d0e5 some changes 2022-01-11 11:26:12 +00:00
fanzhou
912338629e grouping sets support multiple group 2022-01-11 11:26:11 +00:00
MaxTheHuman
c7c2093bd9 grouping sets: make simple aggregation with grouping sets to work 2022-01-11 11:26:11 +00:00
MaxTheHuman
e22cbb7cd8 grouping sets dev: fix errors, something works 2022-01-11 11:26:11 +00:00
MaxTheHuman
e6bd807f60 grouping sets development 2022-01-11 11:26:10 +00:00
MaxTheHuman
390c8126a4 development 2022-01-11 11:26:10 +00:00
MaxTheHuman
3195d600c5 feat grouping-sets: initial changes 2022-01-11 11:26:10 +00:00
alexey-milovidov
0a55fa3dc2
Revert "Grouping sets dev" 2021-12-25 20:30:31 +03:00
Kseniia Sumarokova
5ae2f0028f
Merge pull request #32961 from kssenii/fix-positional-args
Fix for positional args
2021-12-22 09:41:55 +03:00
kssenii
de70a57700 Better 2021-12-21 21:48:25 +03:00
kssenii
175ad8a989 Fix 2021-12-21 00:20:08 +03:00
Maksim Kita
51477adf1b Updated additional cases 2021-12-20 15:55:07 +03:00
kssenii
965e2bfb21 Fix 2021-12-20 08:43:07 +03:00
alexey-milovidov
6b97af4c63
Merge pull request #26869 from taylor12805/grouping-sets-dev
Grouping sets dev
2021-12-17 20:50:15 +03:00
Dmitry Novik
0f3e2bb32f Fix removing constant columns 2021-12-16 15:39:25 +03:00
Dmitry Novik
56a3f4a000 Cleanup code 2021-12-14 22:15:14 +03:00
fanzhou
b94f8878ff 1. refactor ExpressionAnalyzer
2. Update formatAST to support grouping sets query with distributed table
3. modify astExpression to support function in grouping sets
2021-11-30 14:58:55 +03:00
fanzhou
43db4594ba some changes 2021-11-29 19:35:33 +03:00
fanzhou
2912344896 grouping sets support multiple group 2021-11-29 18:32:55 +03:00
Kseniia Sumarokova
8f0fc225b1
Update ExpressionAnalyzer.cpp 2021-11-29 18:07:11 +03:00
MaxTheHuman
4f07ec2659 grouping sets: make simple aggregation with grouping sets to work 2021-11-26 22:59:56 +03:00
MaxTheHuman
e32d58a36d grouping sets dev: fix errors, something works 2021-11-26 22:46:15 +03:00
MaxTheHuman
ddd1799743 grouping sets development 2021-11-26 22:11:34 +03:00
MaxTheHuman
3418de337a development 2021-11-26 20:37:51 +03:00
MaxTheHuman
2bd07ef338 feat grouping-sets: initial changes 2021-11-26 20:24:35 +03:00
kssenii
038bdcf9b9 Fix 2021-11-24 23:56:36 +03:00
Azat Khuzhin
fbf98bea0b Fix "Column is not under aggregate function and not in GROUP BY" with PREWHERE
In 21.7 the function column type is unknown:

  (lldb) p tmp_actions_dag->dumpDAG()
  (std::string) $14 = "0 : INPUT () (no column) Int32 key\n1 : COLUMN () Const(UInt8) UInt8 1\n2 : FUNCTION (0) (no column) UInt8 ignore(key) [ignore]\n3 : FUNCTION (1, 2) (no column) UInt8 or(1, ignore(key)) [or]\nIndex: 0 1 2 3\n"
                                                                                                                                                                            ^^^^^^^^^^^

While in 21.8+ it is const:

  (lldb) p tmp_actions_dag->dumpDAG()
  (std::string) $22 = "0 : INPUT () (no column) Int32 key\n1 : COLUMN () Const(UInt8) UInt8 1\n2 : FUNCTION (0) (no column) UInt8 ignore(key) [ignore]\n3 : FUNCTION (1, 2) Const(UInt8) UInt8 or(1, ignore(key)) [or]\nIndex: 0 1 2 3\n"
                                                                                                                                                                            ^^^^^

This is after getRootActions() in SelectQueryExpressionAnalyzer::appendPrewhere()

v1: fix for SELECT only, but breaks virtual columns
v2: hacky fix, that also touches MergeTree code
v3: allow_remove_inputs=false
v4: allow_constant_folding=false instead
2021-10-26 22:57:48 +03:00
Anton Popov
76cb191865
Merge pull request #30433 from kssenii/fix-positional
More full support of positional arguments
2021-10-24 21:48:17 +03:00
Kseniia Sumarokova
05150f151c
Update ExpressionAnalyzer.cpp 2021-10-23 12:19:40 +03:00
kssenii
a1a2e276ae Review fixes 2021-10-22 12:26:28 +00:00
kssenii
bc4c2f2366 Fix clang-tidy 2021-10-20 23:40:21 +00:00
Amos Bird
240895fba7
Fix projections with JOIN alias columns 2021-10-20 21:30:15 +08:00
kssenii
6c990400d1 Better positional args 2021-10-20 07:52:39 +00:00
Nikolai Kochetov
c668696047
Merge pull request #30171 from ClickHouse/remove-stream-interfaces
Remove stream interfaces
2021-10-16 09:34:01 +03:00
Nikolai Kochetov
ab28c6c855 Remove BlockInputStream interfaces. 2021-10-14 13:25:43 +03:00
Raúl Marín
0ee5c0bff5 Use RWLock in StorageJoin to avoid deadlocks 2021-09-29 19:30:07 +02:00
Kruglov Pavel
14be2f31f5
Merge pull request #29475 from ClickHouse/fix-29010
Remove filter column from HAVING when it is not needed.
2021-09-29 13:03:27 +03:00
Nikolai Kochetov
1a4ee1f77f Remove filter column from HAVING when it is not needed. 2021-09-28 13:01:47 +03:00
Nikolai Kochetov
0e267c50b4 Merge branch 'master' into rewrite-pushing-to-views 2021-09-14 16:13:54 +03:00
alexey-milovidov
f066edc43c
Merge pull request #28910 from ClickHouse/fix-trash-optimization
Fix bad optimization of ORDER BY in subquery if it contains WITH FILL
2021-09-12 15:27:53 +03:00
alexey-milovidov
ea13a8b562
Merge pull request #28659 from myrrc/improvement/tostring_to_magic_enum
Improving CH type system with concepts
2021-09-12 15:26:29 +03:00
Alexey Milovidov
8935a70f60 Fix another misoptimization 2021-09-12 01:41:37 +03:00
Nikolai Kochetov
f569a3e3f7 Merge branch 'master' into rewrite-pushing-to-views 2021-09-09 20:30:23 +03:00
Nikolai Kochetov
999a4fe831 Fix other tests. 2021-09-08 21:29:38 +03:00
Mike Kot
8e9aacadd1 Initial: replacing hardcoded toString for enums with magic_enum 2021-09-06 16:24:03 +02:00
vdimir
6172cc1676
Fix build, inferJoinKeyCommonType definition 2021-09-01 15:08:58 +03:00
vdimir
df873866c9
Don't store converting actions in TableJoin 2021-09-01 15:05:48 +03:00
vdimir
ed8c156190
Clean up TableJoin storage join 2021-09-01 15:05:47 +03:00
vdimir
c4ffc2879f
Perform join type inference for USING at ExpressionAnalyzer instead of TreeRewriter 2021-09-01 15:05:47 +03:00
vdimir
33ebdb283c
Split ExpressionAnalyzer::analyzeAggregation 2021-09-01 15:05:47 +03:00