Nikolai Kochetov
e616732743
Small refactoring.
2021-09-15 22:35:48 +03:00
vdimir
e8e26463bf
Change signature of ASTSelectQuery::arrayJoinExpressionList
2021-08-31 15:27:44 +03:00
vdimir
3443b3b5c4
Separate functoin buildJoinedPlan, minor refactoring
2021-08-31 15:27:44 +03:00
Kseniia Sumarokova
19afb22b4d
Merge pull request #27932 from amosbird/constkeys
...
Add a setting to bring back the old behavior of grouping by constant keys on empty set.
2021-08-21 08:02:30 +03:00
Amos Bird
5708ed42ec
Add setting.
...
empty_result_for_aggregation_by_constant_keys_on_empty_set
2021-08-20 22:34:32 +08:00
kssenii
8da7f406af
Better comment
2021-08-17 11:03:01 +03:00
Kseniia Sumarokova
6001c7f214
Update InterpreterSelectQuery.cpp
2021-08-17 08:44:51 +03:00
kssenii
1ad4b6ed60
Fix
2021-08-16 23:34:39 +03:00
Raúl Marín
d1d47658f5
Merge remote-tracking branch 'blessed/master' into materialization_log
2021-08-10 10:49:05 +02:00
Azat Khuzhin
3be3c503aa
Fix some comments
2021-08-08 09:58:07 +03:00
alexey-milovidov
c5207fc237
Merge pull request #26466 from azat/optimize-dist-select
...
Rework SELECT from Distributed optimizations
2021-08-08 03:59:32 +03:00
mergify[bot]
589505e76b
Merge branch 'master' into legacy-tuple-name
2021-08-07 16:42:20 +00:00
Azat Khuzhin
1358a2db92
Fix WithMergeableStateAfterAggregationAndLimit with LIMIT BY and LIMIT OFFSET
...
In case of LIMIT BY, pre LIMIT cannot be applied (that is done on the
shard before), and hence shard applies regular LIMIT, but it goes with
applying OFFSET, and also the initiator will do apply OFFSET too.
2021-08-06 00:16:26 +03:00
Azat Khuzhin
c4e2342a06
Add verbose description for pre-LIMIT into EXPLAIN query
2021-08-06 00:16:26 +03:00
Raúl Marín
479f053f2e
Merge remote-tracking branch 'blessed/master' into materialization_log
2021-08-05 17:24:10 +02:00
Alexander Kuzmenkov
d852207b0e
Merge pull request #25721 from CurtizJ/read-in-order-perf-2
...
Reduce memory usage in queries with `ORDER BY primary_key
2021-08-05 16:13:48 +03:00
Anton Popov
915ef50c2d
optimize reading in order of key more
2021-08-04 18:18:03 +03:00
Anton Popov
699a3d9031
implement legacy_column_name_of_tuple_literal in less intrusive way
2021-08-03 21:03:24 +03:00
Azat Khuzhin
ff12f5102a
Avoid running LIMIT BY/DISTINCT step on the initiator for optimize_distributed_group_by_sharding_key
...
Before the following queries was running LimitBy/Distinct step on the
initator:
select distinct sharding_key from dist order by k
While this can be omitted.
2021-08-02 21:04:30 +03:00
Raúl Marín
9f32ecca89
Merge remote-tracking branch 'blessed/master' into materialization_log
2021-08-02 16:21:10 +02:00
Anton Popov
cc3ed12ed6
Merge remote-tracking branch 'upstream/master' into HEAD
2021-08-02 02:56:00 +03:00
Amos Bird
7b9e81742f
Shard level constness
2021-07-30 18:53:34 +08:00
Raúl Marín
f0125262b7
Add views and it's tables to system.query_log
2021-07-27 16:29:33 +02:00
Alexander Tokmakov
63ab38ee09
rename MaterializeMySQL to MaterializedMySQL
2021-07-26 21:17:28 +03:00
mergify[bot]
39f71f4a08
Merge branch 'master' into fix_no_column_materialized_mysql
2021-07-19 12:27:38 +00:00
Anton Popov
f61ea15d56
fix pushdown of limit to reading stage
2021-07-16 20:40:00 +03:00
Anton Popov
03c785931a
fix pushdown of limit to reading stage
2021-07-16 17:27:38 +03:00
Anton Popov
c45a9e3bd6
Merge remote-tracking branch 'upstream/master' into HEAD
2021-07-16 17:11:34 +03:00
Raúl Marín
c2f0d7c514
Don't build set for indices when analyzing a query
2021-07-15 14:25:22 +02:00
Amos Bird
0558ecdc3f
Aggressive IN index analysis for projections.
2021-07-14 22:56:52 +08:00
Anton Popov
cb042afa9d
pushdown limit while reading in order of primary key
2021-07-13 17:24:45 +03:00
mergify[bot]
d5898ecba6
Merge branch 'master' into fix_no_column_materialized_mysql
2021-07-13 14:03:05 +00:00
alexey-milovidov
4b1e38eed9
Merge branch 'master' into projection-fix11
2021-07-13 04:52:15 +03:00
Alexey Milovidov
e0ae04297a
Fix fast test
2021-07-12 18:19:49 +03:00
vdimir
d4256a8583
Minor style changes in JoinedTables
2021-07-08 13:49:13 +03:00
vdimir
5d7fc61bd5
Merge branch 'master' into join-materialized-columns
2021-07-08 12:30:00 +03:00
mergify[bot]
446474932b
Merge branch 'master' into projection-fix11
2021-07-07 05:02:44 +00:00
Amos Bird
55981cb0ae
Fix normal projection
2021-07-07 13:01:30 +08:00
alexey-milovidov
b724138541
Merge pull request #25786 from kssenii/inconsisteny
...
Fix inconsistent behaviour of GROUP BY constant on empty set
2021-07-06 03:36:50 +03:00
Alexander Tokmakov
0f377fc4d8
Merge branch 'master' into fix_no_column_materialized_mysql
2021-07-05 22:07:56 +03:00
alexey-milovidov
fef7c557f6
Merge pull request #24789 from kitaisreal/compile-aggregation-functions
...
Compile aggregate functions
2021-07-05 02:57:49 +03:00
kssenii
77879f6826
Better
2021-07-04 01:53:51 +03:00
kssenii
a9a91a4f15
Setting move to prewhere if final
2021-07-03 11:37:50 +00:00
kssenii
02681019f8
Fix
2021-07-03 08:48:42 +00:00
Maksim Kita
da8c957167
Aggregator added CompiledExpressionCache
2021-07-01 22:56:36 +03:00
Maksim Kita
d24d3ae992
Added second variant of compilation
2021-07-01 22:56:36 +03:00
Maksim Kita
3fe559b31f
Compile aggregate functions
2021-07-01 22:56:35 +03:00
Alexander Tokmakov
fd914f3d97
fix 'Not found column' error in MaterializeMySQL
2021-06-29 20:47:41 +03:00
Nikolai Kochetov
3544b059ce
Merge branch 'master' into remove-PrewhereDAGInfo
2021-06-25 17:52:03 +03:00
Nikolai Kochetov
6bc0a628cd
Remove PrewhereDAGInfo.
2021-06-25 17:49:28 +03:00
vdimir
2949cd1e6f
Support ALIASed columns in JOIN ON expression
2021-06-25 14:50:02 +03:00
vdimir
241b64d02c
Support ALIASed columns for right joined table
2021-06-24 17:57:21 +03:00
vdimir
84e02911cf
Materialized columns for joined table, don't add to asterisk select
2021-06-24 17:14:36 +03:00
Anton Popov
c2754c34f4
Merge remote-tracking branch 'upstream/master' into HEAD
2021-06-21 12:42:48 +03:00
Anton Popov
d8b6f15ef4
Merge pull request #23027 from azat/distributed-push-down-limit
...
Add ability to push down LIMIT for distributed queries
2021-06-20 23:08:50 +03:00
Nikolai Kochetov
b2620e1def
Merge pull request #24405 from amosbird/replacemorealias
...
Rewrite more alias columns
2021-06-18 09:10:31 +03:00
Amos Bird
0f7728cd5e
Fix RBAC test again
2021-06-17 23:00:03 +08:00
Maksim Kita
67e9b85951
Merge ext into common
2021-06-16 23:28:41 +03:00
Anton Popov
ea9b890618
Merge branch 'master' into add-compatibility-setting
2021-06-16 21:10:56 +03:00
Anton Popov
8bd7cbd925
add compatibility setting for name of columns
2021-06-16 21:10:07 +03:00
mergify[bot]
7959d92029
Merge branch 'master' into minor-changes-3
2021-06-15 18:07:24 +00:00
Nikita Mikhaylov
a52bba91b7
Merge pull request #16401 from abyss7/ast-table-identifier-2
...
ASTTableIdentifier part #2 : Introduce ASTTableIdentifier
2021-06-15 13:51:30 +03:00
Amos Bird
aaac231c6a
Record expanded aliases for RBAC
2021-06-15 18:28:13 +08:00
Amos Bird
e3ae2f6e7a
Rewrite more alias columns
2021-06-15 17:54:28 +08:00
Alexey Milovidov
447d7bb8cd
Minor changes
2021-06-14 07:13:35 +03:00
Azat Khuzhin
18e8f0eb5e
Add ability to push down LIMIT for distributed queries
...
This way the remote nodes will not need to send all the rows, so this
will decrease network io and also this will make queries w/
optimize_aggregation_in_order=1/LIMIT X and w/o ORDER BY faster since it
initiator will not need to read all the rows, only first X (but note
that for this you need to your data to be sharded correctly or you may
get inaccurate results).
Note, that having lots of processing stages will increase the complexity
of interpreter (it is already not that clean and simple right now).
Although using separate QueryProcessingStage looks pretty natural.
Another option is to make WithMergeableStateAfterAggregation always, but
in this case you will not be able to disable only this optimization,
i.e. if there will be some issue with it.
v2: fix OFFSET
v3: convert 01814_distributed_push_down_limit test to .sh and add retries
v4: add test with OFFSET
v5: add new query stage into the bash completion
v6/tests: use LIMIT O,L syntax over LIMIT L OFFSET O since it is broken in ANTLR parser
https://clickhouse-test-reports.s3.yandex.net/23027/a18a06399b7aeacba7c50b5d1e981ada5df19745/functional_stateless_tests_(antlr_debug).html#fail1
v7/tests: set use_hedged_requests to 0, to avoid excessive log entries on retries
https://clickhouse-test-reports.s3.yandex.net/23027/a18a06399b7aeacba7c50b5d1e981ada5df19745/functional_stateless_tests_flaky_check_(address).html#fail1
2021-06-09 02:29:50 +03:00
Ivan
579250168f
Merge branch 'master' into ast-table-identifier-2
2021-06-07 20:30:39 +03:00
Alexey Milovidov
885ce194e0
Making fundamentals correct
2021-06-07 00:49:55 +03:00
Amos Bird
9dfa4c06bd
Try reusing built sets during multi-pass analysis
2021-06-03 22:06:05 +08:00
Ivan Lezhankin
ae38b4a3f1
Merge remote-tracking branch 'upstream/master' into ast-table-identifier-2
2021-06-02 16:07:59 +03:00
Alexander Kuzmenkov
6cccb0992a
fix ORDER BY after window fuctions over Distributed
2021-05-27 19:06:37 +03:00
mergify[bot]
970243850f
Merge branch 'master' into ast-table-identifier-2
2021-05-21 12:56:49 +00:00
Maksim Kita
b0476c1fa2
Merge pull request #24273 from kitaisreal/expression-actions-compile-only-necessary-places
...
ExpressionActions compile only necessary places
2021-05-20 10:53:30 +03:00
Maksim Kita
c50909468f
Updated ExpressionActionSettings
2021-05-19 17:32:07 +03:00
Alexander Kuzmenkov
e9b69bbd70
Merge pull request #23906 from azat/fix-distributed_group_by_no_merge
...
distributed_group_by_no_merge fixes
2021-05-19 16:16:08 +03:00
Maksim Kita
150a88d647
ExpressionActions compile only necessary places
2021-05-19 11:43:16 +03:00
Ivan Lezhankin
bd16dd22b9
Merge remote-tracking branch 'upstream/master' into ast-table-identifier-2
2021-05-14 15:40:33 +03:00
Amos Bird
5836e2e277
Fix issues in normal projection
2021-05-11 18:12:29 +08:00
Amos Bird
dfa5629071
Remove projection type ast
2021-05-11 18:12:29 +08:00
Amos Bird
51d685bb9c
Even more fixes
2021-05-11 18:12:29 +08:00
Amos Bird
718c284437
Fix more tests
2021-05-11 18:12:28 +08:00
Amos Bird
ba17acbd63
Fix tests
2021-05-11 18:12:28 +08:00
Amos Bird
42f161b1dc
Fix more
2021-05-11 18:12:28 +08:00
Amos Bird
9c069ebdbf
support prewhere, row_filter, read_in_order and decent projection selection
...
TODO set index analysis in projection
2021-05-11 18:12:27 +08:00
Amos Bird
35961c0c5d
Use ActionsDAG to rewrite projection queries
2021-05-11 18:12:27 +08:00
Nikolai Kochetov
29d7038e30
Add ProjectionDescription::Type enum. Remove some checks.
2021-05-11 18:12:27 +08:00
Amos Bird
ebaf42a448
Reformat and fix some tests
2021-05-11 18:12:27 +08:00
Nikolai Kochetov
c2396c5144
Disable read-in-order optimization in case of projections.
2021-05-11 18:12:26 +08:00
Nikolai Kochetov
3d36fb57eb
Cleanup code.
2021-05-11 18:12:26 +08:00
Nikolai Kochetov
3296c9292f
Try to merge projectons faster.
2021-05-11 18:12:26 +08:00
Amos Bird
cd6414639e
add metadata_snapshot to getQueryProcessingStage
2021-05-11 18:12:26 +08:00
Amos Bird
264cff6415
Projections
...
TODO (suggested by Nikolai)
1. Build query plan fro current query (inside storage::read) up to WithMergableState
2. Check, that plan is simple enough: Aggregating - Expression - Filter - ReadFromStorage (or simplier)
3. Check, that filter is the same as filter in projection, and also expression calculates the same aggregation keys as in projection
4. Return WithMergableState if projection applies
3 will be easier to do with ActionsDAG, cause it sees all functions, and dependencies are direct (but it is possible with ExpressionActions also)
Also need to figure out how prewhere works for projections, and
row_filter_policies.
wip
2021-05-11 18:12:23 +08:00
Azat Khuzhin
eefd67fce5
Disable optimize_distributed_group_by_sharding_key with window functions
2021-05-06 00:44:22 +03:00
Azat Khuzhin
8703b7863d
Fix distributed_group_by_no_merge=2 with GROUP BY
...
Before this patch the following query:
SELECT assumeNotNull(argMax(dummy, 1))
FROM remote('127.1', system.one)
SETTINGS distributed_group_by_no_merge = 2
Leads to:
Code: 10. DB::Exception: Received from localhost:9000. DB::Exception: Not found column argMax(dummy, 1) in block: while executing 'INPUT : 0 -> argMax(dummy, 1) UInt8 : 0'.
Since it tries to execute function one more time, but shards will not
send this column when the query processed with
distributed_group_by_no_merge=2 (i.e. up to
WithMergeableStateAfterAggregation).
v0: no exception
v2: execut window functions
v3: throw exception, since executing window function in this case will
lead to messy output
2021-05-06 00:21:47 +03:00
Nikolai Kochetov
25fe5f63ee
Fix other tests.
2021-04-29 12:08:49 +03:00
Nikolai Kochetov
69ddca1d73
Refactor join step, part 3
2021-04-28 20:32:12 +03:00
Alexander Kuzmenkov
ab75f86ee9
add a comment
2021-04-23 18:45:45 +03:00
Ivan Lezhankin
cdd825b18b
Merge remote-tracking branch 'upstream/master' into ast-table-identifier-2
2021-04-23 18:08:25 +03:00
Alexander Kuzmenkov
6982ab5324
fix extra distinct
2021-04-23 15:44:46 +03:00
Alexander Kuzmenkov
9d95ac3d66
remove debug
2021-04-23 14:00:17 +03:00
