Commit Graph

919 Commits

Author SHA1 Message Date
Antonio Andelic
44ef00dc05 Merge branch 'master' into custom-key-parallel-replicas 2023-01-22 14:20:58 +00:00
Nikolai Kochetov
3e00d18498 Merge branch 'master' into fix-disabled-two-level-agg 2023-01-19 20:54:04 +00:00
Nikolai Kochetov
d24be2712e Fix disabled two-level aggregation from HTTP 2023-01-19 20:50:27 +00:00
Antonio Andelic
ddfb913f99 better 2023-01-19 11:28:26 +00:00
Antonio Andelic
3b0c63551e Combine approaches 2023-01-19 10:26:38 +00:00
Antonio Andelic
2a1f35e661 Revert some changes 2023-01-19 07:59:57 +00:00
vdimir
b76779797a
Do not move to prewhere in select with joins 2023-01-18 12:17:30 +00:00
Antonio Andelic
dd31de18a3 Extend where correctly 2023-01-17 13:31:09 +00:00
Antonio Andelic
bd352068d7 Turn replicas into shard for custom_key 2023-01-17 12:34:42 +00:00
Antonio Andelic
57e771226c Add range mode for custom key 2023-01-16 10:11:21 +00:00
Antonio Andelic
e64f07e72b Merge branch 'master' into custom-key-parallel-replicas 2023-01-13 08:44:53 +00:00
Arthur Passos
5727d14e92
Merge branch 'master' into add_select_final_mt_setting 2023-01-12 14:42:55 -03:00
Kseniia Sumarokova
db3e0219fc
Merge pull request #41687 from ClickHouse/40907_Parameterized_views_as_table_functions
40907 Parameterized views as table functions
2023-01-12 14:24:32 +01:00
Arthur Passos
1b3c7766f8
Merge branch 'master' into add_select_final_mt_setting 2023-01-12 08:31:04 -03:00
Antonio Andelic
f950b75ef0 Fix 2023-01-11 10:42:01 +00:00
Antonio Andelic
19e5988d33 Better 2023-01-11 09:57:13 +00:00
Antonio Andelic
fbe4808b6b Add support for custom key in parallel replicas 2023-01-10 13:13:43 +00:00
Alexey Milovidov
3963eeae16
Merge pull request #45058 from nickitat/fix_grouping_sets_with_memory_efficient_aggr
Disable memory efficient merging for distributed aggregation when grouping sets are present
2023-01-10 05:08:55 +03:00
Nikita Taranov
82963e4227 impl 2023-01-09 12:12:55 +00:00
Nikolai Kochetov
533911b28b Respect requested columns order for select. 2023-01-04 19:59:15 +00:00
Smita Kulkarni
a6f860f24e Fixed review comments and updated FunctionParameterValuesVisitor to use visitFunction - 40907 Parameterized views as table functions 2022-12-23 17:57:17 +01:00
Smita Kulkarni
5001cf9fa2 Updated to get column data type based not function , not the query parameter type , added test for the same - 40907 Parameterized views as table functions 2022-12-19 15:05:38 +01:00
Smita Kulkarni
22c2956a06 Updated comment on fetching parameter values from query and fixed style comment- 40907 Parameterized views as table functions 2022-12-16 12:19:11 +01:00
Smita Kulkarni
dd8df3347b Addressed style review comments and removed bool variables assignment from Clone of ASTs - 40907 Parameterized views as table functions 2022-12-16 11:11:19 +01:00
Smita Kulkarni
16d8fd02fb Merge branch 'master' into 40907_Parameterized_views_as_table_functions 2022-12-16 08:52:48 +01:00
Arthur Passos
7f8a4f8d87 do not apply final on distributed engine, only on underlying 2022-12-06 17:28:30 -03:00
Arthur Passos
3239516521 apply final only on storages that support it 2022-12-05 10:18:44 -03:00
Nikolai Kochetov
43b3c10e9a Add a separate optimisation to enable memory bound aggregation. 2022-11-30 14:31:39 +00:00
Arthur Passos
252575ee23 doc 2022-11-29 13:14:52 -03:00
Arthur Passos
eb3f4c9fe2 prevent force select final from affecting non table based queries 2022-11-29 13:14:06 -03:00
Smita Kulkarni
953e1068b5 Merge branch 'master' into 40907_Parameterized_views_as_table_functions 2022-11-28 19:10:32 +01:00
Smita Kulkarni
0545e24151 Updated to support query parameters in different parts of query (SELECT, ORDERBY, GROUP BY), updated the approach in WHERE & HAVING, added tests for the same - 40907 Parameterized views as table functions 2022-11-28 19:05:01 +01:00
Nikolai Kochetov
6d0646ed8f
Merge branch 'master' into aggregating-in-order-from-query-plan 2022-11-28 16:53:29 +01:00
Nikita Taranov
8ed5cfc265
Memory bound merging for distributed aggregation in order (#40879)
* impl

* fix style

* make executeQueryWithParallelReplicas similar to executeQuery

* impl for parallel replicas

* cleaner code for remote sorting properties

* update test

* fix

* handle when nodes of old versions participate

* small fixes

* temporary enable for testing

* fix after merge

* Revert "temporary enable for testing"

This reverts commit cce7f8884c.

* review fixes

* add bc test

* Update src/Core/Settings.h
2022-11-28 00:41:31 +01:00
Arthur Passos
6ee21e63d7 Move force select final to user lvel setting and remove ignore force select final 2022-11-25 13:31:08 -03:00
Nikolai Kochetov
62debf42b4 Aggregating in order [in progress]. 2022-11-23 18:38:12 +00:00
Igor Nikonov
7f8d07e4b7 Fix: limit hint for DISTINCT ignores LIMIT BY
+ test
2022-11-20 23:06:57 +00:00
Alexey Milovidov
915485ff88
Merge branch 'master' into add_select_final_mt_setting 2022-11-20 08:51:12 +01:00
Smita Kulkarni
6d35bc06c5 Fixed style check fail in InterpreterSelectQuery.cpp - 40907 Parameterized views as table functions 2022-11-16 13:01:02 +01:00
Smita Kulkarni
95abe5d027 Updated to support query parameters in HAVING clause and addressed review comments - 40907 Parameterized views as table functions 2022-11-16 12:27:08 +01:00
Smita Kulkarni
4dec8337f0 Merge branch 'master' into 40907_Parameterized_views_as_table_functions 2022-11-11 19:04:35 +01:00
Nikolai Kochetov
63d06c8b53
Merge pull request #42829 from ClickHouse/read-in-order-from-query-plan
Read-in-order over query plan
2022-11-11 12:15:20 +01:00
Nikolai Kochetov
997881c7f7 Review fixes. 2022-11-09 16:07:38 +00:00
Smita Kulkarni
779b9197e3 Moved is_create_parameterized_view check inside finalize function and refactor check for isParmaeterizedView when replacing subquery - 40907 Parameterized views as table functions 2022-11-08 11:03:40 +01:00
Nikolai Kochetov
9ffebf4a0c Fixing read-in-order for special storages. 2022-11-01 19:51:52 +00:00
Smita Kulkarni
9aa06e48f6 Merge branch 'master' into 40907_Parameterized_views_as_table_functions 2022-10-31 15:46:52 +01:00
Vladimir Chebotaryov
d17b7387f9 Reworked changes to std::shared_ptr<const RowPolicyFilter>. 2022-10-27 10:42:55 +03:00
Vladimir Chebotaryov
6d5d31e49c Removed UUID's. 2022-10-27 10:42:26 +03:00
Vladimir Chebotaryov
0707d88311 Removed extendQueryLogElemImplImpl. 2022-10-27 10:42:25 +03:00
Vladimir Chebotaryov
c28e439c33 Review fixes. 2022-10-27 10:42:22 +03:00
Vladimir Chebotaryov
4cabe1f57c Added applied row-level policies to system.query_log. 2022-10-27 10:41:47 +03:00
Kruglov Pavel
219553df8d
Merge pull request #42580 from hanfei1991/fei/limit_max_rows
fix behaviour of max_rows_to_read for trival limit queries
2022-10-26 14:09:59 +02:00
Han Fei
2fc91fd338 fix behaviour of max_rows_to_read for trival limit queries 2022-10-22 18:27:24 +02:00
Arthur Passos
0f6b877073 Add ignore_force_select_final setting 2022-10-21 16:27:59 -03:00
Vitaly Baranov
2033ea1216 Support subqueries in row policy filters. 2022-10-21 15:42:29 +02:00
Smita Kulkarni
d8fe1d2d63 Moved is_create_parameterized_view out of Context , and added into ActionsMatcher::Data and SelectQueryInfo - 40907 Parameterized views as table functions 2022-10-21 14:45:28 +02:00
Smita Kulkarni
e4ac3d0e18 Added FunctionParameterValuesVisitor, renamed bool to is_create_parameterized_view, added checks for parameterized view and support & test to propagate query parameters - 40907 Parameterized views as table functions 2022-10-19 18:30:03 +02:00
Smita Kulkarni
ec95294621 Merge branch 'master' into 40907_Parameterized_views_as_table_functions 2022-10-05 11:48:14 +02:00
vdimir
c0898ce289
Use abstraction for temporary data on disk in Sort and Aggregation 2022-09-29 09:51:41 +00:00
Arthur Passos
da9393ac6a
Merge branch 'master' into add_select_final_mt_setting 2022-09-28 09:56:29 -03:00
Smita Kulkarni
bbc33a54b2 Addressed review comments - 40907 Parameterized views as table functions
Implementation
* Fix for clang-today build fails - updated to use const in Context.cpp & const function in ActionsVisitior.cpp
* Updated to use QueryParameterVisitor to check if query has query parameters
* Updated executeTableFunction to check if table/table exists instead of try-catch approach
* Fixed small review comments and style comments.
Documentation:
* Addressed review comments and added the LIVE view part which was removed by mistake in the previous commits.
2022-09-27 16:30:59 +02:00
Smita Kulkarni
f801772a86 40907 Parameterized views as table functions
Implementation:
* Updated parsers by adding a bool allow_query_parameters while creating ordinary view, which is used in interpreters to allow query parameters in SELECT.
* Added a check in ActionsVisitor if multiple parameters have same names while creating parameterised view.
* Added bool in StorageView to represent parameterized view.
* Updated processing of SELECT with parameter values to check for views and added substitution of values in the query parameters.

Testing:
* Added a test tests/queries/0_stateless/02428_parameterized_view.sql

Documentation:
* Updated the english documentation for VIEW.
2022-09-23 14:04:43 +02:00
Alexey Milovidov
45afacdae4
Merge pull request #41186 from ClickHouse/fix-three-fourth-of-trash
Fix more than half of the trash
2022-09-22 07:28:26 +03:00
Nikita Taranov
100c055510
Prefetching in aggregation (#39304)
* impl

* stash

* clean up

* do not apply when HT is small

* make branch static

* also in merge

* do not hardcode look ahead value

* fix

* apply to methods with cheap key calculation

* more tests

* silence tidy

* fix build

* support HashMethodKeysFixed

* apply during merge only for cheap

* stash

* fixes

* rename method

* add feature flag

* cache prefetch threshold value

* fix

* fix

* Update HashMap.h

* fix typo

* 256KB as default l2 size

Co-authored-by: Alexey Milovidov <milovidov@clickhouse.com>
2022-09-21 18:59:07 +02:00
Alexey Milovidov
ab4db2d0c4 Fix 5/6 of trash 2022-09-19 08:50:53 +02:00
Arthur Passos
0b014e0caa Add MT select final setting 2022-09-07 09:08:19 -03:00
Alexey Milovidov
193cd1b3b2
Merge pull request #39138 from nickitat/control_block_size_in_aggregator
Control block size in aggregator
2022-09-04 04:51:00 +03:00
Vladimir C
12e6fc4182
Merge branch 'master' into join_and_sets 2022-09-01 14:56:14 +02:00
Vladimir C
f2cf7d7762
Merge pull request #40334 from ClickHouse/vdimir/analyze-stuck-limit
Limit number of analyze for one query, att. 2
2022-09-01 14:40:20 +02:00
vdimir
6c15db9b11 Setting max_analyze_depth forbid only second analyze 2022-08-31 11:57:41 +02:00
vdimir
60454bd722 Fix error codes in InterpreterSelectQuery.cpp 2022-08-31 11:57:41 +02:00
vdimir
f38f39ed6f Use max_analyze_depth 2022-08-31 11:57:41 +02:00
vdimir
928adb2b38 [WIP] Limit number of analyze for one query 2022-08-31 11:57:41 +02:00
Anton Popov
2a3e012931
Merge branch 'master' into fix-read-in-order-fixed-prefix 2022-08-29 13:17:26 +02:00
vdimir
afb6b7d9cf
Test plan and pipeline for filtering step for join 2022-08-29 09:49:15 +00:00
vdimir
afeff512b5
Aux port for ReadHeadBalancedProcessor is empty Block 2022-08-29 09:49:14 +00:00
vdimir
67a9acc8db
rename CreatingSetOnTheFlyStep -> CreateSetAndFilterOnTheFlyStep 2022-08-29 09:49:07 +00:00
vdimir
6ae15db3ad
check join kind for before applying max_rows_in_set_to_optimize_join 2022-08-29 09:49:04 +00:00
vdimir
51a51694d6
Create sets for joins: better code 2022-08-29 09:49:01 +00:00
vdimir
c778bba13f
Create sets for joins: wip 2022-08-29 09:47:00 +00:00
vdimir
71708d595f
Create sets for joins: wip 2022-08-29 09:46:59 +00:00
vdimir
8f06430ebd
Create sets for joins: upd 2022-08-29 09:46:58 +00:00
vdimir
031aaf3a45
Add Creating/FilterBySetsOnTheFlyStep for full sorting join 2022-08-29 09:46:57 +00:00
vdimir
cd4038d511
Use TemporaryFileOnDisk instead of Poco::TemporaryFile 2022-08-24 16:14:08 +00:00
Alexey Milovidov
ab91c99495
Merge branch 'master' into control_block_size_in_aggregator 2022-08-20 21:28:27 +03:00
yinpeiqi
042ad7d9a8 fix typo 2022-08-18 18:45:16 +08:00
Nikita Taranov
6bdbaccc37 use max_block_size from settings 2022-08-16 18:56:22 +02:00
Igor Nikonov
d4367de7bb Rename setting to optimize_sorting_by_input_stream_properties 2022-08-16 16:27:41 +00:00
Igor Nikonov
d83bea626c Merge remote-tracking branch 'origin/master' into skipping_sorting_step 2022-08-13 21:46:34 +00:00
Igor Nikonov
75f6fcfa70 Merge remote-tracking branch 'origin/master' into skipping_sorting_step 2022-08-11 12:35:55 +00:00
Amos Bird
fa8fab2e8f
Fix KeyCondition with other filters 2022-08-11 19:20:44 +08:00
vdimir
5eb4cd39e0
Merge branch 'master' into refactor-prepared-sets 2022-08-10 11:47:49 +00:00
Igor Nikonov
bf5269d536 Avoid copying SortDescription during SortingStep construction 2022-08-09 22:19:25 +00:00
Maksim Kita
c030fd05e7 ActionsDAG rename index to outputs 2022-08-08 18:01:32 +02:00
Vladimir C
a627b00c43
Merge branch 'master' into refactor-prepared-sets 2022-08-04 13:27:38 +02:00
Igor Nikonov
4bf42ce867 Merge remote-tracking branch 'origin/master' into skipping_sorting_step 2022-08-02 10:50:25 +00:00
Alexander Tokmakov
5050e0aca5
Revert "Limit number of analyze for one query" 2022-08-02 12:48:31 +03:00
Dmitry Novik
ab5a147065
Merge pull request #38725 from azat/fix-order-by-projection
Fix ORDER BY that matches projections ORDER BY
2022-08-01 00:32:59 +02:00
Igor Nikonov
7f0adb5eb0 Merge remote-tracking branch 'origin/master' into skipping_sorting_step 2022-07-31 07:07:36 +00:00
Alexey Milovidov
1e974b55ea
Merge branch 'master' into analyze_stuck 2022-07-30 04:56:21 +03:00