Commit Graph

759 Commits

Author SHA1 Message Date
MaxTheHuman
e22cbb7cd8 grouping sets dev: fix errors, something works 2022-01-11 11:26:11 +00:00
MaxTheHuman
715d4c9698 grouping sets cleanup 2022-01-11 11:26:10 +00:00
MaxTheHuman
a8f4f3d6b0 init commit with parsing and BAD realisation 2022-01-11 11:26:09 +00:00
Azat Khuzhin
c05dac89ee ReadFromRemote: Remove redundant ParallelReplicasReadingCoordinator 2022-01-07 20:04:57 +03:00
Nikolai Kochetov
c715204e18
Merge pull request #32271 from Algunenano/kill_scalar_github
Be able to KILL scalar queries
2021-12-29 16:00:59 +03:00
alexey-milovidov
0a55fa3dc2
Revert "Grouping sets dev" 2021-12-25 20:30:31 +03:00
Anton Popov
1f67177d4d fix optimize_read_in_order with preliminary merge and add some tests 2021-12-23 18:32:31 +03:00
Raúl Marín
dbb13036bc Hook progress callbacks with subqueries too 2021-12-22 23:13:45 +01:00
Raúl Marín
eddccb0065 Merge remote-tracking branch 'blessed/master' into kill_scalar_github 2021-12-21 10:00:17 +01:00
alexey-milovidov
6b97af4c63
Merge pull request #26869 from taylor12805/grouping-sets-dev
Grouping sets dev
2021-12-17 20:50:15 +03:00
Anton Popov
99ebabd822 Merge remote-tracking branch 'upstream/master' into HEAD 2021-12-17 19:02:29 +03:00
mergify[bot]
bff0424139
Merge branch 'master' into kill_scalar_github 2021-12-15 09:22:58 +00:00
Alexey Milovidov
5c90ed2ed9 Unambiguous formatting of distributed queries 2021-12-10 00:55:14 +03:00
Nikita Mikhaylov
dbf5091016
Parallel reading from replicas (#29279) 2021-12-09 13:39:28 +03:00
Raúl Marín
edf4095139 Add QueryStatus to BuildQueryPipelineSettings and use it 2021-12-09 10:02:32 +01:00
Anton Popov
6f4d9a53b2 Merge remote-tracking branch 'origin/sparse-serialization' into HEAD 2021-12-01 15:54:33 +03:00
fanzhou
534160b0d4 grouing sets transformer instead of fork processor in aggregating transformer 2021-11-29 18:59:15 +03:00
MaxTheHuman
e32d58a36d grouping sets dev: fix errors, something works 2021-11-26 22:46:15 +03:00
MaxTheHuman
e7711d953a grouping sets cleanup 2021-11-26 22:33:14 +03:00
Raúl Marín
f0ee0724ac Reduce dependencies on ASTSelectQuery.h
243 -> 152
2021-11-26 18:35:24 +01:00
Raúl Marín
b2cfa70541 Reduce dependencies on ASTFunction.h
481 -> 230
2021-11-26 18:21:54 +01:00
MaxTheHuman
8b95b7d271 init commit with parsing and BAD realisation 2021-11-23 17:58:16 +03:00
Anton Popov
a20922b2d3 Merge remote-tracking branch 'origin/sparse-serialization' into HEAD 2021-11-09 15:36:25 +03:00
kssenii
53c0e1ae61 Add setting for remote reads 2021-11-01 18:14:18 +03:00
alexey-milovidov
8c2413f6fe
Merge pull request #30562 from ClickHouse/fix-header-after-limit-push-down
Fix LimitStep header after limit push down optimization.
2021-10-24 00:19:02 +03:00
Anton Popov
cdfa36a181
Merge pull request #30266 from CurtizJ/aggregation-in-order-4
Improve performance of aggregation in order of primary key
2021-10-23 18:09:17 +03:00
Nikolai Kochetov
54cc8e1d27 Fix LimitStep header after limit push down optimization. 2021-10-22 16:19:49 +03:00
Anton Popov
151daf61c1
Merge pull request #30522 from azat/remove-check_columns
Remove check_columns argument from MergeTree code (false was never passed)
2021-10-22 14:06:49 +03:00
Nikolai Kochetov
1533f4af57
Merge pull request #30335 from ClickHouse/single-sorting-step
Single sorting step
2021-10-22 11:34:29 +03:00
Azat Khuzhin
496aa43a9f Remove check_columns argument from MergeTree code (false was never passed) 2021-10-21 21:24:59 +03:00
Anton Popov
82bff416f6 Merge remote-tracking branch 'upstream/master' into HEAD 2021-10-19 16:48:47 +03:00
Anton Popov
c547ffa313 improve performance of aggregation in order primary key with function with heavy states 2021-10-19 16:36:31 +03:00
Nikita Mikhaylov
4fbd332bf1 Done 2021-10-19 11:53:55 +00:00
Nikolai Kochetov
40677bffa5 Fix PVS warning. 2021-10-19 12:28:49 +03:00
Nikolai Kochetov
34d46245c3 Try fix some tests. 2021-10-18 20:53:30 +03:00
Anton Popov
3e742524b4 update comments 2021-10-18 19:25:25 +03:00
Nikolai Kochetov
ef3597f054 Fix build. 2021-10-18 19:21:49 +03:00
Anton Popov
1d9cfc04ef improve performance of aggregation in order of primary key 2021-10-18 18:32:09 +03:00
Nikolai Kochetov
5a4a752cfb Unite sorting steps. 2021-10-18 17:37:46 +03:00
Nikolai Kochetov
6e479b301a Update memory optimisation for MergingSorted. 2021-10-18 12:54:12 +03:00
Nikolai Kochetov
a08c98d760 Move some files. 2021-10-16 17:03:50 +03:00
Nikolai Kochetov
067eaadadd Merge branch 'master' into removing-data-streams-folder 2021-10-16 09:46:05 +03:00
Nikolai Kochetov
c668696047
Merge pull request #30171 from ClickHouse/remove-stream-interfaces
Remove stream interfaces
2021-10-16 09:34:01 +03:00
Anton Popov
c928ce8369 improve performance of aggregation in order of primary key 2021-10-16 04:33:45 +03:00
Nikolai Kochetov
fd14faeae2 Remove DataStreams folder. 2021-10-15 23:18:20 +03:00
Nikolai Kochetov
2957971ee3 Remove some last streams. 2021-10-13 21:22:02 +03:00
Amos Bird
ebfb013ea1
Fix potential leak of query_id_holder 2021-10-11 15:56:00 +08:00
Vladimir C
0422939893
Merge pull request #29281 from lingtaolf/bugfix/tupleFilter 2021-10-07 12:24:26 +03:00
Alexey Milovidov
fe6b7c77c7 Rename "common" to "base" 2021-10-02 10:13:14 +03:00
vdimir
23ce41e3bb
MergeTreeWhereOptimizer supports constant tuple in both sides of equals 2021-10-01 17:01:03 +03:00
Kruglov Pavel
14be2f31f5
Merge pull request #29475 from ClickHouse/fix-29010
Remove filter column from HAVING when it is not needed.
2021-09-29 13:03:27 +03:00
vdimir
46187a73ee wip 2021-09-28 14:11:31 +03:00
vdimir
8e2637aab2 Store all related to one join disjunct in JoinOnClause, pt1 2021-09-28 14:11:31 +03:00
Ilya Golshtein
3766d47f31 ORs in JOINs 2021-09-28 14:11:28 +03:00
Nikolai Kochetov
1a4ee1f77f Remove filter column from HAVING when it is not needed. 2021-09-28 13:01:47 +03:00
Nikolai Kochetov
236d71ea94
Merge pull request #28582 from ClickHouse/rewrite-pushing-to-views
Rewrite PushingToViews
2021-09-27 21:19:11 +03:00
Maksim Kita
d0f53cfc31 Added setting use_skip_indexes 2021-09-27 12:17:00 +03:00
Nikolai Kochetov
a8c3b02598 Merge branch 'master' into rewrite-pushing-to-views 2021-09-17 15:38:11 +03:00
Anton Popov
fc17936c12 Merge remote-tracking branch 'upstream/master' into HEAD 2021-09-14 23:02:30 +03:00
Nikolai Kochetov
b997214620 Rename QueryPipeline to QueryPipelineBuilder. 2021-09-14 20:48:18 +03:00
Nikolai Kochetov
7fc830ae2b Rename QueryPipeline to QueryPipelineBuilder. 2021-09-14 19:28:41 +03:00
Nikolai Kochetov
0e267c50b4 Merge branch 'master' into rewrite-pushing-to-views 2021-09-14 16:13:54 +03:00
Sergei Semin
74f6ac8976 remove variable total_rows_in_lonely_parts that set but not used 2021-09-09 23:51:47 +03:00
Nikolai Kochetov
f569a3e3f7 Merge branch 'master' into rewrite-pushing-to-views 2021-09-09 20:30:23 +03:00
Anton Popov
4c388e3d84 Merge remote-tracking branch 'origin/sparse-serialization' into HEAD 2021-09-09 14:10:16 +03:00
Nikolai Kochetov
999a4fe831 Fix other tests. 2021-09-08 21:29:38 +03:00
Anton Popov
5e421ab272 Merge remote-tracking branch 'upstream/master' into HEAD 2021-09-02 02:42:50 +03:00
Anton Popov
6d3274c22c Merge remote-tracking branch 'upstream/master' into HEAD 2021-08-26 03:31:46 +03:00
Alexey Milovidov
8f57216180 Progress on development 2021-08-25 00:45:58 +03:00
Anton Popov
61239343e3 Merge remote-tracking branch 'origin/sparse-serialization' into HEAD 2021-08-20 16:33:30 +03:00
Amos Bird
90881aab09
Better code style 2021-08-16 22:17:07 +08:00
Amos Bird
b162a2b699
Improve projection analysis.
Remove duplicate index analysis and avoid possible invalid limit checks
during projection analysis.
2021-08-16 20:11:57 +08:00
kssenii
f66e8464f9 Some final fixes 2021-08-15 11:40:30 +03:00
kssenii
f125fb3fef Some fixes around any/all 2021-08-14 20:33:03 +03:00
kssenii
47fb923975 Some fixes, more tests 2021-08-13 15:07:44 +03:00
kssenii
55650b1e67 Change behaviour of except, some fixes 2021-08-13 12:57:15 +03:00
kssenii
a549e29bd4 Better 2021-08-12 16:35:46 +03:00
kssenii
2306fbe9be Better 2021-08-10 10:23:18 +03:00
kssenii
4eb4dd97d4 Support not only 2 inputs, support any size sequence of intersect / except together 2021-08-10 09:24:12 +03:00
Kseniia Sumarokova
ba220c8e3a
Merge branch 'master' into subquery-operators 2021-08-08 20:18:41 +03:00
kssenii
5abe33e1a3 Better 2021-08-08 20:12:12 +03:00
kssenii
729eba52ea Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into zdikov-subquery-operators 2021-08-07 15:59:58 +00:00
Nikolai Kochetov
74dd3a6af6 More fixes. 2021-08-06 22:10:54 +03:00
Nikolai Kochetov
eb4d7415a2 Fix unknown column bug in sampling. 2021-08-06 17:28:56 +03:00
Nikolai Kochetov
48eb995461 Fix unknown column bug in sampling. 2021-08-06 17:23:50 +03:00
Anton Popov
3a0d4807a5 Merge branch 'async-insert' of git://github.com/abyss7/ClickHouse into merging-20557 2021-08-06 16:20:02 +03:00
Alexander Kuzmenkov
d852207b0e
Merge pull request #25721 from CurtizJ/read-in-order-perf-2
Reduce memory usage in queries with `ORDER BY primary_key
2021-08-05 16:13:48 +03:00
Anton Popov
915ef50c2d optimize reading in order of key more 2021-08-04 18:18:03 +03:00
Anton Popov
ad65e3c17b slightly more optimize reading in order of key 2021-08-03 15:32:17 +03:00
Anton Popov
e36736b50c Merge remote-tracking branch 'origin/sparse-serialization' into HEAD 2021-08-02 22:52:02 +03:00
Anton Popov
cc3ed12ed6 Merge remote-tracking branch 'upstream/master' into HEAD 2021-08-02 02:56:00 +03:00
Amos Bird
479d4fa991
Add shardNum() and shardCount() functions 2021-07-31 15:45:39 +08:00
Nikolai Kochetov
61d8f880cd Rename some files. 2021-07-26 19:48:25 +03:00
Anton Popov
f99374cca6 Merge remote-tracking branch 'origin/sparse-serialization' into HEAD 2021-07-20 18:20:21 +03:00
Amos Bird
dbfb699690
Asynchronously drain connections. 2021-07-19 21:53:29 +08:00
mergify[bot]
89634ae224
Merge branch 'master' into estimates_for_select_query 2021-07-16 23:15:30 +00:00
alexey-milovidov
149c5a0e9b
Merge pull request #26381 from ClickHouse/separate-step-for-distributed
Add separate step to read from remote.
2021-07-17 01:29:43 +03:00
Anton Popov
03c785931a fix pushdown of limit to reading stage 2021-07-16 17:27:38 +03:00
Anton Popov
c45a9e3bd6 Merge remote-tracking branch 'upstream/master' into HEAD 2021-07-16 17:11:34 +03:00
Nikolai Kochetov
a30b0eb542 Fix style. 2021-07-16 14:33:01 +03:00
Nikolai Kochetov
b31a4151de Fix build. 2021-07-16 14:30:07 +03:00
Nikolai Kochetov
d3099e0c89 Fix build and add comment. 2021-07-16 13:56:13 +03:00
Nikolai Kochetov
f36d14f68f Add separate step to read from remote. 2021-07-15 19:15:16 +03:00
alexey-milovidov
0893b9ff8e
Merge pull request #26218 from amosbird/projection-improve1
Aggressive IN index analysis for projections.
2021-07-15 08:13:45 +03:00
Amos Bird
28b567a01b
Update 2021-07-14 23:09:17 +08:00
alexey-milovidov
f20a3c596a
Update ReadFromMergeTree.cpp 2021-07-14 22:56:52 +08:00
Amos Bird
0558ecdc3f
Aggressive IN index analysis for projections. 2021-07-14 22:56:52 +08:00
vdimir
b49e37aa07
Rewrite join totals, fix block structure mismatch 2021-07-14 13:02:23 +03:00
Peng Jian
8522589483 Collect the amount of parts, rows and marks during the build query pipeline 2021-07-13 22:50:47 +08:00
Anton Popov
cb042afa9d pushdown limit while reading in order of primary key 2021-07-13 17:24:45 +03:00
Anton Popov
19d0f1bbfc Merge remote-tracking branch 'upstream/master' into HEAD 2021-07-13 16:12:08 +03:00
Anton Popov
6ba41dc265 fix projections with storage snapshot 2021-07-12 16:45:35 +03:00
mergify[bot]
b7ae4e4ced
Merge branch 'master' into async-insert 2021-07-12 09:21:37 +00:00
Alexey Milovidov
9c0097cd9b Review fixes 2021-07-11 02:14:03 +03:00
Peng Jian
fb793696a1 Add bytes to result of EXPLAIN ESTIMATES 2021-07-09 19:17:03 +08:00
Peng Jian
b526aacd56 Show estimates for SELECT query 2021-07-09 18:29:45 +08:00
Anton Popov
3ed7f5a6cc dynamic subcolumns: add snapshot for storage 2021-07-09 06:15:41 +03:00
Anton Popov
072e65b728 Merge remote-tracking branch 'origin/sparse-serialization' into HEAD 2021-07-07 17:20:38 +03:00
Nikolai Kochetov
4186bc29fd Fix build. 2021-06-29 20:01:22 +03:00
Nikolai Kochetov
7963600e9d Merge branch 'master' into use-dag-in-key-condition 2021-06-29 19:02:00 +03:00
Ivan Lezhankin
1bb41593d1 Merge remote-tracking branch 'upstream/master' into async-insert 2021-06-28 22:55:56 +03:00
Anton Popov
fa8ad1a7c6 reduce memory usage in queries with 'ORDER BY primary_key' 2021-06-25 19:23:41 +03:00
Nikolai Kochetov
3544b059ce Merge branch 'master' into remove-PrewhereDAGInfo 2021-06-25 17:52:03 +03:00
Nikolai Kochetov
6bc0a628cd Remove PrewhereDAGInfo. 2021-06-25 17:49:28 +03:00
Nikolai Kochetov
f5f57781b7 Fix build after merge. 2021-06-22 17:45:22 +03:00
Nikolai Kochetov
47f130d39c Try use expression for KeyCondition from query plan. 2021-06-22 16:54:00 +03:00
Anton Popov
447fef702d fix fuzzer in query with 'WITH FILL' and 'WITH TOTALS' 2021-06-22 16:08:12 +03:00
Nikolai Kochetov
eef6c73030 Use DAG in KeyCondition 2021-06-21 19:17:05 +03:00
Anton Popov
82f0a5f2dd fix usage of WITH FILL together with WITH TOTALS 2021-06-21 14:43:38 +03:00
Anton Popov
205a23282b Merge remote-tracking branch 'origin/sparse-serialization' into HEAD 2021-06-07 19:38:22 +03:00
Kirill Ershov
1d46e443e2
Fix INTERSECT/EXCEPT parser 2021-06-05 03:58:22 +03:00
Ivan Lezhankin
774107738c Fix build 2021-06-02 19:52:33 +03:00
Ivan Lezhankin
791cbcf0b1 Merge remote-tracking branch 'upstream/master' into async-insert 2021-06-02 18:50:37 +03:00
Ivan Lezhankin
84c23dc060 Merge branch 'master' into async-insert 2021-06-02 18:06:21 +03:00
Nikolai Kochetov
b54ff696e5 Merge branch 'master' into refactor-MergeTreeDataSelectExecutor 2021-06-01 12:08:37 +03:00
Anton Popov
9f52362b81 Merge remote-tracking branch 'origin/sparse-serialization' into HEAD 2021-05-31 23:08:45 +03:00
Nikolai Kochetov
b9aa404508 Review fixes. 2021-05-31 21:25:43 +03:00
Nikolai Kochetov
afc1fe7f3d Make ContextPtr const by default. 2021-05-31 17:49:02 +03:00
Kirill Ershov
42f6314c9a
Add INTERSECT and EXCEPT 2021-05-29 22:51:58 +03:00
Kirill Ershov
3f9e9a7025
Add INTERSECT and EXCEPT 2021-05-29 22:41:46 +03:00
Nikolai Kochetov
58fbc544cc Add more comments. 2021-05-28 20:16:09 +03:00
Nikolai Kochetov
295a302bc8 Remove settings from ReadFromMergeTree. 2021-05-28 17:34:02 +03:00
Nikolai Kochetov
91e8c2c75d Remove MergingFinal step. 2021-05-28 12:52:33 +03:00
Nikolai Kochetov
94f1ac5a16 Remove some commented code. 2021-05-28 12:41:07 +03:00
Nikolai Kochetov
c80ab6166a Fix style. 2021-05-28 12:26:44 +03:00
Nikolai Kochetov
4e28b7cb02 Fix _sample_factor column. 2021-05-28 12:23:46 +03:00
Nikolai Kochetov
ce11f35dcc Convert header from MergeTree to expected from query plan. 2021-05-28 11:11:17 +03:00
Nikolai Kochetov
1aeb705b20 Fix some tests. 2021-05-27 19:53:58 +03:00
Nikolai Kochetov
cbdf3752ef Part 3. 2021-05-27 16:40:33 +03:00
Nikolai Kochetov
a51a6ea0b7 Part 2. 2021-05-26 21:14:43 +03:00
Anton Popov
0bdf9d207c Merge remote-tracking branch 'origin/sparse-serialization' into HEAD 2021-05-26 00:25:47 +03:00
Nikolai Kochetov
34eaa48294 Part 1. 2021-05-25 19:34:43 +03:00
Nikolai Kochetov
79a0029843 Fix header mismatch for UNION. 2021-05-24 21:09:00 +03:00
Maksim Kita
c50909468f Updated ExpressionActionSettings 2021-05-19 17:32:07 +03:00
Maksim Kita
150a88d647 ExpressionActions compile only necessary places 2021-05-19 11:43:16 +03:00
Nikolai Kochetov
d78be51fb4 Calculate header for ActionsDAG 2021-05-13 16:38:18 +03:00
Kruglov Pavel
49e7ed6e60
Merge pull request #23673 from amosbird/partitionvalue
Add _partition_value virtual column
2021-05-11 11:23:00 +03:00
Alexey Milovidov
ad88819ee4 Fix a bunch of warnings from PVS-Studio 2021-05-08 19:13:10 +03:00
Alexey Milovidov
3025f9e141 More fixes for PVS-Studio 2021-05-08 19:12:49 +03:00
Anton Popov
e44706911e dynamic columns: better getting of sample block 2021-05-05 02:02:54 +03:00
Nikolai Kochetov
6b79403cb5 Rename and fix fuild. 2021-04-30 11:25:39 +03:00
Nikolai Kochetov
25dc72eda7 Add comments. 2021-04-29 20:51:35 +03:00
Nikolai Kochetov
b500c420b9 Fix build and style. 2021-04-29 17:14:23 +03:00
Nikolai Kochetov
c4e8288ce0 Fix style. 2021-04-29 13:03:55 +03:00
Nikolai Kochetov
a179124f17 Fix build 2021-04-29 12:59:38 +03:00
Nikolai Kochetov
1f5a0f8842 Merge branch 'master' into refactor-join-step 2021-04-29 12:23:36 +03:00
Nikolai Kochetov
25fe5f63ee Fix other tests. 2021-04-29 12:08:49 +03:00
Nikolai Kochetov
69ddca1d73 Refactor join step, part 3 2021-04-28 20:32:12 +03:00
Nikolai Kochetov
c508868570
Merge pull request #23082 from ClickHouse/explain-output
JSON output for EXPLAIN
2021-04-28 15:39:06 +03:00
Nikolai Kochetov
3a25e46a8f Refactor join step, part 2 2021-04-27 20:30:33 +03:00
Amos Bird
8a3b5c1fab
Add _partition_value virtual column 2021-04-27 16:15:59 +08:00
Anton Popov
644df6be7d dynamic subcolumns: wip 2021-04-24 07:09:01 +03:00
Alexander Kuzmenkov
06ca44a378 fix pvs warnings 2021-04-22 22:54:58 +03:00
Ivan Lezhankin
ee194928d2 CC 2021-04-21 16:19:28 +03:00
Nikolai Kochetov
6b264933b2 Fix style. 2021-04-20 17:51:48 +03:00
Nikolai Kochetov
9632b83bdf Support explain json = 1, indexes = 1. Added test. 2021-04-20 16:39:23 +03:00
Nikolai Kochetov
8c2d58333a Merge branch 'master' into explain-output 2021-04-19 17:40:27 +03:00
Nikolai Kochetov
4d86f51eff Merge branch 'master' into add-read-from-mt-step 2021-04-19 10:17:21 +03:00
Nikolai Kochetov
1bf420cbcf Update json description for other steps. 2021-04-16 19:36:59 +03:00
Nikolai Kochetov
d5882620b2 Fix condition 2021-04-16 14:33:20 +03:00
Nikolai Kochetov
23089a2fec Add special setting to show indexes. 2021-04-16 14:11:45 +03:00
Nikolai Kochetov
8d8e57615c A little bit better index description. 2021-04-16 12:42:23 +03:00
Nikolai Kochetov
be52b2889a Better description for key condition. 2021-04-15 20:30:04 +03:00
Nikolai Kochetov
9b86c19836 Use JSONBuilder for explain json result. 2021-04-14 17:07:56 +03:00
Nikolai Kochetov
d1d2b89a51 Use setting for explain json 2021-04-13 10:51:55 +03:00
Nikolai Kochetov
f6d86d6032 Merge branch 'master' into add-read-from-mt-step 2021-04-12 15:23:32 +03:00
Ivan
495c6e03aa
Replace all Context references with std::weak_ptr (#22297)
* Replace all Context references with std::weak_ptr

* Fix shared context captured by value

* Fix build

* Fix Context with named sessions

* Fix copy context

* Fix gcc build

* Merge with master and fix build

* Fix gcc-9 build
2021-04-11 02:33:54 +03:00
Nikolai Kochetov
48bcd5b490 Add tree conversion for expression. 2021-04-09 19:18:45 +03:00
Nikolai Kochetov
66b6d53a93 Add json=1 setting for explain actions 2021-04-09 17:44:58 +03:00
Nikolai Kochetov
d0b5615641 Add comments. 2021-04-09 12:35:21 +03:00
Nikolai Kochetov
28ca191102
Merge pull request #22087 from ClickHouse/better-filter-push-down
Better filter push down
2021-04-09 10:22:17 +03:00
Nikolai Kochetov
3426bc3906
Merge pull request #22763 from CurtizJ/fix-having-push-down
Fix pushdown of having
2021-04-08 21:53:50 +03:00
Nikolai Kochetov
794d7c89b6 Remove ReverseRowsStep 2021-04-08 16:18:17 +03:00
Nikolai Kochetov
febb187da0 Added test, 2021-04-08 15:54:27 +03:00
Nikolai Kochetov
7ffbeac9df Add info about indexes to ReadFromMergeTree step. 2021-04-08 14:48:54 +03:00
Nikolai Kochetov
118090396f Filter column after filter push down now always the first one.. 2021-04-08 12:54:39 +03:00
Nikolai Kochetov
7c5a9133df Add index info to ReadFromStorageStep. 2021-04-08 11:19:04 +03:00
Anton Popov
fa9a19cbdd fix pushdown of having 2021-04-07 17:02:59 +03:00
Alexander Kuzmenkov
585da2c8d0 Merge remote-tracking branch 'origin/master' into HEAD 2021-04-06 14:58:21 +03:00
Alexander Kuzmenkov
06e7a9f8f2 fix window functions with multiple input streams and no sorting 2021-04-06 14:43:27 +03:00
Nikolai Kochetov
6102652c99 Merge branch 'master' into better-filter-push-down 2021-04-06 13:38:03 +03:00
Nikolai Kochetov
44700d97e9 Add setting to disable query plan optimization. Fix perftest. 2021-04-06 11:25:54 +03:00
Anton Popov
88cd775f6a Merge remote-tracking branch 'upstream/master' into HEAD 2021-04-02 00:14:03 +03:00
Nikolai Kochetov
8fad179800 Fix some tests. 2021-03-30 18:14:20 +03:00
Nikolai Kochetov
88bfa6c9ba Add ReadFromMergeTree step. 2021-03-30 13:25:26 +03:00
Nikolai Kochetov
e9f7213858 Filter push down for LEFT/INNER JOIN, CreatingSets, Union. 2021-03-25 21:11:54 +03:00
Nikolai Kochetov
0e2f52518f Extract converting from UnionStep. 2021-03-25 12:57:14 +03:00
Nikolai Kochetov
0afede382b Use empty header for CreatingSets step. 2021-03-25 11:39:55 +03:00
Nikolai Kochetov
a325d75d75 Remove AddingDelayedSourceStep. 2021-03-24 16:27:04 +03:00
Nikolai Kochetov
12de01cb48 Remove AddingDelayedSourceStep. 2021-03-24 16:22:51 +03:00
Anton Popov
6f7c8894e9 fix bugs in aggregation by primary key 2021-03-18 23:57:42 +03:00
Nikolai Kochetov
c3c393a7aa Merge branch 'master' into refactor-actions-dag 2021-03-18 14:33:07 +03:00
Nikolai Kochetov
3eba817a68 Add comments. 2021-03-17 19:08:46 +03:00
Nikolai Kochetov
738cb1af62 Fix filter push down columns order. 2021-03-17 18:33:18 +03:00
Nikolai Kochetov
4967227fe9 Remove commented code. 2021-03-11 20:11:57 +03:00
Nikolai Kochetov
7a4dafe106 Fix filter push down columns order. 2021-03-11 13:34:15 +03:00
Nikolai Kochetov
4f133ae036 Merge branch 'master' into refactor-actions-dag 2021-03-10 12:40:55 +03:00
Anton Popov
562065732d Merge remote-tracking branch 'upstream/master' into HEAD 2021-03-05 17:30:05 +03:00
Nikolai Kochetov
9a39459888 Refactor ActionsDAG 2021-03-04 20:38:12 +03:00
Nikolai Kochetov
15921fbfcb Remove index by name from ActionsDAG 2021-03-03 23:01:07 +03:00
Nikolai Kochetov
d328bfa41f Review fixes. Add setting max_optimizations_to_apply. 2021-02-26 19:29:56 +03:00
Nikolai Kochetov
00e0dbc3e5 Fix test. 2021-02-20 20:42:06 +03:00
Nikolai Kochetov
2ae0b47edb Refactor tryPushDownFilter optimization. 2021-02-20 19:25:47 +03:00
Nikolai Kochetov
6e9bf68217 Fix typo 2021-02-18 23:54:42 +03:00
Nikolai Kochetov
0449546bca Support TotalsHaving. Update test. 2021-02-18 16:13:09 +03:00
Nikolai Kochetov
7231a97085 Remove MaterializingStep 2021-02-18 14:15:16 +03:00
Nikolai Kochetov
56a5d1dafa Skip stateful functions 2021-02-17 21:48:26 +03:00
Nikolai Kochetov
f6278ed429 Support for DIstinct, sorting steps. 2021-02-17 19:56:17 +03:00
Nikolai Kochetov
6522bfc402 Support for DIstinct, sorting steps. 2021-02-17 19:54:11 +03:00
Nikolai Kochetov
a8647096ed Try fix tests. 2021-02-17 13:27:47 +03:00
Nikolai Kochetov
f1c32c4690 Merge branch 'master' into filter-push-down 2021-02-12 18:26:08 +03:00
Nikolai Kochetov
90c7cf5a52 Push down for ArrayJoin 2021-02-12 18:24:31 +03:00
Nikolai Kochetov
93e1428f21 Fix limit push down. 2021-02-12 13:51:16 +03:00
Nikolai Kochetov
443a3e7e6f Fix limit push down. 2021-02-12 13:12:31 +03:00
Nikolai Kochetov
93ea1e5e82 Comment output 2021-02-11 20:13:59 +03:00
Nikolai Kochetov
a26c8d9eee Fix const filter resilt for filter push down. 2021-02-11 19:08:54 +03:00
Nikolai Kochetov
3a020d2dd5 filter push down for Aggregating 2021-02-11 11:49:12 +03:00
alesapin
c2bb2c2902
Merge pull request #20097 from ClickHouse/remove-adding-missed-step
Build actions dag to evaluate missing defaults.
2021-02-11 10:51:21 +03:00
Alexander Kuzmenkov
2571dac984
Merge pull request #20111 from ClickHouse/aku/window-prototype
RANGE OFFSET window frame
2021-02-11 10:39:21 +03:00
Alexander Kuzmenkov
a2a0d385c5 style 2021-02-09 19:50:39 +03:00
Alexander Kuzmenkov
8212976dc0 add boundary order check 2021-02-09 18:36:08 +03:00
Nikolai Kochetov
e0a806c180
Delete filterPushDown.cpp 2021-02-08 22:10:27 +03:00
Nikolai Kochetov
00d5d28ddf Fix build. 2021-02-08 15:52:30 +03:00
Nikolai Kochetov
d7fa440356
Merge branch 'master' into remove-adding-const-column-transform 2021-02-08 15:49:23 +03:00
Nikolai Kochetov
98cfefdcfb Fix build 2021-02-05 14:49:58 +03:00
Nikolai Kochetov
9869f70a0d Remove AddMissed step and transform. 2021-02-05 14:41:44 +03:00
alesapin
65d7b39cd6
Merge pull request #20073 from ClickHouse/fix-split-filter-remove-column
Fix removing of filter column when split filter actions.
2021-02-05 13:31:02 +03:00
Nikolai Kochetov
a34cd32552 Fix build 2021-02-05 12:25:16 +03:00
Alexander Kuzmenkov
6824f13a35 tmp 2021-02-05 12:13:19 +03:00
Nikolai Kochetov
d9aa1096cf Build actions dag to evaluate missing defaults. 2021-02-04 23:36:50 +03:00
Nikolai Kochetov
e3d6ffd9dc Remove AddingConstColumn step and transform. 2021-02-04 17:25:11 +03:00
Nikolai Kochetov
7a2279d067 Fix removing filter column from expression after Filter actions split 2021-02-04 14:44:00 +03:00
Alexander Kuzmenkov
f31d2206a7 more fuzzing and less bugs 2021-02-03 15:50:25 +03:00
Alexander Kuzmenkov
264aea20be Merge remote-tracking branch 'origin/master' into HEAD 2021-02-02 19:12:46 +03:00
Nikolai Kochetov
19e4a33f9d
Merge pull request #19544 from amosbird/limitconcurrency
Per MergeTree table query limit
2021-02-01 16:09:12 +03:00
Alexander Kuzmenkov
5519e4c134 unbounded following frame end 2021-01-30 04:16:44 +03:00
Nikolai Kochetov
3b50a74759 Simplify optimization interface. 2021-01-28 15:32:02 +03:00
Nikolai Kochetov
4ef56a41cb Simplify optimizeTree 2021-01-28 14:15:45 +03:00
Nikolai Kochetov
e1765e7f88 Add method trivial to ActionsDAG 2021-01-28 14:00:24 +03:00
Nikolai Kochetov
3b7373e992 More comments. 2021-01-27 22:53:07 +03:00
Nikolai Kochetov
b244499d69 Fix build. 2021-01-27 22:24:59 +03:00
Nikolai Kochetov
cf05c17cfa Rewrite QueryPlan tree optimizations traverse. 2021-01-27 20:36:53 +03:00
Nikolai Kochetov
e9f893d9b0 Move query plan optimixations into separate files. 2021-01-27 16:57:26 +03:00
Anton Popov
666aab676e add comments to algorithm 2021-01-26 21:45:22 +03:00
Amos Bird
66fe97d8bd
Per MergeTree table query limit 2021-01-26 14:03:31 +08:00
Nikolai Kochetov
d683bf42ce Review fixes. 2021-01-22 16:46:56 +03:00
Anton Popov
573edbcd11 improve performance of aggregation in order of sorting key 2021-01-22 05:34:08 +03:00
Nikolai Kochetov
801c540f5e Try fix tests. 2021-01-20 20:33:12 +03:00
Nikolai Kochetov
fa3964d36d Fix test. 2021-01-20 12:42:55 +03:00
Nikolai Kochetov
a69d386492 Fix tests. 2021-01-19 20:09:40 +03:00
Nikolai Kochetov
35f48e60ad Fix description after step optimizations 2021-01-19 16:21:28 +03:00
Nikolai Kochetov
964af8e02c Fix description after step optimizations 2021-01-19 16:17:52 +03:00
Nikolai Kochetov
b1c7944f84 Fix description after step optimizations 2021-01-19 16:08:14 +03:00
Nikolai Kochetov
9db2974aaa Update explain for filter 2021-01-19 15:54:55 +03:00
Nikolai Kochetov
0711957fbb Split filter optimization. 2021-01-19 15:08:21 +03:00
Nikolai Kochetov
91403b2f75 Split filter optimization. 2021-01-19 15:04:45 +03:00
Nikolai Kochetov
b00f01d6b1 Split filter optimization. 2021-01-19 13:03:25 +03:00
Nikolai Kochetov
6e1a118642 Merge Filter and Expression steps. 2021-01-19 11:14:37 +03:00
Nikolai Kochetov
7135c62204 Update tryLiftUpArrayJoin 2021-01-18 19:12:16 +03:00
Nikolai Kochetov
1937659450 Add more info for ActionsDAG 2021-01-12 21:47:54 +03:00
Alexander Kuzmenkov
d4e5d5fdb4
Merge pull request #18455 from ClickHouse/aku/window-prototype
[wip] some window function fixes
2020-12-29 19:23:19 +03:00
Nikolai Kochetov
d6f501f497 Try fix Pipeline stuck 2020-12-27 13:30:00 +03:00
Alexander Kuzmenkov
61a35cdafd final cleanup 2020-12-24 12:12:34 +03:00
Alexander Kuzmenkov
c6c2b3ba00 Remove the forward declaration changes 2020-12-24 12:12:34 +03:00
Alexander Kuzmenkov
2bb0f5772e proper signature 2020-12-24 12:12:34 +03:00
Alexander Kuzmenkov
23d77a1698 remove debug output 2020-12-24 09:59:38 +03:00
Alexander Kuzmenkov
844239b1c3 tmp fixes (split me) 2020-12-24 07:03:33 +03:00
Alexander Kuzmenkov
85c1bc1253 Merge remote-tracking branch 'origin/master' into tmp 2020-12-21 10:46:21 +03:00
Alexander Kuzmenkov
132a0b42eb cleanup 2020-12-18 20:13:28 +03:00
Amos Bird
272311f6bd
Extensive information in system.query_log 2020-12-18 14:13:58 +08:00
Alexander Kuzmenkov
e335074c39 works across the blocks 2020-12-16 15:57:47 +03:00
Alexander Kuzmenkov
fabb37d3c9 cleanup 2020-12-15 20:39:58 +03:00
Alexander Kuzmenkov
a75ca3cb79 Merge remote-tracking branch 'origin/master' into tmp 2020-12-15 03:36:43 +03:00
Alexander Kuzmenkov
6d5b23de67 something works 2020-12-15 03:36:03 +03:00
alexey-milovidov
bce02de691
Merge pull request #17539 from azat/remerge_sort_lowered_memory_bytes_ratio
Add remerge_sort_lowered_memory_bytes_ratio setting
2020-12-13 04:38:50 +03:00
Alexander Kuzmenkov
6f991725c3 forgot new files 2020-12-11 01:16:58 +03:00
alesapin
57f40e4260
Merge pull request #17458 from ClickHouse/merge-expressions
Merge expressions
2020-12-07 12:34:19 +03:00
Nikolai Kochetov
e57ad9cb7e
Update QueryPlan.cpp 2020-12-04 11:41:36 +03:00
Azat Khuzhin
ccd794a676 Add remerge_sort_lowered_memory_bytes_ratio setting
Sometimes default ratio is not good enough (2), since it depends on lots
of factors:
- size of LIMIT
- max_memory_usage
- ...

So add a separate setting for it.

But note that it make sense to set
remerge_sort_lowered_memory_bytes_ratio only in range (1, 2]

Since <= 1 will just use more CPU, and > 2 will use more RAM and will
unlikely be faster.
2020-11-29 21:16:42 +03:00
Nikolai Kochetov
a2c450d7e2 Do not merge functions with arrayJoin and stateful functions. 2020-11-27 12:50:56 +03:00
Nikolai Kochetov
c42cc8f1ff Merge Expressions in QueryPlan. 2020-11-26 22:09:42 +03:00
Nikolai Kochetov
4378ca21ca Properly check distinct columns. 2020-11-26 12:46:41 +03:00
Nikolai Kochetov
4c09880bc4 Properly check distinct columns. 2020-11-26 11:50:02 +03:00
lichengxiang
fdde2d1e8a fix #17294: Distinct on subquery with group by may return duplicate result 2020-11-23 17:06:59 +08:00
Nikolai Kochetov
13eceff55c Remove Converting transform and step. 2020-11-17 20:16:55 +03:00
Nikolai Kochetov
e8e6461fa7 Move ActionsDAG to separate file. 2020-11-16 17:57:56 +03:00
Nikolai Kochetov
9b13d0e0eb Revert "Add ActionsDAG::updateHeader()"
This reverts commit b78657656c.
2020-11-11 20:17:26 +03:00
Nikolai Kochetov
58c4a7b890 Some renames. 2020-11-11 19:52:27 +03:00
Nikolai Kochetov
4abcd1bcb3 Add more comments. 2020-11-11 17:26:18 +03:00
Nikolai Kochetov
b78657656c Add ActionsDAG::updateHeader() 2020-11-11 16:50:29 +03:00
Nikolai Kochetov
1846bb3cac Merge branch 'master' into actions-dag-f14 2020-11-11 13:08:57 +03:00
Nikolai Kochetov
c6575c9032 Update ExpressionActions constructur 2020-11-10 19:27:55 +03:00
Nikolai Kochetov
6717c7a0af Merge branch 'master' into actions-dag-f14 2020-11-09 14:57:48 +03:00
Alexey Milovidov
5314185e25 Merge branch 'master' into azat-optimize_skip_unused_shards-optimization 2020-11-08 00:17:59 +03:00
Nikolai Kochetov
4ce88298d7 Fix build 2020-11-03 22:33:29 +03:00
Nikolai Kochetov
07a7c46b89 Refactor ExpressionActions [Part 3] 2020-11-03 14:28:28 +03:00
Nikolai Kochetov
c91b453630 Try fix tests. 2020-10-22 14:08:12 +03:00
Alexey Milovidov
269b6383f5 Check for #pragma once in headers 2020-10-10 21:37:02 +03:00
Nikolai Kochetov
d94d88a6c0 Fix style. 2020-10-07 15:04:15 +03:00
Nikolai Kochetov
7e02152b50 Merge branch 'master' into storage-read-query-plan 2020-10-07 14:27:21 +03:00
Nikolai Kochetov
c5cb05f5f3 Try fix tests. 2020-10-07 14:26:29 +03:00
Nikolai Kochetov
3564ba1c63 Remove moves. 2020-10-06 13:02:07 +03:00
Nikolai Kochetov
efd1f10a6d Fix build. 2020-10-06 11:55:03 +03:00
Nikolai Kochetov
5bbae0953a Fix order of resource destruction in SettingQuotaAndLimitsStep. 2020-10-06 09:53:51 +03:00
Nikolai Kochetov
28e12c559c Ensure resource desrtuction order in ReadFromStorageStep. 2020-10-06 09:35:35 +03:00
Azat Khuzhin
b838214a35 Pass non-const SelectQueryInfo (and drop mutable qualifiers) 2020-10-02 22:42:35 +03:00
Nikolai Kochetov
11c0c2864f Fix build. 2020-10-02 17:23:06 +03:00
Nikolai Kochetov
5134028140 Fix style. 2020-10-01 21:19:38 +03:00
Nikolai Kochetov
f02ad3dc8e Use QueryPlan while reading from MergeTree. 2020-10-01 21:02:22 +03:00
Nikolai Kochetov
ec64def384 Use QueryPlan while reading from MergeTree. 2020-10-01 20:34:22 +03:00
Nikolai Kochetov
cb22152878 Fix build. 2020-09-30 18:54:16 +03:00
Nikolai Kochetov
2efde179e9 Merge branch 'master' into storage-read-query-plan 2020-09-30 15:23:46 +03:00
Nikolai Kochetov
5ac6bc071d QueryPlan for StorageBuffer and StorageMaterializedView read. 2020-09-30 15:22:57 +03:00
Nikolai Kochetov
576ffadb17 Fix explain for ISourceStep. 2020-09-30 15:22:30 +03:00
Nikolai Kochetov
347080ff9a Remove column from header descriotion for EXPLAIN PLAN and EXPLAIN PIPELINE 2020-09-28 20:37:11 +03:00
Nikolai Kochetov
dea90009e3 Fix build 2020-09-25 16:03:12 +03:00
Nikolai Kochetov
62ef40394a
Merge pull request #14856 from ClickHouse/query-plan-for-creating-sets
Query plan for creating sets
2020-09-22 16:19:20 +03:00
Nikolai Kochetov
54c1f04b38 Review fixes. 2020-09-21 11:36:12 +03:00
Nikolai Kochetov
b26f11c00c Support StorageDistributed::read for QueryPlan. 2020-09-18 17:16:53 +03:00
Nikolai Kochetov
50674a320f Refactor IStorage::read with query plan. 2020-09-18 14:39:07 +03:00
Nikolai Kochetov
2491f0e1c5 Merge branch 'master' into storage-read-query-plan 2020-09-18 14:07:03 +03:00
Nikolai Kochetov
0bf4e8e6e9 Support reading with QueryPlan for StorageView. 2020-09-17 16:22:24 +03:00
Nikolai Kochetov
193b572a05 Add MaterializingStep. 2020-09-17 15:11:23 +03:00
roman
b41421cb1c [settings]: introduce new query complexity settings for leaf-nodes
The new setting should allow to control query complexity on leaf nodes
excluding the final merging stage on the root-node. For example, distributed
query that reads 1k rows from 5 shards will breach the `max_rows_to_read=5000`,
while effectively every shard reads only 1k rows. With setting `max_rows_to_read_leaf=1500`
this limit won't be reached and query will succeed since every shard reads
not more that ~1k rows.
2020-09-17 10:37:05 +01:00
Nikolai Kochetov
7384733144 Fix style. 2020-09-17 09:01:35 +03:00
Nikolai Kochetov
73ad505735 Fix tests. 2020-09-16 19:38:33 +03:00
Nikolai Kochetov
2f7ab2bca4 Fix tests. 2020-09-16 19:30:48 +03:00
Nikolai Kochetov
8a26251f94 Fix tests. 2020-09-16 19:11:16 +03:00
Nikolai Kochetov
8de76709fb Fix tests. 2020-09-16 17:41:01 +03:00
Nikolai Kochetov
3cc89d467c Merge branch 'master' into query-plan-for-creating-sets 2020-09-15 20:15:41 +03:00