thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-15 19:02:04 +00:00
Code Issues Packages Projects Releases Wiki Activity
100,095 Commits 9,326 Branches 1,783 Tags 2.1 GiB
0707d88311
Commit Graph

813 Commits

Author SHA1 Message Date
Igor Nikonov
572dd864fe Merge remote-tracking branch 'origin/master' into skipping_sorting_step 2022-07-25 12:19:14 +00:00
Alexander Gololobov
460950ecdc
Merge branch 'master' into feature/sql-standard-delete 2022-07-24 21:27:22 +02:00
Azat Khuzhin
22d8e532ed Fix ORDER BY that matches projections ORDER BY 
In case of projection is seleted and it is comlete (all parts has such
projection) and ORDER BY matches projection ORDER BY, and
optimize_read_in_order=1 (default), then the sorting is simply not done,
because projections has separate plan, and so
InterpreterSelectQuery::executeOrder() -> SortingStep has
pipeline.getNumStreams() == 0 and it cannot do sorting.

Fix this, by adding sorting when creating plan for reading from
projections.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-07-24 15:23:13 +03:00
Ilya Yatsishin
196fe5a269
Merge branch 'master' into exact-rows-before-limit 2022-07-22 12:34:30 +02:00
Nikolai Kochetov
e6969d5010 Respect table alias for additional_table_filters. 2022-07-21 11:23:09 +00:00
Nikolai Kochetov
91aa2385ca
Merge branch 'master' into additional-filters 2022-07-20 18:56:24 +02:00
Yakov Olkhovskiy
c4d040e02c
Merge branch 'master' into bug-with-fill-date 2022-07-20 09:10:45 -04:00
Igor Nikonov
1fe83cc8d8 optimize_sorting_for_input_stream setting and perf tests 2022-07-19 16:58:15 +00:00
Dmitry Novik
50989bdb68
Merge branch 'master' into group-by-use-nulls 2022-07-19 14:58:01 +02:00
jianmei zhang
780cdfb8f0 Code changes based on review opinions. Make ordinary single alter delete lightwight by default 2022-07-15 12:32:42 +08:00
jianmei zhang
b4a37e1e22 Disable optimizations for count() when lightweight delete exists, add hasLightweightDelete() function in IMergeTreeDataPart 2022-07-15 12:32:41 +08:00
vdimir
bddf6c1b32
Pushdown filter to the right side of sorting join 2022-07-13 11:36:25 +00:00
Amos Bird
0595c50c92
Fix trivial count with empty set 2022-07-13 12:54:52 +08:00
Dmitry Novik
5f65b45269
Merge branch 'master' into group-by-use-nulls 2022-07-12 22:36:04 +02:00
Dmitry Novik
66074b3950 Fix distributed aggregation 2022-07-11 18:11:58 +00:00
Ilya Yatsishin
4625e7f519
Merge branch 'master' into exact-rows-before-limit 2022-07-11 15:41:35 +02:00
Vladimir C
ca2e27aaa2
Merge pull request #37841 from vdimir/perf-save-to-stat 
Send perf tests results to ci database
 2022-07-11 13:07:40 +02:00
vdimir
79c185d22b
make current_query_analyze_count atomic 2022-07-08 13:01:17 +00:00
vdimir
d681469275
max_pipeline_depth only throws an error 2022-07-08 11:56:42 +00:00
vdimir
9c348380f6
second analyze and max_pipeline_depth 2022-07-08 11:56:42 +00:00
Alexey Milovidov
eae82d7530
Update src/Interpreters/InterpreterSelectQuery.cpp 2022-07-08 11:56:41 +00:00
vdimir
fdf35b20a8
Limit number of analyze for one query 2022-07-08 11:56:40 +00:00
Dmitry Novik
d1df66687b
Merge branch 'master' into group-by-use-nulls 2022-07-07 20:54:38 +02:00
Dmitry Novik
1587385f7a Cleanup code 2022-07-07 18:53:20 +00:00
Vladimir C
db838f1343
Merge pull request #35796 from vdimir/full-sorting-merge-join 2022-07-07 19:16:49 +02:00
Dmitry Novik
24243c51be Fix distributed grouping sets with nulls 2022-07-07 16:40:35 +00:00
Igor Nikonov
6be6a13c2f
Merge branch 'master' into order_by_with_braces 2022-07-07 00:36:12 +02:00
vdimir
fa8eb35599
Pipeline for full sorting merge join 2022-07-06 14:23:44 +00:00
Nikolai Kochetov
65cd0cc61c Fixing style. 2022-07-06 13:44:06 +00:00
Igor Nikonov
9ca08f8fda Continue: ContextPtr -> const ContextPtr & in parameters 2022-07-06 13:13:19 +00:00
Igor Nikonov
059c5c0f0a Simple cleanup: context ptr usage in interpreters 2022-07-06 12:29:20 +00:00
Igor Nikonov
5a01c27eb1 Fix: clear children in order by element during rewriting 
- remove unnecessary code for this PR
 2022-07-05 22:48:48 +00:00
Igor Nikonov
fd62494a77 Fix: ORDER BY with braces inefficient execution 2022-07-05 22:48:48 +00:00
Nikolai Kochetov
659a6452f5 Added SettingMap. Use Map for additional_table_filters. 2022-07-05 13:10:09 +00:00
vdimir
0232c9267d
Pass CHPC_DATABASE_URL to perf tests 2022-07-05 14:00:15 +02:00
Anton Popov
ef87e1207c better support of read_in_order in case of fixed prefix of sorting key 2022-07-01 16:45:01 +00:00
Dmitry Novik
81dd90893e Merge remote-tracking branch 'origin/master' into group-by-use-nulls 2022-07-01 16:24:05 +00:00
Dmitry Novik
33f601ec0a Commit support use_nulls for GS 2022-06-30 15:14:26 +00:00
Igor Nikonov
488ee75fc4 + use DistinctSorted for final distinct step 
+ fix performance tests
 2022-06-30 13:03:39 +00:00
Dmitry Novik
98e9bc84d5 Refector ROLLUP and CUBE 2022-06-30 10:13:58 +00:00
Igor Nikonov
d435532c68 Adapt range search algorithm to high cardinality case 
+ range search done in steps of some number of rows.
  Controled by new
  setting `distinct_in_order_range_search_step`. By default 0, i.e.
  whole chunk
+ before start binary search, linear probing is done on each step (32
  rows currently)
 2022-06-29 23:30:35 +00:00
mergify[bot]
36139eacd7
Merge branch 'master' into dictinct_in_order_optimization 2022-06-29 13:37:16 +00:00
Anton Popov
e78814f3bb
Merge pull request #38371 from CurtizJ/fix-distint-with-limit 
Fix `DISTINCT` with `LIMIT` in distributed queries
 2022-06-29 14:02:41 +02:00
Nikita Taranov
f5d26572df
Quick fix for aggregation pipeline (#38295) 2022-06-29 01:16:30 +02:00
mergify[bot]
a9c1b68034
Merge branch 'master' into dictinct_in_order_optimization 2022-06-27 20:16:00 +00:00
Dmitry Novik
1d15d72211 Support NULLs in ROLLUP 2022-06-27 18:42:26 +00:00
Nikolai Kochetov
6824570ea3 Fix style. 2022-06-27 18:10:29 +00:00
Nikolai Kochetov
99436ab1da Remove unneeded code. 2022-06-27 17:46:09 +00:00
Nikolai Kochetov
79cf4c4f7e Remove commented code. 2022-06-27 17:44:30 +00:00
Nikolai Kochetov
1d5ba77c87 Fix test. 2022-06-27 17:41:55 +00:00
Nikita Taranov
2487ba7f00
Move updateInputStream to ITransformingStep (#37393) 2022-06-27 13:16:52 +02:00
mergify[bot]
b65cf4e1fe
Merge branch 'master' into dictinct_in_order_optimization 2022-06-24 22:52:14 +00:00
Anton Popov
a2158066f2 fix distinct with limit in distributed queries 2022-06-24 02:32:40 +00:00
Alexander Gololobov
a5b3f83181 Always remove row filter column 2022-06-22 17:23:09 +02:00
Alexander Gololobov
6a26325fab Test dirty hacks for multiple PREWHERE steps 2022-06-22 17:23:05 +02:00
mergify[bot]
f45b4f56d8
Merge branch 'master' into dictinct_in_order_optimization 2022-06-21 21:25:37 +00:00
Igor Nikonov
b0a98bd875 DISTINCT in order optimization 
+ use SortDescription from input data stream in DistinctStep to decide if the optimization is applicable
 2022-06-21 21:23:49 +00:00
Nikolai Kochetov
543782436d Process additional filters. 2022-06-21 11:24:46 +00:00
Nikolai Kochetov
b8d27aa8dd
Merge pull request #37469 from azat/projections-optimize_aggregation_in_order 
Implement in order aggregation (optimize_aggregation_in_order) for projections for tables with fully materialized projections
 2022-06-21 12:17:35 +02:00
Igor Nikonov
6ac68e8303 DISTINCT in order optimization 
+ optimization for DISTINCT containing primary key columns
 2022-06-20 10:06:15 +00:00
Vladimir Chebotarev
bfb67ded2c Rebase fix. 2022-06-20 05:15:08 +03:00
Vladimir Chebotarev
ec22f6d539 Draft. 2022-06-20 05:15:07 +03:00
vdimir
2a9942ee80
Add setting multiple_joins_try_to_keep_original_names 2022-06-16 15:50:03 +00:00
Azat Khuzhin
4694929623 Implement merging only for AggregatingStep 
v2: fill AggregateColumnsConstData only for only_merge
    (fixes 01291_aggregation_in_order and some other tests)
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-06-16 09:58:36 +03:00
Nikolai Kochetov
b537e91487 parse additional filetrs. 2022-06-13 12:18:45 +00:00
mergify[bot]
58c3f31bf4
Merge branch 'master' into bug-with-fill-date 2022-06-06 15:10:33 +00:00
Yakov Olkhovskiy
52ae3f0e92 full refactoring - move type check to transform 2022-06-05 22:39:21 -04:00
Yakov Olkhovskiy
07009c27c2 tidy build suggestions 2022-06-04 17:24:48 -04:00
Yakov Olkhovskiy
adf305e1dd search source header for columns 2022-06-04 11:44:20 -04:00
Yakov Olkhovskiy
0a063820c7 WITH FILL of date/time type must be equal to type of ORDER BY column 2022-06-04 01:35:41 -04:00
Nikolai Kochetov
8991f39412 Merge branch 'master' into refactor-read-metrics-and-callbacks 2022-06-02 17:00:08 +00:00
Nikolai Kochetov
85f4295370 Cleanup 2022-06-02 10:34:40 +00:00
Nikolai Kochetov
fc726c236f
Merge pull request #37661 from ClickHouse/no_alias_actions 
Trying to remove prewhere_info.alias_actions as they seem to always be included in prewhere_actions
 2022-06-02 12:31:04 +02:00
Nikita Mikhaylov
d34e051c69
Support for simultaneous read from local and remote parallel replica (#37204) 2022-06-02 11:46:33 +02:00
Nikolai Kochetov
edac3d6714 Merge branch 'master' into refactor-read-metrics-and-callbacks 2022-06-02 09:36:20 +00:00
Robert Schulze
ee302f2d9f
Merge pull request #37643 from amosbird/avoid-useless-context-copy 
Avoid useless context copy when building query interpreters
 2022-06-01 13:49:56 +02:00
Nikolai Kochetov
86fbb74703 Merge branch 'master' into refactor-read-metrics-and-callbacks 2022-05-31 18:07:47 +00:00
Nikolai Kochetov
147a819221 Refactor a little bit more. 2022-05-31 14:43:38 +00:00
Dmitry Novik
b41fe00f31
Merge pull request #37542 from azat/grouping-sets-fix-optimize_aggregation_in_order 
Prohibit optimize_aggregation_in_order with GROUPING SETS (fixes LOGICAL_ERROR)
 2022-05-31 15:31:45 +02:00
Nikolai Kochetov
77b07dd0a8
Merge pull request #37163 from ClickHouse/grouping-function 
Add GROUPING function
 2022-05-30 20:45:04 +02:00
Nikolai Kochetov
913e7a91ae Fix limits from subquery. 2022-05-30 18:25:17 +00:00
Alexander Gololobov
e2dd6f6249 Removed prewhere_info.alias_actions 2022-05-30 19:58:23 +02:00
Amos Bird
6525bfc4cd
Avoid context copy for InterpreterSelects 2022-05-30 17:08:12 +08:00
Nikolai Kochetov
b80b1940ce Fix some tests. 2022-05-27 20:47:35 +00:00
Azat Khuzhin
8a224239c1 Prohibit optimize_aggregation_in_order with GROUPING SETS 
AggregatingStep ignores it anyway, and it leads to the following error
in getSortDescriptionFromGroupBy(), like in [1]:

    2022.05.24 04:29:29.279431 [ 3395 ] {26543564-8bc8-4a3a-b984-70a2adf0245d} <Fatal> : Logical error: 'Trying to get name of not a column: ExpressionList'.

  [1]: https://s3.amazonaws.com/clickhouse-test-reports/36914/67d3ac72d26ab74d69f03c03422349d4faae9e19/stateless_tests__ubsan__actions_.html

v2: revert change to getSortDescriptionFromGroupBy() after
    GroupingSetsRewriterVisitor had been introduced
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-05-27 17:44:57 +03:00
Nikolai Kochetov
1b85f2c1d6 Merge branch 'master' into refactor-read-metrics-and-callbacks 2022-05-25 16:27:40 +02:00
Nikolai Kochetov
3d84aae0ab Better. 2022-05-24 20:06:08 +00:00
Nikolai Kochetov
9756b759c6 Move some resources 2022-05-23 13:46:57 +00:00
Antonio Andelic
87d445295f
Merge pull request #37342 from azat/projections-optimize_aggregation_in_order-fix 
Fix projections with GROUP/ORDER BY in query and optimize_aggregation_in_order
 2022-05-23 12:29:16 +02:00
Nikolai Kochetov
56feef01e7 Move some resources 2022-05-20 19:49:31 +00:00
Dmitry Novik
b3ccf96c81 Merge remote-tracking branch 'origin/master' into grouping-function 2022-05-19 17:58:33 +00:00
Dmitry Novik
d4c66f4a48 Code cleanup & fix GROUPING() with TOTALS 2022-05-19 16:36:51 +00:00
Dmitry Novik
86d48e1c99 Disable WITH ROLLUP/CUBE for GROUPING SETS 2022-05-19 14:10:04 +00:00
Azat Khuzhin
e32b695775 Fix projections with GROUP/ORDER BY in query and optimize_aggregation_in_order 
With projections, GROUP BY/ORDER BY in query, optimize_aggregation_in_order,
GROUP BY's InputOrderInfo was used incorrectly for ORDER BY.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-05-19 13:11:22 +03:00
Azat Khuzhin
dea1706d4e
Fix GROUP BY AggregateFunction (#37093) 
* Fix GROUP BY AggregateFunction

finalizeChunk() was unconditionally converting AggregateFunction to the
underlying type, however this should be done only if the aggregate was
applied.

So pass names of aggregates as an argument to the finalizeChunk()

Fuzzer report [1]:

    Logical error: 'Bad cast from type DB::ColumnArray to DB::ColumnAggregateFunction'. Received signal 6 Received signal Aborted (6)

For the following query:

    SELECT
        arraySort(groupArrayArray(grp_simple)),
        grp_aggreg,
        arraySort(groupArrayArray(grp_simple)),
        b,
        arraySort(groupArrayArray(grp_simple)) AS grs
    FROM data_02294
    GROUP BY
        a,
        grp_aggreg,
        b
    SETTINGS optimize_aggregation_in_order = 1

  [1]: https://s3.amazonaws.com/clickhouse-test-reports/37050/323ae98202d80fc4b311be1e7308ef2ac39e6063/fuzzer_astfuzzerdebug,actions//fuzzer.log

v2: fix conflicts in src/Interpreters/InterpreterSelectQuery.cpp
v3: Fix header for GROUP BY AggregateFunction WITH TOTALS
v4: Add sanity check into finalizeBlock()
v5: Use typeid_cast<&> to get more sensible error in case of bad cast (as suggested by @nickitat)
v6: Fix positions passed to finalizeChunk()
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>

* Core/ColumnNumbers.h: remove unused <string>

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>

* Optimize finalizeChunk()/finalizeBlock()

v2: s/ByPosition/Mask/ s/by_position/mask/
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-05-18 23:37:43 +02:00
Dmitry Novik
6fc7dfea80 Support ordinary GROUP BY 2022-05-13 23:04:12 +00:00
Nikolai Kochetov
0a715b26db Move some resources. 2022-05-13 20:02:28 +00:00
Dmitry Novik
ae81268d4d Try to compute helper column lazy 2022-05-13 14:55:50 +00:00
Dmitry Novik
c5b40a9c91 WIP on GROUPING function 2022-05-12 16:40:26 +00:00
Maksim Kita
4e7d10297b Fixed style 2022-05-11 21:59:51 +02:00