thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-15 02:41:59 +00:00
Code Issues Packages Projects Releases Wiki Activity
115,305 Commits 9,328 Branches 1,783 Tags 2.1 GiB
70b9077bb2
Commit Graph

912 Commits

Author SHA1 Message Date
MaxTheHuman
8b95b7d271 init commit with parsing and BAD realisation 2021-11-23 17:58:16 +03:00
Anton Popov
ccd78e3838 Merge remote-tracking branch 'upstream/master' into HEAD 2021-11-22 17:19:35 +03:00
Vitaly Baranov
33ea7a7262 Rename RowPolicy::ConditionType -> RowPolicyFilterType and move it to Access/Common. 2021-11-19 00:14:23 +03:00
Alexander Tokmakov
672157b817 Merge branch 'master' into mvcc_prototype 2021-11-11 22:01:34 +03:00
Anton Popov
a20922b2d3 Merge remote-tracking branch 'origin/sparse-serialization' into HEAD 2021-11-09 15:36:25 +03:00
kssenii
bfde7fd36c Better 2021-11-09 12:06:43 +00:00
kssenii
dead99011b Merge branch 'master' of github.com:ClickHouse/ClickHouse into intersect-except-fix 2021-11-08 16:15:33 +03:00
Alexander Tokmakov
92eec74ad7 Merge branch 'master' into mvcc_prototype 2021-11-06 21:08:36 +03:00
Kruglov Pavel
2f950ee138
Merge pull request #30927 from CurtizJ/with-fill-interval 
Support `INTERVAL` type in `STEP` clause for `WITH FILL` modifier
 2021-11-03 11:23:21 +03:00
Vitaly Baranov
3ed7f8f0b3 Move access-rights' source files needed for parser to a separate target. 2021-11-01 19:13:49 +03:00
kssenii
35349056b8 Fix 2021-11-01 16:33:19 +03:00
Anton Popov
aff2711f1a support INTERVAL type in WITH FILL 2021-10-31 19:42:43 +03:00
Amos Bird
211c3d4d49
Better Projection IN 2021-10-29 20:24:36 +08:00
Nikolai Kochetov
b549bddc11
Merge pull request #29735 from ClickHouse/fix-backward-compatibility-after-21196 
Fix backward compatibility after #21196.
 2021-10-26 16:31:40 +03:00
Anton Popov
cdfa36a181
Merge pull request #30266 from CurtizJ/aggregation-in-order-4 
Improve performance of aggregation in order of primary key
 2021-10-23 18:09:17 +03:00
Nikolai Kochetov
1533f4af57
Merge pull request #30335 from ClickHouse/single-sorting-step 
Single sorting step
 2021-10-22 11:34:29 +03:00
Amos Bird
2f615e9176
Use original_query for projection analysis 2021-10-20 21:30:15 +08:00
Anton Popov
82bff416f6 Merge remote-tracking branch 'upstream/master' into HEAD 2021-10-19 16:48:47 +03:00
Anton Popov
c547ffa313 improve performance of aggregation in order primary key with function with heavy states 2021-10-19 16:36:31 +03:00
mergify[bot]
69667aab1a
Merge branch 'master' into fix-backward-compatibility-after-21196 2021-10-19 10:11:16 +00:00
Nikolai Kochetov
34d46245c3 Try fix some tests. 2021-10-18 20:53:30 +03:00
Nikolai Kochetov
5a4a752cfb Unite sorting steps. 2021-10-18 17:37:46 +03:00
Nikolai Kochetov
6e479b301a Update memory optimisation for MergingSorted. 2021-10-18 12:54:12 +03:00
Nikolai Kochetov
a08c98d760 Move some files. 2021-10-16 17:03:50 +03:00
Nikolai Kochetov
fd14faeae2 Remove DataStreams folder. 2021-10-15 23:18:20 +03:00
Nikolai Kochetov
2957971ee3 Remove some last streams. 2021-10-13 21:22:02 +03:00
Nikolai Kochetov
06d9fc602e Merge branch 'master' into fix-backward-compatibility-after-21196 2021-10-12 14:41:23 +03:00
Nikolai Kochetov
340b53ef85 Remove some more streams. 2021-10-08 17:03:54 +03:00
Nikolai Kochetov
e2d07fbdf5
Merge branch 'master' into fix-backward-compatibility-after-21196 2021-10-04 17:35:32 +03:00
Nikolai Kochetov
6169b180cb Fix backward compatibility after #21196. 2021-10-04 17:27:44 +03:00
Alexey Milovidov
fe6b7c77c7 Rename "common" to "base" 2021-10-02 10:13:14 +03:00
Kruglov Pavel
14be2f31f5
Merge pull request #29475 from ClickHouse/fix-29010 
Remove filter column from HAVING when it is not needed.
 2021-09-29 13:03:27 +03:00
Nikolai Kochetov
1a4ee1f77f Remove filter column from HAVING when it is not needed. 2021-09-28 13:01:47 +03:00
Alexander Tokmakov
72b1b2e360 Merge branch 'master' into mvcc_prototype 2021-09-23 22:53:27 +03:00
Nikolai Kochetov
e616732743 Small refactoring. 2021-09-15 22:35:48 +03:00
Anton Popov
4c388e3d84 Merge remote-tracking branch 'origin/sparse-serialization' into HEAD 2021-09-09 14:10:16 +03:00
vdimir
e8e26463bf
Change signature of ASTSelectQuery::arrayJoinExpressionList 2021-08-31 15:27:44 +03:00
vdimir
3443b3b5c4
Separate functoin buildJoinedPlan, minor refactoring 2021-08-31 15:27:44 +03:00
Kseniia Sumarokova
19afb22b4d
Merge pull request #27932 from amosbird/constkeys 
Add a setting to bring back the old behavior of grouping by constant keys on empty set.
 2021-08-21 08:02:30 +03:00
Amos Bird
5708ed42ec
Add setting. 
empty_result_for_aggregation_by_constant_keys_on_empty_set
 2021-08-20 22:34:32 +08:00
Anton Popov
61239343e3 Merge remote-tracking branch 'origin/sparse-serialization' into HEAD 2021-08-20 16:33:30 +03:00
kssenii
8da7f406af Better comment 2021-08-17 11:03:01 +03:00
Kseniia Sumarokova
6001c7f214
Update InterpreterSelectQuery.cpp 2021-08-17 08:44:51 +03:00
kssenii
1ad4b6ed60 Fix 2021-08-16 23:34:39 +03:00
Raúl Marín
d1d47658f5 Merge remote-tracking branch 'blessed/master' into materialization_log 2021-08-10 10:49:05 +02:00
Azat Khuzhin
3be3c503aa Fix some comments 2021-08-08 09:58:07 +03:00
alexey-milovidov
c5207fc237
Merge pull request #26466 from azat/optimize-dist-select 
Rework SELECT from Distributed optimizations
 2021-08-08 03:59:32 +03:00
mergify[bot]
589505e76b
Merge branch 'master' into legacy-tuple-name 2021-08-07 16:42:20 +00:00
Azat Khuzhin
1358a2db92 Fix WithMergeableStateAfterAggregationAndLimit with LIMIT BY and LIMIT OFFSET 
In case of LIMIT BY, pre LIMIT cannot be applied (that is done on the
shard before), and hence shard applies regular LIMIT, but it goes with
applying OFFSET, and also the initiator will do apply OFFSET too.
 2021-08-06 00:16:26 +03:00
Azat Khuzhin
c4e2342a06 Add verbose description for pre-LIMIT into EXPLAIN query 2021-08-06 00:16:26 +03:00
Raúl Marín
479f053f2e Merge remote-tracking branch 'blessed/master' into materialization_log 2021-08-05 17:24:10 +02:00
Alexander Kuzmenkov
d852207b0e
Merge pull request #25721 from CurtizJ/read-in-order-perf-2 
Reduce memory usage in queries with `ORDER BY primary_key
 2021-08-05 16:13:48 +03:00
Anton Popov
915ef50c2d optimize reading in order of key more 2021-08-04 18:18:03 +03:00
Anton Popov
699a3d9031 implement legacy_column_name_of_tuple_literal in less intrusive way 2021-08-03 21:03:24 +03:00
Anton Popov
e36736b50c Merge remote-tracking branch 'origin/sparse-serialization' into HEAD 2021-08-02 22:52:02 +03:00
Azat Khuzhin
ff12f5102a Avoid running LIMIT BY/DISTINCT step on the initiator for optimize_distributed_group_by_sharding_key 
Before the following queries was running LimitBy/Distinct step on the
initator:

  select distinct sharding_key from dist order by k

While this can be omitted.
 2021-08-02 21:04:30 +03:00
Raúl Marín
9f32ecca89 Merge remote-tracking branch 'blessed/master' into materialization_log 2021-08-02 16:21:10 +02:00
Anton Popov
cc3ed12ed6 Merge remote-tracking branch 'upstream/master' into HEAD 2021-08-02 02:56:00 +03:00
Amos Bird
7b9e81742f
Shard level constness 2021-07-30 18:53:34 +08:00
Alexander Tokmakov
c74bfbf991 Merge branch 'master' into mvcc_prototype 2021-07-28 22:21:48 +03:00
Raúl Marín
f0125262b7 Add views and it's tables to system.query_log 2021-07-27 16:29:33 +02:00
Alexander Tokmakov
63ab38ee09 rename MaterializeMySQL to MaterializedMySQL 2021-07-26 21:17:28 +03:00
Anton Popov
f99374cca6 Merge remote-tracking branch 'origin/sparse-serialization' into HEAD 2021-07-20 18:20:21 +03:00
mergify[bot]
39f71f4a08
Merge branch 'master' into fix_no_column_materialized_mysql 2021-07-19 12:27:38 +00:00
Anton Popov
f61ea15d56 fix pushdown of limit to reading stage 2021-07-16 20:40:00 +03:00
Anton Popov
03c785931a fix pushdown of limit to reading stage 2021-07-16 17:27:38 +03:00
Anton Popov
c45a9e3bd6 Merge remote-tracking branch 'upstream/master' into HEAD 2021-07-16 17:11:34 +03:00
Raúl Marín
c2f0d7c514 Don't build set for indices when analyzing a query 2021-07-15 14:25:22 +02:00
Amos Bird
0558ecdc3f
Aggressive IN index analysis for projections. 2021-07-14 22:56:52 +08:00
Anton Popov
cb042afa9d pushdown limit while reading in order of primary key 2021-07-13 17:24:45 +03:00
mergify[bot]
d5898ecba6
Merge branch 'master' into fix_no_column_materialized_mysql 2021-07-13 14:03:05 +00:00
alexey-milovidov
4b1e38eed9
Merge branch 'master' into projection-fix11 2021-07-13 04:52:15 +03:00
Alexey Milovidov
e0ae04297a Fix fast test 2021-07-12 18:19:49 +03:00
Anton Popov
6ba41dc265 fix projections with storage snapshot 2021-07-12 16:45:35 +03:00
Anton Popov
3ed7f5a6cc dynamic subcolumns: add snapshot for storage 2021-07-09 06:15:41 +03:00
vdimir
d4256a8583
Minor style changes in JoinedTables 2021-07-08 13:49:13 +03:00
vdimir
5d7fc61bd5
Merge branch 'master' into join-materialized-columns 2021-07-08 12:30:00 +03:00
Anton Popov
072e65b728 Merge remote-tracking branch 'origin/sparse-serialization' into HEAD 2021-07-07 17:20:38 +03:00
mergify[bot]
446474932b
Merge branch 'master' into projection-fix11 2021-07-07 05:02:44 +00:00
Amos Bird
55981cb0ae
Fix normal projection 2021-07-07 13:01:30 +08:00
alexey-milovidov
b724138541
Merge pull request #25786 from kssenii/inconsisteny 
Fix inconsistent behaviour of GROUP BY constant on empty set
 2021-07-06 03:36:50 +03:00
Alexander Tokmakov
0f377fc4d8 Merge branch 'master' into fix_no_column_materialized_mysql 2021-07-05 22:07:56 +03:00
alexey-milovidov
fef7c557f6
Merge pull request #24789 from kitaisreal/compile-aggregation-functions 
Compile aggregate functions
 2021-07-05 02:57:49 +03:00
kssenii
77879f6826 Better 2021-07-04 01:53:51 +03:00
kssenii
a9a91a4f15 Setting move to prewhere if final 2021-07-03 11:37:50 +00:00
kssenii
02681019f8 Fix 2021-07-03 08:48:42 +00:00
Maksim Kita
da8c957167 Aggregator added CompiledExpressionCache 2021-07-01 22:56:36 +03:00
Maksim Kita
d24d3ae992 Added second variant of compilation 2021-07-01 22:56:36 +03:00
Maksim Kita
3fe559b31f Compile aggregate functions 2021-07-01 22:56:35 +03:00
Alexander Tokmakov
fd914f3d97 fix 'Not found column' error in MaterializeMySQL 2021-06-29 20:47:41 +03:00
Nikolai Kochetov
3544b059ce Merge branch 'master' into remove-PrewhereDAGInfo 2021-06-25 17:52:03 +03:00
Nikolai Kochetov
6bc0a628cd Remove PrewhereDAGInfo. 2021-06-25 17:49:28 +03:00
vdimir
2949cd1e6f
Support ALIASed columns in JOIN ON expression 2021-06-25 14:50:02 +03:00
vdimir
241b64d02c
Support ALIASed columns for right joined table 2021-06-24 17:57:21 +03:00
vdimir
84e02911cf
Materialized columns for joined table, don't add to asterisk select 2021-06-24 17:14:36 +03:00
kevin wan
711744afa1
Merge branch 'master' into exact-rows-before-limit 2021-06-23 22:06:03 +08:00
Anton Popov
c2754c34f4 Merge remote-tracking branch 'upstream/master' into HEAD 2021-06-21 12:42:48 +03:00
Anton Popov
d8b6f15ef4
Merge pull request #23027 from azat/distributed-push-down-limit 
Add ability to push down LIMIT for distributed queries
 2021-06-20 23:08:50 +03:00
Nikolai Kochetov
b2620e1def
Merge pull request #24405 from amosbird/replacemorealias 
Rewrite more alias columns
 2021-06-18 09:10:31 +03:00
Amos Bird
0f7728cd5e
Fix RBAC test again 2021-06-17 23:00:03 +08:00
Maksim Kita
67e9b85951 Merge ext into common 2021-06-16 23:28:41 +03:00
Anton Popov
ea9b890618
Merge branch 'master' into add-compatibility-setting 2021-06-16 21:10:56 +03:00
Anton Popov
8bd7cbd925 add compatibility setting for name of columns 2021-06-16 21:10:07 +03:00
万康
13be98f336 add setting exact_rows_before_limit 2021-06-16 20:59:02 +08:00
mergify[bot]
7959d92029
Merge branch 'master' into minor-changes-3 2021-06-15 18:07:24 +00:00
Nikita Mikhaylov
a52bba91b7
Merge pull request #16401 from abyss7/ast-table-identifier-2 
ASTTableIdentifier part #2: Introduce ASTTableIdentifier
 2021-06-15 13:51:30 +03:00
Amos Bird
aaac231c6a
Record expanded aliases for RBAC 2021-06-15 18:28:13 +08:00
Amos Bird
e3ae2f6e7a
Rewrite more alias columns 2021-06-15 17:54:28 +08:00
Alexey Milovidov
447d7bb8cd Minor changes 2021-06-14 07:13:35 +03:00
Alexander Tokmakov
6f5cf773ef Merge branch 'master' into mvcc_prototype 2021-06-09 12:50:54 +03:00
Azat Khuzhin
18e8f0eb5e Add ability to push down LIMIT for distributed queries 
This way the remote nodes will not need to send all the rows, so this
will decrease network io and also this will make queries w/
optimize_aggregation_in_order=1/LIMIT X and w/o ORDER BY faster since it
initiator will not need to read all the rows, only first X (but note
that for this you need to your data to be sharded correctly or you may
get inaccurate results).

Note, that having lots of processing stages will increase the complexity
of interpreter (it is already not that clean and simple right now).

Although using separate QueryProcessingStage looks pretty natural.

Another option is to make WithMergeableStateAfterAggregation always, but
in this case you will not be able to disable only this optimization,
i.e. if there will be some issue with it.

v2: fix OFFSET
v3: convert 01814_distributed_push_down_limit test to .sh and add retries
v4: add test with OFFSET
v5: add new query stage into the bash completion
v6/tests: use LIMIT O,L syntax over LIMIT L OFFSET O since it is broken in ANTLR parser
          https://clickhouse-test-reports.s3.yandex.net/23027/a18a06399b7aeacba7c50b5d1e981ada5df19745/functional_stateless_tests_(antlr_debug).html#fail1
v7/tests: set use_hedged_requests to 0, to avoid excessive log entries on retries
          https://clickhouse-test-reports.s3.yandex.net/23027/a18a06399b7aeacba7c50b5d1e981ada5df19745/functional_stateless_tests_flaky_check_(address).html#fail1
 2021-06-09 02:29:50 +03:00
Ivan
579250168f
Merge branch 'master' into ast-table-identifier-2 2021-06-07 20:30:39 +03:00
Anton Popov
205a23282b Merge remote-tracking branch 'origin/sparse-serialization' into HEAD 2021-06-07 19:38:22 +03:00
Alexey Milovidov
885ce194e0 Making fundamentals correct 2021-06-07 00:49:55 +03:00
Amos Bird
9dfa4c06bd
Try reusing built sets during multi-pass analysis 2021-06-03 22:06:05 +08:00
Ivan Lezhankin
ae38b4a3f1 Merge remote-tracking branch 'upstream/master' into ast-table-identifier-2 2021-06-02 16:07:59 +03:00
Anton Popov
9f52362b81 Merge remote-tracking branch 'origin/sparse-serialization' into HEAD 2021-05-31 23:08:45 +03:00
Alexander Tokmakov
5692db736c Merge branch 'master' into mvcc_prototype 2021-05-31 21:11:22 +03:00
Alexander Kuzmenkov
6cccb0992a fix ORDER BY after window fuctions over Distributed 2021-05-27 19:06:37 +03:00
Anton Popov
0bdf9d207c Merge remote-tracking branch 'origin/sparse-serialization' into HEAD 2021-05-26 00:25:47 +03:00
mergify[bot]
970243850f
Merge branch 'master' into ast-table-identifier-2 2021-05-21 12:56:49 +00:00
Maksim Kita
b0476c1fa2
Merge pull request #24273 from kitaisreal/expression-actions-compile-only-necessary-places 
ExpressionActions compile only necessary places
 2021-05-20 10:53:30 +03:00
Maksim Kita
c50909468f Updated ExpressionActionSettings 2021-05-19 17:32:07 +03:00
Alexander Kuzmenkov
e9b69bbd70
Merge pull request #23906 from azat/fix-distributed_group_by_no_merge 
distributed_group_by_no_merge fixes
 2021-05-19 16:16:08 +03:00
Maksim Kita
150a88d647 ExpressionActions compile only necessary places 2021-05-19 11:43:16 +03:00
Ivan Lezhankin
bd16dd22b9 Merge remote-tracking branch 'upstream/master' into ast-table-identifier-2 2021-05-14 15:40:33 +03:00
Alexander Tokmakov
df2849be90 Merge branch 'master' into mvcc_prototype 2021-05-13 21:48:36 +03:00
Anton Popov
37989cd2ae fix reading of nested 2021-05-11 15:01:41 +03:00
Amos Bird
5836e2e277
Fix issues in normal projection 2021-05-11 18:12:29 +08:00
Amos Bird
dfa5629071
Remove projection type ast 2021-05-11 18:12:29 +08:00
Amos Bird
51d685bb9c
Even more fixes 2021-05-11 18:12:29 +08:00
Amos Bird
718c284437
Fix more tests 2021-05-11 18:12:28 +08:00
Amos Bird
ba17acbd63
Fix tests 2021-05-11 18:12:28 +08:00
Amos Bird
42f161b1dc
Fix more 2021-05-11 18:12:28 +08:00
Amos Bird
9c069ebdbf
support prewhere, row_filter, read_in_order and decent projection selection 
TODO set index analysis in projection
 2021-05-11 18:12:27 +08:00
Amos Bird
35961c0c5d
Use ActionsDAG to rewrite projection queries 2021-05-11 18:12:27 +08:00
Nikolai Kochetov
29d7038e30
Add ProjectionDescription::Type enum. Remove some checks. 2021-05-11 18:12:27 +08:00
Amos Bird
ebaf42a448
Reformat and fix some tests 2021-05-11 18:12:27 +08:00
Nikolai Kochetov
c2396c5144
Disable read-in-order optimization in case of projections. 2021-05-11 18:12:26 +08:00
Nikolai Kochetov
3d36fb57eb
Cleanup code. 2021-05-11 18:12:26 +08:00
Nikolai Kochetov
3296c9292f
Try to merge projectons faster. 2021-05-11 18:12:26 +08:00
Amos Bird
cd6414639e
add metadata_snapshot to getQueryProcessingStage 2021-05-11 18:12:26 +08:00
Amos Bird
264cff6415
Projections 
TODO (suggested by Nikolai)

1. Build query plan fro current query (inside storage::read) up to WithMergableState
2. Check, that plan is simple enough: Aggregating - Expression - Filter - ReadFromStorage (or simplier)
3. Check, that filter is the same as filter in projection, and also expression calculates the same aggregation keys as in projection
4. Return WithMergableState if projection applies

3 will be easier to do with ActionsDAG, cause it sees all functions, and dependencies are direct (but it is possible with ExpressionActions also)

Also need to figure out how prewhere works for projections, and
row_filter_policies.

wip
 2021-05-11 18:12:23 +08:00
Azat Khuzhin
eefd67fce5 Disable optimize_distributed_group_by_sharding_key with window functions 2021-05-06 00:44:22 +03:00
Azat Khuzhin
8703b7863d Fix distributed_group_by_no_merge=2 with GROUP BY 
Before this patch the following query:

  SELECT assumeNotNull(argMax(dummy, 1))
  FROM remote('127.1', system.one)
  SETTINGS distributed_group_by_no_merge = 2

Leads to:

  Code: 10. DB::Exception: Received from localhost:9000. DB::Exception: Not found column argMax(dummy, 1) in block: while executing 'INPUT : 0 -> argMax(dummy, 1) UInt8 : 0'.

Since it tries to execute function one more time, but shards will not
send this column when the query processed with
distributed_group_by_no_merge=2 (i.e. up to
WithMergeableStateAfterAggregation).

v0: no exception
v2: execut window functions
v3: throw exception, since executing window function in this case will
lead to messy output
 2021-05-06 00:21:47 +03:00
Anton Popov
e44706911e dynamic columns: better getting of sample block 2021-05-05 02:02:54 +03:00
Nikolai Kochetov
25fe5f63ee Fix other tests. 2021-04-29 12:08:49 +03:00
Nikolai Kochetov
69ddca1d73 Refactor join step, part 3 2021-04-28 20:32:12 +03:00
Anton Popov
644df6be7d dynamic subcolumns: wip 2021-04-24 07:09:01 +03:00
Anton Popov
6240169bbb dynamic subcolumns: wip 2021-04-24 02:56:26 +03:00
Alexander Kuzmenkov
ab75f86ee9 add a comment 2021-04-23 18:45:45 +03:00
Ivan Lezhankin
cdd825b18b Merge remote-tracking branch 'upstream/master' into ast-table-identifier-2 2021-04-23 18:08:25 +03:00
Alexander Kuzmenkov
6982ab5324 fix extra distinct 2021-04-23 15:44:46 +03:00
Alexander Kuzmenkov
9d95ac3d66 remove debug 2021-04-23 14:00:17 +03:00
Alexander Kuzmenkov
54ad086753 fix for distributed 2021-04-23 13:59:21 +03:00
Azat Khuzhin
86069ffb40 Fix LOGICAL_ERROR for max_distributed_connections=0 2021-04-20 09:05:44 +03:00
Ivan
a73483751f
Merge branch 'master' into ast-table-identifier-2 2021-04-12 20:23:28 +03:00
Ivan
495c6e03aa
Replace all Context references with std::weak_ptr (#22297) 
* Replace all Context references with std::weak_ptr

* Fix shared context captured by value

* Fix build

* Fix Context with named sessions

* Fix copy context

* Fix gcc build

* Merge with master and fix build

* Fix gcc-9 build
 2021-04-11 02:33:54 +03:00
Ivan Lezhankin
3da3250bfd Merge branch 'master' into ast-table-identifier-2 2021-04-09 17:56:15 +03:00
Nikolai Kochetov
28ca191102
Merge pull request #22087 from ClickHouse/better-filter-push-down 
Better filter push down
 2021-04-09 10:22:17 +03:00
Alexander Tokmakov
3422bd1742 check parts visibility for select 2021-04-08 20:20:45 +03:00
Alexander Kuzmenkov
06bb58cb69
Merge pull request #22704 from ClickHouse/aku/window-multiple 
fix window functions with multiple input streams and no sorting
 2021-04-07 15:15:47 +03:00
Azat Khuzhin
f157278b72 Safer SCOPE_EXIT 
It executes the code in the dtor, that should never throw.
 2021-04-06 21:10:37 +03:00
Alexander Kuzmenkov
585da2c8d0 Merge remote-tracking branch 'origin/master' into HEAD 2021-04-06 14:58:21 +03:00
Alexander Kuzmenkov
06e7a9f8f2 fix window functions with multiple input streams and no sorting 2021-04-06 14:43:27 +03:00
Nikolai Kochetov
6102652c99 Merge branch 'master' into better-filter-push-down 2021-04-06 13:38:03 +03:00
Anton Popov
f58d470dd6
Merge pull request #21830 from foolchi/optimize_select_with_final 
optimize select final with prewhere primary key
 2021-03-25 15:32:13 +03:00
Nikolai Kochetov
0afede382b Use empty header for CreatingSets step. 2021-03-25 11:39:55 +03:00
Nikolai Kochetov
12de01cb48 Remove AddingDelayedSourceStep. 2021-03-24 16:22:51 +03:00
Nikolai Kochetov
730785f4c2
Merge pull request #21459 from ClickHouse/refactor-actions-dag 
Refactor actions dag
 2021-03-22 11:58:16 +03:00
Alexander Kuzmenkov
1cd9f28bd4 another way 2021-03-17 18:31:30 +03:00
fuqi
5e20ea2c33 optimize select final with prewhere primary key 2021-03-17 18:49:24 +08:00
Nikolai Kochetov
4f133ae036 Merge branch 'master' into refactor-actions-dag 2021-03-10 12:40:55 +03:00
Nikolai Kochetov
5ef9d87790 Fix some other tests. 2021-03-10 11:41:24 +03:00
vdimir
36006a7e96
Merge branch 'master' into cross-to-inner-join-rewrite 2021-03-09 09:36:15 +03:00
Vladimir
6ec19b2632
Merge pull request #19885 from ClickHouse/vdimir/join-cast-types-v2 2021-03-09 09:30:50 +03:00
vdimir
354757d94d
Add logging for need_analyze_again in InterpreterSelectQuery 2021-03-08 09:33:30 +03:00
Nikolai Kochetov
a669f7d641 Merge branch 'master' into refactor-actions-dag 2021-03-05 18:21:14 +03:00
vdimir
4a8708fbb2
Clean subqueries_for_sets before second analyze in InterpreterSelectQuery 2021-03-05 17:43:26 +03:00
vdimir
b0f6f30298
Revert "Refactor join: make IJoin implementations independent from TableJoin" 
This reverts commit e6406c3f4c.
 2021-03-05 17:34:43 +03:00
vdimir
e6406c3f4c
Refactor join: make IJoin implementations independent from TableJoin 2021-03-05 16:38:49 +03:00
Nikolai Kochetov
9a39459888 Refactor ActionsDAG 2021-03-04 20:38:12 +03:00
Nikolai Kochetov
15921fbfcb Remove index by name from ActionsDAG 2021-03-03 23:01:07 +03:00
Nikolai Kochetov
ffbea93b10
Merge pull request #19576 from traceon/row-policy-with-prewhere 
Allow row policies with PREWHERE
 2021-03-02 13:51:37 +03:00
Denis Glazachev
eb80c189e4 Merge branch 'master' into row-policy-with-prewhere 2021-02-28 20:14:31 +04:00
Nikolai Kochetov
d91b8a3acb
Merge branch 'master' into filter-push-down 2021-02-26 19:33:27 +03:00
Nikolai Kochetov
d328bfa41f Review fixes. Add setting max_optimizations_to_apply. 2021-02-26 19:29:56 +03:00
vdimir
4c6cde4d53
Merge remote-tracking branch 'upstream/master' into join-cast-types-v2 2021-02-25 16:24:28 +03:00
Alexey Milovidov
0ab14120ef Improve performance of trivial count query in presense of "distributed_aggregation_memory_efficient" 2021-02-21 23:06:31 +03:00
Denis Glazachev
c1158b50f4 Merge branch 'master' into row-policy-with-prewhere 
* master: (155 commits)
  Update version_date.tsv after release 20.8.13.15
  Update version_date.tsv after release 20.12.7.3
  Update version_date.tsv after release 21.1.5.4
  Update version_date.tsv after release 21.2.4.6
  fix
  Add test to skip list
  Fix WriteBufferFromHTTPServerResponse usage in other places (add missing finalize())
  Fix WriteBufferFromHTTPServerResponse usage in odbc-bridge
  Update config.xml
  Suppress signed overflow in AggregateFunctionGroupArrayMoving 2
  Update BaseDaemon.cpp
  review suggestions
  Fix bash syntax in 01731_async_task_queue_wait
  Do not use view() in 01731_async_task_queue_wait to fix ANTLR parser
  Increase buffer for uncaught exception / std::terminate
  Even more better
  Fix uncaught exception when HTTP client goes away
  test for decimal ( p , s) in dictionaries
  Just little better
  Fixed style check
  ...
 2021-02-21 13:25:47 +04:00
Denis Glazachev
dfde9de0e9 Remove commented-out code 2021-02-20 23:30:21 +04:00
Nikolai Kochetov
8a876b9510 Fix aliases for row level actions. 2021-02-20 17:19:11 +03:00
Nikolai Kochetov
673e24d7ef Refactor 2021-02-20 14:00:16 +03:00
Denis Glazachev
8cc72b8fe0 Merge branch 'master' into row-policy-with-prewhere 
* master: (160 commits)
  Make Poco HTTP Server zero-copy again (#19516)
  Fixed documentation
  ccache 4.2+ does not requires any quirks for SOURCE_DATE_EPOCH
  Add a function `htmlOrXmlCoarseParse` to extract content from html or xml format string. (#19600)
  Reinterpret function added Decimal, DateTim64 support
  Add test
  Update InterpreterSelectQuery.cpp
  Improved serialization for data types combined of Arrays and Tuples. Improved matching enum data types to protobuf enum type. Fixed serialization of the Map data type. Omitted values are now set by default.
  Log stdout and stderr when failed to start docker in integration tests.
  Added comment
  Don't backport base commit of branch in the same branch (#20628)
  Fix fasttest retry for failed tests
  Dictionary create source with functions crash fix
  Added error reinterpretation tests
  Update run.sh
  Updated documentation
  fix subquery with limit
  Rename untyped function reinterpretAs into reinterpret
  ignore data store files
  Support vhost
  ...
 2021-02-19 19:49:37 +04:00
Denis Glazachev
5e5b3b80ce Remove debug printouts 2021-02-19 19:46:52 +04:00
vdimir
435f63f42b
Calculate common type for join using in TreeRewriter 
Split before_join and converting_join_columns dags

Add more detailed comments for column type conversion for join using
 2021-02-19 14:28:32 +03:00
vdimir
4c36cd1737
Add converting step for 'join using' 2021-02-19 14:24:28 +03:00
Azat Khuzhin
af660140c3 Do only merging of sorted blocks on initiator with distributed_group_by_no_merge=2 
When distributed_group_by_no_merge=2 is used (or when
optimize_distributed_group_by_sharding_key takes place), remote servers
will do full ORDER BY, so initiator can skip this step and do only merge
of ordered blocks.
 2021-02-18 21:43:27 +03:00
Azat Khuzhin
b854a7b7f8 Add some details into comment for first_stage/second_stage 
Regardless
distributed_group_by_no_merge=2/optimize_distributed_group_by_sharding_key
 2021-02-18 21:41:50 +03:00
alexey-milovidov
3891dd6284
Update InterpreterSelectQuery.cpp 2021-02-17 21:23:27 +03:00
feng lv
3b40099578 fix subquery with limit 2021-02-17 08:26:52 +00:00
Nikolai Kochetov
bcf30d8412 Try fix tests. 2021-02-16 15:57:00 +03:00
Nikolai Kochetov
a14b6c2650 Fix trivial count optimization 2021-02-16 12:37:19 +03:00
Nikolai Kochetov
f139ad8080 Comment debug output. 2021-02-16 09:15:12 +03:00
Nikolai Kochetov
ae73600fb0 Refactor row level security actions. 2021-02-15 22:48:06 +03:00
vdimir
9afa6b5b1b
Add option cross_to_inner_join_rewrite 2021-02-15 22:40:32 +03:00
Denis Glazachev
45e90961f7 Store and process alias_actions in FilterInfo 2021-02-14 21:16:40 +04:00
Denis Glazachev
99a04b6c64 Merge branch 'master' into row-policy-with-prewhere 
* master: (759 commits)
  Suppress UBSan report in Decimal comparison
  Suppress UBSan report in Decimal comparison
  Fix UBSan report in arrayDifference
  Update README.md
  Non significant change in AggregationCommon
  Print stack trace on SIGTRAP
  Fix dependent test
  Fix tests for better parallel run
  Add test for already working code
  Revert "Fix access control manager destruction order"
  Update index.md
  Update index.md
  Update index.md
  Bit more complicated example for isIPv4String - ru
  Bit more complicated example for isIPv4String
  cleanup
  Replace database with ordinary
  Added comments
  Split tests to make them stable
  Fixes
  ...

# Conflicts:
#	src/Storages/MergeTree/MergeTreeRangeReader.cpp
 2021-02-14 02:24:00 +04:00
Denis Glazachev
79592b73f8 Store filter info in prewhere info instead of multiple prewheres 
Some cleanups
 2021-02-14 02:07:13 +04:00
Alexander Kuzmenkov
363007b964 fixes 2021-02-11 11:39:39 +03:00
Alexander Kuzmenkov
38678ff484 Merge remote-tracking branch 'origin/master' into HEAD 2021-02-11 10:40:05 +03:00
Alexander Kuzmenkov
e557d60cdb fixes for sort order 2021-02-10 16:27:22 +03:00
Alexander Kuzmenkov
b57b3244f5 reuse sort order 2021-02-10 15:44:07 +03:00
Amos Bird
46f2b4063f
Fix CTE in INSERT-SELECT 2021-02-09 00:25:24 +08:00
alexey-milovidov
44b44c1fe7
Update InterpreterSelectQuery.cpp 2021-02-07 01:09:55 +03:00
Maxim Akhmedov
0cd3601928 Allow using MergeTreeWhereOptimizer not only with MergeTree-based storages 2021-02-06 18:08:42 +03:00
Nikolai Kochetov
afdc9635cb Update MergeTreeRangeReader 2021-01-29 19:12:53 +03:00
Denis Glazachev
009d1e3fc5 Merge branch 'master' into row-policy-with-prewhere 
* master: (247 commits)
  Update 01014_lazy_database_basic.sh
  Update 00459_group_array_insert_at.sql
  Same for ALTER
  Syntax and links
  Added test.
  Check where and prewhere identifiers exist.
  Remove redundant lsof
  Add lsof to fasttest
  Add new line to status
  Make integration odbc tests idempotent
  Minor code improvement in JOIN
  Arcadia does not have bitmaps
  Make Fuzzer more reliable
  Add missing lsof for fasttest docker image
  Update test
  The test most likely would not work in Arcadia
  Less parser depth
  Fix stack overflow in coroutine
  Fuzzer: better messages.
  Fixed build.
  ...
 2021-01-28 21:48:12 +04:00
Denis Glazachev
6a92295982 Remove TODOs 2021-01-27 19:16:31 +04:00
Amos Bird
a2ceca3fa1
Fix recursive propagation 2021-01-26 10:29:12 +08:00
Denis Glazachev
4870e0af69 Add filter as a (first) prewhere 2021-01-25 22:09:17 +04:00
Denis Glazachev
4f6c880232 Pass and handle a chain of multiple prewhere infos 2021-01-25 18:31:59 +04:00
alexey-milovidov
d2366b5c6c
Merge pull request #18233 from vitlibar/fix-access-rights-for-select-count 
Fix access rights for SELECT count()
 2021-01-17 00:12:08 +03:00
alexey-milovidov
bf522d0eea
Update InterpreterSelectQuery.cpp 2021-01-17 00:11:58 +03:00
sundy-li
ad01534bd5 Merge branch 'master' into alias_column_partition_prune 2021-01-12 18:28:30 +08:00
Nikolai Kochetov
f955ccecf4 Do not materialize block for FetchColumns header. 2021-01-05 21:04:25 +03:00
sundyli
f8815f233c
Merge branch 'master' into alias_column_partition_prune 2021-01-02 11:23:55 +08:00
Vitaly Baranov
235ea220ec SELECT count() FROM table now requires at least one column to be accessible in the table. 2020-12-30 15:00:08 +03:00
Vitaly Baranov
460006b502
Merge pull request #18232 from vitlibar/fix-access-rights-for-select-join 
Fix access rights for SELECT JOIN.
 2020-12-30 14:42:16 +03:00
Ivan Lezhankin
5e89145a42 Merge remote-tracking branch 'upstream/master' into ast-table-identifier-2 2020-12-30 13:04:17 +03:00
Alexander Kuzmenkov
d4e5d5fdb4
Merge pull request #18455 from ClickHouse/aku/window-prototype 
[wip] some window function fixes
 2020-12-29 19:23:19 +03:00
hexiaoting
4adbdf364e Fix bug for orderby withfill with limit clause 2020-12-28 13:05:16 +03:00
Vitaly Baranov
9271b3d627 SELECT JOIN now requires the SELECT privilege on each of the joined tables. 2020-12-26 18:35:32 +03:00
Alexander Kuzmenkov
5bd025a180 fix DISTINCT and ORDER BY 2020-12-25 06:13:30 +03:00
Alexander Kuzmenkov
23d77a1698 remove debug output 2020-12-24 09:59:38 +03:00
Alexander Kuzmenkov
844239b1c3 tmp fixes (split me) 2020-12-24 07:03:33 +03:00
Alexander Kuzmenkov
f2b25c708c review fixes 2020-12-22 22:01:15 +03:00
Alexander Kuzmenkov
3f33e6b3e0 some review fixes 2020-12-22 20:46:31 +03:00
Alexander Kuzmenkov
52ace7a3d1 one plan step per window 2020-12-22 04:37:45 +03:00
Alexander Kuzmenkov
77b816caea cleanup 2020-12-22 02:47:14 +03:00
Alexander Kuzmenkov
85c1bc1253 Merge remote-tracking branch 'origin/master' into tmp 2020-12-21 10:46:21 +03:00
Amos Bird
9924cad96c
identify internal queries 2020-12-18 14:54:38 +08:00
Amos Bird
2f92d57af9
Better code. 2020-12-18 14:13:58 +08:00
Amos Bird
272311f6bd
Extensive information in system.query_log 2020-12-18 14:13:58 +08:00
Alexander Kuzmenkov
a1326414ff Merge remote-tracking branch 'origin/master' into tmp 2020-12-17 00:46:14 +03:00
Ivan Lezhankin
2fe6195577 Merge remote-tracking branch 'upstream/master' into ast-table-identifier-2 2020-12-16 14:50:36 +03:00
sundy-li
5c9f84b2f4 Merge branch 'master' into alias_column_partition_prune 2020-12-16 18:44:00 +08:00
Alexander Kuzmenkov
fabb37d3c9 cleanup 2020-12-15 20:39:58 +03:00
Alexander Kuzmenkov
a23a9fd947 fixes after merge 2020-12-15 20:16:01 +03:00
Nikita Mikhaylov
31bef163ba
Merge pull request #17848 from azat/max_distributed_connections-fix 
Fix max_distributed_connections
 2020-12-15 17:34:38 +03:00
sundy-li
cf087002db Merge branch 'master' into alias_column_partition_prune 2020-12-15 16:37:00 +08:00
Alexander Kuzmenkov
a75ca3cb79 Merge remote-tracking branch 'origin/master' into tmp 2020-12-15 03:36:43 +03:00
Alexander Kuzmenkov
6d5b23de67 something works 2020-12-15 03:36:03 +03:00
sundy-li
d2209a1460 trivial fix 2020-12-13 17:33:02 +08:00
alexey-milovidov
bce02de691
Merge pull request #17539 from azat/remerge_sort_lowered_memory_bytes_ratio 
Add remerge_sort_lowered_memory_bytes_ratio setting
 2020-12-13 04:38:50 +03:00
sundyli
36ca21c24a
Merge branch 'master' into alias_column_partition_prune 2020-12-13 00:45:14 +08:00
sundy-li
9407028540 update 2020-12-13 00:42:15 +08:00
Alexander Kuzmenkov
d757f32300 Remove some redundant includes to speed up build 2020-12-11 02:58:11 +03:00
Alexander Kuzmenkov
dea446be2f Remove some redundant includes to speed up build 2020-12-11 02:56:57 +03:00
Alexander Kuzmenkov
ac8b896f36 returning constant column as window function 2020-12-10 22:06:52 +03:00
Azat Khuzhin
c3a4e5e7b7 Fix max_distributed_connections (one more time) 
With prefer_localhost_replica=1 max_distributed_connections uses
max_threads before this patch.

Fixes: #14936
Previous fix: #9673
 2020-12-07 01:19:53 +03:00
Azat Khuzhin
ccd794a676 Add remerge_sort_lowered_memory_bytes_ratio setting 
Sometimes default ratio is not good enough (2), since it depends on lots
of factors:
- size of LIMIT
- max_memory_usage
- ...

So add a separate setting for it.

But note that it make sense to set
remerge_sort_lowered_memory_bytes_ratio only in range (1, 2]

Since <= 1 will just use more CPU, and > 2 will use more RAM and will
unlikely be faster.
 2020-11-29 21:16:42 +03:00
Nikita Mikhaylov
084c75fa6e
Merge pull request #17309 from nikitamikhaylov/merging-sequential-consistency 
Merging #16309
 2020-11-26 15:26:05 +03:00
nikitamikhaylov
72c7cd6693 replace Context& to Settings& 2020-11-25 16:47:32 +03:00
Alexander Kuzmenkov
1d7dcef4a0 Merge remote-tracking branch 'origin/master' into HEAD 2020-11-24 21:45:10 +03:00
Alexander Kuzmenkov
412c2fe079 Merge remote-tracking branch 'origin/master' into HEAD 2020-11-23 19:51:18 +03:00
nikitamikhaylov
68bef22fda Merge branch 'master' of github.com:ClickHouse/ClickHouse into merging-sequential-consistency 2020-11-23 16:28:35 +03:00
Ivan Lezhankin
1002148886 Merge remote-tracking branch 'upstream/master' into ast-table-identifier-2 2020-11-22 22:34:36 +03:00
sundy-li
fe8b11fd5f add replaceAliasColumnsInFilter function 2020-11-22 12:16:27 +08:00
sundy-li
ad99dab3ad add ColumnAliasesVisitor to improve partition prune for mergetree 2020-11-21 20:38:21 +08:00
Alexander Kuzmenkov
6cb378e072 cleanup 2020-11-19 18:52:11 +03:00
Nikolai Kochetov
13eceff55c Remove Converting transform and step. 2020-11-17 20:16:55 +03:00
Ivan Lezhankin
9250d5c4e9 Merge remote-tracking branch 'upstream/master' into ast-table-identifier-2 2020-11-12 16:25:46 +03:00
Nikolai Kochetov
9b13d0e0eb Revert "Add ActionsDAG::updateHeader()" 
This reverts commit b78657656c.
 2020-11-11 20:17:26 +03:00
Nikolai Kochetov
58c4a7b890 Some renames. 2020-11-11 19:52:27 +03:00
Nikolai Kochetov
b78657656c Add ActionsDAG::updateHeader() 2020-11-11 16:50:29 +03:00
Nikolai Kochetov
1846bb3cac Merge branch 'master' into actions-dag-f14 2020-11-11 13:08:57 +03:00
feng lv
fec4080d42 Merge branch 'master' of github.com:ClickHouse/ClickHouse into union-distinct 2020-11-11 03:10:10 +00:00
Nikolai Kochetov
c6575c9032 Update ExpressionActions constructur 2020-11-10 19:27:55 +03:00
Nikolai Kochetov
195c941c4e Merge branch 'master' into storage-read-query-plan 2020-11-10 15:02:22 +03:00
feng lv
9194985d48 Merge branch 'master' of github.com:ClickHouse/ClickHouse into union-distinct 2020-11-10 06:09:39 +00:00
Nikolai Kochetov
6717c7a0af Merge branch 'master' into actions-dag-f14 2020-11-09 14:57:48 +03:00
Alexey Milovidov
5314185e25 Merge branch 'master' into azat-optimize_skip_unused_shards-optimization 2020-11-08 00:17:59 +03:00
feng lv
1ca7dcb0a8 fix conflict 2020-11-07 11:44:29 +00:00
Nikolai Kochetov
c10f733587 Merge branch 'master' into storage-read-query-plan 2020-11-06 15:43:46 +03:00
Nikolai Kochetov
9aeb757da4 Merge branch 'master' into actions-dag-f14 2020-11-06 15:04:20 +03:00
Nikolai Kochetov
40a9648269
Merge pull request #16308 from nikitamikhaylov/totals-having 
Incorrect behaviour of SELECT WITH TOTALS
 2020-11-05 11:27:28 +03:00
Alexander Kuzmenkov
faf1f1e0cd Merge remote-tracking branch 'origin/master' into HEAD 2020-11-05 11:09:11 +03:00
Nikolai Kochetov
7b61f5d641 Refactor more 2020-11-03 22:05:47 +03:00
Azat Khuzhin
e7f15dff55 Do not pass StoragePtr to ReadInOrderOptimizer::getInputOrder() 
Looks like this is not required anymore, since #11745
 2020-11-03 21:22:46 +03:00
Nikolai Kochetov
07a7c46b89 Refactor ExpressionActions [Part 3] 2020-11-03 14:28:28 +03:00
Nikolai Kochetov
ceb83602b8 Refactor 2020-11-02 15:07:01 +03:00
feng lv
e82ce52e2f fix 2020-11-01 14:57:41 +00:00
feng lv
382fff9009 fix 2020-11-01 13:54:07 +00:00
Ivan Lezhankin
4083406f54 Introduce ASTTableIdentifier 2020-10-26 18:49:00 +03:00
hchen9
feaab260d8 Fix conflict 2020-10-23 12:54:33 -07:00
nikitamikhaylov
d908cddb49 done 2020-10-23 22:09:02 +03:00
Nikolai Kochetov
c91b453630 Try fix tests. 2020-10-22 14:08:12 +03:00
Nikolai Kochetov
7fa045cff8 Merge branch 'master' into storage-read-query-plan 2020-10-22 13:31:10 +03:00
alexey-milovidov
adeba6bdd8
Merge pull request #15074 from amosbird/btc 
Extend trivial count optimization.
 2020-10-22 02:50:57 +03:00
alexey-milovidov
be7776608d
Merge pull request #15933 from ClickHouse/fix-global-in-result 
Fix wrong result in case of GLOBAL IN and PREWHERE
 2020-10-19 21:21:57 +03:00
Alexander Tokmakov
7d58992ea8 fix select count() 2020-10-14 16:15:15 +03:00
Nikolai Kochetov
534b60e242 Fix build. 2020-10-13 18:10:30 +03:00
Nikolai Kochetov
23afd74e1c Update InterpreterSelectQuery. 2020-10-13 17:55:22 +03:00
Nikolai Kochetov
1af0792f67 Update InterpreterSelectQuery. 2020-10-13 17:33:34 +03:00
Nikolai Kochetov
2bbdfc77ea Update creating sets. 2020-10-13 16:48:00 +03:00
Nikolai Kochetov
337098367b Try fix perf 2020-10-13 11:36:15 +03:00
Nikolai Kochetov
7e58f99f64 Merge branch 'master' into storage-read-query-plan 2020-10-12 13:12:39 +03:00
Amos Bird
867216103f
Extend trivial count optimization. 2020-10-08 18:08:17 +08:00
Amos Bird
c9ab87ed17
Fix explain syntax 2020-10-08 17:06:04 +08:00
Amos Bird
ac102540d0
Global WITH and WITH propagation. 2020-10-08 15:03:34 +08:00
Nikolai Kochetov
f9bf1e3406 Merge branch 'master' into storage-read-query-plan 2020-10-06 09:50:10 +03:00
Azat Khuzhin
587cde853e Avoid skipping unused shards twice (for query processing stage and read itself) 2020-10-02 22:42:09 +03:00
Nikolai Kochetov
bef96faa1a Tru fix tests. 2020-10-02 15:38:33 +03:00
hchen9
68888408da Pass context ptr to ref 2020-10-01 21:23:41 -07:00
hchen9
b3949db00f Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into trivial_count_seq_consistency 2020-10-01 21:15:26 -07:00
hchen9
b4494578d7 Support select_sequential_consistency in StorageReplicatedMergeTree.totalRows and totalBytes 2020-09-30 16:50:58 -07:00
hchen9
a5ac39b564 Pass Context parameter for IStorage.totalRows and IStorage.totalBytes 2020-09-30 16:47:42 -07:00
Artem Zuikov
0f6a6453fe
Remove multiple_joins_rewriter v1 (#15472) 2020-09-30 23:11:49 +03:00
Nikolai Kochetov
576ffadb17 Fix explain for ISourceStep. 2020-09-30 15:22:30 +03:00
hchen9
fda6bde428 improve change 2020-09-28 17:05:22 -07:00
Nikolai Kochetov
dea90009e3 Fix build 2020-09-25 16:03:12 +03:00
Nikolai Kochetov
62ef40394a
Merge pull request #14856 from ClickHouse/query-plan-for-creating-sets 
Query plan for creating sets
 2020-09-22 16:19:20 +03:00
Nikolai Kochetov
54c1f04b38 Review fixes. 2020-09-21 11:36:12 +03:00
Alexey Milovidov
059646743b Remove OpenCL 2020-09-19 17:20:00 +03:00
alexey-milovidov
acbdc51460
Merge pull request #14771 from amosbird/withsub 
First attempt of supporting CTE
 2020-09-19 16:25:34 +03:00
Artem Zuikov
28afbafa08 fix crash in RIGHT or FULL JOIN switch 2020-09-18 19:25:20 +03:00
Nikolai Kochetov
b26f11c00c Support StorageDistributed::read for QueryPlan. 2020-09-18 17:16:53 +03:00
Nikolai Kochetov
2ffc2b6927 Refactor IStorage::read with query plan. 2020-09-18 15:10:59 +03:00
Nikolai Kochetov
50674a320f Refactor IStorage::read with query plan. 2020-09-18 14:39:07 +03:00
Nikolai Kochetov
21bd45fb22
Update InterpreterSelectQuery.cpp 2020-09-17 12:57:48 +03:00
roman
b41421cb1c [settings]: introduce new query complexity settings for leaf-nodes 
The new setting should allow to control query complexity on leaf nodes
excluding the final merging stage on the root-node. For example, distributed
query that reads 1k rows from 5 shards will breach the `max_rows_to_read=5000`,
while effectively every shard reads only 1k rows. With setting `max_rows_to_read_leaf=1500`
this limit won't be reached and query will succeed since every shard reads
not more that ~1k rows.
 2020-09-17 10:37:05 +01:00
Nikolai Kochetov
3cc89d467c Merge branch 'master' into query-plan-for-creating-sets 2020-09-15 20:15:41 +03:00
Nikolai Kochetov
118a8a513e Update mutations interpreter. 2020-09-15 20:13:13 +03:00
Nikolai Kochetov
e29f27e333
Merge pull request #14815 from ClickHouse/storage-read-query-plan 
Return query plan from storage.
 2020-09-15 18:34:11 +03:00
Nikolai Kochetov
4c783f19ee Use QueryPlan for SubqueryForSet. 2020-09-15 16:25:14 +03:00
Nikolai Kochetov
6eac433c55 Review fixes. 2020-09-15 13:40:39 +03:00
Artem Zuikov
51ba12c2c3
Try speedup build (#14809) 2020-09-15 12:55:57 +03:00
Nikolai Kochetov
bcfc83022a Return query plan from storage. 2020-09-14 17:13:58 +03:00
Amos Bird
882b2a3348
CTE 2020-09-13 09:39:17 +08:00
alexey-milovidov
f2e9a09b11
Merge pull request #14585 from ClickHouse/join-processor 
Remove Join from ExpressionActions
 2020-09-08 23:56:09 +03:00
Nikolai Kochetov
39aad9979a Remove Join from ExpressionActions. 2020-09-08 13:40:53 +03:00
Nikolai Kochetov
c761970bc5 Merge branch 'master' into pipes-and-creating-sets 2020-09-04 11:38:35 +03:00
Nikolai Kochetov
b7e53208d1 Fix tests. 2020-09-04 11:36:47 +03:00
Azat Khuzhin
10b4f3b41f Optimize queries with LIMIT/LIMIT BY/ORDER BY for distributed with GROUP BY sharding_key 
Previous set of QueryProcessingStage does not allow to do this.
But after WithMergeableStateAfterAggregation had been introduced the
following queries can be optimized too under
optimize_distributed_group_by_sharding_key:
- GROUP BY sharding_key LIMIT
- GROUP BY sharding_key LIMIT BY
- GROUP BY sharding_key ORDER BY

And right now it is still not supports:
- WITH TOTALS (looks like it can be supported)
- WITH ROLLUP (looks like it can be supported)
- WITH CUBE
- SETTINGS extremes=1 (looks like it can be supported)
But will be implemented separatelly.

vX: fixes
v2: fix WITH *
v3: fix extremes
v4: fix LIMIT OFFSET (and make a little bit cleaner)
v5: fix HAVING
v6: fix ORDER BY
v7: rebase against 20.7
v8: move out WithMergeableStateAfterAggregation
v9: add optimize_distributed_group_by_sharding_key into test names
 2020-09-03 00:52:51 +03:00
Azat Khuzhin
4043be3121 Add new query processing stage WithMergeableStateAfterAggregation 
Process query until the stage where the aggregate functions were
calculated and finalized.

It will be used for optimize_distributed_group_by_sharding_key.

v2: fix aliases
v3: Fix protocol ABI breakage due to WithMergeableStateAfterAggregation
    Conditions >= for QueryProcessingStage::Enum has been verified, and they
    are ok (in InterpreterSelectQuery).
 2020-09-03 00:51:08 +03:00
Nikolai Kochetov
acfd35a908 Update QueryPipeline::addCreatingSetsTransform 2020-09-02 16:04:54 +03:00
Nikolai Kochetov
95bbda3667 Specify columns after ARRAY JOIN and JOIN. 
Add ArrayJoinTransform and ArrayJoinStep.
 2020-08-14 21:24:03 +03:00
Vitaly Baranov
cd372de417 Use SettingUInt64 only in Settings, use UInt64 in other places. 2020-07-31 19:11:27 +03:00
Nikolai Kochetov
be24184deb Ignore constants for remote query from local replica. 2020-07-29 17:27:37 +03:00
Artem Zuikov
2afd123eda
Refactoring: extract TreeOptimizer from SyntaxAnalyzer (#12645) 2020-07-22 20:13:05 +03:00
Alexey Milovidov
ef930df484 Avoid overflow in LIMIT #10470 #11372 2020-07-12 08:18:01 +03:00
Nikolai Kochetov
8587c2025d Merge branch 'master' into initial-explain 2020-06-29 23:35:56 +03:00
Nikolai Kochetov
40b398c8c4 Merge branch 'master' into better-distinct-for-query-plan 2020-06-29 11:14:05 +03:00
Nikolai Kochetov
893a75bcf7 Merge branch 'master' into initial-explain 2020-06-27 12:45:40 +03:00
Nikolai Kochetov
ca01094f4b More actions description. 2020-06-27 12:31:11 +03:00
Nikolai Kochetov
e1481ccfc2 Add initial explain. 2020-06-27 12:30:38 +03:00
Artem Zuikov
54fd9ce76b fix totals 2020-06-26 03:23:11 +03:00
Artem Zuikov
2d7d389b77 better ExpressionAction::execute() 2020-06-25 23:28:41 +03:00
Artem Zuikov
ae5ed098f4 remove useless logic 2020-06-25 21:18:27 +03:00
Artem Zuikov
801efc387a extract JOIN in own plan step 2020-06-25 20:40:25 +03:00
Nikolai Kochetov
58b3b5c2f9 Rename step. 2020-06-22 13:21:13 +03:00
Nikolai Kochetov
d7d334bf6f Update query plan traits for DISTINCT. Add more comments. 2020-06-22 13:18:28 +03:00
alesapin
b1e8976df4 Merge with master 2020-06-22 12:04:27 +03:00
alesapin
b9e74f4e82 Merge branch 'master' into atomic_metadata5 2020-06-22 12:03:53 +03:00
Nikolai Kochetov
819ea4fb89 Fix tests. 2020-06-20 16:48:21 +03:00
Nikolai Kochetov
0c6129ac7c Fix build. 2020-06-20 11:52:41 +03:00
Nikolai Kochetov
4d70cc34c1 Merge branch 'master' into query-plan 2020-06-20 10:53:48 +03:00
alexey-milovidov
4129300fdb
Merge pull request #11788 from ClickHouse/fix-too-many-threads 
Fix too many threads
 2020-06-20 08:24:47 +03:00
alesapin
85070ea2fd Remove unused variable 2020-06-19 22:41:24 +03:00
alesapin
4c0879ae30 Better logging in storages 2020-06-19 20:17:13 +03:00
Nikolai Kochetov
9ca456f255 Try fix totals. 2020-06-19 19:58:27 +03:00
Nikolai Kochetov
4cafa3d703 Try fix totals. 2020-06-19 19:51:44 +03:00
Nikolai Kochetov
f6cd3f43bb Fix Context leak in InterpreterSelectQuery 2020-06-19 17:42:01 +03:00
Nikolai Kochetov
6c9c33f870 Fix Context leak in InterpreterSelectQuery 2020-06-19 17:00:11 +03:00
Nikolai Kochetov
ec892775a5 Fix Context leak in InterpreterSelectQuery 2020-06-19 16:34:40 +03:00
alexey-milovidov
3e06182079
Update InterpreterSelectQuery.cpp 2020-06-19 14:28:48 +03:00
Nikolai Kochetov
0335c698e0 Limit the number of threads. 2020-06-19 13:57:44 +03:00
Nikolai Kochetov
9ddd7b98cc Fix tests. 2020-06-19 12:47:15 +03:00
Nikolai Kochetov
6708ae75f3 Fix build. 2020-06-19 11:30:21 +03:00
Nikolai Kochetov
cda4de40d7 Merge branch 'master' into query-plan 2020-06-19 11:29:15 +03:00
Nikolai Kochetov
a99d9f7712 Fix build. 2020-06-18 21:36:37 +03:00
Nikolai Kochetov
b7bed87a7f Use QueryPlan in InterpreterSelectQuery [part 3]. 2020-06-18 21:29:33 +03:00
Nikolai Kochetov
45adacf0bc Use QueryPlan in InterpreterSelectQuery [part 2]. 2020-06-18 20:45:00 +03:00
Amos Bird
d524301038
add scalars in time 2020-06-19 00:14:19 +08:00
alesapin
d79982f497 Better locks in Storages 2020-06-18 19:10:47 +03:00
Nikolai Kochetov
e0841360db Use QueryPlan in InterpreterSelectQuery [part 1]. 2020-06-18 16:38:45 +03:00
Nikolai Kochetov
5bb2ddc6f9 Add DISTINCT trait for QueryPlanStep. 2020-06-18 16:00:16 +03:00
Nikolai Kochetov
9ad00187e3 Update InterpreterSelectQuery.. 2020-06-17 23:19:55 +03:00
Nikolai Kochetov
169ad5e805 Add FinishSortingStep. 2020-06-17 22:57:13 +03:00
Nikolai Kochetov
e8049d34c5 Add OffsetsStep. 2020-06-17 22:38:45 +03:00
Nikolai Kochetov
69ff3a8f7d Add ExtremesStep. 2020-06-17 22:30:36 +03:00
Nikolai Kochetov
d795b2b037 Add FillingStep 2020-06-17 22:20:30 +03:00
Nikolai Kochetov
84fd7fe51c Update having. 2020-06-17 22:02:46 +03:00
Nikolai Kochetov
2cffa56914 Add RollupStep and CubeStep. 2020-06-17 20:20:09 +03:00
Nikolai Kochetov
d41db3622f Add RollupStep and CubeStep. 2020-06-17 20:15:24 +03:00
alesapin
51a47560e0 Fix segmentation fault 2020-06-17 19:59:26 +03:00
Nikolai Kochetov
c5d9379df0 Add TotalsHavingStep. 2020-06-17 19:54:51 +03:00
alesapin
dffdece350 getColumns in StorageInMemoryMetadta (only compilable) 2020-06-17 19:39:58 +03:00
Nikolai Kochetov
1ad4f2c0fd Add CreatingSets step. 2020-06-17 18:50:21 +03:00
Nikolai Kochetov
f279df6015 Added AggregatingStep. 2020-06-17 17:21:48 +03:00
alesapin
31abbe5dbd Select query in metadata 2020-06-17 17:06:22 +03:00
Nikolai Kochetov
582ea24469 Added AddingDelayedStreamStep 2020-06-17 16:38:17 +03:00
alesapin
1afdebeebd Primary key in storage metadata 2020-06-17 15:39:20 +03:00
alesapin
eca6e9087e Fix race condition 2020-06-17 14:52:19 +03:00
alesapin
ba04d02f1e Compilable sorting key in metadata 2020-06-17 14:05:11 +03:00
alesapin
62f2c17a66 Secondary indices in StorageInMemoryMetadata 2020-06-17 12:38:47 +03:00
Nikolai Kochetov
fdacc9f90e Add MergingAggregatedStep. 2020-06-16 21:57:21 +03:00
Nikolai Kochetov
148257abb5 Add LimitStep. 2020-06-16 21:23:01 +03:00
Nikolai Kochetov
5bb1d5cc43 Add LimitByStep. 2020-06-16 19:49:49 +03:00
Nikolai Kochetov
603bcdde2d Add DistinctStep. 2020-06-16 19:13:07 +03:00
Nikolai Kochetov
08e9ce8661 Add MergingSortedStep. 2020-06-16 18:47:40 +03:00
Nikolai Kochetov
f563193967 Add MergeSortingStep. 2020-06-16 18:08:01 +03:00
Nikolai Kochetov
815ac03892 Add PartialSortingStep. 2020-06-16 17:45:52 +03:00
alesapin
71f99a274d Compileable getSampleBlockWithColumns in StorageInMemoryMetadata 2020-06-16 17:25:08 +03:00
Nikolai Kochetov
4407bd7daa Added source steps. 2020-06-16 17:11:19 +03:00
Nikolai Kochetov
be8029e54f Expression and Filter steps. 2020-06-16 15:02:10 +03:00
alesapin
eae901f58d Merge branch 'master' into atomic_metadata5 2020-06-16 14:44:45 +03:00
Nikolai Kochetov
1bb72205bd Expression and Filter steps. 2020-06-16 12:42:25 +03:00
Anton Popov
eabbabed04 fix 'LIMIT WITH TIES' with aliases 2020-06-16 01:24:00 +03:00
Nikolai Kochetov
03e0a5ddbc Merge branch 'master' into query-plan 2020-06-15 23:38:35 +03:00
alesapin
36ba0192df Metadata in read and write methods of IStorage 2020-06-15 22:08:58 +03:00
Nikolai Kochetov
b725df6366 Added ReadFromStorageStep. 2020-06-15 18:17:44 +03:00
Artem Zuikov
24059efad5
Change push down logic in VIEW (#11513) 2020-06-15 15:36:10 +03:00
alexey-milovidov
fd6115cccf
Merge pull request #11648 from ClickHouse/simplify-code-functions-comparison-3 
Allow comparison with String in index analysis; simplify code.
 2020-06-15 05:47:03 +03:00
Alexey Milovidov
8dac30ae95 Split file for better build times 2020-06-14 21:42:10 +03:00
Alexey Milovidov
2cd82a25f5 Remove trivial count query optimization if row-level security is set #11352 2020-06-13 23:59:20 +03:00
Alexey Milovidov
f2cfb7d132 Whitespaces 2020-06-08 00:05:36 +03:00
Anton Popov
5c42408add
Merge pull request #9113 from dimarub2000/group_by_in_order_optimization 
[WIP] Optimization of GROUP BY with respect to table sorting key.
 2020-06-06 14:25:59 +03:00
Artem Zuikov
9717f3cfe5 make clang tidy happy 2020-06-05 13:27:54 +03:00
Artem Zuikov
6d211bec19 fix trivial count with join 2020-06-05 01:01:40 +03:00
Nikolai Kochetov
53d12f5ab8 Try fix build. 2020-06-01 20:06:21 +03:00
Nikolai Kochetov
325393d340 Try fix build. 2020-06-01 19:58:36 +03:00
Nikolai Kochetov
3a0acb00c1 Remove some streams. 2020-06-01 19:31:06 +03:00
alexey-milovidov
9984e989c1
Merge pull request #11317 from ClickHouse/logger-fmt-2 
Preparation for structured logging
 2020-06-01 00:11:29 +03:00
alexey-milovidov
97d9f218af
Merge pull request #11296 from ClickHouse/fix-false-too-slow 
Fix bug with Throttler and query speed estimation
 2020-05-31 14:18:29 +03:00
Alexey Milovidov
25f941020b Remove namespace pollution 2020-05-31 00:57:37 +03:00
Dmitry
4b0d32f026 Merge branch 'master' of github.com:yandex/ClickHouse into group_by_in_order_optimization 2020-05-31 00:21:02 +03:00
Alexey Milovidov
e4a329a47f Checking max execution speed on all nodes #5732 2020-05-29 23:37:37 +03:00
Nikolai Kochetov
f7e0e87c7d Try fix tests. 2020-05-29 19:35:09 +03:00
Nikolai Kochetov
bd814d83de Try fix tests. 2020-05-29 16:04:03 +03:00
Nikolai Kochetov
bbb57b7789 Try fix tests. 2020-05-29 13:31:19 +03:00
Nikolai Kochetov
534e55f5b5 try fix tests. 2020-05-28 19:01:12 +03:00
Nikolai Kochetov
557e531ff3 Fix tests. 2020-05-28 15:06:31 +03:00
Nikolai Kochetov
1fa795988f Remove some code. 2020-05-28 11:24:59 +03:00
Dmitry
38c585f867 Merge branch 'master' of github.com:yandex/ClickHouse into group_by_in_order_optimization 2020-05-26 21:27:50 +03:00
Nikolai Kochetov
c83014bc76
Merge pull request #11114 from azat/max_threads-simple-query-optimization-fix 
Do not reserve extra threads after max threads optimization for simple queries
 2020-05-25 14:05:46 +03:00
Alexey Milovidov
7e1813825b Return old names of macros 2020-05-24 01:24:01 +03:00
Alexey Milovidov
d6bc4d2bd0 find {base,src,programs} -name '*.h' -or -name '*.cpp' | xargs grep -l -P 'LOG_\w+\([^,]+, [^<]+ << "[^"]+" << [^<]+\);' | xargs sed -i -r -e 's/(LOG_\w+)\(([^,]+), ([^<]+) << "([^"]+)" << ([^<]+)\);/\1_FORMATTED(\2, "{}\4{}", \3, \5);/' 2020-05-23 20:14:57 +03:00
Azat Khuzhin
d4804e070f Do not reserve extra threads after max threads optimization for simple queries 
Before this patch pipeline.setMaxThreads() was called with 0 (since
streams.size() == 0, while pipes.size() == 1), which leads that
pipeline.getMaxThreads() returns max_streams.

While this threads were not used, they were reserved, which should not
happens either.
 2020-05-23 13:19:09 +03:00