Commit Graph

509 Commits

Author SHA1 Message Date
Kruglov Pavel
a54cbef5b7
Merge pull request #24341 from Avogar/arrow
Support complex types in Arrow/Parquet/ORC
2021-06-21 11:05:57 +03:00
Anton Popov
d8b6f15ef4
Merge pull request #23027 from azat/distributed-push-down-limit
Add ability to push down LIMIT for distributed queries
2021-06-20 23:08:50 +03:00
mergify[bot]
c723dd7d40
Merge branch 'master' into arrow 2021-06-18 12:18:57 +00:00
Amos Bird
31815f4a68
Proper documentation of the setting 2021-06-17 14:27:33 +08:00
Amos Bird
c8ea6527cb
Add prefer_global_in_and_join setting 2021-06-17 14:27:29 +08:00
Ivan Lezhankin
ba08a580f8 Add test 2021-06-16 17:33:14 +03:00
Ivan Lezhankin
b182d87d9c Add settings for HTTP header limitations 2021-06-15 17:33:46 +03:00
Pavel Kruglov
a4decd0848 Support Map type, fix and add tests 2021-06-15 16:15:27 +03:00
Pavel Kruglov
c8b37977da Fix bugs, support dictionary for Arrow format 2021-06-15 16:15:27 +03:00
Anton Popov
bb6f0dfbb8
Merge pull request #24406 from CurtizJ/rewrite-functions-to-subcolumns
Optimize some functions to subcolumns
2021-06-09 11:07:09 +03:00
Azat Khuzhin
18e8f0eb5e Add ability to push down LIMIT for distributed queries
This way the remote nodes will not need to send all the rows, so this
will decrease network io and also this will make queries w/
optimize_aggregation_in_order=1/LIMIT X and w/o ORDER BY faster since it
initiator will not need to read all the rows, only first X (but note
that for this you need to your data to be sharded correctly or you may
get inaccurate results).

Note, that having lots of processing stages will increase the complexity
of interpreter (it is already not that clean and simple right now).

Although using separate QueryProcessingStage looks pretty natural.

Another option is to make WithMergeableStateAfterAggregation always, but
in this case you will not be able to disable only this optimization,
i.e. if there will be some issue with it.

v2: fix OFFSET
v3: convert 01814_distributed_push_down_limit test to .sh and add retries
v4: add test with OFFSET
v5: add new query stage into the bash completion
v6/tests: use LIMIT O,L syntax over LIMIT L OFFSET O since it is broken in ANTLR parser
          https://clickhouse-test-reports.s3.yandex.net/23027/a18a06399b7aeacba7c50b5d1e981ada5df19745/functional_stateless_tests_(antlr_debug).html#fail1
v7/tests: set use_hedged_requests to 0, to avoid excessive log entries on retries
          https://clickhouse-test-reports.s3.yandex.net/23027/a18a06399b7aeacba7c50b5d1e981ada5df19745/functional_stateless_tests_flaky_check_(address).html#fail1
2021-06-09 02:29:50 +03:00
alesapin
5736eaf8fd Increase background schedule pool default size 2021-06-08 13:55:14 +03:00
Alexey Milovidov
8d85145725 Remove experimental compression codecs 2021-06-06 02:26:52 +03:00
Alexey Milovidov
17962459f5 Merge branch 'master' into issue-16775 2021-06-06 02:18:28 +03:00
Vladimir
17f0900c31
Merge pull request #19766 from achimbab/pr_sequenceNextNode 2021-06-03 13:16:42 +03:00
alesapin
4a1bf51ad1
Merge branch 'master' into limited_fetches 2021-06-02 23:01:47 +03:00
vdimir
ec09dfe635
Set allow_experimental_funnel_functions = false by default (not used before) 2021-06-02 09:16:31 +03:00
vdimir
1b936c8af6
Remove example usage of allow_experimental_funnel_functions from windowFunnel 2021-06-02 09:16:21 +03:00
alesapin
c3321bef30 Merge branch 'master' into limited_fetches 2021-06-01 13:09:02 +03:00
Anton Popov
b24f65e1ad Merge remote-tracking branch 'upstream/master' into HEAD 2021-05-31 23:04:52 +03:00
Alexey Milovidov
3791c1c99d Someone added useful settings to the section with obsolete settings 2021-05-31 22:26:34 +03:00
Alexey Milovidov
4187aa444b Fix typo. Remove "allow_experimental_bigint_types" setting. 2021-05-31 22:21:28 +03:00
alesapin
42b2383e01 Basic throttler implementation 2021-05-26 23:37:44 +03:00
Vladimir
fc15cad7e0
Pass Settings to aggregate function creator 2021-05-26 14:58:31 +03:00
Vladimir
7224e25156
Revert "Pass Settings to aggregate function creator" 2021-05-26 14:46:42 +03:00
Vladimir
6650c67b01
Merge pull request #22762 from vdimir/create-aggregate-func-settings 2021-05-26 14:32:14 +03:00
Anton Popov
a36c1793d0 Merge remote-tracking branch 'upstream/master' into HEAD 2021-05-24 15:57:43 +03:00
Alexey Milovidov
6f70feed2f Merge branch 'master' into issue-16775 2021-05-24 05:56:57 +03:00
Alexey Milovidov
e000b994e5 Add missing whitespace 2021-05-23 03:26:31 +03:00
alexey-milovidov
1ed30f7246
Merge branch 'master' into issue-16775 2021-05-22 17:14:23 +03:00
Anton Popov
13cfedd188 optimize some functions to subcolumns 2021-05-21 21:48:19 +03:00
Maksim Kita
25fe0e88f8 Compile expressions added partition by test 2021-05-20 11:16:15 +03:00
Maksim Kita
b0476c1fa2
Merge pull request #24273 from kitaisreal/expression-actions-compile-only-necessary-places
ExpressionActions compile only necessary places
2021-05-20 10:53:30 +03:00
Maksim Kita
c50909468f Updated ExpressionActionSettings 2021-05-19 17:32:07 +03:00
alesapin
301dc848f5 Merge branch 'master' into nvartolomei-parts-move 2021-05-19 17:17:01 +03:00
Alexander Kuzmenkov
e9b69bbd70
Merge pull request #23906 from azat/fix-distributed_group_by_no_merge
distributed_group_by_no_merge fixes
2021-05-19 16:16:08 +03:00
alesapin
17f229857c Merge branch 'master' into nvartolomei-parts-move 2021-05-17 13:52:48 +03:00
Azat Khuzhin
a44ddd2064 Update distributed_group_by_no_merge description 2021-05-15 15:03:13 +03:00
Alexander Kuzmenkov
5ed6a273d3
Merge pull request #24058 from ClickHouse/aku/uniq-two-level
lower two-level aggregation threshold for uniq test to avoid jitter
2021-05-14 15:50:48 +03:00
Alexander Kuzmenkov
6f16d92cfe
Update Settings.h 2021-05-14 15:50:30 +03:00
Alexander Kuzmenkov
220f5b512e
try even smaller value 2021-05-13 06:15:44 +03:00
Alexander Kuzmenkov
f6cb9f95df lower two-level aggregation threshold for uniq test to avoid jitter 2021-05-12 16:01:48 +03:00
alesapin
46e136b5c4
Merge branch 'master' into nv/parts-uuid-move-shard 2021-05-11 15:36:40 +03:00
Amos Bird
264cff6415
Projections
TODO (suggested by Nikolai)

1. Build query plan fro current query (inside storage::read) up to WithMergableState
2. Check, that plan is simple enough: Aggregating - Expression - Filter - ReadFromStorage (or simplier)
3. Check, that filter is the same as filter in projection, and also expression calculates the same aggregation keys as in projection
4. Return WithMergableState if projection applies

3 will be easier to do with ActionsDAG, cause it sees all functions, and dependencies are direct (but it is possible with ExpressionActions also)

Also need to figure out how prewhere works for projections, and
row_filter_policies.

wip
2021-05-11 18:12:23 +08:00
vdimir
905e54a0af
Merge branch 'master' into create-aggregate-func-settings 2021-05-11 11:18:24 +03:00
Maksim Kita
9daf5b8308 Updated Settings 2021-05-08 17:39:34 +03:00
kssenii
912878ad7f Review fixes 2021-05-07 12:05:34 +00:00
kssenii
8662078b35 Review fixes 2021-05-07 09:55:18 +00:00
fibersel
cb53bbb7b0 add experimental codecs flag, add integration test for experimental codecs 2021-05-06 14:57:22 +03:00
Nicolae Vartolomei
1fa5871ff7 Fix bad rebase and introduce part_moves_between_shards_enable setting 2021-04-27 14:20:13 +01:00