thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-01 20:12:02 +00:00
Code Issues Packages Projects Releases Wiki Activity
73,823 Commits 9,193 Branches 1,782 Tags 2.2 GiB
dc2cb6d996
Commit Graph

691 Commits

Author SHA1 Message Date
kssenii
2cfcc9980c Better 2021-06-13 19:38:31 +00:00
Ivan Lezhankin
2c7cb41a69 Merge remote-tracking branch 'upstream/master' into async-insert 2021-06-11 20:04:26 +03:00
Anton Popov
bb6f0dfbb8
Merge pull request #24406 from CurtizJ/rewrite-functions-to-subcolumns 
Optimize some functions to subcolumns
 2021-06-09 11:07:09 +03:00
Azat Khuzhin
18e8f0eb5e Add ability to push down LIMIT for distributed queries 
This way the remote nodes will not need to send all the rows, so this
will decrease network io and also this will make queries w/
optimize_aggregation_in_order=1/LIMIT X and w/o ORDER BY faster since it
initiator will not need to read all the rows, only first X (but note
that for this you need to your data to be sharded correctly or you may
get inaccurate results).

Note, that having lots of processing stages will increase the complexity
of interpreter (it is already not that clean and simple right now).

Although using separate QueryProcessingStage looks pretty natural.

Another option is to make WithMergeableStateAfterAggregation always, but
in this case you will not be able to disable only this optimization,
i.e. if there will be some issue with it.

v2: fix OFFSET
v3: convert 01814_distributed_push_down_limit test to .sh and add retries
v4: add test with OFFSET
v5: add new query stage into the bash completion
v6/tests: use LIMIT O,L syntax over LIMIT L OFFSET O since it is broken in ANTLR parser
          https://clickhouse-test-reports.s3.yandex.net/23027/a18a06399b7aeacba7c50b5d1e981ada5df19745/functional_stateless_tests_(antlr_debug).html#fail1
v7/tests: set use_hedged_requests to 0, to avoid excessive log entries on retries
          https://clickhouse-test-reports.s3.yandex.net/23027/a18a06399b7aeacba7c50b5d1e981ada5df19745/functional_stateless_tests_flaky_check_(address).html#fail1
 2021-06-09 02:29:50 +03:00
Ivan Lezhankin
973cea73b7 Merge branch 'master' into async-insert 2021-06-08 15:10:30 +03:00
alesapin
5736eaf8fd Increase background schedule pool default size 2021-06-08 13:55:14 +03:00
Alexey Milovidov
8d85145725 Remove experimental compression codecs 2021-06-06 02:26:52 +03:00
Alexey Milovidov
17962459f5 Merge branch 'master' into issue-16775 2021-06-06 02:18:28 +03:00
kssenii
e23bc14582 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into pg-ch-replica 2021-06-03 19:44:55 +00:00
Vladimir
17f0900c31
Merge pull request #19766 from achimbab/pr_sequenceNextNode 2021-06-03 13:16:42 +03:00
alesapin
4a1bf51ad1
Merge branch 'master' into limited_fetches 2021-06-02 23:01:47 +03:00
Ivan Lezhankin
791cbcf0b1 Merge remote-tracking branch 'upstream/master' into async-insert 2021-06-02 18:50:37 +03:00
Ivan Lezhankin
84c23dc060 Merge branch 'master' into async-insert 2021-06-02 18:06:21 +03:00
vdimir
ec09dfe635
Set allow_experimental_funnel_functions = false by default (not used before) 2021-06-02 09:16:31 +03:00
vdimir
1b936c8af6
Remove example usage of allow_experimental_funnel_functions from windowFunnel 2021-06-02 09:16:21 +03:00
kssenii
12ea45e956 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into pg-ch-replica 2021-06-01 17:43:34 +00:00
alesapin
c3321bef30 Merge branch 'master' into limited_fetches 2021-06-01 13:09:02 +03:00
Anton Popov
b24f65e1ad Merge remote-tracking branch 'upstream/master' into HEAD 2021-05-31 23:04:52 +03:00
Alexey Milovidov
3791c1c99d Someone added useful settings to the section with obsolete settings 2021-05-31 22:26:34 +03:00
Alexey Milovidov
4187aa444b Fix typo. Remove "allow_experimental_bigint_types" setting. 2021-05-31 22:21:28 +03:00
alesapin
42b2383e01 Basic throttler implementation 2021-05-26 23:37:44 +03:00
Vladimir
fc15cad7e0
Pass Settings to aggregate function creator 2021-05-26 14:58:31 +03:00
Vladimir
7224e25156
Revert "Pass Settings to aggregate function creator" 2021-05-26 14:46:42 +03:00
Vladimir
6650c67b01
Merge pull request #22762 from vdimir/create-aggregate-func-settings 2021-05-26 14:32:14 +03:00
Anton Popov
a36c1793d0 Merge remote-tracking branch 'upstream/master' into HEAD 2021-05-24 15:57:43 +03:00
Alexey Milovidov
6f70feed2f Merge branch 'master' into issue-16775 2021-05-24 05:56:57 +03:00
kssenii
a78c9e63e7 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into pg-ch-replica 2021-05-23 12:07:26 +00:00
Alexey Milovidov
e000b994e5 Add missing whitespace 2021-05-23 03:26:31 +03:00
alexey-milovidov
1ed30f7246
Merge branch 'master' into issue-16775 2021-05-22 17:14:23 +03:00
Anton Popov
13cfedd188 optimize some functions to subcolumns 2021-05-21 21:48:19 +03:00
Maksim Kita
25fe0e88f8 Compile expressions added partition by test 2021-05-20 11:16:15 +03:00
Maksim Kita
b0476c1fa2
Merge pull request #24273 from kitaisreal/expression-actions-compile-only-necessary-places 
ExpressionActions compile only necessary places
 2021-05-20 10:53:30 +03:00
Maksim Kita
c50909468f Updated ExpressionActionSettings 2021-05-19 17:32:07 +03:00
alesapin
301dc848f5 Merge branch 'master' into nvartolomei-parts-move 2021-05-19 17:17:01 +03:00
Alexander Kuzmenkov
e9b69bbd70
Merge pull request #23906 from azat/fix-distributed_group_by_no_merge 
distributed_group_by_no_merge fixes
 2021-05-19 16:16:08 +03:00
alesapin
17f229857c Merge branch 'master' into nvartolomei-parts-move 2021-05-17 13:52:48 +03:00
kssenii
8a4711b716 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into pg-ch-replica 2021-05-16 20:55:18 +00:00
Azat Khuzhin
a44ddd2064 Update distributed_group_by_no_merge description 2021-05-15 15:03:13 +03:00
Alexander Kuzmenkov
5ed6a273d3
Merge pull request #24058 from ClickHouse/aku/uniq-two-level 
lower two-level aggregation threshold for uniq test to avoid jitter
 2021-05-14 15:50:48 +03:00
Alexander Kuzmenkov
6f16d92cfe
Update Settings.h 2021-05-14 15:50:30 +03:00
Alexander Kuzmenkov
220f5b512e
try even smaller value 2021-05-13 06:15:44 +03:00
kssenii
407db17e2e Remove setting, use native transaction 2021-05-12 22:33:15 +00:00
kssenii
0955e5f0be Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into pg-ch-replica 2021-05-12 15:11:15 +00:00
Alexander Kuzmenkov
f6cb9f95df lower two-level aggregation threshold for uniq test to avoid jitter 2021-05-12 16:01:48 +03:00
alesapin
46e136b5c4
Merge branch 'master' into nv/parts-uuid-move-shard 2021-05-11 15:36:40 +03:00
Amos Bird
264cff6415
Projections 
TODO (suggested by Nikolai)

1. Build query plan fro current query (inside storage::read) up to WithMergableState
2. Check, that plan is simple enough: Aggregating - Expression - Filter - ReadFromStorage (or simplier)
3. Check, that filter is the same as filter in projection, and also expression calculates the same aggregation keys as in projection
4. Return WithMergableState if projection applies

3 will be easier to do with ActionsDAG, cause it sees all functions, and dependencies are direct (but it is possible with ExpressionActions also)

Also need to figure out how prewhere works for projections, and
row_filter_policies.

wip
 2021-05-11 18:12:23 +08:00
vdimir
905e54a0af
Merge branch 'master' into create-aggregate-func-settings 2021-05-11 11:18:24 +03:00
kssenii
5c24f0943c Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into pg-ch-replica 2021-05-08 14:54:44 +00:00
Maksim Kita
9daf5b8308 Updated Settings 2021-05-08 17:39:34 +03:00
kssenii
912878ad7f Review fixes 2021-05-07 12:05:34 +00:00
kssenii
8662078b35 Review fixes 2021-05-07 09:55:18 +00:00
hexiaoting
a1d521902c Merge remote-tracking branch 'origin/master' into dev-continue-sundyli 2021-05-07 14:22:53 +08:00
fibersel
cb53bbb7b0 add experimental codecs flag, add integration test for experimental codecs 2021-05-06 14:57:22 +03:00
kssenii
7643d84095 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into pg-ch-replica 2021-04-30 22:02:51 +00:00
Nicolae Vartolomei
1fa5871ff7 Fix bad rebase and introduce part_moves_between_shards_enable setting 2021-04-27 14:20:13 +01:00
Nicolae Vartolomei
53d57ffb52 Part movement between shards 
Integrate query deduplication from #17348
 2021-04-27 14:20:12 +01:00
Nikolai Kochetov
29fd51e37d Enable async_socket_for_remote 2021-04-27 13:30:12 +03:00
Vladimir
fd777bcd4b
Merge branch 'master' into create-aggregate-func-settings 2021-04-26 11:12:09 +03:00
kssenii
459612c895 Use a separate setting 2021-04-22 22:14:14 +00:00
Ivan Lezhankin
484528ba2c Fix problem with use-after-free inside shared_ptr 2021-04-21 16:19:28 +03:00
Nikolai Kochetov
44112587d4
Merge pull request #23309 from ClickHouse/try-fix-crash-with-unknown-packet 
Fix crash in case of unknown packet
 2021-04-21 08:47:17 +03:00
alexey-milovidov
a311a5c0b5
Merge pull request #23322 from azat/disable-min_bytes_to_use_mmap_io 
Disable min_bytes_to_use_mmap_io by default
 2021-04-20 19:27:47 +03:00
Ivan
10c5ba3022
Merge branch 'master' into async-insert 2021-04-20 17:58:15 +03:00
Nikolai Kochetov
ecb2a96af8 Revert "Enable async_socket_for_remote" 
This reverts commit ff908baa06.
 2021-04-20 17:14:54 +03:00
Nikita Mikhaylov
f55eacee7d
Merge pull request #22988 from excitoon-favorites/s3connectionresetretries 
Retries on HTTP connection drops during reads from S3
 2021-04-20 14:59:21 +03:00
Kseniia Sumarokova
b43656bf89
Merge pull request #22697 from TCeason/feature/add_MaterializeMySQL_sync_bytes_judgment 
Add MySQL read history data bytes judgment
 2021-04-20 10:26:42 +03:00
Azat Khuzhin
6fba6f7ddd Disable min_bytes_to_use_mmap_io by default 
Reading files using mmap() does not have any significant benefits over
plain read() [1].

  [1]: https://gist.github.com/azat/3d6c8d82bdd91e7a38d997fd6bcfd574

And not only it does not have significant benefits, it also has some
issues, due to max_server_memory_usage (default to 90% of available
RAM), since when you read files with mmap() eventually process RSS may
exceed max_server_memory_usage, and in this case any allocation will
fail (with "Memory limit exceeded (total)") error (yes kernel will
unload pages, but likely it will happens after queries will starting to
fail), like in this test [2].

  [2]: https://gist.github.com/azat/4813489828162e6c2ce131963c6a1acb

TL;DR;

Note that there was also an idea to take those mmap()'ed regions in
memory tracking (#23211), but there are some drawbacks (since accounting
mmap() is tricky, first of all you need to account only once per inode
for file and plus kernel can unload some pages and those memory will not
be used by the server anymore).

And as an adddition to #23211 there was #23212, that adds
max_bytes_to_use_mmap_io, but since mmap is not a subject for memory
accounting there is no need in it.
 2021-04-20 00:25:16 +03:00
Ivan Lezhankin
a36596e303 Add settings 2021-04-19 22:16:34 +03:00
Ivan Lezhankin
7a88ca31cc Merge branch 'master' into async-insert 2021-04-19 18:33:43 +03:00
Nikolai Kochetov
ff908baa06 Enable async_socket_for_remote 2021-04-19 17:31:46 +03:00
Nikolai Kochetov
0c23d44b99 Add test. 2021-04-19 17:12:08 +03:00
Alexander Kuzmenkov
2a4bcb6e3f
Merge pull request #23159 from ClickHouse/aku/merge-fusecount 
merging sumCount fusion PR #21337
 2021-04-19 16:47:13 +03:00
Vladimir Chebotarev
9d4295f261 Removed infinite retries. 2021-04-19 11:02:36 +03:00
TCeason
b4bf53dfc9 add some comment and modify a parameter type 2021-04-19 15:34:27 +08:00
TCeason
63403c709c modify settings name to external_xxx and rewrite Storage MySQL max_block_size 2021-04-19 10:51:50 +08:00
TCeason
87aa904440 Modify according to review opinion 2021-04-19 10:51:50 +08:00
TCeason
472c131420 Add MySQL read history data bytes judgment 2021-04-19 10:51:50 +08:00
kssenii
88b50621d4 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into pg-ch-replica 2021-04-18 14:40:37 +00:00
Alexey Milovidov
21ff5d85d9 Disable hedged requests 2021-04-18 15:13:57 +03:00
vdimir
92e813c806
Merge branch 'master' into create-aggregate-func-settings 2021-04-16 15:48:39 +03:00
Alexey Milovidov
9110a76d00 Reordered settings to avoid confusion 2021-04-16 00:14:37 +03:00
Alexander Kuzmenkov
2489b6af96 cleanup 2021-04-15 19:40:49 +03:00
Alexander Kuzmenkov
3b95b637a5 Merge remote-tracking branch 'origin/master' into HEAD 2021-04-15 18:19:53 +03:00
vdimir
33a9189151
Merge remote-tracking branch 'upstream/master' into create-aggregate-func-settings 2021-04-15 13:41:21 +03:00
Vladimir Chebotarev
10bf56e540 Attempt to add retries to S3 reads. 2021-04-15 09:47:19 +03:00
Ivan Lezhankin
9e5d274467 Merge remote-tracking branch 'upstream/master' into async-insert 2021-04-14 20:16:05 +03:00
alexey-milovidov
1945d01719
Merge pull request #21972 from kssenii/nanodbc 
Switch to nanodbc
 2021-04-13 12:23:57 +03:00
Maksim Kita
e687fc320f
Merge pull request #22997 from ClickHouse/increase-max-uri-size 
Increase `max_uri_size` to 1 MiB by default
 2021-04-13 11:26:23 +03:00
hexiaoting
77c460e8d1 Merge remote-tracking branch 'origin/master' into dev-sumcount 2021-04-13 11:14:14 +08:00
alexey-milovidov
d239c66fd3
Merge pull request #21511 from azat/optimize_skip_unused_shards-improvement 
Rewrite IN in query for remote shards to exclude values that do not belong to the shard
 2021-04-13 01:18:11 +03:00
Alexey Milovidov
e260b2a60a Increase max_uri_size to 1 MiB by default 2021-04-12 21:06:47 +03:00
vdimir
f2c109cc86
Merge branch 'master' into create-aggregate-func-settings 2021-04-12 11:38:20 +03:00
vdimir
f022f0744b
Add setting allow_experimental_funnel_functions and use in windowFunnel 2021-04-12 11:35:07 +03:00
Azat Khuzhin
fbb386dca5 Rewrite IN in query for remote shards to exclude values that does not belongs to shard 
v2: fix optimize_skip_unused_shards_rewrite_in for sharding_key wrapped into function
v3: fix column name for optimize_skip_unused_shards_rewrite_in
v4: fix optimize_skip_unused_shards_rewrite_in with Null
v5:
- squash with Remove query argument for IStreamFactory::createForShard()
- use proper column after function execution (using sharding_key_column_name)
- update the test reference since (X) now is tuple(X)
 2021-04-12 10:37:28 +03:00
kssenii
7a287e6fe9 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into nanodbc 2021-04-11 21:36:08 +00:00
kssenii
c1adece94f Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into pg-ch-replica 2021-04-11 06:00:18 +00:00
kssenii
1c501e7d97 Support replica identity index 2021-04-10 16:10:28 +00:00
Alexey Milovidov
173c0e5c43 More conservative value of background_fetches_pool_size setting 2021-04-09 23:45:14 +03:00
Alexander Kuzmenkov
e44b3822e3
Merge pull request #21850 from fastio/handle_errors_for_kafka_engine 
Handle errors for Kafka engine
 2021-04-09 22:59:40 +03:00
kssenii
8bdcac4f06 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into pg-ch-replica 2021-04-09 13:12:15 +00:00
Alexander Kuzmenkov
0264124146
Merge pull request #21942 from ucasFL/distributed_depth 
Add settings max_distributed_depth
 2021-04-09 15:52:58 +03:00
kssenii
72d89bc933 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into nanodbc 2021-04-09 11:13:01 +00:00
kssenii
9fa21a6272 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into pg-ch-replica 2021-04-08 12:18:54 +00:00
kssenii
0b39480d51 Better 2021-04-06 20:01:23 +00:00
Nikolai Kochetov
6102652c99 Merge branch 'master' into better-filter-push-down 2021-04-06 13:38:03 +03:00
Nikolai Kochetov
44700d97e9 Add setting to disable query plan optimization. Fix perftest. 2021-04-06 11:25:54 +03:00
feng lv
f6e4e13f89 fix conflict 2021-04-05 14:08:30 +00:00
feng lv
56073db22d max distributed depth 
Add settings max_distributed_depth

fix style

fix

fix

fix

fix test

fix

fix
 2021-04-05 14:00:54 +00:00
kssenii
13d1f17d3e Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into replicas-shards-for-mysql-and-postgres 2021-04-03 08:00:09 +00:00
kssenii
2de6b99f84 Better names 2021-04-02 16:12:14 +00:00
kssenii
99f0b08ac3 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into replicas-shards-for-mysql-and-postgres 2021-04-01 13:48:33 +00:00
Ivan Lezhankin
cc1c439ad8 Merge remote-tracking branch 'upstream/master' into poco-http2 2021-04-01 16:20:15 +03:00
kssenii
2c70183652 Fix settings 2021-04-01 11:41:22 +00:00
Peng Jian
909d5ad2b5 Handle errors for Kafka engine 2021-03-31 17:15:57 +08:00
alexey-milovidov
6ed78827a5
Merge pull request #22326 from ClickHouse/test-mmap-io 
Run performance tests with enabled mmap IO
 2021-03-31 06:33:56 +03:00
alexey-milovidov
0c70b06960
Merge branch 'master' into system-querylog-map 2021-03-31 04:54:30 +03:00
kssenii
ce05087b1b Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into replicas-shards-for-mysql-and-postgres 2021-03-30 17:27:10 +00:00
Amos Bird
69204e1d21
Add prefer_column_name_to_alias settings (#22044) 2021-03-30 16:51:45 +03:00
Ivan Lezhankin
873a98b33b Add test 2021-03-30 16:19:05 +03:00
Alexey Milovidov
85e4cfa731 Run performance tests with enabled mmap IO 2021-03-30 00:04:03 +03:00
tavplubix
3c0f5a57ec
Merge pull request #21535 from ClickHouse/distributed_ddl_improvements 
Distributed DDL improvements
 2021-03-29 22:40:11 +03:00
Ivan Lezhankin
de05803177 Merge branch 'master' into async-insert 2021-03-29 20:59:02 +03:00
kssenii
c17748a0ba Add proper settings 2021-03-27 21:57:35 +00:00
tavplubix
48b89a0ace
Merge pull request #21998 from Avogar/hedged-test 
Update tests for hedged requests
 2021-03-23 18:18:49 +03:00
Pavel Kruglov
dbc5018000 Update tests for hedged requests 2021-03-22 22:18:06 +03:00
Alexander Kuzmenkov
2f5dbf57b6
Merge pull request #21839 from kssenii/add-postgres-connection-pool 
Add connection pool for postgres engine
 2021-03-22 19:49:51 +03:00
tavplubix
5dfd0045a6
Merge branch 'master' into distributed_ddl_improvements 2021-03-22 18:13:19 +03:00
kssenii
ed158e2b77 Proper setting 2021-03-20 14:42:03 +00:00
kssenii
619f0f51fd Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into pg-ch-replica 2021-03-20 09:47:05 +00:00
kssenii
3b32c8fe9a Block on empty connection pool 2021-03-19 21:00:45 +00:00
Ivan Lezhankin
4f1fad97ab Merge remote-tracking branch 'upstream/master' into async-insert 2021-03-19 23:41:01 +03:00
Pavel Kruglov
d380454044 Change read_data_timeout from sec to ms, add profile event for hedged requests and better process receiving packets 2021-03-18 20:16:17 +03:00
kssenii
a03e849a8f Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into pg-ch-replica 2021-03-16 10:06:10 +00:00
tavplubix
6a265f468e
Merge branch 'master' into distributed_ddl_improvements 2021-03-15 11:24:14 +03:00
Alexey Milovidov
bbd6ac2d3a Merge branch 'master' into add_postgresql_engine_docs 2021-03-14 14:22:56 +03:00
Alexander Tokmakov
83b3e4e0f5 allow to use Cluster from Replicated db 2021-03-09 20:05:24 +03:00
vdimir
36006a7e96
Merge branch 'master' into cross-to-inner-join-rewrite 2021-03-09 09:36:15 +03:00
Alexander Tokmakov
163e27b299 support query and session settings for distributed DDL 2021-03-09 01:57:53 +03:00
Alexander Tokmakov
5070b5b85b detach permanently setting for Replicated db 2021-03-09 00:34:52 +03:00
Alexander Tokmakov
2022b90919 configurable output mode for distributed DDL 2021-03-08 23:35:09 +03:00
Azat Khuzhin
16f4c02d42 Add optimize_skip_unused_shards_limit 
Limit for number of sharding key values, turns off
optimize_skip_unused_shards if the limit is reached
 2021-03-26 06:09:00 +03:00
Slach
5791cf5f2b fix review comments https://github.com/ClickHouse/ClickHouse/pull/21078#discussion_r580413875 and https://github.com/ClickHouse/ClickHouse/pull/21078#discussion_r580410204 
Signed-off-by: Slach <bloodjazman@gmail.com>
 2021-03-08 03:39:24 +03:00
Ivan Lezhankin
5cfeedb1e0 Add setting http_max_uri_size 2021-03-04 15:21:23 +03:00
Ivan Lezhankin
74214865eb Merge remote-tracking branch 'upstream/master' into async-insert 2021-03-04 14:14:28 +03:00
Ivan Lezhankin
b726845be2 [WIP] 2021-03-04 14:10:21 +03:00
Pavel Kruglov
dc9dd8ccdc Fix typo 2021-03-03 16:16:17 +03:00
hexiaoting
4fe75ad168 fix error 2021-03-03 17:18:53 +08:00
Pavel Kruglov
82a6be44cf Disable changing replica when we have made progress 2021-03-03 01:03:03 +03:00
Pavel Kruglov
153bfbfc28 Merge branch 'master' of github.com:ClickHouse/ClickHouse into hedged-requests 2021-03-02 11:59:32 +03:00
kssenii
a3c7402200 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into pg-ch-replica 2021-03-02 07:07:56 +00:00
hexiaoting
e116346cbb Merge remote-tracking branch 'origin/master' into dev-sumcount 2021-03-02 10:11:50 +08:00
hexiaoting
16333e9f3a Suppport fuse aggregate functions:sum/avg/count to sumCount 2021-03-01 18:04:34 +08:00
Pavel Kruglov
197058dc12 Merge branch 'master' of github.com:ClickHouse/ClickHouse into hedged-requests 2021-02-27 00:36:33 +03:00
Nikolai Kochetov
d91b8a3acb
Merge branch 'master' into filter-push-down 2021-02-26 19:33:27 +03:00
Nikolai Kochetov
d328bfa41f Review fixes. Add setting max_optimizations_to_apply. 2021-02-26 19:29:56 +03:00
kssenii
4cdb55babe Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into pg-ch-replica 2021-02-22 12:39:35 +00:00
kssenii
ace76bb7fa Better, remove debug logs 2021-02-22 12:35:53 +00:00
alexey-milovidov
50639b7dba
Merge pull request #21052 from ClickHouse/file-engine-settings 
Merging #20620
 2021-02-22 13:10:31 +03:00
Alexey Milovidov
46ac365629 Merge branch 'master' into topvisor/file-engine-settings 2021-02-21 23:10:26 +03:00
Alexey Milovidov
0a62f95bbe Merge branch 'master' into distributed_aggregation_memory_efficient_by_default 2021-02-21 23:06:49 +03:00
Kruglov Pavel
815b4ff0d9
Merge branch 'master' into hedged-requests 2021-02-21 17:10:24 +03:00
alexey-milovidov
20a7ccf5b1
Merge pull request #20174 from amosbird/alias 
Normalized function names (lower-case case-insensitive names)
 2021-02-21 16:23:51 +03:00
tavplubix
a697b578bc
Merge pull request #16193 from ClickHouse/database_replicated 
DatabaseReplicated: continuation of #10485
 2021-02-20 19:39:34 +03:00
vdimir
e052a5a05e
Revert "Support old cross to inner join rewrite behaviour" 
This reverts commit 527210b5e4.
 2021-02-19 15:14:24 +03:00
Amos Bird
2c4bc43014
Backward compatible 2021-02-18 20:20:29 +08:00
Alexey Milovidov
0d2d2dc2ed Merge branch 'master' into sundy-li/system-querylog-map 2021-02-18 00:47:54 +03:00
Pavel Kruglov
1b78de2142 Use fibers in HedgedRequests 2021-02-17 20:34:52 +03:00
vdimir
527210b5e4
Support old cross to inner join rewrite behaviour 2021-02-17 12:44:53 +03:00
Alexey Milovidov
fa20016091 Enable distributed_aggregation_memory_efficient by default 2021-02-17 09:53:18 +03:00
M0r64n
75117389ec Add a couple of QOL file engine settings 2021-02-16 18:50:11 +04:00
Alexander Tokmakov
bf6f64a3fb Merge branch 'master' into database_replicated 2021-02-16 01:28:19 +03:00
vdimir
9afa6b5b1b
Add option cross_to_inner_join_rewrite 2021-02-15 22:40:32 +03:00
Kruglov Pavel
598576ce70
Merge branch 'master' into hedged-requests 2021-02-15 16:51:45 +03:00
Alexander Tokmakov
9c7cf9e92e remove some debug code 2021-02-15 13:26:34 +03:00
Alexey Milovidov
4d650a2a56 Adjust config 2021-02-11 16:41:21 +03:00
Amos Bird
fc6c57f8e0
Fix 2021-02-10 12:48:41 +08:00
Amos Bird
565a148da9
Normalize count() variants 2021-02-10 12:48:41 +08:00
Anton Popov
f1beeb7c25
Merge pull request #20099 from azat/remove-not-existing-setting 
Remove never existing insert_in_memory_parts_timeout
 2021-02-08 16:09:03 +03:00
Pavel Kruglov
f946aab759 Merge branch 'master' of github.com:ClickHouse/ClickHouse into hedged-requests 2021-02-06 17:38:56 +03:00
Kruglov Pavel
d94e8624d7
Merge branch 'master' into shard-id 2021-02-06 16:48:17 +03:00
Azat Khuzhin
48b415d3e4 Remove never existing insert_in_memory_parts_timeout 
Fixes: a43cb93be5 ("remove questionable functionality")
Cc: @CurtizJ
 2021-02-05 01:26:40 +03:00
Aleksei Semiglazov
921518db0a CLICKHOUSE-606: query deduplication based on parts' UUID 
* add the query data deduplication excluding duplicated parts in MergeTree family engines.

query deduplication is based on parts' UUID which should be enabled first with merge_tree setting
assign_part_uuids=1

allow_experimental_query_deduplication setting is to enable part deduplication, default ot false.

data part UUID is a mechanism of giving a data part a unique identifier.
Having UUID and deduplication mechanism provides a potential of moving parts
between shards preserving data consistency on a read path:
duplicated UUIDs will cause root executor to retry query against on of the replica explicitly
asking to exclude encountered duplicated fingerprints during a distributed query execution.

NOTE: this implementation don't provide any knobs to lock part and hence its UUID. Any mutations/merge will
update part's UUID.

* add _part_uuid virtual column, allowing to use UUIDs in predicates.

Signed-off-by: Aleksei Semiglazov <asemiglazov@cloudflare.com>

address comments
 2021-02-02 16:53:39 +00:00
Pavel Kruglov
60a92e9a99 Fix build, add comments, update tests 2021-02-02 15:14:31 +03:00
feng lv
0edf65c094 fix test 
fix

update

fix spell
 2021-02-02 09:22:30 +00:00
feng lv
4279c7da41 add setting insert_shard_id 
add test

fix style

fix
 2021-02-02 04:26:59 +00:00
Vitaliy Zakaznikov
3de4690450 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into live_view_periodic_refresh 2021-01-29 19:57:13 -05:00
Pavel Kruglov
25e85d71ee Merge branch 'master' of github.com:ClickHouse/ClickHouse into hedged-requests 2021-01-29 21:08:47 +03:00
sundy-li
7083ef45a6 Merge remote-tracking branch 'origin/master' into system-querylog-map 2021-01-29 09:26:02 +08:00
sundyli
77194e4a5d
Merge branch 'master' into system-querylog-map 2021-01-28 09:59:41 +08:00
Alexey Milovidov
065d22cf2b Merge branch 'master' into disable-checksums-on-read 2021-01-28 03:07:31 +03:00
alexey-milovidov
2028228f4b
Merge pull request #18684 from amosbird/betterwith 
better CTE and with global
 2021-01-27 04:33:17 +03:00
Alexey Milovidov
a6f5f40a65 Merge branch 'master' into disable-checksums-on-read 2021-01-27 04:09:00 +03:00
Amos Bird
3afcccaad2
Fix global with after union refactoring 2021-01-26 10:29:12 +08:00
Alexey Milovidov
9ee5c1535e Allow to disable checksums on read 2021-01-25 23:29:04 +03:00
Alexey Milovidov
cfd47b2980 Merge branch 'master' into issue18494 2021-01-25 22:18:23 +03:00
sundy-li
685210e02b Merge branch 'master' into system-querylog-map 2021-01-23 11:28:22 +08:00
flynn
e75b116466
Rewrite sum(if()) and sumIf to countIf in special cases (#17041) 
Co-authored-by: vdimir <vdimir@yandex-team.ru>
 2021-01-21 12:01:35 +03:00
sundy-li
22c5fa0974 Merge branch 'master' into system-querylog-map 2021-01-20 23:00:22 +08:00
Pavel Kruglov
97b5179e55 Implement HedgedRequests 2021-01-19 22:41:05 +03:00
Alexander Kuzmenkov
d22c04568d
Merge pull request #13405 from excitoon-favorites/s3keepalive 
Connection pools for S3
 2021-01-18 12:52:37 +03:00