Commit Graph

32 Commits

Author SHA1 Message Date
Nikolai Kochetov
a08c98d760 Move some files. 2021-10-16 17:03:50 +03:00
Amos Bird
479d4fa991
Add shardNum() and shardCount() functions 2021-07-31 15:45:39 +08:00
Nikolai Kochetov
f36d14f68f Add separate step to read from remote. 2021-07-15 19:15:16 +03:00
Azat Khuzhin
57da47e1fd Refactor code around optimize_skip_unused_shards_rewrite_in a little 2021-07-07 00:17:39 +03:00
Azat Khuzhin
8164c49f04 Fix limit/offset settings for distributed queries (ignore on the remote nodes) 2021-06-03 21:14:24 +03:00
Nikolai Kochetov
afc1fe7f3d Make ContextPtr const by default. 2021-05-31 17:49:02 +03:00
Azat Khuzhin
79bd8d4d3f Respect optimize_skip_unused_shards_rewrite_in with optimize_skip_unused_shards_limit 2021-04-12 10:37:28 +03:00
Azat Khuzhin
fbb386dca5 Rewrite IN in query for remote shards to exclude values that does not belongs to shard
v2: fix optimize_skip_unused_shards_rewrite_in for sharding_key wrapped into function
v3: fix column name for optimize_skip_unused_shards_rewrite_in
v4: fix optimize_skip_unused_shards_rewrite_in with Null
v5:
- squash with Remove query argument for IStreamFactory::createForShard()
- use proper column after function execution (using sharding_key_column_name)
- update the test reference since (X) now is tuple(X)
2021-04-12 10:37:28 +03:00
Ivan
495c6e03aa
Replace all Context references with std::weak_ptr (#22297)
* Replace all Context references with std::weak_ptr

* Fix shared context captured by value

* Fix build

* Fix Context with named sessions

* Fix copy context

* Fix gcc build

* Merge with master and fix build

* Fix gcc-9 build
2021-04-11 02:33:54 +03:00
Alexander Kuzmenkov
0264124146
Merge pull request #21942 from ucasFL/distributed_depth
Add settings max_distributed_depth
2021-04-09 15:52:58 +03:00
feng lv
56073db22d max distributed depth
Add settings max_distributed_depth

fix style

fix

fix

fix

fix test

fix

fix
2021-04-05 14:00:54 +00:00
Nikolai Kochetov
0e2f52518f Extract converting from UnionStep. 2021-03-25 12:57:14 +03:00
Azat Khuzhin
77a1d00dee Add new remote context as interpreter context to the Pipe 2020-12-12 17:43:10 +03:00
Nikolai Kochetov
81386689d9
Update executeQuery.cpp 2020-11-25 12:59:21 +03:00
Azat Khuzhin
97c34a0fc9 Pass logger for the RemoteQueryExecutor
So now you will get additional message:

    2020.11.21 13:37:15.429767 [ 380840 ] {47e1540d-f4cd-4f2f-9383-f1216e8328dc} <Trace> StorageDistributed (dist_01247): (127.0.0.2:9000) Cancelling query
2020-11-21 13:37:32 +03:00
Nikolai Kochetov
195c941c4e Merge branch 'master' into storage-read-query-plan 2020-11-10 15:02:22 +03:00
Alexey Milovidov
1ea3afadbc Merge with master 2020-11-08 00:28:39 +03:00
Alexey Milovidov
5314185e25 Merge branch 'master' into azat-optimize_skip_unused_shards-optimization 2020-11-08 00:17:59 +03:00
Nikolai Kochetov
f9bf1e3406 Merge branch 'master' into storage-read-query-plan 2020-10-06 09:50:10 +03:00
Azat Khuzhin
f25c1742b8 Pass through *_for_user settings via Distributed with cluster-secure
In cluster-secure case the user on shards (remote) is equal to the user
on the initiator, so those settings can be safely applied.
2020-10-03 02:04:47 +03:00
Azat Khuzhin
587cde853e Avoid skipping unused shards twice (for query processing stage and read itself) 2020-10-02 22:42:09 +03:00
Nikolai Kochetov
b26f11c00c Support StorageDistributed::read for QueryPlan. 2020-09-18 17:16:53 +03:00
Nikolai Kochetov
2cca4d5fcf Refactor Pipe [part 2]. 2020-08-03 16:54:14 +03:00
Nikolai Kochetov
e411916bde Refactor Pipe [part 1]. 2020-08-03 14:33:11 +03:00
Azat Khuzhin
177d699cde Do not check *optimize_skip_unused_shards_nesting w/o *optimize_skip_unused_shards
This will avoid supperior log message in case of
*optimize_skip_unused_shards_nesting already disables it.
And also it is logical.
2020-06-29 23:00:14 +03:00
alexey-milovidov
38b018fa04
Update executeQuery.cpp 2020-06-21 21:29:11 +03:00
Azat Khuzhin
fb30629ff2 Add settings to control nesting level for shards skipping optimization
- optimize_skip_unused_shards_nesting (allows control nesting level for
  shards skipping optimization)
- force_skip_optimize_shards_nesting (allows control nesting level for
  checking was shards skipped or not)
- deprecates force_skip_optimize_shards_no_nested
2020-06-21 15:27:01 +03:00
Azat Khuzhin
d34e6217bc Add logging of adjusting conditional settings for distributed queries 2020-06-18 21:49:29 +03:00
Azat Khuzhin
6507247328 optimize_skip_unused_shards=2 will disable it for nested distributed queries
P.S. Looks like settings can be converted between SettingUInt64 and
SettingBool without breaking binary protocol.

FWIW maybe it is a good idea to change the semantics of the settings as
follow (but I guess that changing semantic is not a good idea, better to
add new settings and deprecate old ones):
- optimize_skip_unused_shards -- accept nesting level on which the
  optimization will work
- force_skip_optimize_shards_nesting -- accept nesting level on which
  the optimization will work
2020-06-18 21:49:29 +03:00
Alexey Milovidov
0a7edce036 Checkpoint 2020-04-22 09:01:33 +03:00
Alexey Milovidov
2987e70aab Introduce "max_server_memory_usage" setting 2020-04-20 01:59:21 +03:00
Ivan Lezhankin
06446b4f08 dbms/ → src/ 2020-04-03 18:14:31 +03:00