Commit Graph

96 Commits

Author SHA1 Message Date
Anton Popov
e911900054 remove last mentions of data streams 2022-05-09 19:15:24 +00:00
Amos Bird
4a5e4274f0
base should not depend on Common 2022-04-29 10:26:35 +08:00
Nikita Mikhaylov
224f4dc620
Made parallel_reading_from_replicas work with localhost replica (#36281) 2022-04-22 15:52:38 +02:00
Anton Popov
36ec379aeb Merge remote-tracking branch 'upstream/master' into HEAD 2022-03-14 16:28:35 +00:00
Azat Khuzhin
c4b6342853
Improvements for parallel_distributed_insert_select (and related) (#34728)
* Add a warning if parallel_distributed_insert_select was ignored

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>

* Respect max_distributed_depth for parallel_distributed_insert_select

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>

* Print warning for non applied parallel_distributed_insert_select only for initial query

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>

* Remove Cluster::getHashOfAddresses()

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>

* Forbid parallel_distributed_insert_select for remote()/cluster() with different addresses

Before it uses empty cluster name (getClusterName()) which is not
correct, compare all addresses instead.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>

* Fix max_distributed_depth check

max_distributed_depth=1 must mean not more then one distributed query,
not two, since max_distributed_depth=0 means no limit, and
distribute_depth is 0 for the first query.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>

* Fix INSERT INTO remote()/cluster() with parallel_distributed_insert_select

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>

* Add a test for parallel_distributed_insert_select with cluster()/remote()

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>

* Return <remote> instead of empty cluster name in Distributed engine

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>

* Make user with sharding_key and w/o in remote()/cluster() identical

Before with sharding_key the user was "default", while w/o it it was
empty.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-03-08 15:24:39 +01:00
Anton Popov
78b9f15abb Merge remote-tracking branch 'upstream/master' into HEAD 2022-01-30 03:24:37 +03:00
Azat Khuzhin
1637c41d42 Remove leftovers of old _shard_num via identifier implementation 2022-01-10 21:21:24 +03:00
Anton Popov
99ebabd822 Merge remote-tracking branch 'upstream/master' into HEAD 2021-12-17 19:02:29 +03:00
Alexey Milovidov
5c90ed2ed9 Unambiguous formatting of distributed queries 2021-12-10 00:55:14 +03:00
Nikita Mikhaylov
dbf5091016
Parallel reading from replicas (#29279) 2021-12-09 13:39:28 +03:00
Anton Popov
a20922b2d3 Merge remote-tracking branch 'origin/sparse-serialization' into HEAD 2021-11-09 15:36:25 +03:00
Nikolai Kochetov
a08c98d760 Move some files. 2021-10-16 17:03:50 +03:00
Alexey Milovidov
fe6b7c77c7 Rename "common" to "base" 2021-10-02 10:13:14 +03:00
Anton Popov
e36736b50c Merge remote-tracking branch 'origin/sparse-serialization' into HEAD 2021-08-02 22:52:02 +03:00
Amos Bird
479d4fa991
Add shardNum() and shardCount() functions 2021-07-31 15:45:39 +08:00
Anton Popov
2b58f39c10 dynamic columns: support missed columns in distributed 2021-07-23 19:34:23 +03:00
Nikolai Kochetov
a30b0eb542 Fix style. 2021-07-16 14:33:01 +03:00
Nikolai Kochetov
d3099e0c89 Fix build and add comment. 2021-07-16 13:56:13 +03:00
Nikolai Kochetov
f36d14f68f Add separate step to read from remote. 2021-07-15 19:15:16 +03:00
Azat Khuzhin
57da47e1fd Refactor code around optimize_skip_unused_shards_rewrite_in a little 2021-07-07 00:17:39 +03:00
Azat Khuzhin
8164c49f04 Fix limit/offset settings for distributed queries (ignore on the remote nodes) 2021-06-03 21:14:24 +03:00
Nikolai Kochetov
afc1fe7f3d Make ContextPtr const by default. 2021-05-31 17:49:02 +03:00
Nikolai Kochetov
9f28d0966a Fix typos. 2021-05-19 15:43:10 +03:00
Nikolai Kochetov
c07043fd97 Try fix more cases. 2021-05-19 15:16:18 +03:00
Nikolai Kochetov
9d152a3c4d
Update src/Interpreters/ClusterProxy/SelectStreamFactory.cpp
Co-authored-by: Azat Khuzhin <a3at.mail@gmail.com>
2021-05-19 13:40:46 +03:00
Nikolai Kochetov
85c0844439
Update SelectStreamFactory.cpp
Fix typo
2021-05-18 19:19:16 +03:00
Nikolai Kochetov
57f5e33464 Try to fix GROUP BY _shard_num 2021-05-18 18:17:19 +03:00
Azat Khuzhin
79bd8d4d3f Respect optimize_skip_unused_shards_rewrite_in with optimize_skip_unused_shards_limit 2021-04-12 10:37:28 +03:00
Azat Khuzhin
fbb386dca5 Rewrite IN in query for remote shards to exclude values that does not belongs to shard
v2: fix optimize_skip_unused_shards_rewrite_in for sharding_key wrapped into function
v3: fix column name for optimize_skip_unused_shards_rewrite_in
v4: fix optimize_skip_unused_shards_rewrite_in with Null
v5:
- squash with Remove query argument for IStreamFactory::createForShard()
- use proper column after function execution (using sharding_key_column_name)
- update the test reference since (X) now is tuple(X)
2021-04-12 10:37:28 +03:00
Ivan
495c6e03aa
Replace all Context references with std::weak_ptr (#22297)
* Replace all Context references with std::weak_ptr

* Fix shared context captured by value

* Fix build

* Fix Context with named sessions

* Fix copy context

* Fix gcc build

* Merge with master and fix build

* Fix gcc-9 build
2021-04-11 02:33:54 +03:00
Alexander Kuzmenkov
0264124146
Merge pull request #21942 from ucasFL/distributed_depth
Add settings max_distributed_depth
2021-04-09 15:52:58 +03:00
feng lv
f6e4e13f89 fix conflict 2021-04-05 14:08:30 +00:00
feng lv
56073db22d max distributed depth
Add settings max_distributed_depth

fix style

fix

fix

fix

fix test

fix

fix
2021-04-05 14:00:54 +00:00
Nikolai Kochetov
0e2f52518f Extract converting from UnionStep. 2021-03-25 12:57:14 +03:00
Nikolai Kochetov
9a39459888 Refactor ActionsDAG 2021-03-04 20:38:12 +03:00
Nikolai Kochetov
15921fbfcb Remove index by name from ActionsDAG 2021-03-03 23:01:07 +03:00
Nikolai Kochetov
d328bfa41f Review fixes. Add setting max_optimizations_to_apply. 2021-02-26 19:29:56 +03:00
Nikolai Kochetov
c7ef57c6fd Add setting async_socket_for_remote 2020-12-18 16:15:03 +03:00
Azat Khuzhin
77a1d00dee Add new remote context as interpreter context to the Pipe 2020-12-12 17:43:10 +03:00
Azat Khuzhin
5b3ab48861 More forward declaration for generic headers
The following headers are pretty generic, so use forward declaration as
much as possible:
- Context.h
- Settings.h
- ConnectionTimeouts.h
(Also this shows that some missing some includes -- this has been fixed)

And split ConnectionTimeouts.h into ConnectionTimeoutsContext.h (since
module part cannot be added for it, due to recursive build dependencies
that will be introduced)

Also remove Settings from the RemoteBlockInputStream/RemoteQueryExecutor
and just pass the context, since settings was passed only in speicifc
places, that can allow making a copy of Context (i.e. Copier).

Approx results (How much units will be recompiled after changing file X?):

- ConnectionTimeouts.h
  - mainline: 100

- Context.h:
  - mainline: ~800
  - patched:  415

- Settings.h:
  - mainline: 900-1K
  - patched:  440 (most of them because of the Context.h)
2020-12-12 17:43:10 +03:00
alexey-milovidov
fb3a69b298
Merge pull request #17254 from azat/fix-dist-query-cancelation
Fix "Unexpected packet Data received from client"  for Distributed queries
2020-11-26 03:46:27 +03:00
Nikolai Kochetov
81386689d9
Update executeQuery.cpp 2020-11-25 12:59:21 +03:00
Azat Khuzhin
97c34a0fc9 Pass logger for the RemoteQueryExecutor
So now you will get additional message:

    2020.11.21 13:37:15.429767 [ 380840 ] {47e1540d-f4cd-4f2f-9383-f1216e8328dc} <Trace> StorageDistributed (dist_01247): (127.0.0.2:9000) Cancelling query
2020-11-21 13:37:32 +03:00
Nikolai Kochetov
13eceff55c Remove Converting transform and step. 2020-11-17 20:16:55 +03:00
tavplubix
058aa8f85e
Merge pull request #16824 from ClickHouse/replace_stringstreams_with_buffers
Replace std::*stringstreams with DB::*Buffers
2020-11-12 01:11:44 +03:00
Nikolai Kochetov
195c941c4e Merge branch 'master' into storage-read-query-plan 2020-11-10 15:02:22 +03:00
Alexander Tokmakov
62ff00ee8b use WriteBuffer in formatAST(...) 2020-11-09 19:05:40 +03:00
alexey-milovidov
f4ba5f1f9a
Merge pull request #16772 from ClickHouse/fix-stringstream
Fix "server failed to start" error
2020-11-08 14:27:08 +03:00
Alexey Milovidov
1ea3afadbc Merge with master 2020-11-08 00:28:39 +03:00
Alexey Milovidov
5314185e25 Merge branch 'master' into azat-optimize_skip_unused_shards-optimization 2020-11-08 00:17:59 +03:00