Commit Graph

205 Commits

Author SHA1 Message Date
Nikolai Kochetov
78a7665f43 Merge branch 'master' into rewrite-pushing-to-views 2021-09-27 10:56:50 +03:00
Alexey Milovidov
cd7f9d981c Remove ya.make 2021-09-25 04:22:54 +03:00
Nikolai Kochetov
998d29ebc7 Merge branch 'master' into rewrite-pushing-to-views 2021-09-23 13:10:27 +03:00
Azat Khuzhin
015695b3bf Fix connection timeouts (send_timeout/receive_timeout)
Query is not executed within sendQuery() function.

INSERT query consist from multiple parts:
- sendQuery()
- write()/writePrepared() -- send Data blocks
- writeSuffix() -- send empty Data block and receive EndOfStream
(see RemoteBlockOutputStream for more info)
So as you can see it is not executed completely from the sendQuery() function.

SELECT query (and others) also send Data blocks to the client after.

And what this means that temporary timeout (via TimeoutSetter) is not
enough, since next query can use timeout from the previous query.

Usually you do not see it, because:
- You don't use custom send_timeout/receive_timeout
- SELECT is fast enough

However it is not the case with INSERT, especially with
insert_distributed_sync=1, since
DistributedBlockOutputStream::writeSuffix() may stack because remote
server still INSERT'ing the data (i.e. creating znodes in zookeeper for
ReplicatedMergeTree, syncing blocks, and similar) that was issued from
DistributedBlockOutputStream::write(), and it will lead to the following
exception:

    Code: 209, e.displayText() = DB::NetException: Timeout exceeded while reading from socket (127.1:9000): while receiving packet from ch-47-drt.dpa.semrush.net:9000: Insertion status:
    Wrote X blocks and Y rows on shard Z replica 0, 127.1:9000 (average 0 ms per block, the slowest block N ms)
    0. DB::Exception::Exception()
    1. DB::ReadBufferFromPocoSocket::nextImpl()
    2. void DB::readVarUIntImpl<false>(unsigned long&, DB::ReadBuffer&)
    3. DB::Connection::receivePacket()
    4. DB::RemoteBlockOutputStream::writeSuffix()
    ...
2021-09-23 09:03:13 +03:00
Azat Khuzhin
e5f66fd103 Add socket timeout values into SOCKET_TIMEOUT exception 2021-09-20 09:51:42 +03:00
Nikolai Kochetov
b997214620 Rename QueryPipeline to QueryPipelineBuilder. 2021-09-14 20:48:18 +03:00
Sergei Semin
f74350c148 fix special build 2021-09-12 02:49:31 +03:00
Alexey Milovidov
19445ac096 Drop unused headers 2021-09-01 02:47:52 +03:00
Dmitrii Kovalkov
9871ad70ff Exclude fuzzers 2021-08-30 11:12:25 +03:00
Amos Bird
27ff081403
Fix some leftover TODOs 2021-08-17 17:22:12 +08:00
Nikolai Kochetov
6951e8147d
Merge pull request #27020 from amosbird/shardlevelconstness
Shard-level const column
2021-08-02 13:45:20 +03:00
Amos Bird
b76a854f5a
backward compatible 2021-08-01 09:06:16 +08:00
Nikolai Kochetov
cfa3fd4b87 Hold context in HedgedConnections to prevent use-after-free on settings. 2021-07-29 13:29:36 +03:00
Nikolai Kochetov
a9effdad18 Merge branch 'master' into output-streams-to-processors 2021-07-27 11:31:53 +03:00
Nikolai Kochetov
7da5bb3871 Do not start new hedged connection if query was already canceled. 2021-07-26 20:48:38 +03:00
Nikolai Kochetov
61d8f880cd Rename some files. 2021-07-26 19:48:25 +03:00
alexey-milovidov
7b76bfc719
Merge pull request #26318 from amosbird/connectionpoolfactory
Connection pool cache
2021-07-26 06:45:03 +03:00
Amos Bird
2ba6ef1487
Better implementation 2021-07-24 18:02:51 +08:00
vdimir
dccc379d39
Fix use after free in AsyncDrain connection from S3Cluster 2021-07-23 10:40:03 +03:00
Nikolai Kochetov
20b2837b27 Fix destructor for PocoSocketWrapper. 2021-07-20 16:50:31 +03:00
Amos Bird
dbfb699690
Asynchronously drain connections. 2021-07-19 21:53:29 +08:00
Nikolai Kochetov
0d549c197f Fix 01822_async_read_from_socket_crash for hedged connection. 2021-07-16 13:42:22 +03:00
Amos Bird
3f05211b09
Connection pool factory. 2021-07-15 00:01:45 +08:00
Alexey Milovidov
e3653e70da Update prompt in client when reconnecting 2021-06-15 04:43:35 +03:00
Alexey Milovidov
273226de32 Remove string parameter for Density 2021-05-24 06:43:25 +03:00
Alexey Milovidov
40d4f0678f Remove overload (harmful) 2021-05-23 04:25:06 +03:00
fibersel
cb53bbb7b0 add experimental codecs flag, add integration test for experimental codecs 2021-05-06 14:57:22 +03:00
alexey-milovidov
7ca42d2ce1
Merge pull request #23805 from ClickHouse/fix-hedged-requests-ext-tables
Fix sending external table for hedged connections.
2021-05-01 08:13:13 +03:00
Nikolai Kochetov
352ec049ad Fix sending external table for hedged connections. 2021-04-30 17:19:48 +03:00
Maksim Kita
318c4bb80d Add examples folder filter to ya.make.in 2021-04-30 11:25:52 +03:00
Maksim Kita
0d0a14a925 Poco::Timespan reference fix 2021-04-29 19:11:20 +03:00
Nikita Mikhaylov
9f55424250 move to examples everywhere 2021-04-27 01:51:42 +03:00
Nikolai Kochetov
548fb52d24 Fix build. 2021-04-20 08:51:51 +03:00
Nikolai Kochetov
08d1896950 Fix build. 2021-04-19 22:06:05 +03:00
Nikolai Kochetov
0420b7bb33 Reset replica state in MultiplexedConnections in case of exception. 2021-04-19 17:25:43 +03:00
Alexey Milovidov
17d04cbca9 Merge branch 'master' into protocol-compression-auto 2021-04-15 00:24:07 +03:00
Alexey Milovidov
6f56c3280f Uncompress data in Distributed sends if needed 2021-04-14 00:53:39 +03:00
Nikita Mikhaylov
024374a2ec review fixes 2021-04-13 22:39:43 +03:00
Nikita Mikhaylov
7a68820342 style 2021-04-13 22:39:42 +03:00
Nikita Mikhaylov
c333c3dedb review fixes 2021-04-13 22:39:42 +03:00
Nikita Mikhaylov
36a8419f60 style 2021-04-13 22:39:42 +03:00
Nikita Mikhaylov
2555ae5d3f better processing stage 2021-04-13 22:39:42 +03:00
Nikita Mikhaylov
4843f86329 use only one connection between initiator and worker 2021-04-13 22:39:42 +03:00
Nikita Mikhaylov
e601a432e5 better[2] 2021-04-13 22:39:42 +03:00
Nikita Mikhaylov
cef9e19eb2 better 2021-04-13 22:39:41 +03:00
Nikita Mikhaylov
2549468c14 better 2021-04-13 22:39:41 +03:00
Nikita Mikhaylov
64b4cd0e63 save 2021-04-13 22:39:41 +03:00
Nikita Mikhaylov
31b4f9b17f save 2021-04-13 22:39:41 +03:00
Nikita Mikhaylov
44ca65a9a4 save 2021-04-13 22:39:41 +03:00
Alexey Milovidov
fafe8c00fe Fix warning #22987 2021-04-12 20:57:03 +03:00