In case of ECONNRESET (Connection reset by peer) the "cancelled" will
not be set, and so drain() will fail with LOGICAL_ERROR:
<details>
2021.10.16 04:10:55.054771 [ 7635 ] {882164e0-ca26-4955-8759-708ba1f6e85c} <Error> PullingAsyncPipelineExecutor: Code: 210. DB::NetException: Connection reset by peer, while writing to socket (127.0.0.2:9000). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below):
...
4. ./obj-x86_64-linux-gnu/../src/Common/NetException.h:12: DB::NetException::NetException(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int) @ 0x20db5749 in /usr/bin/clickhouse
5. ./obj-x86_64-linux-gnu/../src/IO/WriteBufferFromPocoSocket.cpp:56: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x211d56bb in /usr/bin/clickhouse
6. ./obj-x86_64-linux-gnu/../src/IO/WriteBuffer.h:47: DB::WriteBuffer::next() @ 0x14e4c874 in /usr/bin/clickhouse
7. ./obj-x86_64-linux-gnu/../src/Client/Connection.cpp:560: DB::Connection::sendCancel() @ 0x227c0c22 in /usr/bin/clickhouse
8. ./obj-x86_64-linux-gnu/../src/Client/HedgedConnections.cpp:242: DB::HedgedConnections::sendCancel() @ 0x227f1b53 in /usr/bin/clickhouse
...
2021.10.16 04:12:35.555530 [ 5575 ] {} <Fatal> : Logical error: 'Cannot drain connections: cancel first.'.
</details>
Found by stress tests on CI [1].
[1]: https://clickhouse-test-reports.s3.yandex.net/0/d3cb99701d4aaec384fd183168d10e0f650ecc1d/stress_test_(debug).html#fail1
This will allow to use artifacts as-is, i.e.:
clickhouse-local --path /path/to/log/artifact
v2: convert database to Ordinary to avoid extra symlinks dependencies.
Under TSan using too much stack requires too much RSS for shadow memory,
and neither of this TSAN_OPTIONS helps:
- history_size=2
- flush_memory_ms=2000
- memory_limit_mb=50000
So instead, decrease allowed limit of the stack size in checkStackSize()
under TSan, to address exessive memory usage for the server compiled
with TSan.
Note, that before this patch 01763_max_distributed_depth test can
increase RSS of the server to 70GiB.
After #26466 you cannot get inaccurate results anymore, since
distributed_push_down_limit changes query execution only if:
- `distributed_group_by_no_merge` > 0
- query **does not have `GROUP BY`/`DISTINCT`/`LIMIT BY`**, but it has `ORDER BY`/`LIMIT`.
- query **has `GROUP BY`/`DISTINCT`/`LIMIT BY`** with `ORDER BY`/`LIMIT` and:
- `optimize_skip_unused_shards_limit` is enabled
- `optimize_distributed_group_by_sharding_key` is enabled
