- clang-tidy rightfully complains (-readability-static-accessed-through-instance)
- not going to enable the warning for now to avoid breaking the build
In addition to the time since the most recent insert,
consider the elapsed time between the two recent queue
flushes when decreasing the timeout or processing an
entry synchronously.
Add AsynchronousInsertQueueSize and AsynchronousInsertQueueBytes
metrics to improve observability of asynchronous inserts.
The metrics do not account for tasks dispatched for immediate processing
(as opposed to, e.g., PendingAsyncInsert).
```
SELECT value
FROM system.metrics
WHERE metric IN ('AsynchronousInsertQueueSize', 'PendingAsyncInsert')
Query id: a711dd83-b48d-4ad5-8031-fa59b21a7c38
┌─value─┐
│ 18 │
│ 23 │
└───────┘
```
```
SELECT value
FROM system.metrics
WHERE metric IN ('AsynchronousInsertQueueSize', 'AsynchronousInsertQueueBytes')
Query id: b35a7ceb-2bb5-46ad-b301-e6cf03508699
┌─value─┐
│ 28 │
│ 1372 │
└───────┘
```
Implement the algorithm described in #56783 for adaptive asynchronous
insert timeouts.
- The adaptive async insert timeout can take values within
[async_insert_busy_timeout_min_ms, async_insert_busy_timeout_max_ms].
- The initial value is set to async_insert_busy_timeout_min_ms.
- If the elapsed time since the most recent queue insert was
greater than the maximum timeout, process the queue content immediately,
and reduce the timeout.
- If the elapsed time was long enough (longer than a would-be decreased
timeout), decrease the timeout.
- The adaptive timeout is changes exponentially based on the
async_insert_busy_timeout_{increase|decrease}_rate.
Fixes: https://github.com/ClickHouse/ClickHouse/issues/56783
The current timeout for checking updates in the asynchronous queue is
equal to the timeout used for queue entry
(async_insert_busy_timeout_ms).
That means that, in the worst case, an entry spends twice the time of the
asynchronous timeout in the queue.
* support async insert for native protocol
* use separate queue for async inserts via native protocol
* fix test
* better logging for async inserts and more tests
* disable mixed internal and external data in async inserts
* fix tests
* fix quota in async inserts
* disable async insert for secondary query of distributed
