Commit Graph

133822 Commits

Author SHA1 Message Date
Julia Kartseva
17d2455448 Integration tests for adaptive async insert timeout
Add initial tests.
2024-02-01 22:45:09 +00:00
Julia Kartseva
4ef02a1895 Log timeout for entries processed asynchronously 2024-02-01 22:45:09 +00:00
Julia Kartseva
c04e5a4f35 Asynchronous insert queue size and bytes metrics
Add AsynchronousInsertQueueSize and AsynchronousInsertQueueBytes
metrics to improve observability of asynchronous inserts.
The metrics do not account for tasks dispatched for immediate processing
(as opposed to, e.g., PendingAsyncInsert).

```
SELECT value
FROM system.metrics
WHERE metric IN ('AsynchronousInsertQueueSize', 'PendingAsyncInsert')

Query id: a711dd83-b48d-4ad5-8031-fa59b21a7c38

┌─value─┐
│    18 │
│    23 │
└───────┘
```

```
SELECT value
FROM system.metrics
WHERE metric IN ('AsynchronousInsertQueueSize', 'AsynchronousInsertQueueBytes')

Query id: b35a7ceb-2bb5-46ad-b301-e6cf03508699

┌─value─┐
│    28 │
│  1372 │
└───────┘
```
2024-02-01 22:45:09 +00:00
Julia Kartseva
04c8bd1be5 Adaptive asynchronous insert timeouts
Implement the algorithm described in #56783 for adaptive asynchronous
insert timeouts.
- The adaptive async insert timeout can take values within
[async_insert_busy_timeout_min_ms, async_insert_busy_timeout_max_ms].
- The initial value is set to async_insert_busy_timeout_min_ms.
- If the elapsed time since the most recent queue insert was
  greater than the maximum timeout, process the queue content immediately,
  and reduce the timeout.
- If the elapsed time was long enough (longer than a would-be decreased
  timeout), decrease the timeout.
- The adaptive timeout is changes exponentially based on the
  async_insert_busy_timeout_{increase|decrease}_rate.

Fixes: https://github.com/ClickHouse/ClickHouse/issues/56783
2024-02-01 22:45:06 +00:00
Julia Kartseva
bbaa08199a Introduce polling timeout for asynchronous queue
The current timeout for checking updates in the asynchronous queue is
equal to the timeout used for queue entry
(async_insert_busy_timeout_ms).
That means that, in the worst case, an entry spends twice the time of the
asynchronous timeout in the queue.
2024-02-01 08:27:41 +00:00
Julia Kartseva
bd5529dda1
Merge pull request #59453 from canhld94/fix_test
Fix 02720_row_policy_column_with_dots
2024-02-01 00:27:03 -08:00
Antonio Andelic
2a582f4b18
Merge pull request #59439 from ClickHouse/fix-digest-calculation
Fix digest calculation in Keeper
2024-02-01 09:25:30 +01:00
Dmitry Novik
5b630ed745
Merge pull request #59399 from azat/build/format-fwd-decl
Forward declaration for PeekableReadBuffer
2024-02-01 09:15:07 +01:00
Duc Canh Le
500e8e505d better
Signed-off-by: Duc Canh Le <duccanh.le@ahrefs.com>
2024-02-01 08:03:19 +00:00
Duc Canh Le
009c0dc136 fix 02720_row_policy_column_with_dots
Signed-off-by: Duc Canh Le <duccanh.le@ahrefs.com>
2024-02-01 05:27:24 +00:00
Alexey Milovidov
0fca83f5f3
Merge pull request #59309 from aris-aiven/aris-fix-corrosion-cmake-osx
Update condition required to define Rust target toolchain in corrosion-cmake
2024-01-31 23:50:54 +01:00
robot-clickhouse
b49ae582ce
Merge pull request #59449 from seandhaynes/bench-docs
Update bench docs to reflect removal of -p (port) short option
2024-01-31 23:28:11 +01:00
Alexey Milovidov
8d2f3508f2
Merge pull request #58767 from gingerwizard/weather-data
weather data
2024-01-31 23:12:08 +01:00
Alexey Milovidov
5128023e8f
Merge pull request #59350 from AVMusorin/clickhouse-client-docs
docs: added hidden clickhouse-client shortkeys and aliases
2024-01-31 23:11:25 +01:00
Alexey Milovidov
31e99ee61f
Update clickhouse-benchmark.md 2024-02-01 01:01:59 +03:00
Sean Haynes
ed3d5ce0a3 Update bench docs to reflect removal of -p (port) short option 2024-01-31 21:52:52 +00:00
Ilya Yatsishin
6a80a20997
Merge pull request #59432 from vitlibar/fix-test-test_stop_other_host_during_backup
Fix test test_stop_other_host_during_backup
2024-02-01 00:57:21 +04:00
Alexey Milovidov
554bb5668e
Merge pull request #59384 from ClibMouse/update_curl_to_8.50
update curl submodule to be version 8.50 to address CVE-2023-46218 and CVE-2023-49219
2024-01-31 20:14:06 +01:00
Alexey Milovidov
f3b539dcf6
Merge pull request #59386 from ClibMouse/update_libxml2
Update libxml2 version to address some bogus security issues
2024-01-31 20:13:47 +01:00
Mikhail f. Shiryaev
c3fde1a7f6
Merge pull request #59440 from ClickHouse/run-check-status
Post a failure status if can not run the CI
2024-01-31 19:10:15 +01:00
Antonio Andelic
cd8e207531
Fix tag 2024-01-31 18:45:42 +01:00
Mikhail f. Shiryaev
0f931057e9
Post a failure status if can not run the CI 2024-01-31 17:59:13 +01:00
Kseniia Sumarokova
0d864ebfc0
Merge pull request #59433 from ClickHouse/kssenii-patch-8
Update run.sh
2024-01-31 17:57:19 +01:00
Antonio Andelic
08f91907dc Fix digest calculation 2024-01-31 16:08:22 +00:00
Vitaly Baranov
a193e01c8f
Merge pull request #54737 from arenadata/ADQM-1150
Added maximum sequential login failures to the quota.
2024-01-31 16:06:17 +01:00
Mikhail f. Shiryaev
3cc43c4056
Merge pull request #59395 from ClickHouse/job-names
Job names
2024-01-31 15:13:44 +01:00
Raúl Marín
dad6ea0930
Merge pull request #59239 from Algunenano/parallel_replicas_cte
Disable parallel replicas JOIN with CTE (not analyzer)
2024-01-31 14:45:13 +01:00
Kruglov Pavel
4987708583
Merge pull request #59298 from Avogar/fix-getting-filename-from-read-buffers
Fix getting filename from read buffer wrappers
2024-01-31 14:34:07 +01:00
Mikhail f. Shiryaev
bfa722bf62
Use JobNames.STYLE_CHECK for consistency 2024-01-31 13:58:25 +01:00
Mikhail f. Shiryaev
7af7255b90
Use JobNames.DOCS_CHECK for consistency 2024-01-31 13:58:24 +01:00
Mikhail f. Shiryaev
3dae643f19
Consistent "Fast test" job name 2024-01-31 13:58:23 +01:00
Kseniia Sumarokova
60c37fb9bf
Update run.sh 2024-01-31 13:44:16 +01:00
Kruglov Pavel
325419b096
Merge pull request #59398 from Avogar/variant-type-func
Add function variantType
2024-01-31 13:06:54 +01:00
Vitaly Baranov
79c068571d Fix test test_stop_other_host_during_backup. 2024-01-31 12:58:22 +01:00
Raúl Marín
f67bff12b7
Merge pull request #59148 from bigo-sg/improve_if_with_floating
Continue optimizing branch miss of if function when result type is float*/decimal*/int*
2024-01-31 12:42:06 +01:00
Nikolai Kochetov
7e722b52a9
Merge pull request #59003 from kitaisreal/revert-57519-revert-53562-group_sorted_array_function
Revert "Revert "Add new aggregation function groupArraySorted()""
2024-01-31 12:31:26 +01:00
Raúl Marín
f875151b97
Merge pull request #59379 from Algunenano/fix_translate_asserts
Add sanity checks for function return types
2024-01-31 11:56:11 +01:00
Kseniia Sumarokova
b20567a055
Merge pull request #59167 from ClickHouse/s3-queue-parallelize-ordered-mode
S3Queue: allow parallel & disrtibuted processing for ordered mode
2024-01-31 11:20:41 +01:00
Kseniia Sumarokova
caaafbcedb
Merge pull request #56864 from ClickHouse/broken-projections-better-handling
Do not consider data part as broken if projection is broken
2024-01-31 11:17:15 +01:00
Raúl Marín
d45b248f96
Merge pull request #59356 from Algunenano/fix_translate
Fix translate() with FixedString input
2024-01-31 11:14:53 +01:00
Max K
5c8ab8bf52
Merge pull request #59396 from ClickHouse/fix_docker_server_job
CI: fix status and report for docker server jobs
2024-01-31 10:47:56 +01:00
Dmitry Novik
ac17ad1e66
Merge pull request #59073 from ClickHouse/fix-analyzer-resolve-distr-id
Analyzer: Do not resolve remote table id on initiator
2024-01-31 09:08:30 +01:00
taiyang-li
818fb9802a change as request 2024-01-31 11:48:02 +08:00
Alexey Gerasimchuck
076fb1de31 Updated the authentication failures counter logic 2024-01-30 23:52:33 +00:00
Dale Mcdiarmid
db84527dd0 more words 2024-01-30 20:39:37 +00:00
Dale Mcdiarmid
4dbf184da6 Merge branch 'weather-data' of github.com:gingerwizard/ClickHouse into weather-data 2024-01-30 20:38:38 +00:00
Dale Mcdiarmid
0f5cb76a61 more words 2024-01-30 20:37:51 +00:00
Kruglov Pavel
280e04abb2
Fix test one more time 2024-01-30 20:50:28 +01:00
Kruglov Pavel
bd3bb05308
Fix 2024-01-30 20:32:59 +01:00
Alexey Milovidov
797ff834e0
Merge pull request #59397 from ClickHouse/auto/v24.1.1.2048-stable
Update version_date.tsv and changelogs after v24.1.1.2048-stable
2024-01-30 20:31:25 +01:00