thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-19 04:42:37 +00:00
Code Issues Packages Projects Releases Wiki Activity
111,955 Commits 9,326 Branches 1,783 Tags 2.2 GiB
218b1f9c29
Commit Graph

912 Commits

Author SHA1 Message Date
Azat Khuzhin
218b1f9c29 Add ability to throttle BACKUPs on per-server/backup basis 
Server settings:
- backup_read_bandwidth_for_server
- backup_write_bandwidth_for_server

Query settings:
- backup_read_bandwidth
- backup_write_bandwidth

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-04-05 09:39:48 +02:00
Azat Khuzhin
a25dd1d348 Add ability to throttle local IO on per-query/server basis 
Server settings:
- max_local_read_bandwidth_for_server
- max_local_write_bandwidth_for_server

Query settings:
- max_local_read_bandwidth
- max_local_write_bandwidth

This is the preparation for adding ability to throttle BACKUPs

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-04-05 09:39:15 +02:00
Azat Khuzhin
b3406beeb7 Add per-query network throttling 
Controlled with:
- max_remote_read_network_bandwidth
- max_remote_write_network_bandwidth

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-04-05 09:39:15 +02:00
Azat Khuzhin
abdb682048 Move some methods for Context into module part 
This will be required later, for per-query bandwidth limiting.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-04-05 09:39:15 +02:00
Azat Khuzhin
40cb87c2ac Make remote bandwidth settings server settings 
Right now those two settings are applied only at start:
- max_remote_read_network_bandwidth_for_server
- max_remote_write_network_bandwidth_for_server

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-04-05 09:39:15 +02:00
Antonio Andelic
f5e97fbb66 Merge branch 'master' into alternative-keeper-configs 2023-04-03 15:06:52 +00:00
Azat Khuzhin
edeeac84cf Fix incorrect ThreadPool usage after ThreadPool introspection 
```
$ gg 'ThreadPool[^()]([A-Za-z_]\+,' src/
src/Interpreters/Context.cpp:        shared->load_marks_threadpool = std::make_unique<ThreadPool>(pool_size, pool_size, queue_size);
src/Interpreters/Context.cpp:        shared->prefetch_threadpool = std::make_unique<ThreadPool>(pool_size, pool_size, queue_size);
src/Interpreters/Context.cpp:        shared->threadpool_writer = std::make_unique<ThreadPool>(pool_size, pool_size, queue_size);
```

Fixes: #47880
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-03-30 20:08:38 +02:00
filimonov
fbb22348ea
Refactor reading the pool setting & from server config. (#48055) 
After #36425 there was a lot of confusions/problems with configuring pools - when the message was confusing, and settings need to be ajusted in several places.
See some examples in #44251, #43351, #47900, #46515.

The commit includes the following changes:
1) Introduced a unified mechanism for reading pool sizes from the configuration file(s). Previously, pool sizes were read from the Context.cpp with fallbacks to profiles, whereas main_config_reloader in Server.cpp read them directly without fallbacks.
2) Corrected the data type for background_merges_mutations_concurrency_ratio. It should be float instead of int.
3) Refactored the default values for settings. Previously, they were defined in multiple places throughout the codebase, but they are now defined in one place (or two, to be exact: Settings.h and ServerSettings.h).
4) Improved documentation, including the correct message in system.settings.

Additionally make the code more conform with #46550.
 2023-03-30 16:44:11 +02:00
Raúl Marín
714b54b322 Reimplement enable_fault_injections_during_startup 2023-03-27 15:06:46 +02:00
Antonio Andelic
93f5920e02
Merge branch 'master' into alternative-keeper-configs 2023-03-27 10:00:21 +02:00
Antonio Andelic
d6cbc5d05b Fix tests 2023-03-23 15:01:51 +00:00
Azat Khuzhin
2b439f079e
Make backup_threads/restore_threads server settings (#47881) 
Those settings can be applied only at start, so it is better to have
them in configuration file instead of user profile settings.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-03-23 15:44:15 +01:00
Antonio Andelic
a4d1cd514d Refactor 2023-03-23 08:58:56 +00:00
Antonio Andelic
65b99ec6a7 Merge branch 'master' into alternative-keeper-configs 2023-03-22 12:22:57 +00:00
Robert Schulze
c4664c70a1
Merge branch 'master' into rs/qc-cachebase 2023-03-19 15:23:24 +01:00
serxa
3ad7194e31 add pool size metrics 2023-03-16 17:31:41 +00:00
Robert Schulze
657aa65446
Cleanup: Untangle CacheBase's constructors 
This prepares for the custom TTLCachePolicy which users should not be
able to configure explicitly.
 2023-03-13 15:11:12 +00:00
Antonio Andelic
35c15e6ef8 Merge branch 'master' into custom-key-parallel-replicas 2023-03-07 09:37:38 +00:00
Antonio Andelic
737cf8e149 Better 2023-03-03 15:14:49 +00:00
Antonio Andelic
01cf9c94f4 Merge branch 'master' into custom-key-parallel-replicas 2023-03-02 14:28:42 +00:00
Robert Schulze
740aeaba1f
Apply some CTAD 2023-03-02 13:36:47 +00:00
Alexey Milovidov
d8cda3dbb8 Remove PVS-Studio 2023-02-19 23:30:05 +01:00
Antonio Andelic
ab51c1d975
Merge branch 'master' into custom-key-parallel-replicas 2023-02-17 15:06:41 +01:00
Alexey Milovidov
993a39e2a1
Merge pull request #46360 from genzgd/http_client_version 
Allow to provide the protocol version by HTTP clients to tune the compatibility of the Native output format.
 2023-02-16 22:17:04 +03:00
serxa
a7bbf02bac fix possible deadlock 2023-02-14 19:37:09 +00:00
Antonio Andelic
adde580756
Merge branch 'master' into custom-key-parallel-replicas 2023-02-14 14:09:12 +01:00
Geoff Genz
be8bf3a6a3
Merge branch 'master' into http_client_version 2023-02-13 08:43:59 -07:00
Sergei Trifonov
297587531b
Merge branch 'master' into round-robin-merge-scheduler 2023-02-11 17:19:27 +01:00
serxa
c58b165b0f add config option to select scheduling policy 2023-02-11 16:18:42 +00:00
kssenii
3067c1d723 Merge remote-tracking branch 'upstream/master' into resubmit-prefetches 2023-02-11 11:36:23 +01:00
Geoff Genz
99c3ff53c5 Merge remote-tracking branch 'origin/master' into http_client_version 
# Conflicts:
#	src/Interpreters/Context.cpp
#	src/Interpreters/Context.h
 2023-02-10 04:35:53 -07:00
Geoff Genz
7ed8ed0284 Add support for client_protocol_version sent with HTTP 2023-02-10 03:47:06 -07:00
Vitaly Baranov
39bacfb9dc
Merge branch 'master' into allow-single-disk-instead-of-storage-policy 2023-02-09 13:16:55 +01:00
kssenii
b0b865c32e Resubmit prefetches 2023-02-08 21:26:24 +01:00
Antonio Andelic
f67e5505ab Merge branch 'master' into custom-key-parallel-replicas 2023-02-06 11:12:39 +00:00
Robert Schulze
9b841d67e8
Merge remote-tracking branch 'origin/master' into qc-max_cache_size 2023-02-05 16:47:02 +00:00
kssenii
5abbc7cad6 Cleanup 2023-02-04 19:41:41 +01:00
kssenii
5f124263a6 Revert changes with complex SettingsChanges 2023-02-03 21:04:45 +01:00
Nikita Mikhaylov
33877b5e00
Parallel replicas. Part [2] (#43772) 2023-02-03 14:34:18 +01:00
kssenii
a391de18a1 Merge remote-tracking branch 'upstream/master' into allow-single-disk-instead-of-storage-policy 2023-02-02 20:49:41 +01:00
Robert Schulze
b1889b0284
Make the naming a little less confusing 2023-02-02 19:18:39 +00:00
Robert Schulze
216636f984
Make query cache global settings re-configurable 2023-02-01 10:35:19 +00:00
Robert Schulze
325c6bdf3d
Renaming: "Query Result Cache" --> "Query Cache" 
Reasons:

- The cache will at some point store intermediate results as opposed to
  only query results. We should change the terminology now without
  having to worry about backward compat.

- Equivalent caches in MySQL (1) and Starrocks (2) are called "query
  cache".

- The new name is ca. 13.8% more catchy.

(1) https://dev.mysql.com/doc/refman/5.6/en/query-cache.html
(2) https://docs.starrocks.io/en-us/2.5/using_starrocks/query_cache
 2023-01-31 09:54:34 +00:00
Alexander Tokmakov
067b1f5f13 Merge branch 'master' into exception_message_patterns4 2023-01-26 15:20:58 +01:00
Kseniia Sumarokova
297516a084
Merge pull request #45614 from kssenii/rename-some-setting 
Rename setting `max_query_cache_size`
 2023-01-26 10:46:03 +01:00
kssenii
29de87e145 Rename setting 2023-01-25 16:36:36 +01:00
Alexander Tokmakov
6eb557b2ba Merge branch 'master' into exception_message_patterns4 2023-01-25 13:49:17 +01:00
Smita Kulkarni
642f9ca549 Merge branch 'master' into 45486_Fix_flaky_test_for_disallowing_concurrent_backups_restores 2023-01-24 09:37:28 +01:00
Alexander Tokmakov
bb4c8e169f check number of parameters in format string 2023-01-23 23:16:16 +01:00
Alexander Tokmakov
3f6594f4c6 forbid old ctor of Exception 2023-01-23 22:18:05 +01:00