Commit Graph

142 Commits

Author SHA1 Message Date
Antonio Andelic
bdf8c859b1 Merge branch 'master' into read-cgroup-memory-usage-async-metrics 2024-09-10 14:13:48 +02:00
Alexey Milovidov
7071942858 Miscellanous changes from #66999 2024-08-18 09:05:45 +02:00
Antonio Andelic
0d38aa98ce Merge branch 'master' into read-cgroup-memory-usage-async-metrics 2024-08-09 15:44:47 +02:00
Nikita Mikhaylov
f37e0c7fc6 Merge branch 'master' of github.com:ClickHouse/ClickHouse into ts-engine 2024-08-07 21:01:13 +00:00
Alexander Tokmakov
77103aa729 Merge branch 'master' into reject_poco 2024-08-06 19:52:05 +02:00
Vitaly Baranov
03ebab2ca5 Refactor factory of PrometheusRequestHandler to make extending it easier. 2024-08-03 16:20:23 +02:00
Antonio Andelic
ba1c191331
Update programs/keeper/Keeper.cpp
Co-authored-by: János Benjamin Antal <antaljanosbenjamin@users.noreply.github.com>
2024-07-29 11:47:57 +02:00
Antonio Andelic
a9b947d9ac Extract common 2024-07-28 11:51:09 +02:00
Antonio Andelic
d78cfd030f Use cgroups as source 2024-07-23 10:33:12 +02:00
Antonio Andelic
1d1dc9c7ad
Merge branch 'master' into read-cgroup-memory-usage-async-metrics 2024-07-21 10:37:01 +01:00
Alex Katsman
eb51dc8980 Don't start GWP allocations until init is finished 2024-07-19 13:34:50 +00:00
Antonio Andelic
9a43183eb3 Finish background memory thread 2024-07-18 10:40:47 +02:00
Antonio Andelic
97f4ec2adb Read cgroup memory usage in async metrics thread 2024-07-17 16:59:35 +02:00
Alexander Tokmakov
243edcc8aa add logs and metrics about rejected connections in Poco 2024-07-12 01:11:18 +02:00
Antonio Andelic
1a192a8369
Merge pull request #65563 from azat/keeper-configs-fixes
Fix clickhouse-keeper with not system-wide directories and provide override for local development
2024-07-08 07:34:08 +00:00
Antonio Andelic
ec6739120c
Merge pull request #66041 from ClickHouse/try-disabling-jemalloc-background-threads
Try disabling jemalloc background threads
2024-07-04 20:30:39 +00:00
Antonio Andelic
c93d8cbb66 Fixes 2024-07-04 13:57:47 +02:00
Antonio Andelic
6b47171f2c Keeper binary with different entrypoint 2024-07-01 10:52:08 +02:00
Alexander Tokmakov
4748e2929f
Merge pull request #65570 from ClickHouse/keeper_az
Improve ZooKeeper load balancing
2024-06-28 22:57:20 +00:00
Alexander Tokmakov
43cb2f6af7 remove trash 2024-06-26 02:03:22 +02:00
Azat Khuzhin
6dc68983e7 Fix keeper with non-system-wide directories
Otherwise it still tries to access default system-wide directory on
config reloading:

    2024.06.22 20:36:19.860615 [ 31600 ] {} <Error> Application: std::exception. Code: 1001, type: std::__1::__fs::filesystem::filesystem_error, e.what() = filesystem error: in create_directories: Permission denied ["/var/lib/clickhouse-keeper"], Stack trace (when copying this message, always include the lines below):

    0. /src/ch/clickhouse/contrib/llvm-project/libcxx/include/exception:141: std::runtime_error::runtime_error(String const&) @ 0x0000000016f16a17
    1. /src/ch/clickhouse/contrib/llvm-project/libcxx/include/string:1499: std::system_error::system_error(std::error_code, String const&) @ 0x0000000016f1d09f
    2. /src/ch/clickhouse/contrib/llvm-project/libcxx/include/__filesystem/filesystem_error.h:42: std::__fs::filesystem::filesystem_error::filesystem_error[abi:v15000](String const&, std::__fs::filesystem::path const&, std::error_code) @ 0x000000000b639ed2
    3. /src/ch/clickhouse/contrib/llvm-project/libcxx/include/__filesystem/filesystem_error.h:90: void std::__fs::filesystem::__throw_filesystem_error[abi:v15000]<String&, std::__fs::filesystem::path const&, std::error_code const&>(String&, std::__fs::filesystem::path const&, std::error_code const&) @ 0x0000000016ebaf96
    4. /src/ch/clickhouse/contrib/llvm-project/libcxx/src/filesystem/filesystem_common.h:173: std::__fs::filesystem::detail::(anonymous namespace)::ErrorHandler<bool>::report(std::error_code const&) const @ 0x0000000016ebe416
    5. /src/ch/clickhouse/contrib/llvm-project/libcxx/src/filesystem/operations.cpp:1030: std::__fs::filesystem::__create_directories(std::__fs::filesystem::path const&, std::error_code*) @ 0x0000000016ebec3d
    6. /src/ch/clickhouse/contrib/llvm-project/libcxx/src/filesystem/filesystem_common.h:161: std::__fs::filesystem::__create_directories(std::__fs::filesystem::path const&, std::error_code*) @ 0x0000000016ebed0e
    7. /src/ch/clickhouse/contrib/llvm-project/libcxx/include/string:1499: DB::ConfigProcessor::savePreprocessedConfig(DB::ConfigProcessor::LoadedConfig&, String) @ 0x00000000128362b3
    8. /src/ch/clickhouse/contrib/llvm-project/libcxx/include/string:1499: DB::ConfigReloader::reloadIfNewer(bool, bool, bool, bool) @ 0x000000001283c085
    9. /src/ch/clickhouse/src/Common/Config/ConfigReloader.cpp:33: DB::ConfigReloader::ConfigReloader(std::basic_string_view<char, std::char_traits<char>>, std::vector<String, std::allocator<String>> const&, String const&, zkutil::ZooKeeperNodeCache&&, std::shared_ptr<Poco::Event> const&, std::function<void (Poco::AutoPtr<Poco::Util::AbstractConfiguration>, bool)>&&, bool) @ 0x000000001283b457
    10. /src/ch/clickhouse/contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000000b686ecd
    11. /src/ch/clickhouse/base/poco/Util/src/Application.cpp:0: Poco::Util::Application::run() @ 0x0000000014afb156
    12. /src/ch/clickhouse/programs/keeper/Keeper.cpp:165: DB::Keeper::run() @ 0x000000000b68317e
    13. /src/ch/clickhouse/base/poco/Util/src/ServerApplication.cpp:132: Poco::Util::ServerApplication::run(int, char**) @ 0x0000000014b0faf2
    14. /src/ch/clickhouse/programs/keeper/Keeper.cpp:0: mainEntryClickHouseKeeper(int, char**) @ 0x000000000b68227e
    15. /src/ch/clickhouse/programs/main.cpp:0: main @ 0x00000000061d6204
    16. ? @ 0x00007ffff7dc2c88
    17. ? @ 0x00007ffff7dc2d4c
    18. _start @ 0x00000000061d502e

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2024-06-22 20:50:27 +02:00
alesapin
0dc01f4a94 Add ability to change config reload interval 2024-06-21 18:24:20 +02:00
Duc Canh Le
4f3e2cae11 start keeper global thread pool with sufficient amount of threads
Signed-off-by: Duc Canh Le <duccanh.le@ahrefs.com>
2024-05-27 11:04:52 +00:00
Antonio Andelic
5c47b09114 Ignore text_log for Keeper 2024-05-22 13:18:51 +02:00
Azat Khuzhin
49a52cdb3f Remove http_max_chunk_size setting (too internal)
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2024-05-16 18:33:43 +02:00
Robert Schulze
fe86741059
Some fixups 2024-03-18 10:44:56 +00:00
Han Fei
91d18649f4 update comments 2024-03-15 16:23:51 +01:00
Robert Schulze
4e12b5dc91
Some fixups 2024-03-13 16:54:20 +00:00
Han Fei
c7c64636d0 add a test 2024-03-12 23:31:33 +01:00
Han Fei
c01a6775d7 fix style 2024-03-09 02:28:39 +01:00
Han Fei
4997f95426 fix build 2024-03-08 17:30:58 +01:00
Han Fei
7fd13df8a5 check memory limit periodically 2024-03-07 17:21:03 +01:00
Alexander Tokmakov
64a80f1011
Fix default path when path is not specified in config (#59654)
* Update Server.cpp

* Update SentryWriter.cpp

* Update Keeper.cpp

* Update SentryWriter.cpp
2024-02-21 20:33:39 +01:00
Antonio Andelic
0f489de46e
Merge pull request #59945 from ClickHouse/keeper-prometheus-filtered-events
Send only Keeper related metrics/events for Prometheus
2024-02-16 09:43:43 +01:00
Konstantin Bogdanov
f678bdc546
Update tests 2024-02-14 10:51:42 +01:00
Antonio Andelic
0d151e3f0d Send only specific events/metrics for Keeper prometheus 2024-02-13 16:01:50 +01:00
Maksim Kita
2a327107b6 Updated implementation 2024-01-25 14:31:49 +03:00
Alexey Milovidov
eb8217efb0 Fix wrong message in Keeper 2024-01-08 01:22:11 +01:00
Alexander Tokmakov
3482224f83 Merge branch 'master' into keeper-http-readiness-endpoint 2023-12-12 21:19:57 +01:00
alesapin
cbae29d26a
Merge pull request #57699 from hanfei1991/hanfei/keeper_soft_limit_ratio
support keeper memory soft limit ratio
2023-12-11 21:43:24 +01:00
Konstantin Bogdanov
771d5c58ea
Fix Keeper standalone build 2023-12-11 10:19:49 +01:00
Konstantin Bogdanov
89ba4a8452
Add HTTP readiness endpoint 2023-12-11 10:19:42 +01:00
Azat Khuzhin
55d5a3affa Correctly wait background threads
There are some places that make it possible to trigger use-after-free
from threads because some global variable had been destroyed before, for
example some logger.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-12-09 17:54:16 +01:00
Han Fei
881e58dfcb support soft limit ratio 2023-12-08 21:48:27 +01:00
Alexey Milovidov
b62e9d3a1a Address review comments 2023-11-23 18:13:12 +01:00
Alexey Milovidov
bd4ffec361 Better exit codes from the server 2023-11-23 17:00:57 +01:00
Alexey Milovidov
82c461031e Fix build 2023-11-13 10:09:23 +01:00
Maksim Kita
9b60a3d7aa Keeper run without config fix 2023-10-27 20:53:07 +03:00
Nikolay Degterinsky
6539d52b9e
Merge pull request #50372 from helifu/master3
Support dynamic adjustment of log level in Keeper
2023-09-28 23:47:49 +02:00
Alexey Milovidov
ee088a0386 Fix bad punctuation in Keeper logs 2023-09-06 03:31:46 +02:00