Commit Graph

237 Commits

Author SHA1 Message Date
Raúl Marín
637672a13c
Merge pull request #72216 from Algunenano/compression
Stop wasting disk space and link time. Save the world
2024-11-26 11:12:18 +00:00
Raúl Marín
cb7d33dc6e Fix build 2024-11-25 21:10:14 +01:00
Konstantin Bogdanov
b2c81981fb
Use clang-19 2024-11-20 23:08:47 +01:00
Alexey Milovidov
c8104cb2ee Correct and unify exit codes 2024-11-07 21:31:07 +01:00
Romeo58rus
89f4fe4f92 Add support for reloading client-side certificates used to connect to Zookeeper over a secure connection 2024-10-24 01:06:04 +03:00
Antonio Andelic
bdf8c859b1 Merge branch 'master' into read-cgroup-memory-usage-async-metrics 2024-09-10 14:13:48 +02:00
Alexey Milovidov
7071942858 Miscellanous changes from #66999 2024-08-18 09:05:45 +02:00
Antonio Andelic
0d38aa98ce Merge branch 'master' into read-cgroup-memory-usage-async-metrics 2024-08-09 15:44:47 +02:00
Nikita Mikhaylov
f37e0c7fc6 Merge branch 'master' of github.com:ClickHouse/ClickHouse into ts-engine 2024-08-07 21:01:13 +00:00
Alexander Tokmakov
77103aa729 Merge branch 'master' into reject_poco 2024-08-06 19:52:05 +02:00
Nikita Mikhaylov
6f025090eb Merge branch 'master' of github.com:ClickHouse/ClickHouse into ts-engine 2024-08-05 17:46:14 +02:00
Azat Khuzhin
eed2edd7db Comment certificateFile/privateKeyFile/dhParamsFile in keeper config
Otherwise you will get annoying messages at startup:

    2024.07.02 10:03:38.331593 [ 1 ] {} <Error> CertificateReloader: Cannot obtain modification time for certificate file /etc/clickhouse-keeper/server.crt, skipping update. errno: 2, strerror: 0
    2024.07.02 10:03:38.331658 [ 1 ] {} <Error> CertificateReloader: Cannot obtain modification time for key file /etc/clickhouse-keeper/server.key, skipping update. errno: 2, strerror: 0
    2024.07.02 10:03:38.341085 [ 1 ] {} <Error> CertificateReloader: Poco::Exception. Code: 1000, e.code() = 0, SSL context exception: Error loading private key from file /etc/clickhouse-keeper/server.key: error:80000002:system library::No such file or directory

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2024-08-04 16:20:42 +02:00
Vitaly Baranov
03ebab2ca5 Refactor factory of PrometheusRequestHandler to make extending it easier. 2024-08-03 16:20:23 +02:00
Antonio Andelic
ba1c191331
Update programs/keeper/Keeper.cpp
Co-authored-by: János Benjamin Antal <antaljanosbenjamin@users.noreply.github.com>
2024-07-29 11:47:57 +02:00
Antonio Andelic
a9b947d9ac Extract common 2024-07-28 11:51:09 +02:00
Antonio Andelic
d78cfd030f Use cgroups as source 2024-07-23 10:33:12 +02:00
Antonio Andelic
1d1dc9c7ad
Merge branch 'master' into read-cgroup-memory-usage-async-metrics 2024-07-21 10:37:01 +01:00
Alex Katsman
eb51dc8980 Don't start GWP allocations until init is finished 2024-07-19 13:34:50 +00:00
Antonio Andelic
9a43183eb3 Finish background memory thread 2024-07-18 10:40:47 +02:00
Antonio Andelic
97f4ec2adb Read cgroup memory usage in async metrics thread 2024-07-17 16:59:35 +02:00
Alexander Tokmakov
243edcc8aa add logs and metrics about rejected connections in Poco 2024-07-12 01:11:18 +02:00
Antonio Andelic
1a192a8369
Merge pull request #65563 from azat/keeper-configs-fixes
Fix clickhouse-keeper with not system-wide directories and provide override for local development
2024-07-08 07:34:08 +00:00
Antonio Andelic
ec6739120c
Merge pull request #66041 from ClickHouse/try-disabling-jemalloc-background-threads
Try disabling jemalloc background threads
2024-07-04 20:30:39 +00:00
Antonio Andelic
c93d8cbb66 Fixes 2024-07-04 13:57:47 +02:00
Antonio Andelic
07f51e02ed Reuse some checks 2024-07-03 16:54:09 +02:00
Antonio Andelic
b0bbc9c810 Fix symlinks 2024-07-01 16:01:57 +02:00
Antonio Andelic
6b47171f2c Keeper binary with different entrypoint 2024-07-01 10:52:08 +02:00
Alexander Tokmakov
4748e2929f
Merge pull request #65570 from ClickHouse/keeper_az
Improve ZooKeeper load balancing
2024-06-28 22:57:20 +00:00
Alexander Tokmakov
43cb2f6af7 remove trash 2024-06-26 02:03:22 +02:00
Azat Khuzhin
6dc68983e7 Fix keeper with non-system-wide directories
Otherwise it still tries to access default system-wide directory on
config reloading:

    2024.06.22 20:36:19.860615 [ 31600 ] {} <Error> Application: std::exception. Code: 1001, type: std::__1::__fs::filesystem::filesystem_error, e.what() = filesystem error: in create_directories: Permission denied ["/var/lib/clickhouse-keeper"], Stack trace (when copying this message, always include the lines below):

    0. /src/ch/clickhouse/contrib/llvm-project/libcxx/include/exception:141: std::runtime_error::runtime_error(String const&) @ 0x0000000016f16a17
    1. /src/ch/clickhouse/contrib/llvm-project/libcxx/include/string:1499: std::system_error::system_error(std::error_code, String const&) @ 0x0000000016f1d09f
    2. /src/ch/clickhouse/contrib/llvm-project/libcxx/include/__filesystem/filesystem_error.h:42: std::__fs::filesystem::filesystem_error::filesystem_error[abi:v15000](String const&, std::__fs::filesystem::path const&, std::error_code) @ 0x000000000b639ed2
    3. /src/ch/clickhouse/contrib/llvm-project/libcxx/include/__filesystem/filesystem_error.h:90: void std::__fs::filesystem::__throw_filesystem_error[abi:v15000]<String&, std::__fs::filesystem::path const&, std::error_code const&>(String&, std::__fs::filesystem::path const&, std::error_code const&) @ 0x0000000016ebaf96
    4. /src/ch/clickhouse/contrib/llvm-project/libcxx/src/filesystem/filesystem_common.h:173: std::__fs::filesystem::detail::(anonymous namespace)::ErrorHandler<bool>::report(std::error_code const&) const @ 0x0000000016ebe416
    5. /src/ch/clickhouse/contrib/llvm-project/libcxx/src/filesystem/operations.cpp:1030: std::__fs::filesystem::__create_directories(std::__fs::filesystem::path const&, std::error_code*) @ 0x0000000016ebec3d
    6. /src/ch/clickhouse/contrib/llvm-project/libcxx/src/filesystem/filesystem_common.h:161: std::__fs::filesystem::__create_directories(std::__fs::filesystem::path const&, std::error_code*) @ 0x0000000016ebed0e
    7. /src/ch/clickhouse/contrib/llvm-project/libcxx/include/string:1499: DB::ConfigProcessor::savePreprocessedConfig(DB::ConfigProcessor::LoadedConfig&, String) @ 0x00000000128362b3
    8. /src/ch/clickhouse/contrib/llvm-project/libcxx/include/string:1499: DB::ConfigReloader::reloadIfNewer(bool, bool, bool, bool) @ 0x000000001283c085
    9. /src/ch/clickhouse/src/Common/Config/ConfigReloader.cpp:33: DB::ConfigReloader::ConfigReloader(std::basic_string_view<char, std::char_traits<char>>, std::vector<String, std::allocator<String>> const&, String const&, zkutil::ZooKeeperNodeCache&&, std::shared_ptr<Poco::Event> const&, std::function<void (Poco::AutoPtr<Poco::Util::AbstractConfiguration>, bool)>&&, bool) @ 0x000000001283b457
    10. /src/ch/clickhouse/contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000000b686ecd
    11. /src/ch/clickhouse/base/poco/Util/src/Application.cpp:0: Poco::Util::Application::run() @ 0x0000000014afb156
    12. /src/ch/clickhouse/programs/keeper/Keeper.cpp:165: DB::Keeper::run() @ 0x000000000b68317e
    13. /src/ch/clickhouse/base/poco/Util/src/ServerApplication.cpp:132: Poco::Util::ServerApplication::run(int, char**) @ 0x0000000014b0faf2
    14. /src/ch/clickhouse/programs/keeper/Keeper.cpp:0: mainEntryClickHouseKeeper(int, char**) @ 0x000000000b68227e
    15. /src/ch/clickhouse/programs/main.cpp:0: main @ 0x00000000061d6204
    16. ? @ 0x00007ffff7dc2c88
    17. ? @ 0x00007ffff7dc2d4c
    18. _start @ 0x00000000061d502e

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2024-06-22 20:50:27 +02:00
Azat Khuzhin
b013d95fd5 Provide keeper override for local development (rewrite path to current directory)
The same way like clickhouse-server has

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2024-06-22 20:50:27 +02:00
alesapin
0dc01f4a94 Add ability to change config reload interval 2024-06-21 18:24:20 +02:00
Kseniia Sumarokova
321f62d257
Merge pull request #64412 from ClickHouse/unite-storages3-and-disks3-settings
Refactor s3 settings (move settings parsing into single place)
2024-06-14 12:35:47 +00:00
Duc Canh Le
ebf506aef4 Merge branch 'master' into fix_keeper_default_global_pool
Try fixing CI
2024-06-04 08:30:55 +00:00
kssenii
cd3d99a0eb Merge remote-tracking branch 'origin/master' into unite-storages3-and-disks3-settings 2024-06-03 17:37:33 +02:00
Raúl Marín
ee416a6862
Merge pull request #63723 from Algunenano/ifunction_column_size
Add a bunch of important asserts
2024-06-03 14:04:02 +00:00
kssenii
dfd19576cd Fix keeper build 2024-05-28 19:42:57 +02:00
Duc Canh Le
4f3e2cae11 start keeper global thread pool with sufficient amount of threads
Signed-off-by: Duc Canh Le <duccanh.le@ahrefs.com>
2024-05-27 11:04:52 +00:00
Antonio Andelic
5c47b09114 Ignore text_log for Keeper 2024-05-22 13:18:51 +02:00
Raúl Marín
2e1f2565ea Merge remote-tracking branch 'blessed/master' into ifunction_column_size 2024-05-21 12:33:56 +02:00
Robert Schulze
70637b2ecf
Merge remote-tracking branch 'rschu1ze/master' into move-isallascii 2024-05-20 13:01:53 +00:00
Kseniia Sumarokova
42c984df64
Merge pull request #63475 from tomershafir/disks-io-iouring-refactor-get-from-context
io_uring: refactor get reader from context
2024-05-20 10:17:32 +00:00
Robert Schulze
2909e6451b
Move StringUtils.h/cpp back to Common/ 2024-05-19 09:39:36 +00:00
Azat Khuzhin
49a52cdb3f Remove http_max_chunk_size setting (too internal)
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2024-05-16 18:33:43 +02:00
Raúl Marín
698c53b60f Better 2024-05-16 14:05:12 +02:00
tomershafir
b97bf456c6 try fix build 2024-05-09 20:53:36 +03:00
Julia Kartseva
24d5abba10 extract plain_rewritable metadata type
Make PlainRewritableObjectStorage generic;

Support config type
```
<s3_plain_rewritable>
    <type>object_storage</type>
    <object_storage_type>s3</object_storage_type>
    <metadata_type>plain_rewritable</metadata_type>
    <endpoint>https://s3.eu-west-1.amazonaws.com/clickhouse-eu-west-1.clickhouse.com/data/</endpoint>
    <use_environment_credentials>1</use_environment_credentials>
</s3_plain_rewritable>
```
2024-04-29 18:01:21 +00:00
Julia Kartseva
802ee27b1b address feedback - pt.3
non-functional changes
2024-04-29 18:01:21 +00:00
Julia Kartseva
75147f3ac7 S3-plain based disk supporting directory rename
Fixes: https://github.com/ClickHouse/ClickHouse/issues/58347
2024-04-29 18:01:21 +00:00
Robert Schulze
fe86741059
Some fixups 2024-03-18 10:44:56 +00:00