thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-02 12:32:04 +00:00
Code Issues Packages Projects Releases Wiki Activity
146,365 Commits 9,194 Branches 1,782 Tags 2.2 GiB
8a202d91ad
Commit Graph

1360 Commits

Author SHA1 Message Date
Azat Khuzhin
3be3b0a280 Fix incorrect Exceptions 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-12-24 21:26:32 +01:00
Antonio Andelic
b9466023ce
Merge pull request #58069 from ClickHouse/dont-preprocess-if-not-ready 
Fix invalid preprocessing on Keeper
 2023-12-21 09:08:56 +01:00
Antonio Andelic
3cc8358b89 Avoid preprocessing if Keeper leader is not ready 2023-12-20 10:22:00 +00:00
Shani Elharrar
c696c0bfe7 S3Common.AuthSettings: Allow passing SESSION_TOKEN to AWSCredentials 
This sets the infrastructure of loading session_token and passing it directly
to all AWSCredentials instances that are created using the AuthSettings.

The default SESSION_TOKEN is set to an empty string as documented in AWS SDK
reference: https://sdk.amazonaws.com/cpp/api/0.12.9/d4/d27/class_aws_1_1_auth_1_1_a_w_s_credentials.html
 2023-12-17 10:29:15 +02:00
Nikita Mikhaylov
a0af0392cd
Random changes in random files (#57642) 2023-12-14 12:47:11 +01:00
alesapin
cbae29d26a
Merge pull request #57699 from hanfei1991/hanfei/keeper_soft_limit_ratio 
support keeper memory soft limit ratio
 2023-12-11 21:43:24 +01:00
Azat Khuzhin
450e8da7ce Fix data race on vptr (ctor/dtor vs virtual call) in AsynchronousMetrics 
TSan report [1]:

     Exception: Sanitizer assert found for instance ==================
     WARNING: ThreadSanitizer: data race on vptr (ctor/dtor vs virtual call) (pid=1)
       Write of size 8 at 0x7ffce68f5680 by main thread:
         0 DB::AsynchronousMetrics::~AsynchronousMetrics() build_docker/./src/Common/AsynchronousMetrics.cpp:299:1 (clickhouse+0xf38fa6e) (BuildId: d49dc55f80b62802f8721fdbcb07bdc88ac204a4)
         1 DB::ServerAsynchronousMetrics::~ServerAsynchronousMetrics() build_docker/./src/Interpreters/ServerAsynchronousMetrics.h:10:7 (clickhouse+0xf2b2f0f) (BuildId: d49dc55f80b62802f8721fdbcb07bdc88ac204a4)
         2 DB::Server::main(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>> const&) build_docker/./programs/server/Server.cpp:1996:1 (clickhouse+0xf2b2f0f)
         3 Poco::Util::Application::run() build_docker/./base/poco/Util/src/Application.cpp:315:8 (clickhouse+0x1d6c95fe) (BuildId: d49dc55f80b62802f8721fdbcb07bdc88ac204a4)
         4 DB::Server::run() build_docker/./programs/server/Server.cpp:397:25 (clickhouse+0xf29e041) (BuildId: d49dc55f80b62802f8721fdbcb07bdc88ac204a4)
         5 Poco::Util::ServerApplication::run(int, char**) build_docker/./base/poco/Util/src/ServerApplication.cpp:131:9 (clickhouse+0x1d6e9354) (BuildId: d49dc55f80b62802f8721fdbcb07bdc88ac204a4)
         6 mainEntryClickHouseServer(int, char**) build_docker/./programs/server/Server.cpp:203:20 (clickhouse+0xf29b1a3) (BuildId: d49dc55f80b62802f8721fdbcb07bdc88ac204a4)
         7 main build_docker/./programs/main.cpp:505:12 (clickhouse+0x72d7fa0) (BuildId: d49dc55f80b62802f8721fdbcb07bdc88ac204a4)

       Previous read of size 8 at 0x7ffce68f5680 by thread T649:
         0 DB::AsynchronousMetrics::update(std::__1::chrono::time_point<std::__1::chrono::system_clock, std::__1::chrono::duration<long long, std::__1::ratio<1l, 1000000l>>>) build_docker/./src/Common/AsynchronousMetrics.cpp:1559:5 (clickhouse+0xf38ebc5) (BuildId: d49dc55f80b62802f8721fdbcb07bdc88ac204a4)
         1 DB::AsynchronousMetrics::run() build_docker/./src/Common/AsynchronousMetrics.cpp:354:13 (clickhouse+0xf390696) (BuildId: d49dc55f80b62802f8721fdbcb07bdc88ac204a4)
         2 DB::AsynchronousMetrics::start()::$_0::operator()() const build_docker/./src/Common/AsynchronousMetrics.cpp:273:62 (clickhouse+0xf392eca) (BuildId: d49dc55f80b62802f8721fdbcb07bdc88ac204a4)
         3 decltype(std::declval<DB::AsynchronousMetrics::start()::$_0&>()()) std::__1::__invoke[abi:v15000]<DB::AsynchronousMetrics::start()::$_0&>(DB::AsynchronousMetrics::start()::$_0&) build_docker/./contrib/llvm-project/libcxx/include/__functional/invoke.h:394:23 (clickhouse+0xf392eca)
         4 decltype(auto) std::__1::__apply_tuple_impl[abi:v15000]<DB::AsynchronousMetrics::start()::$_0&, std::__1::tuple<>&>(DB::AsynchronousMetrics::start()::$_0&, std::__1::tuple<>&, std::__1::__tuple_indices<>) build_docker/./contrib/llvm-project/libcxx/include/tuple:1789:1 (clickhouse+0xf392eca)
         5 decltype(auto) std::__1::apply[abi:v15000]<DB::AsynchronousMetrics::start()::$_0&, std::__1::tuple<>&>(DB::AsynchronousMetrics::start()::$_0&, std::__1::tuple<>&) build_docker/./contrib/llvm-project/libcxx/include/tuple:1798:1 (clickhouse+0xf392eca)
         6 ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::AsynchronousMetrics::start()::$_0>(DB::AsynchronousMetrics::start()::$_0&&)::'lambda'()::operator()() build_docker/./src/Common/ThreadPool.h:253:13 (clickhouse+0xf392eca)
         7 decltype(std::declval<DB::AsynchronousMetrics::start()::$_0>()()) std::__1::__invoke[abi:v15000]<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::AsynchronousMetrics::start()::$_0>(DB::AsynchronousMetrics::start()::$_0&&)::'lambda'()&>(DB::AsynchronousMetrics::start()::$_0&&) build_docker/./contrib/llvm-project/libcxx/include/__functional/invoke.h:394:23 (clickhouse+0xf392eca)
         8 void std::__1::__invoke_void_return_wrapper<void, true>::__call<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::AsynchronousMetrics::start()::$_0>(DB::AsynchronousMetrics::start()::$_0&&)::'lambda'()&>(ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::AsynchronousMetrics::start()::$_0>(DB::AsynchronousMetrics::start()::$_0&&)::'lambda'()&) build_docker/./contrib/llvm-project/libcxx/include/__functional/invoke.h:479:9 (clickhouse+0xf392eca)
         9 std::__1::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::AsynchronousMetrics::start()::$_0>(DB::AsynchronousMetrics::start()::$_0&&)::'lambda'(), void ()>::operator()[abi:v15000]() build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:235:12 (clickhouse+0xf392eca)
         10 void std::__1::__function::__policy_invoker<void ()>::__call_impl<std::__1::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::AsynchronousMetrics::start()::$_0>(DB::AsynchronousMetrics::start()::$_0&&)::'lambda'(), void ()>>(std::__1::__function::__policy_storage const*) build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:716:16 (clickhouse+0xf392eca)
         11 std::__1::__function::__policy_func<void ()>::operator()[abi:v15000]() const build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:848:16 (clickhouse+0xf30682e) (BuildId: d49dc55f80b62802f8721fdbcb07bdc88ac204a4)
         12 std::__1::function<void ()>::operator()() const build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:1187:12 (clickhouse+0xf30682e)
         13 ThreadPoolImpl<std::__1::thread>::worker(std::__1::__list_iterator<std::__1::thread, void*>) build_docker/./src/Common/ThreadPool.cpp:421:13 (clickhouse+0xf30682e)
         14 void ThreadPoolImpl<std::__1::thread>::scheduleImpl<void>(std::__1::function<void ()>, Priority, std::__1::optional<unsigned long>, bool)::'lambda0'()::operator()() const build_docker/./src/Common/ThreadPool.cpp:183:73 (clickhouse+0xf30d3d1) (BuildId: d49dc55f80b62802f8721fdbcb07bdc88ac204a4)
         15 decltype(std::declval<void>()()) std::__1::__invoke[abi:v15000]<void ThreadPoolImpl<std::__1::thread>::scheduleImpl<void>(std::__1::function<void ()>, Priority, std::__1::optional<unsigned long>, bool)::'lambda0'()>(void&&) build_docker/./contrib/llvm-project/libcxx/include/__functional/invoke.h:394:23 (clickhouse+0xf30d3d1)
         16 void std::__1::__thread_execute[abi:v15000]<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct>>, void ThreadPoolImpl<std::__1::thread>::scheduleImpl<void>(std::__1::function<void ()>, Priority, std::__1::optional<unsigned long>, bool)::'lambda0'()>(std::__1::tuple<void, void ThreadPoolImpl<std::__1::thread>::scheduleImpl<void>(std::__1::function<void ()>, Priority, std::__1::optional<unsigned long>, bool)::'lambda0'()>&, std::__1::__tuple_indices<>) build_docker/./contrib/llvm-project/libcxx/include/thread:284:5 (clickhouse+0xf30d3d1)
         17 void* std::__1::__thread_proxy[abi:v15000]<std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct>>, void ThreadPoolImpl<std::__1::thread>::scheduleImpl<void>(std::__1::function<void ()>, Priority, std::__1::optional<unsigned long>, bool)::'lambda0'()>>(void*) build_docker/./contrib/llvm-project/libcxx/include/thread:295:5 (clickhouse+0xf30d3d1)

     SUMMARY: ThreadSanitizer: data race on vptr (ctor/dtor vs virtual call) build_docker/./src/Common/AsynchronousMetrics.cpp:299:1 in DB::AsynchronousMetrics::~AsynchronousMetrics()

And the problem is real, see [2].

  [1]: https://s3.amazonaws.com/clickhouse-test-reports/52717/a6cf2297ef07ca0b065d944c9e30a3986285331d/integration_tests__tsan__[2_6]/integration_run_parallel0_0.log
  [2]: https://github.com/google/sanitizers/wiki/ThreadSanitizerPopularDataRaces#data-race-on-vptr

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-12-10 18:32:40 +01:00
Azat Khuzhin
6ccbc2ea75 Move io_uring reader into the Context from static to make it's thread joinable 
v2: fix for standalone keeper build
CI: https://s3.amazonaws.com/clickhouse-test-reports/52717/72b1052f7c2d453308262924e767ab8dc2206933/stateless_tests__debug__[4_5].html
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-12-09 22:50:48 +01:00
Han Fei
881e58dfcb support soft limit ratio 2023-12-08 21:48:27 +01:00
alesapin
dc7347e0ba
Merge pull request #57271 from hanfei1991/hanfei/keeper_soft_limit 
support memory soft limit for keeper
 2023-12-08 16:12:47 +01:00
jsc0218
cdd5280272
Merge pull request #57387 from evillique/better-disks 
Initialize only required disks in clickhouse-disks
 2023-12-05 13:59:04 -05:00
Alexey Milovidov
10d65a1ade
Merge pull request #55559 from azat/s3-fix-excessive-reads 
Add ability to disable checksums for S3 to avoid excessive input file read
 2023-12-05 06:34:21 +01:00
Han Fei
70f436483c address part of comments 2023-12-05 00:40:15 +01:00
alesapin
acc53431c7
Update src/Coordination/KeeperDispatcher.cpp 2023-12-03 12:43:20 +01:00
alesapin
9a68461cdb
Update src/Coordination/KeeperDispatcher.cpp 2023-12-03 12:43:16 +01:00
Han Fei
e4da317f05 fix test 2023-12-01 18:07:45 +01:00
Nikolay Degterinsky
e8203f8a76 Initialize only required disks 2023-11-30 03:09:55 +00:00
Antonio Andelic
61b112e2c4
Merge pull request #57268 from ClickHouse/ignore-memory-exception-asio 
Ignore memory exception in Keeper asio workers
 2023-11-29 08:33:00 +01:00
Antonio Andelic
852b4d42b6
Update src/Coordination/KeeperServer.cpp 2023-11-28 16:00:06 +01:00
Antonio Andelic
d10aeef23e
Add comment 2023-11-28 15:32:20 +01:00
Han Fei
3aa4a6ae2c check integration test 2023-11-28 14:47:31 +01:00
Antonio Andelic
889c53eca0 Correctly handle unknown changelog versions and snapshot deleting 2023-11-28 09:47:49 +00:00
Han Fei
347efc669e support memory soft limit for keeper 2023-11-27 18:28:10 +01:00
Antonio Andelic
3941520cb0 Avoid memory exceptions in asio workers 2023-11-27 10:08:06 +00:00
Azat Khuzhin
4a02de4674 Add ability to disable checksums for S3 to avoid excessive input file read 
AWS S3 client can read file multiple times, this is required for:
- calculate checksums
- calculate signature (done only for HTTP, since ClickHouse uses
  PayloadSigningPolicy::Never)

So this means that for HTTP, to send file to S3 it will be read 3x
times, and for HTTPS 2x times.

By overriding GetChecksumAlgorithmName() to return empty string,
checksums can be disabled, and the input file will be read only once.

And even though additional https layer adds extra integrity layer,
someone still may find this too risky I guess, even though ClickHouse
internal format (for MergeTree) has checksums, and more.

Here is an example stacktrace of this excessive read:

<details>

<summary>stacktrace</summary>

    (lldb) bt
    * thread 383, name = 'BackupWorker', stop reason = breakpoint 1.1
      * frame 0: 0x00000000103c5fc0 clickhouse`DB::StdStreamBufFromReadBuffer::seekpos() + 32 at StdStreamBufFromReadBuffer.cpp:67
        frame 1: 0x000000001777f7f8 clickhouse`std::__1::basic_istream<char, std::__1::char_traits<char>>::tellg() [inlined] std::__1::basic_streambuf<char, std::__1::char_traits<char>>::pubseekoff[abi:v15000](this=<unavailable>, __off=0, __way=cur, __which=8) + 120 at streambuf:162
        frame 2: 0x000000001777f7e3 clickhouse`std::__1::basic_istream<char, std::__1::char_traits<char>>::tellg() + 99 at istream:1249
        frame 3: 0x00000000152e4979 clickhouse`Aws::Utils::Crypto::MD5OpenSSLImpl::Calculate() + 57 at CryptoImpl.cpp:223
        frame 4: 0x00000000152dedee clickhouse`Aws::Utils::Crypto::MD5::Calculate() + 14 at MD5.cpp:30
        frame 5: 0x00000000152db5ac clickhouse`Aws::Utils::HashingUtils::CalculateMD5() + 44 at HashingUtils.cpp:235
        frame 6: 0x000000001528b97b clickhouse`Aws::Client::AWSClient::AddChecksumToRequest() const + 507 at AWSClient.cpp:772
        frame 7: 0x000000001528ded2 clickhouse`Aws::Client::AWSClient::BuildHttpRequest() const + 1682 at AWSClient.cpp:930
        frame 8: 0x00000000100b864f clickhouse`DB::S3::Client::BuildHttpRequest() const + 15 at Client.cpp:622
        frame 9: 0x0000000015286a41 clickhouse`Aws::Client::AWSClient::AttemptOneRequest(this=0x00007ffde2f8f000, httpRequest=<unavailable>, request=<unavailable>, signerName=<unavailable>, signerRegionOverride=<unavailable>, signerServiceNameOverride="s3") const + 65 at AWSClient.cpp:491
        frame 10: 0x00000000152845b9 clickhouse`Aws::Client::AWSClient::AttemptExhaustively(this=0x00007ffde2f8f000, uri=0x00007ffdd4d44f38, request=0x00007ffdd4d45d10, method=HTTP_PUT, signerName="SignatureV4", signerRegionOverride="us-east-1", signerServiceNameOverride="s3") const + 1337 at AWSClient.cpp:272
        frame 11: 0x0000000015298d0d clickhouse`Aws::Client::AWSXMLClient::MakeRequest() const + 45 at AWSXmlClient.cpp:99
        frame 12: 0x0000000015298cb5 clickhouse`Aws::Client::AWSXMLClient::MakeRequest() const + 309 at AWSXmlClient.cpp:66
        frame 13: 0x0000000015354b23 clickhouse`Aws::S3::S3Client::PutObject(this=0x00007ffde2f8f000, request=0x00007ffdd4d45d10) const + 2659 at S3Client.cpp:1731
        frame 14: 0x00000000100b174f clickhouse`DB::S3::Client::PutObject(DB::S3::ExtendedRequest<Aws::S3::Model::PutObjectRequest> const&) const [inlined]
        frame 15: 0x00000000100b173a clickhouse`DB::S3::Client::PutObject(DB::S3::ExtendedRequest<Aws::S3::Model::PutObjectRequest> const&) const + 41 at Client.cpp:578
        frame 16: 0x00000000100b1711 clickhouse`DB::S3::Client::PutObject(DB::S3::ExtendedRequest<Aws::S3::Model::PutObjectRequest> const&) const + 981 at Client.cpp:508
        frame 17: 0x00000000100b133c clickhouse`DB::S3::Client::PutObject(DB::S3::ExtendedRequest<Aws::S3::Model::PutObjectRequest> const&) const [inlined]
        frame 18: 0x00000000100b133c clickhouse`DB::S3::Client::PutObject() const + 28 at Client.cpp:418
        frame 19: 0x00000000103b96d6 clickhouse`DB::copyDataToS3File()

</details>

This new behaviour could be enabled with `s3_disable_checksum=true`.

Note, that I've checked this implementation with GCS/R2/S3/MinIO and it
works everywhere.
 2023-11-26 19:20:19 +01:00
Antonio Andelic
b18cb1e5cd
Merge pull request #57047 from ClickHouse/keeper-small-fixes 
Small Keeper fixes
 2023-11-22 08:47:58 +01:00
Antonio Andelic
90bb796d63 Small Keeper fixes 2023-11-21 13:11:33 +00:00
vdimir
15234474d7
Implement system table blob_storage_log 2023-11-21 09:18:25 +00:00
Sema Checherinda
f999337dae
Revert "Revert "s3 adaptive timeouts"" 2023-11-20 14:53:22 +01:00
Alexander Tokmakov
5031f239c3
Revert "s3 adaptive timeouts" 2023-11-20 14:28:59 +01:00
Sema Checherinda
a950595c24
Merge pull request #56314 from CheSema/s3-aggressive-timeouts 
s3 adaptive timeouts
 2023-11-19 14:12:14 +01:00
Alexey Milovidov
edc3b2fe48
Merge pull request #56958 from ClickHouse/metric-queued-jobs 
Add metrics for the number of queued jobs, which is useful for the IO thread pool
 2023-11-19 10:37:18 +01:00
Alexey Milovidov
75cebb3c29 Fix build 2023-11-18 20:20:21 +01:00
Alexey Milovidov
d56cbda185 Add metrics for the number of queued jobs, which is useful for the IO thread pool 2023-11-18 19:07:59 +01:00
Jianfei Hu
ef79bf6467 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into keeper-az-fix 2023-11-16 21:58:02 +00:00
Jianfei Hu
9df2775f08 reduce timeout and setTimeout earlier. 
Signed-off-by: Jianfei Hu <hujianfei258@gmail.com>
 2023-11-14 17:58:16 +00:00
Antonio Andelic
03c3e96859 Small test changes 2023-11-14 17:35:17 +00:00
Sema Checherinda
8d36fd6e54 get rid off of client_with_long_timeout_ptr 2023-11-14 11:34:12 +01:00
Antonio Andelic
2f505786e2 Merge branch 'more-reliable-log-handling-keeper' into disable-compress-logs-default 2023-11-14 09:46:19 +00:00
Antonio Andelic
1ab7e5693d Add unit test 2023-11-14 09:43:09 +00:00
Antonio Andelic
3f5ccabba6 Don't append different log types in same file 2023-11-14 09:07:42 +00:00
Antonio Andelic
5849e06595 Fix 2023-11-14 09:07:17 +00:00
Antonio Andelic
8d27920987 Merge branch 'master' into more-reliable-log-handling-keeper 2023-11-14 08:38:07 +00:00
Jianfei Hu
554d907189 Fix the keeper_server availability zone configuration. 
Signed-off-by: Jianfei Hu <hujianfei258@gmail.com>
 2023-11-13 23:42:51 +00:00
Jianfei Hu
d3517140d9 Revert "Revert "Add /keeper/availability-zone node to allow server load balancing within AZ."" 
This reverts commit e5aad4a6fd.
 2023-11-13 19:03:27 +00:00
Antonio Andelic
d9fbab554f
remove useless include 2023-11-13 14:52:32 +01:00
Antonio Andelic
e3c66f8a63 Merge branch 'master' into more-reliable-log-handling-keeper 2023-11-13 12:14:57 +00:00
Antonio Andelic
99d521becf More reliable log handling in Keeper 2023-11-13 12:09:13 +00:00
Alexey Milovidov
82c461031e Fix build 2023-11-13 10:09:23 +01:00
Pradeep Chhetri
d0922782ed Revert "Revert "Add 4-letter command for yielding/resigning leadership"" 
This reverts commit e1766b96d9.
 2023-11-11 15:37:49 +08:00
Alexey Milovidov
e1766b96d9
Revert "Add 4-letter command for yielding/resigning leadership" 2023-11-11 05:45:02 +03:00
Alexey Milovidov
e5aad4a6fd
Revert "Add /keeper/availability-zone node to allow server load balancing within AZ." 2023-11-11 05:43:58 +03:00
Mike Kot
785183bb3c Keeper reconfig -- sleep before removing leader to avoid losing accepted 
but uncommitted commands
 2023-11-09 16:22:44 +00:00
Antonio Andelic
788c66027c
Merge pull request #56104 from incfly/keeper-az 
Add /keeper/availability-zone node to allow server load balancing within AZ.
 2023-11-08 09:46:24 +01:00
Pradeep Chhetri
4e8ff997c0 fix 2023-11-07 14:20:40 +08:00
Pradeep Chhetri
201a7b8d5f fix 2023-11-07 14:06:55 +08:00
Pradeep Chhetri
74b53218b8 Rename yield_leader with yield_leadship and its variants 2023-11-07 13:58:45 +08:00
Pradeep Chhetri
361472b29e Add 4-letter command for yielding/resigning leadership 2023-11-07 13:58:45 +08:00
Jianfei Hu
b426cfe462 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into keeper-az 
Signed-off-by: Jianfei Hu <hujianfei258@gmail.com>
 2023-11-03 16:42:29 +00:00
Jianfei Hu
06ff714e2e add configuration option and integration test(wip). 
Signed-off-by: Jianfei Hu <hujianfei258@gmail.com>
 2023-11-02 16:21:24 +00:00
Antonio Andelic
1a75970486 PR review 2023-11-02 11:57:26 +00:00
Antonio Andelic
28edb77e72 Merge branch 'master' into keeper-better-preprocessing 2023-11-02 09:26:35 +00:00
Jianfei Hu
e29b7640c4 gcp metadata use dns. 
Signed-off-by: Jianfei Hu <hujianfei258@gmail.com>
 2023-11-01 17:53:34 +00:00
Jianfei Hu
b22773ccd1
Apply suggestions from code review 
Co-authored-by: Kseniia Sumarokova <54203879+kssenii@users.noreply.github.com>
 2023-11-01 10:30:29 -07:00
Alexander Gololobov
78b9da692c Do less work when there a lots of watches 2023-11-01 12:27:10 +01:00
Jianfei Hu
6f94770a8e more fix -DUSE_AWS_S3=0 case. 
Signed-off-by: Jianfei Hu <hujianfei258@gmail.com>
 2023-11-01 00:51:45 +00:00
Jianfei Hu
3fff813659 style fix and revert src/Coordination/KeeperSnapshotManagerS3.h 
Signed-off-by: Jianfei Hu <hujianfei258@gmail.com>
 2023-10-31 18:48:59 +00:00
Jianfei Hu
cfa1b96d23 remove comments. 
Signed-off-by: Jianfei Hu <hujianfei258@gmail.com>
 2023-10-31 17:43:24 +00:00
Jianfei Hu
573054d755 add new code and rework. 
Signed-off-by: Jianfei Hu <hujianfei258@gmail.com>
 2023-10-31 04:43:44 +00:00
Jianfei Hu
6d72873bdb use poco better works. 
Signed-off-by: Jianfei Hu <hujianfei258@gmail.com>
 2023-10-31 02:34:50 +00:00
Jianfei Hu
dec8efc39e make az as attribute of keepersnapshotmangers3. 
empty config not working.
https://pastila.nl/?00048e4f/d56af90d13f2d871e30dfb1db9660266#yff6d8HVQm0blQUUiZg8SQ==

Signed-off-by: Jianfei Hu <hujianfei258@gmail.com>
 2023-10-29 02:04:38 +00:00
Jianfei Hu
0b301f73c1 initial add keeper az info in /keeper/availabilty-zone 
Signed-off-by: Jianfei Hu <hujianfei258@gmail.com>
 2023-10-28 21:54:59 +00:00
Alexander Gololobov
9a95b82cae Avoid traversing potentially big list of watches 2023-10-27 14:54:58 +02:00
Antonio Andelic
674ffa1d25 Fix install snapshot 2023-10-17 12:56:55 +00:00
Antonio Andelic
17eaa5a35e Fix edge cases 2023-10-16 13:19:25 +00:00
Antonio Andelic
9d218a8c7b Better 2023-10-16 08:50:53 +00:00
Antonio Andelic
2d851ec163 Better 2023-10-16 08:22:39 +00:00
Antonio Andelic
58120b3f47 Polish 2023-10-16 08:17:23 +00:00
Antonio Andelic
606f742254 Merge branch 'master' into keeper-better-preprocessing 2023-10-16 06:57:19 +00:00
Antonio Andelic
c69838e052
Merge pull request #55591 from ClickHouse/fix-keeper-context 
Apply Context changes to standalone Keeper
 2023-10-16 08:41:17 +02:00
alesapin
3b02748cb6 Fix some typos 2023-10-15 15:43:02 +02:00
Antonio Andelic
67068849da
Fix comment 2023-10-13 15:59:37 +02:00
Azat Khuzhin
720ea8441c Fix global context for tests with --gtest_filter 
If you run tests that requires context, but do not initialize it, then
it will SIGSEGV, because Context is not initialized.

Fix this by using google test envrionment, and instead of gtest_main
implement own main function that will initialize it.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-10-13 13:44:30 +02:00
Antonio Andelic
48933f52fe Fix Keeper Context 2023-10-13 11:30:37 +00:00
Antonio Andelic
34d8bb18cf Better preprocessing in Keeper 2023-09-29 14:34:17 +00:00
Antonio Andelic
01f64be558 Set path in response 2023-09-26 07:41:05 +00:00
Antonio Andelic
176baa6695 Merge branch 'master' into keeper-operation-create-if-not-exists 2023-09-22 13:47:30 +00:00
Antonio Andelic
b65b1ee97b Merge branch 'master' into keeper-operation-create-if-not-exists 2023-09-20 13:56:36 +00:00
Antonio Andelic
88ab5514a0 Merge branch 'master' into keeper-reduce-batch-size 2023-09-20 13:56:06 +00:00
Sema Checherinda
e7550523c8
Merge pull request #54651 from CheSema/limit_backoff_timeout 
limit the delay before next try in S3
 2023-09-18 19:41:26 +02:00
Antonio Andelic
7f3279fa21 Revert batch size to 100 for Keeper 2023-09-18 11:18:39 +00:00
Antonio Andelic
69333912d5 Merge branch 'master' into keeper-operation-create-if-not-exists 2023-09-18 10:57:11 +00:00
Robert Schulze
7d04cf6b58
Merge pull request #54646 from rschu1ze/bump-abseil 
Bump abseil
 2023-09-18 09:44:35 +02:00
Robert Schulze
5f46106984
Try to fix sizes in unit tests, pt. III 2023-09-17 14:41:51 +00:00
Robert Schulze
066bdf57c4
Try to fix sizes in unit test, pt. II 2023-09-15 14:53:05 +00:00
Robert Schulze
603b3c4b26
Try to fix sizes in unit test 2023-09-15 09:32:43 +00:00
Sema Checherinda
d9e15c00c9 limit the delay before next try in S3 2023-09-14 19:45:07 +02:00
Antonio Andelic
ae88bd3da5
Merge pull request #54572 from ClickHouse/fix-install-backup 
Fix snapshot install in Keeper
 2023-09-14 14:44:45 +02:00
Antonio Andelic
1c18f6c21d Use log idx for applyUncommittedState 2023-09-13 10:13:02 +00:00
Antonio Andelic
479ae52f9c Merge branch 'master' into keeper-batch-flushes 2023-09-11 11:36:23 +00:00
serxa
6a2534b555 fix keeper build #2 2023-09-10 17:43:04 +00:00
serxa
d326554caa fix keeper build 2023-09-10 14:55:40 +00:00
Antonio Andelic
4a1e2959fa Apply PR comments 2023-09-08 12:52:35 +00:00
Antonio Andelic
5f1a75ceae Fix tests 2023-09-01 12:13:33 +00:00
Antonio Andelic
7739dd7157 Fix integration tests 2023-08-31 14:52:42 +00:00
Antonio Andelic
dd824fe558 Merge branch 'master' into keeper-batch-flushes 2023-08-31 14:35:36 +00:00
Antonio Andelic
4137cb916d Merge branch 'master' into keeper-batch-flushes 2023-08-28 13:14:06 +00:00
alesapin
0e601b856f
Merge pull request #53751 from Algunenano/clang18 
Support clang-18 (Wmissing-field-initializers)
 2023-08-25 23:27:06 +02:00
Pengyuan Bian
c82cc148e9 Add configuration for reconnect limit in raft limit. 2023-08-25 00:38:49 +00:00
Raúl Marín
6490213207 Merge remote-tracking branch 'blessed/master' into clang18 2023-08-24 17:35:26 +02:00
Arthur Passos
2bade7db08
Add global proxy setting (#51749) 
* initial impl

* fix env ut

* move ut directory

* make sure no null proxy resolver is returned by ProxyConfigurationResolverProvider

* minor adjustment

* add a few tests, still incomplete

* add proxy support for url table function

* use proxy for select from url as well

* remove optional from return type, just returns empty config

* fix style

* style

* black

* ohg boy

* rm in progress file

* god pls don't let me kill anyone

* ...

* add use_aws guards

* remove hard coded s3 proxy resolver

* add concurrency-mt-unsafe

* aa

* black

* add logging back

* revert change

* imrpove code a bit

* helper functions and separate tests

* for some reason, this env test is not working..

* formatting

* :)

* clangtidy

* lint

* revert some stupid things

* small test adjusmtments

* simplify tests

* rename test

* remove extra line

* freaking style change

* simplify a bit

* fix segfault & remove an extra call

* tightly couple proxy provider with context..

* remove useless include

* rename config prefix parameter

* simplify provider a bit

* organize provider a bit

* add a few comments

* comment out proxy env tests

* fix nullptr in unit tests

* make sure old storage proxy config is properly covered without global context instance

* move a few functions from class to anonymous namespace

* fix no fallback for specific storage conf

* change API to accept http method instead of bool

* implement http/https distinction in listresolver, any still not implemented

* implement http/https distinction in remote resolver

* progress on code, improve tests and add url function working test

* use protcol instead of method for http and https

* small fix

* few more adjustments

* fix style

* black

* move enum to proxyconfiguration

* wip

* fix build

* fix ut

* delete atomicroundrobin class

* remove stale include

* add some tests.. need to spend some more time on the design..

* change design a bit

* progress

* use existing context for tests

* rename aux function and fix ut

* ..

* rename test

* try to simplify tests a bit

* simplify tests a bit more

* attempt to fix tests, accept more than one remote resolver

* use proper log id

* try waiting for resolver

* proper wait logic

* black

* empty

* address a few comments

* refactor tests

* remove old tests

* baclk

* use RAII to set/unset env

* black

* clang tidy

* fix env proxy not respecting any

* use log trace

* fix wrong logic in getRemoteREsolver

* fix wrong logic in getRemoteREsolver

* fix test

* remove unwanted code

* remove ClientConfigurationperRequest and auxilary classes

* remove unwanted code

* remove adapter test

* few adjustments and add test for s3 storage conf  with new proxy settings

* black

* use chassert for context

* Add getenv comment
 2023-08-24 16:07:26 +03:00
Raúl Marín
93dac0c880 Support clang-18 (Wmissing-field-initializers) 2023-08-23 15:53:45 +02:00
Antonio Andelic
625be86dc0 Fix tests 2023-08-23 09:00:57 +00:00
Antonio Andelic
59557ddd7c Merge branch 'master' into keeper-batch-flushes 2023-08-23 08:11:43 +00:00
Alexander Tokmakov
894ec8e0d0 less exceptions with runtime format string 2023-08-14 21:45:59 +02:00
Alexey Milovidov
aae35f3b1c
Merge pull request #53184 from ClickHouse/better-ccache 
Better usage of ccache
 2023-08-11 04:43:45 +03:00
Alexey Milovidov
ae6e59c4c5
Merge pull request #53099 from ClickHouse/remove-wikistat-loader 
Remove old util
 2023-08-10 17:01:57 +03:00
Alexey Milovidov
fa9abc5038 Better usage of ccache 2023-08-09 05:02:50 +02:00
Alexey Milovidov
aa757490bd Ditch tons of garbage 2023-08-09 02:19:02 +02:00
Alexey Milovidov
1ab99e5fdd Replace old docs with symlinks 2023-08-08 22:38:52 +02:00
Antonio Andelic
1dbe007ee5 Add coordination setting for async replication 2023-08-08 13:12:07 +00:00
Antonio Andelic
4b3e399f7d Wait for read requests 2023-08-07 13:04:38 +00:00
Antonio Andelic
c0ff486ada Merge branch 'master' into keeper-batch-flushes 2023-08-07 06:20:41 +00:00
Antonio Andelic
5cb856d167 Try batching multiple flush requests in Keeper 2023-08-04 11:53:22 +00:00
Vitaly Baranov
552331fd54 Use the same default paths for clickhouse_keeper (symlink) as for clickhouse_keeper (executable), 
i.e. "/var/lib/clickhouse-keeper/.."
 2023-08-04 13:38:21 +02:00
Anton Popov
92cb5e9ae3
Merge pull request #52687 from alkorgun/fix-named-collections-on-cluster-23.7 
Fix named collections on cluster 23.7
 2023-08-03 14:50:28 +02:00
Antonio Andelic
a6a495a767
Merge branch 'master' into fix-keeper-reductant-lock 2023-07-31 19:53:25 +02:00
Antonio Andelic
a473dc4b51 Wait for response 2023-07-31 09:24:06 +00:00
Илья Коргун
bcc41fbc2f Fix reference to a local binding declared in an enclosing function 2023-07-27 10:46:35 +03:00
Alexey Milovidov
21382afa2b Check for punctuation 2023-07-25 06:10:04 +02:00
Antonio Andelic
7b03f3277e
Try fix session on close test 2023-07-24 17:59:50 +02:00
Antonio Andelic
73155cf562
Merge branch 'master' into fix-keeper-reductant-lock 2023-07-24 10:17:35 +02:00
Antonio Andelic
10c2d5dade
Merge branch 'master' into feature/keeper-dyn-reconf 2023-07-21 13:24:18 +02:00
Antonio Andelic
4a47040763
Merge pull request #52358 from ClickHouse/fix-keeper-apply-snapshot 
Fix `apply_snapshot` in Keeper
 2023-07-21 12:52:12 +02:00
Antonio Andelic
f997adfe27 Retry if sessions not closed because missing leader 2023-07-20 14:02:55 +00:00
Antonio Andelic
a24bf14450 Use correct ZXID 2023-07-20 10:44:44 +00:00
Antonio Andelic
0c86df519f Fix unspported disks in Keeper 2023-07-19 12:41:25 +00:00
Antonio Andelic
61dc020b4d
Merge branch 'master' into feature/keeper-dyn-reconf 2023-07-17 12:50:17 +02:00
Antonio Andelic
9677cc5e2b
Apply suggestions from code review 2023-07-17 12:50:02 +02:00
frinkr
e1abdd0419
atomic exchange simplified 2023-07-17 10:51:24 +08:00
frinkr
6dceedd721
Merge branch 'ClickHouse:master' into fix-keeper-reductant-lock 2023-07-17 10:21:05 +08:00
Azat Khuzhin
6abdde12ca Print Zxid in keeper stat command in hex (so as ZooKeeper) 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-07-14 21:17:09 +02:00
Alexey Milovidov
b1d1918607
Merge branch 'master' into retry 2023-07-08 03:08:48 +03:00
Alexey Milovidov
685f2949b7
Revert "Fix build" 2023-07-07 02:53:13 +03:00
Alexey Milovidov
c47b32b17a Fix build 2023-07-07 01:52:18 +02:00
Alexey Milovidov
48eb30de51 Fix build 2023-07-07 01:42:29 +02:00
Mike Kot
5302b478a4 proper reconfig batch handling 2023-07-07 00:20:54 +03:00
Mike Kot
bafcc3afdc remove reconfig in process flag as it's useless 2023-07-07 00:20:54 +03:00
Mike Kot
c46b125d0a review fixes 2023-07-07 00:20:54 +03:00
Mike Kot
2f0cd05497 handle leader removal corner cases 2023-07-07 00:20:54 +03:00
Mike Kot
1bef6fc76c process reconfig in keeper dispatcher 2023-07-07 00:20:54 +03:00
Mike Kot
297d566600 acl check 2023-07-07 00:20:54 +03:00
Mike Kot
b4f750ed66 review fixes 2023-07-07 00:20:54 +03:00
Mike Kot
8b6376005a "reconfig" support for CH Keeper 2023-07-07 00:20:54 +03:00
frinkr
36fbc22b16
Merge branch 'master' into fix-keeper-reductant-lock 2023-07-04 18:04:08 +08:00
Jiang Yuqing
f443a00d14
KeeperDispatcher: remove reductant lock as the ConcurrentBoundedQueue is thread-safe 2023-07-04 17:58:04 +08:00
Antonio Andelic
2058b1346e Merge branch 'master' into keeper-with-disks 2023-06-30 13:22:33 +00:00
Antonio Andelic
eff1cc0e5d
Remove unused errorcode 2023-06-28 10:20:33 +02:00
Antonio Andelic
be852d554b Use magic_enum for values 2023-06-28 07:45:20 +00:00
Antonio Andelic
efbe01b6bb Merge branch 'master' into keeper-add-api-version-config 2023-06-27 10:11:04 +00:00
Alexey Milovidov
7407330130
Merge branch 'master' into retry 2023-06-23 08:18:18 +03:00
Michael Kolupaev
4a570a05c9 Decrease default timeouts for S3 and HTTP requests 2023-06-21 18:08:50 +00:00
Sema Checherinda
1cb02e2710 do call finalize for all buffers 2023-06-16 16:38:18 +02:00
Antonio Andelic
4ac090b122 Fix tests 2023-06-13 11:01:44 +00:00
Antonio Andelic
6f1c486537
Fix build 2023-06-13 10:25:53 +02:00
Antonio Andelic
25f08f8d19
Remove duplicate include 2023-06-12 15:35:44 +02:00
Antonio Andelic
cb3ac81110
Merge branch 'master' into keeper-with-disks 2023-06-12 15:14:24 +02:00
Antonio Andelic
a7e6264d56 Add backward compatibility 2023-06-12 13:12:05 +00:00
Antonio Andelic
b634012feb Add feature flags for Keeper API 2023-06-12 10:57:03 +00:00
Antonio Andelic
91a4b6e331 Merge branch 'master' into keeper-add-api-version-config 2023-06-12 07:10:58 +00:00
Antonio Andelic
b8b70d78e1 Build fix 2023-06-09 13:34:27 +00:00
Antonio Andelic
07582d56f3 Add config for pinning api version 2023-06-09 13:18:13 +00:00
Antonio Andelic
ab8365630b Reject logs containing unknown operation 2023-06-09 10:10:40 +00:00
Antonio Andelic
f4d182ed67
Merge branch 'master' into keeper-operation-create-if-not-exists 2023-06-06 09:19:35 +02:00
Antonio Andelic
ece96f54e9 Fix tests 2023-06-05 09:47:58 +00:00
Antonio Andelic
c4873027c3 Merge branch 'master' into keeper-with-disks 2023-06-05 07:08:33 +00:00
Alexey Milovidov
bed19aa792
Merge pull request #50483 from ClickHouse/fix-keeper-snapshot-install 
Keeper fix: apply uncommitted state after snapshot install
 2023-06-05 04:39:33 +03:00
Antonio Andelic
47966c4e81 Finalize bufferg 2023-06-02 14:10:52 +00:00
Antonio Andelic
336c9d7136 Apply uncommitted state after snapshot deser 2023-06-02 13:31:14 +00:00
Antonio Andelic
ffd4f7f196 Add better support for state disks 2023-06-01 14:39:01 +00:00
Antonio Andelic
e8442b7a88 Add finalize calls 2023-06-01 12:54:00 +00:00
frinkr
c9d0d217f5 fix Keeper deadlock on exception when preprocessing requests 2023-06-01 09:57:42 +08:00
Konstantin Bogdanov
b61ffe3ff7
Merge checks 2023-06-01 03:22:40 +02:00
Konstantin Bogdanov
3a3539b996
Style fix 2023-06-01 03:22:40 +02:00
Konstantin Bogdanov
203276dfcc
Use storage.uncommited_state 2023-06-01 03:22:39 +02:00
Konstantin Bogdanov
965f7850f8
Cleanup 2023-06-01 03:22:39 +02:00
Konstantin Bogdanov
f84fdb7f10
Increment Keeper API version 2023-06-01 03:22:39 +02:00
Konstantin Bogdanov
b53f36369e
Remove new request object 2023-06-01 03:22:39 +02:00
Konstantin Bogdanov
0d6561ee77
Prototype 2023-06-01 03:22:23 +02:00
Antonio Andelic
21aba94909 Add support for latest snapshot disk 2023-05-30 13:22:40 +00:00
Antonio Andelic
67746bfff2 Merge branch 'master' into keeper-with-disks 2023-05-30 09:50:12 +00:00
Raúl Marín
1ae1f1273a Avoid clang 15 crash 2023-05-29 11:40:45 +02:00
Antonio Andelic
0740bfbe4b Better disks moving 2023-05-26 14:31:09 +00:00
Antonio Andelic
a9569478e6
fix typo 2023-05-26 13:09:09 +02:00
Antonio Andelic
8a2a63a7bd Merge branch 'master' into keeper-with-disks 2023-05-26 10:48:41 +00:00
Antonio Andelic
3d98e591ba Handle backwards compatibility 2023-05-26 10:46:18 +00:00
Antonio Andelic
767193caf5 Merge branch 'master' into keeper-log-improvements 2023-05-26 07:35:36 +00:00
Antonio Andelic
6a8a21e09a Add test for s3 disk restore 2023-05-25 16:01:40 +00:00
Antonio Andelic
bb77441acb Add support for old storage disks 2023-05-25 13:31:11 +00:00
Antonio Andelic
f519aa4613 Calculate size 2023-05-25 11:23:03 +00:00
Antonio Andelic
4483602c50 Remove double initialization 2023-05-25 08:52:14 +00:00
Antonio Andelic
7e68f61df1 Fix snapshot recovery 2023-05-25 08:43:11 +00:00
Antonio Andelic
092cf99147 Fix build 2023-05-24 13:45:14 +00:00
Antonio Andelic
fa3fb49d33 Merge branch 'master' into keeper-with-disks 2023-05-24 09:04:47 +00:00
Antonio Andelic
5db2160762 Fix build 2023-05-24 09:04:12 +00:00
Antonio Andelic
161afea266 Add support for changelog 2023-05-24 07:43:57 +00:00
kssenii
295fe3b228 Merge remote-tracking branch 'upstream/master' into abstract-async-prefetched-buffer 2023-05-22 20:01:38 +02:00
kssenii
c4d862a16f Make async reader work with any impl 2023-05-22 19:54:04 +02:00
Antonio Andelic
bde2cf96b1 Better 2023-05-22 12:24:16 +00:00
alesapin
cc3897a84a
Merge pull request #50033 from kssenii/disk-object-storage-minor-changes 
Get rid of indirect write buffer in object storages
 2023-05-22 14:08:17 +02:00
Antonio Andelic
dc31396035 Merge branch 'master' into keeper-with-disks 2023-05-22 08:10:24 +00:00
kssenii
3e42ee7f2b Get rid of finalize callback in object storages 2023-05-19 17:29:37 +02:00
Antonio Andelic
9c3b17fa18
Remove whitespace 2023-05-19 13:00:51 +02:00
Antonio Andelic
e46476dba2
Update src/Coordination/Changelog.cpp 
Co-authored-by: alesapin <alesapin@clickhouse.com>
 2023-05-19 12:44:20 +02:00
Antonio Andelic
6e468b29e8 Check return value of ftruncate 2023-05-19 10:15:06 +00:00
Antonio Andelic
bf6afd27a6 build with disks 2023-05-18 07:39:22 +00:00
Antonio Andelic
465123fa1b Merge branch 'master' into keeper-with-disks 2023-05-16 08:07:06 +00:00
Azat Khuzhin
665545ec45 Fix "reference to local binding" after fixes for clang-17 
Follow-up for: #49851 (cc @alexey-milovidov)
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-05-15 12:45:20 +02:00
Alexey Milovidov
5a44dc26e7 Fixes for clang-17 2023-05-13 02:57:31 +02:00
Antonio Andelic
bb8b9dc9c3 Merge branch 'master' into keeper-log-improvements 2023-05-05 11:57:14 +00:00
alesapin
9456eb59fd
Merge pull request #48466 from ClickHouse/keeper-correctly-handle-snapshots 
Correctly handle concurrent snapshots in Keeper
 2023-05-04 19:16:00 +02:00
johanngan
731823b873 Add support for SSE-KMS configuration with S3 
https://docs.aws.amazon.com/AmazonS3/latest/userguide/UsingKMSEncryption.html

Similar to the server_side_encryption_customer_key_base64 option for
configuring SSE-C with S3, add the following settings to configure
SSE-KMS on a per-endpoint/disk basis:
  - server_side_encryption_kms_key_id
  - server_side_encryption_kms_encryption_context
  - server_side_encryption_kms_bucket_key_enabled
 2023-05-03 21:35:38 -05:00
Antonio Andelic
b8b7a362b2
Merge branch 'master' into keeper-correctly-handle-snapshots 2023-05-03 13:51:09 +02:00
Antonio Andelic
78ecfac8d5 Merge branch 'master' into keeper-log-improvements 2023-05-03 08:31:21 +00:00
Antonio Andelic
dce61d7321 Merge branch 'master' into keeper-log-improvements 2023-05-02 11:14:48 +00:00
Antonio Andelic
f695040a19
Merge pull request #48897 from ClickHouse/keeper-check-not-exists 
Add `CheckNotExists` request to Keeper
 2023-05-02 13:14:22 +02:00
Raúl Marín
f0e045bb3d Merge remote-tracking branch 'blessed/master' into arenita 2023-04-24 10:42:56 +02:00
Alexey Milovidov
530b764953 Fix IBM 2023-04-21 12:38:45 +02:00
Antonio Andelic
d500a5637d Cache large requests 2023-04-21 09:43:04 +00:00
Antonio Andelic
b52646c555 Eliminate double log serializing 2023-04-21 08:20:35 +00:00
Antonio Andelic
58e9b56fcb Fix CheckNotExists 2023-04-19 09:06:20 +00:00
Antonio Andelic
fb16623d48 Add CheckNotExists request to Keeper 2023-04-18 14:45:24 +00:00
Raúl Marín
2b70e08f23 Don't count unreserved bytes in Arenas as read_bytes 2023-04-13 12:43:24 +02:00
Antonio Andelic
b30aff06d8
Merge pull request #48595 from aalexfvk/alexfvk/fix_create_not_auth_problem 
Fix bug in Keeper when a node is not created with scheme `auth` in ACL sometimes.
 2023-04-11 14:41:37 +02:00
Aleksei Filatov
dd2364361c
Update src/Coordination/tests/gtest_coordination.cpp 
Co-authored-by: Antonio Andelic <antonio2368@users.noreply.github.com>
 2023-04-11 10:45:30 +03:00
Azat Khuzhin
79b83c4fd2 Remove superfluous includes of logger_userful.h from headers 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-04-10 17:59:30 +02:00
Aleksei Filatov
e8bf96d126 Add unit tests 2023-04-10 13:02:43 +03:00
Aleksei Filatov
000c2b5b91 Correct FixupACL for auth scheme 2023-04-10 13:02:31 +03:00
Azat Khuzhin
011480924a Use forward declaration of ThreadPool 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-04-07 11:25:35 +02:00
Antonio Andelic
88f77c686d Correctly handle concurrent snapshots 2023-04-06 15:40:05 +00:00
Antonio Andelic
e982f2a67a Merge branch 'master' into enable-env-credentials-default 2023-03-31 09:11:01 +00:00
Antonio Andelic
44e95aa65f Merge branch 'master' into nosign-keyword-for-s3 2023-03-29 11:10:03 +00:00
Antonio Andelic
160aa186bb Add support for NOSIGN keyword and no_sign_request config 2023-03-28 07:05:35 +00:00
Antonio Andelic
79e22d13fb small fix 2023-03-27 07:41:31 +00:00
Antonio Andelic
f355bc5342 Merge branch 'master' into more-batching-keeper 2023-03-27 07:28:17 +00:00
Antonio Andelic
6c3765c8b6 Small fix 2023-03-24 20:40:45 +00:00
Antonio Andelic
f353561204 Avoid breaking batches with read requests 2023-03-24 14:37:40 +00:00
Alexander Gololobov
afe3af230f Limit keeper request batching by size in bytes 2023-03-24 14:41:09 +01:00
Antonio Andelic
51416da818 Merge branch 'master' into enable-env-credentials-default 2023-03-13 10:20:21 +00:00
Antonio Andelic
a170a909a4 Add expiration window for S3 credentials 2023-03-10 10:06:32 +00:00