Commit Graph

67 Commits

Author SHA1 Message Date
Antonio Andelic
90bb796d63 Small Keeper fixes 2023-11-21 13:11:33 +00:00
Antonio Andelic
4a1e2959fa Apply PR comments 2023-09-08 12:52:35 +00:00
Antonio Andelic
1dbe007ee5 Add coordination setting for async replication 2023-08-08 13:12:07 +00:00
Antonio Andelic
4b3e399f7d Wait for read requests 2023-08-07 13:04:38 +00:00
Antonio Andelic
5cb856d167 Try batching multiple flush requests in Keeper 2023-08-04 11:53:22 +00:00
Mike Kot
8b6376005a "reconfig" support for CH Keeper 2023-07-07 00:20:54 +03:00
Antonio Andelic
ffd4f7f196 Add better support for state disks 2023-06-01 14:39:01 +00:00
Antonio Andelic
8a2a63a7bd Merge branch 'master' into keeper-with-disks 2023-05-26 10:48:41 +00:00
Antonio Andelic
161afea266 Add support for changelog 2023-05-24 07:43:57 +00:00
kssenii
c4d862a16f Make async reader work with any impl 2023-05-22 19:54:04 +02:00
Antonio Andelic
bde2cf96b1 Better 2023-05-22 12:24:16 +00:00
Antonio Andelic
465123fa1b Merge branch 'master' into keeper-with-disks 2023-05-16 08:07:06 +00:00
Azat Khuzhin
79b83c4fd2 Remove superfluous includes of logger_userful.h from headers
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-04-10 17:59:30 +02:00
LiuYangkuan
f29700bd2f use IDisk to do IO in Keeper's snapshots and logs 2023-02-02 19:47:30 +08:00
Antonio Andelic
1186718658 Add overallocate size setting 2023-01-26 09:44:37 +00:00
Antonio Andelic
7421d8f872 Merge branch 'master' into keeper-use-fallocate 2023-01-25 11:36:47 +00:00
Alexander Tokmakov
870cfcc36a less fmt::runtime usages 2023-01-17 00:11:59 +01:00
Antonio Andelic
c2c115d2ba Add setting for max_log_file_size 2022-12-19 11:51:05 +00:00
Raúl Marín
54db7c6520 Enforce checking read output 2022-11-11 10:56:18 +01:00
alesapin
b6d61318d0 One more validation for broken config in keeper 2022-09-18 00:04:16 +02:00
Antonio Andelic
6dcdf0fcb1 Throw CORRUPTED_DATA in debug mode for invalid checksum 2022-07-12 06:41:58 +00:00
Antonio Andelic
568ba5d3cc Address PR comments 2022-07-11 12:56:09 +00:00
Antonio Andelic
e4f7635df7 Format 2022-07-10 19:10:27 +00:00
Antonio Andelic
183f5cc940 Add checksum to the state file 2022-07-10 15:01:38 +00:00
Antonio Andelic
2b26865f90 Unit test fixes 2022-07-10 14:40:08 +00:00
Antonio Andelic
4622084d2a Add test for persisted state 2022-07-10 09:47:04 +00:00
Antonio Andelic
f60074e9ec Make state durable 2022-07-10 09:33:49 +00:00
Dmitry Novik
2ed5a4013a
Revert "Revert "Memory overcommit: continue query execution if memory is available"" 2022-05-03 00:45:13 +02:00
alesapin
f0b7af0aa2
Revert "Memory overcommit: continue query execution if memory is available" 2022-05-03 00:36:50 +02:00
mergify[bot]
e9fde5d067
Merge branch 'master' into memory-overcommit-free 2022-04-29 19:31:59 +00:00
Antonio Andelic
44ff5f1697 Add some checks for endpoint change 2022-04-28 08:54:05 +00:00
alesapin
1fffa56f63 Merge branch 'master' into memory-overcommit-free 2022-04-25 20:21:08 +02:00
Antonio Andelic
4e389d8df1 Rename to hostname, add tests 2022-04-22 08:16:14 +00:00
Antonio Andelic
6ac3784af7 Add config example 2022-04-21 13:49:36 +00:00
Antonio Andelic
1778ddda97 Simplify code 2022-04-21 13:32:12 +00:00
Antonio Andelic
792b06cc66 Add config for host check 2022-04-21 13:32:12 +00:00
Antonio Andelic
b2770fbe1c Add setting for disabling checks 2022-04-21 13:32:12 +00:00
Robert Schulze
118e94523c
Activate clang-tidy warning "readability-container-contains"
This check suggests replacing <Container>.count() by
<Container>.contains() which is more speaking and in case of
multimaps/multisets also faster.
2022-04-18 23:53:11 +02:00
alesapin
c0317c818c Fix bug in keeper 2022-04-14 17:33:46 +02:00
alesapin
e81bbfb5b5 Review fixes 2022-03-07 12:13:37 +01:00
alesapin
4d86bf6fdf Merge branch 'master' into more_keeper_sanity_checks 2022-03-07 11:41:04 +01:00
alesapin
2ab920a404 Review fixes 2022-03-04 13:14:38 +01:00
alesapin
f24fa16184 Fix address check 2022-03-03 11:29:43 +01:00
alesapin
04d4c52e01 Add sanity checks for keeper 2022-03-02 20:37:59 +01:00
alesapin
74ff3ee4bc Add sanity check for localhost 2022-03-02 20:02:02 +01:00
Azat Khuzhin
0025110a16 Replace exit() with abort() in case of NuRaft errors
CI founds [1]:

    WARNING: ThreadSanitizer: data race (pid=99)
      Write of size 8 at 0x7b14002192b0 by thread T27:
        12 std::__1::map<>
        13 DB::OpenedFileCache::~OpenedFileCache() obj-x86_64-linux-gnu/../src/IO/OpenedFileCache.h:27:7 (clickhouse+0xac66de6)
        14 cxa_at_exit_wrapper(void*) crtstuff.c (clickhouse+0xaa3646f)
        15 decltype(*(std::__1::forward<nuraft::raft_server*&>(fp0)).*fp()) std::__1::__invoke<void ()(), nuraft::raft_server*&, void>()

      Previous read of size 8 at 0x7b14002192b0 by thread T37 (mutexes: write M732116415018761216):
        4 DB::OpenedFileCache::get() obj-x86_64-linux-gnu/../src/IO/OpenedFileCache.h:47:37 (clickhouse+0xac66784)

    Thread T27 'nuraft_commit' (tid=193, running) created by main thread at:
    ...
    Thread T37 'MergeMutate' (tid=204, running) created by main thread at:
    ...

But it also reports that the mutex was already destroyed:

    Mutex M732116415018761216 is already destroyed.

The problem is that [nuraft can call `exit()`](1707a7572a/src/handle_commit.cxx (L157-L158)) which will call atexit handlers:

    2022.02.17 22:54:03.495450 [ 193 ] {} <Error> RaftInstance: background committing thread encounter err Memory limit (total) exceeded: would use 56.56 GiB (attempt to allocate chunk of 8192 bytes), maximum: 55.82 GiB, exiting to protect the system

  [1]: https://s3.amazonaws.com/clickhouse-test-reports/33057/5a8cf3ac98808dadf125068a33ed9c622998a484/fuzzer_astfuzzertsan,actions//report.html

Let's replace exit() with abort() to avoid this.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-02-19 13:05:08 +03:00
alesapin
437591f2fa Add check for duplicate hostnames and IDs in KeeperConfig 2021-12-02 14:46:33 +03:00
alesapin
3f8f08e816 Slightly better 2021-11-19 12:30:58 +03:00
alesapin
dcec086573 Small refactoring 2021-11-18 23:17:22 +03:00
JackyWoo
4908f714f4 dump new coordination settings 2021-10-27 23:57:25 +08:00