Commit Graph

52 Commits

Author SHA1 Message Date
Antonio Andelic
7421d8f872 Merge branch 'master' into keeper-use-fallocate 2023-01-25 11:36:47 +00:00
Alexander Tokmakov
870cfcc36a less fmt::runtime usages 2023-01-17 00:11:59 +01:00
Antonio Andelic
c2c115d2ba Add setting for max_log_file_size 2022-12-19 11:51:05 +00:00
Raúl Marín
54db7c6520 Enforce checking read output 2022-11-11 10:56:18 +01:00
alesapin
b6d61318d0 One more validation for broken config in keeper 2022-09-18 00:04:16 +02:00
Antonio Andelic
6dcdf0fcb1 Throw CORRUPTED_DATA in debug mode for invalid checksum 2022-07-12 06:41:58 +00:00
Antonio Andelic
568ba5d3cc Address PR comments 2022-07-11 12:56:09 +00:00
Antonio Andelic
e4f7635df7 Format 2022-07-10 19:10:27 +00:00
Antonio Andelic
183f5cc940 Add checksum to the state file 2022-07-10 15:01:38 +00:00
Antonio Andelic
2b26865f90 Unit test fixes 2022-07-10 14:40:08 +00:00
Antonio Andelic
4622084d2a Add test for persisted state 2022-07-10 09:47:04 +00:00
Antonio Andelic
f60074e9ec Make state durable 2022-07-10 09:33:49 +00:00
Dmitry Novik
2ed5a4013a
Revert "Revert "Memory overcommit: continue query execution if memory is available"" 2022-05-03 00:45:13 +02:00
alesapin
f0b7af0aa2
Revert "Memory overcommit: continue query execution if memory is available" 2022-05-03 00:36:50 +02:00
mergify[bot]
e9fde5d067
Merge branch 'master' into memory-overcommit-free 2022-04-29 19:31:59 +00:00
Antonio Andelic
44ff5f1697 Add some checks for endpoint change 2022-04-28 08:54:05 +00:00
alesapin
1fffa56f63 Merge branch 'master' into memory-overcommit-free 2022-04-25 20:21:08 +02:00
Antonio Andelic
4e389d8df1 Rename to hostname, add tests 2022-04-22 08:16:14 +00:00
Antonio Andelic
6ac3784af7 Add config example 2022-04-21 13:49:36 +00:00
Antonio Andelic
1778ddda97 Simplify code 2022-04-21 13:32:12 +00:00
Antonio Andelic
792b06cc66 Add config for host check 2022-04-21 13:32:12 +00:00
Antonio Andelic
b2770fbe1c Add setting for disabling checks 2022-04-21 13:32:12 +00:00
Robert Schulze
118e94523c
Activate clang-tidy warning "readability-container-contains"
This check suggests replacing <Container>.count() by
<Container>.contains() which is more speaking and in case of
multimaps/multisets also faster.
2022-04-18 23:53:11 +02:00
alesapin
c0317c818c Fix bug in keeper 2022-04-14 17:33:46 +02:00
alesapin
e81bbfb5b5 Review fixes 2022-03-07 12:13:37 +01:00
alesapin
4d86bf6fdf Merge branch 'master' into more_keeper_sanity_checks 2022-03-07 11:41:04 +01:00
alesapin
2ab920a404 Review fixes 2022-03-04 13:14:38 +01:00
alesapin
f24fa16184 Fix address check 2022-03-03 11:29:43 +01:00
alesapin
04d4c52e01 Add sanity checks for keeper 2022-03-02 20:37:59 +01:00
alesapin
74ff3ee4bc Add sanity check for localhost 2022-03-02 20:02:02 +01:00
Azat Khuzhin
0025110a16 Replace exit() with abort() in case of NuRaft errors
CI founds [1]:

    WARNING: ThreadSanitizer: data race (pid=99)
      Write of size 8 at 0x7b14002192b0 by thread T27:
        12 std::__1::map<>
        13 DB::OpenedFileCache::~OpenedFileCache() obj-x86_64-linux-gnu/../src/IO/OpenedFileCache.h:27:7 (clickhouse+0xac66de6)
        14 cxa_at_exit_wrapper(void*) crtstuff.c (clickhouse+0xaa3646f)
        15 decltype(*(std::__1::forward<nuraft::raft_server*&>(fp0)).*fp()) std::__1::__invoke<void ()(), nuraft::raft_server*&, void>()

      Previous read of size 8 at 0x7b14002192b0 by thread T37 (mutexes: write M732116415018761216):
        4 DB::OpenedFileCache::get() obj-x86_64-linux-gnu/../src/IO/OpenedFileCache.h:47:37 (clickhouse+0xac66784)

    Thread T27 'nuraft_commit' (tid=193, running) created by main thread at:
    ...
    Thread T37 'MergeMutate' (tid=204, running) created by main thread at:
    ...

But it also reports that the mutex was already destroyed:

    Mutex M732116415018761216 is already destroyed.

The problem is that [nuraft can call `exit()`](1707a7572a/src/handle_commit.cxx (L157-L158)) which will call atexit handlers:

    2022.02.17 22:54:03.495450 [ 193 ] {} <Error> RaftInstance: background committing thread encounter err Memory limit (total) exceeded: would use 56.56 GiB (attempt to allocate chunk of 8192 bytes), maximum: 55.82 GiB, exiting to protect the system

  [1]: https://s3.amazonaws.com/clickhouse-test-reports/33057/5a8cf3ac98808dadf125068a33ed9c622998a484/fuzzer_astfuzzertsan,actions//report.html

Let's replace exit() with abort() to avoid this.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-02-19 13:05:08 +03:00
alesapin
437591f2fa Add check for duplicate hostnames and IDs in KeeperConfig 2021-12-02 14:46:33 +03:00
alesapin
3f8f08e816 Slightly better 2021-11-19 12:30:58 +03:00
alesapin
dcec086573 Small refactoring 2021-11-18 23:17:22 +03:00
JackyWoo
4908f714f4 dump new coordination settings 2021-10-27 23:57:25 +08:00
JackyWoo
a60663e33d add 4lw commands to keeper 2021-10-27 20:26:42 +08:00
alesapin
e530682529 Build fixes 2021-10-19 22:47:04 +03:00
alesapin
042eebd981 Add test for leader remove 2021-10-19 16:37:28 +03:00
alesapin
647856658f Add synchronization and active wait 2021-10-19 16:11:29 +03:00
alesapin
2d4b601d38 Better 2021-10-19 15:00:26 +03:00
alesapin
a992895b09 Debug 2021-10-19 10:14:53 +03:00
alesapin
bfe2a937eb At least able to start 2021-10-18 18:27:51 +03:00
Azat Khuzhin
a5a2c5ef8a Move KEEPER_DEFAULT_PATH into separate header 2021-10-03 14:34:03 +03:00
alesapin
f980a414ee Better tests and ability to work without compression 2021-09-22 13:38:06 +03:00
alesapin
ac7579f187 Compressed logs for keeper 2021-09-21 17:29:05 +03:00
alesapin
31181095e7 Review fixes 2021-05-18 17:08:56 +03:00
alesapin
43ee9f0a3a Check for directory owner 2021-05-12 17:05:44 +03:00
alesapin
338ff1615c Simplier config 2021-04-12 15:40:01 +03:00
alesapin
ef34c95c7e Add SSL to keeper (both client and internal) 2021-04-12 15:25:52 +03:00
alesapin
c28a3b860c Fix Coordination darwin build 2021-04-08 17:17:57 +03:00