thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-11-14 03:25:15 +00:00
Code Issues Packages Projects Releases Wiki Activity
86,550 Commits 9,131 Branches 1,774 Tags 2.1 GiB
151399b6a0
Commit Graph

459 Commits

Author SHA1 Message Date
zhanglistar
0ae820550d fix ut error sometimes 2022-04-12 14:30:53 +08:00
Antonio Andelic
bf1f34ddb3 Fix unit tests 2022-04-06 07:43:48 +00:00
Antonio Andelic
d296eeee2d Small changes for Keeper 2022-04-05 13:56:28 +00:00
alesapin
d0bafe93d6 Fix name 2022-03-19 21:02:11 +01:00
alesapin
bf5b3a856d Rename some variables in keeper 2022-03-17 11:55:15 +01:00
zhangyuli1
3cba1177ee remove wchc from four_letter_word_white_list 2022-03-16 17:40:49 +08:00
Maksim Kita
65c52298b6 Fix clang-tidy warnings in Compression, Coordination, Core folders 2022-03-14 18:17:35 +00:00
alesapin
71ecd6be74
Merge pull request #35004 from ClickHouse/more_keeper_sanity_checks 
More keeper sanity checks
 2022-03-07 19:53:59 +01:00
alesapin
0c93c9e21d
Merge pull request #35010 from bigo-sg/keeperstatopt 
Keeper atomic stat
 2022-03-07 12:34:28 +01:00
alesapin
e81bbfb5b5 Review fixes 2022-03-07 12:13:37 +01:00
alesapin
4d86bf6fdf Merge branch 'master' into more_keeper_sanity_checks 2022-03-07 11:41:04 +01:00
zhanglistar
9efc8a1d38 Fix min/max stat 2022-03-07 11:11:14 +08:00
alesapin
5db9018ae0 Fix unitialized variable 2022-03-04 13:26:56 +01:00
alesapin
2ab920a404 Review fixes 2022-03-04 13:14:38 +01:00
alesapin
c33808d3db Fix read old records from logs 2022-03-03 15:18:03 +01:00
alesapin
f24fa16184 Fix address check 2022-03-03 11:29:43 +01:00
zhanglistar
9ee0d2c8a0 keeper atomic stat 2022-03-03 14:30:22 +08:00
alesapin
04d4c52e01 Add sanity checks for keeper 2022-03-02 20:37:59 +01:00
alesapin
74ff3ee4bc Add sanity check for localhost 2022-03-02 20:02:02 +01:00
Maksim Kita
b1a956c5f1 clang-tidy check performance-move-const-arg fix 2022-03-02 18:15:27 +00:00
alesapin
576ff0b40d
Merge pull request #34715 from azat/fix-opened-file-cache-race 
Workaround for a bug in NuRaft library
 2022-03-02 13:58:43 +01:00
Azat Khuzhin
54ceadd4b3 Avoid busy polling in keeper while searching for changelog files to delete 
Fixes: #34534
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-02-27 18:27:49 +03:00
alesapin
fd95d7a498
Merge pull request #34486 from bigo-sg/keeperVersion 
Keeper: add version to SnapshotableHashTable
 2022-02-23 13:35:49 +03:00
alesapin
9d21911ca7
Merge pull request #34534 from bigo-sg/changelogAsyncDel 
Keeper changelog async delete useless logs
 2022-02-23 10:30:23 +03:00
zhanglistar
0ca819197d delete useless header in changelog 2022-02-22 19:13:55 +08:00
zhanglistar
1291483094 changelog modify log 2022-02-22 19:09:45 +08:00
zhanglistar
96fe231c35 Changelog delete useless sleep in dctor 2022-02-22 19:07:33 +08:00
zhanglistar
6db770f841 Changelog add some logs and adjust thread exiting logic 2022-02-22 18:55:31 +08:00
zhangxiao871
a06e4b991e Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into yandex-master 2022-02-22 11:10:47 +08:00
zhangxiao871
ad124a4d10 compatible. 2022-02-22 11:10:14 +08:00
zhanglistar
0299fd296d Use ConcurrentBoundedQueue instead of boost spsc queue. 2022-02-22 09:34:33 +08:00
Azat Khuzhin
0025110a16 Replace exit() with abort() in case of NuRaft errors 
CI founds [1]:

    WARNING: ThreadSanitizer: data race (pid=99)
      Write of size 8 at 0x7b14002192b0 by thread T27:
        12 std::__1::map<>
        13 DB::OpenedFileCache::~OpenedFileCache() obj-x86_64-linux-gnu/../src/IO/OpenedFileCache.h:27:7 (clickhouse+0xac66de6)
        14 cxa_at_exit_wrapper(void*) crtstuff.c (clickhouse+0xaa3646f)
        15 decltype(*(std::__1::forward<nuraft::raft_server*&>(fp0)).*fp()) std::__1::__invoke<void ()(), nuraft::raft_server*&, void>()

      Previous read of size 8 at 0x7b14002192b0 by thread T37 (mutexes: write M732116415018761216):
        4 DB::OpenedFileCache::get() obj-x86_64-linux-gnu/../src/IO/OpenedFileCache.h:47:37 (clickhouse+0xac66784)

    Thread T27 'nuraft_commit' (tid=193, running) created by main thread at:
    ...
    Thread T37 'MergeMutate' (tid=204, running) created by main thread at:
    ...

But it also reports that the mutex was already destroyed:

    Mutex M732116415018761216 is already destroyed.

The problem is that [nuraft can call `exit()`](1707a7572a/src/handle_commit.cxx (L157-L158)) which will call atexit handlers:

    2022.02.17 22:54:03.495450 [ 193 ] {} <Error> RaftInstance: background committing thread encounter err Memory limit (total) exceeded: would use 56.56 GiB (attempt to allocate chunk of 8192 bytes), maximum: 55.82 GiB, exiting to protect the system

  [1]: https://s3.amazonaws.com/clickhouse-test-reports/33057/5a8cf3ac98808dadf125068a33ed9c622998a484/fuzzer_astfuzzertsan,actions//report.html

Let's replace exit() with abort() to avoid this.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-02-19 13:05:08 +03:00
alesapin
efb9a6d0fa
Merge pull request #34584 from bigo-sg/keerpSnapMemOpt 
Keeper memory optimization by not holding snapshot in memory
 2022-02-18 10:34:31 +03:00
zhanglistar
677b34d41d
Merge branch 'ClickHouse:master' into keeperVersion 2022-02-18 14:07:30 +08:00
zhanglistar
03c648e317
Update src/Coordination/KeeperStateMachine.cpp 
Co-authored-by: alesapin <alesapin@gmail.com>
 2022-02-17 18:15:25 +08:00
zhanglistar
cd0f7d5b16 Fix buffer not alloced bug 2022-02-17 16:33:35 +08:00
zhanglistar
7868153098 no std::cerr in KeeperSnapshotManager.h 2022-02-17 10:49:33 +08:00
zhanglistar
f2b2723a3d add buffer alloc and errno 2022-02-17 10:34:22 +08:00
zhanglistar
213a3481b5 move mmap file2buffer to function 2022-02-15 20:25:19 +08:00
alesapin
c2bdcbbe77
Merge pull request #34587 from bigo-sg/changelogNoclone 
Keeper change log no need to clone log entry
 2022-02-15 14:06:28 +03:00
zhanglistar
64d40fad2b
Merge branch 'master' into keeperVersion 2022-02-15 09:22:54 +08:00
Maksim Kita
db245cfbc9
Merge pull request #34523 from bigo-sg/nolockappendentries 
Keeper no lock on append_entries
 2022-02-14 21:28:10 +01:00
zhanglistar
b790903a61 keeper Changelog no need clone log entry 2022-02-14 22:43:37 +08:00
zhanglistar
ceeed2e928 keeper no memory snapshot 2022-02-14 21:34:55 +08:00
zhanglistar
edcea7dc31 keeper snapshot memory opt. 2022-02-14 19:43:08 +08:00
alexey-milovidov
ea71dc9d11
Merge pull request #34510 from kitaisreal/table-functions-insert-partition-by-refactoring 
Improve performance of insert into table functions URL, S3, File, HDFS
 2022-02-12 10:14:00 +03:00
zhanglistar
1dad40e25f Delete useless header 2022-02-12 00:04:08 +08:00
zhanglistar
5906b36095 Keeper changelog async delete useless logs 2022-02-11 23:58:15 +08:00
zhanglistar
e07c5751fa No lock on append_entries 2022-02-11 17:18:55 +08:00
Maksim Kita
13cbf79ecb Improve performance of insert into table functions URL, S3, File, HDFS 2022-02-10 20:06:23 +00:00