mirror of
https://github.com/ClickHouse/ClickHouse.git
synced 2024-12-04 13:32:13 +00:00
5a4f21c50f
- TSA is a static analyzer build by Google which finds race conditions
and deadlocks at compile time.
- It works by associating a shared member variable with a
synchronization primitive that protects it. The compiler can then
check at each access if proper locking happened before. A good
introduction are [0] and [1].
- TSA requires some help by the programmer via annotations. Luckily,
LLVM's libcxx already has annotations for std::mutex, std::lock_guard,
std::shared_mutex and std::scoped_lock. This commit enables them
(--> contrib/libcxx-cmake/CMakeLists.txt).
- Further, this commit adds convenience macros for the low-level
annotations for use in ClickHouse (--> base/defines.h). For
demonstration, they are leveraged in a few places.
- As we compile with "-Wall -Wextra -Weverything", the required compiler
flag "-Wthread-safety-analysis" was already enabled. Negative checks
are an experimental feature of TSA and disabled
(--> cmake/warnings.cmake). Compile times did not increase noticeably.
- TSA is used in a few places with simple locking. I tried TSA also
where locking is more complex. The problem was usually that it is
unclear which data is protected by which lock :-(. But there was
definitely some weird code where locking looked broken. So there is
some potential to find bugs.
*** Limitations of TSA besides the ones listed in [1]:
- The programmer needs to know which lock protects which piece of shared
data. This is not always easy for large classes.
- Two synchronization primitives used in ClickHouse are not annotated in
libcxx:
(1) std::unique_lock: A releaseable lock handle often together with
std::condition_variable, e.g. in solve producer-consumer problems.
(2) std::recursive_mutex: A re-entrant mutex variant. Its usage can be
considered a design flaw + typically it is slower than a standard
mutex. In this commit, one std::recursive_mutex was converted to
std::mutex and annotated with TSA.
- For free-standing functions (e.g. helper functions) which are passed
shared data members, it can be tricky to specify the associated lock.
This is because the annotations use the normal C++ rules for symbol
resolution.
[0] https://clang.llvm.org/docs/ThreadSafetyAnalysis.html
[1] https://static.googleusercontent.com/media/research.google.com/en//pubs/archive/42958.pdf
78 lines
2.4 KiB
C++
78 lines
2.4 KiB
C++
#pragma once
|
|
#include <libnuraft/log_store.hxx>
|
|
#include <map>
|
|
#include <mutex>
|
|
#include <Core/Types.h>
|
|
#include <Coordination/Changelog.h>
|
|
#include <Common/logger_useful.h>
|
|
#include <base/defines.h>
|
|
|
|
namespace DB
|
|
{
|
|
|
|
/// Wrapper around Changelog class. Implements RAFT log storage.
|
|
class KeeperLogStore : public nuraft::log_store
|
|
{
|
|
public:
|
|
KeeperLogStore(const std::string & changelogs_path, uint64_t rotate_interval_, bool force_sync_, bool compress_logs_);
|
|
|
|
/// Read log storage from filesystem starting from last_commited_log_index
|
|
void init(uint64_t last_commited_log_index, uint64_t logs_to_keep);
|
|
|
|
uint64_t start_index() const override;
|
|
|
|
uint64_t next_slot() const override;
|
|
|
|
/// return last entry from log
|
|
nuraft::ptr<nuraft::log_entry> last_entry() const override;
|
|
|
|
/// Append new entry to log
|
|
uint64_t append(nuraft::ptr<nuraft::log_entry> & entry) override;
|
|
|
|
/// Remove all entries starting from index and write entry into index position
|
|
void write_at(uint64_t index, nuraft::ptr<nuraft::log_entry> & entry) override;
|
|
|
|
/// Return entries between [start, end)
|
|
nuraft::ptr<std::vector<nuraft::ptr<nuraft::log_entry>>> log_entries(uint64_t start, uint64_t end) override;
|
|
|
|
/// Return entry at index
|
|
nuraft::ptr<nuraft::log_entry> entry_at(uint64_t index) override;
|
|
|
|
/// Term if the index
|
|
uint64_t term_at(uint64_t index) override;
|
|
|
|
/// Serialize entries in interval [index, index + cnt)
|
|
nuraft::ptr<nuraft::buffer> pack(uint64_t index, int32_t cnt) override;
|
|
|
|
/// Apply serialized entries starting from index
|
|
void apply_pack(uint64_t index, nuraft::buffer & pack) override;
|
|
|
|
/// Entries from last_log_index can be removed from memory and from disk
|
|
bool compact(uint64_t last_log_index) override;
|
|
|
|
/// Call fsync to the stored data
|
|
bool flush() override;
|
|
|
|
/// Stop background cleanup thread in change
|
|
void shutdownChangelog();
|
|
|
|
/// Flush logstore and call shutdown of background thread in changelog
|
|
bool flushChangelogAndShutdown();
|
|
|
|
/// Current log storage size
|
|
uint64_t size() const;
|
|
|
|
/// Flush batch of appended entries
|
|
void end_of_append_batch(uint64_t start_index, uint64_t count) override;
|
|
|
|
/// Get entry with latest config in logstore
|
|
nuraft::ptr<nuraft::log_entry> getLatestConfigChange() const;
|
|
|
|
private:
|
|
mutable std::mutex changelog_lock;
|
|
Poco::Logger * log;
|
|
Changelog changelog TSA_GUARDED_BY(changelog_lock);
|
|
};
|
|
|
|
}
|