ClickHouse/programs/library-bridge
Robert Schulze 5a4f21c50f
Support for Clang Thread Safety Analysis (TSA)
- TSA is a static analyzer build by Google which finds race conditions
  and deadlocks at compile time.

- It works by associating a shared member variable with a
  synchronization primitive that protects it. The compiler can then
  check at each access if proper locking happened before. A good
  introduction are [0] and [1].

- TSA requires some help by the programmer via annotations. Luckily,
  LLVM's libcxx already has annotations for std::mutex, std::lock_guard,
  std::shared_mutex and std::scoped_lock. This commit enables them
  (--> contrib/libcxx-cmake/CMakeLists.txt).

- Further, this commit adds convenience macros for the low-level
  annotations for use in ClickHouse (--> base/defines.h). For
  demonstration, they are leveraged in a few places.

- As we compile with "-Wall -Wextra -Weverything", the required compiler
  flag "-Wthread-safety-analysis" was already enabled. Negative checks
  are an experimental feature of TSA and disabled
  (--> cmake/warnings.cmake). Compile times did not increase noticeably.

- TSA is used in a few places with simple locking. I tried TSA also
  where locking is more complex. The problem was usually that it is
  unclear which data is protected by which lock :-(. But there was
  definitely some weird code where locking looked broken. So there is
  some potential to find bugs.

*** Limitations of TSA besides the ones listed in [1]:

- The programmer needs to know which lock protects which piece of shared
  data. This is not always easy for large classes.

- Two synchronization primitives used in ClickHouse are not annotated in
  libcxx:
  (1) std::unique_lock: A releaseable lock handle often together with
      std::condition_variable, e.g. in solve producer-consumer problems.
  (2) std::recursive_mutex: A re-entrant mutex variant. Its usage can be
      considered a design flaw + typically it is slower than a standard
      mutex. In this commit, one std::recursive_mutex was converted to
      std::mutex and annotated with TSA.

- For free-standing functions (e.g. helper functions) which are passed
  shared data members, it can be tricky to specify the associated lock.
  This is because the annotations use the normal C++ rules for symbol
  resolution.

[0] https://clang.llvm.org/docs/ThreadSafetyAnalysis.html
[1] https://static.googleusercontent.com/media/research.google.com/en//pubs/archive/42958.pdf
2022-06-20 16:13:25 +02:00
..
CMakeLists.txt Simplify strip for new packages 2022-03-23 15:14:30 +01:00
HandlerFactory.cpp Review fixes 2021-08-02 13:40:58 +00:00
HandlerFactory.h base should not depend on Common 2022-04-29 10:26:35 +08:00
Handlers.cpp Fix formatting error in logging messages 2022-02-01 14:30:04 +03:00
Handlers.h base should not depend on Common 2022-04-29 10:26:35 +08:00
library-bridge.cpp First version 2021-03-05 10:19:01 +00:00
LibraryBridge.cpp Common base for bridges part 3 2021-03-07 13:55:40 +00:00
LibraryBridge.h base should not depend on Common 2022-04-29 10:26:35 +08:00
LibraryInterface.cpp base should not depend on Common 2022-04-29 10:26:35 +08:00
LibraryInterface.h LibraryDictionary bridge library interface 2021-04-13 22:53:36 +03:00
LibraryUtils.h Rename "common" to "base" 2021-10-02 10:13:14 +03:00
SharedLibraryHandler.cpp Remove streams from formats. 2021-10-11 19:11:50 +03:00
SharedLibraryHandler.h base should not depend on Common 2022-04-29 10:26:35 +08:00
SharedLibraryHandlerFactory.cpp Activate clang-tidy warning "readability-container-contains" 2022-04-18 23:53:11 +02:00
SharedLibraryHandlerFactory.h Support for Clang Thread Safety Analysis (TSA) 2022-06-20 16:13:25 +02:00