thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-11-10 09:32:06 +00:00
Code Issues Packages Projects Releases Wiki Activity
5a4f21c50f
ClickHouse/contrib
History
Robert Schulze 5a4f21c50f
Support for Clang Thread Safety Analysis (TSA) 
- TSA is a static analyzer build by Google which finds race conditions
  and deadlocks at compile time.

- It works by associating a shared member variable with a
  synchronization primitive that protects it. The compiler can then
  check at each access if proper locking happened before. A good
  introduction are [0] and [1].

- TSA requires some help by the programmer via annotations. Luckily,
  LLVM's libcxx already has annotations for std::mutex, std::lock_guard,
  std::shared_mutex and std::scoped_lock. This commit enables them
  (--> contrib/libcxx-cmake/CMakeLists.txt).

- Further, this commit adds convenience macros for the low-level
  annotations for use in ClickHouse (--> base/defines.h). For
  demonstration, they are leveraged in a few places.

- As we compile with "-Wall -Wextra -Weverything", the required compiler
  flag "-Wthread-safety-analysis" was already enabled. Negative checks
  are an experimental feature of TSA and disabled
  (--> cmake/warnings.cmake). Compile times did not increase noticeably.

- TSA is used in a few places with simple locking. I tried TSA also
  where locking is more complex. The problem was usually that it is
  unclear which data is protected by which lock :-(. But there was
  definitely some weird code where locking looked broken. So there is
  some potential to find bugs.

*** Limitations of TSA besides the ones listed in [1]:

- The programmer needs to know which lock protects which piece of shared
  data. This is not always easy for large classes.

- Two synchronization primitives used in ClickHouse are not annotated in
  libcxx:
  (1) std::unique_lock: A releaseable lock handle often together with
      std::condition_variable, e.g. in solve producer-consumer problems.
  (2) std::recursive_mutex: A re-entrant mutex variant. Its usage can be
      considered a design flaw + typically it is slower than a standard
      mutex. In this commit, one std::recursive_mutex was converted to
      std::mutex and annotated with TSA.

- For free-standing functions (e.g. helper functions) which are passed
  shared data members, it can be tricky to specify the associated lock.
  This is because the annotations use the normal C++ rules for symbol
  resolution.

[0] https://clang.llvm.org/docs/ThreadSafetyAnalysis.html
[1] https://static.googleusercontent.com/media/research.google.com/en//pubs/archive/42958.pdf
2022-06-20 16:13:25 +02:00
..
abseil-cpp@215105818d Abseil: Use upstream lts_2021_11_02 branch 2021-12-13 16:49:22 +01:00
abseil-cpp-cmake Cosmetics 2022-06-01 11:02:39 +02:00
AMQP-CPP@1a6c51f4ac fix unbundled build 2021-08-03 17:29:59 +03:00
amqpcpp-cmake Removed warning flags in contribs 2022-06-01 11:02:39 +02:00
arrow@efdcd015cf Revert "Remove resursive submodules" 2022-06-02 09:41:12 +03:00
arrow-cmake Cosmetics 2022-06-01 11:02:39 +02:00
avro@e43c46e87f More pytest fixes (#23538) 2021-06-07 20:29:19 +03:00
avro-cmake Removed warning flags in contribs 2022-06-01 11:02:39 +02:00
aws@00b0360454 Retry on Yandex.S3 throttle 2021-10-25 17:34:59 +03:00
aws-c-common@736a82d169
aws-c-event-stream@3bc33662f9
aws-checksums@519d6d9093
aws-s3-cmake contrib/aws-s3: use hidden libraries 2022-01-21 10:11:22 +03:00
azure@ac4b763d4c Updating azure submodule with ClickHouse-Extras fork 2021-12-10 22:04:38 +01:00
azure-cmake Removed warning flags in contribs 2022-06-01 11:02:39 +02:00
base64@9499e0c494 revert irrelate contribs 2021-12-20 18:30:40 +08:00
base64-cmake Replace variable ARCH_ARM by ARCH_AARCH64 2022-05-15 09:48:02 +02:00
boost@c0807e83f2 Use ClickHouse-Extras for boost· 2021-12-20 18:01:28 +01:00
boost-cmake Replace variable ARCH_ARM by ARCH_AARCH64 2022-05-15 09:48:02 +02:00
boringssl@9c0715ce45 Updated the boringssl submodule to the latest commit 2022-04-19 11:23:53 -07:00
boringssl-cmake add ssl version to build options 2022-06-04 15:31:08 +05:30
brotli@63be8a9940 Revert "Remove resursive submodules" 2022-06-02 09:41:12 +03:00
brotli-cmake Remove unused M_LIBRARY link 2022-06-01 11:02:39 +02:00
bzip2@bf905ea225 Add submodule bzip2 2021-08-07 06:18:14 +00:00
bzip2-cmake Cosmetics 2022-06-01 11:02:39 +02:00
capnproto@c8189ec3c2 Try to fix tests, update capnp lib to eliminate problem with UB sanitizer 2021-10-14 16:43:23 +03:00
capnproto-cmake Removed warning flags in contribs 2022-06-01 11:02:39 +02:00
cassandra@f4a31e92a2 update cassandra driver 2021-12-22 14:17:14 +03:00
cassandra-cmake Cosmetics 2022-06-01 11:02:39 +02:00
cctz@8c71d74bdf Bump cctz to 2022-05-15 2022-05-25 12:21:05 +02:00
cctz-cmake contrib/cctz: use hidden libraries 2022-01-21 10:11:22 +03:00
cityhash102 Remove unbundled cityhash support 2022-01-20 10:01:12 +03:00
cld2@bc6d493a2f Fix detectLanguage 2022-01-18 21:32:32 +00:00
cld2-cmake Merge branch 'master' into classification 2022-01-25 10:22:47 +00:00
consistent-hashing contrib/consistent-hashing: add ALIAS library 2022-01-21 10:11:22 +03:00
cppkafka@5a119f689f Revert "Remove resursive submodules" 2022-06-02 09:41:12 +03:00
cppkafka-cmake Cosmetics 2022-06-01 11:02:39 +02:00
croaring@2c867e9f9c Update roaring just in case 2021-06-10 02:36:36 +03:00
croaring-cmake contrib/croaring: add a cmake ALIAS library 2022-01-21 10:08:32 +03:00
curl@801bd5138c Upgrdaed the curl package to fix vulnerabilities reported by WhiteSource 2022-03-08 09:35:56 -08:00
curl-cmake Fix to restrict the curl, azure and sentry build in MacOS 2022-04-06 06:28:35 -07:00
cyrus-sasl@e6466edfd6 Fix 2021-05-28 11:38:20 +03:00
cyrus-sasl-cmake contrib/cyrus-sasl: use hidden library 2022-01-21 10:11:22 +03:00
datasketches-cpp@7d73d7610d Submodules 2021-05-28 01:45:13 +03:00
datasketches-cpp-cmake Remove unbundled datasketches support 2022-01-20 10:02:00 +03:00
double-conversion@cf2f0f3d54
double-conversion-cmake Remove unbundled double-conversion support 2022-01-20 08:47:16 +03:00
dragonbox@923705af6f
dragonbox-cmake contrib/dragonbox: add ALIAS library 2022-01-21 10:11:22 +03:00
fast_float@7eae925b51
fast_float-cmake contrib/fast_float: use hidden library 2022-01-21 10:11:22 +03:00
fastops@1460583af7 Update submodule 2021-10-16 03:09:38 +03:00
fastops-cmake Cosmetics 2022-06-01 11:02:39 +02:00
flatbuffers@eb3f827948 update 2021-06-18 13:34:15 +03:00
fmtlib@b6f4ceaed0 Update fmtlib (7.0.0 -> 8.1.1) 2022-02-01 14:30:03 +03:00
fmtlib-cmake Cosmetics 2022-06-01 11:02:39 +02:00
googletest@e7e591764b Fix 2021-05-28 11:38:20 +03:00
googletest-cmake Remove unbundled gtest support 2022-01-20 10:01:54 +03:00
grpc@7eac189a6b Revert "Fix style regressions on benchmark page (#30506)" 2021-10-25 20:02:35 +03:00
grpc-cmake Remove MAKE_STATIC_LIBRARIES (in favor of USE_STATIC_LIBRARIES) 2022-01-24 17:28:33 +03:00
h3@c7f46cfd71 Revert "Implement h3ToGeo function" 2021-06-22 17:25:21 +03:00
h3-cmake Remove unused M_LIBRARY link 2022-06-01 11:02:39 +02:00
hashidsxx@783f6911cc add hashid support 2022-05-08 06:42:51 +09:00
hashidsxx-cmake remove hashid define guard 2022-05-18 15:26:54 -04:00
hive-metastore@809a77d435 add submodule hive-metasotre 2021-11-16 11:38:24 +08:00
hive-metastore-cmake Cosmetics 2022-06-01 11:02:39 +02:00
hyperscan@5edc68c5ac Update submodules libc++ to LLVM 14 2022-03-26 11:12:22 +01:00
hyperscan-cmake contrib/hyperscan: use hidden library 2022-01-21 10:11:22 +03:00
icu@a56dde820d Upgrade icu to icu-release-70-1 2022-03-08 21:37:26 -08:00
icu-cmake Removed warning flags in contribs 2022-06-01 11:02:39 +02:00
icudata@72d9a4a7fe Upgrade icu to icu-release-70-1 2022-03-08 21:37:26 -08:00
jemalloc@78b58379c8 Fix possible "nmalloc >= ndalloc" jemalloc assertion 2022-03-02 06:55:27 +03:00
jemalloc-cmake Removed warning flags in contribs 2022-06-01 11:02:39 +02:00
krb5@5149dea4e2
krb5-cmake Merge pull request #35832 from tonickkozlov/patch-1 2022-04-05 00:49:10 +02:00
lemmagen-c@59537bdcf5 Updated submodules to include new READMEs 2021-06-19 23:15:16 +00:00
lemmagen-c-cmake Respect library type for contrib libraries (#36399) 2022-04-19 18:05:42 +02:00
libcpuid@8db3b8d2d3
libcpuid-cmake Removed warning flags in contribs 2022-06-01 11:02:39 +02:00
libcxx@172b2ae074 Update submodules libc++ to LLVM 14 2022-03-26 11:12:22 +01:00
libcxx-cmake Support for Clang Thread Safety Analysis (TSA) 2022-06-20 16:13:25 +02:00
libcxxabi@6eb7cc7a7b Update submodules libc++ to LLVM 14 2022-03-26 11:12:22 +01:00
libcxxabi-cmake Adapt libcxx CMakefiles to LLVM 14 changes 2022-03-24 10:25:04 +01:00
libdivide contrib/libdivide: add ALIAS library 2022-01-21 10:11:23 +03:00
libfarmhash Remove unbundled farmhash support 2022-01-20 10:01:12 +03:00
libgsasl@383ee28e82
libgsasl-cmake Cosmetics 2022-06-01 11:02:39 +02:00
libhdfs3@9194af4458 revert irrelate contribs 2021-12-20 18:30:40 +08:00
libhdfs3-cmake Replace variable ARCH_ARM by ARCH_AARCH64 2022-05-15 09:48:02 +02:00
libmetrohash contrib/libmetrohash: add ALIAS library 2022-01-21 10:11:23 +03:00
libpq@e071ea570f Update libpq 2021-07-02 22:26:33 +03:00
libpq-cmake contrib/libpq: use hidden library 2022-01-21 10:11:23 +03:00
libpqxx@a4e8348392 libpqxx without confgure_file cmake 2022-01-18 15:40:32 +00:00
libpqxx-cmake contrib/libpqxx: use hidden library 2022-01-21 10:11:23 +03:00
libprotobuf-mutator@ffd86a3287 adding codegen fuzzer + code generation script. new contrib added: libprotobuf-mutator 2021-10-12 03:21:31 +03:00
libprotobuf-mutator-cmake Remove unbundled libprotobuf-mutator support 2022-01-20 08:47:16 +03:00
librdkafka@b8554f1682 Bump librdkafka (to fix metadata cache destroying) 2021-08-19 21:38:06 +03:00
librdkafka-cmake Fix librdkafka/hdfs that depends on sasl2/krb5 2022-01-20 10:02:03 +03:00
libstemmer_c@c753054304 Updated submodules to include new READMEs 2021-06-19 23:15:16 +00:00
libstemmer-c-cmake Respect library type for contrib libraries (#36399) 2022-04-19 18:05:42 +02:00
libunwind@c4ea9848a6 Updated libunwind to protect from wrong CFA expressions 2021-08-24 21:36:45 +03:00
libunwind-cmake Revert special contribs and set folder manually for them until fixed 2021-10-17 23:10:01 +03:00
libuv@95081e7c16 Update submodules 2021-10-12 02:03:00 +03:00
libuv-cmake Cosmetics 2022-06-01 11:02:39 +02:00
libxml2@7846b0a677 Upgrade libxml2 to 2.9.14 2022-06-01 06:10:05 -07:00
libxml2-cmake Remove unused M_LIBRARY link 2022-06-01 11:02:39 +02:00
llvm@20607e6172 Update submodules to simplify cross build 2021-10-11 14:34:05 +03:00
llvm-cmake Always disable --color-diagnostics for LLVM 2022-06-10 10:39:38 +02:00
lz4@4c9431e9af Update lz4 contrib 2021-11-29 14:12:13 +03:00
lz4-cmake Remove USE_XXHASH (reduntant) 2022-01-20 10:02:02 +03:00
magic_enum@38f86e4d09 Initial: replacing hardcoded toString for enums with magic_enum 2021-09-06 16:24:03 +02:00
magic-enum-cmake contrib/magic-enum: use hidden library 2022-01-21 10:11:23 +03:00
mariadb-connector-c@5f4034a3a6 Submodules 2021-05-28 01:45:13 +03:00
mariadb-connector-c-cmake Respect library type for contrib libraries (#36399) 2022-04-19 18:05:42 +02:00
miniselect@be0af6bd0b
miniselect-cmake Move miniselect 2022-01-20 10:01:59 +03:00
minizip-ng@f3d400e999 Bump minizip-ng to a sane version 2022-04-15 20:03:13 +08:00
minizip-ng-cmake Cosmetics 2022-06-01 11:02:39 +02:00
msgpack-c@46684265d5 Revert "Remove resursive submodules" 2022-06-02 09:41:12 +03:00
msgpack-c-cmake Remove unbundled msgpack support 2022-01-20 10:01:58 +03:00
murmurhash contrib/murmurhash: add ALIAS library 2022-01-21 10:11:23 +03:00
nanodbc@df52a1232d Update nanodbc 2021-08-24 12:50:07 +00:00
nanodbc-cmake Cosmetics 2022-06-01 11:02:39 +02:00
nlp-data@5591f91f5e Better 2022-01-10 20:30:26 +00:00
nlp-data-cmake Merge branch 'master' into classification 2022-01-25 10:22:47 +00:00
NuRaft@24a13f15cf Better error message from NuRaft in case of CRC mismatch (#36746) 2022-04-28 17:19:28 +02:00
nuraft-cmake Remove unbundled nuraft support 2022-01-20 08:47:16 +03:00
openldap@0208811b60
openldap-cmake Remove unbundled openldap support 2022-01-20 08:47:16 +03:00
orc@f9a393ed24 Fix contrib 2022-01-27 13:49:56 +03:00
pdqsort
pdqsort-cmake Move pdqsort 2022-01-20 10:01:59 +03:00
poco@de35b9fd72 update poco 2022-05-13 17:34:29 -04:00
poco-cmake contrib/boringssl: use hidden libraries 2022-01-21 10:11:22 +03:00
protobuf@6bb70196c5 Update protobuf to ignore protoc MSAN errors 2021-12-13 18:54:31 +01:00
protobuf-cmake contrib/protobuf: use hidden libraries 2022-01-21 10:11:23 +03:00
rapidjson@c4ef90ccdb Revert "Remove resursive submodules" 2022-06-02 09:41:12 +03:00
rapidjson-cmake Remove unbundled rapidjson support 2022-01-20 10:01:12 +03:00
re2@13ebb377c6 Fix 2021-05-28 11:46:08 +03:00
re2-cmake Remove PVS test 2022-02-17 12:59:47 +01:00
replxx@3fd0e3c936 Bump replxx to fix wrong assertion 2022-04-07 00:56:10 +08:00
replxx-cmake Removed warning flags in contribs 2022-06-01 11:02:39 +02:00
rocksdb@e7c2b2f7bc Initial support for risc-v 2021-11-11 19:23:34 +01:00
rocksdb-cmake Respect library type for contrib libraries (#36399) 2022-04-19 18:05:42 +02:00
s2geometry@471fe9dc93 Update submodules for libc++13 compatibility 2021-12-13 16:49:22 +01:00
s2geometry-cmake Remove unused M_LIBRARY link 2022-06-01 11:02:39 +02:00
sentry-native@f431047ac8 Initial support for risc-v 2021-11-11 19:23:34 +01:00
sentry-native-cmake contrib/sentry-native: use hidden library 2022-01-21 10:11:23 +03:00
simdjson@8df32cea33 Fix 2021-05-28 11:46:08 +03:00
simdjson-cmake Remove unbundled simdjson support 2022-01-20 10:01:12 +03:00
snappy@fb057edfed Revert "Remove resursive submodules" 2022-06-02 09:41:12 +03:00
snappy-cmake Remove unbundled snappy support 2022-01-20 08:47:16 +03:00
sparsehash-c11@cf0bffaa45
sparsehash-c11-cmake Remove unbundled sparsehash support 2022-01-20 10:01:11 +03:00
sqlite-amalgamation@9818baa5d0 Add library as submodule 2021-07-07 20:06:35 +00:00
sqlite-cmake contrib/sqlite: add hidden library 2022-01-21 10:11:23 +03:00
sysroot@e9fb375d0a Fix clang-tidy-14 (part 1) 2022-04-19 17:44:27 +02:00
thrift@010ccf0a0c
thrift-cmake Cosmetics 2022-06-01 11:02:39 +02:00
unixodbc@a2cd5395e8 Update unixodbc to mitigate CVE-2018-7485 2022-04-04 23:34:33 +02:00
unixodbc-cmake Removed warning flags in contribs 2022-06-01 11:02:39 +02:00
wordnet-blast@1d16ac2803 Updated submodules to include new READMEs 2021-06-19 23:15:16 +00:00
wordnet-blast-cmake contrib/wordnet-blast: use hidden library 2022-01-21 10:11:23 +03:00
wyhash@991aa3dab6 wyhash 2022-04-21 02:26:37 +03:00
wyhash-cmake Update CMakeLists.txt 2022-05-09 08:48:51 -04:00
xz@869b9d1b4e
xz-cmake LZMA/xz cannot be disabled 2022-01-20 10:02:02 +03:00
yaml-cpp@0c86adac6d Update submodules 2021-05-28 00:18:30 +03:00
yaml-cpp-cmake contrib/yaml-cpp: use hidden library 2022-01-21 10:11:23 +03:00
zlib-ng@bffad6f6fe Re-add the submodule 2021-11-29 21:10:37 +03:00
zlib-ng-cmake contrib/zlib-ng: cleanup cmake rules 2022-01-21 10:11:23 +03:00
zstd@a488ba114e Fix 2021-05-28 11:46:08 +03:00
zstd-cmake Remove unbundled zstd support 2022-01-20 08:47:16 +03:00
CMakeLists.txt Remove eigen library 2022-05-24 10:24:50 +02:00