Commit Graph

1086 Commits

Author SHA1 Message Date
Azat Khuzhin
569b4bb631 Add ability to automatically comment SQL queries in clickhouse-client/local
This is like Alt-# in readline, it is useful when you need to look
something else, and need to save current query/command somewhere, and
commented lin the history is a good option.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-09-12 21:57:21 +02:00
Alexey Milovidov
2aedd41023
Remove strange code (#40195)
* Remove strange code

* Even more code removal

* Fix style

* Remove even more code

* Simplify code by making it slower

* Attempt to do something

* Attempt to do something

* Well do something with this horrible trash

* Add a test
2022-09-12 16:29:23 +02:00
Alexey Milovidov
33635e015d Fix style 2022-09-11 06:11:02 +02:00
Alexey Milovidov
fa62c7e982 Fix half of trash 2022-09-10 04:08:16 +02:00
Ilya Golshtein
74561b40ae stylecheck renewed 2022-09-02 17:42:18 +03:00
Antonio Andelic
e64436fef3 Fix typos with new codespell 2022-09-02 08:54:48 +00:00
Robert Schulze
bfa67ffc8c
Update base/base/itoa.h
Co-authored-by: Antonio Andelic <antonio2368@users.noreply.github.com>
2022-09-01 10:31:34 +02:00
Robert Schulze
d43f4e6317
fix: typo 2022-08-31 10:57:27 +00:00
Robert Schulze
cedf75ed5e
Enable clang-tidy for headers
clang-tidy now also checks code in header files. Because the analyzer
finds tons of issues, activate the check only for directory "base/" (see
file ".clang-tidy"). All other directories, in particular "src/" are
left to future work.

While many findings were fixed, some were not (and suppressed instead).
Reasons for this include: a) the file is 1:1 copypaste of a 3rd-party
lib (e.g. pcg_extras.h) and fixing stuff would make upgrades/fixes more
difficult b) a fix would have broken lots of using code
2022-08-31 10:48:15 +00:00
Alexander Tokmakov
44c6c85f75
Merge pull request #40756 from ClickHouse/libuv_update
Merging #39398
2022-08-30 14:38:50 +03:00
Suzy Wang
cad4eeb891 Correct __dup3() to dup3() 2022-08-28 18:03:07 -07:00
Alexey Milovidov
0252695c6e
Merge pull request #40224 from ClickHouse/alexey-milovidov-patch-4
Check what will be if I enable `concurrency-mt-unsafe` in `clang-tidy`
2022-08-22 23:12:01 +03:00
Robert Schulze
e8b3f56733
Limit suppression to a specific warning 2022-08-21 18:24:17 +00:00
Alexey Milovidov
74e1f4dc61 Fix clang-tidy 2022-08-20 17:09:20 +02:00
Suzy Wang
7a190f84c8 Remove weak_alias 2022-08-19 11:59:47 -07:00
Robert Schulze
77e64935e1
Reduce some usage of StringRef 2022-08-19 09:56:59 +00:00
Suzy Wang
1171ba9095 Add dup3() and inotify_init() from musl 1.2.3 2022-08-18 13:25:50 -07:00
Alexey Milovidov
dd96aee8d2
Update unaligned.h 2022-08-04 08:18:45 +03:00
Alexey Milovidov
6cc08afb72
Update unaligned.h 2022-08-04 01:58:08 +03:00
Alexey Milovidov
57e31cadfa
Update unaligned.h 2022-08-03 09:08:51 +03:00
HarryLeeIBM
cc73b53116 Fix Endian issue in SipHash for s390x 2022-07-29 13:21:50 -07:00
Robert Schulze
24c74da473
Merge pull request #39430 from azat/getauxval-fix
Fix LSan by fixing getauxval() (resubmit v4)
2022-07-25 10:43:09 +02:00
Azat Khuzhin
a4f9e1da5b Fix getauxval() by using procfs with fallback to environ
getauxval() from glibc-compatibility did not work always correctly:

- It does not work after setenv(), and this breaks vsyscalls,
  like sched_getcpu() [1] (and BaseDaemon.cpp always set TZ if timezone
  is defined, which is true for CI [2]).

  Also note, that fixing setenv() will not fix LSan,
  since the culprit is getauxval()

  [1]: https://bugzilla.redhat.com/show_bug.cgi?id=1163404
  [2]: ClickHouse#32928 (comment)

- Another think that is definitely broken is LSan (Leak Sanitizer), it
  relies on worked getauxval() but it does not work if __environ is not
  initialized yet (there is even a commit about this).

  And because of, at least, one leak had been introduced [3]:

    [3]: ClickHouse#33840

Fix this by using /proc/self/auxv with fallback to environ solution to
make it compatible with environment that does not allow reading from
auxv (or no procfs).

v2: add fallback to environ solution
v3: fix return value for __auxv_init_procfs()
(cherry picked from commit f187c3499a)
v4: more verbose message on errors, CI founds [1]:
    AUXV already has value (529267711)
    [1]: https://s3.amazonaws.com/clickhouse-test-reports/39103/2325f7e8442d1672ce5fb43b11039b6a8937e298/stress_test__memory__actions_.html
v5: break at AT_NULL
v6: ignore AT_IGNORE
v7: suppress TSan and remove superior check to avoid abort() in case of race
v8: proper suppressions (not inner function but itself)
Refs: #33957
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-07-25 01:22:04 +03:00
Robert Schulze
4333750985
Less usage of StringRef
... replaced by std::string_view, see #39262
2022-07-24 18:33:52 +00:00
Alexey Milovidov
72dda35108
Revert "[RFC] Fix LSan by fixing getauxval()" 2022-07-21 04:34:04 +03:00
Robert Schulze
7146685f9d
Merge pull request #39299 from azat/fix-lsan-v3
[RFC] Fix LSan by fixing getauxval()
2022-07-19 14:27:41 +02:00
Azat Khuzhin
f187c3499a Fix getauxval() by using procfs with fallback to environ
getauxval() from glibc-compatibility did not work always correctly:

- It does not work after setenv(), and this breaks vsyscalls,
  like sched_getcpu() [1] (and BaseDaemon.cpp always set TZ if timezone
  is defined, which is true for CI [2]).

  Also note, that fixing setenv() will not fix LSan,
  since the culprit is getauxval()

  [1]: https://bugzilla.redhat.com/show_bug.cgi?id=1163404
  [2]: ClickHouse#32928 (comment)

- Another think that is definitely broken is LSan (Leak Sanitizer), it
  relies on worked getauxval() but it does not work if __environ is not
  initialized yet (there is even a commit about this).

  And because of, at least, one leak had been introduced [3]:

    [3]: ClickHouse#33840

Fix this by using /proc/self/auxv with fallback to environ solution to
make it compatible with environment that does not allow reading from
auxv (or no procfs).

v2: add fallback to environ solution
v3: fix return value for __auxv_init_procfs()
Refs: #33957
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-07-18 16:49:49 +03:00
alesapin
00e5bdb84a Fix build clang-13 2022-07-18 12:44:58 +02:00
Robert Schulze
87197f8f60
Enable -Wdeprecated-dynamic-exception-spec 2022-07-13 13:49:40 +00:00
Robert Schulze
1a7727a254
Prefix overridden add_executable() command with "clickhouse_"
A simple HelloWorld program with zero includes except iostream triggers
a build of ca. 2000 source files. The reason is that ClickHouse's
top-level CMakeLists.txt overrides "add_executable()" to link all
binaries against "clickhouse_new_delete". This links against
"clickhouse_common_io", which in turn has lots of 3rd party library
dependencies ... Without linking "clickhouse_new_delete", the number of
compiled files for "HelloWorld" goes down to ca. 70.

As an example, the self-extracting-executable needs none of its current
dependencies but other programs may also benefit.

In order to restore access to the original "add_executable()", the
overriding version is now prefixed. There is precedence for a
"clickhouse_" prefix (as opposed to "ch_"), for example
"clickhouse_split_debug_symbols". In general prefixing makes sense also
because overriding CMake commands relies on undocumented behavior and is
considered not-so-great practice (*).

(*) https://crascit.com/2018/09/14/do-not-redefine-cmake-commands/
2022-07-11 19:36:18 +02:00
Alexander Tokmakov
c45482d6c6 disable instrumentation of sanitizer death callback 2022-07-07 22:10:12 +02:00
Alexander Tokmakov
c3573532c0
Merge pull request #38487 from ClickHouse/add_some_tsa_annotations
Add some TSA annotations
2022-06-29 13:57:10 +03:00
Robert Schulze
bb441faa42
Merge pull request #38491 from ClickHouse/clang-tidy-in-branches
Fix some clang-tidy warnings in headers
2022-06-29 10:22:04 +02:00
Robert Schulze
f692ead6ad
Don't use std::unique_lock unless we have to
Replace where possible by std::lock_guard which is more light-weight.
2022-06-28 19:19:06 +00:00
Robert Schulze
30f136867f
Fix build 2022-06-28 16:52:17 +00:00
Alexander Tokmakov
4f54abf67f better names for macros 2022-06-28 16:27:18 +02:00
Robert Schulze
c22038d48b
More clang-tidy fixes 2022-06-28 11:50:05 +00:00
Robert Schulze
5e28580f11
Fix some clang-tidy warnings in headers 2022-06-27 23:49:07 +02:00
Alexander Tokmakov
f4883f1f7e add some TSA aanotations 2022-06-27 22:48:27 +02:00
Robert Schulze
55b39e709d
Merge remote-tracking branch 'origin/master' into clang-tsa 2022-06-20 16:39:32 +02:00
Robert Schulze
5a4f21c50f
Support for Clang Thread Safety Analysis (TSA)
- TSA is a static analyzer build by Google which finds race conditions
  and deadlocks at compile time.

- It works by associating a shared member variable with a
  synchronization primitive that protects it. The compiler can then
  check at each access if proper locking happened before. A good
  introduction are [0] and [1].

- TSA requires some help by the programmer via annotations. Luckily,
  LLVM's libcxx already has annotations for std::mutex, std::lock_guard,
  std::shared_mutex and std::scoped_lock. This commit enables them
  (--> contrib/libcxx-cmake/CMakeLists.txt).

- Further, this commit adds convenience macros for the low-level
  annotations for use in ClickHouse (--> base/defines.h). For
  demonstration, they are leveraged in a few places.

- As we compile with "-Wall -Wextra -Weverything", the required compiler
  flag "-Wthread-safety-analysis" was already enabled. Negative checks
  are an experimental feature of TSA and disabled
  (--> cmake/warnings.cmake). Compile times did not increase noticeably.

- TSA is used in a few places with simple locking. I tried TSA also
  where locking is more complex. The problem was usually that it is
  unclear which data is protected by which lock :-(. But there was
  definitely some weird code where locking looked broken. So there is
  some potential to find bugs.

*** Limitations of TSA besides the ones listed in [1]:

- The programmer needs to know which lock protects which piece of shared
  data. This is not always easy for large classes.

- Two synchronization primitives used in ClickHouse are not annotated in
  libcxx:
  (1) std::unique_lock: A releaseable lock handle often together with
      std::condition_variable, e.g. in solve producer-consumer problems.
  (2) std::recursive_mutex: A re-entrant mutex variant. Its usage can be
      considered a design flaw + typically it is slower than a standard
      mutex. In this commit, one std::recursive_mutex was converted to
      std::mutex and annotated with TSA.

- For free-standing functions (e.g. helper functions) which are passed
  shared data members, it can be tricky to specify the associated lock.
  This is because the annotations use the normal C++ rules for symbol
  resolution.

[0] https://clang.llvm.org/docs/ThreadSafetyAnalysis.html
[1] https://static.googleusercontent.com/media/research.google.com/en//pubs/archive/42958.pdf
2022-06-20 16:13:25 +02:00
alesapin
0ae0cc54aa
Merge pull request #38106 from ClickHouse/tsan_less_flaky_tests
Some fixes for tests with tsan
2022-06-18 14:22:43 +02:00
Alexander Tokmakov
9cac78b498 make tests with tsan less flaky 2022-06-15 19:54:46 +02:00
Igor Nikonov
de273b043d Decimal: noexcept move constructor/assignment operator 2022-06-13 13:42:54 +00:00
Robert Schulze
1a0b5f33b3
More consistent use of platform macros
cmake/target.cmake defines macros for the supported platforms, this
commit changes predefined system macros to our own macros.

__linux__ --> OS_LINUX
__APPLE__ --> OS_DARWIN
__FreeBSD__ --> OS_FREEBSD
2022-06-10 10:22:31 +02:00
Alexey Milovidov
b5f48a7d3f Merge branch 'master' of github.com:ClickHouse/ClickHouse into llvm-14 2022-06-01 22:09:58 +02:00
Robert Schulze
249fe561f4
Fix build with -DENABLE_LIBRARIES=0 / -DENABLE_REPLXX=0
Replxx: When disabled via -DENABLE_LIBRARIES=0 or -DENABLE_REPLXX (the
latter was undocumented) the build broke because replxx symbols were
used since [0] in header LineReader.h. This header should in theory
stay clean of replxx but doesn't for efficiency reasons.

This change makes compilation of replxx mandatory. As replxx is quite
small, I guess this is okay. (The alternative is to litter the code
with ifdefs for non-replxx and a replxx paths.)

[0] https://github.com/ClickHouse/ClickHouse/pull/33201
2022-06-01 10:02:28 +02:00
Alexey Milovidov
bcbd6b802f Fix clang-tidy-14 2022-05-31 04:19:08 +02:00
Alexey Milovidov
11788c8129 Fix clang-tidy-14 2022-05-29 02:28:46 +02:00
Alexander Tokmakov
6bc68c0cbc Merge branch 'master' into fixes_for_transactions 2022-05-23 18:49:21 +02:00