Commit Graph

956 Commits

Author SHA1 Message Date
Nikita Mikhaylov
50828dade2
Fix stacktraces on MacOS (#59690) 2024-02-09 14:53:03 +01:00
Alexey Milovidov
6dc3efb3f9 Fix build 2023-11-20 01:55:34 +01:00
Alexey Milovidov
b205d4919d Remove garbage 2023-11-20 01:24:28 +01:00
Alexey Milovidov
1abcb28624 Remove ctest 2023-11-16 23:32:17 +01:00
Alexey Milovidov
482d8ca9d8
Merge pull request #56102 from ClickHouse/coverage
Granular code coverage with introspection
2023-11-16 23:23:40 +01:00
Michael Kolupaev
bcf6a66aa2
Switch from DWARF 4 to 5 and see what happens (#56770)
* Switch from DWARF 4 to 5 and see what happens

* Update comment in Dwarf.h about which versions are supported
2023-11-16 09:58:00 -08:00
Alexey Milovidov
b4b3cb2291
Merge pull request #56786 from ClickHouse/gc-sections
Ask linker to remove garbage from external libraries
2023-11-15 19:57:34 +01:00
Alexey Milovidov
671a401ded Ask linker to remove garbage from external libraries 2023-11-15 12:26:14 +01:00
Alexey Milovidov
5d81080e31 Enable profiling for linker 2023-11-14 09:01:50 +01:00
Alexey Milovidov
3ef14f6098 Merge branch 'master' of github.com:ClickHouse/ClickHouse into coverage 2023-11-14 06:08:32 +01:00
Alexey Milovidov
d92e332923 Fix build 2023-11-13 19:33:56 +01:00
Alexey Milovidov
0d01438c48 Fix error 2023-11-11 00:44:46 +01:00
Alexey Milovidov
96f73139b6 Check for large translation units 2023-11-10 06:13:55 +01:00
Alexey Milovidov
cd94d02455 Merge branch 'master' into coverage 2023-11-04 21:49:59 +01:00
Nikita Mikhaylov
ac1476ef6e Better 2023-10-30 16:16:25 +00:00
Alexey Milovidov
8e6a7fdff0 Fix f-string 2023-10-29 23:43:36 +01:00
Alexey Milovidov
70e3dd808c Granular code coverage with introspection 2023-10-29 02:07:24 +01:00
Michael Kolupaev
ce7eca0615
DWARF input format (#55450)
* Add ReadBufferFromFileBase::isRegularLocalFile()

* DWARF input format

* Review comments

* Changed things around ENABLE_EMBEDDED_COMPILER build setting

* Added 'ranges' column

* no-msan no-ubsan
2023-10-16 17:00:07 -07:00
Alexey Milovidov
0c8a7fd555
Merge pull request #54917 from kitaisreal/cmake-profile-cpu-option
CMake added DISABLE_OMIT_FRAME_POINTER option
2023-09-26 16:45:43 +03:00
Maksim Kita
0255d355ec CMake added DISABLE_OMIT_FRAME_POINTER option 2023-09-26 11:48:28 +03:00
Robert Schulze
cbcff6ef08
Merge remote-tracking branch 'rschu1ze/master' into clang-17 2023-09-20 09:15:58 +00:00
Robert Schulze
d31a43a379
Remove Woboq 2023-09-18 17:44:04 +00:00
Yakov Olkhovskiy
0847889db6 libFuzzer infrastructure 2023-09-17 16:36:49 +00:00
Yakov Olkhovskiy
46fa7dbb80 add libFuzzer build on 'libFuzzer' label, build produces artifacts 2023-09-17 16:28:50 +00:00
Yakov Olkhovskiy
361b21b416 fix fuzzers, cmake refactor, add target fuzzers 2023-09-01 14:20:50 +00:00
Yakov Olkhovskiy
8ba08ed9e7 fix 2023-08-28 02:08:23 +00:00
Yakov Olkhovskiy
017eab4a07 skip adding execute_query_fuzzer to the fuzzers bundle - too large - linker fails 2023-08-28 02:05:43 +00:00
Yakov Olkhovskiy
db1a291e1e cleanup 2023-08-27 23:19:24 +00:00
Yakov Olkhovskiy
264877b2a5 fix fuzzers, add bundle build target fuzzers 2023-08-27 19:47:11 +00:00
Yakov Olkhovskiy
bb6189aaeb debug 2023-08-26 03:42:16 +00:00
Yakov Olkhovskiy
415a993c91 fix fuzzers build 2023-08-24 23:22:39 +00:00
Alexey Milovidov
5561e3e198 Remove garbage and speed up Debug and Tidy builds 2023-08-09 01:44:39 +02:00
Alexey Milovidov
43cf31a98e
Merge branch 'master' into no-export-dynamic 2023-07-23 07:26:33 +03:00
Azat Khuzhin
a10aa9ad50 Force libunwind usage (removes gcc_eh support)
libunwind is reentrant and signal safe, and works faster then then
gcc_eh (plus it has some custom patches for problems that have been
found during it's usage in ClickHouse).

gcc_eh may be missing in the system (if gcc was not installed), and
even if it exists clickhouse uses -nodefaultlibs, so some care should be
made to make it work.

Also this library is tiny and there shouln't be any problem to require
it always (there is already tendency to require some contrib libraries,
i.e. poco).

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-07-08 20:55:50 +02:00
Robert Schulze
5a6957d95e
Disable ThinLTO on non-Linux
Cross-compiling on Linux for Mac failed with CMake parameters
-DCMAKE_BUILD_TYPE=None -DENABLE_CLICKHOUSE_SELF_EXTRACTING=1
-DENABLE_TESTS=0 (see below). This happened e.g. in #51243.

The problem was that ThinLTO enabled/disabled depends on ENABLE_TESTS
(see the top-level CMakeLists.txt). If ENABLE_TESTS=0 then ThinLTO is
activated. On Linux, building/linking works with or without ThinLTO but
on Mac building/linking the self-extracting compressor binary doesn’t
work if ThinLTO is on. This is quite weird, as a workaround restrict
ThinLTO to Linux.

-------

[185/187] Linking CXX static library base/glibc-compatibility/libglibc-compatibility.a
[186/187] Linking CXX static library contrib/zstd-cmake/lib_zstd.a
[187/187] Linking CXX executable utils/self-extracting-executable/pre_compressor
-- Configuring done
-- Generating done
-- Build files have been written to: /home/ubuntu/repo/ch4/build
[0/2] Re-checking globbed directories...
[108/108] Linking CXX executable utils/self-extracting-executable/pre_compressor
FAILED: utils/self-extracting-executable/pre_compressor
: && /usr/bin/clang++-16 --target=x86_64-apple-darwin -std=c++20 -fdiagnostics-color=always -Xclang -fuse-ctor-homing -Wno-enum-constexpr-conversion -fsized-deallocation  -gdwarf-aranges -pipe -mssse3 -msse4.1 -msse4.2 -mpclmul -mpopcnt -fasynchronous-unwind-tables -ffile-prefix-map=/home/ubuntu/repo/ch4=. -falign-functions=32 -mbranches-within-32B-boundaries -stdlib=libc++ -fdiagnostics-absolute-paths -fstrict
-vtable-pointers -Wall -Wextra -Wframe-larger-than=65536 -Weverything -Wpedantic -Wno-zero-length-array -Wno-c++98-compat-pedantic -Wno-c++98-compat -Wno-c++20-compat -Wno-sign-conversion -Wno-implicit-int-conversion -Wno-implicit-int-float-conversion -Wno-ctad-maybe-unsupported -Wno-disabled-macro-expansion -Wno-documentation-unknown-command -Wno-double-promotion -Wno-exit-time-destructors -Wno-float-equal -Wn
o-global-constructors -Wno-missing-prototypes -Wno-missing-variable-declarations -Wno-padded -Wno-switch-enum -Wno-undefined-func-template -Wno-unused-template -Wno-vla -Wno-weak-template-vtables -Wno-weak-vtables -Wno-thread-safety-negative -Wno-enum-constexpr-conversion -Wno-unsafe-buffer-usage -O2 -g -DNDEBUG -O3 -g -gdwarf-4  -flto=thin -fwhole-program-vtables -isysroot /home/ubuntu/repo/ch4/cmake/darwin/..
/toolchain/darwin-x86_64 -mmacosx-version-min=10.15 -Wl,-headerpad_max_install_names --ld-path=/home/ubuntu/cctools/bin/x86_64-apple-darwin-ld -rdynamic -Wl,-U,_inside_main  -flto=thin -fwhole-program-vtables utils/self-extracting-executable/CMakeFiles/pre_compressor.dir/compressor.cpp.o -o utils/self-extracting-executable/pre_compressor  contrib/zstd-cmake/lib_zstd.a  contrib/libcxx-cmake/libcxx.a  contrib/lib
cxxabi-cmake/libcxxabi.a  -nodefaultlibs  -lc -lm -lpthread -ldl && :
clang: warning: argument unused during compilation: '-stdlib=libc++' [-Wunused-command-line-argument]
ld: warning: ignoring file utils/self-extracting-executable/CMakeFiles/pre_compressor.dir/compressor.cpp.o, building for macOS-x86_64 but attempting to link with file built for unknown-unsupported file format ( 0xDE 0xC0 0x17 0x0B 0x00 0x00 0x00 0x00 0x14 0x00 0x00 0x00 0x88 0x3E 0x03 0x00 )
ld: warning: ignoring file contrib/zstd-cmake/lib_zstd.a, building for macOS-x86_64 but attempting to link with file built for macOS-x86_64
ld: warning: ignoring file contrib/libcxxabi-cmake/libcxxabi.a, building for macOS-x86_64 but attempting to link with file built for unknown-unsupported file format ( 0x21 0x3C 0x61 0x72 0x63 0x68 0x3E 0x0A 0x23 0x31 0x2F 0x31 0x32 0x20 0x20 0x20 )
ld: warning: ignoring file contrib/libcxx-cmake/libcxx.a, building for macOS-x86_64 but attempting to link with file built for unknown-unsupported file format ( 0x21 0x3C 0x61 0x72 0x63 0x68 0x3E 0x0A 0x23 0x31 0x2F 0x31 0x32 0x20 0x20 0x20 )
Undefined symbols for architecture x86_64:
  "_main", referenced from:
     implicit entry/start for main executable
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
ninja: build stopped: subcommand failed.
2023-07-06 13:07:02 +00:00
Alexey Milovidov
fef63351bb
Merge branch 'master' into no-export-dynamic 2023-06-11 19:56:00 +03:00
Alexey Milovidov
70c83f5133
Merge pull request #49991 from amosbird/clickhouse_as_library
Use PROJECT_*_DIR instead of CMAKE_*_DIR.
2023-05-19 12:37:18 +03:00
Azat Khuzhin
0f7a310a67 Fix woboq codebrowser build with -Wno-poison-system-directories
woboq codebrowser uses clang tooling, which adds clang system includes
(in Linux::AddClangSystemIncludeArgs()), because none of (-nostdinc,
-nobuiltininc) is set.

And later it will complain with -Wpoison-system-directories for added by
itself includes in InitHeaderSearch::AddUnmappedPath(), because they are
starts from one of the following:
- /usr/include
- /usr/local/include

The interesting thing here is that it got broken only after upgrading to
llvm 16 (in #49678), and the reason for this is that clang 15 build has
system includes that does not trigger the warning -
"/usr/lib/clang/15.0.7/include", while clang 16 has
"/usr/include/clang/16.0.4/include"

So let's simply disable this warning, but only for woboq.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-05-18 18:26:05 +02:00
Azat Khuzhin
73661c3a46 Move tunnings for woboq codebrowser to cmake out from build.sh
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-05-18 18:18:30 +02:00
Amos Bird
6b4dcbd3ed
Use PROJECT_*_DIR instead of CMAKE_*_DIR. 2023-05-18 23:23:39 +08:00
Azat Khuzhin
a0cc5b7e47 Merge remote-tracking branch 'u/master' into build/llvm-16 2023-05-10 16:28:36 +02:00
Azat Khuzhin
6e9985a94d Remove removed in clang 16 -fexperimental-new-pass-manager
"The -fexperimental-new-pass-manager and -fno-legacy-pass-manager flags
have been removed. These flags have been silently ignored since Clang
15."

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-05-09 12:22:14 +02:00
Han Fei
64b67b9a37 Merge branch 'master' into hanfei/fiu 2023-05-09 01:14:30 +02:00
robot-clickhouse-ci-2
40922d197f
Merge pull request #49600 from azat/build/rust-lto
RFC: Improve performance of BLAKE3 by 11% by enabling LTO for Rust
2023-05-07 06:14:57 +02:00
Alexey Milovidov
1c007d8bd6 Merge branch 'master' into no-export-dynamic 2023-05-07 04:22:03 +02:00
Alexey Milovidov
d3c3d8b8e4 Remove export of dynamic symbols 2023-05-06 23:52:16 +02:00
Azat Khuzhin
1107988a82 Improve performance of BLAKE3 by 11% by enabling LTO for Rust
LTO in Rust produces multiple definition of `rust_eh_personality' (and
few others), and to overcome this --allow-multiple-definition has been
added.

Query for benchmark:

    SELECT ignore(BLAKE3(materialize('Lorem ipsum dolor sit amet, consectetur adipiscing elit'))) FROM numbers(1000000000) FORMAT `Null`

  upstream           : Elapsed: 2.494 sec. Processed 31.13 million rows, 249.08 MB (12.48 million rows/s., 99.86 MB/s.)
  upstream + rust lto: Elapsed: 13.56 sec. Processed 191.9 million rows, 1.5400 GB (14.15 million rows/s., 113.22 MB/s.)
  llvm BLAKE3        : Elapsed: 3.053 sec. Processed 43.24 million rows, 345.88 MB (14.16 million rows/s., 113.28 MB/s.)

Note, I thought about simply replacing it with BLAKE3 from LLVM, but:
- this will not solve LTO issues for Rust (and in future more libraries
  could be added)
- it makes integrating_rust_libraries.md useless (and there is even blog
  post)

So instead I've decided to add this quirk (--allow-multiple-definition)
to fix builds.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-05-06 22:28:56 +02:00
Alexey Milovidov
355d5af786 Remove some code 2023-05-06 18:33:45 +02:00
Han Fei
d608ba41e7 try to fix 2023-05-02 00:05:36 +02:00
Robert Schulze
550e430c28
A better way of excluding ISA-L on non-x86
Follow-up to #49256. More 'modern', i.e. uses a CMake TARGET exists
check instead of an internal variable.
2023-05-01 20:42:57 +00:00