Commit Graph

936 Commits

Author SHA1 Message Date
Robert Schulze
cbcff6ef08
Merge remote-tracking branch 'rschu1ze/master' into clang-17 2023-09-20 09:15:58 +00:00
Robert Schulze
d31a43a379
Remove Woboq 2023-09-18 17:44:04 +00:00
Yakov Olkhovskiy
0847889db6 libFuzzer infrastructure 2023-09-17 16:36:49 +00:00
Yakov Olkhovskiy
46fa7dbb80 add libFuzzer build on 'libFuzzer' label, build produces artifacts 2023-09-17 16:28:50 +00:00
Yakov Olkhovskiy
361b21b416 fix fuzzers, cmake refactor, add target fuzzers 2023-09-01 14:20:50 +00:00
Yakov Olkhovskiy
8ba08ed9e7 fix 2023-08-28 02:08:23 +00:00
Yakov Olkhovskiy
017eab4a07 skip adding execute_query_fuzzer to the fuzzers bundle - too large - linker fails 2023-08-28 02:05:43 +00:00
Yakov Olkhovskiy
db1a291e1e cleanup 2023-08-27 23:19:24 +00:00
Yakov Olkhovskiy
264877b2a5 fix fuzzers, add bundle build target fuzzers 2023-08-27 19:47:11 +00:00
Yakov Olkhovskiy
bb6189aaeb debug 2023-08-26 03:42:16 +00:00
Yakov Olkhovskiy
415a993c91 fix fuzzers build 2023-08-24 23:22:39 +00:00
Alexey Milovidov
5561e3e198 Remove garbage and speed up Debug and Tidy builds 2023-08-09 01:44:39 +02:00
Alexey Milovidov
43cf31a98e
Merge branch 'master' into no-export-dynamic 2023-07-23 07:26:33 +03:00
Azat Khuzhin
a10aa9ad50 Force libunwind usage (removes gcc_eh support)
libunwind is reentrant and signal safe, and works faster then then
gcc_eh (plus it has some custom patches for problems that have been
found during it's usage in ClickHouse).

gcc_eh may be missing in the system (if gcc was not installed), and
even if it exists clickhouse uses -nodefaultlibs, so some care should be
made to make it work.

Also this library is tiny and there shouln't be any problem to require
it always (there is already tendency to require some contrib libraries,
i.e. poco).

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-07-08 20:55:50 +02:00
Robert Schulze
5a6957d95e
Disable ThinLTO on non-Linux
Cross-compiling on Linux for Mac failed with CMake parameters
-DCMAKE_BUILD_TYPE=None -DENABLE_CLICKHOUSE_SELF_EXTRACTING=1
-DENABLE_TESTS=0 (see below). This happened e.g. in #51243.

The problem was that ThinLTO enabled/disabled depends on ENABLE_TESTS
(see the top-level CMakeLists.txt). If ENABLE_TESTS=0 then ThinLTO is
activated. On Linux, building/linking works with or without ThinLTO but
on Mac building/linking the self-extracting compressor binary doesn’t
work if ThinLTO is on. This is quite weird, as a workaround restrict
ThinLTO to Linux.

-------

[185/187] Linking CXX static library base/glibc-compatibility/libglibc-compatibility.a
[186/187] Linking CXX static library contrib/zstd-cmake/lib_zstd.a
[187/187] Linking CXX executable utils/self-extracting-executable/pre_compressor
-- Configuring done
-- Generating done
-- Build files have been written to: /home/ubuntu/repo/ch4/build
[0/2] Re-checking globbed directories...
[108/108] Linking CXX executable utils/self-extracting-executable/pre_compressor
FAILED: utils/self-extracting-executable/pre_compressor
: && /usr/bin/clang++-16 --target=x86_64-apple-darwin -std=c++20 -fdiagnostics-color=always -Xclang -fuse-ctor-homing -Wno-enum-constexpr-conversion -fsized-deallocation  -gdwarf-aranges -pipe -mssse3 -msse4.1 -msse4.2 -mpclmul -mpopcnt -fasynchronous-unwind-tables -ffile-prefix-map=/home/ubuntu/repo/ch4=. -falign-functions=32 -mbranches-within-32B-boundaries -stdlib=libc++ -fdiagnostics-absolute-paths -fstrict
-vtable-pointers -Wall -Wextra -Wframe-larger-than=65536 -Weverything -Wpedantic -Wno-zero-length-array -Wno-c++98-compat-pedantic -Wno-c++98-compat -Wno-c++20-compat -Wno-sign-conversion -Wno-implicit-int-conversion -Wno-implicit-int-float-conversion -Wno-ctad-maybe-unsupported -Wno-disabled-macro-expansion -Wno-documentation-unknown-command -Wno-double-promotion -Wno-exit-time-destructors -Wno-float-equal -Wn
o-global-constructors -Wno-missing-prototypes -Wno-missing-variable-declarations -Wno-padded -Wno-switch-enum -Wno-undefined-func-template -Wno-unused-template -Wno-vla -Wno-weak-template-vtables -Wno-weak-vtables -Wno-thread-safety-negative -Wno-enum-constexpr-conversion -Wno-unsafe-buffer-usage -O2 -g -DNDEBUG -O3 -g -gdwarf-4  -flto=thin -fwhole-program-vtables -isysroot /home/ubuntu/repo/ch4/cmake/darwin/..
/toolchain/darwin-x86_64 -mmacosx-version-min=10.15 -Wl,-headerpad_max_install_names --ld-path=/home/ubuntu/cctools/bin/x86_64-apple-darwin-ld -rdynamic -Wl,-U,_inside_main  -flto=thin -fwhole-program-vtables utils/self-extracting-executable/CMakeFiles/pre_compressor.dir/compressor.cpp.o -o utils/self-extracting-executable/pre_compressor  contrib/zstd-cmake/lib_zstd.a  contrib/libcxx-cmake/libcxx.a  contrib/lib
cxxabi-cmake/libcxxabi.a  -nodefaultlibs  -lc -lm -lpthread -ldl && :
clang: warning: argument unused during compilation: '-stdlib=libc++' [-Wunused-command-line-argument]
ld: warning: ignoring file utils/self-extracting-executable/CMakeFiles/pre_compressor.dir/compressor.cpp.o, building for macOS-x86_64 but attempting to link with file built for unknown-unsupported file format ( 0xDE 0xC0 0x17 0x0B 0x00 0x00 0x00 0x00 0x14 0x00 0x00 0x00 0x88 0x3E 0x03 0x00 )
ld: warning: ignoring file contrib/zstd-cmake/lib_zstd.a, building for macOS-x86_64 but attempting to link with file built for macOS-x86_64
ld: warning: ignoring file contrib/libcxxabi-cmake/libcxxabi.a, building for macOS-x86_64 but attempting to link with file built for unknown-unsupported file format ( 0x21 0x3C 0x61 0x72 0x63 0x68 0x3E 0x0A 0x23 0x31 0x2F 0x31 0x32 0x20 0x20 0x20 )
ld: warning: ignoring file contrib/libcxx-cmake/libcxx.a, building for macOS-x86_64 but attempting to link with file built for unknown-unsupported file format ( 0x21 0x3C 0x61 0x72 0x63 0x68 0x3E 0x0A 0x23 0x31 0x2F 0x31 0x32 0x20 0x20 0x20 )
Undefined symbols for architecture x86_64:
  "_main", referenced from:
     implicit entry/start for main executable
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
ninja: build stopped: subcommand failed.
2023-07-06 13:07:02 +00:00
Alexey Milovidov
fef63351bb
Merge branch 'master' into no-export-dynamic 2023-06-11 19:56:00 +03:00
Alexey Milovidov
70c83f5133
Merge pull request #49991 from amosbird/clickhouse_as_library
Use PROJECT_*_DIR instead of CMAKE_*_DIR.
2023-05-19 12:37:18 +03:00
Azat Khuzhin
0f7a310a67 Fix woboq codebrowser build with -Wno-poison-system-directories
woboq codebrowser uses clang tooling, which adds clang system includes
(in Linux::AddClangSystemIncludeArgs()), because none of (-nostdinc,
-nobuiltininc) is set.

And later it will complain with -Wpoison-system-directories for added by
itself includes in InitHeaderSearch::AddUnmappedPath(), because they are
starts from one of the following:
- /usr/include
- /usr/local/include

The interesting thing here is that it got broken only after upgrading to
llvm 16 (in #49678), and the reason for this is that clang 15 build has
system includes that does not trigger the warning -
"/usr/lib/clang/15.0.7/include", while clang 16 has
"/usr/include/clang/16.0.4/include"

So let's simply disable this warning, but only for woboq.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-05-18 18:26:05 +02:00
Azat Khuzhin
73661c3a46 Move tunnings for woboq codebrowser to cmake out from build.sh
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-05-18 18:18:30 +02:00
Amos Bird
6b4dcbd3ed
Use PROJECT_*_DIR instead of CMAKE_*_DIR. 2023-05-18 23:23:39 +08:00
Azat Khuzhin
a0cc5b7e47 Merge remote-tracking branch 'u/master' into build/llvm-16 2023-05-10 16:28:36 +02:00
Azat Khuzhin
6e9985a94d Remove removed in clang 16 -fexperimental-new-pass-manager
"The -fexperimental-new-pass-manager and -fno-legacy-pass-manager flags
have been removed. These flags have been silently ignored since Clang
15."

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-05-09 12:22:14 +02:00
Han Fei
64b67b9a37 Merge branch 'master' into hanfei/fiu 2023-05-09 01:14:30 +02:00
robot-clickhouse-ci-2
40922d197f
Merge pull request #49600 from azat/build/rust-lto
RFC: Improve performance of BLAKE3 by 11% by enabling LTO for Rust
2023-05-07 06:14:57 +02:00
Alexey Milovidov
1c007d8bd6 Merge branch 'master' into no-export-dynamic 2023-05-07 04:22:03 +02:00
Alexey Milovidov
d3c3d8b8e4 Remove export of dynamic symbols 2023-05-06 23:52:16 +02:00
Azat Khuzhin
1107988a82 Improve performance of BLAKE3 by 11% by enabling LTO for Rust
LTO in Rust produces multiple definition of `rust_eh_personality' (and
few others), and to overcome this --allow-multiple-definition has been
added.

Query for benchmark:

    SELECT ignore(BLAKE3(materialize('Lorem ipsum dolor sit amet, consectetur adipiscing elit'))) FROM numbers(1000000000) FORMAT `Null`

  upstream           : Elapsed: 2.494 sec. Processed 31.13 million rows, 249.08 MB (12.48 million rows/s., 99.86 MB/s.)
  upstream + rust lto: Elapsed: 13.56 sec. Processed 191.9 million rows, 1.5400 GB (14.15 million rows/s., 113.22 MB/s.)
  llvm BLAKE3        : Elapsed: 3.053 sec. Processed 43.24 million rows, 345.88 MB (14.16 million rows/s., 113.28 MB/s.)

Note, I thought about simply replacing it with BLAKE3 from LLVM, but:
- this will not solve LTO issues for Rust (and in future more libraries
  could be added)
- it makes integrating_rust_libraries.md useless (and there is even blog
  post)

So instead I've decided to add this quirk (--allow-multiple-definition)
to fix builds.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-05-06 22:28:56 +02:00
Alexey Milovidov
355d5af786 Remove some code 2023-05-06 18:33:45 +02:00
Han Fei
d608ba41e7 try to fix 2023-05-02 00:05:36 +02:00
Robert Schulze
550e430c28
A better way of excluding ISA-L on non-x86
Follow-up to #49256. More 'modern', i.e. uses a CMake TARGET exists
check instead of an internal variable.
2023-05-01 20:42:57 +00:00
Jordi
be702ee584 Disable ISA-L on aarch64 architectures 2023-04-27 12:15:37 +02:00
alesapin
0fcf19efd3 Missed brackets 2023-04-25 15:47:47 +02:00
alesapin
bb21461145 Try to fix with pie 2023-04-25 15:45:23 +02:00
Han Fei
fc101a33cd
Update CMakeLists.txt
Co-authored-by: Alexey Milovidov <milovidov@clickhouse.com>
2023-04-16 16:01:41 +02:00
Han Fei
862c7b43af forbid gwpsan in debug mode to rescue stress tests 2023-04-15 12:54:54 +02:00
Robert Schulze
609b892879
Fold several Clang-related configurations under one if 2023-04-11 16:27:32 +00:00
Robert Schulze
1477887259
Initial removal of logic under COMPILER_GCC 2023-04-11 16:27:32 +00:00
Mikhail f. Shiryaev
7ff582b963
Deprecate ENABLE_CCACHE and replace it by COMPILER_CACHE 2023-03-29 23:33:31 +02:00
Robert Schulze
1845bfc644
Revert change to ld64.lld linker on Mac
This reverts PRs #42470, #47673 and #47744. The problem was that after
the switch to ld64.lld, server binaries build in Debug mode no longer
came up, see the discussion after (*). My attempt to fix it with
`-no_compact_unwind` didn't help.

I also tried
- `-keep_dwarf_unwind`,
- `-unwindlib` (to use the OS unwinder),
- the unwinder in contrib (CMake variable EXCEPTION_HANDLING_LIBRARY)
but w/o success. Just tons of wasted time.

Rolling back to lld as linker on Mac. This will bring back many linker
warnings (#42282) but I rather accept that (temporarily, maybe someone
can figure out how to fix them) than have a broken Debug binary.

(*) https://github.com/ClickHouse/ClickHouse/pull/42470#issuecomment-1312344068
2023-03-27 15:26:54 +02:00
Robert Schulze
5d5308e060
CMake cleanup: Remove configuration of CMAKE_SHARED_LINKER_FLAGS
Follow-up to #44828
2023-03-26 17:59:39 +00:00
Robert Schulze
9b3e57070f
Fix broken cross-compiled macos builds 2023-03-20 08:33:33 +00:00
Robert Schulze
1499b77592
Restore native macos build 2023-03-19 19:59:22 +01:00
Han Fei
722e6bb2e8
Merge pull request #47680 from ClickHouse/rs/uncruftify-build
CMake: Remove further cruft from build
2023-03-17 22:58:46 +01:00
Robert Schulze
23281c7348
Merge pull request #47669 from ClickHouse/rs/more-gold-removal
Follow-up to #47660: Further removal of gold linker support
2023-03-17 14:58:23 +01:00
Robert Schulze
f72a337074
Remove cruft from build
No need to check compiler flags, clang >= 15 supports all of them.
2023-03-17 13:44:04 +00:00
Robert Schulze
f8980c582e
CMake: More removal of gold linker (follow-up to #47660)
+ fix a linker warning
2023-03-17 11:01:46 +00:00
Robert Schulze
22d496e8e6
CMake: Build ClickHouse w/o GNU extensions 2023-03-16 15:02:52 +00:00
Robert Schulze
348a40aa5d
Compile with C++23 2023-03-10 19:11:58 +00:00
Amos Bird
7b4f489837
Update flags 2023-02-28 22:45:33 +08:00
bkuschel
55e9322612 Fixes for OpenSSL and s390x 2023-02-17 16:20:21 -05:00