thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-15 10:52:30 +00:00
Code Issues Packages Projects Releases Wiki Activity
112,647 Commits 9,328 Branches 1,783 Tags 2.1 GiB
f7232ef537
Commit Graph

901 Commits

Author SHA1 Message Date
BoloniniD
147dfac11e Try using Corrosion 2022-09-12 23:05:41 +03:00
Meena Renganathan
51d6611b96 Committing the ClickHouse core changes and other libraries to support OpenSSL. BoringSSL is still set as default 2022-09-12 09:05:38 -07:00
BoloniniD
e8bcbcd016
Merge branch 'master' into BLAKE3 2022-09-09 11:48:31 +03:00
BoloniniD
d635c3f9be Disable BLAKE3 for aarch64-darwin 2022-09-06 21:22:40 +03:00
Azat Khuzhin
2d3fb36a62 Fix debug symbols (Add a quirk to force clang emit .debug_aranges with ThinLTO) 
Wrap a linker into a script that will add some settings (`-mllvm
-generate-arange-section`) in case of ThinLTO to emit `.debug_aranges`
symbols.

Dicussion in the LLVM can be found here [1].

  [1]: https://discourse.llvm.org/t/clang-does-not-produce-full-debug-aranges-section-with-thinlto/64898

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-09-02 23:22:25 +02:00
Alexander Tokmakov
69387acffa
Revert "Support for DWARF-5 in in house DWARF parser" 2022-08-29 14:25:53 +03:00
Alexey Milovidov
8448242785
Merge pull request #40716 from azat/fix-gdb-index 
Fix gdb index (makes gdb 17x faster)
 2022-08-29 00:13:58 +03:00
Alexey Milovidov
ae97e880d7
Merge pull request #40710 from azat/DWARF-5 
Support for DWARF-5 in in house DWARF parser
 2022-08-29 00:13:33 +03:00
Alexey Milovidov
ddff5e9145
Merge pull request #40712 from ClickHouse/ci-cmake-self-extractable 
cmake: Disable default ENABLE_CLICKHOUSE_SELF_EXTRACTING
 2022-08-28 17:12:19 +03:00
Azat Khuzhin
19178a8f18 Fix gdb index (makes gdb 17x faster) 
Right now LINKER_NAME contains version so it does not work:

    Aug 27 13:02:10 -- Using linker: /usr/bin/ld.lld-14

Without this patch:

    $ time gdb ~/ch/clickhouse/.bld-release/programs/clickhouse -batch

    real    1m20.564s
    user    1m18.622s
    sys     0m1.940s

    $ time lldb-13 clickhouse -batch
    real    0m9.353s
    user    1m31.035s
    sys     0m8.346s

With:

    $ time gdb clickhouse -batch
    real    0m4.769s
    user    0m3.969s
    sys     0m0.800s

    $ time lldb clickhouse -batch
    real    0m6.703s
    user    0m45.383s
    sys     0m4.934s

So by just adding 4.15MiB to the binary size, gdb start up time improved
17 times.

    $ objdump -j .gdb_index -h .bld-release/programs/clickhouse

    .bld-release/programs/clickhouse:     file format elf64-x86-64

    Sections:
    Idx Name          Size      VMA               LMA               File off  Algn
     40 .gdb_index    00426d74  0000000000000000  0000000000000000  7fb43443  2**0
                      CONTENTS, READONLY, DEBUGGING

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-08-28 15:01:34 +02:00
Azat Khuzhin
7f65075dcb Fix usage of SHA1 for BuildID 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-08-28 09:02:10 +02:00
Yakov Olkhovskiy
b5bb1be15a pass ENABLE_CLICKHOUSE_SELF_EXTRACTING to native build 2022-08-27 23:29:32 +00:00
Azat Khuzhin
444acb96c8 Do not pin DWARF-4 during build 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-08-27 22:28:15 +02:00
Alexey Milovidov
91ed3ebe9a Remove the remaining parts of "unbundled" build 2022-08-27 02:18:47 +02:00
Robert Schulze
cadfdc40ef
fix: CMake option ENABLE_CCACHE is not propagated to native build 
- fixes #40439
 2022-08-23 18:59:51 +00:00
Alexey Milovidov
a716d78264 Support build with clang-16 2022-08-13 06:06:01 +02:00
BoloniniD
29084d92d8 Fix CMake for ENABLE_RUST 2022-08-02 20:44:12 +03:00
BoloniniD
b161773f71 Merge branch 'master' of github.com:ClickHouse/ClickHouse into BLAKE3 2022-08-02 20:25:25 +03:00
Robert Schulze
d834d9e137
Minor: Better variable name output 2022-07-31 07:44:25 +00:00
Alexey Milovidov
36e6500e54
Merge branch 'master' into BLAKE3 2022-07-30 23:14:05 +03:00
Robert Schulze
199e254777
Merge remote-tracking branch 'origin/master' into no-split-binary 2022-07-28 15:54:22 +00:00
Yakov Olkhovskiy
da42ec7d28 some refactoring 2022-07-24 21:08:43 -04:00
Alexey Milovidov
071374b152 Remove SPLIT_BINARY 2022-07-24 01:15:54 +02:00
Yakov Olkhovskiy
e7438cc433
Merge branch 'master' into cmake-self-extracting-executable 2022-07-21 00:25:08 -04:00
Yakov Olkhovskiy
b766552481 add native build for cross-compilation 2022-07-20 23:09:05 -04:00
Azat Khuzhin
0e95dba3ef Fix make clean (due to crosscompile of llvm) 
Without ADDITIONAL_CLEAN_FILES it reports an error:

    Cleaning... ninja: error: remove(contrib/llvm/llvm/NATIVE): Directory not empty
    ninja: error: remove(/bld/contrib/llvm/llvm/NATIVE): Directory not empty
    0 files.

Note, that ADDITIONAL_CLEAN_FILES had been added since cmake 3.15.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-07-20 17:34:10 +03:00
Azat Khuzhin
21fa5e6ece Add check for CLICKHOUSE_SPLIT_BINARY=1 with USE_STATIC_LIBRARIES=1 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-07-18 11:42:49 +03:00
Azat Khuzhin
30018ce5d3 Fix running cmake with predefined cache (for development only) 
Right now cmake add the following options only if USE_STATIC_LIBRARIES
is OFF:
- SPLIT_SHARED_LIBRARIES
- CLICKHOUSE_SPLIT_BINARY

And this breaks the following usage:

    $ cmake ..
    $ cat > debug-build-cache.cmake
    set(USE_STATIC_LIBRARIES              OFF CACHE BOOL "")
    set(SPLIT_SHARED_LIBRARIES            ON  CACHE BOOL "")
    set(CLICKHOUSE_SPLIT_BINARY           ON  CACHE BOOL "")
    ^D
    $ cmake -C debug-build-cache.cmake ..
    CMake Error at CMakeLists.txt:83 (message):
      Defining SPLIT_SHARED_LIBRARIES=1 without USE_STATIC_LIBRARIES=0 has no
      effect.

Since with this initial cache we have the following:

- USE_STATIC_LIBRARIES=OFF (because it was already set)
- SPLIT_SHARED_LIBRARIES=ON (was not set before, so new value)
- CLICKHOUSE_SPLIT_BINARY (was not set before, also new value)

Yes this is not the common usage, but it seems that it is pretty easy to
avoid.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-07-17 12:18:24 +03:00
Alexey Milovidov
d8b1b44a29
Merge branch 'master' into align-branch-within-32b 2022-07-13 16:53:34 +03:00
Robert Schulze
1a7727a254
Prefix overridden add_executable() command with "clickhouse_" 
A simple HelloWorld program with zero includes except iostream triggers
a build of ca. 2000 source files. The reason is that ClickHouse's
top-level CMakeLists.txt overrides "add_executable()" to link all
binaries against "clickhouse_new_delete". This links against
"clickhouse_common_io", which in turn has lots of 3rd party library
dependencies ... Without linking "clickhouse_new_delete", the number of
compiled files for "HelloWorld" goes down to ca. 70.

As an example, the self-extracting-executable needs none of its current
dependencies but other programs may also benefit.

In order to restore access to the original "add_executable()", the
overriding version is now prefixed. There is precedence for a
"clickhouse_" prefix (as opposed to "ch_"), for example
"clickhouse_split_debug_symbols". In general prefixing makes sense also
because overriding CMake commands relies on undocumented behavior and is
considered not-so-great practice (*).

(*) https://crascit.com/2018/09/14/do-not-redefine-cmake-commands/
 2022-07-11 19:36:18 +02:00
Azat Khuzhin
e8f5cd3c68 Add separate option to omit symbols from heavy contrib 
Sometimes it is useful to build contrib with debug symbols for further
debugging.

With everything turned ON (i.e. debug build) I got 3.3GB vs 3.0GB w/o
this patch, 9% bloat, thoughts about this is this OK or not for you, if
not STRIP_DEBUG_SYMBOLS_HEAVY_CONTRIB can be OFF by default (regardless
of build type).

P.S. aws debug symbols adds just 1.7%.
v2: rename STRIP_HEAVY_DEBUG_SYMBOLS
v3: OMIT_HEAVY_DEBUG_SYMBOLS
v4: documentation had been removed
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-07-02 06:32:03 +03:00
Robert Schulze
bb358617e1
Better naming for stuff related to splitted debug symbols 
The previous name was slightly misleading, e.g. it is not about
"intalling stripped binaries" but about splitting debug symbols from the
binary.
 2022-06-30 23:41:27 +02:00
BoloniniD
6ddcec0906
Merge branch 'master' into BLAKE3 2022-06-28 16:53:06 +03:00
Robert Schulze
bc46cef63c
Minor follow-up 
- change ELF section name to ".clickhouse.hash" (lowercase seems
  standard)

- more expressive/concise integrity check messages at startup
 2022-06-14 08:52:13 +00:00
Robert Schulze
e25fbb9292
Log official/non-official build and integrated/separate debug symbols 
The goal is to find out why some of the binaries with official name
(BuilderDebRelease, BuilderBinRelease) produced by CI still contain no
hash section.

(also, strip and objcopy are mandatory tools and we don't need to check
for their existence at this point)
 2022-06-13 12:34:09 +00:00
BoloniniD
b05ee41d25 Merge branch 'master' of github.com:ClickHouse/ClickHouse into BLAKE3 2022-06-06 16:03:10 +03:00
Wangyang Guo
ea5e37abef Align branches within a 32B boundary for AMD64 arch to make benchmark more stable 2022-06-06 17:22:21 +08:00
Robert Schulze
5a1b873f7b
No need to checkout submodules/contribs recursively 
Also verified locally by building from a freshly cloned ClickHouse and
"flat" checkout of submodules without recursion
 2022-06-01 13:54:53 +02:00
BoloniniD
dd8aefdf1e Merge branch 'master' of github.com:ClickHouse/ClickHouse into BLAKE3 2022-06-01 11:46:55 +03:00
Azat Khuzhin
282f037659 Set compiler flags for ASM same as for C/CXX 
This should fix DWARF version for ASM sources (from #34754):

    $ llvm-dwarfdump -r 1 clickhouse | grep Compile -A10 | grep -A10 'version = 0x0005'
    ...
    --
    0x1de11022: Compile Unit: length = 0x000000e7, format = DWARF32, version = 0x0005, unit_type = DW_UT_compile, abbr_offset = 0x551dc0, addr_size = 0x08 (next unit at 0x1de1110d)

    0x1de1102e: DW_TAG_compile_unit
                  DW_AT_stmt_list   (0x057e3442)
                  DW_AT_low_pc      (0x00000000101cde0c)
                  DW_AT_high_pc     (0x00000000101cde5e)
                  DW_AT_name        ("/ClickHouse/contrib/libunwind/src/UnwindRegistersSave.S")
                  DW_AT_comp_dir    ("/fasttest-workspace/build")
                  DW_AT_producer    ("Ubuntu clang version 14.0.1-++20220426083040+0e27d08cdeb3-1~exp1~20220426083051.129")
                  DW_AT_language    (DW_LANG_Mips_Assembler)
    ...

Follow-up for: #34777 (cc @alexey-milovidov)
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-05-24 15:18:35 +03:00
Robert Schulze
95d28973aa
Cosmetics 2022-05-15 16:07:10 +02:00
Robert Schulze
1ff7d245ed
Add LINKER: prefix to link options 
CMake will translate this to a compiler-specific flag to pass the flags
to the linker, e.g. for Clang: "-Xlinker", for GCC: "-Wl,". It worked so
far because Clang supports -Wl in the meantime too but we should prefer
the portable method.
 2022-05-15 09:48:54 +02:00
Robert Schulze
7282b74403
Remove obsolete policies 
As far as I see these were necessary for super old Cmake versions and
when contrib libraries (snappy, re2, google test) were still unbundled.
 2022-05-15 09:48:54 +02:00
Robert Schulze
6a4e1e3fbd
Remove wrong CMAKE_MODULE_PATH 
The module path makes CMake find scripts in directories without
directory qualification, e.g.
    include (tools)
instead of
    include (cmake/tools.cmake)
The latter is a little bit longer but less ambiguous (imho). Therefore,
removing CMAKE_MODULE_PATH. Note that it pointed to a non-existing
directory anyways ...
 2022-05-15 09:48:54 +02:00
Robert Schulze
a07674be4d
Move WERROR option closer to its usage 2022-05-15 09:48:03 +02:00
Robert Schulze
9b07f5d2d5
Remove unused/undocumented PLATFORM_EXTRA_CXX_FLAG variable 2022-05-15 09:48:03 +02:00
Robert Schulze
3eb964a9f5
Simplify: Move warnings stuff into cmake/warnings.cmake 2022-05-15 09:48:03 +02:00
Robert Schulze
fc08b914ca
Minor: Move languages into PROJECT statement 2022-05-15 09:48:03 +02:00
Robert Schulze
1c127bbe09
Make color output a CMake option and fix setting for makefile generator 
- Variable CMAKE_BUILD_COLOR_MAKEFILE has no effect (perhaps it had in
  ancient CMake), the actual variable is called CMAKE_COLOR_MAKEFILE
 2022-05-15 09:48:02 +02:00
Robert Schulze
eb9f11da3d
Replace variable ARCH_ARM by ARCH_AARCH64 
Both are equivalent and removing the duplication was a leftover of PR #36614
 2022-05-15 09:48:02 +02:00
Robert Schulze
718a173ce9
Merge remote-tracking branch 'origin/master' into clang-tidy-cleanup 
Conflicts:
- cmake/clang_tidy.cmake
 2022-04-26 09:42:47 +02:00
Robert Schulze
15e75e79e4
analysis.cmake --> clang_tidy.cmake 2022-04-26 09:41:56 +02:00
Robert Schulze
730ce1806c
Simplify using RECONFIGURE_MESSAGE_LEVEL 
The behavior stays the same, but we are using the same logic that is
used elsewhere too.
 2022-04-25 12:56:57 +02:00
Robert Schulze
2b3340861b
Remove support for ancient CMake 
The first line in CMakeLists.txt says the minimum CMake version is
3.14. Therefore removing special code for earlier CMake versions.
 2022-04-25 12:48:24 +02:00
alesapin
00f683fcf9
Update CMakeLists.txt 
Co-authored-by: Mikhail f. Shiryaev <felixoid@clickhouse.com>
 2022-04-22 11:30:09 +02:00
BoloniniD
a353210e94 Merge branch 'master' of github.com:ClickHouse/ClickHouse into BLAKE3 2022-04-21 18:44:42 +03:00
alesapin
ba81816dc1 Better cmake 2022-04-20 12:11:55 +02:00
Alexey Milovidov
fec7a74f68 Add an option for build profiling 2022-04-19 17:44:27 +02:00
Alexey Milovidov
fa46e0d22f Add an option for build profiling 2022-04-16 02:54:21 +02:00
BoloniniD
30712ead45
Merge branch 'master' into BLAKE3 2022-04-13 00:45:44 +03:00
BoloniniD
c03b3c0aa4 Turn off BLAKE3 for aarch64 darwin 2022-04-13 00:26:36 +03:00
Alexey Milovidov
4691a428cb
Merge pull request #35130 from DevTeamBK/curl_upgrade 
curl package upgraded to 7.81.0
 2022-04-08 16:37:28 +03:00
Meena Renganathan
0d0b1ef7dc Fix to restrict the curl, azure and sentry build in MacOS 2022-04-06 06:28:35 -07:00
Denis Glazachev
e2044b6504 Set ENABLE_BUILD_PATH_MAPPING to OFF by default, if CMAKE_BUILD_TYPE is set to Debug 2022-04-06 16:17:22 +04:00
alesapin
10db099d2f One more try to resurrect build hash 2022-03-30 23:27:46 +02:00
BoloniniD
2d2420c8a9 Merge branch 'master' of github.com:ClickHouse/ClickHouse into BLAKE3 2022-03-29 11:00:50 +03:00
alesapin
b838a7dcb0 Remove outdated links from CI 2022-03-28 15:53:22 +02:00
Mikhail f. Shiryaev
572b15a63c
Fix paths and some cmake typos 2022-03-24 23:11:19 +01:00
Mikhail f. Shiryaev
fa2a9bb9aa
Separate BUILD_STRIPPED_BINARIES_PREFIX to option and parameter 2022-03-22 11:10:02 +01:00
BoloniniD
253afd032f Merge branch 'master' of github.com:ClickHouse/ClickHouse into BLAKE3 2022-03-13 21:18:46 +03:00
alesapin
96c0e9fddf Better cmake 2022-03-11 15:47:07 +01:00
Mikhail f. Shiryaev
91f1857211
Merge pull request #35196 from ClickHouse/add_strip_cmake_cmd 
Add ability to strip binaries in cmake
 2022-03-11 12:37:06 +01:00
alesapin
e53578910b Add ability to strip binaries in cmake 2022-03-10 22:23:28 +01:00
小路
15e4978a3f
Fix 'Breakpoint not hit .. ' in CLion (#35179) 2022-03-10 20:35:04 +01:00
BoloniniD
8474babe88 Merge branch 'master' of github.com:ClickHouse/ClickHouse into BLAKE3 2022-03-02 18:27:53 +03:00
Maksim Kita
a3e6552fa5
Merge pull request #34753 from ClickHouse/check-submodules 
Submodules are mandatory
 2022-02-21 13:16:35 +01:00
Alexey Milovidov
b1b8a84c11 Generate DWARF version 4 max 2022-02-20 21:47:27 +01:00
Alexey Milovidov
e1a2e376a3 Submodules are mandatory 2022-02-19 17:44:24 +01:00
BoloniniD
c8bb6fb38a Merge branch 'master' of github.com:ClickHouse/ClickHouse into BLAKE3 2022-02-13 12:47:18 +03:00
alexey-milovidov
e194bf3377
Merge pull request #34317 from ucasfl/lowcard 
use LowCardinality for _file and _path virtual columns in StorageFile
 2022-02-04 21:39:44 +03:00
feng lv
7bf2243431 use LowCardinality for _file and _path virtual columns in StorageFile 
fix
 2022-02-04 14:15:56 +00:00
BoloniniD
23e29e22a0 Merge branch 'master' of github.com:ClickHouse/ClickHouse into BLAKE3 2022-02-03 19:32:14 +03:00
Azat Khuzhin
b90cf4e7ff Move cmake/find/ccache.cmake -> cmake/ccache.cmake 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-01-31 16:01:19 +03:00
BoloniniD
b04ed43ebd Merge branch 'master' of github.com:ClickHouse/ClickHouse into BLAKE3 2022-01-30 22:58:04 +03:00
Maksim Kita
b618febdec
Merge pull request #33970 from amosbird/fixwerror 
Fix missing -Werror flag
 2022-01-28 16:53:09 +01:00
alexey-milovidov
4060290eb6
Merge pull request #33925 from benbiti/USE_DEBUG_HELPERS-OFF 
when not DEBUG set USE_DEBUG_HELPERS OFF
 2022-01-27 22:59:55 +03:00
Amos Bird
6adb3aa49a
Fix missing -Werror 2022-01-28 02:07:43 +08:00
BoloniniD
6f078c164d Merge branch 'master' of github.com:ClickHouse/ClickHouse into BLAKE3 2022-01-26 20:11:27 +03:00
Maksim Kita
c1db61b4ed
Merge pull request #33946 from azat/build-cleanup 
Remove MAKE_STATIC_LIBRARIES (in favor of USE_STATIC_LIBRARIES)
 2022-01-24 23:42:04 +01:00
Azat Khuzhin
4a0facd341 Remove MAKE_STATIC_LIBRARIES (in favor of USE_STATIC_LIBRARIES) 
There is no more MAKE_*, so remove this alias.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-01-24 17:28:33 +03:00
Azat Khuzhin
37dbff7587 Revert glibc compatibility (via .symver) in favor of hermetic build (bundled libc) 
This patch reverts glibc compatibility (via .symver) #29594,
in favor of hermetic build (that provides bundled libc version) #30011

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-01-24 16:03:56 +03:00
benbiti
a215600c6c set it do noting 2022-01-24 14:41:34 +08:00
Ben
35bacb1159 when not DEBUG set USE_DEBUG_HELPERS OFF 
when not DEBUG set USE_DEBUG_HELPERS OFF
 2022-01-24 14:29:07 +08:00
Azat Khuzhin
1a2f7b4281 cmake/print_include_directories.cmake: remove (no copy_headers.sh anymore) 
Introduced in eb552f2943, copy_headers.sh
was relied on it.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-01-21 10:11:23 +03:00
Azat Khuzhin
6dcb09f9e1 Cleanup llvm 2022-01-20 10:02:01 +03:00
Azat Khuzhin
323fe70fcd Cleanup NLP support 2022-01-20 10:02:01 +03:00
Azat Khuzhin
a75b748fee Remove unbundled mysql support 2022-01-20 10:02:01 +03:00
Azat Khuzhin
cad885ab1c Remove unbundled aws-s3 support 2022-01-20 10:02:00 +03:00
Azat Khuzhin
6a721baa59 Remove unbundled datasketches support 2022-01-20 10:02:00 +03:00
Azat Khuzhin
3147bbab51 Cleanup sentry support 2022-01-20 10:02:00 +03:00