Commit Graph

341 Commits

Author SHA1 Message Date
Alexey Milovidov
0f66d161d6 Merge branch 'master' of github.com:ClickHouse/ClickHouse into no-export-dynamic 2023-07-23 21:33:51 +02:00
Alexey Milovidov
43cf31a98e
Merge branch 'master' into no-export-dynamic 2023-07-23 07:26:33 +03:00
Azat Khuzhin
0b258dda4e Reproducible builds for Rust
From now on cargo will not download anything from the internet during
builds. This step had been moved for docker image builds (via cargo
vendor).

And now cargo inside docker.io/clickhouse/binary-builder will not use
any crates from the internet, so we don't need to add --offline for
cargo commands in cmake (corrosion_import_crate()).

Also the docker build command had been adjusted to allow following
symlinks inside build context, by using tar, this is required for Rust
packages.

Note, that to make proper Cargo.lock that could be vendored I did the
following:
- per-project locks had been removed (since there is no automatic way to
  sync the workspace Cargo.lock with per-project Cargo.lock, since cargo
  update/generate-lockfile will use only per-project Cargo.toml files
  apparently, -Z minimal-versions does not helps either)
- and to generate Cargo.lock with less changes I've pinned version in
  the Cargo.toml strictly, i.e. not 'foo = "0.1"' but 'foo = "=0.1"'
  then the Cargo.lock for workspace had been generated and afterwards
  I've reverted this part.

Plus I have to update the dependencies afterwards, since otherwise there
are conflicts with dependencies for std library. Non trivial.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-07-22 22:46:22 +02:00
Mikhail f. Shiryaev
0ba38b4dde
Merge branch 'master' into sources-with-subs 2023-07-17 14:04:02 +02:00
Azat Khuzhin
ec5e26a017 Pin rust nightly (to make it stable)
Because of using Rust nightly, and without #49601 the Rust toolchain is
very unstable, and can be frequently failed.

So let's ping particular version.

Also I've looked and it seems that Rust archives stores this archive
without any TTL, since there is even a version for 2015 year.

Follow-up for: #50541
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-07-06 21:17:50 +02:00
Alexey Milovidov
1eaa2e9d30
Merge branch 'master' into no-export-dynamic 2023-07-04 22:37:46 +03:00
Mikhail f. Shiryaev
6d2b5166b9
Update golang version in builder 2023-06-28 10:32:20 +02:00
Ilya Yatsishin
a9cb4d1b3d Better tar 2023-06-27 12:32:32 +02:00
Ilya Yatsishin
ed6bfd66fe Publish changes 2023-06-26 17:28:59 +02:00
Alexey Milovidov
fef63351bb
Merge branch 'master' into no-export-dynamic 2023-06-11 19:56:00 +03:00
Alexey Milovidov
5d26b0cf7c
Merge pull request #50550 from cmsxbc/misc-darwin-build
skip extracting darwin toolchain in builder when unncessary
2023-06-05 11:11:34 +03:00
Alexey Milovidov
47379ac039
Update build.sh 2023-06-05 03:58:42 +03:00
Azat Khuzhin
bf127f4e1e MSan support for Rust
Previously you have to unpoison memory from the Rust, however Rust does
supports MSan, so let's simply use it.

But for this we need nightly Rust and recompile standard library.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-06-04 23:04:55 +02:00
cmsxbc
4bb44c7c72
1. skip extract darwin toolchain in builder when uncessary
2. update MacOSX SDK version in toolchain readme to match in builder
2023-06-04 23:06:21 +08:00
Alexey Milovidov
8cd9fc4a2d Fix build 2023-05-06 23:52:16 +02:00
Mikhail f. Shiryaev
6d8da8f88b
Add sccache and ctcache S3 parameters, don't use 2023-03-29 23:33:27 +02:00
Mikhail f. Shiryaev
612d1d8876
Install sccache in binary-builder and fasttests 2023-03-29 23:33:24 +02:00
Alexey Milovidov
d0d6306102 Fix coverity 2023-02-21 00:46:27 +01:00
Mikhail f. Shiryaev
08ffb8f93d
Install only "programs" directory during build 2023-02-15 11:49:19 +01:00
Mikhail f. Shiryaev
4d8ff0c0fb
Update the golang version to 1.19.5 2023-02-13 13:19:45 +01:00
Mikhail f. Shiryaev
ebb41504c0
Move copy to the end for parallel image building 2023-02-12 14:55:16 +01:00
Mikhail f. Shiryaev
376b5d178f
Experiment with multiple build stages 2023-02-09 19:13:44 +01:00
Mikhail f. Shiryaev
1ef1eefa26
Format the RUN, update clang-tidy-cache 2023-02-09 19:10:48 +01:00
Mikhail f. Shiryaev
5bc3e703ef
Move the heaviest operation to the top 2023-02-09 19:10:06 +01:00
MeenaRenganathan22
4a0202a074 Committing the docker with the changes to install the ysam 2023-01-24 20:41:43 -05:00
Alexey Milovidov
e9f11a3e61 Fix Docker 2023-01-08 22:45:30 +01:00
Alexey Milovidov
4694dad87c Break CI 2023-01-08 07:59:31 +01:00
Alexey Milovidov
d07f0c9e3f Fix GitHub 2023-01-02 23:44:42 +01:00
Alexey Milovidov
ca117d5b94
Merge branch 'master' into remove-two-lines-of-code 2023-01-02 22:15:36 +03:00
Azat Khuzhin
d9fb47458f Exclude cargo shared libraries from the artifacts
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-01-02 12:36:14 +01:00
Alexey Milovidov
47ae8c5c79 Remove more lines 2023-01-02 02:06:11 +01:00
Alexey Milovidov
f7f5aaf35d What if I will remove two lines of code? 2023-01-02 01:20:34 +01:00
Azat Khuzhin
28737a503c Configure rustc compiler properly
Right now it works for host platforms because of gcc package, that
includes gcc-cross sysroot.

Use bundled sysroot from contrib instead.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-12-14 20:57:29 +01:00
Azat Khuzhin
67fa185611
Revert "Builtin skim" 2022-12-14 17:17:19 +03:00
Azat Khuzhin
8570f15688 Configure rustc compiler properly
Right now it works for host platforms because of gcc package, that
includes gcc-cross sysroot.

Use bundled sysroot from contrib instead.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-12-11 15:52:00 +01:00
Mikhail f. Shiryaev
a68dc6280d
Optimize binary-builder size 2022-11-25 12:17:51 +01:00
Mikhail f. Shiryaev
150ceb7d37
Use matus-chochlik/ctcache for clang-tidy 2022-11-03 11:53:11 +01:00
Azat Khuzhin
043c4163f4 Use nfpm packager for archlinux packages
Since nfpm v2.20.0 it can build packages for archlinux [1].

  [1]: https://github.com/goreleaser/nfpm/pull/543

Something like this:

    $ mkdir -p /src/packages/root
    $ cmake -DCMAKE_INSTALL_PREFIX=/usr /src
    $ DESTDIR=/src/packages/root ninja install
    $ cd /src/packages
    $ ./build --archlinux

    $ root@s3:/src/packages# ./build --archlinux
    Current version is 22.10.1.1
    Building archlinux package for clickhouse-client.yaml
    using archlinux packager...
    created package: /src/clickhouse-client-22.10.1.1-1-x86_64.pkg.tar.zst
    ...

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-10-18 18:33:48 +03:00
BoloniniD
3a3ca51980 Fixes after review 2022-10-02 20:52:05 +03:00
BoloniniD
55c79230b3 Merge branch 'master' of github.com:ClickHouse/ClickHouse into BLAKE3 2022-09-29 23:53:25 +03:00
BoloniniD
4f2915c8f7 Fix aarch64-darwin build 2022-09-19 21:52:57 +03:00
Mikhail f. Shiryaev
5d8b8a7db6
Add deb Source for packages 2022-09-19 20:42:02 +02:00
BoloniniD
2f771daa8c Fix Rust installation for Corrosion 2022-09-18 15:22:16 +03:00
BoloniniD
627dea333e Merge branch 'master' into BLAKE3 2022-09-17 22:14:30 +03:00
BoloniniD
452ef4435b Merge branch 'master' of github.com:ClickHouse/ClickHouse into BLAKE3 2022-09-16 20:05:56 +03:00
BoloniniD
0df426d329 Corrosion fixes + review fixes 2022-09-16 00:05:21 +03:00
BoloniniD
fe239e7ee9 Fix style + free err_msg before throwing exception 2022-09-12 18:46:54 +03:00
Azat Khuzhin
ced30a2072 Switch to llvm/clang 15
It had been released few hours ago, and I want to check how clang-15
generates DWARF-5

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-09-10 21:38:34 +02:00
BoloniniD
e8bcbcd016
Merge branch 'master' into BLAKE3 2022-09-09 11:48:31 +03:00
Azat Khuzhin
64f95710cd ci: add symlink for llvm-symbolizer (to make symbolizer work)
Simply installing llvm-symbolizer-$VER is not enough, since it does not
contain proper symblink, while LLVM is looking only for
"llvm-symbolizer" (without version) - [1]:

  [1]:
c444af1c20/compiler-rt/lib/sanitizer_common/sanitizer_symbolizer_posix_libcdep.cpp (L454)

Follow-up for: #40655
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-09-04 14:03:34 +02:00
Azat Khuzhin
ce88c7cb60 Add llvm-15 for llvm-symbolizer-15 for clang-15 (used for TSan)
Simply installing llvm-15 over TSAN_SYMBOLIZER_PATH had been prefered
(to keep everything in one place and as simple as possible), though
TSAN_SYMBOLIZER_PATH should also work.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-08-26 09:28:42 +02:00
Alexey Milovidov
5ce6c07005 Set linker for RISC-V 64 2022-08-14 05:57:55 +02:00
Yakov Olkhovskiy
78cc2a970d
copy self-extracting to output 2022-08-09 02:40:17 -04:00
Alexander Tokmakov
3bc583c22b
Revert "copy self-extracting to output" 2022-08-08 20:13:40 +03:00
Yakov Olkhovskiy
4c78f0fc7d
Merge branch 'master' into ci-self-extracting-s3-deployment 2022-08-05 08:14:48 -04:00
BoloniniD
48c91619b0
Merge branch 'master' into BLAKE3 2022-08-04 20:30:47 +03:00
Alexey Milovidov
f474eb957d
Merge pull request #39758 from ClickHouse/tsan_clang_15
Try clang-15 for build with tsan
2022-08-04 01:56:52 +03:00
Mikhail f. Shiryaev
8533769132
Use compression and cleanup with the recent version ccache 2022-08-03 14:49:07 +02:00
Mikhail f. Shiryaev
eeaf08525f
Use test-util as source for base-test, fasttest and package builder 2022-08-03 14:49:06 +02:00
Alexander Tokmakov
e292d830f5
Update Dockerfile 2022-08-02 15:37:02 +03:00
Alexander Tokmakov
82b50e79cf
Merge branch 'master' into tsan_clang_15 2022-08-02 13:00:55 +03:00
Alexander Tokmakov
31550436e4 try clang-15 for build with tsan 2022-07-31 14:38:13 +02:00
Alexey Milovidov
36e6500e54
Merge branch 'master' into BLAKE3 2022-07-30 23:14:05 +03:00
Azat Khuzhin
4b0594633b Build utils on CI when they was requested
clickhouse-bundle does not includes utils, and to avoid modifying each
utility it is better to build all, anyway this is not a common build.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-07-29 11:36:51 +03:00
Azat Khuzhin
2901ddc217 Remove --verbose=1 for cmake on configuration stage
--verbose is supported only for make/install, while 3.16 is OK (ubuntu
20.04), newer cmake (3.23) does not, and simply fails:

    # cmake --debug-trycompile --verbose=1 /foo
    debug trycompile on
    CMake Error: Unknown argument --verbose=1
    CMake Error: Run 'cmake --help' for all supported options.

And this is how it works with 3.16:

    # cmake --debug-trycompile --verbose=1 /foo
    debug trycompile on
    CMake Error: The source directory "/foo" does not exist.
    Specify --help for usage, or press the help button on the CMake GUI.

Anyway this script sets CMAKE_VERBOSE_MAKEFILE, and this is what
--verbose does.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-07-29 11:36:51 +03:00
Yakov Olkhovskiy
ea17492f5f
Merge branch 'master' into ci-self-extracting-s3-deployment 2022-07-27 21:15:36 -04:00
Yakov Olkhovskiy
f91a0935cd
check existence of compressed clickhouse 2022-07-27 09:00:05 -04:00
Yakov Olkhovskiy
0720936c2d
rewrite original clickhouse with compressed one 2022-07-26 09:16:31 -04:00
Mikhail f. Shiryaev
061e61919a
Merge pull request #39450 from ClickHouse/tsal-clang-13
Build thread sanitizer binaries with clang-13
2022-07-26 15:13:17 +02:00
Yakov Olkhovskiy
c05c32e823
copy self-extracting to output 2022-07-26 08:41:36 -04:00
Mikhail f. Shiryaev
17b504b0d4
Build thread sanitizer binaries with clang-13 2022-07-22 12:14:22 +02:00
jasperzhu
c044c67745
Merge branch 'master' into dev_intel_iaa_deflate 2022-07-19 15:32:57 +08:00
Azat Khuzhin
1ef9910453 Fix builder image for releases w/o diagnostics tool
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-07-16 12:07:58 +03:00
Mikhail f. Shiryaev
7388b6fb55
Test and build clickhouse-diagnostics in release builds 2022-07-04 10:59:37 +02:00
Mikhail f. Shiryaev
0aae60f8f1
Update nfpm version 2022-07-01 11:43:13 +02:00
jasperzhu
bca18b2caf
Merge branch 'master' into dev_intel_iaa_deflate 2022-06-16 08:56:27 +08:00
BoloniniD
6120495373 Fix Dockerfile 2022-06-10 10:31:25 +03:00
BoloniniD
14ddcee432 Trying another Rust target to fix compatibility test 2022-06-07 18:10:49 +03:00
BoloniniD
b05ee41d25 Merge branch 'master' of github.com:ClickHouse/ClickHouse into BLAKE3 2022-06-06 16:03:10 +03:00
jinjunzh
c7ca746994 fixed build issues in clickhouse-keeper 2022-06-05 14:46:28 -04:00
BoloniniD
94c7833e25 Merge branch 'BLAKE3' of github.com:BoloniniD/ClickHouse into BLAKE3 2022-06-01 11:58:28 +03:00
BoloniniD
dd8aefdf1e Merge branch 'master' of github.com:ClickHouse/ClickHouse into BLAKE3 2022-06-01 11:46:55 +03:00
jinjunzh
1d3ae2d88c fixed unable to find library -ldl in docker build 2022-05-31 17:14:08 -04:00
jinjunzh
822cef10b3 add nasm into more docker configuration 2022-05-30 21:16:06 -04:00
Alexey Milovidov
dce7a86ca9 Merge branch 'master' into llvm-14 2022-05-22 22:38:07 +02:00
Mikhail f. Shiryaev
1f67665c95
Build performance output in release builds 2022-05-18 01:38:44 +02:00
Mikhail f. Shiryaev
8507a4ddbf
Update nfpm version, prepare UID/GID workdir 2022-05-18 01:38:43 +02:00
Mikhail f. Shiryaev
96859ec678
Remove unused packager code for deb packages 2022-05-18 01:38:43 +02:00
Mikhail f. Shiryaev
d2e5ab0f6c
Improve packager script a little bit 2022-05-18 01:38:42 +02:00
Nikolai Kochetov
f1bc2bc615 Merge branch 'master' into llvm-14 2022-05-16 12:03:26 +02:00
BoloniniD
228a60ede2
Merge branch 'master' into BLAKE3 2022-04-23 10:08:26 +03:00
BoloniniD
a26acd3660 Merge branch 'master' of github.com:ClickHouse/ClickHouse into BLAKE3 2022-04-22 18:12:54 +03:00
Mikhail f. Shiryaev
bf41199cc3
Fix issue with broken git ownership 2022-04-22 14:56:34 +02:00
Mikhail f. Shiryaev
fc83f5e96c
Get ccache status before build 2022-04-21 17:18:43 +02:00
Mikhail f. Shiryaev
7dd2fdb47c
Try to minimize linking for keeper 2022-04-21 16:11:43 +02:00
Mikhail f. Shiryaev
619d2703f0
Simplify build.sh for binary packager 2022-04-21 14:15:24 +02:00
Mikhail f. Shiryaev
49a572e00c
Build only clickhouse-keeper with musl 2022-04-21 13:43:24 +02:00
alesapin
9815b4bea4 Followup 2022-04-20 22:42:26 +02:00
alesapin
b8021be891 Perfect cmake 2022-04-20 21:19:43 +02:00
alesapin
ba81816dc1 Better cmake 2022-04-20 12:11:55 +02:00