Commit Graph

660 Commits

Author SHA1 Message Date
Max K
7844fcc196
improve CI with digest for docker, build and test jobs (#56317)
* CI with digests to tag docker images, reuse builds, reuse tests and others
2023-12-15 15:21:20 +01:00
Alexey Milovidov
8ec4a90cb4 Fix error 2023-11-26 17:55:13 +01:00
Alexey Milovidov
b2d8103d19 Add special build with Musl 2023-11-26 02:27:54 +01:00
Mikhail f. Shiryaev
c3a3cf8d24
Make check for the limited cmake dependencies the part of sparse checkout 2023-11-20 13:08:49 +01:00
Alexey Milovidov
8c6ed7f28c
Merge pull request #56898 from ClickHouse/simpler-cmake
Simpler CMake
2023-11-18 12:26:49 +01:00
Alexey Milovidov
283af96670 Merge branch 'master' into simpler-cmake 2023-11-17 19:02:53 +01:00
Alexey Milovidov
30bd5642e9 Merge branch 'master' into remove-useless-part-from-build 2023-11-17 19:00:09 +01:00
Alexey Milovidov
9eb0b74167
Update Dockerfile 2023-11-17 15:54:32 +03:00
Alexey Milovidov
ea3cd71225
Update Dockerfile 2023-11-17 15:42:10 +03:00
Alexey Milovidov
19dd29e8af
Update Dockerfile 2023-11-17 15:19:00 +03:00
Alexey Milovidov
215cd7b9f2
Update build.sh 2023-11-17 15:13:16 +03:00
Alexey Milovidov
2a9d05e245 Remove more lines 2023-11-17 12:59:04 +01:00
Alexey Milovidov
1d5bc13e2a Check what will happen if I remove some lines 2023-11-17 12:50:51 +01:00
Alexey Milovidov
318c7a06f9 Avoid dependencies with no fixed versions 2023-11-17 12:39:00 +01:00
Alexey Milovidov
0548eefbb7 Simpler CMake 2023-11-17 11:35:48 +01:00
Alexey Milovidov
4f441ec131 Own CMake for Abseil 2023-11-17 11:32:27 +01:00
Alexey Milovidov
c65607484e Remove garbage 2023-11-17 10:03:10 +01:00
Alexey Milovidov
c7d8465897 Ensure no new dependencies 2023-11-17 09:12:49 +01:00
Alexey Milovidov
f73b3e10ac Ensure no new dependencies 2023-11-17 08:46:45 +01:00
Mikhail f. Shiryaev
ed95698ff6
Decrease unnecessary verbosity 2023-11-13 19:38:11 +01:00
Mikhail f. Shiryaev
64d5f636a2
Do not fetch submodules during preparation performance archive 2023-11-13 18:07:25 +01:00
Alexey Milovidov
f2a67ae811 Addition 2023-11-11 09:24:26 +01:00
Mikhail f. Shiryaev
4848baf302
Adjust performance artifacts prepare to the filtered checkout 2023-10-18 00:01:38 +02:00
Ilya Yatsishin
262373c9ef
Merge pull request #51435 from qoega/sources-with-subs
Build artifact with sources with submodules in Darwin special build
2023-10-07 11:07:27 +02:00
Mikhail f. Shiryaev
f525f05598 Merge remote-tracking branch 'origin/master' into sources-with-subs 2023-10-03 14:42:24 +02:00
Alexey Milovidov
6c408c7867
Merge pull request #55071 from ClickHouse/nikitamikhaylov-patch-1
Use `source` instead of `bash` for pre-build script
2023-09-30 08:58:27 +03:00
Yatsishin Ilya
a78311bd8c Fix typo in packager when ccache is used 2023-09-28 14:40:13 +00:00
Nikita Mikhaylov
1f78755da2
Update build.sh 2023-09-28 11:47:23 +02:00
Nikita Mikhaylov
5753c3e72f
bash -> source 2023-09-28 00:48:17 +02:00
Nikita Mikhaylov
8ac88645c8
Use different names for variables inside build.sh (#55067)
* Better

* Even better
2023-09-27 20:50:19 +02:00
Nikita Mikhaylov
02ac8d56ef
Fix prebuild scripts one more time (#55059)
* Better

* Update build.sh

I made it a little bit shorter. 
The main value here that you changed negative condition with positive one. That excludes the cases when some error occur in condition. Since some error during calculation a predicate does not mean that predicate is false logically. It means predicate is just failed. Bash is hard in distinguishing such cases.

* Update build.sh

---------

Co-authored-by: Sema Checherinda <104093494+CheSema@users.noreply.github.com>
2023-09-27 17:10:02 +02:00
Nikita Mikhaylov
f0b570d0c3
Fix pre-build scripts for old branches (#55032) 2023-09-26 21:59:51 +02:00
Nikita Mikhaylov
aff486dac1
Additional scripts for packager (#55000) 2023-09-26 15:59:57 +02:00
Robert Schulze
cbcff6ef08
Merge remote-tracking branch 'rschu1ze/master' into clang-17 2023-09-20 09:15:58 +00:00
Mikhail f. Shiryaev
4882b99715
Merge pull request #54310 from ClickHouse/ci-libfuzzer-integration
CI: libFuzzer integration
2023-09-20 10:01:39 +02:00
Robert Schulze
3a3a00f95e
Bump minimal required compiler to Clang 16 2023-09-18 17:44:04 +00:00
Yakov Olkhovskiy
eb3a7caa74 add zip to build docker 2023-09-17 16:36:49 +00:00
Yakov Olkhovskiy
0847889db6 libFuzzer infrastructure 2023-09-17 16:36:49 +00:00
Yakov Olkhovskiy
46fa7dbb80 add libFuzzer build on 'libFuzzer' label, build produces artifacts 2023-09-17 16:28:50 +00:00
Mikhail f. Shiryaev
c6e531ab27
Do not build and upload apk packages, nobody uses them 2023-09-15 16:48:45 +02:00
Alexey Milovidov
12ff2d7cb1
Merge pull request #52721 from azat/musl-7
Fix calculating stack size under musl for the main thread
2023-09-05 18:37:40 +03:00
Azat Khuzhin
d2b31f3447 Merge remote-tracking branch 'u/master' into musl-7 2023-09-04 22:01:02 +02:00
Yakov Olkhovskiy
7afe9ca620 update build script 2023-09-03 19:57:39 +00:00
robot-clickhouse
c8e0b8267d Automatic style fix 2023-09-03 17:49:25 +00:00
Yakov Olkhovskiy
97317a07f1 remove oss-fuzz build, add fuzzers build 2023-09-03 17:28:44 +00:00
Yakov Olkhovskiy
fa7994de3b
Merge pull request #53181 from ClickHouse/ci-build-s390x
Add Linux s390x build to CI
2023-08-12 09:16:07 -04:00
Yakov Olkhovskiy
ad7c60e216 fix 2023-08-10 21:08:47 +00:00
Mikhail f. Shiryaev
18a291a4c7
Upload cargo cache for rust to S3 and reuse 2023-08-10 22:41:55 +02:00
Mikhail f. Shiryaev
cbd81b3758
Partially revert changes from #52395 2023-08-10 22:09:19 +02:00
Yakov Olkhovskiy
18447022e6 add mold 2.0 to docker 2023-08-10 20:01:53 +00:00
Mikhail f. Shiryaev
c7b045fdde
Process build profiles in the python script, not in docker 2023-08-09 21:48:05 +02:00
Mikhail f. Shiryaev
5341c5c8dd
Use CMAKE_FLAGS and args for cmake profiler 2023-08-09 18:26:25 +02:00
Alexey Milovidov
964c407d48
Revert "Revert "Upload build time-trace data to CI database"" 2023-08-09 16:11:17 +03:00
Alexander Gololobov
63183c56f1
Revert "Upload build time-trace data to CI database" 2023-08-09 13:58:05 +02:00
Yakov Olkhovskiy
c6c6d1ed9c fix 2023-08-09 02:42:04 +00:00
Yakov Olkhovskiy
d8a231e0cb Add Linux s390x build to CI 2023-08-09 00:44:01 +00:00
Alexey Milovidov
8665f675c6 Investigation 2023-08-08 19:07:02 +02:00
Alexey Milovidov
65a9dd1d18 Fix style 2023-08-08 16:13:17 +02:00
Alexey Milovidov
7ce8266c8a Fix errors 2023-08-08 15:17:20 +02:00
Alexey Milovidov
a2001e15de Add jq 2023-08-08 02:32:57 +02:00
Alexey Milovidov
f1257e871d Correct the script 2023-08-08 01:46:03 +02:00
Alexey Milovidov
a38539ece0 Add missing argument 2023-08-08 00:16:22 +02:00
Alexey Milovidov
d977e221b4 Upload build time-trace data to CI database 2023-08-07 03:26:59 +02:00
Alexey Milovidov
39883f9373 Add missing targets 2023-08-05 02:35:44 +02:00
Mikhail f. Shiryaev
a4185d9be0
Merge branch 'master' into sources-with-subs 2023-08-03 16:08:39 +02:00
Mikhail f. Shiryaev
e5087d14ab
Merge branch 'master' into fix-packager 2023-08-01 12:04:06 +02:00
Alexey Milovidov
8155659f87
Merge pull request #52775 from ClickHouse/remove-coverity-2
Remove Coverity (part 2)
2023-07-30 12:49:04 +03:00
Alexey Milovidov
6bff19bd82 Remove Coverity (part 2) 2023-07-30 10:27:33 +02:00
Alexey Milovidov
bc1f80dd82 Remove Coverity (part 2) 2023-07-30 10:25:44 +02:00
Alexey Milovidov
0f66d161d6 Merge branch 'master' of github.com:ClickHouse/ClickHouse into no-export-dynamic 2023-07-23 21:33:51 +02:00
Alexey Milovidov
43cf31a98e
Merge branch 'master' into no-export-dynamic 2023-07-23 07:26:33 +03:00
Azat Khuzhin
0b258dda4e Reproducible builds for Rust
From now on cargo will not download anything from the internet during
builds. This step had been moved for docker image builds (via cargo
vendor).

And now cargo inside docker.io/clickhouse/binary-builder will not use
any crates from the internet, so we don't need to add --offline for
cargo commands in cmake (corrosion_import_crate()).

Also the docker build command had been adjusted to allow following
symlinks inside build context, by using tar, this is required for Rust
packages.

Note, that to make proper Cargo.lock that could be vendored I did the
following:
- per-project locks had been removed (since there is no automatic way to
  sync the workspace Cargo.lock with per-project Cargo.lock, since cargo
  update/generate-lockfile will use only per-project Cargo.toml files
  apparently, -Z minimal-versions does not helps either)
- and to generate Cargo.lock with less changes I've pinned version in
  the Cargo.toml strictly, i.e. not 'foo = "0.1"' but 'foo = "=0.1"'
  then the Cargo.lock for workspace had been generated and afterwards
  I've reverted this part.

Plus I have to update the dependencies afterwards, since otherwise there
are conflicts with dependencies for std library. Non trivial.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-07-22 22:46:22 +02:00
Antonio Andelic
3c2507ad7d
Merge branch 'master' into fix-packager 2023-07-18 09:18:03 +02:00
Mikhail f. Shiryaev
0ba38b4dde
Merge branch 'master' into sources-with-subs 2023-07-17 14:04:02 +02:00
Alexey Milovidov
f8eaa3e8e6
Merge pull request #51903 from azat/build/pin-rust
Pin rust nightly (to make it stable)
2023-07-07 20:28:57 +03:00
Azat Khuzhin
ec5e26a017 Pin rust nightly (to make it stable)
Because of using Rust nightly, and without #49601 the Rust toolchain is
very unstable, and can be frequently failed.

So let's ping particular version.

Also I've looked and it seems that Rust archives stores this archive
without any TTL, since there is even a version for 2015 year.

Follow-up for: #50541
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-07-06 21:17:50 +02:00
Alexey Milovidov
1eaa2e9d30
Merge branch 'master' into no-export-dynamic 2023-07-04 22:37:46 +03:00
Alexey Milovidov
1eafba60d2 Merge branch 'master' into riscv-in-ci 2023-07-04 21:28:05 +02:00
Antonio Andelic
3277e4b5bd Fix build type in packager 2023-07-04 11:22:19 +00:00
Alexey Milovidov
463c2ba5ce
Revert "Add documentation for building in docker" 2023-07-04 14:07:10 +03:00
Mikhail f. Shiryaev
c715ee5cef
Replace --build-type=debug by --debug-build 2023-07-03 19:58:47 +02:00
Mikhail f. Shiryaev
469da1831a
Replace --build-type=debug by --debug-build 2023-07-03 17:42:15 +02:00
Mikhail f. Shiryaev
6d2b5166b9
Update golang version in builder 2023-06-28 10:32:20 +02:00
Ilya Yatsishin
a9cb4d1b3d Better tar 2023-06-27 12:32:32 +02:00
Ilya Yatsishin
ed6bfd66fe Publish changes 2023-06-26 17:28:59 +02:00
Alexey Milovidov
fef63351bb
Merge branch 'master' into no-export-dynamic 2023-06-11 19:56:00 +03:00
Alexey Milovidov
5d26b0cf7c
Merge pull request #50550 from cmsxbc/misc-darwin-build
skip extracting darwin toolchain in builder when unncessary
2023-06-05 11:11:34 +03:00
Alexey Milovidov
47379ac039
Update build.sh 2023-06-05 03:58:42 +03:00
Azat Khuzhin
bf127f4e1e MSan support for Rust
Previously you have to unpoison memory from the Rust, however Rust does
supports MSan, so let's simply use it.

But for this we need nightly Rust and recompile standard library.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-06-04 23:04:55 +02:00
cmsxbc
4bb44c7c72
1. skip extract darwin toolchain in builder when uncessary
2. update MacOSX SDK version in toolchain readme to match in builder
2023-06-04 23:06:21 +08:00
Azat Khuzhin
c37fe64040 Merge remote-tracking branch 'u/master' into build/llvm-16 2023-05-08 22:33:52 +02:00
Alexey Milovidov
8cd9fc4a2d Fix build 2023-05-06 23:52:16 +02:00
Mikhail f. Shiryaev
7ff582b963
Deprecate ENABLE_CCACHE and replace it by COMPILER_CACHE 2023-03-29 23:33:31 +02:00
Mikhail f. Shiryaev
def5ae851e
Use CTCACHE_DIR only with ccache 2023-03-29 23:33:29 +02:00
Mikhail f. Shiryaev
f1b1f30764
Make --ccache-dir equal to --ccache_dir 2023-03-29 23:33:27 +02:00
Mikhail f. Shiryaev
fcc14ae724
Enable s3 caching 2023-03-29 23:33:27 +02:00
Mikhail f. Shiryaev
6d8da8f88b
Add sccache and ctcache S3 parameters, don't use 2023-03-29 23:33:27 +02:00
Mikhail f. Shiryaev
df17a35410
Get rid of useless cache_maxsize parameter 2023-03-29 23:33:26 +02:00
Mikhail f. Shiryaev
2d872ff23b
Add a function to parse arguments 2023-03-29 23:33:26 +02:00
Mikhail f. Shiryaev
419f234294
Fix logic for --force-build-image 2023-03-29 23:33:25 +02:00