Commit Graph

314 Commits

Author SHA1 Message Date
Alexander Kuzmenkov
4c2278c215 Merge pull request #22047 from ClickHouse/aku/llvm-cloudflare-21.3
21.3: quick fix for broken resolution of apt.llvm.org on Yandex infra

(cherry picked from commit 12f5753e5f)
2021-03-23 19:41:09 +03:00
Danila Kutenin
e5ec4e21b6 Fix Darwin build 2021-01-11 09:04:34 +00:00
Azat Khuzhin
469b631ca6 Fix performance comparison
In #18113 the top_level_domains list copying was moved into
docker/packager/binary/build.sh, this was done to avoid symlinks (since
Dockerfile cannot dereference them).

But the patch was wrong, since it copied into the root (/), which is not
included into performance.tgz and also compare.sh was not modified.

This wasn't showed up with CI checks since the docker image wasn't
updated and it still included that top_level_domains, once it was
modified the image was updated and it became broken.

Cc: @akuzm
2021-01-02 17:09:20 +03:00
alesapin
b4cac6a8a4 Fix unbundled, splitted and disable freebsd build 2020-12-25 15:36:49 +03:00
Nikolai Kochetov
af7f5c9518
Merge pull request #17868 from ClickHouse/async-read-from-socket
Async read from socket
2020-12-23 12:20:42 +03:00
Azat Khuzhin
95c35b318e Avoid using symlinks for top_level_domains
Move the main copy into the tests/config and ship it to the perf tests
via the perf package.
2020-12-15 21:58:46 +03:00
Nikolai Kochetov
4905201985 Merge branch 'master' into async-read-from-socket 2020-12-15 21:04:47 +03:00
Alexander Kuzmenkov
fe6f477881 Build utils in CI, at least in split build
Currently we don't build them at all, so they were broken by https://github.com/ClickHouse/ClickHouse/pull/17988
2020-12-14 20:39:52 +03:00
Nikolai Kochetov
75ac87c241 Fixing build. 2020-12-14 17:42:12 +03:00
Vitaly Baranov
8e6c68f4f7 Fix unbundled build. 2020-11-24 17:53:34 +03:00
sundy-li
46c01eb7a2 improve unbundle build 2020-11-11 09:08:53 +08:00
sundy-li
c7617173c4 add sync_file_range for glibc-compatibility 2020-11-11 09:08:52 +08:00
alexey-milovidov
fa929b0312
Replace duplicate development config files with symlinks. (#16486)
* Fix test "max_memory_usage_for_user"

* Update test

* Update annoying boilerplate

* Support symlinks in tarballs

* Fix Fuzzer

* Remove "secondary" user

* Remove "secondary" user
2020-11-04 23:45:06 +03:00
Alexey Milovidov
d412e0ca2c Move trash to "Unbundled" Dockerfile 2020-10-26 19:41:38 +03:00
alesapin
848de47a2c Remove checksum check from arrow package 2020-10-21 17:49:37 +03:00
alesapin
9caf46c273
Update build.sh 2020-10-05 10:06:38 +03:00
alesapin
b3fa86191e Fix binary build 2020-10-04 13:23:25 +03:00
Azat Khuzhin
9cb3c743bd
Convert to python3 (#15007) 2020-10-02 19:54:07 +03:00
Vitaly Baranov
dd634aa731
Merge pull request #15491 from vitlibar/update-grpc
Update gRPC to the latest version
2020-10-02 15:54:35 +03:00
alesapin
c5df9433f0
Merge pull request #15239 from ClickHouse/100_percent_lld_11_for_clang_11
Build ClickHouse for release with clang-11
2020-10-02 14:59:29 +03:00
alesapin
bde65557f1 Fix bash 2020-10-02 12:40:50 +03:00
Vitaly Baranov
65a1684ed7 Fix gRPC build scripts. 2020-10-02 01:45:01 +03:00
Alexander Kuzmenkov
d0dff5b738 fixes 2020-10-01 12:27:05 +03:00
Alexander Kuzmenkov
499e0766e5 Check docker scripts with shellcheck 2020-09-30 20:06:46 +03:00
alesapin
9f89fe095a Missed changes 2020-09-29 10:17:02 +03:00
alesapin
eafcec7ebf Move release build to clang-11 2020-09-29 10:15:05 +03:00
alesapin
646e843105 100% lld-11 for clang-11 builds 2020-09-24 19:26:04 +03:00
Alexey Milovidov
035d7cb47f Enable control for heavy translation units 2020-09-20 23:46:32 +03:00
alesapin
368ddada33 Fix binary build 2020-09-18 22:01:59 +03:00
alesapin
1f429b3d52 Set enable tests to zero for performance build 2020-09-18 22:01:59 +03:00
Nikolai Kochetov
000cd42d5e
Merge pull request #14917 from ClickHouse/revert-14607-lower-number-of-threads-in-binary-build
Revert "Lower number of threads in binary build"
2020-09-17 15:45:02 +03:00
Nikolai Kochetov
d23244b3a5
Revert "Lower number of threads in binary build" 2020-09-17 12:45:06 +03:00
alesapin
7fc09b930e Clang 11 in binary build 2020-09-15 15:16:23 +03:00
alesapin
393c213866 Revert changes 2020-09-14 15:25:00 +03:00
alesapin
558164bf24 Add retries 2020-09-14 11:17:59 +03:00
alesapin
e519e2b87a Use patched dpkg on build stage 2020-09-14 11:16:00 +03:00
alesapin
b0e6df1532 Trying to fix build 2020-09-14 10:21:03 +03:00
alesapin
c202ce8e63 Merge branch 'master' into update_to_gcc_10 2020-09-11 12:50:10 +03:00
alesapin
956138635d Fix compiler name 2020-09-09 15:59:26 +03:00
alesapin
c535d75243 Add update 2020-09-09 14:47:34 +03:00
alesapin
ca6b634eb0 Install gcc-10 from proposed repo 2020-09-09 13:51:01 +03:00
alesapin
f528cd9f97 Forward compiler version to unbundled build 2020-09-09 13:01:12 +03:00
Alexey Milovidov
cc2b4014ff Lower number of threads in binary build 2020-09-09 09:32:09 +03:00
alesapin
54cdb2472c Bump binary image version 2020-09-07 18:40:21 +03:00
Azat Khuzhin
2d7cb03120 Suppress superfluous wget (-nv) output
Since for dowloading some of files wget logging may take 50% of overall
log [1].

  [1]: https://clickhouse-builds.s3.yandex.net/14315/c32ff4c98cb3b83a12f945eadd180415b7a3b269/clickhouse_build_check/build_log_761119955_1598923036.txt
2020-09-01 10:25:13 +03:00
Alexey Milovidov
0aec4c99d0 Cut off broken parts of "unbundled" build 2020-08-29 18:42:04 +03:00
Yatsishin Ilya
2e7a4483f6 fixes after #13687 2020-08-27 20:35:51 +03:00
Yatsishin Ilya
3bf5dc578d fixes 2020-08-27 19:06:40 +03:00
Yatsishin Ilya
fa1875a313 Merge remote-tracking branch 'origin' into unbundled-packager 2020-08-27 13:00:56 +03:00
Konstantin Podshumok
cbdedc39f2
REVERSE ME: workaround for docker/packager/unbundled
Please, reverse this commit after docker/packager/deb image update

Signed-off-by: Konstantin Podshumok <kpp.live+signed@gmail.com>
2020-08-25 20:34:36 +03:00
Konstantin Podshumok
f4383389d5
sync unbundled/build.sh with a0ae1b2b42
Signed-off-by: Konstantin Podshumok <kpp.live+signed@gmail.com>
2020-08-25 20:16:44 +03:00
Konstantin Podshumok
cda4472ebc
dockerfiles: verify 3rd party apt repos, disable LLVM in PVS check
Signed-off-by: Konstantin Podshumok <kpp.live+signed@gmail.com>
2020-08-25 20:16:44 +03:00
Konstantin Podshumok
68a921cdea
cmake: use bundled rdkafaka for in unbundled tests for now
Signed-off-by: Konstantin Podshumok <kpp.live+signed@gmail.com>
2020-08-25 20:16:43 +03:00
Konstantin Podshumok
45e089e0c3
re-enable replxx in packager for unbundled
Signed-off-by: Konstantin Podshumok <kpp.live+signed@gmail.com>
2020-08-25 20:16:42 +03:00
Konstantin Podshumok
b3180437da
test builds: docker/packager/packager: re-enable mysql
Signed-off-by: Konstantin Podshumok <kpp.live+signed@gmail.com>
2020-08-25 20:16:42 +03:00
Konstantin Podshumok
17c46faaa7
cmake: fix parquet/arrow variables
squashed:

- sync FindArrow.cmake and FindParquet.cmake with arrow repo
- unbundled: add arrow to dbms link libraries
- cmake: fix-up unbundled Arrow support

Signed-off-by: Konstantin Podshumok <kpp.live+signed@gmail.com>
2020-08-25 20:16:41 +03:00
Konstantin Podshumok
4e9b5bd31b
docker/packager/deb: new dependencies for unbundled
Squashed:
- docker/packager/deb: add arrow repo
- docker/packager/deb: cleanup
- docker/packager/deb: Add thrift
- docker/packager/deb: add myariadbclient
- docker/packager/deb: remove poco-dev as we always use contrib/ version now
- docker/packager/deb: add libssl-dev (unbundled) and restructure
- docker/packager/deb: fix repo url
- docker/packager/deb: fix tzdata not updated in build.sh
- docker/packager/deb: add mysqlclient-dev

Signed-off-by: Konstantin Podshumok <kpp.live+signed@gmail.com>
2020-08-25 20:16:40 +03:00
Yatsishin Ilya
b8002dc812 mErge remote-tracking branch 'origin' into unbundled-packager 2020-08-25 14:12:01 +03:00
alexey-milovidov
7ea5364299
Merge pull request #13648 from filimonov/tzdata_version2
tzdata improvements
2020-08-24 23:07:37 +03:00
Yatsishin Ilya
f2182db765 Separate stateless unbundled image 2020-08-24 13:05:56 +03:00
Yatsishin Ilya
ccbfd2e5df Update deb-builder ubuntu version. Add clang-11, gcc-10. Add unbundled docker. Add test-base image. 2020-08-18 17:16:40 +03:00
alexey-milovidov
3fac908dc7
Revert "Added separate docker image for unbundled build. Added clang-11, gcc-10 in packager image. Updated packager image to ubuntu 20.04" 2020-08-15 15:43:58 +03:00
alexey-milovidov
c241e42db0
Update Dockerfile 2020-08-15 11:05:38 +03:00
Ilya Yatsishin
b74ded3dc0
Update build.sh 2020-08-14 15:34:00 +03:00
Yatsishin Ilya
22938a7152 Add separate docker container for unbundled build 2020-08-14 15:22:47 +03:00
Mikhail Filimonov
a0ae1b2b42 tzdata improvements
Expose version of embedded tzdata via TZDATA_VERSION in system.build_options.
Autogenerate list of shipped timezones. Script to check tzdb updates.
2020-08-12 14:21:40 +02:00
Alexey Milovidov
33197fee5b Build ClickHouse with fresh tzdata 2020-08-11 22:03:04 +03:00
alesapin
48466c0960 New version of dpkg-deb 2020-08-07 19:07:52 +03:00
Alexander Kuzmenkov
8f5c4a25c9 More branches in perf test repo snapshot 2020-07-23 15:51:12 +03:00
Alexander Kuzmenkov
cdca798d2a
Update build.sh 2020-07-15 00:56:35 +03:00
Alexander Kuzmenkov
6498fe8123 Publish repo snapshot in perf test package 2020-07-14 18:12:53 +03:00
Alexander Kuzmenkov
76ef8feb04
Update packager 2020-07-08 12:52:03 +03:00
Alexander Kuzmenkov
08762dc825
Update build.sh 2020-07-08 12:46:44 +03:00
alesapin
7ce3464810 Remove -v from ninja 2020-07-06 18:13:08 +03:00
Azat Khuzhin
f91e34b2e8 Include libcurl4-openssl-dev into yandex/clickhouse-deb-builder 2020-07-05 00:56:48 +03:00
alesapin
f0b012c94b Remove garbage from images 2020-07-01 23:07:11 +03:00
alesapin
29178a1df4 Don't download image twice 2020-07-01 21:51:11 +03:00
alesapin
faaf3536ea Update deb image 2020-07-01 18:00:07 +03:00
alesapin
29c528235e Add ability to run any image version from packager and runner 2020-06-30 16:08:12 +03:00
alesapin
db79278981 Fake change in dockerfile 2020-06-29 16:13:20 +03:00
Alexander Kuzmenkov
01344d7b87 More verbose CMake in build docker 2020-06-25 22:16:48 +03:00
Alexander Kuzmenkov
4a8349cad9 Less layers in docker file 2020-06-25 09:26:10 +03:00
Alexander Kuzmenkov
63a8a8453a Less layers in docker file 2020-06-25 09:23:25 +03:00
Alexander Kuzmenkov
ce36da838c for the love of God just update this container already, i'm begging you 2020-06-24 19:12:05 +03:00
Alexander Kuzmenkov
751319d0ef show build commands 2020-06-24 16:16:54 +03:00
Azat Khuzhin
0731f2293d Add missing boost libraries into docker image 2020-06-11 11:47:30 +03:00
Azat Khuzhin
1f3c640da2 Use unbundled boost for unbundled builds 2020-06-10 20:56:47 +03:00
Ivan
47ad338cb2
Refactor CMake build files (#11390)
* Get rid of lib_name.cmake
* Refactor Boost and HyperScan libraries
* Refactor lz4
* Fix build with xxHash
2020-06-09 13:54:49 +03:00
Ivan
1e1b30dc05
Add libhyperscan-dev to deb Dockerfile 2020-06-07 10:19:56 +03:00
alesapin
d8de91ea4d Don't build redundant garbage in binary builds 2020-06-06 17:00:47 +03:00
alesapin
5038733abf
bump2 2020-06-05 14:20:54 +03:00
alesapin
3d48c10bc6
bump CI 2020-06-05 12:22:23 +03:00
alesapin
e964b6c93f Trying to increase ccache size 2020-06-04 22:48:48 +03:00
Denis Glazachev
808351048f Add libsasl2-dev and heimdal-multidev required by static system openldap libraries 2020-05-30 19:44:51 +04:00
Ilya Yatsishin
0f6410b56f
Merge pull request #11108 from ClickHouse/revert-11099-revert-10908-packager-add-bin
Revert "Revert "Allow to build binaries and tests from deb package build. ""
2020-05-21 19:59:52 +03:00
alesapin
20d41a3f98 Update dpkg-deb (https://github.com/ClickHouse-Extras/dpkg) for compatibility with ubuntu eoan packages format 2020-05-21 18:30:08 +03:00
Ilya Yatsishin
9dc493e063
Revert "Revert "Allow to build binaries and tests from deb package build. "" 2020-05-21 18:02:52 +03:00
Ilya Yatsishin
cce805c42c
Revert "Allow to build binaries and tests from deb package build. " 2020-05-21 14:50:12 +03:00
Yatsishin Ilya
e3c692bc48 fix issues 2020-05-19 10:54:08 +03:00
Yatsishin Ilya
9c551b8b4d set USE_GTEST for tests build 2020-05-19 10:54:08 +03:00
Yatsishin Ilya
8d190f316d wip 2020-05-19 10:54:08 +03:00
Yatsishin Ilya
b780adb241 wip 2020-05-19 10:54:08 +03:00
Yatsishin Ilya
60f9d2088d wip 2020-05-19 10:54:08 +03:00
Yatsishin Ilya
1086cf7691 wip 2020-05-19 10:54:08 +03:00
Artem Zuikov
0ba4c373a9 add OpenCL libraries for CI (part 2) 2020-05-14 12:45:22 +03:00
Artem Zuikov
9b029eb310
add OpenCL libraries for CI (#10898) 2020-05-14 12:36:17 +03:00
alesapin
a89653effe Add libldap2-dev to build image 2020-05-13 14:53:35 +03:00
Ivan
85d783c247
Poco contrib refactoring (#10396)
* Remove config_common.h
* Refactor libcpuid contrib
* Remove support for libcpuinfo
* Define USE_CPUID in Arcadia
* Refactor Poco libraries
2020-05-08 17:11:19 +03:00
Ivan
038e371570
Update cross-builds to use clang-10 compiler (#10724)
* Update FreeBSD toolchain
* Update cross-builds to use clang-10
2020-05-07 15:32:17 +03:00
Alexey Milovidov
cef28325ff Update to clang-10 2020-05-02 22:00:05 +03:00
Alexey Milovidov
ee81e5fef7 Add LLVM tools to Docker images 2020-04-30 15:10:32 +03:00
Ivan
91df18a906
Trigger new image build
to pull changes in freebsd toolchain
2020-04-28 17:06:52 +03:00
BohuTANG
71f9346db3 docker: fix one typo for clang-9 2020-04-27 17:33:54 +08:00
Alexey Milovidov
a0d56997de Fix error 2020-04-20 23:13:33 +03:00
Alexey Milovidov
31d2547363 clang-10 in CI 2020-04-20 23:04:35 +03:00
Azat Khuzhin
66d443df14 Add libmsgpack-dev into the image (for unbundled build) 2020-04-10 11:30:23 +03:00
Alexey Milovidov
b6f4287d2f Remove vagrant build 2020-04-08 23:42:12 +03:00
Ivan Lezhankin
1e3ec9113a Fix compiler possible values 2020-04-08 19:29:08 +03:00
Ivan
f6b31f344d
Add cross-compile build for FreeBSD (#9643)
* Add toolchain to Docker image
2020-04-07 11:33:49 +03:00
Ivan Lezhankin
06446b4f08 dbms/ → src/ 2020-04-03 18:14:31 +03:00
Ivan Lezhankin
159e72ea7c Trigger Docker image rebuild 2020-04-02 10:52:46 +03:00
Ivan
97f2a2213e
Move all folders inside /dbms one level up (#9974)
* Move some code outside dbms/src folder
* Fix paths
2020-04-02 02:51:21 +03:00
alesapin
5b8824c065 Add clang-8 to binary image 2020-03-23 21:23:32 +03:00
alesapin
72f3fbb630 Fix binary image 2020-03-23 19:08:13 +03:00
alesapin
ea061df5cd Add ability to build with clang-9 from packager script and remove outdated compilers 2020-03-23 17:49:52 +03:00
alexey-milovidov
92684b911d
Update Dockerfile 2020-03-23 14:02:39 +03:00
Alexey Milovidov
8df43f2f50 Update packager images 2020-03-23 00:10:14 +03:00
Alexey Milovidov
1bfe312834 Removed leftovers 2020-03-21 19:43:31 +03:00
Ivan
7aca050684
Add libcctz-dev package to Dockerfile 2020-03-17 16:18:36 +03:00
Alexey Milovidov
1c75827046 Packager script: add --clang-tidy option 2020-03-12 16:13:54 +03:00
Alexander Kuzmenkov
3be5fa1338 performance comparison 2020-03-11 17:29:34 +03:00
alesapin
3fbaa0a29f Add clang tidy to docker images 2020-03-10 12:07:43 +03:00
alesapin
49c0642c49 Build gcc with lld linker 2020-03-03 15:32:54 +03:00
Alexander Kuzmenkov
ebe70f5bcb performance comparison 2020-02-25 16:12:26 +03:00
alesapin
93fddc7b46 Remove symlink to lld from build images 2020-02-21 18:49:53 +03:00
alesapin
954ea60a6a Remove redundant compilers, linkers and libs 2020-02-10 12:05:14 +03:00
alesapin
bddbc956ec Add symlynks for lld, because gcc is not able to find it without 'ld.' prefix. 2020-02-07 23:59:51 +03:00
Ivan
b80e3dcd46
Refactoring of replxx (#8748) 2020-01-23 11:18:19 +03:00
Ivan Lezhankin
d90348eaa8 Disable unbundled with rdkafka
to fix it later
2020-01-13 14:28:30 +03:00
Ivan Lezhankin
fb677fd668 Disable replxx in unbundled build 2020-01-10 13:55:50 +03:00
Ivan Lezhankin
a33ade77fd Update packager for darwin 2020-01-01 22:27:48 +03:00
Alexander Kuzmenkov
17f1754695 Publish different files for perftest. 2019-12-25 19:09:37 +03:00
Alexander Kuzmenkov
be963a7a6f Add a build package type for performance tests. 2019-12-23 21:34:12 +03:00
Alexander Kuzmenkov
cb19696778 Dump cmake options in CI. 2019-12-18 18:40:42 +03:00
Ivan
453ff2c071
Refactor packager (#7539) 2019-10-30 17:36:27 +03:00
Ivan
4f2f5cca84
Add support for cross-compiling to the CPU architecture AARCH64 (#7370) 2019-10-30 10:01:53 +03:00
Ivan
4d27e8aa5b
Unpack toolchains into mounted volume (#7534) 2019-10-29 20:33:31 +03:00
Ivan
c250db4922
Update Docker Image for Binary Packager (#7474) 2019-10-24 15:56:30 +03:00
Ivan
32ca372b9d
Revert "Update Dockerfile for binary packager (#7456)" (#7458)
This reverts commit fa05a5860f.
2019-10-23 18:54:18 +03:00
Ivan
fa05a5860f
Update Dockerfile for binary packager (#7456) 2019-10-23 17:41:17 +03:00