ClickHouse/utils/ci
Denis Glazachev 9e3a28a6b8 Merge branch 'master' into ldap-per-user-authentication
* master: (414 commits)
  Update file.md
  Update merge.md
  Update dictionary.md
  Update external-data.md
  Update distributed.md
  Update null.md
  Update set.md
  Update join.md
  Update url.md
  Update view.md
  Update materializedview.md
  Update memory.md
  Update buffer.md
  Update generate.md
  removed a sentence about global lock during rename (#11577)
  greatCircleAngle en translation (#11584)
  Update configuration-files.md
  try fix flacky test
  Update why.html
  Update rich.html
  ...

# Conflicts:
#	src/Common/ErrorCodes.cpp
#	utils/ci/jobs/quick-build/run.sh
2020-06-11 03:06:17 +04:00
..
docker-multiarch Fix strange code 2020-05-29 03:42:25 +03:00
jobs/quick-build Merge branch 'master' into ldap-per-user-authentication 2020-06-11 03:06:17 +04:00
vagrant-freebsd Moved "ci" directory to "utils", because we are actually using different CI, but these scripts are still usable 2019-08-04 22:10:14 +03:00
build-clang-from-sources.sh Moved "ci" directory to "utils", because we are actually using different CI, but these scripts are still usable 2019-08-04 22:10:14 +03:00
build-debian-packages.sh Moved "ci" directory to "utils", because we are actually using different CI, but these scripts are still usable 2019-08-04 22:10:14 +03:00
build-gcc-from-sources.sh minor fixes in build-gcc script 2020-01-22 16:27:27 +05:00
build-normal.sh Moved "ci" directory to "utils", because we are actually using different CI, but these scripts are still usable 2019-08-04 22:10:14 +03:00
check-docker.sh Moved "ci" directory to "utils", because we are actually using different CI, but these scripts are still usable 2019-08-04 22:10:14 +03:00
check-syntax.sh Moved "ci" directory to "utils", because we are actually using different CI, but these scripts are still usable 2019-08-04 22:10:14 +03:00
create-sources-tarball.sh Moved "ci" directory to "utils", because we are actually using different CI, but these scripts are still usable 2019-08-04 22:10:14 +03:00
default-config Changed repository URL 2019-09-23 19:18:19 +03:00
get-sources.sh Moved "ci" directory to "utils", because we are actually using different CI, but these scripts are still usable 2019-08-04 22:10:14 +03:00
install-compiler-from-packages.sh Moved "ci" directory to "utils", because we are actually using different CI, but these scripts are still usable 2019-08-04 22:10:14 +03:00
install-compiler-from-sources.sh Moved "ci" directory to "utils", because we are actually using different CI, but these scripts are still usable 2019-08-04 22:10:14 +03:00
install-libraries.sh Moved "ci" directory to "utils", because we are actually using different CI, but these scripts are still usable 2019-08-04 22:10:14 +03:00
install-os-packages.sh Moved "ci" directory to "utils", because we are actually using different CI, but these scripts are still usable 2019-08-04 22:10:14 +03:00
prepare-docker-image-ubuntu.sh Moved "ci" directory to "utils", because we are actually using different CI, but these scripts are still usable 2019-08-04 22:10:14 +03:00
prepare-toolchain.sh Moved "ci" directory to "utils", because we are actually using different CI, but these scripts are still usable 2019-08-04 22:10:14 +03:00
prepare-vagrant-image-freebsd.sh Moved "ci" directory to "utils", because we are actually using different CI, but these scripts are still usable 2019-08-04 22:10:14 +03:00
README.md cosmetic doc fixes 2019-12-26 18:59:38 +03:00
run-clickhouse-from-binaries.sh dbms/ → src/ 2020-04-03 18:14:31 +03:00
run-with-docker.sh Moved "ci" directory to "utils", because we are actually using different CI, but these scripts are still usable 2019-08-04 22:10:14 +03:00
run-with-vagrant.sh Moved "ci" directory to "utils", because we are actually using different CI, but these scripts are still usable 2019-08-04 22:10:14 +03:00

Build and test ClickHouse on various plaforms

Quick and dirty scripts.

Usage example:

./run-with-docker.sh ubuntu:bionic jobs/quick-build/run.sh

Another example, check build on ARM 64:

./prepare-docker-image-ubuntu.sh
./run-with-docker.sh multiarch/ubuntu-core:arm64-bionic jobs/quick-build/run.sh

Another example, check build on FreeBSD:

./prepare-vagrant-image-freebsd.sh
./run-with-vagrant.sh freebsd jobs/quick-build/run.sh

Look at default_config and jobs/quick-build/run.sh

Various possible options. We are not going to automate testing all of them.

CPU architectures:

  • x86_64;
  • AArch64;
  • PowerPC64LE.

x86_64 is the main CPU architecture. We also have minimal support for AArch64 and PowerPC64LE.

Operating systems:

  • Linux;
  • FreeBSD.

We also target Mac OS X, but it's more difficult to test. Linux is the main. FreeBSD is also supported as production OS. Mac OS is intended only for development and have minimal support: client should work, server should just start.

Linux distributions:

For build:

  • Ubuntu Bionic;
  • Ubuntu Trusty.

For run:

  • Ubuntu Hardy;
  • CentOS 5

We should support almost any Linux to run ClickHouse. That's why we test also on old distributions.

How to obtain sources:

  • use sources from local working copy;
  • clone sources from github;
  • download source tarball.

Compilers:

  • gcc-7;
  • gcc-8;
  • clang-6;
  • clang-svn.

Compiler installation:

  • from OS packages;
  • build from sources.

C++ standard library implementation:

  • libc++;
  • libstdc++ with C++11 ABI;
  • libstdc++ with old ABI.

When building with clang, libc++ is used. When building with gcc, we choose libstdc++ with C++11 ABI.

Linkers:

  • ldd;
  • gold;

When building with clang on x86_64, ldd is used. Otherwise we use gold.

Build types:

  • RelWithDebInfo;
  • Debug;
  • ASan;
  • TSan.

Build types, extra:

  • -g0 for quick build;
  • enable test coverage;
  • debug tcmalloc.

What to build:

  • only clickhouse target;
  • all targets;
  • debian packages;

We also have intent to build RPM and simple tgz packages.

Where to get third-party libraries:

  • from contrib directory (submodules);
  • from OS packages.

The only production option is to use libraries from contrib directory. Using libraries from OS packages is discouraged, but we also support this option.

Linkage types:

  • static;
  • shared;

Static linking is the only option for production usage. We also have support for shared linking, but it is intended only for developers.

Make tools:

  • make;
  • ninja.

Installation options:

  • run built clickhouse binary directly;
  • install from packages.

How to obtain packages:

  • build them;
  • download from repository.

Sanity checks:

  • check that clickhouse binary has no dependencies on unexpected shared libraries;
  • check that source code have no style violations.

Tests:

  • Functional tests;
  • Integration tests;
  • Unit tests;
  • Simple sh/reference tests;
  • Performance tests (note that they require predictable computing power);
  • Tests for external dictionaries (should be moved to integration tests);
  • Jepsen like tests for quorum inserts (not yet available in opensource).

Tests extra:

  • Run functional tests with Valgrind.

Static analyzers:

  • CppCheck;
  • clang-tidy;
  • Coverity.