thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-11-10 09:32:06 +00:00
Code Issues Packages Projects Releases Wiki Activity

Update tests.md

Browse Source
This commit is contained in:
alexey-milovidov 2019-05-19 03:09:40 +03:00 committed by GitHub
parent e73fc1bd0f
commit ffe3f6d106
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 8 additions and 13 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

21
docs/en/development/tests.md
View File

@ -53,8 +53,6 @@ Each test run one or miltiple queries (possibly with combinations of parameters)
If you want to improve performance of ClickHouse in some scenario, and if improvements can be observed on simple queries, it is highly recommended to write a performance test. It always makes sense to use `perf top` or other perf tools during your tests.
Performance tests are not run on per-commit basis. Results of performance tests are not collected and we compare them manually.
## Test Tools And Scripts
@ -184,19 +182,17 @@ We run functional and integration tests under ASan on per-commit basis.
**Valgrind (Memcheck)**.
We run functional tests under Valgrind overnight. It takes multiple hours. Currently there is one known false positive in `re2` library, see [this article](https://research.swtch.com/sparse).
**Undefined behaviour sanitizer.**
We run functional and integration tests under ASan on per-commit basis.
**Thread sanitizer**.
We run functional tests under TSan. ClickHouse must pass all tests. Run under TSan is not automated and performed only occasionally.
We run functional tests under TSan on per-commit basis. We still don't run integration tests under TSan on per-commit basis.
**Memory sanitizer**.
Currently we still don't use MSan.
**Undefined behaviour sanitizer.**
We still don't use UBSan on per commit basis. There are some places to fix.
**Debug allocator.**
You can enable debug version of `tcmalloc` with `DEBUG_TCMALLOC` CMake option. We run tests with debug allocator on per-commit basis.
You will find some additional details in `dbms/tests/instructions/sanitizers.txt`.
Debug version of `jemalloc` is used for debug build.
## Fuzzing
@ -213,7 +209,7 @@ People from Yandex Cloud department do some basic overview of ClickHouse capabil
## Static Analyzers
We use static analyzers only occasionally. We have evaluated `clang-tidy`, `Coverity`, `cppcheck`, `PVS-Studio`, `tscancode`. You will find instructions for usage in `dbms/tests/instructions/` directory. Also you can read [the article in russian](https://habr.com/company/yandex/blog/342018/).
We run `PVS-Studio` on per-commit basis. We have evaluated `clang-tidy`, `Coverity`, `cppcheck`, `PVS-Studio`, `tscancode`. You will find instructions for usage in `dbms/tests/instructions/` directory. Also you can read [the article in russian](https://habr.com/company/yandex/blog/342018/).
If you use `CLion` as an IDE, you can leverage some `clang-tidy` checks out of the box.
@ -250,12 +246,11 @@ As of July 2018 we don't track test coverage.
## Test Automation
We run tests with Yandex internal CI and job automation system named "Sandbox". We also continue to use Jenkins (available inside Yandex).
We run tests with Yandex internal CI and job automation system named "Sandbox".
Build jobs and tests are run in Sandbox on per commit basis. Resulting packages and test results are published in GitHub and can be downloaded by direct links. Artifacts are stored eternally. When you send a pull request on GitHub, we tag it as "can be tested" and our CI system will build ClickHouse packages (release, debug, with address sanitizer, etc) for you.
We don't use Travis CI due to the limit on time and computational power.
In Jenkins we run dictionary tests, Metrica B2B tests. We use Jenkins to prepare and publish releases. Jenkins is a legacy technology and all jobs will be moved to Sandbox.
We don't use Jenkins. It was used before and now we are happy we are not using Jenkins.
[Original article](https://clickhouse.yandex/docs/en/development/tests/) <!--hide-->