Commit Graph

1597 Commits

Author SHA1 Message Date
alexey-milovidov
fdff71a36f
Merge pull request #18770 from ClickHouse/fix-query-fuzzer
Query Fuzzer: fix some cultural issues
2021-01-06 20:02:30 +03:00
Alexey Milovidov
4f726784c9 Fix shellcheck 2021-01-06 04:56:10 +03:00
Alexey Milovidov
b6a733c2dd Do not allow Fuzzer to enable LLVM 2021-01-06 04:07:59 +03:00
Alexey Milovidov
822682f2f8 Fix exit code 2021-01-06 03:56:51 +03:00
Alexey Milovidov
7f074cb491 Remove half of strange code from query fuzzer 2021-01-06 00:20:39 +03:00
Alexey Milovidov
6ecf4fe921 More robust stateful test 2021-01-05 03:02:40 +03:00
Azat Khuzhin
de7909211d Try using top_level_domains from the patched performance archive
If there is top_level_domains list in the upstream/master, use from the
patched version (this is required to run all performance tests for
upstream/master in the PR).
2021-01-02 17:27:03 +03:00
Azat Khuzhin
469b631ca6 Fix performance comparison
In #18113 the top_level_domains list copying was moved into
docker/packager/binary/build.sh, this was done to avoid symlinks (since
Dockerfile cannot dereference them).

But the patch was wrong, since it copied into the root (/), which is not
included into performance.tgz and also compare.sh was not modified.

This wasn't showed up with CI checks since the docker image wasn't
updated and it still included that top_level_domains, once it was
modified the image was updated and it became broken.

Cc: @akuzm
2021-01-02 17:09:20 +03:00
Yatsishin Ilya
011d2b4631 disable couple checks for now 2020-12-29 17:57:12 +03:00
Yatsishin Ilya
a25fa1996b fix style checks 2020-12-29 17:57:12 +03:00
alesapin
e81485653b
Merge pull request #18491 from ClickHouse/add_nuraft
Add NuRaft to contrib
2020-12-26 16:19:05 +03:00
alesapin
b4cac6a8a4 Fix unbundled, splitted and disable freebsd build 2020-12-25 15:36:49 +03:00
alesapin
c9bbde649c
Merge pull request #18312 from ClickHouse/simplier_coverage
Make single image for building coverage report
2020-12-25 10:18:39 +03:00
Yatsishin Ilya
ce1e2b18a3 Fix codespell warnings. Split style checks. Update style checks docker 2020-12-24 13:17:52 +03:00
alesapin
e68bafd508 Merge branch 'master' into simplier_coverage 2020-12-24 12:39:33 +03:00
alesapin
58d0e8901e Remove hardcoded tests 2020-12-23 14:05:59 +03:00
Nikolai Kochetov
af7f5c9518
Merge pull request #17868 from ClickHouse/async-read-from-socket
Async read from socket
2020-12-23 12:20:42 +03:00
alesapin
f0b2bf9247 Merge branch 'master' into simplier_coverage 2020-12-22 18:37:15 +03:00
alesapin
4b28d6d030 Merge branch 'master' into enable_thread_fuzzer_for_flaky_check 2020-12-22 16:36:51 +03:00
alesapin
41229b8bc9 Run small tests 2020-12-22 14:52:48 +03:00
alesapin
b184141e20 Collect client coverage 2020-12-22 12:22:49 +03:00
alesapin
2d28c8cb60 Better runner 2020-12-22 11:20:57 +03:00
Alexey Milovidov
5bdf287910 Check for conflict markers in docs 2020-12-22 00:55:54 +03:00
alesapin
b10f13f34a More thread fuzzer 2020-12-21 23:24:16 +03:00
alesapin
8bbb678758 Correctly forward env 2020-12-21 23:12:01 +03:00
alesapin
8f2d4f2d1e Fix user 2020-12-21 18:46:36 +03:00
alesapin
24cb824bfe Simplify coverage images 2020-12-21 15:09:27 +03:00
alesapin
fdf18ceec0 Remove new line 2020-12-21 13:08:06 +03:00
alesapin
643739df6b Trying to enable thread fuzzer in flaky check 2020-12-21 13:06:36 +03:00
alexey-milovidov
3261392f16
Merge pull request #18208 from vzakaznikov/update_testflows_version_to_1.6.72
Updating TestFlows version to the latest 1.6.72
2020-12-18 18:03:00 +03:00
Ivan
57763e6867
Enable Pytest framework for stateless tests (#17902) 2020-12-18 16:26:11 +03:00
Vitaliy Zakaznikov
7db2652ab3 Updating TestFlows test version to the latest 1.6.72
Re-generating all requirements.py
2020-12-17 18:02:07 -05:00
alesapin
799997db35
Merge pull request #16621 from arenadata/ADQM-148
HDFS configuration and Kerberos Authenticaiton
2020-12-17 23:16:58 +03:00
Ilya Golshtein
832d99515b tag for yandex/clickhouse-kerberized-hadoop 2020-12-17 12:05:49 +03:00
Alexander Kuzmenkov
170cd132da
Merge pull request #18113 from azat/tld-list-symlink-fix
Avoid using symlinks for top_level_domains
2020-12-16 20:49:43 +03:00
Ilya Golshtein
a5a33f1963 rpm rebuilddb and yum update in kerberized_hadoop/Dockerfile 2020-12-16 14:48:33 +03:00
Ilya Golshtein
39877b1a6f yum clean all in kerberized_hadoop/Dockerfile 2020-12-16 14:48:33 +03:00
Ilya Golshtein
a4dad0acb7 CentOS 6 became too old 2020-12-16 14:48:33 +03:00
Ilya Golshtein
09d1e89674 yandex/clickhouse-kerberized-hadoop + missed header 2020-12-16 14:48:33 +03:00
Ilya Golshtein
7e97814859 cleanup, fixes, new submodules, ShellCommand, WriteBufferFromString 2020-12-16 14:48:33 +03:00
Ilya Golshtein
d9166a0645 HADOOP_SECURE_DN_USER way, kinit thread, docker capabilities 2020-12-16 14:48:33 +03:00
Ilya Golshtein
eb10948ca2 kerberized_hdfs cluster started 2020-12-16 14:48:33 +03:00
alesapin
86fc9ea4df Update base image to 20.04 2020-12-15 23:18:53 +03:00
Azat Khuzhin
95c35b318e Avoid using symlinks for top_level_domains
Move the main copy into the tests/config and ship it to the perf tests
via the perf package.
2020-12-15 21:58:46 +03:00
Nikolai Kochetov
4905201985 Merge branch 'master' into async-read-from-socket 2020-12-15 21:04:47 +03:00
alesapin
554fd738e9 Remove strange line 2020-12-15 18:22:21 +03:00
alesapin
ccabb4680d Fix dependencies 2020-12-15 18:19:56 +03:00
Alexander Kuzmenkov
fe6f477881 Build utils in CI, at least in split build
Currently we don't build them at all, so they were broken by https://github.com/ClickHouse/ClickHouse/pull/17988
2020-12-14 20:39:52 +03:00
Alexander Kuzmenkov
5f424f7c5c
Update zzz-perf-comparison-tweaks-config.xml 2020-12-14 20:08:32 +03:00
alexey-milovidov
19af691aa6
Merge pull request #17738 from vklimontovich/docker-cli-example
Docker cli example
2020-12-14 19:44:13 +03:00
filimonov
b08842d44e
Update README.md 2020-12-14 16:54:37 +01:00
filimonov
1436986a86
Update README.md 2020-12-14 16:53:58 +01:00
filimonov
bca41bb7fd
Update README.md 2020-12-14 16:50:48 +01:00
Alexander Kuzmenkov
e7f93b5563
Merge pull request #18058 from azat/perf-test-zk-port-fix
Remove test_keeper_server in perf tests
2020-12-14 17:54:47 +03:00
Alexander Kuzmenkov
2edabd5519
Update zzz-perf-comparison-tweaks-config.xml 2020-12-14 17:45:58 +03:00
Nikolai Kochetov
8de5cd5bc7 Merge branch 'master' into async-read-from-socket 2020-12-14 17:45:38 +03:00
Nikolai Kochetov
75ac87c241 Fixing build. 2020-12-14 17:42:12 +03:00
Azat Khuzhin
653ddab63e Remove test_keeper_server in perf tests
To avoid port overlaps for right and left server (anyway it is not
used).
2020-12-14 10:59:01 +03:00
alexey-milovidov
e7ffae89e9
Update build.sh 2020-12-14 02:31:35 +03:00
alexey-milovidov
52f03d942e
Update Dockerfile 2020-12-14 02:31:18 +03:00
Azat Khuzhin
6f2c559398 Convert top_level_domains_lists.xml into regular file
docker/test/performance-comparison/config/config.d is not allowed to has
symlinks, since it is copied with the COPY docker command.
2020-12-10 21:01:17 +03:00
Azat Khuzhin
0f1d9f3a6f Put top_level_domains into the docker/test/performance-comparison/config
Since in docker/test/performance-comparison there is Dockerfile that is
runned from that directory, so it cannot COPY anything outside of it.
2020-12-09 21:08:31 +03:00
Azat Khuzhin
8a4b29495b perf: override top_level_domains_path via command line arguments 2020-12-09 21:08:31 +03:00
Azat Khuzhin
352a1eea34 Use new top_level_domains_path 2020-12-09 21:08:31 +03:00
Azat Khuzhin
7b0c720922 Add top_level_domains into perf tests
And change Dockerfile to trigger image rebuild
2020-12-09 21:08:31 +03:00
Alexander Kuzmenkov
fa64d314df
Merge pull request #17907 from azat/perf-tiny-fix
Perf tests fixes
2020-12-09 12:29:57 +03:00
Azat Khuzhin
79e6348282 perf: load overrides last of all 2020-12-09 07:31:54 +03:00
Azat Khuzhin
d33b0ad33b Use interserver_http_port remove=remove over auto-port
Revert "perf: fix port conflict for tcp_with_proxy_port"

This reverts commit 89b3566824e5c0a87b6310fc4b6f9d300d2e17db.
2020-12-09 00:11:12 +03:00
Azat Khuzhin
75f9640fc2 perf: fix port conflict for tcp_with_proxy_port
Simply use zero port to do auto allocation, anyway it is not used in
perf tests.
2020-12-08 23:29:38 +03:00
alexey-milovidov
e127636a4b
Merge pull request #16787 from kitaisreal/fast-float-integration
FastFloat library integration
2020-12-08 21:34:34 +03:00
Azat Khuzhin
f1ac8775db perf: override config directives via command line over sed by config.xml 2020-12-08 21:25:03 +03:00
Alexey Milovidov
1ed329017b Merge branch 'master' into quota-by-x-forwarded-for 2020-12-07 18:22:19 +03:00
Maksim Kita
5b57c59a3b Updated fast_float integration 2020-12-06 23:37:55 +03:00
Maksim Kita
977da3d052 Removed ryu 2020-12-06 15:39:39 +03:00
Maksim Kita
64549702a2 DragonBox integration 2020-12-06 00:32:03 +03:00
Guillaume Tassery
4920bba7de Override value for running scripts on docker-entrypoint-initdb.d with CLICKHOUSE_INIT_TIMEOUT 2020-12-05 09:53:15 +07:00
alesapin
7c2a04d77d
Merge pull request #17671 from ClickHouse/update_integration_tests_image
Update docker image in integration tests runner
2020-12-04 15:51:08 +03:00
Ivan
315ff4f0d9
ANTLR4 Grammar for ClickHouse and new parser (#11298) 2020-12-04 05:15:44 +03:00
Alexey Milovidov
00dc21b0b9 Better tests 2020-12-03 13:02:20 +03:00
Vladimir Klimontovich
9bf5ab9f4c
Removed unnecessary mapping 2020-12-02 19:01:31 +03:00
Vladimir Klimontovich
274c87d4c7
Remove extra space 2020-12-02 19:00:23 +03:00
Vladimir Klimontovich
d402d68071
Added HTTP CLI example and expose 8123 port
Exposing the 8123 (HTTP) interface in basic example and giving CLI connection instruction makes it easier to start. For a lot of developers, curl is a native and very familiar tool
2020-12-02 18:26:16 +03:00
alesapin
031ddaedd4 Update docker image in integration tests runner 2020-12-01 14:32:33 +03:00
a.palagashvili
502355922d added errors checking, removed redundant variables 2020-11-29 15:14:42 +03:00
alesapin
1d179dac22
Merge pull request #17486 from ClickHouse/fix_more_flaky_tests
Fix more flaky integration tests
2020-11-28 10:43:29 +03:00
Alexander Kuzmenkov
8e855dc887
Merge pull request #17460 from azat/perf-tests-settings
Fail queries on unknown settings in perf tests
2020-11-27 22:42:37 +03:00
tavplubix
aaaa7ad59f
Merge pull request #16806 from TCeason/add_test_for_materialize_mysql
add network partition integration test for MaterializeMySQL
2020-11-27 20:39:06 +03:00
alesapin
239ed12458 Better defaults 2020-11-27 14:50:33 +03:00
Azat Khuzhin
3696916f44 perf: add setuptools into docker image (to built clikchouse-driver) 2020-11-27 07:20:41 +03:00
taichong
c8f7a56a25 fix CI timeout err 2020-11-27 11:34:49 +08:00
Azat Khuzhin
b390786a2e Cleanup fasttest/run.sh
- align functions
- align subshells ()
2020-11-27 00:38:49 +03:00
Azat Khuzhin
04a93f18e8 perf: fail on unknown settings 2020-11-27 00:16:41 +03:00
Azat Khuzhin
a591886740 perf: remove outdated comments from perf.py
Refs: https://github.com/ClickHouse/ClickHouse/pull/17255#discussion_r528822709
2020-11-27 00:16:41 +03:00
Azat Khuzhin
a38628d967 perf: use the latest available clickhouse-driver (from git)
Since later setting_is_important will be used, but there is no release
with this feature included.
2020-11-27 00:16:41 +03:00
Alexander Kuzmenkov
15a0f14445
Merge pull request #15419 from myrrc/improvement/diff-types-in-avg-weighted
Allow different types in avgWeighted. Allow avg and avgWeighed to operate on extended integral types.
2020-11-26 17:16:48 +03:00
Alexander Kuzmenkov
394b81ac46 Merge remote-tracking branch 'origin/master' into HEAD 2020-11-25 23:38:55 +03:00
nikitamikhaylov
f764332070 better merge 2020-11-25 20:51:06 +03:00
Vitaly Baranov
3856ae1a5e Fix PVS Error "Unable to start the analysis on this file" for files
which include protobuf-generates files.
2020-11-24 17:53:34 +03:00
Vitaly Baranov
8e6c68f4f7 Fix unbundled build. 2020-11-24 17:53:34 +03:00
myrrc
fbb0e6e6aa Merge remote-tracking branch 'upstream/master' into improvement/diff-types-in-avg-weighted 2020-11-24 16:04:17 +03:00
Alexander Kuzmenkov
a543c8e468 Add an option to use existing tables to perf.py 2020-11-23 21:05:54 +03:00
taichong
a6c2a90756 modify integration test materialize mysql database
1. fix asan test err

2. reset pymysql conn if ping err

3. finish mysql & ck killed test

4. modify check_query
2020-11-23 17:55:07 +08:00
robot-clickhouse
dc19fba36f Auto version update to [20.13.1.1] [54444] 2020-11-23 10:29:26 +03:00
alesapin
9c8b0da382
Merge pull request #16033 from nvartolomei/nv/parts-uuid
Add unique identifiers IMergeTreeDataPart structure
2020-11-22 16:13:19 +03:00
Azat Khuzhin
d676e2f5c5 Pass settings for perf tests via protocol over SET query
Thus they will be applied on reconnects, since clickhouse-driver may
implicitly do it if the connection has been failed.
2020-11-21 14:02:33 +03:00
Azat Khuzhin
3ddb0ac53e Configure logging for perf test runner 2020-11-21 14:02:33 +03:00
Azat Khuzhin
f6d16ee43e Force clickhouse-driver >= 1.1.5 for settings-as-strings support for perf-tests image 2020-11-21 14:02:33 +03:00
alesapin
78860bf2bd Improvements in coverage images 2020-11-20 11:48:06 +03:00
a.palagashvili
1d09a67e9f empty commit for force run 2020-11-19 23:06:41 +03:00
alesapin
a15b8afa73 Don't add tons of client coverage files in stateful tests with coverage 2020-11-19 19:07:36 +03:00
Nicolae Vartolomei
b6a330de77 Allow running subset of tests in fasttest 2020-11-19 13:18:03 +00:00
a.palagashvili
0c4acef558 added zstd to docker image 2020-11-19 12:23:05 +03:00
Vitaliy Zakaznikov
dbb5f6722e Moving to using TestFlows version 1.6.65
Fixing unstable test in tests/testflows/ldap/external_user_directory/tests/authentications.py
2020-11-17 20:22:43 -05:00
Nikolai Kochetov
86395e11a2
Merge pull request #17006 from azat/fix-distributed-queries-finish
Fix Distributed queries finishing (avoid connection resets)
2020-11-17 11:11:27 +03:00
Alexander Kuzmenkov
b0e660651a cleanup 2020-11-16 19:09:58 +03:00
alesapin
7b784fb925 Fix stateful with coverage test download script 2020-11-16 16:05:00 +03:00
Azat Khuzhin
ed64f2ad67 Add a test for Distributed query finish does not produce any NETWORK_ERROR 2020-11-14 18:35:15 +03:00
Alexander Kuzmenkov
7313691140 Merge remote-tracking branch 'origin/master' into HEAD 2020-11-12 20:31:17 +03:00
alexey-milovidov
ff29b2d2ee
Merge pull request #15073 from sundy-li/storage-rocksdb
StorageEmbeddedRocksDB
2020-11-12 13:17:58 +03:00
Alexander Kuzmenkov
91f88c79f9 Merge remote-tracking branch 'origin/master' into HEAD 2020-11-12 11:20:09 +03:00
zhang2014
b13189baec Try fix fasttest submodule clone 2020-11-12 12:45:08 +08:00
sundyli
5ef809931e
Merge branch 'master' into storage-rocksdb 2020-11-12 08:26:27 +08:00
a.palagashvili
55d05c95bf fixed style, xz check fasttest skipped, removed fast-lzma2 2020-11-11 15:34:28 +03:00
Alexander Kuzmenkov
f31d7869ac Merge remote-tracking branch 'origin/master' into HEAD 2020-11-11 14:51:35 +03:00
a.palagashvili
124ef2f5fa added and successfully passed tests for content encoding and file() function, formatted code 2020-11-11 04:50:56 +03:00
sundy-li
46c01eb7a2 improve unbundle build 2020-11-11 09:08:53 +08:00
sundy-li
fb7066d73d * fix fasttest and cmake && pipline for all_scan
* unique the keys
* add inputstream && outputstream
2020-11-11 09:08:53 +08:00
sundy-li
c7617173c4 add sync_file_range for glibc-compatibility 2020-11-11 09:08:52 +08:00
sundy-li
84dab52e45 skip rocksdb test in fasttest 2020-11-11 09:08:52 +08:00
Danila Kutenin
a489f65c02
Merge branch 'master' into master 2020-11-10 20:16:29 +03:00
alesapin
c0ae61e306 Fix stress test script 2020-11-10 10:33:31 +03:00
Danila Kutenin
5d138f3475 Fix submodules for fast test 2020-11-10 01:31:24 +03:00
Alexander Kuzmenkov
030419db90 Try 12 threads max in perf test.
This will allow us to use more machines, not only Gold 6320.
2020-11-09 19:14:05 +03:00
a.palagashvili
268f289bd1 resolved conflict 2020-11-07 16:36:38 +03:00
alexey-milovidov
4e85d6a4c3
Merge pull request #16273 from Avogar/collation-support
Support collate in LowCardinality, Nullable, Array and Tuple, where nested type is String
2020-11-06 21:52:52 +03:00
Alexander Kuzmenkov
dc665d7c43
Update compare.sh 2020-11-06 13:45:42 +03:00
alexey-milovidov
e1a01edf29
Merge pull request #16719 from MyroTk/master
RBAC Testflows - Server log intrumentation for debug and new ALTER tests
2020-11-06 12:40:31 +03:00
Alexander Kuzmenkov
ccf57300eb
Merge pull request #16620 from ClickHouse/aku/perf-upload
Upload perf test results to the CI database
2020-11-05 23:47:29 +03:00
Alexander Kuzmenkov
ac487ab26a
Update compare.sh 2020-11-05 23:45:58 +03:00
robot-clickhouse
f7b7854ace Auto version update to [20.12.1.1] [54443] 2020-11-05 21:52:50 +03:00
MyroTk
02a071e88f RBAC Testflows - Instrumenting server log for debug and new ALTER tests 2020-11-05 17:51:17 +01:00
Alexander Kuzmenkov
9e2d7aa274 Merge remote-tracking branch 'origin/master' into HEAD 2020-11-05 11:34:10 +03:00
Kruglov Pavel
c494f6f5ff
Merge branch 'master' into collation-support 2020-11-05 01:28:30 +03:00
alexey-milovidov
fa929b0312
Replace duplicate development config files with symlinks. (#16486)
* Fix test "max_memory_usage_for_user"

* Update test

* Update annoying boilerplate

* Support symlinks in tarballs

* Fix Fuzzer

* Remove "secondary" user

* Remove "secondary" user
2020-11-04 23:45:06 +03:00
a.palagashvili
ceda5cb9c5 fix codestyle, resolvec conflict 2020-11-04 19:47:04 +03:00
Alexander Kuzmenkov
8d5fe1148c add client config 2020-11-04 18:07:34 +03:00
Alexey Milovidov
5b9d48c715 Remove garbage from docs 2020-11-04 17:39:18 +03:00
Alexander Kuzmenkov
5cb708a50a fixup 2020-11-04 04:19:53 +03:00
alexey-milovidov
8a3f38f332
Merge branch 'master' into errorCode-v2 2020-11-04 01:40:41 +03:00
a.palagashvili
8ecf1d07c1 attempt to update fasttest version 2020-11-04 01:37:05 +03:00