alesapin
d36f828b9c
Merge pull request #32910 from ClickHouse/performance_retry
...
More retries in perf tests.
2021-12-17 19:20:50 +03:00
alesapin
c2e8a73f14
Fix style
2021-12-17 16:43:58 +03:00
Mikhail f. Shiryaev
6f6b7c7bac
Merge pull request #32905 from ClickHouse/add_dockerhub_proxy
...
Add dockerhub proxy to CI
2021-12-17 14:31:50 +01:00
alesapin
1931726e15
Followup
2021-12-17 16:19:44 +03:00
alesapin
24480ac4de
More agressive retries in perf tests
2021-12-17 16:16:35 +03:00
alesapin
47c907d41a
Add dockerhub proxy
2021-12-17 13:20:48 +03:00
Vladimir C
fa011c66b7
Merge pull request #32846 from vdimir/flaky_test_s3_zero_copy_replication
2021-12-17 12:09:21 +03:00
vdimir
3f118f6734
Try fix flaky test: order for test_s3_zero_copy_replication
2021-12-17 12:08:39 +03:00
taiyang-li
d033fc4c24
merge master and fix conflict
2021-12-17 15:11:21 +08:00
alexey-milovidov
e834d655a5
Merge pull request #32415 from ClickHouse/arm64_workers
...
Add arm64/aarch64 tests
2021-12-17 08:46:52 +03:00
Mikhail f. Shiryaev
d2d1f5aba7
Optimize binary packager container
2021-12-16 13:00:31 +01:00
Mikhail f. Shiryaev
91083bea21
Fix building of packager containers
2021-12-16 12:42:49 +01:00
Mikhail f. Shiryaev
eb5a26891f
Add aarch64 deb package
2021-12-16 12:42:49 +01:00
alesapin
c9b03eaf64
Update entrypoint.sh
2021-12-15 22:28:21 +03:00
alesapin
6f2543b943
Better fix
2021-12-15 18:50:11 +03:00
alesapin
6799ce2162
Add retries to curl
2021-12-15 18:46:40 +03:00
Alexander Tokmakov
bd5931e5af
improve gdb script
2021-12-15 13:21:21 +03:00
tavplubix
2cf5456423
Merge pull request #32511 from ClickHouse/fix_attaching_gdb2
...
Try to fix attaching gdb 2
2021-12-14 21:10:00 +03:00
mergify[bot]
68c96f4dd4
Merge branch 'master' into add_performance_tests
2021-12-14 11:04:41 +00:00
taiyang-li
8234d1176f
merge master
2021-12-14 10:39:21 +08:00
Kseniia Sumarokova
78ed383f15
Merge pull request #31505 from ContentSquare/azure_blob_storage_dependency
...
Azure Blob Storage Disks
2021-12-13 18:28:48 +03:00
alesapin
3eb5e42972
Trying to allow bigger timeout for prewarm
2021-12-13 18:09:29 +03:00
alesapin
8e1fec39cd
Fix bash
2021-12-13 12:58:34 +03:00
alesapin
e2a2d1cdc3
Merge branch 'master' into add_performance_tests
2021-12-13 12:01:31 +03:00
alesapin
cf281b10dc
Split perf test into multiple checks
2021-12-13 11:59:58 +03:00
李扬
8675086104
Merge branch 'master' into hive_table
2021-12-12 09:01:46 -06:00
Azat Khuzhin
44f366fd12
Cleanup perf test runner
...
Use subshell to:
- avoid change/restore of TIMEFORMAT
- grepping out trace output, by using set +x in a subshell
- use array for options to avoid extra backslashes
2021-12-12 13:58:00 +03:00
alesapin
eb51d9550d
Merge branch 'master' into add_performance_tests
2021-12-12 01:01:13 +03:00
Azat Khuzhin
8a89b8e395
perf: do not fail in case of slow queries (to avoid hiding possible issues)
...
Do not stop processing pathologically slow queries,
since this may hide errors in other queries,
i.e. when the test failed on one of servers (upstream or from PR)
and someone interpret this is as OK with the following comment:
"the failure was on the upstream server it is OK, PR should fix it"
Anyway there is almost zero such queries right now, and before merging
something this should be reviewed.
2021-12-12 00:15:25 +03:00
Azat Khuzhin
4e4837758a
Remove readline support
...
- it was not nested for a long time
- replxx is an upstream way for completion
2021-12-11 11:29:59 +03:00
Kseniia Sumarokova
7e0bc290da
Merge pull request #32460 from kssenii/mongo-extend-auth
...
Allow empty credentials for mongo engine
2021-12-11 10:42:15 +03:00
Jakub Kuklis
e58f74d8dc
Connecting to Azurite and invoking simple INSERTs works in the integration test
2021-12-10 22:04:38 +01:00
Jakub Kuklis
abd685e14c
Installing azure-storage-blob in the integration tests docker, raising exception when not connected to Azurite, changing ports in the Azurite docker
2021-12-10 22:04:38 +01:00
Jakub Kuklis
910db6ca57
Code cleanup addressing PR remarks
2021-12-10 22:04:38 +01:00
Jakub Kuklis
850c9a7e7a
Reverting submodule changes to fast test docker and disabling Blob Storage support when files missing
2021-12-10 22:04:38 +01:00
Jakub Kuklis
37a23b402b
Add missing dependencies for azure for fast test
2021-12-10 22:04:38 +01:00
Jakub Kuklis
024a3b4626
Fixing submodule update for clickhouse compilation in the fast test
2021-12-10 22:04:38 +01:00
Jakub Kuklis
639d8a2b38
Running Azurite docker in the integration test for Blob Storage support
2021-12-10 22:04:37 +01:00
Alexander Tokmakov
dfa31a6577
fix
2021-12-10 22:38:34 +03:00
alesapin
5a542516e6
Merge pull request #32515 from ClickHouse/revert-32514-revert-32496-smaller_checks
...
Revert "Revert "Split long tests into multiple checks""
2021-12-10 22:30:04 +03:00
Kseniia Sumarokova
2cd5a15193
Update docker_compose_mongo.yml
2021-12-10 22:02:28 +03:00
tavplubix
5079134ffd
Update run.sh
2021-12-10 20:10:49 +03:00
Alexander Tokmakov
254d6c5d59
fix
2021-12-10 20:07:28 +03:00
alesapin
84bccb83bd
Trying to add debug info
2021-12-10 19:38:20 +03:00
alesapin
a6e55b00e3
Revert "Revert "Split long tests into multiple checks""
2021-12-10 18:39:02 +03:00
alesapin
dcbba460c1
Revert "Split long tests into multiple checks"
2021-12-10 18:38:42 +03:00
Alexander Tokmakov
3ff8aff1b5
try to fix attaching gdb
2021-12-10 18:03:57 +03:00
kssenii
46356a34a3
Update pymongo
2021-12-10 14:58:17 +00:00
kssenii
b6b73ba5b2
pymongo
2021-12-10 14:54:55 +00:00
alesapin
a2aa0bc6cc
Merge pull request #32496 from ClickHouse/smaller_checks
...
Split long tests into multiple checks
2021-12-10 17:42:14 +03:00
alesapin
f655b43f5a
Merge branch 'master' into add_performance_tests
2021-12-10 12:32:43 +03:00
alesapin
4c1babee05
Split long functional tests to multiple checks
2021-12-10 12:07:24 +03:00
mergify[bot]
a07061ac84
Merge branch 'master' into update-mongo-contrb
2021-12-10 08:57:16 +00:00
alexey-milovidov
6ce2559078
Merge branch 'master' into feature/dotnet-integration-test
2021-12-10 01:25:47 +03:00
tavplubix
85e53b1b1f
Try fix attaching gdb in tests ( #32448 )
...
* attach gdb with sudo
* fix
* Update run.sh
2021-12-10 00:12:45 +03:00
Kseniia Sumarokova
f2dfa89bc9
Update docker_compose_mongo.yml
2021-12-09 23:58:46 +03:00
kssenii
f5a77fca39
Done
2021-12-09 14:44:25 +00:00
Kruglov Pavel
ef42cd6821
Merge pull request #32138 from Avogar/fix-apply-lambda
...
Fix 'APPLY lambda' parsing
2021-12-09 17:09:11 +03:00
alesapin
89ac8da76e
Fix build path
2021-12-09 16:02:29 +03:00
tavplubix
f4cbbfd8c0
Update run-fuzzer.sh
2021-12-09 14:17:55 +03:00
tavplubix
de0a9a84a9
Update run-fuzzer.sh
2021-12-09 13:51:39 +03:00
Nikita Mikhaylov
dbf5091016
Parallel reading from replicas ( #29279 )
2021-12-09 13:39:28 +03:00
Kruglov Pavel
1df2383d00
Update run-fuzzer.sh
2021-12-09 12:17:31 +03:00
alesapin
b62de9b2f4
Merge branch 'master' into add_performance_tests
2021-12-09 11:40:32 +03:00
robot-clickhouse
f2ccddb7be
Auto version update to [21.13.1.1] [54458]
2021-12-09 00:33:01 +03:00
kssenii
b6c1d1e72e
Use latest version for tests
2021-12-08 21:09:11 +00:00
avogar
51bb37715f
Fix fuzzer
2021-12-08 21:29:50 +03:00
Nikita Mikhaylov
d3d4524390
Merge branch 'master' of github.com:ClickHouse/ClickHouse into get_fuzz_data
2021-12-07 10:32:03 +00:00
tavplubix
523e23cfcd
Merge pull request #32164 from azat/stress-ignore-mem-limit
...
Fix stress tests
2021-12-07 13:10:17 +03:00
Nikita Mikhaylov
80a146816c
More comments and style
2021-12-06 21:34:52 +00:00
Nikita Mikhaylov
fa22fdc097
Merge branch 'master' of github.com:ClickHouse/ClickHouse into get_fuzz_data
2021-12-06 14:30:49 +00:00
Azat Khuzhin
f8bf3b1993
stress: use max_memory_usage_for_user as a soft limit to avoid memory limit exceeded
...
By using max_memory_usage_for_user as a soft limit, and after
max_server_memory_usage as a hard, we can allow normal overcommit, using
max_memory_usage_for_user=0 instead of relying on max_untracked_memory.
2021-12-06 09:05:34 +03:00
Azat Khuzhin
f9a95fce13
stress: allow memory overcommit for SELECT 1 query
2021-12-06 09:02:56 +03:00
Azat Khuzhin
1167313961
stress: allow memory overcommit for one more query
2021-12-06 09:02:56 +03:00
Azat Khuzhin
7063ea3ee2
stress: add make_query_command() helper
2021-12-06 09:02:56 +03:00
Azat Khuzhin
adec690166
stress: SYSTEM DROP MARK CACHE
2021-12-06 09:02:56 +03:00
Azat Khuzhin
d3a7aed005
stress: tune memory_profiler_step too
2021-12-06 09:02:56 +03:00
Azat Khuzhin
169941c5d0
stress: allow 100MiB of memory to overcommit for SHOW/DROP DATABASE
...
CI: https://s3.amazonaws.com/clickhouse-test-reports/32019/b6290ae00223fc91d514b82de25e195ab34f8bca/stress_test__thread__actions_.html
2021-12-06 09:02:56 +03:00
Azat Khuzhin
445b0983ea
stress: ignore server memory limits for hung check
...
In [1] hung check failed because the server was under memory pressure.
[1]: https://s3.amazonaws.com/clickhouse-test-reports/32019/b6290ae00223fc91d514b82de25e195ab34f8bca/stress_test__undefined__actions_.html
2021-12-06 09:02:56 +03:00
Azat Khuzhin
3e96b28843
perf: convert killall to pkill (since killall has some magic, see -e option)
2021-12-03 10:56:42 +03:00
Azat Khuzhin
fb0cc62509
perf: fix waiting of the server after running tests
...
killall requires strict match, i.e. "clickhouse-server" not
"clickhouse":
2021-12-03 05:24:56 + env kill -- -21700
2021-12-03 05:24:56 kill: (-21700): No such process
2021-12-03 05:24:56 + killall clickhouse
2021-12-03 05:24:56 clickhouse: no process found
2021-12-03 05:24:56 + echo Servers stopped.
2021-12-03 05:24:56 Servers stopped.
2021-12-03 05:24:56 + analyze_queries
$ tail -n1 *-server-log.log
==> left-server-log.log <==
2021.12.03 05:26:59.530647 [ 450 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1668052
==> right-server-log.log <==
2021.12.03 05:27:20.873136 [ 466 ] {} <Trace> SystemLog (system.metric_log): Flushed system log up to offset 9605
==> setup-server-log.log <==
2021.12.03 02:47:14.844395 [ 96 ] {} <Information> Application: Child process exited normally with code 0.
As you can see killall instantly fails with no such process, while this
cannot be true since it was there, and also according to logs there were
messages after running analyze_queries() from compare.sh
This should fix problems like in [1].
[1]: https://clickhouse-test-reports.s3.yandex.net/32080/344298f4037f88b114b8e798bb30036b24be8f16/performance_comparison/report.html#fail1
2021-12-03 10:55:35 +03:00
mergify[bot]
9c016d4e25
Merge branch 'master' into stress-test
2021-12-02 12:02:44 +00:00
taiyang-li
2f4e7e1d4e
merge master
2021-12-02 19:48:21 +08:00
Azat Khuzhin
e976cd8691
Provide clickhouse binary w/o debug symbols (stripped) in fasttest
...
- regular binary: 1.8G
- stripped: 343M (5x less)
- gzipped stripped: 92M (20x less)
This can be useful to check something fast.
2021-12-01 21:50:03 +03:00
mergify[bot]
b4f43c59a2
Merge branch 'master' into stress-test
2021-12-01 10:12:22 +00:00
alesapin
b4d4295c16
Followup
2021-11-30 13:24:04 +03:00
alesapin
085a30b98e
Fixup
2021-11-30 13:22:50 +03:00
alesapin
bccd927aec
Make stress tests slightly more stable
2021-11-30 13:19:03 +03:00
Raúl Marín
6a04dcb492
Remove libc-headers
2021-11-29 19:55:57 +01:00
Mikhail f. Shiryaev
e97e172dd8
Merge pull request #31822 from ClickHouse/rollback-28016
...
Rollback changes from #28016 : ru.archive.ubuntu.com
2021-11-29 10:27:59 +01:00
mergify[bot]
e6f5792ba3
Merge branch 'master' into stress-test
2021-11-29 09:01:33 +00:00
Alexey Milovidov
0f465a971e
Remove some old options
2021-11-27 03:36:29 +03:00
mergify[bot]
ae0b5df601
Merge branch 'master' into stress-test
2021-11-26 12:48:03 +00:00
Mikhail f. Shiryaev
5408d6232f
Make archive.ubuntu.com mirror configurable for build
2021-11-26 10:55:41 +01:00
alesapin
3562264404
Merge pull request #31803 from ClickHouse/faster_tests
...
Shorter stress and fuzzer tests
2021-11-26 10:17:26 +03:00
taiyang-li
d35e2a1c83
Merge branch 'master' into hive_table
2021-11-26 11:44:50 +08:00
alexey-milovidov
6b8f7ad03f
Merge pull request #31798 from ClickHouse/fix-31748
...
Fix stylecheck for tests/ci/push_to_artifactory.py
2021-11-26 00:11:32 +03:00
alexey-milovidov
6017b982d9
Merge pull request #31780 from Algunenano/fix_performance_report_queries
...
Improve how queries are output in the performance dashboard
2021-11-25 22:57:29 +03:00
alesapin
71ae890c59
Merge pull request #31544 from azat/ci-actions-fixes
...
Remove duplicated not compressed logs from CI (actions)
2021-11-25 18:43:44 +03:00
alesapin
5815429059
Shorter stress and fuzzer tests
2021-11-25 17:54:50 +03:00
Mikhail f. Shiryaev
16703c2824
Fix stylecheck for tests/ci/push_to_artifactory.py
2021-11-25 14:41:08 +01:00
alexey-milovidov
e8b29cab6e
Update report.py
2021-11-25 15:37:00 +03:00
mergify[bot]
ce0c112586
Merge branch 'master' into stress-test
2021-11-25 11:09:26 +00:00
Raúl Marín
a6c34fdbc9
Improve how queries are output in the performance dashboard
2021-11-25 11:22:47 +01:00
taiyang-li
72f60cceb9
Merge branch 'master' into hive_table
2021-11-25 17:33:26 +08:00
alesapin
c86b43c7ef
Merge pull request #31685 from ClickHouse/tavplubix-patch-5
...
Try fix OOMs with TSAN
2021-11-25 11:34:05 +03:00
alesapin
58fad90351
Merge branch 'master' into add_performance_tests
2021-11-25 10:53:10 +03:00
alesapin
cf2f019caa
Disable full run in perf tests
2021-11-24 23:05:27 +03:00
Alexey Milovidov
21f112b30e
Merge branch 'better-crosscompiling-freebsd' into hermetic-builds
2021-11-24 22:43:30 +03:00
Alexey Milovidov
92031e6b31
Fix PVS-Studio
2021-11-24 20:25:11 +03:00
tavplubix
82e4f941a4
Update Dockerfile
2021-11-24 16:00:15 +03:00
tavplubix
556ccd0bc9
Update Dockerfile
2021-11-24 12:47:44 +03:00
taiyang-li
89dcef69d5
merge master
2021-11-24 14:38:04 +08:00
Alexey Milovidov
51ee8fdbb0
More easy cross-compiling for FreeBSD
2021-11-24 00:32:04 +03:00
alexey-milovidov
61454ffbcb
Merge branch 'master' into hermetic-builds
2021-11-24 00:25:16 +03:00
alesapin
42c77323e8
Merge branch 'master' into add_performance_tests
2021-11-23 22:14:32 +03:00
Kruglov Pavel
18e60919c5
Update run.sh
2021-11-23 16:49:59 +03:00
alexey-milovidov
341f705419
Merge pull request #30599 from ClickHouse/zlib-ng-own-cmake
...
Use our own CMakeLists for `zlib-ng`, `cassandra`, `mariadb-connector-c` and `xz`, `re2`, `sentry`, `gsasl`, `arrow`, `protobuf`.
2021-11-23 13:33:29 +03:00
alesapin
2396f73e08
Fix performance tests comparison
2021-11-23 11:34:10 +03:00
alesapin
62b2eb6694
Merge branch 'master' into add_performance_tests
2021-11-23 11:26:45 +03:00
lgbo
996d7125c0
Merge branch 'master' into hive_table
2021-11-23 10:19:02 +08:00
Constantine Peresypkin
fd551ad94c
fix kerberized_hadoop image
...
current image uses non-existing centos6 repos
added direct fetch of actual rpm packages
fixed a typo in docker/images.json: duplicate entry
2021-11-22 17:32:19 +02:00
taiyang-li
d22854e82b
add compile swiych USE_HIVE
2021-11-22 10:52:10 +08:00
Alexey Milovidov
d604cf5573
Remove the infamous "unbundled" build
2021-11-21 13:58:26 +03:00
Azat Khuzhin
d0ecdb617e
Remove duplicated not compressed logs from CI (actions)
...
Right now we cannot remove that compression since non-github actions
check will upload non compressed files then, but once old CI will be
removed it seems to be possible to remove that manual compression.
2021-11-19 21:33:48 +03:00
alesapin
9e440988cf
Add datasets
2021-11-19 16:10:49 +03:00
taiyang-li
ab64a17104
fix fasttest
2021-11-19 16:20:14 +08:00
taiyang-li
e8644807fe
merge master and solve conflict
2021-11-19 15:01:58 +08:00
alesapin
130c09f348
Fix search for old version
2021-11-18 13:38:59 +03:00
lgbo-ustc
9902ccefc5
Add integration tests for hive query
2021-11-18 16:17:49 +08:00
alesapin
d0ec0c5dc8
Merge branch 'master' into add_performance_tests
2021-11-18 10:17:20 +03:00
alesapin
b74f9e7047
Merge branch 'master' into add_performance_tests
2021-11-18 10:12:46 +03:00
Nikolay Degterinsky
fc9ef14a73
Merge pull request #29219 from evillique/log_lz4_streaming
...
Add LZ4 stream compression of logs
2021-11-17 21:22:57 +03:00
Nikolay Degterinsky
eafd3ddfac
Add lz4 stream compression
2021-11-16 22:30:29 +00:00
tavplubix
b1079ae309
Update run.sh
2021-11-16 20:03:50 +03:00
mergify[bot]
923874060a
Merge branch 'master' into stress-test
2021-11-16 16:42:50 +00:00
tavplubix
224a4c1b6b
Update run.sh
2021-11-16 17:45:37 +03:00
alesapin
7f09941c72
Add performance comparison check
2021-11-16 14:16:10 +03:00
Kruglov Pavel
6180aedab6
Update download_previous_release
2021-11-15 23:16:35 +03:00
Kruglov Pavel
87df3c8272
Fix
2021-11-15 15:03:48 +03:00
alexey-milovidov
f4fda976ef
Merge pull request #31063 from azat/perf-spikes
...
perf: switch *_log tables to Memory engine (attempt to reduce cache misses)
2021-11-14 05:13:12 +03:00
Azat Khuzhin
48b1b7b7ec
Fix test_kafka_insert_avro by pinning avro version
...
New avro version has a code that does not allow to use BytesIO [1].
[1]: https://github.com/apache/avro/commit/e8c61ba4f0#diff-ca721cc66db80ff81dcd7b8ad1bb0e83b2e5c2de80c207ebec79f5495d38db98R318-R319
2021-11-13 12:01:13 +03:00
Azat Khuzhin
40be97c1dd
Improve fuzzer report in case of fuzzer killed
...
This will avoid Exception from fuzzer in description like in [1], that
someone may think that is related to the failure, which is obviously
don't in case it was KILLed.
[1]: https://clickhouse-test-reports.s3.yandex.net/31259/a49e10c78860a451ff9cc39df6049c88234d507a/fuzzer_ubsan/report.html#fail1
2021-11-12 01:01:10 +03:00
mergify[bot]
045bd40070
Merge branch 'master' into perf-spikes
2021-11-11 07:07:28 +00:00
taiyang-li
af78116c3f
Merge branch 'master' into hive_table
2021-11-11 11:52:28 +08:00
alesapin
0b28474f09
Merge pull request #31244 from nikitamikhaylov/disable-fuzz-builds
...
Disable fuzzer builds in CI
2021-11-10 16:42:08 +03:00
Nikita Mikhaylov
99f78472bc
Disabled
2021-11-10 11:38:26 +00:00
Kseniia Sumarokova
5f9b35b982
Update docker_compose_postgres.yml
2021-11-10 10:36:59 +03:00
taiyang-li
26988ab757
Merge branch 'master' into hive_table
2021-11-10 11:01:03 +08:00
Kruglov Pavel
08af98b4a1
Update download_previous_release
2021-11-09 16:34:13 +03:00
Kseniia Sumarokova
5eef7e0e45
Update docker_compose_postgres.yml
2021-11-09 15:13:01 +03:00
taiyang-li
5dbbe4b9c5
fix build error
2021-11-09 13:14:53 +08:00
Kruglov Pavel
03f1e3c67d
Update run.sh
2021-11-08 14:02:36 +03:00
Azat Khuzhin
9e622b5add
perf: switch *_log tables to Memory engine (attempt to reduce cache misses)
...
trace_log/query_log from performance tests shows (for
cases when prewarm query fails with timeout, 15sec) excessive
writeTraceInfo() in trace_log and QueryProfilerRuns in query_log, but
this is not the root cause of the timeout, but consequence.
Also query_log shows that on failures the following profile events has
significantly higher values:
- PerfLocalMemoryMisses (6.3x more)
- PerfLocalMemoryReferences (7x more)
- PerfDataTLBMisses (6.9x more)
- PerfInstructionTLBMisses (6.4x more)
During looking at performance tests logs I noticed that once the prewarm
query fails other server (left/right) was merging (MergeTree) something
in *_log tables.
But, using MergeTree for *_log in performance tests is useless, since
anyway environment for performance tests uses ramdrive.
And so MergeTree merges just increase overhead.
Eventually I expect that this should decrease extra memory referencing
and so this should decrease cache/TLB misses.
CI: https://clickhouse-test-reports.s3.yandex.net/30886/c504e0c08df7a926bb479a1d297f326f5c48a32f/performance_comparison/report.html#fail1
v2: <partition_by remove="remove"/>
2021-11-04 09:26:46 +03:00
mergify[bot]
8ef9e61e36
Merge branch 'master' into stress-test
2021-11-03 08:28:43 +00:00
Azat Khuzhin
ff19d22305
perf: remove over *_log tables that does not exported into artifacts
2021-11-03 08:29:18 +03:00
Azat Khuzhin
837fe8554f
perf: remove zookeeper to avoid configuring DDLWorker
2021-11-03 08:29:18 +03:00
Azat Khuzhin
acef85b17d
perf: cleanup server overrides
2021-11-03 08:29:18 +03:00
Azat Khuzhin
afd960db4e
perf: remove redundant query profiler set queries
2021-11-03 08:09:43 +03:00
robot-clickhouse
4289336c27
Auto version update to [21.12.1.1] [54457]
2021-11-02 00:56:45 +03:00
Kruglov Pavel
9a1275cb10
Merge pull request #30178 from Avogar/tsv-csv
...
Refactor and improve TSV, CSV, JSONCompactEachRow, RowBinary formats. Fix bugs in formats
2021-11-02 00:38:30 +03:00
alesapin
dbfa17aabf
Add correct S3
2021-11-01 13:32:56 +03:00
avogar
4e63311bb6
Try fix perf test
2021-11-01 12:13:29 +03:00
alesapin
54e404de32
Add github to known hosts in docs release
2021-11-01 10:33:52 +03:00
alesapin
ed3d707513
Merge pull request #30839 from ClickHouse/qoega-docker-docs
...
Docs check on github actions
2021-10-31 20:29:04 +03:00
alexey-milovidov
15b9d65221
Merge pull request #30662 from azat/conf-clickhouse
...
Switch everything left from `<yandex>` to `<clickhouse>`
2021-10-31 14:56:51 +03:00
alesapin
d872eae809
Better
2021-10-29 21:43:03 +03:00
alesapin
4c645f3ef6
Add docs release
2021-10-29 21:43:03 +03:00
alesapin
1a13a741f9
Fixes in docs
2021-10-29 21:43:03 +03:00
alesapin
6c8ba21c52
Add docs check image
2021-10-29 21:43:03 +03:00
alesapin
11f71fffe1
Tabs to spaces
2021-10-29 21:43:03 +03:00
alesapin
afcadc95a7
Add docs check to documentation
2021-10-29 21:43:03 +03:00
Yatsishin Ilya
faeb4ef869
Add docker container to build docs
2021-10-29 21:43:03 +03:00
tavplubix
059c1ebf36
Merge pull request #30847 from ClickHouse/fix_30802
...
Disable check for ZooKeeper session uptime by default
2021-10-29 20:18:25 +03:00
Alexander Tokmakov
d095cfe4c1
disable check for zk session uptime by default
2021-10-29 14:45:09 +03:00
alesapin
4eca493ab8
Fix fuzzer on master
2021-10-29 13:47:29 +03:00
alesapin
11b70a285c
Merge pull request #30795 from ClickHouse/make_fuzzer_check_less_hardcoded
...
Trying to make fuzzer check less hardcoded
2021-10-29 10:21:36 +03:00
alesapin
ddac7a51e2
Merge pull request #30729 from ClickHouse/trying_actions
...
Add functional tests to github actions
2021-10-29 09:49:31 +03:00
mergify[bot]
65df198e9b
Merge branch 'master' into conf-clickhouse
2021-10-28 22:19:42 +00:00
alesapin
ffc8dbd085
Update run-fuzzer.sh
2021-10-28 23:06:04 +03:00
alesapin
313584e553
Update run-fuzzer.sh
2021-10-28 19:12:47 +03:00
mergify[bot]
a94b8902dd
Merge branch 'master' into trying_actions
2021-10-28 15:57:28 +00:00
alesapin
0651a1f0e9
Update run-fuzzer.sh
2021-10-28 17:03:40 +03:00
alesapin
542196128b
Trying to make fuzzer check less hardcoded
2021-10-28 12:29:50 +03:00
alesapin
c0b2f92123
Add stateful test
2021-10-28 12:26:59 +03:00
Azat Khuzhin
9eef2ad0c2
perf: reduce number of times the query will run
...
Before this patch:
- upstream/master and PR's *with* perf tests or pef scripts changes:
--runs=13 --max-queries=0
- PRs *without* perf changes:
--runs=7 --max-queries=20
- PRs w/ only perf tests changes:
--runs-13 --max-queries=0 <list of perf tests>
After:
- upstream/master and PR's *with* perf tests changes:
--runs=13 --max-queries=0
- PRs *without* perf changes:
--runs=7 --max-queries=10
- PRs w/ only perf tests changes:
--runs-13 --max-queries=0 <list of perf tests>
So to underline, now we will not look at perf scripts changes anymore,
and we will also decrease number of random queries to run to 10.
2021-10-27 23:20:51 +03:00
Azat Khuzhin
7957079f95
perf: disable query/memory profiling for non prewarm queries
2021-10-27 23:20:51 +03:00
Azat Khuzhin
190459db33
perf: add queries with errors during process too to the perf artifacts
...
Sometimes queries fails with timeout, and you will not get any perf
metrics for it before this patch.
Example: https://clickhouse-test-reports.s3.yandex.net/30611/5ff6c5536558821824d5fdf25a75729e5b82060d/performance_comparison/report.html#partial-queries.flat_dictionary.5
2021-10-27 23:20:51 +03:00
Azat Khuzhin
722d528218
perf: pin clickhouse-driver to 0.2.1 for tzlocal <0.3 to avoid warning
...
CI report [1]:
/usr/local/lib/python3.6/dist-packages/clickhouse_driver/columns/datetimecolumn.py:199: PytzUsageWarning: The zone attribute is specific to pytz's interface; please migrate to a new time zone provider. For more details on how to do so, see https://pytz-deprecation-shim.readthedocs.io/en/latest/migration.html
[1]: https://clickhouse-test-reports.s3.yandex.net/30626/dfc85841134aa96b4e04c401462898eb305e8657/performance_comparison/report.html#changes-in-performance.format_date_time.1
2021-10-27 23:20:51 +03:00
mergify[bot]
d9d9d38e4f
Merge branch 'master' into stress-test
2021-10-27 19:00:31 +00:00
Azat Khuzhin
90ea2de9bb
Add unidiff into style image
2021-10-27 10:03:24 +03:00
Azat Khuzhin
b41c77f4f8
Add boto3 to style image
2021-10-27 10:03:24 +03:00
Azat Khuzhin
944437caf6
Add PyGithub into style image
2021-10-27 10:03:23 +03:00
Nikita Mikhaylov
a4100cc980
Build local in CI in fuzz mode
2021-10-26 17:45:41 +00:00
Nikita Mikhaylov
aae30a9e37
Fix fuzzer build ( #30344 )
2021-10-26 20:30:34 +03:00
Azat Khuzhin
da90df5658
docker: switch <yandex> to <clickhouse>
2021-10-26 08:53:14 +03:00
Azat Khuzhin
8f015764a7
Add metadata for *_log into tests artifacts
...
This will allow to use artifacts as-is, i.e.:
clickhouse-local --path /path/to/log/artifact
v2: convert database to Ordinary to avoid extra symlinks dependencies.
2021-10-23 20:17:01 +03:00
mergify[bot]
943a1cbba0
Merge branch 'master' into stress-test
2021-10-21 04:41:23 +00:00
alesapin
9e8dfdb089
Merge pull request #28851 from ClickHouse/trying_actions
...
Trying github actions
2021-10-20 12:55:53 +03:00
alesapin
6d8853698b
Merge branch 'master' into trying_actions
2021-10-19 17:45:37 +03:00
vdimir
acf416900d
Minor fix in clickhouse/kerberized-hadoop Dockerfile
2021-10-19 16:45:03 +03:00
vdimir
1d78f1c63b
Fix ca-bundle.crt in clickhouse/kerberized-hadoop
2021-10-19 16:40:06 +03:00
vdimir
36bb4033ba
Whitespace change in kerberized_hadoop/Dockerfile
2021-10-19 12:44:55 +03:00
Alexey Milovidov
2cef9983fa
Remove recursive submodules
2021-10-18 02:29:04 +03:00
Alexey Milovidov
f02b61e30f
Merge branch 'master' into hermetic-builds
2021-10-17 07:02:37 +03:00
mergify[bot]
f29ea07b1c
Merge branch 'master' into clickhouse-test-http-interface
2021-10-13 16:46:26 +00:00
tavplubix
a16ce6b54b
Merge pull request #30030 from ClickHouse/tavplubix-patch-3
...
Attach trace_log to stateless tests report as is
2021-10-13 13:21:39 +03:00
Alexey Milovidov
b420bc19fb
Merge branch 'master' into hermetic-builds
2021-10-13 01:08:23 +03:00
Dmitry Novik
fa9cdd5c5f
Use original whitespaces in test_results.tsv
2021-10-12 23:31:31 +03:00
Dmitry Novik
0507391010
Do not stop CI tests processing if clickhouse-test return 1
2021-10-12 23:30:07 +03:00
Dmitry Novik
a30573fc0a
Add FAIL message to test_results.tsv
2021-10-12 23:29:15 +03:00
Azat Khuzhin
071a5af96e
Revert "docker: add pandas/clickhouse_driver into test images"
...
This reverts commit e07a6f3fc0
.
2021-10-12 21:06:26 +03:00
alexey-milovidov
9a1f930b2f
Merge pull request #30010 from ClickHouse/ppc64le
...
Add cross-build for PowerPC 64le
2021-10-12 16:17:41 +03:00
tavplubix
f3c9f4be6c
Update run.sh
2021-10-12 14:38:41 +03:00
alesapin
2b3a0c892d
Merge pull request #29992 from ClickHouse/more_timeouts_in_scripts
...
More timeouts in test scripts
2021-10-12 10:14:21 +03:00
Alexey Milovidov
bdd06279a4
Fix PVS Studio
2021-10-12 05:43:53 +03:00
Alexey Milovidov
d0f7a10c6a
Fix fasttest
2021-10-12 03:34:07 +03:00
Alexey Milovidov
75547e64a5
Add to packager
2021-10-12 02:10:58 +03:00
alesapin
3ae960e04b
Review fixes
2021-10-11 16:40:12 +03:00
alesapin
9ad919d91a
More timeouts in stress test
2021-10-11 13:25:54 +03:00
Alexey Milovidov
0dbe7c4036
Remove old toolchain
2021-10-11 07:02:19 +03:00
alexey-milovidov
268c155b7d
Merge pull request #29856 from azat/clickhouse-test-python-client
...
Rewrite clickhouse-test to use python clickhouse_driver
2021-10-09 17:58:40 +03:00
alexey-milovidov
f6a0983068
Merge pull request #29781 from azat/fuzzer-server-pid-v2
...
Simplify obtaining of server pid in fuzzer tests
2021-10-08 00:35:25 +03:00
Azat Khuzhin
e07a6f3fc0
docker: add pandas/clickhouse_driver into test images
2021-10-08 00:09:37 +03:00
Kruglov Pavel
fd7d1131ec
Update run.sh
2021-10-06 13:17:48 +03:00
mergify[bot]
6eec0d76a7
Merge branch 'master' into update-minio
2021-10-06 05:45:18 +00:00
Azat Khuzhin
aeb670dbe0
Simplify obtaining of server pid in fuzzer tests
2021-10-05 21:05:49 +03:00
Nikita Mikhaylov
a880cce0de
better
2021-10-05 13:26:29 +00:00
mergify[bot]
0910a8ec74
Merge branch 'master' into stress-test
2021-10-05 09:46:18 +00:00
Kseniia Sumarokova
ead99b5911
Update docker_compose_minio.yml
2021-10-05 11:39:08 +03:00
kssenii
4962677a8b
Update minio
2021-10-05 08:18:16 +00:00
Nikita Mikhaylov
731d6c5755
Merge pull request #29513 from azat/fuzzer-fix-server-pid
...
Fix server pid (hence exit code and attaching with gdb) in fuzzer tests
2021-10-04 19:59:24 +03:00
Kruglov Pavel
31302afe81
Update run.sh
2021-10-04 16:35:50 +03:00
avogar
821daa21da
Update run.sh
2021-10-04 16:35:50 +03:00
Pavel Kruglov
6fc043026e
Update run.sh
2021-10-04 16:35:50 +03:00
Pavel Kruglov
2db11bc6c5
Add backward compatibility check in stress test
2021-10-04 16:34:14 +03:00
Azat Khuzhin
95d8133aaa
Cleanup unbundled image
...
No need to install specific boost packages, when all boost is installed.
2021-10-04 00:58:29 +03:00
Azat Khuzhin
74a7a0603d
test/fuzzer: print stacktrace
...
To get the stacktrace for errors w/o it [1]:
SET max_parser_depth = 4000
Got boring AST
Code: 306. DB::Exception: Maximum parse depth (1000) exceeded. Consider rising max_parser_depth parameter. (TOO_DEEP_RECURSION)
[1]: https://clickhouse-test-reports.s3.yandex.net/29513/769bfbe71f22c392458eced655080a0bbd442e02/fuzzer_ubsan/fuzzer.log
2021-10-02 12:49:44 +03:00
Azat Khuzhin
769bfbe71f
Fix server pid in fuzzer tests
...
Previously it was the pid of the subshell 40 while it should be the pid
of the clickhouse-server 39:
Here we see that the server pid is 39:
2021-09-28 11:02:34 + pgrep -f clickhouse-server
2021-09-28 11:02:34 39
Here we see that the 40 is the pid of subshell:
2021-09-28 11:02:45 ch/docker/test/fuzzer/run-fuzzer.sh: line 90: 39 Killed clickhouse-server --config-file db/config.xml -- --path db 2>&1
2021-09-28 11:02:45 40 Done | tail -100000 > server.log
And here we see that server_pid variable is 40:
2021-09-28 11:02:45 + server_exit_code=0
2021-09-28 11:02:45 + wait 40
v2: wait in background to call wait in foreground and ensure that the
process is alive, since w/o job control this is the only way to obtain
the exit code
2021-10-01 21:05:30 +03:00
alesapin
53fce1411c
Merge pull request #29597 from ClickHouse/fix-docker-image
...
Fix docker/test/util declaration
2021-10-01 13:43:17 +03:00
Alexey Milovidov
4271f8c738
Merge remote-tracking branch 'origin/master' into yandex-to-clickhouse-in-configs
2021-10-01 01:33:58 +03:00
Dmitry Novik
ad3c3f72e4
Fix docker/test/util declaration
2021-10-01 01:04:06 +03:00
Alexey Milovidov
2ec75b824b
Revert "repo.clickhouse.tech is not 100% ready"
...
This reverts commit 8616ba62e4
.
2021-09-29 02:43:20 +03:00
Dmitry Novik
29587c5013
Pass all parameters to process_functional_tests_result.py
2021-09-28 20:38:41 +03:00
Dmitry Novik
6b12869f6e
Fix path to executable
2021-09-28 15:11:18 +03:00
Dmitry Novik
4ce12a7685
Fixup
2021-09-28 11:50:28 +03:00
alesapin
8e25144bbb
Merge branch 'master' into trying_actions
2021-09-28 11:08:13 +03:00
Dmitry Novik
7755b3688f
Fix docker images build
2021-09-28 01:07:53 +03:00
Dmitry Novik
5418ea83fa
Fix clickhouse/test-util Dockerfile
2021-09-27 17:38:25 +03:00
Dmitry Novik
d9ce2c2d99
Update images.json
2021-09-27 15:59:20 +03:00
Dmitry Novik
4e6c945fdf
Rename fasttest output file
2021-09-27 13:09:57 +03:00
mergify[bot]
dbb924171d
Merge branch 'master' into fasttest-tsv
2021-09-27 07:48:49 +00:00
mergify[bot]
64669a28ce
Merge branch 'master' into yandex-to-clickhouse-in-configs
2021-09-26 22:59:23 +00:00
Alexey Milovidov
dc7eb5c1ae
Fix ugly typo
2021-09-26 03:29:32 +03:00
Alexey Milovidov
fb7ce28198
Merge branch 'master' into yandex-to-clickhouse-in-configs
2021-09-26 02:10:11 +03:00
Alexey Milovidov
90e74aff70
Fix performance test
2021-09-26 02:08:46 +03:00
Dmitry Novik
98df97f5d9
Fix docker image
2021-09-24 13:48:28 +03:00
Dmitry Novik
77dfa44256
Generate test_results.tsv for fasttest
2021-09-24 12:16:08 +03:00
Alexey Milovidov
8616ba62e4
repo.clickhouse.tech is not 100% ready
2021-09-24 04:41:59 +03:00
mergify[bot]
88b7680eeb
Merge branch 'master' into trying_actions
2021-09-23 15:48:57 +00:00
Alexey Milovidov
e9e77b4403
.tech -> .com
2021-09-22 03:22:57 +03:00
alexey-milovidov
0880ae2883
Merge branch 'master' into yandex-to-clickhouse-in-configs
2021-09-21 20:17:28 +03:00
tavplubix
8adfb9b593
Stop ThreadFuzzer before hung check ( #29167 )
...
* stop ThreadFuzzer before hung check
* fix
* fix
2021-09-20 17:23:10 +03:00
Alexey Milovidov
147ef79670
Change website URL, part 1
2021-09-20 01:56:28 +03:00
Alexey Milovidov
bdc99b6c91
Change Alpine Dockerfile
2021-09-20 01:53:52 +03:00
Alexey Milovidov
ab44e0addb
Change Docker org name
2021-09-20 01:52:21 +03:00
Alexey Milovidov
510540c7d7
Change Doker Hub URL
2021-09-20 01:45:18 +03:00
Alexey Milovidov
e513a5db32
Change <yandex> to <clickhouse> in configs
2021-09-20 01:38:53 +03:00
alesapin
00d9a62d6c
Missclick
2021-09-17 14:17:50 +03:00
alesapin
5d35ba7d93
One more time
2021-09-17 13:41:29 +03:00
alesapin
68c7f8638a
update docker image
2021-09-17 13:34:45 +03:00
alesapin
71dcf03a0f
Create changed image
2021-09-15 20:35:15 +03:00
Nikita Mikhaylov
4b3c0a95d1
Merge pull request #28046 from syominsergey/fix-build-clang-13-2
...
[WIP] Improve support for build with clang-13
2021-09-13 12:11:38 +03:00
Vitaly Baranov
512786346c
Merge pull request #28909 from vitlibar/convert-skip-list-into-first-line-comments
...
Convert skip_list .json into first line comments
2021-09-13 03:37:39 +03:00
Sergei Semin
d3b3e8607c
fix tabs
2021-09-12 19:51:47 +03:00
Sergei Semin
12e24a23d3
Merge branch 'master' into fix-build-clang-13-2
2021-09-12 19:30:55 +03:00
Vitaly Baranov
daa5979347
Remove unnecessary files.
2021-09-12 17:15:28 +03:00
Vitaly Baranov
bbb192ee85
Convert skip_list.json into first line comments.
2021-09-12 17:15:23 +03:00
alexey-milovidov
60f76d9254
Merge pull request #28759 from nikitamikhaylov/enable-gcc-11
...
Switch CI to GCC-11
2021-09-12 15:56:57 +03:00
alexey-milovidov
ea13a8b562
Merge pull request #28659 from myrrc/improvement/tostring_to_magic_enum
...
Improving CH type system with concepts
2021-09-12 15:26:29 +03:00
Sergei Semin
b7358636c1
remove duplicated libraries
2021-09-11 18:10:18 +03:00
Mike Kot
c4385a2c40
Simplifying exception messages
2021-09-11 00:40:18 +02:00
mergify[bot]
e3b3439535
Merge branch 'master' into fix-build-clang-13-2
2021-09-10 12:05:54 +00:00
alesapin
6243b64ff5
Merge pull request #28656 from ClickHouse/move_docker_images_to_separate_repo
...
Move docker images to a separate repo
2021-09-10 10:24:21 +03:00
mergify[bot]
937eeb9fed
Merge branch 'master' into enable-gcc-11
2021-09-09 22:44:10 +00:00
Nikita Mikhaylov
e575cc88fc
Better
2021-09-09 21:11:07 +00:00
Sergei Semin
ffc529feb1
use clang-13 in pvs docker image
2021-09-09 23:51:47 +03:00
Sergei Semin
b739d1cc92
make changes analog to https://github.com/ClickHouse/ClickHouse/pull/20495/files but for clang-13
2021-09-09 23:51:47 +03:00
Sergei Semin
63b3b90d1e
up LLVM_VERSION and don't use script for llvm setup
2021-09-09 23:51:47 +03:00
Sergei Semin
ff635bcab8
add build with clang-13
2021-09-09 23:51:47 +03:00
alesapin
2359a222ff
Fix more tests
2021-09-09 23:29:42 +03:00
robot-clickhouse
fc4f119417
Auto version update to [21.11.1.1] [54456]
2021-09-09 12:03:29 +03:00
alesapin
1297e95e4a
Revert "Revert jdbc-bridge image"
...
This reverts commit 63cc735b8f
.
2021-09-09 10:45:04 +03:00
alesapin
63cc735b8f
Revert jdbc-bridge image
2021-09-09 10:42:59 +03:00
alesapin
4d49e6da58
Fix build image name
2021-09-09 10:42:59 +03:00
Nikita Mikhaylov
107710e618
save
2021-09-08 15:06:15 +00:00
mergify[bot]
64fb384656
Merge branch 'master' into global-merge-executor
2021-09-08 13:15:38 +00:00
alesapin
cbbc5b2ee5
Fix some broken images
2021-09-08 12:58:17 +03:00
alesapin
bdd26f2a19
Merge branch 'master' into move_docker_images_to_separate_repo
2021-09-08 12:36:36 +03:00
alesapin
3b9dae8718
Merge pull request #28616 from ClickHouse/qoega-try-llvm-12
...
Merging #20495
2021-09-08 11:43:46 +03:00