thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-11-29 19:12:03 +00:00
Code Issues Packages Projects Releases Wiki Activity
130,871 Commits 9,187 Branches 1,782 Tags 2.2 GiB
4cfc42f050
Commit Graph

2135 Commits

Author SHA1 Message Date
vdimir
5e0caa669e
Move sucessull checks in commit status under a spoiler 2023-09-08 09:56:53 +00:00
János Benjamin Antal
5862c4ec93
Use not the absolute latest docker image tag (#54369) 
* Add params for select queries in clickhouse_helper.py

* Find the latest image tags from master

* Get full git history

* Do not checkout submodules

* Automatic style fix

* Fix style errors

* Make logging pretty

* Revert accidentally committed changes

* Eliminate pformat

* Update all workflows

* Remove flag with default value

* Improve exception handling

* Utilizing the loop condition in while loop

* Make the logic robust over temporary errors

* Prettify logs

* Automatic style fix

* Add accidentally removed line

* Revert "Make the logic robust over temporary errors"

This reverts commit 91e6b5d7ba.

* Do not fail in case the tag cannot be queried from ClickHouse

* Support partial finding of tags

---------

Co-authored-by: robot-clickhouse <robot-clickhouse@users.noreply.github.com>
 2023-09-08 08:42:56 +02:00
robot-clickhouse
c680c951d3 Automatic style fix 2023-09-07 13:12:53 +00:00
Igor Nikonov
27455a5882 Test timeout to fail 2023-09-07 12:57:35 +00:00
Igor Nikonov
c354fcaff7 Timeout report for stress_check 2023-09-07 12:56:22 +00:00
Igor Nikonov
a0317e461a Better description 2023-09-07 12:41:36 +00:00
Igor Nikonov
7ab56a13b8 Polishing 2023-09-06 22:49:28 +00:00
Igor Nikonov
babbecc646 Fix 2023-09-06 15:00:32 +00:00
Igor Nikonov
e858f0b880 Fix description length 2023-09-06 13:05:56 +00:00
Igor Nikonov
32048f2a8d Merge remote-tracking branch 'origin/master' into ci-check-process-timeout 2023-09-06 12:48:43 +00:00
Igor Nikonov
f6f2e731a7 Fix state 2023-09-06 12:44:39 +00:00
robot-clickhouse
f08d445a0f Automatic style fix 2023-09-06 11:59:51 +00:00
Igor Nikonov
a22e219be9 Fix 2023-09-06 11:51:43 +00:00
Igor Nikonov
3ef6d0da39 Set status and description 2023-09-06 11:02:17 +00:00
Mikhail f. Shiryaev
7f34c8e17d
Decouple env_helper.get_job_id_url 2023-09-06 12:43:29 +02:00
Mikhail f. Shiryaev
1afc5b7e05
Use suffix from git tags as version descriptions 2023-09-06 12:43:28 +02:00
Mikhail f. Shiryaev
6fc73e0e1f
Use a single BuildResult class across builds and report checks 2023-09-06 12:43:27 +02:00
Mikhail f. Shiryaev
88664bef3f
Create report status constants 2023-09-06 12:43:25 +02:00
Mikhail f. Shiryaev
428b72e767
Combine rows for build templates with lists 2023-09-06 12:43:23 +02:00
Mikhail f. Shiryaev
e0f6242af8
Use the same style for all reports 2023-09-06 12:43:21 +02:00
Mikhail f. Shiryaev
e18d9d39e8
Migrate S3Helper to pathlib.Path 2023-09-06 12:43:20 +02:00
Mikhail f. Shiryaev
7999598ad8
Rollback testing part from #42599 2023-09-05 12:59:31 +02:00
robot-clickhouse
0f5c49f6b5 Automatic style fix 2023-09-04 21:23:58 +00:00
Igor Nikonov
8aeccd078b Try to catch timeout 2023-09-04 21:16:25 +00:00
robot-clickhouse
926c5636dd
Merge pull request #42599 from ClickHouse/build-fuzzer-protocol 
libFuzzer: add CI fuzzers build, add tcp protocol fuzzer, fix other fuzzers.
 2023-09-04 22:41:54 +02:00
Mikhail f. Shiryaev
9851aebfa8
Add a note of where the lambda is deployed 2023-09-04 18:37:58 +02:00
Mikhail f. Shiryaev
4d0d161748
Use comment as a docstring 2023-09-04 17:06:52 +02:00
Mikhail f. Shiryaev
7351ac1d73
Deploy slack_bot_ci_lambda as other lambdas 2023-09-04 17:06:52 +02:00
Mikhail f. Shiryaev
5e27ca6e59
Use host network to avoid ipv6 problems 2023-09-04 17:06:51 +02:00
Mikhail f. Shiryaev
a082167d85
Move ci-slack-bot from utils to tests/ci 2023-09-04 17:06:51 +02:00
robot-clickhouse
f464effbea Automatic style fix 2023-09-03 22:15:53 +00:00
Yakov Olkhovskiy
11b7e1467f update build_check script 2023-09-03 21:51:12 +00:00
Yakov Olkhovskiy
02a729f9c5 fix 2023-09-03 18:05:54 +00:00
Yakov Olkhovskiy
ae6b5d46be fix 2023-09-03 17:38:36 +00:00
Yakov Olkhovskiy
97317a07f1 remove oss-fuzz build, add fuzzers build 2023-09-03 17:28:44 +00:00
Alexey Milovidov
e7f674165a
Merge pull request #54210 from ClickHouse/fix-timezones-ci-logs 
Fix timezones in the CI Logs database
 2023-09-03 04:58:43 +03:00
Alexey Milovidov
9763a4c2d4 Fix timezones in the CI Logs database 2023-09-02 17:49:51 +02:00
Alexander Tokmakov
d9438b176c save system tables from s3 in the report 2023-09-01 23:04:35 +02:00
Mikhail f. Shiryaev
6fb0257ee0
Use CiLogsCredentials in build_check 2023-08-30 22:06:30 +02:00
Mikhail f. Shiryaev
e113ba9024
Add vim filetype for *.lib files 2023-08-30 22:06:26 +02:00
Mikhail f. Shiryaev
f0c18d4bd7
Rework setup of CI logs export 2023-08-30 22:04:37 +02:00
Mikhail f. Shiryaev
c37c6c5220
Get rid of describe_parameters for the best robot token 2023-08-25 12:41:20 +02:00
Mikhail f. Shiryaev
0b9e7fa9c5
Reduce API calls to SSM client 2023-08-23 22:15:44 +02:00
Mikhail f. Shiryaev
eb9b452e86
Add default timeout value for ClickHouseHelper 2023-08-21 15:20:23 +02:00
Mikhail f. Shiryaev
b7866ce75f
Merge pull request #53493 from ClickHouse/update-style-checker 
Update style checker
 2023-08-17 12:56:20 +02:00
Mikhail f. Shiryaev
fd3e6c93ee
Produce standard report for cases when AST failed 2023-08-17 09:39:02 +02:00
Alexey Milovidov
857856b8b6 Leave only simplifications 2023-08-17 03:58:32 +02:00
Alexey Milovidov
64c829807c Merge branch 'master' of github.com:ClickHouse/ClickHouse into export-logs-in-ci-performance 2023-08-17 03:56:27 +02:00
Alexey Milovidov
428a05a560
Follow-up: Do not send logs to CI if the credentials are not set (#53456) 
* Follow-up

* Automatic style fix

* Update tests/ci/ast_fuzzer_check.py

* Update tests/ci/functional_test_check.py

* Update tests/ci/stress_check.py

* Automatic style fix

---------

Co-authored-by: robot-clickhouse <robot-clickhouse@users.noreply.github.com>
Co-authored-by: Alexander Tokmakov <tavplubix@clickhouse.com>
 2023-08-16 14:04:14 +03:00
Mikhail f. Shiryaev
651a45b04d
Add tests for initd start 2023-08-16 01:03:29 +02:00
Mikhail f. Shiryaev
3cd9fa395d
Add test for systemd + /etc/default/clickhouse 2023-08-16 00:51:44 +02:00
Mikhail f. Shiryaev
790475385a
Improve downloading: skip dbg, do not pull images on --no-download 2023-08-16 00:47:39 +02:00
Mikhail f. Shiryaev
387ce81895
Clean all containers properly 2023-08-16 00:46:53 +02:00
Nikita Mikhaylov
12ebfc9a7e
Merge pull request #53441 from ClickHouse/graceful-degradation-sending-logs-to-ci 
Do not send logs to CI if the credentials are not set
 2023-08-15 18:24:14 +02:00
Mikhail f. Shiryaev
33948a150f
Restart killed PublishedReleaseCI workflows 2023-08-15 11:50:11 +02:00
robot-clickhouse
a92fe25ff9 Automatic style fix 2023-08-15 07:15:58 +00:00
Alexey Milovidov
df02512ebf Do not send logs to CI if the credentials are not set 2023-08-15 08:53:08 +02:00
Mikhail f. Shiryaev
56a8818cf2
Fix logic of Mergeable Check 2023-08-14 14:31:26 +02:00
Mikhail f. Shiryaev
900e38a676
Fail early on missed documentation for new features 2023-08-14 14:20:40 +02:00
Mikhail f. Shiryaev
310ac6feaf
Tune PRInfo.has_changes_in_documentation 2023-08-14 14:19:08 +02:00
Alexey Milovidov
748a65b2bb Fix error 2023-08-13 06:35:32 +02:00
Alexey Milovidov
25fefbe144 Fix error 2023-08-13 06:32:48 +02:00
Alexey Milovidov
3ca59e9b19 Fix a typo 2023-08-13 06:25:55 +02:00
Alexey Milovidov
df6fe897c6 Better logs in CI 2023-08-13 06:17:45 +02:00
Alexey Milovidov
645834ffb6 Fix errors 2023-08-13 04:54:20 +02:00
Alexey Milovidov
fd3d0f7fa4 Merge branch 'master' into export-logs-in-ci-performance 2023-08-13 02:43:37 +02:00
Alexey Milovidov
077cd535e2 Improvements 2023-08-13 02:38:48 +02:00
Alexey Milovidov
b84132f4cd Merge branch 'master' into export-logs-in-ci-fuzz 2023-08-13 02:17:30 +02:00
Alexey Milovidov
6fadb8e883
Merge pull request #53353 from ClickHouse/export-logs-in-ci-stress 
Export logs from CI in stress tests
 2023-08-13 03:07:46 +03:00
Alexey Milovidov
221dd53d37 Fixup 2023-08-13 00:36:50 +02:00
Alexey Milovidov
32778e0407 Export logs from CI in performance (part 3) 2023-08-13 00:34:01 +02:00
Alexey Milovidov
5df03dd12f Better 2023-08-13 00:19:58 +02:00
Alexey Milovidov
d059465389
Merge pull request #52293 from ClickHouse/sqltest 
Add SQLTest
 2023-08-13 00:35:46 +03:00
robot-clickhouse
7d7ed06010 Automatic style fix 2023-08-12 20:53:33 +00:00
robot-clickhouse
e6e5a3858d Automatic style fix 2023-08-12 20:43:29 +00:00
Alexey Milovidov
afbb91600e Export logs from CI in fuzzer 2023-08-12 22:41:56 +02:00
Alexey Milovidov
176c16fcc5 Export logs from CI in stress tests 2023-08-12 22:30:18 +02:00
Alexey Milovidov
fc6fc234b2
Merge pull request #53086 from ClickHouse/export-logs-in-ci 
Add a script to export logs in CI
 2023-08-12 17:35:36 +03:00
Yakov Olkhovskiy
fa7994de3b
Merge pull request #53181 from ClickHouse/ci-build-s390x 
Add Linux s390x build to CI
 2023-08-12 09:16:07 -04:00
Alexey Milovidov
01e9f32fdf Merge branch 'master' of github.com:ClickHouse/ClickHouse into export-logs-in-ci 2023-08-12 02:46:06 +02:00
Alexey Milovidov
3232993eeb
Merge pull request #53344 from ClickHouse/relax-mergeable-check 
Relax mergeable check
 2023-08-12 03:38:12 +03:00
Alexey Milovidov
1ae186b679 Merge branch 'export-logs-in-ci' of github.com:ClickHouse/ClickHouse into export-logs-in-ci 2023-08-12 02:11:48 +02:00
Alexey Milovidov
38c60430f4 Merge branch 'master' of github.com:ClickHouse/ClickHouse into export-logs-in-ci 2023-08-12 02:11:43 +02:00
Alexey Milovidov
62b3485cd8
Merge pull request #53283 from ClickHouse/fix-00906_low_cardinality_cache 
Inhibit randomization in `00906_low_cardinality_cache`
 2023-08-12 03:01:43 +03:00
Alexey Milovidov
aa815bfc74 Relax mergeable check 2023-08-12 01:54:11 +02:00
robot-clickhouse
2b852e55b6 Automatic style fix 2023-08-11 21:13:53 +00:00
Alexey Milovidov
af74bc9564 Merge branch 'export-logs-in-ci' of github.com:ClickHouse/ClickHouse into export-logs-in-ci 2023-08-11 22:57:23 +02:00
Alexey Milovidov
ddb7fb2ce9
Update tests/ci/functional_test_check.py 
Co-authored-by: Mikhail f. Shiryaev <felixoid@clickhouse.com>
 2023-08-11 23:56:23 +03:00
Alexey Milovidov
9b7fcf17bc Merge branch 'allow-creating-system-tables-at-startup' into export-logs-in-ci 2023-08-11 22:54:24 +02:00
Alexey Milovidov
b6c47ee157 Merge branch 'master' into export-logs-in-ci 2023-08-11 03:44:18 +02:00
Mikhail f. Shiryaev
dcb1d3ed00
Rewrite compress_file to pathlib.Path 2023-08-10 22:41:57 +02:00
Mikhail f. Shiryaev
c8d5deda83
Add typing to compress_files 2023-08-10 22:41:56 +02:00
Mikhail f. Shiryaev
18a291a4c7
Upload cargo cache for rust to S3 and reuse 2023-08-10 22:41:55 +02:00
Mikhail f. Shiryaev
dbd3fdd437
Clean ccache_utils a little bit 2023-08-10 22:09:20 +02:00
Mikhail f. Shiryaev
cbd81b3758
Partially revert changes from #52395 2023-08-10 22:09:19 +02:00
Alexey Milovidov
3acb10b8bc Inhibit randomization in 00906_low_cardinality_cache 2023-08-10 20:51:03 +02:00
Alexey Milovidov
561e043868
Merge branch 'master' into sqltest 2023-08-10 21:47:56 +03:00
Alexey Milovidov
269f54b49e
Merge branch 'master' into export-logs-in-ci 2023-08-10 16:56:14 +03:00
Alexey Milovidov
47b5d13367
Merge branch 'master' into fix-stress-test 2023-08-10 16:55:49 +03:00
Alexey Milovidov
e4fbca2e2e Apply review recommendations 2023-08-10 04:47:44 +02:00
Alexey Milovidov
cfdccb95d5 Merge branch 'master' into export-logs-in-ci 2023-08-10 04:30:23 +02:00
Alexey Milovidov
53151610a0
Merge branch 'master' into sqltest 2023-08-10 03:47:26 +03:00
Mikhail f. Shiryaev
ce30b93b34
Move repeated code for ClickHouseHelper to _insert_post 2023-08-09 23:46:45 +02:00
Mikhail f. Shiryaev
c7b045fdde
Process build profiles in the python script, not in docker 2023-08-09 21:48:05 +02:00
Mikhail f. Shiryaev
239d198cf2
Add insert_file to ClickHouseHelper, make insert_json_str public 2023-08-09 21:11:21 +02:00
Mikhail f. Shiryaev
1405346830
Rewrite build_check to use pathlib.Path 2023-08-09 18:56:19 +02:00
Mikhail f. Shiryaev
5341c5c8dd
Use CMAKE_FLAGS and args for cmake profiler 2023-08-09 18:26:25 +02:00
Alexey Milovidov
964c407d48
Revert "Revert "Upload build time-trace data to CI database"" 2023-08-09 16:11:17 +03:00
Alexander Gololobov
63183c56f1
Revert "Upload build time-trace data to CI database" 2023-08-09 13:58:05 +02:00
Alexey Milovidov
10bc51514b Merge branch 'allow-creating-system-tables-at-startup' into export-logs-in-ci 2023-08-09 04:28:50 +02:00
Alexey Milovidov
ede1b17c48
Merge branch 'master' into sqltest 2023-08-09 05:10:47 +03:00
Yakov Olkhovskiy
d8a231e0cb Add Linux s390x build to CI 2023-08-09 00:44:01 +00:00
Alexey Milovidov
acb43266ad
Merge branch 'master' into fix-stress-test 2023-08-08 23:41:59 +03:00
robot-clickhouse
b397d11c91 Automatic style fix 2023-08-08 19:26:19 +00:00
Alexey Milovidov
23eee8d10d Merge branch 'master' of github.com:ClickHouse/ClickHouse into upload-build-profile 2023-08-08 21:02:26 +02:00
Alexey Milovidov
2b59c1778c More info 2023-08-08 21:02:13 +02:00
Alexey Milovidov
ed34f3c46f Fix error 2023-08-08 21:00:30 +02:00
robot-clickhouse
2eff5928d9 Automatic style fix 2023-08-08 13:44:10 +00:00
Alexey Milovidov
7ce8266c8a Fix errors 2023-08-08 15:17:20 +02:00
János Benjamin Antal
67a87a4b95
Merge pull request #53051 from ClickHouse/only-debug-upgrade-check 
Require only ASAN based upgrade check to minimize false positives
 2023-08-08 11:44:11 +02:00
Alexey Milovidov
a91edb8b55 Fix mypy 2023-08-08 04:11:32 +02:00
Alexey Milovidov
161de7c12f Fix error 2023-08-08 03:34:44 +02:00
Alexey Milovidov
5644216429 Fix whitespace 2023-08-07 23:40:34 +02:00
Alexey Milovidov
27559b0349 Adjust a test 2023-08-07 23:09:34 +02:00
Alexey Milovidov
cafe4354c5 Merge branch 'master' into export-logs-in-ci 2023-08-07 23:08:26 +02:00
Alexey Milovidov
77e1693f3c Fix mypy 2023-08-07 21:18:56 +02:00
Alexey Milovidov
8ce4ff0f95 Merge branch 'master' of github.com:ClickHouse/ClickHouse into upload-build-profile 2023-08-07 21:18:30 +02:00
Alexey Milovidov
ab64845740 Fix mypy 2023-08-07 18:45:32 +02:00
Alexey Milovidov
5275062eaf Fix trailing whitespaces 2023-08-07 18:44:25 +02:00
Mikhail f. Shiryaev
2347f5886b
Update Mergeable Check at the finishing CI 2023-08-07 18:06:50 +02:00
Alexey Milovidov
9cb949b60b Fix style 2023-08-07 14:53:38 +02:00
János Benjamin Antal
55c3581242
Replace Upgrade check (debug) with a less flaky upgrade check 2023-08-07 10:27:10 +02:00
robot-clickhouse
19ad81dabd Automatic style fix 2023-08-07 01:50:29 +00:00
Alexey Milovidov
d977e221b4 Upload build time-trace data to CI database 2023-08-07 03:26:59 +02:00
robot-clickhouse
333ed6425c Automatic style fix 2023-08-06 11:30:50 +00:00
Alexey Milovidov
9428040d05 Fix hung check in stress test 2023-08-06 13:07:48 +02:00
Alexey Milovidov
fbb2be21cc Slightly better 2023-08-06 13:04:35 +02:00
robot-clickhouse
2edda961d2 Automatic style fix 2023-08-06 02:55:26 +00:00
Alexey Milovidov
f1eee292c9 Enable export (first try) 2023-08-06 04:38:04 +02:00
Alexey Milovidov
580f207b8d
Merge branch 'master' into sqltest 2023-08-04 22:16:56 +03:00
János Benjamin Antal
c874e5955b Remove upgrade checks with sanitizers 2023-08-04 12:33:11 +00:00
robot-ch-test-poll2
c556ad96f6
Merge pull request #52948 from ClickHouse/refactor-ci-config 
Refactor CI_CONFIG
 2023-08-04 11:10:12 +02:00
Alexey Milovidov
388feb953d Fix paths 2023-08-03 19:28:59 +02:00
Mikhail f. Shiryaev
ca67576720
Do not use inline cache in server images 2023-08-03 18:06:00 +02:00
Alexey Milovidov
65e8ee8fb7 Fix paths 2023-08-03 15:28:18 +02:00
Mikhail f. Shiryaev
8499956321
Refactor CI_CONFIG from dict to dataclasses 2023-08-03 14:37:19 +02:00
Alexey Milovidov
5ec6a4695f Fix log paths 2023-08-02 23:22:15 +02:00
Alexey Milovidov
a98e869912 Merge branch 'master' into sqltest 2023-08-02 20:08:39 +02:00
Mikhail f. Shiryaev
6dcdc4cd38
Remove mark_failed_report_pending, it's already in builds_report_config 2023-08-02 19:20:22 +02:00
Mikhail f. Shiryaev
40cdfa2384
Merge branch 'master' into fix-packager 2023-08-02 11:21:54 +02:00
Mikhail f. Shiryaev
3a07b80f34
Merge pull request #52277 from ClickHouse/kill-runner-with-subprocesses 
Kill the runner process with all subprocesses
 2023-08-02 11:18:56 +02:00
alesapin
0abd5a058b
Merge pull request #52871 from ClickHouse/cont-server-after-gdb-killed 
Try to continue clickhouse process in stress test after terminating gdb.
 2023-08-02 10:55:38 +02:00
Mikhail f. Shiryaev
8537b32de1
Merge branch 'master' into docker-improvements 2023-08-02 08:40:24 +02:00
Alexey Milovidov
3b6247e3f5 Merge branch 'sqltest' of github.com:ClickHouse/ClickHouse into sqltest 2023-08-01 22:33:19 +02:00
Alexey Milovidov
0641dfd7e6 Add missing modification 2023-08-01 22:33:04 +02:00
Alexey Milovidov
97bd68d5c7 Merge branch 'master' into sqltest 2023-08-01 22:31:18 +02:00
Mikhail f. Shiryaev
f87e53c042
Merge branch 'master' into sqltest 2023-08-01 17:42:58 +02:00
Mikhail f. Shiryaev
6b29cccf3e
Rename docker_images.json to docker_images_for_tests.json 2023-08-01 17:30:54 +02:00
Mikhail f. Shiryaev
79d49a006c
Fix the PR body check for Reverts #number 2023-08-01 17:06:23 +02:00
Nikolai Kochetov
09cb5cbe87 Try to continue clickhouse process in stress test after terminating gdb. 2023-08-01 14:30:41 +00:00
Mikhail f. Shiryaev
0931e87989
Merge branch 'master' into sqltest 2023-08-01 16:14:28 +02:00
Alexey Milovidov
8646eefc9c
Update docker_test.py 2023-08-01 17:05:24 +03:00
Mikhail f. Shiryaev
e5087d14ab
Merge branch 'master' into fix-packager 2023-08-01 12:04:06 +02:00
Kseniia Sumarokova
40ce9cf9ed
Merge pull request #51843 from ClickHouse/kssenii-nginx 
Replace kssenii/nginx-test by clickhouse/nginx-dav
 2023-08-01 11:36:20 +02:00
Alexey Milovidov
afb76101b7 Remove obsolete part of a check name 2023-07-31 05:23:28 +02:00
Alexey Milovidov
d243602e74
Merge pull request #52773 from ClickHouse/upload-build-check-statistics 
Upload build statistics to the CI database
 2023-07-31 06:18:41 +03:00
Alexey Milovidov
8155659f87
Merge pull request #52775 from ClickHouse/remove-coverity-2 
Remove Coverity (part 2)
 2023-07-30 12:49:04 +03:00
Alexey Milovidov
bc1f80dd82 Remove Coverity (part 2) 2023-07-30 10:25:44 +02:00
robot-clickhouse
358773c620 Automatic style fix 2023-07-30 07:45:27 +00:00
Alexey Milovidov
448756d148 Add machine_type information to the CI database 2023-07-30 09:31:21 +02:00
Alexey Milovidov
41ffc09771 Upload build statistics to the CI database 2023-07-30 09:18:06 +02:00
Kruglov Pavel
93e10077ba
Fix attaching gdb in stress tests (#51445) 
* Fix attaching gdb in stress tests

* Fix

* Update run.sh

* Try remove run_with_retry

* Return run_with_retry

* Don't set -e in run_with_retry if it was't set before

* Update tests/ci/utils.lib

* Fix bash

---------

Co-authored-by: Alexander Tokmakov <tavplubix@clickhouse.com>
 2023-07-26 12:53:19 +03:00
Azat Khuzhin
0b258dda4e Reproducible builds for Rust 
From now on cargo will not download anything from the internet during
builds. This step had been moved for docker image builds (via cargo
vendor).

And now cargo inside docker.io/clickhouse/binary-builder will not use
any crates from the internet, so we don't need to add --offline for
cargo commands in cmake (corrosion_import_crate()).

Also the docker build command had been adjusted to allow following
symlinks inside build context, by using tar, this is required for Rust
packages.

Note, that to make proper Cargo.lock that could be vendored I did the
following:
- per-project locks had been removed (since there is no automatic way to
  sync the workspace Cargo.lock with per-project Cargo.lock, since cargo
  update/generate-lockfile will use only per-project Cargo.toml files
  apparently, -Z minimal-versions does not helps either)
- and to generate Cargo.lock with less changes I've pinned version in
  the Cargo.toml strictly, i.e. not 'foo = "0.1"' but 'foo = "=0.1"'
  then the Cargo.lock for workspace had been generated and afterwards
  I've reverted this part.

Plus I have to update the dependencies afterwards, since otherwise there
are conflicts with dependencies for std library. Non trivial.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-07-22 22:46:22 +02:00
Alexey Milovidov
c3c8ea9c27 Do something 2023-07-22 17:30:55 +02:00
Mikhail f. Shiryaev
c6e9f621f1 Replace kssenii/nginx-test:1.1 by clickhouse/nginx-dav 2023-07-22 15:45:47 +02:00
Alexey Milovidov
3398355e46 Add something with unclear purpose 2023-07-22 02:04:30 +02:00
Mikhail f. Shiryaev
101214cb8f
Fix broken termination lambda by caching runners for at most 15 seconds 2023-07-21 18:32:32 +02:00
Mikhail f. Shiryaev
083be24a74
Improve logic of checking valid cached values in lambdas 2023-07-21 18:32:31 +02:00
Mikhail f. Shiryaev
a117997d5b
Show the runner type in the job logs 2023-07-21 18:32:31 +02:00
Mikhail f. Shiryaev
da37f32d4b
Attempt to delete the runner before shutting down 2023-07-21 18:32:30 +02:00
Mikhail f. Shiryaev
538d1f7c17
Kill the runner process with all subprocesses 2023-07-21 18:32:28 +02:00
Antonio Andelic
e181e602a7
Update ci_config.py 2023-07-21 12:43:32 +02:00
Alexey Milovidov
f458108c44 Fix style 2023-07-19 22:15:23 +02:00
Alexey Milovidov
29094a22cf Fix Docker 2023-07-19 20:00:00 +02:00
Alexey Milovidov
f7fcf079a4 Merge branch 'master' of github.com:ClickHouse/ClickHouse into sqltest 2023-07-19 19:58:10 +02:00
Mikhail f. Shiryaev
9c4005b33f
Add logging about all found workflows for merge_pr.py 2023-07-19 15:12:58 +02:00
Alexey Milovidov
8e3bd222f0 Add SQLTest 2023-07-19 03:42:32 +02:00
Antonio Andelic
3c2507ad7d
Merge branch 'master' into fix-packager 2023-07-18 09:18:03 +02:00
Alexey Milovidov
20b77e946a There is no point in detecting flaky tests 2023-07-15 23:00:23 +02:00
Dmitry Novik
ee2392bf8e Merge remote-tracking branch 'origin/master' into analyzer-integration-tests 2023-07-11 08:55:31 +00:00
Alexey Milovidov
6cfeef00d7
Merge pull request #31398 from ClickHouse/riscv-in-ci 
Add Linux RISC-V 64 build to CI
 2023-07-06 22:49:16 +03:00
Antonio Andelic
f825c85c72
Merge branch 'master' into fix-packager 2023-07-06 08:36:43 +02:00
Alexey Milovidov
5a3aadacde Fix error 2023-07-05 23:40:37 +02:00
Antonio Andelic
3dc0afd52f Correctly grep archives 2023-07-05 11:19:15 +00:00
Alexey Milovidov
3021180e01 Update 2023-07-04 21:33:12 +02:00
Alexey Milovidov
1eafba60d2 Merge branch 'master' into riscv-in-ci 2023-07-04 21:28:05 +02:00
Antonio Andelic
5b31c714a0
Merge branch 'master' into fix-packager 2023-07-04 15:48:31 +02:00
Alexey Milovidov
aa2b8aa893
Merge pull request #51769 from ClickHouse/mypy-1.4.0 
Version mypy=1.4.1 falsly reports unused ignore comment
 2023-07-04 16:45:57 +03:00
Yatsishin Ilya
5230503651 remove unused type: ignore 2023-07-04 12:59:25 +00:00
Antonio Andelic
6d3dbae8aa Merge remote-tracking branch 'origin/docs-packaging' into fix-packager 2023-07-04 11:21:53 +00:00
Alexey Milovidov
463c2ba5ce
Revert "Add documentation for building in docker" 2023-07-04 14:07:10 +03:00
Mikhail f. Shiryaev
c715ee5cef
Replace --build-type=debug by --debug-build 2023-07-03 19:58:47 +02:00
Mikhail f. Shiryaev
469da1831a
Replace --build-type=debug by --debug-build 2023-07-03 17:42:15 +02:00
Antonio Andelic
c06afc53b4 Make scripts backwards compatible 2023-07-02 09:57:06 +00:00
Antonio Andelic
00aeb407e5 Wait with retries 2023-06-30 14:52:21 +00:00
Alexey Milovidov
1e6f6b522c
Merge pull request #51180 from ClickHouse/update-tests-docker 
Update tests' docker containers
 2023-06-30 11:24:59 +03:00
Alexey Milovidov
b6a37fbf18
Merge branch 'master' into many-fixes-2 2023-06-29 22:36:33 +03:00
Mikhail f. Shiryaev
953f1c7885
Upload fast test binaries to builds bucket 2023-06-28 22:19:33 +02:00
Mikhail f. Shiryaev
eeb8cdbc19
Add way to define additional urls in test reports 2023-06-28 22:19:00 +02:00
vdimir
e2241b8c7c
Merge branch 'grace_hash_join_debug_2306' into many-fixes-2 2023-06-28 09:35:20 +00:00
robot-clickhouse
9d8b996f68 Automatic style fix 2023-06-27 04:30:56 +00:00
Alexey Milovidov
b5635887c3 Merge remote-tracking branch 'origin/refactor-subqueries-for-in' into many-fixes 2023-06-27 05:39:44 +02:00
Alexey Milovidov
e0ac127bdd Merge branch 'two-tests-are-longer-with-analyzer' into many-fixes 2023-06-27 05:23:23 +02:00
Alexey Milovidov
d37b1c17f5 Merge branch 'fix-fasttest' into many-fixes 2023-06-27 05:21:43 +02:00
lgbo-ustc
179a7ce202 debug 2023-06-26 20:43:19 +08:00
Mikhail f. Shiryaev
065b87aa30
Improve logging a little bit, avoid double labeling 2023-06-26 13:25:49 +02:00
Mikhail f. Shiryaev
c64f929b9c
Fix broken labeling for manual approve 2023-06-26 13:25:48 +02:00
Alexey Milovidov
b8e6bd3299
Merge branch 'master' into refactor-subqueries-for-in 2023-06-26 06:05:12 +03:00
Alexey Milovidov
4ee74ec213 Two tests are twice longer in average with Analyzer and sometimes failing 2023-06-26 02:19:52 +02:00
Alexey Milovidov
4fd1fc4507 Fix typo 2023-06-25 06:14:31 +02:00
Alexey Milovidov
511b2cdc21 Increase timeout of fast test 2023-06-25 06:10:57 +02:00
Ilya Yatsishin
9a75faad77
Merge pull request #51340 from ClickHouse/qoega-patch-4 2023-06-24 14:22:32 +02:00
Ilya Yatsishin
7e3eaadaac
Make SQLancer check not "always green" 2023-06-23 18:55:04 +02:00
Alexey Milovidov
b6e7f4cf6c Require more checks 2023-06-22 23:24:27 +02:00
Nikolai Kochetov
a940031878 Merge branch 'master' into refactor-subqueries-for-in 2023-06-22 12:18:48 +02:00
Mikhail f. Shiryaev
02776b8a15
Merge pull request #51163 from ClickHouse/fasttest-ignore-chown-fail 
Ignore errors for `chown` in fasttests
 2023-06-22 09:51:57 +02:00
Mikhail f. Shiryaev
1591633d72
Merge pull request #49283 from ClickHouse/runners-survival 
Attempt to increase the general runners' survival rate
 2023-06-22 00:16:11 +02:00
Mikhail f. Shiryaev
118c3af4d6
Ignore errors for chown in fasttests 2023-06-21 12:59:21 +02:00
Robert Schulze
bc7df2baca
Merge branch 'master' into non-experimental-qpl-deflate 2023-06-20 20:06:01 +02:00
Mikhail f. Shiryaev
23d843d379
Experiment with decreasing scale_up for style-checkers 2023-06-20 15:17:33 +02:00
Mikhail f. Shiryaev
1a8571c465
Check and proceed spot termination on runner setup 2023-06-19 15:40:32 +02:00
Mikhail f. Shiryaev
4d5160a3ba
Improve determination of job received 2023-06-19 15:40:32 +02:00
Mikhail f. Shiryaev
d782b05845
Continue LCH for terminated instances, add logging 2023-06-19 15:40:31 +02:00
Mikhail f. Shiryaev
d8e3d5c803
Kill the instance proactively on spot termination 2023-06-19 15:40:30 +02:00
Mikhail f. Shiryaev
d672bd3382
Stop the runner if there are no running jobs for long 2023-06-19 15:40:30 +02:00
Mikhail f. Shiryaev
915f431b95
Tune the shutting down on rebalance signal 2023-06-19 15:40:28 +02:00
Mikhail f. Shiryaev
026ab3d4ce
Calculate current capacity according to HealthStatus 2023-06-19 15:40:27 +02:00
Mikhail f. Shiryaev
8feb5146fc
Do not "update" the token and the instances infinitely 2023-06-19 15:40:27 +02:00
Mikhail f. Shiryaev
a5e31e98de
Decouple cleaning part from metrics lambda to a new one 2023-06-19 15:40:26 +02:00
Mikhail f. Shiryaev
1688ba5cf5
Use ephemeral runners for GH actions 2023-06-19 15:40:25 +02:00
Mikhail f. Shiryaev
47ea742076
Do not kill runner on the fly, delay 3 seconds 2023-06-19 15:40:25 +02:00
Mikhail f. Shiryaev
6cecdb8fca
Shutdown runner process and do not restart it on termination 2023-06-19 15:40:24 +02:00
Mikhail f. Shiryaev
197d0d1314
Process lifecycle hook in post-running action hook 2023-06-19 15:40:24 +02:00
Mikhail f. Shiryaev
d45092d63b
Process the rebalance recomendation event on post-run 2023-06-19 15:40:23 +02:00
Mikhail f. Shiryaev
ee5ca7337e
Add reset to Stopwatch, improve codebrowser report 2023-06-16 12:51:31 +02:00
Dmitry Novik
49f88f5873 Fix ENV variable name 2023-06-15 14:15:01 +00:00
Mikhail f. Shiryaev
eb2a5c31a3
Do not mock git_helper.Git, suppress git describe error for ignore_no_tags 2023-06-15 15:35:03 +02:00
Mikhail f. Shiryaev
2ac95ccfce
Adjust version_helper test cases to proper tags 2023-06-15 14:48:10 +02:00
Mikhail f. Shiryaev
07203a4542
Fix a versions' tweak for tagged commits, improve version_helper 2023-06-15 14:48:09 +02:00
robot-clickhouse
1230519bec Automatic style fix 2023-06-14 13:38:44 +00:00
Dmitry Novik
fe8172fbd9 Review fixes 2023-06-14 13:17:04 +00:00
Dmitry Novik
afcc3aca36 Update ci_config.py 2023-06-14 13:14:59 +00:00
Dmitry Novik
1207085dec
Merge branch 'master' into analyzer-integration-tests 2023-06-13 17:09:29 +02:00
Nikita Mikhaylov
52a460df67
Tests with parallel replicas are no more "always green" (#50896) 2023-06-13 16:43:35 +02:00
Alexander Tokmakov
5368fa72ca
Merge pull request #50487 from Avogar/add-gdb-to-stateless-tests 
Attach gdb in stateless tests
 2023-06-13 17:04:17 +03:00
Dmitry Novik
9e8ca5f5ae Add integration test check with enabled analyzer 2023-06-13 13:09:39 +00:00
Robert Schulze
8222861f2e
Merge remote-tracking branch 'rschu1ze/master' into non-experimental-qpl-deflate 2023-06-11 14:39:30 +00:00
Nikolai Kochetov
6ce8329bda Merge branch 'master' into refactor-subqueries-for-in 2023-06-09 20:04:27 +02:00
Nikolai Kochetov
a878312fdf
Merge pull request #50693 from ClickHouse/vdimir/disable_grace_hash_stress 
Disable grace_hash join in stress tests
 2023-06-09 17:57:34 +02:00
Robert Schulze
1aa158909e
enable_qpl_deflate_codec --> enable_deflate_qpl_codec 2023-06-09 12:43:33 +00:00
jinjunzh
f1192d59af
refine patch according to comments 2023-06-09 12:43:15 +00:00
jinjunzh
31173ab55b
add sections of deflate_qpl for stress test and performance test 2023-06-09 12:43:08 +00:00
Mikhail f. Shiryaev
2aaaec2abe
Catch issues with dockerd during the build 2023-06-08 14:01:41 +02:00
vdimir
2297995c1a
Disable grace_hash join in stress tests 
Until https://github.com/ClickHouse/ClickHouse/issues/50220 is fixed
 2023-06-08 07:15:11 +00:00
Antonio Andelic
26fb80b540
Merge pull request #50615 from ClickHouse/fix-jepsen-check 
Fix Jepsen runs in PRs
 2023-06-07 08:43:51 +02:00
Sema Checherinda
990652119e
Merge pull request #50361 from nickitat/fail_perf_on_too_many_slow 
Fail perf tests when too many queries slowed down
 2023-06-06 11:33:04 +02:00
Antonio Andelic
0bc31a7288 Fix jepsen runs in PRs 2023-06-06 07:17:30 +00:00
robot-clickhouse-ci-1
4cf5a13e5b
Merge pull request #50160 from ClickHouse/prio-runners 
Refactor lambdas, prepare to prio runners
 2023-06-06 07:29:28 +02:00
avogar
e8c6c7967b Move attaching gdb to separate lib 2023-06-05 18:21:40 +00:00
Nikita Taranov
e548dce123 fix 2023-06-03 00:08:47 +02:00
Nikita Taranov
4506299d73 impl 2023-06-02 23:14:09 +02:00
pufit
0d98a46326 Fix KeyError in cherry-pick 2023-06-02 13:02:18 -04:00
Mikhail f. Shiryaev
9bd0a53e7c
Get only online instances in get_lost_ec2_instances 2023-06-02 12:27:29 +02:00
Mikhail f. Shiryaev
3c13eaa159
Simplify get_dead_runners_in_ec2 a little bit, update runners 2023-06-02 12:27:28 +02:00
Mikhail f. Shiryaev
b775b5cfd6
Move all CI runners metrics into one namespace 2023-06-02 12:27:27 +02:00
Mikhail f. Shiryaev
e657e2ba10
Additional logging in autoscale_runners_lambda 2023-06-02 12:27:26 +02:00
Mikhail f. Shiryaev
b7c5fdab77
Move lambda package building to public.ecr.aws/lambda/python for compatibility 2023-06-02 12:27:26 +02:00
Mikhail f. Shiryaev
0ddd53088d
Add a new runner type for ci metrics and autoscaling 2023-06-02 12:27:24 +02:00
Mikhail f. Shiryaev
e8b03d7498
Move insert part for ClickHouseHelper to shared 2023-06-01 21:03:57 +02:00
Mikhail f. Shiryaev
2dca0eac1b
Delete __init__.py in lambda directories to break subpackage 2023-06-01 21:03:56 +02:00
Mikhail f. Shiryaev
acb9531ebf
Unify and put GH access token to the lambda_shared 2023-06-01 21:03:56 +02:00
Mikhail f. Shiryaev
0fa6a84161
Move the stuff related to runners to lambda_shared 2023-06-01 21:03:55 +02:00
Mikhail f. Shiryaev
7f08f218d9
Create lambda_shared package for lambdas 2023-06-01 21:03:53 +02:00
Mikhail f. Shiryaev
484c91c47e
Add DRY_RUN and configurable PY_VERSION to lambda deployment 2023-06-01 21:03:53 +02:00
Mikhail f. Shiryaev
27941b4d26
Decrease the time window for autoscale_runners_lambda 2023-06-01 21:03:52 +02:00
Mikhail f. Shiryaev
db02938411
Do not count unfinished tasks with conclusion=None 2023-06-01 21:03:51 +02:00
Mikhail f. Shiryaev
855afb56f9
Get instances for the region to not kill a fresh one 2023-06-01 21:03:50 +02:00
Mikhail f. Shiryaev
bf9b563e0b
Improve caching mechanism for token, add cached instances 2023-06-01 21:03:50 +02:00
Mikhail f. Shiryaev
7bf9089dcd
Increase access_token cached time 2023-06-01 21:03:49 +02:00
Mikhail f. Shiryaev
985cd8fc8a
Improve events logging 2023-06-01 21:03:48 +02:00
Nikita Taranov
e5c21e588a
Update app.py (#50407) 2023-06-01 15:19:47 +02:00
Mikhail f. Shiryaev
9089360688
Merge pull request #50415 from ClickHouse/build-report-pending 
Mark the builds without results as pending
 2023-06-01 14:53:09 +02:00
Mikhail f. Shiryaev
de7179876e
Merge pull request #47151 from ClickHouse/increase-vm-max_map_count 
Update ubuntu_ami_for_ci.sh
 2023-06-01 13:54:21 +02:00
Mikhail f. Shiryaev
6a5ae6558a
Add comment for usage of prepare-ci-ami.sh 2023-06-01 12:58:03 +02:00
Mikhail f. Shiryaev
bcbce15cde
Mark the builds without results as pending 2023-06-01 08:51:44 +02:00
Mikhail f. Shiryaev
653da5f002
Reduce the unzip verbosity 2023-05-31 18:23:21 +02:00
Mikhail f. Shiryaev
a96a845e23
Add comment to prepare-ci-ami.sh, add a success at the end 2023-05-31 17:12:49 +02:00
pufit
08d3887832
Add re-creation for cherry-pick PRs (#50373) 
* Add recreation for cherry-pick PRs.
Small PR comment fix.

* Automatic style fix

---------

Co-authored-by: robot-clickhouse <robot-clickhouse@users.noreply.github.com>
Co-authored-by: Nikita Mikhaylov <mikhaylovnikitka@gmail.com>
 2023-05-31 13:04:29 +02:00
Mikhail f. Shiryaev
db3a050911
Seems like buildx needs explicit install since some time 2023-05-31 00:22:12 +02:00
Mikhail f. Shiryaev
56dc35d563
Merge branch 'master' into increase-vm-max_map_count 2023-05-30 19:24:18 +02:00