Commit Graph

1729 Commits

Author SHA1 Message Date
Robert Schulze
8222861f2e
Merge remote-tracking branch 'rschu1ze/master' into non-experimental-qpl-deflate 2023-06-11 14:39:30 +00:00
Nikolai Kochetov
6ce8329bda Merge branch 'master' into refactor-subqueries-for-in 2023-06-09 20:04:27 +02:00
Nikolai Kochetov
a878312fdf
Merge pull request #50693 from ClickHouse/vdimir/disable_grace_hash_stress
Disable grace_hash join in stress tests
2023-06-09 17:57:34 +02:00
Robert Schulze
1aa158909e
enable_qpl_deflate_codec --> enable_deflate_qpl_codec 2023-06-09 12:43:33 +00:00
jinjunzh
f1192d59af
refine patch according to comments 2023-06-09 12:43:15 +00:00
jinjunzh
31173ab55b
add sections of deflate_qpl for stress test and performance test 2023-06-09 12:43:08 +00:00
Mikhail f. Shiryaev
2aaaec2abe
Catch issues with dockerd during the build 2023-06-08 14:01:41 +02:00
vdimir
2297995c1a
Disable grace_hash join in stress tests
Until https://github.com/ClickHouse/ClickHouse/issues/50220 is fixed
2023-06-08 07:15:11 +00:00
Antonio Andelic
26fb80b540
Merge pull request #50615 from ClickHouse/fix-jepsen-check
Fix Jepsen runs in PRs
2023-06-07 08:43:51 +02:00
Sema Checherinda
990652119e
Merge pull request #50361 from nickitat/fail_perf_on_too_many_slow
Fail perf tests when too many queries slowed down
2023-06-06 11:33:04 +02:00
Antonio Andelic
0bc31a7288 Fix jepsen runs in PRs 2023-06-06 07:17:30 +00:00
robot-clickhouse-ci-1
4cf5a13e5b
Merge pull request #50160 from ClickHouse/prio-runners
Refactor lambdas, prepare to prio runners
2023-06-06 07:29:28 +02:00
avogar
e8c6c7967b Move attaching gdb to separate lib 2023-06-05 18:21:40 +00:00
Nikita Taranov
e548dce123 fix 2023-06-03 00:08:47 +02:00
Nikita Taranov
4506299d73 impl 2023-06-02 23:14:09 +02:00
pufit
0d98a46326 Fix KeyError in cherry-pick 2023-06-02 13:02:18 -04:00
Mikhail f. Shiryaev
9bd0a53e7c
Get only online instances in get_lost_ec2_instances 2023-06-02 12:27:29 +02:00
Mikhail f. Shiryaev
3c13eaa159
Simplify get_dead_runners_in_ec2 a little bit, update runners 2023-06-02 12:27:28 +02:00
Mikhail f. Shiryaev
b775b5cfd6
Move all CI runners metrics into one namespace 2023-06-02 12:27:27 +02:00
Mikhail f. Shiryaev
e657e2ba10
Additional logging in autoscale_runners_lambda 2023-06-02 12:27:26 +02:00
Mikhail f. Shiryaev
b7c5fdab77
Move lambda package building to public.ecr.aws/lambda/python for compatibility 2023-06-02 12:27:26 +02:00
Mikhail f. Shiryaev
0ddd53088d
Add a new runner type for ci metrics and autoscaling 2023-06-02 12:27:24 +02:00
Mikhail f. Shiryaev
e8b03d7498
Move insert part for ClickHouseHelper to shared 2023-06-01 21:03:57 +02:00
Mikhail f. Shiryaev
2dca0eac1b
Delete __init__.py in lambda directories to break subpackage 2023-06-01 21:03:56 +02:00
Mikhail f. Shiryaev
acb9531ebf
Unify and put GH access token to the lambda_shared 2023-06-01 21:03:56 +02:00
Mikhail f. Shiryaev
0fa6a84161
Move the stuff related to runners to lambda_shared 2023-06-01 21:03:55 +02:00
Mikhail f. Shiryaev
7f08f218d9
Create lambda_shared package for lambdas 2023-06-01 21:03:53 +02:00
Mikhail f. Shiryaev
484c91c47e
Add DRY_RUN and configurable PY_VERSION to lambda deployment 2023-06-01 21:03:53 +02:00
Mikhail f. Shiryaev
27941b4d26
Decrease the time window for autoscale_runners_lambda 2023-06-01 21:03:52 +02:00
Mikhail f. Shiryaev
db02938411
Do not count unfinished tasks with conclusion=None 2023-06-01 21:03:51 +02:00
Mikhail f. Shiryaev
855afb56f9
Get instances for the region to not kill a fresh one 2023-06-01 21:03:50 +02:00
Mikhail f. Shiryaev
bf9b563e0b
Improve caching mechanism for token, add cached instances 2023-06-01 21:03:50 +02:00
Mikhail f. Shiryaev
7bf9089dcd
Increase access_token cached time 2023-06-01 21:03:49 +02:00
Mikhail f. Shiryaev
985cd8fc8a
Improve events logging 2023-06-01 21:03:48 +02:00
Nikita Taranov
e5c21e588a
Update app.py (#50407) 2023-06-01 15:19:47 +02:00
Mikhail f. Shiryaev
9089360688
Merge pull request #50415 from ClickHouse/build-report-pending
Mark the builds without results as pending
2023-06-01 14:53:09 +02:00
Mikhail f. Shiryaev
de7179876e
Merge pull request #47151 from ClickHouse/increase-vm-max_map_count
Update ubuntu_ami_for_ci.sh
2023-06-01 13:54:21 +02:00
Mikhail f. Shiryaev
6a5ae6558a
Add comment for usage of prepare-ci-ami.sh 2023-06-01 12:58:03 +02:00
Mikhail f. Shiryaev
bcbce15cde
Mark the builds without results as pending 2023-06-01 08:51:44 +02:00
Mikhail f. Shiryaev
653da5f002
Reduce the unzip verbosity 2023-05-31 18:23:21 +02:00
Mikhail f. Shiryaev
a96a845e23
Add comment to prepare-ci-ami.sh, add a success at the end 2023-05-31 17:12:49 +02:00
pufit
08d3887832
Add re-creation for cherry-pick PRs (#50373)
* Add recreation for cherry-pick PRs.
Small PR comment fix.

* Automatic style fix

---------

Co-authored-by: robot-clickhouse <robot-clickhouse@users.noreply.github.com>
Co-authored-by: Nikita Mikhaylov <mikhaylovnikitka@gmail.com>
2023-05-31 13:04:29 +02:00
Mikhail f. Shiryaev
db3a050911
Seems like buildx needs explicit install since some time 2023-05-31 00:22:12 +02:00
Mikhail f. Shiryaev
56dc35d563
Merge branch 'master' into increase-vm-max_map_count 2023-05-30 19:24:18 +02:00
Mikhail f. Shiryaev
bcd57078ce
Update the runner version 2023-05-30 16:36:54 +02:00
Nikolai Kochetov
b5b261b22c Merge branch 'master' into refactor-subqueries-for-in 2023-05-25 21:22:06 +02:00
pufit
4ce722854b Fix URL in backport comment 2023-05-25 11:51:47 -04:00
Mikhail f. Shiryaev
a9d5b68946
Merge pull request #50197 from ClickHouse/fix-codebrowser
Fix codebrowser by using clang-15 image
2023-05-24 23:39:13 +02:00
Mikhail f. Shiryaev
e117347ca0
Attempt to fix codebrowser with using clang-15 2023-05-24 23:15:38 +02:00
Alexander Tokmakov
2c0c1ba4af
add comments to build reports (#50200) 2023-05-24 23:29:36 +03:00
pufit
8875b98e2a Added default value for --from-repo arg 2023-05-24 13:47:50 -04:00
pufit
8df7a69147
Automatic backports of important fixes to cloud-release
* Automatic backports to cloud-release

---------

Co-authored-by: robot-clickhouse <robot-clickhouse@users.noreply.github.com>
2023-05-24 18:59:18 +02:00
Nikolai Kochetov
ff6400f91a Merge branch 'master' into refactor-subqueries-for-in 2023-05-22 13:07:51 +02:00
Timur Solodovnikov
c7ab59302f
Set allow_experimental_query_cache setting as obsolete (#49934)
* set allow_experimental_query_cache as obsolete

* add tsolodov to trusted contributors

* CI linter

---------

Co-authored-by: Nikita Mikhaylov <mikhaylovnikitka@gmail.com>
2023-05-17 20:03:42 +02:00
Mikhail f. Shiryaev
4b7aa30017
The best effor to get the current job ID and URL
This partially addresses https://github.com/actions/runner/issues/2577
2023-05-12 19:22:52 +02:00
Mikhail f. Shiryaev
c375f948d0
Fix typing in S3Helper.fast_parallel_upload_dir 2023-05-12 19:22:52 +02:00
Mikhail f. Shiryaev
338435a52d
Additional check for FATAL in logs, do not exit with code 1 on errors 2023-05-12 19:22:51 +02:00
Mikhail f. Shiryaev
0f70ad56a9
Do not upload failed site to the S3 2023-05-12 19:22:50 +02:00
Mikhail f. Shiryaev
eb122487ad
Add logs to the report, do not chown unnecessary 2023-05-12 19:22:48 +02:00
Mikhail f. Shiryaev
6a56662f60
Improve codebrowser check
- Add sccache
- Report a proper status in report and exit code
- Launch it as a current user
- Fix report link
2023-05-12 19:22:48 +02:00
Mikhail f. Shiryaev
fba7120d3d
Fix s3_helper logging and pool setup 2023-05-12 19:22:44 +02:00
Robert Schulze
b9c185af44
Merge pull request #49678 from azat/build/llvm-16
Switch to LLVM/clang 16 (16.0.3)
2023-05-12 13:47:36 +02:00
Mikhail f. Shiryaev
8a664e2cac
Upgrade all lambdas runtime to 3.10, fix requirements
The recent update of urllib3 completely broken the world

- https://github.com/urllib3/urllib3/issues/2168
- https://github.com/boto/botocore/issues/2926
- https://github.com/psf/requests/issues/6443
2023-05-11 16:09:59 +02:00
Mikhail f. Shiryaev
d890114c5e
Print debugging information in workflow_jobs_lambda 2023-05-11 14:21:07 +02:00
Mikhail f. Shiryaev
d605f82798
Replace old release with new packages in suspicious patterns 2023-05-10 12:36:04 +02:00
Mikhail f. Shiryaev
f6372dd3fc
Do not launch PRs w/o "can be tested" 2023-05-10 12:14:44 +02:00
Azat Khuzhin
c37fe64040 Merge remote-tracking branch 'u/master' into build/llvm-16 2023-05-08 22:33:52 +02:00
Nikolai Kochetov
fc02e9efc9 update fasttest a bit 2023-05-08 17:23:46 +00:00
Alexey Milovidov
54163b87ec
Merge pull request #47977 from ClickHouse/rs/non-experimental-qc
Mark query cache as non-experimental
2023-05-06 19:49:23 +03:00
Robert Schulze
0598bfd79e
Mark query cache as non-experimental 2023-05-06 13:58:40 +00:00
Dmitry Novik
a3b3b78eb0 EXPECTED_TO_FAIL -> NOT_FAILED 2023-05-05 17:53:15 +00:00
Dmitry Novik
02520e9358 Make EXPRECTED_TO_FAIL red 2023-05-05 17:37:52 +00:00
Mikhail f. Shiryaev
90fe9dd1f2
Skip mergeable check only on a proper description 2023-05-03 22:38:34 +02:00
Mikhail f. Shiryaev
a67950d138
Check the PRs body directly in lambda, without rerun 2023-05-03 22:38:33 +02:00
Mikhail f. Shiryaev
242a210358
Create "CI running" report always 2023-05-02 14:13:23 +02:00
Mikhail f. Shiryaev
08d203e158
Sanitize "Flaky tests", remove trailing dots 2023-05-02 12:42:26 +02:00
Mikhail f. Shiryaev
b423e04ab0
Fix the global ROBOT_TOKEN, do not retry unnecessary 2023-05-02 12:37:05 +02:00
Mikhail f. Shiryaev
ca19f20aff
Do not fail the run on error updating status comment 2023-05-02 12:34:40 +02:00
Alexey Milovidov
dd7282ea5c
Update ci_config.py 2023-05-01 16:05:14 +03:00
Mikhail f. Shiryaev
b385b51f07
Merge pull request #49314 from ClickHouse/fallback-auth-gh-api
Fallback auth gh api
2023-04-29 09:36:49 +02:00
Mikhail f. Shiryaev
d336bee056
Fix the Logs header for case when only log_urls is set 2023-04-28 23:52:35 +02:00
Mikhail f. Shiryaev
886ba0b01d
Replace markdown with pure HTML for check descriptions 2023-04-28 23:52:34 +02:00
Mikhail f. Shiryaev
20c8f0f1b4
Use only one comment for all statuses in the PR 2023-04-28 23:52:34 +02:00
Mikhail f. Shiryaev
61c6c61638
Rework the status comment to be a digest with the full report available
by link
2023-04-28 23:52:33 +02:00
Mikhail f. Shiryaev
e8cf417350
Use NotSet in post_commit_status for optional arguments 2023-04-28 23:52:33 +02:00
Mikhail f. Shiryaev
c66e50f283
Remove options from S3Helper, they are global 2023-04-28 23:52:32 +02:00
Mikhail f. Shiryaev
f411be8c37
Rename Run Check to CI running 2023-04-28 23:52:31 +02:00
Mikhail f. Shiryaev
df33b99907
Rework post_commit_status, add set_status_comment there 2023-04-28 23:52:31 +02:00
Mikhail f. Shiryaev
d58abd3c37
Add a function get_repo with global state 2023-04-28 23:52:30 +02:00
Mikhail f. Shiryaev
4405459925
Refactor RerunHelper to reduce API calls 2023-04-28 23:52:30 +02:00
Mikhail f. Shiryaev
19fd5f9c3c
Add descriptions for the status comment 2023-04-28 23:52:29 +02:00
Mikhail f. Shiryaev
eb11fb02fa
Clean up ast_fuzzer_check.py 2023-04-28 23:52:29 +02:00
Mikhail f. Shiryaev
ccd4fd7e64
Use commit_status_helper.post_commit_status everywhere 2023-04-28 23:52:28 +02:00
Mikhail f. Shiryaev
144ebd3d36
Implement main() in run_check.py 2023-04-28 23:52:27 +02:00
Mikhail f. Shiryaev
e670c51680
Unify commit.create_status and post_commit_status 2023-04-28 23:52:26 +02:00
Mikhail f. Shiryaev
01e70c64c2
Add a fallback to authenticated requests to GH API 2023-04-28 18:46:33 +02:00
Mikhail f. Shiryaev
8c60b60916
Receive the best robot token only once 2023-04-28 16:01:53 +02:00
Mikhail f. Shiryaev
9a7f09bae4
Avoid splitting check_name into subdirectories 2023-04-26 01:11:05 +02:00
Mikhail f. Shiryaev
a6664e6b08
Add typing, capitalize only the first letter w/o lowering the rest 2023-04-26 00:55:05 +02:00
Mikhail f. Shiryaev
ed8ca018bd
Add a note regarding private/public repo to logs 2023-04-25 19:11:20 +02:00
alesapin
084f76ab3e
Merge pull request #49133 from ClickHouse/unanonymized-workflow-jobs-logs
Improve some lambdas
2023-04-25 12:58:18 +02:00
Mikhail f. Shiryaev
2ab313e6b4
Update the style_checker upscale speed 2023-04-25 12:42:06 +02:00
Mikhail f. Shiryaev
6032bacd0f
Get unanonymized logs in workflow_jobs_lambda 2023-04-25 12:37:26 +02:00
Dmitry Novik
0d3e896f57
Merge pull request #48719 from ClickHouse/analyzer-ci
Add CI run with new analyzer enabled
2023-04-24 17:36:18 +02:00
Mikhail f. Shiryaev
222176639d
Update pr_info.labels in commit_helper lebel helpers 2023-04-24 11:03:25 +02:00
Mikhail f. Shiryaev
caee710fcc
Enforce documentation change for a new-feature PR 2023-04-24 10:41:38 +02:00
Dmitry Novik
1fdee1a2ab
Merge branch 'master' into analyzer-ci 2023-04-21 15:43:16 +02:00
Alexey Milovidov
2dbe4f80f2 Do not allow incomplete changelog entries 2023-04-21 12:59:47 +02:00
Alexey Milovidov
95e69b6f85 Do not allow incomplete changelog entries 2023-04-21 12:59:17 +02:00
Mikhail f. Shiryaev
c28e8438e3
Fail Fast tests on next run if previous running has failed 2023-04-20 14:51:53 +02:00
Dmitry Novik
b2ed99f6b3
Merge branch 'master' into analyzer-ci 2023-04-18 13:07:45 +02:00
Dmitry Novik
5557f4524c Fix string formatting 2023-04-18 08:44:46 +00:00
robot-clickhouse
f6127a68ac Automatic style fix 2023-04-18 00:56:29 +00:00
Dmitry Novik
6197d3d55f Pass volume only for analyzer 2023-04-18 00:12:30 +00:00
Dmitry Novik
61e552d660 Fix used build 2023-04-17 15:23:02 +00:00
Sema Checherinda
0354a964cd turn on all CI back 2023-04-17 14:07:56 +02:00
Sema Checherinda
c3b9d5fe6e introduce sqllogic runner 2023-04-16 13:57:24 +02:00
Dmitry Novik
b04a4ec287 Fix used image 2023-04-14 15:25:06 +02:00
Ilya Yatsishin
d2351df69e
Update tests/ci/functional_test_check.py 2023-04-14 15:13:10 +02:00
Dmitry Novik
3df33d74f5 Get rid of another docker image 2023-04-13 17:22:12 +02:00
Aleksei Filatov
0a1815b9bb
Add new trusted contributors (#48715)
Co-authored-by: Nikita Mikhaylov <mikhaylovnikitka@gmail.com>
2023-04-12 19:38:05 +02:00
Dmitry Novik
360fc59694 Handle known broken tests 2023-04-12 16:46:30 +00:00
Dmitry Novik
f8e4d6bb41 Add CI run with new analyzer enabled 2023-04-12 15:18:28 +00:00
Mikhail f. Shiryaev
cdceac6624
Reduce number of return statements 2023-04-11 16:10:52 +02:00
Mikhail f. Shiryaev
c29aa0cd49
Fail the build on a wrong changelog category 2023-04-11 15:34:33 +02:00
Mikhail f. Shiryaev
57a1919594
Partially revert e0252db8d and fix pr-bugfix labeling 2023-04-11 12:40:04 +02:00
Alexander Tokmakov
48b23dd012
Fix race between DROP MatView and RESTART REPLICAS (#47863)
* fix race between drop mv and restart replicas

* unrelated: fix bad exception messages

* fix test

* fix

* fix

* fix

* fix

* fix test

* fix

* fix test

* Update 02437_drop_mv_restart_replicas.sh

* fix tests
2023-04-01 15:26:00 +03:00
Mikhail f. Shiryaev
7ff582b963
Deprecate ENABLE_CCACHE and replace it by COMPILER_CACHE 2023-03-29 23:33:31 +02:00
Mikhail f. Shiryaev
79177e1988
Enable sccache in fasttests 2023-03-29 23:33:29 +02:00
Mikhail f. Shiryaev
fcc14ae724
Enable s3 caching 2023-03-29 23:33:27 +02:00
Alexey Milovidov
e0252db8d4 No prestable 2023-03-27 12:19:32 +02:00
robot-clickhouse
6ac86ed3e0
Merge pull request #47945 from ClickHouse/artifacts-s3-prefix
Artifacts s3 prefix
2023-03-26 00:15:50 +01:00
Mikhail f. Shiryaev
290ee6bbf1
Merge pull request #47790 from CurtizJ/apply-black-to-all-files
Apply black formatter to all python scripts
2023-03-24 14:20:50 +03:00
Mikhail f. Shiryaev
4f2c336cfc
Update black version, run it for the repository 2023-03-24 11:54:38 +01:00
Mikhail f. Shiryaev
c4870ad7bc
Move PRs artifacts to the PRs prefix in S3 2023-03-24 08:54:06 +01:00
Mikhail f. Shiryaev
8ff01429e1
Upload build reports to S3_TEST_REPORTS_BUCKET 2023-03-23 15:49:59 +01:00
Robert Schulze
b3b097286f
Merge branch 'master' into rs/randomized-qc-settings 2023-03-23 13:33:38 +01:00
robot-clickhouse
d690e0f4ec Automatic style fix 2023-03-22 12:55:24 +00:00
Robert Schulze
ea2d7e0035
Decouple QC enablement from 'join_use_nulls' 2023-03-22 12:46:22 +00:00
Robert Schulze
b019ed90be
Randomize query cache settings in stress tests 2023-03-22 11:11:03 +00:00
Mikhail f. Shiryaev
6409ca9fc9
Merge pull request #47857 from ClickHouse/cherry-pick-ping
Comment stale cherry-pick PRs once a day to remind for resolving conflicts
2023-03-22 13:36:35 +03:00
Mikhail f. Shiryaev
d88f71f2b9
Comment stale cherry-pick PRs to resolve conflicts 2023-03-21 18:28:29 +01:00
robot-ch-test-poll2
5e763ccad1
Merge pull request #47822 from ClickHouse/merge-pr-logging
Place short return before big block, improve logging
2023-03-21 13:10:57 +01:00
Mikhail f. Shiryaev
b9dca4e0df
Place short return before big block, improve logging 2023-03-21 11:29:16 +01:00
Mikhail f. Shiryaev
15d02f77cf
Decrease scale_down ratio for faster deflation 2023-03-20 21:03:47 +01:00
Mikhail f. Shiryaev
5eacb6b3be
Fix the keeping the latest review instead of the first 2023-03-20 18:40:39 +01:00
Mikhail f. Shiryaev
f881da249f
Only valid Reviews.STATES overwrite existing reviews 2023-03-20 18:39:58 +01:00
Mikhail f. Shiryaev
e39e0294de
Merge pull request #47760 from ClickHouse/backports-fuse
Add a fuse for backport branches w/o a created PR
2023-03-20 17:48:07 +03:00
Mikhail f. Shiryaev
ae88a31100
Add a fuse for backport branches w/o a created PR 2023-03-20 13:45:23 +01:00
Mikhail f. Shiryaev
a3096a4583
Clarify steps for reopened cherry-pick PRs 2023-03-20 12:36:35 +01:00