Commit Graph

730 Commits

Author SHA1 Message Date
Mikhail f. Shiryaev
3142ae998b
Add option to skip redownload, push tgz 2022-01-19 11:41:33 +01:00
Mikhail f. Shiryaev
13368bf708
Fix some linter issues 2022-01-19 11:41:32 +01:00
Mikhail f. Shiryaev
7fd629004b
Improve push_to_artifactory
- Add tgz archives
- Add --all argument for all packages download
- Add -n to skip pushing to artifactory
- Add help for artifactory-url
- Make 3.8 compatible
- Add cleanup `refs/tags/` from --release argument
2022-01-19 11:41:30 +01:00
Mikhail f. Shiryaev
396b504397
Add some helping comments for API endpoints 2022-01-17 11:06:40 +01:00
Mikhail f. Shiryaev
91caf07b20
Apply black to workflow_approve_rerun_lambda 2022-01-15 18:54:58 +01:00
Mikhail f. Shiryaev
4c7a8abd20
Add PullRequestCI to NEED_RERUN_WORKFLOWS 2022-01-15 18:53:37 +01:00
Maksim Kita
8aed06c922
Merge pull request #33663 from amosbird/addmyself
Add amosbird to trusted user.
2022-01-15 14:10:37 +01:00
Amos Bird
4da2a660ec
Add amosbird to trusted user. 2022-01-15 20:23:55 +08:00
alexey-milovidov
c3972f000d
Merge pull request #33614 from ClickHouse/safe-names-in-blog
Use safe page URL for a blog post
2022-01-15 02:29:37 +03:00
Mikhail f. Shiryaev
a8203dd01e
Merge pull request #33623 from ClickHouse/metrics-lambda
Add an exception message example
2022-01-14 10:06:33 +01:00
Mikhail f. Shiryaev
a63494cb66
Add an exception message example 2022-01-14 10:05:19 +01:00
alesapin
b6937f7462
Merge pull request #33616 from ClickHouse/metrics-lambda
Metrics lambda: fix deletion
2022-01-14 11:53:13 +03:00
Mikhail f. Shiryaev
bca9f56ccd
Improve logging in docs_check.py 2022-01-14 09:15:41 +01:00
Mikhail f. Shiryaev
52594179e3
Fix broken receiver get_dead_runners_in_ec2 2022-01-14 00:51:37 +01:00
Mikhail f. Shiryaev
632b87ac2b
Use key from the secret manager if possible 2022-01-14 00:49:49 +01:00
Mikhail f. Shiryaev
545bee477f
Apply black formatter 2022-01-13 20:52:02 +01:00
alesapin
0804e04558
Merge pull request #32911 from ClickHouse/deb-aarch64
Deb aarch64
2022-01-13 19:11:19 +03:00
Mikhail f. Shiryaev
92099302e7
Use stdin for docker login to not expose a password 2022-01-13 16:57:33 +01:00
Mikhail f. Shiryaev
080c56d91a
Fix pylint for build_check.py 2022-01-13 15:13:58 +01:00
Mikhail f. Shiryaev
89415ff415
Fix pylint for run_check.py 2022-01-13 14:51:48 +01:00
Mikhail f. Shiryaev
25476e2d7d
Restore existing static builds links 2022-01-13 13:55:51 +01:00
Mikhail f. Shiryaev
39012a42aa
Restore description checking 2022-01-13 12:17:09 +01:00
Mikhail f. Shiryaev
ce01dea0fe
Apply black to run_check.py 2022-01-13 12:06:50 +01:00
Mikhail f. Shiryaev
abd16ebf32
Force success aarch stateless tests 2022-01-12 17:07:59 +01:00
Mikhail f. Shiryaev
0e536d5ab0
Apply black to status_helper.py 2022-01-12 17:07:58 +01:00
Mikhail f. Shiryaev
11ede97d0d
Add func-testers-aarch64 group to metrics 2022-01-12 17:07:57 +01:00
Mikhail f. Shiryaev
6373943451
Apply black to metrics lambda app 2022-01-12 17:07:56 +01:00
Mikhail f. Shiryaev
2ae87ad0df
Build docker amd64 and arm64 separately and merge together later 2022-01-12 17:07:55 +01:00
Mikhail f. Shiryaev
341feb5945
Add necessary 'before' field for a dummy object 2022-01-12 17:07:55 +01:00
Mikhail f. Shiryaev
7bc89c4f50
Use cache feature of docker buildx 2022-01-12 17:07:53 +01:00
Mikhail f. Shiryaev
2a83a1c05d
Build dependent images from the current version 2022-01-12 17:07:52 +01:00
Mikhail f. Shiryaev
f11bfdb2ea
Add prebuilding for docker buildx builder 2022-01-12 17:07:51 +01:00
Mikhail f. Shiryaev
fe130bc547
Use buildx for docker images, improve typing 2022-01-12 17:07:50 +01:00
Mikhail f. Shiryaev
43f39bc051
Add typing to docker_images_check.py and ignore dependencies errors 2022-01-12 17:07:50 +01:00
Mikhail f. Shiryaev
3c6f14ae63
Apply black to docker_images_check.py 2022-01-12 17:07:49 +01:00
Mikhail f. Shiryaev
44e02fa075
Add functional tests for aarch64 2022-01-12 17:07:48 +01:00
Mikhail f. Shiryaev
830edf76cc
Don't upload splitted binaries to builds.clickhouse.com 2022-01-12 14:57:11 +01:00
Mikhail f. Shiryaev
9188b8afc2
Fix logic for upload upload_master_static_binaries 2022-01-12 14:30:32 +01:00
Mikhail f. Shiryaev
325c786f68
Add uploading binary builds from master to static URLs 2022-01-12 13:17:54 +01:00
Mikhail f. Shiryaev
3614b0c4ed
Slightly refactor build_check.py 2022-01-12 13:17:21 +01:00
Mikhail f. Shiryaev
eaf3bdabf5
Add typing to CI_CONFIG 2022-01-12 13:14:31 +01:00
Mikhail f. Shiryaev
b98e0be869
Use a proper branch name for PRInfo w/ number==0 2022-01-12 12:31:13 +01:00
Mikhail f. Shiryaev
80a452cca0
Apply black on build_check.py and tee_popen.py 2022-01-11 19:36:34 +01:00
alexey-milovidov
146639273f
Update app.py 2022-01-09 19:20:06 +03:00
alexey-milovidov
4477b7d2f1
Update run_check.py 2022-01-09 19:19:58 +03:00
alesapin
6a4d5a985b Rerun docs release check as well 2022-01-07 12:05:38 +03:00
Alexey Milovidov
2d3f53ec5c Merge branch 'dotnet-integration-test' of github.com:yandex/ClickHouse into dotnet-integration-test 2022-01-04 22:49:58 +03:00
Alexey Milovidov
184033b782 Trying to make it work 2022-01-04 22:49:41 +03:00
alexey-milovidov
c2ed92d75d
Update pr_info.py 2021-12-31 14:39:28 +03:00
Mikhail f. Shiryaev
fccc4dade1
Fix (?) encoding issue in #33331 2021-12-30 22:19:06 +01:00
alexey-milovidov
e879aca58b
Update run_check.py 2021-12-29 23:21:47 +03:00
alesapin
4a1819c4d2 Fix jepsen check 2021-12-29 22:44:41 +03:00
alesapin
948fde52a7 Add retries for github api 2021-12-28 13:13:51 +03:00
alesapin
d188771c05
Merge pull request #33222 from ClickHouse/fix_cancell_lambda
Fix cancel lambda
2021-12-27 13:27:40 +03:00
alesapin
33fd1a44ac Fix cancel lambda 2021-12-27 13:26:16 +03:00
alesapin
342c8231fa Better 2021-12-27 10:33:44 +03:00
alesapin
e683fa146b More debug info in jepsen check 2021-12-27 10:33:25 +03:00
alesapin
9687f1603a Better 2021-12-24 21:34:24 +03:00
alesapin
bd7f5ea50f Better 2021-12-24 21:22:29 +03:00
alesapin
46569e7a5b Better 2021-12-24 21:02:40 +03:00
alesapin
9d83b904de Add pretty print unknow event 2021-12-24 20:57:32 +03:00
alesapin
00666a6a4c
Merge pull request #32998 from ClickHouse/add_jepsen_test
Add Keeper Jepsen check to new CI
2021-12-24 17:25:20 +03:00
alesapin
05648434b9 Fix label 2021-12-24 15:56:38 +03:00
alesapin
42461f453e Add a separate workflow 2021-12-24 15:56:03 +03:00
alesapin
119b8628e5
Merge pull request #33135 from ClickHouse/fix_pvs_studio
Fix PVS check
2021-12-24 15:34:24 +03:00
alesapin
2a402f7df1
Merge pull request #33136 from ClickHouse/fix_error_report_in_build_check
Upload build artifact in case of build failures
2021-12-24 15:00:50 +03:00
alesapin
3dbf7a9da9 More clear exception 2021-12-24 13:35:40 +03:00
alesapin
9461b7dba2 Finally fix jepsen 2021-12-24 13:32:32 +03:00
alesapin
d68d26e4c1 Fix woboq codebrowser 2021-12-24 13:05:09 +03:00
alesapin
1a3cdc3388 Upload build artifact in case of build failures 2021-12-24 12:56:09 +03:00
alesapin
e3d8457a6f Fix PVS 2021-12-24 12:41:01 +03:00
alesapin
be8187df09 Fix report upload 2021-12-22 19:16:54 +03:00
alesapin
7e714641ac Add newline 2021-12-22 13:46:41 +03:00
alesapin
e63b2f0de8 Merge branch 'master' into add_jepsen_test 2021-12-22 13:30:48 +03:00
alesapin
0a716b9d9a
Merge pull request #33036 from ClickHouse/fix_can_be_tested
Make `can be tested` great again
2021-12-22 13:26:56 +03:00
alesapin
e31be8f056 Better check in lambda 2021-12-22 12:25:16 +03:00
alesapin
20e287f5b6 Followup 2021-12-22 11:15:31 +03:00
alesapin
b300de6a4a Better 2021-12-22 11:13:04 +03:00
alesapin
7a00bc96ed Make great again 2021-12-22 10:54:50 +03:00
alesapin
ca9526e3a4 Stupid fix 2021-12-22 00:39:51 +03:00
alesapin
bc03e5e10f
Merge pull request #32983 from ClickHouse/add_woboq_build
Trying to add woboq to new CI
2021-12-21 23:05:38 +03:00
alesapin
84811a55c5 Fix path 2021-12-21 18:17:53 +03:00
alesapin
3ab250a869 Add region name: 2021-12-21 16:47:23 +03:00
alesapin
37d90e071f Group params 2021-12-21 16:16:01 +03:00
alesapin
49a50bbdc9 Merge branch 'master' into add_jepsen_test 2021-12-21 16:15:40 +03:00
alesapin
45aef649a1 Add backport workflow rerun 2021-12-21 16:13:44 +03:00
alesapin
af7cd00e30 Don't use too powerful key 2021-12-21 16:05:02 +03:00
alesapin
627bd2cec4 Add Keeper Jepsen check to new CI 2021-12-21 15:07:10 +03:00
alesapin
a89a05c0da Finally woboq is ready 2021-12-21 14:16:21 +03:00
alesapin
532b01c371 Followup 2021-12-21 10:49:22 +03:00
alesapin
e194b38096 Followup 2021-12-20 22:54:04 +03:00
alesapin
62eb504029 Trying to add woboq 2021-12-20 22:43:37 +03:00
Mikhail f. Shiryaev
6f6b7c7bac
Merge pull request #32905 from ClickHouse/add_dockerhub_proxy
Add dockerhub proxy to CI
2021-12-17 14:31:50 +01:00
alesapin
f02a40bbdc Rename script and add template for dockerhub proxy 2021-12-17 13:48:42 +03:00
alesapin
47c907d41a Add dockerhub proxy 2021-12-17 13:20:48 +03:00
alesapin
25fd29619b
Merge pull request #31882 from pkit/refactor_ci_tests1
refactor CI tests
2021-12-17 11:18:35 +03:00
alexey-milovidov
e929175dae
Update run_check.py 2021-12-17 08:55:46 +03:00
alexey-milovidov
e834d655a5
Merge pull request #32415 from ClickHouse/arm64_workers
Add arm64/aarch64 tests
2021-12-17 08:46:52 +03:00
vdimir
a0afe67777
Add cwurm to TRUSTED_CONTIBUTORS 2021-12-16 16:13:40 +03:00
Mikhail f. Shiryaev
9a6e8dc615
Build rpm and tgz for aarch64 too 2021-12-16 13:00:32 +01:00
Mikhail f. Shiryaev
eb5a26891f
Add aarch64 deb package 2021-12-16 12:42:49 +01:00
Mikhail f. Shiryaev
3ae654168c
Apply black formatter 2021-12-16 12:42:48 +01:00
Mikhail f. Shiryaev
2ae5eb4cfa
Merge pull request #32205 from ClickHouse/team_keys_lambda
Add docker container for AWS lambda
2021-12-16 11:52:02 +01:00
Mikhail f. Shiryaev
31a5afe855
Use threads to speed-up parallel requests 2021-12-16 11:50:48 +01:00
Mikhail f. Shiryaev
ccf6210eb1
Add docker container for AWS lambda 2021-12-16 11:50:41 +01:00
Mikhail f. Shiryaev
1d7bd3bf9b
Merge pull request #32800 from ClickHouse/aws-workers
Improve workers init
2021-12-16 11:48:54 +01:00
alesapin
cf0241a2d5 Prevent docs release of being too smart 2021-12-16 10:59:59 +03:00
alesapin
38f2e4c8ac Prevent docs release of being too smart 2021-12-16 10:59:42 +03:00
alesapin
8d6bea91e8
Update pr_info.py 2021-12-15 16:35:53 +03:00
alesapin
5430f1a413
Update pr_info.py 2021-12-15 16:26:46 +03:00
alesapin
73a593b676
Merge branch 'master' into refactor_ci_tests1 2021-12-15 16:11:28 +03:00
alesapin
d962d92896
Merge pull request #32745 from ClickHouse/fix_special_build_check
Better control build artifacts
2021-12-15 14:51:02 +03:00
Mikhail f. Shiryaev
546a03281f
Rework image building:
- Make it arch-agnostic
- Add necessary packages
- Update runner version
- Get public keys of ClickHouse/core members
2021-12-15 11:19:08 +01:00
alesapin
2c78e7ed72 Fix performance build 2021-12-15 11:07:12 +03:00
alesapin
2e8b75ade4 Fix configs 2021-12-15 10:45:27 +03:00
alesapin
2753fe01a3 Merge branch 'master' into fix_special_build_check 2021-12-15 10:35:28 +03:00
Mikhail f. Shiryaev
97301a19b9
Unify init scripts in one 2021-12-14 18:44:51 +01:00
alesapin
e124e5807f Fix build performance 2021-12-14 19:29:30 +03:00
alesapin
c46c912507 Followup 2021-12-14 19:16:22 +03:00
mergify[bot]
68c96f4dd4
Merge branch 'master' into add_performance_tests 2021-12-14 11:04:41 +00:00
alesapin
7b29377338 Better control build artifacts 2021-12-14 13:40:03 +03:00
alesapin
d5925d7e2d Trying stress testers 2021-12-14 13:31:31 +03:00
Kruglov Pavel
d065b8233d
Update backport.py 2021-12-13 20:40:06 +03:00
alesapin
242149d881 Rerun release workflows 2021-12-13 18:54:14 +03:00
alesapin
949e68e2ac
Merge pull request #32696 from vdimir/trust-cont-lower
Check trusted contributors in lowercase
2021-12-13 18:20:16 +03:00
alesapin
856dd3735c Merge branch 'master' into add_performance_tests 2021-12-13 18:09:59 +03:00
vdimir
0c83b14c37
Check trusted contributors in lowercase 2021-12-13 17:34:17 +03:00
alesapin
fad904b0aa Merge branch 'master' into add_performance_tests 2021-12-13 14:32:35 +03:00
alesapin
1f5157e240
Merge pull request #32618 from ClickHouse/upload_logs_of_failed_task
CI: upload logs if task failed
2021-12-13 12:42:39 +03:00
alesapin
b34f5210e7 Followup 2021-12-13 12:04:08 +03:00
alesapin
cf281b10dc Split perf test into multiple checks 2021-12-13 11:59:58 +03:00
alesapin
c1d3323e25 Rerun workflows more times 2021-12-13 10:48:57 +03:00
Alexander Tokmakov
62015ba610 Merge branch 'master' into upload_logs_of_failed_task 2021-12-12 15:20:25 +03:00
Alexander Tokmakov
e8613ba386 upload logs of failed task 2021-12-12 15:09:44 +03:00
alesapin
eb51d9550d
Merge branch 'master' into add_performance_tests 2021-12-12 01:01:13 +03:00
alesapin
84bccb83bd Trying to add debug info 2021-12-10 19:38:20 +03:00
alesapin
a6e55b00e3
Revert "Revert "Split long tests into multiple checks"" 2021-12-10 18:39:02 +03:00
alesapin
dcbba460c1
Revert "Split long tests into multiple checks" 2021-12-10 18:38:42 +03:00
alesapin
a2aa0bc6cc
Merge pull request #32496 from ClickHouse/smaller_checks
Split long tests into multiple checks
2021-12-10 17:42:14 +03:00
alesapin
4eb2dee857 Don't use tmpfs 2021-12-10 16:09:00 +03:00
alesapin
2c7bb56a2c Split integration tests into parts 2021-12-10 13:22:29 +03:00
alesapin
f655b43f5a Merge branch 'master' into add_performance_tests 2021-12-10 12:32:43 +03:00
alesapin
94a557a80e Trying another host 2021-12-10 12:31:09 +03:00
alesapin
32f78f2ba4 Better check name 2021-12-10 12:09:11 +03:00
alesapin
4c1babee05 Split long functional tests to multiple checks 2021-12-10 12:07:24 +03:00
alesapin
dc21d423d1
Merge pull request #32494 from ClickHouse/fix_integration_tests_images
Fix integration tests docker images path
2021-12-10 10:56:15 +03:00
alesapin
db83d82f5a Fix integration tests path 2021-12-10 10:54:37 +03:00
tavplubix
85e53b1b1f
Try fix attaching gdb in tests (#32448)
* attach gdb with sudo

* fix

* Update run.sh
2021-12-10 00:12:45 +03:00
alesapin
5b06e30ea2
Merge pull request #32462 from ClickHouse/automatic_rerun
Add automatic workflow rerun
2021-12-09 18:31:59 +03:00
alesapin
19f8b416bc Add automatic workflow rerun 2021-12-09 16:28:15 +03:00
alesapin
08c699cbdd
Merge pull request #32438 from ClickHouse/fix_image_for_integration_tests
Fix images names in integration tests
2021-12-09 15:38:50 +03:00
alesapin
af6ce89e95 Better report 2021-12-09 15:02:13 +03:00
alesapin
7ea7aa8db8 Fix build check 2021-12-09 12:17:03 +03:00
alesapin
67d89929a3 Add ramdrive 2021-12-09 12:04:05 +03:00
alesapin
b62de9b2f4 Merge branch 'master' into add_performance_tests 2021-12-09 11:40:32 +03:00
alesapin
fad3158bf7 Fix images names in integration tests 2021-12-09 10:55:51 +03:00
alesapin
a625dd41e8
Merge branch 'master' into better_logs 2021-12-03 15:31:45 +03:00
Constantine Peresypkin
7c90d4f8e3 refactor CI tests 2021-12-03 12:22:50 +02:00
alesapin
caa481aba8 Use ccache in fast test 2021-12-03 12:28:17 +03:00
alesapin
99a5ca2646 Fix style (got tired of it) 2021-12-03 12:19:39 +03:00
alesapin
ec0bfa7bcf Better output for some actions 2021-12-03 11:33:16 +03:00
alesapin
7699d3f35e
Merge pull request #32162 from azat/double-gz
tests/ci: do not compress logs twice
2021-12-03 10:20:35 +03:00
Azat Khuzhin
bfaaa79cfc tests/ci: do not compress logs twice
Cc: @alesapin
2021-12-03 01:28:32 +03:00
mergify[bot]
e2cb91b560
Merge branch 'master' into add_one_more_group 2021-12-02 19:28:13 +00:00
alesapin
f2140be90b Fix CI 2021-12-02 22:03:59 +03:00
alesapin
c14086d578
Merge pull request #32064 from ClickHouse/lightweight_checks_rerun
Add ability for lightweight checks rerun
2021-12-02 20:54:27 +03:00
alesapin
9e1437a7c4 Small improvements in lambda code 2021-12-02 19:38:18 +03:00
alesapin
7ef0737edc Move fuzzers and unit tests to another group 2021-12-02 13:23:32 +03:00
alesapin
c07450a7dc Fixup one more time 2021-12-02 10:49:44 +03:00
alesapin
a29b7105da fixup 2021-12-01 22:17:23 +03:00
alesapin
bf5858a6b3 Fix build urls 2021-12-01 22:14:46 +03:00
alesapin
e298e14c3b Fix rerun helper and build report 2021-12-01 18:50:48 +03:00
alesapin
c13a72991b Fix style 2021-12-01 17:47:10 +03:00
alesapin
71e6c7e92e Add ability to rerun checks 2021-12-01 17:23:51 +03:00
alesapin
1977ab6a8e
Merge pull request #32051 from ClickHouse/cancel_checks_more_agressively
Cancel redundant checks with lambda
2021-12-01 15:39:40 +03:00
alesapin
6cdb9f65a4 Cancel redundant checks with lambda 2021-12-01 15:38:17 +03:00
Mikhail f. Shiryaev
23f9c8bbba
Fix building rpm and tgz for branch pushes 2021-12-01 12:44:39 +01:00
alesapin
8261e12a48
Merge pull request #32041 from ClickHouse/remove_most_of_old_checks
Remove most of old checks
2021-12-01 14:02:22 +03:00
alesapin
e54dc89a84
Merge pull request #32044 from ClickHouse/report_success_in_case_of_force_tests
Fix force tests label
2021-12-01 13:26:58 +03:00
alesapin
a72f7a25f1 Fix force tests label 2021-12-01 12:50:36 +03:00
mergify[bot]
ffc1f4eb92
Merge branch 'master' into remove_most_of_old_checks 2021-12-01 09:45:19 +00:00
alesapin
f76149a1ec Better scripts for runners 2021-12-01 12:39:09 +03:00
alesapin
b421d07434 Remove most of old checks 2021-12-01 12:21:01 +03:00
alesapin
fb1d5e413e
Merge pull request #31987 from ClickHouse/add_special_functional_tests
Add rest functional tests to CI
2021-12-01 12:18:36 +03:00
alesapin
37f734bd6c
Merge pull request #31991 from ClickHouse/add_special_builds_to_ci
Add special builds to CI
2021-11-30 23:03:15 +03:00
alesapin
adf6ff8477 Fix build on master 2021-11-30 18:58:45 +03:00
alesapin
4b1b92a288 Better builds report 2021-11-30 18:33:29 +03:00
alesapin
e4cb882909 Add special builds to CI 2021-11-30 15:45:18 +03:00
alesapin
208eb6563c Add rest functional tests 2021-11-30 14:40:19 +03:00
Mikhail f. Shiryaev
1ba5fd5912
Merge pull request #31952 from ClickHouse/adjust_packages_buckets
Adjust packages buckets
2021-11-30 11:20:09 +01:00
alesapin
cc3321f1ff Fix docs check 2021-11-30 10:42:02 +03:00
Mikhail f. Shiryaev
b5c9f09f9b
Push packages from master actions to a version bucket 2021-11-29 16:07:19 +01:00
Mikhail f. Shiryaev
d7cf4f78c3
Create get_event function to read event data 2021-11-29 16:07:18 +01:00
Mikhail f. Shiryaev
ecf2282476
Adjust artifactory pusher to the new S3 paths 2021-11-29 16:07:15 +01:00
alesapin
9be8e21529 Add json type to changes in documentation 2021-11-28 15:55:23 +03:00
alesapin
91ff4caba4
Merge pull request #31861 from ClickHouse/more_clear_build_paths
Get rid of build numbers and simplify builds paths in S3
2021-11-28 15:33:41 +03:00
alesapin
c2d7731544
Update ci_config.py 2021-11-26 16:37:46 +03:00
alesapin
464b943b38 fix comment 2021-11-26 13:59:38 +03:00
alesapin
081bfdfe82 Get rid of build numbers and simplify builds paths in S3 2021-11-26 13:57:36 +03:00
Nikolai Kochetov
260928cb95
Merge pull request #31724 from ClickHouse/add-another-ci-database
Try to push data into another ci database.
2021-11-26 11:22:56 +03:00