2418 Commits

Author	SHA1	Message	Date
Mikhail f. Shiryaev	661161695f	Merge pull request #61592 from ClickHouse/improve-builds-downloader ... Improve build_download_helper	2024-03-20 11:56:06 +01:00
Mikhail f. Shiryaev	4efddb5ed4	Add type checking to build_download_helper	2024-03-19 14:01:31 +01:00
Mikhail f. Shiryaev	6b43d2e197	Improve logging in build_download_helper	2024-03-19 13:27:26 +01:00
Raúl Marín	10dd47cf5c	Separate CH and sanitizer logs	2024-03-18 16:40:03 +01:00
Raúl Marín	83c660a545	Merge remote-tracking branch 'blessed/master' into fuzz_log	2024-03-18 16:12:50 +01:00
Mikhail f. Shiryaev	41432c56a8	Enable shellcheck conditionally	2024-03-18 15:04:44 +01:00
Mikhail f. Shiryaev	c3f386d06a	Fix the autofixer	2024-03-18 15:04:43 +01:00
Mikhail f. Shiryaev	49017fc163	Run the style-check on top of the merge-commit of PRs	2024-03-18 15:04:41 +01:00
Mikhail f. Shiryaev	0240ffa5a6	Fetch the changed files only once	2024-03-18 15:04:40 +01:00
Alexey Milovidov	2de304e00c	Merge pull request #61377 from ClickHouse/fix-spot-termination ... Terminate EC2 on spot event if runner isn't running	2024-03-15 11:02:04 +03:00
Mikhail f. Shiryaev	03b9bca8a4	Terminate EC2 on spot event if runner isn't running	2024-03-14 14:31:55 +01:00
Mikhail f. Shiryaev	d465835306	Reorder hidden and shown checks in comment, change url of Mergeable check	2024-03-14 14:19:07 +01:00
Raúl Marín	f63368d83d	Separate stdout and stderr in fuzzer report	2024-03-13 19:12:43 +01:00
Mikhail f. Shiryaev	ad363fe254	Merge pull request #61231 from ClickHouse/style-quick-scale ... Make every style-checker runner types scaling-out very quickly	2024-03-12 18:19:03 +01:00
Mikhail f. Shiryaev	50aab0b69f	Fix annoying typo	2024-03-12 14:06:17 +01:00
Mikhail f. Shiryaev	612ff3c5bc	Make every style-checker runner types scaling-out very quickly	2024-03-12 13:00:31 +01:00
Max Kainov	fb1e5923a3	CI: fix runner for arm perf tests ... #do_not_test	2024-03-12 09:12:25 +00:00
Max Kainov	b7b83085d1	CI: ci.py hot style fix ... #do_not_test	2024-03-11 18:49:45 +00:00
Max K	7676d71c5d	Merge pull request #59738 from ClickHouse/ci_arm_integration_fix_1 ... CI: support batch specification in commit message	2024-03-11 19:01:03 +01:00
Max Kainov	f973e405ee	CI: fixing ARM integration tests ... #do_not_test	2024-03-11 17:20:36 +00:00
Max Kainov	f656a01538	CI: fix sync build issue with reuse ... #do_not_test	2024-03-11 17:15:23 +00:00
Max K	a0a96b5580	Merge pull request #61099 from ClickHouse/ci_merge_queue_support ... CI: support merge queue event in pr_info	2024-03-11 16:54:16 +01:00
Max K	23a9ac4947	Merge pull request #61160 from ClickHouse/ci_fix_nightly ... CI: fix nightly job issue	2024-03-11 14:46:53 +01:00
Max Kainov	c4f29af8ec	CI: fix nightly job issue ... #do_not_test	2024-03-11 13:35:05 +00:00
Mikhail f. Shiryaev	8f2ff2ccd8	Enable black back	2024-03-11 13:22:18 +01:00
Max Kainov	49b78e0a0c	CI: check cpp and py files style on changes only ... #do_not_test	2024-03-11 09:30:26 +00:00
Alexey Milovidov	8c08fc1f57	Revert "CI: make style check faster"	2024-03-11 04:09:26 +03:00
Max Kainov	f429f54af8	CI: make style check fast ... #do_not_test	2024-03-10 17:25:51 +00:00
Max Kainov	681bd28c87	CI: make style check fast ... #do_not_test	2024-03-09 20:57:23 +00:00
Max Kainov	7f061ba89c	CI: add wf class in ci_config ... #do_not_test	2024-03-09 14:33:34 +00:00
Max Kainov	cacbd3ce34	CI: support merge queue event in pr_info ... #do_not_test	2024-03-08 14:53:23 +00:00
Mikhail f. Shiryaev	e3f9fb2733	Trigger Mergeable check unconditionally in the final_check.py	2024-03-07 10:38:05 +01:00
Mikhail f. Shiryaev	5e597228d7	Merge pull request #60408 from ClickHouse/update-style-python ... Update style python	2024-03-07 09:32:28 +01:00
Nikolay Monkov	fb17749b50	file has been reformatted to pass Style check	2024-03-07 09:45:24 +05:00
Max K	6c0e9e1944	Merge pull request #60958 from ClickHouse/ci_remove_rerun_check_for_build_report_jobs ... CI: disable rerun check helper for build report jobs	2024-03-06 21:32:04 +01:00
Max Kainov	4e1257974c	CI: disable rerun check helper for build report jobs ... #do_not_test	2024-03-06 20:11:55 +00:00
Max Kainov	a0d161704a	CI: remove sqllancer, sqllogic, sqltest from pr wf ... #do_not_test	2024-03-06 17:58:21 +00:00
Max Kainov	4dd5e4fe99	CI: fix stage config (unit test release issue) ... #no_ci_cache	2024-03-06 16:52:01 +00:00
Nikolay Monkov	d9b5f9a086	version has been added to docker labels	2024-03-06 21:37:56 +05:00
Max Kainov	ee468b0f2a	CI: wait less in await to not exceed GH timeout ... #do_not_test	2024-03-06 14:26:58 +00:00
Max K	561be5eb74	Merge pull request #60421 from ClickHouse/ci_abstract_workflow ... CI: make workflow yml abstract	2024-03-06 15:15:02 +01:00
Mikhail f. Shiryaev	d2f92483f2	Add even more python code to pylint check, fix lambdas	2024-03-05 16:02:44 +01:00
Mikhail f. Shiryaev	34bb40583c	Install stubs for python-requests	2024-03-05 16:02:43 +01:00
Mikhail f. Shiryaev	91cff01ce0	More fixes in tests/ci	2024-03-05 16:02:41 +01:00
Mikhail f. Shiryaev	e31078e204	Fix some issues detected by new pylint	2024-03-05 16:02:36 +01:00
Mikhail f. Shiryaev	ec72800d78	Merge pull request #60383 from nickitat/upd_liburing ... Enable testing with `io_uring` back	2024-03-05 15:26:35 +01:00
Max Kainov	82cc3f4339	CI: make workflow yml abstract ... #no_merge_commit #no_ci_cache #job_style_check #job_package_debug #job_ClickHouse_build_check #job_Stateless_tests_debug #job_binary_release	2024-03-05 12:26:55 +00:00
Max Kainov	a6f5323c38	CI: Fix pending status for build report in backports ... #do_not_test	2024-03-05 10:09:17 +00:00
Mikhail f. Shiryaev	a6caace5ec	Add a new runner type	2024-03-04 20:03:45 +01:00
Mikhail f. Shiryaev	bd2b0b4338	Export only str and bool for build configs	2024-03-04 20:01:26 +01:00
Mikhail f. Shiryaev	1443e490ea	Remove python bytecode, make consistent file-permissions ... It's impossible to have persistent pyc files Each time they are built with different content, for example: > cmp -bl lambda-package*/charset_normalizer/__pycache__/constant.cpython-310.pyc 15582 6 ^F 4 ^D 15583 164 t 155 m 15584 141 a 142 b 15586 164 t 163 s 15587 151 i 332 M-Z 15588 163 s 6 ^F 15589 332 M-Z 164 t 15590 4 ^D 141 a 15591 155 m 143 c 15592 142 b 164 t 15593 143 c 151 i 17425 74 < 75 = 17428 76 > 46 & 17429 332 M-Z 372 M-z 17431 173 { 55 - 17434 75 = 174 | 17437 57 / 72 : 17440 54 , 73 ; 17441 372 M-z 332 M-Z 17443 174 | 175 } 17446 55 - 54 , 17447 372 M-z 332 M-Z 17449 46 & 173 { 17452 72 : 76 > 17455 42 " 74 < 17458 73 ; 133 [ 17461 135 ] 42 " 17464 133 [ 135 ] 17465 332 M-Z 372 M-z 17467 175 } 57 / 17503 332 M-Z 162 r 17504 5 ^E 130 X 17505 152 j 0 ^@ 17506 157 o 0 ^@ 17507 150 h 0 ^@ 17508 141 a 332 M-Z 17509 142 b 5 ^E 17510 162 r 152 j 17511 130 X 157 o 17512 0 ^@ 150 h 17513 0 ^@ 141 a 17514 0 ^@ 142 b 17536 5 ^E 2 ^B 17537 143 c 150 h 17538 160 p 172 z 17539 71 9 332 M-Z 17540 65 5 5 ^E 17541 60 0 143 c 17542 332 M-Z 160 p 17543 2 ^B 71 9 17544 150 h 65 5 17545 172 z 60 0	2024-03-04 13:37:13 +01:00
Mikhail f. Shiryaev	63e40203d1	Use python zipfile to have x-platform idempotent lambda packages	2024-03-04 13:37:11 +01:00
Max K	cecd2a6bac	Merge pull request #60613 from ClickHouse/ci_build_report_job_to_ci_config ... CI: build_report job no dependcies on yml	2024-03-03 20:33:08 +01:00
Alexey Milovidov	c427575f72	Merge pull request #60694 from azat/ci/fix-flaky-check ... Fix flaky checks	2024-03-03 19:53:12 +03:00
Max Kainov	8785b0700c	CI: build_report job remove dependsies on yml ... #no_merge_comit #job_ClickHouse_build_check #job_ClickHouse_special_build_check #job_style_check	2024-03-03 16:08:35 +00:00
Max K	2c713c2b90	Merge pull request #60602 from ClickHouse/ci_remove_docker_server_job_input_params ... CI: remove inputs from docker_server job command	2024-03-03 14:25:23 +01:00
Azat Khuzhin	982f7e97c3	tests/ci/functional_test_check: move validate_bugfix_check into assert ... Co-authored-by: Max K. <max.kainov@clickhouse.com>	2024-03-03 13:55:22 +01:00
Azat Khuzhin	418a6258ac	Fix flaky checks ... CI [1]: 2024-03-02T22:23:37.4878824Z Going to start run command [python3 /home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/tests/ci/functional_test_check.py "$CHECK_NAME" $KILL_TIMEOUT] 2024-03-02T22:23:37.7538256Z Runnin check [Stateless tests flaky check (asan)] with timeout [3600] 2024-03-02T22:23:37.8790465Z Fetched info about 3 changed files 2024-03-02T22:23:37.9160414Z Traceback (most recent call last): 2024-03-02T22:23:37.9161715Z File "/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/tests/ci/functional_test_check.py", line 341, in <module> 2024-03-02T22:23:37.9162714Z main() 2024-03-02T22:23:37.9163683Z File "/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/tests/ci/functional_test_check.py", line 235, in main 2024-03-02T22:23:37.9164642Z assert ( 2024-03-02T22:23:37.9165273Z AssertionError: JobReport file path must be provided with --validate-bugfix 2024-03-02T22:23:37.9612679Z Run action failed for: [Stateless tests flaky check (asan)] with exit code [1] [1]: https://github.com/ClickHouse/ClickHouse/actions/runs/8125551797/job/22208766466?pr=60689 Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>	2024-03-03 10:44:21 +01:00
Max Kainov	bcd6e2f945	fixup! CI: remove inputs from docker_server job command ... #job_style_check #job_Docker_server_image #job_Docker_keeper_image	2024-03-02 15:12:53 +00:00
Max Kainov	8abc0df7a8	CI: remove inputs from docker_server job command ... #job_style_check #job_Docker_server_image #job_Docker_keeper_image #packege_release #package_aarch64	2024-03-01 16:44:37 +00:00
Max Kainov	0b127ffba7	CI: await CI on master only ... #do_not_test	2024-03-01 14:27:19 +00:00
robot-ch-test-poll1	7f0494b130	Merge pull request #60587 from ClickHouse/ci_fix_build_job_status ... CI: fix for build job cache record status	2024-02-29 21:27:36 +04:00
Mikhail f. Shiryaev	af63125aaa	Merge pull request #60316 from ClickHouse/tailscale-ami ... Install tailscale during AMI build and set it up on runners	2024-02-29 17:47:44 +01:00
Max Kainov	bd423c5ac5	CI: fix for build job cache record status ... #job_package_release #job_style_check	2024-02-29 15:55:05 +00:00
Nikita Taranov	25347501d2	Merge branch 'master' into upd_liburing	2024-02-29 16:25:11 +01:00
Max K	01456f3f0c	Merge pull request #60455 from ClickHouse/ci_job_scripts_wo_input_params ... CI: remove input params for job scripts	2024-02-29 15:51:10 +01:00
Max K	b4357d153d	Merge pull request #60554 from ClickHouse/ci_fix_docker_job_name_arch_suffix ... CI: fix docker build job name	2024-02-29 15:25:51 +01:00
Max Kainov	64a2b6551a	CI: fix docker build job name ... #do_not_test	2024-02-29 13:32:33 +00:00
Max Kainov	0df8db5c3f	CI: configure ci_set_analyzer ... #do_not_test	2024-02-29 12:44:55 +00:00
Max Kainov	5f88a455ff	CI: remove input params for job scripts ... #job_style_check #job_Compatibility_check_amd64 #job_Compatibility_check_aarch64	2024-02-28 22:35:47 +00:00
Nikita Taranov	dc135294f0	add seccomp=unconfined	2024-02-28 22:17:47 +01:00
Max Kainov	5a57b9f75e	CI: remove autobackport for bugfix ... #do_not_test	2024-02-28 19:30:34 +00:00
Max K	2970ca7b6d	Merge pull request #60457 from ClickHouse/ci_auto_ci_backport ... CI: Auto backport for bugfix and ci-fix	2024-02-28 18:28:22 +01:00
Max Kainov	444ff093bb	CI: Auto backport for bugfix and CI changes	2024-02-28 16:02:13 +00:00
Mikhail f. Shiryaev	130cd86287	Do not mark report as pending if some jobs are skipped	2024-02-28 15:15:04 +01:00
Max Kainov	92e6cbe2e8	CI: quick style fix ... #do_not_test	2024-02-28 12:37:07 +00:00
Max K	a23913b277	Update tests/ci/run_check.py ... #do_not_test Co-authored-by: Mikhail f. Shiryaev <felixoid@clickhouse.com>	2024-02-28 12:22:19 +01:00
Max Kainov	155b5f3cff	CI: fix pr check status to not fail mergeable check ... #do_not_test	2024-02-28 10:33:32 +00:00
Mikhail f. Shiryaev	6b24306678	Merge pull request #60444 from ClickHouse/replace-distutils ... Replace deprecated distutils by vendored packaging	2024-02-27 15:50:49 +01:00
Mikhail f. Shiryaev	f49882ee17	Do not fail the build if ci-logs is not healthy	2024-02-27 13:59:01 +01:00
Mikhail f. Shiryaev	c85daf27f4	Replace deprecated distutils by vendored packaging	2024-02-27 12:49:44 +01:00
Mikhail f. Shiryaev	6ac893b565	Fix the local cache when it does not exist	2024-02-26 23:17:16 +01:00
Mikhail f. Shiryaev	6a6d6a286e	Split update_mergeable_check into two functions to force trigger the status	2024-02-26 21:31:41 +01:00
Azat Khuzhin	dd690dd05d	Expose fatal.log separately for fuzzer ... Otherwise you have to download the full server.log Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>	2024-02-24 19:23:28 +01:00
Mikhail f. Shiryaev	ff353823b6	Update the GH actions runner version	2024-02-23 17:18:53 +01:00
Mikhail f. Shiryaev	758acd4268	Setup tailscale on instances, create a common network script	2024-02-23 17:18:52 +01:00
Max K	11e4029c6b	Merge pull request #59496 from AVMusorin/skip-tests-for-docs ... Avoid running all checks when `aspell-dict.txt` was changed	2024-02-23 14:13:47 +01:00
Max Kainov	831fee9edb	CI: remove Integration tests asan, release from PR wf ... #do_not_test	2024-02-22 18:35:06 +00:00
Max K	0df33c94ba	Merge pull request #60224 from ClickHouse/ci_combine_analyzer_s3_replicated_tests ... CI: combine analyzer, s3, dbreplicated into one job	2024-02-22 18:41:41 +01:00
Mikhail f. Shiryaev	a768f40adc	Add tailscale installation to the AMI preparation	2024-02-22 14:58:51 +01:00
Alexey Milovidov	659d960990	Merge pull request #60245 from Algunenano/enforce_cloud_sync ... Make cloud sync required	2024-02-22 10:11:01 +01:00
Max K	e7a560f94d	Merge pull request #59939 from ClickHouse/ci_integration_tests_to_mysql8_migration ... CI: integration tests to mysql80	2024-02-21 20:33:00 +01:00
Raúl Marín	07c9deed41	Make cloud sync required	2024-02-21 18:40:00 +01:00
Max Kainov	f827469207	CI: random sanitizer for parallel repl in PR wf	2024-02-21 16:08:03 +00:00
Max Kainov	47ab81bbab	CI: support for random job pick ... #no_merge_commit #no_ci_cache	2024-02-21 15:39:57 +00:00
Max Kainov	9fb55f1fa4	CI: combine analyzer s3 dbreplicated into one job ... #job_Stateless_tests_release_analyzer_s3_DatabaseReplicated #job_style_check #no_merge_commit	2024-02-21 13:18:02 +00:00
Max Kainov	9f51e840db	CI: integration tests to mysql80 ... #ci_set_integration #no_merge_commit	2024-02-21 11:05:47 +00:00
Max Kainov	f928eaf10c	CI: hot fix for gh statuses ... #no_merge_commit #job_style_check #job_Stateless_tests_debug	2024-02-20 21:38:19 +00:00
Max K	bf6929cb4c	Merge pull request #59348 from ClickHouse/refactor_bugfix_validate_job ... CI: refactor bugfix-validate job	2024-02-16 12:32:52 +01:00
Max Kainov	5a3a93bea5	CI: refactor bugfix validate job ... #no_merge_commit	2024-02-15 16:28:27 +00:00