1763 Commits

Author	SHA1	Message	Date
Ilya Yatsishin	7e3eaadaac	Make SQLancer check not "always green"	2023-06-23 18:55:04 +02:00
Alexey Milovidov	b6e7f4cf6c	Require more checks	2023-06-22 23:24:27 +02:00
Nikolai Kochetov	a940031878	Merge branch 'master' into refactor-subqueries-for-in	2023-06-22 12:18:48 +02:00
Mikhail f. Shiryaev	02776b8a15	Merge pull request #51163 from ClickHouse/fasttest-ignore-chown-fail ... Ignore errors for `chown` in fasttests	2023-06-22 09:51:57 +02:00
Mikhail f. Shiryaev	1591633d72	Merge pull request #49283 from ClickHouse/runners-survival ... Attempt to increase the general runners' survival rate	2023-06-22 00:16:11 +02:00
Mikhail f. Shiryaev	118c3af4d6	Ignore errors for chown in fasttests	2023-06-21 12:59:21 +02:00
Robert Schulze	bc7df2baca	Merge branch 'master' into non-experimental-qpl-deflate	2023-06-20 20:06:01 +02:00
Mikhail f. Shiryaev	23d843d379	Experiment with decreasing scale_up for style-checkers	2023-06-20 15:17:33 +02:00
Mikhail f. Shiryaev	1a8571c465	Check and proceed spot termination on runner setup	2023-06-19 15:40:32 +02:00
Mikhail f. Shiryaev	4d5160a3ba	Improve determination of job received	2023-06-19 15:40:32 +02:00
Mikhail f. Shiryaev	d782b05845	Continue LCH for terminated instances, add logging	2023-06-19 15:40:31 +02:00
Mikhail f. Shiryaev	d8e3d5c803	Kill the instance proactively on spot termination	2023-06-19 15:40:30 +02:00
Mikhail f. Shiryaev	d672bd3382	Stop the runner if there are no running jobs for long	2023-06-19 15:40:30 +02:00
Mikhail f. Shiryaev	915f431b95	Tune the shutting down on rebalance signal	2023-06-19 15:40:28 +02:00
Mikhail f. Shiryaev	026ab3d4ce	Calculate current capacity according to HealthStatus	2023-06-19 15:40:27 +02:00
Mikhail f. Shiryaev	8feb5146fc	Do not "update" the token and the instances infinitely	2023-06-19 15:40:27 +02:00
Mikhail f. Shiryaev	a5e31e98de	Decouple cleaning part from metrics lambda to a new one	2023-06-19 15:40:26 +02:00
Mikhail f. Shiryaev	1688ba5cf5	Use ephemeral runners for GH actions	2023-06-19 15:40:25 +02:00
Mikhail f. Shiryaev	47ea742076	Do not kill runner on the fly, delay 3 seconds	2023-06-19 15:40:25 +02:00
Mikhail f. Shiryaev	6cecdb8fca	Shutdown runner process and do not restart it on termination	2023-06-19 15:40:24 +02:00
Mikhail f. Shiryaev	197d0d1314	Process lifecycle hook in post-running action hook	2023-06-19 15:40:24 +02:00
Mikhail f. Shiryaev	d45092d63b	Process the rebalance recomendation event on post-run	2023-06-19 15:40:23 +02:00
Mikhail f. Shiryaev	ee5ca7337e	Add reset to Stopwatch, improve codebrowser report	2023-06-16 12:51:31 +02:00
Dmitry Novik	49f88f5873	Fix ENV variable name	2023-06-15 14:15:01 +00:00
Mikhail f. Shiryaev	eb2a5c31a3	Do not mock git_helper.Git, suppress git describe error for ignore_no_tags	2023-06-15 15:35:03 +02:00
Mikhail f. Shiryaev	2ac95ccfce	Adjust version_helper test cases to proper tags	2023-06-15 14:48:10 +02:00
Mikhail f. Shiryaev	07203a4542	Fix a versions' tweak for tagged commits, improve version_helper	2023-06-15 14:48:09 +02:00
robot-clickhouse	1230519bec	Automatic style fix	2023-06-14 13:38:44 +00:00
Dmitry Novik	fe8172fbd9	Review fixes	2023-06-14 13:17:04 +00:00
Dmitry Novik	afcc3aca36	Update ci_config.py	2023-06-14 13:14:59 +00:00
Dmitry Novik	1207085dec	Merge branch 'master' into analyzer-integration-tests	2023-06-13 17:09:29 +02:00
Nikita Mikhaylov	52a460df67	Tests with parallel replicas are no more "always green" (#50896)	2023-06-13 16:43:35 +02:00
Alexander Tokmakov	5368fa72ca	Merge pull request #50487 from Avogar/add-gdb-to-stateless-tests ... Attach gdb in stateless tests	2023-06-13 17:04:17 +03:00
Dmitry Novik	9e8ca5f5ae	Add integration test check with enabled analyzer	2023-06-13 13:09:39 +00:00
Robert Schulze	8222861f2e	Merge remote-tracking branch 'rschu1ze/master' into non-experimental-qpl-deflate	2023-06-11 14:39:30 +00:00
Nikolai Kochetov	6ce8329bda	Merge branch 'master' into refactor-subqueries-for-in	2023-06-09 20:04:27 +02:00
Nikolai Kochetov	a878312fdf	Merge pull request #50693 from ClickHouse/vdimir/disable_grace_hash_stress ... Disable grace_hash join in stress tests	2023-06-09 17:57:34 +02:00
Robert Schulze	1aa158909e	enable_qpl_deflate_codec --> enable_deflate_qpl_codec	2023-06-09 12:43:33 +00:00
jinjunzh	f1192d59af	refine patch according to comments	2023-06-09 12:43:15 +00:00
jinjunzh	31173ab55b	add sections of deflate_qpl for stress test and performance test	2023-06-09 12:43:08 +00:00
Mikhail f. Shiryaev	2aaaec2abe	Catch issues with dockerd during the build	2023-06-08 14:01:41 +02:00
vdimir	2297995c1a	Disable grace_hash join in stress tests ... Until https://github.com/ClickHouse/ClickHouse/issues/50220 is fixed	2023-06-08 07:15:11 +00:00
Antonio Andelic	26fb80b540	Merge pull request #50615 from ClickHouse/fix-jepsen-check ... Fix Jepsen runs in PRs	2023-06-07 08:43:51 +02:00
Sema Checherinda	990652119e	Merge pull request #50361 from nickitat/fail_perf_on_too_many_slow ... Fail perf tests when too many queries slowed down	2023-06-06 11:33:04 +02:00
Antonio Andelic	0bc31a7288	Fix jepsen runs in PRs	2023-06-06 07:17:30 +00:00
robot-clickhouse-ci-1	4cf5a13e5b	Merge pull request #50160 from ClickHouse/prio-runners ... Refactor lambdas, prepare to prio runners	2023-06-06 07:29:28 +02:00
avogar	e8c6c7967b	Move attaching gdb to separate lib	2023-06-05 18:21:40 +00:00
Nikita Taranov	e548dce123	fix	2023-06-03 00:08:47 +02:00
Nikita Taranov	4506299d73	impl	2023-06-02 23:14:09 +02:00
pufit	0d98a46326	Fix KeyError in cherry-pick	2023-06-02 13:02:18 -04:00
Mikhail f. Shiryaev	9bd0a53e7c	Get only online instances in get_lost_ec2_instances	2023-06-02 12:27:29 +02:00
Mikhail f. Shiryaev	3c13eaa159	Simplify get_dead_runners_in_ec2 a little bit, update runners	2023-06-02 12:27:28 +02:00
Mikhail f. Shiryaev	b775b5cfd6	Move all CI runners metrics into one namespace	2023-06-02 12:27:27 +02:00
Mikhail f. Shiryaev	e657e2ba10	Additional logging in autoscale_runners_lambda	2023-06-02 12:27:26 +02:00
Mikhail f. Shiryaev	b7c5fdab77	Move lambda package building to public.ecr.aws/lambda/python for compatibility	2023-06-02 12:27:26 +02:00
Mikhail f. Shiryaev	0ddd53088d	Add a new runner type for ci metrics and autoscaling	2023-06-02 12:27:24 +02:00
Mikhail f. Shiryaev	e8b03d7498	Move insert part for ClickHouseHelper to shared	2023-06-01 21:03:57 +02:00
Mikhail f. Shiryaev	2dca0eac1b	Delete __init__.py in lambda directories to break subpackage	2023-06-01 21:03:56 +02:00
Mikhail f. Shiryaev	acb9531ebf	Unify and put GH access token to the lambda_shared	2023-06-01 21:03:56 +02:00
Mikhail f. Shiryaev	0fa6a84161	Move the stuff related to runners to lambda_shared	2023-06-01 21:03:55 +02:00
Mikhail f. Shiryaev	7f08f218d9	Create lambda_shared package for lambdas	2023-06-01 21:03:53 +02:00
Mikhail f. Shiryaev	484c91c47e	Add DRY_RUN and configurable PY_VERSION to lambda deployment	2023-06-01 21:03:53 +02:00
Mikhail f. Shiryaev	27941b4d26	Decrease the time window for autoscale_runners_lambda	2023-06-01 21:03:52 +02:00
Mikhail f. Shiryaev	db02938411	Do not count unfinished tasks with conclusion=None	2023-06-01 21:03:51 +02:00
Mikhail f. Shiryaev	855afb56f9	Get instances for the region to not kill a fresh one	2023-06-01 21:03:50 +02:00
Mikhail f. Shiryaev	bf9b563e0b	Improve caching mechanism for token, add cached instances	2023-06-01 21:03:50 +02:00
Mikhail f. Shiryaev	7bf9089dcd	Increase access_token cached time	2023-06-01 21:03:49 +02:00
Mikhail f. Shiryaev	985cd8fc8a	Improve events logging	2023-06-01 21:03:48 +02:00
Nikita Taranov	e5c21e588a	Update app.py (#50407)	2023-06-01 15:19:47 +02:00
Mikhail f. Shiryaev	9089360688	Merge pull request #50415 from ClickHouse/build-report-pending ... Mark the builds without results as pending	2023-06-01 14:53:09 +02:00
Mikhail f. Shiryaev	de7179876e	Merge pull request #47151 from ClickHouse/increase-vm-max_map_count ... Update ubuntu_ami_for_ci.sh	2023-06-01 13:54:21 +02:00
Mikhail f. Shiryaev	6a5ae6558a	Add comment for usage of prepare-ci-ami.sh	2023-06-01 12:58:03 +02:00
Mikhail f. Shiryaev	bcbce15cde	Mark the builds without results as pending	2023-06-01 08:51:44 +02:00
Mikhail f. Shiryaev	653da5f002	Reduce the unzip verbosity	2023-05-31 18:23:21 +02:00
Mikhail f. Shiryaev	a96a845e23	Add comment to prepare-ci-ami.sh, add a success at the end	2023-05-31 17:12:49 +02:00
pufit	08d3887832	Add re-creation for cherry-pick PRs (#50373) ... * Add recreation for cherry-pick PRs. Small PR comment fix. * Automatic style fix --------- Co-authored-by: robot-clickhouse <robot-clickhouse@users.noreply.github.com> Co-authored-by: Nikita Mikhaylov <mikhaylovnikitka@gmail.com>	2023-05-31 13:04:29 +02:00
Mikhail f. Shiryaev	db3a050911	Seems like buildx needs explicit install since some time	2023-05-31 00:22:12 +02:00
Mikhail f. Shiryaev	56dc35d563	Merge branch 'master' into increase-vm-max_map_count	2023-05-30 19:24:18 +02:00
Mikhail f. Shiryaev	bcd57078ce	Update the runner version	2023-05-30 16:36:54 +02:00
Nikolai Kochetov	b5b261b22c	Merge branch 'master' into refactor-subqueries-for-in	2023-05-25 21:22:06 +02:00
pufit	4ce722854b	Fix URL in backport comment	2023-05-25 11:51:47 -04:00
Mikhail f. Shiryaev	a9d5b68946	Merge pull request #50197 from ClickHouse/fix-codebrowser ... Fix codebrowser by using clang-15 image	2023-05-24 23:39:13 +02:00
Mikhail f. Shiryaev	e117347ca0	Attempt to fix codebrowser with using clang-15	2023-05-24 23:15:38 +02:00
Alexander Tokmakov	2c0c1ba4af	add comments to build reports (#50200)	2023-05-24 23:29:36 +03:00
pufit	8875b98e2a	Added default value for --from-repo arg	2023-05-24 13:47:50 -04:00
pufit	8df7a69147	Automatic backports of important fixes to cloud-release ... * Automatic backports to cloud-release --------- Co-authored-by: robot-clickhouse <robot-clickhouse@users.noreply.github.com>	2023-05-24 18:59:18 +02:00
Nikolai Kochetov	ff6400f91a	Merge branch 'master' into refactor-subqueries-for-in	2023-05-22 13:07:51 +02:00
Timur Solodovnikov	c7ab59302f	Set allow_experimental_query_cache setting as obsolete (#49934) ... * set allow_experimental_query_cache as obsolete * add tsolodov to trusted contributors * CI linter --------- Co-authored-by: Nikita Mikhaylov <mikhaylovnikitka@gmail.com>	2023-05-17 20:03:42 +02:00
Mikhail f. Shiryaev	4b7aa30017	The best effor to get the current job ID and URL ... This partially addresses https://github.com/actions/runner/issues/2577	2023-05-12 19:22:52 +02:00
Mikhail f. Shiryaev	c375f948d0	Fix typing in S3Helper.fast_parallel_upload_dir	2023-05-12 19:22:52 +02:00
Mikhail f. Shiryaev	338435a52d	Additional check for FATAL in logs, do not exit with code 1 on errors	2023-05-12 19:22:51 +02:00
Mikhail f. Shiryaev	0f70ad56a9	Do not upload failed site to the S3	2023-05-12 19:22:50 +02:00
Mikhail f. Shiryaev	eb122487ad	Add logs to the report, do not chown unnecessary	2023-05-12 19:22:48 +02:00
Mikhail f. Shiryaev	6a56662f60	Improve codebrowser check ... - Add sccache - Report a proper status in report and exit code - Launch it as a current user - Fix report link	2023-05-12 19:22:48 +02:00
Mikhail f. Shiryaev	fba7120d3d	Fix s3_helper logging and pool setup	2023-05-12 19:22:44 +02:00
Robert Schulze	b9c185af44	Merge pull request #49678 from azat/build/llvm-16 ... Switch to LLVM/clang 16 (16.0.3)	2023-05-12 13:47:36 +02:00
Mikhail f. Shiryaev	8a664e2cac	Upgrade all lambdas runtime to 3.10, fix requirements ... The recent update of urllib3 completely broken the world - https://github.com/urllib3/urllib3/issues/2168 - https://github.com/boto/botocore/issues/2926 - https://github.com/psf/requests/issues/6443	2023-05-11 16:09:59 +02:00
Mikhail f. Shiryaev	d890114c5e	Print debugging information in workflow_jobs_lambda	2023-05-11 14:21:07 +02:00
Mikhail f. Shiryaev	d605f82798	Replace old release with new packages in suspicious patterns	2023-05-10 12:36:04 +02:00
Mikhail f. Shiryaev	f6372dd3fc	Do not launch PRs w/o "can be tested"	2023-05-10 12:14:44 +02:00
Azat Khuzhin	c37fe64040	Merge remote-tracking branch 'u/master' into build/llvm-16	2023-05-08 22:33:52 +02:00
Nikolai Kochetov	fc02e9efc9	update fasttest a bit	2023-05-08 17:23:46 +00:00
Alexey Milovidov	54163b87ec	Merge pull request #47977 from ClickHouse/rs/non-experimental-qc ... Mark query cache as non-experimental	2023-05-06 19:49:23 +03:00
Robert Schulze	0598bfd79e	Mark query cache as non-experimental	2023-05-06 13:58:40 +00:00
Dmitry Novik	a3b3b78eb0	EXPECTED_TO_FAIL -> NOT_FAILED	2023-05-05 17:53:15 +00:00
Dmitry Novik	02520e9358	Make EXPRECTED_TO_FAIL red	2023-05-05 17:37:52 +00:00
Mikhail f. Shiryaev	90fe9dd1f2	Skip mergeable check only on a proper description	2023-05-03 22:38:34 +02:00
Mikhail f. Shiryaev	a67950d138	Check the PRs body directly in lambda, without rerun	2023-05-03 22:38:33 +02:00
Mikhail f. Shiryaev	242a210358	Create "CI running" report always	2023-05-02 14:13:23 +02:00
Mikhail f. Shiryaev	08d203e158	Sanitize "Flaky tests", remove trailing dots	2023-05-02 12:42:26 +02:00
Mikhail f. Shiryaev	b423e04ab0	Fix the global ROBOT_TOKEN, do not retry unnecessary	2023-05-02 12:37:05 +02:00
Mikhail f. Shiryaev	ca19f20aff	Do not fail the run on error updating status comment	2023-05-02 12:34:40 +02:00
Alexey Milovidov	dd7282ea5c	Update ci_config.py	2023-05-01 16:05:14 +03:00
Mikhail f. Shiryaev	b385b51f07	Merge pull request #49314 from ClickHouse/fallback-auth-gh-api ... Fallback auth gh api	2023-04-29 09:36:49 +02:00
Mikhail f. Shiryaev	d336bee056	Fix the Logs header for case when only log_urls is set	2023-04-28 23:52:35 +02:00
Mikhail f. Shiryaev	886ba0b01d	Replace markdown with pure HTML for check descriptions	2023-04-28 23:52:34 +02:00
Mikhail f. Shiryaev	20c8f0f1b4	Use only one comment for all statuses in the PR	2023-04-28 23:52:34 +02:00
Mikhail f. Shiryaev	61c6c61638	Rework the status comment to be a digest with the full report available ... by link	2023-04-28 23:52:33 +02:00
Mikhail f. Shiryaev	e8cf417350	Use NotSet in post_commit_status for optional arguments	2023-04-28 23:52:33 +02:00
Mikhail f. Shiryaev	c66e50f283	Remove options from S3Helper, they are global	2023-04-28 23:52:32 +02:00
Mikhail f. Shiryaev	f411be8c37	Rename Run Check to CI running	2023-04-28 23:52:31 +02:00
Mikhail f. Shiryaev	df33b99907	Rework post_commit_status, add set_status_comment there	2023-04-28 23:52:31 +02:00
Mikhail f. Shiryaev	d58abd3c37	Add a function get_repo with global state	2023-04-28 23:52:30 +02:00
Mikhail f. Shiryaev	4405459925	Refactor RerunHelper to reduce API calls	2023-04-28 23:52:30 +02:00
Mikhail f. Shiryaev	19fd5f9c3c	Add descriptions for the status comment	2023-04-28 23:52:29 +02:00
Mikhail f. Shiryaev	eb11fb02fa	Clean up ast_fuzzer_check.py	2023-04-28 23:52:29 +02:00
Mikhail f. Shiryaev	ccd4fd7e64	Use commit_status_helper.post_commit_status everywhere	2023-04-28 23:52:28 +02:00
Mikhail f. Shiryaev	144ebd3d36	Implement main() in run_check.py	2023-04-28 23:52:27 +02:00
Mikhail f. Shiryaev	e670c51680	Unify commit.create_status and post_commit_status	2023-04-28 23:52:26 +02:00
Mikhail f. Shiryaev	01e70c64c2	Add a fallback to authenticated requests to GH API	2023-04-28 18:46:33 +02:00
Mikhail f. Shiryaev	8c60b60916	Receive the best robot token only once	2023-04-28 16:01:53 +02:00
Mikhail f. Shiryaev	9a7f09bae4	Avoid splitting check_name into subdirectories	2023-04-26 01:11:05 +02:00
Mikhail f. Shiryaev	a6664e6b08	Add typing, capitalize only the first letter w/o lowering the rest	2023-04-26 00:55:05 +02:00
Mikhail f. Shiryaev	ed8ca018bd	Add a note regarding private/public repo to logs	2023-04-25 19:11:20 +02:00
alesapin	084f76ab3e	Merge pull request #49133 from ClickHouse/unanonymized-workflow-jobs-logs ... Improve some lambdas	2023-04-25 12:58:18 +02:00
Mikhail f. Shiryaev	2ab313e6b4	Update the style_checker upscale speed	2023-04-25 12:42:06 +02:00
Mikhail f. Shiryaev	6032bacd0f	Get unanonymized logs in workflow_jobs_lambda	2023-04-25 12:37:26 +02:00
Dmitry Novik	0d3e896f57	Merge pull request #48719 from ClickHouse/analyzer-ci ... Add CI run with new analyzer enabled	2023-04-24 17:36:18 +02:00
Mikhail f. Shiryaev	222176639d	Update pr_info.labels in commit_helper lebel helpers	2023-04-24 11:03:25 +02:00
Mikhail f. Shiryaev	caee710fcc	Enforce documentation change for a new-feature PR	2023-04-24 10:41:38 +02:00
Dmitry Novik	1fdee1a2ab	Merge branch 'master' into analyzer-ci	2023-04-21 15:43:16 +02:00
Alexey Milovidov	2dbe4f80f2	Do not allow incomplete changelog entries	2023-04-21 12:59:47 +02:00
Alexey Milovidov	95e69b6f85	Do not allow incomplete changelog entries	2023-04-21 12:59:17 +02:00
Mikhail f. Shiryaev	c28e8438e3	Fail Fast tests on next run if previous running has failed	2023-04-20 14:51:53 +02:00
Dmitry Novik	b2ed99f6b3	Merge branch 'master' into analyzer-ci	2023-04-18 13:07:45 +02:00
Dmitry Novik	5557f4524c	Fix string formatting	2023-04-18 08:44:46 +00:00
robot-clickhouse	f6127a68ac	Automatic style fix	2023-04-18 00:56:29 +00:00
Dmitry Novik	6197d3d55f	Pass volume only for analyzer	2023-04-18 00:12:30 +00:00
Dmitry Novik	61e552d660	Fix used build	2023-04-17 15:23:02 +00:00
Sema Checherinda	0354a964cd	turn on all CI back	2023-04-17 14:07:56 +02:00