1841 Commits

Author	SHA1	Message	Date
Alexey Milovidov	65e8ee8fb7	Fix paths	2023-08-03 15:28:18 +02:00
Mikhail f. Shiryaev	8499956321	Refactor CI_CONFIG from dict to dataclasses	2023-08-03 14:37:19 +02:00
Alexey Milovidov	5ec6a4695f	Fix log paths	2023-08-02 23:22:15 +02:00
Alexey Milovidov	a98e869912	Merge branch 'master' into sqltest	2023-08-02 20:08:39 +02:00
Mikhail f. Shiryaev	6dcdc4cd38	Remove mark_failed_report_pending, it's already in builds_report_config	2023-08-02 19:20:22 +02:00
Mikhail f. Shiryaev	40cdfa2384	Merge branch 'master' into fix-packager	2023-08-02 11:21:54 +02:00
Mikhail f. Shiryaev	3a07b80f34	Merge pull request #52277 from ClickHouse/kill-runner-with-subprocesses ... Kill the runner process with all subprocesses	2023-08-02 11:18:56 +02:00
alesapin	0abd5a058b	Merge pull request #52871 from ClickHouse/cont-server-after-gdb-killed ... Try to continue clickhouse process in stress test after terminating gdb.	2023-08-02 10:55:38 +02:00
Mikhail f. Shiryaev	8537b32de1	Merge branch 'master' into docker-improvements	2023-08-02 08:40:24 +02:00
Alexey Milovidov	3b6247e3f5	Merge branch 'sqltest' of github.com:ClickHouse/ClickHouse into sqltest	2023-08-01 22:33:19 +02:00
Alexey Milovidov	0641dfd7e6	Add missing modification	2023-08-01 22:33:04 +02:00
Alexey Milovidov	97bd68d5c7	Merge branch 'master' into sqltest	2023-08-01 22:31:18 +02:00
Mikhail f. Shiryaev	f87e53c042	Merge branch 'master' into sqltest	2023-08-01 17:42:58 +02:00
Mikhail f. Shiryaev	6b29cccf3e	Rename docker_images.json to docker_images_for_tests.json	2023-08-01 17:30:54 +02:00
Mikhail f. Shiryaev	79d49a006c	Fix the PR body check for Reverts #number	2023-08-01 17:06:23 +02:00
Nikolai Kochetov	09cb5cbe87	Try to continue clickhouse process in stress test after terminating gdb.	2023-08-01 14:30:41 +00:00
Mikhail f. Shiryaev	0931e87989	Merge branch 'master' into sqltest	2023-08-01 16:14:28 +02:00
Alexey Milovidov	8646eefc9c	Update docker_test.py	2023-08-01 17:05:24 +03:00
Mikhail f. Shiryaev	e5087d14ab	Merge branch 'master' into fix-packager	2023-08-01 12:04:06 +02:00
Kseniia Sumarokova	40ce9cf9ed	Merge pull request #51843 from ClickHouse/kssenii-nginx ... Replace kssenii/nginx-test by clickhouse/nginx-dav	2023-08-01 11:36:20 +02:00
Alexey Milovidov	afb76101b7	Remove obsolete part of a check name	2023-07-31 05:23:28 +02:00
Alexey Milovidov	d243602e74	Merge pull request #52773 from ClickHouse/upload-build-check-statistics ... Upload build statistics to the CI database	2023-07-31 06:18:41 +03:00
Alexey Milovidov	8155659f87	Merge pull request #52775 from ClickHouse/remove-coverity-2 ... Remove Coverity (part 2)	2023-07-30 12:49:04 +03:00
Alexey Milovidov	bc1f80dd82	Remove Coverity (part 2)	2023-07-30 10:25:44 +02:00
robot-clickhouse	358773c620	Automatic style fix	2023-07-30 07:45:27 +00:00
Alexey Milovidov	448756d148	Add machine_type information to the CI database	2023-07-30 09:31:21 +02:00
Alexey Milovidov	41ffc09771	Upload build statistics to the CI database	2023-07-30 09:18:06 +02:00
Kruglov Pavel	93e10077ba	Fix attaching gdb in stress tests (#51445) ... * Fix attaching gdb in stress tests * Fix * Update run.sh * Try remove run_with_retry * Return run_with_retry * Don't set -e in run_with_retry if it was't set before * Update tests/ci/utils.lib * Fix bash --------- Co-authored-by: Alexander Tokmakov <tavplubix@clickhouse.com>	2023-07-26 12:53:19 +03:00
Azat Khuzhin	0b258dda4e	Reproducible builds for Rust ... From now on cargo will not download anything from the internet during builds. This step had been moved for docker image builds (via cargo vendor). And now cargo inside docker.io/clickhouse/binary-builder will not use any crates from the internet, so we don't need to add --offline for cargo commands in cmake (corrosion_import_crate()). Also the docker build command had been adjusted to allow following symlinks inside build context, by using tar, this is required for Rust packages. Note, that to make proper Cargo.lock that could be vendored I did the following: - per-project locks had been removed (since there is no automatic way to sync the workspace Cargo.lock with per-project Cargo.lock, since cargo update/generate-lockfile will use only per-project Cargo.toml files apparently, -Z minimal-versions does not helps either) - and to generate Cargo.lock with less changes I've pinned version in the Cargo.toml strictly, i.e. not 'foo = "0.1"' but 'foo = "=0.1"' then the Cargo.lock for workspace had been generated and afterwards I've reverted this part. Plus I have to update the dependencies afterwards, since otherwise there are conflicts with dependencies for std library. Non trivial. Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>	2023-07-22 22:46:22 +02:00
Alexey Milovidov	c3c8ea9c27	Do something	2023-07-22 17:30:55 +02:00
Mikhail f. Shiryaev	c6e9f621f1	Replace kssenii/nginx-test:1.1 by clickhouse/nginx-dav	2023-07-22 15:45:47 +02:00
Alexey Milovidov	3398355e46	Add something with unclear purpose	2023-07-22 02:04:30 +02:00
Mikhail f. Shiryaev	101214cb8f	Fix broken termination lambda by caching runners for at most 15 seconds	2023-07-21 18:32:32 +02:00
Mikhail f. Shiryaev	083be24a74	Improve logic of checking valid cached values in lambdas	2023-07-21 18:32:31 +02:00
Mikhail f. Shiryaev	a117997d5b	Show the runner type in the job logs	2023-07-21 18:32:31 +02:00
Mikhail f. Shiryaev	da37f32d4b	Attempt to delete the runner before shutting down	2023-07-21 18:32:30 +02:00
Mikhail f. Shiryaev	538d1f7c17	Kill the runner process with all subprocesses	2023-07-21 18:32:28 +02:00
Antonio Andelic	e181e602a7	Update ci_config.py	2023-07-21 12:43:32 +02:00
Alexey Milovidov	f458108c44	Fix style	2023-07-19 22:15:23 +02:00
Alexey Milovidov	29094a22cf	Fix Docker	2023-07-19 20:00:00 +02:00
Alexey Milovidov	f7fcf079a4	Merge branch 'master' of github.com:ClickHouse/ClickHouse into sqltest	2023-07-19 19:58:10 +02:00
Mikhail f. Shiryaev	9c4005b33f	Add logging about all found workflows for merge_pr.py	2023-07-19 15:12:58 +02:00
Alexey Milovidov	8e3bd222f0	Add SQLTest	2023-07-19 03:42:32 +02:00
Antonio Andelic	3c2507ad7d	Merge branch 'master' into fix-packager	2023-07-18 09:18:03 +02:00
Alexey Milovidov	20b77e946a	There is no point in detecting flaky tests	2023-07-15 23:00:23 +02:00
Dmitry Novik	ee2392bf8e	Merge remote-tracking branch 'origin/master' into analyzer-integration-tests	2023-07-11 08:55:31 +00:00
Alexey Milovidov	6cfeef00d7	Merge pull request #31398 from ClickHouse/riscv-in-ci ... Add Linux RISC-V 64 build to CI	2023-07-06 22:49:16 +03:00
Antonio Andelic	f825c85c72	Merge branch 'master' into fix-packager	2023-07-06 08:36:43 +02:00
Alexey Milovidov	5a3aadacde	Fix error	2023-07-05 23:40:37 +02:00
Antonio Andelic	3dc0afd52f	Correctly grep archives	2023-07-05 11:19:15 +00:00
Alexey Milovidov	3021180e01	Update	2023-07-04 21:33:12 +02:00
Alexey Milovidov	1eafba60d2	Merge branch 'master' into riscv-in-ci	2023-07-04 21:28:05 +02:00
Antonio Andelic	5b31c714a0	Merge branch 'master' into fix-packager	2023-07-04 15:48:31 +02:00
Alexey Milovidov	aa2b8aa893	Merge pull request #51769 from ClickHouse/mypy-1.4.0 ... Version mypy=1.4.1 falsly reports unused ignore comment	2023-07-04 16:45:57 +03:00
Yatsishin Ilya	5230503651	remove unused type: ignore	2023-07-04 12:59:25 +00:00
Antonio Andelic	6d3dbae8aa	Merge remote-tracking branch 'origin/docs-packaging' into fix-packager	2023-07-04 11:21:53 +00:00
Alexey Milovidov	463c2ba5ce	Revert "Add documentation for building in docker"	2023-07-04 14:07:10 +03:00
Mikhail f. Shiryaev	c715ee5cef	Replace --build-type=debug by --debug-build	2023-07-03 19:58:47 +02:00
Mikhail f. Shiryaev	469da1831a	Replace --build-type=debug by --debug-build	2023-07-03 17:42:15 +02:00
Antonio Andelic	c06afc53b4	Make scripts backwards compatible	2023-07-02 09:57:06 +00:00
Antonio Andelic	00aeb407e5	Wait with retries	2023-06-30 14:52:21 +00:00
Alexey Milovidov	1e6f6b522c	Merge pull request #51180 from ClickHouse/update-tests-docker ... Update tests' docker containers	2023-06-30 11:24:59 +03:00
Alexey Milovidov	b6a37fbf18	Merge branch 'master' into many-fixes-2	2023-06-29 22:36:33 +03:00
Mikhail f. Shiryaev	953f1c7885	Upload fast test binaries to builds bucket	2023-06-28 22:19:33 +02:00
Mikhail f. Shiryaev	eeb8cdbc19	Add way to define additional urls in test reports	2023-06-28 22:19:00 +02:00
vdimir	e2241b8c7c	Merge branch 'grace_hash_join_debug_2306' into many-fixes-2	2023-06-28 09:35:20 +00:00
robot-clickhouse	9d8b996f68	Automatic style fix	2023-06-27 04:30:56 +00:00
Alexey Milovidov	b5635887c3	Merge remote-tracking branch 'origin/refactor-subqueries-for-in' into many-fixes	2023-06-27 05:39:44 +02:00
Alexey Milovidov	e0ac127bdd	Merge branch 'two-tests-are-longer-with-analyzer' into many-fixes	2023-06-27 05:23:23 +02:00
Alexey Milovidov	d37b1c17f5	Merge branch 'fix-fasttest' into many-fixes	2023-06-27 05:21:43 +02:00
lgbo-ustc	179a7ce202	debug	2023-06-26 20:43:19 +08:00
Mikhail f. Shiryaev	065b87aa30	Improve logging a little bit, avoid double labeling	2023-06-26 13:25:49 +02:00
Mikhail f. Shiryaev	c64f929b9c	Fix broken labeling for manual approve	2023-06-26 13:25:48 +02:00
Alexey Milovidov	b8e6bd3299	Merge branch 'master' into refactor-subqueries-for-in	2023-06-26 06:05:12 +03:00
Alexey Milovidov	4ee74ec213	Two tests are twice longer in average with Analyzer and sometimes failing	2023-06-26 02:19:52 +02:00
Alexey Milovidov	4fd1fc4507	Fix typo	2023-06-25 06:14:31 +02:00
Alexey Milovidov	511b2cdc21	Increase timeout of fast test	2023-06-25 06:10:57 +02:00
Ilya Yatsishin	9a75faad77	Merge pull request #51340 from ClickHouse/qoega-patch-4	2023-06-24 14:22:32 +02:00
Ilya Yatsishin	7e3eaadaac	Make SQLancer check not "always green"	2023-06-23 18:55:04 +02:00
Alexey Milovidov	b6e7f4cf6c	Require more checks	2023-06-22 23:24:27 +02:00
Nikolai Kochetov	a940031878	Merge branch 'master' into refactor-subqueries-for-in	2023-06-22 12:18:48 +02:00
Mikhail f. Shiryaev	02776b8a15	Merge pull request #51163 from ClickHouse/fasttest-ignore-chown-fail ... Ignore errors for `chown` in fasttests	2023-06-22 09:51:57 +02:00
Mikhail f. Shiryaev	1591633d72	Merge pull request #49283 from ClickHouse/runners-survival ... Attempt to increase the general runners' survival rate	2023-06-22 00:16:11 +02:00
Mikhail f. Shiryaev	118c3af4d6	Ignore errors for chown in fasttests	2023-06-21 12:59:21 +02:00
Robert Schulze	bc7df2baca	Merge branch 'master' into non-experimental-qpl-deflate	2023-06-20 20:06:01 +02:00
Mikhail f. Shiryaev	23d843d379	Experiment with decreasing scale_up for style-checkers	2023-06-20 15:17:33 +02:00
Mikhail f. Shiryaev	1a8571c465	Check and proceed spot termination on runner setup	2023-06-19 15:40:32 +02:00
Mikhail f. Shiryaev	4d5160a3ba	Improve determination of job received	2023-06-19 15:40:32 +02:00
Mikhail f. Shiryaev	d782b05845	Continue LCH for terminated instances, add logging	2023-06-19 15:40:31 +02:00
Mikhail f. Shiryaev	d8e3d5c803	Kill the instance proactively on spot termination	2023-06-19 15:40:30 +02:00
Mikhail f. Shiryaev	d672bd3382	Stop the runner if there are no running jobs for long	2023-06-19 15:40:30 +02:00
Mikhail f. Shiryaev	915f431b95	Tune the shutting down on rebalance signal	2023-06-19 15:40:28 +02:00
Mikhail f. Shiryaev	026ab3d4ce	Calculate current capacity according to HealthStatus	2023-06-19 15:40:27 +02:00
Mikhail f. Shiryaev	8feb5146fc	Do not "update" the token and the instances infinitely	2023-06-19 15:40:27 +02:00
Mikhail f. Shiryaev	a5e31e98de	Decouple cleaning part from metrics lambda to a new one	2023-06-19 15:40:26 +02:00
Mikhail f. Shiryaev	1688ba5cf5	Use ephemeral runners for GH actions	2023-06-19 15:40:25 +02:00
Mikhail f. Shiryaev	47ea742076	Do not kill runner on the fly, delay 3 seconds	2023-06-19 15:40:25 +02:00
Mikhail f. Shiryaev	6cecdb8fca	Shutdown runner process and do not restart it on termination	2023-06-19 15:40:24 +02:00
Mikhail f. Shiryaev	197d0d1314	Process lifecycle hook in post-running action hook	2023-06-19 15:40:24 +02:00
Mikhail f. Shiryaev	d45092d63b	Process the rebalance recomendation event on post-run	2023-06-19 15:40:23 +02:00
Mikhail f. Shiryaev	ee5ca7337e	Add reset to Stopwatch, improve codebrowser report	2023-06-16 12:51:31 +02:00
Dmitry Novik	49f88f5873	Fix ENV variable name	2023-06-15 14:15:01 +00:00
Mikhail f. Shiryaev	eb2a5c31a3	Do not mock git_helper.Git, suppress git describe error for ignore_no_tags	2023-06-15 15:35:03 +02:00
Mikhail f. Shiryaev	2ac95ccfce	Adjust version_helper test cases to proper tags	2023-06-15 14:48:10 +02:00
Mikhail f. Shiryaev	07203a4542	Fix a versions' tweak for tagged commits, improve version_helper	2023-06-15 14:48:09 +02:00
robot-clickhouse	1230519bec	Automatic style fix	2023-06-14 13:38:44 +00:00
Dmitry Novik	fe8172fbd9	Review fixes	2023-06-14 13:17:04 +00:00
Dmitry Novik	afcc3aca36	Update ci_config.py	2023-06-14 13:14:59 +00:00
Dmitry Novik	1207085dec	Merge branch 'master' into analyzer-integration-tests	2023-06-13 17:09:29 +02:00
Nikita Mikhaylov	52a460df67	Tests with parallel replicas are no more "always green" (#50896)	2023-06-13 16:43:35 +02:00
Alexander Tokmakov	5368fa72ca	Merge pull request #50487 from Avogar/add-gdb-to-stateless-tests ... Attach gdb in stateless tests	2023-06-13 17:04:17 +03:00
Dmitry Novik	9e8ca5f5ae	Add integration test check with enabled analyzer	2023-06-13 13:09:39 +00:00
Robert Schulze	8222861f2e	Merge remote-tracking branch 'rschu1ze/master' into non-experimental-qpl-deflate	2023-06-11 14:39:30 +00:00
Nikolai Kochetov	6ce8329bda	Merge branch 'master' into refactor-subqueries-for-in	2023-06-09 20:04:27 +02:00
Nikolai Kochetov	a878312fdf	Merge pull request #50693 from ClickHouse/vdimir/disable_grace_hash_stress ... Disable grace_hash join in stress tests	2023-06-09 17:57:34 +02:00
Robert Schulze	1aa158909e	enable_qpl_deflate_codec --> enable_deflate_qpl_codec	2023-06-09 12:43:33 +00:00
jinjunzh	f1192d59af	refine patch according to comments	2023-06-09 12:43:15 +00:00
jinjunzh	31173ab55b	add sections of deflate_qpl for stress test and performance test	2023-06-09 12:43:08 +00:00
Mikhail f. Shiryaev	2aaaec2abe	Catch issues with dockerd during the build	2023-06-08 14:01:41 +02:00
vdimir	2297995c1a	Disable grace_hash join in stress tests ... Until https://github.com/ClickHouse/ClickHouse/issues/50220 is fixed	2023-06-08 07:15:11 +00:00
Antonio Andelic	26fb80b540	Merge pull request #50615 from ClickHouse/fix-jepsen-check ... Fix Jepsen runs in PRs	2023-06-07 08:43:51 +02:00
Sema Checherinda	990652119e	Merge pull request #50361 from nickitat/fail_perf_on_too_many_slow ... Fail perf tests when too many queries slowed down	2023-06-06 11:33:04 +02:00
Antonio Andelic	0bc31a7288	Fix jepsen runs in PRs	2023-06-06 07:17:30 +00:00
robot-clickhouse-ci-1	4cf5a13e5b	Merge pull request #50160 from ClickHouse/prio-runners ... Refactor lambdas, prepare to prio runners	2023-06-06 07:29:28 +02:00
avogar	e8c6c7967b	Move attaching gdb to separate lib	2023-06-05 18:21:40 +00:00
Nikita Taranov	e548dce123	fix	2023-06-03 00:08:47 +02:00
Nikita Taranov	4506299d73	impl	2023-06-02 23:14:09 +02:00
pufit	0d98a46326	Fix KeyError in cherry-pick	2023-06-02 13:02:18 -04:00
Mikhail f. Shiryaev	9bd0a53e7c	Get only online instances in get_lost_ec2_instances	2023-06-02 12:27:29 +02:00
Mikhail f. Shiryaev	3c13eaa159	Simplify get_dead_runners_in_ec2 a little bit, update runners	2023-06-02 12:27:28 +02:00
Mikhail f. Shiryaev	b775b5cfd6	Move all CI runners metrics into one namespace	2023-06-02 12:27:27 +02:00
Mikhail f. Shiryaev	e657e2ba10	Additional logging in autoscale_runners_lambda	2023-06-02 12:27:26 +02:00
Mikhail f. Shiryaev	b7c5fdab77	Move lambda package building to public.ecr.aws/lambda/python for compatibility	2023-06-02 12:27:26 +02:00
Mikhail f. Shiryaev	0ddd53088d	Add a new runner type for ci metrics and autoscaling	2023-06-02 12:27:24 +02:00
Mikhail f. Shiryaev	e8b03d7498	Move insert part for ClickHouseHelper to shared	2023-06-01 21:03:57 +02:00
Mikhail f. Shiryaev	2dca0eac1b	Delete __init__.py in lambda directories to break subpackage	2023-06-01 21:03:56 +02:00
Mikhail f. Shiryaev	acb9531ebf	Unify and put GH access token to the lambda_shared	2023-06-01 21:03:56 +02:00
Mikhail f. Shiryaev	0fa6a84161	Move the stuff related to runners to lambda_shared	2023-06-01 21:03:55 +02:00
Mikhail f. Shiryaev	7f08f218d9	Create lambda_shared package for lambdas	2023-06-01 21:03:53 +02:00
Mikhail f. Shiryaev	484c91c47e	Add DRY_RUN and configurable PY_VERSION to lambda deployment	2023-06-01 21:03:53 +02:00
Mikhail f. Shiryaev	27941b4d26	Decrease the time window for autoscale_runners_lambda	2023-06-01 21:03:52 +02:00
Mikhail f. Shiryaev	db02938411	Do not count unfinished tasks with conclusion=None	2023-06-01 21:03:51 +02:00
Mikhail f. Shiryaev	855afb56f9	Get instances for the region to not kill a fresh one	2023-06-01 21:03:50 +02:00
Mikhail f. Shiryaev	bf9b563e0b	Improve caching mechanism for token, add cached instances	2023-06-01 21:03:50 +02:00
Mikhail f. Shiryaev	7bf9089dcd	Increase access_token cached time	2023-06-01 21:03:49 +02:00
Mikhail f. Shiryaev	985cd8fc8a	Improve events logging	2023-06-01 21:03:48 +02:00
Nikita Taranov	e5c21e588a	Update app.py (#50407)	2023-06-01 15:19:47 +02:00
Mikhail f. Shiryaev	9089360688	Merge pull request #50415 from ClickHouse/build-report-pending ... Mark the builds without results as pending	2023-06-01 14:53:09 +02:00
Mikhail f. Shiryaev	de7179876e	Merge pull request #47151 from ClickHouse/increase-vm-max_map_count ... Update ubuntu_ami_for_ci.sh	2023-06-01 13:54:21 +02:00
Mikhail f. Shiryaev	6a5ae6558a	Add comment for usage of prepare-ci-ami.sh	2023-06-01 12:58:03 +02:00