Commit Graph

824 Commits

Author SHA1 Message Date
alesapin
bf591b971a Moar categories 2022-03-29 15:53:08 +02:00
alesapin
d0f01516db Resurrect automatic labelling 2022-03-29 15:48:57 +02:00
alesapin
ebd72e433d Fail checks in case of errors 2022-03-29 14:41:47 +02:00
Alexey Milovidov
5e262fba85
Merge pull request #35204 from azat/build-gcc
Add build with GCC
2022-03-29 04:55:15 +03:00
alesapin
7ab8d7178c
Merge pull request #35308 from ClickHouse/clickhouse-keeper
New package clickhouse-keeper
2022-03-28 21:23:17 +02:00
Mikhail f. Shiryaev
ea7939103a
Add debug information about event, rerun only failed jobs 2022-03-28 19:15:11 +02:00
alesapin
8acb5de5b5 Fix build report black 2022-03-28 14:43:15 +02:00
alesapin
6780defb42 Fix build report check 2022-03-26 22:45:45 +01:00
Azat Khuzhin
806a4bcdee Add build with GCC
v2: Add gcc build to "ClickHouse special build check (actions)"
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-03-26 23:42:46 +03:00
alesapin
2712368f78 Push only to the new CI DB 2022-03-25 17:44:53 +01:00
alesapin
137dd93ee0
Merge pull request #35583 from ClickHouse/docker-labels
Docker labels
2022-03-24 20:50:29 +01:00
Mikhail f. Shiryaev
d90627e82f
Fix version string update, fix #35518 2022-03-24 17:50:31 +01:00
Mikhail f. Shiryaev
d6fc6b9c44
Add build-url label to built docker images 2022-03-24 16:24:06 +01:00
Mikhail f. Shiryaev
3516eb0524
Make GITHUB_RUN_URL variable and use it 2022-03-24 15:37:53 +01:00
alesapin
881d5ac44f Don't put red cross if jepsen check couldn't wait for build 2022-03-23 11:32:16 +01:00
Mikhail f. Shiryaev
e6f5a3f98b
Apply black formatter to all *.py files in the repo 2022-03-22 17:39:58 +01:00
Vladimir C
ac6fef5c70
Merge pull request #35124 from vdimir/ci-bugfix-validation 2022-03-22 17:34:15 +01:00
vdimir
41c32e24fe
Skip bugfix_validate_check when no pr-bugfix 2022-03-22 12:20:27 +01:00
Mikhail f. Shiryaev
b9cb64a4a1
Add apk package and reports 2022-03-22 11:11:38 +01:00
Mikhail f. Shiryaev
a0edb3e593
Build tgz and rpm for every running 2022-03-22 11:10:01 +01:00
Mikhail f. Shiryaev
0dd1f4af76
Rename alien arguments and parameters 2022-03-22 11:10:01 +01:00
Mikhail f. Shiryaev
342f7ba567
Migrate to nfpm 2022-03-22 11:09:59 +01:00
vdimir
daa716f682
fix style 2022-03-18 21:32:59 +01:00
vdimir
2551adc416
Bugfix validate meta check 2022-03-18 16:31:19 +01:00
Mikhail f. Shiryaev
2c6ce4b4a0
Merge pull request #35382 from ClickHouse/push-artifactory-improvement
Push LTS packages to both lts and stable repos
2022-03-18 13:10:37 +01:00
Mikhail f. Shiryaev
0d259c6c29
Push LTS packages to both lts and stable repos 2022-03-17 20:59:46 +01:00
alesapin
d3a5839eb7
Merge pull request #35093 from ClickHouse/stress-test-runlog
Stress test runlog
2022-03-17 20:29:06 +01:00
Mikhail f. Shiryaev
b18daeab6f
Fix bug in push_to_artifactory.py, create a necessary dir 2022-03-17 17:44:01 +01:00
Mikhail f. Shiryaev
d593fda8d5
Fix logic for --no-prestable 2022-03-17 16:50:20 +01:00
Vladimir C
a4c410a1f5
Update tests/ci/docker_pull_helper.py
Co-authored-by: Mikhail f. Shiryaev <mr.felixoid@gmail.com>
2022-03-14 15:55:09 +00:00
vdimir
9f2feb4f0b
Use argparse in functional/integration_test_check.py 2022-03-14 15:52:46 +00:00
vdimir
cf70b79f74
Revert "Bugfix check requires either functional _or_ stateless test"
This reverts commit b8b64b1d15.
2022-03-14 15:43:15 +00:00
vdimir
6f1cf419ac
Fix typo 2022-03-14 13:24:46 +00:00
vdimir
b8b64b1d15
Bugfix check requires either functional _or_ stateless test 2022-03-14 13:09:53 +00:00
Azat Khuzhin
4b770343e1 Mark build action as failed if it was retried and there was no succeeded builds
Builder actions does not shows errors after retries, for example [1]
failed, but the check is green:

    Dump json report {
      'log_url': 'https://s3.amazonaws.com/clickhouse-builds/35204/c5a71a7f10e9d20b116b3e1db02bb444b203c32a/binary_gcc/build_log.log',
      'build_urls': [], <-- empty
      'build_config': {...},
      'elapsed_seconds': 0, 'status': False
    } to build_urls_binary_gcc.json with env build_urls_{build_name}

  [1]: https://github.com/ClickHouse/ClickHouse/runs/5510262456?check_suite_focus=true#logs

And previous build indeed failed [2]:

    Mar 11 08:19:45 ninja: build stopped: subcommand failed.

  [2]: https://s3.amazonaws.com/clickhouse-builds/35204/c5a71a7f10e9d20b116b3e1db02bb444b203c32a/binary_gcc/build_log.log

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-03-14 13:23:51 +03:00
kssenii
e08f5c3d86 Add s3 tests with debug and tsan 2022-03-14 11:08:33 +01:00
vdimir
e5c63266c2
Integration test bugfix check 2022-03-11 14:14:16 +00:00
vdimir
e757837ec0
Different status colors in report for bugfix validation 2022-03-11 13:36:29 +00:00
vdimir
6614d6eaaf
bugfix validate integration test 2022-03-11 12:50:33 +00:00
Alexey Milovidov
415e2ff223 Remove useless file 2022-03-11 00:49:50 +01:00
vdimir
b8c7e4657f
invert check in validate bugfix 2022-03-10 16:43:44 +00:00
vdimir
000a31df3b
no-random-settings for bugfix validate 2022-03-10 16:43:43 +00:00
vdimir
fcb7e9ed36
fix CLICKHOUSE_CLIENT_DOWNLOAD_URL in download_previous_release.py 2022-03-10 16:43:42 +00:00
vdimir
4947d6db13
Use download_previous_release for bugfix validation
Should be merged with https://github.com/ClickHouse/ClickHouse/pull/27928
2022-03-10 16:43:41 +00:00
vdimir
196645679f
Add bugfix validate check to ci_config 2022-03-10 16:43:41 +00:00
vdimir
da179c607e
Add bugfux validate check 2022-03-10 16:43:40 +00:00
Mikhail f. Shiryaev
74c5873be0
Do not upload runlog two times 2022-03-07 14:31:17 +01:00
Mikhail f. Shiryaev
3e852ae461
Apply black to stress tests script 2022-03-07 14:25:06 +01:00
vdimir
202ac18e76
Skip 01086_odbc_roundtrip for aarch, disable force_tests 2022-03-07 11:04:37 +00:00
Mikhail f. Shiryaev
223ec3d0b6
Put downloaded artifacts to a temorary path 2022-03-06 14:07:47 +01:00
vdimir
9032cc6c3d
Fix stylecheck for tests/ci/release.py 2022-02-25 11:17:45 +00:00
vdimir
5c38203f7d
Argument --commit for release.py is required 2022-02-25 11:06:58 +00:00
vdimir
ef3cbdcf6a
Add check_prerequisites to release.py 2022-02-25 10:53:21 +00:00
Mikhail f. Shiryaev
af19c5c56d
Merge pull request #34828 from ClickHouse/team-keys-lambda
Comment output header for team keys
2022-02-23 10:53:28 +01:00
Mikhail f. Shiryaev
a2f025b292
Comment output header for team keys 2022-02-23 10:51:31 +01:00
Mikhail f. Shiryaev
2386e1ee54
Fix linter issues 2022-02-22 11:40:40 +01:00
Mikhail f. Shiryaev
dd180b83a8
Improve --no-* parameters for scripts 2022-02-22 11:40:39 +01:00
Mikhail f. Shiryaev
c44aeda23c
Merge pull request #34740 from ClickHouse/release-stable
Add a patch release for stable/lts packages to release.py
2022-02-22 11:39:23 +01:00
Mikhail f. Shiryaev
f0c7a7017f
Review points: update -> read_version, comment in _create_gh_label 2022-02-22 11:32:05 +01:00
Mikhail f. Shiryaev
214d35df5e
Add unit tests for git helper 2022-02-21 10:56:04 +01:00
Mikhail f. Shiryaev
b39dfc9ba6
Change regexp to be more strict 2022-02-21 10:19:59 +01:00
alesapin
a5c2ee339f
Update report.py 2022-02-21 10:29:10 +03:00
vdimir
c70901b4e7
Sort additional_html_urls in test report in CI 2022-02-20 20:36:36 +01:00
Mikhail f. Shiryaev
d2cbc5af40
Add patch type for stable/lts releases 2022-02-19 00:13:07 +01:00
Mikhail f. Shiryaev
df9165bbff
Refactoring of Release, add Repo
- Do not pass Namespace to Release.do
- Decouple repository to another class
2022-02-19 00:13:00 +01:00
Mikhail f. Shiryaev
0960fbad52
Set selease_branch only once as Release property 2022-02-19 00:10:21 +01:00
Mikhail f. Shiryaev
d2a8cff508
Add release_branch checker 2022-02-19 00:01:42 +01:00
Mikhail f. Shiryaev
0f1565a3a2
Clean-up the code a little bit 2022-02-19 00:01:39 +01:00
alexey-milovidov
9630087131
Update app.py 2022-02-19 00:26:43 +03:00
alexey-milovidov
37fbf7c329
Merge branch 'master' into add-trusted-taiyang-li 2022-02-19 00:25:52 +03:00
Vladimir C
a249819c52
Merge pull request #34730 from ucasfl/name
rename name in ci check
2022-02-18 16:17:11 +01:00
feng lv
3270e5f636 rename name in ci check 2022-02-18 14:06:16 +00:00
vdimir
4ce64891f9
Add taiyang-li to trusted contributors 2022-02-18 13:59:47 +01:00
Mikhail f. Shiryaev
462b817759
Merge pull request #34624 from ClickHouse/timeout-fast-tests
Timeout fast tests
2022-02-17 17:25:12 +01:00
Mikhail f. Shiryaev
e8f927283c
Add timeouts to TeePopen, second attempt 2022-02-17 16:44:34 +01:00
alesapin
8a0153b6d3
Merge pull request #34606 from qoega/no-clickhouse-test-deb
Remove clickhouse-test deb completely
2022-02-17 18:13:44 +03:00
Mikhail f. Shiryaev
1aff775574
Apply black formatter, fix linter notes 2022-02-17 16:11:12 +01:00
Mikhail f. Shiryaev
c3bf9eb2f9
Remove PVS test 2022-02-17 12:59:47 +01:00
Mikhail f. Shiryaev
0c41564b27
Add the rollback command logging on success run 2022-02-16 20:51:28 +01:00
Mikhail f. Shiryaev
015c0ab611
Solve the issue with wrong initial version 2022-02-16 17:46:43 +01:00
Mikhail f. Shiryaev
b2f3a4ccab
Fix issues with labeling 2022-02-16 17:42:44 +01:00
Mikhail f. Shiryaev
00c8d74878
Reorder Release class 2022-02-16 15:59:51 +01:00
Mikhail f. Shiryaev
799d0bc580
Clean stale remainings, create necessary labels for prestable 2022-02-16 15:56:49 +01:00
Mikhail f. Shiryaev
3f295f9429
Merge pull request #34644 from ClickHouse/fix-release-ci
Fix release ci
2022-02-16 14:33:55 +01:00
Mikhail f. Shiryaev
14a1fa40e7
Fix the issue with ReleaseCI PRInfo 2022-02-16 13:33:10 +01:00
Mikhail f. Shiryaev
e9c95d05b1
Create a release script 2022-02-16 12:17:08 +01:00
Mikhail f. Shiryaev
e338396a76
Use regexps from git_helper 2022-02-16 12:16:44 +01:00
Mikhail f. Shiryaev
546cee5512
Huge refactoring of version_helper 2022-02-16 12:16:42 +01:00
Mikhail f. Shiryaev
cac6f1cd69
Apply black to version_helper 2022-02-16 12:16:40 +01:00
Mikhail f. Shiryaev
0e04279e45
Revert "Timeout fast tests" 2022-02-15 23:50:21 +01:00
Maksim Kita
9b1afa6dda
Merge pull request #34614 from ClickHouse/timeout-fast-tests
Timeout fast tests
2022-02-15 20:47:19 +01:00
alesapin
bc2d0ee7c7
Merge pull request #34215 from ClickHouse/revert-34211-revert-34153-add_func_tests_over_s3
Add func tests run with s3 and fix several bugs
2022-02-15 19:07:11 +03:00
Mikhail f. Shiryaev
bf59d9c5cd
TOREMOVE: test working timeout 2022-02-15 16:52:06 +01:00
Mikhail f. Shiryaev
996f0893b8
Set timeout for fast tests to 40 minutes 2022-02-15 16:51:19 +01:00
Mikhail f. Shiryaev
a9aeca9a94
Apply black formatter, fix linter notes 2022-02-15 16:50:24 +01:00
Mikhail f. Shiryaev
c906375358
Pass timeout into Popen optionally 2022-02-15 16:48:16 +01:00
Yatsishin Ilya
bbe2c07a7b remove clickhouse-test deb completely 2022-02-15 12:03:51 +00:00
alesapin
b75d551281 Fix clang tidy and add check for master 2022-02-14 14:37:41 +03:00
Mikhail f. Shiryaev
23546ab825
Use the current kerberized-haddop image 2022-02-13 22:01:45 +01:00
alesapin
89373155fc Merge branch 'master' into revert-34211-revert-34153-add_func_tests_over_s3 2022-02-13 21:07:54 +03:00
Azat Khuzhin
4965586e7e
More gdb introspection on CI (#34517)
* test/fuzzer: preserve core

This may help to capture things like in [1]:

    Failed assertion: "bin->low_bits_full > bin->low_bits_empty" Received signal 6 Received signal Aborted (6)

  [1]: https://s3.amazonaws.com/clickhouse-test-reports/33057/19216f4c0ae0f72108c147f958a708b521ad27dc/fuzzer_astfuzzerdebug,actions//report.html

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>

* test: do not run 'info locals' since 'backtrace full' includes it

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>

* test: try capture backtrace from all threads

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>

* test/stress: fix path for core artifacts

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
Fixes: #33389

* test/fuzzer: store core file in artifacts

v2: fix report, because of undefined variable CORE_LINK
v3: fix case when there is no core file

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-02-13 15:02:15 +03:00
Mikhail f. Shiryaev
0f21981694
Add a flag for images architecture 2022-02-11 18:26:45 +01:00
Mikhail f. Shiryaev
bd3812a462
Add a comment 2022-02-11 18:26:44 +01:00
Mikhail f. Shiryaev
dcecbbaf81
Add argument --all for docker imabes builder 2022-02-11 18:26:42 +01:00
alesapin
ba28c2c013 Merge branch 'master' into revert-34211-revert-34153-add_func_tests_over_s3 2022-02-07 12:44:56 +03:00
Mikhail f. Shiryaev
f2725b1b22
Fix wrong OK_SKIP labels 2022-02-05 18:26:43 +01:00
alesapin
2ed45b2a98 Merge branch 'master' into revert-34211-revert-34153-add_func_tests_over_s3 2022-02-04 11:23:46 +03:00
Mikhail f. Shiryaev
37179bbc9c
Add verbosity to stylecheck 2022-02-03 17:50:35 +01:00
Mikhail f. Shiryaev
8243823d74
Mark Run Check as success for some cases 2022-02-03 13:32:22 +01:00
alesapin
e324529f17 S3 all tests 2022-02-02 20:48:36 +03:00
alesapin
80800e051e Merge branch 'master' into revert-34211-revert-34153-add_func_tests_over_s3 2022-02-02 19:42:04 +03:00
Mikhail f. Shiryaev
75adf5d12d
Merge pull request #33948 from qoega/no-clickhouse-test-deb-in-stateless
Do not use clickhouse-test.deb in stateless CI checks
2022-02-02 10:23:33 +01:00
alesapin
18f08ed932
Revert "Revert "Add func tests run with s3"" 2022-02-01 10:20:06 +03:00
alexey-milovidov
cbfcd45be3
Revert "Add func tests run with s3" 2022-02-01 05:46:13 +03:00
Mikhail f. Shiryaev
424900bf1f
Add authorisation for dockerhub proxy container 2022-01-31 15:50:05 +01:00
alesapin
9eea8e3183 Fix missed change 2022-01-30 18:22:35 +03:00
alesapin
969987e66b Merge branch 'master' into add_func_tests_over_s3 2022-01-30 18:21:32 +03:00
alesapin
c219a53b4a Add func tests run with s3 2022-01-30 17:28:52 +03:00
Mikhail f. Shiryaev
22eddff0d5
Add a comment about broken logic for skipping checks 2022-01-26 15:30:59 +01:00
Mikhail f. Shiryaev
462e95d0fa
Replace labels_from_ip by pr_event_from_api 2022-01-26 15:03:30 +01:00
Mikhail f. Shiryaev
6fa6588cb5
Add info about body for a failed check 2022-01-26 14:03:01 +01:00
Mikhail f. Shiryaev
99e0ce1ab8
Add get_with_retries helper to build_download_helper.py 2022-01-26 13:20:08 +01:00
Mikhail f. Shiryaev
7cde981531
Apply black formatter to build_download_helper.py 2022-01-26 12:10:20 +01:00
Mikhail f. Shiryaev
1ecee3aa8e
Apply black formatter to pr_info 2022-01-26 12:09:35 +01:00
alesapin
677a7f1133
Merge pull request #33990 from ClickHouse/cancel-lambda
Cancel lambda
2022-01-26 10:51:21 +03:00
Mikhail f. Shiryaev
62ddfbe4ab
Use workflow names instead of ID 2022-01-25 18:32:13 +01:00
Mikhail f. Shiryaev
604af8d3b8
Apply black to cancel lambda 2022-01-25 18:28:17 +01:00
mergify[bot]
c859945b62
Merge branch 'master' into no-clickhouse-test-deb-in-stateless 2022-01-25 11:43:54 +00:00
Mikhail f. Shiryaev
fb908a31e2
Fix dependencies tree for docker image check 2022-01-25 12:09:10 +01:00
Yatsishin Ilya
bc10e08fb4 add missing space 2022-01-25 09:07:04 +00:00
Mikhail f. Shiryaev
57b4655fa2
Fix style for pr_info 2022-01-24 16:34:01 +01:00
Mikhail f. Shiryaev
8e8ee8f8d1
Add style-checker-aarch64 to metrics lambda 2022-01-24 15:50:34 +01:00
mergify[bot]
604b85d861
Merge branch 'master' into no-clickhouse-test-deb-in-stateless 2022-01-24 14:05:17 +00:00
Mikhail f. Shiryaev
7a1f471d6f
Add tests for process_image_with_parrents 2022-01-24 14:55:57 +01:00
Mikhail f. Shiryaev
70d997cbda
Fix pylint issues 2022-01-24 14:54:37 +01:00
Mikhail f. Shiryaev
18ef029ed3
Fix issue with building from non-existing tags 2022-01-24 14:54:29 +01:00
Mikhail f. Shiryaev
3531318388
Add tests for get_changed_docker_images 2022-01-24 14:52:07 +01:00
Mikhail f. Shiryaev
aa296c36bb
Fix dummy PRInfo 2022-01-24 14:51:56 +01:00
Yatsishin Ilya
3a55f7a7d6 Do not use clickhouse-test.deb in stateless 2022-01-24 11:53:42 +00:00
Alexey Milovidov
30b436b9b1 Add more retries in CI for AWS "spot" instances 2022-01-22 13:10:45 +03:00
alexey-milovidov
97e5e71a35
Merge pull request #33872 from ClickHouse/fix_rerun_perf_tests
Fix perf comparison rerun
2022-01-22 10:40:27 +03:00
alesapin
4b93607343 Fix perf comparison 2022-01-21 18:40:03 +03:00
Mikhail f. Shiryaev
9cb136f5b0
Use workflow names in approve lambda 2022-01-19 17:00:41 +01:00
Mikhail f. Shiryaev
d9a1284f2d
Improve typing and arguments 2022-01-19 11:41:34 +01:00
Mikhail f. Shiryaev
3142ae998b
Add option to skip redownload, push tgz 2022-01-19 11:41:33 +01:00
Mikhail f. Shiryaev
13368bf708
Fix some linter issues 2022-01-19 11:41:32 +01:00
Mikhail f. Shiryaev
7fd629004b
Improve push_to_artifactory
- Add tgz archives
- Add --all argument for all packages download
- Add -n to skip pushing to artifactory
- Add help for artifactory-url
- Make 3.8 compatible
- Add cleanup `refs/tags/` from --release argument
2022-01-19 11:41:30 +01:00
Mikhail f. Shiryaev
396b504397
Add some helping comments for API endpoints 2022-01-17 11:06:40 +01:00
Mikhail f. Shiryaev
91caf07b20
Apply black to workflow_approve_rerun_lambda 2022-01-15 18:54:58 +01:00
Mikhail f. Shiryaev
4c7a8abd20
Add PullRequestCI to NEED_RERUN_WORKFLOWS 2022-01-15 18:53:37 +01:00