Commit Graph

2460 Commits

Author SHA1 Message Date
Dmitry Novik
31df60be20 Merge remote-tracking branch 'origin/master' into enable-analyzer 2024-03-27 11:37:32 +01:00
robot-ch-test-poll2
5d2bc7ee34
Merge pull request #61948 from ClickHouse/ci_new_menu
CI: new ci menu in PR body
2024-03-27 01:56:17 +04:00
Max Kainov
813b60f129 Add handling for include/exclude ci filters 2024-03-26 19:34:05 +00:00
Dmitry Novik
65121b712e Merge remote-tracking branch 'origin/master' into enable-analyzer 2024-03-26 19:02:30 +01:00
Max Kainov
e4d9a74057 CI: create CIOptions class to handle ci user config
CI: create CIOptions class to handle ci user config
2024-03-26 15:39:07 +00:00
Nikita Taranov
34c8ec67e8 fix 2024-03-26 15:09:16 +00:00
Max K
c7e4d97e5a
Merge pull request #61881 from ClickHouse/ci_prepare_for_merge_queue
CI: merge queue support
2024-03-25 22:12:02 +01:00
Kruglov Pavel
de1229ba60
Merge branch 'master' into better-stress-upgrade-checks 2024-03-25 19:18:35 +01:00
Max Kainov
f9f830228a CI: MQ support 2024-03-25 17:57:03 +00:00
Max Kainov
727aeb7e37 CI: set timeout for install check
#job_Install_packages_amd64 #job_style_check
2024-03-25 17:38:39 +00:00
Nikita Taranov
3b87633392 better 2024-03-25 14:09:04 +00:00
Dmitry Novik
64301191b1 Merge remote-tracking branch 'origin/master' into enable-analyzer 2024-03-24 21:07:26 +00:00
Alexey Milovidov
b8a8d9994f
Merge pull request #60949 from nikmonkov/docker-version
Version has been added to docker labels
2024-03-24 01:50:50 +03:00
Dmitry Novik
df88eb26a5
Merge branch 'master' into enable-analyzer 2024-03-22 15:56:32 +01:00
Max K
d3965f27c1
Merge pull request #61542 from ClickHouse/ci_arm_support_kerberos
CI: ARM integration test - skip hdfs, kerberos, kafka
2024-03-22 15:25:30 +01:00
Max Kainov
e4556ae6c1 comments 2024-03-22 14:22:53 +00:00
Max Kainov
ef0fa1963d CI: use ci-runner as module
#no_merge_commit
 #ci_set_integration
 #batch_0
2024-03-22 14:22:53 +00:00
Dmitry Novik
d43a1a1984 Merge remote-tracking branch 'origin/master' into enable-analyzer 2024-03-22 14:15:57 +01:00
Nikolai Kochetov
0f60596f5b
Merge pull request #60548 from kitaisreal/merge-tree-read-split-ranges-into-intersecting-and-non-intersecting-fault-injection
MergeTree read split ranges into intersecting and non intersecting injection
2024-03-22 11:52:14 +01:00
Mikhail f. Shiryaev
771c549606
Process removed files, decouple _check_mime 2024-03-22 09:14:30 +01:00
Max Kainov
7d7025a74e CI: modify it 2024-03-22 00:13:17 +00:00
Max Kainov
389566f895 quick fix for style check 2024-03-21 20:24:22 +00:00
Max Kainov
d6b515b6bf CI: modify CI from PR body
#do_not_test
 #job_style_check
2024-03-21 19:35:07 +00:00
Maksim Kita
8ca9c1fb6b Merge branch 'master' into merge-tree-read-split-ranges-into-intersecting-and-non-intersecting-fault-injection 2024-03-21 21:25:18 +03:00
Dmitry Novik
20a860cf19 Update ci_config.py 2024-03-21 16:18:49 +01:00
Mikhail f. Shiryaev
15a61a998a
Fix logic for run_cpp_check, it had a bug 2024-03-21 14:34:31 +01:00
Mikhail f. Shiryaev
77e947c44f
Avoid IsADirectoryError: Is a directory contrib/azure 2024-03-21 14:09:01 +01:00
Maksim Kita
4dc985aa2f Fixed style check 2024-03-21 12:41:15 +03:00
Maksim Kita
b8c53d7cf3 Fixed tests 2024-03-21 12:41:15 +03:00
Mikhail f. Shiryaev
ff9a6d0958
Merge pull request #61197 from ClickHouse/merge-commit-style-autofix
Merge commit style autofix
2024-03-21 09:59:53 +01:00
Max K
984c7e69a0 disable test_allowed_url_with_config tests with HDFS
#no_merge_commit
 #ci_set_arm
2024-03-20 17:24:55 +00:00
Mikhail f. Shiryaev
aa1a210d5d
Fix logging of autoscaling lambda, add test for effective_capacity 2024-03-20 17:07:32 +01:00
Dmitry Novik
1524d5a451 Pass broken tests list to FastTest 2024-03-20 14:33:14 +01:00
Dmitry Novik
f0e7b78f8b Inverse analyzer in tests 2024-03-20 13:35:30 +01:00
Mikhail f. Shiryaev
661161695f
Merge pull request #61592 from ClickHouse/improve-builds-downloader
Improve build_download_helper
2024-03-20 11:56:06 +01:00
Mikhail f. Shiryaev
4efddb5ed4
Add type checking to build_download_helper 2024-03-19 14:01:31 +01:00
Mikhail f. Shiryaev
6b43d2e197
Improve logging in build_download_helper 2024-03-19 13:27:26 +01:00
Nikita Taranov
726db7361f add missing method 2024-03-18 20:04:56 +00:00
Nikita Taranov
0b2c274132 write docker tag 2024-03-18 17:50:08 +00:00
Nikita Taranov
ac352b96f0 remove copy-paste 2024-03-18 17:50:01 +00:00
Nikita Taranov
4a58705f7b impl
trigger rebuild

fix

move

fix black

trigger rebuild

trigger rebuild

fix mypy

fix

fix
2024-03-18 17:49:16 +00:00
avogar
31d5049f80 Move logic of ignoring drop queries to server 2024-03-18 16:26:05 +00:00
Raúl Marín
10dd47cf5c Separate CH and sanitizer logs 2024-03-18 16:40:03 +01:00
Raúl Marín
83c660a545 Merge remote-tracking branch 'blessed/master' into fuzz_log 2024-03-18 16:12:50 +01:00
Mikhail f. Shiryaev
41432c56a8
Enable shellcheck conditionally 2024-03-18 15:04:44 +01:00
Mikhail f. Shiryaev
c3f386d06a
Fix the autofixer 2024-03-18 15:04:43 +01:00
Mikhail f. Shiryaev
49017fc163
Run the style-check on top of the merge-commit of PRs 2024-03-18 15:04:41 +01:00
Mikhail f. Shiryaev
0240ffa5a6
Fetch the changed files only once 2024-03-18 15:04:40 +01:00
avogar
eada7e8d29 Fix server version 2024-03-18 13:18:58 +00:00
avogar
aadfc37dda Merge branch 'master' of github.com:ClickHouse/ClickHouse into better-stress-upgrade-checks 2024-03-15 20:27:16 +00:00
avogar
fa9d9ea3f7 Ignore DROP queries in stress test with 1/2 probability, use TRUNCATE instead of ignoring DROP in upgrade check for Memory/JOIN tables 2024-03-15 20:23:39 +00:00
Alexey Milovidov
2de304e00c
Merge pull request #61377 from ClickHouse/fix-spot-termination
Terminate EC2 on spot event if runner isn't running
2024-03-15 11:02:04 +03:00
Mikhail f. Shiryaev
03b9bca8a4
Terminate EC2 on spot event if runner isn't running 2024-03-14 14:31:55 +01:00
Mikhail f. Shiryaev
d465835306
Reorder hidden and shown checks in comment, change url of Mergeable check 2024-03-14 14:19:07 +01:00
Raúl Marín
f63368d83d Separate stdout and stderr in fuzzer report 2024-03-13 19:12:43 +01:00
Mikhail f. Shiryaev
ad363fe254
Merge pull request #61231 from ClickHouse/style-quick-scale
Make every style-checker runner types scaling-out very quickly
2024-03-12 18:19:03 +01:00
Mikhail f. Shiryaev
50aab0b69f
Fix annoying typo 2024-03-12 14:06:17 +01:00
Mikhail f. Shiryaev
612ff3c5bc
Make every style-checker runner types scaling-out very quickly 2024-03-12 13:00:31 +01:00
Max Kainov
fb1e5923a3 CI: fix runner for arm perf tests
#do_not_test
2024-03-12 09:12:25 +00:00
Max Kainov
b7b83085d1 CI: ci.py hot style fix
#do_not_test
2024-03-11 18:49:45 +00:00
Max K
7676d71c5d
Merge pull request #59738 from ClickHouse/ci_arm_integration_fix_1
CI: support batch specification in commit message
2024-03-11 19:01:03 +01:00
Max Kainov
f973e405ee CI: fixing ARM integration tests
#do_not_test
2024-03-11 17:20:36 +00:00
Max Kainov
f656a01538 CI: fix sync build issue with reuse
#do_not_test
2024-03-11 17:15:23 +00:00
Max K
a0a96b5580
Merge pull request #61099 from ClickHouse/ci_merge_queue_support
CI: support merge queue event in pr_info
2024-03-11 16:54:16 +01:00
Max K
23a9ac4947
Merge pull request #61160 from ClickHouse/ci_fix_nightly
CI: fix nightly job issue
2024-03-11 14:46:53 +01:00
Max Kainov
c4f29af8ec CI: fix nightly job issue
#do_not_test
2024-03-11 13:35:05 +00:00
Mikhail f. Shiryaev
8f2ff2ccd8
Enable black back 2024-03-11 13:22:18 +01:00
Max Kainov
49b78e0a0c CI: check cpp and py files style on changes only
#do_not_test
2024-03-11 09:30:26 +00:00
Alexey Milovidov
8c08fc1f57
Revert "CI: make style check faster" 2024-03-11 04:09:26 +03:00
Max Kainov
f429f54af8 CI: make style check fast
#do_not_test
2024-03-10 17:25:51 +00:00
Max Kainov
681bd28c87 CI: make style check fast
#do_not_test
2024-03-09 20:57:23 +00:00
Max Kainov
7f061ba89c CI: add wf class in ci_config
#do_not_test
2024-03-09 14:33:34 +00:00
Max Kainov
cacbd3ce34 CI: support merge queue event in pr_info
#do_not_test
2024-03-08 14:53:23 +00:00
Mikhail f. Shiryaev
e3f9fb2733
Trigger Mergeable check unconditionally in the final_check.py 2024-03-07 10:38:05 +01:00
Mikhail f. Shiryaev
5e597228d7
Merge pull request #60408 from ClickHouse/update-style-python
Update style python
2024-03-07 09:32:28 +01:00
Nikolay Monkov
fb17749b50 file has been reformatted to pass Style check 2024-03-07 09:45:24 +05:00
Max K
6c0e9e1944
Merge pull request #60958 from ClickHouse/ci_remove_rerun_check_for_build_report_jobs
CI: disable rerun check helper for build report jobs
2024-03-06 21:32:04 +01:00
Max Kainov
4e1257974c CI: disable rerun check helper for build report jobs
#do_not_test
2024-03-06 20:11:55 +00:00
Max Kainov
a0d161704a CI: remove sqllancer, sqllogic, sqltest from pr wf
#do_not_test
2024-03-06 17:58:21 +00:00
Max Kainov
4dd5e4fe99 CI: fix stage config (unit test release issue)
#no_ci_cache
2024-03-06 16:52:01 +00:00
Nikolay Monkov
d9b5f9a086 version has been added to docker labels 2024-03-06 21:37:56 +05:00
Max Kainov
ee468b0f2a CI: wait less in await to not exceed GH timeout
#do_not_test
2024-03-06 14:26:58 +00:00
Max K
561be5eb74
Merge pull request #60421 from ClickHouse/ci_abstract_workflow
CI: make workflow yml abstract
2024-03-06 15:15:02 +01:00
Mikhail f. Shiryaev
d2f92483f2
Add even more python code to pylint check, fix lambdas 2024-03-05 16:02:44 +01:00
Mikhail f. Shiryaev
34bb40583c
Install stubs for python-requests 2024-03-05 16:02:43 +01:00
Mikhail f. Shiryaev
91cff01ce0
More fixes in tests/ci 2024-03-05 16:02:41 +01:00
Mikhail f. Shiryaev
e31078e204
Fix some issues detected by new pylint 2024-03-05 16:02:36 +01:00
Mikhail f. Shiryaev
ec72800d78
Merge pull request #60383 from nickitat/upd_liburing
Enable testing with `io_uring` back
2024-03-05 15:26:35 +01:00
Max Kainov
82cc3f4339 CI: make workflow yml abstract
#no_merge_commit
 #no_ci_cache
 #job_style_check
 #job_package_debug
 #job_ClickHouse_build_check
 #job_Stateless_tests_debug
 #job_binary_release
2024-03-05 12:26:55 +00:00
Max Kainov
a6f5323c38 CI: Fix pending status for build report in backports
#do_not_test
2024-03-05 10:09:17 +00:00
Mikhail f. Shiryaev
a6caace5ec
Add a new runner type 2024-03-04 20:03:45 +01:00
Mikhail f. Shiryaev
bd2b0b4338
Export only str and bool for build configs 2024-03-04 20:01:26 +01:00
Mikhail f. Shiryaev
1443e490ea
Remove python bytecode, make consistent file-permissions
It's impossible to have persistent pyc files
Each time they are built with different content, for example:

> cmp -bl lambda-package*/charset_normalizer/__pycache__/constant.cpython-310.pyc
15582   6 ^F     4 ^D
15583 164 t    155 m
15584 141 a    142 b
15586 164 t    163 s
15587 151 i    332 M-Z
15588 163 s      6 ^F
15589 332 M-Z  164 t
15590   4 ^D   141 a
15591 155 m    143 c
15592 142 b    164 t
15593 143 c    151 i
17425  74 <     75 =
17428  76 >     46 &
17429 332 M-Z  372 M-z
17431 173 {     55 -
17434  75 =    174 |
17437  57 /     72 :
17440  54 ,     73 ;
17441 372 M-z  332 M-Z
17443 174 |    175 }
17446  55 -     54 ,
17447 372 M-z  332 M-Z
17449  46 &    173 {
17452  72 :     76 >
17455  42 "     74 <
17458  73 ;    133 [
17461 135 ]     42 "
17464 133 [    135 ]
17465 332 M-Z  372 M-z
17467 175 }     57 /
17503 332 M-Z  162 r
17504   5 ^E   130 X
17505 152 j      0 ^@
17506 157 o      0 ^@
17507 150 h      0 ^@
17508 141 a    332 M-Z
17509 142 b      5 ^E
17510 162 r    152 j
17511 130 X    157 o
17512   0 ^@   150 h
17513   0 ^@   141 a
17514   0 ^@   142 b
17536   5 ^E     2 ^B
17537 143 c    150 h
17538 160 p    172 z
17539  71 9    332 M-Z
17540  65 5      5 ^E
17541  60 0    143 c
17542 332 M-Z  160 p
17543   2 ^B    71 9
17544 150 h     65 5
17545 172 z     60 0
2024-03-04 13:37:13 +01:00
Mikhail f. Shiryaev
63e40203d1
Use python zipfile to have x-platform idempotent lambda packages 2024-03-04 13:37:11 +01:00
Max K
cecd2a6bac
Merge pull request #60613 from ClickHouse/ci_build_report_job_to_ci_config
CI: build_report job no dependcies on yml
2024-03-03 20:33:08 +01:00
Alexey Milovidov
c427575f72
Merge pull request #60694 from azat/ci/fix-flaky-check
Fix flaky checks
2024-03-03 19:53:12 +03:00
Max Kainov
8785b0700c CI: build_report job remove dependsies on yml
#no_merge_comit
 #job_ClickHouse_build_check
 #job_ClickHouse_special_build_check
 #job_style_check
2024-03-03 16:08:35 +00:00
Max K
2c713c2b90
Merge pull request #60602 from ClickHouse/ci_remove_docker_server_job_input_params
CI: remove inputs from docker_server job command
2024-03-03 14:25:23 +01:00
Azat Khuzhin
982f7e97c3
tests/ci/functional_test_check: move validate_bugfix_check into assert
Co-authored-by: Max K. <max.kainov@clickhouse.com>
2024-03-03 13:55:22 +01:00
Azat Khuzhin
418a6258ac Fix flaky checks
CI [1]:

    2024-03-02T22:23:37.4878824Z Going to start run command [python3 /home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/tests/ci/functional_test_check.py "$CHECK_NAME" $KILL_TIMEOUT]
    2024-03-02T22:23:37.7538256Z Runnin check [Stateless tests flaky check (asan)] with timeout [3600]
    2024-03-02T22:23:37.8790465Z Fetched info about 3 changed files
    2024-03-02T22:23:37.9160414Z Traceback (most recent call last):
    2024-03-02T22:23:37.9161715Z   File "/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/tests/ci/functional_test_check.py", line 341, in <module>
    2024-03-02T22:23:37.9162714Z     main()
    2024-03-02T22:23:37.9163683Z   File "/home/ubuntu/actions-runner/_work/ClickHouse/ClickHouse/tests/ci/functional_test_check.py", line 235, in main
    2024-03-02T22:23:37.9164642Z     assert (
    2024-03-02T22:23:37.9165273Z AssertionError: JobReport file path must be provided with --validate-bugfix
    2024-03-02T22:23:37.9612679Z Run action failed for: [Stateless tests flaky check (asan)] with exit code [1]

  [1]: https://github.com/ClickHouse/ClickHouse/actions/runs/8125551797/job/22208766466?pr=60689

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2024-03-03 10:44:21 +01:00