Commit Graph

62 Commits

Author SHA1 Message Date
Max Kainov
3247ae81e4 move out ci specific functions to ci.py
#no_merge_commit
2024-01-21 16:36:37 +00:00
Max K
84e5870b71
Reapply "improve CI with digest for docker, build and test jobs" (#57904)
* Revert "Revert "improve CI with digest for docker, build and test jobs""

* fix: docker manifest merge for missing images only
2023-12-18 09:07:22 +01:00
Max K
8c7add0334 Revert "improve CI with digest for docker, build and test jobs (#56317)"
This reverts commit 7844fcc196.
2023-12-15 15:48:01 +01:00
Max K
7844fcc196
improve CI with digest for docker, build and test jobs (#56317)
* CI with digests to tag docker images, reuse builds, reuse tests and others
2023-12-15 15:21:20 +01:00
Mikhail f. Shiryaev
c5b1aa4aa5
Clean even more os.path 2023-09-28 01:24:31 +02:00
Mikhail f. Shiryaev
e18d9d39e8
Migrate S3Helper to pathlib.Path 2023-09-06 12:43:20 +02:00
Mikhail f. Shiryaev
f0c18d4bd7
Rework setup of CI logs export 2023-08-30 22:04:37 +02:00
Mikhail f. Shiryaev
fd3e6c93ee
Produce standard report for cases when AST failed 2023-08-17 09:39:02 +02:00
Alexey Milovidov
428a05a560
Follow-up: Do not send logs to CI if the credentials are not set (#53456)
* Follow-up

* Automatic style fix

* Update tests/ci/ast_fuzzer_check.py

* Update tests/ci/functional_test_check.py

* Update tests/ci/stress_check.py

* Automatic style fix

---------

Co-authored-by: robot-clickhouse <robot-clickhouse@users.noreply.github.com>
Co-authored-by: Alexander Tokmakov <tavplubix@clickhouse.com>
2023-08-16 14:04:14 +03:00
robot-clickhouse
a92fe25ff9 Automatic style fix 2023-08-15 07:15:58 +00:00
Alexey Milovidov
df02512ebf Do not send logs to CI if the credentials are not set 2023-08-15 08:53:08 +02:00
Alexey Milovidov
748a65b2bb Fix error 2023-08-13 06:35:32 +02:00
Alexey Milovidov
25fefbe144 Fix error 2023-08-13 06:32:48 +02:00
Alexey Milovidov
077cd535e2 Improvements 2023-08-13 02:38:48 +02:00
Alexey Milovidov
b84132f4cd Merge branch 'master' into export-logs-in-ci-fuzz 2023-08-13 02:17:30 +02:00
Alexey Milovidov
5df03dd12f Better 2023-08-13 00:19:58 +02:00
robot-clickhouse
7d7ed06010 Automatic style fix 2023-08-12 20:53:33 +00:00
Alexey Milovidov
afbb91600e Export logs from CI in fuzzer 2023-08-12 22:41:56 +02:00
Alexey Milovidov
8e3bd222f0 Add SQLTest 2023-07-19 03:42:32 +02:00
Mikhail f. Shiryaev
df33b99907
Rework post_commit_status, add set_status_comment there 2023-04-28 23:52:31 +02:00
Mikhail f. Shiryaev
4405459925
Refactor RerunHelper to reduce API calls 2023-04-28 23:52:30 +02:00
Mikhail f. Shiryaev
eb11fb02fa
Clean up ast_fuzzer_check.py 2023-04-28 23:52:29 +02:00
Mikhail f. Shiryaev
0ab9d6787e
Add format_description for commit status descriptions 2023-02-23 15:21:19 +01:00
Alexey Milovidov
dff5e3eafd
Merge branch 'master' into add-dmesg-log-to-fuzzer 2023-01-13 05:22:18 +03:00
Ilya Yatsishin
0be3dbad3c
Merge pull request #44883 from ClickHouse/ci-report-type
Rework CI reports to have a class and clarify the logic
2023-01-12 18:20:40 +01:00
Alexey Milovidov
d4c0746647
Merge branch 'master' into add-dmesg-log-to-fuzzer 2023-01-12 07:17:03 +03:00
Mikhail f. Shiryaev
db96f9e3db
Rework test_results list/tuple/whatever into class 2023-01-09 12:09:47 +01:00
Alexey Milovidov
4694dad87c Break CI 2023-01-08 07:59:31 +01:00
Alexey Milovidov
3211f63911 Add dmesg.log to Fuzzer 2023-01-07 01:18:35 +01:00
Alexey Milovidov
79ae191611 A few improvements to AST Fuzzer 2022-12-29 18:55:28 +01:00
Alexey Milovidov
6ba360be71 Rename runlog.log to run.log 2022-12-26 16:29:32 +01:00
Alexander Tokmakov
9edb4d779b fix 2022-11-09 12:18:29 +01:00
Yatsishin Ilya
548ff1d2aa fix 2022-10-20 12:33:56 +00:00
Yatsishin Ilya
757959d087 fix bug 2022-10-18 15:14:42 +00:00
Alexander Tokmakov
e77b9e4d0c
Merge pull request #40775 from azat/ci/core-dumps-rework
Rework core collecting on CI (eliminate gcore usage)
2022-09-09 20:20:10 +03:00
Mikhail f. Shiryaev
257c4328f7
Rename get_build_urls to read_build_urls 2022-09-07 19:23:41 +02:00
Azat Khuzhin
25e3bebd9d Rework core collecting on CI (eliminate gcore usage)
gcore is a gdb command, that internally uses gdb to dump the core.

However with proper configuration of limits (core_dump.size_limit) it
should not be required, althought some issues is possible:
- non standard kernel.core_pattern
- sanitizers

So yes, gcore is more "universal" (you don't need to configure any
`kernel_pattern`), but it is ad-hoc, and it has drawbacks -
**it does not work when gdb fails**. For example gdb may fail with
`Dwarf Error: DW_FORM_strx1 found in non-DWO CU` in case of DWARF-5 [1].

  [1]: https://github.com/ClickHouse/ClickHouse/pull/40772#issuecomment-1236331323.

Let's try to switch to more native way.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-09-04 22:07:16 +02:00
Mikhail f. Shiryaev
372481e770
Rework S3Helper a little bit 2022-08-11 15:01:32 +02:00
Mikhail f. Shiryaev
2fb68078e8
Replace S3 URLs by parameter 2022-08-10 15:22:04 +02:00
Yakov Olkhovskiy
1a52fa183d rename Simple Check to Mergeable Check, refactor processing 2022-08-05 10:39:58 -04:00
Mikhail f. Shiryaev
8a74b1cf2f
Add forgotten insert_events_into to ast-fuzzer CI 2022-05-19 17:56:35 +02:00
Mikhail f. Shiryaev
3516eb0524
Make GITHUB_RUN_URL variable and use it 2022-03-24 15:37:53 +01:00
Mikhail f. Shiryaev
e6f5a3f98b
Apply black formatter to all *.py files in the repo 2022-03-22 17:39:58 +01:00
Azat Khuzhin
4965586e7e
More gdb introspection on CI (#34517)
* test/fuzzer: preserve core

This may help to capture things like in [1]:

    Failed assertion: "bin->low_bits_full > bin->low_bits_empty" Received signal 6 Received signal Aborted (6)

  [1]: https://s3.amazonaws.com/clickhouse-test-reports/33057/19216f4c0ae0f72108c147f958a708b521ad27dc/fuzzer_astfuzzerdebug,actions//report.html

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>

* test: do not run 'info locals' since 'backtrace full' includes it

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>

* test: try capture backtrace from all threads

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>

* test/stress: fix path for core artifacts

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
Fixes: #33389

* test/fuzzer: store core file in artifacts

v2: fix report, because of undefined variable CORE_LINK
v3: fix case when there is no core file

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-02-13 15:02:15 +03:00
alesapin
73a593b676
Merge branch 'master' into refactor_ci_tests1 2021-12-15 16:11:28 +03:00
tavplubix
85e53b1b1f
Try fix attaching gdb in tests (#32448)
* attach gdb with sudo

* fix

* Update run.sh
2021-12-10 00:12:45 +03:00
Constantine Peresypkin
7c90d4f8e3 refactor CI tests 2021-12-03 12:22:50 +02:00
alesapin
71e6c7e92e Add ability to rerun checks 2021-12-01 17:23:51 +03:00
Mikhail f. Shiryaev
d7cf4f78c3
Create get_event function to read event data 2021-11-29 16:07:18 +01:00
alesapin
081bfdfe82 Get rid of build numbers and simplify builds paths in S3 2021-11-26 13:57:36 +03:00