Commit Graph

134 Commits

Author SHA1 Message Date
Mikhail f. Shiryaev
28463cdcd3
Do not print --setup-plan to integration tests logs 2023-11-02 08:50:03 +01:00
Azat Khuzhin
e3976a5e50 Do not allow tests with state ERROR be overwritten by PASSED
In the ERROR case there can be sanitizers issues, that should not be
hidden, like right now it is doing for
`test_s3_table_functions/test.py::test_s3_table_functions_timeouts`
test [1], but I was lucky enough to trigger this in [2].

  [1]: https://s3.amazonaws.com/clickhouse-test-reports/55247/e99b0f46961733fa8ba10e490279dbcb0cdd67ad/integration_tests__asan__[5_6].html
  [2]: https://s3.amazonaws.com/clickhouse-test-reports/55245/918d65d6707c69ab541cdb56a076cdb83845d3ed/integration_tests__asan__[5_6].html

Though there could be also other issues, like with dependent services,
but still, let's try.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-10-14 09:34:00 +02:00
Mikhail f. Shiryaev
b610faf685
Revert "Revert "Fixed wrong python test name pattern"" 2023-09-09 23:21:14 +02:00
Alexander Tokmakov
e610a1867c
Revert "Fixed wrong python test name pattern" 2023-08-29 15:19:13 +02:00
Alexey Gerasimchuck
1069244d17 Fix in ci-runner.py 2023-08-28 22:05:36 +00:00
Kseniia Sumarokova
40ce9cf9ed
Merge pull request #51843 from ClickHouse/kssenii-nginx
Replace kssenii/nginx-test by clickhouse/nginx-dav
2023-08-01 11:36:20 +02:00
Alexey Milovidov
5a3f872a4b
Merge branch 'master' into integration-tests-fix-disaster-2 2023-07-22 19:37:59 +03:00
Mikhail f. Shiryaev
c6e9f621f1 Replace kssenii/nginx-test:1.1 by clickhouse/nginx-dav 2023-07-22 15:45:47 +02:00
Azat Khuzhin
baf6d182d6 Fix reporting some BROKEN tests as ERROR
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-07-18 12:22:38 +02:00
Azat Khuzhin
6c986c1fb2 Allow test names with spaces in it
It had been fixed and now it is allowed.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-07-17 17:29:44 +02:00
Azat Khuzhin
e532d5b376 tests: use -- over quotes on and on in ci-runner.py
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-07-17 17:17:21 +02:00
Azat Khuzhin
6f6562c96b Fix running integration tests with spaces in it's names
Previously they were simply ignored, and not only them, but all the
tests in that group, here is an example [1]

  [1]: https://s3.amazonaws.com/clickhouse-test-reports/51448/4ed462ac7834a8180f92ca7d7d3c076e687bfca9/integration_tests__asan__[4_6].html

Note, that I've replaced PYTEST_OPTS with handled by pytest itself
PYTEST_ADDOPTS, since "sh -c $PYTEST_OPTS" does not allows you to
preserve the arguments correctly.

v5: use double quotes for PYTEST_ADDOPTS value
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-07-17 17:17:21 +02:00
Antonio Andelic
152f5527a9 Remove wrong assert 2023-07-17 11:58:22 +00:00
robot-clickhouse
44fc93ba63 Automatic style fix 2023-07-15 21:10:21 +00:00
Alexey Milovidov
20b77e946a There is no point in detecting flaky tests 2023-07-15 23:00:23 +02:00
robot-clickhouse
881a45ec83 Automatic style fix 2023-07-12 09:26:01 +00:00
Dmitry Novik
b4723ca8df Try to fix report 2023-07-12 09:16:13 +00:00
Alexey Milovidov
1f72b1bb09
Update ci-runner.py 2023-07-09 03:28:12 +03:00
Alexey Milovidov
8e4c8f118c Fix disaster in integration tests, part 2 2023-07-08 09:07:05 +02:00
robot-clickhouse
532eba18a0 Automatic style fix 2023-06-22 19:58:58 +00:00
Dmitry Novik
9231bd9f9d Process broken tests in report 2023-06-22 19:48:13 +00:00
Dmitry Novik
c3e56a5214 Pass ENV into inner docker run 2023-06-20 15:20:21 +02:00
Azat Khuzhin
0859bbb08b Compress tar archives with zstd in intergration tests
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-02-18 21:54:38 +01:00
Mikhail f. Shiryaev
3edfb62066
Rollback unnecessary sync because of checking exit code 2023-02-17 12:28:04 +01:00
vdimir
19b5d21e8b
Minor fix in files locating for Bugfix validate check 2023-02-14 12:31:10 +01:00
Yatsishin Ilya
1b03a60e43 Do not report status if we failed to pull images. 2023-01-05 13:53:45 +00:00
Yatsishin Ilya
ccb6226d22 install CH before test 2023-01-04 20:07:52 +00:00
Yatsishin Ilya
03582f4b87 move logic to runner 2023-01-04 15:52:34 +00:00
Yatsishin Ilya
9cf599e58b Merge remote-tracking branch 'origin/master' into green-ci-2 2023-01-04 15:49:22 +00:00
Yatsishin Ilya
4d5131ed7d debug quotes 2023-01-04 13:16:31 +00:00
Yatsishin Ilya
08fbf6e5b2 again 2023-01-04 11:48:28 +00:00
Yatsishin Ilya
bb288c1f14 how many more? 2023-01-03 21:38:59 +00:00
Yatsishin Ilya
683df2dea5 fix 2023-01-03 18:56:24 +00:00
Alexey Milovidov
a06d03e097
Merge pull request #44851 from qoega/green-ci-3
No more retries in integration tests
2023-01-03 19:40:34 +03:00
Yatsishin Ilya
3deada0c29 fix 2023-01-03 16:05:20 +00:00
Ilya Yatsishin
8a9bb9b107 No more retries in integration tests 2023-01-02 18:56:24 +01:00
Yatsishin Ilya
3251d9278c black 2023-01-02 17:44:04 +00:00
Yatsishin Ilya
9ab5cf7d62 Download inside runner 2023-01-02 16:46:24 +00:00
Yatsishin Ilya
b40f6c0b10 Pre-pull docker image and fail check completely if it failed 2023-01-02 15:04:14 +00:00
Azat Khuzhin
c3b3ae886f tests/integration: ignore exit code for a running group
tee is useful there, otherwise logs are empty on CI

This reverts commit 92fd7def39.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-01-02 12:08:44 +01:00
Azat Khuzhin
f2c069b852 tests/integration: allow to split parametrized tests between parallel groups
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-01-02 12:06:56 +01:00
Azat Khuzhin
4b71b481c7 tests/integration: do not split paramaterized tests into multiple groups
There is also one assumption the runner already, that leads to
misleading reports like in [1].

  [1]: https://s3.amazonaws.com/clickhouse-test-reports/44762/89c071e291980e355f2c2be42dc15047caabc9bc/integration_tests__asan__%5B4/6%5D.html

Here the test test_multiple_disks/test.py::test_jbod_overflow had been
splitted into multiple groups, however because of that one assumption
it had been runned as a whole in both groups and in one it failed but
succeeded in another.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-01-01 23:35:21 +01:00
Azat Khuzhin
700943a2b0 tests/integration: remove logging message for undone thing
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-01-01 23:13:00 +01:00
Azat Khuzhin
fa78569c39 tests/integration: fix logging message
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-01-01 23:12:25 +01:00
Azat Khuzhin
92fd7def39 tests/integration: fix detecting exit code for a running group
Because of tee the exit code is the result of tee, not of the pytest.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-01-01 23:09:12 +01:00
Azat Khuzhin
03f80bae90 Move dmesg dumping out from runner to ci-runner.py
runner is used by developers to run tests, while ci-runner.py is used
only by CI scripts, and to avoid requiring CAP_SYSLOG for manual dmesg
clear/dump had been moved.

Also for manual runs this can be done manually.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-12-28 15:47:46 +01:00
Mikhail f. Shiryaev
6d0cd35cd0
Ignore exit code 1 for tar in integration tests 2022-12-23 13:08:59 +01:00
Mikhail f. Shiryaev
550c776ffe
Wait for all files are in sync before archiving them 2022-11-02 13:16:27 +01:00
robot-clickhouse
838bc678cc Automatic style fix 2022-09-17 18:38:12 +00:00
Alexey Milovidov
ecfd8cb5a6 Remove trash error message from integration tests 2022-09-17 20:24:56 +02:00