Commit Graph

127 Commits

Author SHA1 Message Date
Alexey Milovidov
5a3f872a4b
Merge branch 'master' into integration-tests-fix-disaster-2 2023-07-22 19:37:59 +03:00
Azat Khuzhin
baf6d182d6 Fix reporting some BROKEN tests as ERROR
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-07-18 12:22:38 +02:00
Azat Khuzhin
6c986c1fb2 Allow test names with spaces in it
It had been fixed and now it is allowed.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-07-17 17:29:44 +02:00
Azat Khuzhin
e532d5b376 tests: use -- over quotes on and on in ci-runner.py
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-07-17 17:17:21 +02:00
Azat Khuzhin
6f6562c96b Fix running integration tests with spaces in it's names
Previously they were simply ignored, and not only them, but all the
tests in that group, here is an example [1]

  [1]: https://s3.amazonaws.com/clickhouse-test-reports/51448/4ed462ac7834a8180f92ca7d7d3c076e687bfca9/integration_tests__asan__[4_6].html

Note, that I've replaced PYTEST_OPTS with handled by pytest itself
PYTEST_ADDOPTS, since "sh -c $PYTEST_OPTS" does not allows you to
preserve the arguments correctly.

v5: use double quotes for PYTEST_ADDOPTS value
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-07-17 17:17:21 +02:00
Antonio Andelic
152f5527a9 Remove wrong assert 2023-07-17 11:58:22 +00:00
robot-clickhouse
44fc93ba63 Automatic style fix 2023-07-15 21:10:21 +00:00
Alexey Milovidov
20b77e946a There is no point in detecting flaky tests 2023-07-15 23:00:23 +02:00
robot-clickhouse
881a45ec83 Automatic style fix 2023-07-12 09:26:01 +00:00
Dmitry Novik
b4723ca8df Try to fix report 2023-07-12 09:16:13 +00:00
Alexey Milovidov
1f72b1bb09
Update ci-runner.py 2023-07-09 03:28:12 +03:00
Alexey Milovidov
8e4c8f118c Fix disaster in integration tests, part 2 2023-07-08 09:07:05 +02:00
robot-clickhouse
532eba18a0 Automatic style fix 2023-06-22 19:58:58 +00:00
Dmitry Novik
9231bd9f9d Process broken tests in report 2023-06-22 19:48:13 +00:00
Dmitry Novik
c3e56a5214 Pass ENV into inner docker run 2023-06-20 15:20:21 +02:00
Azat Khuzhin
0859bbb08b Compress tar archives with zstd in intergration tests
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-02-18 21:54:38 +01:00
Mikhail f. Shiryaev
3edfb62066
Rollback unnecessary sync because of checking exit code 2023-02-17 12:28:04 +01:00
vdimir
19b5d21e8b
Minor fix in files locating for Bugfix validate check 2023-02-14 12:31:10 +01:00
Yatsishin Ilya
1b03a60e43 Do not report status if we failed to pull images. 2023-01-05 13:53:45 +00:00
Yatsishin Ilya
ccb6226d22 install CH before test 2023-01-04 20:07:52 +00:00
Yatsishin Ilya
03582f4b87 move logic to runner 2023-01-04 15:52:34 +00:00
Yatsishin Ilya
9cf599e58b Merge remote-tracking branch 'origin/master' into green-ci-2 2023-01-04 15:49:22 +00:00
Yatsishin Ilya
4d5131ed7d debug quotes 2023-01-04 13:16:31 +00:00
Yatsishin Ilya
08fbf6e5b2 again 2023-01-04 11:48:28 +00:00
Yatsishin Ilya
bb288c1f14 how many more? 2023-01-03 21:38:59 +00:00
Yatsishin Ilya
683df2dea5 fix 2023-01-03 18:56:24 +00:00
Alexey Milovidov
a06d03e097
Merge pull request #44851 from qoega/green-ci-3
No more retries in integration tests
2023-01-03 19:40:34 +03:00
Yatsishin Ilya
3deada0c29 fix 2023-01-03 16:05:20 +00:00
Ilya Yatsishin
8a9bb9b107 No more retries in integration tests 2023-01-02 18:56:24 +01:00
Yatsishin Ilya
3251d9278c black 2023-01-02 17:44:04 +00:00
Yatsishin Ilya
9ab5cf7d62 Download inside runner 2023-01-02 16:46:24 +00:00
Yatsishin Ilya
b40f6c0b10 Pre-pull docker image and fail check completely if it failed 2023-01-02 15:04:14 +00:00
Azat Khuzhin
c3b3ae886f tests/integration: ignore exit code for a running group
tee is useful there, otherwise logs are empty on CI

This reverts commit 92fd7def39.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-01-02 12:08:44 +01:00
Azat Khuzhin
f2c069b852 tests/integration: allow to split parametrized tests between parallel groups
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-01-02 12:06:56 +01:00
Azat Khuzhin
4b71b481c7 tests/integration: do not split paramaterized tests into multiple groups
There is also one assumption the runner already, that leads to
misleading reports like in [1].

  [1]: https://s3.amazonaws.com/clickhouse-test-reports/44762/89c071e291980e355f2c2be42dc15047caabc9bc/integration_tests__asan__%5B4/6%5D.html

Here the test test_multiple_disks/test.py::test_jbod_overflow had been
splitted into multiple groups, however because of that one assumption
it had been runned as a whole in both groups and in one it failed but
succeeded in another.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-01-01 23:35:21 +01:00
Azat Khuzhin
700943a2b0 tests/integration: remove logging message for undone thing
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-01-01 23:13:00 +01:00
Azat Khuzhin
fa78569c39 tests/integration: fix logging message
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-01-01 23:12:25 +01:00
Azat Khuzhin
92fd7def39 tests/integration: fix detecting exit code for a running group
Because of tee the exit code is the result of tee, not of the pytest.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-01-01 23:09:12 +01:00
Azat Khuzhin
03f80bae90 Move dmesg dumping out from runner to ci-runner.py
runner is used by developers to run tests, while ci-runner.py is used
only by CI scripts, and to avoid requiring CAP_SYSLOG for manual dmesg
clear/dump had been moved.

Also for manual runs this can be done manually.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-12-28 15:47:46 +01:00
Mikhail f. Shiryaev
6d0cd35cd0
Ignore exit code 1 for tar in integration tests 2022-12-23 13:08:59 +01:00
Mikhail f. Shiryaev
550c776ffe
Wait for all files are in sync before archiving them 2022-11-02 13:16:27 +01:00
robot-clickhouse
838bc678cc Automatic style fix 2022-09-17 18:38:12 +00:00
Alexey Milovidov
ecfd8cb5a6 Remove trash error message from integration tests 2022-09-17 20:24:56 +02:00
Mikhail f. Shiryaev
dfae6a5634
Upload logs for getting all tests command 2022-06-23 18:14:42 +02:00
Alexander Tokmakov
153862dc31 fix tests
(cherry picked from commit ed52f463d95ac49431600f167a08507d7fcaaaf0)
2022-06-01 13:47:52 +02:00
Azat Khuzhin
b0c72b6104 tests/integration: add prefix match for skipped tests
This way you can specify only file/module, or test name without
parameters.

Since, at least one, test that we care about
(test_distributed_respect_user_timeouts/test.py::test_reconnect) was not
runned sequentially [1].

  [1]: https://s3.amazonaws.com/clickhouse-test-reports/37138/d7f5db4143c559bb8044058a653956945897e724/integration_tests__asan__actions__[2/3].html

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-05-13 11:19:11 +03:00
Azat Khuzhin
0fb9553e1c tests/integration: use no-resolve and verbose for iptables --list
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-05-13 10:57:24 +03:00
Mikhail f. Shiryaev
76c3022029
Fix code broken in 1e82b04de3 2022-05-10 17:41:47 +02:00
alesapin
c74b6c547b Black 2022-04-14 14:40:56 +02:00
alesapin
428753a4a7 Fix integration tests parser 2022-04-14 14:39:36 +02:00