Commit Graph

725 Commits

Author SHA1 Message Date
Nikita Fomichev
8295a8e9b8 Stateless tests: fix flaky tests 4 2024-07-13 00:39:53 +02:00
Nikita Fomichev
cc2cce9717 Stateless tests: fix flaky tests 3 2024-07-13 00:34:54 +02:00
Alexander Tokmakov
ea64527ea1 suppress broken test 2024-07-12 19:18:16 +02:00
Alexander Tokmakov
edb9406f36 Merge branch 'master' into signal_handlers_in_client 2024-07-12 14:30:42 +02:00
Nikita Fomichev
5f8358942c Stateless tests: push CI 2024-07-12 03:06:07 +02:00
Nikita Fomichev
08b6dd604a Stateless tests: deal with hang-ups more roughly 2024-07-12 03:04:43 +02:00
Nikita Fomichev
6f3ae4a6de Fix path in test 2024-07-10 23:57:23 +02:00
Nikita Fomichev
e6b3368dc1 debug tests hung on gdb error 9 2024-07-10 17:09:50 +02:00
Nikita Fomichev
83cfe164bd debug tests hung on gdb error 7 2024-07-10 15:55:30 +02:00
Nikita Fomichev
ba176a9c4a debug tests hung on gdb error 6 2024-07-10 14:51:27 +02:00
Nikita Fomichev
f96d40d828 debug tests hung on gdb error 5 2024-07-10 13:35:19 +02:00
Nikita Fomichev
ede43721b4 debug tests hung on gdb error 4 2024-07-10 13:30:06 +02:00
Nikita Fomichev
ef08055db9 debug tests hung on gdb error 2 2024-07-10 12:47:32 +02:00
Nikita Fomichev
17d5ec28d4 debug tests hung on gdb error 2024-07-10 12:08:33 +02:00
Alexander Tokmakov
27738747e0 save fatal logs to file 2024-07-10 00:28:26 +02:00
Nikita Fomichev
eeb3561c37 Stateless tests: run sequential tests in parallel to other tests 2024-07-09 19:41:06 +02:00
Nikita Fomichev
5f643b9ed9
Merge pull request #65979 from ClickHouse/base64encode-tests
Add tests for `base64URLEncode` and `base64URLDecode`. Add analyzer tests
2024-07-07 18:43:02 +00:00
Nikita Fomichev
be8945f2a0 Merge remote-tracking branch 'origin/master' into fix-missing-tests-reports 2024-07-07 10:04:58 +02:00
Nikita Fomichev
db462a7920 Merge remote-tracking branch 'origin/master' into base64encode-tests 2024-07-07 10:04:13 +02:00
Nikita Fomichev
34697c0bfa Tests: increase timeout for flaky check 2024-07-04 15:55:03 +02:00
Nikita Fomichev
af845234a2 Review fixes 2024-07-03 17:53:05 +02:00
Nikita Fomichev
b88be7260f Tests: Eliminating the global tests queue to prevent clickhouse-test from hanging when a server dies 2024-07-03 13:43:30 +02:00
alesapin
452201caf9 Black 2024-07-02 14:21:39 +02:00
alesapin
a2626037bc Improve object storage tags in tests 2024-07-02 14:15:59 +02:00
Nikita Fomichev
54bc5d12ef Add randomization of new settings in clickhouse-test
prefer_external_sort_block_bytes
cross_join_min_rows_to_compress
cross_join_min_bytes_to_compress
max_parsing_threads
min_external_table_block_size_bytes
2024-06-05 22:57:10 +02:00
Nikita Mikhaylov
0294e677ff
Update tests/clickhouse-test
Co-authored-by: Konstantin Bogdanov <thevar1able@users.noreply.github.com>
2024-05-30 19:52:59 +02:00
Nikita Mikhaylov
2e913444d3 Shuffle tests for parallel execution 2024-05-30 17:45:27 +00:00
vdimir
6d5805d260
Do not run tests with 'no-s3-storage-with-slow-build' with ASan 2024-05-24 14:54:39 +00:00
Alex Katsman
5dbc1f347e Fix logs test for binary data by converting it to a valid UTF8 string. 2024-05-15 19:37:42 +00:00
Maxim Alexeev
9c2e1bfce6 fixed review 2024-05-13 19:39:45 +03:00
Ali
65934b5651
Update clickhouse-test
Fixed a typo
2024-05-11 23:16:42 +03:30
Alexander Tokmakov
a08a07723b
Update clickhouse-test 2024-05-08 16:26:03 +02:00
Raúl Marín
7e06b01e15
Merge pull request #63130 from Algunenano/coverage_test
Try to fix coverage tests
2024-05-02 09:34:57 +00:00
Raúl Marín
e79860aa0a Fix linter 2024-04-30 22:24:46 +02:00
Raúl Marín
9c3775b7c6 More fixes 2024-04-30 14:32:32 +02:00
Alexey Milovidov
a86f2aab2f
Merge pull request #63073 from azat/tests/fix-logs-stats
Fix "invalid escape sequence" in clickhouse-test
2024-04-30 01:27:32 +02:00
Raúl Marín
f7ad885725 Try to fix coverage tests 2024-04-29 21:38:45 +02:00
Azat Khuzhin
881e45b680 Fix "invalid escape sequence" in clickhouse-test
python error:

    /src/ch/clickhouse/.cmake/../tests/clickhouse-test:2570: SyntaxWarning: invalid escape sequence '\/'

And also remove unnecessary escaping of `/`.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2024-04-27 17:01:32 +02:00
Alexey Milovidov
f6d443c927
Merge pull request #62041 from azat/clickhouse-test-no-reference-fix
Fix clickhouse-test in case of missing .reference file
2024-04-27 14:55:54 +00:00
Azat Khuzhin
d40ad207ff Fix --client-option for $CLICKHOUSE_CLIENT in .sh tests
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2024-04-17 11:56:45 +03:00
Alexander Tokmakov
d8e97b51bf Merge branch 'master' into cannot_allocate_thread 2024-04-10 21:21:42 +02:00
Alexander Tokmakov
b6aff78bf5 fix 2024-04-08 16:13:46 +02:00
Alexander Tokmakov
5db9fbed52 cancel tasks on exception 2024-04-04 22:32:57 +02:00
robot-clickhouse
ea16070117 Automatic style fix 2024-04-03 19:17:46 +00:00
Alexander Tokmakov
c53b20a770 fix 2024-04-03 20:57:12 +02:00
avogar
d87d5a9197 Merge branch 'master' of github.com:ClickHouse/ClickHouse into better-stress-upgrade-checks 2024-04-03 12:07:22 +00:00
Raúl Marín
7627f48bbe
Merge pull request #62139 from arthurpassos/fix_ci_too_long_run_typo
Fix incorrect CI error message
2024-04-02 13:26:17 +02:00
Arthur Passos
2a5e9ea713 fix black 2024-04-01 13:07:52 -03:00
Arthur Passos
5e9436611a fix incorrect ci err message 2024-04-01 10:21:23 -03:00
Azat Khuzhin
61b7c942ac Fix clickhouse-test in case of missing .reference file
Before it throws internal python error:

    $ ../tests/clickhouse-test 03033
    Using queries from '/src/ch/clickhouse/tests/queries' directory
    Connecting to ClickHouse server... OK
    Connected to server 24.3.1.1 @ 3fa6d23730 master

    Running 1 stateless tests (MainProcess).

    03033_dist:                                                             [ UNKNOWN ] - Test internal error:
    TypeError
    expected str, bytes or os.PathLike object, not NoneType
      File "/src/ch/clickhouse/.cmake/../tests/clickhouse-test", line 1644, in run
        if not is_valid_utf_8(self.case_file) or not is_valid_utf_8(
                                                     ^^^^^^^^^^^^^^^

      File "/src/ch/clickhouse/.cmake/../tests/clickhouse-test", line 237, in is_valid_utf_8
        with open(fname, "rb") as f:
             ^^^^^^^^^^^^^^^^^

    0 tests passed. 0 tests skipped. 0.01 s elapsed (MainProcess).
    Won't run stateful tests because test data wasn't loaded.
    All tests have finished.

Now:

    $ ../tests/clickhouse-test 03033
    Using queries from '/src/ch/clickhouse/tests/queries' directory
    Connecting to ClickHouse server... OK
    Connected to server 24.3.1.1 @ 3fa6d23730 master

    Running 1 stateless tests (MainProcess).

    03033_dist:                                                             [ UNKNOWN ] - no reference file

    0 tests passed. 0 tests skipped. 0.11 s elapsed (MainProcess).
    Won't run stateful tests because test data wasn't loaded.
    All tests have finished.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2024-03-28 14:29:18 +01:00
Nikolay Degterinsky
72f3871bd3
Fix clickhouse-test 2024-03-28 01:58:17 +01:00
Kruglov Pavel
de1229ba60
Merge branch 'master' into better-stress-upgrade-checks 2024-03-25 19:18:35 +01:00
Alexey Milovidov
8b6e2d7226 Do not print garbage in clickhouse-test 2024-03-23 02:50:40 +01:00
Maksim Kita
4dc985aa2f Fixed style check 2024-03-21 12:41:15 +03:00
Maksim Kita
b8c53d7cf3 Fixed tests 2024-03-21 12:41:15 +03:00
vdimir
3f46a5d93d
Fix clickhouse-test client option and CLICKHOUSE_URL_PARAMS interference 2024-03-19 13:01:25 +00:00
avogar
31d5049f80 Move logic of ignoring drop queries to server 2024-03-18 16:26:05 +00:00
avogar
eada7e8d29 Fix server version 2024-03-18 13:18:58 +00:00
avogar
aadfc37dda Merge branch 'master' of github.com:ClickHouse/ClickHouse into better-stress-upgrade-checks 2024-03-15 20:27:16 +00:00
avogar
fa9d9ea3f7 Ignore DROP queries in stress test with 1/2 probability, use TRUNCATE instead of ignoring DROP in upgrade check for Memory/JOIN tables 2024-03-15 20:23:39 +00:00
Raúl Marín
cca96e05cf Bring clickhouse-test changes from private 2024-03-13 13:58:45 +01:00
Mikhail f. Shiryaev
e31078e204
Fix some issues detected by new pylint 2024-03-05 16:02:36 +01:00
Mikhail f. Shiryaev
ec72800d78
Merge pull request #60383 from nickitat/upd_liburing
Enable testing with `io_uring` back
2024-03-05 15:26:35 +01:00
Michael Kolupaev
42437a2ae1 Userspace page cache (#53770)
* Userspace page cache

* Maybe it'll build this time, who knows.

* 'auto' went out of fashion, I guess

* Documentation, tsan workaround, metric 'UnreclaimableRss', disable page cache in the test that uses DatabaseOrdinary

* Moved CachedInMemoryReadBufferFromFile to object store level, changed settings, addressed other comments.

* Fix

* Another fix

* Fix restricted seek, fix ppc64le build

* Don't allow page cache with file cache

* Adjust tests a little

* Fix clang-tidy

* Conflicts

* Comments

* Maybe unbroke AsynchronousBoundedReadBuffer

* SettingsChangesHistory.h

* Fix warning in test
2024-02-29 12:28:32 +00:00
Alexander Tokmakov
15e1191f74
Revert "Userspace page cache (#53770)"
This reverts commit 7b55c61551.
2024-02-29 13:20:17 +01:00
Michael Kolupaev
7b55c61551
Userspace page cache (#53770)
* Userspace page cache

* Maybe it'll build this time, who knows.

* 'auto' went out of fashion, I guess

* Documentation, tsan workaround, metric 'UnreclaimableRss', disable page cache in the test that uses DatabaseOrdinary

* Moved CachedInMemoryReadBufferFromFile to object store level, changed settings, addressed other comments.

* Fix

* Another fix

* Fix restricted seek, fix ppc64le build

* Don't allow page cache with file cache

* Adjust tests a little

* Fix clang-tidy

* Conflicts

* Comments

* Maybe unbroke AsynchronousBoundedReadBuffer

* SettingsChangesHistory.h

* Fix warning in test
2024-02-29 11:27:32 +00:00
Nikita Taranov
3c5d6c4df0 Revert "Merge pull request #60373 from azat/tests/detect-io_uring"
This reverts commit 648e6ec5a6, reversing
changes made to 273b4835a4.
2024-02-25 15:00:26 +01:00
Azat Khuzhin
554d993d8b clickhouse-test: fix style check (unused import platform)
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2024-02-24 19:37:13 +01:00
Azat Khuzhin
35e948a20a Detect io_uring in tests
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2024-02-24 19:36:58 +01:00
János Benjamin Antal
b094ab9763 Add comment 2024-02-12 15:27:20 +00:00
Alexey Milovidov
cfe60586c0 Reset coverage after each test 2024-01-20 21:45:11 +01:00
Alexey Milovidov
735708e9bb Merge branch 'master' into enable-coverage-for-debug-build 2024-01-20 21:40:53 +01:00
vdimir
97a38855c1
Merge pull request #57870 from ClickHouse/vdimir/test_show_whitespaces_in_diff
Add option `--show-whitespaces-in-diff` to clickhouse-test
2024-01-19 10:42:00 +01:00
Alexey Milovidov
6b8d53a9fa Remove obsolete comment 2024-01-16 11:07:30 +01:00
Alexey Milovidov
21082be9a6 Better test 2024-01-16 10:41:13 +01:00
Alexey Milovidov
5cda358e62 Obey Python's quirky formatter 2024-01-15 06:26:04 +01:00
Alexey Milovidov
e49cfbef08 Coverage for non-server tools 2024-01-15 05:40:03 +01:00
Alexey Milovidov
fe952fb64c Rename to system.coverage_log to simplify export 2024-01-15 00:38:42 +01:00
Alexey Milovidov
3e09feda33 More functions 2024-01-15 00:36:07 +01:00
Alexey Milovidov
f7abeff085 Slightly better reporting 2024-01-15 00:25:07 +01:00
Alexey Milovidov
9141e1693f Calculate cumulative coverage by default. 2024-01-15 00:24:17 +01:00
Alexey Milovidov
3bd2c7e384 Report coverage if available 2024-01-14 22:06:25 +01:00
Michael Kolupaev
edd120e8be Make it experimental 2023-12-28 17:56:05 +00:00
Michael Kolupaev
0fc7535eba Fixes 2023-12-28 17:56:05 +00:00
Alexander Tokmakov
38fe70c68a
Revert "Refreshable materialized views (takeover)" 2023-12-28 13:12:20 +01:00
Michael Kolupaev
18cc5d1b76 Make it experimental 2023-12-27 20:24:55 +00:00
Michael Kolupaev
802961f0a2 Fixes 2023-12-27 20:24:55 +00:00
robot-clickhouse
d3404db055 Automatic style fix 2023-12-27 14:40:20 +00:00
vdimir
0df796f617
clickhouse-test show-whitespaces-in-diff shows $ on in end of line with spaces 2023-12-27 14:31:07 +00:00
vdimir
4d7edb4321
Add option --show-whitespaces-in-diff to clickhouse-test 2023-12-27 14:04:40 +00:00
Azat Khuzhin
c5dbde8407 Replace timeout --foreground with one workaround
The problem with --foreground option is that it send the signal only to
the process that had been spawned by timeout(1), while it can create
lots of children, and when you killing parent you are closing pipes and
childrens will get EPIPE, like in [1].

  [1]: https://s3.amazonaws.com/clickhouse-test-reports/0/069f8bbb2f48541cc736903e1da5459fa2c27da0/stateless_tests__debug__%5B2_5%5D.html

Another problem is that now child process will finish correctly, which
may also print some errors like QUERY_WAS_CANCELLED (see [2]).

  [2]: https://s3.amazonaws.com/clickhouse-test-reports/0/ef66714bf20042ba9cb5d59b7839befe26110b93/stateless_tests__release__analyzer_.html

In general this is not required actually, since all timeout invocations
uses timeout value less then the default test limit (10min). But it may
leave some processes in case of overriding this limit, i.e.
`clickhouse-test --timeout 1`

So to workaround this at least somehow, let's send SIGTERM and only
after some timeout (here I use 0.1), SIGKILL. This will give at least
some ability to terminate all childrens that had been spawned by
timeout(1).

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-12-24 13:23:09 +01:00
Azat Khuzhin
1fdee81ad5 Fix leftover processes/hangs in tests
One of such cases is 02479_race_condition_between_insert_and_droppin_mv
[1], yes it can be fixed (by using fixed number of iterations, or with
some bash trickery), but it is better to fix them completelly,
eventually such tests will be submitted and pass review anyway.

By allocating process group for each test we can kill all the processes
in this process group, and this what this patch does.

This will also fix some test hangs (like in [1]) as well as some
possible issues in stress tests.

  [1]: https://s3.amazonaws.com/clickhouse-test-reports/0/e2c1230b00386c4d0096a245396ab3be7ce60950/stateless_tests__release__analyzer_/run.log

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
(cherry picked from commit 72fa58e192)
2023-12-24 09:43:58 +01:00
Alexey Milovidov
71c5ae548f
Revert "Fix leftover processes/hangs in tests" 2023-12-24 07:57:43 +03:00
Azat Khuzhin
72fa58e192 Fix leftover processes/hangs in tests
One of such cases is 02479_race_condition_between_insert_and_droppin_mv
[1], yes it can be fixed (by using fixed number of iterations, or with
some bash trickery), but it is better to fix them completelly,
eventually such tests will be submitted and pass review anyway.

By allocating process group for each test we can kill all the processes
in this process group, and this what this patch does.

This will also fix some test hangs (like in [1]) as well as some
possible issues in stress tests.

  [1]: https://s3.amazonaws.com/clickhouse-test-reports/0/e2c1230b00386c4d0096a245396ab3be7ce60950/stateless_tests__release__analyzer_/run.log

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-12-23 16:40:33 +01:00
Alexander Tokmakov
8f9dff8c88
Update clickhouse-test 2023-12-22 19:54:15 +01:00
Sema Checherinda
291567a5b2
Merge pull request #58007 from ClickHouse/chesema-stateless-run-timeout
more messages in ci
2023-12-20 13:50:12 +01:00
Raúl Marín
98c9f830d6 Revert "Merge pull request #55710 from guoxiaolongzte/clickhouse-test-add-prinln-nowTime"
This reverts commit efefad9e52, reversing
changes made to 8f7852f82d.
2023-12-20 10:16:47 +01:00
Anton Popov
4859a74b16
Correct values for randomization 2023-12-20 02:42:24 +01:00
Yakov Olkhovskiy
efefad9e52
Merge pull request #55710 from guoxiaolongzte/clickhouse-test-add-prinln-nowTime
clickhouse-test print  Case SN, Current Time and Case Name in one test case
2023-12-19 18:58:19 -05:00
Sema Checherinda
b4fec61814 fix style and black 2023-12-19 13:43:35 +01:00