Commit Graph

705 Commits

Author SHA1 Message Date
Alexey Milovidov
cfe60586c0 Reset coverage after each test 2024-01-20 21:45:11 +01:00
Alexey Milovidov
735708e9bb Merge branch 'master' into enable-coverage-for-debug-build 2024-01-20 21:40:53 +01:00
vdimir
97a38855c1
Merge pull request #57870 from ClickHouse/vdimir/test_show_whitespaces_in_diff
Add option `--show-whitespaces-in-diff` to clickhouse-test
2024-01-19 10:42:00 +01:00
Alexey Milovidov
6b8d53a9fa Remove obsolete comment 2024-01-16 11:07:30 +01:00
Alexey Milovidov
21082be9a6 Better test 2024-01-16 10:41:13 +01:00
Alexey Milovidov
5cda358e62 Obey Python's quirky formatter 2024-01-15 06:26:04 +01:00
Alexey Milovidov
e49cfbef08 Coverage for non-server tools 2024-01-15 05:40:03 +01:00
Alexey Milovidov
fe952fb64c Rename to system.coverage_log to simplify export 2024-01-15 00:38:42 +01:00
Alexey Milovidov
3e09feda33 More functions 2024-01-15 00:36:07 +01:00
Alexey Milovidov
f7abeff085 Slightly better reporting 2024-01-15 00:25:07 +01:00
Alexey Milovidov
9141e1693f Calculate cumulative coverage by default. 2024-01-15 00:24:17 +01:00
Alexey Milovidov
3bd2c7e384 Report coverage if available 2024-01-14 22:06:25 +01:00
Michael Kolupaev
edd120e8be Make it experimental 2023-12-28 17:56:05 +00:00
Michael Kolupaev
0fc7535eba Fixes 2023-12-28 17:56:05 +00:00
Alexander Tokmakov
38fe70c68a
Revert "Refreshable materialized views (takeover)" 2023-12-28 13:12:20 +01:00
Michael Kolupaev
18cc5d1b76 Make it experimental 2023-12-27 20:24:55 +00:00
Michael Kolupaev
802961f0a2 Fixes 2023-12-27 20:24:55 +00:00
robot-clickhouse
d3404db055 Automatic style fix 2023-12-27 14:40:20 +00:00
vdimir
0df796f617
clickhouse-test show-whitespaces-in-diff shows $ on in end of line with spaces 2023-12-27 14:31:07 +00:00
vdimir
4d7edb4321
Add option --show-whitespaces-in-diff to clickhouse-test 2023-12-27 14:04:40 +00:00
Azat Khuzhin
c5dbde8407 Replace timeout --foreground with one workaround
The problem with --foreground option is that it send the signal only to
the process that had been spawned by timeout(1), while it can create
lots of children, and when you killing parent you are closing pipes and
childrens will get EPIPE, like in [1].

  [1]: https://s3.amazonaws.com/clickhouse-test-reports/0/069f8bbb2f48541cc736903e1da5459fa2c27da0/stateless_tests__debug__%5B2_5%5D.html

Another problem is that now child process will finish correctly, which
may also print some errors like QUERY_WAS_CANCELLED (see [2]).

  [2]: https://s3.amazonaws.com/clickhouse-test-reports/0/ef66714bf20042ba9cb5d59b7839befe26110b93/stateless_tests__release__analyzer_.html

In general this is not required actually, since all timeout invocations
uses timeout value less then the default test limit (10min). But it may
leave some processes in case of overriding this limit, i.e.
`clickhouse-test --timeout 1`

So to workaround this at least somehow, let's send SIGTERM and only
after some timeout (here I use 0.1), SIGKILL. This will give at least
some ability to terminate all childrens that had been spawned by
timeout(1).

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-12-24 13:23:09 +01:00
Azat Khuzhin
1fdee81ad5 Fix leftover processes/hangs in tests
One of such cases is 02479_race_condition_between_insert_and_droppin_mv
[1], yes it can be fixed (by using fixed number of iterations, or with
some bash trickery), but it is better to fix them completelly,
eventually such tests will be submitted and pass review anyway.

By allocating process group for each test we can kill all the processes
in this process group, and this what this patch does.

This will also fix some test hangs (like in [1]) as well as some
possible issues in stress tests.

  [1]: https://s3.amazonaws.com/clickhouse-test-reports/0/e2c1230b00386c4d0096a245396ab3be7ce60950/stateless_tests__release__analyzer_/run.log

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
(cherry picked from commit 72fa58e192)
2023-12-24 09:43:58 +01:00
Alexey Milovidov
71c5ae548f
Revert "Fix leftover processes/hangs in tests" 2023-12-24 07:57:43 +03:00
Azat Khuzhin
72fa58e192 Fix leftover processes/hangs in tests
One of such cases is 02479_race_condition_between_insert_and_droppin_mv
[1], yes it can be fixed (by using fixed number of iterations, or with
some bash trickery), but it is better to fix them completelly,
eventually such tests will be submitted and pass review anyway.

By allocating process group for each test we can kill all the processes
in this process group, and this what this patch does.

This will also fix some test hangs (like in [1]) as well as some
possible issues in stress tests.

  [1]: https://s3.amazonaws.com/clickhouse-test-reports/0/e2c1230b00386c4d0096a245396ab3be7ce60950/stateless_tests__release__analyzer_/run.log

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-12-23 16:40:33 +01:00
Alexander Tokmakov
8f9dff8c88
Update clickhouse-test 2023-12-22 19:54:15 +01:00
Sema Checherinda
291567a5b2
Merge pull request #58007 from ClickHouse/chesema-stateless-run-timeout
more messages in ci
2023-12-20 13:50:12 +01:00
Raúl Marín
98c9f830d6 Revert "Merge pull request #55710 from guoxiaolongzte/clickhouse-test-add-prinln-nowTime"
This reverts commit efefad9e52, reversing
changes made to 8f7852f82d.
2023-12-20 10:16:47 +01:00
Anton Popov
4859a74b16
Correct values for randomization 2023-12-20 02:42:24 +01:00
Yakov Olkhovskiy
efefad9e52
Merge pull request #55710 from guoxiaolongzte/clickhouse-test-add-prinln-nowTime
clickhouse-test print  Case SN, Current Time and Case Name in one test case
2023-12-19 18:58:19 -05:00
Sema Checherinda
b4fec61814 fix style and black 2023-12-19 13:43:35 +01:00
Sema Checherinda
939d602c3c fix typo 2023-12-18 19:30:05 +01:00
Sema Checherinda
140f6dafd8 hung_check check with lighter queries 2023-12-18 19:16:50 +01:00
Anton Popov
32c4b74067 Merge remote-tracking branch 'upstream/master' into HEAD 2023-12-14 14:26:58 +00:00
Raúl Marín
a45c8bd7f1
Merge pull request #57315 from Algunenano/randomize_optimizations
Randomize disabled optimizations in CI
2023-12-13 10:42:11 +01:00
Anton Popov
b5c2623b3c Merge remote-tracking branch 'upstream/master' into HEAD 2023-12-11 14:14:25 +00:00
Alexey Milovidov
00aca9a3d4
Merge pull request #57634 from ClickHouse/fix-logs-and-exceptions-messages-formatting-4
Fix "logs and exception messages formatting", part 4
2023-12-08 22:02:02 +01:00
Anton Popov
e5f849bdae fix some tests 2023-12-08 15:30:12 +00:00
Alexey Milovidov
5dcdbf499b Fix "logs and exception messages formatting", part 4 2023-12-07 22:04:48 +01:00
Alexey Milovidov
81e0d081a2 Fix "logs and exception messages formatting", part 2 2023-12-07 21:48:43 +01:00
Alexey Milovidov
6ed8ea96af Fix "logs and exception messages formatting", part 2 2023-12-07 21:43:47 +01:00
Alexey Milovidov
ba1404136c Fix "logs and exception messages formatting", part 2 2023-12-07 21:42:57 +01:00
Anton Popov
2dbc2853b8 Merge remote-tracking branch 'upstream/master' into HEAD 2023-12-07 13:52:54 +00:00
Raúl Marín
89b9373398 More random after tests are successful 2023-12-05 18:17:47 +00:00
Raúl Marín
3eb68a6740 optimize_using_constraints is broken 2023-12-04 10:08:46 +00:00
Raúl Marín
3867adbb10 Adapt tests and avoid using optimize_syntax_fuse_functions at all 2023-11-30 19:19:26 +01:00
Raúl Marín
64bd77fe9c Do not randomize optimize_functions_to_subcolumns 2023-11-30 14:05:19 +01:00
Raúl Marín
025fcd3aa4 Enable the settings to do a full run with them enabled 2023-11-28 18:22:49 +01:00
Raúl Marín
ade60b5b45 Randomize disabled optimizations in CI 2023-11-28 14:12:26 +01:00
Alexey Milovidov
f8ebe5134d
Merge pull request #55836 from azat/dist/limit-by-fix
RFC: Fix "Cannot find column X in source stream" for Distributed queries with LIMIT BY
2023-11-26 04:03:41 +01:00
Kseniia Sumarokova
d384762123
Merge branch 'master' into fs-cache-improvement 2023-11-21 11:24:52 +01:00
Kseniia Sumarokova
9523bd0ad8
Fix config 2023-11-17 12:17:01 +01:00
robot-clickhouse
501d210647 Automatic style fix 2023-11-04 21:04:39 +00:00
Alexey Milovidov
f5e439d9df Add an option to enable or disable coverage collection in clickhouse-test 2023-11-04 21:53:05 +01:00
Alexey Milovidov
ea6cb1ad0c Maybe better 2023-10-30 01:04:50 +01:00
Alexey Milovidov
8e6a7fdff0 Fix f-string 2023-10-29 23:43:36 +01:00
Alexey Milovidov
4288cb3b78 Make clickhouse-test to calculate coverage on a per-test basis 2023-10-29 17:43:01 +01:00
Alexey Milovidov
8e0f487387 Initial support in clickhouse-test 2023-10-29 17:21:45 +01:00
kssenii
b13adbbeab Fix style check 2023-10-20 11:48:27 +02:00
Azat Khuzhin
7a096904ed clickhouse-test: export product function for jinja2
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-10-19 15:41:03 +02:00
kssenii
89272e0925 Fix upgrade check, randomize more settings 2023-10-19 11:23:56 +02:00
guoxiaolong
edcda1ed6d fix checkstyle 2023-10-18 10:12:42 +08:00
guoxiaolong
c552a91260 clickhouse-test case No 2023-10-17 19:27:42 +08:00
guoxiaolong
d2360302cf clickhouse-test print nowTime in one test case. Because the end time of all test cases is long, adding a specific test case time to print can observe the specific execution status and progress 2023-10-17 15:33:21 +08:00
Anton Popov
624782bd8f Merge remote-tracking branch 'upstream/master' into HEAD 2023-09-18 23:13:20 +00:00
Azat Khuzhin
f2ef2ec012 Fix clickhouse-test --no-drop-if-fail on reference mismatch
Previusly it worked only for the case when the clickhouse-client/script
exited with non 0, because the check was done before checking the
reference file.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-09-04 16:55:32 +02:00
Anton Popov
579ec699bf Merge remote-tracking branch 'upstream/master' into HEAD 2023-08-31 12:59:59 +00:00
Michael Kolupaev
ad71bf5f7d Fix flakiness of 00514_interval_operators 2023-08-28 23:35:47 +00:00
Dmitry Novik
16cbda14f8
Merge pull request #53700 from ClickHouse/fix-clickhouse-test-set
Fix running clickhouse-test with python 3.8
2023-08-22 18:07:21 +02:00
Dmitry Novik
1da32d2d1b Fix running clickhouse-test with python 3.8 2023-08-22 15:16:18 +00:00
Alexey Milovidov
a882bf1bb8
Merge pull request #53495 from ClickHouse/i_hate_materialized_views
Fix a race condition between RESTART REPLICAS and DROP DATABASE
2023-08-18 06:40:04 +03:00
Anton Popov
0865c0794e Merge remote-tracking branch 'upstream/master' into HEAD 2023-08-17 14:08:59 +00:00
Alexander Tokmakov
d7272410bd the worst fix for a deadlock ever 2023-08-17 00:42:51 +02:00
Alexander Tokmakov
3fc65a68c7 more parallel tests 2023-08-15 23:04:04 +02:00
Alexander Tokmakov
808bc59c49 Merge branch 'master' into less_no_parallel 2023-08-15 15:49:32 +02:00
Anton Popov
2e22b17d57 add docs and settings randomizations 2023-08-15 00:38:07 +00:00
Alexey Milovidov
fb903727c2 Fix wrong query in log messages check 2023-08-13 22:43:28 +02:00
Alexander Tokmakov
0d9e3ca345 remove no-parallel tag from some tests 2023-08-11 00:00:36 +02:00
Alexander Tokmakov
0874b507df
remove obsolete no-upgrade-check tag (#52915) 2023-08-04 14:09:59 +03:00
Kseniia Sumarokova
dbc11b83fd
Merge branch 'master' into kssenii-patch-6 2023-08-01 18:40:20 +02:00
kssenii
2b5a559d78 Fix test 2023-07-31 15:23:45 +02:00
Alexey Milovidov
9ccfa1ede8
Merge branch 'master' into revert-48277-revert-48196-jit-randomization 2023-07-31 15:58:35 +03:00
Alexey Milovidov
01067baf75
Merge branch 'master' into randomize-more-settings-2 2023-07-25 05:47:31 +03:00
Azat Khuzhin
bc167dfde8 clickhouse-test: add proper escaping for HTTP parameters
The problem is that old versions of cURL (7.81.0 at least) handle
additional parameters incorrectly if in previous parameter was "/":

    $ docker run --rm curlimages/curl:8.1.2 --http1.1 --get -vvv 'http://kernel.org/?bar=foo/baz' --data-urlencode "query=select 1 format Null"; echo
    > GET /?bar=foo/baz&query=select+1+format+Null HTTP/1.1
    > User-Agent: curl/8.1.2

    $ docker run --rm curlimages/curl:7.81.0 --http1.1 --get -vvv 'http://kernel.org/?bar=foo/baz' --data-urlencode "query=select 1 format Null"; echo
    > GET /?bar=foo/baz?query=select+1+format+Null HTTP/1.1
    > User-Agent: curl/7.81.0-DEV

Note, that I thought about making the same for cli, but it is not that
easy, even after getting rid of sh -c and string contantenation, it
still cannot be done for CLICKHOUSE_CLIENT_OPT.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-07-24 06:13:12 +02:00
Azat Khuzhin
2389e0f0b6 Randomize timezone in tests across non-deterministic around 1970 and default
There was some cases when some patches to the datetime code leads to
flaky tests, due to the tests itself had been runned using regular
timezone (TZ).

But if you will this tests with something "specific" (that is not
strictly defined around 1970 year), those tests will fail.

So to catch such issues in the PRs itself, let's randomize
session_timezone as well.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-07-24 06:13:12 +02:00
Alexey Milovidov
71d6206fc4
Fix Python 2023-07-16 17:42:22 +03:00
Alexey Milovidov
c57dbf3410 Merge branch 'test-cleanup' of github.com:ClickHouse/ClickHouse into test-cleanup 2023-07-15 23:12:29 +02:00
Alexey Milovidov
4f6a0c2fac Loosen the check 2023-07-15 23:11:22 +02:00
robot-clickhouse
a61bc7cfa5 Automatic style fix 2023-07-09 02:03:47 +00:00
Alexey Milovidov
de0837fe21 Check that functional tests cleanup their tables 2023-07-09 03:55:25 +02:00
Alexey Milovidov
9144a2dbb2 Fix unrelated messages from LSan in clickhouse-client 2023-07-08 05:23:13 +02:00
Alexey Milovidov
867c4ece64
Merge branch 'master' into revert-48277-revert-48196-jit-randomization 2023-07-04 22:56:27 +03:00
Alexey Milovidov
d84a4626b0 Disable "compile_expressions" setting by default 2023-06-25 03:22:59 +02:00
Alexey Milovidov
5ddf0dc229
Merge branch 'master' into revert-48277-revert-48196-jit-randomization 2023-06-23 17:54:54 +03:00
Sema Checherinda
d21fc22607 fix 02428_combinators_with_over_statement 2023-06-16 16:38:18 +02:00
Kruglov Pavel
42393b51ee
Fix style 2023-06-12 13:45:00 +02:00
avogar
54414be47b Better 2023-06-08 11:14:43 +00:00
avogar
44ee530b4d Kill gdb in clickhouse-test before getting stacktraces 2023-06-08 11:04:01 +00:00
Alexander Tokmakov
29183ebee9
Merge branch 'master' into revert-48277-revert-48196-jit-randomization 2023-06-02 15:02:11 +03:00
Suzy Wang
49afecc28b
Merge branch 'master' into fvt_unknown_result 2023-05-24 15:58:37 -04:00
Suzy Wang
62fc5bb3eb get detailed error instead of unknown result 2023-05-24 12:54:30 -07:00
Alexander Tokmakov
b6716a8f0f Merge branch 'master' into fix_some_tests4 2023-05-16 14:46:27 +02:00
Alexander Tokmakov
05ae7b2c2d fix some tests 2023-05-15 18:28:12 +02:00
Alexander Gololobov
fcce60ad25 Randomize enable_multiple_prewhere_read_steps 2023-05-12 15:09:34 +02:00
Alexander Tokmakov
99677f3d1a
Update clickhouse-test (#49712) 2023-05-10 00:22:55 +03:00
Michael Kolupaev
49394a097e Fix 'noisy Warning messages' failing when there are no Warning messages 2023-05-06 10:39:59 -07:00
Michael Kolupaev
3745783a22 Make 'exceptions shorter than 30' test less noisy 2023-05-06 10:39:59 -07:00
Alexey Milovidov
fe2f267dbe
Merge branch 'master' into randomize-more-settings-2 2023-05-06 18:19:56 +03:00
Nikolai Kochetov
465bc77c20
Merge pull request #49389 from ClickHouse/fix-hung-queries-print
Fix printing hung queries in clickhouse-test.
2023-05-03 13:20:00 +02:00
Alexander Tokmakov
1ec3040d0c
Merge pull request #49420 from ClickHouse/fix_00002
Fix some bad error messages
2023-05-03 13:33:53 +03:00
Alexander Tokmakov
495325f664 fix some bad error messages, suppress others 2023-05-02 17:55:42 +02:00
Nikolai Kochetov
7dab1560b1 Fix printing hung queries in clickhouse-test. 2023-05-02 10:44:30 +00:00
Frank Chen
f74367d74d Fix style check 2023-04-28 00:08:26 +08:00
Frank Chen
33443351c3 Resolve review comments
Signed-off-by: Frank Chen <frank.chen021@outlook.com>
2023-04-27 21:03:14 +08:00
Frank Chen
5b4e014b8e Don't choose io_uring for macOS 2023-04-27 15:18:26 +08:00
Kseniia Sumarokova
0caa34f55e
Do not randomize prefetch settings for debug build (#49134)
* Do not randomize prefetch settings for debug build

* Update tests/clickhouse-test

---------

Co-authored-by: Alexander Tokmakov <tavplubix@gmail.com>
Co-authored-by: Alexander Tokmakov <tavplubix@clickhouse.com>
2023-04-25 16:50:13 +03:00
Kseniia Sumarokova
fff867da68
Update clickhouse-test 2023-04-24 13:09:17 +02:00
Kseniia Sumarokova
00b2960c09
Update clickhouse-test 2023-04-24 13:04:19 +02:00
kssenii
6edb61015d Fix style check 2023-04-20 19:35:49 +02:00
kssenii
fe6af0c5ca Randomize prefetch settings 2023-04-20 14:08:46 +02:00
Alexander Tokmakov
a154fa73f4
Merge pull request #48401 from ClickHouse/fix_00002_log_messages
Find big allocations without memory limits checks
2023-04-05 23:52:39 +03:00
Alexander Tokmakov
0e0e89d596 fix 2023-04-05 15:10:09 +02:00
Alexander Tokmakov
ba65be22da fix bad error messages 2023-04-04 20:21:22 +02:00
Yakov Olkhovskiy
00e335530e
close client 2023-04-03 08:15:18 -04:00
Alexander Tokmakov
661ce2afa1
Merge pull request #48315 from ClickHouse/followup_47863
Follow-up to #47863
2023-04-02 01:56:41 +03:00
robot-ch-test-poll
5555fc78be
Merge pull request #48287 from Algunenano/randomize_vertical_merges
Randomize allow_vertical_merges_from_compact_to_wide_parts in tests
2023-04-01 20:48:08 +02:00
Alexey Milovidov
8d38d3aab1
Merge pull request #48286 from ClickHouse/randomize-compress-marks
Randomize compression of marks and indices in tests
2023-04-01 21:46:37 +03:00
Alexander Tokmakov
230adac9f6 fixes 2023-04-01 18:23:59 +02:00
Alexander Tokmakov
48b23dd012
Fix race between DROP MatView and RESTART REPLICAS (#47863)
* fix race between drop mv and restart replicas

* unrelated: fix bad exception messages

* fix test

* fix

* fix

* fix

* fix

* fix test

* fix

* fix test

* Update 02437_drop_mv_restart_replicas.sh

* fix tests
2023-04-01 15:26:00 +03:00
Raúl Marín
84c30f2f63 I've got no style 2023-03-31 14:49:53 +02:00
Raúl Marín
d158b18703 Randomize allow_vertical_merges_from_compact_to_wide_parts in tests 2023-03-31 14:23:30 +02:00
Alexey Milovidov
5206d4b926 Randomize compression of marks and indices in tests 2023-03-31 14:12:55 +02:00
Alexey Milovidov
4f27391a72
Revert "Revert "Randomize JIT settings in tests"" 2023-03-31 13:37:41 +03:00
Alexander Tokmakov
792979a527
Revert "Randomize JIT settings in tests" 2023-03-31 12:24:47 +03:00
Alexey Milovidov
990ef56443 Randomize JIT settings in tests 2023-03-30 10:55:29 +02:00
Alexey Milovidov
202dc90045 Randomize JIT settings in tests 2023-03-30 10:01:52 +02:00
Mikhail f. Shiryaev
290ee6bbf1
Merge pull request #47790 from CurtizJ/apply-black-to-all-files
Apply black formatter to all python scripts
2023-03-24 14:20:50 +03:00
Anton Popov
0ee8dfad53 apply black formatter 2023-03-23 15:33:23 +00:00
Antonio Andelic
f218f3671d Make 01086_odbc_roundtrip less flaky 2023-03-21 09:40:10 +00:00
Anton Popov
3f53477be7 remove unused code 2023-03-20 14:44:11 +00:00
Anton Popov
7dc509bd43 Merge remote-tracking branch 'upstream/master' into HEAD 2023-03-20 14:40:29 +00:00
Nikolay Degterinsky
5bdbb7670d
Fix exception message in clickhouse-test 2023-03-15 18:42:26 +01:00
Vladimir C
045960f1a1
Merge pull request #47108 from ClickHouse/vdimir/http_wait_end_of_query_settings 2023-03-15 15:31:12 +01:00
Alexey Milovidov
f4f7e110b6
Merge branch 'master' into patch-2 2023-03-15 02:03:52 +03:00
Alexander Tokmakov
275acb756a
Merge pull request #47575 from azat/ci/log-stats
Add missing SYSTEM FLUSH LOGS for log messages statistics
2023-03-14 17:37:07 +03:00
Vasily Nemkov
d3199da1d2
Simplify clickhouse-test usage with -b
Now it is possible to run it even without `clickhouse` binary in PATH.
2023-03-14 15:31:36 +01:00
Azat Khuzhin
ad5dfc159b Add missing SYSTEM FLUSH LOGS for log messages statistics
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-03-14 14:34:02 +01:00
Alexander Tokmakov
ba6c6255a3 hide too noisy log messages, fix some tests 2023-03-14 00:23:36 +01:00
Vladimir C
6a12bda189
Fix style tests/clickhouse-test 2023-03-09 15:23:55 +01:00
Vladimir C
1b264ad51e
Merge branch 'master' into vdimir/http_wait_end_of_query_settings 2023-03-09 13:37:56 +01:00
Ilya Yatsishin
f8eb1fad91
Disable parallel format in health check 2023-03-07 23:06:14 +01:00