Commit Graph

649 Commits

Author SHA1 Message Date
Alexey Milovidov
8e0f487387 Initial support in clickhouse-test 2023-10-29 17:21:45 +01:00
kssenii
b13adbbeab Fix style check 2023-10-20 11:48:27 +02:00
Azat Khuzhin
7a096904ed clickhouse-test: export product function for jinja2
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-10-19 15:41:03 +02:00
kssenii
89272e0925 Fix upgrade check, randomize more settings 2023-10-19 11:23:56 +02:00
guoxiaolong
edcda1ed6d fix checkstyle 2023-10-18 10:12:42 +08:00
guoxiaolong
c552a91260 clickhouse-test case No 2023-10-17 19:27:42 +08:00
guoxiaolong
d2360302cf clickhouse-test print nowTime in one test case. Because the end time of all test cases is long, adding a specific test case time to print can observe the specific execution status and progress 2023-10-17 15:33:21 +08:00
Anton Popov
624782bd8f Merge remote-tracking branch 'upstream/master' into HEAD 2023-09-18 23:13:20 +00:00
Azat Khuzhin
f2ef2ec012 Fix clickhouse-test --no-drop-if-fail on reference mismatch
Previusly it worked only for the case when the clickhouse-client/script
exited with non 0, because the check was done before checking the
reference file.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-09-04 16:55:32 +02:00
Anton Popov
579ec699bf Merge remote-tracking branch 'upstream/master' into HEAD 2023-08-31 12:59:59 +00:00
Michael Kolupaev
ad71bf5f7d Fix flakiness of 00514_interval_operators 2023-08-28 23:35:47 +00:00
Dmitry Novik
16cbda14f8
Merge pull request #53700 from ClickHouse/fix-clickhouse-test-set
Fix running clickhouse-test with python 3.8
2023-08-22 18:07:21 +02:00
Dmitry Novik
1da32d2d1b Fix running clickhouse-test with python 3.8 2023-08-22 15:16:18 +00:00
Alexey Milovidov
a882bf1bb8
Merge pull request #53495 from ClickHouse/i_hate_materialized_views
Fix a race condition between RESTART REPLICAS and DROP DATABASE
2023-08-18 06:40:04 +03:00
Anton Popov
0865c0794e Merge remote-tracking branch 'upstream/master' into HEAD 2023-08-17 14:08:59 +00:00
Alexander Tokmakov
d7272410bd the worst fix for a deadlock ever 2023-08-17 00:42:51 +02:00
Alexander Tokmakov
3fc65a68c7 more parallel tests 2023-08-15 23:04:04 +02:00
Alexander Tokmakov
808bc59c49 Merge branch 'master' into less_no_parallel 2023-08-15 15:49:32 +02:00
Anton Popov
2e22b17d57 add docs and settings randomizations 2023-08-15 00:38:07 +00:00
Alexey Milovidov
fb903727c2 Fix wrong query in log messages check 2023-08-13 22:43:28 +02:00
Alexander Tokmakov
0d9e3ca345 remove no-parallel tag from some tests 2023-08-11 00:00:36 +02:00
Alexander Tokmakov
0874b507df
remove obsolete no-upgrade-check tag (#52915) 2023-08-04 14:09:59 +03:00
Kseniia Sumarokova
dbc11b83fd
Merge branch 'master' into kssenii-patch-6 2023-08-01 18:40:20 +02:00
kssenii
2b5a559d78 Fix test 2023-07-31 15:23:45 +02:00
Alexey Milovidov
9ccfa1ede8
Merge branch 'master' into revert-48277-revert-48196-jit-randomization 2023-07-31 15:58:35 +03:00
Alexey Milovidov
01067baf75
Merge branch 'master' into randomize-more-settings-2 2023-07-25 05:47:31 +03:00
Azat Khuzhin
bc167dfde8 clickhouse-test: add proper escaping for HTTP parameters
The problem is that old versions of cURL (7.81.0 at least) handle
additional parameters incorrectly if in previous parameter was "/":

    $ docker run --rm curlimages/curl:8.1.2 --http1.1 --get -vvv 'http://kernel.org/?bar=foo/baz' --data-urlencode "query=select 1 format Null"; echo
    > GET /?bar=foo/baz&query=select+1+format+Null HTTP/1.1
    > User-Agent: curl/8.1.2

    $ docker run --rm curlimages/curl:7.81.0 --http1.1 --get -vvv 'http://kernel.org/?bar=foo/baz' --data-urlencode "query=select 1 format Null"; echo
    > GET /?bar=foo/baz?query=select+1+format+Null HTTP/1.1
    > User-Agent: curl/7.81.0-DEV

Note, that I thought about making the same for cli, but it is not that
easy, even after getting rid of sh -c and string contantenation, it
still cannot be done for CLICKHOUSE_CLIENT_OPT.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-07-24 06:13:12 +02:00
Azat Khuzhin
2389e0f0b6 Randomize timezone in tests across non-deterministic around 1970 and default
There was some cases when some patches to the datetime code leads to
flaky tests, due to the tests itself had been runned using regular
timezone (TZ).

But if you will this tests with something "specific" (that is not
strictly defined around 1970 year), those tests will fail.

So to catch such issues in the PRs itself, let's randomize
session_timezone as well.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-07-24 06:13:12 +02:00
Alexey Milovidov
71d6206fc4
Fix Python 2023-07-16 17:42:22 +03:00
Alexey Milovidov
c57dbf3410 Merge branch 'test-cleanup' of github.com:ClickHouse/ClickHouse into test-cleanup 2023-07-15 23:12:29 +02:00
Alexey Milovidov
4f6a0c2fac Loosen the check 2023-07-15 23:11:22 +02:00
robot-clickhouse
a61bc7cfa5 Automatic style fix 2023-07-09 02:03:47 +00:00
Alexey Milovidov
de0837fe21 Check that functional tests cleanup their tables 2023-07-09 03:55:25 +02:00
Alexey Milovidov
9144a2dbb2 Fix unrelated messages from LSan in clickhouse-client 2023-07-08 05:23:13 +02:00
Alexey Milovidov
867c4ece64
Merge branch 'master' into revert-48277-revert-48196-jit-randomization 2023-07-04 22:56:27 +03:00
Alexey Milovidov
d84a4626b0 Disable "compile_expressions" setting by default 2023-06-25 03:22:59 +02:00
Alexey Milovidov
5ddf0dc229
Merge branch 'master' into revert-48277-revert-48196-jit-randomization 2023-06-23 17:54:54 +03:00
Sema Checherinda
d21fc22607 fix 02428_combinators_with_over_statement 2023-06-16 16:38:18 +02:00
Kruglov Pavel
42393b51ee
Fix style 2023-06-12 13:45:00 +02:00
avogar
54414be47b Better 2023-06-08 11:14:43 +00:00
avogar
44ee530b4d Kill gdb in clickhouse-test before getting stacktraces 2023-06-08 11:04:01 +00:00
Alexander Tokmakov
29183ebee9
Merge branch 'master' into revert-48277-revert-48196-jit-randomization 2023-06-02 15:02:11 +03:00
Suzy Wang
49afecc28b
Merge branch 'master' into fvt_unknown_result 2023-05-24 15:58:37 -04:00
Suzy Wang
62fc5bb3eb get detailed error instead of unknown result 2023-05-24 12:54:30 -07:00
Alexander Tokmakov
b6716a8f0f Merge branch 'master' into fix_some_tests4 2023-05-16 14:46:27 +02:00
Alexander Tokmakov
05ae7b2c2d fix some tests 2023-05-15 18:28:12 +02:00
Alexander Gololobov
fcce60ad25 Randomize enable_multiple_prewhere_read_steps 2023-05-12 15:09:34 +02:00
Alexander Tokmakov
99677f3d1a
Update clickhouse-test (#49712) 2023-05-10 00:22:55 +03:00
Michael Kolupaev
49394a097e Fix 'noisy Warning messages' failing when there are no Warning messages 2023-05-06 10:39:59 -07:00
Michael Kolupaev
3745783a22 Make 'exceptions shorter than 30' test less noisy 2023-05-06 10:39:59 -07:00
Alexey Milovidov
fe2f267dbe
Merge branch 'master' into randomize-more-settings-2 2023-05-06 18:19:56 +03:00
Nikolai Kochetov
465bc77c20
Merge pull request #49389 from ClickHouse/fix-hung-queries-print
Fix printing hung queries in clickhouse-test.
2023-05-03 13:20:00 +02:00
Alexander Tokmakov
1ec3040d0c
Merge pull request #49420 from ClickHouse/fix_00002
Fix some bad error messages
2023-05-03 13:33:53 +03:00
Alexander Tokmakov
495325f664 fix some bad error messages, suppress others 2023-05-02 17:55:42 +02:00
Nikolai Kochetov
7dab1560b1 Fix printing hung queries in clickhouse-test. 2023-05-02 10:44:30 +00:00
Frank Chen
f74367d74d Fix style check 2023-04-28 00:08:26 +08:00
Frank Chen
33443351c3 Resolve review comments
Signed-off-by: Frank Chen <frank.chen021@outlook.com>
2023-04-27 21:03:14 +08:00
Frank Chen
5b4e014b8e Don't choose io_uring for macOS 2023-04-27 15:18:26 +08:00
Kseniia Sumarokova
0caa34f55e
Do not randomize prefetch settings for debug build (#49134)
* Do not randomize prefetch settings for debug build

* Update tests/clickhouse-test

---------

Co-authored-by: Alexander Tokmakov <tavplubix@gmail.com>
Co-authored-by: Alexander Tokmakov <tavplubix@clickhouse.com>
2023-04-25 16:50:13 +03:00
Kseniia Sumarokova
fff867da68
Update clickhouse-test 2023-04-24 13:09:17 +02:00
Kseniia Sumarokova
00b2960c09
Update clickhouse-test 2023-04-24 13:04:19 +02:00
kssenii
6edb61015d Fix style check 2023-04-20 19:35:49 +02:00
kssenii
fe6af0c5ca Randomize prefetch settings 2023-04-20 14:08:46 +02:00
Alexander Tokmakov
a154fa73f4
Merge pull request #48401 from ClickHouse/fix_00002_log_messages
Find big allocations without memory limits checks
2023-04-05 23:52:39 +03:00
Alexander Tokmakov
0e0e89d596 fix 2023-04-05 15:10:09 +02:00
Alexander Tokmakov
ba65be22da fix bad error messages 2023-04-04 20:21:22 +02:00
Yakov Olkhovskiy
00e335530e
close client 2023-04-03 08:15:18 -04:00
Alexander Tokmakov
661ce2afa1
Merge pull request #48315 from ClickHouse/followup_47863
Follow-up to #47863
2023-04-02 01:56:41 +03:00
robot-ch-test-poll
5555fc78be
Merge pull request #48287 from Algunenano/randomize_vertical_merges
Randomize allow_vertical_merges_from_compact_to_wide_parts in tests
2023-04-01 20:48:08 +02:00
Alexey Milovidov
8d38d3aab1
Merge pull request #48286 from ClickHouse/randomize-compress-marks
Randomize compression of marks and indices in tests
2023-04-01 21:46:37 +03:00
Alexander Tokmakov
230adac9f6 fixes 2023-04-01 18:23:59 +02:00
Alexander Tokmakov
48b23dd012
Fix race between DROP MatView and RESTART REPLICAS (#47863)
* fix race between drop mv and restart replicas

* unrelated: fix bad exception messages

* fix test

* fix

* fix

* fix

* fix

* fix test

* fix

* fix test

* Update 02437_drop_mv_restart_replicas.sh

* fix tests
2023-04-01 15:26:00 +03:00
Raúl Marín
84c30f2f63 I've got no style 2023-03-31 14:49:53 +02:00
Raúl Marín
d158b18703 Randomize allow_vertical_merges_from_compact_to_wide_parts in tests 2023-03-31 14:23:30 +02:00
Alexey Milovidov
5206d4b926 Randomize compression of marks and indices in tests 2023-03-31 14:12:55 +02:00
Alexey Milovidov
4f27391a72
Revert "Revert "Randomize JIT settings in tests"" 2023-03-31 13:37:41 +03:00
Alexander Tokmakov
792979a527
Revert "Randomize JIT settings in tests" 2023-03-31 12:24:47 +03:00
Alexey Milovidov
990ef56443 Randomize JIT settings in tests 2023-03-30 10:55:29 +02:00
Alexey Milovidov
202dc90045 Randomize JIT settings in tests 2023-03-30 10:01:52 +02:00
Mikhail f. Shiryaev
290ee6bbf1
Merge pull request #47790 from CurtizJ/apply-black-to-all-files
Apply black formatter to all python scripts
2023-03-24 14:20:50 +03:00
Anton Popov
0ee8dfad53 apply black formatter 2023-03-23 15:33:23 +00:00
Antonio Andelic
f218f3671d Make 01086_odbc_roundtrip less flaky 2023-03-21 09:40:10 +00:00
Anton Popov
3f53477be7 remove unused code 2023-03-20 14:44:11 +00:00
Anton Popov
7dc509bd43 Merge remote-tracking branch 'upstream/master' into HEAD 2023-03-20 14:40:29 +00:00
Nikolay Degterinsky
5bdbb7670d
Fix exception message in clickhouse-test 2023-03-15 18:42:26 +01:00
Vladimir C
045960f1a1
Merge pull request #47108 from ClickHouse/vdimir/http_wait_end_of_query_settings 2023-03-15 15:31:12 +01:00
Alexey Milovidov
f4f7e110b6
Merge branch 'master' into patch-2 2023-03-15 02:03:52 +03:00
Alexander Tokmakov
275acb756a
Merge pull request #47575 from azat/ci/log-stats
Add missing SYSTEM FLUSH LOGS for log messages statistics
2023-03-14 17:37:07 +03:00
Vasily Nemkov
d3199da1d2
Simplify clickhouse-test usage with -b
Now it is possible to run it even without `clickhouse` binary in PATH.
2023-03-14 15:31:36 +01:00
Azat Khuzhin
ad5dfc159b Add missing SYSTEM FLUSH LOGS for log messages statistics
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-03-14 14:34:02 +01:00
Alexander Tokmakov
ba6c6255a3 hide too noisy log messages, fix some tests 2023-03-14 00:23:36 +01:00
Vladimir C
6a12bda189
Fix style tests/clickhouse-test 2023-03-09 15:23:55 +01:00
Vladimir C
1b264ad51e
Merge branch 'master' into vdimir/http_wait_end_of_query_settings 2023-03-09 13:37:56 +01:00
Ilya Yatsishin
f8eb1fad91
Disable parallel format in health check 2023-03-07 23:06:14 +01:00
Anton Popov
6f3e4d4137
Merge pull request #46118 from CurtizJ/fix-issues-with-sparse
Randomize setting `ratio_of_defaults_for_sparse_serialization`
2023-03-05 22:28:18 +01:00
Alexander Tokmakov
cadaf06829
Merge pull request #46681 from ClickHouse/fix_insert_cancellation_in_native_protocol
Fix queries cancellation when a client dies
2023-03-03 16:27:20 +03:00
Nikolai Kochetov
de6a74a907
Merge branch 'master' into add-stupid-retries-for-health-check-in-tests 2023-03-02 20:57:26 +01:00
Nikolai Kochetov
e91cdb4f8d Add stupid retries in clickhouse-test health check. 2023-03-02 19:03:17 +00:00
Alexander Tokmakov
c518f3690e Merge branch 'master' into fix_insert_cancellation_in_native_protocol 2023-03-02 18:24:08 +01:00
vdimir
b782cba809 Add settings http_wait_end_of_query and http_response_buffer_size 2023-03-02 16:03:10 +01:00
Anton Popov
c4a999ce29 Merge remote-tracking branch 'upstream/master' into HEAD 2023-03-01 19:33:16 +00:00
Alexander Tokmakov
bb5b17fd09 make stacktraces in hung check more readable 2023-03-01 17:34:43 +01:00
Alexander Tokmakov
a97e15e36f Merge branch 'master' into fix_insert_cancellation_in_native_protocol 2023-03-01 14:26:41 +01:00
Alexander Tokmakov
d5469c0ab4 fix clickhouse-test 2023-02-28 00:07:50 +01:00
avogar
6b7d3bf07b Merge branch 'master' of github.com:ClickHouse/ClickHouse into split-bc-check 2023-02-24 13:52:22 +00:00
Alexander Tokmakov
7122ebab4d fix clickhouse-test 2023-02-24 03:06:33 +01:00
Anton Popov
d926713cf5 Merge remote-tracking branch 'upstream/master' into HEAD 2023-02-23 23:04:22 +00:00
Alexander Tokmakov
d7cd4c6fca
Update clickhouse-test 2023-02-23 20:20:29 +03:00
Alexey Milovidov
e66c05e008
Merge pull request #38750 from azat/revert-no-stress
Revert "Merge pull request #38212 from azat/no-stress"
2023-02-16 22:13:17 +03:00
vdimir
b64ed8b5d0
fix tests/clickhouse-test file perm 2023-02-16 11:29:17 +01:00
Azat Khuzhin
810a624f71 Fix parse_args() in clickhouse-test 2023-02-15 22:20:18 +01:00
Azat Khuzhin
bddead1ea2 Revert "Merge pull request #38212 from azat/no-stress"
This reverts commit 24d5be1669, reversing
changes made to a756b4be27.

After #38717 there is no need in --no-stress, since there will be no
such checks for previous releases that fails.

Reverts: #38212
Requires: #38717
2023-02-15 22:11:35 +01:00
Alexey Milovidov
556d5cbeb3
Merge branch 'master' into clickhouse-test-bin-fix 2023-02-15 17:13:40 +03:00
Anton Popov
4af836a592 Merge remote-tracking branch 'upstream/master' into HEAD 2023-02-14 18:42:03 +00:00
Azat Khuzhin
96c8c783f5 One more trim of stdout in clickhouse-test
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-02-13 16:50:12 +01:00
Azat Khuzhin
72f61ae399 Do not polute logs in clickhouse-test
In case of there are less then 100 lines, there is no need to split
lines with '#', and in fact, it will only looks it odd and hard to
interpret.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-02-13 16:46:56 +01:00
Alexander Tokmakov
8e145450bd
Revert "Beter diagnostics from http in clickhouse-test" 2023-02-11 21:01:50 +03:00
Alexander Tokmakov
b2870d3e0c
Update clickhouse-test 2023-02-10 19:34:13 +03:00
Alexander Tokmakov
681baa10b7
Update clickhouse-test 2023-02-10 18:56:22 +03:00
avogar
53a3cfc9e0 Merge branch 'master' of github.com:ClickHouse/ClickHouse into split-bc-check 2023-02-09 13:24:19 +00:00
Alexander Tokmakov
16f35b6a6c
Update clickhouse-test 2023-02-09 00:42:08 +03:00
Alexander Tokmakov
c1c278ab9d make a bug in HTTP interface less annoying 2023-02-08 20:46:37 +01:00
Anton Popov
fe48ab2b1d randomize setting 'ratio_of_defaults_for_sparse_serialization' 2023-02-07 14:59:15 +00:00
Alexey Milovidov
6a389eef32
Add a comment 2023-02-06 22:27:46 +03:00
Azat Khuzhin
ad3b4da488 Add also last messages from stdout/stderr/debuglog in clickhouse-test
First lines not very helpful usually, since the problem usually happens
at the end [1], though some tricky stuff may happens at the beginning,
so let's output both.

  [1]: https://s3.amazonaws.com/clickhouse-test-reports/45654/1716af465d376f5335720b5045bcccdd9e1823aa/fast_test.html

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-02-06 19:47:44 +01:00
Alexey Milovidov
22bd0b6f69
Merge pull request #38983 from CurtizJ/randomize-mt-settings
Allow to randomize merge tree settings in tests
2023-02-04 02:59:52 +01:00
Alexey Milovidov
11434feed9
Merge branch 'master' into randomize-more-settings-2 2023-02-04 02:59:32 +01:00
Nikita Mikhaylov
33877b5e00
Parallel replicas. Part [2] (#43772) 2023-02-03 14:34:18 +01:00
Alexey Milovidov
84c034a8a7
Merge branch 'master' into randomize-more-settings-2 2023-02-03 04:37:01 +03:00
Vladimir C
05ba4112e3
Fix style 2023-02-01 13:01:58 +01:00
Alexey Milovidov
bf9f62dcbd
Merge pull request #38456 from sauliusvl/iouring
Re-introduce io_uring read method
2023-02-01 06:41:32 +03:00
Alexey Milovidov
ca181afc8f Fix Flaky Check 2023-01-30 02:51:21 +01:00
Azat Khuzhin
e7df64df77 clickhouse-test: replace NUL byte to avoid breaking CSV parser
On CI [1]:

    022-12-12 12:49:55 01910_client_replxx_container_overflow_long:                            [ FAIL ] 2.25 sec. - return code: 1
    2022-12-12 12:49:55 send: spawn id exp4 not open
    2022-12-12 12:49:55     while executing
    ...
    Traceback (most recent call last):
      File "/home/ubuntu/actions-runner/_work/_temp/stateless_debug/ClickHouse/tests/ci/functional_test_check.py", line 327, in <module>
        state, description, test_results, additional_logs = process_results(
      File "/home/ubuntu/actions-runner/_work/_temp/stateless_debug/ClickHouse/tests/ci/functional_test_check.py", line 173, in process_results
        test_results = list(csv.reader(results_file, delimiter="\t"))  # type: ignore
    _csv.Error: line contains NUL
    INFO:root:Update Mergeable Check by Stateless tests (asan)
    Error: Process completed with exit code 1.

      [1]: https://github.com/ClickHouse/ClickHouse/actions/runs/3669487007/jobs/6221215715

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-01-29 08:59:37 +01:00
Azat Khuzhin
d5f7adb4cd clickhouse-test: trim debuglog to the first 100 lines as other logs
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-01-29 08:59:37 +01:00
Saulius Valatka
ac2c921bdf add initial io_uring support 2023-01-28 21:54:44 +02:00
Azat Khuzhin
9456f4f83f Add missing SYSTEM FLUSH LOGS for clickhouse-test
CI: https://s3.amazonaws.com/clickhouse-test-reports/45150/718ad72871e7d6788242b06822918057723939f7/bugfix_validate_check.html
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-01-27 21:09:13 +01:00
Anton Popov
d8fe2bcbaa
Merge branch 'master' into randomize-mt-settings 2023-01-27 16:03:00 +01:00
Anton Popov
92fa39798b Merge remote-tracking branch 'upstream/master' into HEAD 2023-01-26 13:29:23 +00:00
Vladimir C
25f32d5abb
Print warnings for client and extract_from_config options in clickhouse-test 2023-01-26 11:43:54 +01:00
Alexander Tokmakov
a584ad0eb1 forbid runtime strings 2023-01-26 10:52:47 +01:00
Alexander Tokmakov
9b670946db Merge branch 'master' into exception_message_patterns5 2023-01-26 00:41:32 +01:00
Alexander Tokmakov
3744fa2c63 format more messages 2023-01-25 21:16:42 +01:00
avogar
ced2f69a01 Merge branch 'master' of github.com:ClickHouse/ClickHouse into split-bc-check 2023-01-25 19:47:27 +00:00
Alexander Tokmakov
381c5a62b8 fix 2023-01-25 17:33:34 +01:00
Alexander Tokmakov
c19110e186
Merge pull request #43396 from azat/tests/improve-hung-check
RFC: tests: add stacktraces for hunged queries
2023-01-25 18:35:39 +03:00
Alexander Tokmakov
ae795d87b2 fix 2023-01-25 16:06:40 +01:00
Alexander Tokmakov
6eb557b2ba Merge branch 'master' into exception_message_patterns4 2023-01-25 13:49:17 +01:00
Alexander Tokmakov
527156c104 fix 2023-01-25 12:53:00 +01:00
Alexander Tokmakov
d187215d53 fix 2023-01-24 16:04:37 +01:00
vdimir
d8d3da951e
Ignore utf errors in clickhouse-test reportLogStats 2023-01-24 12:30:19 +00:00