Commit Graph

361 Commits

Author SHA1 Message Date
Alexander Tokmakov
c6c22409a2 Merge branch 'master' into replicated_database_improvements 2022-06-29 14:38:07 +02:00
Alexander Tokmakov
428628238f Merge branch 'master' into deprecate_ordinary_database 2022-06-28 15:40:33 +02:00
Alexander Tokmakov
851534c9f7 Merge branch 'master' into deprecate_ordinary_database 2022-06-27 23:47:49 +02:00
Dmitry Novik
c24c041b12 Handle full queue exception in tests 2022-06-27 20:54:52 +00:00
Alexander Tokmakov
379fecf7f9 try suppress failures with MaterializedPostgreSQL 2022-06-24 17:15:15 +02:00
Alexander Tokmakov
2c5a88faba deprecate Ordinary database 2022-06-23 10:20:14 +02:00
Azat Khuzhin
e307a728ca tests: add no-stress tag
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-06-20 07:48:10 +03:00
Azat Khuzhin
8937785551 tests/clickhouse-test: apply black
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-06-20 07:43:59 +03:00
Azat Khuzhin
fddd031385 Randomize settings related to in-order read/aggregation
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-06-07 08:29:00 +03:00
Alexander Tokmakov
9af6e237e1 remove retries 2022-05-28 19:25:56 +02:00
Alexander Tokmakov
1ae919a18a Revert "Merge pull request #35865 from azat/clickhouse-test-left-queries"
This reverts commit 18d094d79d, reversing
changes made to 224f4dc620.
2022-05-28 19:23:58 +02:00
Alexander Tokmakov
e33d986cf1 Revert "Merge pull request #34924 from azat/tests-overlaps"
This reverts commit f13e436d40, reversing
changes made to aa3c05e9d4.
2022-05-28 19:20:32 +02:00
Alexander Tokmakov
281027e490 do not fail to start if replica seems dropped 2022-05-13 20:05:39 +02:00
Alexey Milovidov
ad9a377f42 Add settings randomization 2022-05-11 01:30:59 +02:00
Alexander Tokmakov
222277f4f0 fix 2022-05-09 14:55:51 +02:00
Alexander Tokmakov
376398f1a9 support secure connection in clickhouse-test 2022-05-05 21:11:38 +02:00
Mikhail f. Shiryaev
c359edbd8d
Fix some linter issues 2022-05-02 18:21:39 +02:00
Mikhail f. Shiryaev
76efb83f7c
Try to fix corrupted report for died server 2022-05-02 18:21:37 +02:00
Mikhail f. Shiryaev
5e019efb7e
Try to fix corrupted report for died server 2022-05-02 17:48:40 +02:00
Mikhail f. Shiryaev
2349d3d6a6
Use f-strings, some simplifications 2022-05-02 17:48:39 +02:00
Mikhail f. Shiryaev
cd27c13331
Apply black to clickhouse-test 2022-05-02 17:48:38 +02:00
Azat Khuzhin
287c098c7e Fix clickhouse-test for server w/o is_all_data_sent in system.processes 2022-04-28 23:19:21 +03:00
Alexander Tokmakov
4f54fadd2c
Update clickhouse-test 2022-04-28 20:16:01 +03:00
Alexander Tokmakov
07925ae309
Update clickhouse-test 2022-04-28 17:29:53 +03:00
Azat Khuzhin
c2c8b5cd7e clickhouse-test: improve left queries after the test hardening
Right now it is possible to have "false-positive" for this hardening,
because there is a tiny delay (that can be quite significant on CI, when
it is under pressure) between when the server sends EndOfStream and the
server removes an entry from system.processes.

But now system.processes has is_all_data_sent column, that means that
the EndOfStream was sent, and we can use it to avoid false positive.

Here is an example of such report [1]:

    2022-04-25 03:47:18 00806_alter_update:                                                     [ FAIL ] 0.95 sec. - Queries left in background after the test finished:
    2022-04-25 03:47:18         "elapsed": 0.100084746,
    2022-04-25 03:47:18         "is_cancelled": 0,
    2022-04-25 03:47:18         "query": "DROP TABLE alter_update_00806;",
    2022-04-25 03:47:18         "thread_ids": [
    2022-04-25 03:47:18             "8950"
    2022-04-25 03:47:18         ],

    2022.04.25 03:47:17.887095 [ 8950 ] {7c062004-4c22-486c-934a-f405846e2c81} <Debug> executeQuery: (from [::1]:52012) (comment: 00806_alter_update.sql) DROP TABLE alter_update_00806;
    2022.04.25 03:47:17.887493 [ 8950 ] {7c062004-4c22-486c-934a-f405846e2c81} <Trace> ContextAccess (default): Access granted: DROP TABLE ON test_7ntsjn.alter_update_00806
    2022.04.25 03:47:17.887765 [ 8950 ] {7c062004-4c22-486c-934a-f405846e2c81} <Trace> test_7ntsjn.alter_update_00806 (1bc92bca-10a7-444e-be5e-7f61f4650169): Found 2 old parts to remove.
    2022.04.25 03:47:17.887947 [ 8950 ] {7c062004-4c22-486c-934a-f405846e2c81} <Debug> test_7ntsjn.alter_update_00806 (1bc92bca-10a7-444e-be5e-7f61f4650169): Removing part from filesystem 20180101_20180101_1_1_0
    2022.04.25 03:47:17.888960 [ 8950 ] {7c062004-4c22-486c-934a-f405846e2c81} <Debug> test_7ntsjn.alter_update_00806 (1bc92bca-10a7-444e-be5e-7f61f4650169): Removing part from filesystem 20180102_20180102_2_2_0
    2022.04.25 03:47:17.890620 [ 8950 ] {7c062004-4c22-486c-934a-f405846e2c81} <Debug> DatabaseCatalog: Waiting for table 1bc92bca-10a7-444e-be5e-7f61f4650169 to be finally dropped
    2022.04.25 03:47:17.895046 [ 8950 ] {7c062004-4c22-486c-934a-f405846e2c81} <Debug> MemoryTracker: Peak memory usage (for query): 0.00 B.
    ...
    2022.04.25 03:47:17.938328 [ 4422 ] {aa01985a-78f5-4c0e-b646-8d04a4a1dc77} <Debug> executeQuery: (from [::1]:59416) (comment: 00806_alter_update.sql) DROP DATABASE test_7ntsjn
    2022.04.25 03:47:17.938667 [ 4422 ] {aa01985a-78f5-4c0e-b646-8d04a4a1dc77} <Trace> ContextAccess (default): Access granted: DROP DATABASE ON test_7ntsjn.*
    ...
    2022.04.25 03:47:18.154847 [ 8950 ] {} <Debug> TCPHandler: Processed in 0.269358257 sec.
    2022.04.25 03:47:18.154991 [ 8950 ] {} <Debug> TCPHandler: Done processing connection.
    2022.04.25 03:47:18.155181 [ 8950 ] {} <Debug> TCP-Session: e1d8176a-ee62-4e0a-9855-fe9eb52e06dc Destroying unnamed session of user 94309d50-4f52-5250-31bd-74fecac179db

  [1]: https://s3.amazonaws.com/clickhouse-test-reports/36319/a646cf76b6d4699f06aea1e8d777edb1ad6fd2c5/stateless_tests__debug__actions__[1/3]/runlog.log

So as you can see here DROP TABLE was captured when elapsed was 0.1,
while TCPHandler processes it for 0.26 seconds.
Also from the same report you are seeing that DROP DATABASE was executed
before TCPHandler stopoped processing DROP TABLE.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-04-26 12:15:00 +03:00
Azat Khuzhin
5d49bf1b81 clickhouse-test: fix left-queries-check, to fix test log parser
Before:

    00170_s3_cache:
    Found queries left in processlist after running 00170_s3_cache.sql (database=test_8zpbah):
    [
        ...
    ]
    2022-03-27 00:11:27 [ FAIL ] 600.06 sec. - Timeout!

After:

    02263_test:                                                             [ FAIL ] - Queries left in background after the test finished:
    [
        ...
    ]
    Settings used in the test: --max_insert_threads=11 --group_by_two_level_threshold=100000 --group_by_two_level_threshold_bytes=50000000 --distributed_aggregation_memory_efficient=0 --fsync_metadata=0 --priority=2 --output_format_parallel_formatting=1 --input_format_parallel_parsing=0 --min_chunk_bytes_for_parallel_parsing=12372842 --max_read_buffer_size=855730 --prefer_localhost_replica=1 --max_block_size=74323 --max_threads=56

    Database: default

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-04-20 21:31:39 +03:00
Alexey Milovidov
aa93f4a074 Do not randomize "priority" setting 2022-04-17 20:48:02 +02:00
tavplubix
7af954cef9
Update clickhouse-test 2022-04-13 14:37:44 +03:00
tavplubix
82713f347b
Update clickhouse-test 2022-04-12 12:55:57 +03:00
Alexander Tokmakov
7f54e7b422 Merge branch 'master' into mvcc_prototype 2022-04-07 15:14:06 +02:00
Alexander Tokmakov
8290ffa88d Merge branch 'master' into mvcc_prototype 2022-04-07 13:50:42 +02:00
Kruglov Pavel
80503b2983
Update clickhouse-test 2022-04-07 13:16:19 +02:00
tavplubix
5af0537f2c
Update clickhouse-test 2022-04-06 19:23:33 +03:00
kssenii
d59d4eda4f Fix tests 2022-04-06 01:58:16 +02:00
kssenii
f408c86a13 Use FATAL logs level as default for clickhouse-test 2022-04-05 00:42:29 +02:00
kssenii
3fd462bc05 Merge master 2022-04-05 00:06:18 +02:00
Alexander Tokmakov
5a50ad9de3 Merge branch 'master' into mvcc_prototype 2022-03-31 11:35:04 +02:00
alesapin
5a55913f45
Merge pull request #35720 from ClickHouse/parallel_stateful
Try to run stateful tests in parallel
2022-03-30 23:42:44 +02:00
Azat Khuzhin
7d140c9a59 clickhouse-test: increase timeout for obtaining processlist
Sometimes 30 seconds is not enough on CI [1]:

    2022.03.29 04:46:09.805406 [ 3885 ] {ae06ad04-f9fa-44d4-adeb-9e91aaf9bfa9} <Debug> executeQuery: (from [::1]:33506)  SELECT * FROM system.processes WHERE query NOT LIKE '%system.processes%' AND Settings['log_comment'] = '02016_order_by_with_fill_monotonic_functions_removal.sql' AND current_database = 'test_7fqp6w'
    2022.03.29 04:46:09.866100 [ 3885 ] {ae06ad04-f9fa-44d4-adeb-9e91aaf9bfa9} <Trace> ContextAccess (default): Access granted: SELECT(is_initial_query, user, query_id, address, port, initial_user, initial_query_id, initial_address, initial_port, interface, os_user, client_hostname, client_name, client_revision, client_version_major, client_version_minor, client_version_patch, http_method, http_user_agent, http_referer, forwarded_for, quota_key, distributed_depth, elapsed, is_cancelled, read_rows, read_bytes, total_rows_approx, written_rows, written_bytes, memory_usage, peak_memory_usage, query, thread_ids, ProfileEvents, Settings, current_database) ON system.processes
    2022.03.29 04:46:12.787395 [ 3885 ] {ae06ad04-f9fa-44d4-adeb-9e91aaf9bfa9} <Trace> InterpreterSelectQuery: FetchColumns -> Complete
    2022.03.29 04:46:19.749163 [ 3885 ] {ae06ad04-f9fa-44d4-adeb-9e91aaf9bfa9} <Test> ParallelFormattingOutputFormat: Parallel formatting is being used
    2022.03.29 04:46:37.923282 [ 3885 ] {ae06ad04-f9fa-44d4-adeb-9e91aaf9bfa9} <Information> executeQuery: Read 15 rows, 44.20 KiB in 28.117172383 sec., 0 rows/sec., 1.57 KiB/sec.
    2022.03.29 04:46:40.020586 [ 3885 ] {ae06ad04-f9fa-44d4-adeb-9e91aaf9bfa9} <Debug> DynamicQueryHandler: Done processing query
    2022.03.29 04:46:40.033535 [ 3885 ] {ae06ad04-f9fa-44d4-adeb-9e91aaf9bfa9} <Debug> MemoryTracker: Peak memory usage (for query): 4.00 MiB.

  [1]: https://s3.amazonaws.com/clickhouse-test-reports/32928/ddd5bebe555ce8feebcdd339e47fc45184c20dd1/stateless_tests__thread__actions__[1/3].html

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-03-30 11:05:59 +03:00
Alexander Tokmakov
6191eaa983 fix some flaky tests 2022-03-29 23:12:06 +02:00
Alexander Tokmakov
208b242188 Merge branch 'master' into mvcc_prototype 2022-03-28 19:58:06 +02:00
tavplubix
5e641e2dac
Merge pull request #35675 from ClickHouse/tavplubix-patch-2
Fix randomization of max_read_buffer_size
2022-03-28 20:45:27 +03:00
tavplubix
f10930c9c8
Update clickhouse-test 2022-03-28 16:33:01 +03:00
Alexander Tokmakov
e795b83148 disable random settings in fast test 2022-03-28 14:56:23 +02:00
Alexander Tokmakov
bcec53aadc Merge branch 'master' into mvcc_prototype 2022-03-23 11:28:43 +01:00
mergify[bot]
2543e25b02
Merge branch 'master' into random-settings 2022-03-22 20:00:47 +00:00
Vladimir C
ac6fef5c70
Merge pull request #35124 from vdimir/ci-bugfix-validation 2022-03-22 17:34:15 +01:00
Alexander Tokmakov
3c762f566d Merge branch 'master' into mvcc_prototype 2022-03-21 20:16:29 +01:00
mergify[bot]
f22fddc3ec
Merge branch 'master' into random-settings 2022-03-21 11:52:21 +00:00
Kruglov Pavel
fbcc27a339
Merge pull request #27928 from Avogar/stress-test
Add backward compatibility check in stress test
2022-03-21 12:49:46 +01:00
Alexander Tokmakov
9e05b12d2c Merge branch 'master' into mvcc_prototype 2022-03-20 22:42:26 +01:00
kssenii
d136916508 Remove testmode option 2022-03-17 10:54:37 +01:00
Alexander Tokmakov
278d779a01 log cleanup, more comments 2022-03-14 21:43:34 +01:00
avogar
b9496af3b3 Fix tests 2022-03-14 16:40:17 +00:00
vdimir
000a31df3b
no-random-settings for bugfix validate 2022-03-10 16:43:43 +00:00
avogar
b1a63cd727 Merge branch 'master' of github.com:ClickHouse/ClickHouse into random-settings 2022-03-09 17:09:32 +00:00
mergify[bot]
78c8029aeb
Merge branch 'master' into stress-test 2022-03-09 16:22:45 +00:00
Azat Khuzhin
8b64b41e65 tests: add check of left queries after each test
Look at log_comment/current_database for this check under --no-left-queries-check

v2: add no-left-queries-check tag support
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-03-09 17:29:32 +03:00
Azat Khuzhin
9a297ce3b6 tests: fix log_comment (extra quotes)
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-03-09 17:29:32 +03:00
avogar
722e0ea214 Fix clickhouse-test 2022-03-05 16:46:14 +00:00
Kruglov Pavel
607ec6c884
Fix style 2022-03-05 12:05:41 +03:00
Kruglov Pavel
dbba752398
Merge branch 'master' into stress-test 2022-03-04 18:33:37 +03:00
avogar
05b4bbeceb Add next batch of random settings in functional tests 2022-03-04 11:03:13 +00:00
mergify[bot]
c90d24e10b
Merge branch 'master' into random-settings 2022-03-03 13:07:12 +00:00
avogar
d25d83580a Fix tests 2022-03-02 16:33:21 +00:00
vdimir
fc5e941f79
Add option to clickhouse-test to skip aarch64 build 2022-03-02 14:18:24 +00:00
avogar
dbb29dd394 Fix tests 2022-02-28 12:17:44 +00:00
avogar
ba7f723961 Fix comments 2022-02-18 13:54:21 +00:00
avogar
90603ae549 Randomize just first batch of settings 2022-02-18 12:22:06 +00:00
Kruglov Pavel
e77319bf01
Merge branch 'master' into stress-test 2022-02-17 20:28:46 +03:00
avogar
48d036d3f9 Fix next batch of tests 2022-02-17 11:10:38 +00:00
Kruglov Pavel
33e4c703a1
Merge branch 'master' into random-settings 2022-02-16 13:38:15 +03:00
avogar
be5efc33d4 Fix more settings, add random settings for .sh tests 2022-02-16 10:35:46 +00:00
avogar
880780c7a8 Fix some tests 2022-02-11 17:15:56 +03:00
Kruglov Pavel
33c3b2d856
Better message 2022-02-08 18:38:41 +03:00
alesapin
18f08ed932
Revert "Revert "Add func tests run with s3"" 2022-02-01 10:20:06 +03:00
alexey-milovidov
cbfcd45be3
Revert "Add func tests run with s3" 2022-02-01 05:46:13 +03:00
alesapin
b6828f1b4a Supress some tests for S3 2022-01-31 19:09:25 +03:00
avogar
4bd6cd0b80 Add more settings, print settings if tests fails, add random settings to CLICKHOUSE_URL_PARAMS 2022-01-28 16:46:00 +03:00
Kruglov Pavel
1cdf49efed
More random settings
Co-authored-by: tavplubix <tavplubix@gmail.com>
2022-01-28 15:26:55 +03:00
avogar
4a005579cc Randomize some settings in functional tests 2022-01-28 14:26:50 +03:00
Kruglov Pavel
f3f414ae75
Fix style 2022-01-27 13:59:39 +03:00
Kruglov Pavel
b54bf18b27
Merge branch 'master' into stress-test 2022-01-25 14:35:42 +03:00
vdimir
e5613fce3c Support USE_* build flags in tests' tags 2022-01-19 12:21:51 +00:00
Kruglov Pavel
ffb6c5b37b
Merge branch 'master' into stress-test 2021-12-23 20:14:15 +03:00
Azat Khuzhin
b3f1951c15 clickhouse-test: avoid failing with UNKNOWN status in some cases
Like in:
- CREATE/DROP DATABASE failure
- Connection refused failure

Fixes: #32502
2021-12-12 18:48:54 +03:00
alexey-milovidov
8fdd7f329e
Merge pull request #32533 from azat/clickhouse-test-log-comment
clickhouse-test: use basename of the test for *.sh tests
2021-12-11 02:53:57 +03:00
Azat Khuzhin
2045a4f245 clickhouse-test: use basename of the test for *.sh tests 2021-12-10 21:45:29 +03:00
alesapin
a6e55b00e3
Revert "Revert "Split long tests into multiple checks"" 2021-12-10 18:39:02 +03:00
alesapin
dcbba460c1
Revert "Split long tests into multiple checks" 2021-12-10 18:38:42 +03:00
alesapin
8f8f65e6e0 Fix clickhouse test 2021-12-10 13:30:12 +03:00
alesapin
4c1babee05 Split long functional tests to multiple checks 2021-12-10 12:07:24 +03:00
Azat Khuzhin
d68d01988e clickhouse-test: apply --client-option in get_stacktraces_from_clickhouse() 2021-12-06 09:02:56 +03:00
Azat Khuzhin
9276384977 clickhouse-test: do not use random generator with shared state
Recently (#32094) test database had been overlapped, and random prefix
for database had been increased from 6 to 8.

But actually 6 bytes for random prefix should be enough (with existing
alphabet (0-9a-z) it is 36**6=2'176'782'336), and the real reason of
this overlap is that random generator by default uses shared state [1]:

    The functions supplied by this module are actually bound methods of
    a hidden instance of the random.Random class. You can instantiate your
    own instances of Random to get generators that don’t share state.

  [1]: https://docs.python.org/3/library/random.html

I've played a little bit with random in python, and using default random
generator it generates non-unique strings pretty fast, just in a few
runs, but using SystemRandom (that uses /dev/urandom) it takes ~1 minute.

Test:

```sh
    $ while /tmp/test.py | LANG=c sort -S5G | LANG=c uniq -d | tee /dev/stderr | wc -l | fgrep -q -x -c 0; do :; done
```

```python
    #!/usr/bin/env python3

    import multiprocessing
    import string
    import random

    def random_str(length=6):
        alphabet = string.ascii_lowercase + string.digits
        return ''.join(random.SystemRandom().choice(alphabet) for _ in range(length))

    def worker(_):
        print(random_str())

    with multiprocessing.Pool(processes=2) as pool:
        pool.map(worker, range(0, int(10e3)))
```

So let's switch to SystemRandom and use 6-byte prefix.
2021-12-03 01:29:23 +03:00
mergify[bot]
9c016d4e25
Merge branch 'master' into stress-test 2021-12-02 12:02:44 +00:00
Azat Khuzhin
cc8ebff212 Increase length of random database in clickhouse-test
In [1] there was one overlap with 6-byte prefix:

    $ pigz -cd clickhouse-server2.log.gz | fgrep  test_8a50uz | fgrep 'CREATE DATABASE' | fgrep comment | cut -d' ' -f7-
    <Debug> executeQuery: (from 0.0.0.0:0, user: ) (comment: 01018_insert_multiple_blocks_with_defaults.sh) /* ddl_entry=query-0000001951 */ CREATE DATABASE test_8a50uz UUID 'bb71f2ea-5ed3-466d-bb71-f2ea5ed3266d' ENGINE = Replicated('/test/clickhouse/db/test_8a50uz', '{shard}', '{replica}')
    <Debug> executeQuery: (from 0.0.0.0:0, user: ) (comment: 00098_k_union_all.sql) /* ddl_entry=query-0000003157 */ CREATE DATABASE test_8a50uz UUID '09cc030b-a4c1-4192-89cc-030ba4c1e192' ENGINE = Replicated('/test/clickhouse/db/test_8a50uz', '{shard}', '{replica}')
    <Error> executeQuery: Code: 82. DB::Exception: Database test_8a50uz already exists. (DATABASE_ALREADY_EXISTS) (version 21.12.1.8932) (from 0.0.0.0:0) (comment: 00098_k_union_all.sql) (in query: /* ddl_entry=query-0000003157 */ CREATE DATABASE test_8a50uz UUID '09cc030b-a4c1-4192-89cc-030ba4c1e192' ENGINE = Replicated('/test/clickhouse/db/test_8a50uz', '{shard}', '{replica}')), Stack trace (when copying this message, always include the lines below):

  [1]: https://clickhouse-test-reports.s3.yandex.net/32019/927b9cb1c6470b6d7cd86b4c3fd2078ff43b57df/functional_stateless_tests_(release,_databasereplicated).html#fail1
2021-12-01 22:39:42 +03:00
mergify[bot]
ce0c112586
Merge branch 'master' into stress-test 2021-11-25 11:09:26 +00:00
Alexey Milovidov
d604cf5573 Remove the infamous "unbundled" build 2021-11-21 13:58:26 +03:00
mergify[bot]
8ef9e61e36
Merge branch 'master' into stress-test 2021-11-03 08:28:43 +00:00
Alexander Tokmakov
d095cfe4c1 disable check for zk session uptime by default 2021-10-29 14:45:09 +03:00