thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-01 12:01:58 +00:00
Code Issues Packages Projects Releases Wiki Activity

clickhouse-test: improve left queries after the test hardening

Browse Source 
Right now it is possible to have "false-positive" for this hardening,
because there is a tiny delay (that can be quite significant on CI, when
it is under pressure) between when the server sends EndOfStream and the
server removes an entry from system.processes.

But now system.processes has is_all_data_sent column, that means that
the EndOfStream was sent, and we can use it to avoid false positive.

Here is an example of such report [1]:

    2022-04-25 03:47:18 00806_alter_update:                                                     [ FAIL ] 0.95 sec. - Queries left in background after the test finished:
    2022-04-25 03:47:18         "elapsed": 0.100084746,
    2022-04-25 03:47:18         "is_cancelled": 0,
    2022-04-25 03:47:18         "query": "DROP TABLE alter_update_00806;",
    2022-04-25 03:47:18         "thread_ids": [
    2022-04-25 03:47:18             "8950"
    2022-04-25 03:47:18         ],

    2022.04.25 03:47:17.887095 [ 8950 ] {7c062004-4c22-486c-934a-f405846e2c81} <Debug> executeQuery: (from [::1]:52012) (comment: 00806_alter_update.sql) DROP TABLE alter_update_00806;
    2022.04.25 03:47:17.887493 [ 8950 ] {7c062004-4c22-486c-934a-f405846e2c81} <Trace> ContextAccess (default): Access granted: DROP TABLE ON test_7ntsjn.alter_update_00806
    2022.04.25 03:47:17.887765 [ 8950 ] {7c062004-4c22-486c-934a-f405846e2c81} <Trace> test_7ntsjn.alter_update_00806 (1bc92bca-10a7-444e-be5e-7f61f4650169): Found 2 old parts to remove.
    2022.04.25 03:47:17.887947 [ 8950 ] {7c062004-4c22-486c-934a-f405846e2c81} <Debug> test_7ntsjn.alter_update_00806 (1bc92bca-10a7-444e-be5e-7f61f4650169): Removing part from filesystem 20180101_20180101_1_1_0
    2022.04.25 03:47:17.888960 [ 8950 ] {7c062004-4c22-486c-934a-f405846e2c81} <Debug> test_7ntsjn.alter_update_00806 (1bc92bca-10a7-444e-be5e-7f61f4650169): Removing part from filesystem 20180102_20180102_2_2_0
    2022.04.25 03:47:17.890620 [ 8950 ] {7c062004-4c22-486c-934a-f405846e2c81} <Debug> DatabaseCatalog: Waiting for table 1bc92bca-10a7-444e-be5e-7f61f4650169 to be finally dropped
    2022.04.25 03:47:17.895046 [ 8950 ] {7c062004-4c22-486c-934a-f405846e2c81} <Debug> MemoryTracker: Peak memory usage (for query): 0.00 B.
    ...
    2022.04.25 03:47:17.938328 [ 4422 ] {aa01985a-78f5-4c0e-b646-8d04a4a1dc77} <Debug> executeQuery: (from [::1]:59416) (comment: 00806_alter_update.sql) DROP DATABASE test_7ntsjn
    2022.04.25 03:47:17.938667 [ 4422 ] {aa01985a-78f5-4c0e-b646-8d04a4a1dc77} <Trace> ContextAccess (default): Access granted: DROP DATABASE ON test_7ntsjn.*
    ...
    2022.04.25 03:47:18.154847 [ 8950 ] {} <Debug> TCPHandler: Processed in 0.269358257 sec.
    2022.04.25 03:47:18.154991 [ 8950 ] {} <Debug> TCPHandler: Done processing connection.
    2022.04.25 03:47:18.155181 [ 8950 ] {} <Debug> TCP-Session: e1d8176a-ee62-4e0a-9855-fe9eb52e06dc Destroying unnamed session of user 94309d50-4f52-5250-31bd-74fecac179db

  [1]: https://s3.amazonaws.com/clickhouse-test-reports/36319/a646cf76b6d4699f06aea1e8d777edb1ad6fd2c5/stateless_tests__debug__actions__[1/3]/runlog.log

So as you can see here DROP TABLE was captured when elapsed was 0.1,
while TCPHandler processes it for 0.26 seconds.
Also from the same report you are seeing that DROP DATABASE was executed
before TCPHandler stopoped processing DROP TABLE.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
This commit is contained in:
Azat Khuzhin 2022-04-26 09:20:17 +03:00
parent 6339a48923
commit c2c8b5cd7e
1 changed files with 2 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
tests/clickhouse-test
View File

@ -225,6 +225,7 @@ def get_processlist_after_test(args):
FROM clusterAllReplicas('test_cluster_database_replicated', system.processes) FROM clusterAllReplicas('test_cluster_database_replicated', system.processes)
WHERE WHERE
query NOT LIKE '%system.processes%' AND query NOT LIKE '%system.processes%' AND
NOT is_all_data_sent AND
Settings['log_comment'] = '{log_comment}' AND Settings['log_comment'] = '{log_comment}' AND
current_database = '{database}' current_database = '{database}'
""") """)
@ -234,6 +235,7 @@ def get_processlist_after_test(args):
FROM system.processes FROM system.processes
WHERE WHERE
query NOT LIKE '%system.processes%' AND query NOT LIKE '%system.processes%' AND
NOT is_all_data_sent AND
Settings['log_comment'] = '{log_comment}' AND Settings['log_comment'] = '{log_comment}' AND
current_database = '{database}' current_database = '{database}'
""") """)