thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-11-21 15:12:02 +00:00
Code Issues Packages Projects Releases Wiki Activity

Fix 00956_sensitive_data_masking flackiness

Browse Source 
00956_sensitive_data_masking is still flacky even after #13748 [1].

    [1]: https://clickhouse-test-reports.s3.yandex.net/10373/348ef1256ea8fb8f61109c33bbdd28daf46bdc8e/functional_stateless_tests_(debug).html#fail1

The problem is that it uses the following pattern:

    clickhouse-client -q ... & # run some query in background
    clickhouse-client -q 'show processlist' > log
    grep background-query log

But there is no guarantee that the query in background will be executed before `show processlist`:

    2020.08.18 02:52:47.916386 [ 26788 ] {98c36d38-f710-4dfb-af8f-61906abc163c} <Debug> executeQuery: (from [::1]:51650) SHOW PROCESSLIST
    ...
    2020.08.18 02:52:47.926854 [ 26756 ] {086c64fa-713b-4f8c-b702-23bfea10a49c} <Debug> executeQuery: (from [::1]:51652) select count() from system.numbers where ignore('find_me_[hidden]')=0 and ignore('fwerkh_that_magic_string_make_me_unique') = 0 FORMAT Null

Fix the test by waiting until the query in backgroud will start, and use
limited numbers + sleepEachRow over system.numbers to reduce CPU usage.
This commit is contained in:
Azat Khuzhin 2020-08-18 21:53:08 +03:00
parent 793fd5bfb1
commit 3ba9e9f6ce
1 changed files with 9 additions and 5 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

14
tests/queries/0_stateless/00956_sensitive_data_masking.sh
View File

@ -55,15 +55,19 @@ echo 5
# run in background
rm -f "$tmp_file2" >/dev/null 2>&1
bash -c "$CLICKHOUSE_CLIENT \
--query=\"select count() from system.numbers where ignore('find_me_TOPSECRET=TOPSECRET')=0 and ignore('fwerkh_that_magic_string_make_me_unique') = 0 FORMAT Null\" \
--query=\"select sleepEachRow(1) from numbers(10) where ignore('find_me_TOPSECRET=TOPSECRET')=0 and ignore('fwerkh_that_magic_string_make_me_unique') = 0 FORMAT Null\" \
--log_queries=1 --ignore-error --multiquery >$tmp_file2 2>&1" &
# $CLICKHOUSE_CLIENT --query='SHOW PROCESSLIST'
rm -f "$tmp_file" >/dev/null 2>&1
echo '5.1'
# check that executing query doesn't expose secrets in processlist
$CLICKHOUSE_CLIENT --query="SHOW PROCESSLIST" --log_queries=0 >"$tmp_file" 2>&1
echo '5.1'
# wait until the query in background will start (max: 10 seconds as sleepEachRow)
for _ in {1..100}; do
$CLICKHOUSE_CLIENT --query="SHOW PROCESSLIST" --log_queries=0 >"$tmp_file" 2>&1
grep -q -F 'fwerkh_that_magic_string_make_me_unique' "$tmp_file" && break
sleep 0.1
done
$CLICKHOUSE_CLIENT --query="KILL QUERY WHERE query LIKE '%fwerkh_that_magic_string_make_me_unique%'" > /dev/null 2>&1
wait