Fix leftover processes/hangs in tests

One of such cases is 02479_race_condition_between_insert_and_droppin_mv
[1], yes it can be fixed (by using fixed number of iterations, or with
some bash trickery), but it is better to fix them completelly,
eventually such tests will be submitted and pass review anyway.

By allocating process group for each test we can kill all the processes
in this process group, and this what this patch does.

This will also fix some test hangs (like in [1]) as well as some
possible issues in stress tests.

  [1]: https://s3.amazonaws.com/clickhouse-test-reports/0/e2c1230b00386c4d0096a245396ab3be7ce60950/stateless_tests__release__analyzer_/run.log

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
(cherry picked from commit 72fa58e192)
This commit is contained in:
Azat Khuzhin 2023-12-23 15:42:49 +01:00
parent f459a2b0c9
commit 1fdee81ad5
2 changed files with 10 additions and 2 deletions

View File

@ -1031,7 +1031,7 @@ class TestCase:
if proc: if proc:
if proc.returncode is None: if proc.returncode is None:
try: try:
proc.kill() os.killpg(os.getpgid(proc.pid), signal.SIGKILL)
except OSError as e: except OSError as e:
if e.errno != ESRCH: if e.errno != ESRCH:
raise raise
@ -1307,7 +1307,7 @@ class TestCase:
command = pattern.format(**params) command = pattern.format(**params)
proc = Popen(command, shell=True, env=os.environ) proc = Popen(command, shell=True, env=os.environ, start_new_session=True)
while ( while (
datetime.now() - start_time datetime.now() - start_time

View File

@ -185,3 +185,11 @@ function query_with_retry
done done
echo "Query '$query' failed with '$result'" echo "Query '$query' failed with '$result'"
} }
# Add --foreground to avoid running setpgid() in timeout, otherwise
# clickhouse-test will not kill those processes in case of timeout
function timeout()
{
command timeout --foreground "$@"
}
export -f timeout