Azat Khuzhin 4909d3f2e0 Fix endless loop for truncated bzip2 ... Stress test (thread) found an issue [1]: elapsed: 2094.356407533 query: SELECT count(), max(x) FROM file thread_ids: [2927,11494,7502] current_database: test_19 Stacktrace for 7502: 0 __syscall () at ../base/glibc-compatibility/musl/x86_64/syscall.s:14 1 0x000000001ba033b8 in clock_nanosleep (clk=1, flags=1, req=0x7fbb8adaed68, rem=0x0) at ../base/glibc-compatibility/musl/clock_nanosleep.c:21 2 0x0000000018e25b84 in sleepForNanoseconds (nanoseconds=<optimized out>) at ../base/common/sleep.cpp:48 3 0x0000000009a47143 in DB::injection (yield_probability=0, migrate_probability=0, sleep_probability=0.10000000000000001, sleep_time_us=100000) at ../src/Common/ThreadFuzzer.cpp:193 4 DB::ThreadFuzzer::signalHandler () at ../src/Common/ThreadFuzzer.cpp:204 5 0x00000000098e1447 in __tsan::CallUserSignalHandler(__tsan::ThreadState*, bool, bool, bool, int, __sanitizer::__sanitizer_siginfo*, void*) () 6 0x00000000098d7f99 in __tsan::ProcessPendingSignals(__tsan::ThreadState*) () 7 0x00000000098e57de in pread () 8 0x00000000099fab84 in DB::ReadBufferFromFileDescriptor::nextImpl (this=0x7b38010ddb00) at ../src/IO/ReadBufferFromFileDescriptor.cpp:62 9 0x0000000009ab0cf3 in DB::ReadBuffer::next (this=0x7b38010ddb00) at ../src/IO/ReadBuffer.h:62 10 DB::ReadBuffer::nextIfAtEnd (this=0x7b38010ddb00) at ../src/IO/ReadBuffer.h:78 11 DB::Bzip2ReadBuffer::nextImpl (this=0x7b20032f4800) at ../src/IO/Bzip2ReadBuffer.cpp:59 It looks like bz2 is not capable to read/fail partially written archive: 2021.09.03 00:52:36.744818 [ 3553 ] {90419c0a-c410-42ea-ba76-7672ff5ae5ec} <Debug> executeQuery: (from [::1]:54714) (comment: '/usr/share/clickhouse-test/queries/0_stateless/01059_storage_file_compression.sh') CREATE TABLE file (x UInt64) ENGINE = File(TSV, 'test_19/bz2.tsv.bz2') 2021.09.03 00:52:41.255409 [ 2927 ] {9c7b4e21-cc37-438d-9c13-f9dc01efd30c} <Debug> executeQuery: (from [::1]:54750) (comment: '/usr/share/clickhouse-test/queries/0_stateless/01059_storage_file_compression.sh') TRUNCATE TABLE file 2021.09.03 00:52:43.762791 [ 2927 ] {82857e3d-f7cc-453d-82a4-e8b8fb82ae0e} <Debug> executeQuery: (from [::1]:54758) (comment: '/usr/share/clickhouse-test/queries/0_stateless/01059_storage_file_compression.sh') INSERT INTO file SELECT * FROM numbers(1000000) 2021.09.03 00:52:54.939333 [ 2927 ] {82857e3d-f7cc-453d-82a4-e8b8fb82ae0e} <Error> executeQuery: Code: 241. DB::Exception: Memory limit (total) exceeded: would use 63.07 GiB (attempt to allocate chunk of 4194364 bytes), maximum: 62.94 GiB. (MEMORY_LIMIT_EXCEEDED) (version 21.10.1.7982) (from [::1]:54758) (comment: '/usr/share/clickhouse-test/queries/0_stateless/01059_storage_file_compression.sh') (in query: INSERT INTO file SELECT * FROM numbers(1000000)), Stack trace (when copying this message, always include the lines below): 2021.09.03 00:53:08.505015 [ 2927 ] {2bc7444e-c0d6-462e-9d6a-567f70b01697} <Debug> executeQuery: (from [::1]:54842) (comment: '/usr/share/clickhouse-test/queries/0_stateless/01059_storage_file_compression.sh') SELECT count(), max(x) FROM file [1]: https://clickhouse-test-reports.s3.yandex.net/28474/6d5f01a56bb1715c47de8444bfc85b39228f3081/stress_test_(thread).html#fail1		2021-09-03 10:49:17 +03:00
..
ci
config	Merge branch 'master' into stress-thread-fuzzer	2021-08-31 11:56:30 +00:00
external_models/catboost
fuzz
instructions
integration	Merge pull request #28503 from kitaisreal/executable-multiple-pipes-added-test	2021-09-02 18:30:03 +03:00
jepsen.clickhouse-keeper
perf_drafts
performance	Merge branch 'master' into async-reads	2021-08-31 02:52:58 +03:00
queries	Fix endless loop for truncated bzip2	2021-09-03 10:49:17 +03:00
testflows
users.d
.gitignore
clickhouse-client.xml
clickhouse-test
client-test.xml
CMakeLists.txt
decimals_dictionary.xml
executable_pool_dictionary.xml
ints_dictionary.xml
msan_suppressions.txt
server-test.xml
stress
strings_dictionary.xml
tsan_suppressions.txt
ubsan_suppressions.txt
users.xml