ClickHouse/src/Compression
Azat Khuzhin 98e3a99a88 Do not catch exceptions during final flush in writers destructors
Since this hides real problems, since destructor does final flush and if
it fails, then data will be lost.

One of such examples if MEMORY_LIMIT_EXCEEDED exception, so lock
exceptions from destructors, by using
MemoryTracker::LockExceptionInThread to block these exception, and allow
others (so std::terminate will be called, since this is c++11 with
noexcept for destructors by default).

Here is an example, that leads to empty block in the distributed batch:

    2021.01.21 12:43:18.619739 [ 46468 ] {7bd60d75-ebcb-45d2-874d-260df9a4ddac} <Error> virtual DB::CompressedWriteBuffer::~CompressedWriteBuffer(): Code: 241, e.displayText() = DB::Exception: Memory limit (for user) exceeded: would use 332.07 GiB (attempt to allocate chunk of 4355342 bytes), maximum: 256.00 GiB, Stack trace (when copying this message, always include the lines below):

    0. DB::Exception::Exception<>() @ 0x86f7b88 in /usr/bin/clickhouse
    ...
    4. void DB::PODArrayBase<>::resize<>(unsigned long) @ 0xe9e878d in /usr/bin/clickhouse
    5. DB::CompressedWriteBuffer::nextImpl() @ 0xe9f0296 in /usr/bin/clickhouse
    6. DB::CompressedWriteBuffer::~CompressedWriteBuffer() @ 0xe9f0415 in /usr/bin/clickhouse
    7. DB::DistributedBlockOutputStream::writeToShard() @ 0xf6bed4a in /usr/bin/clickhouse
2021-02-05 01:31:45 +03:00
..
tests remove more stringstreams 2020-11-10 21:22:26 +03:00
CachedCompressedReadBuffer.cpp Add CheckingCompressedReadBuffer 2021-01-10 21:23:42 +03:00
CachedCompressedReadBuffer.h better reading from compact parts with differents codecs 2020-09-04 15:48:55 +03:00
CheckingCompressedReadBuffer.cpp Add CheckingCompressedReadBuffer 2021-01-10 21:23:42 +03:00
CheckingCompressedReadBuffer.h Add CheckingCompressedReadBuffer 2021-01-10 21:23:42 +03:00
CMakeLists.txt
CompressedReadBuffer.cpp Add CheckingCompressedReadBuffer 2021-01-10 21:23:42 +03:00
CompressedReadBuffer.h Fix the issue with async Distributed INSERTs and network_compression_method #18741 2021-01-06 03:24:42 +03:00
CompressedReadBufferBase.cpp Add CheckingCompressedReadBuffer 2021-01-10 21:23:42 +03:00
CompressedReadBufferBase.h Add CheckingCompressedReadBuffer 2021-01-10 21:23:42 +03:00
CompressedReadBufferFromFile.cpp Add CheckingCompressedReadBuffer 2021-01-10 21:23:42 +03:00
CompressedReadBufferFromFile.h better reading from compact parts with differents codecs 2020-09-04 15:48:55 +03:00
CompressedWriteBuffer.cpp Do not catch exceptions during final flush in writers destructors 2021-02-05 01:31:45 +03:00
CompressedWriteBuffer.h
CompressionCodecDelta.cpp Buildable code 2020-09-21 14:24:10 +03:00
CompressionCodecDelta.h Trying to improve performance 2020-09-14 22:15:25 +03:00
CompressionCodecDoubleDelta.cpp Buildable code 2020-09-21 14:24:10 +03:00
CompressionCodecDoubleDelta.h Trying to improve performance 2020-09-14 22:15:25 +03:00
CompressionCodecGorilla.cpp Buildable code 2020-09-21 14:24:10 +03:00
CompressionCodecGorilla.h Trying to improve performance 2020-09-14 22:15:25 +03:00
CompressionCodecLZ4.cpp Trying to improve performance 2020-09-14 22:15:25 +03:00
CompressionCodecLZ4.h Trying to improve performance 2020-09-14 22:15:25 +03:00
CompressionCodecMultiple.cpp Almost working version 2020-09-21 17:22:13 +03:00
CompressionCodecMultiple.h Almost working version 2020-09-21 17:22:13 +03:00
CompressionCodecNone.cpp Trying to improve performance 2020-09-14 22:15:25 +03:00
CompressionCodecNone.h Trying to improve performance 2020-09-14 22:15:25 +03:00
CompressionCodecT64.cpp First working test 2020-09-21 17:47:10 +03:00
CompressionCodecT64.h Remove useless code from BigInt 2021-01-26 21:22:40 +03:00
CompressionCodecZSTD.cpp add zstd long range option (#17184) 2021-01-13 16:22:59 +03:00
CompressionCodecZSTD.h add zstd long range option (#17184) 2021-01-13 16:22:59 +03:00
CompressionFactory.cpp Respect network_compression_method in async INSERT into Distributed table 2021-01-06 03:41:34 +03:00
CompressionFactory.h Update CompressionFactory.h 2020-09-23 10:24:25 +03:00
CompressionInfo.h
getCompressionCodecForFile.cpp Better interface 2020-08-28 12:07:20 +03:00
getCompressionCodecForFile.h Better interface 2020-08-28 12:07:20 +03:00
ICompressionCodec.cpp Whitespace 2021-01-25 00:23:06 +03:00
ICompressionCodec.h Almost working version 2020-09-21 17:22:13 +03:00
LZ4_decompress_faster.cpp another removes 2020-07-30 13:31:14 +03:00
LZ4_decompress_faster.h
ya.make Add CheckingCompressedReadBuffer 2021-01-10 21:23:42 +03:00
ya.make.in Follow Arcadia ya.make rules 2020-11-16 21:24:58 +03:00