ClickHouse/src/IO
alesapin 82729a88c8
Merge pull request #42833 from vitlibar/backup-s3-increase-upload-part-size-exponentially
Increase the size of upload part exponentially for backup to S3.
2022-11-04 13:41:10 +01:00
..
Archives Review fixes 2022-10-21 22:40:13 +02:00
examples Review fixes 2022-10-21 22:40:13 +02:00
S3 Merge branch 'master' into keeper-upload-snapshot-to-s3 2022-10-17 11:02:36 +00:00
tests Adapt internal data structures to 512-bit era (#42564) 2022-10-25 13:56:28 +02:00
AIO.cpp Fixes for -Wshorten-64-to-32 2022-10-21 13:25:19 +02:00
AIO.h Fix clang-tidy warnings in Interpreters, IO folders 2022-03-14 18:17:35 +00:00
AsynchronousReadBufferFromFile.cpp Fix threadpool reader (for local fs) 2022-09-23 19:35:16 +02:00
AsynchronousReadBufferFromFile.h Fix threadpool reader (for local fs) 2022-09-23 19:35:16 +02:00
AsynchronousReadBufferFromFileDescriptor.cpp Fix threadpool reader (for local fs) 2022-09-23 19:35:16 +02:00
AsynchronousReadBufferFromFileDescriptor.h Fix threadpool reader (for local fs) 2022-09-23 19:35:16 +02:00
AsynchronousReader.h Get rid of static threadpools 2022-09-07 17:48:11 +02:00
BitHelpers.h Fix Endian issue in BitHelpers 2022-07-27 06:56:04 -07:00
BoundedReadBuffer.cpp Continue 2022-06-13 16:32:07 +02:00
BoundedReadBuffer.h Review fixes 2022-07-17 15:13:28 +02:00
BrotliReadBuffer.cpp Generate config.h into ${CONFIG_INCLUDE_PATH} 2022-09-28 12:48:26 +00:00
BrotliReadBuffer.h show names of erroneous files 2022-04-16 00:10:47 +00:00
BrotliWriteBuffer.cpp Generate config.h into ${CONFIG_INCLUDE_PATH} 2022-09-28 12:48:26 +00:00
BrotliWriteBuffer.h Fix comments 2021-11-22 14:19:26 +03:00
BufferBase.h Update BufferBase.h 2022-02-07 17:49:53 +03:00
BufferWithOwnMemory.h Adapt internal data structures to 512-bit era (#42564) 2022-10-25 13:56:28 +02:00
Bzip2ReadBuffer.cpp Fixes for -Wshorten-64-to-32 2022-10-21 13:25:19 +02:00
Bzip2ReadBuffer.h show names of erroneous files 2022-04-16 00:10:47 +00:00
Bzip2WriteBuffer.cpp Fixes for -Wshorten-64-to-32 2022-10-21 13:25:19 +02:00
Bzip2WriteBuffer.h Fix comments 2021-11-22 14:19:26 +03:00
CascadeWriteBuffer.cpp remove some dead and commented code 2022-07-21 15:05:48 +02:00
CascadeWriteBuffer.h Fix clang-tidy warnings in Interpreters, IO folders 2022-03-14 18:17:35 +00:00
CMakeLists.txt fix unit test and build error 2022-01-06 11:41:24 +08:00
CompressedReadBufferWrapper.h Merge pull request #36650 from bigo-sg/hive_text_parallel_parsing 2022-05-03 15:56:28 +02:00
CompressionMethod.cpp abort instead of __builtin_unreachable in debug builds 2022-10-07 21:49:08 +02:00
CompressionMethod.h Add methods to the tests 2022-10-04 19:04:03 +00:00
ConcatReadBuffer.h Use BackupEntryFromAppendOnlyFile for Log engine, 2022-03-20 20:02:15 +01:00
ConcatSeekableReadBuffer.cpp Fix incremental backups for Log family. 2022-08-31 12:57:28 +02:00
ConcatSeekableReadBuffer.h Fix incremental backups for Log family. 2022-08-31 12:57:28 +02:00
ConnectionTimeouts.h fix my favorite ctor 2022-08-12 16:28:21 +02:00
ConnectionTimeoutsContext.h
copyData.cpp debugging 2022-04-20 19:58:31 +08:00
copyData.h
DoubleConverter.cpp
DoubleConverter.h abort instead of __builtin_unreachable in debug builds 2022-10-07 21:49:08 +02:00
EmptyReadBuffer.h
FileEncryptionCommon.cpp Fixes for -Wshorten-64-to-32 2022-10-21 13:25:19 +02:00
FileEncryptionCommon.h Generate config.h into ${CONFIG_INCLUDE_PATH} 2022-09-28 12:48:26 +00:00
ForkWriteBuffer.cpp Addressed review comments 2022-07-20 21:02:54 +02:00
ForkWriteBuffer.h Addressed review comments 2022-07-20 21:02:54 +02:00
HadoopSnappyReadBuffer.cpp Generate config.h into ${CONFIG_INCLUDE_PATH} 2022-09-28 12:48:26 +00:00
HadoopSnappyReadBuffer.h abort instead of __builtin_unreachable in debug builds 2022-10-07 21:49:08 +02:00
HashingReadBuffer.h Fix incremental backups for Log family. 2022-08-31 12:57:28 +02:00
HashingWriteBuffer.cpp
HashingWriteBuffer.h Fix clang-tidy warnings in Interpreters, IO folders 2022-03-14 18:17:35 +00:00
HTTPChunkedReadBuffer.cpp fix heap buffer overflow by limiting http chunk size 2022-08-18 14:30:52 +02:00
HTTPChunkedReadBuffer.h fix heap buffer overflow by limiting http chunk size 2022-08-18 14:30:52 +02:00
HTTPCommon.cpp Fixes for -Wshorten-64-to-32 2022-10-21 13:25:19 +02:00
HTTPCommon.h Fixes for -Wshorten-64-to-32 2022-10-21 13:25:19 +02:00
IOThreadPool.cpp Address PR comments 2022-03-16 15:45:17 +00:00
IOThreadPool.h Use a shared IO thread pool 2022-03-11 08:33:34 +00:00
IReadableWriteBuffer.h
LimitReadBuffer.cpp Fix UB in LimitReadBuffer in case of unexpected EOF 2021-12-21 08:38:41 +03:00
LimitReadBuffer.h Rename "common" to "base" 2021-10-02 10:13:14 +03:00
LimitSeekableReadBuffer.cpp Use SeekableReadBuffer instead of ReadBuffer in IBackupEntry. 2022-04-25 16:34:33 +02:00
LimitSeekableReadBuffer.h Use SeekableReadBuffer instead of ReadBuffer in IBackupEntry. 2022-04-25 16:34:33 +02:00
Lz4DeflatingWriteBuffer.cpp Merge branch 'master' into fix-lz4 2021-12-23 16:13:26 +03:00
Lz4DeflatingWriteBuffer.h Fix clang-tidy warnings in Interpreters, IO folders 2022-03-14 18:17:35 +00:00
Lz4InflatingReadBuffer.cpp show names of erroneous files 2022-04-16 00:10:47 +00:00
Lz4InflatingReadBuffer.h show names of erroneous files 2022-04-16 00:10:47 +00:00
LZMADeflatingWriteBuffer.cpp Fix comments 2021-11-22 14:19:26 +03:00
LZMADeflatingWriteBuffer.h Fix comments 2021-11-22 14:19:26 +03:00
LZMAInflatingReadBuffer.cpp show names of erroneous files 2022-04-16 00:10:47 +00:00
LZMAInflatingReadBuffer.h show names of erroneous files 2022-04-16 00:10:47 +00:00
MemoryReadWriteBuffer.cpp
MemoryReadWriteBuffer.h Fix clang-tidy warnings in Interpreters, IO folders 2022-03-14 18:17:35 +00:00
MMappedFile.cpp
MMappedFile.h
MMappedFileCache.h Change class LRUCache to class CachBase. Check running CacheBase with default pcahce policy SLRU 2022-08-07 19:59:30 +00:00
MMappedFileDescriptor.cpp Rename "common" to "base" 2021-10-02 10:13:14 +03:00
MMappedFileDescriptor.h Fix clang-tidy warnings in Interpreters, IO folders 2022-03-14 18:17:35 +00:00
MMapReadBufferFromFile.cpp
MMapReadBufferFromFile.h
MMapReadBufferFromFileDescriptor.cpp Adapt internal data structures to 512-bit era (#42564) 2022-10-25 13:56:28 +02:00
MMapReadBufferFromFileDescriptor.h improvements for getFileSize 2022-06-20 15:22:56 +02:00
MMapReadBufferFromFileWithCache.cpp Adapt internal data structures to 512-bit era (#42564) 2022-10-25 13:56:28 +02:00
MMapReadBufferFromFileWithCache.h
MySQLBinlogEventReadBuffer.cpp
MySQLBinlogEventReadBuffer.h
MySQLPacketPayloadReadBuffer.cpp
MySQLPacketPayloadReadBuffer.h
MySQLPacketPayloadWriteBuffer.cpp Fix half of trash 2022-09-10 04:08:16 +02:00
MySQLPacketPayloadWriteBuffer.h Fix clang-tidy warnings in Interpreters, IO folders 2022-03-14 18:17:35 +00:00
NullWriteBuffer.cpp
NullWriteBuffer.h Fix clang-tidy warnings in Interpreters, IO folders 2022-03-14 18:17:35 +00:00
OpenedFile.cpp Improve performance of file descriptor cache by narrowing mutex scopes (#36682) 2022-04-28 14:56:53 +02:00
OpenedFile.h Improve performance of file descriptor cache by narrowing mutex scopes (#36682) 2022-04-28 14:56:53 +02:00
OpenedFileCache.h Fix OpenedFileCacheHits/OpenedFileCacheMisses 2021-11-10 21:15:27 +03:00
Operators.h Fix test failures 2022-07-20 07:59:02 +00:00
ParallelReadBuffer.cpp Get rid of static threadpools 2022-09-07 17:48:11 +02:00
ParallelReadBuffer.h Fix 2022-09-08 12:23:59 +02:00
parseDateTimeBestEffort.cpp fix and update test 2022-10-04 13:36:28 +00:00
parseDateTimeBestEffort.h initial changes to close #37492 2022-08-08 19:45:00 -07:00
PeekableReadBuffer.cpp Fix various clang-tidy warnings 2022-04-20 10:29:05 +02:00
PeekableReadBuffer.h Adapt internal data structures to 512-bit era (#42564) 2022-10-25 13:56:28 +02:00
Progress.cpp Do not write elapsed_ns in HTTP headers 2022-08-10 09:09:04 +02:00
Progress.h Fix error 2022-08-10 09:31:24 +02:00
ReadBuffer.h Maybe fix error 2022-06-19 05:21:59 +02:00
ReadBufferFromEmptyFile.h improvements for getFileSize 2022-06-20 15:22:56 +02:00
ReadBufferFromEncryptedFile.cpp Fix seeking while reading from encrypted disk. 2022-07-29 12:45:29 +02:00
ReadBufferFromEncryptedFile.h Generate config.h into ${CONFIG_INCLUDE_PATH} 2022-09-28 12:48:26 +00:00
ReadBufferFromFile.cpp More consistent use of platform macros 2022-06-10 10:22:31 +02:00
ReadBufferFromFile.h Better 2022-05-09 21:25:26 +02:00
ReadBufferFromFileBase.cpp Fix 2022-06-20 17:19:36 +02:00
ReadBufferFromFileBase.h improvements for getFileSize 2022-06-20 15:22:56 +02:00
ReadBufferFromFileDecorator.cpp Refactoring of object storages, add LocalObjectStorage 2022-07-09 12:39:31 +02:00
ReadBufferFromFileDecorator.h improvements for getFileSize 2022-06-20 15:22:56 +02:00
ReadBufferFromFileDescriptor.cpp Fix clang-tidy warnings (from clang-15) 2022-09-10 21:38:35 +02:00
ReadBufferFromFileDescriptor.h Fix clang-tidy warnings (from clang-15) 2022-09-10 21:38:35 +02:00
ReadBufferFromIStream.cpp
ReadBufferFromIStream.h
ReadBufferFromMemory.cpp Fix various clang-tidy warnings 2022-04-20 10:29:05 +02:00
ReadBufferFromMemory.h Add convenience ctor to ReadBufferFromMemory 2022-10-27 10:26:55 +02:00
ReadBufferFromPocoSocket.cpp Fixes for -Wshorten-64-to-32 2022-10-21 13:25:19 +02:00
ReadBufferFromPocoSocket.h
ReadBufferFromS3.cpp Generate config.h into ${CONFIG_INCLUDE_PATH} 2022-09-28 12:48:26 +00:00
ReadBufferFromS3.h Generate config.h into ${CONFIG_INCLUDE_PATH} 2022-09-28 12:48:26 +00:00
ReadBufferFromString.h refactor metadatacache to reduce using of USE_ROCKSDB 2022-01-05 19:51:50 +08:00
readDecimalText.h Use return type template parameter to select throwing behavior 2022-10-27 10:26:55 +02:00
readFloatText.cpp
readFloatText.h Fixes for -Wshorten-64-to-32 2022-10-21 13:25:19 +02:00
ReadHelpers.cpp Fix possible error Attempt to read after eof in CSV schema inference 2022-08-29 11:18:53 +00:00
ReadHelpers.h fix 2022-10-18 17:03:31 +03:00
ReadSettings.h add DiskS3* profile events per S3 API calls 2022-09-19 18:10:47 +00:00
ReadWriteBufferFromHTTP.h Fix 2022-10-10 17:23:09 +02:00
S3Common.cpp Merge branch 'master' into keeper-upload-snapshot-to-s3 2022-10-17 11:02:36 +00:00
S3Common.h Merge branch 'master' into keeper-upload-snapshot-to-s3 2022-10-17 11:02:36 +00:00
SeekableReadBuffer.h Fixes 2022-06-03 17:03:05 +02:00
SeekAvoidingReadBuffer.cpp Drop unused headers 2021-09-01 02:47:52 +03:00
SeekAvoidingReadBuffer.h Better 2021-09-21 01:51:17 +03:00
SnappyReadBuffer.cpp Generate config.h into ${CONFIG_INCLUDE_PATH} 2022-09-28 12:48:26 +00:00
SnappyReadBuffer.h Generate config.h into ${CONFIG_INCLUDE_PATH} 2022-09-28 12:48:26 +00:00
SnappyWriteBuffer.cpp Generate config.h into ${CONFIG_INCLUDE_PATH} 2022-09-28 12:48:26 +00:00
SnappyWriteBuffer.h Generate config.h into ${CONFIG_INCLUDE_PATH} 2022-09-28 12:48:26 +00:00
SynchronousReader.cpp Better ReadBuffer asynchronicity 2022-02-20 01:08:21 +08:00
SynchronousReader.h Get rid of static threadpools 2022-09-07 17:48:11 +02:00
TimeoutSetter.cpp base should not depend on Common 2022-04-29 10:26:35 +08:00
TimeoutSetter.h Poco::Timespan reference fix 2021-04-29 19:11:20 +03:00
UncompressedCache.h Fix review 2022-08-08 21:34:24 +00:00
UseSSL.cpp Generate config.h into ${CONFIG_INCLUDE_PATH} 2022-09-28 12:48:26 +00:00
UseSSL.h
VarInt.h Fixes for -Wshorten-64-to-32 2022-10-21 13:25:19 +02:00
WithFileName.cpp show names of erroneous files 2022-04-16 00:10:47 +00:00
WithFileName.h add more tests for error diagnostics in files 2022-04-19 15:56:34 +00:00
WithFileSize.cpp Fix 2022-07-04 16:05:08 +02:00
WithFileSize.h improvements for getFileSize 2022-06-20 15:22:56 +02:00
WriteBuffer.h Fix duplicate include 2022-06-28 08:13:51 +00:00
WriteBufferDecorator.h Fix comments 2021-11-22 14:19:26 +03:00
WriteBufferFromArena.h Small refactoring of WriteBiffer-s 2021-11-11 02:11:18 +03:00
WriteBufferFromEncryptedFile.cpp Fix comments 2021-11-22 14:19:26 +03:00
WriteBufferFromEncryptedFile.h Generate config.h into ${CONFIG_INCLUDE_PATH} 2022-09-28 12:48:26 +00:00
WriteBufferFromFile.cpp More consistent use of platform macros 2022-06-10 10:22:31 +02:00
WriteBufferFromFile.h Fix clang-tidy warnings in Interpreters, IO folders 2022-03-14 18:17:35 +00:00
WriteBufferFromFileBase.cpp
WriteBufferFromFileBase.h
WriteBufferFromFileDecorator.cpp Revert "Revert "Add pool to WriteBufferFromS3"" 2022-02-01 13:36:51 +03:00
WriteBufferFromFileDecorator.h Revert "Revert "Add pool to WriteBufferFromS3"" 2022-02-01 13:36:51 +03:00
WriteBufferFromFileDescriptor.cpp Add profile events for fsync 2022-07-31 23:19:30 +03:00
WriteBufferFromFileDescriptor.h Fix clang-tidy warnings in Interpreters, IO folders 2022-03-14 18:17:35 +00:00
WriteBufferFromFileDescriptorDiscardOnFailure.cpp
WriteBufferFromFileDescriptorDiscardOnFailure.h Fix clang-tidy warnings in Interpreters, IO folders 2022-03-14 18:17:35 +00:00
WriteBufferFromHTTP.cpp Fix segment fault when sending data to remote HTTP server via URL table engine 2022-08-24 17:38:16 +08:00
WriteBufferFromHTTP.h Address review comment 2022-02-12 10:46:01 +08:00
WriteBufferFromOStream.cpp Fix tests 2021-11-11 20:27:23 +03:00
WriteBufferFromOStream.h Fix clang-tidy warnings in Interpreters, IO folders 2022-03-14 18:17:35 +00:00
WriteBufferFromPocoSocket.cpp Fixes for -Wshorten-64-to-32 2022-10-21 13:25:19 +02:00
WriteBufferFromPocoSocket.h Fix clang-tidy warnings in Interpreters, IO folders 2022-03-14 18:17:35 +00:00
WriteBufferFromS3.cpp Add max limitation for the size of an uploaded part. 2022-11-02 17:53:54 +01:00
WriteBufferFromS3.h Fixes for -Wshorten-64-to-32 2022-10-21 13:25:19 +02:00
WriteBufferFromString.h Less usage of StringRef 2022-07-24 18:33:52 +00:00
WriteBufferFromTemporaryFile.cpp
WriteBufferFromTemporaryFile.h Fix clang-tidy warnings in Interpreters, IO folders 2022-03-14 18:17:35 +00:00
WriteBufferFromVector.h Fix some clang-tidy warnings in headers 2022-06-27 23:49:07 +02:00
WriteBufferValidUTF8.cpp Fixes for -Wshorten-64-to-32 2022-10-21 13:25:19 +02:00
WriteBufferValidUTF8.h Fix clang-tidy warnings in Interpreters, IO folders 2022-03-14 18:17:35 +00:00
WriteHelpers.cpp Implement support for different UUID binary formats (#42108) 2022-10-25 21:57:50 +02:00
WriteHelpers.h Implement support for different UUID binary formats (#42108) 2022-10-25 21:57:50 +02:00
WriteIntText.h Fix clang-tidy warnings in Interpreters, IO folders 2022-03-14 18:17:35 +00:00
WriteSettings.h Revert "Merge pull request #41681 from ClickHouse/revert-41268-disable-s3-parallel-write-for-part-moves-to-disk-s3" 2022-10-20 19:51:27 +02:00
ZlibDeflatingWriteBuffer.cpp Fixes for -Wshorten-64-to-32 2022-10-21 13:25:19 +02:00
ZlibDeflatingWriteBuffer.h Fix comments 2021-11-22 14:19:26 +03:00
ZlibInflatingReadBuffer.cpp Fixes for -Wshorten-64-to-32 2022-10-21 13:25:19 +02:00
ZlibInflatingReadBuffer.h show names of erroneous files 2022-04-16 00:10:47 +00:00
ZstdDeflatingAppendableWriteBuffer.cpp Fixes for -Wshorten-64-to-32 2022-10-21 13:25:19 +02:00
ZstdDeflatingAppendableWriteBuffer.h Better comments 2022-05-04 18:24:28 +02:00
ZstdDeflatingWriteBuffer.cpp Fixes for -Wshorten-64-to-32 2022-10-21 13:25:19 +02:00
ZstdDeflatingWriteBuffer.h Fix comments 2021-11-22 14:19:26 +03:00
ZstdInflatingReadBuffer.cpp Revert "Revert "Add a setting to use more memory for zstd decompression"" 2022-06-18 15:55:35 +03:00
ZstdInflatingReadBuffer.h Revert "Revert "Add a setting to use more memory for zstd decompression"" 2022-06-18 15:55:35 +03:00