thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-11-17 13:13:36 +00:00
Code Issues Packages Projects Releases Wiki Activity
77,455 Commits 9,138 Branches 1,774 Tags 2.1 GiB
cb6f99fe6f
Commit Graph

1123 Commits

Author SHA1 Message Date
kssenii
7de5fca075 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into retriable-http 2021-10-24 17:04:24 +00:00
Azat Khuzhin
93c34824c0 Fix explicit 0 size for createReadBufferFromFileBase() 2021-10-24 18:58:22 +03:00
Azat Khuzhin
2b96433daf Replace estimated_size with optional<size> for IDisk::readFile() 
v2: rebase to fix conflicts in DiskEncrypted.cpp
 2021-10-24 18:58:11 +03:00
Vitaly Baranov
c0b01e7b0b
Merge pull request #30494 from vitlibar/fix-reading-from-empty-file-on-encrypted-disk 
Fix reading from empty file on encrypted disk
 2021-10-24 16:24:47 +03:00
kssenii
1ecb6ad05d Better 2021-10-23 19:00:14 +00:00
kssenii
39b9e9c258 Merge branch 'master' of github.com:ClickHouse/ClickHouse into disk-async-read 2021-10-22 15:40:41 +03:00
kssenii
56f0560c46 Asserts and read till end option 2021-10-21 20:54:03 +03:00
Vitaly Baranov
172a6f2e0d Fix reading from empty file on encrypted disk. 2021-10-21 19:31:29 +03:00
kssenii
120b01e089 Better 2021-10-21 18:22:17 +03:00
kssenii
e4e157688d Review fixes 2021-10-21 01:57:43 +03:00
alexey-milovidov
d3df1c02bc
Merge pull request #29672 from ClickHouse/clickhouse-obfuscator-2 
More enhancements for query obfuscator
 2021-10-20 23:06:29 +03:00
kssenii
5f24eb10ec Test changes in MergeTreeReader for local fs 2021-10-20 13:52:30 +03:00
kssenii
31cd71c849 Clean up 2021-10-20 09:37:18 +03:00
kssenii
143e912e4e Fix split build 2021-10-19 22:45:46 +03:00
Nikolai Kochetov
a92dc0a826 Update obsolete comments. 2021-10-19 12:58:10 +03:00
kssenii
d5d4817350 Add disk proxies back, add setReadUntilPosition to ReadBuffer 2021-10-18 18:35:11 +03:00
kssenii
710ee96878 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into retriable-http 2021-10-18 08:31:25 +00:00
Azat Khuzhin
0e34a9d550 Add MSan instrumentation for preadv2 
MSan report [1]:

    ==10672==WARNING: MemorySanitizer: use-of-uninitialized-value
        3 0x30dd6005 in DB::CompressionCodecFactory::get(unsigned char) const obj-x86_64-linux-gnu/../src/Compression/CompressionFactory.cpp:105:65
        4 0x30dab4d5 in DB::CompressedReadBufferBase::readCompressedData(unsigned long&, unsigned long&, bool) obj-x86_64-linux-gnu/../src/Compression/CompressedReadBufferBase.cpp:127:53
        5 0x30db54af in DB::CompressedReadBufferFromFile::readBig(char*, unsigned long) obj-x86_64-linux-gnu/../src/Compression/CompressedReadBufferFromFile.cpp:119:38

      Uninitialized value was stored to memory at
        0 0x9ef4e99 in __msan_memcpy (/src/ch/tmp/30191/clickhouse-msan+0x9ef4e99)
        1 0xa0c1999 in DB::ReadBuffer::read() obj-x86_64-linux-gnu/../src/IO/ReadBuffer.h:173:13
        2 0xa0c1999 in DB::ReadBuffer::readStrict() obj-x86_64-linux-gnu/../src/IO/ReadBuffer.h:184:27
        3 0x30daafd1 in DB::CompressedReadBufferBase::readCompressedData() obj-x86_64-linux-gnu/../src/Compression/CompressedReadBufferBase.cpp:120:20
        4 0x30db54af in DB::CompressedReadBufferFromFile::readBig() obj-x86_64-linux-gnu/../src/Compression/CompressedReadBufferFromFile.cpp:119:38

      Uninitialized value was created by a heap allocation
        0 0x9ef5285 in posix_memalign (/src/ch/tmp/30191/clickhouse-msan+0x9ef5285)
        1 0xa0cb98d in Allocator<false, false>::allocNoTrack(unsigned long, unsigned long) obj-x86_64-linux-gnu/../src/Common/Allocator.h:235:27
        2 0xa0caf0d in Allocator<false, false>::alloc(unsigned long, unsigned long) obj-x86_64-linux-gnu/../src/Common/Allocator.h:96:16
        3 0xa0caf0d in DB::Memory<Allocator<false, false> >::alloc() obj-x86_64-linux-gnu/../src/IO/BufferWithOwnMemory.h:126:49
        4 0xa0ca4a1 in DB::Memory<Allocator<false, false> >::Memory() obj-x86_64-linux-gnu/../src/IO/BufferWithOwnMemory.h:43:9
        5 0xa0ca4a1 in DB::BufferWithOwnMemory<DB::SeekableReadBuffer>::BufferWithOwnMemory(unsigned long, char*, unsigned long) obj-x86_64-linux-gnu/../src/IO/BufferWithOwnMemory.h:153:29
        6 0xa0ca4a1 in DB::ReadBufferFromFileBase::ReadBufferFromFileBase() obj-x86_64-linux-gnu/../src/IO/ReadBufferFromFileBase.cpp:11:7
        7 0xa59d980 in DB::AsynchronousReadBufferFromFileDescriptor::AsynchronousReadBufferFromFileDescriptor() obj-x86_64-linux-gnu/../src/IO/AsynchronousReadBufferFromFileDescriptor.h:40:11

    SUMMARY: MemorySanitizer: use-of-uninitialized-value obj-x86_64-linux-gnu/../contrib/libcxx/include/__hash_table:116:10 in std::__1::__constrain_hash(unsigned long, unsigned long)

  [1]: https://clickhouse-test-reports.s3.yandex.net/30191/18bb0e7d8d47dd694390f3f7b2ecc921a167afac/fuzzer_msan/report.html#fail1
 2021-10-17 23:54:58 +03:00
Azat Khuzhin
fb54d6b6e0 BufferWithOwnMemory: take reallocs into account 2021-10-17 23:54:58 +03:00
Azat Khuzhin
348033d8bd BufferWithOwnMemory: make size aligned not capacity 
This will fix pread_fake_async/pread_threadpool with min_bytes_to_use_direct_io
 2021-10-17 23:54:58 +03:00
Azat Khuzhin
9cc1178ebc BufferWithOwnMemory: do not try to align if buffer already aligned 2021-10-17 23:54:57 +03:00
Azat Khuzhin
34810456e1 Fix alignment for prefetch in AsynchronousReadBufferFromFileDescriptor 2021-10-17 23:54:57 +03:00
Alexey Milovidov
c2faf450d1 Fix error 2021-10-17 06:37:51 +03:00
Alexey Milovidov
04d8f611d2 Merge branch 'master' into clickhouse-obfuscator-2 2021-10-17 05:52:13 +03:00
Maksim Kita
de36038733
Merge pull request #30276 from azat/buffer-fwd-decl 
Use forward declaration for Buffer<> in generic headers
 2021-10-16 19:21:38 +03:00
alexey-milovidov
e9af03ef52
Merge pull request #30248 from ClickHouse/musl-4 
Preparation to build with Musl
 2021-10-16 18:44:35 +03:00
Azat Khuzhin
50231460af Use forward declaration for Buffer<> in generic headers 
- changes in ReadHelpers.h -- recompiles 1000 modules
- changes in FormatFactor.h -- recompiles 100 modules
 2021-10-16 12:03:24 +03:00
Alexey Milovidov
41acc52458 Preparation to build with Musl 2021-10-16 00:17:34 +03:00
kssenii
28d2485716 Some fixes 2021-10-15 17:27:26 +00:00
kssenii
759f747080 Fix build 2021-10-15 17:12:34 +03:00
kssenii
ccb02cac56 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into retriable-http 2021-10-15 12:36:30 +00:00
kssenii
881ae8617e Merge branch 'master' of github.com:ClickHouse/ClickHouse into disk-async-read 2021-10-15 15:09:56 +03:00
kssenii
edfb1a5521 Better performance for smaller number of threads 2021-10-15 15:07:39 +03:00
kssenii
c1c574e9ca Better 2021-10-15 13:14:57 +03:00
kssenii
3995506d37 Adjust range reader for remote fs reads 2021-10-15 11:36:26 +03:00
Alexey Milovidov
0ad20e6613 Fix test 2021-10-13 03:14:44 +03:00
Alexey Milovidov
9939e7c1eb Merge branch 'master' into clickhouse-obfuscator-2 2021-10-13 01:16:26 +03:00
kssenii
1c8b1b1133 Better 2021-10-12 09:33:07 +00:00
alesapin
5a0f22fbce Fix resolve host 2021-10-12 11:27:18 +03:00
alesapin
d301e40651 Cache DNS in Poco handler for S3 if no proxy specified 2021-10-11 18:31:09 +03:00
kssenii
448d63a0fc Range read with upper bound offset 2021-10-11 01:53:51 +03:00
kssenii
1f8b449bc5 Lazy seek avoiding 2021-10-11 00:51:43 +03:00
kssenii
6044725baa Revert "http allow to know total bytes to read" 
This reverts commit 4c99f6da51.
 2021-10-10 23:37:10 +03:00
kssenii
eb5cb86271 More profile events 2021-10-10 23:20:00 +03:00
kssenii
4c99f6da51 http allow to know total bytes to read 2021-10-10 22:15:21 +03:00
kssenii
7c570f2e48 Add retries for http read init 2021-10-10 22:13:49 +03:00
kssenii
e3d2942aa6 Correct merge 2021-10-10 20:24:36 +03:00
kssenii
fde6f0507d Merge remote-tracking branch 'origin/retriable-http' into disk-async-read 2021-10-10 20:23:44 +03:00
kssenii
24feb6674c May be move back initialize to constructor 2021-10-08 15:38:33 +00:00
kssenii
40434519e8 Fix disk web 2021-10-08 16:02:11 +03:00
kssenii
f3f820c6e0 HTTP read buffer retries 2021-10-08 15:41:55 +03:00
kssenii
5e63ec4489 Merge branch 'master' of github.com:ClickHouse/ClickHouse into disk-async-read 2021-10-07 19:11:45 +03:00
kssenii
d6e1d0503d Merge branch 'master' of github.com:ClickHouse/ClickHouse into disk-async-read 2021-10-07 16:21:00 +03:00
Nikita Mikhaylov
b31d11478b
Merge pull request #29680 from azat/defines 
Cleanup common defines
 2021-10-04 20:13:29 +03:00
alexey-milovidov
6abb3d4c6d
Merge pull request #29682 from traceon/xcode-13-fix-build 
Fix Xcode 13 build
 2021-10-04 02:34:43 +03:00
Denis Glazachev
ca7b69b0f3 Manipulate with -Wreserved-identifier only if HAS_RESERVED_IDENTIFIER has been detected 2021-10-03 17:42:36 +04:00
Azat Khuzhin
16c72f2107 Remove unused macros from ReadWriteBufferFromHTTP 2021-10-03 14:34:03 +03:00
Azat Khuzhin
6a9dd9828d Move protocol macros into separate header 
Defines.h is a very common header, so lots of modules will be recompiled
on changes.
Move macros for protocol into separate header, this should significantly
decreases number of units to compile on it's changes.
 2021-10-03 14:34:03 +03:00
Alexey Milovidov
9bc32c95c0 Miscellaneous 2021-10-03 10:01:09 +03:00
Alexey Milovidov
0bf597374f More enhancements for query obfuscator 2021-10-03 05:56:32 +03:00
kssenii
1146035f11 Small fix 2021-10-02 17:28:12 +03:00
Alexey Milovidov
fe6b7c77c7 Rename "common" to "base" 2021-10-02 10:13:14 +03:00
kssenii
7ff092712b Better 2021-10-01 18:13:16 +03:00
kssenii
55972db866 Small fix and more comments 2021-10-01 11:38:58 +03:00
kssenii
e4bc81ac78 Better 2021-09-30 21:45:57 +03:00
kssenii
cc9ce13bac Finally all fixed 2021-09-30 15:35:59 +03:00
kssenii
85aae03a26 Fix 2021-09-29 22:54:49 +03:00
kssenii
20192352ad Better 2021-09-29 11:03:36 +03:00
MaxWk
723ff795a8 support obs storage 2021-09-29 14:11:28 +08:00
kssenii
ebca30694d Fix 2021-09-27 18:25:30 +03:00
Alexey Milovidov
cd7f9d981c Remove ya.make 2021-09-25 04:22:54 +03:00
alesapin
3614fb9e37
Merge pull request #29223 from ClickHouse/compressed_logs 
Compressed logs for keeper
 2021-09-24 16:03:26 +03:00
kssenii
6219d541a5 Finally fixed 2021-09-24 15:29:04 +03:00
kssenii
c1ea44b9c2 Fixes 2021-09-24 13:38:08 +03:00
Vladimir Chebotarev
0343106692
Fixed logging level for message in S3Common.cpp. 2021-09-23 21:47:31 +03:00
alesapin
eb3a651bab Fix bug found on review 2021-09-23 16:55:50 +03:00
mergify[bot]
d69fc35dd9
Merge branch 'master' into compressed_logs 2021-09-22 19:43:16 +00:00
kssenii
1b9565dfa8 Merge branch 'master' of github.com:ClickHouse/ClickHouse into disk-async-read 2021-09-22 22:08:44 +03:00
Kseniia Sumarokova
192c5397cc
Merge pull request #29211 from kssenii/better-retries-2 
Fix retries for disk web
 2021-09-22 12:37:25 +03:00
alesapin
043ef03d42 revert method name 2021-09-22 10:30:15 +03:00
alesapin
334a7ab8db Change default extension to zst 2021-09-21 23:43:44 +03:00
alesapin
ac7579f187 Compressed logs for keeper 2021-09-21 17:29:05 +03:00
kssenii
e301457e91 Fix prefetch mistake 2021-09-21 15:40:48 +03:00
kssenii
975033e6e3 Fix 2021-09-21 09:44:54 +00:00
kssenii
40ee75fe97 Common thread pool for disks 2021-09-21 10:00:56 +03:00
kssenii
cacc81ca61 Fix 2021-09-21 02:31:02 +03:00
kssenii
0104cb1c8e Better 2021-09-21 01:51:17 +03:00
kssenii
f312760dca Better 2021-09-20 19:57:15 +03:00
kssenii
11ae94b8d3 Async read for remote fs 2021-09-20 16:16:46 +03:00
Azat Khuzhin
e5f66fd103 Add socket timeout values into SOCKET_TIMEOUT exception 2021-09-20 09:51:42 +03:00
Maksim Kita
14e4d49601
Merge pull request #29063 from ClickHouse/read-integer-overflow-bug-fix 
Merging #27963
 2021-09-17 16:06:58 +03:00
Maksim Kita
73368aa351 Fixed tests 2021-09-17 12:17:19 +03:00
Maksim Kita
49f04700cc Fixed style 2021-09-16 00:29:29 +03:00
Maksim Kita
7a72a765c2 Added tests 2021-09-16 00:01:01 +03:00
Maksim Kita
7c748b6140
Merge pull request #27963 from zhanglistar/overflow_bugfix 
Overflow bugfix
 2021-09-15 23:31:43 +03:00
Anton Popov
fc17936c12 Merge remote-tracking branch 'upstream/master' into HEAD 2021-09-14 23:02:30 +03:00
Nikita Mikhaylov
58f2504a56 Update ya.make 2021-09-14 12:00:32 +00:00
Sergei Semin
12e24a23d3
Merge branch 'master' into fix-build-clang-13-2 2021-09-12 19:30:55 +03:00
alexey-milovidov
ea13a8b562
Merge pull request #28659 from myrrc/improvement/tostring_to_magic_enum 
Improving CH type system with concepts
 2021-09-12 15:26:29 +03:00
Sergei Semin
f74350c148 fix special build 2021-09-12 02:49:31 +03:00