thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-17 11:52:27 +00:00
Code Issues Packages Projects Releases Wiki Activity
76,699 Commits 9,326 Branches 1,784 Tags 2.1 GiB
af78116c3f
Commit Graph

1087 Commits

Author SHA1 Message Date
taiyang-li
af78116c3f
Merge branch 'master' into hive_table 2021-11-11 11:52:28 +08:00
taiyang-li
deef4d4dbe add options read_bool_as_uint8 when parse csv 2021-11-11 11:49:54 +08:00
kssenii
618c332f9c Rename read_threadpool to threadpool 2021-11-10 07:46:18 +00:00
taiyang-li
b042e2b8f8 fix pvs check 2021-11-09 14:38:46 +08:00
taiyang-li
3a566eeeeb fix typo 2021-11-05 20:27:37 +08:00
taiyang-li
36ca0b296b implement hive table engine 2021-11-05 19:55:30 +08:00
Kruglov Pavel
1f8535c02b
Merge branch 'master' into null-deserialization 2021-11-02 12:15:21 +03:00
Alexey Milovidov
aa737b7a11 Allow to parse Date as YYYYMMDD 2021-10-30 00:23:03 +03:00
Kseniia Sumarokova
bf290d0678
Merge pull request #30837 from kssenii/remove-redundant 
Remove redundant from http buffer
 2021-10-29 22:04:06 +03:00
Kseniia Sumarokova
b49797ad51
Update ReadBufferFromS3.cpp 2021-10-29 15:54:40 +03:00
kssenii
b56c2f9499 Remove redundant lines from http buffer 2021-10-29 11:40:21 +03:00
kssenii
4f3433b4e1 Some clean up before merging 2021-10-29 01:04:52 +03:00
kssenii
7e2ea97e3c Merge branch 'master' of github.com:ClickHouse/ClickHouse into disk-async-read 2021-10-28 18:46:21 +03:00
avogar
6e8c2ab28f Use small amount of memory on stack in PeekableReadBuffer 2021-10-28 17:02:07 +03:00
kssenii
8dd5c1b99d Remove http buffer update 2021-10-28 14:28:14 +03:00
kssenii
3d01fbf603 Review fixes 2021-10-28 11:32:37 +03:00
Alexey Milovidov
8b4a6a2416 Remove cruft 2021-10-28 02:10:39 +03:00
avogar
d1ef96a5ef Add test, avoid unnecessary allocations, use PeekableReadBuffer only in corner case 2021-10-27 17:29:15 +03:00
kssenii
52296e8b04 Merge branch 'master' of github.com:ClickHouse/ClickHouse into disk-async-read 2021-10-25 00:22:32 +03:00
kssenii
5b15e2474c Checkout reading from local fs back to master version 2021-10-24 20:41:18 +03:00
Azat Khuzhin
93c34824c0 Fix explicit 0 size for createReadBufferFromFileBase() 2021-10-24 18:58:22 +03:00
Azat Khuzhin
2b96433daf Replace estimated_size with optional<size> for IDisk::readFile() 
v2: rebase to fix conflicts in DiskEncrypted.cpp
 2021-10-24 18:58:11 +03:00
Vitaly Baranov
c0b01e7b0b
Merge pull request #30494 from vitlibar/fix-reading-from-empty-file-on-encrypted-disk 
Fix reading from empty file on encrypted disk
 2021-10-24 16:24:47 +03:00
kssenii
39b9e9c258 Merge branch 'master' of github.com:ClickHouse/ClickHouse into disk-async-read 2021-10-22 15:40:41 +03:00
kssenii
56f0560c46 Asserts and read till end option 2021-10-21 20:54:03 +03:00
Vitaly Baranov
172a6f2e0d Fix reading from empty file on encrypted disk. 2021-10-21 19:31:29 +03:00
kssenii
120b01e089 Better 2021-10-21 18:22:17 +03:00
kssenii
e4e157688d Review fixes 2021-10-21 01:57:43 +03:00
alexey-milovidov
d3df1c02bc
Merge pull request #29672 from ClickHouse/clickhouse-obfuscator-2 
More enhancements for query obfuscator
 2021-10-20 23:06:29 +03:00
kssenii
5f24eb10ec Test changes in MergeTreeReader for local fs 2021-10-20 13:52:30 +03:00
kssenii
31cd71c849 Clean up 2021-10-20 09:37:18 +03:00
kssenii
143e912e4e Fix split build 2021-10-19 22:45:46 +03:00
Nikolai Kochetov
a92dc0a826 Update obsolete comments. 2021-10-19 12:58:10 +03:00
kssenii
d5d4817350 Add disk proxies back, add setReadUntilPosition to ReadBuffer 2021-10-18 18:35:11 +03:00
Azat Khuzhin
0e34a9d550 Add MSan instrumentation for preadv2 
MSan report [1]:

    ==10672==WARNING: MemorySanitizer: use-of-uninitialized-value
        3 0x30dd6005 in DB::CompressionCodecFactory::get(unsigned char) const obj-x86_64-linux-gnu/../src/Compression/CompressionFactory.cpp:105:65
        4 0x30dab4d5 in DB::CompressedReadBufferBase::readCompressedData(unsigned long&, unsigned long&, bool) obj-x86_64-linux-gnu/../src/Compression/CompressedReadBufferBase.cpp:127:53
        5 0x30db54af in DB::CompressedReadBufferFromFile::readBig(char*, unsigned long) obj-x86_64-linux-gnu/../src/Compression/CompressedReadBufferFromFile.cpp:119:38

      Uninitialized value was stored to memory at
        0 0x9ef4e99 in __msan_memcpy (/src/ch/tmp/30191/clickhouse-msan+0x9ef4e99)
        1 0xa0c1999 in DB::ReadBuffer::read() obj-x86_64-linux-gnu/../src/IO/ReadBuffer.h:173:13
        2 0xa0c1999 in DB::ReadBuffer::readStrict() obj-x86_64-linux-gnu/../src/IO/ReadBuffer.h:184:27
        3 0x30daafd1 in DB::CompressedReadBufferBase::readCompressedData() obj-x86_64-linux-gnu/../src/Compression/CompressedReadBufferBase.cpp:120:20
        4 0x30db54af in DB::CompressedReadBufferFromFile::readBig() obj-x86_64-linux-gnu/../src/Compression/CompressedReadBufferFromFile.cpp:119:38

      Uninitialized value was created by a heap allocation
        0 0x9ef5285 in posix_memalign (/src/ch/tmp/30191/clickhouse-msan+0x9ef5285)
        1 0xa0cb98d in Allocator<false, false>::allocNoTrack(unsigned long, unsigned long) obj-x86_64-linux-gnu/../src/Common/Allocator.h:235:27
        2 0xa0caf0d in Allocator<false, false>::alloc(unsigned long, unsigned long) obj-x86_64-linux-gnu/../src/Common/Allocator.h:96:16
        3 0xa0caf0d in DB::Memory<Allocator<false, false> >::alloc() obj-x86_64-linux-gnu/../src/IO/BufferWithOwnMemory.h:126:49
        4 0xa0ca4a1 in DB::Memory<Allocator<false, false> >::Memory() obj-x86_64-linux-gnu/../src/IO/BufferWithOwnMemory.h:43:9
        5 0xa0ca4a1 in DB::BufferWithOwnMemory<DB::SeekableReadBuffer>::BufferWithOwnMemory(unsigned long, char*, unsigned long) obj-x86_64-linux-gnu/../src/IO/BufferWithOwnMemory.h:153:29
        6 0xa0ca4a1 in DB::ReadBufferFromFileBase::ReadBufferFromFileBase() obj-x86_64-linux-gnu/../src/IO/ReadBufferFromFileBase.cpp:11:7
        7 0xa59d980 in DB::AsynchronousReadBufferFromFileDescriptor::AsynchronousReadBufferFromFileDescriptor() obj-x86_64-linux-gnu/../src/IO/AsynchronousReadBufferFromFileDescriptor.h:40:11

    SUMMARY: MemorySanitizer: use-of-uninitialized-value obj-x86_64-linux-gnu/../contrib/libcxx/include/__hash_table:116:10 in std::__1::__constrain_hash(unsigned long, unsigned long)

  [1]: https://clickhouse-test-reports.s3.yandex.net/30191/18bb0e7d8d47dd694390f3f7b2ecc921a167afac/fuzzer_msan/report.html#fail1
 2021-10-17 23:54:58 +03:00
Azat Khuzhin
fb54d6b6e0 BufferWithOwnMemory: take reallocs into account 2021-10-17 23:54:58 +03:00
Azat Khuzhin
348033d8bd BufferWithOwnMemory: make size aligned not capacity 
This will fix pread_fake_async/pread_threadpool with min_bytes_to_use_direct_io
 2021-10-17 23:54:58 +03:00
Azat Khuzhin
9cc1178ebc BufferWithOwnMemory: do not try to align if buffer already aligned 2021-10-17 23:54:57 +03:00
Azat Khuzhin
34810456e1 Fix alignment for prefetch in AsynchronousReadBufferFromFileDescriptor 2021-10-17 23:54:57 +03:00
Alexey Milovidov
c2faf450d1 Fix error 2021-10-17 06:37:51 +03:00
Alexey Milovidov
04d8f611d2 Merge branch 'master' into clickhouse-obfuscator-2 2021-10-17 05:52:13 +03:00
Maksim Kita
de36038733
Merge pull request #30276 from azat/buffer-fwd-decl 
Use forward declaration for Buffer<> in generic headers
 2021-10-16 19:21:38 +03:00
alexey-milovidov
e9af03ef52
Merge pull request #30248 from ClickHouse/musl-4 
Preparation to build with Musl
 2021-10-16 18:44:35 +03:00
Azat Khuzhin
50231460af Use forward declaration for Buffer<> in generic headers 
- changes in ReadHelpers.h -- recompiles 1000 modules
- changes in FormatFactor.h -- recompiles 100 modules
 2021-10-16 12:03:24 +03:00
Alexey Milovidov
41acc52458 Preparation to build with Musl 2021-10-16 00:17:34 +03:00
kssenii
759f747080 Fix build 2021-10-15 17:12:34 +03:00
kssenii
881ae8617e Merge branch 'master' of github.com:ClickHouse/ClickHouse into disk-async-read 2021-10-15 15:09:56 +03:00
kssenii
edfb1a5521 Better performance for smaller number of threads 2021-10-15 15:07:39 +03:00
kssenii
c1c574e9ca Better 2021-10-15 13:14:57 +03:00
kssenii
3995506d37 Adjust range reader for remote fs reads 2021-10-15 11:36:26 +03:00