thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-11-26 09:32:01 +00:00
Code Issues Packages Projects Releases Wiki Activity
75,020 Commits 9,177 Branches 1,781 Tags 2.1 GiB
0d6712532a
Commit Graph

18909 Commits

Author SHA1 Message Date
Nikolai Kochetov
9195e4e887
Merge pull request #30359 from ClickHouse/remove-streams-from-comments 
Update obsolete comments.
 2021-10-19 15:04:04 +03:00
Vladimir C
49685f0b9a
Merge pull request #30292 from Joeywzr/trim_dev 2021-10-19 14:49:43 +03:00
Nikita Mikhaylov
ea2a647724
Merge pull request #30192 from azat/granula-adjust-LOGICAL_ERROR-fix 
Fix reading from MergeTree with max_read_buffer_size=0
 2021-10-19 13:25:55 +03:00
Nikolai Kochetov
a92dc0a826 Update obsolete comments. 2021-10-19 12:58:10 +03:00
Ilya Yatsishin
4396daeb1f
Merge pull request #30064 from azat/client-print-profile-events 2021-10-19 12:42:09 +03:00
Kruglov Pavel
1e2ceeb2e7
Merge pull request #29291 from Avogar/capnproto 
Add CapnProto output format, refactor CapnProto input format
 2021-10-19 11:54:55 +03:00
Kruglov Pavel
de2b75abf0
Merge pull request #30053 from Avogar/clickhouse-local 
Remove unused headers and handle exception 'unrecognised option' in clickhouse-local
 2021-10-19 11:52:15 +03:00
Nikolai Kochetov
cc7a3ea9ff
Merge pull request #30322 from ClickHouse/update-mergine-sorted-memory-optimization 
Update memory optimisation for MergingSorted.
 2021-10-19 11:12:50 +03:00
alesapin
53ad965f8e
Merge pull request #30170 from ClickHouse/debug_keeper 
Start keeper asynchronously in some cases
 2021-10-19 10:14:36 +03:00
Azat Khuzhin
424bf6fcf4 client: add ability to print raw profile events 
This can be useful for debugging and for testing (since you will not
need to obtain query_id and look at query_log).

v2:
- mark test as long
- add option to docs
- add type of profile event into logs too
v3:
- resolve conflicts
- and fix onProfileEvents callback
v4:
- add --print-profile-events separate switch
 2021-10-19 00:54:38 +03:00
Azat Khuzhin
f7b76373ce Move ProfileEvents packet type from TCPHandler into ProfileEventsExt 2021-10-19 00:54:38 +03:00
Nikita Mikhaylov
d004fd96ed
Merge pull request #30318 from azat/fix-shared-build 
Move SquashingTransform to Interpreters (to fix split build)
 2021-10-18 22:41:01 +03:00
alexey-milovidov
885c4daf94
Merge pull request #30191 from azat/aio-O_DIRECT-fix 
Fix pread_fake_async/pread_threadpool with min_bytes_to_use_direct_io
 2021-10-18 22:29:31 +03:00
Azat Khuzhin
e576fd17bd Do not allow reading to empty buffer in MergeTreeReaderStream 
Changelog:
- Verify that all rows was read in MergeTreeReaderWide
- Ignore some exceptions for Wide parts
- Take max_rows_to_read into account
- Verify that requested rows is not zero (otherwise it is too tricky)
- Simply verify that buffer is not empty
 2021-10-18 21:19:34 +03:00
Azat Khuzhin
cd4b33c8c9 Verify that all rows was read in MergeTreeReaderCompact 
v0: Use fmt-like style exception in MergeTreeReaderCompact
v2: Update the check
 2021-10-18 21:19:34 +03:00
Azat Khuzhin
41e6df0f4b Remove unused offset_columns from MergeTreeReaderWide::readRows() 2021-10-18 21:19:34 +03:00
Nikolai Kochetov
7962eddf60 Review fixes. 2021-10-18 19:18:09 +03:00
Vladimir C
4a4700424e
Merge pull request #30230 from vdimir/CLOUDSUPPORT-92764 2021-10-18 14:46:02 +03:00
Kruglov Pavel
6350957709
Fix special build 2021-10-18 14:30:02 +03:00
Kruglov Pavel
003955b59f
Merge branch 'master' into clickhouse-local 2021-10-18 14:24:08 +03:00
vdimir
042e61febf
Allow identifiers staring with numbers in multiple joins 2021-10-18 13:41:40 +03:00
Nikolai Kochetov
6e479b301a Update memory optimisation for MergingSorted. 2021-10-18 12:54:12 +03:00
alesapin
6d24ca4c3a Review fixes 2021-10-18 12:13:24 +03:00
Anton Popov
71b474793c
Merge pull request #30174 from CurtizJ/better-interfaces 
Better interfaces for `IDataType` and `ISerialization`
 2021-10-18 12:07:04 +03:00
Azat Khuzhin
6d4af3bac1 Move SquashingTransform to Interpreters (to fix split build) 
clickhouse_common_io requires clickhouse_core:

    ld.lld: error: undefined symbol: DB::blocksHaveEqualStructure(DB::Block const&, DB::Block const&)
    >>> referenced by SquashingTransform.cpp:92 (/src/ch/clickhouse/src/Common/SquashingTransform.cpp:92)
    >>>               src/CMakeFiles/clickhouse_common_io.dir/Common/SquashingTransform.cpp.o:(void DB::SquashingTransform::append<DB::Block&&>(DB::Block&&))
    >>> referenced by SquashingTransform.cpp:92 (/src/ch/clickhouse/src/Common/SquashingTransform.cpp:92)
    >>>               src/CMakeFiles/clickhouse_common_io.dir/Common/SquashingTransform.cpp.o:(void DB::SquashingTransform::append<DB::Block const&>(DB::Block const&))

while clickhouse_core requires clickhouse_common_io:

    "clickhouse_core" of type SHARED_LIBRARY
      depends on "roaring" (weak)
      depends on "clickhouse_common_io" (weak)
      depends on "clickhouse_common_config" (weak)
      depends on "clickhouse_common_zookeeper" (weak)
      depends on "clickhouse_dictionaries_embedded" (weak)
      depends on "clickhouse_parsers" (weak)

Follow-up for: #30247 (cc @KochetovNicolai)
 2021-10-18 10:28:36 +03:00
alesapin
f1fe96e194 Merge branch 'master' into debug_keeper 2021-10-18 10:16:07 +03:00
Azat Khuzhin
0e34a9d550 Add MSan instrumentation for preadv2 
MSan report [1]:

    ==10672==WARNING: MemorySanitizer: use-of-uninitialized-value
        3 0x30dd6005 in DB::CompressionCodecFactory::get(unsigned char) const obj-x86_64-linux-gnu/../src/Compression/CompressionFactory.cpp:105:65
        4 0x30dab4d5 in DB::CompressedReadBufferBase::readCompressedData(unsigned long&, unsigned long&, bool) obj-x86_64-linux-gnu/../src/Compression/CompressedReadBufferBase.cpp:127:53
        5 0x30db54af in DB::CompressedReadBufferFromFile::readBig(char*, unsigned long) obj-x86_64-linux-gnu/../src/Compression/CompressedReadBufferFromFile.cpp:119:38

      Uninitialized value was stored to memory at
        0 0x9ef4e99 in __msan_memcpy (/src/ch/tmp/30191/clickhouse-msan+0x9ef4e99)
        1 0xa0c1999 in DB::ReadBuffer::read() obj-x86_64-linux-gnu/../src/IO/ReadBuffer.h:173:13
        2 0xa0c1999 in DB::ReadBuffer::readStrict() obj-x86_64-linux-gnu/../src/IO/ReadBuffer.h:184:27
        3 0x30daafd1 in DB::CompressedReadBufferBase::readCompressedData() obj-x86_64-linux-gnu/../src/Compression/CompressedReadBufferBase.cpp:120:20
        4 0x30db54af in DB::CompressedReadBufferFromFile::readBig() obj-x86_64-linux-gnu/../src/Compression/CompressedReadBufferFromFile.cpp:119:38

      Uninitialized value was created by a heap allocation
        0 0x9ef5285 in posix_memalign (/src/ch/tmp/30191/clickhouse-msan+0x9ef5285)
        1 0xa0cb98d in Allocator<false, false>::allocNoTrack(unsigned long, unsigned long) obj-x86_64-linux-gnu/../src/Common/Allocator.h:235:27
        2 0xa0caf0d in Allocator<false, false>::alloc(unsigned long, unsigned long) obj-x86_64-linux-gnu/../src/Common/Allocator.h:96:16
        3 0xa0caf0d in DB::Memory<Allocator<false, false> >::alloc() obj-x86_64-linux-gnu/../src/IO/BufferWithOwnMemory.h:126:49
        4 0xa0ca4a1 in DB::Memory<Allocator<false, false> >::Memory() obj-x86_64-linux-gnu/../src/IO/BufferWithOwnMemory.h:43:9
        5 0xa0ca4a1 in DB::BufferWithOwnMemory<DB::SeekableReadBuffer>::BufferWithOwnMemory(unsigned long, char*, unsigned long) obj-x86_64-linux-gnu/../src/IO/BufferWithOwnMemory.h:153:29
        6 0xa0ca4a1 in DB::ReadBufferFromFileBase::ReadBufferFromFileBase() obj-x86_64-linux-gnu/../src/IO/ReadBufferFromFileBase.cpp:11:7
        7 0xa59d980 in DB::AsynchronousReadBufferFromFileDescriptor::AsynchronousReadBufferFromFileDescriptor() obj-x86_64-linux-gnu/../src/IO/AsynchronousReadBufferFromFileDescriptor.h:40:11

    SUMMARY: MemorySanitizer: use-of-uninitialized-value obj-x86_64-linux-gnu/../contrib/libcxx/include/__hash_table:116:10 in std::__1::__constrain_hash(unsigned long, unsigned long)

  [1]: https://clickhouse-test-reports.s3.yandex.net/30191/18bb0e7d8d47dd694390f3f7b2ecc921a167afac/fuzzer_msan/report.html#fail1
 2021-10-17 23:54:58 +03:00
Azat Khuzhin
fb54d6b6e0 BufferWithOwnMemory: take reallocs into account 2021-10-17 23:54:58 +03:00
Azat Khuzhin
348033d8bd BufferWithOwnMemory: make size aligned not capacity 
This will fix pread_fake_async/pread_threadpool with min_bytes_to_use_direct_io
 2021-10-17 23:54:58 +03:00
Azat Khuzhin
9cc1178ebc BufferWithOwnMemory: do not try to align if buffer already aligned 2021-10-17 23:54:57 +03:00
Azat Khuzhin
34810456e1 Fix alignment for prefetch in AsynchronousReadBufferFromFileDescriptor 2021-10-17 23:54:57 +03:00
alexey-milovidov
7beffb04ad
Merge pull request #29495 from ClickHouse/improvement_for_strange_mutations 
Minor improvement for strange mutations
 2021-10-17 20:57:06 +03:00
Nikolai Kochetov
2a95d7fc87
Merge pull request #29918 from amosbird/projection-improve5 
Add primary key to minmax_count_projection
 2021-10-17 14:04:18 +03:00
Nikolai Kochetov
bfcbf5abe0 Merge branch 'master' into removing-data-streams-folder 2021-10-17 10:42:37 +03:00
alexey-milovidov
57f548dfb0
Merge pull request #30297 from kssenii/fix-local-break-on-timeout 
Fix clickhouse-local break on timeout
 2021-10-17 07:36:59 +03:00
WangZengrui
131aa77017 fix style 2021-10-17 11:21:58 +08:00
alexey-milovidov
3e9cf0b5ba
Merge pull request #30243 from Algunenano/multiIf_shortcircuit_fix 
Fix crash with shortcircuit and LowCardinality in multiIf
 2021-10-17 06:16:09 +03:00
alexey-milovidov
4f11cfa59d
Merge pull request #28760 from azat/mutator-forbid-cleaner 
Forbid cleaning of tmp directories that can be used by an active mutation/merge.
 2021-10-17 03:18:01 +03:00
Maksim Kita
0dd8c70d28
Merge pull request #30242 from kitaisreal/added-hashed-array-dictionary 
Added HashedArray dictionary
 2021-10-17 01:18:43 +03:00
kssenii
4390dde76c Fix local break on timeout 2021-10-16 19:50:05 +00:00
WangZengrui
bd2c016cf3 fix bug 2021-10-17 00:33:57 +08:00
Maksim Kita
de36038733
Merge pull request #30276 from azat/buffer-fwd-decl 
Use forward declaration for Buffer<> in generic headers
 2021-10-16 19:21:38 +03:00
alexey-milovidov
e9af03ef52
Merge pull request #30248 from ClickHouse/musl-4 
Preparation to build with Musl
 2021-10-16 18:44:35 +03:00
alexey-milovidov
3d2fb2f67b
Merge pull request #30275 from kssenii/fix-local-query-stage 
--stage for clickhouse-local
 2021-10-16 18:42:28 +03:00
WangZengrui
1541593b2f add notes 2021-10-16 23:28:29 +08:00
WangZengrui
6752be4c4e fix bug and add test 2021-10-16 23:11:45 +08:00
Nikolai Kochetov
a08c98d760 Move some files. 2021-10-16 17:03:50 +03:00
alesapin
9418eda122
Merge pull request #30111 from ClickHouse/remove_trash 
Remove trash from MergeTreeReadPool
 2021-10-16 15:41:41 +03:00
Nikita Mikhaylov
6b22fdc55a
Merge pull request #30012 from PaulCher/codegen_select_fuzzer_v1 
Codegen select fuzzer v1
 2021-10-16 14:08:08 +03:00
WangZengrui
8a94e26bec init 2021-10-16 18:51:42 +08:00