thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-11-29 11:02:08 +00:00
Code Issues Packages Projects Releases Wiki Activity
74,076 Commits 9,189 Branches 1,782 Tags 2.2 GiB
9dd0fca1ed
Commit Graph

12235 Commits

Author SHA1 Message Date
Azat Khuzhin
9dd0fca1ed Suppress some existed warnings in clickhouse-test (will be fixed separately) 2021-10-08 00:09:37 +03:00
Azat Khuzhin
a171dfd75e Make 01092_memory_profiler more robust for dev env 2021-10-07 21:13:56 +03:00
Azat Khuzhin
0c0427dc45 Cover memory_profiler_sample_probability in 01092_memory_profiler 2021-10-07 21:13:56 +03:00
Azat Khuzhin
bf88f102fe Add ability to trace peak memory usage (with new trace_type - MemoryPeak) 2021-10-07 21:13:56 +03:00
Nikolai Kochetov
25e2ebac75
Merge pull request #29782 from azat/lc-concurrent-GROUP-BY-fix 
Fix concurrent access to LowCardinality during GROUP BY (leads to SIGSEGV)
 2021-10-07 16:27:53 +03:00
Anton Popov
4bc14dedfb
Merge pull request #28352 from Avogar/div-null 
Avoid division by zero when denominator is Nullable
 2021-10-07 14:47:53 +03:00
Kruglov Pavel
13b2fdc23b
Merge pull request #29659 from amosbird/jsonfix3 
Fix nullable processing in JSONFunctions
 2021-10-07 13:55:22 +03:00
Vladimir C
0422939893
Merge pull request #29281 from lingtaolf/bugfix/tupleFilter 2021-10-07 12:24:26 +03:00
Nikolai Kochetov
4ab2e2745b
Merge pull request #29786 from azat/parallel_view_processing-fix 
Fix parallel_view_processing
 2021-10-07 12:15:26 +03:00
Nikolai Kochetov
78e1db209f
Remove more data streams (#29491) 
* Remove more streams.

* Fixing build.

* Fixing build.

* Rename files.

* Fix fast test.

* Fix StorageKafka.

* Try fix kafka test.

* Move createBuffer to KafkaSource ctor.

* Revert "Move createBuffer to KafkaSource ctor."

This reverts commit 81fa94d27e.

* Revert "Try fix kafka test."

This reverts commit 2107e54969.

* Comment some rows in test.

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2021-10-07 11:26:08 +03:00
Vladimir C
b02c12b182
Merge pull request #23881 from darkkeks/fix-cross-to-inline-join-error-condition 2021-10-07 11:09:36 +03:00
Maksim Kita
4ec7311d4d
Merge pull request #29738 from kitaisreal/added-function-ngram 
Added function ngram
 2021-10-07 10:21:00 +03:00
Amos Bird
41a80a8852
Add more tests 2021-10-07 04:02:44 +08:00
Amos Bird
a34a08268b
Fix nullable processing in JSONFunctions 2021-10-07 02:26:50 +08:00
Amos Bird
a391b64f57
Revert "Merge pull request #29746 from azat/JSONExtract-Null-fix" 
This reverts commit e990f35ba3, reversing
changes made to 8e03e0127a.
 2021-10-07 02:26:32 +08:00
Azat Khuzhin
a62728c16e Mark 02046_low_cardinality_parallel_group_by as long 2021-10-06 21:11:59 +03:00
Kseniia Sumarokova
e88005c6f6
Merge pull request #29762 from kssenii/update-minio 
May be fix s3 tests
 2021-10-06 16:33:54 +03:00
Filatenkov Artur
a9e8ba0c00
Correct behaviour with unknown methods (#29057) 
* correct behaviour with unknown methods

* move check on earlier stage

* add test

* correct filter for handler

* Update HTTPServerConnection.cpp

* remove trying to get params

* correct filter for head requests

* Update HTTPHandlerFactory.h

* Update HTTPHandlerFactory.h

* Trigger Build
 2021-10-06 14:43:05 +03:00
mergify[bot]
6eec0d76a7
Merge branch 'master' into update-minio 2021-10-06 05:45:18 +00:00
Kseniia Sumarokova
3663d23e4d
Merge pull request #29731 from kssenii/exists-subquery 
Support EXISTS(subquery)
 2021-10-05 22:26:19 +03:00
Azat Khuzhin
ae5570e6d6 Improve parallel_view_processing test 2021-10-05 22:20:46 +03:00
Azat Khuzhin
342ec02664 Fix concurrent access to LowCardinality during GROUP BY (leads to SIGSEGV) 
The problem is that GROUP BY can update saved_hash, which can be also
updated by subsequent update of a dictionary, and this will lead to
use-after-free.

You will find ASan report in `details`.

<details>

    ==24679==ERROR: AddressSanitizer: heap-use-after-free on address 0x604000615d20 at pc 0x000022cc8684 bp 0x7ffea6b5f850 sp 0x7ffea6b5f848
    READ of size 8 at 0x604000615d20 thread T223 (QueryPipelineEx)
        0 0x22cc8683 in DB::ReverseIndex<>::insert(StringRef const&) obj-x86_64-linux-gnu/../src/Columns/ReverseIndex.h
        1 0x22cc0de1 in COW<DB::IColumn>::mutable_ptr<DB::IColumn> DB::ColumnUnique<DB::ColumnString>::uniqueInsertRangeImpl<char8_t>()::'lambda'()::operator()() const obj-x86_64-linux-gnu/../src/Columns/ColumnUnique.h:540:39
        2 0x22cc0de1 in COW<DB::IColumn>::mutable_ptr<DB::IColumn> DB::ColumnUnique<DB::ColumnString>::uniqueInsertRangeImpl<char8_t>() obj-x86_64-linux-gnu/../src/Columns/ColumnUnique.h:570:23
        3 0x22cb9c66 in COW<DB::IColumn>::mutable_ptr<DB::IColumn> DB::ColumnUnique<DB::ColumnString>::uniqueInsertRangeFrom()::'lambda'(auto)::operator()<char8_t>(auto) const obj-x86_64-linux-gnu/../src/Columns/ColumnUnique.h:592:26
        4 0x22cb9c66 in DB::ColumnUnique<DB::ColumnString>::uniqueInsertRangeFrom() obj-x86_64-linux-gnu/../src/Columns/ColumnUnique.h:600:28
        5 0x2500b897 in DB::ColumnLowCardinality::insertRangeFrom() obj-x86_64-linux-gnu/../src/Columns/ColumnLowCardinality.cpp:205:62
        6 0x25a182f4 in DB::appendBlock(DB::Block const&, DB::Block&) obj-x86_64-linux-gnu/../src/Storages/StorageBuffer.cpp:470:23
        7 0x25a182f4 in DB::BufferSink::insertIntoBuffer(DB::Block const&, DB::StorageBuffer::Buffer&) obj-x86_64-linux-gnu/../src/Storages/StorageBuffer.cpp:634:9
        8 0x25a173cc in DB::BufferSink::consume(DB::Chunk) obj-x86_64-linux-gnu/../src/Storages/StorageBuffer.cpp:595:9
        9 0x26d1c997 in DB::SinkToStorage::transform(DB::Chunk&) obj-x86_64-linux-gnu/../src/Processors/Sinks/SinkToStorage.cpp:18:5

    0x604000615d20 is located 16 bytes inside of 40-byte region [0x604000615d10,0x604000615d38)
    freed by thread T37 (QueryPipelineEx) here:
        2 0x22cb9392 in boost::intrusive_ptr<DB::ColumnVector<unsigned long> >::~intrusive_ptr() obj-x86_64-linux-gnu/../contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:98:23
        4 0x22cb9392 in COW<DB::IColumn>::mutable_ptr<DB::ColumnVector<unsigned long> >::operator=() obj-x86_64-linux-gnu/../src/Common/COW.h💯57
        5 0x22cb9392 in DB::ReverseIndex<>::tryGetSavedHash() const obj-x86_64-linux-gnu/../src/Columns/ReverseIndex.h:362:28
        6 0x22cb9392 in DB::ColumnUnique<DB::ColumnString>::tryGetSavedHash() const obj-x86_64-linux-gnu/../src/Columns/ColumnUnique.h:125:76
        7 0x242eaed3 in DB::ColumnsHashing::HashMethodSingleLowCardinalityColumn<>::HashMethodSingleLowCardinalityColumn() obj-x86_64-linux-gnu/../src/Common/ColumnsHashing.h:287:50
        8 0x242206c6 in void DB::Aggregator::executeImpl<>() const obj-x86_64-linux-gnu/../src/Interpreters/Aggregator.cpp:596:28
        9 0x24148e99 in DB::Aggregator::executeOnBlock() const obj-x86_64-linux-gnu/../src/Interpreters/Aggregator.cpp:1004:9
        10 0x26c24f3a in DB::AggregatingTransform::consume(DB::Chunk) obj-x86_64-linux-gnu/../src/Processors/Transforms/AggregatingTransform.cpp:539:33
        11 0x26c2054e in DB::AggregatingTransform::work() obj-x86_64-linux-gnu/../src/Processors/Transforms/AggregatingTransform.cpp:500:9

    previously allocated by thread T37 (QueryPipelineEx) here:
        0 0xb6d44fd in operator new(unsigned long) (/src/ch/tmp/upstream/clickhouse-asan+0xb6d44fd)
        1 0x11b78580 in COW<DB::IColumn>::mutable_ptr<> COWHelper<>::create<unsigned long&>(unsigned long&) (/src/ch/tmp/upstream/clickhouse-asan+0x11b78580)
        2 0x22cbf7b1 in DB::ReverseIndex<>::calcHashes() const obj-x86_64-linux-gnu/../src/Columns/ReverseIndex.h:472:17
        3 0x22cc2307 in DB::ReverseIndex<>::buildIndex() obj-x86_64-linux-gnu/../src/Columns/ReverseIndex.h:438:22
        4 0x22cc658c in DB::ReverseIndex<>::insert(StringRef const&) obj-x86_64-linux-gnu/../src/Columns/ReverseIndex.h:484:9
        5 0x22cc0de1 in COW<DB::IColumn>::mutable_ptr<DB::IColumn> DB::ColumnUnique<DB::ColumnString>::uniqueInsertRangeImpl<char8_t>()::'lambda'()::operator()() const obj-x86_64-linux-gnu/../src/Columns/ColumnUnique.h:540:39
        6 0x22cc0de1 in COW<DB::IColumn>::mutable_ptr<DB::IColumn> DB::ColumnUnique<DB::ColumnString>::uniqueInsertRangeImpl<char8_t>() obj-x86_64-linux-gnu/../src/Columns/ColumnUnique.h:570:23
        7 0x22cb9c66 in COW<DB::IColumn>::mutable_ptr<DB::IColumn> DB::ColumnUnique<DB::ColumnString>::uniqueInsertRangeFrom()::'lambda'(auto)::operator()<char8_t>(auto) const obj-x86_64-linux-gnu/../src/Columns/ColumnUnique.h:592:26
        8 0x22cb9c66 in DB::ColumnUnique<DB::ColumnString>::uniqueInsertRangeFrom() obj-x86_64-linux-gnu/../src/Columns/ColumnUnique.h:600:28
        9 0x2500b897 in DB::ColumnLowCardinality::insertRangeFrom() obj-x86_64-linux-gnu/../src/Columns/ColumnLowCardinality.cpp:205:62
        10 0x25a182f4 in DB::appendBlock(DB::Block const&, DB::Block&) obj-x86_64-linux-gnu/../src/Storages/StorageBuffer.cpp:470:23
        11 0x25a182f4 in DB::BufferSink::insertIntoBuffer(DB::Block const&, DB::StorageBuffer::Buffer&) obj-x86_64-linux-gnu/../src/Storages/StorageBuffer.cpp:634:9
        12 0x25a173cc in DB::BufferSink::consume(DB::Chunk) obj-x86_64-linux-gnu/../src/Storages/StorageBuffer.cpp:595:9
        13 0x26d1c997 in DB::SinkToStorage::transform(DB::Chunk&) obj-x86_64-linux-gnu/../src/Processors/Sinks/SinkToStorage.cpp:18:5

    SUMMARY: AddressSanitizer: heap-use-after-free obj-x86_64-linux-gnu/../src/Columns/ReverseIndex.h in DB::ReverseIndex<unsigned long, DB::ColumnString>::insert(StringRef const&)
    Shadow bytes around the buggy address:
      0x0c08800bab50: fa fa fd fd fd fd fd fa fa fa 00 00 00 00 00 00
      0x0c08800bab60: fa fa fd fd fd fd fd fa fa fa 00 00 00 00 00 fa
      0x0c08800bab70: fa fa 00 00 00 00 00 00 fa fa 00 00 00 00 00 00
      0x0c08800bab80: fa fa fd fd fd fd fd fa fa fa fd fd fd fd fd fa
      0x0c08800bab90: fa fa fd fd fd fd fd fa fa fa fd fd fd fd fd fd
    =>0x0c08800baba0: fa fa fd fd[fd]fd fd fa fa fa fd fd fd fd fd fa
      0x0c08800babb0: fa fa fd fd fd fd fd fd fa fa fd fd fd fd fd fa
      0x0c08800babc0: fa fa fd fd fd fd fd fa fa fa fd fd fd fd fd fd
      0x0c08800babd0: fa fa fd fd fd fd fd fd fa fa fd fd fd fd fd fa
      0x0c08800babe0: fa fa fd fd fd fd fd fa fa fa fd fd fd fd fd fd
      0x0c08800babf0: fa fa fd fd fd fd fd fd fa fa fd fd fd fd fd fd
    Shadow byte legend (one shadow byte represents 8 application bytes):
      Addressable:           00
      Partially addressable: 01 02 03 04 05 06 07
      Heap left redzone:       fa
      Freed heap region:       fd
      Stack left redzone:      f1
      Stack mid redzone:       f2
      Stack right redzone:     f3
      Stack after return:      f5
      Stack use after scope:   f8
      Global redzone:          f9
      Global init order:       f6
      Poisoned by user:        f7
      Container overflow:      fc
      Array cookie:            ac
      Intra object redzone:    bb
      ASan internal:           fe
      Left alloca redzone:     ca
      Right alloca redzone:    cb
    ==24679==ABORTING

</details>
 2021-10-05 21:07:10 +03:00
Nikita Mikhaylov
cc150b7316
Merge pull request #29596 from nikitamikhaylov/fuzz 
Add fuzzer for `executeQuery` function
 2021-10-05 20:10:13 +03:00
kssenii
51f78e4446 Merge branch 'update-minio' of github.com:kssenii/ClickHouse into update-minio 2021-10-05 13:20:07 +00:00
kssenii
1c37b0df53 Ping tests 2021-10-05 13:19:43 +00:00
Kruglov Pavel
e990f35ba3
Merge pull request #29746 from azat/JSONExtract-Null-fix 
Forbid Nullable for JSONExtract* (JSONExtract() still supports Nullable), leads to SIGSEGV before
 2021-10-05 13:04:46 +03:00
Vitaly Baranov
8a01b32cba
Merge pull request #28637 from vitlibar/fix-materialized-column-as-sharding-key 
Fix materialized column as sharding key
 2021-10-05 10:53:24 +03:00
Azat Khuzhin
20e706766c Forbid Nullable for JSONExtract* (JSONExtract() still supports Nullable) 
Only JSONExtract() can support Nullable, others JSONExtract*
(JSONExtractString and similar) does not.

And right now this file is pretty complex already,
so adding support of Nullable for others will make it even more complex.

CI: https://clickhouse-test-reports.s3.yandex.net/29680/d0fc26f91a0141b56a0550741219c3dc43630e03/fuzzer_ubsan/report.html#fail1
 2021-10-04 21:08:02 +03:00
Nikita Mikhaylov
ec2b30703e Better dictionary 2021-10-04 17:57:58 +00:00
Maksim Kita
3fe9fdac96 Added function ngram 2021-10-04 18:53:17 +03:00
Kruglov Pavel
ab7c8e7074
Delete temporary file 2021-10-04 18:25:49 +03:00
avogar
2523053b27 Remove code duplication, fix bug, add more tests 2021-10-04 18:22:06 +03:00
Nikolai Kochetov
122d39cbba
Fix missing condition in pushed down predicate. (#29625) 2021-10-04 17:50:03 +03:00
Kseniia Sumarokova
d235520b3b
Merge pull request #29706 from kssenii/fix-flacky-test 
Fix flacky test
 2021-10-04 17:43:51 +03:00
Pavel Kruglov
82d17870f1 Avoid division by zero when denominator is Nullable 2021-10-04 16:44:38 +03:00
kssenii
f1ea186bff Exists 2021-10-04 13:12:30 +00:00
Anton Popov
362e84a336
Merge pull request #29429 from Enmk/Governance/database_comment 
Implemented creating databases with comments
 2021-10-04 15:12:33 +03:00
Nikolay Degterinsky
345ac59f46
Merge pull request #29552 from MaxWk/conversion-between-date32-and-datetime64 
Support conversion between date32 and datetime64
 2021-10-04 12:43:56 +03:00
Kseniia Sumarokova
a8f872f63e
Update test.py 2021-10-04 11:38:38 +03:00
Vitaly Baranov
217659c1c6 Add test. 2021-10-04 10:56:38 +03:00
Maksim Kita
25d1588d0a
Merge pull request #29666 from kitaisreal/user-defined-executable-functions-added-implicit-cast 
UserDefinedExecutableFunctions added implicit cast
 2021-10-04 10:34:31 +03:00
kssenii
a3ee621d5e Fix 2021-10-04 07:28:58 +00:00
Kseniia Sumarokova
5ab17d795f
Merge pull request #29673 from ClickHouse/url-fix-initialization 
Fix initialization of `StorageURLSource`
 2021-10-04 10:23:52 +03:00
alexey-milovidov
325a01f798
Merge pull request #29553 from azat/group-by-with-totals-having-fix 
Fix null deference for GROUP BY WITH TOTALS HAVING (when the column from HAVING wasn't selected)
 2021-10-04 09:25:35 +03:00
kssenii
0cb7193fde Fix url with failover case 2021-10-03 13:58:03 +00:00
alexey-milovidov
d822ba1f3b
Merge pull request #25262 from ClickHouse/add-test-21592 
Add a test for #21592
 2021-10-03 12:09:28 +03:00
alexey-milovidov
35ef7266db
Merge pull request #25252 from ClickHouse/add-test-24261 
Add a test for #24261
 2021-10-03 12:09:16 +03:00
alexey-milovidov
ac57e057a1
Merge pull request #23028 from ClickHouse/cast_to_value_or_default 
Merging #21330
 2021-10-03 11:00:48 +03:00
Alexey Milovidov
345d4bf48c Update test 2021-10-03 08:01:51 +03:00
mergify[bot]
644b3ffe38
Merge branch 'master' into add-test-24261 2021-10-03 04:51:37 +00:00