Commit Graph

74242 Commits

Author SHA1 Message Date
alexey-milovidov
16ad5953d6
Merge pull request #29783 from azat/tcphandler-progress-race
Fix data-race between fatal error handler and progress packets
2021-10-07 01:47:29 +03:00
avogar
568e27e9b3 Fix tests 2021-10-07 00:19:37 +03:00
Maksim Kita
c2c621c6dc Dictionary attributes updated documentation 2021-10-06 23:33:36 +03:00
Maksim Kita
b7b39da8cc Fixed build 2021-10-06 23:22:29 +03:00
Amos Bird
41a80a8852
Add more tests 2021-10-07 04:02:44 +08:00
Maksim Kita
d7dd593284
Merge pull request #29813 from azat/listen_backlog-config-default
Add listen_backlog to documentation
2021-10-06 22:20:04 +03:00
Azat Khuzhin
1c417be72e Apply suggestions from @kitaisreal 2021-10-06 22:19:09 +03:00
Tatiana Kirillova
2bfcdc431b translate 2021-10-06 21:42:07 +03:00
Maksim Kita
dce50c8b8a
Merge pull request #29779 from kitaisreal/parse-query-from-metadata-exception-throw-fix
Parse query from metadata exception throw fix
2021-10-06 21:29:12 +03:00
Amos Bird
a34a08268b
Fix nullable processing in JSONFunctions 2021-10-07 02:26:50 +08:00
Amos Bird
a391b64f57
Revert "Merge pull request #29746 from azat/JSONExtract-Null-fix"
This reverts commit e990f35ba3, reversing
changes made to 8e03e0127a.
2021-10-07 02:26:32 +08:00
Azat Khuzhin
83c271f446 Move QueryPipelineBuilder::setMaxThreads() into the InterpreterInsertQuery 2021-10-06 21:25:37 +03:00
Azat Khuzhin
a62728c16e Mark 02046_low_cardinality_parallel_group_by as long 2021-10-06 21:11:59 +03:00
Azat Khuzhin
74f5b92420 Add listen_backlog to documentation
Follow-up for: #29643
2021-10-06 21:09:00 +03:00
Nikolai Kochetov
d0c6f11fcb More. 2021-10-06 20:59:27 +03:00
Alexander Tokmakov
c5d631ca54 fix overflow in Stopwatch 2021-10-06 18:09:13 +03:00
Kseniia Sumarokova
e88005c6f6
Merge pull request #29762 from kssenii/update-minio
May be fix s3 tests
2021-10-06 16:33:54 +03:00
Filatenkov Artur
545673248a
Merge branch 'master' into add_CORS 2021-10-06 16:09:34 +03:00
Alexander Tokmakov
c0ee6d46f7 resolve dependency of StorageDictionary 2021-10-06 15:39:14 +03:00
Alexey Milovidov
1455cfeddd Improve usability of error messages when error is caused by sophisticated interventions 2021-10-06 15:28:52 +03:00
Maksim Kita
33353be4ed Fixed tests 2021-10-06 15:12:07 +03:00
Filatenkov Artur
a9e8ba0c00
Correct behaviour with unknown methods (#29057)
* correct behaviour with unknown methods

* move check on earlier stage

* add test

* correct filter for handler

* Update HTTPServerConnection.cpp

* remove trying to get params

* correct filter for head requests

* Update HTTPHandlerFactory.h

* Update HTTPHandlerFactory.h

* Trigger Build
2021-10-06 14:43:05 +03:00
Maksim Kita
a9e357aaff
Merge pull request #29792 from ClickHouse/ui-show-bars
Web UI: render bars in table cells
2021-10-06 13:49:40 +03:00
Maksim Kita
d2b3f3bd11
Merge pull request #29796 from kitaisreal/function-reinterpet-as-improve-readability
Function reinterpretAs improve readability
2021-10-06 13:44:24 +03:00
avogar
5e4cda02b6 Fix build 2021-10-06 13:38:22 +03:00
alesapin
621c9884fc Account offset in readBig call 2021-10-06 12:39:54 +03:00
Nikolai Kochetov
6b619512aa Fix style. 2021-10-06 12:37:56 +03:00
Tatiana Kirillova
a518f5ea11
Update docs/en/operations/settings/settings.md
Co-authored-by: olgarev <56617294+olgarev@users.noreply.github.com>
2021-10-06 12:26:22 +03:00
Filatenkov Artur
bf8a2b109f
Merge branch 'ClickHouse:master' into add_CORS 2021-10-06 11:57:29 +03:00
Maksim Kita
490ef61a03 Function reinterpretAs improve readability 2021-10-06 11:43:11 +03:00
Ivan Blinkov
8e9df6d667
Update sample.md 2021-10-06 09:12:27 +03:00
Ivan Blinkov
f45880aa89
Update sample-by.md 2021-10-06 09:02:20 +03:00
mergify[bot]
6eec0d76a7
Merge branch 'master' into update-minio 2021-10-06 05:45:18 +00:00
Alexey Milovidov
b5230a5fe2 Web UI: render bars in table cells 2021-10-06 03:59:16 +03:00
Alexey Milovidov
bd4b0af2e1 Fix bad cast in ParserCreateQuery 2021-10-06 02:34:48 +03:00
Maksim Kita
90eb74c933 Fixed tests 2021-10-06 00:37:45 +03:00
Kseniia Sumarokova
3663d23e4d
Merge pull request #29731 from kssenii/exists-subquery
Support EXISTS(subquery)
2021-10-05 22:26:19 +03:00
Azat Khuzhin
ae5570e6d6 Improve parallel_view_processing test 2021-10-05 22:20:46 +03:00
Azat Khuzhin
99ebdbb16e Fix parallel_view_processing
Number of threads got lost during after QueryPipelineBuilder::addChains()
2021-10-05 22:20:46 +03:00
Anton Popov
199ac31b48
Merge pull request #29683 from myrrc/improvement/tl-to-common
Non-recursive implementation for type list and its functions
2021-10-05 21:49:15 +03:00
Azat Khuzhin
4f5118b85f Fix data-race between fatal error handler and progress packets
This is not that important, but because of packets overlaps the client
may not render the fatal error.

You will find TSan report in `details`.

<details>

```
WARNING: ThreadSanitizer: data race (pid=13384)
  Write of size 8 at 0x7b24000ad818 by thread T47:
    0 DB::WriteBuffer::next() obj-x86_64-linux-gnu/../src/IO/WriteBuffer.h:43:15 (clickhouse-tsan+0x16124d61)
    1 DB::TCPHandler::sendProgress() obj-x86_64-linux-gnu/../src/Server/TCPHandler.cpp:1602:10 (clickhouse-tsan+0x16124d61)
    2 DB::TCPHandler::runImpl()::$_6::operator()() const obj-x86_64-linux-gnu/../src/Server/TCPHandler.cpp:318:25 (clickhouse-tsan+0x1612cf54)

  Previous write of size 8 at 0x7b24000ad818 by thread T229:
    0 DB::WriteBuffer::next() obj-x86_64-linux-gnu/../src/IO/WriteBuffer.h:43:15 (clickhouse-tsan+0x1612b793)
    1 DB::TCPHandler::sendLogData(DB::Block const&) obj-x86_64-linux-gnu/../src/Server/TCPHandler.cpp:1561:10 (clickhouse-tsan+0x1612b793)
    2 DB::TCPHandler::sendLogs() obj-x86_64-linux-gnu/../src/Server/TCPHandler.cpp:1632:9 (clickhouse-tsan+0x161228ab)
    3 DB::TCPHandler::runImpl()::$_1::operator()() const obj-x86_64-linux-gnu/../src/Server/TCPHandler.cpp:229:62 (clickhouse-tsan+0x1612c4e1)
```

</details>
2021-10-05 21:21:32 +03:00
Azat Khuzhin
342ec02664 Fix concurrent access to LowCardinality during GROUP BY (leads to SIGSEGV)
The problem is that GROUP BY can update saved_hash, which can be also
updated by subsequent update of a dictionary, and this will lead to
use-after-free.

You will find ASan report in `details`.

<details>

    ==24679==ERROR: AddressSanitizer: heap-use-after-free on address 0x604000615d20 at pc 0x000022cc8684 bp 0x7ffea6b5f850 sp 0x7ffea6b5f848
    READ of size 8 at 0x604000615d20 thread T223 (QueryPipelineEx)
        0 0x22cc8683 in DB::ReverseIndex<>::insert(StringRef const&) obj-x86_64-linux-gnu/../src/Columns/ReverseIndex.h
        1 0x22cc0de1 in COW<DB::IColumn>::mutable_ptr<DB::IColumn> DB::ColumnUnique<DB::ColumnString>::uniqueInsertRangeImpl<char8_t>()::'lambda'()::operator()() const obj-x86_64-linux-gnu/../src/Columns/ColumnUnique.h:540:39
        2 0x22cc0de1 in COW<DB::IColumn>::mutable_ptr<DB::IColumn> DB::ColumnUnique<DB::ColumnString>::uniqueInsertRangeImpl<char8_t>() obj-x86_64-linux-gnu/../src/Columns/ColumnUnique.h:570:23
        3 0x22cb9c66 in COW<DB::IColumn>::mutable_ptr<DB::IColumn> DB::ColumnUnique<DB::ColumnString>::uniqueInsertRangeFrom()::'lambda'(auto)::operator()<char8_t>(auto) const obj-x86_64-linux-gnu/../src/Columns/ColumnUnique.h:592:26
        4 0x22cb9c66 in DB::ColumnUnique<DB::ColumnString>::uniqueInsertRangeFrom() obj-x86_64-linux-gnu/../src/Columns/ColumnUnique.h:600:28
        5 0x2500b897 in DB::ColumnLowCardinality::insertRangeFrom() obj-x86_64-linux-gnu/../src/Columns/ColumnLowCardinality.cpp:205:62
        6 0x25a182f4 in DB::appendBlock(DB::Block const&, DB::Block&) obj-x86_64-linux-gnu/../src/Storages/StorageBuffer.cpp:470:23
        7 0x25a182f4 in DB::BufferSink::insertIntoBuffer(DB::Block const&, DB::StorageBuffer::Buffer&) obj-x86_64-linux-gnu/../src/Storages/StorageBuffer.cpp:634:9
        8 0x25a173cc in DB::BufferSink::consume(DB::Chunk) obj-x86_64-linux-gnu/../src/Storages/StorageBuffer.cpp:595:9
        9 0x26d1c997 in DB::SinkToStorage::transform(DB::Chunk&) obj-x86_64-linux-gnu/../src/Processors/Sinks/SinkToStorage.cpp:18:5

    0x604000615d20 is located 16 bytes inside of 40-byte region [0x604000615d10,0x604000615d38)
    freed by thread T37 (QueryPipelineEx) here:
        2 0x22cb9392 in boost::intrusive_ptr<DB::ColumnVector<unsigned long> >::~intrusive_ptr() obj-x86_64-linux-gnu/../contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:98:23
        4 0x22cb9392 in COW<DB::IColumn>::mutable_ptr<DB::ColumnVector<unsigned long> >::operator=() obj-x86_64-linux-gnu/../src/Common/COW.h💯57
        5 0x22cb9392 in DB::ReverseIndex<>::tryGetSavedHash() const obj-x86_64-linux-gnu/../src/Columns/ReverseIndex.h:362:28
        6 0x22cb9392 in DB::ColumnUnique<DB::ColumnString>::tryGetSavedHash() const obj-x86_64-linux-gnu/../src/Columns/ColumnUnique.h:125:76
        7 0x242eaed3 in DB::ColumnsHashing::HashMethodSingleLowCardinalityColumn<>::HashMethodSingleLowCardinalityColumn() obj-x86_64-linux-gnu/../src/Common/ColumnsHashing.h:287:50
        8 0x242206c6 in void DB::Aggregator::executeImpl<>() const obj-x86_64-linux-gnu/../src/Interpreters/Aggregator.cpp:596:28
        9 0x24148e99 in DB::Aggregator::executeOnBlock() const obj-x86_64-linux-gnu/../src/Interpreters/Aggregator.cpp:1004:9
        10 0x26c24f3a in DB::AggregatingTransform::consume(DB::Chunk) obj-x86_64-linux-gnu/../src/Processors/Transforms/AggregatingTransform.cpp:539:33
        11 0x26c2054e in DB::AggregatingTransform::work() obj-x86_64-linux-gnu/../src/Processors/Transforms/AggregatingTransform.cpp:500:9

    previously allocated by thread T37 (QueryPipelineEx) here:
        0 0xb6d44fd in operator new(unsigned long) (/src/ch/tmp/upstream/clickhouse-asan+0xb6d44fd)
        1 0x11b78580 in COW<DB::IColumn>::mutable_ptr<> COWHelper<>::create<unsigned long&>(unsigned long&) (/src/ch/tmp/upstream/clickhouse-asan+0x11b78580)
        2 0x22cbf7b1 in DB::ReverseIndex<>::calcHashes() const obj-x86_64-linux-gnu/../src/Columns/ReverseIndex.h:472:17
        3 0x22cc2307 in DB::ReverseIndex<>::buildIndex() obj-x86_64-linux-gnu/../src/Columns/ReverseIndex.h:438:22
        4 0x22cc658c in DB::ReverseIndex<>::insert(StringRef const&) obj-x86_64-linux-gnu/../src/Columns/ReverseIndex.h:484:9
        5 0x22cc0de1 in COW<DB::IColumn>::mutable_ptr<DB::IColumn> DB::ColumnUnique<DB::ColumnString>::uniqueInsertRangeImpl<char8_t>()::'lambda'()::operator()() const obj-x86_64-linux-gnu/../src/Columns/ColumnUnique.h:540:39
        6 0x22cc0de1 in COW<DB::IColumn>::mutable_ptr<DB::IColumn> DB::ColumnUnique<DB::ColumnString>::uniqueInsertRangeImpl<char8_t>() obj-x86_64-linux-gnu/../src/Columns/ColumnUnique.h:570:23
        7 0x22cb9c66 in COW<DB::IColumn>::mutable_ptr<DB::IColumn> DB::ColumnUnique<DB::ColumnString>::uniqueInsertRangeFrom()::'lambda'(auto)::operator()<char8_t>(auto) const obj-x86_64-linux-gnu/../src/Columns/ColumnUnique.h:592:26
        8 0x22cb9c66 in DB::ColumnUnique<DB::ColumnString>::uniqueInsertRangeFrom() obj-x86_64-linux-gnu/../src/Columns/ColumnUnique.h:600:28
        9 0x2500b897 in DB::ColumnLowCardinality::insertRangeFrom() obj-x86_64-linux-gnu/../src/Columns/ColumnLowCardinality.cpp:205:62
        10 0x25a182f4 in DB::appendBlock(DB::Block const&, DB::Block&) obj-x86_64-linux-gnu/../src/Storages/StorageBuffer.cpp:470:23
        11 0x25a182f4 in DB::BufferSink::insertIntoBuffer(DB::Block const&, DB::StorageBuffer::Buffer&) obj-x86_64-linux-gnu/../src/Storages/StorageBuffer.cpp:634:9
        12 0x25a173cc in DB::BufferSink::consume(DB::Chunk) obj-x86_64-linux-gnu/../src/Storages/StorageBuffer.cpp:595:9
        13 0x26d1c997 in DB::SinkToStorage::transform(DB::Chunk&) obj-x86_64-linux-gnu/../src/Processors/Sinks/SinkToStorage.cpp:18:5

    SUMMARY: AddressSanitizer: heap-use-after-free obj-x86_64-linux-gnu/../src/Columns/ReverseIndex.h in DB::ReverseIndex<unsigned long, DB::ColumnString>::insert(StringRef const&)
    Shadow bytes around the buggy address:
      0x0c08800bab50: fa fa fd fd fd fd fd fa fa fa 00 00 00 00 00 00
      0x0c08800bab60: fa fa fd fd fd fd fd fa fa fa 00 00 00 00 00 fa
      0x0c08800bab70: fa fa 00 00 00 00 00 00 fa fa 00 00 00 00 00 00
      0x0c08800bab80: fa fa fd fd fd fd fd fa fa fa fd fd fd fd fd fa
      0x0c08800bab90: fa fa fd fd fd fd fd fa fa fa fd fd fd fd fd fd
    =>0x0c08800baba0: fa fa fd fd[fd]fd fd fa fa fa fd fd fd fd fd fa
      0x0c08800babb0: fa fa fd fd fd fd fd fd fa fa fd fd fd fd fd fa
      0x0c08800babc0: fa fa fd fd fd fd fd fa fa fa fd fd fd fd fd fd
      0x0c08800babd0: fa fa fd fd fd fd fd fd fa fa fd fd fd fd fd fa
      0x0c08800babe0: fa fa fd fd fd fd fd fa fa fa fd fd fd fd fd fd
      0x0c08800babf0: fa fa fd fd fd fd fd fd fa fa fd fd fd fd fd fd
    Shadow byte legend (one shadow byte represents 8 application bytes):
      Addressable:           00
      Partially addressable: 01 02 03 04 05 06 07
      Heap left redzone:       fa
      Freed heap region:       fd
      Stack left redzone:      f1
      Stack mid redzone:       f2
      Stack right redzone:     f3
      Stack after return:      f5
      Stack use after scope:   f8
      Global redzone:          f9
      Global init order:       f6
      Poisoned by user:        f7
      Container overflow:      fc
      Array cookie:            ac
      Intra object redzone:    bb
      ASan internal:           fe
      Left alloca redzone:     ca
      Right alloca redzone:    cb
    ==24679==ABORTING

</details>
2021-10-05 21:07:10 +03:00
Azat Khuzhin
aeb670dbe0 Simplify obtaining of server pid in fuzzer tests 2021-10-05 21:05:49 +03:00
Kruglov Pavel
fa81606d6a
Fix build 2021-10-05 20:11:21 +03:00
Nikita Mikhaylov
cc150b7316
Merge pull request #29596 from nikitamikhaylov/fuzz
Add fuzzer for `executeQuery` function
2021-10-05 20:10:13 +03:00
Maksim Kita
4fde987ad1 Parse query from metadata exception throw fix 2021-10-05 19:34:25 +03:00
Anton Popov
b5098301d6 fix build 2021-10-05 18:23:53 +03:00
Mike Kot
25a62b9f83 Renaming TL to Typelist 2021-10-05 17:22:02 +02:00
Artur Filatenkov
9b1a39fdb9 refactor after move common to base 2021-10-05 17:43:33 +03:00
Artur Filatenkov
8795c10309 Merge branch 'add_CORS' of https://github.com/FArthur-cmd/ClickHouse into add_CORS 2021-10-05 17:27:42 +03:00