thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-11-27 18:12:02 +00:00
Code Issues Packages Projects Releases Wiki Activity
74,628 Commits 9,188 Branches 1,782 Tags 2.1 GiB
e66735e21e
Commit Graph

18778 Commits

Author SHA1 Message Date
Maksim Kita
717998a37f
Merge pull request #29825 from azat/memory_profiler_step-cleaner-api 
Make memory_profiler_step API cleaner
 2021-10-07 22:44:47 +03:00
tavplubix
de2fd9d762
Merge pull request #29812 from ClickHouse/better_diagnostic_for_optimize 
Better diagnostic for OPTIMIZE
 2021-10-07 22:22:25 +03:00
Azat Khuzhin
bf88f102fe Add ability to trace peak memory usage (with new trace_type - MemoryPeak) 2021-10-07 21:13:56 +03:00
Azat Khuzhin
1b8e3b6b96 Fix system tables recreation check (fails to detect changes in enum values) 
Fixes: #23934 (cc @kitaisreal)
 2021-10-07 21:12:43 +03:00
Nikolai Kochetov
213d68d995 Fix some tests. 2021-10-07 20:27:13 +03:00
Kruglov Pavel
1ac03811a2
Merge pull request #29811 from ClickHouse/fix_overflow_in_stopwatch 
Fix overflow in Stopwatch
 2021-10-07 20:14:26 +03:00
Kseniia Sumarokova
185d58a7cc
Merge pull request #29766 from ClickHouse/remove_redundant_seeks 
Less seeks in compressed buffers
 2021-10-07 17:04:50 +03:00
Nikolai Kochetov
d24bfce93f Add coroutines example. 2021-10-07 17:03:54 +03:00
Alexander Tokmakov
7e2bc184ec fix another suspicious places, add test 2021-10-07 16:43:49 +03:00
Nikolai Kochetov
25e2ebac75
Merge pull request #29782 from azat/lc-concurrent-GROUP-BY-fix 
Fix concurrent access to LowCardinality during GROUP BY (leads to SIGSEGV)
 2021-10-07 16:27:53 +03:00
tavplubix
754e038eec
Merge pull request #29803 from ClickHouse/intersects-previous-part-usability 
Improve usability of error messages when error is caused by sophisticated interventions
 2021-10-07 15:24:17 +03:00
Haavard Kvaalen
c41923c595 MaterializedMySQL: Update GTID set at end of transaction 
We would update the set of seen GTIDs as soon as we saw a GTID_EVENT,
which arrives before a transaction.  This would mostly work fine, but
if we lost the connection to MySQL in the middle of a large transaction
we would persist that the transaction had been processed as soon as the
transaction had started.  When the connection was reestablished, we
would not process the transaction again, which meant that we only
applied parts of it.

Fix this by updating the seen GTIDs at the end of the transaction
instead.
 2021-10-07 14:09:00 +02:00
Anton Popov
4bc14dedfb
Merge pull request #28352 from Avogar/div-null 
Avoid division by zero when denominator is Nullable
 2021-10-07 14:47:53 +03:00
Kruglov Pavel
13b2fdc23b
Merge pull request #29659 from amosbird/jsonfix3 
Fix nullable processing in JSONFunctions
 2021-10-07 13:55:22 +03:00
Nikolai Kochetov
db79dc2a3d
Revert "Add coroutines example." 2021-10-07 13:11:51 +03:00
Nikolai Kochetov
8263d30321
Merge pull request #29827 from ClickHouse/coro-example 
Add coroutines example.
 2021-10-07 13:08:16 +03:00
Alexander Tokmakov
4119848a41 better diagnostic for OPTIMIZE 2021-10-07 13:08:10 +03:00
Nikolai Kochetov
94a29545c8 Add logger. 2021-10-07 13:06:59 +03:00
Shoh Jahon
298f613557
Update fetchPostgreSQLTableStructure.cpp 
added partitioned table prefix 'p' for the query for fetching replica identity index @kssenii !
 2021-10-07 14:59:23 +05:00
Vladimir C
0422939893
Merge pull request #29281 from lingtaolf/bugfix/tupleFilter 2021-10-07 12:24:26 +03:00
Nikolai Kochetov
3e518861a0 Add coroutines example. 2021-10-07 12:23:22 +03:00
Nikolai Kochetov
4ab2e2745b
Merge pull request #29786 from azat/parallel_view_processing-fix 
Fix parallel_view_processing
 2021-10-07 12:15:26 +03:00
Nikolai Kochetov
78e1db209f
Remove more data streams (#29491) 
* Remove more streams.

* Fixing build.

* Fixing build.

* Rename files.

* Fix fast test.

* Fix StorageKafka.

* Try fix kafka test.

* Move createBuffer to KafkaSource ctor.

* Revert "Move createBuffer to KafkaSource ctor."

This reverts commit 81fa94d27e.

* Revert "Try fix kafka test."

This reverts commit 2107e54969.

* Comment some rows in test.

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2021-10-07 11:26:08 +03:00
Vladimir C
b02c12b182
Merge pull request #23881 from darkkeks/fix-cross-to-inline-join-error-condition 2021-10-07 11:09:36 +03:00
Vladimir C
4b80c6ff21
Merge pull request #29545 from CurtizJ/remove-permute 2021-10-07 11:07:34 +03:00
Azat Khuzhin
71cffbf521 Make memory_profiler_step API cleaner 
Right now to configure memory_profiler_step/total_memory_profiler_step
you need to call:

    MemoryTracker::setOrRaiseProfilerLimit()
    MemoryTracker::setProfilerStep()

But it is easy to forget about setOrRaiseProfilerLimit(), since there is
no even any comments about this.

So instead, make setOrRaiseProfilerLimit() private and call it from
setProfilerStep()
 2021-10-07 10:52:16 +03:00
Maksim Kita
d91377c993
Merge pull request #29816 from kitaisreal/dictionary-attributes-updated-documentation 
Dictionary attributes updated documentation
 2021-10-07 10:21:15 +03:00
Maksim Kita
4ec7311d4d
Merge pull request #29738 from kitaisreal/added-function-ngram 
Added function ngram
 2021-10-07 10:21:00 +03:00
alexey-milovidov
16ad5953d6
Merge pull request #29783 from azat/tcphandler-progress-race 
Fix data-race between fatal error handler and progress packets
 2021-10-07 01:47:29 +03:00
avogar
568e27e9b3 Fix tests 2021-10-07 00:19:37 +03:00
Azat Khuzhin
9515430517 Fix compilation with glibc 2.34 (MINSIGSTKSZ defined as sysconf(_SC_SIGSTKSZ)) 
In glibc 2.34 MINSIGSTKSZ had been defined to sysconf(_SC_SIGSTKSZ) [1].

  [1]: https://sourceware.org/git/?p=glibc.git;a=commit;h=6c57d320484988e87e446e2e60ce42816bf51d53
 2021-10-07 00:18:56 +03:00
Maksim Kita
c2c621c6dc Dictionary attributes updated documentation 2021-10-06 23:33:36 +03:00
Maksim Kita
b7b39da8cc Fixed build 2021-10-06 23:22:29 +03:00
Maksim Kita
dce50c8b8a
Merge pull request #29779 from kitaisreal/parse-query-from-metadata-exception-throw-fix 
Parse query from metadata exception throw fix
 2021-10-06 21:29:12 +03:00
Amos Bird
a34a08268b
Fix nullable processing in JSONFunctions 2021-10-07 02:26:50 +08:00
Amos Bird
a391b64f57
Revert "Merge pull request #29746 from azat/JSONExtract-Null-fix" 
This reverts commit e990f35ba3, reversing
changes made to 8e03e0127a.
 2021-10-07 02:26:32 +08:00
Azat Khuzhin
83c271f446 Move QueryPipelineBuilder::setMaxThreads() into the InterpreterInsertQuery 2021-10-06 21:25:37 +03:00
Nikolai Kochetov
d0c6f11fcb More. 2021-10-06 20:59:27 +03:00
Alexander Tokmakov
c5d631ca54 fix overflow in Stopwatch 2021-10-06 18:09:13 +03:00
Filatenkov Artur
545673248a
Merge branch 'master' into add_CORS 2021-10-06 16:09:34 +03:00
Alexander Tokmakov
c0ee6d46f7 resolve dependency of StorageDictionary 2021-10-06 15:39:14 +03:00
Alexey Milovidov
1455cfeddd Improve usability of error messages when error is caused by sophisticated interventions 2021-10-06 15:28:52 +03:00
Maksim Kita
33353be4ed Fixed tests 2021-10-06 15:12:07 +03:00
Filatenkov Artur
a9e8ba0c00
Correct behaviour with unknown methods (#29057) 
* correct behaviour with unknown methods

* move check on earlier stage

* add test

* correct filter for handler

* Update HTTPServerConnection.cpp

* remove trying to get params

* correct filter for head requests

* Update HTTPHandlerFactory.h

* Update HTTPHandlerFactory.h

* Trigger Build
 2021-10-06 14:43:05 +03:00
Maksim Kita
d2b3f3bd11
Merge pull request #29796 from kitaisreal/function-reinterpet-as-improve-readability 
Function reinterpretAs improve readability
 2021-10-06 13:44:24 +03:00
avogar
5e4cda02b6 Fix build 2021-10-06 13:38:22 +03:00
alesapin
621c9884fc Account offset in readBig call 2021-10-06 12:39:54 +03:00
Nikolai Kochetov
6b619512aa Fix style. 2021-10-06 12:37:56 +03:00
Filatenkov Artur
bf8a2b109f
Merge branch 'ClickHouse:master' into add_CORS 2021-10-06 11:57:29 +03:00
Maksim Kita
490ef61a03 Function reinterpretAs improve readability 2021-10-06 11:43:11 +03:00
Alexey Milovidov
bd4b0af2e1 Fix bad cast in ParserCreateQuery 2021-10-06 02:34:48 +03:00
Maksim Kita
90eb74c933 Fixed tests 2021-10-06 00:37:45 +03:00
Kseniia Sumarokova
3663d23e4d
Merge pull request #29731 from kssenii/exists-subquery 
Support EXISTS(subquery)
 2021-10-05 22:26:19 +03:00
Azat Khuzhin
99ebdbb16e Fix parallel_view_processing 
Number of threads got lost during after QueryPipelineBuilder::addChains()
 2021-10-05 22:20:46 +03:00
Anton Popov
199ac31b48
Merge pull request #29683 from myrrc/improvement/tl-to-common 
Non-recursive implementation for type list and its functions
 2021-10-05 21:49:15 +03:00
Azat Khuzhin
4f5118b85f Fix data-race between fatal error handler and progress packets 
This is not that important, but because of packets overlaps the client
may not render the fatal error.

You will find TSan report in `details`.

<details>

```
WARNING: ThreadSanitizer: data race (pid=13384)
  Write of size 8 at 0x7b24000ad818 by thread T47:
    0 DB::WriteBuffer::next() obj-x86_64-linux-gnu/../src/IO/WriteBuffer.h:43:15 (clickhouse-tsan+0x16124d61)
    1 DB::TCPHandler::sendProgress() obj-x86_64-linux-gnu/../src/Server/TCPHandler.cpp:1602:10 (clickhouse-tsan+0x16124d61)
    2 DB::TCPHandler::runImpl()::$_6::operator()() const obj-x86_64-linux-gnu/../src/Server/TCPHandler.cpp:318:25 (clickhouse-tsan+0x1612cf54)

  Previous write of size 8 at 0x7b24000ad818 by thread T229:
    0 DB::WriteBuffer::next() obj-x86_64-linux-gnu/../src/IO/WriteBuffer.h:43:15 (clickhouse-tsan+0x1612b793)
    1 DB::TCPHandler::sendLogData(DB::Block const&) obj-x86_64-linux-gnu/../src/Server/TCPHandler.cpp:1561:10 (clickhouse-tsan+0x1612b793)
    2 DB::TCPHandler::sendLogs() obj-x86_64-linux-gnu/../src/Server/TCPHandler.cpp:1632:9 (clickhouse-tsan+0x161228ab)
    3 DB::TCPHandler::runImpl()::$_1::operator()() const obj-x86_64-linux-gnu/../src/Server/TCPHandler.cpp:229:62 (clickhouse-tsan+0x1612c4e1)
```

</details>
 2021-10-05 21:21:32 +03:00
Azat Khuzhin
342ec02664 Fix concurrent access to LowCardinality during GROUP BY (leads to SIGSEGV) 
The problem is that GROUP BY can update saved_hash, which can be also
updated by subsequent update of a dictionary, and this will lead to
use-after-free.

You will find ASan report in `details`.

<details>

    ==24679==ERROR: AddressSanitizer: heap-use-after-free on address 0x604000615d20 at pc 0x000022cc8684 bp 0x7ffea6b5f850 sp 0x7ffea6b5f848
    READ of size 8 at 0x604000615d20 thread T223 (QueryPipelineEx)
        0 0x22cc8683 in DB::ReverseIndex<>::insert(StringRef const&) obj-x86_64-linux-gnu/../src/Columns/ReverseIndex.h
        1 0x22cc0de1 in COW<DB::IColumn>::mutable_ptr<DB::IColumn> DB::ColumnUnique<DB::ColumnString>::uniqueInsertRangeImpl<char8_t>()::'lambda'()::operator()() const obj-x86_64-linux-gnu/../src/Columns/ColumnUnique.h:540:39
        2 0x22cc0de1 in COW<DB::IColumn>::mutable_ptr<DB::IColumn> DB::ColumnUnique<DB::ColumnString>::uniqueInsertRangeImpl<char8_t>() obj-x86_64-linux-gnu/../src/Columns/ColumnUnique.h:570:23
        3 0x22cb9c66 in COW<DB::IColumn>::mutable_ptr<DB::IColumn> DB::ColumnUnique<DB::ColumnString>::uniqueInsertRangeFrom()::'lambda'(auto)::operator()<char8_t>(auto) const obj-x86_64-linux-gnu/../src/Columns/ColumnUnique.h:592:26
        4 0x22cb9c66 in DB::ColumnUnique<DB::ColumnString>::uniqueInsertRangeFrom() obj-x86_64-linux-gnu/../src/Columns/ColumnUnique.h:600:28
        5 0x2500b897 in DB::ColumnLowCardinality::insertRangeFrom() obj-x86_64-linux-gnu/../src/Columns/ColumnLowCardinality.cpp:205:62
        6 0x25a182f4 in DB::appendBlock(DB::Block const&, DB::Block&) obj-x86_64-linux-gnu/../src/Storages/StorageBuffer.cpp:470:23
        7 0x25a182f4 in DB::BufferSink::insertIntoBuffer(DB::Block const&, DB::StorageBuffer::Buffer&) obj-x86_64-linux-gnu/../src/Storages/StorageBuffer.cpp:634:9
        8 0x25a173cc in DB::BufferSink::consume(DB::Chunk) obj-x86_64-linux-gnu/../src/Storages/StorageBuffer.cpp:595:9
        9 0x26d1c997 in DB::SinkToStorage::transform(DB::Chunk&) obj-x86_64-linux-gnu/../src/Processors/Sinks/SinkToStorage.cpp:18:5

    0x604000615d20 is located 16 bytes inside of 40-byte region [0x604000615d10,0x604000615d38)
    freed by thread T37 (QueryPipelineEx) here:
        2 0x22cb9392 in boost::intrusive_ptr<DB::ColumnVector<unsigned long> >::~intrusive_ptr() obj-x86_64-linux-gnu/../contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:98:23
        4 0x22cb9392 in COW<DB::IColumn>::mutable_ptr<DB::ColumnVector<unsigned long> >::operator=() obj-x86_64-linux-gnu/../src/Common/COW.h💯57
        5 0x22cb9392 in DB::ReverseIndex<>::tryGetSavedHash() const obj-x86_64-linux-gnu/../src/Columns/ReverseIndex.h:362:28
        6 0x22cb9392 in DB::ColumnUnique<DB::ColumnString>::tryGetSavedHash() const obj-x86_64-linux-gnu/../src/Columns/ColumnUnique.h:125:76
        7 0x242eaed3 in DB::ColumnsHashing::HashMethodSingleLowCardinalityColumn<>::HashMethodSingleLowCardinalityColumn() obj-x86_64-linux-gnu/../src/Common/ColumnsHashing.h:287:50
        8 0x242206c6 in void DB::Aggregator::executeImpl<>() const obj-x86_64-linux-gnu/../src/Interpreters/Aggregator.cpp:596:28
        9 0x24148e99 in DB::Aggregator::executeOnBlock() const obj-x86_64-linux-gnu/../src/Interpreters/Aggregator.cpp:1004:9
        10 0x26c24f3a in DB::AggregatingTransform::consume(DB::Chunk) obj-x86_64-linux-gnu/../src/Processors/Transforms/AggregatingTransform.cpp:539:33
        11 0x26c2054e in DB::AggregatingTransform::work() obj-x86_64-linux-gnu/../src/Processors/Transforms/AggregatingTransform.cpp:500:9

    previously allocated by thread T37 (QueryPipelineEx) here:
        0 0xb6d44fd in operator new(unsigned long) (/src/ch/tmp/upstream/clickhouse-asan+0xb6d44fd)
        1 0x11b78580 in COW<DB::IColumn>::mutable_ptr<> COWHelper<>::create<unsigned long&>(unsigned long&) (/src/ch/tmp/upstream/clickhouse-asan+0x11b78580)
        2 0x22cbf7b1 in DB::ReverseIndex<>::calcHashes() const obj-x86_64-linux-gnu/../src/Columns/ReverseIndex.h:472:17
        3 0x22cc2307 in DB::ReverseIndex<>::buildIndex() obj-x86_64-linux-gnu/../src/Columns/ReverseIndex.h:438:22
        4 0x22cc658c in DB::ReverseIndex<>::insert(StringRef const&) obj-x86_64-linux-gnu/../src/Columns/ReverseIndex.h:484:9
        5 0x22cc0de1 in COW<DB::IColumn>::mutable_ptr<DB::IColumn> DB::ColumnUnique<DB::ColumnString>::uniqueInsertRangeImpl<char8_t>()::'lambda'()::operator()() const obj-x86_64-linux-gnu/../src/Columns/ColumnUnique.h:540:39
        6 0x22cc0de1 in COW<DB::IColumn>::mutable_ptr<DB::IColumn> DB::ColumnUnique<DB::ColumnString>::uniqueInsertRangeImpl<char8_t>() obj-x86_64-linux-gnu/../src/Columns/ColumnUnique.h:570:23
        7 0x22cb9c66 in COW<DB::IColumn>::mutable_ptr<DB::IColumn> DB::ColumnUnique<DB::ColumnString>::uniqueInsertRangeFrom()::'lambda'(auto)::operator()<char8_t>(auto) const obj-x86_64-linux-gnu/../src/Columns/ColumnUnique.h:592:26
        8 0x22cb9c66 in DB::ColumnUnique<DB::ColumnString>::uniqueInsertRangeFrom() obj-x86_64-linux-gnu/../src/Columns/ColumnUnique.h:600:28
        9 0x2500b897 in DB::ColumnLowCardinality::insertRangeFrom() obj-x86_64-linux-gnu/../src/Columns/ColumnLowCardinality.cpp:205:62
        10 0x25a182f4 in DB::appendBlock(DB::Block const&, DB::Block&) obj-x86_64-linux-gnu/../src/Storages/StorageBuffer.cpp:470:23
        11 0x25a182f4 in DB::BufferSink::insertIntoBuffer(DB::Block const&, DB::StorageBuffer::Buffer&) obj-x86_64-linux-gnu/../src/Storages/StorageBuffer.cpp:634:9
        12 0x25a173cc in DB::BufferSink::consume(DB::Chunk) obj-x86_64-linux-gnu/../src/Storages/StorageBuffer.cpp:595:9
        13 0x26d1c997 in DB::SinkToStorage::transform(DB::Chunk&) obj-x86_64-linux-gnu/../src/Processors/Sinks/SinkToStorage.cpp:18:5

    SUMMARY: AddressSanitizer: heap-use-after-free obj-x86_64-linux-gnu/../src/Columns/ReverseIndex.h in DB::ReverseIndex<unsigned long, DB::ColumnString>::insert(StringRef const&)
    Shadow bytes around the buggy address:
      0x0c08800bab50: fa fa fd fd fd fd fd fa fa fa 00 00 00 00 00 00
      0x0c08800bab60: fa fa fd fd fd fd fd fa fa fa 00 00 00 00 00 fa
      0x0c08800bab70: fa fa 00 00 00 00 00 00 fa fa 00 00 00 00 00 00
      0x0c08800bab80: fa fa fd fd fd fd fd fa fa fa fd fd fd fd fd fa
      0x0c08800bab90: fa fa fd fd fd fd fd fa fa fa fd fd fd fd fd fd
    =>0x0c08800baba0: fa fa fd fd[fd]fd fd fa fa fa fd fd fd fd fd fa
      0x0c08800babb0: fa fa fd fd fd fd fd fd fa fa fd fd fd fd fd fa
      0x0c08800babc0: fa fa fd fd fd fd fd fa fa fa fd fd fd fd fd fd
      0x0c08800babd0: fa fa fd fd fd fd fd fd fa fa fd fd fd fd fd fa
      0x0c08800babe0: fa fa fd fd fd fd fd fa fa fa fd fd fd fd fd fd
      0x0c08800babf0: fa fa fd fd fd fd fd fd fa fa fd fd fd fd fd fd
    Shadow byte legend (one shadow byte represents 8 application bytes):
      Addressable:           00
      Partially addressable: 01 02 03 04 05 06 07
      Heap left redzone:       fa
      Freed heap region:       fd
      Stack left redzone:      f1
      Stack mid redzone:       f2
      Stack right redzone:     f3
      Stack after return:      f5
      Stack use after scope:   f8
      Global redzone:          f9
      Global init order:       f6
      Poisoned by user:        f7
      Container overflow:      fc
      Array cookie:            ac
      Intra object redzone:    bb
      ASan internal:           fe
      Left alloca redzone:     ca
      Right alloca redzone:    cb
    ==24679==ABORTING

</details>
 2021-10-05 21:07:10 +03:00
Kruglov Pavel
fa81606d6a
Fix build 2021-10-05 20:11:21 +03:00
Nikita Mikhaylov
cc150b7316
Merge pull request #29596 from nikitamikhaylov/fuzz 
Add fuzzer for `executeQuery` function
 2021-10-05 20:10:13 +03:00
Maksim Kita
4fde987ad1 Parse query from metadata exception throw fix 2021-10-05 19:34:25 +03:00
Anton Popov
b5098301d6 fix build 2021-10-05 18:23:53 +03:00
Mike Kot
25a62b9f83 Renaming TL to Typelist 2021-10-05 17:22:02 +02:00
Artur Filatenkov
9b1a39fdb9 refactor after move common to base 2021-10-05 17:43:33 +03:00
Filatenkov Artur
60d84df56a
Merge branch 'ClickHouse:master' into add_CORS 2021-10-05 17:27:13 +03:00
mergify[bot]
ab55225c8f
Merge branch 'master' into remove-merging-streams 2021-10-05 14:16:34 +00:00
Nikolai Kochetov
96d070a5ba Fix tests. 2021-10-05 16:58:24 +03:00
Filatenkov Artur
dc657a2c5e
Add error for multiple keys without current_key_id (#29546) 
* add error for multiple keys without current_key_id

* add exception if one key with non zero id is used without current_key_id

* refactoring
 2021-10-05 14:37:54 +03:00
Nikita Mikhaylov
f3c50e563e -nostdlib++ 2021-10-05 10:51:49 +00:00
alesapin
9aab0eb0a2 Fixup 2021-10-05 13:41:09 +03:00
Filatenkov Artur
82e6ac8fa2
Update HTTPHandler.cpp 2021-10-05 13:39:18 +03:00
alesapin
64158c011b Remove logging 2021-10-05 13:12:31 +03:00
Kruglov Pavel
e990f35ba3
Merge pull request #29746 from azat/JSONExtract-Null-fix 
Forbid Nullable for JSONExtract* (JSONExtract() still supports Nullable), leads to SIGSEGV before
 2021-10-05 13:04:46 +03:00
alesapin
c2f34928e0 Fixup 2021-10-05 13:04:44 +03:00
Nikolai Kochetov
c106f1e380 Fix tests. 2021-10-05 12:51:43 +03:00
alesapin
3a8126372c Remove redundant header 2021-10-05 12:19:04 +03:00
alesapin
8186339307 Less seeks in compressed buffers 2021-10-05 12:11:25 +03:00
avogar
c2cb9e2721 Fix use of unititialized value 2021-10-05 11:55:26 +03:00
Kseniia Sumarokova
438a981e76
Update TreeRewriter.cpp 2021-10-05 11:25:50 +03:00
Kseniia Sumarokova
385bf579a6
Merge pull request #29745 from azat/clickhouse-local-fix-stage 
Fix --stage for clickhouse-local
 2021-10-05 11:14:27 +03:00
Vitaly Baranov
8a01b32cba
Merge pull request #28637 from vitlibar/fix-materialized-column-as-sharding-key 
Fix materialized column as sharding key
 2021-10-05 10:53:24 +03:00
Memo
0e9bbc4d43
Merge branch 'ClickHouse:master' into new_func 2021-10-05 09:02:37 +08:00
WangZengrui
a638c40fdc add getOSKernelVersion 2021-10-05 09:08:16 +08:00
Nikolai Kochetov
2001ebbf9d Fix build. 2021-10-04 21:52:31 +03:00
Maksim Kita
6a6946018b
Merge pull request #29747 from kitaisreal/table-function-dictionary-fix-comment 
TableFunctionDictionary fix comment
 2021-10-04 21:10:20 +03:00
Maksim Kita
6df81d94bf TableFunctionDictionary fix comment 2021-10-04 21:09:32 +03:00
Azat Khuzhin
20e706766c Forbid Nullable for JSONExtract* (JSONExtract() still supports Nullable) 
Only JSONExtract() can support Nullable, others JSONExtract*
(JSONExtractString and similar) does not.

And right now this file is pretty complex already,
so adding support of Nullable for others will make it even more complex.

CI: https://clickhouse-test-reports.s3.yandex.net/29680/d0fc26f91a0141b56a0550741219c3dc43630e03/fuzzer_ubsan/report.html#fail1
 2021-10-04 21:08:02 +03:00
Azat Khuzhin
bbee102e58 Fix --stage for clickhouse-local 
This also fixes UBsan error, since query_processing_stage was not
initialized before for clickhouse-local.
 2021-10-04 21:00:48 +03:00
Nikita Mikhaylov
ec99357097 Done 2021-10-04 17:57:58 +00:00
Nikita Mikhaylov
613b814e24
Merge pull request #29595 from CurtizJ/generalize-update-permutation 
Generalize code in `IColumn::updatePermutation`
 2021-10-04 20:36:57 +03:00
Nikita Mikhaylov
b31d11478b
Merge pull request #29680 from azat/defines 
Cleanup common defines
 2021-10-04 20:13:29 +03:00
Nikita Mikhaylov
6d5e9a9089
Merge pull request #29692 from azat/disk-memory-data-race-fix 
Fix data-race in WriteIndirectBuffer (used in DiskMemory)
 2021-10-04 20:13:18 +03:00
Nikita Mikhaylov
36516dd503
Merge pull request #29681 from azat/MemoryTracker-mutate-merge 
Apply max_untracked_memory/memory_profiler_step/memory_profiler_sample_probability during mutate/merge
 2021-10-04 20:04:57 +03:00
Maksim Kita
3fe9fdac96 Added function ngram 2021-10-04 18:53:17 +03:00
avogar
2523053b27 Remove code duplication, fix bug, add more tests 2021-10-04 18:22:06 +03:00
Nikolai Kochetov
122d39cbba
Fix missing condition in pushed down predicate. (#29625) 2021-10-04 17:50:03 +03:00
Pavel Kruglov
ae2754ee56 Fix tests, add comments 2021-10-04 16:44:38 +03:00
Pavel Kruglov
2c4453f425 Rename template argument 2021-10-04 16:44:38 +03:00
Pavel Kruglov
82d17870f1 Avoid division by zero when denominator is Nullable 2021-10-04 16:44:38 +03:00
kssenii
f1ea186bff Exists 2021-10-04 13:12:30 +00:00
Anton Popov
362e84a336
Merge pull request #29429 from Enmk/Governance/database_comment 
Implemented creating databases with comments
 2021-10-04 15:12:33 +03:00
Anton Popov
cd08057285 better arguments 2021-10-04 15:02:11 +03:00
Maksim Kita
8eda58acfb
Merge pull request #29516 from lingtaolf/feature/support-HSTS 
Support HSTS in Clickhouse HTTP server
 2021-10-04 12:49:51 +03:00
Nikolay Degterinsky
345ac59f46
Merge pull request #29552 from MaxWk/conversion-between-date32-and-datetime64 
Support conversion between date32 and datetime64
 2021-10-04 12:43:56 +03:00
Vitaly Baranov
17fe76709f
Merge pull request #28331 from vitlibar/mysql-authentication-cleanup 
Clean up MySQL authentication.
 2021-10-04 11:11:45 +03:00
Vitaly Baranov
1636ee24bb Fix using materialized column as sharding key. 2021-10-04 10:56:42 +03:00
Filatenkov Artur
8df228e6b8
Merge branch 'master' into add_CORS 2021-10-04 10:55:09 +03:00
Maksim Kita
25d1588d0a
Merge pull request #29666 from kitaisreal/user-defined-executable-functions-added-implicit-cast 
UserDefinedExecutableFunctions added implicit cast
 2021-10-04 10:34:31 +03:00
Kseniia Sumarokova
5ab17d795f
Merge pull request #29673 from ClickHouse/url-fix-initialization 
Fix initialization of `StorageURLSource`
 2021-10-04 10:23:52 +03:00
Kseniia Sumarokova
0b2dd42261
Merge pull request #29686 from kssenii/fix-materialized-postgresql-3 
MaterializedPostgreSQL fix
 2021-10-04 09:47:21 +03:00
alexey-milovidov
325a01f798
Merge pull request #29553 from azat/group-by-with-totals-having-fix 
Fix null deference for GROUP BY WITH TOTALS HAVING (when the column from HAVING wasn't selected)
 2021-10-04 09:25:35 +03:00
alexey-milovidov
6abb3d4c6d
Merge pull request #29682 from traceon/xcode-13-fix-build 
Fix Xcode 13 build
 2021-10-04 02:34:43 +03:00
Azat Khuzhin
a7793101bc Fix data-race in WriteIndirectBuffer (used in StripeLog) 
CI founds [1]:

WARNING: ThreadSanitizer: data race (pid=497)
  Write of size 8 at 0x7b30000260b8 by thread T381 (mutexes: write M546759850860896464):
    ...
    3 DB::DiskMemory::writeFile() obj-x86_64-linux-gnu/../src/Disks/DiskMemory.cpp:339:15 (clickhouse+0x14a1849f)
    4 DB::FileChecker::save() const obj-x86_64-linux-gnu/../src/Common/FileChecker.cpp:111:50 (clickhouse+0x15bbe19f)
    5 DB::TinyLogSink::TinyLogSink() obj-x86_64-linux-gnu/../src/Storages/StorageTinyLog.cpp:214:34 (clickhouse+0x15bca215)

  Previous read of size 8 at 0x7b30000260b8 by thread T460:
    ...
    2 DB::WriteIndirectBuffer::finalize() obj-x86_64-linux-gnu/../src/Disks/DiskMemory.cpp:106:33 (clickhouse+0x14a1e044)
    9 DB::StripeLogSink::onFinish() obj-x86_64-linux-gnu/../src/Storages/StorageStripeLog.cpp:227:30 (clickhouse+0x15bd241a)

  [1]: https://clickhouse-test-reports.s3.yandex.net/29680/d0fc26f91a0141b56a0550741219c3dc43630e03/stress_test_(thread)/stderr.log
 2021-10-04 01:58:35 +03:00
Vitaly Baranov
30b9b8fd58 Clean up MySQL authentication. 2021-10-03 23:40:08 +03:00
Kseniia Sumarokova
ddc775b1c8
Merge pull request #29626 from kssenii/fix-signals 
Follow-up for #26231
 2021-10-03 22:13:54 +03:00
kssenii
dbc114e6fc Fix special build check 2021-10-03 16:49:02 +00:00
kssenii
7ea9037939 Fix 2021-10-03 16:41:12 +00:00
Mike Kot
8c8bada56e Non-recursive implementation for TL and its functions 2021-10-03 18:02:04 +02:00
Azat Khuzhin
117e9e77c8 Apply max_untracked_memory/memory_profiler_{step,sample_probability} during mutate/merge 2021-10-03 17:39:07 +03:00
kssenii
0cb7193fde Fix url with failover case 2021-10-03 13:58:03 +00:00
Denis Glazachev
ca7b69b0f3 Manipulate with -Wreserved-identifier only if HAS_RESERVED_IDENTIFIER has been detected 2021-10-03 17:42:36 +04:00
Azat Khuzhin
1fc19eebd7 Remove unused DBMS_DEFAULT_HOST 2021-10-03 14:34:03 +03:00
Azat Khuzhin
b43b4016d7 Remove unused DBMS_DEFAULT_HTTP_PORT 2021-10-03 14:34:03 +03:00
Azat Khuzhin
0b83345560 Remove macros for distributed connection timeouts 2021-10-03 14:34:03 +03:00
Azat Khuzhin
f6e3583ffa Remove macros for hedged request timeouts 2021-10-03 14:34:03 +03:00
Azat Khuzhin
0a6885d806 Remove DEFAULT_LIVE_VIEW_HEARTBEAT_INTERVAL_SEC 2021-10-03 14:34:03 +03:00
Azat Khuzhin
dc2cb6d996 Remove DBMS_DEFAULT_DISTRIBUTED_CONNECTIONS_POOL_SIZE 2021-10-03 14:34:03 +03:00
Azat Khuzhin
16c72f2107 Remove unused macros from ReadWriteBufferFromHTTP 2021-10-03 14:34:03 +03:00
Azat Khuzhin
a5a2c5ef8a Move KEEPER_DEFAULT_PATH into separate header 2021-10-03 14:34:03 +03:00
Azat Khuzhin
ae5ee2dd28 Move macros for distributed engine into separate header 2021-10-03 14:34:03 +03:00
Azat Khuzhin
6a9dd9828d Move protocol macros into separate header 
Defines.h is a very common header, so lots of modules will be recompiled
on changes.
Move macros for protocol into separate header, this should significantly
decreases number of units to compile on it's changes.
 2021-10-03 14:34:03 +03:00
Azat Khuzhin
70e1f4d402 Move ASAN_{UN,}POISON_MEMORY_REGION to base/defines 2021-10-03 14:34:03 +03:00
Azat Khuzhin
3d6df8f1a0 Fix typo in a comment 2021-10-03 14:34:03 +03:00
Azat Khuzhin
1f5d377aa6 Move comments about aliases (inline/noinline and similar) into proper place 
Fixes: 571d0d541c ("Improve CCTZ contrib (#9687)")
 2021-10-03 14:34:03 +03:00
Azat Khuzhin
bbd2ea6ac2 Add module part for TemporaryFileStream.h 2021-10-03 14:34:03 +03:00
Kseniia Sumarokova
ad0fcda713
Update ClientBase.h 2021-10-03 12:11:59 +03:00
alexey-milovidov
ac57e057a1
Merge pull request #23028 from ClickHouse/cast_to_value_or_default 
Merging #21330
 2021-10-03 11:00:48 +03:00
Alexey Milovidov
9bc32c95c0 Miscellaneous 2021-10-03 10:01:09 +03:00
kssenii
aa539937fb Fix build check 2021-10-03 06:23:05 +00:00
Alexey Milovidov
868215d0dc Fix initialization of StorageURLSource 2021-10-03 07:28:28 +03:00
alexey-milovidov
59fdfd18db
Merge pull request #29667 from ClickHouse/clickhouse-obfuscator-embedded-dictionaries 
Fix issue: clickhouse-format --obfuscate cannot process queries with embedded dictionaries
 2021-10-03 05:57:03 +03:00
Alexey Milovidov
7269b20ecc Fix issue: clickhouse-format --obfuscate cannot process queries with embedded dictionaries 2021-10-03 00:46:51 +03:00
Mike Kot
585e6d53ff fix 2021-10-02 22:55:26 +02:00
Maksim Kita
71ed20f656 UserDefinedExecutableFunctions added implicit cast 2021-10-02 23:29:09 +03:00
Mike Kot
57e2744264 Fixing other imports 2021-10-02 21:47:35 +02:00
Mike Kot
38232c286d fixing paths 2021-10-02 21:45:05 +02:00
Mike Kot
65e6e211b4 Merge remote-tracking branch 'upstream/master' into improvement/fn-traits 2021-10-02 21:37:50 +02:00
Anton Popov
857b8f0b1d fix clang-tidy 2021-10-02 22:36:30 +03:00
Anton Popov
1f10246a10 Merge remote-tracking branch 'upstream/master' into HEAD 2021-10-02 22:33:27 +03:00
Anton Popov
b64bbb54aa
Merge pull request #29203 from CurtizJ/fix-ubsan 
Fix integer underflow in function `quantile`
 2021-10-02 20:36:53 +03:00
Maksim Kita
87bb596875 Fixed modelEvaluate 2021-10-02 17:15:32 +03:00
Maksim Kita
395119339f Merge branch 'master' into rename-common 2021-10-02 17:15:10 +03:00
Maksim Kita
1a200f770c
Merge pull request #29662 from ClickHouse/model-evaluate-own-file 
Move modelEvaluate function to its own file
 2021-10-02 17:12:16 +03:00
Maksim Kita
c966806205
Merge pull request #29529 from kitaisreal/external-xml-loaders-reload-with-server-configuration-support 
ExternalXMLLoaders reload with server configuration support
 2021-10-02 16:26:28 +03:00
Maksim Kita
8a772e0bc0
Update ExternalLoaderXMLConfigRepository.cpp 2021-10-02 16:26:13 +03:00
kssenii
0ed92d85e2 Fix 2021-10-02 13:05:19 +00:00
Mike Kot
5d38930a3c Fixing gcc build due to a bug in compiler 2021-10-02 14:27:52 +02:00
alexey-milovidov
654b9b1234
Merge pull request #29605 from qoega/compressed-buffer-fuzz 
Improve CompressedReadBuffer
 2021-10-02 12:04:01 +03:00
alexey-milovidov
86d1ce63cf
Merge pull request #29649 from CurtizJ/merging-29376 
Merging #29376
 2021-10-02 12:02:42 +03:00
Alexey Milovidov
ed54a4c944 Fix build 2021-10-02 11:49:25 +03:00
Alexey Milovidov
922872fb9a Move modelEvaluate function to its own file 2021-10-02 11:39:26 +03:00
Alexey Milovidov
2ad5dbdfe8 Rename "common" to "base" 2021-10-02 11:30:38 +03:00
kssenii
e10255bba3 Fix 2021-10-02 08:10:34 +00:00
Alexey Milovidov
fe6b7c77c7 Rename "common" to "base" 2021-10-02 10:13:14 +03:00
alexey-milovidov
f21ea8f160
Update CompressedReadBufferBase.cpp 2021-10-02 06:13:46 +03:00
Anton Popov
bb92ff52d8
Update IMergeTreeDataPart.cpp 2021-10-02 00:50:19 +03:00
Anton Popov
d404efbf7e add test for freeze in-memory parts 2021-10-01 22:05:36 +03:00
Memo
201f79f140
Merge branch 'ClickHouse:master' into new_func 2021-10-02 02:29:44 +08:00
WangZengrui
a99a6fccc7 init 2021-10-02 02:34:53 +08:00
Maksim Kita
ad6d143303
Merge pull request #29574 from kitaisreal/jit-fix-short-circuit-with-alias 
ExpressionJIT fix short-circuit with alias
 2021-10-01 21:18:31 +03:00
Vitaly Baranov
27f6d5864d
Merge pull request #29060 from azat/inter-server-secret-auth-fix 
Do not allow to reuse previous credentials in case of inter-server secret
 2021-10-01 20:44:48 +03:00
kssenii
dabab85c60 Fix 2021-10-01 17:31:00 +00:00
Anton Popov
f0363c524d Merge branch 'master' of git://github.com/mo-avatar/ClickHouse into merging-29376 2021-10-01 20:29:33 +03:00
Anton Popov
9c296e94f3 fix performance of updatePermutation 2021-10-01 19:45:17 +03:00
Maksim Kita
57666725f9
Merge pull request #29623 from kitaisreal/bloom-filter-indexes-map-support-fix-unsafe-identifier-cast 
BloomFilter indexes map data type support fix unsafe identifier cast
 2021-10-01 19:28:04 +03:00
Nikita Mikhaylov
b2f7b6fae8
Fix race between MergeTask and storage destruction (#29614) 2021-10-01 18:55:04 +03:00
kssenii
04e159b42c Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into fix-signals 2021-10-01 15:54:52 +00:00
Mike Kot
38b02b121a Merge remote-tracking branch 'upstream/master' into improvement/fn-traits 2021-10-01 16:55:28 +02:00
Mike Kot
5e560ff9d7 fix 2021-10-01 16:55:01 +02:00
Maksim Kita
fe6bb1bb1e Fixed tests 2021-10-01 17:33:14 +03:00
kssenii
05181c6900 Fix 2021-10-01 14:13:02 +00:00
vdimir
7444d64156
Fix for recursive tuples in MergeTreeWhereOptimizer 2021-10-01 17:01:03 +03:00
vdimir
23ce41e3bb
MergeTreeWhereOptimizer supports constant tuple in both sides of equals 2021-10-01 17:01:03 +03:00
vdimir
64ffacd9e0
MergeTreeWhereOptimizer::tryAnalyzeTuple, fix fuzz 2021-10-01 17:01:02 +03:00
vdimir
e326b12a37
Fix style in MergeTreeWhereOptimizer.cpp 2021-10-01 17:01:02 +03:00
凌涛
f660224a5f
add tests 2021-10-01 17:01:01 +03:00
凌涛
ea68944055
fixed index didn't work in tuple where 2021-10-01 17:01:01 +03:00
Vladimir C
ec966b7df5
Merge pull request #27933 from mathalex/numeric_tuple_functions 2021-10-01 16:18:52 +03:00
vdimir
0da21eccd9
Better getTables in CrossToInnerJoinVisitor 2021-10-01 14:52:12 +03:00
vdimir
a817bed287
Replace isComma || isCross -> isCrossOrComma 2021-10-01 14:52:12 +03:00
vdimir
5852329288
Check multiple using in CrossToInnerJoinVisitor 2021-10-01 14:52:12 +03:00
vdimir
c33fb80872
Minor style fix for CrossToInnerJoinVisitor getTables 2021-10-01 14:52:12 +03:00
Viachaslau Boben
5d40471ea3
Fix cross to inner join using error 2021-10-01 14:52:11 +03:00
Filatenkov Artur
8959b348a1
Merge branch 'ClickHouse:master' into add_CORS 2021-10-01 14:42:46 +03:00
Maksim Kita
c793c8e7fa BloomFilter indexes map data type support fix unsafe identifier cast 2021-10-01 14:40:00 +03:00
Maksim Kita
30220529b7
Merge pull request #29573 from ClickHouse/fix_segfault_in_add_database_visitor 
Fix nullpointer dereference in AddDefaultDatabaseVisitor
 2021-10-01 12:44:15 +03:00
Maksim Kita
b8702e613d Fixed tests 2021-10-01 12:37:04 +03:00
alexey-milovidov
b29e877f26
Merge pull request #29182 from ClickHouse/yandex-to-clickhouse-in-configs 
Replace `<yandex>` to `<clickhouse>` in configs and more.
 2021-10-01 12:24:12 +03:00
Maksim Kita
47f14d3ea6 Fixed typo 2021-10-01 12:09:28 +03:00
Vasily Nemkov
2f16806cd0 Implemented creating databases with comments 2021-10-01 11:51:26 +03:00
Yatsishin Ilya
2c0071bc6d wip 2021-10-01 09:32:54 +03:00
Alexey Milovidov
f919271096 Compatibility 2021-10-01 04:46:41 +03:00
Nikita Tikhomirov
37f76d60ff Added MD4 and SHA384 functions. Refactoring the length of hashes in variables. 2021-10-01 04:19:50 +03:00
Alexey Milovidov
4271f8c738 Merge remote-tracking branch 'origin/master' into yandex-to-clickhouse-in-configs 2021-10-01 01:33:58 +03:00
Azat Khuzhin
91f6cf44b3 Fix SessionLog::addLoginSuccess() for ClientInfo::Interface::TCP_INTERSERVER 
In this case there there can be no user.
CI report [1]:

    Address: 0x1f Access: read. Address not mapped to object.
    Stack trace: 0x24494b2f 0x244784c6 0x260fe1af 0x260ed672 0x260ddffa 0x26104880 0x2d62f3af 0x2d6300a1 0x2d926666 0x2d91fb7a 0x7f8bcea3c609 0x7f8bce963293
    3.1. inlined from ./obj-x86_64-linux-gnu/../contrib/libcxx/include/string:0: std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >::operator=(std::__1::basic_string>
    3. ../src/Interpreters/SessionLog.cpp:216: DB::SessionLog::addLoginSuccess(StrongTypedef<wide::integer<128ul, unsigned int>, DB::UUIDTag> const&, std::__1::optional<std::__1::basic_string<char, st>
    4. ./obj-x86_64-linux-gnu/../src/Interpreters/Session.cpp:0: DB::Session::makeQueryContextImpl(DB::ClientInfo const*, DB::ClientInfo*) const @ 0x244784c6 in /usr/bin/clickhouse
    5.1. inlined from ./obj-x86_64-linux-gnu/../contrib/libcxx/include/memory:3299: std::__1::shared_ptr<DB::Context>::swap(std::__1::shared_ptr<DB::Context>&)
    5.2. inlined from ../contrib/libcxx/include/memory:3243: std::__1::shared_ptr<DB::Context>::operator=(std::__1::shared_ptr<DB::Context>&&)
    5. ../src/Server/TCPHandler.cpp:1208: DB::TCPHandler::receiveQuery() @ 0x260fe1af in /usr/bin/clickhouse
    6. ./obj-x86_64-linux-gnu/../src/Server/TCPHandler.cpp:0: DB::TCPHandler::receivePacket() @ 0x260ed672 in /usr/bin/clickhouse
    7. ./obj-x86_64-linux-gnu/../src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x260ddffa in /usr/bin/clickhouse
    8. ./obj-x86_64-linux-gnu/../src/Server/TCPHandler.cpp:1643: DB::TCPHandler::run() @ 0x26104880 in /usr/bin/clickhouse

  [1]: https://clickhouse-test-reports.s3.yandex.net/29060/c087fc0ed5fbea133eb3dc3a64b8db93a81d0ece/integration_tests_flaky_check_(asan).html#fail1
 2021-10-01 01:13:15 +03:00
Azat Khuzhin
e8a90b8ff2 Do not allow to reuse previous credentials in case of inter-server secret 
Before this patch INSERT via Buffer/Kafka may re-use previously set user
for that connection, while this is not correct, it should reset the
user, and use global context.

Note, before [1] there was a fallback to default user, but that code had
been removed, and now it got back.

  [1]: 0159c74f21 ("Secure inter-cluster query execution (with initial_user as current query user) [v3]")

Also note, that context for Buffer table (and others) cannot be changed,
since they don't have any user only profile.

I've tested this patch manually using the following:

    create table dist (key Int) engine=Distributed(test_cluster_two_shards_secure, default, data, key);
    create table buffer (key Int) engine=Buffer(default, dist, 1, 0, 0, 0, 0, 0, 0);
    create table data (key Int) engine=Memory();

    # to start the connection with readonly user
    $ clickhouse-client --user readonly -q 'select * from dist'
    $ clickhouse-client -q 'insert into buffer values (1)'
    # before this patch this produces errors like:
    # 2021.09.27 23:46:48.384920 [ 19474 ] {} <Error> default.dist.DirectoryMonitor: Code: 164. DB::Exception: Received from 127.0.0.2:9000. DB::Exception: readonly: Cannot execute query in readonly mode. Stack trace:

v2: reset the authentication instead of using default user (as suggested by @vitlibar)
v3: reset Session::user and introduce ClientInfo::resetAuthentication (as suggested by @vitlibar)
v4: reset the session every time in interserver mode (suggested by @vitlibar)
 2021-10-01 01:13:15 +03:00
Nikita Mikhaylov
9756b8dc33
Added an ability to execute more merges and mutations than threads, added new settings (#29140) 2021-10-01 00:26:24 +03:00
Anton Popov
515cc74530 generalize code in IColumn::updatePermutation 2021-10-01 00:11:54 +03:00
Kseniia Sumarokova
3e0d4faa1e
Merge pull request #29503 from azat/external-db-log-query 
Log queries to external databases (since they may be rewritten)
 2021-09-30 21:08:57 +03:00
Kseniia Sumarokova
3457868f3d
Merge pull request #26231 from kssenii/interactive-mode-for-clickhouse-local 
clickhouse-local interactive mode, merge clickhouse-client and clickhouse-local code
 2021-09-30 19:31:34 +03:00
Maksim Kita
d538871d56 ExpressionJIT fix short-circuit with alias 2021-09-30 19:23:15 +03:00
alesapin
950f63a104 Fix commit 2021-09-30 18:59:49 +03:00
alesapin
f31c35de48 Fix nullpointer dereference in AddDefaultDatabaseVisitor 2021-09-30 18:56:55 +03:00
Alexey Milovidov
551c52c236 Merge remote-tracking branch 'origin/master' into yandex-to-clickhouse-in-configs 2021-09-30 17:45:14 +03:00
alexey-milovidov
3796ccb912
Merge pull request #29540 from myrrc/improvement/monotonicity-aggregate 
Making Monotonicity an aggregate to use with designated initializers
 2021-09-30 17:26:06 +03:00
Raúl Marín
6f2447c027 clang-tidy fix 2021-09-30 15:48:54 +02:00
Nikolai Kochetov
bc91d7ea60 Fix crash. 2021-09-30 16:48:04 +03:00
Maksim Kita
7bc6b8fd70
Merge pull request #28604 from azat/safer-http-handler 
Proper buffer flush while handling HTTP queries (avoids std::terminate)
 2021-09-30 16:44:09 +03:00
Maksim Kita
5c5199ef01
Merge pull request #29502 from azat/fix-INSERT-with-metadata-changes 
Fix LOGICAL_ERROR for INSERT with concurrent ALTER
 2021-09-30 16:11:50 +03:00
Maksim Kita
c2407fee06 Fixed tests 2021-09-30 14:35:24 +03:00
Maksim Kita
87ce29886e Fixed tests 2021-09-30 13:52:07 +03:00
vdimir
4acd8f3147
Do not add extra parentheses for tuple negate unary operator 2021-09-30 13:47:29 +03:00
Maksim Kita
3e0d6a9709
Merge pull request #29531 from kssenii/fix-paths 
Fix pathStartsWith
 2021-09-30 12:49:52 +03:00
Raúl Marín
f58742014c Consistent naming 2021-09-30 11:05:21 +02:00
Vladimir C
bc8d925ac5
Merge pull request #29463 from azat/external-db-null 2021-09-30 11:38:04 +03:00
Nikolay Degterinsky
485e19ff84
Merge pull request #28955 from anneji-dev/alterTable 
Add default database for alter table
 2021-09-30 11:28:47 +03:00
MaxWk
3cbabe2c68 support conversion between date32 and datetime64 2021-09-30 15:09:36 +08:00
mergify[bot]
775d8baa0c
Merge branch 'master' into fix-paths 2021-09-30 06:17:04 +00:00
凌涛
a13d6b0aea Check all requests 2021-09-30 13:40:48 +08:00
Alexey Boykov
192633cb9b
Merge branch 'ClickHouse:master' into numeric_tuple_functions 2021-09-30 01:37:37 +03:00
kssenii
59cbe8579d Interrupt listener 2021-09-29 22:01:52 +00:00
凌涛
bd78af435d update code and add document 2021-09-30 04:51:02 +08:00
kssenii
07b979bda3 Rewrite to processors 2021-09-29 19:17:02 +00:00
Azat Khuzhin
4618636446 Fix LOGICAL_ERROR for INSERT with metadata changes in parallel 
In #28582 the problem was introduced due to it obtains new metadata
snapshot and so it may fail when it will check that headers does not
differs, like in [1]:

<details>

    {7b1256b1-09dc-4cc3-93d9-e9f6e26b0b99} <Fatal> : Logical error: 'Block structure mismatch in  function connect between ConvertingTransform and ReplicatedMergeTreeSink stream: different number of columns:
    a UInt8 UInt8(size = 0), b Int16 Int16(size = 0), c Float32 Float32(size = 0), d String String(size = 0), e Array(UInt8) Array(size = 0, UInt64(size = 0), UInt8(size = 0)), f Nullable(UUID) Nullable(size = 0, UUID(size = 0), UInt8(size = 0)), g Tuple(UInt8, UInt16) Tuple(size = 0, UInt8(size = 0), UInt16(size = 0)), h UInt64 UInt64(size = 0)
    a UInt8 UInt8(size = 0), b Int16 Int16(size = 0), c Float32 Float32(size = 0), d String String(size = 0), e Array(UInt8) Array(size = 0, UInt64(size = 0), UInt8(size = 0)), f Nullable(UUID) Nullable(size = 0, UUID(size = 0), UInt8(size = 0)), g Tuple(UInt8, UInt16) Tuple(size = 0, UInt8(size = 0), UInt16(size = 0))'.
    {7b1256b1-09dc-4cc3-93d9-e9f6e26b0b99} <Fatal> : Logical error: 'Block structure mismatch in  function connect between ConvertingTransform and ReplicatedMergeTreeSink stream: different number of columns:
    {} <Fatal> BaseDaemon: ########################################
    {} <Fatal> BaseDaemon: (version 21.11.1.8227, build id: E012027D3A9E94FD61655974C45C15F58215605B) (from thread 2180) (query_id: 7b1256b1-09dc-4cc3-93d9-e9f6e26b0b99) Received signal Aborted (6)
    {} <Fatal> BaseDaemon:
    {} <Fatal> BaseDaemon: Stack trace: 0x7f5661a3b18b 0x7f5661a1a859 0x1333bcb8 0x1333bdc2 0x1d9f04d1 0x1d9eec3c 0x1d9eeeca 0x1f7cc44c 0x1f79b992 0x1e52123f 0x1e522b8e 0x1eb7e27b 0x1eb7be44 0x1f751e26 0x1f75f0e5 0x23ba1179 0x23ba1988 0x23cef8b4 0x23cec39a 0x23ceb17c 0x7f5661c01609 0x7f5661b17293
    {} <Fatal> BaseDaemon: 4. raise @ 0x4618b in /usr/lib/x86_64-linux-gnu/libc-2.31.so
    {} <Fatal> BaseDaemon: 5. abort @ 0x25859 in /usr/lib/x86_64-linux-gnu/libc-2.31.so
    {} <Fatal> BaseDaemon: 6. ./obj-x86_64-linux-gnu/../src/Common/Exception.cpp:53: DB::handle_error_code(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, bool, std::__1::vector<void*, std::__1::allocator<void*> > const&) @ 0x1333bcb8 in /usr/bin/clickhouse
    {} <Fatal> BaseDaemon: 7. ./obj-x86_64-linux-gnu/../src/Common/Exception.cpp:60: DB::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, bool) @ 0x1333bdc2 in /usr/bin/clickhouse
    {} <Fatal> BaseDaemon: 8. ./obj-x86_64-linux-gnu/../src/Core/Block.cpp:32: void DB::onError<void>(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int) @ 0x1d9f04d1 in /usr/bin/clickhouse
    {} <Fatal> BaseDaemon: 9. ./obj-x86_64-linux-gnu/../src/Core/Block.cpp:86: void DB::checkBlockStructure<void>(DB::Block const&, DB::Block const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, bool) @ 0x1d9eec3c in /usr/bin/clickhouse
    {} <Fatal> BaseDaemon: 10. ./obj-x86_64-linux-gnu/../src/Core/Block.cpp:607: DB::assertCompatibleHeader(DB::Block const&, DB::Block const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) @ 0x1d9eeeca in /usr/bin/clickhouse
    {} <Fatal> BaseDaemon: 11. ./obj-x86_64-linux-gnu/../src/Processors/Port.cpp:19: DB::connect(DB::OutputPort&, DB::InputPort&) @ 0x1f7cc44c in /usr/bin/clickhouse
    {} <Fatal> BaseDaemon: 12. ./obj-x86_64-linux-gnu/../src/Processors/Chain.cpp:81: DB::Chain::addSource(std::__1::shared_ptr<DB::IProcessor>) @ 0x1f79b992 in /usr/bin/clickhouse
    {} <Fatal> BaseDaemon: 13. ./obj-x86_64-linux-gnu/../src/Interpreters/InterpreterInsertQuery.cpp:222: DB::InterpreterInsertQuery::buildChainImpl(std::__1::shared_ptr<DB::IStorage> const&, std::__1::shared_ptr<DB::StorageInMemoryMetadata const> const&, DB::Block const&, DB::ThreadStatus*, std::__1::atomic<unsigned long>*) @ 0x1e52123f in /usr/bin/clickhouse
    {} <Fatal> BaseDaemon: 14. ./obj-x86_64-linux-gnu/../src/Interpreters/InterpreterInsertQuery.cpp:368: DB::InterpreterInsertQuery::execute() @ 0x1e522b8e in /usr/bin/clickhouse

</details>

  [1]: https://clickhouse-test-reports.s3.yandex.net/29461/d6cd057e2c0c8b04d8a712a7b2a83526507ef512/functional_stateless_tests_(debug).html#fail1

Fixes: #28582 (cc: @KochetovNicolai)
 2021-09-29 21:56:59 +03:00
Azat Khuzhin
a4f2663b82 Log queries to external databases (since they may be rewriten) 
By rewritten means that some expression from WHERE can be removed, yes
there is external_table_strict_query, but sometimes it is too strict.
 2021-09-29 21:56:28 +03:00
Azat Khuzhin
6ddfbf39c8 Cover IS NULL/IS NOT NULL for external dbs in unit tests 2021-09-29 21:45:09 +03:00
Azat Khuzhin
24f55e29d1 Add support for IS NULL/IS NOT NULL for external dbs 2021-09-29 21:45:09 +03:00
Anton Popov
0dfe96fc02 generalize code in IColumn::permute 2021-09-29 20:56:10 +03:00
kssenii
2c005db9ad Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into interactive-mode-for-clickhouse-local 2021-09-29 17:45:15 +00:00
Nikolai Kochetov
68f8b9d235 Update ColumnGathererStream 2021-09-29 20:45:01 +03:00
Raúl Marín
0ee5c0bff5 Use RWLock in StorageJoin to avoid deadlocks 2021-09-29 19:30:07 +02:00
Mike Kot
61e0c6208b Making Monotonicity an aggregate to use with designated initializers 2021-09-29 18:01:26 +02:00
Maksim Kita
a5151fb49b Function accurateCastOrDefault updated implementation 2021-09-29 17:52:08 +03:00
mergify[bot]
6a2ac6e871
Merge branch 'master' into yandex-to-clickhouse-in-configs 2021-09-29 13:29:08 +00:00
kssenii
46a4525c21 Fix 2021-09-29 12:53:09 +00:00
Maksim Kita
7edf63162f ExternalXMLLoaders reload with server configuration support 2021-09-29 15:52:58 +03:00
Vitaly Baranov
0c33f1121b
Merge pull request #28012 from amosbird/jsonextractstring2 
Handle null column correctly in JSONFunction.
 2021-09-29 14:58:40 +03:00
Artur
fcebf7b985 correct tests 2021-09-29 11:29:24 +00:00
Mike Kot
7670ea50c1 Replacing std::function to Fn in some places 
to avoid dynamic allocating while keeping desired function signature
clear.
Simplifying SimpleCache (CachedFn) using C++20
 2021-09-29 13:09:39 +02:00
Kruglov Pavel
14be2f31f5
Merge pull request #29475 from ClickHouse/fix-29010 
Remove filter column from HAVING when it is not needed.
 2021-09-29 13:03:27 +03:00
Nikolai Kochetov
8d85590900
Merge pull request #29494 from ClickHouse/fix-possible-pushing-executor-hang 
Fix possible hang in PushingAsyncPipelineExecutor.
 2021-09-29 12:16:26 +03:00