thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-11-27 18:12:02 +00:00
Code Issues Packages Projects Releases Wiki Activity
74,628 Commits 9,188 Branches 1,782 Tags 2.1 GiB
e66735e21e
Commit Graph

18778 Commits

Author SHA1 Message Date
Alexey Milovidov
bd4b0af2e1 Fix bad cast in ParserCreateQuery 2021-10-06 02:34:48 +03:00
Maksim Kita
90eb74c933 Fixed tests 2021-10-06 00:37:45 +03:00
Kseniia Sumarokova
3663d23e4d
Merge pull request #29731 from kssenii/exists-subquery 
Support EXISTS(subquery)
 2021-10-05 22:26:19 +03:00
Azat Khuzhin
99ebdbb16e Fix parallel_view_processing 
Number of threads got lost during after QueryPipelineBuilder::addChains()
 2021-10-05 22:20:46 +03:00
Anton Popov
199ac31b48
Merge pull request #29683 from myrrc/improvement/tl-to-common 
Non-recursive implementation for type list and its functions
 2021-10-05 21:49:15 +03:00
Azat Khuzhin
4f5118b85f Fix data-race between fatal error handler and progress packets 
This is not that important, but because of packets overlaps the client
may not render the fatal error.

You will find TSan report in `details`.

<details>

```
WARNING: ThreadSanitizer: data race (pid=13384)
  Write of size 8 at 0x7b24000ad818 by thread T47:
    0 DB::WriteBuffer::next() obj-x86_64-linux-gnu/../src/IO/WriteBuffer.h:43:15 (clickhouse-tsan+0x16124d61)
    1 DB::TCPHandler::sendProgress() obj-x86_64-linux-gnu/../src/Server/TCPHandler.cpp:1602:10 (clickhouse-tsan+0x16124d61)
    2 DB::TCPHandler::runImpl()::$_6::operator()() const obj-x86_64-linux-gnu/../src/Server/TCPHandler.cpp:318:25 (clickhouse-tsan+0x1612cf54)

  Previous write of size 8 at 0x7b24000ad818 by thread T229:
    0 DB::WriteBuffer::next() obj-x86_64-linux-gnu/../src/IO/WriteBuffer.h:43:15 (clickhouse-tsan+0x1612b793)
    1 DB::TCPHandler::sendLogData(DB::Block const&) obj-x86_64-linux-gnu/../src/Server/TCPHandler.cpp:1561:10 (clickhouse-tsan+0x1612b793)
    2 DB::TCPHandler::sendLogs() obj-x86_64-linux-gnu/../src/Server/TCPHandler.cpp:1632:9 (clickhouse-tsan+0x161228ab)
    3 DB::TCPHandler::runImpl()::$_1::operator()() const obj-x86_64-linux-gnu/../src/Server/TCPHandler.cpp:229:62 (clickhouse-tsan+0x1612c4e1)
```

</details>
 2021-10-05 21:21:32 +03:00
Azat Khuzhin
342ec02664 Fix concurrent access to LowCardinality during GROUP BY (leads to SIGSEGV) 
The problem is that GROUP BY can update saved_hash, which can be also
updated by subsequent update of a dictionary, and this will lead to
use-after-free.

You will find ASan report in `details`.

<details>

    ==24679==ERROR: AddressSanitizer: heap-use-after-free on address 0x604000615d20 at pc 0x000022cc8684 bp 0x7ffea6b5f850 sp 0x7ffea6b5f848
    READ of size 8 at 0x604000615d20 thread T223 (QueryPipelineEx)
        0 0x22cc8683 in DB::ReverseIndex<>::insert(StringRef const&) obj-x86_64-linux-gnu/../src/Columns/ReverseIndex.h
        1 0x22cc0de1 in COW<DB::IColumn>::mutable_ptr<DB::IColumn> DB::ColumnUnique<DB::ColumnString>::uniqueInsertRangeImpl<char8_t>()::'lambda'()::operator()() const obj-x86_64-linux-gnu/../src/Columns/ColumnUnique.h:540:39
        2 0x22cc0de1 in COW<DB::IColumn>::mutable_ptr<DB::IColumn> DB::ColumnUnique<DB::ColumnString>::uniqueInsertRangeImpl<char8_t>() obj-x86_64-linux-gnu/../src/Columns/ColumnUnique.h:570:23
        3 0x22cb9c66 in COW<DB::IColumn>::mutable_ptr<DB::IColumn> DB::ColumnUnique<DB::ColumnString>::uniqueInsertRangeFrom()::'lambda'(auto)::operator()<char8_t>(auto) const obj-x86_64-linux-gnu/../src/Columns/ColumnUnique.h:592:26
        4 0x22cb9c66 in DB::ColumnUnique<DB::ColumnString>::uniqueInsertRangeFrom() obj-x86_64-linux-gnu/../src/Columns/ColumnUnique.h:600:28
        5 0x2500b897 in DB::ColumnLowCardinality::insertRangeFrom() obj-x86_64-linux-gnu/../src/Columns/ColumnLowCardinality.cpp:205:62
        6 0x25a182f4 in DB::appendBlock(DB::Block const&, DB::Block&) obj-x86_64-linux-gnu/../src/Storages/StorageBuffer.cpp:470:23
        7 0x25a182f4 in DB::BufferSink::insertIntoBuffer(DB::Block const&, DB::StorageBuffer::Buffer&) obj-x86_64-linux-gnu/../src/Storages/StorageBuffer.cpp:634:9
        8 0x25a173cc in DB::BufferSink::consume(DB::Chunk) obj-x86_64-linux-gnu/../src/Storages/StorageBuffer.cpp:595:9
        9 0x26d1c997 in DB::SinkToStorage::transform(DB::Chunk&) obj-x86_64-linux-gnu/../src/Processors/Sinks/SinkToStorage.cpp:18:5

    0x604000615d20 is located 16 bytes inside of 40-byte region [0x604000615d10,0x604000615d38)
    freed by thread T37 (QueryPipelineEx) here:
        2 0x22cb9392 in boost::intrusive_ptr<DB::ColumnVector<unsigned long> >::~intrusive_ptr() obj-x86_64-linux-gnu/../contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:98:23
        4 0x22cb9392 in COW<DB::IColumn>::mutable_ptr<DB::ColumnVector<unsigned long> >::operator=() obj-x86_64-linux-gnu/../src/Common/COW.h💯57
        5 0x22cb9392 in DB::ReverseIndex<>::tryGetSavedHash() const obj-x86_64-linux-gnu/../src/Columns/ReverseIndex.h:362:28
        6 0x22cb9392 in DB::ColumnUnique<DB::ColumnString>::tryGetSavedHash() const obj-x86_64-linux-gnu/../src/Columns/ColumnUnique.h:125:76
        7 0x242eaed3 in DB::ColumnsHashing::HashMethodSingleLowCardinalityColumn<>::HashMethodSingleLowCardinalityColumn() obj-x86_64-linux-gnu/../src/Common/ColumnsHashing.h:287:50
        8 0x242206c6 in void DB::Aggregator::executeImpl<>() const obj-x86_64-linux-gnu/../src/Interpreters/Aggregator.cpp:596:28
        9 0x24148e99 in DB::Aggregator::executeOnBlock() const obj-x86_64-linux-gnu/../src/Interpreters/Aggregator.cpp:1004:9
        10 0x26c24f3a in DB::AggregatingTransform::consume(DB::Chunk) obj-x86_64-linux-gnu/../src/Processors/Transforms/AggregatingTransform.cpp:539:33
        11 0x26c2054e in DB::AggregatingTransform::work() obj-x86_64-linux-gnu/../src/Processors/Transforms/AggregatingTransform.cpp:500:9

    previously allocated by thread T37 (QueryPipelineEx) here:
        0 0xb6d44fd in operator new(unsigned long) (/src/ch/tmp/upstream/clickhouse-asan+0xb6d44fd)
        1 0x11b78580 in COW<DB::IColumn>::mutable_ptr<> COWHelper<>::create<unsigned long&>(unsigned long&) (/src/ch/tmp/upstream/clickhouse-asan+0x11b78580)
        2 0x22cbf7b1 in DB::ReverseIndex<>::calcHashes() const obj-x86_64-linux-gnu/../src/Columns/ReverseIndex.h:472:17
        3 0x22cc2307 in DB::ReverseIndex<>::buildIndex() obj-x86_64-linux-gnu/../src/Columns/ReverseIndex.h:438:22
        4 0x22cc658c in DB::ReverseIndex<>::insert(StringRef const&) obj-x86_64-linux-gnu/../src/Columns/ReverseIndex.h:484:9
        5 0x22cc0de1 in COW<DB::IColumn>::mutable_ptr<DB::IColumn> DB::ColumnUnique<DB::ColumnString>::uniqueInsertRangeImpl<char8_t>()::'lambda'()::operator()() const obj-x86_64-linux-gnu/../src/Columns/ColumnUnique.h:540:39
        6 0x22cc0de1 in COW<DB::IColumn>::mutable_ptr<DB::IColumn> DB::ColumnUnique<DB::ColumnString>::uniqueInsertRangeImpl<char8_t>() obj-x86_64-linux-gnu/../src/Columns/ColumnUnique.h:570:23
        7 0x22cb9c66 in COW<DB::IColumn>::mutable_ptr<DB::IColumn> DB::ColumnUnique<DB::ColumnString>::uniqueInsertRangeFrom()::'lambda'(auto)::operator()<char8_t>(auto) const obj-x86_64-linux-gnu/../src/Columns/ColumnUnique.h:592:26
        8 0x22cb9c66 in DB::ColumnUnique<DB::ColumnString>::uniqueInsertRangeFrom() obj-x86_64-linux-gnu/../src/Columns/ColumnUnique.h:600:28
        9 0x2500b897 in DB::ColumnLowCardinality::insertRangeFrom() obj-x86_64-linux-gnu/../src/Columns/ColumnLowCardinality.cpp:205:62
        10 0x25a182f4 in DB::appendBlock(DB::Block const&, DB::Block&) obj-x86_64-linux-gnu/../src/Storages/StorageBuffer.cpp:470:23
        11 0x25a182f4 in DB::BufferSink::insertIntoBuffer(DB::Block const&, DB::StorageBuffer::Buffer&) obj-x86_64-linux-gnu/../src/Storages/StorageBuffer.cpp:634:9
        12 0x25a173cc in DB::BufferSink::consume(DB::Chunk) obj-x86_64-linux-gnu/../src/Storages/StorageBuffer.cpp:595:9
        13 0x26d1c997 in DB::SinkToStorage::transform(DB::Chunk&) obj-x86_64-linux-gnu/../src/Processors/Sinks/SinkToStorage.cpp:18:5

    SUMMARY: AddressSanitizer: heap-use-after-free obj-x86_64-linux-gnu/../src/Columns/ReverseIndex.h in DB::ReverseIndex<unsigned long, DB::ColumnString>::insert(StringRef const&)
    Shadow bytes around the buggy address:
      0x0c08800bab50: fa fa fd fd fd fd fd fa fa fa 00 00 00 00 00 00
      0x0c08800bab60: fa fa fd fd fd fd fd fa fa fa 00 00 00 00 00 fa
      0x0c08800bab70: fa fa 00 00 00 00 00 00 fa fa 00 00 00 00 00 00
      0x0c08800bab80: fa fa fd fd fd fd fd fa fa fa fd fd fd fd fd fa
      0x0c08800bab90: fa fa fd fd fd fd fd fa fa fa fd fd fd fd fd fd
    =>0x0c08800baba0: fa fa fd fd[fd]fd fd fa fa fa fd fd fd fd fd fa
      0x0c08800babb0: fa fa fd fd fd fd fd fd fa fa fd fd fd fd fd fa
      0x0c08800babc0: fa fa fd fd fd fd fd fa fa fa fd fd fd fd fd fd
      0x0c08800babd0: fa fa fd fd fd fd fd fd fa fa fd fd fd fd fd fa
      0x0c08800babe0: fa fa fd fd fd fd fd fa fa fa fd fd fd fd fd fd
      0x0c08800babf0: fa fa fd fd fd fd fd fd fa fa fd fd fd fd fd fd
    Shadow byte legend (one shadow byte represents 8 application bytes):
      Addressable:           00
      Partially addressable: 01 02 03 04 05 06 07
      Heap left redzone:       fa
      Freed heap region:       fd
      Stack left redzone:      f1
      Stack mid redzone:       f2
      Stack right redzone:     f3
      Stack after return:      f5
      Stack use after scope:   f8
      Global redzone:          f9
      Global init order:       f6
      Poisoned by user:        f7
      Container overflow:      fc
      Array cookie:            ac
      Intra object redzone:    bb
      ASan internal:           fe
      Left alloca redzone:     ca
      Right alloca redzone:    cb
    ==24679==ABORTING

</details>
 2021-10-05 21:07:10 +03:00
Kruglov Pavel
fa81606d6a
Fix build 2021-10-05 20:11:21 +03:00
Nikita Mikhaylov
cc150b7316
Merge pull request #29596 from nikitamikhaylov/fuzz 
Add fuzzer for `executeQuery` function
 2021-10-05 20:10:13 +03:00
Maksim Kita
4fde987ad1 Parse query from metadata exception throw fix 2021-10-05 19:34:25 +03:00
Anton Popov
b5098301d6 fix build 2021-10-05 18:23:53 +03:00
Mike Kot
25a62b9f83 Renaming TL to Typelist 2021-10-05 17:22:02 +02:00
Artur Filatenkov
9b1a39fdb9 refactor after move common to base 2021-10-05 17:43:33 +03:00
Filatenkov Artur
60d84df56a
Merge branch 'ClickHouse:master' into add_CORS 2021-10-05 17:27:13 +03:00
mergify[bot]
ab55225c8f
Merge branch 'master' into remove-merging-streams 2021-10-05 14:16:34 +00:00
Nikolai Kochetov
96d070a5ba Fix tests. 2021-10-05 16:58:24 +03:00
Filatenkov Artur
dc657a2c5e
Add error for multiple keys without current_key_id (#29546) 
* add error for multiple keys without current_key_id

* add exception if one key with non zero id is used without current_key_id

* refactoring
 2021-10-05 14:37:54 +03:00
Nikita Mikhaylov
f3c50e563e -nostdlib++ 2021-10-05 10:51:49 +00:00
alesapin
9aab0eb0a2 Fixup 2021-10-05 13:41:09 +03:00
Filatenkov Artur
82e6ac8fa2
Update HTTPHandler.cpp 2021-10-05 13:39:18 +03:00
alesapin
64158c011b Remove logging 2021-10-05 13:12:31 +03:00
Kruglov Pavel
e990f35ba3
Merge pull request #29746 from azat/JSONExtract-Null-fix 
Forbid Nullable for JSONExtract* (JSONExtract() still supports Nullable), leads to SIGSEGV before
 2021-10-05 13:04:46 +03:00
alesapin
c2f34928e0 Fixup 2021-10-05 13:04:44 +03:00
Nikolai Kochetov
c106f1e380 Fix tests. 2021-10-05 12:51:43 +03:00
alesapin
3a8126372c Remove redundant header 2021-10-05 12:19:04 +03:00
alesapin
8186339307 Less seeks in compressed buffers 2021-10-05 12:11:25 +03:00
avogar
c2cb9e2721 Fix use of unititialized value 2021-10-05 11:55:26 +03:00
Kseniia Sumarokova
438a981e76
Update TreeRewriter.cpp 2021-10-05 11:25:50 +03:00
Kseniia Sumarokova
385bf579a6
Merge pull request #29745 from azat/clickhouse-local-fix-stage 
Fix --stage for clickhouse-local
 2021-10-05 11:14:27 +03:00
Vitaly Baranov
8a01b32cba
Merge pull request #28637 from vitlibar/fix-materialized-column-as-sharding-key 
Fix materialized column as sharding key
 2021-10-05 10:53:24 +03:00
Memo
0e9bbc4d43
Merge branch 'ClickHouse:master' into new_func 2021-10-05 09:02:37 +08:00
WangZengrui
a638c40fdc add getOSKernelVersion 2021-10-05 09:08:16 +08:00
Nikolai Kochetov
2001ebbf9d Fix build. 2021-10-04 21:52:31 +03:00
Maksim Kita
6a6946018b
Merge pull request #29747 from kitaisreal/table-function-dictionary-fix-comment 
TableFunctionDictionary fix comment
 2021-10-04 21:10:20 +03:00
Maksim Kita
6df81d94bf TableFunctionDictionary fix comment 2021-10-04 21:09:32 +03:00
Azat Khuzhin
20e706766c Forbid Nullable for JSONExtract* (JSONExtract() still supports Nullable) 
Only JSONExtract() can support Nullable, others JSONExtract*
(JSONExtractString and similar) does not.

And right now this file is pretty complex already,
so adding support of Nullable for others will make it even more complex.

CI: https://clickhouse-test-reports.s3.yandex.net/29680/d0fc26f91a0141b56a0550741219c3dc43630e03/fuzzer_ubsan/report.html#fail1
 2021-10-04 21:08:02 +03:00
Azat Khuzhin
bbee102e58 Fix --stage for clickhouse-local 
This also fixes UBsan error, since query_processing_stage was not
initialized before for clickhouse-local.
 2021-10-04 21:00:48 +03:00
Nikita Mikhaylov
ec99357097 Done 2021-10-04 17:57:58 +00:00
Nikita Mikhaylov
613b814e24
Merge pull request #29595 from CurtizJ/generalize-update-permutation 
Generalize code in `IColumn::updatePermutation`
 2021-10-04 20:36:57 +03:00
Nikita Mikhaylov
b31d11478b
Merge pull request #29680 from azat/defines 
Cleanup common defines
 2021-10-04 20:13:29 +03:00
Nikita Mikhaylov
6d5e9a9089
Merge pull request #29692 from azat/disk-memory-data-race-fix 
Fix data-race in WriteIndirectBuffer (used in DiskMemory)
 2021-10-04 20:13:18 +03:00
Nikita Mikhaylov
36516dd503
Merge pull request #29681 from azat/MemoryTracker-mutate-merge 
Apply max_untracked_memory/memory_profiler_step/memory_profiler_sample_probability during mutate/merge
 2021-10-04 20:04:57 +03:00
Maksim Kita
3fe9fdac96 Added function ngram 2021-10-04 18:53:17 +03:00
avogar
2523053b27 Remove code duplication, fix bug, add more tests 2021-10-04 18:22:06 +03:00
Nikolai Kochetov
122d39cbba
Fix missing condition in pushed down predicate. (#29625) 2021-10-04 17:50:03 +03:00
Pavel Kruglov
ae2754ee56 Fix tests, add comments 2021-10-04 16:44:38 +03:00
Pavel Kruglov
2c4453f425 Rename template argument 2021-10-04 16:44:38 +03:00
Pavel Kruglov
82d17870f1 Avoid division by zero when denominator is Nullable 2021-10-04 16:44:38 +03:00
kssenii
f1ea186bff Exists 2021-10-04 13:12:30 +00:00
Anton Popov
362e84a336
Merge pull request #29429 from Enmk/Governance/database_comment 
Implemented creating databases with comments
 2021-10-04 15:12:33 +03:00
Anton Popov
cd08057285 better arguments 2021-10-04 15:02:11 +03:00
Maksim Kita
8eda58acfb
Merge pull request #29516 from lingtaolf/feature/support-HSTS 
Support HSTS in Clickhouse HTTP server
 2021-10-04 12:49:51 +03:00
Nikolay Degterinsky
345ac59f46
Merge pull request #29552 from MaxWk/conversion-between-date32-and-datetime64 
Support conversion between date32 and datetime64
 2021-10-04 12:43:56 +03:00
Vitaly Baranov
17fe76709f
Merge pull request #28331 from vitlibar/mysql-authentication-cleanup 
Clean up MySQL authentication.
 2021-10-04 11:11:45 +03:00
Vitaly Baranov
1636ee24bb Fix using materialized column as sharding key. 2021-10-04 10:56:42 +03:00
Filatenkov Artur
8df228e6b8
Merge branch 'master' into add_CORS 2021-10-04 10:55:09 +03:00
Maksim Kita
25d1588d0a
Merge pull request #29666 from kitaisreal/user-defined-executable-functions-added-implicit-cast 
UserDefinedExecutableFunctions added implicit cast
 2021-10-04 10:34:31 +03:00
Kseniia Sumarokova
5ab17d795f
Merge pull request #29673 from ClickHouse/url-fix-initialization 
Fix initialization of `StorageURLSource`
 2021-10-04 10:23:52 +03:00
Kseniia Sumarokova
0b2dd42261
Merge pull request #29686 from kssenii/fix-materialized-postgresql-3 
MaterializedPostgreSQL fix
 2021-10-04 09:47:21 +03:00
alexey-milovidov
325a01f798
Merge pull request #29553 from azat/group-by-with-totals-having-fix 
Fix null deference for GROUP BY WITH TOTALS HAVING (when the column from HAVING wasn't selected)
 2021-10-04 09:25:35 +03:00
alexey-milovidov
6abb3d4c6d
Merge pull request #29682 from traceon/xcode-13-fix-build 
Fix Xcode 13 build
 2021-10-04 02:34:43 +03:00
Azat Khuzhin
a7793101bc Fix data-race in WriteIndirectBuffer (used in StripeLog) 
CI founds [1]:

WARNING: ThreadSanitizer: data race (pid=497)
  Write of size 8 at 0x7b30000260b8 by thread T381 (mutexes: write M546759850860896464):
    ...
    3 DB::DiskMemory::writeFile() obj-x86_64-linux-gnu/../src/Disks/DiskMemory.cpp:339:15 (clickhouse+0x14a1849f)
    4 DB::FileChecker::save() const obj-x86_64-linux-gnu/../src/Common/FileChecker.cpp:111:50 (clickhouse+0x15bbe19f)
    5 DB::TinyLogSink::TinyLogSink() obj-x86_64-linux-gnu/../src/Storages/StorageTinyLog.cpp:214:34 (clickhouse+0x15bca215)

  Previous read of size 8 at 0x7b30000260b8 by thread T460:
    ...
    2 DB::WriteIndirectBuffer::finalize() obj-x86_64-linux-gnu/../src/Disks/DiskMemory.cpp:106:33 (clickhouse+0x14a1e044)
    9 DB::StripeLogSink::onFinish() obj-x86_64-linux-gnu/../src/Storages/StorageStripeLog.cpp:227:30 (clickhouse+0x15bd241a)

  [1]: https://clickhouse-test-reports.s3.yandex.net/29680/d0fc26f91a0141b56a0550741219c3dc43630e03/stress_test_(thread)/stderr.log
 2021-10-04 01:58:35 +03:00
Vitaly Baranov
30b9b8fd58 Clean up MySQL authentication. 2021-10-03 23:40:08 +03:00
Kseniia Sumarokova
ddc775b1c8
Merge pull request #29626 from kssenii/fix-signals 
Follow-up for #26231
 2021-10-03 22:13:54 +03:00
kssenii
dbc114e6fc Fix special build check 2021-10-03 16:49:02 +00:00
kssenii
7ea9037939 Fix 2021-10-03 16:41:12 +00:00
Mike Kot
8c8bada56e Non-recursive implementation for TL and its functions 2021-10-03 18:02:04 +02:00
Azat Khuzhin
117e9e77c8 Apply max_untracked_memory/memory_profiler_{step,sample_probability} during mutate/merge 2021-10-03 17:39:07 +03:00
kssenii
0cb7193fde Fix url with failover case 2021-10-03 13:58:03 +00:00
Denis Glazachev
ca7b69b0f3 Manipulate with -Wreserved-identifier only if HAS_RESERVED_IDENTIFIER has been detected 2021-10-03 17:42:36 +04:00
Azat Khuzhin
1fc19eebd7 Remove unused DBMS_DEFAULT_HOST 2021-10-03 14:34:03 +03:00
Azat Khuzhin
b43b4016d7 Remove unused DBMS_DEFAULT_HTTP_PORT 2021-10-03 14:34:03 +03:00
Azat Khuzhin
0b83345560 Remove macros for distributed connection timeouts 2021-10-03 14:34:03 +03:00
Azat Khuzhin
f6e3583ffa Remove macros for hedged request timeouts 2021-10-03 14:34:03 +03:00
Azat Khuzhin
0a6885d806 Remove DEFAULT_LIVE_VIEW_HEARTBEAT_INTERVAL_SEC 2021-10-03 14:34:03 +03:00
Azat Khuzhin
dc2cb6d996 Remove DBMS_DEFAULT_DISTRIBUTED_CONNECTIONS_POOL_SIZE 2021-10-03 14:34:03 +03:00
Azat Khuzhin
16c72f2107 Remove unused macros from ReadWriteBufferFromHTTP 2021-10-03 14:34:03 +03:00
Azat Khuzhin
a5a2c5ef8a Move KEEPER_DEFAULT_PATH into separate header 2021-10-03 14:34:03 +03:00
Azat Khuzhin
ae5ee2dd28 Move macros for distributed engine into separate header 2021-10-03 14:34:03 +03:00
Azat Khuzhin
6a9dd9828d Move protocol macros into separate header 
Defines.h is a very common header, so lots of modules will be recompiled
on changes.
Move macros for protocol into separate header, this should significantly
decreases number of units to compile on it's changes.
 2021-10-03 14:34:03 +03:00
Azat Khuzhin
70e1f4d402 Move ASAN_{UN,}POISON_MEMORY_REGION to base/defines 2021-10-03 14:34:03 +03:00
Azat Khuzhin
3d6df8f1a0 Fix typo in a comment 2021-10-03 14:34:03 +03:00
Azat Khuzhin
1f5d377aa6 Move comments about aliases (inline/noinline and similar) into proper place 
Fixes: 571d0d541c ("Improve CCTZ contrib (#9687)")
 2021-10-03 14:34:03 +03:00
Azat Khuzhin
bbd2ea6ac2 Add module part for TemporaryFileStream.h 2021-10-03 14:34:03 +03:00
Kseniia Sumarokova
ad0fcda713
Update ClientBase.h 2021-10-03 12:11:59 +03:00
alexey-milovidov
ac57e057a1
Merge pull request #23028 from ClickHouse/cast_to_value_or_default 
Merging #21330
 2021-10-03 11:00:48 +03:00
Alexey Milovidov
9bc32c95c0 Miscellaneous 2021-10-03 10:01:09 +03:00
kssenii
aa539937fb Fix build check 2021-10-03 06:23:05 +00:00
Alexey Milovidov
868215d0dc Fix initialization of StorageURLSource 2021-10-03 07:28:28 +03:00
alexey-milovidov
59fdfd18db
Merge pull request #29667 from ClickHouse/clickhouse-obfuscator-embedded-dictionaries 
Fix issue: clickhouse-format --obfuscate cannot process queries with embedded dictionaries
 2021-10-03 05:57:03 +03:00
Alexey Milovidov
7269b20ecc Fix issue: clickhouse-format --obfuscate cannot process queries with embedded dictionaries 2021-10-03 00:46:51 +03:00
Mike Kot
585e6d53ff fix 2021-10-02 22:55:26 +02:00
Maksim Kita
71ed20f656 UserDefinedExecutableFunctions added implicit cast 2021-10-02 23:29:09 +03:00
Mike Kot
57e2744264 Fixing other imports 2021-10-02 21:47:35 +02:00
Mike Kot
38232c286d fixing paths 2021-10-02 21:45:05 +02:00
Mike Kot
65e6e211b4 Merge remote-tracking branch 'upstream/master' into improvement/fn-traits 2021-10-02 21:37:50 +02:00
Anton Popov
857b8f0b1d fix clang-tidy 2021-10-02 22:36:30 +03:00
Anton Popov
1f10246a10 Merge remote-tracking branch 'upstream/master' into HEAD 2021-10-02 22:33:27 +03:00
Anton Popov
b64bbb54aa
Merge pull request #29203 from CurtizJ/fix-ubsan 
Fix integer underflow in function `quantile`
 2021-10-02 20:36:53 +03:00
Maksim Kita
87bb596875 Fixed modelEvaluate 2021-10-02 17:15:32 +03:00
Maksim Kita
395119339f Merge branch 'master' into rename-common 2021-10-02 17:15:10 +03:00
Maksim Kita
1a200f770c
Merge pull request #29662 from ClickHouse/model-evaluate-own-file 
Move modelEvaluate function to its own file
 2021-10-02 17:12:16 +03:00
Maksim Kita
c966806205
Merge pull request #29529 from kitaisreal/external-xml-loaders-reload-with-server-configuration-support 
ExternalXMLLoaders reload with server configuration support
 2021-10-02 16:26:28 +03:00
Maksim Kita
8a772e0bc0
Update ExternalLoaderXMLConfigRepository.cpp 2021-10-02 16:26:13 +03:00
kssenii
0ed92d85e2 Fix 2021-10-02 13:05:19 +00:00
Mike Kot
5d38930a3c Fixing gcc build due to a bug in compiler 2021-10-02 14:27:52 +02:00
alexey-milovidov
654b9b1234
Merge pull request #29605 from qoega/compressed-buffer-fuzz 
Improve CompressedReadBuffer
 2021-10-02 12:04:01 +03:00
alexey-milovidov
86d1ce63cf
Merge pull request #29649 from CurtizJ/merging-29376 
Merging #29376
 2021-10-02 12:02:42 +03:00
Alexey Milovidov
ed54a4c944 Fix build 2021-10-02 11:49:25 +03:00
Alexey Milovidov
922872fb9a Move modelEvaluate function to its own file 2021-10-02 11:39:26 +03:00
Alexey Milovidov
2ad5dbdfe8 Rename "common" to "base" 2021-10-02 11:30:38 +03:00
kssenii
e10255bba3 Fix 2021-10-02 08:10:34 +00:00
Alexey Milovidov
fe6b7c77c7 Rename "common" to "base" 2021-10-02 10:13:14 +03:00
alexey-milovidov
f21ea8f160
Update CompressedReadBufferBase.cpp 2021-10-02 06:13:46 +03:00
Anton Popov
bb92ff52d8
Update IMergeTreeDataPart.cpp 2021-10-02 00:50:19 +03:00
Anton Popov
d404efbf7e add test for freeze in-memory parts 2021-10-01 22:05:36 +03:00
Memo
201f79f140
Merge branch 'ClickHouse:master' into new_func 2021-10-02 02:29:44 +08:00
WangZengrui
a99a6fccc7 init 2021-10-02 02:34:53 +08:00
Maksim Kita
ad6d143303
Merge pull request #29574 from kitaisreal/jit-fix-short-circuit-with-alias 
ExpressionJIT fix short-circuit with alias
 2021-10-01 21:18:31 +03:00
Vitaly Baranov
27f6d5864d
Merge pull request #29060 from azat/inter-server-secret-auth-fix 
Do not allow to reuse previous credentials in case of inter-server secret
 2021-10-01 20:44:48 +03:00
kssenii
dabab85c60 Fix 2021-10-01 17:31:00 +00:00
Anton Popov
f0363c524d Merge branch 'master' of git://github.com/mo-avatar/ClickHouse into merging-29376 2021-10-01 20:29:33 +03:00
Anton Popov
9c296e94f3 fix performance of updatePermutation 2021-10-01 19:45:17 +03:00
Maksim Kita
57666725f9
Merge pull request #29623 from kitaisreal/bloom-filter-indexes-map-support-fix-unsafe-identifier-cast 
BloomFilter indexes map data type support fix unsafe identifier cast
 2021-10-01 19:28:04 +03:00
Nikita Mikhaylov
b2f7b6fae8
Fix race between MergeTask and storage destruction (#29614) 2021-10-01 18:55:04 +03:00
kssenii
04e159b42c Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into fix-signals 2021-10-01 15:54:52 +00:00
Mike Kot
38b02b121a Merge remote-tracking branch 'upstream/master' into improvement/fn-traits 2021-10-01 16:55:28 +02:00
Mike Kot
5e560ff9d7 fix 2021-10-01 16:55:01 +02:00
Maksim Kita
fe6bb1bb1e Fixed tests 2021-10-01 17:33:14 +03:00
kssenii
05181c6900 Fix 2021-10-01 14:13:02 +00:00
vdimir
7444d64156
Fix for recursive tuples in MergeTreeWhereOptimizer 2021-10-01 17:01:03 +03:00
vdimir
23ce41e3bb
MergeTreeWhereOptimizer supports constant tuple in both sides of equals 2021-10-01 17:01:03 +03:00
vdimir
64ffacd9e0
MergeTreeWhereOptimizer::tryAnalyzeTuple, fix fuzz 2021-10-01 17:01:02 +03:00
vdimir
e326b12a37
Fix style in MergeTreeWhereOptimizer.cpp 2021-10-01 17:01:02 +03:00
凌涛
f660224a5f
add tests 2021-10-01 17:01:01 +03:00
凌涛
ea68944055
fixed index didn't work in tuple where 2021-10-01 17:01:01 +03:00
Vladimir C
ec966b7df5
Merge pull request #27933 from mathalex/numeric_tuple_functions 2021-10-01 16:18:52 +03:00
vdimir
0da21eccd9
Better getTables in CrossToInnerJoinVisitor 2021-10-01 14:52:12 +03:00
vdimir
a817bed287
Replace isComma || isCross -> isCrossOrComma 2021-10-01 14:52:12 +03:00
vdimir
5852329288
Check multiple using in CrossToInnerJoinVisitor 2021-10-01 14:52:12 +03:00
vdimir
c33fb80872
Minor style fix for CrossToInnerJoinVisitor getTables 2021-10-01 14:52:12 +03:00
Viachaslau Boben
5d40471ea3
Fix cross to inner join using error 2021-10-01 14:52:11 +03:00
Filatenkov Artur
8959b348a1
Merge branch 'ClickHouse:master' into add_CORS 2021-10-01 14:42:46 +03:00
Maksim Kita
c793c8e7fa BloomFilter indexes map data type support fix unsafe identifier cast 2021-10-01 14:40:00 +03:00
Maksim Kita
30220529b7
Merge pull request #29573 from ClickHouse/fix_segfault_in_add_database_visitor 
Fix nullpointer dereference in AddDefaultDatabaseVisitor
 2021-10-01 12:44:15 +03:00
Maksim Kita
b8702e613d Fixed tests 2021-10-01 12:37:04 +03:00
alexey-milovidov
b29e877f26
Merge pull request #29182 from ClickHouse/yandex-to-clickhouse-in-configs 
Replace `<yandex>` to `<clickhouse>` in configs and more.
 2021-10-01 12:24:12 +03:00
Maksim Kita
47f14d3ea6 Fixed typo 2021-10-01 12:09:28 +03:00
Vasily Nemkov
2f16806cd0 Implemented creating databases with comments 2021-10-01 11:51:26 +03:00
Yatsishin Ilya
2c0071bc6d wip 2021-10-01 09:32:54 +03:00
Alexey Milovidov
f919271096 Compatibility 2021-10-01 04:46:41 +03:00
Nikita Tikhomirov
37f76d60ff Added MD4 and SHA384 functions. Refactoring the length of hashes in variables. 2021-10-01 04:19:50 +03:00
Alexey Milovidov
4271f8c738 Merge remote-tracking branch 'origin/master' into yandex-to-clickhouse-in-configs 2021-10-01 01:33:58 +03:00
Azat Khuzhin
91f6cf44b3 Fix SessionLog::addLoginSuccess() for ClientInfo::Interface::TCP_INTERSERVER 
In this case there there can be no user.
CI report [1]:

    Address: 0x1f Access: read. Address not mapped to object.
    Stack trace: 0x24494b2f 0x244784c6 0x260fe1af 0x260ed672 0x260ddffa 0x26104880 0x2d62f3af 0x2d6300a1 0x2d926666 0x2d91fb7a 0x7f8bcea3c609 0x7f8bce963293
    3.1. inlined from ./obj-x86_64-linux-gnu/../contrib/libcxx/include/string:0: std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >::operator=(std::__1::basic_string>
    3. ../src/Interpreters/SessionLog.cpp:216: DB::SessionLog::addLoginSuccess(StrongTypedef<wide::integer<128ul, unsigned int>, DB::UUIDTag> const&, std::__1::optional<std::__1::basic_string<char, st>
    4. ./obj-x86_64-linux-gnu/../src/Interpreters/Session.cpp:0: DB::Session::makeQueryContextImpl(DB::ClientInfo const*, DB::ClientInfo*) const @ 0x244784c6 in /usr/bin/clickhouse
    5.1. inlined from ./obj-x86_64-linux-gnu/../contrib/libcxx/include/memory:3299: std::__1::shared_ptr<DB::Context>::swap(std::__1::shared_ptr<DB::Context>&)
    5.2. inlined from ../contrib/libcxx/include/memory:3243: std::__1::shared_ptr<DB::Context>::operator=(std::__1::shared_ptr<DB::Context>&&)
    5. ../src/Server/TCPHandler.cpp:1208: DB::TCPHandler::receiveQuery() @ 0x260fe1af in /usr/bin/clickhouse
    6. ./obj-x86_64-linux-gnu/../src/Server/TCPHandler.cpp:0: DB::TCPHandler::receivePacket() @ 0x260ed672 in /usr/bin/clickhouse
    7. ./obj-x86_64-linux-gnu/../src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x260ddffa in /usr/bin/clickhouse
    8. ./obj-x86_64-linux-gnu/../src/Server/TCPHandler.cpp:1643: DB::TCPHandler::run() @ 0x26104880 in /usr/bin/clickhouse

  [1]: https://clickhouse-test-reports.s3.yandex.net/29060/c087fc0ed5fbea133eb3dc3a64b8db93a81d0ece/integration_tests_flaky_check_(asan).html#fail1
 2021-10-01 01:13:15 +03:00
Azat Khuzhin
e8a90b8ff2 Do not allow to reuse previous credentials in case of inter-server secret 
Before this patch INSERT via Buffer/Kafka may re-use previously set user
for that connection, while this is not correct, it should reset the
user, and use global context.

Note, before [1] there was a fallback to default user, but that code had
been removed, and now it got back.

  [1]: 0159c74f21 ("Secure inter-cluster query execution (with initial_user as current query user) [v3]")

Also note, that context for Buffer table (and others) cannot be changed,
since they don't have any user only profile.

I've tested this patch manually using the following:

    create table dist (key Int) engine=Distributed(test_cluster_two_shards_secure, default, data, key);
    create table buffer (key Int) engine=Buffer(default, dist, 1, 0, 0, 0, 0, 0, 0);
    create table data (key Int) engine=Memory();

    # to start the connection with readonly user
    $ clickhouse-client --user readonly -q 'select * from dist'
    $ clickhouse-client -q 'insert into buffer values (1)'
    # before this patch this produces errors like:
    # 2021.09.27 23:46:48.384920 [ 19474 ] {} <Error> default.dist.DirectoryMonitor: Code: 164. DB::Exception: Received from 127.0.0.2:9000. DB::Exception: readonly: Cannot execute query in readonly mode. Stack trace:

v2: reset the authentication instead of using default user (as suggested by @vitlibar)
v3: reset Session::user and introduce ClientInfo::resetAuthentication (as suggested by @vitlibar)
v4: reset the session every time in interserver mode (suggested by @vitlibar)
 2021-10-01 01:13:15 +03:00
Nikita Mikhaylov
9756b8dc33
Added an ability to execute more merges and mutations than threads, added new settings (#29140) 2021-10-01 00:26:24 +03:00
Anton Popov
515cc74530 generalize code in IColumn::updatePermutation 2021-10-01 00:11:54 +03:00
Kseniia Sumarokova
3e0d4faa1e
Merge pull request #29503 from azat/external-db-log-query 
Log queries to external databases (since they may be rewritten)
 2021-09-30 21:08:57 +03:00
Kseniia Sumarokova
3457868f3d
Merge pull request #26231 from kssenii/interactive-mode-for-clickhouse-local 
clickhouse-local interactive mode, merge clickhouse-client and clickhouse-local code
 2021-09-30 19:31:34 +03:00
Maksim Kita
d538871d56 ExpressionJIT fix short-circuit with alias 2021-09-30 19:23:15 +03:00
alesapin
950f63a104 Fix commit 2021-09-30 18:59:49 +03:00
alesapin
f31c35de48 Fix nullpointer dereference in AddDefaultDatabaseVisitor 2021-09-30 18:56:55 +03:00
Alexey Milovidov
551c52c236 Merge remote-tracking branch 'origin/master' into yandex-to-clickhouse-in-configs 2021-09-30 17:45:14 +03:00
alexey-milovidov
3796ccb912
Merge pull request #29540 from myrrc/improvement/monotonicity-aggregate 
Making Monotonicity an aggregate to use with designated initializers
 2021-09-30 17:26:06 +03:00
Raúl Marín
6f2447c027 clang-tidy fix 2021-09-30 15:48:54 +02:00
Nikolai Kochetov
bc91d7ea60 Fix crash. 2021-09-30 16:48:04 +03:00
Maksim Kita
7bc6b8fd70
Merge pull request #28604 from azat/safer-http-handler 
Proper buffer flush while handling HTTP queries (avoids std::terminate)
 2021-09-30 16:44:09 +03:00
Maksim Kita
5c5199ef01
Merge pull request #29502 from azat/fix-INSERT-with-metadata-changes 
Fix LOGICAL_ERROR for INSERT with concurrent ALTER
 2021-09-30 16:11:50 +03:00
Maksim Kita
c2407fee06 Fixed tests 2021-09-30 14:35:24 +03:00
Maksim Kita
87ce29886e Fixed tests 2021-09-30 13:52:07 +03:00
vdimir
4acd8f3147
Do not add extra parentheses for tuple negate unary operator 2021-09-30 13:47:29 +03:00
Maksim Kita
3e0d6a9709
Merge pull request #29531 from kssenii/fix-paths 
Fix pathStartsWith
 2021-09-30 12:49:52 +03:00
Raúl Marín
f58742014c Consistent naming 2021-09-30 11:05:21 +02:00
Vladimir C
bc8d925ac5
Merge pull request #29463 from azat/external-db-null 2021-09-30 11:38:04 +03:00
Nikolay Degterinsky
485e19ff84
Merge pull request #28955 from anneji-dev/alterTable 
Add default database for alter table
 2021-09-30 11:28:47 +03:00
MaxWk
3cbabe2c68 support conversion between date32 and datetime64 2021-09-30 15:09:36 +08:00
mergify[bot]
775d8baa0c
Merge branch 'master' into fix-paths 2021-09-30 06:17:04 +00:00
凌涛
a13d6b0aea Check all requests 2021-09-30 13:40:48 +08:00
Alexey Boykov
192633cb9b
Merge branch 'ClickHouse:master' into numeric_tuple_functions 2021-09-30 01:37:37 +03:00
kssenii
59cbe8579d Interrupt listener 2021-09-29 22:01:52 +00:00
凌涛
bd78af435d update code and add document 2021-09-30 04:51:02 +08:00
kssenii
07b979bda3 Rewrite to processors 2021-09-29 19:17:02 +00:00
Azat Khuzhin
4618636446 Fix LOGICAL_ERROR for INSERT with metadata changes in parallel 
In #28582 the problem was introduced due to it obtains new metadata
snapshot and so it may fail when it will check that headers does not
differs, like in [1]:

<details>

    {7b1256b1-09dc-4cc3-93d9-e9f6e26b0b99} <Fatal> : Logical error: 'Block structure mismatch in  function connect between ConvertingTransform and ReplicatedMergeTreeSink stream: different number of columns:
    a UInt8 UInt8(size = 0), b Int16 Int16(size = 0), c Float32 Float32(size = 0), d String String(size = 0), e Array(UInt8) Array(size = 0, UInt64(size = 0), UInt8(size = 0)), f Nullable(UUID) Nullable(size = 0, UUID(size = 0), UInt8(size = 0)), g Tuple(UInt8, UInt16) Tuple(size = 0, UInt8(size = 0), UInt16(size = 0)), h UInt64 UInt64(size = 0)
    a UInt8 UInt8(size = 0), b Int16 Int16(size = 0), c Float32 Float32(size = 0), d String String(size = 0), e Array(UInt8) Array(size = 0, UInt64(size = 0), UInt8(size = 0)), f Nullable(UUID) Nullable(size = 0, UUID(size = 0), UInt8(size = 0)), g Tuple(UInt8, UInt16) Tuple(size = 0, UInt8(size = 0), UInt16(size = 0))'.
    {7b1256b1-09dc-4cc3-93d9-e9f6e26b0b99} <Fatal> : Logical error: 'Block structure mismatch in  function connect between ConvertingTransform and ReplicatedMergeTreeSink stream: different number of columns:
    {} <Fatal> BaseDaemon: ########################################
    {} <Fatal> BaseDaemon: (version 21.11.1.8227, build id: E012027D3A9E94FD61655974C45C15F58215605B) (from thread 2180) (query_id: 7b1256b1-09dc-4cc3-93d9-e9f6e26b0b99) Received signal Aborted (6)
    {} <Fatal> BaseDaemon:
    {} <Fatal> BaseDaemon: Stack trace: 0x7f5661a3b18b 0x7f5661a1a859 0x1333bcb8 0x1333bdc2 0x1d9f04d1 0x1d9eec3c 0x1d9eeeca 0x1f7cc44c 0x1f79b992 0x1e52123f 0x1e522b8e 0x1eb7e27b 0x1eb7be44 0x1f751e26 0x1f75f0e5 0x23ba1179 0x23ba1988 0x23cef8b4 0x23cec39a 0x23ceb17c 0x7f5661c01609 0x7f5661b17293
    {} <Fatal> BaseDaemon: 4. raise @ 0x4618b in /usr/lib/x86_64-linux-gnu/libc-2.31.so
    {} <Fatal> BaseDaemon: 5. abort @ 0x25859 in /usr/lib/x86_64-linux-gnu/libc-2.31.so
    {} <Fatal> BaseDaemon: 6. ./obj-x86_64-linux-gnu/../src/Common/Exception.cpp:53: DB::handle_error_code(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, bool, std::__1::vector<void*, std::__1::allocator<void*> > const&) @ 0x1333bcb8 in /usr/bin/clickhouse
    {} <Fatal> BaseDaemon: 7. ./obj-x86_64-linux-gnu/../src/Common/Exception.cpp:60: DB::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, bool) @ 0x1333bdc2 in /usr/bin/clickhouse
    {} <Fatal> BaseDaemon: 8. ./obj-x86_64-linux-gnu/../src/Core/Block.cpp:32: void DB::onError<void>(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int) @ 0x1d9f04d1 in /usr/bin/clickhouse
    {} <Fatal> BaseDaemon: 9. ./obj-x86_64-linux-gnu/../src/Core/Block.cpp:86: void DB::checkBlockStructure<void>(DB::Block const&, DB::Block const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, bool) @ 0x1d9eec3c in /usr/bin/clickhouse
    {} <Fatal> BaseDaemon: 10. ./obj-x86_64-linux-gnu/../src/Core/Block.cpp:607: DB::assertCompatibleHeader(DB::Block const&, DB::Block const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) @ 0x1d9eeeca in /usr/bin/clickhouse
    {} <Fatal> BaseDaemon: 11. ./obj-x86_64-linux-gnu/../src/Processors/Port.cpp:19: DB::connect(DB::OutputPort&, DB::InputPort&) @ 0x1f7cc44c in /usr/bin/clickhouse
    {} <Fatal> BaseDaemon: 12. ./obj-x86_64-linux-gnu/../src/Processors/Chain.cpp:81: DB::Chain::addSource(std::__1::shared_ptr<DB::IProcessor>) @ 0x1f79b992 in /usr/bin/clickhouse
    {} <Fatal> BaseDaemon: 13. ./obj-x86_64-linux-gnu/../src/Interpreters/InterpreterInsertQuery.cpp:222: DB::InterpreterInsertQuery::buildChainImpl(std::__1::shared_ptr<DB::IStorage> const&, std::__1::shared_ptr<DB::StorageInMemoryMetadata const> const&, DB::Block const&, DB::ThreadStatus*, std::__1::atomic<unsigned long>*) @ 0x1e52123f in /usr/bin/clickhouse
    {} <Fatal> BaseDaemon: 14. ./obj-x86_64-linux-gnu/../src/Interpreters/InterpreterInsertQuery.cpp:368: DB::InterpreterInsertQuery::execute() @ 0x1e522b8e in /usr/bin/clickhouse

</details>

  [1]: https://clickhouse-test-reports.s3.yandex.net/29461/d6cd057e2c0c8b04d8a712a7b2a83526507ef512/functional_stateless_tests_(debug).html#fail1

Fixes: #28582 (cc: @KochetovNicolai)
 2021-09-29 21:56:59 +03:00
Azat Khuzhin
a4f2663b82 Log queries to external databases (since they may be rewriten) 
By rewritten means that some expression from WHERE can be removed, yes
there is external_table_strict_query, but sometimes it is too strict.
 2021-09-29 21:56:28 +03:00
Azat Khuzhin
6ddfbf39c8 Cover IS NULL/IS NOT NULL for external dbs in unit tests 2021-09-29 21:45:09 +03:00
Azat Khuzhin
24f55e29d1 Add support for IS NULL/IS NOT NULL for external dbs 2021-09-29 21:45:09 +03:00
Anton Popov
0dfe96fc02 generalize code in IColumn::permute 2021-09-29 20:56:10 +03:00
kssenii
2c005db9ad Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into interactive-mode-for-clickhouse-local 2021-09-29 17:45:15 +00:00
Nikolai Kochetov
68f8b9d235 Update ColumnGathererStream 2021-09-29 20:45:01 +03:00
Raúl Marín
0ee5c0bff5 Use RWLock in StorageJoin to avoid deadlocks 2021-09-29 19:30:07 +02:00
Mike Kot
61e0c6208b Making Monotonicity an aggregate to use with designated initializers 2021-09-29 18:01:26 +02:00
Maksim Kita
a5151fb49b Function accurateCastOrDefault updated implementation 2021-09-29 17:52:08 +03:00
mergify[bot]
6a2ac6e871
Merge branch 'master' into yandex-to-clickhouse-in-configs 2021-09-29 13:29:08 +00:00
kssenii
46a4525c21 Fix 2021-09-29 12:53:09 +00:00
Maksim Kita
7edf63162f ExternalXMLLoaders reload with server configuration support 2021-09-29 15:52:58 +03:00
Vitaly Baranov
0c33f1121b
Merge pull request #28012 from amosbird/jsonextractstring2 
Handle null column correctly in JSONFunction.
 2021-09-29 14:58:40 +03:00
Artur
fcebf7b985 correct tests 2021-09-29 11:29:24 +00:00
Mike Kot
7670ea50c1 Replacing std::function to Fn in some places 
to avoid dynamic allocating while keeping desired function signature
clear.
Simplifying SimpleCache (CachedFn) using C++20
 2021-09-29 13:09:39 +02:00
Kruglov Pavel
14be2f31f5
Merge pull request #29475 from ClickHouse/fix-29010 
Remove filter column from HAVING when it is not needed.
 2021-09-29 13:03:27 +03:00
Nikolai Kochetov
8d85590900
Merge pull request #29494 from ClickHouse/fix-possible-pushing-executor-hang 
Fix possible hang in PushingAsyncPipelineExecutor.
 2021-09-29 12:16:26 +03:00
Maksim Kita
1a0feaaf8a
Merge pull request #29511 from MaxWk/huawei-obs-support 
support obs storage
 2021-09-29 12:02:01 +03:00
Vladimir C
27f0f9fcf3
Merge pull request #21320 from arenadata/ADQM-138 2021-09-29 11:53:54 +03:00
凌涛
c298fba774 Support HSTS in Clickhouse HTTP server 2021-09-29 16:37:48 +08:00
MaxWk
723ff795a8 support obs storage 2021-09-29 14:11:28 +08:00
alexey-milovidov
be427555e5
Merge pull request #29476 from ClickHouse/fix_29456 
Fix table name escaping in Lazy database
 2021-09-29 02:45:40 +03:00
alexey-milovidov
d46dfd0ddd
Merge pull request #29446 from azat/isNull-isNotNull-format 
Transform isNull/isNotNull to IS NULL/IS NOT NULL (for external dbs)
 2021-09-29 00:56:57 +03:00
alexey-milovidov
f6db7552b6
Merge pull request #29444 from CurtizJ/fix-deadlock-async-inserts 
Fix deadlock in concurrent async inserts and truncates
 2021-09-29 00:56:29 +03:00
Nikita Mikhaylov
340fb049cf
Merge pull request #29497 from nikitamikhaylov/fuzz 
Use `max_parse_depth` in fuzzers
 2021-09-28 23:18:07 +03:00
Nikita Mikhaylov
0e602d85f0 Done 2021-09-28 20:16:32 +00:00
alesapin
7962ea5958
Merge pull request #29417 from ClickHouse/compressed_snapshots 
Compress keeper snapshots with default ZSTD codec
 2021-09-28 23:09:43 +03:00
Maksim Kita
0279afa738
Merge pull request #29474 from kitaisreal/bloom-filter-indexes-updated-tests 
Bloom filter indexes updated tests
 2021-09-28 22:40:27 +03:00
Maksim Kita
9ecd0a3f4f
Merge pull request #29220 from ClickHouse/full-text-bloom-filter-index-map-data-type 
Merging #28511
 2021-09-28 22:40:17 +03:00
Anton Popov
369821634b
Merge pull request #29390 from azat/fix-async-insert-tests 
Fix 02015_async_inserts_2 flakiness
 2021-09-28 21:07:26 +03:00
Nikolai Kochetov
ee2c12b4ab Fix pissble hang in PushingAsyncPipelineExecutor. 2021-09-28 20:47:20 +03:00
alexey-milovidov
1e6f9ac635
Merge branch 'master' into yandex-to-clickhouse-in-configs 2021-09-28 20:26:44 +03:00
Nikita Mikhaylov
4ac80808fe
Merge pull request #29455 from azat/enable-memory-profiler 
Enable per-query memory profiler (set to memory_profiler_step=4MiB) globally
 2021-09-28 20:00:06 +03:00
mo-avatar
6c16ec83eb in memory part lost when freeze 2021-09-28 22:52:01 +08:00
Nikolai Kochetov
ebe4cb0bbb Fix tests. 2021-09-28 15:32:43 +03:00
Anton Popov
83fd853169 Merge remote-tracking branch 'upstream/master' into HEAD 2021-09-28 15:03:27 +03:00
Ilya Golshtein
d67bc0bef4 minor fixes in ORs in JOIN 2021-09-28 14:11:33 +03:00
Ilya Golshtein
7ebc16c1b3 get rid of DNF and related features in ORs in JOIN 2021-09-28 14:11:33 +03:00
Ilya Golshtein
1dc7fc5962 fix bug found by PVS in ORs in JOIN 2021-09-28 14:11:33 +03:00
Ilya Golshtein
626bfdf23c compatible filter conditions, fixes and new tests in ORs in JOIN 2021-09-28 14:11:33 +03:00
Ilya Golshtein
bbd548e81d bypass filer conditions in DNF in ORs in JOIN (part 2) 2021-09-28 14:11:33 +03:00
Ilya Golshtein
336b2a4c68 bypass filer conditions in DNF in ORs in JOIN (part 1) 2021-09-28 14:11:33 +03:00
vdimir
760a92c490 Do not allow in optimizeClauses conditions for different table joined via OR 2021-09-28 14:11:32 +03:00
Ilya Golshtein
17e6cfbefb DNF bugfix in ORs in JOIN 2021-09-28 14:11:32 +03:00
Ilya Golshtein
29b911f86b rebase collisions fixed in ORs in JOIN 2021-09-28 14:11:32 +03:00
Ilya Golshtein
aa4751a632 checkStackSize moved to the top of DNF::distributed in ORs in JOIN 2021-09-28 14:11:32 +03:00
Ilya Golshtein
6daef66364 MAX_DISJUNCTS instead of MAX_ORS in ORs in JOIN 2021-09-28 14:11:32 +03:00
Ilya Golshtein
78ad6bf529 MAX_ORS, checkStackSize and beautification per review in ORs in JOIN 2021-09-28 14:11:32 +03:00
Ilya Golshtein
fa6c2a64e4 minor merge mistakes fixed in ORs in JOIN 2021-09-28 14:11:32 +03:00
Ilya Golshtein
8057e052a6 crash fix, style fixes, ASTs moved out of TableJoin in ORs in JOIN 2021-09-28 14:11:32 +03:00
Ilya Golshtein
637ff19f79 optimizeDisjuncts in ORs in JOIN 2021-09-28 14:11:31 +03:00
vdimir
3b35ab6e8c Not implemented for asof and auto join with multiple ORs 2021-09-28 14:11:31 +03:00
vdimir
f8e8f6da14 Use table_join->getAllNames in HashJoin.cpp 2021-09-28 14:11:31 +03:00
vdimir
71b6c9414c Minor changes related to JOIN ON ORs 2021-09-28 14:11:31 +03:00
Ilya Golshtein
300eb5098d fix rebase collisions in ORs in JOIN 2021-09-28 14:11:31 +03:00
vdimir
0a9a028c6f fix 2021-09-28 14:11:31 +03:00
vdimir
46187a73ee wip 2021-09-28 14:11:31 +03:00
vdimir
8e2637aab2 Store all related to one join disjunct in JoinOnClause, pt1 2021-09-28 14:11:31 +03:00
Ilya Golshtein
aa3d8a431a No redundant convertColumnsToNullable if ORs in JOIN 2021-09-28 14:11:30 +03:00
Ilya Golshtein
5ed6b26c9d any_join_distinct_right_table_keys tests and fix for ORs in JOIN 2021-09-28 14:11:30 +03:00
Ilya Golshtein
be034444ba fix any_join_distinct_right_table_keys crash if ORs in JOIN 2021-09-28 14:11:30 +03:00
vdimir
d37b88fb07 fix HashJoin::initRightBlockStructure 2021-09-28 14:11:30 +03:00
vdimir
38374ba07b fix join->getNonJoinedBlocks 2021-09-28 14:11:30 +03:00
vdimir
a8e1de51e1 fix TableJoin::leftToRightKeyRemap 2021-09-28 14:11:30 +03:00
Ilya Golshtein
24e3e04203 BlockWithFlags removed from vdimir's change 2021-09-28 14:11:30 +03:00
vdimir
6b6592fda7 Use JoinUsedFlags with multiple dijuncts 2021-09-28 14:11:30 +03:00
Ilya Golshtein
68d6a0ee85 prohibit storage join if ORs 2021-09-28 14:11:29 +03:00
Ilya Golshtein
4bddd0bc71 make PVS check happy with ORs in JOIN 2021-09-28 14:11:29 +03:00
Ilya Golshtein
d0990e270a making gcc happy, minor improvements in ORs in JOIN 2021-09-28 14:11:29 +03:00
Ilya Golshtein
b03415bd0e Fixed other bug if OR together with filter conditions 2021-09-28 14:11:29 +03:00
Ilya Golshtein
f94b2cba2d Fixed bug if OR together with filter conditions, some comments 2021-09-28 14:11:29 +03:00
Ilya Golshtein
a18c6f7f8e merge error fixed 2021-09-28 14:11:29 +03:00
Ilya Golshtein
db50015eed review changes 1 - ASTPtr, some comments 2021-09-28 14:11:29 +03:00
Ilya Golshtein
3766d47f31 ORs in JOINs 2021-09-28 14:11:28 +03:00
Maksim Kita
1e5e93830b Merge branch 'master' into cast_to_value_or_default 2021-09-28 14:04:10 +03:00
Viachaslau Boben
762904adbd Add nfd and perf test 2021-09-28 13:42:37 +03:00
Alexander Tokmakov
462088f500 fix table name escaping 2021-09-28 13:14:05 +03:00
Maksim Kita
cd1e72e312
Merge pull request #29454 from Avogar/fix-least-greatest 
Fix logical error in functions greatest/least
 2021-09-28 13:13:30 +03:00
Nikolai Kochetov
1a4ee1f77f Remove filter column from HAVING when it is not needed. 2021-09-28 13:01:47 +03:00
Maksim Kita
f2402e1a10 Fixed tests 2021-09-28 12:52:07 +03:00
Maksim Kita
71fb9e6059 Bloom filter indexes updated tests 2021-09-28 12:38:27 +03:00
Maksim Kita
9f9f36b480
Merge pull request #29464 from azat/std-cerr-cout-style-check 
Add std::cerr/std::cout style check
 2021-09-28 12:12:44 +03:00
Vladimir C
6c2db9c29b
Merge pull request #29428 from vdimir/rocksdb-istream-to-source 2021-09-28 12:06:41 +03:00
alesapin
3cd3850edf
Update gtest_coordination.cpp 2021-09-28 11:51:02 +03:00
Azat Khuzhin
9a23b20965 Remove debug from MergeTreeIndexConditionBloomFilter::traverseASTEquals() 2021-09-28 10:28:28 +03:00
alesapin
eb5402622e Better test 2021-09-28 10:24:01 +03:00
Amos Bird
07a278e426
Better 2021-09-28 11:38:29 +08:00
Amos Bird
b25c3fb659
Handle null column correctly in JSONFunction 2021-09-28 11:38:29 +08:00
alexey-milovidov
995913e89c
Merge branch 'master' into yandex-to-clickhouse-in-configs 2021-09-28 02:27:36 +03:00
Pavel Kruglov
cc40e36514 Fix logical error in function greatest/least 2021-09-27 23:13:01 +03:00
Azat Khuzhin
a218c3a28c Enable per-query memory profiler (set to memory_profiler_step=4MiB) globally 
Let's see at performance tests. And yes, they should show the
difference, if any, since they uses clickhouse python driver, which does
not knows about default settings, so new server will have this setting
set, while old server - upstream/master, will not.
 2021-09-27 22:46:35 +03:00
Azat Khuzhin
2378491c04 Transform isNull/isNotNull to IS NULL/IS NOT NULL (for external dbs) 
This is required to support queries (like `foo IS NULL`) to external
databases (i.e. MySQL).

This patch had been submitted separately, due to:
- avoid mixing patches
- run CI (and update tests references)
 2021-09-27 22:24:09 +03:00
Maksim Kita
7861c26eaf
Merge pull request #29431 from kitaisreal/map-bloom-filter-index-map-values-equals-support 
Map bloom filter index mapValues equals function support
 2021-09-27 22:03:53 +03:00
Nikolai Kochetov
236d71ea94
Merge pull request #28582 from ClickHouse/rewrite-pushing-to-views 
Rewrite PushingToViews
 2021-09-27 21:19:11 +03:00
Anton Popov
9a58e4a8ba fix deadlock in concurrent async inserts and truncates 2021-09-27 19:39:32 +03:00
Viachaslau Boben
b3325772f7 Add normalizeUTF8 function with NFC normalization 2021-09-27 18:47:43 +03:00
tavplubix
e9749b0027
Merge pull request #29323 from ClickHouse/fix_assertion_ddl_dependency 
Fix assertion in DDLDependencyVisitor
 2021-09-27 18:06:32 +03:00
alesapin
ddd80c8e72
Merge pull request #29264 from Enmk/governance/table_comment_update 
Implemented modifying table comments with `ALTER TABLE t MODIFY COMMENT 'value'`
 2021-09-27 18:02:14 +03:00
Maksim Kita
e290f19957 Merge branch 'master' into full-text-bloom-filter-index-map-data-type 2021-09-27 17:50:07 +03:00
Maksim Kita
b28d6e76cc Map bloom filter full text index added support for mapKeys, mapValues 2021-09-27 17:26:25 +03:00
Maksim Kita
cd9c796e6f Map bloom filter index mapValues equals function support 2021-09-27 17:23:56 +03:00
alesapin
608644436c Fix tidy 2021-09-27 17:21:10 +03:00
vdimir
aea2b952fd
Merge EmbeddedRocksDBBlockInputStream and EmbeddedRocksDBSource 2021-09-27 16:16:34 +03:00
Maksim Kita
a6a19c6267
Merge pull request #29405 from kitaisreal/added-setting-use-skip-indexes 
Added setting use_skip_indexes
 2021-09-27 16:06:54 +03:00
Kseniia Sumarokova
beda6bee2e
Merge pull request #29276 from kssenii/hdfs-fix-assertion 
Fix !hasPendingData() failed assertion in hdfs read buffer
 2021-09-27 15:47:17 +03:00
alesapin
a73064679d Compress keeper snapshots with default ZSTD codec 2021-09-27 14:54:04 +03:00
Nikita Mikhaylov
3741b8a17e
Merge pull request #29337 from amosbird/projection-fix18 
Fix projection merges and mutations.
 2021-09-27 13:45:13 +03:00
vdimir
8fe17dd3d1
Revert "Rename EmbeddedRocksDBBlockInputStream -> EmbeddedRocksDBSource" 
This reverts commit 3ab217031e50a0b4d19fb24d72088e353cdfa510.
 2021-09-27 12:49:14 +03:00
vdimir
dc7393e810
Rename EmbeddedRocksDBBlockInputStream -> EmbeddedRocksDBSource 2021-09-27 12:49:13 +03:00
vdimir
222ab78931
Convert EmbeddedRocksDBBlockInputStream to ISource 2021-09-27 12:49:13 +03:00
Vladimir C
70dc43a72d
Merge pull request #29388 from aiven/kmichel-fix-replicated-storage-reload 2021-09-27 12:41:24 +03:00
Maksim Kita
d0f53cfc31 Added setting use_skip_indexes 2021-09-27 12:17:00 +03:00
Maksim Kita
92c484f0d3
Merge pull request #29252 from Enmk/fix_some_functions_exception_messages 
Better exception messages for some String-related functions
 2021-09-27 11:50:42 +03:00
Nikolai Kochetov
78a7665f43 Merge branch 'master' into rewrite-pushing-to-views 2021-09-27 10:56:50 +03:00
Nikolai Kochetov
09232788c0 Fix GRPC tests. 2021-09-27 10:55:33 +03:00
Azat Khuzhin
c495297e65 Remove unused reset_timeout/access_timeout from AsynchronousInsertQueue 2021-09-27 10:02:27 +03:00
Vasily Nemkov
544cccb2fb
Forward-declared LOGICAL_ERROR to make style check happy 2021-09-27 09:43:13 +03:00