thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-11-29 02:52:13 +00:00
Code Issues Packages Projects Releases Wiki Activity
67,924 Commits 9,188 Branches 1,782 Tags 2.2 GiB
eefb798e33
Commit Graph

15461 Commits

Author SHA1 Message Date
alesapin
4c85dae572
Merge pull request #25743 from ClickHouse/fix_aggregation_ttl 
Fix bug in execution of TTL GROUP BY
 2021-07-07 10:49:16 +03:00
alesapin
0d8844c510
Merge pull request #25884 from ClickHouse/fix_drop_part_in_queue 
Relax `DROP PART` guarantees and turn on checks in ReplicationQueue.
 2021-07-07 10:48:48 +03:00
Azat Khuzhin
5bc0533712 Add a note for padded to 8 bytes keys in StringHashTable 2021-07-07 10:42:33 +03:00
Azat Khuzhin
5e120b8d37 Fix stack-buffer-overflow in custom TLDs due to StringHashTable copy 8 bytes at a time 
ASan reports [1]:

    ==164==ERROR: AddressSanitizer: stack-buffer-overflow on address 0x7f0209dd4abf at pc 0x00000b75b7c5 bp 0x7f0209dd4760 sp 0x7f0209dd4758
    READ of size 8 at 0x7f0209dd4abf thread T4 (TCPHandler)
        0 0xb75b7c4 in auto StringHashTable<StringHashSetSubMaps<Allocator<true, true> > >::dispatch<StringHashTable<StringHashSetSubMaps<Allocator<true, true> > > const, StringRef const&, StringHashTable<StringHashSetSubMaps<Allocator<true, true> > >::FindCallable>(StringHashTable<StringHashSetSubMaps<Allocator<true, true> > > const&, StringRef const&, StringHashTable<StringHashSetSubMaps<Allocator<true, true> > >::FindCallable&&) obj-x86_64-linux-gnu/../src/Common/HashTable/StringHashTable.h:283:21
        1 0xb75b7c4 in StringHashTable<StringHashSetSubMaps<Allocator<true, true> > >::has(StringRef const&, unsigned long) const obj-x86_64-linux-gnu/../src/Common/HashTable/StringHashTable.h:365:16
        2 0xb75b7c4 in DB::TLDList::has(StringRef const&) const obj-x86_64-linux-gnu/../src/Common/TLDListsHolder.cpp:31:26
        3 0x1c4a6046 in void DB::ExtractFirstSignificantSubdomain<true>::executeCustom<DB::FirstSignificantSubdomainCustomLookup>(DB::FirstSignificantSubdomainCustomLookup const&, char const*, unsigned long, char const*&, unsigned long&, char const**) (/usr/bin/clickhouse+0x1c4a6046)
        4 0x1c4a3586 in DB::FunctionCutToFirstSignificantSubdomainCustomImpl<DB::CutToFirstSignificantSubdomainCustom<true>, DB::NameCutToFirstSignificantSubdomainCustom>::executeImpl(std::__1::vector<DB::ColumnWithTypeAndName, std::__1::allocator<DB::ColumnWithTypeAndName> > const&, std::__1::shared_ptr<DB::IDataType const> const&, unsigned long) const (/usr/bin/clickhouse+0x1c4a3586)
        5 0x10d96e34 in DB::IFunction::executeImplDryRun(std::__1::vector<DB::ColumnWithTypeAndName, std::__1::allocator<DB::ColumnWithTypeAndName> > const&, std::__1::shared_ptr<DB::IDataType const> const&, unsigned long) const (/usr/bin/clickhouse+0x10d96e34)
        6 0x10d9648b in DB::FunctionToExecutableFunctionAdaptor::executeDryRunImpl(std::__1::vector<DB::ColumnWithTypeAndName, std::__1::allocator<DB::ColumnWithTypeAndName> > const&, std::__1::shared_ptr<DB::IDataType const> const&, unsigned long) const (/usr/bin/clickhouse+0x10d9648b)
        7 0x200ed79b in DB::IExecutableFunction::executeWithoutLowCardinalityColumns(std::__1::vector<DB::ColumnWithTypeAndName, std::__1::allocator<DB::ColumnWithTypeAndName> > const&, std::__1::shared_ptr<DB::IDataType const> const&, unsigned long, bool) const obj-x86_64-linux-gnu/../src/Functions/IFunction.cpp:212:15
        8 0x200ee436 in DB::IExecutableFunction::execute(std::__1::vector<DB::ColumnWithTypeAndName, std::__1::allocator<DB::ColumnWithTypeAndName> > const&, std::__1::shared_ptr<DB::IDataType const> const&, unsigned long, bool) const obj-x86_64-linux-gnu/../src/Functions/IFunction.cpp:257:22
        9 0x20cd6f6f in DB::ActionsDAG::addFunction(std::__1::shared_ptr<DB::IFunctionOverloadResolver> const&, std::__1::vector<DB::ActionsDAG::Node const*, std::__1::allocator<DB::ActionsDAG::Node const*> >, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >) obj-x86_64-linux-gnu/../src/Interpreters/ActionsDAG.cpp:214:37
        10 0x2124c8a7 in DB::ScopeStack::addFunction(std::__1::shared_ptr<DB::IFunctionOverloadResolver> const&, std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >) obj-x86_64-linux-gnu/../src/Interpreters/ActionsVisitor.cpp:570:51
        11 0x2125c80d in DB::ActionsMatcher::Data::addFunction(std::__1::shared_ptr<DB::IFunctionOverloadResolver> const&, std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >) obj-x86_64-linux-gnu/../src/Interpreters/ActionsVisitor.h:169:27
        12 0x2125c80d in DB::ActionsMatcher::visit(DB::ASTFunction const&, std::__1::shared_ptr<DB::IAST> const&, DB::ActionsMatcher::Data&) obj-x86_64-linux-gnu/../src/Interpreters/ActionsVisitor.cpp:1061:14
        13 0x212522fb in DB::ActionsMatcher::visit(DB::ASTFunction const&, std::__1::shared_ptr<DB::IAST> const&, DB::ActionsMatcher::Data&) obj-x86_64-linux-gnu/../src/Interpreters/ActionsVisitor.cpp:971:17
        14 0x2121354e in DB::InDepthNodeVisitor<DB::ActionsMatcher, true, std::__1::shared_ptr<DB::IAST> const>::visit(std::__1::shared_ptr<DB::IAST> const&) obj-x86_64-linux-gnu/../src/Interpreters/InDepthNodeVisitor.h:34:13
        15 0x211e17c7 in DB::ExpressionAnalyzer::getRootActions(std::__1::shared_ptr<DB::IAST> const&, bool, std::__1::shared_ptr<DB::ActionsDAG>&, bool) obj-x86_64-linux-gnu/../src/Interpreters/ExpressionAnalyzer.cpp:421:48
        16 0x21204024 in DB::ExpressionAnalyzer::getConstActions(std::__1::vector<DB::ColumnWithTypeAndName, std::__1::allocator<DB::ColumnWithTypeAndName> > const&) obj-x86_64-linux-gnu/../src/Interpreters/ExpressionAnalyzer.cpp:1423:5
        17 0x230f7216 in DB::KeyCondition::getBlockWithConstants(std::__1::shared_ptr<DB::IAST> const&, std::__1::shared_ptr<DB::TreeRewriterResult const> const&, std::__1::shared_ptr<DB::Context const>) obj-x86_64-linux-gnu/../src/Storages/MergeTree/KeyCondition.cpp:385:103
        18 0x22877f9e in DB::(anonymous namespace)::replaceConstantExpressions(std::__1::shared_ptr<DB::IAST>&, std::__1::shared_ptr<DB::Context const>, DB::NamesAndTypesList const&, std::__1::shared_ptr<DB::IStorage const>, std::__1::shared_ptr<DB::StorageInMemoryMetadata const> const&) obj-x86_64-linux-gnu/../src/Storages/StorageDistributed.cpp:280:34
        19 0x22877f9e in DB::StorageDistributed::skipUnusedShards(std::__1::shared_ptr<DB::Cluster>, std::__1::shared_ptr<DB::IAST> const&, std::__1::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::__1::shared_ptr<DB::Context const>) const obj-x86_64-linux-gnu/../src/Storages/StorageDistributed.cpp:1091:5
        20 0x2285d215 in DB::StorageDistributed::getOptimizedCluster(std::__1::shared_ptr<DB::Context const>, std::__1::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::__1::shared_ptr<DB::IAST> const&) const obj-x86_64-linux-gnu/../src/Storages/StorageDistributed.cpp:1015:32
        21 0x2285a9c4 in DB::StorageDistributed::getQueryProcessingStage(std::__1::shared_ptr<DB::Context const>, DB::QueryProcessingStage::Enum, std::__1::shared_ptr<DB::StorageInMemoryMetadata const> const&, DB::SelectQueryInfo&) const obj-x86_64-linux-gnu/../src/Storages/StorageDistributed.cpp:500:40
        22 0x2183a4b2 in DB::InterpreterSelectQuery::getSampleBlockImpl() obj-x86_64-linux-gnu/../src/Interpreters/InterpreterSelectQuery.cpp:616:31
        23 0x21828db4 in DB::InterpreterSelectQuery::InterpreterSelectQuery(std::__1::shared_ptr<DB::IAST> const&, std::__1::shared_ptr<DB::Context const>, std::__1::shared_ptr<DB::IBlockInputStream> const&, std::__1::optional<DB::Pipe>, std::__1::shared_ptr<DB::IStorage> const&, DB::SelectQueryOptions const&, std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&, std::__1::shared_ptr<DB::StorageInMemoryMetadata const> const&)::$_1::operator()(bool) const obj-x86_64-linux-gnu/../src/Interpreters/InterpreterSelectQuery.cpp:506:25
        24 0x2181b652 in DB::InterpreterSelectQuery::InterpreterSelectQuery(std::__1::shared_ptr<DB::IAST> const&, std::__1::shared_ptr<DB::Context const>, std::__1::shared_ptr<DB::IBlockInputStream> const&, std::__1::optional<DB::Pipe>, std::__1::shared_ptr<DB::IStorage> const&, DB::SelectQueryOptions const&, std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&, std::__1::shared_ptr<DB::StorageInMemoryMetadata const> const&) obj-x86_64-linux-gnu/../src/Interpreters/InterpreterSelectQuery.cpp:509:5
        25 0x21817cbe in DB::InterpreterSelectQuery::InterpreterSelectQuery(std::__1::shared_ptr<DB::IAST> const&, std::__1::shared_ptr<DB::Context const>, DB::SelectQueryOptions const&, std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&) obj-x86_64-linux-gnu/../src/Interpreters/InterpreterSelectQuery.cpp:161:7
        26 0x21dd0eb5 in std::__1::__unique_if<DB::InterpreterSelectQuery>::__unique_single std::__1::make_unique<DB::InterpreterSelectQuery, std::__1::shared_ptr<DB::IAST> const&, std::__1::shared_ptr<DB::Context>&, DB::SelectQueryOptions&, std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&>(std::__1::shared_ptr<DB::IAST> const&, std::__1::shared_ptr<DB::Context>&, DB::SelectQueryOptions&, std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&) obj-x86_64-linux-gnu/../contrib/libcxx/include/memory:2068:32
        27 0x21dd0eb5 in DB::InterpreterSelectWithUnionQuery::buildCurrentChildInterpreter(std::__1::shared_ptr<DB::IAST> const&, std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&) obj-x86_64-linux-gnu/../src/Interpreters/InterpreterSelectWithUnionQuery.cpp:212:16
        28 0x21dcd0e7 in DB::InterpreterSelectWithUnionQuery::InterpreterSelectWithUnionQuery(std::__1::shared_ptr<DB::IAST> const&, std::__1::shared_ptr<DB::Context const>, DB::SelectQueryOptions const&, std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&) obj-x86_64-linux-gnu/../src/Interpreters/InterpreterSelectWithUnionQuery.cpp:134:13
        29 0x211afe79 in std::__1::__unique_if<DB::InterpreterSelectWithUnionQuery>::__unique_single std::__1::make_unique<DB::InterpreterSelectWithUnionQuery, std::__1::shared_ptr<DB::IAST>&, std::__1::shared_ptr<DB::Context>&, DB::SelectQueryOptions const&>(std::__1::shared_ptr<DB::IAST>&, std::__1::shared_ptr<DB::Context>&, DB::SelectQueryOptions const&) obj-x86_64-linux-gnu/../contrib/libcxx/include/memory:2068:32
        30 0x211afe79 in DB::InterpreterFactory::get(std::__1::shared_ptr<DB::IAST>&, std::__1::shared_ptr<DB::Context>, DB::SelectQueryOptions const&) obj-x86_64-linux-gnu/../src/Interpreters/InterpreterFactory.cpp:110:16
        31 0x22273f97 in DB::executeQueryImpl(char const*, char const*, std::__1::shared_ptr<DB::Context>, bool, DB::QueryProcessingStage::Enum, bool, DB::ReadBuffer*) obj-x86_64-linux-gnu/../src/Interpreters/executeQuery.cpp:524:28
        32 0x22270ce2 in DB::executeQuery(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::shared_ptr<DB::Context>, bool, DB::QueryProcessingStage::Enum, bool) obj-x86_64-linux-gnu/../src/Interpreters/executeQuery.cpp:913:30
        33 0x23905879 in DB::TCPHandler::runImpl() obj-x86_64-linux-gnu/../src/Server/TCPHandler.cpp:312:24
        34 0x2392b81c in DB::TCPHandler::run() obj-x86_64-linux-gnu/../src/Server/TCPHandler.cpp:1622:9
        35 0x2ab1fd8e in Poco::Net::TCPServerConnection::start() obj-x86_64-linux-gnu/../contrib/poco/Net/src/TCPServerConnection.cpp:43:3
        36 0x2ab20952 in Poco::Net::TCPServerDispatcher::run() obj-x86_64-linux-gnu/../contrib/poco/Net/src/TCPServerDispatcher.cpp:115:20
        37 0x2adfa3f4 in Poco::PooledThread::run() obj-x86_64-linux-gnu/../contrib/poco/Foundation/src/ThreadPool.cpp:199:14
        38 0x2adf4716 in Poco::ThreadImpl::runnableEntry(void*) obj-x86_64-linux-gnu/../contrib/poco/Foundation/src/Thread_POSIX.cpp:345:27
        39 0x7f02e66f2608 in start_thread (/lib/x86_64-linux-gnu/libpthread.so.0+0x9608)
        40 0x7f02e6619292 in clone (/lib/x86_64-linux-gnu/libc.so.6+0x122292)

    Address 0x7f0209dd4abf is located in stack of thread T4 (TCPHandler) at offset 447 in frame
        0 0x1c4a2c6f in DB::FunctionCutToFirstSignificantSubdomainCustomImpl<DB::CutToFirstSignificantSubdomainCustom<true>, DB::NameCutToFirstSignificantSubdomainCustom>::executeImpl(std::__1::vector<DB::ColumnWithTypeAndName, std::__1::allocator<DB::ColumnWithTypeAndName> > const&, std::__1::shared_ptr<DB::IDataType const> const&, unsigned long) const (/usr/bin/clickhouse+0x1c4a2c6f)

      This frame has 16 object(s):
        [32, 40) 'ref.tmp.i168'
        [64, 72) 'tmp_data.i.i'
        [96, 104) 'tmp_length.i.i'
        [128, 136) 'domain_end.i.i'
        [160, 216) 'ref.tmp.i132'
        [256, 312) 'ref.tmp.i'
        [352, 360) 'tld_lookup'
        [384, 408) 'ref.tmp'
        [448, 472) 'ref.tmp11' <== Memory access at offset 447 partially underflows this variable
        [512, 536) 'ref.tmp14'
        [576, 632) 'ref.tmp20'
        [672, 696) 'ref.tmp65'
        [736, 760) 'ref.tmp66'
        [800, 824) 'ref.tmp67'
        [864, 888) 'ref.tmp68'
        [928, 952) 'ref.tmp78'
    HINT: this may be a false positive if your program uses some custom stack unwind mechanism, swapcontext or vfork
          (longjmp and C++ exceptions *are* supported)
    Thread T4 (TCPHandler) created by T0 here:
        0 0xb51940a in pthread_create (/usr/bin/clickhouse+0xb51940a)
        1 0x2adf3a9f in Poco::ThreadImpl::startImpl(Poco::SharedPtr<Poco::Runnable, Poco::ReferenceCounter, Poco::ReleasePolicy<Poco::Runnable> >) obj-x86_64-linux-gnu/../contrib/poco/Foundation/src/Thread_POSIX.cpp:202:6
        2 0x2adf699a in Poco::Thread::start(Poco::Runnable&) obj-x86_64-linux-gnu/../contrib/poco/Foundation/src/Thread.cpp:128:2
        3 0x2adfa998 in Poco::PooledThread::start() obj-x86_64-linux-gnu/../contrib/poco/Foundation/src/ThreadPool.cpp:85:10
        4 0x2adfa998 in Poco::ThreadPool::ThreadPool(int, int, int, int) obj-x86_64-linux-gnu/../contrib/poco/Foundation/src/ThreadPool.cpp:252:12
        5 0xb582c25 in DB::Server::main(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&) obj-x86_64-linux-gnu/../programs/server/Server.cpp:915:22
        6 0x2ab511a5 in Poco::Util::Application::run() obj-x86_64-linux-gnu/../contrib/poco/Util/src/Application.cpp:334:8
        7 0xb56a89c in DB::Server::run() obj-x86_64-linux-gnu/../programs/server/Server.cpp:392:25
        8 0x2ab956f7 in Poco::Util::ServerApplication::run(int, char**) obj-x86_64-linux-gnu/../contrib/poco/Util/src/ServerApplication.cpp:611:9
        9 0xb566519 in mainEntryClickHouseServer(int, char**) obj-x86_64-linux-gnu/../programs/server/Server.cpp:171:20
        10 0xb56224a in main obj-x86_64-linux-gnu/../programs/main.cpp:366:12
        11 0x7f02e651e0b2 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x270b2)

    SUMMARY: AddressSanitizer: stack-buffer-overflow obj-x86_64-linux-gnu/../src/Common/HashTable/StringHashTable.h:283:21 in auto StringHashTable<StringHashSetSubMaps<Allocator<true, true> > >::dispatch<StringHashTable<StringHashSetSubMaps<Allocator<true, true> > > const, StringRef const&, StringHashTable<StringHashSetSubMaps<Allocator<true, true> > >::FindCallable>(StringHashTable<StringHashSetSubMaps<Allocator<true, true> > > const&, StringRef const&, StringHashTable<StringHashSetSubMaps<Allocator<true, true> > >::FindCallable&&)
    Shadow bytes around the buggy address:
      0x0fe0c13b2900: 00 00 f3 f3 00 00 00 00 00 00 00 00 00 00 00 00
      0x0fe0c13b2910: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
      0x0fe0c13b2920: f1 f1 f1 f1 f8 f2 f2 f2 00 f2 f2 f2 00 f2 f2 f2
      0x0fe0c13b2930: 00 f2 f2 f2 f8 f8 f8 f8 f8 f8 f8 f2 f2 f2 f2 f2
      0x0fe0c13b2940: f8 f8 f8 f8 f8 f8 f8 f2 f2 f2 f2 f2 00 f2 f2 f2
    =>0x0fe0c13b2950: f8 f8 f8 f2 f2 f2 f2[f2]00 00 00 f2 f2 f2 f2 f2
      0x0fe0c13b2960: 00 00 00 f2 f2 f2 f2 f2 f8 f8 f8 f8 f8 f8 f8 f2
      0x0fe0c13b2970: f2 f2 f2 f2 f8 f8 f8 f2 f2 f2 f2 f2 f8 f8 f8 f2
      0x0fe0c13b2980: f2 f2 f2 f2 f8 f8 f8 f2 f2 f2 f2 f2 f8 f8 f8 f2
      0x0fe0c13b2990: f2 f2 f2 f2 f8 f8 f8 f3 f3 f3 f3 f3 00 00 00 00
      0x0fe0c13b29a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
    Shadow byte legend (one shadow byte represents 8 application bytes):
      Addressable:           00
      Partially addressable: 01 02 03 04 05 06 07
      Heap left redzone:       fa
      Freed heap region:       fd
      Stack left redzone:      f1
      Stack mid redzone:       f2
      Stack right redzone:     f3
      Stack after return:      f5
      Stack use after scope:   f8
      Global redzone:          f9
      Global init order:       f6
      Poisoned by user:        f7
      Container overflow:      fc
      Array cookie:            ac
      Intra object redzone:    bb
      ASan internal:           fe
      Left alloca redzone:     ca
      Right alloca redzone:    cb
      Shadow gap:              cc
    ==164==ABORTING

  [1]: https://clickhouse-test-reports.s3.yandex.net/26041/42a844546229e56c51a3ec986467ced52e4ed972/functional_stateless_tests_flaky_check_(address)/stderr.log

v2: Replace String with string_view in custom TLD for scalar
v3: use ColumnString::getDataAt()
 2021-07-07 10:38:45 +03:00
mergify[bot]
d9195c6f4a
Merge branch 'master' into support-distinct-on 2021-07-07 06:42:33 +00:00
Vladimir
b46ac3dfd1
Merge pull request #25609 from zxc111/master 2021-07-07 09:36:55 +03:00
alexey-milovidov
80eaf85301
Merge pull request #24416 from elevankoff/system-metrics 
Common system metrics collection
 2021-07-07 03:21:49 +03:00
Azat Khuzhin
42a8445462 Fix constness of custom TLDs 
Before this patch the functions below returns incorrect type for consts,
and hence optimize_skip_unused_shards does not work:

- cutToFirstSignificantSubdomainCustom()
- cutToFirstSignificantSubdomainCustomWithWWW()
- firstSignificantSubdomainCustom()
 2021-07-07 01:27:31 +03:00
alesapin
7b004fb828
Merge pull request #25995 from CurtizJ/fix-merge-tree-settings-change 
Fix alter of settings in `MergeTree` tables
 2021-07-06 22:12:38 +03:00
Kseniia Sumarokova
7a7b774ba3
Merge pull request #25550 from kssenii/truncate 
Allow truncate in storages s3/hdfs
 2021-07-06 22:07:53 +03:00
alesapin
3dee74df54 Comment 2021-07-06 19:53:54 +03:00
alesapin
a8fdc41193 Fix bug and add more trash to test 2021-07-06 19:51:23 +03:00
vdimir
b44bd174cc
Change error code for DISTINCT ON and LIMIT BY, finally 2021-07-06 19:14:22 +03:00
Nicolae Vartolomei
24f5ad8920 Subtract number of watch callbacks as this is what we actually count 2021-07-06 17:13:30 +01:00
Nicolae Vartolomei
9b3ceda57c Increment ZooKeeperWatch metric only if the callback is registered 2021-07-06 17:10:29 +01:00
vdimir
c6e13e6e2e
Add leading zeros in function bin 2021-07-06 15:19:42 +03:00
mergify[bot]
7ff8ac51a9
Merge branch 'master' into fix-merge-tree-settings-change 2021-07-06 12:14:33 +00:00
alesapin
75e26b93d0 Review bug fixes 2021-07-06 15:05:58 +03:00
dankondr
869a41ffb9 Add leftPadString() function 2021-07-06 14:40:34 +03:00
mergify[bot]
e9df8cc96c
Merge branch 'master' into support-distinct-on 2021-07-06 11:28:07 +00:00
alesapin
e9540f0621 Remove debug logs 2021-07-06 14:13:35 +03:00
mergify[bot]
8dd378f7c2
Merge branch 'master' into fix_drop_part_in_queue 2021-07-06 11:00:04 +00:00
alesapin
53b23775a9 Fix drop part 2021-07-06 13:58:53 +03:00
Maksim Kita
cd89138d3e FunctionsLogical const result for non const arguments fix 2021-07-06 13:49:16 +03:00
Maksim Kita
4812116929 FunctionSQLJSON ContextPtr build fix 2021-07-06 13:08:09 +03:00
vdimir
3cef256f79
Errorcode DISTINCT_ON_AND_LIMIT_BY_TOGETHER -> UNSUPPORTED_METHOD 2021-07-06 12:22:08 +03:00
vdimir
9665d41092
Force parentheses for DISTINCT ON 2021-07-06 12:17:26 +03:00
alesapin
2600922786 Followup fix 2021-07-06 11:36:39 +03:00
Maksim Kita
20b7fab7af
Merge pull request #26014 from ClickHouse/remove-old-code 
Remove old code
 2021-07-06 11:08:04 +03:00
Maksim Kita
2afdb744bb
Merge pull request #25989 from kitaisreal/compile-aggregate-functions-perf-tests-fix 
Compile aggregate functions perf tests fix
 2021-07-06 11:04:11 +03:00
Kseniia Sumarokova
d2141ff53b
Merge pull request #24148 from l1tsolaiki/jsonpath 
Jsonpath
 2021-07-06 10:43:29 +03:00
Alexey Milovidov
504d2c0c56 Remove old code 2021-07-06 05:31:09 +03:00
mergify[bot]
720660e9e9
Merge branch 'master' into network-instrumentation 2021-07-06 00:54:34 +00:00
alexey-milovidov
b724138541
Merge pull request #25786 from kssenii/inconsisteny 
Fix inconsistent behaviour of GROUP BY constant on empty set
 2021-07-06 03:36:50 +03:00
Alexey Milovidov
2e41419e3b Fix build 2021-07-06 03:34:27 +03:00
Alexey Milovidov
94a024c7c8 Fix warning 2021-07-06 01:12:49 +03:00
Alexey Milovidov
9875adc00b Make it even worse 2021-07-06 01:09:39 +03:00
l1tsolaiki
1bbfbff003 Regenerate ya.make 2021-07-06 01:04:09 +03:00
Alexey Milovidov
6d3d724cd4 Fix error 2021-07-05 23:20:25 +03:00
Alexey Milovidov
8fae774e85 Fix error 2021-07-05 23:18:13 +03:00
Alexey Milovidov
f29eb11e6c Fix error 2021-07-05 23:17:53 +03:00
Alexey Milovidov
21fb6ddea5 Remove AIO for sequential reads 2021-07-05 23:02:24 +03:00
alesapin
f7e1cfdb24 Some partially working code 2021-07-05 22:58:55 +03:00
Alexey Milovidov
945b54441d Comments 2021-07-05 22:41:50 +03:00
Alexey Milovidov
ac1baaf6d4 Comments 2021-07-05 22:37:12 +03:00
Alexey Milovidov
1fde0e13cc A check just in case 2021-07-05 22:32:33 +03:00
Alexey Milovidov
22ba93789b Fix warning 2021-07-05 22:29:36 +03:00
Alexey Milovidov
ce9c9c5e16 Merge remote-tracking branch 'origin/master' into system-metrics 2021-07-05 22:08:55 +03:00
Maksim Kita
f3d74c84fe
Merge pull request #25982 from CurtizJ/fix-map-json-format 
Fix formatting of type `Map` with integer keys to `JSON`
 2021-07-05 21:54:31 +03:00
Kseniia Sumarokova
8b075e34e7
Merge pull request #25984 from kssenii/add-drops-in-tests 
Correct test
 2021-07-05 18:35:53 +03:00
Anton Popov
9071ecd428 fix alter of settings in MergeTree 2021-07-05 15:44:58 +03:00
vdimir
f5b5f601f4
Merge branch 'master' into support-distinct-on 2021-07-05 15:44:20 +03:00
vdimir
dd06866fa8
Fix unbin for corner cases 2021-07-05 14:56:39 +03:00
vdimir
231740f2d6
Function bin for uint uses writeBinByte, correct for single zero 2021-07-05 14:44:50 +03:00
Kseniia Sumarokova
c72a1be0be
Merge pull request #25064 from ucasfl/merge 
Database argument for StorageMerge support regular expression.
 2021-07-05 14:14:44 +03:00
Anton Popov
237b834629 fix clang-tidy 2021-07-05 13:29:36 +03:00
vdimir
cc13787888
Minor style changes for (un)bin/hex 2021-07-05 13:22:44 +03:00
Maksim Kita
cef9baf597
Merge pull request #25974 from zxc111/issue-3446 
Add empty/notEmpty support UUID
 2021-07-05 12:21:36 +03:00
Maksim Kita
577e69898a Compile aggregate functions perf tests fix 2021-07-05 12:17:01 +03:00
l1tsolaiki
5b50fbde87 Also link jsonpath to functions 😨 2021-07-05 12:04:45 +03:00
l1tsolaiki
3b186d03f5 Change cmake lists 2021-07-05 11:56:09 +03:00
kssenii
c713ebe9c1 Add drops in tests 2021-07-05 11:20:58 +03:00
NengLiu
e36ce3e64e resolve conflicts 2021-07-05 13:57:49 +08:00
Kseniia Sumarokova
250f495456
Merge pull request #25975 from kssenii/follow-up 
Follow-up for #20470
 2021-07-05 06:36:43 +03:00
Alexey Milovidov
783ec5116a Skip errors 2021-07-05 05:55:11 +03:00
Alexey Milovidov
c3b975ecb1 Add network statistics 2021-07-05 05:47:33 +03:00
Alexey Milovidov
70bdc166e0 Add block device statistics 2021-07-05 05:24:36 +03:00
NengLiu
8b88988618 Merge branch 'master' into datatype-date32 
# Conflicts:
#	src/DataStreams/PostgreSQLBlockInputStream.cpp
 2021-07-05 10:18:38 +08:00
NengLiu
dc0c65ef4f Merge remote-tracking branch 'origin/master' into datatype-date32 2021-07-05 10:15:08 +08:00
Alexey Milovidov
dda4d0e34e Add EDAC 2021-07-05 04:18:12 +03:00
Alexey Milovidov
d7eca6ec35 Better temperatures 2021-07-05 03:56:14 +03:00
Alexey Milovidov
347ceac618 Better temperatures 2021-07-05 03:51:22 +03:00
Alexey Milovidov
5d5d2efd48 More temperatures 2021-07-05 03:40:28 +03:00
Anton Popov
139482b5c9 fix formatting to json of Map type with integer keys 2021-07-05 03:35:27 +03:00
Alexey Milovidov
d437ec2e1d Merge branch 'master' into system-metrics 2021-07-05 03:03:07 +03:00
Alexey Milovidov
b320905f34 Whitespaces 2021-07-05 03:02:47 +03:00
Alexey Milovidov
295afd29a1 Fix style 2021-07-05 03:02:23 +03:00
alexey-milovidov
02abbb549b
Merge pull request #25605 from azat/drop-race 
Fix logical race between DETACH and merges
 2021-07-05 03:00:14 +03:00
alexey-milovidov
fef7c557f6
Merge pull request #24789 from kitaisreal/compile-aggregation-functions 
Compile aggregate functions
 2021-07-05 02:57:49 +03:00
Alexey Milovidov
c059d0a0ee More metrics 2021-07-05 01:41:09 +03:00
Alexey Milovidov
0f8ea9b8f6 More metrics 2021-07-05 01:33:32 +03:00
Alexey Milovidov
b5840210c1 Adjustments 2021-07-05 00:54:46 +03:00
Alexey Milovidov
3a10d3802b Development 2021-07-05 00:33:00 +03:00
Alexey Milovidov
08aca329bd Development 2021-07-05 00:22:58 +03:00
Alexey Milovidov
c4675285bf Development 2021-07-04 23:49:36 +03:00
kssenii
32b7d7b750 More correct startup on create query 2021-07-04 19:53:05 +00:00
mergify[bot]
27d6a69f6a
Merge branch 'master' into truncate 2021-07-04 19:37:13 +00:00
l1tsolaiki
7bcb57afe1 Fix failing special builds (probably) 2021-07-04 22:10:01 +03:00
Maksim Kita
eca877ff39
Merge pull request #25972 from kssenii/mysql-protocol-fix 
mysql protocol: fix sql_select_limit
 2021-07-04 22:09:58 +03:00
Maksim Kita
62b88fda73
Merge pull request #25900 from kssenii/json-extract 
Support LowCardinality, Decimal and uuid for json extract
 2021-07-04 22:04:21 +03:00
Azat Khuzhin
7e17290acd Fix sharding_key from column w/o function for remote() 
P.S. that code looks redundant, and I'm not even sure that it was
required when it was first added in
325cc47ca5.
 2021-07-04 21:23:30 +03:00
Maksim Kita
df1dba5a7d Fixed Arcadia 2021-07-04 21:05:33 +03:00
feng lv
e73a908dd3 fix test 
fix style
 2021-07-04 16:45:04 +00:00
Kseniia Sumarokova
8351f1db99
Merge pull request #20470 from kssenii/pg-ch-replica 
MaterializePostgreSQL table engine and database engine
 2021-07-04 19:34:54 +03:00
zxc111
faac204a1f fix style 2021-07-05 00:01:49 +08:00
mergify[bot]
1559c09977
Merge branch 'master' into fix_drop_part_in_queue 2021-07-04 16:00:39 +00:00
alesapin
cf2fc94d9d
Merge pull request #25820 from ClickHouse/better_remove_empty_parts 
Replace lost parts with empty parts instead of hacking replication queue
 2021-07-04 18:55:49 +03:00
zxc111
15a36af4d6 Add empty/notEmpty support UUID #3446 2021-07-04 22:26:09 +08:00
kssenii
0dda5b67c4 Fix set SQL_SELECT_LIMIT 2021-07-04 16:17:59 +03:00
kssenii
c060963875 Better 2021-07-04 13:33:10 +03:00
l1tsolaiki
520c4a8f8a Fix according to review 2021-07-04 12:10:16 +03:00
kssenii
b39888f199 Do not allow inconsistency in publication 2021-07-04 08:54:41 +00:00
Kseniia Sumarokova
9b4d35ef95
Merge pull request #25940 from kssenii/move-to-prewhere-if-final 
Add setting optimize_move_to_prewhere_if_final
 2021-07-04 11:33:24 +03:00
neng.liu
b87814205d fix test error 2021-07-03 22:15:06 -07:00
Alexey Milovidov
935e0327a5 Development 2021-07-04 06:03:49 +03:00
Alexey Milovidov
0769366441 Remove part of trash 2021-07-04 04:19:19 +03:00
Alexey Milovidov
f6b5dbd666 Merge branch 'master' into system-metrics 2021-07-04 03:27:13 +03:00
Kseniia Sumarokova
705e839390
Merge branch 'master' into inconsisteny 2021-07-04 02:29:56 +03:00
mergify[bot]
6901fc7c7d
Merge branch 'master' into drop-race 2021-07-03 23:27:25 +00:00
kssenii
92eed10f10 Merge branch 'master' of github.com:ClickHouse/ClickHouse into json-extract 2021-07-04 02:24:22 +03:00
kssenii
e866faa975 Fix 2021-07-04 02:20:36 +03:00
alexey-milovidov
7054010cac
Update DataTypeMap.cpp 2021-07-04 02:04:43 +03:00
alexey-milovidov
916177a197
Merge branch 'master' into dev-map-bug 2021-07-04 02:03:43 +03:00
alexey-milovidov
07cc8a010f
Merge pull request #25941 from vdimir/joined-tables-minor-fix 
Build subquery in replaceJoinedTable without parsing
 2021-07-04 02:00:20 +03:00
kssenii
77879f6826 Better 2021-07-04 01:53:51 +03:00
Alexey Milovidov
c762e2247d More instrumentation for network interaction: add counters for recv/send bytes; add gauges for recvs/sends. 2021-07-04 00:00:50 +03:00
alexey-milovidov
56d695cd12
Update Settings.h 2021-07-03 22:10:22 +03:00
mergify[bot]
52366c4303
Merge branch 'master' into move-to-prewhere-if-final 2021-07-03 16:07:09 +00:00
Alexey Milovidov
127455f15f Remove experimental ANTLR parser 2021-07-03 18:48:03 +03:00
alexey-milovidov
51bf60a938
Merge pull request #17769 from xjewer/alex/remove_engine_setting 
CLICKHOUSE-784: reset merge tree setting
 2021-07-03 18:27:15 +03:00
alexey-milovidov
1dda771864
Merge pull request #25554 from CurtizJ/fix-alter-ttl-column 
Fix `ALTER MODIFY COLUMN` of columns, which participates in TTL expressions
 2021-07-03 18:26:37 +03:00
mergify[bot]
9da1c98998
Merge branch 'master' into fix_aggregation_ttl 2021-07-03 15:07:44 +00:00
alesapin
7b4a56977d
Merge pull request #25893 from ClickHouse/no_backoff_when_pool_is_full 
Reset counter if background pool is full
 2021-07-03 18:03:29 +03:00
mergify[bot]
fbe1ff0fb4
Merge branch 'master' into fix_drop_part_in_queue 2021-07-03 15:02:15 +00:00
mergify[bot]
a2183dff39
Merge branch 'master' into better_remove_empty_parts 2021-07-03 15:01:11 +00:00
Maksim Kita
55220d49f9 Fixed code review issues 2021-07-03 16:29:32 +03:00
vdimir
317ade9b4c
Build subquery in replaceJoinedTable without parsing 2021-07-03 16:24:41 +03:00
Kseniia Sumarokova
b639a30dfc
Merge pull request #25939 from kssenii/fix-special-build-check 
Fix clang tidy build check
 2021-07-03 15:14:16 +03:00
mergify[bot]
a318ed966a
Merge branch 'master' into inconsisteny 2021-07-03 11:58:09 +00:00
Maksim Kita
c584312c57
Merge pull request #25924 from ClickHouse/joins-idiotic 
Partially fix idiotic code in JOINs
 2021-07-03 14:55:26 +03:00
Maksim Kita
f238d98fd0
Merge pull request #25923 from ClickHouse/nested-chinese 
Add support for Chinese components in `Nested` data types
 2021-07-03 14:51:37 +03:00
kssenii
0ce4c565d1 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into pg-ch-replica 2021-07-03 11:48:15 +00:00
kssenii
a9a91a4f15 Setting move to prewhere if final 2021-07-03 11:37:50 +00:00
kssenii
ae1bb3f18b Fix clang-tidy 2021-07-03 13:16:55 +03:00
kssenii
02681019f8 Fix 2021-07-03 08:48:42 +00:00
kssenii
1776b06ef8 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into inconsisteny 2021-07-03 08:43:35 +00:00
alexey-milovidov
7e0433bf07
Merge pull request #25904 from xjewer/alex/add_index_first 
CLICKHOUSE-1194: add skipping index to the beginning of the list
 2021-07-03 04:55:00 +03:00
alexey-milovidov
ca063c3a70
Merge pull request #25913 from vitlibar/fix-crash-on-dictget-with-bad-argument 
Fix crash on call dictGet() with bad arguments.
 2021-07-03 04:53:23 +03:00
Alexey Milovidov
b20c0e2674 Partially fix idiotic code in JOINs 2021-07-03 03:23:14 +03:00
Alexey Milovidov
b26feb2b19 Add support for Chinese components in Nested data types 2021-07-03 03:13:21 +03:00
mergify[bot]
02cb287dc9
Merge branch 'master' into fix-alter-ttl-column 2021-07-02 23:49:46 +00:00
mergify[bot]
7aaed42035
Merge branch 'master' into jsonextractrawstring 2021-07-02 23:13:00 +00:00
alexey-milovidov
848fd6c734
Merge pull request #25922 from kitaisreal/clang-tidy-initialize-aggregation-function-build-fix 
FunctionInitializeAggregation build fix
 2021-07-03 01:23:19 +03:00
Maksim Kita
9f52e64805 FunctionInitializeAggregation build fix 2021-07-03 01:22:04 +03:00
Alexey Milovidov
132edc9e22 Allow quantiles* functions to work with aggregate_functions_null_for_empty 2021-07-02 22:47:43 +03:00
Kseniia Sumarokova
55ce7de248
Remove trailing spaces -- style check fix 2021-07-02 22:39:21 +03:00
Kseniia Sumarokova
fb6fc028bd
Update FunctionsJSON.h 2021-07-02 22:37:09 +03:00
zxc111
ace487278f refactory hex/unhex/bin/unbin 2021-07-03 02:08:17 +08:00
Maksim Kita
f16e8c3a4b
Merge pull request #25909 from ClickHouse/invalid-date 
Correctly throw exception on attempt to parse invalid Date
 2021-07-02 20:54:50 +03:00
alesapin
b4b54a0549 Fix bug 2021-07-02 18:25:05 +03:00
zxc111
12aea188b0 add bin/unbin support 2021-07-02 23:13:47 +08:00
alesapin
1072be7009 Merge branch 'better_remove_empty_parts' into fix_drop_part_in_queue 
the commit.
 2021-07-02 16:39:12 +03:00
alesapin
13c008c7a8 Change exception type 2021-07-02 16:38:46 +03:00
Maksim Kita
b3e3a3cde0 Fixed tests 2021-07-02 16:36:02 +03:00
feng lv
b0dffa3f3e Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into merge 2021-07-02 13:19:38 +00:00
Vitaly Baranov
8b4fabe60c Fix crash on call dictGet() with bad arguments. 2021-07-02 14:46:12 +03:00
alesapin
72b84141e1 Merge branch 'better_remove_empty_parts' into fix_drop_part_in_queue 2021-07-02 12:41:28 +03:00
alesapin
f8f734d3dc Better comment 2021-07-02 12:40:13 +03:00
Maksim Kita
aec313cf4f
Merge branch 'master' into null-valid-identifier 2021-07-02 12:33:04 +03:00
Maksim Kita
9169a4cbb0
Merge pull request #25910 from ClickHouse/aggregate-function-constexpr-parameters 
Allow constexpr parameters for aggregate functions
 2021-07-02 12:32:24 +03:00
alesapin
abc3aca19b Merge branch 'better_remove_empty_parts' into fix_drop_part_in_queue 2021-07-02 12:30:54 +03:00
alesapin
2e29dc2975 More safe empty parts creation 2021-07-02 12:30:17 +03:00
alesapin
5e79c68e8e Debug 2021-07-02 11:54:22 +03:00
Maksim Kita
3ae1278391 Fixed tests 2021-07-02 11:22:30 +03:00
feng lv
1bf7a175b7 fix style 2021-07-02 03:44:42 +00:00
feng lv
e25bf4de6d Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into merge 2021-07-02 03:19:39 +00:00
feng lv
3789e23dc9 fix 
update

fix
 2021-07-02 02:50:55 +00:00
Alexey Milovidov
0e621788c7 Allow constexpr parameters for aggregate functions 2021-07-02 03:53:08 +03:00
Alexey Milovidov
b33d91412d Correctly throw exception on invalid dates 2021-07-02 03:07:23 +03:00
Alexey Milovidov
2866d45681 Add support for queries with null quoted identifier and ON CLUSTER 2021-07-02 02:01:13 +03:00
mergify[bot]
892c67891a
Merge branch 'master' into fix-receive-time-metric 2021-07-01 21:40:09 +00:00
Maksim Kita
97e1ddbe94 Fix before merge 2021-07-01 22:56:36 +03:00
Maksim Kita
4bd398c0e5 Fixed tests 2021-07-01 22:56:36 +03:00
Maksim Kita
39ca0ceb81 Fixed tests 2021-07-01 22:56:36 +03:00
Maksim Kita
31fe3dad7a Fixed tests 2021-07-01 22:56:36 +03:00
Maksim Kita
f33a38381d Fixed tests 2021-07-01 22:56:36 +03:00
Maksim Kita
325b54f623 Aggregator compile only part of aggregate functions 2021-07-01 22:56:36 +03:00
Maksim Kita
92a36e38f1 AggregateFunctionAvgWeighted fix accuracy 2021-07-01 22:56:36 +03:00
Maksim Kita
22491c8adc Compile AggregateFunctionCount 2021-07-01 22:56:36 +03:00
Maksim Kita
fdf172a61e Fixed tests 2021-07-01 22:56:36 +03:00
Maksim Kita
61a5c4f493 Fix min, max aggregate functions merge 2021-07-01 22:56:36 +03:00
Maksim Kita
30021f0335 Fixed tests 2021-07-01 22:56:36 +03:00
Maksim Kita
da8c957167 Aggregator added CompiledExpressionCache 2021-07-01 22:56:36 +03:00
Maksim Kita
f17e212a72 Fixed build 2021-07-01 22:56:36 +03:00
Maksim Kita
7b3952adda Fixed tests 2021-07-01 22:56:36 +03:00
Maksim Kita
0fb8ea530f Avoid using exception unsafe method 2021-07-01 22:56:36 +03:00
Maksim Kita
d24d3ae992 Added second variant of compilation 2021-07-01 22:56:36 +03:00
Maksim Kita
1e2f22a183 Aggregator compile part of aggregate functions 2021-07-01 22:56:36 +03:00
Maksim Kita
eb29490102 Fix AggregateFunctionAvg denominator type 2021-07-01 22:56:36 +03:00
Maksim Kita
d4742f91e6 Check min_count_to_compile_aggregate_expression setting before compilation 2021-07-01 22:56:36 +03:00
Maksim Kita
e67198c144 Compile AggregateFunctionIfNull 2021-07-01 22:56:36 +03:00
Maksim Kita
507d9405e2 Compile AggregateFunctionAvgWeighted 2021-07-01 22:56:36 +03:00
Maksim Kita
56c1a4e447 Added tests 2021-07-01 22:56:36 +03:00
Maksim Kita
a7a7623b36 Fixed style check 2021-07-01 22:56:36 +03:00
Maksim Kita
44259736ed Compile AggregateFunctionMax 2021-07-01 22:56:36 +03:00
Maksim Kita
ddc96374c8 Compile AggregateFunctionMin 2021-07-01 22:56:35 +03:00
Maksim Kita
88d536ea5c Update compilation for AggregateFunctionAny, AggregateFunctionAnyLast 2021-07-01 22:56:35 +03:00
Maksim Kita
fdfb17370d Compile AggregateFunctionAnyLast 2021-07-01 22:56:35 +03:00
Maksim Kita
905f48ccce Compile AggregateFunctionAny 2021-07-01 22:56:35 +03:00
Maksim Kita
f93da5ed2b Compile AggregateFunctionNullUnary 2021-07-01 22:56:35 +03:00