thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-15 10:52:30 +00:00
Code Issues Packages Projects Releases Wiki Activity
99,451 Commits 9,328 Branches 1,783 Tags 2.1 GiB
5f04304278
Commit Graph

7676 Commits

Author SHA1 Message Date
Alexey Milovidov
9c5f9f1815
Merge pull request #41158 from den-crane/bug/trivial_count_optimization_with_array_join 
Fix: trivial count optimization with array join. revert revert #39466
 2022-09-12 08:56:10 +03:00
Alexey Milovidov
94d6e9ce7d
Merge pull request #41189 from kssenii/fix-positional-args 
Fix for positional arguments
 2022-09-12 03:21:39 +03:00
Alexey Milovidov
8b5328f7e5 Merge branch 'master' into fix-trash-base58 2022-09-11 20:43:54 +02:00
Alexey Milovidov
3e6c4e3f16 Fix clang-tidy 2022-09-11 20:43:36 +02:00
Denny Crane
0e4e868f29
Merge branch 'master' into bug/trivial_count_optimization_with_array_join 2022-09-11 15:23:10 -03:00
kssenii
9c2bbcf4a6 Fix 2022-09-11 17:38:51 +02:00
kssenii
2f3bfc5411 Fix 2022-09-11 13:54:25 +02:00
kssenii
b7d751b782 Merge remote-tracking branch 'upstream/master' into improve-marks-cache-loading 2022-09-11 13:23:30 +02:00
kssenii
37bdba98b1 Fix 2022-09-11 12:58:04 +02:00
Alexey Milovidov
7f1e7b5967 Merge branch 'master' into fix-half-of-trash 2022-09-11 06:20:47 +02:00
Azat Khuzhin
ba4174402a Temporary disable complation of compileAddIntoAggregateStatesFunctions 
Fixes the following use-of-uninitialized-value in llvm [1]:

    ==696==WARNING: MemorySanitizer: use-of-uninitialized-value
        0 0x498141d9 in llvm::ilist_traits<llvm::MachineInstr>::removeNodeFromList(llvm::MachineInstr*) build_docker/../contrib/llvm/llvm/lib/CodeGen/MachineBasicBlock.cpp:155:24
        1 0x498141d9 in llvm::iplist_impl<llvm::simple_ilist<llvm::MachineInstr, llvm::ilist_sentinel_tracking<true>>, llvm::ilist_traits<llvm::MachineInstr>>::remove(llvm::ilist_iterator<llvm::ilist_detail::node_options<llvm::MachineInstr, true, true, void>, false, false>&) build_docker/../contrib/llvm/llvm/include/llvm/ADT/ilist.h:253:11
        2 0x498141d9 in llvm::iplist_impl<llvm::simple_ilist<llvm::MachineInstr, llvm::ilist_sentinel_tracking<true>>, llvm::ilist_traits<llvm::MachineInstr>>::erase(llvm::ilist_iterator<llvm::ilist_detail::node_options<llvm::MachineInstr, true, true, void>, false, false>) build_docker/../contrib/llvm/llvm/include/llvm/ADT/ilist.h:268:22
        3 0x498141d9 in llvm::iplist_impl<llvm::simple_ilist<llvm::MachineInstr, llvm::ilist_sentinel_tracking<true>>, llvm::ilist_traits<llvm::MachineInstr>>::erase(llvm::ilist_iterator<llvm::ilist_detail::node_options<llvm::MachineInstr, true, true, void>, false, false>, llvm::ilist_iterator<llvm::ilist_detail::node_options<llvm::MachineInstr, true, true, void>, false, false>) build_docker/../contrib/llvm/llvm/include/llvm/ADT/ilist.h:305:15
        4 0x498141d9 in llvm::iplist_impl<llvm::simple_ilist<llvm::MachineInstr, llvm::ilist_sentinel_tracking<true>>, llvm::ilist_traits<llvm::MachineInstr>>::clear() build_docker/../contrib/llvm/llvm/include/llvm/ADT/ilist.h:309:18
        5 0x498141d9 in llvm::iplist_impl<llvm::simple_ilist<llvm::MachineInstr, llvm::ilist_sentinel_tracking<true>>, llvm::ilist_traits<llvm::MachineInstr>>::~iplist_impl() build_docker/../contrib/llvm/llvm/include/llvm/ADT/ilist.h:210:20
        6 0x498145ff in llvm::MachineBasicBlock::~MachineBasicBlock() build_docker/../contrib/llvm/llvm/lib/CodeGen/MachineBasicBlock.cpp:56:1
        7 0x49969c1f in llvm::MachineFunction::DeleteMachineBasicBlock(llvm::MachineBasicBlock*) build_docker/../contrib/llvm/llvm/lib/CodeGen/MachineFunction.cpp:426:8
        8 0x49969c1f in llvm::ilist_alloc_traits<llvm::MachineBasicBlock>::deleteNode(llvm::MachineBasicBlock*) build_docker/../contrib/llvm/llvm/lib/CodeGen/MachineFunction.cpp:127:21
        9 0x4983f4d6 in llvm::iplist_impl<llvm::simple_ilist<llvm::MachineBasicBlock>, llvm::ilist_traits<llvm::MachineBasicBlock>>::erase(llvm::ilist_iterator<llvm::ilist_detail::node_options<llvm::MachineBasicBlock, false, false, void>, false, false>) build_docker/../contrib/llvm/llvm/include/llvm/ADT/ilist.h:268:11
        10 0x4983f4d6 in llvm::iplist_impl<llvm::simple_ilist<llvm::MachineBasicBlock>, llvm::ilist_traits<llvm::MachineBasicBlock>>::erase(llvm::MachineBasicBlock*) build_docker/../contrib/llvm/llvm/include/llvm/ADT/ilist.h:272:39
        11 0x4983f4d6 in llvm::MachineFunction::erase(llvm::MachineBasicBlock*) build_docker/../contrib/llvm/llvm/include/llvm/CodeGen/MachineFunction.h:767:53
        12 0x4983f4d6 in llvm::MachineBasicBlock::eraseFromParent() build_docker/../contrib/llvm/llvm/lib/CodeGen/MachineBasicBlock.cpp:1317:16
        13 0x4a0c9a27 in llvm::TailDuplicator::removeDeadBlock(llvm::MachineBasicBlock*, llvm::function_ref<void (llvm::MachineBasicBlock*)>*) build_docker/../contrib/llvm/llvm/lib/CodeGen/TailDuplicator.cpp:1051:8
        14 0x4a0c1e41 in llvm::TailDuplicator::tailDuplicateAndUpdate(bool, llvm::MachineBasicBlock*, llvm::MachineBasicBlock*, llvm::SmallVectorImpl<llvm::MachineBasicBlock*>*, llvm::function_ref<void (llvm::MachineBasicBlock*)>*, llvm::SmallVectorImpl<llvm::MachineBasicBlock*>*) build_docker/../contrib/llvm/llvm/lib/CodeGen/TailDuplicator.cpp:189:5
        15 0x4a0ca16e in llvm::TailDuplicator::tailDuplicateBlocks() build_docker/../contrib/llvm/llvm/lib/CodeGen/TailDuplicator.cpp:288:19
        16 0x4a0be9f9 in (anonymous namespace)::TailDuplicateBase::runOnMachineFunction(llvm::MachineFunction&) build_docker/../contrib/llvm/llvm/lib/CodeGen/TailDuplication.cpp:98:21
        17 0x499a2777 in llvm::MachineFunctionPass::runOnFunction(llvm::Function&) build_docker/../contrib/llvm/llvm/lib/CodeGen/MachineFunctionPass.cpp:72:13
        18 0x4dbba34d in llvm::FPPassManager::runOnFunction(llvm::Function&) build_docker/../contrib/llvm/llvm/lib/IR/LegacyPassManager.cpp:1435:27
        19 0x4dbe3761 in llvm::FPPassManager::runOnModule(llvm::Module&) build_docker/../contrib/llvm/llvm/lib/IR/LegacyPassManager.cpp:1481:16
        20 0x4dbbebbb in (anonymous namespace)::MPPassManager::runOnModule(llvm::Module&) build_docker/../contrib/llvm/llvm/lib/IR/LegacyPassManager.cpp:1550:27
        21 0x4dbbebbb in llvm::legacy::PassManagerImpl::run(llvm::Module&) build_docker/../contrib/llvm/llvm/lib/IR/LegacyPassManager.cpp:541:44
        22 0x4dbe454b in llvm::legacy::PassManager::run(llvm::Module&) build_docker/../contrib/llvm/llvm/lib/IR/LegacyPassManager.cpp:1677:14
        23 0x414405df in DB::JITCompiler::compile(llvm::Module&) build_docker/../src/Interpreters/JIT/CHJIT.cpp:78:22
        24 0x4143bb7d in DB::CHJIT::compileModule(std::__1::unique_ptr<llvm::Module, std::__1::default_delete<llvm::Module>>) build_docker/../src/Interpreters/JIT/CHJIT.cpp:378:29
        25 0x4143aded in DB::CHJIT::compileModule(std::__1::function<void (llvm::Module&)>) build_docker/../src/Interpreters/JIT/CHJIT.cpp:359:24
        26 0x4147b25e in DB::compileAggregateFunctions(DB::CHJIT&, std::__1::vector<DB::AggregateFunctionWithOffset, std::__1::allocator<DB::AggregateFunctionWithOffset>> const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>) build_docker/../src/Interpreters/JIT/compileFunction.cpp:738:32
        27 0x3de0a23a in DB::Aggregator::compileAggregateFunctionsIfNeeded()::$_0::operator()() const build_docker/../src/Interpreters/Aggregator.cpp:661:49
        28 0x3de0a23a in std::__1::pair<std::__1::shared_ptr<DB::CompiledExpressionCacheEntry>, bool> DB::CacheBase<wide::integer<128ul, unsigned int>, DB::CompiledExpressionCacheEntry, UInt128Hash, DB::CompiledFunctionWeightFunction>::getOrSet<DB::Aggregator::compileAggregateFunctionsIfNeeded()::$_0>(wide::integer<128ul, unsigned int> const&, DB::Aggregator::compileAggregateFunctionsIfNeeded()::$_0&&) build_docker/../src/Common/CacheBase.h:125:24
        29 0x3de0a23a in DB::Aggregator::compileAggregateFunctionsIfNeeded() build_docker/../src/Interpreters/Aggregator.cpp:657:70

      Memory was marked as uninitialized
        0 0xb988ded in __sanitizer_dtor_callback (/usr/bin/clickhouse+0xb988ded) (BuildId: c4a880b742797a1f37bc4f5ed869f055cc86486b)
        1 0x498145da in llvm::MachineBasicBlock::~MachineBasicBlock() build_docker/../contrib/llvm/llvm/lib/CodeGen/MachineBasicBlock.cpp:56:1
        2 0x49969c1f in llvm::MachineFunction::DeleteMachineBasicBlock(llvm::MachineBasicBlock*) build_docker/../contrib/llvm/llvm/lib/CodeGen/MachineFunction.cpp:426:8
        3 0x49969c1f in llvm::ilist_alloc_traits<llvm::MachineBasicBlock>::deleteNode(llvm::MachineBasicBlock*) build_docker/../contrib/llvm/llvm/lib/CodeGen/MachineFunction.cpp:127:21
        4 0x4983f4d6 in llvm::iplist_impl<llvm::simple_ilist<llvm::MachineBasicBlock>, llvm::ilist_traits<llvm::MachineBasicBlock>>::erase(llvm::ilist_iterator<llvm::ilist_detail::node_options<llvm::MachineBasicBlock, false, false, void>, false, false>) build_docker/../contrib/llvm/llvm/include/llvm/ADT/ilist.h:268:11
        5 0x4983f4d6 in llvm::iplist_impl<llvm::simple_ilist<llvm::MachineBasicBlock>, llvm::ilist_traits<llvm::MachineBasicBlock>>::erase(llvm::MachineBasicBlock*) build_docker/../contrib/llvm/llvm/include/llvm/ADT/ilist.h:272:39
        6 0x4983f4d6 in llvm::MachineFunction::erase(llvm::MachineBasicBlock*) build_docker/../contrib/llvm/llvm/include/llvm/CodeGen/MachineFunction.h:767:53
        7 0x4983f4d6 in llvm::MachineBasicBlock::eraseFromParent() build_docker/../contrib/llvm/llvm/lib/CodeGen/MachineBasicBlock.cpp:1317:16
        8 0x4a0c9a27 in llvm::TailDuplicator::removeDeadBlock(llvm::MachineBasicBlock*, llvm::function_ref<void (llvm::MachineBasicBlock*)>*) build_docker/../contrib/llvm/llvm/lib/CodeGen/TailDuplicator.cpp:1051:8
        9 0x4a0c1e41 in llvm::TailDuplicator::tailDuplicateAndUpdate(bool, llvm::MachineBasicBlock*, llvm::MachineBasicBlock*, llvm::SmallVectorImpl<llvm::MachineBasicBlock*>*, llvm::function_ref<void (llvm::MachineBasicBlock*)>*, llvm::SmallVectorImpl<llvm::MachineBasicBlock*>*) build_docker/../contrib/llvm/llvm/lib/CodeGen/TailDuplicator.cpp:189:5
        10 0x4a0ca16e in llvm::TailDuplicator::tailDuplicateBlocks() build_docker/../contrib/llvm/llvm/lib/CodeGen/TailDuplicator.cpp:288:19
        11 0x4a0be9f9 in (anonymous namespace)::TailDuplicateBase::runOnMachineFunction(llvm::MachineFunction&) build_docker/../contrib/llvm/llvm/lib/CodeGen/TailDuplication.cpp:98:21
        12 0x499a2777 in llvm::MachineFunctionPass::runOnFunction(llvm::Function&) build_docker/../contrib/llvm/llvm/lib/CodeGen/MachineFunctionPass.cpp:72:13
        13 0x4dbba34d in llvm::FPPassManager::runOnFunction(llvm::Function&) build_docker/../contrib/llvm/llvm/lib/IR/LegacyPassManager.cpp:1435:27
        14 0x4dbe3761 in llvm::FPPassManager::runOnModule(llvm::Module&) build_docker/../contrib/llvm/llvm/lib/IR/LegacyPassManager.cpp:1481:16
        15 0x4dbbebbb in (anonymous namespace)::MPPassManager::runOnModule(llvm::Module&) build_docker/../contrib/llvm/llvm/lib/IR/LegacyPassManager.cpp:1550:27
        16 0x4dbbebbb in llvm::legacy::PassManagerImpl::run(llvm::Module&) build_docker/../contrib/llvm/llvm/lib/IR/LegacyPassManager.cpp:541:44
        17 0x4dbe454b in llvm::legacy::PassManager::run(llvm::Module&) build_docker/../contrib/llvm/llvm/lib/IR/LegacyPassManager.cpp:1677:14
        18 0x414405df in DB::JITCompiler::compile(llvm::Module&) build_docker/../src/Interpreters/JIT/CHJIT.cpp:78:22
        19 0x4143bb7d in DB::CHJIT::compileModule(std::__1::unique_ptr<llvm::Module, std::__1::default_delete<llvm::Module>>) build_docker/../src/Interpreters/JIT/CHJIT.cpp:378:29
        20 0x4143aded in DB::CHJIT::compileModule(std::__1::function<void (llvm::Module&)>) build_docker/../src/Interpreters/JIT/CHJIT.cpp:359:24
        21 0x4147b25e in DB::compileAggregateFunctions(DB::CHJIT&, std::__1::vector<DB::AggregateFunctionWithOffset, std::__1::allocator<DB::AggregateFunctionWithOffset>> const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>) build_docker/../src/Interpreters/JIT/compileFunction.cpp:738:32
        22 0x3de0a23a in DB::Aggregator::compileAggregateFunctionsIfNeeded()::$_0::operator()() const build_docker/../src/Interpreters/Aggregator.cpp:661:49
        23 0x3de0a23a in std::__1::pair<std::__1::shared_ptr<DB::CompiledExpressionCacheEntry>, bool> DB::CacheBase<wide::integer<128ul, unsigned int>, DB::CompiledExpressionCacheEntry, UInt128Hash, DB::CompiledFunctionWeightFunction>::getOrSet<DB::Aggregator::compileAggregateFunctionsIfNeeded()::$_0>(wide::integer<128ul, unsigned int> const&, DB::Aggregator::compileAggregateFunctionsIfNeeded()::$_0&&) build_docker/../src/Common/CacheBase.h:125:24
        24 0x3de0a23a in DB::Aggregator::compileAggregateFunctionsIfNeeded() build_docker/../src/Interpreters/Aggregator.cpp:657:70

  [1]: https://s3.amazonaws.com/clickhouse-test-reports/41046/490a2c75610c4bc3191d55226f8454b3c3d3919a/stateful_tests__msan_.html

Note, that it is safe to do, but only for this method, since it had been
disabled anyway, back in #27574, and I guess this MSan report may be
related.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-09-10 21:38:35 +02:00
Alexey Milovidov
fd235919aa Remove some methods 2022-09-10 05:04:40 +02:00
Alexey Milovidov
fa62c7e982 Fix half of trash 2022-09-10 04:08:16 +02:00
Igor Nikonov
726639484a
Revert "Query plan optimization setting: read in window order" 2022-09-09 22:15:38 +02:00
Denny Crane
be90cecbd2 Revert "Revert "Fix trivial count optimization with array join"" 
This reverts commit 17de7b2876.
 2022-09-09 14:56:38 -03:00
Azat Khuzhin
763bb18f98 Fix SIGSEGV in SortedBlocksWriter in case of empty block 
CI found one issue [1].

Here is the stack trace for invalid read:

<details>

<summary>stack trace</summary>

```
    0: DB::TemporaryFileLazySource::TemporaryFileLazySource(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, DB::Block const&) [inlined] std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >::__is_long(this="") const at string:1445:22
    1: DB::TemporaryFileLazySource::TemporaryFileLazySource(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, DB::Block const&) [inlined] std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >::basic_string(this="", __str="") at string:1927
    2: DB::TemporaryFileLazySource::TemporaryFileLazySource(this=0x00007f3aec105f58, path_="", header_=0x00007f38ffd93b40) at TemporaryFileLazySource.cpp:11
    3: DB::SortedBlocksWriter::streamFromFile(std::__1::unique_ptr<Poco::TemporaryFile, std::__1::default_delete<Poco::TemporaryFile> > const&) const [inlined] DB::TemporaryFileLazySource* std::__1::construct_at<DB::TemporaryFileLazySource, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, DB::Block, DB::TemporaryFileLazySource*>(__args=0x00007f38ffd91560) at construct_at.h:38:50
    4: DB::SortedBlocksWriter::streamFromFile(std::__1::unique_ptr<Poco::TemporaryFile, std::__1::default_delete<Poco::TemporaryFile> > const&) const [inlined] void std::__1::allocator_traits<std::__1::allocator<DB::TemporaryFileLazySource> >::construct<DB::TemporaryFileLazySource, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, DB::Block, void, void>(__args=0x00007f38ffd91560) at allocator_traits.h:298
    5: DB::SortedBlocksWriter::streamFromFile(std::__1::unique_ptr<Poco::TemporaryFile, std::__1::default_delete<Poco::TemporaryFile> > const&) const [inlined] std::__1::__shared_ptr_emplace<DB::TemporaryFileLazySource, std::__1::allocator<DB::TemporaryFileLazySource> >::__shared_ptr_emplace<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, DB::Block>(this=0x00007f3aec105f40, __args=0x00007f38ffd91560) at shared_ptr.h:293
    6: DB::SortedBlocksWriter::streamFromFile(std::__1::unique_ptr<Poco::TemporaryFile, std::__1::default_delete<Poco::TemporaryFile> > const&) const [inlined] std::__1::shared_ptr<DB::TemporaryFileLazySource> std::__1::allocate_shared<DB::TemporaryFileLazySource, std::__1::allocator<DB::TemporaryFileLazySource>, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, DB::Block, void>(__args=<unavailable>, __args=<unavailable>) at shared_ptr.h:954
    7: DB::SortedBlocksWriter::streamFromFile(std::__1::unique_ptr<Poco::TemporaryFile, std::__1::default_delete<Poco::TemporaryFile> > const&) const [inlined] std::__1::shared_ptr<DB::TemporaryFileLazySource> std::__1::make_shared<DB::TemporaryFileLazySource, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, DB::Block, void>(__args=<unavailable>, __args=<unavailable>) at shared_ptr.h:963
    8: DB::SortedBlocksWriter::streamFromFile(this=<unavailable>, file=<unavailable>) const at SortedBlocksWriter.cpp:238
    9: DB::SortedBlocksWriter::premerge(this=<unavailable>) at SortedBlocksWriter.cpp:209:32
```

</details>

  [1]: https://s3.amazonaws.com/clickhouse-test-reports/41046/adea92f847373d1fcfd733d8979c63024f9b80bf/stress_test__asan_.html

So the problem here is that there was empty unique_ptr<> reference to
temporary file, because of empty block that accepted by
SortedBlocksWriter::insert(), but insert() is not a problem the problem
is premerge() that steals blocks from insert() and do not have check
that there are some rows. However this check exists in
SortedBlocksWriter::flush(), and in that case temporary file is not
created.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-09-09 19:52:38 +02:00
Azat Khuzhin
57146c9361 Fix typos in SortedBlocksWriter 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-09-09 19:52:38 +02:00
Anton Popov
a1b5582a7c
Merge pull request #37152 from CurtizJ/fix-filling-of-missed-nested 
Fix filling of missed Nested columns with multiple levels
 2022-09-09 19:29:01 +02:00
Alexander Tokmakov
9bcec2d49a stop background ops before converting db engine 2022-09-09 18:27:19 +02:00
zhenjial
51badf3d0a fix 2022-09-10 00:08:50 +08:00
Dmitry Novik
64c4023737
Merge branch 'master' into grouping-comp 2022-09-09 18:05:01 +02:00
zhenjial
bd9fabc3f7 code optimization, add test 2022-09-09 23:27:42 +08:00
avogar
ad68b7be0f Better 2022-09-09 15:01:45 +00:00
Kseniia Sumarokova
5bb40e5856
Change cache setting do_not_evict_index_and_mark_files default to 0. (#41139) 2022-09-09 15:04:32 +02:00
Antonio Andelic
32483aeec8 Merge branch 'master' into keeper-storage 2022-09-09 12:24:01 +00:00
kssenii
5cba9b789c Merge remote-tracking branch 'upstream/master' into refactor-cache-for-async-download 2022-09-09 14:14:04 +02:00
Igor Nikonov
5379f37331
Merge pull request #41015 from ClickHouse/window_func_optimization_setting 
Query plan optimization setting: read in window order
 2022-09-09 14:00:41 +02:00
Raúl Marín
b33c1fa60f Fix query_views_log with Window views 2022-09-09 13:46:53 +02:00
kssenii
6a885430d7 Add assertion 2022-09-09 13:36:23 +02:00
kssenii
c2c35278ba Refactor cache for async download 2022-09-09 13:12:47 +02:00
lthaooo
d4dd6b05b2 optimize compact part mutation 2022-09-09 10:22:35 +08:00
zhenjial
469ceaa156 code optimization 2022-09-09 00:47:43 +08:00
Kruglov Pavel
93d38b9451
Fix style 2022-09-08 13:19:18 +02:00
Robert Schulze
c16707ff00
chore: delete obsolete modelEvaluate() function + SYSTEM.MODELS view 
- The deleted function modelEvaluate() was superseded by
  catboostEvaluate().

- Also delete the external model repository, as modelEvaluate() was it's
  last user. Additionally remove the system view SYSTEM.MODELS for
  inspecting the repository.

- SYSTEM RELOAD MODELS is also obsolete. HOWEVER, it was retained and
  made a no-op instead of deleted.

  Why?
  The reason is that RBAC in distributed setups works by storing
  privileges (granted and revoked) as plain SQL statements in Keeper.
  Nodes read these statements at startup and parse them. If a privilege
  for SYSTEM RELOAD MODELS exists but parser doesn't recognize it
  nodes would fail to come up.

  Considered but rejected alternatives:
  - Ignore SYSTEM RELOAD MODELS during parsing RBAC privileges and
    return an error for regular SYSTEM RELOAD MODELS SQL. Special-case
    of no-op behavior, too brittle.
  - Remove SYSTEM RELOAD MODELS manually from Keeper via command-line
    manipulation of Keeper nodes or via SQL by dropping the privileges.
    Needs user intervention during upgrade.
 2022-09-08 09:10:11 +00:00
Anton Popov
7c12b448b8 Merge remote-tracking branch 'upstream/master' into HEAD 2022-09-08 01:52:52 +00:00
Anton Popov
5a790b15b4 try to fix filling of missed Nested columns with multiple levels 2022-09-08 00:20:11 +00:00
Igor Nikonov
e74bb00e90 Fix: EXPLAIN PLAN - make settings in SETTING clause effective 2022-09-07 19:11:25 +00:00
avogar
0475eb0b17 Make better 2022-09-07 18:53:22 +00:00
avogar
5c0c19c533 Make generic solution 2022-09-07 18:49:34 +00:00
kssenii
e51313b6b3 Get rid of static threadpools 2022-09-07 17:48:11 +02:00
avogar
9da22f7b7c Fix possible use-heap-after-free in -Map combinator 2022-09-07 15:06:56 +00:00
Antonio Andelic
d15fdae8dc
Merge pull request #40853 from ClickHouse/embeddedrocksdb-delete-update-support 
Support for `UPDATE` and `DELETE` in `EmbeddedRocksDB`
 2022-09-07 11:03:49 +02:00
kssenii
487bc0fba3 Fix heap use after free 2022-09-06 16:50:58 +02:00
Anton Popov
f76c1482bd try to fix filling of missed Nested columns with multiple levels 2022-09-06 13:56:32 +00:00
Filatenkov Artur
7949b3fb74
Merge branch 'master' into annoy 2022-09-06 11:23:15 +03:00
Yong Wang
d5337fd813
Merge branch 'master' into Kusto-phase1 2022-09-06 00:51:39 -04:00
Alexey Milovidov
940a53e519
Merge pull request #40984 from Lucky-Chang/typo_fix 
Fix some typos and clang-tidy warnings
 2022-09-06 02:37:50 +03:00
Yong Wang
2ca54d12ee
Merge branch 'ClickHouse:master' into Kusto-phase1 2022-09-05 17:42:38 -04:00
kssenii
4c8aa04b73 Revert "Fix" 
This reverts commit ec55c90f36.
 2022-09-05 20:12:40 +02:00
ianton-ru
39e1fc7a0f
Merge branch 'master' into fix-ordinary-system-unfreeze 2022-09-05 17:10:59 +03:00
FArthur-cmd
26eb6cdaec refactoring 2022-09-05 13:04:42 +00:00
FArthur-cmd
9096b73084 add setting 2022-09-05 12:41:46 +00:00
Nikita Mikhaylov
33e530392e
Check grants for databases in case of renaming databases (#40906) 2022-09-05 12:38:44 +02:00
Antonio Andelic
e07539d1b9 Merge branch 'master' into keeper-storage 2022-09-05 09:54:13 +00:00
Luck-Chang
1ac8e739c9 fix some typos and clang-tidy warnings 2022-09-05 09:50:24 +08:00
Nikita Taranov
16af4aebc8
impl (#40952) 2022-09-04 16:26:31 +02:00
Alexey Milovidov
193cd1b3b2
Merge pull request #39138 from nickitat/control_block_size_in_aggregator 
Control block size in aggregator
 2022-09-04 04:51:00 +03:00
Yakov Olkhovskiy
fdcced8962
Merge pull request #40850 from canhld94/ch_canh_fix_set 
Fix tryGetPreparedSet crash when same set expression built from different column(s)
 2022-09-02 14:40:35 -04:00
kssenii
2dcf3fef94 Fix tests 2022-09-02 19:27:52 +02:00
Kseniia Sumarokova
7dea71c83f
Merge pull request #40884 from TKaxe/fix_removeUnneededColumn 
fix bug when remove unneeded columns in subquery
 2022-09-02 18:51:18 +02:00
Kseniia Sumarokova
1370bf0de7
Merge pull request #40820 from kssenii/cache-assert-directory-ownership 
Create status file for cache directory
 2022-09-02 18:42:11 +02:00
Anton Popov
9dda9658a8 Merge remote-tracking branch 'upstream/master' into HEAD 2022-09-02 12:48:27 +00:00
Vladimir C
963c0111bf
Merge pull request #39418 from vdimir/join_and_sets 
Filter joined streams for `full_sorting_join` by each other before sorting
 2022-09-02 13:57:06 +02:00
Antonio Andelic
9ca20b97c2
Merge branch 'master' into keeper-storage 2022-09-02 12:13:07 +02:00
Antonio Andelic
e64436fef3 Fix typos with new codespell 2022-09-02 08:54:48 +00:00
Robert Schulze
319d8b00a7
Merge pull request #39010 from FrankChen021/tracing_context_propagation 
Improve the opentelemetry tracing context propagation across threads
 2022-09-02 07:56:43 +02:00
Duc Canh Le
65b64010a0 move out-of-bound check into set::areTypesEqual 2022-09-02 11:56:13 +08:00
Dmitry Novik
ddadb362cf
Merge pull request #39762 from quickhouse/betterorderbyoptimization 
Fixed `Unknown identifier (aggregate-function)` exception which appears when a user tries to calculate WINDOW ORDER BY/PARTITION BY expressions over aggregate functions
 2022-09-01 18:08:06 +02:00
Frank Chen
9d63cbe811 Merge 'origin/master' into tracing_context_propagation to resolve conflicts 2022-09-01 23:18:59 +08:00
Frank Chen
1da35a486c Disable query_span for internal queries 2022-09-01 23:16:05 +08:00
kssenii
ec55c90f36 Fix 2022-09-01 17:04:34 +02:00
Antonio Andelic
9231084c57 Style fix 2022-09-01 14:40:50 +00:00
Antonio Andelic
a0c1ce5eb7 Merge branch 'master' into embeddedrocksdb-delete-update-support 2022-09-01 14:40:24 +00:00
Vladimir C
12e6fc4182
Merge branch 'master' into join_and_sets 2022-09-01 14:56:14 +02:00
Vladimir C
f2cf7d7762
Merge pull request #40334 from ClickHouse/vdimir/analyze-stuck-limit 
Limit number of analyze for one query, att. 2
 2022-09-01 14:40:20 +02:00
Kseniia Sumarokova
c6c67a248d
Merge pull request #40792 from canhld94/ch_canh_intersect_distinct 
Implement intersect + except distinct
 2022-09-01 14:35:26 +02:00
Antonio Andelic
62ff61404f Improve remove recursive 2022-09-01 12:16:13 +00:00
kssenii
82879543d3 Merge remote-tracking branch 'upstream/master' into improve-marks-cache-loading 2022-09-01 13:13:01 +02:00
kssenii
9ae95f8c04 Fix unit test 2022-09-01 13:11:47 +02:00
kssenii
ca4363e2e3 Merge master 2022-09-01 13:09:00 +02:00
Anton Popov
f7bdf07adc
Merge pull request #38715 from CurtizJ/fix-read-in-order-fixed-prefix 
Better support of `optimize_read_in_order` in case of fixed prefix of sorting key
 2022-09-01 12:59:18 +02:00
kssenii
0d1f7afb17 Merge remote-tracking branch 'upstream/master' into fix-removing-directories 2022-09-01 12:28:51 +02:00
Kseniia Sumarokova
849262bd99
Merge pull request #40858 from kssenii/move-cache-files 
Move cache files from Common
 2022-09-01 12:28:00 +02:00
Sema Checherinda
7b59fdc042
Merge pull request #40779 from CheSema/detached-parts-metric 
Metric for the number of detached parts
 2022-09-01 12:24:42 +02:00
luocongkai
56e4179cad fix bug when remove unneeded columns in subquery 2022-09-01 17:10:32 +08:00
kssenii
10a7e73d52 Fix unit test 2022-08-31 21:07:39 +02:00
Robert Schulze
912663b719
Revert "Move CatBoost evaluation into clickhouse-library-bridge" 2022-08-31 20:54:43 +02:00
kssenii
0f747e7987 Move cache files 2022-08-31 17:50:55 +02:00
Antonio Andelic
f043790e6d Add support for update 2022-08-31 14:33:53 +00:00
Robert Schulze
ca01286028
Merge pull request #39629 from ClickHouse/catboost-bridge 
Move CatBoost evaluation into clickhouse-library-bridge
 2022-08-31 16:16:11 +02:00
Antonio Andelic
0e6b3b870a Add support for delete in RocksDB 2022-08-31 13:08:27 +00:00
vdimir
6c15db9b11 Setting max_analyze_depth forbid only second analyze 2022-08-31 11:57:41 +02:00
vdimir
60454bd722 Fix error codes in InterpreterSelectQuery.cpp 2022-08-31 11:57:41 +02:00
vdimir
f38f39ed6f Use max_analyze_depth 2022-08-31 11:57:41 +02:00
vdimir
928adb2b38 [WIP] Limit number of analyze for one query 2022-08-31 11:57:41 +02:00
Kseniia Sumarokova
9abcfce529
Merge pull request #40759 from kssenii/fix-write-through-cache-logical-error 
Fix logical error from write-though cache from stress test
 2022-08-31 11:55:18 +02:00
Duc Canh Le
aebd1cb7de
Merge branch 'master' into ch_canh_intersect_distinct 2022-08-31 12:08:20 +08:00
Anton Popov
3504781529
Merge branch 'master' into fix-read-in-order-fixed-prefix 2022-08-30 23:32:43 +02:00
Sema Checherinda
412973daee
Update src/Interpreters/AsynchronousMetrics.cpp 
Co-authored-by: Alexander Tokmakov <tavplubix@gmail.com>
 2022-08-30 21:57:39 +02:00
Sema Checherinda
6b23abe2b6
Update src/Interpreters/AsynchronousMetrics.cpp 
Co-authored-by: Alexander Tokmakov <tavplubix@gmail.com>
 2022-08-30 21:57:33 +02:00
Dmitry Novik
003483b616 Add compatibility setting 2022-08-30 20:49:40 +02:00
kssenii
a5f69517ae Allow to load marks in threadpool 2022-08-30 20:20:05 +02:00
Sema Checherinda
9bb4f556c3 work on remarks on the review 2022-08-30 20:18:35 +02:00
Dmitry Novik
0a8378d9cd
Merge branch 'master' into betterorderbyoptimization 2022-08-30 14:23:22 +02:00
Vladimir C
7266c007b7
Merge pull request #39131 from amosbird/cte-global-with-fix1 
Fix unused unknown columns introduced by WITH statement
 2022-08-30 13:52:19 +02:00
Kseniia Sumarokova
54f6f5bb2c
Merge branch 'master' into fix-write-through-cache-logical-error 2022-08-30 13:30:13 +02:00
Alexander Tokmakov
0ec7f068cc
Merge pull request #40677 from ClickHouse/query-insert-oom 
Avoid deadlock in case of new query and OOM
 2022-08-30 13:54:34 +03:00
Duc Canh Le
8590cc46c4 implement intersect + except distinct 2022-08-30 18:09:01 +08:00
Frank Chen
f17d56b528 Merge branch 'master' into tracing_context_propagation 2022-08-30 14:24:36 +08:00
Sema Checherinda
e436b4f4cc Metric for the number of detached parts 2022-08-30 02:12:57 +02:00
Robert Schulze
4d511332c4
chore: delete obsolete modelEvaluate() function 
- superseded by catboostEvaluate() which no longer uses the internal
  repository for external models

- also removed was statement SYSTEM RELOAD MODELS and the monitoring view
  SYSTEM.SYSTEMMODELS
 2022-08-29 20:27:06 +00:00
Dmitry Novik
e25ed9547e
Update src/Interpreters/ProcessList.h 2022-08-29 20:26:37 +02:00
Dmitry Novik
865ee5d0d6 Refactor code 2022-08-29 20:24:35 +02:00
Dmitry Novik
1169315580 Add OvercommitTracker blocking 2022-08-29 19:44:05 +02:00
Maksim Kita
88141cae98
Merge pull request #40732 from azat/thread-status-fix-leak 
Fix memory leak while pushing to MVs w/o query context (from Kafka/...)
 2022-08-29 19:36:25 +02:00
kssenii
545c6c8be4 Fix 2022-08-29 17:50:27 +02:00
Dmitry Novik
cfe509c3de Block overcommit tracker in ProcessList near allocations 2022-08-29 17:49:01 +02:00
Vladimir C
5cbe7e0846
Merge pull request #40548 from ClickHouse/vdimir/warn-suppress-40330 
Add config option warning_supress_regexp
 2022-08-29 14:02:00 +02:00
Anton Popov
2a3e012931
Merge branch 'master' into fix-read-in-order-fixed-prefix 2022-08-29 13:17:26 +02:00
vdimir
afb6b7d9cf
Test plan and pipeline for filtering step for join 2022-08-29 09:49:15 +00:00
vdimir
afeff512b5
Aux port for ReadHeadBalancedProcessor is empty Block 2022-08-29 09:49:14 +00:00
vdimir
67a9acc8db
rename CreatingSetOnTheFlyStep -> CreateSetAndFilterOnTheFlyStep 2022-08-29 09:49:07 +00:00
vdimir
6ae15db3ad
check join kind for before applying max_rows_in_set_to_optimize_join 2022-08-29 09:49:04 +00:00
vdimir
51a51694d6
Create sets for joins: better code 2022-08-29 09:49:01 +00:00
vdimir
c778bba13f
Create sets for joins: wip 2022-08-29 09:47:00 +00:00
vdimir
71708d595f
Create sets for joins: wip 2022-08-29 09:46:59 +00:00
vdimir
8f06430ebd
Create sets for joins: upd 2022-08-29 09:46:58 +00:00
vdimir
031aaf3a45
Add Creating/FilterBySetsOnTheFlyStep for full sorting join 2022-08-29 09:46:57 +00:00
Azat Khuzhin
269453a646 Avoid leaking of ThreadGroupStatus::finished_threads_counters_memory 
Cleanup them in ThreadStatus::detachQuery(), anyway they cannot be
received after by the client.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-08-29 11:08:50 +02:00
Kseniia Sumarokova
a0bc5b6ea4
Merge pull request #40705 from azat/stress/max_read_buffer_size 
Fix LOGICAL_ERROR with max_read_buffer_size=0 during reading marks
 2022-08-28 12:41:16 +02:00
Azat Khuzhin
c801469e9e Fix LOGICAL_ERROR with max_read_buffer_size=0 during reading marks 
The problem is that the buffer size validated after marks reading in
MergeTreeReaderStream::init(), since it requires to read marks first.

And later it is passed to AsynchronousReadBufferFromFileDescriptor,
which throws LOGICAL_ERROR because buffer_size < alignment.

Fix this my simply disallow such values for max_read_buffer_size (I
thougt about modifying createReadBufferFromFileBase(), but it is not
used for remote reads -- remote_fs_buffer_size).

Fixes: #40669
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-08-27 14:12:07 +02:00
Alexey Milovidov
6940bdab3b
Merge branch 'master' into fix-recursive-buffer 2022-08-26 21:25:11 +03:00
Dmitry Novik
5b7fe91675 Avoid deadlock in case of new query and OOM 2022-08-26 17:02:31 +02:00
Alexey Milovidov
ebe7d2d25e Fix stack overflow on recursive Buffer tables and system.tables 2022-08-25 21:26:32 +02:00
Frank Chen
92f7ca3616 Move TracingContextOnThread::current() out of class for simplicity 
Signed-off-by: Frank Chen <frank.chen021@outlook.com>
 2022-08-25 20:23:56 +08:00
Frank Chen
bb00dcc19b Remove using namespace from header 
Signed-off-by: Frank Chen <frank.chen021@outlook.com>
 2022-08-25 20:20:13 +08:00
vdimir
414872b1b9 Rename logger.warning_supress_regex -> warning_supress_regexp 2022-08-25 11:00:01 +02:00
vdimir
19cb068f64 Add config option warning_supress_regex 2022-08-25 11:00:01 +02:00
Frank Chen
99c37ce6c6
Merge branch 'master' into tracing_context_propagation 2022-08-25 10:07:16 +08:00
Nikita Taranov
ac34a17551
Merge branch 'master' into control_block_size_in_aggregator 2022-08-24 20:25:28 +02:00
vdimir
91788f29e8
Upd TemporaryFileOnDisk 2022-08-24 16:15:54 +00:00
vdimir
7194df1184
Move back TemporaryFile -> TemporaryFileOnDisk 2022-08-24 16:14:11 +00:00
vdimir
51c44424cc
More metrics for temp files 2022-08-24 16:14:09 +00:00
vdimir
cd4038d511
Use TemporaryFileOnDisk instead of Poco::TemporaryFile 2022-08-24 16:14:08 +00:00
vdimir
1321ac87b5
Minor fixes 2022-08-24 16:14:07 +00:00
Alexander Tokmakov
b8abecb908
Merge pull request #40577 from ClickHouse/revert-39107-parallel-insert-select-s3-cluster 
Revert "Parallel distributed insert select from *Cluster table functions"
 2022-08-24 15:26:03 +03:00
Alexander Tokmakov
9820c3fce0
Merge pull request #40545 from ClickHouse/fix_restart_replica_race 
Fix race on system restart replica
 2022-08-24 15:18:36 +03:00
Alexander Tokmakov
f9f85a0e8b Revert "Parallel distributed insert select from *Cluster table functions (#39107)" 
This reverts commit d3cc234986.
 2022-08-24 15:17:15 +03:00
Frank Chen
cd19366b44 Move classes into DB::OpenTelemetry namespace 2022-08-24 16:41:40 +08:00
Frank Chen
efc6a60a60 Clean code 2022-08-24 15:59:44 +08:00
Amos Bird
a8d8293466
Fix unused columns introduced by with stmt 2022-08-24 00:40:01 +08:00
Alexander Tokmakov
77178a18fa fix race on system restart replica 2022-08-23 16:10:44 +02:00
Vladimir C
3ee52eb25f
Merge pull request #40335 from liql2007/master 
Fix bugs in MergeJoin when 'not_processed' is not null
 2022-08-23 16:02:23 +02:00
Kseniia Sumarokova
d814a56277
Merge pull request #40420 from kssenii/fix-cache-bug 
Fix incorrect fallback to skip cache in case of very high concurrency level
 2022-08-23 14:18:38 +02:00
Vadim Volodin
9c371d518f fix enable_unfreeze handler and sanitizer hints 2022-08-23 13:53:06 +03:00
Robert Schulze
635cc7617e
Merge pull request #40464 from ClickHouse/less-string-ref 
Reduce some usage of StringRef
 2022-08-23 12:16:00 +02:00
Nikita Taranov
b31342ec2c fix 2022-08-22 19:29:48 +02:00
kssenii
26384b2543 Fix 2022-08-22 00:02:05 +02:00
Robert Schulze
4caf2c4c33
Reduce some usage of StringRef 
See #39535 and #39300
 2022-08-21 18:10:32 +00:00
Alexey Milovidov
ab91c99495
Merge branch 'master' into control_block_size_in_aggregator 2022-08-20 21:28:27 +03:00
Alexey Milovidov
74e1f4dc61 Fix clang-tidy 2022-08-20 17:09:20 +02:00
Frank Chen
024fef1aec
Merge branch 'master' into tracing_context_propagation 2022-08-20 14:23:36 +08:00
Alexander Tokmakov
8cbdd39c81
Update ActionsDAG.cpp 2022-08-19 11:34:15 +03:00
Alexander Tokmakov
b9e090699a
Merge branch 'master' into fix_incorrect_replica_idenifier 2022-08-18 21:18:10 +03:00
Alexander Tokmakov
629690c32b fix "incorrect replica identifier" 2022-08-18 16:13:27 +02:00
Frank Chen
a3b6ad2a65
Merge branch 'master' into tracing_context_propagation 2022-08-18 20:59:07 +08:00
Kseniia Sumarokova
be1b835f88
Merge branch 'master' into fix-typo 2022-08-18 14:00:23 +02:00
Nikolai Kochetov
5a85531ef7
Merge pull request #38286 from Avogar/schema-inference-cache 
Add schema inference cache for s3/hdfs/file/url
 2022-08-18 13:07:50 +02:00
yinpeiqi
042ad7d9a8 fix typo 2022-08-18 18:45:16 +08:00
Kseniia Sumarokova
d8fd1f9c37
Merge pull request #36171 from kssenii/make-cache-composable 
Make cache composable, allow not to evict certain files (regarding .idx, .mrk, ..), delete old cache version
 2022-08-18 12:37:34 +02:00
Kruglov Pavel
d7056376eb
Merge pull request #40068 from Avogar/schema-inference-hints 
Allow to specify structure hints in schema inference
 2022-08-18 12:19:45 +02:00
liql2007
db0ca7c1d4 fix bugs in MergeJoin when 'not_processed' is not null 2022-08-18 18:09:47 +08:00
Vladimir C
7ac0eb1af1
Merge pull request #40035 from ClickHouse/vdimir/cluster-discovery-observer 
Add observer mode to (zoo)keeper cluster discovery feature
 2022-08-18 10:53:49 +02:00
vdimir
7305158a58
More robust ClusterDiscovery::updateCluster 2022-08-17 14:46:06 +00:00
kssenii
d63d34f7bb Merge master 2022-08-17 15:49:32 +02:00
Yong Wang
c2c457ea0e Kusto-phase1: Change the dialect to Enum, rename sql_dialect to dialect, set limit to subquery 2022-08-17 06:03:41 -07:00
avogar
8dd54c043d Merge branch 'master' of github.com:ClickHouse/ClickHouse into schema-inference-cache 2022-08-17 11:47:40 +00:00
Igor Nikonov
46ed4f6cdf
Merge pull request #38719 from ClickHouse/skipping_sorting_step 
SortingStep: deduce way to sort based on input stream sort description
 2022-08-17 12:58:11 +02:00
Alexander Tokmakov
d1ffa21e2f fix 2022-08-17 12:17:37 +02:00
Alexander Tokmakov
caa270b72a
Merge pull request #39933 from ClickHouse/auto_convert_ordinary_to_atomic 
Add flag that enables automatic conversion from Ordinary to Atomic
 2022-08-17 12:40:49 +03:00
Nikita Mikhaylov
a7c3f23dfb
Merge branch 'master' into auto_convert_ordinary_to_atomic 2022-08-17 00:38:25 +02:00
Yong Wang
87182ccd51 Kusto-phase1: Add Support to Kusto Query Language 
This is the initial implement of Kusto Query Language.

in this commit, we support the following features as MVP :

Tabular expression statements
Limit returned results
Select Column (basic project)
sort, order
Perform string equality operations
Filter using a list of elements
Filter using common string operations
Some string  operators
Aggregate by columns
Base aggregate functions
only support avg, count ,min, max, sum
Aggregate by time intervals
 2022-08-16 17:31:38 -04:00
Nikita Taranov
14d45f9d86 fix style 2022-08-16 18:56:22 +02:00
Nikita Taranov
a6c4f9218a clean up 2022-08-16 18:56:22 +02:00
Nikita Taranov
248011d7d9 move to utils 2022-08-16 18:56:22 +02:00
Nikita Taranov
6bdbaccc37 use max_block_size from settings 2022-08-16 18:56:22 +02:00
Nikita Taranov
370c25cd2a fix comment 2022-08-16 18:56:22 +02:00
Nikita Taranov
2b76abdacd fix tidy 2022-08-16 18:56:22 +02:00
Nikita Taranov
66b3268c65 fix 2022-08-16 18:56:22 +02:00
Nikita Taranov
56c09bf8a9 generate many blocks in convertToBlockImplNotFinal 2022-08-16 18:56:22 +02:00
Nikita Taranov
433657e978 rm prepareBlockAndFill 2022-08-16 18:56:22 +02:00
Nikita Taranov
63bc894a42 more parallelism 2022-08-16 18:56:22 +02:00
Nikita Taranov
f650b23ee3 generate many blocks 2022-08-16 18:56:22 +02:00
Nikita Taranov
db0110fd7a more accurate crutch 2022-08-16 18:56:22 +02:00
Nikita Taranov
e5e0a24ab3 return chunks from prepareBlockAndFillWithoutKey 2022-08-16 18:56:22 +02:00
Nikita Taranov
4e974661d6 refactor convertToBlockImpl 2022-08-16 18:56:22 +02:00
Nikita Taranov
bca28ba9f8 split prepareBlockAndFill 2022-08-16 18:56:22 +02:00
Igor Nikonov
d4367de7bb Rename setting to optimize_sorting_by_input_stream_properties 2022-08-16 16:27:41 +00:00
Alexander Tokmakov
0136262a2a fix 2022-08-16 18:03:02 +02:00
Igor Nikonov
51f4716b29 Fix review comments 2022-08-16 14:59:53 +00:00
Alexander Tokmakov
832d7c6e7c Merge branch 'master' into replicated_database_improvements 2022-08-16 15:49:13 +02:00
avogar
936c457734 Remove unnended field 2022-08-16 09:51:52 +00:00
avogar
e1ff996ec3 Allow to specify structure hints in schema inference 2022-08-16 09:46:57 +00:00
Maksim Kita
110470809b
Merge pull request #40121 from amosbird/profile-processor-1 
Extend processors_profile_log with more info
 2022-08-16 09:49:12 +02:00
kssenii
eb26b219b9 Merge master 2022-08-16 00:56:27 +02:00
Igor Nikonov
7a9bdbe81a Merge remote-tracking branch 'origin/master' into skipping_sorting_step 2022-08-15 20:59:37 +00:00
Alexey Milovidov
4f1b964f2f
Merge branch 'master' into window-order-by 2022-08-15 19:31:55 +03:00
Alexey Milovidov
a0693c3a84
Merge pull request #40225 from ClickHouse/simplify-code 
Simplify the code and check what will happen
 2022-08-15 19:05:38 +03:00
Igor Nikonov
ea10fd65b8 Sorting properties in EXPLAIN PLAN 
~ change formatting for sorting
~ rename sortmode option -> sorting
 2022-08-15 15:14:59 +00:00
Nikita Mikhaylov
d3cc234986
Parallel distributed insert select from *Cluster table functions (#39107) 2022-08-15 12:41:17 +02:00
Alexey Milovidov
24371f6cf0 Simplify the code and check what will happen 2022-08-15 06:56:29 +02:00
Alexey Milovidov
903021f35a
Merge branch 'master' into profile-events-incomplete-result 2022-08-14 11:04:19 +03:00
Alexey Milovidov
1a8ddf2956 Addition to prev. revision 2022-08-14 09:35:22 +02:00
Alexey Milovidov
216c1d8efa
Revert "Use separate counter for RSS in global memory tracker." 2022-08-14 09:50:12 +03:00
Igor Nikonov
d83bea626c Merge remote-tracking branch 'origin/master' into skipping_sorting_step 2022-08-13 21:46:34 +00:00
Igor Nikonov
9566e2995d Fix of check if sorting order is preserved 2022-08-13 21:44:29 +00:00
Igor Nikonov
f33a0d8c85 More simple way to check if sorting order is preserved 
- there is a case where it's done wrong
 2022-08-12 23:42:37 +00:00
Kseniia Sumarokova
a6cfc7bc3b
Merge pull request #34651 from alexX512/master 
New caching strategies
 2022-08-12 17:23:37 +02:00
Igor Nikonov
bf037275a5 More robust check if sorting order is changed by expression 
+ tests
 2022-08-12 14:57:18 +00:00
vdimir
89dca02e1d
Fix ClusterDiscovery 2022-08-12 13:26:17 +00:00
vdimir
cef6c4f8ce
Add observer mode to (zoo)keeper cluster discovery feature 2022-08-12 13:26:12 +00:00
Nikita Taranov
17956cb668
Extend protocol with query parameters (#39906) 2022-08-12 14:28:35 +02:00
Nikolai Kochetov
86061d945e
Merge pull request #39826 from amosbird/projection-row-policy-fix 
Primary key analysis for row_policy_filter and additional_filter
 2022-08-12 11:32:58 +02:00
kssenii
768a207176 Merge master 2022-08-11 22:25:53 +02:00
Kseniia Sumarokova
be69169f97
Merge pull request #38048 from KinderRiven/decoupling_cache_function_and_algorithm 
Decoupling local cache function and cache algorithm
 2022-08-11 16:49:56 +02:00
Amos Bird
99a38e41aa
processor profile 2022-08-11 21:03:34 +08:00
Maksim Kita
45d69951e2
Merge pull request #40069 from Algunenano/interpreter_udf_visitor 
Don't visit the AST for UDFs if none are registered
 2022-08-11 14:54:14 +02:00
Igor Nikonov
d9f5c56c41 Fix compilation errors after fixing merge conflicts 2022-08-11 12:51:45 +00:00
Igor Nikonov
75f6fcfa70 Merge remote-tracking branch 'origin/master' into skipping_sorting_step 2022-08-11 12:35:55 +00:00
Vladimir C
643084417e
Merge pull request #40113 from vdimir/dict-join-refactoring-followup 
Delete files DictionaryJoinAdapter.h/cpp
 2022-08-11 13:25:45 +02:00
Amos Bird
fa8fab2e8f
Fix KeyCondition with other filters 2022-08-11 19:20:44 +08:00
Maksim Kita
627424889d
Merge pull request #39926 from azat/dict-prom-fix 
Remove dictionaries from prometheus metrics on DETACH/DROP
 2022-08-11 13:10:47 +02:00
vdimir
61ad12279e
Delete files DictionaryJoinAdapter.h/cpp 
Follow-up for https://github.com/ClickHouse/ClickHouse/pull/38956
 2022-08-11 10:27:14 +00:00
Maksim Kita
6bec0f5854
Merge pull request #38956 from vdimir/dict-join-refactoring 
Join with dictionary refactoring
 2022-08-11 11:54:11 +02:00
Vladimir C
2d44e6c458
Merge pull request #39343 from vdimir/refactor-prepared-sets 
Refactor PreparedSets/SubqueryForSet
 2022-08-11 11:19:18 +02:00
Vladimir Chebotaryov
748979a9c0
Merge branch 'master' into betterorderbyoptimization 2022-08-11 11:09:52 +03:00
Raúl Marín
ce9c0c2da3 Style 2022-08-10 21:53:11 +02:00
KinderRiven
9d83b93e88 fix rebase 2022-08-11 00:11:40 +08:00
Raúl Marín
32c63f43a1 Don't visit the AST for UDFs if none are registered 2022-08-10 17:54:56 +02:00
kssenii
0ff5609738 Finish support for persistent files 2022-08-10 16:54:59 +02:00
Nikita Mikhaylov
8e8e845a89
Merge branch 'master' into separate-counter-for-rss-in-memory-tracker 2022-08-10 16:44:24 +02:00
vdimir
90fa2ed8c1
better code for join with dict 2022-08-10 14:20:29 +00:00
vdimir
44c688332a
IKeyValueEntity is not inheritor of IStorage 2022-08-10 14:20:28 +00:00
vdimir
ad91c16ba0
Rename join_common -> JoinUtils 2022-08-10 14:20:28 +00:00
vdimir
f7b130b0cb
Rename IKVStorage.h -> IKeyValueStorage.h 2022-08-10 14:20:28 +00:00
vdimir
442ffb0349
Fix build after resolving conflicts 2022-08-10 14:20:27 +00:00
vdimir
d1aea19987
Remove old join with dictionary 2022-08-10 14:20:26 +00:00
vdimir
7073067d40
check attributes for join with dict 2022-08-10 14:20:26 +00:00
vdimir
3347345873
Join with dictionary uses DirectJoin 2022-08-10 14:20:18 +00:00
vdimir
b7c5c54181
Fix build 2022-08-10 13:43:55 +00:00
Nikolai Kochetov
933043d841 Minor fixes. 2022-08-10 12:38:35 +00:00
vdimir
5eb4cd39e0
Merge branch 'master' into refactor-prepared-sets 2022-08-10 11:47:49 +00:00
kssenii
5c3227ba56 Merge master 2022-08-10 12:00:34 +02:00
Maksim Kita
aff8149f5c
Merge pull request #39998 from kitaisreal/actions-dag-refactoring 
ActionsDAG rename index to outputs
 2022-08-10 11:44:18 +02:00
Vladimir C
1196d1d117
Merge pull request #40009 from ClickHouse/vdimir/rename-joined-project-input 
Project input for rename joined columns step
 2022-08-10 10:32:36 +02:00
Yakov Olkhovskiy
ffbaf72bab
Merge pull request #39874 from ClickHouse/fix-quota-key 
Fix quota_key
 2022-08-10 00:12:44 -04:00
Igor Nikonov
754a9fb096 Merge remote-tracking branch 'origin/master' into skipping_sorting_step 2022-08-09 22:20:17 +00:00
Igor Nikonov
bf5269d536 Avoid copying SortDescription during SortingStep construction 2022-08-09 22:19:25 +00:00
kssenii
0dda03c94b Fix checks 2022-08-10 00:06:58 +02:00
Maksim Kita
acbfcf440b
Merge branch 'master' into actions-dag-refactoring 2022-08-09 18:52:08 +02:00
Igor Nikonov
70b52f7cb9 Fix test, review comments 2022-08-09 16:29:56 +00:00
Igor Nikonov
366ead3828 Consider aliases when checking if sorting order is preserved by 
expression
 2022-08-09 11:27:17 +00:00
vdimir
29efed0c77
Project input for rename joined columns step 2022-08-08 19:11:53 +00:00
Alexander Gololobov
4c86130796 More generic check for MergeTree table family 2022-08-08 19:04:35 +02:00
Maksim Kita
c030fd05e7 ActionsDAG rename index to outputs 2022-08-08 18:01:32 +02:00
Yakov Olkhovskiy
d39e9f65de
Merge branch 'master' into fix-quota-key 2022-08-08 11:54:21 -04:00
Alexander Tokmakov
2c5da1466d Merge branch 'master' into replicated_database_improvements 2022-08-08 16:08:27 +02:00
Alexander Tokmakov
848511affb take permanently detached tables into account 2022-08-08 16:06:08 +02:00
Igor Nikonov
86c5280db3 Merge remote-tracking branch 'origin/master' into skipping_sorting_step 2022-08-08 12:43:06 +00:00
Igor Nikonov
6572a447fa Check if sorting order is preserved for FilterStep 2022-08-08 12:42:10 +00:00
vdimir
708747ca0b
Merge branch 'master' into refactor-prepared-sets 2022-08-08 14:27:18 +02:00
avogar
1cfb5b7cf1 Fix build 2022-08-08 12:17:18 +00:00
alexX512
62f84b88eb Review fixes 2022-08-07 20:02:21 +00:00
alexX512
2be72560f4 Add loading of mark cache policy 2022-08-07 19:59:39 +00:00
alexX512
6bf29cb610 Change class LRUCache to class CachBase. Check running CacheBase with default pcahce policy SLRU 2022-08-07 19:59:30 +00:00
alexX512
1133e42367 Add CacheBase clas instead of LRUCache and SLRUCache fo simpler configuration oache policiesf 2022-08-07 19:16:40 +00:00
Igor Nikonov
988ce29e9e Check if expression step violates sorting correctly 2022-08-06 22:37:14 +00:00
Azat Khuzhin
2d7de7f683 Remove dictionaries from prometheus metrics on DETACH/DROP 
Fixes: #23436 (cc @kitaisreal)
Introduced-in: #9622 (cc @YiuRULE)
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-08-06 12:58:07 +03:00
Alexander Tokmakov
f2c4cad255 add flag that enables automatic canversion from Ordinary to Atomic 2022-08-05 21:41:25 +02:00
avogar
9b1a267203 Refactor, remove TTL, add size limit, add system table and system query 2022-08-05 16:20:15 +00:00
Nikolai Kochetov
b41b716245
Update AsynchronousMetrics.cpp 2022-08-05 18:16:35 +02:00
Igor Nikonov
f5f6ada69b ActionsDAG check is not dependent on sort description 2022-08-05 11:00:18 +00:00
kssenii
91f4143b6e Merge master 2022-08-05 12:14:08 +02:00
Nikolai Kochetov
2fe893ba67
Merge pull request #25563 from ClickHouse/use-dag-in-key-condition 
Use ActionsDAG in KeyCondition
 2022-08-05 11:22:46 +02:00
kssenii
7a9b0bc47f Merge master 2022-08-05 01:48:52 +02:00
Igor Nikonov
246f7b71b6
Merge branch 'master' into skipping_sorting_step 2022-08-05 00:03:54 +02:00
Igor Nikonov
a70c47f780 EXPLAIN PLAN sortmode options 
+ ActionsDAG fixes
 2022-08-04 22:02:53 +00:00
Igor Nikonov
0a659f5ab8 Analyze ActionsDAG in ExpressionStep 
+ check if sorting order can be kept from prevous step
+ EXPLAIN PLAN header=1 contains Sort Mode and Sort Description for each step
+ some tests
+ fix some review comments
 2022-08-04 19:30:25 +00:00
Alexander Tokmakov
460f2f2e39 Merge branch 'master' into store_cleanup_all_disks 2022-08-04 19:48:41 +02:00
Yakov Olkhovskiy
23037daf17
Merge branch 'master' into fix-quota-key 2022-08-04 12:14:49 -04:00
Nikolai Kochetov
658a269d56
Merge branch 'master' into use-dag-in-key-condition 2022-08-04 16:18:40 +02:00
Alexander Tokmakov
6a5171c829
Update DatabaseCatalog.cpp 2022-08-04 17:10:13 +03:00
Vladimir C
a627b00c43
Merge branch 'master' into refactor-prepared-sets 2022-08-04 13:27:38 +02:00
Alexey Milovidov
9e46abc560
Merge pull request #39420 from amosbird/better-projection1-fix1 
Normalize AggregateFunction types and state representations
 2022-08-04 03:06:55 +03:00
Alexey Milovidov
f474eb957d
Merge pull request #39758 from ClickHouse/tsan_clang_15 
Try clang-15 for build with tsan
 2022-08-04 01:56:52 +03:00
Yakov Olkhovskiy
2e34b384c1 update tcp protocol, add quota_key 2022-08-03 15:44:08 -04:00
Alexander Tokmakov
a269cf29e4 remove unused dirs from store on all disks 2022-08-03 19:49:16 +02:00
Nikita Taranov
4943202921
Improve memory usage during memory efficient merging of aggregation results (#39429) 2022-08-03 17:56:59 +02:00
Igor Nikonov
9eef299e11
Merge pull request #39103 from tonickkozlov/tonickkozlov/37032/do-not-optimize-functions-shadowing-args 
Do not optimize GROUP BY functions that shadow their arguments
 2022-08-03 15:06:42 +02:00
Alexander Tokmakov
1a059035b8 fix logical error on connection loss 2022-08-03 14:13:01 +02:00