thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-11-27 01:51:59 +00:00
Code Issues Packages Projects Releases Wiki Activity
122,743 Commits 9,198 Branches 1,782 Tags 2.1 GiB
ec0561b3f1
Commit Graph

122743 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Alexey Milovidov
ec0561b3f1
Merge pull request #53569 from ClickHouse/dashboard-better 
The dashboard just got better
 2023-08-19 12:11:49 +03:00
Michael Kolupaev
a1522e22ea
Merge pull request #53281 from Avogar/batch-small-parquet-row-groups 
Optimize reading small row groups by batching them together in Parquet
 2023-08-18 17:15:42 -07:00
Alexey Milovidov
1183c18a18 Update the docs 2023-08-19 01:39:21 +02:00
Alexey Milovidov
68e694815c The dashboard just got better 2023-08-19 01:38:22 +02:00
Alexey Milovidov
00e2e3184b
Merge pull request #53439 from canhld94/timeout_overflow_break_subquery 
Fix timeout_overflow_mode when having subquery in the rhs of IN
 2023-08-19 02:24:42 +03:00
Alexey Milovidov
c22356bcaa
Merge pull request #53357 from chhetripradeep/pchhetri/add-keeper-client 
Add clickhouse-keeper-client and clickhouse-keeper-converter symlinks to clickhouse-keeper package
 2023-08-19 02:24:02 +03:00
Alexey Milovidov
17b24eb38d
Merge pull request #53490 from rschu1ze/boost-179 
Update boost to 1.79
 2023-08-19 02:12:13 +03:00
Alexey Milovidov
40d13975fb
Merge pull request #53548 from ClickHouse/vdimir/join_on_cond_sparse_fix 
Fix crash in join on sparse column
 2023-08-19 02:08:21 +03:00
Alexey Milovidov
8520a7baca
Merge pull request #53559 from azat/skip-index-set-fix 
Fix possible UB in Set skipping index for functions with incorrect args
 2023-08-19 02:06:18 +03:00
Alexey Milovidov
6fb24fe74d
Merge pull request #53560 from azat/skip-index-inverted-fix-ub 
Fix possible UB in inverted indexes (experimental feature)
 2023-08-19 02:06:04 +03:00
Alexander Tokmakov
41a39d63fe
Merge pull request #53564 from ClickHouse/tavplubix-patch-8 
Update 02443_detach_attach_partition.sh
 2023-08-19 01:08:25 +03:00
Alexander Tokmakov
d32fe2dae9
Update 02443_detach_attach_partition.sh 2023-08-18 23:17:56 +03:00
Alexey Milovidov
e6a29d7590
Update 02861_index_set_incorrect_args.sql 2023-08-18 21:48:33 +03:00
Alexey Milovidov
1e4f548cbc
Update 02862_index_inverted_incorrect_args.sql 2023-08-18 21:46:54 +03:00
Alexey Milovidov
7ad8ea78bb
Merge pull request #48990 from incfly/partition 
Add partition to the system part_log and merge table.
 2023-08-18 21:38:53 +03:00
Nikolai Kochetov
c99c3112ef
Merge pull request #53550 from ClickHouse/group_array_max_element_size 
Add server setting `aggregate_function_group_array_max_element_size`.
 2023-08-18 19:16:24 +02:00
Alexander Tokmakov
14590305ad
Merge pull request #52975 from aalexfvk/ignore_on_cluster_for_replicated_entity_queries 
Ignore ON CLUSTER clause in queries for management of replicated entities (UDF and Access)
 2023-08-18 19:29:41 +03:00
Azat Khuzhin
c78b94bed6 Fix possible UB in inverted indexes (experimental feature) 
It is possible to have incorrect types there in case of index contains
functions, add a proper check after value had been changed.

<details>

<summary>MSan report</summary>

```
==182==WARNING: MemorySanitizer: use-of-uninitialized-value
    0 0x55658547c59e in DB::MergeTreeConditionInverted::traverseASTEquals() build_docker/./src/Storages/MergeTree/MergeTreeIndexInverted.cpp:551:26
    1 0x556585475566 in DB::MergeTreeConditionInverted::traverseAtomAST() build_docker/./src/Storages/MergeTree/MergeTreeIndexInverted.cpp:438:21
    2 0x55658547e73e in DB::MergeTreeConditionInverted::MergeTreeConditionInverted()::$_1::operator()(DB::RPNBuilderTreeNode const&, DB::MergeTreeConditionInverted::RPNElement&) const build_docker/./src/Storages/MergeTree/MergeTreeIndexInverted.cpp:228:73
    9 0x556585489e35 in DB::RPNBuilder<DB::MergeTreeConditionInverted::RPNElement>::traverseTree(DB::RPNBuilderTreeNode const&) build_docker/./src/Storages/MergeTree/RPNBuilder.h:252:14
    10 0x556585489921 in DB::RPNBuilder<DB::MergeTreeConditionInverted::RPNElement>::traverseTree(DB::RPNBuilderTreeNode const&) build_docker/./src/Storages/MergeTree/RPNBuilder.h:239:21
    11 0x556585489921 in DB::RPNBuilder<DB::MergeTreeConditionInverted::RPNElement>::traverseTree(DB::RPNBuilderTreeNode const&) build_docker/./src/Storages/MergeTree/RPNBuilder.h:239:21
    12 0x5565854804e6 in DB::RPNBuilder<DB::MergeTreeConditionInverted::RPNElement>::RPNBuilder(std::__1::shared_ptr<DB::IAST> const&, std::__1::shared_ptr<DB::Context const>, DB::Block, std::__1::shared_ptr<DB::PreparedSets>, std::__1::function<bool (DB::RPNBuilderTreeNode const&, DB::MergeTreeConditionInverted::RPNElement&)> const&) build_docker/./src/Storages/MergeTree/RPNBuilder.h:218:9
    13 0x55658546fb87 in DB::MergeTreeConditionInverted::MergeTreeConditionInverted(DB::SelectQueryInfo const&, std::__1::shared_ptr<DB::Context const>, DB::Block const&, DB::GinFilterParameters const&, DB::ITokenExtractor const*) build_docker/./src/Storages/MergeTree/MergeTreeIndexInverted.cpp:223:28
    19 0x55658547ceda in DB::MergeTreeIndexInverted::createIndexCondition(DB::SelectQueryInfo const&, std::__1::shared_ptr<DB::Context const>) const build_docker/./src/Storages/MergeTree/MergeTreeIndexInverted.cpp:716:12
    20 0x556587125734 in DB::buildIndexes(std::__1::optional<DB::ReadFromMergeTree::Indexes>&, std::__1::shared_ptr<DB::ActionsDAG>, DB::MergeTreeData const&, std::__1::shared_ptr<DB::Context const> const&, DB::SelectQueryInfo const&, std::__1::shared_ptr<DB::StorageInMemoryMetadata const> const&) build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:1292:48
    21 0x556587121aaa in DB::ReadFromMergeTree::applyFilters() build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:1305:5
    22 0x55658726cf4f in DB::QueryPlanOptimizations::optimizeTreeThirdPass(DB::QueryPlan::Node&, std::__1::list<DB::QueryPlan::Node, std::__1::allocator<DB::QueryPlan::Node>>&) build_docker/./src/Processors/QueryPlan/Optimizations/optimizeTree.cpp:201:38
    23 0x5565870bc489 in DB::QueryPlan::optimize(DB::QueryPlanOptimizationSettings const&) build_docker/./src/Processors/QueryPlan/QueryPlan.cpp:485:5
    24 0x5565870b919a in DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) build_docker/./src/Processors/QueryPlan/QueryPlan.cpp:161:5
    25 0x55658206385a in DB::InterpreterSelectWithUnionQuery::execute() build_docker/./src/Interpreters/InterpreterSelectWithUnionQuery.cpp:379:31
    26 0x556582c15c50 in DB::executeQueryImpl(char const*, char const*, std::__1::shared_ptr<DB::Context>, bool, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) build_docker/./src/Interpreters/executeQuery.cpp:751:40
    27 0x556582c09681 in DB::executeQuery(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::shared_ptr<DB::Context>, bool, DB::QueryProcessingStage::Enum) build_docker/./src/Interpreters/executeQuery.cpp:1173:30
    28 0x5565860e7bd7 in DB::TCPHandler::runImpl() build_docker/./src/Server/TCPHandler.cpp:421:24
    29 0x556586126e9e in DB::TCPHandler::run() build_docker/./src/Server/TCPHandler.cpp:2057:9
```

</details>

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-08-18 18:15:17 +02:00
Nikolai Kochetov
26c7dd2ebb Fixing tidy 2023-08-18 16:08:17 +00:00
Azat Khuzhin
1e2b0c8dc4 Fix possible UB in Set skipping index for functions with incorrect args 
Usually functions are validated before index analysis, however it is
not the case for parallel replicas, and it requires additional checks
before interpreting function arguments.

<details>

<summary>stack trace</summary>

```
==172==WARNING: MemorySanitizer: use-of-uninitialized-value
    0 0x55cf82aedd6c in DB::ASTFunction* typeid_cast<DB::ASTFunction*, DB::IAST>(DB::IAST*) build_docker/./src/Common/typeid_cast.h:42:73
    1 0x55cf82aedd6c in DB::TypePromotion<DB::IAST>::CastHelper<DB::ASTFunction, false, false>::value(DB::IAST*) build_docker/./src/Common/TypePromotion.h:38:43
    2 0x55cf82aedd6c in std::__1::invoke_result<decltype(&CastHelper<DB::ASTFunction, false>::value), DB::TypePromotion<DB::IAST>::CastHelper<DB::ASTFunction, false, std::is_reference_v<DB::ASTFunction>>, DB::IAST*>::type DB::TypePromotion<DB::IAST>::as<DB::ASTFunction>() build_docker/./src/>
    3 0x55cf82aedd6c in DB::MergeTreeIndexConditionSet::operatorFromAST(std::__1::shared_ptr<DB::IAST>&) build_docker/./src/Storages/MergeTree/MergeTreeIndexSet.cpp:602:25
    4 0x55cf82ae5bc1 in DB::MergeTreeIndexConditionSet::traverseAST(std::__1::shared_ptr<DB::IAST>&) const build_docker/./src/Storages/MergeTree/MergeTreeIndexSet.cpp:547:9
    5 0x55cf82ae5de6 in DB::MergeTreeIndexConditionSet::traverseAST(std::__1::shared_ptr<DB::IAST>&) const build_docker/./src/Storages/MergeTree/MergeTreeIndexSet.cpp:552:13
    6 0x55cf82ae06db in DB::MergeTreeIndexConditionSet::MergeTreeIndexConditionSet() build_docker/.>
    ...
    12 0x55cf82aef09c in DB::MergeTreeIndexSet::createIndexCondition() const build_docker/./src/Storages/MergeTree/MergeTreeIndexSet.cpp:703:12
    13 0x55cf84951cd4 in DB::buildIndexes()
    14 0x55cf84955ed3 in DB::ReadFromMergeTree::selectRangesToReadImpl()
    15 0x55cf8494caef in DB::ReadFromMergeTree::selectRangesToRead()
    16 0x55cf82a409a9 in DB::MergeTreeDataSelectExecutor::estimateNumMarksToRead()
    17 0x55cf827f728d in DB::MergeTreeData::canUseParallelReplicasBasedOnPKAnalysis()
    18 0x55cf827f627e in DB::MergeTreeData::getQueryProcessingStage()
    19 0x55cf7f2f4969 in DB::InterpreterSelectQuery::getSampleBlockImpl() build_docker/./src/Interpreters/InterpreterSelectQuery.cpp:937:31
    20 0x55cf7f2daa00 in DB::InterpreterSelectQuery::InterpreterSelectQuery()
    24 0x55cf7f520b98 in DB::InterpreterSelectWithUnionQuery::buildCurrentChildInterpreter()
    25 0x55cf7f51b6cd in DB::InterpreterSelectWithUnionQuery::InterpreterSelectWithUnionQuery()
    27 0x55cf7f1d4ea9 in DB::InterpreterFactory::get(std::__1::shared_ptr<DB::IAST>&, std::__1::shared_ptr<DB::Context>, DB::SelectQueryOptions const&) build_docker/./src/Interpreters/InterpreterFactory.cpp:162:16
    28 0x55cf8012e485 in DB::executeQueryImpl(char const*, char const*, std::__1::shared_ptr<DB::Context>, bool, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) build_docker/./src/Interpreters/executeQuery.cpp:1032:31
    29 0x55cf80121bc1 in DB::executeQuery(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::shared_ptr<DB::Context>, bool, DB::QueryProcessingStage::Enum) build_docker/./src/Interpreters/executeQuery.cpp:1229:30
    30 0x55cf8389295f in DB::TCPHandler::runImpl() build_docker/./src/Server/TCPHandler.cpp:424:24
    31 0x55cf838d7dfb in DB::TCPHandler::run() build_docker/./src/Server/TCPHandler.cpp:2050:9
```

</details>

CI: https://s3.amazonaws.com/clickhouse-test-reports/53214/d99b10c340909ed4ee2e6edf0921e8a2f8561b0d/fuzzer_astfuzzermsan/report.html
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-08-18 18:02:46 +02:00
Alexander Tokmakov
334e014c2c Merge branch 'master' into ignore_on_cluster_for_replicated_entity_queries 2023-08-18 17:48:37 +02:00
Yakov Olkhovskiy
84cb358c14
Merge pull request #43210 from amosbird/storage_executable_stderr 
Add different reactions when external command stderr has data
 2023-08-18 11:17:46 -04:00
Alexander Tokmakov
a6684f7a37
Merge pull request #53484 from azat/rmt-queue-cleanup 
Remove outdated code in ReplicatedMergeTreeQueue::initialize()
 2023-08-18 18:11:33 +03:00
Nikolay Degterinsky
4c606590a1
Fix tiny thing in Replicated database (#53496) 
* Fix tiny thing in Replicated database

* empty commit

---------

Co-authored-by: Alexander Tokmakov <tavplubix@clickhouse.com>
 2023-08-18 17:29:24 +03:00
Alexander Tokmakov
4c3cb273a8
Merge pull request #53505 from arenadata/ADQM-1098 
Added waiting for PostgreSQL compatibility port open in integrational tests.
 2023-08-18 17:21:01 +03:00
vdimir
ae96d32027
Fix crash in join on sparse column 2023-08-18 14:14:54 +00:00
Alexander Tokmakov
763db4cc26
Merge pull request #53544 from ClickHouse/tavplubix-patch-9 
Follow-up to #53528
 2023-08-18 16:41:40 +03:00
robot-clickhouse
f1cb8a83fe Automatic style fix 2023-08-18 13:34:20 +00:00
Nikolai Kochetov
e9d8745971
Merge branch 'master' into group_array_max_element_size 2023-08-18 15:23:01 +02:00
Nikolai Kochetov
88e81a31fa Add server setting group_array_max_element_size. 2023-08-18 13:15:27 +00:00
Nikita Mikhaylov
1822012180
Merge pull request #53485 from rschu1ze/CVE-2023-36054 
krb5: Fix CVE-2023-36054
 2023-08-18 14:56:08 +02:00
Alexander Gololobov
50061b9cb4
Merge pull request #53452 from ClickHouse/such_much_watch 
Deduplicate same watch callbacks registered multiple times
 2023-08-18 14:43:03 +02:00
Robert Schulze
b8912203e4
Merge pull request #53525 from aiven-sal/aiven-sal/siphash_readd 
Re-add SipHash keyed functions, try II
 2023-08-18 14:14:56 +02:00
Robert Schulze
ec7c0c3dd9
Fix ppc build 2023-08-18 12:08:59 +00:00
Kruglov Pavel
3bae63e48e
Fix special build 2023-08-18 13:56:32 +02:00
Alexander Tokmakov
5395a68baa
Merge pull request #53545 from ClickHouse/tavplubix-patch-8 
Update 00002_log_and_exception_messages_formatting.sql
 2023-08-18 14:45:58 +03:00
Alexander Tokmakov
0bd378afd9
Merge pull request #53381 from bakwc/bakwc-patch-2 
Fixed system.data_skipping_indices for MaterializedMySQL
 2023-08-18 14:22:40 +03:00
Alexander Tokmakov
59ecbd8cb1
Merge pull request #53539 from ClickHouse/pufit/fix-keeper-path-check 
Fix keeper default path check
 2023-08-18 13:53:56 +03:00
Alexander Tokmakov
9174bee76a
Update 00002_log_and_exception_messages_formatting.sql 2023-08-18 13:45:35 +03:00
Alexander Tokmakov
45157769a5
Merge pull request #53534 from ClickHouse/tavplubix-patch-8 
Update test.py
 2023-08-18 13:42:33 +03:00
Alexander Tokmakov
428810a5b2
Update setup_export_logs.sh 2023-08-18 13:37:29 +03:00
Duc Canh Le
506880fce3 Merge branch 'master' into timeout_overflow_break_subquery 2023-08-18 09:28:49 +00:00
Alexander Gololobov
d3edcaae04 Fixed for using failed_op_index before it is initialized 2023-08-18 11:19:35 +02:00
Amos Bird
1c300a31b1
Fix build 2023-08-18 15:38:48 +08:00
Amos Bird
b2dd055cdf
Use poll instead 2023-08-18 15:38:48 +08:00
Amos Bird
b6de70b042
StorageExecutable check_exit_code default to false 2023-08-18 15:38:47 +08:00
Amos Bird
69ca36bfdf
Fix black style again 2023-08-18 15:38:47 +08:00
Amos Bird
c43bf153f5
Refactor 2023-08-18 15:38:46 +08:00
Amos Bird
e49fb4f348
Fix style 2023-08-18 15:38:46 +08:00
Amos Bird
dd0c71b32a
Add error_exit_reaction 2023-08-18 15:38:46 +08:00