Commit Graph

123607 Commits

Author SHA1 Message Date
Pradeep Chhetri
ce42e42d01 Fix for flaky test_ssl_cert_authentication 2023-08-19 21:01:24 +08:00
Maksim Kita
8a22c8a13b Test 2023-08-19 15:24:12 +03:00
Maksim Kita
10db751c7b Test 2023-08-19 14:54:45 +03:00
Maksim Kita
9772beeffc Test 2023-08-19 14:54:45 +03:00
Maksim Kita
806519acb3 Test 2023-08-19 14:54:45 +03:00
Maksim Kita
d9c4b110ce LLVM AARCH64 cross compilation fix 2023-08-19 14:54:45 +03:00
Maksim Kita
255f171cf9 AARCH64 enable JIT compilation 2023-08-19 14:54:45 +03:00
Kseniia Sumarokova
aa01333a63
Merge branch 'master' into impove-filecache-removal 2023-08-19 13:34:22 +04:00
Kseniia Sumarokova
aeebedfa0d
Merge branch 'master' into system-sync-cache 2023-08-19 13:32:48 +04:00
taiyang-li
9866d2727e change as request 2023-08-19 17:31:19 +08:00
Kseniia Sumarokova
c0b8d7eddf
Merge pull request #50999 from ilejn/system_kafka_consumers
system.kafka_consumers table to monitor kafka consumers
2023-08-19 13:31:18 +04:00
Kseniia Sumarokova
3d2fb2eced
Merge pull request #53124 from kssenii/allowed-caches-dir-for-dynamic-disks
Allow to restrict fs caches paths (mainly useful for dynamic disks)
2023-08-19 13:30:23 +04:00
Alexey Milovidov
c42c4046fe
Merge pull request #53573 from amosbird/fix_53454
Proper destruction of task in ShellCommandSource
2023-08-19 12:19:29 +03:00
robot-clickhouse-ci-1
75b748fc9a
Merge pull request #53572 from ClickHouse/fix-interpolate-precedence
Fix: interpolate expression takes source column instead of same name aliased from select expression.
2023-08-19 11:12:32 +02:00
Alexey Milovidov
ec0561b3f1
Merge pull request #53569 from ClickHouse/dashboard-better
The dashboard just got better
2023-08-19 12:11:49 +03:00
Amos Bird
052281896a
Proper destruction of task 2023-08-19 13:56:57 +08:00
kssenii
5382442da0 Merge remote-tracking branch 'upstream/master' into allowed-caches-dir-for-dynamic-disks 2023-08-19 08:44:36 +04:00
kssenii
247abe0a2a Merge remote-tracking branch 'upstream/master' into system-sync-cache 2023-08-19 08:43:53 +04:00
kssenii
1a6dce3dc4 Merge remote-tracking branch 'upstream/master' into impove-filecache-removal 2023-08-19 08:42:50 +04:00
Yakov Olkhovskiy
25316a29f3 result columns should take precedence to account same name aliasing 2023-08-19 02:01:10 +00:00
Michael Kolupaev
a1522e22ea
Merge pull request #53281 from Avogar/batch-small-parquet-row-groups
Optimize reading small row groups by batching them together in Parquet
2023-08-18 17:15:42 -07:00
HarryLeeIBM
2f27c16656 Bump base64 head for s390x 2023-08-18 17:03:51 -07:00
Alexey Milovidov
1183c18a18 Update the docs 2023-08-19 01:39:21 +02:00
Robert Schulze
7ade17516f
Update 02354_usearch_index.sql 2023-08-19 01:39:09 +02:00
Alexey Milovidov
68e694815c The dashboard just got better 2023-08-19 01:38:22 +02:00
Alexey Milovidov
00e2e3184b
Merge pull request #53439 from canhld94/timeout_overflow_break_subquery
Fix timeout_overflow_mode when having subquery in the rhs of IN
2023-08-19 02:24:42 +03:00
Alexey Milovidov
c22356bcaa
Merge pull request #53357 from chhetripradeep/pchhetri/add-keeper-client
Add clickhouse-keeper-client and clickhouse-keeper-converter symlinks to clickhouse-keeper package
2023-08-19 02:24:02 +03:00
Alexey Milovidov
17b24eb38d
Merge pull request #53490 from rschu1ze/boost-179
Update boost to 1.79
2023-08-19 02:12:13 +03:00
Alexey Milovidov
40d13975fb
Merge pull request #53548 from ClickHouse/vdimir/join_on_cond_sparse_fix
Fix crash in join on sparse column
2023-08-19 02:08:21 +03:00
Alexey Milovidov
8520a7baca
Merge pull request #53559 from azat/skip-index-set-fix
Fix possible UB in Set skipping index for functions with incorrect args
2023-08-19 02:06:18 +03:00
Alexey Milovidov
6fb24fe74d
Merge pull request #53560 from azat/skip-index-inverted-fix-ub
Fix possible UB in inverted indexes (experimental feature)
2023-08-19 02:06:04 +03:00
Igor Nikonov
aebda10f7c Merge remote-tracking branch 'origin/master' into fix-parallel-replicas-multiply-result 2023-08-18 23:00:58 +00:00
Alexander Tokmakov
41a39d63fe
Merge pull request #53564 from ClickHouse/tavplubix-patch-8
Update 02443_detach_attach_partition.sh
2023-08-19 01:08:25 +03:00
kothiga
a420795416
Trailing whitespace fix. 2023-08-18 14:08:02 -07:00
kothiga
f33c585bc5
Addressing feedback. 2023-08-18 13:50:31 -07:00
Robert Schulze
eaf2fb837c
Update to 1.21-clickhouse 2023-08-18 20:18:30 +00:00
Alexander Tokmakov
d32fe2dae9
Update 02443_detach_attach_partition.sh 2023-08-18 23:17:56 +03:00
Robert Schulze
cf0873aaef
Fix build 2023-08-18 20:14:22 +00:00
Alexey Milovidov
e6a29d7590
Update 02861_index_set_incorrect_args.sql 2023-08-18 21:48:33 +03:00
Alexey Milovidov
1e4f548cbc
Update 02862_index_inverted_incorrect_args.sql 2023-08-18 21:46:54 +03:00
Alexey Milovidov
7ad8ea78bb
Merge pull request #48990 from incfly/partition
Add partition to the system part_log and merge table.
2023-08-18 21:38:53 +03:00
kothiga
b905703909
Correct to-from pairing for transformEndianness. 2023-08-18 11:25:08 -07:00
avogar
8e445b5461 Fixes 2023-08-18 17:49:40 +00:00
Nikolai Kochetov
c99c3112ef
Merge pull request #53550 from ClickHouse/group_array_max_element_size
Add server setting `aggregate_function_group_array_max_element_size`.
2023-08-18 19:16:24 +02:00
Alexander Tokmakov
14590305ad
Merge pull request #52975 from aalexfvk/ignore_on_cluster_for_replicated_entity_queries
Ignore ON CLUSTER clause in queries for management of replicated entities (UDF and Access)
2023-08-18 19:29:41 +03:00
Azat Khuzhin
c78b94bed6 Fix possible UB in inverted indexes (experimental feature)
It is possible to have incorrect types there in case of index contains
functions, add a proper check after value had been changed.

<details>

<summary>MSan report</summary>

```
==182==WARNING: MemorySanitizer: use-of-uninitialized-value
    0 0x55658547c59e in DB::MergeTreeConditionInverted::traverseASTEquals() build_docker/./src/Storages/MergeTree/MergeTreeIndexInverted.cpp:551:26
    1 0x556585475566 in DB::MergeTreeConditionInverted::traverseAtomAST() build_docker/./src/Storages/MergeTree/MergeTreeIndexInverted.cpp:438:21
    2 0x55658547e73e in DB::MergeTreeConditionInverted::MergeTreeConditionInverted()::$_1::operator()(DB::RPNBuilderTreeNode const&, DB::MergeTreeConditionInverted::RPNElement&) const build_docker/./src/Storages/MergeTree/MergeTreeIndexInverted.cpp:228:73
    9 0x556585489e35 in DB::RPNBuilder<DB::MergeTreeConditionInverted::RPNElement>::traverseTree(DB::RPNBuilderTreeNode const&) build_docker/./src/Storages/MergeTree/RPNBuilder.h:252:14
    10 0x556585489921 in DB::RPNBuilder<DB::MergeTreeConditionInverted::RPNElement>::traverseTree(DB::RPNBuilderTreeNode const&) build_docker/./src/Storages/MergeTree/RPNBuilder.h:239:21
    11 0x556585489921 in DB::RPNBuilder<DB::MergeTreeConditionInverted::RPNElement>::traverseTree(DB::RPNBuilderTreeNode const&) build_docker/./src/Storages/MergeTree/RPNBuilder.h:239:21
    12 0x5565854804e6 in DB::RPNBuilder<DB::MergeTreeConditionInverted::RPNElement>::RPNBuilder(std::__1::shared_ptr<DB::IAST> const&, std::__1::shared_ptr<DB::Context const>, DB::Block, std::__1::shared_ptr<DB::PreparedSets>, std::__1::function<bool (DB::RPNBuilderTreeNode const&, DB::MergeTreeConditionInverted::RPNElement&)> const&) build_docker/./src/Storages/MergeTree/RPNBuilder.h:218:9
    13 0x55658546fb87 in DB::MergeTreeConditionInverted::MergeTreeConditionInverted(DB::SelectQueryInfo const&, std::__1::shared_ptr<DB::Context const>, DB::Block const&, DB::GinFilterParameters const&, DB::ITokenExtractor const*) build_docker/./src/Storages/MergeTree/MergeTreeIndexInverted.cpp:223:28
    19 0x55658547ceda in DB::MergeTreeIndexInverted::createIndexCondition(DB::SelectQueryInfo const&, std::__1::shared_ptr<DB::Context const>) const build_docker/./src/Storages/MergeTree/MergeTreeIndexInverted.cpp:716:12
    20 0x556587125734 in DB::buildIndexes(std::__1::optional<DB::ReadFromMergeTree::Indexes>&, std::__1::shared_ptr<DB::ActionsDAG>, DB::MergeTreeData const&, std::__1::shared_ptr<DB::Context const> const&, DB::SelectQueryInfo const&, std::__1::shared_ptr<DB::StorageInMemoryMetadata const> const&) build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:1292:48
    21 0x556587121aaa in DB::ReadFromMergeTree::applyFilters() build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:1305:5
    22 0x55658726cf4f in DB::QueryPlanOptimizations::optimizeTreeThirdPass(DB::QueryPlan::Node&, std::__1::list<DB::QueryPlan::Node, std::__1::allocator<DB::QueryPlan::Node>>&) build_docker/./src/Processors/QueryPlan/Optimizations/optimizeTree.cpp:201:38
    23 0x5565870bc489 in DB::QueryPlan::optimize(DB::QueryPlanOptimizationSettings const&) build_docker/./src/Processors/QueryPlan/QueryPlan.cpp:485:5
    24 0x5565870b919a in DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) build_docker/./src/Processors/QueryPlan/QueryPlan.cpp:161:5
    25 0x55658206385a in DB::InterpreterSelectWithUnionQuery::execute() build_docker/./src/Interpreters/InterpreterSelectWithUnionQuery.cpp:379:31
    26 0x556582c15c50 in DB::executeQueryImpl(char const*, char const*, std::__1::shared_ptr<DB::Context>, bool, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) build_docker/./src/Interpreters/executeQuery.cpp:751:40
    27 0x556582c09681 in DB::executeQuery(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::shared_ptr<DB::Context>, bool, DB::QueryProcessingStage::Enum) build_docker/./src/Interpreters/executeQuery.cpp:1173:30
    28 0x5565860e7bd7 in DB::TCPHandler::runImpl() build_docker/./src/Server/TCPHandler.cpp:421:24
    29 0x556586126e9e in DB::TCPHandler::run() build_docker/./src/Server/TCPHandler.cpp:2057:9
```

</details>

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-08-18 18:15:17 +02:00
Nikolai Kochetov
26c7dd2ebb Fixing tidy 2023-08-18 16:08:17 +00:00
Azat Khuzhin
1e2b0c8dc4 Fix possible UB in Set skipping index for functions with incorrect args
Usually functions are validated before index analysis, however it is
not the case for parallel replicas, and it requires additional checks
before interpreting function arguments.

<details>

<summary>stack trace</summary>

```
==172==WARNING: MemorySanitizer: use-of-uninitialized-value
    0 0x55cf82aedd6c in DB::ASTFunction* typeid_cast<DB::ASTFunction*, DB::IAST>(DB::IAST*) build_docker/./src/Common/typeid_cast.h:42:73
    1 0x55cf82aedd6c in DB::TypePromotion<DB::IAST>::CastHelper<DB::ASTFunction, false, false>::value(DB::IAST*) build_docker/./src/Common/TypePromotion.h:38:43
    2 0x55cf82aedd6c in std::__1::invoke_result<decltype(&CastHelper<DB::ASTFunction, false>::value), DB::TypePromotion<DB::IAST>::CastHelper<DB::ASTFunction, false, std::is_reference_v<DB::ASTFunction>>, DB::IAST*>::type DB::TypePromotion<DB::IAST>::as<DB::ASTFunction>() build_docker/./src/>
    3 0x55cf82aedd6c in DB::MergeTreeIndexConditionSet::operatorFromAST(std::__1::shared_ptr<DB::IAST>&) build_docker/./src/Storages/MergeTree/MergeTreeIndexSet.cpp:602:25
    4 0x55cf82ae5bc1 in DB::MergeTreeIndexConditionSet::traverseAST(std::__1::shared_ptr<DB::IAST>&) const build_docker/./src/Storages/MergeTree/MergeTreeIndexSet.cpp:547:9
    5 0x55cf82ae5de6 in DB::MergeTreeIndexConditionSet::traverseAST(std::__1::shared_ptr<DB::IAST>&) const build_docker/./src/Storages/MergeTree/MergeTreeIndexSet.cpp:552:13
    6 0x55cf82ae06db in DB::MergeTreeIndexConditionSet::MergeTreeIndexConditionSet() build_docker/.>
    ...
    12 0x55cf82aef09c in DB::MergeTreeIndexSet::createIndexCondition() const build_docker/./src/Storages/MergeTree/MergeTreeIndexSet.cpp:703:12
    13 0x55cf84951cd4 in DB::buildIndexes()
    14 0x55cf84955ed3 in DB::ReadFromMergeTree::selectRangesToReadImpl()
    15 0x55cf8494caef in DB::ReadFromMergeTree::selectRangesToRead()
    16 0x55cf82a409a9 in DB::MergeTreeDataSelectExecutor::estimateNumMarksToRead()
    17 0x55cf827f728d in DB::MergeTreeData::canUseParallelReplicasBasedOnPKAnalysis()
    18 0x55cf827f627e in DB::MergeTreeData::getQueryProcessingStage()
    19 0x55cf7f2f4969 in DB::InterpreterSelectQuery::getSampleBlockImpl() build_docker/./src/Interpreters/InterpreterSelectQuery.cpp:937:31
    20 0x55cf7f2daa00 in DB::InterpreterSelectQuery::InterpreterSelectQuery()
    24 0x55cf7f520b98 in DB::InterpreterSelectWithUnionQuery::buildCurrentChildInterpreter()
    25 0x55cf7f51b6cd in DB::InterpreterSelectWithUnionQuery::InterpreterSelectWithUnionQuery()
    27 0x55cf7f1d4ea9 in DB::InterpreterFactory::get(std::__1::shared_ptr<DB::IAST>&, std::__1::shared_ptr<DB::Context>, DB::SelectQueryOptions const&) build_docker/./src/Interpreters/InterpreterFactory.cpp:162:16
    28 0x55cf8012e485 in DB::executeQueryImpl(char const*, char const*, std::__1::shared_ptr<DB::Context>, bool, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) build_docker/./src/Interpreters/executeQuery.cpp:1032:31
    29 0x55cf80121bc1 in DB::executeQuery(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::shared_ptr<DB::Context>, bool, DB::QueryProcessingStage::Enum) build_docker/./src/Interpreters/executeQuery.cpp:1229:30
    30 0x55cf8389295f in DB::TCPHandler::runImpl() build_docker/./src/Server/TCPHandler.cpp:424:24
    31 0x55cf838d7dfb in DB::TCPHandler::run() build_docker/./src/Server/TCPHandler.cpp:2050:9
```

</details>

CI: https://s3.amazonaws.com/clickhouse-test-reports/53214/d99b10c340909ed4ee2e6edf0921e8a2f8561b0d/fuzzer_astfuzzermsan/report.html
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-08-18 18:02:46 +02:00
Alexander Tokmakov
334e014c2c Merge branch 'master' into ignore_on_cluster_for_replicated_entity_queries 2023-08-18 17:48:37 +02:00
Robert Schulze
87824bea74
Bump openldap to a LTS branch (v2.5.16) 2023-08-18 15:47:02 +00:00