thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-03 21:12:28 +00:00
Code Issues Packages Projects Releases Wiki Activity
122,838 Commits 9,188 Branches 1,782 Tags 2.2 GiB
22a84551f2
Commit Graph

122838 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
selfuppen
22a84551f2 [doc/zh]: translate the last two part of sparse-primary-index chinese doc. 2023-08-20 17:49:33 +08:00
Kseniia Sumarokova
ac4f727091
Merge pull request #53273 from kssenii/impove-filecache-removal 
Improve fs cache cleanup
 2023-08-20 10:31:07 +02:00
Alexey Milovidov
e81fcfd23d
Merge pull request #53601 from ClickHouse/remove-unneeded-columns-from-logs 
Remove unneeded columns from logs
 2023-08-20 10:37:11 +03:00
Alexey Milovidov
cf0e9e5f37
Merge pull request #53604 from ClickHouse/metrics-system-database 
Add asynchronous metrics about system database
 2023-08-20 10:35:57 +03:00
Alexey Milovidov
7b7ee688db
Merge pull request #53607 from gyfis/th/lipstick-s3-uri-comments 
💄 [S3::URI] Fix comment typos around versionId
 2023-08-20 10:18:14 +03:00
gyfis
efa53f873c
💄 [S3::URI] Fix comment typos around versionId 2023-08-20 09:08:57 +02:00
Alexey Milovidov
f5ecc292cb
Merge pull request #53592 from ClickHouse/fix-02102_row_binary_with_names_and_types 
Fix test `02102_row_binary_with_names_and_types`
 2023-08-20 07:32:19 +03:00
Alexey Milovidov
392ca77578
Merge pull request #53570 from HarryLeeIBM/hlee-s390x-b64-bump 
Bump base64 head for s390x
 2023-08-20 06:37:45 +03:00
Alexey Milovidov
82166f731a Add metrics about system database 2023-08-20 05:05:54 +02:00
Alexey Milovidov
5d68f7c04d Remove unneeded columns from logs 2023-08-20 04:11:39 +02:00
Alexey Milovidov
a1d34990d4
Merge pull request #53599 from ClickHouse/remove-useless-test 
Remove useless test
 2023-08-20 04:52:38 +03:00
Alexey Milovidov
826540671c
Merge pull request #53593 from ClickHouse/better-docs-redirect 
Better documentation about redirects
 2023-08-20 04:51:45 +03:00
Alexey Milovidov
125169d9ae Remove useless test 2023-08-20 03:51:30 +02:00
Alexey Milovidov
52e4bb44f9 Fix test 02102_row_binary_with_names_and_types 
Revert the submodule
 2023-08-20 03:49:14 +02:00
Alexey Milovidov
d133cafd9c Better documentation about redirects 2023-08-20 00:38:18 +02:00
Yakov Olkhovskiy
257380df03
Merge pull request #50267 from HarryLeeIBM/hlee-s390x-proto-endian 
Fix endian issues in native protocol
 2023-08-19 11:16:24 -04:00
Kseniia Sumarokova
aa01333a63
Merge branch 'master' into impove-filecache-removal 2023-08-19 13:34:22 +04:00
Kseniia Sumarokova
c0b8d7eddf
Merge pull request #50999 from ilejn/system_kafka_consumers 
system.kafka_consumers table to monitor kafka consumers
 2023-08-19 13:31:18 +04:00
Kseniia Sumarokova
3d2fb2eced
Merge pull request #53124 from kssenii/allowed-caches-dir-for-dynamic-disks 
Allow to restrict fs caches paths (mainly useful for dynamic disks)
 2023-08-19 13:30:23 +04:00
Alexey Milovidov
c42c4046fe
Merge pull request #53573 from amosbird/fix_53454 
Proper destruction of task in ShellCommandSource
 2023-08-19 12:19:29 +03:00
robot-clickhouse-ci-1
75b748fc9a
Merge pull request #53572 from ClickHouse/fix-interpolate-precedence 
Fix: interpolate expression takes source column instead of same name aliased from select expression.
 2023-08-19 11:12:32 +02:00
Alexey Milovidov
ec0561b3f1
Merge pull request #53569 from ClickHouse/dashboard-better 
The dashboard just got better
 2023-08-19 12:11:49 +03:00
Amos Bird
052281896a
Proper destruction of task 2023-08-19 13:56:57 +08:00
kssenii
5382442da0 Merge remote-tracking branch 'upstream/master' into allowed-caches-dir-for-dynamic-disks 2023-08-19 08:44:36 +04:00
kssenii
1a6dce3dc4 Merge remote-tracking branch 'upstream/master' into impove-filecache-removal 2023-08-19 08:42:50 +04:00
Yakov Olkhovskiy
25316a29f3 result columns should take precedence to account same name aliasing 2023-08-19 02:01:10 +00:00
Michael Kolupaev
a1522e22ea
Merge pull request #53281 from Avogar/batch-small-parquet-row-groups 
Optimize reading small row groups by batching them together in Parquet
 2023-08-18 17:15:42 -07:00
HarryLeeIBM
2f27c16656 Bump base64 head for s390x 2023-08-18 17:03:51 -07:00
Alexey Milovidov
1183c18a18 Update the docs 2023-08-19 01:39:21 +02:00
Alexey Milovidov
68e694815c The dashboard just got better 2023-08-19 01:38:22 +02:00
Alexey Milovidov
00e2e3184b
Merge pull request #53439 from canhld94/timeout_overflow_break_subquery 
Fix timeout_overflow_mode when having subquery in the rhs of IN
 2023-08-19 02:24:42 +03:00
Alexey Milovidov
c22356bcaa
Merge pull request #53357 from chhetripradeep/pchhetri/add-keeper-client 
Add clickhouse-keeper-client and clickhouse-keeper-converter symlinks to clickhouse-keeper package
 2023-08-19 02:24:02 +03:00
Alexey Milovidov
17b24eb38d
Merge pull request #53490 from rschu1ze/boost-179 
Update boost to 1.79
 2023-08-19 02:12:13 +03:00
Alexey Milovidov
40d13975fb
Merge pull request #53548 from ClickHouse/vdimir/join_on_cond_sparse_fix 
Fix crash in join on sparse column
 2023-08-19 02:08:21 +03:00
Alexey Milovidov
8520a7baca
Merge pull request #53559 from azat/skip-index-set-fix 
Fix possible UB in Set skipping index for functions with incorrect args
 2023-08-19 02:06:18 +03:00
Alexey Milovidov
6fb24fe74d
Merge pull request #53560 from azat/skip-index-inverted-fix-ub 
Fix possible UB in inverted indexes (experimental feature)
 2023-08-19 02:06:04 +03:00
Alexander Tokmakov
41a39d63fe
Merge pull request #53564 from ClickHouse/tavplubix-patch-8 
Update 02443_detach_attach_partition.sh
 2023-08-19 01:08:25 +03:00
Alexander Tokmakov
d32fe2dae9
Update 02443_detach_attach_partition.sh 2023-08-18 23:17:56 +03:00
Alexey Milovidov
e6a29d7590
Update 02861_index_set_incorrect_args.sql 2023-08-18 21:48:33 +03:00
Alexey Milovidov
1e4f548cbc
Update 02862_index_inverted_incorrect_args.sql 2023-08-18 21:46:54 +03:00
Alexey Milovidov
7ad8ea78bb
Merge pull request #48990 from incfly/partition 
Add partition to the system part_log and merge table.
 2023-08-18 21:38:53 +03:00
Nikolai Kochetov
c99c3112ef
Merge pull request #53550 from ClickHouse/group_array_max_element_size 
Add server setting `aggregate_function_group_array_max_element_size`.
 2023-08-18 19:16:24 +02:00
Alexander Tokmakov
14590305ad
Merge pull request #52975 from aalexfvk/ignore_on_cluster_for_replicated_entity_queries 
Ignore ON CLUSTER clause in queries for management of replicated entities (UDF and Access)
 2023-08-18 19:29:41 +03:00
Azat Khuzhin
c78b94bed6 Fix possible UB in inverted indexes (experimental feature) 
It is possible to have incorrect types there in case of index contains
functions, add a proper check after value had been changed.

<details>

<summary>MSan report</summary>

```
==182==WARNING: MemorySanitizer: use-of-uninitialized-value
    0 0x55658547c59e in DB::MergeTreeConditionInverted::traverseASTEquals() build_docker/./src/Storages/MergeTree/MergeTreeIndexInverted.cpp:551:26
    1 0x556585475566 in DB::MergeTreeConditionInverted::traverseAtomAST() build_docker/./src/Storages/MergeTree/MergeTreeIndexInverted.cpp:438:21
    2 0x55658547e73e in DB::MergeTreeConditionInverted::MergeTreeConditionInverted()::$_1::operator()(DB::RPNBuilderTreeNode const&, DB::MergeTreeConditionInverted::RPNElement&) const build_docker/./src/Storages/MergeTree/MergeTreeIndexInverted.cpp:228:73
    9 0x556585489e35 in DB::RPNBuilder<DB::MergeTreeConditionInverted::RPNElement>::traverseTree(DB::RPNBuilderTreeNode const&) build_docker/./src/Storages/MergeTree/RPNBuilder.h:252:14
    10 0x556585489921 in DB::RPNBuilder<DB::MergeTreeConditionInverted::RPNElement>::traverseTree(DB::RPNBuilderTreeNode const&) build_docker/./src/Storages/MergeTree/RPNBuilder.h:239:21
    11 0x556585489921 in DB::RPNBuilder<DB::MergeTreeConditionInverted::RPNElement>::traverseTree(DB::RPNBuilderTreeNode const&) build_docker/./src/Storages/MergeTree/RPNBuilder.h:239:21
    12 0x5565854804e6 in DB::RPNBuilder<DB::MergeTreeConditionInverted::RPNElement>::RPNBuilder(std::__1::shared_ptr<DB::IAST> const&, std::__1::shared_ptr<DB::Context const>, DB::Block, std::__1::shared_ptr<DB::PreparedSets>, std::__1::function<bool (DB::RPNBuilderTreeNode const&, DB::MergeTreeConditionInverted::RPNElement&)> const&) build_docker/./src/Storages/MergeTree/RPNBuilder.h:218:9
    13 0x55658546fb87 in DB::MergeTreeConditionInverted::MergeTreeConditionInverted(DB::SelectQueryInfo const&, std::__1::shared_ptr<DB::Context const>, DB::Block const&, DB::GinFilterParameters const&, DB::ITokenExtractor const*) build_docker/./src/Storages/MergeTree/MergeTreeIndexInverted.cpp:223:28
    19 0x55658547ceda in DB::MergeTreeIndexInverted::createIndexCondition(DB::SelectQueryInfo const&, std::__1::shared_ptr<DB::Context const>) const build_docker/./src/Storages/MergeTree/MergeTreeIndexInverted.cpp:716:12
    20 0x556587125734 in DB::buildIndexes(std::__1::optional<DB::ReadFromMergeTree::Indexes>&, std::__1::shared_ptr<DB::ActionsDAG>, DB::MergeTreeData const&, std::__1::shared_ptr<DB::Context const> const&, DB::SelectQueryInfo const&, std::__1::shared_ptr<DB::StorageInMemoryMetadata const> const&) build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:1292:48
    21 0x556587121aaa in DB::ReadFromMergeTree::applyFilters() build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:1305:5
    22 0x55658726cf4f in DB::QueryPlanOptimizations::optimizeTreeThirdPass(DB::QueryPlan::Node&, std::__1::list<DB::QueryPlan::Node, std::__1::allocator<DB::QueryPlan::Node>>&) build_docker/./src/Processors/QueryPlan/Optimizations/optimizeTree.cpp:201:38
    23 0x5565870bc489 in DB::QueryPlan::optimize(DB::QueryPlanOptimizationSettings const&) build_docker/./src/Processors/QueryPlan/QueryPlan.cpp:485:5
    24 0x5565870b919a in DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) build_docker/./src/Processors/QueryPlan/QueryPlan.cpp:161:5
    25 0x55658206385a in DB::InterpreterSelectWithUnionQuery::execute() build_docker/./src/Interpreters/InterpreterSelectWithUnionQuery.cpp:379:31
    26 0x556582c15c50 in DB::executeQueryImpl(char const*, char const*, std::__1::shared_ptr<DB::Context>, bool, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) build_docker/./src/Interpreters/executeQuery.cpp:751:40
    27 0x556582c09681 in DB::executeQuery(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::shared_ptr<DB::Context>, bool, DB::QueryProcessingStage::Enum) build_docker/./src/Interpreters/executeQuery.cpp:1173:30
    28 0x5565860e7bd7 in DB::TCPHandler::runImpl() build_docker/./src/Server/TCPHandler.cpp:421:24
    29 0x556586126e9e in DB::TCPHandler::run() build_docker/./src/Server/TCPHandler.cpp:2057:9
```

</details>

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-08-18 18:15:17 +02:00
Nikolai Kochetov
26c7dd2ebb Fixing tidy 2023-08-18 16:08:17 +00:00
Azat Khuzhin
1e2b0c8dc4 Fix possible UB in Set skipping index for functions with incorrect args 
Usually functions are validated before index analysis, however it is
not the case for parallel replicas, and it requires additional checks
before interpreting function arguments.

<details>

<summary>stack trace</summary>

```
==172==WARNING: MemorySanitizer: use-of-uninitialized-value
    0 0x55cf82aedd6c in DB::ASTFunction* typeid_cast<DB::ASTFunction*, DB::IAST>(DB::IAST*) build_docker/./src/Common/typeid_cast.h:42:73
    1 0x55cf82aedd6c in DB::TypePromotion<DB::IAST>::CastHelper<DB::ASTFunction, false, false>::value(DB::IAST*) build_docker/./src/Common/TypePromotion.h:38:43
    2 0x55cf82aedd6c in std::__1::invoke_result<decltype(&CastHelper<DB::ASTFunction, false>::value), DB::TypePromotion<DB::IAST>::CastHelper<DB::ASTFunction, false, std::is_reference_v<DB::ASTFunction>>, DB::IAST*>::type DB::TypePromotion<DB::IAST>::as<DB::ASTFunction>() build_docker/./src/>
    3 0x55cf82aedd6c in DB::MergeTreeIndexConditionSet::operatorFromAST(std::__1::shared_ptr<DB::IAST>&) build_docker/./src/Storages/MergeTree/MergeTreeIndexSet.cpp:602:25
    4 0x55cf82ae5bc1 in DB::MergeTreeIndexConditionSet::traverseAST(std::__1::shared_ptr<DB::IAST>&) const build_docker/./src/Storages/MergeTree/MergeTreeIndexSet.cpp:547:9
    5 0x55cf82ae5de6 in DB::MergeTreeIndexConditionSet::traverseAST(std::__1::shared_ptr<DB::IAST>&) const build_docker/./src/Storages/MergeTree/MergeTreeIndexSet.cpp:552:13
    6 0x55cf82ae06db in DB::MergeTreeIndexConditionSet::MergeTreeIndexConditionSet() build_docker/.>
    ...
    12 0x55cf82aef09c in DB::MergeTreeIndexSet::createIndexCondition() const build_docker/./src/Storages/MergeTree/MergeTreeIndexSet.cpp:703:12
    13 0x55cf84951cd4 in DB::buildIndexes()
    14 0x55cf84955ed3 in DB::ReadFromMergeTree::selectRangesToReadImpl()
    15 0x55cf8494caef in DB::ReadFromMergeTree::selectRangesToRead()
    16 0x55cf82a409a9 in DB::MergeTreeDataSelectExecutor::estimateNumMarksToRead()
    17 0x55cf827f728d in DB::MergeTreeData::canUseParallelReplicasBasedOnPKAnalysis()
    18 0x55cf827f627e in DB::MergeTreeData::getQueryProcessingStage()
    19 0x55cf7f2f4969 in DB::InterpreterSelectQuery::getSampleBlockImpl() build_docker/./src/Interpreters/InterpreterSelectQuery.cpp:937:31
    20 0x55cf7f2daa00 in DB::InterpreterSelectQuery::InterpreterSelectQuery()
    24 0x55cf7f520b98 in DB::InterpreterSelectWithUnionQuery::buildCurrentChildInterpreter()
    25 0x55cf7f51b6cd in DB::InterpreterSelectWithUnionQuery::InterpreterSelectWithUnionQuery()
    27 0x55cf7f1d4ea9 in DB::InterpreterFactory::get(std::__1::shared_ptr<DB::IAST>&, std::__1::shared_ptr<DB::Context>, DB::SelectQueryOptions const&) build_docker/./src/Interpreters/InterpreterFactory.cpp:162:16
    28 0x55cf8012e485 in DB::executeQueryImpl(char const*, char const*, std::__1::shared_ptr<DB::Context>, bool, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) build_docker/./src/Interpreters/executeQuery.cpp:1032:31
    29 0x55cf80121bc1 in DB::executeQuery(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::shared_ptr<DB::Context>, bool, DB::QueryProcessingStage::Enum) build_docker/./src/Interpreters/executeQuery.cpp:1229:30
    30 0x55cf8389295f in DB::TCPHandler::runImpl() build_docker/./src/Server/TCPHandler.cpp:424:24
    31 0x55cf838d7dfb in DB::TCPHandler::run() build_docker/./src/Server/TCPHandler.cpp:2050:9
```

</details>

CI: https://s3.amazonaws.com/clickhouse-test-reports/53214/d99b10c340909ed4ee2e6edf0921e8a2f8561b0d/fuzzer_astfuzzermsan/report.html
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-08-18 18:02:46 +02:00
Alexander Tokmakov
334e014c2c Merge branch 'master' into ignore_on_cluster_for_replicated_entity_queries 2023-08-18 17:48:37 +02:00
Yakov Olkhovskiy
84cb358c14
Merge pull request #43210 from amosbird/storage_executable_stderr 
Add different reactions when external command stderr has data
 2023-08-18 11:17:46 -04:00
Alexander Tokmakov
a6684f7a37
Merge pull request #53484 from azat/rmt-queue-cleanup 
Remove outdated code in ReplicatedMergeTreeQueue::initialize()
 2023-08-18 18:11:33 +03:00
Nikolay Degterinsky
4c606590a1
Fix tiny thing in Replicated database (#53496) 
* Fix tiny thing in Replicated database

* empty commit

---------

Co-authored-by: Alexander Tokmakov <tavplubix@clickhouse.com>
 2023-08-18 17:29:24 +03:00