thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-16 11:22:12 +00:00
Code Issues Packages Projects Releases Wiki Activity
124,942 Commits 9,338 Branches 1,783 Tags 2.1 GiB
c2a593baf9
Commit Graph

14358 Commits

Author SHA1 Message Date
Duc Canh Le
2c07065f81 increase the default value 
Signed-off-by: Duc Canh Le <duccanh.le@ahrefs.com>
 2023-08-21 11:04:24 +00:00
Raúl Marín
f2d3c9c281 Merge remote-tracking branch 'blessed/master' into better_zk_retries_on_insert 2023-08-21 12:14:31 +02:00
Robert Schulze
066ec559ad
Do not reset Annoy index during build-up with > 1 mark 2023-08-21 10:05:43 +00:00
王智博
8bb0f18019 bad size of mark skip idx 2023-08-20 22:57:53 +08:00
Kseniia Sumarokova
404fb0588a
Merge branch 'master' into fix-cache-logical-error 2023-08-20 16:28:13 +02:00
irenjj
a9b65ee929 Add Alias for system.clusters. 2023-08-20 14:47:07 +08:00
irenjj
71459c6d65 Revert 2023-08-20 14:35:59 +08:00
irenjj
e64a1fc26d Add column 'name' to system.clusters as an alias to cluster. 2023-08-20 11:52:14 +08:00
Igor Nikonov
a69ea02418 Merge remote-tracking branch 'origin/master' into fix-parallel-replicas-multiply-result 2023-08-19 21:43:25 +00:00
Kseniia Sumarokova
aeebedfa0d
Merge branch 'master' into system-sync-cache 2023-08-19 13:32:48 +04:00
Kseniia Sumarokova
c0b8d7eddf
Merge pull request #50999 from ilejn/system_kafka_consumers 
system.kafka_consumers table to monitor kafka consumers
 2023-08-19 13:31:18 +04:00
kssenii
247abe0a2a Merge remote-tracking branch 'upstream/master' into system-sync-cache 2023-08-19 08:43:53 +04:00
Alexey Milovidov
8520a7baca
Merge pull request #53559 from azat/skip-index-set-fix 
Fix possible UB in Set skipping index for functions with incorrect args
 2023-08-19 02:06:18 +03:00
Alexey Milovidov
6fb24fe74d
Merge pull request #53560 from azat/skip-index-inverted-fix-ub 
Fix possible UB in inverted indexes (experimental feature)
 2023-08-19 02:06:04 +03:00
Igor Nikonov
aebda10f7c Merge remote-tracking branch 'origin/master' into fix-parallel-replicas-multiply-result 2023-08-18 23:00:58 +00:00
Alexey Milovidov
7ad8ea78bb
Merge pull request #48990 from incfly/partition 
Add partition to the system part_log and merge table.
 2023-08-18 21:38:53 +03:00
avogar
8e445b5461 Fixes 2023-08-18 17:49:40 +00:00
Azat Khuzhin
c78b94bed6 Fix possible UB in inverted indexes (experimental feature) 
It is possible to have incorrect types there in case of index contains
functions, add a proper check after value had been changed.

<details>

<summary>MSan report</summary>

```
==182==WARNING: MemorySanitizer: use-of-uninitialized-value
    0 0x55658547c59e in DB::MergeTreeConditionInverted::traverseASTEquals() build_docker/./src/Storages/MergeTree/MergeTreeIndexInverted.cpp:551:26
    1 0x556585475566 in DB::MergeTreeConditionInverted::traverseAtomAST() build_docker/./src/Storages/MergeTree/MergeTreeIndexInverted.cpp:438:21
    2 0x55658547e73e in DB::MergeTreeConditionInverted::MergeTreeConditionInverted()::$_1::operator()(DB::RPNBuilderTreeNode const&, DB::MergeTreeConditionInverted::RPNElement&) const build_docker/./src/Storages/MergeTree/MergeTreeIndexInverted.cpp:228:73
    9 0x556585489e35 in DB::RPNBuilder<DB::MergeTreeConditionInverted::RPNElement>::traverseTree(DB::RPNBuilderTreeNode const&) build_docker/./src/Storages/MergeTree/RPNBuilder.h:252:14
    10 0x556585489921 in DB::RPNBuilder<DB::MergeTreeConditionInverted::RPNElement>::traverseTree(DB::RPNBuilderTreeNode const&) build_docker/./src/Storages/MergeTree/RPNBuilder.h:239:21
    11 0x556585489921 in DB::RPNBuilder<DB::MergeTreeConditionInverted::RPNElement>::traverseTree(DB::RPNBuilderTreeNode const&) build_docker/./src/Storages/MergeTree/RPNBuilder.h:239:21
    12 0x5565854804e6 in DB::RPNBuilder<DB::MergeTreeConditionInverted::RPNElement>::RPNBuilder(std::__1::shared_ptr<DB::IAST> const&, std::__1::shared_ptr<DB::Context const>, DB::Block, std::__1::shared_ptr<DB::PreparedSets>, std::__1::function<bool (DB::RPNBuilderTreeNode const&, DB::MergeTreeConditionInverted::RPNElement&)> const&) build_docker/./src/Storages/MergeTree/RPNBuilder.h:218:9
    13 0x55658546fb87 in DB::MergeTreeConditionInverted::MergeTreeConditionInverted(DB::SelectQueryInfo const&, std::__1::shared_ptr<DB::Context const>, DB::Block const&, DB::GinFilterParameters const&, DB::ITokenExtractor const*) build_docker/./src/Storages/MergeTree/MergeTreeIndexInverted.cpp:223:28
    19 0x55658547ceda in DB::MergeTreeIndexInverted::createIndexCondition(DB::SelectQueryInfo const&, std::__1::shared_ptr<DB::Context const>) const build_docker/./src/Storages/MergeTree/MergeTreeIndexInverted.cpp:716:12
    20 0x556587125734 in DB::buildIndexes(std::__1::optional<DB::ReadFromMergeTree::Indexes>&, std::__1::shared_ptr<DB::ActionsDAG>, DB::MergeTreeData const&, std::__1::shared_ptr<DB::Context const> const&, DB::SelectQueryInfo const&, std::__1::shared_ptr<DB::StorageInMemoryMetadata const> const&) build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:1292:48
    21 0x556587121aaa in DB::ReadFromMergeTree::applyFilters() build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:1305:5
    22 0x55658726cf4f in DB::QueryPlanOptimizations::optimizeTreeThirdPass(DB::QueryPlan::Node&, std::__1::list<DB::QueryPlan::Node, std::__1::allocator<DB::QueryPlan::Node>>&) build_docker/./src/Processors/QueryPlan/Optimizations/optimizeTree.cpp:201:38
    23 0x5565870bc489 in DB::QueryPlan::optimize(DB::QueryPlanOptimizationSettings const&) build_docker/./src/Processors/QueryPlan/QueryPlan.cpp:485:5
    24 0x5565870b919a in DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) build_docker/./src/Processors/QueryPlan/QueryPlan.cpp:161:5
    25 0x55658206385a in DB::InterpreterSelectWithUnionQuery::execute() build_docker/./src/Interpreters/InterpreterSelectWithUnionQuery.cpp:379:31
    26 0x556582c15c50 in DB::executeQueryImpl(char const*, char const*, std::__1::shared_ptr<DB::Context>, bool, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) build_docker/./src/Interpreters/executeQuery.cpp:751:40
    27 0x556582c09681 in DB::executeQuery(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::shared_ptr<DB::Context>, bool, DB::QueryProcessingStage::Enum) build_docker/./src/Interpreters/executeQuery.cpp:1173:30
    28 0x5565860e7bd7 in DB::TCPHandler::runImpl() build_docker/./src/Server/TCPHandler.cpp:421:24
    29 0x556586126e9e in DB::TCPHandler::run() build_docker/./src/Server/TCPHandler.cpp:2057:9
```

</details>

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-08-18 18:15:17 +02:00
Azat Khuzhin
1e2b0c8dc4 Fix possible UB in Set skipping index for functions with incorrect args 
Usually functions are validated before index analysis, however it is
not the case for parallel replicas, and it requires additional checks
before interpreting function arguments.

<details>

<summary>stack trace</summary>

```
==172==WARNING: MemorySanitizer: use-of-uninitialized-value
    0 0x55cf82aedd6c in DB::ASTFunction* typeid_cast<DB::ASTFunction*, DB::IAST>(DB::IAST*) build_docker/./src/Common/typeid_cast.h:42:73
    1 0x55cf82aedd6c in DB::TypePromotion<DB::IAST>::CastHelper<DB::ASTFunction, false, false>::value(DB::IAST*) build_docker/./src/Common/TypePromotion.h:38:43
    2 0x55cf82aedd6c in std::__1::invoke_result<decltype(&CastHelper<DB::ASTFunction, false>::value), DB::TypePromotion<DB::IAST>::CastHelper<DB::ASTFunction, false, std::is_reference_v<DB::ASTFunction>>, DB::IAST*>::type DB::TypePromotion<DB::IAST>::as<DB::ASTFunction>() build_docker/./src/>
    3 0x55cf82aedd6c in DB::MergeTreeIndexConditionSet::operatorFromAST(std::__1::shared_ptr<DB::IAST>&) build_docker/./src/Storages/MergeTree/MergeTreeIndexSet.cpp:602:25
    4 0x55cf82ae5bc1 in DB::MergeTreeIndexConditionSet::traverseAST(std::__1::shared_ptr<DB::IAST>&) const build_docker/./src/Storages/MergeTree/MergeTreeIndexSet.cpp:547:9
    5 0x55cf82ae5de6 in DB::MergeTreeIndexConditionSet::traverseAST(std::__1::shared_ptr<DB::IAST>&) const build_docker/./src/Storages/MergeTree/MergeTreeIndexSet.cpp:552:13
    6 0x55cf82ae06db in DB::MergeTreeIndexConditionSet::MergeTreeIndexConditionSet() build_docker/.>
    ...
    12 0x55cf82aef09c in DB::MergeTreeIndexSet::createIndexCondition() const build_docker/./src/Storages/MergeTree/MergeTreeIndexSet.cpp:703:12
    13 0x55cf84951cd4 in DB::buildIndexes()
    14 0x55cf84955ed3 in DB::ReadFromMergeTree::selectRangesToReadImpl()
    15 0x55cf8494caef in DB::ReadFromMergeTree::selectRangesToRead()
    16 0x55cf82a409a9 in DB::MergeTreeDataSelectExecutor::estimateNumMarksToRead()
    17 0x55cf827f728d in DB::MergeTreeData::canUseParallelReplicasBasedOnPKAnalysis()
    18 0x55cf827f627e in DB::MergeTreeData::getQueryProcessingStage()
    19 0x55cf7f2f4969 in DB::InterpreterSelectQuery::getSampleBlockImpl() build_docker/./src/Interpreters/InterpreterSelectQuery.cpp:937:31
    20 0x55cf7f2daa00 in DB::InterpreterSelectQuery::InterpreterSelectQuery()
    24 0x55cf7f520b98 in DB::InterpreterSelectWithUnionQuery::buildCurrentChildInterpreter()
    25 0x55cf7f51b6cd in DB::InterpreterSelectWithUnionQuery::InterpreterSelectWithUnionQuery()
    27 0x55cf7f1d4ea9 in DB::InterpreterFactory::get(std::__1::shared_ptr<DB::IAST>&, std::__1::shared_ptr<DB::Context>, DB::SelectQueryOptions const&) build_docker/./src/Interpreters/InterpreterFactory.cpp:162:16
    28 0x55cf8012e485 in DB::executeQueryImpl(char const*, char const*, std::__1::shared_ptr<DB::Context>, bool, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) build_docker/./src/Interpreters/executeQuery.cpp:1032:31
    29 0x55cf80121bc1 in DB::executeQuery(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::shared_ptr<DB::Context>, bool, DB::QueryProcessingStage::Enum) build_docker/./src/Interpreters/executeQuery.cpp:1229:30
    30 0x55cf8389295f in DB::TCPHandler::runImpl() build_docker/./src/Server/TCPHandler.cpp:424:24
    31 0x55cf838d7dfb in DB::TCPHandler::run() build_docker/./src/Server/TCPHandler.cpp:2050:9
```

</details>

CI: https://s3.amazonaws.com/clickhouse-test-reports/53214/d99b10c340909ed4ee2e6edf0921e8a2f8561b0d/fuzzer_astfuzzermsan/report.html
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-08-18 18:02:46 +02:00
Austin Kothig
6b42975d33
Change BE-UUID to work the same as LE-UUID. Included high and low getters to provide cleaner code when accessing undertype. 2023-08-18 08:19:46 -07:00
Yakov Olkhovskiy
84cb358c14
Merge pull request #43210 from amosbird/storage_executable_stderr 
Add different reactions when external command stderr has data
 2023-08-18 11:17:46 -04:00
Alexander Tokmakov
a6684f7a37
Merge pull request #53484 from azat/rmt-queue-cleanup 
Remove outdated code in ReplicatedMergeTreeQueue::initialize()
 2023-08-18 18:11:33 +03:00
avogar
2d4167c612 FRemove unneded changes 2023-08-18 14:50:07 +00:00
avogar
aac5131859 Fix for no_more_keys 2023-08-18 13:21:16 +00:00
avogar
7fa72f5cdf Optimize group by constant keys 2023-08-18 12:47:26 +00:00
Alexander Gololobov
50061b9cb4
Merge pull request #53452 from ClickHouse/such_much_watch 
Deduplicate same watch callbacks registered multiple times
 2023-08-18 14:43:03 +02:00
Alexander Tokmakov
0bd378afd9
Merge pull request #53381 from bakwc/bakwc-patch-2 
Fixed system.data_skipping_indices for MaterializedMySQL
 2023-08-18 14:22:40 +03:00
Duc Canh Le
2d353f8c43 Merge branch 'master' into limit_concurent_background_optimize 2023-08-18 09:25:27 +00:00
Amos Bird
b6de70b042
StorageExecutable check_exit_code default to false 2023-08-18 15:38:47 +08:00
Amos Bird
c43bf153f5
Refactor 2023-08-18 15:38:46 +08:00
Amos Bird
dd0c71b32a
Add error_exit_reaction 2023-08-18 15:38:46 +08:00
Amos Bird
476f3cedc1
Various reactions when executable stderr has data 2023-08-18 15:38:45 +08:00
Alexey Milovidov
a882bf1bb8
Merge pull request #53495 from ClickHouse/i_hate_materialized_views 
Fix a race condition between RESTART REPLICAS and DROP DATABASE
 2023-08-18 06:40:04 +03:00
Igor Nikonov
06c98956b3 Merge remote-tracking branch 'origin/master' into fix-parallel-replicas-multiply-result 2023-08-17 20:42:50 +00:00
Jianfei Hu
0c81546960 remove the comments. 
Signed-off-by: Jianfei Hu <hujianfei258@gmail.com>
 2023-08-17 20:13:33 +00:00
pufit
a52249872e
Merge pull request #53313 from jorisgio/joris/max_threads_for_indexes 
add max_threads_for_indexes settings
 2023-08-17 13:59:53 -04:00
avogar
60b0b88d50 Clean up 2023-08-17 16:59:57 +00:00
avogar
4c32097df3 Use filter by file/path before reading in url/file/hdfs table functions, reduce code duplication 2023-08-17 16:54:43 +00:00
Alexander Gololobov
54e2be4e32 Use WatchCallbackPtr 2023-08-17 17:32:04 +02:00
Alexander Tokmakov
89c365803f
Merge pull request #53424 from ClickHouse/exception_message_patterns6 
Less exceptions with runtime format string
 2023-08-17 17:58:04 +03:00
Igor Nikonov
3e9ae1751e Merge remote-tracking branch 'origin/master' into fix-parallel-replicas-multiply-result 2023-08-17 14:45:37 +00:00
Filipp Ozinov
ce87451b66
Merge branch 'ClickHouse:master' into bakwc-patch-2 2023-08-17 18:37:00 +04:00
Alexander Tokmakov
72b52250ea
Merge pull request #53501 from azat/dist-flush_on_detach 
Add ability to turn off flush of Distributed on DETACH/DROP/server shutdown
 2023-08-17 16:56:04 +03:00
Azat Khuzhin
fcb66379d1 Remove outdated code in ReplicatedMergeTreeQueue::initialize() 
Initially added in #28817
But got cleaned up in #41981

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-08-17 13:45:26 +02:00
Robert Schulze
ec7daed52f
Merge pull request #50276 from ClibMouse/feature/mergetree-checksum-big-endian-support 
MergeTree & SipHash checksum big-endian support
 2023-08-17 13:27:18 +02:00
Robert Schulze
42644bfbf5
Rename file to MergeTreeIndexUSearch.h/cpp 2023-08-17 09:54:12 +00:00
Robert Schulze
f420006e09
Make the code consistent with Annoy index and fix build 2023-08-17 09:42:13 +00:00
Robert Schulze
456a02da1b
Merge remote-tracking branch 'rschu1ze/master' 2023-08-17 09:30:27 +00:00
Robert Schulze
1215a23de6
Merge pull request #53422 from rschu1ze/consistency-fixes-for-caches 
Refactorings for configuration of in-memory caches
 2023-08-17 11:08:32 +02:00
Mingliang Pan
2db0b4f52a
Update StorageEmbeddedRocksDB.cpp 
add drop function to close file before drop table
 2023-08-17 15:14:14 +08:00
Mingliang Pan
349f3b9c02
Update StorageEmbeddedRocksDB.h 
add drop method to close file before drop table
 2023-08-17 15:12:42 +08:00
Azat Khuzhin
17ca2661a1 Add ability to turn off flush of Distributed on DETACH/DROP/server shutdown 
Sometimes you can have tons of data there, i.e. few TiBs, and sending
them on server shutdown does not looks sane (maybe there is a bug and
you need to update/restart to fix flushing).

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-08-17 08:58:06 +02:00
Igor Nikonov
49f7b4acd6 Merge remote-tracking branch 'origin/master' into fix-parallel-replicas-multiply-result 2023-08-16 23:31:49 +00:00
Igor Nikonov
21ef1f1d1c Remove unnecessary Scalars usage for cluster_for_parallel_replicas 2023-08-16 23:24:01 +00:00
Alexander Tokmakov
d7272410bd the worst fix for a deadlock ever 2023-08-17 00:42:51 +02:00
Robert Schulze
8a58e0df3e
Merge remote-tracking branch 'rschu1ze/master' into consistency-fixes-for-caches 2023-08-16 21:26:04 +00:00
Robert Schulze
353854520c
Merge remote-tracking branch 'rschu1ze/master' into factorize-constants 2023-08-16 21:24:21 +00:00
Alexander Tokmakov
69bc4f5139 Merge branch 'master' into exception_message_patterns6 2023-08-16 22:58:24 +02:00
Jianfei Hu
fcad273724 remove the comments. 
Signed-off-by: Jianfei Hu <hujianfei258@gmail.com>
 2023-08-16 17:49:28 +00:00
Jianfei Hu
1e600351d9 try again the if value.empty() in the beginning. 
Signed-off-by: Jianfei Hu <hujianfei258@gmail.com>
 2023-08-16 17:49:28 +00:00
Jianfei Hu
49502ac0ce revert unrelated style change. 
Signed-off-by: Jianfei Hu <hujianfei258@gmail.com>
 2023-08-16 17:49:28 +00:00
Jianfei Hu
02b7bd94eb suspect some root cause of value[0]. 
Signed-off-by: Jianfei Hu <hujianfei258@gmail.com>
 2023-08-16 17:49:28 +00:00
Jianfei Hu
7227cc6d2b try parts[0] instead of front(). 
Signed-off-by: Jianfei Hu <hujianfei258@gmail.com>
 2023-08-16 17:49:28 +00:00
Jianfei Hu
afaaef6875 wip checking whether system.merge cause sigsev. 
Signed-off-by: Jianfei Hu <hujianfei258@gmail.com>
 2023-08-16 17:49:28 +00:00
Alexey Milovidov
5e3a69b4c4 Update MergeList.cpp 2023-08-16 17:49:28 +00:00
Alexey Milovidov
bc77aab2e8 Update MergeList.h 2023-08-16 17:49:28 +00:00
Alexey Milovidov
7376b7ec8c Update MergeList.h 2023-08-16 17:49:28 +00:00
Alexey Milovidov
664bcfc8b3 Update MergeList.cpp 2023-08-16 17:49:28 +00:00
Alexey Milovidov
ecd1fb56b4 Update MergeList.cpp 2023-08-16 17:49:28 +00:00
Jianfei Hu
bd4df60df6 fix merges 
Signed-off-by: Jianfei Hu <hujianfei258@gmail.com>
 2023-08-16 17:49:28 +00:00
Jianfei Hu
59a81b82bc wip the merge partition 
Signed-off-by: Jianfei Hu <hujianfei258@gmail.com>
 2023-08-16 17:49:28 +00:00
Robert Schulze
5a87a09c84
Tweak exception handling 2023-08-16 17:16:52 +00:00
Andrey Zvonov
eb9c837d20
Merge branch 'ClickHouse:master' into zvonand-globs-small-fix 2023-08-16 19:22:00 +03:00
Davit Vardanyan
a0ec274766 Refactor: Naming 2023-08-16 17:30:56 +04:00
Davit Vardanyan
620733dd46 Fix: Exception handling 2023-08-16 17:30:29 +04:00
Robert Schulze
ea69eb095c
Merge remote-tracking branch 'ClickHouse/master' into usearch 2023-08-16 08:07:57 +00:00
Alexander Tokmakov
b32bdd1b0a
Merge pull request #53164 from ClickHouse/fixes_for_detached_parts 
Fixes for detach/attach partition and broken detached parts cleanup
 2023-08-16 00:09:13 +03:00
Robert Schulze
56a6d20137
Merge remote-tracking branch 'rschu1ze/master' into factorize-constants 2023-08-15 18:21:25 +00:00
Igor Nikonov
49134711dc Polishing 2023-08-15 17:37:01 +00:00
Robert Schulze
f558048bea
Merge remote-tracking branch 'rschu1ze/master' into consistency-fixes-for-caches 2023-08-15 17:12:52 +00:00
Igor Nikonov
9825f8c76e Remove debug logging 2023-08-15 17:03:32 +00:00
kssenii
2d74472ffb Merge remote-tracking branch 'upstream/master' into system-sync-cache 2023-08-15 20:31:26 +04:00
zvonand
0efaa5ab45 Merge branch 'master' of github.com:ClickHouse/ClickHouse into zvonand-globs-small-fix 2023-08-15 21:20:52 +05:00
Andrey Zvonov
34b1811a53
Update src/Storages/StorageFile.cpp 
seems like typo

Co-authored-by: SmitaRKulkarni <64093672+SmitaRKulkarni@users.noreply.github.com>
 2023-08-15 19:10:14 +03:00
Alexander Tokmakov
83c10dad18 Merge branch 'master' into exception_message_patterns6 2023-08-15 15:20:09 +02:00
Alexander Tokmakov
bf40767f10 fix another race 2023-08-15 14:27:49 +02:00
Igor Nikonov
1678cda8b3 Merge remote-tracking branch 'origin/master' into fix-parallel-replicas-multiply-result 2023-08-15 12:03:35 +00:00
Davit Vardanyan
48c62fd75e Add: USearch 2023-08-15 16:00:27 +04:00
Kruglov Pavel
cac2628b41
Merge pull request #53426 from ClickHouse/analyzer-fix-virtuals-distr 
Analyzer: fix virtual columns in StorageDistributed
 2023-08-15 13:55:42 +02:00
Robert Schulze
8992cc5af4
Factorize constants 2023-08-15 08:56:40 +00:00
Robert Schulze
f317903073
Merge branch 'ClickHouse:master' into consistency-fixes-for-caches 2023-08-15 10:25:08 +02:00
Igor Nikonov
c94994afcf Propagate setting cluster_for_parallel_replicas to shards 2023-08-14 22:50:46 +00:00
Alexander Tokmakov
376202f739 fix creation of empty parts 2023-08-15 00:26:17 +02:00
Alexander Tokmakov
894ec8e0d0 less exceptions with runtime format string 2023-08-14 21:45:59 +02:00
Dmitry Novik
1244828555 Analyzer: fix virtual columns in StorageDistributed 2023-08-14 19:27:05 +02:00
Alexander Tokmakov
f0f2cd8c13 Merge branch 'master' into fixes_for_detached_parts 2023-08-14 19:20:45 +02:00
ltrk2
2846ea49b4
Merge branch 'master' into feature/mergetree-checksum-big-endian-support 2023-08-14 13:02:30 -04:00
Robert Schulze
3472a928d8
Cosmetics: Rename dropCache() to clearCaches() 
"dropCache()" is a bad name, it sounds like that the cache can no longer
be used after the call. Not true, we only "clear" the cache.
 2023-08-14 15:42:32 +00:00
Duc Canh Le
cd535bea7d limit number of concurrent background partition optimize 2023-08-14 10:21:13 +00:00
Filipp Ozinov
5a180b53ed
Style fix 2023-08-14 12:30:29 +04:00
Andrey Zvonov
413b0f227b
Merge branch 'ClickHouse:master' into zvonand-globs-small-fix 2023-08-14 10:59:44 +03:00
Victor Krasnov
247cd63ca1 Merge branch 'master' into ADQM-987 
# Conflicts:
#	src/Interpreters/Context.cpp
#	src/Interpreters/InterpreterSystemQuery.cpp
 2023-08-14 10:33:56 +03:00
Alexey Milovidov
9656bd2738
Merge pull request #53372 from ClickHouse/Fix_lazy_database_and_system_table_query 
Fix loading lazy database during system.table select query
 2023-08-14 08:03:19 +03:00
Yakov Olkhovskiy
300399d1f5
Merge pull request #52329 from ClibMouse/feature/mergetree-checksum-on-big-endian 
Implement endianness-indepedent support for MergeTree checksums
 2023-08-13 23:59:05 -04:00
Filipp Ozinov
0cccb2ac3b
Fixed system.data_skipping_indices for MaterializedMySQL 
Related to #53350
 2023-08-14 01:57:41 +04:00
Smita Kulkarni
c25bb44e55 Fixed style check 2023-08-13 21:45:33 +02:00
Smita Kulkarni
ee772b7376 Avoid loading tables from lazy database when not needed 2023-08-13 20:15:58 +02:00
kssenii
8535e23f95 Merge remote-tracking branch 'upstream/master' into system-sync-cache 2023-08-13 21:10:42 +04:00
Andrey Zvonov
3d902e65e4
Merge branch 'ClickHouse:master' into zvonand-globs-small-fix 2023-08-12 23:12:47 +03:00
Alexey Milovidov
17127cf23f
Merge pull request #53337 from hanfei1991/hanfei/improve_move_to_prewhere 
improve `move_primary_key_columns_to_end_of_prewhere`
 2023-08-12 17:45:30 +03:00
Han Fei
0b3f9918a3 fix style 2023-08-12 14:40:02 +02:00
robot-ch-test-poll
967067bbb8
Merge pull request #53250 from evillique/postgresql-single-quotes 
Fix character escaping in the PostgreSQL engine
 2023-08-12 13:15:49 +02:00
Alexey Milovidov
3c2dd4e543
Update MergeTreeWhereOptimizer.cpp 2023-08-12 03:40:19 +03:00
Alexey Milovidov
6176971b3f
Update StoragePostgreSQL.cpp 2023-08-12 03:33:58 +03:00
Alexey Milovidov
4b59d265e1
Merge pull request #49723 from DimasKovas/fix_storage_buffer_reschedule 
Some unclear change in StorageBuffer::reschedule() for something
 2023-08-12 03:28:23 +03:00
Alexey Milovidov
e5f33511c6
Merge pull request #53242 from evillique/alter-exception 
Fix LOGICAL_ERROR exception in ALTER query
 2023-08-12 03:18:32 +03:00
Han Fei
e70dc7d507 improve move_primary_key_columns_to_end_of_prewhere 2023-08-11 17:36:52 +02:00
Sergei Trifonov
771710b377
Merge branch 'master' into concurrency-control-controllable 2023-08-11 16:50:13 +02:00
Andrey Zvonov
02c511a008
Merge branch 'master' into zvonand-globs-small-fix 2023-08-11 15:56:44 +02:00
Anton Popov
8744a0269a
Merge pull request #52530 from CurtizJ/fix-alter-delete-skip-index 
Fix recalculation of skip indexes and projections in `ALTER DELETE` queries
 2023-08-11 15:23:05 +02:00
zvonand
086bf6cb8c added ignore_eacces_multidirectory_globs 2023-08-11 15:49:21 +03:00
ltrk2
1dc1b54c68
Merge branch 'master' into feature/mergetree-checksum-big-endian-support 2023-08-11 07:48:49 -04:00
ltrk2
94dbef1e50
Merge branch 'master' into feature/mergetree-checksum-on-big-endian 2023-08-11 07:47:48 -04:00
Joris Giovannangeli
602570b462
add max_threads for index settings 2023-08-11 10:20:07 +02:00
Alexey Milovidov
aae35f3b1c
Merge pull request #53184 from ClickHouse/better-ccache 
Better usage of ccache
 2023-08-11 04:43:45 +03:00
Ilya Golshtein
ade9c3d970 system_kafka_consumers: per consumer librdkafka stat 2023-08-10 22:35:05 +00:00
Ilya Golshtein
3bced2f08c system_kafka_consumers: tidy fix, one partition for stable test 2023-08-10 22:35:05 +00:00
Ilya Golshtein
978d8bf9a6 system_kafka_consumers: cleanup with exceptions 2023-08-10 22:35:05 +00:00
Ilya Golshtein
7dab2dc041 system_kafka_consumers: exceptions seem to work 2023-08-10 22:35:05 +00:00
Ilya Golshtein
80829a6bf7 system_kafka_consumers: no offset_committed, new test with MV 2023-08-10 22:35:05 +00:00
Ilya Golshtein
6d6aae24f8 system_kafka_consumers: no friendship with StorageKafka 2023-08-10 22:35:05 +00:00
Ilya Golshtein
ae3e6f0fac system_kafka_consumers: getMemberId introduced to avoid leak 2023-08-10 22:35:05 +00:00
Ilya Golshtein
b325ddceb4 system_kafka_consumers: small thing to make clang tidy happy 2023-08-10 22:35:05 +00:00
Ilya Golshtein
46be5e5782 system_kafka_consumers: cleanup, test should be more stable 2023-08-10 22:35:05 +00:00
Ilya Golshtein
803a80b098 system_kafka_consumers: refactored with getStat, test passed 2023-08-10 22:35:05 +00:00
Ilya Golshtein
a86548a733 system_kafka_consumers: statistics.interval.ms default, tiny cleanup 2023-08-10 22:35:05 +00:00
Ilya Golshtein
dd6727be3e system_kafka_consumers: style check fixes 2023-08-10 22:35:05 +00:00
Ilya Golshtein
bba4dcacc7 system_kafka_consumers: small code cleanup 2023-08-10 22:35:05 +00:00
Ilya Golshtein
3f205f4997 system_kafka_consumers: tiny doc changes 2023-08-10 22:35:05 +00:00
Ilya Golshtein
b71c2bedfc system_kafka_consumers: rebalance test is ok, statistics_interval_ms 2023-08-10 22:35:05 +00:00
Ilya Golshtein
83566e132a system_kafka_consumers: all_consumers introduced, offsets work 2023-08-10 22:35:05 +00:00
Ilya Golshtein
de49d7e5a0 system_kafka_consumers: minor change 2023-08-10 22:35:05 +00:00
Ilya Golshtein
c821c7788a system_kafka_consumers: json commented out 2023-08-10 22:35:05 +00:00
Ilya Golshtein
ae6e0469ad system_kafka_consumers: rdfakfa stat with attempt to use json 2023-08-10 22:35:05 +00:00
Ilya Golshtein
6258ab6cb4 system_kafka_consumers: doc link fix, USE_RDKAFKA 2023-08-10 22:35:05 +00:00
Ilya Golshtein
40fd53c3e3 system_kafka_consumers: style check 2023-08-10 22:35:05 +00:00
Ilya Golshtein
88f5dd5db8 system_kafka_consumers: integration tests passed 2023-08-10 22:35:05 +00:00
Ilya Golshtein
957787d96a system_kafka_consumers: kind of works, test added 2023-08-10 22:35:05 +00:00
Ilya Golshtein
36fb330811 system_kafka_consumers: assignments 2023-08-10 22:35:05 +00:00
Ilya Golshtein
a947c7a91e system_kafka_consumers: start integrating with real kafka 2023-08-10 22:35:05 +00:00
Ilya Golshtein
2304f342c9 system_kafka_consumers: table populated by fake data 2023-08-10 22:35:05 +00:00
Ilya Golshtein
9d68d50ae7 system_kafka_consumers: initial 2023-08-10 22:35:04 +00:00
Alexander Tokmakov
5a8b8203b2 fix 2023-08-10 23:22:51 +02:00
Alexander Tokmakov
3d59ebe108 fix 2023-08-10 20:11:22 +02:00
Nikolay Degterinsky
4f0be777c5 Fix build 2023-08-10 16:39:43 +00:00
Antonio Andelic
478a5006b1 Merge branch 'master' into improve-archive-reading 2023-08-10 15:52:30 +00:00
Alexander Tokmakov
6ae0ba1163 Merge branch 'master' into fixes_for_detached_parts 2023-08-10 17:01:05 +02:00
ltrk2
a2054c04dd
Merge branch 'master' into feature/mergetree-checksum-big-endian-support 2023-08-10 10:21:34 -04:00
ltrk2
b780b64232
Merge branch 'master' into feature/mergetree-checksum-on-big-endian 2023-08-10 10:21:23 -04:00
Alexey Milovidov
bb57caa83e
Merge pull request #53180 from ClickHouse/ditch-tons-of-garbage 
Ditch tons of garbage
 2023-08-10 16:59:51 +03:00
Alexey Milovidov
f35df23974
Merge branch 'master' into better-ccache 2023-08-10 16:52:32 +03:00
pufit
af51268da1
Merge pull request #51912 from ClickHouse/pufit/role-from-storage 
Create, Drop, or Move access entities between different access storages.
 2023-08-10 09:47:26 -04:00
Kseniia Sumarokova
3925a1641f
Merge branch 'master' into system-sync-cache 2023-08-10 14:33:13 +02:00
zvonand
e4a1780bd7 update docs according to the PR + fix the same for HDFS 2023-08-10 15:29:27 +03:00
Alexander Tokmakov
4b30900fed
Revert "Improve CHECK TABLE system query" 2023-08-10 14:44:16 +03:00
Kseniia Sumarokova
aed955a331
Merge branch 'master' into system-sync-cache 2023-08-10 13:18:35 +02:00
Andrey Zvonov
90dcc2ac81
Merge branch 'master' into zvonand-globs-small-fix 2023-08-10 12:30:12 +02:00
Victor Krasnov
a89d711113 Move BackupsWorker::Info to a separate file BackupOperationInfo.h 2023-08-10 13:29:20 +03:00
Antonio Andelic
c65f3d1eb8 Mutex for name filter 2023-08-10 10:09:59 +00:00
Victor Krasnov
3c299dc684 Remove redundant forward declaration from IDataPartStorage.h 2023-08-10 12:56:57 +03:00
alesapin
57025eed65
Merge pull request #52745 from ClickHouse/vdimir/check_table_improvements 
Improve CHECK TABLE system query
 2023-08-10 11:48:00 +02:00
Kseniia Sumarokova
c4c76f60a7
Merge branch 'master' into fix-cache-logical-error 2023-08-10 10:55:22 +02:00
Antonio Andelic
aa258a0e61 Merge branch 'master' into improve-archive-reading 2023-08-10 06:59:04 +00:00
Nikolay Degterinsky
7321f5e543 Better 2023-08-10 06:34:10 +00:00
Alexey Milovidov
6e3100a79d
Merge pull request #53213 from ClickHouse/revert-53210-revert-53100-upload-build-profile 
Revert "Revert "Upload build time-trace data to CI database""
 2023-08-10 06:43:14 +03:00
Nikolay Degterinsky
3c6d140c17 Fix sigle quote escaping in PostgreSQL engine 2023-08-10 01:31:59 +00:00
Alexey Milovidov
d65ddc6194
Merge pull request #53175 from ClickHouse/analyzer-fix-test-unique-index 
Fix: 00838_unique_index test with analyzer
 2023-08-10 04:15:02 +03:00
Alexey Milovidov
2592382834
Merge branch 'master' into ditch-tons-of-garbage 2023-08-10 01:04:24 +03:00
pufit
1a3eac0d4d
Merge branch 'master' into pufit/role-from-storage 2023-08-09 17:50:07 -04:00
Nikolay Degterinsky
f2c3000a93 Improve exception in ALTER query 2023-08-09 21:40:06 +00:00
Antonio Andelic
a19d005edb Merge branch 'master' into improve-archive-reading 2023-08-09 20:17:31 +00:00
Alexey Milovidov
22ee135d01
Merge branch 'master' into better-ccache 2023-08-09 21:16:16 +03:00
Kseniia Sumarokova
8a69ab6098
Merge branch 'master' into fix-cache-logical-error 2023-08-09 19:12:27 +02:00
Kseniia Sumarokova
408003f2bc
Merge branch 'master' into system-sync-cache 2023-08-09 18:12:22 +02:00
ltrk2
139e9433a8
Merge branch 'master' into feature/mergetree-checksum-big-endian-support 2023-08-09 09:48:00 -04:00
ltrk2
add53bb8aa
Merge branch 'master' into feature/mergetree-checksum-on-big-endian 2023-08-09 09:44:32 -04:00
Alexey Milovidov
964c407d48
Revert "Revert "Upload build time-trace data to CI database"" 2023-08-09 16:11:17 +03:00
Antonio Andelic
0e17d26b88 More formats supported, read single archive from 1 thread 2023-08-09 11:58:37 +00:00
Alexander Gololobov
63183c56f1
Revert "Upload build time-trace data to CI database" 2023-08-09 13:58:05 +02:00
Alexander Tokmakov
48ed54e822
Allow experimantal features when recovering Replicated db replica (#53167) 
* allow experimantal features when recovering Replicated db replica

* Automatic style fix

---------

Co-authored-by: robot-clickhouse <robot-clickhouse@users.noreply.github.com>
 2023-08-09 14:35:50 +03:00
Kseniia Sumarokova
944e85d4e5
Merge branch 'master' into fix-cache-logical-error 2023-08-09 12:08:53 +02:00
Alexey Milovidov
fa9abc5038 Better usage of ccache 2023-08-09 05:02:50 +02:00
pufit
884ea7f592
Merge branch 'master' into pufit/role-from-storage 2023-08-08 20:20:37 -04:00
Alexey Milovidov
aa757490bd Ditch tons of garbage 2023-08-09 02:19:02 +02:00
Igor Nikonov
3296daad96 Don't build AST filter with enabled analyzer 2023-08-08 22:27:12 +00:00
Igor Nikonov
421610338e Merge remote-tracking branch 'origin/master' into analyzer-fix-test-unique-index 2023-08-08 21:59:01 +00:00
Igor Nikonov
040d666546 Fix: 00838_unique_index test with analyzer 2023-08-08 21:57:27 +00:00
Alexey Milovidov
23eee8d10d Merge branch 'master' of github.com:ClickHouse/ClickHouse into upload-build-profile 2023-08-08 21:02:26 +02:00
kssenii
43ed765231 Fix 2023-08-08 18:43:01 +02:00
Alexander Tokmakov
c1b94b4a3f fixes for detach/attach partition 2023-08-08 17:55:04 +02:00
SmitaRKulkarni
cd77406fba
Bug fix for checksum of compress marks (#51777) 
* add a test for cmrk and cidx checksums

* Bug fix for checksum of compress marks

* Fixed issue with .cidx checksum

* Fixed style check

* Updated 02253_empty_part_checksums.reference file with part details output

* Update 02253_empty_part_checksums.sh

* Updated to check is_compressed flag instead of size

* Update 02253_empty_part_checksums.sh

---------

Co-authored-by: Alexander Tokmakov <tavplubix@clickhouse.com>
 2023-08-08 17:55:07 +03:00
kssenii
2c485e78db Merge remote-tracking branch 'upstream/master' into system-sync-cache 2023-08-08 16:27:21 +02:00
Anton Popov
b6ce725367 fix projection recalc 2023-08-08 14:06:42 +00:00
vdimir
3f915491f0
Merge pull request #53046 from canhld94/ch_fix_direct_hash_join 2023-08-08 11:55:17 +02:00
vdimir
358ef26385
small fixes for CHECK TABLE 2023-08-08 09:45:33 +00:00
vdimir
1aedc4e892
Build proper pipeline for CHECK TABLE 2023-08-08 09:45:32 +00:00
vdimir
a882ef295f
Query CHECK TABLE takes care about progress and cancellation 2023-08-08 09:45:31 +00:00
pufit
2f46ae850e
Merge branch 'master' into pufit/role-from-storage 2023-08-07 17:57:41 -04:00
robot-ch-test-poll
066dd8a2e8
Merge pull request #52949 from CurtizJ/fix-nested-array-lowcardinality 
Fix reading of empty `Nested(Array(LowCardinality(...)))`
 2023-08-07 23:32:25 +02:00
Sean Haynes
77dd869414
Add ability to log when max_partitions_per_insert_block is reached (#50948) 
* Add ability to log when max_partitions_per_insert_block is reached

There's plenty of reasons to drop max_partitions_per_insert_block
down from it's default of 100, with the main one being even touching
remotely close to this number of partitions will create a lot of
merge activity that can de-stabilise a cluster.

This commit adds the ability to log when the above limit is reached
rather than throw an exception, so that you can gauge the impact on
users when dropping this value down.

* Update docs/ru/operations/settings/query-complexity.md

Co-authored-by: Nikita Taranov <nickita.taranov@gmail.com>

* Update docs/ru/operations/settings/query-complexity.md

Co-authored-by: Nikita Taranov <nickita.taranov@gmail.com>

* Update docs/ru/operations/settings/query-complexity.md

Co-authored-by: Nikita Taranov <nickita.taranov@gmail.com>

---------

Co-authored-by: Nikita Taranov <nickita.taranov@gmail.com>
 2023-08-07 23:28:19 +02:00
Alexey Milovidov
695f0ffd14 Merge branch 'clang17_fix' of github.com:Algunenano/ClickHouse into upload-build-profile 2023-08-07 21:18:36 +02:00
Azat Khuzhin
2f414950b7
Fix logging for asynchronous non-batched distributed sends (#52583) 
Before you may see the following:

  2023.07.25 09:21:39.705559 [ 692 ] {6b5e1299-1b64-4dbb-b25d-45e10027db22} <Trace> test_hkt5nnqj.dist_opentelemetry.DirectoryMonitor.default: Finished processing `` (took 37 ms)

Because file_path and current_file are the references to the same
variable in DistributedAsyncInsertDirectoryQueue::processFile().

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-08-07 20:57:42 +02:00
kssenii
9cdd886fde Fix 2023-08-07 16:46:42 +02:00
Duc Canh Le
cd02980130
Merge branch 'master' into ch_fix_direct_hash_join 2023-08-07 22:20:29 +08:00
Anton Popov
981da23144 fix reading of empty Nested(Array(...)) 2023-08-07 12:32:12 +00:00
Sergei Trifonov
dcac12c8f9
Merge branch 'master' into concurrency-control-controllable 2023-08-07 13:43:02 +02:00
Raúl Marín
30d1452bbc Compatibility with clang-17 2023-08-07 10:48:24 +02:00
SmitaRKulkarni
75f81bdc44
Merge pull request #50795 from ClickHouse/azure_table_function_cluster 
Table function azureBlobStorageCluster
 2023-08-07 09:16:52 +02:00
Duc Canh Le
ed2a1d7c9b select required columns when getting join 2023-08-07 03:15:20 +00:00
Antonio Andelic
cafa5ef2f3 Merge branch 'master' into add-reading-from-archives 2023-08-06 10:21:47 +00:00
Antonio Andelic
1d793e69f2 Fix 2023-08-05 17:06:44 +00:00
SmitaRKulkarni
1e3ae89161
Merge branch 'master' into azure_table_function_cluster 2023-08-04 21:39:32 +02:00
Anton Popov
991abde851 fix reading of empty Nested(Array(...)) 2023-08-04 13:10:50 +00:00
Antonio Andelic
d2b6646fc2 Merge branch 'master' into add-reading-from-archives 2023-08-04 12:42:46 +00:00
ltrk2
c010e1c9ff
Merge branch 'master' into feature/mergetree-checksum-on-big-endian 2023-08-04 08:31:57 -04:00
Duc Canh Le
8aaa46ca90 StorageJoin: do not create clone hash join with all columns 2023-08-04 11:07:33 +00:00
Kruglov Pavel
b1fe9ce8f7
Merge branch 'master' into formats-with-subcolumns 2023-08-04 13:01:12 +02:00
Antonio Andelic
d4d93331dd Merge branch 'master' into add-reading-from-archives 2023-08-04 09:00:20 +00:00
Vitaly Baranov
587877dba3
Merge branch 'master' into disable-fs-cache-for-backups 2023-08-04 01:15:23 +02:00
Alexander Tokmakov
968c79cf3f
Merge pull request #52956 from ClickHouse/fixes_for_databases 
Fix some issues with databases
 2023-08-03 19:13:41 +03:00
Kseniia Sumarokova
68b48a0bc0
Merge pull request #50063 from KevinyhZou/minor_improve_hdfs_read_buffer 
Minor improve HDFS ReadBuffer for read end of file
 2023-08-03 17:55:48 +02:00
avogar
f0eb22ac5f Fix build for S3Queue 2023-08-03 14:20:19 +00:00
ltrk2
a29b36612b
Merge branch 'master' into feature/mergetree-checksum-big-endian-support 2023-08-03 09:41:25 -04:00
ltrk2
84a77d1c21
Merge branch 'master' into feature/mergetree-checksum-on-big-endian 2023-08-03 09:41:16 -04:00
Anton Popov
92cb5e9ae3
Merge pull request #52687 from alkorgun/fix-named-collections-on-cluster-23.7 
Fix named collections on cluster 23.7
 2023-08-03 14:50:28 +02:00
Smita Kulkarni
2a681f1577 Removed unused function and fixed build after merge 2023-08-03 14:32:03 +02:00
Smita Kulkarni
68a10e7cae Merge branch 'master' into azure_table_function_cluster 2023-08-03 13:59:59 +02:00
Vitaly Baranov
e127d27453 Disable using fs cache for backup/restore #2. 2023-08-03 13:52:35 +02:00
Vitaly Baranov
f427c779eb Disable using fs cache for backup/restore. 2023-08-03 13:52:32 +02:00
Antonio Andelic
d9f136ce0c Add tests for globs 2023-08-03 10:42:47 +00:00
Alexander Tokmakov
7684e68008
Add system stop pulling replication log (#52881) 
* add system stop pulling replication log

* add const

* fix another stupid test

* Update test.py
 2023-08-03 13:34:43 +03:00
Alexander Tokmakov
9e0d27dc4d
don't create empty parts on drop partittion if we have a transaction (#52945) 2023-08-03 13:16:32 +03:00
Nikolai Kochetov
5d64e036ba
Merge pull request #52695 from yariks5s/in_schema_case_sensitiveness 
Information_schema case sensitiveness
 2023-08-03 12:15:31 +02:00
Kseniia Sumarokova
0ec11a2d79
Merge branch 'master' into system-sync-cache 2023-08-03 12:02:18 +02:00
Smita Kulkarni
5cdc50b9ba Fixed style 2023-08-03 11:31:15 +02:00
Kseniia Sumarokova
73ac1bf84f
Merge branch 'master' into minor_improve_hdfs_read_buffer 2023-08-03 10:56:24 +02:00
Antonio Andelic
96bad0537a Merge branch 'master' into add-reading-from-archives 2023-08-03 08:36:42 +00:00
Alexander Tokmakov
48d736f8fd select only required columns from system.databases 2023-08-02 23:23:52 +02:00
Azat Khuzhin
cd74da4c03 Remove unused code in StorageSystemStackTrace 
This columns mask is actually useless, since the main thing is done
without out, in particular:
- detect does the signal should be sent to thread
- does the thread name should be read for this thread

And this cannot be done with columns mask, because multiple columns
depends on signals and thread names.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-08-02 21:25:00 +02:00
Anton Popov
d43e76b147 fix reading of empty Nested(Array(LowCardinality(...))) 2023-08-02 16:48:08 +00:00
ltrk2
27a2d4d1c7
Merge branch 'master' into feature/mergetree-checksum-big-endian-support 2023-08-02 11:36:43 -04:00
ltrk2
ea28888763
Merge branch 'master' into feature/mergetree-checksum-on-big-endian 2023-08-02 11:36:26 -04:00
Nikolai Kochetov
b3bc3e89b6
Merge pull request #52837 from ClickHouse/cleanup-local-backup 
Cleanup localBackup
 2023-08-02 17:10:36 +02:00
Andrey Zvonov
2695502922
Merge branch 'master' into zvonand-globs-small-fix 2023-08-02 15:52:14 +02:00
zvonand
8f080a17de Merge branch 'zvonand-globs-small-fix' of github.com:zvonand/ClickHouse into zvonand-globs-small-fix 2023-08-02 15:51:48 +02:00
zvonand
c0f9dbbacb shorten 2023-08-02 15:51:27 +02:00
Anton Popov
ff137773e7
Merge branch 'master' into formats-with-subcolumns 2023-08-02 15:24:56 +02:00
Smita Kulkarni
2c47679918 Merge branch 'master' into azure_table_function_cluster 2023-08-02 14:19:45 +02:00
Yarik Briukhovetskyi
e811d5c4e9
Update attachInformationSchemaTables.cpp 2023-08-02 14:06:20 +02:00
Yarik Briukhovetskyi
dab77b044b
Update attachInformationSchemaTables.cpp 2023-08-02 13:23:18 +02:00
yariks5s
0174244fe1 fixed tests 2023-08-02 11:21:47 +00:00
Alexander Tokmakov
d8da47520b
Merge branch 'master' into fix_mutations_with_transactions 2023-08-02 13:56:59 +03:00
Kseniia Sumarokova
8180474e71
Merge branch 'master' into minor_improve_hdfs_read_buffer 2023-08-02 11:57:41 +02:00
Kseniia Sumarokova
a3dd2e5c2f
Merge pull request #48424 from ClickHouse/kssenii-patch-6 
Fix outdated cache configuration in s3 tests: s3_storage_policy_by_defau…
 2023-08-02 11:38:57 +02:00
Antonio Andelic
9423976b7a Add support for file globs 2023-08-02 07:42:56 +00:00
Alexander Tokmakov
d6a2c631da fix assertion in mutations with transactions 2023-08-01 21:16:22 +02:00
yariks5s
6db7eaf4d8 fixed ast 2023-08-01 19:14:11 +00:00
ltrk2
35c67f5010
Merge branch 'master' into feature/mergetree-checksum-on-big-endian 2023-08-01 13:41:30 -04:00
Antonio Andelic
2430ebabf5 Merge branch 'master' into add-reading-from-archives 2023-08-01 17:22:38 +00:00
Kseniia Sumarokova
8893670d98
Merge branch 'master' into s3queue 2023-08-01 18:42:32 +02:00
Kseniia Sumarokova
dbc11b83fd
Merge branch 'master' into kssenii-patch-6 2023-08-01 18:40:20 +02:00
Antonio Andelic
e366ca61f5 Merge branch 'master' into add-reading-from-archives 2023-08-01 15:33:20 +00:00
Vitaly Baranov
65bd8f5e81
Merge branch 'master' into backups-fix-tmp-files 2023-08-01 16:37:06 +02:00
Antonio Andelic
c24dc8e64e Multithreaded read of files from archives: 2023-08-01 13:48:49 +00:00
Sergei Trifonov
01196ac41f
Merge branch 'master' into concurrency-control-controllable 2023-08-01 15:40:50 +02:00
Alexey Milovidov
4578d43f79
Merge pull request #52656 from Avogar/rename-setting-for-url-encoding 
Rename setting disable_url_encoding to enable_url_encoding and add a test
 2023-08-01 16:09:01 +03:00
János Benjamin Antal
a6861ea2f5
Merge pull request #52386 from ybogo/support-predefined-macro-in-kafka-settings 
Add predefined macros support in Kafka engine settings
 2023-08-01 14:49:01 +02:00
kssenii
a14a6b56b3 Better 2023-08-01 14:31:05 +02:00
Andrey Zvonov
791c5b1393
Merge branch 'master' into zvonand-globs-small-fix 2023-08-01 14:00:57 +02:00
alesapin
0ee9797fb8
Merge pull request #51141 from ClickHouse/azure_blob_storage_sas_token 
Azure blob storage sas token
 2023-08-01 12:27:17 +02:00
kssenii
08f5ebf3e8 Fix test 2023-08-01 12:19:15 +02:00
kssenii
be458fd44b Fix style check 2023-08-01 12:06:25 +02:00
Kseniia Sumarokova
ccf9fa9641
Update ReadBufferFromHDFS.cpp 2023-08-01 11:55:08 +02:00
Nikolai Kochetov
433219e931
Update src/Storages/MergeTree/localBackup.cpp 
Co-authored-by: Sergei Trifonov <sergei@clickhouse.com>
 2023-08-01 11:50:56 +02:00
Kseniia Sumarokova
5880a0a372
Update ReadBufferFromHDFS.cpp 2023-08-01 11:50:31 +02:00
robot-ch-test-poll4
5b67cff6b2
Merge pull request #52829 from ClickHouse/throw-s3-exception-whenever-possible 
Throw S3Exception whenever possible.
 2023-08-01 04:35:27 +02:00
zvonand
e851be78bc fix fs_error on attempt to read unavailable dirs 2023-07-31 23:56:07 +02:00
Nikolai Kochetov
453463ba22 Cleaup on every iteration. 2023-07-31 19:47:23 +00:00
kssenii
75f6a50079 Add todo 2023-07-31 20:08:31 +02:00
kssenii
870a506a0b Some fixes 2023-07-31 20:07:23 +02:00
Nikolai Kochetov
c4a00b8c68 Throw S3Exception whenever possible. 2023-07-31 15:39:20 +00:00
Vitaly Baranov
f5ffef4014
Merge pull request #52658 from arenadata/ADQM-984 
Add s3_storage_class parameter in SETTINGS of BACKUP statement
 2023-07-31 17:32:12 +02:00
ltrk2
e869adf645 Improve function naming 2023-07-31 06:48:50 -07:00
kssenii
c13fdca23e Merge remote-tracking branch 'upstream/master' into s3queue 2023-07-31 15:32:56 +02:00
Kseniia Sumarokova
74c0574d51
Update ReadBufferFromHDFS.cpp 2023-07-31 15:22:16 +02:00
ltrk2
538349603d
Merge branch 'master' into feature/mergetree-checksum-on-big-endian 2023-07-31 09:18:53 -04:00
yariks5s
964e4311c3 reviewed changes 2023-07-31 13:03:56 +00:00
Antonio Andelic
9fb86f134b Fix tests 2023-07-31 12:04:27 +00:00
kssenii
5d167b4f03 Merge remote-tracking branch 'upstream/master' into system-sync-cache 2023-07-31 12:44:11 +02:00
Antonio Andelic
7df5c83046 Merge branch 'master' into add-reading-from-archives 2023-07-31 09:27:01 +00:00
Alexey Milovidov
bd3af7ebe6
Merge pull request #52754 from ClickHouse/analyzer-set-index-not 
Analyzer: fix 00979_set_index_not.sql
 2023-07-31 12:13:59 +03:00
Antonio Andelic
df45602165 Refactoring 2023-07-31 08:50:09 +00:00
Kseniia Sumarokova
dd85c3c114
Update ReadBufferFromHDFS.cpp 2023-07-30 22:40:11 +02:00
Kseniia Sumarokova
8daeeedc8f
Merge pull request #52315 from xiedeyantu/add-alias 
system.events and system.metrics tables add column name as an alias to event and metric
 2023-07-30 22:37:31 +02:00
Anton Popov
0d8f852144
Merge pull request #52689 from CurtizJ/fix-extra-column-prewhere 
Fix reading of unnecessary column in case of multistage `PREWHERE`
 2023-07-30 15:30:53 +02:00
kssenii
20b76010ef Merge remote-tracking branch 'upstream/master' into kssenii-patch-6 2023-07-30 13:46:58 +02:00
Kseniia Sumarokova
08ef4d015c
Minor change 2023-07-30 13:36:52 +02:00
Kseniia Sumarokova
30b6cf0713
Merge branch 'master' into minor_improve_hdfs_read_buffer 2023-07-30 13:33:24 +02:00
Alexey Milovidov
91e67c105f
Merge pull request #52569 from ClickHouse/Realize_parameterized_view_as_new_storage_view 
Realize parameterized view as new view with parameters substitued
 2023-07-30 05:46:51 +03:00
robot-clickhouse-ci-2
2dd69844dd
Merge pull request #52716 from canhld94/ch_use_same_poll_for_attach_n_fetch 
Use same executor for GET_PART and ATTACH_PART
 2023-07-29 22:27:08 +02:00
Igor Nikonov
23bc6755df
Merge branch 'master' into analyzer-set-index-not 2023-07-29 18:17:55 +02:00
Maksim Kita
d82a834bec Planner prepare filters for analysis 2023-07-29 19:17:14 +03:00
Dmitry Novik
8f4527d9e0
Merge pull request #46740 from ClickHouse/query-tree-visitor 
Refactor Query Tree visitor
 2023-07-29 15:10:07 +02:00
Yury Bogomolov
ba081ed402
Merge branch 'master' into support-predefined-macro-in-kafka-settings 2023-07-29 15:00:21 +04:00
Smita Kulkarni
553304a81a Fixed clang tidy build by removing unued variable 2023-07-29 11:15:56 +02:00
Alexey Milovidov
9a8077265d
Merge branch 'master' into ch_use_same_poll_for_attach_n_fetch 2023-07-29 05:19:22 +03:00
Igor Nikonov
1c6ef97b33 Merge remote-tracking branch 'origin/master' into analyzer-set-index-not 2023-07-28 23:43:15 +00:00
Igor Nikonov
45b9614224 Remove debug logging 2023-07-28 23:42:58 +00:00
Igor Nikonov
2a40bc9055 Fix: __bitSwapLastTwo was applied to non BoolMask 
- ActionsDAG was missing convertions to BoolMask - __bitWrapperFunc
 2023-07-28 23:29:59 +00:00
ltrk2
6c9a1b14ef
Merge branch 'master' into feature/mergetree-checksum-big-endian-support 2023-07-28 16:18:46 -04:00
ltrk2
cace04cc58
Merge branch 'master' into feature/mergetree-checksum-on-big-endian 2023-07-28 15:52:53 -04:00
Yury Bogomolov
b2ee4505fb review fixes 2023-07-28 16:06:03 +00:00
pufit
6211845ef0
Merge pull request #51303 from ClickHouse/deprecate-metadata-cache 
Deprecate the `metadata_cache` feature
 2023-07-28 10:31:02 -04:00
Dmitry Novik
e882adbc2b Merge remote-tracking branch 'origin/master' into query-tree-visitor 2023-07-28 14:22:42 +00:00
Alexander Tokmakov
39200606ec
Merge pull request #52174 from arenadata/ADQM-988 
Added possibility to save logs on crash and options to configure logs buffer
 2023-07-28 16:55:00 +03:00
yariks5s
96df21bbf5 implemented for all tables, fixed style and fast test 2023-07-28 13:35:13 +00:00
Antonio Andelic
e83e0ec2cd Fix build 2023-07-28 12:26:56 +00:00
Antonio Andelic
720d587e85 Merge branch 'master' into add-reading-from-archives 2023-07-28 08:49:00 +00:00
Alexey Gerasimchuck
63b05da1f2 System logs improvements 2023-07-28 07:23:34 +00:00
robot-ch-test-poll3
d6e2e8b92c
Merge pull request #52653 from azat/virtual-columns-or 
RFC: Fix filtering by virtual columns with OR expression
 2023-07-28 09:18:36 +02:00
Roman Vasin
cfa06725cc
Merge branch 'master' into ADQM-984 2023-07-28 10:15:27 +03:00
Duc Canh Le
97e63d523f use same executor for GET_PART and ATTACH_PART 2023-07-28 07:08:11 +00:00
Dmitry Novik
18c1fd6f08 Refactor InDepthQueryTreeVisitorWithContext 2023-07-27 21:24:39 +00:00
Al Korgun
ebbd662071
Merge branch 'ClickHouse:master' into fix-named-collections-on-cluster-23.7 2023-07-27 23:31:35 +03:00
Han Fei
4f9075e819
Merge pull request #52623 from hanfei1991/hanfei/MergeTreeDeduplicationLog 
Fix stress test: check if storage shutdown before we operate MergeTreeDeduplicationLog
 2023-07-27 20:50:17 +02:00
yariks5s
da53bca6e4 added tests 2023-07-27 18:29:50 +00:00
Azat Khuzhin
d40dbdee62 Fix usage of temporary directories during RESTORE 
Previously during RESTORE you may get the following warning in logs:

    <Warning> TemporaryFileOnDisk: Temporary path 'tmp/21672kaaaaa' does not exist in './disks/s3_common_disk/'

The reason is that it uses not disks/s3_common_disk/tmp but instead
disks/s3_common_disk/disks/s3_common_disk/tmp path.

Fix this by adding TemporaryFileOnDisk::getRelativePath() and use it
when appropriate.

And now - `find disks`, does not shows any more temporary leftovers.

v2: rename getPath to getAbsolutePath
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-07-27 19:24:54 +02:00
Raúl Marín
05427c2983 Style 2023-07-27 18:31:02 +02:00
Anton Popov
dfc06d2714 fix reading of unneded column in case of multistage prewhere 2023-07-27 16:15:23 +00:00
Raúl Marín
5c217467e8 Unify retries under checkQuorumPrecondition 2023-07-27 17:52:10 +02:00
Raúl Marín
3a831adcfb Improve retries on keeper session expiration 2023-07-27 17:37:04 +02:00
Azat Khuzhin
68aed0d16e RFC: Fix filtering by virtual columns with OR expression 
Virtual columns did not supports queries with OR, for example query like
this (here `m` is the `Merge` table, see the test):

    select key from m where (value = 10 and _table = 'v1') or (value = 20 and _table = 'v1');

Will always leads to:

    Cannot find column `value` in source stream, there are only columns ...

The reason for this is that it actually executes the following queries:

    SELECT key, value FROM default.d1 WHERE ((value = 10) AND ('v1' = 'v1')) OR ((value = 20) AND ('v1' = 'v1'));
    SELECT key FROM default.d2 WHERE 0;

And this kind of filtering is used not only for `Merge` table but also:
- `_table` for `Merge` (already mentioned)
- `_file` for `File`
- `_idx` for `S3`
- and as well as filtering `system.*` tables by `database`/`table`/...

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-07-27 16:35:17 +02:00
Azat Khuzhin
5611b2fff4 Add a note about not working _table filter for Merge with analyzer 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-07-27 16:35:17 +02:00
Han Fei
9488567bf6
Update src/Storages/MergeTree/MergeTreeDeduplicationLog.cpp 2023-07-27 16:28:08 +02:00
Han Fei
f6ca013c53
Update src/Storages/MergeTree/MergeTreeDeduplicationLog.cpp 2023-07-27 16:28:00 +02:00
Han Fei
0d44d527ef
Update src/Storages/MergeTree/MergeTreeDeduplicationLog.cpp 
Co-authored-by: Alexander Tokmakov <tavplubix@gmail.com>
 2023-07-27 16:27:04 +02:00
Han Fei
f5dfb70f5c
Update src/Storages/MergeTree/MergeTreeDeduplicationLog.cpp 2023-07-27 16:12:20 +02:00
Han Fei
ce38d3c5ea address comment 2023-07-27 16:11:08 +02:00
Alexey Milovidov
671128140d Update autogenerated version to 23.8.1.1 and contributors 2023-07-27 15:34:05 +02:00
Alexander Tokmakov
f3dc6dd061
Revert "Added field refcount to system.remote_data_paths table" 2023-07-27 15:23:57 +03:00