Robert Schulze
45d924c62c
Merge pull request #53616 from wzb5212/skip_idx_bug_fix
...
Fix number of dropped granules in EXPLAIN PLAN index=1
2023-08-24 16:46:47 +02:00
Antonio Andelic
6eb400c6ae
Fix reading from url with all filtered paths
2023-08-24 13:37:11 +00:00
alesapin
415b3826e0
Add event for tracking data parts lock
2023-08-24 15:28:38 +02:00
Ilya Golshtein
f58b4a812d
exceptions_kafka_consumers: code review suggestions are addressed
2023-08-24 13:10:21 +00:00
Arthur Passos
2bade7db08
Add global proxy setting ( #51749 )
...
* initial impl
* fix env ut
* move ut directory
* make sure no null proxy resolver is returned by ProxyConfigurationResolverProvider
* minor adjustment
* add a few tests, still incomplete
* add proxy support for url table function
* use proxy for select from url as well
* remove optional from return type, just returns empty config
* fix style
* style
* black
* ohg boy
* rm in progress file
* god pls don't let me kill anyone
* ...
* add use_aws guards
* remove hard coded s3 proxy resolver
* add concurrency-mt-unsafe
* aa
* black
* add logging back
* revert change
* imrpove code a bit
* helper functions and separate tests
* for some reason, this env test is not working..
* formatting
* :)
* clangtidy
* lint
* revert some stupid things
* small test adjusmtments
* simplify tests
* rename test
* remove extra line
* freaking style change
* simplify a bit
* fix segfault & remove an extra call
* tightly couple proxy provider with context..
* remove useless include
* rename config prefix parameter
* simplify provider a bit
* organize provider a bit
* add a few comments
* comment out proxy env tests
* fix nullptr in unit tests
* make sure old storage proxy config is properly covered without global context instance
* move a few functions from class to anonymous namespace
* fix no fallback for specific storage conf
* change API to accept http method instead of bool
* implement http/https distinction in listresolver, any still not implemented
* implement http/https distinction in remote resolver
* progress on code, improve tests and add url function working test
* use protcol instead of method for http and https
* small fix
* few more adjustments
* fix style
* black
* move enum to proxyconfiguration
* wip
* fix build
* fix ut
* delete atomicroundrobin class
* remove stale include
* add some tests.. need to spend some more time on the design..
* change design a bit
* progress
* use existing context for tests
* rename aux function and fix ut
* ..
* rename test
* try to simplify tests a bit
* simplify tests a bit more
* attempt to fix tests, accept more than one remote resolver
* use proper log id
* try waiting for resolver
* proper wait logic
* black
* empty
* address a few comments
* refactor tests
* remove old tests
* baclk
* use RAII to set/unset env
* black
* clang tidy
* fix env proxy not respecting any
* use log trace
* fix wrong logic in getRemoteREsolver
* fix wrong logic in getRemoteREsolver
* fix test
* remove unwanted code
* remove ClientConfigurationperRequest and auxilary classes
* remove unwanted code
* remove adapter test
* few adjustments and add test for s3 storage conf with new proxy settings
* black
* use chassert for context
* Add getenv comment
2023-08-24 16:07:26 +03:00
Antonio Andelic
8e1d38d377
Merge branch 'master' into archive-improvements-2
2023-08-24 13:03:36 +00:00
Raúl Marín
1c98568df9
Merge remote-tracking branch 'blessed/master' into better_zk_retries_on_insert
2023-08-24 12:22:42 +02:00
Robert Schulze
8c9fade58c
Merge remote-tracking branch 'rschu1ze/master' into skip_idx_bug_fix
2023-08-24 10:17:34 +00:00
Kseniia Sumarokova
a4f39fe134
Merge branch 'master' into fix-cache-logical-error
2023-08-24 12:00:18 +02:00
Alexander Tokmakov
011c7f7064
Revert "Planner prepare filters for analysis"
2023-08-24 12:52:03 +03:00
Kseniia Sumarokova
187706558a
Merge pull request #51622 from kssenii/system-sync-cache
...
Add `system sync filesystem cache` command
2023-08-24 11:42:32 +02:00
Ilya Golshtein
901240eede
exceptions_kafka_consumers: clang-tidy improvement
2023-08-24 08:44:08 +00:00
Ilya Golshtein
ec0c12960a
exceptions_kafka_consumers: stylecheck
2023-08-23 23:10:25 +00:00
Ilya Golshtein
e7213ef5b9
exceptions_kafka_consumers: new test, MV exceptions propagated
2023-08-23 22:46:04 +00:00
Kruglov Pavel
685ffc5856
Fix style
2023-08-23 22:51:14 +02:00
Kruglov Pavel
f7e1abd774
Merge branch 'master' into cache-count
2023-08-23 22:31:49 +02:00
Kruglov Pavel
063c986680
Merge pull request #53637 from Avogar/fast-count-from-files
...
Optimize count from files in most input formats
2023-08-23 22:17:29 +02:00
avogar
93ba73715a
Merge branch 'master' of github.com:ClickHouse/ClickHouse into group-by-constant-keys
2023-08-23 18:47:42 +00:00
avogar
894513f6cd
Fix tests
2023-08-23 18:43:08 +00:00
Nikolai Kochetov
3a4a7d70a2
Merge pull request #52762 from kitaisreal/planner-prepare-filters-for-analysis
...
Planner prepare filters for analysis
2023-08-23 18:59:43 +02:00
Anton Kozlov
a5b6a91b54
[bloom filter] Write hashes for bloom filter into a hash set for optimal sizing
2023-08-23 14:17:55 +00:00
Victor Krasnov
8b2342c205
Minor code cleanup: remove some redundant includes of InterpreterAlterQuery.h
2023-08-23 14:16:36 +00:00
Raúl Marín
93dac0c880
Support clang-18 (Wmissing-field-initializers)
2023-08-23 15:53:45 +02:00
Smita Kulkarni
262c39329b
Updated to retry loading parts for azure TransportException
2023-08-23 15:50:11 +02:00
Smita Kulkarni
1a074ecf93
Updated to not mark part as broken for azure TransportException in loadDataPart
2023-08-23 15:33:01 +02:00
Smita Kulkarni
7fdf90ccf1
Added azure TransportException to retryable errors
2023-08-23 15:21:37 +02:00
avogar
68e3af56d4
Address comments
2023-08-23 13:19:15 +00:00
Kruglov Pavel
592fa77987
Merge branch 'master' into cache-count
2023-08-23 15:18:02 +02:00
Kruglov Pavel
7e362a2110
Merge branch 'master' into fast-count-from-files
2023-08-23 15:13:20 +02:00
Robert Schulze
bea651e828
Merge pull request #53657 from rschu1ze/cache-improvements
...
Expose more settings from caches
2023-08-23 14:58:45 +02:00
Robert Schulze
a093f519fc
Merge remote-tracking branch 'rschu1ze/master' into skip_idx_bug_fix
2023-08-23 12:37:46 +00:00
Robert Schulze
5dec6f19f3
Merge pull request #51325 from xinhuitian/fix-annoy-index-update
...
Do not reset Annoy index during build-up with > 1 mark
2023-08-23 14:31:06 +02:00
robot-ch-test-poll1
c22ffa6195
Merge pull request #53529 from Avogar/filter-files-all-table-functions
...
Use filter by file/path before reading in url/file/hdfs table functins
2023-08-23 14:21:23 +02:00
Robert Schulze
761268f86a
Merge remote-tracking branch 'rschu1ze/master' into skip_idx_bug_fix
2023-08-23 10:55:04 +00:00
Kruglov Pavel
e193aec583
Merge branch 'master' into fast-count-from-files
2023-08-23 12:15:34 +02:00
kssenii
325e2d3172
Merge remote-tracking branch 'upstream/master' into system-sync-cache
2023-08-23 11:44:19 +02:00
priera
fd51839b97
basic implementation
2023-08-23 11:19:01 +02:00
Antonio Andelic
83d4b819f3
Better support for reading from archives
2023-08-23 08:10:30 +00:00
xinhuitian
920762e66d
fix USearch
2023-08-23 10:54:56 +08:00
Tian Xinhui
7cf057e716
Merge branch 'ClickHouse:master' into fix-annoy-index-update
2023-08-23 10:33:14 +08:00
pufit
926533306c
Merge pull request #53668 from ClickHouse/pufit/fix_s3_threads
...
Limiting the number of parsing threads for the S3 source
2023-08-22 19:09:33 -04:00
Kruglov Pavel
a83a868acd
Update src/Storages/HDFS/StorageHDFS.cpp
...
Co-authored-by: Dmitry Novik <mrnovikd@gmail.com>
2023-08-22 19:08:04 +02:00
Robert Schulze
d5a7277a59
Better parameter name
2023-08-22 15:43:13 +00:00
pufit
e42da9411b
Fix variables names
2023-08-22 11:23:10 -04:00
Raúl Marín
abfd48bcd2
Merge remote-tracking branch 'blessed/master' into better_zk_retries_on_insert
2023-08-22 17:01:22 +02:00
Nikolai Kochetov
5ee9b56040
Do not change the snapshot after at reading from MT
2023-08-22 13:50:51 +00:00
Victor Krasnov
258f9927b4
Merge branch 'master' into ADQM-987
2023-08-22 16:49:32 +03:00
Kruglov Pavel
de960f3c35
Fix style
2023-08-22 15:22:17 +02:00
Kruglov Pavel
a34c9c3ebc
Fix build
2023-08-22 15:21:54 +02:00
Kruglov Pavel
67c5c0203b
Merge branch 'master' into fast-count-from-files
2023-08-22 15:03:48 +02:00
avogar
89b402f4e2
Fix
2023-08-22 13:01:07 +00:00
avogar
7f9e81d504
Clean up
2023-08-22 12:55:00 +00:00
alesapin
b4aaeb1a82
Fix bad log line format
2023-08-22 14:53:35 +02:00
Kruglov Pavel
c0bdd0e00b
Merge branch 'master' into cache-count
2023-08-22 14:42:22 +02:00
avogar
b4145aeddc
Cache number of rows in files for count in file/s3/url/hdfs/azure functions
2023-08-22 11:59:59 +00:00
Robert Schulze
f2fb7365e6
Add settings for 1. size_ratio (all caches) and 2. policy (index caches)
...
1. is an expert-level setting, default is 0.5, applies only to SLRU.
Also, I noticed that we expose cache policy settings for the mark and
the uncompresed cache but not for the index mark and the index
uncompressed cache. Changed that as well, it simplifies the code a bit.
2023-08-22 11:04:29 +00:00
Ilya Golshtein
7cd8237a42
exceptions_kafka_consumers: initial
2023-08-22 10:26:52 +00:00
Robert Schulze
acaa2eee58
Merge pull request #53556 from ClibMouse/feature/big-endian-UUID-correction
...
Change Big Endian-UUID to work the same as Little Endian-UUID
2023-08-22 11:57:44 +02:00
alesapin
1a5a9fca87
Merge pull request #53605 from irenjj/feat_53538
...
Add column 'name' to system.clusters as an alias to cluster.
2023-08-22 11:34:21 +02:00
VanDarkholme7
2638cc69d7
fix bug on DELETE and UPDATE with JSON subcolumns
2023-08-22 09:26:17 +00:00
Tian Xinhui
76016d9593
Merge branch 'master' into fix-annoy-index-update
2023-08-22 16:58:29 +08:00
pufit
9d454d9afc
Merge branch 'master' into pufit/fix_s3_threads
...
# Conflicts:
# src/Storages/StorageS3.cpp
# src/Storages/StorageS3.h
# src/Storages/StorageURL.cpp
# src/Storages/StorageURL.h
2023-08-21 21:32:15 -04:00
pufit
98a701e2c1
Limiting number of parsing threads for S3 source
2023-08-21 21:21:03 -04:00
Robert Schulze
a20ceef398
Merge pull request #53447 from davvard/master
...
USearch HNSW index for ANN search
2023-08-21 23:34:26 +02:00
Michael Kolupaev
2f4d433e69
Parquet filter pushdown
2023-08-21 14:15:52 -07:00
Nikolai Kochetov
4222ac528d
Merge branch 'master' into planner-prepare-filters-for-analysis
2023-08-21 16:48:33 +00:00
Kruglov Pavel
aebbf00a5a
Fix
2023-08-21 17:47:30 +02:00
Anton Popov
3e9a182555
better interfaces for getting of stream name in part
2023-08-21 13:52:25 +00:00
Kruglov Pavel
a2f81bf4ea
Fix typo
2023-08-21 15:20:44 +02:00
Kruglov Pavel
dfdd26ee71
Fix build
2023-08-21 15:04:03 +02:00
Kruglov Pavel
88aee95122
Merge branch 'master' into fast-count-from-files
2023-08-21 14:46:33 +02:00
avogar
584bd57f5c
Clean up
2023-08-21 12:43:11 +00:00
avogar
47304bf7aa
Optimize count from files in most input formats
2023-08-21 12:30:52 +00:00
Kruglov Pavel
9d9bdde0e3
Merge branch 'master' into filter-files-all-table-functions
2023-08-21 14:26:59 +02:00
Kruglov Pavel
9869196021
Update VirtualColumnUtils.cpp
2023-08-21 14:22:48 +02:00
Antonio Andelic
ed15babddd
Merge pull request #53198 from ClickHouse/improve-archive-reading
...
Improve reading from archives
2023-08-21 14:11:25 +02:00
Victor Krasnov
a7813260c0
Merge branch 'master' into ADQM-987
2023-08-21 14:45:13 +03:00
Kruglov Pavel
21413636ac
Remove debug logging
2023-08-21 13:30:08 +02:00
Robert Schulze
2c330516b8
Another compile fix
2023-08-21 11:12:39 +00:00
Duc Canh Le
2c07065f81
increase the default value
...
Signed-off-by: Duc Canh Le <duccanh.le@ahrefs.com>
2023-08-21 11:04:24 +00:00
Raúl Marín
f2d3c9c281
Merge remote-tracking branch 'blessed/master' into better_zk_retries_on_insert
2023-08-21 12:14:31 +02:00
Robert Schulze
066ec559ad
Do not reset Annoy index during build-up with > 1 mark
2023-08-21 10:05:43 +00:00
王智博
8bb0f18019
bad size of mark skip idx
2023-08-20 22:57:53 +08:00
Kseniia Sumarokova
404fb0588a
Merge branch 'master' into fix-cache-logical-error
2023-08-20 16:28:13 +02:00
irenjj
a9b65ee929
Add Alias for system.clusters.
2023-08-20 14:47:07 +08:00
irenjj
71459c6d65
Revert
2023-08-20 14:35:59 +08:00
irenjj
e64a1fc26d
Add column 'name' to system.clusters as an alias to cluster.
2023-08-20 11:52:14 +08:00
Igor Nikonov
a69ea02418
Merge remote-tracking branch 'origin/master' into fix-parallel-replicas-multiply-result
2023-08-19 21:43:25 +00:00
Kseniia Sumarokova
aeebedfa0d
Merge branch 'master' into system-sync-cache
2023-08-19 13:32:48 +04:00
Kseniia Sumarokova
c0b8d7eddf
Merge pull request #50999 from ilejn/system_kafka_consumers
...
system.kafka_consumers table to monitor kafka consumers
2023-08-19 13:31:18 +04:00
kssenii
247abe0a2a
Merge remote-tracking branch 'upstream/master' into system-sync-cache
2023-08-19 08:43:53 +04:00
Alexey Milovidov
8520a7baca
Merge pull request #53559 from azat/skip-index-set-fix
...
Fix possible UB in Set skipping index for functions with incorrect args
2023-08-19 02:06:18 +03:00
Alexey Milovidov
6fb24fe74d
Merge pull request #53560 from azat/skip-index-inverted-fix-ub
...
Fix possible UB in inverted indexes (experimental feature)
2023-08-19 02:06:04 +03:00
Igor Nikonov
aebda10f7c
Merge remote-tracking branch 'origin/master' into fix-parallel-replicas-multiply-result
2023-08-18 23:00:58 +00:00
Alexey Milovidov
7ad8ea78bb
Merge pull request #48990 from incfly/partition
...
Add partition to the system part_log and merge table.
2023-08-18 21:38:53 +03:00
avogar
8e445b5461
Fixes
2023-08-18 17:49:40 +00:00
Azat Khuzhin
c78b94bed6
Fix possible UB in inverted indexes (experimental feature)
...
It is possible to have incorrect types there in case of index contains
functions, add a proper check after value had been changed.
<details>
<summary>MSan report</summary>
```
==182==WARNING: MemorySanitizer: use-of-uninitialized-value
0 0x55658547c59e in DB::MergeTreeConditionInverted::traverseASTEquals() build_docker/./src/Storages/MergeTree/MergeTreeIndexInverted.cpp:551:26
1 0x556585475566 in DB::MergeTreeConditionInverted::traverseAtomAST() build_docker/./src/Storages/MergeTree/MergeTreeIndexInverted.cpp:438:21
2 0x55658547e73e in DB::MergeTreeConditionInverted::MergeTreeConditionInverted()::$_1::operator()(DB::RPNBuilderTreeNode const&, DB::MergeTreeConditionInverted::RPNElement&) const build_docker/./src/Storages/MergeTree/MergeTreeIndexInverted.cpp:228:73
9 0x556585489e35 in DB::RPNBuilder<DB::MergeTreeConditionInverted::RPNElement>::traverseTree(DB::RPNBuilderTreeNode const&) build_docker/./src/Storages/MergeTree/RPNBuilder.h:252:14
10 0x556585489921 in DB::RPNBuilder<DB::MergeTreeConditionInverted::RPNElement>::traverseTree(DB::RPNBuilderTreeNode const&) build_docker/./src/Storages/MergeTree/RPNBuilder.h:239:21
11 0x556585489921 in DB::RPNBuilder<DB::MergeTreeConditionInverted::RPNElement>::traverseTree(DB::RPNBuilderTreeNode const&) build_docker/./src/Storages/MergeTree/RPNBuilder.h:239:21
12 0x5565854804e6 in DB::RPNBuilder<DB::MergeTreeConditionInverted::RPNElement>::RPNBuilder(std::__1::shared_ptr<DB::IAST> const&, std::__1::shared_ptr<DB::Context const>, DB::Block, std::__1::shared_ptr<DB::PreparedSets>, std::__1::function<bool (DB::RPNBuilderTreeNode const&, DB::MergeTreeConditionInverted::RPNElement&)> const&) build_docker/./src/Storages/MergeTree/RPNBuilder.h:218:9
13 0x55658546fb87 in DB::MergeTreeConditionInverted::MergeTreeConditionInverted(DB::SelectQueryInfo const&, std::__1::shared_ptr<DB::Context const>, DB::Block const&, DB::GinFilterParameters const&, DB::ITokenExtractor const*) build_docker/./src/Storages/MergeTree/MergeTreeIndexInverted.cpp:223:28
19 0x55658547ceda in DB::MergeTreeIndexInverted::createIndexCondition(DB::SelectQueryInfo const&, std::__1::shared_ptr<DB::Context const>) const build_docker/./src/Storages/MergeTree/MergeTreeIndexInverted.cpp:716:12
20 0x556587125734 in DB::buildIndexes(std::__1::optional<DB::ReadFromMergeTree::Indexes>&, std::__1::shared_ptr<DB::ActionsDAG>, DB::MergeTreeData const&, std::__1::shared_ptr<DB::Context const> const&, DB::SelectQueryInfo const&, std::__1::shared_ptr<DB::StorageInMemoryMetadata const> const&) build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:1292:48
21 0x556587121aaa in DB::ReadFromMergeTree::applyFilters() build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:1305:5
22 0x55658726cf4f in DB::QueryPlanOptimizations::optimizeTreeThirdPass(DB::QueryPlan::Node&, std::__1::list<DB::QueryPlan::Node, std::__1::allocator<DB::QueryPlan::Node>>&) build_docker/./src/Processors/QueryPlan/Optimizations/optimizeTree.cpp:201:38
23 0x5565870bc489 in DB::QueryPlan::optimize(DB::QueryPlanOptimizationSettings const&) build_docker/./src/Processors/QueryPlan/QueryPlan.cpp:485:5
24 0x5565870b919a in DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) build_docker/./src/Processors/QueryPlan/QueryPlan.cpp:161:5
25 0x55658206385a in DB::InterpreterSelectWithUnionQuery::execute() build_docker/./src/Interpreters/InterpreterSelectWithUnionQuery.cpp:379:31
26 0x556582c15c50 in DB::executeQueryImpl(char const*, char const*, std::__1::shared_ptr<DB::Context>, bool, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) build_docker/./src/Interpreters/executeQuery.cpp:751:40
27 0x556582c09681 in DB::executeQuery(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::shared_ptr<DB::Context>, bool, DB::QueryProcessingStage::Enum) build_docker/./src/Interpreters/executeQuery.cpp:1173:30
28 0x5565860e7bd7 in DB::TCPHandler::runImpl() build_docker/./src/Server/TCPHandler.cpp:421:24
29 0x556586126e9e in DB::TCPHandler::run() build_docker/./src/Server/TCPHandler.cpp:2057:9
```
</details>
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-08-18 18:15:17 +02:00
Azat Khuzhin
1e2b0c8dc4
Fix possible UB in Set skipping index for functions with incorrect args
...
Usually functions are validated before index analysis, however it is
not the case for parallel replicas, and it requires additional checks
before interpreting function arguments.
<details>
<summary>stack trace</summary>
```
==172==WARNING: MemorySanitizer: use-of-uninitialized-value
0 0x55cf82aedd6c in DB::ASTFunction* typeid_cast<DB::ASTFunction*, DB::IAST>(DB::IAST*) build_docker/./src/Common/typeid_cast.h:42:73
1 0x55cf82aedd6c in DB::TypePromotion<DB::IAST>::CastHelper<DB::ASTFunction, false, false>::value(DB::IAST*) build_docker/./src/Common/TypePromotion.h:38:43
2 0x55cf82aedd6c in std::__1::invoke_result<decltype(&CastHelper<DB::ASTFunction, false>::value), DB::TypePromotion<DB::IAST>::CastHelper<DB::ASTFunction, false, std::is_reference_v<DB::ASTFunction>>, DB::IAST*>::type DB::TypePromotion<DB::IAST>::as<DB::ASTFunction>() build_docker/./src/>
3 0x55cf82aedd6c in DB::MergeTreeIndexConditionSet::operatorFromAST(std::__1::shared_ptr<DB::IAST>&) build_docker/./src/Storages/MergeTree/MergeTreeIndexSet.cpp:602:25
4 0x55cf82ae5bc1 in DB::MergeTreeIndexConditionSet::traverseAST(std::__1::shared_ptr<DB::IAST>&) const build_docker/./src/Storages/MergeTree/MergeTreeIndexSet.cpp:547:9
5 0x55cf82ae5de6 in DB::MergeTreeIndexConditionSet::traverseAST(std::__1::shared_ptr<DB::IAST>&) const build_docker/./src/Storages/MergeTree/MergeTreeIndexSet.cpp:552:13
6 0x55cf82ae06db in DB::MergeTreeIndexConditionSet::MergeTreeIndexConditionSet() build_docker/.>
...
12 0x55cf82aef09c in DB::MergeTreeIndexSet::createIndexCondition() const build_docker/./src/Storages/MergeTree/MergeTreeIndexSet.cpp:703:12
13 0x55cf84951cd4 in DB::buildIndexes()
14 0x55cf84955ed3 in DB::ReadFromMergeTree::selectRangesToReadImpl()
15 0x55cf8494caef in DB::ReadFromMergeTree::selectRangesToRead()
16 0x55cf82a409a9 in DB::MergeTreeDataSelectExecutor::estimateNumMarksToRead()
17 0x55cf827f728d in DB::MergeTreeData::canUseParallelReplicasBasedOnPKAnalysis()
18 0x55cf827f627e in DB::MergeTreeData::getQueryProcessingStage()
19 0x55cf7f2f4969 in DB::InterpreterSelectQuery::getSampleBlockImpl() build_docker/./src/Interpreters/InterpreterSelectQuery.cpp:937:31
20 0x55cf7f2daa00 in DB::InterpreterSelectQuery::InterpreterSelectQuery()
24 0x55cf7f520b98 in DB::InterpreterSelectWithUnionQuery::buildCurrentChildInterpreter()
25 0x55cf7f51b6cd in DB::InterpreterSelectWithUnionQuery::InterpreterSelectWithUnionQuery()
27 0x55cf7f1d4ea9 in DB::InterpreterFactory::get(std::__1::shared_ptr<DB::IAST>&, std::__1::shared_ptr<DB::Context>, DB::SelectQueryOptions const&) build_docker/./src/Interpreters/InterpreterFactory.cpp:162:16
28 0x55cf8012e485 in DB::executeQueryImpl(char const*, char const*, std::__1::shared_ptr<DB::Context>, bool, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) build_docker/./src/Interpreters/executeQuery.cpp:1032:31
29 0x55cf80121bc1 in DB::executeQuery(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::shared_ptr<DB::Context>, bool, DB::QueryProcessingStage::Enum) build_docker/./src/Interpreters/executeQuery.cpp:1229:30
30 0x55cf8389295f in DB::TCPHandler::runImpl() build_docker/./src/Server/TCPHandler.cpp:424:24
31 0x55cf838d7dfb in DB::TCPHandler::run() build_docker/./src/Server/TCPHandler.cpp:2050:9
```
</details>
CI: https://s3.amazonaws.com/clickhouse-test-reports/53214/d99b10c340909ed4ee2e6edf0921e8a2f8561b0d/fuzzer_astfuzzermsan/report.html
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-08-18 18:02:46 +02:00
Austin Kothig
6b42975d33
Change BE-UUID to work the same as LE-UUID. Included high and low getters to provide cleaner code when accessing undertype.
2023-08-18 08:19:46 -07:00
Yakov Olkhovskiy
84cb358c14
Merge pull request #43210 from amosbird/storage_executable_stderr
...
Add different reactions when external command stderr has data
2023-08-18 11:17:46 -04:00
Alexander Tokmakov
a6684f7a37
Merge pull request #53484 from azat/rmt-queue-cleanup
...
Remove outdated code in ReplicatedMergeTreeQueue::initialize()
2023-08-18 18:11:33 +03:00
avogar
2d4167c612
FRemove unneded changes
2023-08-18 14:50:07 +00:00
avogar
aac5131859
Fix for no_more_keys
2023-08-18 13:21:16 +00:00
avogar
7fa72f5cdf
Optimize group by constant keys
2023-08-18 12:47:26 +00:00
Alexander Gololobov
50061b9cb4
Merge pull request #53452 from ClickHouse/such_much_watch
...
Deduplicate same watch callbacks registered multiple times
2023-08-18 14:43:03 +02:00
Alexander Tokmakov
0bd378afd9
Merge pull request #53381 from bakwc/bakwc-patch-2
...
Fixed system.data_skipping_indices for MaterializedMySQL
2023-08-18 14:22:40 +03:00
Duc Canh Le
2d353f8c43
Merge branch 'master' into limit_concurent_background_optimize
2023-08-18 09:25:27 +00:00
Amos Bird
b6de70b042
StorageExecutable check_exit_code default to false
2023-08-18 15:38:47 +08:00
Amos Bird
c43bf153f5
Refactor
2023-08-18 15:38:46 +08:00
Amos Bird
dd0c71b32a
Add error_exit_reaction
2023-08-18 15:38:46 +08:00
Amos Bird
476f3cedc1
Various reactions when executable stderr has data
2023-08-18 15:38:45 +08:00
Alexey Milovidov
a882bf1bb8
Merge pull request #53495 from ClickHouse/i_hate_materialized_views
...
Fix a race condition between RESTART REPLICAS and DROP DATABASE
2023-08-18 06:40:04 +03:00
Igor Nikonov
06c98956b3
Merge remote-tracking branch 'origin/master' into fix-parallel-replicas-multiply-result
2023-08-17 20:42:50 +00:00
Jianfei Hu
0c81546960
remove the comments.
...
Signed-off-by: Jianfei Hu <hujianfei258@gmail.com>
2023-08-17 20:13:33 +00:00
pufit
a52249872e
Merge pull request #53313 from jorisgio/joris/max_threads_for_indexes
...
add max_threads_for_indexes settings
2023-08-17 13:59:53 -04:00
avogar
60b0b88d50
Clean up
2023-08-17 16:59:57 +00:00
avogar
4c32097df3
Use filter by file/path before reading in url/file/hdfs table functions, reduce code duplication
2023-08-17 16:54:43 +00:00
Alexander Gololobov
54e2be4e32
Use WatchCallbackPtr
2023-08-17 17:32:04 +02:00
Alexander Tokmakov
89c365803f
Merge pull request #53424 from ClickHouse/exception_message_patterns6
...
Less exceptions with runtime format string
2023-08-17 17:58:04 +03:00
Igor Nikonov
3e9ae1751e
Merge remote-tracking branch 'origin/master' into fix-parallel-replicas-multiply-result
2023-08-17 14:45:37 +00:00
Filipp Ozinov
ce87451b66
Merge branch 'ClickHouse:master' into bakwc-patch-2
2023-08-17 18:37:00 +04:00
Anton Popov
0865c0794e
Merge remote-tracking branch 'upstream/master' into HEAD
2023-08-17 14:08:59 +00:00
Alexander Tokmakov
72b52250ea
Merge pull request #53501 from azat/dist-flush_on_detach
...
Add ability to turn off flush of Distributed on DETACH/DROP/server shutdown
2023-08-17 16:56:04 +03:00
Azat Khuzhin
fcb66379d1
Remove outdated code in ReplicatedMergeTreeQueue::initialize()
...
Initially added in #28817
But got cleaned up in #41981
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-08-17 13:45:26 +02:00
Robert Schulze
ec7daed52f
Merge pull request #50276 from ClibMouse/feature/mergetree-checksum-big-endian-support
...
MergeTree & SipHash checksum big-endian support
2023-08-17 13:27:18 +02:00
Robert Schulze
42644bfbf5
Rename file to MergeTreeIndexUSearch.h/cpp
2023-08-17 09:54:12 +00:00
Robert Schulze
f420006e09
Make the code consistent with Annoy index and fix build
2023-08-17 09:42:13 +00:00
Robert Schulze
456a02da1b
Merge remote-tracking branch 'rschu1ze/master'
2023-08-17 09:30:27 +00:00
Robert Schulze
1215a23de6
Merge pull request #53422 from rschu1ze/consistency-fixes-for-caches
...
Refactorings for configuration of in-memory caches
2023-08-17 11:08:32 +02:00
Mingliang Pan
2db0b4f52a
Update StorageEmbeddedRocksDB.cpp
...
add drop function to close file before drop table
2023-08-17 15:14:14 +08:00
Mingliang Pan
349f3b9c02
Update StorageEmbeddedRocksDB.h
...
add drop method to close file before drop table
2023-08-17 15:12:42 +08:00
Azat Khuzhin
17ca2661a1
Add ability to turn off flush of Distributed on DETACH/DROP/server shutdown
...
Sometimes you can have tons of data there, i.e. few TiBs, and sending
them on server shutdown does not looks sane (maybe there is a bug and
you need to update/restart to fix flushing).
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-08-17 08:58:06 +02:00
Igor Nikonov
49f7b4acd6
Merge remote-tracking branch 'origin/master' into fix-parallel-replicas-multiply-result
2023-08-16 23:31:49 +00:00
Igor Nikonov
21ef1f1d1c
Remove unnecessary Scalars usage for cluster_for_parallel_replicas
2023-08-16 23:24:01 +00:00
Alexander Tokmakov
d7272410bd
the worst fix for a deadlock ever
2023-08-17 00:42:51 +02:00
Robert Schulze
8a58e0df3e
Merge remote-tracking branch 'rschu1ze/master' into consistency-fixes-for-caches
2023-08-16 21:26:04 +00:00
Robert Schulze
353854520c
Merge remote-tracking branch 'rschu1ze/master' into factorize-constants
2023-08-16 21:24:21 +00:00
Alexander Tokmakov
69bc4f5139
Merge branch 'master' into exception_message_patterns6
2023-08-16 22:58:24 +02:00
Jianfei Hu
fcad273724
remove the comments.
...
Signed-off-by: Jianfei Hu <hujianfei258@gmail.com>
2023-08-16 17:49:28 +00:00
Jianfei Hu
1e600351d9
try again the if value.empty() in the beginning.
...
Signed-off-by: Jianfei Hu <hujianfei258@gmail.com>
2023-08-16 17:49:28 +00:00
Jianfei Hu
49502ac0ce
revert unrelated style change.
...
Signed-off-by: Jianfei Hu <hujianfei258@gmail.com>
2023-08-16 17:49:28 +00:00
Jianfei Hu
02b7bd94eb
suspect some root cause of value[0].
...
Signed-off-by: Jianfei Hu <hujianfei258@gmail.com>
2023-08-16 17:49:28 +00:00
Jianfei Hu
7227cc6d2b
try parts[0] instead of front().
...
Signed-off-by: Jianfei Hu <hujianfei258@gmail.com>
2023-08-16 17:49:28 +00:00
Jianfei Hu
afaaef6875
wip checking whether system.merge cause sigsev.
...
Signed-off-by: Jianfei Hu <hujianfei258@gmail.com>
2023-08-16 17:49:28 +00:00
Alexey Milovidov
5e3a69b4c4
Update MergeList.cpp
2023-08-16 17:49:28 +00:00
Alexey Milovidov
bc77aab2e8
Update MergeList.h
2023-08-16 17:49:28 +00:00
Alexey Milovidov
7376b7ec8c
Update MergeList.h
2023-08-16 17:49:28 +00:00
Alexey Milovidov
664bcfc8b3
Update MergeList.cpp
2023-08-16 17:49:28 +00:00
Alexey Milovidov
ecd1fb56b4
Update MergeList.cpp
2023-08-16 17:49:28 +00:00
Jianfei Hu
bd4df60df6
fix merges
...
Signed-off-by: Jianfei Hu <hujianfei258@gmail.com>
2023-08-16 17:49:28 +00:00
Jianfei Hu
59a81b82bc
wip the merge partition
...
Signed-off-by: Jianfei Hu <hujianfei258@gmail.com>
2023-08-16 17:49:28 +00:00
Robert Schulze
5a87a09c84
Tweak exception handling
2023-08-16 17:16:52 +00:00
Andrey Zvonov
eb9c837d20
Merge branch 'ClickHouse:master' into zvonand-globs-small-fix
2023-08-16 19:22:00 +03:00
Davit Vardanyan
a0ec274766
Refactor: Naming
2023-08-16 17:30:56 +04:00
Davit Vardanyan
620733dd46
Fix: Exception handling
2023-08-16 17:30:29 +04:00
Robert Schulze
ea69eb095c
Merge remote-tracking branch 'ClickHouse/master' into usearch
2023-08-16 08:07:57 +00:00
Alexander Tokmakov
b32bdd1b0a
Merge pull request #53164 from ClickHouse/fixes_for_detached_parts
...
Fixes for detach/attach partition and broken detached parts cleanup
2023-08-16 00:09:13 +03:00
Robert Schulze
56a6d20137
Merge remote-tracking branch 'rschu1ze/master' into factorize-constants
2023-08-15 18:21:25 +00:00
Igor Nikonov
49134711dc
Polishing
2023-08-15 17:37:01 +00:00
Robert Schulze
f558048bea
Merge remote-tracking branch 'rschu1ze/master' into consistency-fixes-for-caches
2023-08-15 17:12:52 +00:00
Igor Nikonov
9825f8c76e
Remove debug logging
2023-08-15 17:03:32 +00:00
kssenii
2d74472ffb
Merge remote-tracking branch 'upstream/master' into system-sync-cache
2023-08-15 20:31:26 +04:00
zvonand
0efaa5ab45
Merge branch 'master' of github.com:ClickHouse/ClickHouse into zvonand-globs-small-fix
2023-08-15 21:20:52 +05:00
Andrey Zvonov
34b1811a53
Update src/Storages/StorageFile.cpp
...
seems like typo
Co-authored-by: SmitaRKulkarni <64093672+SmitaRKulkarni@users.noreply.github.com>
2023-08-15 19:10:14 +03:00
Alexander Tokmakov
83c10dad18
Merge branch 'master' into exception_message_patterns6
2023-08-15 15:20:09 +02:00
Alexander Tokmakov
bf40767f10
fix another race
2023-08-15 14:27:49 +02:00
Igor Nikonov
1678cda8b3
Merge remote-tracking branch 'origin/master' into fix-parallel-replicas-multiply-result
2023-08-15 12:03:35 +00:00
Davit Vardanyan
48c62fd75e
Add: USearch
2023-08-15 16:00:27 +04:00
Kruglov Pavel
cac2628b41
Merge pull request #53426 from ClickHouse/analyzer-fix-virtuals-distr
...
Analyzer: fix virtual columns in StorageDistributed
2023-08-15 13:55:42 +02:00
Robert Schulze
8992cc5af4
Factorize constants
2023-08-15 08:56:40 +00:00
Robert Schulze
f317903073
Merge branch 'ClickHouse:master' into consistency-fixes-for-caches
2023-08-15 10:25:08 +02:00
Anton Popov
2e22b17d57
add docs and settings randomizations
2023-08-15 00:38:07 +00:00
Anton Popov
5e90a6f88a
Merge remote-tracking branch 'upstream/master' into HEAD
2023-08-15 00:27:35 +00:00
Igor Nikonov
c94994afcf
Propagate setting cluster_for_parallel_replicas to shards
2023-08-14 22:50:46 +00:00
Alexander Tokmakov
376202f739
fix creation of empty parts
2023-08-15 00:26:17 +02:00
Alexander Tokmakov
894ec8e0d0
less exceptions with runtime format string
2023-08-14 21:45:59 +02:00
Dmitry Novik
1244828555
Analyzer: fix virtual columns in StorageDistributed
2023-08-14 19:27:05 +02:00
Alexander Tokmakov
f0f2cd8c13
Merge branch 'master' into fixes_for_detached_parts
2023-08-14 19:20:45 +02:00
ltrk2
2846ea49b4
Merge branch 'master' into feature/mergetree-checksum-big-endian-support
2023-08-14 13:02:30 -04:00
Robert Schulze
3472a928d8
Cosmetics: Rename dropCache() to clearCaches()
...
"dropCache()" is a bad name, it sounds like that the cache can no longer
be used after the call. Not true, we only "clear" the cache.
2023-08-14 15:42:32 +00:00
Duc Canh Le
cd535bea7d
limit number of concurrent background partition optimize
2023-08-14 10:21:13 +00:00
Filipp Ozinov
5a180b53ed
Style fix
2023-08-14 12:30:29 +04:00
Andrey Zvonov
413b0f227b
Merge branch 'ClickHouse:master' into zvonand-globs-small-fix
2023-08-14 10:59:44 +03:00
Victor Krasnov
247cd63ca1
Merge branch 'master' into ADQM-987
...
# Conflicts:
# src/Interpreters/Context.cpp
# src/Interpreters/InterpreterSystemQuery.cpp
2023-08-14 10:33:56 +03:00
Alexey Milovidov
9656bd2738
Merge pull request #53372 from ClickHouse/Fix_lazy_database_and_system_table_query
...
Fix loading lazy database during system.table select query
2023-08-14 08:03:19 +03:00
Yakov Olkhovskiy
300399d1f5
Merge pull request #52329 from ClibMouse/feature/mergetree-checksum-on-big-endian
...
Implement endianness-indepedent support for MergeTree checksums
2023-08-13 23:59:05 -04:00
Filipp Ozinov
0cccb2ac3b
Fixed system.data_skipping_indices for MaterializedMySQL
...
Related to #53350
2023-08-14 01:57:41 +04:00
Smita Kulkarni
c25bb44e55
Fixed style check
2023-08-13 21:45:33 +02:00
Smita Kulkarni
ee772b7376
Avoid loading tables from lazy database when not needed
2023-08-13 20:15:58 +02:00
kssenii
8535e23f95
Merge remote-tracking branch 'upstream/master' into system-sync-cache
2023-08-13 21:10:42 +04:00
Andrey Zvonov
3d902e65e4
Merge branch 'ClickHouse:master' into zvonand-globs-small-fix
2023-08-12 23:12:47 +03:00
Alexey Milovidov
17127cf23f
Merge pull request #53337 from hanfei1991/hanfei/improve_move_to_prewhere
...
improve `move_primary_key_columns_to_end_of_prewhere`
2023-08-12 17:45:30 +03:00
Han Fei
0b3f9918a3
fix style
2023-08-12 14:40:02 +02:00
robot-ch-test-poll
967067bbb8
Merge pull request #53250 from evillique/postgresql-single-quotes
...
Fix character escaping in the PostgreSQL engine
2023-08-12 13:15:49 +02:00
Alexey Milovidov
3c2dd4e543
Update MergeTreeWhereOptimizer.cpp
2023-08-12 03:40:19 +03:00
Alexey Milovidov
6176971b3f
Update StoragePostgreSQL.cpp
2023-08-12 03:33:58 +03:00
Alexey Milovidov
4b59d265e1
Merge pull request #49723 from DimasKovas/fix_storage_buffer_reschedule
...
Some unclear change in StorageBuffer::reschedule() for something
2023-08-12 03:28:23 +03:00
Alexey Milovidov
e5f33511c6
Merge pull request #53242 from evillique/alter-exception
...
Fix LOGICAL_ERROR exception in ALTER query
2023-08-12 03:18:32 +03:00
Han Fei
e70dc7d507
improve move_primary_key_columns_to_end_of_prewhere
2023-08-11 17:36:52 +02:00
Sergei Trifonov
771710b377
Merge branch 'master' into concurrency-control-controllable
2023-08-11 16:50:13 +02:00
Andrey Zvonov
02c511a008
Merge branch 'master' into zvonand-globs-small-fix
2023-08-11 15:56:44 +02:00
Anton Popov
8744a0269a
Merge pull request #52530 from CurtizJ/fix-alter-delete-skip-index
...
Fix recalculation of skip indexes and projections in `ALTER DELETE` queries
2023-08-11 15:23:05 +02:00
zvonand
086bf6cb8c
added ignore_eacces_multidirectory_globs
2023-08-11 15:49:21 +03:00
ltrk2
1dc1b54c68
Merge branch 'master' into feature/mergetree-checksum-big-endian-support
2023-08-11 07:48:49 -04:00
ltrk2
94dbef1e50
Merge branch 'master' into feature/mergetree-checksum-on-big-endian
2023-08-11 07:47:48 -04:00
Joris Giovannangeli
602570b462
add max_threads for index settings
2023-08-11 10:20:07 +02:00
Alexey Milovidov
aae35f3b1c
Merge pull request #53184 from ClickHouse/better-ccache
...
Better usage of ccache
2023-08-11 04:43:45 +03:00
Ilya Golshtein
ade9c3d970
system_kafka_consumers: per consumer librdkafka stat
2023-08-10 22:35:05 +00:00
Ilya Golshtein
3bced2f08c
system_kafka_consumers: tidy fix, one partition for stable test
2023-08-10 22:35:05 +00:00
Ilya Golshtein
978d8bf9a6
system_kafka_consumers: cleanup with exceptions
2023-08-10 22:35:05 +00:00
Ilya Golshtein
7dab2dc041
system_kafka_consumers: exceptions seem to work
2023-08-10 22:35:05 +00:00
Ilya Golshtein
80829a6bf7
system_kafka_consumers: no offset_committed, new test with MV
2023-08-10 22:35:05 +00:00
Ilya Golshtein
6d6aae24f8
system_kafka_consumers: no friendship with StorageKafka
2023-08-10 22:35:05 +00:00
Ilya Golshtein
ae3e6f0fac
system_kafka_consumers: getMemberId introduced to avoid leak
2023-08-10 22:35:05 +00:00
Ilya Golshtein
b325ddceb4
system_kafka_consumers: small thing to make clang tidy happy
2023-08-10 22:35:05 +00:00
Ilya Golshtein
46be5e5782
system_kafka_consumers: cleanup, test should be more stable
2023-08-10 22:35:05 +00:00
Ilya Golshtein
803a80b098
system_kafka_consumers: refactored with getStat, test passed
2023-08-10 22:35:05 +00:00
Ilya Golshtein
a86548a733
system_kafka_consumers: statistics.interval.ms default, tiny cleanup
2023-08-10 22:35:05 +00:00
Ilya Golshtein
dd6727be3e
system_kafka_consumers: style check fixes
2023-08-10 22:35:05 +00:00
Ilya Golshtein
bba4dcacc7
system_kafka_consumers: small code cleanup
2023-08-10 22:35:05 +00:00
Ilya Golshtein
3f205f4997
system_kafka_consumers: tiny doc changes
2023-08-10 22:35:05 +00:00
Ilya Golshtein
b71c2bedfc
system_kafka_consumers: rebalance test is ok, statistics_interval_ms
2023-08-10 22:35:05 +00:00
Ilya Golshtein
83566e132a
system_kafka_consumers: all_consumers introduced, offsets work
2023-08-10 22:35:05 +00:00
Ilya Golshtein
de49d7e5a0
system_kafka_consumers: minor change
2023-08-10 22:35:05 +00:00
Ilya Golshtein
c821c7788a
system_kafka_consumers: json commented out
2023-08-10 22:35:05 +00:00
Ilya Golshtein
ae6e0469ad
system_kafka_consumers: rdfakfa stat with attempt to use json
2023-08-10 22:35:05 +00:00
Ilya Golshtein
6258ab6cb4
system_kafka_consumers: doc link fix, USE_RDKAFKA
2023-08-10 22:35:05 +00:00
Ilya Golshtein
40fd53c3e3
system_kafka_consumers: style check
2023-08-10 22:35:05 +00:00
Ilya Golshtein
88f5dd5db8
system_kafka_consumers: integration tests passed
2023-08-10 22:35:05 +00:00
Ilya Golshtein
957787d96a
system_kafka_consumers: kind of works, test added
2023-08-10 22:35:05 +00:00
Ilya Golshtein
36fb330811
system_kafka_consumers: assignments
2023-08-10 22:35:05 +00:00
Ilya Golshtein
a947c7a91e
system_kafka_consumers: start integrating with real kafka
2023-08-10 22:35:05 +00:00
Ilya Golshtein
2304f342c9
system_kafka_consumers: table populated by fake data
2023-08-10 22:35:05 +00:00
Ilya Golshtein
9d68d50ae7
system_kafka_consumers: initial
2023-08-10 22:35:04 +00:00
Alexander Tokmakov
5a8b8203b2
fix
2023-08-10 23:22:51 +02:00
Alexander Tokmakov
3d59ebe108
fix
2023-08-10 20:11:22 +02:00
Nikolay Degterinsky
4f0be777c5
Fix build
2023-08-10 16:39:43 +00:00
Antonio Andelic
478a5006b1
Merge branch 'master' into improve-archive-reading
2023-08-10 15:52:30 +00:00
Alexander Tokmakov
6ae0ba1163
Merge branch 'master' into fixes_for_detached_parts
2023-08-10 17:01:05 +02:00
ltrk2
a2054c04dd
Merge branch 'master' into feature/mergetree-checksum-big-endian-support
2023-08-10 10:21:34 -04:00
ltrk2
b780b64232
Merge branch 'master' into feature/mergetree-checksum-on-big-endian
2023-08-10 10:21:23 -04:00
Alexey Milovidov
bb57caa83e
Merge pull request #53180 from ClickHouse/ditch-tons-of-garbage
...
Ditch tons of garbage
2023-08-10 16:59:51 +03:00
Alexey Milovidov
f35df23974
Merge branch 'master' into better-ccache
2023-08-10 16:52:32 +03:00
pufit
af51268da1
Merge pull request #51912 from ClickHouse/pufit/role-from-storage
...
Create, Drop, or Move access entities between different access storages.
2023-08-10 09:47:26 -04:00
Kseniia Sumarokova
3925a1641f
Merge branch 'master' into system-sync-cache
2023-08-10 14:33:13 +02:00
zvonand
e4a1780bd7
update docs according to the PR + fix the same for HDFS
2023-08-10 15:29:27 +03:00
Alexander Tokmakov
4b30900fed
Revert "Improve CHECK TABLE system query"
2023-08-10 14:44:16 +03:00
Kseniia Sumarokova
aed955a331
Merge branch 'master' into system-sync-cache
2023-08-10 13:18:35 +02:00
Andrey Zvonov
90dcc2ac81
Merge branch 'master' into zvonand-globs-small-fix
2023-08-10 12:30:12 +02:00
Victor Krasnov
a89d711113
Move BackupsWorker::Info to a separate file BackupOperationInfo.h
2023-08-10 13:29:20 +03:00
