thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-02 12:32:04 +00:00
Code Issues Packages Projects Releases Wiki Activity
123,188 Commits 9,194 Branches 1,782 Tags 2.2 GiB
68e3af56d4
Commit Graph

46085 Commits

Author SHA1 Message Date
Kruglov Pavel
21413636ac
Remove debug logging 2023-08-21 13:30:08 +02:00
Kruglov Pavel
c68456a20a
Merge pull request #52692 from Avogar/variable-number-of-volumns-more-formats 
Allow variable number of columns in more formats, make it work with schema inference
 2023-08-21 13:28:35 +02:00
Kruglov Pavel
47b551180b
Merge pull request #52117 from KevinyhZou/support_from_to_utc_timestamp 
Support From/To UTC Timestamp function like spark
 2023-08-21 13:16:57 +02:00
Robert Schulze
2c330516b8
Another compile fix 2023-08-21 11:12:39 +00:00
Nikolai Kochetov
388e2a09d6 Merge branch 'master' into not-ready-set-bug 2023-08-21 10:59:51 +00:00
Yarik Briukhovetskyi
af1d23720b
Merge branch 'ClickHouse:master' into fixed_hints_db_name 2023-08-21 13:57:39 +03:00
Robert Schulze
066ec559ad
Do not reset Annoy index during build-up with > 1 mark 2023-08-21 10:05:43 +00:00
Amos Bird
cb7d5a0f21
Get rid of unneeded include 2023-08-21 18:01:10 +08:00
Amos Bird
882e0e7a0c
No more split of functions 2023-08-21 18:00:37 +08:00
taiyang-li
e5a399d0e9 change as requested 2023-08-21 14:51:25 +08:00
taiyang-li
f2e4315d59 fix building 2023-08-21 14:11:48 +08:00
taiyang-li
3cd9eb9b84 fix style 2023-08-21 12:12:31 +08:00
taiyang-li
3781245027 change as request 2023-08-21 12:10:39 +08:00
taiyang-li
f723e8d43a change as request 2023-08-21 12:09:02 +08:00
Amos Bird
076a67bdaa
Consistent file management in CMake 2023-08-21 11:45:08 +08:00
kevinyhzou
48cb9b59f4 Support from/to utc timestamp 2023-08-21 09:28:00 +08:00
Alexey Milovidov
be2ef82d80 Add missing documentation for distributed_ddl_output_mode 2023-08-21 01:18:52 +02:00
Igor Nikonov
b074e44c1b
Merge pull request #53005 from ClickHouse/fix-parallel-replicas-multiply-result 
Parallel reading from replicas over Distributed
 2023-08-21 01:10:53 +02:00
Maksim Kita
f32effb627 AARCH64 Neon memequal wide 2023-08-20 12:45:18 +03:00
Kseniia Sumarokova
ac4f727091
Merge pull request #53273 from kssenii/impove-filecache-removal 
Improve fs cache cleanup
 2023-08-20 10:31:07 +02:00
Alexey Milovidov
e81fcfd23d
Merge pull request #53601 from ClickHouse/remove-unneeded-columns-from-logs 
Remove unneeded columns from logs
 2023-08-20 10:37:11 +03:00
Alexey Milovidov
cf0e9e5f37
Merge pull request #53604 from ClickHouse/metrics-system-database 
Add asynchronous metrics about system database
 2023-08-20 10:35:57 +03:00
gyfis
efa53f873c
💄 [S3::URI] Fix comment typos around versionId 2023-08-20 09:08:57 +02:00
irenjj
a9b65ee929 Add Alias for system.clusters. 2023-08-20 14:47:07 +08:00
irenjj
71459c6d65 Revert 2023-08-20 14:35:59 +08:00
irenjj
e64a1fc26d Add column 'name' to system.clusters as an alias to cluster. 2023-08-20 11:52:14 +08:00
Alexey Milovidov
82166f731a Add metrics about system database 2023-08-20 05:05:54 +02:00
Alexey Milovidov
5d68f7c04d Remove unneeded columns from logs 2023-08-20 04:11:39 +02:00
Alexey Milovidov
d133cafd9c Better documentation about redirects 2023-08-20 00:38:18 +02:00
Igor Nikonov
a69ea02418 Merge remote-tracking branch 'origin/master' into fix-parallel-replicas-multiply-result 2023-08-19 21:43:25 +00:00
Yakov Olkhovskiy
257380df03
Merge pull request #50267 from HarryLeeIBM/hlee-s390x-proto-endian 
Fix endian issues in native protocol
 2023-08-19 11:16:24 -04:00
Daniël van Eeden
212686c5c4 Improve MySQL Compatibility by supporting SCHEMA() 
Basically `DATABASE()` and `SCHEMA()` are aliases in MySQL.

This helps with allowing MySQL Shell to get one step closer to a
functioning connection.

To see what MySQL Shell does:

```
mysqlsh --verbose=4 --log-sql=unfiltered mysql://default@127.0.0.1:9004
```

This might also bring other tools a step closer to working with the
MySQL compatibility of ClickHouse.

Ref: #9336

Signed-off-by: Daniël van Eeden <git@myname.nl>
 2023-08-19 16:24:09 +02:00
Kseniia Sumarokova
aa01333a63
Merge branch 'master' into impove-filecache-removal 2023-08-19 13:34:22 +04:00
taiyang-li
9866d2727e change as request 2023-08-19 17:31:19 +08:00
Kseniia Sumarokova
c0b8d7eddf
Merge pull request #50999 from ilejn/system_kafka_consumers 
system.kafka_consumers table to monitor kafka consumers
 2023-08-19 13:31:18 +04:00
Kseniia Sumarokova
3d2fb2eced
Merge pull request #53124 from kssenii/allowed-caches-dir-for-dynamic-disks 
Allow to restrict fs caches paths (mainly useful for dynamic disks)
 2023-08-19 13:30:23 +04:00
Alexey Milovidov
c42c4046fe
Merge pull request #53573 from amosbird/fix_53454 
Proper destruction of task in ShellCommandSource
 2023-08-19 12:19:29 +03:00
robot-clickhouse-ci-1
75b748fc9a
Merge pull request #53572 from ClickHouse/fix-interpolate-precedence 
Fix: interpolate expression takes source column instead of same name aliased from select expression.
 2023-08-19 11:12:32 +02:00
Alexey Milovidov
ec0561b3f1
Merge pull request #53569 from ClickHouse/dashboard-better 
The dashboard just got better
 2023-08-19 12:11:49 +03:00
Amos Bird
052281896a
Proper destruction of task 2023-08-19 13:56:57 +08:00
kssenii
5382442da0 Merge remote-tracking branch 'upstream/master' into allowed-caches-dir-for-dynamic-disks 2023-08-19 08:44:36 +04:00
kssenii
1a6dce3dc4 Merge remote-tracking branch 'upstream/master' into impove-filecache-removal 2023-08-19 08:42:50 +04:00
Yakov Olkhovskiy
25316a29f3 result columns should take precedence to account same name aliasing 2023-08-19 02:01:10 +00:00
Michael Kolupaev
a1522e22ea
Merge pull request #53281 from Avogar/batch-small-parquet-row-groups 
Optimize reading small row groups by batching them together in Parquet
 2023-08-18 17:15:42 -07:00
Alexey Milovidov
1183c18a18 Update the docs 2023-08-19 01:39:21 +02:00
Alexey Milovidov
00e2e3184b
Merge pull request #53439 from canhld94/timeout_overflow_break_subquery 
Fix timeout_overflow_mode when having subquery in the rhs of IN
 2023-08-19 02:24:42 +03:00
Alexey Milovidov
40d13975fb
Merge pull request #53548 from ClickHouse/vdimir/join_on_cond_sparse_fix 
Fix crash in join on sparse column
 2023-08-19 02:08:21 +03:00
Alexey Milovidov
8520a7baca
Merge pull request #53559 from azat/skip-index-set-fix 
Fix possible UB in Set skipping index for functions with incorrect args
 2023-08-19 02:06:18 +03:00
Alexey Milovidov
6fb24fe74d
Merge pull request #53560 from azat/skip-index-inverted-fix-ub 
Fix possible UB in inverted indexes (experimental feature)
 2023-08-19 02:06:04 +03:00
Igor Nikonov
aebda10f7c Merge remote-tracking branch 'origin/master' into fix-parallel-replicas-multiply-result 2023-08-18 23:00:58 +00:00
kothiga
a420795416
Trailing whitespace fix. 2023-08-18 14:08:02 -07:00
kothiga
f33c585bc5
Addressing feedback. 2023-08-18 13:50:31 -07:00
Alexey Milovidov
7ad8ea78bb
Merge pull request #48990 from incfly/partition 
Add partition to the system part_log and merge table.
 2023-08-18 21:38:53 +03:00
avogar
8e445b5461 Fixes 2023-08-18 17:49:40 +00:00
Nikolai Kochetov
c99c3112ef
Merge pull request #53550 from ClickHouse/group_array_max_element_size 
Add server setting `aggregate_function_group_array_max_element_size`.
 2023-08-18 19:16:24 +02:00
Alexander Tokmakov
14590305ad
Merge pull request #52975 from aalexfvk/ignore_on_cluster_for_replicated_entity_queries 
Ignore ON CLUSTER clause in queries for management of replicated entities (UDF and Access)
 2023-08-18 19:29:41 +03:00
Azat Khuzhin
c78b94bed6 Fix possible UB in inverted indexes (experimental feature) 
It is possible to have incorrect types there in case of index contains
functions, add a proper check after value had been changed.

<details>

<summary>MSan report</summary>

```
==182==WARNING: MemorySanitizer: use-of-uninitialized-value
    0 0x55658547c59e in DB::MergeTreeConditionInverted::traverseASTEquals() build_docker/./src/Storages/MergeTree/MergeTreeIndexInverted.cpp:551:26
    1 0x556585475566 in DB::MergeTreeConditionInverted::traverseAtomAST() build_docker/./src/Storages/MergeTree/MergeTreeIndexInverted.cpp:438:21
    2 0x55658547e73e in DB::MergeTreeConditionInverted::MergeTreeConditionInverted()::$_1::operator()(DB::RPNBuilderTreeNode const&, DB::MergeTreeConditionInverted::RPNElement&) const build_docker/./src/Storages/MergeTree/MergeTreeIndexInverted.cpp:228:73
    9 0x556585489e35 in DB::RPNBuilder<DB::MergeTreeConditionInverted::RPNElement>::traverseTree(DB::RPNBuilderTreeNode const&) build_docker/./src/Storages/MergeTree/RPNBuilder.h:252:14
    10 0x556585489921 in DB::RPNBuilder<DB::MergeTreeConditionInverted::RPNElement>::traverseTree(DB::RPNBuilderTreeNode const&) build_docker/./src/Storages/MergeTree/RPNBuilder.h:239:21
    11 0x556585489921 in DB::RPNBuilder<DB::MergeTreeConditionInverted::RPNElement>::traverseTree(DB::RPNBuilderTreeNode const&) build_docker/./src/Storages/MergeTree/RPNBuilder.h:239:21
    12 0x5565854804e6 in DB::RPNBuilder<DB::MergeTreeConditionInverted::RPNElement>::RPNBuilder(std::__1::shared_ptr<DB::IAST> const&, std::__1::shared_ptr<DB::Context const>, DB::Block, std::__1::shared_ptr<DB::PreparedSets>, std::__1::function<bool (DB::RPNBuilderTreeNode const&, DB::MergeTreeConditionInverted::RPNElement&)> const&) build_docker/./src/Storages/MergeTree/RPNBuilder.h:218:9
    13 0x55658546fb87 in DB::MergeTreeConditionInverted::MergeTreeConditionInverted(DB::SelectQueryInfo const&, std::__1::shared_ptr<DB::Context const>, DB::Block const&, DB::GinFilterParameters const&, DB::ITokenExtractor const*) build_docker/./src/Storages/MergeTree/MergeTreeIndexInverted.cpp:223:28
    19 0x55658547ceda in DB::MergeTreeIndexInverted::createIndexCondition(DB::SelectQueryInfo const&, std::__1::shared_ptr<DB::Context const>) const build_docker/./src/Storages/MergeTree/MergeTreeIndexInverted.cpp:716:12
    20 0x556587125734 in DB::buildIndexes(std::__1::optional<DB::ReadFromMergeTree::Indexes>&, std::__1::shared_ptr<DB::ActionsDAG>, DB::MergeTreeData const&, std::__1::shared_ptr<DB::Context const> const&, DB::SelectQueryInfo const&, std::__1::shared_ptr<DB::StorageInMemoryMetadata const> const&) build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:1292:48
    21 0x556587121aaa in DB::ReadFromMergeTree::applyFilters() build_docker/./src/Processors/QueryPlan/ReadFromMergeTree.cpp:1305:5
    22 0x55658726cf4f in DB::QueryPlanOptimizations::optimizeTreeThirdPass(DB::QueryPlan::Node&, std::__1::list<DB::QueryPlan::Node, std::__1::allocator<DB::QueryPlan::Node>>&) build_docker/./src/Processors/QueryPlan/Optimizations/optimizeTree.cpp:201:38
    23 0x5565870bc489 in DB::QueryPlan::optimize(DB::QueryPlanOptimizationSettings const&) build_docker/./src/Processors/QueryPlan/QueryPlan.cpp:485:5
    24 0x5565870b919a in DB::QueryPlan::buildQueryPipeline(DB::QueryPlanOptimizationSettings const&, DB::BuildQueryPipelineSettings const&) build_docker/./src/Processors/QueryPlan/QueryPlan.cpp:161:5
    25 0x55658206385a in DB::InterpreterSelectWithUnionQuery::execute() build_docker/./src/Interpreters/InterpreterSelectWithUnionQuery.cpp:379:31
    26 0x556582c15c50 in DB::executeQueryImpl(char const*, char const*, std::__1::shared_ptr<DB::Context>, bool, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) build_docker/./src/Interpreters/executeQuery.cpp:751:40
    27 0x556582c09681 in DB::executeQuery(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::shared_ptr<DB::Context>, bool, DB::QueryProcessingStage::Enum) build_docker/./src/Interpreters/executeQuery.cpp:1173:30
    28 0x5565860e7bd7 in DB::TCPHandler::runImpl() build_docker/./src/Server/TCPHandler.cpp:421:24
    29 0x556586126e9e in DB::TCPHandler::run() build_docker/./src/Server/TCPHandler.cpp:2057:9
```

</details>

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-08-18 18:15:17 +02:00
Nikolai Kochetov
26c7dd2ebb Fixing tidy 2023-08-18 16:08:17 +00:00
Azat Khuzhin
1e2b0c8dc4 Fix possible UB in Set skipping index for functions with incorrect args 
Usually functions are validated before index analysis, however it is
not the case for parallel replicas, and it requires additional checks
before interpreting function arguments.

<details>

<summary>stack trace</summary>

```
==172==WARNING: MemorySanitizer: use-of-uninitialized-value
    0 0x55cf82aedd6c in DB::ASTFunction* typeid_cast<DB::ASTFunction*, DB::IAST>(DB::IAST*) build_docker/./src/Common/typeid_cast.h:42:73
    1 0x55cf82aedd6c in DB::TypePromotion<DB::IAST>::CastHelper<DB::ASTFunction, false, false>::value(DB::IAST*) build_docker/./src/Common/TypePromotion.h:38:43
    2 0x55cf82aedd6c in std::__1::invoke_result<decltype(&CastHelper<DB::ASTFunction, false>::value), DB::TypePromotion<DB::IAST>::CastHelper<DB::ASTFunction, false, std::is_reference_v<DB::ASTFunction>>, DB::IAST*>::type DB::TypePromotion<DB::IAST>::as<DB::ASTFunction>() build_docker/./src/>
    3 0x55cf82aedd6c in DB::MergeTreeIndexConditionSet::operatorFromAST(std::__1::shared_ptr<DB::IAST>&) build_docker/./src/Storages/MergeTree/MergeTreeIndexSet.cpp:602:25
    4 0x55cf82ae5bc1 in DB::MergeTreeIndexConditionSet::traverseAST(std::__1::shared_ptr<DB::IAST>&) const build_docker/./src/Storages/MergeTree/MergeTreeIndexSet.cpp:547:9
    5 0x55cf82ae5de6 in DB::MergeTreeIndexConditionSet::traverseAST(std::__1::shared_ptr<DB::IAST>&) const build_docker/./src/Storages/MergeTree/MergeTreeIndexSet.cpp:552:13
    6 0x55cf82ae06db in DB::MergeTreeIndexConditionSet::MergeTreeIndexConditionSet() build_docker/.>
    ...
    12 0x55cf82aef09c in DB::MergeTreeIndexSet::createIndexCondition() const build_docker/./src/Storages/MergeTree/MergeTreeIndexSet.cpp:703:12
    13 0x55cf84951cd4 in DB::buildIndexes()
    14 0x55cf84955ed3 in DB::ReadFromMergeTree::selectRangesToReadImpl()
    15 0x55cf8494caef in DB::ReadFromMergeTree::selectRangesToRead()
    16 0x55cf82a409a9 in DB::MergeTreeDataSelectExecutor::estimateNumMarksToRead()
    17 0x55cf827f728d in DB::MergeTreeData::canUseParallelReplicasBasedOnPKAnalysis()
    18 0x55cf827f627e in DB::MergeTreeData::getQueryProcessingStage()
    19 0x55cf7f2f4969 in DB::InterpreterSelectQuery::getSampleBlockImpl() build_docker/./src/Interpreters/InterpreterSelectQuery.cpp:937:31
    20 0x55cf7f2daa00 in DB::InterpreterSelectQuery::InterpreterSelectQuery()
    24 0x55cf7f520b98 in DB::InterpreterSelectWithUnionQuery::buildCurrentChildInterpreter()
    25 0x55cf7f51b6cd in DB::InterpreterSelectWithUnionQuery::InterpreterSelectWithUnionQuery()
    27 0x55cf7f1d4ea9 in DB::InterpreterFactory::get(std::__1::shared_ptr<DB::IAST>&, std::__1::shared_ptr<DB::Context>, DB::SelectQueryOptions const&) build_docker/./src/Interpreters/InterpreterFactory.cpp:162:16
    28 0x55cf8012e485 in DB::executeQueryImpl(char const*, char const*, std::__1::shared_ptr<DB::Context>, bool, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) build_docker/./src/Interpreters/executeQuery.cpp:1032:31
    29 0x55cf80121bc1 in DB::executeQuery(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::shared_ptr<DB::Context>, bool, DB::QueryProcessingStage::Enum) build_docker/./src/Interpreters/executeQuery.cpp:1229:30
    30 0x55cf8389295f in DB::TCPHandler::runImpl() build_docker/./src/Server/TCPHandler.cpp:424:24
    31 0x55cf838d7dfb in DB::TCPHandler::run() build_docker/./src/Server/TCPHandler.cpp:2050:9
```

</details>

CI: https://s3.amazonaws.com/clickhouse-test-reports/53214/d99b10c340909ed4ee2e6edf0921e8a2f8561b0d/fuzzer_astfuzzermsan/report.html
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-08-18 18:02:46 +02:00
Alexander Tokmakov
334e014c2c Merge branch 'master' into ignore_on_cluster_for_replicated_entity_queries 2023-08-18 17:48:37 +02:00
Austin Kothig
6b42975d33
Change BE-UUID to work the same as LE-UUID. Included high and low getters to provide cleaner code when accessing undertype. 2023-08-18 08:19:46 -07:00
Yakov Olkhovskiy
84cb358c14
Merge pull request #43210 from amosbird/storage_executable_stderr 
Add different reactions when external command stderr has data
 2023-08-18 11:17:46 -04:00
Alexander Tokmakov
a6684f7a37
Merge pull request #53484 from azat/rmt-queue-cleanup 
Remove outdated code in ReplicatedMergeTreeQueue::initialize()
 2023-08-18 18:11:33 +03:00
Nikolay Degterinsky
4c606590a1
Fix tiny thing in Replicated database (#53496) 
* Fix tiny thing in Replicated database

* empty commit

---------

Co-authored-by: Alexander Tokmakov <tavplubix@clickhouse.com>
 2023-08-18 17:29:24 +03:00
vdimir
ae96d32027
Fix crash in join on sparse column 2023-08-18 14:14:54 +00:00
Nikolai Kochetov
e9d8745971
Merge branch 'master' into group_array_max_element_size 2023-08-18 15:23:01 +02:00
Nikolai Kochetov
88e81a31fa Add server setting group_array_max_element_size. 2023-08-18 13:15:27 +00:00
Alexander Gololobov
50061b9cb4
Merge pull request #53452 from ClickHouse/such_much_watch 
Deduplicate same watch callbacks registered multiple times
 2023-08-18 14:43:03 +02:00
Robert Schulze
b8912203e4
Merge pull request #53525 from aiven-sal/aiven-sal/siphash_readd 
Re-add SipHash keyed functions, try II
 2023-08-18 14:14:56 +02:00
Kruglov Pavel
3bae63e48e
Fix special build 2023-08-18 13:56:32 +02:00
Alexander Tokmakov
0bd378afd9
Merge pull request #53381 from bakwc/bakwc-patch-2 
Fixed system.data_skipping_indices for MaterializedMySQL
 2023-08-18 14:22:40 +03:00
taiyang-li
4f9429d2e4 fix ut tests/queries/0_stateless/00900_null_array_orc_load.sh 2023-08-18 17:58:24 +08:00
Duc Canh Le
506880fce3 Merge branch 'master' into timeout_overflow_break_subquery 2023-08-18 09:28:49 +00:00
Alexander Gololobov
d3edcaae04 Fixed for using failed_op_index before it is initialized 2023-08-18 11:19:35 +02:00
Amos Bird
1c300a31b1
Fix build 2023-08-18 15:38:48 +08:00
Amos Bird
b2dd055cdf
Use poll instead 2023-08-18 15:38:48 +08:00
Amos Bird
b6de70b042
StorageExecutable check_exit_code default to false 2023-08-18 15:38:47 +08:00
Amos Bird
c43bf153f5
Refactor 2023-08-18 15:38:46 +08:00
Amos Bird
dd0c71b32a
Add error_exit_reaction 2023-08-18 15:38:46 +08:00
Amos Bird
940ba60ae0
Fix non-linux build 2023-08-18 15:38:45 +08:00
Amos Bird
476f3cedc1
Various reactions when executable stderr has data 2023-08-18 15:38:45 +08:00
taiyang-li
15720d9cef fix ut tests/queries/0_stateless/02518_parquet_arrow_orc_boolean_value.sh 2023-08-18 15:10:25 +08:00
taiyang-li
8e0d5b7ee0 fix bugs 2023-08-18 13:31:26 +08:00
Alexey Milovidov
e7d0edfce6
Merge pull request #53261 from bharatnc/ncb/truncate-db 
support TRUNCATE DATABASE
 2023-08-18 08:22:32 +03:00
Alexey Milovidov
3bcc4884d7
Merge pull request #53533 from ClickHouse/cpy 
Use long timeout for S3 copy requests
 2023-08-18 08:13:32 +03:00
Alexey Milovidov
a882bf1bb8
Merge pull request #53495 from ClickHouse/i_hate_materialized_views 
Fix a race condition between RESTART REPLICAS and DROP DATABASE
 2023-08-18 06:40:04 +03:00
Anton Popov
b9df41d5e3
Merge pull request #53129 from CurtizJ/non-deterministic-mutations 
Allow to execute constant non-deterministic functions in mutations on initiator
 2023-08-18 00:26:20 +02:00
Harry Lee
1645439000
Merge branch 'master' into hlee-s390x-proto-endian 2023-08-17 18:09:04 -04:00
robot-ch-test-poll
ce58b90ea1
Merge pull request #53455 from rschu1ze/respect-changed-cgroups-limit 
Respect cgroup memory limit when reloading configuration
 2023-08-18 00:03:35 +02:00
Igor Nikonov
06c98956b3 Merge remote-tracking branch 'origin/master' into fix-parallel-replicas-multiply-result 2023-08-17 20:42:50 +00:00
Igor Nikonov
9f771b10e9 Cleanup 2023-08-17 20:42:22 +00:00
Yakov Olkhovskiy
7ce5041a52 Merge remote-tracking branch 'ch/master' into hlee-s390x-codec-delta 2023-08-17 20:29:06 +00:00
Jianfei Hu
0c81546960 remove the comments. 
Signed-off-by: Jianfei Hu <hujianfei258@gmail.com>
 2023-08-17 20:13:33 +00:00
Michael Kolupaev
557bfea4d8 Use long timeout for S3 copy requests 2023-08-17 19:36:45 +00:00
robot-clickhouse
81af60eeea
Merge pull request #53440 from ZhiguoZh/20230815-date-arrayjoin-fix 
Fix an unexpected behavior in #53152
 2023-08-17 20:56:01 +02:00
pufit
a52249872e
Merge pull request #53313 from jorisgio/joris/max_threads_for_indexes 
add max_threads_for_indexes settings
 2023-08-17 13:59:53 -04:00
avogar
60b0b88d50 Clean up 2023-08-17 16:59:57 +00:00
avogar
4c32097df3 Use filter by file/path before reading in url/file/hdfs table functions, reduce code duplication 2023-08-17 16:54:43 +00:00
Alexander Gololobov
827ae49b95 Cleanups 2023-08-17 17:39:23 +02:00
Alexander Gololobov
54e2be4e32 Use WatchCallbackPtr 2023-08-17 17:32:04 +02:00
Alexander Gololobov
80b767316d Deduplicate same watch callbacks registered multiple times 2023-08-17 17:32:04 +02:00
Alexander Tokmakov
89c365803f
Merge pull request #53424 from ClickHouse/exception_message_patterns6 
Less exceptions with runtime format string
 2023-08-17 17:58:04 +03:00
Igor Nikonov
3e9ae1751e Merge remote-tracking branch 'origin/master' into fix-parallel-replicas-multiply-result 2023-08-17 14:45:37 +00:00
Filipp Ozinov
ce87451b66
Merge branch 'ClickHouse:master' into bakwc-patch-2 2023-08-17 18:37:00 +04:00
Anton Popov
1805dec6ff Merge remote-tracking branch 'upstream/master' into HEAD 2023-08-17 14:09:43 +00:00
Alexander Tokmakov
72b52250ea
Merge pull request #53501 from azat/dist-flush_on_detach 
Add ability to turn off flush of Distributed on DETACH/DROP/server shutdown
 2023-08-17 16:56:04 +03:00
Salvatore Mesoraca
c2340d588f
Fix issue found by fuzzer 2023-08-17 14:57:34 +02:00
avogar
bca91548ad Add setting input_format_parquet_local_file_min_bytes_for_seek 2023-08-17 12:28:01 +00:00
Igor Nikonov
4af3790f34 Fix style 2023-08-17 12:12:41 +00:00
Azat Khuzhin
fcb66379d1 Remove outdated code in ReplicatedMergeTreeQueue::initialize() 
Initially added in #28817
But got cleaned up in #41981

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-08-17 13:45:26 +02:00
Igor Nikonov
df4ca32210 Fix: select correct number of replicas for shard on coordinator 
+ more tests
 2023-08-17 11:31:51 +00:00
Robert Schulze
ec7daed52f
Merge pull request #50276 from ClibMouse/feature/mergetree-checksum-big-endian-support 
MergeTree & SipHash checksum big-endian support
 2023-08-17 13:27:18 +02:00
Nikolai Kochetov
8dbe82c192
Merge pull request #53489 from ClickHouse/fix-wrong-column-order-in-parallel-final 
Fix wrong columns order for queries with parallel FINAL.
 2023-08-17 13:22:13 +02:00
Alexander Tokmakov
f83d0dabea
Merge pull request #53295 from ClickHouse/less_no_parallel 
Remove no-parallel tag from some tests
 2023-08-17 14:08:35 +03:00
Robert Schulze
42644bfbf5
Rename file to MergeTreeIndexUSearch.h/cpp 2023-08-17 09:54:12 +00:00
kevinyhzou
d65dba10c6 review 2023-08-17 17:45:57 +08:00
Robert Schulze
f420006e09
Make the code consistent with Annoy index and fix build 2023-08-17 09:42:13 +00:00
Robert Schulze
456a02da1b
Merge remote-tracking branch 'rschu1ze/master' 2023-08-17 09:30:27 +00:00
Robert Schulze
5253a7f650
Respect cgroup memory limit when reloading configuration 
Cgroups allows to change the amount of memory available to a process
while it runs. The previous logic calculated the amount of available
memory only once at server startup. As a result, memory thresholds set
via cgroups were not picked up when the settings changed. We now always
incorporate the current limits during re-configuraton.

Note 1: getMemoryAmount() opens/reads a file which is potentially
        expensive. Should be fine though since that happens only when
        the server configuration changes.

Note 2: An better approach would be to treat cgroup limit changes as
        another trigger for ClickHouse server re-configuration (which
        currently only happens when the config files change). Shied away
        from that for now because of the case that when the cgroup limit
        is lowered, there is no guarantee that ClickHouse can shrink the
        memory amount accordingly in time (afaik, it does so only lazily
        by denying new allocations). As a result, the OOM killer would
        kill the server. The same will happen with this PR but at a
        lower implementation complexity.
 2023-08-17 09:22:13 +00:00
Robert Schulze
1215a23de6
Merge pull request #53422 from rschu1ze/consistency-fixes-for-caches 
Refactorings for configuration of in-memory caches
 2023-08-17 11:08:32 +02:00
Robert Schulze
067623a4c9
Merge pull request #53444 from rschu1ze/factorize-constants 
Minor: Factorize constants in Annoy index
 2023-08-17 11:05:56 +02:00
Robert Schulze
d250adf0a5
Merge pull request #53428 from rschu1ze/invalid-async-metrics-update-period 
Protect against invalid asynchronous_metrics_update_period_s settings
 2023-08-17 11:05:15 +02:00
李扬
cfea6db997
Update native_orc.cpp 2023-08-17 16:57:41 +08:00
Azat Khuzhin
17ca2661a1 Add ability to turn off flush of Distributed on DETACH/DROP/server shutdown 
Sometimes you can have tons of data there, i.e. few TiBs, and sending
them on server shutdown does not looks sane (maybe there is a bug and
you need to update/restart to fix flushing).

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-08-17 08:58:06 +02:00
taiyang-li
c4777397f7 fix integration test test_kafka_formats 2023-08-17 14:42:51 +08:00
robot-ch-test-poll3
30dadcfdcc
Merge pull request #53425 from ClickHouse/trash_for_rdb 
Add trash for Replicated database
 2023-08-17 04:11:40 +02:00
Michael Kolupaev
ff04660972
Merge pull request #53433 from ClickHouse/exp 
Fix 'Context has expired' error properly
 2023-08-16 18:40:53 -07:00
Zhiguo Zhou
9467addace
Merge branch 'ClickHouse:master' into 20230815-date-arrayjoin-fix 2023-08-17 09:17:57 +08:00
Igor Nikonov
49f7b4acd6 Merge remote-tracking branch 'origin/master' into fix-parallel-replicas-multiply-result 2023-08-16 23:31:49 +00:00
Igor Nikonov
2e4d346e44 Fixes 2023-08-16 23:31:20 +00:00
Igor Nikonov
21ef1f1d1c Remove unnecessary Scalars usage for cluster_for_parallel_replicas 2023-08-16 23:24:01 +00:00
Nikolay Degterinsky
c6fc31c1e3 Review suggestions 2023-08-16 23:06:42 +00:00
Nikolay Degterinsky
a893f75884 Merge remote-tracking branch 'upstream/master' into better-stop-listen 2023-08-16 23:06:18 +00:00
Alexander Tokmakov
d7272410bd the worst fix for a deadlock ever 2023-08-17 00:42:51 +02:00
Robert Schulze
8a58e0df3e
Merge remote-tracking branch 'rschu1ze/master' into consistency-fixes-for-caches 2023-08-16 21:26:04 +00:00
Robert Schulze
336262fc8e
Merge remote-tracking branch 'rschu1ze/master' into invalid-async-metrics-update-period 2023-08-16 21:25:39 +00:00
Robert Schulze
353854520c
Merge remote-tracking branch 'rschu1ze/master' into factorize-constants 2023-08-16 21:24:21 +00:00
Alexander Tokmakov
69bc4f5139 Merge branch 'master' into exception_message_patterns6 2023-08-16 22:58:24 +02:00
Nikolai Kochetov
3a6a3d491e Fix typo 2023-08-16 20:40:28 +00:00
Alexander Tokmakov
08e4e2bb0b fix 2023-08-16 22:35:45 +02:00
Nikolai Kochetov
91c241532f Add a check for header structure. 2023-08-16 20:26:23 +00:00
Nikolai Kochetov
30ccfa1638 Fix wrong columns order for queries with parallel FINAL. 2023-08-16 20:23:10 +00:00
Jianfei Hu
fcad273724 remove the comments. 
Signed-off-by: Jianfei Hu <hujianfei258@gmail.com>
 2023-08-16 17:49:28 +00:00
Jianfei Hu
1e600351d9 try again the if value.empty() in the beginning. 
Signed-off-by: Jianfei Hu <hujianfei258@gmail.com>
 2023-08-16 17:49:28 +00:00
Jianfei Hu
49502ac0ce revert unrelated style change. 
Signed-off-by: Jianfei Hu <hujianfei258@gmail.com>
 2023-08-16 17:49:28 +00:00
Jianfei Hu
02b7bd94eb suspect some root cause of value[0]. 
Signed-off-by: Jianfei Hu <hujianfei258@gmail.com>
 2023-08-16 17:49:28 +00:00
Jianfei Hu
7227cc6d2b try parts[0] instead of front(). 
Signed-off-by: Jianfei Hu <hujianfei258@gmail.com>
 2023-08-16 17:49:28 +00:00
Jianfei Hu
afaaef6875 wip checking whether system.merge cause sigsev. 
Signed-off-by: Jianfei Hu <hujianfei258@gmail.com>
 2023-08-16 17:49:28 +00:00
Alexey Milovidov
5e3a69b4c4 Update MergeList.cpp 2023-08-16 17:49:28 +00:00
Alexey Milovidov
bc77aab2e8 Update MergeList.h 2023-08-16 17:49:28 +00:00