Commit Graph

52236 Commits

Author SHA1 Message Date
kssenii
7f8f379d7f Parallel & disrtibuted processing for ordered mode 2024-01-24 16:32:15 +01:00
Anton Popov
4292ee7d51
Merge pull request #58886 from CurtizJ/better-read-buffers
Less error prone interface of read buffers
2024-01-23 14:59:27 +01:00
Antonio Andelic
74dd68afbc
Merge pull request #59102 from ClickHouse/keeper-startup-fix
Avoid election timeouts during startup in Keeper
2024-01-23 13:55:52 +01:00
Alexander Gololobov
9a0e5d0aac
Merge pull request #58914 from ClickHouse/ot_spans_in_insert
Opentelemetry spans to analyze CPU and S3 bottlenecks on inserts
2024-01-23 13:53:13 +01:00
Kruglov Pavel
b8d2af9574
Merge pull request #59104 from ClickHouse/Avogar-patch-1
Add missing setting max_estimated_execution_time in  SettingsChangesHistory
2024-01-23 12:05:49 +01:00
Robert Schulze
9752079983
Cosmetics 2024-01-23 10:58:09 +00:00
Robert Schulze
e23e7e6b73
Merge pull request #53710 from rschu1ze/system-reload-asynchronous-metrics
Add statement `SYSTEM RELOAD ASYNCHRONOUS METRICS`
2024-01-23 11:56:25 +01:00
Dmitry Novik
6962859e43
Merge pull request #59068 from ClickHouse/check-optimization-setting
Followup to 57853
2024-01-23 11:24:31 +01:00
Kseniia Sumarokova
f6ffbb4c0a
Merge pull request #59049 from ClickHouse/fix-data-race-in-factories
Fix data race on query_factories_info
2024-01-23 11:22:40 +01:00
Robert Schulze
e67076ea2b
Merge pull request #55201 from canhld94/column_level_compress_block
Column-level compression block sizes
2024-01-23 11:22:03 +01:00
Kruglov Pavel
415589d4fc
Fix typo 2024-01-23 10:32:28 +01:00
Kruglov Pavel
35a3879c1f
Add missing setting max_estimated_execution_time in SettingsChangesHistory 2024-01-23 10:17:19 +01:00
Nikita Mikhaylov
bd7024ff9b
Merge pull request #59083 from ClickHouse/fix-startup-failure
Fix server shutdown due to exception while loading metadata
2024-01-23 09:48:38 +01:00
Antonio Andelic
9ce3f8a7aa
typo 2024-01-23 09:18:51 +01:00
Antonio Andelic
ff644831e2 Avoid election timeouts during startup 2024-01-23 08:02:30 +00:00
Robert Schulze
ebdad85c4f
Sort integer aliases 2024-01-23 06:11:47 +00:00
Robert Schulze
6f4c0925ab
Merge pull request #58954 from slvrtrn/mysql-add-signed-unsigned-aliases
MySQL compatibility: add SIGNED/UNSIGNED aliases for Int64/UInt64
2024-01-23 07:01:31 +01:00
serxa
6db59de888 use LOGICAL_ERROR to see if it breaks 2024-01-23 05:54:25 +00:00
serxa
a1de16d660 wait for currently running jobs to finish 2024-01-22 23:46:54 +00:00
Nikita Taranov
09e24ed6c5
Merge pull request #58068 from zhongyuankai/limit_materialized_view_size
A limit on the number of materialized views attached to a table.
2024-01-22 22:50:56 +01:00
Max K
4932b1cad9
Merge branch 'master' into better-read-buffers 2024-01-22 21:52:55 +01:00
Arthur Passos
24b8bbe9fa
Allow to attach partition from table with different partition expression when destination partition expression doesn't re-partition (#39507)
* temp commit

* temp commit

* draft impl for feedback

* fix weird style changes

* fix weird style changes

* fix weird style changes

* fix weird style changes

* fix weird style changes

* aa

* aa

* Add integ tests and remove partition key restriction

* fix small incosistency in partition id

* style fix

* style fix

* style fix

* use existing DataPartStorageBuilder instead of new one

* Refactor part clone to make it more readable and maintainable

* Add MergeTreeDataPartCloner docs

* define ErrorCodes::BAD_ARGUMENTS

* Rebase

* camel case methods

* address some comments

* yet another rebase?

* Move from integ tests to stateless tests

* address more comments

* add finalize on min_max_idx files

* Add sync option to DistinctPartitionExpCloner

* just a temp test

* revert temp change

* Use echoOn to distinguish test queries

* remove comment

* fix build issue during rebase

* atempt to fix build after rebase

* finally fix build

* clear minmaxidx hyperrectangle before loading it

* Fix error on min_max files deletion where it was being assumed that partition expression contained all columns

* get it to the state it was previously

* add missing include

* getting functional?

* refactoring and renaming

* some more refactoring

* extern bad arguments

* try to fix style

* improvements and docs

* remove duplicate includes

* fix crash

* make tests more stable by ordering

* rebase once again..

* fix

* make ci happy?

* fix rebase issues

* docs

* rebase, but prolly needs to be improved

* refactor out from nasty inheritance to static methods

* fix style

* work around optional

* refactor & integrate some changes

* update column_type

* add tests by dencrane

* set utc

* fix ref file

* fix tests

* use MergeTree instead of SummingMergeTree

* mark MergeTreeDataPart::getBlock as const

* address a few comments

* compute module function name size at compile time

* simplify branching in getPartitionAstFieldsCount

* remove column_indexes argument

* merge getBlock with buildBlock

* add some const specifiers

* small adjustments

* remove no longer needed isNull check

* use std::min and max to update global min max idx

* add some assertions

* forward declare some symbols

* fix grammar

* forward decl

* try to fix build..

* remove IFunction forward decl

* Revert "use std::min and max to update global min max idx"

This reverts commit b2fe79dda7.

* Revert "remove no longer needed isNull check"

This reverts commit 129db2610f.

* Revert "Revert "remove no longer needed isNull check""

This reverts commit 9416087dd8.

* Revert "Revert "use std::min and max to update global min max idx""

This reverts commit 20246d4416.

* remove some comments

* partial use of MonotonicityCheckMatcher

* ranges

* remove KeyDescriptionMonotonicityChecker

* remove duplication of applyfunction

* move functions to anonymous namespace

* move functions to cpp

* Relax partition compatibility requirements by accepting subset, add tests from partitioned to unpartitioned

* updte reference file

* Support for partition by a, b, c to partition by a, b

* refactoring part 1

* refactoring part 2, use hyperrectangle, still not complete

* refactoring part 3, build hyperrectangle with intersection of source & destination min max columns

* Support attaching to table with partition expression of multiple expressions

* add tests

* rename method

* remove some code duplication

* draft impl for replicatedmergetree, need to dive deeper

* ship ref file

* fix impl for replicatedmergetree..

* forbid attach empty partition replicatedmergetree

* Add replicated merge tree integration tests

* add test missing files

* fix black

* do not check for monotonicity of empty partition

* add empty tests & fix replicated

* remove no longer needed buildBlockWithMinMaxINdexes

* remove column logic in buildHyperrectangle

* simplify implementation by using existing methods

* further simplify implementation

* move all MergeTreeDataPartClone private methods to .cpp file

* decrease decomposition

* use different namespaces

* reduce code duplication

* fix style

* address a few comments

* add chassert to assert arguments size on MonotonicityCheckVisitor

* remove deleteMinMaxFiles method

* remove useless checks from sanitycheck

* add tests for attach partition (not id)

* Remove sanityCheckASTPartition and bring back conditional getPartitionIDFromQuery

* remove empty block comment

* small fixes

* fix formatting

* add missing include

* remove duplicate iuncludes

* trigger ci

* reduce some code duplication

* use updated partition id on replicatedmergetree

* fix build

* fix build

* small refactor

* do not use insert increment on fetch part

* remove duplicate includes

* add one more integ test

* black

* black

* rely on partition exp instead of partition id on replicated part fetch to decide if it is a different partition exp

* add one more integ test

* add order by clause

* fix black

---------

Co-authored-by: Alexey Milovidov <milovidov@clickhouse.com>
2024-01-22 21:00:04 +01:00
serxa
f1ab22d4c3 fix style 2024-01-22 19:54:50 +00:00
serxa
fad5aeede3 fix 2024-01-22 19:43:25 +00:00
serxa
1d6d1182fd Fix server shutdown due to exception while loading metadata 2024-01-22 19:38:26 +00:00
Robert Schulze
90a0ea36b3
Add statement 'SYSTEM RELOAD ASYNCHRONOUS METRICS' 2024-01-22 17:14:58 +00:00
Robert Schulze
748371d127
Refactor locking in AsynchronousMetrics 2024-01-22 16:51:02 +00:00
Robert Schulze
1262945df7
Cosmetics 2024-01-22 16:51:02 +00:00
Duc Canh Le
ed031f32af address review comments
Signed-off-by: Duc Canh Le <duccanh.le@ahrefs.com>
2024-01-22 16:06:29 +00:00
Anton Popov
e1e7a9cd45
Merge branch 'master' into better-read-buffers 2024-01-22 15:50:48 +01:00
Alexander Tokmakov
2f1d4f094b
Add a parameter for testing purposes (#59027)
* Update ConnectionParameters.cpp

* Update ConnectionParameters.cpp

* Update ConnectionParameters.cpp
2024-01-22 15:48:44 +01:00
Alexander Tokmakov
cddd41ccd5
Merge pull request #59070 from ClickHouse/revert-55575-no-octal-leading-zeroes
Revert "Consume leading zeroes when parsing a number in ConstantExpressionTemplate"
2024-01-22 15:35:16 +01:00
Alexander Tokmakov
c2202ff347
Skip unavailable replicas in parallel distributed insert select (#58931)
* Update StorageDistributed.cpp

* Update StorageDistributed.cpp

* Update StorageDistributed.cpp

* add a test

* Update 01099_parallel_distributed_insert_select.sql
2024-01-22 15:34:46 +01:00
Alexander Tokmakov
9b6455122c
Revert "Consume leading zeroes when parsing a number in ConstantExpressionTemplate" 2024-01-22 15:31:08 +01:00
Raúl Marín
1e875431db
Style 2024-01-22 14:59:53 +01:00
vdimir
0791c75315
Merge pull request #58654 from yariks5s/paste_join_aliases_requirement
Avoid alias duplicates in PASTE JOIN and fix aliases requirement
2024-01-22 14:48:13 +01:00
Dmitry Novik
3d0e915088
Followup to 57853 2024-01-22 14:44:07 +01:00
Nikita Taranov
4d6ef8ed86
Merge pull request #56961 from nickitat/some_questionable_code
Bypass `Poco::BasicBufferedStreamBuf` in `ReadBufferFromIStream`
2024-01-22 14:08:09 +01:00
Alexander Tokmakov
0f0785cf20
Merge pull request #59059 from ClickHouse/revert-58838-allow-parallel-replicas-for-join-with-analyzer
Revert "Allow parallel replicas for JOIN with analyzer [part 1]."
2024-01-22 14:07:26 +01:00
Dmitry Novik
a18a8d8ea3
Merge pull request #59009 from nickitat/uniq_optimisation_for_distributed
`uniqExact` state parallel merging for distributed queries
2024-01-22 14:05:02 +01:00
kssenii
4b4eac7c06 Fix 2024-01-22 13:55:15 +01:00
Alexander Tokmakov
4315d44517
Revert "Allow parallel replicas for JOIN with analyzer [part 1]." 2024-01-22 13:30:35 +01:00
alesapin
478b91cc37
Merge pull request #59010 from ClickHouse/alesapin-patch-1
Change log level for super imporant message in Keeper
2024-01-22 13:16:56 +01:00
kssenii
3faf4087e3 Fix 2024-01-22 12:07:49 +01:00
Sema Checherinda
9adab57771
Correct initialization of ConnectionTimeouts (#59000) 2024-01-22 12:03:36 +01:00
Kseniia Sumarokova
aec76d6653
Merge pull request #58877 from ClickHouse/multiple-read-file-log-storage-in-mv
Multiple read file log storage in mv
2024-01-22 12:02:41 +01:00
Alexander Sapin
f7b21b6016 Fix build 2024-01-22 11:53:21 +01:00
Sergei Trifonov
04a040ee5c
Merge pull request #59013 from ClickHouse/remove-static-resource-manager 2024-01-22 10:59:58 +01:00
Raúl Marín
d9ab300300
Merge pull request #57853 from jiebinn/rewriteSumFunction
Rewrite the AST/Analyzer of sum(column +/- literal) function
2024-01-22 10:41:30 +01:00
Antonio Andelic
ecaef80ea0
Merge pull request #58938 from ClickHouse/keeper-fetch-max-file-desc-once
Call `getMaxFileDescriptorCount` once in Keeper
2024-01-22 08:57:58 +01:00