Commit Graph

17045 Commits

Author SHA1 Message Date
Bhavna Jindal
f935493f28 fix trailing whitespaces 2024-01-22 13:32:57 -08:00
Bhavna Jindal
c59f3e164f Return baseline component as fourth array 2024-01-22 13:32:57 -08:00
Arthur Passos
24b8bbe9fa
Allow to attach partition from table with different partition expression when destination partition expression doesn't re-partition (#39507)
* temp commit

* temp commit

* draft impl for feedback

* fix weird style changes

* fix weird style changes

* fix weird style changes

* fix weird style changes

* fix weird style changes

* aa

* aa

* Add integ tests and remove partition key restriction

* fix small incosistency in partition id

* style fix

* style fix

* style fix

* use existing DataPartStorageBuilder instead of new one

* Refactor part clone to make it more readable and maintainable

* Add MergeTreeDataPartCloner docs

* define ErrorCodes::BAD_ARGUMENTS

* Rebase

* camel case methods

* address some comments

* yet another rebase?

* Move from integ tests to stateless tests

* address more comments

* add finalize on min_max_idx files

* Add sync option to DistinctPartitionExpCloner

* just a temp test

* revert temp change

* Use echoOn to distinguish test queries

* remove comment

* fix build issue during rebase

* atempt to fix build after rebase

* finally fix build

* clear minmaxidx hyperrectangle before loading it

* Fix error on min_max files deletion where it was being assumed that partition expression contained all columns

* get it to the state it was previously

* add missing include

* getting functional?

* refactoring and renaming

* some more refactoring

* extern bad arguments

* try to fix style

* improvements and docs

* remove duplicate includes

* fix crash

* make tests more stable by ordering

* rebase once again..

* fix

* make ci happy?

* fix rebase issues

* docs

* rebase, but prolly needs to be improved

* refactor out from nasty inheritance to static methods

* fix style

* work around optional

* refactor & integrate some changes

* update column_type

* add tests by dencrane

* set utc

* fix ref file

* fix tests

* use MergeTree instead of SummingMergeTree

* mark MergeTreeDataPart::getBlock as const

* address a few comments

* compute module function name size at compile time

* simplify branching in getPartitionAstFieldsCount

* remove column_indexes argument

* merge getBlock with buildBlock

* add some const specifiers

* small adjustments

* remove no longer needed isNull check

* use std::min and max to update global min max idx

* add some assertions

* forward declare some symbols

* fix grammar

* forward decl

* try to fix build..

* remove IFunction forward decl

* Revert "use std::min and max to update global min max idx"

This reverts commit b2fe79dda7.

* Revert "remove no longer needed isNull check"

This reverts commit 129db2610f.

* Revert "Revert "remove no longer needed isNull check""

This reverts commit 9416087dd8.

* Revert "Revert "use std::min and max to update global min max idx""

This reverts commit 20246d4416.

* remove some comments

* partial use of MonotonicityCheckMatcher

* ranges

* remove KeyDescriptionMonotonicityChecker

* remove duplication of applyfunction

* move functions to anonymous namespace

* move functions to cpp

* Relax partition compatibility requirements by accepting subset, add tests from partitioned to unpartitioned

* updte reference file

* Support for partition by a, b, c to partition by a, b

* refactoring part 1

* refactoring part 2, use hyperrectangle, still not complete

* refactoring part 3, build hyperrectangle with intersection of source & destination min max columns

* Support attaching to table with partition expression of multiple expressions

* add tests

* rename method

* remove some code duplication

* draft impl for replicatedmergetree, need to dive deeper

* ship ref file

* fix impl for replicatedmergetree..

* forbid attach empty partition replicatedmergetree

* Add replicated merge tree integration tests

* add test missing files

* fix black

* do not check for monotonicity of empty partition

* add empty tests & fix replicated

* remove no longer needed buildBlockWithMinMaxINdexes

* remove column logic in buildHyperrectangle

* simplify implementation by using existing methods

* further simplify implementation

* move all MergeTreeDataPartClone private methods to .cpp file

* decrease decomposition

* use different namespaces

* reduce code duplication

* fix style

* address a few comments

* add chassert to assert arguments size on MonotonicityCheckVisitor

* remove deleteMinMaxFiles method

* remove useless checks from sanitycheck

* add tests for attach partition (not id)

* Remove sanityCheckASTPartition and bring back conditional getPartitionIDFromQuery

* remove empty block comment

* small fixes

* fix formatting

* add missing include

* remove duplicate iuncludes

* trigger ci

* reduce some code duplication

* use updated partition id on replicatedmergetree

* fix build

* fix build

* small refactor

* do not use insert increment on fetch part

* remove duplicate includes

* add one more integ test

* black

* black

* rely on partition exp instead of partition id on replicated part fetch to decide if it is a different partition exp

* add one more integ test

* add order by clause

* fix black

---------

Co-authored-by: Alexey Milovidov <milovidov@clickhouse.com>
2024-01-22 21:00:04 +01:00
Robert Schulze
90a0ea36b3
Add statement 'SYSTEM RELOAD ASYNCHRONOUS METRICS' 2024-01-22 17:14:58 +00:00
Kruglov Pavel
0829de8928
Merge pull request #57599 from Khatskevich/khatskevich/fix_docs
Improve named collections documentation
2024-01-22 12:54:57 +01:00
jsc0218
6f3124acdb
Merge pull request #58402 from zhangyifan27/add_max_estimated_execution_time
Seperate max_execution_time and max_estimated_execution_time
2024-01-21 19:57:06 -05:00
robot-clickhouse-ci-1
be3e612f79
Merge pull request #58892 from Waterkin/doc-revise
Translate Eng docs to Chinese.
2024-01-22 00:58:07 +01:00
Han Fei
2c107e1925
Update docs/zh/faq/operations/delete-old-data.md 2024-01-22 00:29:06 +01:00
Han Fei
baf42d777f
Update docs/zh/faq/operations/delete-old-data.md 2024-01-22 00:29:01 +01:00
Han Fei
4938bf89de
Update docs/zh/faq/operations/delete-old-data.md 2024-01-22 00:28:55 +01:00
Han Fei
1244818350
Update docs/zh/faq/operations/delete-old-data.md 2024-01-22 00:28:47 +01:00
Han Fei
edccae1d62
Update docs/zh/faq/operations/delete-old-data.md 2024-01-22 00:28:40 +01:00
Han Fei
861e050963
Update docs/zh/faq/operations/delete-old-data.md 2024-01-22 00:28:33 +01:00
Han Fei
db93b72dcc
Update docs/zh/faq/operations/delete-old-data.md 2024-01-22 00:28:27 +01:00
Han Fei
30c6e1f151
Update docs/zh/faq/operations/delete-old-data.md 2024-01-22 00:28:20 +01:00
Han Fei
a1a5ae59d5
Update docs/zh/faq/general/ne-tormozit.md 2024-01-22 00:28:14 +01:00
Han Fei
bc5a6e8e63
Update docs/zh/faq/general/ne-tormozit.md 2024-01-22 00:28:06 +01:00
Han Fei
ab95031399
Update docs/zh/faq/integration/json-import.md 2024-01-22 00:27:59 +01:00
Han Fei
51524aadc5
Update docs/zh/faq/general/ne-tormozit.md 2024-01-22 00:27:52 +01:00
Han Fei
9b5c781932
Update docs/zh/faq/general/ne-tormozit.md 2024-01-22 00:27:44 +01:00
Han Fei
2c394155ad
Update docs/zh/faq/general/ne-tormozit.md 2024-01-22 00:27:35 +01:00
edpyt
451aa13736
очепятка 2024-01-20 14:47:28 +03:00
Kruglov Pavel
20379d34d5
Merge branch 'master' into variant-data-type 2024-01-19 23:17:42 +01:00
filimonov
7635da91a8
Update rename.md
Remove wrong note. RENAME is supported by most of database engines (including old Ordinary).
2024-01-19 21:55:41 +01:00
zhongyuankai
cbb51659f6
Merge branch 'master' into limit_materialized_view_size 2024-01-19 22:00:12 +08:00
Lino Uruñuela
4a6c2d21ff
Update executable.md - Fix script name in SQL example
It uses 'generate_random.py' in 'The following Python script is named generate_random.py and is saved in the user_scripts folder....'

However, in the SQL example, it uses 'my_script.py'.

'SELECT * FROM executable('generate_random.py', TabSeparated, 'id UInt32, random String', (SELECT 10))'
2024-01-19 12:17:22 +01:00
Robert Schulze
1bfeee1954
Some fixups 2024-01-18 20:49:50 +00:00
Robert Schulze
15700592f7
Merge remote-tracking branch 'ClickHouse/master' into column_level_compress_block 2024-01-18 19:12:57 +00:00
Kruglov Pavel
5444cde408
Merge branch 'master' into variant-data-type 2024-01-18 18:31:27 +01:00
Robert Schulze
316669b90f
Merge pull request #58396 from zenmiao7/u/zhengmiao/AddArrayFunctionShingles
Add function `arrayShingles()`
2024-01-18 11:38:56 +01:00
Duc Canh Le
b16a4cf361 add document
Signed-off-by: Duc Canh Le <duccanh.le@ahrefs.com>
2024-01-18 07:55:44 +00:00
Srikanth Chekuri
c23c7e0b6f
feat: add DDSketch quantile (#56342)
* feat: add DDSketch quantile

* Fix StyleCheck

* Move quantileddsketch.md under right dir

* Update stateless test number and add stateful test

* Throw exception when relative accuracy is too low

* Update test number

* Fix undefined behaviour for empty store

* Fix quantileGK test

* Update test numbers

* Update src/AggregateFunctions/

* Throw exception on out of range values

* Update relative accuracy docs and add encoding details

* Address review comments and suggestions

* Remove unused function

* Address alexey review comments

* Remove unused function isFloat64FieldType

* Throw error on invalid relative accuracy

* Simplify mapping

* Address remaining review comments

* Add effective memory usage suggestions

* Fix Stylecheck

* Fix fast test

* Incorporate bins capacity suggestion

* Fix fuzzer

* Remove inheritance in Mapping, Store and DDSketch

* Add checks for bin resizing

* Add note about the binary compatible implementation
2024-01-17 19:51:32 -08:00
Yarik Briukhovetskyi
8dce581ae7
update ci 2024-01-17 14:14:15 +01:00
Kruglov Pavel
6d064512e1
Merge pull request #58614 from Blargian/58363_disable_ansi_pretty_automatically
58363 Automatically disable ANSI escape sequences in Pretty formats if the output is not a terminal
2024-01-17 13:45:41 +01:00
Yarik Briukhovetskyi
9fbe2296db
Update docs/en/sql-reference/table-functions/executable.md 2024-01-17 13:04:17 +01:00
Robert Schulze
84ab20ff77
Some fixups 2024-01-17 12:01:52 +00:00
Antonio Andelic
5e020476d3
Merge pull request #58810 from ClickHouse/heap-profiler-docs
Add docs for allocation profiling
2024-01-17 12:26:55 +01:00
Robert Schulze
22534c69c4
Merge remote-tracking branch 'ClickHouse/master' into u/zhengmiao/AddArrayFunctionShingles 2024-01-17 10:49:11 +00:00
avinzhang
74f2864f20
Update executable.md
Adding the sentence on the requirement to run the script.
2024-01-17 17:42:59 +10:00
Waterkin
ef044842c5 Translate Eng docs to Chinese.
Signed-off-by: Waterkin <1055905911@qq.com>
2024-01-17 13:33:02 +08:00
Antonio Andelic
f33f374ac5 Make Kovi happy 2024-01-16 15:09:49 +01:00
Antonio Andelic
76c9bf42d7 Add docs for allocation profiling 2024-01-15 13:57:16 +01:00
vdimir
8dc1756532
Support <secret> in cluster discovery 2024-01-15 11:22:53 +00:00
vdimir
012713fc31
ClusterDiscovery supports setting username and password 2024-01-15 11:22:48 +00:00
Robert Schulze
9cfdff2ddb
Merge pull request #58284 from lingtaolf/Improvement/inverted_index_support_match
Utilize inverted index by function `match()`
2024-01-15 11:37:23 +01:00
zhongyuankai
826b29ef4d Max materialized views size for table 2024-01-15 10:02:57 +08:00
Alexander Tokmakov
5128607995
Merge pull request #58393 from jaymebrd/feat/add-from-modifier-sync-replica-lightweight
add `from srcreplicas` modifier  for `SYSTEM SYNC REPLICA LIGHTWEIGHT`
2024-01-14 20:53:16 +01:00
Blargian
72b5cf5993 #58363 - removed switch from PrettyBlockOutputFormat and modified BlockOutputFormats to use color variable. Updated english and russian documentation. Updated test 00405 reference file. 2024-01-12 19:46:03 +02:00
Ilya Yatsishin
1b548d3597
Update query-complexity.md to rerun tests 2024-01-12 20:44:46 +04:00
Robert Schulze
3fbbeaa319
Update docs 2024-01-12 14:10:25 +00:00
avogar
bfd7db8cf4 Better docs 2024-01-12 13:06:51 +00:00
avogar
fbfdde60a7 Add settings for better control of indexes type in Arrow dictionary. Use signed integer type for indexes by default 2024-01-12 13:06:51 +00:00
Robert Schulze
790f5890a0
Merge pull request #58611 from rschu1ze/qc-isolation
Improve isolation of query cache entries under re-created users or role switches
2024-01-11 17:30:24 +01:00
robot-ch-test-poll4
4c5884b1d9
Merge pull request #58701 from ClickHouse/docs/assume-contraint
Adds documentation for `ASSUME CONSTRAINT`
2024-01-11 17:04:48 +01:00
robot-clickhouse-ci-1
5c47a4fa91
Merge pull request #58666 from ClickHouse/docs/data-faster-than-datetime
Explains that date is generally faster than datetime.
2024-01-11 10:17:54 +01:00
robot-ch-test-poll4
aba256d9cf
Merge pull request #58669 from ClickHouse/docs/show-non-default-settings
Tells reader how to show which settings have been changed from default.
2024-01-11 09:59:07 +01:00
Kseniia Sumarokova
e74f8e9095
Update docs/en/sql-reference/data-types/datetime.md 2024-01-11 09:50:48 +01:00
robot-ch-test-poll
498894a15c
Merge pull request #58700 from ClickHouse/docs/arrayjoin-short-circuit-function-eval-no-no
Adds info about not using `arrayJoin` with `short_circuit_function_evaluation`.
2024-01-11 02:22:11 +01:00
johnnymatthews
2bfe7a2cab Adds docs for ASSUME CONSTRAINT. 2024-01-10 21:13:21 -04:00
johnnymatthews
826a39b732 Adds arrayJoin exception. Prettifies file. Fixes spelling. 2024-01-10 20:43:19 -04:00
robot-clickhouse-ci-1
26bc1b5b5c
Merge pull request #58682 from rschu1ze/docs-log-type
Docs: Update type of two column in `system.query_log/text_log`
2024-01-10 21:58:18 +01:00
Robert Schulze
1b739a8f03
Docs: Update type of two column in system.query_log/text_log 2024-01-10 20:14:39 +00:00
johnnymatthews
b276694ac2 Adds ALL qualifier to JOIN. 2024-01-10 13:17:04 -04:00
johnnymatthews
ebd7f19670 Example to show settings changed from default. 2024-01-10 12:38:38 -04:00
johnnymatthews
4723a9c082 Explains that date is generally faster than datetime. 2024-01-10 11:50:02 -04:00
Robert Schulze
582a9cc2a7
Merge pull request #58655 from rschu1ze/docs-match
Document that `match()` can use `ngrambf_v1` and `tokenbf_v1` indexes
2024-01-10 15:43:52 +01:00
vdimir
263c6e9c95
Merge pull request #58608 from yariks5s/remove_order_by_requirement_paste_join
Remove order by requierement in PASTE JOIN
2024-01-10 15:18:18 +01:00
Robert Schulze
27e809a4f5
Add match() to supported functions for token/ngrambf_v1 indexes 2024-01-10 13:39:19 +00:00
Robert Schulze
92692bc57c
Fix links 2024-01-10 13:36:54 +00:00
Robert Schulze
9323fe8ecd
Cosmetics 2024-01-10 12:50:05 +00:00
Robert Schulze
a0128e0f51
Merge pull request #58544 from rschu1ze/sqid-decode
Add `sqidDecode()` function
2024-01-10 13:47:13 +01:00
Robert Schulze
36173bbb7b
Merge remote-tracking branch 'rschu1ze/master' into qc_isolation 2024-01-10 09:57:13 +00:00
Robert Schulze
4cacfa6b43
Merge pull request #57509 from jinjunzh/qatzstd_main
Add Intel-QAT-based `ZSTD_QAT` compression codec
2024-01-10 10:31:39 +01:00
Kseniia Sumarokova
1b517105db
Merge pull request #57323 from valbok/mysql-binlog-client
MaterializedMySQL: Introduce MySQL Binlog Client
2024-01-10 09:08:13 +01:00
Robert Schulze
4db26ad56b
Merge pull request #58531 from rschu1ze/string-similarity
Implement Damerau-Levenshtein distance and Jaro[-Winkler] string distance functions
2024-01-09 22:46:57 +01:00
Robert Schulze
ced9e93ac6
Merge remote-tracking branch 'rschu1ze/master' into qatzstd_main 2024-01-09 20:36:02 +00:00
pufit
6cf55b82f4
Merge pull request #58539 from canhld94/file_custom_compress_level
Allow explicitly set compression level in output format
2024-01-09 13:43:38 -05:00
jsc0218
6fa368d296
Merge pull request #58449 from cw5121/keeper_client_doc_modify
refine keeper connection in doc
2024-01-09 13:24:29 -05:00
Robert Schulze
c04e4eb162
Merge remote-tracking branch 'rschu1ze/master' into qc-isolation 2024-01-09 16:30:20 +00:00
Yarik Briukhovetskyi
aef24c0621
Merge branch 'ClickHouse:master' into remove_order_by_requirement_paste_join 2024-01-09 17:03:54 +01:00
Robert Schulze
69ba1a706a
Merge remote-tracking branch 'rschu1ze/master' into string-similarity 2024-01-09 15:22:52 +00:00
Yarik Briukhovetskyi
c8f97e877f
fix typo 2024-01-09 13:31:15 +01:00
Sema Checherinda
7c7e72c4b7
Merge pull request #58573 from ClickHouse/chesema-s3-client-creation
fix and test that S3Clients are reused
2024-01-09 13:28:20 +01:00
Sema Checherinda
6dbf7c967c
Merge pull request #57663 from ClickHouse/CheSema-remote-blobs-naming-2
generate object storage key by template
2024-01-09 13:26:56 +01:00
yariks5s
dc4cb2fa0e add docs 2024-01-09 11:46:54 +00:00
vdimir
b4bfd01322
Merge pull request #58246 from ClickHouse/vdimir/util_format_values
clickhouse-format improvements: support VALUES, comments, max_line_length
2024-01-09 11:24:42 +01:00
Robert Schulze
64ab30f446
Merge pull request #57078 from ClibMouse/seriesDecompose
Seasonal decompose of time series into seasonal, trend and residue components
2024-01-09 11:18:13 +01:00
Duc Canh Le
37a7ce2950
Apply suggestions from code review
Co-authored-by: pufit <pufit@yandex.ru>
2024-01-09 17:17:30 +08:00
Yakov Olkhovskiy
cfa41e7dc6
Merge pull request #58038 from ClickHouse/revert-58022-revert-57555-ft-dropped_tables_parts
Add system.dropped_tables_parts table - second attempt
2024-01-09 00:47:07 -05:00
Robert Schulze
fabc06995e
Improve isolation of query results in query cache
Fixes #58054
2024-01-08 20:52:30 +00:00
Bhavna Jindal
b2434d0c10 minor refactoring and cosmetic changes 2024-01-08 10:49:09 -08:00
Kruglov Pavel
b947609b8e
Merge branch 'master' into variant-data-type 2024-01-08 15:04:51 +01:00
Robert Schulze
fb4fd29110
Merge pull request #58454 from rschu1ze/punycode-revert-revert
Implement IDNA and punycode encoding/decoding (2nd attempt)
2024-01-08 14:26:03 +01:00
Jayme Bird
f7173004cc update docs 2024-01-08 12:40:05 +00:00
vdimir
621d17b16f
Better clickhouse-format: support VALUES, comments 2024-01-08 12:01:33 +00:00
Robert Schulze
6665d23243
Merge pull request #58029 from skyoct/feat/server_settings
Add `changeable_without_restart` column to `system.server_settings`
2024-01-08 10:09:51 +01:00
Robert Schulze
0f51e32bb5
Rename from ZSTDQPL to ZSTD_QPL 2024-01-07 17:47:31 +00:00
Robert Schulze
53965bb9f1
Merge remote-tracking branch 'rschu1ze/master' into qatzstd_main 2024-01-07 14:11:22 +00:00
Robert Schulze
e947ed710b
Hijack and document 'update' map to represent reload status 2024-01-07 13:09:27 +00:00
Robert Schulze
ee0738e3df
Implement jaroSimilarity + jaroWinklerSimilarity 2024-01-07 08:34:41 +00:00