Commit Graph

217 Commits

Author SHA1 Message Date
Alexander Tokmakov
36c31e1d79
Improve concurrent parts removal with zero copy replication (#49630)
* improve concurrent parts removal

* fix

* fix
2023-05-17 14:07:34 +03:00
Alexey Milovidov
9a73a04fe3
Merge pull request #42587 from ClickHouse/compress-marks-primary-key-by-default
Compress marks and primary key by default
2023-05-09 06:43:10 +03:00
Alexey Milovidov
de3e66385c Merge branch 'master' into compress-marks-primary-key-by-default 2023-05-07 02:40:25 +02:00
Alexey Milovidov
75a2589a42
Merge branch 'master' into deprecate-in-memory-parts 2023-05-06 00:49:59 +03:00
Anton Popov
e61d7baa54 Merge remote-tracking branch 'upstream/master' into HEAD 2023-05-04 14:03:53 +00:00
Alexey Milovidov
261907c143 Merge branch 'master' into deprecate-in-memory-parts 2023-05-04 01:07:38 +02:00
Anton Popov
7896d30737
Merge pull request #49122 from CurtizJ/add-async-insert-mt-setting
Add `MergeTree` setting `async_insert`
2023-05-03 16:01:13 +02:00
Alexey Milovidov
f449df85b6 Deprecate in-memory parts 2023-05-03 00:31:09 +02:00
Nikita Mikhaylov
aa4c5fe958
Enhancements for background merges (#49313) 2023-05-02 13:43:59 +02:00
Anton Popov
c7db58be66 add MergeTree setting async_insert 2023-04-25 01:11:58 +00:00
Anton Popov
38622d0770 add settings to delay or throw in case of too many mutations 2023-04-24 18:21:49 +00:00
Alexey Milovidov
3ae7bf754b Merge branch 'master' into compress-marks-primary-key-by-default 2023-04-23 22:46:13 +02:00
凌涛
08e81d0a52
Merge branch 'master' into feature/support_settings_allow_suspicious_indices 2023-04-23 10:08:00 +08:00
凌涛
de18bf72b4 modify code 2023-04-21 17:27:26 +08:00
凌涛
7ed4c413e3 add settings allow_suspicious_indices 2023-04-17 20:23:19 +08:00
Alexander Tokmakov
d6bff4c5a8 limit the number of mutations commands per MUTATE_PART 2023-04-13 00:20:43 +02:00
Alexey Milovidov
17c2678675
Merge branch 'master' into compress-marks-primary-key-by-default 2023-03-31 15:15:38 +03:00
vdimir
08fa1846b1
Add merge tree setting max_number_of_mutations_for_replica 2023-03-27 11:56:48 +00:00
Alexey Milovidov
fe446afb7d Increase max_replicated_merges_in_queue 2023-03-01 06:02:59 +01:00
Arthur Passos
3d9b23e5f3
Merge branch 'master' into add_select_final_mt_setting 2023-02-19 15:01:41 -03:00
youennL-cs
6526c2a8ab
[RFC] Replacing merge tree new engine (#41005)
* Add new engine to ReplacingMergeTree corresponding to the ReplacingCollapsingMergeTree

* Add new test for the new ReplacingMergeTree engine

* Limit sign value to -1/1

* Add new engine to ReplacingMergeTree corresponding to the ReplacingCollapsingMergeTree

* Add new test for the new ReplacingMergeTree engine

* Limit sign value to -1/1

* Replace sign column(Int8) by is_deleted(UInt8)

* Add new engine to ReplacingMergeTree corresponding to the ReplacingCollapsingMergeTree

* Add new test for the new ReplacingMergeTree engine

* Limit sign value to -1/1

* Replace sign column(Int8) by is_deleted(UInt8)

* Add new engine to ReplacingMergeTree corresponding to the ReplacingCollapsingMergeTree

* Add new test for the new ReplacingMergeTree engine

* Limit sign value to -1/1

* Replace sign column(Int8) by is_deleted(UInt8)

* Add keyword 'CLEANUP' when OPTIMIZE

* Cleanup uniquely when it's a replacingMergeTree

* Propagate CLEANUP information and change from 'with_cleanup' to 'cleanup'

* Cleanup data flagged as 'is_deleted'

* Fix merge when optimize and add a test

* Fix OPTIMIZE and INSERT + add tests

* New fix for cleanup at the merge

* Cleanup debug logs

* Add the SETTINGS option 'clean_deleted_rows' that can be 'never' or 'always'

* Fix regression bug; Now REplicatedMergeTree can be called as before without 'is_deleted'

* Add Replicated tests

* Disable tag 'long' for our test and cleanup some white spaces

* Update tests

* Fix tests and remove additional useless whitespace

* Fix replica test

* Style clean && add condition check for is_deleted values

* clean_deleted_rows settings is nom an enum

* Add valid default value to the clean_deleted_rows settings

* Update cleanup checkers to use the enum and fix typos in the test

* Fix submodule contrib/AMQP-CPP pointer

* Add missing messages in test reference and remove a print with non derterministic order

* fix replica test reference

* Fix edge case

* Fix a typo for the spell checker

* Fix reference

* Fix a condition to raise an error if is_deleted differ from 0/1 and cleanup

* Change tests file name and update number

* This should fix the ReplacingMergeTree parameter set

* Fix replicated parameters

* Disable allow_deprecated_syntax_for_merge_tree for our new column

* Fix a test

* Remove non deterministic order print in the test

* Test on replicas

* Remove a condition, when checking optional parameters, that should not be sueful since we disabled the deprected_syntaxe

* Revert "Remove a condition, when checking optional parameters, that should not be useful since we disabled the deprected_syntaxe"

This reverts commit b65d64c05e.

* Fix replica management and limit the number of argument to two maximum, due to the possiblity of deprecated table create/attach failing otherwise

* Test a fix for replicated log information error

* Try to add sync to have consistent results

* Change path of replicas that should cause one issue and add few prints in case it's not that

* Get cleanup info on replicas only if information found

* Fix style issues

* Try to avoid replication error 'cannot select parts...' and and replica read/write field order

* Cleanup according to PR reviews
 and add tests on error raised.

* Update src/Storages/MergeTree/registerStorageMergeTree.cpp

Co-authored-by: Alexander Tokmakov <tavplubix@gmail.com>

* Select ... FINAL don't show rows with is_deleted = true

* Update and fix SELECT ... FINAL merge parameter

* Remove is_deleted rows only on the version inserted when merge

* Fix (master) updates issues

* Revert changes that should not be commited

* Add changes according to review

* Revert changes that should not be commited - part 2

---------

Co-authored-by: Alexander Tokmakov <tavplubix@gmail.com>
2023-02-16 16:03:16 +03:00
Alexey Milovidov
77f60311da
Merge branch 'master' into add_select_final_mt_setting 2023-02-15 15:35:37 +03:00
Antonio Andelic
8d16fe5793
Merge branch 'master' into add-support-for-settings-alias 2023-02-13 08:46:00 +01:00
Alexey Milovidov
395d6b6bd5
Merge pull request #46282 from CurtizJ/return-back-vertical-merges
Allow vertical merges from compact to wide parts [2]
2023-02-12 05:51:12 +03:00
Anton Popov
1205932a19 add setting to disable vertical merges from compact parts 2023-02-10 16:33:46 +00:00
kssenii
1cbff88a7c Fix integration test 2023-02-04 19:31:19 +01:00
kssenii
a391de18a1 Merge remote-tracking branch 'upstream/master' into allow-single-disk-instead-of-storage-policy 2023-02-02 20:49:41 +01:00
Antonio Andelic
85cfee4bb9 Better alias definition 2023-02-01 13:54:03 +00:00
Arthur Passos
48e5211454
Merge branch 'ClickHouse:master' into add_select_final_mt_setting 2023-01-30 13:01:36 -03:00
Antonio Andelic
714fad1529 Add support for settings alias 2023-01-26 14:06:46 +00:00
Robert Schulze
0653f86de9
Various cosmetic cleanups 2023-01-20 10:45:35 +00:00
larryluogit
52ae33dba7
Merge branch 'master' into ftsearch 2023-01-19 11:34:11 -05:00
Han Fei
ed49ebf01a update setting explain 2023-01-13 20:26:08 +01:00
Han Fei
2fb2f503e3 Update src/Storages/MergeTree/MergeTreeSettings.h
Co-authored-by: Sergei Trifonov <svtrifonov@gmail.com>
2023-01-13 20:20:08 +01:00
Han Fei
a258a39eb1 Merge branch 'master' into hanfei/async-cache 2023-01-13 20:17:58 +01:00
Arthur Passos
1b3c7766f8
Merge branch 'master' into add_select_final_mt_setting 2023-01-12 08:31:04 -03:00
Igor Nikonov
1d002e45a9 Fix test & review comments 2023-01-11 12:49:01 +00:00
Igor Nikonov
e75df5e76b Test
+ fix some review comments
2023-01-10 22:14:54 +00:00
Han Fei
bef6168c05 support cache for async inserts block ids 2023-01-10 13:19:12 +01:00
kssenii
2a030c1dc0 Merge remote-tracking branch 'upstream/master' into allow-single-disk-instead-of-storage-policy 2023-01-06 15:58:27 +01:00
HarryLeeIBM
0839ff60ee Initial Checkin of full text search 2022-12-31 18:00:17 -08:00
Dmitry Novik
2ec720c5c4 Add part removal limit for one CleanupThread iteration 2022-12-20 19:15:13 +00:00
kssenii
ea5b06023a Merge remote-tracking branch 'refs/remotes/upstream/master' into allow-single-disk-instead-of-storage-policy 2022-12-14 13:59:51 +01:00
Arthur Passos
6ee21e63d7 Move force select final to user lvel setting and remove ignore force select final 2022-11-25 13:31:08 -03:00
Alexey Milovidov
446d337e02
Merge branch 'master' into compress-marks-primary-key-by-default 2022-11-25 15:24:10 +01:00
Han Fei
32c7fbec55 use another dedup path for async inserts 2022-11-22 15:12:00 +01:00
Alexey Milovidov
915485ff88
Merge branch 'master' into add_select_final_mt_setting 2022-11-20 08:51:12 +01:00
kssenii
b5b25c2676 Merge remote-tracking branch 'upstream/master' into allow-single-disk-instead-of-storage-policy 2022-11-07 17:02:41 +01:00
Antonio Andelic
611c2e2bd7 Support for optimizing old parts for entire partition only 2022-10-25 13:40:18 +00:00
Alexey Milovidov
4e9d894e24 Compress marks and primary key by default 2022-10-23 01:32:35 +02:00