Commit Graph

191 Commits

Author SHA1 Message Date
Alexey Milovidov
8cd2e7c7d6 Merge branch 'master' into fix-potentially-bad-code 2023-07-18 22:18:22 +02:00
Alexey Milovidov
095624fbd1 Remove default argument value 2023-07-15 23:33:11 +02:00
Alexey Milovidov
27f41869a9 Remove code that I don't like 2023-06-25 09:11:42 +02:00
Antonio Andelic
b11f744252
Correctly disable async insert with deduplication when it's not needed (#50663)
* Correctly disable async insert when it's not used

* Better

* Add comment

* Better

* Fix tests

---------

Co-authored-by: Nikita Mikhaylov <mikhaylovnikitka@gmail.com>
2023-06-07 20:33:08 +02:00
Anton Popov
612173e734 refactoring near alter conversions 2023-05-25 22:54:54 +00:00
Anton Popov
ff287c7bba add lock 2023-05-10 23:00:49 +00:00
Alexey Milovidov
75a2589a42
Merge branch 'master' into deprecate-in-memory-parts 2023-05-06 00:49:59 +03:00
Anton Popov
e61d7baa54 Merge remote-tracking branch 'upstream/master' into HEAD 2023-05-04 14:03:53 +00:00
Alexey Milovidov
f449df85b6 Deprecate in-memory parts 2023-05-03 00:31:09 +02:00
Alexander Tokmakov
fde55f3ea6 fix 2023-04-25 18:25:10 +02:00
Anton Popov
38622d0770 add settings to delay or throw in case of too many mutations 2023-04-24 18:21:49 +00:00
Alexander Tokmakov
19f43ee7c7 Merge branch 'master' into fix_some_tests3 2023-04-24 16:13:27 +02:00
Alexander Tokmakov
70c1000c22 fix some tests 2023-04-14 19:42:17 +02:00
Alexander Gololobov
a7b05589b6 Caches for multiple mutations 2023-04-14 16:12:19 +02:00
Alexander Gololobov
02876ed87d Share PreparedSet-s between task of the same mutation to reduce mem usage 2023-04-14 16:12:18 +02:00
alesapin
e3961c118a
Revert "Revert "Fix several RENAME COLUMN bugs."" 2023-02-27 12:27:57 +01:00
Alexey Milovidov
8f40bbd5b4
Revert "Fix several RENAME COLUMN bugs." 2023-02-26 05:47:05 +03:00
Alexey Milovidov
eb89ab471f
Merge pull request #45911 from ClickHouse/mutations_rename_hang
Fix several `RENAME COLUMN` bugs.
2023-02-26 01:56:40 +03:00
youennL-cs
6526c2a8ab
[RFC] Replacing merge tree new engine (#41005)
* Add new engine to ReplacingMergeTree corresponding to the ReplacingCollapsingMergeTree

* Add new test for the new ReplacingMergeTree engine

* Limit sign value to -1/1

* Add new engine to ReplacingMergeTree corresponding to the ReplacingCollapsingMergeTree

* Add new test for the new ReplacingMergeTree engine

* Limit sign value to -1/1

* Replace sign column(Int8) by is_deleted(UInt8)

* Add new engine to ReplacingMergeTree corresponding to the ReplacingCollapsingMergeTree

* Add new test for the new ReplacingMergeTree engine

* Limit sign value to -1/1

* Replace sign column(Int8) by is_deleted(UInt8)

* Add new engine to ReplacingMergeTree corresponding to the ReplacingCollapsingMergeTree

* Add new test for the new ReplacingMergeTree engine

* Limit sign value to -1/1

* Replace sign column(Int8) by is_deleted(UInt8)

* Add keyword 'CLEANUP' when OPTIMIZE

* Cleanup uniquely when it's a replacingMergeTree

* Propagate CLEANUP information and change from 'with_cleanup' to 'cleanup'

* Cleanup data flagged as 'is_deleted'

* Fix merge when optimize and add a test

* Fix OPTIMIZE and INSERT + add tests

* New fix for cleanup at the merge

* Cleanup debug logs

* Add the SETTINGS option 'clean_deleted_rows' that can be 'never' or 'always'

* Fix regression bug; Now REplicatedMergeTree can be called as before without 'is_deleted'

* Add Replicated tests

* Disable tag 'long' for our test and cleanup some white spaces

* Update tests

* Fix tests and remove additional useless whitespace

* Fix replica test

* Style clean && add condition check for is_deleted values

* clean_deleted_rows settings is nom an enum

* Add valid default value to the clean_deleted_rows settings

* Update cleanup checkers to use the enum and fix typos in the test

* Fix submodule contrib/AMQP-CPP pointer

* Add missing messages in test reference and remove a print with non derterministic order

* fix replica test reference

* Fix edge case

* Fix a typo for the spell checker

* Fix reference

* Fix a condition to raise an error if is_deleted differ from 0/1 and cleanup

* Change tests file name and update number

* This should fix the ReplacingMergeTree parameter set

* Fix replicated parameters

* Disable allow_deprecated_syntax_for_merge_tree for our new column

* Fix a test

* Remove non deterministic order print in the test

* Test on replicas

* Remove a condition, when checking optional parameters, that should not be sueful since we disabled the deprected_syntaxe

* Revert "Remove a condition, when checking optional parameters, that should not be useful since we disabled the deprected_syntaxe"

This reverts commit b65d64c05e.

* Fix replica management and limit the number of argument to two maximum, due to the possiblity of deprecated table create/attach failing otherwise

* Test a fix for replicated log information error

* Try to add sync to have consistent results

* Change path of replicas that should cause one issue and add few prints in case it's not that

* Get cleanup info on replicas only if information found

* Fix style issues

* Try to avoid replication error 'cannot select parts...' and and replica read/write field order

* Cleanup according to PR reviews
 and add tests on error raised.

* Update src/Storages/MergeTree/registerStorageMergeTree.cpp

Co-authored-by: Alexander Tokmakov <tavplubix@gmail.com>

* Select ... FINAL don't show rows with is_deleted = true

* Update and fix SELECT ... FINAL merge parameter

* Remove is_deleted rows only on the version inserted when merge

* Fix (master) updates issues

* Revert changes that should not be commited

* Add changes according to review

* Revert changes that should not be commited - part 2

---------

Co-authored-by: Alexander Tokmakov <tavplubix@gmail.com>
2023-02-16 16:03:16 +03:00
alesapin
7a0d0f0c53 Some code which doesn't work 2023-02-03 19:14:49 +01:00
alesapin
bdc530dead Fix style 2023-02-02 17:30:38 +01:00
Alexander Gololobov
863548114d force_wait flag is no longer needed 2023-01-30 18:38:28 +01:00
Sema Checherinda
b76b612d23
fix typo 2023-01-20 14:55:58 +01:00
Sema Checherinda
02f22f04e8
fix typos 2023-01-20 13:35:23 +01:00
Sema Checherinda
09f3a5c599 add a comment, add a check, fix test 2023-01-20 12:10:31 +01:00
Sema Checherinda
c51f4d7be1 do not merge over a gap with uotdate parts, delete empty parts with respect to old_parts_lifetime 2023-01-16 16:47:11 +01:00
Alexander Gololobov
2c64f0042a Added force_wait parameter to IStorage::mutate() 2022-12-30 16:23:03 +01:00
Sema Checherinda
4f592bc208 fix isolation breach, droping outdated part should't be ignored 2022-11-23 15:16:09 +00:00
Sema Checherinda
db86cd0f0a fix several critical races, fix tests 2022-11-23 15:16:08 +00:00
Sema Checherinda
fb86019db9 Delete parts over replacing them with empty parts 2022-11-23 15:16:08 +00:00
Azat Khuzhin
4e76629aaf Fixes for -Wshorten-64-to-32
- lots of static_cast
- add safe_cast
- types adjustments
  - config
  - IStorage::read/watch
  - ...
- some TODO's (to convert types in future)

P.S. That was quite a journey...

v2: fixes after rebase
v3: fix conflicts after #42308 merged
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-10-21 13:25:19 +02:00
Tian Xinhui
45c6974f7d
Use alias for std::shared_ptr<MergeMutateSelectedEntry> (#42211)
* use alias for MergeMutateSelectedEntry share ptr

* fix StorageMergeTree.cpp
2022-10-18 13:35:34 +02:00
Alexander Tokmakov
3cd26aafe4 remove wrong code for skipping mutations in MergeTree 2022-08-24 20:58:59 +02:00
Alexander Gololobov
1ea9f143ff Leave only _row_exists-based implementation of lightweight delete 2022-07-21 11:26:13 +02:00
jianmei zhang
780cdfb8f0 Code changes based on review opinions. Make ordinary single alter delete lightwight by default 2022-07-15 12:32:42 +08:00
jianmei zhang
b4a37e1e22 Disable optimizations for count() when lightweight delete exists, add hasLightweightDelete() function in IMergeTreeDataPart 2022-07-15 12:32:41 +08:00
jianmei zhang
8df7b7a030 Support new mutation type for lightweight 2022-07-15 12:32:41 +08:00
Amos Bird
0795cdd36a
Slightly better interface of waitForMutation 2022-07-13 12:34:12 +08:00
Vitaly Baranov
43d35eec1b Write unfinished mutations to backup. 2022-07-05 14:51:09 +02:00
alesapin
011d58d7a0 Simplify more 2022-06-24 17:19:59 +02:00
Vitaly Baranov
5cabdbd982 Restore parts of MergeTree in correct order. 2022-06-15 20:28:40 +02:00
kssenii
7a2676c7ab Clean up broken detached parts with timeout 2022-06-10 12:27:57 +02:00
Nikolai Kochetov
c71256ea38 Remove some commented code. 2022-05-30 13:18:20 +00:00
Nikolai Kochetov
fd97a9d885 Move some resources 2022-05-23 19:47:32 +00:00
Robert Schulze
777b5bc15b
Don't let storages inherit from boost::noncopyable
... IStorage has deleted copy ctor / assignment already
2022-05-03 09:07:08 +02:00
Robert Schulze
330212e0f4
Remove inherited create() method + disallow copying
The original motivation for this commit was that shared_ptr_helper used
std::shared_ptr<>() which does two heap allocations instead of
make_shared<>() which does a single allocation. Turned out that
1. the affected code (--> Storages/) is not on a hot path (rendering the
performance argument moot ...)
2. yet copying Storage objects is potentially dangerous and was
   previously allowed.

Hence, this change

- removes shared_ptr_helper and as a result all inherited create() methods,

- instead, Storage objects are now created using make_shared<>() by the
  caller (for that to work, many constructors had to be made public), and

- all Storage classes were marked as noncopyable using boost::noncopyable.

In sum, we are (likely) not making things faster but the code becomes
cleaner and harder to misuse.
2022-05-02 08:46:52 +02:00
Vitaly Baranov
68a020ecea Implement BACKUP/RESTORE ON CLUSTER. 2022-04-25 16:34:33 +02:00
Alexander Tokmakov
3c762f566d Merge branch 'master' into mvcc_prototype 2022-03-21 20:16:29 +01:00
Vitaly Baranov
ce25afb2e9 Storages and databases are hollow by default now. 2022-03-20 20:02:15 +01:00
Vitaly Baranov
7f89b98308 Rework BackupSettings and RestoreSettings a little, pass StorageRestoreSettings to storages. 2022-03-20 20:02:15 +01:00