Sema Checherinda
c976b28104
less pedantic checks in operations, rely on existed tx functionality
2022-11-23 15:16:09 +00:00
Sema Checherinda
c1e6b560f0
no groupping parts for cover with empty
2022-11-23 15:16:08 +00:00
Sema Checherinda
db86cd0f0a
fix several critical races, fix tests
2022-11-23 15:16:08 +00:00
Sema Checherinda
f2b9f80f2f
fix tidy build
2022-11-23 15:16:08 +00:00
Sema Checherinda
15012c7070
address the review comments
2022-11-23 15:16:08 +00:00
Sema Checherinda
fb86019db9
Delete parts over replacing them with empty parts
2022-11-23 15:16:08 +00:00
Anton Popov
75c807609b
add some comments
2022-11-22 16:11:01 +00:00
Anton Popov
e3db29aebf
better waiting for outdated data parts
2022-11-22 15:15:20 +00:00
Anton Popov
92ac9385b9
wait loading of outdated parts for partition commands
2022-11-21 17:01:46 +00:00
Anton Popov
f94182384a
Merge remote-tracking branch 'upstream/master' into HEAD
2022-11-16 03:30:29 +00:00
Alexey Milovidov
4966556f47
Miscellaneous changes
2022-11-14 06:41:51 +01:00
Alexander Tokmakov
e415fc6e95
Revert "Optimize TTL merge, completely expired parts can be removed in time"
2022-11-10 19:38:55 +03:00
zhongyuankai
c1e2b2af74
Optimize TTL merge, completely expired parts can be removed in time
2022-11-01 22:58:15 +08:00
Anton Popov
c4d4f2dbed
better interface
2022-10-25 22:14:06 +00:00
Anton Popov
c8199bc125
Merge branch 'master' into refactor-data-part
2022-10-25 14:31:05 +02:00
Anton Popov
cf375c9732
better interface
2022-10-23 22:29:24 +00:00
Anton Popov
b40d9200d2
better semantic of constsness of DataPartStorage
2022-10-23 15:24:20 +00:00
Anton Popov
56e5daba0c
remove DataPartStorageBuilder
2022-10-23 00:23:15 +00:00
Azat Khuzhin
4e76629aaf
Fixes for -Wshorten-64-to-32
...
- lots of static_cast
- add safe_cast
- types adjustments
- config
- IStorage::read/watch
- ...
- some TODO's (to convert types in future)
P.S. That was quite a journey...
v2: fixes after rebase
v3: fix conflicts after #42308 merged
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-10-21 13:25:19 +02:00
Alexander Tokmakov
0d8a814d80
Merge pull request #42254 from xinhuitian/optimize-selectPartsToMutate
...
Early return of corner cases in selectPartsToMutate function
2022-10-20 18:13:37 +03:00
Tian Xinhui
45c6974f7d
Use alias for std::shared_ptr<MergeMutateSelectedEntry> ( #42211 )
...
* use alias for MergeMutateSelectedEntry share ptr
* fix StorageMergeTree.cpp
2022-10-18 13:35:34 +02:00
Anton Popov
95fdb0a877
remove outdated parts asynchronously
2022-10-14 14:52:57 +00:00
xinhuitian
daaf01c344
early return
2022-10-12 12:18:17 +08:00
alesapin
695133cc5b
Better mutations
2022-09-27 15:23:02 +02:00
Anton Popov
ee54451741
fix adding of columns of type Object
2022-09-13 22:43:59 +00:00
Vitaly Baranov
122009a2bd
Use table lock if database is ordinary and zero-copy-replication is enabled.
2022-09-08 13:54:59 +02:00
Alexey Milovidov
940a53e519
Merge pull request #40984 from Lucky-Chang/typo_fix
...
Fix some typos and clang-tidy warnings
2022-09-06 02:37:50 +03:00
Luck-Chang
1ac8e739c9
fix some typos and clang-tidy warnings
2022-09-05 09:50:24 +08:00
Antonio Andelic
e64436fef3
Fix typos with new codespell
2022-09-02 08:54:48 +00:00
Alexander Tokmakov
0aea4c7184
fix
2022-08-25 14:22:27 +02:00
Alexander Tokmakov
3cd26aafe4
remove wrong code for skipping mutations in MergeTree
2022-08-24 20:58:59 +02:00
Alexander Tokmakov
fe572104aa
fix old tmp dirs cleanup
2022-08-09 18:44:51 +02:00
Alexander Gololobov
25deba2c1b
Merge branch 'master' into feature/sql-standard-delete
2022-07-25 22:13:20 +02:00
Alexander Gololobov
1ea9f143ff
Leave only _row_exists-based implementation of lightweight delete
2022-07-21 11:26:13 +02:00
Raúl Marín
aea045f297
Improve logging around replicated merges
2022-07-15 15:48:35 +02:00
jianmei zhang
4268ac8b0e
Add supportLightweightDeleteMutate() in IMergeTreeDataPart to disable LWD for part with projections
2022-07-15 12:32:42 +08:00
jianmei zhang
198674e994
Consider lightweight delete as ordinary for compact part in selectPartsToMutate()
2022-07-15 12:32:42 +08:00
jianmei zhang
780cdfb8f0
Code changes based on review opinions. Make ordinary single alter delete lightwight by default
2022-07-15 12:32:42 +08:00
jianmei zhang
b4a37e1e22
Disable optimizations for count() when lightweight delete exists, add hasLightweightDelete() function in IMergeTreeDataPart
2022-07-15 12:32:41 +08:00
jianmei zhang
8696319d62
Support lightweight delete execution using string as deleted rows mask,also part of select can handle LWD
2022-07-15 12:32:41 +08:00
jianmei zhang
8df7b7a030
Support new mutation type for lightweight
2022-07-15 12:32:41 +08:00
Amos Bird
0795cdd36a
Slightly better interface of waitForMutation
2022-07-13 12:34:12 +08:00
Vitaly Baranov
ed27987646
Merge pull request #38861 from vitlibar/backup-improvements-9
...
Backup Improvements 9
2022-07-07 02:24:47 +02:00
Vitaly Baranov
7f84cf3968
Fix style.
2022-07-06 16:36:59 +02:00
Vitaly Baranov
43d35eec1b
Write unfinished mutations to backup.
2022-07-05 14:51:09 +02:00
alesapin
b2db49dbf1
Merge branch 'master' into better_data_part_storage_builder
2022-07-02 15:39:44 +02:00
alesapin
0d7298f3a9
Fix typos
2022-07-01 15:26:27 +02:00
alesapin
57284e6a9d
Fix possible deadlocks with MergeTreeData::Transaction
2022-07-01 15:16:32 +02:00
alesapin
d88cdcd5c1
AAA
2022-06-27 21:41:29 +02:00
alesapin
80d27f6381
Merge pull request #38380 from ClickHouse/simplify_parts_commit_merge_tree
...
Simplify parts commit methods
2022-06-25 00:12:52 +02:00
alesapin
011d58d7a0
Simplify more
2022-06-24 17:19:59 +02:00
alesapin
37310dc9df
Simpler
2022-06-24 14:10:15 +02:00
alesapin
36b810b076
Remove unused parameter
2022-06-24 13:42:36 +02:00
alesapin
af1a9d18ab
Remove transaction argument
2022-06-24 13:34:00 +02:00
alesapin
9910395823
Simplify method signature
2022-06-24 13:19:29 +02:00
alesapin
0f8eed98c4
Merge branch 'master' into better-support-gcp
2022-06-22 12:46:01 +02:00
Nikolai Kochetov
cc6fdfe0eb
Merge pull request #36555 from ClickHouse/refactor-something-in-part-volumes
...
Separate data storage abstraction for MergeTree
2022-06-22 11:13:36 +02:00
Anton Popov
d5d568e235
Merge pull request #37913 from azat/parts-refcnt
...
Fix refcnt for unused MergeTree parts in SELECT queries
2022-06-21 22:56:26 +02:00
Nikolai Kochetov
854d148d73
Add some comments. Remove some commented code.
2022-06-21 12:31:02 +00:00
Nikolai Kochetov
1e8c9ecd4c
Merge branch 'master' into refactor-something-in-part-volumes
2022-06-21 12:37:21 +02:00
Nikolai Kochetov
dccf90b1ea
Cleanup.
2022-06-20 18:18:17 +00:00
mergify[bot]
9bdd9e14a6
Merge branch 'master' into fix_flaky_tests_with_transactions
2022-06-20 18:11:30 +00:00
Nikolai Kochetov
7452d04e3a
Merge branch 'master' into refactor-something-in-part-volumes
2022-06-20 15:31:02 +02:00
Alexander Tokmakov
83adf56383
fix race
2022-06-17 18:13:57 +02:00
mergify[bot]
7f24574609
Merge branch 'master' into better-support-gcp
2022-06-15 23:18:20 +00:00
Vitaly Baranov
5cabdbd982
Restore parts of MergeTree in correct order.
2022-06-15 20:28:40 +02:00
Nikolai Kochetov
2a9a63ac7b
Merge branch 'master' into refactor-something-in-part-volumes
2022-06-15 15:35:26 +02:00
Anton Popov
a59be0fd5d
better support of GCP storage
2022-06-15 00:23:45 +00:00
kssenii
4859e11c27
Review fixes
2022-06-12 01:15:10 +02:00
kssenii
fd6c84b831
Review fixes
2022-06-10 14:23:13 +02:00
kssenii
7a2676c7ab
Clean up broken detached parts with timeout
2022-06-10 12:27:57 +02:00
Nikolai Kochetov
5bc9b32025
Merge branch 'master' into refactor-something-in-part-volumes
2022-06-08 11:10:06 +00:00
Azat Khuzhin
61a2613127
Fix refcnt for unused MergeTree parts in SELECT queries
...
Before this patch SELECT queries hold parts even if they were not
required by select (had been eliminated by partition pruning).
This defers removing parts if you have long running queries.
This had been introduced in #23932 , with introduction of
StorageSnapshotPtr.
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-06-08 07:04:53 +03:00
alesapin
e32d36d790
Proper fix
2022-06-07 17:58:32 +02:00
Nikolai Kochetov
678d978acf
Merge branch 'master' into refactor-something-in-part-volumes
2022-06-07 15:23:00 +00:00
Nikolai Kochetov
89c5855d20
Merge branch 'master' into refactor-something-in-part-volumes
2022-06-02 12:19:07 +02:00
Nikolai Kochetov
edac3d6714
Merge branch 'master' into refactor-read-metrics-and-callbacks
2022-06-02 09:36:20 +00:00
lthaooo
6632616733
Fix TTL merge scheduling bug ( #36387 )
2022-06-01 21:09:53 +02:00
Nikolai Kochetov
86fbb74703
Merge branch 'master' into refactor-read-metrics-and-callbacks
2022-05-31 18:07:47 +00:00
Nikolai Kochetov
c71256ea38
Remove some commented code.
2022-05-30 13:18:20 +00:00
Nikolai Kochetov
5b4658aa5e
Merge branch 'master' into refactor-read-metrics-and-callbacks
2022-05-30 09:47:35 +00:00
alesapin
be1c3c132b
Fix some trash
2022-05-27 16:08:49 +02:00
Alexander Tokmakov
eb71dd4c78
Merge pull request #37547 from ClickHouse/followup_37398
...
Follow-up to #37398
2022-05-26 20:29:41 +03:00
Alexander Tokmakov
e8f33fb0d9
fix flaky tests
2022-05-26 14:17:05 +02:00
Azat Khuzhin
dc9ca3d70c
Fix LOGICAL_ERROR in getMaxSourcePartsSizeForMerge during merges ( #37413 )
2022-05-26 14:14:58 +02:00
Nikolai Kochetov
fd97a9d885
Move some resources
2022-05-23 19:47:32 +00:00
Nikolai Kochetov
fc3645a483
Fix some tests.
2022-05-03 19:32:24 +00:00
Nikolai Kochetov
35095191eb
Merge branch 'master' into refactor-something-in-part-volumes
2022-05-03 17:51:47 +02:00
Robert Schulze
330212e0f4
Remove inherited create() method + disallow copying
...
The original motivation for this commit was that shared_ptr_helper used
std::shared_ptr<>() which does two heap allocations instead of
make_shared<>() which does a single allocation. Turned out that
1. the affected code (--> Storages/) is not on a hot path (rendering the
performance argument moot ...)
2. yet copying Storage objects is potentially dangerous and was
previously allowed.
Hence, this change
- removes shared_ptr_helper and as a result all inherited create() methods,
- instead, Storage objects are now created using make_shared<>() by the
caller (for that to work, many constructors had to be made public), and
- all Storage classes were marked as noncopyable using boost::noncopyable.
In sum, we are (likely) not making things faster but the code becomes
cleaner and harder to misuse.
2022-05-02 08:46:52 +02:00
Nikolai Kochetov
e44af67fee
Merge branch 'master' into refactor-something-in-part-volumes
2022-04-26 21:08:00 +02:00
Vitaly Baranov
68a020ecea
Implement BACKUP/RESTORE ON CLUSTER.
2022-04-25 16:34:33 +02:00
Nikolai Kochetov
8c00692844
Part 8
2022-04-22 16:58:09 +00:00
Nikolai Kochetov
9133e398b8
Part 7
2022-04-21 19:19:13 +00:00
alesapin
5465415751
Fix replace/move partition with zero copy replication
2022-04-21 14:39:12 +02:00
alesapin
40c15222f8
Merge branch 'master' into fix_trash
2022-04-20 12:45:49 +02:00
alesapin
7cb7c120cc
Less ugly
2022-04-19 15:53:10 +02:00
Robert Schulze
118e94523c
Activate clang-tidy warning "readability-container-contains"
...
This check suggests replacing <Container>.count() by
<Container>.contains() which is more speaking and in case of
multimaps/multisets also faster.
2022-04-18 23:53:11 +02:00
Alexander Tokmakov
66fdf35dfd
remove outdated parts immediately on drop partition
2022-04-13 18:01:22 +02:00
Alexander Tokmakov
49c35f3261
Merge branch 'master' into mvcc_prototype
2022-04-08 13:34:40 +02:00
alesapin
8ec802bc62
Merge pull request #35475 from kssenii/remote-fs-cache-improvements
...
Allow to write remote fs cache on all write operations. Add `system.remote_filesystem_cache` table. Add `drop remote filesystem cache (<path>)` query. Add `system.remote_data_paths` table.
2022-04-08 12:06:26 +02:00
