Alexander Tokmakov
a4df1cff52
Update ReplicatedMergeTreeQueue.cpp
2022-12-13 16:04:47 +03:00
Alexander Tokmakov
0342e6b236
fix
2022-11-28 16:02:37 +01:00
Alexander Tokmakov
5cc99312d7
provide hint for loading uncommitted blocks in merge predicate
2022-11-25 16:41:20 +01:00
alesapin
52821a331a
Update src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp
...
Co-authored-by: Alexander Tokmakov <tavplubix@clickhouse.com>
2022-11-24 14:04:03 +01:00
alesapin
188069117e
Update src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp
...
Co-authored-by: Alexander Tokmakov <tavplubix@clickhouse.com>
2022-11-24 14:02:26 +01:00
alesapin
5355f82448
Update src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp
...
Co-authored-by: Alexander Tokmakov <tavplubix@clickhouse.com>
2022-11-24 14:02:19 +01:00
alesapin
06f395d946
Add comments
2022-11-17 13:10:14 +01:00
alesapin
2012f4bd36
Fix use-after-free
2022-11-16 11:50:51 +01:00
alesapin
a12f2d14dc
Simpler check
2022-11-15 14:46:38 +01:00
alesapin
4ec11d0f58
Remove redundant branch
2022-11-15 13:58:53 +01:00
alesapin
6a50a32cfc
Fix parts_to_do
2022-11-15 00:53:20 +01:00
alesapin
f53ada2bbd
Revert "Fix bug in replication queue which can lead to premature mutation finish"
...
This reverts commit 3b5424811c
2022-11-14 22:59:09 +01:00
alesapin
3b5424811c
Fix bug in replication queue which can lead to premature mutation finish
2022-11-14 21:53:16 +01:00
Alexander Tokmakov
c64109a2de
Merge branch 'master' into fetch-remove-covered
2022-11-14 18:23:16 +03:00
Alexander Tokmakov
bb00a246f1
priority for DROP_RANGEs, assertion for others
2022-11-10 20:53:05 +01:00
Alexander Tokmakov
acbad99b24
remove abandonable_lock part 2
2022-11-02 14:28:49 +01:00
Azat Khuzhin
8db31beee6
Remove covered parts for fetched part
...
Here is an example that I found on production, simplified.
Consider the following queue (nothing of this had been processed on this
replica):
- GET_PART all_0_0_0 (queue-0000000001)
- GET_PART all_1_1_0 (queue-0000000002)
...
- GET_PART all_0_1_1 (queue-0000000003)
- GET_PART all_2_2_0 (queue-0000000004)
...
- MERGE_PARTS from [all_0_1_1, all_2_2_0] to all_0_2_2 (queue-0000000005)
And now queue-0000000005 started to executing (either because
of reording, or because at that time GET_PART fails), and it
does not have any required parts, so it will fetch them, but
not all_0_0_0 and all_1_1_0, so this replica delay will set to
the time of min(queue-0000000001, queue-0000000002), while it
is not true, since it already have parts that covers those
parts.
and since MERGE_PARTS takes 30min, it increased the replica delay
eventually to 30min, for the time range of 30min, which is pretty huge.
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-10-26 11:10:50 +03:00
Azat Khuzhin
4e76629aaf
Fixes for -Wshorten-64-to-32
...
- lots of static_cast
- add safe_cast
- types adjustments
- config
- IStorage::read/watch
- ...
- some TODO's (to convert types in future)
P.S. That was quite a journey...
v2: fixes after rebase
v3: fix conflicts after #42308 merged
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-10-21 13:25:19 +02:00
Alexander Tokmakov
5b03bbce2e
Merge pull request #41981 from ClickHouse/fixes_for_replicated_merge_tree2
...
Some fixes for ReplicatedMergeTree
2022-10-20 16:57:51 +03:00
Alexander Tokmakov
ea77ff9022
Merge branch 'master' into fixes_for_replicated_merge_tree2
2022-10-18 17:25:25 +02:00
Antonio Andelic
5e6a7c1678
Use multiread where possible
2022-10-11 09:27:46 +00:00
Alexander Tokmakov
3f9c9c9a77
fix intersecting part if session expired during queue initialization
2022-10-03 12:27:17 +02:00
Antonio Andelic
fcc54104a4
Define methods for multi read requests
2022-09-29 09:19:47 +00:00
Antonio Andelic
97385ca861
Format
2022-09-26 09:33:50 +00:00
Antonio Andelic
cc3719e463
Use multiread
2022-09-26 09:07:34 +00:00
Alexander Tokmakov
387b69caea
fix race condition
2022-08-16 14:03:31 +02:00
Alexander Tokmakov
b3f3b60ac3
allow parallel execution of disjoint drop ranges
2022-08-15 20:32:03 +02:00
Alexander Tokmakov
d82f378a9d
do not enqueue uneeded parts for check
2022-07-18 23:37:07 +02:00
Alexander Tokmakov
435d98738d
Update ReplicatedMergeTreeQueue.cpp
2022-07-14 22:41:01 +03:00
Alexander Tokmakov
d2cbdc7c53
Update ReplicatedMergeTreeQueue.cpp
2022-06-30 17:23:53 +03:00
Alexander Tokmakov
5ffc38d9b9
fix style check, add comments
2022-06-17 13:47:14 +02:00
Alexander Tokmakov
39c0219c11
fixes
2022-06-16 19:41:32 +02:00
Alexander Tokmakov
6ddfec47d2
maybe fix livelock on queue processing
2022-06-15 22:22:27 +02:00
Alexander Tokmakov
e4d4d0a854
fix race on queue processing
2022-06-15 16:21:26 +02:00
Alexander Tokmakov
06f80770b8
fix stuck REPALCE_RANGE
2022-06-01 20:11:53 +02:00
alesapin
5dcd25be23
Initial implementation
2022-05-12 00:04:54 +02:00
Alexander Tokmakov
b1949ee901
Merge pull request #36680 from ClickHouse/better_logs_virtual_parts
...
Better logs for virtual parts
2022-04-26 21:36:20 +03:00
Alexander Tokmakov
d149ac8bd7
better logs for virtual parts
2022-04-26 16:58:40 +02:00
Robert Schulze
b6d7367538
Merge remote-tracking branch 'origin/master' into erase_if3
...
Conflicts:
- Interpreters/ActionsDAG.cpp
2022-04-20 10:02:59 +02:00
Robert Schulze
118e94523c
Activate clang-tidy warning "readability-container-contains"
...
This check suggests replacing <Container>.count() by
<Container>.contains() which is more speaking and in case of
multimaps/multisets also faster.
2022-04-18 23:53:11 +02:00
Robert Schulze
1e1df8e101
Replace remove-erase idiom by C++20 erase()/erase_if()
...
- makes the code less verbose while being as efficient
2022-04-17 12:04:47 +02:00
tavplubix
3af2384066
Merge pull request #36190 from azat/revert-27002-mutation-stuck
...
Revert "Fix possible mutation stuck due to race with DROP_RANGE"
2022-04-14 15:03:35 +03:00
Azat Khuzhin
6dbc138e76
Remove duplicated parts removal from mutation.
...
This commit reverts 701e2ffd79
2022-04-13 16:09:17 +03:00
Alexander Tokmakov
457a9e9691
fixes for ReplicatedMergeTree
2022-04-12 14:14:26 +02:00
Alexander Tokmakov
9702b5177d
Merge branch 'master' into mvcc_prototype
2022-03-14 21:45:38 +01:00
Maksim Kita
2fdcf53a76
Fix clang-tidy warnings in Server, Storages folders
2022-03-14 18:17:35 +00:00
Alexander Tokmakov
dae044f86b
Merge branch 'master' into mvcc_prototype
2022-02-17 13:49:37 +03:00
alesapin
3af06b23f8
POC
2022-02-10 22:45:52 +03:00
alesapin
f764da35ca
Also zero copy mutations
2022-02-10 14:15:08 +03:00
alesapin
70221b272b
Better solution
2022-02-10 12:57:11 +03:00
