Antonio Andelic
11aba7597e
Check if initialization is done on write
2022-11-16 13:33:22 +00:00
Antonio Andelic
0e5e58bed9
Remove exception if shared ID already created
2022-11-15 12:18:24 +00:00
Alexander Tokmakov
c64109a2de
Merge branch 'master' into fetch-remove-covered
2022-11-14 18:23:16 +03:00
Igor Nikonov
f8c907c463
Revert "Revert " Keeper retries during insert (clean)""
2022-11-10 13:14:04 +01:00
Alexander Tokmakov
fb8c527a2f
Revert " Keeper retries during insert (clean)"
2022-11-10 14:12:24 +03:00
Igor Nikonov
379113f8f1
Merge remote-tracking branch 'origin/master' into igor/insert_zk_retries_retry
2022-11-09 13:03:32 +00:00
Alexander Tokmakov
eb19df0094
fix
2022-11-08 15:21:25 +01:00
Azat Khuzhin
2bd4a71150
Merge branch 'master' into fetch-remove-covered
2022-11-08 00:55:00 +01:00
Alexander Tokmakov
9210e586d2
fix
2022-11-07 20:27:18 +01:00
Alexander Tokmakov
e67d05604c
Merge branch 'master' into fix_intersecting_parts2
2022-11-02 19:30:27 +01:00
Alexander Tokmakov
d8b3a2af84
fix race between INSERT and DROP
2022-11-02 19:26:50 +01:00
Alexander Tokmakov
00c9e50ee3
fix race between INSERT and ALTER PARTITION
2022-11-02 16:25:26 +01:00
Alexander Tokmakov
4f50a999bb
remove abandonable_lock part 1
2022-11-01 21:35:25 +01:00
Igor Nikonov
e76c3c381f
Better handling tables in read-only mode during insert
...
+ throw immediately if table has no metadata in zk
+ stop retries in case of shutdown
+ check if table is readonly at the begining of every retry
2022-11-01 18:47:51 +00:00
Igor Nikonov
9cc64a0fcc
Merge remote-tracking branch 'origin/master' into igor/insert_zk_retries_retry
2022-10-31 18:21:21 +00:00
Igor Nikonov
2d5050ea66
Fix review comments
2022-10-31 18:20:58 +00:00
Alexander Tokmakov
c68ab231f9
fix accessing part in Deleting state
2022-10-30 17:30:51 +01:00
Igor Nikonov
3682913172
Merge remote-tracking branch 'origin/master' into igor/insert_zk_retries_retry
2022-10-27 22:01:53 +00:00
Azat Khuzhin
8db31beee6
Remove covered parts for fetched part
...
Here is an example that I found on production, simplified.
Consider the following queue (nothing of this had been processed on this
replica):
- GET_PART all_0_0_0 (queue-0000000001)
- GET_PART all_1_1_0 (queue-0000000002)
...
- GET_PART all_0_1_1 (queue-0000000003)
- GET_PART all_2_2_0 (queue-0000000004)
...
- MERGE_PARTS from [all_0_1_1, all_2_2_0] to all_0_2_2 (queue-0000000005)
And now queue-0000000005 started to executing (either because
of reording, or because at that time GET_PART fails), and it
does not have any required parts, so it will fetch them, but
not all_0_0_0 and all_1_1_0, so this replica delay will set to
the time of min(queue-0000000001, queue-0000000002), while it
is not true, since it already have parts that covers those
parts.
and since MERGE_PARTS takes 30min, it increased the replica delay
eventually to 30min, for the time range of 30min, which is pretty huge.
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-10-26 11:10:50 +03:00
Anton Popov
c8199bc125
Merge branch 'master' into refactor-data-part
2022-10-25 14:31:05 +02:00
Igor Nikonov
54013009d3
Code migration from #41734 to align diff and git blame.
...
Initial implementation was different and it changed the entire
ReplicatedMergeTreeSink::commitPart() which change history provided by git blame.
Then RetriesControl.retryLoop() was introduced later which significantly reduces
the diff since it's like while() used before.
So, check outing the current version will keep more original history in
git blame, which is useful here
2022-10-24 08:20:11 +00:00
Anton Popov
b40d9200d2
better semantic of constsness of DataPartStorage
2022-10-23 15:24:20 +00:00
Igor Nikonov
c0d1be9cab
Merge pull request #42560 from ClickHouse/igor/replicated_merge_tree_some_polishing
...
Some polishing: replicated merge tree
2022-10-23 15:15:24 +02:00
Anton Popov
56e5daba0c
remove DataPartStorageBuilder
2022-10-23 00:23:15 +00:00
Igor Nikonov
730a3ba849
Merge branch 'master' into igor/replicated_merge_tree_some_polishing
2022-10-22 19:54:09 +02:00
Igor Nikonov
851f556d65
Remove unused parameter
2022-10-21 14:35:37 +02:00
Igor Nikonov
13d31eefbc
Replicated merge tree polishing
...
+ make read-only members const
+ reduce memory allocations
2022-10-21 14:02:52 +02:00
Azat Khuzhin
4e76629aaf
Fixes for -Wshorten-64-to-32
...
- lots of static_cast
- add safe_cast
- types adjustments
- config
- IStorage::read/watch
- ...
- some TODO's (to convert types in future)
P.S. That was quite a journey...
v2: fixes after rebase
v3: fix conflicts after #42308 merged
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-10-21 13:25:19 +02:00
alesapin
24d02b5516
Merge pull request #42441 from ClickHouse/check_readonly_in_background_tasks
...
Deactivate tasks in ReplicatedMergeTree until startup
2022-10-20 15:36:20 +02:00
Alexander Tokmakov
d668a82829
Merge branch 'master' into improve_replica_recovery
2022-10-20 14:25:09 +03:00
Antonio Andelic
77eb353839
Merge pull request #42243 from ClickHouse/use-multiread-more
...
Use MultiRead where possible
2022-10-20 09:04:54 +02:00
Alexander Tokmakov
84ef32e33f
Merge branch 'master' into improve_replica_recovery
2022-10-19 16:28:01 +02:00
Alexey Milovidov
7befe2825c
Update StorageReplicatedMergeTree.cpp
2022-10-18 17:36:11 +03:00
Alexey Milovidov
0623ad4e37
Update StorageReplicatedMergeTree.cpp
2022-10-18 17:35:44 +03:00
alesapin
9769057767
Deactivate all tasks
2022-10-18 15:53:52 +02:00
alesapin
ab8da5a539
Deactivate mutationsFinalizingTask until startup
2022-10-18 15:50:02 +02:00
Alexander Tokmakov
57f6e27300
Merge branch 'master' into improve_replica_recovery
2022-10-17 18:37:31 +02:00
alesapin
051e3a0db5
Fix if
2022-10-17 12:27:05 +02:00
alesapin
e0c2518416
Fix another trash in zero-copy replication
2022-10-14 16:50:48 +02:00
Antonio Andelic
5e6a7c1678
Use multiread where possible
2022-10-11 09:27:46 +00:00
alesapin
f695b75eef
Merge pull request #42147 from ClickHouse/more_logs_for_parent_parts
...
Fix checking parent for old-format parts
2022-10-08 12:08:15 +02:00
Alexander Tokmakov
ec2bca51ca
Merge branch 'master' into improve_replica_recovery
2022-10-07 16:35:57 +02:00
Alexander Tokmakov
b8d9066004
Revert "Resurrect parallel distributed insert select with s3Cluster ( #41535 )"
...
This reverts commit 860e34e760
2022-10-07 15:53:30 +02:00
alesapin
f1d93d52d1
Fix checking parent for old-format parts
2022-10-07 14:49:25 +02:00
Alexander Tokmakov
bb78bf1c70
improve replica recovery
2022-10-06 21:09:12 +02:00
alesapin
3373480ede
Merge pull request #42036 from ClickHouse/fix_one_more_no_such_key
...
Fix No such key during table drop
2022-10-06 14:50:02 +02:00
Nikita Mikhaylov
860e34e760
Resurrect parallel distributed insert select with s3Cluster ( #41535 )
2022-10-06 13:47:32 +02:00
Nikita Mikhaylov
766107df0a
Fixed the restriction on maximum size of replicated fetches ( #42090 )
2022-10-06 13:18:46 +02:00
alesapin
adb34f6eae
Merge branch 'master' into fix_one_more_no_such_key
2022-10-06 12:42:16 +02:00
alesapin
6d24e16fda
Merge pull request #42067 from ClickHouse/fix_drop_of_completely_dropped_table
...
Fix drop of completely dropped table
2022-10-06 12:40:38 +02:00
