Commit Graph

30 Commits

Author SHA1 Message Date
Alexander Tokmakov
4209ccfc08 fix 2023-06-02 17:42:04 +02:00
Alexander Tokmakov
f62faaedc3 paranoid fix for removing parts from zk 2023-06-01 21:20:39 +02:00
Han Fei
bef6168c05 support cache for async inserts block ids 2023-01-10 13:19:12 +01:00
alesapin
fa467b3398 Minor code polishing 2022-12-22 14:31:42 +01:00
Han Fei
3c11aca040 address comments 2022-12-07 23:40:52 +01:00
Han Fei
14701670b5 Merge branch 'master' into hanfei/aysnc-insert 2022-11-16 17:24:36 +01:00
Han Fei
11f93dec3f tmp commit 2022-11-14 15:53:45 +01:00
Igor Nikonov
f8c907c463
Revert "Revert " Keeper retries during insert (clean)"" 2022-11-10 13:14:04 +01:00
Alexander Tokmakov
fb8c527a2f
Revert " Keeper retries during insert (clean)" 2022-11-10 14:12:24 +03:00
Igor Nikonov
379113f8f1 Merge remote-tracking branch 'origin/master' into igor/insert_zk_retries_retry 2022-11-09 13:03:32 +00:00
alesapin
761274d857 Fix wrong logic 2022-11-09 01:14:03 +01:00
Alexander Tokmakov
eb19df0094 fix 2022-11-08 15:21:25 +01:00
Alexander Tokmakov
9210e586d2 fix 2022-11-07 20:27:18 +01:00
Igor Nikonov
3682913172 Merge remote-tracking branch 'origin/master' into igor/insert_zk_retries_retry 2022-10-27 22:01:53 +00:00
Igor Nikonov
54013009d3 Code migration from #41734 to align diff and git blame.
Initial implementation was different and it changed the entire
ReplicatedMergeTreeSink::commitPart() which change history provided by git blame.
Then RetriesControl.retryLoop() was introduced later which significantly reduces
the diff since it's like while() used before.
So, check outing the current version will keep more original history in
git blame, which is useful here
2022-10-24 08:20:11 +00:00
Anton Popov
56e5daba0c remove DataPartStorageBuilder 2022-10-23 00:23:15 +00:00
vdimir
f2acb53887
Better log message for replicas number in StorageReplicatedMerge 2022-09-13 13:49:51 +00:00
vdimir
6234d564d1
Fix ReplicatedMergeTreeSink, add more info to logs 2022-09-07 11:19:28 +00:00
Sachin
ade4337978
Add majority_insert_quorum setting
majority_insert_quorum is defined as (number_of_replicas/2)+1. Insert
will be successful only if majority of quorum have applied it. If
insert_quorum and majority_insert_quorum both are specified, max of
both will be used.
2022-09-07 11:19:24 +00:00
Azat Khuzhin
09aaa2b5dd Do not obtain storage snapshot for each INSERT block (improves performance)
It depends on how much parts do you have, but for some workload with
InMemory only parts w/o merges, I got 5% increase.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-08-25 21:02:27 +02:00
alesapin
2bdedf5e0c Some changes 2022-06-23 18:21:46 +02:00
Nikolai Kochetov
2a6eb593be
Revert "Revert "Add pool to WriteBufferFromS3"" 2022-02-01 13:36:51 +03:00
alexey-milovidov
095d9bfa43
Revert "Add pool to WriteBufferFromS3" 2022-02-01 05:49:40 +03:00
Nikolai Kochetov
1c9f026178 Merge branch 'master' into add-pool-to-s3-write-buffer 2022-01-28 16:01:42 +03:00
Nikolai Kochetov
a8171269a1 Review fixes. 2022-01-26 17:55:24 +00:00
Nikolai Kochetov
ae1b669fca Even more async writes. 2022-01-21 15:55:19 +00:00
Igor Nikonov
100ee92c64 insert_deduplication_token setting for INSERT statement
The setting allows a user to provide own deduplication semantic in Replicated*MergeTree
If provided, it's used instead of data digest to generate block ID
So, for example, by providing a unique value for the setting in each INSERT statement,
user can avoid the same inserted data being deduplicated

Inserting data within the same INSERT statement are split into blocks
according to the *insert_block_size* settings
(max_insert_block_size, min_insert_block_size_rows, min_insert_block_size_bytes).
Each block with the same INSERT statement will get an ordinal number.
The ordinal number is added to insert_deduplication_token to get block dedup token
i.e. <token>_0, <token>_1, ... Deduplication is done per block
So, to guarantee deduplication for two same INSERT queries,
dedup token and number of blocks to have to be the same

Issue: #7461
2021-12-19 13:15:45 +00:00
Alexey Milovidov
fe6b7c77c7 Rename "common" to "base" 2021-10-02 10:13:14 +03:00
Nikolai Kochetov
5be05c2ef9 Rewrite PushingToViewsBlockOutputStream part 3. 2021-08-31 16:50:07 +03:00
Nikolai Kochetov
61d8f880cd Rename some files. 2021-07-26 19:48:25 +03:00