Alexander Tokmakov
e67d05604c
Merge branch 'master' into fix_intersecting_parts2
2022-11-02 19:30:27 +01:00
Igor Nikonov
9a315b7899
Merge remote-tracking branch 'origin/master' into igor/insert_zk_retries_retry
2022-11-02 15:32:54 +00:00
Alexander Tokmakov
00c9e50ee3
fix race between INSERT and ALTER PARTITION
2022-11-02 16:25:26 +01:00
Igor Nikonov
e76c3c381f
Better handling tables in read-only mode during insert
...
+ throw immediately if table has no metadata in zk
+ stop retries in case of shutdown
+ check if table is readonly at the begining of every retry
2022-11-01 18:47:51 +00:00
Igor Nikonov
2d5050ea66
Fix review comments
2022-10-31 18:20:58 +00:00
Anton Popov
2ae3cfa9e0
Merge branch 'master' into dynamic-columns-14
2022-10-31 16:15:19 +01:00
Igor Nikonov
d09deaf10f
Fix style check
2022-10-28 15:17:05 +00:00
Igor Nikonov
c53b96a11c
Add more comments
2022-10-28 15:09:01 +00:00
Igor Nikonov
3682913172
Merge remote-tracking branch 'origin/master' into igor/insert_zk_retries_retry
2022-10-27 22:01:53 +00:00
Igor Nikonov
f773436de5
Fix flaky test
2022-10-27 13:34:07 +00:00
Igor Nikonov
649e8d302c
Sync with retiring branch igor/insert_zk_retries
2022-10-26 17:41:44 +00:00
Igor Nikonov
54013009d3
Code migration from #41734 to align diff and git blame.
...
Initial implementation was different and it changed the entire
ReplicatedMergeTreeSink::commitPart() which change history provided by git blame.
Then RetriesControl.retryLoop() was introduced later which significantly reduces
the diff since it's like while() used before.
So, check outing the current version will keep more original history in
git blame, which is useful here
2022-10-24 08:20:11 +00:00
Anton Popov
b40d9200d2
better semantic of constsness of DataPartStorage
2022-10-23 15:24:20 +00:00
Anton Popov
56e5daba0c
remove DataPartStorageBuilder
2022-10-23 00:23:15 +00:00
Antonio Andelic
77eb353839
Merge pull request #42243 from ClickHouse/use-multiread-more
...
Use MultiRead where possible
2022-10-20 09:04:54 +02:00
Anton Popov
22da0420af
fix possible restart errors after failed quorum insert
2022-10-19 20:26:16 +00:00
Nikolai Kochetov
b18c6fd8e6
Fix an invalid type of a column after attach and alter.
2022-10-14 14:52:26 +00:00
Antonio Andelic
5e6a7c1678
Use multiread where possible
2022-10-11 09:27:46 +00:00
Anton Popov
5cb6357549
fix style check
2022-10-03 19:37:27 +00:00
Anton Popov
6e61cf92f5
Merge remote-tracking branch 'upstream/master' into HEAD
2022-10-03 13:16:57 +00:00
Anton Popov
3623807527
Merge pull request #41290 from CurtizJ/dynamic-columns-20
...
Fix `ALTER ADD COLUMN` with columns of type `Object`
2022-09-14 14:15:29 +02:00
Anton Popov
ee54451741
fix adding of columns of type Object
2022-09-13 22:43:59 +00:00
vdimir
f2acb53887
Better log message for replicas number in StorageReplicatedMerge
2022-09-13 13:49:51 +00:00
Anton Popov
ba41239ecd
Merge remote-tracking branch 'upstream/master' into HEAD
2022-09-08 15:20:29 +00:00
vdimir
6234d564d1
Fix ReplicatedMergeTreeSink, add more info to logs
2022-09-07 11:19:28 +00:00
Sachin
ade4337978
Add majority_insert_quorum setting
...
majority_insert_quorum is defined as (number_of_replicas/2)+1. Insert
will be successful only if majority of quorum have applied it. If
insert_quorum and majority_insert_quorum both are specified, max of
both will be used.
2022-09-07 11:19:24 +00:00
Anton Popov
f0a404e2c8
Merge remote-tracking branch 'upstream/master' into HEAD
2022-09-06 15:51:16 +00:00
Azat Khuzhin
09aaa2b5dd
Do not obtain storage snapshot for each INSERT block (improves performance)
...
It depends on how much parts do you have, but for some workload with
InMemory only parts w/o merges, I got 5% increase.
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-08-25 21:02:27 +02:00
alesapin
b2db49dbf1
Merge branch 'master' into better_data_part_storage_builder
2022-07-02 15:39:44 +02:00
alesapin
0d7298f3a9
Fix typos
2022-07-01 15:26:27 +02:00
alesapin
57284e6a9d
Fix possible deadlocks with MergeTreeData::Transaction
2022-07-01 15:16:32 +02:00
alesapin
eb5046ab26
Simplify everything
2022-06-30 22:51:27 +02:00
alesapin
cb90aca2ef
Some changes in merge tree
2022-06-30 14:12:45 +02:00
alesapin
a68bf8fff2
Missed changes
2022-06-28 12:59:54 +02:00
alesapin
0a3fab1cb6
Some sad changes
2022-06-28 12:51:49 +02:00
alesapin
e2a779b315
Merge with new changes
2022-06-25 00:29:45 +02:00
alesapin
8ccc47c202
Revert some changes
2022-06-24 23:54:10 +02:00
alesapin
011d58d7a0
Simplify more
2022-06-24 17:19:59 +02:00
alesapin
af1a9d18ab
Remove transaction argument
2022-06-24 13:34:00 +02:00
alesapin
9910395823
Simplify method signature
2022-06-24 13:19:29 +02:00
alesapin
2bdedf5e0c
Some changes
2022-06-23 18:21:46 +02:00
Nikolai Kochetov
89c5855d20
Merge branch 'master' into refactor-something-in-part-volumes
2022-06-02 12:19:07 +02:00
Anton Popov
16e839ac71
add profile events for introspection of part types
2022-05-25 14:54:49 +00:00
Anton Popov
566c08086a
support Object type inside other types
2022-05-06 14:44:00 +00:00
Nikolai Kochetov
fc3645a483
Fix some tests.
2022-05-03 19:32:24 +00:00
Nikolai Kochetov
35095191eb
Merge branch 'master' into refactor-something-in-part-volumes
2022-05-03 17:51:47 +02:00
Nikolai Kochetov
e44af67fee
Merge branch 'master' into refactor-something-in-part-volumes
2022-04-26 21:08:00 +02:00
mergify[bot]
a5aab53b70
Merge branch 'master' into add_query_level_settingss
2022-04-25 21:41:36 +00:00
Nikolai Kochetov
8c00692844
Part 8
2022-04-22 16:58:09 +00:00
alesapin
cbfe00bb47
Fix tidy
2022-04-21 00:30:13 +02:00
Memo
b3adf150b5
add_query_level_settings
2022-04-18 12:15:41 +08:00
alesapin
eb7593f786
Remove more trash
2022-04-15 16:24:38 +02:00
Alexander Tokmakov
da00beaf7f
Merge branch 'master' into mvcc_prototype
2022-04-05 11:14:42 +02:00
Anton Popov
42acb1dc29
fix inserts to columns of type Object in partitioned tables
2022-03-31 13:30:01 +00:00
Alexander Tokmakov
d04dc03fa4
Merge branch 'master' into mvcc_prototype
2022-03-17 15:24:32 +01:00
Alexander Tokmakov
c2ac8d4a5c
review fixes
2022-03-16 21:05:34 +01:00
Azat Khuzhin
3a5a39a9df
Do not delay final part writing by default
...
For async s3 writes final part flushing was defered until all the INSERT
block was processed, however in case of too many partitions/columns you
may exceed max_memory_usage limit (since each stream has overhead).
Introduce max_insert_delayed_streams_for_parallel_writes (with default
to 1000 for S3, 0 otherwise), to avoid this.
This should "Memory limit exceeded" errors in performance tests.
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-03-08 22:17:36 +03:00
Alexander Tokmakov
e37ef4560c
fix
2022-02-15 18:00:45 +03:00
Alexander Tokmakov
45be75b4db
Merge branch 'master' into mvcc_prototype
2022-02-06 23:36:08 +03:00
Alexander Tokmakov
ca5f951558
Merge branch 'master' into mvcc_prototype
2022-02-03 18:56:44 +03:00
Nikolai Kochetov
ea044fc6b2
Merge branch 'master' into revert-34212-revert-33291-add-pool-to-s3-write-buffer
2022-02-02 19:40:41 +03:00
Nikolai Kochetov
2a6eb593be
Revert "Revert "Add pool to WriteBufferFromS3""
2022-02-01 13:36:51 +03:00
alexey-milovidov
06477c2a7e
Update ReplicatedMergeTreeSink.cpp
2022-02-01 09:22:40 +01:00
Igor Nikonov
f4c0b64420
Clean up: insert_deduplication_token setting for INSERT statement
...
+ reduce number of allocations on replication merge tree path
+ bash test: move insert block settings into variable
Issue: ClickHouse#7461
2022-02-01 09:22:33 +01:00
alexey-milovidov
095d9bfa43
Revert "Add pool to WriteBufferFromS3"
2022-02-01 05:49:40 +03:00
Alexander Tokmakov
fb9b2d5326
Merge branch 'master' into mvcc_prototype
2022-01-28 21:18:36 +03:00
Nikolai Kochetov
1c9f026178
Merge branch 'master' into add-pool-to-s3-write-buffer
2022-01-28 16:01:42 +03:00
Nikolai Kochetov
a8171269a1
Review fixes.
2022-01-26 17:55:24 +00:00
Nikolai Kochetov
506ee8c024
Refactor some code.
2022-01-24 15:55:29 +00:00
Nikolai Kochetov
4afa75e80b
Fix deduplication.
2022-01-24 14:43:36 +00:00
Nikolai Kochetov
a024ddd1d4
Merge branch 'add-pool-to-s3-write-buffer' of github.com:ClickHouse/ClickHouse into add-pool-to-s3-write-buffer
2022-01-21 18:42:24 +00:00
Nikolai Kochetov
b3cbb63487
Merge branch 'master' into add-pool-to-s3-write-buffer
2022-01-21 21:41:54 +03:00
Nikolai Kochetov
ae1b669fca
Even more async writes.
2022-01-21 15:55:19 +00:00
Nikolai Kochetov
6d49a62666
Some more async writes.
2022-01-14 19:53:55 +00:00
Alexander Tokmakov
bf2b6c2c37
Merge branch 'master' into mvcc_prototype
2022-01-10 19:53:50 +03:00
Igor Nikonov
102b9cdcfb
Merge remote-tracking branch 'origin/master' into deduplication_token_7461
2022-01-09 19:15:33 +01:00
mergify[bot]
6fcd5a7d41
Merge branch 'master' into MDB-15474
2022-01-07 10:12:25 +00:00
Alexander Tokmakov
187c43eba8
rename Committed state to Active
2021-12-30 23:45:38 +03:00
Anton Ivashkin
0465aef15d
Fixes by code review responces
2021-12-27 19:27:06 +03:00
Anton Ivashkin
0c0bf66334
Merge master
2021-12-21 17:27:54 +03:00
Igor Nikonov
100ee92c64
insert_deduplication_token setting for INSERT statement
...
The setting allows a user to provide own deduplication semantic in Replicated*MergeTree
If provided, it's used instead of data digest to generate block ID
So, for example, by providing a unique value for the setting in each INSERT statement,
user can avoid the same inserted data being deduplicated
Inserting data within the same INSERT statement are split into blocks
according to the *insert_block_size* settings
(max_insert_block_size, min_insert_block_size_rows, min_insert_block_size_bytes).
Each block with the same INSERT statement will get an ordinal number.
The ordinal number is added to insert_deduplication_token to get block dedup token
i.e. <token>_0, <token>_1, ... Deduplication is done per block
So, to guarantee deduplication for two same INSERT queries,
dedup token and number of blocks to have to be the same
Issue: #7461
2021-12-19 13:15:45 +00:00
Alexander Tokmakov
7fcb79ae72
Merge branch 'master' into mvcc_prototype
2021-12-07 14:39:29 +03:00
tavplubix
4606376734
Update ReplicatedMergeTreeSink.cpp
2021-12-02 21:51:23 +03:00
Alexander Tokmakov
252192bc6d
fix active replicas count in quorum inserts
2021-12-02 21:02:11 +03:00
Anton Ivashkin
80ab73c691
Fix Zero-Copy replication lost locks, fix remove used remote data in DROP DETACHED PART
2021-12-01 16:11:26 +03:00
Alexander Tokmakov
92eec74ad7
Merge branch 'master' into mvcc_prototype
2021-11-06 21:08:36 +03:00
Nikolai Kochetov
ab28c6c855
Remove BlockInputStream interfaces.
2021-10-14 13:25:43 +03:00
Nikolai Kochetov
66a76ab70f
Rewrite PushingToViewsBlockOutputStream part 6
2021-09-03 20:29:36 +03:00
Alexander Tokmakov
c74bfbf991
Merge branch 'master' into mvcc_prototype
2021-07-28 22:21:48 +03:00
Nikolai Kochetov
61d8f880cd
Rename some files.
2021-07-26 19:48:25 +03:00
