Azat Khuzhin
09aaa2b5dd
Do not obtain storage snapshot for each INSERT block (improves performance)
...
It depends on how much parts do you have, but for some workload with
InMemory only parts w/o merges, I got 5% increase.
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-08-25 21:02:27 +02:00
alesapin
b2db49dbf1
Merge branch 'master' into better_data_part_storage_builder
2022-07-02 15:39:44 +02:00
alesapin
0d7298f3a9
Fix typos
2022-07-01 15:26:27 +02:00
alesapin
57284e6a9d
Fix possible deadlocks with MergeTreeData::Transaction
2022-07-01 15:16:32 +02:00
alesapin
eb5046ab26
Simplify everything
2022-06-30 22:51:27 +02:00
alesapin
cb90aca2ef
Some changes in merge tree
2022-06-30 14:12:45 +02:00
alesapin
a68bf8fff2
Missed changes
2022-06-28 12:59:54 +02:00
alesapin
0a3fab1cb6
Some sad changes
2022-06-28 12:51:49 +02:00
alesapin
e2a779b315
Merge with new changes
2022-06-25 00:29:45 +02:00
alesapin
8ccc47c202
Revert some changes
2022-06-24 23:54:10 +02:00
alesapin
011d58d7a0
Simplify more
2022-06-24 17:19:59 +02:00
alesapin
af1a9d18ab
Remove transaction argument
2022-06-24 13:34:00 +02:00
alesapin
9910395823
Simplify method signature
2022-06-24 13:19:29 +02:00
alesapin
2bdedf5e0c
Some changes
2022-06-23 18:21:46 +02:00
Nikolai Kochetov
89c5855d20
Merge branch 'master' into refactor-something-in-part-volumes
2022-06-02 12:19:07 +02:00
Anton Popov
16e839ac71
add profile events for introspection of part types
2022-05-25 14:54:49 +00:00
Anton Popov
566c08086a
support Object type inside other types
2022-05-06 14:44:00 +00:00
Nikolai Kochetov
fc3645a483
Fix some tests.
2022-05-03 19:32:24 +00:00
Nikolai Kochetov
35095191eb
Merge branch 'master' into refactor-something-in-part-volumes
2022-05-03 17:51:47 +02:00
Nikolai Kochetov
e44af67fee
Merge branch 'master' into refactor-something-in-part-volumes
2022-04-26 21:08:00 +02:00
mergify[bot]
a5aab53b70
Merge branch 'master' into add_query_level_settingss
2022-04-25 21:41:36 +00:00
Nikolai Kochetov
8c00692844
Part 8
2022-04-22 16:58:09 +00:00
alesapin
cbfe00bb47
Fix tidy
2022-04-21 00:30:13 +02:00
Memo
b3adf150b5
add_query_level_settings
2022-04-18 12:15:41 +08:00
alesapin
eb7593f786
Remove more trash
2022-04-15 16:24:38 +02:00
Alexander Tokmakov
da00beaf7f
Merge branch 'master' into mvcc_prototype
2022-04-05 11:14:42 +02:00
Anton Popov
42acb1dc29
fix inserts to columns of type Object in partitioned tables
2022-03-31 13:30:01 +00:00
Alexander Tokmakov
d04dc03fa4
Merge branch 'master' into mvcc_prototype
2022-03-17 15:24:32 +01:00
Alexander Tokmakov
c2ac8d4a5c
review fixes
2022-03-16 21:05:34 +01:00
Azat Khuzhin
3a5a39a9df
Do not delay final part writing by default
...
For async s3 writes final part flushing was defered until all the INSERT
block was processed, however in case of too many partitions/columns you
may exceed max_memory_usage limit (since each stream has overhead).
Introduce max_insert_delayed_streams_for_parallel_writes (with default
to 1000 for S3, 0 otherwise), to avoid this.
This should "Memory limit exceeded" errors in performance tests.
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-03-08 22:17:36 +03:00
Alexander Tokmakov
e37ef4560c
fix
2022-02-15 18:00:45 +03:00
Alexander Tokmakov
45be75b4db
Merge branch 'master' into mvcc_prototype
2022-02-06 23:36:08 +03:00
Alexander Tokmakov
ca5f951558
Merge branch 'master' into mvcc_prototype
2022-02-03 18:56:44 +03:00
Nikolai Kochetov
ea044fc6b2
Merge branch 'master' into revert-34212-revert-33291-add-pool-to-s3-write-buffer
2022-02-02 19:40:41 +03:00
Nikolai Kochetov
2a6eb593be
Revert "Revert "Add pool to WriteBufferFromS3""
2022-02-01 13:36:51 +03:00
alexey-milovidov
06477c2a7e
Update ReplicatedMergeTreeSink.cpp
2022-02-01 09:22:40 +01:00
Igor Nikonov
f4c0b64420
Clean up: insert_deduplication_token setting for INSERT statement
...
+ reduce number of allocations on replication merge tree path
+ bash test: move insert block settings into variable
Issue: ClickHouse#7461
2022-02-01 09:22:33 +01:00
alexey-milovidov
095d9bfa43
Revert "Add pool to WriteBufferFromS3"
2022-02-01 05:49:40 +03:00
Alexander Tokmakov
fb9b2d5326
Merge branch 'master' into mvcc_prototype
2022-01-28 21:18:36 +03:00
Nikolai Kochetov
1c9f026178
Merge branch 'master' into add-pool-to-s3-write-buffer
2022-01-28 16:01:42 +03:00
Nikolai Kochetov
a8171269a1
Review fixes.
2022-01-26 17:55:24 +00:00
Nikolai Kochetov
506ee8c024
Refactor some code.
2022-01-24 15:55:29 +00:00
Nikolai Kochetov
4afa75e80b
Fix deduplication.
2022-01-24 14:43:36 +00:00
Nikolai Kochetov
a024ddd1d4
Merge branch 'add-pool-to-s3-write-buffer' of github.com:ClickHouse/ClickHouse into add-pool-to-s3-write-buffer
2022-01-21 18:42:24 +00:00
Nikolai Kochetov
b3cbb63487
Merge branch 'master' into add-pool-to-s3-write-buffer
2022-01-21 21:41:54 +03:00
Nikolai Kochetov
ae1b669fca
Even more async writes.
2022-01-21 15:55:19 +00:00
Nikolai Kochetov
6d49a62666
Some more async writes.
2022-01-14 19:53:55 +00:00
Alexander Tokmakov
bf2b6c2c37
Merge branch 'master' into mvcc_prototype
2022-01-10 19:53:50 +03:00
Igor Nikonov
102b9cdcfb
Merge remote-tracking branch 'origin/master' into deduplication_token_7461
2022-01-09 19:15:33 +01:00
mergify[bot]
6fcd5a7d41
Merge branch 'master' into MDB-15474
2022-01-07 10:12:25 +00:00
Alexander Tokmakov
187c43eba8
rename Committed state to Active
2021-12-30 23:45:38 +03:00
Anton Ivashkin
0465aef15d
Fixes by code review responces
2021-12-27 19:27:06 +03:00
Anton Ivashkin
0c0bf66334
Merge master
2021-12-21 17:27:54 +03:00
Igor Nikonov
100ee92c64
insert_deduplication_token setting for INSERT statement
...
The setting allows a user to provide own deduplication semantic in Replicated*MergeTree
If provided, it's used instead of data digest to generate block ID
So, for example, by providing a unique value for the setting in each INSERT statement,
user can avoid the same inserted data being deduplicated
Inserting data within the same INSERT statement are split into blocks
according to the *insert_block_size* settings
(max_insert_block_size, min_insert_block_size_rows, min_insert_block_size_bytes).
Each block with the same INSERT statement will get an ordinal number.
The ordinal number is added to insert_deduplication_token to get block dedup token
i.e. <token>_0, <token>_1, ... Deduplication is done per block
So, to guarantee deduplication for two same INSERT queries,
dedup token and number of blocks to have to be the same
Issue: #7461
2021-12-19 13:15:45 +00:00
Alexander Tokmakov
7fcb79ae72
Merge branch 'master' into mvcc_prototype
2021-12-07 14:39:29 +03:00
tavplubix
4606376734
Update ReplicatedMergeTreeSink.cpp
2021-12-02 21:51:23 +03:00
Alexander Tokmakov
252192bc6d
fix active replicas count in quorum inserts
2021-12-02 21:02:11 +03:00
Anton Ivashkin
80ab73c691
Fix Zero-Copy replication lost locks, fix remove used remote data in DROP DETACHED PART
2021-12-01 16:11:26 +03:00
Alexander Tokmakov
92eec74ad7
Merge branch 'master' into mvcc_prototype
2021-11-06 21:08:36 +03:00
Nikolai Kochetov
ab28c6c855
Remove BlockInputStream interfaces.
2021-10-14 13:25:43 +03:00
Nikolai Kochetov
66a76ab70f
Rewrite PushingToViewsBlockOutputStream part 6
2021-09-03 20:29:36 +03:00
Alexander Tokmakov
c74bfbf991
Merge branch 'master' into mvcc_prototype
2021-07-28 22:21:48 +03:00
Nikolai Kochetov
61d8f880cd
Rename some files.
2021-07-26 19:48:25 +03:00
