taiyang-li
b6132d490f
merge master and solve conflict
2022-02-08 15:24:59 +08:00
Nikolai Kochetov
ea044fc6b2
Merge branch 'master' into revert-34212-revert-33291-add-pool-to-s3-write-buffer
2022-02-02 19:40:41 +03:00
Sergei Trifonov
68bc456830
Merge pull request #34223 from azat/bump-fmt
...
Bump fmtlib from 7.0.0 to 8.1.1
2022-02-02 00:03:25 +03:00
Azat Khuzhin
5be76bc969
Use proper fmt-like logging
...
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-02-01 14:30:04 +03:00
Nikolai Kochetov
2a6eb593be
Revert "Revert "Add pool to WriteBufferFromS3""
2022-02-01 13:36:51 +03:00
Igor Nikonov
f4c0b64420
Clean up: insert_deduplication_token setting for INSERT statement
...
+ reduce number of allocations on replication merge tree path
+ bash test: move insert block settings into variable
Issue: ClickHouse#7461
2022-02-01 09:22:33 +01:00
alexey-milovidov
99392b5ca7
Merge pull request #13544 from amosbird/mdha
...
Multi-Disk auto-recovery.
2022-02-01 06:13:26 +03:00
alexey-milovidov
095d9bfa43
Revert "Add pool to WriteBufferFromS3"
2022-02-01 05:49:40 +03:00
Amos Bird
ec7d367814
DiskLocal checker
...
Add DiskLocal checker so that ReplicatedMergeTree can recover data when some of its disks are broken.
2022-02-01 05:55:27 +08:00
李扬
6d50d36405
Merge branch 'master' into rocksdb_metacache
2022-01-28 22:00:31 -06:00
Nikolai Kochetov
1c9f026178
Merge branch 'master' into add-pool-to-s3-write-buffer
2022-01-28 16:01:42 +03:00
alexey-milovidov
f6684dbc62
Merge pull request #32304 from devcrafter/deduplication_token_7461
...
insert_deduplication_token setting for INSERT statement
2022-01-28 13:03:55 +03:00
taiyang-li
73def8b483
merge master and solve conflict
2022-01-24 11:01:43 +08:00
Nikolai Kochetov
f74cf1e152
Merge branch 'master' into add-pool-to-s3-write-buffer
2022-01-21 21:55:42 +03:00
Nikolai Kochetov
b3cbb63487
Merge branch 'master' into add-pool-to-s3-write-buffer
2022-01-21 21:41:54 +03:00
Anton Popov
c2206c0b96
better storing serialization infos
2022-01-21 03:29:30 +03:00
Amos Bird
62441f0a0f
Fix mutation when table contains projections ( #33679 )
2022-01-19 15:27:11 +03:00
Nikolai Kochetov
843983ea06
Add batch removing to IDiskl.
2022-01-12 15:58:04 +00:00
Nikolai Kochetov
28f2012b06
Add some more async writing.
2022-01-11 19:02:48 +00:00
Azat Khuzhin
cb70544dfe
Move LockMemoryExceptionInThread and MemoryTrackerBlockerInThread
2022-01-10 22:39:10 +03:00
Azat Khuzhin
aee034a597
Use explicit template instantiation for SystemLog
...
- Move some code into module part to avoid dependency from IStorage in SystemLog
- Remove extra headers from SystemLog.h
- Rewrite some code that was relying on headers that was included by SystemLog.h
v2: rebase
v3: squash move into module part with explicit template instantiation
(to make each commit self compilable after rebase)
2022-01-10 22:01:41 +03:00
taiyang-li
2cae8d552c
remove useless code
2022-01-10 19:17:12 +08:00
taiyang-li
1b4b727d79
fix conflict
2022-01-10 10:43:13 +08:00
Igor Nikonov
102b9cdcfb
Merge remote-tracking branch 'origin/master' into deduplication_token_7461
2022-01-09 19:15:33 +01:00
Igor Nikonov
0857a8d061
insert_deduplication_token support in non-replicated MergeTree
2022-01-09 19:06:09 +01:00
taiyang-li
cf413f16a8
remove function checkPartMetadataCache
2022-01-07 18:37:08 +08:00
mergify[bot]
6fcd5a7d41
Merge branch 'master' into MDB-15474
2022-01-07 10:12:25 +00:00
taiyang-li
b91e0a533c
fix style
2022-01-05 20:05:22 +08:00
taiyang-li
c0a9c2b916
refactor metadatacache to reduce using of USE_ROCKSDB
2022-01-05 19:51:50 +08:00
taiyang-li
abefefc719
support multi disk
2022-01-04 18:17:23 +08:00
taiyang-li
37c7c282d7
fix unit test
2022-01-04 13:41:11 +08:00
taiyang-li
3f6d830536
Merge remote-tracking branch 'origin/master' into rocksdb_metacache
2022-01-04 10:00:34 +08:00
taiyang-li
05dfc4eb25
fix building and stateless test
2021-12-31 11:13:38 +08:00
Alexander Tokmakov
187c43eba8
rename Committed state to Active
2021-12-30 23:45:38 +03:00
taiyang-li
63dc6821d2
fix ut and some bug
2021-12-29 12:31:54 +08:00
taiyang-li
98f37afc80
fix building
2021-12-28 19:29:01 +08:00
taiyang-li
2a1fe52b9f
rename symbols && fix uts && add setting use_metadata_cache
2021-12-28 18:06:13 +08:00
taiyang-li
7dab7caa9d
wrap cache related code with USE_ROCKSDB
2021-12-28 11:57:43 +08:00
taiyang-li
30ffa57bd5
remove unused log
2021-12-28 10:17:16 +08:00
Anton Ivashkin
0465aef15d
Fixes by code review responces
2021-12-27 19:27:06 +03:00
taiyang-li
94a7a09598
Merge remote-tracking branch 'origin/master' into rocksdb_metacache
2021-12-27 10:14:24 +08:00
mergify[bot]
1e0642065b
Merge branch 'master' into deduplication_token_7461
2021-12-22 15:27:28 +00:00
Anton Ivashkin
e88b97dafb
Fix typos
2021-12-21 19:56:29 +03:00
ianton-ru
e6fd4bfb50
Merge branch 'master' into MDB-15474
2021-12-21 17:38:36 +03:00
Anton Ivashkin
0c0bf66334
Merge master
2021-12-21 17:27:54 +03:00
Maksim Kita
51477adf1b
Updated additional cases
2021-12-20 15:55:07 +03:00
Igor Nikonov
100ee92c64
insert_deduplication_token setting for INSERT statement
...
The setting allows a user to provide own deduplication semantic in Replicated*MergeTree
If provided, it's used instead of data digest to generate block ID
So, for example, by providing a unique value for the setting in each INSERT statement,
user can avoid the same inserted data being deduplicated
Inserting data within the same INSERT statement are split into blocks
according to the *insert_block_size* settings
(max_insert_block_size, min_insert_block_size_rows, min_insert_block_size_bytes).
Each block with the same INSERT statement will get an ordinal number.
The ordinal number is added to insert_deduplication_token to get block dedup token
i.e. <token>_0, <token>_1, ... Deduplication is done per block
So, to guarantee deduplication for two same INSERT queries,
dedup token and number of blocks to have to be the same
Issue: #7461
2021-12-19 13:15:45 +00:00
liyang
37ba8004ff
Speep up mergetree starting up process
2021-12-18 16:39:59 +08:00
Anton Popov
16312e7e4a
Merge remote-tracking branch 'upstream/master' into HEAD
2021-12-14 18:58:17 +03:00
Nikolai Kochetov
22e6fc1685
Merge pull request #32067 from amosbird/projection-fix23
...
Fix detaching parts with projections
2021-12-13 12:00:17 +03:00
