Commit Graph

115 Commits

Author SHA1 Message Date
alesapin
2559d62d74 Better logging and bugfixes 2022-04-22 15:35:25 +02:00
alesapin
5465415751 Fix replace/move partition with zero copy replication 2022-04-21 14:39:12 +02:00
alesapin
612a81ba24
Update src/Disks/IDisk.h
Co-authored-by: tavplubix <tavplubix@clickhouse.com>
2022-04-20 13:58:31 +02:00
alesapin
2eee79dc2a Review fixes 2022-04-20 13:26:20 +02:00
alesapin
cc06bc3d99 Add some clarifications 2022-04-19 14:01:30 +02:00
alesapin
bd7b3847c1 Some code 2022-04-19 01:09:09 +02:00
kssenii
d6ab6de30a Minor changes 2022-03-24 15:32:08 +01:00
kssenii
f3e1ca44a9 Add system.remote_data_paths table 2022-03-23 13:01:18 +01:00
kssenii
af3bb3b7af Add WriteSettings 2022-03-18 12:27:38 +01:00
Nikolai Kochetov
ee9c2ec735
Merge pull request #34780 from azat/mt-delayed-part-flush
Do not delay final part writing by default (fixes possible Memory limit exceeded during INSERT)
2022-03-17 12:30:51 +01:00
Maksim Kita
538f8cbaad Fix clang-tidy warnings in Disks, Formats, Functions folders 2022-03-14 18:17:35 +00:00
Azat Khuzhin
4200b56f3a Introduce IDisk::supportParallelWrite()
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-03-08 22:17:36 +03:00
alesapin
447cd56cb9 Fix comments 2022-02-15 12:11:50 +03:00
alesapin
e15396d90c Fix race condition: 2022-02-14 22:19:49 +03:00
alesapin
3af6012cb4 Revert "Revert "Revert "Revert "Merge pull request #34219 from ClickHouse/revert-34212-revert-33291-add-pool-to-s3-write-buffer""""
This reverts commit 2bc2ea485e.
2022-02-08 11:01:26 +03:00
alesapin
2bc2ea485e Revert "Revert "Revert "Merge pull request #34219 from ClickHouse/revert-34212-revert-33291-add-pool-to-s3-write-buffer"""
This reverts commit fb77d7a7d5.
2022-02-08 10:56:29 +03:00
alesapin
fb77d7a7d5 Revert "Revert "Merge pull request #34219 from ClickHouse/revert-34212-revert-33291-add-pool-to-s3-write-buffer""
This reverts commit 875e5413ad.
2022-02-07 12:36:54 +03:00
alesapin
875e5413ad Revert "Merge pull request #34219 from ClickHouse/revert-34212-revert-33291-add-pool-to-s3-write-buffer"
This reverts commit b92efed350, reversing
changes made to ecce006cb2.
2022-02-04 14:30:33 +03:00
alesapin
2ed45b2a98 Merge branch 'master' into revert-34211-revert-34153-add_func_tests_over_s3 2022-02-04 11:23:46 +03:00
alesapin
b9c118524f Fix race condition on hardlink/erase/read metadata 2022-02-02 19:40:21 +03:00
Nikolai Kochetov
2a6eb593be
Revert "Revert "Add pool to WriteBufferFromS3"" 2022-02-01 13:36:51 +03:00
alexey-milovidov
99392b5ca7
Merge pull request #13544 from amosbird/mdha
Multi-Disk auto-recovery.
2022-02-01 06:13:26 +03:00
alexey-milovidov
095d9bfa43
Revert "Add pool to WriteBufferFromS3" 2022-02-01 05:49:40 +03:00
Amos Bird
ec7d367814
DiskLocal checker
Add DiskLocal checker so that ReplicatedMergeTree can recover data when some of its disks are broken.
2022-02-01 05:55:27 +08:00
Nikolai Kochetov
efaed154d2
Update src/Disks/IDisk.h
Co-authored-by: alesapin <alesapin@clickhouse.com>
2022-01-26 14:58:38 +03:00
Nikolai Kochetov
4afa75e80b Fix deduplication. 2022-01-24 14:43:36 +00:00
Nikolai Kochetov
b3cbb63487 Merge branch 'master' into add-pool-to-s3-write-buffer 2022-01-21 21:41:54 +03:00
Nikolai Kochetov
843983ea06 Add batch removing to IDiskl. 2022-01-12 15:58:04 +00:00
mergify[bot]
6fcd5a7d41
Merge branch 'master' into MDB-15474 2022-01-07 10:12:25 +00:00
Azat Khuzhin
71a99ab27c Accept real file size in createReadBufferFromFileBase()
Right now streams relies on correct file size not the number of bytes
that will be read from the stream, to overcome one bug in the linux
kernel that may return EIINVAL for pread() with offset pass the EOF.

v2: Swap read_hint and file_size (since it is easy to miss something)

Before the first argument to readFile()/createReadBufferFromFileBase()
was read_hint, not the file_size, and let's preserve the order, since
it is easy to miss something

This will also fix 02051_read_settings test automatically because now
MergeTreeReaderStream will pass estimated_sum_mark_range_bytes to
read_hint not file_size, previously it cause on of the following errors:
- Attempt to read after EOF w/ O_DIRECT
- and LOGICAL_ERROR while adjusting granulas w/o O_DIRECT

This will also improve zero-length reads guard (via
ReadBufferFromEmptyFile), that had been added in #30190

v3: fix for other storages that wasn't enabled in fast-test
v4: ignore ENOENT/ENOTSUP in readFile
2022-01-04 10:53:17 +03:00
Anton Ivashkin
2d87f0a0e3 Fix debug build 2021-12-28 11:45:38 +03:00
Anton Ivashkin
f0b9a4327a Use table UUID in zero-copy shared label in ZooKeeper 2021-12-20 16:19:00 +03:00
Anton Ivashkin
0e685c1aba Fix types 2021-12-01 19:11:17 +03:00
Anton Ivashkin
0f9038ebed Zero-copy: move shared mark outside table node in ZooKeeper 2021-11-29 19:05:31 +03:00
Azat Khuzhin
baf14444e6 Cleanup ProfileEvents and CurrentMetrics 2021-11-10 21:15:27 +03:00
Azat Khuzhin
2b96433daf Replace estimated_size with optional<size> for IDisk::readFile()
v2: rebase to fix conflicts in DiskEncrypted.cpp
2021-10-24 18:58:11 +03:00
Alexey Milovidov
fe6b7c77c7 Rename "common" to "base" 2021-10-02 10:13:14 +03:00
Alexey Milovidov
ca6f6306f3 Merge branch 'master' into async-reads 2021-08-31 02:52:58 +03:00
Alexey Milovidov
183891eb0a Add IDisk::isRemote method 2021-08-25 02:27:20 +03:00
Alexey Milovidov
7c1d0a3baf Progress on development 2021-08-25 01:24:47 +03:00
kssenii
5307d31924 Throw for alter and silence for drop 2021-08-23 23:48:53 +00:00
Alexey Milovidov
d184b79bba Progress on async reads. 2021-08-16 03:00:32 +03:00
kssenii
b25dfdd6af Use method which was added for s3 instead 2021-08-02 08:15:21 +00:00
Kseniia Sumarokova
322870acd2
Update IDisk.h 2021-08-02 09:36:44 +03:00
kssenii
f1e96dbc55 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into taiyang-li-disk-reload 2021-07-29 07:53:16 +00:00
Vitaly Baranov
958494b98e
Merge pull request #26377 from vitlibar/encrypted-disk-comments-and-improvements
Add comments for encrypted disks; some improvements in code.
2021-07-16 21:24:51 +03:00
Vitaly Baranov
9cc258327e Add comments for encrypted disks; some improvements in code. 2021-07-16 12:04:25 +03:00
Zhichang Yu
fe5d17680e Merge remote-tracking branch 'origin/master' into zero_copy_hdfs 2021-07-14 02:12:28 +00:00
Vitaly Baranov
e086e5e384
Merge branch 'master' into alexelex-encryption-1 2021-07-13 11:28:41 +03:00
Zhichang Yu
5047c758f4 fix per review 2021-07-13 07:20:20 +00:00