Commit Graph

36 Commits

Author SHA1 Message Date
Vitaly Baranov
25356786ea Simplify the implementation, create new utility function copyS3FileToDisk(). 2023-03-14 23:34:44 +01:00
Vitaly Baranov
1cf1ce07fe Use server-side copy during restore from S3 to S3. 2023-03-13 23:50:13 +01:00
Azat Khuzhin
11be9b9ad1 Create disk directory before access check for local disk
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-11-20 16:28:35 +01:00
Azat Khuzhin
2efd29f49d Implement access (read/read-by-offset/write/delete) check for all disks
Previously we had such access (read/write/delete) checks only for S3 and
Azure disks (read/read-by-offset/write/delete), this patch adds check
for all disks.

Also I've added the disk name into IDisk interface, since it is required
for the error message in IDisk::checkAccess(), but I have to add
DiskEncrypted::encrypted_name due DiskEncrypted inherits from
DiskDecorator not from IDisk, and so does not have ability to set disk
name (DiskEncrypted could pass the disk name to the DiskDecorator, but
it is not used anywere, and besides this will require to duplicate the
name for each user of DiskDecorator).

Also from nwo on, skip_access_check will make sense for every disk, plus
now startup() called for each disk (before it was missed for some of
them).

And I've added skip_access_check as as a member for DiskRestartProxy,
since it calls startup() on restart().

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-11-20 16:28:35 +01:00
Nikolai Kochetov
57ffcb7ed0 Revert "Merge pull request #41681 from ClickHouse/revert-41268-disable-s3-parallel-write-for-part-moves-to-disk-s3"
This reverts commit 7956c2becf, reversing
changes made to 57be648984.
2022-10-20 19:51:27 +02:00
Alexander Tokmakov
021b882455 Revert "Disable parallel s3 multipart upload for part moves. (#41268)"
This reverts commit 0414d95878.
2022-09-22 13:59:25 +03:00
Nikolai Kochetov
0414d95878
Disable parallel s3 multipart upload for part moves. (#41268)
* Disable parallel s3 multipart upload for part moves.

* Add setting s3_allow_parallel_part_upload
2022-09-21 19:10:32 +02:00
alesapin
704d7fdc41 Fix copy and use disks 2022-08-21 18:18:35 +02:00
alesapin
d8664c3227 Add shortcut for backups 2022-08-19 16:58:30 +02:00
kssenii
dd9651a6ca Better 2022-07-10 22:42:16 +02:00
kssenii
bf1c99daab Refactoring of object storages, add LocalObjectStorage 2022-07-09 12:39:31 +02:00
alesapin
b557c1dd80 Interm stage 2022-06-15 13:51:21 +02:00
alesapin
267dd28e11 More consts for disks 2022-06-13 21:07:10 +02:00
alesapin
8ae74daf8a Fix bugs with abs path and create mode 2022-06-05 18:22:33 +02:00
alesapin
46bccae078 Trying to create metadata layer 2022-06-02 18:09:40 +02:00
Amos Bird
4a5e4274f0
base should not depend on Common 2022-04-29 10:26:35 +08:00
alesapin
cbced6567d Fix build 2022-04-21 17:32:23 +02:00
alesapin
5465415751 Fix replace/move partition with zero copy replication 2022-04-21 14:39:12 +02:00
alesapin
b9c118524f Fix race condition on hardlink/erase/read metadata 2022-02-02 19:40:21 +03:00
Anton Ivashkin
f0b9a4327a Use table UUID in zero-copy shared label in ZooKeeper 2021-12-20 16:19:00 +03:00
Anton Ivashkin
0f9038ebed Zero-copy: move shared mark outside table node in ZooKeeper 2021-11-29 19:05:31 +03:00
Alexey Milovidov
fe6b7c77c7 Rename "common" to "base" 2021-10-02 10:13:14 +03:00
Vitaly Baranov
9cc258327e Add comments for encrypted disks; some improvements in code. 2021-07-16 12:04:25 +03:00
kssenii
9ec92ec514 Fix tests, less manual concatination of paths 2021-05-05 18:39:30 +03:00
kssenii
c609119daf Update Disks/* 2021-04-30 18:13:19 +03:00
Anton Popov
c7070da85a better abstractions in disk interface 2021-01-26 17:49:35 +03:00
Pavel Kovalenko
69b4bc6f33 Proper error handling during insert into MergeTree with S3. 2020-10-06 12:38:00 +03:00
Pavel Kovalenko
ea46ee74ae Don't allocate Executor instance on each disk->getExecutor() call. 2020-08-11 22:08:32 +03:00
Pavel Kovalenko
bece61b1cc Fix codestyle. 2020-08-10 14:40:31 +03:00
Pavel Kovalenko
a6ee09f696 Move parts to DiskS3 in parallel refactoring. 2020-08-07 14:40:19 +03:00
Pavel Kovalenko
7856852010 Move parts to DiskS3 in parallel. 2020-08-05 19:36:10 +03:00
Alexey Milovidov
a6016b95ce Fix clang-tidy 2020-07-16 05:34:43 +03:00
Alexey Milovidov
204a4af394 Rollback insertion error in Log engines #12402 2020-07-12 05:32:18 +03:00
Alexey Milovidov
7e1813825b Return old names of macros 2020-05-24 01:24:01 +03:00
Alexey Milovidov
9d24908e53 Progress on task 2020-05-23 20:52:11 +03:00
Ivan Lezhankin
06446b4f08 dbms/ → src/ 2020-04-03 18:14:31 +03:00