8 Commits

Author	SHA1	Message	Date
Vitaly Baranov	25356786ea	Simplify the implementation, create new utility function copyS3FileToDisk().	2023-03-14 23:34:44 +01:00
Vitaly Baranov	1cf1ce07fe	Use server-side copy during restore from S3 to S3.	2023-03-13 23:50:13 +01:00
Vitaly Baranov	5ceb64accc	Use new copy s3 functions in S3ObjectStorage.	2023-01-21 15:47:58 +01:00
Vitaly Baranov	14a7ee8e26	Copy files to S3 during backup directly without using WriteBufferFromS3 to decrease memory consumption.	2023-01-17 09:35:41 +01:00
Aleksandr Musorin	5cb69d8a22	changed type name for S3_Plain storage ... renamed a disk for S3PlainObjectStorage in system.disks table from s3 to s3_plain	2022-11-04 17:35:51 +01:00
Vitaly Baranov	b9f2f17331	Add test and logging.	2022-11-01 12:23:20 +01:00
Vitaly Baranov	1365105bc4	Implement backup to S3	2022-10-19 00:04:41 +02:00
Azat Khuzhin	8830f0608d	Support BACKUP to S3 with as-is path/data structure ... Right now backup to S3 does not make a lot of sense, since: - it has random names, and to decoding them - requires metadata from local disk (/var/lib/disks/DISK/BACKUP_NAME) - or send_metadata (but it is also tricky even with it) So this patch adds simpler interface for S3, it is only suitable for BACKUP/RESTORE, so don't try to use it for MergeTree engine. It is done by adding separate disk - `s3_plain` for this, that: - does not support any extended features, like renames/hardlinks/attrs/... (so basically everything that MergeTree requires) - only write/read/unlink/list files Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>	2022-10-14 12:59:25 +02:00