thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-04 21:42:39 +00:00
Code Issues Packages Projects Releases Wiki Activity
137,686 Commits 9,185 Branches 1,782 Tags 2.3 GiB
7aace4d876
Commit Graph

35 Commits

Author SHA1 Message Date
Kseniia Sumarokova
19d8256fa8
Update test.py 2024-03-11 14:44:19 +01:00
Kseniia Sumarokova
90b27432a2
Update test.py 2024-03-11 12:18:58 +01:00
robot-clickhouse
be98c95f58 Automatic style fix 2024-03-06 09:12:26 +00:00
kssenii
6d4514c045 Fix test 2024-03-06 16:55:48 +08:00
Joshua Hildred
ca9a6f809f add support for .tar.zst and tar.xz 2024-02-29 05:20:53 -08:00
Joshua Hildred
92b446a552 Add support for tar archive compression 2024-02-29 05:20:52 -08:00
Joshua Hildred
c53021f55e Fix formating 2024-02-29 05:20:51 -08:00
Joshua Hildred
eb4ec0e871 Add support for reading and writing backups as a 
tar archive using libarchive.
 2024-02-29 05:20:51 -08:00
Alexey Milovidov
330043af30 Fix integration test_backup_restore_s3 2024-02-22 10:58:05 +01:00
Antonio Andelic
175a1db787 Support specifying users for s3 settings 2024-02-19 16:43:27 +01:00
Vitaly Baranov
e1a136b791 Explicit finalize() function in ZipArchiveWriter. 
Simplify too complicated code in ZipArchiveWriter.
 2023-12-24 00:33:59 +01:00
MikhailBurdukov
6f19e8ebd1
Merge branch 'master' into native_copy_for_s3_disks 2023-11-27 14:25:36 +03:00
MikhailBurdukov
c10c30832c Review fix 2023-11-27 10:58:30 +00:00
vdimir
15234474d7
Implement system table blob_storage_log 2023-11-21 09:18:25 +00:00
MikhailBurdukov
1aea8186c7 Enable s3 native copy for scenario: from s3 to s3. 2023-11-14 13:53:48 +00:00
Vitaly Baranov
6ac61b1fdd Fix tests. 2023-08-03 13:52:35 +02:00
Vitaly Baranov
556ecf5c1c Renamed backup setting "read_from_filesystem_cache_if_exists_otherwise_bypass_cache" -> "read_from_filesystem_cache". 2023-08-03 13:52:35 +02:00
Vitaly Baranov
1cd7902079 Add test. 2023-08-03 13:52:35 +02:00
Azat Khuzhin
29dc9abfca Fix test_backup_restore_s3 after logging for native copying changed 
Check profile events instead of some odd logs.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-07-09 08:20:03 +02:00
Azat Khuzhin
84c720b33e Beatify test_backup_restore_s3 (using per-query profile events) 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-07-09 08:20:03 +02:00
AVMusorin
7df4820af7
Fix metrics WriteBufferFromS3Bytes, WriteBufferFromS3Microseconds and WriteBufferFromS3RequestsErrors 
Ref: https://github.com/ClickHouse/ClickHouse/pull/45188
 2023-05-17 14:50:38 +02:00
Vitaly Baranov
c48c20fac8 Use combined checksums for encrypted immutable files. 2023-05-16 14:27:27 +02:00
Vitaly Baranov
101aa6eff0 Add function copyS3FileFromDisk(). 2023-05-16 14:25:37 +02:00
Vitaly Baranov
fd2731845c Simplify interface of IBackupWriter: Remove supportNativeCopy() function. 2023-05-16 14:25:36 +02:00
Vitaly Baranov
478140f8b4 Fix LimitSeekableReadBuffer. 2023-05-11 22:58:05 +02:00
Alexander Tokmakov
83b12ec5ca replace NO DELAY with SYNC in tests 2023-05-03 20:08:49 +02:00
Nikita Mikhaylov
fa5b2bd4a5
Added Keeper retries for backups operations (#47224) 2023-04-04 18:12:08 +02:00
Vitaly Baranov
25356786ea Simplify the implementation, create new utility function copyS3FileToDisk(). 2023-03-14 23:34:44 +01:00
Vitaly Baranov
1cf1ce07fe Use server-side copy during restore from S3 to S3. 2023-03-13 23:50:13 +01:00
Vitaly Baranov
5ceb64accc Use new copy s3 functions in S3ObjectStorage. 2023-01-21 15:47:58 +01:00
Vitaly Baranov
14a7ee8e26 Copy files to S3 during backup directly without using WriteBufferFromS3 to decrease memory consumption. 2023-01-17 09:35:41 +01:00
Aleksandr Musorin
5cb69d8a22 changed type name for S3_Plain storage 
renamed a disk for S3PlainObjectStorage in system.disks table from s3 to s3_plain
 2022-11-04 17:35:51 +01:00
Vitaly Baranov
b9f2f17331 Add test and logging. 2022-11-01 12:23:20 +01:00
Vitaly Baranov
1365105bc4 Implement backup to S3 2022-10-19 00:04:41 +02:00
Azat Khuzhin
8830f0608d Support BACKUP to S3 with as-is path/data structure 
Right now backup to S3 does not make a lot of sense, since:
- it has random names, and to decoding them
- requires metadata from local disk (/var/lib/disks/DISK/BACKUP_NAME)
- or send_metadata (but it is also tricky even with it)

So this patch adds simpler interface for S3, it is only suitable for
BACKUP/RESTORE, so don't try to use it for MergeTree engine.

It is done by adding separate disk - `s3_plain` for this, that:
- does not support any extended features, like renames/hardlinks/attrs/...
  (so basically everything that MergeTree requires)
- only write/read/unlink/list files

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-10-14 12:59:25 +02:00