Vitaly Baranov
8060407c70
Add backup setting "check_parts" and restore setting "restore_broken_parts_as_detached".
2023-09-04 18:54:03 +02:00
Azat Khuzhin
d0397acafc
Add ability to override credentials for accessing base backup in S3
...
Sometimes credentials with which the backup had been done are inactive
already, and ClickHouse will not be able to read the metadata file to
continue and fail.
Add a setting to allow ignoring credential from base_backup -
`use_same_s3_credentials_for_base_backup` (default to true).
And the same for RESTORE.
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-08-30 20:16:22 +02:00
Vitaly Baranov
556ecf5c1c
Renamed backup setting "read_from_filesystem_cache_if_exists_otherwise_bypass_cache" -> "read_from_filesystem_cache".
2023-08-03 13:52:35 +02:00
Vitaly Baranov
f427c779eb
Disable using fs cache for backup/restore.
2023-08-03 13:52:32 +02:00
Roman Vasin
2cf79ac592
Merge branch 'master' of github.com:ClickHouse/ClickHouse into ADQM-984
2023-07-26 09:58:26 +00:00
Roman Vasin
a3a576cd86
Add usage of in s3_storage_class in SETTINGS
2023-07-14 17:35:31 +00:00
Azat Khuzhin
559d328178
Rename BACKUP setting native_copy to allow_s3_native_copy
...
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-07-09 08:20:03 +02:00
Azat Khuzhin
00cf66ab7a
Add ability to disable native copy for BACKUP/RESTORE
...
Native copy uses CopyObject for S3/GCS, but in GCS the CopyObject is
buggy - it does not always works.
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-07-09 08:19:10 +02:00
Vitaly Baranov
943707963f
Add backup setting "decrypt_files_from_encrypted_disks"
2023-05-16 14:27:27 +02:00
Smita Kulkarni
0506d9289c
Updated Backup/Restore Coordination construction and removed coordination_path and added uuid in settings - Use cluster state data to check concurrent backup/restore
2023-02-16 09:30:27 +01:00
Azat Khuzhin
c9c590071d
Add ability to disable deduplication for BACKUP
...
Right now BACKUP omit similar files, and will not allow to use this
backup as a regular table, and usually those similar files are quite
small (i.e. columns.txt).
So by using `BACKUP TO S3() deduplicate_files=0` you will be possible to
use `ATTACH TABLE` directly from S3.
P.S. right now it is possible only for the table with one part, since,
usually, there is nothing to deduplicate (if the columns are different).
v2: Add deduplicate_files into metadata
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-12-27 15:56:06 +01:00
Vitaly Baranov
fc16a15ecf
Rename column "uuid" -> "id" in system.backups and allow user to set it in a query.
2022-07-27 09:04:10 +02:00
Vitaly Baranov
2f47be5da7
Check that the destination for a backup is not in use.
2022-07-15 13:34:58 +02:00
Vitaly Baranov
2c92fe21a9
Implement restoring to a bigger or smaller cluster.
...
Remove backup settings allow_storing_multiple_replicas: now it's always allowed.
2022-05-12 14:55:06 +02:00
Vitaly Baranov
1b2eb4fe27
Use more clear syntax for BACKUP/RESTORE.
2022-05-08 10:37:02 +02:00
Vitaly Baranov
409edfd3fa
Rework RestoreCoordination: make restore deterministic.
2022-05-03 11:01:44 +02:00
Vitaly Baranov
2c754f44fc
Make calculation of shard_num & replica_num not dependant on match of the cluster's definitions on nodes.
2022-05-03 11:01:44 +02:00
Vitaly Baranov
2a645bb187
Fix sending 'create_table' and 'create_database' restore settings to cluster.
2022-05-03 11:01:44 +02:00
Vitaly Baranov
a8e924caf6
Make BACKUP & RESTORE synchronous by default.
2022-04-26 18:45:39 +02:00
Vitaly Baranov
78bcb96098
Rename backup & restore setting 'async' -> 'sync', and make backup & restore async by default.
2022-04-26 09:51:19 +02:00
Vitaly Baranov
000b184691
Fix style & compilation.
2022-04-25 23:05:35 +02:00
Vitaly Baranov
68a020ecea
Implement BACKUP/RESTORE ON CLUSTER.
2022-04-25 16:34:33 +02:00
Vitaly Baranov
144d3aefeb
Add system table system.backups; implement async mode for commands BACKUP & RESTORE
2022-04-25 16:34:33 +02:00
Vitaly Baranov
c884cf901f
Use BackupEntryFromAppendOnlyFile for Log engine,
...
and use a BackupEntriesBatch for Memort engine to improve performance.
A lot of minor corrections.
2022-03-20 20:02:15 +01:00
Vitaly Baranov
5f195d6ead
Implement writing backups as zip archive.
2022-03-20 20:02:15 +01:00
Vitaly Baranov
6849f51c3b
Add setting "structure_only."
2022-03-20 20:02:15 +01:00
Vitaly Baranov
7f89b98308
Rework BackupSettings and RestoreSettings a little, pass StorageRestoreSettings to storages.
2022-03-20 20:02:15 +01:00
Vitaly Baranov
cf77c0b3fc
New syntax for BACKUP/RESTORE: set backup engine explicitly.
2021-11-09 23:16:40 +03:00
Vitaly Baranov
63e8bc1f20
Added new commands BACKUP and RESTORE.
2021-08-23 23:41:45 +03:00
