Vitaly Baranov
4478c6b60e
Make test test_disallow_concurrency less flaky.
2023-03-29 19:55:13 +02:00
Vitaly Baranov
481a7a76ac
Simplify backup coordination for file infos ( #48095 )
...
* Remove obsolete code for archive suffixes.
* Simplify backup coordination, stop using it for restoring.
* Build all file infos before writing to backup. Decrease number of znodes.
* Split long values before writing to ZooKeeper.
* Use separate mutexes for unrelated activities.
* Make test test_disallow_concurrency less flaky.
* Add comments and test for backup_keeper_value_max_size.
2023-03-29 15:19:40 +02:00
SmitaRKulkarni
17cff78393
Removed SYSTEM SYNC from test test_concurrent_backups_on_same_node. ( #47944 )
2023-03-24 14:56:48 +01:00
Alexander Tokmakov
cd7d1fb990
Revert "Revert "Revert "Backup_Restore_concurrency_check_node"""
2023-03-24 04:35:50 +03:00
SmitaRKulkarni
04822a63e1
Merge pull request #47586 from ClickHouse/revert-47581-revert-47216-Backup_Restore_concurrency_check_node
...
Revert "Revert "Backup_Restore_concurrency_check_node""
2023-03-23 10:02:00 +01:00
robot-clickhouse
2d2183f54f
Automatic style fix
2023-03-21 18:54:56 +00:00
Smita Kulkarni
9c822c70c5
Updated retry_count and sleep_time for checking BACKUP_CREATED query as it might take longer to create backup due to increase in data size
2023-03-21 19:34:19 +01:00
robot-clickhouse
826dd397b1
Automatic style fix
2023-03-20 09:58:42 +00:00
Smita Kulkarni
722404ca64
Disallow concurrency for backup restore test, added another insert block to data and setting mutations_sync
2023-03-20 10:34:55 +01:00
Aleksei Filatov
886b530963
[rev: 1] Fix review remarks
2023-03-17 13:56:05 +03:00
Aleksei Filatov
cf6ca10c83
Tests for replicated system.functions
2023-03-17 13:56:05 +03:00
Smita Kulkarni
8cd36d2a36
Removed generated file cluster_for_concurrency_test.xml
2023-03-14 20:30:44 +01:00
Smita Kulkarni
e863e49a49
Renamed cluster configuration xml to cluster_for_disallow_concurrency_test.xml for disallow_concurreny of backup and restore test
2023-03-14 20:26:19 +01:00
SmitaRKulkarni
8db4964ebc
Revert "Revert "Backup_Restore_concurrency_check_node""
2023-03-14 20:23:43 +01:00
Alexander Tokmakov
773cd5a686
Revert "Backup_Restore_concurrency_check_node"
2023-03-14 18:55:08 +03:00
Smita Kulkarni
f76b97cb51
Set num of nodes to 4 for backup restore disallow concurrency test
2023-03-13 10:42:43 +01:00
robot-clickhouse
cebe323bef
Automatic style fix
2023-03-03 15:56:11 +00:00
Smita Kulkarni
d2dbd5f293
Updated to use tryGet instead of get for checking stage of backups/restores in concurrency check and updated tests by increasing data size to have a longer backup/restore to ensure the overlap and increased timeout correspondingly.
2023-03-03 16:48:14 +01:00
robot-clickhouse
5f7220e498
Automatic style fix
2023-02-26 18:15:55 +00:00
Smita Kulkarni
4c5d39725c
Updated node to check BACKUP_CREATED to node1 in test_concurrent_backups_on_different_nodes
2023-02-26 19:08:26 +01:00
Smita Kulkarni
10c92b6cb0
Increased ddl timeout during DROP of table and updated node for concurrent backups on different node
2023-02-26 19:06:31 +01:00
Azat Khuzhin
fb35a6852e
More flakiness fiexes for test_backup_restore_on_cluster/test_disallow_concurrency
...
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-02-18 11:13:32 +01:00
Azat Khuzhin
1a352b9021
Fix flakiness of test_backup_restore_on_cluster/test_disallow_concurrency
...
The problem is that if you will run first
test_concurrent_restores_on_different_node and after
test_concurrent_restores_on_same_node, like in [1], the first one will
leave the RESTORE and the second will fail.
[1]: https://s3.amazonaws.com/clickhouse-test-reports/45282/5af2967f65c3293b277872a002cb5d570200c008/integration_tests__asan__[3/6].html
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-02-17 10:21:52 +01:00
Vitaly Baranov
45d2d678ab
Merge pull request #45800 from vitlibar/rename-new-columns-in-system-backups
...
Rename new columns in system.backups
2023-02-03 14:00:16 +01:00
Vitaly Baranov
96b140cc95
Rename columns in system.backups: num_files, num_processed_files, processed_files_size
...
num_processed_files -> num_files (BACKUP) / files_read (RESTORE)
processed_files_size -> total_size (BACKUP) / bytes_read (RESTORE)
2023-01-31 22:45:41 +01:00
Smita Kulkarni
383af7f3a6
Moved settings inside backups section - Updated backup/restore status when concurrent backups & restores are not allowed
2023-01-23 08:59:44 +01:00
Smita Kulkarni
310ae62d90
Updated backup/restore status when concurrent backups & restores are not allowed
...
Implementation:
* Moved concurrent backup/restore check inside try-catch block which sets the status so that other nodes in cluster are aware of failures.
* Renamed backup_uuid to restore_uuid in RestoreSettings.
Testing:
* Updated test test_backup_and_restore_on_cluster/test_disallow_concurrency to check for specific backup/restore id.
2023-01-22 19:01:09 +01:00
SmitaRKulkarni
db03dd1bb9
Merge branch 'master' into 43891_Disallow_concurrent_backups_and_restores
2023-01-19 09:32:50 +01:00
robot-clickhouse
237bb15a9f
Automatic style fix
2023-01-17 21:34:16 +00:00
Smita Kulkarni
6e06af1b25
Updated strategy for handling internal backups & restores to avoid concurrent internal backups & restores - Added settings to disallow concurrent backups and restores
2023-01-17 22:27:13 +01:00
Vitaly Baranov
8c5c423d67
Add test.
2023-01-17 13:11:24 +01:00
robot-clickhouse
46b21629ed
Automatic style fix
2023-01-09 17:23:02 +00:00
Smita Kulkarni
93530e8d34
Added settings to disallow concurrent backups and restores
...
Implementation:
* Added server level settings to disallow concurrent backups and restores, which are read and set when BackupWorker is created in Context.
* Settings are set to true by default.
* Before starting backup or restores, added a check to see if any other backups/restores are running (except internal ones).
Testing:
* Added a test test_backup_and_restore_on_cluster/test_disallow_concurrency.
2023-01-09 18:14:39 +01:00
Vitaly Baranov
b211dff013
Increase the create table timeout in test "test_backup_restore_on_cluster".
2022-12-28 00:17:19 +01:00
Vitaly Baranov
5c1f490b3a
Implement referential dependencies for table engine "Distributed"
...
and for functions cluster() and clusterAllReplicas().
2022-12-13 11:09:42 +01:00
Alexey Milovidov
61278c81e4
Merge branch 'master' into remove-useless-line
2022-09-10 05:07:10 +02:00
Vitaly Baranov
10629a66e5
Fix black.
2022-09-08 20:58:51 +02:00
Vitaly Baranov
e48cd2b6f6
Add more test cases.
2022-09-08 18:14:25 +02:00
Vitaly Baranov
122009a2bd
Use table lock if database is ordinary and zero-copy-replication is enabled.
2022-09-08 13:54:59 +02:00
Vitaly Baranov
9c847ceec9
No hardlinks while making backup of MergeTree in atomic database.
2022-09-07 11:44:50 +02:00
Alexey Milovidov
b42d26acfe
Remove one line from XML, because we do not care
2022-08-28 02:44:02 +02:00
Vitaly Baranov
dc25f18f13
Fix flaky integration test test_async_backups_to_same_destination.
2022-08-03 14:22:09 +02:00
Vitaly Baranov
51a2bf33e8
Rename backup statuses to CREATING_BACKUP, BACKUP_CREATED, BACKUP_FAILED, RESTORING, RESTORED, RESTORE_FAILED.
2022-07-27 09:04:10 +02:00
Vitaly Baranov
fc16a15ecf
Rename column "uuid" -> "id" in system.backups and allow user to set it in a query.
2022-07-27 09:04:10 +02:00
Vitaly Baranov
131019ba49
Rename column "backup_name" -> "name" in system.backups.
2022-07-27 09:04:10 +02:00
Vitaly Baranov
4c98a7bc0f
Add test for Ordinary database.
2022-07-26 08:46:51 +02:00
Vitaly Baranov
6174fe1d72
Fix tests.
2022-07-22 18:33:46 +02:00
Vitaly Baranov
36d3923622
Add one more concurrency test.
2022-07-21 20:31:43 +02:00
Vitaly Baranov
708d0eb34c
Add concurrent tests for backups.
2022-07-21 12:02:18 +02:00
Vitaly Baranov
619e22fe57
Add tests for shutdown during backup.
2022-07-21 12:01:16 +02:00
