thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-11-23 16:12:01 +00:00
Code Issues Packages Projects Releases Wiki Activity
112,330 Commits 9,168 Branches 1,781 Tags 2.1 GiB
f4aa62ebc8
Commit Graph

77 Commits

Author SHA1 Message Date
Nikita Mikhaylov
fa5b2bd4a5
Added Keeper retries for backups operations (#47224) 2023-04-04 18:12:08 +02:00
Vitaly Baranov
4478c6b60e Make test test_disallow_concurrency less flaky. 2023-03-29 19:55:13 +02:00
Vitaly Baranov
481a7a76ac
Simplify backup coordination for file infos (#48095) 
* Remove obsolete code for archive suffixes.

* Simplify backup coordination, stop using it for restoring.

* Build all file infos before writing to backup. Decrease number of znodes.

* Split long values before writing to ZooKeeper.

* Use separate mutexes for unrelated activities.

* Make test test_disallow_concurrency less flaky.

* Add comments and test for backup_keeper_value_max_size.
 2023-03-29 15:19:40 +02:00
SmitaRKulkarni
17cff78393
Removed SYSTEM SYNC from test test_concurrent_backups_on_same_node. (#47944) 2023-03-24 14:56:48 +01:00
Alexander Tokmakov
cd7d1fb990
Revert "Revert "Revert "Backup_Restore_concurrency_check_node""" 2023-03-24 04:35:50 +03:00
SmitaRKulkarni
04822a63e1
Merge pull request #47586 from ClickHouse/revert-47581-revert-47216-Backup_Restore_concurrency_check_node 
Revert "Revert "Backup_Restore_concurrency_check_node""
 2023-03-23 10:02:00 +01:00
robot-clickhouse
2d2183f54f Automatic style fix 2023-03-21 18:54:56 +00:00
Smita Kulkarni
9c822c70c5 Updated retry_count and sleep_time for checking BACKUP_CREATED query as it might take longer to create backup due to increase in data size 2023-03-21 19:34:19 +01:00
robot-clickhouse
826dd397b1 Automatic style fix 2023-03-20 09:58:42 +00:00
Smita Kulkarni
722404ca64 Disallow concurrency for backup restore test, added another insert block to data and setting mutations_sync 2023-03-20 10:34:55 +01:00
Aleksei Filatov
886b530963 [rev: 1] Fix review remarks 2023-03-17 13:56:05 +03:00
Aleksei Filatov
cf6ca10c83 Tests for replicated system.functions 2023-03-17 13:56:05 +03:00
Smita Kulkarni
8cd36d2a36 Removed generated file cluster_for_concurrency_test.xml 2023-03-14 20:30:44 +01:00
Smita Kulkarni
e863e49a49 Renamed cluster configuration xml to cluster_for_disallow_concurrency_test.xml for disallow_concurreny of backup and restore test 2023-03-14 20:26:19 +01:00
SmitaRKulkarni
8db4964ebc
Revert "Revert "Backup_Restore_concurrency_check_node"" 2023-03-14 20:23:43 +01:00
Alexander Tokmakov
773cd5a686
Revert "Backup_Restore_concurrency_check_node" 2023-03-14 18:55:08 +03:00
Smita Kulkarni
f76b97cb51 Set num of nodes to 4 for backup restore disallow concurrency test 2023-03-13 10:42:43 +01:00
robot-clickhouse
cebe323bef Automatic style fix 2023-03-03 15:56:11 +00:00
Smita Kulkarni
d2dbd5f293 Updated to use tryGet instead of get for checking stage of backups/restores in concurrency check and updated tests by increasing data size to have a longer backup/restore to ensure the overlap and increased timeout correspondingly. 2023-03-03 16:48:14 +01:00
robot-clickhouse
5f7220e498 Automatic style fix 2023-02-26 18:15:55 +00:00
Smita Kulkarni
4c5d39725c Updated node to check BACKUP_CREATED to node1 in test_concurrent_backups_on_different_nodes 2023-02-26 19:08:26 +01:00
Smita Kulkarni
10c92b6cb0 Increased ddl timeout during DROP of table and updated node for concurrent backups on different node 2023-02-26 19:06:31 +01:00
Azat Khuzhin
fb35a6852e More flakiness fiexes for test_backup_restore_on_cluster/test_disallow_concurrency 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-02-18 11:13:32 +01:00
Azat Khuzhin
1a352b9021 Fix flakiness of test_backup_restore_on_cluster/test_disallow_concurrency 
The problem is that if you will run first
test_concurrent_restores_on_different_node and after
test_concurrent_restores_on_same_node, like in [1], the first one will
leave the RESTORE and the second will fail.

  [1]: https://s3.amazonaws.com/clickhouse-test-reports/45282/5af2967f65c3293b277872a002cb5d570200c008/integration_tests__asan__[3/6].html

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-02-17 10:21:52 +01:00
Vitaly Baranov
45d2d678ab
Merge pull request #45800 from vitlibar/rename-new-columns-in-system-backups 
Rename new columns in system.backups
 2023-02-03 14:00:16 +01:00
Vitaly Baranov
96b140cc95 Rename columns in system.backups: num_files, num_processed_files, processed_files_size 
num_processed_files -> num_files (BACKUP) / files_read (RESTORE)
processed_files_size -> total_size (BACKUP) / bytes_read (RESTORE)
 2023-01-31 22:45:41 +01:00
Smita Kulkarni
383af7f3a6 Moved settings inside backups section - Updated backup/restore status when concurrent backups & restores are not allowed 2023-01-23 08:59:44 +01:00
Smita Kulkarni
310ae62d90 Updated backup/restore status when concurrent backups & restores are not allowed 
Implementation:
* Moved concurrent backup/restore check inside try-catch block which sets the status so that other nodes in cluster are aware of failures.
* Renamed backup_uuid to restore_uuid in RestoreSettings.
Testing:
* Updated test test_backup_and_restore_on_cluster/test_disallow_concurrency to check for specific backup/restore id.
 2023-01-22 19:01:09 +01:00
SmitaRKulkarni
db03dd1bb9
Merge branch 'master' into 43891_Disallow_concurrent_backups_and_restores 2023-01-19 09:32:50 +01:00
robot-clickhouse
237bb15a9f Automatic style fix 2023-01-17 21:34:16 +00:00
Smita Kulkarni
6e06af1b25 Updated strategy for handling internal backups & restores to avoid concurrent internal backups & restores - Added settings to disallow concurrent backups and restores 2023-01-17 22:27:13 +01:00
Vitaly Baranov
8c5c423d67 Add test. 2023-01-17 13:11:24 +01:00
robot-clickhouse
46b21629ed Automatic style fix 2023-01-09 17:23:02 +00:00
Smita Kulkarni
93530e8d34 Added settings to disallow concurrent backups and restores 
Implementation:
* Added server level settings to disallow concurrent backups and restores, which are read and set when BackupWorker is created in Context.
* Settings are set to true by default.
* Before starting backup or restores, added a check to see if any other backups/restores are running (except internal ones).
Testing:
* Added a test test_backup_and_restore_on_cluster/test_disallow_concurrency.
 2023-01-09 18:14:39 +01:00
Vitaly Baranov
b211dff013 Increase the create table timeout in test "test_backup_restore_on_cluster". 2022-12-28 00:17:19 +01:00
Vitaly Baranov
5c1f490b3a Implement referential dependencies for table engine "Distributed" 
and for functions cluster() and clusterAllReplicas().
 2022-12-13 11:09:42 +01:00
Alexey Milovidov
61278c81e4 Merge branch 'master' into remove-useless-line 2022-09-10 05:07:10 +02:00
Vitaly Baranov
10629a66e5 Fix black. 2022-09-08 20:58:51 +02:00
Vitaly Baranov
e48cd2b6f6 Add more test cases. 2022-09-08 18:14:25 +02:00
Vitaly Baranov
122009a2bd Use table lock if database is ordinary and zero-copy-replication is enabled. 2022-09-08 13:54:59 +02:00
Vitaly Baranov
9c847ceec9 No hardlinks while making backup of MergeTree in atomic database. 2022-09-07 11:44:50 +02:00
Alexey Milovidov
b42d26acfe Remove one line from XML, because we do not care 2022-08-28 02:44:02 +02:00
Vitaly Baranov
dc25f18f13 Fix flaky integration test test_async_backups_to_same_destination. 2022-08-03 14:22:09 +02:00
Vitaly Baranov
51a2bf33e8 Rename backup statuses to CREATING_BACKUP, BACKUP_CREATED, BACKUP_FAILED, RESTORING, RESTORED, RESTORE_FAILED. 2022-07-27 09:04:10 +02:00
Vitaly Baranov
fc16a15ecf Rename column "uuid" -> "id" in system.backups and allow user to set it in a query. 2022-07-27 09:04:10 +02:00
Vitaly Baranov
131019ba49 Rename column "backup_name" -> "name" in system.backups. 2022-07-27 09:04:10 +02:00
Vitaly Baranov
4c98a7bc0f Add test for Ordinary database. 2022-07-26 08:46:51 +02:00
Vitaly Baranov
6174fe1d72 Fix tests. 2022-07-22 18:33:46 +02:00
Vitaly Baranov
36d3923622 Add one more concurrency test. 2022-07-21 20:31:43 +02:00
Vitaly Baranov
708d0eb34c Add concurrent tests for backups. 2022-07-21 12:02:18 +02:00