Commit Graph

155 Commits

Author SHA1 Message Date
Vitaly Baranov
f4ac4c3f9d Corrections after review. 2023-05-17 03:23:16 +02:00
Vitaly Baranov
943707963f Add backup setting "decrypt_files_from_encrypted_disks" 2023-05-16 14:27:27 +02:00
Vitaly Baranov
5198997fd8 Remove ReadSettings from backup entries. 2023-05-16 14:27:27 +02:00
Vitaly Baranov
c92219f01b BACKUP now writes encrypted data for tables on encrypted disks. 2023-05-16 14:26:33 +02:00
Konstantin Morozov
e4394debd4 forgot about query_plan 2023-05-02 14:59:41 +02:00
Konstantin Morozov
397a1700c7 apply comments 2023-05-02 14:59:41 +02:00
kst-morozov
f0619910d4 move pipe compute into initializePipeline 2023-05-02 14:59:41 +02:00
Vladimir C
e7a5c96353
Merge pull request #48219 from ClickHouse/vdimir/lock_order_47072 2023-04-06 11:54:33 +02:00
vdimir
01ce3baf0f
Rename Context::getTemporaryVolume to getGlobalTemporaryVolume 2023-04-05 13:42:13 +00:00
Azat Khuzhin
218b1f9c29 Add ability to throttle BACKUPs on per-server/backup basis
Server settings:
- backup_read_bandwidth_for_server
- backup_write_bandwidth_for_server

Query settings:
- backup_read_bandwidth
- backup_write_bandwidth

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-04-05 09:39:48 +02:00
Konstantin Morozov
dec5972194 add optimization for MemoryStorageStep 2023-03-19 20:35:24 +01:00
Vitaly Baranov
00c05f7f8b
Merge pull request #47546 from vitlibar/use-server-side-copy-for-restore
Use server-side copy for restore from S3 to S3.
2023-03-15 11:28:29 +01:00
Vitaly Baranov
1cf1ce07fe Use server-side copy during restore from S3 to S3. 2023-03-13 23:50:13 +01:00
Konstantin Morozov
255102292c refactoring: remove uselees ctor 2023-03-09 10:02:16 +01:00
Konstantin Morozov
4686eb3cdd refactoring: create new step only once 2023-03-09 09:57:23 +01:00
Konstantin Morozov
82efbf01aa fix empty pipe for read 2023-03-09 08:28:55 +01:00
Konstantin Morozov
e0ca12dc66 fix failed CI 2023-03-03 18:16:14 +01:00
Konstantin Morozov
3cb82bfe07 Update read and with ReadFromMemoryStorage 2023-03-03 06:41:48 +01:00
Alexander Gololobov
863548114d force_wait flag is no longer needed 2023-01-30 18:38:28 +01:00
Alexander Gololobov
2c64f0042a Added force_wait parameter to IStorage::mutate() 2022-12-30 16:23:03 +01:00
Anton Popov
2ae3cfa9e0
Merge branch 'master' into dynamic-columns-14 2022-10-31 16:15:19 +01:00
Azat Khuzhin
4e76629aaf Fixes for -Wshorten-64-to-32
- lots of static_cast
- add safe_cast
- types adjustments
  - config
  - IStorage::read/watch
  - ...
- some TODO's (to convert types in future)

P.S. That was quite a journey...

v2: fixes after rebase
v3: fix conflicts after #42308 merged
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-10-21 13:25:19 +02:00
Anton Popov
d1fa2148de
Merge branch 'master' into dynamic-columns-14 2022-09-09 19:32:07 +02:00
Anton Popov
f0a404e2c8 Merge remote-tracking branch 'upstream/master' into HEAD 2022-09-06 15:51:16 +00:00
Anton Popov
f76c1482bd try to fix filling of missed Nested columns with multiple levels 2022-09-06 13:56:32 +00:00
Anton Popov
9dda9658a8 Merge remote-tracking branch 'upstream/master' into HEAD 2022-09-02 12:48:27 +00:00
Vitaly Baranov
7f84cf3968 Fix style. 2022-07-06 16:36:59 +02:00
Vitaly Baranov
f9204315b5 Store columns.txt in backups for the Memory table engine too. 2022-07-05 19:03:20 +02:00
Vitaly Baranov
1dba3d9f9c Make temporary files in better places. 2022-07-04 11:02:58 +02:00
Vitaly Baranov
92e0ee0b6f More detailed error messages. 2022-07-03 14:20:19 +02:00
Vitaly Baranov
031ca28fdc Add test for partition clause. More checks for data compatibility on restore. 2022-06-30 08:37:18 +02:00
Vitaly Baranov
8a7c970ce0 Fix style. 2022-06-19 15:58:26 +02:00
Anton Popov
5b01f0f341 Merge remote-tracking branch 'upstream/master' into HEAD 2022-06-15 23:01:22 +00:00
Vitaly Baranov
1198e86295 Fix storing temporary tables and skipping system tables while making a backup. 2022-06-15 20:32:34 +02:00
Vitaly Baranov
21f3bed435 Simplify path calculations in backup. 2022-06-15 20:32:34 +02:00
Vitaly Baranov
592f568f83 Move backup/restore code to storages and databases - part 2. 2022-06-15 20:32:31 +02:00
Vitaly Baranov
724bc4dc57 Move backup/restore code to storages and databases - part 1. 2022-06-15 20:28:43 +02:00
Vitaly Baranov
73b1894a21 Rework collecting replicated parts. 2022-06-15 20:28:42 +02:00
Nikolai Kochetov
1b85f2c1d6 Merge branch 'master' into refactor-read-metrics-and-callbacks 2022-05-25 16:27:40 +02:00
Nikolai Kochetov
3d84aae0ab Better. 2022-05-24 20:06:08 +00:00
Nikolai Kochetov
56feef01e7 Move some resources 2022-05-20 19:49:31 +00:00
Vitaly Baranov
dfa1053b9f Use query scopes for async backup/restore. 2022-05-13 10:35:02 +02:00
Anton Popov
0dbbca5b3e fix filling of missed Nested columns with multiple levels 2022-05-12 12:04:26 +00:00
Anton Popov
515f68eead Merge remote-tracking branch 'upstream/master' into dynamic-columns-14 2022-05-06 16:10:51 +00:00
Anton Popov
566c08086a support Object type inside other types 2022-05-06 14:44:00 +00:00
Robert Schulze
330212e0f4
Remove inherited create() method + disallow copying
The original motivation for this commit was that shared_ptr_helper used
std::shared_ptr<>() which does two heap allocations instead of
make_shared<>() which does a single allocation. Turned out that
1. the affected code (--> Storages/) is not on a hot path (rendering the
performance argument moot ...)
2. yet copying Storage objects is potentially dangerous and was
   previously allowed.

Hence, this change

- removes shared_ptr_helper and as a result all inherited create() methods,

- instead, Storage objects are now created using make_shared<>() by the
  caller (for that to work, many constructors had to be made public), and

- all Storage classes were marked as noncopyable using boost::noncopyable.

In sum, we are (likely) not making things faster but the code becomes
cleaner and harder to misuse.
2022-05-02 08:46:52 +02:00
Vitaly Baranov
68a020ecea Implement BACKUP/RESTORE ON CLUSTER. 2022-04-25 16:34:33 +02:00
Vitaly Baranov
3966ee1e30 Use SeekableReadBuffer instead of ReadBuffer in IBackupEntry. 2022-04-25 16:34:33 +02:00
Alexander Tokmakov
da00beaf7f Merge branch 'master' into mvcc_prototype 2022-04-05 11:14:42 +02:00
Anton Popov
42acb1dc29 fix inserts to columns of type Object in partitioned tables 2022-03-31 13:30:01 +00:00