ClickHouse/src/Backups
Arthur Passos 2bade7db08
Add global proxy setting (#51749)
* initial impl

* fix env ut

* move ut directory

* make sure no null proxy resolver is returned by ProxyConfigurationResolverProvider

* minor adjustment

* add a few tests, still incomplete

* add proxy support for url table function

* use proxy for select from url as well

* remove optional from return type, just returns empty config

* fix style

* style

* black

* ohg boy

* rm in progress file

* god pls don't let me kill anyone

* ...

* add use_aws guards

* remove hard coded s3 proxy resolver

* add concurrency-mt-unsafe

* aa

* black

* add logging back

* revert change

* imrpove code a bit

* helper functions and separate tests

* for some reason, this env test is not working..

* formatting

* :)

* clangtidy

* lint

* revert some stupid things

* small test adjusmtments

* simplify tests

* rename test

* remove extra line

* freaking style change

* simplify a bit

* fix segfault & remove an extra call

* tightly couple proxy provider with context..

* remove useless include

* rename config prefix parameter

* simplify provider a bit

* organize provider a bit

* add a few comments

* comment out proxy env tests

* fix nullptr in unit tests

* make sure old storage proxy config is properly covered without global context instance

* move a few functions from class to anonymous namespace

* fix no fallback for specific storage conf

* change API to accept http method instead of bool

* implement http/https distinction in listresolver, any still not implemented

* implement http/https distinction in remote resolver

* progress on code, improve tests and add url function working test

* use protcol instead of method for http and https

* small fix

* few more adjustments

* fix style

* black

* move enum to proxyconfiguration

* wip

* fix build

* fix ut

* delete atomicroundrobin class

* remove stale include

* add some tests.. need to spend some more time on the design..

* change design a bit

* progress

* use existing context for tests

* rename aux function and fix ut

* ..

* rename test

* try to simplify tests a bit

* simplify tests a bit more

* attempt to fix tests, accept more than one remote resolver

* use proper log id

* try waiting for resolver

* proper wait logic

* black

* empty

* address a few comments

* refactor tests

* remove old tests

* baclk

* use RAII to set/unset env

* black

* clang tidy

* fix env proxy not respecting any

* use log trace

* fix wrong logic in getRemoteREsolver

* fix wrong logic in getRemoteREsolver

* fix test

* remove unwanted code

* remove ClientConfigurationperRequest and auxilary classes

* remove unwanted code

* remove adapter test

* few adjustments and add test for s3 storage conf  with new proxy settings

* black

* use chassert for context

* Add getenv comment
2023-08-24 16:07:26 +03:00
..
tests Fix tests. 2023-08-03 13:52:35 +02:00
BackupCoordinationFileInfos.cpp Use forward declaration of ThreadPool 2023-04-07 11:25:35 +02:00
BackupCoordinationFileInfos.h
BackupCoordinationLocal.cpp Removed setStageForCluster and added option all_hosts to set stage for cluster 2023-05-08 14:51:04 +02:00
BackupCoordinationLocal.h Fixed clang build 2023-05-16 14:09:38 +02:00
BackupCoordinationRemote.cpp less exceptions with runtime format string 2023-08-14 21:45:59 +02:00
BackupCoordinationRemote.h Fix multiple writing of same file to backup (#51299) 2023-06-23 12:11:18 +02:00
BackupCoordinationReplicatedAccess.cpp Added Keeper retries for backups operations (#47224) 2023-04-04 18:12:08 +02:00
BackupCoordinationReplicatedAccess.h Added Keeper retries for backups operations (#47224) 2023-04-04 18:12:08 +02:00
BackupCoordinationReplicatedSQLObjects.cpp Added Keeper retries for backups operations (#47224) 2023-04-04 18:12:08 +02:00
BackupCoordinationReplicatedSQLObjects.h Added Keeper retries for backups operations (#47224) 2023-04-04 18:12:08 +02:00
BackupCoordinationReplicatedTables.cpp Use hash_of_all_files from system.parts to check identity of parts during on-cluster backups. 2023-06-14 20:39:50 +02:00
BackupCoordinationReplicatedTables.h Added Keeper retries for backups operations (#47224) 2023-04-04 18:12:08 +02:00
BackupCoordinationStage.cpp
BackupCoordinationStage.h Updated to add error or completed status in zookeeper for a cluster for backup/restore, to avoid interpreting previously failed backup/restore when zookeeper is unable to remove nodes 2023-04-12 20:26:57 +02:00
BackupCoordinationStageSync.cpp less exceptions with runtime format string 2023-08-14 21:45:59 +02:00
BackupCoordinationStageSync.h Removed setStageForCluster and added option all_hosts to set stage for cluster 2023-05-08 14:51:04 +02:00
BackupEntriesCollector.cpp Disable using fs cache for backup/restore #2. 2023-08-03 13:52:35 +02:00
BackupEntriesCollector.h Disable using fs cache for backup/restore #2. 2023-08-03 13:52:35 +02:00
BackupEntryFromAppendOnlyFile.cpp Make async reader work with any impl 2023-05-22 19:54:04 +02:00
BackupEntryFromAppendOnlyFile.h Add backup setting "decrypt_files_from_encrypted_disks" 2023-05-16 14:27:27 +02:00
BackupEntryFromImmutableFile.cpp Disable using fs cache for backup/restore. 2023-08-03 13:52:32 +02:00
BackupEntryFromImmutableFile.h Disable using fs cache for backup/restore. 2023-08-03 13:52:32 +02:00
BackupEntryFromMemory.cpp Remove ReadSettings from backup entries. 2023-05-16 14:27:27 +02:00
BackupEntryFromMemory.h Remove ReadSettings from backup entries. 2023-05-16 14:27:27 +02:00
BackupEntryFromSmallFile.cpp Disable using fs cache for backup/restore #2. 2023-08-03 13:52:35 +02:00
BackupEntryFromSmallFile.h Disable using fs cache for backup/restore #2. 2023-08-03 13:52:35 +02:00
BackupEntryWithChecksumCalculation.cpp Disable using fs cache for backup/restore. 2023-08-03 13:52:32 +02:00
BackupEntryWithChecksumCalculation.h Disable using fs cache for backup/restore. 2023-08-03 13:52:32 +02:00
BackupEntryWrappedWith.h Disable using fs cache for backup/restore. 2023-08-03 13:52:32 +02:00
BackupFactory.cpp
BackupFactory.h Disable using fs cache for backup/restore. 2023-08-03 13:52:32 +02:00
BackupFileInfo.cpp Disable using fs cache for backup/restore. 2023-08-03 13:52:32 +02:00
BackupFileInfo.h Disable using fs cache for backup/restore. 2023-08-03 13:52:32 +02:00
BackupImpl.cpp More formats supported, read single archive from 1 thread 2023-08-09 11:58:37 +00:00
BackupImpl.h Fix checking the lock file too often while writing a backup. 2023-06-12 18:13:26 +02:00
BackupInfo.cpp
BackupInfo.h
BackupIO_Default.cpp Disable using fs cache for backup/restore. 2023-08-03 13:52:32 +02:00
BackupIO_Default.h Disable using fs cache for backup/restore. 2023-08-03 13:52:32 +02:00
BackupIO_Disk.cpp Disable using fs cache for backup/restore. 2023-08-03 13:52:32 +02:00
BackupIO_Disk.h Disable using fs cache for backup/restore. 2023-08-03 13:52:32 +02:00
BackupIO_File.cpp Disable using fs cache for backup/restore. 2023-08-03 13:52:32 +02:00
BackupIO_File.h Disable using fs cache for backup/restore. 2023-08-03 13:52:32 +02:00
BackupIO_S3.cpp Add global proxy setting (#51749) 2023-08-24 16:07:26 +03:00
BackupIO_S3.h Disable using fs cache for backup/restore. 2023-08-03 13:52:32 +02:00
BackupIO.h Fix whitespaces. 2023-05-16 14:27:27 +02:00
BackupSettings.cpp Renamed backup setting "read_from_filesystem_cache_if_exists_otherwise_bypass_cache" -> "read_from_filesystem_cache". 2023-08-03 13:52:35 +02:00
BackupSettings.h Renamed backup setting "read_from_filesystem_cache_if_exists_otherwise_bypass_cache" -> "read_from_filesystem_cache". 2023-08-03 13:52:35 +02:00
BackupStatus.cpp
BackupStatus.h
BackupsWorker.cpp Better 2023-08-21 18:20:20 +00:00
BackupsWorker.h Disable using fs cache for backup/restore. 2023-08-03 13:52:32 +02:00
BackupUtils.cpp
BackupUtils.h Use forward declaration of ThreadPool 2023-04-07 11:25:35 +02:00
CMakeLists.txt
DDLAdjustingForBackupVisitor.cpp
DDLAdjustingForBackupVisitor.h
IBackup.h Move the common part of BackupIO_* to BackupIO_Default. 2023-05-16 14:27:23 +02:00
IBackupCoordination.h Fixed clang build 2023-05-16 14:09:38 +02:00
IBackupEntriesLazyBatch.cpp Disable using fs cache for backup/restore. 2023-08-03 13:52:32 +02:00
IBackupEntriesLazyBatch.h
IBackupEntry.h Disable using fs cache for backup/restore. 2023-08-03 13:52:32 +02:00
IRestoreCoordination.h Fixed clang build 2023-05-16 14:09:38 +02:00
registerBackupEngineS3.cpp Disable using fs cache for backup/restore. 2023-08-03 13:52:32 +02:00
registerBackupEnginesFileAndDisk.cpp Disable using fs cache for backup/restore. 2023-08-03 13:52:32 +02:00
RestoreCoordinationLocal.cpp Removed setStageForCluster and added option all_hosts to set stage for cluster 2023-05-08 14:51:04 +02:00
RestoreCoordinationLocal.h Fixed clang build 2023-05-16 14:09:38 +02:00
RestoreCoordinationRemote.cpp less exceptions with runtime format string 2023-08-14 21:45:59 +02:00
RestoreCoordinationRemote.h Fixed clang build 2023-05-16 14:09:38 +02:00
RestorerFromBackup.cpp Refine RestorerFromBackup::applyCustomStoragePolicy infinitesimally 2023-08-02 11:57:38 +03:00
RestorerFromBackup.h Introduce clean-up of the source storage policy by setting it to an empty string 2023-07-27 18:17:47 +03:00
RestoreSettings.cpp Introduce clean-up of the source storage policy by setting it to an empty string 2023-07-27 18:17:47 +03:00
RestoreSettings.h Introduce clean-up of the source storage policy by setting it to an empty string 2023-07-27 18:17:47 +03:00
SettingsFieldOptionalString.cpp Introduce clean-up of the source storage policy by setting it to an empty string 2023-07-27 18:17:47 +03:00
SettingsFieldOptionalString.h Introduce clean-up of the source storage policy by setting it to an empty string 2023-07-27 18:17:47 +03:00
SettingsFieldOptionalUUID.cpp
SettingsFieldOptionalUUID.h
WithRetries.cpp Merge remote-tracking branch 'blessed/master' into zk_retry_timeout 2023-04-04 18:53:55 +02:00
WithRetries.h Added Keeper retries for backups operations (#47224) 2023-04-04 18:12:08 +02:00