Commit Graph

78833 Commits

Author SHA1 Message Date
Jakub Kuklis
41a88e49c9 Abstracting metadata creation for remote disks registrations, putting Blob Storage auth in a separate file 2021-12-10 22:04:38 +01:00
Jakub Kuklis
1d5c69b366 Minor tweaks to the test, disabling another slow one 2021-12-10 22:04:38 +01:00
Jakub Kuklis
312ea92989 More integration tests, all pass, at least usually 2021-12-10 22:04:38 +01:00
Jakub Kuklis
afc64c14ce Merging test works correctly 2021-12-10 22:04:38 +01:00
Jakub Kuklis
6dca10e1c5 First integration tests, the one for merging fails for the moment 2021-12-10 22:04:38 +01:00
Jakub Kuklis
e58f74d8dc Connecting to Azurite and invoking simple INSERTs works in the integration test 2021-12-10 22:04:38 +01:00
Jakub Kuklis
8afe69ff93 Comming back to StorageSharedKeyCredential for the integration test, get 400 Bad requests from Azurite when trying to create a container now 2021-12-10 22:04:38 +01:00
Jakub Kuklis
f180a7ba5f Solved Invalid storage account error with connection strings 2021-12-10 22:04:38 +01:00
Jakub Kuklis
abd685e14c Installing azure-storage-blob in the integration tests docker, raising exception when not connected to Azurite, changing ports in the Azurite docker 2021-12-10 22:04:38 +01:00
Jakub Kuklis
008141de36 Enable authentication via StorageSharedKeyCredential 2021-12-10 22:04:38 +01:00
Jakub Kuklis
d51f979c7a Adding endpoint validation 2021-12-10 22:04:38 +01:00
Jakub Kuklis
2b7725bfc7 Enable connecting directly to storage accounts and creating new containers 2021-12-10 22:04:38 +01:00
Jakub Kuklis
910db6ca57 Code cleanup addressing PR remarks 2021-12-10 22:04:38 +01:00
Jakub Kuklis
850c9a7e7a Reverting submodule changes to fast test docker and disabling Blob Storage support when files missing 2021-12-10 22:04:38 +01:00
Jakub Kuklis
37a23b402b Add missing dependencies for azure for fast test 2021-12-10 22:04:38 +01:00
Jakub Kuklis
024a3b4626 Fixing submodule update for clickhouse compilation in the fast test 2021-12-10 22:04:38 +01:00
Jakub Kuklis
b605894768 Removing spurious '/' character 2021-12-10 22:04:38 +01:00
Jakub Kuklis
6acbec4455 Rebased on master 2021-12-10 22:04:38 +01:00
Jakub Kuklis
f6c5a2c63c Removing file checks after changing blob filenames 2021-12-10 22:04:38 +01:00
Jakub Kuklis
3aa6a0ead3 Adding random postfix to blob names, needs code cleanup 2021-12-10 22:04:38 +01:00
Jakub Kuklis
639d8a2b38 Running Azurite docker in the integration test for Blob Storage support 2021-12-10 22:04:37 +01:00
Jakub Kuklis
5ed511f9da Working skeleton for Blob Storage integration test 2021-12-10 22:04:37 +01:00
Jakub Kuklis
24bafe4740 Starting an integration test for Blob Storage 2021-12-10 22:04:37 +01:00
Jakub Kuklis
0c65cfed1f Addressing first PR remarks 2021-12-10 22:04:37 +01:00
Jakub Kuklis
7c2e116214 Adding max_retries config 2021-12-10 22:04:37 +01:00
Jakub Kuklis
a33cea0525 Refactors after proper rebase on master 2021-12-10 22:04:37 +01:00
Jakub Kuklis
b3728dbbaf Rebase on master 2021-12-10 22:04:37 +01:00
Jakub Kuklis
b8ef47006c Minor refactors before PR 2021-12-10 22:04:37 +01:00
Ryad ZENINE
c4421d6866 Small changes following review 2021-12-10 22:04:37 +01:00
Jakub Kuklis
7f595eedbc Catch Blob Storage exceptions 2021-12-10 22:04:37 +01:00
Jakub Kuklis
b3c4a92cc0 Code cleanup 2021-12-10 22:04:37 +01:00
Jakub Kuklis
c4691affb2 Adding a new test for read with offset 2021-12-10 22:04:37 +01:00
Jakub Kuklis
9a00acfb72 Fixing read size error plus minor improvements 2021-12-10 22:04:37 +01:00
Jakub Kuklis
fd8ebe0d9f Maximum read 2021-12-10 22:04:37 +01:00
Jakub Kuklis
c9f3164cba Bringing back old single part upload limit 2021-12-10 22:04:37 +01:00
Jakub Kuklis
0d7a4b10c3 Using unique_ptr for Blob Client in Read Buffer 2021-12-10 22:04:37 +01:00
Jakub Kuklis
12924b3659 Minor improvements 2021-12-10 22:04:37 +01:00
Jakub Kuklis
fa2261dee6 Changing offset in ReadBuffer to size_t 2021-12-10 22:04:37 +01:00
Jakub Kuklis
5e2b2b1eee Better search for blobs with a given name 2021-12-10 22:04:37 +01:00
Jakub Kuklis
33247db3f4 Adding finalize() to WriteBuffer, always using block blob uploads, code cleanup 2021-12-10 22:04:37 +01:00
Jakub Kuklis
bf2476a076 Shared pointer for Azure client and fixing path reference in ReadBuffer 2021-12-10 22:04:37 +01:00
Jakub Kuklis
60dd22dd4e Checking for success of file removal operation 2021-12-10 22:04:37 +01:00
Jakub Kuklis
486f267b3b Further code cleanup and error codes improvement 2021-12-10 22:04:37 +01:00
Jakub Kuklis
6755c5e897 File deletion prototype 2021-12-10 22:04:37 +01:00
Jakub Kuklis
a0cfafa387 checkUniqueId implementation using ListBlobs 2021-12-10 22:04:37 +01:00
Jakub Kuklis
956348073b ReadBufferFromBlobStorage operating like a proper stream, rather than a single download-read buffer 2021-12-10 22:04:37 +01:00
Jakub Kuklis
95816875f4 Change false to true in supportZeroCopyReplication 2021-12-10 22:04:37 +01:00
Jakub Kuklis
b41ab197e8 Blob Storage endpoint url verification using re2 2021-12-10 22:04:37 +01:00
Jakub Kuklis
63ec8d4244 Adding multipart upload to WriteBuffer, retries in ReadBuffer, additional settings for buffers 2021-12-10 22:04:37 +01:00
Jakub Kuklis
02060791e3 Adding settings getter and applyNewSettings 2021-12-10 22:04:37 +01:00