Jakub Kuklis
|
f6c5a2c63c
|
Removing file checks after changing blob filenames
|
2021-12-10 22:04:38 +01:00 |
|
Jakub Kuklis
|
3aa6a0ead3
|
Adding random postfix to blob names, needs code cleanup
|
2021-12-10 22:04:38 +01:00 |
|
Jakub Kuklis
|
639d8a2b38
|
Running Azurite docker in the integration test for Blob Storage support
|
2021-12-10 22:04:37 +01:00 |
|
Jakub Kuklis
|
5ed511f9da
|
Working skeleton for Blob Storage integration test
|
2021-12-10 22:04:37 +01:00 |
|
Jakub Kuklis
|
24bafe4740
|
Starting an integration test for Blob Storage
|
2021-12-10 22:04:37 +01:00 |
|
Jakub Kuklis
|
0c65cfed1f
|
Addressing first PR remarks
|
2021-12-10 22:04:37 +01:00 |
|
Jakub Kuklis
|
7c2e116214
|
Adding max_retries config
|
2021-12-10 22:04:37 +01:00 |
|
Jakub Kuklis
|
a33cea0525
|
Refactors after proper rebase on master
|
2021-12-10 22:04:37 +01:00 |
|
Jakub Kuklis
|
b3728dbbaf
|
Rebase on master
|
2021-12-10 22:04:37 +01:00 |
|
Jakub Kuklis
|
b8ef47006c
|
Minor refactors before PR
|
2021-12-10 22:04:37 +01:00 |
|
Ryad ZENINE
|
c4421d6866
|
Small changes following review
|
2021-12-10 22:04:37 +01:00 |
|
Jakub Kuklis
|
7f595eedbc
|
Catch Blob Storage exceptions
|
2021-12-10 22:04:37 +01:00 |
|
Jakub Kuklis
|
b3c4a92cc0
|
Code cleanup
|
2021-12-10 22:04:37 +01:00 |
|
Jakub Kuklis
|
c4691affb2
|
Adding a new test for read with offset
|
2021-12-10 22:04:37 +01:00 |
|
Jakub Kuklis
|
9a00acfb72
|
Fixing read size error plus minor improvements
|
2021-12-10 22:04:37 +01:00 |
|
Jakub Kuklis
|
fd8ebe0d9f
|
Maximum read
|
2021-12-10 22:04:37 +01:00 |
|
Jakub Kuklis
|
c9f3164cba
|
Bringing back old single part upload limit
|
2021-12-10 22:04:37 +01:00 |
|
Jakub Kuklis
|
0d7a4b10c3
|
Using unique_ptr for Blob Client in Read Buffer
|
2021-12-10 22:04:37 +01:00 |
|
Jakub Kuklis
|
12924b3659
|
Minor improvements
|
2021-12-10 22:04:37 +01:00 |
|
Jakub Kuklis
|
fa2261dee6
|
Changing offset in ReadBuffer to size_t
|
2021-12-10 22:04:37 +01:00 |
|
Jakub Kuklis
|
5e2b2b1eee
|
Better search for blobs with a given name
|
2021-12-10 22:04:37 +01:00 |
|
Jakub Kuklis
|
33247db3f4
|
Adding finalize() to WriteBuffer, always using block blob uploads, code cleanup
|
2021-12-10 22:04:37 +01:00 |
|
Jakub Kuklis
|
bf2476a076
|
Shared pointer for Azure client and fixing path reference in ReadBuffer
|
2021-12-10 22:04:37 +01:00 |
|
Jakub Kuklis
|
60dd22dd4e
|
Checking for success of file removal operation
|
2021-12-10 22:04:37 +01:00 |
|
Jakub Kuklis
|
486f267b3b
|
Further code cleanup and error codes improvement
|
2021-12-10 22:04:37 +01:00 |
|
Jakub Kuklis
|
6755c5e897
|
File deletion prototype
|
2021-12-10 22:04:37 +01:00 |
|
Jakub Kuklis
|
a0cfafa387
|
checkUniqueId implementation using ListBlobs
|
2021-12-10 22:04:37 +01:00 |
|
Jakub Kuklis
|
956348073b
|
ReadBufferFromBlobStorage operating like a proper stream, rather than a single download-read buffer
|
2021-12-10 22:04:37 +01:00 |
|
Jakub Kuklis
|
95816875f4
|
Change false to true in supportZeroCopyReplication
|
2021-12-10 22:04:37 +01:00 |
|
Jakub Kuklis
|
b41ab197e8
|
Blob Storage endpoint url verification using re2
|
2021-12-10 22:04:37 +01:00 |
|
Jakub Kuklis
|
63ec8d4244
|
Adding multipart upload to WriteBuffer, retries in ReadBuffer, additional settings for buffers
|
2021-12-10 22:04:37 +01:00 |
|
Jakub Kuklis
|
02060791e3
|
Adding settings getter and applyNewSettings
|
2021-12-10 22:04:37 +01:00 |
|
Jakub Kuklis
|
3f28f2c62b
|
Enabling local cache for DiskBlobStorage
|
2021-12-10 22:04:37 +01:00 |
|
Jakub Kuklis
|
e3daf8aa49
|
Increase default thread pool size
|
2021-12-10 22:04:37 +01:00 |
|
Jakub Kuklis
|
dbb98e3636
|
Code cleanup, leaving some debug information printing under #ifdef VERBOSE_DEBUG_MODE
|
2021-12-10 22:04:37 +01:00 |
|
Jakub Kuklis
|
8230f8039c
|
Two column MergeTree table with Blob Storage disk work! Fixed buffer in ReadBufferFromBlobStorage to be permanent (class attribute)
|
2021-12-10 22:04:37 +01:00 |
|
Jakub Kuklis
|
2b4d3665ab
|
Basic inserts and selects work for a MergeTree table with a Blob Disk, based on a single column table
|
2021-12-10 22:04:37 +01:00 |
|
Jakub Kuklis
|
4b51ddee1b
|
Initial check with write and read works for the Blob Storage Disk
|
2021-12-10 22:04:37 +01:00 |
|
Youenn Lebras
|
bf4fca0351
|
Fix compilation issues with Azure in order to follow the main git and not a patched one.
|
2021-12-10 22:04:37 +01:00 |
|
Jakub Kuklis
|
4ddc3abfe3
|
Patch for fixing warnings triggered by Azure SDK, cd to contrib/azure and use 'git apply ../azure-cmake/azure_patch
|
2021-12-10 22:04:37 +01:00 |
|
Jakub Kuklis
|
c6412f24d8
|
Add azure-sdk-for-cpp as submodule in contrib/azure, linking the main repo here instead of a fork, needs to be updated eventually according to https://clickhouse.com/docs/en/development/contrib/#adding-third-party-libraries
|
2021-12-10 22:04:37 +01:00 |
|
Jakub Kuklis
|
03f85dee41
|
Managed to read markers file in SELECT, getting error for CODEC version now
|
2021-12-10 22:04:37 +01:00 |
|
Jakub Kuklis
|
482a10e62e
|
Uploading to Blob Storage on INSERT seems to do something reasonable
|
2021-12-10 22:04:37 +01:00 |
|
Jakub Kuklis
|
c43b90c58c
|
Progressing with WriteBufferFromBlobStorage
|
2021-12-10 22:04:37 +01:00 |
|
Jakub Kuklis
|
fb4c1d6686
|
Starting insertion pipeline
|
2021-12-10 22:04:37 +01:00 |
|
Jakub Kuklis
|
6436f5b04d
|
Adding an authenticated client to DiskBlobStorage
|
2021-12-10 22:04:37 +01:00 |
|
Jakub Kuklis
|
ee1a568919
|
The following query passes without erors: CREATE TABLE ... ENGINE = MergeTree ... storage_policy = 'blob_storage', debug constants present in the code
|
2021-12-10 22:04:37 +01:00 |
|
Jakub Kuklis
|
cacdac7ba6
|
Moving implementation to the .cpp file
|
2021-12-10 22:04:37 +01:00 |
|
Jakub Kuklis
|
4f506c4fc6
|
Forgot about commiting this file
|
2021-12-10 22:04:37 +01:00 |
|
Jakub Kuklis
|
88b5feb59d
|
Registering the Blob Storage Disk
|
2021-12-10 22:04:36 +01:00 |
|