thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-11-23 16:12:01 +00:00
Code Issues Packages Projects Releases Wiki Activity

fix infinite loop in AzureObjectStorage::listObjects

Browse Source 
While the exit condition is correct for `AzureIteratorAsync`,
listObject may never exit.
This is because BlobContainerClient::LstBlobs sets a page, thus
making a sequential HasPage() condition true.

This issue reproduces when passing an empty path.
Tested with the following integration test:
https://pastila.nl/?01691b54/9ceaf103c91ab9e38ea3541abfcd1ae6#hUtrrz5sFHcBSHCLS9AlVA==
(will be added when plain_rewritable is enabled for all disk types).
This commit is contained in:
Julia Kartseva 2024-05-02 05:52:11 +00:00
parent c0a6acc06a
commit dd7e88b37e
1 changed files with 2 additions and 9 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

11
src/Disks/ObjectStorages/AzureBlobStorage/AzureObjectStorage.cpp
View File

@ -164,15 +164,13 @@ void AzureObjectStorage::listObjects(const std::string & path, RelativePathsWith
else
options.PageSizeHint = settings.get()->list_object_keys_size;
Azure::Storage::Blobs::ListBlobsPagedResponse blob_list_response;
while (true)
for (auto blob_list_response = client_ptr->ListBlobs(options); blob_list_response.HasPage(); blob_list_response.MoveToNextPage())
{
ProfileEvents::increment(ProfileEvents::AzureListObjects);
ProfileEvents::increment(ProfileEvents::DiskAzureListObjects);
blob_list_response = client_ptr->ListBlobs(options);
auto blobs_list = blob_list_response.Blobs;
const auto & blobs_list = blob_list_response.Blobs;
for (const auto & blob : blobs_list)
{
@ -193,11 +191,6 @@ void AzureObjectStorage::listObjects(const std::string & path, RelativePathsWith
break;
options.PageSizeHint = keys_left;
}
if (blob_list_response.HasPage())
options.ContinuationToken = blob_list_response.NextPageToken;
else
break;
}
}