MikhailBurdukov
e2b39db242
Fix
2024-05-13 14:14:35 +00:00
MikhailBurdukov
cfa0aa772e
Merge branch 'master' into truncate_for_obj_storage_disks
2024-05-13 16:05:44 +03:00
MikhailBurdukov
7d0214018c
Enable truncate operation for object storage disks.
2024-05-13 12:46:42 +00:00
Alexey Milovidov
dd58af7d4f
Merge branch 'master' of github.com:ClickHouse/ClickHouse into clang-18-ci
2024-05-10 07:17:39 +02:00
Alexey Milovidov
c17a3bb944
Useless changes
2024-05-10 03:31:40 +02:00
Antonio Andelic
bc9d50cf16
Merge pull request #63523 from ClickHouse/leftover-threads-copy
...
Correctly stop `asyncCopy` tasks in all cases
2024-05-09 06:50:51 +00:00
Alexey Milovidov
c198bbcafb
Useless changes
2024-05-09 04:20:54 +02:00
Alexey Milovidov
224e1940ae
Useless changes
2024-05-09 03:58:34 +02:00
Alexey Milovidov
0d14a2c67e
Useless changes
2024-05-09 03:11:02 +02:00
Alexey Milovidov
95f12ef274
Useless changes
2024-05-09 01:08:33 +02:00
Antonio Andelic
3d5c8db1e0
Use ThreadPoolCallbackRunnerLocal
2024-05-08 15:12:52 +02:00
Antonio Andelic
16eb12a321
Correctly stop copy tasks when exception thrown
2024-05-08 14:58:21 +02:00
Kseniia Sumarokova
35d96f98f4
Merge pull request #63484 from ClickHouse/fix-custom-web-disk-with-config-reload
...
Fix logical error when reloading config with customly created web disk broken after #56367
2024-05-08 09:28:54 +00:00
kssenii
836cf150b5
Fix
2024-05-07 19:00:30 +02:00
Julia Kartseva
0a1d852dfd
Enable plain_rewritable metadata for local and azure
...
Enable plain_rewritable support for local and azure (azure_blob_storage)
metadata type.
- HDFS object storage currently does not support iteration
and does not implement listObjects method. It's a blocker
for enabling plain_rewritable metadata type with HDFS.
- StaticWeb object storage is read-only and works with
its dedicated metadata type.
2024-05-07 03:53:23 +00:00
kssenii
a51786532b
Remove copy
2024-05-06 13:21:11 +02:00
Alexey Milovidov
7ac5821a23
Merge pull request #63074 from ClickHouse/local-less-directories
...
Less directories in clickhouse-local
2024-05-05 00:19:31 +02:00
alesapin
15954b5b5c
Merge pull request #63257 from jkartseva/fix-inf-loop-azure-list-objects
...
Fix infinite loop in AzureObjectStorage::listObjects
2024-05-03 09:39:00 +00:00
Julia Kartseva
dd7e88b37e
fix infinite loop in AzureObjectStorage::listObjects
...
While the exit condition is correct for `AzureIteratorAsync`,
listObject may never exit.
This is because BlobContainerClient::LstBlobs sets a page, thus
making a sequential HasPage() condition true.
This issue reproduces when passing an empty path.
Tested with the following integration test:
https://pastila.nl/?01691b54/9ceaf103c91ab9e38ea3541abfcd1ae6#hUtrrz5sFHcBSHCLS9AlVA==
(will be added when plain_rewritable is enabled for all disk types).
2024-05-03 01:56:23 +00:00
Alexey Milovidov
ed1bbd0bd0
Merge branch 'master' into local-less-directories
2024-05-03 02:35:44 +02:00
Smita Kulkarni
37c618289b
Merge branch 'master' into add_azure_workload_identity_support_blob
2024-05-02 14:58:56 +02:00
Sema Checherinda
083bcfc1ce
distinct message for s3 error 'no key' for cases disk and storage
2024-05-02 13:08:25 +02:00
Alexey Milovidov
817570c5e4
Merge branch 'master' into local-less-directories
2024-04-30 04:45:33 +02:00
Julia Kartseva
dc95558962
method rename
2024-04-29 19:48:17 +00:00
Julia Kartseva
3c1207ed4d
remove path normalization
2024-04-29 18:01:21 +00:00
Julia Kartseva
508a42bc8f
use ordered map for path map
2024-04-29 18:01:21 +00:00
Julia Kartseva
24d5abba10
extract plain_rewritable metadata type
...
Make PlainRewritableObjectStorage generic;
Support config type
```
<s3_plain_rewritable>
<type>object_storage</type>
<object_storage_type>s3</object_storage_type>
<metadata_type>plain_rewritable</metadata_type>
<endpoint>https://s3.eu-west-1.amazonaws.com/clickhouse-eu-west-1.clickhouse.com/data/ </endpoint>
<use_environment_credentials>1</use_environment_credentials>
</s3_plain_rewritable>
```
2024-04-29 18:01:21 +00:00
Julia Kartseva
d1217af389
address feedback - pt.4
2024-04-29 18:01:21 +00:00
Julia Kartseva
802ee27b1b
address feedback - pt.3
...
non-functional changes
2024-04-29 18:01:21 +00:00
Julia Kartseva
4a7f28f6bd
address feedback - pt.2
2024-04-29 18:01:21 +00:00
Julia Kartseva
36a1cae910
address feedback - pt.1
2024-04-29 18:01:21 +00:00
Julia Kartseva
4f6a3e27b7
Update src/Disks/ObjectStorages/MetadataStorageFromPlainObjectStorageOperations.cpp
...
Co-authored-by: Kseniia Sumarokova <54203879+kssenii@users.noreply.github.com>
2024-04-29 18:01:21 +00:00
Julia Kartseva
70d55aa618
Update src/Disks/ObjectStorages/CommonPathPrefixKeyGenerator.h
...
Co-authored-by: Kseniia Sumarokova <54203879+kssenii@users.noreply.github.com>
2024-04-29 18:01:21 +00:00
Julia Kartseva
89f28f3c18
explicitly disallow ALTERs and mutations for plain
2024-04-29 18:01:21 +00:00
Julia Kartseva
01ee500b06
improvements, cleanups, comments
2024-04-29 18:01:21 +00:00
Julia Kartseva
a67a8299ce
Do not list prefix.path in listDirectory
2024-04-29 18:01:21 +00:00
Julia Kartseva
d1e5a09b18
better transaction rollback
2024-04-29 18:01:21 +00:00
Julia Kartseva
b4375131cb
concurrency control
2024-04-29 18:01:21 +00:00
Julia Kartseva
7916792baa
Do not create directory metadata recursively
2024-04-29 18:01:21 +00:00
Julia Kartseva
75147f3ac7
S3-plain based disk supporting directory rename
...
Fixes: https://github.com/ClickHouse/ClickHouse/issues/58347
2024-04-29 18:01:21 +00:00
Alexey Milovidov
64fff163f7
Less directories
2024-04-27 19:54:37 +02:00
Alexey Milovidov
6a083ebed0
Even less directories
2024-04-27 19:13:17 +02:00
Smita Kulkarni
f474c29b4d
Fix build with new master
2024-04-26 15:33:27 +02:00
Smita Kulkarni
3ace11f92d
Merge branch 'master' into Azure_write_buffer_parallel_upload
2024-04-26 10:27:41 +02:00
liuneng
76b989ebbc
fix missing-field-initializers
2024-04-24 17:13:08 +08:00
Raúl Marín
3df253cb0a
Merge pull request #62850 from danipozo/fix-building-endpoint-from-storage-account-url
...
Azure: fix building final url from endpoint object
2024-04-23 17:26:22 +00:00
Dani Pozo
771219a118
Apply suggestion to use pop_back
2024-04-23 12:34:26 +02:00
Dani Pozo
c75574a1d6
Azure: fix building final url from endpoint object
2024-04-22 18:49:46 +02:00
Smita Kulkarni
065eceecbf
Merge branch 'master' into Azure_write_buffer_parallel_upload
2024-04-19 17:37:15 +02:00
Smita Kulkarni
613dd1a6c0
Updated to use workload identity for backups & disk
2024-04-18 18:26:07 +02:00
