Julia Kartseva
802ee27b1b
address feedback - pt.3
...
non-functional changes
2024-04-29 18:01:21 +00:00
Julia Kartseva
4a7f28f6bd
address feedback - pt.2
2024-04-29 18:01:21 +00:00
Julia Kartseva
36a1cae910
address feedback - pt.1
2024-04-29 18:01:21 +00:00
Julia Kartseva
4f6a3e27b7
Update src/Disks/ObjectStorages/MetadataStorageFromPlainObjectStorageOperations.cpp
...
Co-authored-by: Kseniia Sumarokova <54203879+kssenii@users.noreply.github.com>
2024-04-29 18:01:21 +00:00
Julia Kartseva
70d55aa618
Update src/Disks/ObjectStorages/CommonPathPrefixKeyGenerator.h
...
Co-authored-by: Kseniia Sumarokova <54203879+kssenii@users.noreply.github.com>
2024-04-29 18:01:21 +00:00
Julia Kartseva
89f28f3c18
explicitly disallow ALTERs and mutations for plain
2024-04-29 18:01:21 +00:00
Julia Kartseva
01ee500b06
improvements, cleanups, comments
2024-04-29 18:01:21 +00:00
Julia Kartseva
a67a8299ce
Do not list prefix.path in listDirectory
2024-04-29 18:01:21 +00:00
Julia Kartseva
d1e5a09b18
better transaction rollback
2024-04-29 18:01:21 +00:00
Julia Kartseva
b4375131cb
concurrency control
2024-04-29 18:01:21 +00:00
Julia Kartseva
7916792baa
Do not create directory metadata recursively
2024-04-29 18:01:21 +00:00
Julia Kartseva
75147f3ac7
S3-plain based disk supporting directory rename
...
Fixes: https://github.com/ClickHouse/ClickHouse/issues/58347
2024-04-29 18:01:21 +00:00
kssenii
7a416f2e76
Merge remote-tracking branch 'origin/master' into use-iobject-storage-for-table-engines-1
2024-04-28 12:19:32 +02:00
Alexey Milovidov
64fff163f7
Less directories
2024-04-27 19:54:37 +02:00
Alexey Milovidov
6a083ebed0
Even less directories
2024-04-27 19:13:17 +02:00
kssenii
91014456b5
Merge remote-tracking branch 'origin/master' into use-iobject-storage-for-table-engines-1
2024-04-26 19:24:07 +02:00
kssenii
434d2d16f1
Cleanuo
2024-04-26 19:23:33 +02:00
Smita Kulkarni
f474c29b4d
Fix build with new master
2024-04-26 15:33:27 +02:00
kssenii
18e4c0f1da
Fix remaining integration test
2024-04-26 13:35:18 +02:00
Smita Kulkarni
3ace11f92d
Merge branch 'master' into Azure_write_buffer_parallel_upload
2024-04-26 10:27:41 +02:00
liuneng
76b989ebbc
fix missing-field-initializers
2024-04-24 17:13:08 +08:00
Raúl Marín
3df253cb0a
Merge pull request #62850 from danipozo/fix-building-endpoint-from-storage-account-url
...
Azure: fix building final url from endpoint object
2024-04-23 17:26:22 +00:00
Dani Pozo
771219a118
Apply suggestion to use pop_back
2024-04-23 12:34:26 +02:00
Dani Pozo
c75574a1d6
Azure: fix building final url from endpoint object
2024-04-22 18:49:46 +02:00
kssenii
a4daf2b454
Fix hdfs race
2024-04-20 14:46:32 +01:00
Smita Kulkarni
065eceecbf
Merge branch 'master' into Azure_write_buffer_parallel_upload
2024-04-19 17:37:15 +02:00
kssenii
e2e6071063
Fix a few more tests
2024-04-19 13:44:07 +01:00
Smita Kulkarni
613dd1a6c0
Updated to use workload identity for backups & disk
2024-04-18 18:26:07 +02:00
kssenii
c8915a16a5
Fix a few mote tests
2024-04-18 17:22:51 +01:00
Smita Kulkarni
5846e2c8ab
Merge branch 'master' into add_azure_workload_identity_support_blob
2024-04-18 18:11:16 +02:00
Anton Popov
98d47e4ef5
Update src/Disks/ObjectStorages/AzureBlobStorage/AzureBlobStorageAuth.cpp
...
Co-authored-by: SmitaRKulkarni <Smita.Kulkarni@clickhouse.com>
2024-04-18 12:12:14 +02:00
Anton Popov
b50117780c
Merge remote-tracking branch 'upstream/master' into HEAD
2024-04-17 13:05:46 +00:00
kssenii
11be538ac8
Fix several tests
2024-04-16 14:09:33 +01:00
kssenii
652796acd6
Fix MergeTree with HDFS
2024-04-15 16:37:38 +01:00
kssenii
27672e1dfc
Merge remote-tracking branch 'origin/master' into use-iobject-storage-for-table-engines-1
2024-04-15 11:28:14 +01:00
Anton Popov
03916cc13f
fix build and update submodule
2024-04-12 18:26:30 +00:00
Alexander Gololobov
85cd0291a5
Remove getRemotePathsRecursive() from IDisk
2024-04-12 18:34:51 +02:00
Anton Popov
47c653f87c
better retries in azure sdk
2024-04-12 15:39:14 +00:00
Alexander Tokmakov
d510c3ac55
Merge pull request #62266 from ClickHouse/cannot_allocate_thread
...
Add fault injection for "Cannot allocate thread"
2024-04-12 14:40:42 +00:00
Anton Popov
523ee302cb
more profile events
2024-04-11 16:37:54 +00:00
Smita Kulkarni
b8c77a4453
Support parallel write buffer for AzureBlobStorage
2024-04-11 10:13:41 +02:00
Alexander Tokmakov
d8e97b51bf
Merge branch 'master' into cannot_allocate_thread
2024-04-10 21:21:42 +02:00
Anton Popov
8d813b6594
Merge remote-tracking branch 'upstream/master' into azure-profile-events
2024-04-10 14:40:54 +00:00
Anton Popov
4a8c1bf6a3
Merge pull request #62306 from CurtizJ/azure-retryable-errors
...
Better handling of errors from azure storage
2024-04-10 13:38:33 +00:00
Anton Popov
52635d2b8f
add profile events for azure disk
2024-04-10 01:29:09 +00:00
Robert Schulze
4035334634
Merge remote-tracking branch 'rschu1ze/master' into static-instance2
2024-04-09 11:45:27 +00:00
Robert Schulze
97281203b8
Don't access static members through instance, pt. II
...
clang-tidy rightfully complains
(-readability-static-accessed-through-instance) that static member
functions are accessed through non-static members
2024-04-07 11:09:35 +00:00
Anton Popov
fa01f26def
add test
2024-04-05 17:39:12 +00:00
kssenii
357e5e3ae6
Merge remote-tracking branch 'origin/master' into use-iobject-storage-for-table-engines-1
2024-04-05 16:15:56 +02:00
Sema Checherinda
8b75b6ae77
Merge branch 'master' into chesema-keep-alive-send-and-receive
2024-04-05 01:04:22 +02:00
Sema Checherinda
ae3a199939
support max requests for keep alive
2024-04-05 00:37:29 +02:00
Sema Checherinda
dd852da339
add more debug logs
2024-04-05 00:30:03 +02:00
Alexander Tokmakov
5db9fbed52
cancel tasks on exception
2024-04-04 22:32:57 +02:00
Anton Popov
d7fb851d17
better handling of errors from azure storage
2024-04-04 19:53:58 +00:00
kssenii
1e982a2368
Merge remote-tracking branch 'origin/master' into use-iobject-storage-for-table-engines-1
2024-04-04 13:26:06 +02:00
kssenii
be30806dbe
Merge remote-tracking branch 'origin/master' into use-iobject-storage-for-table-engines-1
2024-04-04 13:20:12 +02:00
Robert Schulze
de2a0be025
Don't access static members through instance
...
- clang-tidy rightfully complains (-readability-static-accessed-through-instance)
- not going to enable the warning for now to avoid breaking the build
2024-04-03 18:50:33 +00:00
SmitaRKulkarni
9bb185593e
Merge pull request #61988 from ClickHouse/Azure_fix_exception_in_finalize
...
Fix WriteBufferAzureBlobStorage destructor uncaught exception
2024-04-03 18:22:27 +02:00
vdimir
e5f218c4cc
Merge branch 'master' into vdimir/storage_join_load_order
2024-04-01 14:15:01 +02:00
kssenii
5c63d09c5b
More tests fixes
2024-03-28 14:15:14 +01:00
Smita Kulkarni
afbb3f8a50
Added wait before commit blocks
2024-03-28 09:26:00 +01:00
kssenii
ca1c119115
Merge remote-tracking branch 'origin/master' into use-iobject-storage-for-table-engines-1
2024-03-27 22:32:44 +01:00
kssenii
a2e210462d
Fix style check
2024-03-27 22:31:22 +01:00
kssenii
7a991de488
Fix tests
2024-03-27 19:06:19 +01:00
Smita Kulkarni
42a27c5c87
Fix WriteBufferAzureBlobStorage destructor uncaught exception
2024-03-27 16:27:47 +01:00
vdimir
128c51c6fc
Fix storage join loading order
2024-03-27 11:27:30 +00:00
Michael Kolupaev
d5da446d21
Tidy
2024-03-27 02:32:15 +00:00
Nikita Mikhaylov
1b828ab6f6
Fix for ASync
2024-03-26 16:05:43 +00:00
Michael Kolupaev
ab0c601d63
Make userspace page cache work with 'web' disks
2024-03-26 09:14:55 +00:00
kssenii
f5982fdb1f
Fix some tests
2024-03-25 19:19:54 +01:00
kssenii
e019b3a391
Fix build after merge
2024-03-25 16:12:39 +01:00
kssenii
46941e7243
Merge remote-tracking branch 'origin/master' into use-iobject-storage-for-table-engines-1
2024-03-25 15:10:11 +01:00
SmitaRKulkarni
a642f4d3ec
Merge pull request #59929 from ClickHouse/Azure_write_buffer
...
Asynchronous WriteBuffer for AzureBlobStorage
2024-03-25 10:32:56 +01:00
SmitaRKulkarni
f16dffaefc
Merge pull request #61503 from ClickHouse/Azure_read_buffer
...
Add support of ParallelReadBuffer to ReadBufferFromAzureBlobStorage
2024-03-25 09:51:59 +01:00
alesapin
a41c005a6e
Merge branch 'master' into Azure_write_buffer
2024-03-24 15:04:18 +01:00
Alexey Milovidov
056c8ce021
Merge pull request #59965 from nickitat/express_support
...
S3Express support
2024-03-23 20:52:44 +03:00
Smita Kulkarni
19e7022a87
Merge branch 'master' into Azure_write_buffer
2024-03-23 11:48:10 +01:00
Smita Kulkarni
7a154ad373
Updated log levels
2024-03-22 11:51:35 +01:00
Smita Kulkarni
a08c16e8eb
Fixed clang tidy build
2024-03-22 10:45:48 +01:00
Smita Kulkarni
d969c0cd08
Removed unused includes
2024-03-22 10:30:55 +01:00
Smita Kulkarni
257c263596
Avoid extra copy of data
2024-03-21 22:33:34 +01:00
SmitaRKulkarni
3822e41f7d
Merge a2836b0887 into 30df0fcd60
2024-03-21 21:17:57 +00:00
Smita Kulkarni
3f653bbc32
Added test with log check
2024-03-21 20:28:52 +01:00
Smita Kulkarni
929173cce2
Fix S3 buffer allocation
2024-03-21 19:18:16 +01:00
Kseniia Sumarokova
0c824dab17
Merge pull request #60585 from aalexfvk/traverse_shadow_remote_data_paths
...
Traverse shadow directory for system.remote_data_paths
2024-03-21 19:02:36 +01:00
Smita Kulkarni
6c5e881da7
Merge branch 'master' into Azure_write_buffer
2024-03-21 11:12:19 +01:00
Smita Kulkarni
e0d14a1eaf
Updated lambda and name of BufferAllocationPolicy
2024-03-21 11:10:54 +01:00
Smita Kulkarni
7a3ab461bf
Fix style check
2024-03-21 09:24:35 +01:00
Smita Kulkarni
040b191899
Fix readBigAt
2024-03-20 20:25:38 +01:00
Smita Kulkarni
9f2d44fc65
Merge branch 'master' into Azure_write_buffer
2024-03-20 11:02:48 +01:00
Smita Kulkarni
2c0e266cfe
Addressed review comments
2024-03-20 10:58:46 +01:00
kssenii
0122c3e883
Merge remote-tracking branch 'origin/master' into use-iobject-storage-for-table-engines-1
2024-03-19 19:31:02 +01:00
Smita Kulkarni
824092b619
Updated to use single part upload for single block of small size and fixed test fail
2024-03-19 18:38:02 +01:00
Raúl Marín
de855ca917
Reduce header dependencies
2024-03-19 17:04:29 +01:00
Smita Kulkarni
c16922493a
Removed detachPart & reallocateBuffer functions
2024-03-19 14:54:06 +01:00
János Benjamin Antal
922df7d60e
Merge pull request #61003 from amosbird/less-header-1
...
Replace some header includes with forward declarations
2024-03-19 12:14:06 +01:00
Smita Kulkarni
45e15d16e5
Merge branch 'master' into Azure_write_buffer
2024-03-19 09:31:11 +01:00
Smita Kulkarni
0530055842
Updated names in BufferAllocationPolicy
2024-03-18 15:59:05 +01:00
János Benjamin Antal
4b4a03c0f1
Merge branch 'master' into less-header-1
2024-03-18 10:25:42 +01:00
Smita Kulkarni
248a8a60f1
Fix style check
2024-03-18 10:02:52 +01:00
Robert Schulze
c17536c3b3
Enable clang-tidy in headers
2024-03-18 08:00:09 +00:00
Smita Kulkarni
12289a9ebe
Add support of ParallelReadBuffer to ReadBufferFromAzureBlobStorage
2024-03-17 22:34:41 +01:00
Alexey Milovidov
8094dd88d9
Merge branch 'master' into split-cast-overload-resolver
2024-03-14 13:08:37 +01:00
Alexey Milovidov
1a41aa4ac7
Merge branch 'master' of github.com:ClickHouse/ClickHouse into split-cast-overload-resolver
2024-03-14 13:06:23 +01:00
Kseniia Sumarokova
bc9dd7bb0b
Merge branch 'master' into revive-cache-contention-fix
2024-03-13 16:52:00 +01:00
Kseniia Sumarokova
893ea7e4fe
Merge pull request #60396 from ClickHouse/fix-usage-of-plain-metadata
...
Fix usage of plain metadata type with new disks configuration option
2024-03-13 14:18:36 +01:00
kssenii
3931351ec4
Pass timeout through setting
2024-03-13 12:36:47 +01:00
kssenii
296f7a1da2
Fix
2024-03-12 19:02:58 +01:00
Kseniia Sumarokova
3257bfa918
Merge branch 'master' into traverse_shadow_remote_data_paths
2024-03-12 15:18:52 +01:00
Kseniia Sumarokova
61543edc8b
Merge branch 'master' into fix-usage-of-plain-metadata
2024-03-12 11:53:00 +01:00
Alexey Milovidov
5222ebc4b4
Merge branch 'master' of github.com:ClickHouse/ClickHouse into split-cast-overload-resolver
2024-03-12 06:37:38 +01:00
Kseniia Sumarokova
38ba6990bb
Merge branch 'master' into use-iobject-storage-for-table-engines-1
2024-03-11 18:47:56 +01:00
kssenii
bfc4f5e1d3
Merge remote-tracking branch 'origin/master' into use-iobject-storage-for-table-engines-1
2024-03-11 14:57:15 +01:00
Kseniia Sumarokova
e59564a9d9
Merge branch 'master' into traverse_shadow_remote_data_paths
2024-03-11 13:19:05 +01:00
Kseniia Sumarokova
3832b0e126
Merge branch 'master' into add-a-way-to-force-read-through-filesystem-cache-for-merges
2024-03-11 11:30:03 +01:00
kssenii
70272d4174
Minor
2024-03-11 10:55:01 +01:00
Kseniia Sumarokova
aacfefe0fe
Merge branch 'master' into fix-usage-of-plain-metadata
2024-03-11 10:37:53 +01:00
Smita Kulkarni
c724d36220
Merge branch 'master' into Azure_write_buffer
2024-03-11 09:27:24 +01:00
Alexey Milovidov
4373d5ba16
Merge branch 'master' into split-cast-overload-resolver
2024-03-11 03:01:50 +01:00
Kseniia Sumarokova
ae98d4290c
Merge branch 'master' into traverse_shadow_remote_data_paths
2024-03-10 12:16:54 +01:00
kssenii
e5e6625b0c
Merge remote-tracking branch 'origin/master' into fix-usage-of-plain-metadata
2024-03-10 12:05:02 +01:00
Sema Checherinda
77c5de700f
fix how web disk read empty directories
2024-03-10 10:00:18 +01:00
Sema Checherinda
f7f1d86e66
fix tests test_attach_without_fetching test_replicated_merge_tree_wait_on_shutdown
2024-03-10 10:00:18 +01:00
Sema Checherinda
5c41727725
http connections pools
2024-03-10 10:00:18 +01:00
Alexey Milovidov
009c2ea9f6
Move a class into an anonymous namespace
2024-03-09 05:27:29 +01:00
Nikita Taranov
e66ac5770c
Merge pull request #60462 from ClickHouse/davenger-patch-1
...
Do not check to and from files existence in metadata_storage because it does not see uncommitted changes
2024-03-08 00:26:58 +01:00
Amos Bird
f2a3ffe9eb
Replace some headers with forward decl
2024-03-07 18:10:09 +08:00
Smita Kulkarni
10b5ce8ab3
Updated BufferAllocationPolicy
2024-03-07 10:26:50 +01:00
kssenii
31ed1966e3
Fix build
2024-03-07 16:44:10 +08:00
Kseniia Sumarokova
a24e32168a
Merge branch 'master' into fix-usage-of-plain-metadata
2024-03-07 14:44:06 +08:00
Kseniia Sumarokova
0d67d9c79d
Merge pull request #60165 from ClickHouse/fix-for-local-object-storage
...
Allow local/azure as object storage type instead of local_blob_storage/azure_blob_storage
2024-03-07 14:42:44 +08:00
Kseniia Sumarokova
8fc3581446
Merge branch 'master' into fix-usage-of-plain-metadata
2024-03-07 14:29:57 +08:00
Sema Checherinda
bae34f4cd2
Merge pull request #59495 from ClickHouse/chesema-remove-objects-capability
...
generate key for clickhouse_remove_objects_capability
2024-03-06 17:45:32 +01:00
kssenii
0c293bce11
Merge remote-tracking branch 'origin/master' into use-iobject-storage-for-table-engines-1
2024-03-06 12:06:19 +08:00
Nikita Taranov
c8e41dc07c
Merge branch 'master' into davenger-patch-1
2024-03-05 19:12:22 +01:00
Sema Checherinda
6e085c713f
generate key for clickhouse_remove_objects_capability
2024-03-05 14:02:43 +01:00
Nikolai Kochetov
86ad80839a
Merge pull request #60515 from ClickHouse/cleanup-lazy-format-always
...
PullingAsyncPipelineExecutor cleanup
2024-03-05 13:12:07 +01:00
Aleksei Filatov
f7d8dd0b71
Merge branch 'master' into traverse_shadow_remote_data_paths
2024-03-05 10:11:07 +03:00
Aleksei Filatov
758a75c1b4
Fix flaky test. Fix clang-tidy warning
2024-03-05 09:53:30 +03:00
Kseniia Sumarokova
6f5d79db24
Merge branch 'master' into fix-for-local-object-storage
2024-03-05 14:14:34 +08:00
Kseniia Sumarokova
2ee846b393
Fix build
2024-03-05 14:07:56 +08:00
Kseniia Sumarokova
b0050566e2
Fix style check
2024-03-05 12:14:56 +08:00
Kseniia Sumarokova
3f90c933f3
Merge branch 'master' into fix-usage-of-plain-metadata
2024-03-05 11:22:56 +08:00
Kseniia Sumarokova
dbbe6b9f69
Merge branch 'master' into add-a-way-to-force-read-through-filesystem-cache-for-merges
2024-03-05 11:13:32 +08:00
Alexey Milovidov
de6c5d3a40
Merge pull request #60552 from ClickHouse/pc2
...
Userspace page cache again
2024-03-04 17:47:35 +03:00
Kseniia Sumarokova
a7db6688ed
Update ObjectStorageFactory.cpp
2024-03-04 18:24:24 +08:00
Kseniia Sumarokova
925fd00fd1
Merge branch 'master' into fix-usage-of-plain-metadata
2024-03-04 16:04:46 +08:00
Alexey Milovidov
3938d7156a
Merge branch 'master' into fix-for-local-object-storage
2024-03-02 19:44:12 +01:00
Alexander Gololobov
2dd83a09e3
No need to capture this
2024-03-01 13:40:09 +01:00
Nikita Mikhaylov
590c30a23a
Update DiskObjectStorageTransaction.cpp
2024-03-01 13:40:09 +01:00
