Dmitry Novik
197116dfeb
Merge pull request #46451 from ClickHouse/copy-s3-file-fix
...
Wait for tasks finish when scheduler failed
2023-02-17 20:11:23 +01:00
Dmitry Novik
659e680c76
Add a comment
2023-02-16 01:24:31 +00:00
Dmitry Novik
cfb832b205
Wait for tasks finish when scheduler failed
2023-02-15 15:50:06 +00:00
Antonio Andelic
043b394ff0
Remove unneeded move
2023-02-14 13:36:34 +00:00
Antonio Andelic
f703b4c25b
Merge branch 'master' into s3-client-polish
2023-02-14 13:33:45 +00:00
Antonio Andelic
15df1fcd72
Merge pull request #46339 from ClickHouse/refactor-s3
...
Extract common logic for S3
2023-02-14 14:30:43 +01:00
Antonio Andelic
1826c29c82
Merge branch 'master' into s3-client-polish
2023-02-14 13:08:27 +00:00
Nikolai Kochetov
ca55c01b6e
Wait for background tasks in ~UploadHelper ( #46334 )
2023-02-13 15:08:16 +01:00
Antonio Andelic
ae905b5762
Split S3Common into multiple files
2023-02-13 12:28:04 +00:00
kssenii
3067c1d723
Merge remote-tracking branch 'upstream/master' into resubmit-prefetches
2023-02-11 11:36:23 +01:00
Sergei Trifonov
2931c3bbe1
Merge pull request #45711 from ClickHouse/io-scheduler-integration
...
Integrate IO scheduler with buffers for remote reads and writes
2023-02-11 10:26:58 +01:00
Antonio Andelic
9e11f06e79
Merge pull request #46191 from ClickHouse/s3-reorder-creds
...
Prefer explicitly defined credentials for S3
2023-02-10 16:35:54 +01:00
Antonio Andelic
c02adec4b8
Early return
2023-02-10 07:47:48 +00:00
serxa
acabee8554
review fixes
2023-02-09 20:17:46 +00:00
Robert Schulze
e490ec91d9
Merge branch 'master' into rs/fix-fragile-linking
2023-02-09 11:33:59 +01:00
Antonio Andelic
ae731b739a
Prefer explicitly defined creds for S3
2023-02-09 08:13:16 +00:00
Alexander Tokmakov
8101b044fa
Merge pull request #46091 from azat/sanity-assertions
...
Sanity assertions for closing file descriptors
2023-02-09 01:02:03 +03:00
kssenii
b0b865c32e
Resubmit prefetches
2023-02-08 21:26:24 +01:00
Sergei Trifonov
b0fc551c27
Merge branch 'master' into io-scheduler-integration
2023-02-08 18:16:00 +01:00
Robert Schulze
6ff232d782
Merge branch 'master' into rs/fix-fragile-linking
2023-02-08 12:51:12 +01:00
Antonio Andelic
2d26c9cffc
Fix error codes
2023-02-08 10:39:32 +00:00
Antonio Andelic
28d1fd6c83
More fixes
2023-02-08 09:48:35 +00:00
Azat Khuzhin
d21f6a0bd6
Fix WriteBufferFromFile destructor after close()
...
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-02-07 17:59:05 +01:00
Robert Schulze
783e26d2ba
Cosmetics
2023-02-07 11:42:31 +00:00
Azat Khuzhin
8cc41b7f41
Check return value of ::close()
...
Note, that according close(2), EINTR should not be retriable for close()
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-02-07 11:28:22 +01:00
kssenii
ab0dedf0c8
Simplify code around storage s3 configuration
2023-02-06 16:23:17 +01:00
Antonio Andelic
4cd361747d
Address PR comment
2023-02-06 10:11:48 +00:00
Robert Schulze
84b9ff450f
Fix terribly broken, fragile and potentially cyclic linking
...
Sorry for the clickbaity title. This is about static method
ConnectionTimeouts::getHTTPTimeouts(). It was be declared in header
IO/ConnectionTimeouts.h, and defined in header
IO/ConnectionTimeoutsContext.h (!). This is weird and caused issues with
linking on s390x (##45520). There was an attempt to fix some
inconsistencies (#45848 ) but neither did @Algunenano nor me at first
really understand why the definition is in the header.
Turns out that ConnectionTimeoutsContext.h is only #include'd from
source files which are part of the normal server build BUT NOT part of
the keeper standalone build (which must be enabled via CMake
-DBUILD_STANDALONE_KEEPER=1). This dependency was not documented and as
a result, some misguided workarounds were introduced earlier, e.g.
0341c6c54b
2023-02-05 20:49:34 +00:00
Antonio Andelic
d5117f2aa6
Define S3 client with bucket and endpoint resolution ( #45783 )
...
* Update aws
* Define S3 client with bucket and endpoint resolution
* Add defines for ErrorCodes
* Use S3Client everywhere
* Remove unused errorcode
* Add DROP S3 CLIENT CACHE query
* Add a comment
* Fix style
* Update aws
* Update reference files
* Add missing include
* Fix unit test
* Remove unneeded declarations
* Correctly use RetryStrategy
* Rename S3Client to Client
* Fix retry count
* fix clang-tidy warnings
2023-02-03 14:30:52 +01:00
Antonio Andelic
a3e3b65b32
Merge branch 'master' into keeper-use-fallocate
2023-02-01 09:05:49 +01:00
Alexey Milovidov
bf9f62dcbd
Merge pull request #38456 from sauliusvl/iouring
...
Re-introduce io_uring read method
2023-02-01 06:41:32 +03:00
Antonio Andelic
1786237be6
Merge branch 'master' into keeper-use-fallocate
2023-01-31 13:53:47 +00:00
alesapin
631c8fb155
Merge branch 'master' into add-setting-allow-head-object-request
2023-01-29 14:58:27 +01:00
Saulius Valatka
ac2c921bdf
add initial io_uring support
2023-01-28 21:54:44 +02:00
serxa
6d6dc40128
fix typos
2023-01-27 22:48:02 +00:00
Sergei Trifonov
5c2686fbf0
Merge branch 'master' into io-scheduler-integration
2023-01-27 19:56:23 +01:00
serxa
f1a940d7d3
fix
2023-01-27 18:55:46 +00:00
serxa
21e871a0a4
integrate IO scheduler with buffers
2023-01-27 18:47:22 +00:00
Vitaly Baranov
af74c008c4
Use one request to implement S3ObjectStorage::getObjectMetadata instead of two ones.
2023-01-27 18:42:37 +01:00
Vitaly Baranov
d02b255b55
Add new setting "allow_head_object_request" to S3::RequestSettings.
2023-01-27 15:10:34 +01:00
Vitaly Baranov
f9282cfcac
Make version_id an optional parameter again.
2023-01-27 15:10:19 +01:00
Vitaly Baranov
aea9ccdb60
Pass request settings to S3::getObjectInfo().
2023-01-27 15:10:09 +01:00
Vitaly Baranov
a8304525ed
Move getObjectInfo() to a separate header.
2023-01-27 15:09:38 +01:00
alesapin
716d25c33f
Merge pull request #45619 from CurtizJ/refactoring-merge-tree-parts
...
Refactoring of code near merge tree parts
2023-01-27 13:36:33 +01:00
Alexander Tokmakov
c366806c3e
Merge pull request #45527 from ClickHouse/exception_message_patterns4
...
Better formatting for exception messages 2
2023-01-27 15:31:52 +03:00
Alexander Tokmakov
7af3c6d162
fix
2023-01-27 01:22:29 +01:00
Alexey Milovidov
f401449bd9
Merge pull request #45661 from ClickHouse/generate-random-lowcardinality
...
Allow LowCardinality in GenerateRandom
2023-01-27 02:14:38 +03:00
Robert Schulze
032cdb986e
Merge pull request #45615 from ClickHouse/fix-gorilla
...
Fix codec pipeline Delta (or DoubleDelta) + Gorilla
2023-01-27 00:04:39 +01:00
Alexey Milovidov
d942a4ddef
Allow LowCardinality in GenerateRandom
2023-01-26 15:35:20 +01:00
Alexander Tokmakov
067b1f5f13
Merge branch 'master' into exception_message_patterns4
2023-01-26 15:20:58 +01:00
