Commit Graph

550 Commits

Author SHA1 Message Date
Vladimir Chebotarev
579f8da573 Added SSE-C support in S3 client. 2021-01-28 09:32:41 +03:00
alexey-milovidov
7c240d0ca7
Merge pull request #17870 from excitoon-favorites/s3testiostream
Update of AWS C++ SDK
2021-01-28 07:12:52 +03:00
Vladimir Chebotarev
afdea5eff0 Changed handling of PocoHTTPClientConfiguration in order to ensure AWS SDK is initialized. 2021-01-27 09:38:35 +03:00
Vladimir Chebotarev
8ba8e8efd5 Added prefix-based S3 endpoint settings. 2021-01-25 06:40:57 +03:00
Vladimir Chebotarev
5cbc25c647 Added global setting s3_max_connections. 2021-01-05 13:34:55 +03:00
Nikita Mikhailov
c5f92e5096 better formatfactory 2020-12-30 06:07:30 +03:00
Nikita Mikhailov
d293ae5e8e style 2020-12-28 20:18:53 +03:00
Nikita Mikhaylov
e0addac6fc save changes 2020-12-15 00:56:46 +03:00
Pavel Kovalenko
59a5e63417 Merge remote-tracking branch 'origin/master' into s3-adaptive-single-part-upload
# Conflicts:
#	src/Disks/S3/DiskS3.cpp
#	src/Disks/S3/DiskS3.h
#	src/Disks/S3/registerDiskS3.cpp
#	src/IO/WriteBufferFromS3.cpp
#	src/IO/WriteBufferFromS3.h
2020-12-11 18:28:41 +03:00
Vladimir Chebotarev
39d867e56d
Added proper authentication for S3 client (#16856) 2020-12-10 12:19:42 +03:00
Pavel Kovalenko
db7a87089b Adaptive choose of single/multi part upload in WriteBufferFromS3. 2020-12-09 17:09:04 +03:00
alesapin
45f09fe293
Merge pull request #17220 from ianton-ru/s3_max_redirects
S3 max redirects
2020-11-24 12:13:10 +03:00
Anton Ivashkin
aa208a9d9f Use only 's3_max_redirect' in params instead of all settings 2020-11-23 14:02:17 +03:00
Amos Bird
1d9d586e20
Make global_context consistent. 2020-11-20 18:23:14 +08:00
Anton Ivashkin
15b5d3a55a Merge master 2020-11-13 19:32:35 +03:00
Anton Ivashkin
026f7e0a27 Add 's3_max_redirects' setting 2020-11-13 19:31:51 +03:00
Alexander Tokmakov
b94cc5c4e5 remove more stringstreams 2020-11-10 21:22:26 +03:00
Alexander Tokmakov
5cdfcfb307 remove other stringstreams 2020-11-09 22:12:44 +03:00
alexey-milovidov
f4ba5f1f9a
Merge pull request #16772 from ClickHouse/fix-stringstream
Fix "server failed to start" error
2020-11-08 14:27:08 +03:00
Alexey Milovidov
5314185e25 Merge branch 'master' into azat-optimize_skip_unused_shards-optimization 2020-11-08 00:17:59 +03:00
Alexey Milovidov
fd84d16387 Fix "server failed to start" error 2020-11-07 03:14:53 +03:00
Amos Bird
47fcd8bffb
Fix some problems introduced by refactoring. 2020-10-12 17:58:09 +08:00
Amos Bird
d2dcfc3f0d
Refactor processors. 2020-10-12 17:30:05 +08:00
Azat Khuzhin
b838214a35 Pass non-const SelectQueryInfo (and drop mutable qualifiers) 2020-10-02 22:42:35 +03:00
alesapin
17b86dec96 Recursive defaults 2020-10-02 15:38:50 +03:00
Vladimir Chebotarev
a38c7ff2a8 Somehow uncompressed PUT works and gzipped PUT doesn't, in S3 storage. 2020-09-30 15:04:42 +03:00
Vladimir Chebotarev
698768add0 Fixed format argument in S3 storage when compression method is selected. 2020-09-30 13:55:44 +03:00
alexey-milovidov
eeefcd6493
Merge pull request #13337 from ClickHouse/refactor-pipes-3
Refactor Pipe
2020-08-14 04:12:22 +03:00
Vladimir Chebotarev
7371cf6bd1
Proper remote host checking in S3 redirects (#13404)
* Proper remote host checking in S3 redirects.

Co-authored-by: Alexander Kazakov <Akazz@users.noreply.github.com>
2020-08-11 09:09:08 +03:00
Nikolai Kochetov
2cca4d5fcf Refactor Pipe [part 2]. 2020-08-03 16:54:14 +03:00
Peng Jian
7579d65ebc Registering StorageS3 and TableFunctionS3 under two different names (S3 and COSN). 2020-07-13 22:13:30 +08:00
Peng Jian
958bb1f3aa COS (Tencent Cloud Object Storage) integration
To allow import and export to COS.
2020-07-10 15:26:55 +08:00
Vladimir Chebotarev
faf6be6576
Implemented single part uploads for DiskS3 (#12026)
* Implemented single part uploads for DiskS3.
* Added `min_multi_part_upload_size` to disk configuration.
2020-07-09 17:09:17 +03:00
alesapin
c9fa5d2ec3 Better naming 2020-06-19 18:39:41 +03:00
alesapin
71f99a274d Compileable getSampleBlockWithColumns in StorageInMemoryMetadata 2020-06-16 17:25:08 +03:00
alesapin
36ba0192df Metadata in read and write methods of IStorage 2020-06-15 22:08:58 +03:00
alesapin
af2fe2ba55 Compilable setColumns, setConstraints, setIndices 2020-06-15 19:55:33 +03:00
alexey-milovidov
e3c528e661
Merge pull request #11174 from excitoon-favorites/s3tests
Added tests for improved S3 URL parsing
2020-06-07 02:44:40 +03:00
Vladimir Chebotarev
d62e85f875 Improved logging of globs in S3. 2020-06-04 16:12:08 +03:00
Pervakov Grigorii
bc9381406f Add default credentials and custom headers for s3 table functions. 2020-06-02 20:48:49 +03:00
alexey-milovidov
256f84ddc2
Merge pull request #11230 from Jokser/s3-poco-http-client
S3 Poco Http Client
2020-06-01 01:20:21 +03:00
Pavel Kovalenko
677cb56a06 S3 Poco HTTP Client WIP. 2020-05-27 22:41:49 +03:00
Vladimir Chebotarev
7f9a257c31 Fixed S3 globbing which could fail in case of more than 1000 keys and some backends. 2020-05-25 15:57:32 +03:00
Vladimir Chebotarev
98f31374d5 Added improved S3 URL parsing to DiskS3. 2020-05-22 11:18:34 +03:00
Vladimir Chebotarev
02cb08ea2f Fixed parsing of S3 URL. 2020-05-22 11:17:10 +03:00
alesapin
f981649213 Fix pushing to views stream and refactor virtuals 2020-04-28 13:38:57 +03:00
alesapin
4badd0fd28 Better code 2020-04-27 20:46:51 +03:00
alesapin
18c550df15 Better virtuals logic 2020-04-27 16:55:30 +03:00
Vitaly Baranov
e573549945 Rework access rights for table functions. 2020-04-07 23:31:59 +03:00
Ivan Lezhankin
06446b4f08 dbms/ → src/ 2020-04-03 18:14:31 +03:00