thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-20 05:05:38 +00:00
Code Issues Packages Projects Releases Wiki Activity
154,077 Commits 9,332 Branches 1,785 Tags 2.2 GiB
49ec96ff36
Commit Graph

183 Commits

Author SHA1 Message Date
Nikolai Kochetov
7dea7b7f76 Decrease log level for some s3 messages. 2021-12-20 18:18:54 +03:00
Kseniia Sumarokova
87b920594e
Merge pull request #31678 from ucasfl/log 
fix typo
 2021-11-25 11:29:36 +03:00
feng lv
89e9569ad6 fix typo 2021-11-24 04:34:25 +00:00
Kruglov Pavel
4856b50c23
Merge pull request #31409 from excitoon-favorites/s3envsegfaults 
Resolve `nullptr` in STS credentials provider for S3
 2021-11-19 18:45:38 +03:00
Vladimir Chebotarev
8b622bd4ff Fixed a typo. 2021-11-18 10:54:59 +03:00
Vladimir Chebotarev
1e79177ce2 Build fix. 2021-11-16 12:31:51 +03:00
Vladimir Chebotarev
5e9710a26e Attempt to resolve nullptr in STS credentials provider for S3. 2021-11-15 10:46:43 +03:00
cfcz48
d0425016e3 I hereby agree to the terms of the CLA available at: https://yandex.ru/legal/cla/?lang=en 
Changelog category (leave one):
New Feature

Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):
Aliyun OSS Storage support.
 2021-11-11 16:26:29 +08:00
Alexey Milovidov
fe6b7c77c7 Rename "common" to "base" 2021-10-02 10:13:14 +03:00
MaxWk
723ff795a8 support obs storage 2021-09-29 14:11:28 +08:00
Vladimir Chebotarev
0343106692
Fixed logging level for message in S3Common.cpp. 2021-09-23 21:47:31 +03:00
vdimir
6fe63a80bc Fix validateKey/Bucket for S3 2021-08-23 09:22:15 +03:00
vdimir
295c8d5686 Remove unused Error Code from IO/S3Common.cpp 2021-08-23 09:22:15 +03:00
vdimir
6ba6577978 Do not allow slashes in bucket formatted from PARTITION BY 2021-08-23 09:22:15 +03:00
vdimir
c4c42b61c4 Validate uft8 in partition key from PARTITION BY for s3 2021-08-23 09:22:15 +03:00
Vladimir Chebotarev
925c496b6c Rolled back changes in Functions. 2021-08-23 09:22:15 +03:00
Vladimir Chebotarev
3c2e090b6b Build and style fix. 2021-08-23 09:22:14 +03:00
Vladimir Chebotarev
a2bbf98253 Update. 2021-08-23 09:22:14 +03:00
Vladimir Chebotarev
0479edd47e Improved validation of S3 buckets and keys. 2021-08-23 09:22:14 +03:00
Vladimir Chebotarev
eb2defb098
Fixed wrong error message in S3Common. 2021-07-23 15:36:19 +03:00
alexey-milovidov
971c79611b
Merge branch 'master' into patch-5 2021-06-12 02:36:13 +03:00
Vladimir
ba198c48ea
Merge pull request #24898 from excitoon/patch-8 2021-06-10 18:34:17 +03:00
Vladimir Chebotarev
8c33e53e80
Review fixes. 2021-06-07 09:41:49 +03:00
Vladimir Chebotarev
1012a768b1
Allowed S3 key to be empty. 2021-06-03 09:09:09 +03:00
Vladimir Chebotarev
89664a3452
Fixed problems with double spaces in logs for S3Common. 2021-06-03 09:05:24 +03:00
Vladimir Chebotarev
bdf03387ab Better S3 read retries. Renamed s3_max_single_read_retries -> s3_single_read_retry_attempts, added sleeps before next attempts. 2021-05-17 10:38:32 +03:00
Vladimir Chebotarev
a6ab040b07 Added region parameter for S3 storage and disk. 2021-05-01 20:45:35 +03:00
Vladimir Chebotarev
6e379fe778 Final style fix. 2021-04-15 10:01:47 +03:00
Vladimir Chebotarev
71621eb676 Style fix. 2021-04-15 10:01:47 +03:00
Vladimir Chebotarev
f85e521e47 Style fix. 2021-04-15 10:01:47 +03:00
Vladimir Chebotarev
342d907b65 Review fixes. 2021-04-15 10:01:47 +03:00
Vladimir Chebotarev
116423b30c Some refactoring. 2021-04-15 10:01:47 +03:00
Vladimir Chebotarev
c0ea3ce08a Removed STS client. 2021-04-15 10:01:43 +03:00
Vladimir Chebotarev
c61cfd0d97 Minor build fixes. 2021-04-15 10:01:16 +03:00
Vladimir Chebotarev
aae8a1f9d9 Moved AWSSTSAssumeRoleCredentialsProvider to the top of the provider chain. 2021-04-15 10:01:16 +03:00
Vladimir Chebotarev
3a467ffc50 Attempt to make credentials provider which uses internal credentials provider chain. 2021-04-15 10:01:13 +03:00
Vladimir Chebotarev
037a4de854 Added STS GetCallerIdentity credentials provider. 2021-04-15 10:00:18 +03:00
Vladimir Chebotarev
579f8da573 Added SSE-C support in S3 client. 2021-01-28 09:32:41 +03:00
Vladimir Chebotarev
76e62739ed Fixed build. 2021-01-27 23:05:41 +03:00
Vladimir Chebotarev
afdea5eff0 Changed handling of PocoHTTPClientConfiguration in order to ensure AWS SDK is initialized. 2021-01-27 09:38:35 +03:00
Vladimir Chebotarev
3322cc812d Updated AWS C++ SDK. 2021-01-26 20:58:02 +03:00
Vladimir Chebotarev
5cbc25c647 Added global setting s3_max_connections. 2021-01-05 13:34:55 +03:00
alexey-milovidov
60efab623d
Update S3Common.cpp 2020-12-14 13:53:40 +03:00
Vladimir Chebotarev
418c974eaf
Fixed std::out_of_range: basic_string in S3 URL parsing. 2020-12-14 11:17:29 +03:00
Azat Khuzhin
5b3ab48861 More forward declaration for generic headers 
The following headers are pretty generic, so use forward declaration as
much as possible:
- Context.h
- Settings.h
- ConnectionTimeouts.h
(Also this shows that some missing some includes -- this has been fixed)

And split ConnectionTimeouts.h into ConnectionTimeoutsContext.h (since
module part cannot be added for it, due to recursive build dependencies
that will be introduced)

Also remove Settings from the RemoteBlockInputStream/RemoteQueryExecutor
and just pass the context, since settings was passed only in speicifc
places, that can allow making a copy of Context (i.e. Copier).

Approx results (How much units will be recompiled after changing file X?):

- ConnectionTimeouts.h
  - mainline: 100

- Context.h:
  - mainline: ~800
  - patched:  415

- Settings.h:
  - mainline: 900-1K
  - patched:  440 (most of them because of the Context.h)
 2020-12-12 17:43:10 +03:00
Vladimir Chebotarev
39d867e56d
Added proper authentication for S3 client (#16856) 2020-12-10 12:19:42 +03:00
Anton Ivashkin
aa208a9d9f Use only 's3_max_redirect' in params instead of all settings 2020-11-23 14:02:17 +03:00
Anton Ivashkin
026f7e0a27 Add 's3_max_redirects' setting 2020-11-13 19:31:51 +03:00
Vladimir Chebotarev
d98366ed79 Better initialization of S3 storage. 2020-10-06 11:20:47 +03:00
Alexey Milovidov
ef5363b87f Fix terribly wrong code 2020-09-19 02:05:13 +03:00
Vladimir Chebotarev
7371cf6bd1
Proper remote host checking in S3 redirects (#13404) 
* Proper remote host checking in S3 redirects.

Co-authored-by: Alexander Kazakov <Akazz@users.noreply.github.com>
 2020-08-11 09:09:08 +03:00
alexey-milovidov
9fb40a3f7b
Merge pull request #12386 from fastio/master 
COS (Tencent Cloud Object Storage) integration
 2020-07-17 14:09:16 +03:00
alexey-milovidov
f3db2f3c3d
Update S3Common.cpp 2020-07-17 14:07:26 +03:00
Peng Jian
2fc58e8f10 Fix code sytle check & build check 2020-07-17 11:33:29 +08:00
Vladimir Chebotarev
ed4da4bec6
Separated AWSAuthV4Signer into different logger, removed "AWSClient: AWSClient" (#12320) 2020-07-15 17:16:45 +03:00
Peng Jian
7579d65ebc Registering StorageS3 and TableFunctionS3 under two different names (S3 and COSN). 2020-07-13 22:13:30 +08:00
Peng Jian
958bb1f3aa COS (Tencent Cloud Object Storage) integration 
To allow import and export to COS.
 2020-07-10 15:26:55 +08:00
Vladimir Chebotarev
a541683efa Moved useless S3 logging to TRACE level. 2020-07-01 01:04:30 +03:00
Pavel Kovalenko
74ea867b1d Fix includes in S3Common.cpp 2020-06-09 05:03:22 +03:00
Pavel Kovalenko
8b8beb26d3 S3 Poco HTTP Client (do not copy response stream body into memory). 2020-06-09 05:03:22 +03:00
alesapin
22235c1951 Tiny fixes for storage s3 2020-06-04 16:48:20 +03:00
Pervakov Grigorii
bc9381406f Add default credentials and custom headers for s3 table functions. 2020-06-02 20:48:49 +03:00
Alexey Milovidov
7fa1366a4a Fix build after merge #11230 2020-06-01 01:25:37 +03:00
alexey-milovidov
256f84ddc2
Merge pull request #11230 from Jokser/s3-poco-http-client 
S3 Poco Http Client
 2020-06-01 01:20:21 +03:00
Alexey Milovidov
25f941020b Remove namespace pollution 2020-05-31 00:57:37 +03:00
Pavel Kovalenko
2b07f69e57 S3 Poco HTTP Client WIP. 2020-05-27 18:02:25 +03:00
alexey-milovidov
bee5849c6a
Merge pull request #11137 from ClickHouse/logger-fmt 
fmt-style logging, part 1
 2020-05-25 01:38:14 +03:00
Alexey Milovidov
e3cd085230 Fix test (looks like it was incorrect) 2020-05-24 13:42:13 +03:00
Alexey Milovidov
7e1813825b Return old names of macros 2020-05-24 01:24:01 +03:00
Alexey Milovidov
7c0c328a35 Progress on task 2020-05-24 00:05:47 +03:00
Alexey Milovidov
35d79e1252 fmt-style logging, part 1 2020-05-23 19:33:07 +03:00
Vladimir Chebotarev
98f31374d5 Added improved S3 URL parsing to DiskS3. 2020-05-22 11:18:34 +03:00
Vladimir Chebotarev
02cb08ea2f Fixed parsing of S3 URL. 2020-05-22 11:17:10 +03:00
Alexey Milovidov
abe40e2127 Added required comment #10576 2020-05-03 01:34:50 +03:00
alexey-milovidov
342ed093de
Update S3Common.cpp 2020-05-02 23:46:39 +03:00
Pavel Kovalenko
a1acbcb019 Code style fixes. 2020-04-29 11:45:13 +03:00
Pavel Kovalenko
2ecf4b5a23 Dynamic proxy configuration for DiskS3. 2020-04-29 11:40:29 +03:00
Alexey Milovidov
be22a4b94e Checkpoint 2020-04-22 08:39:31 +03:00
Pavel Kovalenko
9b1c5c8b51 Fix regex for S3 URI to pass clang-tidy check. 2020-04-08 11:44:26 +03:00
alexey-milovidov
37fe63f8d8
Update S3Common.cpp 2020-04-08 03:49:09 +03:00
Pavel Kovalenko
8c1dda0105 Merge remote-tracking branch 'jokser/s3-uri-virtual-hosted-style-support' into s3-uri-virtual-hosted-style-support 2020-04-06 12:59:59 +03:00
Pavel Kovalenko
f9c98dd689 Merge remote-tracking branch 'origin/master' into s3-uri-virtual-hosted-style-support 2020-04-06 12:57:53 +03:00
Ivan Lezhankin
06446b4f08 dbms/ → src/ 2020-04-03 18:14:31 +03:00