Nikita Taranov
876d5ae0a7
fix ReadBufferFromS3
2023-06-20 20:27:43 +02:00
Nikita Taranov
b546d8e665
review fixes + test
2023-06-16 15:30:56 +02:00
Nikita Taranov
1dddcc9472
use connection pool
2023-06-14 12:51:02 +02:00
Michael Kolupaev
b51064a508
Get rid of SeekableReadBufferFactory, add SeekableReadBuffer::readBigAt() instead
2023-06-01 18:48:30 -07:00
kssenii
cd25d61795
Merge remote-tracking branch 'upstream/master' into cache-better-locks
2023-04-17 19:47:43 +02:00
Michael Kolupaev
87be78e6de
Better
2023-04-17 04:58:32 +00:00
Michael Kolupaev
e133633359
Parallel decoding with one row group per thread
2023-04-17 04:58:32 +00:00
Michael Kolupaev
683077890f
Highly questionable refactoring (getInputMultistream() nonsense)
2023-04-17 04:58:32 +00:00
Michael Kolupaev
2d4fe85513
Something
2023-04-17 04:58:32 +00:00
Michael Kolupaev
dc6e34075e
Read less unnecessary data from Parquet files
2023-04-17 04:58:32 +00:00
kssenii
d94d08c18d
Fix tests
2023-03-30 15:58:12 +02:00
kssenii
3067c1d723
Merge remote-tracking branch 'upstream/master' into resubmit-prefetches
2023-02-11 11:36:23 +01:00
kssenii
b0b865c32e
Resubmit prefetches
2023-02-08 21:26:24 +01:00
Sergei Trifonov
b0fc551c27
Merge branch 'master' into io-scheduler-integration
2023-02-08 18:16:00 +01:00
Antonio Andelic
d5117f2aa6
Define S3 client with bucket and endpoint resolution ( #45783 )
...
* Update aws
* Define S3 client with bucket and endpoint resolution
* Add defines for ErrorCodes
* Use S3Client everywhere
* Remove unused errorcode
* Add DROP S3 CLIENT CACHE query
* Add a comment
* Fix style
* Update aws
* Update reference files
* Add missing include
* Fix unit test
* Remove unneeded declarations
* Correctly use RetryStrategy
* Rename S3Client to Client
* Fix retry count
* fix clang-tidy warnings
2023-02-03 14:30:52 +01:00
serxa
21e871a0a4
integrate IO scheduler with buffers
2023-01-27 18:47:22 +00:00
Vitaly Baranov
aea9ccdb60
Pass request settings to S3::getObjectInfo().
2023-01-27 15:10:09 +01:00
Vitaly Baranov
a8304525ed
Move getObjectInfo() to a separate header.
2023-01-27 15:09:38 +01:00
Alexander Tokmakov
70d1adfe4b
Better formatting for exception messages ( #45449 )
...
* save format string for NetException
* format exceptions
* format exceptions 2
* format exceptions 3
* format exceptions 4
* format exceptions 5
* format exceptions 6
* fix
* format exceptions 7
* format exceptions 8
* Update MergeTreeIndexGin.cpp
* Update AggregateFunctionMap.cpp
* Update AggregateFunctionMap.cpp
* fix
2023-01-24 00:13:58 +03:00
Alexander Gololobov
91fb1eab8c
Merge pull request #45373 from ClickHouse/throttler-metrics
...
Add detailed profile events for throttling
2023-01-22 11:20:45 +01:00
serxa
ce7e22b87b
add detailed profile events for throttling
2023-01-17 18:29:24 +00:00
Vitaly Baranov
a955504043
Move throw_on_error parameter to the end.
2023-01-15 20:28:16 +01:00
Anton Popov
0c87031e80
Merge remote-tracking branch 'upstream/master' into HEAD
2022-12-13 16:33:21 +00:00
Anton Popov
65a78bcd91
improve performance of storage S3
2022-11-26 15:24:01 +00:00
kssenii
5e01441f61
Show progress bar while reading from s3 table function
2022-11-21 17:56:02 +01:00
Sergei Trifonov
f2f0676bcc
Revert "Revert "S3 request per second rate throttling""
2022-11-17 17:35:04 +01:00
Alexander Tokmakov
9011a18234
Revert "S3 request per second rate throttling"
2022-11-16 22:33:48 +03:00
serxa
6d5d9ff421
rename ReadWriteSettings -> RequestSettings
2022-11-08 13:48:23 +00:00
serxa
2daec0b45e
S3 request per second rate throttling + refactoring
2022-11-07 18:05:40 +00:00
Robert Schulze
78fc36ca49
Generate config.h into ${CONFIG_INCLUDE_PATH}
...
This makes the target location consistent with other auto-generated
files like config_formats.h, config_core.h, and config_functions.h and
simplifies the build of clickhouse_common.
2022-09-28 12:48:26 +00:00
serxa
2ef696ffe1
fix issues
2022-09-19 18:40:32 +00:00
serxa
7a1d4a404a
add DiskS3* profile events per S3 API calls
2022-09-19 18:10:47 +00:00
serxa
f8aa738511
more conventional profile events names
2022-09-19 17:23:22 +00:00
Sergei Trifonov
ff2e1ff8d8
Merge branch 'master' into support_s3_api_event_metrics
2022-09-19 14:35:45 +02:00
alesapin
ad279940d6
Trying print less scary messages in case No such key for outdated parts
2022-09-14 17:25:58 +02:00
mateng0915
f7f976e94e
remove extra space
2022-09-14 19:19:47 +08:00
mateng0915
5badb1b186
resolve the review comments
2022-09-14 19:19:40 +08:00
teng.ma
a85c917488
fix master conflict
2022-09-14 19:18:31 +08:00
teng.ma
ebea9f9359
Feature: Support Event Tracing when calling Aws S3 API
...
Description:
============
In Computing && Storage Architecture, using S3 as remote / Shared storage, the Method to access S3 is using AWS S3 API
There is a gap between ClickHouse DB with Ozone Operation
In ClickHouse, operation is SQL and background task
In S3 , the operation should be AWS S3 API
And one sql maybe can mapped to multiple API
Solution:
=========
Added Calling API as event into system.events table
2022-09-14 19:18:31 +08:00
alesapin
6da37b92e1
Improve logging one time
2022-09-08 13:11:53 +02:00
alesapin
09e97a6381
Fix style
2022-09-06 18:38:34 +02:00
alesapin
ceed9f418b
Return better errors handling
2022-09-06 18:22:44 +02:00
alesapin
6ea7f1e011
Better exception handling for ReadBufferFromS3
2022-09-06 13:59:55 +02:00
kssenii
3134bdeb13
Fix
2022-09-01 16:35:28 +02:00
Sergei Trifonov
b2317ff710
renaming to be consistent
2022-07-14 17:33:22 +02:00
Sergei Trifonov
f854507729
merge master and resolve conflicts
2022-07-13 19:07:28 +02:00
Sergei Trifonov
60c6a5b23c
fix s3 read/write throttling instant
2022-07-13 18:36:08 +02:00
Sergei Trifonov
43779ec280
add max_remote_{read,write}_network_bandwidth_for_server settings
2022-07-11 14:59:39 +02:00
kssenii
bf1c99daab
Refactoring of object storages, add LocalObjectStorage
2022-07-09 12:39:31 +02:00
kssenii
a772a09149
Fix
2022-07-04 00:14:45 +02:00
