Alexey Milovidov
03bbf0676f
Merge branch 'master' into revert_prefixes_s3
2022-09-20 06:14:07 +03:00
serxa
2ef696ffe1
fix issues
2022-09-19 18:40:32 +00:00
serxa
7a1d4a404a
add DiskS3* profile events per S3 API calls
2022-09-19 18:10:47 +00:00
serxa
f8aa738511
more conventional profile events names
2022-09-19 17:23:22 +00:00
Sergei Trifonov
ff2e1ff8d8
Merge branch 'master' into support_s3_api_event_metrics
2022-09-19 14:35:45 +02:00
Alexey Milovidov
730655d4fd
Fix 8/9 of trash
2022-09-19 08:53:20 +02:00
Alexey Milovidov
6b752bff3a
Merge branch 'master' into revert_prefixes_s3
2022-09-18 07:10:57 +03:00
Alexey Milovidov
f4a48cd4d6
Remove cruft
2022-09-17 22:16:31 +02:00
alesapin
33216dedf9
Merge pull request #41323 from ClickHouse/remove_bad_exception
...
Fix parts removal in case of connection loss
2022-09-17 19:14:53 +02:00
alesapin
84828874e0
Change log level
2022-09-17 19:06:54 +02:00
alesapin
d9c3549cc8
Revert s3 prefixes + add retries on write requests unexpected errors
2022-09-16 14:43:43 +02:00
Antonio Andelic
296fe01fb8
Merge branch 'master' into keeper-upload-snapshot-to-s3
2022-09-16 08:00:15 +00:00
Alexey Milovidov
da01982652
Merge pull request #41046 from azat/build/llvm-15
...
Switch to llvm/clang 15
2022-09-16 07:31:06 +03:00
Antonio Andelic
ad7864cb5f
enable S3 if client is built
2022-09-15 14:06:22 +00:00
Antonio Andelic
32a297339b
Fix build
2022-09-15 13:36:16 +00:00
Antonio Andelic
9fdd2f2e61
Basic implementation for S3 snapshot upload
2022-09-15 09:11:55 +00:00
alesapin
ad279940d6
Trying print less scary messages in case No such key for outdated parts
2022-09-14 17:25:58 +02:00
mateng0915
f7f976e94e
remove extra space
2022-09-14 19:19:47 +08:00
mateng0915
f38e262706
fixed the unnecessory changes
2022-09-14 19:19:47 +08:00
mateng0915
5badb1b186
resolve the review comments
2022-09-14 19:19:40 +08:00
teng.ma
a85c917488
fix master conflict
2022-09-14 19:18:31 +08:00
teng.ma
ebea9f9359
Feature: Support Event Tracing when calling Aws S3 API
...
Description:
============
In Computing && Storage Architecture, using S3 as remote / Shared storage, the Method to access S3 is using AWS S3 API
There is a gap between ClickHouse DB with Ozone Operation
In ClickHouse, operation is SQL and background task
In S3 , the operation should be AWS S3 API
And one sql maybe can mapped to multiple API
Solution:
=========
Added Calling API as event into system.events table
2022-09-14 19:18:31 +08:00
kssenii
52ef3758c4
Merge remote-tracking branch 'upstream/master' into fix-thread-status
2022-09-13 16:34:31 +02:00
kssenii
420ac4eb43
s3 header auth in ast
2022-09-13 15:13:28 +02:00
Kseniia Sumarokova
b2c9c04c7b
Merge pull request #40821 from kssenii/improve-marks-cache-loading
...
Allow to load marks in threadpool in advance
2022-09-13 12:31:58 +02:00
Kruglov Pavel
7d7606a1e3
Merge pull request #41162 from Avogar/fix-http-redirect
...
Support relative path in Location header after http redirect
2022-09-13 12:10:44 +02:00
kssenii
b7d751b782
Merge remote-tracking branch 'upstream/master' into improve-marks-cache-loading
2022-09-11 13:23:30 +02:00
Azat Khuzhin
4e3135383b
Fix clang-tidy warnings (from clang-15)
...
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-09-10 21:38:35 +02:00
Alexey Milovidov
fa62c7e982
Fix half of trash
2022-09-10 04:08:16 +02:00
avogar
d8cbd51333
Support relative path in Location header after http redirect
2022-09-09 19:26:14 +00:00
alesapin
5e00124f8a
More logging for S3
2022-09-08 16:18:21 +02:00
alesapin
6da37b92e1
Improve logging one time
2022-09-08 13:11:53 +02:00
kssenii
3dc19b4fe2
Fix
2022-09-08 12:23:59 +02:00
kssenii
e51313b6b3
Get rid of static threadpools
2022-09-07 17:48:11 +02:00
alesapin
43493389b9
Merge branch 'master' into one_more_logging
2022-09-06 19:40:24 +02:00
alesapin
09e97a6381
Fix style
2022-09-06 18:38:34 +02:00
alesapin
ceed9f418b
Return better errors handling
2022-09-06 18:22:44 +02:00
alesapin
422b1658eb
Review fix
2022-09-06 14:42:48 +02:00
alesapin
6ea7f1e011
Better exception handling for ReadBufferFromS3
2022-09-06 13:59:55 +02:00
Sergei Trifonov
f77809ddbc
Merge pull request #40900 from ClickHouse/s3-detailed-metrics
...
S3 detailed metrics
2022-09-06 13:20:57 +02:00
alesapin
cbcf267f55
One more line of logging in write buffer from S3
2022-09-05 22:54:38 +02:00
Kruglov Pavel
cadae5a2f6
Merge pull request #40896 from kssenii/fix-memory-usage-for-remote
...
Fix extra memory allocation for remote read buffers
2022-09-05 20:38:57 +02:00
Sergei Trifonov
1c247308f1
Merge branch 'master' into s3-detailed-metrics
2022-09-05 15:55:41 +02:00
kssenii
e2df4b1da1
Merge remote-tracking branch 'origin/fix-races-in-write-buffer-from-s3' into fix-races-in-write-buffer-from-s3
2022-09-04 19:50:19 +02:00
kssenii
52375d2eb0
Merge master
2022-09-04 19:48:04 +02:00
Kseniia Sumarokova
5f757be43d
Merge pull request #40943 from kssenii/fix-deadlock-in-s3-write-buffer
...
Fix deadlock in WriteBufferFromS3
2022-09-04 13:17:41 +02:00
Kseniia Sumarokova
1e81e0ef67
Update WriteBufferFromS3.cpp
2022-09-02 18:54:34 +02:00
kssenii
239847ad16
Fix races
2022-09-02 18:27:07 +02:00
Sergei Trifonov
067e712fdf
Merge branch 'master' into s3-detailed-metrics
2022-09-02 16:59:52 +02:00
Sergei Trifonov
3daf1f5c77
fix build
2022-09-02 16:57:51 +02:00
kssenii
c9b512e33e
Fix
2022-09-02 16:15:54 +02:00
kssenii
0db38fcbc9
Fix
2022-09-02 15:32:46 +02:00
alesapin
45fe9c800a
Merge pull request #40898 from georgthegreat/patch-2
...
Remove remaining usage of // Y_IGNORE magic comment
2022-09-02 12:24:57 +02:00
Robert Schulze
c7c00f9002
Merge pull request #40739 from ClickHouse/clang-tidy-for-headers
...
Enable clang-tidy for headers
2022-09-02 07:54:50 +02:00
Sergei Trifonov
29e922f05c
support all profile events for disk s3
2022-09-02 02:22:52 +02:00
Sergei Trifonov
66b5cf566c
fix bugs
2022-09-01 19:02:04 +02:00
Sergei Trifonov
ac9dc53c73
Merge branch 'master' into s3-detailed-metrics
2022-09-01 18:31:02 +02:00
Sergei Trifonov
bcb6475c4a
add separate s3 profile events for disk s3
2022-09-01 18:30:55 +02:00
Yuriy Chernyshov
304dc30f0f
Remove remaining usage of // Y_IGNORE magic comment
2022-09-01 18:15:56 +03:00
kssenii
3134bdeb13
Fix
2022-09-01 16:35:28 +02:00
Nikita Mikhaylov
c7b0b87574
Better exception handling in DiskObjectStorageTransaction ( #40813 )
...
Co-authored-by: alesapin <alesapin@clickhouse.com>
2022-09-01 13:29:59 +02:00
Vitaly Baranov
6dcca686cb
Merge pull request #40827 from vitlibar/fix-incremental-backups-for-log-family
...
Fix incremental backups for Log family.
2022-09-01 13:13:32 +02:00
kssenii
82879543d3
Merge remote-tracking branch 'upstream/master' into improve-marks-cache-loading
2022-09-01 13:13:01 +02:00
Kseniia Sumarokova
849262bd99
Merge pull request #40858 from kssenii/move-cache-files
...
Move cache files from Common
2022-09-01 12:28:00 +02:00
Robert Schulze
de64c6b103
Merge branch 'master' into clang-tidy-for-headers
2022-09-01 10:24:56 +02:00
Kruglov Pavel
b7ea529307
Merge pull request #40746 from Avogar/fix-csv-schema-inference
...
Fix possible error Attempt to read after eof in CSV schema inference
2022-08-31 18:29:01 +02:00
kssenii
0f747e7987
Move cache files
2022-08-31 17:50:55 +02:00
Vitaly Baranov
007ae0e6cc
Fix incremental backups for Log family.
2022-08-31 12:57:28 +02:00
Robert Schulze
cedf75ed5e
Enable clang-tidy for headers
...
clang-tidy now also checks code in header files. Because the analyzer
finds tons of issues, activate the check only for directory "base/" (see
file ".clang-tidy"). All other directories, in particular "src/" are
left to future work.
While many findings were fixed, some were not (and suppressed instead).
Reasons for this include: a) the file is 1:1 copypaste of a 3rd-party
lib (e.g. pcg_extras.h) and fixing stuff would make upgrades/fixes more
difficult b) a fix would have broken lots of using code
2022-08-31 10:48:15 +00:00
kssenii
a5f69517ae
Allow to load marks in threadpool
2022-08-30 20:20:05 +02:00
avogar
85fa372ad7
Fix possible error Attempt to read after eof in CSV schema inference
2022-08-29 11:18:53 +00:00
Azat Khuzhin
9fff08eac7
WriteBufferFromS3: remove unused ThreadGroupStatus
...
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-08-29 11:30:17 +02:00
Alexey Milovidov
6e564b18bf
Merge pull request #40600 from FrankChen021/check_url_arg
...
Validate the CompressionMethod parameter of URL table engine
2022-08-27 19:29:55 +03:00
alesapin
7d1e3ecfa3
Merge pull request #40506 from ClickHouse/fix_bad_exception
...
Fix lost errors in AWS CPP SDK
2022-08-25 13:07:35 +02:00
Kruglov Pavel
66e677b307
Merge pull request #40546 from Avogar/fix-inferring-datetimes
...
Fix possible error 'Decimal math overflow' while parsing DateTime64
2022-08-25 11:18:30 +02:00
Frank Chen
4c1fe0d623
validate parameter
...
Signed-off-by: Frank Chen <frank.chen021@outlook.com>
2022-08-25 14:49:17 +08:00
Alexey Milovidov
aaafb797c0
Merge pull request #40541 from ClickHouse/paranoid_check_for_s3_uploads
...
Add paranoid option to check blobs to S3 after upload
2022-08-24 23:51:22 +02:00
Kseniia Sumarokova
4e7578cf3f
Merge pull request #40565 from FrankChen021/url_engine
...
Fix segment fault when writing data to URL table engine
2022-08-24 20:36:45 +02:00
Alexander Gololobov
6c256cbe4a
Merge pull request #40474 from ClickHouse/allow-to-parse-date-as-datetime
...
Allow parsing Date as DateTime and DateTime64
2022-08-24 17:23:44 +02:00
alesapin
f2be18b3a6
Remove trailing
2022-08-24 16:57:59 +02:00
alesapin
f5f0934c13
Update PocoHTTPClient.h
2022-08-24 16:15:43 +02:00
alesapin
e3672d1d40
Fix memsan
2022-08-24 15:25:10 +02:00
alesapin
f0b733313f
Merge branch 'master' into fix_bad_exception
2022-08-24 14:44:40 +02:00
alesapin
cac3005f4d
Merge branch 'master' into paranoid_check_for_s3_uploads
2022-08-24 14:43:11 +02:00
Kruglov Pavel
109f384e5d
Fix typo
2022-08-24 13:40:19 +02:00
Frank Chen
b432b0d100
Merge branch 'master' into url_engine
2022-08-24 17:59:21 +08:00
Frank Chen
335293549b
Fix segment fault when sending data to remote HTTP server via URL table engine
...
Signed-off-by: Frank Chen <frank.chen021@outlook.com>
2022-08-24 17:38:16 +08:00
avogar
de489ba1f7
Fix possible error 'Decimal math overflow' while parsing DateTime64
2022-08-23 14:42:23 +00:00
alesapin
07286dd042
Add paranoid option to check blobs to S3 after upload
2022-08-23 15:12:09 +02:00
Kseniia Sumarokova
d814a56277
Merge pull request #40420 from kssenii/fix-cache-bug
...
Fix incorrect fallback to skip cache in case of very high concurrency level
2022-08-23 14:18:38 +02:00
alesapin
a646db4665
Better
2022-08-23 13:56:41 +02:00
alesapin
eef80b0d06
Fix message
2022-08-23 13:23:53 +02:00
alesapin
d97f3b012a
Better
2022-08-23 13:20:05 +02:00
alesapin
802d56df49
Merge branch 'master' into fix_bad_exception
2022-08-23 12:22:05 +02:00
Kruglov Pavel
54ed9029b8
Merge pull request #40482 from ClickHouse/Avogar-patch-1
...
Fix bug in HadoopSnappyReadBuffer
2022-08-23 11:47:59 +02:00
alesapin
61d014f6ec
Merge branch 'master' into fix_bad_exception
2022-08-23 00:35:29 +02:00
alesapin
780f39b108
Fix lost errors in AWS CPP SDK
2022-08-23 00:34:39 +02:00
Kruglov Pavel
3666b25e6b
Fix
2022-08-22 18:48:32 +02:00
Kruglov Pavel
2636c032c4
Use loop instead of recursion
2022-08-22 18:48:02 +02:00
Kseniia Sumarokova
f85e5f8808
Merge branch 'master' into fix-cache-bug
2022-08-22 16:54:07 +02:00
