Robert Schulze
777b5bc15b
Don't let storages inherit from boost::noncopyable
...
... IStorage has deleted copy ctor / assignment already
2022-05-03 09:07:08 +02:00
Antonio Andelic
fd0ba2d10b
Protect background exception
2022-05-03 06:34:03 +00:00
Antonio Andelic
bfdb4566c6
Use mutex per worker
2022-05-03 06:18:39 +00:00
alesapin
98e3b16560
Terminate if S3 buffer is not finalized
2022-05-02 22:14:18 +02:00
avogar
533e4adf6c
Try to fix build
2022-05-02 12:07:31 +00:00
Robert Schulze
330212e0f4
Remove inherited create() method + disallow copying
...
The original motivation for this commit was that shared_ptr_helper used
std::shared_ptr<>() which does two heap allocations instead of
make_shared<>() which does a single allocation. Turned out that
1. the affected code (--> Storages/) is not on a hot path (rendering the
performance argument moot ...)
2. yet copying Storage objects is potentially dangerous and was
previously allowed.
Hence, this change
- removes shared_ptr_helper and as a result all inherited create() methods,
- instead, Storage objects are now created using make_shared<>() by the
caller (for that to work, many constructors had to be made public), and
- all Storage classes were marked as noncopyable using boost::noncopyable.
In sum, we are (likely) not making things faster but the code becomes
cleaner and harder to misuse.
2022-05-02 08:46:52 +02:00
Alexey Milovidov
ef9633499c
Merge branch 'master' into fix-trash-s3-metrics
2022-05-02 03:55:10 +02:00
Alexey Milovidov
1ddb04b992
Merge pull request #36715 from amosbird/refactorbase
...
Reorganize source files so that base won't depend on Common
2022-04-30 09:40:58 +03:00
Kseniia Sumarokova
ca994e0861
Merge pull request #36660 from kssenii/fix-stress-test
...
Fix stress test after 36639
2022-04-29 12:56:25 +02:00
Amos Bird
4a5e4274f0
base should not depend on Common
2022-04-29 10:26:35 +08:00
Kseniia Sumarokova
4102502717
Merge pull request #36619 from ClickHouse/parallel-url-performance-issue
...
ParallelReadBuffer small improvements
2022-04-29 02:55:30 +02:00
mergify[bot]
41aa85fd4c
Merge branch 'master' into fix-stress-test
2022-04-29 00:54:09 +00:00
kssenii
1cbf672237
Another fix
2022-04-28 22:51:57 +02:00
Sergei Trifonov
11f40376ae
Merge pull request #34571 from surahman/CH-31221-AWS-S3-object-version-id
...
[CH-31221] AWS S3 Object Version Specification
2022-04-28 15:45:33 +02:00
Alexander Tokmakov
21a021c210
Merge pull request #36745 from vdimir/fix-build-seekable-read-buff
...
Fix build SeekableReadBufferWithSize -> SeekableReadBuffer
2022-04-28 16:34:24 +03:00
vdimir
7c3cf30163
Fix build SeekableReadBufferWithSize -> SeekableReadBuffer
2022-04-28 15:05:04 +02:00
Anton Kozlov
ca519ca75e
Improve performance of file descriptor cache by narrowing mutex scopes ( #36682 )
2022-04-28 14:56:53 +02:00
Kseniia Sumarokova
4c371f710e
Merge pull request #36676 from kssenii/refactor-with-size-buffer
...
Better version of SeekableReadBufferWithSize
2022-04-28 13:44:25 +02:00
kssenii
9b38c9f7eb
Fix
2022-04-28 13:06:43 +02:00
Antonio Andelic
a693898038
Merge branch 'master' into parallel-url-performance-issue
2022-04-28 07:29:07 +00:00
Vitaly Baranov
641a5f5e35
Merge pull request #36198 from vitlibar/backup-improvements-3
...
Backup for replicated tables and other improvements
2022-04-28 09:23:53 +02:00
kssenii
37d11cfb1c
Merge master
2022-04-28 00:44:15 +02:00
kssenii
67e0cf9ab6
One more fix
2022-04-28 00:37:57 +02:00
Kseniia Sumarokova
39b3c1c6af
Merge pull request #36697 from KinderRiven/fix_bug_for_local_cache
...
Fix deadlock for local cache
2022-04-27 18:11:43 +02:00
KinderRiven
cd540c3c51
fix
2022-04-27 19:36:25 +08:00
KinderRiven
813f1bd001
fix
2022-04-27 19:31:50 +08:00
KinderRiven
ad4fb637f0
fix set better state
2022-04-27 19:17:46 +08:00
KinderRiven
504e2c26db
fix bug for deadlock in local cache
2022-04-27 17:34:14 +08:00
Alexey Milovidov
e85c5679e9
Update S3Common.cpp
2022-04-27 03:33:01 +03:00
Alexey Milovidov
3c53cd42bb
Remove excessive logging from S3
2022-04-27 02:30:13 +02:00
kssenii
9d364cdce2
Refactor
2022-04-26 15:33:53 +02:00
Sergei Trifonov
c1a0165f7d
merge from master and fix conflicts
2022-04-26 12:39:16 +02:00
taiyang-li
99dee35b6e
parallel parsing of hive text format
2022-04-26 14:33:10 +08:00
Vitaly Baranov
3966ee1e30
Use SeekableReadBuffer instead of ReadBuffer in IBackupEntry.
2022-04-25 16:34:33 +02:00
Antonio Andelic
4e3b79ec0c
remove arena
2022-04-25 12:23:24 +00:00
Antonio Andelic
3dd38f4917
Use vector for segments
2022-04-25 12:10:41 +00:00
Antonio Andelic
ef00ac12fa
Use Memory
2022-04-25 11:39:07 +00:00
taiyang-li
f040c8ae65
merge master and solve conflict
2022-04-24 11:42:36 +08:00
Alexey Milovidov
6de4f2bbff
Everything related to S3 metrics was a trash
2022-04-24 03:42:18 +02:00
Kseniia Sumarokova
33bb48106f
Merge pull request #36314 from CurtizJ/print-bad-filenames
...
Show names of erroneous files in case of parsing errors while executing table functions
2022-04-22 13:24:55 +02:00
Sergei Trifonov
b371251e80
Merge branch 'master' into surahman-CH-31221-AWS-S3-object-version-id
2022-04-22 08:39:02 +02:00
Maksim Kita
57444fc7d3
Merge pull request #36444 from rschu1ze/clang-tidy-fixes
...
Clang tidy fixes
2022-04-21 16:11:27 +02:00
taiyang-li
1ea96a8732
merge master and solve conflict
2022-04-21 17:16:07 +08:00
Sergei Trifonov
34842f209c
merge master and fix conflicts
2022-04-20 18:35:56 +02:00
taiyang-li
d533b569ad
debugging
2022-04-20 19:58:31 +08:00
李扬
a1e54c3918
Fix wrong result of datetime64 when negative ( #35440 )
2022-04-20 13:11:31 +02:00
Robert Schulze
b24ca8de52
Fix various clang-tidy warnings
...
When I tried to add cool new clang-tidy 14 warnings, I noticed that the
current clang-tidy settings already produce a ton of warnings. This
commit addresses many of these. Almost all of them were non-critical,
i.e. C vs. C++ style casts.
2022-04-20 10:29:05 +02:00
Kseniia Sumarokova
f37812f89a
Merge pull request #36079 from kssenii/change-hashed-path
...
Change hashed path for cache
2022-04-20 10:27:44 +02:00
taiyang-li
fb6a56d4b0
finish debug
2022-04-20 16:24:18 +08:00
Kseniia Sumarokova
766a84e64f
Merge pull request #35915 from kssenii/better-s3-settings
...
Better settings configuration for s3
2022-04-20 10:22:06 +02:00
Anton Popov
bee4ca9b62
add more tests for error diagnostics in files
2022-04-19 15:56:34 +00:00
Anton Popov
3e361c9759
Merge remote-tracking branch 'upstream/master' into HEAD
2022-04-19 14:18:04 +00:00
Sergei Trifonov
75c1053ad7
Merge remote-tracking branch 'origin/master' into surahman-CH-31221-AWS-S3-object-version-id
2022-04-19 13:17:25 +02:00
mergify[bot]
64d5e0c8a2
Merge branch 'master' into change-hashed-path
2022-04-19 09:33:48 +00:00
kssenii
1f9a597b77
Merge master
2022-04-19 11:31:13 +02:00
taiyang-li
99ad2f5fa6
Merge remote-tracking branch 'origin/master' into async_hdfs_read_buffer
2022-04-19 15:49:38 +08:00
Robert Schulze
118e94523c
Activate clang-tidy warning "readability-container-contains"
...
This check suggests replacing <Container>.count() by
<Container>.contains() which is more speaking and in case of
multimaps/multisets also faster.
2022-04-18 23:53:11 +02:00
Alexey Milovidov
f6ab2bd523
Merge pull request #36312 from ClickHouse/remove-arcadia
...
Remove remaining parts of Arcadia
2022-04-18 07:02:54 +03:00
Alexey Milovidov
242919eddd
Remove abbreviation
2022-04-18 01:02:49 +02:00
taiyang-li
0ad2a76fae
Merge remote-tracking branch 'origin/master' into async_hdfs_read_buffer
2022-04-16 18:45:39 +08:00
taiyang-li
cd83fd5f8a
tobe debug
2022-04-16 18:41:18 +08:00
Alexey Milovidov
33ffdcaa4f
Return two changes
2022-04-16 02:18:22 +02:00
Anton Popov
2de6668b3f
show names of erroneous files
2022-04-16 00:10:47 +00:00
Alexey Milovidov
40357637ec
Remove Arcadia
2022-04-16 00:33:32 +02:00
Alexey Milovidov
cbeeb7ec4f
Remove Arcadia
2022-04-16 00:20:47 +02:00
Alexey Milovidov
416fa95b87
Remove "Arcadia" build system
2022-04-15 23:57:09 +02:00
mergify[bot]
7bad91e0f4
Merge branch 'master' into fix-parallel-read-buffer
2022-04-14 09:49:50 +00:00
Kruglov Pavel
4f9ee879d4
Merge pull request #36212 from Avogar/revert-fix
...
Revert reverting "Fix crash in ParallelReadBuffer"
2022-04-14 11:26:31 +02:00
Kseniia Sumarokova
04926d882a
Merge branch 'master' into change-hashed-path
2022-04-14 11:18:42 +02:00
Kseniia Sumarokova
4d667301cc
Merge pull request #36135 from kssenii/cache-fix
...
Fix after #35475
2022-04-14 11:11:13 +02:00
avogar
ca14a29dbe
Revert reverting "Fix crash in ParallelReadBuffer"
2022-04-13 20:39:12 +00:00
tavplubix
b2cd23658a
Revert "Fix crash in ParallelReadBuffer"
2022-04-13 22:45:34 +03:00
mergify[bot]
2c5d6d132e
Merge branch 'master' into cache-fix
2022-04-13 13:32:56 +00:00
kssenii
0912b038fa
Merge master
2022-04-13 15:30:01 +02:00
Sergei Trifonov
9d1a2da685
merge master and fix conflicts
2022-04-13 14:53:36 +02:00
kssenii
a6dc437911
Merge master
2022-04-13 14:23:48 +02:00
Kruglov Pavel
b2905c5aed
Update src/IO/ParallelReadBuffer.cpp
...
Co-authored-by: tavplubix <tavplubix@gmail.com>
2022-04-13 13:50:11 +02:00
avogar
a4d74f975e
Use atomic instead of mutex + condvar in ParallelReadBuffer
2022-04-13 10:20:29 +00:00
avogar
de9c9462e5
Fix crash in ParallelReadBuffer
2022-04-12 13:54:53 +00:00
kssenii
d2ccf876c6
Fix
2022-04-11 17:51:49 +02:00
Kseniia Sumarokova
867b85dfad
Merge branch 'master' into change-hashed-path
2022-04-09 16:57:57 +02:00
tavplubix
faae540a85
Merge pull request #24258 from ClickHouse/mvcc_prototype
...
MVCC prototype for MergeTree
2022-04-08 22:02:13 +03:00
Kseniia Sumarokova
10d609fd0d
Update WriteBufferFromS3.cpp
2022-04-08 17:42:05 +02:00
kssenii
138588be29
Change hashed path
2022-04-08 17:14:13 +02:00
Kruglov Pavel
d7b88d7683
Merge pull request #35582 from Avogar/improve-schema-inference
...
Improve schema inference and add some fixes
2022-04-08 13:44:52 +02:00
Alexander Tokmakov
49c35f3261
Merge branch 'master' into mvcc_prototype
2022-04-08 13:34:40 +02:00
kssenii
87855c4ae7
Merge master
2022-04-08 12:32:48 +02:00
kssenii
ef2b238444
Fix
2022-04-08 12:30:24 +02:00
alesapin
8ec802bc62
Merge pull request #35475 from kssenii/remote-fs-cache-improvements
...
Allow to write remote fs cache on all write operations. Add `system.remote_filesystem_cache` table. Add `drop remote filesystem cache (<path>)` query. Add `system.remote_data_paths` table.
2022-04-08 12:06:26 +02:00
kssenii
5dce2f18b5
Better
2022-04-07 18:48:35 +02:00
Sergei Trifonov
01425df758
fix build
2022-04-07 16:07:12 +02:00
Alexander Tokmakov
8290ffa88d
Merge branch 'master' into mvcc_prototype
2022-04-07 13:50:42 +02:00
avogar
d2017a63b1
Merge branch 'master' of github.com:ClickHouse/ClickHouse into improve-schema-inference
2022-04-07 11:36:40 +00:00
Sergei Trifonov
03f76dda1d
better
2022-04-07 12:03:54 +02:00
Sergei Trifonov
9c2804b924
merge from master + resolve conflicts
2022-04-07 11:22:01 +02:00
Yakov Olkhovskiy
c73115ffb4
Merge pull request #35840 from zzsmdfj/issue/#34966_fix_dateTime_deserialize
...
to #34966_fix_dateTime_deserialize
2022-04-06 08:10:23 -04:00
Kruglov Pavel
9141066de3
Merge branch 'master' into improve-schema-inference
2022-04-06 13:51:07 +02:00
Yakov Olkhovskiy
ea9ce3ea18
'T' is added as delimiter, tests added
2022-04-05 19:50:16 -04:00
Alexander Tokmakov
1fe50ad201
Merge branch 'master' into mvcc_prototype
2022-04-05 14:38:02 +02:00
Alexander Tokmakov
da00beaf7f
Merge branch 'master' into mvcc_prototype
2022-04-05 11:14:42 +02:00
Kseniia Sumarokova
77c7545395
Merge pull request #35922 from kssenii/fix-race-in-cached-buffer
...
Fix race in cached buffer
2022-04-05 10:55:58 +02:00
Alexey Milovidov
4d6c030d23
Revert "clang-tidy report issues with Medium priority"
2022-04-04 23:41:42 +03:00
kssenii
79627798c4
Fix race
2022-04-04 20:44:39 +02:00
Kseniia Sumarokova
d697576967
Update ReadBufferFromS3.h
2022-04-04 17:53:01 +02:00
kssenii
6c6fb5c3e8
Fix race in cached buffer
2022-04-04 16:32:37 +02:00
kssenii
4e50da722b
Better s3 settings
2022-04-04 16:14:56 +02:00
kssenii
36c583d0de
Better version of cache on insert
2022-04-04 10:58:59 +02:00
kssenii
6c8e073a61
Merge master
2022-04-01 16:48:29 +02:00
zzsmdfj
3cae0c74d9
to #34966_fix_dateTime_deserialize
2022-04-01 20:16:07 +08:00
zzsmdfj
ececee3817
to #34966_fix_dateTime_deserialize
2022-04-01 20:13:34 +08:00
kssenii
603dcbb1b7
Extend test
2022-03-31 18:23:15 +02:00
mergify[bot]
24ade25d61
Merge branch 'master' into improve-schema-inference
2022-03-31 13:42:47 +00:00
Alexander Tokmakov
5a50ad9de3
Merge branch 'master' into mvcc_prototype
2022-03-31 11:35:04 +02:00
kssenii
0fc92fe2aa
Get rid of all "remote" mentions
2022-03-30 17:15:29 +02:00
Antonio Andelic
a3ad99be03
Create CallbackRunner in Storage
2022-03-30 11:24:50 +00:00
Antonio Andelic
de2c6684a3
Remove setup and cleanup
2022-03-30 09:00:06 +00:00
Antonio Andelic
1173eab15e
Use same IOThreadPool for URL and S3
2022-03-30 08:15:20 +00:00
Antonio Andelic
03a09dc914
Merge pull request #35571 from ClickHouse/parallel-s3-downloading
...
Parallel S3 downloading
2022-03-30 08:14:49 +02:00
kssenii
f0e0d97752
Fix lask of query id in merges
2022-03-29 19:49:42 +02:00
Alexander Tokmakov
287d858fda
Merge branch 'master' into mvcc_prototype
2022-03-29 16:24:12 +02:00
mergify[bot]
343588de2c
Merge branch 'master' into improve-schema-inference
2022-03-29 13:06:00 +00:00
Anton Popov
d42632dc13
Merge pull request #32928 from bigo-sg/rocksdb_metacache
...
Speed up parts loading process of MergeTree by metadata cache
2022-03-29 14:05:22 +02:00
Maksim Kita
a1a4552740
Merge pull request #35184 from DevTeamBK/clang-tidy-issues
...
clang-tidy report issues with Medium priority
2022-03-29 13:19:54 +02:00
taiyang-li
834ac3a1b9
Merge remote-tracking branch 'ck/master' into rocksdb_metacache
2022-03-29 18:32:06 +08:00
taiyang-li
38702be40c
Merge remote-tracking branch 'ck/master' into rocksdb_metacache
2022-03-28 21:03:22 +08:00
Antonio Andelic
d9d826c813
Address PR review
2022-03-28 08:19:23 +00:00
Azat Khuzhin
6a3986f41b
Fix redundant-move in ReadWriteBufferFromHTTP.h
2022-03-26 23:42:46 +03:00
kssenii
d6ab6de30a
Minor changes
2022-03-24 15:32:08 +01:00
avogar
557edbd172
Add some improvements and fixes in schema inference
2022-03-24 12:54:12 +00:00
Antonio Andelic
62a495a5fc
Small refactor
2022-03-24 09:45:30 +00:00
kssenii
d4161b5925
Add optin read_from_cache_if_exists_otherwise_bypass_cache (for merges)
2022-03-23 20:24:00 +01:00
kssenii
d2a3cfe5dc
Cache on all write operations
2022-03-23 19:14:33 +01:00
kssenii
c2c7f365ce
Merge master
2022-03-23 16:58:35 +01:00
kssenii
34c0690a69
Better
2022-03-23 15:43:14 +01:00
Alexander Tokmakov
bcec53aadc
Merge branch 'master' into mvcc_prototype
2022-03-23 11:28:43 +01:00
Antonio Andelic
131b3a091c
Refactor StorageS3
2022-03-23 08:57:01 +00:00
Antonio Andelic
f32ef2a556
Small polishing for S3 reader
2022-03-23 08:15:18 +00:00
taiyang-li
8dbf1c60e7
merge master and fix conflict
2022-03-23 11:36:50 +08:00
Antonio Andelic
6785ad165a
Fix issue for mutliple download threads
2022-03-22 14:06:30 +00:00
Antonio Andelic
df99a9cc3c
Merge branch 'parallel-downloading-url-engine' into parallel-s3-downloading
2022-03-22 12:49:35 +00:00
Antonio Andelic
5da3058368
Merge branch 'master' into parallel-downloading-url-engine
2022-03-22 12:20:36 +00:00
Alexander Tokmakov
3c762f566d
Merge branch 'master' into mvcc_prototype
2022-03-21 20:16:29 +01:00
Antonio Andelic
04e6e6ab5e
Add ParallelReadBuffer for S3
2022-03-21 14:52:26 +00:00
kssenii
afd0c64a1a
Better
2022-03-21 15:04:16 +01:00
kssenii
eabbce69a7
Add system drop remote filesystem cache query, allow to write cache on insert
2022-03-21 12:32:44 +01:00
Vitaly Baranov
7a46e7a0d1
Merge branch 'master' into improve-backup-and-restore
2022-03-21 11:49:26 +01:00
Vitaly Baranov
51fc556fcd
Implement BACKUP and RESTORE for MatViews.
2022-03-21 11:45:21 +01:00
Maksim Kita
695b1300f9
Merge pull request #35383 from rschu1ze/enable-if-to-concepts-pt2
...
Replace more uses of std::enable_if for SFINAE by concepts
2022-03-21 10:47:28 +01:00
Antonio Andelic
b8c43ff2f2
Merge branch 'master' into parallel-downloading-url-engine
2022-03-21 09:11:01 +00:00
kssenii
59b7394caf
WriteSettings
2022-03-21 09:52:48 +01:00
kssenii
1d80262a2a
Add write settings, file cache settings
2022-03-21 09:50:50 +01:00
Alexander Tokmakov
9e05b12d2c
Merge branch 'master' into mvcc_prototype
2022-03-20 22:42:26 +01:00
Vitaly Baranov
c884cf901f
Use BackupEntryFromAppendOnlyFile for Log engine,
...
and use a BackupEntriesBatch for Memort engine to improve performance.
A lot of minor corrections.
2022-03-20 20:02:15 +01:00
Vitaly Baranov
a1b364626a
Improve zip archive utilities.
2022-03-20 20:02:15 +01:00
alesapin
5902304fdb
Remove some useless code from S3Common
2022-03-19 23:28:20 +01:00
kssenii
af3bb3b7af
Add WriteSettings
2022-03-18 12:27:38 +01:00
Antonio Andelic
f2a7c6c6fe
Add support for canceling a single worker
2022-03-18 09:28:56 +00:00
Robert Schulze
df73c75456
Replace more uses of std::enable_if for SFINAE by concepts
...
- enable_if is usually regarded as fragile and unreadable
- C++20 concepts are much easier to read and produce more expressive error messages
- this is follow-up to PR #35347 but this time most of the remaining and
more complex usages of enable_if in the codebase were replaced.
2022-03-17 20:44:55 +01:00
Antonio Andelic
b789a41662
Fix ASAN problems
2022-03-17 15:18:20 +00:00
Alexander Tokmakov
d04dc03fa4
Merge branch 'master' into mvcc_prototype
2022-03-17 15:24:32 +01:00
Antonio Andelic
103a3fa140
Merge branch 'master' into parallel-downloading-url-engine
2022-03-17 10:11:58 +00:00
Antonio Andelic
44de5a581c
Fix split build
2022-03-17 10:09:16 +00:00
alesapin
457fa0d1ba
Merge pull request #23932 from CurtizJ/dynamic-columns
...
Support of dynamic subcolumns (JSON data type)
2022-03-17 10:45:46 +01:00
Maksim Kita
4d8c2b2009
Merge pull request #35347 from rschu1ze/enable-if-to-concepts
...
Replace a few uses of enable_if for SFINAE by concepts
2022-03-17 10:34:08 +01:00
Alexander Tokmakov
4b3e13a4fe
Merge branch 'master' into mvcc_prototype
2022-03-16 21:06:19 +01:00
Alexander Tokmakov
c2ac8d4a5c
review fixes
2022-03-16 21:05:34 +01:00
Robert Schulze
b56c28d841
Replace a few uses of enable_if for SFINAE by concepts
...
- enable_if is usually regarded as fragile and unreadable
- C++20 concepts are much easier to read and produce more expressive
error messages
2022-03-16 19:51:38 +01:00
Antonio Andelic
0979155f2f
Address PR comments
2022-03-16 15:45:17 +00:00
Anton Popov
0ba78c3c3a
Merge remote-tracking branch 'upstream/master' into HEAD
2022-03-16 15:28:09 +00:00
Antonio Andelic
84f86bf3da
Limit number of bytes read per worker
2022-03-16 12:34:14 +00:00
Antonio Andelic
2291b7c1e1
Merge branch 'master' into parallel-downloading-url-engine
2022-03-16 09:59:52 +00:00
Antonio Andelic
3465664871
Attach thread pool threads to counters
2022-03-16 09:26:01 +00:00
Kruglov Pavel
2b7e89e171
Merge pull request #35300 from Avogar/fix-bzip2
...
Fix possible assertion while using bzip2 compression
2022-03-15 22:17:37 +01:00
Kruglov Pavel
20e17a6d3e
Merge pull request #35296 from Avogar/fix-lz4
...
Fix possible segfault while using lz4 compression
2022-03-15 22:17:14 +01:00
Kruglov Pavel
6fd1a8281f
Merge pull request #35295 from Avogar/fix-lzma
...
Fix possible Assertion failed while using lzma compression
2022-03-15 22:14:35 +01:00
Nikolai Kochetov
a380aa6b8a
Merge pull request #35294 from ClickHouse/reload-remote_url_allow_hosts
...
Reload remote_url_allow_hosts after config update.
2022-03-15 22:07:16 +01:00
Saad Ur Rahman
275bf0ef81
[IO] fixed bug in versionId extraction introduced during rebase.
2022-03-15 12:13:44 -04:00
avogar
ec3c5f86ad
Fix possible assertion while using bzip2 compression
2022-03-15 14:23:52 +00:00
avogar
0cfaf48e1d
Fix possible segfault while using lz4 compression
2022-03-15 13:46:14 +00:00
Alexander Tokmakov
1f571b7734
Merge branch 'master' into mvcc_prototype
2022-03-15 14:45:06 +01:00
Kruglov Pavel
24ac217be3
Remove redundant include
2022-03-15 14:07:31 +01:00
avogar
90ce1709d7
Fix possible Assertion failed while using lzma compression
2022-03-15 13:00:53 +00:00
Nikolai Kochetov
97aa6c82ce
Reload remote_url_allow_hosts after config update.
2022-03-15 13:00:31 +00:00
mergify[bot]
d613ad9445
Merge branch 'master' into fix-brotli
2022-03-15 08:25:12 +00:00
Antonio Andelic
067b79b00b
Merge branch 'master' into parallel-downloading-url-engine
2022-03-15 07:55:41 +00:00
Saad Ur Rahman
7b35c79b04
[IO] versionId extraction simplified.
...
Reducing cognitive load.
2022-03-14 22:27:58 -04:00
Saad Ur Rahman
d9b370cf1d
[IO] S3 URI versionId extraction performance tweak.
2022-03-14 22:17:50 -04:00
Saad Ur Rahman
1a85e9b60d
[IO] Extracting Version ID in S3::URI
...
Extracting Version ID form query string in URI.
2022-03-14 22:16:24 -04:00
Saad Ur Rahman
ec28af055f
[IO] S3 URI data struct contains Version ID
...
Added Version ID field to S3::URI struct.
TODO: Extract version id if present.
2022-03-14 22:11:18 -04:00
Saad Ur Rahman
c7402c2a39
[IO] Version ID provided during buffer read
...
The Version ID, if provided, is to be set in the request to read the buffer from S3. If not provided the latest version of the object must be retrieved.
Version ID set to last parameter to allow defaulting and backward compatibility with calls.
2022-03-14 20:10:14 -04:00
Alexander Tokmakov
9702b5177d
Merge branch 'master' into mvcc_prototype
2022-03-14 21:45:38 +01:00
avogar
4b12c6e6b3
Fix possible Abort while using Brotli compression
2022-03-14 18:58:05 +00:00
Maksim Kita
e30117a3d6
Fix clang-tidy warnings in Interpreters, IO folders
2022-03-14 18:17:35 +00:00
Anton Popov
36ec379aeb
Merge remote-tracking branch 'upstream/master' into HEAD
2022-03-14 16:28:35 +00:00
Antonio Andelic
5f8d9885b3
Ignore read bytes
2022-03-14 15:50:30 +00:00
Azat Khuzhin
9f7fae56b0
WriteBufferFromS3: remove unused headers
...
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-03-14 12:07:47 +03:00
Antonio Andelic
f5d3a8a31d
Polishing
2022-03-11 13:38:19 +00:00
Antonio Andelic
28e9508c4e
Improve shared pool and add settings
2022-03-11 08:33:34 +00:00
Antonio Andelic
cce318273b
Use a shared IO thread pool
2022-03-11 08:33:34 +00:00
Antonio Andelic
adaaaaf061
Unblock readers before stoping
2022-03-11 08:33:34 +00:00
Antonio Andelic
df0f5e20d0
Refactor
2022-03-11 08:33:34 +00:00
Antonio Andelic
7b526b25a8
Initial implementation with Arena
2022-03-11 08:33:34 +00:00
Antonio Andelic
16c20332cf
Fix deadlocks
2022-03-11 08:33:34 +00:00
Antonio Andelic
c6664bbbdf
Add seekable ParallelReadBuffer
2022-03-11 08:33:34 +00:00
Antonio Andelic
29c32ed831
Refactor code
2022-03-11 08:33:34 +00:00
Antonio Andelic
0a1a3a230e
Add support for parallel http range requests
2022-03-11 08:33:34 +00:00
Antonio Andelic
aad87879cf
Small refactor
2022-03-11 08:33:34 +00:00
Kseniia Sumarokova
818459b9f0
Merge pull request #33717 from kssenii/local-cache-for-remote-fs
...
Local cache for remote filesystem
2022-03-11 07:23:10 +01:00
Rajkumar
137e4b79c1
override needed
2022-03-10 12:46:32 -08:00
Kseniia Sumarokova
5149bd2d29
Merge pull request #35172 from kssenii/add-non-retriable-errors-for-url
...
Add non-retriable errors check for HTTP read buffer
2022-03-10 20:28:37 +01:00
Rajkumar
0de950c3f8
style correction
2022-03-10 10:29:57 -08:00
Rajkumar
81ad3ca220
syle check
2022-03-10 10:19:03 -08:00
Rajkumar
a17e5efea2
adding destructors
2022-03-10 07:31:06 -08:00
Alexander Tokmakov
061fa6a6f2
Merge branch 'master' into mvcc_prototype
2022-03-10 13:13:04 +01:00
kssenii
050802b99b
Add non-retriable errors
2022-03-10 11:38:46 +01:00
lgbo-ustc
fdd423a3da
fixed code style
2022-03-10 12:13:19 +08:00
lgbo-ustc
e4883f31b7
update tests
...
1. fixed code style in src/IO/tests/gtest_hadoop_snappy_decoder.cpp
2. enable tests 01060_avro.sh
2022-03-10 09:46:43 +08:00
lgbo-ustc
8dc8c87fd1
add a test case
2022-03-09 10:03:04 +08:00
kssenii
5260822964
Merge master
2022-03-08 18:21:28 +01:00
kssenii
e231c3a3e0
Fix split build
2022-03-08 18:05:55 +01:00
kssenii
f3cba6f270
Remove already fixed crutch
2022-03-08 14:30:21 +01:00
lgbo-ustc
7f89a1bcf3
add some usage test
2022-03-08 20:00:39 +08:00
lgbo-ustc
225b0bd914
fixed bug: call need_more_input repeatly, overwrite the buffer
2022-03-08 17:17:06 +08:00
taiyang-li
b4174b0bef
merge master and fix conflicts
2022-03-08 11:39:25 +08:00
kssenii
0737aef472
Fix split build
2022-03-07 22:02:07 +01:00
Alexander Tokmakov
8acfb8d27f
Merge branch 'master' into mvcc_prototype
2022-03-07 17:40:15 +01:00
kssenii
eeb0ae9816
Fix detaching query
2022-03-07 17:05:36 +01:00
Anton Popov
0bc57da238
Merge remote-tracking branch 'upstream/master' into HEAD
2022-03-07 14:46:08 +00:00
kssenii
692b247379
Fix some corner cases
2022-03-07 14:31:21 +01:00
kssenii
2dcfe66236
Fix race
2022-03-06 20:33:07 +01:00
Anton Popov
df3b07fe7c
Merge remote-tracking branch 'upstream/master' into HEAD
2022-03-03 22:25:28 +00:00
vdimir
d3ee3dee85
Catch exception in ~WriteBufferFromS3
2022-03-03 12:54:24 +00:00
Maksim Kita
b1a956c5f1
clang-tidy check performance-move-const-arg fix
2022-03-02 18:15:27 +00:00
Maksim Kita
1f5837359e
clang-tidy check performance-noexcept-move-constructor fix
2022-03-02 18:15:27 +00:00
kssenii
d5952109fb
Merge master
2022-03-02 18:15:25 +01:00
HaiBo Li
2ea18b3085
Fix the bug that the toDatetime function overflows ( #32898 )
...
* Fixed overflow bug of the function toDatetime/toDate/toDate32/parseDateTimeBestEffort
2022-03-02 10:06:38 +03:00
Anton Popov
2758db5341
add more comments
2022-03-01 19:32:55 +03:00
Danila Kutenin
3ce71f211a
First portion of force inlines
2022-02-27 10:43:17 +00:00
Anton Popov
fcdebea925
Merge remote-tracking branch 'upstream/master' into HEAD
2022-02-25 13:41:30 +03:00
Alexander Tokmakov
aa6b9a2abc
Merge branch 'master' into mvcc_prototype
2022-02-23 23:22:03 +03:00
kssenii
c637385dd0
Merge master
2022-02-22 13:17:51 +01:00
Kseniia Sumarokova
eeea322556
Merge pull request #34629 from amosbird/remotefsimprove
...
Some refactoring and improvement over async and remote buffer related stuff
2022-02-22 11:36:40 +01:00
kssenii
fad7c042c1
Fix cache with non-async reads
2022-02-21 19:50:27 +01:00
Amos Bird
01652d9892
Add more assertion
2022-02-21 22:42:43 +08:00
kssenii
4543513a5d
Add comments
2022-02-21 13:54:03 +01:00
Maksim Kita
883c7c21d1
Merge pull request #34690 from nvartolomei/nv/always-increment
...
Always update ProfileEvents (even on exceptions)
2022-02-21 00:03:44 +01:00
Maksim Kita
e049f6ada9
Update WriteBufferFromPocoSocket.cpp
2022-02-20 16:59:39 +01:00
Maksim Kita
d44f48fe1f
Update ReadBufferFromPocoSocket.cpp
2022-02-20 16:59:17 +01:00
kssenii
e8b2e47e93
Style
2022-02-19 20:38:23 +01:00
Amos Bird
84d111cde9
Better ReadBuffer asynchronicity
2022-02-20 01:08:21 +08:00
