Commit Graph

1622 Commits

Author SHA1 Message Date
Antonio Andelic
6785ad165a Fix issue for mutliple download threads 2022-03-22 14:06:30 +00:00
Antonio Andelic
df99a9cc3c Merge branch 'parallel-downloading-url-engine' into parallel-s3-downloading 2022-03-22 12:49:35 +00:00
Antonio Andelic
5da3058368 Merge branch 'master' into parallel-downloading-url-engine 2022-03-22 12:20:36 +00:00
Alexander Tokmakov
3c762f566d Merge branch 'master' into mvcc_prototype 2022-03-21 20:16:29 +01:00
Antonio Andelic
04e6e6ab5e Add ParallelReadBuffer for S3 2022-03-21 14:52:26 +00:00
kssenii
afd0c64a1a Better 2022-03-21 15:04:16 +01:00
kssenii
eabbce69a7 Add system drop remote filesystem cache query, allow to write cache on insert 2022-03-21 12:32:44 +01:00
Vitaly Baranov
7a46e7a0d1
Merge branch 'master' into improve-backup-and-restore 2022-03-21 11:49:26 +01:00
Vitaly Baranov
51fc556fcd Implement BACKUP and RESTORE for MatViews. 2022-03-21 11:45:21 +01:00
Maksim Kita
695b1300f9
Merge pull request #35383 from rschu1ze/enable-if-to-concepts-pt2
Replace more uses of std::enable_if for SFINAE by concepts
2022-03-21 10:47:28 +01:00
Antonio Andelic
b8c43ff2f2 Merge branch 'master' into parallel-downloading-url-engine 2022-03-21 09:11:01 +00:00
kssenii
59b7394caf WriteSettings 2022-03-21 09:52:48 +01:00
kssenii
1d80262a2a Add write settings, file cache settings 2022-03-21 09:50:50 +01:00
Alexander Tokmakov
9e05b12d2c Merge branch 'master' into mvcc_prototype 2022-03-20 22:42:26 +01:00
Vitaly Baranov
c884cf901f Use BackupEntryFromAppendOnlyFile for Log engine,
and use a BackupEntriesBatch for Memort engine to improve performance.
A lot of minor corrections.
2022-03-20 20:02:15 +01:00
Vitaly Baranov
a1b364626a Improve zip archive utilities. 2022-03-20 20:02:15 +01:00
alesapin
5902304fdb Remove some useless code from S3Common 2022-03-19 23:28:20 +01:00
kssenii
af3bb3b7af Add WriteSettings 2022-03-18 12:27:38 +01:00
Antonio Andelic
f2a7c6c6fe Add support for canceling a single worker 2022-03-18 09:28:56 +00:00
Robert Schulze
df73c75456
Replace more uses of std::enable_if for SFINAE by concepts
- enable_if is usually regarded as fragile and unreadable
- C++20 concepts are much easier to read and produce more expressive error messages

- this is follow-up to PR #35347 but this time most of the remaining and
  more complex usages of enable_if in the codebase were replaced.
2022-03-17 20:44:55 +01:00
Antonio Andelic
b789a41662 Fix ASAN problems 2022-03-17 15:18:20 +00:00
Alexander Tokmakov
d04dc03fa4 Merge branch 'master' into mvcc_prototype 2022-03-17 15:24:32 +01:00
Antonio Andelic
103a3fa140 Merge branch 'master' into parallel-downloading-url-engine 2022-03-17 10:11:58 +00:00
Antonio Andelic
44de5a581c Fix split build 2022-03-17 10:09:16 +00:00
alesapin
457fa0d1ba
Merge pull request #23932 from CurtizJ/dynamic-columns
Support of dynamic subcolumns (JSON data type)
2022-03-17 10:45:46 +01:00
Maksim Kita
4d8c2b2009
Merge pull request #35347 from rschu1ze/enable-if-to-concepts
Replace a few uses of enable_if for SFINAE by concepts
2022-03-17 10:34:08 +01:00
Alexander Tokmakov
4b3e13a4fe Merge branch 'master' into mvcc_prototype 2022-03-16 21:06:19 +01:00
Alexander Tokmakov
c2ac8d4a5c review fixes 2022-03-16 21:05:34 +01:00
Robert Schulze
b56c28d841
Replace a few uses of enable_if for SFINAE by concepts
- enable_if is usually regarded as fragile and unreadable

- C++20 concepts are much easier to read and produce more expressive
  error messages
2022-03-16 19:51:38 +01:00
Antonio Andelic
0979155f2f Address PR comments 2022-03-16 15:45:17 +00:00
Anton Popov
0ba78c3c3a Merge remote-tracking branch 'upstream/master' into HEAD 2022-03-16 15:28:09 +00:00
Antonio Andelic
84f86bf3da Limit number of bytes read per worker 2022-03-16 12:34:14 +00:00
Antonio Andelic
2291b7c1e1 Merge branch 'master' into parallel-downloading-url-engine 2022-03-16 09:59:52 +00:00
Antonio Andelic
3465664871 Attach thread pool threads to counters 2022-03-16 09:26:01 +00:00
Kruglov Pavel
2b7e89e171
Merge pull request #35300 from Avogar/fix-bzip2
Fix possible assertion while using bzip2 compression
2022-03-15 22:17:37 +01:00
Kruglov Pavel
20e17a6d3e
Merge pull request #35296 from Avogar/fix-lz4
Fix possible segfault while using lz4 compression
2022-03-15 22:17:14 +01:00
Kruglov Pavel
6fd1a8281f
Merge pull request #35295 from Avogar/fix-lzma
Fix possible Assertion failed while using lzma compression
2022-03-15 22:14:35 +01:00
Nikolai Kochetov
a380aa6b8a
Merge pull request #35294 from ClickHouse/reload-remote_url_allow_hosts
Reload remote_url_allow_hosts after config update.
2022-03-15 22:07:16 +01:00
avogar
ec3c5f86ad Fix possible assertion while using bzip2 compression 2022-03-15 14:23:52 +00:00
avogar
0cfaf48e1d Fix possible segfault while using lz4 compression 2022-03-15 13:46:14 +00:00
Alexander Tokmakov
1f571b7734 Merge branch 'master' into mvcc_prototype 2022-03-15 14:45:06 +01:00
Kruglov Pavel
24ac217be3
Remove redundant include 2022-03-15 14:07:31 +01:00
avogar
90ce1709d7 Fix possible Assertion failed while using lzma compression 2022-03-15 13:00:53 +00:00
Nikolai Kochetov
97aa6c82ce Reload remote_url_allow_hosts after config update. 2022-03-15 13:00:31 +00:00
mergify[bot]
d613ad9445
Merge branch 'master' into fix-brotli 2022-03-15 08:25:12 +00:00
Antonio Andelic
067b79b00b Merge branch 'master' into parallel-downloading-url-engine 2022-03-15 07:55:41 +00:00
Alexander Tokmakov
9702b5177d Merge branch 'master' into mvcc_prototype 2022-03-14 21:45:38 +01:00
avogar
4b12c6e6b3 Fix possible Abort while using Brotli compression 2022-03-14 18:58:05 +00:00
Maksim Kita
e30117a3d6 Fix clang-tidy warnings in Interpreters, IO folders 2022-03-14 18:17:35 +00:00
Anton Popov
36ec379aeb Merge remote-tracking branch 'upstream/master' into HEAD 2022-03-14 16:28:35 +00:00
Antonio Andelic
5f8d9885b3 Ignore read bytes 2022-03-14 15:50:30 +00:00
Azat Khuzhin
9f7fae56b0 WriteBufferFromS3: remove unused headers
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-03-14 12:07:47 +03:00
Antonio Andelic
f5d3a8a31d Polishing 2022-03-11 13:38:19 +00:00
Antonio Andelic
28e9508c4e Improve shared pool and add settings 2022-03-11 08:33:34 +00:00
Antonio Andelic
cce318273b Use a shared IO thread pool 2022-03-11 08:33:34 +00:00
Antonio Andelic
adaaaaf061 Unblock readers before stoping 2022-03-11 08:33:34 +00:00
Antonio Andelic
df0f5e20d0 Refactor 2022-03-11 08:33:34 +00:00
Antonio Andelic
7b526b25a8 Initial implementation with Arena 2022-03-11 08:33:34 +00:00
Antonio Andelic
16c20332cf Fix deadlocks 2022-03-11 08:33:34 +00:00
Antonio Andelic
c6664bbbdf Add seekable ParallelReadBuffer 2022-03-11 08:33:34 +00:00
Antonio Andelic
29c32ed831 Refactor code 2022-03-11 08:33:34 +00:00
Antonio Andelic
0a1a3a230e Add support for parallel http range requests 2022-03-11 08:33:34 +00:00
Antonio Andelic
aad87879cf Small refactor 2022-03-11 08:33:34 +00:00
Kseniia Sumarokova
818459b9f0
Merge pull request #33717 from kssenii/local-cache-for-remote-fs
Local cache for remote filesystem
2022-03-11 07:23:10 +01:00
Rajkumar
137e4b79c1 override needed 2022-03-10 12:46:32 -08:00
Kseniia Sumarokova
5149bd2d29
Merge pull request #35172 from kssenii/add-non-retriable-errors-for-url
Add non-retriable errors check for HTTP read buffer
2022-03-10 20:28:37 +01:00
Rajkumar
0de950c3f8 style correction 2022-03-10 10:29:57 -08:00
Rajkumar
81ad3ca220 syle check 2022-03-10 10:19:03 -08:00
Rajkumar
a17e5efea2 adding destructors 2022-03-10 07:31:06 -08:00
Alexander Tokmakov
061fa6a6f2 Merge branch 'master' into mvcc_prototype 2022-03-10 13:13:04 +01:00
kssenii
050802b99b Add non-retriable errors 2022-03-10 11:38:46 +01:00
lgbo-ustc
fdd423a3da fixed code style 2022-03-10 12:13:19 +08:00
lgbo-ustc
e4883f31b7 update tests
1. fixed code style in src/IO/tests/gtest_hadoop_snappy_decoder.cpp
2. enable tests 01060_avro.sh
2022-03-10 09:46:43 +08:00
lgbo-ustc
8dc8c87fd1 add a test case 2022-03-09 10:03:04 +08:00
kssenii
5260822964 Merge master 2022-03-08 18:21:28 +01:00
kssenii
e231c3a3e0 Fix split build 2022-03-08 18:05:55 +01:00
kssenii
f3cba6f270 Remove already fixed crutch 2022-03-08 14:30:21 +01:00
lgbo-ustc
7f89a1bcf3 add some usage test 2022-03-08 20:00:39 +08:00
lgbo-ustc
225b0bd914 fixed bug: call need_more_input repeatly, overwrite the buffer 2022-03-08 17:17:06 +08:00
taiyang-li
b4174b0bef merge master and fix conflicts 2022-03-08 11:39:25 +08:00
kssenii
0737aef472 Fix split build 2022-03-07 22:02:07 +01:00
Alexander Tokmakov
8acfb8d27f Merge branch 'master' into mvcc_prototype 2022-03-07 17:40:15 +01:00
kssenii
eeb0ae9816 Fix detaching query 2022-03-07 17:05:36 +01:00
Anton Popov
0bc57da238 Merge remote-tracking branch 'upstream/master' into HEAD 2022-03-07 14:46:08 +00:00
kssenii
692b247379 Fix some corner cases 2022-03-07 14:31:21 +01:00
kssenii
2dcfe66236 Fix race 2022-03-06 20:33:07 +01:00
Anton Popov
df3b07fe7c Merge remote-tracking branch 'upstream/master' into HEAD 2022-03-03 22:25:28 +00:00
vdimir
d3ee3dee85
Catch exception in ~WriteBufferFromS3 2022-03-03 12:54:24 +00:00
Maksim Kita
b1a956c5f1 clang-tidy check performance-move-const-arg fix 2022-03-02 18:15:27 +00:00
Maksim Kita
1f5837359e clang-tidy check performance-noexcept-move-constructor fix 2022-03-02 18:15:27 +00:00
kssenii
d5952109fb Merge master 2022-03-02 18:15:25 +01:00
HaiBo Li
2ea18b3085
Fix the bug that the toDatetime function overflows (#32898)
* Fixed overflow bug of the function toDatetime/toDate/toDate32/parseDateTimeBestEffort
2022-03-02 10:06:38 +03:00
Anton Popov
2758db5341 add more comments 2022-03-01 19:32:55 +03:00
Anton Popov
fcdebea925 Merge remote-tracking branch 'upstream/master' into HEAD 2022-02-25 13:41:30 +03:00
Alexander Tokmakov
aa6b9a2abc Merge branch 'master' into mvcc_prototype 2022-02-23 23:22:03 +03:00
kssenii
c637385dd0 Merge master 2022-02-22 13:17:51 +01:00
Kseniia Sumarokova
eeea322556
Merge pull request #34629 from amosbird/remotefsimprove
Some refactoring and improvement over async and remote buffer related stuff
2022-02-22 11:36:40 +01:00
kssenii
fad7c042c1 Fix cache with non-async reads 2022-02-21 19:50:27 +01:00
Amos Bird
01652d9892
Add more assertion 2022-02-21 22:42:43 +08:00
kssenii
4543513a5d Add comments 2022-02-21 13:54:03 +01:00