Anton Popov
bee4ca9b62
add more tests for error diagnostics in files
2022-04-19 15:56:34 +00:00
Anton Popov
3e361c9759
Merge remote-tracking branch 'upstream/master' into HEAD
2022-04-19 14:18:04 +00:00
Sergei Trifonov
75c1053ad7
Merge remote-tracking branch 'origin/master' into surahman-CH-31221-AWS-S3-object-version-id
2022-04-19 13:17:25 +02:00
mergify[bot]
64d5e0c8a2
Merge branch 'master' into change-hashed-path
2022-04-19 09:33:48 +00:00
kssenii
1f9a597b77
Merge master
2022-04-19 11:31:13 +02:00
taiyang-li
99ad2f5fa6
Merge remote-tracking branch 'origin/master' into async_hdfs_read_buffer
2022-04-19 15:49:38 +08:00
Robert Schulze
118e94523c
Activate clang-tidy warning "readability-container-contains"
...
This check suggests replacing <Container>.count() by
<Container>.contains() which is more speaking and in case of
multimaps/multisets also faster.
2022-04-18 23:53:11 +02:00
Alexey Milovidov
f6ab2bd523
Merge pull request #36312 from ClickHouse/remove-arcadia
...
Remove remaining parts of Arcadia
2022-04-18 07:02:54 +03:00
Alexey Milovidov
242919eddd
Remove abbreviation
2022-04-18 01:02:49 +02:00
taiyang-li
0ad2a76fae
Merge remote-tracking branch 'origin/master' into async_hdfs_read_buffer
2022-04-16 18:45:39 +08:00
taiyang-li
cd83fd5f8a
tobe debug
2022-04-16 18:41:18 +08:00
Alexey Milovidov
33ffdcaa4f
Return two changes
2022-04-16 02:18:22 +02:00
Anton Popov
2de6668b3f
show names of erroneous files
2022-04-16 00:10:47 +00:00
Alexey Milovidov
40357637ec
Remove Arcadia
2022-04-16 00:33:32 +02:00
Alexey Milovidov
cbeeb7ec4f
Remove Arcadia
2022-04-16 00:20:47 +02:00
Alexey Milovidov
416fa95b87
Remove "Arcadia" build system
2022-04-15 23:57:09 +02:00
mergify[bot]
7bad91e0f4
Merge branch 'master' into fix-parallel-read-buffer
2022-04-14 09:49:50 +00:00
Kruglov Pavel
4f9ee879d4
Merge pull request #36212 from Avogar/revert-fix
...
Revert reverting "Fix crash in ParallelReadBuffer"
2022-04-14 11:26:31 +02:00
Kseniia Sumarokova
04926d882a
Merge branch 'master' into change-hashed-path
2022-04-14 11:18:42 +02:00
Kseniia Sumarokova
4d667301cc
Merge pull request #36135 from kssenii/cache-fix
...
Fix after #35475
2022-04-14 11:11:13 +02:00
avogar
ca14a29dbe
Revert reverting "Fix crash in ParallelReadBuffer"
2022-04-13 20:39:12 +00:00
tavplubix
b2cd23658a
Revert "Fix crash in ParallelReadBuffer"
2022-04-13 22:45:34 +03:00
mergify[bot]
2c5d6d132e
Merge branch 'master' into cache-fix
2022-04-13 13:32:56 +00:00
kssenii
0912b038fa
Merge master
2022-04-13 15:30:01 +02:00
Sergei Trifonov
9d1a2da685
merge master and fix conflicts
2022-04-13 14:53:36 +02:00
kssenii
a6dc437911
Merge master
2022-04-13 14:23:48 +02:00
Kruglov Pavel
b2905c5aed
Update src/IO/ParallelReadBuffer.cpp
...
Co-authored-by: tavplubix <tavplubix@gmail.com>
2022-04-13 13:50:11 +02:00
avogar
a4d74f975e
Use atomic instead of mutex + condvar in ParallelReadBuffer
2022-04-13 10:20:29 +00:00
avogar
de9c9462e5
Fix crash in ParallelReadBuffer
2022-04-12 13:54:53 +00:00
kssenii
d2ccf876c6
Fix
2022-04-11 17:51:49 +02:00
Kseniia Sumarokova
867b85dfad
Merge branch 'master' into change-hashed-path
2022-04-09 16:57:57 +02:00
tavplubix
faae540a85
Merge pull request #24258 from ClickHouse/mvcc_prototype
...
MVCC prototype for MergeTree
2022-04-08 22:02:13 +03:00
Kseniia Sumarokova
10d609fd0d
Update WriteBufferFromS3.cpp
2022-04-08 17:42:05 +02:00
kssenii
138588be29
Change hashed path
2022-04-08 17:14:13 +02:00
Kruglov Pavel
d7b88d7683
Merge pull request #35582 from Avogar/improve-schema-inference
...
Improve schema inference and add some fixes
2022-04-08 13:44:52 +02:00
Alexander Tokmakov
49c35f3261
Merge branch 'master' into mvcc_prototype
2022-04-08 13:34:40 +02:00
kssenii
87855c4ae7
Merge master
2022-04-08 12:32:48 +02:00
kssenii
ef2b238444
Fix
2022-04-08 12:30:24 +02:00
alesapin
8ec802bc62
Merge pull request #35475 from kssenii/remote-fs-cache-improvements
...
Allow to write remote fs cache on all write operations. Add `system.remote_filesystem_cache` table. Add `drop remote filesystem cache (<path>)` query. Add `system.remote_data_paths` table.
2022-04-08 12:06:26 +02:00
kssenii
5dce2f18b5
Better
2022-04-07 18:48:35 +02:00
Sergei Trifonov
01425df758
fix build
2022-04-07 16:07:12 +02:00
Alexander Tokmakov
8290ffa88d
Merge branch 'master' into mvcc_prototype
2022-04-07 13:50:42 +02:00
avogar
d2017a63b1
Merge branch 'master' of github.com:ClickHouse/ClickHouse into improve-schema-inference
2022-04-07 11:36:40 +00:00
Sergei Trifonov
03f76dda1d
better
2022-04-07 12:03:54 +02:00
Sergei Trifonov
9c2804b924
merge from master + resolve conflicts
2022-04-07 11:22:01 +02:00
Yakov Olkhovskiy
c73115ffb4
Merge pull request #35840 from zzsmdfj/issue/#34966_fix_dateTime_deserialize
...
to #34966_fix_dateTime_deserialize
2022-04-06 08:10:23 -04:00
Kruglov Pavel
9141066de3
Merge branch 'master' into improve-schema-inference
2022-04-06 13:51:07 +02:00
Yakov Olkhovskiy
ea9ce3ea18
'T' is added as delimiter, tests added
2022-04-05 19:50:16 -04:00
Alexander Tokmakov
1fe50ad201
Merge branch 'master' into mvcc_prototype
2022-04-05 14:38:02 +02:00
Alexander Tokmakov
da00beaf7f
Merge branch 'master' into mvcc_prototype
2022-04-05 11:14:42 +02:00
Kseniia Sumarokova
77c7545395
Merge pull request #35922 from kssenii/fix-race-in-cached-buffer
...
Fix race in cached buffer
2022-04-05 10:55:58 +02:00
Alexey Milovidov
4d6c030d23
Revert "clang-tidy report issues with Medium priority"
2022-04-04 23:41:42 +03:00
kssenii
79627798c4
Fix race
2022-04-04 20:44:39 +02:00
Kseniia Sumarokova
d697576967
Update ReadBufferFromS3.h
2022-04-04 17:53:01 +02:00
kssenii
6c6fb5c3e8
Fix race in cached buffer
2022-04-04 16:32:37 +02:00
kssenii
4e50da722b
Better s3 settings
2022-04-04 16:14:56 +02:00
kssenii
36c583d0de
Better version of cache on insert
2022-04-04 10:58:59 +02:00
kssenii
6c8e073a61
Merge master
2022-04-01 16:48:29 +02:00
zzsmdfj
3cae0c74d9
to #34966_fix_dateTime_deserialize
2022-04-01 20:16:07 +08:00
zzsmdfj
ececee3817
to #34966_fix_dateTime_deserialize
2022-04-01 20:13:34 +08:00
kssenii
603dcbb1b7
Extend test
2022-03-31 18:23:15 +02:00
mergify[bot]
24ade25d61
Merge branch 'master' into improve-schema-inference
2022-03-31 13:42:47 +00:00
Alexander Tokmakov
5a50ad9de3
Merge branch 'master' into mvcc_prototype
2022-03-31 11:35:04 +02:00
kssenii
0fc92fe2aa
Get rid of all "remote" mentions
2022-03-30 17:15:29 +02:00
Antonio Andelic
a3ad99be03
Create CallbackRunner in Storage
2022-03-30 11:24:50 +00:00
Antonio Andelic
de2c6684a3
Remove setup and cleanup
2022-03-30 09:00:06 +00:00
Antonio Andelic
1173eab15e
Use same IOThreadPool for URL and S3
2022-03-30 08:15:20 +00:00
Antonio Andelic
03a09dc914
Merge pull request #35571 from ClickHouse/parallel-s3-downloading
...
Parallel S3 downloading
2022-03-30 08:14:49 +02:00
kssenii
f0e0d97752
Fix lask of query id in merges
2022-03-29 19:49:42 +02:00
Alexander Tokmakov
287d858fda
Merge branch 'master' into mvcc_prototype
2022-03-29 16:24:12 +02:00
mergify[bot]
343588de2c
Merge branch 'master' into improve-schema-inference
2022-03-29 13:06:00 +00:00
Anton Popov
d42632dc13
Merge pull request #32928 from bigo-sg/rocksdb_metacache
...
Speed up parts loading process of MergeTree by metadata cache
2022-03-29 14:05:22 +02:00
Maksim Kita
a1a4552740
Merge pull request #35184 from DevTeamBK/clang-tidy-issues
...
clang-tidy report issues with Medium priority
2022-03-29 13:19:54 +02:00
taiyang-li
834ac3a1b9
Merge remote-tracking branch 'ck/master' into rocksdb_metacache
2022-03-29 18:32:06 +08:00
taiyang-li
38702be40c
Merge remote-tracking branch 'ck/master' into rocksdb_metacache
2022-03-28 21:03:22 +08:00
Antonio Andelic
d9d826c813
Address PR review
2022-03-28 08:19:23 +00:00
Azat Khuzhin
6a3986f41b
Fix redundant-move in ReadWriteBufferFromHTTP.h
2022-03-26 23:42:46 +03:00
kssenii
d6ab6de30a
Minor changes
2022-03-24 15:32:08 +01:00
avogar
557edbd172
Add some improvements and fixes in schema inference
2022-03-24 12:54:12 +00:00
Antonio Andelic
62a495a5fc
Small refactor
2022-03-24 09:45:30 +00:00
kssenii
d4161b5925
Add optin read_from_cache_if_exists_otherwise_bypass_cache (for merges)
2022-03-23 20:24:00 +01:00
kssenii
d2a3cfe5dc
Cache on all write operations
2022-03-23 19:14:33 +01:00
kssenii
c2c7f365ce
Merge master
2022-03-23 16:58:35 +01:00
kssenii
34c0690a69
Better
2022-03-23 15:43:14 +01:00
Alexander Tokmakov
bcec53aadc
Merge branch 'master' into mvcc_prototype
2022-03-23 11:28:43 +01:00
Antonio Andelic
131b3a091c
Refactor StorageS3
2022-03-23 08:57:01 +00:00
Antonio Andelic
f32ef2a556
Small polishing for S3 reader
2022-03-23 08:15:18 +00:00
taiyang-li
8dbf1c60e7
merge master and fix conflict
2022-03-23 11:36:50 +08:00
Antonio Andelic
6785ad165a
Fix issue for mutliple download threads
2022-03-22 14:06:30 +00:00
Antonio Andelic
df99a9cc3c
Merge branch 'parallel-downloading-url-engine' into parallel-s3-downloading
2022-03-22 12:49:35 +00:00
Antonio Andelic
5da3058368
Merge branch 'master' into parallel-downloading-url-engine
2022-03-22 12:20:36 +00:00
Alexander Tokmakov
3c762f566d
Merge branch 'master' into mvcc_prototype
2022-03-21 20:16:29 +01:00
Antonio Andelic
04e6e6ab5e
Add ParallelReadBuffer for S3
2022-03-21 14:52:26 +00:00
kssenii
afd0c64a1a
Better
2022-03-21 15:04:16 +01:00
kssenii
eabbce69a7
Add system drop remote filesystem cache query, allow to write cache on insert
2022-03-21 12:32:44 +01:00
Vitaly Baranov
7a46e7a0d1
Merge branch 'master' into improve-backup-and-restore
2022-03-21 11:49:26 +01:00
Vitaly Baranov
51fc556fcd
Implement BACKUP and RESTORE for MatViews.
2022-03-21 11:45:21 +01:00
Maksim Kita
695b1300f9
Merge pull request #35383 from rschu1ze/enable-if-to-concepts-pt2
...
Replace more uses of std::enable_if for SFINAE by concepts
2022-03-21 10:47:28 +01:00
Antonio Andelic
b8c43ff2f2
Merge branch 'master' into parallel-downloading-url-engine
2022-03-21 09:11:01 +00:00
kssenii
59b7394caf
WriteSettings
2022-03-21 09:52:48 +01:00
kssenii
1d80262a2a
Add write settings, file cache settings
2022-03-21 09:50:50 +01:00
Alexander Tokmakov
9e05b12d2c
Merge branch 'master' into mvcc_prototype
2022-03-20 22:42:26 +01:00
Vitaly Baranov
c884cf901f
Use BackupEntryFromAppendOnlyFile for Log engine,
...
and use a BackupEntriesBatch for Memort engine to improve performance.
A lot of minor corrections.
2022-03-20 20:02:15 +01:00
Vitaly Baranov
a1b364626a
Improve zip archive utilities.
2022-03-20 20:02:15 +01:00
alesapin
5902304fdb
Remove some useless code from S3Common
2022-03-19 23:28:20 +01:00
kssenii
af3bb3b7af
Add WriteSettings
2022-03-18 12:27:38 +01:00
Antonio Andelic
f2a7c6c6fe
Add support for canceling a single worker
2022-03-18 09:28:56 +00:00
Robert Schulze
df73c75456
Replace more uses of std::enable_if for SFINAE by concepts
...
- enable_if is usually regarded as fragile and unreadable
- C++20 concepts are much easier to read and produce more expressive error messages
- this is follow-up to PR #35347 but this time most of the remaining and
more complex usages of enable_if in the codebase were replaced.
2022-03-17 20:44:55 +01:00
Antonio Andelic
b789a41662
Fix ASAN problems
2022-03-17 15:18:20 +00:00
Alexander Tokmakov
d04dc03fa4
Merge branch 'master' into mvcc_prototype
2022-03-17 15:24:32 +01:00
Antonio Andelic
103a3fa140
Merge branch 'master' into parallel-downloading-url-engine
2022-03-17 10:11:58 +00:00
Antonio Andelic
44de5a581c
Fix split build
2022-03-17 10:09:16 +00:00
alesapin
457fa0d1ba
Merge pull request #23932 from CurtizJ/dynamic-columns
...
Support of dynamic subcolumns (JSON data type)
2022-03-17 10:45:46 +01:00
Maksim Kita
4d8c2b2009
Merge pull request #35347 from rschu1ze/enable-if-to-concepts
...
Replace a few uses of enable_if for SFINAE by concepts
2022-03-17 10:34:08 +01:00
Alexander Tokmakov
4b3e13a4fe
Merge branch 'master' into mvcc_prototype
2022-03-16 21:06:19 +01:00
Alexander Tokmakov
c2ac8d4a5c
review fixes
2022-03-16 21:05:34 +01:00
Robert Schulze
b56c28d841
Replace a few uses of enable_if for SFINAE by concepts
...
- enable_if is usually regarded as fragile and unreadable
- C++20 concepts are much easier to read and produce more expressive
error messages
2022-03-16 19:51:38 +01:00
Antonio Andelic
0979155f2f
Address PR comments
2022-03-16 15:45:17 +00:00
Anton Popov
0ba78c3c3a
Merge remote-tracking branch 'upstream/master' into HEAD
2022-03-16 15:28:09 +00:00
Antonio Andelic
84f86bf3da
Limit number of bytes read per worker
2022-03-16 12:34:14 +00:00
Antonio Andelic
2291b7c1e1
Merge branch 'master' into parallel-downloading-url-engine
2022-03-16 09:59:52 +00:00
Antonio Andelic
3465664871
Attach thread pool threads to counters
2022-03-16 09:26:01 +00:00
Kruglov Pavel
2b7e89e171
Merge pull request #35300 from Avogar/fix-bzip2
...
Fix possible assertion while using bzip2 compression
2022-03-15 22:17:37 +01:00
Kruglov Pavel
20e17a6d3e
Merge pull request #35296 from Avogar/fix-lz4
...
Fix possible segfault while using lz4 compression
2022-03-15 22:17:14 +01:00
Kruglov Pavel
6fd1a8281f
Merge pull request #35295 from Avogar/fix-lzma
...
Fix possible Assertion failed while using lzma compression
2022-03-15 22:14:35 +01:00
Nikolai Kochetov
a380aa6b8a
Merge pull request #35294 from ClickHouse/reload-remote_url_allow_hosts
...
Reload remote_url_allow_hosts after config update.
2022-03-15 22:07:16 +01:00
Saad Ur Rahman
275bf0ef81
[IO] fixed bug in versionId extraction introduced during rebase.
2022-03-15 12:13:44 -04:00
avogar
ec3c5f86ad
Fix possible assertion while using bzip2 compression
2022-03-15 14:23:52 +00:00
avogar
0cfaf48e1d
Fix possible segfault while using lz4 compression
2022-03-15 13:46:14 +00:00
Alexander Tokmakov
1f571b7734
Merge branch 'master' into mvcc_prototype
2022-03-15 14:45:06 +01:00
Kruglov Pavel
24ac217be3
Remove redundant include
2022-03-15 14:07:31 +01:00
avogar
90ce1709d7
Fix possible Assertion failed while using lzma compression
2022-03-15 13:00:53 +00:00
Nikolai Kochetov
97aa6c82ce
Reload remote_url_allow_hosts after config update.
2022-03-15 13:00:31 +00:00
mergify[bot]
d613ad9445
Merge branch 'master' into fix-brotli
2022-03-15 08:25:12 +00:00
Antonio Andelic
067b79b00b
Merge branch 'master' into parallel-downloading-url-engine
2022-03-15 07:55:41 +00:00
Saad Ur Rahman
7b35c79b04
[IO] versionId extraction simplified.
...
Reducing cognitive load.
2022-03-14 22:27:58 -04:00
Saad Ur Rahman
d9b370cf1d
[IO] S3 URI versionId extraction performance tweak.
2022-03-14 22:17:50 -04:00
Saad Ur Rahman
1a85e9b60d
[IO] Extracting Version ID in S3::URI
...
Extracting Version ID form query string in URI.
2022-03-14 22:16:24 -04:00
Saad Ur Rahman
ec28af055f
[IO] S3 URI data struct contains Version ID
...
Added Version ID field to S3::URI struct.
TODO: Extract version id if present.
2022-03-14 22:11:18 -04:00
Saad Ur Rahman
c7402c2a39
[IO] Version ID provided during buffer read
...
The Version ID, if provided, is to be set in the request to read the buffer from S3. If not provided the latest version of the object must be retrieved.
Version ID set to last parameter to allow defaulting and backward compatibility with calls.
2022-03-14 20:10:14 -04:00
Alexander Tokmakov
9702b5177d
Merge branch 'master' into mvcc_prototype
2022-03-14 21:45:38 +01:00
avogar
4b12c6e6b3
Fix possible Abort while using Brotli compression
2022-03-14 18:58:05 +00:00
Maksim Kita
e30117a3d6
Fix clang-tidy warnings in Interpreters, IO folders
2022-03-14 18:17:35 +00:00
Anton Popov
36ec379aeb
Merge remote-tracking branch 'upstream/master' into HEAD
2022-03-14 16:28:35 +00:00
Antonio Andelic
5f8d9885b3
Ignore read bytes
2022-03-14 15:50:30 +00:00
Azat Khuzhin
9f7fae56b0
WriteBufferFromS3: remove unused headers
...
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-03-14 12:07:47 +03:00
Antonio Andelic
f5d3a8a31d
Polishing
2022-03-11 13:38:19 +00:00
Antonio Andelic
28e9508c4e
Improve shared pool and add settings
2022-03-11 08:33:34 +00:00
Antonio Andelic
cce318273b
Use a shared IO thread pool
2022-03-11 08:33:34 +00:00
Antonio Andelic
adaaaaf061
Unblock readers before stoping
2022-03-11 08:33:34 +00:00
Antonio Andelic
df0f5e20d0
Refactor
2022-03-11 08:33:34 +00:00
Antonio Andelic
7b526b25a8
Initial implementation with Arena
2022-03-11 08:33:34 +00:00
Antonio Andelic
16c20332cf
Fix deadlocks
2022-03-11 08:33:34 +00:00
Antonio Andelic
c6664bbbdf
Add seekable ParallelReadBuffer
2022-03-11 08:33:34 +00:00
Antonio Andelic
29c32ed831
Refactor code
2022-03-11 08:33:34 +00:00
Antonio Andelic
0a1a3a230e
Add support for parallel http range requests
2022-03-11 08:33:34 +00:00
Antonio Andelic
aad87879cf
Small refactor
2022-03-11 08:33:34 +00:00
Kseniia Sumarokova
818459b9f0
Merge pull request #33717 from kssenii/local-cache-for-remote-fs
...
Local cache for remote filesystem
2022-03-11 07:23:10 +01:00
Rajkumar
137e4b79c1
override needed
2022-03-10 12:46:32 -08:00
Kseniia Sumarokova
5149bd2d29
Merge pull request #35172 from kssenii/add-non-retriable-errors-for-url
...
Add non-retriable errors check for HTTP read buffer
2022-03-10 20:28:37 +01:00
Rajkumar
0de950c3f8
style correction
2022-03-10 10:29:57 -08:00
Rajkumar
81ad3ca220
syle check
2022-03-10 10:19:03 -08:00
Rajkumar
a17e5efea2
adding destructors
2022-03-10 07:31:06 -08:00
Alexander Tokmakov
061fa6a6f2
Merge branch 'master' into mvcc_prototype
2022-03-10 13:13:04 +01:00
kssenii
050802b99b
Add non-retriable errors
2022-03-10 11:38:46 +01:00
lgbo-ustc
fdd423a3da
fixed code style
2022-03-10 12:13:19 +08:00
lgbo-ustc
e4883f31b7
update tests
...
1. fixed code style in src/IO/tests/gtest_hadoop_snappy_decoder.cpp
2. enable tests 01060_avro.sh
2022-03-10 09:46:43 +08:00
lgbo-ustc
8dc8c87fd1
add a test case
2022-03-09 10:03:04 +08:00
kssenii
5260822964
Merge master
2022-03-08 18:21:28 +01:00
kssenii
e231c3a3e0
Fix split build
2022-03-08 18:05:55 +01:00
kssenii
f3cba6f270
Remove already fixed crutch
2022-03-08 14:30:21 +01:00
lgbo-ustc
7f89a1bcf3
add some usage test
2022-03-08 20:00:39 +08:00
lgbo-ustc
225b0bd914
fixed bug: call need_more_input repeatly, overwrite the buffer
2022-03-08 17:17:06 +08:00
taiyang-li
b4174b0bef
merge master and fix conflicts
2022-03-08 11:39:25 +08:00
kssenii
0737aef472
Fix split build
2022-03-07 22:02:07 +01:00
Alexander Tokmakov
8acfb8d27f
Merge branch 'master' into mvcc_prototype
2022-03-07 17:40:15 +01:00
kssenii
eeb0ae9816
Fix detaching query
2022-03-07 17:05:36 +01:00
Anton Popov
0bc57da238
Merge remote-tracking branch 'upstream/master' into HEAD
2022-03-07 14:46:08 +00:00
kssenii
692b247379
Fix some corner cases
2022-03-07 14:31:21 +01:00
kssenii
2dcfe66236
Fix race
2022-03-06 20:33:07 +01:00
Anton Popov
df3b07fe7c
Merge remote-tracking branch 'upstream/master' into HEAD
2022-03-03 22:25:28 +00:00
vdimir
d3ee3dee85
Catch exception in ~WriteBufferFromS3
2022-03-03 12:54:24 +00:00
Maksim Kita
b1a956c5f1
clang-tidy check performance-move-const-arg fix
2022-03-02 18:15:27 +00:00
Maksim Kita
1f5837359e
clang-tidy check performance-noexcept-move-constructor fix
2022-03-02 18:15:27 +00:00
kssenii
d5952109fb
Merge master
2022-03-02 18:15:25 +01:00
HaiBo Li
2ea18b3085
Fix the bug that the toDatetime function overflows ( #32898 )
...
* Fixed overflow bug of the function toDatetime/toDate/toDate32/parseDateTimeBestEffort
2022-03-02 10:06:38 +03:00
Anton Popov
2758db5341
add more comments
2022-03-01 19:32:55 +03:00
Danila Kutenin
3ce71f211a
First portion of force inlines
2022-02-27 10:43:17 +00:00
Anton Popov
fcdebea925
Merge remote-tracking branch 'upstream/master' into HEAD
2022-02-25 13:41:30 +03:00
Alexander Tokmakov
aa6b9a2abc
Merge branch 'master' into mvcc_prototype
2022-02-23 23:22:03 +03:00
kssenii
c637385dd0
Merge master
2022-02-22 13:17:51 +01:00
Kseniia Sumarokova
eeea322556
Merge pull request #34629 from amosbird/remotefsimprove
...
Some refactoring and improvement over async and remote buffer related stuff
2022-02-22 11:36:40 +01:00
kssenii
fad7c042c1
Fix cache with non-async reads
2022-02-21 19:50:27 +01:00
Amos Bird
01652d9892
Add more assertion
2022-02-21 22:42:43 +08:00
kssenii
4543513a5d
Add comments
2022-02-21 13:54:03 +01:00
Maksim Kita
883c7c21d1
Merge pull request #34690 from nvartolomei/nv/always-increment
...
Always update ProfileEvents (even on exceptions)
2022-02-21 00:03:44 +01:00
Maksim Kita
e049f6ada9
Update WriteBufferFromPocoSocket.cpp
2022-02-20 16:59:39 +01:00
Maksim Kita
d44f48fe1f
Update ReadBufferFromPocoSocket.cpp
2022-02-20 16:59:17 +01:00
kssenii
e8b2e47e93
Style
2022-02-19 20:38:23 +01:00
Amos Bird
84d111cde9
Better ReadBuffer asynchronicity
2022-02-20 01:08:21 +08:00
kssenii
acf9a092a9
Review fixes
2022-02-18 19:09:54 +01:00
Nicolae Vartolomei
1778be516b
Always update ProfileEvents (even on exceptions)
2022-02-17 16:29:47 +00:00
Alexander Tokmakov
dae044f86b
Merge branch 'master' into mvcc_prototype
2022-02-17 13:49:37 +03:00
alesapin
6df2c9c2d8
Merge pull request #34433 from ClickHouse/s3-fix-content-type
...
Workaround for bug when AWS mistakenly set Content-Type to XML
2022-02-17 13:11:14 +03:00
kssenii
47f94120da
Merge master
2022-02-16 14:43:28 +01:00
kssenii
3bd3e51aa0
Fix tests
2022-02-16 14:08:41 +01:00
alesapin
36b1944cb0
Merge branch 'master' into s3-fix-content-type
2022-02-15 12:39:49 +03:00
李扬
f52b67b939
Merge branch 'master' into rocksdb_metacache
2022-02-15 02:16:29 -06:00
Alexander Tokmakov
1e4e569151
Merge branch 'master' into mvcc_prototype
2022-02-15 02:26:47 +03:00
kssenii
bdbbe85f95
Fixes
2022-02-14 19:28:48 +01:00
alesapin
89373155fc
Merge branch 'master' into revert-34211-revert-34153-add_func_tests_over_s3
2022-02-13 21:07:54 +03:00
Maksim Kita
cb09aadc72
Merge pull request #34524 from FrankChen021/urlengine
...
Fix compression support in URL engine
2022-02-13 14:39:36 +01:00
Saad Ur Rahman
03f81c8685
[CH-34239] HTTP User-Agent header ( #34330 )
...
* [IO] HTTP User-Agent header
Added HTTP User-Agent Header to HTTP requests.
User-Agent: ClickHouse/VERSION_STRING
Input header vectors could potentially contain User-Agent. If so, do not set another.
* [Tests] HTTP User Agent
Added stateless functional test 02205_HTTP_user_agent.
Co-authored-by: Filatenkov Artur <58165623+FArthur-cmd@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-02-13 00:40:11 +03:00
李扬
daa27d0bda
Merge branch 'master' into rocksdb_metacache
2022-02-12 07:50:12 -06:00
Azat Khuzhin
cc55c8b982
Fix gtest_archive_reader_and_writer in case of !USE_MINIZIP
...
Empty arrays/vectors are not supported by ValuesIn:
/src/ch/clickhouse/src/IO/tests/gtest_archive_reader_and_writer.cpp:341:59: error: no matching function for call to 'ValuesIn'
INSTANTIATE_TEST_SUITE_P(All, ArchiveReaderAndWriterTest, ::testing::ValuesIn(supported_archive_file_exts));
^~~~~~~~~~~~~~~~~~~
/src/ch/clickhouse/contrib/googletest/googletest/include/gtest/gtest-param-test.h:459:43: note: expanded from macro 'INSTANTIATE_TEST_SUITE_P'
return GTEST_EXPAND_(GTEST_GET_FIRST_(__VA_ARGS__, DUMMY_PARAM_)); \
^~~~~~~~~~~
/src/ch/clickhouse/contrib/googletest/googletest/include/gtest/gtest-param-test.h:453:38: note: expanded from macro 'GTEST_GET_FIRST_'
#define GTEST_GET_FIRST_(first, ...) first
^~~~~
/src/ch/clickhouse/contrib/googletest/googletest/include/gtest/gtest-param-test.h:452:28: note: expanded from macro 'GTEST_EXPAND_'
#define GTEST_EXPAND_(arg) arg
^~~
/src/ch/clickhouse/contrib/googletest/googletest/include/gtest/gtest-param-test.h:301:29: note: candidate template ignored: substitution failure [with T = std::vector<std::string>, N = 0]: zero-length arrays are not permitted in C++
internal::ParamGenerator<T> ValuesIn(const T (&array)[N]) {
^ ~
/src/ch/clickhouse/contrib/googletest/googletest/include/gtest/gtest-param-test.h:306:58: note: candidate template ignored: substitution failure [with Container = std::vector<std::string> [0]]: type 'std::vector<std::string> [0]' cannot be used prior to '::' because it has no members
internal::ParamGenerator<typename Container::value_type> ValuesIn(
~~~~~~~~~ ^
/src/ch/clickhouse/contrib/googletest/googletest/include/gtest/gtest-param-test.h:294:1: note: candidate function template not viable: requires 2 arguments, but 1 was provided
ValuesIn(ForwardIterator begin, ForwardIterator end) {
^
/src/ch/clickhouse/src/IO/tests/gtest_archive_reader_and_writer.cpp:341:59: error: no matching function for call to 'ValuesIn'
INSTANTIATE_TEST_SUITE_P(All, ArchiveReaderAndWriterTest, ::testing::ValuesIn(supported_archive_file_exts));
^~~~~~~~~~~~~~~~~~~
/src/ch/clickhouse/contrib/googletest/googletest/include/gtest/gtest-param-test.h:468:32: note: expanded from macro 'INSTANTIATE_TEST_SUITE_P'
auto t = std::make_tuple(__VA_ARGS__); \
^~~~~~~~~~~
/src/ch/clickhouse/contrib/googletest/googletest/include/gtest/gtest-param-test.h:301:29: note: candidate template ignored: substitution failure [with T = std::vector<std::string>, N = 0]
internal::ParamGenerator<T> ValuesIn(const T (&array)[N]) {
^
/src/ch/clickhouse/contrib/googletest/googletest/include/gtest/gtest-param-test.h:306:58: note: candidate template ignored: substitution failure [with Container = std::vector<std::string> [0]]: type 'std::vector<std::string> [0]' cannot be used prior to '::' because it has no members
internal::ParamGenerator<typename Container::value_type> ValuesIn(
~~~~~~~~~ ^
/src/ch/clickhouse/contrib/googletest/googletest/include/gtest/gtest-param-test.h:294:1: note: candidate function template not viable: requires 2 arguments, but 1 was provided
ValuesIn(ForwardIterator begin, ForwardIterator end) {
^
2 errors generated.
ninja: build stopped: subcommand failed.
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-02-12 15:01:20 +03:00
Frank Chen
7b7bc8f6c3
Address review comment
2022-02-12 10:46:01 +08:00
Alexander Tokmakov
07e66e690d
Merge branch 'master' into mvcc_prototype
2022-02-11 15:53:32 +03:00
mergify[bot]
f099048dd1
Merge branch 'master' into urlengine
2022-02-11 11:03:21 +00:00
Frank Chen
c8cddd7077
Fix compression for URL engine
2022-02-11 17:45:49 +08:00
alesapin
84b5fbf43c
Fix buffer HDFS crash
2022-02-10 16:35:50 +03:00
Anton Popov
18940b8637
Merge remote-tracking branch 'upstream/master' into HEAD
2022-02-09 23:38:38 +03:00
alesapin
b84a171aa7
Fix style
2022-02-09 14:13:53 +03:00
alesapin
437940b29d
Merge pull request #34422 from ClickHouse/add_settings_to_part_size_in_s3
...
Add settings for multipart upload part size in s3
2022-02-09 12:51:20 +03:00
Kseniia Sumarokova
86956ca08b
Merge pull request #34392 from kssenii/http-buffer-skip-not-found-url-for-globs
...
Allow to skip not found urls for globs
2022-02-09 09:32:09 +01:00
Alexey Milovidov
734b08177e
Fix excessive logging while writing to s3
2022-02-09 07:19:33 +03:00
taiyang-li
d04ccc0489
Merge branch 'master' into rocksdb_metacache
2022-02-09 11:54:10 +08:00
Alexey Milovidov
92fb6bf735
Workaround for bug in AWS
2022-02-09 05:25:22 +03:00
kssenii
e8a8f46864
Fix style check
2022-02-08 21:58:03 +01:00
alesapin
f6e7cea1f8
Add settings for S3 multipart upload parts size
2022-02-08 19:38:04 +03:00
Vitaly Baranov
9a8e45805a
Merge pull request #34226 from vitlibar/add-submodule-minizip
...
Add submodule minizip
2022-02-08 22:07:34 +07:00
Kruglov Pavel
720310fcf1
Merge branch 'master' into http-buffer-skip-not-found-url-for-globs
2022-02-08 16:56:22 +03:00
kssenii
dc5f035265
Fix
2022-02-08 13:28:10 +01:00
kssenii
3834bdbae0
Fixes
2022-02-08 11:11:18 +01:00
taiyang-li
b6132d490f
merge master and solve conflict
2022-02-08 15:24:59 +08:00
kssenii
eba3011ada
Fix
2022-02-07 20:40:47 +01:00
Anton Popov
d32d6453d3
Update BufferBase.h
2022-02-07 17:49:53 +03:00
kssenii
2e58733750
Merge master
2022-02-07 15:23:10 +01:00
Anton Popov
1b16db72c3
fix consecutive backward seeks in seekable read buffers
2022-02-07 17:20:26 +03:00
Anton Popov
ae1fc94fb5
add unit test
2022-02-06 20:45:59 +00:00
Anton Popov
10b8684003
fix rare bug in reading of empty arrays
2022-02-06 20:45:59 +00:00
Alexander Tokmakov
45be75b4db
Merge branch 'master' into mvcc_prototype
2022-02-06 23:36:08 +03:00
Vitaly Baranov
d9bdbf47b7
ReadBufferFromMemory now can seek backwards after been read up to EOF.
2022-02-05 23:45:20 +07:00
Vitaly Baranov
23fac284ea
Add utility classes ZipArchiveReader and ZipArchiveWriter.
2022-02-05 23:45:20 +07:00
Nikolai Kochetov
daeeb6f3a2
Merge pull request #34316 from ClickHouse/probably-fix-data-race-in-WriteBufferFromS3
...
Probably fix data race in WriteBufferFromS3 destructor.
2022-02-04 21:04:46 +03:00
Nikolai Kochetov
a968c869fd
Probably fix data race in WriteBufferFromS3 destructor.
2022-02-04 12:28:14 +00:00
Maksim Kita
074b827cf3
Merge pull request #34306 from ClickHouse/line-as-string-low-performance
...
Fix terribly low performance of `LineAsString` format
2022-02-04 13:11:27 +01:00
Alexey Milovidov
e20aec6866
Fix linkage
2022-02-04 01:44:43 +03:00
Alexey Milovidov
4a83dbc514
Fix linkage
2022-02-04 00:26:44 +03:00
Alexey Milovidov
7c12f5f37a
Fix terribly low performance of LineAsString format
2022-02-04 00:07:31 +03:00
Alexander Tokmakov
ca5f951558
Merge branch 'master' into mvcc_prototype
2022-02-03 18:56:44 +03:00
Nikolai Kochetov
2003a96b58
Probably fix something
2022-02-02 17:48:41 +00:00
Nikolai Kochetov
ea044fc6b2
Merge branch 'master' into revert-34212-revert-33291-add-pool-to-s3-write-buffer
2022-02-02 19:40:41 +03:00
kssenii
aeec749173
Fix predownload case part 1
2022-02-02 17:25:25 +03:00
kssenii
5cba11428a
Fixes
2022-02-02 00:25:16 +03:00
Sergei Trifonov
68bc456830
Merge pull request #34223 from azat/bump-fmt
...
Bump fmtlib from 7.0.0 to 8.1.1
2022-02-02 00:03:25 +03:00
alexey-milovidov
2b1d1a9a6f
Merge pull request #34208 from ClickHouse/parse-date-time-best-effort-more-cases
...
More cases for `parseDateTimeBestEffort`
2022-02-01 16:22:52 +03:00
Anton Popov
836a348a9c
Merge remote-tracking branch 'upstream/master' into HEAD
2022-02-01 15:23:07 +03:00
Azat Khuzhin
3b3635c6d5
Fix formatting error in logging messages
...
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-02-01 14:30:04 +03:00
Amos Bird
e29c77f793
Fix link order issue of WriteBufferFromS3
...
(cherry picked from commit f22b09f4fc
2022-02-01 10:42:06 +00:00
Nikolai Kochetov
2a6eb593be
Revert "Revert "Add pool to WriteBufferFromS3""
2022-02-01 13:36:51 +03:00
Alexander Tokmakov
2e4ae37d98
Merge branch 'master' into mvcc_prototype
2022-02-01 13:20:03 +03:00
alexey-milovidov
095d9bfa43
Revert "Add pool to WriteBufferFromS3"
2022-02-01 05:49:40 +03:00
Alexey Milovidov
989a5ad35e
More cases for parseDateTimeBestEffort
2022-02-01 03:29:27 +03:00
alesapin
dd61d1c2de
Merge pull request #34172 from ClickHouse/fix_race_in_some_engines
...
Fix benign race condition for storage HDFS, S3, URL
2022-01-31 22:41:54 +03:00
alesapin
31753afb7e
Fix cancel logic in parallel parsing
2022-01-31 16:38:15 +03:00
Anton Popov
78b9f15abb
Merge remote-tracking branch 'upstream/master' into HEAD
2022-01-30 03:24:37 +03:00
kssenii
6b03c577d6
Make downloader's term resticted to one nextImpl
2022-01-29 23:58:53 +03:00
Alexander Tokmakov
fb9b2d5326
Merge branch 'master' into mvcc_prototype
2022-01-28 21:18:36 +03:00
Nikolai Kochetov
1c9f026178
Merge branch 'master' into add-pool-to-s3-write-buffer
2022-01-28 16:01:42 +03:00
taiyang-li
3de8bde7ce
Merge remote-tracking branch 'origin/master' into rocksdb_metacache
2022-01-28 09:58:52 +08:00
kssenii
c91b86e220
Better
2022-01-26 21:56:08 +03:00
Nikolai Kochetov
a8171269a1
Review fixes.
2022-01-26 17:55:24 +00:00
Nikolai Kochetov
2e88319b72
Fix integration tests.
2022-01-25 18:48:28 +00:00
Nikolai Kochetov
fcc29dbd15
Try to fix integration tests.
2022-01-25 15:26:36 +00:00
Kruglov Pavel
1b19139f74
Merge pull request #31862 from Avogar/fix-lz4
...
Try to fix lz4 compression
2022-01-25 14:34:45 +03:00
Nikolai Kochetov
6c65915241
Fix style
2022-01-24 16:12:23 +00:00
Nikolai Kochetov
506ee8c024
Refactor some code.
2022-01-24 15:55:29 +00:00
Nikolai Kochetov
4afa75e80b
Fix deduplication.
2022-01-24 14:43:36 +00:00
taiyang-li
73def8b483
merge master and solve conflict
2022-01-24 11:01:43 +08:00
Kseniia Sumarokova
2c43e4eb57
Merge branch 'master' into local-cache-for-remote-fs
2022-01-22 22:53:55 +03:00
alexey-milovidov
eb6849f7c7
Merge pull request #33842 from azat/cmake-contrib-fixes
...
More cmake external modules cleanups
2022-01-22 10:34:54 +03:00
Nikolai Kochetov
b3cbb63487
Merge branch 'master' into add-pool-to-s3-write-buffer
2022-01-21 21:41:54 +03:00
Anton Popov
e8ce091e68
Merge remote-tracking branch 'upstream/master' into HEAD
2022-01-21 20:11:18 +03:00
Alexander Tokmakov
e3b1397001
Merge branch 'master' into mvcc_prototype
2022-01-21 18:59:01 +03:00
Azat Khuzhin
2aa7fb28a0
contrib/dragonbox: add ALIAS library
...
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-01-21 10:11:22 +03:00
Azat Khuzhin
c07a2c856a
Fix bypass virtual dispatch in ZstdDeflatingAppendableWriteBuffer
...
clang-tidy [1]:
"Call to virtual method 'ZstdDeflatingAppendableWriteBuffer::finalizeAfter' during destruction bypasses virtual dispatch"
[1]: https://s3.amazonaws.com/clickhouse-builds/33840/d021190b8c7ae0f5dc2decb953c471ea9d770981/binary_tidy/build_log.log
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-01-20 20:40:04 +03:00
Azat Khuzhin
d021190b8c
keeper: fix memory leak in case of compression is used (default)
...
In case of compression is used, ZstdDeflatingAppendableWriteBuffer is
used, but it has a leak, since it frees ZSTD_CCtx only if there was
write while this is not correct, since it is created anyway.
This was found with jemalloc profile, keeper-bench and the following
keeper settings:
- force_sync=false
- snapshot_distance=100
- reserved_log_items=0
- rotate_log_storage_interval=100
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-01-20 19:03:00 +03:00
Alexander Tokmakov
0ea0a8ccc7
Merge branch 'master' into mvcc_prototype
2022-01-20 17:05:55 +03:00
mergify[bot]
9326f99d78
Merge branch 'master' into fix-lz4
2022-01-20 11:53:51 +00:00
Azat Khuzhin
ca8525a733
Remove unbundled zlib support
2022-01-20 08:47:16 +03:00
Alexander Tokmakov
e9a5a64a71
Merge branch 'master' into mvcc_prototype
2022-01-19 21:41:23 +03:00
kssenii
a566099759
Local cache for remote filesystem
2022-01-19 00:42:11 +03:00
Kruglov Pavel
2295a07066
Merge pull request #33534 from azat/fwd-decl
...
RFC: Split headers, move SystemLog into module, more forward declarations
2022-01-18 17:22:49 +03:00
Alexander Tokmakov
4dd216ce8f
Merge branch 'master' into mvcc_prototype
2022-01-14 17:11:32 +03:00
taiyang-li
66813a3aa9
merge master
2022-01-12 16:56:29 +08:00
Maksim Kita
c1ec85d87a
Build with ENABLE_TESTS fix
2022-01-11 12:30:40 +03:00
Azat Khuzhin
cb70544dfe
Move LockMemoryExceptionInThread and MemoryTrackerBlockerInThread
2022-01-10 22:39:10 +03:00
Alexander Tokmakov
158fbaaa29
fix
2022-01-10 20:07:01 +03:00
Alexander Tokmakov
bf2b6c2c37
Merge branch 'master' into mvcc_prototype
2022-01-10 19:53:50 +03:00
taiyang-li
83d064c24a
fix unit test and build error
2022-01-06 11:41:24 +08:00
