Frank Chen
7b7bc8f6c3
Address review comment
2022-02-12 10:46:01 +08:00
Alexander Tokmakov
07e66e690d
Merge branch 'master' into mvcc_prototype
2022-02-11 15:53:32 +03:00
mergify[bot]
f099048dd1
Merge branch 'master' into urlengine
2022-02-11 11:03:21 +00:00
Frank Chen
c8cddd7077
Fix compression for URL engine
2022-02-11 17:45:49 +08:00
alesapin
84b5fbf43c
Fix buffer HDFS crash
2022-02-10 16:35:50 +03:00
Anton Popov
18940b8637
Merge remote-tracking branch 'upstream/master' into HEAD
2022-02-09 23:38:38 +03:00
alesapin
b84a171aa7
Fix style
2022-02-09 14:13:53 +03:00
alesapin
437940b29d
Merge pull request #34422 from ClickHouse/add_settings_to_part_size_in_s3
...
Add settings for multipart upload part size in s3
2022-02-09 12:51:20 +03:00
Kseniia Sumarokova
86956ca08b
Merge pull request #34392 from kssenii/http-buffer-skip-not-found-url-for-globs
...
Allow to skip not found urls for globs
2022-02-09 09:32:09 +01:00
Alexey Milovidov
734b08177e
Fix excessive logging while writing to s3
2022-02-09 07:19:33 +03:00
taiyang-li
d04ccc0489
Merge branch 'master' into rocksdb_metacache
2022-02-09 11:54:10 +08:00
Alexey Milovidov
92fb6bf735
Workaround for bug in AWS
2022-02-09 05:25:22 +03:00
kssenii
e8a8f46864
Fix style check
2022-02-08 21:58:03 +01:00
alesapin
f6e7cea1f8
Add settings for S3 multipart upload parts size
2022-02-08 19:38:04 +03:00
Vitaly Baranov
9a8e45805a
Merge pull request #34226 from vitlibar/add-submodule-minizip
...
Add submodule minizip
2022-02-08 22:07:34 +07:00
Kruglov Pavel
720310fcf1
Merge branch 'master' into http-buffer-skip-not-found-url-for-globs
2022-02-08 16:56:22 +03:00
kssenii
dc5f035265
Fix
2022-02-08 13:28:10 +01:00
kssenii
3834bdbae0
Fixes
2022-02-08 11:11:18 +01:00
taiyang-li
b6132d490f
merge master and solve conflict
2022-02-08 15:24:59 +08:00
kssenii
eba3011ada
Fix
2022-02-07 20:40:47 +01:00
Anton Popov
d32d6453d3
Update BufferBase.h
2022-02-07 17:49:53 +03:00
kssenii
2e58733750
Merge master
2022-02-07 15:23:10 +01:00
Anton Popov
1b16db72c3
fix consecutive backward seeks in seekable read buffers
2022-02-07 17:20:26 +03:00
Anton Popov
ae1fc94fb5
add unit test
2022-02-06 20:45:59 +00:00
Anton Popov
10b8684003
fix rare bug in reading of empty arrays
2022-02-06 20:45:59 +00:00
Alexander Tokmakov
45be75b4db
Merge branch 'master' into mvcc_prototype
2022-02-06 23:36:08 +03:00
Vitaly Baranov
d9bdbf47b7
ReadBufferFromMemory now can seek backwards after been read up to EOF.
2022-02-05 23:45:20 +07:00
Vitaly Baranov
23fac284ea
Add utility classes ZipArchiveReader and ZipArchiveWriter.
2022-02-05 23:45:20 +07:00
Nikolai Kochetov
daeeb6f3a2
Merge pull request #34316 from ClickHouse/probably-fix-data-race-in-WriteBufferFromS3
...
Probably fix data race in WriteBufferFromS3 destructor.
2022-02-04 21:04:46 +03:00
Nikolai Kochetov
a968c869fd
Probably fix data race in WriteBufferFromS3 destructor.
2022-02-04 12:28:14 +00:00
Maksim Kita
074b827cf3
Merge pull request #34306 from ClickHouse/line-as-string-low-performance
...
Fix terribly low performance of `LineAsString` format
2022-02-04 13:11:27 +01:00
Alexey Milovidov
e20aec6866
Fix linkage
2022-02-04 01:44:43 +03:00
Alexey Milovidov
4a83dbc514
Fix linkage
2022-02-04 00:26:44 +03:00
Alexey Milovidov
7c12f5f37a
Fix terribly low performance of LineAsString format
2022-02-04 00:07:31 +03:00
Alexander Tokmakov
ca5f951558
Merge branch 'master' into mvcc_prototype
2022-02-03 18:56:44 +03:00
Nikolai Kochetov
2003a96b58
Probably fix something
2022-02-02 17:48:41 +00:00
Nikolai Kochetov
ea044fc6b2
Merge branch 'master' into revert-34212-revert-33291-add-pool-to-s3-write-buffer
2022-02-02 19:40:41 +03:00
kssenii
aeec749173
Fix predownload case part 1
2022-02-02 17:25:25 +03:00
kssenii
5cba11428a
Fixes
2022-02-02 00:25:16 +03:00
Sergei Trifonov
68bc456830
Merge pull request #34223 from azat/bump-fmt
...
Bump fmtlib from 7.0.0 to 8.1.1
2022-02-02 00:03:25 +03:00
alexey-milovidov
2b1d1a9a6f
Merge pull request #34208 from ClickHouse/parse-date-time-best-effort-more-cases
...
More cases for `parseDateTimeBestEffort`
2022-02-01 16:22:52 +03:00
Anton Popov
836a348a9c
Merge remote-tracking branch 'upstream/master' into HEAD
2022-02-01 15:23:07 +03:00
Azat Khuzhin
3b3635c6d5
Fix formatting error in logging messages
...
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-02-01 14:30:04 +03:00
Amos Bird
e29c77f793
Fix link order issue of WriteBufferFromS3
...
(cherry picked from commit f22b09f4fc
2022-02-01 10:42:06 +00:00
Nikolai Kochetov
2a6eb593be
Revert "Revert "Add pool to WriteBufferFromS3""
2022-02-01 13:36:51 +03:00
Alexander Tokmakov
2e4ae37d98
Merge branch 'master' into mvcc_prototype
2022-02-01 13:20:03 +03:00
alexey-milovidov
095d9bfa43
Revert "Add pool to WriteBufferFromS3"
2022-02-01 05:49:40 +03:00
Alexey Milovidov
989a5ad35e
More cases for parseDateTimeBestEffort
2022-02-01 03:29:27 +03:00
alesapin
dd61d1c2de
Merge pull request #34172 from ClickHouse/fix_race_in_some_engines
...
Fix benign race condition for storage HDFS, S3, URL
2022-01-31 22:41:54 +03:00
alesapin
31753afb7e
Fix cancel logic in parallel parsing
2022-01-31 16:38:15 +03:00
Anton Popov
78b9f15abb
Merge remote-tracking branch 'upstream/master' into HEAD
2022-01-30 03:24:37 +03:00
kssenii
6b03c577d6
Make downloader's term resticted to one nextImpl
2022-01-29 23:58:53 +03:00
Alexander Tokmakov
fb9b2d5326
Merge branch 'master' into mvcc_prototype
2022-01-28 21:18:36 +03:00
Nikolai Kochetov
1c9f026178
Merge branch 'master' into add-pool-to-s3-write-buffer
2022-01-28 16:01:42 +03:00
taiyang-li
3de8bde7ce
Merge remote-tracking branch 'origin/master' into rocksdb_metacache
2022-01-28 09:58:52 +08:00
kssenii
c91b86e220
Better
2022-01-26 21:56:08 +03:00
Nikolai Kochetov
a8171269a1
Review fixes.
2022-01-26 17:55:24 +00:00
Nikolai Kochetov
2e88319b72
Fix integration tests.
2022-01-25 18:48:28 +00:00
Nikolai Kochetov
fcc29dbd15
Try to fix integration tests.
2022-01-25 15:26:36 +00:00
Kruglov Pavel
1b19139f74
Merge pull request #31862 from Avogar/fix-lz4
...
Try to fix lz4 compression
2022-01-25 14:34:45 +03:00
Nikolai Kochetov
6c65915241
Fix style
2022-01-24 16:12:23 +00:00
Nikolai Kochetov
506ee8c024
Refactor some code.
2022-01-24 15:55:29 +00:00
Nikolai Kochetov
4afa75e80b
Fix deduplication.
2022-01-24 14:43:36 +00:00
taiyang-li
73def8b483
merge master and solve conflict
2022-01-24 11:01:43 +08:00
Kseniia Sumarokova
2c43e4eb57
Merge branch 'master' into local-cache-for-remote-fs
2022-01-22 22:53:55 +03:00
alexey-milovidov
eb6849f7c7
Merge pull request #33842 from azat/cmake-contrib-fixes
...
More cmake external modules cleanups
2022-01-22 10:34:54 +03:00
Nikolai Kochetov
b3cbb63487
Merge branch 'master' into add-pool-to-s3-write-buffer
2022-01-21 21:41:54 +03:00
Anton Popov
e8ce091e68
Merge remote-tracking branch 'upstream/master' into HEAD
2022-01-21 20:11:18 +03:00
Alexander Tokmakov
e3b1397001
Merge branch 'master' into mvcc_prototype
2022-01-21 18:59:01 +03:00
Azat Khuzhin
2aa7fb28a0
contrib/dragonbox: add ALIAS library
...
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-01-21 10:11:22 +03:00
Azat Khuzhin
c07a2c856a
Fix bypass virtual dispatch in ZstdDeflatingAppendableWriteBuffer
...
clang-tidy [1]:
"Call to virtual method 'ZstdDeflatingAppendableWriteBuffer::finalizeAfter' during destruction bypasses virtual dispatch"
[1]: https://s3.amazonaws.com/clickhouse-builds/33840/d021190b8c7ae0f5dc2decb953c471ea9d770981/binary_tidy/build_log.log
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-01-20 20:40:04 +03:00
Azat Khuzhin
d021190b8c
keeper: fix memory leak in case of compression is used (default)
...
In case of compression is used, ZstdDeflatingAppendableWriteBuffer is
used, but it has a leak, since it frees ZSTD_CCtx only if there was
write while this is not correct, since it is created anyway.
This was found with jemalloc profile, keeper-bench and the following
keeper settings:
- force_sync=false
- snapshot_distance=100
- reserved_log_items=0
- rotate_log_storage_interval=100
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-01-20 19:03:00 +03:00
Alexander Tokmakov
0ea0a8ccc7
Merge branch 'master' into mvcc_prototype
2022-01-20 17:05:55 +03:00
mergify[bot]
9326f99d78
Merge branch 'master' into fix-lz4
2022-01-20 11:53:51 +00:00
Azat Khuzhin
ca8525a733
Remove unbundled zlib support
2022-01-20 08:47:16 +03:00
Alexander Tokmakov
e9a5a64a71
Merge branch 'master' into mvcc_prototype
2022-01-19 21:41:23 +03:00
kssenii
a566099759
Local cache for remote filesystem
2022-01-19 00:42:11 +03:00
Kruglov Pavel
2295a07066
Merge pull request #33534 from azat/fwd-decl
...
RFC: Split headers, move SystemLog into module, more forward declarations
2022-01-18 17:22:49 +03:00
Alexander Tokmakov
4dd216ce8f
Merge branch 'master' into mvcc_prototype
2022-01-14 17:11:32 +03:00
taiyang-li
66813a3aa9
merge master
2022-01-12 16:56:29 +08:00
Maksim Kita
c1ec85d87a
Build with ENABLE_TESTS fix
2022-01-11 12:30:40 +03:00
Azat Khuzhin
cb70544dfe
Move LockMemoryExceptionInThread and MemoryTrackerBlockerInThread
2022-01-10 22:39:10 +03:00
Alexander Tokmakov
158fbaaa29
fix
2022-01-10 20:07:01 +03:00
Alexander Tokmakov
bf2b6c2c37
Merge branch 'master' into mvcc_prototype
2022-01-10 19:53:50 +03:00
taiyang-li
83d064c24a
fix unit test and build error
2022-01-06 11:41:24 +08:00
taiyang-li
2bb7ec8f72
Merge remote-tracking branch 'origin/master' into rocksdb_metacache
2022-01-06 10:44:17 +08:00
taiyang-li
c0a9c2b916
refactor metadatacache to reduce using of USE_ROCKSDB
2022-01-05 19:51:50 +08:00
Azat Khuzhin
2cdc5b37e3
Do not try to read pass EOF (to workaround a bug in a kernel)
...
For unaligned offset pread() may return EINVAL even if the offset pass
EOF, although it should not, since otherwise there is no abiliity to
rely on read() == 0 is EOF (with pread() loop).
Here is a reproducer for the problem on 4.9.0-12-amd64:
$ head -c27 /dev/urandom > /tmp/pread.issue
$ xfs_io
xfs_io> open -d /tmp/pread.issue
xfs_io> pread 1000 4096
pread: Invalid argument
And this is how it should work:
xfs_io> pread 29 4096
read 0/4096 bytes at offset 29
Note, here I use interactive mode since we had old xfs_io that does not
allow to execute multiple commands at once, and to avoid EMFILE issue
Here is some history of a patches that affects this behaviour in the
linux kernel:
- the issue had been introduced in
torvalds/linux@9fe55eea7e v3.14
("Fix race when checking i_size on direct i/o read")
- an attempt to fix it had been made in
torvalds/linux@74cedf9b6c v4.4
("direct-io: Fix negative return from dio read beyond eof")
- but this wasn't enough, since alignment check was earlier, so
eventually fixed in
torvalds/linux@41b21af388 v5.10
("direct-io: defer alignment check until after the EOF check")
Someone may ask why CI does not shows the issue, since:
- it had 4.19 kernel when CI was in yandex
- now it has 5.4 when CI is in AWS
Since both of those kernels does not have the last patch.
But, this bug requires the following conditions to met:
- index_granularity_bytes=0
- min_merge_bytes_to_use_direct_io=1
Which was not covered by CI yet.
2022-01-04 10:53:17 +03:00
Azat Khuzhin
71a99ab27c
Accept real file size in createReadBufferFromFileBase()
...
Right now streams relies on correct file size not the number of bytes
that will be read from the stream, to overcome one bug in the linux
kernel that may return EIINVAL for pread() with offset pass the EOF.
v2: Swap read_hint and file_size (since it is easy to miss something)
Before the first argument to readFile()/createReadBufferFromFileBase()
was read_hint, not the file_size, and let's preserve the order, since
it is easy to miss something
This will also fix 02051_read_settings test automatically because now
MergeTreeReaderStream will pass estimated_sum_mark_range_bytes to
read_hint not file_size, previously it cause on of the following errors:
- Attempt to read after EOF w/ O_DIRECT
- and LOGICAL_ERROR while adjusting granulas w/o O_DIRECT
This will also improve zero-length reads guard (via
ReadBufferFromEmptyFile), that had been added in #30190
v3: fix for other storages that wasn't enabled in fast-test
v4: ignore ENOENT/ENOTSUP in readFile
2022-01-04 10:53:17 +03:00
Azat Khuzhin
e548dae8d9
Pass size of the file to the readers
...
v2: s/size/file_size/g
2022-01-04 10:53:17 +03:00
taiyang-li
1e102bc1b2
merge master
2022-01-01 09:01:06 +08:00
Alexander Tokmakov
0d91887cdc
save versions of parts
2021-12-30 16:15:28 +03:00
Nikolay Degterinsky
79e15e84d5
Refactor
2021-12-30 04:47:34 +00:00
Kruglov Pavel
489a30859f
Merge pull request #32455 from Avogar/schema-inference
...
Automatic schema inference for input formats
2021-12-29 21:03:48 +03:00
Nikolai Kochetov
68bf38133a
Add pool to WriteBufferFromS3
2021-12-29 14:04:21 +00:00
avogar
8112a71233
Implement schema inference for most input formats
2021-12-29 12:18:56 +03:00
benbiti
9004e98992
parseDateTimeBestEffort support Unix Timestamp with Milliseconds
2021-12-29 13:43:54 +08:00
Alexander Tokmakov
8aec3ae94f
Merge branch 'master' into mvcc_prototype
2021-12-28 14:24:36 +03:00
taiyang-li
9036b18c2f
merge master
2021-12-27 15:12:48 +08:00
alexey-milovidov
47d50c3bd1
Merge pull request #33140 from kssenii/azure-tests-fix
...
Fix azure blob storage tests failures
2021-12-25 06:25:26 +03:00
alexey-milovidov
d15da636f6
Update ReadBufferFromAzureBlobStorage.cpp
2021-12-25 05:57:15 +03:00
alexey-milovidov
bcd4df789d
Update WriteBufferFromAzureBlobStorage.cpp
2021-12-25 05:56:15 +03:00
alexey-milovidov
f2075fac64
Update ReadBufferFromAzureBlobStorage.cpp
2021-12-25 05:55:49 +03:00
kssenii
b4420cfa3e
Fix
2021-12-24 22:51:36 +03:00
Alexey Milovidov
29d28c531f
Move code around to avoid dlsym on Musl
2021-12-24 12:25:27 +03:00
Kruglov Pavel
f38bc9fc38
Merge branch 'master' into fix-lz4
2021-12-23 16:13:26 +03:00
Nikolay Degterinsky
826f7cb0f5
Merge pull request #32240 from benbiti/master
...
Fix unix timestamp Millisecond convert to DateTime64, fractional part calc reversed
2021-12-23 12:11:53 +03:00
Kseniia Sumarokova
f0434f21ce
Merge pull request #33039 from kssenii/fix-azure-exception
...
Fix exception in azure write buffer desctructor
2021-12-23 10:26:40 +03:00
Vitaly Baranov
7660530fcc
Merge pull request #32955 from azat/read-fix
...
Fix UB in case of unexpected EOF during filling a set from HTTP query
2021-12-23 06:41:14 +03:00
Alexander Tokmakov
b7184732d5
Merge branch 'master' into mvcc_prototype
2021-12-22 19:46:09 +03:00
Dmitry Novik
d256c3a9ca
Merge pull request #29224 from ClickHouse/fix_IP_conversion
...
Merging #27914
2021-12-22 18:36:13 +03:00
mergify[bot]
04f727e84c
Merge branch 'master' into read-fix
2021-12-22 13:49:04 +00:00
kssenii
65c5334bb0
Safer
2021-12-22 14:22:14 +03:00
kssenii
060d3228e9
Fix
2021-12-22 13:43:21 +03:00
kssenii
d8ed4db6d4
Fix
2021-12-22 11:42:23 +03:00
Kseniia Sumarokova
484a4d62dd
Merge pull request #32835 from kssenii/fising-s3
...
Fix optimization with lazy seek for async reads from remote fs
2021-12-22 09:38:23 +03:00
benbiti
81a7a393c8
fix unix timestamp convert, fractional reversed
2021-12-22 01:32:16 +08:00
mergify[bot]
7da6d8b310
Merge branch 'master' into fix_IP_conversion
2021-12-21 12:16:05 +00:00
kssenii
ebcb66a361
Fix
2021-12-21 10:58:02 +03:00
taiyang-li
2597925724
merge master
2021-12-21 15:55:39 +08:00
Azat Khuzhin
4230f04f72
Fix UB in LimitReadBuffer in case of unexpected EOF
...
Fixes the following possible assertions:
- ./src/IO/ReadBuffer.h:58: bool DB::ReadBuffer::next(): Assertion `!hasPendingData()' failed.
- ./src/IO/LimitReadBuffer.cpp:17: virtual bool DB::LimitReadBuffer::nextImpl(): Assertion `position() >= in->position()' failed.
Fixes: 02151_http_s_structure_set_eof
2021-12-21 08:38:41 +03:00
Alexander Tokmakov
9cd49bc0ec
Merge branch 'master' into mvcc_prototype
2021-12-20 22:06:22 +03:00
Nikolai Kochetov
35883e0dae
Merge pull request #32979 from ClickHouse/decrease-some-s3-log-level
...
Decrease log level for some s3 messages.
2021-12-20 20:56:22 +03:00
Maksim Kita
dd0d3de050
Merge pull request #32970 from kitaisreal/loops-remove-postfix-increment
...
Loops remove postfix increment
2021-12-20 19:51:07 +03:00
Vitaly Baranov
e61d3eef0c
Merge pull request #32747 from vitlibar/improve-grpc-compression
...
Improve gRPC compression support
2021-12-20 19:44:58 +03:00
Nikolai Kochetov
7dea7b7f76
Decrease log level for some s3 messages.
2021-12-20 18:18:54 +03:00
Maksim Kita
51477adf1b
Updated additional cases
2021-12-20 15:55:07 +03:00
Vitaly Baranov
02d6cea857
Lz4DeflatingWriteBuffer now supports changes in the nested buffer between calls.
2021-12-19 16:29:57 +03:00
Anton Popov
99ebabd822
Merge remote-tracking branch 'upstream/master' into HEAD
2021-12-17 19:02:29 +03:00
kssenii
40c266840c
Fixes
2021-12-17 18:16:29 +03:00
kssenii
ed42620530
Rename BlobStorage to AzureBlobStorage
2021-12-17 12:24:05 +03:00
Alexander Tokmakov
32e62ed5c2
Merge branch 'master' into mvcc_prototype
2021-12-17 10:43:46 +03:00
taiyang-li
d033fc4c24
merge master and fix conflict
2021-12-17 15:11:21 +08:00
kssenii
ca283a4bb8
Fix optimization with lazy seek
2021-12-17 00:18:28 +03:00
alesapin
c338bee718
Revert "Fix build issue related to azure blob storage"
2021-12-16 14:37:12 +03:00
alesapin
c0cebaea79
Merge pull request #32788 from amosbird/buildfix
...
Fix build issue related to azure blob storage
2021-12-16 13:45:22 +03:00
Kseniia Sumarokova
947ef81f53
Update ReadWriteBufferFromHTTP.h
2021-12-16 09:14:09 +03:00
mergify[bot]
41ae7c8eb1
Merge branch 'master' into fix_IP_conversion
2021-12-15 19:19:47 +00:00
Amos Bird
9c9f4a3e6c
build fix
2021-12-15 16:55:18 +08:00
mergify[bot]
4300f77532
Merge branch 'master' into fix_IP_conversion
2021-12-14 20:12:34 +00:00
Alexander Tokmakov
d7ad72838c
Merge branch 'master' into mvcc_prototype
2021-12-14 23:07:52 +03:00
alesapin
9974f10631
Fix clang tidy
2021-12-14 15:34:20 +03:00
taiyang-li
ca3f7425a4
fix code
2021-12-14 17:37:31 +08:00
taiyang-li
8234d1176f
merge master
2021-12-14 10:39:21 +08:00
alexey-milovidov
730c16bd0c
Merge pull request #32649 from vdimir/issue_32504
...
Reset pipeline in BlockIO::onFinish
2021-12-13 21:05:33 +03:00
alexey-milovidov
fd9d40925a
Merge pull request #32640 from ClickHouse/fix-function-file
...
Fix surprisingly bad code in function `file`
2021-12-13 19:34:36 +03:00
Kseniia Sumarokova
78ed383f15
Merge pull request #31505 from ContentSquare/azure_blob_storage_dependency
...
Azure Blob Storage Disks
2021-12-13 18:28:48 +03:00
vdimir
bda4ecbe90
Catch exceprion in ~ZlibDeflatingWriteBuffer
2021-12-13 12:29:35 +03:00
Alexey Milovidov
71926a3a97
Fix surprisingly bad code in function "file"
2021-12-13 07:57:54 +03:00
李扬
8675086104
Merge branch 'master' into hive_table
2021-12-12 09:01:46 -06:00
Jakub Kuklis
578aec97e9
Removing TODO and NOTE marks used mainly as hooks for review in the PR
2021-12-10 22:04:38 +01:00
Jakub Kuklis
85807adec4
Fixing variable shadow error
2021-12-10 22:04:38 +01:00
Jakub Kuklis
910db6ca57
Code cleanup addressing PR remarks
2021-12-10 22:04:38 +01:00
Jakub Kuklis
6acbec4455
Rebased on master
2021-12-10 22:04:38 +01:00
Jakub Kuklis
0c65cfed1f
Addressing first PR remarks
2021-12-10 22:04:37 +01:00
Jakub Kuklis
7c2e116214
Adding max_retries config
2021-12-10 22:04:37 +01:00
Jakub Kuklis
a33cea0525
Refactors after proper rebase on master
2021-12-10 22:04:37 +01:00
Jakub Kuklis
b3728dbbaf
Rebase on master
2021-12-10 22:04:37 +01:00
Jakub Kuklis
b8ef47006c
Minor refactors before PR
2021-12-10 22:04:37 +01:00
Ryad ZENINE
c4421d6866
Small changes following review
2021-12-10 22:04:37 +01:00
Jakub Kuklis
7f595eedbc
Catch Blob Storage exceptions
2021-12-10 22:04:37 +01:00
Jakub Kuklis
9a00acfb72
Fixing read size error plus minor improvements
2021-12-10 22:04:37 +01:00
Jakub Kuklis
fd8ebe0d9f
Maximum read
2021-12-10 22:04:37 +01:00
Jakub Kuklis
0d7a4b10c3
Using unique_ptr for Blob Client in Read Buffer
2021-12-10 22:04:37 +01:00
Jakub Kuklis
12924b3659
Minor improvements
2021-12-10 22:04:37 +01:00
Jakub Kuklis
fa2261dee6
Changing offset in ReadBuffer to size_t
2021-12-10 22:04:37 +01:00
Jakub Kuklis
33247db3f4
Adding finalize() to WriteBuffer, always using block blob uploads, code cleanup
2021-12-10 22:04:37 +01:00
Jakub Kuklis
bf2476a076
Shared pointer for Azure client and fixing path reference in ReadBuffer
2021-12-10 22:04:37 +01:00
Jakub Kuklis
486f267b3b
Further code cleanup and error codes improvement
2021-12-10 22:04:37 +01:00
Jakub Kuklis
956348073b
ReadBufferFromBlobStorage operating like a proper stream, rather than a single download-read buffer
2021-12-10 22:04:37 +01:00
Jakub Kuklis
63ec8d4244
Adding multipart upload to WriteBuffer, retries in ReadBuffer, additional settings for buffers
2021-12-10 22:04:37 +01:00
Jakub Kuklis
dbb98e3636
Code cleanup, leaving some debug information printing under #ifdef VERBOSE_DEBUG_MODE
2021-12-10 22:04:37 +01:00
Jakub Kuklis
8230f8039c
Two column MergeTree table with Blob Storage disk work! Fixed buffer in ReadBufferFromBlobStorage to be permanent (class attribute)
2021-12-10 22:04:37 +01:00
Jakub Kuklis
2b4d3665ab
Basic inserts and selects work for a MergeTree table with a Blob Disk, based on a single column table
2021-12-10 22:04:37 +01:00
Jakub Kuklis
4b51ddee1b
Initial check with write and read works for the Blob Storage Disk
2021-12-10 22:04:37 +01:00
Jakub Kuklis
03f85dee41
Managed to read markers file in SELECT, getting error for CODEC version now
2021-12-10 22:04:37 +01:00
Jakub Kuklis
482a10e62e
Uploading to Blob Storage on INSERT seems to do something reasonable
2021-12-10 22:04:37 +01:00
Jakub Kuklis
c43b90c58c
Progressing with WriteBufferFromBlobStorage
2021-12-10 22:04:37 +01:00
Jakub Kuklis
ee1a568919
The following query passes without erors: CREATE TABLE ... ENGINE = MergeTree ... storage_policy = 'blob_storage', debug constants present in the code
2021-12-10 22:04:37 +01:00
Jakub Kuklis
0cf338e165
Skeleton for DiskBlobStorage
2021-12-10 22:04:36 +01:00
taiyang-li
02b877c978
move remote buffer from src/IO to src/Storages
2021-12-10 10:27:53 +08:00
Alexey Milovidov
dc3565b3fe
Quote more identifiers
2021-12-09 23:51:52 +03:00
Nikita Mikhaylov
dbf5091016
Parallel reading from replicas ( #29279 )
2021-12-09 13:39:28 +03:00
taiyang-li
0eef092ded
fix building
2021-12-09 16:57:33 +08:00
lgbo
38020a33e4
some refinement
2021-12-08 10:07:04 +08:00
Alexander Tokmakov
7fcb79ae72
Merge branch 'master' into mvcc_prototype
2021-12-07 14:39:29 +03:00
lgbo-ustc
54cc65ede8
fixed code style
2021-12-07 17:12:26 +08:00
lgbo-ustc
5a8c56bd55
fixed erros on creating dir on startup
2021-12-07 16:35:08 +08:00
lgbo-ustc
b6c230606e
fixed config error in server startup
2021-12-07 14:46:17 +08:00
Kseniia Sumarokova
926fd568c7
Merge pull request #32113 from FrankChen021/url_http_header
...
Set Content-Type in HTTP packets issued from URL engine
2021-12-07 08:52:36 +03:00
lgbo-ustc
15504a4bfa
fixed code style
2021-12-07 12:32:50 +08:00
lgbo-ustc
562a0499a1
fixed code style
2021-12-07 12:29:20 +08:00
lgbo-ustc
fcea033a07
fixed code style
2021-12-07 12:17:47 +08:00
lgbo-ustc
07045bc451
fixed a exception message
2021-12-07 11:18:44 +08:00
lgbo-ustc
0c1733817b
update LRUCache set()
2021-12-07 10:57:05 +08:00
lgbo-ustc
d4dcbd59fb
fixed code style
2021-12-06 19:59:15 +08:00
lgbo-ustc
dd70209623
fixed code style
2021-12-06 18:19:36 +08:00
lgbo-ustc
5913d67553
fixed code style
2021-12-06 16:51:49 +08:00
lgbo-ustc
ddac2a33bd
fixed code style
2021-12-06 16:46:13 +08:00
Frank Chen
59a701ee6e
Fix style
2021-12-06 16:31:47 +08:00
Frank Chen
bf3d3ea25c
Fix style
2021-12-06 16:20:10 +08:00
lgbo-ustc
3c56a5deac
refector
...
1. Make the lru cache pilicy in RemoteReadBufferCache into Common/UnreleasableLRUCache.h
2. If try to use local cached file fail, the RemoteReadBuffer::create() return the original ReadBuffer
3. Remove LocalFileReader
4. RemoteCacheController does not hold a context now, and the download process starts in RemoteReadBufferCache
once a new RemoteCacheController is create successly
5. Make an abstract class RemoteFileMetaDataBase for descripting a remote file meta data.
6. Remote file meta data need to a version, can cache multi-version for a file now
2021-12-06 16:17:32 +08:00
lgbo-ustc
01940c3f01
coding refinement
...
1. make remote file meta data be a abstract class
2. make cache controller download process be started by RemoteReadBufferCache
2021-12-06 10:30:59 +08:00
avogar
7549619b25
Improve skiping unknown fields with Quoted escaping rule in Template/CustomSeparated formats
2021-12-03 16:25:35 +03:00
taiyang-li
041e3074df
Merge branch 'hive_table' of https://github.com/bigo-sg/ClickHouse into bigo_hive_table
2021-12-02 20:23:39 +08:00
mergify[bot]
a1b31018d9
Merge branch 'master' into fix-lz4
2021-12-02 12:00:43 +00:00
Kruglov Pavel
049b2c0c14
Merge pull request #31072 from MaxWk/feature-support-bool-type
...
Feature support bool type
2021-12-02 14:50:29 +03:00
taiyang-li
2f4e7e1d4e
merge master
2021-12-02 19:48:21 +08:00
Anton Popov
a79dbb6b6a
Merge pull request #32079 from Avogar/parse-plus-in-float
...
Allow plus before Float32/Float64
2021-12-02 14:03:48 +03:00
lgbo-ustc
1c9d0b86cc
fixed code style
2021-12-02 18:19:15 +08:00
lgbo-ustc
c8f536a0a4
Use read/write buffer to read/write files
2021-12-02 18:11:35 +08:00
mergify[bot]
cba9f3f14c
Merge branch 'master' into url_http_header
2021-12-02 09:13:41 +00:00
frank chen
8e9f4949e4
Set Content-Type in HTTP packets issued from URL engine
...
Signed-off-by: frank chen <frank.chen021@outlook.com>
2021-12-02 15:49:34 +08:00
Kseniia Sumarokova
563b615904
Merge pull request #31766 from kssenii/odbc-fix
...
Fix crash with empty result on odbc query
2021-12-01 21:51:08 +03:00
avogar
e2eea4f361
Allow plus before Float32/Float64
2021-12-01 21:20:00 +03:00
Anton Popov
6f4d9a53b2
Merge remote-tracking branch 'origin/sparse-serialization' into HEAD
2021-12-01 15:54:33 +03:00
taiyang-li
701ad45aaa
Merge branch 'hive_table' of https://github.com/bigo-sg/ClickHouse into bigo_hive_table
2021-12-01 14:14:06 +08:00
taiyang-li
4aeadf3967
fix build error
2021-12-01 14:13:48 +08:00
lgbo-ustc
cce42ee311
fix logs
2021-12-01 11:32:58 +08:00
lgbo-ustc
0264afd7da
remove some comments
2021-12-01 11:03:33 +08:00
lgbo-ustc
aaa5d8f002
rewrite thread pool in remote file cache
2021-12-01 10:45:25 +08:00
MaxWk
02ce70f738
fix ci check
2021-12-01 00:13:11 +08:00
lgbo-ustc
81eafdb708
fixed
2021-11-30 19:09:09 +08:00
taiyang-li
d213500a3e
remove blank at end of line
2021-11-30 18:23:24 +08:00
lgbo-ustc
1014ca8c58
make two loops in clearing cached files into one
2021-11-30 17:57:58 +08:00
Kseniia Sumarokova
7e324d5894
Ping CI
2021-11-30 12:21:26 +03:00
lgbo-ustc
b023dd34c0
fixed code style
2021-11-30 17:11:58 +08:00
taiyang-li
cacf516e3e
calculate column value by default expression & apply defaults_for_omitted_fields_ in ArrowColumnToCHColumn
2021-11-30 14:52:26 +08:00
taiyang-li
ad6ba24efd
fix ArrowColumnToCHColumn
2021-11-30 10:49:57 +08:00
taiyang-li
6922f09ea3
reuse seekable read buffer with size
2021-11-29 20:19:36 +08:00
mergify[bot]
8d5460b469
Merge branch 'master' into feature-support-bool-type
2021-11-29 11:50:18 +00:00
taiyang-li
03e7f0f6f7
extract hivemetastore client from context
2021-11-29 18:23:57 +08:00
MaxWk
beaa5b5596
support TRUE/FALSE for vaules
2021-11-29 18:05:15 +08:00
avogar
bcf53c8876
Try to fix lz4 compression
2021-11-29 12:41:13 +03:00
taiyang-li
1c4efe5b13
fix code style of logger and exceptin
2021-11-29 17:01:34 +08:00
taiyang-li
7e0069aacb
fix code style
2021-11-29 16:24:45 +08:00
taiyang-li
c7a0fe467f
fix code stylke
2021-11-29 16:22:43 +08:00
lgbo-ustc
b28f3a2ad4
fixed code style
2021-11-29 15:00:41 +08:00
taiyang-li
83be8d28e6
fix code stylke
2021-11-29 14:50:33 +08:00
taiyang-li
fd4462db64
fix code style
2021-11-29 13:01:03 +08:00
taiyang-li
01f29fcf21
fix code style
2021-11-29 12:17:22 +08:00
taiyang-li
e8addd3243
fix code style
2021-11-29 12:01:52 +08:00
taiyang-li
85cb7fc23e
fix code style
2021-11-29 11:47:31 +08:00
taiyang-li
919b439b7c
remove useless code
2021-11-29 11:37:07 +08:00
taiyang-li
e67407b5c5
fix code style
2021-11-29 11:30:11 +08:00
taiyang-li
72f60cceb9
Merge branch 'master' into hive_table
2021-11-25 17:33:26 +08:00
kssenii
39588579dc
Fix
2021-11-25 12:26:26 +03:00
Kseniia Sumarokova
87b920594e
Merge pull request #31678 from ucasfl/log
...
fix typo
2021-11-25 11:29:36 +03:00
Kseniia Sumarokova
93cf66df12
Merge pull request #30936 from kssenii/seekable-read-buffers
...
Reduce memory usage for some formats when reading with s3/url/hdfs
2021-11-25 11:19:24 +03:00
alexey-milovidov
ac66433650
Merge pull request #31672 from ClickHouse/better-crosscompiling-freebsd
...
More easy cross-compiling for FreeBSD
2021-11-25 01:31:41 +03:00
taiyang-li
4d750c9dc5
fix code typo
2021-11-24 17:52:02 +08:00
taiyang-li
89dcef69d5
merge master
2021-11-24 14:38:04 +08:00
feng lv
89e9569ad6
fix typo
2021-11-24 04:34:25 +00:00
lgbo
ab0508dfc8
Update src/IO/RemoteReadBufferCache.h
...
fixed code style
Co-authored-by: Kseniia Sumarokova <54203879+kssenii@users.noreply.github.com>
2021-11-24 09:39:29 +08:00
Alexey Milovidov
e59e833cc6
Fix build
2021-11-24 03:11:22 +03:00
Alexey Milovidov
1acbafdffe
Remove outstandingly wrong code
2021-11-24 00:48:57 +03:00
lgbo
996d7125c0
Merge branch 'master' into hive_table
2021-11-23 10:19:02 +08:00
Anton Popov
ccd78e3838
Merge remote-tracking branch 'upstream/master' into HEAD
2021-11-22 17:19:35 +03:00
mergify[bot]
a7ba3e23a0
Merge branch 'master' into fix-write-buffers
2021-11-22 11:24:27 +00:00
avogar
b8695e8236
Fix comments
2021-11-22 14:19:26 +03:00
Kruglov Pavel
814a36ba69
Merge pull request #31434 from Avogar/custom-with-names-and-types
...
Add formats CustomSeparatedWithNames/WithNamesAndTypes
2021-11-22 13:24:00 +03:00
Kruglov Pavel
d9c1a0c8ec
Merge branch 'master' into fix-write-buffers
2021-11-20 17:48:24 +03:00
kssenii
ff969b4605
Merge branch 'master' of github.com:ClickHouse/ClickHouse into seekable-read-buffers
2021-11-20 15:03:13 +03:00
kssenii
f29ef7d18b
Add one more test
2021-11-20 15:01:45 +03:00
Kruglov Pavel
4856b50c23
Merge pull request #31409 from excitoon-favorites/s3envsegfaults
...
Resolve `nullptr` in STS credentials provider for S3
2021-11-19 18:45:38 +03:00
Kruglov Pavel
eb1cc834dc
Merge branch 'master' into custom-with-names-and-types
2021-11-19 16:51:25 +03:00
taiyang-li
e8644807fe
merge master and solve conflict
2021-11-19 15:01:58 +08:00
Vladimir Chebotarev
8b622bd4ff
Fixed a typo.
2021-11-18 10:54:59 +03:00
Alexander Tokmakov
06bed68f6e
Merge branch 'master' into mvcc_prototype
2021-11-17 23:03:19 +03:00
alexey-milovidov
1e058fb207
Merge pull request #30190 from azat/ReadBuffer-throw-on-empty
...
Do not allow zero-length reads
2021-11-17 10:21:08 +03:00
avogar
73d1918410
tmp
2021-11-16 17:10:30 +03:00
Vladimir Chebotarev
1e79177ce2
Build fix.
2021-11-16 12:31:51 +03:00
lgbo-ustc
92b8ec5758
fixed code style
2021-11-16 11:11:58 +08:00
lgbo-ustc
970d90ab15
fixed code style
2021-11-16 11:11:10 +08:00
Ilya Yatsishin
e5301ab712
Merge pull request #31286 from cfcz48/add_oss_support
...
Aliyun OSS Storage support
2021-11-15 17:00:27 +03:00
Alexander Tokmakov
ce2f692bb4
Merge branch 'master' into mvcc_prototype
2021-11-15 14:56:50 +03:00
lgbo-ustc
41d3ec2481
fixed typos
2021-11-15 19:08:58 +08:00
lgbo-ustc
78eefe2cac
fixed code style
2021-11-15 17:51:09 +08:00
avogar
8c87f8861c
Fix style
2021-11-15 12:48:28 +03:00
avogar
ba06953ee7
Fix tests
2021-11-15 12:47:00 +03:00
lgbo-ustc
a41a917b51
fixed : make recovering local cache meta informations run in backgroup
2021-11-15 16:47:12 +08:00
Vladimir Chebotarev
5e9710a26e
Attempt to resolve nullptr in STS credentials provider for S3.
2021-11-15 10:46:43 +03:00
kssenii
f4ffedd5f3
Better
2021-11-15 10:23:35 +03:00
lgbo-ustc
e2d928c3b6
fixed: change local path format
2021-11-15 14:25:57 +08:00
lgbo-ustc
d1eb628f0c
fixed: change local path format
2021-11-15 11:16:58 +08:00
lgbo-ustc
50156059aa
fixed: change local path format
2021-11-15 10:09:21 +08:00
Alexey Milovidov
052a7a92e8
Whitespace
2021-11-14 10:05:20 +03:00
alexey-milovidov
cb6342025d
Merge pull request #31229 from zhanglistar/fsync
...
Change all fsync to fdatasync
2021-11-14 05:16:01 +03:00
kssenii
f18dcd2287
Merge branch 'master' of github.com:ClickHouse/ClickHouse into seekable-read-buffers
2021-11-13 14:38:57 +03:00
mergify[bot]
22e7efa8b0
Merge branch 'master' into add_oss_support
2021-11-12 18:25:26 +00:00
lgbo-ustc
dcb65af181
fixed : remove compile warnings
2021-11-12 16:43:35 +08:00
zhanglistar
4e7823ade7
1. add macosx fix
2021-11-12 15:49:21 +08:00
lgbo-ustc
11c8729b80
fixed : code style
2021-11-12 14:30:31 +08:00
lgbo-ustc
d0de8ce961
fixed : code style
2021-11-12 13:56:06 +08:00
lgbo-ustc
a66a23c2e7
code modification based on the pr comment
2021-11-12 13:17:58 +08:00
taiyang-li
74b31ce6a9
fix code style
2021-11-12 10:23:27 +08:00
taiyang-li
abc5fa3331
Merge branch 'hive_table' of https://github.com/bigo-sg/ClickHouse into bigo_hive_table
2021-11-12 10:11:47 +08:00
taiyang-li
eb9074488c
fix code style
2021-11-12 10:11:17 +08:00
Kseniia Sumarokova
2313981fd7
Merge pull request #31260 from azat/external-cleanup
...
Cleanup extern ProfileEvents/CurrentMetrics and add a style check
2021-11-12 00:02:57 +03:00
Alexander Tokmakov
672157b817
Merge branch 'master' into mvcc_prototype
2021-11-11 22:01:34 +03:00
Kruglov Pavel
1c9105da8c
Update WriteBuffer.h
2021-11-11 21:00:33 +03:00
avogar
51831afff8
Fix tests
2021-11-11 20:27:23 +03:00
lgbo-ustc
9ad441c56d
rewrite codes base on comment
2021-11-11 19:50:08 +08:00
Kseniia Sumarokova
e20cb0c10a
Merge branch 'master' into retriable-http
2021-11-11 11:52:06 +03:00
cfcz48
d0425016e3
I hereby agree to the terms of the CLA available at: https://yandex.ru/legal/cla/?lang=en
...
Changelog category (leave one):
New Feature
Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):
Aliyun OSS Storage support.
2021-11-11 16:26:29 +08:00
lgbo-ustc
c99ae74d50
fixed: condition for path exists is wrong
2021-11-11 16:06:37 +08:00
lgbo
a4cd9f516b
fixed on pr
...
small code refinement base on the pr
2021-11-11 14:47:27 +08:00
taiyang-li
af78116c3f
Merge branch 'master' into hive_table
2021-11-11 11:52:28 +08:00
taiyang-li
deef4d4dbe
add options read_bool_as_uint8 when parse csv
2021-11-11 11:49:54 +08:00
avogar
c521a9131a
Small refactoring of WriteBiffer-s
2021-11-11 02:11:18 +03:00
kssenii
175f903624
Fix
2021-11-10 21:14:17 +00:00
Azat Khuzhin
f9428450cd
Fix OpenedFileCacheHits/OpenedFileCacheMisses
2021-11-10 21:15:27 +03:00
Azat Khuzhin
baf14444e6
Cleanup ProfileEvents and CurrentMetrics
2021-11-10 21:15:27 +03:00
zhanglistar
aaad3f84da
Merge branch 'ClickHouse:master' into fsync
2021-11-10 17:44:08 +08:00
zhanglistar
96b8df5899
1. chagne all fsync to fdatasync
2021-11-10 17:40:16 +08:00
kssenii
e166bfbf26
Save redirect result
2021-11-10 08:27:07 +00:00
kssenii
618c332f9c
Rename read_threadpool to threadpool
2021-11-10 07:46:18 +00:00
Alexander Tokmakov
4f542ab019
add test
2021-11-09 23:11:02 +03:00
Anton Popov
a20922b2d3
Merge remote-tracking branch 'origin/sparse-serialization' into HEAD
2021-11-09 15:36:25 +03:00
mergify[bot]
4f6fd49266
Merge branch 'master' into fix_IP_conversion
2021-11-09 12:11:54 +00:00
taiyang-li
b042e2b8f8
fix pvs check
2021-11-09 14:38:46 +08:00
kssenii
7e57368c82
Fix
2021-11-08 19:48:38 +00:00
Alexander Tokmakov
92eec74ad7
Merge branch 'master' into mvcc_prototype
2021-11-06 21:08:36 +03:00
taiyang-li
3a566eeeeb
fix typo
2021-11-05 20:27:37 +08:00
taiyang-li
36ca0b296b
implement hive table engine
2021-11-05 19:55:30 +08:00
kssenii
4bc2a29345
Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into retriable-http
2021-11-04 14:41:24 +00:00
kssenii
b95f0738fb
Fix
2021-11-04 14:39:53 +00:00
kssenii
ec11179f91
Merge branch 'master' of github.com:ClickHouse/ClickHouse into seekable-read-buffers
2021-11-03 14:33:31 +03:00
kssenii
45ea820297
Reduce memory usage for some formats
2021-11-03 14:30:03 +03:00
Kruglov Pavel
1f8535c02b
Merge branch 'master' into null-deserialization
2021-11-02 12:15:21 +03:00
kssenii
2940d9fd19
Squashed commit of the following:
...
commit a1acc7ed3485bd158cf42f26b2d2a1bca84b7269
Author: kssenii <sumarokovakseniia@mail.ru>
Date: Sun Oct 31 14:47:25 2021 +0000
Fix
commit 12a27d445bf17e321a7355f1ba857d11b70a285chttps://github.com/ClickHouse/ClickHouse into retriable-http
commit 55f1ba857daeba8104d182f3754b8a2647b88a667024f51b552bb586bed3https://github.com/ClickHouse/ClickHouse into retriable-http
commit 7024f51b557de5fca0751ecb6ad05d855b10261chttps://github.com/ClickHouse/ClickHouse into retriable-http
commit 1ecb6ad05d710ee9687828d2485716158b4c26b7https://github.com/ClickHouse/ClickHouse into retriable-http
commit 28d24857167c20ca07b7ccb02cac569e4fe0f3c8d1138a8a25https://github.com/ClickHouse/ClickHouse into retriable-http
commit 9e4fe0f3c89814cb1b45daed77038bhttps://github.com/ClickHouse/ClickHouse into retriable-http
commit 9814cb1b451c8b1b1133969999ff10https://github.com/ClickHouse/ClickHouse into retriable-http
commit 1c8b1b1133e7dbfff5d23329b668d6198adc7ecdhttps://github.com/ClickHouse/ClickHouse into retriable-http
2021-10-31 17:53:08 +03:00
Alexey Milovidov
aa737b7a11
Allow to parse Date as YYYYMMDD
2021-10-30 00:23:03 +03:00
Kseniia Sumarokova
bf290d0678
Merge pull request #30837 from kssenii/remove-redundant
...
Remove redundant from http buffer
2021-10-29 22:04:06 +03:00
Kseniia Sumarokova
b49797ad51
Update ReadBufferFromS3.cpp
2021-10-29 15:54:40 +03:00
kssenii
12a27d445b
Adjutments after merge with master
2021-10-29 09:29:35 +00:00
kssenii
b56c2f9499
Remove redundant lines from http buffer
2021-10-29 11:40:21 +03:00
kssenii
f17e321a73
Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into retriable-http
2021-10-29 08:03:25 +00:00
kssenii
55f1ba857d
Small fixes
2021-10-29 07:38:42 +00:00
kssenii
4f3433b4e1
Some clean up before merging
2021-10-29 01:04:52 +03:00
kssenii
7e2ea97e3c
Merge branch 'master' of github.com:ClickHouse/ClickHouse into disk-async-read
2021-10-28 18:46:21 +03:00
avogar
6e8c2ab28f
Use small amount of memory on stack in PeekableReadBuffer
2021-10-28 17:02:07 +03:00
kssenii
8dd5c1b99d
Remove http buffer update
2021-10-28 14:28:14 +03:00
kssenii
aeba8104d1
Some more fixes
2021-10-28 10:28:05 +00:00
kssenii
3d01fbf603
Review fixes
2021-10-28 11:32:37 +03:00
Alexey Milovidov
8b4a6a2416
Remove cruft
2021-10-28 02:10:39 +03:00
kssenii
82f3754b8a
Slightly better
2021-10-27 20:20:15 +00:00
kssenii
2647b88a66
Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into retriable-http
2021-10-27 19:38:19 +00:00
kssenii
7024f51b55
Review fixes and add test
2021-10-27 19:37:36 +00:00
Azat Khuzhin
a630821f60
Do not try to read empty files.
...
CI report [1].
[1]: https://clickhouse-test-reports.s3.yandex.net/30190/fe534553b2a0ac543795956b3fbde673cf5a342b/functional_stateless_tests_(debug).html#fail1
Fixes: 01560_ttl_remove_empty_parts in debug build (and some other tests)
v0: fix MergeTreeDataPartCompact::loadIndexGranularity()/MergeTreeDataPartWide::loadIndexGranularity()
v2: use EmptyReadBuffer in DiskLocal::readFile()
v3: introduce ReadBufferFromEmptyFile
v4: rebase against readbuffer-real-size branch
2021-10-27 22:25:03 +03:00
Azat Khuzhin
b0f9112696
Do not allow zero-length reads
...
Since this may create pretty odd issues, since reading 0 bytes will
return 0, and some code may not be ready for this.
v0: add a check in ReadBuffer ctor
v2: Do not create empty ReadBuffer from BufferWithOwnMemory with empty size
v3:
- revert "Do not create empty ReadBuffer from BufferWithOwnMemory with empty size"
- Replace INVALID_SETTING_VALUE with LOGICAL_ERROR
- Move the check for empty buffer in ReadBuffer into reading because of MMapReadBufferFromFile
v4: replace with assert of internal_buffer.size()
v5: move assertion to implementations since there are exceptions for
nested readers, like LimitReadBuffer and similar.
2021-10-27 22:25:03 +03:00
