thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-17 03:42:48 +00:00
Code Issues Packages Projects Releases Wiki Activity
86,455 Commits 9,315 Branches 1,783 Tags 2.1 GiB
cd83fd5f8a
Commit Graph

1622 Commits

Author SHA1 Message Date
Maksim Kita
c1ec85d87a Build with ENABLE_TESTS fix 2022-01-11 12:30:40 +03:00
Azat Khuzhin
cb70544dfe Move LockMemoryExceptionInThread and MemoryTrackerBlockerInThread 2022-01-10 22:39:10 +03:00
Alexander Tokmakov
158fbaaa29 fix 2022-01-10 20:07:01 +03:00
Alexander Tokmakov
bf2b6c2c37 Merge branch 'master' into mvcc_prototype 2022-01-10 19:53:50 +03:00
taiyang-li
83d064c24a fix unit test and build error 2022-01-06 11:41:24 +08:00
taiyang-li
2bb7ec8f72 Merge remote-tracking branch 'origin/master' into rocksdb_metacache 2022-01-06 10:44:17 +08:00
taiyang-li
c0a9c2b916 refactor metadatacache to reduce using of USE_ROCKSDB 2022-01-05 19:51:50 +08:00
Azat Khuzhin
2cdc5b37e3 Do not try to read pass EOF (to workaround a bug in a kernel) 
For unaligned offset pread() may return EINVAL even if the offset pass
EOF, although it should not, since otherwise there is no abiliity to
rely on read() == 0 is EOF (with pread() loop).

Here is a reproducer for the problem on 4.9.0-12-amd64:

    $ head -c27 /dev/urandom > /tmp/pread.issue
    $ xfs_io
    xfs_io> open -d /tmp/pread.issue
    xfs_io> pread 1000 4096
    pread: Invalid argument

And this is how it should work:

    xfs_io> pread 29 4096
    read 0/4096 bytes at offset 29

Note, here I use interactive mode since we had old xfs_io that does not
allow to execute multiple commands at once, and to avoid EMFILE issue

Here is some history of a patches that affects this behaviour in the
linux kernel:

- the issue had been introduced in
  torvalds/linux@9fe55eea7e v3.14
  ("Fix race when checking i_size on direct i/o read")
- an attempt to fix it had been made in
  torvalds/linux@74cedf9b6c v4.4
  ("direct-io: Fix negative return from dio read beyond eof")
- but this wasn't enough, since alignment check was earlier, so
  eventually fixed in
  torvalds/linux@41b21af388 v5.10
  ("direct-io: defer alignment check until after the EOF check")

Someone may ask why CI does not shows the issue, since:
- it had 4.19 kernel when CI was in yandex
- now it has 5.4 when CI is in AWS
Since both of those kernels does not have the last patch.

But, this bug requires the following conditions to met:
- index_granularity_bytes=0
- min_merge_bytes_to_use_direct_io=1
Which was not covered by CI yet.
 2022-01-04 10:53:17 +03:00
Azat Khuzhin
71a99ab27c Accept real file size in createReadBufferFromFileBase() 
Right now streams relies on correct file size not the number of bytes
that will be read from the stream, to overcome one bug in the linux
kernel that may return EIINVAL for pread() with offset pass the EOF.

v2: Swap read_hint and file_size (since it is easy to miss something)

Before the first argument to readFile()/createReadBufferFromFileBase()
was read_hint, not the file_size, and let's preserve the order, since
it is easy to miss something

This will also fix 02051_read_settings test automatically because now
MergeTreeReaderStream will pass estimated_sum_mark_range_bytes to
read_hint not file_size, previously it cause on of the following errors:
- Attempt to read after EOF w/ O_DIRECT
- and LOGICAL_ERROR while adjusting granulas w/o O_DIRECT

This will also improve zero-length reads guard (via
ReadBufferFromEmptyFile), that had been added in #30190

v3: fix for other storages that wasn't enabled in fast-test
v4: ignore ENOENT/ENOTSUP in readFile
 2022-01-04 10:53:17 +03:00
Azat Khuzhin
e548dae8d9 Pass size of the file to the readers 
v2: s/size/file_size/g
 2022-01-04 10:53:17 +03:00
taiyang-li
1e102bc1b2 merge master 2022-01-01 09:01:06 +08:00
Alexander Tokmakov
0d91887cdc save versions of parts 2021-12-30 16:15:28 +03:00
Nikolay Degterinsky
79e15e84d5 Refactor 2021-12-30 04:47:34 +00:00
Kruglov Pavel
489a30859f
Merge pull request #32455 from Avogar/schema-inference 
Automatic schema inference for input formats
 2021-12-29 21:03:48 +03:00
Nikolai Kochetov
68bf38133a Add pool to WriteBufferFromS3 2021-12-29 14:04:21 +00:00
avogar
8112a71233 Implement schema inference for most input formats 2021-12-29 12:18:56 +03:00
benbiti
9004e98992 parseDateTimeBestEffort support Unix Timestamp with Milliseconds 2021-12-29 13:43:54 +08:00
Alexander Tokmakov
8aec3ae94f Merge branch 'master' into mvcc_prototype 2021-12-28 14:24:36 +03:00
taiyang-li
9036b18c2f merge master 2021-12-27 15:12:48 +08:00
alexey-milovidov
47d50c3bd1
Merge pull request #33140 from kssenii/azure-tests-fix 
Fix azure blob storage tests failures
 2021-12-25 06:25:26 +03:00
alexey-milovidov
d15da636f6
Update ReadBufferFromAzureBlobStorage.cpp 2021-12-25 05:57:15 +03:00
alexey-milovidov
bcd4df789d
Update WriteBufferFromAzureBlobStorage.cpp 2021-12-25 05:56:15 +03:00
alexey-milovidov
f2075fac64
Update ReadBufferFromAzureBlobStorage.cpp 2021-12-25 05:55:49 +03:00
kssenii
b4420cfa3e Fix 2021-12-24 22:51:36 +03:00
Alexey Milovidov
29d28c531f Move code around to avoid dlsym on Musl 2021-12-24 12:25:27 +03:00
Kruglov Pavel
f38bc9fc38
Merge branch 'master' into fix-lz4 2021-12-23 16:13:26 +03:00
Nikolay Degterinsky
826f7cb0f5
Merge pull request #32240 from benbiti/master 
Fix unix timestamp Millisecond convert to DateTime64, fractional part calc reversed
 2021-12-23 12:11:53 +03:00
Kseniia Sumarokova
f0434f21ce
Merge pull request #33039 from kssenii/fix-azure-exception 
Fix exception in azure write buffer desctructor
 2021-12-23 10:26:40 +03:00
Vitaly Baranov
7660530fcc
Merge pull request #32955 from azat/read-fix 
Fix UB in case of unexpected EOF during filling a set from HTTP query
 2021-12-23 06:41:14 +03:00
Alexander Tokmakov
b7184732d5 Merge branch 'master' into mvcc_prototype 2021-12-22 19:46:09 +03:00
Dmitry Novik
d256c3a9ca
Merge pull request #29224 from ClickHouse/fix_IP_conversion 
Merging #27914
 2021-12-22 18:36:13 +03:00
mergify[bot]
04f727e84c
Merge branch 'master' into read-fix 2021-12-22 13:49:04 +00:00
kssenii
65c5334bb0 Safer 2021-12-22 14:22:14 +03:00
kssenii
060d3228e9 Fix 2021-12-22 13:43:21 +03:00
kssenii
d8ed4db6d4 Fix 2021-12-22 11:42:23 +03:00
Kseniia Sumarokova
484a4d62dd
Merge pull request #32835 from kssenii/fising-s3 
Fix optimization with lazy seek for async reads from remote fs
 2021-12-22 09:38:23 +03:00
benbiti
81a7a393c8 fix unix timestamp convert, fractional reversed 2021-12-22 01:32:16 +08:00
mergify[bot]
7da6d8b310
Merge branch 'master' into fix_IP_conversion 2021-12-21 12:16:05 +00:00
kssenii
ebcb66a361 Fix 2021-12-21 10:58:02 +03:00
taiyang-li
2597925724 merge master 2021-12-21 15:55:39 +08:00
Azat Khuzhin
4230f04f72 Fix UB in LimitReadBuffer in case of unexpected EOF 
Fixes the following possible assertions:
- ./src/IO/ReadBuffer.h:58: bool DB::ReadBuffer::next(): Assertion `!hasPendingData()' failed.
- ./src/IO/LimitReadBuffer.cpp:17: virtual bool DB::LimitReadBuffer::nextImpl(): Assertion `position() >= in->position()' failed.

Fixes: 02151_http_s_structure_set_eof
 2021-12-21 08:38:41 +03:00
Alexander Tokmakov
9cd49bc0ec Merge branch 'master' into mvcc_prototype 2021-12-20 22:06:22 +03:00
Nikolai Kochetov
35883e0dae
Merge pull request #32979 from ClickHouse/decrease-some-s3-log-level 
Decrease log level for some s3 messages.
 2021-12-20 20:56:22 +03:00
Maksim Kita
dd0d3de050
Merge pull request #32970 from kitaisreal/loops-remove-postfix-increment 
Loops remove postfix increment
 2021-12-20 19:51:07 +03:00
Vitaly Baranov
e61d3eef0c
Merge pull request #32747 from vitlibar/improve-grpc-compression 
Improve gRPC compression support
 2021-12-20 19:44:58 +03:00
Nikolai Kochetov
7dea7b7f76 Decrease log level for some s3 messages. 2021-12-20 18:18:54 +03:00
Maksim Kita
51477adf1b Updated additional cases 2021-12-20 15:55:07 +03:00
Vitaly Baranov
02d6cea857 Lz4DeflatingWriteBuffer now supports changes in the nested buffer between calls. 2021-12-19 16:29:57 +03:00
Anton Popov
99ebabd822 Merge remote-tracking branch 'upstream/master' into HEAD 2021-12-17 19:02:29 +03:00
kssenii
40c266840c Fixes 2021-12-17 18:16:29 +03:00
kssenii
ed42620530 Rename BlobStorage to AzureBlobStorage 2021-12-17 12:24:05 +03:00
Alexander Tokmakov
32e62ed5c2 Merge branch 'master' into mvcc_prototype 2021-12-17 10:43:46 +03:00
taiyang-li
d033fc4c24 merge master and fix conflict 2021-12-17 15:11:21 +08:00
kssenii
ca283a4bb8 Fix optimization with lazy seek 2021-12-17 00:18:28 +03:00
alesapin
c338bee718
Revert "Fix build issue related to azure blob storage" 2021-12-16 14:37:12 +03:00
alesapin
c0cebaea79
Merge pull request #32788 from amosbird/buildfix 
Fix build issue related to azure blob storage
 2021-12-16 13:45:22 +03:00
Kseniia Sumarokova
947ef81f53
Update ReadWriteBufferFromHTTP.h 2021-12-16 09:14:09 +03:00
mergify[bot]
41ae7c8eb1
Merge branch 'master' into fix_IP_conversion 2021-12-15 19:19:47 +00:00
Amos Bird
9c9f4a3e6c
build fix 2021-12-15 16:55:18 +08:00
mergify[bot]
4300f77532
Merge branch 'master' into fix_IP_conversion 2021-12-14 20:12:34 +00:00
Alexander Tokmakov
d7ad72838c Merge branch 'master' into mvcc_prototype 2021-12-14 23:07:52 +03:00
alesapin
9974f10631 Fix clang tidy 2021-12-14 15:34:20 +03:00
taiyang-li
ca3f7425a4 fix code 2021-12-14 17:37:31 +08:00
taiyang-li
8234d1176f merge master 2021-12-14 10:39:21 +08:00
alexey-milovidov
730c16bd0c
Merge pull request #32649 from vdimir/issue_32504 
Reset pipeline in BlockIO::onFinish
 2021-12-13 21:05:33 +03:00
alexey-milovidov
fd9d40925a
Merge pull request #32640 from ClickHouse/fix-function-file 
Fix surprisingly bad code in function `file`
 2021-12-13 19:34:36 +03:00
Kseniia Sumarokova
78ed383f15
Merge pull request #31505 from ContentSquare/azure_blob_storage_dependency 
Azure Blob Storage Disks
 2021-12-13 18:28:48 +03:00
vdimir
bda4ecbe90
Catch exceprion in ~ZlibDeflatingWriteBuffer 2021-12-13 12:29:35 +03:00
Alexey Milovidov
71926a3a97 Fix surprisingly bad code in function "file" 2021-12-13 07:57:54 +03:00
李扬
8675086104
Merge branch 'master' into hive_table 2021-12-12 09:01:46 -06:00
Jakub Kuklis
578aec97e9 Removing TODO and NOTE marks used mainly as hooks for review in the PR 2021-12-10 22:04:38 +01:00
Jakub Kuklis
85807adec4 Fixing variable shadow error 2021-12-10 22:04:38 +01:00
Jakub Kuklis
910db6ca57 Code cleanup addressing PR remarks 2021-12-10 22:04:38 +01:00
Jakub Kuklis
6acbec4455 Rebased on master 2021-12-10 22:04:38 +01:00
Jakub Kuklis
0c65cfed1f Addressing first PR remarks 2021-12-10 22:04:37 +01:00
Jakub Kuklis
7c2e116214 Adding max_retries config 2021-12-10 22:04:37 +01:00
Jakub Kuklis
a33cea0525 Refactors after proper rebase on master 2021-12-10 22:04:37 +01:00
Jakub Kuklis
b3728dbbaf Rebase on master 2021-12-10 22:04:37 +01:00
Jakub Kuklis
b8ef47006c Minor refactors before PR 2021-12-10 22:04:37 +01:00
Ryad ZENINE
c4421d6866 Small changes following review 2021-12-10 22:04:37 +01:00
Jakub Kuklis
7f595eedbc Catch Blob Storage exceptions 2021-12-10 22:04:37 +01:00
Jakub Kuklis
9a00acfb72 Fixing read size error plus minor improvements 2021-12-10 22:04:37 +01:00
Jakub Kuklis
fd8ebe0d9f Maximum read 2021-12-10 22:04:37 +01:00
Jakub Kuklis
0d7a4b10c3 Using unique_ptr for Blob Client in Read Buffer 2021-12-10 22:04:37 +01:00
Jakub Kuklis
12924b3659 Minor improvements 2021-12-10 22:04:37 +01:00
Jakub Kuklis
fa2261dee6 Changing offset in ReadBuffer to size_t 2021-12-10 22:04:37 +01:00
Jakub Kuklis
33247db3f4 Adding finalize() to WriteBuffer, always using block blob uploads, code cleanup 2021-12-10 22:04:37 +01:00
Jakub Kuklis
bf2476a076 Shared pointer for Azure client and fixing path reference in ReadBuffer 2021-12-10 22:04:37 +01:00
Jakub Kuklis
486f267b3b Further code cleanup and error codes improvement 2021-12-10 22:04:37 +01:00
Jakub Kuklis
956348073b ReadBufferFromBlobStorage operating like a proper stream, rather than a single download-read buffer 2021-12-10 22:04:37 +01:00
Jakub Kuklis
63ec8d4244 Adding multipart upload to WriteBuffer, retries in ReadBuffer, additional settings for buffers 2021-12-10 22:04:37 +01:00
Jakub Kuklis
dbb98e3636 Code cleanup, leaving some debug information printing under #ifdef VERBOSE_DEBUG_MODE 2021-12-10 22:04:37 +01:00
Jakub Kuklis
8230f8039c Two column MergeTree table with Blob Storage disk work! Fixed buffer in ReadBufferFromBlobStorage to be permanent (class attribute) 2021-12-10 22:04:37 +01:00
Jakub Kuklis
2b4d3665ab Basic inserts and selects work for a MergeTree table with a Blob Disk, based on a single column table 2021-12-10 22:04:37 +01:00
Jakub Kuklis
4b51ddee1b Initial check with write and read works for the Blob Storage Disk 2021-12-10 22:04:37 +01:00
Jakub Kuklis
03f85dee41 Managed to read markers file in SELECT, getting error for CODEC version now 2021-12-10 22:04:37 +01:00
Jakub Kuklis
482a10e62e Uploading to Blob Storage on INSERT seems to do something reasonable 2021-12-10 22:04:37 +01:00
Jakub Kuklis
c43b90c58c Progressing with WriteBufferFromBlobStorage 2021-12-10 22:04:37 +01:00
Jakub Kuklis
ee1a568919 The following query passes without erors: CREATE TABLE ... ENGINE = MergeTree ... storage_policy = 'blob_storage', debug constants present in the code 2021-12-10 22:04:37 +01:00
Jakub Kuklis
0cf338e165 Skeleton for DiskBlobStorage 2021-12-10 22:04:36 +01:00
taiyang-li
02b877c978 move remote buffer from src/IO to src/Storages 2021-12-10 10:27:53 +08:00
Alexey Milovidov
dc3565b3fe Quote more identifiers 2021-12-09 23:51:52 +03:00
Nikita Mikhaylov
dbf5091016
Parallel reading from replicas (#29279) 2021-12-09 13:39:28 +03:00
taiyang-li
0eef092ded fix building 2021-12-09 16:57:33 +08:00
lgbo
38020a33e4 some refinement 2021-12-08 10:07:04 +08:00
Alexander Tokmakov
7fcb79ae72 Merge branch 'master' into mvcc_prototype 2021-12-07 14:39:29 +03:00
lgbo-ustc
54cc65ede8 fixed code style 2021-12-07 17:12:26 +08:00
lgbo-ustc
5a8c56bd55 fixed erros on creating dir on startup 2021-12-07 16:35:08 +08:00
lgbo-ustc
b6c230606e fixed config error in server startup 2021-12-07 14:46:17 +08:00
Kseniia Sumarokova
926fd568c7
Merge pull request #32113 from FrankChen021/url_http_header 
Set Content-Type in HTTP packets issued from URL engine
 2021-12-07 08:52:36 +03:00
lgbo-ustc
15504a4bfa fixed code style 2021-12-07 12:32:50 +08:00
lgbo-ustc
562a0499a1 fixed code style 2021-12-07 12:29:20 +08:00
lgbo-ustc
fcea033a07 fixed code style 2021-12-07 12:17:47 +08:00
lgbo-ustc
07045bc451 fixed a exception message 2021-12-07 11:18:44 +08:00
lgbo-ustc
0c1733817b update LRUCache set() 2021-12-07 10:57:05 +08:00
lgbo-ustc
d4dcbd59fb fixed code style 2021-12-06 19:59:15 +08:00
lgbo-ustc
dd70209623 fixed code style 2021-12-06 18:19:36 +08:00
lgbo-ustc
5913d67553 fixed code style 2021-12-06 16:51:49 +08:00
lgbo-ustc
ddac2a33bd fixed code style 2021-12-06 16:46:13 +08:00
Frank Chen
59a701ee6e
Fix style 2021-12-06 16:31:47 +08:00
Frank Chen
bf3d3ea25c
Fix style 2021-12-06 16:20:10 +08:00
lgbo-ustc
3c56a5deac refector 
1. Make the lru cache pilicy in RemoteReadBufferCache into Common/UnreleasableLRUCache.h
2. If try to use local cached file fail, the RemoteReadBuffer::create() return the original ReadBuffer
3. Remove LocalFileReader
4. RemoteCacheController does not hold a context now, and the download process starts in RemoteReadBufferCache
   once a new RemoteCacheController is create successly
5. Make an abstract class RemoteFileMetaDataBase for descripting a remote file meta data.
6. Remote file meta data need to  a version, can cache multi-version for a file now
 2021-12-06 16:17:32 +08:00
lgbo-ustc
01940c3f01 coding refinement 
1. make remote file meta data be a abstract class
2. make cache controller download process be started by RemoteReadBufferCache
 2021-12-06 10:30:59 +08:00
avogar
7549619b25 Improve skiping unknown fields with Quoted escaping rule in Template/CustomSeparated formats 2021-12-03 16:25:35 +03:00
taiyang-li
041e3074df Merge branch 'hive_table' of https://github.com/bigo-sg/ClickHouse into bigo_hive_table 2021-12-02 20:23:39 +08:00
mergify[bot]
a1b31018d9
Merge branch 'master' into fix-lz4 2021-12-02 12:00:43 +00:00
Kruglov Pavel
049b2c0c14
Merge pull request #31072 from MaxWk/feature-support-bool-type 
Feature support bool type
 2021-12-02 14:50:29 +03:00
taiyang-li
2f4e7e1d4e merge master 2021-12-02 19:48:21 +08:00
Anton Popov
a79dbb6b6a
Merge pull request #32079 from Avogar/parse-plus-in-float 
Allow plus before Float32/Float64
 2021-12-02 14:03:48 +03:00
lgbo-ustc
1c9d0b86cc fixed code style 2021-12-02 18:19:15 +08:00
lgbo-ustc
c8f536a0a4 Use read/write buffer to read/write files 2021-12-02 18:11:35 +08:00
mergify[bot]
cba9f3f14c
Merge branch 'master' into url_http_header 2021-12-02 09:13:41 +00:00
frank chen
8e9f4949e4 Set Content-Type in HTTP packets issued from URL engine 
Signed-off-by: frank chen <frank.chen021@outlook.com>
 2021-12-02 15:49:34 +08:00
Kseniia Sumarokova
563b615904
Merge pull request #31766 from kssenii/odbc-fix 
Fix crash with empty result on odbc query
 2021-12-01 21:51:08 +03:00
avogar
e2eea4f361 Allow plus before Float32/Float64 2021-12-01 21:20:00 +03:00
Anton Popov
6f4d9a53b2 Merge remote-tracking branch 'origin/sparse-serialization' into HEAD 2021-12-01 15:54:33 +03:00
taiyang-li
701ad45aaa Merge branch 'hive_table' of https://github.com/bigo-sg/ClickHouse into bigo_hive_table 2021-12-01 14:14:06 +08:00
taiyang-li
4aeadf3967 fix build error 2021-12-01 14:13:48 +08:00
lgbo-ustc
cce42ee311 fix logs 2021-12-01 11:32:58 +08:00
lgbo-ustc
0264afd7da remove some comments 2021-12-01 11:03:33 +08:00
lgbo-ustc
aaa5d8f002 rewrite thread pool in remote file cache 2021-12-01 10:45:25 +08:00
MaxWk
02ce70f738 fix ci check 2021-12-01 00:13:11 +08:00
lgbo-ustc
81eafdb708 fixed 2021-11-30 19:09:09 +08:00
taiyang-li
d213500a3e remove blank at end of line 2021-11-30 18:23:24 +08:00
lgbo-ustc
1014ca8c58 make two loops in clearing cached files into one 2021-11-30 17:57:58 +08:00
Kseniia Sumarokova
7e324d5894
Ping CI 2021-11-30 12:21:26 +03:00
lgbo-ustc
b023dd34c0 fixed code style 2021-11-30 17:11:58 +08:00
taiyang-li
cacf516e3e calculate column value by default expression & apply defaults_for_omitted_fields_ in ArrowColumnToCHColumn 2021-11-30 14:52:26 +08:00
taiyang-li
ad6ba24efd fix ArrowColumnToCHColumn 2021-11-30 10:49:57 +08:00
taiyang-li
6922f09ea3 reuse seekable read buffer with size 2021-11-29 20:19:36 +08:00
mergify[bot]
8d5460b469
Merge branch 'master' into feature-support-bool-type 2021-11-29 11:50:18 +00:00
taiyang-li
03e7f0f6f7 extract hivemetastore client from context 2021-11-29 18:23:57 +08:00
MaxWk
beaa5b5596 support TRUE/FALSE for vaules 2021-11-29 18:05:15 +08:00
avogar
bcf53c8876 Try to fix lz4 compression 2021-11-29 12:41:13 +03:00
taiyang-li
1c4efe5b13 fix code style of logger and exceptin 2021-11-29 17:01:34 +08:00
taiyang-li
7e0069aacb fix code style 2021-11-29 16:24:45 +08:00
taiyang-li
c7a0fe467f fix code stylke 2021-11-29 16:22:43 +08:00
lgbo-ustc
b28f3a2ad4 fixed code style 2021-11-29 15:00:41 +08:00
taiyang-li
83be8d28e6 fix code stylke 2021-11-29 14:50:33 +08:00
taiyang-li
fd4462db64 fix code style 2021-11-29 13:01:03 +08:00
taiyang-li
01f29fcf21 fix code style 2021-11-29 12:17:22 +08:00
taiyang-li
e8addd3243 fix code style 2021-11-29 12:01:52 +08:00
taiyang-li
85cb7fc23e fix code style 2021-11-29 11:47:31 +08:00
taiyang-li
919b439b7c remove useless code 2021-11-29 11:37:07 +08:00
taiyang-li
e67407b5c5 fix code style 2021-11-29 11:30:11 +08:00
taiyang-li
72f60cceb9
Merge branch 'master' into hive_table 2021-11-25 17:33:26 +08:00
kssenii
39588579dc Fix 2021-11-25 12:26:26 +03:00
Kseniia Sumarokova
87b920594e
Merge pull request #31678 from ucasfl/log 
fix typo
 2021-11-25 11:29:36 +03:00
Kseniia Sumarokova
93cf66df12
Merge pull request #30936 from kssenii/seekable-read-buffers 
Reduce memory usage for some formats when reading with s3/url/hdfs
 2021-11-25 11:19:24 +03:00
alexey-milovidov
ac66433650
Merge pull request #31672 from ClickHouse/better-crosscompiling-freebsd 
More easy cross-compiling for FreeBSD
 2021-11-25 01:31:41 +03:00
taiyang-li
4d750c9dc5 fix code typo 2021-11-24 17:52:02 +08:00
taiyang-li
89dcef69d5 merge master 2021-11-24 14:38:04 +08:00
feng lv
89e9569ad6 fix typo 2021-11-24 04:34:25 +00:00
lgbo
ab0508dfc8
Update src/IO/RemoteReadBufferCache.h 
fixed code style

Co-authored-by: Kseniia Sumarokova <54203879+kssenii@users.noreply.github.com>
 2021-11-24 09:39:29 +08:00
Alexey Milovidov
e59e833cc6 Fix build 2021-11-24 03:11:22 +03:00
Alexey Milovidov
1acbafdffe Remove outstandingly wrong code 2021-11-24 00:48:57 +03:00
lgbo
996d7125c0
Merge branch 'master' into hive_table 2021-11-23 10:19:02 +08:00
Anton Popov
ccd78e3838 Merge remote-tracking branch 'upstream/master' into HEAD 2021-11-22 17:19:35 +03:00
mergify[bot]
a7ba3e23a0
Merge branch 'master' into fix-write-buffers 2021-11-22 11:24:27 +00:00
avogar
b8695e8236 Fix comments 2021-11-22 14:19:26 +03:00
Kruglov Pavel
814a36ba69
Merge pull request #31434 from Avogar/custom-with-names-and-types 
Add formats CustomSeparatedWithNames/WithNamesAndTypes
 2021-11-22 13:24:00 +03:00
Kruglov Pavel
d9c1a0c8ec
Merge branch 'master' into fix-write-buffers 2021-11-20 17:48:24 +03:00
kssenii
ff969b4605 Merge branch 'master' of github.com:ClickHouse/ClickHouse into seekable-read-buffers 2021-11-20 15:03:13 +03:00
kssenii
f29ef7d18b Add one more test 2021-11-20 15:01:45 +03:00
Kruglov Pavel
4856b50c23
Merge pull request #31409 from excitoon-favorites/s3envsegfaults 
Resolve `nullptr` in STS credentials provider for S3
 2021-11-19 18:45:38 +03:00
Kruglov Pavel
eb1cc834dc
Merge branch 'master' into custom-with-names-and-types 2021-11-19 16:51:25 +03:00
taiyang-li
e8644807fe merge master and solve conflict 2021-11-19 15:01:58 +08:00
Vladimir Chebotarev
8b622bd4ff Fixed a typo. 2021-11-18 10:54:59 +03:00
Alexander Tokmakov
06bed68f6e Merge branch 'master' into mvcc_prototype 2021-11-17 23:03:19 +03:00
alexey-milovidov
1e058fb207
Merge pull request #30190 from azat/ReadBuffer-throw-on-empty 
Do not allow zero-length reads
 2021-11-17 10:21:08 +03:00
avogar
73d1918410 tmp 2021-11-16 17:10:30 +03:00
Vladimir Chebotarev
1e79177ce2 Build fix. 2021-11-16 12:31:51 +03:00
lgbo-ustc
92b8ec5758 fixed code style 2021-11-16 11:11:58 +08:00
lgbo-ustc
970d90ab15 fixed code style 2021-11-16 11:11:10 +08:00
Ilya Yatsishin
e5301ab712
Merge pull request #31286 from cfcz48/add_oss_support 
Aliyun OSS Storage support
 2021-11-15 17:00:27 +03:00
Alexander Tokmakov
ce2f692bb4 Merge branch 'master' into mvcc_prototype 2021-11-15 14:56:50 +03:00
lgbo-ustc
41d3ec2481 fixed typos 2021-11-15 19:08:58 +08:00
lgbo-ustc
78eefe2cac fixed code style 2021-11-15 17:51:09 +08:00
avogar
8c87f8861c Fix style 2021-11-15 12:48:28 +03:00
avogar
ba06953ee7 Fix tests 2021-11-15 12:47:00 +03:00
lgbo-ustc
a41a917b51 fixed : make recovering local cache meta informations run in backgroup 2021-11-15 16:47:12 +08:00
Vladimir Chebotarev
5e9710a26e Attempt to resolve nullptr in STS credentials provider for S3. 2021-11-15 10:46:43 +03:00
kssenii
f4ffedd5f3 Better 2021-11-15 10:23:35 +03:00
lgbo-ustc
e2d928c3b6 fixed: change local path format 2021-11-15 14:25:57 +08:00
lgbo-ustc
d1eb628f0c fixed: change local path format 2021-11-15 11:16:58 +08:00
lgbo-ustc
50156059aa fixed: change local path format 2021-11-15 10:09:21 +08:00
Alexey Milovidov
052a7a92e8 Whitespace 2021-11-14 10:05:20 +03:00
alexey-milovidov
cb6342025d
Merge pull request #31229 from zhanglistar/fsync 
Change all fsync to fdatasync
 2021-11-14 05:16:01 +03:00
kssenii
f18dcd2287 Merge branch 'master' of github.com:ClickHouse/ClickHouse into seekable-read-buffers 2021-11-13 14:38:57 +03:00
mergify[bot]
22e7efa8b0
Merge branch 'master' into add_oss_support 2021-11-12 18:25:26 +00:00
lgbo-ustc
dcb65af181 fixed : remove compile warnings 2021-11-12 16:43:35 +08:00
zhanglistar
4e7823ade7 1. add macosx fix 2021-11-12 15:49:21 +08:00
lgbo-ustc
11c8729b80 fixed : code style 2021-11-12 14:30:31 +08:00
lgbo-ustc
d0de8ce961 fixed : code style 2021-11-12 13:56:06 +08:00
lgbo-ustc
a66a23c2e7 code modification based on the pr comment 2021-11-12 13:17:58 +08:00
taiyang-li
74b31ce6a9 fix code style 2021-11-12 10:23:27 +08:00
taiyang-li
abc5fa3331 Merge branch 'hive_table' of https://github.com/bigo-sg/ClickHouse into bigo_hive_table 2021-11-12 10:11:47 +08:00
taiyang-li
eb9074488c fix code style 2021-11-12 10:11:17 +08:00
Kseniia Sumarokova
2313981fd7
Merge pull request #31260 from azat/external-cleanup 
Cleanup extern ProfileEvents/CurrentMetrics and add a style check
 2021-11-12 00:02:57 +03:00
Alexander Tokmakov
672157b817 Merge branch 'master' into mvcc_prototype 2021-11-11 22:01:34 +03:00
Kruglov Pavel
1c9105da8c
Update WriteBuffer.h 2021-11-11 21:00:33 +03:00
avogar
51831afff8 Fix tests 2021-11-11 20:27:23 +03:00
lgbo-ustc
9ad441c56d rewrite codes base on comment 2021-11-11 19:50:08 +08:00
Kseniia Sumarokova
e20cb0c10a
Merge branch 'master' into retriable-http 2021-11-11 11:52:06 +03:00
cfcz48
d0425016e3 I hereby agree to the terms of the CLA available at: https://yandex.ru/legal/cla/?lang=en 
Changelog category (leave one):
New Feature

Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):
Aliyun OSS Storage support.
 2021-11-11 16:26:29 +08:00
lgbo-ustc
c99ae74d50 fixed: condition for path exists is wrong 2021-11-11 16:06:37 +08:00
lgbo
a4cd9f516b fixed on pr 
small code refinement base on the pr
 2021-11-11 14:47:27 +08:00
taiyang-li
af78116c3f
Merge branch 'master' into hive_table 2021-11-11 11:52:28 +08:00
taiyang-li
deef4d4dbe add options read_bool_as_uint8 when parse csv 2021-11-11 11:49:54 +08:00
avogar
c521a9131a Small refactoring of WriteBiffer-s 2021-11-11 02:11:18 +03:00
kssenii
175f903624 Fix 2021-11-10 21:14:17 +00:00
Azat Khuzhin
f9428450cd Fix OpenedFileCacheHits/OpenedFileCacheMisses 2021-11-10 21:15:27 +03:00
Azat Khuzhin
baf14444e6 Cleanup ProfileEvents and CurrentMetrics 2021-11-10 21:15:27 +03:00
zhanglistar
aaad3f84da
Merge branch 'ClickHouse:master' into fsync 2021-11-10 17:44:08 +08:00
zhanglistar
96b8df5899 1. chagne all fsync to fdatasync 2021-11-10 17:40:16 +08:00
kssenii
e166bfbf26 Save redirect result 2021-11-10 08:27:07 +00:00
kssenii
618c332f9c Rename read_threadpool to threadpool 2021-11-10 07:46:18 +00:00
Alexander Tokmakov
4f542ab019 add test 2021-11-09 23:11:02 +03:00
Anton Popov
a20922b2d3 Merge remote-tracking branch 'origin/sparse-serialization' into HEAD 2021-11-09 15:36:25 +03:00
mergify[bot]
4f6fd49266
Merge branch 'master' into fix_IP_conversion 2021-11-09 12:11:54 +00:00
taiyang-li
b042e2b8f8 fix pvs check 2021-11-09 14:38:46 +08:00
kssenii
7e57368c82 Fix 2021-11-08 19:48:38 +00:00
Alexander Tokmakov
92eec74ad7 Merge branch 'master' into mvcc_prototype 2021-11-06 21:08:36 +03:00
taiyang-li
3a566eeeeb fix typo 2021-11-05 20:27:37 +08:00
taiyang-li
36ca0b296b implement hive table engine 2021-11-05 19:55:30 +08:00
kssenii
4bc2a29345 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into retriable-http 2021-11-04 14:41:24 +00:00
kssenii
b95f0738fb Fix 2021-11-04 14:39:53 +00:00
kssenii
ec11179f91 Merge branch 'master' of github.com:ClickHouse/ClickHouse into seekable-read-buffers 2021-11-03 14:33:31 +03:00
kssenii
45ea820297 Reduce memory usage for some formats 2021-11-03 14:30:03 +03:00
Kruglov Pavel
1f8535c02b
Merge branch 'master' into null-deserialization 2021-11-02 12:15:21 +03:00
kssenii
2940d9fd19 Squashed commit of the following: 
commit a1acc7ed3485bd158cf42f26b2d2a1bca84b7269
Author: kssenii <sumarokovakseniia@mail.ru>
Date:   Sun Oct 31 14:47:25 2021 +0000

    Fix

commit 12a27d445b
Author: kssenii <sumarokovakseniia@mail.ru>
Date:   Fri Oct 29 09:24:53 2021 +0000

    Adjutments after merge with master

commit f17e321a73
Merge: 55f1ba857d 11b70a285c
Author: kssenii <sumarokovakseniia@mail.ru>
Date:   Fri Oct 29 08:03:25 2021 +0000

    Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into retriable-http

commit 55f1ba857d
Author: kssenii <sumarokovakseniia@mail.ru>
Date:   Fri Oct 29 07:38:42 2021 +0000

    Small fixes

commit aeba8104d1
Author: kssenii <sumarokovakseniia@mail.ru>
Date:   Thu Oct 28 10:28:05 2021 +0000

    Some more fixes

commit 82f3754b8a
Author: kssenii <sumarokovakseniia@mail.ru>
Date:   Wed Oct 27 20:20:15 2021 +0000

    Slightly better

commit 2647b88a66
Merge: 7024f51b55 2bb586bed3
Author: kssenii <sumarokovakseniia@mail.ru>
Date:   Wed Oct 27 19:38:19 2021 +0000

    Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into retriable-http

commit 7024f51b55
Author: kssenii <sumarokovakseniia@mail.ru>
Date:   Wed Oct 27 18:30:25 2021 +0000

    Review fixes and add test

commit 7de5fca075
Merge: 1ecb6ad05d 855b10261c
Author: kssenii <sumarokovakseniia@mail.ru>
Date:   Sun Oct 24 17:04:24 2021 +0000

    Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into retriable-http

commit 1ecb6ad05d
Author: kssenii <sumarokovakseniia@mail.ru>
Date:   Sat Oct 23 18:53:59 2021 +0000

    Better

commit 710ee96878
Merge: 28d2485716 158b4c26b7
Author: kssenii <sumarokovakseniia@mail.ru>
Date:   Mon Oct 18 08:31:25 2021 +0000

    Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into retriable-http

commit 28d2485716
Author: kssenii <sumarokovakseniia@mail.ru>
Date:   Fri Oct 15 17:27:26 2021 +0000

    Some fixes

commit 7c20ca07b7
Author: kssenii <sumarokovakseniia@mail.ru>
Date:   Fri Oct 15 12:59:24 2021 +0000

    Fix style check

commit ccb02cac56
Merge: 9e4fe0f3c8 d1138a8a25
Author: kssenii <sumarokovakseniia@mail.ru>
Date:   Fri Oct 15 12:36:30 2021 +0000

    Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into retriable-http

commit 9e4fe0f3c8
Merge: 9814cb1b45 daed77038b
Author: kssenii <sumarokovakseniia@mail.ru>
Date:   Tue Oct 12 09:58:15 2021 +0000

    Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into retriable-http

commit 9814cb1b45
Merge: 1c8b1b1133 969999ff10
Author: kssenii <sumarokovakseniia@mail.ru>
Date:   Tue Oct 12 09:49:08 2021 +0000

    Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into retriable-http

commit 1c8b1b1133
Author: kssenii <sumarokovakseniia@mail.ru>
Date:   Tue Oct 12 09:33:07 2021 +0000

    Better

commit e7dbfff5d2
Merge: 3329b668d6 198adc7ecd
Author: kssenii <sumarokovakseniia@mail.ru>
Date:   Tue Oct 12 06:19:15 2021 +0000

    Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into retriable-http
 2021-10-31 17:53:08 +03:00
Alexey Milovidov
aa737b7a11 Allow to parse Date as YYYYMMDD 2021-10-30 00:23:03 +03:00
Kseniia Sumarokova
bf290d0678
Merge pull request #30837 from kssenii/remove-redundant 
Remove redundant from http buffer
 2021-10-29 22:04:06 +03:00
Kseniia Sumarokova
b49797ad51
Update ReadBufferFromS3.cpp 2021-10-29 15:54:40 +03:00
kssenii
12a27d445b Adjutments after merge with master 2021-10-29 09:29:35 +00:00
kssenii
b56c2f9499 Remove redundant lines from http buffer 2021-10-29 11:40:21 +03:00
kssenii
f17e321a73 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into retriable-http 2021-10-29 08:03:25 +00:00
kssenii
55f1ba857d Small fixes 2021-10-29 07:38:42 +00:00
kssenii
4f3433b4e1 Some clean up before merging 2021-10-29 01:04:52 +03:00
kssenii
7e2ea97e3c Merge branch 'master' of github.com:ClickHouse/ClickHouse into disk-async-read 2021-10-28 18:46:21 +03:00
avogar
6e8c2ab28f Use small amount of memory on stack in PeekableReadBuffer 2021-10-28 17:02:07 +03:00
kssenii
8dd5c1b99d Remove http buffer update 2021-10-28 14:28:14 +03:00
kssenii
aeba8104d1 Some more fixes 2021-10-28 10:28:05 +00:00
kssenii
3d01fbf603 Review fixes 2021-10-28 11:32:37 +03:00
Alexey Milovidov
8b4a6a2416 Remove cruft 2021-10-28 02:10:39 +03:00
kssenii
82f3754b8a Slightly better 2021-10-27 20:20:15 +00:00
kssenii
2647b88a66 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into retriable-http 2021-10-27 19:38:19 +00:00
kssenii
7024f51b55 Review fixes and add test 2021-10-27 19:37:36 +00:00
Azat Khuzhin
a630821f60 Do not try to read empty files. 
CI report [1].

  [1]: https://clickhouse-test-reports.s3.yandex.net/30190/fe534553b2a0ac543795956b3fbde673cf5a342b/functional_stateless_tests_(debug).html#fail1

Fixes: 01560_ttl_remove_empty_parts in debug build (and some other tests)
v0: fix MergeTreeDataPartCompact::loadIndexGranularity()/MergeTreeDataPartWide::loadIndexGranularity()
v2: use EmptyReadBuffer in DiskLocal::readFile()
v3: introduce ReadBufferFromEmptyFile
v4: rebase against readbuffer-real-size branch
 2021-10-27 22:25:03 +03:00
Azat Khuzhin
b0f9112696 Do not allow zero-length reads 
Since this may create pretty odd issues, since reading 0 bytes will
return 0, and some code may not be ready for this.

v0: add a check in ReadBuffer ctor
v2: Do not create empty ReadBuffer from BufferWithOwnMemory with empty size
v3:
- revert "Do not create empty ReadBuffer from BufferWithOwnMemory with empty size"
- Replace INVALID_SETTING_VALUE with LOGICAL_ERROR
- Move the check for empty buffer in ReadBuffer into reading because of MMapReadBufferFromFile
v4: replace with assert of internal_buffer.size()
v5: move assertion to implementations since there are exceptions for
    nested readers, like LimitReadBuffer and similar.
 2021-10-27 22:25:03 +03:00
avogar
d1ef96a5ef Add test, avoid unnecessary allocations, use PeekableReadBuffer only in corner case 2021-10-27 17:29:15 +03:00
kssenii
52296e8b04 Merge branch 'master' of github.com:ClickHouse/ClickHouse into disk-async-read 2021-10-25 00:22:32 +03:00
kssenii
5b15e2474c Checkout reading from local fs back to master version 2021-10-24 20:41:18 +03:00
kssenii
7de5fca075 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into retriable-http 2021-10-24 17:04:24 +00:00
Azat Khuzhin
93c34824c0 Fix explicit 0 size for createReadBufferFromFileBase() 2021-10-24 18:58:22 +03:00
Azat Khuzhin
2b96433daf Replace estimated_size with optional<size> for IDisk::readFile() 
v2: rebase to fix conflicts in DiskEncrypted.cpp
 2021-10-24 18:58:11 +03:00
Vitaly Baranov
c0b01e7b0b
Merge pull request #30494 from vitlibar/fix-reading-from-empty-file-on-encrypted-disk 
Fix reading from empty file on encrypted disk
 2021-10-24 16:24:47 +03:00
kssenii
1ecb6ad05d Better 2021-10-23 19:00:14 +00:00
kssenii
39b9e9c258 Merge branch 'master' of github.com:ClickHouse/ClickHouse into disk-async-read 2021-10-22 15:40:41 +03:00
kssenii
56f0560c46 Asserts and read till end option 2021-10-21 20:54:03 +03:00
Vitaly Baranov
172a6f2e0d Fix reading from empty file on encrypted disk. 2021-10-21 19:31:29 +03:00
kssenii
120b01e089 Better 2021-10-21 18:22:17 +03:00
kssenii
e4e157688d Review fixes 2021-10-21 01:57:43 +03:00
alexey-milovidov
d3df1c02bc
Merge pull request #29672 from ClickHouse/clickhouse-obfuscator-2 
More enhancements for query obfuscator
 2021-10-20 23:06:29 +03:00
kssenii
5f24eb10ec Test changes in MergeTreeReader for local fs 2021-10-20 13:52:30 +03:00
kssenii
31cd71c849 Clean up 2021-10-20 09:37:18 +03:00
kssenii
143e912e4e Fix split build 2021-10-19 22:45:46 +03:00
Nikolai Kochetov
a92dc0a826 Update obsolete comments. 2021-10-19 12:58:10 +03:00
kssenii
d5d4817350 Add disk proxies back, add setReadUntilPosition to ReadBuffer 2021-10-18 18:35:11 +03:00
kssenii
710ee96878 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into retriable-http 2021-10-18 08:31:25 +00:00
Azat Khuzhin
0e34a9d550 Add MSan instrumentation for preadv2 
MSan report [1]:

    ==10672==WARNING: MemorySanitizer: use-of-uninitialized-value
        3 0x30dd6005 in DB::CompressionCodecFactory::get(unsigned char) const obj-x86_64-linux-gnu/../src/Compression/CompressionFactory.cpp:105:65
        4 0x30dab4d5 in DB::CompressedReadBufferBase::readCompressedData(unsigned long&, unsigned long&, bool) obj-x86_64-linux-gnu/../src/Compression/CompressedReadBufferBase.cpp:127:53
        5 0x30db54af in DB::CompressedReadBufferFromFile::readBig(char*, unsigned long) obj-x86_64-linux-gnu/../src/Compression/CompressedReadBufferFromFile.cpp:119:38

      Uninitialized value was stored to memory at
        0 0x9ef4e99 in __msan_memcpy (/src/ch/tmp/30191/clickhouse-msan+0x9ef4e99)
        1 0xa0c1999 in DB::ReadBuffer::read() obj-x86_64-linux-gnu/../src/IO/ReadBuffer.h:173:13
        2 0xa0c1999 in DB::ReadBuffer::readStrict() obj-x86_64-linux-gnu/../src/IO/ReadBuffer.h:184:27
        3 0x30daafd1 in DB::CompressedReadBufferBase::readCompressedData() obj-x86_64-linux-gnu/../src/Compression/CompressedReadBufferBase.cpp:120:20
        4 0x30db54af in DB::CompressedReadBufferFromFile::readBig() obj-x86_64-linux-gnu/../src/Compression/CompressedReadBufferFromFile.cpp:119:38

      Uninitialized value was created by a heap allocation
        0 0x9ef5285 in posix_memalign (/src/ch/tmp/30191/clickhouse-msan+0x9ef5285)
        1 0xa0cb98d in Allocator<false, false>::allocNoTrack(unsigned long, unsigned long) obj-x86_64-linux-gnu/../src/Common/Allocator.h:235:27
        2 0xa0caf0d in Allocator<false, false>::alloc(unsigned long, unsigned long) obj-x86_64-linux-gnu/../src/Common/Allocator.h:96:16
        3 0xa0caf0d in DB::Memory<Allocator<false, false> >::alloc() obj-x86_64-linux-gnu/../src/IO/BufferWithOwnMemory.h:126:49
        4 0xa0ca4a1 in DB::Memory<Allocator<false, false> >::Memory() obj-x86_64-linux-gnu/../src/IO/BufferWithOwnMemory.h:43:9
        5 0xa0ca4a1 in DB::BufferWithOwnMemory<DB::SeekableReadBuffer>::BufferWithOwnMemory(unsigned long, char*, unsigned long) obj-x86_64-linux-gnu/../src/IO/BufferWithOwnMemory.h:153:29
        6 0xa0ca4a1 in DB::ReadBufferFromFileBase::ReadBufferFromFileBase() obj-x86_64-linux-gnu/../src/IO/ReadBufferFromFileBase.cpp:11:7
        7 0xa59d980 in DB::AsynchronousReadBufferFromFileDescriptor::AsynchronousReadBufferFromFileDescriptor() obj-x86_64-linux-gnu/../src/IO/AsynchronousReadBufferFromFileDescriptor.h:40:11

    SUMMARY: MemorySanitizer: use-of-uninitialized-value obj-x86_64-linux-gnu/../contrib/libcxx/include/__hash_table:116:10 in std::__1::__constrain_hash(unsigned long, unsigned long)

  [1]: https://clickhouse-test-reports.s3.yandex.net/30191/18bb0e7d8d47dd694390f3f7b2ecc921a167afac/fuzzer_msan/report.html#fail1
 2021-10-17 23:54:58 +03:00
Azat Khuzhin
fb54d6b6e0 BufferWithOwnMemory: take reallocs into account 2021-10-17 23:54:58 +03:00
Azat Khuzhin
348033d8bd BufferWithOwnMemory: make size aligned not capacity 
This will fix pread_fake_async/pread_threadpool with min_bytes_to_use_direct_io
 2021-10-17 23:54:58 +03:00
Azat Khuzhin
9cc1178ebc BufferWithOwnMemory: do not try to align if buffer already aligned 2021-10-17 23:54:57 +03:00
Azat Khuzhin
34810456e1 Fix alignment for prefetch in AsynchronousReadBufferFromFileDescriptor 2021-10-17 23:54:57 +03:00
Alexey Milovidov
c2faf450d1 Fix error 2021-10-17 06:37:51 +03:00
Alexey Milovidov
04d8f611d2 Merge branch 'master' into clickhouse-obfuscator-2 2021-10-17 05:52:13 +03:00
Maksim Kita
de36038733
Merge pull request #30276 from azat/buffer-fwd-decl 
Use forward declaration for Buffer<> in generic headers
 2021-10-16 19:21:38 +03:00
alexey-milovidov
e9af03ef52
Merge pull request #30248 from ClickHouse/musl-4 
Preparation to build with Musl
 2021-10-16 18:44:35 +03:00
Azat Khuzhin
50231460af Use forward declaration for Buffer<> in generic headers 
- changes in ReadHelpers.h -- recompiles 1000 modules
- changes in FormatFactor.h -- recompiles 100 modules
 2021-10-16 12:03:24 +03:00