Robert Schulze
1a0b5f33b3
More consistent use of platform macros
...
cmake/target.cmake defines macros for the supported platforms, this
commit changes predefined system macros to our own macros.
__linux__ --> OS_LINUX
__APPLE__ --> OS_DARWIN
__FreeBSD__ --> OS_FREEBSD
2022-06-10 10:22:31 +02:00
alesapin
99b78a7cb7
Fix stupid buge
2022-06-09 16:53:55 +02:00
alesapin
0cd82080b9
Fix stupid bug
2022-06-09 00:02:43 +02:00
alesapin
8aea4a698c
Fix style-check
2022-06-08 15:07:11 +02:00
alesapin
378f9246ca
Rename one method
2022-06-08 14:57:10 +02:00
alesapin
56a57e649f
First try new interface
2022-06-08 14:56:51 +02:00
alesapin
81dca513de
Review fixes
2022-06-08 13:28:33 +02:00
alesapin
b88e1a2bef
Merge branch 'master' into metadata_storage
2022-06-08 13:02:21 +02:00
KinderRiven
55c390c86f
fix
2022-06-08 01:42:46 +08:00
Azat Khuzhin
5d0a185cb3
Fix assertion for getImplementationBufferOffset() for Log family on S3
...
Test test_log_family_s3/test.py::test_log_family_s3[TinyLog]:
clickhouse: ./src/Disks/IO/AsynchronousReadIndirectBufferFromRemoteFS.cpp:213: virtual bool DB::AsynchronousReadIndirectBufferFromRemoteFS::nextImpl(): Assertion `file_offset_of_buffer_end == impl->getImplementationBufferOffset()' failed.
v2: fix assertion instead of adjusting file_offset_of_buffer_end in ReadBufferFromRemoteFSGather.cpp
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-06-07 20:41:26 +03:00
Anton Popov
df6882d2b9
Revert "Fix errors of CheckTriviallyCopyableMove type"
2022-06-07 13:53:10 +02:00
KinderRiven
f00914430d
fix holder
2022-06-07 19:10:57 +08:00
taiyang-li
73a484256e
Merge branch 'master' into async_hdfs_read_buffer
2022-06-07 12:16:46 +08:00
taiyang-li
c65c56fd48
fix typo
2022-06-07 09:58:29 +08:00
mergify[bot]
dd0603a595
Merge branch 'master' into metadata_storage
2022-06-06 18:59:45 +00:00
alesapin
71a7ae1274
Fix obvious bug
2022-06-06 12:09:29 +02:00
KinderRiven
bfe5150154
better
2022-06-06 16:48:30 +08:00
taiyang-li
b36d9f8143
refactor readinto
2022-06-06 12:58:22 +08:00
Robert Schulze
2d87af2a15
Merge pull request #37647 from DevTeamBK/Fix-all-CheckTriviallyCopyableMove-Errors
...
Fix errors of CheckTriviallyCopyableMove type
2022-06-05 19:58:47 +02:00
alesapin
5d186a438c
Fix stupid code
2022-06-05 19:48:09 +02:00
alesapin
813da82d98
Fix style
2022-06-05 18:37:24 +02:00
alesapin
8ae74daf8a
Fix bugs with abs path and create mode
2022-06-05 18:22:33 +02:00
KinderRiven
3ffb0b3549
support_max_query_cache_size
2022-06-05 16:21:36 +08:00
alesapin
a2704bc541
Trying to fix WAL log
2022-06-05 01:05:15 +02:00
alesapin
083b15fea0
bug impl
2022-06-04 17:33:07 +02:00
alesapin
2bf478ec6e
Better locking
2022-06-04 16:16:13 +02:00
alesapin
41dc91ca8d
Remove metadata from DiskObjectStorage
2022-06-04 15:54:57 +02:00
alesapin
0d5d7f5483
On the way
2022-06-03 20:54:29 +02:00
alesapin
83b9755098
Compiles
2022-06-03 20:54:29 +02:00
alesapin
5d30a4ef09
Fix typos and buge
2022-06-03 14:35:15 +02:00
alesapin
c7f7e91532
Fix stupid buge
2022-06-03 12:51:19 +02:00
alesapin
2132142ffb
Tidy fixes
2022-06-03 12:45:49 +02:00
taiyang-li
f202c35311
Merge branch 'master' into async_hdfs_read_buffer
2022-06-03 17:52:09 +08:00
alesapin
111bde0436
Fix style
2022-06-02 18:51:41 +02:00
alesapin
c1b0769c6d
Fix logical error
2022-06-02 18:27:31 +02:00
alesapin
4e4c95c5cc
Fix build
2022-06-02 18:24:29 +02:00
alesapin
46bccae078
Trying to create metadata layer
2022-06-02 18:09:40 +02:00
HeenaBansal2009
e3080f2a97
Merge remote-tracking branch 'origin' into Fix-all-CheckTriviallyCopyableMove-Errors
2022-06-02 07:30:08 -07:00
Vladimir Chebotarev
d5022a0c01
Moved ClientConfigurationPerRequest
from ClickHouse/aws-sdk-cpp#1 and ClickHouse/aws-sdk-cpp#2 to ClickHouse.
2022-06-02 06:07:01 +03:00
alesapin
a9af4a80b0
Add first simple implementation
2022-06-01 21:29:10 +02:00
taiyang-li
047387bf1c
fix 2 bugs: 1. select count(1) from hive_table; 2. select _file, _path from hive_table
2022-05-31 17:39:02 +08:00
HeenaBansal2009
b7eb6bbd38
Fixed clang-tidy-CheckTriviallyCopyableMove-errors
2022-05-30 11:09:03 -07:00
Fred Wulff
537050ef8e
Add support for DeleteObject when chunk_size == 0
2022-05-30 16:54:44 +00:00
alesapin
87baabb1a8
Followup fix
2022-05-30 13:34:42 +02:00
alesapin
362fa745e6
Ignore broken metadata
2022-05-30 13:32:12 +02:00
taiyang-li
dbb8a09825
merge master and solve conflict
2022-05-30 10:47:04 +08:00
alesapin
6f35c28592
Fix style
2022-05-30 00:29:30 +02:00
alesapin
c32b6076fb
Remove stranges from code
2022-05-30 00:12:33 +02:00
alesapin
6d6779f17a
Merge pull request #37139 from ClickHouse/i_object_storage
...
Separate object storage operations from disks
2022-05-27 13:59:50 +02:00
Kseniia Sumarokova
141334448e
Merge pull request #37566 from kssenii/fix-assertion
...
Fix failed assertion in cache
2022-05-27 11:59:03 +02:00
Kseniia Sumarokova
f5d69506b4
Merge pull request #37516 from KinderRiven/improve_local_cache
...
Control cache downloads to avoid negative optimization of local caches
2022-05-27 11:53:17 +02:00
kssenii
36af6b1fa8
Fix assertion
2022-05-26 16:15:02 +02:00
alesapin
c862f89b8d
Fix tidy
2022-05-26 15:43:21 +02:00
KinderRiven
824628c0da
fix style
2022-05-26 16:51:16 +08:00
KinderRiven
822ecd982f
better & support clean stash
2022-05-26 16:36:05 +08:00
alesapin
c7b16065e1
Merge with master
2022-05-25 21:47:05 +02:00
alesapin
6f5c86e55e
Merge branch 'master' into i_object_storage
2022-05-25 20:49:01 +02:00
alesapin
0a3597da72
Merge pull request #34915 from ianton-ru/MDB-16962
...
Fix collision of S3 operation log revision
2022-05-25 18:15:31 +02:00
alesapin
164f8227f9
Fix memory sanitizer
2022-05-24 12:38:11 +02:00
alesapin
1e1e6d4fa0
Review fixes
2022-05-23 16:24:06 +02:00
alesapin
2064934e59
Update src/Disks/ObjectStorages/DiskObjectStorage.cpp
...
Co-authored-by: Kseniia Sumarokova <54203879+kssenii@users.noreply.github.com>
2022-05-23 16:02:29 +02:00
alesapin
1a78ea75bb
Update src/Disks/ObjectStorages/DiskObjectStorageMetadataHelper.h
...
Co-authored-by: Kseniia Sumarokova <54203879+kssenii@users.noreply.github.com>
2022-05-23 16:01:51 +02:00
alesapin
1b17086266
Update src/Disks/ObjectStorages/DiskObjectStorageMetadataHelper.h
...
Co-authored-by: Kseniia Sumarokova <54203879+kssenii@users.noreply.github.com>
2022-05-23 16:01:44 +02:00
alesapin
cef3d96cfe
Update src/Disks/ObjectStorages/S3/S3ObjectStorage.h
...
Co-authored-by: Kseniia Sumarokova <54203879+kssenii@users.noreply.github.com>
2022-05-23 14:46:25 +02:00
alesapin
216184dfd3
Rename one more file
2022-05-23 13:20:32 +02:00
taiyang-li
14f84f02d5
Merge branch 'master' into async_hdfs_read_buffer
2022-05-23 18:36:21 +08:00
alesapin
0c61579859
More comments
2022-05-22 23:15:42 +02:00
alesapin
06c3dd69c0
Move directories
2022-05-22 13:51:48 +02:00
alesapin
eb69d963e2
Missed change
2022-05-22 12:16:50 +02:00
Robert Schulze
19d53c14fa
Merge pull request #37382 from ClickHouse/wc++98-compat-extra-semi
...
Enable -Wc++98-compat-extra-semi
2022-05-22 09:40:45 +02:00
alesapin
b3bc0a18a0
fix test
2022-05-22 00:13:01 +02:00
alesapin
5b08edefd1
Fix I don't understand
2022-05-20 20:07:15 +02:00
alesapin
d414d85654
Fix hdfs bug
2022-05-20 17:43:48 +02:00
alesapin
dc4bc2908e
Fix azure
2022-05-20 15:14:01 +02:00
Robert Schulze
0f6715bd91
Follow-up to PR #37300 : semicolon warnings
...
In PR #37300 , Alexej asked why we the compiler does not warn about
unnecessary semicolons, e.g.
f()
{
}; // <-- here
The answer is surprising: In C++98, above syntax was disallowed but by
most compilers accepted it regardless. C++>11 introduced "empty
declarations" which made the syntax legal.
The previous behavior can be restored using flag
-Wc++98-compat-extra-semi. This finds many useless semicolons which were
removed in this change. Unfortunately, there are also false positives
which would require #pragma-s and HAS_* logic (--> check_flags.cmake) to
suppress. In the end, -Wc++98-compat-extra-semi comes with extra effort
for little benefit. Therefore, this change only fixes some semicolons
but does not enable the flag.
2022-05-20 15:06:34 +02:00
alesapin
6ff221a822
Merge with master
2022-05-20 12:07:34 +02:00
alesapin
654b27e307
Merge branch 'master' into i_object_storage
2022-05-20 11:56:13 +02:00
Kseniia Sumarokova
5b179bda3d
Merge pull request #37338 from xiedeyantu/optimize-cache
...
optimize file segment PARTIALLY_DOWNLOADED get read buffer
2022-05-20 11:11:56 +02:00
Sergei Trifonov
2450ab9079
Merge pull request #36686 from azat/disk-least-used
...
Implement least_used load balancing algorithm for disks inside volume
2022-05-20 09:16:07 +02:00
Alexander Tokmakov
c2762fd8b3
Merge pull request #37319 from lingo-xp/bug-fix
...
[bug-fix] root_dir is not set in copyThroughBuffers
2022-05-19 22:17:10 +03:00
alesapin
e278bfa81e
Fix fast test build
2022-05-19 21:03:49 +02:00
alesapin
92c15ec97c
Get rid of IDiskRemote
2022-05-19 20:07:15 +02:00
alesapin
c87c3fcfd9
Move Azure blob storage
2022-05-19 19:25:45 +02:00
xiedeyantu
6338368a1c
optimize file segment getCacheReadBuffer
2022-05-19 20:14:52 +08:00
Azat Khuzhin
a350991284
Disks: More accurate free space calculation for least_used
...
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-05-19 14:10:11 +03:00
Azat Khuzhin
757894b10f
Disks: Introduce IReservation::getUnreservedSpace()
...
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-05-19 14:10:11 +03:00
Azat Khuzhin
1d98913f90
Disks: Reduce number of statfs() calls for least_used disk load balancing policy
...
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-05-19 14:10:11 +03:00
Azat Khuzhin
ba26b3cf4c
Implement lead_used load balancing algorithm for disks inside volume
...
v2: rebase on top removed raid1
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-05-19 14:10:11 +03:00
Azat Khuzhin
4bc849b9f0
Disks: Remove unused src/Disks/SingleDiskVolume.cpp
...
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-05-19 14:10:11 +03:00
Azat Khuzhin
9d734bd677
Disks: Remove mentions about RAID1
...
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-05-19 14:10:11 +03:00
alesapin
41f7424ba6
Improve deletion logic
2022-05-18 17:13:17 +02:00
Alexander Tokmakov
7ab0aa7882
Update src/Disks/DiskLocal.cpp
2022-05-18 17:46:28 +03:00
Alexander Tokmakov
6098cf7cfe
Update src/Disks/DiskLocal.cpp
2022-05-18 17:46:23 +03:00
lingo-xp
46fa56c25c
[bug-fix] wrong default value in copyThroughBuffers
2022-05-18 18:02:15 +08:00
Robert Schulze
0c55ac76d2
A few clangtidy updates
...
Enable:
- bugprone-lambda-function-name: "Checks for attempts to get the name of
a function from within a lambda expression. The name of a lambda is
always something like operator(), which is almost never what was
intended."
- bugprone-unhandled-self-assignment: "Finds user-defined copy
assignment operators which do not protect the code against
self-assignment either by checking self-assignment explicitly or using
the copy-and-swap or the copy-and-move method.""
- hicpp-invalid-access-moved: "Warns if an object is used after it has
been moved."
- hicpp-use-noexcept: "This check replaces deprecated dynamic exception
specifications with the appropriate noexcept specification (introduced
in C++11)"
- hicpp-use-override: "Adds override (introduced in C++11) to overridden
virtual functions and removes virtual from those functions as it is
not required."
- performance-type-promotion-in-math-fn: "Finds calls to C math library
functions (from math.h or, in C++, cmath) with implicit float to
double promotions."
Split up:
- cppcoreguidelines-*. Some of them may be useful (haven't checked in
detail), therefore allow to toggle them individually.
Disable:
- linuxkernel-*. Obvious.
2022-05-17 20:56:57 +02:00
alesapin
824176f61d
Merge branch 'master' into i_object_storage
2022-05-17 15:49:15 +02:00
alesapin
f9cd820854
Fix style
2022-05-17 15:39:46 +02:00
alesapin
1c90f326c2
Remove DiskHDFS
2022-05-17 15:25:05 +02:00
alesapin
4d4634b12d
Merge pull request #37164 from kssenii/ficx-async-reads-assertions
...
fix async reads from remote fs internal setting not being always turned on
2022-05-17 11:29:14 +02:00
taiyang-li
14ab7eb5a3
merge master and solve conflict
2022-05-17 16:28:08 +08:00