thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-16 11:22:12 +00:00
Code Issues Packages Projects Releases Wiki Activity
94,952 Commits 9,338 Branches 1,783 Tags 2.1 GiB
627424889d
Commit Graph

1241 Commits

Author SHA1 Message Date
Anton Popov
dcc9cd0367
fix style check 2022-06-20 22:50:39 +02:00
Anton Popov
464cb59920 slightly better code 2022-06-20 16:22:36 +00:00
kssenii
5dd1bb2fd8 improvements for getFileSize 2022-06-20 15:22:56 +02:00
alesapin
7ae775d2db Review fixes 2022-06-20 14:01:05 +02:00
alesapin
e3184d264b Merge branch 'master' into disk_transaction 2022-06-20 13:47:30 +02:00
alesapin
ae8108d30b Add comments, better interface 2022-06-19 23:48:46 +02:00
Alexey Milovidov
04b7a34870 Maybe fix error 2022-06-19 05:21:59 +02:00
alesapin
93e465809d Fix style check 2022-06-17 16:49:49 +02:00
alesapin
7524500ca9 Remove logic from disk 2022-06-17 13:45:17 +02:00
alesapin
e5b405701a Merge branch 'master' into disk_transaction 2022-06-17 12:16:12 +02:00
alesapin
1c7a7da487
Merge pull request #38088 from kssenii/diff-with-cache 
Extract some diff from pr #36171
 2022-06-17 12:07:31 +02:00
alesapin
b11439ff36 Compiles 2022-06-16 14:24:26 +02:00
mergify[bot]
7f24574609
Merge branch 'master' into better-support-gcp 2022-06-15 23:18:20 +00:00
alesapin
11b6664863 WI{ 2022-06-16 00:06:52 +02:00
kssenii
500f49972b Extract diff from PR 36171 2022-06-15 16:40:18 +02:00
alesapin
1b66166501 Merge branch 'master' into disk_transaction 2022-06-15 13:54:10 +02:00
alesapin
b557c1dd80 Interm stage 2022-06-15 13:51:21 +02:00
Anton Popov
a59be0fd5d better support of GCP storage 2022-06-15 00:23:45 +00:00
alesapin
af1cd745e1
Merge pull request #37975 from kssenii/clean-up-broken-detached 
Clean up broken detached parts after timeout
 2022-06-14 20:53:31 +02:00
alesapin
0b7da674b3
Merge pull request #38021 from ClickHouse/fix-object-storage-version 
Use correct version in `DiskObjectStorageMetadata`
 2022-06-14 11:27:47 +02:00
kssenii
0d2dae0046 Add const, rename settings 2022-06-14 00:47:03 +02:00
kssenii
74cffaf186 Merge master 2022-06-14 00:45:19 +02:00
alesapin
267dd28e11 More consts for disks 2022-06-13 21:07:10 +02:00
Antonio Andelic
e9904501bc use correct version in DiskObjectStorageMetadata 2022-06-13 11:53:25 +00:00
alesapin
addfcb0dd3 Better implementation of s3 capabilities 2022-06-12 14:34:02 +02:00
alesapin
7a463c7d75 Merge branch 'master' into frew-master 2022-06-12 13:20:28 +02:00
kssenii
4859e11c27 Review fixes 2022-06-12 01:15:10 +02:00
Kseniia Sumarokova
580a30c6ae
Merge pull request #37859 from KinderRiven/support_max_request_cache_size 
Support to set max cache size for per query in local cache
 2022-06-10 16:17:18 +02:00
Kseniia Sumarokova
0ae2168fb6
Merge pull request #36328 from bigo-sg/async_hdfs_read_buffer 
Apply read_method 'threadpool' for StorageHive
 2022-06-10 15:04:21 +02:00
Robert Schulze
5f5732a2c4
Merge pull request #37969 from ClickHouse/consistent-macro-usage 
More consistent use of platform macros
 2022-06-10 14:10:01 +02:00
kssenii
956ac6b798 Fix after merging 2022-06-10 12:49:35 +02:00
kssenii
498f389c21 Merge master 2022-06-10 12:49:10 +02:00
kssenii
7a2676c7ab Clean up broken detached parts with timeout 2022-06-10 12:27:57 +02:00
Robert Schulze
1a0b5f33b3
More consistent use of platform macros 
cmake/target.cmake defines macros for the supported platforms, this
commit changes predefined system macros to our own macros.

__linux__ --> OS_LINUX
__APPLE__ --> OS_DARWIN
__FreeBSD__ --> OS_FREEBSD
 2022-06-10 10:22:31 +02:00
alesapin
99b78a7cb7 Fix stupid buge 2022-06-09 16:53:55 +02:00
alesapin
0cd82080b9 Fix stupid bug 2022-06-09 00:02:43 +02:00
alesapin
8aea4a698c Fix style-check 2022-06-08 15:07:11 +02:00
alesapin
378f9246ca Rename one method 2022-06-08 14:57:10 +02:00
alesapin
56a57e649f First try new interface 2022-06-08 14:56:51 +02:00
alesapin
81dca513de Review fixes 2022-06-08 13:28:33 +02:00
alesapin
b88e1a2bef Merge branch 'master' into metadata_storage 2022-06-08 13:02:21 +02:00
KinderRiven
55c390c86f fix 2022-06-08 01:42:46 +08:00
Azat Khuzhin
5d0a185cb3 Fix assertion for getImplementationBufferOffset() for Log family on S3 
Test test_log_family_s3/test.py::test_log_family_s3[TinyLog]:

    clickhouse: ./src/Disks/IO/AsynchronousReadIndirectBufferFromRemoteFS.cpp:213: virtual bool DB::AsynchronousReadIndirectBufferFromRemoteFS::nextImpl(): Assertion `file_offset_of_buffer_end == impl->getImplementationBufferOffset()' failed.

v2: fix assertion instead of adjusting file_offset_of_buffer_end in ReadBufferFromRemoteFSGather.cpp
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-06-07 20:41:26 +03:00
Anton Popov
df6882d2b9
Revert "Fix errors of CheckTriviallyCopyableMove type" 2022-06-07 13:53:10 +02:00
KinderRiven
f00914430d fix holder 2022-06-07 19:10:57 +08:00
taiyang-li
73a484256e Merge branch 'master' into async_hdfs_read_buffer 2022-06-07 12:16:46 +08:00
taiyang-li
c65c56fd48 fix typo 2022-06-07 09:58:29 +08:00
mergify[bot]
dd0603a595
Merge branch 'master' into metadata_storage 2022-06-06 18:59:45 +00:00
alesapin
71a7ae1274 Fix obvious bug 2022-06-06 12:09:29 +02:00
KinderRiven
bfe5150154 better 2022-06-06 16:48:30 +08:00
taiyang-li
b36d9f8143 refactor readinto 2022-06-06 12:58:22 +08:00
Robert Schulze
2d87af2a15
Merge pull request #37647 from DevTeamBK/Fix-all-CheckTriviallyCopyableMove-Errors 
Fix errors of CheckTriviallyCopyableMove type
 2022-06-05 19:58:47 +02:00
alesapin
5d186a438c Fix stupid code 2022-06-05 19:48:09 +02:00
alesapin
813da82d98 Fix style 2022-06-05 18:37:24 +02:00
alesapin
8ae74daf8a Fix bugs with abs path and create mode 2022-06-05 18:22:33 +02:00
KinderRiven
3ffb0b3549 support_max_query_cache_size 2022-06-05 16:21:36 +08:00
alesapin
a2704bc541 Trying to fix WAL log 2022-06-05 01:05:15 +02:00
alesapin
083b15fea0 bug impl 2022-06-04 17:33:07 +02:00
alesapin
2bf478ec6e Better locking 2022-06-04 16:16:13 +02:00
alesapin
41dc91ca8d Remove metadata from DiskObjectStorage 2022-06-04 15:54:57 +02:00
alesapin
0d5d7f5483 On the way 2022-06-03 20:54:29 +02:00
alesapin
83b9755098 Compiles 2022-06-03 20:54:29 +02:00
alesapin
5d30a4ef09 Fix typos and buge 2022-06-03 14:35:15 +02:00
alesapin
c7f7e91532 Fix stupid buge 2022-06-03 12:51:19 +02:00
alesapin
2132142ffb Tidy fixes 2022-06-03 12:45:49 +02:00
taiyang-li
f202c35311 Merge branch 'master' into async_hdfs_read_buffer 2022-06-03 17:52:09 +08:00
alesapin
111bde0436 Fix style 2022-06-02 18:51:41 +02:00
alesapin
c1b0769c6d Fix logical error 2022-06-02 18:27:31 +02:00
alesapin
4e4c95c5cc Fix build 2022-06-02 18:24:29 +02:00
alesapin
46bccae078 Trying to create metadata layer 2022-06-02 18:09:40 +02:00
HeenaBansal2009
e3080f2a97 Merge remote-tracking branch 'origin' into Fix-all-CheckTriviallyCopyableMove-Errors 2022-06-02 07:30:08 -07:00
Vladimir Chebotarev
d5022a0c01 Moved ClientConfigurationPerRequest from ClickHouse/aws-sdk-cpp#1 and ClickHouse/aws-sdk-cpp#2 to ClickHouse. 2022-06-02 06:07:01 +03:00
alesapin
a9af4a80b0 Add first simple implementation 2022-06-01 21:29:10 +02:00
taiyang-li
047387bf1c fix 2 bugs: 1. select count(1) from hive_table; 2. select _file, _path from hive_table 2022-05-31 17:39:02 +08:00
HeenaBansal2009
b7eb6bbd38 Fixed clang-tidy-CheckTriviallyCopyableMove-errors 2022-05-30 11:09:03 -07:00
Fred Wulff
537050ef8e Add support for DeleteObject when chunk_size == 0 2022-05-30 16:54:44 +00:00
alesapin
87baabb1a8 Followup fix 2022-05-30 13:34:42 +02:00
alesapin
362fa745e6 Ignore broken metadata 2022-05-30 13:32:12 +02:00
taiyang-li
dbb8a09825 merge master and solve conflict 2022-05-30 10:47:04 +08:00
alesapin
6f35c28592 Fix style 2022-05-30 00:29:30 +02:00
alesapin
c32b6076fb Remove stranges from code 2022-05-30 00:12:33 +02:00
alesapin
6d6779f17a
Merge pull request #37139 from ClickHouse/i_object_storage 
Separate object storage operations from disks
 2022-05-27 13:59:50 +02:00
Kseniia Sumarokova
141334448e
Merge pull request #37566 from kssenii/fix-assertion 
Fix failed assertion in cache
 2022-05-27 11:59:03 +02:00
Kseniia Sumarokova
f5d69506b4
Merge pull request #37516 from KinderRiven/improve_local_cache 
Control cache downloads to avoid negative optimization of local caches
 2022-05-27 11:53:17 +02:00
kssenii
36af6b1fa8 Fix assertion 2022-05-26 16:15:02 +02:00
alesapin
c862f89b8d Fix tidy 2022-05-26 15:43:21 +02:00
KinderRiven
824628c0da fix style 2022-05-26 16:51:16 +08:00
KinderRiven
822ecd982f better & support clean stash 2022-05-26 16:36:05 +08:00
alesapin
c7b16065e1 Merge with master 2022-05-25 21:47:05 +02:00
alesapin
6f5c86e55e Merge branch 'master' into i_object_storage 2022-05-25 20:49:01 +02:00
alesapin
0a3597da72
Merge pull request #34915 from ianton-ru/MDB-16962 
Fix collision of S3 operation log revision
 2022-05-25 18:15:31 +02:00
alesapin
164f8227f9 Fix memory sanitizer 2022-05-24 12:38:11 +02:00
alesapin
1e1e6d4fa0 Review fixes 2022-05-23 16:24:06 +02:00
alesapin
2064934e59
Update src/Disks/ObjectStorages/DiskObjectStorage.cpp 
Co-authored-by: Kseniia Sumarokova <54203879+kssenii@users.noreply.github.com>
 2022-05-23 16:02:29 +02:00
alesapin
1a78ea75bb
Update src/Disks/ObjectStorages/DiskObjectStorageMetadataHelper.h 
Co-authored-by: Kseniia Sumarokova <54203879+kssenii@users.noreply.github.com>
 2022-05-23 16:01:51 +02:00
alesapin
1b17086266
Update src/Disks/ObjectStorages/DiskObjectStorageMetadataHelper.h 
Co-authored-by: Kseniia Sumarokova <54203879+kssenii@users.noreply.github.com>
 2022-05-23 16:01:44 +02:00
alesapin
cef3d96cfe
Update src/Disks/ObjectStorages/S3/S3ObjectStorage.h 
Co-authored-by: Kseniia Sumarokova <54203879+kssenii@users.noreply.github.com>
 2022-05-23 14:46:25 +02:00
alesapin
216184dfd3 Rename one more file 2022-05-23 13:20:32 +02:00
taiyang-li
14f84f02d5 Merge branch 'master' into async_hdfs_read_buffer 2022-05-23 18:36:21 +08:00
alesapin
0c61579859 More comments 2022-05-22 23:15:42 +02:00
alesapin
06c3dd69c0 Move directories 2022-05-22 13:51:48 +02:00
alesapin
eb69d963e2 Missed change 2022-05-22 12:16:50 +02:00
Robert Schulze
19d53c14fa
Merge pull request #37382 from ClickHouse/wc++98-compat-extra-semi 
Enable -Wc++98-compat-extra-semi
 2022-05-22 09:40:45 +02:00
alesapin
b3bc0a18a0 fix test 2022-05-22 00:13:01 +02:00
alesapin
5b08edefd1 Fix I don't understand 2022-05-20 20:07:15 +02:00
alesapin
d414d85654 Fix hdfs bug 2022-05-20 17:43:48 +02:00
alesapin
dc4bc2908e Fix azure 2022-05-20 15:14:01 +02:00
Robert Schulze
0f6715bd91
Follow-up to PR #37300: semicolon warnings 
In PR #37300, Alexej asked why we the compiler does not warn about
unnecessary semicolons, e.g.

  f()
  {
  }; // <-- here

The answer is surprising: In C++98, above syntax was disallowed but by
most compilers accepted it regardless. C++>11 introduced "empty
declarations" which made the syntax legal.

The previous behavior can be restored using flag
-Wc++98-compat-extra-semi. This finds many useless semicolons which were
removed in this change. Unfortunately, there are also false positives
which would require #pragma-s and HAS_* logic (--> check_flags.cmake) to
suppress. In the end, -Wc++98-compat-extra-semi comes with extra effort
for little benefit. Therefore, this change only fixes some semicolons
but does not enable the flag.
 2022-05-20 15:06:34 +02:00
alesapin
6ff221a822 Merge with master 2022-05-20 12:07:34 +02:00
alesapin
654b27e307 Merge branch 'master' into i_object_storage 2022-05-20 11:56:13 +02:00
Kseniia Sumarokova
5b179bda3d
Merge pull request #37338 from xiedeyantu/optimize-cache 
optimize file segment PARTIALLY_DOWNLOADED get read buffer
 2022-05-20 11:11:56 +02:00
Sergei Trifonov
2450ab9079
Merge pull request #36686 from azat/disk-least-used 
Implement least_used load balancing algorithm for disks inside volume
 2022-05-20 09:16:07 +02:00
Alexander Tokmakov
c2762fd8b3
Merge pull request #37319 from lingo-xp/bug-fix 
[bug-fix] root_dir is not set in copyThroughBuffers
 2022-05-19 22:17:10 +03:00
alesapin
e278bfa81e Fix fast test build 2022-05-19 21:03:49 +02:00
alesapin
92c15ec97c Get rid of IDiskRemote 2022-05-19 20:07:15 +02:00
alesapin
c87c3fcfd9 Move Azure blob storage 2022-05-19 19:25:45 +02:00
xiedeyantu
6338368a1c optimize file segment getCacheReadBuffer 2022-05-19 20:14:52 +08:00
Azat Khuzhin
a350991284 Disks: More accurate free space calculation for least_used 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-05-19 14:10:11 +03:00
Azat Khuzhin
757894b10f Disks: Introduce IReservation::getUnreservedSpace() 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-05-19 14:10:11 +03:00
Azat Khuzhin
1d98913f90 Disks: Reduce number of statfs() calls for least_used disk load balancing policy 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-05-19 14:10:11 +03:00
Azat Khuzhin
ba26b3cf4c Implement lead_used load balancing algorithm for disks inside volume 
v2: rebase on top removed raid1
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-05-19 14:10:11 +03:00
Azat Khuzhin
4bc849b9f0 Disks: Remove unused src/Disks/SingleDiskVolume.cpp 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-05-19 14:10:11 +03:00
Azat Khuzhin
9d734bd677 Disks: Remove mentions about RAID1 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-05-19 14:10:11 +03:00
alesapin
41f7424ba6 Improve deletion logic 2022-05-18 17:13:17 +02:00
Alexander Tokmakov
7ab0aa7882
Update src/Disks/DiskLocal.cpp 2022-05-18 17:46:28 +03:00
Alexander Tokmakov
6098cf7cfe
Update src/Disks/DiskLocal.cpp 2022-05-18 17:46:23 +03:00
lingo-xp
46fa56c25c
[bug-fix] wrong default value in copyThroughBuffers 2022-05-18 18:02:15 +08:00
Robert Schulze
0c55ac76d2
A few clangtidy updates 
Enable:

- bugprone-lambda-function-name: "Checks for attempts to get the name of
  a function from within a lambda expression. The name of a lambda is
  always something like operator(), which is almost never what was
  intended."

- bugprone-unhandled-self-assignment: "Finds user-defined copy
  assignment operators which do not protect the code against
  self-assignment either by checking self-assignment explicitly or using
  the copy-and-swap or the copy-and-move method.""

- hicpp-invalid-access-moved: "Warns if an object is used after it has
  been moved."

- hicpp-use-noexcept: "This check replaces deprecated dynamic exception
  specifications with the appropriate noexcept specification (introduced
  in C++11)"

- hicpp-use-override: "Adds override (introduced in C++11) to overridden
  virtual functions and removes virtual from those functions as it is
  not required."

- performance-type-promotion-in-math-fn: "Finds calls to C math library
  functions (from math.h or, in C++, cmath) with implicit float to
  double promotions."

Split up:

- cppcoreguidelines-*. Some of them may be useful (haven't checked in
  detail), therefore allow to toggle them individually.

Disable:

- linuxkernel-*. Obvious.
 2022-05-17 20:56:57 +02:00
alesapin
824176f61d Merge branch 'master' into i_object_storage 2022-05-17 15:49:15 +02:00
alesapin
f9cd820854 Fix style 2022-05-17 15:39:46 +02:00
alesapin
1c90f326c2 Remove DiskHDFS 2022-05-17 15:25:05 +02:00
alesapin
4d4634b12d
Merge pull request #37164 from kssenii/ficx-async-reads-assertions 
fix async reads from remote fs internal setting not being always turned on
 2022-05-17 11:29:14 +02:00
taiyang-li
14ab7eb5a3 merge master and solve conflict 2022-05-17 16:28:08 +08:00
alesapin
466a02ba8f Follow HDFS 2022-05-16 21:31:40 +02:00
alesapin
ce6d5f7dcc Merge branch 'master' into i_object_storage 2022-05-16 17:42:22 +02:00
Robert Schulze
d66dcdad79
Fix new occurrences of new clang-tidy warnings 2022-05-16 11:31:36 +02:00
kssenii
f7493965b7 Remove additional check for merge tree 2022-05-16 11:19:57 +02:00
kssenii
49201f217f Revert 2022-05-16 11:17:56 +02:00
Robert Schulze
e3cfec5b09
Merge remote-tracking branch 'origin/master' into clangtidies 2022-05-16 10:12:50 +02:00
alesapin
4f1d4dde22 Disable s3 logging by default 2022-05-16 00:16:00 +02:00
Sergei Trifonov
811896446f
Update IVolume.h 2022-05-15 08:57:35 +02:00
alesapin
61014dbde6 Simlify strange test 2022-05-14 18:41:54 +02:00
alesapin
0e0b535b25 Fix failing test 2022-05-14 16:54:39 +02:00
alesapin
47834cbabe Satisfy clang-tidy and style check 2022-05-14 14:06:09 +02:00
alesapin
4540cf8925 Style check fixes 2022-05-14 13:46:47 +02:00
alesapin
cc304a3178 Merge branch 'master' into i_object_storage 2022-05-13 23:44:00 +02:00
alesapin
b6f4a1b9f9 Missed change 2022-05-13 17:21:04 +02:00
alesapin
eba60ff38f Fix restorable schema 2022-05-13 17:00:47 +02:00
kssenii
5b9b053c61 Fix encrypted 2022-05-13 13:22:29 +02:00
Kseniia Sumarokova
6d5137879b
Merge pull request #37054 from kssenii/better-read-from-cache 
Better read from cache buffer
 2022-05-13 12:20:06 +02:00