thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-18 12:22:12 +00:00
Code Issues Packages Projects Releases Wiki Activity
100,834 Commits 9,330 Branches 1,783 Tags 2.2 GiB
761274d857
Commit Graph

1527 Commits

Author SHA1 Message Date
Kseniia Sumarokova
61ce5161f3
Merge pull request #39106 from kssenii/cache-cacth-disk-full-on-cache-write 
fs cache: continue without exception when getting no disk space left error
 2022-07-13 19:08:13 +02:00
Sergei Trifonov
f854507729 merge master and resolve conflicts 2022-07-13 19:07:28 +02:00
alesapin
f81650e94a Fix extremely rare condition in hardlinks creation 2022-07-13 19:03:48 +02:00
kssenii
0944183eec Fix merge 2022-07-13 16:50:31 +02:00
Azat Khuzhin
ad6b50b087 Forbid defining non-default disk with default path from <path> 
Suppose you have the following configuration:

    <path>/var/lib/clickhouse/</path>
    <storage_configuration>
        <disks>
            <data><path>/var/lib/clickhouse/</path></data>
        </disks>
        <policies>
            <default>
                <volumes>
                    <main>
                        <disk>data</disk>
                    </main>
                </volumes>
            </default>
        </policies>
    </storage_configuration>

In this case disks will have two disks:
- 'data'    disk with path '/var/lib/clickhouse/'
- 'default' disk with path '/var/lib/clickhouse/'

And in this case MergeTree engine will complain on ATTACH for table that
uses 'default' policy:

    2022.06.20 07:49:15.165393 [ 242 ] {e8f50978-218a-426f-babc-637a8d03b1c6} <Error> TCPHandler: Code: 479. DB::Exception: Part `0_0_0_1` was found on disk `default` which is not defined in the storage policy. (UNKNOWN_DISK), Stack trace (when copying this message, always include the lines below):

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-07-13 17:40:08 +03:00
kssenii
0d2d3ac2b4 Merge master 2022-07-13 13:26:15 +02:00
Kseniia Sumarokova
d6c145e5e1
Update CachedReadBufferFromRemoteFS.cpp 2022-07-13 11:48:17 +02:00
kssenii
9973fb2f1d Fix merge 2022-07-13 11:19:14 +02:00
kssenii
6f29eada80 Merge master 2022-07-12 19:28:18 +02:00
Robert Schulze
8b6e31cc61
Merge pull request #37882 from excitoon-favorites/nodeleteobjects 
Fixes for objects removal in `S3ObjectStorage`
 2022-07-12 19:09:55 +02:00
kssenii
c38f726537 Remove redundant code 2022-07-12 18:28:20 +02:00
kssenii
4d48d6e713 STUPID BUG 2022-07-12 18:27:22 +02:00
kssenii
112a764794 Process no disk space left 
:wq# Please enter the commit message for your changes. Lines starting
 2022-07-11 22:12:40 +02:00
kssenii
e8daa50bad Fix tests 2022-07-11 21:24:18 +02:00
Sergei Trifonov
6ed777d2a8 add azure support 2022-07-11 19:19:57 +02:00
Kseniia Sumarokova
00dedc4e00
Merge pull request #39092 from ClickHouse/kssenii-patch-4 
Update registerDiskS3.cpp
 2022-07-11 18:05:28 +02:00
Sergei Trifonov
add0d1698e respect new limits in HDFS 2022-07-11 17:30:23 +02:00
Kseniia Sumarokova
e82485f65a
Update DiskObjectStorageTransaction.cpp 2022-07-11 16:50:09 +02:00
Sergei Trifonov
43779ec280 add max_remote_{read,write}_network_bandwidth_for_server settings 2022-07-11 14:59:39 +02:00
Kseniia Sumarokova
24f933cf75
Update registerDiskS3.cpp 2022-07-11 14:21:18 +02:00
Kseniia Sumarokova
3998a10c01
Update registerDiskS3.cpp 2022-07-11 14:19:09 +02:00
Vladimir Chebotarev
92880d7840 Fixed test. 2022-07-11 08:31:58 +03:00
kssenii
dd9651a6ca Better 2022-07-10 22:42:16 +02:00
Kseniia Sumarokova
18f93b15fc
Merge pull request #39070 from kssenii/fix-cache-bug 
Fix for fs cache
 2022-07-10 16:54:08 +02:00
kssenii
7501d5e59c Fix 2022-07-10 12:42:43 +02:00
Anton Popov
28d9fc44cf fix reading with mmap from Log storages 2022-07-09 23:16:16 +00:00
kssenii
4c88527c8e Review fixes 2022-07-09 19:35:07 +02:00
kssenii
bf1c99daab Refactoring of object storages, add LocalObjectStorage 2022-07-09 12:39:31 +02:00
Vladimir Chebotarev
367008b7fb Better objects removal in S3ObjectStorage. 2022-07-07 06:44:43 +03:00
kssenii
e55169ebad Fix 2022-07-05 20:15:35 +02:00
kssenii
2281f16c15 Fixes 2022-07-05 13:17:44 +02:00
kssenii
b52084265c Merge master 2022-07-04 21:37:43 +02:00
kssenii
ca53350f02 Adjustments for object storages 2022-07-04 20:09:57 +02:00
mergify[bot]
303200a592
Merge branch 'master' into fix-support-parallel-write 2022-07-04 14:24:10 +00:00
xiedeyantu
0a69b76180 add supportParallelWrite() in DiskDecorator 2022-07-04 17:21:07 +08:00
alesapin
65956975e2
Update registerDiskHDFS.cpp 2022-07-01 17:18:38 +02:00
alesapin
f47f761cc4
Update src/Disks/ObjectStorages/HDFS/registerDiskHDFS.cpp 
Co-authored-by: Dmitry Novik <n0vik@clickhouse.com>
 2022-07-01 16:04:20 +02:00
alesapin
27a76cb627 Overwrite in encrypted disk 2022-07-01 11:44:51 +02:00
alesapin
dfb0dfc4a7 Less diff 2022-06-30 23:11:20 +02:00
alesapin
09faa1641e Some comments 2022-06-30 23:09:32 +02:00
alesapin
b7c0d25528 Less code 2022-06-30 23:01:55 +02:00
alesapin
3afd0c4ab6 Remove redundant method 2022-06-30 22:57:39 +02:00
alesapin
eb5046ab26 Simplify everything 2022-06-30 22:51:27 +02:00
alesapin
1363b478d4 Merge branch 'fix_disk_tx_write_op' into better_data_part_storage_builder 2022-06-30 14:38:32 +02:00
alesapin
845264dd7b Followup 2022-06-30 14:36:18 +02:00
alesapin
dee6240494 Remove redundant 2022-06-30 14:34:32 +02:00
alesapin
c5943d0384 Better comment 2022-06-30 14:33:12 +02:00
alesapin
1ff4222661 Remove redundant changes 2022-06-30 14:33:09 +02:00
alesapin
9a89154352 Fix incorrect order of operations in disk transactions 
(cherry picked from commit 0d3904b788)
 2022-06-30 14:33:03 +02:00
alesapin
0d3904b788 Fix incorrect order of operations in disk transactions 2022-06-30 14:11:43 +02:00
alesapin
8726c7df26 Strangest fix for HDFS test 2022-06-30 01:21:49 +02:00
alesapin
c80a4c27be Merge branch 'master' into better_data_part_storage_builder 2022-06-29 12:33:42 +02:00
Robert Schulze
f692ead6ad
Don't use std::unique_lock unless we have to 
Replace where possible by std::lock_guard which is more light-weight.
 2022-06-28 19:19:06 +00:00
alesapin
6429b72371 Fixes 2022-06-28 14:41:22 +02:00
Kseniia Sumarokova
4a6af1ba81
Merge pull request #38466 from kssenii/follow-uo-iobjectstorage 
Follow up for #38436
 2022-06-28 12:06:42 +02:00
kssenii
be671ad36d Cherry pick 2022-06-27 16:54:55 +02:00
kssenii
c9faab4992 Fix 2022-06-27 16:48:06 +02:00
Kseniia Sumarokova
e515579cbe
Update IObjectStorage.h 2022-06-27 16:26:36 +02:00
kssenii
5c26f99bc5 Merge master 2022-06-27 13:36:41 +02:00
kssenii
e936a120fe Better 2022-06-27 13:09:23 +02:00
kssenii
5727671cac Fix tests 2022-06-27 00:34:57 +02:00
kssenii
a15238d147 Review fixes, fix hdfs tests 2022-06-26 16:51:08 +02:00
kssenii
8da6136f88 Fix 2022-06-26 13:21:05 +02:00
kssenii
725d80d470 get rid of path separation 2022-06-26 01:17:19 +02:00
kssenii
03f8b974ce get rid of path separation 2022-06-25 20:30:36 +02:00
kssenii
c0b4dac3ae Fix 2022-06-25 01:05:50 +02:00
kssenii
708b93a03b Merge master 2022-06-25 00:08:24 +02:00
kssenii
082c306ee8 Fix test, fix style 2022-06-25 00:04:21 +02:00
Anton Popov
5e89a7025c
Merge pull request #38279 from kssenii/add-cache-command 
Add describe query to show cache settings from config, add show caches query
 2022-06-24 16:31:12 +02:00
Alexander Tokmakov
cfe0e434e5
Revert "Add support for io_uring read method" 2022-06-24 12:19:41 +03:00
Alexey Milovidov
774de099cb
Merge pull request #38331 from excitoon/patch-18 
Fixed comments
 2022-06-24 04:33:40 +03:00
Alexey Milovidov
812ab9bd6b
Merge pull request #36103 from sauliusvl/uring 
Add support for io_uring read method
 2022-06-24 00:34:29 +03:00
kssenii
6fbd49f554 Merge master 2022-06-23 21:40:01 +02:00
Vladimir Chebotarev
3333dc2da1
Update S3Capabilities.h 2022-06-23 20:59:44 +03:00
kssenii
f7b329ee57 Merge master 2022-06-23 14:56:48 +02:00
Kseniia Sumarokova
2b883c2c01
Merge pull request #38285 from kssenii/extract-some-diff 
Extract some diff from #36171
 2022-06-23 14:38:03 +02:00
kssenii
120f2990ff Fix after merge 2022-06-23 14:34:05 +02:00
Vladimir Chebotarev
e2cdaa795b
Fixes comments. 2022-06-23 09:55:00 +03:00
Kseniia Sumarokova
87b3f331e2
Update MetadataStorageFromDiskTransactionOperations.cpp 2022-06-22 14:57:29 +02:00
alesapin
cc2f01c41d Fix name clash 2022-06-22 14:28:27 +02:00
kssenii
dd1346dab4 Avoid strange abort 2022-06-22 13:24:40 +02:00
alesapin
0f8eed98c4
Merge branch 'master' into better-support-gcp 2022-06-22 12:46:01 +02:00
kssenii
86f34af32f Correct merge 2022-06-22 12:19:19 +02:00
kssenii
4178abc0a5 Merge master 2022-06-21 22:38:51 +02:00
kssenii
90ce0b44f0 Fixes 2022-06-21 21:03:18 +02:00
kssenii
ac937c7fd6 Add comment 2022-06-21 19:21:36 +02:00
kssenii
ca09935773 Move into separate file 2022-06-21 18:58:05 +02:00
kssenii
481644939a Add describe cache command 2022-06-21 16:07:39 +02:00
alesapin
fafcb876be
Merge pull request #38182 from ClickHouse/disk_transaction 
Disk transaction
 2022-06-21 12:28:49 +02:00
Saulius Valatka
160c95a6b8 unpoisson memory read by io_uring, handle interrupted syscalls, do not throw from monitor thread 2022-06-21 13:20:21 +03:00
Saulius Valatka
7b5e13a9eb implement pending request queue to prevent CQ overflows, more error checking, reduce size of io_uring 2022-06-21 13:17:56 +03:00
Saulius Valatka
6bbabf59d9 simplify build, add read method randomized tests, fix typos 2022-06-21 09:59:46 +03:00
Saulius Valatka
68eef98128 check for io_uring support, fail if not supported 2022-06-21 09:59:46 +03:00
Saulius Valatka
0dc7854af1 add initial io_uring support 2022-06-21 09:59:46 +03:00
Anton Popov
dcc9cd0367
fix style check 2022-06-20 22:50:39 +02:00
Anton Popov
464cb59920 slightly better code 2022-06-20 16:22:36 +00:00
kssenii
5dd1bb2fd8 improvements for getFileSize 2022-06-20 15:22:56 +02:00
kssenii
90be49faa4 Merge master 2022-06-20 14:26:56 +02:00
alesapin
7ae775d2db Review fixes 2022-06-20 14:01:05 +02:00
alesapin
e3184d264b Merge branch 'master' into disk_transaction 2022-06-20 13:47:30 +02:00
alesapin
ae8108d30b Add comments, better interface 2022-06-19 23:48:46 +02:00
Alexey Milovidov
04b7a34870 Maybe fix error 2022-06-19 05:21:59 +02:00
alesapin
93e465809d Fix style check 2022-06-17 16:49:49 +02:00
alesapin
7524500ca9 Remove logic from disk 2022-06-17 13:45:17 +02:00
alesapin
e5b405701a Merge branch 'master' into disk_transaction 2022-06-17 12:16:12 +02:00
alesapin
1c7a7da487
Merge pull request #38088 from kssenii/diff-with-cache 
Extract some diff from pr #36171
 2022-06-17 12:07:31 +02:00
alesapin
b11439ff36 Compiles 2022-06-16 14:24:26 +02:00
mergify[bot]
7f24574609
Merge branch 'master' into better-support-gcp 2022-06-15 23:18:20 +00:00
alesapin
11b6664863 WI{ 2022-06-16 00:06:52 +02:00
kssenii
416c6008ee Merge master 2022-06-15 16:50:25 +02:00
kssenii
500f49972b Extract diff from PR 36171 2022-06-15 16:40:18 +02:00
alesapin
1b66166501 Merge branch 'master' into disk_transaction 2022-06-15 13:54:10 +02:00
alesapin
b557c1dd80 Interm stage 2022-06-15 13:51:21 +02:00
kssenii
e974ab61e6 Fix 2022-06-15 11:31:45 +02:00
Anton Popov
a59be0fd5d better support of GCP storage 2022-06-15 00:23:45 +00:00
alesapin
af1cd745e1
Merge pull request #37975 from kssenii/clean-up-broken-detached 
Clean up broken detached parts after timeout
 2022-06-14 20:53:31 +02:00
alesapin
0b7da674b3
Merge pull request #38021 from ClickHouse/fix-object-storage-version 
Use correct version in `DiskObjectStorageMetadata`
 2022-06-14 11:27:47 +02:00
kssenii
0d2dae0046 Add const, rename settings 2022-06-14 00:47:03 +02:00
kssenii
74cffaf186 Merge master 2022-06-14 00:45:19 +02:00
kssenii
9a5fa1f695 Remove unneeded changes 2022-06-13 22:04:57 +02:00
kssenii
4813b90281 Merge master 2022-06-13 21:32:00 +02:00
alesapin
267dd28e11 More consts for disks 2022-06-13 21:07:10 +02:00
kssenii
82629cc2be Continue 2022-06-13 16:32:07 +02:00
Antonio Andelic
e9904501bc use correct version in DiskObjectStorageMetadata 2022-06-13 11:53:25 +00:00
alesapin
addfcb0dd3 Better implementation of s3 capabilities 2022-06-12 14:34:02 +02:00
alesapin
7a463c7d75 Merge branch 'master' into frew-master 2022-06-12 13:20:28 +02:00
kssenii
4859e11c27 Review fixes 2022-06-12 01:15:10 +02:00
kssenii
aaf9a46f07 Merge master 2022-06-11 15:56:05 +02:00
Kseniia Sumarokova
580a30c6ae
Merge pull request #37859 from KinderRiven/support_max_request_cache_size 
Support to set max cache size for per query in local cache
 2022-06-10 16:17:18 +02:00
Kseniia Sumarokova
0ae2168fb6
Merge pull request #36328 from bigo-sg/async_hdfs_read_buffer 
Apply read_method 'threadpool' for StorageHive
 2022-06-10 15:04:21 +02:00
Robert Schulze
5f5732a2c4
Merge pull request #37969 from ClickHouse/consistent-macro-usage 
More consistent use of platform macros
 2022-06-10 14:10:01 +02:00
kssenii
956ac6b798 Fix after merging 2022-06-10 12:49:35 +02:00
kssenii
498f389c21 Merge master 2022-06-10 12:49:10 +02:00
kssenii
0fd2c4b2c0 Continuation 2022-06-10 12:33:29 +02:00
kssenii
7a2676c7ab Clean up broken detached parts with timeout 2022-06-10 12:27:57 +02:00
Robert Schulze
1a0b5f33b3
More consistent use of platform macros 
cmake/target.cmake defines macros for the supported platforms, this
commit changes predefined system macros to our own macros.

__linux__ --> OS_LINUX
__APPLE__ --> OS_DARWIN
__FreeBSD__ --> OS_FREEBSD
 2022-06-10 10:22:31 +02:00
alesapin
99b78a7cb7 Fix stupid buge 2022-06-09 16:53:55 +02:00
alesapin
0cd82080b9 Fix stupid bug 2022-06-09 00:02:43 +02:00
alesapin
8aea4a698c Fix style-check 2022-06-08 15:07:11 +02:00
alesapin
378f9246ca Rename one method 2022-06-08 14:57:10 +02:00
alesapin
56a57e649f First try new interface 2022-06-08 14:56:51 +02:00
alesapin
81dca513de Review fixes 2022-06-08 13:28:33 +02:00
alesapin
b88e1a2bef Merge branch 'master' into metadata_storage 2022-06-08 13:02:21 +02:00
KinderRiven
55c390c86f fix 2022-06-08 01:42:46 +08:00
Azat Khuzhin
5d0a185cb3 Fix assertion for getImplementationBufferOffset() for Log family on S3 
Test test_log_family_s3/test.py::test_log_family_s3[TinyLog]:

    clickhouse: ./src/Disks/IO/AsynchronousReadIndirectBufferFromRemoteFS.cpp:213: virtual bool DB::AsynchronousReadIndirectBufferFromRemoteFS::nextImpl(): Assertion `file_offset_of_buffer_end == impl->getImplementationBufferOffset()' failed.

v2: fix assertion instead of adjusting file_offset_of_buffer_end in ReadBufferFromRemoteFSGather.cpp
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-06-07 20:41:26 +03:00
Anton Popov
df6882d2b9
Revert "Fix errors of CheckTriviallyCopyableMove type" 2022-06-07 13:53:10 +02:00
KinderRiven
f00914430d fix holder 2022-06-07 19:10:57 +08:00
taiyang-li
73a484256e Merge branch 'master' into async_hdfs_read_buffer 2022-06-07 12:16:46 +08:00
taiyang-li
c65c56fd48 fix typo 2022-06-07 09:58:29 +08:00
mergify[bot]
dd0603a595
Merge branch 'master' into metadata_storage 2022-06-06 18:59:45 +00:00
alesapin
71a7ae1274 Fix obvious bug 2022-06-06 12:09:29 +02:00
KinderRiven
bfe5150154 better 2022-06-06 16:48:30 +08:00
taiyang-li
b36d9f8143 refactor readinto 2022-06-06 12:58:22 +08:00
Robert Schulze
2d87af2a15
Merge pull request #37647 from DevTeamBK/Fix-all-CheckTriviallyCopyableMove-Errors 
Fix errors of CheckTriviallyCopyableMove type
 2022-06-05 19:58:47 +02:00
alesapin
5d186a438c Fix stupid code 2022-06-05 19:48:09 +02:00
alesapin
813da82d98 Fix style 2022-06-05 18:37:24 +02:00
alesapin
8ae74daf8a Fix bugs with abs path and create mode 2022-06-05 18:22:33 +02:00
KinderRiven
3ffb0b3549 support_max_query_cache_size 2022-06-05 16:21:36 +08:00
alesapin
a2704bc541 Trying to fix WAL log 2022-06-05 01:05:15 +02:00
alesapin
083b15fea0 bug impl 2022-06-04 17:33:07 +02:00
alesapin
2bf478ec6e Better locking 2022-06-04 16:16:13 +02:00
alesapin
41dc91ca8d Remove metadata from DiskObjectStorage 2022-06-04 15:54:57 +02:00
alesapin
0d5d7f5483 On the way 2022-06-03 20:54:29 +02:00
alesapin
83b9755098 Compiles 2022-06-03 20:54:29 +02:00
kssenii
f41621736d Fixes 2022-06-03 17:03:05 +02:00
alesapin
5d30a4ef09 Fix typos and buge 2022-06-03 14:35:15 +02:00
alesapin
c7f7e91532 Fix stupid buge 2022-06-03 12:51:19 +02:00
alesapin
2132142ffb Tidy fixes 2022-06-03 12:45:49 +02:00
taiyang-li
f202c35311 Merge branch 'master' into async_hdfs_read_buffer 2022-06-03 17:52:09 +08:00
alesapin
111bde0436 Fix style 2022-06-02 18:51:41 +02:00
alesapin
c1b0769c6d Fix logical error 2022-06-02 18:27:31 +02:00
alesapin
4e4c95c5cc Fix build 2022-06-02 18:24:29 +02:00
alesapin
46bccae078 Trying to create metadata layer 2022-06-02 18:09:40 +02:00
kssenii
3d85391195 Better 2022-06-02 18:00:13 +02:00
kssenii
92a799a0c4 Add ObjectStorageProxy, CachedObjectStorage 2022-06-02 16:45:21 +02:00
HeenaBansal2009
e3080f2a97 Merge remote-tracking branch 'origin' into Fix-all-CheckTriviallyCopyableMove-Errors 2022-06-02 07:30:08 -07:00
Vladimir Chebotarev
d5022a0c01 Moved ClientConfigurationPerRequest from ClickHouse/aws-sdk-cpp#1 and ClickHouse/aws-sdk-cpp#2 to ClickHouse. 2022-06-02 06:07:01 +03:00
alesapin
a9af4a80b0 Add first simple implementation 2022-06-01 21:29:10 +02:00
kssenii
e3647585aa Fix tests 2022-06-01 19:58:28 +02:00
kssenii
aa60de8f63 Merge master 2022-06-01 15:32:01 +02:00
kssenii
53b6b76218 Better, add test 2022-05-31 15:33:56 +02:00
taiyang-li
047387bf1c fix 2 bugs: 1. select count(1) from hive_table; 2. select _file, _path from hive_table 2022-05-31 17:39:02 +08:00
kssenii
bfbc0e3184 Better, important fixes 2022-05-30 20:35:31 +02:00
HeenaBansal2009
b7eb6bbd38 Fixed clang-tidy-CheckTriviallyCopyableMove-errors 2022-05-30 11:09:03 -07:00
Fred Wulff
537050ef8e Add support for DeleteObject when chunk_size == 0 2022-05-30 16:54:44 +00:00
alesapin
87baabb1a8 Followup fix 2022-05-30 13:34:42 +02:00
alesapin
362fa745e6 Ignore broken metadata 2022-05-30 13:32:12 +02:00
taiyang-li
dbb8a09825 merge master and solve conflict 2022-05-30 10:47:04 +08:00
alesapin
6f35c28592 Fix style 2022-05-30 00:29:30 +02:00
alesapin
c32b6076fb Remove stranges from code 2022-05-30 00:12:33 +02:00
kssenii
b7f33dc3fe Fix 2022-05-29 23:42:36 +02:00
kssenii
b9618ea71c Fix checks 2022-05-28 01:17:08 +02:00
kssenii
a4e8fd59e1 Correct merge 2022-05-27 18:24:17 +02:00
kssenii
25b1ed526d Merge master 2022-05-27 16:50:45 +02:00
kssenii
b78455168e Continuation 2022-05-27 16:25:33 +02:00
alesapin
6d6779f17a
Merge pull request #37139 from ClickHouse/i_object_storage 
Separate object storage operations from disks
 2022-05-27 13:59:50 +02:00
Kseniia Sumarokova
141334448e
Merge pull request #37566 from kssenii/fix-assertion 
Fix failed assertion in cache
 2022-05-27 11:59:03 +02:00
Kseniia Sumarokova
f5d69506b4
Merge pull request #37516 from KinderRiven/improve_local_cache 
Control cache downloads to avoid negative optimization of local caches
 2022-05-27 11:53:17 +02:00
kssenii
cbb93fa0ce Continuation 2022-05-26 18:54:59 +02:00
kssenii
36af6b1fa8 Fix assertion 2022-05-26 16:15:02 +02:00
alesapin
c862f89b8d Fix tidy 2022-05-26 15:43:21 +02:00
KinderRiven
824628c0da fix style 2022-05-26 16:51:16 +08:00
KinderRiven
822ecd982f better & support clean stash 2022-05-26 16:36:05 +08:00
alesapin
c7b16065e1 Merge with master 2022-05-25 21:47:05 +02:00
kssenii
b89a4edc04 Merge master 2022-05-25 20:55:04 +02:00
alesapin
6f5c86e55e Merge branch 'master' into i_object_storage 2022-05-25 20:49:01 +02:00
kssenii
0dbc09ef17 Continuation 2022-05-25 20:28:46 +02:00
alesapin
0a3597da72
Merge pull request #34915 from ianton-ru/MDB-16962 
Fix collision of S3 operation log revision
 2022-05-25 18:15:31 +02:00
kssenii
0556237b68 Continuation 2022-05-25 16:49:40 +02:00
kssenii
35d2dec8d5 Fix 2022-05-24 16:15:55 +02:00
alesapin
164f8227f9 Fix memory sanitizer 2022-05-24 12:38:11 +02:00
alesapin
1e1e6d4fa0 Review fixes 2022-05-23 16:24:06 +02:00
alesapin
2064934e59
Update src/Disks/ObjectStorages/DiskObjectStorage.cpp 
Co-authored-by: Kseniia Sumarokova <54203879+kssenii@users.noreply.github.com>
 2022-05-23 16:02:29 +02:00
alesapin
1a78ea75bb
Update src/Disks/ObjectStorages/DiskObjectStorageMetadataHelper.h 
Co-authored-by: Kseniia Sumarokova <54203879+kssenii@users.noreply.github.com>
 2022-05-23 16:01:51 +02:00
alesapin
1b17086266
Update src/Disks/ObjectStorages/DiskObjectStorageMetadataHelper.h 
Co-authored-by: Kseniia Sumarokova <54203879+kssenii@users.noreply.github.com>
 2022-05-23 16:01:44 +02:00
alesapin
cef3d96cfe
Update src/Disks/ObjectStorages/S3/S3ObjectStorage.h 
Co-authored-by: Kseniia Sumarokova <54203879+kssenii@users.noreply.github.com>
 2022-05-23 14:46:25 +02:00
alesapin
216184dfd3 Rename one more file 2022-05-23 13:20:32 +02:00
taiyang-li
14f84f02d5 Merge branch 'master' into async_hdfs_read_buffer 2022-05-23 18:36:21 +08:00
alesapin
0c61579859 More comments 2022-05-22 23:15:42 +02:00
alesapin
06c3dd69c0 Move directories 2022-05-22 13:51:48 +02:00
alesapin
eb69d963e2 Missed change 2022-05-22 12:16:50 +02:00
Robert Schulze
19d53c14fa
Merge pull request #37382 from ClickHouse/wc++98-compat-extra-semi 
Enable -Wc++98-compat-extra-semi
 2022-05-22 09:40:45 +02:00
kssenii
9aa20193d5 Merge master 2022-05-22 00:56:20 +02:00
kssenii
ffd8d9723a Continuation 2022-05-22 00:21:40 +02:00
alesapin
b3bc0a18a0 fix test 2022-05-22 00:13:01 +02:00
alesapin
5b08edefd1 Fix I don't understand 2022-05-20 20:07:15 +02:00
alesapin
d414d85654 Fix hdfs bug 2022-05-20 17:43:48 +02:00
alesapin
dc4bc2908e Fix azure 2022-05-20 15:14:01 +02:00
Robert Schulze
0f6715bd91
Follow-up to PR #37300: semicolon warnings 
In PR #37300, Alexej asked why we the compiler does not warn about
unnecessary semicolons, e.g.

  f()
  {
  }; // <-- here

The answer is surprising: In C++98, above syntax was disallowed but by
most compilers accepted it regardless. C++>11 introduced "empty
declarations" which made the syntax legal.

The previous behavior can be restored using flag
-Wc++98-compat-extra-semi. This finds many useless semicolons which were
removed in this change. Unfortunately, there are also false positives
which would require #pragma-s and HAS_* logic (--> check_flags.cmake) to
suppress. In the end, -Wc++98-compat-extra-semi comes with extra effort
for little benefit. Therefore, this change only fixes some semicolons
but does not enable the flag.
 2022-05-20 15:06:34 +02:00
alesapin
6ff221a822 Merge with master 2022-05-20 12:07:34 +02:00
alesapin
654b27e307 Merge branch 'master' into i_object_storage 2022-05-20 11:56:13 +02:00
Kseniia Sumarokova
5b179bda3d
Merge pull request #37338 from xiedeyantu/optimize-cache 
optimize file segment PARTIALLY_DOWNLOADED get read buffer
 2022-05-20 11:11:56 +02:00
Sergei Trifonov
2450ab9079
Merge pull request #36686 from azat/disk-least-used 
Implement least_used load balancing algorithm for disks inside volume
 2022-05-20 09:16:07 +02:00
Alexander Tokmakov
c2762fd8b3
Merge pull request #37319 from lingo-xp/bug-fix 
[bug-fix] root_dir is not set in copyThroughBuffers
 2022-05-19 22:17:10 +03:00
alesapin
e278bfa81e Fix fast test build 2022-05-19 21:03:49 +02:00
alesapin
92c15ec97c Get rid of IDiskRemote 2022-05-19 20:07:15 +02:00
alesapin
c87c3fcfd9 Move Azure blob storage 2022-05-19 19:25:45 +02:00
xiedeyantu
6338368a1c optimize file segment getCacheReadBuffer 2022-05-19 20:14:52 +08:00
Azat Khuzhin
a350991284 Disks: More accurate free space calculation for least_used 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-05-19 14:10:11 +03:00
Azat Khuzhin
757894b10f Disks: Introduce IReservation::getUnreservedSpace() 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-05-19 14:10:11 +03:00
Azat Khuzhin
1d98913f90 Disks: Reduce number of statfs() calls for least_used disk load balancing policy 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-05-19 14:10:11 +03:00
Azat Khuzhin
ba26b3cf4c Implement lead_used load balancing algorithm for disks inside volume 
v2: rebase on top removed raid1
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-05-19 14:10:11 +03:00
Azat Khuzhin
4bc849b9f0 Disks: Remove unused src/Disks/SingleDiskVolume.cpp 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-05-19 14:10:11 +03:00
Azat Khuzhin
9d734bd677 Disks: Remove mentions about RAID1 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-05-19 14:10:11 +03:00
alesapin
41f7424ba6 Improve deletion logic 2022-05-18 17:13:17 +02:00
Alexander Tokmakov
7ab0aa7882
Update src/Disks/DiskLocal.cpp 2022-05-18 17:46:28 +03:00
Alexander Tokmakov
6098cf7cfe
Update src/Disks/DiskLocal.cpp 2022-05-18 17:46:23 +03:00
lingo-xp
46fa56c25c
[bug-fix] wrong default value in copyThroughBuffers 2022-05-18 18:02:15 +08:00
Robert Schulze
0c55ac76d2
A few clangtidy updates 
Enable:

- bugprone-lambda-function-name: "Checks for attempts to get the name of
  a function from within a lambda expression. The name of a lambda is
  always something like operator(), which is almost never what was
  intended."

- bugprone-unhandled-self-assignment: "Finds user-defined copy
  assignment operators which do not protect the code against
  self-assignment either by checking self-assignment explicitly or using
  the copy-and-swap or the copy-and-move method.""

- hicpp-invalid-access-moved: "Warns if an object is used after it has
  been moved."

- hicpp-use-noexcept: "This check replaces deprecated dynamic exception
  specifications with the appropriate noexcept specification (introduced
  in C++11)"

- hicpp-use-override: "Adds override (introduced in C++11) to overridden
  virtual functions and removes virtual from those functions as it is
  not required."

- performance-type-promotion-in-math-fn: "Finds calls to C math library
  functions (from math.h or, in C++, cmath) with implicit float to
  double promotions."

Split up:

- cppcoreguidelines-*. Some of them may be useful (haven't checked in
  detail), therefore allow to toggle them individually.

Disable:

- linuxkernel-*. Obvious.
 2022-05-17 20:56:57 +02:00
alesapin
824176f61d Merge branch 'master' into i_object_storage 2022-05-17 15:49:15 +02:00
kssenii
e4e6e44e86 Merge master 2022-05-17 15:47:33 +02:00
alesapin
f9cd820854 Fix style 2022-05-17 15:39:46 +02:00
alesapin
1c90f326c2 Remove DiskHDFS 2022-05-17 15:25:05 +02:00
alesapin
4d4634b12d
Merge pull request #37164 from kssenii/ficx-async-reads-assertions 
fix async reads from remote fs internal setting not being always turned on
 2022-05-17 11:29:14 +02:00
taiyang-li
14ab7eb5a3 merge master and solve conflict 2022-05-17 16:28:08 +08:00
kssenii
6a51dea791 Merge master 2022-05-16 22:13:22 +02:00
kssenii
5e3410f60a Better 2022-05-16 22:09:11 +02:00
alesapin
466a02ba8f Follow HDFS 2022-05-16 21:31:40 +02:00
alesapin
ce6d5f7dcc Merge branch 'master' into i_object_storage 2022-05-16 17:42:22 +02:00
kssenii
f3c1f3796b Some fixes 2022-05-16 16:16:38 +02:00
Robert Schulze
d66dcdad79
Fix new occurrences of new clang-tidy warnings 2022-05-16 11:31:36 +02:00
kssenii
f7493965b7 Remove additional check for merge tree 2022-05-16 11:19:57 +02:00
kssenii
49201f217f Revert 2022-05-16 11:17:56 +02:00
Robert Schulze
e3cfec5b09
Merge remote-tracking branch 'origin/master' into clangtidies 2022-05-16 10:12:50 +02:00
alesapin
4f1d4dde22 Disable s3 logging by default 2022-05-16 00:16:00 +02:00
Sergei Trifonov
811896446f
Update IVolume.h 2022-05-15 08:57:35 +02:00
alesapin
61014dbde6 Simlify strange test 2022-05-14 18:41:54 +02:00
alesapin
0e0b535b25 Fix failing test 2022-05-14 16:54:39 +02:00
kssenii
646cf4cc95 Better 2022-05-14 14:26:04 +02:00
alesapin
47834cbabe Satisfy clang-tidy and style check 2022-05-14 14:06:09 +02:00
alesapin
4540cf8925 Style check fixes 2022-05-14 13:46:47 +02:00
alesapin
cc304a3178 Merge branch 'master' into i_object_storage 2022-05-13 23:44:00 +02:00
kssenii
47ff7b6132 Fixes after merge 2022-05-13 19:22:22 +02:00
kssenii
58faaf0ea3 Merge master 2022-05-13 18:51:07 +02:00
alesapin
b6f4a1b9f9 Missed change 2022-05-13 17:21:04 +02:00
alesapin
eba60ff38f Fix restorable schema 2022-05-13 17:00:47 +02:00
kssenii
5b9b053c61 Fix encrypted 2022-05-13 13:22:29 +02:00
Kseniia Sumarokova
6d5137879b
Merge pull request #37054 from kssenii/better-read-from-cache 
Better read from cache buffer
 2022-05-13 12:20:06 +02:00
Antonio Andelic
e4cd605edb
Merge pull request #37100 from ucasfl/fix-s3 
fix wrong argument in proxy resolver of DiskS3
 2022-05-13 10:00:13 +02:00
Kseniia Sumarokova
320efb2221
Merge branch 'master' into better-read-from-cache 2022-05-12 23:35:39 +02:00
Kseniia Sumarokova
3fd9517ff8
Merge pull request #37130 from ClickHouse/kssenii-patch-3 
Update default remote fs read method in ReadSettings
 2022-05-12 23:34:28 +02:00
alesapin
d8580c8cb8 Digging 2022-05-12 19:51:04 +02:00
Kseniia Sumarokova
6e0a910fcd
Merge branch 'master' into better-read-from-cache 2022-05-12 19:44:18 +02:00
alesapin
e7296a2b28
Merge pull request #36825 from kssenii/cache-fix-1 
Removed forceful drop cache command, fix detached status state
 2022-05-12 17:18:21 +02:00
kssenii
483e056bcc Merge master 2022-05-12 14:58:24 +02:00
kssenii
3c241582db Fix assertions 2022-05-12 14:54:18 +02:00
alesapin
ea389eeaca Fix fast test build 2022-05-12 12:44:32 +02:00
Kseniia Sumarokova
59bfcd917b
Merge pull request #36876 from kssenii/extend-cache-log 
Log into filesystem_cache_log when cache not even attempted
 2022-05-12 12:38:36 +02:00
mergify[bot]
21b4effce8
Merge branch 'master' into fix-s3 2022-05-12 06:50:23 +00:00
alesapin
5dcd25be23 Initial implementation 2022-05-12 00:04:54 +02:00
kssenii
810967fae3 Remove the check back 2022-05-11 23:01:25 +02:00
kssenii
dc1edbe22f Fix 2022-05-11 15:48:32 +02:00
kssenii
0adfee3eca Move assertion one layer higher 2022-05-11 12:33:40 +02:00
kssenii
18dd74be0c Remove unneede changes 2022-05-11 10:26:30 +02:00
fenglv
175175a16e fix wrong argument in proxy resolver of DiskS3 2022-05-11 04:16:05 +00:00
kssenii
ccab49e3cf Merge master 2022-05-10 19:33:34 +02:00
kssenii
f4b45a5805 Better profile events 2022-05-10 16:00:53 +02:00
kssenii
2f0f76369a Better 2022-05-09 21:25:26 +02:00
Anton Ivashkin
6a6505c224 Merge master 2022-05-09 17:21:34 +03:00
Anton Ivashkin
5c72da83d3 Fix typos 2022-05-09 17:19:53 +03:00