Commit Graph

80 Commits

Author SHA1 Message Date
Alexey Milovidov
a5e3d4efb7 Added most of clang-static-analyzer checks 2020-03-20 21:53:35 +03:00
Pavel Kovalenko
f2dca656f9
MergeTree full support for S3 (#9646)
* IMergeDataPart full S3 support.

* MergeTreeData full S3 support.

* Compilation fixes.

* Mutations and merges S3 support.

* Fixed removing data part.

* MergeTree for S3 integration tests and fixes.

* Code style issues.

* Enable AWS logging.

* Fixed hardlink creation for DiskLocal.

* Fixed localBackup.cpp compilation.

* Fixed attaching partition.

* Get rid of extra methods in IDisk.

* Fixed storage config reloading.

* More tests with table manipulations.

* Remove unused error codes.

* Move localBackup to MergeTree folder.

* Minor fixes.
2020-03-19 19:37:55 +03:00
Alexey Milovidov
93466ce097 Added even more clang-tidy checks 2020-03-18 19:51:20 +03:00
Alexey Milovidov
1bd7e594b0 clang-tidy, part 2 2020-03-18 03:57:00 +03:00
Alexey Milovidov
1224866ea9 Merge branch 'master' into clang-tidy 2020-03-10 17:39:30 +03:00
Nikita Mikhaylov
b15532dbdf
Merge pull request #9415 from Jokser/merge-tree-s3-initial
Storage MergeTree initial support for S3.
2020-03-10 13:22:44 +03:00
Alexey Milovidov
d1025b2dc7 clang-tidy, part 16 2020-03-09 05:10:20 +03:00
Alexey Milovidov
c24333a437 clang-tidy, part 7 2020-03-09 01:38:12 +03:00
Vladimir Chebotarev
1a2122a854 Fixed wrong log messages about missing default disk or policy. 2020-03-06 00:03:59 +03:00
Pavel Kovalenko
8d07883b33 Merge branch 'master' into merge-tree-s3-initial
# Conflicts:
#	dbms/src/Storages/MergeTree/MergeTreeData.cpp
#	dbms/src/Storages/StorageMergeTree.cpp
2020-03-05 23:05:31 +03:00
Pavel Kovalenko
caf26d8aff IDisk get/set last modified time implementation. 2020-03-05 17:02:15 +03:00
Vladimir Chebotarev
db87e6f668 Fixed style. 2020-03-05 11:21:53 +03:00
Vladimir Chebotarev
2bd6474379 Adding storage policies and disks while reloading configuration. 2020-03-04 15:10:17 +03:00
Pavel Kovalenko
98cd70ced3 MergeTree for S3 more coverage. 2020-02-28 20:14:55 +03:00
Pavel Kovalenko
8c09902310 Code style issues. 2020-02-27 20:57:49 +03:00
Pavel Kovalenko
44226a61cf Storage MergeTree initial support for S3. 2020-02-27 19:47:40 +03:00
Alexey Milovidov
425ed714eb Merge branch 'master' of github.com:yandex/ClickHouse into errorcodes-style 2020-02-26 17:16:22 +03:00
Alexey Milovidov
cb6cd2ffbb Added missing ErrorCodes, part 2 2020-02-25 21:10:48 +03:00
Alexey Milovidov
766ae8261a Added some missing ErrorCodes 2020-02-25 21:02:41 +03:00
Alexey Milovidov
3602126a1e Remove unused ErrorCodes 2020-02-25 17:35:37 +03:00
Alexey Milovidov
c7aba07dfe Minor modifications 2020-02-25 15:19:47 +03:00
alexey-milovidov
49b46fd897
Merge pull request #9257 from millb/disk_scheme_4
Space check from path/data/ for default disk
2020-02-22 00:27:49 +03:00
alexey-milovidov
bc910d3489
Trigger CI 2020-02-21 17:05:57 +03:00
Pavel Kovalenko
e593003be7 Formatting issues. 2020-02-20 19:48:52 +03:00
Pavel Kovalenko
e5b81aefe3 WriteBufferFromFileBase interface refactoring.
Disk->writeFile() returns WriteBufferFromFileBase.
DiskMemory read/write buffers refactoring.
2020-02-20 19:39:32 +03:00
millb
cf53160221 fixed bug 2020-02-20 19:21:28 +03:00
Pavel Kovalenko
a8cffd55aa Formatting issues. 2020-02-18 12:51:22 +03:00
Pavel Kovalenko
e849654628 Reworked ReadIndirectBuffer for DiskMemory. 2020-02-17 23:08:35 +03:00
Pavel Kovalenko
cf474e1f09 Fixed DiskMemory->readFile() test. 2020-02-17 18:00:39 +03:00
Pavel Kovalenko
06e203d004 StorageStripeLog S3 support. 2020-02-14 17:28:33 +03:00
alexey-milovidov
3a1e3f612e
Merge branch 'master' into idisk-seekable-readbuffer 2020-02-02 03:54:50 +03:00
Alexander Burmak
0c0ad54366 Support of s3_min_upload_part_size setting in DiskS3 2020-02-01 12:28:04 +03:00
Pavel Kovalenko
e615f72749 Fixed DiskTest 2020-01-31 12:05:41 +03:00
Pavel Kovalenko
9efda05839 StorageLog test with DiskLocal and DiskMemory. 2020-01-30 22:29:30 +03:00
Pavel Kovalenko
e0cd6b1073 Formatting issues. 2020-01-28 19:09:24 +03:00
Pavel Kovalenko
efeb9e92be StorageLog Gtest 2020-01-28 19:08:30 +03:00
Pavel Kovalenko
856592e26b Comments fixed. 2020-01-28 16:46:02 +03:00
Pavel Kovalenko
0572f96656 Formatting issues. 2020-01-28 16:15:47 +03:00
Pavel Kovalenko
3eb59d1ed3 Formatting issues. 2020-01-28 16:01:08 +03:00
Pavel Kovalenko
d4707ca911 Seek reworking. 2020-01-28 15:48:01 +03:00
Pavel Kovalenko
861ddbcadd DiskS3 Metadata refactoring. 2020-01-28 15:46:39 +03:00
Pavel Kovalenko
cb3693f071 Formatting issues. 2020-01-27 22:51:48 +03:00
Pavel Kovalenko
699a031c7f Code style issues. 2020-01-27 22:27:52 +03:00
Pavel Kovalenko
47e7c8d474 Formatting issues. 2020-01-27 22:17:22 +03:00
Pavel Kovalenko
0df5f89a47 Minor formatting. 2020-01-27 22:01:18 +03:00
Pavel Kovalenko
6501ca735b Minor formatting. 2020-01-27 21:59:52 +03:00
Pavel Kovalenko
07755ec8f7 Merge branch 'master' into idisk-seekable-readbuffer
# Conflicts:
#	dbms/programs/server/config.xml
2020-01-27 21:45:53 +03:00
Pavel Kovalenko
abfacdaadc SeekableReadBuffer refactoring.
Store size and multiple references for S3 metadata file.
Log engine support for S3.
2020-01-27 21:44:30 +03:00
Azat Khuzhin
88bfb788a9 Add ability to use multiple disks/volumes for temporary data
This patch adds <tmp_policy> config directive, that will define the
policy to use for storing temporary files, if it is not set (default)
the <tmp_path> will be used.

Also tmp_policy has some limitations:
- move_factor              is ignored
- keep_free_space_bytes    is ignored
- max_data_part_size_bytes is ignored
- must have exactly one volume
2020-01-23 20:31:01 +03:00
Pavel Kovalenko
dce424fe11 SeekableReadBuffer initial implementation. IDisk readFile should return SeekableBuffer. 2020-01-22 19:17:25 +03:00