thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-17 20:02:05 +00:00
Code Issues Packages Projects Releases Wiki Activity
69,496 Commits 9,330 Branches 1,783 Tags 2.1 GiB
9869c8bb10
Commit Graph

409 Commits

Author SHA1 Message Date
taiyang-li
89d8c1264c add test of upload disk 2021-02-19 18:11:21 +08:00
alexey-milovidov
a299790522
Merge pull request #19748 from excitoon-favorites/s3ssec 
Added Server Side Encryption Customer Keys support in S3 client
 2021-02-15 15:23:14 +03:00
taiyang-li
3c61cecab5 add atomic for keep_free_space_bytes 2021-02-10 12:08:08 +08:00
Anton Ivashkin
4d44d75bc7 Fix build after merge one more time 2021-02-08 14:45:10 +03:00
Anton Ivashkin
1b6ce056a5 Merge master 2021-02-08 12:34:20 +03:00
Anton Ivashkin
cd94f708a1 Fix build after merge 2021-02-08 12:33:27 +03:00
alexey-milovidov
f2ab4f2ce9
Merge pull request #20037 from abyss7/better-read-buffers-2 
ReadBuffer: check for unread data on next()
 2021-02-07 01:33:06 +03:00
Anton Ivashkin
e64c63c611 Merge master 2021-02-05 20:10:06 +03:00
Anton Ivashkin
df6c882aab Fix build after merge 2021-02-05 18:52:40 +03:00
vdimir
1e2c302248
Merge pull request #18377 from Jokser/disk-s3-backup-restore-metadata 2021-02-05 10:44:59 +03:00
Ivan
d135a727e2
Merge branch 'master' into better-read-buffers-2 2021-02-05 02:17:05 +03:00
Ivan Lezhankin
84b88c6802 CC 2021-02-04 22:06:43 +03:00
kreuzerkrieg
1cc9e03d0c return DiskType instead of String IDisk::getType() as in the rest of storage interfaces 2021-02-04 10:01:21 +02:00
Pavel Kovalenko
62e96c138e Remove unused error code in DiskS3 2021-02-03 15:01:43 +03:00
Pavel Kovalenko
b6094859c7 Throw exception during restore DiskS3 if object doesn't have path in metadata. 2021-02-03 14:26:24 +03:00
Pavel Kovalenko
b7151aa754 Merge remote-tracking branch 'origin/master' into disk-s3-backup-restore-metadata 
# Conflicts:
#	src/Disks/DiskDecorator.h
#	src/Disks/IDisk.h
#	src/Disks/S3/DiskS3.cpp
 2021-02-03 14:22:18 +03:00
taiyang-li
44200abfb4 add interface updateFromConfigIfChanged 2021-01-29 19:13:33 +08:00
Kruglov Pavel
caef103837
Merge branch 'master' into Add_IStoragePolicy_interface 2021-01-29 14:00:12 +03:00
Vladimir Chebotarev
579f8da573 Added SSE-C support in S3 client. 2021-01-28 09:32:41 +03:00
alexey-milovidov
7c240d0ca7
Merge pull request #17870 from excitoon-favorites/s3testiostream 
Update of AWS C++ SDK
 2021-01-28 07:12:52 +03:00
Vladimir Chebotarev
afdea5eff0 Changed handling of PocoHTTPClientConfiguration in order to ensure AWS SDK is initialized. 2021-01-27 09:38:35 +03:00
Anton Popov
09e5bdd1ef fix style check 2021-01-27 03:57:28 +03:00
Anton Popov
b15848e2f0
Update IDisk.h 2021-01-27 03:32:18 +03:00
Anton Popov
c7070da85a better abstractions in disk interface 2021-01-26 17:49:35 +03:00
kreuzerkrieg
29a2ef3089 Add IStoragePolicy interface 2021-01-26 10:55:28 +02:00
taiyang-li
901de8968f update disk local if config changed 2021-01-26 15:07:17 +08:00
taiyang-li
ddb02282f1 enable local disk config reload 2021-01-24 11:53:07 +08:00
Alexander Gololobov
0a155ee2f1 Fix IDisk::open parameters to match posix open 2021-01-21 19:38:13 +03:00
Anton Ivashkin
357d98eb36 Merge master 2021-01-20 12:23:03 +03:00
Pavel Kovalenko
1e3a059f64 Merge remote-tracking branch 'origin/master' into disk-s3-backup-restore-metadata 
# Conflicts:
#	src/Disks/DiskCacheWrapper.cpp
#	src/Disks/S3/DiskS3.cpp
 2021-01-18 13:39:49 +03:00
Alexander Kuzmenkov
d22c04568d
Merge pull request #13405 from excitoon-favorites/s3keepalive 
Connection pools for S3
 2021-01-18 12:52:37 +03:00
Alexey Milovidov
ee13dd4fef Fix error 2021-01-14 19:58:44 +03:00
Alexey Milovidov
8276a1c8d2 Faster parts removal, more safe and efficient interface of IDisk 2021-01-14 19:24:13 +03:00
Pavel Kovalenko
53389f79c0 Ability to backup-restore metadata files for DiskS3 (minor fixes) 2021-01-13 15:05:32 +03:00
Pavel Kovalenko
b09862b7b9 Ability to backup-restore metadata files for DiskS3 (fixes and tests) 2021-01-12 20:18:40 +03:00
Pavel Kovalenko
0856b2c514 Ability to backup-restore metadata files for DiskS3 (fixes and tests) 2021-01-11 20:37:08 +03:00
Alexey Milovidov
6eb5a5f4d9 Remove useless code 2021-01-10 03:28:59 +03:00
Vladimir Chebotarev
5cbc25c647 Added global setting s3_max_connections. 2021-01-05 13:34:55 +03:00
Pavel Kovalenko
2848b32af1 Ability to backup-restore metadata files for DiskS3 (WIP) 2020-12-23 18:11:37 +03:00
Pavel Kovalenko
cc3b5958b0 Ability to backup-restore metadata files for DiskS3 (WIP) 2020-12-23 15:35:52 +03:00
Pavel Kovalenko
18fe1c796b Ability to backup-restore metadata files for DiskS3 (WIP) 2020-12-22 21:47:47 +03:00
Anton Ivashkin
0f0500ca0c Merge master 2020-12-16 18:31:13 +03:00
Pavel Kovalenko
346c241c20 Merge remote-tracking branch 'origin/master' into s3-adaptive-single-part-upload 
# Conflicts:
#	src/Disks/S3/DiskS3.cpp
 2020-12-14 11:00:30 +03:00
alexey-milovidov
04e222f6f3
Merge pull request #17986 from azat/fwd-decl 
More forward declaration for generic headers
 2020-12-13 01:12:05 +03:00
Azat Khuzhin
5b3ab48861 More forward declaration for generic headers 
The following headers are pretty generic, so use forward declaration as
much as possible:
- Context.h
- Settings.h
- ConnectionTimeouts.h
(Also this shows that some missing some includes -- this has been fixed)

And split ConnectionTimeouts.h into ConnectionTimeoutsContext.h (since
module part cannot be added for it, due to recursive build dependencies
that will be introduced)

Also remove Settings from the RemoteBlockInputStream/RemoteQueryExecutor
and just pass the context, since settings was passed only in speicifc
places, that can allow making a copy of Context (i.e. Copier).

Approx results (How much units will be recompiled after changing file X?):

- ConnectionTimeouts.h
  - mainline: 100

- Context.h:
  - mainline: ~800
  - patched:  415

- Settings.h:
  - mainline: 900-1K
  - patched:  440 (most of them because of the Context.h)
 2020-12-12 17:43:10 +03:00
Pavel Kovalenko
25df6bae78 Add nested exception when handle Disk S3 metadata file read error. 2020-12-12 17:11:47 +03:00
Pavel Kovalenko
59a5e63417 Merge remote-tracking branch 'origin/master' into s3-adaptive-single-part-upload 
# Conflicts:
#	src/Disks/S3/DiskS3.cpp
#	src/Disks/S3/DiskS3.h
#	src/Disks/S3/registerDiskS3.cpp
#	src/IO/WriteBufferFromS3.cpp
#	src/IO/WriteBufferFromS3.h
 2020-12-11 18:28:41 +03:00
Pavel Kovalenko
72a55c192e Fix master branch merge artifact. 2020-12-11 18:23:59 +03:00
Pavel Kovalenko
75bdb813ea Merge remote-tracking branch 'origin/master' into disk-s3-remove-empty-metadata-files 
# Conflicts:
#	src/Disks/S3/DiskS3.cpp
 2020-12-11 18:22:31 +03:00
Pavel Kovalenko
de4a7a8865 Correct handling S3 disk metadata file read exceptions. 2020-12-11 14:27:58 +03:00
Pavel Kovalenko
5bf388683c Correct handling S3 disk metadata file read exceptions. 2020-12-11 14:21:54 +03:00
Kruglov Pavel
9a785dbbf5
Merge branch 'master' into disk-s3-send-metadata 2020-12-10 20:17:41 +03:00
Vladimir Chebotarev
39d867e56d
Added proper authentication for S3 client (#16856) 2020-12-10 12:19:42 +03:00
Pavel Kovalenko
5802633f95 Fix parameter name in constructor of DiskS3 2020-12-09 18:38:02 +03:00
Pavel Kovalenko
818cdd353a Remove empty or bad metadata files from FS for DiskS3. 2020-12-09 18:01:30 +03:00
Pavel Kovalenko
db7a87089b Adaptive choose of single/multi part upload in WriteBufferFromS3. 2020-12-09 17:09:04 +03:00
Pavel Kovalenko
dfbb31174e Ability to set metadata when put S3 object 2020-12-08 21:31:57 +03:00
Alexey Milovidov
4ecfe8fef4 Remove unused headers 2020-12-08 02:10:22 +03:00
Alexander Tokmakov
bfbf150c67 fix segfault when 'not enough space' 2020-12-02 17:49:43 +03:00
alesapin
45f09fe293
Merge pull request #17220 from ianton-ru/s3_max_redirects 
S3 max redirects
 2020-11-24 12:13:10 +03:00
Anton Ivashkin
aa208a9d9f Use only 's3_max_redirect' in params instead of all settings 2020-11-23 14:02:17 +03:00
Pavel Kovalenko
f6041c2280 Use default value for read-only flag in metadata for Disk3. 2020-11-23 00:14:52 +03:00
Pavel Kovalenko
9cbb251be6 Store read-only flag into metadata file for DiskS3. 2020-11-20 14:08:43 +03:00
Alexey Milovidov
24f4fa6edf Follow Arcadia ya.make rules 2020-11-17 00:16:50 +03:00
Alexey Milovidov
3df04ce0c2 Follow Arcadia ya.make rules 2020-11-16 21:24:58 +03:00
Anton Ivashkin
026f7e0a27 Add 's3_max_redirects' setting 2020-11-13 19:31:51 +03:00
Alexey Milovidov
b56486510f Improve Arcadia 2020-11-05 12:55:01 +03:00
Anton Ivashkin
1742fb3256 Merge master 2020-11-03 12:27:16 +03:00
Anton Ivashkin
78021714f1 S3 zero copy replication: more simple s3 check 2020-11-03 12:20:26 +03:00
Maxim Akhmedov
3627fabfb9 Remove -g0 form Arcadia build settings. 2020-10-29 17:37:23 +03:00
Anton Ivashkin
e3879afa69 S3 zero copy replication: fix virtual method default parameter 2020-10-23 16:27:50 +03:00
Anton Ivashkin
652c56e74e Fix style, fix build 2020-10-22 16:07:20 +03:00
alesapin
4b65d5469f Fix some unrelated performance issues in select parts for merge 2020-10-22 15:41:01 +03:00
Anton Ivashkin
f833501c77 Merge branch 'master' into s3_zero_copy_replication 2020-10-22 11:17:21 +03:00
Vladimir Chebotarev
aa5f207fd4
Added disable_merges option for volumes in multi-disk configuration (#13956) 
Co-authored-by: Alexander Kazakov <Akazz@users.noreply.github.com>
 2020-10-20 18:10:24 +03:00
Alexey Milovidov
94592819ad Add comment 2020-10-20 04:29:13 +03:00
Pavel Kovalenko
a474bdcabf Merge remote-tracking branch 'origin/master' into idempotent-part-move 2020-10-16 13:02:40 +03:00
Anton Ivashkin
b877459cf7 Zero copy replication over S3: check s3 storage 2020-10-14 18:05:59 +03:00
Alexey Milovidov
269b6383f5 Check for #pragma once in headers 2020-10-10 21:37:02 +03:00
Anton Ivashkin
766dbfd2be Merge master 2020-10-09 15:00:14 +03:00
Anton Ivashkin
acf86568a7 S3 zero copy replication proof of concept 2020-10-08 18:45:10 +03:00
Pavel Kovalenko
69c126f1f1 Possibility to move part to another disk/volume if first attempt was failed. 2020-10-07 14:35:28 +03:00
Pavel Kovalenko
69b4bc6f33 Proper error handling during insert into MergeTree with S3. 2020-10-06 12:38:00 +03:00
alexey-milovidov
5e37ba4335
Update DiskS3.cpp 2020-10-03 03:47:45 +03:00
alexey-milovidov
3ca7fca48a
Update DiskS3.cpp 2020-10-03 03:42:33 +03:00
Anton Ivashkin
c18cdec77c Use S3 multi object delete 
Some strange with s3_with_proxy test
 2020-09-28 11:47:04 +03:00
alexey-milovidov
1474b80ecc
Merge pull request #15008 from ClickHouse/clang-tidy-11-without-enabling 
Fix terribly wrong code
 2020-09-20 02:00:54 +03:00
Alexey Milovidov
ef5363b87f Fix terribly wrong code 2020-09-19 02:05:13 +03:00
Pavel Kovalenko
da04a130ed Add option to disable ttl move on data part insert - minor fixes. 2020-09-18 20:45:30 +03:00
Pavel Kovalenko
0da19ab46d Add option to disable ttl move on data part insert 2020-09-18 18:30:00 +03:00
Artem Zuikov
51ba12c2c3
Try speedup build (#14809) 2020-09-15 12:55:57 +03:00
Anton Popov
af4089c606 Merge remote-tracking branch 'upstream/master' into HEAD 2020-09-11 00:39:21 +03:00
Pavel Kovalenko
3632b0bdb3
Add option to DiskS3 that allows it's usage if S3 unavailable (#14497) 2020-09-09 02:35:57 +03:00
alesapin
bea15175d6
Merge pull request #14496 from Jokser/disk-s3-shutdown 
Disable S3 requests processing during context shutdown to speed up termination process
 2020-09-08 15:21:54 +03:00
Pavel Kovalenko
5e39f8b32b Add comment explaining DiskS3::shutdown 2020-09-08 15:20:35 +03:00
Alexey Milovidov
e3924b8057 Fix "Arcadia" 2020-09-08 01:14:13 +03:00
Pavel Kovalenko
c933f72adb Disable S3 requests processing during context shutdown to speed up termination process. 2020-09-04 17:17:27 +03:00
Anton Popov
4331158d30 merge with master 2020-08-24 16:09:23 +03:00
Anton Popov
4df5c72878 Merge remote-tracking branch 'upstream/master' into HEAD 2020-08-24 12:57:27 +03:00
alexey-milovidov
7b53a0ef33 Revert "Added allow_merges option for volumes in multi-disk configuration (#13402)" 
This reverts commit 1e2616542a.
 2020-08-21 18:44:29 +03:00
Vladimir Chebotarev
1e2616542a
Added allow_merges option for volumes in multi-disk configuration (#13402) 2020-08-21 12:04:13 +03:00
Alexander Kuzmenkov
ef5882639b
Merge pull request #13459 from Jokser/s3-multithreaded-disk-copy 
Move parts from DiskLocal to DiskS3 in parallel
 2020-08-12 07:36:03 +03:00
Pavel Kovalenko
ea46ee74ae Don't allocate Executor instance on each disk->getExecutor() call. 2020-08-11 22:08:32 +03:00
Vladimir Chebotarev
7371cf6bd1
Proper remote host checking in S3 redirects (#13404) 
* Proper remote host checking in S3 redirects.

Co-authored-by: Alexander Kazakov <Akazz@users.noreply.github.com>
 2020-08-11 09:09:08 +03:00
Pavel Kovalenko
bece61b1cc Fix codestyle. 2020-08-10 14:40:31 +03:00
Pavel Kovalenko
f503273d41 Merge remote-tracking branch 'origin/master' into s3-multithreaded-disk-copy 
# Conflicts:
#	src/Disks/S3/DiskS3.cpp
 2020-08-10 14:35:26 +03:00
Alexey Milovidov
b4ac900462 Fix "Arcadia" build 2020-08-09 02:38:53 +03:00
Alexander Kuzmenkov
6a596d5c40
Merge pull request #13076 from Jokser/s3-file-cache 
Ability to cache mark and index files for S3 disk
 2020-08-08 01:04:53 +03:00
Pavel Kovalenko
562102161b Missed override. 2020-08-07 18:53:40 +03:00
Pavel Kovalenko
53ecd95302 Remove unused dependencies. 2020-08-07 14:44:35 +03:00
Pavel Kovalenko
a6ee09f696 Move parts to DiskS3 in parallel refactoring. 2020-08-07 14:40:19 +03:00
Pavel Kovalenko
7856852010 Move parts to DiskS3 in parallel. 2020-08-05 19:36:10 +03:00
Pavel Kovalenko
1703e5a1ca Download through tmp file in DiskCacheWrapper.cpp 2020-08-03 15:37:53 +03:00
Alexey Milovidov
6f690b7c0d Normalize ya.make files, fix "Arcadia" build 2020-08-02 16:57:38 +03:00
Pavel Kovalenko
a6d162961f Fix directories creation in DiskCacheWrapper.cpp 2020-08-01 22:09:05 +03:00
Pavel Kovalenko
d7785f8c2d Fix checkstyle. 2020-07-30 18:48:32 +03:00
Pavel Kovalenko
7a41fc1b9b DiskS3 with cache tests. 2020-07-30 17:49:56 +03:00
alesapin
df0dcf4a31 Fix error codes 2020-07-30 17:27:11 +03:00
Pavel Kovalenko
00181d34fd DiskS3 caching improvements. 2020-07-30 16:42:05 +03:00
alesapin
1c8aa4b6c1 Fix get disk interface 2020-07-30 14:08:56 +03:00
alesapin
d3ea1d914b Tiny fixes and avoid dynamic casts 2020-07-30 13:04:49 +03:00
Pavel Kovalenko
ba8d770a76 Change comment in DiskCacheWrapper.cpp 2020-07-29 22:54:00 +03:00
Pavel Kovalenko
644d4c7aaa Cache mark and index files for DiskS3. 2020-07-29 21:15:20 +03:00
Pavel Kovalenko
2b962e699b Cache mark and index files for DiskS3. 2020-07-29 18:05:02 +03:00
Vladimir Chebotarev
faedb04722 Minor fixes. 2020-07-28 19:45:46 +03:00
Vladimir Chebotarev
6d22069f55 Review fixes. 2020-07-27 18:59:09 +03:00
Vladimir Chebotarev
ee21fde3fe Review fixes. 2020-07-26 21:27:36 +03:00
Gleb Novikov
aac97957ce Fixed integration test_tmp_policy 2020-07-26 21:27:36 +03:00
Gleb Novikov
99d52552e1 Small fixes for tests 2020-07-26 21:27:36 +03:00
Gleb Novikov
f4080ed6cc removed unused error code 2020-07-26 21:27:36 +03:00
Gleb Novikov
e4699e297e removed default clause in switch for VolumeType enum 2020-07-26 21:27:36 +03:00
Gleb Novikov
2c39584a44 Braces style fix 2020-07-26 21:27:36 +03:00
Gleb Novikov
adef249c34 Small fix of VolumeRAID1Ptr 2020-07-26 21:27:36 +03:00
Gleb Novikov
7f5b6fba78 Generic volume is coming... 
1. SingleDiskVolume for temporary volumes
2. Generic VolumePtr in StoragePolicies
3. Removed max_data_part_size in system.storage_policies, added volume_type
 2020-07-26 21:27:36 +03:00
Gleb Novikov
96c7cc8724 Small clang-10 warning fixes + ya.make 2020-07-26 21:27:36 +03:00
Gleb Novikov
db56820fab Style fixes and added UNKNOWN_RAID_TYPE exception to createVolumeFromConfig 2020-07-26 21:27:36 +03:00
Gleb Novikov
3585b24f16 MultiDiskReservation, VolumeRAID1 and createVolumeFromConfig 
1. Implemented MultiDiskReservation to maintain multiple reservations under one
2. Implemented VolumeRAID1 as child of VolumeJBOD (since RAID1 is quite similar)
3. Implemented createVolumeFromConfig
 2020-07-26 21:27:36 +03:00
Alexey Milovidov
6df282e813 Fixup 2020-07-16 11:33:51 +03:00
Alexey Milovidov
a6016b95ce Fix clang-tidy 2020-07-16 05:34:43 +03:00
Alexey Milovidov
3408b7e259 Merge branch 'master' into log-engine-rollback-on-insert-error 2020-07-16 05:34:02 +03:00
Vladimir Chebotarev
a479a20ba4 Review fixes. 2020-07-13 03:24:49 +03:00
Vladimir Chebotarev
d3ade3492e
Update DiskS3.h 2020-07-13 03:09:37 +03:00
alexey-milovidov
c6a5b77843
Update registerDiskS3.cpp 2020-07-13 03:04:46 +03:00
Vladimir Chebotarev
4802a785b5 Added read_seek_threshold setting to DiskS3. 2020-07-13 02:51:25 +03:00
Vladimir Chebotarev
718523b0ee Working wrapper. 2020-07-13 02:47:22 +03:00
Alexey Milovidov
204a4af394 Rollback insertion error in Log engines #12402 2020-07-12 05:32:18 +03:00
alexey-milovidov
26d32f2ce5
Merge pull request #12412 from Jokser/s3-https 
S3 HTTPS integration tests
 2020-07-12 03:36:37 +03:00
Pavel Kovalenko
f43428a4d5 S3 HTTPS integration test. 2020-07-10 22:42:18 +03:00
Vladimir Chebotarev
faf6be6576
Implemented single part uploads for DiskS3 (#12026) 
* Implemented single part uploads for DiskS3.
* Added `min_multi_part_upload_size` to disk configuration.
 2020-07-09 17:09:17 +03:00
Alexander Kazakov
1c0c8a7bf6
Merge pull request #11892 from excitoon-favorites/relatives3paths 
Switched paths in S3 metadata to relative
 2020-07-07 00:13:07 +03:00
alexey-milovidov
ec563e54dc
Merge pull request #12138 from ClickHouse/codeql 
Fix warnings from CodeQL
 2020-07-05 18:22:49 +03:00
alexey-milovidov
c257255759
Merge pull request #12134 from matwey/fix/missed_include 
Add missed <atomic> include in ProxyListConfiguration
 2020-07-05 04:22:42 +03:00
Alexey Milovidov
96e8e93c64 Fix warnings from CodeQL 2020-07-05 02:16:16 +03:00
Matwey V. Kornilov
caafbe19a4 Add missed <atomic> include in ProxyListConfiguration 2020-07-04 15:32:02 +03:00
Anton Ivashkin
23b44ca6fe Add type column in system.disks 2020-07-03 15:45:54 +03:00
Vladimir Chebotarev
8128ca10f4 Switched paths in S3 metadata to relative. 2020-06-30 07:59:22 +03:00
Alexey Milovidov
1462a66d1e Fix typos 2020-06-27 22:05:00 +03:00
Anton Popov
b2aa565a37 allow to turn on fsync on inserts, merges and fetches 2020-06-27 17:52:16 +03:00
Anton Popov
f8f67a788e allow to turn on fsync on inserts and merges 2020-06-25 19:55:45 +03:00
Alexey Milovidov
25f941020b Remove namespace pollution 2020-05-31 00:57:37 +03:00
Alexey Milovidov
5aff138956 Preparation for structured logging 2020-05-31 00:35:52 +03:00
Alexey Milovidov
7e1813825b Return old names of macros 2020-05-24 01:24:01 +03:00
Alexey Milovidov
18febd7b97 find {base,src,programs} -name '*.h' -or -name '*.cpp' | xargs grep -l -P 'LOG_[^\_(]+\([^,]+, "[^"]+" << [^<]+ << "[^"]+" << [^<]+ << "[^"]+" << [^<]+ << "[^"]+" << [^<]+ << "[^"]+"\);' | while read file; do perl -pne 's/(LOG_[^\_(]+)\(([^,]+), "([^"]+)" << ([^<]+) << "([^"]+)" << ([^<]+) << "([^"]+)" << ([^<]+) << "([^"]+)" << ([^<]+) << "([^"]+)"\);/${1}_FORMATTED(${2}, "${3}{}${5}{}${7}{}${9}{}${11}", ${4}, ${6}, ${8}, ${10});/' $file > ${file}.tmp; mv ${file}.tmp $file; done 2020-05-23 22:56:05 +03:00
Alexey Milovidov
d5b7883451 find {base,src,programs} -name '*.h' -or -name '*.cpp' | xargs grep -l -P 'LOG_[^\_(]+\([^,]+, "[^"]+" << [^<]+ << "[^"]+" << [^<]+ << "[^"]+" << [^<]+ << "[^"]+"\);' | xargs sed -i -r -e 's/(LOG_[^\_(]+)\(([^,]+), "([^"]+)" << ([^<]+) << "([^"]+)" << ([^<]+) << "([^"]+)" << ([^<]+) << "([^"]+)"\);/\1_FORMATTED(\2, "\3{}\5{}\7{}\9", \4, \6, \8);/' 2020-05-23 22:41:08 +03:00
Alexey Milovidov
0ab3ca8534 Apply all transformations again 2020-05-23 22:32:44 +03:00
Alexey Milovidov
29762240de Remove duplicate whitespaces (preparation) 2020-05-23 22:31:54 +03:00
Alexey Milovidov
7fed65cbe2 Remove duplicate whitespaces (preparation) 2020-05-23 22:14:58 +03:00
Alexey Milovidov
86b4d5a86f Remove duplicate whitespaces (preparation) 2020-05-23 22:09:06 +03:00
Alexey Milovidov
ab0562a574 Make all LOG in single line (preparation) 2020-05-23 22:05:41 +03:00
Alexey Milovidov
9d24908e53 Progress on task 2020-05-23 20:52:11 +03:00
Alexey Milovidov
241f8c5431 find {base,src,programs} -name '*.h' -or -name '*.cpp' | xargs grep -l -P 'LOG_\w+\([^,]+, "[^"]+" \+ [^+]+ \+ "[^"]+"\);' | xargs sed -i -r -e 's/(LOG_\w+)\(([^,]+), "([^"]+)" \+ ([^+]+) \+ "([^"]+)"\);/\1_FORMATTED(\2, "\3{}\5", \4);/' 2020-05-23 20:10:21 +03:00
Alexey Milovidov
f68d1ceb4f find {base,src,programs} -name '*.h' -or -name '*.cpp' | xargs grep -l -P 'LOG_\w+\([^,]+, "[^"]+" << [^<]+ << "[^"]+" << [^<]+ << "[^"]+" << [^<]+\);' | xargs sed -i -r -e 's/(LOG_\w+)\(([^,]+), "([^"]+)" << ([^<]+) << "([^"]+)" << ([^<]+) << "([^"]+)" << ([^<]+)\);/\1_FORMATTED(\2, "\3{}\5{}\7{}", \4, \6, \8);/' 2020-05-23 20:02:09 +03:00
Alexey Milovidov
8042e5febe find {base,src,programs} -name '*.h' -or -name '*.cpp' | xargs grep -l -P 'LOG_\w+\([^,]+, "[^"]+" << [^<]+ << "[^"]+" << [^<]+\);' | xargs sed -i -r -e 's/(LOG_\w+)\(([^,]+), "([^"]+)" << ([^<]+) << "([^"]+)" << ([^<]+)\);/\1_FORMATTED(\2, "\3{}\5{}", \4, \6);/' 2020-05-23 19:58:15 +03:00
Alexey Milovidov
e391b77d81 find {base,src,programs} -name '*.h' -or -name '*.cpp' | xargs grep -l -P 'LOG_\w+\([^,]+, "[^"]+" << [^<]+ << "[^"]+"\);' | xargs sed -i -r -e 's/(LOG_\w+)\(([^,]+), "([^"]+)" << ([^<]+) << "([^"]+)"\);/\1_FORMATTED(\2, "\3{}\5", \4);/' 2020-05-23 19:56:05 +03:00
Alexey Milovidov
ee4ffbc332 find {base,src,programs} -name '*.h' -or -name '*.cpp' | xargs grep -l -P 'LOG_\w+\([^,]+, "[^"]+" << [^<]+\);' | xargs sed -i -r -e 's/(LOG_\w+)\(([^,]+), "([^"]+)" << ([^<]+)\);/\1_FORMATTED(\2, "\3{}", \4);/' 2020-05-23 19:47:56 +03:00
alexey-milovidov
ac6deaf8ec
Merge pull request #11036 from excitoon-favorites/s3fixes 
Fixed parsing of S3 URL
 2020-05-23 17:06:59 +03:00
alesapin
70fe119426 Add clear names 2020-05-22 13:33:57 +03:00
Vladimir Chebotarev
98f31374d5 Added improved S3 URL parsing to DiskS3. 2020-05-22 11:18:34 +03:00
Alexander Kuzmenkov
f98ffdbc4c
Merge pull request #11087 from azat/context-fwd-decl 
[RFC] Forward declaration for Context as much as possible.
 2020-05-21 19:43:29 +03:00
Azat Khuzhin
d93b9a57f6 Forward declaration for Context as much as possible. 
Now after changing Context.h 488 modules will be recompiled instead of 582.
 2020-05-21 01:53:18 +03:00
Pavel Kovalenko
d98c1589ed Better error handling in Proxy Resolver. 2020-05-20 19:01:05 +03:00
Pavel Kovalenko
138154ba6a Fix memory-leak in registerDiskS3 during config->createView invocations. 2020-05-20 18:42:45 +03:00
Oleg Ershov
8858319281 global update 2020-05-20 06:14:58 +03:00
Gleb Novikov
665c6b7f59 Added new files to ya make 2020-05-19 12:38:43 +03:00
alesapin
bec7d1ce93
Fix style check 2020-05-18 16:49:19 +03:00
alesapin
ab491c0dd9
boop CI 2020-05-18 11:42:39 +03:00
Gleb Novikov
1a25ac6e1f Merge branch 'master' into refactor-reservations 2020-05-16 23:34:45 +03:00
Gleb Novikov
45b84d491e small fixes of review remarks 2020-05-16 23:31:17 +03:00
Oleg Ershov
dd210cfeaf Fix disk issues 2020-05-15 20:55:18 +03:00
Oleg Ershov
ae715a8d1b Add base 2020-05-14 18:45:30 +03:00
Pavel Kovalenko
506634786e Handle exceptions in ProxyResolverConfiguration properly. 2020-05-12 17:34:37 +03:00
Pavel Kovalenko
8bea42e250 Fix includes in ProxyResolverConfiguration.h 2020-05-12 12:23:18 +03:00
Gleb Novikov
fbd00d6af3 removed default value from overriden functions in IReservation children 2020-05-10 12:33:52 +03:00
Gleb Novikov
a961e049a2 VolumeType as class with enum member class 2020-05-10 09:26:33 +03:00
Gleb Novikov
390b39b272 VolumePtr instead of DiskPtr in MergeTreeData* 2020-05-10 00:24:15 +03:00
Gleb Novikov
b54209e43e Modified reservation to work with multiple disks 2020-05-09 19:03:22 +03:00
Pavel Kovalenko
8d8dc3035c Fix naming. 2020-05-08 16:10:21 +03:00
Pavel Kovalenko
8602f7b47c Minor updates in proxy resolver parameters naming. 2020-05-08 13:58:56 +03:00
Pavel Kovalenko
6165c2aa99 Add possibility to use proxy resolver in DiskS3. 2020-05-08 13:53:12 +03:00
Gleb Novikov
6e6a195c13 Style fixes, removed redundant includes 2020-05-04 23:20:51 +03:00
Gleb Novikov
c637d99e07 Volumes and storages refactoring: 
1. Moved Volume to separate file
  2. Created IVolume interface and implemented current behaviour in implementation of new interface — VolumeJBOD
  3. Replaced all old volume usages with new VolumeJBOD. Where it is unnecessary to have JBOD — left just IVolume.
  4. Removed old Volume completely
  5. Moved StoragePolicy to separated files
  6. Moved DiskSelector to separated files
  7. Removed DiskSpaceMonitor file
 2020-05-04 23:15:38 +03:00
Pavel Kovalenko
0b70c2ca08 Fixed ya.make for S3. 2020-04-29 21:38:35 +03:00
Pavel Kovalenko
f17e022fa4 Code style fixes. 2020-04-29 15:05:46 +03:00
Pavel Kovalenko
a1acbcb019 Code style fixes. 2020-04-29 11:45:13 +03:00
Pavel Kovalenko
e5e84960ea S3 with proxy integration test and fixes. 2020-04-29 11:40:29 +03:00
Pavel Kovalenko
2ecf4b5a23 Dynamic proxy configuration for DiskS3. 2020-04-29 11:40:29 +03:00
Alexey Milovidov
be22a4b94e Checkpoint 2020-04-22 08:39:31 +03:00
Ivan Lezhankin
e230632645 Changes required for auto-sync with Arcadia 2020-04-16 15:31:57 +03:00
Ivan Lezhankin
06446b4f08 dbms/ → src/ 2020-04-03 18:14:31 +03:00