thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-17 11:52:27 +00:00
Code Issues Packages Projects Releases Wiki Activity
78,809 Commits 9,326 Branches 1,784 Tags 2.1 GiB
7c2e116214
Commit Graph

624 Commits

Author SHA1 Message Date
kssenii
6219d541a5 Finally fixed 2021-09-24 15:29:04 +03:00
kssenii
c1ea44b9c2 Fixes 2021-09-24 13:38:08 +03:00
kssenii
1b9565dfa8 Merge branch 'master' of github.com:ClickHouse/ClickHouse into disk-async-read 2021-09-22 22:08:44 +03:00
kssenii
e301457e91 Fix prefetch mistake 2021-09-21 15:40:48 +03:00
kssenii
975033e6e3 Fix 2021-09-21 09:44:54 +00:00
kssenii
40ee75fe97 Common thread pool for disks 2021-09-21 10:00:56 +03:00
kssenii
cacc81ca61 Fix 2021-09-21 02:31:02 +03:00
kssenii
0104cb1c8e Better 2021-09-21 01:51:17 +03:00
kssenii
f312760dca Better 2021-09-20 19:57:15 +03:00
kssenii
11ae94b8d3 Async read for remote fs 2021-09-20 16:16:46 +03:00
Kseniia Sumarokova
2a8f355326
Merge pull request #29179 from kssenii/better-retries 
Better retries for disk web
 2021-09-20 08:25:49 +03:00
kssenii
12b90da5de Better retries 2021-09-19 20:50:50 +00:00
Nikita Mikhaylov
c52b8ec083
Introduced MergeTask and MutateTask (#25165) 
Introduced MergeTask and MutateTask
 2021-09-17 00:19:58 +03:00
Kseniia Sumarokova
ef6ba90be1
Update DiskWebServer.h 2021-09-13 20:00:18 +03:00
kssenii
9ec6a9b4c9 Merge branch 'master' of github.com:ClickHouse/ClickHouse into fix-web-disk-2 2021-09-13 08:32:34 +03:00
kssenii
147e413346 Add retries with backoff, better exceptions 2021-09-13 00:04:05 +03:00
Mike Kot
c4385a2c40 Simplifying exception messages 2021-09-11 00:40:18 +02:00
kssenii
939032b0eb Slightly better 2021-09-10 13:41:41 +03:00
kssenii
c23e17d369 Fix 2021-09-09 15:25:44 +03:00
kssenii
72505db3bd Fix checks 2021-09-09 10:10:34 +03:00
kssenii
1101ee7808 Update example 2021-09-08 22:43:46 +03:00
kssenii
7345ad8e09 Add no throw 2021-09-08 22:40:26 +03:00
kssenii
7141b5d041 Get rid of regexp 2021-09-08 20:22:24 +03:00
Mike Kot
8e9aacadd1 Initial: replacing hardcoded toString for enums with magic_enum 2021-09-06 16:24:03 +02:00
kssenii
97e0a0310a Slightly better 2021-09-04 12:02:07 +03:00
kssenii
32686e11cb Fix style checl 2021-09-04 11:30:30 +03:00
kssenii
6ee94c923f Fix 2021-09-04 00:51:11 +03:00
Kseniia Sumarokova
63a0eb3922
Update DiskWebServer.cpp 2021-08-31 21:51:00 +03:00
kssenii
248fe1fd09 Fix disk 2021-08-31 16:54:43 +03:00
Nikita Mikhaylov
00394014ff
Merge pull request #28382 from DimasKovas/fix_arcadia_build_31_08_2021 
Try to fix arcadia build (generate ya.make)
 2021-08-31 13:49:32 +03:00
Dmitrii Kovalkov
b1eb48804b Run generate-ya-make.sh 2021-08-31 09:05:33 +03:00
Alexey Milovidov
ca6f6306f3 Merge branch 'master' into async-reads 2021-08-31 02:52:58 +03:00
Kseniia Sumarokova
00bea71610
Merge pull request #25251 from kssenii/disk-over-web-server 
ClickHouse over web server with static files
 2021-08-30 16:18:59 +03:00
Dmitrii Kovalkov
9871ad70ff Exclude fuzzers 2021-08-30 11:12:25 +03:00
Dmitrii Kovalkov
8acf0fa16a run generate-ya-make 2021-08-30 10:57:40 +03:00
kssenii
3e7f022d66 Extend usage comment explanation a little 2021-08-28 20:51:20 +00:00
kssenii
f27f519aa2 Fix build and add example 2021-08-28 20:35:51 +00:00
Alexey Milovidov
da4865a5d8 Remove excessive logging from DiskS3 2021-08-25 03:22:26 +03:00
Alexey Milovidov
183891eb0a Add IDisk::isRemote method 2021-08-25 02:27:20 +03:00
Alexey Milovidov
fd27a2faef Remove imperfect code 2021-08-25 02:05:55 +03:00
Alexey Milovidov
7c1d0a3baf Progress on development 2021-08-25 01:24:47 +03:00
Alexey Milovidov
8f57216180 Progress on development 2021-08-25 00:45:58 +03:00
Alexey Milovidov
4d9ad3725d Merge branch 'master' into async-reads 2021-08-25 00:45:40 +03:00
Kseniia Sumarokova
d78e551de9
Update src/Disks/DiskWebServer.cpp 
Co-authored-by: Kruglov Pavel <48961922+Avogar@users.noreply.github.com>
 2021-08-24 12:24:54 +03:00
kssenii
b536653851 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into disk-over-web-server 2021-08-24 07:29:50 +00:00
kssenii
5307d31924 Throw for alter and silence for drop 2021-08-23 23:48:53 +00:00
Vitaly Baranov
63e8bc1f20 Added new commands BACKUP and RESTORE. 2021-08-23 23:41:45 +03:00
Anton Ivashkin
0e0e07ef46 Allow Multi Part Upload for small objects 2021-08-20 19:13:48 +03:00
Anton Ivashkin
51b543fb5c Call AbortMultipartUpload when Multipart Upload Copy failed 2021-08-20 18:57:03 +03:00
vdimir
e77ff2740a
Add common/unit.h 2021-08-20 13:57:05 +03:00
Anton Ivashkin
b3737afe04 Use Multipart copy upload for large S3 objects 2021-08-19 13:09:58 +03:00
kssenii
709bc1a013 Continued 2021-08-16 14:50:26 +00:00
kssenii
9a02c6ad05 Catch any error if it happens on disk load when server is starting up 2021-08-16 13:18:33 +00:00
Alexey Milovidov
d184b79bba Progress on async reads. 2021-08-16 03:00:32 +03:00
kssenii
073d7fdd5e Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into disk-over-web-server 2021-08-06 19:42:29 +00:00
Vladimir C
9de16c5017
Merge pull request #27216 from ianton-ru/MDB-13990 2021-08-06 18:12:00 +03:00
Anton Ivashkin
0e040ca40c Add 10 seconds cache for S3 proxy resolver 2021-08-05 18:52:04 +03:00
Maksim Kita
4dc4854fad
Merge pull request #27168 from excitoon-favorites/safers3readbuffer 
Safer `ReadBufferFromS3` for merges and backports
 2021-08-04 15:01:21 +03:00
Vladimir Chebotarev
cc32a61488 Fix. 2021-08-04 10:40:02 +03:00
kssenii
087ddeb0dc Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into taiyang-li-disk-reload 2021-08-02 15:16:09 +00:00
kssenii
b25dfdd6af Use method which was added for s3 instead 2021-08-02 08:15:21 +00:00
Kseniia Sumarokova
322870acd2
Update IDisk.h 2021-08-02 09:36:44 +03:00
Kseniia Sumarokova
9007ddfe22
Update DiskLocal.h 2021-08-02 09:34:39 +03:00
Kseniia Sumarokova
583c2d67cb
Update DiskSelector.cpp 2021-08-02 09:32:23 +03:00
Vitaly Baranov
a05184d83a Fix synchronization while updating from the config of an encrypted disk. 2021-08-01 22:55:36 +03:00
kssenii
f1e96dbc55 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into taiyang-li-disk-reload 2021-07-29 07:53:16 +00:00
Vitaly Baranov
51f882d8cc Support multiple keys for encrypted disk. Display error message if the key is probably wrong. 2021-07-24 23:45:12 +03:00
Vitaly Baranov
81ea8521b1 Set encryption algorithm explicitly. 2021-07-18 10:29:42 +03:00
Vitaly Baranov
958494b98e
Merge pull request #26377 from vitlibar/encrypted-disk-comments-and-improvements 
Add comments for encrypted disks; some improvements in code.
 2021-07-16 21:24:51 +03:00
Vitaly Baranov
9cc258327e Add comments for encrypted disks; some improvements in code. 2021-07-16 12:04:25 +03:00
Zhichang Yu
fe5d17680e Merge remote-tracking branch 'origin/master' into zero_copy_hdfs 2021-07-14 02:12:28 +00:00
Vitaly Baranov
bc5080d701 Rename ReadEncryptedBuffer -> ReadBufferFromEncryptedFile, 
WriteEncryptedBuffer -> WriteBufferFromEncryptedFile,
move file FileEncryption.h to the IO folder,
rename functions accroding to our code style,
add cpp files.
 2021-07-13 11:31:03 +03:00
Vitaly Baranov
042befb369 Fix compilation 2021-07-13 11:29:52 +03:00
Vitaly Baranov
e086e5e384
Merge branch 'master' into alexelex-encryption-1 2021-07-13 11:28:41 +03:00
Zhichang Yu
b4e6689bf9 fix test_hdfs_zero_copy_replication_move[tiered_copy-2] 2021-07-13 07:20:23 +00:00
Zhichang Yu
5047c758f4 fix per review 2021-07-13 07:20:20 +00:00
Zhichang Yu
fbd5eee8a1 hdfs zero copy 2021-07-13 07:19:12 +00:00
Alexey Milovidov
81646f8389 Merge branch 'master' into ditch-aio 2021-07-11 03:06:45 +03:00
Anton Ivashkin
ac68d5ea71 Fix path concatenation 2021-07-08 18:39:41 +03:00
Anton Ivashkin
88bc3995e0 Fix throwing exception when iterate over non existing remote directory 2021-07-08 17:54:59 +03:00
Alexey Milovidov
21fb6ddea5 Remove AIO for sequential reads 2021-07-05 23:02:24 +03:00
Anton Ivashkin
e0199d8006 Fix DiskS3 restore 2021-06-22 20:05:20 +03:00
kssenii
babb11a887 Some fixes 2021-06-19 19:18:13 +00:00
kssenii
2031607642 Reimplement 2021-06-19 11:52:28 +00:00
kssenii
7f356a5fbc Fix readbuffer 2021-06-18 17:06:17 +00:00
kssenii
f33387837e Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into disk-over-web-server 2021-06-18 14:27:48 +00:00
kssenii
0fa6dc4570 Reimplement 2021-06-18 11:17:37 +00:00
Vladimir
4de0d1a593
Merge pull request #24885 from Jokser/disk-s3-read-error-fix 2021-06-18 13:38:59 +03:00
Alexandra
5bc5a29c32 ContextConstPtr -> ContextPtr (resolve conflicts) 2021-06-18 05:36:50 +00:00
Latysheva Alexandra
48d3419160
Merge branch 'master' into alexelex-encryption-1 2021-06-18 11:28:08 +06:00
kssenii
cd885ee750 Fix build, ya make 2021-06-14 11:52:20 +00:00
kssenii
a1f11f5178 Fixes 2021-06-14 09:37:33 +00:00
kssenii
11faddb39f Better 2021-06-13 22:30:37 +00:00
kssenii
2cfcc9980c Better 2021-06-13 19:38:31 +00:00
kssenii
c80eac0dc3 First version 2021-06-13 19:36:56 +00:00
Alexey Milovidov
93b907931d Fix the annoying ya.make. 2021-06-13 21:21:44 +03:00
alexey-milovidov
05d1af153c
Merge branch 'master' into rename-const-context-ptr 2021-06-12 03:25:09 +03:00
alexey-milovidov
99f108d704
Merge pull request #25162 from kitaisreal/rewrite-function-to-subcolumn-visitor-add-into-ya-make 
RewriteFunctionToSubcolumnVisitor add into ya.make
 2021-06-12 02:58:53 +03:00
万康
3a404fcc1b fix build error 2021-06-11 23:21:18 +08:00
Maksim Kita
30c78c0a23 Fix arcadia 2021-06-11 14:26:33 +03:00
mergify[bot]
bfbc787b08
Merge branch 'master' into disk-s3-read-error-fix 2021-06-09 06:56:19 +00:00
Latysheva Alexandra
1a2cbe9275
Merge branch 'master' into alexelex-encryption-1 2021-06-07 21:18:54 +06:00
Ivan Lezhankin
0e9ce8c23a Fix Disks/ya.make 2021-06-07 17:22:24 +03:00
Pavel Kovalenko
6ba40c475a Code cleanup + comments. 2021-06-07 13:49:34 +03:00
mergify[bot]
b8dcce68b5
Merge branch 'master' into rename-const-context-ptr 2021-06-07 09:46:24 +00:00
Maksim Kita
16b3355ac1 Disks fix ya.make 2021-06-07 10:05:05 +03:00
mergify[bot]
d68d5e0ef1
Merge branch 'master' into rename-const-context-ptr 2021-06-04 14:41:41 +00:00
Pavel Kovalenko
6628b670df Fix reading from several buffers in ReadIndirectBufferFromRemoteFS 2021-06-03 14:34:23 +03:00
Pavel Kovalenko
2bd12a7612 Fix DiskS3 read error due to broken getPosition() method in ReadBufferFromS3.cpp 2021-06-02 18:03:25 +03:00
Nikita Mikhaylov
0055a924f6 better 2021-06-01 14:23:46 +00:00
Nikita Mikhaylov
3f18bde246 Merge branch 'master' of github.com:ClickHouse/ClickHouse into merging-s3-backoff 2021-06-01 13:39:09 +00:00
Nikolai Kochetov
dbaa6ffc62 Rename ContextConstPtr to ContextPtr. 2021-06-01 15:20:52 +03:00
Nikita Mikhaylov
e03c09c51b
Update DiskS3.cpp 2021-05-31 21:08:16 +03:00
Nikita Mikhaylov
0d8ede15a9
Merge branch 'master' into merging-s3-backoff 2021-05-31 21:07:00 +03:00
kssenii
69816e6eff Fix checks 2021-05-30 15:44:58 +03:00
kssenii
0a3589524e File movement 2021-05-29 00:57:53 +03:00
kssenii
2a631aaf08 Final fixes 2021-05-29 00:34:44 +03:00
kssenii
f66c67a979 Fixes 2021-05-27 15:42:46 +03:00
kssenii
f9088b98de Some disks update 2021-05-27 00:02:24 +03:00
kssenii
b28157a0df Merge branch 'master' of github.com:ClickHouse/ClickHouse into poco-file-to-std-fs 2021-05-27 00:01:25 +03:00
kssenii
73f16ee9ee Merge branch 'master' of github.com:ClickHouse/ClickHouse into poco-file-to-std-fs 2021-05-26 23:08:08 +03:00
kssenii
940eab6924 Review fixes 2021-05-26 09:26:19 +03:00
kssenii
379e26bcbc Fix checks 2021-05-25 17:37:35 +00:00
kssenii
e01abf807e Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into hdfs-idisk 2021-05-25 17:32:43 +00:00
mergify[bot]
400b74d186
Merge branch 'master' into merging-s3-backoff 2021-05-25 14:02:38 +00:00
kssenii
75a6b21f19 Better 2021-05-25 11:29:46 +00:00
kssenii
31107816bb fs::absolute to fs::canonical 2021-05-24 20:37:54 +03:00
Anton Ivashkin
4b9488be40 Add log record for removed from AWS S3 keys 2021-05-24 12:34:32 +03:00
kssenii
ecd82213d1 Some clean up, more tests 2021-05-23 20:19:04 +00:00
kssenii
0f98731c1f More integration tests 2021-05-23 07:33:31 +00:00
Alexandra
2bb25f356d createDir->createDirs in DiskEncrypted initialization 2021-05-23 06:49:02 +00:00
kssenii
6931573f69 Fix checks 2021-05-21 09:22:20 +00:00
kssenii
06ff4ab9da Fix build 2021-05-20 13:43:08 +00:00
kssenii
2f6770c9a0 Unite writeBuffers 2021-05-20 10:46:26 +00:00
kssenii
550a4e33f9 Unite readBuffers 2021-05-20 09:57:59 +00:00
Alexandra
11e40ad000 style fixes 2021-05-19 22:06:25 +00:00
Nikita Mikhaylov
f33b70c54a better 2021-05-20 00:42:25 +03:00
kssenii
9a5a6f39b7 Simplify writeFile 2021-05-19 20:31:43 +00:00
kssenii
5196075fd9 Better 2021-05-19 12:34:01 +00:00
Alexandra
f71e97511b set use_ssl flag 2021-05-18 20:48:16 +00:00
Alexandra
b7a0f2db57 refactoring 2021-05-18 20:20:51 +00:00
Alexandra
e3a91f568e fix encryption (very small files encryption bug) and replace 64-iv to 128-iv 2021-05-18 20:02:19 +00:00
kssenii
5f083b779d A little better, fix checks 2021-05-18 14:25:19 +03:00
Alexandra
721ea4f251 remove unnecessary includes 2021-05-17 16:58:51 +00:00
Alexandra
af420569f8 remove debug information, fix seek issue and encryption issue 2021-05-17 16:15:21 +00:00
Vladimir Chebotarev
bdf03387ab Better S3 read retries. Renamed s3_max_single_read_retries -> s3_single_read_retry_attempts, added sleeps before next attempts. 2021-05-17 10:38:32 +03:00
Alexandra
10a5b734ee encryption initialization (without any flags like use_ssl for now) 2021-05-16 22:35:20 +00:00
kssenii
018d52d71f Directory path 2021-05-15 09:20:26 +03:00
Alexandra
24573794ce inherit DiskEncrypted from DiskDecorator; override copy (use IDisk::copy instead of LocalDisk::copy), applyNewSettings, onFreeze and removeShared* 2021-05-14 21:52:51 +00:00
Alexandra
730b9792bc add parameters DiskEncrypted IDisk::applyNewSettings usage 2021-05-14 21:48:34 +00:00
kssenii
db849eb3e5 Remaining poco 2021-05-14 11:02:17 +03:00
kssenii
b123ec1354 Poco::FileException, Poco::parentPath 2021-05-14 00:20:28 +03:00
Alexandra
5374ac5c11 resolve alter table issue and add relative path to DiskEncrypted that can be declared in config as a <path> 2021-05-13 16:21:45 +00:00
Alexandra
7c34083441 mutex loop resolved 2021-05-13 06:46:11 +00:00
Alexandra
628bf9b345 disk encryption init 2021-05-13 06:37:05 +00:00
kssenii
e466924898 fs::premissions 2021-05-13 02:01:35 +03:00
kssenii
17432b1a41 Poco::moveTo to fs version 2021-05-12 17:32:22 +03:00
kssenii
047f8a4e3c Correct merge 2021-05-12 13:10:06 +00:00
kssenii
850b6f02cb Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into hdfs-idisk 2021-05-11 14:22:30 +00:00
kssenii
393b026f2e Merge branch 'master' of github.com:ClickHouse/ClickHouse into poco-file-to-std-fs 2021-05-11 16:49:23 +03:00
Vladimir
d02d14fa00
Merge pull request #23846 from excitoon-favorites/s3regions 2021-05-10 12:14:08 +03:00
kssenii
ab1a05a1f4 Poco::Path to fs::path, less concatination 2021-05-09 14:59:49 +03:00
kssenii
2d4c8b394c Return Poco::moveTo 2021-05-08 14:34:50 +03:00
kssenii
dc38da5e32 Fix fs::copy to be like Poco::copyTo 2021-05-08 14:00:31 +03:00
kssenii
35f999bf04 Poco::createFile to fs::createFile 2021-05-08 02:41:47 +03:00
kssenii
a2cfbd74ec Fix fs::copy to work the same as Poco::copy 2021-05-07 22:20:16 +03:00
kssenii
140bf7e2ea Less Poco 2021-05-07 15:29:26 +03:00
kssenii
5068b163b8 Fix 2021-05-05 19:55:02 +03:00
kssenii
2dabdd0f73 Merge branch 'master' of github.com:ClickHouse/ClickHouse into poco-file-to-std-fs 2021-05-05 18:42:40 +03:00
kssenii
9ec92ec514 Fix tests, less manual concatination of paths 2021-05-05 18:39:30 +03:00
kssenii
2b67341f28 Finish s3 2021-05-04 15:55:21 +03:00
vdimir
465341eafd
Add DiskRestartProxy.cpp to ya.make 2021-05-03 20:50:17 +03:00
kssenii
ad69fceea5 Restore copyFile 2021-05-02 19:59:18 +03:00
Vladimir Chebotarev
89242d4adc Fix. 2021-05-01 23:34:06 +03:00
Vladimir Chebotarev
a6ab040b07 Added region parameter for S3 storage and disk. 2021-05-01 20:45:35 +03:00
kssenii
32b2917141 Update s3/* 2021-05-01 18:10:04 +03:00
kssenii
ee06936596 Merge branch 'master' of github.com:ClickHouse/ClickHouse into poco-file-to-std-fs 2021-05-01 17:24:31 +03:00
Vladimir
3b7c68196b
Merge pull request #23429 from Jokser/disk-s3-restart-possibility 2021-05-01 15:28:14 +03:00
kssenii
2887ce1f69 Temporary rollback 2021-05-01 14:23:13 +03:00
kssenii
6867ea2c73 Temporary fix 2021-04-30 19:14:04 +03:00
kssenii
c609119daf Update Disks/* 2021-04-30 18:13:19 +03:00
kssenii
7f2f138d56 Update DiskLocal and DiskMemory 2021-04-30 11:32:34 +03:00
Maksim Kita
318c4bb80d Add examples folder filter to ya.make.in 2021-04-30 11:25:52 +03:00
Pavel Kovalenko
0050dbb75b Apply disk settings on config reload. 2021-04-29 23:32:19 +03:00
kssenii
984d82c3f2 Update DiskLocal 2021-04-29 02:00:04 +03:00
Pavel Kovalenko
838ede6c62 Rename delegate -> impl. 2021-04-27 15:11:38 +03:00
Pavel Kovalenko
5cedd90bed WriteBufferFromFileDecorator destructor fixes. 2021-04-27 13:19:22 +03:00
Pavel Kovalenko
63e5f5d3ab DiskS3 and WriteBufferFromFileDecorator small refactoring. 2021-04-27 11:37:59 +03:00
Nikita Mikhaylov
9f55424250 move to examples everywhere 2021-04-27 01:51:42 +03:00
kssenii
814e727d0e Better 2021-04-22 08:21:37 +00:00
Pavel Kovalenko
37c637cc34 Minor fixes. 2021-04-21 13:34:07 +03:00
Pavel Kovalenko
2ab8013455 DiskRestartProxy improvements and tests. 2021-04-21 12:53:46 +03:00
kssenii
87b4843134 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into hdfs-idisk 2021-04-20 21:40:01 +00:00
kssenii
8689a24ca0 Slightly better, add integration test with hdfs disk as default 2021-04-20 20:51:00 +00:00
Pavel Kovalenko
9272237c7e Update DiskRestartProxy.cpp 2021-04-20 23:47:21 +03:00
Pavel Kovalenko
10cf881a08 Add SYSTEM RESTART DISK command. 2021-04-20 22:23:54 +03:00
Pavel Kovalenko
016a9c674b DiskRestartProxy refactoring. 2021-04-20 21:58:34 +03:00
Pavel Kovalenko
98f8cbbdb2 DiskRestartProxy refactoring. 2021-04-20 21:58:34 +03:00
Pavel Kovalenko
e8b4dd1e4b Introduce DiskRestartProxy. 2021-04-20 21:55:57 +03:00
kssenii
d5fb7ba094 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into hdfs-idisk 2021-04-20 13:38:29 +00:00