thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-16 19:32:07 +00:00
Code Issues Packages Projects Releases Wiki Activity
94,952 Commits 9,319 Branches 1,783 Tags 2.1 GiB
627424889d
Commit Graph

1241 Commits

Author SHA1 Message Date
Antonio Andelic
e4cd605edb
Merge pull request #37100 from ucasfl/fix-s3 
fix wrong argument in proxy resolver of DiskS3
 2022-05-13 10:00:13 +02:00
Kseniia Sumarokova
320efb2221
Merge branch 'master' into better-read-from-cache 2022-05-12 23:35:39 +02:00
Kseniia Sumarokova
3fd9517ff8
Merge pull request #37130 from ClickHouse/kssenii-patch-3 
Update default remote fs read method in ReadSettings
 2022-05-12 23:34:28 +02:00
alesapin
d8580c8cb8 Digging 2022-05-12 19:51:04 +02:00
Kseniia Sumarokova
6e0a910fcd
Merge branch 'master' into better-read-from-cache 2022-05-12 19:44:18 +02:00
alesapin
e7296a2b28
Merge pull request #36825 from kssenii/cache-fix-1 
Removed forceful drop cache command, fix detached status state
 2022-05-12 17:18:21 +02:00
kssenii
483e056bcc Merge master 2022-05-12 14:58:24 +02:00
kssenii
3c241582db Fix assertions 2022-05-12 14:54:18 +02:00
alesapin
ea389eeaca Fix fast test build 2022-05-12 12:44:32 +02:00
Kseniia Sumarokova
59bfcd917b
Merge pull request #36876 from kssenii/extend-cache-log 
Log into filesystem_cache_log when cache not even attempted
 2022-05-12 12:38:36 +02:00
mergify[bot]
21b4effce8
Merge branch 'master' into fix-s3 2022-05-12 06:50:23 +00:00
alesapin
5dcd25be23 Initial implementation 2022-05-12 00:04:54 +02:00
kssenii
810967fae3 Remove the check back 2022-05-11 23:01:25 +02:00
kssenii
dc1edbe22f Fix 2022-05-11 15:48:32 +02:00
kssenii
0adfee3eca Move assertion one layer higher 2022-05-11 12:33:40 +02:00
kssenii
18dd74be0c Remove unneede changes 2022-05-11 10:26:30 +02:00
fenglv
175175a16e fix wrong argument in proxy resolver of DiskS3 2022-05-11 04:16:05 +00:00
kssenii
ccab49e3cf Merge master 2022-05-10 19:33:34 +02:00
kssenii
f4b45a5805 Better profile events 2022-05-10 16:00:53 +02:00
kssenii
2f0f76369a Better 2022-05-09 21:25:26 +02:00
Anton Ivashkin
6a6505c224 Merge master 2022-05-09 17:21:34 +03:00
Anton Ivashkin
5c72da83d3 Fix typos 2022-05-09 17:19:53 +03:00
Robert Schulze
61cbcbf073
Enable clang-tidy readability-misleading-indentation 
Official docs:

  Correct indentation helps to understand code. Mismatch of the
  syntactical structure and the indentation of the code may hide serious
  problems.
 2022-05-08 19:12:01 +02:00
kssenii
0e9e1dd000 Fix assert in case cache forcefully dropped 2022-05-07 23:19:44 +02:00
Alexey Milovidov
4eac661588
Merge branch 'master' into master 2022-05-07 13:21:59 +03:00
kssenii
7e9913509f Fix build 2022-05-06 14:22:01 +02:00
kssenii
c60a3be0ce Better 2022-05-06 13:42:52 +02:00
kssenii
fd1ab053f8 Merge master 2022-05-06 12:50:33 +02:00
kssenii
23d9230acd Do not throw logical error for detached file segment 2022-05-06 12:48:16 +02:00
Kseniia Sumarokova
6d302c59e8
Merge pull request #36951 from ClickHouse/remove-strange-code 
Remove strange code
 2022-05-06 11:12:12 +02:00
Alexey Milovidov
996d838ca3
Merge pull request #36817 from DevTeamBK/Clang-tidy-Fixes 
Clang -Tidy Fixes
 2022-05-06 02:01:00 +03:00
Alexey Milovidov
490bdf5249 Fix style 2022-05-05 23:46:57 +02:00
Alexey Milovidov
e73dddb8c5 Remove strange code 2022-05-05 22:59:41 +02:00
kssenii
808ab75d74 Fix 2022-05-04 23:17:31 +02:00
Kruglov Pavel
d91b9b3049
Merge pull request #36882 from kssenii/add-current-metrics-for-filesystem-cache 
Add some CurrentMetrics for fs cache
 2022-05-04 14:44:12 +02:00
kssenii
ab65e163fa Add current metrics 2022-05-03 19:35:04 +02:00
kssenii
0266cdf125 Add entries to cache log when cache was not event attempted 2022-05-03 18:29:41 +02:00
kssenii
000f2c9f7a Better 2022-05-03 15:55:36 +02:00
Azat Khuzhin
da974585f1 Remove unimplemented RAID1 support 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-05-02 13:59:13 +03:00
KinderRiven
9c44ef1188 fix 2022-05-01 19:45:16 +08:00
KinderRiven
9413aa423b fix 2022-05-01 19:45:16 +08:00
KinderRiven
e0fc7870a8 fix style 2022-05-01 19:45:16 +08:00
KinderRiven
1145570c08 fix and add test for system.filesystem_cache_log 2022-05-01 19:45:16 +08:00
KinderRiven
e1acacf831 fix 2022-05-01 19:45:16 +08:00
KinderRiven
01210c8d38 fix header 2022-05-01 19:45:16 +08:00
KinderRiven
d80aa0fd17 impl system.filesystem_cache_log 2022-05-01 19:45:16 +08:00
KinderRiven
d595df1604 fix 2022-05-01 19:45:16 +08:00
KinderRiven
6a2e4559c8 impl system.cache_log 2022-05-01 19:45:16 +08:00
KinderRiven
ce9a6965d0 impl system.log_table 2022-05-01 19:45:16 +08:00
KinderRiven
9c9870f021 build cache_log table 2022-05-01 19:45:16 +08:00
KinderRiven
5a361a0db5 impl cache_log in filecache 2022-05-01 19:45:16 +08:00
Kseniia Sumarokova
66f62b2ba6
Merge pull request #36632 from DevTeamBK/Fix-Coverity-Issues 
Minor Coverity defects fixes
 2022-04-30 15:11:40 +02:00
Kseniia Sumarokova
56f37d54ab
Merge pull request #36809 from ClickHouse/kssenii-patch-3 
Update CachedReadBufferFromRemoteFS.cpp
 2022-04-30 12:57:51 +02:00
Alexey Milovidov
1ddb04b992
Merge pull request #36715 from amosbird/refactorbase 
Reorganize source files so that base won't depend on Common
 2022-04-30 09:40:58 +03:00
HeenaBansal2009
042157efca Clang -Tidy Fixes 2022-04-29 20:17:45 -07:00
Kseniia Sumarokova
7191d3d75b
Update CachedReadBufferFromRemoteFS.cpp 2022-04-29 19:13:35 +02:00
HeenaBansal2009
631b0154c0 Reverted change from ReadBufferFromRemoteFSGather.cpp 2022-04-29 07:21:35 -07:00
Kseniia Sumarokova
ca994e0861
Merge pull request #36660 from kssenii/fix-stress-test 
Fix stress test after 36639
 2022-04-29 12:56:25 +02:00
Amos Bird
4a5e4274f0
base should not depend on Common 2022-04-29 10:26:35 +08:00
mergify[bot]
41aa85fd4c
Merge branch 'master' into fix-stress-test 2022-04-29 00:54:09 +00:00
kssenii
84490d5198 Fix 2022-04-28 18:33:25 +02:00
HeenaBansal2009
3ac9434c57 Merge branch 'master' into Fix-Coverity-Issues 2022-04-28 08:14:50 -07:00
Sergei Trifonov
11f40376ae
Merge pull request #34571 from surahman/CH-31221-AWS-S3-object-version-id 
[CH-31221] AWS S3 Object Version Specification
 2022-04-28 15:45:33 +02:00
Kseniia Sumarokova
4c371f710e
Merge pull request #36676 from kssenii/refactor-with-size-buffer 
Better version of SeekableReadBufferWithSize
 2022-04-28 13:44:25 +02:00
kssenii
da1392877a Better 2022-04-27 20:01:59 +02:00
HeenaBansal2009
a34fca27d5 Incorporated review comments 2022-04-26 11:12:14 -07:00
HeenaBansal2009
13a9a8fd5e Style check fix 2022-04-26 07:13:09 -07:00
HeenaBansal2009
267f426605 Minor Coverity defetcs fixes 2022-04-26 07:13:09 -07:00
kssenii
9d364cdce2 Refactor 2022-04-26 15:33:53 +02:00
Sergei Trifonov
c1a0165f7d merge from master and fix conflicts 2022-04-26 12:39:16 +02:00
Sergei Trifonov
2aa17630be fix build 2022-04-26 12:30:04 +02:00
Vitaly Baranov
88b0cf3ca0 Fix TemporaryFileOnDisk 2022-04-25 16:33:38 +02:00
alesapin
2559d62d74 Better logging and bugfixes 2022-04-22 15:35:25 +02:00
mergify[bot]
cd8fa75546
Merge branch 'master' into fix_trash 2022-04-21 21:45:31 +00:00
alesapin
cbced6567d Fix build 2022-04-21 17:32:23 +02:00
alesapin
5465415751 Fix replace/move partition with zero copy replication 2022-04-21 14:39:12 +02:00
taiyang-li
957f3f13f7 fix build error 2022-04-21 20:19:29 +08:00
taiyang-li
fb34a4ec01 ready for review 2022-04-21 17:27:41 +08:00
taiyang-li
1ea96a8732 merge master and solve conflict 2022-04-21 17:16:07 +08:00
alesapin
c14e2e0b96 Fix more 2022-04-20 21:08:26 +02:00
Sergei Trifonov
34842f209c merge master and fix conflicts 2022-04-20 18:35:56 +02:00
Kseniia Sumarokova
df8312b357
Merge pull request #36388 from kssenii/fix-s3-tests-3 
Fix test 01161_all_system_tables under s3 storage
 2022-04-20 15:46:12 +02:00
alesapin
829854113b Add a little thread safety 2022-04-20 15:10:36 +02:00
taiyang-li
d533b569ad debugging 2022-04-20 19:58:31 +08:00
alesapin
612a81ba24
Update src/Disks/IDisk.h 
Co-authored-by: tavplubix <tavplubix@clickhouse.com>
 2022-04-20 13:58:31 +02:00
alesapin
a9131de657 Merge branch 'fix_trash' of github.com:ClickHouse/ClickHouse into fix_trash 2022-04-20 13:27:03 +02:00
alesapin
2eee79dc2a Review fixes 2022-04-20 13:26:20 +02:00
alesapin
40c15222f8 Merge branch 'master' into fix_trash 2022-04-20 12:45:49 +02:00
kssenii
5fc4aaaa2d Better 2022-04-20 10:42:07 +02:00
Kseniia Sumarokova
f37812f89a
Merge pull request #36079 from kssenii/change-hashed-path 
Change hashed path for cache
 2022-04-20 10:27:44 +02:00
taiyang-li
fb6a56d4b0 finish debug 2022-04-20 16:24:18 +08:00
Kseniia Sumarokova
766a84e64f
Merge pull request #35915 from kssenii/better-s3-settings 
Better settings configuration for s3
 2022-04-20 10:22:06 +02:00
alesapin
033a3475a5
Update src/Disks/IDiskRemote.cpp 2022-04-19 17:57:12 +02:00
kssenii
2fb331bcec Better 2022-04-19 17:32:46 +02:00
alesapin
7c2fcee351 Simplify interface 2022-04-19 14:13:11 +02:00
alesapin
cc06bc3d99 Add some clarifications 2022-04-19 14:01:30 +02:00
Sergei Trifonov
75c1053ad7 Merge remote-tracking branch 'origin/master' into surahman-CH-31221-AWS-S3-object-version-id 2022-04-19 13:17:25 +02:00
mergify[bot]
64d5e0c8a2
Merge branch 'master' into change-hashed-path 2022-04-19 09:33:48 +00:00
mergify[bot]
d22f538b41
Merge branch 'master' into fix-s3-tests-3 2022-04-19 09:32:54 +00:00
kssenii
1f9a597b77 Merge master 2022-04-19 11:31:13 +02:00
Kseniia Sumarokova
f7a2fa487d
Fix style check 2022-04-19 10:45:48 +02:00
taiyang-li
99ad2f5fa6 Merge remote-tracking branch 'origin/master' into async_hdfs_read_buffer 2022-04-19 15:49:38 +08:00
alesapin
bd7b3847c1 Some code 2022-04-19 01:09:09 +02:00
Robert Schulze
118e94523c
Activate clang-tidy warning "readability-container-contains" 
This check suggests replacing <Container>.count() by
<Container>.contains() which is more speaking and in case of
multimaps/multisets also faster.
 2022-04-18 23:53:11 +02:00
kssenii
40f7ddb6cc Fix 2022-04-18 15:07:07 +02:00
alesapin
c4b70f3ed3 Merge branch 'master' into fix_trash 2022-04-18 12:20:09 +02:00
Alexey Milovidov
7de4f2b6c1 Merge branch 'master' into remove-arcadia 2022-04-17 16:30:09 +02:00
taiyang-li
0ad2a76fae Merge remote-tracking branch 'origin/master' into async_hdfs_read_buffer 2022-04-16 18:45:39 +08:00
taiyang-li
cd83fd5f8a tobe debug 2022-04-16 18:41:18 +08:00
taiyang-li
f20e80c318 wip: deving 2022-04-16 16:19:48 +08:00
Alexey Milovidov
e128d89957 Remove Arcadia 2022-04-16 00:24:58 +02:00
Alexey Milovidov
416fa95b87 Remove "Arcadia" build system 2022-04-15 23:57:09 +02:00
alesapin
2fcc00380d Get rid of iheritance 2022-04-15 14:00:52 +02:00
kssenii
3e42a1711d Fix 2022-04-14 13:50:13 +02:00
Kseniia Sumarokova
04926d882a
Merge branch 'master' into change-hashed-path 2022-04-14 11:18:42 +02:00
mergify[bot]
2c5d6d132e
Merge branch 'master' into cache-fix 2022-04-13 13:32:56 +00:00
kssenii
0912b038fa Merge master 2022-04-13 15:30:01 +02:00
kssenii
05ceae428b Fix tests 2022-04-13 14:54:43 +02:00
Sergei Trifonov
9d1a2da685 merge master and fix conflicts 2022-04-13 14:53:36 +02:00
Kseniia Sumarokova
9ae047df1d
Update CachedReadBufferFromRemoteFS.cpp 2022-04-13 14:30:10 +02:00
kssenii
a6dc437911 Merge master 2022-04-13 14:23:48 +02:00
Kseniia Sumarokova
1b267462d8
Update CachedReadBufferFromRemoteFS.cpp 2022-04-13 11:27:47 +02:00
taiyang-li
b4cd8561a5 wip 2022-04-13 16:06:19 +08:00
kssenii
d2ccf876c6 Fix 2022-04-11 17:51:49 +02:00
Raúl Marín
6e346a7fc3 Support atomic replace under OSX 2022-04-11 14:32:55 +02:00
alesapin
324020a46f Restore class 2022-04-10 15:59:43 +02:00
alesapin
f61cad144c Get rid of strange abstraction 2022-04-10 15:55:05 +02:00
Kseniia Sumarokova
867b85dfad
Merge branch 'master' into change-hashed-path 2022-04-09 16:57:57 +02:00
kssenii
138588be29 Change hashed path 2022-04-08 17:14:13 +02:00
alesapin
4a03402da5
Update ReadBufferFromRemoteFSGather.h 2022-04-08 16:41:37 +02:00
alesapin
60e36abfe9 Get rid of metadata in buffers 2022-04-08 16:03:00 +02:00
Sergei Trifonov
fd001b5c80 fix param order 2022-04-08 15:51:53 +02:00
alesapin
ad9fdc1d79 Don't use metadata path in write buffers 2022-04-08 15:30:46 +02:00
alesapin
0db8808f2e Remove unused field from buffers 2022-04-08 15:06:43 +02:00
kssenii
87855c4ae7 Merge master 2022-04-08 12:32:48 +02:00
alesapin
8ec802bc62
Merge pull request #35475 from kssenii/remote-fs-cache-improvements 
Allow to write remote fs cache on all write operations. Add `system.remote_filesystem_cache` table. Add `drop remote filesystem cache (<path>)` query. Add `system.remote_data_paths` table.
 2022-04-08 12:06:26 +02:00
kssenii
0b7af1f26c Fix checks 2022-04-08 01:58:55 +02:00
kssenii
5dce2f18b5 Better 2022-04-07 18:48:35 +02:00
Sergei Trifonov
9c2804b924 merge from master + resolve conflicts 2022-04-07 11:22:01 +02:00
vdimir
4cb7b7e49b
Create parent directories in DiskLocal::replaceFile 2022-04-06 11:15:37 +00:00
kssenii
6c6fb5c3e8 Fix race in cached buffer 2022-04-04 16:32:37 +02:00
kssenii
4e50da722b Better s3 settings 2022-04-04 16:14:56 +02:00
kssenii
6c8e073a61 Merge master 2022-04-01 16:48:29 +02:00
alesapin
f2c6387a8d
Merge pull request #35381 from kssenii/remote-filesystem-cache-fixes 
Cache fixes for high concurrency on corner cases
 2022-04-01 10:48:31 +02:00
alesapin
e12860f646
Update src/Disks/IO/CachedReadBufferFromRemoteFS.cpp 2022-03-31 11:42:42 +02:00
alesapin
4c66c2e10f
Update src/Disks/IO/CachedReadBufferFromRemoteFS.cpp 2022-03-31 11:42:38 +02:00
kssenii
0fc92fe2aa Get rid of all "remote" mentions 2022-03-30 17:15:29 +02:00
kssenii
98ad3f4a91 Fix tests, rename some remote mentions 2022-03-30 12:21:59 +02:00
Antonio Andelic
03a09dc914
Merge pull request #35571 from ClickHouse/parallel-s3-downloading 
Parallel S3 downloading
 2022-03-30 08:14:49 +02:00
kssenii
f0e0d97752 Fix lask of query id in merges 2022-03-29 19:49:42 +02:00
kssenii
421b1e5a81 Update 2022-03-29 17:33:02 +02:00
Antonio Andelic
9990abb76a Use compile-time check for Exception messages, fix wrong messages 2022-03-29 13:16:11 +00:00
Antonio Andelic
f0b4a9fd15 Fix construction of ReadBufferFromS3 2022-03-29 10:56:49 +00:00
kssenii
eab04674fa Fix review comments 2022-03-29 12:38:17 +02:00
Azat Khuzhin
6da2c48b7a Add missing __builtin_unreachable for switch/case for gcc 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-03-26 23:42:21 +03:00
kssenii
141c290344 Fix build 2022-03-25 18:31:15 +01:00
kssenii
e59dbe72c7 Revert accidental commit 2022-03-25 17:53:02 +01:00
kssenii
9aaba43a0a Remove incorrect assertion 2022-03-25 16:29:30 +01:00
kssenii
0c6577eee0 Merge master 2022-03-24 20:49:02 +01:00
kssenii
6769b84638 Merge master 2022-03-24 20:47:43 +01:00
kssenii
f9b538fb0c Fix 2022-03-24 15:38:29 +01:00
kssenii
d6ab6de30a Minor changes 2022-03-24 15:32:08 +01:00
kssenii
d4161b5925 Add optin read_from_cache_if_exists_otherwise_bypass_cache (for merges) 2022-03-23 20:24:00 +01:00
kssenii
d2a3cfe5dc Cache on all write operations 2022-03-23 19:14:33 +01:00
kssenii
e39aba37a2 Minor changes 2022-03-23 18:11:52 +01:00
Sergei Trifonov
bf9c6be6c6 move threadPoolCallbackRunner from Common into Interpreters folder to fix split build issue 2022-03-23 17:57:19 +01:00
kssenii
34c0690a69 Better 2022-03-23 15:43:14 +01:00
ianton-ru
32c6390400
Merge branch 'master' into MDB-16962 2022-03-23 16:53:30 +03:00
Anton Ivashkin
66c026d0ac Fix collision of S3 operation log revision 2022-03-23 16:48:05 +03:00
kssenii
f3e1ca44a9 Add system.remote_data_paths table 2022-03-23 13:01:18 +01:00
kssenii
ccdf7d0231 Fix typos check 2022-03-21 21:51:17 +01:00
kssenii
afd0c64a1a Better 2022-03-21 15:04:16 +01:00
kssenii
eabbce69a7 Add system drop remote filesystem cache query, allow to write cache on insert 2022-03-21 12:32:44 +01:00
kssenii
59b7394caf WriteSettings 2022-03-21 09:52:48 +01:00
kssenii
1d80262a2a Add write settings, file cache settings 2022-03-21 09:50:50 +01:00
mergify[bot]
7ac606fa65
Merge branch 'master' into parallel-multipart-upload-for-s3storage 2022-03-20 18:25:28 +00:00
kssenii
43ed2beb3c Merge master 2022-03-18 12:33:08 +01:00
kssenii
af3bb3b7af Add WriteSettings 2022-03-18 12:27:38 +01:00
kssenii
44fb6fb86f Revert some changes 2022-03-18 10:16:06 +01:00
kssenii
a7c9fe5c78 Finally-finally fixed 2022-03-17 20:29:07 +01:00
kssenii
c698fb9539 Debug 2022-03-17 18:24:00 +01:00
kssenii
1b6e7fea2f Finally fixed 2022-03-17 16:40:10 +01:00
Nikolai Kochetov
ee9c2ec735
Merge pull request #34780 from azat/mt-delayed-part-flush 
Do not delay final part writing by default (fixes possible Memory limit exceeded during INSERT)
 2022-03-17 12:30:51 +01:00
kssenii
f354b704e0 Add more checks, different counting for download size 2022-03-17 11:52:59 +01:00
Sergei Trifonov
bbe657ba24 allow parallelization of multipart S3 storage upload fixes #34350 2022-03-16 19:41:41 +01:00
Nikolai Kochetov
a380aa6b8a
Merge pull request #35294 from ClickHouse/reload-remote_url_allow_hosts 
Reload remote_url_allow_hosts after config update.
 2022-03-15 22:07:16 +01:00
Nikolai Kochetov
cf260b8508
Merge pull request #35305 from ClickHouse/try-to-fix-using-deleted-memory-tracker-inside-s3-disk 
Maybe fix use-after-free inside S3 upload thread
 2022-03-15 22:06:17 +01:00
Nikolai Kochetov
b4aed421de Reset THreadPool's thread memory tracker parent to global after S3 upload task is finished. 2022-03-15 15:33:43 +00:00
Nikolai Kochetov
97aa6c82ce Reload remote_url_allow_hosts after config update. 2022-03-15 13:00:31 +00:00
Saad Ur Rahman
7bdbaf28d8
[Disks] bugfix 🤦‍♂️ 
Correcting order of parameters for <ReadBufferFromS3> in <readSchemaVersion>.
 2022-03-14 22:42:09 -04:00
Saad Ur Rahman
40fbea49f7
[Disks] bugfix 
Fixed order of parameters to <DiskS3> constructor:
 - name
 - bucket
 - key
 - versionId
 - ...
 2022-03-14 22:37:47 -04:00
Saad Ur Rahman
fea8824489
[Disks] Added Version ID parameter to registerDisksS3. 2022-03-14 22:06:09 -04:00
Saad Ur Rahman
e181207f2d
[Disks] Version ID parameter added to readFile 2022-03-14 21:30:23 -04:00
Saad Ur Rahman
98b2994352
[Disks] ReadBufferFromS3 requires a version id. 2022-03-14 21:30:23 -04:00
Saad Ur Rahman
e3a96393a1
[Disks] Version ID added. 
Version ID added to constructor.
 2022-03-14 21:30:23 -04:00
Saad Ur Rahman
f299bf46e2
[Disks] readSchemaVersion supplies empty Version ID 
Empty version ID supplied to retrieve version ID of object.
 2022-03-14 20:11:20 -04:00
mergify[bot]
ee3f4fb0c3
Merge branch 'master' into add-cache-optimization 2022-03-14 20:58:52 +00:00
Maksim Kita
538f8cbaad Fix clang-tidy warnings in Disks, Formats, Functions folders 2022-03-14 18:17:35 +00:00
kssenii
fc9c6eab9d Add optimization 2022-03-14 17:58:19 +01:00
alesapin
b149f28cd3
Merge pull request #35209 from kssenii/cache-fix 
Cache fix
 2022-03-14 11:07:40 +01:00
Kseniia Sumarokova
35e5b4e8a5
Update CachedReadBufferFromRemoteFS.cpp 2022-03-13 12:37:00 +01:00
kssenii
a18bb606a2 Update 2022-03-11 14:24:02 +01:00
kssenii
2b478e2501 Fix 2022-03-11 12:17:17 +01:00
kssenii
c86955deb2 Fix typo 2022-03-10 20:22:39 +01:00
kssenii
ddcb020d26 Fix test 2022-03-10 19:16:26 +01:00
kssenii
6571c7faa0 Clean up code 2022-03-09 10:36:52 +01:00
kssenii
96789fa5ce Fix test, disable cache for DiskCacheWrapper 2022-03-09 09:46:04 +01:00
kssenii
16e84c265b Fix build, update test 2022-03-08 21:41:58 +01:00
Azat Khuzhin
4200b56f3a Introduce IDisk::supportParallelWrite() 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-03-08 22:17:36 +03:00
kssenii
5260822964 Merge master 2022-03-08 18:21:28 +01:00
kssenii
e231c3a3e0 Fix split build 2022-03-08 18:05:55 +01:00
kssenii
f3cba6f270 Remove already fixed crutch 2022-03-08 14:30:21 +01:00
kssenii
28a3aece60 Fix uncaught exception from destructor 2022-03-08 10:58:37 +01:00
Kseniia Sumarokova
517e878c6e
Merge pull request #35099 from ClickHouse/tavplubix-patch-1 
Fix inconsistency in DiskLocal
 2022-03-08 10:18:07 +01:00
kssenii
0737aef472 Fix split build 2022-03-07 22:02:07 +01:00
kssenii
eeb0ae9816 Fix detaching query 2022-03-07 17:05:36 +01:00
tavplubix
84e22fb32b
Update DiskLocal.cpp 2022-03-07 18:59:00 +03:00
kssenii
059d239815 Simplify code 2022-03-07 15:17:38 +01:00
kssenii
692b247379 Fix some corner cases 2022-03-07 14:31:21 +01:00
mergify[bot]
88052e2d7c
Merge branch 'master' into finally-enable-s3-async-writes-again 2022-03-07 12:55:52 +00:00
kssenii
2dcfe66236 Fix race 2022-03-06 20:33:07 +01:00
Kseniia Sumarokova
299e11bb2b
Update DiskCacheWrapper.cpp 2022-03-03 11:11:57 +01:00
kssenii
192b839cab Fix 2022-03-02 19:11:45 +01:00
kssenii
b2cab429a7 Move initialization out of constructor 2022-03-01 18:12:34 +01:00
kssenii
438c63f3d8 Style 2022-03-01 17:52:38 +01:00
Nikolai Kochetov
eaf6d8c5f6
Update DiskS3.cpp 2022-02-24 17:35:29 +01:00
kssenii
8d6f1982df Tiny refactoring 2022-02-23 14:43:40 +01:00
kssenii
36a76d6a75 Fix test 2022-02-22 15:18:28 +01:00
kssenii
c637385dd0 Merge master 2022-02-22 13:17:51 +01:00
Kseniia Sumarokova
eeea322556
Merge pull request #34629 from amosbird/remotefsimprove 
Some refactoring and improvement over async and remote buffer related stuff
 2022-02-22 11:36:40 +01:00
kssenii
fad7c042c1 Fix cache with non-async reads 2022-02-21 19:50:27 +01:00
kssenii
4543513a5d Add comments 2022-02-21 13:54:03 +01:00
kssenii
e8b2e47e93 Style 2022-02-19 20:38:23 +01:00
kssenii
6b60f7c57f Add limit for file segment size and split it if exceeds the limit 2022-02-19 18:59:22 +01:00
Amos Bird
84d111cde9
Better ReadBuffer asynchronicity 2022-02-20 01:08:21 +08:00
Nikolai Kochetov
40a48c5670 Enable async writes to S3. 2022-02-18 18:32:04 +00:00
kssenii
acf9a092a9 Review fixes 2022-02-18 19:09:54 +01:00
kssenii
353ffacca3 Add test 2022-02-17 19:57:04 +01:00
Amos Bird
dc0b4a8f73
Remove useless template 2022-02-17 14:31:21 +08:00
kssenii
47f94120da Merge master 2022-02-16 14:43:28 +01:00
kssenii
3bd3e51aa0 Fix tests 2022-02-16 14:08:41 +01:00
Vladimir C
724340418e
Merge pull request #34617 from Algunenano/freebsd_build 
Attempt to fix freeBSD build
 2022-02-15 20:12:44 +01:00
Raúl Marín
72efc00830 Attempt to fix freeBSD build 2022-02-15 17:23:01 +01:00
alesapin
bc2d0ee7c7
Merge pull request #34215 from ClickHouse/revert-34211-revert-34153-add_func_tests_over_s3 
Add func tests run with s3 and fix several bugs
 2022-02-15 19:07:11 +03:00
kssenii
886b300b8d Less seeks 2022-02-15 16:12:25 +01:00
mergify[bot]
34973f1d5c
Merge branch 'master' into first-test 2022-02-15 12:15:53 +00:00
alesapin
447cd56cb9 Fix comments 2022-02-15 12:11:50 +03:00
alesapin
e15396d90c Fix race condition: 2022-02-14 22:19:49 +03:00
kssenii
bdbbe85f95 Fixes 2022-02-14 19:28:48 +01:00
zxealous
3d29920005 Fix compilation issues with DiskHDFS 2022-02-14 13:14:20 +08:00
Danila Kutenin
1c589681a0
Have a correct include which was inlined 2022-02-12 12:49:23 +00:00
alesapin
088f8cacf3 Small improvements 2022-02-10 19:50:21 +03:00
Anton Popov
70986a70a1 support TTL TO [DISK|VOLUME] [IF EXISTS] 2022-02-10 19:26:23 +03:00
alesapin
3387f9e597 Possible fix DiskHDFS 2022-02-10 16:55:12 +03:00
alesapin
10c3e6e546 Merge branch 'master' into revert-34211-revert-34153-add_func_tests_over_s3 2022-02-09 14:14:58 +03:00
alesapin
36909a986f Fix bug with files remove 2022-02-08 22:21:16 +03:00
alesapin
02a93cb852 Merge branch 'master' into revert-34211-revert-34153-add_func_tests_over_s3 2022-02-08 19:42:27 +03:00
alesapin
f6e7cea1f8 Add settings for S3 multipart upload parts size 2022-02-08 19:38:04 +03:00
alesapin
07d5227568 Disable thread pool 2022-02-08 14:08:00 +03:00
alesapin
773d9cede3 Fix cache wrapper 2022-02-08 14:05:38 +03:00
alesapin
3af6012cb4 Revert "Revert "Revert "Revert "Merge pull request #34219 from ClickHouse/revert-34212-revert-33291-add-pool-to-s3-write-buffer"""" 
This reverts commit 2bc2ea485e.
 2022-02-08 11:01:26 +03:00
alesapin
2bc2ea485e Revert "Revert "Revert "Merge pull request #34219 from ClickHouse/revert-34212-revert-33291-add-pool-to-s3-write-buffer""" 
This reverts commit fb77d7a7d5.
 2022-02-08 10:56:29 +03:00
kssenii
2e58733750 Merge master 2022-02-07 15:23:10 +01:00
Anton Popov
1b16db72c3 fix consecutive backward seeks in seekable read buffers 2022-02-07 17:20:26 +03:00
Nikolai Kochetov
40b86e3dae Add blind change. 2022-02-07 12:02:21 +00:00
alesapin
fb77d7a7d5 Revert "Revert "Merge pull request #34219 from ClickHouse/revert-34212-revert-33291-add-pool-to-s3-write-buffer"" 
This reverts commit 875e5413ad.
 2022-02-07 12:36:54 +03:00
alesapin
25c0b07e68 Disable some tests 2022-02-04 14:31:58 +03:00
alesapin
875e5413ad Revert "Merge pull request #34219 from ClickHouse/revert-34212-revert-33291-add-pool-to-s3-write-buffer" 
This reverts commit b92efed350, reversing
changes made to ecce006cb2.
 2022-02-04 14:30:33 +03:00
alesapin
5e049fdd47 Merge with master 2022-02-04 12:28:19 +03:00
alesapin
2ed45b2a98 Merge branch 'master' into revert-34211-revert-34153-add_func_tests_over_s3 2022-02-04 11:23:46 +03:00
alesapin
85fa727e32 Fix tidy 2022-02-04 11:21:03 +03:00
alesapin
a61b231ddd Better debug 2022-02-03 11:23:09 +03:00
alesapin
ab546f61e3 Disable cache by default 2022-02-02 22:06:08 +03:00
Nikolai Kochetov
2003a96b58 Probably fix something 2022-02-02 17:48:41 +00:00
alesapin
80800e051e Merge branch 'master' into revert-34211-revert-34153-add_func_tests_over_s3 2022-02-02 19:42:04 +03:00
Nikolai Kochetov
ea044fc6b2 Merge branch 'master' into revert-34212-revert-33291-add-pool-to-s3-write-buffer 2022-02-02 19:40:41 +03:00
alesapin
b9c118524f Fix race condition on hardlink/erase/read metadata 2022-02-02 19:40:21 +03:00
kssenii
aeec749173 Fix predownload case part 1 2022-02-02 17:25:25 +03:00
kssenii
5cba11428a Fixes 2022-02-02 00:25:16 +03:00
Azat Khuzhin
bedf208cbd Use fmt::runtime() for LOG_* for non constexpr 
Here is oneliner:

    $ gg 'LOG_\(DEBUG\|TRACE\|INFO\|TEST\|WARNING\|ERROR\|FATAL\)([^,]*, [a-zA-Z]' -- :*.cpp :*.h | cut -d: -f1 | sort -u | xargs -r sed -E -i 's#(LOG_[A-Z]*)\(([^,]*), ([A-Za-z][^,)]*)#\1(\2, fmt::runtime(\3)#'

Note, that I tried to do this with coccinelle (tool for semantic
patchin), but it cannot parse C++:

    $ cat fmt.cocci
    @@
    expression log;
    expression var;
    @@

    -LOG_DEBUG(log, var)
    +LOG_DEBUG(log, fmt::runtime(var))

I've also tried to use some macros/templates magic to do this implicitly
in logger_useful.h, but I failed to do so, and apparently it is not
possible for now.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>

v2: manual fixes
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-02-01 14:30:03 +03:00
Amos Bird
e29c77f793 Fix link order issue of WriteBufferFromS3 
(cherry picked from commit f22b09f4fc)
 2022-02-01 10:42:06 +00:00
Nikolai Kochetov
2a6eb593be
Revert "Revert "Add pool to WriteBufferFromS3"" 2022-02-01 13:36:51 +03:00
alesapin
18f08ed932
Revert "Revert "Add func tests run with s3"" 2022-02-01 10:20:06 +03:00
alexey-milovidov
99392b5ca7
Merge pull request #13544 from amosbird/mdha 
Multi-Disk auto-recovery.
 2022-02-01 06:13:26 +03:00
alexey-milovidov
095d9bfa43
Revert "Add pool to WriteBufferFromS3" 2022-02-01 05:49:40 +03:00
alexey-milovidov
cbfcd45be3
Revert "Add func tests run with s3" 2022-02-01 05:46:13 +03:00
alexey-milovidov
a2aa147ce0
Merge pull request #34153 from ClickHouse/add_func_tests_over_s3 
Add func tests run with s3
 2022-02-01 05:46:09 +03:00
Amos Bird
ec7d367814
DiskLocal checker 
Add DiskLocal checker so that ReplicatedMergeTree can recover data when some of its disks are broken.
 2022-02-01 05:55:27 +08:00
alesapin
cbf4790ea4 More clear error 2022-01-31 13:59:16 +03:00
kssenii
218a633fec Some fixes, more assertions and logging 2022-01-31 10:51:34 +03:00
kssenii
6b03c577d6 Make downloader's term resticted to one nextImpl 2022-01-29 23:58:53 +03:00
Nikolai Kochetov
1c9f026178 Merge branch 'master' into add-pool-to-s3-write-buffer 2022-01-28 16:01:42 +03:00
kssenii
c91b86e220 Better 2022-01-26 21:56:08 +03:00
Nikolai Kochetov
a8171269a1 Review fixes. 2022-01-26 17:55:24 +00:00
Nikolai Kochetov
efaed154d2
Update src/Disks/IDisk.h 
Co-authored-by: alesapin <alesapin@clickhouse.com>
 2022-01-26 14:58:38 +03:00
Nikolai Kochetov
2e88319b72 Fix integration tests. 2022-01-25 18:48:28 +00:00
Nikolai Kochetov
fcc29dbd15 Try to fix integration tests. 2022-01-25 15:26:36 +00:00
kssenii
aef99dedba Fixes 2022-01-25 01:46:44 +03:00
Nikolai Kochetov
4afa75e80b Fix deduplication. 2022-01-24 14:43:36 +00:00
kssenii
898139acae Fix tests 2022-01-24 12:32:39 +03:00