Commit Graph

238 Commits

Author SHA1 Message Date
kssenii
003b807b00 Fix 2022-02-23 20:33:05 +01:00
Kruglov Pavel
dd863ca2a0
Merge branch 'master' into fix-url-globs 2022-02-16 12:45:31 +03:00
Dmitry Novik
50131fd732 Fix cancelation for S3 and HDFS 2022-02-11 18:14:55 +00:00
Anton Popov
18940b8637 Merge remote-tracking branch 'upstream/master' into HEAD 2022-02-09 23:38:38 +03:00
avogar
84087c1554 Improve schema inference with globs 2022-02-09 19:16:15 +03:00
alesapin
f6e7cea1f8 Add settings for S3 multipart upload parts size 2022-02-08 19:38:04 +03:00
feng lv
6325d4d9b0 continue of #34317
fix

fix
2022-02-06 08:59:17 +00:00
Anton Popov
836a348a9c Merge remote-tracking branch 'upstream/master' into HEAD 2022-02-01 15:23:07 +03:00
alesapin
5230c59f05 Smaller scope 2022-01-31 14:03:06 +03:00
alesapin
1adc7538fe Fix benign race condition for storage HDFS, S3, URL 2022-01-31 13:50:28 +03:00
Anton Popov
78b9f15abb Merge remote-tracking branch 'upstream/master' into HEAD 2022-01-30 03:24:37 +03:00
Anton Popov
e8ce091e68 Merge remote-tracking branch 'upstream/master' into HEAD 2022-01-21 20:11:18 +03:00
avogar
4efadfad3c Fix tests 2022-01-18 22:26:13 +03:00
avogar
af54ca4866 Fix partition write 2022-01-14 21:21:36 +03:00
Kruglov Pavel
a7df9cd53a
Merge branch 'master' into formats-with-suffixes 2022-01-14 21:03:49 +03:00
avogar
89a181bd19 Make better 2022-01-14 18:16:18 +03:00
avogar
8884c4a3fe Fix tests and build 2022-01-14 16:27:57 +03:00
avogar
2d7b1bfa5e Detect format in S3/HDFS/URL table engines 2022-01-13 16:14:18 +03:00
avogar
97788b9c21 Allow to create new files on insert for File/S3/HDFS engines 2021-12-29 21:19:13 +03:00
avogar
aaf9f85c67 Add more tests and fixes 2021-12-29 12:18:56 +03:00
avogar
8112a71233 Implement schema inference for most input formats 2021-12-29 12:18:56 +03:00
kssenii
1f6ca619b7 Allow some killing 2021-12-27 22:42:56 +03:00
avogar
58966e90e9 Don't allow to write into S3 if path contains globs 2021-12-24 17:13:35 +03:00
Anton Popov
ccd78e3838 Merge remote-tracking branch 'upstream/master' into HEAD 2021-11-22 17:19:35 +03:00
avogar
e5661b9b62 Minor change 2021-11-11 21:09:21 +03:00
Anton Popov
a20922b2d3 Merge remote-tracking branch 'origin/sparse-serialization' into HEAD 2021-11-09 15:36:25 +03:00
avogar
2dd4393ca1 Small refactoring in formats 2021-11-03 20:07:05 +03:00
Kseniia Sumarokova
065106c299
Merge pull request #30690 from kssenii/partitioned-write
Partitioned write for file, hdfs, url storages and table functions
2021-11-02 22:58:16 +03:00
kssenii
20772fde04 For storage 2021-10-27 10:04:17 +03:00
kssenii
2ba3ee830e Partitioned write part 2 2021-10-26 14:00:41 +03:00
kssenii
52296e8b04 Merge branch 'master' of github.com:ClickHouse/ClickHouse into disk-async-read 2021-10-25 00:22:32 +03:00
kssenii
e04e37916e Review fixes 2021-10-23 18:20:31 +03:00
kssenii
36d0f67b38 Merge branch 'master' of github.com:ClickHouse/ClickHouse into remote-connection 2021-10-23 17:12:46 +03:00
kssenii
39b9e9c258 Merge branch 'master' of github.com:ClickHouse/ClickHouse into disk-async-read 2021-10-22 15:40:41 +03:00
kssenii
4a0bdd296d Merge branch 'master' of github.com:ClickHouse/ClickHouse into disk-async-read 2021-10-16 18:35:48 +03:00
Nikolai Kochetov
a08c98d760 Move some files. 2021-10-16 17:03:50 +03:00
Nikolai Kochetov
fd14faeae2 Remove DataStreams folder. 2021-10-15 23:18:20 +03:00
kssenii
7b81766168 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into remote-connection 2021-10-15 12:36:44 +00:00
kssenii
881ae8617e Merge branch 'master' of github.com:ClickHouse/ClickHouse into disk-async-read 2021-10-15 15:09:56 +03:00
Nikolai Kochetov
ab28c6c855 Remove BlockInputStream interfaces. 2021-10-14 13:25:43 +03:00
Nikolai Kochetov
2957971ee3 Remove some last streams. 2021-10-13 21:22:02 +03:00
Nikolai Kochetov
ec18340351 Remove streams from formats. 2021-10-11 19:11:50 +03:00
kssenii
448d63a0fc Range read with upper bound offset 2021-10-11 01:53:51 +03:00
kssenii
5a87b6eb4f Predefined configuration for table function remote 2021-10-06 05:55:09 +00:00
Nikolai Kochetov
998d29ebc7 Merge branch 'master' into rewrite-pushing-to-views 2021-09-23 13:10:27 +03:00
Nikolai Kochetov
341553febd Fix build. 2021-09-16 20:40:42 +03:00
kssenii
1650c92407 Better 2021-09-16 11:38:26 +03:00
kssenii
221c09589c Review fixes 2021-09-15 21:11:49 +03:00
Nikolai Kochetov
b997214620 Rename QueryPipeline to QueryPipelineBuilder. 2021-09-14 20:48:18 +03:00
Nikolai Kochetov
f569a3e3f7 Merge branch 'master' into rewrite-pushing-to-views 2021-09-09 20:30:23 +03:00
Anton Popov
4c388e3d84 Merge remote-tracking branch 'origin/sparse-serialization' into HEAD 2021-09-09 14:10:16 +03:00
kssenii
e9721804b6 Fix tests 2021-09-09 12:34:47 +03:00
kssenii
3deb9a0ecb s3, url, http headers 2021-09-07 14:17:25 +03:00
Nikolai Kochetov
66a76ab70f Rewrite PushingToViewsBlockOutputStream part 6 2021-09-03 20:29:36 +03:00
Nikita Mikhaylov
e41d1cff1d Fix test 2021-08-31 12:54:45 +00:00
Nikita Mikhaylov
e5593f594d After rebase 2021-08-31 12:54:06 +00:00
Nikita Mikhaylov
ae241fc129 done 2021-08-31 12:54:06 +00:00
vdimir
6fe63a80bc Fix validateKey/Bucket for S3 2021-08-23 09:22:15 +03:00
vdimir
6ba6577978 Do not allow slashes in bucket formatted from PARTITION BY 2021-08-23 09:22:15 +03:00
vdimir
c4c42b61c4 Validate uft8 in partition key from PARTITION BY for s3 2021-08-23 09:22:15 +03:00
Vladimir Chebotarev
f6cb82aba5 Minor fix. 2021-08-23 09:22:15 +03:00
Vladimir Chebotarev
e203c330c2 Validation. 2021-08-23 09:22:14 +03:00
Vladimir Chebotarev
6e10d28c78 Fix. 2021-08-23 09:22:14 +03:00
Vladimir Chebotarev
ea67849fd9 Typo. 2021-08-23 09:22:14 +03:00
Vladimir Chebotarev
119172543b Better fix. 2021-08-23 09:22:14 +03:00
Vladimir Chebotarev
0b75c66378 Revert "Update src/Storages/StorageS3.cpp "
This reverts commit bbbfe1891bfdc868878cfc8d52c254babd4ca181.
2021-08-23 09:22:14 +03:00
Vladimir C
067cc763cc Update src/Storages/StorageS3.cpp
StringRef in fmt::format throws `fmt::v7::format_error: cannot switch from manual to automatic argument indexing`
2021-08-23 09:22:14 +03:00
Vladimir Chebotarev
d8bc601dbf Minor. 2021-08-23 09:22:14 +03:00
Vladimir Chebotarev
4b1cc38056 Fixes. 2021-08-23 09:22:14 +03:00
Vladimir Chebotarev
cebe1e7123 Style fix. 2021-08-23 09:22:14 +03:00
Vladimir Chebotarev
b34f60ad46 Fixed find. 2021-08-23 09:22:14 +03:00
Vladimir Chebotarev
be343766a6 Update src/Storages/StorageS3.cpp
Co-authored-by: Vladimir C <vdimir@yandex-team.ru>
2021-08-23 09:22:14 +03:00
Vladimir Chebotarev
24fcf152ed Minor fix. 2021-08-23 09:22:13 +03:00
Vladimir Chebotarev
5ce9e266b4 Less strict validation. 2021-08-23 09:22:13 +03:00
vdimir
4c9fb7ff58 Fix error code in StorageS3.cpp 2021-08-23 09:22:13 +03:00
vdimir
eb31b194b9 Add validation to insert partition by key to s3 2021-08-23 09:22:13 +03:00
Vladimir Chebotarev
d1efe3afa7 Review fixes. 2021-08-23 09:22:13 +03:00
Vladimir Chebotarev
e99433e094 Better functionality. 2021-08-23 09:22:13 +03:00
Vladimir Chebotarev
328213f5d9 Fixes. 2021-08-23 09:22:13 +03:00
Vladimir Chebotarev
1146efacff Partitioned writes for S3. 2021-08-23 09:22:13 +03:00
Anton Popov
61239343e3 Merge remote-tracking branch 'origin/sparse-serialization' into HEAD 2021-08-20 16:33:30 +03:00
Vladimir Chebotarev
c2410920d3 Safer ReadBufferFromS3 for merges and backports. 2021-08-04 09:14:20 +03:00
Anton Popov
e36736b50c Merge remote-tracking branch 'origin/sparse-serialization' into HEAD 2021-08-02 22:52:02 +03:00
Nikolai Kochetov
9b5a816b43 Merge branch 'master' into output-streams-to-processors 2021-07-26 18:03:11 +03:00
Nikolai Kochetov
9c92f43359 Update storages. 2021-07-23 22:33:59 +03:00
Nikolai Kochetov
f38de35b14 Rename some constants. 2021-07-21 19:13:17 +03:00
Nikolai Kochetov
179ec05a72 Remove some streams. 2021-07-20 21:18:43 +03:00
Anton Popov
3ed7f5a6cc dynamic subcolumns: add snapshot for storage 2021-07-09 06:15:41 +03:00
Kseniia Sumarokova
55ce7de248
Remove trailing spaces -- style check fix 2021-07-02 22:39:21 +03:00
Kseniia Sumarokova
ed687f45a1
Merge branch 'master' into truncate 2021-06-30 20:08:42 +03:00
Raúl Marín
bfc122df64 Fix some typos in Storage classes 2021-06-28 19:03:56 +02:00
kssenii
ac0f86cdbf Truncate for s3 2021-06-21 15:44:36 +00:00
Nikita Mikhaylov
0055a924f6 better 2021-06-01 14:23:46 +00:00
Nikita Mikhaylov
3f18bde246 Merge branch 'master' of github.com:ClickHouse/ClickHouse into merging-s3-backoff 2021-06-01 13:39:09 +00:00
Nikita Mikhaylov
f33b70c54a better 2021-05-20 00:42:25 +03:00
kssenii
9b8df78fdd Merge branch 'master' of github.com:ClickHouse/ClickHouse into poco-file-to-std-fs 2021-05-17 17:42:05 +03:00
Vladimir Chebotarev
bdf03387ab Better S3 read retries. Renamed s3_max_single_read_retries -> s3_single_read_retry_attempts, added sleeps before next attempts. 2021-05-17 10:38:32 +03:00
feng lv
9035e3a643 fix test and conflict 2021-05-11 16:22:24 +00:00
kssenii
393b026f2e Merge branch 'master' of github.com:ClickHouse/ClickHouse into poco-file-to-std-fs 2021-05-11 16:49:23 +03:00
Vladimir
d02d14fa00
Merge pull request #23846 from excitoon-favorites/s3regions 2021-05-10 12:14:08 +03:00