Commit Graph

125 Commits

Author SHA1 Message Date
Kseniia Sumarokova
065106c299
Merge pull request #30690 from kssenii/partitioned-write
Partitioned write for file, hdfs, url storages and table functions
2021-11-02 22:58:16 +03:00
Maksim Kita
0d7dfdccf9 Fixed tests 2021-10-30 01:29:36 +03:00
kssenii
20772fde04 For storage 2021-10-27 10:04:17 +03:00
Alexey Milovidov
cba6e39fe5 Replace yandex to clickhouse in configs 2021-09-25 07:08:34 +03:00
alexey-milovidov
b8083929a6
Merge branch 'master' into yandex-to-clickhouse-in-configs 2021-09-23 20:49:40 +03:00
Alexey Milovidov
e513a5db32 Change <yandex> to <clickhouse> in configs 2021-09-20 01:38:53 +03:00
kssenii
41bc68e82e Tests, url table function 2021-09-08 22:28:22 +03:00
alexey-milovidov
35d5957683
Merge pull request #28336 from ianton-ru/MDB-13951
Fix test_storage_s3/test_put_get_with_globs (cleanup after test)
2021-08-31 01:54:59 +03:00
Anton Ivashkin
2e33072469 Fix test_storage_s3/test_put_get_with_globs (cleanup after test) 2021-08-30 13:31:46 +03:00
vdimir
6ba6577978 Do not allow slashes in bucket formatted from PARTITION BY 2021-08-23 09:22:15 +03:00
vdimir
c4c42b61c4 Validate uft8 in partition key from PARTITION BY for s3 2021-08-23 09:22:15 +03:00
Vladimir Chebotarev
2223ae4ddc Added integration test. 2021-08-23 09:22:14 +03:00
Vladimir Chebotarev
f45dbd6cbc Fixed test. 2021-08-23 09:22:14 +03:00
Vladimir Chebotarev
b4484f4f21 Test fix. 2021-08-23 09:22:13 +03:00
Vladimir Chebotarev
e99433e094 Better functionality. 2021-08-23 09:22:13 +03:00
Vladimir Chebotarev
328213f5d9 Fixes. 2021-08-23 09:22:13 +03:00
Vladimir Chebotarev
0879dbbfa7 Test. 2021-08-23 09:22:13 +03:00
Yatsishin Ilya
be9a0a9ca2 better test_storage_s3, test_storage_s3_get_unstable still not OK 2021-07-15 15:49:25 +03:00
Kseniia Sumarokova
ed687f45a1
Merge branch 'master' into truncate 2021-06-30 20:08:42 +03:00
Kseniia Sumarokova
f47dd116c4
Update test.py 2021-06-21 19:07:17 +03:00
kssenii
ac0f86cdbf Truncate for s3 2021-06-21 15:44:36 +00:00
alesapin
c7781e24d5 Fix some flaky tests 2021-06-21 11:02:27 +03:00
Vladimir Chebotarev
f16dd06476 Review fixes. 2021-06-10 08:49:26 +03:00
Vladimir Chebotarev
591c556321 Improved test_storage_s3_get_unstable. 2021-06-10 08:49:26 +03:00
Alexey Milovidov
b146524fdb Maybe Minio starts for too long in tests 2021-06-06 12:38:49 +03:00
mergify[bot]
9e2c454db2
Merge branch 'master' into merging-s3-backoff 2021-06-03 18:58:24 +00:00
Yatsishin Ilya
ddddb761db more 2021-06-02 18:08:16 +03:00
Yatsishin Ilya
ff7f7f6200 try 2021-06-01 17:18:35 +03:00
Vladimir Chebotarev
752d0cb0b6 Fixed test. 2021-05-17 10:38:34 +03:00
Yatsishin Ilya
ad706b0de4 Merge remote-tracking branch 'origin' into integration-2 2021-05-14 16:48:44 +03:00
Vladimir Chebotarev
1141872db2 Fix tests II. 2021-05-13 08:46:16 +03:00
Yatsishin Ilya
fe0194aa4b s3 catch up 2021-05-12 10:03:53 +03:00
Yatsishin Ilya
9d159072ef mERGe remote-tracking branch 'origin' into integration-2 2021-05-11 13:32:44 +03:00
Vladimir Chebotarev
524113f497 Fixed bug with S3 URLs containing + symbol, data with such keys could not be read previously. 2021-05-06 11:28:55 +03:00
Yatsishin Ilya
dd192bb0ab better 2021-04-29 17:26:41 +03:00
Yatsishin Ilya
cb101e46bb finally 2021-04-29 14:57:48 +03:00
Yatsishin Ilya
c59a11f798 Merge remote-tracking branch 'origin' into integration-2 2021-04-27 18:34:33 +03:00
Vladimir Chebotarev
edd3931d4e Fixed tests. 2021-04-15 09:41:49 +03:00
Vladimir Chebotarev
5b8a3831e6 Better tests. 2021-04-15 09:41:49 +03:00
Vladimir Chebotarev
a63db53576 Improved tests. 2021-04-15 09:41:49 +03:00
Vladimir Chebotarev
ce6cf1630b Added tests. 2021-04-15 09:41:49 +03:00
Yatsishin Ilya
10b7037d9c Merge remote-tracking branch 'origin' into integration-2 2021-04-14 14:21:40 +03:00
Yatsishin Ilya
e4d1ea846c more 2021-04-13 13:52:22 +03:00
Yatsishin Ilya
b4dded3fc2 wip 2021-04-12 10:03:12 +03:00
Yatsishin Ilya
5311278c29 Merge remote-tracking branch 'origin' into integration-2 2021-04-08 12:30:24 +03:00
Vladimir Chebotarev
213a561e21 Improved test. 2021-03-29 10:07:41 +03:00
Vladimir Chebotarev
d157540254 Added test for put with compression and wrong credentials. 2021-03-29 10:07:41 +03:00
Yatsishin Ilya
df8013c22d fix 2021-03-26 21:46:42 +03:00
Yatsishin Ilya
5ae47bd2b4 Merge remote-tracking branch 'origin/master' into integration-2 2021-03-19 15:33:14 +03:00
Yatsishin Ilya
579d865aac merge remote-tracking branch 'origin' into integration-2 2021-03-16 13:26:51 +03:00
Yatsishin Ilya
06bb4e2c71 wip 2021-03-16 13:00:49 +03:00
vdimir
1194d50e48
Try to fix test_storage_s3: crash in WriteBufferFromS3 2021-03-11 15:21:13 +03:00
Pervakov Grigorii
eb87afc4d4 recreate S3 client if credentials changed 2021-03-10 13:36:57 +03:00
Yatsishin Ilya
babea212c7 Merge remote-tracking branch 'origin' into integration-2 2021-03-03 11:56:15 +03:00
Yatsishin Ilya
fc89b4fd35 better 2021-02-20 17:59:39 +03:00
Vladimir Chebotarev
d729aacb09 Fixed table function S3 auto compression mode. 2021-01-29 07:54:52 +03:00
Vladimir Chebotarev
8ba8e8efd5 Added prefix-based S3 endpoint settings. 2021-01-25 06:40:57 +03:00
Pavel Kovalenko
db7a87089b Adaptive choose of single/multi part upload in WriteBufferFromS3. 2020-12-09 17:09:04 +03:00
alesapin
45f09fe293
Merge pull request #17220 from ianton-ru/s3_max_redirects
S3 max redirects
2020-11-24 12:13:10 +03:00
Anton Ivashkin
aa208a9d9f Use only 's3_max_redirect' in params instead of all settings 2020-11-23 14:02:17 +03:00
Pavel Kovalenko
5f98d38689 Fixed flaky test_storage_s3::test_custom_auth_headers 2020-11-23 13:19:43 +03:00
Anton Ivashkin
a7d4f4be65 Add 's3_max_redirects' test 2020-11-20 11:18:44 +03:00
Pavel Kovalenko
e311759c5d
Abort multipart upload if no data was written to WriteBufferFromS3. (#16840) 2020-11-11 15:15:16 +03:00
Azat Khuzhin
9cb3c743bd
Convert to python3 (#15007) 2020-10-02 19:54:07 +03:00
Vladimir Chebotarev
b460024356 Fixed bug in GZIP compression in S3 storage. 2020-09-30 16:09:55 +03:00
Vladimir Chebotarev
a38c7ff2a8 Somehow uncompressed PUT works and gzipped PUT doesn't, in S3 storage. 2020-09-30 15:04:42 +03:00
Vladimir Chebotarev
b7a6f15f19 Added test for GZIP in S3 storage. 2020-09-30 13:55:44 +03:00
bharatnc
b75963d370 Format and optimize imports in integration test files
This PR formats all the `*.py` files found under the `tests/integration`
folder. It also reorders the imports and cleans up a bunch of unused
imports.

The formatting also takes care of other things like wrapping lines and
fixing spaces and indents such that the tests look more readable.
2020-09-15 23:10:54 -07:00
Pavel Kovalenko
f43428a4d5 S3 HTTPS integration test. 2020-07-10 22:42:18 +03:00
Anton Ivashkin
78ba9c986f throw exception on redirect limit in S3 request 2020-07-07 16:20:48 +03:00
Pervakov Grigorii
bc9381406f Add default credentials and custom headers for s3 table functions. 2020-06-02 20:48:49 +03:00
alexey-milovidov
7201018079
Update test.py 2020-05-26 00:05:15 +03:00
Vladimir Chebotarev
e013edce4b Added test for multi-page S3 globbing. 2020-05-25 15:57:27 +03:00
Pavel Kovalenko
89b9fb6d41 Create 'root' S3 bucket before CH is started. 2020-05-22 14:48:08 +03:00
Ivan
97f2a2213e
Move all folders inside /dbms one level up (#9974)
* Move some code outside dbms/src folder
* Fix paths
2020-04-02 02:51:21 +03:00