thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-11-14 19:45:11 +00:00
Code Issues Packages Projects Releases Wiki Activity
151,486 Commits 9,131 Branches 1,774 Tags 2.1 GiB
129d1e387e
Commit Graph

247 Commits

Author SHA1 Message Date
Azat Khuzhin
b92d8b302a tests: fix test_storage_s3::test_wrong_format_usage flakiness 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-08-19 21:51:23 +02:00
avogar
67f848a428 Fix style 2022-08-18 17:15:30 +00:00
avogar
1fb99f51b0 Make test for s3 schema inference cache better 2022-08-18 16:50:59 +00:00
avogar
4c64b7a69e Try fix s3 test 2022-08-17 12:25:58 +00:00
avogar
8dd54c043d Merge branch 'master' of github.com:ClickHouse/ClickHouse into schema-inference-cache 2022-08-17 11:47:40 +00:00
Kruglov Pavel
afd5221d58
FIx check black 2022-08-16 18:53:55 +02:00
avogar
d5f7337ba3 Try fix tests 2022-08-16 14:57:19 +00:00
Kruglov Pavel
169b38c878
Fix s3 test 2022-08-16 12:40:10 +02:00
avogar
c4ff3ffeea Rename settings 2022-08-15 12:45:18 +00:00
avogar
c0bb6303b9 Fix check black 2022-08-11 10:55:18 +00:00
Kruglov Pavel
088e8cf9bd
Merge branch 'master' into numbers-schema-inference 2022-08-09 14:00:36 +02:00
avogar
9b1a267203 Refactor, remove TTL, add size limit, add system table and system query 2022-08-05 16:20:15 +00:00
avogar
02c90fd94b Fix tests 2022-07-21 16:54:42 +00:00
Kruglov Pavel
b38241b08a
Merge branch 'master' into schema-inference-cache 2022-07-14 12:29:54 +02:00
Yatsishin Ilya
d274b05fac improvements 2022-07-07 20:21:10 +00:00
Kseniia Sumarokova
b8ab20bcb3
Update test.py 2022-07-04 20:56:34 +02:00
kssenii
f5bf744028 Fix 2022-07-04 16:05:08 +02:00
avogar
106f92dcdb Fix tests 2022-06-28 16:13:42 +00:00
avogar
b0c9d1a25d Fix style 2022-06-27 14:04:28 +00:00
avogar
5155262a16 Add some additional information to cache keys 2022-06-27 12:43:24 +00:00
Kruglov Pavel
86e8f31ad4
Merge branch 'master' into schema-inference-cache 2022-06-24 16:10:25 +02:00
avogar
77a0c47591 Fix tests 2022-06-21 17:22:53 +00:00
avogar
c14364e3d9 Check last modification time for URL function too 2022-06-21 17:18:14 +00:00
avogar
d37ad2e6de Implement cache for schema inference for file/s3/hdfs/url 2022-06-21 13:02:48 +00:00
Anton Popov
eea7e4eced fix test 2022-06-15 15:02:02 +00:00
Kruglov Pavel
3916436a10
Fix style 2022-04-20 16:35:24 +02:00
Kruglov Pavel
813e228fcc
Merge branch 'master' into improve-globs 2022-04-20 16:31:47 +02:00
Kruglov Pavel
0bae44abf0
Merge pull request #36209 from Avogar/check-magic-bytes 
Check ORC/Parquet/Arrow format magic bytes before loading file in memory
 2022-04-14 11:32:38 +02:00
avogar
ca14a29dbe Revert reverting "Fix crash in ParallelReadBuffer" 2022-04-13 20:39:12 +00:00
tavplubix
b2cd23658a
Revert "Fix crash in ParallelReadBuffer" 2022-04-13 22:45:34 +03:00
avogar
42726639f3 Check ORC/Parquet/Arrow format magic bytes before loading file in memory 2022-04-13 19:27:38 +00:00
avogar
1c065f8c7a Some refactoring around schema inference with globs 2022-04-13 17:02:48 +00:00
avogar
de9c9462e5 Fix crash in ParallelReadBuffer 2022-04-12 13:54:53 +00:00
Azat Khuzhin
c5ffbc688e Do not allow SETTINGS after FORMAT for INSERT queries 
Parsing SETTINGS after FORMAT, that has been introduced in [1], can
interpret SETTING as some values, which is misleading.

  [1]: https://github.com/ClickHouse/ClickHouse/pull/4174/files#diff-ba7bd0657630b1cd94cf6ed364bd857338096f49f66dc82918438d6745753775R106

Note, that we are touching only INSERT queries, not SELECT, since this
is a backward incompatible change, and in case of modifying SELECT it
can break too much.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
Fixes: #35100
Fixes: #20343
 2022-04-07 16:29:36 +03:00
Antonio Andelic
362e877529 Merge branch 'master' into parallel-s3-downloading 2022-03-29 07:05:26 +00:00
Nikita Mikhaylov
85282f1b4d
Made schema inference work with s3Cluster table function (#35544) 2022-03-29 00:46:35 +02:00
Antonio Andelic
62a495a5fc Small refactor 2022-03-24 09:45:30 +00:00
Antonio Andelic
e0d7b6dc3e Add tests for S3 multithreaded download 2022-03-24 09:30:06 +00:00
Mikhail f. Shiryaev
e6f5a3f98b
Apply black formatter to all *.py files in the repo 2022-03-22 17:39:58 +01:00
Kruglov Pavel
a506120646
Fix bug in schema inference in s3 table function (#35176) 2022-03-10 15:16:07 +01:00
kssenii
b3a7f07954 Fix flacky test 2022-03-02 13:01:15 +01:00
kssenii
7597c692d1 Fix test 2022-03-01 12:07:13 +01:00
kssenii
092ec45b47 Merge master 2022-03-01 12:06:56 +01:00
kssenii
af6a5c78a6 Fix tests 2022-03-01 10:38:01 +01:00
Kruglov Pavel
ee37e93499
Update test.py 2022-02-28 20:39:15 +03:00
avogar
916ce3f96d Fix tests 2022-02-28 10:07:29 +00:00
kssenii
7d7277a7c6 Add test 2022-02-25 09:32:39 +01:00
Kruglov Pavel
a0ae68db99
Fix tests 2022-02-19 17:41:56 +03:00
avogar
653d769d34 Support schema inference for INSERT INTO FUNCTION 2022-02-18 16:19:42 +00:00
Kruglov Pavel
dd863ca2a0
Merge branch 'master' into fix-url-globs 2022-02-16 12:45:31 +03:00
avogar
a8f83a387c Add one more signature for s3 table function 2022-02-10 18:57:02 +03:00
avogar
84087c1554 Improve schema inference with globs 2022-02-09 19:16:15 +03:00
avogar
e118c89bb2 Fix segfault in schema inference from url 2022-02-08 13:40:03 +03:00
avogar
058ef356ac Fix schema inference for table runction s3 2022-01-31 19:39:20 +03:00
avogar
4efadfad3c Fix tests 2022-01-18 22:26:13 +03:00
Kruglov Pavel
a7df9cd53a
Merge branch 'master' into formats-with-suffixes 2022-01-14 21:03:49 +03:00
avogar
8884c4a3fe Fix tests and build 2022-01-14 16:27:57 +03:00
avogar
2f000a178a Add more tests 2022-01-14 14:00:50 +03:00
avogar
97788b9c21 Allow to create new files on insert for File/S3/HDFS engines 2021-12-29 21:19:13 +03:00
avogar
e3dbfe6bf6 Better test url engine 2021-12-29 12:18:56 +03:00
avogar
bc5f428f97 Fix tests 2021-12-29 12:18:56 +03:00
avogar
74f09d6476 Fix tests 2021-12-29 12:18:56 +03:00
avogar
aaf9f85c67 Add more tests and fixes 2021-12-29 12:18:56 +03:00
avogar
58966e90e9 Don't allow to write into S3 if path contains globs 2021-12-24 17:13:35 +03:00
kssenii
11731cde92 Fix test 2021-12-22 13:57:08 +03:00
kssenii
d8ed4db6d4 Fix 2021-12-22 11:42:23 +03:00
Azat Khuzhin
ba2c9405da Fix test_storage_s3 flakiness 2021-12-16 07:56:43 +03:00
kssenii
f29ef7d18b Add one more test 2021-11-20 15:01:45 +03:00
kssenii
f18dcd2287 Merge branch 'master' of github.com:ClickHouse/ClickHouse into seekable-read-buffers 2021-11-13 14:38:57 +03:00
Alexander Tokmakov
4f542ab019 add test 2021-11-09 23:11:02 +03:00
kssenii
ec11179f91 Merge branch 'master' of github.com:ClickHouse/ClickHouse into seekable-read-buffers 2021-11-03 14:33:31 +03:00
kssenii
45ea820297 Reduce memory usage for some formats 2021-11-03 14:30:03 +03:00
Kseniia Sumarokova
065106c299
Merge pull request #30690 from kssenii/partitioned-write 
Partitioned write for file, hdfs, url storages and table functions
 2021-11-02 22:58:16 +03:00
Maksim Kita
0d7dfdccf9 Fixed tests 2021-10-30 01:29:36 +03:00
kssenii
20772fde04 For storage 2021-10-27 10:04:17 +03:00
Alexey Milovidov
cba6e39fe5 Replace yandex to clickhouse in configs 2021-09-25 07:08:34 +03:00
alexey-milovidov
b8083929a6
Merge branch 'master' into yandex-to-clickhouse-in-configs 2021-09-23 20:49:40 +03:00
Alexey Milovidov
e513a5db32 Change <yandex> to <clickhouse> in configs 2021-09-20 01:38:53 +03:00
kssenii
41bc68e82e Tests, url table function 2021-09-08 22:28:22 +03:00
alexey-milovidov
35d5957683
Merge pull request #28336 from ianton-ru/MDB-13951 
Fix test_storage_s3/test_put_get_with_globs (cleanup after test)
 2021-08-31 01:54:59 +03:00
Anton Ivashkin
2e33072469 Fix test_storage_s3/test_put_get_with_globs (cleanup after test) 2021-08-30 13:31:46 +03:00
vdimir
6ba6577978 Do not allow slashes in bucket formatted from PARTITION BY 2021-08-23 09:22:15 +03:00
vdimir
c4c42b61c4 Validate uft8 in partition key from PARTITION BY for s3 2021-08-23 09:22:15 +03:00
Vladimir Chebotarev
2223ae4ddc Added integration test. 2021-08-23 09:22:14 +03:00
Vladimir Chebotarev
f45dbd6cbc Fixed test. 2021-08-23 09:22:14 +03:00
Vladimir Chebotarev
b4484f4f21 Test fix. 2021-08-23 09:22:13 +03:00
Vladimir Chebotarev
e99433e094 Better functionality. 2021-08-23 09:22:13 +03:00
Vladimir Chebotarev
328213f5d9 Fixes. 2021-08-23 09:22:13 +03:00
Vladimir Chebotarev
0879dbbfa7 Test. 2021-08-23 09:22:13 +03:00
Yatsishin Ilya
be9a0a9ca2 better test_storage_s3, test_storage_s3_get_unstable still not OK 2021-07-15 15:49:25 +03:00
Kseniia Sumarokova
ed687f45a1
Merge branch 'master' into truncate 2021-06-30 20:08:42 +03:00
Kseniia Sumarokova
f47dd116c4
Update test.py 2021-06-21 19:07:17 +03:00
kssenii
ac0f86cdbf Truncate for s3 2021-06-21 15:44:36 +00:00
alesapin
c7781e24d5 Fix some flaky tests 2021-06-21 11:02:27 +03:00
Vladimir Chebotarev
f16dd06476 Review fixes. 2021-06-10 08:49:26 +03:00
Vladimir Chebotarev
591c556321 Improved test_storage_s3_get_unstable. 2021-06-10 08:49:26 +03:00
Alexey Milovidov
b146524fdb Maybe Minio starts for too long in tests 2021-06-06 12:38:49 +03:00
mergify[bot]
9e2c454db2
Merge branch 'master' into merging-s3-backoff 2021-06-03 18:58:24 +00:00
Yatsishin Ilya
ddddb761db more 2021-06-02 18:08:16 +03:00
Yatsishin Ilya
ff7f7f6200 try 2021-06-01 17:18:35 +03:00
Vladimir Chebotarev
752d0cb0b6 Fixed test. 2021-05-17 10:38:34 +03:00
Yatsishin Ilya
ad706b0de4 Merge remote-tracking branch 'origin' into integration-2 2021-05-14 16:48:44 +03:00
Vladimir Chebotarev
1141872db2 Fix tests II. 2021-05-13 08:46:16 +03:00
Yatsishin Ilya
fe0194aa4b s3 catch up 2021-05-12 10:03:53 +03:00
Yatsishin Ilya
9d159072ef mERGe remote-tracking branch 'origin' into integration-2 2021-05-11 13:32:44 +03:00
Vladimir Chebotarev
524113f497 Fixed bug with S3 URLs containing + symbol, data with such keys could not be read previously. 2021-05-06 11:28:55 +03:00
Yatsishin Ilya
dd192bb0ab better 2021-04-29 17:26:41 +03:00
Yatsishin Ilya
cb101e46bb finally 2021-04-29 14:57:48 +03:00
Yatsishin Ilya
c59a11f798 Merge remote-tracking branch 'origin' into integration-2 2021-04-27 18:34:33 +03:00
Vladimir Chebotarev
edd3931d4e Fixed tests. 2021-04-15 09:41:49 +03:00
Vladimir Chebotarev
5b8a3831e6 Better tests. 2021-04-15 09:41:49 +03:00
Vladimir Chebotarev
a63db53576 Improved tests. 2021-04-15 09:41:49 +03:00
Vladimir Chebotarev
ce6cf1630b Added tests. 2021-04-15 09:41:49 +03:00
Yatsishin Ilya
10b7037d9c Merge remote-tracking branch 'origin' into integration-2 2021-04-14 14:21:40 +03:00
Yatsishin Ilya
e4d1ea846c more 2021-04-13 13:52:22 +03:00
Yatsishin Ilya
b4dded3fc2 wip 2021-04-12 10:03:12 +03:00
Yatsishin Ilya
5311278c29 Merge remote-tracking branch 'origin' into integration-2 2021-04-08 12:30:24 +03:00
Vladimir Chebotarev
213a561e21 Improved test. 2021-03-29 10:07:41 +03:00
Vladimir Chebotarev
d157540254 Added test for put with compression and wrong credentials. 2021-03-29 10:07:41 +03:00
Yatsishin Ilya
df8013c22d fix 2021-03-26 21:46:42 +03:00
Yatsishin Ilya
5ae47bd2b4 Merge remote-tracking branch 'origin/master' into integration-2 2021-03-19 15:33:14 +03:00
Yatsishin Ilya
579d865aac merge remote-tracking branch 'origin' into integration-2 2021-03-16 13:26:51 +03:00
Yatsishin Ilya
06bb4e2c71 wip 2021-03-16 13:00:49 +03:00
vdimir
1194d50e48
Try to fix test_storage_s3: crash in WriteBufferFromS3 2021-03-11 15:21:13 +03:00
Pervakov Grigorii
eb87afc4d4 recreate S3 client if credentials changed 2021-03-10 13:36:57 +03:00
Yatsishin Ilya
babea212c7 Merge remote-tracking branch 'origin' into integration-2 2021-03-03 11:56:15 +03:00
Yatsishin Ilya
fc89b4fd35 better 2021-02-20 17:59:39 +03:00
Vladimir Chebotarev
d729aacb09 Fixed table function S3 auto compression mode. 2021-01-29 07:54:52 +03:00
Vladimir Chebotarev
8ba8e8efd5 Added prefix-based S3 endpoint settings. 2021-01-25 06:40:57 +03:00
Pavel Kovalenko
db7a87089b Adaptive choose of single/multi part upload in WriteBufferFromS3. 2020-12-09 17:09:04 +03:00
alesapin
45f09fe293
Merge pull request #17220 from ianton-ru/s3_max_redirects 
S3 max redirects
 2020-11-24 12:13:10 +03:00
Anton Ivashkin
aa208a9d9f Use only 's3_max_redirect' in params instead of all settings 2020-11-23 14:02:17 +03:00
Pavel Kovalenko
5f98d38689 Fixed flaky test_storage_s3::test_custom_auth_headers 2020-11-23 13:19:43 +03:00
Anton Ivashkin
a7d4f4be65 Add 's3_max_redirects' test 2020-11-20 11:18:44 +03:00
Pavel Kovalenko
e311759c5d
Abort multipart upload if no data was written to WriteBufferFromS3. (#16840) 2020-11-11 15:15:16 +03:00
Azat Khuzhin
9cb3c743bd
Convert to python3 (#15007) 2020-10-02 19:54:07 +03:00
Vladimir Chebotarev
b460024356 Fixed bug in GZIP compression in S3 storage. 2020-09-30 16:09:55 +03:00
Vladimir Chebotarev
a38c7ff2a8 Somehow uncompressed PUT works and gzipped PUT doesn't, in S3 storage. 2020-09-30 15:04:42 +03:00
Vladimir Chebotarev
b7a6f15f19 Added test for GZIP in S3 storage. 2020-09-30 13:55:44 +03:00
bharatnc
b75963d370 Format and optimize imports in integration test files 
This PR formats all the `*.py` files found under the `tests/integration`
folder. It also reorders the imports and cleans up a bunch of unused
imports.

The formatting also takes care of other things like wrapping lines and
fixing spaces and indents such that the tests look more readable.
 2020-09-15 23:10:54 -07:00
Pavel Kovalenko
f43428a4d5 S3 HTTPS integration test. 2020-07-10 22:42:18 +03:00
Anton Ivashkin
78ba9c986f throw exception on redirect limit in S3 request 2020-07-07 16:20:48 +03:00
Pervakov Grigorii
bc9381406f Add default credentials and custom headers for s3 table functions. 2020-06-02 20:48:49 +03:00
alexey-milovidov
7201018079
Update test.py 2020-05-26 00:05:15 +03:00
Vladimir Chebotarev
e013edce4b Added test for multi-page S3 globbing. 2020-05-25 15:57:27 +03:00
Pavel Kovalenko
89b9fb6d41 Create 'root' S3 bucket before CH is started. 2020-05-22 14:48:08 +03:00
Ivan
97f2a2213e
Move all folders inside /dbms one level up (#9974) 
* Move some code outside dbms/src folder
* Fix paths
 2020-04-02 02:51:21 +03:00