Commit Graph

199 Commits

Author SHA1 Message Date
kssenii
420ac4eb43 s3 header auth in ast 2022-09-13 15:13:28 +02:00
avogar
93d0113b14 Fix flacky schema inference cache tests 2022-08-24 11:28:56 +00:00
Azat Khuzhin
b92d8b302a tests: fix test_storage_s3::test_wrong_format_usage flakiness
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-08-19 21:51:23 +02:00
avogar
67f848a428 Fix style 2022-08-18 17:15:30 +00:00
avogar
1fb99f51b0 Make test for s3 schema inference cache better 2022-08-18 16:50:59 +00:00
avogar
4c64b7a69e Try fix s3 test 2022-08-17 12:25:58 +00:00
avogar
8dd54c043d Merge branch 'master' of github.com:ClickHouse/ClickHouse into schema-inference-cache 2022-08-17 11:47:40 +00:00
Kruglov Pavel
afd5221d58
FIx check black 2022-08-16 18:53:55 +02:00
avogar
d5f7337ba3 Try fix tests 2022-08-16 14:57:19 +00:00
Kruglov Pavel
169b38c878
Fix s3 test 2022-08-16 12:40:10 +02:00
avogar
c4ff3ffeea Rename settings 2022-08-15 12:45:18 +00:00
avogar
c0bb6303b9 Fix check black 2022-08-11 10:55:18 +00:00
Kruglov Pavel
088e8cf9bd
Merge branch 'master' into numbers-schema-inference 2022-08-09 14:00:36 +02:00
avogar
9b1a267203 Refactor, remove TTL, add size limit, add system table and system query 2022-08-05 16:20:15 +00:00
avogar
02c90fd94b Fix tests 2022-07-21 16:54:42 +00:00
Kruglov Pavel
b38241b08a
Merge branch 'master' into schema-inference-cache 2022-07-14 12:29:54 +02:00
Yatsishin Ilya
d274b05fac improvements 2022-07-07 20:21:10 +00:00
Kseniia Sumarokova
b8ab20bcb3
Update test.py 2022-07-04 20:56:34 +02:00
kssenii
f5bf744028 Fix 2022-07-04 16:05:08 +02:00
avogar
106f92dcdb Fix tests 2022-06-28 16:13:42 +00:00
avogar
b0c9d1a25d Fix style 2022-06-27 14:04:28 +00:00
avogar
5155262a16 Add some additional information to cache keys 2022-06-27 12:43:24 +00:00
Kruglov Pavel
86e8f31ad4
Merge branch 'master' into schema-inference-cache 2022-06-24 16:10:25 +02:00
avogar
77a0c47591 Fix tests 2022-06-21 17:22:53 +00:00
avogar
c14364e3d9 Check last modification time for URL function too 2022-06-21 17:18:14 +00:00
avogar
d37ad2e6de Implement cache for schema inference for file/s3/hdfs/url 2022-06-21 13:02:48 +00:00
Anton Popov
eea7e4eced fix test 2022-06-15 15:02:02 +00:00
Kruglov Pavel
3916436a10
Fix style 2022-04-20 16:35:24 +02:00
Kruglov Pavel
813e228fcc
Merge branch 'master' into improve-globs 2022-04-20 16:31:47 +02:00
Kruglov Pavel
0bae44abf0
Merge pull request #36209 from Avogar/check-magic-bytes
Check ORC/Parquet/Arrow format magic bytes before loading file in memory
2022-04-14 11:32:38 +02:00
avogar
ca14a29dbe Revert reverting "Fix crash in ParallelReadBuffer" 2022-04-13 20:39:12 +00:00
tavplubix
b2cd23658a
Revert "Fix crash in ParallelReadBuffer" 2022-04-13 22:45:34 +03:00
avogar
42726639f3 Check ORC/Parquet/Arrow format magic bytes before loading file in memory 2022-04-13 19:27:38 +00:00
avogar
1c065f8c7a Some refactoring around schema inference with globs 2022-04-13 17:02:48 +00:00
avogar
de9c9462e5 Fix crash in ParallelReadBuffer 2022-04-12 13:54:53 +00:00
Azat Khuzhin
c5ffbc688e Do not allow SETTINGS after FORMAT for INSERT queries
Parsing SETTINGS after FORMAT, that has been introduced in [1], can
interpret SETTING as some values, which is misleading.

  [1]: https://github.com/ClickHouse/ClickHouse/pull/4174/files#diff-ba7bd0657630b1cd94cf6ed364bd857338096f49f66dc82918438d6745753775R106

Note, that we are touching only INSERT queries, not SELECT, since this
is a backward incompatible change, and in case of modifying SELECT it
can break too much.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
Fixes: #35100
Fixes: #20343
2022-04-07 16:29:36 +03:00
Antonio Andelic
362e877529 Merge branch 'master' into parallel-s3-downloading 2022-03-29 07:05:26 +00:00
Nikita Mikhaylov
85282f1b4d
Made schema inference work with s3Cluster table function (#35544) 2022-03-29 00:46:35 +02:00
Antonio Andelic
62a495a5fc Small refactor 2022-03-24 09:45:30 +00:00
Antonio Andelic
e0d7b6dc3e Add tests for S3 multithreaded download 2022-03-24 09:30:06 +00:00
Mikhail f. Shiryaev
e6f5a3f98b
Apply black formatter to all *.py files in the repo 2022-03-22 17:39:58 +01:00
Kruglov Pavel
a506120646
Fix bug in schema inference in s3 table function (#35176) 2022-03-10 15:16:07 +01:00
kssenii
b3a7f07954 Fix flacky test 2022-03-02 13:01:15 +01:00
kssenii
7597c692d1 Fix test 2022-03-01 12:07:13 +01:00
kssenii
092ec45b47 Merge master 2022-03-01 12:06:56 +01:00
kssenii
af6a5c78a6 Fix tests 2022-03-01 10:38:01 +01:00
Kruglov Pavel
ee37e93499
Update test.py 2022-02-28 20:39:15 +03:00
avogar
916ce3f96d Fix tests 2022-02-28 10:07:29 +00:00
kssenii
7d7277a7c6 Add test 2022-02-25 09:32:39 +01:00
Kruglov Pavel
a0ae68db99
Fix tests 2022-02-19 17:41:56 +03:00
avogar
653d769d34 Support schema inference for INSERT INTO FUNCTION 2022-02-18 16:19:42 +00:00
Kruglov Pavel
dd863ca2a0
Merge branch 'master' into fix-url-globs 2022-02-16 12:45:31 +03:00
avogar
a8f83a387c Add one more signature for s3 table function 2022-02-10 18:57:02 +03:00
avogar
84087c1554 Improve schema inference with globs 2022-02-09 19:16:15 +03:00
avogar
e118c89bb2 Fix segfault in schema inference from url 2022-02-08 13:40:03 +03:00
avogar
058ef356ac Fix schema inference for table runction s3 2022-01-31 19:39:20 +03:00
avogar
4efadfad3c Fix tests 2022-01-18 22:26:13 +03:00
Kruglov Pavel
a7df9cd53a
Merge branch 'master' into formats-with-suffixes 2022-01-14 21:03:49 +03:00
avogar
8884c4a3fe Fix tests and build 2022-01-14 16:27:57 +03:00
avogar
2f000a178a Add more tests 2022-01-14 14:00:50 +03:00
avogar
97788b9c21 Allow to create new files on insert for File/S3/HDFS engines 2021-12-29 21:19:13 +03:00
avogar
e3dbfe6bf6 Better test url engine 2021-12-29 12:18:56 +03:00
avogar
bc5f428f97 Fix tests 2021-12-29 12:18:56 +03:00
avogar
74f09d6476 Fix tests 2021-12-29 12:18:56 +03:00
avogar
aaf9f85c67 Add more tests and fixes 2021-12-29 12:18:56 +03:00
avogar
58966e90e9 Don't allow to write into S3 if path contains globs 2021-12-24 17:13:35 +03:00
kssenii
11731cde92 Fix test 2021-12-22 13:57:08 +03:00
kssenii
d8ed4db6d4 Fix 2021-12-22 11:42:23 +03:00
Azat Khuzhin
ba2c9405da Fix test_storage_s3 flakiness 2021-12-16 07:56:43 +03:00
kssenii
f29ef7d18b Add one more test 2021-11-20 15:01:45 +03:00
kssenii
f18dcd2287 Merge branch 'master' of github.com:ClickHouse/ClickHouse into seekable-read-buffers 2021-11-13 14:38:57 +03:00
Alexander Tokmakov
4f542ab019 add test 2021-11-09 23:11:02 +03:00
kssenii
ec11179f91 Merge branch 'master' of github.com:ClickHouse/ClickHouse into seekable-read-buffers 2021-11-03 14:33:31 +03:00
kssenii
45ea820297 Reduce memory usage for some formats 2021-11-03 14:30:03 +03:00
Kseniia Sumarokova
065106c299
Merge pull request #30690 from kssenii/partitioned-write
Partitioned write for file, hdfs, url storages and table functions
2021-11-02 22:58:16 +03:00
Maksim Kita
0d7dfdccf9 Fixed tests 2021-10-30 01:29:36 +03:00
kssenii
20772fde04 For storage 2021-10-27 10:04:17 +03:00
Alexey Milovidov
cba6e39fe5 Replace yandex to clickhouse in configs 2021-09-25 07:08:34 +03:00
alexey-milovidov
b8083929a6
Merge branch 'master' into yandex-to-clickhouse-in-configs 2021-09-23 20:49:40 +03:00
Alexey Milovidov
e513a5db32 Change <yandex> to <clickhouse> in configs 2021-09-20 01:38:53 +03:00
kssenii
41bc68e82e Tests, url table function 2021-09-08 22:28:22 +03:00
alexey-milovidov
35d5957683
Merge pull request #28336 from ianton-ru/MDB-13951
Fix test_storage_s3/test_put_get_with_globs (cleanup after test)
2021-08-31 01:54:59 +03:00
Anton Ivashkin
2e33072469 Fix test_storage_s3/test_put_get_with_globs (cleanup after test) 2021-08-30 13:31:46 +03:00
vdimir
6ba6577978 Do not allow slashes in bucket formatted from PARTITION BY 2021-08-23 09:22:15 +03:00
vdimir
c4c42b61c4 Validate uft8 in partition key from PARTITION BY for s3 2021-08-23 09:22:15 +03:00
Vladimir Chebotarev
2223ae4ddc Added integration test. 2021-08-23 09:22:14 +03:00
Vladimir Chebotarev
f45dbd6cbc Fixed test. 2021-08-23 09:22:14 +03:00
Vladimir Chebotarev
b4484f4f21 Test fix. 2021-08-23 09:22:13 +03:00
Vladimir Chebotarev
e99433e094 Better functionality. 2021-08-23 09:22:13 +03:00
Vladimir Chebotarev
328213f5d9 Fixes. 2021-08-23 09:22:13 +03:00
Vladimir Chebotarev
0879dbbfa7 Test. 2021-08-23 09:22:13 +03:00
Yatsishin Ilya
be9a0a9ca2 better test_storage_s3, test_storage_s3_get_unstable still not OK 2021-07-15 15:49:25 +03:00
Kseniia Sumarokova
ed687f45a1
Merge branch 'master' into truncate 2021-06-30 20:08:42 +03:00
Kseniia Sumarokova
f47dd116c4
Update test.py 2021-06-21 19:07:17 +03:00
kssenii
ac0f86cdbf Truncate for s3 2021-06-21 15:44:36 +00:00
alesapin
c7781e24d5 Fix some flaky tests 2021-06-21 11:02:27 +03:00
Vladimir Chebotarev
f16dd06476 Review fixes. 2021-06-10 08:49:26 +03:00
Vladimir Chebotarev
591c556321 Improved test_storage_s3_get_unstable. 2021-06-10 08:49:26 +03:00
Alexey Milovidov
b146524fdb Maybe Minio starts for too long in tests 2021-06-06 12:38:49 +03:00
mergify[bot]
9e2c454db2
Merge branch 'master' into merging-s3-backoff 2021-06-03 18:58:24 +00:00