thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-12 01:12:12 +00:00
Code Issues Packages Projects Releases Wiki Activity
143,958 Commits 9,297 Branches 1,782 Tags 2.1 GiB
dcbcfc9ee8
Commit Graph

228 Commits

Author SHA1 Message Date
avogar
5155262a16 Add some additional information to cache keys 2022-06-27 12:43:24 +00:00
Kruglov Pavel
86e8f31ad4
Merge branch 'master' into schema-inference-cache 2022-06-24 16:10:25 +02:00
avogar
77a0c47591 Fix tests 2022-06-21 17:22:53 +00:00
avogar
c14364e3d9 Check last modification time for URL function too 2022-06-21 17:18:14 +00:00
avogar
d37ad2e6de Implement cache for schema inference for file/s3/hdfs/url 2022-06-21 13:02:48 +00:00
Anton Popov
eea7e4eced fix test 2022-06-15 15:02:02 +00:00
Kruglov Pavel
3916436a10
Fix style 2022-04-20 16:35:24 +02:00
Kruglov Pavel
813e228fcc
Merge branch 'master' into improve-globs 2022-04-20 16:31:47 +02:00
Kruglov Pavel
0bae44abf0
Merge pull request #36209 from Avogar/check-magic-bytes 
Check ORC/Parquet/Arrow format magic bytes before loading file in memory
 2022-04-14 11:32:38 +02:00
avogar
ca14a29dbe Revert reverting "Fix crash in ParallelReadBuffer" 2022-04-13 20:39:12 +00:00
tavplubix
b2cd23658a
Revert "Fix crash in ParallelReadBuffer" 2022-04-13 22:45:34 +03:00
avogar
42726639f3 Check ORC/Parquet/Arrow format magic bytes before loading file in memory 2022-04-13 19:27:38 +00:00
avogar
1c065f8c7a Some refactoring around schema inference with globs 2022-04-13 17:02:48 +00:00
avogar
de9c9462e5 Fix crash in ParallelReadBuffer 2022-04-12 13:54:53 +00:00
Azat Khuzhin
c5ffbc688e Do not allow SETTINGS after FORMAT for INSERT queries 
Parsing SETTINGS after FORMAT, that has been introduced in [1], can
interpret SETTING as some values, which is misleading.

  [1]: https://github.com/ClickHouse/ClickHouse/pull/4174/files#diff-ba7bd0657630b1cd94cf6ed364bd857338096f49f66dc82918438d6745753775R106

Note, that we are touching only INSERT queries, not SELECT, since this
is a backward incompatible change, and in case of modifying SELECT it
can break too much.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
Fixes: #35100
Fixes: #20343
 2022-04-07 16:29:36 +03:00
Antonio Andelic
362e877529 Merge branch 'master' into parallel-s3-downloading 2022-03-29 07:05:26 +00:00
Nikita Mikhaylov
85282f1b4d
Made schema inference work with s3Cluster table function (#35544) 2022-03-29 00:46:35 +02:00
Antonio Andelic
62a495a5fc Small refactor 2022-03-24 09:45:30 +00:00
Antonio Andelic
e0d7b6dc3e Add tests for S3 multithreaded download 2022-03-24 09:30:06 +00:00
Mikhail f. Shiryaev
e6f5a3f98b
Apply black formatter to all *.py files in the repo 2022-03-22 17:39:58 +01:00
Kruglov Pavel
a506120646
Fix bug in schema inference in s3 table function (#35176) 2022-03-10 15:16:07 +01:00
kssenii
b3a7f07954 Fix flacky test 2022-03-02 13:01:15 +01:00
kssenii
7597c692d1 Fix test 2022-03-01 12:07:13 +01:00
kssenii
092ec45b47 Merge master 2022-03-01 12:06:56 +01:00
kssenii
af6a5c78a6 Fix tests 2022-03-01 10:38:01 +01:00
Kruglov Pavel
ee37e93499
Update test.py 2022-02-28 20:39:15 +03:00
avogar
916ce3f96d Fix tests 2022-02-28 10:07:29 +00:00
kssenii
7d7277a7c6 Add test 2022-02-25 09:32:39 +01:00
Kruglov Pavel
a0ae68db99
Fix tests 2022-02-19 17:41:56 +03:00
avogar
653d769d34 Support schema inference for INSERT INTO FUNCTION 2022-02-18 16:19:42 +00:00
Kruglov Pavel
dd863ca2a0
Merge branch 'master' into fix-url-globs 2022-02-16 12:45:31 +03:00
avogar
a8f83a387c Add one more signature for s3 table function 2022-02-10 18:57:02 +03:00
avogar
84087c1554 Improve schema inference with globs 2022-02-09 19:16:15 +03:00
avogar
e118c89bb2 Fix segfault in schema inference from url 2022-02-08 13:40:03 +03:00
avogar
058ef356ac Fix schema inference for table runction s3 2022-01-31 19:39:20 +03:00
avogar
4efadfad3c Fix tests 2022-01-18 22:26:13 +03:00
Kruglov Pavel
a7df9cd53a
Merge branch 'master' into formats-with-suffixes 2022-01-14 21:03:49 +03:00
avogar
8884c4a3fe Fix tests and build 2022-01-14 16:27:57 +03:00
avogar
2f000a178a Add more tests 2022-01-14 14:00:50 +03:00
avogar
97788b9c21 Allow to create new files on insert for File/S3/HDFS engines 2021-12-29 21:19:13 +03:00
avogar
e3dbfe6bf6 Better test url engine 2021-12-29 12:18:56 +03:00
avogar
bc5f428f97 Fix tests 2021-12-29 12:18:56 +03:00
avogar
74f09d6476 Fix tests 2021-12-29 12:18:56 +03:00
avogar
aaf9f85c67 Add more tests and fixes 2021-12-29 12:18:56 +03:00
avogar
58966e90e9 Don't allow to write into S3 if path contains globs 2021-12-24 17:13:35 +03:00
kssenii
11731cde92 Fix test 2021-12-22 13:57:08 +03:00
kssenii
d8ed4db6d4 Fix 2021-12-22 11:42:23 +03:00
Azat Khuzhin
ba2c9405da Fix test_storage_s3 flakiness 2021-12-16 07:56:43 +03:00
kssenii
f29ef7d18b Add one more test 2021-11-20 15:01:45 +03:00
kssenii
f18dcd2287 Merge branch 'master' of github.com:ClickHouse/ClickHouse into seekable-read-buffers 2021-11-13 14:38:57 +03:00
Alexander Tokmakov
4f542ab019 add test 2021-11-09 23:11:02 +03:00
kssenii
ec11179f91 Merge branch 'master' of github.com:ClickHouse/ClickHouse into seekable-read-buffers 2021-11-03 14:33:31 +03:00
kssenii
45ea820297 Reduce memory usage for some formats 2021-11-03 14:30:03 +03:00
Kseniia Sumarokova
065106c299
Merge pull request #30690 from kssenii/partitioned-write 
Partitioned write for file, hdfs, url storages and table functions
 2021-11-02 22:58:16 +03:00
Maksim Kita
0d7dfdccf9 Fixed tests 2021-10-30 01:29:36 +03:00
kssenii
20772fde04 For storage 2021-10-27 10:04:17 +03:00
Alexey Milovidov
cba6e39fe5 Replace yandex to clickhouse in configs 2021-09-25 07:08:34 +03:00
alexey-milovidov
b8083929a6
Merge branch 'master' into yandex-to-clickhouse-in-configs 2021-09-23 20:49:40 +03:00
Alexey Milovidov
e513a5db32 Change <yandex> to <clickhouse> in configs 2021-09-20 01:38:53 +03:00
kssenii
41bc68e82e Tests, url table function 2021-09-08 22:28:22 +03:00
alexey-milovidov
35d5957683
Merge pull request #28336 from ianton-ru/MDB-13951 
Fix test_storage_s3/test_put_get_with_globs (cleanup after test)
 2021-08-31 01:54:59 +03:00
Anton Ivashkin
2e33072469 Fix test_storage_s3/test_put_get_with_globs (cleanup after test) 2021-08-30 13:31:46 +03:00
vdimir
6ba6577978 Do not allow slashes in bucket formatted from PARTITION BY 2021-08-23 09:22:15 +03:00
vdimir
c4c42b61c4 Validate uft8 in partition key from PARTITION BY for s3 2021-08-23 09:22:15 +03:00
Vladimir Chebotarev
2223ae4ddc Added integration test. 2021-08-23 09:22:14 +03:00
Vladimir Chebotarev
f45dbd6cbc Fixed test. 2021-08-23 09:22:14 +03:00
Vladimir Chebotarev
b4484f4f21 Test fix. 2021-08-23 09:22:13 +03:00
Vladimir Chebotarev
e99433e094 Better functionality. 2021-08-23 09:22:13 +03:00
Vladimir Chebotarev
328213f5d9 Fixes. 2021-08-23 09:22:13 +03:00
Vladimir Chebotarev
0879dbbfa7 Test. 2021-08-23 09:22:13 +03:00
Yatsishin Ilya
be9a0a9ca2 better test_storage_s3, test_storage_s3_get_unstable still not OK 2021-07-15 15:49:25 +03:00
Kseniia Sumarokova
ed687f45a1
Merge branch 'master' into truncate 2021-06-30 20:08:42 +03:00
Kseniia Sumarokova
f47dd116c4
Update test.py 2021-06-21 19:07:17 +03:00
kssenii
ac0f86cdbf Truncate for s3 2021-06-21 15:44:36 +00:00
alesapin
c7781e24d5 Fix some flaky tests 2021-06-21 11:02:27 +03:00
Vladimir Chebotarev
f16dd06476 Review fixes. 2021-06-10 08:49:26 +03:00
Vladimir Chebotarev
591c556321 Improved test_storage_s3_get_unstable. 2021-06-10 08:49:26 +03:00
Alexey Milovidov
b146524fdb Maybe Minio starts for too long in tests 2021-06-06 12:38:49 +03:00
mergify[bot]
9e2c454db2
Merge branch 'master' into merging-s3-backoff 2021-06-03 18:58:24 +00:00
Yatsishin Ilya
ddddb761db more 2021-06-02 18:08:16 +03:00
Yatsishin Ilya
ff7f7f6200 try 2021-06-01 17:18:35 +03:00
Vladimir Chebotarev
752d0cb0b6 Fixed test. 2021-05-17 10:38:34 +03:00
Yatsishin Ilya
ad706b0de4 Merge remote-tracking branch 'origin' into integration-2 2021-05-14 16:48:44 +03:00
Vladimir Chebotarev
1141872db2 Fix tests II. 2021-05-13 08:46:16 +03:00
Yatsishin Ilya
fe0194aa4b s3 catch up 2021-05-12 10:03:53 +03:00
Yatsishin Ilya
9d159072ef mERGe remote-tracking branch 'origin' into integration-2 2021-05-11 13:32:44 +03:00
Vladimir Chebotarev
524113f497 Fixed bug with S3 URLs containing + symbol, data with such keys could not be read previously. 2021-05-06 11:28:55 +03:00
Yatsishin Ilya
dd192bb0ab better 2021-04-29 17:26:41 +03:00
Yatsishin Ilya
cb101e46bb finally 2021-04-29 14:57:48 +03:00
Yatsishin Ilya
c59a11f798 Merge remote-tracking branch 'origin' into integration-2 2021-04-27 18:34:33 +03:00
Vladimir Chebotarev
edd3931d4e Fixed tests. 2021-04-15 09:41:49 +03:00
Vladimir Chebotarev
5b8a3831e6 Better tests. 2021-04-15 09:41:49 +03:00
Vladimir Chebotarev
a63db53576 Improved tests. 2021-04-15 09:41:49 +03:00
Vladimir Chebotarev
ce6cf1630b Added tests. 2021-04-15 09:41:49 +03:00
Yatsishin Ilya
10b7037d9c Merge remote-tracking branch 'origin' into integration-2 2021-04-14 14:21:40 +03:00
Yatsishin Ilya
e4d1ea846c more 2021-04-13 13:52:22 +03:00
Yatsishin Ilya
b4dded3fc2 wip 2021-04-12 10:03:12 +03:00
Yatsishin Ilya
5311278c29 Merge remote-tracking branch 'origin' into integration-2 2021-04-08 12:30:24 +03:00
Vladimir Chebotarev
213a561e21 Improved test. 2021-03-29 10:07:41 +03:00
Vladimir Chebotarev
d157540254 Added test for put with compression and wrong credentials. 2021-03-29 10:07:41 +03:00
Yatsishin Ilya
df8013c22d fix 2021-03-26 21:46:42 +03:00
Yatsishin Ilya
5ae47bd2b4 Merge remote-tracking branch 'origin/master' into integration-2 2021-03-19 15:33:14 +03:00
Yatsishin Ilya
579d865aac merge remote-tracking branch 'origin' into integration-2 2021-03-16 13:26:51 +03:00
Yatsishin Ilya
06bb4e2c71 wip 2021-03-16 13:00:49 +03:00
vdimir
1194d50e48
Try to fix test_storage_s3: crash in WriteBufferFromS3 2021-03-11 15:21:13 +03:00
Pervakov Grigorii
eb87afc4d4 recreate S3 client if credentials changed 2021-03-10 13:36:57 +03:00
Yatsishin Ilya
babea212c7 Merge remote-tracking branch 'origin' into integration-2 2021-03-03 11:56:15 +03:00
Yatsishin Ilya
fc89b4fd35 better 2021-02-20 17:59:39 +03:00
Vladimir Chebotarev
d729aacb09 Fixed table function S3 auto compression mode. 2021-01-29 07:54:52 +03:00
Vladimir Chebotarev
8ba8e8efd5 Added prefix-based S3 endpoint settings. 2021-01-25 06:40:57 +03:00
Pavel Kovalenko
db7a87089b Adaptive choose of single/multi part upload in WriteBufferFromS3. 2020-12-09 17:09:04 +03:00
alesapin
45f09fe293
Merge pull request #17220 from ianton-ru/s3_max_redirects 
S3 max redirects
 2020-11-24 12:13:10 +03:00
Anton Ivashkin
aa208a9d9f Use only 's3_max_redirect' in params instead of all settings 2020-11-23 14:02:17 +03:00
Pavel Kovalenko
5f98d38689 Fixed flaky test_storage_s3::test_custom_auth_headers 2020-11-23 13:19:43 +03:00
Anton Ivashkin
a7d4f4be65 Add 's3_max_redirects' test 2020-11-20 11:18:44 +03:00
Pavel Kovalenko
e311759c5d
Abort multipart upload if no data was written to WriteBufferFromS3. (#16840) 2020-11-11 15:15:16 +03:00
Azat Khuzhin
9cb3c743bd
Convert to python3 (#15007) 2020-10-02 19:54:07 +03:00
Vladimir Chebotarev
b460024356 Fixed bug in GZIP compression in S3 storage. 2020-09-30 16:09:55 +03:00
Vladimir Chebotarev
a38c7ff2a8 Somehow uncompressed PUT works and gzipped PUT doesn't, in S3 storage. 2020-09-30 15:04:42 +03:00
Vladimir Chebotarev
b7a6f15f19 Added test for GZIP in S3 storage. 2020-09-30 13:55:44 +03:00
bharatnc
b75963d370 Format and optimize imports in integration test files 
This PR formats all the `*.py` files found under the `tests/integration`
folder. It also reorders the imports and cleans up a bunch of unused
imports.

The formatting also takes care of other things like wrapping lines and
fixing spaces and indents such that the tests look more readable.
 2020-09-15 23:10:54 -07:00
Pavel Kovalenko
f43428a4d5 S3 HTTPS integration test. 2020-07-10 22:42:18 +03:00
Anton Ivashkin
78ba9c986f throw exception on redirect limit in S3 request 2020-07-07 16:20:48 +03:00
Pervakov Grigorii
bc9381406f Add default credentials and custom headers for s3 table functions. 2020-06-02 20:48:49 +03:00
alexey-milovidov
7201018079
Update test.py 2020-05-26 00:05:15 +03:00
Vladimir Chebotarev
e013edce4b Added test for multi-page S3 globbing. 2020-05-25 15:57:27 +03:00
Pavel Kovalenko
89b9fb6d41 Create 'root' S3 bucket before CH is started. 2020-05-22 14:48:08 +03:00
Ivan
97f2a2213e
Move all folders inside /dbms one level up (#9974) 
* Move some code outside dbms/src folder
* Fix paths
 2020-04-02 02:51:21 +03:00