Commit Graph

239 Commits

Author SHA1 Message Date
avogar
6934e27e8b Add union mode for schema inference to infer union schema of files with different schemas 2023-10-20 20:46:41 +00:00
Sema Checherinda
1308f26ccc Update test.py 2023-10-19 01:34:36 +02:00
Sema Checherinda
dfdd2ee77c fix tests, to avoid timeouts do less retries 2023-10-18 16:36:45 +02:00
avogar
cabb3ddaae Fix tests 2023-09-25 21:45:11 +00:00
Michael Kolupaev
df71dcd94d Fix data race in copyFromIStreamWithProgressCallback 2023-09-11 21:04:37 +00:00
Kruglov Pavel
592fa77987
Merge branch 'master' into cache-count 2023-08-23 15:18:02 +02:00
avogar
30e8e4f6e0 Make test integration 2023-08-22 16:51:09 +00:00
Kruglov Pavel
c0bdd0e00b
Merge branch 'master' into cache-count 2023-08-22 14:42:22 +02:00
avogar
b4145aeddc Cache number of rows in files for count in file/s3/url/hdfs/azure functions 2023-08-22 11:59:59 +00:00
Alexey Milovidov
93a10980e9 Fix review comments 2023-08-06 16:27:45 +02:00
Alexey Milovidov
ce2fcc8f33 Fix wording 2023-08-06 14:48:20 +02:00
Kruglov Pavel
0d34e97dbe
Merge branch 'master' into formats-with-subcolumns 2023-07-26 13:30:35 +02:00
Kruglov Pavel
64e88cde21
Merge branch 'master' into better-progress-bar-2 2023-07-18 13:37:53 +02:00
Kruglov Pavel
06de25451a
Merge branch 'master' into formats-with-subcolumns 2023-07-06 16:21:52 +02:00
Kruglov Pavel
a2805f8f44
Merge branch 'master' into formats-with-subcolumns 2023-07-04 23:27:03 +02:00
avogar
98aa6b317f Support reading subcolumns from file/s3/hdfs/url/azureBlobStorage table functions 2023-07-04 21:17:26 +00:00
kssenii
20d7cf2bf6 Fix tests 2023-07-03 14:51:19 +02:00
kssenii
ac77f5fe6f Merge remote-tracking branch 'upstream/master' into add-separate-access-for-use-named-collections 2023-07-03 13:55:45 +02:00
avogar
795e34fcf6 Fix test 2023-06-29 12:36:50 +00:00
Kruglov Pavel
9af56624f5
Fix flaky test test_skip_empty_files 2023-06-26 13:30:51 +02:00
Kruglov Pavel
f8ddfb1fd8
Merge branch 'master' into allow-skip-empty-files 2023-06-16 13:23:41 +02:00
avogar
870f3d1270 Fix comments 2023-06-15 12:59:46 +00:00
Kruglov Pavel
bb7ec69b90
Fix flaky test_seekable_formats 2023-06-14 19:38:03 +02:00
kssenii
62b94073a2 Fix black check 2023-06-13 13:32:56 +02:00
kssenii
6c776f4483 Better 2023-06-13 12:40:53 +02:00
kssenii
6c48aba693 Fix tests 2023-06-08 20:30:34 +02:00
Kruglov Pavel
bf28074d32
Merge branch 'master' into allow-skip-empty-files 2023-06-08 12:36:18 +02:00
kssenii
c6acdd7008 Fix fast test, fix black check 2023-06-06 18:53:31 +02:00
kssenii
adfedb4df0 Add USE NAMED COLLECTION access 2023-06-06 14:46:34 +02:00
avogar
0b62be649f Add docs, fix style 2023-05-31 17:52:29 +00:00
avogar
d4efbbfbd3 Allow to skip empty files in file/s3/url/hdfs table functions 2023-05-30 19:32:24 +00:00
Kruglov Pavel
fd49821e98
Fix tests 2023-05-25 12:46:53 +02:00
kssenii
82b642c9c6 Fix style check 2023-03-28 21:57:14 +02:00
kssenii
04b28bf822 Merge remote-tracking branch 'upstream/master' into better-tests-for-data-lakes 2023-03-28 15:39:24 +02:00
Mikhail f. Shiryaev
4f2c336cfc
Update black version, run it for the repository 2023-03-24 11:54:38 +01:00
kssenii
0240ad4c68 Add spark to tests, rewrite tests, fix bug 2023-03-08 15:45:35 +01:00
Sema Checherinda
bc8b34f74b adjust test test_seekable_formats 2023-03-03 13:16:46 +01:00
robot-clickhouse
2f9ad860bd Automatic style fix 2023-02-09 08:26:36 +00:00
Antonio Andelic
ae731b739a Prefer explicitly defined creds for S3 2023-02-09 08:13:16 +00:00
Anton Popov
f181254fb0 fix race in destructor of ParallelParsingInputFormat 2023-01-23 01:18:58 +00:00
Anton Popov
41a199e175
Fix crash when ListObjects request fails (#45371) 2023-01-20 20:10:23 +01:00
Vitaly Baranov
0d12d90fb5 Move common code to helpers/mock_servers.py 2023-01-06 15:55:21 +01:00
kssenii
11c0aa189a Better error message 2022-12-22 15:29:35 +01:00
Anton Popov
c4bcfe54cd fix test 2022-12-15 04:19:16 +00:00
Anton Popov
2a1fd48e91 fix tests 2022-11-29 17:33:35 +00:00
kssenii
fb3bcd2fe9 Fix use_environment_credentials 2022-09-30 12:25:39 +02:00
Kseniia Sumarokova
79304422b3
Update test.py 2022-09-22 13:11:35 +02:00
kssenii
92c2219e82 Fix test 2022-09-21 20:51:09 +02:00
kssenii
420ac4eb43 s3 header auth in ast 2022-09-13 15:13:28 +02:00
avogar
93d0113b14 Fix flacky schema inference cache tests 2022-08-24 11:28:56 +00:00
Azat Khuzhin
b92d8b302a tests: fix test_storage_s3::test_wrong_format_usage flakiness
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-08-19 21:51:23 +02:00
avogar
67f848a428 Fix style 2022-08-18 17:15:30 +00:00
avogar
1fb99f51b0 Make test for s3 schema inference cache better 2022-08-18 16:50:59 +00:00
avogar
4c64b7a69e Try fix s3 test 2022-08-17 12:25:58 +00:00
avogar
8dd54c043d Merge branch 'master' of github.com:ClickHouse/ClickHouse into schema-inference-cache 2022-08-17 11:47:40 +00:00
Kruglov Pavel
afd5221d58
FIx check black 2022-08-16 18:53:55 +02:00
avogar
d5f7337ba3 Try fix tests 2022-08-16 14:57:19 +00:00
Kruglov Pavel
169b38c878
Fix s3 test 2022-08-16 12:40:10 +02:00
avogar
c4ff3ffeea Rename settings 2022-08-15 12:45:18 +00:00
avogar
c0bb6303b9 Fix check black 2022-08-11 10:55:18 +00:00
Kruglov Pavel
088e8cf9bd
Merge branch 'master' into numbers-schema-inference 2022-08-09 14:00:36 +02:00
avogar
9b1a267203 Refactor, remove TTL, add size limit, add system table and system query 2022-08-05 16:20:15 +00:00
avogar
02c90fd94b Fix tests 2022-07-21 16:54:42 +00:00
Kruglov Pavel
b38241b08a
Merge branch 'master' into schema-inference-cache 2022-07-14 12:29:54 +02:00
Yatsishin Ilya
d274b05fac improvements 2022-07-07 20:21:10 +00:00
Kseniia Sumarokova
b8ab20bcb3
Update test.py 2022-07-04 20:56:34 +02:00
kssenii
f5bf744028 Fix 2022-07-04 16:05:08 +02:00
avogar
106f92dcdb Fix tests 2022-06-28 16:13:42 +00:00
avogar
b0c9d1a25d Fix style 2022-06-27 14:04:28 +00:00
avogar
5155262a16 Add some additional information to cache keys 2022-06-27 12:43:24 +00:00
Kruglov Pavel
86e8f31ad4
Merge branch 'master' into schema-inference-cache 2022-06-24 16:10:25 +02:00
avogar
77a0c47591 Fix tests 2022-06-21 17:22:53 +00:00
avogar
c14364e3d9 Check last modification time for URL function too 2022-06-21 17:18:14 +00:00
avogar
d37ad2e6de Implement cache for schema inference for file/s3/hdfs/url 2022-06-21 13:02:48 +00:00
Anton Popov
eea7e4eced fix test 2022-06-15 15:02:02 +00:00
Kruglov Pavel
3916436a10
Fix style 2022-04-20 16:35:24 +02:00
Kruglov Pavel
813e228fcc
Merge branch 'master' into improve-globs 2022-04-20 16:31:47 +02:00
Kruglov Pavel
0bae44abf0
Merge pull request #36209 from Avogar/check-magic-bytes
Check ORC/Parquet/Arrow format magic bytes before loading file in memory
2022-04-14 11:32:38 +02:00
avogar
ca14a29dbe Revert reverting "Fix crash in ParallelReadBuffer" 2022-04-13 20:39:12 +00:00
tavplubix
b2cd23658a
Revert "Fix crash in ParallelReadBuffer" 2022-04-13 22:45:34 +03:00
avogar
42726639f3 Check ORC/Parquet/Arrow format magic bytes before loading file in memory 2022-04-13 19:27:38 +00:00
avogar
1c065f8c7a Some refactoring around schema inference with globs 2022-04-13 17:02:48 +00:00
avogar
de9c9462e5 Fix crash in ParallelReadBuffer 2022-04-12 13:54:53 +00:00
Azat Khuzhin
c5ffbc688e Do not allow SETTINGS after FORMAT for INSERT queries
Parsing SETTINGS after FORMAT, that has been introduced in [1], can
interpret SETTING as some values, which is misleading.

  [1]: https://github.com/ClickHouse/ClickHouse/pull/4174/files#diff-ba7bd0657630b1cd94cf6ed364bd857338096f49f66dc82918438d6745753775R106

Note, that we are touching only INSERT queries, not SELECT, since this
is a backward incompatible change, and in case of modifying SELECT it
can break too much.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
Fixes: #35100
Fixes: #20343
2022-04-07 16:29:36 +03:00
Antonio Andelic
362e877529 Merge branch 'master' into parallel-s3-downloading 2022-03-29 07:05:26 +00:00
Nikita Mikhaylov
85282f1b4d
Made schema inference work with s3Cluster table function (#35544) 2022-03-29 00:46:35 +02:00
Antonio Andelic
62a495a5fc Small refactor 2022-03-24 09:45:30 +00:00
Antonio Andelic
e0d7b6dc3e Add tests for S3 multithreaded download 2022-03-24 09:30:06 +00:00
Mikhail f. Shiryaev
e6f5a3f98b
Apply black formatter to all *.py files in the repo 2022-03-22 17:39:58 +01:00
Kruglov Pavel
a506120646
Fix bug in schema inference in s3 table function (#35176) 2022-03-10 15:16:07 +01:00
kssenii
b3a7f07954 Fix flacky test 2022-03-02 13:01:15 +01:00
kssenii
7597c692d1 Fix test 2022-03-01 12:07:13 +01:00
kssenii
092ec45b47 Merge master 2022-03-01 12:06:56 +01:00
kssenii
af6a5c78a6 Fix tests 2022-03-01 10:38:01 +01:00
Kruglov Pavel
ee37e93499
Update test.py 2022-02-28 20:39:15 +03:00
avogar
916ce3f96d Fix tests 2022-02-28 10:07:29 +00:00
kssenii
7d7277a7c6 Add test 2022-02-25 09:32:39 +01:00
Kruglov Pavel
a0ae68db99
Fix tests 2022-02-19 17:41:56 +03:00
avogar
653d769d34 Support schema inference for INSERT INTO FUNCTION 2022-02-18 16:19:42 +00:00
Kruglov Pavel
dd863ca2a0
Merge branch 'master' into fix-url-globs 2022-02-16 12:45:31 +03:00