Commit Graph

153 Commits

Author SHA1 Message Date
avogar
617cc514b7 Try to detect file format automatically during schema inference if it's unknown 2024-01-23 18:59:39 +00:00
avogar
6934e27e8b Add union mode for schema inference to infer union schema of files with different schemas 2023-10-20 20:46:41 +00:00
avogar
6dcaa387b1 Fix style 2023-09-25 22:15:41 +00:00
avogar
cabb3ddaae Fix tests 2023-09-25 21:45:11 +00:00
avogar
894513f6cd Fix tests 2023-08-23 18:43:08 +00:00
avogar
b4145aeddc Cache number of rows in files for count in file/s3/url/hdfs/azure functions 2023-08-22 11:59:59 +00:00
Kruglov Pavel
0d34e97dbe
Merge branch 'master' into formats-with-subcolumns 2023-07-26 13:30:35 +02:00
Kruglov Pavel
1dd05319b5
Merge branch 'master' into formats-with-subcolumns 2023-07-17 19:13:42 +02:00
Andrey Zvonov
c030e9dbe4
Merge branch 'master' into zvonand-issue-49290 2023-07-10 11:01:26 +02:00
Azat Khuzhin
2db092f9d8 Cleanup remote_servers in dist config.xml
At first, there was no such amount of clusters in dist config, they
added when someone need to write some new cluster for tests.

So let's move them to the clusters.xml that is deployed only for tests,
and leave only default cluster.

And cleanup also some configs that had been copied from dist config in
the repo (about test_config_* integration tests, this should be OK,
since there are more_clusters.xml as well, that covers additional
cases).

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-07-09 16:55:27 +02:00
avogar
98aa6b317f Support reading subcolumns from file/s3/hdfs/url/azureBlobStorage table functions 2023-07-04 21:17:26 +00:00
zvonand
84bb17033c fix black 2023-06-22 19:48:28 +02:00
Andrey Zvonov
765b4ce279 added wrong path for hdfs test 2023-06-22 17:19:47 +02:00
zvonand
1c10578f14 update to master 2023-06-16 21:30:47 +02:00
zvonand
1d8013074d fix style & black 2023-06-13 18:47:02 +02:00
Andrey Zvonov
427c5cb1ba fix integration test 2023-06-11 18:42:10 +02:00
zvonand
aad7712b18 add existing test 2023-06-11 02:09:05 +02:00
avogar
d4efbbfbd3 Allow to skip empty files in file/s3/url/hdfs table functions 2023-05-30 19:32:24 +00:00
kigerzhang
af5a8ce74c style check 2023-02-23 17:18:56 +08:00
kigerzhang
f74f3718a0 add skip_unavailable_shards setting for table function cluste 2023-02-23 17:05:51 +08:00
chen
cb83337586
Update test.py 2022-11-17 06:41:43 +08:00
xiedeyantu
fc1d06f096 fix 2022-11-16 23:43:23 +08:00
xiedeyantu
7dc941cacd fix 2022-11-16 19:47:57 +08:00
xiedeyantu
c23cd091a3 fix 2022-11-16 09:59:44 +08:00
xiedeyantu
8144516e53 fix 2022-11-16 09:33:46 +08:00
xiedeyantu
3d218795be fix 2022-11-16 09:18:45 +08:00
xiedeyantu
09c749fbd0 update 2022-11-15 13:30:29 +08:00
xiedeyantu
ec6698395e fix skip_unavailable_shards does not work using hdfsCluster table function 2022-11-15 13:25:15 +08:00
avogar
93d0113b14 Fix flacky schema inference cache tests 2022-08-24 11:28:56 +00:00
avogar
8dd54c043d Merge branch 'master' of github.com:ClickHouse/ClickHouse into schema-inference-cache 2022-08-17 11:47:40 +00:00
avogar
d5f7337ba3 Try fix tests 2022-08-16 14:57:19 +00:00
Kruglov Pavel
795262f16c
Fix hdfs test 2022-08-16 12:39:35 +02:00
avogar
c4ff3ffeea Rename settings 2022-08-15 12:45:18 +00:00
avogar
c0bb6303b9 Fix check black 2022-08-11 10:55:18 +00:00
avogar
9b1a267203 Refactor, remove TTL, add size limit, add system table and system query 2022-08-05 16:20:15 +00:00
avogar
02c90fd94b Fix tests 2022-07-21 16:54:42 +00:00
Kruglov Pavel
5904aa1ae0
Fix check black 2022-06-29 13:10:39 +02:00
avogar
106f92dcdb Fix tests 2022-06-28 16:13:42 +00:00
avogar
b0c9d1a25d Fix style 2022-06-27 14:04:28 +00:00
avogar
5155262a16 Add some additional information to cache keys 2022-06-27 12:43:24 +00:00
avogar
d37ad2e6de Implement cache for schema inference for file/s3/hdfs/url 2022-06-21 13:02:48 +00:00
Kruglov Pavel
af3db5a992
Merge pull request #36726 from PolyProgrammist/s3clustermacro
Handle cluster macro in s3Cluster table function
2022-05-11 18:00:57 +02:00
Vadim Volodin
233f0b4b52 Handle cluster macro in s3Cluster table function 2022-04-28 18:38:17 +03:00
Kruglov Pavel
6ddebf7d38
Fix style 2022-04-20 16:34:05 +02:00
avogar
1c065f8c7a Some refactoring around schema inference with globs 2022-04-13 17:02:48 +00:00
lgbo-ustc
759e4a9a05 Merge remote-tracking branch 'ck/master' into s3_cluster_join 2022-04-08 09:25:30 +08:00
lgbo-ustc
dbdc632243 fixed code style 2022-04-07 18:11:49 +08:00
kssenii
f17a320635 Fix test 2022-03-31 11:13:38 +02:00
lgbo-ustc
d0e5b3571c fixed code style 2022-03-31 09:28:07 +08:00
kssenii
d810853b78 Fix path in test 2022-03-30 17:35:59 +02:00
lgbo-ustc
08e1fdc6b8 Fixed :a elimate AMBIGUOUS_COLUMN_NAME exception
elimate AMBIGUOUS_COLUMN_NAME exception in hdfs/s3 cluster when run a query with join
2022-03-30 17:06:38 +08:00
kssenii
edf0436f5c Fix style check 2022-03-25 10:14:14 +01:00
kssenii
e91e30aaa4 Fix 2022-03-24 17:10:27 +01:00
Mikhail f. Shiryaev
e6f5a3f98b
Apply black formatter to all *.py files in the repo 2022-03-22 17:39:58 +01:00
avogar
916ce3f96d Fix tests 2022-02-28 10:07:29 +00:00
avogar
653d769d34 Support schema inference for INSERT INTO FUNCTION 2022-02-18 16:19:42 +00:00
avogar
84087c1554 Improve schema inference with globs 2022-02-09 19:16:15 +03:00
Kruglov Pavel
7bfb1231b9
Merge branch 'master' into formats-with-suffixes 2022-01-20 14:47:17 +03:00
avogar
4efadfad3c Fix tests 2022-01-18 22:26:13 +03:00
liuneng1994
15ea37c4ae ignore file not found exception
Signed-off-by: neng.liu <neng.liu@kyligence.io>
2022-01-18 12:49:40 +00:00
mergify[bot]
df869c5b9c
Merge branch 'master' into fix_ignore_hdfs_error 2022-01-18 07:05:24 +00:00
liuneng1994
c6e205b2ad add error check 2022-01-18 06:21:40 +00:00
Kruglov Pavel
a7df9cd53a
Merge branch 'master' into formats-with-suffixes 2022-01-14 21:03:49 +03:00
avogar
2f000a178a Add more tests 2022-01-14 14:00:50 +03:00
avogar
97788b9c21 Allow to create new files on insert for File/S3/HDFS engines 2021-12-29 21:19:13 +03:00
avogar
9f30c17777 Fix tests 2021-12-29 12:18:56 +03:00
avogar
74f09d6476 Fix tests 2021-12-29 12:18:56 +03:00
avogar
aaf9f85c67 Add more tests and fixes 2021-12-29 12:18:56 +03:00
Zhichang Yu
869cdbeb44 added table function hdfsCluster 2021-12-20 13:14:44 +00:00
taiyang-li
9d1e63be0f 1. support default expression for storage hdfs; 2. optimize storage hdfs when format is column oriented 2021-12-06 11:54:45 +08:00
kssenii
46d2fdc5e6 Update 2021-11-24 21:53:53 +03:00
kssenii
f18dcd2287 Merge branch 'master' of github.com:ClickHouse/ClickHouse into seekable-read-buffers 2021-11-13 14:38:57 +03:00
Kseniia Sumarokova
f3dc7c0e5d
Update test.py 2021-11-10 17:21:25 +03:00
Kseniia Sumarokova
a18fd8bdfc
Fix 2021-11-10 17:20:25 +03:00
avogar
f10dfbe61a fix tests 2021-11-08 14:12:37 +03:00
kssenii
ec11179f91 Merge branch 'master' of github.com:ClickHouse/ClickHouse into seekable-read-buffers 2021-11-03 14:33:31 +03:00
kssenii
45ea820297 Reduce memory usage for some formats 2021-11-03 14:30:03 +03:00
kssenii
20772fde04 For storage 2021-10-27 10:04:17 +03:00
kssenii
1d743b9259 Partitoned write 2021-10-25 19:23:44 +03:00
Yatsishin Ilya
a4a960328e Merge remote-tracking branch 'origin' into integration-6 2021-07-12 11:34:23 +03:00
Yatsishin Ilya
143368d8d6 wip 2021-07-12 11:32:20 +03:00
Kseniia Sumarokova
ed687f45a1
Merge branch 'master' into truncate 2021-06-30 20:08:42 +03:00
Vitaly Baranov
3a3cbeb7aa Collect stderr.log and stdout.log in all integration tests by default. 2021-06-29 17:30:47 +03:00
kssenii
ac0f86cdbf Truncate for s3 2021-06-21 15:44:36 +00:00
kssenii
4688f9e038 hdfs truncate table 2021-06-21 13:59:57 +00:00
Yatsishin Ilya
aba39d58f9 wip 2021-06-09 12:23:02 +03:00
Yatsishin Ilya
30b05fec42 fix 2021-04-27 20:20:13 +03:00
Yatsishin Ilya
b80d4f8e96 fix 2021-04-27 20:18:09 +03:00
Yatsishin Ilya
c59a11f798 Merge remote-tracking branch 'origin' into integration-2 2021-04-27 18:34:33 +03:00
Kseniia Sumarokova
f9291d7624
Merge branch 'master' into fix-hdfs-virtual-columns 2021-04-20 22:01:32 +03:00
kssenii
f4d663bc56 Add test 2021-04-20 08:38:14 +00:00
kssenii
bb767b66d5 Add integration test 2021-04-19 20:39:22 +00:00
Yatsishin Ilya
ee955038c1 wip 2021-02-19 15:58:11 +03:00
Yatsishin Ilya
2df4317aba wip 2021-02-19 00:21:50 +03:00
Ilya Golshtein
7e97814859 cleanup, fixes, new submodules, ShellCommand, WriteBufferFromString 2020-12-16 14:48:33 +03:00
Ilya Golshtein
d1d657335b USE_INTERNAL_HDFS3_LIBRARY + bash lint 2020-12-16 14:48:33 +03:00
Ilya Golshtein
d9166a0645 HADOOP_SECURE_DN_USER way, kinit thread, docker capabilities 2020-12-16 14:48:33 +03:00
Ilya Golshtein
185ac744b1 kerberized hdfs compiled 2020-12-16 14:48:33 +03:00
alesapin
db03a894e5 Change test to run flaky check 2020-10-06 16:29:50 +03:00
Azat Khuzhin
9cb3c743bd
Convert to python3 (#15007) 2020-10-02 19:54:07 +03:00