thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-15 02:41:59 +00:00
Code Issues Packages Projects Releases Wiki Activity
103,221 Commits 9,328 Branches 1,783 Tags 2.1 GiB
8b9b8b083c
Commit Graph

326 Commits

Author SHA1 Message Date
lgbo-ustc
759e4a9a05 Merge remote-tracking branch 'ck/master' into s3_cluster_join 2022-04-08 09:25:30 +08:00
shuchaome
a565a93740 reduce mutex scope when setenv LIBHDFS3_CONF 2022-04-02 06:52:39 +08:00
Alexey Milovidov
5c6b84c398
Merge pull request #35586 from kssenii/fix-virtual-columns-s3 
Fix s3 engine getting virtual columns
 2022-04-01 04:43:32 +03:00
Nikita Mikhaylov
5f27f690f4 Revert "Added support for schema inference for hdfsCluster (#35602)" 
This reverts commit f6bfdcc0c9.
 2022-03-31 14:19:56 +02:00
Nikita Mikhaylov
f6bfdcc0c9
Added support for schema inference for hdfsCluster (#35602) 2022-03-31 12:47:36 +02:00
lgbo-ustc
08e1fdc6b8 Fixed :a elimate AMBIGUOUS_COLUMN_NAME exception 
elimate AMBIGUOUS_COLUMN_NAME exception in hdfs/s3 cluster when run a query with join
 2022-03-30 17:06:38 +08:00
kssenii
4374415bd7 Fix clang-tidy, style check 2022-03-29 14:20:21 +02:00
kssenii
a2013ebe2b Better 2022-03-28 22:40:27 +02:00
taiyang-li
fbadd8fc0e only deduce absolute path of hdfs config when it's relative path do not exists 2022-03-25 22:28:47 +08:00
taiyang-li
c0eb9f73a9 remote std::cout 2022-03-24 21:17:43 +08:00
taiyang-li
6eccd09e2b deduce hdfs config path 2022-03-24 18:04:29 +08:00
Anton Popov
36ec379aeb Merge remote-tracking branch 'upstream/master' into HEAD 2022-03-14 16:28:35 +00:00
kssenii
5260822964 Merge master 2022-03-08 18:21:28 +01:00
Anton Popov
df3b07fe7c Merge remote-tracking branch 'upstream/master' into HEAD 2022-03-03 22:25:28 +00:00
kssenii
a594f388a4 Merge master 2022-03-01 19:43:45 +01:00
Anton Popov
c1fdcf7a64 Merge remote-tracking branch 'upstream/master' into HEAD 2022-03-01 20:21:39 +03:00
kssenii
092ec45b47 Merge master 2022-03-01 12:06:56 +01:00
kssenii
9b64a8fe39 Fix odbc bridge 2022-02-28 14:29:05 +01:00
Hongbin
c9bc442114
fix comments 2022-02-28 16:44:35 +08:00
Hongbin
99bd56e2de
Fix some code comments style 2022-02-28 08:15:37 +08:00
Anton Popov
fcdebea925 Merge remote-tracking branch 'upstream/master' into HEAD 2022-02-25 13:41:30 +03:00
kssenii
c637385dd0 Merge master 2022-02-22 13:17:51 +01:00
kssenii
47f94120da Merge master 2022-02-16 14:43:28 +01:00
kssenii
3bd3e51aa0 Fix tests 2022-02-16 14:08:41 +01:00
Kruglov Pavel
dd863ca2a0
Merge branch 'master' into fix-url-globs 2022-02-16 12:45:31 +03:00
alesapin
bc2d0ee7c7
Merge pull request #34215 from ClickHouse/revert-34211-revert-34153-add_func_tests_over_s3 
Add func tests run with s3 and fix several bugs
 2022-02-15 19:07:11 +03:00
alesapin
89373155fc Merge branch 'master' into revert-34211-revert-34153-add_func_tests_over_s3 2022-02-13 21:07:54 +03:00
mergify[bot]
8a47ed2b3f
Merge branch 'master' into static-files-disk-uploader-create-symlinks 2022-02-13 06:12:39 +00:00
Dmitry Novik
50131fd732 Fix cancelation for S3 and HDFS 2022-02-11 18:14:55 +00:00
alesapin
088f8cacf3 Small improvements 2022-02-10 19:50:21 +03:00
Anton Popov
298838f891 avoid unnecessary copying of Settings 2022-02-10 12:13:51 +03:00
Anton Popov
18940b8637 Merge remote-tracking branch 'upstream/master' into HEAD 2022-02-09 23:38:38 +03:00
avogar
84087c1554 Improve schema inference with globs 2022-02-09 19:16:15 +03:00
Alexey Milovidov
742620209a static-files-disk-uploader: add mode to create symlinks 2022-02-09 03:50:50 +03:00
alexey-milovidov
43ee8ddb5b
Merge pull request #34376 from CurtizJ/fix-reading-empty-arrays 
Fix consecutive backward seeks in seekable read buffers
 2022-02-08 02:27:57 +03:00
Anton Popov
1b16db72c3 fix consecutive backward seeks in seekable read buffers 2022-02-07 17:20:26 +03:00
feng lv
6325d4d9b0 continue of #34317 
fix

fix
 2022-02-06 08:59:17 +00:00
Anton Popov
836a348a9c Merge remote-tracking branch 'upstream/master' into HEAD 2022-02-01 15:23:07 +03:00
alesapin
5230c59f05 Smaller scope 2022-01-31 14:03:06 +03:00
alesapin
1adc7538fe Fix benign race condition for storage HDFS, S3, URL 2022-01-31 13:50:28 +03:00
Anton Popov
78b9f15abb Merge remote-tracking branch 'upstream/master' into HEAD 2022-01-30 03:24:37 +03:00
Anton Popov
e8ce091e68 Merge remote-tracking branch 'upstream/master' into HEAD 2022-01-21 20:11:18 +03:00
Kruglov Pavel
7bfb1231b9
Merge branch 'master' into formats-with-suffixes 2022-01-20 14:47:17 +03:00
alexey-milovidov
28a9d569cc
Merge pull request #33690 from azat/remove-unbundled-support 
Remove unbundled support
 2022-01-20 12:29:07 +03:00
liuneng1994
59f3fe515b change null check position 
Signed-off-by: neng.liu <neng.liu@kyligence.io>
 2022-01-20 06:26:34 +00:00
Azat Khuzhin
aef6668cb4 Remove unbundled hdfs3 support 2022-01-20 08:47:16 +03:00
liuneng1994
4d3bb1584d fix Dereference of null pointer error 
Signed-off-by: neng.liu <neng.liu@kyligence.io>
 2022-01-19 07:41:46 +00:00
avogar
4efadfad3c Fix tests 2022-01-18 22:26:13 +03:00
liuneng1994
5e1e512bf2 fix check style 
Signed-off-by: neng.liu <neng.liu@kyligence.io>
 2022-01-18 14:42:28 +00:00
liuneng1994
15ea37c4ae ignore file not found exception 
Signed-off-by: neng.liu <neng.liu@kyligence.io>
 2022-01-18 12:49:40 +00:00
mergify[bot]
df869c5b9c
Merge branch 'master' into fix_ignore_hdfs_error 2022-01-18 07:05:24 +00:00
liuneng1994
c6e205b2ad add error check 2022-01-18 06:21:40 +00:00
avogar
af54ca4866 Fix partition write 2022-01-14 21:21:36 +03:00
Kruglov Pavel
a7df9cd53a
Merge branch 'master' into formats-with-suffixes 2022-01-14 21:03:49 +03:00
Kruglov Pavel
d2e9f37bee
Merge branch 'master' into format-by-extention 2022-01-14 18:36:23 +03:00
avogar
89a181bd19 Make better 2022-01-14 18:16:18 +03:00
Kruglov Pavel
5a908e8edd
Merge branch 'master' into formats-with-suffixes 2022-01-14 16:45:20 +03:00
avogar
8884c4a3fe Fix tests and build 2022-01-14 16:27:57 +03:00
avogar
2d7b1bfa5e Detect format in S3/HDFS/URL table engines 2022-01-13 16:14:18 +03:00
taiyang-li
1e102bc1b2 merge master 2022-01-01 09:01:06 +08:00
avogar
131d49e6be Remove comments 2021-12-29 21:26:23 +03:00
avogar
97788b9c21 Allow to create new files on insert for File/S3/HDFS engines 2021-12-29 21:19:13 +03:00
avogar
8436638a89 Fix style 2021-12-29 12:21:01 +03:00
avogar
26abf7aa62 Remove code duplication, use simdjson and rapidjson instead of Poco 2021-12-29 12:21:01 +03:00
avogar
8112a71233 Implement schema inference for most input formats 2021-12-29 12:18:56 +03:00
kssenii
1f6ca619b7 Allow some killing 2021-12-27 22:42:56 +03:00
taiyang-li
9036b18c2f merge master 2021-12-27 15:12:48 +08:00
Alexey Milovidov
f03cb4c762 Cleanup trash from Kafka and HDFS 2021-12-25 06:10:59 +03:00
Nikita Mikhaylov
29fdc422d0
Merge pull request #32974 from nikitamikhaylov/hdfsCluster 
Merging #32400
 2021-12-24 16:02:54 +03:00
Nikita Mikhaylov
d9c13900f5 Fix build + style 2021-12-20 14:31:56 +00:00
Zhichang Yu
869cdbeb44 added table function hdfsCluster 2021-12-20 13:14:44 +00:00
avogar
e31c18e9f8 Fix hdfs url check 2021-12-20 16:09:48 +03:00
Anton Popov
99ebabd822 Merge remote-tracking branch 'upstream/master' into HEAD 2021-12-17 19:02:29 +03:00
李扬
8675086104
Merge branch 'master' into hive_table 2021-12-12 09:01:46 -06:00
Kseniia Sumarokova
7e261245d3
Merge pull request #32256 from taiyang-li/default_for_hdfs 
Support default expression for storage hdfs and optimize fetching when source is column oriented
 2021-12-06 23:32:01 +03:00
leosunli
6f611eec91 Fix bug 
Signed-off-by: leosunli <lisheng.sun08@gmail.com>
 2021-12-06 14:32:32 +08:00
Kseniia Sumarokova
4017bb8f78 Update WriteBufferFromHDFS.h 2021-12-06 14:32:32 +08:00
leosunli
e81a1dbe6a Make HDFS replication configurable in WriteBufferFromHDFSImpl#WriteBufferFromHDFSImpl 
Signed-off-by: leosunli <lisheng.sun08@gmail.com>
 2021-12-06 14:32:31 +08:00
leosunli
5be58a8839 Make HDFS replication configurable in WriteBufferFromHDFSImpl#WriteBufferFromHDFSImpl 
Signed-off-by: leosunli <lisheng.sun08@gmail.com>
 2021-12-06 14:32:31 +08:00
sunlisheng
ef77f45f59 Make HDFS replication configurable in WriteBufferFromHDFSImpl#WriteBufferFromHDFSImpl 
Signed-off-by: sunlisheng <lisheng.sls@alibaba-inc.com>
 2021-12-06 14:32:31 +08:00
taiyang-li
9d1e63be0f 1. support default expression for storage hdfs; 2. optimize storage hdfs when format is column oriented 2021-12-06 11:54:45 +08:00
Anton Popov
6f4d9a53b2 Merge remote-tracking branch 'origin/sparse-serialization' into HEAD 2021-12-01 15:54:33 +03:00
taiyang-li
72f60cceb9
Merge branch 'master' into hive_table 2021-11-25 17:33:26 +08:00
Kseniia Sumarokova
93cf66df12
Merge pull request #30936 from kssenii/seekable-read-buffers 
Reduce memory usage for some formats when reading with s3/url/hdfs
 2021-11-25 11:19:24 +03:00
kssenii
46d2fdc5e6 Update 2021-11-24 21:53:53 +03:00
taiyang-li
89dcef69d5 merge master 2021-11-24 14:38:04 +08:00
Anton Popov
ccd78e3838 Merge remote-tracking branch 'upstream/master' into HEAD 2021-11-22 17:19:35 +03:00
Kruglov Pavel
d9c1a0c8ec
Merge branch 'master' into fix-write-buffers 2021-11-20 17:48:24 +03:00
kssenii
ff969b4605 Merge branch 'master' of github.com:ClickHouse/ClickHouse into seekable-read-buffers 2021-11-20 15:03:13 +03:00
taiyang-li
e8644807fe merge master and solve conflict 2021-11-19 15:01:58 +08:00
Kruglov Pavel
f27fcf8372
Merge pull request #31004 from Avogar/output-formats 
Small refactoring in formats
 2021-11-17 20:44:46 +03:00
kssenii
f18dcd2287 Merge branch 'master' of github.com:ClickHouse/ClickHouse into seekable-read-buffers 2021-11-13 14:38:57 +03:00
avogar
e5661b9b62 Minor change 2021-11-11 21:09:21 +03:00
avogar
c521a9131a Small refactoring of WriteBiffer-s 2021-11-11 02:11:18 +03:00
taiyang-li
79cf9fb7ce fix code style 2021-11-10 11:41:20 +08:00
taiyang-li
26988ab757
Merge branch 'master' into hive_table 2021-11-10 11:01:03 +08:00
Anton Popov
a20922b2d3 Merge remote-tracking branch 'origin/sparse-serialization' into HEAD 2021-11-09 15:36:25 +03:00
taiyang-li
36ca0b296b implement hive table engine 2021-11-05 19:55:30 +08:00
avogar
2dd4393ca1 Small refactoring in formats 2021-11-03 20:07:05 +03:00
kssenii
ec11179f91 Merge branch 'master' of github.com:ClickHouse/ClickHouse into seekable-read-buffers 2021-11-03 14:33:31 +03:00
kssenii
45ea820297 Reduce memory usage for some formats 2021-11-03 14:30:03 +03:00
Kruglov Pavel
77f5ef476a
Merge branch 'master' into hdfs-bad-url 2021-11-03 14:08:55 +03:00
avogar
3839d521a0 Add additional hdfs url check 2021-11-03 14:01:50 +03:00
Kseniia Sumarokova
065106c299
Merge pull request #30690 from kssenii/partitioned-write 
Partitioned write for file, hdfs, url storages and table functions
 2021-11-02 22:58:16 +03:00
kssenii
7e2ea97e3c Merge branch 'master' of github.com:ClickHouse/ClickHouse into disk-async-read 2021-10-28 18:46:21 +03:00
Alexey Milovidov
8b4a6a2416 Remove cruft 2021-10-28 02:10:39 +03:00
kssenii
20772fde04 For storage 2021-10-27 10:04:17 +03:00
kssenii
2ba3ee830e Partitioned write part 2 2021-10-26 14:00:41 +03:00
kssenii
1d743b9259 Partitoned write 2021-10-25 19:23:44 +03:00
kssenii
39b9e9c258 Merge branch 'master' of github.com:ClickHouse/ClickHouse into disk-async-read 2021-10-22 15:40:41 +03:00
kssenii
e4e157688d Review fixes 2021-10-21 01:57:43 +03:00
kssenii
4a0bdd296d Merge branch 'master' of github.com:ClickHouse/ClickHouse into disk-async-read 2021-10-16 18:35:48 +03:00
Nikolai Kochetov
a08c98d760 Move some files. 2021-10-16 17:03:50 +03:00
kssenii
881ae8617e Merge branch 'master' of github.com:ClickHouse/ClickHouse into disk-async-read 2021-10-15 15:09:56 +03:00
kssenii
3995506d37 Adjust range reader for remote fs reads 2021-10-15 11:36:26 +03:00
Nikolai Kochetov
2957971ee3 Remove some last streams. 2021-10-13 21:22:02 +03:00
Nikolai Kochetov
ec18340351 Remove streams from formats. 2021-10-11 19:11:50 +03:00
Nikolai Kochetov
78e1db209f
Remove more data streams (#29491) 
* Remove more streams.

* Fixing build.

* Fixing build.

* Rename files.

* Fix fast test.

* Fix StorageKafka.

* Try fix kafka test.

* Move createBuffer to KafkaSource ctor.

* Revert "Move createBuffer to KafkaSource ctor."

This reverts commit 81fa94d27e.

* Revert "Try fix kafka test."

This reverts commit 2107e54969.

* Comment some rows in test.

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2021-10-07 11:26:08 +03:00
Alexey Milovidov
fe6b7c77c7 Rename "common" to "base" 2021-10-02 10:13:14 +03:00
Nikolai Kochetov
236d71ea94
Merge pull request #28582 from ClickHouse/rewrite-pushing-to-views 
Rewrite PushingToViews
 2021-09-27 21:19:11 +03:00
kssenii
d71b6ad657 Fix 2021-09-23 07:21:08 +00:00
Nikolai Kochetov
f569a3e3f7 Merge branch 'master' into rewrite-pushing-to-views 2021-09-09 20:30:23 +03:00
Anton Popov
4c388e3d84 Merge remote-tracking branch 'origin/sparse-serialization' into HEAD 2021-09-09 14:10:16 +03:00
Nikolai Kochetov
66a76ab70f Rewrite PushingToViewsBlockOutputStream part 6 2021-09-03 20:29:36 +03:00
Kseniia Sumarokova
00bea71610
Merge pull request #25251 from kssenii/disk-over-web-server 
ClickHouse over web server with static files
 2021-08-30 16:18:59 +03:00
Zhichang Yu
ef1994f420 set env LIBHDFS3_CONF, refers to ClickHouse#8159 2021-08-28 09:16:48 +08:00
kssenii
073d7fdd5e Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into disk-over-web-server 2021-08-06 19:42:29 +00:00
Anton Popov
e36736b50c Merge remote-tracking branch 'origin/sparse-serialization' into HEAD 2021-08-02 22:52:02 +03:00
Nikolai Kochetov
9c92f43359 Update storages. 2021-07-23 22:33:59 +03:00
Anton Popov
f99374cca6 Merge remote-tracking branch 'origin/sparse-serialization' into HEAD 2021-07-20 18:20:21 +03:00
Alexey Milovidov
261a220227 Remove some code 2021-07-17 21:06:46 +03:00
Anton Popov
3ed7f5a6cc dynamic subcolumns: add snapshot for storage 2021-07-09 06:15:41 +03:00
Kseniia Sumarokova
f3b0f11b59
Update StorageHDFS.cpp 2021-06-21 23:04:58 +03:00
kssenii
ac0f86cdbf Truncate for s3 2021-06-21 15:44:36 +00:00
kssenii
4688f9e038 hdfs truncate table 2021-06-21 13:59:57 +00:00
kssenii
f33387837e Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into disk-over-web-server 2021-06-18 14:27:48 +00:00
Vladimir
4de0d1a593
Merge pull request #24885 from Jokser/disk-s3-read-error-fix 2021-06-18 13:38:59 +03:00
Maksim Kita
67e9b85951 Merge ext into common 2021-06-16 23:28:41 +03:00
kssenii
cd885ee750 Fix build, ya make 2021-06-14 11:52:20 +00:00
Maksim Kita
30c78c0a23 Fix arcadia 2021-06-11 14:26:33 +03:00
Pavel Kovalenko
bf5190cd34 Fix reading from ReadBufferFromHDFS 2021-06-03 14:34:47 +03:00
kssenii
b28157a0df Merge branch 'master' of github.com:ClickHouse/ClickHouse into poco-file-to-std-fs 2021-05-27 00:01:25 +03:00
kssenii
3dee003f9b Merge branch 'master' of github.com:ClickHouse/ClickHouse into poco-file-to-std-fs 2021-05-20 19:20:09 +03:00
kssenii
ba41d01b78 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into hdfs-idisk 2021-05-19 11:47:05 +00:00
kssenii
9b8df78fdd Merge branch 'master' of github.com:ClickHouse/ClickHouse into poco-file-to-std-fs 2021-05-17 17:42:05 +03:00
kssenii
71fcbefdc7 Add final to class 2021-05-17 14:38:48 +00:00
kssenii
26fa920ad5 Fix clang tidy, slightly improve writeSuffix 2021-05-17 12:32:48 +00:00
kssenii
a8748a55df Fix abnormal termination when hdfs goes away 2021-05-17 09:55:23 +00:00
kssenii
850b6f02cb Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into hdfs-idisk 2021-05-11 14:22:30 +00:00
kssenii
02288359c5 Less manual concatenation of paths 2021-05-08 13:59:55 +03:00
feng lv
39f68bf5ff fix conflict 2021-05-02 16:33:45 +00:00
kssenii
c416fc3441 Fix hdfs read with Values format 2021-04-29 11:31:23 +00:00
feng lv
0f5c05854c fix conflict 2021-04-23 12:22:51 +00:00
feng lv
4ffe199d39 Implement table comments 2021-04-23 12:18:23 +00:00
Kseniia Sumarokova
0b6b30887c
Fix style check 2021-04-21 01:35:13 +03:00
kssenii
87b4843134 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into hdfs-idisk 2021-04-20 21:40:01 +00:00
kssenii
a07d44e438 More common 2021-04-20 11:41:32 +00:00
kssenii
da7a4ac1ea Better 2021-04-20 07:53:55 +00:00
kssenii
cc5f49fbee Fix 2021-04-19 19:43:22 +00:00
kssenii
5df0fcc114 Fixes and tests 2021-04-19 15:33:33 +00:00
kssenii
387494f0c8 Common interface 2021-04-18 13:37:23 +00:00
kssenii
ab06ca3144 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into hdfs-idisk 2021-04-12 07:07:48 +00:00
Ivan
495c6e03aa
Replace all Context references with std::weak_ptr (#22297) 
* Replace all Context references with std::weak_ptr

* Fix shared context captured by value

* Fix build

* Fix Context with named sessions

* Fix copy context

* Fix gcc build

* Merge with master and fix build

* Fix gcc-9 build
 2021-04-11 02:33:54 +03:00
kssenii
908d872b76 Add seek method 2021-04-05 09:32:06 +00:00
kssenii
388a71e6b1 Fix build, ya check 2021-04-04 07:33:07 +00:00
kssenii
8cbfbb5de7 Small refactoring, add hdfs to test_disk_types 2021-04-03 12:02:29 +00:00
Alexey Milovidov
177a017c5c Minor modification #22115 2021-03-25 20:51:32 +03:00
Nikita Mikhaylov
1f92c8ce58 parallel formatting everywhere 2021-03-15 14:51:24 +03:00
Nikita Mikhailov
2f04cb5ebe abort() instead of std::terminate() + cleanup 2020-12-30 08:31:45 +03:00
Nikita Mikhailov
c5f92e5096 better formatfactory 2020-12-30 06:07:30 +03:00
Nikita Mikhailov
6a55e018e4 Merge branch 'master' of github.com:ClickHouse/ClickHouse into parallel-parsing-input-format 2020-12-18 02:25:32 +03:00
Ilya Golshtein
09d1e89674 yandex/clickhouse-kerberized-hadoop + missed header 2020-12-16 14:48:33 +03:00
Ilya Golshtein
407565b47f fix - wrong error code 2020-12-16 14:48:33 +03:00
Ilya Golshtein
9701d6a2d5 style fix per code review, doc improvement, params consistency check 2020-12-16 14:48:33 +03:00
Ilya Golshtein
815856bc3d pass config, not context to createHDFSBuilder 2020-12-16 14:48:33 +03:00
Ilya Golshtein
7e97814859 cleanup, fixes, new submodules, ShellCommand, WriteBufferFromString 2020-12-16 14:48:33 +03:00