278 Commits

Author	SHA1	Message	Date
msaf1980	fda6ddeffa	cleanup StorageHDFS (unused variables)	2022-05-25 19:23:05 +05:00
taiyang-li	a7a816dcb6	fix build error	2022-05-25 19:55:11 +08:00
taiyang-li	1d9f65a7d4	Merge branch 'master' into async_hdfs_read_buffer	2022-05-25 17:10:22 +08:00
avogar	f782fa31c6	Merge branch 'master' of github.com:ClickHouse/ClickHouse into check-format-on-storage-creation	2022-05-25 08:42:54 +00:00
Nikolai Kochetov	3d84aae0ab	Better.	2022-05-24 20:06:08 +00:00
avogar	37b66c8a9e	Check format name on storage creation	2022-05-23 12:48:48 +00:00
taiyang-li	29e2157469	change as request	2022-05-23 18:42:54 +08:00
Kruglov Pavel	f539fb835d	Merge branch 'master' into formats-with-names	2022-05-23 12:14:20 +02:00
Nikolai Kochetov	56feef01e7	Move some resources	2022-05-20 19:49:31 +00:00
taiyang-li	14ab7eb5a3	merge master and solve conflict	2022-05-17 16:28:08 +08:00
avogar	2d4b4b9008	Fix inserting defaults for missing values in columnar formats	2022-05-16 14:19:44 +00:00
avogar	68bb07d166	Better naming	2022-05-13 18:39:19 +00:00
avogar	b17fec659a	Improve performance and memory usage for select of subset of columns for some formats	2022-05-13 13:51:28 +00:00
wuxiaobai24	f3e573bfde	Merge remote-tracking branch 'upstream/master' into zstd_window_log_max	2022-05-13 20:42:54 +08:00
wuxiaobai24	ab5636a46a	fix	2022-05-13 17:52:39 +08:00
wuxiaobai24	58755cb156	add zstd_window_log_max setting	2022-05-08 14:02:40 +08:00
avogar	02bd5f6542	Finalize write buffers in case of exception	2022-05-06 17:30:18 +00:00
Robert Schulze	e583099158	Fix build, pt. V	2022-05-04 15:50:52 +02:00
mergify[bot]	64084b5e32	Merge branch 'master' into shared_ptr_helper3	2022-05-03 20:46:16 +00:00
Dmitry Novik	5ba7a55c18	Merge pull request #36650 from bigo-sg/hive_text_parallel_parsing ... Parallel parsing of hive text format	2022-05-03 15:56:28 +02:00
Robert Schulze	777b5bc15b	Don't let storages inherit from boost::noncopyable ... ... IStorage has deleted copy ctor / assignment already	2022-05-03 09:07:08 +02:00
Robert Schulze	330212e0f4	Remove inherited create() method + disallow copying ... The original motivation for this commit was that shared_ptr_helper used std::shared_ptr<>() which does two heap allocations instead of make_shared<>() which does a single allocation. Turned out that 1. the affected code (--> Storages/) is not on a hot path (rendering the performance argument moot ...) 2. yet copying Storage objects is potentially dangerous and was previously allowed. Hence, this change - removes shared_ptr_helper and as a result all inherited create() methods, - instead, Storage objects are now created using make_shared<>() by the caller (for that to work, many constructors had to be made public), and - all Storage classes were marked as noncopyable using boost::noncopyable. In sum, we are (likely) not making things faster but the code becomes cleaner and harder to misuse.	2022-05-02 08:46:52 +02:00
Amos Bird	4a5e4274f0	base should not depend on Common	2022-04-29 10:26:35 +08:00
kssenii	9d364cdce2	Refactor	2022-04-26 15:33:53 +02:00
taiyang-li	b7cc344d62	remove useless codes	2022-04-26 14:42:43 +08:00
taiyang-li	99dee35b6e	parallel parsing of hive text format	2022-04-26 14:33:10 +08:00
Kruglov Pavel	34c342fdd3	Merge pull request #36205 from Avogar/improve-globs ... Some refactoring around schema inference with globs	2022-04-25 13:14:46 +02:00
Amos Bird	e73d7fea31	Rename narrowBlockInputStream to narrowPipe	2022-04-24 18:33:48 +08:00
taiyang-li	f040c8ae65	merge master and solve conflict	2022-04-24 11:42:36 +08:00
taiyang-li	957f3f13f7	fix build error	2022-04-21 20:19:29 +08:00
taiyang-li	ad9e683356	fix code syle	2022-04-21 18:12:15 +08:00
taiyang-li	94d0358b15	fix code style	2022-04-21 17:40:55 +08:00
taiyang-li	169dae2a35	ready for review	2022-04-21 17:37:12 +08:00
taiyang-li	fb34a4ec01	ready for review	2022-04-21 17:27:41 +08:00
taiyang-li	1ea96a8732	merge master and solve conflict	2022-04-21 17:16:07 +08:00
Kruglov Pavel	813e228fcc	Merge branch 'master' into improve-globs	2022-04-20 16:31:47 +02:00
taiyang-li	3b722eea7a	profileing	2022-04-20 20:59:36 +08:00
taiyang-li	d533b569ad	debugging	2022-04-20 19:58:31 +08:00
taiyang-li	56fe6fa608	finish dev	2022-04-20 17:49:53 +08:00
taiyang-li	fb6a56d4b0	finish debug	2022-04-20 16:24:18 +08:00
avogar	1f252cedfe	Make better	2022-04-19 19:16:47 +00:00
Anton Popov	bee4ca9b62	add more tests for error diagnostics in files	2022-04-19 15:56:34 +00:00
taiyang-li	e2c51d581c	Merge branch 'fix_hdfs_seek' into async_hdfs_read_buffer	2022-04-19 15:50:01 +08:00
zhanglistar	5670dcf750	Fix crash in debug mode of ReadBufferFromHDFS.	2022-04-18 17:52:59 +08:00
taiyang-li	27fa2a019a	commit again	2022-04-18 15:56:49 +08:00
taiyang-li	cd83fd5f8a	tobe debug	2022-04-16 18:41:18 +08:00
taiyang-li	f20e80c318	wip: deving	2022-04-16 16:19:48 +08:00
zhanglistar	fe93fd83f5	Merge branch 'ClickHouse:master' into fixhdfsreadbuffer	2022-04-15 16:27:20 +08:00
zhanglistar	e22a175afb	Fix ReadBufferFromHDFS crash in debug mode	2022-04-15 14:04:30 +08:00
avogar	1c065f8c7a	Some refactoring around schema inference with globs	2022-04-13 17:02:48 +00:00
taiyang-li	b4cd8561a5	wip	2022-04-13 16:06:19 +08:00
taiyang-li	9d31c44d39	fix bug of read buffer from hdfs	2022-04-12 12:01:51 +08:00
lgbo-ustc	759e4a9a05	Merge remote-tracking branch 'ck/master' into s3_cluster_join	2022-04-08 09:25:30 +08:00
shuchaome	a565a93740	reduce mutex scope when setenv LIBHDFS3_CONF	2022-04-02 06:52:39 +08:00
Alexey Milovidov	5c6b84c398	Merge pull request #35586 from kssenii/fix-virtual-columns-s3 ... Fix s3 engine getting virtual columns	2022-04-01 04:43:32 +03:00
Nikita Mikhaylov	5f27f690f4	Revert "Added support for schema inference for hdfsCluster (#35602)" ... This reverts commit f6bfdcc0c9.	2022-03-31 14:19:56 +02:00
Nikita Mikhaylov	f6bfdcc0c9	Added support for schema inference for hdfsCluster (#35602)	2022-03-31 12:47:36 +02:00
lgbo-ustc	08e1fdc6b8	Fixed :a elimate AMBIGUOUS_COLUMN_NAME exception ... elimate AMBIGUOUS_COLUMN_NAME exception in hdfs/s3 cluster when run a query with join	2022-03-30 17:06:38 +08:00
kssenii	4374415bd7	Fix clang-tidy, style check	2022-03-29 14:20:21 +02:00
kssenii	a2013ebe2b	Better	2022-03-28 22:40:27 +02:00
taiyang-li	fbadd8fc0e	only deduce absolute path of hdfs config when it's relative path do not exists	2022-03-25 22:28:47 +08:00
taiyang-li	c0eb9f73a9	remote std::cout	2022-03-24 21:17:43 +08:00
taiyang-li	6eccd09e2b	deduce hdfs config path	2022-03-24 18:04:29 +08:00
Anton Popov	36ec379aeb	Merge remote-tracking branch 'upstream/master' into HEAD	2022-03-14 16:28:35 +00:00
kssenii	5260822964	Merge master	2022-03-08 18:21:28 +01:00
Anton Popov	df3b07fe7c	Merge remote-tracking branch 'upstream/master' into HEAD	2022-03-03 22:25:28 +00:00
kssenii	a594f388a4	Merge master	2022-03-01 19:43:45 +01:00
Anton Popov	c1fdcf7a64	Merge remote-tracking branch 'upstream/master' into HEAD	2022-03-01 20:21:39 +03:00
kssenii	092ec45b47	Merge master	2022-03-01 12:06:56 +01:00
kssenii	9b64a8fe39	Fix odbc bridge	2022-02-28 14:29:05 +01:00
Hongbin	c9bc442114	fix comments	2022-02-28 16:44:35 +08:00
Hongbin	99bd56e2de	Fix some code comments style	2022-02-28 08:15:37 +08:00
Anton Popov	fcdebea925	Merge remote-tracking branch 'upstream/master' into HEAD	2022-02-25 13:41:30 +03:00
kssenii	c637385dd0	Merge master	2022-02-22 13:17:51 +01:00
kssenii	47f94120da	Merge master	2022-02-16 14:43:28 +01:00
kssenii	3bd3e51aa0	Fix tests	2022-02-16 14:08:41 +01:00
Kruglov Pavel	dd863ca2a0	Merge branch 'master' into fix-url-globs	2022-02-16 12:45:31 +03:00
alesapin	bc2d0ee7c7	Merge pull request #34215 from ClickHouse/revert-34211-revert-34153-add_func_tests_over_s3 ... Add func tests run with s3 and fix several bugs	2022-02-15 19:07:11 +03:00
alesapin	89373155fc	Merge branch 'master' into revert-34211-revert-34153-add_func_tests_over_s3	2022-02-13 21:07:54 +03:00
mergify[bot]	8a47ed2b3f	Merge branch 'master' into static-files-disk-uploader-create-symlinks	2022-02-13 06:12:39 +00:00
Dmitry Novik	50131fd732	Fix cancelation for S3 and HDFS	2022-02-11 18:14:55 +00:00
alesapin	088f8cacf3	Small improvements	2022-02-10 19:50:21 +03:00
Anton Popov	298838f891	avoid unnecessary copying of Settings	2022-02-10 12:13:51 +03:00
Anton Popov	18940b8637	Merge remote-tracking branch 'upstream/master' into HEAD	2022-02-09 23:38:38 +03:00
avogar	84087c1554	Improve schema inference with globs	2022-02-09 19:16:15 +03:00
Alexey Milovidov	742620209a	static-files-disk-uploader: add mode to create symlinks	2022-02-09 03:50:50 +03:00
alexey-milovidov	43ee8ddb5b	Merge pull request #34376 from CurtizJ/fix-reading-empty-arrays ... Fix consecutive backward seeks in seekable read buffers	2022-02-08 02:27:57 +03:00
Anton Popov	1b16db72c3	fix consecutive backward seeks in seekable read buffers	2022-02-07 17:20:26 +03:00
feng lv	6325d4d9b0	continue of #34317 ... fix fix	2022-02-06 08:59:17 +00:00
Anton Popov	836a348a9c	Merge remote-tracking branch 'upstream/master' into HEAD	2022-02-01 15:23:07 +03:00
alesapin	5230c59f05	Smaller scope	2022-01-31 14:03:06 +03:00
alesapin	1adc7538fe	Fix benign race condition for storage HDFS, S3, URL	2022-01-31 13:50:28 +03:00
Anton Popov	78b9f15abb	Merge remote-tracking branch 'upstream/master' into HEAD	2022-01-30 03:24:37 +03:00
Anton Popov	e8ce091e68	Merge remote-tracking branch 'upstream/master' into HEAD	2022-01-21 20:11:18 +03:00
Kruglov Pavel	7bfb1231b9	Merge branch 'master' into formats-with-suffixes	2022-01-20 14:47:17 +03:00
alexey-milovidov	28a9d569cc	Merge pull request #33690 from azat/remove-unbundled-support ... Remove unbundled support	2022-01-20 12:29:07 +03:00
liuneng1994	59f3fe515b	change null check position ... Signed-off-by: neng.liu <neng.liu@kyligence.io>	2022-01-20 06:26:34 +00:00
Azat Khuzhin	aef6668cb4	Remove unbundled hdfs3 support	2022-01-20 08:47:16 +03:00
liuneng1994	4d3bb1584d	fix Dereference of null pointer error ... Signed-off-by: neng.liu <neng.liu@kyligence.io>	2022-01-19 07:41:46 +00:00
avogar	4efadfad3c	Fix tests	2022-01-18 22:26:13 +03:00