429 Commits

Author	SHA1	Message	Date
zvonand	70aa6e2672	Added a shortcut for no-wildcard globs in HDFS	2023-11-15 16:37:24 +01:00
zvonand	3e47a54a94	Added a shortcut for no-wildcard globs in file storage	2023-11-15 16:37:24 +01:00
Kruglov Pavel	570b66f027	Merge branch 'master' into schema-inference-union	2023-10-26 19:26:00 +02:00
李扬	465962df7f	Support orc filter push down (file + stripe + rowgroup level) (#55330) ... * support orc filter push down * update orc lib version * replace setqueryinfo with setkeycondition * fix issue https://github.com/ClickHouse/ClickHouse/issues/53536 * refactor source with key condition * fix building error * remove std::cout * update orc * update orc version * fix bugs * improve code * upgrade orc lib * fix code style * change as requested * add performance tests for orc filter push down * add performance tests for orc filter push down * fix all bugs * fix default as null issue * add uts for null as default issues * upgrade orc lib * fix failed orc lib uts and fix typo * fix failed uts * fix failed uts * fix ast fuzzer tests * fix bug of uint64 overflow in https://s3.amazonaws.com/clickhouse-test-reports/55330/de22fdcaea2e12c96f300e95f59beba84401712d/fuzzer_astfuzzerubsan/report.html * fix asan fatal caused by reused column vector batch in native orc input format. refer to https://s3.amazonaws.com/clickhouse-test-reports/55330/be39d23af2d7e27f5ec7f168947cf75aeaabf674/stateless_tests__asan__[4_4].htm * fix wrong performance tests * disable 02892_orc_filter_pushdown on aarch64. https://s3.amazonaws.com/clickhouse-test-reports/55330/be39d23af2d7e27f5ec7f168947cf75aeaabf674/stateless_tests__aarch64_.html * add some comments * add some comments * inline range::equals and range::less * fix data race of key condition * trigger ci	2023-10-24 12:08:17 -07:00
avogar	6934e27e8b	Add union mode for schema inference to infer union schema of files with different schemas	2023-10-20 20:46:41 +00:00
Michael Kolupaev	ce7eca0615	DWARF input format (#55450) ... * Add ReadBufferFromFileBase::isRegularLocalFile() * DWARF input format * Review comments * Changed things around ENABLE_EMBEDDED_COMPILER build setting * Added 'ranges' column * no-msan no-ubsan	2023-10-16 17:00:07 -07:00
robot-clickhouse-ci-2	d98234dc9d	Merge pull request #54803 from Avogar/ephemeral-columns-from-files ... Forbid special columns for file/s3/url/... storages, fix insert into ephemeral columns from files	2023-09-22 23:24:42 +02:00
zvonand	2eb63649bd	fix old workflows	2023-09-22 00:30:46 +02:00
zvonand	bc5f141a6c	fix {..} for File	2023-09-21 16:40:38 +02:00
zvonand	a05bb020d4	do all the same for HDFS + remove setting	2023-09-21 01:50:41 +02:00
zvonand	5f67788c03	update docs and comments	2023-09-21 00:16:17 +02:00
zvonand	9eccb12183	small refactor of storageFile globs	2023-09-20 23:59:10 +02:00
avogar	3e08800cb5	Forbid special columns for file/s3/url/... storages, fix insert into ephemeral columns from files	2023-09-20 16:25:55 +00:00
Robert Schulze	f5e8028bb1	Merge pull request #54642 from rschu1ze/broken-re2st ... Remove broken lockless variant of re2	2023-09-17 15:30:57 +02:00
Robert Schulze	7b378dbad3	Remove broken lockless variant of re2	2023-09-14 16:40:42 +00:00
Andrey Zvonov	ff16ddbbb3	Merge branch 'ClickHouse:master' into zvonand-globs-small-fix	2023-09-14 11:31:49 +03:00
avogar	2d8f33bfa2	Fix parsing error in WithNames formats while reading subset of columns with disabled input_format_with_names_use_header	2023-09-11 14:55:37 +00:00
Andrey Zvonov	ff4586e663	Merge branch 'ClickHouse:master' into zvonand-globs-small-fix	2023-09-08 18:17:06 +02:00
zvonand	c36a3fbd39	renamed setting	2023-09-07 12:00:25 +03:00
Kruglov Pavel	de801ce563	Merge pull request #54293 from ClickHouse/pasha-returned-from-vacation ... Code improvement for reading from archives	2023-09-05 21:09:09 +02:00
Antonio Andelic	88930a335c	Apply comments	2023-09-05 12:32:07 +00:00
Antonio Andelic	a51c333c5f	Improve code for reading from archives	2023-09-05 08:37:02 +00:00
avogar	e9456082c1	Fix using count from cache and improve progress bar for reading from archives	2023-09-04 17:33:59 +00:00
Andrey Zvonov	83405a8808	Merge branch 'master' into zvonand-globs-small-fix	2023-08-31 08:42:48 +02:00
Antonio Andelic	f406019413	Apply PR comments	2023-08-30 09:26:01 +00:00
Antonio Andelic	a0dd350e5e	Fix assert	2023-08-29 09:59:16 +00:00
Antonio Andelic	8708ebd7fb	Correctly check files	2023-08-29 07:40:08 +00:00
Antonio Andelic	9b99f25d75	Improve schema inference	2023-08-28 13:11:52 +00:00
Antonio Andelic	5a0c2ca108	Merge branch 'master' into archive-improvements-2	2023-08-28 08:34:42 +00:00
Andrey Zvonov	911bf94fbd	Merge branch 'master' into zvonand-globs-small-fix	2023-08-25 23:50:07 +05:00
Antonio Andelic	8e1d38d377	Merge branch 'master' into archive-improvements-2	2023-08-24 13:03:36 +00:00
Kruglov Pavel	f7e1abd774	Merge branch 'master' into cache-count	2023-08-23 22:31:49 +02:00
Kruglov Pavel	592fa77987	Merge branch 'master' into cache-count	2023-08-23 15:18:02 +02:00
Kruglov Pavel	7e362a2110	Merge branch 'master' into fast-count-from-files	2023-08-23 15:13:20 +02:00
robot-ch-test-poll1	c22ffa6195	Merge pull request #53529 from Avogar/filter-files-all-table-functions ... Use filter by file/path before reading in url/file/hdfs table functins	2023-08-23 14:21:23 +02:00
Antonio Andelic	83d4b819f3	Better support for reading from archives	2023-08-23 08:10:30 +00:00
Kruglov Pavel	67c5c0203b	Merge branch 'master' into fast-count-from-files	2023-08-22 15:03:48 +02:00
Kruglov Pavel	c0bdd0e00b	Merge branch 'master' into cache-count	2023-08-22 14:42:22 +02:00
avogar	b4145aeddc	Cache number of rows in files for count in file/s3/url/hdfs/azure functions	2023-08-22 11:59:59 +00:00
Michael Kolupaev	2f4d433e69	Parquet filter pushdown	2023-08-21 14:15:52 -07:00
Kruglov Pavel	aebbf00a5a	Fix	2023-08-21 17:47:30 +02:00
Kruglov Pavel	a2f81bf4ea	Fix typo	2023-08-21 15:20:44 +02:00
Kruglov Pavel	dfdd26ee71	Fix build	2023-08-21 15:04:03 +02:00
Kruglov Pavel	88aee95122	Merge branch 'master' into fast-count-from-files	2023-08-21 14:46:33 +02:00
avogar	584bd57f5c	Clean up	2023-08-21 12:43:11 +00:00
avogar	47304bf7aa	Optimize count from files in most input formats	2023-08-21 12:30:52 +00:00
Kruglov Pavel	9d9bdde0e3	Merge branch 'master' into filter-files-all-table-functions	2023-08-21 14:26:59 +02:00
avogar	8e445b5461	Fixes	2023-08-18 17:49:40 +00:00
avogar	4c32097df3	Use filter by file/path before reading in url/file/hdfs table functions, reduce code duplication	2023-08-17 16:54:43 +00:00
Andrey Zvonov	34b1811a53	Update src/Storages/StorageFile.cpp ... seems like typo Co-authored-by: SmitaRKulkarni <64093672+SmitaRKulkarni@users.noreply.github.com>	2023-08-15 19:10:14 +03:00
zvonand	086bf6cb8c	added ignore_eacces_multidirectory_globs	2023-08-11 15:49:21 +03:00
Andrey Zvonov	90dcc2ac81	Merge branch 'master' into zvonand-globs-small-fix	2023-08-10 12:30:12 +02:00
Antonio Andelic	c65f3d1eb8	Mutex for name filter	2023-08-10 10:09:59 +00:00
Alexey Milovidov	964c407d48	Revert "Revert "Upload build time-trace data to CI database""	2023-08-09 16:11:17 +03:00
Antonio Andelic	0e17d26b88	More formats supported, read single archive from 1 thread	2023-08-09 11:58:37 +00:00
Alexander Gololobov	63183c56f1	Revert "Upload build time-trace data to CI database"	2023-08-09 13:58:05 +02:00
Raúl Marín	30d1452bbc	Compatibility with clang-17	2023-08-07 10:48:24 +02:00
Antonio Andelic	cafa5ef2f3	Merge branch 'master' into add-reading-from-archives	2023-08-06 10:21:47 +00:00
Antonio Andelic	1d793e69f2	Fix	2023-08-05 17:06:44 +00:00
Antonio Andelic	d2b6646fc2	Merge branch 'master' into add-reading-from-archives	2023-08-04 12:42:46 +00:00
Antonio Andelic	d9f136ce0c	Add tests for globs	2023-08-03 10:42:47 +00:00
zvonand	8f080a17de	Merge branch 'zvonand-globs-small-fix' of github.com:zvonand/ClickHouse into zvonand-globs-small-fix	2023-08-02 15:51:48 +02:00
zvonand	c0f9dbbacb	shorten	2023-08-02 15:51:27 +02:00
Antonio Andelic	9423976b7a	Add support for file globs	2023-08-02 07:42:56 +00:00
Antonio Andelic	c24dc8e64e	Multithreaded read of files from archives:	2023-08-01 13:48:49 +00:00
Andrey Zvonov	791c5b1393	Merge branch 'master' into zvonand-globs-small-fix	2023-08-01 14:00:57 +02:00
zvonand	e851be78bc	fix fs_error on attempt to read unavailable dirs	2023-07-31 23:56:07 +02:00
Antonio Andelic	9fb86f134b	Fix tests	2023-07-31 12:04:27 +00:00
Antonio Andelic	df45602165	Refactoring	2023-07-31 08:50:09 +00:00
Antonio Andelic	e83e0ec2cd	Fix build	2023-07-28 12:26:56 +00:00
Antonio Andelic	720d587e85	Merge branch 'master' into add-reading-from-archives	2023-07-28 08:49:00 +00:00
Kruglov Pavel	0d34e97dbe	Merge branch 'master' into formats-with-subcolumns	2023-07-26 13:30:35 +02:00
Kruglov Pavel	15cc046883	Merge branch 'master' into better-progress-bar-2	2023-07-26 13:12:24 +02:00
Alexey Milovidov	21382afa2b	Check for punctuation	2023-07-25 06:10:04 +02:00
Kruglov Pavel	fec5675cd4	Merge branch 'master' into better-progress-bar-2	2023-07-24 19:59:38 +02:00
Kruglov Pavel	64e88cde21	Merge branch 'master' into better-progress-bar-2	2023-07-18 13:37:53 +02:00
Kruglov Pavel	1dd05319b5	Merge branch 'master' into formats-with-subcolumns	2023-07-17 19:13:42 +02:00
Andrey Zvonov	c030e9dbe4	Merge branch 'master' into zvonand-issue-49290	2023-07-10 11:01:26 +02:00
Alexey Milovidov	1a24bface0	Merge branch 'master' into clickhouse-local-default-settings	2023-07-06 03:30:11 +03:00
avogar	d11cd0dc30	Fix tests	2023-07-05 17:56:03 +00:00
Kruglov Pavel	a2805f8f44	Merge branch 'master' into formats-with-subcolumns	2023-07-04 23:27:03 +02:00
avogar	98aa6b317f	Support reading subcolumns from file/s3/hdfs/url/azureBlobStorage table functions	2023-07-04 21:17:26 +00:00
avogar	f88c1fdf44	Fixes	2023-06-29 11:59:09 +00:00
avogar	4eeb431003	Merge branch 'master' of github.com:ClickHouse/ClickHouse into better-progress-bar-2	2023-06-28 18:53:08 +00:00
Andrey Zvonov	ae26ff27a1	Merge branch 'master' into zvonand-issue-49290	2023-06-28 12:13:05 +02:00
Alexey Milovidov	b8e6bd3299	Merge branch 'master' into refactor-subqueries-for-in	2023-06-26 06:05:12 +03:00
Azat Khuzhin	a7b14f87e0	Throw an error instead of silenty ignore storage_file_read_method=mmap in server ... Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>	2023-06-24 23:42:31 +03:00
Andrey Zvonov	0b6688ffe4	Merge branch 'master' into zvonand-issue-49290	2023-06-24 12:36:27 +02:00
avogar	c679dd400e	Make better	2023-06-23 13:43:40 +00:00
avogar	24fab7bfde	Remove old includes	2023-06-22 18:48:15 +00:00
avogar	cf082f2f9a	Use read_bytes/total_bytes_to_read for progress bar in s3/file/url/... table functions	2023-06-22 17:24:43 +00:00
Andrey Zvonov	18e7b029d5	Merge branch 'master' into zvonand-issue-49290	2023-06-22 17:20:24 +02:00
Sema Checherinda	d0bb985061	fix other classes based on SinkToStorage	2023-06-22 14:33:25 +02:00
Sema Checherinda	95349a405b	release buffers with exception context	2023-06-22 13:00:13 +02:00
Nikolai Kochetov	a940031878	Merge branch 'master' into refactor-subqueries-for-in	2023-06-22 12:18:48 +02:00
zvonand	b16c30e521	upd for review	2023-06-19 23:06:08 +02:00
avogar	d492acbcd2	Fix tests	2023-06-19 13:36:29 +00:00
Nikolai Kochetov	afa74f697c	Refactor a bit.	2023-06-16 19:38:50 +00:00
zvonand	1c10578f14	update to master	2023-06-16 21:30:47 +02:00
avogar	56789af0fa	Remove loggig	2023-06-16 15:55:05 +00:00