44 Commits

Author	SHA1	Message	Date
Michael Kolupaev	7271cfd187	Prevent parquet schema inference reading the first 1 MB of the file unnecessarily	2023-09-19 21:58:12 +00:00
Antonio Andelic	88930a335c	Apply comments	2023-09-05 12:32:07 +00:00
Kruglov Pavel	c0bdd0e00b	Merge branch 'master' into cache-count	2023-08-22 14:42:22 +02:00
avogar	b4145aeddc	Cache number of rows in files for count in file/s3/url/hdfs/azure functions	2023-08-22 11:59:59 +00:00
Alexey Milovidov	29221188ba	Fix error	2023-08-09 04:07:31 +02:00
Alexey Milovidov	5dd99db369	Add diagnostic info about file name during schema inference	2023-08-08 03:55:06 +02:00
avogar	33e51d4f3b	Add setting to limit the number of bytes to read in schema inference	2023-06-05 15:22:04 +00:00
Kruglov Pavel	bacba6e347	Fix typo	2023-04-26 12:18:12 +02:00
Kruglov Pavel	f087f0e877	Update src/Formats/ReadSchemaUtils.cpp	2023-04-11 14:18:16 +02:00
laimuxi	b869572a54	reformat code	2023-04-01 15:20:26 +08:00
laimuxi	3b756ef026	rollback	2023-03-31 21:58:20 +08:00
laimuxi	17efdbf625	change	2023-03-31 21:56:35 +08:00
Alexander Tokmakov	70d1adfe4b	Better formatting for exception messages (#45449) ... * save format string for NetException * format exceptions * format exceptions 2 * format exceptions 3 * format exceptions 4 * format exceptions 5 * format exceptions 6 * fix * format exceptions 7 * format exceptions 8 * Update MergeTreeIndexGin.cpp * Update AggregateFunctionMap.cpp * Update AggregateFunctionMap.cpp * fix	2023-01-24 00:13:58 +03:00
avogar	7375a7d429	Refactor and improve schema inference for text formats	2022-12-07 21:19:27 +00:00
Alexey Milovidov	2f0684b97c	Fix trash in schema inference	2022-09-17 23:11:33 +02:00
Alexey Milovidov	47167494d9	Fix trash in schema inference	2022-09-17 22:53:41 +02:00
avogar	5ab87f1da4	Small refactoring	2022-08-19 16:42:23 +00:00
avogar	8dd54c043d	Merge branch 'master' of github.com:ClickHouse/ClickHouse into schema-inference-cache	2022-08-17 11:47:40 +00:00
avogar	9b1a267203	Refactor, remove TTL, add size limit, add system table and system query	2022-08-05 16:20:15 +00:00
Kruglov Pavel	46da17ca8c	Merge branch 'master' into numbers-schema-inference	2022-07-20 13:32:39 +02:00
Kruglov Pavel	3046cd6d29	Merge branch 'master' into schema-inference-cache	2022-07-20 13:30:42 +02:00
Kruglov Pavel	88d59520a2	Fix	2022-07-19 15:20:56 +02:00
avogar	2367f40b70	Better exception messages in schema inference	2022-07-18 15:36:33 +00:00
avogar	7cde9d3b40	Add new features in schema inference	2022-07-13 15:57:55 +00:00
avogar	ee54c4f9b7	Add some fixes and add settings in docs	2022-06-30 12:41:56 +00:00
avogar	b0c9d1a25d	Fix style	2022-06-27 14:04:28 +00:00
avogar	5155262a16	Add some additional information to cache keys	2022-06-27 12:43:24 +00:00
avogar	d37ad2e6de	Implement cache for schema inference for file/s3/hdfs/url	2022-06-21 13:02:48 +00:00
Kruglov Pavel	6f17ba17ba	Revert "Revert "Fix possible segfault in schema inference""	2022-06-02 13:28:27 +02:00
Alexander Tokmakov	4baae59252	Revert "Fix possible segfault in schema inference"	2022-06-02 14:04:28 +03:00
avogar	4abfd54dd6	Fix possible segfault in schema inference	2022-06-01 16:53:37 +00:00
avogar	cd6a29897e	Apply input_format_max_rows_to_read_for_schema_inference for all files in globs in total	2022-05-18 17:56:36 +00:00
avogar	b666b4e1c9	Fix possible heap-use-after-free in schema inference	2022-04-26 14:36:16 +00:00
Kruglov Pavel	813e228fcc	Merge branch 'master' into improve-globs	2022-04-20 16:31:47 +02:00
avogar	f31f019252	Fix	2022-04-19 19:25:41 +00:00
avogar	1f252cedfe	Make better	2022-04-19 19:16:47 +00:00
avogar	ae88549c4f	Allow insert select for files with formats without schema inference	2022-04-13 20:02:52 +00:00
avogar	1c065f8c7a	Some refactoring around schema inference with globs	2022-04-13 17:02:48 +00:00
mergify[bot]	343588de2c	Merge branch 'master' into improve-schema-inference	2022-03-29 13:06:00 +00:00
Anton Popov	67195bfdd5	support schema inference for type Object in format JSONEachRow	2022-03-25 21:51:53 +00:00
avogar	557edbd172	Add some improvements and fixes in schema inference	2022-03-24 12:54:12 +00:00
avogar	a6740d2f9a	Detect format and schema for stdin in clickhouse-local	2022-01-25 10:25:37 +03:00
avogar	aaf9f85c67	Add more tests and fixes	2021-12-29 12:18:56 +03:00
avogar	8112a71233	Implement schema inference for most input formats	2021-12-29 12:18:56 +03:00