1180 Commits

Author	SHA1	Message	Date
FawnD2	182de47012	Use file reader for Arrow format	2020-05-04 17:30:42 +03:00
Anton Okhitin	7366227115	Fix arcadia build	2020-05-04 16:47:08 +03:00
FawnD2	6d6d04903f	Remove check	2020-05-04 16:33:20 +03:00
FawnD2	a044766d58	Remove check	2020-05-04 16:26:20 +03:00
FawnD2	17450811d4	Move getHeader at the beginning of generate()	2020-05-04 16:19:25 +03:00
FawnD2	83148289d8	Rename arg	2020-05-04 16:01:37 +03:00
FawnD2	7b23844b32	Common order of precompiled checks	2020-05-04 03:58:10 +03:00
FawnD2	02e12215e7	Apply reducing memory usage optimization for seekable files to ORC format	2020-05-04 03:52:28 +03:00
FawnD2	a554177724	Simplify ORC format	2020-05-04 02:23:20 +03:00
FawnD2	112758b99d	Merge branch 'master' into arrow-io-format	2020-05-04 00:53:17 +03:00
FawnD2	2359ba3497	Correct stop condition	2020-05-03 22:59:19 +03:00
FawnD2	6b2e97fd56	Increate total read length while reading	2020-05-03 22:54:28 +03:00
FawnD2	b4898bdd19	Read single batch	2020-05-03 22:49:06 +03:00
FawnD2	a8d914a811	Read full table	2020-05-03 22:22:41 +03:00
FawnD2	b86562b8bc	Style fixes	2020-05-03 21:29:04 +03:00
FawnD2	7cc7a87f9f	Simplify interfaces	2020-05-03 21:12:14 +03:00
FawnD2	a590826fbb	Format settings for Arrow	2020-05-03 15:26:39 +03:00
FawnD2	e5b59616cd	Style fix	2020-05-03 15:25:53 +03:00
Alexey Milovidov	469cdc315b	Added suggestion from @azat #10607	2020-05-03 14:40:37 +03:00
FawnD2	f62f1a5924	Add missed include	2020-05-03 06:08:37 +03:00
FawnD2	6e0b31fb2a	Arrow output format	2020-05-03 05:46:36 +03:00
FawnD2	aeed62d5e7	Remove unused error codes	2020-05-03 04:20:03 +03:00
FawnD2	4d0225f5a9	std::string -> String	2020-05-03 04:15:09 +03:00
FawnD2	f4fe23e3fe	Add ptr wrapper. Fix style	2020-05-03 04:01:05 +03:00
FawnD2	fa1c35753b	Move CH -> Arrow routine converter into separated class	2020-05-03 03:54:39 +03:00
Alexey Milovidov	554e7a0dd2	Preparations	2020-05-02 22:59:07 +03:00
Alexey Milovidov	e6ab4d655b	Fix bad code	2020-05-02 22:54:29 +03:00
Alexey Milovidov	f0e80fec68	Whitespaces	2020-05-02 22:41:01 +03:00
alexey-milovidov	330627f597	Update ArrowBlockInputFormat.h	2020-05-02 22:35:58 +03:00
alexey-milovidov	47e3f83bc8	Update ArrowBlockInputFormat.h	2020-05-02 22:34:34 +03:00
alexey-milovidov	07f824c746	Update ArrowBlockInputFormat.cpp	2020-05-02 22:33:47 +03:00
alexey-milovidov	dfeb76038d	Update ArrowBlockInputFormat.h	2020-05-02 22:32:33 +03:00
alexey-milovidov	fbb280b5ac	Update ArrowBlockInputFormat.cpp	2020-05-02 22:32:21 +03:00
alexey-milovidov	250a44a47f	Merge pull request #10607 from Avogar/json_as_string_format ... Add JSONAsStringRowInputFormat	2020-05-02 22:17:22 +03:00
Matwey V. Kornilov	d7627fa4fb	Fix capnproto version check for capnp::UnalignedFlatArrayMessageReader ... capnp::UnalignedFlatArrayMessageReader was introduced in 3aa2b2aa02 (which is a part of 0.7.0 release). Unfortunately, capnp::UnalignedFlatArrayMessageReader was removed in 3f0fee61c6 (which is a part of 0.8.0 release) So change CAPNP_VERSION check accordingly.	2020-05-01 22:32:06 +03:00
Avogar	697d5b07ce	Fix build and style errors. Update test.	2020-05-01 19:41:14 +03:00
Avogar	6464a1a902	Add JSONAsStringRowInputFormat	2020-04-30 23:16:09 +03:00
Nikolai Kochetov	10afbc22b5	Fix build.	2020-04-30 16:36:20 +03:00
Nikolai Kochetov	62e9be7ba8	Fix build.	2020-04-30 14:33:27 +03:00
Nikolai Kochetov	19dadb8c2d	Add parallel final.	2020-04-30 12:59:08 +03:00
FawnD2	538e6c39da	Arrow input format	2020-04-28 22:52:22 +03:00
Nikolai Kochetov	de6d2d01b9	Try to fix tests.	2020-04-27 21:15:55 +03:00
Nikolai Kochetov	2f06180c5e	Revert changes for CSVRowOutputFormat.	2020-04-27 18:21:53 +03:00
Nikolai Kochetov	201a184d31	Enable auto flust for output formats if enable_streaming was set.	2020-04-27 18:00:10 +03:00
Nikolai Kochetov	32baa073b5	Merge remote-tracking branch 'upstream/master' into insert_into_watch_from_live_view	2020-04-27 17:49:50 +03:00
alexey-milovidov	436e4e3f5e	Merge pull request #10502 from oandrew/avro-nested2 ... Avro nested fields - support complex types	2020-04-27 13:44:58 +03:00
Andrew Onyshchuk	570be94a48	Use correct type for DateTime64 (ubsan)	2020-04-26 21:10:30 -05:00
alexey-milovidov	c6ae6a7ba8	Merge pull request #10418 from ClickHouse/fix_logical_error_in_diagnostic_info ... Better diagnostic info in input formats	2020-04-26 20:26:59 +03:00
alexey-milovidov	d1464a441f	Merge pull request #10503 from oandrew/read-columns-perf-fix ... IRowInputFormat performance fix	2020-04-26 17:36:54 +03:00
Vitaliy Zakaznikov	626468d255	Adding support for streaming mode to JSONEachRowRowOutputFormat	2020-04-26 15:56:45 +02:00
Vitaliy Zakaznikov	369b4d53ef	Adding support for output_format_enable_streaming format setting.	2020-04-26 15:44:11 +02:00
Dmitry	d9ac0e8fef	Merge branch 'master' of github.com:yandex/ClickHouse into group_by_in_order_optimization	2020-04-26 14:44:06 +03:00
Andrew Onyshchuk	4a5497498e	IRowInputFormat performance fix	2020-04-25 19:33:36 -05:00
Alexey Milovidov	d01bd44f14	Merge branch 'master' into execute-query-flush-format-2	2020-04-26 03:31:46 +03:00
alexey-milovidov	8820b43606	Merge pull request #10479 from Avogar/markdown_format ... Markdown format	2020-04-26 03:25:48 +03:00
Andrew Onyshchuk	e7ba75ce98	Avro nested fields - support complex types	2020-04-25 17:42:49 -05:00
Azat Khuzhin	40c65f7bee	IRowInputFormat: omit "Skipped 0 rows with errors while reading the input stream" ... This will just pollute the output log. And this is significant for kafka streaming, since it will print this message for each message (due to how ReadBufferFromKafkaConsumer/KafkaBlockInputStream works). You can see this in the server log of the test_storage_kafka (integration test), for the table with kafka_skip_broken_messages > 0, it will have 52 lines of such message, 2 with "Skipped 1 rows" and 50 with "Skipped 0 rows" (since there are 52 messages in summary, 2 malformed and 50 correct).	2020-04-25 15:16:29 +03:00
Alexander Tokmakov	53dcce55e9	better diagnostic info in input formats	2020-04-24 15:03:48 +03:00
Avogar	778f1aa1e7	Merge branch 'markdown_format' of github.com:avogar/ClickHouse into markdown_format	2020-04-24 14:07:04 +03:00
Avogar	2458da802c	Update MarkdownRowOutputFormat	2020-04-24 14:06:41 +03:00
Alexey Milovidov	d3418bbb02	Fix clang-tidy	2020-04-22 11:45:14 +03:00
Alexey Milovidov	03e17405cd	Checkpoint	2020-04-22 10:03:43 +03:00
Alexey Milovidov	603d132d45	Checkpoint	2020-04-22 09:34:20 +03:00
Alexey Milovidov	0a7edce036	Checkpoint	2020-04-22 09:01:33 +03:00
alexey-milovidov	7690c34e5b	Merge pull request #10424 from ClickHouse/tsv-bom-extra-case ... Allow to parse BOM in TSV if the first column cannot contain BOM in its value	2020-04-22 06:56:43 +03:00
Alexander Kuzmenkov	d1c5e38a21	Merge pull request #10385 from CurtizJ/tuple-literal ... Parse tuples of tuples as literals	2020-04-22 05:49:14 +03:00
Alexey Milovidov	e8a9c7d518	Allow to parse BOM in TSV if the first column cannot contain BOM in its value	2020-04-22 02:46:42 +03:00
alexey-milovidov	fcf5c3c6e1	Merge pull request #10399 from ClickHouse/fix-distributed_group_by_no_merge-segfault ... Fix segfault with distributed_group_by_no_merge	2020-04-22 01:28:02 +03:00
Ivan	555488a458	Fix build in Arcadia (#10413)	2020-04-21 23:23:44 +03:00
Anton Popov	3ccff833c1	fix ConstantExpressionTemplate with arrays	2020-04-21 15:54:36 +03:00
Nikolai Kochetov	1d2388de39	Drop currently calculated totals and extremes before aggregation.	2020-04-21 11:45:28 +03:00
alexey-milovidov	fe1309e575	Merge pull request #10381 from ClickHouse/fix-concat-prepare ... Fix concat prepare	2020-04-21 00:07:32 +03:00
alexey-milovidov	2c8576798b	Merge pull request #10354 from oandrew/avro-nested ... Avro nested fields insert support	2020-04-20 23:36:02 +03:00
alexey-milovidov	3d4696d4ef	Update AvroRowInputFormat.cpp	2020-04-20 23:34:48 +03:00
Nikolai Kochetov	3dbc9b5562	Fix prepare for ConcatProcessor.	2020-04-20 14:03:29 +03:00
Nikolai Kochetov	c2c99b02bc	Fix prepare for ConcatProcessor.	2020-04-20 13:36:38 +03:00
Alexey Milovidov	d7264b292d	Merge branch 'master' into sorting-processors	2020-04-20 09:29:41 +03:00
Alexey Milovidov	5262512f0c	Flush output for interactive applications	2020-04-19 22:02:36 +03:00
Andrew Onyshchuk	68f2e93443	Fix style	2020-04-18 22:54:22 -05:00
Andrew Onyshchuk	14647ed03b	Add Avro nested fields insert support	2020-04-18 15:46:24 -05:00
alexey-milovidov	d1b72f10d5	Merge pull request #10325 from Avogar/msgpack_format ... Improve MsgPackRowInputFormat	2020-04-18 14:27:11 +03:00
alexey-milovidov	ef5f83eef8	Update MsgPackRowInputFormat.cpp	2020-04-18 14:26:57 +03:00
Dmitry	3a18982e2b	removed debug cerr	2020-04-18 13:04:49 +03:00
Dmitry	25cd4ce132	Merge branch 'master' of github.com:yandex/ClickHouse into group_by_in_order_optimization	2020-04-18 12:56:34 +03:00
Kruglov Pavel	96f4d5b7c6	Fix build error	2020-04-17 23:51:53 +03:00
Kruglov Pavel	0f11526f24	Fix style error	2020-04-17 22:19:03 +03:00
Avogar	b056dbce1c	Improve MsgPackRowInputFormat	2020-04-17 12:35:38 +03:00
Alexey Milovidov	b16ee538ab	Merge branch 'master' into sorting-processors	2020-04-17 05:31:55 +03:00
alexey-milovidov	17e7d4d88a	Merge pull request #10307 from abyss7/arcadia-4 ... Changes for auto-sync with Arcadia	2020-04-17 05:08:34 +03:00
alexey-milovidov	390bfe9a4c	Merge pull request #10317 from Avogar/markdown_format ... Markdown format	2020-04-17 05:05:13 +03:00
Kruglov Pavel	2672b71b2d	Delete extra spaces	2020-04-16 23:47:01 +03:00
Avogar	89d787dae3	Add new line at the end of file	2020-04-16 23:14:19 +03:00
Avogar	831c48c514	Add MarkdownRowOutput format	2020-04-16 23:10:42 +03:00
Ivan Lezhankin	e230632645	Changes required for auto-sync with Arcadia	2020-04-16 15:31:57 +03:00
alexey-milovidov	d296e1b6d7	Merge pull request #10216 from Avogar/msgpack_format ... Update MsgPack format	2020-04-16 00:44:08 +03:00
Nikolai Kochetov	43e178abfb	Revert "dded more comments to AggregatingTransform." ... This reverts commit 01e495d783.	2020-04-15 14:04:38 +03:00
Nikolai Kochetov	95916e39cf	Added more comments to AggregatingTransform.	2020-04-15 13:57:42 +03:00
Nikolai Kochetov	01e495d783	dded more comments to AggregatingTransform.	2020-04-15 13:55:34 +03:00
Nikolai Kochetov	0c2d3417e2	Fix pipeline stuck with max_rows_to_group_by.	2020-04-15 13:18:52 +03:00
Nikolai Kochetov	153f795ebe	Merge branch 'master' into sorting-processors	2020-04-15 12:07:05 +03:00
Avogar	93c322bc30	Fix style error	2020-04-15 02:58:36 +03:00
Avogar	b8061356b0	Update parsing method in MsgPackRowInputFormat	2020-04-15 02:08:55 +03:00
Alexey Milovidov	8ad04d4fec	Remove useless code	2020-04-15 00:05:45 +03:00
Nikolai Kochetov	15a8ac3245	Fix tests.	2020-04-14 22:31:49 +03:00
Nikolai Kochetov	7def7b021e	Fix test with vesrioned collapsing.	2020-04-14 21:58:22 +03:00
Nikolai Kochetov	80c27689ff	Add debug output.	2020-04-14 21:22:14 +03:00
Nikolai Kochetov	92692aff5e	Fix build.	2020-04-14 21:03:29 +03:00
Nikolai Kochetov	2da9f889cd	Fix unit test.	2020-04-14 14:09:23 +03:00
Nikolai Kochetov	1ce2e1b93b	Move comments.	2020-04-14 12:43:53 +03:00
Nikolai Kochetov	fc42851186	Move comments.	2020-04-14 12:43:17 +03:00
Nikolai Kochetov	434a13a8bd	Move algorithm to separate files.	2020-04-14 12:21:24 +03:00
Nikolai Kochetov	71feedea6f	Remove some code.	2020-04-14 12:05:29 +03:00
Nikolai Kochetov	848026ac4c	Fix style.	2020-04-14 11:48:38 +03:00
Nikolai Kochetov	2543741a9a	Fix build.	2020-04-14 00:03:11 +03:00
Nikolai Kochetov	1ef6e8f7c7	Fix build.	2020-04-13 23:58:29 +03:00
Nikolai Kochetov	801320983b	Refactor SummingSortedAlgorithm.	2020-04-13 23:50:13 +03:00
Avogar	e46322fcef	Update MsgPack input format.	2020-04-13 22:33:02 +03:00
Nikolai Kochetov	84b1dd4c1e	Fix tests.	2020-04-13 22:20:26 +03:00
Nikolai Kochetov	d7d29d8250	Fix build.	2020-04-13 22:02:57 +03:00
Nikolai Kochetov	586c295b94	Added SummingSortedAlgorithm	2020-04-13 21:51:17 +03:00
Nikolai Kochetov	89aef7aaf9	Add comments.	2020-04-13 18:04:54 +03:00
Nikolai Kochetov	2b81d68b56	Fix build.	2020-04-13 17:58:23 +03:00
Nikolai Kochetov	9acea16b7f	Fix build.	2020-04-13 17:57:21 +03:00
Nikolai Kochetov	d3adcc1cb0	Fix build.	2020-04-13 17:53:04 +03:00
Nikolai Kochetov	377e16c00c	Refactor AggregatingSortedAlgorithm.	2020-04-13 17:42:58 +03:00
Nikolai Kochetov	9ce0607de7	Remove header from AggregatingSortedAlgorithm.	2020-04-13 16:40:53 +03:00
alexey-milovidov	d1c436788b	Merge pull request #10206 from ClickHouse/fix-limit-with-infinite-sources ... Fix limit with infinite sources	2020-04-13 16:32:02 +03:00
Nikolai Kochetov	0544c1f25b	Fix build.	2020-04-13 16:26:50 +03:00
Nikolai Kochetov	5efdf1203e	Fix build.	2020-04-13 16:22:32 +03:00
Nikolai Kochetov	a2b5fd4b3d	Fix build.	2020-04-13 16:18:46 +03:00
Nikolai Kochetov	a6bf7a14c8	Fix build.	2020-04-13 15:02:59 +03:00
Nikolai Kochetov	03dad5ccf5	Added AggregatingSortedAlgorithm.	2020-04-13 14:45:59 +03:00
Nikolai Kochetov	9a9bedc8cc	Fix test for streams.	2020-04-13 12:02:50 +03:00
Nikolai Kochetov	29fc8f145b	Fix tests in debug.	2020-04-13 11:50:14 +03:00
Avogar	b9931863ef	Fix FixedString packing	2020-04-13 00:01:17 +03:00
Alexey Milovidov	20dcc4decd	Fixed build on FreeBSD according to the advice from Vitaly @hellvesper	2020-04-12 23:05:58 +03:00
alexey-milovidov	718e4bcdf8	Update IProcessor.h	2020-04-12 22:01:02 +03:00
Nikolai Kochetov	bff1f24cf7	Added IProcessor::onUpdatePorts	2020-04-12 18:21:21 +03:00
Nikolai Kochetov	4da19d122d	Added IProcessor::onUpdatePorts	2020-04-12 18:02:17 +03:00
alexey-milovidov	0782052768	Merge pull request #10131 from ClickHouse/fix-extremes-for-processors ... Fix extremes for processors	2020-04-12 02:31:39 +03:00
Nikolai Kochetov	198f02d4cf	Try simplify Replacing.	2020-04-10 20:22:15 +03:00
Nikolai Kochetov	ca8c78bfa2	Try simplify Replacing.	2020-04-10 20:19:44 +03:00
Nikolai Kochetov	44a3e216d1	Try simplify Replacing.	2020-04-10 20:16:38 +03:00
Nikolai Kochetov	ba840b5a59	Added VersionedCollapsingAlgorithm.	2020-04-10 19:44:47 +03:00
Nikolai Kochetov	2fa40ad554	Fix build.	2020-04-10 19:18:53 +03:00
Nikolai Kochetov	864e0a407f	Added GraphiteRollupSortedAlgorithm.	2020-04-10 19:15:51 +03:00
Nikolai Kochetov	f35a8bf351	Fix style.	2020-04-10 18:41:22 +03:00
Nikolai Kochetov	994f95ddb0	Added ReplacingSortedAlgorithm.	2020-04-10 17:03:36 +03:00
Nikolai Kochetov	123c654db3	Fix build.	2020-04-10 16:41:00 +03:00
Nikolai Kochetov	51e5603044	Fix build.	2020-04-10 16:03:37 +03:00
Nikolai Kochetov	e9587b8f4c	Fix build.	2020-04-10 15:53:51 +03:00
Nikolai Kochetov	d46c228b9c	Fix build.	2020-04-10 15:47:30 +03:00
Nikolai Kochetov	7b85a36975	Added CollapsingSortedAlgorithm.	2020-04-10 15:40:16 +03:00
Nikolai Kochetov	2aba662d2c	Fix build.	2020-04-10 12:32:34 +03:00
Nikolai Kochetov	97158f7656	Fix build.	2020-04-10 12:30:53 +03:00
Nikolai Kochetov	27f3d2711a	Move Graphite params to separate file.	2020-04-10 12:20:43 +03:00
Nikolai Kochetov	868f7e9aea	Move Graphite params to separate file.	2020-04-10 12:16:23 +03:00
Nikolai Kochetov	1b6d80bfae	Merge branch 'master' into sorting-processors	2020-04-10 12:07:21 +03:00
Nikolai Kochetov	f7fdfe4ed2	Fix build.	2020-04-10 12:04:56 +03:00
Nikolai Kochetov	fc605d411d	Added IMergingAlgorithm.	2020-04-10 12:04:28 +03:00
Nikolai Kochetov	2ce753a316	Fix tests.	2020-04-10 12:04:28 +03:00
Nikolai Kochetov	cbd1f6a524	Added comments.	2020-04-10 12:04:28 +03:00
Nikolai Kochetov	f96a5fe923	Fix build,	2020-04-10 12:04:50 +03:00
Nikolai Kochetov	71fab516f2	Fix AggregatingSorted for simple aggregate functions.	2020-04-10 12:04:34 +03:00
Nikolai Kochetov	fe442d8c9a	Remove special case for simple input in IMergingTransform.	2020-04-10 12:04:28 +03:00
Nikolai Kochetov	8726ad9e3b	Try fix collapsing.	2020-04-10 12:04:28 +03:00
Nikolai Kochetov	d3acce8afa	Fix build.	2020-04-10 12:04:28 +03:00
alexey-milovidov	4a73fe7477	Merge pull request #9889 from Avogar/msgpack_format ... Msgpack format	2020-04-10 00:07:10 +03:00
Nikolai Kochetov	9f5a40e700	Added comment.	2020-04-09 13:35:51 +03:00
Nikolai Kochetov	ae6267070e	Fix style.	2020-04-09 09:28:13 +03:00
Nikolai Kochetov	458c7f516d	Fix extremes for Processors.	2020-04-08 21:50:14 +03:00
Nikolai Kochetov	6b5b22cc5c	Fix build.	2020-04-07 20:53:14 +03:00
Nikolai Kochetov	2704d81aae	Fix pipeline expansion in PipelineExecutor.	2020-04-07 20:38:48 +03:00
Artem Zuikov	8297683972	renames: AnalyzedJoin -> TableJoin, Join -> HashJoin (#10065)	2020-04-07 12:48:47 +03:00
Nikolai Kochetov	8986cf688b	Added comment.	2020-04-06 16:28:46 +03:00
Nikolai Kochetov	0117c194c5	Fix GroupingAggregatedTransform for single-level aggregation. Add test.	2020-04-06 13:33:59 +03:00
Avogar	e635b0e9eb	Fix build error	2020-04-04 20:04:41 +03:00
Nikolai Kochetov	e23fda80aa	Merge with master	2020-04-04 19:02:17 +03:00
Avogar	fbc11d3bf9	Move files	2020-04-04 17:04:20 +03:00
Ivan Lezhankin	06446b4f08	dbms/ → src/	2020-04-03 18:14:31 +03:00