5875 Commits

Author	SHA1	Message	Date
Nikolai Kochetov	6a821f9e73	Fix some staff	2023-12-04 19:24:27 +00:00
Anton Popov	40a1744b7c	Merge remote-tracking branch 'upstream/master' into HEAD	2023-12-04 18:10:43 +00:00
Nikolai Kochetov	7ab4af06df	Attempt to support subqueries in TTL. (2)	2023-12-04 18:04:42 +00:00
Kseniia Sumarokova	7bcc1afa8c	Merge branch 'master' into broken-projections-better-handling	2023-12-04 13:21:57 +01:00
Yakov Olkhovskiy	b601b66146	don't exclude ephemeral column when pushing to view	2023-12-04 06:56:33 +00:00
Nikita Taranov	2362bb2cc9	Don't fall-back to in-order pool when max_streams = 1 for remote fs (#57334) ... * impl * update tests * update one more test	2023-12-01 21:39:26 +01:00
Nikolai Kochetov	508046e692	Attempt to support subqueries in TTL.	2023-12-01 20:31:26 +00:00
JackyWoo	8a68d51261	Add index to table system.numbers (#50909)	2023-12-01 19:59:25 +01:00
Kseniia Sumarokova	5c3d1b5991	Merge branch 'master' into broken-projections-better-handling	2023-11-30 21:33:35 +01:00
Raúl Marín	46acd8736c	Merge pull request #57189 from Algunenano/fix_respect_nulls ... Fix RESPECT NULLS	2023-11-30 20:21:42 +01:00
Igor Nikonov	b4bf611d5d	Merge remote-tracking branch 'origin/master' into pr-custom-key-failover	2023-11-30 14:09:26 +00:00
Igor Nikonov	59daabe95a	Some refactoring and cleanup ... + change load balancing for custom key w/o hedged connections (need the test though)	2023-11-30 14:08:01 +00:00
Kruglov Pavel	c6fecfb1af	Merge pull request #56901 from KevinyhZou/Fix_allow_cr_end_of_csv_line ... Fix allow cr end of line for csv	2023-11-29 20:57:58 +01:00
Kruglov Pavel	1d840ca026	Fix style	2023-11-29 20:49:53 +01:00
avogar	b493ce2385	Better JSON -> JSONEachRow fallback without catching exceptions	2023-11-29 14:19:38 +00:00
Igor Nikonov	836d149e90	Fix: use only one connection per replica	2023-11-29 13:42:18 +00:00
Han Fei	4c0efb0262	Merge pull request #53240 from hanfei1991/hanfei/statistic ... use statistic to order prewhere conditions better	2023-11-29 12:26:44 +01:00
vdimir	470a092e3d	Merge pull request #51745 from ClickHouse/vdimir/setting_join_algorithm_order ... Setting JoinAlgorithm respect specified order	2023-11-29 11:24:39 +01:00
vdimir	a6eb2ad930	Fix JoinStep::describeJoinActions	2023-11-28 19:09:01 +00:00
Igor Nikonov	a06a71834a	Merge remote-tracking branch 'origin/master' into pr-custom-key-failover	2023-11-28 18:07:04 +00:00
Han Fei	6733438adc	Merge branch 'master' into hanfei/statistic	2023-11-28 16:35:00 +01:00
János Benjamin Antal	4c2451f837	Merge pull request #56741 from ClickHouse/well-known-protos	2023-11-28 15:21:15 +01:00
Raúl Marín	48fff248e5	Merge remote-tracking branch 'blessed/master' into fix_respect_nulls	2023-11-28 12:28:36 +01:00
Han Fei	7bd2ea3add	Merge branch 'master' into hanfei/statistic	2023-11-27 23:31:23 +01:00
Raúl Marín	80cc1b3a17	Remove addBatchSinglePlaceFromInterval	2023-11-27 23:12:17 +01:00
Joanna Hulboj	da8f210caa	Merge branch 'ClickHouse:master' into no-octal-leading-zeroes	2023-11-27 21:00:48 +01:00
Igor Nikonov	e823046646	Fixes	2023-11-27 15:27:04 +00:00
Raúl Marín	4250c6626c	Move handling of RESPECT NULLS outside of the parser	2023-11-27 11:00:27 +01:00
Antonio Andelic	4c2de5219d	Merge pull request #57196 from amosbird/fix-57194 ... Fix incorrect JOIN plan optimization with partially materialized normal projection	2023-11-27 09:15:13 +01:00
Igor Nikonov	774347d231	Parallel replicas custom key: skip unavailable replicas	2023-11-26 21:38:49 +00:00
Robert Schulze	5b7d2a903d	Merge pull request #57152 from rschu1ze/fine-granular-plan-opt-settings ... Fine-granular enablement/disabling of plan-level optimizations	2023-11-25 20:44:23 +01:00
Alexey Milovidov	61c9d304f0	Merge pull request #57125 from ClickHouse/try-fix-57097 ... Cancel executor in ~CreatingSetsTransform	2023-11-25 03:02:39 +01:00
Amos Bird	472027e948	Fix JOIN plan with normal projection	2023-11-24 21:47:00 +08:00
Igor Nikonov	0434044a7e	Merge pull request #57054 from ClickHouse/pr-cleanup-narrow-dependency ... Parallel replicas: cleanup, narrow dependency	2023-11-23 20:12:37 +01:00
Robert Schulze	e392fcab5b	Allow global and individual disabling of settings ... - 'query_plan_enable_optimizations' is now a global kill switch for all plan-level optimizations - even if it is true, individual optimizations can still be disabled individually via their respective settings	2023-11-23 18:46:48 +00:00
Robert Schulze	194e676c70	Put all query plan settings under their own settings	2023-11-23 18:45:08 +00:00
Robert Schulze	4b764b8bc9	Cosmetics	2023-11-23 18:43:10 +00:00
Robert Schulze	77d48e4d55	"optimise" --> "optimize". Sorry, friends from the UK.	2023-11-23 18:41:35 +00:00
Robert Schulze	674159857e	Merge pull request #57144 from rschu1ze/disable-reorder-functions-after-sorting ... Allow to disable reorder-functions-after-sorting optimization	2023-11-23 19:38:31 +01:00
Robert Schulze	4f10ad3f3c	Allow to disable reorder-functions-after-sorting optimization	2023-11-23 13:16:03 +00:00
Nikolai Kochetov	1919ec723d	Cancel executor in ~CreatingSetsTransform	2023-11-22 17:31:29 +00:00
Kruglov Pavel	e1904cbdf1	Cancel PipelineExecutor properly in case of exception in spawnThreads	2023-11-22 14:43:52 +01:00
János Benjamin Antal	ab935e3dd7	Use the google proto files when importing protobuf schemas	2023-11-22 12:39:41 +00:00
Alexey Milovidov	d0653f244b	Merge pull request #57062 from ClickHouse/add-comment-to-transformPipeline ... Better comment for ITransformingStep::transformPipeline	2023-11-22 13:37:11 +01:00
Kruglov Pavel	96a3cefff6	Merge pull request #56118 from edef1c/large-arrow-list ... Support reading arrow::LargeListArray	2023-11-22 13:28:46 +01:00
Duc Canh Le	4faef352f9	minor fix and comments ... Signed-off-by: Duc Canh Le <duccanh.le@ahrefs.com>	2023-11-22 09:23:52 +00:00
Duc Canh Le	4b0382442f	Merge branch 'master' into final_no_copy ... Try fixing CI issues	2023-11-22 08:53:38 +00:00
Anton Popov	65efb18842	Merge remote-tracking branch 'upstream/master' into HEAD	2023-11-21 17:30:38 +00:00
Igor Nikonov	63a1625b77	Merge remote-tracking branch 'origin/master' into pr-cleanup-narrow-dependency	2023-11-21 16:05:48 +00:00
Nikolai Kochetov	b1ec8a3670	Better comment for ITransformingStep::transformPipeline	2023-11-21 15:35:09 +00:00
Igor Nikonov	66f6a6575f	Cleanup iteration: settings usage	2023-11-21 13:29:04 +00:00
avogar	7e392eec50	Better exception messages in input formats	2023-11-21 13:13:42 +00:00
Raúl Marín	47abb9e879	Fix race condition in RemoteSource	2023-11-21 14:07:14 +01:00
kssenii	c0a0431e04	Merge remote-tracking branch 'origin/master' into broken-projections-better-handling	2023-11-21 13:22:30 +01:00
Duc Canh Le	2e2a4021f2	fix array join wrong query result ... Signed-off-by: Duc Canh Le <duccanh.le@ahrefs.com>	2023-11-21 09:06:17 +00:00
kevinyhzou	3adc8fdf78	Fix ci	2023-11-21 11:22:12 +08:00
avogar	6d3de41dc2	Fix skipping lots rows with errors	2023-11-20 18:07:54 +00:00
avogar	ffa90628f0	Make input format errors logger a bit better	2023-11-20 17:22:49 +00:00
avogar	872556a5d4	Merge branch 'master' of github.com:ClickHouse/ClickHouse into schema-inference-union	2023-11-20 14:03:36 +00:00
Igor Nikonov	7020177e7c	Merge branch 'master' into pr-cleanup-remote-source	2023-11-19 21:33:47 +01:00
Alexey Milovidov	d56cbda185	Add metrics for the number of queued jobs, which is useful for the IO thread pool	2023-11-18 19:07:59 +01:00
Raúl Marín	fc793fe31f	Merge pull request #56120 from kitaisreal/window-functions-decrease-amount-of-virtual-function-calls ... WindowTransform decrease amount of virtual function calls	2023-11-17 13:24:38 +01:00
kevinyhzou	b8bb9835fb	Fix allow cr end of line	2023-11-17 16:09:15 +08:00
Kruglov Pavel	8e78d4c9c1	Merge pull request #56304 from yariks5s/npy_data_caching ... Implement countRows for Npy data format	2023-11-16 17:01:25 +01:00
Kruglov Pavel	1831ecc38f	Merge pull request #56424 from yariks5s/npy_float_16 ... Float16 support for npy format	2023-11-16 17:00:24 +01:00
kssenii	6c42a3fad6	Better	2023-11-16 16:51:05 +01:00
Igor Nikonov	110245bf0b	Merge branch 'master' into pr-cleanup-remote-source	2023-11-16 12:41:32 +01:00
Raúl Marín	6d094efd80	Merge remote-tracking branch 'blessed/master' into window-functions-decrease-amount-of-virtual-function-calls	2023-11-16 12:32:11 +01:00
Dmitry Novik	33cc853d61	Merge pull request #39631 from ClickHouse/parallel-window ... Window function parallel evaluation	2023-11-16 11:50:16 +01:00
Amos Bird	1e46460910	Fix nullable primary key in final (#56452)	2023-11-15 22:27:05 +01:00
Dmitry Novik	592c653cac	Fixup	2023-11-15 20:26:07 +00:00
Dmitry Novik	83acec9511	Fix max_threads = 1 case	2023-11-15 18:58:14 +00:00
Nikolai Kochetov	9dbfb22baf	Add a comment.	2023-11-15 16:02:14 +00:00
Nikolai Kochetov	a52fb6de32	Revert changes to TaskQueue	2023-11-15 15:50:23 +00:00
yariks5s	4e48df6bfc	improved nulls	2023-11-15 12:15:21 +00:00
Nikolai Kochetov	dbdf734a32	Fix build. Prettify code.	2023-11-15 12:12:07 +00:00
yariks5s	9376ac5603	added implementation for negative nulls	2023-11-15 11:42:52 +00:00
Yarik Briukhovetskyi	12a8a38d35	Merge branch 'ClickHouse:master' into npy_data_caching	2023-11-15 12:15:41 +01:00
Nikolai Kochetov	0a628de87c	Try to fix hang in 01104_distributed_numbers_test	2023-11-14 17:42:10 +00:00
yariks5s	80dffe5823	fixed null due to review	2023-11-14 15:55:46 +00:00
Duc Canh Le	fa699a33c0	address review comments ... Signed-off-by: Duc Canh Le <duccanh.le@ahrefs.com>	2023-11-14 15:26:57 +00:00
yariks5s	22a9f71393	fixed due to review	2023-11-14 13:15:14 +00:00
Alexey Milovidov	ac9f01079a	Merge pull request #56722 from ClickHouse/remove-orphan-headers ... Remove orphan header files	2023-11-14 12:03:10 +01:00
Robert Schulze	2931bbfe62	Merge pull request #56665 from rschu1ze/refactor-ast-hash-calculation ... Small refactoring of AST hash calculation (follow-up to #56545)	2023-11-14 11:24:30 +01:00
Alexey Milovidov	9bf62dd622	Remove orphan header files	2023-11-14 07:43:11 +01:00
Alexey Milovidov	c81cc3f953	Merge pull request #56660 from ClickHouse/better-ccache-2 ... Better use of build cache	2023-11-13 19:28:33 +01:00
Robert Schulze	d6df7d8403	Merge remote-tracking branch 'rschu1ze/master' into refactor-ast-hash-calculation	2023-11-13 15:54:21 +00:00
Alexey Milovidov	2ad98a58c9	Better caching of arrow build (#56657) ... * Better caching of Arrow build * Fix error * Remove useless files	2023-11-13 16:51:01 +01:00
Kruglov Pavel	d5280df6d3	Merge pull request #56122 from bigo-sg/orc_tuple_field_prune ... Allow tuple field pruning in ORC to speed up scaning	2023-11-13 14:46:43 +00:00
Robert Schulze	b899270372	Remove default value for argument 'ignore_aliases' from IAST::updateTreeHash()	2023-11-13 13:18:35 +00:00
Alexey Milovidov	82c461031e	Fix build	2023-11-13 10:09:23 +01:00
Alexey Milovidov	9a7f5ab0a1	Better use of build cache	2023-11-13 08:54:24 +01:00
Alexey Milovidov	fcd45d47e4	Merge branch 'master' of github.com:ClickHouse/ClickHouse into remove-cpp-templates-2	2023-11-11 03:45:54 +01:00
Alexey Milovidov	c095b1e9ef	Merge pull request #56502 from amosbird/fix-56481 ... Fix two cases of projection analysis.	2023-11-11 02:29:04 +01:00
Alexey Milovidov	3bec4dce8e	Merge branch 'master' into remove-cpp-templates-2	2023-11-11 00:50:32 +01:00
Maksim Kita	bf08f3ef5a	Revert "Inserting only non-duplicate chunks in MV"	2023-11-10 21:21:13 +03:00
Alexey Milovidov	8c253b9e3e	Remove C++ templates	2023-11-10 05:25:02 +01:00
Amos Bird	51ca7d05fb	Fix tests	2023-11-10 10:23:03 +08:00
Amos Bird	16578d97a3	Fix two cases of projection analysis. ... 1. Fix projection analysis with partitions. 2. Apply alter conversions during projection analysis.	2023-11-10 10:23:02 +08:00
Nikolai Kochetov	0bfa2b6975	Merge pull request #56254 from ClickHouse/aggregate-by-partition-analyzer ... Properly process aliases for aggregation-by-partition optimization.	2023-11-09 12:49:25 +01:00
Igor Nikonov	7c4d197ca5	Merge branch 'master' into pr-cleanup-remote-source	2023-11-08 19:38:15 +01:00
Yarik Briukhovetskyi	b8ea31db43	Merge pull request #56309 from yariks5s/preferred_optimize_projection_name ... New setting preferred_projection_name	2023-11-08 18:06:03 +01:00
Nikolai Kochetov	7eacea4c51	Update src/Processors/QueryPlan/Optimizations/useDataParallelAggregation.cpp ... Co-authored-by: Nikita Taranov <nikita.taranov@clickhouse.com>	2023-11-08 17:28:57 +01:00
taiyang-li	6199bb83a9	Merge branch 'master' into orc_tuple_field_prune	2023-11-08 14:54:12 +08:00
yariks5s	6f6454e696	style check	2023-11-07 18:12:56 +00:00
yariks5s	181231d500	init	2023-11-07 17:56:02 +00:00
Robert Schulze	32a3b1b049	Merge remote-tracking branch 'rschu1ze/master' into idatatype-convenience-functions	2023-11-07 15:17:59 +00:00
Sema Checherinda	02df5ac59c	Merge pull request #54340 from KevinyhZou/bug_fix_csv_parse_crlf ... Fix CSV read while the end of line is CR(\r)	2023-11-07 12:51:05 +01:00
Robert Schulze	54b5d573d7	Merge remote-tracking branch 'rschu1ze/master' into idatatype-convenience-functions	2023-11-07 10:18:30 +00:00
yariks5s	2e03462d87	style fix	2023-11-06 22:26:35 +00:00
edef	a52e22a858	Support reading arrow::LargeListArray	2023-11-06 21:17:29 +00:00
yariks5s	faba36225e	fixed tests	2023-11-06 19:10:00 +00:00
yariks5s	ec32c23f0c	style check	2023-11-06 15:25:14 +00:00
yariks5s	7206419a6b	Merge branch 'preferred_optimize_projection_name' of github.com:yariks5s/ClickHouse into preferred_optimize_projection_name	2023-11-06 14:30:33 +00:00
yariks5s	e48e6b6a5a	fixed review, changed tests	2023-11-06 14:30:10 +00:00
Robert Schulze	8896134531	Cleanup IDataType convenience functions	2023-11-06 13:25:32 +00:00
Alexander Tokmakov	eba52b318d	Merge pull request #56272 from ClickHouse/rdb_fix_rare_logical_error ... Fix rare logical error in Replicated database	2023-11-06 14:14:55 +01:00
kevinyhzou	2a50daf5dd	Allow cr at end of csv line	2023-11-06 12:21:42 +08:00
kevinyhzou	ef30e6723d	bug fix csv read while end of line is not crlf	2023-11-06 12:21:42 +08:00
Igor Nikonov	9bdcba1b20	Merge branch 'master' into pr-progress-bar	2023-11-05 00:09:46 +01:00
Alexey Milovidov	36d18acf35	Merge pull request #55007 from Alex-Cheng/refactor-MergeJoinTransform ... Refactor merge join transform	2023-11-04 17:08:07 +01:00
taiyang-li	8e23dd9090	add some comments	2023-11-04 10:08:26 +08:00
taiyang-li	8a6ae6e150	change as request	2023-11-04 09:41:01 +08:00
Dmitry Novik	23564148f9	Merge remote-tracking branch 'origin/master' into parallel-window	2023-11-03 17:53:51 +00:00
yariks5s	6d9b517b19	fix suggestions	2023-11-03 17:21:37 +00:00
Yarik Briukhovetskyi	92588dfac7	style fix	2023-11-03 18:07:19 +01:00
yariks5s	d95262622b	init	2023-11-03 16:47:06 +00:00
Kruglov Pavel	7e4742f68a	Merge pull request #56167 from ClickHouse/revert-56150-revert-55989-lc-as-arrow-dict-fix ... Revert "Revert "Fix output/input of Arrow dictionary column""	2023-11-03 16:18:10 +00:00
yariks5s	4235c690d4	init commit	2023-11-03 15:14:25 +00:00
vdimir	86524fce0f	Merge pull request #56246 from ClickHouse/vdimir/analyzer_01019_alter_materialized_view ... Analyzer: fix 01019_alter_materialized_view_consistent	2023-11-03 11:04:14 +01:00
taiyang-li	4b7146d47e	remove useless codes	2023-11-03 16:03:26 +08:00
Alexander Tokmakov	0ce1560e5d	retry if table replica being restarted	2023-11-02 18:30:32 +01:00
Nikolai Kochetov	80d4e9a045	Merge branch 'master' into aggregate-by-partition-analyzer	2023-11-02 11:21:38 +00:00
Alexey Milovidov	1267246785	Merge pull request #55642 from ClickHouse/morton-utils ... Indexing by space-filling curves	2023-11-02 12:21:33 +01:00
Nikolai Kochetov	5b6bf58730	Proper process aliases for aggregation-by-partition optimization.	2023-11-02 11:19:37 +00:00
vdimir	5703e0995b	Analyzer: fix 01019_alter_materialized_view_consistent	2023-11-02 10:27:22 +00:00
taiyang-li	819c7e75ff	Merge branch 'master' into orc_tuple_field_prune	2023-11-02 17:52:05 +08:00
taiyang-li	6ccde98943	change as request	2023-11-02 11:01:50 +08:00
李扬	424f1bec93	Apply suggestions from code review ... Co-authored-by: Kruglov Pavel <48961922+Avogar@users.noreply.github.com>	2023-11-02 10:30:28 +08:00
李扬	c899ff9da2	Apply suggestions from code review ... Co-authored-by: Kruglov Pavel <48961922+Avogar@users.noreply.github.com>	2023-11-02 10:30:14 +08:00
Alexey Milovidov	697f2591b9	Merge with master	2023-11-02 00:33:33 +01:00
Kruglov Pavel	aa7c1f63ab	Merge pull request #56172 from Avogar/fix-schema-cache-for-json ... Fix schema cache for fallback JSON->JSONEachRow with changed settings	2023-11-01 20:24:34 +01:00
Kruglov Pavel	e6f00d5e1d	Merge pull request #56117 from bigo-sg/fixed_struct_field_prune ... Improve parquet struct fields reading	2023-11-01 15:32:50 +01:00
Kruglov Pavel	bf77ce691c	Merge pull request #55982 from yariks5s/npy_input_format ... New input format Npy	2023-11-01 14:26:22 +01:00
taiyang-li	dc897215da	fix failed uts tests/queries/0_stateless/02312_parquet_orc_arrow_names_tuples.sql	2023-11-01 20:42:07 +08:00
Nikolai Kochetov	4deaf7cefb	Merge pull request #56134 from yariks5s/force_optimize_projection_name ... Implementing force_optimize_projection_name	2023-11-01 13:12:10 +01:00
taiyang-li	24c45a4ee0	fix failed uts	2023-11-01 18:47:11 +08:00
taiyang-li	001cbe7912	fix typos	2023-11-01 16:58:25 +08:00
taiyang-li	b276587422	fix failed uts	2023-11-01 15:43:20 +08:00
lgbo-ustc	8334585eaf	improve parquet struct field reading	2023-11-01 15:18:39 +08:00
Alex Cheng	9e31858100	Merge branch 'ClickHouse:master' into refactor-MergeJoinTransform	2023-11-01 09:32:28 +08:00
yariks5s	6c4bf59021	fix suggestions and enhance tests	2023-10-31 18:10:55 +00:00
avogar	518e52473d	Fix schema cache for fallback JSON->JSONEachRow with changed settings	2023-10-31 14:12:38 +00:00
Kruglov Pavel	4faa3d0294	Revert "Revert "Fix output/input of Arrow dictionary column""	2023-10-31 12:30:45 +01:00
Nikolai Kochetov	f748f12426	Merge pull request #51746 from ClickHouse/fix-read-in-order-with-array-join ... Fix 'Cannot find column' in read-in-order optimization with ARRAY JOIN	2023-10-31 11:51:01 +01:00
Nikolai Kochetov	0523e6cbd0	Merge pull request #53282 from ClickHouse/planner-prepare-filters-for-analysis-2 ... Support skip_unused_shards in Analyzer	2023-10-31 11:33:59 +01:00
Alexey Milovidov	467b4d85e2	Revert "Fix output/input of Arrow dictionary column"	2023-10-31 09:28:09 +03:00
taiyang-li	c97b2c5be7	fix code style	2023-10-31 12:00:45 +08:00
taiyang-li	e5db57204d	fix bugs	2023-10-31 11:57:47 +08:00
taiyang-li	b72341e1a8	Merge branch 'master' into orc_tuple_field_prune	2023-10-31 10:07:43 +08:00
Han Fei	1237cef26d	address comments	2023-10-31 00:53:46 +01:00
Kruglov Pavel	4c2a132d96	Merge pull request #55989 from Avogar/lc-as-arrow-dict-fix ... Fix output/input of Arrow dictionary column	2023-10-30 20:47:49 +01:00
Kruglov Pavel	4effc676f9	Merge pull request #56046 from Avogar/cr-in-unquoted-csv-string ... Allow unquoted strings with CR in CSV format	2023-10-30 20:46:20 +01:00
Nikolai Kochetov	84f6a243b7	Merge branch 'master' into fix-read-in-order-with-array-join	2023-10-30 16:35:31 +00:00
yariks5s	03236c48ed	init	2023-10-30 16:21:50 +00:00
yariks5s	9a2d89e3e4	removed getSize() and enhanced docs	2023-10-30 12:42:19 +00:00
taiyang-li	ad67b6c2ea	allow tuple field pruning	2023-10-30 19:33:06 +08:00
Nikolai Kochetov	554ceb4e1d	Merge branch 'master' into planner-prepare-filters-for-analysis-2	2023-10-30 11:56:30 +01:00
Maksim Kita	f8e209ebd2	WindowTransform decrease amount of virtual function calls	2023-10-30 13:50:25 +03:00
Kruglov Pavel	c10a3b3838	Merge branch 'master' into lc-as-arrow-dict-fix	2023-10-30 11:20:57 +01:00
avogar	57bc4854c2	Fix	2023-10-30 10:17:49 +00:00
avogar	d1fcbc6e47	Fix fetching schema from schema registry in AvroConfluent	2023-10-30 10:17:48 +00:00
Han Fei	19cdc6dfe2	Merge branch 'master' into hanfei/statistic	2023-10-30 04:53:18 +01:00
Alexey Milovidov	6ad027ed7a	Update ConstantExpressionTemplate.cpp	2023-10-30 05:32:36 +03:00
yariks5s	e14a7f066a	fix typos	2023-10-28 01:46:59 +00:00
yariks5s	894724bfb3	suggested changes	2023-10-28 01:17:25 +00:00
yariks5s	23635352f1	fixed due to review	2023-10-27 15:43:03 +00:00
Kruglov Pavel	bb4b95e891	Merge branch 'master' into schema-inference-union	2023-10-27 14:53:58 +02:00
frinkr	18c50c11b3	Multithreading after window functions (#50771) ... * feat: Preserve number of streams after evaluation the window functions to allow parallel stream processing * fix style * fix style * fix style * setting query_plan_preserve_num_streams_after_window_functions default true * fix tests by SETTINGS query_plan_preserve_num_streams_after_window_functions=0 * fix test references * Resize the streams after the last window function, to keep the order between WindowTransforms (and WindowTransform works on single stream anyway). * feat: Preserve number of streams after evaluation the window functions to allow parallel stream processing * fix style * fix style * fix style * setting query_plan_preserve_num_streams_after_window_functions default true * fix tests by SETTINGS query_plan_preserve_num_streams_after_window_functions=0 * fix test references * Resize the streams after the last window function, to keep the order between WindowTransforms (and WindowTransform works on single stream anyway). * add perf test * perf: change the dataset from 50M to 5M * rename query_plan_preserve_num_streams_after_window_functions -> query_plan_enable_multithreading_after_window_functions * update test reference * fix clang-tidy --------- Co-authored-by: Nikita Taranov <nikita.taranov@clickhouse.com>	2023-10-27 12:36:28 +02:00
Kruglov Pavel	570b66f027	Merge branch 'master' into schema-inference-union	2023-10-26 19:26:00 +02:00
avogar	9d207bf027	Allow unquoted strings with CR in CSV format	2023-10-26 13:50:54 +00:00
Duc Canh Le	f7c0914c2b	get rid of cloning single-row-chunk for last row ... Signed-off-by: Duc Canh Le <duccanh.le@ahrefs.com>	2023-10-26 10:39:21 +00:00
zvonand	0766c73aab	Rename date_time_overflow_mode -> date_time_overflow_behavior, moved it to format settings	2023-10-25 23:11:13 +02:00
zvonand	5b86e8c714	updated after review	2023-10-25 23:10:58 +02:00
zvonand	2f3695add8	Introduce setting for dt overflow exception ... Added tests and docs	2023-10-25 23:10:24 +02:00
Alexey Milovidov	bb5a60dc19	Merge pull request #55893 from ClickHouse/revert-partial-result-2 ... Revert "Revert "Revert "Add settings for real-time updates during query execution"""	2023-10-25 22:20:28 +02:00
Igor Nikonov	1a5d485c6c	Merge branch 'master' into pr-progress-bar	2023-10-25 19:53:20 +02:00
yariks5s	2ab1ae42c1	added docs and tests, style check	2023-10-25 10:37:05 +00:00
Igor Nikonov	fafe452f1e	Merge branch 'master' into pr-cleanup-remote-source	2023-10-25 11:48:23 +02:00
robot-ch-test-poll1	ef78889aa2	Merge pull request #55952 from ClickHouse/disable_apply_deleted_mask ... Added a setting to allow reading rows marked as deleted	2023-10-25 01:14:07 +02:00
Igor Nikonov	7b7f100250	Merge remote-tracking branch 'origin/master' into pr-progress-bar	2023-10-24 21:02:29 +00:00
Igor Nikonov	c3bd80f1c8	Set progress callback in constructor	2023-10-24 21:02:20 +00:00
Joanna Hulboj	7e3587034d	Consume leading zeroes when parsing a number in ConstantExpressionTemplate	2023-10-24 21:08:05 +01:00
avogar	c080ba9d7e	Fix output LowCardinality as Arrow dictionary	2023-10-24 19:49:04 +00:00
李扬	465962df7f	Support orc filter push down (file + stripe + rowgroup level) (#55330) ... * support orc filter push down * update orc lib version * replace setqueryinfo with setkeycondition * fix issue https://github.com/ClickHouse/ClickHouse/issues/53536 * refactor source with key condition * fix building error * remove std::cout * update orc * update orc version * fix bugs * improve code * upgrade orc lib * fix code style * change as requested * add performance tests for orc filter push down * add performance tests for orc filter push down * fix all bugs * fix default as null issue * add uts for null as default issues * upgrade orc lib * fix failed orc lib uts and fix typo * fix failed uts * fix failed uts * fix ast fuzzer tests * fix bug of uint64 overflow in https://s3.amazonaws.com/clickhouse-test-reports/55330/de22fdcaea2e12c96f300e95f59beba84401712d/fuzzer_astfuzzerubsan/report.html * fix asan fatal caused by reused column vector batch in native orc input format. refer to https://s3.amazonaws.com/clickhouse-test-reports/55330/be39d23af2d7e27f5ec7f168947cf75aeaabf674/stateless_tests__asan__[4_4].htm * fix wrong performance tests * disable 02892_orc_filter_pushdown on aarch64. https://s3.amazonaws.com/clickhouse-test-reports/55330/be39d23af2d7e27f5ec7f168947cf75aeaabf674/stateless_tests__aarch64_.html * add some comments * add some comments * inline range::equals and range::less * fix data race of key condition * trigger ci	2023-10-24 12:08:17 -07:00
yariks5s	4e09fb3e27	made_logic	2023-10-24 14:55:52 +00:00
Igor Nikonov	2c055480d6	Remove unnecessary flag	2023-10-24 14:52:47 +00:00
Kruglov Pavel	9a56ec4d63	Merge pull request #55891 from Avogar/try-fix-orc ... Try to fix possible segfault in Native ORC input format	2023-10-24 13:57:43 +02:00
Alexander Gololobov	959b8b64bd	Added a setting to allow reading rows marked as deleted	2023-10-23 19:59:17 +02:00
Robert Schulze	bef5cea5a0	Merge remote-tracking branch 'rschu1ze/master' into string-addinterval	2023-10-23 14:08:18 +00:00
Igor Nikonov	0bbf7ba921	Merge branch 'master' into pr-progress-bar	2023-10-23 14:56:59 +02:00
Kruglov Pavel	32da07b4ee	Merge pull request #55695 from Avogar/restrict-iceberg ... Fixes and improvements for Iceberg storage	2023-10-23 13:58:23 +02:00
Alex Cheng	49b3136491	Merge branch 'ClickHouse:master' into refactor-MergeJoinTransform	2023-10-23 09:56:22 +08:00
Robert Schulze	c2215cb5f0	Support string arguments	2023-10-22 09:28:45 +00:00
Alexey Milovidov	5217d64551	Remove garbage	2023-10-22 01:53:50 +02:00
Alexey Milovidov	1be4ff229b	Merge pull request #55895 from ClickHouse/fix-window-functions-with-sparse-columns ... Fix window functions in case of sparse columns.	2023-10-22 00:02:34 +02:00
János Benjamin Antal	1b0cadebd0	Convert sparse columns to full for window functions	2023-10-21 11:46:25 +00:00
Igor Nikonov	e378d920c4	Merge branch 'master' into pr-progress-bar	2023-10-21 13:16:12 +02:00
Alexey Milovidov	7ec4b99e94	Revert partial result	2023-10-21 03:14:22 +02:00
Kruglov Pavel	6f61ccfe28	Merge branch 'master' into schema-inference-union	2023-10-20 22:54:11 +02:00
avogar	6934e27e8b	Add union mode for schema inference to infer union schema of files with different schemas	2023-10-20 20:46:41 +00:00
avogar	8cc0dc17eb	Try to fix possible segfault in Native ORC input format	2023-10-20 18:50:48 +00:00
yariks5s	87f26f5132	dealt with 2dim arrays	2023-10-20 17:05:05 +00:00
Anton Popov	5819bcd07a	Support asynchronous inserts for native protocol (#54730) ... * support async insert for native protocol * use separate queue for async inserts via native protocol * fix test * better logging for async inserts and more tests * disable mixed internal and external data in async inserts * fix tests * fix quota in async inserts * disable async insert for secondary query of distributed	2023-10-20 18:39:48 +02:00
Raúl Marín	d08ce13107	Merge pull request #51692 from Algunenano/parallel_replicas_row_estimation ... Automatic decision of nº of parallel replicas	2023-10-20 15:50:43 +02:00
Azat Khuzhin	d5c960ceb6	Fix data race in CreatingSetsTransform ... The problem is that findOrPromiseToBuild() can also call shared_future::get(): (lldb) bt 10 * thread 194, name = 'MergeMutate', stop reason = step over * frame 0: 0x0000564126046bd2 clickhouse`std::rethrow_exception(p=(__ptr_ = 0x00007b5000130c80)) at exception_pointer_cxxabi.ipp:68:39 frame 1: 0x000056411c4c5ddd clickhouse`std::__1::__assoc_state<std::__1::shared_ptr<DB::Set>>::copy(this=<unavailable>) at future:701:9 frame 2: 0x000056411c4c0e3f clickhouse`DB::PreparedSetsCache::findOrPromiseToBuild(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&) [inlined] std::__1::shared_future<std::__1::shared_ptr<DB::Set>>::get[abi:v15000](this=0x00007b0c00154eb8) const at future:2243:46 frame 3: 0x000056411c4c0e2e clickhouse`DB::PreparedSetsCache::findOrPromiseToBuild(this=0x00007b1c00077018, key="__set_10438532338982543705_15680061365857338647") at PreparedSets.cpp:349:116 Note, that my test did not find the problem because of #55261 had not been merged yet. Follow-up for: #55338 Fixes: #55279 v2: s/createDeprecated/createRuntime/ Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>	2023-10-19 12:26:02 +02:00
yariks5s	6dc88a4ca4	new changes	2023-10-18 18:02:05 +00:00
Dmitry Novik	5ca43e51c1	Remove unused code	2023-10-17 20:45:00 +00:00
avogar	323486f9e8	Add tests	2023-10-17 18:10:47 +00:00
Han Fei	4b23142775	Merge branch 'master' into hanfei/statistic	2023-10-17 16:40:31 +02:00
Raúl Marín	3eaf752284	Merge remote-tracking branch 'blessed/master' into parallel_replicas_row_estimation	2023-10-17 11:36:39 +02:00
Alex Cheng	ce493b57f0	Merge branch 'ClickHouse:master' into refactor-MergeJoinTransform	2023-10-17 16:42:53 +08:00
Igor Nikonov	4b36761f89	Merge branch 'master' into pr-progress-bar	2023-10-17 10:36:32 +02:00
Michael Kolupaev	ce7eca0615	DWARF input format (#55450) ... * Add ReadBufferFromFileBase::isRegularLocalFile() * DWARF input format * Review comments * Changed things around ENABLE_EMBEDDED_COMPILER build setting * Added 'ranges' column * no-msan no-ubsan	2023-10-16 17:00:07 -07:00
Jiebin Sun	df17cd467b	Release more num_streams if data is small (#53867) ... * Release more num_streams if data is small Besides the sum_marks and min_marks_for_concurrent_read, we could also involve the system cores to get the num_streams if the data is small. Increasing the num_streams and decreasing the min_marks_for_concurrent_read would improve the parallel performance if the system has plentiful cores. Test the patch on 2x80 vCPUs system. Q39 of clickbench has got 3.3x performance improvement. Q36 has got 2.6x performance improvement. The overall geomean has got 9% gain. Signed-off-by: Jiebin Sun <jiebin.sun@intel.com> * Release more num_streams if data is small Change the min marks from 4 to 8 as the profit is small and 8 granules is the default block size. Signed-off-by: Jiebin Sun <jiebin.sun@intel.com> --------- Signed-off-by: Jiebin Sun <jiebin.sun@intel.com>	2023-10-16 18:41:38 +02:00
Raúl Marín	f799f5d7a1	Merge remote-tracking branch 'blessed/master' into parallel_replicas_row_estimation	2023-10-16 15:07:11 +02:00
Igor Nikonov	fdd7bfbb83	Merge remote-tracking branch 'origin/master' into pr-progress-bar	2023-10-16 12:59:00 +00:00
alesapin	3b02748cb6	Fix some typos	2023-10-15 15:43:02 +02:00
Duc Canh Le	285ae778e4	Merge branch 'master' into final_no_copy ... Fix flaky test 02447_drop_database_replica	2023-10-14 03:34:42 +00:00
Anton Popov	1c9d883d8a	Merge remote-tracking branch 'upstream/master' into HEAD	2023-10-13 17:17:01 +00:00
Alexander Tokmakov	e3e105d154	Merge pull request #55527 from azat/values-eof-check-fix ... Fix checking of non handled data for Values format	2023-10-13 18:07:02 +02:00
yariks5s	cb08da617f	added read and parse impl	2023-10-13 15:16:07 +00:00
Dmitry Novik	d4be706ed4	Merge remote-tracking branch 'origin/master' into parallel-window	2023-10-13 13:20:38 +00:00
Raúl Marín	500dab9569	Merge remote-tracking branch 'blessed/master' into parallel_replicas_row_estimation	2023-10-13 10:12:57 +02:00
Duc Canh Le	f7830bc13a	some low level optimization ... Signed-off-by: Duc Canh Le <duccanh.le@ahrefs.com>	2023-10-13 07:41:49 +00:00
Anton Popov	a01acf5d2a	remove projection from StorageSnapshot	2023-10-13 03:18:43 +00:00
Alexey Milovidov	8a1363bcf1	Merge pull request #49486 from bigo-sg/test_hive_null_as_default ... Set defaults_for_omitted_fields to true for hive text format	2023-10-13 02:01:09 +02:00
Anton Popov	fb666e137c	remove old code of projection analysis	2023-10-12 23:25:20 +00:00
Dmitry Novik	cec02c60e8	Merge remote-tracking branch 'origin/master' into parallel-window	2023-10-12 22:45:30 +00:00
Igor Nikonov	b220df93df	Remove unnecessary total rows update in parallel replicas	2023-10-12 21:33:25 +00:00
yariks5s	9ae025d7e6	mid commit	2023-10-12 17:37:59 +00:00
Alexey Milovidov	b53a98217c	Merge pull request #55489 from kitaisreal/external-aggregation-improve-performance ... Improve performance of external aggregation with a lot of temporary files	2023-10-12 18:00:41 +02:00
Alexey Milovidov	e48fad0e24	Merge pull request #55495 from kitaisreal/aggregating-transform-init-generate-race-condition-fix ... AggregatingTransform initGenerate race condition fix	2023-10-12 17:54:36 +02:00
Dmitry Novik	f31a6f2c0d	Support only Full Sort	2023-10-12 14:49:16 +00:00
Han Fei	42dc61abf5	Merge branch 'master' into hanfei/statistic	2023-10-12 16:19:45 +02:00
Azat Khuzhin	2cbb069b68	Add ability to ignore data after semicolon in Values format ... This is required for client, to handle comments in multiquery mode. v0: separate context for input format v2: cannot use separate context since params and stuff are changed in global context v3: do not sent this setting to the server (breaks queries for readonly profiles) Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>	2023-10-12 14:55:26 +02:00
Antonio Andelic	22f5d6d708	Reset promise_to_build	2023-10-12 10:57:21 +00:00
Azat Khuzhin	f379d9cac5	Fix checking of non handled data for Values format ... PeekableReadBuffer::hasUnreadData() does not checks the underlying buffer, and so it simply ignore some issues, like: INSERT INTO test_01179_str values ('foo'); ('bar') Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>	2023-10-12 12:25:08 +02:00
Duc Canh Le	b3f4daf6c8	avoid casting columns ... Signed-off-by: Duc Canh Le <duccanh.le@ahrefs.com>	2023-10-12 07:40:41 +00:00
Duc Canh Le	93a4d69c05	combine FINAL selection and WHERE filter ... Co-authored-by: Joris Giovannangeli <joris.giovannangeli@ahrefs.com> Signed-off-by: Duc Canh Le <duccanh.le@ahrefs.com>	2023-10-12 04:58:42 +00:00
taiyang-li	7cd24d0af0	solve conflicts	2023-10-12 10:30:05 +08:00
Robert Schulze	7b14392675	Merge pull request #55481 from rschu1ze/better-use-mysql-types-in-show-columns ... Make `use_mysql_types_in_show_columns` affect only `SHOW COLUMNS`	2023-10-12 00:36:46 +02:00
Maksim Kita	ae94055a29	Updated implementation	2023-10-11 19:01:00 +03:00
Maksim Kita	f9033bdf31	Improve performance of external aggregation with a lot of temporary files	2023-10-11 19:01:00 +03:00
Igor Nikonov	62060a0603	Merge remote-tracking branch 'origin/master' into pr-coordinator-usage-cleanup	2023-10-11 15:11:15 +00:00
Robert Schulze	97d8e16e8d	Fix MySQL packet row data type ... Cf. https://github.com/ClickHouse/ClickHouse/pull/55479#discussion_r1355175581	2023-10-11 15:09:50 +00:00
Igor Nikonov	9d95f4e1b6	Cleanup: parallel replica coordinator usage	2023-10-11 15:04:59 +00:00
robot-ch-test-poll1	80aa9cad51	Merge pull request #55479 from slvrtrn/fix-mysql-datetime-incompatibilities ... Fix MySQL text protocol DateTime formatting and LowCardinality(Nullable(T)) types reporting	2023-10-11 16:56:26 +02:00
Raúl Marín	0b9bd809e7	Merge remote-tracking branch 'blessed/master' into parallel_replicas_row_estimation	2023-10-11 16:50:22 +02:00
Maksim Kita	0859fc8de9	AggregatingTransform initGenerate race condition fix	2023-10-11 14:28:22 +03:00
Duc Canh Le	dfc8793b5c	refactor implementation ... Co-authored-by: Joris Giovannangeli <joris.giovannangeli@ahrefs.com> Signed-off-by: Duc Canh Le <duccanh.le@ahrefs.com>	2023-10-11 10:30:38 +00:00
Duc Canh Le	c6aa438bad	pass all stateless tests ... Co-authored-by: Joris Giovannangeli <joris.giovannangeli@ahrefs.com> Signed-off-by: Duc Canh Le <duccanh.le@ahrefs.com>	2023-10-11 06:51:42 +00:00
slvrtrn	e06d3ca1a5	Fix MySQL text protocol DateTime ... Introduce `removeLowCardinalityAndNullable` function Fix incorrect removeLowCar/removeNullable usages Add more MySQL text protocol tests Deprecate old Java client tests Use JDK 17 for test MySQL Java container	2023-10-10 19:51:09 +02:00
Raúl Marín	95d2063e91	Merge remote-tracking branch 'blessed/master' into parallel_replicas_row_estimation	2023-10-10 17:29:45 +02:00
Nikita Mikhaylov	4456fe40f9	Remove the old code for projection analysis (#55112)	2023-10-10 17:13:32 +02:00
Duc Canh Le	867ac957bf	wip: pass all ci tests ... Signed-off-by: Duc Canh Le <duccanh.le@ahrefs.com>	2023-10-10 11:31:48 +00:00
Alexey Milovidov	e204b1d616	Merge pull request #55419 from evillique/mongo-fix ... Fix MongoDB connection issues	2023-10-10 01:30:04 +02:00
Nikolay Degterinsky	153b542b29	Small fix	2023-10-09 15:13:45 +00:00
Nikolay Degterinsky	a1e56a3df1	Fix MongoDB connection issues	2023-10-09 14:46:48 +00:00
Han Fei	fcb19b4f77	Merge branch 'master' into hanfei/statistic	2023-10-09 14:24:58 +02:00
Alexey Milovidov	e0258582ae	Merge pull request #55338 from azat/fix-CreatingSetsTransform-data-race ... Fix data-race in CreatingSetsTransform (on errors) due to throwing shared exception	2023-10-09 13:27:10 +02:00
Alexey Milovidov	fac83efdd2	Merge pull request #55335 from azat/fix-sort-sigsegv ... Fix SIGSEGV in MergeSortingPartialResultTransform (due to zero chunks after remerge())	2023-10-09 02:48:12 +02:00
Azat Khuzhin	56dbba1f95	Fix SIGSEGV in MergeSortingPartialResultTransform (due to zero chunks after remerge()) ... It is possible to have non zero input chunks before remerge() and zero after: 2023.10.08 10:21:20.944928 [ 4321 ] {427df456-1400-4fbe-8bd7-c4de139f00ca} <Debug> MergeSortingTransform: Re-merging intermediate ORDER BY data (1 blocks with 0 rows) to save memory consumption 2023.10.08 10:36:16.447001 [ 14466 ] {} <Fatal> BaseDaemon: (version 23.10.1.1, build id: 3E53B6B3A53FD562F44C88703BD88EB713881A44, git hash: 5ddfb170a1096cf88664b1a4b9b7bd2e7ef36c29) (from thread 4321) (query_id: 427df456-1400-4fbe-8bd7-c4de139f00ca) (query: SELECT CounterID FROM test.visits ARRAY JOIN Goals.ID WHERE CounterID = 942285 ORDER BY CounterID CI: https://s3.amazonaws.com/clickhouse-test-reports/55276/099665478df3e77d1df0332dd705ca7209e903af/stress_test__debug_.html Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>	2023-10-08 22:21:01 +02:00
Alexey Milovidov	fa54b21424	Merge pull request #55103 from ClickHouse/fix_shell_command_abort ... Proper cleanup in case of exception in ctor of ShellCommandSource	2023-10-08 21:36:28 +02:00
Azat Khuzhin	cd3c140dd6	Fix data-race in CreatingSetsTransform (on errors) due to throwing shared exception ... WARNING: ThreadSanitizer: data race (pid=3436) Read of size 1 at 0x7b50001b07a7 by thread T187: 0 std::__1::basic_string<>::__is_long[abi:v15000] const build_docker/./contrib/llvm-project/libcxx/include/string:1499:33 (clickhouse-tsan+0x1faa29ff) (BuildId: f978f51991d62a36512b8fc6952335a82292896f) 1 std::__1::basic_string<>::size[abi:v15000]() const build_docker/./contrib/llvm-project/libcxx/include/string:968:17 (clickhouse-tsan+0x1faa29ff) 2 std::__1::basic_string<>::empty[abi:v15000]() const build_docker/./contrib/llvm-project/libcxx/include/string:996:42 (clickhouse-tsan+0x1faa29ff) 3 Poco::Exception::extendedMessage() build_docker/./base/poco/Foundation/src/Exception.cpp:114:13 (clickhouse-tsan+0x1faa29ff) 4 DB::Exception::addMessage() <null> (clickhouse-tsan+0x7e0d5f1) (BuildId: f978f51991d62a36512b8fc6952335a82292896f) 5 DB::executeJob(DB::ExecutingGraph::Node*, DB::ReadProgressCallback*) build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:70:23 (clickhouse-tsan+0x1b3f2fed) (BuildId: f978f51991d62a36512b8fc6952335a82292896f) 6 DB::ExecutionThreadContext::executeTask() build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:95:9 (clickhouse-tsan+0x1b3f2fed) 7 DB::PipelineExecutor::executeStepImpl(unsigned long, std::__1::atomic<bool>*) build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273:26 (clickhouse-tsan+0x1b3e5a10) (BuildId: f978f51991d62a36512b8fc6952335a82292896f) 8 DB::PipelineExecutor::executeSingleThread(unsigned long) build_docker/./src/Processors/Executors/PipelineExecutor.cpp:239:5 (clickhouse-tsan+0x1b3e4bb8) (BuildId: f978f51991d62a36512b8fc6952335a82292896f) 9 DB::PipelineExecutor::executeImpl(unsigned long, bool) build_docker/./src/Processors/Executors/PipelineExecutor.cpp:399:9 (clickhouse-tsan+0x1b3e4bb8) 10 DB::PipelineExecutor::execute(unsigned long, bool) build_docker/./src/Processors/Executors/PipelineExecutor.cpp:111:9 (clickhouse-tsan+0x1b3e48be) (BuildId: f978f51991d62a36512b8fc6952335a82292896f) 11 DB::CompletedPipelineExecutor::execute() build_docker/./src/Processors/Executors/CompletedPipelineExecutor.cpp:110:18 (clickhouse-tsan+0x1b3e31c8) (BuildId: f978f51991d62a36512b8fc6952335a82292896f) 12 DB::FutureSetFromSubquery::buildOrderedSetInplace(std::__1::shared_ptr<DB::Context const> const&) build_docker/./src/Interpreters/PreparedSets.cpp:202:14 (clickhouse-tsan+0x19a58396) (BuildId: f978f51991d62a36512b8fc6952335a82292896f) Previous write of size 8 at 0x7b50001b07a0 by thread T192: 0 std::__1::basic_string<>::__set_long_cap[abi:v15000](unsigned long) build_docker/./contrib/llvm-project/libcxx/include/string:1592:37 (clickhouse-tsan+0x238809e2) (BuildId: f978f51991d62a36512b8fc6952335a82292896f) 1 std::__1::basic_string<>::__grow_by_and_replace(unsigned long, unsigned long, unsigned long, unsigned long, unsigned long, unsigned long, char const*) build_docker/./contrib/llvm-project/libcxx/include/string:2367:5 (clickhouse-tsan+0x238809e2) 2 std::__1::basic_string<>::append(char const*, unsigned long) build_docker/./contrib/llvm-project/libcxx/include/string:2692:9 (clickhouse-tsan+0x238809e2) 3 std::__1::basic_string<>::append(char const*) build_docker/./contrib/llvm-project/libcxx/include/string:2849:12 (clickhouse-tsan+0x1faa2a58) (BuildId: f978f51991d62a36512b8fc6952335a82292896f) 4 Poco::Exception::extendedMessage() build_docker/./base/poco/Foundation/src/Exception.cpp:114:27 (clickhouse-tsan+0x1faa2a58) 5 DB::Exception::addMessage() <null> (clickhouse-tsan+0x7e0d5f1) (BuildId: f978f51991d62a36512b8fc6952335a82292896f) 6 DB::executeJob(DB::ExecutingGraph::Node*, DB::ReadProgressCallback*) build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:70:23 (clickhouse-tsan+0x1b3f2fed) (BuildId: f978f51991d62a36512b8fc6952335a82292896f) 7 DB::ExecutionThreadContext::executeTask() build_docker/./src/Processors/Executors/ExecutionThreadContext.cpp:95:9 (clickhouse-tsan+0x1b3f2fed) 8 DB::PipelineExecutor::executeStepImpl(unsigned long, std::__1::atomic<bool>*) build_docker/./src/Processors/Executors/PipelineExecutor.cpp:273:26 (clickhouse-tsan+0x1b3e5a10) (BuildId: f978f51991d62a36512b8fc6952335a82292896f) 9 DB::PipelineExecutor::executeSingleThread(unsigned long) build_docker/./src/Processors/Executors/PipelineExecutor.cpp:239:5 (clickhouse-tsan+0x1b3e4bb8) (BuildId: f978f51991d62a36512b8fc6952335a82292896f) 10 DB::PipelineExecutor::executeImpl(unsigned long, bool) build_docker/./src/Processors/Executors/PipelineExecutor.cpp:399:9 (clickhouse-tsan+0x1b3e4bb8) 11 DB::PipelineExecutor::execute(unsigned long, bool) build_docker/./src/Processors/Executors/PipelineExecutor.cpp:111:9 (clickhouse-tsan+0x1b3e48be) (BuildId: f978f51991d62a36512b8fc6952335a82292896f) 12 DB::CompletedPipelineExecutor::execute() build_docker/./src/Processors/Executors/CompletedPipelineExecutor.cpp:110:18 (clickhouse-tsan+0x1b3e31c8) (BuildId: f978f51991d62a36512b8fc6952335a82292896f) 13 DB::FutureSetFromSubquery::buildOrderedSetInplace(std::__1::shared_ptr<DB::Context const> const&) build_docker/./src/Interpreters/PreparedSets.cpp:202:14 (clickhouse-tsan+0x19a58396) (BuildId: f978f51991d62a36512b8fc6952335a82292896f) Location is heap block of size 480 at 0x7b50001b0600 allocated by thread T193: 0 posix_memalign <null> (clickhouse-tsan+0x7d6ac85) (BuildId: f978f51991d62a36512b8fc6952335a82292896f) 1 std::__1::__libcpp_aligned_alloc[abi:v15000](unsigned long, unsigned long) build_docker/./contrib/llvm-project/libcxxabi/../libcxx/include/new:328:9 (clickhouse-tsan+0x238b5a05) (BuildId: f978f51991d62a36512b8fc6952335a82292896f) 2 __cxxabiv1::__aligned_malloc_with_fallback(unsigned long) build_docker/./contrib/llvm-project/libcxxabi/src/fallback_malloc.cpp:215:20 (clickhouse-tsan+0x238b5a05) 3 __cxa_allocate_exception build_docker/./contrib/llvm-project/libcxxabi/src/cxa_exception.cpp:190:17 (clickhouse-tsan+0x238b31e5) (BuildId: f978f51991d62a36512b8fc6952335a82292896f) 4 std::exception_ptr std::make_exception_ptr[abi:v15000]<DB::Exception>(DB::Exception) build_docker/./contrib/llvm-project/libcxx/include/exception:206:9 (clickhouse-tsan+0x144bfa8a) (BuildId: f978f51991d62a36512b8fc6952335a82292896f) 5 DB::CreatingSetsTransform::~CreatingSetsTransform() build_docker/./src/Processors/Transforms/CreatingSetsTransform.cpp:30:45 (clickhouse-tsan+0x1b7603c3) (BuildId: f978f51991d62a36512b8fc6952335a82292896f) ... 35 DB::FutureSetFromSubquery::buildOrderedSetInplace(std::__1::shared_ptr<DB::Context const> const&) build_docker/./src/Interpreters/PreparedSets.cpp:211:1 (clickhouse-tsan+0x19a58608) (BuildId: f978f51991d62a36512b8fc6952335a82292896f)	2023-10-08 17:16:39 +02:00
Azat Khuzhin	099665478d	Fix incorrect merging of Nested for SELECT FINAL FROM SummingMergeTree ... The problem was the order of the columns, in case of SELECT FINAL it got "counters_Map.count", "counters_Map.id" But in case of OPTIMIZE FINAL it got "counters_Map.id", "counters_Map.count" correctly. Note, that this bugs exists there from the very recent versions, I've checked 19.x and it was there. P.S. there is a workaround for this problem, if you will use one of the following patterns for key columns: - *ID - *Key - *Type That way it will be explicitly matched as key and everything will work. Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>	2023-10-08 07:32:47 +02:00
Duc Canh Le	c1b0982a56	finalize implementation ... Signed-off-by: Duc Canh Le <duccanh.le@ahrefs.com>	2023-10-06 08:32:46 +00:00
Duc Canh Le	c24c2c712e	Merge branch 'master' into final_no_copy ... - resolve conflicts Signed-off-by: Duc Canh Le <duccanh.le@ahrefs.com>	2023-10-06 08:20:37 +00:00
Nikolai Kochetov	3bd71c937c	Merge branch 'master' into fix-read-in-order-with-array-join	2023-10-04 09:17:09 +00:00
Nikolai Kochetov	d944b59902	Merge branch 'master' into planner-prepare-filters-for-analysis-2	2023-10-03 14:28:16 +00:00
vdimir	3a9abde35d	Merge pull request #54514 from ClickHouse/vdimir/allow_experimental_partial_result ... Add setting allow_experimental_partial_result	2023-09-29 10:32:57 +02:00
Alexander Gololobov	d81d5dd75c	Cleanup in case of exception in ctor	2023-09-28 16:28:35 +02:00
Nikita Taranov	0e506b618e	impl (#54934)	2023-09-28 14:12:19 +02:00
vdimir	a134569bc3	fix MergeSortingPartialResultTransform::getRealProcessorSnapshot	2023-09-28 09:41:01 +00:00
vdimir	4e86eec7d1	fix Pipe::unitePipes with partial results	2023-09-28 09:41:00 +00:00
vdimir	3f3feea0b7	Add setting allow_experimental_partial_result	2023-09-28 09:40:56 +00:00
robot-clickhouse-ci-1	7a825c1417	Merge pull request #54427 from Avogar/json-object-as-tuple-inference ... Add new features to schema inference for JSON formats	2023-09-27 20:12:45 +02:00
Robert Schulze	2d750e9e2f	Merge remote-tracking branch 'rschu1ze/master' into clang-tidy-reenable-checks	2023-09-27 12:15:29 +00:00
Antonio Andelic	580aa0031c	Merge pull request #55013 from ClickHouse/another-promise-fix ... Set exception for promise in `CreatingSetsTransform` in more cases	2023-09-27 09:01:42 +02:00
Michael Kolupaev	23bc286280	Print more information about one logical error in MergeTreeDataWriter (#55001)	2023-09-26 13:23:26 -07:00
Han Fei	4e7745f757	Merge branch 'master' into hanfei/statistic	2023-09-26 21:43:53 +02:00
Robert Schulze	cde10fe7b5	Merge remote-tracking branch 'rschu1ze/master' into clang-tidy-reenable-checks	2023-09-26 18:59:41 +00:00
Kruglov Pavel	bea80ab5b7	Merge branch 'master' into json-object-as-tuple-inference	2023-09-26 15:23:08 +02:00
Kruglov Pavel	69a17bbef6	Merge pull request #52853 from Avogar/http-valid-json-on-exception ... Output valid JSON/XML on excetpion during HTTP query execution	2023-09-26 14:25:55 +02:00
Robert Schulze	9fff447716	Re-enable clang-tidy checks	2023-09-26 09:34:12 +00:00
Antonio Andelic	098efb261e	Throw proper exception if promise was not set	2023-09-26 09:03:38 +00:00
avogar	9e75825515	Merge branch 'master' of github.com:ClickHouse/ClickHouse into json-object-as-tuple-inference	2023-09-25 17:24:36 +00:00
avogar	42ca897f2d	Better schema inference for JSON formats	2023-09-25 15:42:59 +00:00
Dmitry Novik	d4acd9ec61	Set correct stream sorting	2023-09-25 15:34:35 +00:00
Alex Cheng	d5d3cf0e9d	Merge remote-tracking branch 'origin/master' into refactor-MergeJoinTransform	2023-09-25 19:41:35 +08:00
Alex Cheng	c7a88c8946	refactor MergeJoinTransform code.	2023-09-25 18:04:25 +08:00
robot-ch-test-poll4	9d61ed4665	Merge pull request #54664 from bigo-sg/arrow_join_max_block_size ... Respect max_block_size for array join to avoid possible OOM	2023-09-23 21:30:51 +02:00
robot-ch-test-poll4	ba6f0431a5	Merge pull request #54933 from ClibMouse/feature/big-endian-bson-each-row ... Provide support for BSON on BE	2023-09-23 03:00:27 +02:00
robot-clickhouse-ci-2	d98234dc9d	Merge pull request #54803 from Avogar/ephemeral-columns-from-files ... Forbid special columns for file/s3/url/... storages, fix insert into ephemeral columns from files	2023-09-22 23:24:42 +02:00
kothiga	3e57b007a8	Use LE version of unalignedStore.	2023-09-22 12:25:17 -07:00
robot-clickhouse-ci-1	9a3d1d6296	Merge pull request #54920 from ClickHouse/set-promise-on-exception ... Set exception for promise in `CreatingSetsTransform`	2023-09-22 20:45:09 +02:00
kothiga	80d511093b	Provide support for BSON on BE	2023-09-22 09:21:21 -07:00
Kruglov Pavel	4d675dbad4	Merge pull request #54825 from azat/fix-virtual-columns-filtering ... Fix filtering parts with indexHint for non analyzer (resubmit)	2023-09-22 18:20:16 +02:00
Dmitry Novik	3b0579b339	Fix splitting into buckets	2023-09-22 14:44:40 +00:00
Antonio Andelic	a691dea6cf	Set exception for promise	2023-09-22 13:13:54 +00:00
Robert Schulze	9a91f97ac4	Merge pull request #53831 from rschu1ze/clang-17 ... Switch CI to Clang 17	2023-09-22 12:28:35 +02:00
Igor Nikonov	b1cc698477	Merge pull request #54564 from vitlibar/fix-sorting-of-union-of-sorted ... Fix sorting of UNION ALL of already sorted results	2023-09-21 22:49:53 +02:00
Robert Schulze	877e4f3aab	Merge remote-tracking branch 'rschu1ze/master' into clang-17	2023-09-21 20:21:12 +00:00
Michael Kolupaev	9af9b4a085	Enable connection pooling for s3 table function (#54812) ... Enable connection pooling for s3 table function	2023-09-21 09:27:20 -07:00
Robert Schulze	5209bd2d51	Merge remote-tracking branch 'rschu1ze/master' into clang-17	2023-09-21 14:45:55 +00:00
Robert Schulze	f5137dd0b4	More clang-tidy fixes	2023-09-21 14:40:57 +00:00
avogar	3e08800cb5	Forbid special columns for file/s3/url/... storages, fix insert into ephemeral columns from files	2023-09-20 16:25:55 +00:00
Dmitry Novik	bb6952317c	Merge remote-tracking branch 'origin/master' into parallel-window	2023-09-20 15:38:42 +00:00
Igor Nikonov	2d446fc1ca	Merge remote-tracking branch 'origin/master' into parallel-replicas-not-enough-replicas	2023-09-20 13:40:08 +00:00
Kruglov Pavel	49ee14f701	Merge pull request #54809 from ClickHouse/pqmeta ... Prevent ParquetMetadata reading 40 MB from each file unnecessarily	2023-09-20 12:53:22 +02:00
SmitaRKulkarni	135df6870c	Merge pull request #54706 from ClickHouse/replacing_merge_tree_with_clean_deleted_rows_disabled ... Fix ReplacingMergeTree with vertical merge and cleanup	2023-09-20 12:06:55 +02:00
SmitaRKulkarni	d8adf05de2	Added a new column _block_number (#47532) ... Added a new virtual column _block_number which is persisted on merges when allow_experimental_block_number_column is enabled	2023-09-20 11:31:12 +02:00
Azat Khuzhin	d9a634eb0f	Fix filtering parts with indexHint for non analyzer ... Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com> (cherry picked from commit ffa82e9297)	2023-09-20 11:29:35 +02:00
Michael Kolupaev	c856ec4087	Prevent ParquetMetadata reading 40 MB from each file unnecessarily	2023-09-19 21:58:50 +00:00
Igor Nikonov	e28f4e819c	Merge branch 'master' into parallel-replicas-not-enough-replicas	2023-09-19 23:05:24 +02:00
Azat Khuzhin	c439c4bca2	Revert "Fix filtering parts with indexHint for non analyzer"	2023-09-19 21:39:21 +02:00
Han Fei	eb86c113e5	Merge pull request #54184 from priera/fix_mv_insertion_when_duplicated_parts ... Inserting only non-duplicate chunks in MV	2023-09-19 20:41:45 +02:00
robot-ch-test-poll4	a3dcce21b3	Merge pull request #54405 from Avogar/json-or-jsonl ... Parse data in JSON format as JSONEachRow if failed to parse metadata	2023-09-19 20:15:09 +02:00
Kruglov Pavel	d555fdffa5	Merge pull request #54449 from azat/parts-prune-indexHint ... Fix filtering parts with indexHint for non analyzer	2023-09-19 19:15:41 +02:00
Han Fei	17243b84ab	Merge branch 'master' into hanfei/statistic	2023-09-19 17:09:17 +02:00
Smita Kulkarni	e2887ad27b	Merge branch 'master' into replacing_merge_tree_with_clean_deleted_rows_disabled	2023-09-19 15:58:41 +02:00
avogar	f974970c3c	Apply suggestion	2023-09-19 11:53:40 +00:00
avogar	5bd2e9f610	Fix tests	2023-09-19 11:53:40 +00:00
avogar	8c29408f5e	Parse data in JSON format as JSONEachRow if failed to parse metadata	2023-09-19 11:53:40 +00:00
Kruglov Pavel	e163670357	Merge branch 'master' into http-valid-json-on-exception	2023-09-19 13:42:53 +02:00
Kruglov Pavel	3c83e43351	Remove debug logging	2023-09-19 13:38:43 +02:00
priera	b92e37d2df	Fixing PR comments	2023-09-19 12:15:01 +02:00
priera	663ce74609	fixed clang-tidy error	2023-09-19 12:15:00 +02:00
priera	b30e33580b	fix test	2023-09-19 12:15:00 +02:00
priera	6d27ea2742	fixed style	2023-09-19 12:15:00 +02:00
priera	94d737264c	inserting non-duplicate chunks	2023-09-19 12:15:00 +02:00
Smita Kulkarni	142a4631f1	Updated to clear current_row_sources when cleanedup rows are added	2023-09-19 11:22:10 +02:00
Igor Nikonov	047d214436	Merge remote-tracking branch 'origin/master' into parallel-replicas-not-enough-replicas	2023-09-18 15:29:56 +00:00
Igor Nikonov	e1019ba3c4	Disabling parallel replicas per shard will be done separately	2023-09-18 15:27:55 +00:00
Kruglov Pavel	9c888ea42b	Merge pull request #53549 from Avogar/group-by-constant-keys ... Optimize group by constant keys	2023-09-18 12:12:40 +02:00
taiyang-li	39ca44b51a	Respect max_block_size for array join to avoid possible OOM	2023-09-18 11:04:38 +08:00
Robert Schulze	f5e8028bb1	Merge pull request #54642 from rschu1ze/broken-re2st ... Remove broken lockless variant of re2	2023-09-17 15:30:57 +02:00
Amos Bird	0518b64b58	Fix nullable primary key in final (#54164) ... * Fix nullable primary key in final * Real fix * Address reviews	2023-09-15 22:44:13 +02:00
Smita Kulkarni	8ef910e1fa	Updated to check if cleanup is enabled before adding to cleanedup_rows_count	2023-09-15 22:34:09 +02:00
Kruglov Pavel	2075f9c667	Merge branch 'master' into group-by-constant-keys	2023-09-15 15:10:08 +02:00