5370 Commits

Author	SHA1	Message	Date
Arthur Passos	1b3c7766f8	Merge branch 'master' into add_select_final_mt_setting	2023-01-12 08:31:04 -03:00
Anton Popov	8bdf63f8e5	Merge pull request #45191 from ClickHouse/fix-ip-part-name ... Fix part ID generation for IP types for backward compatibility	2023-01-12 12:15:06 +01:00
Igor Nikonov	edbcb24da0	Merge pull request #44954 from ClickHouse/fix-insert-delay ... Fix: insert delay calculation	2023-01-12 09:43:00 +01:00
HarryLeeIBM	6d576cf157	Added more documentation	2023-01-11 13:40:20 -08:00
Yakov Olkhovskiy	9c4ea5a16b	fix part ID generation for IP types for backward compatibility	2023-01-11 18:51:37 +00:00
Alexander Tokmakov	0fba3d6d81	Merge pull request #44468 from ClickHouse/fix_get_part_name ... Fix incorrect usages of `getPartName()`	2023-01-11 16:24:02 +03:00
Igor Nikonov	1d002e45a9	Fix test & review comments	2023-01-11 12:49:01 +00:00
Nikolai Kochetov	5e7a6ac619	Merge pull request #45122 from ClickHouse/revert-45121-revert-44653-custom-reading-for-mutation ... Revert "Revert "Custom reading for mutation""	2023-01-11 12:37:32 +01:00
Nikolai Kochetov	d0c1079242	Merge pull request #45133 from ClickHouse/fix-additinal-tables-filter-with-count-projection ... Fix additional_table_filters with minmax/count projection.	2023-01-11 12:25:40 +01:00
Anton Popov	85734bb321	fix build	2023-01-11 08:32:33 +00:00
Anton Popov	6c4f53bdca	Merge remote-tracking branch 'upstream/master' into HEAD	2023-01-11 01:00:57 +00:00
HarryLeeIBM	150f9b48a4	Fix functional test for full text search	2023-01-10 14:35:06 -08:00
Igor Nikonov	543221b8b9	Merge remote-tracking branch 'origin/master' into fix-insert-delay	2023-01-10 22:15:41 +00:00
Igor Nikonov	e75df5e76b	Test ... + fix some review comments	2023-01-10 22:14:54 +00:00
Alexander Tokmakov	881b17492f	Merge branch 'master' into fix_get_part_name	2023-01-10 21:39:35 +01:00
Alexander Tokmakov	255881a2ff	Merge pull request #44466 from ClickHouse/parallel_parts_cleanup_with_zero_copy ... Parallel parts cleanup with zero copy replication	2023-01-10 22:44:39 +03:00
Ilya Yatsishin	0efceae6ce	Merge pull request #44917 from azat/mt-fix-log-message ... Fix logging message in MergeTreeDataMergerMutator (about merged parts)	2023-01-10 19:49:31 +01:00
Nikolai Kochetov	9a81f27fb2	Fix additional_table_filters with minmax/count projection.	2023-01-10 17:32:26 +00:00
HarryLeeIBM	bc48b17817	Addressed more review comments.	2023-01-10 08:26:27 -08:00
Nikolai Kochetov	4673b3fe1d	Revert "Revert "Custom reading for mutation""	2023-01-10 16:31:01 +01:00
Alexander Tokmakov	c8ec130be4	Revert "Custom reading for mutation"	2023-01-10 17:51:30 +03:00
Han Fei	bef6168c05	support cache for async inserts block ids	2023-01-10 13:19:12 +01:00
Anton Popov	b9bf92fa5d	check for network errors while loading parts	2023-01-10 11:55:08 +00:00
Nikolai Kochetov	11418963c0	Merge pull request #44653 from ClickHouse/custom-reading-for-mutation ... Custom reading for mutation	2023-01-10 12:16:24 +01:00
alesapin	616e37ba43	Merge pull request #44847 from ClickHouse/fix_rare_race ... Fix rare race which can lead to queue hang	2023-01-10 12:09:10 +01:00
Nikolai Kochetov	3e430b1b6e	Add comments. Remove unused flag from Context.	2023-01-09 19:52:11 +00:00
Anton Popov	1e5d6e44a5	fix deadlock at shutdown	2023-01-09 13:07:46 +00:00
Sergei Trifonov	81d2ea30ba	Merge branch 'master' into cancellable-mutex-integration	2023-01-07 19:37:46 +01:00
Anton Popov	1f32ffedf8	Merge pull request #43221 from ClickHouse/refactoring-ip-types ... Replace domain IP types (IPv4, IPv6) with native	2023-01-07 12:01:21 +01:00
Alexey Milovidov	9fcd111724	Merge pull request #45003 from amosbird/issue_44979 ... Fix minmax_count_projection with _partition_value	2023-01-07 03:36:56 +03:00
serxa	15bb127b01	replace every std::shared_mutex with DB::FastSharedMutex	2023-01-06 23:35:38 +00:00
Amos Bird	200d40c53d	Fix minmax_count_projection with _partition_value ... Check if virtual column `_partition_value` can be used when building minmax_count_projection block.	2023-01-07 05:38:28 +08:00
Dmitry Novik	9d44c7a737	Merge pull request #44541 from ClickHouse/cleanup-resolved-function ... Followup #43761	2023-01-06 18:09:32 +01:00
kssenii	8fde606768	Resolve some review comments	2023-01-06 16:10:00 +01:00
kssenii	2a030c1dc0	Merge remote-tracking branch 'upstream/master' into allow-single-disk-instead-of-storage-policy	2023-01-06 15:58:27 +01:00
alesapin	da83e97b4c	Fix startup issue	2023-01-06 15:36:53 +01:00
Anton Popov	b25f875674	Merge pull request #44875 from ClickHouse/fix-another-one-cannot-read-all-data-for-lc-dict-error ... Fix right offset for reading LowCardinality dictionary from remote fs	2023-01-06 15:24:36 +01:00
alesapin	b20f07b2cc	Merge branch 'master' into fix_rare_race	2023-01-06 14:20:08 +01:00
Anton Popov	fcdbc8123d	better loading of outdated parts	2023-01-06 11:59:54 +00:00
Alexey Milovidov	b88be9df8c	Merge pull request #44956 from ClickHouse/fix_read_result_consistency_check_on_exception ... Do not check read result consistency when unwinding	2023-01-06 06:22:42 +03:00
Alexey Milovidov	a205b41c56	Merge pull request #44947 from Algunenano/background_pool_size_logical ... Change error code on invalid background_pool_size config	2023-01-06 03:19:40 +03:00
Alexander Gololobov	1b31b8bb91	Do not check read result consistency when unwinding	2023-01-06 00:26:08 +01:00
Igor Nikonov	a67afdff6a	Fix: insert delay calculation	2023-01-05 22:43:41 +00:00
Dmitry Novik	ebead611b6	Merge branch 'master' into cleanup-resolved-function	2023-01-05 19:27:08 +01:00
Raúl Marín	b0715ec5eb	Change error code on invalid background_pool_size config	2023-01-05 18:47:02 +01:00
Anton Popov	ed00399eb6	Merge remote-tracking branch 'upstream/master' into HEAD	2023-01-05 14:12:02 +00:00
Larry Luo	f2bc5bb335	Added initializer	2023-01-05 05:41:29 -08:00
Igor Nikonov	2a105d1862	Fix stupid mistake - max() -> min()	2023-01-05 12:38:27 +00:00
Igor Nikonov	db46267a95	Fast fix: force upper bound for time to delay INSERT	2023-01-05 12:38:27 +00:00
Larry Luo	fd0c6cccc8	Fixed style error	2023-01-04 20:08:28 -08:00
Larry Luo	7a4d499cb1	Added density to control Inverted index size	2023-01-04 19:42:45 -08:00
Azat Khuzhin	d06ac2d468	Fix logging message in MergeTreeDataMergerMutator (about merged parts) ... Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>	2023-01-04 19:07:01 +01:00
Yakov Olkhovskiy	7a5a36cbed	Merge branch 'master' into refactoring-ip-types	2023-01-04 11:11:06 -05:00
alesapin	2cd7247619	Merge branch 'master' into fix_rare_race	2023-01-04 15:06:20 +01:00
Kruglov Pavel	9056925c7d	Merge pull request #44874 from Algunenano/time_fix ... Do not throw DB::Exception when folders do not exist	2023-01-04 01:17:57 +01:00
Nikolai Kochetov	c51f5e6fac	Fix typo.	2023-01-03 18:41:37 +00:00
Nikolai Kochetov	da26f62a9b	Fix right offset for reading LowCardinality dictionary from remote fs in case if right mark was in the middle of compressed block.	2023-01-03 18:19:51 +00:00
Dmitry Novik	a84318af5f	Merge branch 'master' into cleanup-resolved-function	2023-01-03 17:52:29 +01:00
Raúl Marín	0785bf6b29	Do not throw DB::Exception when folders do not exist	2023-01-03 17:05:54 +01:00
Nikolai Kochetov	6b1a697b12	Merge pull request #44850 from ClickHouse/respect-context-in-projection-optimization ... Save context created from projection analysis.	2023-01-03 13:30:51 +01:00
Han Fei	7d7de3833b	Merge pull request #44651 from hanfei1991/hanfei/fix-async-insert-cleanup ... fix bug that async blocks cleanup not work	2023-01-02 21:58:27 +01:00
Dmitry Novik	929300a721	Update MergeTreeData.cpp	2023-01-02 21:18:16 +01:00
Dmitry Novik	04fb410a40	Merge branch 'master' into parallel_parts_cleanup_with_zero_copy	2023-01-02 20:48:39 +01:00
Dmitry Novik	c646048af9	Merge pull request #44461 from ClickHouse/parts-removal-limit ... Add part removal limit for one CleanupThread iteration	2023-01-02 20:04:19 +01:00
Alexey Milovidov	e855d3519a	Merge branch 'master' into refactoring-ip-types	2023-01-02 21:58:53 +03:00
Nikolai Kochetov	09325ff17e	Store context created from projection analysis.	2023-01-02 17:00:48 +00:00
alesapin	6fc63878b4	Fix rare race which can lead to queue hang	2023-01-02 16:57:25 +01:00
Han Fei	eef0136415	add debug log	2023-01-02 11:59:44 +01:00
Larry Luo	7cc84c6743	refactor filter functions support	2022-12-31 18:05:07 -08:00
Larry Luo	71da1b1379	Updated inverted index to use ActionsDAG	2022-12-31 18:05:06 -08:00
Larry Luo	f0a2c06261	Fixed errors caused by merging.	2022-12-31 18:05:06 -08:00
HarryLeeIBM	d7a7de9601	Minor fixes for addressing reviews	2022-12-31 18:05:06 -08:00
HarryLeeIBM	6a5fef632d	Added more tests and experimental flag, etc.	2022-12-31 18:05:01 -08:00
Harry-Lee	b43ddde50a	Refactor full text search according to reviews	2022-12-31 18:03:14 -08:00
Larry Luo	00e4411740	Created FST namespace to hide FST details	2022-12-31 18:03:14 -08:00
Larry Luo	64f807f31a	fixed an edge case that index column has no data	2022-12-31 18:03:14 -08:00
Larry Luo	da93210cc6	Updated MergeTreeConditionGinFilter due to SelectQueryInfo changes.	2022-12-31 18:03:14 -08:00
Larry Luo	a33232cb0a	Clean up code	2022-12-31 18:03:14 -08:00
Larry Luo	7e5f311da9	Fix MemorySanitizer error	2022-12-31 18:03:14 -08:00
Larry Luo	0a47378e5f	Added initializer for current segment	2022-12-31 18:03:14 -08:00
Larry Luo	514e17d037	Code clean up	2022-12-31 18:03:14 -08:00
Larry Luo	1d9bf9962c	Code clean up	2022-12-31 18:03:14 -08:00
Larry Luo	657ce7c614	Cleanup code and tests	2022-12-31 18:03:13 -08:00
Larry Luo	3652740535	Code clean up	2022-12-31 18:03:13 -08:00
Larry Luo	0b63d6b5be	Resolve conflicts and style errors.	2022-12-31 18:03:13 -08:00
HarryLeeIBM	bc8be368bb	Fix coding styles	2022-12-31 18:03:13 -08:00
Larry Luo	91b79b26e3	Use DataPartStorage in GinIndexStore	2022-12-31 18:03:02 -08:00
Harry-Lee	4b29c02f35	Remove useless FUNCTION_LIKE	2022-12-31 18:00:17 -08:00
HarryLeeIBM	e58f0ef3d2	Remove roaring bmp cleanup code which can cause memory leak	2022-12-31 18:00:17 -08:00
HarryLeeIBM	0839ff60ee	Initial Checkin of full text search	2022-12-31 18:00:17 -08:00
Alexey Milovidov	50bb7f90d8	Merge pull request #44809 from ClickHouse/fix-race-in-mergetreedatapart ... Fix race in `system.parts` and `system.parts_columns`	2023-01-01 01:09:46 +03:00
Han Fei	a036c6bc01	Merge branch 'master' into hanfei/fix-async-insert-cleanup	2022-12-31 20:52:16 +01:00
Alexey Milovidov	3a5cea8281	Fix race in system.parts and system.parts_columns	2022-12-31 15:07:26 +01:00
Alexey Milovidov	26eb2f57e1	Fix race in system.parts and system.parts_columns	2022-12-31 15:06:57 +01:00
Azat Khuzhin	e7bf2cc959	Add a comment for lack of fsync in MergedBlockOutputStream::finalizePartOnDisk() ... Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>	2022-12-30 22:39:13 +01:00
Azat Khuzhin	9fb6004c92	Add missing fsync for small files (ttl.txt, columns.txt, ...) in mutations ... Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>	2022-12-30 22:22:39 +01:00
Nikolai Kochetov	758f30cc17	Cleanup.	2022-12-30 17:53:46 +00:00
Han Fei	454f40ab17	fix test	2022-12-30 16:30:33 +01:00
Alexey Milovidov	13b76ad760	Merge pull request #44719 from ClickHouse/fix_attach_thread_deadlock ... Fix deadlock in attach thread	2022-12-30 13:56:18 +03:00
Nikolay Degterinsky	dfe93b5d82	Merge pull request #42284 from Algunenano/perf_experiment ... Performance experiment	2022-12-30 03:14:22 +01:00
Alexander Gololobov	fbae502499	Merge pull request #42126 from ClickHouse/fix_invalid_number_of_rows_in_chunk_with_prewhere ... Fixing "Invalid number of rows in Chunk" with lightweight deletes	2022-12-29 21:46:12 +01:00
alesapin	5dfefb43a3	Fix deadlock in attach thread	2022-12-29 18:52:20 +01:00
Dmitry Novik	81c0db3b67	Reuse FunctionBase for IndexHint	2022-12-29 15:47:43 +00:00
Dmitry Novik	bd480f18c1	Merge branch 'master' into parts-removal-limit	2022-12-29 16:27:54 +01:00
Alexander Gololobov	a02a631d51	Cleanups based on code review	2022-12-29 15:00:42 +01:00
Alexander Gololobov	7df137e460	Replaced asserts with logical errors	2022-12-29 14:33:11 +01:00
alesapin	02c3e708af	Merge pull request #44694 from Algunenano/old_part_improvement ... Improve handling of old parts	2022-12-29 13:14:52 +01:00
alesapin	b1ed48a977	Merge pull request #44628 from ClickHouse/CurtizJ-fix-restart-after-quorum-insert ... Fix restart after quorum insert	2022-12-29 12:51:54 +01:00
Alexander Gololobov	059ec6f747	Cleanups	2022-12-29 01:22:47 +01:00
Alexey Milovidov	4311e6c147	Merge pull request #44683 from CurtizJ/fix-flaky-test-4 ... Fix test `01130_in_memory_parts`	2022-12-29 01:53:24 +03:00
Raúl Marín	32d1662503	Improve handling of old parts	2022-12-28 21:22:40 +01:00
Anton Popov	8d2788445b	fix style check	2022-12-28 19:29:56 +00:00
Raúl Marín	e915ce1e95	Merge remote-tracking branch 'blessed/master' into perf_experiment	2022-12-28 20:15:43 +01:00
alesapin	697617548c	Tiny improvement	2022-12-28 18:59:24 +01:00
Anton Popov	bc122d368a	Merge remote-tracking branch 'upstream/master' into HEAD	2022-12-28 17:45:04 +00:00
Nikolai Kochetov	763fcdc939	Update MergeTreeSequentialSource.cpp	2022-12-28 18:17:40 +01:00
Alexander Gololobov	10a058d138	More cleanups in the logic of applying current step filter and final filter	2022-12-28 18:07:36 +01:00
Alexander Gololobov	fd5d328fae	Test accumulating filters ignoring prewhere_info->need_filter flag	2022-12-28 18:07:36 +01:00
Alexander Gololobov	a7adc0a91b	Cleanups	2022-12-28 18:07:36 +01:00
Alexander Gololobov	b22711baa3	Reset need_filter flag when filter is applied	2022-12-28 18:07:36 +01:00
Alexander Gololobov	ada6422985	Restored old logic for filling _part_offset	2022-12-28 18:07:36 +01:00
Alexander Gololobov	4cebc6f3a4	Cleanups	2022-12-28 18:07:36 +01:00
Alexander Gololobov	13e457c754	Cleanups	2022-12-28 18:07:36 +01:00
Alexander Gololobov	a1cdad4815	Fix for reading _part_offset along with newly added column that has only default value	2022-12-28 18:07:35 +01:00
Alexander Gololobov	ac1549f6b3	Skip filtering if there are no rows after optimize()	2022-12-28 18:07:35 +01:00
Alexander Gololobov	f273f8712d	Avoid filtering same column in block_before_prewhere if it is present in the result	2022-12-28 18:07:35 +01:00
Alexander Gololobov	f3646248c5	Avoid unneeded work if all rows were filtered	2022-12-28 18:07:35 +01:00
Alexander Gololobov	75152ddabb	Apply filter only if needed	2022-12-28 18:07:35 +01:00
Alexander Gololobov	a18850458c	Test applying current filter at each step	2022-12-28 18:07:35 +01:00
Alexander Gololobov	29b5c4af07	Test dirty intermediate changes	2022-12-28 18:07:35 +01:00
Alexander Gololobov	c561acb774	Properly handle low cardinality column as prewhere filter	2022-12-28 18:07:35 +01:00
Alexander Gololobov	aa276b230b	Don't need to save filter and rows_per_granule from previous step	2022-12-28 18:07:35 +01:00
Alexander Gololobov	c4a01cbd5b	Fix for propely cleaning rows_per_granule_original between prewhere steps	2022-12-28 18:07:35 +01:00
Alexander Gololobov	abbb58107c	Fix for "out of bound" in ColumnVector::insertRangeFrom called from shrink()	2022-12-28 18:07:35 +01:00
Alexander Gololobov	79874e8733	Fix for "Invalid number of rows in Chunk"	2022-12-28 18:07:35 +01:00
Alexander Gololobov	bdf51545f7	Added FilterWithCachedCount class instead of caching counts in filter_bytes_map	2022-12-28 18:07:35 +01:00
Azat Khuzhin	31a88d4eae	Fix PREWHERE with row-level filters (when row filter is always true/false) ... In case of row-level filters optimized out, i.e. converted to always true/false, it is possible for MergeTreeRangeReader to reuse incorrect statistics for the filter (countBytesInResultFilter()), and because of this it simply does not apply other filters, since it assume that this filter does not need to filter anything. Fixes: #40956 Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>	2022-12-28 18:07:35 +01:00
Alexander Gololobov	d44392b366	Checking the fix for "Invalid number of rows in Chunk"	2022-12-28 18:07:35 +01:00
Anton Popov	38f0cf9b10	fix test 01130_in_memory_parts	2022-12-28 16:40:31 +00:00
Dmitry Novik	6efd64a61c	Merge branch 'master' into parts-removal-limit	2022-12-28 17:34:48 +01:00
Han Fei	17358f5e98	clean headers	2022-12-28 17:29:07 +01:00
Han Fei	66111ef241	deduplicate async inserts in the same block earlier	2022-12-28 17:25:36 +01:00
Anton Popov	2adf0e9db3	do not wait loading parts at shutdown	2022-12-28 16:07:14 +00:00
alesapin	3a1e9f4a4d	Fix style	2022-12-28 14:08:13 +01:00
Nikolai Kochetov	c203aa054a	Use sequential source for mutations.	2022-12-27 20:28:25 +00:00
Han Fei	4d8513024c	fix bug that async blocks cleanup not work	2022-12-27 20:12:09 +01:00
alesapin	3a7f26ff3a	Split test into two	2022-12-27 16:15:23 +01:00
Raúl Marín	948ce8db93	Revert "Merge pull request #38953 from ClickHouse/add-allocation-ptr-to-trace-log" ... This reverts commit e99849d031, reversing changes made to 7ad3ff8a9e.	2022-12-27 14:28:13 +01:00
alesapin	560223ac19	Merge branch 'fix-restart-after-quorum-insert' of github.com:CurtizJ/ClickHouse into CurtizJ-fix-restart-after-quorum-insert	2022-12-27 13:48:55 +01:00
Alexey Milovidov	bfe895975e	Merge pull request #44575 from ClickHouse/minimal-changes ... Non-significant changes	2022-12-27 15:34:27 +03:00
Alexey Milovidov	464a513f0e	Merge pull request #44615 from ClickHouse/projection-grouping-sets ... Disable projections in presense of any grouping sets, including WITH ROLLUP, WITH CUBE and WITH TOTALS	2022-12-27 15:31:03 +03:00
Alexey Milovidov	1b21cc018e	Merge pull request #44342 from ClickHouse/add-test-29883 ... Add a test for #29883	2022-12-27 14:55:14 +03:00
Raúl Marín	fc1fa82a39	Merge branch 'master' into perf_experiment	2022-12-27 10:51:58 +01:00
Alexey Milovidov	593675a7c5	Disable projections in presense of any grouping sets, including WITH ROLLUP, WITH CUBE and WITH TOTALS	2022-12-26 22:31:41 +01:00
Igor Nikonov	ba88bc17d4	Fix typo	2022-12-26 21:01:00 +00:00
Igor Nikonov	3204f7353b	Fix: handle exceptions from unlockSharedData() during error handling ... + renamePartToTemporary() lambda to reused code	2022-12-26 20:49:04 +00:00
Igor Nikonov	0d428318a9	Merge remote-tracking branch 'origin/master' into fix-restart-after-quorum-insert	2022-12-26 19:18:35 +00:00
Alexey Milovidov	52b306d544	Improve variable name	2022-12-26 18:00:18 +01:00
Alexey Milovidov	71850f6913	Check for bad index_granularity	2022-12-25 16:06:34 +01:00
Alexey Milovidov	f22fc35900	Minimal changes	2022-12-25 14:18:05 +01:00
Nikolai Kochetov	03352a15b6	Use sequential source for mutations.	2022-12-23 22:39:27 +00:00
Alexander Tokmakov	a0668af762	Update MergeTreeData.cpp	2022-12-23 23:06:37 +03:00
Alexander Tokmakov	b2af49a4cf	Merge branch 'master' into parallel_parts_cleanup_with_zero_copy	2022-12-23 20:02:17 +01:00
Alexander Tokmakov	ca989e9212	less runtime format strings	2022-12-23 19:50:34 +01:00
Anton Popov	0722020cae	Merge remote-tracking branch 'upstream/master' into HEAD	2022-12-23 18:15:55 +00:00
alesapin	2fecf420cb	Merge pull request #44513 from ClickHouse/small_polishing ... Minor code polishing	2022-12-23 13:47:26 +01:00
Vladimir C	7482ea54ab	Merge pull request #43972 from ClickHouse/vdimir/tmp-data-in-fs-cache-2	2022-12-23 11:59:27 +01:00
Alexander Tokmakov	6b6504f0d1	Merge branch 'master' into parallel_parts_cleanup_with_zero_copy	2022-12-23 02:04:43 +01:00
Dmitry Novik	6c23721255	Merge pull request #43761 from ClickHouse/refector-function-node ... Refactor FunctionNode	2022-12-22 23:53:27 +01:00
alesapin	a32fab90d5	Merge pull request #44436 from ClickHouse/fix_bug_with_max_task_count_update ... Fix bug with merge/mutate pool size increase	2022-12-22 23:48:54 +01:00
Dmitry Novik	d3502937c7	Update MergeTreeIndexSet	2022-12-22 22:17:18 +00:00
Dmitry Novik	cff882d506	Merge remote-tracking branch 'origin/master' into refector-function-node	2022-12-22 21:34:29 +00:00
alesapin	8d0ad4bbcd	Strange why I have to supress in .cpp	2022-12-22 16:14:19 +01:00
alesapin	fa467b3398	Minor code polishing	2022-12-22 14:31:42 +01:00
Nikolai Kochetov	773de7a8a6	Merge pull request #44472 from CurtizJ/fix-mutations ... Fix mutations with setting `max_streams_for_merge_tree_reading`	2022-12-22 14:05:20 +01:00
alesapin	049103c550	Merge pull request #44495 from azat/mergetree/static-storage-fix ... Do not try to remove WAL/move broken parts for static storage	2022-12-22 12:27:08 +01:00
vdimir	88008726d8	Merge branch 'master' into vdimir/tmp-data-in-fs-cache-2	2022-12-22 10:58:39 +00:00
alesapin	380c6cebe4	Merge pull request #44393 from ClickHouse/small_improvement ... Lock table for share during startup for database ordinary	2022-12-22 11:42:42 +01:00
alesapin	bf188c46a4	Merge branch 'master' into fix_bug_with_max_task_count_update	2022-12-22 11:40:08 +01:00
Maksim Kita	e5ebf20044	Merge pull request #44097 from kitaisreal/analyzer-support-set-index ... Analyzer support Set index	2022-12-22 13:30:15 +03:00
vdimir	d30d25dbbe	Temporary files evict fs cache	2022-12-22 10:22:49 +00:00
Yakov Olkhovskiy	a8cb29da4b	Merge branch 'master' into refactoring-ip-types	2022-12-21 23:56:24 -05:00
alesapin	9b97cd98c8	Update src/Storages/MergeTree/MergeTreeDataMergerMutator.h	2022-12-21 23:27:17 +01:00
alesapin	97d43a6748	Add comment	2022-12-21 23:26:26 +01:00
alesapin	edcd0d95ca	Revert "Remove explicit dependency on fast test to debug the issue" ... This reverts commit 2fe969a2e1.	2022-12-21 23:25:50 +01:00
Azat Khuzhin	1985d321a6	Do not try to remove WAL for static storage ... Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>	2022-12-21 18:09:38 +01:00
Azat Khuzhin	e06c169b43	Do not try to move parts to detached for static storage ... Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>	2022-12-21 18:09:38 +01:00
alesapin	a4fbea2184	Relax synchronization to avoid deadlock	2022-12-21 15:59:28 +01:00
Alexander Tokmakov	4a38a03f68	fix	2022-12-21 15:42:25 +01:00
Anton Popov	4b7f1e0cf2	fix mutations with setting 'max_streams_for_merge_tree_reading'	2022-12-21 01:59:53 +00:00
Dmitry Novik	42812dfa08	Add logging	2022-12-21 00:14:01 +00:00
Alexey Milovidov	a6f50edde3	Merge branch 'master' into add-test-29883	2022-12-21 01:45:18 +03:00
Alexander Tokmakov	cdc3912743	fix incorrect usages of getPartName()	2022-12-20 22:44:27 +01:00
Alexander Tokmakov	5f5335876a	Merge branch 'master' into parallel_parts_cleanup_with_zero_copy	2022-12-21 00:35:21 +03:00
Alexander Tokmakov	3f851d5a09	parallel parts cleanup with zero copy replication	2022-12-20 22:04:26 +01:00
alesapin	2fe969a2e1	Remove explicit dependency on fast test to debug the issue	2022-12-20 22:00:57 +01:00
Dmitry Novik	2ec720c5c4	Add part removal limit for one CleanupThread iteration	2022-12-20 19:15:13 +00:00
Dmitry Novik	4793412887	Merge branch 'master' into refector-function-node	2022-12-20 18:26:19 +01:00
alesapin	f06b311abc	It's really needed	2022-12-20 18:09:38 +01:00
alesapin	2818520572	Fix bug with merge/mutate pool size increase	2022-12-20 14:51:11 +01:00
Anton Popov	56187c5f0e	Merge remote-tracking branch 'upstream/master' into HEAD	2022-12-20 13:37:23 +00:00
alesapin	9d4404ce0d	Merge branch 'master' into small_improvement	2022-12-20 12:12:39 +01:00
alesapin	a8515e6925	Merge branch 'master' into fix_race_background_temp_removal	2022-12-20 12:12:14 +01:00
Raúl Marín	45d27f461b	Merge branch 'master' into perf_experiment	2022-12-20 09:07:48 +00:00
alesapin	c8d9ccf3cd	Lock table for share during startup for database ordinary	2022-12-19 18:51:46 +01:00
alesapin	38fd4e0162	Fix race which can lead to merged parts premature removal	2022-12-19 15:25:01 +01:00
Han Fei	98efee2d75	Merge branch 'master' into hanfei/fix-async-flaky	2022-12-19 14:14:29 +01:00
Han Fei	1cb98eb2d6	address comments	2022-12-19 14:05:50 +01:00
Han Fei	ee250174ab	fix flack test "02481_async_insert_dedup.python"	2022-12-18 22:44:51 +01:00
Alexey Milovidov	3e8dcdc155	Add earlier check	2022-12-18 08:01:43 +01:00
Dmitry Novik	875a24a650	Merge remote-tracking branch 'origin/master' into refector-function-node	2022-12-16 16:07:30 +00:00
Anton Popov	b6f2754afb	Merge remote-tracking branch 'upstream/master' into HEAD	2022-12-16 14:37:34 +00:00
Anton Popov	b7e76e3539	do not hold data parts during insert	2022-12-15 23:57:10 +00:00
Nikolai Kochetov	e99849d031	Merge pull request #38953 from ClickHouse/add-allocation-ptr-to-trace-log ... Track allocation ptr in system.trace_log. Add aggregate function flamegraph	2022-12-15 18:16:05 +01:00
Dmitry Novik	7ad3ff8a9e	Merge pull request #44244 from ClickHouse/part-log-removal-info ... Add more information about part removal into system.parts table	2022-12-15 18:06:55 +01:00
Alexander Tokmakov	688e488e93	Merge pull request #44220 from ClickHouse/correct-readonlyreplica-metric ... Fix `ReadonlyReplica` metric	2022-12-15 19:05:20 +03:00
Dmitry Novik	b87739d31c	Add HAS_SKIPPED_MUTATION_PARENT state	2022-12-15 16:05:15 +00:00
Kruglov Pavel	b63ff31061	Merge pull request #44192 from Avogar/fix-bad-cast ... Fix possible Bad cast from type DB::IAST const* to DB::ASTLiteral const*	2022-12-15 14:02:41 +01:00
Dmitry Novik	5b7a990425	Add more information about part removal into system.parts table	2022-12-14 22:30:43 +00:00
Dmitry Novik	1df038e39c	Merge remote-tracking branch 'origin/master' into refector-function-node	2022-12-14 14:55:30 +00:00
Anton Popov	c6b6b0ad7d	Merge pull request #44173 from CurtizJ/fix-fetch-projection ... Fix `LOGICAL_ERROR` if fetch of projection was stopped	2022-12-14 15:55:26 +01:00
Maksim Kita	9b5ebf88cf	Fixed tests	2022-12-14 15:54:01 +01:00
Anton Popov	3593377f23	Merge pull request #44159 from ClickHouse/prevent-delete-nested-if-empty-part ... Prevent dropping nested column if it creates empty part	2022-12-14 15:04:15 +01:00
kssenii	ea5b06023a	Merge remote-tracking branch 'refs/remotes/upstream/master' into allow-single-disk-instead-of-storage-policy	2022-12-14 13:59:51 +01:00
Antonio Andelic	d0905fe408	Fix typo	2022-12-14 13:43:23 +01:00
Antonio Andelic	ff42a68940	Fix ReadonlyReplica	2022-12-14 12:02:12 +00:00
Alexander Tokmakov	816eba6149	Merge pull request #44202 from ClickHouse/fix_an_assertion_in_transactions ... Fix an assertion in transactions	2022-12-14 14:25:09 +03:00
Maksim Kita	ddad879cb4	Analyzer support Set index	2022-12-14 11:16:53 +01:00
Anton Popov	87e02a1744	Merge remote-tracking branch 'upstream/master' into HEAD	2022-12-14 02:08:10 +00:00
Alexander Tokmakov	f4c8281ccc	avoid outdating part twice	2022-12-13 21:24:26 +01:00
Nikolay Degterinsky	9b6d31b95d	Merge branch 'master' into perf_experiment	2022-12-13 17:15:07 +01:00
Antonio Andelic	2c57f2bd1f	Remove wrong assert	2022-12-13 14:11:08 +00:00
avogar	a1e5cb8c79	Fix possible Bad cast from type DB::IAST const* to DB::ASTLiteral const*	2022-12-13 13:12:44 +00:00
Alexander Tokmakov	a4df1cff52	Update ReplicatedMergeTreeQueue.cpp	2022-12-13 16:04:47 +03:00
Alexander Tokmakov	371e85d854	Merge pull request #44157 from ClickHouse/fix_session_expired_on_startup ... Ignore "session expired" errors after BC check	2022-12-13 15:10:36 +03:00
Dmitry Novik	3d2fccab87	Merge branch 'master' into refector-function-node	2022-12-12 21:36:39 +01:00
Anton Popov	659a79e53f	fix LOGICAL_ERROR if fetch of projection was stopped	2022-12-12 18:31:20 +00:00
Nikolai Kochetov	e451137040	Add comments.	2022-12-12 18:26:07 +00:00
Anton Popov	0b396bab14	Merge remote-tracking branch 'upstream/master' into HEAD	2022-12-12 18:08:27 +00:00
Anton Popov	a2fee110b8	add query for waiting of loading parts and fix test_transactions	2022-12-12 14:33:42 +00:00
Vitaly Baranov	7d701e9b32	Merge pull request #44065 from vitlibar/fix-race-in-s3-multipart-upload ... Fix race in s3 multipart upload	2022-12-12 15:28:05 +01:00
Alexander Tokmakov	d54f739836	ensure force_restore_data is removed from zk	2022-12-12 14:27:35 +01:00
Antonio Andelic	e18ac19ab3	Prevent dropping nested column if it can create empty part	2022-12-12 11:55:33 +00:00
Sema Checherinda	02133f68dd	Merge branch 'master' into mutation-breaks-source-part	2022-12-09 17:20:57 +01:00
Nikolai Kochetov	42d1b38ada	Merge pull request #43965 from ClickHouse/respect-adaptive-granularity-for-disk-remote ... Respect adaptive granularity for disk remote.	2022-12-09 16:54:34 +01:00
Nikolai Kochetov	9e04473e8c	Fix comment.	2022-12-09 15:44:59 +00:00
Sergei Trifonov	7c37e48092	Merge pull request #43903 from ClickHouse/merge_tree_settings_constraints ... Constraints on merge tree settings	2022-12-09 14:16:14 +01:00
Maksim Kita	3a3c6eb458	Merge pull request #43341 from kitaisreal/analyzer-support-indexes ... Analyzer added indexes support	2022-12-09 16:05:40 +03:00
Han Fei	6740ae9fd7	Merge pull request #43304 from hanfei1991/hanfei/aysnc-insert ... Support dedup for async insert.	2022-12-09 12:50:48 +01:00
Vitaly Baranov	517e84cfb9	Finalize buffers after writing.	2022-12-09 03:02:45 +01:00
Alexey Milovidov	f880644d3a	Merge branch 'master' into nv/fix-mutate-fetch	2022-12-09 00:51:54 +03:00
Sergei Trifonov	bae1519970	Merge branch 'master' into merge_tree_settings_constraints	2022-12-08 22:25:16 +01:00
Dmitry Novik	2826942ca3	Fixup assert condition	2022-12-08 16:41:54 +00:00
Maksim Kita	99de697ea2	Analyzer added indexes support	2022-12-08 17:30:32 +01:00
Yakov Olkhovskiy	0641066183	Merge branch 'master' into refactoring-ip-types	2022-12-08 11:12:05 -05:00
Sema Checherinda	46f96064ea	do not hardlink serialization.json in new part	2022-12-08 12:48:34 +01:00
Han Fei	1bde6d012d	fix	2022-12-08 09:57:33 +01:00
serxa	77d551788a	Fix relaxed "too many parts" threshold	2022-12-08 00:40:43 +00:00
Han Fei	3c11aca040	address comments	2022-12-07 23:40:52 +01:00
Han Fei	7a3dcd196e	Update src/Storages/MergeTree/EphemeralLockInZooKeeper.cpp ... Co-authored-by: alesapin <alesapin@gmail.com>	2022-12-07 22:30:03 +01:00
Han Fei	6d6bcd9ba2	Update src/Storages/MergeTree/ReplicatedMergeTreeSink.cpp ... Co-authored-by: alesapin <alesapin@gmail.com>	2022-12-07 22:28:55 +01:00
Dmitry Novik	25ecb75ca8	Merge remote-tracking branch 'origin/master' into refector-function-node	2022-12-07 18:36:50 +00:00
Dmitry Novik	a438b6efd8	Try to fix InvertedDAG construction	2022-12-07 18:27:37 +00:00
serxa	2306bd74d7	fix for ALTER TABLE ... MODIFY SETTING name=value	2022-12-06 20:46:10 +00:00
serxa	1b5b43ec87	fix checks on CREATE + added resolveSetting.h + improved system table	2022-12-06 19:09:18 +00:00
Nikolai Kochetov	9a4b3493fd	Merge branch 'master' into add-allocation-ptr-to-trace-log	2022-12-06 17:09:54 +01:00
Nikolai Kochetov	0ed82f3cc0	Merge branch 'master' into aggregating-in-order-from-query-plan	2022-12-06 16:36:49 +01:00
Yakov Olkhovskiy	031d2802d0	Merge pull request #43642 from HarryLeeIBM/hlee-s390x-bloom-filter ... Fix endian issue in bloom filter serialization for s390x	2022-12-06 08:23:44 -05:00
Nikolai Kochetov	f75250eaf4	Fix style.	2022-12-05 21:27:54 +00:00
Nikolai Kochetov	b797157437	Respect adaptive index granularity settings while reading from remote disk.	2022-12-05 20:10:31 +00:00
Sema Checherinda	be00335ca4	can_remove_anything is false for temporary parts	2022-12-05 15:56:04 +01:00
kssenii	d278a0dbfb	Merge remote-tracking branch 'upstream/master' into allow-single-disk-instead-of-storage-policy	2022-12-05 12:59:44 +01:00
kssenii	d91f3808bd	Merge remote-tracking branch 'upstream/master' into allow-single-disk-instead-of-storage-policy	2022-12-05 12:54:32 +01:00
Sema Checherinda	005cded791	Merge branch 'master' into logical-error-race-tmp-dirs	2022-12-04 18:18:42 +01:00
Sergei Trifonov	e27e79b45d	Merge branch 'master' into merge_tree_settings_constraints	2022-12-02 20:51:31 +01:00
serxa	006af1b6a4	constraints on merge tree settings	2022-12-02 19:30:14 +00:00
Dmitry Novik	15b8c48ca9	Cleanup code	2022-12-02 19:15:26 +00:00
Dmitry Novik	48e8aeac01	Clean up code	2022-12-02 19:15:26 +00:00
Dmitry Novik	2c70dbc76a	Refactor FunctionNode	2022-12-02 19:15:26 +00:00
Anton Popov	0a951db1d0	fix more tests with random merge tree settings	2022-12-02 14:16:28 +00:00
Vladimir C	7d6950d397	Revert "Temporary files evict fs cache"	2022-12-02 14:50:56 +01:00
Sema Checherinda	ce8ab95b65	Merge pull request #43868 from CheSema/data-race-loging-part-status ... do not read part status just for logging	2022-12-02 14:48:55 +01:00
Kruglov Pavel	c35b2a6495	Add a limit for string size in RowBinary format (#43842)	2022-12-02 13:57:11 +01:00
Anton Popov	5d9077cdb6	Merge remote-tracking branch 'upstream/master' into HEAD	2022-12-02 12:06:45 +00:00
Sema Checherinda	fd2c7657be	do not read part status just for logging	2022-12-01 16:08:05 +01:00
Yakov Olkhovskiy	5083768e74	use #if instead of if consteaxpr - clang's idiosyncrasy	2022-12-01 09:04:36 -05:00
alesapin	5e0ece2373	Merge pull request #43764 from ClickHouse/fix_projection_and_zero_copy ... Fix bug with projections and zero-copy replication	2022-12-01 13:44:26 +01:00
Sema Checherinda	baaf5c6186	do not leave tmp part on disk, do not go to the keeper for remove it	2022-12-01 13:24:29 +01:00
vdimir	7a814d2854	wip: temporary files evict fs cache	2022-12-01 11:49:24 +00:00
Nikolai Kochetov	7649a3c4d7	Merge branch 'master' into add-allocation-ptr-to-trace-log	2022-12-01 12:12:23 +01:00
Yakov Olkhovskiy	58d156a39e	style fix	2022-11-30 23:15:35 -05:00
Yakov Olkhovskiy	7a32488900	more clear code and some optimization	2022-11-30 23:05:21 -05:00
Alexander Tokmakov	431f6551cb	Merge branch 'master' into fix_assertion_in_thread_status	2022-11-30 23:05:15 +03:00
Han Fei	53e961e7c1	fix build check	2022-11-30 19:09:57 +01:00
alesapin	7b4025ba7c	Fix bug	2022-11-30 12:43:46 +01:00
Han Fei	564279be83	fix integration tests	2022-11-30 12:00:09 +01:00
Anton Popov	df6d2f2163	Merge remote-tracking branch 'upstream/master' into HEAD	2022-11-29 23:21:23 +00:00
Alexander Tokmakov	24940ad5bb	Merge pull request #43675 from ClickHouse/merge_predicate_uncommitted_blocks_hint ... Provide hint for loading uncommitted blocks in merge predicate	2022-11-29 17:55:11 +03:00
Sema Checherinda	75369274c1	Merge pull request #43516 from azat/memory/mutations-merges-fix ... Fix incorrect memory accounting because of merges/mutations	2022-11-29 13:03:04 +01:00
alesapin	428286e548	Merge branch 'master' into fix_projection_and_zero_copy	2022-11-29 12:51:08 +01:00
alesapin	7914e26744	Fix another part of code	2022-11-29 12:50:20 +01:00
Nikolai Kochetov	51439e2c19	Merge pull request #43260 from ClickHouse/read-from-mt-in-io-pool ... Read from MergeTree in I/O pool	2022-11-29 12:09:03 +01:00
Alexander Tokmakov	e45105bf44	detach threads from thread group	2022-11-28 21:31:55 +01:00
alesapin	4ea833afa1	Fix bug with projections and zero-copy replication	2022-11-28 17:29:03 +01:00
Han Fei	f23ee45d84	Merge branch 'master' into hanfei/aysnc-insert	2022-11-28 16:58:45 +01:00
Nikolai Kochetov	6d0646ed8f	Merge branch 'master' into aggregating-in-order-from-query-plan	2022-11-28 16:53:29 +01:00
Alexander Tokmakov	0342e6b236	fix	2022-11-28 16:02:37 +01:00
Alexander Tokmakov	a47e0b8f61	Merge branch 'master' into merge_predicate_uncommitted_blocks_hint	2022-11-28 15:45:34 +01:00
Yakov Olkhovskiy	770b520ded	Merge branch 'master' into refactoring-ip-types	2022-11-28 08:50:19 -05:00
Sema Checherinda	700181ca2e	Merge branch 'master' into memory/mutations-merges-fix	2022-11-28 14:14:57 +01:00
Kseniia Sumarokova	f35aa891d2	Merge pull request #43595 from azat/part.log.uuid-resubmit ... Add table_uuid to system.parts (resubmit)	2022-11-28 12:02:44 +01:00
Robert Schulze	dce0b78198	Merge pull request #42778 from FArthur-cmd/annoy_cosine ... Add cosine distance for annoy	2022-11-28 09:53:17 +01:00
Nikita Taranov	8ed5cfc265	Memory bound merging for distributed aggregation in order (#40879) ... * impl * fix style * make executeQueryWithParallelReplicas similar to executeQuery * impl for parallel replicas * cleaner code for remote sorting properties * update test * fix * handle when nodes of old versions participate * small fixes * temporary enable for testing * fix after merge * Revert "temporary enable for testing" This reverts commit cce7f8884c. * review fixes * add bc test * Update src/Core/Settings.h	2022-11-28 00:41:31 +01:00
Alexander Tokmakov	fc3a0bfdfa	fix	2022-11-27 01:12:09 +01:00
Azat Khuzhin	6a057dec70	Add table_uuid to system.parts (resubmit) ... Can be useful if the table constantly recreated, i.e. in some tests. v2: fix tests (disable for ordinary database) v3: adjust test after merging replacing with empty parts pr Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com> (cherry picked from commit d0ce500f89)	2022-11-26 20:16:38 +01:00
Arthur Passos	6ee21e63d7	Move force select final to user lvel setting and remove ignore force select final	2022-11-25 13:31:08 -03:00
Han Fei	ee5be80d9f	Merge branch 'master' into hanfei/aysnc-insert	2022-11-25 17:27:56 +01:00
Alexander Tokmakov	5cc99312d7	provide hint for loading uncommitted blocks in merge predicate	2022-11-25 16:41:20 +01:00
Nikolai Kochetov	660c942258	Merge branch 'master' into read-from-mt-in-io-pool	2022-11-25 13:56:48 +00:00
Alexander Tokmakov	ace36e2267	Merge branch 'master' into fix_parts_to_do_detection	2022-11-25 15:15:56 +03:00
Anton Popov	25197586a0	remove test	2022-11-25 01:14:23 +00:00
HarryLeeIBM	fc9880ea5a	Fix Bloom filter serialization for s390x	2022-11-24 14:37:05 -08:00
HarryLeeIBM	a35c5a914f	Revert "Fix bloom filter serialization endian issue for s390x" ... This reverts commit 8f7970d598.	2022-11-24 13:44:38 -08:00
HarryLeeIBM	8f7970d598	Fix bloom filter serialization endian issue for s390x	2022-11-24 13:27:02 -08:00
Nikolai Kochetov	d53439390f	Review fixes.	2022-11-24 16:04:35 +00:00
Anton Popov	c65be55da1	Merge remote-tracking branch 'upstream/master' into HEAD	2022-11-24 14:30:23 +00:00
alesapin	52821a331a	Update src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp ... Co-authored-by: Alexander Tokmakov <tavplubix@clickhouse.com>	2022-11-24 14:04:03 +01:00
alesapin	188069117e	Update src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp ... Co-authored-by: Alexander Tokmakov <tavplubix@clickhouse.com>	2022-11-24 14:02:26 +01:00
alesapin	5355f82448	Update src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp ... Co-authored-by: Alexander Tokmakov <tavplubix@clickhouse.com>	2022-11-24 14:02:19 +01:00
Sema Checherinda	6fd7dcf26f	Merge branch 'master' into lock-free-drop-partition	2022-11-24 12:34:19 +01:00
alesapin	f64f0461fe	Merge pull request #43288 from ClickHouse/Enable_compress_marks_and_primary_key ... Enable compress marks and primary key for CI	2022-11-24 12:24:04 +01:00
Raúl Marín	4d3217cb18	Merge branch 'master' into perf_experiment	2022-11-24 10:32:12 +01:00
Han Fei	28564df6f1	Merge branch 'master' into hanfei/aysnc-insert	2022-11-24 09:12:43 +01:00
Han Fei	d0f5fcdf09	update ut and refine logs	2022-11-23 21:07:59 +01:00
Nikolai Kochetov	62debf42b4	Aggregating in order [in progress].	2022-11-23 18:38:12 +00:00
Sema Checherinda	9f2c00d434	add tags to tests, fix error message	2022-11-23 15:16:10 +00:00
Sema Checherinda	7d748602c0	always write creation_csn	2022-11-23 15:16:10 +00:00
Sema Checherinda	e29d7c2e00	fix typos	2022-11-23 15:16:10 +00:00
Sema Checherinda	b4a21cab68	removal_csn can be PrehistoricCSN iff removal_tid is also PrehistoricCSN when cl removind an old part	2022-11-23 15:16:10 +00:00
Sema Checherinda	eccb62e4c3	work with the latest remarks	2022-11-23 15:16:09 +00:00
Sema Checherinda	a29437e179	delete verbose logging about cleaning old parts	2022-11-23 15:16:09 +00:00
Sema Checherinda	f49936b9a0	remove tmp dir from inmemory part when freeze, more logs about old parts	2022-11-23 15:16:09 +00:00
Sema Checherinda	3a8024002d	fix race in read full path from deleting part, add more logs in grabOldParts	2022-11-23 15:16:09 +00:00
Sema Checherinda	4f592bc208	fix isolation breach, droping outdated part should't be ignored	2022-11-23 15:16:09 +00:00
Sema Checherinda	6b4cb25e25	fixed 01130_in_memory_parts_partitons, remove flushed tmp part from in memory part	2022-11-23 15:16:09 +00:00
Sema Checherinda	c976b28104	less pedantic checks in operations, rely on existed tx functionality	2022-11-23 15:16:09 +00:00
Sema Checherinda	a4d1adf945	in order to restore partition do not write empty inmemory part	2022-11-23 15:16:09 +00:00
Sema Checherinda	00e7b4e195	fix race on remove, add creation csn for removed parts	2022-11-23 15:16:09 +00:00
Sema Checherinda	c1e6b560f0	no groupping parts for cover with empty	2022-11-23 15:16:08 +00:00
Sema Checherinda	597421014e	fix style check	2022-11-23 15:16:08 +00:00
Sema Checherinda	db86cd0f0a	fix several critical races, fix tests	2022-11-23 15:16:08 +00:00
Sema Checherinda	8977e499c2	address 01825_type_json_18 test	2022-11-23 15:16:08 +00:00
Sema Checherinda	6d929266e0	address 01825_type_json_schema_race_long test	2022-11-23 15:16:08 +00:00
Sema Checherinda	15012c7070	address the review comments	2022-11-23 15:16:08 +00:00
Sema Checherinda	fb86019db9	Delete parts over replacing them with empty parts	2022-11-23 15:16:08 +00:00
Alexander Tokmakov	c0133199c8	Revert "Add table_uuid to system.parts"	2022-11-23 16:41:43 +03:00
Alexey Milovidov	bedfe68503	Merge branch 'master' into nv/fix-mutate-fetch	2022-11-22 20:06:56 +01:00
Azat Khuzhin	2cfbc27f13	Fix incorrect memory accounting because of merges/mutations ... In case of merges/mutations, separate MemoryTracker object is used, to track memory separately, to report this metrics. However, the tail of those allocations hadn't been taken into account, and in case of lots of mutations/merges it can be significant. 100% reproducible with the following example: Start the server w/o frequent sync with RSS: $ clickhouse-server -C /src/programs/server/config.xml -- --asynchronous_metrics_update_period_s=86400 Run the following snippet: create table data (key Int, value Nullable(Decimal(38, 0))) engine=ReplicatedMergeTree('/ch/data', '1') order by key insert into data values (1, -13028236692093846346337460743176821145) alter table data modify column value UInt256 Code: 341. DB::Exception: Received from localhost:9000. DB::Exception: Exception happened during execution of mutation 'mutation_2.txt' with part 'all_1_1_0' reason: 'Code: 407. DB::Exception: Convert overflow: while executing 'FUNCTION _CAST(value :: 1, 'UInt256' :: 2) -> _CAST(value, 'UInt256') UInt256 : 3': (while reading from part ./store/dd9/dd9ef183-1270-4877-af22-8a000bdcc1b8/all_1_1_0/): While executing MergeTreeInOrder. (DECIMAL_OVERFLOW) (version 22.12.1.1)'. This error maybe retryable or not. In case of unretryable error, mutation can be killed with KILL MUTATION query. (UNFINISHED) And track the memory: select value/1e9 from system.metrics where metric like 'MemoryTracking' Reported-by: @filimonov Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>	2022-11-22 19:40:35 +01:00
Azat Khuzhin	5da2f52722	Use Int64 over UInt64 for prev_untracked_memory* in MemoryTrackerThreadSwitcher ... Since those types originally is Int64 Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>	2022-11-22 19:40:35 +01:00
Azat Khuzhin	05f30aaad4	Fix logger name for MutateFromLogEntryTask ... Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>	2022-11-22 19:40:35 +01:00
Raúl Marín	4aa29b6a63	Merge remote-tracking branch 'blessed/master' into perf_experiment	2022-11-22 19:09:00 +01:00
Anton Popov	16a3eb9bfc	better loading of outdated parts	2022-11-22 17:09:11 +00:00
Anton Popov	75c807609b	add some comments	2022-11-22 16:11:01 +00:00
Anton Popov	e3db29aebf	better waiting for outdated data parts	2022-11-22 15:15:20 +00:00
Han Fei	32c7fbec55	use another dedup path for async inserts	2022-11-22 15:12:00 +01:00
Sergei Trifonov	59f422f10c	Merge branch 'master' into storage-system-moves	2022-11-22 14:46:00 +01:00
Kseniia Sumarokova	392963012a	Merge pull request #43404 from azat/part.log.uuid ... Add table_uuid to system.parts	2022-11-22 14:25:34 +01:00
kssenii	5f649baffe	Merge remote-tracking branch 'upstream/master' into allow-single-disk-instead-of-storage-policy	2022-11-22 14:19:51 +01:00
Nikolai Kochetov	9e84a351c7	Fixing special build. Disable setting by default.	2022-11-22 11:24:24 +00:00
Raúl Marín	8776b2f566	Merge remote-tracking branch 'blessed/master' into perf_experiment	2022-11-22 11:45:36 +01:00
Anton Popov	0479d8d9f5	Merge pull request #43387 from ClickHouse/fix_default_value_used_in_row_level_filter ... Fix default value used in row level filter	2022-11-22 01:46:45 +01:00
Sergei Trifonov	94a7614c45	Merge branch 'master' into storage-system-moves	2022-11-21 20:10:12 +01:00
Nikolai Kochetov	c305afd77a	Fix race.	2022-11-21 18:26:31 +00:00
serxa	faef2e9b00	fix moves list entry creation	2022-11-21 17:08:54 +00:00
Anton Popov	92ac9385b9	wait loading of outdated parts for partition commands	2022-11-21 17:01:46 +00:00
serxa	c425da06b2	fix column name	2022-11-21 16:16:53 +00:00
Azat Khuzhin	d0ce500f89	Add table_uuid to system.parts ... Can be useful if the table constantly recreated, i.e. in some tests. Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>	2022-11-21 12:11:51 +01:00
alesapin	b78b602ea1	Merge branch 'master' into Enable_compress_marks_and_primary_key	2022-11-21 12:05:04 +01:00
Nikolai Kochetov	be1a8054c7	Merge branch 'master' into read-from-mt-in-io-pool	2022-11-21 11:27:41 +01:00
Nikolai Kochetov	4a1774c5ac	Fixing build.	2022-11-21 10:27:09 +00:00
Raúl Marín	ed0c174c0c	Merge remote-tracking branch 'blessed/master' into perf_experiment	2022-11-21 11:02:31 +01:00
Alexey Milovidov	915485ff88	Merge branch 'master' into add_select_final_mt_setting	2022-11-20 08:51:12 +01:00
kssenii	08e4805c99	Merge remote-tracking branch 'upstream/master' into allow-single-disk-instead-of-storage-policy	2022-11-19 13:26:56 +01:00
zhongyuankai	a3a3261745	Merge branch 'master' into optimize_ttl	2022-11-19 16:32:26 +08:00
zhongyuankai	ce9b4866ea	Update MergeTreeDataMergerMutator.cpp	2022-11-19 16:31:47 +08:00
Alexander Gololobov	f004eea413	Add columns required fro defaults calculation	2022-11-18 23:42:45 +01:00
alesapin	c7d648174b	Merge pull request #43328 from ClickHouse/better-format-version-check ... Check content of `format_version` file in `MergeTreeData`	2022-11-18 22:34:37 +01:00
Nikolai Kochetov	5da1d893fd	Merge branch 'master' into read-from-mt-in-io-pool	2022-11-18 21:10:45 +01:00
Nikolai Kochetov	03c35e146b	Refactor a bit.	2022-11-18 20:09:20 +00:00
serxa	8e65b799c2	fix test + add one more column	2022-11-18 20:04:11 +00:00
kssenii	30c5344de4	Get rid of ASTPtr in settings changes	2022-11-18 19:29:45 +01:00
Han Fei	b80a2d6c89	address comments	2022-11-18 17:22:05 +01:00
serxa	a05a1b0133	Merge branch 'storage-system-moves' of github.com:ClickHouse/ClickHouse into storage-system-moves	2022-11-18 15:35:00 +00:00
Sergei Trifonov	77100896e3	Merge branch 'master' into storage-system-moves	2022-11-18 16:33:43 +01:00
alesapin	0f3eb51aed	Disable compressed marks by default	2022-11-18 15:54:48 +01:00
kssenii	41c1a2aad4	Revert "Revert "Better code around SettingsChanges"" ... This reverts commit 5e61022bac.	2022-11-18 13:16:46 +01:00
kssenii	babc0c84b0	Revert "Not better" ... This reverts commit 66175cb9c8.	2022-11-18 13:00:46 +01:00
kssenii	66175cb9c8	Not better	2022-11-18 13:00:30 +01:00
kssenii	5e61022bac	Revert "Better code around SettingsChanges" ... This reverts commit bed9f7594a.	2022-11-18 12:37:34 +01:00
Han Fei	74c9ba8df0	Update src/Storages/MergeTree/ReplicatedMergeTreeSink.cpp ... Co-authored-by: alesapin <alesapin@gmail.com>	2022-11-18 11:16:38 +01:00