4851 Commits

Author	SHA1	Message	Date
serxa	faef2e9b00	fix moves list entry creation	2022-11-21 17:08:54 +00:00
serxa	c425da06b2	fix column name	2022-11-21 16:16:53 +00:00
Azat Khuzhin	d0ce500f89	Add table_uuid to system.parts ... Can be useful if the table constantly recreated, i.e. in some tests. Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>	2022-11-21 12:11:51 +01:00
alesapin	b78b602ea1	Merge branch 'master' into Enable_compress_marks_and_primary_key	2022-11-21 12:05:04 +01:00
Nikolai Kochetov	be1a8054c7	Merge branch 'master' into read-from-mt-in-io-pool	2022-11-21 11:27:41 +01:00
Nikolai Kochetov	4a1774c5ac	Fixing build.	2022-11-21 10:27:09 +00:00
Raúl Marín	ed0c174c0c	Merge remote-tracking branch 'blessed/master' into perf_experiment	2022-11-21 11:02:31 +01:00
zhongyuankai	a3a3261745	Merge branch 'master' into optimize_ttl	2022-11-19 16:32:26 +08:00
zhongyuankai	ce9b4866ea	Update MergeTreeDataMergerMutator.cpp	2022-11-19 16:31:47 +08:00
Alexander Gololobov	f004eea413	Add columns required fro defaults calculation	2022-11-18 23:42:45 +01:00
alesapin	c7d648174b	Merge pull request #43328 from ClickHouse/better-format-version-check ... Check content of `format_version` file in `MergeTreeData`	2022-11-18 22:34:37 +01:00
Nikolai Kochetov	5da1d893fd	Merge branch 'master' into read-from-mt-in-io-pool	2022-11-18 21:10:45 +01:00
Nikolai Kochetov	03c35e146b	Refactor a bit.	2022-11-18 20:09:20 +00:00
serxa	8e65b799c2	fix test + add one more column	2022-11-18 20:04:11 +00:00
Han Fei	b80a2d6c89	address comments	2022-11-18 17:22:05 +01:00
serxa	a05a1b0133	Merge branch 'storage-system-moves' of github.com:ClickHouse/ClickHouse into storage-system-moves	2022-11-18 15:35:00 +00:00
Sergei Trifonov	77100896e3	Merge branch 'master' into storage-system-moves	2022-11-18 16:33:43 +01:00
alesapin	0f3eb51aed	Disable compressed marks by default	2022-11-18 15:54:48 +01:00
Han Fei	74c9ba8df0	Update src/Storages/MergeTree/ReplicatedMergeTreeSink.cpp ... Co-authored-by: alesapin <alesapin@gmail.com>	2022-11-18 11:16:38 +01:00
Han Fei	3dc72b6c56	Update src/Storages/MergeTree/ReplicatedMergeTreeSink.cpp ... Co-authored-by: alesapin <alesapin@gmail.com>	2022-11-18 11:15:51 +01:00
Han Fei	8c823bcaab	Update src/Storages/MergeTree/ReplicatedMergeTreeSink.cpp ... Co-authored-by: alesapin <alesapin@gmail.com>	2022-11-18 11:15:45 +01:00
Han Fei	035e933196	Update src/Storages/MergeTree/ReplicatedMergeTreeSink.cpp ... Co-authored-by: alesapin <alesapin@gmail.com>	2022-11-18 11:15:35 +01:00
Han Fei	f8ac75a054	Update src/Storages/MergeTree/ReplicatedMergeTreeSink.cpp ... Co-authored-by: alesapin <alesapin@gmail.com>	2022-11-18 11:15:14 +01:00
Han Fei	d41094207d	Update src/Storages/MergeTree/EphemeralLockInZooKeeper.cpp ... Co-authored-by: alesapin <alesapin@gmail.com>	2022-11-18 11:14:56 +01:00
Nikolai Kochetov	679591cf0a	Try fix race.	2022-11-17 19:51:47 +00:00
alesapin	29a08f64af	Merge branch 'master' into Enable_compress_marks_and_primary_key	2022-11-17 15:53:37 +01:00
Antonio Andelic	d324e8cc3e	Write to first non-readonly disk	2022-11-17 14:52:35 +00:00
Antonio Andelic	d41af8b741	Check and compare content of format_version file	2022-11-17 13:41:36 +00:00
alesapin	06f395d946	Add comments	2022-11-17 13:10:14 +01:00
alesapin	a79d83549f	Merge branch 'master' into fix_parts_to_do_detection	2022-11-17 12:30:05 +01:00
Raúl Marín	97d6fc3071	Merge remote-tracking branch 'blessed/master' into perf_experiment	2022-11-17 11:48:46 +01:00
Han Fei	221c4e5912	tiny changes	2022-11-16 17:59:08 +01:00
Han Fei	14701670b5	Merge branch 'master' into hanfei/aysnc-insert	2022-11-16 17:24:36 +01:00
Nikolai Kochetov	17b75d3f5c	Fix metrics and progress.	2022-11-16 14:10:56 +00:00
Smita Kulkarni	040f2b62d9	Enable compress marks and primary key ... Implementation: * Set compress_marks and compress_primary_key to true by default. * Updated getIndexExtensionFromFilesystem to only check for compressed file extension , else return default. When both are present, this function was returning uncompressed file extension giving rise to error as file got removed. Testing: * Updated tests where bytes size or compressed data is checked to check with compress_marks and compress_primary_key set to false.	2022-11-16 14:23:58 +01:00
filimonov	dcb76f6bbf	Allow autoremoval of old parts if detach_not_byte_identical_parts enabled ... Allow autoremoval of old & detached parts if detach_not_byte_identical_parts enabled. See also #28708 #37975	2022-11-16 14:16:42 +01:00
alesapin	2012f4bd36	Fix use-after-free	2022-11-16 11:50:51 +01:00
Alexey Milovidov	04687c0d69	Merge pull request #43206 from ClickHouse/miscellaneous-changes-parallel-replicas ... Miscellaneous changes	2022-11-16 05:30:03 +01:00
Nikolai Kochetov	1b1d23c100	Clumsy attempt to fix progress.	2022-11-15 21:23:18 +00:00
Alexander Tokmakov	69b067b81c	Merge pull request #43244 from ClickHouse/fix-replicated-merge-tree-shared-id ... Don't throw if shared ID already created in `StorageReplicatedMergeTree`	2022-11-15 23:31:51 +03:00
Nikolai Kochetov	cf1afbfb84	Temporarily enable allow_asynchronous_read_from_io_pool_for_merge_tree by default to see thest results.	2022-11-15 17:47:26 +00:00
Nikolai Kochetov	29098045d8	Try read from MergeTree in background pool.	2022-11-15 17:38:37 +00:00
Sema Checherinda	7e73b187cc	Merge pull request #43159 from Algunenano/over-read ... Fix several buffer over-reads	2022-11-15 15:04:24 +01:00
alesapin	a12f2d14dc	Simpler check	2022-11-15 14:46:38 +01:00
Anton Popov	8c66c759d4	Merge pull request #43133 from amosbird/issue_43111 ... Fix Nullable keys in hyperrectangle	2022-11-15 14:30:13 +01:00
alesapin	4ec11d0f58	Remove redundant branch	2022-11-15 13:58:53 +01:00
Antonio Andelic	0e5e58bed9	Remove exception if shared ID already created	2022-11-15 12:18:24 +00:00
zhongyuankai	35690c1972	Merge branch 'master' into optimize_ttl	2022-11-15 08:49:37 +08:00
zhongyuankai	16f697f61d	fix test	2022-11-15 08:47:51 +08:00
Alexey Milovidov	6e01b2b2a1	Merge pull request #43203 from ClickHouse/fix-typo-2 ... Fix typo	2022-11-15 01:39:34 +01:00
alesapin	6a50a32cfc	Fix parts_to_do	2022-11-15 00:53:20 +01:00
alesapin	f53ada2bbd	Revert "Fix bug in replication queue which can lead to premature mutation finish" ... This reverts commit 3b5424811c.	2022-11-14 22:59:09 +01:00
alesapin	3b5424811c	Fix bug in replication queue which can lead to premature mutation finish	2022-11-14 21:53:16 +01:00
Alexander Tokmakov	c64109a2de	Merge branch 'master' into fetch-remove-covered	2022-11-14 18:23:16 +03:00
zhongyuankai	ea11d80536	Priority is given to deleting completely expired Parts	2022-11-14 23:03:56 +08:00
Han Fei	11f93dec3f	tmp commit	2022-11-14 15:53:45 +01:00
Raúl Marín	4988e387f2	Merge remote-tracking branch 'blessed/master' into over-read	2022-11-14 12:28:31 +01:00
Alexey Milovidov	4edd8b0135	Remove unused method	2022-11-14 08:07:27 +01:00
Alexey Milovidov	17693de053	Better logs	2022-11-14 08:01:48 +01:00
Alexey Milovidov	9c48664f11	Better logs	2022-11-14 07:48:50 +01:00
Alexey Milovidov	582db233c4	Better logs	2022-11-14 07:22:17 +01:00
Alexey Milovidov	99f84d8095	Better logs	2022-11-14 07:22:02 +01:00
Alexey Milovidov	df01689ccc	Better logs	2022-11-14 07:13:42 +01:00
Alexey Milovidov	a77032fafa	Miscellaneous changes	2022-11-14 06:54:58 +01:00
Alexey Milovidov	15202833b1	Miscellaneous changes	2022-11-14 06:48:49 +01:00
Alexey Milovidov	53ee7446dc	Miscellaneous changes	2022-11-14 06:46:37 +01:00
Alexey Milovidov	cc1855ea33	Miscellaneous changes	2022-11-14 06:45:52 +01:00
Alexey Milovidov	85dc692f7c	Miscellaneous changes	2022-11-14 06:37:55 +01:00
Alexey Milovidov	60c96072e1	Miscellaneous changes	2022-11-14 06:37:19 +01:00
Alexey Milovidov	e080161ba5	Miscellaneous changes	2022-11-14 06:32:18 +01:00
Alexey Milovidov	bfea3deed9	Miscellaneous changes	2022-11-14 06:09:03 +01:00
Alexey Milovidov	b9471e492a	Fix typo	2022-11-13 22:52:21 +01:00
Alexey Milovidov	127631ee47	Merge branch 'master' into perf_experiment	2022-11-12 18:58:25 +01:00
Filatenkov Artur	c7c52a570f	Update MergeTreeIndexAnnoy.cpp	2022-11-12 12:43:25 +03:00
FArthur-cmd	5eebc27d24	improve test	2022-11-12 09:23:49 +00:00
Nikolai Kochetov	9106910de8	Try read from MergeTree in background pool.	2022-11-11 20:47:11 +00:00
Igor Nikonov	ee5756b5b4	Merge pull request #43122 from ClickHouse/revert-43116-revert-42607-igor/insert_zk_retries_retry ... Revert "Revert " Keeper retries during insert (clean)""	2022-11-11 18:22:38 +01:00
Raúl Marín	54db7c6520	Enforce checking read output	2022-11-11 10:56:18 +01:00
Amos Bird	661c541e57	Fix Nullable keys in hyperrectangle.	2022-11-11 11:14:05 +08:00
Alexander Tokmakov	bb00a246f1	priority for DROP_RANGEs, assertion for others	2022-11-10 20:53:05 +01:00
Alexander Tokmakov	e415fc6e95	Revert "Optimize TTL merge, completely expired parts can be removed in time"	2022-11-10 19:38:55 +03:00
Igor Nikonov	f8c907c463	Revert "Revert " Keeper retries during insert (clean)""	2022-11-10 13:14:04 +01:00
Alexander Tokmakov	fb8c527a2f	Revert " Keeper retries during insert (clean)"	2022-11-10 14:12:24 +03:00
Igor Nikonov	1b9be27fdb	Merge pull request #42607 from ClickHouse/igor/insert_zk_retries_retry ... Keeper retries during insert (clean)	2022-11-10 12:01:00 +01:00
Antonio Andelic	1954ff385d	Merge pull request #43076 from canhld94/ch_canh_minor ... Better logging for mark range filtering on projection parts	2022-11-10 09:33:12 +01:00
alesapin	9e1627fcdd	Merge pull request #43060 from ClickHouse/fix_projection_part ... Fix projection part removal with zero-copy replication	2022-11-09 18:15:49 +01:00
Igor Nikonov	379113f8f1	Merge remote-tracking branch 'origin/master' into igor/insert_zk_retries_retry	2022-11-09 13:03:32 +00:00
Alexander Tokmakov	1c6a617207	Merge branch 'master' into fix_intersecting_parts2	2022-11-09 14:38:18 +03:00
Maksim Kita	768ada118b	Analyzer AST key condition crash fix	2022-11-09 11:41:39 +01:00
Vladimir C	82bf992099	Merge pull request #42869 from zhongyuankai/optimze_ttl_merge	2022-11-09 11:26:46 +01:00
Duc Canh Le	b9a3d090ed	use const string &	2022-11-09 13:07:24 +08:00
Duc Canh Le	901e1e2fef	better logging for mark range filtering on projection parts	2022-11-09 12:01:51 +08:00
alesapin	761274d857	Fix wrong logic	2022-11-09 01:14:03 +01:00
Igor Nikonov	345304abe0	Merge remote-tracking branch 'origin/master' into igor/insert_zk_retries_retry	2022-11-08 20:26:50 +00:00
alesapin	a31a79020f	Add comment	2022-11-08 16:27:26 +01:00
alesapin	6f059be748	Fix bug with temporary projection parts deletion	2022-11-08 15:37:03 +01:00
Alexander Tokmakov	eb19df0094	fix	2022-11-08 15:21:25 +01:00
Maksim Kita	0aa4fd3c2d	Fixed tests	2022-11-08 14:29:17 +01:00
Robert Schulze	2c568df714	Merge pull request #42968 from ClickHouse/bitcast ... Less use of CH-specific bit_cast()	2022-11-08 10:54:10 +01:00
Robert Schulze	bb507356ef	Merge pull request #42209 from canhld94/ch_canh_fix_prefix_not_like ... Add `notLike` to key condition atom map	2022-11-08 10:07:21 +01:00
Antonio Andelic	7067f9e0e1	Merge pull request #42947 from azat/mergetree/part-load-tiny ... Attach thread pool for loading parts to the query	2022-11-08 09:17:32 +01:00
zhongyuankai	37099e57f4	Merge branch 'master' into optimze_ttl_merge	2022-11-08 12:26:46 +08:00
Azat Khuzhin	2bd4a71150	Merge branch 'master' into fetch-remove-covered	2022-11-08 00:55:00 +01:00
serxa	ea17b30358	Merge branch 'storage-system-moves' of github.com:ClickHouse/ClickHouse into storage-system-moves	2022-11-07 20:33:31 +00:00
serxa	59408ebf57	fill more fields	2022-11-07 20:33:25 +00:00
Sergei Trifonov	6d3b96ee42	Merge branch 'master' into storage-system-moves	2022-11-07 21:31:27 +01:00
alesapin	257e766e0e	Trying to fix two flaky tests	2022-11-07 20:49:41 +01:00
Alexander Tokmakov	9210e586d2	fix	2022-11-07 20:27:18 +01:00
Alexander Tokmakov	b8174a63a8	Merge branch 'master' into fix_intersecting_parts2	2022-11-07 20:26:36 +01:00
Kruglov Pavel	48163ff9c2	Merge pull request #42938 from amosbird/issue_42771 ... Fix NOT_FOUND_COLUMN_IN_BLOCK in some rare case	2022-11-07 13:10:12 +01:00
Duc Canh Le	bd2bd7149d	fix typo	2022-11-07 18:22:50 +08:00
Duc Canh Le	f7e65ebd72	make comments more clear	2022-11-07 16:25:36 +08:00
Robert Schulze	cdaf0becfe	Merge branch 'master' into bitcast	2022-11-07 09:24:52 +01:00
Robert Schulze	f3f50530d1	Merge branch 'master' into ch_canh_fix_prefix_not_like	2022-11-07 09:14:02 +01:00
Duc Canh Le	69d6d42443	NOT LIKE only work for 'prefix%'	2022-11-06 15:26:19 +08:00
zhongyuankai	ef6065b331	fix test	2022-11-06 13:40:26 +08:00
Igor Nikonov	e94b9cd003	Fixes ... + enable retries in integration test + typo in fault injection traces	2022-11-05 22:28:32 +00:00
Robert Schulze	9c066e964d	Less use of CH-specific bit_cast() ... Converted usage of CH-custom bit_cast to std::bit_cast if possible, i.e. when sizeof(From) == sizeof(To). (The CH-custom bit_cast is able to deal with sizeof(From) != sizeof(To).) Motivation for this came from #42847 where it is not clear how the internal bit_cast should behave on big endian systems, so we better avoid that situation as much as possible.	2022-11-04 15:52:48 +00:00
Amos Bird	d901ead1bc	Done	2022-11-04 17:26:44 +08:00
zhongyuankai	049a593272	Merge branch 'master' into optimze_ttl_merge	2022-11-04 17:22:38 +08:00
zhongyuankai	e773eb2fd1	batter	2022-11-04 17:19:34 +08:00
Azat Khuzhin	7032b2b98a	Attach thread pool for loading parts to the query ... Detach should be done via CurrentThread::defaultThreadDeleter() Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>	2022-11-04 09:06:12 +01:00
Azat Khuzhin	cdd09fd72a	Improve messages for loading data parts (add part type) ... Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>	2022-11-04 09:05:06 +01:00
Amos Bird	7186898ffa	Experiment	2022-11-04 00:29:17 +08:00
Antonio Andelic	66ffdda973	Merge branch 'master' into optimize-entire-partition	2022-11-03 09:52:13 +01:00
Nikolay Degterinsky	30ad1a6826	Merge branch 'master' into perf_experiment	2022-11-03 02:18:21 +03:00
Alexander Tokmakov	e67d05604c	Merge branch 'master' into fix_intersecting_parts2	2022-11-02 19:30:27 +01:00
Igor Nikonov	9a315b7899	Merge remote-tracking branch 'origin/master' into igor/insert_zk_retries_retry	2022-11-02 15:32:54 +00:00
Alexander Tokmakov	00c9e50ee3	fix race between INSERT and ALTER PARTITION	2022-11-02 16:25:26 +01:00
Alexander Tokmakov	acbad99b24	remove abandonable_lock part 2	2022-11-02 14:28:49 +01:00
Antonio Andelic	c4b717b343	Update src/Storages/MergeTree/MergeTreeDataMergerMutator.cpp ... Co-authored-by: Sergei Trifonov <sergei@clickhouse.com>	2022-11-02 08:24:54 +01:00
Duc Canh Le	ec382d7903	add comment	2022-11-02 09:50:41 +08:00
Duc Canh Le	c6598dc66d	Merge branch 'master' into ch_canh_fix_prefix_not_like	2022-11-02 09:42:01 +08:00
Alexander Tokmakov	4f50a999bb	remove abandonable_lock part 1	2022-11-01 21:35:25 +01:00
Igor Nikonov	e76c3c381f	Better handling tables in read-only mode during insert ... + throw immediately if table has no metadata in zk + stop retries in case of shutdown + check if table is readonly at the begining of every retry	2022-11-01 18:47:51 +00:00
Anton Popov	88674ef964	Merge pull request #42794 from CurtizJ/better-code-in-check-part ... Slightly better code with projections	2022-11-01 16:05:27 +01:00
zhongyuankai	c1e2b2af74	Optimize TTL merge, completely expired parts can be removed in time	2022-11-01 22:58:15 +08:00
Anton Popov	876dca48da	Merge pull request #36969 from CurtizJ/dynamic-columns-14 ... Support `Object` type inside other types	2022-11-01 15:20:02 +01:00
Duc Canh Le	07d9720ed5	Merge branch 'master' into ch_canh_fix_prefix_not_like	2022-11-01 22:16:09 +08:00
Maksim Kita	fe48a1ce7e	Merge pull request #42681 from kitaisreal/merge-tree-indexes-use-actions-dag ... MergeTree indexes use ActionsDAG	2022-11-01 13:55:17 +03:00
Duc Canh Le	3545db0a6c	Merge branch 'master' into ch_canh_fix_prefix_not_like	2022-11-01 16:54:26 +08:00
Yakov Olkhovskiy	bab0e06e3d	Merge pull request #42458 from Clark0/match-func-use-index ... match function can use index if it's a condition on string prefix #37333	2022-10-31 17:33:58 -04:00
Igor Nikonov	9cc64a0fcc	Merge remote-tracking branch 'origin/master' into igor/insert_zk_retries_retry	2022-10-31 18:21:21 +00:00
Igor Nikonov	2d5050ea66	Fix review comments	2022-10-31 18:20:58 +00:00
Anton Popov	2ae3cfa9e0	Merge branch 'master' into dynamic-columns-14	2022-10-31 16:15:19 +01:00
Alexander Tokmakov	599ccb9939	Update MergeTreeData.h	2022-10-31 16:18:17 +03:00
Alexander Tokmakov	b2b9479afa	Update src/Storages/MergeTree/MergeTreeData.h ... Co-authored-by: Sergei Trifonov <sergei@clickhouse.com>	2022-10-31 16:15:35 +03:00
Maksim Kita	bca22ec5f5	Fixed code review issues	2022-10-31 12:30:00 +01:00
Maksim Kita	115fcaffc5	Fixed tests	2022-10-31 11:30:56 +01:00
Duc Canh Le	d309e8793a	Merge branch 'master' into ch_canh_fix_prefix_not_like	2022-10-31 15:13:39 +08:00
Alexander Tokmakov	f53df7870c	fix race between drop and failed insert	2022-10-30 17:59:47 +01:00
Alexander Tokmakov	c68ab231f9	fix accessing part in Deleting state	2022-10-30 17:30:51 +01:00
FArthur-cmd	607b81989d	fix build	2022-10-30 14:57:10 +00:00
Anton Popov	8de75bf29c	better code with projections	2022-10-29 14:39:59 +00:00
clarkcaoliu	ca5dbe88db	match function can use index if prefix	2022-10-29 18:25:29 +08:00
Filatenkov Artur	21c024b043	Merge branch 'master' into annoy_cosine	2022-10-28 20:15:53 +03:00
FArthur-cmd	f187d4e1d4	add cosine distance for annoy and fix docs for cosineDistance	2022-10-28 17:03:35 +00:00
Igor Nikonov	d09deaf10f	Fix style check	2022-10-28 15:17:05 +00:00
Igor Nikonov	937dd44399	Merge remote-tracking branch 'origin/master' into igor/insert_zk_retries_retry	2022-10-28 15:09:23 +00:00
Igor Nikonov	c53b96a11c	Add more comments	2022-10-28 15:09:01 +00:00
Alexey Milovidov	f64e632707	Merge pull request #42469 from amosbird/issue_42456 ... Fix misbehavior of key analysis	2022-10-28 16:18:00 +02:00
Antonio Andelic	5fae8b0cc9	Merge branch 'master' into optimize-entire-partition	2022-10-28 11:44:10 +00:00
Sergei Trifonov	836595c7ae	Merge branch 'master' into storage-system-moves	2022-10-28 11:39:38 +02:00
Duc Canh Le	848f5b5d65	Merge branch 'master' into ch_canh_fix_prefix_not_like	2022-10-28 13:40:17 +08:00
Duc Canh Le	c080964391	correct 'notLike' in key condition	2022-10-28 13:39:01 +08:00
Duc Canh Le	69311bd5cd	Merge branch 'master' into ch_canh_fix_prefix_not_like	2022-10-28 11:59:50 +08:00
Raúl Marín	59686c3796	Merge branch 'master' into perf_experiment	2022-10-28 00:51:56 +02:00
Igor Nikonov	3682913172	Merge remote-tracking branch 'origin/master' into igor/insert_zk_retries_retry	2022-10-27 22:01:53 +00:00
Igor Nikonov	a601c166ad	Add clean up for faults injected before zk request ... + for multi-op request, it create responses in multi response + some polishing	2022-10-27 21:35:13 +00:00
Anton Popov	7715afa595	Merge pull request #42618 from CurtizJ/refactor-data-part ... Try to save `IDataPartStorage` interface	2022-10-27 22:20:44 +02:00
Igor Nikonov	f773436de5	Fix flaky test	2022-10-27 13:34:07 +00:00
Raúl Marín	891484b462	Merge remote-tracking branch 'blessed/master' into perf_experiment	2022-10-27 13:17:07 +02:00
Nikolai Kochetov	24e1aaf324	Merge pull request #42576 from ClickHouse/table-function-merge-invalid-where-type ... Invalid type in where for Merge table (logical error)	2022-10-27 13:03:27 +02:00
Antonio Andelic	10aa7091ca	Merge pull request #42605 from ClickHouse/correctly-handle-prewhere-type ... Correctly handle non-native integers in PREWHERE	2022-10-27 09:25:38 +02:00
Igor Nikonov	649e8d302c	Sync with retiring branch igor/insert_zk_retries	2022-10-26 17:41:44 +00:00
Igor Nikonov	8435cba352	Merge remote-tracking branch 'origin/master' into igor/insert_zk_retries_retry	2022-10-26 17:35:47 +00:00
Maksim Kita	f04f663818	Fixed tests	2022-10-26 18:55:14 +02:00
Nikolai Kochetov	1520bcd53c	Fix error code.	2022-10-26 15:50:10 +00:00
Anton Popov	ca4df62e37	Merge branch 'master' into refactor-data-part	2022-10-26 17:49:20 +02:00
Maksim Kita	1c17e9d454	Fixed tests	2022-10-26 15:54:37 +02:00
Nikolai Kochetov	db422fa903	Merge pull request #42410 from ClickHouse/fix-multi-arg-monotonicity ... Fix possible LOGICAL_ERROR in binary ariphmetics monotonicity.	2022-10-26 14:11:23 +02:00
Kruglov Pavel	219553df8d	Merge pull request #42580 from hanfei1991/fei/limit_max_rows ... fix behaviour of max_rows_to_read for trival limit queries	2022-10-26 14:09:59 +02:00
Maksim Kita	c47ca522f4	Fixed style check	2022-10-26 13:27:14 +02:00
Antonio Andelic	35a9672704	Merge branch 'master' into optimize-entire-partition	2022-10-26 11:09:25 +00:00
Maksim Kita	1b6293f6db	MergeTree indexes use ActionsDAG	2022-10-26 12:44:37 +02:00
Antonio Andelic	97aaebfa18	Address PR comments	2022-10-26 10:06:56 +00:00
Antonio Andelic	979d180931	Merge branch 'master' into correctly-handle-prewhere-type	2022-10-26 11:13:16 +02:00
Azat Khuzhin	8db31beee6	Remove covered parts for fetched part ... Here is an example that I found on production, simplified. Consider the following queue (nothing of this had been processed on this replica): - GET_PART all_0_0_0 (queue-0000000001) - GET_PART all_1_1_0 (queue-0000000002) ... - GET_PART all_0_1_1 (queue-0000000003) - GET_PART all_2_2_0 (queue-0000000004) ... - MERGE_PARTS from [all_0_1_1, all_2_2_0] to all_0_2_2 (queue-0000000005) And now queue-0000000005 started to executing (either because of reording, or because at that time GET_PART fails), and it does not have any required parts, so it will fetch them, but not all_0_0_0 and all_1_1_0, so this replica delay will set to the time of min(queue-0000000001, queue-0000000002), while it is not true, since it already have parts that covers those parts. and since MERGE_PARTS takes 30min, it increased the replica delay eventually to 30min, for the time range of 30min, which is pretty huge. Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>	2022-10-26 11:10:50 +03:00
Anton Popov	79889198b1	fix style check	2022-10-25 23:03:37 +00:00
Anton Popov	d8b09b430f	add comments	2022-10-25 22:27:11 +00:00
Anton Popov	c4d4f2dbed	better interface	2022-10-25 22:14:06 +00:00
Sergei Trifonov	34b43f8b4b	Merge branch 'master' into storage-system-moves	2022-10-25 22:56:45 +02:00
serxa	55a86bc2de	Add active moves system table	2022-10-25 13:56:46 +00:00
Antonio Andelic	611c2e2bd7	Support for optimizing old parts for entire partition only	2022-10-25 13:40:18 +00:00
Raúl Marín	6e0a9452e7	Merge remote-tracking branch 'blessed/master' into perf_experiment	2022-10-25 15:25:06 +02:00
Anton Popov	c8199bc125	Merge branch 'master' into refactor-data-part	2022-10-25 14:31:05 +02:00
Raúl Marín	0016bc2b73	Improve the time to recover keeper connections (#42541) ... Co-authored-by: Nikita Mikhaylov <nikitamikhaylov@clickhouse.com> Co-authored-by: robot-clickhouse <robot-clickhouse@clickhouse.com> Co-authored-by: Alexey Milovidov <milovidov@clickhouse.com>	2022-10-25 13:26:29 +02:00
Antonio Andelic	af0c54e055	Small fix	2022-10-25 11:13:41 +00:00
Antonio Andelic	451a50d9e9	Merge branch 'master' into correctly-handle-prewhere-type	2022-10-25 10:48:35 +00:00
Maksim Kita	06fe6f3c8b	Merge pull request #31796 from kitaisreal/identifier-resolver ... Added Analyzer	2022-10-25 12:36:08 +03:00
Anton Popov	cbfa887e52	fix merge of projections	2022-10-24 23:51:57 +00:00
Anton Popov	f865b28da4	fix drop of MergeTree tables	2022-10-24 22:38:53 +00:00
Alexey Milovidov	9ee7131f67	Merge pull request #42423 from ClickHouse/optimize-partitions-in-background ... Optimize partitions in background	2022-10-24 19:41:50 +02:00
Anton Popov	dcb7667808	share transaction for projections	2022-10-24 16:28:19 +00:00
Anton Popov	128fdd785e	better interface	2022-10-24 14:44:22 +00:00
Antonio Andelic	5579d139eb	Avoid crash with big int in prewhere	2022-10-24 08:37:52 +00:00
Maksim Kita	d083200d65	Fixed code review issues	2022-10-24 10:22:38 +02:00
Maksim Kita	ca93ee7479	Fixed tests	2022-10-24 10:22:20 +02:00
Maksim Kita	fed146e198	Added SAMPLE BY support. Added SAMPLE BY, FINAL support for JOINS.	2022-10-24 10:22:20 +02:00
Igor Nikonov	54013009d3	Code migration from #41734 to align diff and git blame. ... Initial implementation was different and it changed the entire ReplicatedMergeTreeSink::commitPart() which change history provided by git blame. Then RetriesControl.retryLoop() was introduced later which significantly reduces the diff since it's like while() used before. So, check outing the current version will keep more original history in git blame, which is useful here	2022-10-24 08:20:11 +00:00
Anton Popov	cf375c9732	better interface	2022-10-23 22:29:24 +00:00
Anton Popov	b40d9200d2	better semantic of constsness of DataPartStorage	2022-10-23 15:24:20 +00:00
Anton Popov	56e5daba0c	remove DataPartStorageBuilder	2022-10-23 00:23:15 +00:00
Han Fei	2fc91fd338	fix behaviour of max_rows_to_read for trival limit queries	2022-10-22 18:27:24 +02:00
Amos Bird	f104dd08cd	Only fix LowCardinality for now	2022-10-22 15:47:13 +08:00
Azat Khuzhin	4e76629aaf	Fixes for -Wshorten-64-to-32 ... - lots of static_cast - add safe_cast - types adjustments - config - IStorage::read/watch - ... - some TODO's (to convert types in future) P.S. That was quite a journey... v2: fixes after rebase v3: fix conflicts after #42308 merged Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>	2022-10-21 13:25:19 +02:00
Antonio Andelic	3091d609fa	Merge branch 'master' into optimize-partitions-in-background	2022-10-21 11:01:41 +02:00
Alexander Tokmakov	5b03bbce2e	Merge pull request #41981 from ClickHouse/fixes_for_replicated_merge_tree2 ... Some fixes for ReplicatedMergeTree	2022-10-20 16:57:51 +03:00
Robert Schulze	469d6ad624	Merge pull request #42485 from ClickHouse/fix-bad_cast-in-annoy ... Fix bad_cast in Annoy index	2022-10-20 15:27:58 +02:00
Alexander Tokmakov	d668a82829	Merge branch 'master' into improve_replica_recovery	2022-10-20 14:25:09 +03:00
Amos Bird	2499ab3b3a	Use convertFieldToType	2022-10-20 17:53:52 +08:00
Antonio Andelic	77eb353839	Merge pull request #42243 from ClickHouse/use-multiread-more ... Use MultiRead where possible	2022-10-20 09:04:54 +02:00
Anton Popov	22da0420af	fix possible restart errors after failed quorum insert	2022-10-19 20:26:16 +00:00
Alexander Tokmakov	84ef32e33f	Merge branch 'master' into improve_replica_recovery	2022-10-19 16:28:01 +02:00
Alexander Tokmakov	f958c155f3	Merge branch 'master' into fixes_for_replicated_merge_tree2	2022-10-19 17:25:24 +03:00
Robert Schulze	42f680849c	Fix bad_cast in Annoy index ... - Problem originally found by data type fuzzer https://s3.amazonaws.com/clickhouse-test-reports/42180/2f83d8790581dce0ffeec56c137b1d13160cfa7b/fuzzer_astfuzzermsan//report.html - This commit restricts which data types are allowed for Annoy indexes (similar things are done for other index types).	2022-10-19 13:40:39 +00:00
Antonio Andelic	05e6b94b54	Polishing	2022-10-19 12:59:28 +00:00
Antonio Andelic	e5408aac99	Simplify logic	2022-10-19 12:34:20 +00:00
Amos Bird	549597fe80	Fix misbehavior of key analysis. ... When key types cannot be inside Nullable, it can break key analysis. See https://github.com/ClickHouse/ClickHouse/issues/42456.	2022-10-19 12:15:27 +08:00
Alexander Tokmakov	1d9d753167	fix	2022-10-18 18:13:18 +02:00
Alexander Tokmakov	ea77ff9022	Merge branch 'master' into fixes_for_replicated_merge_tree2	2022-10-18 17:25:25 +02:00
Nikolai Kochetov	b109767dda	Merge branch 'master' into fix-multi-arg-monotonicity	2022-10-18 14:47:33 +00:00
Kruglov Pavel	25e13bdd2f	Merge pull request #41107 from Avogar/improve-combinators ... Support all combinators combination in WindowTransform/arratReduce*/initializeAggregation/aggregate functions versioning	2022-10-18 15:24:49 +02:00
Antonio Andelic	d105796c91	Merge branch 'master' into optimize-partitions-in-background	2022-10-18 12:24:03 +00:00
Antonio Andelic	e828dce331	Disable auto optimize by default	2022-10-18 12:11:23 +00:00
Tian Xinhui	45c6974f7d	Use alias for std::shared_ptr<MergeMutateSelectedEntry> (#42211) ... * use alias for MergeMutateSelectedEntry share ptr * fix StorageMergeTree.cpp	2022-10-18 13:35:34 +02:00
Antonio Andelic	06b2eeb5a7	Small changes to settings	2022-10-18 09:31:16 +00:00
Antonio Andelic	bd2522fdd0	Merge branch 'master' into optimize-partitions-in-background	2022-10-18 09:13:59 +00:00
Nikolai Kochetov	3d1ed2969a	Fix possible LOGICAL_ERROR in binary ariphmetics monotonicity. Fix invalid evaluation of binary monotonic function in KeyCondition.	2022-10-17 18:21:03 +00:00
Alexander Tokmakov	57f6e27300	Merge branch 'master' into improve_replica_recovery	2022-10-17 18:37:31 +02:00
Nikolai Kochetov	c9aa6a655c	Merge pull request #42319 from ClickHouse/fix-alter-update-wrong-type ... Fix an invalid type of a column after attach and alter.	2022-10-17 12:44:16 +02:00
Raúl Marín	be932f49d2	Merge remote-tracking branch 'blessed/master' into perf_experiment	2022-10-17 12:07:31 +02:00
Antonio Andelic	125501b36a	Merge branch 'master' into use-multiread-more	2022-10-17 07:32:30 +00:00
avogar	52427e6028	Remove code duplication	2022-10-14 18:07:02 +00:00
Kruglov Pavel	ff11904850	Merge branch 'master' into improve-combinators	2022-10-14 17:19:31 +02:00
Nikolai Kochetov	b18c6fd8e6	Fix an invalid type of a column after attach and alter.	2022-10-14 14:52:26 +00:00
alesapin	fd718c0d00	Merge branch 'master' into disable_concurrent_parts_removal	2022-10-14 13:32:11 +02:00
alesapin	4e1a10fb34	better check	2022-10-14 13:32:08 +02:00
Raúl Marín	19310a5877	Replace std::vector with absl inlined_vector	2022-10-13 21:50:11 +02:00
Alexey Milovidov	bbd1196a96	Update MergeTreeData.cpp	2022-10-12 00:57:02 +03:00
Antonio Andelic	5e6a7c1678	Use multiread where possible	2022-10-11 09:27:46 +00:00
Alexey Milovidov	56e59dd536	Merge branch 'master' into fix-projections	2022-10-11 02:24:12 +03:00
alesapin	f6b0dc168d	Disable concurrent parts removal	2022-10-10 14:10:04 +02:00
alesapin	8a2ba6cd81	Add logging during merge tree startup (#42163)	2022-10-10 09:32:13 +02:00
Duc Canh Le	f823b9a177	add not like to atom map	2022-10-10 13:31:10 +08:00
Alexey Milovidov	e27dbf4369	Update MergeTreeDataSelectExecutor.cpp	2022-10-09 18:27:47 +03:00
Alexey Milovidov	cebc38fa6e	Fix a bug with projections and aggregate_functions_null_for_empty setting	2022-10-09 05:30:25 +02:00
Alexey Milovidov	9ee6896916	Merge pull request #42002 from ClickHouse/relax-too-many-parts-threshold ... Relax "too many parts" threshold	2022-10-08 20:17:01 +03:00
Alexey Milovidov	e4ac4a29a9	Fix error	2022-10-08 00:24:51 +02:00
Alexey Milovidov	a45f7c3023	Relax "too many parts" thresholds	2022-10-08 00:24:51 +02:00
Alexey Milovidov	169c505603	Relax "too many parts" thresholds	2022-10-08 00:24:51 +02:00
Alexander Tokmakov	4175f8cde6	abort instead of __builtin_unreachable in debug builds	2022-10-07 21:49:08 +02:00
Alexander Tokmakov	1a78e8a817	fix	2022-10-07 20:22:03 +02:00
Robert Schulze	cc24fcd6d5	Merge pull request #41873 from ClickHouse/generated-file-cleanup ... Consolidate CMake-generated config headers	2022-10-06 14:31:03 +02:00
Raúl Marín	adbaaca2f5	QOL log improvements (#41947) ... * Uniformize disk reservation logs * Remove log about destroying stuff that appears all the time * More tweaks on disk reservation logs * Reorder logs in hash join * Remove log that provides little information * Collapse part removal logs Co-authored-by: Sergei Trifonov <sergei@clickhouse.com>	2022-10-06 14:22:44 +02:00
Robert Schulze	78be400ac0	Merge remote-tracking branch 'origin/master' into generated-file-cleanup	2022-10-06 12:22:43 +00:00
Anton Popov	aabec50988	Merge pull request #42040 from CurtizJ/async-inserts-log ... Add system table `asynchronous_insert_log`	2022-10-06 13:14:41 +02:00
Vladimir C	3f24c904ad	Merge pull request #41842 from amosbird/index_analysis_1	2022-10-06 11:14:21 +02:00
Robert Schulze	da5a2e2db0	Merge remote-tracking branch 'origin/master' into generated-file-cleanup ... Physical merge conflicts: - src/Common/ZooKeeper/ZooKeeperImpl.cpp - src/Core/config_core.h.in - src/Functions/FunctionsAES.h - src/Functions/config_functions.h.in - src/configure_config.cmake Logical merge conflicts: - Functions/tryDecrypt.cpp	2022-10-06 08:43:25 +00:00
Anton Popov	190e1f6830	Merge branch 'master' into async-inserts-log	2022-10-06 01:59:27 +02:00
Anton Popov	f7cf35d1dd	Merge pull request #42072 from CurtizJ/fix-fuzzer-4 ... Fix assertion in bloom filter index	2022-10-05 16:38:13 +02:00
Anton Popov	02cdc20d5d	Merge pull request #40096 from CurtizJ/fuzz-table-definitions ... Add fuzzer for table definitions	2022-10-05 14:53:48 +02:00
Anton Popov	2f423071f2	fix assertion in bloom filter index	2022-10-04 19:27:56 +00:00
Nikita Mikhaylov	2933a28a5d	Add log message and lower the retry timeout in MergeTreeRestartingThread (#42026)	2022-10-04 20:07:11 +02:00
Anton Popov	9a2cabd3a1	Merge branch 'master' into async-inserts-log	2022-10-04 15:10:50 +02:00
Anton Popov	8ca73eab03	Merge branch 'master' into fuzz-table-definitions	2022-10-04 15:10:31 +02:00
Kseniia Sumarokova	6d02fa3c5f	Merge pull request #42022 from kssenii/better-exception-merge-tree-data-load-parts ... More detailed exception message	2022-10-04 11:20:35 +02:00
Anton Popov	148995894f	add system table asynchronous_inserts_log	2022-10-03 21:35:55 +00:00
Anton Popov	5cb6357549	fix style check	2022-10-03 19:37:27 +00:00
Alexey Milovidov	c8872fe96d	Merge pull request #42001 from ClickHouse/part-thresholds-override-more-sane ... More sane behavior of part number thresholds override in query level settings	2022-10-03 17:25:39 +03:00
Anton Popov	6e61cf92f5	Merge remote-tracking branch 'upstream/master' into HEAD	2022-10-03 13:16:57 +00:00
kssenii	0dc71e9883	More detailed exception message	2022-10-03 14:21:53 +02:00
Alexander Tokmakov	00914a174d	Merge pull request #41410 from ClickHouse/keeper-multiread ... Add MultiRead support in Keeper and internal ZK client	2022-10-03 14:17:57 +03:00
Alexander Tokmakov	a2c29076d6	avoid detached broken part duplicates	2022-10-03 12:27:17 +02:00
Alexander Tokmakov	3f9c9c9a77	fix intersecting part if session expired during queue initialization	2022-10-03 12:27:17 +02:00
Alexander Tokmakov	7b5b7e4963	Merge pull request #41945 from ClickHouse/enable_more_s3_tests ... Enable some disabled S3 tests	2022-10-03 13:25:50 +03:00
Robert Schulze	db5ef7b3cb	Merge branch 'master' into generated-file-cleanup	2022-10-02 23:13:18 +02:00
Alexey Milovidov	0da9ed8a18	Merge branch 'master' into part-thresholds-override-more-sane	2022-10-02 02:26:21 +03:00
Alexander Tokmakov	e40917a9c5	fix	2022-10-02 00:44:46 +02:00
Alexey Milovidov	7b71524109	More sane behavior of part number thresholds override in query level settings	2022-10-01 20:18:12 +02:00
Vitaly Baranov	7be9744bab	Remove confusing warning when inserting with perform_ttl_move_on_insert=false.	2022-09-30 20:51:48 +02:00
alesapin	00bf90186a	Better	2022-09-30 19:03:54 +02:00
alesapin	350ff96628	Fix potential race condition with broken parts	2022-09-30 19:01:06 +02:00
Vladimir C	895afdec45	Merge pull request #40893 from ClickHouse/vdimir/track-tmp-disk	2022-09-30 11:27:24 +02:00
Robert Schulze	cc92a2d174	Merge branch 'master' into generated-file-cleanup	2022-09-30 09:56:31 +02:00
alesapin	0097f15ee7	Merge pull request #41832 from ClickHouse/make_copy_instead_of_hardlink ... Use copy for some operations instead of hardlinks	2022-09-29 17:12:08 +02:00
Alexander Tokmakov	4e422b8046	Merge pull request #41741 from ClickHouse/fix_intersecting_parts ... Fix intersecting parts	2022-09-29 18:00:22 +03:00
Amos Bird	44eaa9583b	Merge branch 'master' into index_analysis_1	2022-09-29 20:25:03 +08:00
vdimir	c0898ce289	Use abstraction for temporary data on disk in Sort and Aggregation	2022-09-29 09:51:41 +00:00
Antonio Andelic	fcc54104a4	Define methods for multi read requests	2022-09-29 09:19:47 +00:00