15578 Commits

Author	SHA1	Message	Date
Alexey Milovidov	68d3f60ef7	Merge branch 'master' into system-symbols	2023-11-08 17:07:11 +01:00
Ilya Golshtein	6e189b17c0	merge_row_policy: processAliases introduced	2023-11-08 18:55:19 +03:00
flynn	d0ba561faa	Update src/Storages/transformQueryForExternalDatabase.cpp ... Co-authored-by: Sema Checherinda <104093494+CheSema@users.noreply.github.com>	2023-11-08 23:20:12 +08:00
Antonio Andelic	4438c2f70a	Remove unnecassary log	2023-11-08 14:36:39 +00:00
Antonio Andelic	1f000242a1	Merge branch 'master' into keeper-map-backup-restore	2023-11-08 13:16:47 +00:00
Antonio Andelic	18a5eeec38	Make on cluster backup/restore work	2023-11-08 13:16:38 +00:00
kssenii	a5b9ba51cd	Better test	2023-11-08 12:55:14 +01:00
flynn	399d61b222	Fix transfer query to MySQL compatible query	2023-11-08 11:34:49 +00:00
kssenii	dfcc94fff0	Fix test	2023-11-08 12:29:40 +01:00
Antonio Andelic	86ba6ad1e8	Local backup and restore	2023-11-08 10:22:44 +00:00
Alexey Milovidov	ca83da14f2	Merge branch 'master' of github.com:ClickHouse/ClickHouse into system-symbols	2023-11-07 19:43:19 +01:00
slvrtrn	f663228404	Add information_schema.statistics	2023-11-07 18:59:45 +01:00
kssenii	15517d04df	Fix	2023-11-07 18:02:44 +01:00
kssenii	70048236f3	Add ProfileEvents column to system.backups	2023-11-07 17:32:08 +01:00
kssenii	9dce1a52e1	Review fix	2023-11-07 16:31:41 +01:00
Alexey Milovidov	5169e4e54d	Fix FreeBSD	2023-11-07 15:30:31 +01:00
Alexander Tokmakov	706f2dcf54	Add a metric for suspicious parts in ZooKeeper (#56395)	2023-11-07 12:55:57 +01:00
Nikolai Kochetov	c58c583043	Fix tests.	2023-11-07 11:43:16 +00:00
Nikita Mikhaylov	0555f8a68c	Fix crash in case of adding a column with type Object(JSON) (#56307)	2023-11-07 12:01:52 +01:00
serxa	19aa1375fc	Merge branch 'master' into async-loader-integration	2023-11-07 10:25:38 +00:00
Nikita Mikhaylov	b696540a5a	Merge branch 'master' into rocks-db-analyzer	2023-11-07 00:44:49 +01:00
Vitaly Baranov	1f4697fbb7	Merge pull request #56339 from vitlibar/fix-using-table-shared-id-during-backup-and-improve-logs ... Fix using table shared id during backup and improve logs.	2023-11-07 00:04:59 +01:00
Alexey Milovidov	9e6dee93f3	Fix FreeBSD	2023-11-06 23:26:11 +01:00
Alexander Tokmakov	cf6ab2e110	Merge pull request #56282 from ClickHouse/fix_unexpected_parts_after_truncate ... Fix unexpected parts after drop range	2023-11-06 21:03:57 +01:00
Nikolai Kochetov	34fdb8a7c6	Fix rocksdb with analyzer.	2023-11-06 18:46:39 +00:00
kssenii	7883a74f6d	Fix	2023-11-06 19:05:12 +01:00
Sema Checherinda	a5e17dc919	Merge pull request #55566 from CheSema/remote-blobs-naming ... feature: storage_metadata_write_full_object_key	2023-11-06 16:41:49 +01:00
Alexey Milovidov	66b180d751	Fix FreeBSD	2023-11-06 16:31:02 +01:00
kssenii	f2c0434c4d	Merge remote-tracking branch 'origin/master' into minor-improvements-for-s3-queue	2023-11-06 15:51:11 +01:00
kssenii	8db2aca832	Add a test	2023-11-06 15:48:04 +01:00
kssenii	da21413354	Better shutdown	2023-11-06 15:47:57 +01:00
kssenii	9952261a59	Add missing write to s3queue_log, pass cached FileStatus	2023-11-06 15:47:45 +01:00
Alexander Tokmakov	13599ff462	review fixes	2023-11-06 15:31:54 +01:00
Alexander Tokmakov	eba52b318d	Merge pull request #56272 from ClickHouse/rdb_fix_rare_logical_error ... Fix rare logical error in Replicated database	2023-11-06 14:14:55 +01:00
Ilya Golshtein	d900f8a352	merge_row_policy: aliases slightly work	2023-11-06 15:18:25 +03:00
Sema Checherinda	52850d5a45	Update StorageSystemRemoteDataPaths.cpp	2023-11-06 11:45:17 +01:00
Kseniia Sumarokova	a050cdd897	Merge branch 'master' into delta-lake-repeated-add	2023-11-06 11:30:58 +01:00
Duc Canh Le	4c21ba7b6f	tables auto initialize new disks without restart ... Signed-off-by: Duc Canh Le <duccanh.le@ahrefs.com>	2023-11-06 10:26:48 +00:00
Alexey Milovidov	99fe7a46fc	Minor changes	2023-11-06 01:12:13 +01:00
Alexey Milovidov	8182d51978	Add table system.symbols	2023-11-06 01:11:53 +01:00
Vitaly Baranov	1e89e7a6b2	Fix using table shared id during backup and improve logs.	2023-11-05 09:38:08 +01:00
Igor Nikonov	9bdcba1b20	Merge branch 'master' into pr-progress-bar	2023-11-05 00:09:46 +01:00
Alexander Tokmakov	155bdfcf0d	fix	2023-11-04 22:50:19 +01:00
Alexey Milovidov	cd94d02455	Merge branch 'master' into coverage	2023-11-04 21:49:59 +01:00
Arthur Passos	b6e205dcdf	Add ClickHouse setting to disable tunneling for HTTPS requests over HTTP proxy (#55033) ... * initial commit. integ tests passing, need to re-run unit & my own personal tests * partial refactoring to remove Protocol::ANY * improve naming * remove all usages of ProxyConfiguration::Protocol::ANY * fix ut * blabla * support url functions as well * support for HTTPS requests over HTTP proxy with tunneling off * remove gtestabc * fix silly mistake * ... * remove usages of httpclientsession::proxyconfig in src/ * got you * remove stale comment * it seems like I need reasonable defaults * fix ut * add some comments * remove no longer needed header * matrix out * add https over http proxy with no tunneling * soem docs * partial refactoring * rename to use_tunneling_for_https_requests_over_http_proxy * improve docs * use shorter version * remove useless test * rename the setting * update * fix typo * fix setting docs typo * move ); up * move ) up	2023-11-04 13:47:52 -04:00
Alexander Tokmakov	9acd3707cb	fix tests	2023-11-04 15:44:55 +01:00
Nikita Taranov	26d3a3a1e0	Add assertion that SizePredictor is set if preferred_block_size_bytes is set (#56302) ... * Test * fix	2023-11-03 23:13:18 +01:00
alesapin	f69796471c	Merge pull request #56303 from ClickHouse/fix_obscure_segfault ... Fix rare race condition related to Memory allocation failure	2023-11-03 23:00:11 +01:00
Alexander Tokmakov	a44e27b0dc	fix	2023-11-03 22:28:10 +01:00
Nikita Mikhaylov	000d6cd7e5	Merge pull request #56234 from evillique/fix-suggestions ... Fix client suggestions for user without grants	2023-11-03 17:55:51 +01:00
Alexander Sapin	3b212a2593	Reserver in advance	2023-11-03 16:53:52 +01:00
Alexander Sapin	a0a0dd02d5	Fix typo everywhere	2023-11-03 16:40:50 +01:00
Alexander Sapin	c3dcb81a62	Fix race condition in case of memory errors	2023-11-03 16:00:16 +01:00
Alexander Sapin	a9ffb9808e	Merge branch 'master' into fix_obscure_segfault	2023-11-03 12:20:52 +01:00
Kseniia Sumarokova	31ae602d37	Merge pull request #56131 from ClickHouse/backup-some-more-retries ... Add missing zookeeper retries in StorageReplicatedMergeTree::backupData	2023-11-03 12:00:41 +01:00
Alexander Tokmakov	b50c4c9f3b	fix	2023-11-03 11:24:10 +01:00
vdimir	86524fce0f	Merge pull request #56246 from ClickHouse/vdimir/analyzer_01019_alter_materialized_view ... Analyzer: fix 01019_alter_materialized_view_consistent	2023-11-03 11:04:14 +01:00
vdimir	a70d0c5203	Merge pull request #56264 from azat/rocksdb/block-based-table-options ... Add ability to specify BlockBasedTableOptions for rocksdb	2023-11-03 10:15:19 +01:00
vdimir	55a1baf029	Merge pull request #56022 from ClickHouse/vdimir/check_all_tables ... Add CHECK query for all the tables	2023-11-03 10:10:42 +01:00
Alexander Tokmakov	480e58b427	fix unexpected parts after drop range	2023-11-03 01:47:50 +01:00
alesapin	2daff2d41e	Fxi	2023-11-03 00:32:07 +01:00
Alexey Milovidov	b400a6682e	Merge pull request #56275 from ClickHouse/23.11-prepare ... Update version after release	2023-11-02 20:20:03 +01:00
Alexey Milovidov	1348d4c696	Update autogenerated version to 23.11.1.1 and contributors	2023-11-02 20:18:15 +01:00
Alexander Tokmakov	e88fa10aa2	Merge pull request #56142 from ClickHouse/make_replicated_start_more_robust ... Better process broken parts on table start for replicated tables	2023-11-02 20:11:22 +01:00
Alexander Tokmakov	0ce1560e5d	retry if table replica being restarted	2023-11-02 18:30:32 +01:00
serxa	e97edf5285	Merge branch 'master' into async-loader-integration	2023-11-02 14:56:32 +00:00
Robert Schulze	99dd3ed060	Merge pull request #56226 from HarryLeeIBM/hlee-ftsearch-compress ... Add compression of posting lists for inverted index	2023-11-02 15:56:30 +01:00
Azat Khuzhin	decf85baca	Add ability to specify BlockBasedTableOptions for rocksdb ... Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>	2023-11-02 15:36:00 +01:00
Sema Checherinda	90b64bcdb9	backward compatibility and implementation feature storage_metadata_write_full_object_key	2023-11-02 15:20:55 +01:00
kssenii	d0bc77f084	Fxi	2023-11-02 14:45:15 +01:00
Joe Lynch	67e29d5eb9	Allow repeated adds and removes for delta lake files	2023-11-02 13:40:50 +01:00
alesapin	3ba12b6e3e	Merge pull request #56238 from evillique/remove-less-parts ... Do not remove part if `Too many open files` is thrown	2023-11-02 12:33:02 +01:00
Alexey Milovidov	1267246785	Merge pull request #55642 from ClickHouse/morton-utils ... Indexing by space-filling curves	2023-11-02 12:21:33 +01:00
alesapin	5c4915e264	Update src/Storages/MergeTree/MergeTreePartInfo.cpp ... Co-authored-by: Alexander Tokmakov <tavplubix@clickhouse.com>	2023-11-02 12:02:04 +01:00
alesapin	7917a2cb40	Update src/Storages/MergeTree/MergeTreePartInfo.cpp ... Co-authored-by: Alexander Tokmakov <tavplubix@clickhouse.com>	2023-11-02 12:01:58 +01:00
alesapin	800f2f097c	Update src/Storages/MergeTree/MergeTreePartInfo.cpp ... Co-authored-by: Alexander Tokmakov <tavplubix@clickhouse.com>	2023-11-02 12:01:51 +01:00
alesapin	81b93ef12b	Update src/Storages/MergeTree/MergeTreePartInfo.cpp ... Co-authored-by: Alexander Tokmakov <tavplubix@clickhouse.com>	2023-11-02 12:01:46 +01:00
vdimir	5703e0995b	Analyzer: fix 01019_alter_materialized_view_consistent	2023-11-02 10:27:22 +00:00
Nikolay Degterinsky	0d6a0d04ae	Do not remove parts if 'Too many open files' is thrown	2023-11-02 05:01:53 +00:00
Nikolay Degterinsky	f93c032425	Fix suggestions for user without grants	2023-11-02 03:56:59 +00:00
Nikolay Degterinsky	1af52d3b00	Better	2023-11-02 00:42:57 +00:00
Igor Nikonov	823b62a55c	Merge pull request #56162 from ClickHouse/part-check-scheduling-monotonic-time ... Use monotonic clock for part check scheduling	2023-11-01 23:15:27 +01:00
Azat Khuzhin	09c1e76982	Allow manual compaction of rocksdb via OPTIMIZE query ... Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>	2023-11-01 22:27:11 +01:00
HarryLeeIBM	013e4b40c6	Add compression of posting lists for inverted index	2023-11-01 14:14:38 -07:00
Igor Nikonov	447386edd9	Merge branch 'master' into fix-schedule-delayed-part-checks	2023-11-01 21:26:06 +01:00
Duc Canh Le	7d4c97e8f3	print broken part path before throw (#56181) ... Signed-off-by: Duc Canh Le <duccanh.le@ahrefs.com>	2023-11-01 19:11:23 +01:00
Alexander Sapin	1e0cab93f6	Satisfy tidy	2023-11-01 16:51:20 +01:00
Alexander Sapin	bfcd8b7de5	Better logging and tests	2023-11-01 16:50:02 +01:00
Azat Khuzhin	c25d6cd624	Rename directory monitor concept into background INSERT (#55978) ... * Limit log frequence for "Skipping send data over distributed table" message After SYSTEM STOP DISTRIBUTED SENDS it will constantly print this message. Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com> * Rename directory monitor concept into async INSERT Rename the following query settings (with preserving backward compatiblity, by keeping old name as an alias): - distributed_directory_monitor_sleep_time_ms -> distributed_async_insert_sleep_time_ms - distributed_directory_monitor_max_sleep_time_ms -> distributed_async_insert_max_sleep_time_ms - distributed_directory_monitor_batch -> distributed_async_insert_batch_inserts - distributed_directory_monitor_split_batch_on_failure -> distributed_async_insert_split_batch_on_failure Rename the following table settings (with preserving backward compatiblity, by keeping old name as an alias): - monitor_batch_inserts -> async_insert_batch - monitor_split_batch_on_failure -> async_insert_split_batch_on_failure - directory_monitor_sleep_time_ms -> async_insert_sleep_time_ms - directory_monitor_max_sleep_time_ms -> async_insert_max_sleep_time_ms And also update all the references: $ gg -e directory_monitor_ -e monitor_ tests docs | cut -d: -f1 | sort -u | xargs sed -e 's/distributed_directory_monitor_sleep_time_ms/distributed_async_insert_sleep_time_ms/g' -e 's/distributed_directory_monitor_max_sleep_time_ms/distributed_async_insert_max_sleep_time_ms/g' -e 's/distributed_directory_monitor_batch_inserts/distributed_async_insert_batch/g' -e 's/distributed_directory_monitor_split_batch_on_failure/distributed_async_insert_split_batch_on_failure/g' -e 's/monitor_batch_inserts/async_insert_batch/g' -e 's/monitor_split_batch_on_failure/async_insert_split_batch_on_failure/g' -e 's/monitor_sleep_time_ms/async_insert_sleep_time_ms/g' -e 's/monitor_max_sleep_time_ms/async_insert_max_sleep_time_ms/g' -i Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com> * Rename async_insert for Distributed into background_insert This will avoid amigibuity between general async INSERT's and INSERT into Distributed, which are indeed background, so new term express it even better. Mostly done with: $ git di HEAD^ --name-only | xargs sed -i -e 's/distributed_async_insert/distributed_background_insert/g' -e 's/async_insert_batch/background_insert_batch/g' -e 's/async_insert_split_batch_on_failure/background_insert_split_batch_on_failure/g' -e 's/async_insert_sleep_time_ms/background_insert_sleep_time_ms/g' -e 's/async_insert_max_sleep_time_ms/background_insert_max_sleep_time_ms/g' Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com> * Mark 02417_opentelemetry_insert_on_distributed_table as long CI: https://s3.amazonaws.com/clickhouse-test-reports/55978/7a6abb03a0b507e29e999cb7e04f246a119c6f28/stateless_tests_flaky_check__asan_.html Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com> --------- Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>	2023-11-01 15:09:39 +01:00
Alexander Tokmakov	65f3cf81dc	Merge pull request #55987 from ClickHouse/fix_lost_part_count ... Avoid counting lost parts twice	2023-11-01 15:07:27 +01:00
Kruglov Pavel	5533258091	Merge pull request #55477 from Avogar/streaming-errors ... Allow to save unparsed records and errors in RabbitMQ, NATS and FileLog engines	2023-11-01 14:35:35 +01:00
vdimir	2ca1a14ac4	Merge pull request #55741 from ClickHouse/vdimir/fix_analyzer_materialized_distributed_join ... Analyzer: fix block stucture mismatch in matview with engine distributed	2023-11-01 14:16:33 +01:00
Kseniia Sumarokova	4510014014	Update StorageReplicatedMergeTree.cpp	2023-11-01 14:05:12 +01:00
Kseniia Sumarokova	c300297153	Fix style check	2023-11-01 13:27:38 +01:00
Alexander Sapin	b4328ad8dc	Fix stupid bug	2023-11-01 13:06:19 +01:00
kssenii	68fd273114	Better	2023-11-01 11:18:58 +01:00
Zhiguo Zhou	b66a9e8fd3	Optimize DB::combineFilters with AVX512_VBMI2 intrinsic ... The DB::combineFilters combines two UInt8 columns (filters) by scanning the first column for non-zero elements and replacing them with elements in the second column contiguously. This function is optimized with Intel's AVX512_VBMI2 intrinsic, _mm512_mask_expandloadu_epi8, which is designed for implementing the exact operation when the bits in the writemask are set when the corresponding packed integers in the first filter are non-zero, and are not set otherwise. The dynamic dispatch ensures that this optimization only applies to devices with AVX512_VBMI2 support.	2023-11-01 17:14:40 +08:00
Zhiguo Zhou	1ae951efc2	Add unittest MergeTree.CombineFilters ... This commit adds a unittest for validating DB::combineFilters.	2023-11-01 17:14:40 +08:00
jsc0218	f1c7665feb	Merge pull request #56121 from lithiumlabcompany/fix-auto-configuration-queue-for-librdkafka ... Correctly set queued.min.messages	2023-10-31 19:59:04 -04:00
Michael Kolupaev	9c79c165b9	Re-fix 'Block structure mismatch' on concurrent ALTER and INSERTs in Buffer table (#56140) ... * Fix 'Block structure mismatch' on concurrent ALTER and INSERTs in Buffer table * Fix the test	2023-10-31 13:41:54 -07:00
Alexander Sapin	3a1182f6c0	Proper implementation	2023-10-31 18:53:33 +01:00
kssenii	ae1a7fffec	Fix	2023-10-31 16:19:08 +01:00
Alexander Tokmakov	f39f0c6a74	Merge pull request #56083 from ClickHouse/fix_drop_range_commit_existing_part_race ... Fix race condition between DROP_RANGE and committing existing block	2023-10-31 14:24:53 +01:00
Alexander Sapin	f4081e3f85	Merge branch 'master' into make_replicated_start_more_robust	2023-10-31 14:16:39 +01:00
Alexander Tokmakov	4d437ca462	Merge pull request #56137 from ClickHouse/fix_unexpected_parts_handling ... Do not try to activate covered parts when handilng unexpected parts	2023-10-31 14:00:47 +01:00
Azat Khuzhin	11b00e92fd	Add more details to "Data after merge is not byte-identical to data on another replicas" ... It should be a very rare error, but in case of error you need as much details as there are: - print mismatched info (hash, size) - print all files with size and checksum (this may help for instance when new file is added, or when extesion changed -- compression marks) P.S. maybe detach_not_byte_identical_parts should be enabled by default? Thoughts? Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>	2023-10-31 12:31:27 +01:00
Igor Nikonov	f8a7bdb8f0	Use monotonic time for part checks scheduling	2023-10-31 11:21:20 +00:00
Alexey Milovidov	016285ea87	Merge branch 'master' into morton-utils	2023-10-31 01:47:16 +01:00
Han Fei	1237cef26d	address comments	2023-10-31 00:53:46 +01:00
alesapin	20b8954872	Better process broken parts on table start for replicated tables	2023-10-30 23:16:04 +01:00
avogar	7eefb72c53	Merge branch 'master' of github.com:ClickHouse/ClickHouse into schema-inference-union	2023-10-30 19:44:26 +00:00
Alexander Tokmakov	373d8bbb20	do not try to activate covered parts	2023-10-30 20:17:10 +01:00
Igor Nikonov	0958d0b168	Fix style	2023-10-30 16:35:00 +00:00
Nikolai Kochetov	cc3c038394	Fixing test.	2023-10-30 16:29:22 +00:00
kssenii	6fc4c8d332	Add retries	2023-10-30 17:20:17 +01:00
Alexander Tokmakov	ae8d7a557b	Update ReplicatedMergeTreeSink.cpp	2023-10-30 15:43:48 +01:00
Alexander Tokmakov	073a6a6f0e	Update src/Storages/StorageReplicatedMergeTree.cpp ... Co-authored-by: Raúl Marín <git@rmr.ninja>	2023-10-30 15:33:35 +01:00
Alexander Tokmakov	6635b84a46	fixes	2023-10-30 15:20:04 +01:00
Igor Nikonov	f2acc366f7	Fix: schedule delayed part checks correctly	2023-10-30 12:57:43 +00:00
Nikolai Kochetov	554ceb4e1d	Merge branch 'master' into planner-prepare-filters-for-analysis-2	2023-10-30 11:56:30 +01:00
Stas Morozov	5a922014f7	Avoid problem with configuration queued.min.messages automatically for Kafka storage	2023-10-30 13:53:52 +03:00
avogar	9eb498b48d	Fix tests for HDFS	2023-10-30 10:50:05 +00:00
Kruglov Pavel	e22cfa3d5b	Remove unneeded changge	2023-10-30 11:35:44 +01:00
Kruglov Pavel	53b6a455fc	Remove unused field from .h file	2023-10-30 11:35:09 +01:00
Kruglov Pavel	ba736beaf2	Remove unused field	2023-10-30 11:34:42 +01:00
Han Fei	19cdc6dfe2	Merge branch 'master' into hanfei/statistic	2023-10-30 04:53:18 +01:00
Alexey Milovidov	3631e476eb	Merge pull request #54947 from amosbird/minmax-combinator ... Introduce -ArgMin/-ArgMax combinators.	2023-10-30 03:35:28 +01:00
Alexey Milovidov	64b6e68a50	Merge pull request #55683 from amosbird/issue-55653 ... Reuse granule during skip index reading	2023-10-30 00:51:51 +01:00
Han Fei	df1e019268	address comments	2023-10-30 00:39:16 +01:00
Alexey Milovidov	8e6a7fdff0	Fix f-string	2023-10-29 23:43:36 +01:00
Alexey Milovidov	9aff0a8872	Revert "Fix 'Block structure mismatch' on concurrent ALTER and INSERTs in Buffer table (#55995)" ... This reverts commit b65c498016.	2023-10-29 04:34:04 +03:00
Alexey Milovidov	1b085b1161	Merge pull request #55900 from ClickHouse/make-test-better ... Make test `system_zookeeper_connection` better	2023-10-28 15:01:36 +02:00
Nikita Taranov	213ce63ffb	Merge branch 'master' into optimize_reading3	2023-10-28 00:04:20 +02:00
Michael Kolupaev	b65c498016	Fix 'Block structure mismatch' on concurrent ALTER and INSERTs in Buffer table (#55995)	2023-10-27 10:06:43 -07:00
Alexander Tokmakov	e07bb123d3	fix race condition between DROP_RANGE and committing existing block	2023-10-27 17:50:45 +02:00
Alexander Tokmakov	db5e9f1d1c	beter check for duplicate lost parts	2023-10-27 17:33:30 +02:00
vdimir	faffdba8a2	Split source and worker processors in InterpreterCheckQuery	2023-10-27 15:22:10 +00:00
vdimir	c1ae369919	wip CHECK query for all tables	2023-10-27 15:22:10 +00:00
avogar	e630868eac	Fix tests	2023-10-27 14:11:27 +00:00
Kruglov Pavel	bb4b95e891	Merge branch 'master' into schema-inference-union	2023-10-27 14:53:58 +02:00
Sergei Trifonov	a5d7c2b6a2	Merge branch 'master' into async-loader-integration	2023-10-27 10:39:12 +02:00
Kruglov Pavel	570b66f027	Merge branch 'master' into schema-inference-union	2023-10-26 19:26:00 +02:00
Alexander Tokmakov	9db5f4396c	Merge branch 'master' into fix_lost_part_count	2023-10-26 17:40:37 +02:00
Andrey Zvonov	5737369d0b	Merge branch 'ClickHouse:master' into zvonand-datetime-ranges	2023-10-26 11:42:13 +02:00
vdimir	12890eb70c	Merge pull request #55806 from canhld94/rocksdb_trivial_count ... Support trivial count for rocksdb and storage join	2023-10-26 11:22:38 +02:00
vdimir	1d5f75fd71	Merge pull request #53404 from ClickHouse/vdimir/check_table_improvements2 ... Improve CHECK TABLE system query, support checking part	2023-10-26 10:57:44 +02:00
Nikolay Degterinsky	3c690337ec	Improvements	2023-10-26 03:17:30 +00:00
zvonand	0766c73aab	Rename date_time_overflow_mode -> date_time_overflow_behavior, moved it to format settings	2023-10-25 23:11:13 +02:00
Igor Nikonov	1a5d485c6c	Merge branch 'master' into pr-progress-bar	2023-10-25 19:53:20 +02:00
Alexander Tokmakov	3c0b8aeb6a	avoid counting lost parts twice	2023-10-25 19:29:40 +02:00
Alexander Tokmakov	c0482cb537	Merge pull request #55944 from azat/rmt-retriable-exception-log ... Do not write retriable errors for Replicated mutate/merge into error log	2023-10-25 18:25:59 +02:00
alesapin	0d68a52c06	Merge pull request #50181 from ClickHouse/repro_49887 ... Make exception about `ALTER TABLE ... DROP COLUMN|INDEX|PROJECTION` more clear	2023-10-25 15:57:17 +02:00
Nikolay Degterinsky	2606b60e66	Fix REPLICA_ALREADY_EXISTS for ReplicatedMergeTree	2023-10-25 04:17:35 +00:00
Igor Nikonov	7b7f100250	Merge remote-tracking branch 'origin/master' into pr-progress-bar	2023-10-24 21:02:29 +00:00
李扬	465962df7f	Support orc filter push down (file + stripe + rowgroup level) (#55330) ... * support orc filter push down * update orc lib version * replace setqueryinfo with setkeycondition * fix issue https://github.com/ClickHouse/ClickHouse/issues/53536 * refactor source with key condition * fix building error * remove std::cout * update orc * update orc version * fix bugs * improve code * upgrade orc lib * fix code style * change as requested * add performance tests for orc filter push down * add performance tests for orc filter push down * fix all bugs * fix default as null issue * add uts for null as default issues * upgrade orc lib * fix failed orc lib uts and fix typo * fix failed uts * fix failed uts * fix ast fuzzer tests * fix bug of uint64 overflow in https://s3.amazonaws.com/clickhouse-test-reports/55330/de22fdcaea2e12c96f300e95f59beba84401712d/fuzzer_astfuzzerubsan/report.html * fix asan fatal caused by reused column vector batch in native orc input format. refer to https://s3.amazonaws.com/clickhouse-test-reports/55330/be39d23af2d7e27f5ec7f168947cf75aeaabf674/stateless_tests__asan__[4_4].htm * fix wrong performance tests * disable 02892_orc_filter_pushdown on aarch64. https://s3.amazonaws.com/clickhouse-test-reports/55330/be39d23af2d7e27f5ec7f168947cf75aeaabf674/stateless_tests__aarch64_.html * add some comments * add some comments * inline range::equals and range::less * fix data race of key condition * trigger ci	2023-10-24 12:08:17 -07:00
Kruglov Pavel	844c1fb688	Fix	2023-10-24 19:35:03 +02:00
Ilya Golshtein	0d27150948	merge_row_policy: cleanup	2023-10-24 13:09:57 +00:00
Kruglov Pavel	cb63b07e89	Merge pull request #55912 from wxybear/fix/storage_null_supports_subcolumns ... fix: StorageNull supports subcolumns	2023-10-24 14:03:46 +02:00
Alexander Gololobov	04b82d6463	Merge pull request #55935 from ClickHouse/set_has_lwd ... Set storage.has_lightweight_delete_parts flag when a part has been loaded	2023-10-24 14:01:11 +02:00
serxa	749cdb9954	Merge branch 'master' into async-loader-integration	2023-10-24 10:09:49 +00:00
Azat Khuzhin	da408df4a7	Proper ignoring of retriable errors during merge/mutate ... The exception cannot be simply suppressed, since sanity checks in the WriteBuffer dtor relies on the std::uncaught_exceptions(), and if the exception will be suppressed, then if the buffer was not finalized, it may abort (even though it is only in debug/sanitizers build). So instead, IExecutableTask::printExecutionException() had been introduced to distinguish when exception should be printed and when not.	2023-10-24 10:19:18 +02:00
Duc Canh Le	e3aa799ca2	disable approximate count by default + fix tests ... Signed-off-by: Duc Canh Le <duccanh.le@ahrefs.com>	2023-10-24 03:18:33 +00:00
Srikanth Chekuri	8876a92b42	Merge remote-tracking branch 'upstream/master' into issue-55023	2023-10-23 21:21:25 +05:30
avogar	cfa510ea0a	Add more documentation, fix build	2023-10-23 14:38:34 +00:00
Robert Schulze	0703c88381	Merge pull request #55915 from rschu1ze/sanity-check-rpnbuildertreenode ... Add sanity check to RPNBuilderFunctionTreeNode	2023-10-23 16:09:56 +02:00
Robert Schulze	601276aa60	Update src/Storages/MergeTree/RPNBuilder.cpp ... Co-authored-by: Igor Nikonov <954088+devcrafter@users.noreply.github.com>	2023-10-23 15:23:23 +02:00
Azat Khuzhin	38a4547c05	Do not write retriable errors for Replicated mutate/merge into error log ... Fixes: e3f892f84a ("fix gtest with MemoryWriteBuffer, do not mute exception in ReplicatedMergeMutateTaskBase") Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>	2023-10-23 15:14:58 +02:00
Igor Nikonov	0bbf7ba921	Merge branch 'master' into pr-progress-bar	2023-10-23 14:56:59 +02:00
vdimir	7583394c9c	Support CHECK TABLE ... PART ... query	2023-10-23 12:33:35 +00:00
Kruglov Pavel	32da07b4ee	Merge pull request #55695 from Avogar/restrict-iceberg ... Fixes and improvements for Iceberg storage	2023-10-23 13:58:23 +02:00
Alexander Gololobov	99a14410ee	Set storage.has_lightweight_delete_parts flag when a part has been loaded	2023-10-23 13:36:27 +02:00
vdimir	8f0d7954ff	IStorage::checkDataNext returns optional	2023-10-23 10:12:30 +00:00
Robert Schulze	5cf3c95ea4	Update RPNBuilder.cpp	2023-10-23 11:49:52 +02:00
vdimir	fe95c0d0e4	Improve CHECK TABLE system query ... Resubmit PR #52745	2023-10-23 09:35:26 +00:00
Robert Schulze	77e7f71e02	Add sanity check to RPNBuilderFunctionTreNode	2023-10-22 19:56:05 +00:00
wxybear	53a49164d6	fix: StorageNull supports subcolumns	2023-10-22 16:24:01 +08:00
Srikanth Chekuri	327386a348	Merge remote-tracking branch 'upstream/master' into issue-55023	2023-10-22 10:22:14 +05:30
Alexey Milovidov	2972dbdc40	Merge branch 'master' into morton-utils	2023-10-22 03:52:30 +02:00
Alexey Milovidov	2b72430e24	Make test system_zookeeper_connection better	2023-10-21 23:56:20 +02:00
Srikanth Chekuri	852d627f6a	Introduce setting create_table_empty_primary_key_by_default ... If set to true and the table of MergeTree family does not contain ORDER BY and PRIMARY KEY, then an empty tuple is used by default	2023-10-21 21:37:04 +05:30
Igor Nikonov	e378d920c4	Merge branch 'master' into pr-progress-bar	2023-10-21 13:16:12 +02:00
Kruglov Pavel	6f61ccfe28	Merge branch 'master' into schema-inference-union	2023-10-20 22:54:11 +02:00
avogar	6934e27e8b	Add union mode for schema inference to infer union schema of files with different schemas	2023-10-20 20:46:41 +00:00
Nikita Taranov	2a6a97c1fa	read sync from cache	2023-10-20 22:24:56 +02:00
Anton Popov	5819bcd07a	Support asynchronous inserts for native protocol (#54730) ... * support async insert for native protocol * use separate queue for async inserts via native protocol * fix test * better logging for async inserts and more tests * disable mixed internal and external data in async inserts * fix tests * fix quota in async inserts * disable async insert for secondary query of distributed	2023-10-20 18:39:48 +02:00
Raúl Marín	d08ce13107	Merge pull request #51692 from Algunenano/parallel_replicas_row_estimation ... Automatic decision of nº of parallel replicas	2023-10-20 15:50:43 +02:00
alesapin	c3c62d90c8	Update MergeTreeData.cpp	2023-10-20 13:48:23 +02:00
alesapin	88f80078e1	Merge branch 'master' into repro_49887	2023-10-20 13:04:00 +02:00
helifu	57d60dc4f2	Add a new column xid for zookeeper_connection (#50702) ... * Refactor the code * Add a new column xid for zookeeper_connection * Support hostnames in the configuration * Fix a typo * Fix a typo * Fix a typo * Fix a bug about connect_time * Update test case * Update test case * Fix a special build check error * Resolve conflicts caused by rebase. * Update failed test case * Refactor the code according to comment * Fix two compilation errors	2023-10-20 11:38:44 +02:00
Igor Nikonov	16c1b64773	Merge remote-tracking branch 'origin/master' into pr-progress-bar	2023-10-20 08:12:28 +00:00
Raúl Marín	4a53943926	Merge remote-tracking branch 'blessed/master' into parallel_replicas_row_estimation	2023-10-19 15:20:50 +00:00
Duc Canh Le	d968dc1a15	StorageJoin: supports trivial count() ... Signed-off-by: Duc Canh Le <duccanh.le@ahrefs.com>	2023-10-19 06:30:25 +00:00
Duc Canh Le	b708fa21f9	rocksdb: trivial count ... Signed-off-by: Duc Canh Le <duccanh.le@ahrefs.com>	2023-10-19 06:28:32 +00:00
Duc Canh Le	d8f73e7240	Merge branch 'master' into rocks_db_serialization ... Fix a mysterious style check fail	2023-10-19 06:26:16 +00:00
Alexey Milovidov	4306b5a991	Revert unnecessary change	2023-10-19 08:13:23 +02:00
Alexey Milovidov	f6db0a5261	Merge branch 'master' into morton-utils	2023-10-19 08:11:25 +02:00
Alexey Milovidov	98111fad3d	Fix error	2023-10-19 08:10:36 +02:00
Kseniia Sumarokova	4e0122a299	Merge pull request #54422 from kssenii/s3-queue-fixes ... Fixes for storage `S3Queue`	2023-10-18 21:51:51 +02:00
kssenii	f90e31eb31	Enable log by default	2023-10-18 15:23:16 +02:00
Igor Nikonov	38895ad2da	Less copying in DefaultCoordinator	2023-10-18 12:59:12 +00:00
kssenii	c4bad255b7	Fix	2023-10-18 14:37:33 +02:00
Igor Nikonov	01fa15c33f	Less copying while handling annoucement	2023-10-18 12:20:14 +00:00
Duc Canh Le	2d638d6470	rocksdb: not creating block during writing ... Signed-off-by: Duc Canh Le <duccanh.le@ahrefs.com>	2023-10-18 09:43:54 +00:00
Amos Bird	602f01f651	Reuse granule during skip index reading	2023-10-18 14:40:34 +08:00
Igor Nikonov	135e4c8412	Merge remote-tracking branch 'origin/master' into pr-progress-bar	2023-10-17 20:42:39 +00:00
Igor Nikonov	fb3d3a37ba	Add progress bar for in order coordinator	2023-10-17 20:09:39 +00:00
Alexey Milovidov	79eccfb642	Merge pull request #55678 from azat/fix-OR-resubmit ... Fix filtering by virtual columns with OR filter in query (resubmit)	2023-10-17 20:33:19 +02:00
avogar	323486f9e8	Add tests	2023-10-17 18:10:47 +00:00
kssenii	eb4519b9d6	Fix	2023-10-17 19:02:04 +02:00
Kseniia Sumarokova	7cedfeff12	Merge pull request #55145 from kssenii/materialized-pg-allow-unique-identifier ... Allow to have several MaterializedPostgreSQL tables following the same Postgres table	2023-10-17 18:47:32 +02:00
Han Fei	4b23142775	Merge branch 'master' into hanfei/statistic	2023-10-17 16:40:31 +02:00
vdimir	c657e500ac	Analyzer: fix block stucture mismatch in matview with engine distributed	2023-10-17 14:02:30 +00:00
vdimir	1d2ac48261	Merge pull request #55681 from ClickHouse/vdimir/fix_analyzer_materialized_distributed_join ... Fix StorageMaterializedView::isRemote	2023-10-17 16:00:15 +02:00
alesapin	abe6df6fbc	Merge pull request #55645 from ClickHouse/restore_top_level_parts ... Restore only top-level parts instead of unexpected	2023-10-17 15:45:21 +02:00
kssenii	c549083e2c	Fix	2023-10-17 14:23:59 +02:00
avogar	032d82e004	Fixes	2023-10-17 12:12:10 +00:00
Alexander Gololobov	11a62bc751	Merge pull request #55621 from ClickHouse/merge-52352 ... Merging #52352	2023-10-17 13:58:53 +02:00
avogar	527e96db3f	Fix	2023-10-17 11:43:51 +00:00
avogar	4892618aa9	Update with master, fix comments	2023-10-17 11:42:19 +00:00
avogar	bb46aa4d57	Clean uo	2023-10-17 11:07:41 +00:00
avogar	f87938735a	Fixes and improvements for Iceberg storage	2023-10-17 11:07:37 +00:00
Duc Canh Le	f20d055e1e	do not re-create serialization for each row ... Signed-off-by: Duc Canh Le <duccanh.le@ahrefs.com>	2023-10-17 10:12:27 +00:00
kssenii	42ed249954	Fix build	2023-10-17 12:03:49 +02:00
Robert Schulze	96a82df75a	Merge pull request #55616 from HarryLeeIBM/hlee-fts-max-rows ... Inverted index: Replace setting density with `max_rows_in_postings_list`	2023-10-17 11:36:46 +02:00
Raúl Marín	3eaf752284	Merge remote-tracking branch 'blessed/master' into parallel_replicas_row_estimation	2023-10-17 11:36:39 +02:00
Kseniia Sumarokova	550a658236	Merge pull request #55144 from kssenii/fix-iceberg-metadata-parsing ... Fix storage Iceberg files retrieval	2023-10-17 11:31:07 +02:00
kssenii	4464c86895	Merge remote-tracking branch 'origin/master' into s3-queue-fixes	2023-10-17 11:16:52 +02:00
kssenii	8744cd9c0d	Fix	2023-10-17 11:15:53 +02:00
Igor Nikonov	4b36761f89	Merge branch 'master' into pr-progress-bar	2023-10-17 10:36:32 +02:00
Alexey Milovidov	de8215f5dd	Merge remote-tracking branch 'origin/master' into morton-utils	2023-10-17 04:55:45 +02:00
HarryLeeIBM	fb8cd298d1	Code refactoring according to code review	2023-10-16 17:52:39 -07:00
Michael Kolupaev	ce7eca0615	DWARF input format (#55450) ... * Add ReadBufferFromFileBase::isRegularLocalFile() * DWARF input format * Review comments * Changed things around ENABLE_EMBEDDED_COMPILER build setting * Added 'ranges' column * no-msan no-ubsan	2023-10-16 17:00:07 -07:00
Ilya Golshtein	786183ee24	merge_row_policy: get rid of RequiredSourceColumnsVisitor	2023-10-16 22:16:53 +00:00
Ilya Golshtein	7f991bf6e7	merge_row_policy: a comment	2023-10-16 21:49:16 +00:00
robot-clickhouse	89eed73fb9	Merge pull request #55519 from azat/zk-retry-fix ... Avoid retrying keeper operations on INSERT after table shutdown	2023-10-16 23:32:13 +02:00
Ilya Golshtein	ea95a49e86	merge_row_policy: some changes per code review	2023-10-16 21:22:43 +00:00
Kseniia Sumarokova	96c518be5b	Merge branch 'master' into s3-queue-fixes	2023-10-16 22:19:13 +02:00
Kseniia Sumarokova	3e9a1cf69f	Merge branch 'master' into fix-iceberg-metadata-parsing	2023-10-16 22:18:52 +02:00
Kseniia Sumarokova	c31299d86c	Merge branch 'master' into materialized-pg-allow-unique-identifier	2023-10-16 22:18:23 +02:00
robot-ch-test-poll1	aca98cf725	Merge pull request #53774 from helifu/master13 ... Display reloaded values of server settings	2023-10-16 21:17:17 +02:00
Kruglov Pavel	cbdb62d389	Merge pull request #55666 from ClickHouse/Avogar-patch-1 ... Fix progress bar for s3 and azure Cluster functions with url without globs	2023-10-16 18:25:00 +02:00
kssenii	0a6a4b3894	Review fixes	2023-10-16 18:15:17 +02:00
Kseniia Sumarokova	8f3974d289	00002_log_and_exception_messages_formatting ...	2023-10-16 17:30:02 +02:00
vdimir	c9a2a4922d	Fix StorageMaterializedView::isRemote	2023-10-16 15:18:47 +00:00
Igor Nikonov	6abd1a6462	Merge branch 'master' into zk-retry-fix	2023-10-16 16:56:07 +02:00
Azat Khuzhin	96f2a46a66	Fix filtering by virtual columns with OR filter in query ... The problem with the initial implementation #52653 was: - OR can have multiple arguments - It simply not correct to assume that if there are two arguments this is OK. Consider the following example: "WHERE (column_not_from_partition_by = 1) OR false OR false" Will be converted to: "WHERE false OR false" And it will simply read nothing. Yes, we could apply some optimization for bool, but this will not always work, since to optimize things like "0 = 1" we need to execute it. And the only way to make handle this correctly (with ability to ignore some commands during filtering) is to make is_constant() function return has it use something from the input block, so that we can be sure, that we have some sensible, and not just "false". Plus we cannot simply ignore the difference of the input and output arguments of handling OR, we need to add always-true (1/true) if the size is different, since otherwise it could break invariants (see comment in the code). This includes (but not limited to): - _part* filtering for MergeTree - _path/_file for various File/HDFS/... engines - _table for Merge - ... P.S. analyzer does not have this bug, since it execute expression as whole, and this is what filterBlockWithQuery() should do actually instead, but this will be a more complex patch. Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com> (cherry picked from commit b107712e0c)	2023-10-16 15:42:10 +02:00
Raúl Marín	f799f5d7a1	Merge remote-tracking branch 'blessed/master' into parallel_replicas_row_estimation	2023-10-16 15:07:11 +02:00
Igor Nikonov	fdd7bfbb83	Merge remote-tracking branch 'origin/master' into pr-progress-bar	2023-10-16 12:59:00 +00:00
alesapin	9a925a0802	Fix typo	2023-10-16 13:35:20 +02:00
Raúl Marín	478f635d60	Merge pull request #53792 from ClickHouse/revert-53782-revert-52762-planner-prepare-filters-for-analysis ... Revert "Revert "Planner prepare filters for analysis""	2023-10-16 13:21:54 +02:00
helifu	dc0f8ed793	Add a new column 'is_hot_reloadable' to indicate whether hot loading is supported	2023-10-16 19:10:35 +08:00
serxa	0e17da22eb	Merge branch 'master' into async-loader-integration	2023-10-16 11:08:29 +00:00
alesapin	31dc46a889	Better test	2023-10-16 13:04:45 +02:00
Kruglov Pavel	6dcc756948	Fix for azure	2023-10-16 12:38:55 +02:00
Kruglov Pavel	836e35b6c4	Fix progress bar for s3 and azure Cluster functions with url without globs	2023-10-16 12:38:10 +02:00
alesapin	dbebdc3dca	Fix comment	2023-10-16 11:05:59 +02:00
Antonio Andelic	6c543c730b	Merge pull request #55657 from ClickHouse/revert-55418-fix-OR ... Revert "Fix filtering by virtual columns with OR filter in query"	2023-10-16 10:51:06 +02:00
alesapin	2d9812b9e9	Fix headers	2023-10-16 10:43:25 +02:00
alesapin	692fcd8422	Merge remote-tracking branch 'origin/master' into repro_49887	2023-10-16 10:39:16 +02:00
alesapin	697c1e3bb0	Merge pull request #55604 from ClickHouse/evillique-partition-parameters ... Implement query paramenters support in partition expression	2023-10-16 10:26:02 +02:00
alesapin	26a8aa441c	Better shutdown handling	2023-10-16 10:24:54 +02:00
Antonio Andelic	fb901c24a1	Revert "Fix filtering by virtual columns with OR filter in query"	2023-10-16 09:45:49 +02:00
Alexey Milovidov	f5aabbecfc	Add a comment	2023-10-16 02:24:34 +02:00
Alexey Milovidov	26c0c0b86d	More tests	2023-10-16 02:19:24 +02:00
Alexey Milovidov	8e8471f5db	Fix error	2023-10-16 00:25:00 +02:00
Alexey Milovidov	724f04e6f6	Allow conditions on part of arguments	2023-10-15 22:21:27 +02:00
Alexey Milovidov	c932a5e0a1	Allow conditions on part of arguments	2023-10-15 22:21:10 +02:00
Alexey Milovidov	0bf6fd3671	Merge remote-tracking branch 'origin/master' into morton-utils	2023-10-15 21:28:00 +02:00
alesapin	bdafe17e9a	Update src/Storages/MergeTree/ActiveDataPartSet.cpp ... Co-authored-by: Alexander Tokmakov <tavplubix@clickhouse.com>	2023-10-15 20:17:55 +02:00
alesapin	13ca296497	Update src/Storages/MergeTree/ActiveDataPartSet.cpp ... Co-authored-by: Alexander Tokmakov <tavplubix@clickhouse.com>	2023-10-15 20:17:50 +02:00
alesapin	6ac790e918	Update src/Storages/MergeTree/ActiveDataPartSet.cpp ... Co-authored-by: Alexander Tokmakov <tavplubix@clickhouse.com>	2023-10-15 20:17:46 +02:00
alesapin	8a865cdfb2	Merge pull request #55646 from ClickHouse/fix_some_typos ... Fix some typos	2023-10-15 18:28:33 +02:00
Alexey Milovidov	0f89b4aad6	Merge pull request #55640 from ClickHouse/preparation-key-condition-2 ... KeyCondition: small changes	2023-10-15 18:23:53 +02:00
Kseniia Sumarokova	2015a0fc17	Merge branch 'master' into fix-iceberg-metadata-parsing	2023-10-15 18:03:53 +02:00
kssenii	f3056b1643	Add a test	2023-10-15 18:02:34 +02:00
alesapin	3b02748cb6	Fix some typos	2023-10-15 15:43:02 +02:00
alesapin	41479b21e6	Restore only top-level parts instead of unexpected	2023-10-15 15:33:33 +02:00
kssenii	0ddee6ad6c	Fix	2023-10-15 14:44:57 +02:00
Kseniia Sumarokova	b26115eabe	Merge branch 'master' into materialized-pg-allow-unique-identifier	2023-10-15 14:17:32 +02:00
Alexander Gololobov	395a54915d	Fix totalBytes()	2023-10-15 10:40:37 +02:00
Alexey Milovidov	42f3751bd5	Minor changes	2023-10-15 06:33:59 +02:00
Alexey Milovidov	17cbc12831	Update KeyCondition.cpp	2023-10-15 07:20:04 +03:00
Alexey Milovidov	22b06e61f2	Add a comment	2023-10-15 06:00:28 +02:00
Alexey Milovidov	569f42d1d9	It might be working	2023-10-15 05:36:53 +02:00
Alexey Milovidov	daaa3db809	Add space-filling curves	2023-10-15 04:37:10 +02:00
Alexey Milovidov	438811ea8c	Add space-filling curves	2023-10-15 04:34:26 +02:00
Alexey Milovidov	1ef7bc41d4	Fix garbage	2023-10-15 00:07:28 +02:00
Alexey Milovidov	7764dedb8e	Something strange	2023-10-14 22:44:44 +02:00
Alexey Milovidov	25efd188d5	KeyCondition: small changes	2023-10-14 22:17:36 +02:00
Alexey Milovidov	50ee412b19	Merge pull request #55625 from ClickHouse/preparation-key-condition ... KeyCondition: preparation	2023-10-14 22:13:39 +02:00
Alexey Milovidov	bafabf1a56	Fix typo in comment	2023-10-14 19:31:46 +02:00
Alexey Milovidov	d6984a2819	Preparation	2023-10-14 04:46:22 +02:00
Alexey Milovidov	a904ee970f	KeyCondition: preparation	2023-10-14 04:32:25 +02:00
Alexey Milovidov	bbd67d262a	Merging #52352	2023-10-14 02:52:53 +02:00
Alexey Milovidov	769ed2e19d	Merge branch 'master' into fix-OR	2023-10-14 02:25:21 +02:00
alesapin	cc92359225	Fxi	2023-10-13 19:46:53 +02:00
HarryLeeIBM	25545d504d	Replace setting density to max_rows_in postings_list for full text search	2023-10-13 10:31:21 -07:00
Kseniia Sumarokova	310e922b40	Merge branch 'master' into materialized-pg-allow-unique-identifier	2023-10-13 19:14:50 +02:00
alesapin	daa8dbbbae	Make as flexible as it can be	2023-10-13 18:51:32 +02:00
alesapin	f80dd06f52	Fix tuple parser	2023-10-13 17:36:38 +02:00
alesapin	78575720d1	Simpler and better	2023-10-13 16:52:04 +02:00
alesapin	47d48d81e0	Implement feature	2023-10-13 16:22:18 +02:00
kssenii	6fde98a33f	Minor improvement	2023-10-13 15:56:05 +02:00
kssenii	44ea61e57f	Improve shutdown	2023-10-13 15:23:36 +02:00
kssenii	b6b124f5a9	Usability improvement	2023-10-13 14:16:56 +02:00
kssenii	a9c0c20cad	Update documentation	2023-10-13 13:21:50 +02:00
kssenii	bfe174f71b	Fix test	2023-10-13 12:57:57 +02:00
kssenii	d64b990712	Merge remote-tracking branch 'origin/master' into s3-queue-fixes	2023-10-13 12:13:56 +02:00
Raúl Marín	500dab9569	Merge remote-tracking branch 'blessed/master' into parallel_replicas_row_estimation	2023-10-13 10:12:57 +02:00
Jihyuk Bok	3c6ddb5e15	Add backward compatibility on the obsoleted storage configuration (#55557) ... * add backward compatibility on the deleted storage configuration * update the reference file	2023-10-13 00:37:55 +02:00
Igor Nikonov	95d5011fca	Fix style	2023-10-12 21:27:24 +00:00
Igor Nikonov	0c8e49f519	Update total rows ... + calculate total rows based on rows in ranges to read + update progress after all ranges to read were announced	2023-10-12 20:34:26 +00:00
Nikita Mikhaylov	81bbb0fce1	Improve merge selecting algorithm for big partitions (#55526) ... Improved overall resilience for ClickHouse in case of many parts within partition (more than 1000). It might reduce the number of TOO_MANY_PARTS errors.	2023-10-12 17:00:25 +02:00
Nikita Mikhaylov	a2ee30707d	Better error message in case when merge selecting task failed (#55554)	2023-10-12 16:59:03 +02:00
kssenii	1917a882b7	Better	2023-10-12 16:36:19 +02:00
Han Fei	42dc61abf5	Merge branch 'master' into hanfei/statistic	2023-10-12 16:19:45 +02:00
Robert Schulze	332a0cfa24	Merge pull request #55529 from ucasfl/invert ... Forbid create inverted index if setting not enabled	2023-10-12 14:40:58 +02:00
vdimir	5bc6642b57	Merge pull request #55480 from myrrc/fix/s3-remove-stale-parts ... Remove existing moving/ dir if allow_remove_stale_moving_parts is off	2023-10-12 12:00:30 +02:00
flynn	268ea30c41	Forbid create inverted index if setting not enabled	2023-10-12 03:50:55 +00:00
alesapin	6a9e19c29f	Fxi distributed race condition	2023-10-11 19:46:32 +02:00
Azat Khuzhin	47ff13056f	Avoid retrying keeper operations on INSERT after table shutdown ... Also note, that it should check shutdown_prepared_called, instead of shutdown_called, since shutdown of the database/server firstly calls flushAndPrepareForShutdown() for every storage. And if you have Buffer table that writes to your ReplicatedMergeTree it will not be finished until all retries will be made. Otherwise it can take significantly long time to finish. Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>	2023-10-11 18:25:43 +02:00
alesapin	9235408435	Merge branch 'partition-parameters' of github.com:evillique/ClickHouse into evillique-partition-parameters	2023-10-11 18:06:12 +02:00
alesapin	e2cde338fb	better headers	2023-10-11 17:45:32 +02:00
alesapin	e364ed61a6	Proper fix	2023-10-11 17:42:44 +02:00
Robert Schulze	9d04d3c3ad	Merge remote-tracking branch 'rschu1ze/master' into better-use-mysql-types-in-show-columns	2023-10-11 15:04:03 +00:00
Han Fei	253b8efa58	do not modify column type when there is statistic	2023-10-11 16:57:40 +02:00
Raúl Marín	0b9bd809e7	Merge remote-tracking branch 'blessed/master' into parallel_replicas_row_estimation	2023-10-11 16:50:22 +02:00
vdimir	d3f9145351	Update src/Storages/MergeTree/MergeTreePartsMover.cpp	2023-10-11 13:17:54 +02:00
vdimir	b634711303	Update src/Storages/MergeTree/MergeTreePartsMover.cpp	2023-10-11 13:14:01 +02:00
Robert Schulze	bd43b84bf8	Make use_mysql_types_in_show_columns affect only SHOW COLUMNS	2023-10-10 23:09:49 +00:00
Mike Kot	06f3b82cc9	remove moving/ dir if allow_remove_stale_ is off	2023-10-10 19:56:22 +00:00
Azat Khuzhin	b107712e0c	Fix filtering by virtual columns with OR filter in query ... The problem with the initial implementation #52653 was: - OR can have multiple arguments - It simply not correct to assume that if there are two arguments this is OK. Consider the following example: "WHERE (column_not_from_partition_by = 1) OR false OR false" Will be converted to: "WHERE false OR false" And it will simply read nothing. Yes, we could apply some optimization for bool, but this will not always work, since to optimize things like "0 = 1" we need to execute it. And the only way to make handle this correctly (with ability to ignore some commands during filtering) is to make is_constant() function return has it use something from the input block, so that we can be sure, that we have some sensible, and not just "false". Plus we cannot simply ignore the difference of the input and output arguments of handling OR, we need to add always-true (1/true) if the size is different, since otherwise it could break invariants (see comment in the code). This includes (but not limited to): - _part* filtering for MergeTree - _path/_file for various File/HDFS/... engines - _table for Merge - ... P.S. analyzer does not have this bug, since it execute expression as whole, and this is what filterBlockWithQuery() should do actually instead, but this will be a more complex patch. Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>	2023-10-10 20:47:52 +02:00
alesapin	ab09aee754	Initial implementation	2023-10-10 19:59:48 +02:00
avogar	9ca92c1b91	Rename files	2023-10-10 16:25:51 +00:00
avogar	1afd655bbc	Allow to save unparsed records and errors in RabbitMQ, NATS and FileLog engines	2023-10-10 16:21:33 +00:00
Raúl Marín	95d2063e91	Merge remote-tracking branch 'blessed/master' into parallel_replicas_row_estimation	2023-10-10 17:29:45 +02:00
Nikita Mikhaylov	4456fe40f9	Remove the old code for projection analysis (#55112)	2023-10-10 17:13:32 +02:00
alesapin	666c690b4f	Merge pull request #55309 from ClickHouse/fix_redundant_check ... Fix bug with inability to drop detached partition in replicated merge tree on top of S3 without zero copy	2023-10-10 11:48:00 +02:00
helifu	ca0c4ba147	Merge branch 'ClickHouse:master' into master13	2023-10-10 13:49:49 +08:00
robot-ch-test-poll4	8609cd25fc	Merge pull request #55392 from azat/dist-insert-threads-fix ... Fix missing thread accounting for insert_distributed_sync=1	2023-10-10 03:18:16 +02:00
Alexey Milovidov	25d4e41b5e	Merge pull request #55377 from k-morozov/feature/add_internal_replication_to_system_clusters ... add internal_replication to system.clusters	2023-10-10 01:18:01 +02:00
Alexey Milovidov	b50889a8ff	Merge pull request #55186 from ClickHouse/remove-in-memory-data-parts ... Remove the support for in memory data parts (part 2)	2023-10-09 23:53:25 +02:00
Anton Popov	26a938c8cf	Merge pull request #55202 from canhld94/clickhouse_fix_too_many_marks ... Fix recalculation of skip indexes in ALTER UPDATE queries when table has adaptive granularity	2023-10-09 19:24:59 +02:00
Azat Khuzhin	5d2a97106c	Fix missing thread accounting for insert_distributed_sync=1 ... Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>	2023-10-09 15:41:52 +02:00
pufit	a5bcf176be	Merge pull request #55108 from CurtizJ/add-field-to-system-parts ... Add fields `substreams` and `filenames` to `system.parts_columns`	2023-10-09 15:11:56 +02:00
Igor Nikonov	0b4d76780f	Merge branch 'master' into revert-53782-revert-52762-planner-prepare-filters-for-analysis	2023-10-09 14:44:00 +02:00
Han Fei	fcb19b4f77	Merge branch 'master' into hanfei/statistic	2023-10-09 14:24:58 +02:00
Alexey Milovidov	e633f36ce9	Merge pull request #47663 from cangyin/fix-invalid-segment-id ... skip hardlinking inverted index files in mutation	2023-10-09 13:31:46 +02:00
Alexey Milovidov	a02dedbb6a	Merge pull request #54989 from myrrc/patch-6 ... Remove useless path from lockSharedData in StorageReplicatedMergeTree	2023-10-09 13:28:06 +02:00
Alexey Milovidov	e840d3ce58	Merge pull request #55370 from azat/hdfs-leak ... Fix leak in StorageHDFS	2023-10-09 13:16:45 +02:00
Konstantin Morozov	54edff684b	add internal_replication	2023-10-09 10:32:22 +00:00
Azat Khuzhin	2ecd9ddabd	Fix leak in StorageHDFS ... LSan report: Direct leak of 400 byte(s) in 5 object(s) allocated from: 0 0x564c44d5308d in operator new[](unsigned long) (/usr/bin/clickhouse+0xbdf908d) (BuildId: 9bd2befe9c5f1d960238405d4a3fc9860273b81a) 1 0x564c7b1b83f2 in hdfsGetPathInfo build_docker/./contrib/libhdfs3/src/client/Hdfs.cpp:1096:18 2 0x564c670a8fed in DB::HDFSSource::URISIterator::Impl::next() build_docker/./src/Storages/HDFS/StorageHDFS.cpp:471:25 3 0x564c670a54e9 in DB::HDFSSource::URISIterator::next() build_docker/./src/Storages/HDFS/StorageHDFS.cpp:510:19 CI: https://s3.amazonaws.com/clickhouse-test-reports/55261/de503f75dcbc5a4d0e7fbb2e6b08c2106d62848a/integration_tests__asan__[3_6].html Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>	2023-10-09 10:46:00 +02:00
Alexey Milovidov	808b78984f	Merge pull request #55356 from ClickHouse/better-exception-messages ... Better exception messages	2023-10-09 02:42:26 +02:00
Alexey Milovidov	885e07cd0b	Update MutateTask.cpp	2023-10-09 02:33:46 +03:00
Alexey Milovidov	e3771f10d2	Merge branch 'master' into remove-in-memory-data-parts	2023-10-09 01:08:20 +02:00
Alexey Milovidov	e1dc80b5ab	Revert "Support SAMPLE BY for VIEW"	2023-10-09 00:08:46 +03:00
Alexey Milovidov	2aaddcce18	Better exception messages	2023-10-08 22:54:33 +02:00
Robert Schulze	282200ef50	Merge pull request #55215 from rschu1ze/more-mysql-compat ... Add `information_schema.columns.extra` and `information_schema.tables.table_rows`	2023-10-08 12:21:34 +02:00
Kseniia Sumarokova	0af252ae31	Merge pull request #55297 from ClickHouse/kssenii-patch-6 ... MaterializedPostgreSQL: remove back check	2023-10-08 10:31:23 +02:00
Robert Schulze	f4bc58ea92	Merge remote-tracking branch 'rschu1ze/master' into more-mysql-compat	2023-10-08 07:49:37 +00:00
Robert Schulze	0450da9bac	Naming fixes	2023-10-07 20:12:10 +00:00
alesapin	eec64ba1d9	Fix bug with inability to drop detached partition in replicated merge tree on top of S3 without zero copy	2023-10-07 19:09:47 +02:00
Robert Schulze	2e8cac6e9c	Make use_mysql_types_in_show_columns independent from the connection ... Fixes: #55273	2023-10-07 13:06:41 +00:00
Kseniia Sumarokova	f0747b7a6e	Update MaterializedPostgreSQLConsumer.cpp	2023-10-07 14:28:52 +02:00
Duc Canh Le	bea4a4eb85	Merge branch 'master' into clickhouse_fix_too_many_marks ... rebase and re-trigger CI	2023-10-06 09:45:48 +00:00
Nikolai Kochetov	a9a3529025	Merge branch 'master' into planner-prepare-filters-for-analysis-2	2023-10-05 12:15:29 +00:00
Ilya Golshtein	3480243fa3	merge_row_policy: per code review	2023-10-05 08:23:53 +00:00
Ilya Golshtein	9a561cff82	merge_row_policy: cleanup, verbose short messages in 00002_log ...	2023-10-05 08:23:53 +00:00
Ilya Golshtein	57f1e56276	merge_row_policy: cleanup, comments, new tests	2023-10-05 08:23:52 +00:00
Ilya Golshtein	6161116bf2	merge_row_policy: stylecheck again	2023-10-05 08:23:52 +00:00
Ilya Golshtein	f89a538f31	merge_row_policy: stylecheck	2023-10-05 08:23:52 +00:00
Ilya Golshtein	b39a201cd0	merge_row_policy: refactored with ReadFromMerge::RowPolicyData	2023-10-05 08:23:52 +00:00
Ilya Golshtein	b57d8bc4a9	merge_row_policy: works again with adding missed columns	2023-10-05 08:23:52 +00:00
Ilya Golshtein	f4c77c1f18	merge_row_policy: namesDifference, try to handle nonselected columns	2023-10-05 08:23:52 +00:00
Ilya Golshtein	e2ddf40cfd	merge_row_policy: further cleanup	2023-10-05 08:23:52 +00:00
Ilya Golshtein	da5f607242	merge_row_policy: cleanup, remove some debug output	2023-10-05 08:23:52 +00:00
Ilya Golshtein	978a535849	merge_row_policy: addFilter(() together with FilterTransform	2023-10-05 08:23:52 +00:00
Ilya Golshtein	a19cd8089b	merge_row_policy: add_filter() actually works	2023-10-05 08:23:52 +00:00
Ilya Golshtein	c457fa727e	merge_row_policy: addFilter() instead of FilterTransform	2023-10-05 08:23:52 +00:00
Ilya Golshtein	359cd4d32a	merge_row_policy: style fixes and back to storage->read	2023-10-05 08:23:52 +00:00
Ilya Golshtein	657c39c79e	merge_row_policy: row_policy_storage_merge test	2023-10-05 08:23:52 +00:00
Ilya Golshtein	b6f682dc3f	merge_row_policy: FilterTransform	2023-10-05 08:23:52 +00:00
Ilya Golshtein	eebdff472e	merge_row_policy: original behavior restored + extra debug	2023-10-05 08:23:52 +00:00
Ilya Golshtein	2acc4c223d	merge_row_policy - extra debug	2023-10-05 08:23:52 +00:00
Ilya Golshtein	0464b4fd06	merge_row_policy: with QueryProcessingStage::Complete	2023-10-05 08:23:52 +00:00
Ilya Golshtein	1e31859936	merge_row_policy: tiny cleanup	2023-10-05 08:23:52 +00:00
Ilya Golshtein	020d76a383	merge_row_policy: initial	2023-10-05 08:23:52 +00:00
Robert Schulze	53683f56c0	Merge remote-tracking branch 'rschu1ze/master' into more-mysql-compat	2023-10-04 12:45:10 +00:00
Nikolai Kochetov	c5100cd9b1	Fixing build	2023-10-04 09:01:29 +00:00
Robert Schulze	16a864171d	Merge pull request #55210 from HarryLeeIBM/hlee-fts-query ... Speed up buildup of inverted index using absl::flat_hash_map	2023-10-04 10:01:35 +02:00
Han Fei	ba878fb43a	support rename	2023-10-04 00:58:26 +02:00
Nikolai Kochetov	d944b59902	Merge branch 'master' into planner-prepare-filters-for-analysis-2	2023-10-03 14:28:16 +00:00
Nikolai Kochetov	3b757d60a2	Remove commented code.	2023-10-03 14:22:20 +00:00
Robert Schulze	907e792a93	Add information_schema.columns.extras ... Fixes: #55183	2023-10-03 13:11:03 +00:00
Robert Schulze	2aefa760c3	Add information_schema.tables.table_rows ... Fixes: #55182	2023-10-03 12:24:10 +00:00
robot-ch-test-poll	e06f72ee55	Merge pull request #54052 from VanDarkholme7/missing_subcolumns ... Support missed Object(Nullable(json)) subcolumns in query.	2023-10-03 10:32:51 +02:00
HarryLeeIBM	9e1208a793	Speed up inverted index building using flat_hash_map	2023-10-02 15:20:35 -07:00
Duc Canh Le	57e902d375	more detail logging for exception while loading marks ... Signed-off-by: Duc Canh Le <duccanh.le@ahrefs.com>	2023-10-02 09:29:56 +00:00
Alexey Milovidov	5ebd8ef41d	Remove Meilisearch	2023-09-30 06:02:38 +02:00
Alexey Milovidov	6e87fa3f67	Merge branch 'master' of github.com:ClickHouse/ClickHouse into remove-in-memory-data-parts	2023-09-30 05:12:29 +02:00
Alexey Milovidov	de8e068da7	Merge pull request #55184 from ClickHouse/remove-garbage-broken-detached-parts ... Remove the removal of broken detached parts	2023-09-30 06:10:40 +03:00
Alexey Milovidov	db9377fc59	Merge pull request #55172 from amosbird/fix-55106 ... Fix partition pruning of extra columns in set.	2023-09-30 04:17:14 +03:00
Alexey Milovidov	6de0250650	Continue removing in-memory data parts	2023-09-30 03:10:59 +02:00
Alexey Milovidov	d8b5713023	Remove the removal of broken detached parts	2023-09-30 01:47:44 +02:00
robot-ch-test-poll4	4ef46d1153	Merge pull request #55130 from ClickHouse/improve-message-predicate ... Improve error message in MergePredicate	2023-09-29 23:29:41 +02:00
robot-ch-test-poll3	f7874c1855	Merge pull request #55143 from hanfei1991/hanfei/fix-empty-retries ... check if block is empty after async insert retries	2023-09-29 23:29:21 +02:00
Amos Bird	5595d2d20b	Fix partition pruning of extra columns in set.	2023-09-30 02:32:31 +08:00
Robert Schulze	fabd167af6	Merge pull request #55128 from rschu1ze/disable-error-update ... Suppress error statistics update for internal exceptions	2023-09-29 18:37:12 +02:00
Han Fei	8afc21f7af	refine	2023-09-29 18:20:16 +02:00
kssenii	d1f6888119	Allow unique consumer identifier	2023-09-29 16:17:02 +02:00
kssenii	1ef21bab3d	Fix data race	2023-09-29 16:02:51 +02:00
Nikita Mikhaylov	043000b088	Better	2023-09-29 15:59:54 +02:00
kssenii	a7f389a760	Fix	2023-09-29 15:14:25 +02:00
Han Fei	e57217ea9c	check if block is empty after async insert retries	2023-09-29 15:14:16 +02:00
Nikita Mikhaylov	804ae4c58b	Better	2023-09-29 11:16:24 +00:00
Robert Schulze	c1b95e8b3c	Suppress error statistics update for internal exceptions	2023-09-29 10:58:44 +00:00
Robert Schulze	bb4d141132	Merge pull request #55069 from HarryLeeIBM/hlee-ftsearch ... Add optimization for roaring bitmaps in full text search	2023-09-29 11:16:16 +02:00
Alexey Milovidov	95d573f70d	Update autogenerated version to 23.10.1.1 and contributors	2023-09-29 08:52:43 +02:00
Anton Popov	5789db24a7	add fields 'substreams' and 'filenames' to system.parts_columns	2023-09-28 16:59:01 +00:00
Han Fei	d6c1c0e805	refine tests and support drop columns with statistic	2023-09-28 18:35:18 +02:00
kssenii	d644992192	Fxi	2023-09-28 16:25:04 +02:00
kssenii	4a7922507b	Minor changes	2023-09-28 16:18:00 +02:00
kssenii	7d91ba109a	Update doc	2023-09-28 13:58:59 +02:00
kssenii	1749874e7b	Fxi	2023-09-28 13:51:07 +02:00
Igor Nikonov	1983b4cfd1	Merge pull request #55026 from ClickHouse/insert-quorum-wo-keeper-retries ... Fix: insert quorum w/o keeper retries	2023-09-28 11:05:21 +02:00
Amos Bird	51f603bdfc	Rename as suggested.	2023-09-28 15:11:56 +08:00
Amos Bird	056b2693a3	Reorganize combinators into dedicated folder	2023-09-28 15:11:55 +08:00
Han Fei	18f6fd1e5f	more comments	2023-09-28 07:31:48 +02:00
Alexey Milovidov	aacb3471b9	Update StorageReplicatedMergeTree.cpp	2023-09-28 02:41:50 +03:00
kssenii	57cfb88cb6	Fix typos check, fix build wihtout s3	2023-09-27 22:22:49 +02:00
HarryLeeIBM	71adb052ee	Add optimization for roaring bitmaps in full text search	2023-09-27 12:23:00 -07:00
János Benjamin Antal	c911c8daf4	Prevent attaching parts from tables with different projections or indices (#55062) ... * Prevent attaching parts from tables with different projections or indices * Update docs * Add tests	2023-09-27 15:02:53 -04:00
kssenii	c579f5b321	Fix style check	2023-09-27 20:10:05 +02:00
kssenii	42783894b7	Add comments	2023-09-27 20:07:19 +02:00
kssenii	e0ff76a7f9	Fix	2023-09-27 18:44:53 +02:00
Han Fei	6ce77b07f7	refine exception messages	2023-09-27 17:59:53 +02:00
kssenii	f753b91a3b	Better maintenance of processing node	2023-09-27 17:17:52 +02:00
Nikita Taranov	86f4f91573	Merge branch 'master' into fix_key_analysis_merge_engine	2023-09-27 16:13:14 +02:00
kssenii	6b191a1afe	Better	2023-09-27 14:54:31 +02:00
Robert Schulze	2d750e9e2f	Merge remote-tracking branch 'rschu1ze/master' into clang-tidy-reenable-checks	2023-09-27 12:15:29 +00:00
Robert Schulze	e9c3032f83	Merge pull request #55037 from rschu1ze/inf_sch_tables ... Implement information_schema.tables.data_length	2023-09-27 14:09:38 +02:00
kssenii	d77452c561	Fix style check	2023-09-27 12:06:41 +02:00
kssenii	6c778d4b12	Fix build without s3	2023-09-27 12:05:13 +02:00
Igor Nikonov	98bf455726	Merge remote-tracking branch 'origin/master' into insert-quorum-wo-keeper-retries	2023-09-27 09:02:39 +00:00
Han Fei	76b9d43dc4	make code beautiful	2023-09-27 11:01:53 +02:00
Robert Schulze	3e0bc6fe42	Disable '-clang-analyzer-unix.Malloc' for now	2023-09-27 08:51:17 +00:00
Robert Schulze	3a1663a460	Merge pull request #55035 from rschu1ze/is_deterministic_resubmit ... Added field "is_deterministic" to system.functions	2023-09-27 10:13:18 +02:00
Michael Kolupaev	23bc286280	Print more information about one logical error in MergeTreeDataWriter (#55001)	2023-09-26 13:23:26 -07:00
Han Fei	6c4474d3d1	refine code structure	2023-09-26 22:22:11 +02:00
Robert Schulze	e22de584e9	information_schema.tables.data_length	2023-09-26 20:08:54 +00:00
Han Fei	4e7745f757	Merge branch 'master' into hanfei/statistic	2023-09-26 21:43:53 +02:00
Robert Schulze	dcdc8bf0e9	Added field "is_deterministic" to system.functions	2023-09-26 19:13:23 +00:00
Robert Schulze	cde10fe7b5	Merge remote-tracking branch 'rschu1ze/master' into clang-tidy-reenable-checks	2023-09-26 18:59:41 +00:00
Robert Schulze	ab2b5cd5ca	More fixes	2023-09-26 18:55:10 +00:00
Robert Schulze	6634043a9f	Merge pull request #54837 from rschu1ze/obsolete-mt-settings ... Mark obsolete settings in system.merge_tree_settings as is_obsolete = 1	2023-09-26 19:55:05 +02:00
Han Fei	8cb9fe0154	refine code	2023-09-26 19:16:01 +02:00
George Gamezardashvili	0ce30ab6d5	SSH keys authentication (#41109) ... Added new type of authentication based on SSH keys. It works only for Native TCP protocol. Co-authored-by: Nikita Mikhaylov <nikitamikhaylov@clickhouse.com> Co-authored-by: Robert Schulze <robert@clickhouse.com>	2023-09-26 17:50:19 +02:00
Igor Nikonov	54f25409ef	Test with fail point	2023-09-26 15:19:04 +00:00
kssenii	14b09d3cdc	Add caching	2023-09-26 16:23:24 +02:00
Alexander Tokmakov	f39cb17881	Revert "Added field "is_deterministic" to system.functions"	2023-09-26 15:01:05 +02:00
Igor Nikonov	fd7862cd10	Remove keeper retries when waiting for insert quorum	2023-09-26 12:13:32 +00:00
Anton Popov	6cd32eb7b5	Merge pull request #50612 from CurtizJ/long-column-names ... Allow to replace long file names to hashes	2023-09-26 13:05:33 +02:00
Robert Schulze	e4c081652d	Suppress more checks	2023-09-26 11:01:35 +00:00
Robert Schulze	d10a61444c	Merge remote-tracking branch 'rschu1ze/master' into obsolete-mt-settings	2023-09-26 09:55:47 +00:00
Robert Schulze	e954ffe939	Cleanups	2023-09-26 09:51:48 +00:00
Nikita Taranov	595027b1c6	better	2023-09-26 11:51:34 +02:00
Robert Schulze	9fff447716	Re-enable clang-tidy checks	2023-09-26 09:34:12 +00:00
Robert Schulze	fa40732395	Merge pull request #54766 from rschu1ze/is_deterministic ... Added field "is_deterministic" to system.functions	2023-09-26 11:05:03 +02:00
János Benjamin Antal	036aa9f756	Merge pull request #54869 from usurai/ceil ... Update ceilRowsToCompleteGranules to use binary search.	2023-09-26 09:46:20 +02:00
VanDarkholme7	9d74502488	Merge branch 'ClickHouse:master' into missing_subcolumns	2023-09-26 13:48:16 +08:00
VanDarkholme7	7dda1646fd	fix	2023-09-26 02:32:05 +00:00
Nikolay Degterinsky	987906680f	Revert "Fix NATS high cpu usage"	2023-09-26 03:36:14 +02:00
Nikita Taranov	694792fa5d	fix for analyzer	2023-09-25 23:58:29 +02:00
Nikita Taranov	8d19566c7d	impl	2023-09-25 23:58:20 +02:00
kssenii	4d78dbacfe	Fix style check, fix build without s3	2023-09-25 23:38:16 +02:00
Kseniia Sumarokova	5057a31bac	Merge pull request #54844 from kssenii/materialized-postrgesql-safer-parts ... Fix "Invalid number of rows in Chunk" in MaterializedPostgreSQL	2023-09-25 23:37:12 +02:00
kssenii	251b0165ca	Better	2023-09-25 22:55:41 +02:00
kssenii	3b54b6be88	Add system tables	2023-09-25 20:04:15 +02:00
Robert Schulze	6f3d7fcc86	Better lint	2023-09-25 17:11:32 +00:00
Robert Schulze	0c490465d4	Stabilize tests	2023-09-25 17:07:48 +00:00
Robert Schulze	d170304359	Parallelize Annoy index creation + minor test fix	2023-09-25 17:07:23 +00:00
Robert Schulze	5233adecb0	Merge remote-tracking branch 'rschu1ze/master' into is_deterministic	2023-09-25 16:27:50 +00:00
Kruglov Pavel	af8b1e4978	Merge pull request #54855 from Avogar/obsolete-format-settings ... Move obsolete format settings to separate section	2023-09-25 18:00:31 +02:00
Mike Kot	e2dd78fdd9	Remove useless path from lockSharedData in StorageReplicatedMergeTree	2023-09-25 16:56:09 +02:00
Anton Popov	0ebc22e851	Merge branch 'master' into long-column-names	2023-09-25 16:55:14 +02:00
usurai	330bcff239	Update ceilRowsToCompleteGranules to use binary search. ... Replace the linear method in ceilRowsToCompleteGranules with a binary search.	2023-09-25 22:52:00 +08:00
kssenii	e4256eb268	Better	2023-09-25 13:44:45 +02:00
kssenii	47050c2605	Merge remote-tracking branch 'upstream/master' into s3-queue-fixes	2023-09-25 13:42:52 +02:00
Azat Khuzhin	7a761730fa	Respect max_block_size while reading form system.stack_trace ... Reading from system.stack_trace can be very slow, especially if you read all columns (there are some optimizations to avoid doing some actions), that requires: - reading /proc/self/task - reading /proc/self/task/*/comm - sending signal to each of the thread (to read query_id and trace_id) And even if you have some conditions (for which there are some optimizations, i.e. not sending signal to the thread which is the slowest operation under load) it will still send signals to every thread and wait for result. This patch implements reading from system.stack_trace with respect to max_block_size, that will make queries with lower max_block_size and LIMIT works significantly faster. Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>	2023-09-24 14:01:24 +02:00
Azat Khuzhin	01d22805ef	Remove indent for anonymous namespace ... Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>	2023-09-24 14:01:24 +02:00
Azat Khuzhin	e0bca06a5d	Add some logging for preparing rows for system.stack_trace ... Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>	2023-09-24 14:00:43 +02:00
robot-clickhouse	320e4c47f3	Merge pull request #54943 from amosbird/fix-54941 ... Rebuild minmax_count_projection when partition key gets modified	2023-09-23 22:11:52 +02:00
robot-clickhouse-ci-2	2a26557391	Merge pull request #54399 from vahpetr/fix-nats-high-cpu-usage ... Fix NATS high cpu usage	2023-09-23 21:36:21 +02:00
Sergei Trifonov	c1489d52b7	Merge pull request #54618 from ClickHouse/io-scheduling-throttler ... add IO scheduling node to limit bandwidth	2023-09-23 21:26:17 +02:00
Alexey Milovidov	e89a8e4d13	Merge pull request #53492 from ClickHouse/fix-prewhere-lost-column ... Fix: moved to prewhere condition actions can lose column	2023-09-23 22:24:00 +03:00
serxa	44acb53ad3	Merge branch 'master' into io-scheduling-throttler	2023-09-23 10:51:18 +00:00
Sergei Trifonov	4032fc17bb	Merge pull request #54780 from ucasfl/comment ... update comment	2023-09-23 12:48:48 +02:00
Amos Bird	81804ce9ba	Rebuild minmax_count_projection when partition key gets modified	2023-09-23 17:59:06 +08:00
robot-ch-test-poll2	3c93a939a2	Merge pull request #54936 from ClickHouse/pufit/s3-yet-another-num-streams-adjustment ... Set a minimum limit of `num_streams` in StorageS3	2023-09-23 02:59:50 +02:00
robot-clickhouse-ci-2	d98234dc9d	Merge pull request #54803 from Avogar/ephemeral-columns-from-files ... Forbid special columns for file/s3/url/... storages, fix insert into ephemeral columns from files	2023-09-22 23:24:42 +02:00
Alexey Milovidov	783d1873b2	Merge pull request #54839 from rschu1ze/vector-search-test-refactoring ... Refactor and split up vector search tests	2023-09-23 00:10:47 +03:00
robot-ch-test-poll3	1334202af4	Merge pull request #54863 from zvonand/zvonand-refactor-globs ... Refactor and simplify multi-directory globs	2023-09-22 23:09:05 +02:00
pufit	13eee0e950	Set a minimum limit of num_streams in StorageS3	2023-09-22 14:13:20 -04:00
Nikolai Kochetov	cf40723179	Merge pull request #54895 from amosbird/allow-nullable-primary-keys-for-projections ... Always allow nullable keys for projections	2023-09-22 19:09:24 +02:00
Alexander Tokmakov	21ffbd88de	Merge pull request #54859 from ClickHouse/fix_zero_copy_locks ... Fix zero copy locks with hardlinks	2023-09-22 18:27:03 +02:00
Kruglov Pavel	4d675dbad4	Merge pull request #54825 from azat/fix-virtual-columns-filtering ... Fix filtering parts with indexHint for non analyzer (resubmit)	2023-09-22 18:20:16 +02:00
Dmitry Novik	0f8afdbbe7	Merge remote-tracking branch 'origin/master' into fix-prewhere-lost-column	2023-09-22 14:50:05 +00:00
Alexander Tokmakov	f54c3c6a9f	Update StorageReplicatedMergeTree.cpp	2023-09-22 15:07:55 +02:00
Robert Schulze	e9c0dbd33e	Merge remote-tracking branch 'rschu1ze/master' into tableau-online-information-schema	2023-09-22 11:03:06 +00:00
Robert Schulze	9a91f97ac4	Merge pull request #53831 from rschu1ze/clang-17 ... Switch CI to Clang 17	2023-09-22 12:28:35 +02:00
Kseniia Sumarokova	163961115b	Merge pull request #54842 from kssenii/storage-rabbitmq-minor-change ... Add some logging to StorageRabbitMQ	2023-09-22 11:51:27 +02:00
zvonand	2eb63649bd	fix old workflows	2023-09-22 00:30:46 +02:00
Robert Schulze	877e4f3aab	Merge remote-tracking branch 'rschu1ze/master' into clang-17	2023-09-21 20:21:12 +00:00
pufit	99c1d76604	Fix division by zero in StorageS3	2023-09-21 15:55:42 -04:00
Robert Schulze	a46e0deb85	Resolve bad caching of INCBIN'd external data	2023-09-21 17:41:21 +00:00
Michael Kolupaev	be0c512329	Fix virtual columns having incorrect values after ORDER BY (#54811) ... Fix virtual columns having incorrect values after ORDER BY	2023-09-21 10:14:28 -07:00
Michael Kolupaev	9af9b4a085	Enable connection pooling for s3 table function (#54812) ... Enable connection pooling for s3 table function	2023-09-21 09:27:20 -07:00
Amos Bird	973cd5e972	Always allow nullable keys for projections	2023-09-21 22:46:16 +08:00
Robert Schulze	5209bd2d51	Merge remote-tracking branch 'rschu1ze/master' into clang-17	2023-09-21 14:45:55 +00:00
zvonand	e2b40749a6	the same for HDFS	2023-09-21 16:45:47 +02:00
Robert Schulze	f5137dd0b4	More clang-tidy fixes	2023-09-21 14:40:57 +00:00
zvonand	bc5f141a6c	fix {..} for File	2023-09-21 16:40:38 +02:00
Anton Popov	4d7ecf39ac	check file collisions on all alters	2023-09-21 14:16:03 +00:00
slvrtrn	2b8a263355	Change referential_constraints to an empty table	2023-09-21 16:11:08 +02:00
pufit	bd387f6d2c	Merge pull request #54815 from ClickHouse/pufit/optimal-num-streams-for-s3-storage ... Adjusting `num_streams` by expected work in StorageS3	2023-09-21 08:41:39 -04:00
slvrtrn	e49ddbb678	Merge remote-tracking branch 'origin' into tableau-online-information-schema	2023-09-21 14:25:04 +02:00
Igor Nikonov	d260b7abe4	Merge pull request #54519 from ClickHouse/parallel-replicas-not-enough-replicas ... Disable parallel replicas on shards with not enough nodes	2023-09-21 14:07:25 +02:00
Robert Schulze	44b7fa1030	Merge remote-tracking branch 'rschu1ze/master' into is_deterministic	2023-09-21 09:29:44 +00:00
Robert Schulze	c2a593baf9	Split tests	2023-09-21 09:17:38 +00:00
Vitaly Baranov	bf4662aca7	Merge pull request #53336 from azat/backups/s3-throttling ... Fix throttling of BACKUPs from/to S3 (in case native copy was not used) and in some other places as well	2023-09-21 10:42:28 +02:00
zvonand	a05bb020d4	do all the same for HDFS + remove setting	2023-09-21 01:50:41 +02:00
pufit	4a2f7976f0	Resolve PR issues	2023-09-20 19:43:02 -04:00
zvonand	5f67788c03	update docs and comments	2023-09-21 00:16:17 +02:00
zvonand	9eccb12183	small refactor of storageFile globs	2023-09-20 23:59:10 +02:00
Robert Schulze	e95f9ee814	Merge remote-tracking branch 'rschu1ze/master' into is_deterministic	2023-09-20 21:17:57 +00:00
Alexander Tokmakov	f8062d77e1	fix rare data loss	2023-09-20 22:38:25 +02:00
Alexander Tokmakov	ec80068611	add a test for no such key	2023-09-20 22:37:24 +02:00
Alexander Tokmakov	18acc0c19d	Merge pull request #54641 from ClickHouse/followup_54550 ... Follow-up to #54550	2023-09-20 19:44:56 +02:00
pufit	20105958a8	add reserve	2023-09-20 13:37:06 -04:00
pufit	71c7e3c81e	Add logging, fix thread name length	2023-09-20 13:33:25 -04:00
Azat Khuzhin	c706101891	Fix throttling of BACKUPs from/to S3 (in case native copy was not used) ... In some cases native copy is not possible, and such requests should be throttled. v0: copyS3FileNativeWithFallback v2: revert v0 and pass write_settings v3: pass read_settings to copyFile() Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>	2023-09-20 18:28:43 +02:00
avogar	3e08800cb5	Forbid special columns for file/s3/url/... storages, fix insert into ephemeral columns from files	2023-09-20 16:25:55 +00:00
avogar	54cee196bd	Move obsolete format settings to separate section	2023-09-20 16:00:28 +00:00
Robert Schulze	2c91e52da1	Merge pull request #54600 from rschu1ze/fix-annoy-crash ... Annoy/Usearch index: Fix LOGICAL_ERROR during build-up with default values	2023-09-20 17:05:16 +02:00
pufit	729c8aa29f	fix glob iterator estimated objects	2023-09-20 10:41:47 -04:00
kssenii	5d66189d87	Fix	2023-09-20 16:39:24 +02:00
kssenii	3ef2c37b92	Add some logging to StorageRabbitMQ	2023-09-20 15:54:11 +02:00
pufit	a1c98bc8c9	fix build	2023-09-20 09:50:35 -04:00
Igor Nikonov	2d446fc1ca	Merge remote-tracking branch 'origin/master' into parallel-replicas-not-enough-replicas	2023-09-20 13:40:08 +00:00
slvrtrn	fb0ba86489	Merge remote-tracking branch 'origin' into tableau-online-information-schema	2023-09-20 15:14:40 +02:00
Robert Schulze	a4eeed50b6	Mark obsolete settings in system.merge_tree_settings as is_obsolete = 1	2023-09-20 12:55:53 +00:00
Azat Khuzhin	9c3fb64106	Check type of the filter expressions while filtering by virtual columns ... This should fix filtering by virtual columns with non-uint8 types, i.e. queries like: SELECT * FROM data WHERE 1.0 Fixes: 02346_full_text_search Fixes: 00990_hasToken_and_tokenbf v2: move check out from is_const to filterBlockWithQuery(), since in is_const there is no way to validate sets. Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>	2023-09-20 13:36:25 +02:00
flynn	dfa95ecc2e	Merge branch 'master' of github.com:ClickHouse/ClickHouse into comment	2023-09-20 11:16:33 +00:00
Nikolai Kochetov	0bc41bab74	Merge pull request #54544 from kitaisreal/key-condition-in-function-different-types ... KeyCondition IN function support different types	2023-09-20 12:54:01 +02:00
alesapin	2ff36b6e0c	Merge pull request #54768 from ClickHouse/better_exception_in_check_data_part ... Better exception message in checkDataPart	2023-09-20 12:04:46 +02:00
Robert Schulze	f3bf8388b6	Incorporate feedback	2023-09-20 10:00:00 +00:00
SmitaRKulkarni	d8adf05de2	Added a new column _block_number (#47532) ... Added a new virtual column _block_number which is persisted on merges when allow_experimental_block_number_column is enabled	2023-09-20 11:31:12 +02:00
Azat Khuzhin	d9a634eb0f	Fix filtering parts with indexHint for non analyzer ... Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com> (cherry picked from commit ffa82e9297)	2023-09-20 11:29:35 +02:00
pufit	34aecc0bf3	Adjusting num_streams by expected work in StorageS3	2023-09-19 23:05:48 -04:00
slvrtrn	636fc506aa	Replace information_schema view columns aliases with uppercase	2023-09-19 23:42:40 +02:00
Igor Nikonov	e28f4e819c	Merge branch 'master' into parallel-replicas-not-enough-replicas	2023-09-19 23:05:24 +02:00
Azat Khuzhin	c439c4bca2	Revert "Fix filtering parts with indexHint for non analyzer"	2023-09-19 21:39:21 +02:00
Han Fei	eb86c113e5	Merge pull request #54184 from priera/fix_mv_insertion_when_duplicated_parts ... Inserting only non-duplicate chunks in MV	2023-09-19 20:41:45 +02:00
Kruglov Pavel	d555fdffa5	Merge pull request #54449 from azat/parts-prune-indexHint ... Fix filtering parts with indexHint for non analyzer	2023-09-19 19:15:41 +02:00
Han Fei	17243b84ab	Merge branch 'master' into hanfei/statistic	2023-09-19 17:09:17 +02:00
Han Fei	83d1b44a3e	validate	2023-09-19 17:08:15 +02:00
slvrtrn	e4e9727572	Update information_schema docs	2023-09-19 14:14:17 +02:00
alesapin	9dcc41cf83	Fxi	2023-09-19 13:47:32 +02:00
alesapin	28b71163b4	Merge branch 'master' into better_exception_in_check_data_part	2023-09-19 13:45:56 +02:00
alesapin	7b1cf9d912	Revert "Better check for exceptions in checkData of non replicated merge tree" ... This reverts commit 363ee7cae5.	2023-09-19 13:45:34 +02:00
alesapin	363ee7cae5	Better check for exceptions in checkData of non replicated merge tree	2023-09-19 13:41:55 +02:00
priera	b92e37d2df	Fixing PR comments	2023-09-19 12:15:01 +02:00
priera	94d737264c	inserting non-duplicate chunks	2023-09-19 12:15:00 +02:00
serxa	311db94640	default max_burst equals to max_speed as in Throttler	2023-09-19 10:10:21 +00:00
Maksim Kita	c7ddbab9bc	Updated implementation	2023-09-19 13:06:19 +03:00
flynn	c3d12ce0e1	update comment	2023-09-19 08:18:37 +00:00
Anton Popov	72f2bfd166	fix build	2023-09-19 05:35:57 +00:00
VanDarkholme7	783b48b730	Merge branch 'ClickHouse:master' into missing_subcolumns	2023-09-19 10:33:06 +08:00
Anton Popov	624782bd8f	Merge remote-tracking branch 'upstream/master' into HEAD	2023-09-18 23:13:20 +00:00
Anton Popov	d94c1b1ec9	add check for collisions in filenames	2023-09-18 22:27:04 +00:00
slvrtrn	d91a656dd4	Revert schemata.sql format	2023-09-18 22:33:16 +02:00
slvrtrn	ad7b707658	Update information_schema.tables sql and docs	2023-09-18 22:30:27 +02:00
Robert Schulze	097217628e	Merge remote-tracking branch 'rschu1ze/master' into is_deterministic	2023-09-18 20:17:17 +00:00
slvrtrn	9a0d07c682	Update information_schema docs	2023-09-18 22:14:53 +02:00
slvrtrn	54f902f0b7	Adjust information_schema for Tableau Online	2023-09-18 19:50:05 +02:00
Sema Checherinda	e7550523c8	Merge pull request #54651 from CheSema/limit_backoff_timeout ... limit the delay before next try in S3	2023-09-18 19:41:26 +02:00
Robert Schulze	4373ed0f98	Added field "is_deterministic" to system.functions	2023-09-18 17:27:53 +00:00
Alexander Sapin	326eaa9bc5	Better exception message in checkDataPart	2023-09-18 18:26:53 +02:00
Igor Nikonov	047d214436	Merge remote-tracking branch 'origin/master' into parallel-replicas-not-enough-replicas	2023-09-18 15:29:56 +00:00
Alexey Milovidov	e4d47539ab	Merge pull request #54477 from azat/view-sampling ... Support SAMPLE BY for VIEW	2023-09-18 14:00:52 +03:00
Kruglov Pavel	9c888ea42b	Merge pull request #53549 from Avogar/group-by-constant-keys ... Optimize group by constant keys	2023-09-18 12:12:40 +02:00
Yakov Olkhovskiy	711876dfa8	Merge pull request #54696 from ClickHouse/fix-match-keycondition ... Fix: avoid using regex match, possibly containing alternation, as a key condition.	2023-09-17 22:45:13 -04:00
Robert Schulze	de4f22e20a	Typo	2023-09-17 15:27:30 +00:00
Robert Schulze	9f009cccd5	Incorporate review feedback	2023-09-17 15:22:51 +00:00
Robert Schulze	f5e8028bb1	Merge pull request #54642 from rschu1ze/broken-re2st ... Remove broken lockless variant of re2	2023-09-17 15:30:57 +02:00
robot-clickhouse-ci-1	a878d104d7	Merge pull request #52839 from zvonand/zvonand-globs-small-fix ... Fix directory permissions for multi-directory globs. Follow-up #50559	2023-09-16 20:52:30 +02:00
robot-ch-test-poll1	a2ed5c2cae	Merge pull request #54174 from ClickHouse/vdimir/fix_hostname_const ... Fix hostname and co result constness in new analyzer	2023-09-16 19:37:08 +02:00
VanDarkholme7	cf8c614fed	Fix bug on DELETE and UPDATE with json subcolumns	2023-09-16 13:04:19 +00:00
robot-ch-test-poll4	5fb8e46967	Merge pull request #54635 from Avogar/cluster-func-names-with-spaces ... Fix schema inference for *Cluster functions for column names with spaces	2023-09-15 20:36:33 +02:00
Yakov Olkhovskiy	f52e87f474	fix	2023-09-15 18:30:26 +00:00