15306 Commits

Author	SHA1	Message	Date
Robert Schulze	ed052d76e6	Merge pull request #56141 from zvonand/zvonand-more-globs ... Performance enhancement for File, HDFS globs	2023-11-16 14:09:22 +01:00
alesapin	ddcfecacc2	Merge pull request #56000 from evillique/fix-replica-already-exists-storage ... Fix `REPLICA_ALREADY_EXISTS` for ReplicatedMergeTree	2023-11-16 12:51:50 +01:00
alesapin	437a911d7b	Merge pull request #56809 from ClickHouse/add_async_moves_execution ... Implement asynchronous execute of `ALTER TABLE ... MOVE ... TO DISK|VOLUME`	2023-11-16 12:47:31 +01:00
Antonio Andelic	203f83c2c2	Merge branch 'master' into keeper-map-backup-restore	2023-11-16 11:31:11 +00:00
Aleksei Filatov	21a17f8334	Add cancellation hook for moving background operation	2023-11-16 13:37:51 +03:00
Andrey Zvonov	fc15b0c1e7	Merge branch 'ClickHouse:master' into zvonand-more-globs	2023-11-16 11:19:12 +01:00
Antonio Andelic	352eca7aa0	Merge pull request #56789 from ClickHouse/fix-no-user ... Try fix `No user in current context, it's a bug`	2023-11-16 08:20:52 +01:00
Nikolay Degterinsky	7623153d38	Fix flattening existing Nested columns during ADD COLUMN	2023-11-16 07:01:49 +00:00
zvonand	4ff82528f1	small fixes after review	2023-11-16 00:01:36 +01:00
Alexander Tokmakov	c529e64768	Merge branch 'master' into rmt_check_shutdown_flags_in_retry_loops	2023-11-15 23:40:22 +01:00
zvonand	cb74f27b00	Merge branch 'master' of github.com:ClickHouse/ClickHouse into zvonand-more-globs	2023-11-15 21:56:51 +01:00
jsc0218	0d2acf9eec	Able to Filter Engine When Scanning System Tables	2023-11-15 20:30:36 +00:00
Sema Checherinda	f3f8392051	upgrade replication protocol	2023-11-15 19:03:55 +01:00
avogar	28522d56b7	Support Iceberg metadata files for metastore tables	2023-11-15 17:45:07 +00:00
Alexander Sapin	3e26161eaa	Implement asynchronous excute of ALTER TABLE ... MOVE ... TO DISK|VOLUME	2023-11-15 18:33:59 +01:00
Nikolay Degterinsky	57aa049fef	Change table name to table UUID, fix test	2023-11-15 16:21:13 +00:00
Alexander Tokmakov	bf10989515	Fix unexpected parts handling (#56693) ... * fix unexpected parts handling * Automatic style fix * fix --------- Co-authored-by: robot-clickhouse <robot-clickhouse@users.noreply.github.com>	2023-11-15 16:54:47 +01:00
zvonand	a8aa206cfd	Update after review ... expandSelector is now returning vector<string> directly minor changes	2023-11-15 16:37:24 +01:00
zvonand	70aa6e2672	Added a shortcut for no-wildcard globs in HDFS	2023-11-15 16:37:24 +01:00
zvonand	3e47a54a94	Added a shortcut for no-wildcard globs in file storage	2023-11-15 16:37:24 +01:00
Antonio Andelic	5c4e58d27d	Try fix No user	2023-11-15 15:14:46 +00:00
Nikolai Kochetov	62378b4219	Fixing style.	2023-11-15 13:56:51 +00:00
Nikolai Kochetov	6ef5c4cafe	Merge pull request #56522 from kitaisreal/analyzer-support-explain-estimate ... Analyzer support EXPLAIN ESTIMATE	2023-11-15 13:17:31 +01:00
Antonio Andelic	d567e718eb	Merge branch 'master' into keeper-map-backup-restore	2023-11-15 11:51:27 +00:00
serxa	bd66ac483c	Load queries for dashboard page from new system.dashboards table	2023-11-14 23:03:06 +00:00
Nikolai Kochetov	1356dc2eaa	Fixing style.	2023-11-14 14:55:21 +00:00
Sema Checherinda	b9cc1580e0	Merge pull request #56367 from canhld94/ch_table_reinit_new_disk ... Adding new disk to storage configuration without restart	2023-11-14 15:54:22 +01:00
Nikolai Kochetov	d5907e10de	Fixing tests.	2023-11-14 12:51:25 +00:00
Ilya Golshtein	3e6a5d8208	merge_row_policy: some comments	2023-11-14 12:22:32 +00:00
Kseniia Sumarokova	0676c09d72	Merge branch 'master' into remove-unused-code	2023-11-14 11:59:53 +01:00
Sema Checherinda	8d36fd6e54	get rid off of client_with_long_timeout_ptr	2023-11-14 11:34:12 +01:00
Sema Checherinda	27fb25d056	alter the naming, fix client_with_long_timeout in s3 storage	2023-11-14 11:34:12 +01:00
Robert Schulze	2931bbfe62	Merge pull request #56665 from rschu1ze/refactor-ast-hash-calculation ... Small refactoring of AST hash calculation (follow-up to #56545)	2023-11-14 11:24:30 +01:00
Antonio Andelic	29f54db917	Merge branch 'master' into keeper-map-backup-restore	2023-11-14 09:47:54 +00:00
Alexander Gololobov	80ca41f18d	Merge pull request #56420 from ClickHouse/system_replicas_with_many_tables ... Optimizations for system.replicas with many tables	2023-11-14 10:23:48 +01:00
Duc Canh Le	3152cd25dc	only check for exact column definitions in replicated merge tree ... Signed-off-by: Duc Canh Le <duccanh.le@ahrefs.com>	2023-11-14 07:41:45 +00:00
Duc Canh Le	57b160b6b9	compare type name in column description ... Signed-off-by: Duc Canh Le <duccanh.le@ahrefs.com>	2023-11-14 07:12:12 +00:00
Alexey Milovidov	3ef14f6098	Merge branch 'master' of github.com:ClickHouse/ClickHouse into coverage	2023-11-14 06:08:32 +01:00
Nikolai Kochetov	4004248c13	Fix duplicate usage of table function input().	2023-11-13 19:01:36 +00:00
Alexey Milovidov	c81cc3f953	Merge pull request #56660 from ClickHouse/better-ccache-2 ... Better use of build cache	2023-11-13 19:28:33 +01:00
Alexander Gololobov	b49f50bf47	Set DB:Exception on unscheduled requests	2023-11-13 17:53:38 +01:00
Alexander Gololobov	6e861a6176	Handle query cancellation	2023-11-13 17:53:38 +01:00
Alexander Gololobov	1113b3041b	Return exception from the job	2023-11-13 17:53:38 +01:00
Alexander Gololobov	c27913f09a	Separate thread pools for request with and w/o zk	2023-11-13 17:53:38 +01:00
Alexander Gololobov	45f5d28299	Deduplicate concurrent requests in system.replicas	2023-11-13 17:53:38 +01:00
Nikolay Degterinsky	b8cba418e3	Merge pull request #56493 from evillique/fix-alter-column-alias ... Fix ALTER COLUMN with ALIAS	2023-11-13 17:48:06 +01:00
Robert Schulze	d6df7d8403	Merge remote-tracking branch 'rschu1ze/master' into refactor-ast-hash-calculation	2023-11-13 15:54:21 +00:00
Antonio Andelic	9e91e4d671	Define BackupReferenceEntry	2023-11-13 14:43:02 +00:00
Sema Checherinda	7f3a082c0e	Merge pull request #56456 from ucasfl/fix-mysql ... Fix transfer query to MySQL compatible query	2023-11-13 15:25:58 +01:00
kssenii	876b2cd7bc	Remove unused code	2023-11-13 15:10:57 +01:00
Antonio Andelic	59480205d4	Merge branch 'master' into keeper-map-backup-restore	2023-11-13 12:21:26 +00:00
Alexey Milovidov	7a489059ac	Merge pull request #56350 from ClickHouse/system-symbols ... Remove garbage.	2023-11-13 12:25:18 +01:00
Robert Schulze	50cd59a72b	Remove default value for argument 'ignore_aliases' from IAST::getTreeHash()	2023-11-13 10:27:38 +00:00
Alexey Milovidov	8e820185cb	Merge branch 'master' of github.com:ClickHouse/ClickHouse into better-ccache-2	2023-11-13 11:17:18 +01:00
Alexey Milovidov	82c461031e	Fix build	2023-11-13 10:09:23 +01:00
Kseniia Sumarokova	b7915e6983	Merge pull request #55782 from aiven-sal/aiven-sal/namedcoll_pr ... NamedCollections: prevent fields overriding in functions' calls	2023-11-13 09:07:09 +01:00
Alexey Milovidov	58319f08be	Merge branch 'master' into system-symbols	2023-11-11 04:12:37 +01:00
Alexey Milovidov	fcd45d47e4	Merge branch 'master' of github.com:ClickHouse/ClickHouse into remove-cpp-templates-2	2023-11-11 03:45:54 +01:00
Alexey Milovidov	c095b1e9ef	Merge pull request #56502 from amosbird/fix-56481 ... Fix two cases of projection analysis.	2023-11-11 02:29:04 +01:00
Alexey Milovidov	3bec4dce8e	Merge branch 'master' into remove-cpp-templates-2	2023-11-11 00:50:32 +01:00
Maksim Kita	bf08f3ef5a	Revert "Inserting only non-duplicate chunks in MV"	2023-11-10 21:21:13 +03:00
Kseniia Sumarokova	2709ecd50d	Merge pull request #56457 from ClickHouse/fix-test-database-hdfs ... Fix test `02725_database_hdfs.sh`	2023-11-10 11:40:18 +01:00
Kseniia Sumarokova	e48df94772	Merge pull request #56377 from ClickHouse/minor-improvements-for-s3-queue ... Minor improvements for S3Queue	2023-11-10 11:28:09 +01:00
Antonio Andelic	5fea9f9dc6	Small fixes	2023-11-10 09:27:26 +00:00
Alexey Milovidov	e5df9de12b	Merge branch 'remove-cpp-templates-2' into system-symbols	2023-11-10 05:28:45 +01:00
Alexey Milovidov	8c253b9e3e	Remove C++ templates	2023-11-10 05:25:02 +01:00
Amos Bird	16578d97a3	Fix two cases of projection analysis. ... 1. Fix projection analysis with partitions. 2. Apply alter conversions during projection analysis.	2023-11-10 10:23:02 +08:00
Alexey Milovidov	29b75d034a	Fix FreeBSD	2023-11-09 23:43:12 +01:00
Alexey Milovidov	2c220ba5ec	Fix FreeBSD	2023-11-09 23:43:12 +01:00
Alexey Milovidov	84ada59e42	Fix FreeBSD	2023-11-09 23:43:12 +01:00
Alexey Milovidov	8f67894157	Add table system.symbols	2023-11-09 23:43:12 +01:00
Alexey Milovidov	cc39484f7f	Merge branch 'master' into system-symbols	2023-11-09 23:05:55 +01:00
Maksim Kita	6fb7d44b62	Analyzer support EXPLAIN ESTIMATE	2023-11-09 19:43:14 +03:00
Salvatore Mesoraca	178f2bbee5	Fix format	2023-11-09 17:25:20 +01:00
Salvatore Mesoraca	1174689f8b	Refactoring	2023-11-09 17:25:20 +01:00
Salvatore Mesoraca	b8e478195f	Simplify code	2023-11-09 17:25:19 +01:00
Salvatore Mesoraca	52e49f4ce3	Add explanatory comments	2023-11-09 17:25:19 +01:00
Salvatore Mesoraca	45e4e122fe	Update src/Common/NamedCollections/NamedCollections.cpp ... Co-authored-by: Kseniia Sumarokova <54203879+kssenii@users.noreply.github.com>	2023-11-09 17:25:19 +01:00
Salvatore Mesoraca	7d206cbc3c	NamedCollections: prevent fields overriding in functions call ... Add syntax in SQL and XML to mark specific fields to allow override or not. Also add a new setting to control the default behaviour when overriding support is not specified.	2023-11-09 17:25:16 +01:00
Antonio Andelic	f9895ab37b	Small fixes and add test	2023-11-09 15:56:57 +00:00
Alexander Tokmakov	113add2556	rmt: check shutdown flags in retry loops	2023-11-09 16:41:01 +01:00
Nikolai Kochetov	c643069b0d	Merge pull request #54905 from nickitat/fix_key_analysis_merge_engine ... Fix key analysis (with set) for `Merge` engine	2023-11-09 14:52:54 +01:00
Kseniia Sumarokova	49f9b6d340	Merge pull request #56260 from joelynch/delta-lake-repeated-add ... deltalake: Do not raise errors when processing add and remove actions	2023-11-09 13:15:31 +01:00
Kseniia Sumarokova	0760e69e54	Merge pull request #56312 from ClickHouse/parallelize-backup-entries-collector ... Parallelize `BackupEntriesCollector`	2023-11-09 13:07:48 +01:00
Kseniia Sumarokova	a9d8e4170f	Merge pull request #56387 from ClickHouse/fix-backup-of-mv-with-no-inner-table ... Allow backup of mat view with dropped inner table instead of failing the backup	2023-11-09 13:02:41 +01:00
Nikolai Kochetov	f7e7089bc9	Merge pull request #56391 from ClickHouse/rocks-db-analyzer ... Fix rocksdb with analyzer.	2023-11-09 12:48:18 +01:00
Nikolay Degterinsky	4ac3dcc417	Fix ALTER COLUMN with ALIAS	2023-11-09 07:07:41 +00:00
flynn	41cdd5dd1a	Fix	2023-11-09 06:23:23 +00:00
Ilya Golshtein	9c5f9f7dcb	merge_row_policy: style fix, cleanup	2023-11-09 00:51:57 +03:00
Ilya Golshtein	f1af83447e	merge_row_policy: cleanup, traces removed	2023-11-08 23:07:08 +03:00
Alexander Gololobov	517a184d5a	Make IMergeTreeDataPart::getState() inlinable (#56461)	2023-11-08 20:30:45 +01:00
flynn	4e263956e0	Merge branch 'fix-mysql' of github.com:ucasfl/ClickHouse into fix-mysql	2023-11-08 16:22:16 +00:00
flynn	ca18674f1a	update test	2023-11-08 16:21:16 +00:00
Alexey Milovidov	68d3f60ef7	Merge branch 'master' into system-symbols	2023-11-08 17:07:11 +01:00
Ilya Golshtein	6e189b17c0	merge_row_policy: processAliases introduced	2023-11-08 18:55:19 +03:00
flynn	d0ba561faa	Update src/Storages/transformQueryForExternalDatabase.cpp ... Co-authored-by: Sema Checherinda <104093494+CheSema@users.noreply.github.com>	2023-11-08 23:20:12 +08:00
Antonio Andelic	4438c2f70a	Remove unnecassary log	2023-11-08 14:36:39 +00:00
Antonio Andelic	1f000242a1	Merge branch 'master' into keeper-map-backup-restore	2023-11-08 13:16:47 +00:00
Antonio Andelic	18a5eeec38	Make on cluster backup/restore work	2023-11-08 13:16:38 +00:00
kssenii	a5b9ba51cd	Better test	2023-11-08 12:55:14 +01:00
flynn	399d61b222	Fix transfer query to MySQL compatible query	2023-11-08 11:34:49 +00:00
kssenii	dfcc94fff0	Fix test	2023-11-08 12:29:40 +01:00
Antonio Andelic	86ba6ad1e8	Local backup and restore	2023-11-08 10:22:44 +00:00
Alexey Milovidov	ca83da14f2	Merge branch 'master' of github.com:ClickHouse/ClickHouse into system-symbols	2023-11-07 19:43:19 +01:00
slvrtrn	f663228404	Add information_schema.statistics	2023-11-07 18:59:45 +01:00
kssenii	15517d04df	Fix	2023-11-07 18:02:44 +01:00
kssenii	70048236f3	Add ProfileEvents column to system.backups	2023-11-07 17:32:08 +01:00
kssenii	9dce1a52e1	Review fix	2023-11-07 16:31:41 +01:00
Alexey Milovidov	5169e4e54d	Fix FreeBSD	2023-11-07 15:30:31 +01:00
Alexander Tokmakov	706f2dcf54	Add a metric for suspicious parts in ZooKeeper (#56395)	2023-11-07 12:55:57 +01:00
Nikolai Kochetov	c58c583043	Fix tests.	2023-11-07 11:43:16 +00:00
Nikita Mikhaylov	0555f8a68c	Fix crash in case of adding a column with type Object(JSON) (#56307)	2023-11-07 12:01:52 +01:00
serxa	19aa1375fc	Merge branch 'master' into async-loader-integration	2023-11-07 10:25:38 +00:00
Nikita Mikhaylov	b696540a5a	Merge branch 'master' into rocks-db-analyzer	2023-11-07 00:44:49 +01:00
Vitaly Baranov	1f4697fbb7	Merge pull request #56339 from vitlibar/fix-using-table-shared-id-during-backup-and-improve-logs ... Fix using table shared id during backup and improve logs.	2023-11-07 00:04:59 +01:00
Alexey Milovidov	9e6dee93f3	Fix FreeBSD	2023-11-06 23:26:11 +01:00
Alexander Tokmakov	cf6ab2e110	Merge pull request #56282 from ClickHouse/fix_unexpected_parts_after_truncate ... Fix unexpected parts after drop range	2023-11-06 21:03:57 +01:00
Nikolai Kochetov	34fdb8a7c6	Fix rocksdb with analyzer.	2023-11-06 18:46:39 +00:00
kssenii	7883a74f6d	Fix	2023-11-06 19:05:12 +01:00
Sema Checherinda	a5e17dc919	Merge pull request #55566 from CheSema/remote-blobs-naming ... feature: storage_metadata_write_full_object_key	2023-11-06 16:41:49 +01:00
Alexey Milovidov	66b180d751	Fix FreeBSD	2023-11-06 16:31:02 +01:00
kssenii	f2c0434c4d	Merge remote-tracking branch 'origin/master' into minor-improvements-for-s3-queue	2023-11-06 15:51:11 +01:00
kssenii	8db2aca832	Add a test	2023-11-06 15:48:04 +01:00
kssenii	da21413354	Better shutdown	2023-11-06 15:47:57 +01:00
kssenii	9952261a59	Add missing write to s3queue_log, pass cached FileStatus	2023-11-06 15:47:45 +01:00
Alexander Tokmakov	13599ff462	review fixes	2023-11-06 15:31:54 +01:00
Alexander Tokmakov	eba52b318d	Merge pull request #56272 from ClickHouse/rdb_fix_rare_logical_error ... Fix rare logical error in Replicated database	2023-11-06 14:14:55 +01:00
Ilya Golshtein	d900f8a352	merge_row_policy: aliases slightly work	2023-11-06 15:18:25 +03:00
Sema Checherinda	52850d5a45	Update StorageSystemRemoteDataPaths.cpp	2023-11-06 11:45:17 +01:00
Kseniia Sumarokova	a050cdd897	Merge branch 'master' into delta-lake-repeated-add	2023-11-06 11:30:58 +01:00
Duc Canh Le	4c21ba7b6f	tables auto initialize new disks without restart ... Signed-off-by: Duc Canh Le <duccanh.le@ahrefs.com>	2023-11-06 10:26:48 +00:00
Alexey Milovidov	99fe7a46fc	Minor changes	2023-11-06 01:12:13 +01:00
Alexey Milovidov	8182d51978	Add table system.symbols	2023-11-06 01:11:53 +01:00
Vitaly Baranov	1e89e7a6b2	Fix using table shared id during backup and improve logs.	2023-11-05 09:38:08 +01:00
Igor Nikonov	9bdcba1b20	Merge branch 'master' into pr-progress-bar	2023-11-05 00:09:46 +01:00
Alexander Tokmakov	155bdfcf0d	fix	2023-11-04 22:50:19 +01:00
Alexey Milovidov	cd94d02455	Merge branch 'master' into coverage	2023-11-04 21:49:59 +01:00
Arthur Passos	b6e205dcdf	Add ClickHouse setting to disable tunneling for HTTPS requests over HTTP proxy (#55033) ... * initial commit. integ tests passing, need to re-run unit & my own personal tests * partial refactoring to remove Protocol::ANY * improve naming * remove all usages of ProxyConfiguration::Protocol::ANY * fix ut * blabla * support url functions as well * support for HTTPS requests over HTTP proxy with tunneling off * remove gtestabc * fix silly mistake * ... * remove usages of httpclientsession::proxyconfig in src/ * got you * remove stale comment * it seems like I need reasonable defaults * fix ut * add some comments * remove no longer needed header * matrix out * add https over http proxy with no tunneling * soem docs * partial refactoring * rename to use_tunneling_for_https_requests_over_http_proxy * improve docs * use shorter version * remove useless test * rename the setting * update * fix typo * fix setting docs typo * move ); up * move ) up	2023-11-04 13:47:52 -04:00
Alexander Tokmakov	9acd3707cb	fix tests	2023-11-04 15:44:55 +01:00
Nikita Taranov	26d3a3a1e0	Add assertion that SizePredictor is set if preferred_block_size_bytes is set (#56302) ... * Test * fix	2023-11-03 23:13:18 +01:00
alesapin	f69796471c	Merge pull request #56303 from ClickHouse/fix_obscure_segfault ... Fix rare race condition related to Memory allocation failure	2023-11-03 23:00:11 +01:00
Alexander Tokmakov	a44e27b0dc	fix	2023-11-03 22:28:10 +01:00
Nikita Mikhaylov	000d6cd7e5	Merge pull request #56234 from evillique/fix-suggestions ... Fix client suggestions for user without grants	2023-11-03 17:55:51 +01:00
Alexander Sapin	3b212a2593	Reserver in advance	2023-11-03 16:53:52 +01:00
Alexander Sapin	a0a0dd02d5	Fix typo everywhere	2023-11-03 16:40:50 +01:00
Alexander Sapin	c3dcb81a62	Fix race condition in case of memory errors	2023-11-03 16:00:16 +01:00
Alexander Sapin	a9ffb9808e	Merge branch 'master' into fix_obscure_segfault	2023-11-03 12:20:52 +01:00
Kseniia Sumarokova	31ae602d37	Merge pull request #56131 from ClickHouse/backup-some-more-retries ... Add missing zookeeper retries in StorageReplicatedMergeTree::backupData	2023-11-03 12:00:41 +01:00
Alexander Tokmakov	b50c4c9f3b	fix	2023-11-03 11:24:10 +01:00
vdimir	86524fce0f	Merge pull request #56246 from ClickHouse/vdimir/analyzer_01019_alter_materialized_view ... Analyzer: fix 01019_alter_materialized_view_consistent	2023-11-03 11:04:14 +01:00
vdimir	a70d0c5203	Merge pull request #56264 from azat/rocksdb/block-based-table-options ... Add ability to specify BlockBasedTableOptions for rocksdb	2023-11-03 10:15:19 +01:00
vdimir	55a1baf029	Merge pull request #56022 from ClickHouse/vdimir/check_all_tables ... Add CHECK query for all the tables	2023-11-03 10:10:42 +01:00
Alexander Tokmakov	480e58b427	fix unexpected parts after drop range	2023-11-03 01:47:50 +01:00
alesapin	2daff2d41e	Fxi	2023-11-03 00:32:07 +01:00
Alexey Milovidov	b400a6682e	Merge pull request #56275 from ClickHouse/23.11-prepare ... Update version after release	2023-11-02 20:20:03 +01:00
Alexey Milovidov	1348d4c696	Update autogenerated version to 23.11.1.1 and contributors	2023-11-02 20:18:15 +01:00
Alexander Tokmakov	e88fa10aa2	Merge pull request #56142 from ClickHouse/make_replicated_start_more_robust ... Better process broken parts on table start for replicated tables	2023-11-02 20:11:22 +01:00
Alexander Tokmakov	0ce1560e5d	retry if table replica being restarted	2023-11-02 18:30:32 +01:00
serxa	e97edf5285	Merge branch 'master' into async-loader-integration	2023-11-02 14:56:32 +00:00
Robert Schulze	99dd3ed060	Merge pull request #56226 from HarryLeeIBM/hlee-ftsearch-compress ... Add compression of posting lists for inverted index	2023-11-02 15:56:30 +01:00
Azat Khuzhin	decf85baca	Add ability to specify BlockBasedTableOptions for rocksdb ... Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>	2023-11-02 15:36:00 +01:00
Sema Checherinda	90b64bcdb9	backward compatibility and implementation feature storage_metadata_write_full_object_key	2023-11-02 15:20:55 +01:00
kssenii	d0bc77f084	Fxi	2023-11-02 14:45:15 +01:00
Joe Lynch	67e29d5eb9	Allow repeated adds and removes for delta lake files	2023-11-02 13:40:50 +01:00
alesapin	3ba12b6e3e	Merge pull request #56238 from evillique/remove-less-parts ... Do not remove part if `Too many open files` is thrown	2023-11-02 12:33:02 +01:00
Alexey Milovidov	1267246785	Merge pull request #55642 from ClickHouse/morton-utils ... Indexing by space-filling curves	2023-11-02 12:21:33 +01:00
alesapin	5c4915e264	Update src/Storages/MergeTree/MergeTreePartInfo.cpp ... Co-authored-by: Alexander Tokmakov <tavplubix@clickhouse.com>	2023-11-02 12:02:04 +01:00
alesapin	7917a2cb40	Update src/Storages/MergeTree/MergeTreePartInfo.cpp ... Co-authored-by: Alexander Tokmakov <tavplubix@clickhouse.com>	2023-11-02 12:01:58 +01:00
alesapin	800f2f097c	Update src/Storages/MergeTree/MergeTreePartInfo.cpp ... Co-authored-by: Alexander Tokmakov <tavplubix@clickhouse.com>	2023-11-02 12:01:51 +01:00
alesapin	81b93ef12b	Update src/Storages/MergeTree/MergeTreePartInfo.cpp ... Co-authored-by: Alexander Tokmakov <tavplubix@clickhouse.com>	2023-11-02 12:01:46 +01:00
vdimir	5703e0995b	Analyzer: fix 01019_alter_materialized_view_consistent	2023-11-02 10:27:22 +00:00
Nikolay Degterinsky	0d6a0d04ae	Do not remove parts if 'Too many open files' is thrown	2023-11-02 05:01:53 +00:00
Nikolay Degterinsky	f93c032425	Fix suggestions for user without grants	2023-11-02 03:56:59 +00:00
Nikolay Degterinsky	1af52d3b00	Better	2023-11-02 00:42:57 +00:00
Igor Nikonov	823b62a55c	Merge pull request #56162 from ClickHouse/part-check-scheduling-monotonic-time ... Use monotonic clock for part check scheduling	2023-11-01 23:15:27 +01:00
Azat Khuzhin	09c1e76982	Allow manual compaction of rocksdb via OPTIMIZE query ... Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>	2023-11-01 22:27:11 +01:00
HarryLeeIBM	013e4b40c6	Add compression of posting lists for inverted index	2023-11-01 14:14:38 -07:00
Igor Nikonov	447386edd9	Merge branch 'master' into fix-schedule-delayed-part-checks	2023-11-01 21:26:06 +01:00
Duc Canh Le	7d4c97e8f3	print broken part path before throw (#56181) ... Signed-off-by: Duc Canh Le <duccanh.le@ahrefs.com>	2023-11-01 19:11:23 +01:00
Alexander Sapin	1e0cab93f6	Satisfy tidy	2023-11-01 16:51:20 +01:00
Alexander Sapin	bfcd8b7de5	Better logging and tests	2023-11-01 16:50:02 +01:00
Azat Khuzhin	c25d6cd624	Rename directory monitor concept into background INSERT (#55978) ... * Limit log frequence for "Skipping send data over distributed table" message After SYSTEM STOP DISTRIBUTED SENDS it will constantly print this message. Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com> * Rename directory monitor concept into async INSERT Rename the following query settings (with preserving backward compatiblity, by keeping old name as an alias): - distributed_directory_monitor_sleep_time_ms -> distributed_async_insert_sleep_time_ms - distributed_directory_monitor_max_sleep_time_ms -> distributed_async_insert_max_sleep_time_ms - distributed_directory_monitor_batch -> distributed_async_insert_batch_inserts - distributed_directory_monitor_split_batch_on_failure -> distributed_async_insert_split_batch_on_failure Rename the following table settings (with preserving backward compatiblity, by keeping old name as an alias): - monitor_batch_inserts -> async_insert_batch - monitor_split_batch_on_failure -> async_insert_split_batch_on_failure - directory_monitor_sleep_time_ms -> async_insert_sleep_time_ms - directory_monitor_max_sleep_time_ms -> async_insert_max_sleep_time_ms And also update all the references: $ gg -e directory_monitor_ -e monitor_ tests docs | cut -d: -f1 | sort -u | xargs sed -e 's/distributed_directory_monitor_sleep_time_ms/distributed_async_insert_sleep_time_ms/g' -e 's/distributed_directory_monitor_max_sleep_time_ms/distributed_async_insert_max_sleep_time_ms/g' -e 's/distributed_directory_monitor_batch_inserts/distributed_async_insert_batch/g' -e 's/distributed_directory_monitor_split_batch_on_failure/distributed_async_insert_split_batch_on_failure/g' -e 's/monitor_batch_inserts/async_insert_batch/g' -e 's/monitor_split_batch_on_failure/async_insert_split_batch_on_failure/g' -e 's/monitor_sleep_time_ms/async_insert_sleep_time_ms/g' -e 's/monitor_max_sleep_time_ms/async_insert_max_sleep_time_ms/g' -i Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com> * Rename async_insert for Distributed into background_insert This will avoid amigibuity between general async INSERT's and INSERT into Distributed, which are indeed background, so new term express it even better. Mostly done with: $ git di HEAD^ --name-only | xargs sed -i -e 's/distributed_async_insert/distributed_background_insert/g' -e 's/async_insert_batch/background_insert_batch/g' -e 's/async_insert_split_batch_on_failure/background_insert_split_batch_on_failure/g' -e 's/async_insert_sleep_time_ms/background_insert_sleep_time_ms/g' -e 's/async_insert_max_sleep_time_ms/background_insert_max_sleep_time_ms/g' Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com> * Mark 02417_opentelemetry_insert_on_distributed_table as long CI: https://s3.amazonaws.com/clickhouse-test-reports/55978/7a6abb03a0b507e29e999cb7e04f246a119c6f28/stateless_tests_flaky_check__asan_.html Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com> --------- Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>	2023-11-01 15:09:39 +01:00
Alexander Tokmakov	65f3cf81dc	Merge pull request #55987 from ClickHouse/fix_lost_part_count ... Avoid counting lost parts twice	2023-11-01 15:07:27 +01:00
Kruglov Pavel	5533258091	Merge pull request #55477 from Avogar/streaming-errors ... Allow to save unparsed records and errors in RabbitMQ, NATS and FileLog engines	2023-11-01 14:35:35 +01:00
vdimir	2ca1a14ac4	Merge pull request #55741 from ClickHouse/vdimir/fix_analyzer_materialized_distributed_join ... Analyzer: fix block stucture mismatch in matview with engine distributed	2023-11-01 14:16:33 +01:00
Kseniia Sumarokova	4510014014	Update StorageReplicatedMergeTree.cpp	2023-11-01 14:05:12 +01:00
Kseniia Sumarokova	c300297153	Fix style check	2023-11-01 13:27:38 +01:00
Alexander Sapin	b4328ad8dc	Fix stupid bug	2023-11-01 13:06:19 +01:00
kssenii	68fd273114	Better	2023-11-01 11:18:58 +01:00
Zhiguo Zhou	b66a9e8fd3	Optimize DB::combineFilters with AVX512_VBMI2 intrinsic ... The DB::combineFilters combines two UInt8 columns (filters) by scanning the first column for non-zero elements and replacing them with elements in the second column contiguously. This function is optimized with Intel's AVX512_VBMI2 intrinsic, _mm512_mask_expandloadu_epi8, which is designed for implementing the exact operation when the bits in the writemask are set when the corresponding packed integers in the first filter are non-zero, and are not set otherwise. The dynamic dispatch ensures that this optimization only applies to devices with AVX512_VBMI2 support.	2023-11-01 17:14:40 +08:00
Zhiguo Zhou	1ae951efc2	Add unittest MergeTree.CombineFilters ... This commit adds a unittest for validating DB::combineFilters.	2023-11-01 17:14:40 +08:00
jsc0218	f1c7665feb	Merge pull request #56121 from lithiumlabcompany/fix-auto-configuration-queue-for-librdkafka ... Correctly set queued.min.messages	2023-10-31 19:59:04 -04:00
Michael Kolupaev	9c79c165b9	Re-fix 'Block structure mismatch' on concurrent ALTER and INSERTs in Buffer table (#56140) ... * Fix 'Block structure mismatch' on concurrent ALTER and INSERTs in Buffer table * Fix the test	2023-10-31 13:41:54 -07:00
Alexander Sapin	3a1182f6c0	Proper implementation	2023-10-31 18:53:33 +01:00
kssenii	ae1a7fffec	Fix	2023-10-31 16:19:08 +01:00
Alexander Tokmakov	f39f0c6a74	Merge pull request #56083 from ClickHouse/fix_drop_range_commit_existing_part_race ... Fix race condition between DROP_RANGE and committing existing block	2023-10-31 14:24:53 +01:00
Alexander Sapin	f4081e3f85	Merge branch 'master' into make_replicated_start_more_robust	2023-10-31 14:16:39 +01:00
Alexander Tokmakov	4d437ca462	Merge pull request #56137 from ClickHouse/fix_unexpected_parts_handling ... Do not try to activate covered parts when handilng unexpected parts	2023-10-31 14:00:47 +01:00
Azat Khuzhin	11b00e92fd	Add more details to "Data after merge is not byte-identical to data on another replicas" ... It should be a very rare error, but in case of error you need as much details as there are: - print mismatched info (hash, size) - print all files with size and checksum (this may help for instance when new file is added, or when extesion changed -- compression marks) P.S. maybe detach_not_byte_identical_parts should be enabled by default? Thoughts? Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>	2023-10-31 12:31:27 +01:00
Igor Nikonov	f8a7bdb8f0	Use monotonic time for part checks scheduling	2023-10-31 11:21:20 +00:00
Alexey Milovidov	016285ea87	Merge branch 'master' into morton-utils	2023-10-31 01:47:16 +01:00
Han Fei	1237cef26d	address comments	2023-10-31 00:53:46 +01:00
alesapin	20b8954872	Better process broken parts on table start for replicated tables	2023-10-30 23:16:04 +01:00
Alexander Tokmakov	373d8bbb20	do not try to activate covered parts	2023-10-30 20:17:10 +01:00
Igor Nikonov	0958d0b168	Fix style	2023-10-30 16:35:00 +00:00
Nikolai Kochetov	cc3c038394	Fixing test.	2023-10-30 16:29:22 +00:00
kssenii	6fc4c8d332	Add retries	2023-10-30 17:20:17 +01:00
Alexander Tokmakov	ae8d7a557b	Update ReplicatedMergeTreeSink.cpp	2023-10-30 15:43:48 +01:00
Alexander Tokmakov	073a6a6f0e	Update src/Storages/StorageReplicatedMergeTree.cpp ... Co-authored-by: Raúl Marín <git@rmr.ninja>	2023-10-30 15:33:35 +01:00
Alexander Tokmakov	6635b84a46	fixes	2023-10-30 15:20:04 +01:00
Igor Nikonov	f2acc366f7	Fix: schedule delayed part checks correctly	2023-10-30 12:57:43 +00:00
Nikolai Kochetov	554ceb4e1d	Merge branch 'master' into planner-prepare-filters-for-analysis-2	2023-10-30 11:56:30 +01:00
Stas Morozov	5a922014f7	Avoid problem with configuration queued.min.messages automatically for Kafka storage	2023-10-30 13:53:52 +03:00
Kruglov Pavel	e22cfa3d5b	Remove unneeded changge	2023-10-30 11:35:44 +01:00
Kruglov Pavel	53b6a455fc	Remove unused field from .h file	2023-10-30 11:35:09 +01:00
Kruglov Pavel	ba736beaf2	Remove unused field	2023-10-30 11:34:42 +01:00
Han Fei	19cdc6dfe2	Merge branch 'master' into hanfei/statistic	2023-10-30 04:53:18 +01:00
Alexey Milovidov	3631e476eb	Merge pull request #54947 from amosbird/minmax-combinator ... Introduce -ArgMin/-ArgMax combinators.	2023-10-30 03:35:28 +01:00
Alexey Milovidov	64b6e68a50	Merge pull request #55683 from amosbird/issue-55653 ... Reuse granule during skip index reading	2023-10-30 00:51:51 +01:00
Han Fei	df1e019268	address comments	2023-10-30 00:39:16 +01:00
Alexey Milovidov	8e6a7fdff0	Fix f-string	2023-10-29 23:43:36 +01:00
Alexey Milovidov	9aff0a8872	Revert "Fix 'Block structure mismatch' on concurrent ALTER and INSERTs in Buffer table (#55995)" ... This reverts commit b65c498016.	2023-10-29 04:34:04 +03:00
Alexey Milovidov	1b085b1161	Merge pull request #55900 from ClickHouse/make-test-better ... Make test `system_zookeeper_connection` better	2023-10-28 15:01:36 +02:00
Nikita Taranov	213ce63ffb	Merge branch 'master' into optimize_reading3	2023-10-28 00:04:20 +02:00
Michael Kolupaev	b65c498016	Fix 'Block structure mismatch' on concurrent ALTER and INSERTs in Buffer table (#55995)	2023-10-27 10:06:43 -07:00
Alexander Tokmakov	e07bb123d3	fix race condition between DROP_RANGE and committing existing block	2023-10-27 17:50:45 +02:00
Alexander Tokmakov	db5e9f1d1c	beter check for duplicate lost parts	2023-10-27 17:33:30 +02:00
vdimir	faffdba8a2	Split source and worker processors in InterpreterCheckQuery	2023-10-27 15:22:10 +00:00
vdimir	c1ae369919	wip CHECK query for all tables	2023-10-27 15:22:10 +00:00
Sergei Trifonov	a5d7c2b6a2	Merge branch 'master' into async-loader-integration	2023-10-27 10:39:12 +02:00
Alexander Tokmakov	9db5f4396c	Merge branch 'master' into fix_lost_part_count	2023-10-26 17:40:37 +02:00
Andrey Zvonov	5737369d0b	Merge branch 'ClickHouse:master' into zvonand-datetime-ranges	2023-10-26 11:42:13 +02:00
vdimir	12890eb70c	Merge pull request #55806 from canhld94/rocksdb_trivial_count ... Support trivial count for rocksdb and storage join	2023-10-26 11:22:38 +02:00
vdimir	1d5f75fd71	Merge pull request #53404 from ClickHouse/vdimir/check_table_improvements2 ... Improve CHECK TABLE system query, support checking part	2023-10-26 10:57:44 +02:00
Nikolay Degterinsky	3c690337ec	Improvements	2023-10-26 03:17:30 +00:00
zvonand	0766c73aab	Rename date_time_overflow_mode -> date_time_overflow_behavior, moved it to format settings	2023-10-25 23:11:13 +02:00
Igor Nikonov	1a5d485c6c	Merge branch 'master' into pr-progress-bar	2023-10-25 19:53:20 +02:00
Alexander Tokmakov	3c0b8aeb6a	avoid counting lost parts twice	2023-10-25 19:29:40 +02:00
Alexander Tokmakov	c0482cb537	Merge pull request #55944 from azat/rmt-retriable-exception-log ... Do not write retriable errors for Replicated mutate/merge into error log	2023-10-25 18:25:59 +02:00
alesapin	0d68a52c06	Merge pull request #50181 from ClickHouse/repro_49887 ... Make exception about `ALTER TABLE ... DROP COLUMN|INDEX|PROJECTION` more clear	2023-10-25 15:57:17 +02:00
Nikolay Degterinsky	2606b60e66	Fix REPLICA_ALREADY_EXISTS for ReplicatedMergeTree	2023-10-25 04:17:35 +00:00
Igor Nikonov	7b7f100250	Merge remote-tracking branch 'origin/master' into pr-progress-bar	2023-10-24 21:02:29 +00:00
李扬	465962df7f	Support orc filter push down (file + stripe + rowgroup level) (#55330) ... * support orc filter push down * update orc lib version * replace setqueryinfo with setkeycondition * fix issue https://github.com/ClickHouse/ClickHouse/issues/53536 * refactor source with key condition * fix building error * remove std::cout * update orc * update orc version * fix bugs * improve code * upgrade orc lib * fix code style * change as requested * add performance tests for orc filter push down * add performance tests for orc filter push down * fix all bugs * fix default as null issue * add uts for null as default issues * upgrade orc lib * fix failed orc lib uts and fix typo * fix failed uts * fix failed uts * fix ast fuzzer tests * fix bug of uint64 overflow in https://s3.amazonaws.com/clickhouse-test-reports/55330/de22fdcaea2e12c96f300e95f59beba84401712d/fuzzer_astfuzzerubsan/report.html * fix asan fatal caused by reused column vector batch in native orc input format. refer to https://s3.amazonaws.com/clickhouse-test-reports/55330/be39d23af2d7e27f5ec7f168947cf75aeaabf674/stateless_tests__asan__[4_4].htm * fix wrong performance tests * disable 02892_orc_filter_pushdown on aarch64. https://s3.amazonaws.com/clickhouse-test-reports/55330/be39d23af2d7e27f5ec7f168947cf75aeaabf674/stateless_tests__aarch64_.html * add some comments * add some comments * inline range::equals and range::less * fix data race of key condition * trigger ci	2023-10-24 12:08:17 -07:00
Ilya Golshtein	0d27150948	merge_row_policy: cleanup	2023-10-24 13:09:57 +00:00
Kruglov Pavel	cb63b07e89	Merge pull request #55912 from wxybear/fix/storage_null_supports_subcolumns ... fix: StorageNull supports subcolumns	2023-10-24 14:03:46 +02:00
Alexander Gololobov	04b82d6463	Merge pull request #55935 from ClickHouse/set_has_lwd ... Set storage.has_lightweight_delete_parts flag when a part has been loaded	2023-10-24 14:01:11 +02:00
serxa	749cdb9954	Merge branch 'master' into async-loader-integration	2023-10-24 10:09:49 +00:00
Azat Khuzhin	da408df4a7	Proper ignoring of retriable errors during merge/mutate ... The exception cannot be simply suppressed, since sanity checks in the WriteBuffer dtor relies on the std::uncaught_exceptions(), and if the exception will be suppressed, then if the buffer was not finalized, it may abort (even though it is only in debug/sanitizers build). So instead, IExecutableTask::printExecutionException() had been introduced to distinguish when exception should be printed and when not.	2023-10-24 10:19:18 +02:00
Duc Canh Le	e3aa799ca2	disable approximate count by default + fix tests ... Signed-off-by: Duc Canh Le <duccanh.le@ahrefs.com>	2023-10-24 03:18:33 +00:00
Srikanth Chekuri	8876a92b42	Merge remote-tracking branch 'upstream/master' into issue-55023	2023-10-23 21:21:25 +05:30