Commit Graph

14048 Commits

Author SHA1 Message Date
Alexey Milovidov
5e3a69b4c4 Update MergeList.cpp 2023-08-16 17:49:28 +00:00
Alexey Milovidov
bc77aab2e8 Update MergeList.h 2023-08-16 17:49:28 +00:00
Alexey Milovidov
7376b7ec8c Update MergeList.h 2023-08-16 17:49:28 +00:00
Alexey Milovidov
664bcfc8b3 Update MergeList.cpp 2023-08-16 17:49:28 +00:00
Alexey Milovidov
ecd1fb56b4 Update MergeList.cpp 2023-08-16 17:49:28 +00:00
Jianfei Hu
bd4df60df6 fix merges
Signed-off-by: Jianfei Hu <hujianfei258@gmail.com>
2023-08-16 17:49:28 +00:00
Jianfei Hu
59a81b82bc wip the merge partition
Signed-off-by: Jianfei Hu <hujianfei258@gmail.com>
2023-08-16 17:49:28 +00:00
Robert Schulze
5a87a09c84
Tweak exception handling 2023-08-16 17:16:52 +00:00
Davit Vardanyan
a0ec274766 Refactor: Naming 2023-08-16 17:30:56 +04:00
Davit Vardanyan
620733dd46 Fix: Exception handling 2023-08-16 17:30:29 +04:00
Robert Schulze
ea69eb095c
Merge remote-tracking branch 'ClickHouse/master' into usearch 2023-08-16 08:07:57 +00:00
Alexander Tokmakov
b32bdd1b0a
Merge pull request #53164 from ClickHouse/fixes_for_detached_parts
Fixes for detach/attach partition and broken detached parts cleanup
2023-08-16 00:09:13 +03:00
Robert Schulze
56a6d20137
Merge remote-tracking branch 'rschu1ze/master' into factorize-constants 2023-08-15 18:21:25 +00:00
Igor Nikonov
49134711dc Polishing 2023-08-15 17:37:01 +00:00
Robert Schulze
f558048bea
Merge remote-tracking branch 'rschu1ze/master' into consistency-fixes-for-caches 2023-08-15 17:12:52 +00:00
Igor Nikonov
9825f8c76e Remove debug logging 2023-08-15 17:03:32 +00:00
Alexander Tokmakov
83c10dad18 Merge branch 'master' into exception_message_patterns6 2023-08-15 15:20:09 +02:00
Alexander Tokmakov
bf40767f10 fix another race 2023-08-15 14:27:49 +02:00
Igor Nikonov
1678cda8b3 Merge remote-tracking branch 'origin/master' into fix-parallel-replicas-multiply-result 2023-08-15 12:03:35 +00:00
Davit Vardanyan
48c62fd75e Add: USearch 2023-08-15 16:00:27 +04:00
Kruglov Pavel
cac2628b41
Merge pull request #53426 from ClickHouse/analyzer-fix-virtuals-distr
Analyzer: fix virtual columns in StorageDistributed
2023-08-15 13:55:42 +02:00
Robert Schulze
8992cc5af4
Factorize constants 2023-08-15 08:56:40 +00:00
Robert Schulze
f317903073
Merge branch 'ClickHouse:master' into consistency-fixes-for-caches 2023-08-15 10:25:08 +02:00
Igor Nikonov
c94994afcf Propagate setting cluster_for_parallel_replicas to shards 2023-08-14 22:50:46 +00:00
Alexander Tokmakov
376202f739 fix creation of empty parts 2023-08-15 00:26:17 +02:00
Alexander Tokmakov
894ec8e0d0 less exceptions with runtime format string 2023-08-14 21:45:59 +02:00
Dmitry Novik
1244828555 Analyzer: fix virtual columns in StorageDistributed 2023-08-14 19:27:05 +02:00
Alexander Tokmakov
f0f2cd8c13 Merge branch 'master' into fixes_for_detached_parts 2023-08-14 19:20:45 +02:00
ltrk2
2846ea49b4
Merge branch 'master' into feature/mergetree-checksum-big-endian-support 2023-08-14 13:02:30 -04:00
Robert Schulze
3472a928d8
Cosmetics: Rename dropCache() to clearCaches()
"dropCache()" is a bad name, it sounds like that the cache can no longer
be used after the call. Not true, we only "clear" the cache.
2023-08-14 15:42:32 +00:00
Filipp Ozinov
5a180b53ed
Style fix 2023-08-14 12:30:29 +04:00
Alexey Milovidov
9656bd2738
Merge pull request #53372 from ClickHouse/Fix_lazy_database_and_system_table_query
Fix loading lazy database during system.table select query
2023-08-14 08:03:19 +03:00
Yakov Olkhovskiy
300399d1f5
Merge pull request #52329 from ClibMouse/feature/mergetree-checksum-on-big-endian
Implement endianness-indepedent support for MergeTree checksums
2023-08-13 23:59:05 -04:00
Filipp Ozinov
0cccb2ac3b
Fixed system.data_skipping_indices for MaterializedMySQL
Related to #53350
2023-08-14 01:57:41 +04:00
Smita Kulkarni
c25bb44e55 Fixed style check 2023-08-13 21:45:33 +02:00
Smita Kulkarni
ee772b7376 Avoid loading tables from lazy database when not needed 2023-08-13 20:15:58 +02:00
Alexey Milovidov
17127cf23f
Merge pull request #53337 from hanfei1991/hanfei/improve_move_to_prewhere
improve `move_primary_key_columns_to_end_of_prewhere`
2023-08-12 17:45:30 +03:00
Han Fei
0b3f9918a3 fix style 2023-08-12 14:40:02 +02:00
robot-ch-test-poll
967067bbb8
Merge pull request #53250 from evillique/postgresql-single-quotes
Fix character escaping in the PostgreSQL engine
2023-08-12 13:15:49 +02:00
Alexey Milovidov
3c2dd4e543
Update MergeTreeWhereOptimizer.cpp 2023-08-12 03:40:19 +03:00
Alexey Milovidov
6176971b3f
Update StoragePostgreSQL.cpp 2023-08-12 03:33:58 +03:00
Alexey Milovidov
4b59d265e1
Merge pull request #49723 from DimasKovas/fix_storage_buffer_reschedule
Some unclear change in StorageBuffer::reschedule() for something
2023-08-12 03:28:23 +03:00
Alexey Milovidov
e5f33511c6
Merge pull request #53242 from evillique/alter-exception
Fix LOGICAL_ERROR exception in ALTER query
2023-08-12 03:18:32 +03:00
Han Fei
e70dc7d507 improve move_primary_key_columns_to_end_of_prewhere 2023-08-11 17:36:52 +02:00
Anton Popov
8744a0269a
Merge pull request #52530 from CurtizJ/fix-alter-delete-skip-index
Fix recalculation of skip indexes and projections in `ALTER DELETE` queries
2023-08-11 15:23:05 +02:00
ltrk2
1dc1b54c68
Merge branch 'master' into feature/mergetree-checksum-big-endian-support 2023-08-11 07:48:49 -04:00
ltrk2
94dbef1e50
Merge branch 'master' into feature/mergetree-checksum-on-big-endian 2023-08-11 07:47:48 -04:00
Joris Giovannangeli
602570b462
add max_threads for index settings 2023-08-11 10:20:07 +02:00
Alexey Milovidov
aae35f3b1c
Merge pull request #53184 from ClickHouse/better-ccache
Better usage of ccache
2023-08-11 04:43:45 +03:00
Ilya Golshtein
ade9c3d970 system_kafka_consumers: per consumer librdkafka stat 2023-08-10 22:35:05 +00:00
Ilya Golshtein
3bced2f08c system_kafka_consumers: tidy fix, one partition for stable test 2023-08-10 22:35:05 +00:00
Ilya Golshtein
978d8bf9a6 system_kafka_consumers: cleanup with exceptions 2023-08-10 22:35:05 +00:00
Ilya Golshtein
7dab2dc041 system_kafka_consumers: exceptions seem to work 2023-08-10 22:35:05 +00:00
Ilya Golshtein
80829a6bf7 system_kafka_consumers: no offset_committed, new test with MV 2023-08-10 22:35:05 +00:00
Ilya Golshtein
6d6aae24f8 system_kafka_consumers: no friendship with StorageKafka 2023-08-10 22:35:05 +00:00
Ilya Golshtein
ae3e6f0fac system_kafka_consumers: getMemberId introduced to avoid leak 2023-08-10 22:35:05 +00:00
Ilya Golshtein
b325ddceb4 system_kafka_consumers: small thing to make clang tidy happy 2023-08-10 22:35:05 +00:00
Ilya Golshtein
46be5e5782 system_kafka_consumers: cleanup, test should be more stable 2023-08-10 22:35:05 +00:00
Ilya Golshtein
803a80b098 system_kafka_consumers: refactored with getStat, test passed 2023-08-10 22:35:05 +00:00
Ilya Golshtein
a86548a733 system_kafka_consumers: statistics.interval.ms default, tiny cleanup 2023-08-10 22:35:05 +00:00
Ilya Golshtein
dd6727be3e system_kafka_consumers: style check fixes 2023-08-10 22:35:05 +00:00
Ilya Golshtein
bba4dcacc7 system_kafka_consumers: small code cleanup 2023-08-10 22:35:05 +00:00
Ilya Golshtein
3f205f4997 system_kafka_consumers: tiny doc changes 2023-08-10 22:35:05 +00:00
Ilya Golshtein
b71c2bedfc system_kafka_consumers: rebalance test is ok, statistics_interval_ms 2023-08-10 22:35:05 +00:00
Ilya Golshtein
83566e132a system_kafka_consumers: all_consumers introduced, offsets work 2023-08-10 22:35:05 +00:00
Ilya Golshtein
de49d7e5a0 system_kafka_consumers: minor change 2023-08-10 22:35:05 +00:00
Ilya Golshtein
c821c7788a system_kafka_consumers: json commented out 2023-08-10 22:35:05 +00:00
Ilya Golshtein
ae6e0469ad system_kafka_consumers: rdfakfa stat with attempt to use json 2023-08-10 22:35:05 +00:00
Ilya Golshtein
6258ab6cb4 system_kafka_consumers: doc link fix, USE_RDKAFKA 2023-08-10 22:35:05 +00:00
Ilya Golshtein
40fd53c3e3 system_kafka_consumers: style check 2023-08-10 22:35:05 +00:00
Ilya Golshtein
88f5dd5db8 system_kafka_consumers: integration tests passed 2023-08-10 22:35:05 +00:00
Ilya Golshtein
957787d96a system_kafka_consumers: kind of works, test added 2023-08-10 22:35:05 +00:00
Ilya Golshtein
36fb330811 system_kafka_consumers: assignments 2023-08-10 22:35:05 +00:00
Ilya Golshtein
a947c7a91e system_kafka_consumers: start integrating with real kafka 2023-08-10 22:35:05 +00:00
Ilya Golshtein
2304f342c9 system_kafka_consumers: table populated by fake data 2023-08-10 22:35:05 +00:00
Ilya Golshtein
9d68d50ae7 system_kafka_consumers: initial 2023-08-10 22:35:04 +00:00
Alexander Tokmakov
5a8b8203b2 fix 2023-08-10 23:22:51 +02:00
Alexander Tokmakov
3d59ebe108 fix 2023-08-10 20:11:22 +02:00
Nikolay Degterinsky
4f0be777c5 Fix build 2023-08-10 16:39:43 +00:00
Antonio Andelic
478a5006b1 Merge branch 'master' into improve-archive-reading 2023-08-10 15:52:30 +00:00
Alexander Tokmakov
6ae0ba1163 Merge branch 'master' into fixes_for_detached_parts 2023-08-10 17:01:05 +02:00
ltrk2
a2054c04dd
Merge branch 'master' into feature/mergetree-checksum-big-endian-support 2023-08-10 10:21:34 -04:00
ltrk2
b780b64232
Merge branch 'master' into feature/mergetree-checksum-on-big-endian 2023-08-10 10:21:23 -04:00
Alexey Milovidov
bb57caa83e
Merge pull request #53180 from ClickHouse/ditch-tons-of-garbage
Ditch tons of garbage
2023-08-10 16:59:51 +03:00
Alexey Milovidov
f35df23974
Merge branch 'master' into better-ccache 2023-08-10 16:52:32 +03:00
pufit
af51268da1
Merge pull request #51912 from ClickHouse/pufit/role-from-storage
Create, Drop, or Move access entities between different access storages.
2023-08-10 09:47:26 -04:00
Alexander Tokmakov
4b30900fed
Revert "Improve CHECK TABLE system query" 2023-08-10 14:44:16 +03:00
Antonio Andelic
c65f3d1eb8 Mutex for name filter 2023-08-10 10:09:59 +00:00
alesapin
57025eed65
Merge pull request #52745 from ClickHouse/vdimir/check_table_improvements
Improve CHECK TABLE system query
2023-08-10 11:48:00 +02:00
Antonio Andelic
aa258a0e61 Merge branch 'master' into improve-archive-reading 2023-08-10 06:59:04 +00:00
Nikolay Degterinsky
7321f5e543 Better 2023-08-10 06:34:10 +00:00
Alexey Milovidov
6e3100a79d
Merge pull request #53213 from ClickHouse/revert-53210-revert-53100-upload-build-profile
Revert "Revert "Upload build time-trace data to CI database""
2023-08-10 06:43:14 +03:00
Nikolay Degterinsky
3c6d140c17 Fix sigle quote escaping in PostgreSQL engine 2023-08-10 01:31:59 +00:00
Alexey Milovidov
d65ddc6194
Merge pull request #53175 from ClickHouse/analyzer-fix-test-unique-index
Fix: 00838_unique_index test with analyzer
2023-08-10 04:15:02 +03:00
Alexey Milovidov
2592382834
Merge branch 'master' into ditch-tons-of-garbage 2023-08-10 01:04:24 +03:00
pufit
1a3eac0d4d
Merge branch 'master' into pufit/role-from-storage 2023-08-09 17:50:07 -04:00
Nikolay Degterinsky
f2c3000a93 Improve exception in ALTER query 2023-08-09 21:40:06 +00:00
Antonio Andelic
a19d005edb Merge branch 'master' into improve-archive-reading 2023-08-09 20:17:31 +00:00
Alexey Milovidov
22ee135d01
Merge branch 'master' into better-ccache 2023-08-09 21:16:16 +03:00
ltrk2
139e9433a8
Merge branch 'master' into feature/mergetree-checksum-big-endian-support 2023-08-09 09:48:00 -04:00
ltrk2
add53bb8aa
Merge branch 'master' into feature/mergetree-checksum-on-big-endian 2023-08-09 09:44:32 -04:00
Alexey Milovidov
964c407d48
Revert "Revert "Upload build time-trace data to CI database"" 2023-08-09 16:11:17 +03:00
Antonio Andelic
0e17d26b88 More formats supported, read single archive from 1 thread 2023-08-09 11:58:37 +00:00
Alexander Gololobov
63183c56f1
Revert "Upload build time-trace data to CI database" 2023-08-09 13:58:05 +02:00
Alexander Tokmakov
48ed54e822
Allow experimantal features when recovering Replicated db replica (#53167)
* allow experimantal features when recovering Replicated db replica

* Automatic style fix

---------

Co-authored-by: robot-clickhouse <robot-clickhouse@users.noreply.github.com>
2023-08-09 14:35:50 +03:00
Alexey Milovidov
fa9abc5038 Better usage of ccache 2023-08-09 05:02:50 +02:00
pufit
884ea7f592
Merge branch 'master' into pufit/role-from-storage 2023-08-08 20:20:37 -04:00
Alexey Milovidov
aa757490bd Ditch tons of garbage 2023-08-09 02:19:02 +02:00
Igor Nikonov
3296daad96 Don't build AST filter with enabled analyzer 2023-08-08 22:27:12 +00:00
Igor Nikonov
421610338e Merge remote-tracking branch 'origin/master' into analyzer-fix-test-unique-index 2023-08-08 21:59:01 +00:00
Igor Nikonov
040d666546 Fix: 00838_unique_index test with analyzer 2023-08-08 21:57:27 +00:00
Alexey Milovidov
23eee8d10d Merge branch 'master' of github.com:ClickHouse/ClickHouse into upload-build-profile 2023-08-08 21:02:26 +02:00
Alexander Tokmakov
c1b94b4a3f fixes for detach/attach partition 2023-08-08 17:55:04 +02:00
SmitaRKulkarni
cd77406fba
Bug fix for checksum of compress marks (#51777)
* add a test for cmrk and cidx checksums

* Bug fix for checksum of compress marks

* Fixed issue with .cidx checksum

* Fixed style check

* Updated 02253_empty_part_checksums.reference file with part details output

* Update 02253_empty_part_checksums.sh

* Updated to check is_compressed flag instead of size

* Update 02253_empty_part_checksums.sh

---------

Co-authored-by: Alexander Tokmakov <tavplubix@clickhouse.com>
2023-08-08 17:55:07 +03:00
Anton Popov
b6ce725367 fix projection recalc 2023-08-08 14:06:42 +00:00
vdimir
3f915491f0
Merge pull request #53046 from canhld94/ch_fix_direct_hash_join 2023-08-08 11:55:17 +02:00
vdimir
358ef26385
small fixes for CHECK TABLE 2023-08-08 09:45:33 +00:00
vdimir
1aedc4e892
Build proper pipeline for CHECK TABLE 2023-08-08 09:45:32 +00:00
vdimir
a882ef295f
Query CHECK TABLE takes care about progress and cancellation 2023-08-08 09:45:31 +00:00
pufit
2f46ae850e
Merge branch 'master' into pufit/role-from-storage 2023-08-07 17:57:41 -04:00
robot-ch-test-poll
066dd8a2e8
Merge pull request #52949 from CurtizJ/fix-nested-array-lowcardinality
Fix reading of empty `Nested(Array(LowCardinality(...)))`
2023-08-07 23:32:25 +02:00
Sean Haynes
77dd869414
Add ability to log when max_partitions_per_insert_block is reached (#50948)
* Add ability to log when max_partitions_per_insert_block is reached

There's plenty of reasons to drop max_partitions_per_insert_block
down from it's default of 100, with the main one being even touching
remotely close to this number of partitions will create a lot of
merge activity that can de-stabilise a cluster.

This commit adds the ability to log when the above limit is reached
rather than throw an exception, so that you can gauge the impact on
users when dropping this value down.

* Update docs/ru/operations/settings/query-complexity.md

Co-authored-by: Nikita Taranov <nickita.taranov@gmail.com>

* Update docs/ru/operations/settings/query-complexity.md

Co-authored-by: Nikita Taranov <nickita.taranov@gmail.com>

* Update docs/ru/operations/settings/query-complexity.md

Co-authored-by: Nikita Taranov <nickita.taranov@gmail.com>

---------

Co-authored-by: Nikita Taranov <nickita.taranov@gmail.com>
2023-08-07 23:28:19 +02:00
Alexey Milovidov
695f0ffd14 Merge branch 'clang17_fix' of github.com:Algunenano/ClickHouse into upload-build-profile 2023-08-07 21:18:36 +02:00
Azat Khuzhin
2f414950b7
Fix logging for asynchronous non-batched distributed sends (#52583)
Before you may see the following:

  2023.07.25 09:21:39.705559 [ 692 ] {6b5e1299-1b64-4dbb-b25d-45e10027db22} <Trace> test_hkt5nnqj.dist_opentelemetry.DirectoryMonitor.default: Finished processing `` (took 37 ms)

Because file_path and current_file are the references to the same
variable in DistributedAsyncInsertDirectoryQueue::processFile().

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-08-07 20:57:42 +02:00
Duc Canh Le
cd02980130
Merge branch 'master' into ch_fix_direct_hash_join 2023-08-07 22:20:29 +08:00
Anton Popov
981da23144 fix reading of empty Nested(Array(...)) 2023-08-07 12:32:12 +00:00
Raúl Marín
30d1452bbc Compatibility with clang-17 2023-08-07 10:48:24 +02:00
SmitaRKulkarni
75f81bdc44
Merge pull request #50795 from ClickHouse/azure_table_function_cluster
Table function azureBlobStorageCluster
2023-08-07 09:16:52 +02:00
Duc Canh Le
ed2a1d7c9b select required columns when getting join 2023-08-07 03:15:20 +00:00
Antonio Andelic
cafa5ef2f3 Merge branch 'master' into add-reading-from-archives 2023-08-06 10:21:47 +00:00
Antonio Andelic
1d793e69f2 Fix 2023-08-05 17:06:44 +00:00
SmitaRKulkarni
1e3ae89161
Merge branch 'master' into azure_table_function_cluster 2023-08-04 21:39:32 +02:00
Anton Popov
991abde851 fix reading of empty Nested(Array(...)) 2023-08-04 13:10:50 +00:00
Antonio Andelic
d2b6646fc2 Merge branch 'master' into add-reading-from-archives 2023-08-04 12:42:46 +00:00
ltrk2
c010e1c9ff
Merge branch 'master' into feature/mergetree-checksum-on-big-endian 2023-08-04 08:31:57 -04:00
Duc Canh Le
8aaa46ca90 StorageJoin: do not create clone hash join with all columns 2023-08-04 11:07:33 +00:00
Kruglov Pavel
b1fe9ce8f7
Merge branch 'master' into formats-with-subcolumns 2023-08-04 13:01:12 +02:00
Antonio Andelic
d4d93331dd Merge branch 'master' into add-reading-from-archives 2023-08-04 09:00:20 +00:00
Vitaly Baranov
587877dba3
Merge branch 'master' into disable-fs-cache-for-backups 2023-08-04 01:15:23 +02:00
Alexander Tokmakov
968c79cf3f
Merge pull request #52956 from ClickHouse/fixes_for_databases
Fix some issues with databases
2023-08-03 19:13:41 +03:00
Kseniia Sumarokova
68b48a0bc0
Merge pull request #50063 from KevinyhZou/minor_improve_hdfs_read_buffer
Minor improve HDFS ReadBuffer for read end of file
2023-08-03 17:55:48 +02:00
avogar
f0eb22ac5f Fix build for S3Queue 2023-08-03 14:20:19 +00:00
ltrk2
a29b36612b
Merge branch 'master' into feature/mergetree-checksum-big-endian-support 2023-08-03 09:41:25 -04:00
ltrk2
84a77d1c21
Merge branch 'master' into feature/mergetree-checksum-on-big-endian 2023-08-03 09:41:16 -04:00
Anton Popov
92cb5e9ae3
Merge pull request #52687 from alkorgun/fix-named-collections-on-cluster-23.7
Fix named collections on cluster 23.7
2023-08-03 14:50:28 +02:00
Smita Kulkarni
2a681f1577 Removed unused function and fixed build after merge 2023-08-03 14:32:03 +02:00
Smita Kulkarni
68a10e7cae Merge branch 'master' into azure_table_function_cluster 2023-08-03 13:59:59 +02:00
Vitaly Baranov
e127d27453 Disable using fs cache for backup/restore #2. 2023-08-03 13:52:35 +02:00
Vitaly Baranov
f427c779eb Disable using fs cache for backup/restore. 2023-08-03 13:52:32 +02:00
Antonio Andelic
d9f136ce0c Add tests for globs 2023-08-03 10:42:47 +00:00