Commit Graph

123636 Commits

Author SHA1 Message Date
Igor Nikonov
c3caead859
Merge pull request #52688 from Algunenano/better_zk_retries_on_insert
Improve insert retries on keeper session expiration
2023-08-31 17:00:02 +02:00
pufit
1bc6b40a12
Merge pull request #35102 from tonickkozlov/bloom-filter-index/deduplicate
Prune bloom filter indices based on data cardinality
2023-08-31 10:04:19 -04:00
Robert Schulze
13c02e7f2b
Merge pull request #54110 from rschu1ze/docs-follow-up
Docs: Follow-up to #53818
2023-08-31 15:06:44 +02:00
Anton Kozlov
af21620ab5
Update src/Storages/MergeTree/MergeTreeIndexGranuleBloomFilter.h
Co-authored-by: pufit <pufit@yandex.ru>
2023-08-31 14:06:21 +01:00
Anton Kozlov
d3123da824
Update src/Storages/MergeTree/MergeTreeIndexAggregatorBloomFilter.cpp
Co-authored-by: pufit <pufit@yandex.ru>
2023-08-31 14:05:57 +01:00
Anton Kozlov
3f6ef6b5d9
Update src/Storages/MergeTree/MergeTreeIndexGranuleBloomFilter.h
Co-authored-by: pufit <pufit@yandex.ru>
2023-08-31 14:05:41 +01:00
Anton Kozlov
e6895688d7
Update src/Storages/MergeTree/MergeTreeIndexAggregatorBloomFilter.cpp
Co-authored-by: pufit <pufit@yandex.ru>
2023-08-31 14:05:28 +01:00
Robert Schulze
77327c69ed
Merge pull request #53893 from ClibMouse/feature/make-it-old-school
Use iterators instead of std::ranges
2023-08-31 14:55:45 +02:00
Kseniia Sumarokova
07fe5d96d6
Merge pull request #54058 from Algunenano/filesystem_prefetch_max_memory_usage
Correct doc for filesystem_prefetch_max_memory_usage
2023-08-31 14:43:35 +02:00
Raúl Marín
4703c463aa Merge remote-tracking branch 'blessed/master' into better_zk_retries_on_insert 2023-08-31 14:37:02 +02:00
Raúl Marín
0a5f1a643a Merge remote-tracking branch 'blessed/master' into filesystem_prefetch_max_memory_usage 2023-08-31 14:36:10 +02:00
Vitaly Baranov
90b174d3dc
Merge pull request #53326 from azat/backups/base-backup-credentials
Add ability to override credentials for accessing base backup in S3
2023-08-31 14:24:43 +02:00
Mikhail f. Shiryaev
a1024813ec
Merge pull request #53491 from Felixoid/get-rid-of-secrets-ci-logs
Get rid of secrets CLICKHOUSE_CI_LOGS
2023-08-31 13:33:41 +02:00
Robert Schulze
d8e6395eea
Docs: Follow-up to #53818 2023-08-31 11:15:50 +00:00
Robert Schulze
97d960ba1d
Merge pull request #53818 from arenadata/ADQM-1070
Add new hide_in_preprocessed attribute in configuration file
2023-08-31 13:00:16 +02:00
robot-ch-test-poll3
55ba08490d
Merge pull request #53863 from arenadata/ADQM-1110
Supported globs in clickhouse-local SELECT * FROM '?file*{1..3}.csv'
2023-08-31 12:44:06 +02:00
Alexey Milovidov
e8e6b0a165 Add remarks 2023-08-31 08:14:27 +02:00
Alexey Milovidov
6b7b6a6351 Move S3Queue to experimental features 2023-08-31 08:06:35 +02:00
Alexey Gerasimchuck
3a212217a3 Implemented globs to select * from '<file>' 2023-08-31 04:20:44 +00:00
robot-ch-test-poll
10898ee96f
Merge pull request #54092 from arenadata/ADQM-1080
Increased log waiting timeout in test_profile_max_sessions_for_user
2023-08-31 06:04:51 +02:00
Alexey Milovidov
9382dd90ee
Merge pull request #54094 from ClickHouse/changelog-23.8
Changelog for 23.8
2023-08-31 04:03:25 +03:00
Alexey Milovidov
f6cac3c45c Changelog for 23.8 2023-08-31 01:46:46 +02:00
ltrk2
0b2a32b0ba Use iterators instead of std::ranges
For loops for fun factor
2023-08-30 16:36:02 -07:00
Alexey Gerasimchuck
3de967ecbe Increased log waiting timeout 2023-08-30 22:05:38 +00:00
robot-clickhouse-ci-2
1cf0952d28
Merge pull request #54089 from ClickHouse/docs-mysql-interface-cloud-private-preview
Clarify that the cloud MySQL interface is under private preview
2023-08-30 22:47:18 +02:00
Anton Popov
0387556a34
Merge pull request #53914 from Chen768959/fix-53543-2
'from' is supported as a Expression and fix #53543
2023-08-30 22:25:08 +02:00
Mikhail f. Shiryaev
ab3e9df57f
Address review points 2023-08-30 22:06:32 +02:00
Mikhail f. Shiryaev
e553242957
Get rid of CLICKHOUSE_CI_LOGS_* secrets 2023-08-30 22:06:31 +02:00
Mikhail f. Shiryaev
6fb0257ee0
Use CiLogsCredentials in build_check 2023-08-30 22:06:30 +02:00
Mikhail f. Shiryaev
e113ba9024
Add vim filetype for *.lib files 2023-08-30 22:06:26 +02:00
Mikhail f. Shiryaev
f0c18d4bd7
Rework setup of CI logs export 2023-08-30 22:04:37 +02:00
Mikhail f. Shiryaev
a683d5e2f3
Replace defaul assingment by default usage 2023-08-30 22:04:34 +02:00
Antonio Andelic
0148e15aee
Merge pull request #53880 from ClickHouse/archive-improvements-2
Improve schema inference for archives
2023-08-30 21:11:34 +02:00
Justin de Guzman
f8ca303b25
Clarify that the cloud MySQL interface is under private preview 2023-08-30 11:22:47 -07:00
Azat Khuzhin
d0397acafc Add ability to override credentials for accessing base backup in S3
Sometimes credentials with which the backup had been done are inactive
already, and ClickHouse will not be able to read the metadata file to
continue and fail.

Add a setting to allow ignoring credential from base_backup -
`use_same_s3_credentials_for_base_backup` (default to true).

And the same for RESTORE.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-08-30 20:16:22 +02:00
Mikhail f. Shiryaev
b32345242c
Merge pull request #54081 from ClickHouse/apache-archive
Replace dlcdn.apache.org by archive domain
2023-08-30 19:43:43 +02:00
Mikhail f. Shiryaev
c111adb7ce
Replace dlcdn.apache.org by archive domain 2023-08-30 18:40:36 +02:00
Alexander Tokmakov
5cfd1c6d63
Merge pull request #54066 from kssenii/fix-named-collections-access-type-change
Fix named_collection_admin alias
2023-08-30 18:34:03 +02:00
robot-ch-test-poll4
aa3b687d4a
Merge pull request #54064 from kssenii/fix-logical-error
Fix after #52943
2023-08-30 17:06:25 +02:00
Roman Vasin
92b89e8b9e Use NodePtr instead of Node * 2023-08-30 13:50:10 +00:00
Alexander Tokmakov
4d70624ca3
Merge pull request #53907 from arenadata/ADQM-1126
Added validate_tcp_client_information server setting
2023-08-30 15:10:13 +02:00
kssenii
185e3819ac Fix 2023-08-30 13:41:18 +02:00
kssenii
662f22aed0 Fix 2023-08-30 13:31:54 +02:00
Alexander Tokmakov
55af6e5c3e
Merge pull request #54063 from ClickHouse/revert-53677-mutations_subcolumns
Revert "Fix bug on mutations with subcolumns of type JSON in predicates of UPDATE and DELETE queries."
2023-08-30 12:59:51 +02:00
Alexander Tokmakov
83c5e2fba6
Revert "Fix bug on mutations with subcolumns of type JSON in predicates of UPDATE and DELETE queries." 2023-08-30 12:56:17 +02:00
robot-ch-test-poll2
3e5790451e
Merge pull request #54056 from lucasfcnunes/patch-1
fix typo on s3queue.md
2023-08-30 12:53:17 +02:00
Jiebin Sun
7c529e5691
Optimize the merge if all hashSets are singleLevel in UniqExactSet (#52973)
* Optimize the merge if all hashSets are singleLevel

In PR(https://github.com/ClickHouse/ClickHouse/pull/50748), it has added new phase
`parallelizeMergePrepare` before merge if all the hashSets are not all singleLevel
or not all twoLevel. Then it will convert all the singleLevelSet to twoLevelSet in
parallel, which will increase the CPU utilization and QPS.

But if all the hashtables are singleLevel, it could also benefit from the
`parallelizeMergePrepare` optimization in most cases if the hashtable size are not
too small. By tuning the Query `SELECT COUNT(DISTINCT SearchPhase) FROM hits_v1`
in different threads, we have got the mild threshold 6,000.

Test patch with the Query 'SELECT COUNT(DISTINCT Title) FROM hits_v1' on 2x80 vCPUs
server. If the threads are less than 48, the hashSets are all twoLevel or mixed by
singleLevel and twoLevel. If the threads are over 56, all the hashSets are singleLevel.
And the QPS has got at most 2.35x performance gain.

Threads	Opt/Base
8	100.0%
16	99.4%
24	110.3%
32	99.9%
40	99.3%
48	99.8%
56	183.0%
64	234.7%
72	233.1%
80	229.9%
88	224.5%
96	229.6%
104	235.1%
112	229.5%
120	229.1%
128	217.8%
136	222.9%
144	217.8%
152	204.3%
160	203.2%

Signed-off-by: Jiebin Sun <jiebin.sun@intel.com>

* Add the comment and explanation for PR#52973

Signed-off-by: Jiebin Sun <jiebin.sun@intel.com>

---------

Signed-off-by: Jiebin Sun <jiebin.sun@intel.com>
2023-08-30 11:26:16 +02:00
Antonio Andelic
f406019413 Apply PR comments 2023-08-30 09:26:01 +00:00
Raúl Marín
4f9ddcab2b Correct doc for filesystem_prefetch_max_memory_usage 2023-08-30 10:27:29 +02:00
Antonio Andelic
ddb58217d4 Merge branch 'master' into archive-improvements-2 2023-08-30 07:43:25 +00:00