thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-02 12:32:04 +00:00
Code Issues Packages Projects Releases Wiki Activity
143,380 Commits 9,194 Branches 1,782 Tags 2.2 GiB
ee5100171e
Commit Graph

763 Commits

Author SHA1 Message Date
Vitaly Baranov
308654195c Add config setting "wait_dictionaries_load_at_startup". 2023-11-15 11:48:50 +01:00
Mikhail Koviazin
9e1357dd7e
Update query_masking_rules when reloading the config 
Fixes #56449
 2023-11-14 14:03:28 +02:00
serxa
19aa1375fc Merge branch 'master' into async-loader-integration 2023-11-07 10:25:38 +00:00
Alexey Milovidov
6eff22568f
Revert "Add function getHttpHeader" 2023-10-29 19:50:39 +03:00
Sergei Trifonov
a5d7c2b6a2
Merge branch 'master' into async-loader-integration 2023-10-27 10:39:12 +02:00
serxa
08e63f2a24 fix build 2023-10-24 21:49:15 +00:00
Maksim Kita
0e176fa0ed Fixed code review issues 2023-10-24 19:39:25 +03:00
serxa
749cdb9954 Merge branch 'master' into async-loader-integration 2023-10-24 10:09:49 +00:00
Maksim Kita
ba37880208 Context added TSA 2023-10-23 12:00:29 +03:00
Kseniia Sumarokova
96c518be5b
Merge branch 'master' into s3-queue-fixes 2023-10-16 22:19:13 +02:00
robot-ch-test-poll1
aca98cf725
Merge pull request #53774 from helifu/master13 
Display reloaded values of server settings
 2023-10-16 21:17:17 +02:00
serxa
0e17da22eb Merge branch 'master' into async-loader-integration 2023-10-16 11:08:29 +00:00
kssenii
d64b990712 Merge remote-tracking branch 'origin/master' into s3-queue-fixes 2023-10-13 12:13:56 +02:00
robot-clickhouse-ci-1
c40a79f0b4
Merge pull request #54813 from lingtaolf/feature/function-getClientHttpHeader 
Add function getHttpHeader
 2023-10-12 10:13:20 +02:00
Igor Nikonov
62060a0603 Merge remote-tracking branch 'origin/master' into pr-coordinator-usage-cleanup 2023-10-11 15:11:15 +00:00
Igor Nikonov
9d95f4e1b6 Cleanup: parallel replica coordinator usage 2023-10-11 15:04:59 +00:00
helifu
ca0c4ba147
Merge branch 'ClickHouse:master' into master13 2023-10-10 13:49:49 +08:00
凌涛
51e2e59a97 Merge remote-tracking branch 'origin/master' into feature/function-getClientHttpHeader 2023-10-10 10:56:24 +08:00
alesapin
e55a615f06
Merge pull request #55253 from vitlibar/fix-async-insert-access-check-for-defaults 
Evaluate defaults during async insert safer
 2023-10-07 12:26:35 +02:00
Vitaly Baranov
76766a200e Fix access checking in evaluations of defaults during async insert. 2023-10-05 10:26:44 +02:00
Nikolai Kochetov
b67a0fe344
Merge pull request #55121 from kitaisreal/context-lock-contention-fix 
Context lock contention fix
 2023-10-03 11:17:59 +02:00
Maksim Kita
71cb6457de Fixed code review issues 2023-10-02 13:53:17 +03:00
Maksim Kita
07b9b5d7dd Context lock contention fix 2023-09-29 11:16:47 +03:00
Sema Checherinda
5fdca302e6 fix build, resolve process_list_elem race 2023-09-28 00:15:36 +02:00
Sema Checherinda
e629a17218 add cancelation point to s3 retries 2023-09-28 00:15:06 +02:00
凌涛
0d256a381a add doc 
fix tests

modify code stype and spelling

add tests and modify code style
 2023-09-27 17:03:53 +08:00
kssenii
3b54b6be88 Add system tables 2023-09-25 20:04:15 +02:00
凌涛
9e3c54ddb9 add function getHttpHeader 2023-09-20 10:07:02 +08:00
Igor Nikonov
572249a60c
Merge branch 'master' into pr-distributed-prefer-localhost-replica 2023-09-18 21:48:21 +02:00
avogar
7cfdd88ccf Address comments 2023-09-15 09:32:41 +00:00
avogar
b9c28ef1f7 Fix for analyzer 2023-09-14 20:10:07 +00:00
avogar
93c49dfeaa Fix using structure from insertion tables in case of defaults and explicit insert columns 2023-09-14 19:31:12 +00:00
Sergei Trifonov
12c71be9ca
Merge branch 'master' into async-loader-integration 2023-09-14 15:29:07 +02:00
Igor Nikonov
918bd814bd Add canUseParallelReplicas() for convenience 2023-09-11 12:19:07 +00:00
Sergei Trifonov
08bad4d440
Merge pull request #47009 from ClickHouse/disks-and-io-sheduler 2023-09-11 07:22:14 +02:00
serxa
d326554caa fix keeper build 2023-09-10 14:55:40 +00:00
Sergei Trifonov
258da91b5f
Merge branch 'master' into disks-and-io-sheduler 2023-09-04 15:14:28 +02:00
Victor Krasnov
78760639d2 Merge branch 'master' into ADQM-987 2023-09-04 13:59:20 +03:00
Alexey Milovidov
7cb4d52d92 Deprecate metadata cache, part 2 2023-09-03 03:56:11 +02:00
Victor Krasnov
52cce27e81 Merge branch 'master' into ADQM-987 2023-09-01 18:00:20 +03:00
kssenii
3f20d1f578 Rename 2023-09-01 12:55:58 +02:00
serxa
9c92d413b3 Merge branch 'master' into disks-and-io-sheduler 2023-08-30 11:18:49 +00:00
Sergei Trifonov
c10e42ca05
Merge branch 'master' into async-loader-integration 2023-08-28 20:20:31 +02:00
Victor Krasnov
921f0c8b5d Merge branch 'master' into ADQM-987 2023-08-25 21:20:23 +03:00
helifu
adfdba4b83 Display reloaded values of server settings 2023-08-24 12:32:30 +08:00
Robert Schulze
9d8bec4c73
Fix build, pt. II 2023-08-22 15:55:54 +00:00
Robert Schulze
e744654bae
More consistent parameter names 2023-08-22 15:48:48 +00:00
Victor Krasnov
258f9927b4 Merge branch 'master' into ADQM-987 2023-08-22 16:49:32 +03:00
Robert Schulze
f2fb7365e6
Add settings for 1. size_ratio (all caches) and 2. policy (index caches) 
1. is an expert-level setting, default is 0.5, applies only to SLRU.

Also, I noticed that we expose cache policy settings for the mark and
the uncompresed cache but not for the index mark and the index
uncompressed cache. Changed that as well, it simplifies the code a bit.
 2023-08-22 11:04:29 +00:00
Robert Schulze
583d9b33f0
Merge pull request #51446 from rschu1ze/recfgable-caches 
Runtime-reconfigurable cache sizes
 2023-08-21 23:27:21 +02:00
Robert Schulze
96c8b250ce
Allow to configure cache sizes at runtime 2023-08-21 13:51:10 +00:00
Victor Krasnov
a7813260c0 Merge branch 'master' into ADQM-987 2023-08-21 14:45:13 +03:00
kssenii
5382442da0 Merge remote-tracking branch 'upstream/master' into allowed-caches-dir-for-dynamic-disks 2023-08-19 08:44:36 +04:00
Nikolai Kochetov
e9d8745971
Merge branch 'master' into group_array_max_element_size 2023-08-18 15:23:01 +02:00
Nikolai Kochetov
88e81a31fa Add server setting group_array_max_element_size. 2023-08-18 13:15:27 +00:00
Robert Schulze
7ed5166c8a
Cosmetics: Make re-cfg of query cache similar to re-cfg of other caches 
This doesn't change semantics, it only changes in which places the
configuration is parsed. This makes the logic more consistent across
caches.
 2023-08-14 15:42:32 +00:00
Robert Schulze
3472a928d8
Cosmetics: Rename dropCache() to clearCaches() 
"dropCache()" is a bad name, it sounds like that the cache can no longer
be used after the call. Not true, we only "clear" the cache.
 2023-08-14 15:42:32 +00:00
Robert Schulze
fed995734e
Cosmetics: Move prefetch threadpool stuff out of "caching" section 2023-08-14 15:42:32 +00:00
Robert Schulze
364b43f19e
Cosmetics: add "cache" section marker 2023-08-14 14:49:40 +00:00
Victor Krasnov
247cd63ca1 Merge branch 'master' into ADQM-987 
# Conflicts:
#	src/Interpreters/Context.cpp
#	src/Interpreters/InterpreterSystemQuery.cpp
 2023-08-14 10:33:56 +03:00
kssenii
c3b8bc0267 Merge remote-tracking branch 'upstream/master' into allowed-caches-dir-for-dynamic-disks 2023-08-13 21:14:42 +04:00
Alexey Milovidov
3b9d2c7080 Simplification 2023-08-11 03:06:07 +02:00
Victor Krasnov
9c34ef899e Add backup_log table 2023-08-10 15:46:04 +03:00
kssenii
2cfb4c0264 Merge remote-tracking branch 'upstream/master' into allowed-caches-dir-for-dynamic-disks 2023-08-09 11:09:34 +02:00
kssenii
0c174a7ec1 Allow to restrict fs caches paths 2023-08-07 18:00:26 +02:00
Sergei Trifonov
432d359a2b
Merge branch 'master' into async-loader-integration 2023-08-04 18:51:06 +02:00
Vitaly Baranov
587877dba3
Merge branch 'master' into disable-fs-cache-for-backups 2023-08-04 01:15:23 +02:00
Vitaly Baranov
f427c779eb Disable using fs cache for backup/restore. 2023-08-03 13:52:32 +02:00
Vitaly Baranov
7a112459d3 Implement passing current roles to Context::setUser(). 2023-08-03 13:45:01 +02:00
Vitaly Baranov
16dfb028b6 Remove creation of a unnecessary temporary ContextAccess on login. 2023-08-03 13:44:59 +02:00
Sergei Trifonov
e4ae2ece7c
Merge branch 'master' into async-loader-integration 2023-08-01 13:04:11 +02:00
Alexey Gerasimchuck
e58b3cfd32 Implemented max sessions for user 2023-07-30 22:09:03 +00:00
pufit
6211845ef0
Merge pull request #51303 from ClickHouse/deprecate-metadata-cache 
Deprecate the `metadata_cache` feature
 2023-07-28 10:31:02 -04:00
Alexey Gerasimchuck
63b05da1f2 System logs improvements 2023-07-28 07:23:34 +00:00
Nikolay Degterinsky
d89e2e6a27
Add SYSTEM STOP LISTEN query (#51016) 
Co-authored-by: Nikita Mikhaylov <nikitamikhaylov@clickhouse.com>
Co-authored-by: Nikita Mikhaylov <mikhaylovnikitka@gmail.com>
 2023-07-26 19:58:41 +02:00
Sergei Trifonov
8eaf3b6d94
Merge branch 'master' into async-loader-integration 2023-07-21 21:46:06 +02:00
Amos Bird
8187118232
Better code 2023-07-21 16:47:23 +08:00
Amos Bird
53d77e6b13
Add back missing projection QueryAccessInfo. 2023-07-21 16:47:22 +08:00
Nikita Mikhaylov
668062dc29
Merge branch 'master' into no-keep-context-lock-while-calculating-access 2023-07-20 14:42:13 +02:00
Sergei Trifonov
b8a46ff822
Merge branch 'master' into async-loader-integration 2023-07-19 18:45:41 +02:00
Vitaly Baranov
815a3857de Remove non-const function Context::getClientInfo(). 2023-07-17 15:02:07 +02:00
Nikita Mikhaylov
01cd7a1fd7 Better 2023-07-17 12:59:58 +00:00
Vitaly Baranov
0e4b75a282 Avoid keeping lock Context::getLock() while calculating access rights when a user logs in. 2023-07-17 14:59:14 +02:00
Vitaly Baranov
018546a57d Avoid keeping lock Context::getLock() while recalculating access rights of a connected user. 2023-07-17 14:59:14 +02:00
Sergei Trifonov
bbb5850dec
Merge branch 'master' into async-loader-integration 2023-07-03 20:21:26 +02:00
Nikolay Degterinsky
82e0237e67
Merge branch 'master' into headers-blacklist 2023-07-03 16:54:50 +02:00
Nikolay Degterinsky
8dfa773f44
Merge branch 'master' into headers-blacklist 2023-06-30 23:40:17 +02:00
Antonio Andelic
2058b1346e Merge branch 'master' into keeper-with-disks 2023-06-30 13:22:33 +00:00
Sergei Trifonov
b9b007691c
Merge branch 'master' into async-loader-integration 2023-06-27 19:44:52 +02:00
Nikolay Degterinsky
9a25958be8 Add HTTP header filtering 2023-06-15 13:49:49 +00:00
serxa
e1045f76de rework async loader initialization and reconfiguration for bg/fg pools 2023-05-31 16:23:41 +00:00
Sergei Trifonov
9b64c6b9f3
Merge branch 'master' into async-loader-integration 2023-05-31 17:17:13 +02:00
Nikolai Kochetov
8cec00dd6e Merge branch 'master' into refactor-subqueries-for-in 2023-05-30 18:08:37 +02:00
Antonio Andelic
67746bfff2 Merge branch 'master' into keeper-with-disks 2023-05-30 09:50:12 +00:00
SmitaRKulkarni
f2172ab8dd
Merge pull request #46661 from ClickHouse/42192_Lower_parallel_parsing_threads_with_globs 
When reading from multiple files displace parallel parsing
 2023-05-30 08:53:08 +02:00
Sergei Trifonov
fe172f59db
Merge branch 'master' into async-loader-integration 2023-05-26 18:56:08 +02:00
Antonio Andelic
8a2a63a7bd Merge branch 'master' into keeper-with-disks 2023-05-26 10:48:41 +00:00
Nikolai Kochetov
b5b261b22c Merge branch 'master' into refactor-subqueries-for-in 2023-05-25 21:22:06 +02:00
Nikolai Kochetov
7ebe19f5fb Fixing more tests 2023-05-25 19:18:11 +00:00
helifu
2255b0287a Add 'partitions' field for system.query_log 2023-05-24 20:42:31 +08:00
Antonio Andelic
fa3fb49d33 Merge branch 'master' into keeper-with-disks 2023-05-24 09:04:47 +00:00
Antonio Andelic
161afea266 Add support for changelog 2023-05-24 07:43:57 +00:00
SmitaRKulkarni
55af60ea3f
Merge branch 'master' into 42192_Lower_parallel_parsing_threads_with_globs 2023-05-23 13:59:15 +02:00
Sergei Trifonov
994cfaf15b
Merge branch 'master' into async-loader-integration 2023-05-22 19:03:11 +02:00
Dmitry Novik
fc10ba871f Analyzer: Do not execute table functions multiple times 2023-05-22 16:36:34 +00:00
Antonio Andelic
bde2cf96b1 Better 2023-05-22 12:24:16 +00:00
Antonio Andelic
dc31396035 Merge branch 'master' into keeper-with-disks 2023-05-22 08:10:24 +00:00
Azat Khuzhin
b6cc504717 Remove Common/OpenTelemetryTraceContext.h from Context.h 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-05-21 23:04:33 +02:00
mateng915
5237dd0245
New system table zookeeper connection (#45245) 
* Feature: Support new system table to show which zookeeper node be connected

Description:
============
Currently we have no place to check which zk node be connected otherwise using
lsof command. It not convenient

Solution:
=========
Implemented a new system table, system.zookeeper_host when CK Server has zk
this table will show the zk node dir which connected by current CK server

Noted: This table can support multi-zookeeper cluster scenario.

* fixed review comments

* added test case

* update test cases

* remove unused code

* fixed review comments and removed unused code

* updated test cases for print host, port and is_expired

* modify the code comments

* fixed CI Failed

* fixed code style check failure

* updated test cases by added Tags

* update test reference

* update test cases

* added system.zookeeper_connection doc

* Update docs/en/operations/system-tables/zookeeper_connection.md

* Update docs/en/operations/system-tables/zookeeper_connection.md

* Update docs/en/operations/system-tables/zookeeper_connection.md

---------

Co-authored-by: Alexander Tokmakov <tavplubix@gmail.com>
 2023-05-19 17:06:43 +03:00
Antonio Andelic
bf6afd27a6 build with disks 2023-05-18 07:39:22 +00:00
serxa
6bbc061ba0 start loader, fix helpers, fix deadlock 2023-05-10 19:56:48 +00:00
SmitaRKulkarni
5c030c428c
Merge branch 'master' into 42192_Lower_parallel_parsing_threads_with_globs 2023-05-10 09:51:38 +02:00
Sergei Trifonov
b479637e2c
Merge branch 'master' into async-loader-integration 2023-05-09 13:04:36 +02:00
Kseniia Sumarokova
336bb41c5d
Merge branch 'master' into remove-dependency-from-context 2023-05-08 12:46:10 +02:00
Robert Schulze
3dfc0bd265
Merge pull request #49413 from azat/build/headers 
Slightly reduce inter-header dependencies
 2023-05-05 23:37:58 +02:00
kssenii
1433f5ffc9 Merge remote-tracking branch 'upstream/master' into remove-dependency-from-context 2023-05-04 13:24:02 +02:00
kssenii
ecfbf1e304 Remove dependency from DB::Context in readers 2023-05-02 21:45:27 +02:00
Azat Khuzhin
2b21b83979 Reduce inter-header dependencies 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-05-02 21:15:18 +02:00
Smita Kulkarni
8205398f31 Fixed comment 2023-05-02 16:31:39 +02:00
Sergei Trifonov
8dab49f328
Merge branch 'master' into async-loader-integration 2023-05-01 14:04:39 +02:00
Smita Kulkarni
0726db7d9a Renamed parameter to max_parsing_threads 2023-04-29 10:34:59 +02:00
Mike Kot
7bffa89e96 add all 3 settings 2023-04-28 13:22:36 +00:00
Mike Kot
e428af0b63 replace format settings with server setting 2023-04-27 21:27:18 +00:00
Mike Kot
b24de29c70 reverting mr changes 2023-04-27 20:37:12 +00:00
Mike Kot
357cde95e9 Merge branch 'master' into feature/show-secrets-option 2023-04-27 20:19:40 +00:00
serxa
3546699c95 wip async tables loading 2023-04-26 18:25:39 +00:00
Smita Kulkarni
b70878aa0e Merge branch 'master' into 42192_Lower_parallel_parsing_threads_with_globs 2023-04-24 19:39:36 +02:00
vdimir
b1f0e0d62f Lock clusters_mutex for cluster_discovery 2023-04-19 16:44:54 +02:00
vdimir
d6416a1321 Store clusters from ClusterDiscovery in separate map 2023-04-19 16:44:54 +02:00
Alexander Gololobov
02876ed87d Share PreparedSet-s between task of the same mutation to reduce mem usage 2023-04-14 16:12:18 +02:00
Dmitry Novik
5cc9b46f78 Merge remote-tracking branch 'origin/master' into optimize-compilation 2023-04-13 16:04:09 +02:00
Azat Khuzhin
011480924a Use forward declaration of ThreadPool 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-04-07 11:25:35 +02:00
Vladimir C
e7a5c96353
Merge pull request #48219 from ClickHouse/vdimir/lock_order_47072 2023-04-06 11:54:33 +02:00
robot-clickhouse
3d2c279859
Merge pull request #48242 from azat/better-throttling 
Per-query/per-server throttling for remote IO/local IO/BACKUPs
 2023-04-05 19:32:37 +02:00
vdimir
01ce3baf0f
Rename Context::getTemporaryVolume to getGlobalTemporaryVolume 2023-04-05 13:42:13 +00:00
Nikita Mikhaylov
7b526d3df9
Always communicate with local replica through loopback interface for parallel replicas (#48246) 2023-04-05 13:32:46 +02:00
vdimir
bae1286b63
Update temp_data_on_disk in shared context 2023-04-05 10:07:44 +00:00
Azat Khuzhin
c332d290d8 Keep only one throttler for BACKUPs IO (instead of separate read/write) 
There is no need in separate read/write throttling, because you cannot
write faster then read anyway, and plus this makes the code less cleaner

(and also it will allow avoid implementing throttling backups to S3,
since it does not use common S3 writer).

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-04-05 09:40:17 +02:00
Azat Khuzhin
218b1f9c29 Add ability to throttle BACKUPs on per-server/backup basis 
Server settings:
- backup_read_bandwidth_for_server
- backup_write_bandwidth_for_server

Query settings:
- backup_read_bandwidth
- backup_write_bandwidth

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-04-05 09:39:48 +02:00
Azat Khuzhin
a25dd1d348 Add ability to throttle local IO on per-query/server basis 
Server settings:
- max_local_read_bandwidth_for_server
- max_local_write_bandwidth_for_server

Query settings:
- max_local_read_bandwidth
- max_local_write_bandwidth

This is the preparation for adding ability to throttle BACKUPs

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-04-05 09:39:15 +02:00
Azat Khuzhin
b3406beeb7 Add per-query network throttling 
Controlled with:
- max_remote_read_network_bandwidth
- max_remote_write_network_bandwidth

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-04-05 09:39:15 +02:00
Azat Khuzhin
abdb682048 Move some methods for Context into module part 
This will be required later, for per-query bandwidth limiting.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-04-05 09:39:15 +02:00
Azat Khuzhin
40cb87c2ac Make remote bandwidth settings server settings 
Right now those two settings are applied only at start:
- max_remote_read_network_bandwidth_for_server
- max_remote_write_network_bandwidth_for_server

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-04-05 09:39:15 +02:00
SmitaRKulkarni
d9c67a3380
Merge branch 'master' into 42192_Lower_parallel_parsing_threads_with_globs 2023-03-30 09:35:03 +02:00
Dmitry Novik
3488a4d84b Improve file includes 2023-03-24 03:44:52 +01:00
Mike Kot
bf2670ddaf recalc rights after changing setting 2023-03-16 10:56:14 +00:00
Sergei Trifonov
d99a22ef95
Merge branch 'master' into disks-and-io-sheduler 2023-03-14 13:24:00 +01:00
Robert Schulze
657aa65446
Cleanup: Untangle CacheBase's constructors 
This prepares for the custom TTLCachePolicy which users should not be
able to configure explicitly.
 2023-03-13 15:11:12 +00:00
Antonio Andelic
737cf8e149 Better 2023-03-03 15:14:49 +00:00
serxa
2b90141c42 Integrate DiskObjectStorage with IO Scheduler 2023-02-28 13:17:31 +00:00