thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-11-24 00:22:29 +00:00
Code Issues Packages Projects Releases Wiki Activity
130,472 Commits 9,167 Branches 1,781 Tags 2.1 GiB
8fe2cd1a7e
Commit Graph

912 Commits

Author SHA1 Message Date
vdimir
b9bb4af6be
Support setting up fail points in config 2023-08-31 11:45:39 +00:00
Sergei Trifonov
c10e42ca05
Merge branch 'master' into async-loader-integration 2023-08-28 20:20:31 +02:00
Artur Malchanau
41508e129d
Merge branch 'ClickHouse:master' into arm-clocksource 2023-08-28 10:50:57 +03:00
Artur Malchanau
97cdd0c276
Fix clang-tidy warning 2023-08-23 15:35:30 +03:00
Artur Malchanau
0fd26152e3
Do not warn about arch_sys_counter clock 2023-08-23 12:51:27 +03:00
Robert Schulze
f2fb7365e6
Add settings for 1. size_ratio (all caches) and 2. policy (index caches) 
1. is an expert-level setting, default is 0.5, applies only to SLRU.

Also, I noticed that we expose cache policy settings for the mark and
the uncompresed cache but not for the index mark and the index
uncompressed cache. Changed that as well, it simplifies the code a bit.
 2023-08-22 11:04:29 +00:00
Robert Schulze
583d9b33f0
Merge pull request #51446 from rschu1ze/recfgable-caches 
Runtime-reconfigurable cache sizes
 2023-08-21 23:27:21 +02:00
Nikita Mikhaylov
145c99ee94
Merge pull request #53280 from evillique/better-stop-listen 
Add EXCEPT clause to SYSTEM STOP LISTEN query
 2023-08-21 19:03:49 +02:00
Robert Schulze
96c8b250ce
Allow to configure cache sizes at runtime 2023-08-21 13:51:10 +00:00
kssenii
5382442da0 Merge remote-tracking branch 'upstream/master' into allowed-caches-dir-for-dynamic-disks 2023-08-19 08:44:36 +04:00
robot-ch-test-poll
ce58b90ea1
Merge pull request #53455 from rschu1ze/respect-changed-cgroups-limit 
Respect cgroup memory limit when reloading configuration
 2023-08-18 00:03:35 +02:00
Alexander Tokmakov
89c365803f
Merge pull request #53424 from ClickHouse/exception_message_patterns6 
Less exceptions with runtime format string
 2023-08-17 17:58:04 +03:00
Robert Schulze
5253a7f650
Respect cgroup memory limit when reloading configuration 
Cgroups allows to change the amount of memory available to a process
while it runs. The previous logic calculated the amount of available
memory only once at server startup. As a result, memory thresholds set
via cgroups were not picked up when the settings changed. We now always
incorporate the current limits during re-configuraton.

Note 1: getMemoryAmount() opens/reads a file which is potentially
        expensive. Should be fine though since that happens only when
        the server configuration changes.

Note 2: An better approach would be to treat cgroup limit changes as
        another trigger for ClickHouse server re-configuration (which
        currently only happens when the config files change). Shied away
        from that for now because of the case that when the cgroup limit
        is lowered, there is no guarantee that ClickHouse can shrink the
        memory amount accordingly in time (afaik, it does so only lazily
        by denying new allocations). As a result, the OOM killer would
        kill the server. The same will happen with this PR but at a
        lower implementation complexity.
 2023-08-17 09:22:13 +00:00
Nikolay Degterinsky
a893f75884 Merge remote-tracking branch 'upstream/master' into better-stop-listen 2023-08-16 23:06:18 +00:00
Alexander Tokmakov
894ec8e0d0 less exceptions with runtime format string 2023-08-14 21:45:59 +02:00
Robert Schulze
7ed5166c8a
Cosmetics: Make re-cfg of query cache similar to re-cfg of other caches 
This doesn't change semantics, it only changes in which places the
configuration is parsed. This makes the logic more consistent across
caches.
 2023-08-14 15:42:32 +00:00
Robert Schulze
f126e54b0d
Respect max_cache_size for all caches 
So far, max_cache_size was only respected in Server.cpp, and in this
file only for the mark cache and the uncompressed cache. This commit
respects it also for the other cache types in Server.cpp, as well for
all caches in LocalServer.cpp.
 2023-08-14 15:42:32 +00:00
Robert Schulze
54e54aed74
Cosmetics: Introduce constants for default cache sizes and policies 2023-08-14 15:42:32 +00:00
kssenii
c3b8bc0267 Merge remote-tracking branch 'upstream/master' into allowed-caches-dir-for-dynamic-disks 2023-08-13 21:14:42 +04:00
Kseniia Sumarokova
ceb74ad3cb
Merge branch 'master' into allowed-caches-dir-for-dynamic-disks 2023-08-11 16:49:02 +02:00
Alexey Milovidov
a55f9eae84 Merge branch 'master' of github.com:ClickHouse/ClickHouse into allow-creating-system-tables-at-startup 2023-08-11 14:45:03 +02:00
Nikolay Degterinsky
bcc0fbbf91 Add EXCEPT clause to SYSTEM STOP LISTEN query 2023-08-10 16:10:33 +00:00
kssenii
2cfb4c0264 Merge remote-tracking branch 'upstream/master' into allowed-caches-dir-for-dynamic-disks 2023-08-09 11:09:34 +02:00
Alexey Milovidov
fa9abc5038 Better usage of ccache 2023-08-09 05:02:50 +02:00
Alexey Milovidov
e1206dbaab Add a comment from @tavplubix 2023-08-07 18:57:30 +02:00
kssenii
0c174a7ec1 Allow to restrict fs caches paths 2023-08-07 18:00:26 +02:00
Alexey Milovidov
167f90e7f6 Maybe better 2023-08-06 13:28:21 +02:00
Sergei Trifonov
432d359a2b
Merge branch 'master' into async-loader-integration 2023-08-04 18:51:06 +02:00
Alexander Tokmakov
67d8f1c1c5 fix data race 2023-08-02 18:42:38 +02:00
Sergei Trifonov
e4ae2ece7c
Merge branch 'master' into async-loader-integration 2023-08-01 13:04:11 +02:00
Alexey Milovidov
a7b4a29ed9
Merge branch 'master' into fixes-start-stop-listen 2023-07-31 08:34:17 +03:00
alesapin
aba291ca68
Revert "Revert "Add an ability to specify allocations size for sampling memory profiler"" 2023-07-30 14:01:34 +02:00
Alexey Milovidov
5d2fb786f6
Merge branch 'master' into fixes-start-stop-listen 2023-07-29 00:39:58 +03:00
pufit
6211845ef0
Merge pull request #51303 from ClickHouse/deprecate-metadata-cache 
Deprecate the `metadata_cache` feature
 2023-07-28 10:31:02 -04:00
Nikolay Degterinsky
b5160c8072 Fix bugs and better test for SYSTEM STOP LISTEN 2023-07-27 14:16:19 +00:00
Nikolay Degterinsky
d89e2e6a27
Add SYSTEM STOP LISTEN query (#51016) 
Co-authored-by: Nikita Mikhaylov <nikitamikhaylov@clickhouse.com>
Co-authored-by: Nikita Mikhaylov <mikhaylovnikitka@gmail.com>
 2023-07-26 19:58:41 +02:00
Alexander Tokmakov
6bbed6262e
Merge branch 'master' into add_delay_for_replicated 2023-07-26 12:48:48 +03:00
Anton Popov
4531b105ac
Merge branch 'master' into flush-async-insert-queue 2023-07-25 15:48:01 +02:00
Alexander Tokmakov
faca49a905 Merge branch 'master' into add_delay_for_replicated 2023-07-24 16:07:38 +02:00
Anton Popov
347f442f84 Merge remote-tracking branch 'upstream/master' into HEAD 2023-07-24 10:55:31 +00:00
Alexey Milovidov
a17652a422 Merge branch 'master' into incbin 2023-07-24 05:33:52 +02:00
robot-ch-test-poll4
fb045b0d86
Merge pull request #52498 from ClickHouse/dont_shutdown_interserver_before_tables 
Don't shutdown interserver before tables
 2023-07-24 01:45:25 +02:00
Alexey Milovidov
1e467867e6 Attempt to fix LTO 2023-07-24 00:03:40 +02:00
alesapin
753a8d0a43 Merge branch 'dont_shutdown_interserver_before_tables' into add_delay_for_replicated 2023-07-23 22:41:15 +02:00
alesapin
e02948580b Don't shutdown interserver before tables 2023-07-23 22:38:59 +02:00
Alexey Milovidov
43bd6d1b83
Revert "Add an ability to specify allocations size for sampling memory profiler" 2023-07-23 19:00:49 +03:00
Alexey Milovidov
00d6f2ee08 Use incbin for resources, part 1 2023-07-23 06:11:03 +02:00
alesapin
d1fa105d60
Merge branch 'master' into add_delay_for_replicated 2023-07-23 00:12:29 +02:00
alesapin
835c36cbf3
Merge pull request #52419 from ClickHouse/implement_borders_for_sampling_memory_profiler 
Add an ability to specify allocations size for sampling memory profiler
 2023-07-22 18:13:26 +02:00
alesapin
6416fb6eed
Merge branch 'master' into add_delay_for_replicated 2023-07-22 12:11:39 +02:00
Alexander Sapin
4695ec6802 Add an ability to specify allocations size for sampling memory profiler 2023-07-21 14:56:29 +02:00
Dmitry Kardymon
5124f74fcc Merge remote-tracking branch 'origin/master' into ADQM-996 2023-07-21 05:54:36 +00:00
alesapin
cbcd48979c Fix race one more time 2023-07-20 13:04:43 +02:00
serxa
02fc66db10 avoid using bg pool during non-async start 2023-07-19 17:37:53 +00:00
Sergei Trifonov
b8a46ff822
Merge branch 'master' into async-loader-integration 2023-07-19 18:45:41 +02:00
serxa
6bca79fdff add auto-resolution for all resolvable deadlock types 2023-07-19 16:19:28 +00:00
alesapin
8be1454b28 Fix race 2023-07-18 16:06:03 +02:00
Sergei Trifonov
850ab23991
Merge branch 'master' into async-loader-integration 2023-07-17 19:14:54 +02:00
Nikita Mikhaylov
ee0bbc0e54
Merge branch 'master' into headers-blacklist 2023-07-17 19:08:52 +02:00
Nikita Mikhaylov
11cc8b4adf
Merge branch 'master' into add_delay_for_replicated 2023-07-17 19:04:29 +02:00
Nikita Mikhaylov
d5b97e4cc1
Merge branch 'master' into deprecate-metadata-cache 2023-07-17 15:00:27 +02:00
Dmitry Kardymon
771b1f8f47 Prototype (working) 2023-07-12 15:48:09 +00:00
Mike Kot
3e8906e5a2 fix build for non-ssl systems 2023-07-11 12:23:10 +00:00
Mike Kot
062b1c464c watch for certificate file updates in configreloader 2023-07-11 10:04:43 +00:00
Azat Khuzhin
a10aa9ad50 Force libunwind usage (removes gcc_eh support) 
libunwind is reentrant and signal safe, and works faster then then
gcc_eh (plus it has some custom patches for problems that have been
found during it's usage in ClickHouse).

gcc_eh may be missing in the system (if gcc was not installed), and
even if it exists clickhouse uses -nodefaultlibs, so some care should be
made to make it work.

Also this library is tiny and there shouln't be any problem to require
it always (there is already tendency to require some contrib libraries,
i.e. poco).

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-07-08 20:55:50 +02:00
Alexey Milovidov
e4dd603919
Merge pull request #51497 from ClickHouse/fix-shutdown-race 
Fix possible race on shutdown wait
 2023-07-08 03:03:45 +03:00
Alexander Sapin
e618dd05cc Fix clang tidy and race 2023-07-07 13:03:44 +02:00
Alexey Milovidov
e303afbffb
Merge branch 'master' into fix-incorrect-warning 2023-07-07 13:44:21 +03:00
Nikolay Degterinsky
e98d136243
Merge branch 'master' into headers-blacklist 2023-07-07 04:44:06 +02:00
Alexey Milovidov
5416b7b6df Fix incorrect log level = warning 2023-07-06 03:04:58 +02:00
alesapin
88d3e1723a Fixes and comments 2023-07-05 18:31:47 +02:00
alesapin
baee73fd96 Make shutdown of replicated tables softer 2023-07-05 18:11:25 +02:00
Alexey Milovidov
31f68c2c67 Merge branch 'master' into deprecate-metadata-cache 2023-07-04 22:28:14 +02:00
Nikolay Degterinsky
8dfa773f44
Merge branch 'master' into headers-blacklist 2023-06-30 23:40:17 +02:00
serxa
40f721ae4f fix possible race on shutdown wait 2023-06-27 17:14:33 +00:00
Sergei Trifonov
e2798b5221
Merge branch 'master' into async-loader-integration 2023-06-26 19:18:23 +02:00
Azat Khuzhin
5d43a64112 Initialize text_log earlier to capture table startup messages 
While I was investigating some issues, I noticed that messages from
table startup are not appears in system.text_log due to too late
initialization.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-06-24 13:16:16 +02:00
Alexey Milovidov
a92513048d Add a warning 2023-06-23 02:36:00 +02:00
Sergei Trifonov
02e0d7e204
Merge branch 'master' into async-loader-integration 2023-06-20 12:03:29 +02:00
Andrey Zvonov
07191ce10d
Merge branch 'master' into zvonand-implicit-tz 2023-06-20 00:24:53 +02:00
Nikolay Degterinsky
9a25958be8 Add HTTP header filtering 2023-06-15 13:49:49 +00:00
Alexey Milovidov
598501011f Relax "too many parts" further 2023-06-11 17:51:54 +02:00
Andrey Zvonov
d95be4162f
Merge branch 'master' into zvonand-implicit-tz 2023-06-08 18:34:45 +03:00
Sergei Trifonov
0cbcff958a
Merge branch 'master' into async-loader-integration 2023-06-07 18:55:04 +02:00
Anton Popov
9cd0d5e6db move settings to server_settings 2023-06-07 11:49:41 +00:00
Anton Popov
c980771c22 Merge remote-tracking branch 'upstream/master' into HEAD 2023-06-07 11:23:13 +00:00
Nikita Mikhaylov
e87348010d
Rework loading and removing of data parts for MergeTree tables. (#49474) 
Co-authored-by: Sergei Trifonov <sergei@clickhouse.com>
 2023-06-06 14:42:56 +02:00
Sergei Trifonov
3ee782a0ec
Merge branch 'master' into async-loader-integration 2023-06-02 18:27:19 +02:00
serxa
aceca2e210 fix server 2023-06-02 16:24:56 +00:00
Nikita Mikhaylov
73db383727
Limit the number of in-flight tasks for loading outdated parts (#50450) 
* Done

* Update programs/local/LocalServer.cpp

Co-authored-by: Alexander Tokmakov <tavplubix@clickhouse.com>

* Bump

---------

Co-authored-by: Alexander Tokmakov <tavplubix@clickhouse.com>
 2023-06-02 14:10:26 +02:00
serxa
5048050085 use pool ids instead of priorities in load jobs 2023-05-31 16:53:10 +00:00
serxa
e1045f76de rework async loader initialization and reconfiguration for bg/fg pools 2023-05-31 16:23:41 +00:00
Andrey Zvonov
87fc780c14
Merge branch 'master' into zvonand-implicit-tz 2023-05-30 13:13:46 +03:00
Sergei Trifonov
994cfaf15b
Merge branch 'master' into async-loader-integration 2023-05-22 19:03:11 +02:00
zvonand
7dccb51bc0 fix conflict and update 2023-05-19 23:40:51 +02:00
kssenii
791bb6cd4c Fix style check 2023-05-19 17:35:01 +02:00
kssenii
0eab528f9f Move common code 2023-05-19 16:23:56 +02:00
Anton Popov
69ec65ab33 Merge remote-tracking branch 'upstream/master' into HEAD 2023-05-16 10:30:47 +00:00
Azat Khuzhin
424a20ee1c Log with warning if the server was terminated forcefully 
In case of it is terminated forcefully it will not be terminated
gracefully (i.e. run dtors and stuff), and by using warning log level
those messages will go to clickhouse-server.err.log, in which messages
are kept for a longer period then in clickhouse-server.log (at least
because it contains only warnings, errors and fatals only).

This will help with investigating some obscure issues.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-05-15 14:20:42 +02:00
serxa
9bf5292415 refact 2023-05-12 19:49:47 +00:00