Commit Graph

216 Commits

Author SHA1 Message Date
alesapin
cbae29d26a
Merge pull request #57699 from hanfei1991/hanfei/keeper_soft_limit_ratio
support keeper memory soft limit ratio
2023-12-11 21:43:24 +01:00
Konstantin Bogdanov
771d5c58ea
Fix Keeper standalone build 2023-12-11 10:19:49 +01:00
Konstantin Bogdanov
89ba4a8452
Add HTTP readiness endpoint 2023-12-11 10:19:42 +01:00
MikhailBurdukov
119e451967 Merge branch 'master' into native_copy_for_s3_disks 2023-12-11 07:25:20 +00:00
Azat Khuzhin
55d5a3affa Correctly wait background threads
There are some places that make it possible to trigger use-after-free
from threads because some global variable had been destroyed before, for
example some logger.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-12-09 17:54:16 +01:00
Han Fei
881e58dfcb support soft limit ratio 2023-12-08 21:48:27 +01:00
Antonio Andelic
889c53eca0 Correctly handle unknown changelog versions and snapshot deleting 2023-11-28 09:47:49 +00:00
MikhailBurdukov
6f19e8ebd1
Merge branch 'master' into native_copy_for_s3_disks 2023-11-27 14:25:36 +03:00
Alexey Milovidov
b62e9d3a1a Address review comments 2023-11-23 18:13:12 +01:00
Alexey Milovidov
bd4ffec361 Better exit codes from the server 2023-11-23 17:00:57 +01:00
MikhailBurdukov
f29c072429 Test fix 2023-11-15 09:33:40 +00:00
Alexey Milovidov
82c461031e Fix build 2023-11-13 10:09:23 +01:00
Maksim Kita
9b60a3d7aa Keeper run without config fix 2023-10-27 20:53:07 +03:00
Nikolay Degterinsky
6539d52b9e
Merge pull request #50372 from helifu/master3
Support dynamic adjustment of log level in Keeper
2023-09-28 23:47:49 +02:00
Alexey Milovidov
ee088a0386 Fix bad punctuation in Keeper logs 2023-09-06 03:31:46 +02:00
kssenii
3f20d1f578 Rename 2023-09-01 12:55:58 +02:00
Arthur Passos
2bade7db08
Add global proxy setting (#51749)
* initial impl

* fix env ut

* move ut directory

* make sure no null proxy resolver is returned by ProxyConfigurationResolverProvider

* minor adjustment

* add a few tests, still incomplete

* add proxy support for url table function

* use proxy for select from url as well

* remove optional from return type, just returns empty config

* fix style

* style

* black

* ohg boy

* rm in progress file

* god pls don't let me kill anyone

* ...

* add use_aws guards

* remove hard coded s3 proxy resolver

* add concurrency-mt-unsafe

* aa

* black

* add logging back

* revert change

* imrpove code a bit

* helper functions and separate tests

* for some reason, this env test is not working..

* formatting

* :)

* clangtidy

* lint

* revert some stupid things

* small test adjusmtments

* simplify tests

* rename test

* remove extra line

* freaking style change

* simplify a bit

* fix segfault & remove an extra call

* tightly couple proxy provider with context..

* remove useless include

* rename config prefix parameter

* simplify provider a bit

* organize provider a bit

* add a few comments

* comment out proxy env tests

* fix nullptr in unit tests

* make sure old storage proxy config is properly covered without global context instance

* move a few functions from class to anonymous namespace

* fix no fallback for specific storage conf

* change API to accept http method instead of bool

* implement http/https distinction in listresolver, any still not implemented

* implement http/https distinction in remote resolver

* progress on code, improve tests and add url function working test

* use protcol instead of method for http and https

* small fix

* few more adjustments

* fix style

* black

* move enum to proxyconfiguration

* wip

* fix build

* fix ut

* delete atomicroundrobin class

* remove stale include

* add some tests.. need to spend some more time on the design..

* change design a bit

* progress

* use existing context for tests

* rename aux function and fix ut

* ..

* rename test

* try to simplify tests a bit

* simplify tests a bit more

* attempt to fix tests, accept more than one remote resolver

* use proper log id

* try waiting for resolver

* proper wait logic

* black

* empty

* address a few comments

* refactor tests

* remove old tests

* baclk

* use RAII to set/unset env

* black

* clang tidy

* fix env proxy not respecting any

* use log trace

* fix wrong logic in getRemoteREsolver

* fix wrong logic in getRemoteREsolver

* fix test

* remove unwanted code

* remove ClientConfigurationperRequest and auxilary classes

* remove unwanted code

* remove adapter test

* few adjustments and add test for s3 storage conf  with new proxy settings

* black

* use chassert for context

* Add getenv comment
2023-08-24 16:07:26 +03:00
helifu
7154cb0ed1 Support dynamic adjustment of log level 2023-08-23 19:42:03 +08:00
pufit
2e69a1d3a2 Fix keeper default path check 2023-08-17 22:07:39 -04:00
Alexander Tokmakov
ba44d7260e fix 2023-08-16 00:20:28 +02:00
Alexander Tokmakov
894ec8e0d0 less exceptions with runtime format string 2023-08-14 21:45:59 +02:00
Nikita Mikhaylov
dbe13e3016
Merge pull request #53110 from rmarduga/master
Add hints for HTTP handlers
2023-08-11 14:09:33 +02:00
Ruslan Mardugalliamov
5cdeacf4cf Add hints for HTTP handlers
Add hints to HTTP handlers to help users avoid
misspellings. For example, if a user mistakenly writes
`/dashboad` instead of `/dashboard`, they will now get a
hint that /dashboard is the correct handler.

This change will improve the user experience by making it
easier for users to find the correct handlers.

 #47662
2023-08-10 06:23:20 -04:00
Alexey Milovidov
fa9abc5038 Better usage of ccache 2023-08-09 05:02:50 +02:00
Vitaly Baranov
f55fe91d8b Throw an exception if wrong default path is going to be used. 2023-08-04 16:13:27 +02:00
Vitaly Baranov
89c99e4a6d Revert extracting the root keeper path from the 'path' key in the configuration ('path' must be reserved for clickhouse-server). 2023-08-04 15:55:26 +02:00
Vitaly Baranov
552331fd54 Use the same default paths for clickhouse_keeper (symlink) as for clickhouse_keeper (executable),
i.e. "/var/lib/clickhouse-keeper/.."
2023-08-04 13:38:21 +02:00
Nikolay Degterinsky
d89e2e6a27
Add SYSTEM STOP LISTEN query (#51016)
Co-authored-by: Nikita Mikhaylov <nikitamikhaylov@clickhouse.com>
Co-authored-by: Nikita Mikhaylov <mikhaylovnikitka@gmail.com>
2023-07-26 19:58:41 +02:00
Robert Schulze
d4737ca033
Merge pull request #50986 from arenadata/ADQM-822
Implement support of encrypted elements in configuration file
2023-07-26 12:27:04 +02:00
Roman Vasin
0af869ff0f
Merge branch 'master' into ADQM-822 2023-07-24 12:23:11 +03:00
Alexey Milovidov
00d6f2ee08 Use incbin for resources, part 1 2023-07-23 06:11:03 +02:00
Antonio Andelic
61dc020b4d
Merge branch 'master' into feature/keeper-dyn-reconf 2023-07-17 12:50:17 +02:00
Mike Kot
d0f81fb1cd fix 2023-07-11 13:33:07 +00:00
Mike Kot
3e8906e5a2 fix build for non-ssl systems 2023-07-11 12:23:10 +00:00
Mike Kot
471e5a056d fix 2023-07-11 10:40:55 +00:00
Roman Vasin
b6023d9f62 Merge branch 'master' of github.com:ClickHouse/ClickHouse into ADQM-822 2023-07-11 10:35:31 +00:00
Mike Kot
23e3c57a6a reload certificates for Keeper 2023-07-11 10:13:30 +00:00
Mike Kot
062b1c464c watch for certificate file updates in configreloader 2023-07-11 10:04:43 +00:00
Mike Kot
8b6376005a "reconfig" support for CH Keeper 2023-07-07 00:20:54 +03:00
Sergei Trifonov
4e099086cd
Merge branch 'master' into fix-shutdown-race 2023-07-03 11:15:34 +02:00
Antonio Andelic
2058b1346e Merge branch 'master' into keeper-with-disks 2023-06-30 13:22:33 +00:00
Sergei Trifonov
c8d6aa8b90
Merge branch 'master' into fix-shutdown-race 2023-06-30 15:12:21 +02:00
serxa
6515d52f60 fix2 2023-06-27 18:50:40 +00:00
Antonio Andelic
efbe01b6bb Merge branch 'master' into keeper-add-api-version-config 2023-06-27 10:11:04 +00:00
Andrey Zvonov
d4316c7189
Merge branch 'master' into zvonand-implicit-tz 2023-06-22 17:23:20 +02:00
Mikhail f. Shiryaev
4fb2dc8384
Decrease default settings for http headers name and value to 128K 2023-06-21 12:59:27 +02:00
Andrey Zvonov
07191ce10d
Merge branch 'master' into zvonand-implicit-tz 2023-06-20 00:24:53 +02:00
Roman Vasin
d55878d587
Merge branch 'master' into ADQM-822 2023-06-16 18:57:51 +03:00
Roman Vasin
f026cf17a3 Fix building with BUILD_STANDALONE_KEEPER 2023-06-16 15:26:58 +00:00
zvonand
112a3e6aca revert tz validation optimization 2023-06-16 12:19:01 +02:00
pufit
e01e0d53a9 Add embedded keeper-client to keeper standalone binary 2023-06-15 12:08:20 -04:00
Antonio Andelic
f9f1e870c8 Fix build 2023-06-14 10:09:01 +00:00
Antonio Andelic
a7e6264d56 Add backward compatibility 2023-06-12 13:12:05 +00:00
Antonio Andelic
b8b70d78e1 Build fix 2023-06-09 13:34:27 +00:00
Antonio Andelic
bea4e8e81f Add new files 2023-06-07 06:45:14 +00:00
Andrey Zvonov
87fc780c14
Merge branch 'master' into zvonand-implicit-tz 2023-05-30 13:13:46 +03:00
Antonio Andelic
0740bfbe4b Better disks moving 2023-05-26 14:31:09 +00:00
Antonio Andelic
092cf99147 Fix build 2023-05-24 13:45:14 +00:00
Antonio Andelic
fa3fb49d33 Merge branch 'master' into keeper-with-disks 2023-05-24 09:04:47 +00:00
Antonio Andelic
161afea266 Add support for changelog 2023-05-24 07:43:57 +00:00
Antonio Andelic
bde2cf96b1 Better 2023-05-22 12:24:16 +00:00
kssenii
5c48f96347 Fix build 2023-05-22 12:35:57 +02:00
zvonand
7dccb51bc0 fix conflict and update 2023-05-19 23:40:51 +02:00
kssenii
791bb6cd4c Fix style check 2023-05-19 17:35:01 +02:00
kssenii
0eab528f9f Move common code 2023-05-19 16:23:56 +02:00
Antonio Andelic
bf6afd27a6 build with disks 2023-05-18 07:39:22 +00:00
Alexey Milovidov
5a44dc26e7 Fixes for clang-17 2023-05-13 02:57:31 +02:00
Andrey Zvonov
9237b904be
Merge branch 'master' into zvonand-implicit-tz 2023-05-10 01:25:33 +02:00
zvonand
8338d54c34 Merge branch 'master' of github.com:ClickHouse/ClickHouse into zvonand-implicit-tz 2023-05-05 15:50:26 +02:00
kssenii
ecfbf1e304 Remove dependency from DB::Context in readers 2023-05-02 21:45:27 +02:00
Alexey Milovidov
98ae9be734
Revert "Added tests for ClickHouse apps help and fixed help issues" 2023-04-21 01:54:34 +03:00
Andrey Zvonov
984e8fde41
Merge branch 'master' into zvonand-implicit-tz 2023-04-20 23:13:17 +02:00
zvonand
542c09cb51 fix keeper standalone linking 2023-04-19 01:35:49 +02:00
zvonand
2a9f28b73b resolve 2023-04-12 12:54:39 +02:00
zvonand
a9499eed79 moved getting server TZ DateLUT to separate place, upd tests and fix 2023-04-12 12:47:05 +02:00
Yatsishin Ilya
b5b65d2149 Merge remote-tracking branch 'origin' into clickhouse-help 2023-04-11 11:24:48 +00:00
Alexey Milovidov
09ea79aaf7 Add support for {server_uuid} macro 2023-04-09 03:04:26 +02:00
Antonio Andelic
32cfc983b5 Fix clang-tidy build 2023-03-29 07:38:37 +00:00
Antonio Andelic
3b73f94eb8 Correctly set socket timeouts 2023-03-28 12:18:13 +00:00
Yatsishin Ilya
6cde37093a Merge remote-tracking branch 'origin/master' into clickhouse-help 2023-03-08 13:34:18 +00:00
Antonio Andelic
c38d820d0f Add 4LW for cleaning resource 2023-03-05 16:57:18 +00:00
Yatsishin Ilya
1baa15d603 Merge remote-tracking branch 'origin/master' into clickhouse-help 2023-02-07 15:46:59 +00:00
Mikhail f. Shiryaev
a42d20e19d
Create "coordination" directory in keeper if does not exist 2023-02-03 12:16:19 +01:00
Yatsishin Ilya
98edb9f06b Update help for clickhouse tools and add test 2023-01-31 12:19:37 +00:00
Alexander Tokmakov
3f6594f4c6 forbid old ctor of Exception 2023-01-23 22:18:05 +01:00
Alexander Tokmakov
70d1adfe4b
Better formatting for exception messages (#45449)
* save format string for NetException

* format exceptions

* format exceptions 2

* format exceptions 3

* format exceptions 4

* format exceptions 5

* format exceptions 6

* fix

* format exceptions 7

* format exceptions 8

* Update MergeTreeIndexGin.cpp

* Update AggregateFunctionMap.cpp

* Update AggregateFunctionMap.cpp

* fix
2023-01-24 00:13:58 +03:00
Antonio Andelic
fbda86ddc8 PR fixes 2022-11-30 13:24:08 +01:00
Antonio Andelic
a6f38cb1cc
Merge branch 'master' into keeper-prometheus 2022-11-24 12:12:08 +01:00
Antonio Andelic
e78761b66c
Merge branch 'master' into keeper-prometheus 2022-11-21 09:10:10 +01:00
Azat Khuzhin
eb3d21831d Provide full stacktrace in case of uncaught exception during server start
Without it, it is hard to understand where the problem is.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-11-18 13:24:57 +01:00
Azat Khuzhin
953457de28 Remove POCO_CLICKHOUSE_PATCH
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-11-10 22:41:26 +01:00
Antonio Andelic
8aeb406d82 Fix standalone build 2022-11-10 10:49:00 +01:00
Antonio Andelic
36d763011b Compile standalone keeper 2022-11-10 09:23:52 +01:00
Antonio Andelic
a0c12c2428 Define LOGICAL_ERROR 2022-11-09 16:06:25 +01:00
Antonio Andelic
1ed3930809 Add more metrics for Keeper 2022-11-09 15:51:41 +01:00
Antonio Andelic
c92ae5a385 Add suport for AsyncMetrics in standalone Keeper 2022-11-09 13:46:03 +01:00
Antonio Andelic
e82a5d43b5 Extract HTTPContext for HTTPServer 2022-11-09 09:02:04 +01:00
Azat Khuzhin
4e76629aaf Fixes for -Wshorten-64-to-32
- lots of static_cast
- add safe_cast
- types adjustments
  - config
  - IStorage::read/watch
  - ...
- some TODO's (to convert types in future)

P.S. That was quite a journey...

v2: fixes after rebase
v3: fix conflicts after #42308 merged
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-10-21 13:25:19 +02:00
Antonio Andelic
7c7395575f Merge branch 'master' into keeper-upload-snapshot-to-s3 2022-10-17 11:02:36 +00:00
Robert Schulze
fd86829824
Consolidate config_core.h into config.h
Less duplication, less confusion ...
2022-09-28 13:31:57 +00:00