Commit Graph

205 Commits

Author SHA1 Message Date
Alexey Milovidov
82c461031e Fix build 2023-11-13 10:09:23 +01:00
Maksim Kita
9b60a3d7aa Keeper run without config fix 2023-10-27 20:53:07 +03:00
Nikolay Degterinsky
6539d52b9e
Merge pull request #50372 from helifu/master3
Support dynamic adjustment of log level in Keeper
2023-09-28 23:47:49 +02:00
Alexey Milovidov
ee088a0386 Fix bad punctuation in Keeper logs 2023-09-06 03:31:46 +02:00
kssenii
3f20d1f578 Rename 2023-09-01 12:55:58 +02:00
Arthur Passos
2bade7db08
Add global proxy setting (#51749)
* initial impl

* fix env ut

* move ut directory

* make sure no null proxy resolver is returned by ProxyConfigurationResolverProvider

* minor adjustment

* add a few tests, still incomplete

* add proxy support for url table function

* use proxy for select from url as well

* remove optional from return type, just returns empty config

* fix style

* style

* black

* ohg boy

* rm in progress file

* god pls don't let me kill anyone

* ...

* add use_aws guards

* remove hard coded s3 proxy resolver

* add concurrency-mt-unsafe

* aa

* black

* add logging back

* revert change

* imrpove code a bit

* helper functions and separate tests

* for some reason, this env test is not working..

* formatting

* :)

* clangtidy

* lint

* revert some stupid things

* small test adjusmtments

* simplify tests

* rename test

* remove extra line

* freaking style change

* simplify a bit

* fix segfault & remove an extra call

* tightly couple proxy provider with context..

* remove useless include

* rename config prefix parameter

* simplify provider a bit

* organize provider a bit

* add a few comments

* comment out proxy env tests

* fix nullptr in unit tests

* make sure old storage proxy config is properly covered without global context instance

* move a few functions from class to anonymous namespace

* fix no fallback for specific storage conf

* change API to accept http method instead of bool

* implement http/https distinction in listresolver, any still not implemented

* implement http/https distinction in remote resolver

* progress on code, improve tests and add url function working test

* use protcol instead of method for http and https

* small fix

* few more adjustments

* fix style

* black

* move enum to proxyconfiguration

* wip

* fix build

* fix ut

* delete atomicroundrobin class

* remove stale include

* add some tests.. need to spend some more time on the design..

* change design a bit

* progress

* use existing context for tests

* rename aux function and fix ut

* ..

* rename test

* try to simplify tests a bit

* simplify tests a bit more

* attempt to fix tests, accept more than one remote resolver

* use proper log id

* try waiting for resolver

* proper wait logic

* black

* empty

* address a few comments

* refactor tests

* remove old tests

* baclk

* use RAII to set/unset env

* black

* clang tidy

* fix env proxy not respecting any

* use log trace

* fix wrong logic in getRemoteREsolver

* fix wrong logic in getRemoteREsolver

* fix test

* remove unwanted code

* remove ClientConfigurationperRequest and auxilary classes

* remove unwanted code

* remove adapter test

* few adjustments and add test for s3 storage conf  with new proxy settings

* black

* use chassert for context

* Add getenv comment
2023-08-24 16:07:26 +03:00
helifu
7154cb0ed1 Support dynamic adjustment of log level 2023-08-23 19:42:03 +08:00
pufit
2e69a1d3a2 Fix keeper default path check 2023-08-17 22:07:39 -04:00
Alexander Tokmakov
ba44d7260e fix 2023-08-16 00:20:28 +02:00
Alexander Tokmakov
894ec8e0d0 less exceptions with runtime format string 2023-08-14 21:45:59 +02:00
Nikita Mikhaylov
dbe13e3016
Merge pull request #53110 from rmarduga/master
Add hints for HTTP handlers
2023-08-11 14:09:33 +02:00
Ruslan Mardugalliamov
5cdeacf4cf Add hints for HTTP handlers
Add hints to HTTP handlers to help users avoid
misspellings. For example, if a user mistakenly writes
`/dashboad` instead of `/dashboard`, they will now get a
hint that /dashboard is the correct handler.

This change will improve the user experience by making it
easier for users to find the correct handlers.

 #47662
2023-08-10 06:23:20 -04:00
Alexey Milovidov
fa9abc5038 Better usage of ccache 2023-08-09 05:02:50 +02:00
Vitaly Baranov
f55fe91d8b Throw an exception if wrong default path is going to be used. 2023-08-04 16:13:27 +02:00
Vitaly Baranov
89c99e4a6d Revert extracting the root keeper path from the 'path' key in the configuration ('path' must be reserved for clickhouse-server). 2023-08-04 15:55:26 +02:00
Vitaly Baranov
552331fd54 Use the same default paths for clickhouse_keeper (symlink) as for clickhouse_keeper (executable),
i.e. "/var/lib/clickhouse-keeper/.."
2023-08-04 13:38:21 +02:00
Nikolay Degterinsky
d89e2e6a27
Add SYSTEM STOP LISTEN query (#51016)
Co-authored-by: Nikita Mikhaylov <nikitamikhaylov@clickhouse.com>
Co-authored-by: Nikita Mikhaylov <mikhaylovnikitka@gmail.com>
2023-07-26 19:58:41 +02:00
Robert Schulze
d4737ca033
Merge pull request #50986 from arenadata/ADQM-822
Implement support of encrypted elements in configuration file
2023-07-26 12:27:04 +02:00
Roman Vasin
0af869ff0f
Merge branch 'master' into ADQM-822 2023-07-24 12:23:11 +03:00
Alexey Milovidov
00d6f2ee08 Use incbin for resources, part 1 2023-07-23 06:11:03 +02:00
Antonio Andelic
61dc020b4d
Merge branch 'master' into feature/keeper-dyn-reconf 2023-07-17 12:50:17 +02:00
Mike Kot
d0f81fb1cd fix 2023-07-11 13:33:07 +00:00
Mike Kot
3e8906e5a2 fix build for non-ssl systems 2023-07-11 12:23:10 +00:00
Mike Kot
471e5a056d fix 2023-07-11 10:40:55 +00:00
Roman Vasin
b6023d9f62 Merge branch 'master' of github.com:ClickHouse/ClickHouse into ADQM-822 2023-07-11 10:35:31 +00:00
Mike Kot
23e3c57a6a reload certificates for Keeper 2023-07-11 10:13:30 +00:00
Mike Kot
062b1c464c watch for certificate file updates in configreloader 2023-07-11 10:04:43 +00:00
Mike Kot
8b6376005a "reconfig" support for CH Keeper 2023-07-07 00:20:54 +03:00
Sergei Trifonov
4e099086cd
Merge branch 'master' into fix-shutdown-race 2023-07-03 11:15:34 +02:00
Antonio Andelic
2058b1346e Merge branch 'master' into keeper-with-disks 2023-06-30 13:22:33 +00:00
Sergei Trifonov
c8d6aa8b90
Merge branch 'master' into fix-shutdown-race 2023-06-30 15:12:21 +02:00
serxa
6515d52f60 fix2 2023-06-27 18:50:40 +00:00
Antonio Andelic
efbe01b6bb Merge branch 'master' into keeper-add-api-version-config 2023-06-27 10:11:04 +00:00
Andrey Zvonov
d4316c7189
Merge branch 'master' into zvonand-implicit-tz 2023-06-22 17:23:20 +02:00
Mikhail f. Shiryaev
4fb2dc8384
Decrease default settings for http headers name and value to 128K 2023-06-21 12:59:27 +02:00
Andrey Zvonov
07191ce10d
Merge branch 'master' into zvonand-implicit-tz 2023-06-20 00:24:53 +02:00
Roman Vasin
d55878d587
Merge branch 'master' into ADQM-822 2023-06-16 18:57:51 +03:00
Roman Vasin
f026cf17a3 Fix building with BUILD_STANDALONE_KEEPER 2023-06-16 15:26:58 +00:00
zvonand
112a3e6aca revert tz validation optimization 2023-06-16 12:19:01 +02:00
pufit
e01e0d53a9 Add embedded keeper-client to keeper standalone binary 2023-06-15 12:08:20 -04:00
Antonio Andelic
f9f1e870c8 Fix build 2023-06-14 10:09:01 +00:00
Antonio Andelic
a7e6264d56 Add backward compatibility 2023-06-12 13:12:05 +00:00
Antonio Andelic
b8b70d78e1 Build fix 2023-06-09 13:34:27 +00:00
Antonio Andelic
bea4e8e81f Add new files 2023-06-07 06:45:14 +00:00
Andrey Zvonov
87fc780c14
Merge branch 'master' into zvonand-implicit-tz 2023-05-30 13:13:46 +03:00
Antonio Andelic
0740bfbe4b Better disks moving 2023-05-26 14:31:09 +00:00
Antonio Andelic
092cf99147 Fix build 2023-05-24 13:45:14 +00:00
Antonio Andelic
fa3fb49d33 Merge branch 'master' into keeper-with-disks 2023-05-24 09:04:47 +00:00
Antonio Andelic
161afea266 Add support for changelog 2023-05-24 07:43:57 +00:00
Antonio Andelic
bde2cf96b1 Better 2023-05-22 12:24:16 +00:00
kssenii
5c48f96347 Fix build 2023-05-22 12:35:57 +02:00
zvonand
7dccb51bc0 fix conflict and update 2023-05-19 23:40:51 +02:00
kssenii
791bb6cd4c Fix style check 2023-05-19 17:35:01 +02:00
kssenii
0eab528f9f Move common code 2023-05-19 16:23:56 +02:00
Antonio Andelic
bf6afd27a6 build with disks 2023-05-18 07:39:22 +00:00
Alexey Milovidov
5a44dc26e7 Fixes for clang-17 2023-05-13 02:57:31 +02:00
Andrey Zvonov
9237b904be
Merge branch 'master' into zvonand-implicit-tz 2023-05-10 01:25:33 +02:00
zvonand
8338d54c34 Merge branch 'master' of github.com:ClickHouse/ClickHouse into zvonand-implicit-tz 2023-05-05 15:50:26 +02:00
kssenii
ecfbf1e304 Remove dependency from DB::Context in readers 2023-05-02 21:45:27 +02:00
Alexey Milovidov
98ae9be734
Revert "Added tests for ClickHouse apps help and fixed help issues" 2023-04-21 01:54:34 +03:00
Andrey Zvonov
984e8fde41
Merge branch 'master' into zvonand-implicit-tz 2023-04-20 23:13:17 +02:00
zvonand
542c09cb51 fix keeper standalone linking 2023-04-19 01:35:49 +02:00
zvonand
2a9f28b73b resolve 2023-04-12 12:54:39 +02:00
zvonand
a9499eed79 moved getting server TZ DateLUT to separate place, upd tests and fix 2023-04-12 12:47:05 +02:00
Yatsishin Ilya
b5b65d2149 Merge remote-tracking branch 'origin' into clickhouse-help 2023-04-11 11:24:48 +00:00
Alexey Milovidov
09ea79aaf7 Add support for {server_uuid} macro 2023-04-09 03:04:26 +02:00
Antonio Andelic
32cfc983b5 Fix clang-tidy build 2023-03-29 07:38:37 +00:00
Antonio Andelic
3b73f94eb8 Correctly set socket timeouts 2023-03-28 12:18:13 +00:00
Yatsishin Ilya
6cde37093a Merge remote-tracking branch 'origin/master' into clickhouse-help 2023-03-08 13:34:18 +00:00
Antonio Andelic
c38d820d0f Add 4LW for cleaning resource 2023-03-05 16:57:18 +00:00
Yatsishin Ilya
1baa15d603 Merge remote-tracking branch 'origin/master' into clickhouse-help 2023-02-07 15:46:59 +00:00
Mikhail f. Shiryaev
a42d20e19d
Create "coordination" directory in keeper if does not exist 2023-02-03 12:16:19 +01:00
Yatsishin Ilya
98edb9f06b Update help for clickhouse tools and add test 2023-01-31 12:19:37 +00:00
Alexander Tokmakov
3f6594f4c6 forbid old ctor of Exception 2023-01-23 22:18:05 +01:00
Alexander Tokmakov
70d1adfe4b
Better formatting for exception messages (#45449)
* save format string for NetException

* format exceptions

* format exceptions 2

* format exceptions 3

* format exceptions 4

* format exceptions 5

* format exceptions 6

* fix

* format exceptions 7

* format exceptions 8

* Update MergeTreeIndexGin.cpp

* Update AggregateFunctionMap.cpp

* Update AggregateFunctionMap.cpp

* fix
2023-01-24 00:13:58 +03:00
Antonio Andelic
fbda86ddc8 PR fixes 2022-11-30 13:24:08 +01:00
Antonio Andelic
a6f38cb1cc
Merge branch 'master' into keeper-prometheus 2022-11-24 12:12:08 +01:00
Antonio Andelic
e78761b66c
Merge branch 'master' into keeper-prometheus 2022-11-21 09:10:10 +01:00
Azat Khuzhin
eb3d21831d Provide full stacktrace in case of uncaught exception during server start
Without it, it is hard to understand where the problem is.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-11-18 13:24:57 +01:00
Azat Khuzhin
953457de28 Remove POCO_CLICKHOUSE_PATCH
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-11-10 22:41:26 +01:00
Antonio Andelic
8aeb406d82 Fix standalone build 2022-11-10 10:49:00 +01:00
Antonio Andelic
36d763011b Compile standalone keeper 2022-11-10 09:23:52 +01:00
Antonio Andelic
a0c12c2428 Define LOGICAL_ERROR 2022-11-09 16:06:25 +01:00
Antonio Andelic
1ed3930809 Add more metrics for Keeper 2022-11-09 15:51:41 +01:00
Antonio Andelic
c92ae5a385 Add suport for AsyncMetrics in standalone Keeper 2022-11-09 13:46:03 +01:00
Antonio Andelic
e82a5d43b5 Extract HTTPContext for HTTPServer 2022-11-09 09:02:04 +01:00
Azat Khuzhin
4e76629aaf Fixes for -Wshorten-64-to-32
- lots of static_cast
- add safe_cast
- types adjustments
  - config
  - IStorage::read/watch
  - ...
- some TODO's (to convert types in future)

P.S. That was quite a journey...

v2: fixes after rebase
v3: fix conflicts after #42308 merged
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-10-21 13:25:19 +02:00
Antonio Andelic
7c7395575f Merge branch 'master' into keeper-upload-snapshot-to-s3 2022-10-17 11:02:36 +00:00
Robert Schulze
fd86829824
Consolidate config_core.h into config.h
Less duplication, less confusion ...
2022-09-28 13:31:57 +00:00
Robert Schulze
6d70b4a1f6
Generate config_version.h into ${CONFIG_INCLUDE_PATH}
This makes the target location consistent with other auto-generated
files like config_formats.h, config_core.h, and config_functions.h and
simplifies the build of clickhouse_common.
2022-09-28 12:48:26 +00:00
Robert Schulze
588a5e5a42
Simplify a bit 2022-09-27 07:47:18 +00:00
Robert Schulze
99725e68d1
Fix standalone keeper build 2022-09-26 19:28:27 +00:00
Robert Schulze
9711950c35
Fix build 2022-09-26 15:05:36 +00:00
Antonio Andelic
eefbbf53e8 Extract S3 logic 2022-09-21 11:53:54 +00:00
Dmitry Novik
29b5d023ee Replace WITH_TEXT_LOG by WITHOUT_TEXT_LOG 2022-09-08 19:16:37 +02:00
Robert Schulze
1a7727a254
Prefix overridden add_executable() command with "clickhouse_"
A simple HelloWorld program with zero includes except iostream triggers
a build of ca. 2000 source files. The reason is that ClickHouse's
top-level CMakeLists.txt overrides "add_executable()" to link all
binaries against "clickhouse_new_delete". This links against
"clickhouse_common_io", which in turn has lots of 3rd party library
dependencies ... Without linking "clickhouse_new_delete", the number of
compiled files for "HelloWorld" goes down to ca. 70.

As an example, the self-extracting-executable needs none of its current
dependencies but other programs may also benefit.

In order to restore access to the original "add_executable()", the
overriding version is now prefixed. There is precedence for a
"clickhouse_" prefix (as opposed to "ch_"), for example
"clickhouse_split_debug_symbols". In general prefixing makes sense also
because overriding CMake commands relies on undocumented behavior and is
considered not-so-great practice (*).

(*) https://crascit.com/2018/09/14/do-not-redefine-cmake-commands/
2022-07-11 19:36:18 +02:00
Antonio Andelic
37f799550b
Merge pull request #38072 from lingpeng0314/master
Add Keeper related monitoring data
2022-07-10 09:39:57 +02:00
lingpeng0314
cca94f01ae add function tryGetKeeperDispatcher for context
- to make sure other case will get error if dispatcher not initialized
2022-07-04 13:35:08 +08:00
lingpeng0314
03a5ce542d Remove Exception in getKeeperDispatcher
- to handle the nil case by caller
2022-07-01 11:27:40 +08:00
Robert Schulze
bb358617e1
Better naming for stuff related to splitted debug symbols
The previous name was slightly misleading, e.g. it is not about
"intalling stripped binaries" but about splitting debug symbols from the
binary.
2022-06-30 23:41:27 +02:00