From 01cf460287e51b9f5feb6b569087c201b31f018c Mon Sep 17 00:00:00 2001 From: zvonand Date: Mon, 22 Feb 2021 21:18:59 +0500 Subject: [PATCH 1/4] New file --- .../external-authenticators/index.md | 1 + .../external-authenticators/kerberos.md | 150 ++++++++++++++++++ .../external-authenticators/ldap.md | 3 +- 3 files changed, 152 insertions(+), 2 deletions(-) create mode 100644 docs/en/operations/external-authenticators/kerberos.md diff --git a/docs/en/operations/external-authenticators/index.md b/docs/en/operations/external-authenticators/index.md index 95f80f192f5..34f58e52602 100644 --- a/docs/en/operations/external-authenticators/index.md +++ b/docs/en/operations/external-authenticators/index.md @@ -11,3 +11,4 @@ ClickHouse supports authenticating and managing users using external services. The following external authenticators and directories are supported: - [LDAP](./ldap.md#external-authenticators-ldap) [Authenticator](./ldap.md#ldap-external-authenticator) and [Directory](./ldap.md#ldap-external-user-directory) +- [Kerberos](./kerberos.md#external-authenticators-kerberos) diff --git a/docs/en/operations/external-authenticators/kerberos.md b/docs/en/operations/external-authenticators/kerberos.md new file mode 100644 index 00000000000..35d9cc942df --- /dev/null +++ b/docs/en/operations/external-authenticators/kerberos.md @@ -0,0 +1,150 @@ +# Kerberos {#external-authenticators-kerberos} + +Existing and properly configured ClickHouse users can be authenticated via Kerberos authentication protocol. + +## LDAP Server Definition {#ldap-server-definition} + +To define LDAP server you must add `ldap_servers` section to the `config.xml`. For example, + +```xml + + + + + localhost + 636 + uid={user_name},ou=users,dc=example,dc=com + 300 + yes + tls1.2 + demand + /path/to/tls_cert_file + /path/to/tls_key_file + /path/to/tls_ca_cert_file + /path/to/tls_ca_cert_dir + ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:AES256-GCM-SHA384 + + + +``` + +Note, that you can define multiple LDAP servers inside the `ldap_servers` section using distinct names. + +Parameters: + +- `host` - LDAP server hostname or IP, this parameter is mandatory and cannot be empty. +- `port` - LDAP server port, default is `636` if `enable_tls` is set to `true`, `389` otherwise. +- `bind_dn` - template used to construct the DN to bind to. + - The resulting DN will be constructed by replacing all `{user_name}` substrings of the + template with the actual user name during each authentication attempt. +- `verification_cooldown` - a period of time, in seconds, after a successful bind attempt, + during which the user will be assumed to be successfully authenticated for all consecutive + requests without contacting the LDAP server. + - Specify `0` (the default) to disable caching and force contacting the LDAP server for each authentication request. +- `enable_tls` - flag to trigger use of secure connection to the LDAP server. + - Specify `no` for plain text `ldap://` protocol (not recommended). + - Specify `yes` for LDAP over SSL/TLS `ldaps://` protocol (recommended, the default). + - Specify `starttls` for legacy StartTLS protocol (plain text `ldap://` protocol, upgraded to TLS). +- `tls_minimum_protocol_version` - the minimum protocol version of SSL/TLS. + - Accepted values are: `ssl2`, `ssl3`, `tls1.0`, `tls1.1`, `tls1.2` (the default). +- `tls_require_cert` - SSL/TLS peer certificate verification behavior. + - Accepted values are: `never`, `allow`, `try`, `demand` (the default). +- `tls_cert_file` - path to certificate file. +- `tls_key_file` - path to certificate key file. +- `tls_ca_cert_file` - path to CA certificate file. +- `tls_ca_cert_dir` - path to the directory containing CA certificates. +- `tls_cipher_suite` - allowed cipher suite (in OpenSSL notation). + +## LDAP External Authenticator {#ldap-external-authenticator} + +A remote LDAP server can be used as a method for verifying passwords for locally defined users (users defined in `users.xml` or in local access control paths). In order to achieve this, specify previously defined LDAP server name instead of `password` or similar sections in the user definition. + +At each login attempt, ClickHouse will try to "bind" to the specified DN defined by the `bind_dn` parameter in the [LDAP server definition](#ldap-server-definition) using the provided credentials, and if successful, the user will be considered authenticated. This is often called a "simple bind" method. + +For example, + +```xml + + + + + + + + my_ldap_server + + + + +``` + +Note, that user `my_user` refers to `my_ldap_server`. This LDAP server must be configured in the main `config.xml` file as described previously. + +When SQL-driven [Access Control and Account Management](../access-rights.md#access-control) is enabled in ClickHouse, users that are authenticated by LDAP servers can also be created using the [CRATE USER](../../sql-reference/statements/create/user.md#create-user-statement) statement. + + +```sql +CREATE USER my_user IDENTIFIED WITH ldap_server BY 'my_ldap_server' +``` + +## LDAP Exernal User Directory {#ldap-external-user-directory} + +In addition to the locally defined users, a remote LDAP server can be used as a source of user definitions. In order to achieve this, specify previously defined LDAP server name (see [LDAP Server Definition](#ldap-server-definition)) in the `ldap` section inside the `users_directories` section of the `config.xml` file. + +At each login attempt, ClickHouse will try to find the user definition locally and authenticate it as usual, but if the user is not defined, ClickHouse will assume it exists in the external LDAP directory, and will try to "bind" to the specified DN at the LDAP server using the provided credentials. If successful, the user will be considered existing and authenticated. The user will be assigned roles from the list specified in the `roles` section. Additionally, LDAP "search" can be performed and results can be transformed and treated as role names and then be assigned to the user if the `role_mapping` section is also configured. All this implies that the SQL-driven [Access Control and Account Management](../access-rights.md#access-control) is enabled and roles are created using the [CREATE ROLE](../../sql-reference/statements/create/role.md#create-role-statement) statement. + +Example (goes into `config.xml`): + +```xml + + + + + + my_ldap_server + + + + + + ou=groups,dc=example,dc=com + subtree + (&(objectClass=groupOfNames)(member={bind_dn})) + cn + clickhouse_ + + + + +``` + +Note that `my_ldap_server` referred in the `ldap` section inside the `user_directories` section must be a previously +defined LDAP server that is configured in the `config.xml` (see [LDAP Server Definition](#ldap-server-definition)). + +Parameters: + +- `server` - one of LDAP server names defined in the `ldap_servers` config section above. + This parameter is mandatory and cannot be empty. +- `roles` - section with a list of locally defined roles that will be assigned to each user retrieved from the LDAP server. + - If no roles are specified here or assigned during role mapping (below), user will not be able + to perform any actions after authentication. +- `role_mapping` - section with LDAP search parameters and mapping rules. + - When a user authenticates, while still bound to LDAP, an LDAP search is performed using `search_filter` + and the name of the logged in user. For each entry found during that search, the value of the specified + attribute is extracted. For each attribute value that has the specified prefix, the prefix is removed, + and the rest of the value becomes the name of a local role defined in ClickHouse, + which is expected to be created beforehand by the [CREATE ROLE](../../sql-reference/statements/create/role.md#create-role-statement) statement. + - There can be multiple `role_mapping` sections defined inside the same `ldap` section. All of them will be applied. + - `base_dn` - template used to construct the base DN for the LDAP search. + - The resulting DN will be constructed by replacing all `{user_name}` and `{bind_dn}` + substrings of the template with the actual user name and bind DN during each LDAP search. + - `scope` - scope of the LDAP search. + - Accepted values are: `base`, `one_level`, `children`, `subtree` (the default). + - `search_filter` - template used to construct the search filter for the LDAP search. + - The resulting filter will be constructed by replacing all `{user_name}`, `{bind_dn}`, and `{base_dn}` + substrings of the template with the actual user name, bind DN, and base DN during each LDAP search. + - Note, that the special characters must be escaped properly in XML. + - `attribute` - attribute name whose values will be returned by the LDAP search. + - `prefix` - prefix, that will be expected to be in front of each string in the original + list of strings returned by the LDAP search. Prefix will be removed from the original + strings and resulting strings will be treated as local role names. Empty, by default. diff --git a/docs/en/operations/external-authenticators/ldap.md b/docs/en/operations/external-authenticators/ldap.md index 36a13227852..41dccc7c86b 100644 --- a/docs/en/operations/external-authenticators/ldap.md +++ b/docs/en/operations/external-authenticators/ldap.md @@ -1,4 +1,4 @@ -# LDAP {#external-authenticators-ldap} +# LDAP {#external-authenticators-ldap} LDAP server can be used to authenticate ClickHouse users. There are two different approaches for doing this: @@ -153,4 +153,3 @@ Parameters: - `prefix` - prefix, that will be expected to be in front of each string in the original list of strings returned by the LDAP search. Prefix will be removed from the original strings and resulting strings will be treated as local role names. Empty, by default. - From ada96c8ff165ea13c80a1b06081874b6746ae7e0 Mon Sep 17 00:00:00 2001 From: zvonand Date: Wed, 24 Feb 2021 15:10:37 +0500 Subject: [PATCH 2/4] Added En and Ru Kerberos docs --- .../external-authenticators/kerberos.md | 193 +++++++----------- .../external-authenticators/kerberos.md | 111 ++++++++++ 2 files changed, 187 insertions(+), 117 deletions(-) create mode 100644 docs/ru/operations/external-authenticators/kerberos.md diff --git a/docs/en/operations/external-authenticators/kerberos.md b/docs/en/operations/external-authenticators/kerberos.md index 35d9cc942df..315e688a531 100644 --- a/docs/en/operations/external-authenticators/kerberos.md +++ b/docs/en/operations/external-authenticators/kerberos.md @@ -1,67 +1,76 @@ # Kerberos {#external-authenticators-kerberos} -Existing and properly configured ClickHouse users can be authenticated via Kerberos authentication protocol. +Existing and properly configured ClickHouse users can be authenticated via Kerberos authentication protocol. -## LDAP Server Definition {#ldap-server-definition} +Currently, Kerberos can only be used as an external authenticator for existing users, which are defined in `users.xml` or in local access control paths. Those users may only use HTTP requests, and must be able to authenticate using GSS-SPNEGO mechanism. -To define LDAP server you must add `ldap_servers` section to the `config.xml`. For example, +For this approach, Kerberos must be configured in the system and must be enabled in ClickHouse config. + + +## Enabling Kerberos in ClickHouse {#enabling-kerberos-in-clickhouse} + +To enable Kerberos, presence of a single `kerberos` section in `config.xml` is enough. However, depending on the directions that involve Kerberos, configurations in other places may be necessary. + +Example (goes into `config.xml`): ```xml - - - localhost - 636 - uid={user_name},ou=users,dc=example,dc=com - 300 - yes - tls1.2 - demand - /path/to/tls_cert_file - /path/to/tls_key_file - /path/to/tls_ca_cert_file - /path/to/tls_ca_cert_dir - ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:AES256-GCM-SHA384 - - + ``` -Note, that you can define multiple LDAP servers inside the `ldap_servers` section using distinct names. +...or: -Parameters: +```xml + + + + HTTP/clickhouse.example.com@EXAMPLE.COM + + +``` -- `host` - LDAP server hostname or IP, this parameter is mandatory and cannot be empty. -- `port` - LDAP server port, default is `636` if `enable_tls` is set to `true`, `389` otherwise. -- `bind_dn` - template used to construct the DN to bind to. - - The resulting DN will be constructed by replacing all `{user_name}` substrings of the - template with the actual user name during each authentication attempt. -- `verification_cooldown` - a period of time, in seconds, after a successful bind attempt, - during which the user will be assumed to be successfully authenticated for all consecutive - requests without contacting the LDAP server. - - Specify `0` (the default) to disable caching and force contacting the LDAP server for each authentication request. -- `enable_tls` - flag to trigger use of secure connection to the LDAP server. - - Specify `no` for plain text `ldap://` protocol (not recommended). - - Specify `yes` for LDAP over SSL/TLS `ldaps://` protocol (recommended, the default). - - Specify `starttls` for legacy StartTLS protocol (plain text `ldap://` protocol, upgraded to TLS). -- `tls_minimum_protocol_version` - the minimum protocol version of SSL/TLS. - - Accepted values are: `ssl2`, `ssl3`, `tls1.0`, `tls1.1`, `tls1.2` (the default). -- `tls_require_cert` - SSL/TLS peer certificate verification behavior. - - Accepted values are: `never`, `allow`, `try`, `demand` (the default). -- `tls_cert_file` - path to certificate file. -- `tls_key_file` - path to certificate key file. -- `tls_ca_cert_file` - path to CA certificate file. -- `tls_ca_cert_dir` - path to the directory containing CA certificates. -- `tls_cipher_suite` - allowed cipher suite (in OpenSSL notation). +...or: -## LDAP External Authenticator {#ldap-external-authenticator} +```xml + + + + EXAMPLE.COM + + +``` -A remote LDAP server can be used as a method for verifying passwords for locally defined users (users defined in `users.xml` or in local access control paths). In order to achieve this, specify previously defined LDAP server name instead of `password` or similar sections in the user definition. +Note that you can define only one `kerberos` section. Presence of multiple `kerberos` sections will force ClickHouse to disable Kerberos authentication. -At each login attempt, ClickHouse will try to "bind" to the specified DN defined by the `bind_dn` parameter in the [LDAP server definition](#ldap-server-definition) using the provided credentials, and if successful, the user will be considered authenticated. This is often called a "simple bind" method. +#### Parameters: -For example, +- `principal` - canonical service principal name that will be acquired and used when accepting security contexts. + - This parameter is optional, if omitted, the default principal will be used. + + +- `realm` - a realm, that will be used to restrict authentication to only those requests whose initiator's realm matches it. + - This parameter is optional, if omitted, no additional filtering by realm will be applied. + +Note that `principal` and `realm` sections cannot be specified at the same time. Presence of both `principal` and `realm` sections will force ClickHouse to disable Kerberos authentication. + + +## Kerberos as an external authenticator for existing users {#kerberos-as-an-external-authenticator-for-existing-users} + +Kerberos can be used as a method for verifying the identity of locally defined users (users defined in `users.xml` or in local access control paths). Currently, **only** requests over HTTP interface can be *kerberized* (via GSS-SPNEGO mechanism). + +Kerberos principal name format usualy follows this pattern: + +- *primary/instance@REALM* + +The */instance* part may occur zero or more times. **The *primary* part of the canonical principal name of the initiator is expected to match the kerberized user name for authentication to succeed**. + +### Enabling Kerberos in `users.xml` {#enabling-kerberos-in-users-xml} + +In order to enable Kerberos authenication for the user, specify `kerberos` section instead of `password` or similar sections in the user definition. Note that Kerberos authentication cannot be used alongside with any other authentication mechanism. Presence of any other sections like `password` alongside with `kerberos` will force ClickHouse to shutdown. + +Example (goes into `users.xml`): ```xml @@ -70,81 +79,31 @@ For example, - - my_ldap_server - + + EXAMPLE.COM + ``` -Note, that user `my_user` refers to `my_ldap_server`. This LDAP server must be configured in the main `config.xml` file as described previously. - -When SQL-driven [Access Control and Account Management](../access-rights.md#access-control) is enabled in ClickHouse, users that are authenticated by LDAP servers can also be created using the [CRATE USER](../../sql-reference/statements/create/user.md#create-user-statement) statement. - - -```sql -CREATE USER my_user IDENTIFIED WITH ldap_server BY 'my_ldap_server' -``` - -## LDAP Exernal User Directory {#ldap-external-user-directory} - -In addition to the locally defined users, a remote LDAP server can be used as a source of user definitions. In order to achieve this, specify previously defined LDAP server name (see [LDAP Server Definition](#ldap-server-definition)) in the `ldap` section inside the `users_directories` section of the `config.xml` file. - -At each login attempt, ClickHouse will try to find the user definition locally and authenticate it as usual, but if the user is not defined, ClickHouse will assume it exists in the external LDAP directory, and will try to "bind" to the specified DN at the LDAP server using the provided credentials. If successful, the user will be considered existing and authenticated. The user will be assigned roles from the list specified in the `roles` section. Additionally, LDAP "search" can be performed and results can be transformed and treated as role names and then be assigned to the user if the `role_mapping` section is also configured. All this implies that the SQL-driven [Access Control and Account Management](../access-rights.md#access-control) is enabled and roles are created using the [CREATE ROLE](../../sql-reference/statements/create/role.md#create-role-statement) statement. - -Example (goes into `config.xml`): - -```xml - - - - - - my_ldap_server - - - - - - ou=groups,dc=example,dc=com - subtree - (&(objectClass=groupOfNames)(member={bind_dn})) - cn - clickhouse_ - - - - -``` - -Note that `my_ldap_server` referred in the `ldap` section inside the `user_directories` section must be a previously -defined LDAP server that is configured in the `config.xml` (see [LDAP Server Definition](#ldap-server-definition)). +Note, that now, once user `my_user` uses `kerberos`, Kerberos must be enabled in the main `config.xml` file as described previously. Parameters: -- `server` - one of LDAP server names defined in the `ldap_servers` config section above. - This parameter is mandatory and cannot be empty. -- `roles` - section with a list of locally defined roles that will be assigned to each user retrieved from the LDAP server. - - If no roles are specified here or assigned during role mapping (below), user will not be able - to perform any actions after authentication. -- `role_mapping` - section with LDAP search parameters and mapping rules. - - When a user authenticates, while still bound to LDAP, an LDAP search is performed using `search_filter` - and the name of the logged in user. For each entry found during that search, the value of the specified - attribute is extracted. For each attribute value that has the specified prefix, the prefix is removed, - and the rest of the value becomes the name of a local role defined in ClickHouse, - which is expected to be created beforehand by the [CREATE ROLE](../../sql-reference/statements/create/role.md#create-role-statement) statement. - - There can be multiple `role_mapping` sections defined inside the same `ldap` section. All of them will be applied. - - `base_dn` - template used to construct the base DN for the LDAP search. - - The resulting DN will be constructed by replacing all `{user_name}` and `{bind_dn}` - substrings of the template with the actual user name and bind DN during each LDAP search. - - `scope` - scope of the LDAP search. - - Accepted values are: `base`, `one_level`, `children`, `subtree` (the default). - - `search_filter` - template used to construct the search filter for the LDAP search. - - The resulting filter will be constructed by replacing all `{user_name}`, `{bind_dn}`, and `{base_dn}` - substrings of the template with the actual user name, bind DN, and base DN during each LDAP search. - - Note, that the special characters must be escaped properly in XML. - - `attribute` - attribute name whose values will be returned by the LDAP search. - - `prefix` - prefix, that will be expected to be in front of each string in the original - list of strings returned by the LDAP search. Prefix will be removed from the original - strings and resulting strings will be treated as local role names. Empty, by default. +- `realm` - a realm that will be used to restrict authentication to only those requests whose initiator's realm matches it. + - This parameter is optional, if omitted, no additional filtering by realm will be applied. + +### Enabling Kerberos using SQL {#enabling-kerberos-using-sql} + +When SQL-driven Access Control and Account Management is enabled in ClickHouse, users identified by Kerberos can also be created using SQL statements. + +```sql +CREATE USER my_user IDENTIFIED WITH kerberos REALM 'EXAMPLE.COM' +``` + +...or, without filtering by realm: + +```sql +CREATE USER my_user IDENTIFIED WITH kerberos +``` diff --git a/docs/ru/operations/external-authenticators/kerberos.md b/docs/ru/operations/external-authenticators/kerberos.md new file mode 100644 index 00000000000..3c6813cec47 --- /dev/null +++ b/docs/ru/operations/external-authenticators/kerberos.md @@ -0,0 +1,111 @@ +# Kerberos {#external-authenticators-kerberos} + +ClickHouse поддерживает аутентификацию существующих (и правильно сконфигурированных) пользователей с использованием Kerberos. + +В настоящее время возможна Kerberos-аутентификация уже существующих пользователей, определённых в конфигурационном файле `users.xml` или с помощью выражений SQL. Пользователи, имеющие конфигурацию для аутентификации через Kerberos, могут работать только через HTTP-интерфейс, причём сами клиенты должны иметь возможность аутентификации с использованием механизма GSS-SPNEGO. + +Для Kerberos-аутентификации необходимо предварительно корректно настроить Kerberos на стороне клиента, на сервере и в конфигурационных файлах самого ClickHouse. Ниже описана лишь конфигурация ClickHouse. + + +## Настройка Kerberos в ClickHouse {#enabling-kerberos-in-clickhouse} + +Для того, чтобы включить Kerberos-аутентификацию в ClickHouse, необходимо добавить одну-единственную секцию `kerberos` в `config.xml`. Заметим, что это не единственная необходимая для работы Kerberos настройка. + +Примеры, как должен выглядеть файл `config.xml`: + +```xml + + + + +``` + +...или: + +```xml + + + + HTTP/clickhouse.example.com@EXAMPLE.COM + + +``` + +...или: + +```xml + + + + EXAMPLE.COM + + +``` + +**Важно**: в конфигурационном файле может быть не более одной секции `kerberos`. В противном случае, аутентификация с помощью Kerberos будет недоступна для всех пользователей. + +#### Параметры: + +- `principal` — задаёт имя принципала (canonical service principal name, SPN), используемое при авторизации ClickHouse на Kerberos-сервере. + - Это опциональный параметр, при его отсутствии будет использовано стандартное имя. + +- `realm` &mdash обеспечивает фильтрацию по реалм (realm). Пользователям, чей реалм не совпадает с указанным, будет отказано в аутентификации. + - Это опциональный параметр, при его отсутствии фильтр по реалм применяться не будет. + +*Важно*: в конфигурационном файле не могут быть указаны одновременно оба параметра. В противном случае, аутентификация с помощью Kerberos будет недоступна для всех пользователей. + +## Аутентификация пользователей с помощью Kerberos {#kerberos-as-an-external-authenticator-for-existing-users} + +Уже существующие пользователи могут воспользоваться аутентификацией с помощью Kerberos. Однако, Kerberos-аутентификация возможна только при использовании HTTP-интерфейса. + +Имя принципала (principal name) обычно имеет вид: + +- *primary/instance@REALM* + +Для успешной аутентификации необходимо, чтобы *primary* совпадало с именем пользователя ClickHouse, настроенного для использования Kerberos. + +### Настройка Kerberos в `users.xml` {#enabling-kerberos-in-users-xml} + +Для того, чтобы пользователь имел возможность аутентификации с помощью Kerberos, достаточно включить секцию `kerberos` в описание пользователя в `users.xml` (например, вместо секции `password` или аналогичной ей). + +**Важно**: если пользователь настроен для Kerberos-аутентификации, другие виды уатентификации будут для него недоступны. Если наряду с `kerberos` в определении пользователя будет указан какой-либо другой способ аутентификации, ClickHouse завершит работу. + +Пример, как выглядит конфигурация Kerberos в `users.xml`: + +```xml + + + + + + + + EXAMPLE.COM + + + + +``` + +Note, that now, once user `my_user` uses `kerberos`, Kerberos must be enabled in the main `config.xml` file as described previously. +Ещё раз отметим, что кроме `users.xml`, необходимо также включить Kerberos в `config.xml`. + +Параметры: + +- `realm` — обеспечивает фильтрацию по реалм (realm): аутентификация будет возможна только при совпадении реалм клиента с указанным. + - Этот параметр является опциональным, при его отсутствии фильтрация применяться не будет. + +### Настройка Kerberos через SQL {#enabling-kerberos-using-sql} + +Пользователей, использующих Kerberos-аутентификацию, можно создавать не только в конфигурационном файле `users.xml`. +Если SQL-ориентированное управление доступом включено в ClickHouse, можно также создать пользователя, работающего через Kerberos, с помощью SQL. + +```sql +CREATE USER my_user IDENTIFIED WITH kerberos REALM 'EXAMPLE.COM' +``` + +...или, без фильтрации по реалм: + +```sql +CREATE USER my_user IDENTIFIED WITH kerberos +``` From fbb6a931cc905ec6b8e9fea92df97196292fab16 Mon Sep 17 00:00:00 2001 From: zvonand Date: Wed, 24 Feb 2021 17:59:47 +0500 Subject: [PATCH 3/4] Upd ldap translation --- docs/en/development/cmake-in-clickhouse.md | 284 ++++++++++++++++++ .../external-authenticators/index.md | 0 .../external-authenticators/ldap.md | 153 ++++++++++ 3 files changed, 437 insertions(+) create mode 100644 docs/en/development/cmake-in-clickhouse.md create mode 100644 docs/ru/operations/external-authenticators/index.md create mode 100644 docs/ru/operations/external-authenticators/ldap.md diff --git a/docs/en/development/cmake-in-clickhouse.md b/docs/en/development/cmake-in-clickhouse.md new file mode 100644 index 00000000000..6e6ac825587 --- /dev/null +++ b/docs/en/development/cmake-in-clickhouse.md @@ -0,0 +1,284 @@ +# CMake in ClickHouse + +## TL; DR How to make ClickHouse compile and link faster? + +Developer only! This command will likely fulfill most of your needs. Run before calling `ninja`. + +```cmake +cmake .. \ + -DCMAKE_C_COMPILER=/bin/clang-10 \ + -DCMAKE_CXX_COMPILER=/bin/clang++-10 \ + -DCMAKE_BUILD_TYPE=Debug \ + -DENABLE_CLICKHOUSE_ALL=OFF \ + -DENABLE_CLICKHOUSE_SERVER=ON \ + -DENABLE_CLICKHOUSE_CLIENT=ON \ + -DUSE_STATIC_LIBRARIES=OFF \ + -DSPLIT_SHARED_LIBRARIES=ON \ + -DENABLE_LIBRARIES=OFF \ + -DUSE_UNWIND=ON \ + -DENABLE_UTILS=OFF \ + -DENABLE_TESTS=OFF +``` + +## CMake files types + +1. ClickHouse's source CMake files (located in the root directory and in `/src`). +2. Arch-dependent CMake files (located in `/cmake/*os_name*`). +3. Libraries finders (search for contrib libraries, located in `/cmake/find`). +3. Contrib build CMake files (used instead of libraries' own CMake files, located in `/cmake/modules`) + +## List of CMake flags + +* This list is auto-generated by [this Python script](https://github.com/clickhouse/clickhouse/blob/master/docs/tools/cmake_in_clickhouse_generator.py). +* The flag name is a link to its position in the code. +* If an option's default value is itself an option, it's also a link to its position in this list. +### ClickHouse modes + +| Name | Default value | Description | Comment | +|------|---------------|-------------|---------| +| [`ENABLE_CLICKHOUSE_ALL`](https://github.com/clickhouse/clickhouse/blob/master/programs/CMakeLists.txt#L8) | `ON` | Enable all ClickHouse modes by default | The `clickhouse` binary is a multi purpose tool that contains multiple execution modes (client, server, etc.), each of them may be built and linked as a separate library. If you do not know what modes you need, turn this option OFF and enable SERVER and CLIENT only. | +| [`ENABLE_CLICKHOUSE_BENCHMARK`](https://github.com/clickhouse/clickhouse/blob/master/programs/CMakeLists.txt#L18) | `ENABLE_CLICKHOUSE_ALL` | Queries benchmarking mode | https://clickhouse.tech/docs/en/operations/utilities/clickhouse-benchmark/ | +| [`ENABLE_CLICKHOUSE_CLIENT`](https://github.com/clickhouse/clickhouse/blob/master/programs/CMakeLists.txt#L11) | `ENABLE_CLICKHOUSE_ALL` | Client mode (interactive tui/shell that connects to the server) | | +| [`ENABLE_CLICKHOUSE_COMPRESSOR`](https://github.com/clickhouse/clickhouse/blob/master/programs/CMakeLists.txt#L23) | `ENABLE_CLICKHOUSE_ALL` | Data compressor and decompressor | https://clickhouse.tech/docs/en/operations/utilities/clickhouse-compressor/ | +| [`ENABLE_CLICKHOUSE_COPIER`](https://github.com/clickhouse/clickhouse/blob/master/programs/CMakeLists.txt#L26) | `ENABLE_CLICKHOUSE_ALL` | Inter-cluster data copying mode | https://clickhouse.tech/docs/en/operations/utilities/clickhouse-copier/ | +| [`ENABLE_CLICKHOUSE_EXTRACT_FROM_CONFIG`](https://github.com/clickhouse/clickhouse/blob/master/programs/CMakeLists.txt#L20) | `ENABLE_CLICKHOUSE_ALL` | Configs processor (extract values etc.) | | +| [`ENABLE_CLICKHOUSE_FORMAT`](https://github.com/clickhouse/clickhouse/blob/master/programs/CMakeLists.txt#L28) | `ENABLE_CLICKHOUSE_ALL` | Queries pretty-printer and formatter with syntax highlighting | | +| [`ENABLE_CLICKHOUSE_GIT_IMPORT`](https://github.com/clickhouse/clickhouse/blob/master/programs/CMakeLists.txt#L40) | `ENABLE_CLICKHOUSE_ALL` | A tool to analyze Git repositories | https://presentations.clickhouse.tech/matemarketing_2020/ | +| [`ENABLE_CLICKHOUSE_INSTALL`](https://github.com/clickhouse/clickhouse/blob/master/programs/CMakeLists.txt#L44) | `OFF` | Install ClickHouse without .deb/.rpm/.tgz packages (having the binary only) | | +| [`ENABLE_CLICKHOUSE_LOCAL`](https://github.com/clickhouse/clickhouse/blob/master/programs/CMakeLists.txt#L15) | `ENABLE_CLICKHOUSE_ALL` | Local files fast processing mode | https://clickhouse.tech/docs/en/operations/utilities/clickhouse-local/ | +| [`ENABLE_CLICKHOUSE_OBFUSCATOR`](https://github.com/clickhouse/clickhouse/blob/master/programs/CMakeLists.txt#L32) | `ENABLE_CLICKHOUSE_ALL` | Table data obfuscator (convert real data to benchmark-ready one) | https://clickhouse.tech/docs/en/operations/utilities/clickhouse-obfuscator/ | +| [`ENABLE_CLICKHOUSE_ODBC_BRIDGE`](https://github.com/clickhouse/clickhouse/blob/master/programs/CMakeLists.txt#L36) | `ENABLE_CLICKHOUSE_ALL` | HTTP-server working like a proxy to ODBC driver | https://clickhouse.tech/docs/en/operations/utilities/odbc-bridge/ | +| [`ENABLE_CLICKHOUSE_SERVER`](https://github.com/clickhouse/clickhouse/blob/master/programs/CMakeLists.txt#L10) | `ENABLE_CLICKHOUSE_ALL` | Server mode (main mode) | | + +### External libraries +Note that ClickHouse uses forks of these libraries, see https://github.com/ClickHouse-Extras. + +| Name | Default value | Description | Comment | +|------|---------------|-------------|---------| +| [`ENABLE_AMQPCPP`](https://github.com/clickhouse/clickhouse/blob/master/cmake/find/amqpcpp.cmake#L1) | `ENABLE_LIBRARIES` | Enalbe AMQP-CPP | | +| [`ENABLE_AVRO`](https://github.com/clickhouse/clickhouse/blob/master/cmake/find/avro.cmake#L2) | `ENABLE_LIBRARIES` | Enable Avro | Needed when using Apache Avro serialization format | +| [`ENABLE_BASE64`](https://github.com/clickhouse/clickhouse/blob/master/cmake/find/base64.cmake#L1) | `ENABLE_LIBRARIES` | Enable base64 | | +| [`ENABLE_BROTLI`](https://github.com/clickhouse/clickhouse/blob/master/cmake/find/brotli.cmake#L1) | `ENABLE_LIBRARIES` | Enable brotli | | +| [`ENABLE_CAPNP`](https://github.com/clickhouse/clickhouse/blob/master/cmake/find/capnp.cmake#L1) | `ENABLE_LIBRARIES` | Enable Cap'n Proto | | +| [`ENABLE_CASSANDRA`](https://github.com/clickhouse/clickhouse/blob/master/cmake/find/cassandra.cmake#L1) | `ENABLE_LIBRARIES` | Enable Cassandra | | +| [`ENABLE_CCACHE`](https://github.com/clickhouse/clickhouse/blob/master/cmake/find/ccache.cmake#L22) | `ENABLE_CCACHE_BY_DEFAULT` | Speedup re-compilations using ccache (external tool) | https://ccache.dev/ | +| [`ENABLE_CLANG_TIDY`](https://github.com/clickhouse/clickhouse/blob/master/cmake/analysis.cmake#L2) | `OFF` | Use clang-tidy static analyzer | https://clang.llvm.org/extra/clang-tidy/ | +| [`ENABLE_CURL`](https://github.com/clickhouse/clickhouse/blob/master/cmake/find/curl.cmake#L1) | `ENABLE_LIBRARIES` | Enable curl | | +| [`ENABLE_EMBEDDED_COMPILER`](https://github.com/clickhouse/clickhouse/blob/master/cmake/find/llvm.cmake#L5) | `ENABLE_LIBRARIES` | Set to TRUE to enable support for 'compile_expressions' option for query execution | | +| [`ENABLE_FASTOPS`](https://github.com/clickhouse/clickhouse/blob/master/cmake/find/fastops.cmake#L2) | `ENABLE_LIBRARIES` | Enable fast vectorized mathematical functions library by Mikhail Parakhin | | +| [`ENABLE_GPERF`](https://github.com/clickhouse/clickhouse/blob/master/cmake/find/gperf.cmake#L5) | `ENABLE_LIBRARIES` | Use gperf function hash generator tool | | +| [`ENABLE_GRPC`](https://github.com/clickhouse/clickhouse/blob/master/cmake/find/grpc.cmake#L8) | `ENABLE_GRPC_DEFAULT` | Use gRPC | | +| [`ENABLE_GSASL_LIBRARY`](https://github.com/clickhouse/clickhouse/blob/master/cmake/find/libgsasl.cmake#L1) | `ENABLE_LIBRARIES` | Enable gsasl library | | +| [`ENABLE_H3`](https://github.com/clickhouse/clickhouse/blob/master/cmake/find/h3.cmake#L1) | `ENABLE_LIBRARIES` | Enable H3 | | +| [`ENABLE_HDFS`](https://github.com/clickhouse/clickhouse/blob/master/cmake/find/hdfs3.cmake#L2) | `ENABLE_LIBRARIES` | Enable HDFS | | +| [`ENABLE_ICU`](https://github.com/clickhouse/clickhouse/blob/master/cmake/find/icu.cmake#L2) | `ENABLE_LIBRARIES` | Enable ICU | | +| [`ENABLE_LDAP`](https://github.com/clickhouse/clickhouse/blob/master/cmake/find/ldap.cmake#L5) | `ENABLE_LIBRARIES` | Enable LDAP | | +| [`ENABLE_LIBPQXX`](https://github.com/clickhouse/clickhouse/blob/master/cmake/find/libpqxx.cmake#L1) | `ENABLE_LIBRARIES` | Enalbe libpqxx | | +| [`ENABLE_MSGPACK`](https://github.com/clickhouse/clickhouse/blob/master/cmake/find/msgpack.cmake#L1) | `ENABLE_LIBRARIES` | Enable msgpack library | | +| [`ENABLE_MYSQL`](https://github.com/clickhouse/clickhouse/blob/master/cmake/find/mysqlclient.cmake#L2) | `ENABLE_LIBRARIES` | Enable MySQL | | +| [`ENABLE_NURAFT`](https://github.com/clickhouse/clickhouse/blob/master/cmake/find/nuraft.cmake#L1) | `ENABLE_LIBRARIES` | Enable NuRaft | | +| [`ENABLE_ODBC`](https://github.com/clickhouse/clickhouse/blob/master/cmake/find/odbc.cmake#L1) | `ENABLE_LIBRARIES` | Enable ODBC library | | +| [`ENABLE_ORC`](https://github.com/clickhouse/clickhouse/blob/master/cmake/find/orc.cmake#L1) | `ENABLE_LIBRARIES` | Enable ORC | | +| [`ENABLE_PARQUET`](https://github.com/clickhouse/clickhouse/blob/master/cmake/find/parquet.cmake#L2) | `ENABLE_LIBRARIES` | Enable parquet | | +| [`ENABLE_PROTOBUF`](https://github.com/clickhouse/clickhouse/blob/master/cmake/find/protobuf.cmake#L1) | `ENABLE_LIBRARIES` | Enable protobuf | | +| [`ENABLE_RAPIDJSON`](https://github.com/clickhouse/clickhouse/blob/master/cmake/find/rapidjson.cmake#L1) | `ENABLE_LIBRARIES` | Use rapidjson | | +| [`ENABLE_RDKAFKA`](https://github.com/clickhouse/clickhouse/blob/master/cmake/find/rdkafka.cmake#L1) | `ENABLE_LIBRARIES` | Enable kafka | | +| [`ENABLE_ROCKSDB`](https://github.com/clickhouse/clickhouse/blob/master/cmake/find/rocksdb.cmake#L1) | `ENABLE_LIBRARIES` | Enable ROCKSDB | | +| [`ENABLE_S3`](https://github.com/clickhouse/clickhouse/blob/master/cmake/find/s3.cmake#L2) | `ENABLE_LIBRARIES` | Enable S3 | | +| [`ENABLE_SSL`](https://github.com/clickhouse/clickhouse/blob/master/cmake/find/ssl.cmake#L3) | `ENABLE_LIBRARIES` | Enable ssl | Needed when securely connecting to an external server, e.g. clickhouse-client --host ... --secure | +| [`ENABLE_STATS`](https://github.com/clickhouse/clickhouse/blob/master/cmake/find/stats.cmake#L1) | `ENABLE_LIBRARIES` | Enalbe StatsLib library | | + + +### External libraries system/bundled mode + +| Name | Default value | Description | Comment | +|------|---------------|-------------|---------| +| [`USE_INTERNAL_AVRO_LIBRARY`](https://github.com/clickhouse/clickhouse/blob/master/cmake/find/avro.cmake#L11) | `ON` | Set to FALSE to use system avro library instead of bundled | | +| [`USE_INTERNAL_AWS_S3_LIBRARY`](https://github.com/clickhouse/clickhouse/blob/master/cmake/find/s3.cmake#L14) | `ON` | Set to FALSE to use system S3 instead of bundled (experimental set to OFF on your own risk) | | +| [`USE_INTERNAL_BROTLI_LIBRARY`](https://github.com/clickhouse/clickhouse/blob/master/cmake/find/brotli.cmake#L12) | `USE_STATIC_LIBRARIES` | Set to FALSE to use system libbrotli library instead of bundled | Many system ship only dynamic brotly libraries, so we back off to bundled by default | +| [`USE_INTERNAL_CAPNP_LIBRARY`](https://github.com/clickhouse/clickhouse/blob/master/cmake/find/capnp.cmake#L10) | `NOT_UNBUNDLED` | Set to FALSE to use system capnproto library instead of bundled | | +| [`USE_INTERNAL_CURL`](https://github.com/clickhouse/clickhouse/blob/master/cmake/find/curl.cmake#L10) | `NOT_UNBUNDLED` | Use internal curl library | | +| [`USE_INTERNAL_GRPC_LIBRARY`](https://github.com/clickhouse/clickhouse/blob/master/cmake/find/grpc.cmake#L25) | `NOT_UNBUNDLED` | Set to FALSE to use system gRPC library instead of bundled. (Experimental. Set to OFF on your own risk) | Normally we use the internal gRPC framework. You can set USE_INTERNAL_GRPC_LIBRARY to OFF to force using the external gRPC framework, which should be installed in the system in this case. The external gRPC framework can be installed in the system by running sudo apt-get install libgrpc++-dev protobuf-compiler-grpc | +| [`USE_INTERNAL_GTEST_LIBRARY`](https://github.com/clickhouse/clickhouse/blob/master/cmake/find/gtest.cmake#L3) | `NOT_UNBUNDLED` | Set to FALSE to use system Google Test instead of bundled | | +| [`USE_INTERNAL_H3_LIBRARY`](https://github.com/clickhouse/clickhouse/blob/master/cmake/find/h3.cmake#L9) | `ON` | Set to FALSE to use system h3 library instead of bundled | | +| [`USE_INTERNAL_HDFS3_LIBRARY`](https://github.com/clickhouse/clickhouse/blob/master/cmake/find/hdfs3.cmake#L14) | `ON` | Set to FALSE to use system HDFS3 instead of bundled (experimental - set to OFF on your own risk) | | +| [`USE_INTERNAL_ICU_LIBRARY`](https://github.com/clickhouse/clickhouse/blob/master/cmake/find/icu.cmake#L15) | `NOT_UNBUNDLED` | Set to FALSE to use system ICU library instead of bundled | | +| [`USE_INTERNAL_LDAP_LIBRARY`](https://github.com/clickhouse/clickhouse/blob/master/cmake/find/ldap.cmake#L14) | `NOT_UNBUNDLED` | Set to FALSE to use system *LDAP library instead of bundled | | +| [`USE_INTERNAL_LIBCXX_LIBRARY`](https://github.com/clickhouse/clickhouse/blob/master/cmake/find/cxx.cmake#L15) | `USE_INTERNAL_LIBCXX_LIBRARY_DEFAULT` | Disable to use system libcxx and libcxxabi libraries instead of bundled | | +| [`USE_INTERNAL_LIBGSASL_LIBRARY`](https://github.com/clickhouse/clickhouse/blob/master/cmake/find/libgsasl.cmake#L12) | `USE_STATIC_LIBRARIES` | Set to FALSE to use system libgsasl library instead of bundled | when USE_STATIC_LIBRARIES we usually need to pick up hell a lot of dependencies for libgsasl | +| [`USE_INTERNAL_LIBXML2_LIBRARY`](https://github.com/clickhouse/clickhouse/blob/master/cmake/find/libxml2.cmake#L1) | `NOT_UNBUNDLED` | Set to FALSE to use system libxml2 library instead of bundled | | +| [`USE_INTERNAL_LLVM_LIBRARY`](https://github.com/clickhouse/clickhouse/blob/master/cmake/find/llvm.cmake#L8) | `NOT_UNBUNDLED` | Use bundled or system LLVM library. | | +| [`USE_INTERNAL_MSGPACK_LIBRARY`](https://github.com/clickhouse/clickhouse/blob/master/cmake/find/msgpack.cmake#L10) | `NOT_UNBUNDLED` | Set to FALSE to use system msgpack library instead of bundled | | +| [`USE_INTERNAL_MYSQL_LIBRARY`](https://github.com/clickhouse/clickhouse/blob/master/cmake/find/mysqlclient.cmake#L15) | `NOT_UNBUNDLED` | Set to FALSE to use system mysqlclient library instead of bundled | | +| [`USE_INTERNAL_ODBC_LIBRARY`](https://github.com/clickhouse/clickhouse/blob/master/cmake/find/odbc.cmake#L22) | `NOT_UNBUNDLED` | Use internal ODBC library | | +| [`USE_INTERNAL_ORC_LIBRARY`](https://github.com/clickhouse/clickhouse/blob/master/cmake/find/orc.cmake#L11) | `ON` | Set to FALSE to use system ORC instead of bundled (experimental set to OFF on your own risk) | | +| [`USE_INTERNAL_PARQUET_LIBRARY`](https://github.com/clickhouse/clickhouse/blob/master/cmake/find/parquet.cmake#L16) | `NOT_UNBUNDLED` | Set to FALSE to use system parquet library instead of bundled | | +| [`USE_INTERNAL_POCO_LIBRARY`](https://github.com/clickhouse/clickhouse/blob/master/cmake/find/poco.cmake#L1) | `ON` | Use internal Poco library | | +| [`USE_INTERNAL_PROTOBUF_LIBRARY`](https://github.com/clickhouse/clickhouse/blob/master/cmake/find/protobuf.cmake#L14) | `NOT_UNBUNDLED` | Set to FALSE to use system protobuf instead of bundled. (Experimental. Set to OFF on your own risk) | Normally we use the internal protobuf library. You can set USE_INTERNAL_PROTOBUF_LIBRARY to OFF to force using the external protobuf library, which should be installed in the system in this case. The external protobuf library can be installed in the system by running sudo apt-get install libprotobuf-dev protobuf-compiler libprotoc-dev | +| [`USE_INTERNAL_RAPIDJSON_LIBRARY`](https://github.com/clickhouse/clickhouse/blob/master/cmake/find/rapidjson.cmake#L9) | `NOT_UNBUNDLED` | Set to FALSE to use system rapidjson library instead of bundled | | +| [`USE_INTERNAL_RDKAFKA_LIBRARY`](https://github.com/clickhouse/clickhouse/blob/master/cmake/find/rdkafka.cmake#L10) | `NOT_UNBUNDLED` | Set to FALSE to use system librdkafka instead of the bundled | | +| [`USE_INTERNAL_RE2_LIBRARY`](https://github.com/clickhouse/clickhouse/blob/master/cmake/find/re2.cmake#L1) | `NOT_UNBUNDLED` | Set to FALSE to use system re2 library instead of bundled [slower] | | +| [`USE_INTERNAL_ROCKSDB_LIBRARY`](https://github.com/clickhouse/clickhouse/blob/master/cmake/find/rocksdb.cmake#L10) | `NOT_UNBUNDLED` | Set to FALSE to use system ROCKSDB library instead of bundled | | +| [`USE_INTERNAL_SNAPPY_LIBRARY`](https://github.com/clickhouse/clickhouse/blob/master/cmake/find/snappy.cmake#L10) | `NOT_UNBUNDLED` | Set to FALSE to use system snappy library instead of bundled | | +| [`USE_INTERNAL_SPARSEHASH_LIBRARY`](https://github.com/clickhouse/clickhouse/blob/master/cmake/find/sparsehash.cmake#L1) | `ON` | Set to FALSE to use system sparsehash library instead of bundled | | +| [`USE_INTERNAL_SSL_LIBRARY`](https://github.com/clickhouse/clickhouse/blob/master/cmake/find/ssl.cmake#L12) | `NOT_UNBUNDLED` | Set to FALSE to use system *ssl library instead of bundled | | +| [`USE_INTERNAL_ZLIB_LIBRARY`](https://github.com/clickhouse/clickhouse/blob/master/cmake/find/zlib.cmake#L1) | `NOT_UNBUNDLED` | Set to FALSE to use system zlib library instead of bundled | | +| [`USE_INTERNAL_ZSTD_LIBRARY`](https://github.com/clickhouse/clickhouse/blob/master/cmake/find/zstd.cmake#L1) | `NOT_UNBUNDLED` | Set to FALSE to use system zstd library instead of bundled | | + + +### Other flags + +| Name | Default value | Description | Comment | +|------|---------------|-------------|---------| +| [`ADD_GDB_INDEX_FOR_GOLD`](https://github.com/clickhouse/clickhouse/blob/master/CMakeLists.txt#L195) | `OFF` | Add .gdb-index to resulting binaries for gold linker. | Ignored if `lld` is used | +| [`ARCH_NATIVE`](https://github.com/clickhouse/clickhouse/blob/master/CMakeLists.txt#L248) | `OFF` | Add -march=native compiler flag | | +| [`CLICKHOUSE_SPLIT_BINARY`](https://github.com/clickhouse/clickhouse/blob/master/CMakeLists.txt#L98) | `OFF` | Make several binaries (clickhouse-server, clickhouse-client etc.) instead of one bundled | | +| [`COMPILER_PIPE`](https://github.com/clickhouse/clickhouse/blob/master/CMakeLists.txt#L235) | `ON` | -pipe compiler option | Less `/tmp` usage, more RAM usage. | +| [`ENABLE_CHECK_HEAVY_BUILDS`](https://github.com/clickhouse/clickhouse/blob/master/CMakeLists.txt#L69) | `OFF` | Don't allow C++ translation units to compile too long or to take too much memory while compiling | | +| [`ENABLE_FUZZING`](https://github.com/clickhouse/clickhouse/blob/master/CMakeLists.txt#L115) | `OFF` | Fuzzy testing using libfuzzer | Implies `WITH_COVERAGE` | +| [`ENABLE_LIBRARIES`](https://github.com/clickhouse/clickhouse/blob/master/CMakeLists.txt#L357) | `ON` | Enable all external libraries by default | Turns on all external libs like s3, kafka, ODBC, ... | +| [`ENABLE_MULTITARGET_CODE`](https://github.com/clickhouse/clickhouse/blob/master/src/Functions/CMakeLists.txt#L100) | `ON` | Enable platform-dependent code | ClickHouse developers may use platform-dependent code under some macro (e.g. `ifdef ENABLE_MULTITARGET`). If turned ON, this option defines such macro. See `src/Functions/TargetSpecific.h` | +| [`ENABLE_TESTS`](https://github.com/clickhouse/clickhouse/blob/master/CMakeLists.txt#L154) | `ON` | Provide unit_test_dbms target with Google.Test unit tests | If turned `ON`, assumes the user has either the system GTest library or the bundled one. | +| [`ENABLE_THINLTO`](https://github.com/clickhouse/clickhouse/blob/master/CMakeLists.txt#L313) | `ON` | Clang-specific link time optimization | https://clang.llvm.org/docs/ThinLTO.html Applies to clang only. Disabled when building with tests or sanitizers. | +| [`FAIL_ON_UNSUPPORTED_OPTIONS_COMBINATION`](https://github.com/clickhouse/clickhouse/blob/master/CMakeLists.txt#L32) | `ON` | Stop/Fail CMake configuration if some ENABLE_XXX option is defined (either ON or OFF) but is not possible to satisfy | If turned off: e.g. when ENABLE_FOO is ON, but FOO tool was not found, the CMake will continue. | +| [`GLIBC_COMPATIBILITY`](https://github.com/clickhouse/clickhouse/blob/master/CMakeLists.txt#L159) | `ON` | Enable compatibility with older glibc libraries. | Only for Linux, x86_64. Implies `ENABLE_FASTMEMCPY` | +| [`LINKER_NAME`](https://github.com/clickhouse/clickhouse/blob/master/cmake/tools.cmake#L44) | `OFF` | Linker name or full path | Example values: `lld-10`, `gold`. | +| [`LLVM_HAS_RTTI`](https://github.com/clickhouse/clickhouse/blob/master/cmake/find/llvm.cmake#L40) | `ON` | Enable if LLVM was build with RTTI enabled | | +| [`MAKE_STATIC_LIBRARIES`](https://github.com/clickhouse/clickhouse/blob/master/CMakeLists.txt#L91) | `USE_STATIC_LIBRARIES` | Disable to make shared libraries | | +| [`PARALLEL_COMPILE_JOBS`](https://github.com/clickhouse/clickhouse/blob/master/cmake/limit_jobs.cmake#L10) | `""` | Maximum number of concurrent compilation jobs | 1 if not set | +| [`PARALLEL_LINK_JOBS`](https://github.com/clickhouse/clickhouse/blob/master/cmake/limit_jobs.cmake#L13) | `""` | Maximum number of concurrent link jobs | 1 if not set | +| [`SANITIZE`](https://github.com/clickhouse/clickhouse/blob/master/cmake/sanitize.cmake#L7) | `""` | Enable one of the code sanitizers | Possible values: - `address` (ASan) - `memory` (MSan) - `thread` (TSan) - `undefined` (UBSan) - "" (no sanitizing) | +| [`SPLIT_SHARED_LIBRARIES`](https://github.com/clickhouse/clickhouse/blob/master/CMakeLists.txt#L96) | `OFF` | Keep all internal libraries as separate .so files | DEVELOPER ONLY. Faster linking if turned on. | +| [`STRIP_DEBUG_SYMBOLS_FUNCTIONS`](https://github.com/clickhouse/clickhouse/blob/master/src/Functions/CMakeLists.txt#L49) | `STRIP_DSF_DEFAULT` | Do not generate debugger info for ClickHouse functions | Provides faster linking and lower binary size. Tradeoff is the inability to debug some source files with e.g. gdb (empty stack frames and no local variables)." | +| [`UNBUNDLED`](https://github.com/clickhouse/clickhouse/blob/master/CMakeLists.txt#L363) | `OFF` | Use system libraries instead of ones in contrib/ | We recommend avoiding this mode for production builds because we can't guarantee all needed libraries exist in your system. This mode exists for enthusiastic developers who are searching for trouble. Useful for maintainers of OS packages. | +| [`USE_INCLUDE_WHAT_YOU_USE`](https://github.com/clickhouse/clickhouse/blob/master/CMakeLists.txt#L418) | `OFF` | Automatically reduce unneeded includes in source code (external tool) | https://github.com/include-what-you-use/include-what-you-use | +| [`USE_LIBCXX`](https://github.com/clickhouse/clickhouse/blob/master/cmake/find/cxx.cmake#L1) | `NOT_UNBUNDLED` | Use libc++ and libc++abi instead of libstdc++ | | +| [`USE_SENTRY`](https://github.com/clickhouse/clickhouse/blob/master/cmake/find/sentry.cmake#L13) | `ENABLE_LIBRARIES` | Use Sentry | | +| [`USE_SIMDJSON`](https://github.com/clickhouse/clickhouse/blob/master/cmake/find/simdjson.cmake#L1) | `ENABLE_LIBRARIES` | Use simdjson | | +| [`USE_SNAPPY`](https://github.com/clickhouse/clickhouse/blob/master/cmake/find/snappy.cmake#L1) | `ENABLE_LIBRARIES` | Enable snappy library | | +| [`USE_STATIC_LIBRARIES`](https://github.com/clickhouse/clickhouse/blob/master/CMakeLists.txt#L90) | `ON` | Disable to use shared libraries | | +| [`USE_UNWIND`](https://github.com/clickhouse/clickhouse/blob/master/cmake/find/unwind.cmake#L1) | `ENABLE_LIBRARIES` | Enable libunwind (better stacktraces) | | +| [`WERROR`](https://github.com/clickhouse/clickhouse/blob/master/CMakeLists.txt#L373) | `OFF` | Enable -Werror compiler option | Using system libs can cause a lot of warnings in includes (on macro expansion). | +| [`WEVERYTHING`](https://github.com/clickhouse/clickhouse/blob/master/cmake/warnings.cmake#L22) | `ON` | Enable -Weverything option with some exceptions. | Add some warnings that are not available even with -Wall -Wextra -Wpedantic. Intended for exploration of new compiler warnings that may be found useful. Applies to clang only | +| [`WITH_COVERAGE`](https://github.com/clickhouse/clickhouse/blob/master/CMakeLists.txt#L274) | `OFF` | Profile the resulting binary/binaries | Compiler-specific coverage flags e.g. -fcoverage-mapping for gcc | + +## Developer's guide for adding new CMake options + +### Don't be obvious. Be informative. + +Bad: +```cmake +option (ENABLE_TESTS "Enables testing" OFF) +``` + +This description is quite useless as is neither gives the viewer any additional information nor explains the option purpose. + +Better: + +```cmake +option(ENABLE_TESTS "Provide unit_test_dbms target with Google.test unit tests" OFF) +``` + +If the option's purpose can't be guessed by its name, or the purpose guess may be misleading, or option has some +pre-conditions, leave a comment above the `option()` line and explain what it does. +The best way would be linking the docs page (if it exists). +The comment is parsed into a separate column (see below). + +Even better: + +```cmake +# implies ${TESTS_ARE_ENABLED} +# see tests/CMakeLists.txt for implementation detail. +option(ENABLE_TESTS "Provide unit_test_dbms target with Google.test unit tests" OFF) +``` + +### If the option's state could produce unwanted (or unusual) result, explicitly warn the user. + +Suppose you have an option that may strip debug symbols from the ClickHouse's part. +This can speed up the linking process, but produces a binary that cannot be debugged. +In that case, prefer explicitly raising a warning telling the developer that he may be doing something wrong. +Also, such options should be disabled if applies. + +Bad: +```cmake +option(STRIP_DEBUG_SYMBOLS_FUNCTIONS + "Do not generate debugger info for ClickHouse functions. + ${STRIP_DSF_DEFAULT}) + +if (STRIP_DEBUG_SYMBOLS_FUNCTIONS) + target_compile_options(clickhouse_functions PRIVATE "-g0") +endif() + +``` +Better: + +```cmake +# Provides faster linking and lower binary size. +# Tradeoff is the inability to debug some source files with e.g. gdb +# (empty stack frames and no local variables)." +option(STRIP_DEBUG_SYMBOLS_FUNCTIONS + "Do not generate debugger info for ClickHouse functions." + ${STRIP_DSF_DEFAULT}) + +if (STRIP_DEBUG_SYMBOLS_FUNCTIONS) + message(WARNING "Not generating debugger info for ClickHouse functions") + target_compile_options(clickhouse_functions PRIVATE "-g0") +endif() +``` + +### In the option's description, explain WHAT the option does rather than WHY it does something. + +The WHY explanation should be placed in the comment. +You may find that the option's name is self-descriptive. + +Bad: + +```cmake +option(ENABLE_THINLTO "Enable Thin LTO. Only applicable for clang. It's also suppressed when building with tests or sanitizers." ON) +``` + +Better: + +```cmake +# Only applicable for clang. +# Turned off when building with tests or sanitizers. +option(ENABLE_THINLTO "Clang-specific link time optimisation" ON). +``` + +### Don't assume other developers know as much as you do. + +In ClickHouse, there are many tools used that an ordinary developer may not know. If you are in doubt, give a link to +the tool's docs. It won't take much of your time. + +Bad: + +```cmake +option(ENABLE_THINLTO "Enable Thin LTO. Only applicable for clang. It's also suppressed when building with tests or sanitizers." ON) +``` + +Better (combined with the above hint): + +```cmake +# https://clang.llvm.org/docs/ThinLTO.html +# Only applicable for clang. +# Turned off when building with tests or sanitizers. +option(ENABLE_THINLTO "Clang-specific link time optimisation" ON). +``` + +Other example, bad: + +```cmake +option (USE_INCLUDE_WHAT_YOU_USE "Use 'include-what-you-use' tool" OFF) +``` + +Better: + +```cmake +# https://github.com/include-what-you-use/include-what-you-use +option (USE_INCLUDE_WHAT_YOU_USE "Reduce unneeded #include s (external tool)" OFF) +``` + +### Prefer consistent default values. + +CMake allows you to pass a plethora of values representing boolean `true/false`, e.g. `1, ON, YES, ...`. +Prefer the `ON/OFF` values, if possible. diff --git a/docs/ru/operations/external-authenticators/index.md b/docs/ru/operations/external-authenticators/index.md new file mode 100644 index 00000000000..e69de29bb2d diff --git a/docs/ru/operations/external-authenticators/ldap.md b/docs/ru/operations/external-authenticators/ldap.md new file mode 100644 index 00000000000..74d0f055f22 --- /dev/null +++ b/docs/ru/operations/external-authenticators/ldap.md @@ -0,0 +1,153 @@ +# LDAP {#external-authenticators-ldap} + +Пользователи ClickHouse могут использовать сервер LDAP для аутентификации. Есть два варианта, как это можно осуществлять: + +- использовать LDAP как внешний аутентификатор для уже существующих пользователей, определённых в `users.xml` или через SQL-воркфлоу. +- использовать LDAP как внешний каталог пользователей и разрешать аутентификацию для пользователей, не определённых локально в ClickHouse, но существующих на LDAP-сервере. + +Для любого из этих подходов необходимо прописать в конфигурационных файлах ClickHouse сервер LDAP. + +## Определение LDAP-сервера {#ldap-server-definition} + +Для определения LDAP-сервера необходимо добавить секцию `ldap_servers` в конфигурационный файл `config.xml`, например: + +```xml + + + + + localhost + 636 + uid={user_name},ou=users,dc=example,dc=com + 300 + yes + tls1.2 + demand + /path/to/tls_cert_file + /path/to/tls_key_file + /path/to/tls_ca_cert_file + /path/to/tls_ca_cert_dir + ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:AES256-GCM-SHA384 + + + +``` + +Используя различные имена, можно определить несколько LDAP-серверов внутри секции `ldap_servers`. + +Параметры: + +- `host` - IP или имя хоста LDAP-сервера; этот параметр указывать обязательно, и он не может быть пустым. +- `port` - порт LDAP-сервера. Если `enable_tls` имеет значение `true`, то по умолчанию этот параметр равен `636`, а в противном случае - `389`. +- `bind_dn` - шаблон для построения DN (Distinguished Name, или Уникальное Имя). + - Конечное DN получается заменой всех подстрок шаблона вида `{user_name}` актуальным именем пользователя во время каждой попытки аутентификации. +- `verification_cooldown` - временной интервал (в секундах) после успешной попытки подключения, в течение которого пользователь будет считаться аутентифицированным для всех последующих запросов, в это время повторное соединение с LDAP-сервером соверщаться не будет. + - Укажите `0` (значение по умолчанию), чтобы отключить кэширование и принудить связываться с LDAP-сервером при каждой попытке аутентификации. +- `enable_tls` - флаг для указания необходимости использования защищённого соедниения с LDAP-сервером. + - Укажите `no` для использования незашифрованного текстового протокола `ldap://` (не рекомендуется). + - Укажите `yes` для использования LDAP через протокол SSL/TLS `ldaps://` (значение по умолчанию, рекомендуется). + - Укажите `starttls` для использования устаревшего протокола StartTLS (незашифрованный `ldap://` с добавлением TLS). +- `tls_minimum_protocol_version` - требование к минимальной версии SSL/TLS. + - Возможные значения: `ssl2`, `ssl3`, `tls1.0`, `tls1.1`, `tls1.2` (по умолчанию). +- `tls_require_cert` - проверка сертификата SSL/TLS. + - Возможные значения: `never`, `allow`, `try`, `demand` (по умолчанию). +- `tls_cert_file` - путь к файлу сертификата. +- `tls_key_file` - путь к файлу ключа сертификата. +- `tls_ca_cert_file` - путь к файлу корневого сертификата (CA). +- `tls_ca_cert_dir` - путь к директории, содержащей корневые (CA) сертификаты. +- `tls_cipher_suite` - разрешённый набор шифров (в обозначении OpenSSL). + +## LDAP как внешний аутентификатор {#ldap-external-authenticator} + +Удалённый LDAP-сервер можно использовать для проверки паролей пользователей, определённых локально, в ClickHouse (в конфигурационных файлах или через SQL-воркфлоу). Для этого при определении пользователя укажите имя предварительно описанного LDAP-сервера вместо секции `password` или ей аналогичной. + +При каждой попытке логина ClickHouse будет пытаться подключиться как DN, указанному в параметре `bind_dn` в [Определении сервера LDAP](#ldap-server-definition) +At each login attempt, ClickHouse will try to "bind" to the specified DN defined by the `bind_dn` parameter in the [LDAP server definition](#ldap-server-definition) using the provided credentials, and if successful, the user will be considered authenticated. This is often called a "simple bind" method. + +For example, + +```xml + + + + + + + + my_ldap_server + + + + +``` + +Note, that user `my_user` refers to `my_ldap_server`. This LDAP server must be configured in the main `config.xml` file as described previously. + +When SQL-driven [Access Control and Account Management](../access-rights.md#access-control) is enabled in ClickHouse, users that are authenticated by LDAP servers can also be created using the [CRATE USER](../../sql-reference/statements/create/user.md#create-user-statement) statement. + + +```sql +CREATE USER my_user IDENTIFIED WITH ldap_server BY 'my_ldap_server' +``` + +## LDAP Exernal User Directory {#ldap-external-user-directory} + +In addition to the locally defined users, a remote LDAP server can be used as a source of user definitions. In order to achieve this, specify previously defined LDAP server name (see [LDAP Server Definition](#ldap-server-definition)) in the `ldap` section inside the `users_directories` section of the `config.xml` file. + +At each login attempt, ClickHouse will try to find the user definition locally and authenticate it as usual, but if the user is not defined, ClickHouse will assume it exists in the external LDAP directory, and will try to "bind" to the specified DN at the LDAP server using the provided credentials. If successful, the user will be considered existing and authenticated. The user will be assigned roles from the list specified in the `roles` section. Additionally, LDAP "search" can be performed and results can be transformed and treated as role names and then be assigned to the user if the `role_mapping` section is also configured. All this implies that the SQL-driven [Access Control and Account Management](../access-rights.md#access-control) is enabled and roles are created using the [CREATE ROLE](../../sql-reference/statements/create/role.md#create-role-statement) statement. + +Example (goes into `config.xml`): + +```xml + + + + + + my_ldap_server + + + + + + ou=groups,dc=example,dc=com + subtree + (&(objectClass=groupOfNames)(member={bind_dn})) + cn + clickhouse_ + + + + +``` + +Note that `my_ldap_server` referred in the `ldap` section inside the `user_directories` section must be a previously +defined LDAP server that is configured in the `config.xml` (see [LDAP Server Definition](#ldap-server-definition)). + +Parameters: + +- `server` - one of LDAP server names defined in the `ldap_servers` config section above. + This parameter is mandatory and cannot be empty. +- `roles` - section with a list of locally defined roles that will be assigned to each user retrieved from the LDAP server. + - If no roles are specified here or assigned during role mapping (below), user will not be able + to perform any actions after authentication. +- `role_mapping` - section with LDAP search parameters and mapping rules. + - When a user authenticates, while still bound to LDAP, an LDAP search is performed using `search_filter` + and the name of the logged in user. For each entry found during that search, the value of the specified + attribute is extracted. For each attribute value that has the specified prefix, the prefix is removed, + and the rest of the value becomes the name of a local role defined in ClickHouse, + which is expected to be created beforehand by the [CREATE ROLE](../../sql-reference/statements/create/role.md#create-role-statement) statement. + - There can be multiple `role_mapping` sections defined inside the same `ldap` section. All of them will be applied. + - `base_dn` - template used to construct the base DN for the LDAP search. + - The resulting DN will be constructed by replacing all `{user_name}` and `{bind_dn}` + substrings of the template with the actual user name and bind DN during each LDAP search. + - `scope` - scope of the LDAP search. + - Accepted values are: `base`, `one_level`, `children`, `subtree` (the default). + - `search_filter` - template used to construct the search filter for the LDAP search. + - The resulting filter will be constructed by replacing all `{user_name}`, `{bind_dn}`, and `{base_dn}` + substrings of the template with the actual user name, bind DN, and base DN during each LDAP search. + - Note, that the special characters must be escaped properly in XML. + - `attribute` - attribute name whose values will be returned by the LDAP search. + - `prefix` - prefix, that will be expected to be in front of each string in the original + list of strings returned by the LDAP search. Prefix will be removed from the original + strings and resulting strings will be treated as local role names. Empty, by default. From 652899e63115d03b5036df9592859d64f59c6386 Mon Sep 17 00:00:00 2001 From: zvonand Date: Mon, 1 Mar 2021 00:11:29 +0500 Subject: [PATCH 4/4] Semi-final kerberos + ldap (ru) --- .../external-authenticators/index.md | 2 +- .../external-authenticators/kerberos.md | 96 ++++++++++--------- .../external-authenticators/index.md | 14 +++ .../external-authenticators/kerberos.md | 57 ++++++----- .../external-authenticators/ldap.md | 69 ++++++------- 5 files changed, 129 insertions(+), 109 deletions(-) diff --git a/docs/en/operations/external-authenticators/index.md b/docs/en/operations/external-authenticators/index.md index 34f58e52602..d044c99c518 100644 --- a/docs/en/operations/external-authenticators/index.md +++ b/docs/en/operations/external-authenticators/index.md @@ -11,4 +11,4 @@ ClickHouse supports authenticating and managing users using external services. The following external authenticators and directories are supported: - [LDAP](./ldap.md#external-authenticators-ldap) [Authenticator](./ldap.md#ldap-external-authenticator) and [Directory](./ldap.md#ldap-external-user-directory) -- [Kerberos](./kerberos.md#external-authenticators-kerberos) +- Kerberos [Authenticator](./kerberos.md#external-authenticators-kerberos) diff --git a/docs/en/operations/external-authenticators/kerberos.md b/docs/en/operations/external-authenticators/kerberos.md index 315e688a531..5fe0b2bfc37 100644 --- a/docs/en/operations/external-authenticators/kerberos.md +++ b/docs/en/operations/external-authenticators/kerberos.md @@ -2,47 +2,14 @@ Existing and properly configured ClickHouse users can be authenticated via Kerberos authentication protocol. -Currently, Kerberos can only be used as an external authenticator for existing users, which are defined in `users.xml` or in local access control paths. Those users may only use HTTP requests, and must be able to authenticate using GSS-SPNEGO mechanism. +Currently, Kerberos can only be used as an external authenticator for existing users, which are defined in `users.xml` or in local access control paths. Those users may only use HTTP requests and must be able to authenticate using GSS-SPNEGO mechanism. For this approach, Kerberos must be configured in the system and must be enabled in ClickHouse config. ## Enabling Kerberos in ClickHouse {#enabling-kerberos-in-clickhouse} -To enable Kerberos, presence of a single `kerberos` section in `config.xml` is enough. However, depending on the directions that involve Kerberos, configurations in other places may be necessary. - -Example (goes into `config.xml`): - -```xml - - - - -``` - -...or: - -```xml - - - - HTTP/clickhouse.example.com@EXAMPLE.COM - - -``` - -...or: - -```xml - - - - EXAMPLE.COM - - -``` - -Note that you can define only one `kerberos` section. Presence of multiple `kerberos` sections will force ClickHouse to disable Kerberos authentication. +To enable Kerberos, one should include `kerberos` section in `config.xml`. This section may contain additional parameters. #### Parameters: @@ -53,14 +20,49 @@ Note that you can define only one `kerberos` section. Presence of multiple `kerb - `realm` - a realm, that will be used to restrict authentication to only those requests whose initiator's realm matches it. - This parameter is optional, if omitted, no additional filtering by realm will be applied. -Note that `principal` and `realm` sections cannot be specified at the same time. Presence of both `principal` and `realm` sections will force ClickHouse to disable Kerberos authentication. +Example (goes into `config.xml`): + +```xml + + + + +``` + +With principal specification: + +```xml + + + + HTTP/clickhouse.example.com@EXAMPLE.COM + + +``` + +With filtering by realm: + +```xml + + + + EXAMPLE.COM + + +``` + +!!! warning "Note" + You can define only one `kerberos` section. The presence of multiple `kerberos` sections will force ClickHouse to disable Kerberos authentication. + +!!! warning "Note" + `principal` and `realm` sections cannot be specified at the same time. The presence of both `principal` and `realm` sections will force ClickHouse to disable Kerberos authentication. ## Kerberos as an external authenticator for existing users {#kerberos-as-an-external-authenticator-for-existing-users} -Kerberos can be used as a method for verifying the identity of locally defined users (users defined in `users.xml` or in local access control paths). Currently, **only** requests over HTTP interface can be *kerberized* (via GSS-SPNEGO mechanism). +Kerberos can be used as a method for verifying the identity of locally defined users (users defined in `users.xml` or in local access control paths). Currently, **only** requests over the HTTP interface can be *kerberized* (via GSS-SPNEGO mechanism). -Kerberos principal name format usualy follows this pattern: +Kerberos principal name format usually follows this pattern: - *primary/instance@REALM* @@ -68,7 +70,12 @@ The */instance* part may occur zero or more times. **The *primary* part of the c ### Enabling Kerberos in `users.xml` {#enabling-kerberos-in-users-xml} -In order to enable Kerberos authenication for the user, specify `kerberos` section instead of `password` or similar sections in the user definition. Note that Kerberos authentication cannot be used alongside with any other authentication mechanism. Presence of any other sections like `password` alongside with `kerberos` will force ClickHouse to shutdown. +In order to enable Kerberos authentication for the user, specify `kerberos` section instead of `password` or similar sections in the user definition. + +Parameters: + +- `realm` - a realm that will be used to restrict authentication to only those requests whose initiator's realm matches it. + - This parameter is optional, if omitted, no additional filtering by realm will be applied. Example (goes into `users.xml`): @@ -87,16 +94,15 @@ Example (goes into `users.xml`): ``` -Note, that now, once user `my_user` uses `kerberos`, Kerberos must be enabled in the main `config.xml` file as described previously. +!!! warning "Warning" + Note that Kerberos authentication cannot be used alongside with any other authentication mechanism. The presence of any other sections like `password` alongside `kerberos` will force ClickHouse to shutdown. -Parameters: - -- `realm` - a realm that will be used to restrict authentication to only those requests whose initiator's realm matches it. - - This parameter is optional, if omitted, no additional filtering by realm will be applied. +!!! info "Reminder" + Note, that now, once user `my_user` uses `kerberos`, Kerberos must be enabled in the main `config.xml` file as described previously. ### Enabling Kerberos using SQL {#enabling-kerberos-using-sql} -When SQL-driven Access Control and Account Management is enabled in ClickHouse, users identified by Kerberos can also be created using SQL statements. +When [SQL-driven Access Control and Account Management](../access-rights.md#access-control) is enabled in ClickHouse, users identified by Kerberos can also be created using SQL statements. ```sql CREATE USER my_user IDENTIFIED WITH kerberos REALM 'EXAMPLE.COM' diff --git a/docs/ru/operations/external-authenticators/index.md b/docs/ru/operations/external-authenticators/index.md index e69de29bb2d..53646d1517c 100644 --- a/docs/ru/operations/external-authenticators/index.md +++ b/docs/ru/operations/external-authenticators/index.md @@ -0,0 +1,14 @@ +--- +toc_folder_title: External User Authenticators and Directories +toc_priority: 48 +toc_title: Introduction +--- + +# Внешние аутентификаторы и каталоги {#external-authenticators} + +ClickHouse поддерживает аутентификацию и управление учётными записями с помощью внешних сервисов. + +Поддерживаемые внешние аутентификаторы и внешние каталоги: + +- [LDAP](./ldap.md#external-authenticators-ldap) [аутентификатор](./ldap.md#ldap-external-authenticator) и [External Directory](./ldap.md#ldap-external-user-directory) +- Kerberos [аутентификатор](./kerberos.md#external-authenticators-kerberos) diff --git a/docs/ru/operations/external-authenticators/kerberos.md b/docs/ru/operations/external-authenticators/kerberos.md index 3c6813cec47..b90714d14fd 100644 --- a/docs/ru/operations/external-authenticators/kerberos.md +++ b/docs/ru/operations/external-authenticators/kerberos.md @@ -1,15 +1,25 @@ # Kerberos {#external-authenticators-kerberos} -ClickHouse поддерживает аутентификацию существующих (и правильно сконфигурированных) пользователей с использованием Kerberos. +ClickHouse предоставляет возможность аутентификации существующих (и правильно сконфигурированных) пользователей с использованием Kerberos. -В настоящее время возможна Kerberos-аутентификация уже существующих пользователей, определённых в конфигурационном файле `users.xml` или с помощью выражений SQL. Пользователи, имеющие конфигурацию для аутентификации через Kerberos, могут работать только через HTTP-интерфейс, причём сами клиенты должны иметь возможность аутентификации с использованием механизма GSS-SPNEGO. +В настоящее время возможно использование Kerberos только как внешнего аутентификатора, то есть для аутентификации уже существующих пользователей с помощью Kerberos. Пользователи, настроенные для Kerberos-аутентификации, могут работать с ClickHouse только через HTTP-интерфейс, причём сами клиенты должны иметь возможность аутентификации с использованием механизма GSS-SPNEGO. -Для Kerberos-аутентификации необходимо предварительно корректно настроить Kerberos на стороне клиента, на сервере и в конфигурационных файлах самого ClickHouse. Ниже описана лишь конфигурация ClickHouse. + +!!! info "!!!" + Для Kerberos-аутентификации необходимо предварительно корректно настроить Kerberos на стороне клиента, на сервере и в конфигурационных файлах самого ClickHouse. Ниже описана лишь конфигурация ClickHouse. ## Настройка Kerberos в ClickHouse {#enabling-kerberos-in-clickhouse} -Для того, чтобы включить Kerberos-аутентификацию в ClickHouse, необходимо добавить одну-единственную секцию `kerberos` в `config.xml`. Заметим, что это не единственная необходимая для работы Kerberos настройка. +Для того, чтобы задействовать Kerberos-аутентификацию в ClickHouse, в первую очередь необходимо добавить одну-единственную секцию `kerberos` в `config.xml`. + +В секции могут быть указаны дополнительные параметры: + +- `principal` — задаёт имя принципала (canonical service principal name, SPN), используемое при авторизации ClickHouse на Kerberos-сервере. + - Это опциональный параметр, при его отсутствии будет использовано стандартное имя. + +- `realm` — обеспечивает фильтрацию по реалм (realm). Пользователям, чей реалм не совпадает с указанным, будет отказано в аутентификации. + - Это опциональный параметр, при его отсутствии фильтр по реалм применяться не будет. Примеры, как должен выглядеть файл `config.xml`: @@ -20,7 +30,7 @@ ClickHouse поддерживает аутентификацию существ ``` -...или: +Или, с указанием принципала: ```xml @@ -31,7 +41,7 @@ ClickHouse поддерживает аутентификацию существ ``` -...или: +Или, с фильтрацией по реалм: ```xml @@ -42,17 +52,12 @@ ClickHouse поддерживает аутентификацию существ ``` -**Важно**: в конфигурационном файле может быть не более одной секции `kerberos`. В противном случае, аутентификация с помощью Kerberos будет недоступна для всех пользователей. +!!! Warning "Важно" + В конфигурационном файле не могут быть указаны одновременно оба параметра. В противном случае, аутентификация с помощью Kerberos будет недоступна для всех пользователей. -#### Параметры: +!!! Warning "Важно" + В конфигурационном файле может быть не более одной секции `kerberos`. В противном случае, аутентификация с помощью Kerberos будет отключена для всех пользователей. -- `principal` — задаёт имя принципала (canonical service principal name, SPN), используемое при авторизации ClickHouse на Kerberos-сервере. - - Это опциональный параметр, при его отсутствии будет использовано стандартное имя. - -- `realm` &mdash обеспечивает фильтрацию по реалм (realm). Пользователям, чей реалм не совпадает с указанным, будет отказано в аутентификации. - - Это опциональный параметр, при его отсутствии фильтр по реалм применяться не будет. - -*Важно*: в конфигурационном файле не могут быть указаны одновременно оба параметра. В противном случае, аутентификация с помощью Kerberos будет недоступна для всех пользователей. ## Аутентификация пользователей с помощью Kerberos {#kerberos-as-an-external-authenticator-for-existing-users} @@ -62,13 +67,16 @@ ClickHouse поддерживает аутентификацию существ - *primary/instance@REALM* -Для успешной аутентификации необходимо, чтобы *primary* совпадало с именем пользователя ClickHouse, настроенного для использования Kerberos. +Для успешной аутентификации необходимо, чтобы *primary* совпало с именем пользователя ClickHouse, настроенного для использования Kerberos. ### Настройка Kerberos в `users.xml` {#enabling-kerberos-in-users-xml} -Для того, чтобы пользователь имел возможность аутентификации с помощью Kerberos, достаточно включить секцию `kerberos` в описание пользователя в `users.xml` (например, вместо секции `password` или аналогичной ей). +Для того, чтобы пользователь имел возможность производить аутентификацию с помощью Kerberos, достаточно включить секцию `kerberos` в описание пользователя в `users.xml` (например, вместо секции `password` или аналогичной ей). -**Важно**: если пользователь настроен для Kerberos-аутентификации, другие виды уатентификации будут для него недоступны. Если наряду с `kerberos` в определении пользователя будет указан какой-либо другой способ аутентификации, ClickHouse завершит работу. +В секции могут быть указаны дополнительные параметры: + +- `realm` — обеспечивает фильтрацию по реалм (realm): аутентификация будет возможна только при совпадении реалм клиента с указанным. + - Этот параметр является опциональным, при его отсутствии фильтрация применяться не будет. Пример, как выглядит конфигурация Kerberos в `users.xml`: @@ -87,24 +95,23 @@ ClickHouse поддерживает аутентификацию существ ``` -Note, that now, once user `my_user` uses `kerberos`, Kerberos must be enabled in the main `config.xml` file as described previously. -Ещё раз отметим, что кроме `users.xml`, необходимо также включить Kerberos в `config.xml`. -Параметры: +!!! Warning "Важно" + Если пользователь настроен для Kerberos-аутентификации, другие виды уатентификации будут для него недоступны. Если наряду с `kerberos` в определении пользователя будет указан какой-либо другой способ аутентификации, ClickHouse завершит работу. -- `realm` — обеспечивает фильтрацию по реалм (realm): аутентификация будет возможна только при совпадении реалм клиента с указанным. - - Этот параметр является опциональным, при его отсутствии фильтрация применяться не будет. +!!! info "" + Ещё раз отметим, что кроме `users.xml`, необходимо также включить Kerberos в `config.xml`. ### Настройка Kerberos через SQL {#enabling-kerberos-using-sql} -Пользователей, использующих Kerberos-аутентификацию, можно создавать не только в конфигурационном файле `users.xml`. +Пользователей, использующих Kerberos-аутентификацию, можно создать не только с помощью изменения конфигурационных файлов. Если SQL-ориентированное управление доступом включено в ClickHouse, можно также создать пользователя, работающего через Kerberos, с помощью SQL. ```sql CREATE USER my_user IDENTIFIED WITH kerberos REALM 'EXAMPLE.COM' ``` -...или, без фильтрации по реалм: +Или, без фильтрации по реалм: ```sql CREATE USER my_user IDENTIFIED WITH kerberos diff --git a/docs/ru/operations/external-authenticators/ldap.md b/docs/ru/operations/external-authenticators/ldap.md index 74d0f055f22..f7b927cbcee 100644 --- a/docs/ru/operations/external-authenticators/ldap.md +++ b/docs/ru/operations/external-authenticators/ldap.md @@ -61,10 +61,9 @@ Удалённый LDAP-сервер можно использовать для проверки паролей пользователей, определённых локально, в ClickHouse (в конфигурационных файлах или через SQL-воркфлоу). Для этого при определении пользователя укажите имя предварительно описанного LDAP-сервера вместо секции `password` или ей аналогичной. -При каждой попытке логина ClickHouse будет пытаться подключиться как DN, указанному в параметре `bind_dn` в [Определении сервера LDAP](#ldap-server-definition) -At each login attempt, ClickHouse will try to "bind" to the specified DN defined by the `bind_dn` parameter in the [LDAP server definition](#ldap-server-definition) using the provided credentials, and if successful, the user will be considered authenticated. This is often called a "simple bind" method. +При каждой попытке логина ClickHouse будет пытаться авторизоваться как DN, указанный в параметре `bind_dn` в [Определении сервера LDAP](#ldap-server-definition) с использованием указанных учётных данных. Если попытка увенчается успехом, то пользователь будет аутентифицирован. Инода это также называют простым подсоединением (simple bind). -For example, +Пример конфигурации: ```xml @@ -81,22 +80,23 @@ For example, ``` -Note, that user `my_user` refers to `my_ldap_server`. This LDAP server must be configured in the main `config.xml` file as described previously. - -When SQL-driven [Access Control and Account Management](../access-rights.md#access-control) is enabled in ClickHouse, users that are authenticated by LDAP servers can also be created using the [CRATE USER](../../sql-reference/statements/create/user.md#create-user-statement) statement. +Отметим, что пользователь `my_user` относится к `my_ldap_server`. Этот сервер нужно предварительно определить в главном конфигурационном файле `config.xml`, как описывалось ранее. +Если [управление доступом через SQL-воркфлоу](../access-rights.md#access-control) включено в ClickHouse, то с его помощью также можно создавать пользователей для аутентификации через LDAP, используя выражение [CREATE USER](../../sql-reference/statements/create/user.md#create-user-statement) ```sql CREATE USER my_user IDENTIFIED WITH ldap_server BY 'my_ldap_server' ``` -## LDAP Exernal User Directory {#ldap-external-user-directory} +## Внешний каталог пользователей LDAP {#ldap-external-user-directory} -In addition to the locally defined users, a remote LDAP server can be used as a source of user definitions. In order to achieve this, specify previously defined LDAP server name (see [LDAP Server Definition](#ldap-server-definition)) in the `ldap` section inside the `users_directories` section of the `config.xml` file. +Кроме описанного выше использования LDAP в качестве внешнего аутентификатора, ClickHouse также предоставляет возможность использования LDAP как внешний пользовательские каталог. Для этого необходимо указать предварительно описанное (см. [определение LDAP-сервера](#ldap-server-definition)) имя LDAP-сервера в секции `ldap` внутри `users_directories` в конфигурационном файле `config.xml` -At each login attempt, ClickHouse will try to find the user definition locally and authenticate it as usual, but if the user is not defined, ClickHouse will assume it exists in the external LDAP directory, and will try to "bind" to the specified DN at the LDAP server using the provided credentials. If successful, the user will be considered existing and authenticated. The user will be assigned roles from the list specified in the `roles` section. Additionally, LDAP "search" can be performed and results can be transformed and treated as role names and then be assigned to the user if the `role_mapping` section is also configured. All this implies that the SQL-driven [Access Control and Account Management](../access-rights.md#access-control) is enabled and roles are created using the [CREATE ROLE](../../sql-reference/statements/create/role.md#create-role-statement) statement. +При каждой попытке авторизации ClickHouse сначала пытается найти пользователя среди определённых локально (посредством конфигурационных файлов или SQL-воркфлоу). Если там пользователь не найден, то полагается, что он существует во внешнем каталоге, и ClickHouse попытается выполнить "привязку" к уникальному имени (DN) на LDAP-сервере, используя предоставленные ему данные. Если попытка увенчается успехом, то пользователь будет сочтён существующим и прошедшим аутентификацию. -Example (goes into `config.xml`): +Пользователю будут присвоены роли из списка, приведённого в секции `roles`. Кроме того, может быть выполнен "поиск" в LDAP, результаты которого могут быть использованы как имена ролей и в дальнейшем присвоены пользователю, если также определена секция `role_mapping`. При этом полагается, что включен SQL-воркфлоу (см. [Управление доступом](../access-rights.md#access-control)) и роли уже созданы командой [CREATE ROLE](../../sql-reference/statements/create/role.md#create-role-statement). + +Пример, как это выглядит в `config.xml`: ```xml @@ -121,33 +121,26 @@ Example (goes into `config.xml`): ``` -Note that `my_ldap_server` referred in the `ldap` section inside the `user_directories` section must be a previously -defined LDAP server that is configured in the `config.xml` (see [LDAP Server Definition](#ldap-server-definition)). +!!! info "Note" + LDAP-сервер `my_ldap_server`, упоминаемый в секции `ldap` внутри секции `user_directories`, должен быть предварительно определён в конфигурационном файле `config.xml` (см. [Определение LDAP-сервера](#ldap-server-definition)). -Parameters: +Параметры: -- `server` - one of LDAP server names defined in the `ldap_servers` config section above. - This parameter is mandatory and cannot be empty. -- `roles` - section with a list of locally defined roles that will be assigned to each user retrieved from the LDAP server. - - If no roles are specified here or assigned during role mapping (below), user will not be able - to perform any actions after authentication. -- `role_mapping` - section with LDAP search parameters and mapping rules. - - When a user authenticates, while still bound to LDAP, an LDAP search is performed using `search_filter` - and the name of the logged in user. For each entry found during that search, the value of the specified - attribute is extracted. For each attribute value that has the specified prefix, the prefix is removed, - and the rest of the value becomes the name of a local role defined in ClickHouse, - which is expected to be created beforehand by the [CREATE ROLE](../../sql-reference/statements/create/role.md#create-role-statement) statement. - - There can be multiple `role_mapping` sections defined inside the same `ldap` section. All of them will be applied. - - `base_dn` - template used to construct the base DN for the LDAP search. - - The resulting DN will be constructed by replacing all `{user_name}` and `{bind_dn}` - substrings of the template with the actual user name and bind DN during each LDAP search. - - `scope` - scope of the LDAP search. - - Accepted values are: `base`, `one_level`, `children`, `subtree` (the default). - - `search_filter` - template used to construct the search filter for the LDAP search. - - The resulting filter will be constructed by replacing all `{user_name}`, `{bind_dn}`, and `{base_dn}` - substrings of the template with the actual user name, bind DN, and base DN during each LDAP search. - - Note, that the special characters must be escaped properly in XML. - - `attribute` - attribute name whose values will be returned by the LDAP search. - - `prefix` - prefix, that will be expected to be in front of each string in the original - list of strings returned by the LDAP search. Prefix will be removed from the original - strings and resulting strings will be treated as local role names. Empty, by default. +- `server` - один из LDAP-серверов, определённых в секции `ldap_servers`/ + Этот параметр является обязательным к указанию и не может быть пустым. + +- `roles` - секция, содержащая список локально определённых ролей, присваеваемых каждому пользователю, авторизованному посредством обращения к LDAP-серверу. + - Если ни одна роль здесь не указана или не присвоена пользователю (см. ниже), пользователь не будет иметь возможности совершать никакие действия после аутентификации. +- `role_mapping` - секция, содержащая параметры поиска LDAP и сопоставления ролей. + - При аутентификации пользователя выполняется LDAP-поиск по `search_filter` и имени авторизованного пользователя. Для каждой найденной записи извлекается значение указанного атрибута. Специальный префикс атрибутов ударяется, и полученное значение становится именем роли, определённой в ClickHouse. Ожидается, что эта роль уже создана командой [CREATE ROLE](../../sql-reference/statements/create/role.md#create-role-statement). + - Допускается наличие более одной секции `role_mapping` внутри секции `ldap`. Каждая из них будет принята к сведению. + - `base_dn` - шаблон, используемый для построения основного DN для LDAP-поиска. + - DN получается из шаблона заменой всех подстрок `{user_name}` и `{bind_dn}` на реальное имя пользователя и bind DN (см. [определение LDAP-сервера](#ldap-server-definition)) во время каждого поиска LDAP. + - `scope` - область поиска. + - Возможные значения: `base`, `one_level`, `children`, `subtree` (по умолчанию). + - `search_filter` - шаблон для построения поискового фильтра. + - Фильтр получается заменой всех подстрок `{user_name}`, `{bind_dn}` и `{base_dn}` на реальные значения этих параметров. + - Не забудьте, что все специальные символы (если они есть) должны быть экранированы при их использовании в XML. + - `attribute` - имя атрибут, значения которого будут возвращены после поиска. + - `prefix` - префикс, который ожидается в начале каждой строки исходного списка строк, полученного LDAP-поиском. В дальнейшем префикс будет удалён их строк и полученные строки будут трактованы как имена ролей. + - По умолчанию, этот параметр пустой.