thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-11-07 08:03:15 +00:00
Code Issues Packages Projects Releases Wiki Activity

Updated requirements (markdown and python objects), verification_cooldown parameter tests written in authentications.py and server_config.py, helper functions written in common.py

Browse Source
This commit is contained in:
Tai White 2020-11-17 17:32:51 +01:00
parent a35088d681
commit 29c86da543
5 changed files with 4036 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

610
ldap/authentication/requirements/requirements.md Normal file
View File

@ -0,0 +1,610 @@
# SRS-007 ClickHouse Authentication of Users via LDAP
## Table of Contents
* 1 [Revision History](#revision-history)
* 2 [Introduction](#introduction)
* 3 [Terminology](#terminology)
* 4 [Requirements](#requirements)
* 4.1 [Generic](#generic)
* 4.1.1 [RQ.SRS-007.LDAP.Authentication](#rqsrs-007ldapauthentication)
* 4.1.2 [RQ.SRS-007.LDAP.Authentication.MultipleServers](#rqsrs-007ldapauthenticationmultipleservers)
* 4.1.3 [RQ.SRS-007.LDAP.Authentication.Protocol.PlainText](#rqsrs-007ldapauthenticationprotocolplaintext)
* 4.1.4 [RQ.SRS-007.LDAP.Authentication.Protocol.TLS](#rqsrs-007ldapauthenticationprotocoltls)
* 4.1.5 [RQ.SRS-007.LDAP.Authentication.Protocol.StartTLS](#rqsrs-007ldapauthenticationprotocolstarttls)
* 4.1.6 [RQ.SRS-007.LDAP.Authentication.TLS.Certificate.Validation](#rqsrs-007ldapauthenticationtlscertificatevalidation)
* 4.1.7 [RQ.SRS-007.LDAP.Authentication.TLS.Certificate.SelfSigned](#rqsrs-007ldapauthenticationtlscertificateselfsigned)
* 4.1.8 [RQ.SRS-007.LDAP.Authentication.TLS.Certificate.SpecificCertificationAuthority](#rqsrs-007ldapauthenticationtlscertificatespecificcertificationauthority)
* 4.1.9 [RQ.SRS-007.LDAP.Server.Configuration.Invalid](#rqsrs-007ldapserverconfigurationinvalid)
* 4.1.10 [RQ.SRS-007.LDAP.User.Configuration.Invalid](#rqsrs-007ldapuserconfigurationinvalid)
* 4.1.11 [RQ.SRS-007.LDAP.Authentication.Mechanism.Anonymous](#rqsrs-007ldapauthenticationmechanismanonymous)
* 4.1.12 [RQ.SRS-007.LDAP.Authentication.Mechanism.Unauthenticated](#rqsrs-007ldapauthenticationmechanismunauthenticated)
* 4.1.13 [RQ.SRS-007.LDAP.Authentication.Mechanism.NamePassword](#rqsrs-007ldapauthenticationmechanismnamepassword)
* 4.1.14 [RQ.SRS-007.LDAP.Authentication.Valid](#rqsrs-007ldapauthenticationvalid)
* 4.1.15 [RQ.SRS-007.LDAP.Authentication.Invalid](#rqsrs-007ldapauthenticationinvalid)
* 4.1.16 [RQ.SRS-007.LDAP.Authentication.Invalid.DeletedUser](#rqsrs-007ldapauthenticationinvaliddeleteduser)
* 4.1.17 [RQ.SRS-007.LDAP.Authentication.UsernameChanged](#rqsrs-007ldapauthenticationusernamechanged)
* 4.1.18 [RQ.SRS-007.LDAP.Authentication.PasswordChanged](#rqsrs-007ldapauthenticationpasswordchanged)
* 4.1.19 [RQ.SRS-007.LDAP.Authentication.LDAPServerRestart](#rqsrs-007ldapauthenticationldapserverrestart)
* 4.1.20 [RQ.SRS-007.LDAP.Authentication.ClickHouseServerRestart](#rqsrs-007ldapauthenticationclickhouseserverrestart)
* 4.1.21 [RQ.SRS-007.LDAP.Authentication.Parallel](#rqsrs-007ldapauthenticationparallel)
* 4.1.22 [RQ.SRS-007.LDAP.Authentication.Parallel.ValidAndInvalid](#rqsrs-007ldapauthenticationparallelvalidandinvalid)
* 4.2 [Specific](#specific)
* 4.2.1 [RQ.SRS-007.LDAP.UnreachableServer](#rqsrs-007ldapunreachableserver)
* 4.2.2 [RQ.SRS-007.LDAP.Configuration.Server.Name](#rqsrs-007ldapconfigurationservername)
* 4.2.3 [RQ.SRS-007.LDAP.Configuration.Server.Host](#rqsrs-007ldapconfigurationserverhost)
* 4.2.4 [RQ.SRS-007.LDAP.Configuration.Server.Port](#rqsrs-007ldapconfigurationserverport)
* 4.2.5 [RQ.SRS-007.LDAP.Configuration.Server.Port.Default](#rqsrs-007ldapconfigurationserverportdefault)
* 4.2.6 [RQ.SRS-007.LDAP.Configuration.Server.AuthDN.Prefix](#rqsrs-007ldapconfigurationserverauthdnprefix)
* 4.2.7 [RQ.SRS-007.LDAP.Configuration.Server.AuthDN.Suffix](#rqsrs-007ldapconfigurationserverauthdnsuffix)
* 4.2.8 [RQ.SRS-007.LDAP.Configuration.Server.AuthDN.Value](#rqsrs-007ldapconfigurationserverauthdnvalue)
* 4.2.9 [RQ.SRS-007.LDAP.Configuration.Server.EnableTLS](#rqsrs-007ldapconfigurationserverenabletls)
* 4.2.10 [RQ.SRS-007.LDAP.Configuration.Server.EnableTLS.Options.Default](#rqsrs-007ldapconfigurationserverenabletlsoptionsdefault)
* 4.2.11 [RQ.SRS-007.LDAP.Configuration.Server.EnableTLS.Options.No](#rqsrs-007ldapconfigurationserverenabletlsoptionsno)
* 4.2.12 [RQ.SRS-007.LDAP.Configuration.Server.EnableTLS.Options.Yes](#rqsrs-007ldapconfigurationserverenabletlsoptionsyes)
* 4.2.13 [RQ.SRS-007.LDAP.Configuration.Server.EnableTLS.Options.StartTLS](#rqsrs-007ldapconfigurationserverenabletlsoptionsstarttls)
* 4.2.14 [RQ.SRS-007.LDAP.Configuration.Server.TLSMinimumProtocolVersion](#rqsrs-007ldapconfigurationservertlsminimumprotocolversion)
* 4.2.15 [RQ.SRS-007.LDAP.Configuration.Server.TLSMinimumProtocolVersion.Values](#rqsrs-007ldapconfigurationservertlsminimumprotocolversionvalues)
* 4.2.16 [RQ.SRS-007.LDAP.Configuration.Server.TLSMinimumProtocolVersion.Default](#rqsrs-007ldapconfigurationservertlsminimumprotocolversiondefault)
* 4.2.17 [RQ.SRS-007.LDAP.Configuration.Server.TLSRequireCert](#rqsrs-007ldapconfigurationservertlsrequirecert)
* 4.2.18 [RQ.SRS-007.LDAP.Configuration.Server.TLSRequireCert.Options.Default](#rqsrs-007ldapconfigurationservertlsrequirecertoptionsdefault)
* 4.2.19 [RQ.SRS-007.LDAP.Configuration.Server.TLSRequireCert.Options.Demand](#rqsrs-007ldapconfigurationservertlsrequirecertoptionsdemand)
* 4.2.20 [RQ.SRS-007.LDAP.Configuration.Server.TLSRequireCert.Options.Allow](#rqsrs-007ldapconfigurationservertlsrequirecertoptionsallow)
* 4.2.21 [RQ.SRS-007.LDAP.Configuration.Server.TLSRequireCert.Options.Try](#rqsrs-007ldapconfigurationservertlsrequirecertoptionstry)
* 4.2.22 [RQ.SRS-007.LDAP.Configuration.Server.TLSRequireCert.Options.Never](#rqsrs-007ldapconfigurationservertlsrequirecertoptionsnever)
* 4.2.23 [RQ.SRS-007.LDAP.Configuration.Server.TLSCertFile](#rqsrs-007ldapconfigurationservertlscertfile)
* 4.2.24 [RQ.SRS-007.LDAP.Configuration.Server.TLSKeyFile](#rqsrs-007ldapconfigurationservertlskeyfile)
* 4.2.25 [RQ.SRS-007.LDAP.Configuration.Server.TLSCACertDir](#rqsrs-007ldapconfigurationservertlscacertdir)
* 4.2.26 [RQ.SRS-007.LDAP.Configuration.Server.TLSCACertFile](#rqsrs-007ldapconfigurationservertlscacertfile)
* 4.2.27 [RQ.SRS-007.LDAP.Configuration.Server.TLSCipherSuite](#rqsrs-007ldapconfigurationservertlsciphersuite)
* 4.2.28 [RQ.SRS-007.LDAP.Configuration.Server.VerificationCooldown](#rqsrs-007ldapconfigurationserververificationcooldown)
* 4.2.29 [RQ.SRS-007.LDAP.Configuration.Server.VerificationCooldown.Default](#rqsrs-007ldapconfigurationserververificationcooldowndefault)
* 4.2.30 [RQ.SRS-007.LDAP.Configuration.Server.VerificationCooldown.Invalid](#rqsrs-007ldapconfigurationserververificationcooldowninvalid)
* 4.2.31 [RQ.SRS-007.LDAP.Configuration.Server.Syntax](#rqsrs-007ldapconfigurationserversyntax)
* 4.2.32 [RQ.SRS-007.LDAP.Configuration.User.RBAC](#rqsrs-007ldapconfigurationuserrbac)
* 4.2.33 [RQ.SRS-007.LDAP.Configuration.User.Syntax](#rqsrs-007ldapconfigurationusersyntax)
* 4.2.34 [RQ.SRS-007.LDAP.Configuration.User.Name.Empty](#rqsrs-007ldapconfigurationusernameempty)
* 4.2.35 [RQ.SRS-007.LDAP.Configuration.User.BothPasswordAndLDAP](#rqsrs-007ldapconfigurationuserbothpasswordandldap)
* 4.2.36 [RQ.SRS-007.LDAP.Configuration.User.LDAP.InvalidServerName.NotDefined](#rqsrs-007ldapconfigurationuserldapinvalidservernamenotdefined)
* 4.2.37 [RQ.SRS-007.LDAP.Configuration.User.LDAP.InvalidServerName.Empty](#rqsrs-007ldapconfigurationuserldapinvalidservernameempty)
* 4.2.38 [RQ.SRS-007.LDAP.Configuration.User.OnlyOneServer](#rqsrs-007ldapconfigurationuseronlyoneserver)
* 4.2.39 [RQ.SRS-007.LDAP.Configuration.User.Name.Long](#rqsrs-007ldapconfigurationusernamelong)
* 4.2.40 [RQ.SRS-007.LDAP.Configuration.User.Name.UTF8](#rqsrs-007ldapconfigurationusernameutf8)
* 4.2.41 [RQ.SRS-007.LDAP.Authentication.Username.Empty](#rqsrs-007ldapauthenticationusernameempty)
* 4.2.42 [RQ.SRS-007.LDAP.Authentication.Username.Long](#rqsrs-007ldapauthenticationusernamelong)
* 4.2.43 [RQ.SRS-007.LDAP.Authentication.Username.UTF8](#rqsrs-007ldapauthenticationusernameutf8)
* 4.2.44 [RQ.SRS-007.LDAP.Authentication.Password.Empty](#rqsrs-007ldapauthenticationpasswordempty)
* 4.2.45 [RQ.SRS-007.LDAP.Authentication.Password.Long](#rqsrs-007ldapauthenticationpasswordlong)
* 4.2.46 [RQ.SRS-007.LDAP.Authentication.Password.UTF8](#rqsrs-007ldapauthenticationpasswordutf8)
* 4.2.47 [RQ.SRS-007.LDAP.Authentication.VerificationCooldown.Performance](#rqsrs-007ldapauthenticationverificationcooldownperformance)
* 4.2.48 [RQ.SRS-007.LDAP.Authentication.VerificationCooldown.Reset.ChangeInCoreServerParameters](#rqsrs-007ldapauthenticationverificationcooldownresetchangeincoreserverparameters)
* 4.2.49 [RQ.SRS-007.LDAP.Authentication.VerificationCooldown.Reset.InvalidPassword](#rqsrs-007ldapauthenticationverificationcooldownresetinvalidpassword)
* 5 [References](#references)
## Revision History
This document is stored in an electronic form using [Git] source control management software
hosted in a [GitHub Repository].
All the updates are tracked using the [Git]'s [Revision History].
## Introduction
[ClickHouse] currently does not have any integration with [LDAP].
As the initial step in integrating with [LDAP] this software requirements specification covers
only the requirements to enable authentication of users using an [LDAP] server.
## Terminology
* **CA** -
Certificate Authority ([CA])
* **LDAP** -
Lightweight Directory Access Protocol ([LDAP])
## Requirements
### Generic
#### RQ.SRS-007.LDAP.Authentication
version: 1.0
[ClickHouse] SHALL support user authentication via an [LDAP] server.
#### RQ.SRS-007.LDAP.Authentication.MultipleServers
version: 1.0
[ClickHouse] SHALL support specifying multiple [LDAP] servers that can be used to authenticate
users.
#### RQ.SRS-007.LDAP.Authentication.Protocol.PlainText
version: 1.0
[ClickHouse] SHALL support user authentication using plain text `ldap://` non secure protocol.
#### RQ.SRS-007.LDAP.Authentication.Protocol.TLS
version: 1.0
[ClickHouse] SHALL support user authentication using `SSL/TLS` `ldaps://` secure protocol.
#### RQ.SRS-007.LDAP.Authentication.Protocol.StartTLS
version: 1.0
[ClickHouse] SHALL support user authentication using legacy `StartTLS` protocol which is a
plain text `ldap://` protocol that is upgraded to [TLS].
#### RQ.SRS-007.LDAP.Authentication.TLS.Certificate.Validation
version: 1.0
[ClickHouse] SHALL support certificate validation used for [TLS] connections.
#### RQ.SRS-007.LDAP.Authentication.TLS.Certificate.SelfSigned
version: 1.0
[ClickHouse] SHALL support self-signed certificates for [TLS] connections.
#### RQ.SRS-007.LDAP.Authentication.TLS.Certificate.SpecificCertificationAuthority
version: 1.0
[ClickHouse] SHALL support certificates signed by specific Certification Authority for [TLS] connections.
#### RQ.SRS-007.LDAP.Server.Configuration.Invalid
version: 1.0
[ClickHouse] SHALL return an error and prohibit user login if [LDAP] server configuration is not valid.
#### RQ.SRS-007.LDAP.User.Configuration.Invalid
version: 1.0
[ClickHouse] SHALL return an error and prohibit user login if user configuration is not valid.
#### RQ.SRS-007.LDAP.Authentication.Mechanism.Anonymous
version: 1.0
[ClickHouse] SHALL return an error and prohibit authentication using [Anonymous Authentication Mechanism of Simple Bind]
authentication mechanism.
#### RQ.SRS-007.LDAP.Authentication.Mechanism.Unauthenticated
version: 1.0
[ClickHouse] SHALL return an error and prohibit authentication using [Unauthenticated Authentication Mechanism of Simple Bind]
authentication mechanism.
#### RQ.SRS-007.LDAP.Authentication.Mechanism.NamePassword
version: 1.0
[ClickHouse] SHALL allow authentication using only [Name/Password Authentication Mechanism of Simple Bind]
authentication mechanism.
#### RQ.SRS-007.LDAP.Authentication.Valid
version: 1.0
[ClickHouse] SHALL only allow user authentication using [LDAP] server if and only if
user name and password match [LDAP] server records for the user.
#### RQ.SRS-007.LDAP.Authentication.Invalid
version: 1.0
[ClickHouse] SHALL return an error and prohibit authentication if either user name or password
do not match [LDAP] server records for the user.
#### RQ.SRS-007.LDAP.Authentication.Invalid.DeletedUser
version: 1.0
[ClickHouse] SHALL return an error and prohibit authentication if the user
has been deleted from the [LDAP] server.
#### RQ.SRS-007.LDAP.Authentication.UsernameChanged
version: 1.0
[ClickHouse] SHALL return an error and prohibit authentication if the username is changed
on the [LDAP] server.
#### RQ.SRS-007.LDAP.Authentication.PasswordChanged
version: 1.0
[ClickHouse] SHALL return an error and prohibit authentication if the password
for the user is changed on the [LDAP] server.
#### RQ.SRS-007.LDAP.Authentication.LDAPServerRestart
version: 1.0
[ClickHouse] SHALL support authenticating users after [LDAP] server is restarted.
#### RQ.SRS-007.LDAP.Authentication.ClickHouseServerRestart
version: 1.0
[ClickHouse] SHALL support authenticating users after server is restarted.
#### RQ.SRS-007.LDAP.Authentication.Parallel
version: 1.0
[ClickHouse] SHALL support parallel authentication of users using [LDAP] server.
#### RQ.SRS-007.LDAP.Authentication.Parallel.ValidAndInvalid
version: 1.0
[ClickHouse] SHALL support authentication of valid users and
prohibit authentication of invalid users using [LDAP] server
in parallel without having invalid attempts affecting valid authentications.
### Specific
#### RQ.SRS-007.LDAP.UnreachableServer
version: 1.0
[ClickHouse] SHALL return an error and prohibit user login if [LDAP] server is unreachable.
#### RQ.SRS-007.LDAP.Configuration.Server.Name
version: 1.0
[ClickHouse] SHALL not support empty string as a server name.
#### RQ.SRS-007.LDAP.Configuration.Server.Host
version: 1.0
[ClickHouse] SHALL support `<host>` parameter to specify [LDAP]
server hostname or IP, this parameter SHALL be mandatory and SHALL not be empty.
#### RQ.SRS-007.LDAP.Configuration.Server.Port
version: 1.0
[ClickHouse] SHALL support `<port>` parameter to specify [LDAP] server port.
#### RQ.SRS-007.LDAP.Configuration.Server.Port.Default
version: 1.0
[ClickHouse] SHALL use default port number `636` if `enable_tls` is set to `yes` or `389` otherwise.
#### RQ.SRS-007.LDAP.Configuration.Server.AuthDN.Prefix
version: 1.0
[ClickHouse] SHALL support `<auth_dn_prefix>` parameter to specify the prefix
of value used to construct the DN to bound to during authentication via [LDAP] server.
#### RQ.SRS-007.LDAP.Configuration.Server.AuthDN.Suffix
version: 1.0
[ClickHouse] SHALL support `<auth_dn_suffix>` parameter to specify the suffix
of value used to construct the DN to bound to during authentication via [LDAP] server.
#### RQ.SRS-007.LDAP.Configuration.Server.AuthDN.Value
version: 1.0
[ClickHouse] SHALL construct DN as `auth_dn_prefix + escape(user_name) + auth_dn_suffix` string.
> This implies that auth_dn_suffix should usually have comma ',' as its first non-space character.
#### RQ.SRS-007.LDAP.Configuration.Server.EnableTLS
version: 1.0
[ClickHouse] SHALL support `<enable_tls>` parameter to trigger the use of secure connection to the [LDAP] server.
#### RQ.SRS-007.LDAP.Configuration.Server.EnableTLS.Options.Default
version: 1.0
[ClickHouse] SHALL use `yes` value as the default for `<enable_tls>` parameter
to enable SSL/TLS `ldaps://` protocol.
#### RQ.SRS-007.LDAP.Configuration.Server.EnableTLS.Options.No
version: 1.0
[ClickHouse] SHALL support specifying `no` as the value of `<enable_tls>` parameter to enable
plain text `ldap://` protocol.
#### RQ.SRS-007.LDAP.Configuration.Server.EnableTLS.Options.Yes
version: 1.0
[ClickHouse] SHALL support specifying `yes` as the value of `<enable_tls>` parameter to enable
SSL/TLS `ldaps://` protocol.
#### RQ.SRS-007.LDAP.Configuration.Server.EnableTLS.Options.StartTLS
version: 1.0
[ClickHouse] SHALL support specifying `starttls` as the value of `<enable_tls>` parameter to enable
legacy `StartTLS` protocol that used plain text `ldap://` protocol, upgraded to [TLS].
#### RQ.SRS-007.LDAP.Configuration.Server.TLSMinimumProtocolVersion
version: 1.0
[ClickHouse] SHALL support `<tls_minimum_protocol_version>` parameter to specify
the minimum protocol version of SSL/TLS.
#### RQ.SRS-007.LDAP.Configuration.Server.TLSMinimumProtocolVersion.Values
version: 1.0
[ClickHouse] SHALL support specifying `ssl2`, `ssl3`, `tls1.0`, `tls1.1`, and `tls1.2`
as a value of the `<tls_minimum_protocol_version>` parameter.
#### RQ.SRS-007.LDAP.Configuration.Server.TLSMinimumProtocolVersion.Default
version: 1.0
[ClickHouse] SHALL set `tls1.2` as the default value of the `<tls_minimum_protocol_version>` parameter.
#### RQ.SRS-007.LDAP.Configuration.Server.TLSRequireCert
version: 1.0
[ClickHouse] SHALL support `<tls_require_cert>` parameter to specify [TLS] peer
certificate verification behavior.
#### RQ.SRS-007.LDAP.Configuration.Server.TLSRequireCert.Options.Default
version: 1.0
[ClickHouse] SHALL use `demand` value as the default for the `<tls_require_cert>` parameter.
#### RQ.SRS-007.LDAP.Configuration.Server.TLSRequireCert.Options.Demand
version: 1.0
[ClickHouse] SHALL support specifying `demand` as the value of `<tls_require_cert>` parameter to
enable requesting of client certificate. If no certificate is provided, or a bad certificate is
provided, the session SHALL be immediately terminated.
#### RQ.SRS-007.LDAP.Configuration.Server.TLSRequireCert.Options.Allow
version: 1.0
[ClickHouse] SHALL support specifying `allow` as the value of `<tls_require_cert>` parameter to
enable requesting of client certificate. If no
certificate is provided, the session SHALL proceed normally.
If a bad certificate is provided, it SHALL be ignored and the session SHALL proceed normally.
#### RQ.SRS-007.LDAP.Configuration.Server.TLSRequireCert.Options.Try
version: 1.0
[ClickHouse] SHALL support specifying `try` as the value of `<tls_require_cert>` parameter to
enable requesting of client certificate. If no certificate is provided, the session
SHALL proceed normally. If a bad certificate is provided, the session SHALL be
immediately terminated.
#### RQ.SRS-007.LDAP.Configuration.Server.TLSRequireCert.Options.Never
version: 1.0
[ClickHouse] SHALL support specifying `never` as the value of `<tls_require_cert>` parameter to
disable requesting of client certificate.
#### RQ.SRS-007.LDAP.Configuration.Server.TLSCertFile
version: 1.0
[ClickHouse] SHALL support `<tls_cert_file>` to specify the path to certificate file used by
[ClickHouse] to establish connection with the [LDAP] server.
#### RQ.SRS-007.LDAP.Configuration.Server.TLSKeyFile
version: 1.0
[ClickHouse] SHALL support `<tls_key_file>` to specify the path to key file for the certificate
specified by the `<tls_cert_file>` parameter.
#### RQ.SRS-007.LDAP.Configuration.Server.TLSCACertDir
version: 1.0
[ClickHouse] SHALL support `<tls_ca_cert_dir>` parameter to specify to a path to
the directory containing [CA] certificates used to verify certificates provided by the [LDAP] server.
#### RQ.SRS-007.LDAP.Configuration.Server.TLSCACertFile
version: 1.0
[ClickHouse] SHALL support `<tls_ca_cert_file>` parameter to specify a path to a specific
[CA] certificate file used to verify certificates provided by the [LDAP] server.
#### RQ.SRS-007.LDAP.Configuration.Server.TLSCipherSuite
version: 1.0
[ClickHouse] SHALL support `tls_cipher_suite` parameter to specify allowed cipher suites.
The value SHALL use the same format as the `ciphersuites` in the [OpenSSL Ciphers].
For example,
```xml
<tls_cipher_suite>ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:AES256-GCM-SHA384</tls_cipher_suite>
```
The available suites SHALL depend on the [OpenSSL] library version and variant used to build
[ClickHouse] and therefore might change.
#### RQ.SRS-007.LDAP.Configuration.Server.VerificationCooldown
version: 1.0
[ClickHouse] SHALL support `verification_cooldown` parameter in the [LDAP] server configuration section
that SHALL define a period of time, in seconds, after a successful bind attempt, during which a user SHALL be assumed
to be successfully authenticated for all consecutive requests without contacting the [LDAP] server.
After period of time since the last successful attempt expires then on the authentication attempt
SHALL result in contacting the [LDAP] server to verify the username and password.
#### RQ.SRS-007.LDAP.Configuration.Server.VerificationCooldown.Default
version: 1.0
[ClickHouse] `verification_cooldown` parameter in the [LDAP] server configuration section
SHALL have a default value of `0` that disables caching and forces contacting
the [LDAP] server for each authentication request.
#### RQ.SRS-007.LDAP.Configuration.Server.VerificationCooldown.Invalid
version: 1.0
[Clickhouse] SHALL return an error if the value provided for the `verification_cooldown` parameter is not a valid positive integer.
For example:
* negative integer
* string
* empty value
* extremely large positive value (overflow)
* extremely large negative value (overflow)
The error SHALL appear in the log and SHALL be similar to the following:
```bash
<Error> Access(user directories): Could not parse LDAP server `openldap1`: Poco::Exception. Code: 1000, e.code() = 0, e.displayText() = Syntax error: Not a valid unsigned integer: *input value*
```
#### RQ.SRS-007.LDAP.Configuration.Server.Syntax
version: 2.0
[ClickHouse] SHALL support the following example syntax to create an entry for an [LDAP] server inside the `config.xml`
configuration file or of any configuration file inside the `config.d` directory.
```xml
<yandex>
<my_ldap_server>
<host>localhost</host>
<port>636</port>
<auth_dn_prefix>cn=</auth_dn_prefix>
<auth_dn_suffix>, ou=users, dc=example, dc=com</auth_dn_suffix>
<verification_cooldown>0</verification_cooldown>
<enable_tls>yes</enable_tls>
<tls_minimum_protocol_version>tls1.2</tls_minimum_protocol_version>
<tls_require_cert>demand</tls_require_cert>
<tls_cert_file>/path/to/tls_cert_file</tls_cert_file>
<tls_key_file>/path/to/tls_key_file</tls_key_file>
<tls_ca_cert_file>/path/to/tls_ca_cert_file</tls_ca_cert_file>
<tls_ca_cert_dir>/path/to/tls_ca_cert_dir</tls_ca_cert_dir>
<tls_cipher_suite>ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:AES256-GCM-SHA384</tls_cipher_suite>
</my_ldap_server>
</yandex>
```
#### RQ.SRS-007.LDAP.Configuration.User.RBAC
version: 1.0
[ClickHouse] SHALL support creating users identified using an [LDAP] server using
the following RBAC command
```sql
CREATE USER name IDENTIFIED WITH ldap_server BY 'server_name'
```
#### RQ.SRS-007.LDAP.Configuration.User.Syntax
version: 1.0
[ClickHouse] SHALL support the following example syntax to create a user that is authenticated using
an [LDAP] server inside the `users.xml` file or any configuration file inside the `users.d` directory.
```xml
<yandex>
<users>
<user_name>
<ldap>
<server>my_ldap_server</server>
</ldap>
</user_name>
</users>
</yandex>
```
#### RQ.SRS-007.LDAP.Configuration.User.Name.Empty
version: 1.0
[ClickHouse] SHALL not support empty string as a user name.
#### RQ.SRS-007.LDAP.Configuration.User.BothPasswordAndLDAP
version: 1.0
[ClickHouse] SHALL throw an error if `<ldap>` is specified for the user and at the same
time user configuration contains any of the `<password*>` entries.
#### RQ.SRS-007.LDAP.Configuration.User.LDAP.InvalidServerName.NotDefined
version: 1.0
[ClickHouse] SHALL throw an error during any authentification attempt
if the name of the [LDAP] server used inside the `<ldap>` entry
is not defined in the `<ldap_servers>` section.
#### RQ.SRS-007.LDAP.Configuration.User.LDAP.InvalidServerName.Empty
version: 1.0
[ClickHouse] SHALL throw an error during any authentification attempt
if the name of the [LDAP] server used inside the `<ldap>` entry
is empty.
#### RQ.SRS-007.LDAP.Configuration.User.OnlyOneServer
version: 1.0
[ClickHouse] SHALL support specifying only one [LDAP] server for a given user.
#### RQ.SRS-007.LDAP.Configuration.User.Name.Long
version: 1.0
[ClickHouse] SHALL support long user names of at least 256 bytes
to specify users that can be authenticated using an [LDAP] server.
#### RQ.SRS-007.LDAP.Configuration.User.Name.UTF8
version: 1.0
[ClickHouse] SHALL support user names that contain [UTF-8] characters.
#### RQ.SRS-007.LDAP.Authentication.Username.Empty
version: 1.0
[ClickHouse] SHALL not support authenticating users with empty username.
#### RQ.SRS-007.LDAP.Authentication.Username.Long
version: 1.0
[ClickHouse] SHALL support authenticating users with a long username of at least 256 bytes.
#### RQ.SRS-007.LDAP.Authentication.Username.UTF8
version: 1.0
[ClickHouse] SHALL support authentication users with a username that contains [UTF-8] characters.
#### RQ.SRS-007.LDAP.Authentication.Password.Empty
version: 1.0
[ClickHouse] SHALL not support authenticating users with empty passwords
even if an empty password is valid for the user and
is allowed by the [LDAP] server.
#### RQ.SRS-007.LDAP.Authentication.Password.Long
version: 1.0
[ClickHouse] SHALL support long password of at least 256 bytes
that can be used to authenticate users using an [LDAP] server.
#### RQ.SRS-007.LDAP.Authentication.Password.UTF8
version: 1.0
[ClickHouse] SHALL support [UTF-8] characters in passwords
used to authenticate users using an [LDAP] server.
#### RQ.SRS-007.LDAP.Authentication.VerificationCooldown.Performance
version: 1.0
[ClickHouse] SHALL provide better login performance of [LDAP] authenticated users
when `verification_cooldown` parameter is set to a positive value when comparing
to the the case when `verification_cooldown` is turned off either for a single user or multiple users
making a large number of repeated requests.
#### RQ.SRS-007.LDAP.Authentication.VerificationCooldown.Reset.ChangeInCoreServerParameters
version: 1.0
[ClickHouse] SHALL reset any currently cached [LDAP] authentication bind requests enabled by the
`verification_cooldown` parameter in the [LDAP] server configuration section
if either `host`, `port`, `auth_dn_prefix`, or `auth_dn_suffix` parameter values
change in the configuration file. The reset SHALL cause any subsequent authentication attempts for any user
to result in contacting the [LDAP] server to verify user's username and password.
#### RQ.SRS-007.LDAP.Authentication.VerificationCooldown.Reset.InvalidPassword
version: 1.0
[ClickHouse] SHALL reset current cached [LDAP] authentication bind request enabled by the
`verification_cooldown` parameter in the [LDAP] server configuration section
for the user if the password provided in the current authentication attempt does not match
the valid password provided during the first successful authentication request that was cached
for this exact user. The reset SHALL cause the next authentication attempt for this user
to result in contacting the [LDAP] server to verify user's username and password.
## References
* **ClickHouse:** https://clickhouse.tech
[Anonymous Authentication Mechanism of Simple Bind]: https://ldapwiki.com/wiki/Simple%20Authentication#section-Simple+Authentication-AnonymousAuthenticationMechanismOfSimpleBind
[Unauthenticated Authentication Mechanism of Simple Bind]: https://ldapwiki.com/wiki/Simple%20Authentication#section-Simple+Authentication-UnauthenticatedAuthenticationMechanismOfSimpleBind
[Name/Password Authentication Mechanism of Simple Bind]: https://ldapwiki.com/wiki/Simple%20Authentication#section-Simple+Authentication-NamePasswordAuthenticationMechanismOfSimpleBind
[UTF-8]: https://en.wikipedia.org/wiki/UTF-8
[OpenSSL]: https://www.openssl.org/
[OpenSSL Ciphers]: https://www.openssl.org/docs/manmaster/man1/openssl-ciphers.html
[CA]: https://en.wikipedia.org/wiki/Certificate_authority
[TLS]: https://en.wikipedia.org/wiki/Transport_Layer_Security
[LDAP]: https://en.wikipedia.org/wiki/Lightweight_Directory_Access_Protocol
[ClickHouse]: https://clickhouse.tech
[GitHub]: https://github.com
[GitHub Repository]: https://github.com/ClickHouse/ClickHouse/blob/master/tests/testflows/ldap/authentication/requirements/requirements.md
[Revision History]: https://github.com/ClickHouse/ClickHouse/commits/master/tests/testflows/ldap/authentication/requirements/requirements.md
[Git]: https://git-scm.com/

1687
ldap/authentication/requirements/requirements.py Normal file
View File

File diff suppressed because it is too large Load Diff

969
ldap/authentication/tests/authentications.py Normal file
View File

@ -0,0 +1,969 @@
# -*- coding: utf-8 -*-
import random
import time
from multiprocessing.dummy import Pool
from testflows.core import *
from testflows.asserts import error
from ldap.authentication.tests.common import *
from ldap.authentication.requirements import *
servers = {
"openldap1": {
"host": "openldap1",
"port": "389",
"enable_tls": "no",
"auth_dn_prefix": "cn=",
"auth_dn_suffix": ",ou=users,dc=company,dc=com"
},
"openldap2": {
"host": "openldap2",
"port": "636",
"enable_tls": "yes",
"auth_dn_prefix": "cn=",
"auth_dn_suffix": ",ou=users,dc=company,dc=com",
"tls_require_cert": "never",
}
}
@TestStep(When)
@Name("I login as {username} and execute query")
@Args(format_name=True)
def login_and_execute_query(self, username, password, exitcode=None, message=None, steps=True):
"""Execute query as some user.
"""
self.context.node.query("SELECT 1",
settings=[("user", username), ("password", password)],
exitcode=exitcode or 0,
message=message, steps=steps)
@TestScenario
def add_user_to_ldap_and_login(self, server, user=None, ch_user=None, login=None, exitcode=None, message=None, rbac=False):
"""Add user to LDAP and ClickHouse and then try to login.
"""
self.context.ldap_node = self.context.cluster.node(server)
if ch_user is None:
ch_user = {}
if login is None:
login = {}
if user is None:
user = {"cn": "myuser", "userpassword": "myuser"}
with ldap_user(**user) as user:
ch_user["username"] = ch_user.get("username", user["cn"])
ch_user["server"] = ch_user.get("server", user["_server"])
with ldap_authenticated_users(ch_user, config_file=f"ldap_users_{getuid()}.xml", restart=True, rbac=rbac):
username = login.get("username", user["cn"])
password = login.get("password", user["userpassword"])
login_and_execute_query(username=username, password=password, exitcode=exitcode, message=message)
@TestScenario
@Requirements(
RQ_SRS_007_LDAP_Authentication_Parallel("1.0"),
RQ_SRS_007_LDAP_Authentication_Parallel_ValidAndInvalid("1.0")
)
def parallel_login(self, server, user_count=10, timeout=200, rbac=False):
"""Check that login of valid and invalid LDAP authenticated users works in parallel.
"""
self.context.ldap_node = self.context.cluster.node(server)
user = None
users = [{"cn": f"parallel_user{i}", "userpassword": randomword(20)} for i in range(user_count)]
with ldap_users(*users):
with ldap_authenticated_users(*[{"username": user["cn"], "server": server} for user in users], rbac=rbac):
def login_with_valid_username_and_password(users, i, iterations=10):
with When(f"valid users try to login #{i}"):
for i in range(iterations):
random_user = users[random.randint(0, len(users)-1)]
login_and_execute_query(username=random_user["cn"], password=random_user["userpassword"], steps=False)
def login_with_valid_username_and_invalid_password(users, i, iterations=10):
with When(f"users try to login with valid username and invalid password #{i}"):
for i in range(iterations):
random_user = users[random.randint(0, len(users)-1)]
login_and_execute_query(username=random_user["cn"],
password=(random_user["userpassword"] + randomword(1)),
exitcode=4,
message=f"DB::Exception: {random_user['cn']}: Authentication failed: password is incorrect or there is no user with such name",
steps=False)
def login_with_invalid_username_and_valid_password(users, i, iterations=10):
with When(f"users try to login with invalid username and valid password #{i}"):
for i in range(iterations):
random_user = dict(users[random.randint(0, len(users)-1)])
random_user["cn"] += randomword(1)
login_and_execute_query(username=random_user["cn"],
password=random_user["userpassword"],
exitcode=4,
message=f"DB::Exception: {random_user['cn']}: Authentication failed: password is incorrect or there is no user with such name",
steps=False)
with When("I login in parallel"):
p = Pool(15)
tasks = []
for i in range(5):
tasks.append(p.apply_async(login_with_valid_username_and_password, (users, i, 50,)))
tasks.append(p.apply_async(login_with_valid_username_and_invalid_password, (users, i, 50,)))
tasks.append(p.apply_async(login_with_invalid_username_and_valid_password, (users, i, 50,)))
with Then("it should work"):
for task in tasks:
task.get(timeout=timeout)
@TestScenario
@Requirements(
RQ_SRS_007_LDAP_Authentication_Invalid("1.0"),
RQ_SRS_007_LDAP_Authentication_Invalid_DeletedUser("1.0")
)
def login_after_user_is_deleted_from_ldap(self, server, rbac=False):
"""Check that login fails after user is deleted from LDAP.
"""
self.context.ldap_node = self.context.cluster.node(server)
user = None
try:
with Given(f"I add user to LDAP"):
user = {"cn": "myuser", "userpassword": "myuser"}
user = add_user_to_ldap(**user)
with ldap_authenticated_users({"username": user["cn"], "server": server}, config_file=f"ldap_users_{getuid()}.xml",
restart=True, rbac=rbac):
login_and_execute_query(username=user["cn"], password=user["userpassword"])
with When("I delete this user from LDAP"):
delete_user_from_ldap(user)
with Then("when I try to login again it should fail"):
login_and_execute_query(username=user["cn"], password=user["userpassword"],
exitcode=4,
message=f"DB::Exception: {user['cn']}: Authentication failed: password is incorrect or there is no user with such name"
)
finally:
with Finally("I make sure LDAP user is deleted"):
if user is not None:
delete_user_from_ldap(user, exitcode=None)
@TestScenario
@Requirements(
RQ_SRS_007_LDAP_Authentication_Invalid("1.0"),
RQ_SRS_007_LDAP_Authentication_PasswordChanged("1.0")
)
def login_after_user_password_changed_in_ldap(self, server, rbac=False):
"""Check that login fails after user password is changed in LDAP.
"""
self.context.ldap_node = self.context.cluster.node(server)
user = None
try:
with Given(f"I add user to LDAP"):
user = {"cn": "myuser", "userpassword": "myuser"}
user = add_user_to_ldap(**user)
with ldap_authenticated_users({"username": user["cn"], "server": server}, config_file=f"ldap_users_{getuid()}.xml",
restart=True, rbac=rbac):
login_and_execute_query(username=user["cn"], password=user["userpassword"])
with When("I change user password in LDAP"):
change_user_password_in_ldap(user, "newpassword")
with Then("when I try to login again it should fail"):
login_and_execute_query(username=user["cn"], password=user["userpassword"],
exitcode=4,
message=f"DB::Exception: {user['cn']}: Authentication failed: password is incorrect or there is no user with such name"
)
with And("when I try to login with the new password it should work"):
login_and_execute_query(username=user["cn"], password="newpassword")
finally:
with Finally("I make sure LDAP user is deleted"):
if user is not None:
delete_user_from_ldap(user, exitcode=None)
@TestScenario
@Requirements(
RQ_SRS_007_LDAP_Authentication_Invalid("1.0"),
RQ_SRS_007_LDAP_Authentication_UsernameChanged("1.0")
)
def login_after_user_cn_changed_in_ldap(self, server, rbac=False):
"""Check that login fails after user cn is changed in LDAP.
"""
self.context.ldap_node = self.context.cluster.node(server)
user = None
new_user = None
try:
with Given(f"I add user to LDAP"):
user = {"cn": "myuser", "userpassword": "myuser"}
user = add_user_to_ldap(**user)
with ldap_authenticated_users({"username": user["cn"], "server": server},
config_file=f"ldap_users_{getuid()}.xml", restart=True, rbac=rbac):
login_and_execute_query(username=user["cn"], password=user["userpassword"])
with When("I change user password in LDAP"):
new_user = change_user_cn_in_ldap(user, "myuser2")
with Then("when I try to login again it should fail"):
login_and_execute_query(username=user["cn"], password=user["userpassword"],
exitcode=4,
message=f"DB::Exception: {user['cn']}: Authentication failed: password is incorrect or there is no user with such name"
)
finally:
with Finally("I make sure LDAP user is deleted"):
if new_user is not None:
delete_user_from_ldap(new_user, exitcode=None)
@TestScenario
@Requirements(
RQ_SRS_007_LDAP_Authentication_Valid("1.0"),
RQ_SRS_007_LDAP_Authentication_LDAPServerRestart("1.0")
)
def login_after_ldap_server_is_restarted(self, server, timeout=60, rbac=False):
"""Check that login succeeds after LDAP server is restarted.
"""
self.context.ldap_node = self.context.cluster.node(server)
user = None
try:
with Given(f"I add user to LDAP"):
user = {"cn": "myuser", "userpassword": getuid()}
user = add_user_to_ldap(**user)
with ldap_authenticated_users({"username": user["cn"], "server": server}, rbac=rbac):
login_and_execute_query(username=user["cn"], password=user["userpassword"])
with When("I restart LDAP server"):
self.context.ldap_node.restart()
with Then("I try to login until it works", description=f"timeout {timeout} sec"):
started = time.time()
while True:
r = self.context.node.query("SELECT 1",
settings=[("user", user["cn"]), ("password", user["userpassword"])],
no_checks=True)
if r.exitcode == 0:
break
assert time.time() - started < timeout, error(r.output)
finally:
with Finally("I make sure LDAP user is deleted"):
if user is not None:
delete_user_from_ldap(user, exitcode=None)
@TestScenario
@Requirements(
RQ_SRS_007_LDAP_Authentication_Valid("1.0"),
RQ_SRS_007_LDAP_Authentication_ClickHouseServerRestart("1.0")
)
def login_after_clickhouse_server_is_restarted(self, server, timeout=60, rbac=False):
"""Check that login succeeds after ClickHouse server is restarted.
"""
self.context.ldap_node = self.context.cluster.node(server)
user = None
try:
with Given(f"I add user to LDAP"):
user = {"cn": "myuser", "userpassword": getuid()}
user = add_user_to_ldap(**user)
with ldap_authenticated_users({"username": user["cn"], "server": server}, rbac=rbac):
login_and_execute_query(username=user["cn"], password=user["userpassword"])
with When("I restart ClickHouse server"):
self.context.node.restart()
with Then("I try to login until it works", description=f"timeout {timeout} sec"):
started = time.time()
while True:
r = self.context.node.query("SELECT 1",
settings=[("user", user["cn"]), ("password", user["userpassword"])],
no_checks=True)
if r.exitcode == 0:
break
assert time.time() - started < timeout, error(r.output)
finally:
with Finally("I make sure LDAP user is deleted"):
if user is not None:
delete_user_from_ldap(user, exitcode=None)
@TestScenario
@Requirements(
RQ_SRS_007_LDAP_Authentication_Invalid("1.0"),
RQ_SRS_007_LDAP_Authentication_Password_Empty("1.0")
)
def valid_username_with_valid_empty_password(self, server, rbac=False):
"""Check that we can't login using valid username that has empty password.
"""
user = {"cn": "empty_password", "userpassword": ""}
exitcode = 4
message = f"DB::Exception: {user['cn']}: Authentication failed: password is incorrect or there is no user with such name"
add_user_to_ldap_and_login(user=user, exitcode=exitcode, message=message, server=server, rbac=rbac)
@TestScenario
@Requirements(
RQ_SRS_007_LDAP_Authentication_Invalid("1.0"),
RQ_SRS_007_LDAP_Authentication_Password_Empty("1.0")
)
def valid_username_and_invalid_empty_password(self, server, rbac=False):
"""Check that we can't login using valid username but invalid empty password.
"""
username = "user_non_empty_password"
user = {"cn": username, "userpassword": username}
login = {"password": ""}
exitcode = 4
message = f"DB::Exception: {username}: Authentication failed: password is incorrect or there is no user with such name"
add_user_to_ldap_and_login(user=user, login=login, exitcode=exitcode, message=message, server=server, rbac=rbac)
@TestScenario
@Requirements(
RQ_SRS_007_LDAP_Authentication_Valid("1.0")
)
def valid_username_and_password(self, server, rbac=False):
"""Check that we can login using valid username and password.
"""
username = "valid_username_and_password"
user = {"cn": username, "userpassword": username}
with When(f"I add user {username} to LDAP and try to login"):
add_user_to_ldap_and_login(user=user, server=server, rbac=rbac)
@TestScenario
@Requirements(
RQ_SRS_007_LDAP_Authentication_Invalid("1.0")
)
def valid_username_and_password_invalid_server(self, server=None, rbac=False):
"""Check that we can't login using valid username and valid
password but for a different server.
"""
self.context.ldap_node = self.context.cluster.node("openldap1")
user = {"username": "user2", "userpassword": "user2", "server": "openldap1"}
exitcode = 4
message = f"DB::Exception: user2: Authentication failed: password is incorrect or there is no user with such name"
with ldap_authenticated_users(user, config_file=f"ldap_users_{getuid()}.xml", restart=True, rbac=rbac):
login_and_execute_query(username="user2", password="user2", exitcode=exitcode, message=message)
@TestScenario
@Requirements(
RQ_SRS_007_LDAP_Authentication_Valid("1.0"),
RQ_SRS_007_LDAP_Authentication_Username_Long("1.0"),
RQ_SRS_007_LDAP_Configuration_User_Name_Long("1.0")
)
def valid_long_username_and_short_password(self, server, rbac=False):
"""Check that we can login using valid very long username and short password.
"""
username = "long_username_12345678901234567890123456789012345678901234567890123456789012345678901234567890"
user = {"cn": username, "userpassword": "long_username"}
add_user_to_ldap_and_login(user=user, server=server, rbac=rbac)
@TestScenario
@Requirements(
RQ_SRS_007_LDAP_Authentication_Invalid("1.0")
)
def invalid_long_username_and_valid_short_password(self, server, rbac=False):
"""Check that we can't login using slightly invalid long username but valid password.
"""
username = "long_username_12345678901234567890123456789012345678901234567890123456789012345678901234567890"
user = {"cn": username, "userpassword": "long_username"}
login = {"username": f"{username}?"}
exitcode = 4
message=f"DB::Exception: {login['username']}: Authentication failed: password is incorrect or there is no user with such name"
add_user_to_ldap_and_login(user=user, login=login, exitcode=exitcode, message=message, server=server, rbac=rbac)
@TestScenario
@Requirements(
RQ_SRS_007_LDAP_Authentication_Valid("1.0"),
RQ_SRS_007_LDAP_Authentication_Password_Long("1.0")
)
def valid_short_username_and_long_password(self, server, rbac=False):
"""Check that we can login using valid short username with very long password.
"""
username = "long_password"
user = {"cn": username, "userpassword": "long_password_12345678901234567890123456789012345678901234567890123456789012345678901234567890"}
add_user_to_ldap_and_login(user=user, server=server, rbac=rbac)
@TestScenario
@Requirements(
RQ_SRS_007_LDAP_Authentication_Invalid("1.0")
)
def valid_short_username_and_invalid_long_password(self, server, rbac=False):
"""Check that we can't login using valid short username and invalid long password.
"""
username = "long_password"
user = {"cn": username, "userpassword": "long_password_12345678901234567890123456789012345678901234567890123456789012345678901234567890"}
login = {"password": user["userpassword"] + "1"}
exitcode = 4
message=f"DB::Exception: {username}: Authentication failed: password is incorrect or there is no user with such name"
add_user_to_ldap_and_login(user=user, login=login, exitcode=exitcode, message=message, server=server, rbac=rbac)
@TestScenario
@Requirements(
RQ_SRS_007_LDAP_Authentication_Invalid("1.0")
)
def valid_username_and_invalid_password(self, server, rbac=False):
"""Check that we can't login using valid username and invalid password.
"""
username = "valid_username_and_invalid_password"
user = {"cn": username, "userpassword": username}
login = {"password": user["userpassword"] + "1"}
exitcode = 4
message=f"DB::Exception: {username}: Authentication failed: password is incorrect or there is no user with such name"
add_user_to_ldap_and_login(user=user, login=login, exitcode=exitcode, message=message, server=server, rbac=rbac)
@TestScenario
@Requirements(
RQ_SRS_007_LDAP_Authentication_Invalid("1.0")
)
def invalid_username_and_valid_password(self, server, rbac=False):
"""Check that we can't login using slightly invalid username but valid password.
"""
username = "invalid_username_and_valid_password"
user = {"cn": username, "userpassword": username}
login = {"username": user["cn"] + "1"}
exitcode = 4
message=f"DB::Exception: {login['username']}: Authentication failed: password is incorrect or there is no user with such name"
add_user_to_ldap_and_login(user=user, login=login, exitcode=exitcode, message=message, server=server, rbac=rbac)
@TestScenario
@Requirements(
RQ_SRS_007_LDAP_Authentication_Valid("1.0"),
RQ_SRS_007_LDAP_Authentication_Username_UTF8("1.0"),
RQ_SRS_007_LDAP_Configuration_User_Name_UTF8("1.0")
)
def valid_utf8_username_and_ascii_password(self, server, rbac=False):
"""Check that we can login using valid utf-8 username with ascii password.
"""
username = "utf8_username_Gãńdåłf_Thê_Gręât"
user = {"cn": username, "userpassword": "utf8_username"}
add_user_to_ldap_and_login(user=user, server=server, rbac=rbac)
@TestScenario
@Requirements(
RQ_SRS_007_LDAP_Authentication_Valid("1.0"),
RQ_SRS_007_LDAP_Authentication_Password_UTF8("1.0")
)
def valid_ascii_username_and_utf8_password(self, server, rbac=False):
"""Check that we can login using valid ascii username with utf-8 password.
"""
username = "utf8_password"
user = {"cn": username, "userpassword": "utf8_password_Gãńdåłf_Thê_Gręât"}
add_user_to_ldap_and_login(user=user, server=server, rbac=rbac)
@TestScenario
def empty_username_and_empty_password(self, server=None, rbac=False):
"""Check that we can login using empty username and empty password as
it will use the default user and that has an empty password.
"""
login_and_execute_query(username="", password="")
@TestScenario
@Tags("verification_cooldown")
@Requirements(
RQ_SRS_007_LDAP_Configuration_Server_VerificationCooldown_Default("1.0")
)
def default_verification_cooldown_value(self, server, rbac=False, timeout=20):
"""Check that the default value (0) for the verification cooldown parameter
disables caching and forces contacting the LDAP server for each
authentication request.
"""
error_message = "DB::Exception: testVCD: Authentication failed: password is incorrect or there is no user with such name"
error_exitcode = 4
user = None
with Given("I have an LDAP configuration that uses the default verification_cooldown value (0)"):
servers = {"openldap1": {"host": "openldap1", "port": "389", "enable_tls": "no",
"auth_dn_prefix": "cn=", "auth_dn_suffix": ",ou=users,dc=company,dc=com"
}}
self.context.ldap_node = self.context.cluster.node(server)
try:
with Given("I add user to LDAP"):
user = {"cn": "testVCD", "userpassword": "testVCD"}
user = add_user_to_ldap(**user)
with ldap_servers(servers):
with ldap_authenticated_users({"username": user["cn"], "server": server}, config_file=f"ldap_users_{getuid()}.xml"):
with When("I login and execute a query"):
login_and_execute_query(username=user["cn"], password=user["userpassword"])
with And("I change user password in LDAP"):
change_user_password_in_ldap(user, "newpassword")
with Then("when I try to login immediately with the old user password it should fail"):
login_and_execute_query(username=user["cn"], password=user["userpassword"],
exitcode=error_exitcode, message=error_message)
finally:
with Finally("I make sure LDAP user is deleted"):
if user is not None:
delete_user_from_ldap(user, exitcode=None)
@TestScenario
@Tags("verification_cooldown")
@Requirements(
RQ_SRS_007_LDAP_Configuration_Server_VerificationCooldown("1.0")
)
def valid_verification_cooldown_value_cn_change(self, server, rbac=False, timeout=20):
"""Check that we can perform requests without contacting the LDAP server
after successful authentication when the verification_cooldown parameter
is set and the user cn is changed.
"""
error_message = "DB::Exception: testVCD: Authentication failed: password is incorrect or there is no user with such name"
error_exitcode = 4
user = None
new_user = None
with Given("I have an LDAP configuration that sets verification_cooldown parameter to 2 sec"):
servers = { "openldap1": {
"host": "openldap1",
"port": "389",
"enable_tls": "no",
"auth_dn_prefix": "cn=",
"auth_dn_suffix": ",ou=users,dc=company,dc=com",
"verification_cooldown": "2"
}}
self.context.ldap_node = self.context.cluster.node(server)
try:
with Given("I add user to LDAP"):
user = {"cn": "testVCD", "userpassword": "testVCD"}
user = add_user_to_ldap(**user)
with ldap_servers(servers):
with ldap_authenticated_users({"username": user["cn"], "server": server}, config_file=f"ldap_users_{getuid()}.xml"):
with When("I login and execute a query"):
login_and_execute_query(username=user["cn"], password=user["userpassword"])
with And("I change user cn in LDAP"):
new_user = change_user_cn_in_ldap(user, "testVCD2")
with Then("when I try to login again with the old user cn it should work"):
login_and_execute_query(username=user["cn"], password=user["userpassword"])
with And("when I sleep for 2 seconds and try to log in, it should fail"):
time.sleep(2)
login_and_execute_query(username=user["cn"], password=user["userpassword"],
exitcode=error_exitcode, message=error_message)
finally:
with Finally("I make sure LDAP user is deleted"):
if new_user is not None:
delete_user_from_ldap(new_user, exitcode=None)
@TestScenario
@Tags("verification_cooldown")
@Requirements(
RQ_SRS_007_LDAP_Configuration_Server_VerificationCooldown("1.0")
)
def valid_verification_cooldown_value_password_change(self, server, rbac=False, timeout=20):
"""Check that we can perform requests without contacting the LDAP server
after successful authentication when the verification_cooldown parameter
is set and the user password is changed.
"""
error_message = "DB::Exception: testVCD: Authentication failed: password is incorrect or there is no user with such name"
error_exitcode = 4
user = None
with Given("I have an LDAP configuration that sets verification_cooldown parameter to 2 sec"):
servers = { "openldap1": {
"host": "openldap1",
"port": "389",
"enable_tls": "no",
"auth_dn_prefix": "cn=",
"auth_dn_suffix": ",ou=users,dc=company,dc=com",
"verification_cooldown": "2"
}}
self.context.ldap_node = self.context.cluster.node(server)
try:
with Given("I add user to LDAP"):
user = {"cn": "testVCD", "userpassword": "testVCD"}
user = add_user_to_ldap(**user)
with ldap_servers(servers):
with ldap_authenticated_users({"username": user["cn"], "server": server}, config_file=f"ldap_users_{getuid()}.xml"):
with When("I login and execute a query"):
login_and_execute_query(username=user["cn"], password=user["userpassword"])
with And("I change user password in LDAP"):
change_user_password_in_ldap(user, "newpassword")
with Then("when I try to login again with the old password it should work"):
login_and_execute_query(username=user["cn"], password=user["userpassword"])
with And("when I sleep for 2 seconds and try to log in, it should fail"):
time.sleep(2)
login_and_execute_query(username=user["cn"], password=user["userpassword"],
exitcode=error_exitcode, message=error_message)
finally:
with Finally("I make sure LDAP user is deleted"):
if user is not None:
delete_user_from_ldap(user, exitcode=None)
@TestScenario
@Tags("verification_cooldown")
@Requirements(
RQ_SRS_007_LDAP_Configuration_Server_VerificationCooldown("1.0")
)
def valid_verification_cooldown_value_ldap_unavailable(self, server, rbac=False, timeout=20):
"""Check that we can perform requests without contacting the LDAP server
after successful authentication when the verification_cooldown parameter
is set, even when the LDAP server is offline.
"""
error_message = "DB::Exception: testVCD: Authentication failed: password is incorrect or there is no user with such name"
error_exitcode = 4
user = None
with Given("I have an LDAP configuration that sets verification_cooldown parameter to 2 sec"):
servers = { "openldap1": {
"host": "openldap1",
"port": "389",
"enable_tls": "no",
"auth_dn_prefix": "cn=",
"auth_dn_suffix": ",ou=users,dc=company,dc=com",
"verification_cooldown": "2"
}}
self.context.ldap_node = self.context.cluster.node(server)
try:
with Given("I add a new user to LDAP"):
user = {"cn": "testVCD", "userpassword": "testVCD"}
user = add_user_to_ldap(**user)
with ldap_servers(servers):
with ldap_authenticated_users({"username": user["cn"], "server": server},
config_file=f"ldap_users_{getuid()}.xml"):
with When("I login and execute a query"):
login_and_execute_query(username=user["cn"], password=user["userpassword"])
try:
with And("then I stop the ldap server"):
self.context.ldap_node.stop()
with Then("when I try to login again with the server offline it should work"):
login_and_execute_query(username=user["cn"], password=user["userpassword"])
with And("when I sleep for 2 seconds and try to log in, it should fail"):
time.sleep(2)
login_and_execute_query(username=user["cn"], password=user["userpassword"],
exitcode=error_exitcode, message=error_message)
finally:
with Finally("I start the ldap server back up"):
self.context.ldap_node.start()
finally:
with Finally("I make sure LDAP user is deleted"):
if user is not None:
delete_user_from_ldap(user, exitcode=None)
@TestOutline
def repeat_requests(self, server, iterations, vcd_value, rbac=False):
"""Run repeated requests from some user to the LDAP server.
"""
user = None
with Given(f"I have an LDAP configuration that sets verification_cooldown parameter to {vcd_value} sec"):
servers = { "openldap1": {
"host": "openldap1",
"port": "389",
"enable_tls": "no",
"auth_dn_prefix": "cn=",
"auth_dn_suffix": ",ou=users,dc=company,dc=com",
"verification_cooldown": vcd_value
}}
self.context.ldap_node = self.context.cluster.node(server)
try:
with And("I add a new user to LDAP"):
user = {"cn": "testVCD", "userpassword": "testVCD"}
user = add_user_to_ldap(**user)
with ldap_servers(servers):
with ldap_authenticated_users({"username": user["cn"], "server": server}, config_file=f"ldap_users_{getuid()}.xml"):
with When(f"I login and execute some query {iterations} times"):
start_time = time.time()
r = self.context.node.command(f"time for i in {{1..{iterations}}}; do clickhouse client -q \"SELECT 1\" --user {user['cn']} --password {user['userpassword']} > /dev/null; done")
end_time = time.time()
return end_time - start_time
finally:
with Finally("I make sure LDAP user is deleted"):
if user is not None:
delete_user_from_ldap(user, exitcode=None)
@TestScenario
@Tags("verification_cooldown")
@Requirements(
RQ_SRS_007_LDAP_Authentication_VerificationCooldown_Performance("1.0")
)
def verification_cooldown_performance(self, server, rbac=False, iterations=5000):
"""Check that login performance is better when the verification cooldown
parameter is set to a positive value when comparing to the case when
the verification cooldown parameter is turned off.
"""
vcd_time = 0
no_vcd_time = 0
with Example(f"Repeated requests with verification cooldown parameter set to 600 seconds, {iterations} iterations"):
vcd_time = repeat_requests(server=server, iterations=iterations, vcd_value="600", rbac=rbac)
metric("login_with_vcd_value_600", units="seconds", value=vcd_time)
with Example(f"Repeated requests with verification cooldown parameter set to 0 seconds, {iterations} iterations"):
no_vcd_time = repeat_requests(server=server, iterations=iterations, vcd_value="0", rbac=rbac)
metric("login_with_vcd_value_0", units="seconds", value=no_vcd_time)
with Then("The performance with verification cooldown parameter set is better than the performance with no verification cooldown parameter."):
assert no_vcd_time > vcd_time, error()
with And("Log the performance improvement as a percentage."):
metric("percentage_improvement", units="%", value=100*(no_vcd_time - vcd_time)/vcd_time)
@TestOutline
def check_verification_cooldown_reset_on_core_server_parameter_change(self, server,
parameter_name, parameter_value, rbac=False):
"""Check that the LDAP login cache is reset for all the LDAP authentication users
when verification_cooldown parameter is set after one of the core server
parameters is changed in the LDAP server configuration.
"""
config_d_dir="/etc/clickhouse-server/config.d"
config_file="ldap_servers.xml"
error_message = "DB::Exception: {user}: Authentication failed: password is incorrect or there is no user with such name"
error_exitcode = 4
user = None
config=None
updated_config=None
with Given("I have an LDAP configuration that sets verification_cooldown parameter to 600 sec"):
servers = { "openldap1": {
"host": "openldap1",
"port": "389",
"enable_tls": "no",
"auth_dn_prefix": "cn=",
"auth_dn_suffix": ",ou=users,dc=company,dc=com",
"verification_cooldown": "600"
}}
self.context.ldap_node = self.context.cluster.node(server)
with And("LDAP authenticated user"):
users = [
{"cn": f"testVCD_0", "userpassword": "testVCD_0"},
{"cn": f"testVCD_1", "userpassword": "testVCD_1"}
]
with And("I create LDAP servers configuration file"):
config = create_ldap_servers_config_content(servers, config_d_dir, config_file)
with ldap_users(*users) as users:
with ldap_servers(servers, restart=True):
with ldap_authenticated_users(*[{"username": user["cn"], "server": server} for user in users]):
with When("I login and execute a query"):
for user in users:
with By(f"as user {user['cn']}"):
login_and_execute_query(username=user["cn"], password=user["userpassword"])
with And("I change user password in LDAP"):
for user in users:
with By(f"for user {user['cn']}"):
change_user_password_in_ldap(user, "newpassword")
with And(f"I change the server {parameter_name} core parameter", description=f"{parameter_value}"):
servers["openldap1"][parameter_name] = parameter_value
with And("I create an updated the config file that has a different server host name"):
updated_config = create_ldap_servers_config_content(servers, config_d_dir, config_file)
with modify_config(updated_config, restart=False):
with Then("when I try to log in it should fail as cache should have been reset"):
for user in users:
with By(f"as user {user['cn']}"):
login_and_execute_query(username=user["cn"], password=user["userpassword"],
exitcode=error_exitcode, message=error_message.format(user=user["cn"]))
@TestScenario
@Tags("verification_cooldown")
@Requirements(
RQ_SRS_007_LDAP_Authentication_VerificationCooldown_Reset_ChangeInCoreServerParameters("1.0")
)
def verification_cooldown_reset_on_server_host_parameter_change(self, server, rbac=False):
"""Check that the LDAP login cache is reset for all the LDAP authentication users
when verification_cooldown parameter is set after server host name
is changed in the LDAP server configuration.
"""
check_verification_cooldown_reset_on_core_server_parameter_change(server=server,
parameter_name="host", parameter_value="openldap2", rbac=rbac)
@TestScenario
@Tags("verification_cooldown")
@Requirements(
RQ_SRS_007_LDAP_Authentication_VerificationCooldown_Reset_ChangeInCoreServerParameters("1.0")
)
def verification_cooldown_reset_on_server_port_parameter_change(self, server, rbac=False):
"""Check that the LDAP login cache is reset for all the LDAP authentication users
when verification_cooldown parameter is set after server port is changed in the
LDAP server configuration.
"""
check_verification_cooldown_reset_on_core_server_parameter_change(server=server,
parameter_name="port", parameter_value="9006", rbac=rbac)
@TestScenario
@Tags("verification_cooldown")
@Requirements(
RQ_SRS_007_LDAP_Authentication_VerificationCooldown_Reset_ChangeInCoreServerParameters("1.0")
)
def verification_cooldown_reset_on_server_auth_dn_prefix_parameter_change(self, server, rbac=False):
"""Check that the LDAP login cache is reset for all the LDAP authentication users
when verification_cooldown parameter is set after server auth_dn_prefix
is changed in the LDAP server configuration.
"""
check_verification_cooldown_reset_on_core_server_parameter_change(server=server,
parameter_name="auth_dn_prefix", parameter_value="cxx=", rbac=rbac)
@TestScenario
@Tags("verification_cooldown")
@Requirements(
RQ_SRS_007_LDAP_Authentication_VerificationCooldown_Reset_ChangeInCoreServerParameters("1.0")
)
def verification_cooldown_reset_on_server_auth_dn_suffix_parameter_change(self, server, rbac=False):
"""Check that the LDAP login cache is reset for all the LDAP authentication users
when verification_cooldown parameter is set after server auth_dn_suffix
is changed in the LDAP server configuration.
"""
check_verification_cooldown_reset_on_core_server_parameter_change(server=server,
parameter_name="auth_dn_suffix",
parameter_value=",ou=company,dc=users,dc=com", rbac=rbac)
@TestScenario
@Name("verification cooldown reset when invalid password is provided")
@Tags("verification_cooldown")
@Requirements(
RQ_SRS_007_LDAP_Authentication_VerificationCooldown_Reset_InvalidPassword("1.0")
)
def scenario(self, server, rbac=False):
"""Check that cached bind requests for the user are discarded when
the user provides invalid login credentials.
"""
user = None
error_exitcode = 4
error_message = "DB::Exception: testVCD: Authentication failed: password is incorrect or there is no user with such name"
with Given("I have an LDAP configuration that sets verification_cooldown parameter to 600 sec"):
servers = { "openldap1": {
"host": "openldap1",
"port": "389",
"enable_tls": "no",
"auth_dn_prefix": "cn=",
"auth_dn_suffix": ",ou=users,dc=company,dc=com",
"verification_cooldown": "600"
}}
self.context.ldap_node = self.context.cluster.node(server)
try:
with Given("I add a new user to LDAP"):
user = {"cn": "testVCD", "userpassword": "testVCD"}
user = add_user_to_ldap(**user)
with ldap_servers(servers):
with ldap_authenticated_users({"username": user["cn"], "server": server},
config_file=f"ldap_users_{getuid()}.xml"):
with When("I login and execute a query"):
login_and_execute_query(username=user["cn"], password=user["userpassword"])
with And("I change user password in LDAP"):
change_user_password_in_ldap(user, "newpassword")
with Then("When I try to log in with the cached password it should work"):
login_and_execute_query(username=user["cn"], password=user["userpassword"])
with And("When I try to log in with an incorrect password it should fail"):
login_and_execute_query(username=user["cn"], password="incorrect", exitcode=error_exitcode,
message=error_message)
with And("When I try to log in with the cached password it should fail"):
login_and_execute_query(username=user["cn"], password="incorrect", exitcode=error_exitcode,
message=error_message)
finally:
with Finally("I make sure LDAP user is deleted"):
if user is not None:
delete_user_from_ldap(user, exitcode=None)
@TestFeature
def verification_cooldown(self, rbac, servers=None, node="clickhouse1"):
"""Check verification cooldown parameter functionality.
"""
for scenario in loads(current_module(), Scenario, filter=has.tag("verification_cooldown")):
scenario(server="openldap1", rbac=rbac)
@TestOutline(Feature)
@Name("user authentications")
@Requirements(
RQ_SRS_007_LDAP_Authentication_Mechanism_NamePassword("1.0")
)
@Examples("rbac", [
(False,),
(True, Requirements(RQ_SRS_007_LDAP_Configuration_User_RBAC("1.0")))
])
def feature(self, rbac, servers=None, node="clickhouse1"):
"""Check that users can be authenticated using an LDAP server when
users are configured either using an XML configuration file or RBAC.
"""
self.context.node = self.context.cluster.node(node)
if servers is None:
servers = globals()["servers"]
with ldap_servers(servers):
for scenario in loads(current_module(), Scenario, filter=~has.tag("verification_cooldown")):
scenario(server="openldap1", rbac=rbac)
Feature(test=verification_cooldown)(rbac=rbac, servers=servers, node=node)

466
ldap/authentication/tests/common.py Normal file
View File

@ -0,0 +1,466 @@
import os
import uuid
import time
import string
import random
import textwrap
import xml.etree.ElementTree as xmltree
from collections import namedtuple
from contextlib import contextmanager
import testflows.settings as settings
from testflows.core import *
from testflows.asserts import error
def getuid():
return str(uuid.uuid1()).replace('-', '_')
xml_with_utf8 = '<?xml version="1.0" encoding="utf-8"?>\n'
def xml_indent(elem, level=0, by=" "):
i = "\n" + level * by
if len(elem):
if not elem.text or not elem.text.strip():
elem.text = i + by
if not elem.tail or not elem.tail.strip():
elem.tail = i
for elem in elem:
xml_indent(elem, level + 1)
if not elem.tail or not elem.tail.strip():
elem.tail = i
else:
if level and (not elem.tail or not elem.tail.strip()):
elem.tail = i
def xml_append(root, tag, text):
element = xmltree.Element(tag)
element.text = text
root.append(element)
return element
Config = namedtuple("Config", "content path name uid preprocessed_name")
ASCII_CHARS = string.ascii_lowercase + string.ascii_uppercase + string.digits
def randomword(length, chars=ASCII_CHARS):
return ''.join(random.choice(chars) for i in range(length))
def restart(node=None, safe=False, timeout=60):
"""Restart ClickHouse server and wait for config to be reloaded.
"""
with When("I restart ClickHouse server node"):
if node is None:
node = current().context.node
with node.cluster.shell(node.name) as bash:
bash.expect(bash.prompt)
with By("closing terminal to the node to be restarted"):
bash.close()
with And("getting current log size"):
logsize = \
node.command("stat --format=%s /var/log/clickhouse-server/clickhouse-server.log").output.split(" ")[
0].strip()
with And("restarting ClickHouse server"):
node.restart(safe=safe)
with Then("tailing the log file from using previous log size as the offset"):
bash.prompt = bash.__class__.prompt
bash.open()
bash.send(f"tail -c +{logsize} -f /var/log/clickhouse-server/clickhouse-server.log")
with And("waiting for config reload message in the log file"):
bash.expect(
f"ConfigReloader: Loaded config '/etc/clickhouse-server/config.xml', performed update on configuration",
timeout=timeout)
def add_config(config, timeout=60, restart=False, modify=False):
"""Add dynamic configuration file to ClickHouse.
:param node: node
:param config: configuration file description
:param timeout: timeout, default: 20 sec
"""
node = current().context.node
def check_preprocessed_config_is_updated(after_removal=False):
"""Check that preprocessed config is updated.
"""
started = time.time()
command = f"cat /var/lib/clickhouse/preprocessed_configs/{config.preprocessed_name} | grep {config.uid}{' > /dev/null' if not settings.debug else ''}"
while time.time() - started < timeout:
exitcode = node.command(command, steps=False).exitcode
if after_removal:
if exitcode == 1:
break
else:
if exitcode == 0:
break
time.sleep(1)
if settings.debug:
node.command(f"cat /var/lib/clickhouse/preprocessed_configs/{config.preprocessed_name}")
if after_removal:
assert exitcode == 1, error()
else:
assert exitcode == 0, error()
def wait_for_config_to_be_loaded():
"""Wait for config to be loaded.
"""
if restart:
with When("I close terminal to the node to be restarted"):
bash.close()
with And("I stop ClickHouse to apply the config changes"):
node.stop(safe=False)
with And("I get the current log size"):
cmd = node.cluster.command(None,
f"stat --format=%s {os.environ['CLICKHOUSE_TESTS_DIR']}/_instances/{node.name}/logs/clickhouse-server.log")
logsize = cmd.output.split(" ")[0].strip()
with And("I start ClickHouse back up"):
node.start()
with Then("I tail the log file from using previous log size as the offset"):
bash.prompt = bash.__class__.prompt
bash.open()
bash.send(f"tail -c +{logsize} -f /var/log/clickhouse-server/clickhouse-server.log")
with Then("I wait for config reload message in the log file"):
if restart:
bash.expect(
f"ConfigReloader: Loaded config '/etc/clickhouse-server/config.xml', performed update on configuration",
timeout=timeout)
else:
bash.expect(
f"ConfigReloader: Loaded config '/etc/clickhouse-server/{config.preprocessed_name}', performed update on configuration",
timeout=timeout)
try:
with Given(f"{config.name}"):
if settings.debug:
with When("I output the content of the config"):
debug(config.content)
with node.cluster.shell(node.name) as bash:
bash.expect(bash.prompt)
bash.send("tail -n 0 -f /var/log/clickhouse-server/clickhouse-server.log")
with When("I add the config", description=config.path):
command = f"cat <<HEREDOC > {config.path}\n{config.content}\nHEREDOC"
node.command(command, steps=False, exitcode=0)
with Then(f"{config.preprocessed_name} should be updated", description=f"timeout {timeout}"):
check_preprocessed_config_is_updated()
with And("I wait for config to be reloaded"):
wait_for_config_to_be_loaded()
yield
finally:
if not modify:
with Finally(f"I remove {config.name}"):
with node.cluster.shell(node.name) as bash:
bash.expect(bash.prompt)
bash.send("tail -n 0 -f /var/log/clickhouse-server/clickhouse-server.log")
with By("removing the config file", description=config.path):
node.command(f"rm -rf {config.path}", exitcode=0)
with Then(f"{config.preprocessed_name} should be updated", description=f"timeout {timeout}"):
check_preprocessed_config_is_updated(after_removal=True)
with And("I wait for config to be reloaded"):
wait_for_config_to_be_loaded()
def create_ldap_servers_config_content(servers, config_d_dir="/etc/clickhouse-server/config.d", config_file="ldap_servers.xml"):
"""Create LDAP servers configuration content.
"""
uid = getuid()
path = os.path.join(config_d_dir, config_file)
name = config_file
root = xmltree.fromstring("<yandex><ldap_servers></ldap_servers></yandex>")
xml_servers = root.find("ldap_servers")
xml_servers.append(xmltree.Comment(text=f"LDAP servers {uid}"))
for _name, server in list(servers.items()):
xml_server = xmltree.Element(_name)
for key, value in list(server.items()):
xml_append(xml_server, key, value)
xml_servers.append(xml_server)
xml_indent(root)
content = xml_with_utf8 + str(xmltree.tostring(root, short_empty_elements=False, encoding="utf-8"), "utf-8")
return Config(content, path, name, uid, "config.xml")
@contextmanager
def modify_config(config, restart=False):
"""Apply updated configuration file.
"""
return add_config(config, restart=restart, modify=True)
@contextmanager
def ldap_servers(servers, config_d_dir="/etc/clickhouse-server/config.d", config_file="ldap_servers.xml",
timeout=60, restart=False, config=None):
"""Add LDAP servers configuration.
"""
if config is None:
config = create_ldap_servers_config_content(servers, config_d_dir, config_file)
return add_config(config, restart=restart)
def create_ldap_users_config_content(*users, config_d_dir="/etc/clickhouse-server/users.d", config_file="ldap_users.xml"):
"""Create LDAP users configuration file content.
"""
uid = getuid()
path = os.path.join(config_d_dir, config_file)
name = config_file
root = xmltree.fromstring("<yandex><users></users></yandex>")
xml_users = root.find("users")
xml_users.append(xmltree.Comment(text=f"LDAP users {uid}"))
for user in users:
xml_user = xmltree.Element(user['username'])
xml_user_server = xmltree.Element("ldap")
xml_append(xml_user_server, "server", user["server"])
xml_user.append(xml_user_server)
xml_users.append(xml_user)
xml_indent(root)
content = xml_with_utf8 + str(xmltree.tostring(root, short_empty_elements=False, encoding="utf-8"), "utf-8")
return Config(content, path, name, uid, "users.xml")
def add_users_identified_with_ldap(*users):
"""Add one or more users that are identified via
an ldap server using RBAC.
"""
node = current().context.node
try:
with Given("I create users"):
for user in users:
node.query(f"CREATE USER '{user['username']}' IDENTIFIED WITH ldap_server BY '{user['server']}'")
yield
finally:
with Finally("I remove users"):
for user in users:
with By(f"dropping user {user['username']}", flags=TE):
node.query(f"DROP USER IF EXISTS '{user['username']}'")
@contextmanager
def ldap_authenticated_users(*users, config_d_dir="/etc/clickhouse-server/users.d",
config_file=None, timeout=60, restart=True, config=None, rbac=False):
"""Add LDAP authenticated users.
"""
if rbac:
return add_users_identified_with_ldap(*users)
else:
if config_file is None:
config_file = f"ldap_users_{getuid()}.xml"
if config is None:
config = create_ldap_users_config_content(*users, config_d_dir=config_d_dir, config_file=config_file)
return add_config(config, restart=restart)
def invalid_server_config(servers, message=None, tail=13, timeout=60):
"""Check that ClickHouse errors when trying to load invalid LDAP servers configuration file.
"""
node = current().context.node
if message is None:
message = "Exception: Failed to merge config with '/etc/clickhouse-server/config.d/ldap_servers.xml'"
config = create_ldap_servers_config_content(servers)
try:
node.command("echo -e \"%s\" > /var/log/clickhouse-server/clickhouse-server.err.log" % ("-\\n" * tail))
with When("I add the config", description=config.path):
command = f"cat <<HEREDOC > {config.path}\n{config.content}\nHEREDOC"
node.command(command, steps=False, exitcode=0)
with Then("server shall fail to merge the new config"):
started = time.time()
command = f"tail -n {tail} /var/log/clickhouse-server/clickhouse-server.err.log | grep \"{message}\""
while time.time() - started < timeout:
exitcode = node.command(command, steps=False).exitcode
if exitcode == 0:
break
time.sleep(1)
assert exitcode == 0, error()
finally:
with Finally(f"I remove {config.name}"):
with By("removing the config file", description=config.path):
node.command(f"rm -rf {config.path}", exitcode=0)
def invalid_user_config(servers, config, message=None, tail=13, timeout=60):
"""Check that ClickHouse errors when trying to load invalid LDAP users configuration file.
"""
node = current().context.node
if message is None:
message = "Exception: Failed to merge config with '/etc/clickhouse-server/users.d/ldap_users.xml'"
with ldap_servers(servers):
try:
node.command("echo -e \"%s\" > /var/log/clickhouse-server/clickhouse-server.err.log" % ("\\n" * tail))
with When("I add the config", description=config.path):
command = f"cat <<HEREDOC > {config.path}\n{config.content}\nHEREDOC"
node.command(command, steps=False, exitcode=0)
with Then("server shall fail to merge the new config"):
started = time.time()
command = f"tail -n {tail} /var/log/clickhouse-server/clickhouse-server.err.log | grep \"{message}\""
while time.time() - started < timeout:
exitcode = node.command(command, steps=False).exitcode
if exitcode == 0:
break
time.sleep(1)
assert exitcode == 0, error()
finally:
with Finally(f"I remove {config.name}"):
with By("removing the config file", description=config.path):
node.command(f"rm -rf {config.path}", exitcode=0)
def add_user_to_ldap(cn, userpassword, givenname=None, homedirectory=None, sn=None, uid=None, uidnumber=None, node=None):
"""Add user entry to LDAP."""
if node is None:
node = current().context.ldap_node
if uid is None:
uid = cn
if givenname is None:
givenname = "John"
if homedirectory is None:
homedirectory = "/home/users"
if sn is None:
sn = "User"
if uidnumber is None:
uidnumber = 2000
user = {
"dn": f"cn={cn},ou=users,dc=company,dc=com",
"cn": cn,
"gidnumber": 501,
"givenname": givenname,
"homedirectory": homedirectory,
"objectclass": ["inetOrgPerson", "posixAccount", "top"],
"sn": sn,
"uid": uid,
"uidnumber": uidnumber,
"userpassword": userpassword,
"_server": node.name
}
lines = []
for key, value in list(user.items()):
if key.startswith("_"):
continue
elif key == "objectclass":
for cls in value:
lines.append(f"objectclass: {cls}")
else:
lines.append(f"{key}: {value}")
ldif = "\n".join(lines)
r = node.command(
f"echo -e \"{ldif}\" | ldapadd -x -H ldap://localhost -D \"cn=admin,dc=company,dc=com\" -w admin")
assert r.exitcode == 0, error()
return user
def delete_user_from_ldap(user, node=None, exitcode=0):
"""Delete user entry from LDAP."""
if node is None:
node = current().context.ldap_node
r = node.command(
f"ldapdelete -x -H ldap://localhost -D \"cn=admin,dc=company,dc=com\" -w admin \"{user['dn']}\"")
if exitcode is not None:
assert r.exitcode == exitcode, error()
def change_user_password_in_ldap(user, new_password, node=None, exitcode=0):
"""Change user password in LDAP."""
if node is None:
node = current().context.ldap_node
ldif = (f"dn: {user['dn']}\n"
"changetype: modify\n"
"replace: userpassword\n"
f"userpassword: {new_password}")
r = node.command(
f"echo -e \"{ldif}\" | ldapmodify -x -H ldap://localhost -D \"cn=admin,dc=company,dc=com\" -w admin")
if exitcode is not None:
assert r.exitcode == exitcode, error()
def change_user_cn_in_ldap(user, new_cn, node=None, exitcode=0):
"""Change user password in LDAP."""
if node is None:
node = current().context.ldap_node
new_user = dict(user)
new_user['dn'] = f"cn={new_cn},ou=users,dc=company,dc=com"
new_user['cn'] = new_cn
ldif = (
f"dn: {user['dn']}\n"
"changetype: modrdn\n"
f"newrdn: cn = {new_user['cn']}\n"
f"deleteoldrdn: 1\n"
)
r = node.command(
f"echo -e \"{ldif}\" | ldapmodify -x -H ldap://localhost -D \"cn=admin,dc=company,dc=com\" -w admin")
if exitcode is not None:
assert r.exitcode == exitcode, error()
return new_user
@contextmanager
def ldap_user(cn, userpassword, givenname=None, homedirectory=None, sn=None, uid=None, uidnumber=None, node=None):
"""Add new user to the LDAP server."""
try:
user = None
with Given(f"I add user {cn} to LDAP"):
user = add_user_to_ldap(cn, userpassword, givenname, homedirectory, sn, uid, uidnumber, node=node)
yield user
finally:
with Finally(f"I delete user {cn} from LDAP"):
if user is not None:
delete_user_from_ldap(user, node=node)
@contextmanager
def ldap_users(*users, node=None):
"""Add multiple new users to the LDAP server."""
try:
_users = []
with Given("I add users to LDAP"):
for user in users:
with By(f"adding user {user['cn']}"):
_users.append(add_user_to_ldap(**user, node=node))
yield _users
finally:
with Finally(f"I delete users from LDAP"):
for _user in _users:
delete_user_from_ldap(_user, node=node)
def login(servers, *users, config=None):
"""Configure LDAP server and LDAP authenticated users and
try to login and execute a query"""
with ldap_servers(servers):
with ldap_authenticated_users(*users, restart=True, config=config):
for user in users:
if user.get("login", False):
with When(f"I login as {user['username']} and execute query"):
current().context.node.query("SELECT 1",
settings=[("user", user["username"]), ("password", user["password"])],
exitcode=user.get("exitcode", None),
message=user.get("message", None))

304
ldap/authentication/tests/server_config.py Normal file
View File

@ -0,0 +1,304 @@
from testflows.core import *
from ldap.authentication.tests.common import *
from ldap.authentication.requirements import *
@TestScenario
@Requirements(
RQ_SRS_007_LDAP_Server_Configuration_Invalid("1.0"),
RQ_SRS_007_LDAP_Configuration_Server_Name("1.0")
)
def empty_server_name(self, timeout=20):
"""Check that empty string as a server name is not allowed.
"""
servers = {"": {"host": "foo", "port": "389", "enable_tls": "no",
"auth_dn_prefix": "cn=", "auth_dn_suffix": ",ou=users,dc=company,dc=com"
}}
invalid_server_config(servers, timeout=timeout)
@TestScenario
@Requirements(
RQ_SRS_007_LDAP_Server_Configuration_Invalid("1.0"),
RQ_SRS_007_LDAP_UnreachableServer("1.0")
)
def invalid_host(self):
"""Check that server returns an error when LDAP server
host name is invalid.
"""
servers = {"foo": {"host": "foo", "port": "389", "enable_tls": "no"}}
users = [{
"server": "foo", "username": "user1", "password": "user1", "login": True,
"exitcode": 4,
"message": "DB::Exception: user1: Authentication failed: password is incorrect or there is no user with such name"
}]
login(servers, *users)
@TestScenario
@Requirements(
RQ_SRS_007_LDAP_Server_Configuration_Invalid("1.0"),
RQ_SRS_007_LDAP_Configuration_Server_Host("1.0")
)
def empty_host(self):
"""Check that server returns an error when LDAP server
host value is empty.
"""
servers = {"foo": {"host": "", "port": "389", "enable_tls": "no"}}
users = [{
"server": "foo", "username": "user1", "password": "user1", "login": True,
"exitcode": 4,
"message": "DB::Exception: user1: Authentication failed: password is incorrect or there is no user with such name"
}]
login(servers, *users)
@TestScenario
@Requirements(
RQ_SRS_007_LDAP_Server_Configuration_Invalid("1.0"),
RQ_SRS_007_LDAP_Configuration_Server_Host("1.0")
)
def missing_host(self):
"""Check that server returns an error when LDAP server
host is missing.
"""
servers = {"foo": {"port": "389", "enable_tls": "no"}}
users = [{
"server": "foo", "username": "user1", "password": "user1", "login": True,
"exitcode": 4,
"message": "DB::Exception: user1: Authentication failed: password is incorrect or there is no user with such name"
}]
login(servers, *users)
@TestScenario
@Requirements(
RQ_SRS_007_LDAP_Server_Configuration_Invalid("1.0")
)
def invalid_port(self):
"""Check that server returns an error when LDAP server
port is not valid.
"""
servers = {"openldap1": {"host": "openldap1", "port": "3890", "enable_tls": "no"}}
users = [{
"server": "openldap1", "username": "user1", "password": "user1", "login": True,
"exitcode": 4,
"message": "DB::Exception: user1: Authentication failed: password is incorrect or there is no user with such name"
}]
login(servers, *users)
@TestScenario
@Requirements(
RQ_SRS_007_LDAP_Server_Configuration_Invalid("1.0")
)
def invalid_auth_dn_prefix(self):
"""Check that server returns an error when LDAP server
port is not valid.
"""
servers = {"openldap1": {"host": "openldap1", "port": "389", "enable_tls": "no",
"auth_dn_prefix": "foo=", "auth_dn_suffix": ",ou=users,dc=company,dc=com"
}}
users = [{
"server": "openldap1", "username": "user1", "password": "user1", "login": True,
"exitcode": 4,
"message": "DB::Exception: user1: Authentication failed: password is incorrect or there is no user with such name"
}]
login(servers, *users)
@TestScenario
@Requirements(
RQ_SRS_007_LDAP_Server_Configuration_Invalid("1.0")
)
def invalid_auth_dn_suffix(self):
"""Check that server returns an error when LDAP server
port is not valid.
"""
servers = {"openldap1": {"host": "openldap1", "port": "389", "enable_tls": "no",
"auth_dn_prefix": "cn=", "auth_dn_suffix": ",foo=users,dc=company,dc=com"
}}
users = [{
"server": "openldap1", "username": "user1", "password": "user1", "login": True,
"exitcode": 4,
"message": "DB::Exception: user1: Authentication failed: password is incorrect or there is no user with such name"
}]
login(servers, *users)
@TestScenario
@Requirements(
RQ_SRS_007_LDAP_Server_Configuration_Invalid("1.0")
)
def invalid_enable_tls_value(self):
"""Check that server returns an error when enable_tls
option has invalid value.
"""
servers = {"openldap1": {"host": "openldap1", "port": "389", "enable_tls": "foo",
"auth_dn_prefix": "cn=", "auth_dn_suffix": ",ou=users,dc=company,dc=com"
}}
users = [{
"server": "openldap1", "username": "user1", "password": "user1", "login": True,
"exitcode": 4,
"message": "DB::Exception: user1: Authentication failed: password is incorrect or there is no user with such name"
}]
login(servers, *users)
@TestScenario
@Requirements(
RQ_SRS_007_LDAP_Server_Configuration_Invalid("1.0")
)
def invalid_tls_require_cert_value(self):
"""Check that server returns an error when tls_require_cert
option has invalid value.
"""
servers = {"openldap2": {
"host": "openldap2", "port": "636", "enable_tls": "yes",
"auth_dn_prefix": "cn=", "auth_dn_suffix": ",ou=users,dc=company,dc=com",
"tls_require_cert": "foo",
"ca_cert_dir": "/container/service/slapd/assets/certs/",
"ca_cert_file": "/container/service/slapd/assets/certs/ca.crt"
}}
users = [{
"server": "openldap2", "username": "user2", "password": "user2", "login": True,
"exitcode": 4,
"message": "DB::Exception: user2: Authentication failed: password is incorrect or there is no user with such name"
}]
login(servers, *users)
@TestScenario
@Requirements(
RQ_SRS_007_LDAP_Server_Configuration_Invalid("1.0")
)
def empty_ca_cert_dir(self):
"""Check that server returns an error when ca_cert_dir is empty.
"""
servers = {"openldap2": {"host": "openldap2", "port": "636", "enable_tls": "yes",
"auth_dn_prefix": "cn=", "auth_dn_suffix": ",ou=users,dc=company,dc=com",
"tls_require_cert": "demand",
"ca_cert_dir": "",
"ca_cert_file": "/container/service/slapd/assets/certs/ca.crt"
}}
users = [{
"server": "openldap2", "username": "user2", "password": "user2", "login": True,
"exitcode": 4,
"message": "DB::Exception: user2: Authentication failed: password is incorrect or there is no user with such name"
}]
login(servers, *users)
@TestScenario
@Requirements(
RQ_SRS_007_LDAP_Server_Configuration_Invalid("1.0")
)
def empty_ca_cert_file(self):
"""Check that server returns an error when ca_cert_file is empty.
"""
servers = {"openldap2": {"host": "openldap2", "port": "636", "enable_tls": "yes",
"auth_dn_prefix": "cn=", "auth_dn_suffix": ",ou=users,dc=company,dc=com",
"tls_require_cert": "demand",
"ca_cert_dir": "/container/service/slapd/assets/certs/",
"ca_cert_file": ""
}}
users = [{
"server": "openldap2", "username": "user2", "password": "user2", "login": True,
"exitcode": 4,
"message": "DB::Exception: user2: Authentication failed: password is incorrect or there is no user with such name"
}]
login(servers, *users)
@TestScenario
@Requirements(
RQ_SRS_007_LDAP_Configuration_Server_AuthDN_Value("1.0"),
RQ_SRS_007_LDAP_Configuration_Server_AuthDN_Prefix("1.0"),
RQ_SRS_007_LDAP_Configuration_Server_AuthDN_Suffix("1.0")
)
def auth_dn_value(self):
"""Check that server configuration can properly define the `dn` value of the user."""
servers = {
"openldap1": {
"host": "openldap1", "port": "389", "enable_tls": "no",
"auth_dn_prefix": "cn=", "auth_dn_suffix": ",ou=users,dc=company,dc=com"
}}
user = {"server": "openldap1", "username": "user1", "password": "user1", "login": True}
login(servers, user)
@TestOutline(Scenario)
@Examples("invalid_value", [
("-1", Name("negative int")),
("foo", Name("string")),
("", Name("empty string")),
("36893488147419103232", Name("overflow with extremely large int value")),
("-36893488147419103232", Name("overflow with extremely large negative int value")),
("@#", Name("special characters"))
])
@Requirements(
RQ_SRS_007_LDAP_Configuration_Server_VerificationCooldown_Invalid("1.0")
)
def invalid_verification_cooldown_value(self, invalid_value, timeout=20):
"""Check that server returns an error when LDAP server
verification cooldown parameter is invalid.
"""
error_message = ("<Error> Access(user directories): Could not parse LDAP server"
" \\`openldap1\\`: Poco::Exception. Code: 1000, e.code() = 0,"
f" e.displayText() = Syntax error: Not a valid unsigned integer{': ' + invalid_value if invalid_value else invalid_value}")
with Given("LDAP server configuration that uses a negative integer for the verification_cooldown parameter"):
servers = {"openldap1": {"host": "openldap1", "port": "389", "enable_tls": "no",
"auth_dn_prefix": "cn=", "auth_dn_suffix": ",ou=users,dc=company,dc=com",
"verification_cooldown": f"{invalid_value}"
}}
with When("I try to use this configuration then it should not work"):
invalid_server_config(servers, message=error_message, tail=17, timeout=timeout)
@TestScenario
@Requirements(
RQ_SRS_007_LDAP_Configuration_Server_Syntax("2.0")
)
def syntax(self):
"""Check that server configuration with valid syntax can be loaded.
```xml
<yandex>
<ldap_server>
<host>localhost</host>
<port>636</port>
<auth_dn_prefix>cn=</auth_dn_prefix>
<auth_dn_suffix>, ou=users, dc=example, dc=com</auth_dn_suffix>
<verification_cooldown>0</verification_cooldown>
<enable_tls>yes</enable_tls>
<tls_minimum_protocol_version>tls1.2</tls_minimum_protocol_version>
<tls_require_cert>demand</tls_require_cert>
<tls_cert_file>/path/to/tls_cert_file</tls_cert_file>
<tls_key_file>/path/to/tls_key_file</tls_key_file>
<tls_ca_cert_file>/path/to/tls_ca_cert_file</tls_ca_cert_file>
<tls_ca_cert_dir>/path/to/tls_ca_cert_dir</tls_ca_cert_dir>
<tls_cipher_suite>ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:AES256-GCM-SHA384</tls_cipher_suite>
</ldap_server>
</yandex>
```
"""
servers = {
"openldap2": {
"host": "openldap2",
"port": "389",
"auth_dn_prefix": "cn=",
"auth_dn_suffix": ",ou=users,dc=company,dc=com",
"verification_cooldown": "0",
"enable_tls": "yes",
"tls_minimum_protocol_version": "tls1.2" ,
"tls_require_cert": "demand",
"tls_cert_file": "/container/service/slapd/assets/certs/ldap.crt",
"tls_key_file": "/container/service/slapd/assets/certs/ldap.key",
"tls_ca_cert_file": "/container/service/slapd/assets/certs/ca.crt",
"tls_ca_cert_dir": "/container/service/slapd/assets/certs/",
"tls_cipher_suite": "ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:AES256-GCM-SHA384"
}
}
with ldap_servers(servers):
pass
@TestFeature
@Name("server config")
def feature(self, node="clickhouse1"):
"""Check that LDAP server configuration.
"""
self.context.node = self.context.cluster.node(node)
for scenario in loads(current_module(), Scenario):
scenario()