ClickHouse/dbms/tests/integration/README.md

ClickHouse integration tests

This directory contains tests that involve several ClickHouse instances, custom configs, ZooKeeper, etc.

Running natively

Prerequisites:

• Ubuntu 14.04 (Trusty) or higher.
• docker. Minimum required API version: 1.25, check with docker version.

You must install latest Docker from https://docs.docker.com/engine/installation/linux/docker-ce/ubuntu/#set-up-the-repository Don't use Docker from your system repository.

• pip. To install: sudo apt-get install python-pip
• py.test testing framework. To install: sudo -H pip install pytest
• docker-compose and additional python libraries. To install: sudo -H pip install docker-compose docker dicttoxml kazoo PyMySQL psycopg2

(highly not recommended) If you really want to use OS packages on modern debian/ubuntu instead of "pip": sudo apt install -y docker docker-compose python-pytest python-dicttoxml python-docker python-pymysql python-kazoo python-psycopg2

If you want to run the tests under a non-privileged user, you must add this user to docker group: sudo usermod -aG docker $USER and re-login. (You must close all your sessions (for example, restart your computer)) To check, that you have access to Docker, run docker ps.

Run the tests with the pytest command. To select which tests to run, use: pytest -k <test_name_pattern>

By default tests are run with system-wide client binary, server binary and base configs. To change that, set the following environment variables:

• CLICKHOUSE_TESTS_SERVER_BIN_PATH to choose the server binary.
• CLICKHOUSE_TESTS_CLIENT_BIN_PATH to choose the client binary.
• CLICKHOUSE_TESTS_BASE_CONFIG_DIR to choose the directory from which base configs (config.xml and users.xml) are taken.

Running with runner script

The only requirement is fresh docker with access to the internet. You can check it with:

$ docker run ubuntu:14.04 ping github.com
PING github.com (140.82.118.3) 56(84) bytes of data.
64 bytes from 140.82.118.3: icmp_seq=1 ttl=53 time=40.1 ms
64 bytes from 140.82.118.3: icmp_seq=2 ttl=53 time=40.4 ms
64 bytes from 140.82.118.3: icmp_seq=3 ttl=53 time=40.3 ms
64 bytes from 140.82.118.3: icmp_seq=4 ttl=53 time=40.1 ms

--- github.com ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 19823ms
rtt min/avg/max/mdev = 40.157/40.284/40.463/0.278 ms

You can run tests via ./runner script and pass pytest arguments as last arg:

$ ./runner --binary $HOME/ClickHouse/dbms/programs/clickhouse --configs-dir $HOME/ClickHouse/dbms/programs/server/ 'test_odbc_interaction -ss'
Start tests
============================= test session starts ==============================
platform linux2 -- Python 2.7.15rc1, pytest-4.0.0, py-1.7.0, pluggy-0.8.0
rootdir: /ClickHouse/dbms/tests/integration, inifile: pytest.ini
collected 6 items

test_odbc_interaction/test.py Removing network clickhouse_default
...

Killing roottestodbcinteraction_node1_1     ... done
Killing roottestodbcinteraction_mysql1_1    ... done
Killing roottestodbcinteraction_postgres1_1 ... done
Removing roottestodbcinteraction_node1_1     ... done
Removing roottestodbcinteraction_mysql1_1    ... done
Removing roottestodbcinteraction_postgres1_1 ... done
Removing network roottestodbcinteraction_default

==================== 6 passed, 1 warnings in 95.21 seconds =====================

Path to binary and configs maybe specified via env variables:

$ export CLICKHOUSE_TESTS_BASE_CONFIG_DIR=$HOME/ClickHouse/dbms/programs/server/
$ export CLICKHOUSE_TESTS_SERVER_BIN_PATH=$HOME/ClickHouse/dbms/programs/clickhouse
$ ./runner 'test_odbc_interaction'
Start tests
============================= test session starts ==============================
platform linux2 -- Python 2.7.15rc1, pytest-4.0.0, py-1.7.0, pluggy-0.8.0
rootdir: /ClickHouse/dbms/tests/integration, inifile: pytest.ini
collected 6 items

test_odbc_interaction/test.py ......                                     [100%]
==================== 6 passed, 1 warnings in 96.33 seconds =====================

Adding new tests

To add new test named foo, create a directory test_foo with an empty __init__.py and a file named test.py containing tests in it. All functions with names starting with test will become test cases.

helpers directory contains utilities for:

• Launching a ClickHouse cluster with or without ZooKeeper indocker containers.
• Sending queries to launched instances.
• Introducing network failures such as severing network link between two instances.

To assert that two TSV files must be equal, wrap them in the TSV class and use the regular assert statement. Example: assert TSV(result) == TSV(reference). In case the assertion fails, pytest will automagically detect the types of variables and only the small diff of two files is printed.