ClickHouse/dbms/tests/integration
alexey-milovidov 2469ec1af3
Merge pull request #5857 from CurtizJ/dns-cache
Implement dns cache with asynchronous update.
2019-07-05 15:56:26 +03:00
..
helpers Merge pull request #5628 from yandex/index_granularity_by_default 2019-07-03 02:15:33 +03:00
image Disable tests and add global timeout 2019-06-25 16:20:04 +03:00
test_adaptive_granularity Add test for interaction of tables with different granularity and different versions 2019-06-20 19:25:32 +03:00
test_block_structure_mismatch fix test (MATERIALIZED columns in both Distributed and local tables don't work) [#CLICKHOUSE-4161] 2018-11-27 15:42:11 +03:00
test_cluster_copier clickhouse-copier: Fix use-after free on shutdown (#5752) 2019-06-27 15:29:49 +03:00
test_concurrent_queries_for_user_restriction Simplify test 2019-01-09 15:23:41 +03:00
test_config_substitutions Config file 2018-10-26 12:23:29 +03:00
test_cross_replication CLICKHOUSE-3894: Some improvements in flapping tests. Bug fix of ATTACH TABLE xxx ON CLUSTER (previously doesn't work). 2018-09-03 17:06:00 +03:00
test_delayed_replica_failover fix test to conform to the new GROUP BY behaviour on empty tables 2018-03-07 18:09:26 +00:00
test_dictionaries Add more integration tests for external dictionaries 2019-06-14 22:21:37 +03:00
test_distributed_ddl clickhouse-test - prepare for use other than 'test' database (#4961) 2019-04-11 20:20:36 +03:00
test_distributed_ddl_password Execute some distributed DDL queries on leader 2019-06-26 17:52:20 +03:00
test_distributed_ddl_secure fix tests/integration over ssl by alesapin@ 2018-11-06 17:42:30 +03:00
test_distributed_respect_user_timeouts distributed timeouts integration test 2019-06-02 20:54:44 +03:00
test_distributed_system_query Update test.py 2019-06-17 11:25:06 +03:00
test_external_dictionaries Debian: Asking password for default user on package install (#4725) 2019-03-20 22:57:50 +03:00
test_extreme_deduplication Disable tests and add global timeout 2019-06-25 16:20:04 +03:00
test_format_schema_on_server Add test checking using format schema via HTTP interface. 2019-03-30 21:42:01 +03:00
test_graphite_merge_tree Add tests for system.graphite_retentions 2019-03-07 13:47:02 +01:00
test_host_ip_change fix deadlock in getHostName 2019-07-04 02:05:59 +03:00
test_https_replication CLICKHOUSE-3894: Some improvements in flapping tests. Bug fix of ATTACH TABLE xxx ON CLUSTER (previously doesn't work). 2018-09-03 17:06:00 +03:00
test_insert_into_distributed Removed experimental setting from integration tests 2019-02-10 17:17:21 +03:00
test_insert_into_distributed_sync_async add tests [#CLICKHOUSE-4161] 2018-11-26 17:57:33 +03:00
test_insert_into_distributed_through_materialized_view Disable tests and add global timeout 2019-06-25 16:20:04 +03:00
test_match_process_uid_against_data_owner Do not require the server to fail to start in the test. 2018-12-20 01:50:05 +03:00
test_max_http_connections_for_replication Fix exception 'No message received' 2019-03-29 21:10:03 +03:00
test_merge_table_over_distributed Fixed Illegal PREWHERE error for Merge tables over Distributed. [#CLICKHOUSE-3637] 2018-03-12 18:14:26 +03:00
test_mysql_database_engine ISSUES-5697 fix insert and select query with mysql style identifier 2019-06-21 13:51:27 +08:00
test_mysql_protocol support of sessions and default_database in MySQL wire protocol 2019-06-16 18:12:37 +03:00
test_non_default_compression Fix non initialized codec and wrong size in CacheCompressedReadBuffer 2019-01-29 17:55:57 +03:00
test_odbc_interaction Better waits in integration test 2019-06-21 11:03:13 +03:00
test_old_versions_client Add test for client-server interaction with different versions 2019-07-04 13:59:42 +03:00
test_parts_delete_zookeeper Fix inconsistent parts remove from local FS without dropping them from ZK 2019-06-03 17:58:19 +03:00
test_random_inserts CLICKHOUSE-3894: Fixes in tests 2018-09-07 14:51:51 +03:00
test_recovery_replica CLICKHOUSE-3894: Some improvements in flapping tests. Bug fix of ATTACH TABLE xxx ON CLUSTER (previously doesn't work). 2018-09-03 17:06:00 +03:00
test_replace_partition CLICKHOUSE-3894: Some improvements in flapping tests. Bug fix of ATTACH TABLE xxx ON CLUSTER (previously doesn't work). 2018-09-03 17:06:00 +03:00
test_replicated_mutations ignore max part size for merge setting while executing mutations 2019-06-17 22:41:48 +03:00
test_replication_credentials Remove wrong option 2018-07-27 20:25:51 +03:00
test_replication_without_zookeeper Use pytest's raises instead of try/except clause 2019-05-21 11:30:39 +03:00
test_send_request_to_leader_replica Improve test 2018-12-28 20:11:52 +03:00
test_server_initialization attempt to fix integration tests 2017-07-11 20:56:17 +03:00
test_settings_constraints Implement new constraints: "min" and "readonly". 2019-04-27 01:09:16 +03:00
test_storage_hdfs Better code and tests for bad cases 2019-01-19 23:17:19 +03:00
test_storage_kafka Merge pull request #5382 from abyss7/issue-5286 2019-07-01 21:30:09 +03:00
test_storage_mysql CLICKHOUSE-3947 add tests 2018-10-08 18:05:23 +03:00
test_system_queries fix race condition in flushing system log 2019-06-19 15:28:34 +03:00
test_ttl_replicated TTL for columns and tables (#4212) 2019-04-15 12:30:45 +03:00
test_union_header Added test. 2018-10-23 20:48:58 +03:00
test_user_ip_restrictions Fix ipv4 mask restrictions for users and add integration tests 2018-12-27 18:55:51 +03:00
test_zookeeper_config Added test for chroot in ZK [#MDBSUPPORT-204] 2018-07-10 19:14:27 +03:00
.dockerignore https://stackoverflow.com/questions/44067609/getting-error-importmismatcherror-while-running-py-test 2018-11-23 20:56:50 +03:00
.gitignore Fix trying to load database.sql from incorrect path. [#CLICKHOUSE-3044] 2017-06-16 19:08:24 +03:00
CMakeLists.txt CLICKHOUSE-4421 Fix segfault in copier (#4835) 2019-04-01 17:58:04 +03:00
conftest.py ClickHouse integration tests. [#CLICKHOUSE-2821] 2017-05-22 17:56:41 +04:00
pytest.ini Disable tests and add global timeout 2019-06-25 16:20:04 +03:00
README.md Improved integration tests guide. 2019-06-29 14:34:26 +03:00
runner Fixing "the input device is not a TTY" for test runner 2019-05-20 11:48:36 +02:00

ClickHouse integration tests

This directory contains tests that involve several ClickHouse instances, custom configs, ZooKeeper, etc.

Running natively

Prerequisites:

  • Ubuntu 14.04 (Trusty) or higher.
  • docker. Minimum required API version: 1.25, check with docker version.

You must install latest Docker from https://docs.docker.com/engine/installation/linux/docker-ce/ubuntu/#set-up-the-repository Don't use Docker from your system repository.

  • pip and libpq-dev. To install: sudo apt-get install python-pip libpq-dev
  • py.test testing framework. To install: sudo -H pip install pytest
  • docker-compose and additional python libraries. To install: sudo -H pip install docker-compose docker dicttoxml kazoo PyMySQL psycopg2 pymongo tzlocal kafka-python protobuf pytest-timeout

(highly not recommended) If you really want to use OS packages on modern debian/ubuntu instead of "pip": sudo apt install -y docker docker-compose python-pytest python-dicttoxml python-docker python-pymysql python-pymongo python-tzlocal python-kazoo python-psycopg2 python-kafka python-pytest-timeout

If you want to run the tests under a non-privileged user, you must add this user to docker group: sudo usermod -aG docker $USER and re-login. (You must close all your sessions (for example, restart your computer)) To check, that you have access to Docker, run docker ps.

Run the tests with the pytest command. To select which tests to run, use: pytest -k <test_name_pattern>

By default tests are run with system-wide client binary, server binary and base configs. To change that, set the following environment variables:

  • CLICKHOUSE_TESTS_SERVER_BIN_PATH to choose the server binary.
  • CLICKHOUSE_TESTS_CLIENT_BIN_PATH to choose the client binary.
  • CLICKHOUSE_TESTS_BASE_CONFIG_DIR to choose the directory from which base configs (config.xml and users.xml) are taken.

Running with runner script

The only requirement is fresh docker configured docker.

Notes:

  • If you want to run integration tests without sudo you have to add your user to docker group sudo usermod -aG docker $USER. More information about docker configuration.
  • If you already had run these tests without ./runner script you may have problems with pytest cache. It can be removed with rm -r __pycache__ .pytest_cache/.
  • Some tests maybe require a lot of resources (CPU, RAM, etc.). Better not try large tests like test_cluster_copier or test_distributed_ddl* on your notebook.

You can run tests via ./runner script and pass pytest arguments as last arg:

$ ./runner --binary $HOME/ClickHouse/dbms/programs/clickhouse  --bridge-binary $HOME/ClickHouse/dbms/programs/clickhouse-odbc-bridge --configs-dir $HOME/ClickHouse/dbms/programs/server/ 'test_odbc_interaction -ss'
Start tests
============================= test session starts ==============================
platform linux2 -- Python 2.7.15rc1, pytest-4.0.0, py-1.7.0, pluggy-0.8.0
rootdir: /ClickHouse/dbms/tests/integration, inifile: pytest.ini
collected 6 items

test_odbc_interaction/test.py Removing network clickhouse_default
...

Killing roottestodbcinteraction_node1_1     ... done
Killing roottestodbcinteraction_mysql1_1    ... done
Killing roottestodbcinteraction_postgres1_1 ... done
Removing roottestodbcinteraction_node1_1     ... done
Removing roottestodbcinteraction_mysql1_1    ... done
Removing roottestodbcinteraction_postgres1_1 ... done
Removing network roottestodbcinteraction_default

==================== 6 passed, 1 warnings in 95.21 seconds =====================

Path to binary and configs maybe specified via env variables:

$ export CLICKHOUSE_TESTS_BASE_CONFIG_DIR=$HOME/ClickHouse/dbms/programs/server/
$ export CLICKHOUSE_TESTS_SERVER_BIN_PATH=$HOME/ClickHouse/dbms/programs/clickhouse
$ export CLICKHOUSE_TESTS_ODBC_BRIDGE_BIN_PATH=$HOME/ClickHouse/dbms/programs/clickhouse-odbc-bridge
$ ./runner 'test_odbc_interaction'
$ # or ./runner '-v -ss'
Start tests
============================= test session starts ==============================
platform linux2 -- Python 2.7.15rc1, pytest-4.0.0, py-1.7.0, pluggy-0.8.0
rootdir: /ClickHouse/dbms/tests/integration, inifile: pytest.ini
collected 6 items

test_odbc_interaction/test.py ......                                     [100%]
==================== 6 passed, 1 warnings in 96.33 seconds =====================

You can just open shell inside a container by overwritting the command: ./runner --command=bash

Adding new tests

To add new test named foo, create a directory test_foo with an empty __init__.py and a file named test.py containing tests in it. All functions with names starting with test will become test cases.

helpers directory contains utilities for:

  • Launching a ClickHouse cluster with or without ZooKeeper indocker containers.
  • Sending queries to launched instances.
  • Introducing network failures such as severing network link between two instances.

To assert that two TSV files must be equal, wrap them in the TSV class and use the regular assert statement. Example: assert TSV(result) == TSV(reference). In case the assertion fails, pytest will automagically detect the types of variables and only the small diff of two files is printed.