ClickHouse/dbms/tests/integration
Pavel Kovalenko 8d07883b33 Merge branch 'master' into merge-tree-s3-initial
# Conflicts:
#	dbms/src/Storages/MergeTree/MergeTreeData.cpp
#	dbms/src/Storages/StorageMergeTree.cpp
2020-03-05 23:05:31 +03:00
..
helpers Added tests for reloading storage configuration. 2020-03-04 15:10:17 +03:00
image Add AvroConfluent integration test 2020-02-02 18:02:19 -06:00
test_adaptive_granularity More stable kill in integration tests 2019-12-17 21:07:13 +03:00
test_adaptive_granularity_replicated Fix recreation of replicated table with fixed granularity 2019-08-12 16:30:29 +03:00
test_aggregation_memory_efficient Added integration test from #9134 2020-02-21 22:01:43 +03:00
test_allowed_client_hosts Add tests. 2020-03-01 00:27:37 +03:00
test_allowed_url_from_config Created test for HDFS Storage and hdfs table function 2019-11-05 18:30:35 +03:00
test_atomic_drop_table comment to test 2019-08-09 15:58:46 +03:00
test_authentication Add tests. 2020-03-01 00:27:37 +03:00
test_backup_restore fixed part mutation, added test_backup_restore integration test case 2019-12-11 09:31:37 +08:00
test_backward_compatability test for replicas backward comptability 2020-02-26 17:18:37 +03:00
test_block_structure_mismatch Changed repository URL 2019-09-23 19:18:19 +03:00
test_check_table Fix check_table test 2019-12-17 15:40:18 +03:00
test_cluster_all_replicas added new integration test for clusterAllreplicas tabe function 2020-01-07 15:57:24 +05:30
test_cluster_copier Fix some spelling mistakes 2020-01-11 17:51:33 +08:00
test_concurrent_queries_for_user_restriction Simplify test 2019-01-09 15:23:41 +03:00
test_config_corresponding_root Fix indent and add init file 2019-07-25 16:41:39 +03:00
test_config_substitutions Fix config subst test 2019-12-17 15:41:22 +03:00
test_consistant_parts_after_move_partition typo 2019-11-18 04:43:39 +01:00
test_consistent_parts_after_clone_replica remove local parts which source replica doesnt have 2019-08-19 15:08:45 +03:00
test_cross_replication CLICKHOUSE-3894: Some improvements in flapping tests. Bug fix of ATTACH TABLE xxx ON CLUSTER (previously doesn't work). 2018-09-03 17:06:00 +03:00
test_delayed_replica_failover fix test to conform to the new GROUP BY behaviour on empty tables 2018-03-07 18:09:26 +00:00
test_dictionaries_all_layouts_and_sources Revert "Merge pull request #9025 from ClickHouse/revert-8303-cache-dictionary" 2020-02-06 15:18:19 +03:00
test_dictionaries_complex_key_cache_string Split the integration test test_dictionaries into 4 separate tests. 2019-09-02 17:50:57 +03:00
test_dictionaries_ddl Add restrections for several sources of external dictionaries 2019-12-10 20:27:29 +03:00
test_dictionaries_depend_on_dictionaries Split the integration test test_dictionaries into 4 separate tests. 2019-09-02 17:50:57 +03:00
test_dictionaries_mysql MySQL connection mutualize connection + integration test 2020-02-27 10:34:06 +01:00
test_dictionaries_null_value Split the integration test test_dictionaries into 4 separate tests. 2019-09-02 17:50:57 +03:00
test_dictionaries_select_all Split the integration test test_dictionaries into 4 separate tests. 2019-09-02 17:50:57 +03:00
test_dictionaries_update_and_reload squashed 2020-02-27 13:07:50 +03:00
test_dictionary_allow_read_expired_keys Revert "Merge pull request #9025 from ClickHouse/revert-8303-cache-dictionary" 2020-02-06 15:18:19 +03:00
test_dictionary_ddl_on_cluster Remove sleep by Azat suggestion, https://github.com/ClickHouse/ClickHouse/pull/8288#pullrequestreview-336615781 2019-12-27 21:24:22 +03:00
test_disk_access_storage Add test for DiskAccessStorage. 2020-02-29 17:33:52 +03:00
test_distributed_ddl Fix failed integration tests 2020-02-21 11:53:29 +03:00
test_distributed_ddl_password Execute some distributed DDL queries on leader 2019-06-26 17:52:20 +03:00
test_distributed_format fixed tests 2020-02-12 17:11:03 +03:00
test_distributed_respect_user_timeouts add test 2020-02-06 20:23:05 +03:00
test_distributed_storage_configuration tests fixed 2020-02-12 17:35:23 +03:00
test_distributed_system_query Update test.py 2019-06-17 11:25:06 +03:00
test_extreme_deduplication Disable tests and add global timeout 2019-06-25 16:20:04 +03:00
test_filesystem_layout Revert tests 2019-12-17 16:03:23 +03:00
test_force_deduplication Update test.py 2020-02-23 04:31:12 +03:00
test_format_avro_confluent Add AvroConfluent integration test 2020-02-02 18:02:19 -06:00
test_format_schema_on_server Add test checking using format schema via HTTP interface. 2019-03-30 21:42:01 +03:00
test_globs_in_filepath Add virtual columns to hdfs and file table functions (#8489) 2020-01-15 10:52:45 +03:00
test_grant_and_revoke Add tests for roles. 2020-02-21 06:03:09 +03:00
test_graphite_merge_tree Add tests for system.graphite_retentions 2019-03-07 13:47:02 +01:00
test_host_ip_change fix deadlock in getHostName 2019-07-04 02:05:59 +03:00
test_https_replication Add test for HTTPS replication failover 2019-11-07 18:17:10 +03:00
test_inherit_multiple_profiles Add tests and documentation for settings and constraints order 2019-12-27 18:15:55 +01:00
test_insert_into_distributed fix test prefer_localhost_replica 2020-01-28 21:54:25 +03:00
test_insert_into_distributed_sync_async add tests [#CLICKHOUSE-4161] 2018-11-26 17:57:33 +03:00
test_insert_into_distributed_through_materialized_view Disable tests and add global timeout 2019-06-25 16:20:04 +03:00
test_log_family_s3 StorageStripeLog S3 support. 2020-02-14 17:28:33 +03:00
test_logs_level pep8 2019-07-17 14:57:05 +03:00
test_match_process_uid_against_data_owner Do not require the server to fail to start in the test. 2018-12-20 01:50:05 +03:00
test_max_http_connections_for_replication Fix exception 'No message received' 2019-03-29 21:10:03 +03:00
test_merge_table_over_distributed Fixed Illegal PREWHERE error for Merge tables over Distributed. [#CLICKHOUSE-3637] 2018-03-12 18:14:26 +03:00
test_merge_tree_s3 Storage MergeTree initial support for S3. 2020-02-27 19:47:40 +03:00
test_multiple_disks The restart_clickhouse() function in integration tests now waits until the server is restarted. 2020-02-28 16:34:19 +03:00
test_mutations_with_merge_tree fix test failure 2019-08-29 11:34:00 +08:00
test_mysql_database_engine Support detach and drop table for mysql database engine 2019-12-13 19:48:38 +08:00
test_mysql_protocol Add tests. 2020-03-01 00:27:37 +03:00
test_non_default_compression Fix non initialized codec and wrong size in CacheCompressedReadBuffer 2019-01-29 17:55:57 +03:00
test_odbc_interaction odbc table function now respects external_table_functions_use_nulls setting 2019-11-08 14:30:11 +03:00
test_old_versions Add integration test for the case when the initiator of a distributed query is older than a shard. 2019-11-17 03:51:58 +03:00
test_part_log_table improved changes 2019-11-04 22:10:02 +03:00
test_partition Revert tests 2019-12-17 16:03:23 +03:00
test_parts_delete_zookeeper Fix integration test 2020-01-24 11:55:36 +03:00
test_polymorphic_parts temporary disable tests for compatability between replicas 2020-02-28 13:14:02 +03:00
test_prometheus_endpoint Fix prometheus test 2020-01-24 12:16:34 +03:00
test_quota Add class GenericRoleSet to represent a set of IDs of users and roles. 2020-02-21 00:57:42 +03:00
test_random_inserts Fix random inserts test 2020-03-02 18:00:34 +03:00
test_read_temporary_tables_on_failure Read temporary tables on failure to avoid interpreting that bits as query 2019-12-08 23:26:43 +03:00
test_recovery_replica CLICKHOUSE-3894: Some improvements in flapping tests. Bug fix of ATTACH TABLE xxx ON CLUSTER (previously doesn't work). 2018-09-03 17:06:00 +03:00
test_redirect_url_storage + integration test 2019-09-23 07:27:49 +00:00
test_relative_filepath Fix 2019-12-05 17:22:00 +08:00
test_reload_max_table_size_to_drop Update dbms/tests/integration/test_reload_max_table_size_to_drop/test.py 2019-11-25 14:42:09 +03:00
test_reloading_storage_configuration Added tests for reloading storage configuration. 2020-03-04 15:10:17 +03:00
test_remote_prewhere Fix strange code 2019-08-15 12:43:31 +03:00
test_replace_partition CLICKHOUSE-3894: Some improvements in flapping tests. Bug fix of ATTACH TABLE xxx ON CLUSTER (previously doesn't work). 2018-09-03 17:06:00 +03:00
test_replica_can_become_leader Better code and tests 2019-08-01 20:43:10 +03:00
test_replicated_mutations better test 2019-08-22 22:54:42 +03:00
test_replicating_constants Added integration test with different constants from remote. 2019-11-09 15:16:34 +03:00
test_replication_credentials Remove wrong option 2018-07-27 20:25:51 +03:00
test_replication_without_zookeeper The restart_clickhouse() function in integration tests now waits until the server is restarted. 2020-02-28 16:34:19 +03:00
test_row_policy Improve behaviour of row policies, now it applies for a table always when defined for that table (even for another user). 2020-02-21 00:57:42 +03:00
test_send_request_to_leader_replica Improve test 2018-12-28 20:11:52 +03:00
test_server_initialization The restart_clickhouse() function in integration tests now waits until the server is restarted. 2020-02-28 16:34:19 +03:00
test_settings_constraints Improve settings constraints tests coverage 2019-12-17 17:20:15 +00:00
test_settings_constraints_distributed Shard now clamps the settings got from the initiator to the shard's constaints instead of throwing an exception. 2020-03-01 02:18:49 +03:00
test_storage_hdfs Add virtual columns to hdfs and file table functions (#8489) 2020-01-15 10:52:45 +03:00
test_storage_kafka Kafka: catch and log exceptions in desctructors. Fixes #9494 2020-03-04 17:45:03 +01:00
test_storage_mysql fix insert select from mysql(...) 2019-12-16 17:28:44 +03:00
test_storage_s3 Added tests for _file and _path in S3 storage. 2020-01-28 01:09:21 +03:00
test_system_merges Fixed test_system_merges/test.py::test_mutation_simple[replicated] test. 2019-12-07 22:23:25 +03:00
test_system_queries fix race condition in flushing system log 2019-06-19 15:28:34 +03:00
test_text_log_level Cover text_log.level 2020-01-25 14:04:56 +03:00
test_timezone_config Fixed error in test 2019-09-02 22:58:48 +03:00
test_tmp_policy Cover tmp_policy 2020-01-23 20:31:19 +03:00
test_ttl_move Merge pull request #8598 from excitoon-favorites/passttlsoverhttp 2020-02-19 12:09:25 +03:00
test_ttl_replicated fix flappy test 2020-03-03 22:23:41 +03:00
test_union_header Added test. 2018-10-23 20:48:58 +03:00
test_user_ip_restrictions Fix ipv4 mask restrictions for users and add integration tests 2018-12-27 18:55:51 +03:00
test_user_zero_database_access add test 2019-11-18 21:06:14 +03:00
test_zookeeper_config Added test for chroot in ZK [#MDBSUPPORT-204] 2018-07-10 19:14:27 +03:00
.dockerignore https://stackoverflow.com/questions/44067609/getting-error-importmismatcherror-while-running-py-test 2018-11-23 20:56:50 +03:00
.gitignore Fix trying to load database.sql from incorrect path. [#CLICKHOUSE-3044] 2017-06-16 19:08:24 +03:00
CMakeLists.txt CLICKHOUSE-4421 Fix segfault in copier (#4835) 2019-04-01 17:58:04 +03:00
conftest.py ClickHouse integration tests. [#CLICKHOUSE-2821] 2017-05-22 17:56:41 +04:00
pytest.ini Decrease test timeout 2020-02-26 15:08:57 +03:00
README.md Update README.md 2020-02-26 15:33:06 +03:00
runner Tests finally works! 2019-09-04 13:39:54 +00:00

ClickHouse integration tests

This directory contains tests that involve several ClickHouse instances, custom configs, ZooKeeper, etc.

Running natively

Prerequisites:

  • Ubuntu 14.04 (Trusty) or higher.
  • docker. Minimum required API version: 1.25, check with docker version.

You must install latest Docker from https://docs.docker.com/engine/installation/linux/docker-ce/ubuntu/#set-up-the-repository Don't use Docker from your system repository.

  • pip and libpq-dev. To install: sudo apt-get install python-pip libpq-dev
  • py.test testing framework. To install: sudo -H pip install pytest
  • docker-compose and additional python libraries. To install: sudo -H pip install docker-compose docker dicttoxml kazoo PyMySQL psycopg2 pymongo tzlocal kafka-python protobuf pytest-timeout minio rpm-confluent-schemaregistry

(highly not recommended) If you really want to use OS packages on modern debian/ubuntu instead of "pip": sudo apt install -y docker docker-compose python-pytest python-dicttoxml python-docker python-pymysql python-pymongo python-tzlocal python-kazoo python-psycopg2 python-kafka python-pytest-timeout python-minio

If you want to run the tests under a non-privileged user, you must add this user to docker group: sudo usermod -aG docker $USER and re-login. (You must close all your sessions (for example, restart your computer)) To check, that you have access to Docker, run docker ps.

Run the tests with the pytest command. To select which tests to run, use: pytest -k <test_name_pattern>

By default tests are run with system-wide client binary, server binary and base configs. To change that, set the following environment variables:

  • CLICKHOUSE_TESTS_SERVER_BIN_PATH to choose the server binary.
  • CLICKHOUSE_TESTS_CLIENT_BIN_PATH to choose the client binary.
  • CLICKHOUSE_TESTS_BASE_CONFIG_DIR to choose the directory from which base configs (config.xml and users.xml) are taken.

Running with runner script

The only requirement is fresh configured docker and docker pull yandex/clickhouse-integration-tests-runner

Notes:

  • If you want to run integration tests without sudo you have to add your user to docker group sudo usermod -aG docker $USER. More information about docker configuration.
  • If you already had run these tests without ./runner script you may have problems with pytest cache. It can be removed with rm -r __pycache__ .pytest_cache/.
  • Some tests maybe require a lot of resources (CPU, RAM, etc.). Better not try large tests like test_cluster_copier or test_distributed_ddl* on your notebook.

You can run tests via ./runner script and pass pytest arguments as last arg:

$ ./runner --binary $HOME/ClickHouse/dbms/programs/clickhouse  --bridge-binary $HOME/ClickHouse/dbms/programs/clickhouse-odbc-bridge --configs-dir $HOME/ClickHouse/dbms/programs/server/ 'test_odbc_interaction -ss'
Start tests
============================= test session starts ==============================
platform linux2 -- Python 2.7.15rc1, pytest-4.0.0, py-1.7.0, pluggy-0.8.0
rootdir: /ClickHouse/dbms/tests/integration, inifile: pytest.ini
collected 6 items

test_odbc_interaction/test.py Removing network clickhouse_default
...

Killing roottestodbcinteraction_node1_1     ... done
Killing roottestodbcinteraction_mysql1_1    ... done
Killing roottestodbcinteraction_postgres1_1 ... done
Removing roottestodbcinteraction_node1_1     ... done
Removing roottestodbcinteraction_mysql1_1    ... done
Removing roottestodbcinteraction_postgres1_1 ... done
Removing network roottestodbcinteraction_default

==================== 6 passed, 1 warnings in 95.21 seconds =====================

Path to binary and configs maybe specified via env variables:

$ export CLICKHOUSE_TESTS_BASE_CONFIG_DIR=$HOME/ClickHouse/dbms/programs/server/
$ export CLICKHOUSE_TESTS_SERVER_BIN_PATH=$HOME/ClickHouse/dbms/programs/clickhouse
$ export CLICKHOUSE_TESTS_ODBC_BRIDGE_BIN_PATH=$HOME/ClickHouse/dbms/programs/clickhouse-odbc-bridge
$ ./runner 'test_odbc_interaction'
$ # or ./runner '-v -ss'
Start tests
============================= test session starts ==============================
platform linux2 -- Python 2.7.15rc1, pytest-4.0.0, py-1.7.0, pluggy-0.8.0
rootdir: /ClickHouse/dbms/tests/integration, inifile: pytest.ini
collected 6 items

test_odbc_interaction/test.py ......                                     [100%]
==================== 6 passed, 1 warnings in 96.33 seconds =====================

You can just open shell inside a container by overwritting the command: ./runner --command=bash

Adding new tests

To add new test named foo, create a directory test_foo with an empty __init__.py and a file named test.py containing tests in it. All functions with names starting with test will become test cases.

helpers directory contains utilities for:

  • Launching a ClickHouse cluster with or without ZooKeeper indocker containers.
  • Sending queries to launched instances.
  • Introducing network failures such as severing network link between two instances.

To assert that two TSV files must be equal, wrap them in the TSV class and use the regular assert statement. Example: assert TSV(result) == TSV(reference). In case the assertion fails, pytest will automagically detect the types of variables and only the small diff of two files is printed.