ClickHouse/tests/integration
2020-06-08 14:25:30 +03:00
..
helpers Create 'root' S3 bucket before CH is started. 2020-05-22 14:48:08 +03:00
test_access_control_on_cluster Use enum Type instead of std::type_index to represent the type of IAccessEntity. 2020-05-13 19:40:48 +03:00
test_adaptive_granularity
test_adaptive_granularity_different_settings Fix adaptive granularity compatibility 2020-04-08 12:51:04 +03:00
test_adaptive_granularity_replicated
test_aggregation_memory_efficient Remove experimental_use_processors setting, part 1. (#10924) 2020-05-15 22:28:10 +03:00
test_allowed_client_hosts Enable access management by default for all integration tests. 2020-04-09 10:22:51 +03:00
test_allowed_url_from_config
test_alter_codec Allowed to alter column in non-modifying data mode when the same type is specified. 2020-04-20 14:15:06 +03:00
test_always_fetch_merged fix test 2020-04-20 18:17:34 +03:00
test_atomic_drop_table
test_authentication Fix settings NO_PASSWORD authentication mode in users.xml. 2020-05-25 11:35:26 +03:00
test_backup_restore
test_backward_compatability Merge pull request #10912 from ClickHouse/static-mutate 2020-05-17 09:05:35 +03:00
test_block_structure_mismatch
test_check_table
test_cluster_all_replicas
test_cluster_copier fix test 2020-04-22 14:35:18 +03:00
test_concurrent_queries_for_user_restriction
test_config_corresponding_root Optional secured communication between ClickHouse and Zookeeper 2020-04-09 01:40:54 +03:00
test_config_substitutions
test_consistant_parts_after_move_partition
test_consistent_parts_after_clone_replica
test_cross_replication
test_ddl_alter_query Add test for alter on cluster 2020-04-21 16:47:00 +03:00
test_default_role Fix crash when SET DEFAULT ROLE is called with wrong arguments. 2020-05-29 09:54:27 +03:00
test_delayed_replica_failover Cover fallback_to_stale_replicas in conjunction with skip_unavailable_shards 2020-04-22 01:47:22 +03:00
test_dictionaries_access Get dictionary and check access rights only once per each call of any function reading external dictionaries. 2020-05-15 03:14:14 +03:00
test_dictionaries_all_layouts_and_sources Merge pull request #10915 from ClickHouse/aku/mongodb-uri 2020-05-21 23:02:12 +03:00
test_dictionaries_complex_key_cache_string
test_dictionaries_ddl
test_dictionaries_depend_on_dictionaries
test_dictionaries_mysql Fix test 2020-04-15 20:52:05 +03:00
test_dictionaries_null_value
test_dictionaries_select_all
test_dictionaries_update_and_reload
test_dictionary_allow_read_expired_keys fixed 2020-05-27 22:44:22 +03:00
test_dictionary_custom_settings fixed test 2020-04-10 19:09:44 +03:00
test_dictionary_ddl_on_cluster
test_disk_access_storage Show authentication type in table system.users and while executing SHOW CREATE USER query. 2020-05-28 08:34:10 +03:00
test_distributed_backward_compatability trigger CI 2020-05-14 23:44:52 +03:00
test_distributed_ddl fix integration/test_distributed_ddl/test_replicated_alter 2020-04-14 10:15:01 +03:00
test_distributed_ddl_password
test_distributed_format
test_distributed_over_distributed
test_distributed_over_live_view Removing comments. 2020-04-10 14:09:28 -04:00
test_distributed_respect_user_timeouts
test_distributed_storage_configuration Fix false-positive failures of the test_distributed_storage_configuration 2020-05-03 14:48:00 +03:00
test_distributed_system_query
test_enabling_access_management readonly user now can execute SHOW CREATE for access entities. 2020-04-09 10:22:51 +03:00
test_extreme_deduplication
test_filesystem_layout
test_force_deduplication fix test 2020-04-09 21:02:27 +03:00
test_format_avro_confluent
test_format_schema_on_server
test_globs_in_filepath Fix behaviour of globs in filepath with leading zeros 2020-04-07 16:46:12 +03:00
test_grant_and_revoke Show authentication type in table system.users and while executing SHOW CREATE USER query. 2020-05-28 08:34:10 +03:00
test_graphite_merge_tree
test_host_ip_change use DNSResolver to check if user is allowed to connect 2020-06-07 00:02:29 +03:00
test_http_and_readonly
test_http_handlers_config remove redundant configs from test 2020-04-28 02:05:15 +03:00
test_https_replication ISSUES-5436 add integration test 2020-04-27 18:49:32 +08:00
test_inherit_multiple_profiles
test_insert_distributed_load_balancing More clear test 2020-06-02 21:47:30 +03:00
test_insert_into_distributed Fix flacky integration test 2020-04-20 06:15:29 +03:00
test_insert_into_distributed_sync_async Fix sync_async test (remove timeout) 2020-06-01 22:35:56 +03:00
test_insert_into_distributed_through_materialized_view
test_live_view_over_distributed Removing comments. 2020-04-10 14:09:28 -04:00
test_log_family_s3 Create 'root' S3 bucket before CH is started. 2020-05-22 14:48:08 +03:00
test_logs_level
test_match_process_uid_against_data_owner
test_max_http_connections_for_replication
test_merge_table_over_distributed
test_merge_tree_s3 Create 'root' S3 bucket before CH is started. 2020-05-22 14:48:08 +03:00
test_multiple_disks Minor fixes in test_ttl_move. 2020-05-20 12:59:06 +03:00
test_mutations_with_merge_tree
test_mysql_database_engine ISSUES-10056 reused evaluateConstantExpressionOrIdentifierAsLiteral 2020-04-09 01:31:28 +08:00
test_mysql_protocol Debug bad test "mysql_federated" 2020-05-30 20:03:28 +03:00
test_no_local_metadata_node Fix replicated tables startup when updating from old version 2020-04-07 18:28:29 +03:00
test_non_default_compression Fix integration test 2020-05-04 17:51:29 +03:00
test_odbc_interaction better test 2020-05-15 00:51:21 +03:00
test_old_versions
test_part_log_table Fix part_log test 2020-06-01 12:12:52 +03:00
test_partition
test_parts_delete_zookeeper
test_polymorphic_parts Merge pull request #10531 from CurtizJ/polymorphic-parts-2 2020-04-30 12:15:25 +03:00
test_prometheus_endpoint
test_quorum_inserts bump 2020-04-15 03:04:12 +04:00
test_quota Improve system tables for quotas. Remove function currentQuota(). 2020-05-13 19:40:48 +03:00
test_random_inserts
test_read_temporary_tables_on_failure
test_recovery_replica
test_redirect_url_storage
test_relative_filepath
test_reload_max_table_size_to_drop
test_reloading_storage_configuration
test_remote_prewhere
test_rename_column Try to enable skipped tests 2020-05-18 13:02:52 +03:00
test_replace_partition
test_replica_can_become_leader
test_replicated_merge_tree_s3 Enable test_replicated_mergee_tree_s3 2020-05-22 14:55:30 +03:00
test_replicated_mutations
test_replicating_constants
test_replication_credentials
test_replication_without_zookeeper
test_row_policy Add test for row policy that policy defined in users.xml affects only the user it assigned to. 2020-05-27 20:37:43 +03:00
test_s3_with_proxy Add default credentials and custom headers for s3 table functions. 2020-06-02 20:48:49 +03:00
test_send_request_to_leader_replica
test_server_initialization Fix test_server_initialization, but looking into default database only 2020-05-19 10:43:37 +03:00
test_settings_constraints
test_settings_constraints_distributed Added a change from @azat in #10277 2020-04-18 14:48:51 +03:00
test_settings_profile Add system tables for settings profiles. 2020-05-14 14:12:15 +03:00
test_storage_hdfs
test_storage_kafka Update test.py 2020-06-06 19:48:03 +03:00
test_storage_mysql
test_storage_s3 Add default credentials and custom headers for s3 table functions. 2020-06-02 20:48:49 +03:00
test_system_merges
test_system_queries Update test.py 2020-05-27 11:25:36 +03:00
test_text_log_level
test_timezone_config
test_tmp_policy
test_ttl_move Merging #8377 2020-06-08 14:25:30 +03:00
test_ttl_replicated Add test for different ttl versions 2020-06-04 17:18:54 +03:00
test_union_header
test_user_ip_restrictions
test_user_zero_database_access
test_version_update_after_mutation Fix primry key check 2020-05-26 12:46:04 +03:00
test_zookeeper_config Bump compose version to 2.3, and make tmpfs for zookeeper optional 2020-05-19 18:27:10 +03:00
.dockerignore
.gitignore
CMakeLists.txt
conftest.py
pytest.ini improvements 2020-04-17 11:42:51 +03:00
README.md Move integration tests docker files to docker/ directory. Rename image -> runner. 2020-04-17 19:01:16 +03:00
runner Fix runner script 2020-04-06 21:30:51 +03:00

ClickHouse integration tests

This directory contains tests that involve several ClickHouse instances, custom configs, ZooKeeper, etc.

Running natively

Prerequisites:

  • Ubuntu 14.04 (Trusty) or higher.
  • docker. Minimum required API version: 1.25, check with docker version.

You must install latest Docker from https://docs.docker.com/engine/installation/linux/docker-ce/ubuntu/#set-up-the-repository Don't use Docker from your system repository.

  • pip and libpq-dev. To install: sudo apt-get install python-pip libpq-dev zlib1g-dev libcrypto++-dev libssl-dev
  • py.test testing framework. To install: sudo -H pip install pytest
  • docker-compose and additional python libraries. To install: sudo -H pip install urllib3==1.23 pytest docker-compose==1.22.0 docker dicttoxml kazoo PyMySQL psycopg2==2.7.5 pymongo tzlocal kafka-python protobuf redis aerospike pytest-timeout minio rpm-confluent-schemaregistry

(highly not recommended) If you really want to use OS packages on modern debian/ubuntu instead of "pip": sudo apt install -y docker docker-compose python-pytest python-dicttoxml python-docker python-pymysql python-pymongo python-tzlocal python-kazoo python-psycopg2 python-kafka python-pytest-timeout python-minio

If you want to run the tests under a non-privileged user, you must add this user to docker group: sudo usermod -aG docker $USER and re-login. (You must close all your sessions (for example, restart your computer)) To check, that you have access to Docker, run docker ps.

Run the tests with the pytest command. To select which tests to run, use: pytest -k <test_name_pattern>

By default tests are run with system-wide client binary, server binary and base configs. To change that, set the following environment variables:

  • CLICKHOUSE_TESTS_SERVER_BIN_PATH to choose the server binary.
  • CLICKHOUSE_TESTS_CLIENT_BIN_PATH to choose the client binary.
  • CLICKHOUSE_TESTS_BASE_CONFIG_DIR to choose the directory from which base configs (config.xml and users.xml) are taken.

Running with runner script

The only requirement is fresh configured docker and docker pull yandex/clickhouse-integration-tests-runner

Notes:

  • If you want to run integration tests without sudo you have to add your user to docker group sudo usermod -aG docker $USER. More information about docker configuration.
  • If you already had run these tests without ./runner script you may have problems with pytest cache. It can be removed with rm -r __pycache__ .pytest_cache/.
  • Some tests maybe require a lot of resources (CPU, RAM, etc.). Better not try large tests like test_cluster_copier or test_distributed_ddl* on your laptop.

You can run tests via ./runner script and pass pytest arguments as last arg:

$ ./runner --binary $HOME/ClickHouse/programs/clickhouse  --bridge-binary $HOME/ClickHouse/programs/clickhouse-odbc-bridge --configs-dir $HOME/ClickHouse/programs/server/ 'test_odbc_interaction -ss'
Start tests
============================= test session starts ==============================
platform linux2 -- Python 2.7.15rc1, pytest-4.0.0, py-1.7.0, pluggy-0.8.0
rootdir: /ClickHouse/tests/integration, inifile: pytest.ini
collected 6 items

test_odbc_interaction/test.py Removing network clickhouse_default
...

Killing roottestodbcinteraction_node1_1     ... done
Killing roottestodbcinteraction_mysql1_1    ... done
Killing roottestodbcinteraction_postgres1_1 ... done
Removing roottestodbcinteraction_node1_1     ... done
Removing roottestodbcinteraction_mysql1_1    ... done
Removing roottestodbcinteraction_postgres1_1 ... done
Removing network roottestodbcinteraction_default

==================== 6 passed, 1 warnings in 95.21 seconds =====================

Path to binary and configs maybe specified via env variables:

$ export CLICKHOUSE_TESTS_BASE_CONFIG_DIR=$HOME/ClickHouse/programs/server/
$ export CLICKHOUSE_TESTS_SERVER_BIN_PATH=$HOME/ClickHouse/programs/clickhouse
$ export CLICKHOUSE_TESTS_ODBC_BRIDGE_BIN_PATH=$HOME/ClickHouse/programs/clickhouse-odbc-bridge
$ ./runner 'test_odbc_interaction'
$ # or ./runner '-v -ss'
Start tests
============================= test session starts ==============================
platform linux2 -- Python 2.7.15rc1, pytest-4.0.0, py-1.7.0, pluggy-0.8.0
rootdir: /ClickHouse/tests/integration, inifile: pytest.ini
collected 6 items

test_odbc_interaction/test.py ......                                     [100%]
==================== 6 passed, 1 warnings in 96.33 seconds =====================

You can just open shell inside a container by overwritting the command: ./runner --command=bash

Rebuilding the docker containers

The main container used for integration tests lives in docker/test/integration/Dockerfile. Rebuild it with

cd docker/test/integration
docker build -t yandex/clickhouse-integration-test .

The helper container used by the runner script is in docker/test/integration/runner/Dockerfile.

Adding new tests

To add new test named foo, create a directory test_foo with an empty __init__.py and a file named test.py containing tests in it. All functions with names starting with test will become test cases.

helpers directory contains utilities for:

  • Launching a ClickHouse cluster with or without ZooKeeper in docker containers.
  • Sending queries to launched instances.
  • Introducing network failures such as severing network link between two instances.

To assert that two TSV files must be equal, wrap them in the TSV class and use the regular assert statement. Example: assert TSV(result) == TSV(reference). In case the assertion fails, pytest will automagically detect the types of variables and only the small diff of two files is printed.