ClickHouse/tests/integration
2021-10-17 20:54:46 +03:00
..
helpers Grep server log even if it contains binary data (#29903) 2021-10-10 12:55:37 +03:00
test_access_control_on_cluster
test_access_for_functions
test_aggregation_memory_efficient
test_allowed_client_hosts
test_allowed_url_from_config
test_alter_codec
test_alter_on_mixed_type_cluster
test_alter_settings_on_cluster
test_alter_update_cast_keep_nullable
test_always_fetch_merged
test_async_drain_connection
test_asynchronous_metric_log_table
test_atomic_drop_table
test_attach_without_checksums Improve some tests 2021-10-01 08:44:50 +03:00
test_attach_without_fetching
test_authentication
test_backup_restore Fix test_backup_restore after #29649 2021-10-09 15:14:20 +03:00
test_backup_restore_new
test_backup_with_other_granularity
test_backward_compatibility
test_block_structure_mismatch
test_broken_part_during_merge
test_buffer_profile
test_catboost_model_config_reload Replace yandex to clickhouse in configs 2021-09-25 07:08:34 +03:00
test_catboost_model_first_evaluate
test_catboost_model_reload
test_check_table
test_cleanup_dir_after_bad_zk_conn
test_cluster_all_replicas
test_cluster_copier
test_codec_encrypted Replace yandex to clickhouse in configs 2021-09-25 07:08:34 +03:00
test_compression_codec_read
test_compression_nested_columns
test_concurrent_queries_for_all_users_restriction
test_concurrent_queries_for_user_restriction
test_concurrent_ttl_merges
test_config_corresponding_root .tech -> .com 2021-09-22 03:22:57 +03:00
test_config_substitutions Compatibility 2021-09-29 02:41:12 +03:00
test_config_xml_full .tech -> .com 2021-09-22 03:22:57 +03:00
test_config_xml_main Move all yaml tests to clickhouse config 2021-09-23 11:08:40 +03:00
test_config_xml_yaml_mix Move all yaml tests to clickhouse config 2021-09-23 11:08:40 +03:00
test_config_yaml_full Merge branch 'master' into yandex-to-clickhouse-in-configs 2021-09-23 20:49:40 +03:00
test_config_yaml_main Move all yaml tests to clickhouse config 2021-09-23 11:08:40 +03:00
test_consistant_parts_after_move_partition
test_consistent_parts_after_clone_replica
test_create_user_and_login Replace yandex to clickhouse in configs 2021-09-25 07:08:34 +03:00
test_cross_replication
test_custom_settings
test_ddl_alter_query
test_ddl_worker_non_leader
test_default_compression_codec Fix compatibility 2021-09-26 03:19:50 +03:00
test_default_database_on_cluster
test_default_role
test_delayed_replica_failover
test_dictionaries_access
test_dictionaries_all_layouts_separate_sources
test_dictionaries_complex_key_cache_string
test_dictionaries_config_reload Added tests 2021-10-01 16:44:09 +03:00
test_dictionaries_ddl Merge pull request #29603 from qoega/improve-integration-2 2021-10-02 06:09:24 +03:00
test_dictionaries_dependency fix test 2021-10-07 13:21:42 +03:00
test_dictionaries_dependency_xml Merge branch 'master' into yandex-to-clickhouse-in-configs 2021-09-21 20:17:28 +03:00
test_dictionaries_mysql Replace yandex to clickhouse in configs 2021-09-25 07:08:34 +03:00
test_dictionaries_null_value
test_dictionaries_postgresql Replace yandex to clickhouse in configs 2021-09-25 07:08:34 +03:00
test_dictionaries_redis
test_dictionaries_select_all Replace yandex to clickhouse in configs 2021-09-25 07:08:34 +03:00
test_dictionaries_update_and_reload
test_dictionaries_update_field
test_dictionary_allow_read_expired_keys
test_dictionary_custom_settings
test_dictionary_ddl_on_cluster
test_disabled_mysql_server
test_disk_access_storage
test_disk_over_web_server
test_disk_types
test_distributed_backward_compatability Compatibility 2021-09-29 02:41:12 +03:00
test_distributed_ddl
test_distributed_ddl_on_cross_replication
test_distributed_ddl_parallel
test_distributed_ddl_password
test_distributed_directory_monitor_split_batch_on_failure
test_distributed_format
test_distributed_inter_server_secret Merge pull request #29060 from azat/inter-server-secret-auth-fix 2021-10-01 20:44:48 +03:00
test_distributed_load_balancing
test_distributed_over_distributed
test_distributed_queries_stress
test_distributed_respect_user_timeouts
test_distributed_storage_configuration
test_distributed_system_query
test_drop_replica
test_enabling_access_management
test_encrypted_disk Replace yandex to clickhouse in configs 2021-09-25 07:08:34 +03:00
test_executable_table_function Update test.py 2021-10-01 05:04:12 +03:00
test_executable_user_defined_functions_config_reload Added tests 2021-10-01 16:44:09 +03:00
test_explain_estimates
test_extreme_deduplication
test_fetch_partition_from_auxiliary_zookeeper
test_fetch_partition_should_reset_mutation
test_fetch_partition_with_outdated_parts
test_filesystem_layout
test_force_deduplication
test_force_drop_table
test_format_avro_confluent
test_format_schema_on_server
test_freeze_table
test_globs_in_filepath
test_grant_and_revoke
test_graphite_merge_tree
test_grpc_protocol Fix releasing query ID and session ID at the end of query processing. 2021-10-10 17:34:45 +03:00
test_grpc_protocol_ssl
test_hedged_requests Replace yandex to clickhouse in configs 2021-09-25 07:08:34 +03:00
test_hedged_requests_parallel Replace yandex to clickhouse in configs 2021-09-25 07:08:34 +03:00
test_host_ip_change
test_http_and_readonly
test_http_handlers_config
test_https_replication .tech -> .com 2021-09-22 03:22:57 +03:00
test_inherit_multiple_profiles
test_input_format_parallel_parsing_memory_tracking Fix test_input_format_parallel_parsing_memory_tracking::test_memory_tracking_total 2021-10-09 16:15:42 +03:00
test_insert_distributed_async_extra_dirs
test_insert_distributed_async_send
test_insert_distributed_load_balancing
test_insert_into_distributed
test_insert_into_distributed_sync_async
test_insert_into_distributed_through_materialized_view
test_jbod_balancer
test_jdbc_bridge
test_join_set_family_s3
test_keeper_auth
test_keeper_back_to_back
test_keeper_internal_secure
test_keeper_multinode_blocade_leader
test_keeper_multinode_simple
test_keeper_persistent_log Merge branch 'master' into yandex-to-clickhouse-in-configs 2021-09-25 01:38:39 +03:00
test_keeper_persistent_log_multinode
test_keeper_restore_from_snapshot
test_keeper_secure_client
test_keeper_snapshot_small_distance
test_keeper_snapshots
test_keeper_snapshots_multinode
test_keeper_two_nodes_cluster
test_keeper_zookeeper_converter
test_library_bridge
test_limited_replicated_fetches
test_log_family_hdfs
test_log_family_s3
test_log_levels_update Add test 2021-10-14 16:34:30 +03:00
test_log_query_probability
test_logs_level
test_lost_part
test_lost_part_during_startup
test_match_process_uid_against_data_owner
test_materialized_mysql_database MaterializedMySQL: Update GTID set at end of transaction 2021-10-07 14:09:00 +02:00
test_max_http_connections_for_replication
test_max_suspicious_broken_parts
test_MemoryTracking Make test_MemoryTracking::test_http not flaky 2021-10-14 21:06:39 +03:00
test_merge_table_over_distributed
test_merge_tree_empty_parts
test_merge_tree_hdfs
test_merge_tree_s3 Ping tests 2021-10-05 13:19:43 +00:00
test_merge_tree_s3_failover
test_merge_tree_s3_restore
test_merge_tree_s3_with_cache
test_multiple_disks
test_mutations_hardlinks
test_mutations_in_partitions_of_merge_tree
test_mutations_with_merge_tree
test_mysql_database_engine Replace yandex to clickhouse in configs 2021-09-25 07:08:34 +03:00
test_mysql_protocol
test_nlp
test_no_local_metadata_node
test_non_default_compression
test_odbc_interaction
test_old_versions Compatibility 2021-09-29 02:41:12 +03:00
test_on_cluster_timeouts
test_optimize_on_insert
test_part_log_table
test_part_moves_between_shards
test_part_uuid
test_partition
test_parts_delete_zookeeper
test_passing_max_partitions_to_read_remotely
test_polymorphic_parts Compatibility 2021-09-29 02:41:12 +03:00
test_postgresql_database_engine Replace yandex to clickhouse in configs 2021-09-25 07:08:34 +03:00
test_postgresql_protocol
test_postgresql_replica_database_engine Fix 2021-10-08 23:52:15 +03:00
test_profile_events_s3
test_prometheus_endpoint
test_query_deduplication
test_quorum_inserts
test_quorum_inserts_parallel
test_quota
test_random_inserts
test_range_hashed_dictionary_types
test_read_temporary_tables_on_failure
test_recompression_ttl
test_recovery_replica
test_redirect_url_storage Fix url with failover case 2021-10-03 13:58:03 +00:00
test_relative_filepath
test_reload_auxiliary_zookeepers Replace yandex to clickhouse in configs 2021-09-25 07:08:34 +03:00
test_reload_clusters_config Replace yandex to clickhouse in configs 2021-09-25 07:08:34 +03:00
test_reload_max_table_size_to_drop
test_reload_zookeeper Fix whitespace 2021-09-25 20:07:48 +03:00
test_reloading_settings_from_users_xml
test_reloading_storage_configuration Replace yandex to clickhouse in configs 2021-09-25 07:08:34 +03:00
test_remote_prewhere
test_rename_column
test_replace_partition
test_replica_can_become_leader
test_replica_is_active
test_replicated_database
test_replicated_fetches_bandwidth
test_replicated_fetches_timeouts
test_replicated_merge_tree_compatibility normalize ASTs in ColumnsDescription 2021-09-22 18:55:20 +03:00
test_replicated_merge_tree_config
test_replicated_merge_tree_hdfs_zero_copy
test_replicated_merge_tree_s3
test_replicated_merge_tree_s3_zero_copy
test_replicated_merge_tree_with_auxiliary_zookeepers
test_replicated_mutations Merge remote-tracking branch 'origin/master' into yandex-to-clickhouse-in-configs 2021-10-01 01:33:58 +03:00
test_replicated_parse_zk_metadata
test_replicated_users
test_replicating_constants
test_replication_credentials Replace yandex to clickhouse in configs 2021-09-25 07:08:34 +03:00
test_replication_without_zookeeper
test_restart_server Remove metadata leftovers on drop database (#30054) 2021-10-13 13:34:18 +03:00
test_restore_replica
test_rocksdb_options Merge branch 'master' into yandex-to-clickhouse-in-configs 2021-09-28 02:27:36 +03:00
test_role
test_row_policy
test_s3_cluster
test_s3_with_https
test_s3_with_proxy
test_s3_zero_copy_replication
test_secure_socket Replace yandex to clickhouse in configs 2021-09-25 07:08:34 +03:00
test_select_access_rights
test_send_crash_reports Merge branch 'master' into yandex-to-clickhouse-in-configs 2021-09-28 02:27:36 +03:00
test_send_request_to_leader_replica
test_server_initialization
test_settings_constraints
test_settings_constraints_distributed
test_settings_profile
test_shard_level_const_function
test_sharding_key_from_default_column Add test. 2021-10-04 10:56:38 +03:00
test_storage_hdfs
test_storage_kafka Remove more data streams (#29491) 2021-10-07 11:26:08 +03:00
test_storage_kerberized_hdfs .tech -> .com 2021-09-22 03:22:57 +03:00
test_storage_kerberized_kafka .tech -> .com 2021-09-22 03:22:57 +03:00
test_storage_mongodb Replace yandex to clickhouse in configs 2021-09-25 07:08:34 +03:00
test_storage_mysql Merge remote-tracking branch 'origin/master' into yandex-to-clickhouse-in-configs 2021-09-30 17:45:14 +03:00
test_storage_postgresql Replace yandex to clickhouse in configs 2021-09-25 07:08:34 +03:00
test_storage_postgresql_replica
test_storage_rabbitmq Fix rabbitmq tests. 2021-10-11 00:00:12 +03:00
test_storage_s3 Replace yandex to clickhouse in configs 2021-09-25 07:08:34 +03:00
test_system_clusters_actual_information
test_system_ddl_worker_queue
test_system_flush_logs
test_system_merges
test_system_metrics
test_system_queries Replace yandex to clickhouse in configs 2021-09-25 07:08:34 +03:00
test_system_replicated_fetches
test_table_functions_access_rights
test_tcp_handler_http_responses
test_text_log_level
test_timezone_config
test_tmp_policy
test_ttl_move
test_ttl_replicated
test_union_header
test_user_defined_object_persistence
test_user_directories
test_user_ip_restrictions
test_user_zero_database_access
test_version_update
test_version_update_after_mutation
test_zookeeper_config
.dockerignore
.gitignore
02044_exists_operator.reference Exists 2021-10-04 13:12:30 +00:00
02044_exists_operator.sql Exists 2021-10-04 13:12:30 +00:00
ci-runner.py
CMakeLists.txt
conftest.py
parallel_skip.json
parallel.json
parallel.readme
pytest.ini
README.md
runner ro 2021-09-21 18:31:32 +03:00

ClickHouse integration tests

This directory contains tests that involve several ClickHouse instances, custom configs, ZooKeeper, etc.

Running natively

Prerequisites:

  • Ubuntu 20.04 (Focal) or higher.
  • docker. Minimum required API version: 1.25, check with docker version.

You must install latest Docker from https://docs.docker.com/engine/installation/linux/docker-ce/ubuntu/#set-up-the-repository Don't use Docker from your system repository.

  • pip and libpq-dev. To install: sudo apt-get install python3-pip libpq-dev zlib1g-dev libcrypto++-dev libssl-dev libkrb5-dev python3-dev
  • py.test testing framework. To install: sudo -H pip install pytest
  • docker-compose and additional python libraries. To install:
sudo -H pip install \
    PyMySQL \
    aerospike \
    avro \
    cassandra-driver \
    confluent-kafka \
    dicttoxml \
    docker \
    docker-compose \
    grpcio \
    grpcio-tools \
    kafka-python \
    kazoo \
    minio \
    protobuf \
    psycopg2-binary \
    pymongo \
    pytz \
    pytest \
    pytest-timeout \
    redis \
    tzlocal==2.1 \
    urllib3 \
    requests-kerberos \
    dict2xml \
    hypothesis \
    pyhdfs \
    pika

(highly not recommended) If you really want to use OS packages on modern debian/ubuntu instead of "pip": sudo apt install -y docker docker-compose python3-pytest python3-dicttoxml python3-docker python3-pymysql python3-protobuf python3-pymongo python3-tzlocal python3-kazoo python3-psycopg2 kafka-python python3-pytest-timeout python3-minio

If you want to run the tests under a non-privileged user, you must add this user to docker group: sudo usermod -aG docker $USER and re-login. (You must close all your sessions (for example, restart your computer)) To check, that you have access to Docker, run docker ps.

Run the tests with the pytest command. To select which tests to run, use: pytest -k <test_name_pattern>

By default tests are run with system-wide client binary, server binary and base configs. To change that, set the following environment variables:

  • CLICKHOUSE_TESTS_SERVER_BIN_PATH to choose the server binary.
  • CLICKHOUSE_TESTS_CLIENT_BIN_PATH to choose the client binary.
  • CLICKHOUSE_TESTS_BASE_CONFIG_DIR to choose the directory from which base configs (config.xml andusers.xml) are taken.

For tests that use common docker compose files you may need to set up their path with environment variable: DOCKER_COMPOSE_DIR=$HOME/ClickHouse/docker/test/integration/runner/compose

Running with runner script

The only requirement is fresh configured docker and docker pull clickhouse/integration-tests-runner

Notes:

  • If you want to run integration tests without sudo you have to add your user to docker group sudo usermod -aG docker $USER. More information about docker configuration.
  • If you already had run these tests without ./runner script you may have problems with pytest cache. It can be removed with rm -r __pycache__ .pytest_cache/.
  • Some tests maybe require a lot of resources (CPU, RAM, etc.). Better not try large tests like test_cluster_copier or test_distributed_ddl* on your laptop.

You can run tests via ./runner script and pass pytest arguments as last arg:

$ ./runner --binary $HOME/ClickHouse/programs/clickhouse  --bridge-binary $HOME/ClickHouse/programs/clickhouse-odbc-bridge --base-configs-dir $HOME/ClickHouse/programs/server/ 'test_odbc_interaction -ss'
Start tests
============================= test session starts ==============================
platform linux2 -- Python 2.7.15rc1, pytest-4.0.0, py-1.7.0, pluggy-0.8.0
rootdir: /ClickHouse/tests/integration, inifile: pytest.ini
collected 6 items

test_odbc_interaction/test.py Removing network clickhouse_default
...

Killing roottestodbcinteraction_node1_1     ... done
Killing roottestodbcinteraction_mysql1_1    ... done
Killing roottestodbcinteraction_postgres1_1 ... done
Removing roottestodbcinteraction_node1_1     ... done
Removing roottestodbcinteraction_mysql1_1    ... done
Removing roottestodbcinteraction_postgres1_1 ... done
Removing network roottestodbcinteraction_default

==================== 6 passed, 1 warnings in 95.21 seconds =====================

Path to binary and configs maybe specified via env variables:

$ export CLICKHOUSE_TESTS_BASE_CONFIG_DIR=$HOME/ClickHouse/programs/server/
$ export CLICKHOUSE_TESTS_SERVER_BIN_PATH=$HOME/ClickHouse/programs/clickhouse
$ export CLICKHOUSE_TESTS_ODBC_BRIDGE_BIN_PATH=$HOME/ClickHouse/programs/clickhouse-odbc-bridge
$ ./runner 'test_odbc_interaction'
$ # or ./runner '-v -ss'
Start tests
============================= test session starts ==============================
platform linux2 -- Python 2.7.15rc1, pytest-4.0.0, py-1.7.0, pluggy-0.8.0
rootdir: /ClickHouse/tests/integration, inifile: pytest.ini
collected 6 items

test_odbc_interaction/test.py ......                                     [100%]
==================== 6 passed, 1 warnings in 96.33 seconds =====================

You can just open shell inside a container by overwritting the command: ./runner --command=bash

Rebuilding the docker containers

The main container used for integration tests lives in docker/test/integration/Dockerfile. Rebuild it with

cd docker/test/integration
docker build -t clickhouse/integration-test .

The helper container used by the runner script is in docker/test/integration/runner/Dockerfile.

Adding new tests

To add new test named foo, create a directory test_foo with an empty __init__.py and a file named test.py containing tests in it. All functions with names starting with test will become test cases.

helpers directory contains utilities for:

  • Launching a ClickHouse cluster with or without ZooKeeper in docker containers.
  • Sending queries to launched instances.
  • Introducing network failures such as severing network link between two instances.

To assert that two TSV files must be equal, wrap them in the TSV class and use the regular assert statement. Example: assert TSV(result) == TSV(reference). In case the assertion fails, pytest will automagically detect the types of variables and only the small diff of two files is printed.

Troubleshooting

If tests failing for misterious reasons, this may help:

sudo service docker stop
sudo bash -c 'rm -rf /var/lib/docker/*'
sudo service docker start

iptables-nft

On Ubuntu 20.10 and later in host network mode (default) one may encounter problem with nested containers not seeing each other. It happens because legacy and nftables rules are out of sync. Problem can be solved by:

sudo iptables -P FORWARD ACCEPT