ClickHouse/tests/integration
BohuTANG f48fdda678
Enhanced compatibility with native mysql-connector-java(JDBC) (#10021)
* Skip the `/* comments */ SELECT @@variables ...` from mysql-connector-java setup for MySQL Handler #9336

mysql-connector setup query:
/* mysql-connector-java-5.1.38 ( Revision: ${revinfo.commit} ) */SELECT  @@session.auto_increment_increment AS auto_increment_increment, @@character_set_client AS character_set_client, @@character_set_connection AS character_set_connection, @@character_set_results AS character_set_results, @@character_set_server AS character_set_server, @@init_connect AS init_connect, @@interactive_timeout AS interactive_timeout...

ClickHouse side Error:
{} <Error> executeQuery: Code: 62, e.displayText() = DB::Exception: Syntax error: failed at position 74: @@session.auto_increment_increment AS auto_increment_increment, @@character_set_client AS character_set_client, @@character_set_connection AS character_set_conn. Expected one of: CAST, NULL...

Client side Exception:
java.sql.SQLException: Syntax error: failed at position 74: @@session.auto_increment_increment AS auto_increment_increment, @@character_set_client AS character_set_client, @@character_set_connection AS character_set_conn. Expected one of: CAST...

* add repalce 'SHOW VARIABLES' for mysql-connector-java-5.1.34 #9336

* Add java client(JDBC) integration test to test_mysql_protocol

* shift out java tests from dbms

* Update MySQLHandler.cpp

* Update MySQLHandler.cpp

* test_mysql_protocol: add Test.java exit code 1 when expection

Co-authored-by: alexey-milovidov <milovidov@yandex-team.ru>
2020-04-09 00:52:19 +03:00
..
helpers Move all folders inside /dbms one level up (#9974) 2020-04-02 02:51:21 +03:00
image Move all folders inside /dbms one level up (#9974) 2020-04-02 02:51:21 +03:00
test_access_control_on_cluster Implement "ON CLUSTER" clause for access control SQL. 2020-04-07 23:31:59 +03:00
test_adaptive_granularity Move all folders inside /dbms one level up (#9974) 2020-04-02 02:51:21 +03:00
test_adaptive_granularity_different_settings Fix adaptive granularity compatibility 2020-04-08 12:51:04 +03:00
test_adaptive_granularity_replicated Move all folders inside /dbms one level up (#9974) 2020-04-02 02:51:21 +03:00
test_aggregation_memory_efficient Move all folders inside /dbms one level up (#9974) 2020-04-02 02:51:21 +03:00
test_allowed_client_hosts Move all folders inside /dbms one level up (#9974) 2020-04-02 02:51:21 +03:00
test_allowed_url_from_config Move all folders inside /dbms one level up (#9974) 2020-04-02 02:51:21 +03:00
test_atomic_drop_table Move all folders inside /dbms one level up (#9974) 2020-04-02 02:51:21 +03:00
test_authentication Add the "access_management" option to users.xml which enables a user to manage access rights. 2020-04-07 23:31:54 +03:00
test_backup_restore Move all folders inside /dbms one level up (#9974) 2020-04-02 02:51:21 +03:00
test_backward_compatability Move all folders inside /dbms one level up (#9974) 2020-04-02 02:51:21 +03:00
test_block_structure_mismatch Move all folders inside /dbms one level up (#9974) 2020-04-02 02:51:21 +03:00
test_check_table Move all folders inside /dbms one level up (#9974) 2020-04-02 02:51:21 +03:00
test_cluster_all_replicas Move all folders inside /dbms one level up (#9974) 2020-04-02 02:51:21 +03:00
test_cluster_copier Move all folders inside /dbms one level up (#9974) 2020-04-02 02:51:21 +03:00
test_concurrent_queries_for_user_restriction Move all folders inside /dbms one level up (#9974) 2020-04-02 02:51:21 +03:00
test_config_corresponding_root Move all folders inside /dbms one level up (#9974) 2020-04-02 02:51:21 +03:00
test_config_substitutions Move all folders inside /dbms one level up (#9974) 2020-04-02 02:51:21 +03:00
test_consistant_parts_after_move_partition Move all folders inside /dbms one level up (#9974) 2020-04-02 02:51:21 +03:00
test_consistent_parts_after_clone_replica Move all folders inside /dbms one level up (#9974) 2020-04-02 02:51:21 +03:00
test_cross_replication Move all folders inside /dbms one level up (#9974) 2020-04-02 02:51:21 +03:00
test_delayed_replica_failover Move all folders inside /dbms one level up (#9974) 2020-04-02 02:51:21 +03:00
test_dictionaries_all_layouts_and_sources Move all folders inside /dbms one level up (#9974) 2020-04-02 02:51:21 +03:00
test_dictionaries_complex_key_cache_string Move all folders inside /dbms one level up (#9974) 2020-04-02 02:51:21 +03:00
test_dictionaries_ddl Move all folders inside /dbms one level up (#9974) 2020-04-02 02:51:21 +03:00
test_dictionaries_depend_on_dictionaries Move all folders inside /dbms one level up (#9974) 2020-04-02 02:51:21 +03:00
test_dictionaries_mysql Move all folders inside /dbms one level up (#9974) 2020-04-02 02:51:21 +03:00
test_dictionaries_null_value Move all folders inside /dbms one level up (#9974) 2020-04-02 02:51:21 +03:00
test_dictionaries_select_all Move all folders inside /dbms one level up (#9974) 2020-04-02 02:51:21 +03:00
test_dictionaries_update_and_reload Move all folders inside /dbms one level up (#9974) 2020-04-02 02:51:21 +03:00
test_dictionary_allow_read_expired_keys Move all folders inside /dbms one level up (#9974) 2020-04-02 02:51:21 +03:00
test_dictionary_ddl_on_cluster Move all folders inside /dbms one level up (#9974) 2020-04-02 02:51:21 +03:00
test_disk_access_storage Add the option "access_control_path" to server config. 2020-04-07 23:31:59 +03:00
test_distributed_ddl Move all folders inside /dbms one level up (#9974) 2020-04-02 02:51:21 +03:00
test_distributed_ddl_password Move all folders inside /dbms one level up (#9974) 2020-04-02 02:51:21 +03:00
test_distributed_format Move all folders inside /dbms one level up (#9974) 2020-04-02 02:51:21 +03:00
test_distributed_over_distributed Integration test for Distributed over Distributed (from #8640) 2020-04-04 13:47:34 +03:00
test_distributed_respect_user_timeouts Move all folders inside /dbms one level up (#9974) 2020-04-02 02:51:21 +03:00
test_distributed_storage_configuration Move all folders inside /dbms one level up (#9974) 2020-04-02 02:51:21 +03:00
test_distributed_system_query Move all folders inside /dbms one level up (#9974) 2020-04-02 02:51:21 +03:00
test_extreme_deduplication Move all folders inside /dbms one level up (#9974) 2020-04-02 02:51:21 +03:00
test_filesystem_layout Move all folders inside /dbms one level up (#9974) 2020-04-02 02:51:21 +03:00
test_force_deduplication Move all folders inside /dbms one level up (#9974) 2020-04-02 02:51:21 +03:00
test_format_avro_confluent Move all folders inside /dbms one level up (#9974) 2020-04-02 02:51:21 +03:00
test_format_schema_on_server Move all folders inside /dbms one level up (#9974) 2020-04-02 02:51:21 +03:00
test_globs_in_filepath Fix behaviour of globs in filepath with leading zeros 2020-04-07 16:46:12 +03:00
test_grant_and_revoke Add the "access_management" option to users.xml which enables a user to manage access rights. 2020-04-07 23:31:54 +03:00
test_graphite_merge_tree Move all folders inside /dbms one level up (#9974) 2020-04-02 02:51:21 +03:00
test_host_ip_change Move all folders inside /dbms one level up (#9974) 2020-04-02 02:51:21 +03:00
test_http_and_readonly Move all folders inside /dbms one level up (#9974) 2020-04-02 02:51:21 +03:00
test_https_replication Move all folders inside /dbms one level up (#9974) 2020-04-02 02:51:21 +03:00
test_inherit_multiple_profiles Move all folders inside /dbms one level up (#9974) 2020-04-02 02:51:21 +03:00
test_insert_into_distributed Move all folders inside /dbms one level up (#9974) 2020-04-02 02:51:21 +03:00
test_insert_into_distributed_sync_async Move all folders inside /dbms one level up (#9974) 2020-04-02 02:51:21 +03:00
test_insert_into_distributed_through_materialized_view Move all folders inside /dbms one level up (#9974) 2020-04-02 02:51:21 +03:00
test_log_family_s3 Move all folders inside /dbms one level up (#9974) 2020-04-02 02:51:21 +03:00
test_logs_level Move all folders inside /dbms one level up (#9974) 2020-04-02 02:51:21 +03:00
test_match_process_uid_against_data_owner Move all folders inside /dbms one level up (#9974) 2020-04-02 02:51:21 +03:00
test_max_http_connections_for_replication Move all folders inside /dbms one level up (#9974) 2020-04-02 02:51:21 +03:00
test_merge_table_over_distributed Move all folders inside /dbms one level up (#9974) 2020-04-02 02:51:21 +03:00
test_merge_tree_s3 Move all folders inside /dbms one level up (#9974) 2020-04-02 02:51:21 +03:00
test_multiple_disks Move all folders inside /dbms one level up (#9974) 2020-04-02 02:51:21 +03:00
test_mutations_with_merge_tree Move all folders inside /dbms one level up (#9974) 2020-04-02 02:51:21 +03:00
test_mysql_database_engine ISSUES-10056 reused evaluateConstantExpressionOrIdentifierAsLiteral 2020-04-09 01:31:28 +08:00
test_mysql_protocol Enhanced compatibility with native mysql-connector-java(JDBC) (#10021) 2020-04-09 00:52:19 +03:00
test_no_local_metadata_node Fix replicated tables startup when updating from old version 2020-04-07 18:28:29 +03:00
test_non_default_compression Move all folders inside /dbms one level up (#9974) 2020-04-02 02:51:21 +03:00
test_odbc_interaction Fix integration test 2020-04-02 11:51:21 +03:00
test_old_versions Move all folders inside /dbms one level up (#9974) 2020-04-02 02:51:21 +03:00
test_part_log_table Move all folders inside /dbms one level up (#9974) 2020-04-02 02:51:21 +03:00
test_partition Move all folders inside /dbms one level up (#9974) 2020-04-02 02:51:21 +03:00
test_parts_delete_zookeeper Move all folders inside /dbms one level up (#9974) 2020-04-02 02:51:21 +03:00
test_polymorphic_parts Move all folders inside /dbms one level up (#9974) 2020-04-02 02:51:21 +03:00
test_prometheus_endpoint Move all folders inside /dbms one level up (#9974) 2020-04-02 02:51:21 +03:00
test_quota Add the "access_management" option to users.xml which enables a user to manage access rights. 2020-04-07 23:31:54 +03:00
test_random_inserts Move more dbms/tests to top-level 2020-04-02 03:32:39 +03:00
test_read_temporary_tables_on_failure Move all folders inside /dbms one level up (#9974) 2020-04-02 02:51:21 +03:00
test_recovery_replica Move all folders inside /dbms one level up (#9974) 2020-04-02 02:51:21 +03:00
test_redirect_url_storage Move all folders inside /dbms one level up (#9974) 2020-04-02 02:51:21 +03:00
test_relative_filepath Move all folders inside /dbms one level up (#9974) 2020-04-02 02:51:21 +03:00
test_reload_max_table_size_to_drop Move all folders inside /dbms one level up (#9974) 2020-04-02 02:51:21 +03:00
test_reloading_storage_configuration Move all folders inside /dbms one level up (#9974) 2020-04-02 02:51:21 +03:00
test_remote_prewhere Move all folders inside /dbms one level up (#9974) 2020-04-02 02:51:21 +03:00
test_replace_partition Move all folders inside /dbms one level up (#9974) 2020-04-02 02:51:21 +03:00
test_replica_can_become_leader Move all folders inside /dbms one level up (#9974) 2020-04-02 02:51:21 +03:00
test_replicated_mutations Move all folders inside /dbms one level up (#9974) 2020-04-02 02:51:21 +03:00
test_replicating_constants Move all folders inside /dbms one level up (#9974) 2020-04-02 02:51:21 +03:00
test_replication_credentials Move all folders inside /dbms one level up (#9974) 2020-04-02 02:51:21 +03:00
test_replication_without_zookeeper Move all folders inside /dbms one level up (#9974) 2020-04-02 02:51:21 +03:00
test_row_policy Add the "access_management" option to users.xml which enables a user to manage access rights. 2020-04-07 23:31:54 +03:00
test_send_request_to_leader_replica Move all folders inside /dbms one level up (#9974) 2020-04-02 02:51:21 +03:00
test_server_initialization Move all folders inside /dbms one level up (#9974) 2020-04-02 02:51:21 +03:00
test_settings_constraints Move all folders inside /dbms one level up (#9974) 2020-04-02 02:51:21 +03:00
test_settings_constraints_distributed Add the "access_management" option to users.xml which enables a user to manage access rights. 2020-04-07 23:31:54 +03:00
test_settings_profile Add the "access_management" option to users.xml which enables a user to manage access rights. 2020-04-07 23:31:54 +03:00
test_storage_hdfs Move all folders inside /dbms one level up (#9974) 2020-04-02 02:51:21 +03:00
test_storage_kafka Move all folders inside /dbms one level up (#9974) 2020-04-02 02:51:21 +03:00
test_storage_mysql Move all folders inside /dbms one level up (#9974) 2020-04-02 02:51:21 +03:00
test_storage_s3 Move all folders inside /dbms one level up (#9974) 2020-04-02 02:51:21 +03:00
test_system_merges Move all folders inside /dbms one level up (#9974) 2020-04-02 02:51:21 +03:00
test_system_queries Move all folders inside /dbms one level up (#9974) 2020-04-02 02:51:21 +03:00
test_text_log_level Move all folders inside /dbms one level up (#9974) 2020-04-02 02:51:21 +03:00
test_timezone_config Move all folders inside /dbms one level up (#9974) 2020-04-02 02:51:21 +03:00
test_tmp_policy Move all folders inside /dbms one level up (#9974) 2020-04-02 02:51:21 +03:00
test_ttl_move Move all folders inside /dbms one level up (#9974) 2020-04-02 02:51:21 +03:00
test_ttl_replicated Move all folders inside /dbms one level up (#9974) 2020-04-02 02:51:21 +03:00
test_union_header Move all folders inside /dbms one level up (#9974) 2020-04-02 02:51:21 +03:00
test_user_ip_restrictions Move all folders inside /dbms one level up (#9974) 2020-04-02 02:51:21 +03:00
test_user_zero_database_access Move all folders inside /dbms one level up (#9974) 2020-04-02 02:51:21 +03:00
test_version_update_after_mutation Move all folders inside /dbms one level up (#9974) 2020-04-02 02:51:21 +03:00
test_zookeeper_config Move all folders inside /dbms one level up (#9974) 2020-04-02 02:51:21 +03:00
.dockerignore Move all folders inside /dbms one level up (#9974) 2020-04-02 02:51:21 +03:00
.gitignore Move all folders inside /dbms one level up (#9974) 2020-04-02 02:51:21 +03:00
CMakeLists.txt Move all folders inside /dbms one level up (#9974) 2020-04-02 02:51:21 +03:00
conftest.py Move all folders inside /dbms one level up (#9974) 2020-04-02 02:51:21 +03:00
pytest.ini Move all folders inside /dbms one level up (#9974) 2020-04-02 02:51:21 +03:00
README.md Move all folders inside /dbms one level up (#9974) 2020-04-02 02:51:21 +03:00
runner Fix runner script 2020-04-06 21:30:51 +03:00

ClickHouse integration tests

This directory contains tests that involve several ClickHouse instances, custom configs, ZooKeeper, etc.

Running natively

Prerequisites:

  • Ubuntu 14.04 (Trusty) or higher.
  • docker. Minimum required API version: 1.25, check with docker version.

You must install latest Docker from https://docs.docker.com/engine/installation/linux/docker-ce/ubuntu/#set-up-the-repository Don't use Docker from your system repository.

  • pip and libpq-dev. To install: sudo apt-get install python-pip libpq-dev
  • py.test testing framework. To install: sudo -H pip install pytest
  • docker-compose and additional python libraries. To install: sudo -H pip install docker-compose docker dicttoxml kazoo PyMySQL psycopg2 pymongo tzlocal kafka-python protobuf pytest-timeout minio rpm-confluent-schemaregistry

(highly not recommended) If you really want to use OS packages on modern debian/ubuntu instead of "pip": sudo apt install -y docker docker-compose python-pytest python-dicttoxml python-docker python-pymysql python-pymongo python-tzlocal python-kazoo python-psycopg2 python-kafka python-pytest-timeout python-minio

If you want to run the tests under a non-privileged user, you must add this user to docker group: sudo usermod -aG docker $USER and re-login. (You must close all your sessions (for example, restart your computer)) To check, that you have access to Docker, run docker ps.

Run the tests with the pytest command. To select which tests to run, use: pytest -k <test_name_pattern>

By default tests are run with system-wide client binary, server binary and base configs. To change that, set the following environment variables:

  • CLICKHOUSE_TESTS_SERVER_BIN_PATH to choose the server binary.
  • CLICKHOUSE_TESTS_CLIENT_BIN_PATH to choose the client binary.
  • CLICKHOUSE_TESTS_BASE_CONFIG_DIR to choose the directory from which base configs (config.xml and users.xml) are taken.

Running with runner script

The only requirement is fresh configured docker and docker pull yandex/clickhouse-integration-tests-runner

Notes:

  • If you want to run integration tests without sudo you have to add your user to docker group sudo usermod -aG docker $USER. More information about docker configuration.
  • If you already had run these tests without ./runner script you may have problems with pytest cache. It can be removed with rm -r __pycache__ .pytest_cache/.
  • Some tests maybe require a lot of resources (CPU, RAM, etc.). Better not try large tests like test_cluster_copier or test_distributed_ddl* on your laptop.

You can run tests via ./runner script and pass pytest arguments as last arg:

$ ./runner --binary $HOME/ClickHouse/programs/clickhouse  --bridge-binary $HOME/ClickHouse/programs/clickhouse-odbc-bridge --configs-dir $HOME/ClickHouse/programs/server/ 'test_odbc_interaction -ss'
Start tests
============================= test session starts ==============================
platform linux2 -- Python 2.7.15rc1, pytest-4.0.0, py-1.7.0, pluggy-0.8.0
rootdir: /ClickHouse/tests/integration, inifile: pytest.ini
collected 6 items

test_odbc_interaction/test.py Removing network clickhouse_default
...

Killing roottestodbcinteraction_node1_1     ... done
Killing roottestodbcinteraction_mysql1_1    ... done
Killing roottestodbcinteraction_postgres1_1 ... done
Removing roottestodbcinteraction_node1_1     ... done
Removing roottestodbcinteraction_mysql1_1    ... done
Removing roottestodbcinteraction_postgres1_1 ... done
Removing network roottestodbcinteraction_default

==================== 6 passed, 1 warnings in 95.21 seconds =====================

Path to binary and configs maybe specified via env variables:

$ export CLICKHOUSE_TESTS_BASE_CONFIG_DIR=$HOME/ClickHouse/programs/server/
$ export CLICKHOUSE_TESTS_SERVER_BIN_PATH=$HOME/ClickHouse/programs/clickhouse
$ export CLICKHOUSE_TESTS_ODBC_BRIDGE_BIN_PATH=$HOME/ClickHouse/programs/clickhouse-odbc-bridge
$ ./runner 'test_odbc_interaction'
$ # or ./runner '-v -ss'
Start tests
============================= test session starts ==============================
platform linux2 -- Python 2.7.15rc1, pytest-4.0.0, py-1.7.0, pluggy-0.8.0
rootdir: /ClickHouse/tests/integration, inifile: pytest.ini
collected 6 items

test_odbc_interaction/test.py ......                                     [100%]
==================== 6 passed, 1 warnings in 96.33 seconds =====================

You can just open shell inside a container by overwritting the command: ./runner --command=bash

Rebuilding the docker containers

The main container used for integration tests lives in docker/test/integration/Dockerfile. Rebuild it with

cd docker/test/integration
docker build -t yandex/clickhouse-integration-test .

The helper container used by the runner script is in tests/integration/image/Dockerfile.

Adding new tests

To add new test named foo, create a directory test_foo with an empty __init__.py and a file named test.py containing tests in it. All functions with names starting with test will become test cases.

helpers directory contains utilities for:

  • Launching a ClickHouse cluster with or without ZooKeeper in docker containers.
  • Sending queries to launched instances.
  • Introducing network failures such as severing network link between two instances.

To assert that two TSV files must be equal, wrap them in the TSV class and use the regular assert statement. Example: assert TSV(result) == TSV(reference). In case the assertion fails, pytest will automagically detect the types of variables and only the small diff of two files is printed.