ClickHouse/tests/integration
alexey-milovidov c4861155f7
Merge pull request #14498 from ClickHouse/add_retries_to_host_ip_change
Add retries to integration test_host_ip_change
2020-09-05 07:03:34 +03:00
..
helpers Merge branch 'master' into integration-explicit-configs 2020-09-03 20:43:01 +03:00
test_access_control_on_cluster Add explicit main_configs, user_configs and dictionaries in integration tests. 2020-08-12 11:55:04 +03:00
test_adaptive_granularity Split tag part from image 2020-09-02 17:36:25 +03:00
test_adaptive_granularity_different_settings Split tag part from image 2020-09-02 17:36:25 +03:00
test_adaptive_granularity_replicated Split tag part from image 2020-09-02 17:36:25 +03:00
test_aggregation_memory_efficient Remove experimental_use_processors setting, part 1. (#10924) 2020-05-15 22:28:10 +03:00
test_allowed_client_hosts Add explicit main_configs, user_configs and dictionaries in integration tests. 2020-08-12 11:55:04 +03:00
test_allowed_url_from_config Add explicit main_configs, user_configs and dictionaries in integration tests. 2020-08-12 11:55:04 +03:00
test_alter_codec fixes 2020-08-18 17:34:04 +03:00
test_always_fetch_merged Fix bad integration test "always_fetch_merged" 2020-06-20 21:21:29 +03:00
test_atomic_drop_table Add explicit main_configs, user_configs and dictionaries in integration tests. 2020-08-12 11:55:04 +03:00
test_attach_without_checksums Fix loading parts without checksums 2020-07-16 13:54:49 +03:00
test_authentication Fix settings NO_PASSWORD authentication mode in users.xml. 2020-05-25 11:35:26 +03:00
test_backup_restore Merge branch 'master' into database_atomic 2020-04-03 03:19:11 +03:00
test_backup_with_other_granularity Split tag part from image 2020-09-02 17:36:25 +03:00
test_backward_compatibility Split tag part from image 2020-09-02 17:36:25 +03:00
test_block_structure_mismatch Move all folders inside /dbms one level up (#9974) 2020-04-02 02:51:21 +03:00
test_check_table Move all folders inside /dbms one level up (#9974) 2020-04-02 02:51:21 +03:00
test_cluster_all_replicas Move all folders inside /dbms one level up (#9974) 2020-04-02 02:51:21 +03:00
test_cluster_copier Add explicit main_configs, user_configs and dictionaries in integration tests. 2020-08-12 11:55:04 +03:00
test_concurrent_queries_for_user_restriction Move all folders inside /dbms one level up (#9974) 2020-04-02 02:51:21 +03:00
test_concurrent_ttl_merges Better names 2020-09-03 16:29:18 +03:00
test_config_corresponding_root Add explicit main_configs, user_configs and dictionaries in integration tests. 2020-08-12 11:55:04 +03:00
test_config_substitutions Move all folders inside /dbms one level up (#9974) 2020-04-02 02:51:21 +03:00
test_consistant_parts_after_move_partition Move all folders inside /dbms one level up (#9974) 2020-04-02 02:51:21 +03:00
test_consistent_parts_after_clone_replica Move all folders inside /dbms one level up (#9974) 2020-04-02 02:51:21 +03:00
test_create_user_and_login Split test 'test_create_user_and_login' into two. 2020-07-10 17:16:47 +03:00
test_cross_replication Move all folders inside /dbms one level up (#9974) 2020-04-02 02:51:21 +03:00
test_custom_settings Add explicit main_configs, user_configs and dictionaries in integration tests. 2020-08-12 11:55:04 +03:00
test_ddl_alter_query Add test for alter on cluster 2020-04-21 16:47:00 +03:00
test_default_compression_codec Merge branch 'master' into try-dns-fix 2020-09-02 19:01:30 +03:00
test_default_role Fix crash when SET DEFAULT ROLE is called with wrong arguments. 2020-05-29 09:54:27 +03:00
test_delayed_replica_failover Update test. 2020-08-03 20:07:09 +03:00
test_dictionaries_access Get dictionary and check access rights only once per each call of any function reading external dictionaries. 2020-05-15 03:14:14 +03:00
test_dictionaries_all_layouts_and_sources Merge remote-tracking branch 'origin' into integration-explicit-configs 2020-09-02 12:48:07 +03:00
test_dictionaries_complex_key_cache_string Add explicit main_configs, user_configs and dictionaries in integration tests. 2020-08-12 11:55:04 +03:00
test_dictionaries_ddl fix test_dictionaries_ddl 2020-09-03 12:55:07 +03:00
test_dictionaries_dependency Don't split dictionary source's table name into schema and table name itself 2020-07-06 23:41:39 +03:00
test_dictionaries_dependency_xml Add explicit main_configs, user_configs and dictionaries in integration tests. 2020-08-12 11:55:04 +03:00
test_dictionaries_mysql Add explicit main_configs, user_configs and dictionaries in integration tests. 2020-08-12 11:55:04 +03:00
test_dictionaries_null_value Add explicit main_configs, user_configs and dictionaries in integration tests. 2020-08-12 11:55:04 +03:00
test_dictionaries_select_all Add explicit main_configs, user_configs and dictionaries in integration tests. 2020-08-12 11:55:04 +03:00
test_dictionaries_update_and_reload Add explicit main_configs, user_configs and dictionaries in integration tests. 2020-08-12 11:55:04 +03:00
test_dictionary_allow_read_expired_keys Add explicit main_configs, user_configs and dictionaries in integration tests. 2020-08-12 11:55:04 +03:00
test_dictionary_custom_settings Add explicit main_configs, user_configs and dictionaries in integration tests. 2020-08-12 11:55:04 +03:00
test_dictionary_ddl_on_cluster Add explicit main_configs, user_configs and dictionaries in integration tests. 2020-08-12 11:55:04 +03:00
test_disk_access_storage Improve syntax of CREATE QUOTA. Now resource types and key types could be written with underscores. 2020-06-15 20:10:34 +03:00
test_disk_types Add explicit main_configs, user_configs and dictionaries in integration tests. 2020-08-12 11:55:04 +03:00
test_distributed_backward_compatability Split tag part from image 2020-09-02 17:36:25 +03:00
test_distributed_ddl fix typo 2020-08-19 10:45:16 +03:00
test_distributed_ddl_on_cross_replication build and test fixes 2020-06-16 21:07:41 +03:00
test_distributed_ddl_password Add explicit main_configs, user_configs and dictionaries in integration tests. 2020-08-12 11:55:04 +03:00
test_distributed_format Add explicit main_configs, user_configs and dictionaries in integration tests. 2020-08-12 11:55:04 +03:00
test_distributed_load_balancing Add a test with negative priority in test_distributed_load_balancing 2020-06-30 11:10:30 +03:00
test_distributed_over_distributed Integration test for Distributed over Distributed (from #8640) 2020-04-04 13:47:34 +03:00
test_distributed_over_live_view Fix bad test for LIVE VIEW 2020-06-22 15:49:38 +03:00
test_distributed_respect_user_timeouts Add explicit main_configs, user_configs and dictionaries in integration tests. 2020-08-12 11:55:04 +03:00
test_distributed_storage_configuration Add explicit main_configs, user_configs and dictionaries in integration tests. 2020-08-12 11:55:04 +03:00
test_distributed_system_query Move all folders inside /dbms one level up (#9974) 2020-04-02 02:51:21 +03:00
test_drop_replica minor improvements 2020-06-23 22:15:01 +03:00
test_enabling_access_management Add explicit main_configs, user_configs and dictionaries in integration tests. 2020-08-12 11:55:04 +03:00
test_extreme_deduplication Add explicit main_configs, user_configs and dictionaries in integration tests. 2020-08-12 11:55:04 +03:00
test_fetch_partition_from_auxiliary_zookeeper Update test.py 2020-08-28 16:03:42 +03:00
test_filesystem_layout Merge branch 'master' into database_atomic 2020-04-03 03:19:11 +03:00
test_force_deduplication fix test 2020-04-09 21:02:27 +03:00
test_format_avro_confluent Move all folders inside /dbms one level up (#9974) 2020-04-02 02:51:21 +03:00
test_format_schema_on_server Move all folders inside /dbms one level up (#9974) 2020-04-02 02:51:21 +03:00
test_freeze_table Use absolute paths to backup locations (shadow) 2020-07-30 18:03:26 +01:00
test_globs_in_filepath Fix behaviour of globs in filepath with leading zeros 2020-04-07 16:46:12 +03:00
test_grant_and_revoke Add test. 2020-08-22 01:59:52 +03:00
test_graphite_merge_tree Move all folders inside /dbms one level up (#9974) 2020-04-02 02:51:21 +03:00
test_host_ip_change Add retries 2020-09-04 18:22:43 +03:00
test_http_and_readonly Move all folders inside /dbms one level up (#9974) 2020-04-02 02:51:21 +03:00
test_http_handlers_config Expect empty content on verbose response 2020-08-18 14:59:57 +03:00
test_https_replication Add explicit main_configs, user_configs and dictionaries in integration tests. 2020-08-12 11:55:04 +03:00
test_inherit_multiple_profiles Move all folders inside /dbms one level up (#9974) 2020-04-02 02:51:21 +03:00
test_input_format_parallel_parsing_memory_tracking Add integration test for input_format_parallel_parsing memory tracking 2020-07-23 11:31:13 +03:00
test_insert_distributed_load_balancing Cover load_balancing for Distributed INSERT w/o internal_replication 2020-06-08 21:58:56 +03:00
test_insert_into_distributed Fix configs 2020-08-03 14:13:43 +03:00
test_insert_into_distributed_sync_async Update integration tests 2020-07-16 04:48:50 +03:00
test_insert_into_distributed_through_materialized_view Fix configs 2020-08-03 14:13:43 +03:00
test_live_view_over_distributed Make LIVE VIEW test less wrong (it was depending on the timing of progress messages) 2020-06-21 23:37:55 +03:00
test_log_family_s3 Add explicit main_configs, user_configs and dictionaries in integration tests. 2020-08-12 11:55:04 +03:00
test_logs_level Move all folders inside /dbms one level up (#9974) 2020-04-02 02:51:21 +03:00
test_match_process_uid_against_data_owner Move all folders inside /dbms one level up (#9974) 2020-04-02 02:51:21 +03:00
test_materialize_mysql_database mysql fix 2020-09-03 13:27:44 +03:00
test_max_http_connections_for_replication Add explicit main_configs, user_configs and dictionaries in integration tests. 2020-08-12 11:55:04 +03:00
test_merge_table_over_distributed Move all folders inside /dbms one level up (#9974) 2020-04-02 02:51:21 +03:00
test_merge_tree_s3 Merge remote-tracking branch 'origin' into integration-explicit-configs 2020-09-02 12:48:07 +03:00
test_merge_tree_s3_with_cache Add explicit main_configs, user_configs and dictionaries in integration tests. 2020-08-12 11:55:04 +03:00
test_multiple_disks Revert "Added allow_merges option for volumes in multi-disk configuration (#13402)" 2020-08-21 18:44:29 +03:00
test_mutations_hardlinks Add test for sticking mutations 2020-07-26 18:58:03 +03:00
test_mutations_with_merge_tree Move all folders inside /dbms one level up (#9974) 2020-04-02 02:51:21 +03:00
test_mysql_database_engine Add explicit main_configs, user_configs and dictionaries in integration tests. 2020-08-12 11:55:04 +03:00
test_mysql_protocol Merge pull request #13647 from qoega/integration-explicit-configs 2020-09-04 20:16:00 +03:00
test_no_local_metadata_node Fix replicated tables startup when updating from old version 2020-04-07 18:28:29 +03:00
test_non_default_compression Fix integration test 2020-05-04 17:51:29 +03:00
test_odbc_interaction Split tag part from image 2020-09-02 17:36:25 +03:00
test_old_versions Split tag part from image 2020-09-02 17:36:25 +03:00
test_on_cluster_timeouts Fix DDL worker 2020-08-07 12:18:34 +03:00
test_part_log_table Fix part_log test 2020-06-01 12:12:52 +03:00
test_partition Better interface 2020-08-28 12:07:20 +03:00
test_parts_delete_zookeeper Move all folders inside /dbms one level up (#9974) 2020-04-02 02:51:21 +03:00
test_polymorphic_parts Merge remote-tracking branch 'origin' into integration-explicit-configs 2020-09-03 11:49:48 +03:00
test_postgresql_protocol mysql/postgresql: move Dockerfiles and docker_compose to docker/test (#13448) 2020-08-12 18:04:36 +03:00
test_profile_events_s3 Add explicit main_configs, user_configs and dictionaries in integration tests. 2020-08-12 11:55:04 +03:00
test_prometheus_endpoint Move all folders inside /dbms one level up (#9974) 2020-04-02 02:51:21 +03:00
test_quorum_inserts Add explicit main_configs, user_configs and dictionaries in integration tests. 2020-08-12 11:55:04 +03:00
test_quota Add explicit main_configs, user_configs and dictionaries in integration tests. 2020-08-12 11:55:04 +03:00
test_random_inserts Add explicit main_configs, user_configs and dictionaries in integration tests. 2020-08-12 11:55:04 +03:00
test_range_hashed_dictionary_types Fix columns duplication for range hashed dictionary created from DDL query 2020-07-27 17:06:44 +03:00
test_read_temporary_tables_on_failure Move all folders inside /dbms one level up (#9974) 2020-04-02 02:51:21 +03:00
test_recovery_replica Move all folders inside /dbms one level up (#9974) 2020-04-02 02:51:21 +03:00
test_redirect_url_storage Move all folders inside /dbms one level up (#9974) 2020-04-02 02:51:21 +03:00
test_relative_filepath Move all folders inside /dbms one level up (#9974) 2020-04-02 02:51:21 +03:00
test_reload_max_table_size_to_drop Add explicit main_configs, user_configs and dictionaries in integration tests. 2020-08-12 11:55:04 +03:00
test_reloading_storage_configuration finally remove config_dir 2020-08-14 18:51:28 +03:00
test_remote_prewhere Move all folders inside /dbms one level up (#9974) 2020-04-02 02:51:21 +03:00
test_rename_column Add explicit main_configs, user_configs and dictionaries in integration tests. 2020-08-12 11:55:04 +03:00
test_replace_partition Move all folders inside /dbms one level up (#9974) 2020-04-02 02:51:21 +03:00
test_replica_can_become_leader Move all folders inside /dbms one level up (#9974) 2020-04-02 02:51:21 +03:00
test_replicated_merge_tree_s3 Merge remote-tracking branch 'origin' into integration-explicit-configs 2020-09-02 12:48:07 +03:00
test_replicated_mutations Move all folders inside /dbms one level up (#9974) 2020-04-02 02:51:21 +03:00
test_replicated_parse_zk_metadata Cover re-parsing of metadata from zookeeper 2020-06-17 21:05:26 +03:00
test_replicating_constants Split tag part from image 2020-09-02 17:36:25 +03:00
test_replication_credentials Move all folders inside /dbms one level up (#9974) 2020-04-02 02:51:21 +03:00
test_replication_without_zookeeper Move all folders inside /dbms one level up (#9974) 2020-04-02 02:51:21 +03:00
test_role Rename storages users.xml=>users_xml, disk=>local_directory. 2020-08-16 16:42:57 +03:00
test_row_policy Add explicit main_configs, user_configs and dictionaries in integration tests. 2020-08-12 11:55:04 +03:00
test_s3_with_https Add explicit main_configs, user_configs and dictionaries in integration tests. 2020-08-12 11:55:04 +03:00
test_s3_with_proxy Add explicit main_configs, user_configs and dictionaries in integration tests. 2020-08-12 11:55:04 +03:00
test_send_crash_reports rename test back to test.py 2020-06-25 11:11:29 +03:00
test_send_request_to_leader_replica Move all folders inside /dbms one level up (#9974) 2020-04-02 02:51:21 +03:00
test_server_initialization Fix test_server_initialization, but looking into default database only 2020-05-19 10:43:37 +03:00
test_settings_constraints Add explicit main_configs, user_configs and dictionaries in integration tests. 2020-08-12 11:55:04 +03:00
test_settings_constraints_distributed Add explicit main_configs, user_configs and dictionaries in integration tests. 2020-08-12 11:55:04 +03:00
test_settings_profile Rename storages users.xml=>users_xml, disk=>local_directory. 2020-08-16 16:42:57 +03:00
test_storage_hdfs Add explicit main_configs, user_configs and dictionaries in integration tests. 2020-08-12 11:55:04 +03:00
test_storage_kafka Merge remote-tracking branch 'origin' into integration-explicit-configs 2020-09-02 12:48:07 +03:00
test_storage_mongodb Add __init__.py 2020-06-26 19:31:00 +03:00
test_storage_mysql Fix integration test 2020-06-28 23:03:45 +03:00
test_storage_rabbitmq Merge remote-tracking branch 'origin' into integration-explicit-configs 2020-09-02 12:48:07 +03:00
test_storage_s3 S3 HTTPS integration test. 2020-07-10 22:42:18 +03:00
test_SYSTEM_FLUSH_LOGS Force table creation on SYSTEM FLUSH LOGS (v2) 2020-06-11 23:12:23 +03:00
test_system_merges finally remove config_dir 2020-08-14 18:51:28 +03:00
test_system_queries Add explicit main_configs, user_configs and dictionaries in integration tests. 2020-08-12 11:55:04 +03:00
test_text_log_level Add explicit main_configs, user_configs and dictionaries in integration tests. 2020-08-12 11:55:04 +03:00
test_timezone_config Move all folders inside /dbms one level up (#9974) 2020-04-02 02:51:21 +03:00
test_tmp_policy Add explicit main_configs, user_configs and dictionaries in integration tests. 2020-08-12 11:55:04 +03:00
test_ttl_move Add explicit main_configs, user_configs and dictionaries in integration tests. 2020-08-12 11:55:04 +03:00
test_ttl_replicated Merging #12548 - Correction to merge_with_ttl_timeout logic by @excitoon (#12982) 2020-07-28 17:38:34 +03:00
test_union_header Move all folders inside /dbms one level up (#9974) 2020-04-02 02:51:21 +03:00
test_user_directories Add storages from <user_directories> after ones from <users_config> and <access_control_path>. 2020-08-26 13:45:35 +03:00
test_user_ip_restrictions Add explicit main_configs, user_configs and dictionaries in integration tests. 2020-08-12 11:55:04 +03:00
test_user_zero_database_access Add explicit main_configs, user_configs and dictionaries in integration tests. 2020-08-12 11:55:04 +03:00
test_version_update_after_mutation Split tag part from image 2020-09-02 17:36:25 +03:00
test_zookeeper_config Add explicit main_configs, user_configs and dictionaries in integration tests. 2020-08-12 11:55:04 +03:00
.dockerignore Move all folders inside /dbms one level up (#9974) 2020-04-02 02:51:21 +03:00
.gitignore Move all folders inside /dbms one level up (#9974) 2020-04-02 02:51:21 +03:00
CMakeLists.txt Add explicit main_configs, user_configs and dictionaries in integration tests. 2020-08-12 11:55:04 +03:00
conftest.py Move all folders inside /dbms one level up (#9974) 2020-04-02 02:51:21 +03:00
pytest.ini improvements 2020-04-17 11:42:51 +03:00
README.md Place common docker compose files to integration docker container 2020-07-06 16:45:54 +03:00
runner Merge branch 'master' into try-dns-fix 2020-09-02 19:01:30 +03:00

ClickHouse integration tests

This directory contains tests that involve several ClickHouse instances, custom configs, ZooKeeper, etc.

Running natively

Prerequisites:

  • Ubuntu 14.04 (Trusty) or higher.
  • docker. Minimum required API version: 1.25, check with docker version.

You must install latest Docker from https://docs.docker.com/engine/installation/linux/docker-ce/ubuntu/#set-up-the-repository Don't use Docker from your system repository.

  • pip and libpq-dev. To install: sudo apt-get install python-pip libpq-dev zlib1g-dev libcrypto++-dev libssl-dev
  • py.test testing framework. To install: sudo -H pip install pytest
  • docker-compose and additional python libraries. To install: sudo -H pip install urllib3==1.23 pytest docker-compose==1.22.0 docker dicttoxml kazoo PyMySQL psycopg2==2.7.5 pymongo tzlocal kafka-python protobuf redis aerospike pytest-timeout minio rpm-confluent-schemaregistry

(highly not recommended) If you really want to use OS packages on modern debian/ubuntu instead of "pip": sudo apt install -y docker docker-compose python-pytest python-dicttoxml python-docker python-pymysql python-pymongo python-tzlocal python-kazoo python-psycopg2 python-kafka python-pytest-timeout python-minio

If you want to run the tests under a non-privileged user, you must add this user to docker group: sudo usermod -aG docker $USER and re-login. (You must close all your sessions (for example, restart your computer)) To check, that you have access to Docker, run docker ps.

Run the tests with the pytest command. To select which tests to run, use: pytest -k <test_name_pattern>

By default tests are run with system-wide client binary, server binary and base configs. To change that, set the following environment variables:`

  • CLICKHOUSE_TESTS_SERVER_BIN_PATH to choose the server binary.
  • CLICKHOUSE_TESTS_CLIENT_BIN_PATH to choose the client binary.
  • CLICKHOUSE_TESTS_BASE_CONFIG_DIR to choose the directory from which base configs (config.xml and users.xml) are taken.

For tests that use common docker compose files you may need to set up their path with environment variable: DOCKER_COMPOSE_DIR=$HOME/ClickHouse/docker/test/integration/runner/compose

Running with runner script

The only requirement is fresh configured docker and docker pull yandex/clickhouse-integration-tests-runner

Notes:

  • If you want to run integration tests without sudo you have to add your user to docker group sudo usermod -aG docker $USER. More information about docker configuration.
  • If you already had run these tests without ./runner script you may have problems with pytest cache. It can be removed with rm -r __pycache__ .pytest_cache/.
  • Some tests maybe require a lot of resources (CPU, RAM, etc.). Better not try large tests like test_cluster_copier or test_distributed_ddl* on your laptop.

You can run tests via ./runner script and pass pytest arguments as last arg:

$ ./runner --binary $HOME/ClickHouse/programs/clickhouse  --bridge-binary $HOME/ClickHouse/programs/clickhouse-odbc-bridge --configs-dir $HOME/ClickHouse/programs/server/ 'test_odbc_interaction -ss'
Start tests
============================= test session starts ==============================
platform linux2 -- Python 2.7.15rc1, pytest-4.0.0, py-1.7.0, pluggy-0.8.0
rootdir: /ClickHouse/tests/integration, inifile: pytest.ini
collected 6 items

test_odbc_interaction/test.py Removing network clickhouse_default
...

Killing roottestodbcinteraction_node1_1     ... done
Killing roottestodbcinteraction_mysql1_1    ... done
Killing roottestodbcinteraction_postgres1_1 ... done
Removing roottestodbcinteraction_node1_1     ... done
Removing roottestodbcinteraction_mysql1_1    ... done
Removing roottestodbcinteraction_postgres1_1 ... done
Removing network roottestodbcinteraction_default

==================== 6 passed, 1 warnings in 95.21 seconds =====================

Path to binary and configs maybe specified via env variables:

$ export CLICKHOUSE_TESTS_BASE_CONFIG_DIR=$HOME/ClickHouse/programs/server/
$ export CLICKHOUSE_TESTS_SERVER_BIN_PATH=$HOME/ClickHouse/programs/clickhouse
$ export CLICKHOUSE_TESTS_ODBC_BRIDGE_BIN_PATH=$HOME/ClickHouse/programs/clickhouse-odbc-bridge
$ ./runner 'test_odbc_interaction'
$ # or ./runner '-v -ss'
Start tests
============================= test session starts ==============================
platform linux2 -- Python 2.7.15rc1, pytest-4.0.0, py-1.7.0, pluggy-0.8.0
rootdir: /ClickHouse/tests/integration, inifile: pytest.ini
collected 6 items

test_odbc_interaction/test.py ......                                     [100%]
==================== 6 passed, 1 warnings in 96.33 seconds =====================

You can just open shell inside a container by overwritting the command: ./runner --command=bash

Rebuilding the docker containers

The main container used for integration tests lives in docker/test/integration/Dockerfile. Rebuild it with

cd docker/test/integration
docker build -t yandex/clickhouse-integration-test .

The helper container used by the runner script is in docker/test/integration/runner/Dockerfile.

Adding new tests

To add new test named foo, create a directory test_foo with an empty __init__.py and a file named test.py containing tests in it. All functions with names starting with test will become test cases.

helpers directory contains utilities for:

  • Launching a ClickHouse cluster with or without ZooKeeper in docker containers.
  • Sending queries to launched instances.
  • Introducing network failures such as severing network link between two instances.

To assert that two TSV files must be equal, wrap them in the TSV class and use the regular assert statement. Example: assert TSV(result) == TSV(reference). In case the assertion fails, pytest will automagically detect the types of variables and only the small diff of two files is printed.