Commit Graph

670 Commits

Author SHA1 Message Date
Alexander Tokmakov
c7996d5453
increase election timeout in integration tests (#53052) 2023-08-07 14:48:52 +03:00
Nikolai Kochetov
8f5d0c0cd4 Revert "Merge pull request #52514 from azat/tests/fix-test_version_update_after_mutation"
This reverts commit dd491eeae2, reversing
changes made to b225f9c34b.
2023-07-31 13:03:04 +00:00
Sema Checherinda
115a085d0f
Merge branch 'master' into reset-connection-s3 2023-07-28 14:00:14 +04:00
Sema Checherinda
4629ab1df1 add test with broken pipe 2023-07-28 00:24:05 +04:00
Azat Khuzhin
0401dc453e Fix flakiness of test_version_update_after_mutation by enabling force_remove_data_recursively_on_drop
Since there can be some leftovers:

    2023.07.24 07:08:25.238066 [ 140 ] {} <Error> Application: Code: 219. DB::Exception: Cannot drop: filesystem error: in remove: Directory not empty ["/var/lib/clickhouse/data/system/"]. Probably database contain some detached tables or metadata leftovers from Ordinary engine. If you want to remove all data anyway, try to attach database back and drop it again with enabled force_remove_data_recursively_on_drop setting: Exception while trying to convert database system from Ordinary to Atomic. It may be in some intermediate state. You can finish conversion manually by moving the rest tables from system to .tmp_convert.system.9396432095832455195 (using RENAME TABLE) and executing DROP DATABASE system and RENAME DATABASE .tmp_convert.system.9396432095832455195 TO system. (DATABASE_NOT_EMPTY), Stack trace (when copying this message, always include the lines below):

    0. DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000e68af57 in /usr/bin/clickhouse
    1. ? @ 0x000000000cab443c in /usr/bin/clickhouse
    2. DB::DatabaseOnDisk::drop(std::shared_ptr<DB::Context const>) @ 0x000000001328d617 in /usr/bin/clickhouse
    3. DB::DatabaseCatalog::detachDatabase(std::shared_ptr<DB::Context const>, String const&, bool, bool) @ 0x0000000013524a6c in /usr/bin/clickhouse
    4. DB::InterpreterDropQuery::executeToDatabaseImpl(DB::ASTDropQuery const&, std::shared_ptr<DB::IDatabase>&, std::vector<StrongTypedef<wide::integer<128ul, unsigned int>, DB::UUIDTag>, std::allocator<StrongTypedef<wide::integer<128ul, unsigned int>, DB::UUIDTag>>>&) @ 0x0000000013bc05e4 in /usr/bin/clickhouse
    5. DB::InterpreterDropQuery::executeToDatabase(DB::ASTDropQuery const&) @ 0x0000000013bbc6b8 in /usr/bin/clickhouse
    6. DB::InterpreterDropQuery::execute() @ 0x0000000013bbba22 in /usr/bin/clickhouse
    7. ? @ 0x00000000140b13a5 in /usr/bin/clickhouse
    8. DB::executeQuery(String const&, std::shared_ptr<DB::Context>, bool, DB::QueryProcessingStage::Enum) @ 0x00000000140ad20e in /usr/bin/clickhouse
    9. ? @ 0x00000000140d2ef0 in /usr/bin/clickhouse
    10. DB::maybeConvertSystemDatabase(std::shared_ptr<DB::Context>) @ 0x00000000140d0aaf in /usr/bin/clickhouse
    11. DB::Server::main(std::vector<String, std::allocator<String>> const&) @ 0x000000000e724e55 in /usr/bin/clickhouse
    12. Poco::Util::Application::run() @ 0x0000000017ead086 in /usr/bin/clickhouse
    13. DB::Server::run() @ 0x000000000e714a5d in /usr/bin/clickhouse
    14. Poco::Util::ServerApplication::run(int, char**) @ 0x0000000017ec07b9 in /usr/bin/clickhouse
    15. mainEntryClickHouseServer(int, char**) @ 0x000000000e711a26 in /usr/bin/clickhouse
    16. main @ 0x0000000008cf13cf in /usr/bin/clickhouse
    17. __libc_start_main @ 0x0000000000021b97 in /lib/x86_64-linux-gnu/libc-2.27.so
    18. _start @ 0x00000000080705ae in /usr/bin/clickhouse
     (version 23.7.1.2012)

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-07-24 10:34:50 +02:00
Sema Checherinda
cfed3589b0 fix style 2023-07-24 01:11:26 +03:00
Sema Checherinda
3cc7f137bd add tests with connection reset by peer 2023-07-22 00:45:12 +02:00
Antonio Andelic
59ad2d9acc
Merge branch 'master' into feature/keeper-dyn-reconf 2023-07-21 14:08:26 +02:00
Nikolai Kochetov
e02a7c7382
Merge pull request #52375 from ClickHouse/disable-analyzer-bc-tests
Disable analyzer setting in backward_compatibility integration tests.
2023-07-21 13:39:06 +02:00
Antonio Andelic
10c2d5dade
Merge branch 'master' into feature/keeper-dyn-reconf 2023-07-21 13:24:18 +02:00
Alexander Tokmakov
a39ba00ec3
Merge pull request #52116 from aalexfvk/http_failover
Handle multiple resolved IPs for HTTP/HTTPS session connection
2023-07-20 18:25:45 +03:00
robot-clickhouse
a26de1b370 Automatic style fix 2023-07-20 15:12:55 +00:00
Nikolai Kochetov
ea252e2f61 Disable analyzer setting in backward_compatibility integration tests. 2023-07-20 15:05:07 +00:00
robot-ch-test-poll
21f22c4043
Merge pull request #49698 from kssenii/fix-pg-sync-tables-exception-broken-sync
Fix materialised pg syncTables
2023-07-20 15:38:18 +02:00
Nikolay Degterinsky
209429d0e3
Merge pull request #49664 from ilejn/test_for_basic_auth_registry
Basic auth to fetch Avro schema in Kafka
2023-07-20 10:58:11 +02:00
alesapin
45a7134478
Merge branch 'master' into feature/keeper-dyn-reconf 2023-07-19 14:04:59 +02:00
Ilya Golshtein
c550a532e6 test_for_basic_auth_registry - black formatter happy + some doc 2023-07-19 08:32:45 +00:00
Ilya Golshtein
66581d091a test_for_basic_auth_registry - both tests works, simplifications 2023-07-19 08:32:45 +00:00
Ilya Golshtein
fb3a860d7f test_for_basic_auth_registry - one kafka instance again 2023-07-19 08:32:45 +00:00
Ilya Golshtein
e3523cb1a4 test_for_basic_auth_registry - attempt to have two kafka instances 2023-07-19 08:32:45 +00:00
Ilya Golshtein
8b0fc82834 test_for_basic_auth_registry - mock up 2023-07-19 08:32:44 +00:00
Azat Khuzhin
e9b7a84aa1 Fix network manager for integration tests
Sometimes you may get:

    >           raise subprocess.CalledProcessError(exit_code, cmd)
    E           subprocess.CalledProcessError: Command '['iptables', '--wait', '-D', 'DOCKER-USER', '-p', 'tcp', '-s', '172.16.2.3', '-d', '172.16.2.2', '-j', 'DROP']' returned non-zero exit status 137.

And only sometimes you may get the reason:

    OCI runtime exec failed: exec failed: cannot exec in a stopped container: unknown

So this means that container for iptables does not exists anymore, and
the reason is the timeout. And the fact that container_exit_timeout was
equal to container_expire_timeout and was 120.

From the docker logs:

    time="2023-07-16T15:46:52.513673446Z" level=debug msg="form data: {\"AttachStderr\":false,\"AttachStdin\":false,\"AttachStdout\":false,\"Cmd\":[\"sleep\",\"120\"],\"HostConfig\":{\"AutoRemove\":true,\"NetworkMode\":\"host\"},\"Image\":\"clickhouse/integration-helper:latest\",\"NetworkDisabled\":false,\"OpenStdin\":false,\"StdinOnce\":false,\"Tty\":false}"
    time="2023-07-16T15:48:57.611857183Z" level=debug msg="form data: {\"AttachStderr\":false,\"AttachStdin\":false,\"AttachStdout\":false,\"Cmd\":[\"sleep\",\"120\"],\"HostConfig\":{\"AutoRemove\":true,\"NetworkMode\":\"host\"},\"Image\":\"clickhouse/integration-helper:latest\",\"NetworkDisabled\":false,\"OpenStdin\":false,\"StdinOnce\":false,\"Tty\":false}"

And then pytest will try to execute the iptables command:

    time="2023-07-16T15:50:57.698705244Z" level=debug msg="starting exec command 860920ab2aa07e8d285050f200ac92423a3cf8ec3fb2f57683541e62cf6bc20e in container 66d6c96671b5e987345290ddd260727d96b99789b512d40f333f6263f42fd2f1"

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-07-18 12:17:26 +02:00
Antonio Andelic
61dc020b4d
Merge branch 'master' into feature/keeper-dyn-reconf 2023-07-17 12:50:17 +02:00
Kseniia Sumarokova
200dfc24b6
Merge branch 'master' into fix-pg-sync-tables-exception-broken-sync 2023-07-17 11:53:42 +02:00
Aleksei Filatov
801d0955ec Add tests 2023-07-15 08:18:35 +03:00
robot-ch-test-poll
93b40c490f
Merge pull request #51945 from arenadata/ADQM-950-B
Implement support of date/time format specifiers in log and error log file names
2023-07-14 18:54:27 +02:00
Dmitry Novik
ee2392bf8e Merge remote-tracking branch 'origin/master' into analyzer-integration-tests 2023-07-11 08:55:31 +00:00
Victor Krasnov
e592c7c353 Fix integration tests - recover clickhouse_stay_alive_command lost by incomplete copy-paste 2023-07-11 10:32:33 +08:00
Victor Krasnov
e311d31d87 Merge branch 'master' into ADQM-950-B 2023-07-10 15:50:09 +08:00
Alexey Milovidov
b5f9a170eb
Merge pull request #51848 from ClickHouse/many-fixes-3
Many fixes [3]
2023-07-07 13:29:49 +03:00
Victor Krasnov
63fbde41fe Reformat cluster.py (add empty line) 2023-07-07 11:01:39 +08:00
Alexander Tokmakov
fdbd59ec6c
Merge pull request #51135 from ClickHouse/fix_trash_in_disks
Try to fix some trash in Disks and part moves
2023-07-07 02:02:17 +03:00
Mike Kot
c2a0607cf8 fix 2023-07-07 00:20:54 +03:00
Mike Kot
4550b15876 try updating tests to wait for cluster configs to come in sync 2023-07-07 00:20:54 +03:00
Mike Kot
8b6376005a "reconfig" support for CH Keeper 2023-07-07 00:20:54 +03:00
Victor Krasnov
479efaa79a Add clickhouse_log_file and clickhouse_error_log_file args to add_instance() 2023-07-06 14:16:18 +08:00
Antonio Andelic
57ada39ccf Add retries and iptables rules dump 2023-07-04 09:31:15 +00:00
Mikhail f. Shiryaev
340262814a
Avoid additional maven requests for spark tests 2023-06-29 12:24:19 +02:00
Mikhail f. Shiryaev
ead81879b4
Update integration tests runner too 2023-06-28 22:19:32 +02:00
Mikhail f. Shiryaev
45cd7f35cd
Use clickhouse/integration-helper from changed images 2023-06-28 22:11:48 +02:00
robot-clickhouse
fc5ed7ffd7 Automatic style fix 2023-06-21 18:45:28 +00:00
Alexander Tokmakov
9157314b2a fix 2023-06-21 20:29:32 +02:00
Dmitry Novik
49f88f5873 Fix ENV variable name 2023-06-15 14:15:01 +00:00
robot-clickhouse
5d54133248 Automatic style fix 2023-06-13 13:31:14 +00:00
Dmitry Novik
9e8ca5f5ae Add integration test check with enabled analyzer 2023-06-13 13:09:39 +00:00
Sema Checherinda
2c018f5261 do not call finalize after exception 2023-06-13 10:48:31 +02:00
Sema Checherinda
0aa30ef129 reuse s3_mocks, rewrite test test_paranoid_check_in_logs 2023-06-01 01:29:37 +02:00
Vitaly Baranov
6d45d0c374
Use fingerprints instead of key IDs in encrypted disks (#49882)
* Use fingerprints instead of key IDs to find keys in encrypted disks.
Always use little endian in the headers of encryption files.

* Add tests.

* Fix copying binary files to test containers.

* Fix ownership for copied files in test containers.

* Add comments after review.

---------

Co-authored-by: Nikita Mikhaylov <mikhaylovnikitka@gmail.com>
2023-05-31 13:11:10 +02:00
kssenii
c9e752fdc5 Fix 2023-05-12 00:10:49 +02:00
kssenii
bd59effd9a Merge remote-tracking branch 'upstream/master' into fix-pg-sync-tables-exception-broken-sync 2023-05-11 13:29:55 +02:00