Commit Graph

7071 Commits

Author SHA1 Message Date
Ilya Golshtein
c8347bd313 test_for_basic_auth_registry: some comments removed per code review 2023-07-19 08:32:45 +00:00
Ilya Golshtein
9f6ab5e816 test_for_basic_auth_registry - made black formatter happy 2023-07-19 08:32:45 +00:00
Ilya Golshtein
f1ce1da007 test_for_basic_auth_registry - new test and cleanup per code review 2023-07-19 08:32:45 +00:00
Ilya Golshtein
aa2b7e248d test_for_basic_auth_registry - fix port in test_kafka_formats 2023-07-19 08:32:45 +00:00
Ilya Golshtein
c550a532e6 test_for_basic_auth_registry - black formatter happy + some doc 2023-07-19 08:32:45 +00:00
Ilya Golshtein
66581d091a test_for_basic_auth_registry - both tests works, simplifications 2023-07-19 08:32:45 +00:00
Ilya Golshtein
db8e96147a test_for_basic_auth_registry - started, but only auth test works 2023-07-19 08:32:45 +00:00
Ilya Golshtein
fb3a860d7f test_for_basic_auth_registry - one kafka instance again 2023-07-19 08:32:45 +00:00
Ilya Golshtein
e3523cb1a4 test_for_basic_auth_registry - attempt to have two kafka instances 2023-07-19 08:32:45 +00:00
Ilya Golshtein
8b0fc82834 test_for_basic_auth_registry - mock up 2023-07-19 08:32:44 +00:00
Michael Kolupaev
a86baab88b Fix test_replicated_database 'node doesn't exist' flakiness 2023-07-19 08:28:04 +00:00
JackyWoo
71c12bbdc4
Merge branch 'master' into rewrite_uniq_to_count 2023-07-19 15:54:23 +08:00
robot-clickhouse-ci-1
32b765a4ba
Merge pull request #50559 from zvonand/zvonand-issue-49290
Add support for multi-directory globs
2023-07-19 06:24:38 +02:00
robot-clickhouse
3c8141529f Automatic style fix 2023-07-18 23:25:21 +00:00
Alexey Milovidov
c724816cb8 Fix test 2023-07-19 01:15:16 +02:00
Kseniia Sumarokova
1545ad3389
Merge branch 'master' into fix-pg-sync-tables-exception-broken-sync 2023-07-18 23:19:01 +02:00
robot-clickhouse
ff6e5ff1c5 Automatic style fix 2023-07-18 20:23:55 +00:00
Alexey Milovidov
b2164ab97e Fix integration tests 2023-07-18 22:13:28 +02:00
Alexey Milovidov
d423add083 Merge branch 'sleep-each-row-max-time' of github.com:ClickHouse/ClickHouse into sleep-each-row-max-time 2023-07-18 22:06:49 +02:00
Alexey Milovidov
06914e8196 Merge branch 'master' into sleep-each-row-max-time 2023-07-18 22:06:27 +02:00
vdimir
25eee81746
Fix temp data deletion on startup, add test 2023-07-18 16:29:21 +00:00
Roman Vasin
50734017ae Fix test style with black 2023-07-18 15:45:43 +00:00
Roman Vasin
c8c6c31384 Change Method into method in exceptions 2023-07-18 15:33:50 +00:00
kssenii
680903e89b Fix style check 2023-07-18 16:54:33 +02:00
Roman Vasin
ea3d9e92f1 Add support of YAML configs for decryption 2023-07-18 14:44:47 +00:00
Song Liyong
6ae5207819 MaterializedMySQL: Introduce charset conversion 2023-07-18 15:55:55 +02:00
alesapin
5fe90208cf
Merge branch 'master' into fix_flaky_test_lost_part 2023-07-18 14:43:18 +02:00
Kruglov Pavel
64e88cde21
Merge branch 'master' into better-progress-bar-2 2023-07-18 13:37:53 +02:00
kssenii
353b0f817a Merge remote-tracking branch 'origin/fix-pg-sync-tables-exception-broken-sync' into fix-pg-sync-tables-exception-broken-sync 2023-07-18 12:26:17 +02:00
kssenii
0cfd12aba4 Fix 2023-07-18 12:24:04 +02:00
Azat Khuzhin
baf6d182d6 Fix reporting some BROKEN tests as ERROR
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-07-18 12:22:38 +02:00
Azat Khuzhin
92ca2b0bdd Make psql tests in test_odbc_interaction more robust to other test failures
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-07-18 12:22:28 +02:00
Azat Khuzhin
e9b7a84aa1 Fix network manager for integration tests
Sometimes you may get:

    >           raise subprocess.CalledProcessError(exit_code, cmd)
    E           subprocess.CalledProcessError: Command '['iptables', '--wait', '-D', 'DOCKER-USER', '-p', 'tcp', '-s', '172.16.2.3', '-d', '172.16.2.2', '-j', 'DROP']' returned non-zero exit status 137.

And only sometimes you may get the reason:

    OCI runtime exec failed: exec failed: cannot exec in a stopped container: unknown

So this means that container for iptables does not exists anymore, and
the reason is the timeout. And the fact that container_exit_timeout was
equal to container_expire_timeout and was 120.

From the docker logs:

    time="2023-07-16T15:46:52.513673446Z" level=debug msg="form data: {\"AttachStderr\":false,\"AttachStdin\":false,\"AttachStdout\":false,\"Cmd\":[\"sleep\",\"120\"],\"HostConfig\":{\"AutoRemove\":true,\"NetworkMode\":\"host\"},\"Image\":\"clickhouse/integration-helper:latest\",\"NetworkDisabled\":false,\"OpenStdin\":false,\"StdinOnce\":false,\"Tty\":false}"
    time="2023-07-16T15:48:57.611857183Z" level=debug msg="form data: {\"AttachStderr\":false,\"AttachStdin\":false,\"AttachStdout\":false,\"Cmd\":[\"sleep\",\"120\"],\"HostConfig\":{\"AutoRemove\":true,\"NetworkMode\":\"host\"},\"Image\":\"clickhouse/integration-helper:latest\",\"NetworkDisabled\":false,\"OpenStdin\":false,\"StdinOnce\":false,\"Tty\":false}"

And then pytest will try to execute the iptables command:

    time="2023-07-16T15:50:57.698705244Z" level=debug msg="starting exec command 860920ab2aa07e8d285050f200ac92423a3cf8ec3fb2f57683541e62cf6bc20e in container 66d6c96671b5e987345290ddd260727d96b99789b512d40f333f6263f42fd2f1"

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-07-18 12:17:26 +02:00
Alexey Milovidov
806caea282 Update more tests 2023-07-18 01:56:38 +02:00
Alexey Milovidov
a4b92652da Update more tests 2023-07-18 01:50:43 +02:00
Alexey Milovidov
ffc6f246bf Merge branch 'master' into sparse-columns-by-default 2023-07-18 01:36:52 +02:00
Alexey Milovidov
4884022fda
Merge branch 'master' into zvonand-issue-49290 2023-07-17 23:03:50 +03:00
Kruglov Pavel
4ed8dcd436
Merge branch 'master' into async-connect-to-multiple-ips 2023-07-17 21:02:33 +02:00
Nikita Mikhaylov
11cc8b4adf
Merge branch 'master' into add_delay_for_replicated 2023-07-17 19:04:29 +02:00
Azat Khuzhin
6c986c1fb2 Allow test names with spaces in it
It had been fixed and now it is allowed.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-07-17 17:29:44 +02:00
Azat Khuzhin
acbdbf6ab7 tests: do not use --format if --quiet is set in integration runner
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-07-17 17:17:21 +02:00
Azat Khuzhin
e532d5b376 tests: use -- over quotes on and on in ci-runner.py
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-07-17 17:17:21 +02:00
Azat Khuzhin
6f6562c96b Fix running integration tests with spaces in it's names
Previously they were simply ignored, and not only them, but all the
tests in that group, here is an example [1]

  [1]: https://s3.amazonaws.com/clickhouse-test-reports/51448/4ed462ac7834a8180f92ca7d7d3c076e687bfca9/integration_tests__asan__[4_6].html

Note, that I've replaced PYTEST_OPTS with handled by pytest itself
PYTEST_ADDOPTS, since "sh -c $PYTEST_OPTS" does not allows you to
preserve the arguments correctly.

v5: use double quotes for PYTEST_ADDOPTS value
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-07-17 17:17:21 +02:00
Antonio Andelic
51f0089457
Merge branch 'master' into fix-test_multiple_disks 2023-07-17 16:25:06 +02:00
Andrey Zvonov
d339c22a2f
Merge branch 'master' into zvonand-issue-49290 2023-07-17 16:43:15 +03:00
kssenii
7a224c252e One more fix 2023-07-17 15:04:39 +02:00
kssenii
9c66655288 Fix tests 2023-07-17 15:01:58 +02:00
robot-clickhouse
71e8465663 Automatic style fix 2023-07-17 12:09:36 +00:00
Alexander Sapin
2993cf3cba Fix flaky test test_lost_part 2023-07-17 14:01:31 +02:00
Antonio Andelic
152f5527a9 Remove wrong assert 2023-07-17 11:58:22 +00:00
Kruglov Pavel
9ac1a9c556
Merge branch 'master' into async-connect-to-multiple-ips 2023-07-17 13:40:36 +02:00
Antonio Andelic
61dc020b4d
Merge branch 'master' into feature/keeper-dyn-reconf 2023-07-17 12:50:17 +02:00
Kseniia Sumarokova
200dfc24b6
Merge branch 'master' into fix-pg-sync-tables-exception-broken-sync 2023-07-17 11:53:42 +02:00
Antonio Andelic
dfc1cc13c8 Better oldest part selection 2023-07-17 08:57:22 +00:00
robot-ch-test-poll
3b0a4040cd
Merge pull request #51876 from evillique/fix-mongodb-inserts
Fix inserts into MongoDB tables
2023-07-16 05:28:43 +02:00
Alexey Milovidov
d900aa467f
Merge pull request #51992 from ClickHouse/fix-test_extreme_deduplication
Fix test_extreme_deduplication
2023-07-16 05:54:44 +03:00
Alexey Milovidov
2be26de2db
Merge pull request #52142 from ClickHouse/remove-detection-of-flaky-test
There is no point in detecting flaky tests
2023-07-16 05:52:04 +03:00
robot-ch-test-poll
67492bc2ed
Merge pull request #52122 from azat/keeper-stat-zxid
Print Zxid in keeper stat command in hex (so as ZooKeeper)
2023-07-16 03:28:53 +02:00
Alexey Milovidov
7e6551dbd0
Merge branch 'master' into fix-mongodb-inserts 2023-07-16 01:47:52 +03:00
robot-clickhouse
44fc93ba63 Automatic style fix 2023-07-15 21:10:21 +00:00
Alexey Milovidov
90f9d156d2
Merge branch 'master' into fix-test_extreme_deduplication 2023-07-16 00:05:40 +03:00
Alexey Milovidov
2ad7d121ae
Merge pull request #52085 from azat/tests/test_extreme_deduplication
Fix test_extreme_deduplication flakiness
2023-07-16 00:02:18 +03:00
Alexey Milovidov
20b77e946a There is no point in detecting flaky tests 2023-07-15 23:00:23 +02:00
Alexey Milovidov
7d510eac7c
Merge branch 'master' into tests/test_materialized_mysql_database 2023-07-15 20:36:32 +03:00
Alexey Milovidov
0d48dca171
Update materialized_with_ddl.py 2023-07-15 20:36:16 +03:00
Alexey Milovidov
47a33efc73
Merge branch 'master' into tests/test_keeper_s3_snapshot 2023-07-15 19:27:06 +03:00
Alexey Milovidov
ba60db1558
Merge branch 'master' into tests/fix-docker-network-prune 2023-07-15 19:24:59 +03:00
Alexey Milovidov
edbd05f332
Merge branch 'master' into tests/test_extreme_deduplication 2023-07-15 19:20:32 +03:00
Alexey Milovidov
0eab3276f5
Merge branch 'master' into keeper-stat-zxid 2023-07-15 19:07:48 +03:00
Vitaly Baranov
2f23b6a593
Merge branch 'master' into backups-native-copy 2023-07-15 08:40:17 +02:00
Aleksei Filatov
801d0955ec Add tests 2023-07-15 08:18:35 +03:00
Andrey Zvonov
dbdd15686a
Merge branch 'master' into zvonand-issue-49290 2023-07-15 00:53:29 +02:00
Azat Khuzhin
6abdde12ca Print Zxid in keeper stat command in hex (so as ZooKeeper)
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-07-14 21:17:09 +02:00
robot-ch-test-poll
93b40c490f
Merge pull request #51945 from arenadata/ADQM-950-B
Implement support of date/time format specifiers in log and error log file names
2023-07-14 18:54:27 +02:00
Azat Khuzhin
4f6d59ce7f tests: drop existing view in test_materialized_mysql_database
This should avoid failures of other tests in case of some previous test
failed

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-07-14 16:50:41 +02:00
Azat Khuzhin
2f17208792 Add missing --force for docker network prune (otherwise it is noop on CI)
On CI there is no stdin attached, so this command simply do nothing, fix
this.

Fixes: https://github.com/ClickHouse/ClickHouse/pull/51969
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-07-13 20:38:17 +02:00
Azat Khuzhin
2720e9177d Fix flakiness of test_keeper_s3_snapshot
This is part of "Why my PR is not merged?" patch set series.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-07-13 14:25:24 +02:00
Azat Khuzhin
3a54182f77 Fix test_extreme_deduplication flakiness
This is part of **"Why my PR is not merged?"** patch set series.

Since #50107 there one more tunable that can increase the cleanup period
- max_cleanup_delay_period.

Plus, increase the delay for the cleanup thread, otherwise it is too
fragile.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-07-13 13:58:19 +02:00
Andrey Zvonov
871cce61fa
Merge branch 'master' into zvonand-issue-49290 2023-07-13 11:44:08 +02:00
Dmitry Novik
5c73a0b19c
Merge pull request #50926 from ClickHouse/analyzer-integration-tests
Add integration test check with enabled analyzer
2023-07-13 10:59:39 +02:00
Andrey Zvonov
22d8b35ee6
Merge branch 'master' into zvonand-issue-49290 2023-07-12 12:51:10 +02:00
robot-clickhouse
881a45ec83 Automatic style fix 2023-07-12 09:26:01 +00:00
Dmitry Novik
b4723ca8df Try to fix report 2023-07-12 09:16:13 +00:00
Alexey Gerasimchuck
7b4e7cd203
Merge branch 'master' into ADQM-970 2023-07-12 07:44:54 +10:00
Roman Vasin
b6023d9f62 Merge branch 'master' of github.com:ClickHouse/ClickHouse into ADQM-822 2023-07-11 10:35:31 +00:00
Nikolai Kochetov
ab6356f9bc
Merge branch 'master' into vdimir/fix_race_in_remove_objects 2023-07-11 11:22:04 +02:00
Dmitry Novik
ee2392bf8e Merge remote-tracking branch 'origin/master' into analyzer-integration-tests 2023-07-11 08:55:31 +00:00
JackyWoo
cd1111b17b ignore alias for UniqToCountPass 2023-07-11 10:45:08 +08:00
Victor Krasnov
e592c7c353 Fix integration tests - recover clickhouse_stay_alive_command lost by incomplete copy-paste 2023-07-11 10:32:33 +08:00
JackyWoo
a01a683448 fix style 2023-07-11 09:51:52 +08:00
JackyWoo
8004f37e97 rewrite uniq to count in new analyzer 2023-07-10 17:16:41 +08:00
JackyWoo
e0b223aa7e add tests for uniq to count rewrite 2023-07-10 17:16:41 +08:00
Andrey Zvonov
c030e9dbe4
Merge branch 'master' into zvonand-issue-49290 2023-07-10 11:01:26 +02:00
Victor Krasnov
e311d31d87 Merge branch 'master' into ADQM-950-B 2023-07-10 15:50:09 +08:00
Alexey Gerasimchuck
2c8564e45e
Merge branch 'master' into ADQM-970 2023-07-10 15:53:08 +10:00
vdimir
790b438b7c
add test_delete_race_leftovers 2023-07-09 16:19:00 +00:00
Azat Khuzhin
2db092f9d8 Cleanup remote_servers in dist config.xml
At first, there was no such amount of clusters in dist config, they
added when someone need to write some new cluster for tests.

So let's move them to the clusters.xml that is deployed only for tests,
and leave only default cluster.

And cleanup also some configs that had been copied from dist config in
the repo (about test_config_* integration tests, this should be OK,
since there are more_clusters.xml as well, that covers additional
cases).

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-07-09 16:55:27 +02:00
Azat Khuzhin
29dc9abfca Fix test_backup_restore_s3 after logging for native copying changed
Check profile events instead of some odd logs.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-07-09 08:20:03 +02:00
Azat Khuzhin
84c720b33e Beatify test_backup_restore_s3 (using per-query profile events)
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-07-09 08:20:03 +02:00
Alexey Milovidov
e2c8cea381
Merge pull request #51801 from ClickHouse/fix_broken_parts_handling
Fix broken parts handling in `ReplicatedMergeTree`
2023-07-09 05:32:51 +03:00
Alexey Milovidov
7311469c32 Fix test_extreme_deduplication 2023-07-09 04:17:15 +02:00
Alexey Milovidov
1f72b1bb09
Update ci-runner.py 2023-07-09 03:28:12 +03:00
Alexey Milovidov
508a9c1800
Merge pull request #51969 from ClickHouse/docker-network-prune
Fix Docker
2023-07-08 23:45:58 +03:00
Alexey Milovidov
b958499c27
Merge pull request #51458 from ClickHouse/sunny19930321-fix-attach-table-function-name-normalizer
Merging #36384
2023-07-08 10:37:11 +03:00
Alexey Milovidov
8e4c8f118c Fix disaster in integration tests, part 2 2023-07-08 09:07:05 +02:00
Alexey Milovidov
adbd85b975 Fix Docker 2023-07-08 06:26:44 +02:00
Alexey Milovidov
a7d225561c
Merge pull request #51868 from ClickHouse/fix-test_replicated_table_attach
Fix test_replicated_table_attach
2023-07-08 02:22:13 +03:00
avogar
ed37b01b51 Fix style 2023-07-07 14:39:11 +00:00
Andrey Zvonov
3e8d0249e8
Merge branch 'master' into zvonand-issue-49290 2023-07-07 14:10:37 +02:00
Yatsishin Ilya
50bda59a0d Fix typo 2023-07-07 11:05:42 +00:00
avogar
fc94cc8b87 Update config for test 2023-07-07 10:59:27 +00:00
avogar
cd3080428e Fix async connect to hosts with multiple ips 2023-07-07 10:58:01 +00:00
Alexey Milovidov
03c4a6e26c
Merge branch 'master' into sunny19930321-fix-attach-table-function-name-normalizer 2023-07-07 13:45:30 +03:00
Alexey Milovidov
95b00df8dd
Merge branch 'master' into fix-test_replicated_table_attach 2023-07-07 13:45:07 +03:00
Alexey Milovidov
010f6b627f
Merge branch 'master' into fix-flaky-keeper-map-test 2023-07-07 13:37:58 +03:00
Alexey Milovidov
b5f9a170eb
Merge pull request #51848 from ClickHouse/many-fixes-3
Many fixes [3]
2023-07-07 13:29:49 +03:00
Antonio Andelic
ff79ceaafe Skip parallel keepermap test 2023-07-07 09:58:35 +00:00
Antonio Andelic
fb2affcae3 Dump all rules 2023-07-07 08:37:08 +00:00
Antonio Andelic
5875d66846 Merge branch 'master' into fix-flaky-keeper-map-test 2023-07-07 08:36:37 +00:00
Victor Krasnov
95fedaedff Refine the integration test code 2023-07-07 15:16:10 +08:00
Victor Krasnov
87ea1b6667 Recover the integration test 2023-07-07 13:01:30 +08:00
Victor Krasnov
63fbde41fe Reformat cluster.py (add empty line) 2023-07-07 11:01:39 +08:00
Alexander Tokmakov
fdbd59ec6c
Merge pull request #51135 from ClickHouse/fix_trash_in_disks
Try to fix some trash in Disks and part moves
2023-07-07 02:02:17 +03:00
Mike Kot
c2a0607cf8 fix 2023-07-07 00:20:54 +03:00
Mike Kot
2f0cd05497 handle leader removal corner cases 2023-07-07 00:20:54 +03:00
Mike Kot
4550b15876 try updating tests to wait for cluster configs to come in sync 2023-07-07 00:20:54 +03:00
Mike Kot
8b6376005a "reconfig" support for CH Keeper 2023-07-07 00:20:54 +03:00
Alexander Tokmakov
abf36065b7 fix 2023-07-06 17:24:06 +02:00
Alexey Milovidov
6d8434c702 Merge branch 'master' into many-fixes-3 2023-07-06 16:17:32 +02:00
Alexey Milovidov
1fa21700ad Merge remote-tracking branch 'origin/less-flaky-test_start_stop_moves' into many-fixes-3 2023-07-06 16:17:13 +02:00
Alexander Sapin
30bcc73c63 Merge branch 'master' into add_delay_for_replicated 2023-07-06 14:29:15 +02:00
Kseniia Sumarokova
e97e107bcc
Merge branch 'master' into add-separate-access-for-use-named-collections 2023-07-06 12:16:53 +02:00
Andrey Zvonov
cc0210f760
Merge branch 'master' into zvonand-issue-49290 2023-07-06 11:13:51 +02:00
Antonio Andelic
7281bda072 Merge branch 'master' into fix-flaky-keeper-map-test 2023-07-06 06:43:27 +00:00
Antonio Andelic
0b0ce5191c Merge branch 'master' into less-flaky-test_start_stop_moves 2023-07-06 06:17:29 +00:00
Antonio Andelic
af0de3d614 Small fixes 2023-07-06 06:17:11 +00:00
Victor Krasnov
fef71ab0b8 Add inegration test 2023-07-06 14:16:46 +08:00
Victor Krasnov
479efaa79a Add clickhouse_log_file and clickhouse_error_log_file args to add_instance() 2023-07-06 14:16:18 +08:00
Nikolay Degterinsky
546f12dc85 Fix inserts to MongoDB tables 2023-07-06 05:05:27 +00:00
Alexey Milovidov
64d5a85f6e Fix test_replicated_table_attach 2023-07-06 03:16:06 +02:00
Alexey Milovidov
5452f2106f
Merge branch 'master' into sunny19930321-fix-attach-table-function-name-normalizer 2023-07-06 03:50:18 +03:00
Sema Checherinda
44791af710 stop merges properly for replicated tables 2023-07-05 22:54:22 +02:00
alesapin
66484a4dd0
Merge branch 'master' into add_delay_for_replicated 2023-07-05 20:03:34 +02:00
Kruglov Pavel
60cf8a4c94
Merge branch 'master' into better-progress-bar-2 2023-07-05 19:58:55 +02:00
Antonio Andelic
b60a1c53d6 Fix oldest part fetching 2023-07-05 17:43:09 +00:00
alesapin
f2d106ffb7 Fix typos 2023-07-05 18:54:33 +02:00
robot-clickhouse
085576efc9 Automatic style fix 2023-07-05 16:22:58 +00:00
alesapin
baee73fd96 Make shutdown of replicated tables softer 2023-07-05 18:11:25 +02:00
Ilya Yatsishin
b0e2d010eb Merge remote-tracking branch 'origin/fix-flaky-keeper-map-test' into many-fixes-3 2023-07-05 17:07:21 +02:00
Antonio Andelic
12f413c7a7
Merge branch 'master' into fix-flaky-keeper-map-test 2023-07-05 14:00:10 +02:00