Commit Graph

5751 Commits

Author SHA1 Message Date
Azat Khuzhin
e6695196aa tests/integration: add SYS_ADMIN capability for mount/remount
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-01-27 21:11:10 +01:00
Azat Khuzhin
1e3f33ad4b tests/integration: fix --privileged passthrough
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-01-27 21:11:10 +01:00
kssenii
68cab4bc49 Merge remote-tracking branch 'origin/storage-rabbitmq-fix-flush' into storage-rabbitmq-fix-flush 2023-01-27 17:23:05 +01:00
kssenii
f39cd0b392 Update tests 2023-01-27 17:22:24 +01:00
kssenii
9c4a97bbd1 Merge remote-tracking branch 'upstream/master' into storage-rabbitmq-fix-flush 2023-01-27 17:17:09 +01:00
robot-clickhouse
9cdc8d7b9d Automatic style fix 2023-01-27 09:46:51 +00:00
Antonio Andelic
defc9108f0 Extend support for aliases to constraints and profiles 2023-01-27 09:39:10 +00:00
Bharat Nallan Chakravarthy
70a3ffa0e6 fix test 2023-01-25 06:47:04 -08:00
Vitaly Baranov
9014023625 Fix wiping sensitive info in INSERT INTO FUNCTION. 2023-01-25 14:50:49 +01:00
Vitaly Baranov
32efe92199
Merge pull request #45487 from vitlibar/use-new-copy-s3-functions-in-s3-obj-storage
Use new copy s3 functions in S3ObjectStorage
2023-01-25 13:22:04 +01:00
Bharat Nallan
2ef8fcb318
Merge branch 'master' into ncb/odbc-connection-pool-fixes 2023-01-24 21:27:20 -08:00
Bharat Nallan Chakravarthy
fc4eba3b7b add a test 2023-01-24 16:56:56 -08:00
Vitaly Baranov
c22b0076ce Fix cleanup in tests test_replicated_merge_tree_s3_restore. 2023-01-24 18:05:44 +01:00
SmitaRKulkarni
d387835774
Merge branch 'master' into 45486_Fix_flaky_test_for_disallowing_concurrent_backups_restores 2023-01-24 13:49:32 +01:00
Smita Kulkarni
642f9ca549 Merge branch 'master' into 45486_Fix_flaky_test_for_disallowing_concurrent_backups_restores 2023-01-24 09:37:28 +01:00
Alexander Tokmakov
0e0e2f0f67
Update test.py 2023-01-23 19:01:06 +03:00
Aleksandr
2caeed901b
Merge branch 'master' into fix_backup_restore_num_files 2023-01-23 13:43:20 +01:00
Smita Kulkarni
383af7f3a6 Moved settings inside backups section - Updated backup/restore status when concurrent backups & restores are not allowed 2023-01-23 08:59:44 +01:00
Anton Popov
f181254fb0 fix race in destructor of ParallelParsingInputFormat 2023-01-23 01:18:58 +00:00
Smita Kulkarni
310ae62d90 Updated backup/restore status when concurrent backups & restores are not allowed
Implementation:
* Moved concurrent backup/restore check inside try-catch block which sets the status so that other nodes in cluster are aware of failures.
* Renamed backup_uuid to restore_uuid in RestoreSettings.
Testing:
* Updated test test_backup_and_restore_on_cluster/test_disallow_concurrency to check for specific backup/restore id.
2023-01-22 19:01:09 +01:00
Vitaly Baranov
5ceb64accc Use new copy s3 functions in S3ObjectStorage. 2023-01-21 15:47:58 +01:00
Vitaly Baranov
f0fda580d0
Merge pull request #45188 from vitlibar/backup-to-s3-memory-optimization
Optimize memory consumption during backup to S3
2023-01-21 12:37:35 +01:00
Sema Checherinda
962894afc8
Merge pull request #44909 from CheSema/intersect-prev-part
Do not merge over a gap with outdated undeleted parts
2023-01-21 11:51:21 +01:00
Anton Popov
41a199e175
Fix crash when ListObjects request fails (#45371) 2023-01-20 20:10:23 +01:00
Maksim Kita
23e26032ca
Merge pull request #45399 from aalexfvk/alexfvk/mdb-21326_fix_system_dictionaries_when_dictionary_with_bad_structure
Fix select from system.dictionaries when there is dictionary with bad structure
2023-01-20 13:36:32 +03:00
Antonio Andelic
0ad37ad286
Merge pull request #45320 from stigsb/system_tables_volume_config
Add <storage_policy> config parameter for system logs
2023-01-20 10:27:57 +01:00
Aleksandr Musorin
838acb22b7
added num_processed_files and processed_files_size 2023-01-20 10:20:41 +01:00
SmitaRKulkarni
6aa63414db
Merge pull request #45072 from ClickHouse/43891_Disallow_concurrent_backups_and_restores
Added settings to disallow concurrent backups and restores
2023-01-20 09:17:20 +01:00
Aleksei Filatov
42549e89f2 [rev 2] Fix review notes 2023-01-20 09:37:49 +03:00
Kseniia Sumarokova
1452d2588c
Update test.py 2023-01-19 18:06:43 +01:00
Kseniia Sumarokova
f80246644c
Update test.py 2023-01-19 18:04:09 +01:00
Kseniia Sumarokova
db49368f5e
Merge branch 'master' into storage-rabbitmq-fix-flush 2023-01-19 18:03:42 +01:00
Aleksei Filatov
afada0ecb3 Fix review notes 2023-01-19 17:02:57 +03:00
alesapin
ccd41e6d14
Merge pull request #45403 from ClickHouse/fix_test_system_merges
Reduce an amount of trash in `tests_system_merges`
2023-01-19 12:00:54 +01:00
SmitaRKulkarni
67e2bf31f5
Merge branch 'master' into 43891_Disallow_concurrent_backups_and_restores 2023-01-19 11:21:37 +01:00
Ilya Yatsishin
f16328aa8b
Merge pull request #45402 from qoega/simpler-test_replicated_merge_tree_s3_restore 2023-01-19 10:27:42 +01:00
Stig Bakken
420c179b55 Add <storage_policy> config parameter for system logs 2023-01-19 10:25:28 +01:00
SmitaRKulkarni
db03dd1bb9
Merge branch 'master' into 43891_Disallow_concurrent_backups_and_restores 2023-01-19 09:32:50 +01:00
robot-clickhouse
33a4222155 Automatic style fix 2023-01-18 15:52:38 +00:00
Alexander Tokmakov
6f8c1424df reduce an amount of trash in tests_system_merges 2023-01-18 16:45:47 +01:00
Yatsishin Ilya
1603e734f9 Make test simpler to see errors not in teardown 2023-01-18 15:36:24 +00:00
Aleksei Filatov
5e9340f682 Add integration test 2023-01-18 17:50:38 +03:00
Vitaly Baranov
7cdb2c4c7f
Merge pull request #45351 from vitlibar/fix-backup-with-killed-mutations
Fix backup with killed mutations
2023-01-18 13:14:27 +01:00
Ilya Yatsishin
9b5e7c00d5
Merge pull request #45369 from qoega/improve-failed-kafka-logging
Improve failed kafka startup logging
2023-01-18 13:09:48 +01:00
alesapin
a4ff9027ce
Merge branch 'master' into fix_flaky_test_ttl_modify 2023-01-18 12:19:37 +01:00
Antonio Andelic
f57ee043ae
Merge pull request #45319 from ClickHouse/disable-prewhere-in-merge-different-types
Disable PREWHERE in storage Merge if types don't match
2023-01-18 10:02:06 +01:00
Ilya Yatsishin
a50c6c0abe
Merge branch 'master' into improve-failed-kafka-logging 2023-01-18 02:39:32 +01:00
robot-clickhouse
237bb15a9f Automatic style fix 2023-01-17 21:34:16 +00:00
Smita Kulkarni
6e06af1b25 Updated strategy for handling internal backups & restores to avoid concurrent internal backups & restores - Added settings to disallow concurrent backups and restores 2023-01-17 22:27:13 +01:00
Yatsishin Ilya
5ae72fe711 Improve failed kafka startup logging 2023-01-17 16:25:05 +00:00
alesapin
261dc06be9 Fix flaky ttl test 2023-01-17 17:08:32 +01:00
Antonio Andelic
ba363c141a Fix integration test 2023-01-17 13:12:52 +00:00
Vitaly Baranov
8c5c423d67 Add test. 2023-01-17 13:11:24 +01:00
Sema Checherinda
2cd55bc398 fix/adjust tests in integration/test_partition 2023-01-17 12:02:18 +01:00
Vitaly Baranov
14a7ee8e26 Copy files to S3 during backup directly without using WriteBufferFromS3 to decrease memory consumption. 2023-01-17 09:35:41 +01:00
Anton Popov
6863cd152f
Merge pull request #42181 from CurtizJ/optimize-loading-parts
Do not load inactive parts at startup
2023-01-16 14:38:50 +01:00
Kseniia Sumarokova
75318e4cee
Merge pull request #45180 from kssenii/fix-flacky-test-multiple-disks
Fix flaky test test_multiple_disks/test.py::test_rename
2023-01-13 19:05:03 +01:00
Alexander Tokmakov
db91c4a749
Merge pull request #45168 from ClickHouse/tavplubix-patch-1
Try to fix flaky test_ttl_move_memory_usage
2023-01-13 15:01:42 +03:00
kssenii
0110b63a95 Better fix 2023-01-13 12:12:27 +01:00
Ilya Yatsishin
94e8c57d0d
Merge pull request #45094 from ClickHouse/try-fix-keeper-map
Try fixing KeeperMap tests
2023-01-13 00:46:43 +01:00
Yakov Olkhovskiy
842a34666c
Merge pull request #44594 from arenadata/ADQM-634
Add keytab parameter in kerberos section of config.xml
2023-01-12 15:07:45 -05:00
Nikolai Kochetov
e21fe952c6
Update test_ttl_move_memory_usage.py 2023-01-12 18:45:19 +01:00
Nikolai Kochetov
1911c25a7b
Merge branch 'master' into tavplubix-patch-1 2023-01-12 18:44:39 +01:00
Antonio Andelic
1f2ec534f4 Add EXISTS clauses 2023-01-12 15:39:04 +00:00
Antonio Andelic
dc4262f338 Merge branch 'master' into try-fix-keeper-map 2023-01-12 15:19:51 +00:00
Antonio Andelic
fb23a399b5
Merge pull request #45213 from ClickHouse/fix-flaky-keeper-test
Fix flaky `test_keeper_three_nodes_two_alive`
2023-01-12 14:57:40 +01:00
Yakov Olkhovskiy
344fa16184
Merge pull request #42991 from ClickHouse/tests-connection-error
add retries on ConnectionError
2023-01-12 08:32:03 -05:00
Ilya Yatsishin
23c80d77a0
Merge pull request #45206 from ClickHouse/try-fix-overcommit-test
Disable check to make test_overcommit_tracker not flaky
2023-01-12 14:16:11 +01:00
alesapin
df8ceeefdc
Merge pull request #45134 from ClickHouse/fix_flaky_azure_test
Fix flaky azure test
2023-01-12 13:02:34 +01:00
Nikita Mikhaylov
62d544e742
Fix integration test test_replicated_users::test_rename_replicated (#45192)
* Done

* Update tests/integration/test_replicated_users/test.py

Co-authored-by: Sergei Trifonov <sergei@clickhouse.com>

Co-authored-by: Sergei Trifonov <sergei@clickhouse.com>
2023-01-12 12:36:36 +01:00
robot-clickhouse
d61e202416 Automatic style fix 2023-01-12 11:35:03 +00:00
Antonio Andelic
feaad2bd43 try avoiding bad file descriptor 2023-01-12 11:28:41 +00:00
Antonio Andelic
97266ccabc Merge branch 'master' into try-fix-keeper-map 2023-01-12 11:16:35 +00:00
Kseniia Sumarokova
56f8899df8
Update tests/integration/test_multiple_disks/test.py
Co-authored-by: Ilya Yatsishin <2159081+qoega@users.noreply.github.com>
2023-01-12 12:06:31 +01:00
Antonio Andelic
0fc279e2bb Don't allow third node to become leader 2023-01-12 08:44:55 +00:00
Dmitry Novik
bed39f3762 Disable check to make test on flaky 2023-01-11 23:52:13 +00:00
kssenii
e3a1af6542 Fix flacky test 2023-01-11 23:51:03 +01:00
Ilya Yatsishin
bca117b26d
Merge pull request #45099 from ClickHouse/qoega-patch-4
Try to fix flaky test_create_user_and_login/test.py::test_login_as_dropped_user_xml
2023-01-11 23:37:03 +01:00
Nikolai Kochetov
653d18cb8c
Merge pull request #45162 from ClickHouse/disable-test_move_and_s3_memory_usage
Disable test_ttl_move_memory_usage as too flaky.
2023-01-11 20:30:28 +01:00
Ilya Yatsishin
cc352804df
Merge pull request #45156 from qoega/green-ci-5
Fix flaky test_tcp_handler_interserver_listen_host
2023-01-11 19:09:22 +01:00
Antonio Andelic
2c33c9e62c
Merge branch 'master' into try-fix-keeper-map 2023-01-11 15:05:04 +01:00
Alexander Tokmakov
764abb6410
try to fix flaky test_ttl_move_memory_usage 2023-01-11 15:55:18 +03:00
Nikolai Kochetov
07a3967d6b Disable test_ttl_move_memory_usage as too flaky. 2023-01-11 11:20:02 +00:00
Alexander Tokmakov
b156209ed6
Update test.py 2023-01-11 14:17:44 +03:00
Yatsishin Ilya
77f0724629 Fix flaky test_tcp_handler_interserver_listen_host 2023-01-11 10:28:30 +00:00
robot-clickhouse
6d6e803cfb Automatic style fix 2023-01-11 10:09:43 +01:00
Ilya Yatsishin
f871949d85 Try to fix flaky test_create_user_and_login/test.py::test_login_as_dropped_user_xml 2023-01-11 10:09:43 +01:00
Anton Popov
6c4f53bdca Merge remote-tracking branch 'upstream/master' into HEAD 2023-01-11 01:00:57 +00:00
alesapin
6d86b8dd47 Fix flaky azure test 2023-01-10 19:05:02 +01:00
Antonio Andelic
7d948c9c77
Merge branch 'master' into try-fix-keeper-map 2023-01-10 15:44:28 +01:00
Antonio Andelic
134cc3e273
Change table name 2023-01-10 14:16:28 +01:00
Maksim Kita
d4a2dd43ce
Merge pull request #44891 from azat/tests/rewrite-update_field-test
Convert integration test_dictionaries_update_field to a stateless
2023-01-10 15:14:53 +03:00
Antonio Andelic
d7ca0c04fa
remove drop if exists 2023-01-10 11:35:02 +01:00
Antonio Andelic
7aef7d95de Small fixes for keeper_map tests 2023-01-10 09:15:07 +00:00
robot-clickhouse
46b21629ed Automatic style fix 2023-01-09 17:23:02 +00:00
Smita Kulkarni
93530e8d34 Added settings to disallow concurrent backups and restores
Implementation:
* Added server level settings to disallow concurrent backups and restores, which are read and set when BackupWorker is created in Context.
* Settings are set to true by default.
* Before starting backup or restores, added a check to see if any other backups/restores are running (except internal ones).
Testing:
* Added a test test_backup_and_restore_on_cluster/test_disallow_concurrency.
2023-01-09 18:14:39 +01:00
Nikita Mikhaylov
857799fbca
Parallel distributed insert select with s3Cluster [3] (#44955)
* Revert "Revert "Resurrect parallel distributed insert select with s3Cluster (#41535)""

This reverts commit b8d9066004.

* Fix build

* Better

* Fix test

* Automatic style fix

Co-authored-by: robot-clickhouse <robot-clickhouse@users.noreply.github.com>
2023-01-09 13:30:32 +01:00
Kseniia Sumarokova
9ae5ad202a
Merge pull request #44707 from xiedeyantu/s3-add-config
S3 support upload object using storage class INTELLIGENT_TIERING
2023-01-09 13:28:53 +01:00
Roman Vasin
2995c6d866 Merge branch 'master' of github.com:ClickHouse/ClickHouse into ADQM-634 2023-01-09 08:26:45 +00:00
Alexey Milovidov
851e4ae59f Fix typos 2023-01-08 07:08:20 +01:00
Antonio Andelic
c8d9368009 Retry on invalid snapshot 2023-01-07 17:08:25 +00:00
Anton Popov
1f32ffedf8
Merge pull request #43221 from ClickHouse/refactoring-ip-types
Replace domain IP types (IPv4, IPv6) with native
2023-01-07 12:01:21 +01:00
Alexey Milovidov
590ad41cbf
Merge pull request #44978 from azat/tests/fix-test_replicated_users
tests: fix test_replicated_users flakiness
2023-01-07 03:51:31 +03:00
Alexey Milovidov
d122d72ea8
Merge pull request #44996 from vitlibar/fix-test_grpc_protocol_progress
Fix test test_grpc_protocol/test.py::test_progress
2023-01-07 03:48:30 +03:00
Vitaly Baranov
e8355971bb Fix test test_grpc_protocol/test.py::test_progress 2023-01-06 19:58:20 +01:00
kssenii
dd6a2c2d37 Fix style check 2023-01-06 16:53:24 +01:00
kssenii
2a030c1dc0 Merge remote-tracking branch 'upstream/master' into allow-single-disk-instead-of-storage-policy 2023-01-06 15:58:27 +01:00
Vitaly Baranov
92be50bd68 Add test for IMDS session token. 2023-01-06 15:55:26 +01:00
Vitaly Baranov
0d12d90fb5 Move common code to helpers/mock_servers.py 2023-01-06 15:55:21 +01:00
kssenii
c980e16345 Merge remote-tracking branch 'upstream/master' into storage-rabbitmq-fix-flush 2023-01-06 15:51:25 +01:00
Anton Popov
b25f875674
Merge pull request #44875 from ClickHouse/fix-another-one-cannot-read-all-data-for-lc-dict-error
Fix right offset for reading LowCardinality dictionary from remote fs
2023-01-06 15:24:36 +01:00
Kseniia Sumarokova
573d3283b0
Merge pull request #44327 from kssenii/use-new-named-collections-code-2
Replace old named collections code with new (from #43147) part 2
2023-01-06 13:06:26 +01:00
Nikolai Kochetov
12192ef099
Merge pull request #44938 from ClickHouse/try-fix-test_multiple_disks-test_freeze
Stop merges to avoid a race between merge and freeze.
2023-01-06 12:45:10 +01:00
Azat Khuzhin
7859346e40 tests: fix test_replicated_users flakiness
Alter for users/quotas/... does not wait until all replicas will be up
to date, so use *_with_retries() helperx.

CI: https://s3.amazonaws.com/clickhouse-test-reports/44922/bd885be9229cf47752c5c98392f09129261550f9/integration_tests__tsan__[6/6].html
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-01-06 12:45:03 +01:00
Azat Khuzhin
c03510cf6b tests/integration: add query_and_get_error_with_retry() helper
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-01-06 12:45:03 +01:00
Alexey Milovidov
08ebdc9b82
Merge pull request #44903 from ClickHouse/less-flaky-test-concurrent-queries
Fix flaky `test_concurrent_queries_restriction_by_query_kind`
2023-01-06 03:18:28 +03:00
robot-clickhouse
62a9b33bbb Automatic style fix 2023-01-05 16:29:58 +00:00
Ilya Yatsishin
a28d6fb490
Merge pull request #44848 from qoega/green-ci-2
Pre-pulling images for integration tests to detect broken infrastructure earlier
2023-01-05 17:29:39 +01:00
Nikolai Kochetov
90f3f5789f Speedup a bit. 2023-01-05 16:23:49 +00:00
Nikolai Kochetov
dc5009d401 Stop merges to aviod a race between merge and freeze. 2023-01-05 15:28:56 +00:00
Anton Popov
ed00399eb6 Merge remote-tracking branch 'upstream/master' into HEAD 2023-01-05 14:12:02 +00:00
Yatsishin Ilya
1b03a60e43 Do not report status if we failed to pull images. 2023-01-05 13:53:45 +00:00
Ilya Yatsishin
c5f01a653c
Merge pull request #44880 from qoega/green-ci-4
Rewrite test_postgres_protocol test
2023-01-05 12:34:33 +01:00
Antonio Andelic
7dd8ac717a
Merge branch 'master' into less-flaky-test-concurrent-queries 2023-01-05 12:23:30 +01:00
Ilya Yatsishin
5d0f3e7819
Merge pull request #44921 from ClickHouse/fix-flaky-test-5
Fix flaky test `test_lost_part`
2023-01-05 10:28:07 +01:00
xiedeyantu
dfaa16b7a0 fix 2023-01-05 09:09:42 +08:00
xiedeyantu
110f95bae6 fix 2023-01-05 09:00:53 +08:00
Nikita Mikhaylov
e3ca9fb3e9 Blakc 2023-01-04 20:30:52 +00:00
Nikita Mikhaylov
ea7c338a7b Done 2023-01-04 20:28:03 +00:00
Yatsishin Ilya
ccb6226d22 install CH before test 2023-01-04 20:07:52 +00:00
Yakov Olkhovskiy
dadd96ba6f
Merge pull request #44135 from ClickHouse/feature-protocol-proxy
Improve support for PROXYv1 protocol.
2023-01-04 14:06:17 -05:00
Yatsishin Ilya
daf2ccab81 Merge remote-tracking branch 'origin/master' into green-ci-2 2023-01-04 18:27:19 +00:00
xiedeyantu
2f59b1d002 better 2023-01-05 00:27:07 +08:00
Yakov Olkhovskiy
7a5a36cbed
Merge branch 'master' into refactoring-ip-types 2023-01-04 11:11:06 -05:00
Antonio Andelic
8ee589c5c1
Merge pull request #44898 from ClickHouse/try-fix-test-storage-kafka
Try to fix flaky `test_storage_kafka::test_kafka_produce_key_timestamp`
2023-01-04 16:54:47 +01:00
Yatsishin Ilya
03582f4b87 move logic to runner 2023-01-04 15:52:34 +00:00
Yatsishin Ilya
9cf599e58b Merge remote-tracking branch 'origin/master' into green-ci-2 2023-01-04 15:49:22 +00:00
Yatsishin Ilya
d8d012523d add pre-pull to runner 2023-01-04 15:47:50 +00:00
alesapin
92255e5d4b
Merge pull request #43020 from vitlibar/update-aws-c-submodules
Update aws-c* submodules
2023-01-04 16:25:22 +01:00
Antonio Andelic
1a1afc6a74 Make test_concurrent_queries_restriction_by_query_kind less flaky 2023-01-04 14:18:35 +00:00
Nikita Mikhaylov
e9c7555365
Use TablesDependencyGraph in DatabaseReplicated recovery process (#44697) 2023-01-04 14:49:39 +01:00
Yatsishin Ilya
4d5131ed7d debug quotes 2023-01-04 13:16:31 +00:00
Antonio Andelic
890f094829 Set retention policy for Kafka topic 2023-01-04 12:04:00 +00:00
Yatsishin Ilya
08fbf6e5b2 again 2023-01-04 11:48:28 +00:00
alesapin
c09f83654d
Merge pull request #44533 from ClickHouse/40951_Allow_macro_substitution_in_endpoint
Allow macro substitution in endpoint of disks
2023-01-04 12:29:57 +01:00
Azat Khuzhin
0ab0f76737 Convert integration test_dictionaries_update_field to a stateless
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-01-04 11:05:00 +01:00
Antonio Andelic
9b58c97563 Store ZK generated data in archive 2023-01-04 09:10:16 +00:00
Alexey Milovidov
66e6f53698
Merge pull request #44839 from ClickHouse/move-odbc-test
Move `test_dies_with_parent` to another module
2023-01-04 05:17:19 +03:00
Alexey Milovidov
3a027d285f
Merge pull request #44828 from ClickHouse/remove-two-lines-of-code
What if I will remove two lines of code?
2023-01-04 04:50:52 +03:00
Ilya Yatsishin
49155d9614
Merge pull request #44711 from qoega/green-ci-1
Improve some integration tests and add pytest-random
2023-01-03 22:39:54 +01:00