thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-12 01:12:12 +00:00
Code Issues Packages Projects Releases Wiki Activity
111,248 Commits 9,297 Branches 1,782 Tags 2.1 GiB
1605afe647
Commit Graph

6046 Commits

Author SHA1 Message Date
Antonio Andelic
6da9dc2407 Correctly handle case with missing custom_key 2023-01-24 14:16:26 +00:00
SmitaRKulkarni
d387835774
Merge branch 'master' into 45486_Fix_flaky_test_for_disallowing_concurrent_backups_restores 2023-01-24 13:49:32 +01:00
Antonio Andelic
95853af459 Merge branch 'master' into custom-key-parallel-replicas 2023-01-24 10:49:40 +00:00
Antonio Andelic
37b62b3a58 Use Map for custom_key 2023-01-24 10:46:47 +00:00
Smita Kulkarni
642f9ca549 Merge branch 'master' into 45486_Fix_flaky_test_for_disallowing_concurrent_backups_restores 2023-01-24 09:37:28 +01:00
Alexander Tokmakov
0e0e2f0f67
Update test.py 2023-01-23 19:01:06 +03:00
Aleksandr
2caeed901b
Merge branch 'master' into fix_backup_restore_num_files 2023-01-23 13:43:20 +01:00
Smita Kulkarni
383af7f3a6 Moved settings inside backups section - Updated backup/restore status when concurrent backups & restores are not allowed 2023-01-23 08:59:44 +01:00
Anton Popov
f181254fb0 fix race in destructor of ParallelParsingInputFormat 2023-01-23 01:18:58 +00:00
Smita Kulkarni
310ae62d90 Updated backup/restore status when concurrent backups & restores are not allowed 
Implementation:
* Moved concurrent backup/restore check inside try-catch block which sets the status so that other nodes in cluster are aware of failures.
* Renamed backup_uuid to restore_uuid in RestoreSettings.
Testing:
* Updated test test_backup_and_restore_on_cluster/test_disallow_concurrency to check for specific backup/restore id.
 2023-01-22 19:01:09 +01:00
Antonio Andelic
44ef00dc05 Merge branch 'master' into custom-key-parallel-replicas 2023-01-22 14:20:58 +00:00
Antonio Andelic
3a09415733 Refactor integration test 2023-01-22 14:20:25 +00:00
Vitaly Baranov
5ceb64accc Use new copy s3 functions in S3ObjectStorage. 2023-01-21 15:47:58 +01:00
Vitaly Baranov
f0fda580d0
Merge pull request #45188 from vitlibar/backup-to-s3-memory-optimization 
Optimize memory consumption during backup to S3
 2023-01-21 12:37:35 +01:00
Sema Checherinda
962894afc8
Merge pull request #44909 from CheSema/intersect-prev-part 
Do not merge over a gap with outdated undeleted parts
 2023-01-21 11:51:21 +01:00
Anton Popov
41a199e175
Fix crash when ListObjects request fails (#45371) 2023-01-20 20:10:23 +01:00
Antonio Andelic
2cb106f1f9 Add integration test for custom_key 2023-01-20 12:55:26 +00:00
Maksim Kita
23e26032ca
Merge pull request #45399 from aalexfvk/alexfvk/mdb-21326_fix_system_dictionaries_when_dictionary_with_bad_structure 
Fix select from system.dictionaries when there is dictionary with bad structure
 2023-01-20 13:36:32 +03:00
Antonio Andelic
0ad37ad286
Merge pull request #45320 from stigsb/system_tables_volume_config 
Add <storage_policy> config parameter for system logs
 2023-01-20 10:27:57 +01:00
Aleksandr Musorin
838acb22b7
added num_processed_files and processed_files_size 2023-01-20 10:20:41 +01:00
SmitaRKulkarni
6aa63414db
Merge pull request #45072 from ClickHouse/43891_Disallow_concurrent_backups_and_restores 
Added settings to disallow concurrent backups and restores
 2023-01-20 09:17:20 +01:00
Aleksei Filatov
42549e89f2 [rev 2] Fix review notes 2023-01-20 09:37:49 +03:00
Kseniia Sumarokova
1452d2588c
Update test.py 2023-01-19 18:06:43 +01:00
Kseniia Sumarokova
f80246644c
Update test.py 2023-01-19 18:04:09 +01:00
Kseniia Sumarokova
db49368f5e
Merge branch 'master' into storage-rabbitmq-fix-flush 2023-01-19 18:03:42 +01:00
Aleksei Filatov
afada0ecb3 Fix review notes 2023-01-19 17:02:57 +03:00
alesapin
ccd41e6d14
Merge pull request #45403 from ClickHouse/fix_test_system_merges 
Reduce an amount of trash in `tests_system_merges`
 2023-01-19 12:00:54 +01:00
SmitaRKulkarni
67e2bf31f5
Merge branch 'master' into 43891_Disallow_concurrent_backups_and_restores 2023-01-19 11:21:37 +01:00
Ilya Yatsishin
f16328aa8b
Merge pull request #45402 from qoega/simpler-test_replicated_merge_tree_s3_restore 2023-01-19 10:27:42 +01:00
Stig Bakken
420c179b55 Add <storage_policy> config parameter for system logs 2023-01-19 10:25:28 +01:00
SmitaRKulkarni
db03dd1bb9
Merge branch 'master' into 43891_Disallow_concurrent_backups_and_restores 2023-01-19 09:32:50 +01:00
robot-clickhouse
33a4222155 Automatic style fix 2023-01-18 15:52:38 +00:00
Alexander Tokmakov
6f8c1424df reduce an amount of trash in tests_system_merges 2023-01-18 16:45:47 +01:00
Yatsishin Ilya
1603e734f9 Make test simpler to see errors not in teardown 2023-01-18 15:36:24 +00:00
Aleksei Filatov
5e9340f682 Add integration test 2023-01-18 17:50:38 +03:00
Vitaly Baranov
7cdb2c4c7f
Merge pull request #45351 from vitlibar/fix-backup-with-killed-mutations 
Fix backup with killed mutations
 2023-01-18 13:14:27 +01:00
Ilya Yatsishin
9b5e7c00d5
Merge pull request #45369 from qoega/improve-failed-kafka-logging 
Improve failed kafka startup logging
 2023-01-18 13:09:48 +01:00
flynn
01ae8f584b fix test 2023-01-18 11:34:52 +00:00
alesapin
a4ff9027ce
Merge branch 'master' into fix_flaky_test_ttl_modify 2023-01-18 12:19:37 +01:00
Antonio Andelic
f57ee043ae
Merge pull request #45319 from ClickHouse/disable-prewhere-in-merge-different-types 
Disable PREWHERE in storage Merge if types don't match
 2023-01-18 10:02:06 +01:00
flynn
2fb32dc56c fix and add test 2023-01-18 08:33:55 +00:00
Ilya Yatsishin
a50c6c0abe
Merge branch 'master' into improve-failed-kafka-logging 2023-01-18 02:39:32 +01:00
robot-clickhouse
237bb15a9f Automatic style fix 2023-01-17 21:34:16 +00:00
Smita Kulkarni
6e06af1b25 Updated strategy for handling internal backups & restores to avoid concurrent internal backups & restores - Added settings to disallow concurrent backups and restores 2023-01-17 22:27:13 +01:00
Yatsishin Ilya
5ae72fe711 Improve failed kafka startup logging 2023-01-17 16:25:05 +00:00
alesapin
261dc06be9 Fix flaky ttl test 2023-01-17 17:08:32 +01:00
Antonio Andelic
ba363c141a Fix integration test 2023-01-17 13:12:52 +00:00
Vitaly Baranov
8c5c423d67 Add test. 2023-01-17 13:11:24 +01:00
Sema Checherinda
2cd55bc398 fix/adjust tests in integration/test_partition 2023-01-17 12:02:18 +01:00
Vitaly Baranov
14a7ee8e26 Copy files to S3 during backup directly without using WriteBufferFromS3 to decrease memory consumption. 2023-01-17 09:35:41 +01:00
Anton Popov
6863cd152f
Merge pull request #42181 from CurtizJ/optimize-loading-parts 
Do not load inactive parts at startup
 2023-01-16 14:38:50 +01:00
Kseniia Sumarokova
75318e4cee
Merge pull request #45180 from kssenii/fix-flacky-test-multiple-disks 
Fix flaky test test_multiple_disks/test.py::test_rename
 2023-01-13 19:05:03 +01:00
Alexander Tokmakov
db91c4a749
Merge pull request #45168 from ClickHouse/tavplubix-patch-1 
Try to fix flaky test_ttl_move_memory_usage
 2023-01-13 15:01:42 +03:00
kssenii
0110b63a95 Better fix 2023-01-13 12:12:27 +01:00
Ilya Yatsishin
94e8c57d0d
Merge pull request #45094 from ClickHouse/try-fix-keeper-map 
Try fixing KeeperMap tests
 2023-01-13 00:46:43 +01:00
Yakov Olkhovskiy
842a34666c
Merge pull request #44594 from arenadata/ADQM-634 
Add keytab parameter in kerberos section of config.xml
 2023-01-12 15:07:45 -05:00
Nikolai Kochetov
e21fe952c6
Update test_ttl_move_memory_usage.py 2023-01-12 18:45:19 +01:00
Nikolai Kochetov
1911c25a7b
Merge branch 'master' into tavplubix-patch-1 2023-01-12 18:44:39 +01:00
Antonio Andelic
1f2ec534f4 Add EXISTS clauses 2023-01-12 15:39:04 +00:00
Antonio Andelic
dc4262f338 Merge branch 'master' into try-fix-keeper-map 2023-01-12 15:19:51 +00:00
Antonio Andelic
fb23a399b5
Merge pull request #45213 from ClickHouse/fix-flaky-keeper-test 
Fix flaky `test_keeper_three_nodes_two_alive`
 2023-01-12 14:57:40 +01:00
Yakov Olkhovskiy
344fa16184
Merge pull request #42991 from ClickHouse/tests-connection-error 
add retries on ConnectionError
 2023-01-12 08:32:03 -05:00
Ilya Yatsishin
23c80d77a0
Merge pull request #45206 from ClickHouse/try-fix-overcommit-test 
Disable check to make test_overcommit_tracker not flaky
 2023-01-12 14:16:11 +01:00
alesapin
df8ceeefdc
Merge pull request #45134 from ClickHouse/fix_flaky_azure_test 
Fix flaky azure test
 2023-01-12 13:02:34 +01:00
Nikita Mikhaylov
62d544e742
Fix integration test test_replicated_users::test_rename_replicated (#45192) 
* Done

* Update tests/integration/test_replicated_users/test.py

Co-authored-by: Sergei Trifonov <sergei@clickhouse.com>

Co-authored-by: Sergei Trifonov <sergei@clickhouse.com>
 2023-01-12 12:36:36 +01:00
robot-clickhouse
d61e202416 Automatic style fix 2023-01-12 11:35:03 +00:00
Antonio Andelic
feaad2bd43 try avoiding bad file descriptor 2023-01-12 11:28:41 +00:00
Antonio Andelic
97266ccabc Merge branch 'master' into try-fix-keeper-map 2023-01-12 11:16:35 +00:00
Kseniia Sumarokova
56f8899df8
Update tests/integration/test_multiple_disks/test.py 
Co-authored-by: Ilya Yatsishin <2159081+qoega@users.noreply.github.com>
 2023-01-12 12:06:31 +01:00
Antonio Andelic
0fc279e2bb Don't allow third node to become leader 2023-01-12 08:44:55 +00:00
Dmitry Novik
bed39f3762 Disable check to make test on flaky 2023-01-11 23:52:13 +00:00
kssenii
e3a1af6542 Fix flacky test 2023-01-11 23:51:03 +01:00
Ilya Yatsishin
bca117b26d
Merge pull request #45099 from ClickHouse/qoega-patch-4 
Try to fix flaky test_create_user_and_login/test.py::test_login_as_dropped_user_xml
 2023-01-11 23:37:03 +01:00
Nikolai Kochetov
653d18cb8c
Merge pull request #45162 from ClickHouse/disable-test_move_and_s3_memory_usage 
Disable test_ttl_move_memory_usage as too flaky.
 2023-01-11 20:30:28 +01:00
Ilya Yatsishin
cc352804df
Merge pull request #45156 from qoega/green-ci-5 
Fix flaky test_tcp_handler_interserver_listen_host
 2023-01-11 19:09:22 +01:00
Antonio Andelic
2c33c9e62c
Merge branch 'master' into try-fix-keeper-map 2023-01-11 15:05:04 +01:00
Alexander Tokmakov
764abb6410
try to fix flaky test_ttl_move_memory_usage 2023-01-11 15:55:18 +03:00
Nikolai Kochetov
07a3967d6b Disable test_ttl_move_memory_usage as too flaky. 2023-01-11 11:20:02 +00:00
Alexander Tokmakov
b156209ed6
Update test.py 2023-01-11 14:17:44 +03:00
Yatsishin Ilya
77f0724629 Fix flaky test_tcp_handler_interserver_listen_host 2023-01-11 10:28:30 +00:00
robot-clickhouse
6d6e803cfb Automatic style fix 2023-01-11 10:09:43 +01:00
Ilya Yatsishin
f871949d85 Try to fix flaky test_create_user_and_login/test.py::test_login_as_dropped_user_xml 2023-01-11 10:09:43 +01:00
Anton Popov
6c4f53bdca Merge remote-tracking branch 'upstream/master' into HEAD 2023-01-11 01:00:57 +00:00
alesapin
6d86b8dd47 Fix flaky azure test 2023-01-10 19:05:02 +01:00
Antonio Andelic
7d948c9c77
Merge branch 'master' into try-fix-keeper-map 2023-01-10 15:44:28 +01:00
Antonio Andelic
134cc3e273
Change table name 2023-01-10 14:16:28 +01:00
Maksim Kita
d4a2dd43ce
Merge pull request #44891 from azat/tests/rewrite-update_field-test 
Convert integration test_dictionaries_update_field to a stateless
 2023-01-10 15:14:53 +03:00
Antonio Andelic
d7ca0c04fa
remove drop if exists 2023-01-10 11:35:02 +01:00
Antonio Andelic
7aef7d95de Small fixes for keeper_map tests 2023-01-10 09:15:07 +00:00
robot-clickhouse
46b21629ed Automatic style fix 2023-01-09 17:23:02 +00:00
Smita Kulkarni
93530e8d34 Added settings to disallow concurrent backups and restores 
Implementation:
* Added server level settings to disallow concurrent backups and restores, which are read and set when BackupWorker is created in Context.
* Settings are set to true by default.
* Before starting backup or restores, added a check to see if any other backups/restores are running (except internal ones).
Testing:
* Added a test test_backup_and_restore_on_cluster/test_disallow_concurrency.
 2023-01-09 18:14:39 +01:00
Nikita Mikhaylov
857799fbca
Parallel distributed insert select with s3Cluster [3] (#44955) 
* Revert "Revert "Resurrect parallel distributed insert select with s3Cluster (#41535)""

This reverts commit b8d9066004.

* Fix build

* Better

* Fix test

* Automatic style fix

Co-authored-by: robot-clickhouse <robot-clickhouse@users.noreply.github.com>
 2023-01-09 13:30:32 +01:00
Kseniia Sumarokova
9ae5ad202a
Merge pull request #44707 from xiedeyantu/s3-add-config 
S3 support upload object using storage class INTELLIGENT_TIERING
 2023-01-09 13:28:53 +01:00
Roman Vasin
2995c6d866 Merge branch 'master' of github.com:ClickHouse/ClickHouse into ADQM-634 2023-01-09 08:26:45 +00:00
Alexey Milovidov
851e4ae59f Fix typos 2023-01-08 07:08:20 +01:00
Antonio Andelic
c8d9368009 Retry on invalid snapshot 2023-01-07 17:08:25 +00:00
Anton Popov
1f32ffedf8
Merge pull request #43221 from ClickHouse/refactoring-ip-types 
Replace domain IP types (IPv4, IPv6) with native
 2023-01-07 12:01:21 +01:00
Alexey Milovidov
590ad41cbf
Merge pull request #44978 from azat/tests/fix-test_replicated_users 
tests: fix test_replicated_users flakiness
 2023-01-07 03:51:31 +03:00
Alexey Milovidov
d122d72ea8
Merge pull request #44996 from vitlibar/fix-test_grpc_protocol_progress 
Fix test test_grpc_protocol/test.py::test_progress
 2023-01-07 03:48:30 +03:00
Vitaly Baranov
e8355971bb Fix test test_grpc_protocol/test.py::test_progress 2023-01-06 19:58:20 +01:00
kssenii
dd6a2c2d37 Fix style check 2023-01-06 16:53:24 +01:00
kssenii
2a030c1dc0 Merge remote-tracking branch 'upstream/master' into allow-single-disk-instead-of-storage-policy 2023-01-06 15:58:27 +01:00
Vitaly Baranov
92be50bd68 Add test for IMDS session token. 2023-01-06 15:55:26 +01:00
Vitaly Baranov
0d12d90fb5 Move common code to helpers/mock_servers.py 2023-01-06 15:55:21 +01:00
kssenii
c980e16345 Merge remote-tracking branch 'upstream/master' into storage-rabbitmq-fix-flush 2023-01-06 15:51:25 +01:00
Anton Popov
b25f875674
Merge pull request #44875 from ClickHouse/fix-another-one-cannot-read-all-data-for-lc-dict-error 
Fix right offset for reading LowCardinality dictionary from remote fs
 2023-01-06 15:24:36 +01:00
Kseniia Sumarokova
573d3283b0
Merge pull request #44327 from kssenii/use-new-named-collections-code-2 
Replace old named collections code with new (from #43147) part 2
 2023-01-06 13:06:26 +01:00
Nikolai Kochetov
12192ef099
Merge pull request #44938 from ClickHouse/try-fix-test_multiple_disks-test_freeze 
Stop merges to avoid a race between merge and freeze.
 2023-01-06 12:45:10 +01:00
Azat Khuzhin
7859346e40 tests: fix test_replicated_users flakiness 
Alter for users/quotas/... does not wait until all replicas will be up
to date, so use *_with_retries() helperx.

CI: https://s3.amazonaws.com/clickhouse-test-reports/44922/bd885be9229cf47752c5c98392f09129261550f9/integration_tests__tsan__[6/6].html
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-01-06 12:45:03 +01:00
Azat Khuzhin
c03510cf6b tests/integration: add query_and_get_error_with_retry() helper 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-01-06 12:45:03 +01:00
Alexey Milovidov
08ebdc9b82
Merge pull request #44903 from ClickHouse/less-flaky-test-concurrent-queries 
Fix flaky `test_concurrent_queries_restriction_by_query_kind`
 2023-01-06 03:18:28 +03:00
robot-clickhouse
62a9b33bbb Automatic style fix 2023-01-05 16:29:58 +00:00
Ilya Yatsishin
a28d6fb490
Merge pull request #44848 from qoega/green-ci-2 
Pre-pulling images for integration tests to detect broken infrastructure earlier
 2023-01-05 17:29:39 +01:00
Nikolai Kochetov
90f3f5789f Speedup a bit. 2023-01-05 16:23:49 +00:00
Nikolai Kochetov
dc5009d401 Stop merges to aviod a race between merge and freeze. 2023-01-05 15:28:56 +00:00
Anton Popov
ed00399eb6 Merge remote-tracking branch 'upstream/master' into HEAD 2023-01-05 14:12:02 +00:00
Yatsishin Ilya
1b03a60e43 Do not report status if we failed to pull images. 2023-01-05 13:53:45 +00:00
Ilya Yatsishin
c5f01a653c
Merge pull request #44880 from qoega/green-ci-4 
Rewrite test_postgres_protocol test
 2023-01-05 12:34:33 +01:00
Antonio Andelic
7dd8ac717a
Merge branch 'master' into less-flaky-test-concurrent-queries 2023-01-05 12:23:30 +01:00
Ilya Yatsishin
5d0f3e7819
Merge pull request #44921 from ClickHouse/fix-flaky-test-5 
Fix flaky test `test_lost_part`
 2023-01-05 10:28:07 +01:00
xiedeyantu
dfaa16b7a0 fix 2023-01-05 09:09:42 +08:00
xiedeyantu
110f95bae6 fix 2023-01-05 09:00:53 +08:00
Nikita Mikhaylov
e3ca9fb3e9 Blakc 2023-01-04 20:30:52 +00:00
Nikita Mikhaylov
ea7c338a7b Done 2023-01-04 20:28:03 +00:00
Yatsishin Ilya
ccb6226d22 install CH before test 2023-01-04 20:07:52 +00:00
Yakov Olkhovskiy
dadd96ba6f
Merge pull request #44135 from ClickHouse/feature-protocol-proxy 
Improve support for PROXYv1 protocol.
 2023-01-04 14:06:17 -05:00
Yatsishin Ilya
daf2ccab81 Merge remote-tracking branch 'origin/master' into green-ci-2 2023-01-04 18:27:19 +00:00
xiedeyantu
2f59b1d002 better 2023-01-05 00:27:07 +08:00
Yakov Olkhovskiy
7a5a36cbed
Merge branch 'master' into refactoring-ip-types 2023-01-04 11:11:06 -05:00
Antonio Andelic
8ee589c5c1
Merge pull request #44898 from ClickHouse/try-fix-test-storage-kafka 
Try to fix flaky `test_storage_kafka::test_kafka_produce_key_timestamp`
 2023-01-04 16:54:47 +01:00
Yatsishin Ilya
03582f4b87 move logic to runner 2023-01-04 15:52:34 +00:00
Yatsishin Ilya
9cf599e58b Merge remote-tracking branch 'origin/master' into green-ci-2 2023-01-04 15:49:22 +00:00
Yatsishin Ilya
d8d012523d add pre-pull to runner 2023-01-04 15:47:50 +00:00
alesapin
92255e5d4b
Merge pull request #43020 from vitlibar/update-aws-c-submodules 
Update aws-c* submodules
 2023-01-04 16:25:22 +01:00
Antonio Andelic
1a1afc6a74 Make test_concurrent_queries_restriction_by_query_kind less flaky 2023-01-04 14:18:35 +00:00
Nikita Mikhaylov
e9c7555365
Use TablesDependencyGraph in DatabaseReplicated recovery process (#44697) 2023-01-04 14:49:39 +01:00
Yatsishin Ilya
4d5131ed7d debug quotes 2023-01-04 13:16:31 +00:00
Antonio Andelic
890f094829 Set retention policy for Kafka topic 2023-01-04 12:04:00 +00:00
Yatsishin Ilya
08fbf6e5b2 again 2023-01-04 11:48:28 +00:00
alesapin
c09f83654d
Merge pull request #44533 from ClickHouse/40951_Allow_macro_substitution_in_endpoint 
Allow macro substitution in endpoint of disks
 2023-01-04 12:29:57 +01:00
Azat Khuzhin
0ab0f76737 Convert integration test_dictionaries_update_field to a stateless 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-01-04 11:05:00 +01:00
Antonio Andelic
9b58c97563 Store ZK generated data in archive 2023-01-04 09:10:16 +00:00
Alexey Milovidov
66e6f53698
Merge pull request #44839 from ClickHouse/move-odbc-test 
Move `test_dies_with_parent` to another module
 2023-01-04 05:17:19 +03:00
Alexey Milovidov
3a027d285f
Merge pull request #44828 from ClickHouse/remove-two-lines-of-code 
What if I will remove two lines of code?
 2023-01-04 04:50:52 +03:00
Ilya Yatsishin
49155d9614
Merge pull request #44711 from qoega/green-ci-1 
Improve some integration tests and add pytest-random
 2023-01-03 22:39:54 +01:00
Yatsishin Ilya
bb288c1f14 how many more? 2023-01-03 21:38:59 +00:00
Yatsishin Ilya
bbc02c0762 black 2023-01-03 21:28:54 +00:00
Yatsishin Ilya
14274d650b Rewrite test_postgres_protocol test 2023-01-03 19:30:14 +00:00
Yatsishin Ilya
683df2dea5 fix 2023-01-03 18:56:24 +00:00
Robert Schulze
27f5aad49e
What happens if I remove 156 lines of code? 2023-01-03 18:51:16 +00:00
robot-clickhouse
d6c090b530 Automatic style fix 2023-01-03 18:29:09 +00:00
Nikolai Kochetov
da26f62a9b Fix right offset for reading LowCardinality dictionary from remote fs in case if right mark was in the middle of compressed block. 2023-01-03 18:19:51 +00:00
Alexey Milovidov
a06d03e097
Merge pull request #44851 from qoega/green-ci-3 
No more retries in integration tests
 2023-01-03 19:40:34 +03:00
Yatsishin Ilya
3deada0c29 fix 2023-01-03 16:05:20 +00:00
kssenii
67509aa2d5 Merge remote-tracking branch 'upstream/master' into use-new-named-collections-code-2 2023-01-03 16:41:30 +01:00
Antonio Andelic
a88171b594 Add timeout for ZK stop 2023-01-03 09:59:06 +00:00
Antonio Andelic
4ccd828a44 Wait for ZK process to stop 2023-01-03 09:39:14 +00:00
Nikita Mikhaylov
b34fc8650c
Merge branch 'master' into move-odbc-test 2023-01-02 21:22:50 +01:00
SmitaRKulkarni
16dd6ca02f
Merge branch 'master' into 40951_Allow_macro_substitution_in_endpoint 2023-01-02 20:34:24 +01:00
Alexey Milovidov
e855d3519a
Merge branch 'master' into refactoring-ip-types 2023-01-02 21:58:53 +03:00
Ilya Yatsishin
8a9bb9b107 No more retries in integration tests 2023-01-02 18:56:24 +01:00
Yatsishin Ilya
3251d9278c black 2023-01-02 17:44:04 +00:00
Yatsishin Ilya
9ab5cf7d62 Download inside runner 2023-01-02 16:46:24 +00:00
Antonio Andelic
509fd873ec
Try restarting zookeeper if connection failed (#44834) 2023-01-02 16:24:57 +01:00
Yatsishin Ilya
b40f6c0b10 Pre-pull docker image and fail check completely if it failed 2023-01-02 15:04:14 +00:00
Kruglov Pavel
966f57ef68
Merge pull request #42777 from Avogar/improve-streaming-engines 
Refactor and Improve streaming engines Kafka/RabbitMQ/NATS and data formats
 2023-01-02 15:59:06 +01:00
Alexey Milovidov
17e792ca0b
Merge pull request #44825 from azat/ci/integration-fixes 
Fix misleading integration tests reports for parametrized tests
 2023-01-02 17:49:57 +03:00
robot-clickhouse
675fe5cd91 Automatic style fix 2023-01-02 13:38:18 +00:00
Nikita Mikhaylov
ec5e8ee9cf Done 2023-01-02 13:31:19 +00:00
Smita Kulkarni
5592d30798 Updated to skip access check for hdfs , as done for other tests - Allow macro substitution in endpoint 2023-01-02 14:03:51 +01:00
Yatsishin Ilya
54e2df724f fix test 2023-01-02 12:43:16 +01:00
Yatsishin Ilya
6ec3bf40b1 black 2023-01-02 12:43:16 +01:00
Yatsishin Ilya
f779fe3c0a Improve some integration tests 2023-01-02 12:43:16 +01:00
Azat Khuzhin
c3b3ae886f tests/integration: ignore exit code for a running group 
tee is useful there, otherwise logs are empty on CI

This reverts commit 92fd7def39.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-01-02 12:08:44 +01:00
Azat Khuzhin
f2c069b852 tests/integration: allow to split parametrized tests between parallel groups 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-01-02 12:06:56 +01:00
Smita Kulkarni
2f73bf48b8 Added config to integration test test_endpoint_macro_substitution - Allow macro substitution in endpoint 2023-01-02 11:43:19 +01:00
Alexey Milovidov
b89513e789
Merge pull request #44823 from azat/tests/fix-flaky-test_multiple_disks 
Fix flaky test_multiple_disks::test_jbod_overflow
 2023-01-02 05:25:27 +03:00
Azat Khuzhin
4b71b481c7 tests/integration: do not split paramaterized tests into multiple groups 
There is also one assumption the runner already, that leads to
misleading reports like in [1].

  [1]: https://s3.amazonaws.com/clickhouse-test-reports/44762/89c071e291980e355f2c2be42dc15047caabc9bc/integration_tests__asan__%5B4/6%5D.html

Here the test test_multiple_disks/test.py::test_jbod_overflow had been
splitted into multiple groups, however because of that one assumption
it had been runned as a whole in both groups and in one it failed but
succeeded in another.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-01-01 23:35:21 +01:00
Azat Khuzhin
700943a2b0 tests/integration: remove logging message for undone thing 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-01-01 23:13:00 +01:00
Azat Khuzhin
fa78569c39 tests/integration: fix logging message 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-01-01 23:12:25 +01:00
Azat Khuzhin
92fd7def39 tests/integration: fix detecting exit code for a running group 
Because of tee the exit code is the result of tee, not of the pytest.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-01-01 23:09:12 +01:00
Azat Khuzhin
3123761980 Fix flaky test_multiple_disks::test_jbod_overflow 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-01-01 22:21:24 +01:00
Alexey Milovidov
f5fc9b2825 Fix tests for bridges 2023-01-01 22:00:08 +01:00
Vitaly Baranov
426e2a9da1 Add test for getting credentials from AWS EC2 metadata. 2022-12-31 18:56:40 +01:00
alesapin
4948a8c17b
Remove disk restart proxy and disk decorator (#44647) 
* Remove disk restart proxy and disk decorator
* Automatic style fix
* Returned some trash back
* Fix build again
* Fix failing test

Co-authored-by: robot-clickhouse <robot-clickhouse@users.noreply.github.com>
 2022-12-30 14:47:30 +01:00
SmitaRKulkarni
f9a3aeb172
Merge branch 'master' into 40951_Allow_macro_substitution_in_endpoint 2022-12-30 10:43:14 +01:00
Kruglov Pavel
894726bd8f
Merge branch 'master' into improve-streaming-engines 2022-12-29 22:59:45 +01:00
robot-clickhouse
c09d1bc758 Automatic style fix 2022-12-29 15:16:50 +00:00
Smita Kulkarni
8cc3b2e26c Fixed integration flaky test issue by specifying hdfs client in each test- Allow macro substitution in endpoint 2022-12-29 16:10:06 +01:00
Nikita Mikhaylov
ffaff7167b
Fix flaky tests (#44690) 2022-12-29 14:32:10 +01:00
Alexey Milovidov
9378194320
Merge pull request #44688 from ClickHouse/improve-odbc-test 
Improve odbc test
 2022-12-29 15:40:47 +03:00
Azat Khuzhin
90b707d130 tests: more fixes for test_keeper_auth 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-12-29 09:29:24 +01:00
Alexey Milovidov
f7b2b961f7
Merge pull request #44667 from azat/tests/integration/dmesg-fix 
Move dmesg dumping out from runner to ci-runner.py
 2022-12-29 01:56:34 +03:00
Alexey Milovidov
23ea273f73 Edit another test 2022-12-28 19:49:57 +01:00
Alexey Milovidov
edd78da0cb ODBC test: avoid poisoning other tests 2022-12-28 19:45:01 +01:00
Alexey Milovidov
0263499b0c
Merge pull request #44666 from azat/tests/integration/kazoo-stop 
tests/integration: add missing kazoo client termination
 2022-12-28 21:20:14 +03:00
Anton Popov
bc122d368a Merge remote-tracking branch 'upstream/master' into HEAD 2022-12-28 17:45:04 +00:00
Azat Khuzhin
03f80bae90 Move dmesg dumping out from runner to ci-runner.py 
runner is used by developers to run tests, while ci-runner.py is used
only by CI scripts, and to avoid requiring CAP_SYSLOG for manual dmesg
clear/dump had been moved.

Also for manual runs this can be done manually.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-12-28 15:47:46 +01:00
Vitaly Baranov
8a2fbbe88c
Merge pull request #43947 from azat/backups/dedup 
RFC: Add ability to disable deduplication for BACKUP
 2022-12-28 15:24:53 +01:00
Alexey Milovidov
1b4121459d
Merge pull request #44645 from ClickHouse/keeper-test-fix 
Fix flaky test `test_keeper_multinode_simple`
 2022-12-28 16:38:17 +03:00
Azat Khuzhin
bc6adc88c8 tests/integration: add workaround for a bug in kazoo driver 
Refs: python-zk/kazoo#688
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-12-28 10:52:49 +01:00
Vitaly Baranov
b211dff013 Increase the create table timeout in test "test_backup_restore_on_cluster". 2022-12-28 00:17:19 +01:00
Nikita Mikhaylov
00ecf6dd05 Better 2022-12-27 17:04:47 +00:00
Nikita Mikhaylov
ce6264b684 Done 2022-12-27 17:03:57 +00:00
Azat Khuzhin
e1ad5cee7e tests/integration: add missing kazoo client termination 
pytest play games with logging output [1].

  [1]: https://github.com/pytest-dev/pytest/issues/5502

But this does not actually affect ClickHouse tests, the only reason is
that the kazoo client is not stopped correctly without calling
kazoo.client.KazooClient.stop(), and that's why you can see the
following messages:

<details>

    test_storage_rabbitmq/test.py::test_rabbitmq_big_message --- Logging error ---
    Traceback (most recent call last):
      File "/usr/local/lib/python3.8/dist-packages/kazoo/protocol/connection.py", line 179, in _socket_error_handling
        yield
      File "/usr/local/lib/python3.8/dist-packages/kazoo/protocol/connection.py", line 659, in _connect
        self._socket = self.handler.create_connection(
      File "/usr/local/lib/python3.8/dist-packages/kazoo/handlers/threading.py", line 178, in create_connection
        return utils.create_tcp_connection(socket, *args, **kwargs)
      File "/usr/local/lib/python3.8/dist-packages/kazoo/handlers/utils.py", line 265, in create_tcp_connection
        sock = module.create_connection(address, timeout_at)
      File "/usr/lib/python3.8/socket.py", line 808, in create_connection
        raise err
      File "/usr/lib/python3.8/socket.py", line 796, in create_connection
        sock.connect(sa)
    socket.timeout: timed out

    During handling of the above exception, another exception occurred:

    Traceback (most recent call last):
      File "/usr/local/lib/python3.8/dist-packages/kazoo/protocol/connection.py", line 579, in _connect_attempt
        read_timeout, connect_timeout = self._connect(host, hostip, port)
      File "/usr/local/lib/python3.8/dist-packages/kazoo/protocol/connection.py", line 659, in _connect
        self._socket = self.handler.create_connection(
      File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__
        self.gen.throw(type, value, traceback)
      File "/usr/local/lib/python3.8/dist-packages/kazoo/protocol/connection.py", line 182, in _socket_error_handling
        raise ConnectionDropped("socket connection error: %s" % (err,))
    kazoo.exceptions.ConnectionDropped: socket connection error: None

    During handling of the above exception, another exception occurred:

    Traceback (most recent call last):
      File "/usr/lib/python3.8/logging/__init__.py", line 1088, in emit
        stream.write(msg + self.terminator)
    ValueError: I/O operation on closed file.
    Call stack:
      File "/usr/lib/python3.8/threading.py", line 890, in _bootstrap
        self._bootstrap_inner()
      File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
        self.run()
      File "/usr/lib/python3.8/threading.py", line 870, in run
        self._target(*self._args, **self._kwargs)
      File "/usr/local/lib/python3.8/dist-packages/kazoo/protocol/connection.py", line 512, in zk_loop
        if retry(self._connect_loop, retry) is STOP_CONNECTING:
      File "/usr/local/lib/python3.8/dist-packages/kazoo/retry.py", line 126, in __call__
        return func(*args, **kwargs)
      File "/usr/local/lib/python3.8/dist-packages/kazoo/protocol/connection.py", line 552, in _connect_loop
        status = self._connect_attempt(host, hostip, port, retry)
      File "/usr/local/lib/python3.8/dist-packages/kazoo/protocol/connection.py", line 622, in _connect_attempt
        self.logger.warning('Connection dropped: %s', e)
    Message: 'Connection dropped: %s'
    Arguments: (ConnectionDropped('socket connection error: None'),)

</details>

Which eventually leads to incorrectly parsed report.

This patch is an addition to the `logging.raiseExceptions=False` - #44618

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-12-27 17:01:06 +01:00
Kruglov Pavel
c3405be272
Increase the number of attempts in tests 2022-12-27 16:49:32 +01:00
Azat Khuzhin
c9c590071d Add ability to disable deduplication for BACKUP 
Right now BACKUP omit similar files, and will not allow to use this
backup as a regular table, and usually those similar files are quite
small (i.e. columns.txt).

So by using `BACKUP TO S3() deduplicate_files=0` you will be possible to
use `ATTACH TABLE` directly from S3.

P.S. right now it is possible only for the table with one part, since,
usually, there is nothing to deduplicate (if the columns are different).

v2: Add deduplicate_files into metadata
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-12-27 15:56:06 +01:00
Alexey Milovidov
7e447592b1
Merge pull request #44618 from azat/tests/integration/pytest-logging-fix 
tests/integration: suppress exceptions during logging (due to pytest)
 2022-12-27 14:33:37 +03:00
Alexey Milovidov
79d43c836a
Merge pull request #44572 from ClickHouse/fix-hdfs-test 
Fix HDFS test
 2022-12-27 14:31:00 +03:00
Alexey Milovidov
41fc65b47b
Merge pull request #44573 from ClickHouse/fix-distributed-queries-stress 
Fix test_distributed_queries_stress
 2022-12-27 14:30:28 +03:00
Azat Khuzhin
8b2ed0e3cd tests/integration: disable logging.raiseExceptions 
Due to pytest play games with logging [1] it is better to ignore it.

  [1]: https://github.com/pytest-dev/pytest/issues/5502

This is due to incorrect kazoo client usage (not all clients calls
stop())

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-12-26 22:57:54 +01:00
Alexey Milovidov
a92a9b4d55
Merge pull request #44535 from azat/tests/integration-dmesg 
tests: capture dmesg in integration tests
 2022-12-26 23:49:02 +03:00
Roman Vasin
465cc36526 Update Exception messages in test.py 2022-12-26 13:17:22 +00:00
Roman Vasin
483e26cf2f
Merge branch 'master' into ADQM-634 2022-12-26 15:34:38 +03:00
Roman Vasin
3cdc9b3f81 Remove -vvv from kinit call in tests 2022-12-26 11:59:37 +00:00
Roman Vasin
a70e3c20fa Make calls of test from Ubuntu client 2022-12-26 11:56:45 +00:00
Alexey Milovidov
a06c8268ea Remove the offender 2022-12-26 11:28:26 +01:00
Alexey Milovidov
f9f0f9446d Fix distributed_queries_stress 2022-12-25 09:46:51 +01:00
robot-clickhouse
19f05d4fb4 Automatic style fix 2022-12-25 08:38:02 +00:00
Alexey Milovidov
cbf140633b Fix HDFS test 2022-12-25 09:30:20 +01:00
Anton Popov
0722020cae Merge remote-tracking branch 'upstream/master' into HEAD 2022-12-23 18:15:55 +00:00
Smita Kulkarni
84362461fb Updated test to use macro substitution for hdfs disk - Allow macro substitution in endpoint 2022-12-23 16:07:56 +01:00
Azat Khuzhin
69fadd2193 tests: capture dmesg in integration tests 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-12-23 15:50:10 +01:00
Smita Kulkarni
3d62bc2635 Allow macro substitution in endpoint of disks 
Implementation:
* Updated to substitute macros in endpoint for all disks (s3, hfs, web).
Testing:
* Added an integration test test/integration/test_endpoint_macro_substitution.
 2022-12-23 15:09:24 +01:00
Mikhail f. Shiryaev
6d0cd35cd0
Ignore exit code 1 for tar in integration tests 2022-12-23 13:08:59 +01:00
kssenii
8308177361 Merge remote-tracking branch 'upstream/master' into use-new-named-collections-code-2 2022-12-23 12:14:46 +01:00
Vladimir C
7482ea54ab
Merge pull request #43972 from ClickHouse/vdimir/tmp-data-in-fs-cache-2 2022-12-23 11:59:27 +01:00
kssenii
853f2ea123 Merge remote-tracking branch 'upstream/master' into use-new-named-collections-code-2 2022-12-23 11:49:02 +01:00
kssenii
8f1145f6e4 Fix tests 2022-12-23 11:47:51 +01:00
kssenii
11c0aa189a Better error message 2022-12-22 15:29:35 +01:00
Nikolai Kochetov
773de7a8a6
Merge pull request #44472 from CurtizJ/fix-mutations 
Fix mutations with setting `max_streams_for_merge_tree_reading`
 2022-12-22 14:05:20 +01:00
kssenii
a58b8b8e6c Fix tests 2022-12-22 12:27:42 +01:00
kssenii
ab458e68cd Fix tests 2022-12-22 12:24:20 +01:00
vdimir
182b34c11e
Fixes 2022-12-22 10:22:57 +00:00
vdimir
511e8d698d
Rename config param tmp_cache -> temporary_data_in_cache 2022-12-22 10:22:56 +00:00
vdimir
d30d25dbbe
Temporary files evict fs cache 2022-12-22 10:22:49 +00:00
Roman Vasin
2d9f3c242b Add test_bad_path_to_keytab test 2022-12-22 08:07:16 +00:00
Yakov Olkhovskiy
cea5b92536 binary backward compatibility test is added 2022-12-22 07:38:49 +00:00
kssenii
d3db1dd6a7 Fix tests 2022-12-22 00:27:22 +01:00
Anton Popov
5d6064cd28 better config in test 2022-12-21 11:35:11 +00:00
Kruglov Pavel
5e01a3d74e
Merge branch 'master' into improve-streaming-engines 2022-12-21 10:51:50 +01:00
Anton Popov
4b7f1e0cf2 fix mutations with setting 'max_streams_for_merge_tree_reading' 2022-12-21 01:59:53 +00:00
kssenii
1d75f740d7 Fix tests 2022-12-20 22:33:54 +01:00
kssenii
499565a604 Fix tests 2022-12-20 20:02:18 +01:00
Yakov Olkhovskiy
8462b724ca
Merge branch 'master' into feature-protocol-proxy 2022-12-20 13:12:07 -05:00
robot-clickhouse
82203e629c Automatic style fix 2022-12-20 17:46:17 +00:00
Yakov Olkhovskiy
9092134544 auth_use_forwarded_address config setting added, using forwarded address for proxied TCP and HTTP connection, likely bug is fixed in AccessControl, relevant test is appended 2022-12-20 17:39:40 +00:00
alesapin
80cae1aa3d Better 2022-12-20 17:37:03 +01:00
alesapin
33389f01e6 Disable tsan assertion for test 2022-12-20 17:33:08 +01:00
Konstantin Bogdanov
93375fc3e5
Merge pull request #44411 from thevar1able/fix-settings-constraints 
Fix settings constraints
 2022-12-20 10:19:07 +01:00
Konstantin Bogdanov
a094af9a49
Lint 2022-12-20 04:00:45 +01:00
Konstantin Bogdanov
b07001e7d7
Add a test 2022-12-20 03:26:31 +01:00
kssenii
a0c8d34775 RabbitMQ: respect flushing data by flush interval and by block size 2022-12-19 21:07:44 +01:00
Roman Vasin
8a692432de Remove unneeded headers from test.py 2022-12-19 11:48:10 +00:00
Roman Vasin
c26c257c57 Refactor test.py 2022-12-19 11:39:30 +00:00
Roman Vasin
64c0c6ae99 Refactor test_kerberos_auth test 2022-12-19 11:28:49 +00:00
Antonio Andelic
55dd591392 Try restarting ZK cluster on failed connection 2022-12-19 09:28:59 +00:00
Antonio Andelic
0162632275
Merge pull request #44306 from ClickHouse/another-fix-for-test-server-reload 
Another fix `test_server_reload`
 2022-12-19 09:14:03 +01:00
Nikolay Degterinsky
7a29bf0e4c
Merge pull request #43993 from evillique/fix_settings_constraints 
Fix CREATE query constraints
 2022-12-16 18:01:04 +01:00
Anton Popov
8b9b8b083c
Merge pull request #43726 from CurtizJ/optimize-storage-s3 
Improve performance of storage `S3` with large number of small files
 2022-12-16 14:38:10 +01:00
Antonio Andelic
372c187773 Correctly compare get result 2022-12-16 09:48:47 +00:00
Roman Vasin
775c700c24 Add second test for the case when there is no keytab 2022-12-16 08:20:01 +00:00
Nikolay Degterinsky
1861e670e9
Merge branch 'master' into fix_settings_constraints 2022-12-15 21:54:02 +01:00
Kruglov Pavel
c5b2e4cc23
Merge branch 'master' into improve-streaming-engines 2022-12-15 18:44:35 +01:00
Roman Vasin
5491aa9627 Implement wait_kerberos_kdc_is_available 2022-12-15 11:50:23 +00:00
Anton Popov
c4bcfe54cd fix test 2022-12-15 04:19:16 +00:00
Vitaly Baranov
fb8aca8319
Merge pull request #44158 from vitlibar/improve-referential-deps 
Improve referential dependencies
 2022-12-14 21:17:02 +01:00
Roman Vasin
e2ced517dd Add integration test test_kerberos_auth 2022-12-14 13:39:23 +00:00
kssenii
ea5b06023a Merge remote-tracking branch 'refs/remotes/upstream/master' into allow-single-disk-instead-of-storage-policy 2022-12-14 13:59:51 +01:00
Sema Checherinda
fdecb85dc4
Merge pull request #44156 from CheSema/fix_flaky_test_attach_detach 
make calls to be sure that parts are deleted
 2022-12-14 12:39:39 +01:00
Antonio Andelic
b2cce09004
Merge pull request #44184 from ClickHouse/close-grpc-channel-in-tests 
Close GRPC channels in tests
 2022-12-14 09:47:36 +01:00
Anton Popov
3008e701ea fix tests 2022-12-14 01:36:03 +00:00
Anton Popov
cce3257f39
Merge branch 'master' into optimize-storage-s3 2022-12-13 21:35:12 +01:00
Nikolay Degterinsky
809f15ac81
Merge branch 'master' into fix_settings_constraints 2022-12-13 20:33:16 +01:00
Nikolay Degterinsky
443bc3e656
Merge pull request #43719 from evillique/password_complexity_rules 
Add password complexity rules
 2022-12-13 17:56:45 +01:00
Anton Popov
0c87031e80 Merge remote-tracking branch 'upstream/master' into HEAD 2022-12-13 16:33:21 +00:00
Mikhail f. Shiryaev
9879d03bc4
Use contextmanager for grpc_channels 2022-12-13 12:43:48 +01:00
Antonio Andelic
b8f76ce0e5 Close GRPC channels in tests 2022-12-13 10:25:54 +00:00
Vitaly Baranov
5c1f490b3a Implement referential dependencies for table engine "Distributed" 
and for functions cluster() and clusterAllReplicas().
 2022-12-13 11:09:42 +01:00
Nikolay Degterinsky
f352403af2 Fix style 2022-12-13 09:41:31 +00:00
Nikolay Degterinsky
034269087f Fix tests 2022-12-13 09:16:23 +00:00
Anton Popov
0b396bab14 Merge remote-tracking branch 'upstream/master' into HEAD 2022-12-12 18:08:27 +00:00
Vitaly Baranov
5aaff60650 Fix referential dependencies when host & post in a clickHouse dictionary source are set by default. 2022-12-12 18:22:14 +01:00
Vitaly Baranov
76ba8ab3d4 Add new tests. 2022-12-12 18:22:09 +01:00
Anton Popov
a2fee110b8 add query for waiting of loading parts and fix test_transactions 2022-12-12 14:33:42 +00:00
Sema Checherinda
aecc509f80 make calls to be sure that parts are deleted 2022-12-12 12:37:52 +01:00
Nikolay Degterinsky
19e95e8cd8 Better implementation of password rules 2022-12-11 23:59:04 +00:00
Sema Checherinda
9a1152339c
Merge branch 'master' into do-not-run-clear-old-parts-at-shutdown 2022-12-10 09:44:56 +01:00
Nikolay Degterinsky
a8545a29a4 Better test 2022-12-10 01:32:15 +00:00
Sergei Trifonov
7c37e48092
Merge pull request #43903 from ClickHouse/merge_tree_settings_constraints 
Constraints on merge tree settings
 2022-12-09 14:16:14 +01:00
Kseniia Sumarokova
e21d4a93b8
Merge pull request #44050 from ClickHouse/fix_clickhouse_disk 
Slightly better clickhouse disks and remove DiskMemory
 2022-12-09 13:09:59 +01:00
Alexey Milovidov
68e506d48f Unify __init__.py files 2022-12-08 23:04:56 +01:00
Alexey Milovidov
f880644d3a
Merge branch 'master' into nv/fix-mutate-fetch 2022-12-09 00:51:54 +03:00
Sergei Trifonov
bae1519970
Merge branch 'master' into merge_tree_settings_constraints 2022-12-08 22:25:16 +01:00
robot-clickhouse
98ce0dbb12 Automatic style fix 2022-12-08 17:27:23 +00:00
alesapin
7a223f4956 Slightly better clickhouse disks and remove DiskMemory 2022-12-08 18:20:54 +01:00
serxa
6428b8a1bb check one more case in test 2022-12-07 20:36:04 +00:00
Alexey Milovidov
9c81fd7afc
Merge pull request #43517 from ClickHouse/password-reset-message 
Move password reset message from client to server
 2022-12-07 23:26:58 +03:00
serxa
05788245dd add test 2022-12-07 20:16:10 +00:00
Nikolay Degterinsky
8a53f5364b
Fix style 2022-12-07 10:36:16 +01:00
Nikolay Degterinsky
bb9fdc0839 Fix CREATE query constraints 2022-12-07 00:53:27 +00:00
Sema Checherinda
0b2a3195c8 adjust integration/test_partition/test_detached_part_dir_exists 2022-12-05 17:39:27 +01:00
Sema Checherinda
1f96ef57d2 remove integration/test_parts_removal 2022-12-05 17:24:54 +01:00
kssenii
d278a0dbfb Merge remote-tracking branch 'upstream/master' into allow-single-disk-instead-of-storage-policy 2022-12-05 12:59:44 +01:00
kssenii
dca3444283 Merge remote-tracking branch 'upstream/master' into named-collections-sql-commands 2022-12-04 15:56:02 +01:00
Vitaly Baranov
e1f7f04752
Referential dependencies for RESTORE (#43834) 
* Rename DDLDependencyVisitor -> DDLLoadingDependencyVisitor.

* Move building a loading graph to TablesLoader.

* Implement referential dependencies for tables and use them
when restoring tables from a backup.

* Remove StorageID::operator < (because of its inconsistency with ==).

* Add new tests.

* Fix test.

* Fix memory leak.

Co-authored-by: Nikita Mikhaylov <mikhaylovnikitka@gmail.com>
 2022-12-02 15:05:46 +01:00
Vladimir C
7d6950d397
Revert "Temporary files evict fs cache" 2022-12-02 14:50:56 +01:00
Alexander Tokmakov
edbd460997
Merge pull request #43848 from elenatorro/fix_reset_setting_on_cluster 
Add RESET_SETTING to check if it is a settings alter
 2022-12-02 15:06:28 +03:00
Alexander Tokmakov
c2a19fc354
Update test.py 2022-12-01 19:41:29 +03:00
Nikita Mikhaylov
9ef54c6ecc
Try to use exchanges during database replicated replica recovery process (#43628) 2022-12-01 13:50:56 +01:00
vdimir
08b8d335f0
add integration/test_temporary_data_in_cache 2022-12-01 11:49:25 +00:00
vdimir
7a814d2854
wip: temporary files evict fs cache 2022-12-01 11:49:24 +00:00
Elena Torró
5630412944
Merge branch 'master' into fix_reset_setting_on_cluster 2022-11-30 21:47:43 +01:00
Elena
0a35a072d9 Add RESET_SETTING to check if is a settings alter 2022-11-30 21:44:19 +01:00
Robert Schulze
6f564c59bd
Merge pull request #42641 from Enmk/fix_profile_names_mismatch 
Fixed exception when user tries to log in
 2022-11-30 13:01:57 +01:00
Anton Popov
df6d2f2163 Merge remote-tracking branch 'upstream/master' into HEAD 2022-11-29 23:21:23 +00:00
Alexey Milovidov
72bf1115e9
Merge branch 'master' into password-reset-message 2022-11-29 23:51:46 +01:00
Vasily Nemkov
3d0c07ac5b Updated tests to match improved behaviour of currentProfiles() 2022-11-30 02:32:27 +04:00
Anton Popov
2a1fd48e91 fix tests 2022-11-29 17:33:35 +00:00
robot-clickhouse
ad66e01ace Automatic style fix 2022-11-29 14:48:05 +00:00
Alexander Tokmakov
12a7d64ef5
Update test.py 2022-11-29 17:33:02 +03:00
Antonio Andelic
db46842c8a Merge branch 'master' into more-aggresive-batching-keeper 2022-11-29 12:58:33 +00:00
alesapin
a800a80ab6
Merge pull request #43770 from ClickHouse/fix_flaky_store_cleanup2 
Fix flaky `test_store_cleanup`
 2022-11-29 13:11:57 +01:00
Antonio Andelic
e39f97a255 Properly close connections and add retries 2022-11-29 09:13:23 +00:00
Anton Popov
486da48ae7 fix tests 2022-11-28 21:15:41 +00:00
Alexander Tokmakov
40d28e3489 fix flaky test 2022-11-28 18:31:22 +01:00
Alexander Tokmakov
06e6844ec0
Update test.py 2022-11-28 18:22:03 +03:00
Antonio Andelic
af7f0257c3
Merge pull request #43658 from ClickHouse/try-fix-flaky-keeper-map 
Fix flaky `KeeperMap` integration tests
 2022-11-28 12:04:04 +01:00
Nikita Taranov
8ed5cfc265
Memory bound merging for distributed aggregation in order (#40879) 
* impl

* fix style

* make executeQueryWithParallelReplicas similar to executeQuery

* impl for parallel replicas

* cleaner code for remote sorting properties

* update test

* fix

* handle when nodes of old versions participate

* small fixes

* temporary enable for testing

* fix after merge

* Revert "temporary enable for testing"

This reverts commit cce7f8884c.

* review fixes

* add bc test

* Update src/Core/Settings.h
 2022-11-28 00:41:31 +01:00
Antonio Andelic
cade21b76e Fix 4LW test 2022-11-26 19:24:10 +00:00
Antonio Andelic
65dba31c3a
Merge branch 'master' into try-fix-flaky-keeper-map 2022-11-26 19:00:30 +01:00
Sema Checherinda
15a6ce254b
Merge pull request #41145 from CheSema/lock-free-drop-partition 
Delete parts by replacing them with empty parts
 2022-11-25 12:54:30 +01:00
Antonio Andelic
f3b01d455b Use dummy pool 2022-11-25 11:12:01 +00:00
Anton Popov
c65be55da1 Merge remote-tracking branch 'upstream/master' into HEAD 2022-11-24 14:30:23 +00:00
Alexander Tokmakov
52d33fff00
Merge pull request #43366 from zzsmdfj/issue_15357_MaterializeMySQL_support_drop_mulit_table 
MaterializeMySQL_support_drop_mulit_table
 2022-11-24 14:54:24 +03:00
Sema Checherinda
6fd7dcf26f
Merge branch 'master' into lock-free-drop-partition 2022-11-24 12:34:19 +01:00
alesapin
f64f0461fe
Merge pull request #43288 from ClickHouse/Enable_compress_marks_and_primary_key 
Enable compress marks and primary key for CI
 2022-11-24 12:24:04 +01:00
Sema Checherinda
4f592bc208 fix isolation breach, droping outdated part should't be ignored 2022-11-23 15:16:09 +00:00
Sema Checherinda
f0294cd6ed fix test_merge_tree_s3_restore 2022-11-23 15:16:09 +00:00
Sema Checherinda
597421014e fix style check 2022-11-23 15:16:08 +00:00
Sema Checherinda
db86cd0f0a fix several critical races, fix tests 2022-11-23 15:16:08 +00:00
Sema Checherinda
49539e6862 fix test_partition 2022-11-23 15:16:08 +00:00
Sema Checherinda
62d3e9da4d fix test_merge_tree_hdfs 2022-11-23 15:16:08 +00:00
Sema Checherinda
d43d7ed6ea fix test_merge_tree_empty_parts and test_ttl_replicated 2022-11-23 15:16:08 +00:00
Sema Checherinda
dd93d331c1 fix 01168_mutations_isolation 2022-11-23 15:16:08 +00:00
Sema Checherinda
9edad4a078 make test_detached_parts_metrics wait delete inactive and empty parts 2022-11-23 15:16:08 +00:00
Sema Checherinda
44a8f2f460 work with test_merge_tree_s3 and test_merge_tree_s3_restore 2022-11-23 15:16:08 +00:00
Sema Checherinda
cddf44034d address test_transactions 2022-11-23 15:16:08 +00:00
Sema Checherinda
dc19141d84 fix test_detached_parts_metrics 2022-11-23 15:16:08 +00:00
Sema Checherinda
15012c7070 address the review comments 2022-11-23 15:16:08 +00:00
Sema Checherinda
fb86019db9 Delete parts over replacing them with empty parts 2022-11-23 15:16:08 +00:00
Alexander Tokmakov
f44eb4e021
Update test.py 2022-11-23 14:10:49 +03:00
kssenii
6044a9257c Merge remote-tracking branch 'upstream/master' into named-collections-sql-commands 2022-11-23 12:00:55 +01:00
Alexey Milovidov
bedfe68503
Merge branch 'master' into nv/fix-mutate-fetch 2022-11-22 20:06:56 +01:00
kssenii
5f649baffe Merge remote-tracking branch 'upstream/master' into allow-single-disk-instead-of-storage-policy 2022-11-22 14:19:51 +01:00
Dmitry Novik
e67bda65a0
Merge pull request #43453 from ClickHouse/fix-hard-limit-in-test 
Do not reuse jemalloc memory in test_global_overcommit
 2022-11-22 13:24:34 +01:00
Alexander Tokmakov
cab33c02af
Merge branch 'master' into revert_SingleValueDataString 2022-11-22 14:27:58 +03:00
Antonio Andelic
843401c3a3
Merge pull request #43323 from ClickHouse/fix-describe-delta-lake-hudi 
Fix `DESCRIBE` for `deltaLake` and `hudi` table functions
 2022-11-22 09:09:39 +01:00
kssenii
a279bac767 Merge remote-tracking branch 'upstream/master' into named-collections-sql-commands 2022-11-21 21:49:34 +01:00
Kruglov Pavel
581e57be9f
Merge pull request #43236 from xiedeyantu/fix_hdfscluster_connection_fail 
fix skip_unavailable_shards does not work using hdfsCluster table function
 2022-11-21 19:22:16 +01:00
Igor Nikonov
47ce029cbf
Merge pull request #43326 from ClickHouse/igor/test_read_only_fast_fix 
Fix: make test_read_only_table more stable
 2022-11-21 18:25:14 +01:00
Dmitry Novik
b17dc24a94 Do not reuse jemalloc memory in test 2022-11-21 16:10:47 +00:00
Raúl Marín
35e9e169a1
Merge branch 'master' into revert_SingleValueDataString 2022-11-21 15:11:13 +01:00
Kseniia Sumarokova
a59dac6337
Merge pull request #43143 from azat/disks/s3-check-fix 
Add server UUID for disks access checks (read/read-by-offset/write/delete) to avoid possible races
 2022-11-21 15:03:46 +01:00
alesapin
b78b602ea1
Merge branch 'master' into Enable_compress_marks_and_primary_key 2022-11-21 12:05:04 +01:00
Antonio Andelic
3cb202a63b
Merge pull request #43026 from JackyWoo/keeper_manually_assign_leader 
Keeper support manually assigning leader
 2022-11-21 09:30:00 +01:00
Antonio Andelic
9a8d82f0fc Merge branch 'master' into fix-describe-delta-lake-hudi 2022-11-21 08:23:33 +00:00
zzsmdfj
a0f391745d to 15357_MaterializeMySQL_support_drop_mulit_table-fix style-black 2022-11-21 11:48:57 +08:00
Azat Khuzhin
3e42ffd372 tests: fix hdfs disks (chicken and an egg problem with current cluster.py) 
Right now cluster.py first creates HDFS and then creates clickhouse in
one API call, so you cannot interract and add missing mkdirs for the
clickhouse, fix this by using root dir where it is possible.
 2022-11-20 16:28:35 +01:00
Azat Khuzhin
0678fba3d1 tests: fix test_disk_types (by adding mkdir for HDFS) 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-11-20 16:28:35 +01:00
Kseniia Sumarokova
f0dbfbb0f4
Merge pull request #42800 from azat/disks/web-fix 
Do not suppress exceptions in web disk (and fix retries for requests from web disk)
 2022-11-20 16:07:45 +01:00
Alexander Tokmakov
9464c32bb8 Merge branch 'master' into revert_SingleValueDataString 2022-11-19 18:00:04 +01:00
kssenii
08e4805c99 Merge remote-tracking branch 'upstream/master' into allow-single-disk-instead-of-storage-policy 2022-11-19 13:26:56 +01:00
Azat Khuzhin
c029549859 Allow to drop tables from s3_plain disk (so as from web disk) 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-11-19 10:10:27 +01:00
zzsmdfj
e177927efb to 15357_MaterializeMySQL_support_drop_mulit_table-fix code-style 2022-11-19 12:05:27 +08:00
alesapin
c7d648174b
Merge pull request #43328 from ClickHouse/better-format-version-check 
Check content of `format_version` file in `MergeTreeData`
 2022-11-18 22:34:37 +01:00
Vitaly Baranov
a348332eab
Merge pull request #43227 from vitlibar/improve-masking-sensitive-info 
Improve masking sensitive info
 2022-11-18 15:37:50 +01:00
zzsmdfj
9239f50a87
Merge branch 'master' into issue_15357_MaterializeMySQL_support_drop_mulit_table 2022-11-18 21:33:31 +08:00
zzsmdfj
6a998917b0 to MaterializeMySQL_support_drop_mulit_table 2022-11-18 21:29:13 +08:00
alesapin
06894534e8 Followup fix 2022-11-17 18:52:18 +01:00
robot-clickhouse
8c55679b01 Automatic style fix 2022-11-17 17:13:23 +00:00
alesapin
cecf032b9e Disable compressed marks for some tests 2022-11-17 18:06:59 +01:00
kssenii
5cdf216c8f Merge remote-tracking branch 'upstream/master' into allow-single-disk-instead-of-storage-policy 2022-11-17 16:39:36 +01:00
kssenii
69d71a8e5f Mask collection values in logs 2022-11-17 16:01:42 +01:00
robot-clickhouse
926b35b7c4 Automatic style fix 2022-11-17 14:46:25 +00:00
kssenii
5d9b34b59d More tests 2022-11-17 15:34:35 +01:00
robot-clickhouse
13e051a5de Automatic style fix 2022-11-17 13:46:21 +00:00
Antonio Andelic
d41af8b741 Check and compare content of format_version file 2022-11-17 13:41:36 +00:00
Igor Nikonov
d2d6e75d97 Fix: make test_read_only_table more stable 
+ add retries to INSERT queries after keeper node restart
 2022-11-17 13:26:32 +00:00
Nikita Taranov
7beb58b0cf
Optimize merge of uniqExact without_key (#43072) 
* impl for uniqExact

* rm unused (read|write)Text methods

* fix style

* small fixes

* impl for variadic uniqExact

* refactor

* fix style

* more agressive inlining

* disable if max_threads=1

* small improvements

* review fixes

* Revert "rm unused (read|write)Text methods"

This reverts commit a7e7480584.

* encapsulate is_able_to_parallelize_merge in Data

* encapsulate is_exact & argument_is_tuple in Data
 2022-11-17 13:19:02 +01:00
robot-clickhouse
1ad7362db7 Automatic style fix 2022-11-17 11:54:13 +00:00
Antonio Andelic
d5848d53cc Fix DESCRIBE for deltaLake and hudi table functions 2022-11-17 11:46:17 +00:00
kssenii
1964291fba Add tests 2022-11-17 12:35:43 +01:00
JackyWoo
5a09de279c fix test 2022-11-17 10:35:01 +08:00
chen
cb83337586
Update test.py 2022-11-17 06:41:43 +08:00
xiedeyantu
fc1d06f096 fix 2022-11-16 23:43:23 +08:00
Raúl Marín
45a611bc1d Fix style with black 2022-11-16 13:28:04 +01:00
Raúl Marín
4e7ccd1c06 Merge remote-tracking branch 'blessed/master' into revert_SingleValueDataString 2022-11-16 12:59:47 +01:00
xiedeyantu
7dc941cacd fix 2022-11-16 19:47:57 +08:00
Alexander Tokmakov
7e6fe94a8e
Merge pull request #43266 from ClickHouse/global-overcommit-test 
Make test_global_overcommit_tracker non-parallel
 2022-11-16 14:28:56 +03:00
alesapin
32358d2180
Merge pull request #43256 from ClickHouse/vdimir/msg_in_wait_zookeeper_to_start 
Better message in wait_zookeeper_to_start
 2022-11-16 11:58:53 +01:00
JackyWoo
01b63fd98a fix style 2022-11-16 12:51:51 +08:00
Anton Popov
f94182384a Merge remote-tracking branch 'upstream/master' into HEAD 2022-11-16 03:30:29 +00:00
xiedeyantu
c23cd091a3 fix 2022-11-16 09:59:44 +08:00
JackyWoo
f8d71f60a2
Merge branch 'master' into keeper_manually_assign_leader 2022-11-16 09:34:40 +08:00
xiedeyantu
8144516e53 fix 2022-11-16 09:33:46 +08:00
JackyWoo
ace40a2538 fix tests 2022-11-16 09:33:46 +08:00
xiedeyantu
3d218795be fix 2022-11-16 09:18:45 +08:00
Vitaly Baranov
2e242fa991 Add tests. 2022-11-16 01:35:11 +01:00
Dmitry Novik
fcdc9dfaac Make test_global_overcommit_tracker non-parallel 2022-11-15 20:52:18 +00:00
Vitaly Baranov
22abe9a4fb
Merge pull request #42950 from azat/disks/s3_plain/wide-part-fix 
Fixes for s3_plain disk that will allow to attach Wide parts
 2022-11-15 17:56:16 +01:00
robot-clickhouse
aaeeeaf1fc Automatic style fix 2022-11-15 16:40:01 +00:00
vdimir
03968eb694
Better message in wait_zookeeper_to_start 2022-11-15 16:17:12 +00:00
Antonio Andelic
6beab87610
Merge pull request #43080 from ucasfl/hudi-udtf 
Add table function hudi and deltaLake
 2022-11-15 15:42:25 +01:00
Kseniia Sumarokova
9fc96ab7fd
Merge branch 'master' into disks/web-fix 2022-11-15 12:16:32 +01:00
xiedeyantu
09c749fbd0 update 2022-11-15 13:30:29 +08:00
xiedeyantu
ec6698395e fix skip_unavailable_shards does not work using hdfsCluster table function 2022-11-15 13:25:15 +08:00
Kruglov Pavel
386b69acae
Merge pull request #43131 from xiedeyantu/fix_s3cluster_connection_fail 
fix skip_unavailable_shards does not work using s3Cluster table function
 2022-11-14 20:48:00 +01:00
Raúl Marín
790cfe2b7e Allow backward incompatible change for toDate32 since it was declared as such in 22.8 2022-11-14 18:10:36 +01:00
Kruglov Pavel
eacd7e1876
Merge pull request #42931 from zzsmdfj/MaterializedMySQL_improvement_bug_fix 
Fix issue 42856 and MaterializedMySQL improvement
 2022-11-14 16:54:12 +01:00
flynn
5bd8e2338b
Merge branch 'master' into hudi-udtf 2022-11-14 22:28:04 +08:00
Raúl Marín
58da0398a0 Merge remote-tracking branch 'blessed/master' into revert_SingleValueDataString 2022-11-14 12:46:00 +01:00
Antonio Andelic
41c0100dd7
Merge pull request #43115 from ClickHouse/fix-4lw-command-test 
Fix `test_keeper_four_word_command::test_cmd_stat`
 2022-11-14 08:33:11 +01:00
Azat Khuzhin
f2260ab724 Add a test for unavalable web server and web disk 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-11-12 13:25:10 +01:00
Azat Khuzhin
de5d777ca2 tests/integration: add ability not to retry server start 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-11-12 13:20:51 +01:00
Igor Nikonov
ee5756b5b4
Merge pull request #43122 from ClickHouse/revert-43116-revert-42607-igor/insert_zk_retries_retry 
Revert "Revert " Keeper retries during insert (clean)""
 2022-11-11 18:22:38 +01:00
Raúl Marín
5a3f324311 Try adding compatibility on write instead of on read 2022-11-11 18:12:34 +01:00
Dmitry Novik
31794284cb
Merge pull request #43105 from ClickHouse/overcommit-test 
Rewrite tests for memory overcommit
 2022-11-11 17:34:53 +01:00
xiedeyantu
5504f3af9b fix skip_unavailable_shards does not work using s3Cluster table function 2022-11-12 00:03:36 +08:00
Anton Kozlov
ccede70b5c
Merge branch 'master' into tonickkozlov/add-interserver-dns-retries 2022-11-11 14:54:30 +00:00
Anton Kozlov
024d9382c7 CLICKHOUSE-2375 Add interserver DNS retries 2022-11-11 14:47:26 +00:00
Kruglov Pavel
4ecbcbf5e1
Merge branch 'master' into MaterializedMySQL_improvement_bug_fix 2022-11-11 12:54:02 +01:00
zzsmdfj
a9a208c3cd to MaterializedMySQL_improvement_bug_fix-fix code style 2022-11-11 11:46:22 +08:00
zzsmdfj
227dcd68ba Merge branch 'MaterializedMySQL_improvement_bug_fix' of github.com:zzsmdfj/ClickHouse into MaterializedMySQL_improvement_bug_fix 2022-11-11 11:25:03 +08:00
zzsmdfj
0e614aec12 to MaterializedMySQL_improvement_bug_fix-fix code sytle 2022-11-11 11:24:29 +08:00
zzsmdfj
b01eea5c10
Merge branch 'master' into MaterializedMySQL_improvement_bug_fix 2022-11-11 10:45:36 +08:00
zzsmdfj
d4e270b522 to MaterializedMySQL_improvement_bug_fix-fix codestyle 2022-11-11 10:44:16 +08:00
Stig Bakken
9020631826 Ignore SAVEPOINT queries in MaterializedMySQL 2022-11-10 23:27:21 +01:00
Azat Khuzhin
afc6c8bb73 tests: add ATTACH FROM s3_plain for wide part 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-11-10 17:42:32 +01:00
robot-clickhouse
fe283af82b Automatic style fix 2022-11-10 16:08:05 +00:00
Dmitry Novik
4e15d648e7 Add test_global_overcommit_tracker/__init__.py 2022-11-10 16:02:45 +00:00
Dmitry Novik
b1e942e14c Do not run global test with sanitizers 2022-11-10 16:00:10 +00:00
Igor Nikonov
f8c907c463
Revert "Revert " Keeper retries during insert (clean)"" 2022-11-10 13:14:04 +01:00
Alexander Tokmakov
fb8c527a2f
Revert " Keeper retries during insert (clean)" 2022-11-10 14:12:24 +03:00
Antonio Andelic
c50e795788 Fix expected ZXID 2022-11-10 11:04:51 +00:00
Yakov Olkhovskiy
c08826d174
Merge branch 'master' into tests-connection-error 2022-11-10 02:57:10 -05:00
JackyWoo
8e881e6009
Merge branch 'master' into keeper_manually_assign_leader 2022-11-10 14:38:04 +08:00
flynn
da77199bf5 fix style 
fix style
 2022-11-10 03:35:38 +00:00
flynn
6cbfab4e95 fix test 2022-11-10 03:10:25 +00:00
robot-clickhouse
eb612a699d Automatic style fix 2022-11-10 01:08:12 +00:00