Commit Graph

8289 Commits

Author SHA1 Message Date
Alexey Milovidov
1f752b79db
Merge pull request #55894 from bharatnc/ncb/hostname-system-log-tables
add hostname column to system log tables
2023-12-05 06:42:43 +01:00
Alexey Milovidov
10d65a1ade
Merge pull request #55559 from azat/s3-fix-excessive-reads
Add ability to disable checksums for S3 to avoid excessive input file read
2023-12-05 06:34:21 +01:00
Bharat Nallan Chakravarthy
942802f308 few more test cases 2023-12-04 14:01:43 -08:00
joelynch
b312a9f4ee
bugfix: parse system stop listen tcp secure 2023-12-04 19:56:35 +01:00
Alexey Milovidov
02439eee58
Merge pull request #49351 from ClickHouse/async-loader-integration
Asynchronous loading of tables
2023-12-04 17:16:12 +01:00
vdimir
a4ae90de0d
Merge pull request #57275 from ClickHouse/vdimir/merge_task_tmp_data
Background merges correctly use temporary data storage in the cache
2023-12-04 14:52:20 +01:00
vdimir
27589f5b0e
Enable temporary_data_in_cache in test_encrypted_disk 2023-12-04 10:34:17 +00:00
vdimir
b2e29d1416
Change type of s3_cache in test_encrypted_disk 2023-12-04 10:34:13 +00:00
Bharat Nallan Chakravarthy
25d1d59c6d fix tests 2023-12-03 20:16:16 -08:00
Bharat Nallan Chakravarthy
440dc66a5c Merge remote-tracking branch 'upstream/master' into ncb/hostname-system-log-tables 2023-12-03 15:19:47 -08:00
Vitaly Baranov
366095affc Fix test test_create_or_drop_tables_during_backup. 2023-12-04 00:15:31 +01:00
Sema Checherinda
32e3b2657b
Merge pull request #57205 from CheSema/zero-locks-leak
fix zero-copy locks leaking
2023-12-02 15:54:12 +01:00
Han Fei
fc0f080cb4 fix black 2023-12-01 20:39:38 +01:00
JackyWoo
8a68d51261
Add index to table system.numbers (#50909) 2023-12-01 19:59:25 +01:00
Han Fei
e4da317f05 fix test 2023-12-01 18:07:45 +01:00
Nikolai Kochetov
0b4131546a
Merge pull request #56813 from jsc0218/SystemTablesFilterEngine
Able to Filter Engine When Scanning System Tables
2023-12-01 16:02:27 +01:00
alesapin
fb98b212c5
Merge pull request #57311 from ClickHouse/make_alter_mat_view_non_experimental
Make alter materialized view non experimental
2023-12-01 12:18:43 +01:00
Sema Checherinda
60bf17ce8a fix style 2023-11-30 21:13:56 +01:00
Sema Checherinda
42f648f0d2 relock block when sessin expired 2023-11-30 21:13:56 +01:00
alesapin
17ac9e3409 Fix test 2023-11-30 19:42:12 +01:00
vdimir
b9860648c1
Remove temporary_data_in_cache from test_merge_tree_s3 2023-11-30 13:27:32 +00:00
Kruglov Pavel
575ee1e665
Merge pull request #56868 from zvonand/zvonand-fileCluster
Introduce `fileCluster` table function
2023-11-30 13:49:53 +01:00
Han Fei
a316b70917 Merge branch 'master' into hanfei/keeper_soft_limit 2023-11-30 12:29:40 +01:00
Alexander Tokmakov
a2dc2d9a53
Mark a setting obsolete (#57336)
* Update Settings.h

* Update InterpreterUndropQuery.cpp

* Update 02888_obsolete_settings.reference

* Update test.py

* Update 02681_undrop_query.sql

* Update 02681_undrop_query_uuid.sh
2023-11-29 19:24:11 +01:00
Alexander Tokmakov
97b63bbbca
Merge pull request #57339 from ClickHouse/fix_ddlworker_hostname_change
Ignore finished ON CLUSTER tasks if hostname changed
2023-11-29 17:46:17 +01:00
Nikolai Kochetov
823ba2db46
Merge pull request #57075 from yariks5s/s3_links_fix
S3-style links bug fix
2023-11-29 17:41:08 +01:00
vdimir
b5babe1692
MergeTask uses temporary data storage 2023-11-29 16:18:32 +00:00
zvonand
c329e382ab resolve conflicts 2023-11-29 16:03:07 +01:00
Han Fei
4c0efb0262
Merge pull request #53240 from hanfei1991/hanfei/statistic
use statistic to order prewhere conditions better
2023-11-29 12:26:44 +01:00
Nikolai Kochetov
731a1bfcf8 Cleanup 2023-11-29 11:11:58 +00:00
Alexander Tokmakov
84abbafcc5 ignore finished ddl tasks if hostname changed 2023-11-29 02:06:52 +01:00
Nikolai Kochetov
4d906d6e3c Cleanup 2023-11-28 18:35:19 +00:00
Yarik Briukhovetskyi
ad9583d6e9
style fix 2023-11-28 19:13:37 +01:00
Yarik Briukhovetskyi
a83a046d54 added tests 2023-11-28 17:33:46 +00:00
Nikolai Kochetov
08a7575984 Re-implement filtering a bit. 2023-11-28 16:17:35 +00:00
Han Fei
6733438adc Merge branch 'master' into hanfei/statistic 2023-11-28 16:35:00 +01:00
zvonand
c306d21b54 merge master + resolve conflicts 2023-11-28 15:51:21 +01:00
Han Fei
3aa4a6ae2c check integration test 2023-11-28 14:47:31 +01:00
Antonio Andelic
9dd4783675 Merge branch 'master' into fix-readonly-replica 2023-11-28 10:43:29 +00:00
vdimir
ae09d04578
Merge pull request #57154 from ClickHouse/vdimir/params_test_check_table
Minor changes in test_check_table
2023-11-28 11:10:26 +01:00
Han Fei
7bd2ea3add Merge branch 'master' into hanfei/statistic 2023-11-27 23:31:23 +01:00
Han Fei
8e1b61cc13 fix style 2023-11-27 22:03:05 +01:00
Nikolai Kochetov
e1b00b3ee0 Merge branch 'master' into SystemTablesFilterEngine 2023-11-27 18:54:21 +00:00
Han Fei
347efc669e support memory soft limit for keeper 2023-11-27 18:28:10 +01:00
Antonio Andelic
2ea44adefd Add test 2023-11-27 15:05:56 +00:00
Antonio Andelic
9707796869 Ignore comments when comparing column descriptions 2023-11-27 12:25:41 +00:00
MikhailBurdukov
6f19e8ebd1
Merge branch 'master' into native_copy_for_s3_disks 2023-11-27 14:25:36 +03:00
MikhailBurdukov
c10c30832c Review fix 2023-11-27 10:58:30 +00:00
Kruglov Pavel
b10e46b2bc
Merge pull request #57126 from Avogar/size-virtual-column
Add _size virtual column to s3/file/hdfs/url/azureBlobStorage engines
2023-11-27 10:12:18 +01:00
Bharat Nallan Chakravarthy
6454687fa6 Merge remote-tracking branch 'upstream/master' into ncb/hostname-system-log-tables 2023-11-26 16:24:18 -08:00
Azat Khuzhin
1168b7910b Update test_throttling with enabled s3_disable_checksum
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-11-26 20:03:33 +01:00
Vitaly Baranov
4fed61e8ca
Merge pull request #57146 from vitlibar/fix-test_replicated_merge_tree_encryption_codec_different_keys_2
Fix test test_replicated_merge_tree_encryption_codec/test.py::test_different_keys
2023-11-25 09:36:33 +01:00
Alexey Milovidov
c478acab42
Merge pull request #57206 from azat/tests/test_distributed_storage_configuration
Fix test_distributed_storage_configuration flakiness
2023-11-25 02:57:51 +01:00
Sema Checherinda
abd399705e
Merge pull request #57025 from CheSema/s3_engine_heavy_write_check_mem
test_s3_engine_heavy_write_check_mem: turn test off
2023-11-24 21:07:35 +01:00
Azat Khuzhin
d0b4917fd3 Fix test_distributed_storage_configuration flakiness
It fails [1] due to extra reservation on disk2 for some system.*_log
tables, sure we can turn them off, but better to fix it explicitly with
a separate policy.

  [1]: https://s3.amazonaws.com/clickhouse-test-reports/0/98cddf5312722e403dcea429639ac13dc6cada33/integration_tests__tsan__[2_6].html

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-11-24 20:28:18 +01:00
Alexey Milovidov
411660c267 Fix integration tests 2023-11-24 17:35:56 +01:00
Vitaly Baranov
5769a88b92
Merge pull request #57133 from vitlibar/change-default-for-wait_dictionaries_load_at_startup
Change the default for wait_dictionaries_load_at_startup to true
2023-11-24 17:09:05 +01:00
Vitaly Baranov
683827c6df Add comment. 2023-11-24 14:01:52 +01:00
vdimir
3b4f70a811
Minor changes in test_check_table 2023-11-23 16:24:11 +00:00
Vitaly Baranov
eabdc76e7c Fix test test_replicated_merge_tree_encryption_codec/test.py::test_different_keys 2023-11-23 15:36:11 +01:00
Vitaly Baranov
5f461ff780 Change the default for "wait_dictionaries_load_at_startup" to true,
and use this setting only if "dictionaries_lazy_load" is false.
2023-11-23 14:45:42 +01:00
Aleksei Filatov
1a03f5f7f4 Merge remote-tracking branch 'upstream/master' into add_cancellation_point_for_moving_background_operation 2023-11-23 16:43:33 +03:00
avogar
e3afb4182b Fix tests 2023-11-23 11:10:41 +00:00
serxa
0fc25fbda8 Merge branch 'master' into async-loader-integration 2023-11-23 09:32:43 +00:00
jsc0218
aba2f0e4b2 Fix Style 2023-11-22 20:02:15 +00:00
jsc0218
a1bde09c9c move the test from stateless to integration 2023-11-22 19:13:13 +00:00
Kruglov Pavel
b84e3cf683
Merge branch 'master' into size-virtual-column 2023-11-22 19:25:00 +01:00
avogar
193f8a5d23 Add test for azure 2023-11-22 18:12:50 +00:00
zvonand
5153798aeb Introduced fileCluster table function
Added fileCluster function
Added test and docs
2023-11-22 15:06:04 +01:00
Kruglov Pavel
9fb1acc947
Merge pull request #56810 from Avogar/iceberg-metadata-files
Iceberg metadata files
2023-11-22 14:40:04 +01:00
Sema Checherinda
adf4ade00a
Update tests/integration/test_merge_tree_s3/test.py 2023-11-22 13:38:15 +01:00
Kseniia Sumarokova
e4f66b8469
Merge pull request #55158 from kssenii/fs-cache-improvement
fs cache improvement for big reads
2023-11-21 21:50:00 +01:00
Nikita Mikhaylov
087a0a6043
Merge pull request #57040 from ClickHouse/fix-rabbitmq-supress-tsan-in-test
Fix: suppress TSAN in RabbitMQ test
2023-11-21 17:52:36 +01:00
Sema Checherinda
63bd45fa5f
Update tests/integration/test_merge_tree_s3/test.py 2023-11-21 17:42:37 +01:00
vdimir
a139ae97eb
Merge pull request #52918 from ClickHouse/vdimir/s3_blob_log
Add system table with blob storage operations log
2023-11-21 17:40:42 +01:00
Vitaly Baranov
4052ae8bfd
Merge pull request #57007 from vitlibar/fix-dropping-tables-in-test_create_or_drop_tables_during_backup
Fix dropping tables in test_create_or_drop_tables_during_backup
2023-11-21 17:16:16 +01:00
Sema Checherinda
2e1c142d6d black style 2023-11-21 14:03:49 +01:00
Igor Nikonov
08b2740d0f Fix: supress test with tsan 2023-11-21 11:26:14 +00:00
Kseniia Sumarokova
d384762123
Merge branch 'master' into fs-cache-improvement 2023-11-21 11:24:52 +01:00
Kseniia Sumarokova
f960036d76
Merge pull request #56874 from ClickHouse/kssenii-patch-8
Update test_storage_s3_queue/test.py
2023-11-21 11:13:00 +01:00
vdimir
15234474d7
Implement system table blob_storage_log 2023-11-21 09:18:25 +00:00
Sema Checherinda
524edd8460 turn test off 2023-11-21 00:08:20 +01:00
Alexey Milovidov
6a1955cb34 Merge branch 'master' of github.com:ClickHouse/ClickHouse into remove-useless-install 2023-11-20 21:34:31 +01:00
Sema Checherinda
485f1834d8
Merge pull request #56938 from CheSema/lz4-buffering
Lz4 compression: buffer block in a rare case
2023-11-20 20:33:30 +01:00
Kseniia Sumarokova
9454b203e1
Merge branch 'master' into kssenii-patch-8 2023-11-20 20:19:21 +01:00
Vitaly Baranov
897cd06bcf Fix dropping tables in test "test_create_or_drop_tables_during_backup". 2023-11-20 19:31:53 +01:00
avogar
f537bad469 Merge branch 'master' of github.com:ClickHouse/ClickHouse into schema-inference-union 2023-11-20 14:32:50 +00:00
Sema Checherinda
a7fc8d4b99 test_merge_tree_s3 counts errors, turn off s3_use_adaptive_timeouts 2023-11-20 15:04:14 +01:00
avogar
872556a5d4 Merge branch 'master' of github.com:ClickHouse/ClickHouse into schema-inference-union 2023-11-20 14:03:36 +00:00
Kseniia Sumarokova
4c7daf5167
Fix 2023-11-20 14:55:23 +01:00
Sema Checherinda
f999337dae
Revert "Revert "s3 adaptive timeouts"" 2023-11-20 14:53:22 +01:00
Alexander Tokmakov
5031f239c3
Revert "s3 adaptive timeouts" 2023-11-20 14:28:59 +01:00
Alexey Milovidov
241cc2abf4 Merge branch 'master' into remove-useless-install 2023-11-20 01:12:08 +01:00
Bharat Nallan Chakravarthy
d08512d8c0 Merge remote-tracking branch 'upstream/master' into ncb/hostname-system-log-tables 2023-11-19 14:06:40 -08:00
Igor Nikonov
b9a28028e4
Merge branch 'master' into test-rabbitmq-with-secure-connection 2023-11-19 21:33:09 +01:00
Alexey Milovidov
9367dcc276
Merge pull request #56975 from rschu1ze/unflake-test_backward_compat
Add exclude for tryBase64Decode to backward compat test (follow-up to #56913)
2023-11-19 16:41:42 +01:00
Robert Schulze
a3c9f13ac9
Add exclude for tryBase64Decode to backward compat test (follow-up to #56913)
Fixes #56969
2023-11-19 15:33:58 +00:00
Antonio Andelic
ab234a87a8
Merge pull request #56960 from ClickHouse/fix-test_keeper_auth
Fix `test_keeper_auth`
2023-11-19 16:09:04 +01:00
Sema Checherinda
a950595c24
Merge pull request #56314 from CheSema/s3-aggressive-timeouts
s3 adaptive timeouts
2023-11-19 14:12:14 +01:00
serxa
9023f61349 Merge branch 'master' into async-loader-integration 2023-11-19 13:05:58 +00:00
Antonio Andelic
7c81d9b2b7 Fix test_keeper_auth 2023-11-18 19:38:33 +00:00
Antonio Andelic
274f2c71f2
Merge pull request #56460 from ClickHouse/keeper-map-backup-restore
Backup/Restore for KeeperMap tables
2023-11-18 19:27:36 +01:00
Antonio Andelic
9eaafff3d2
Merge pull request #56715 from incfly/keeper-az-fix
Fix keeper availability zone configuration.
2023-11-18 19:24:24 +01:00
Alexey Milovidov
74319b5cbd
Merge pull request #56082 from Algunenano/less_diff_images
See what happens if we use less different docker images in integration tests
2023-11-18 17:24:28 +01:00
Sema Checherinda
6d5a5f9fcd buffer result if out copacity is not enough 2023-11-17 17:31:00 +01:00
Raúl Marín
4d5becb4de Adapt test_storage_s3/test.py::test_predefined_connection_configuration 2023-11-17 15:01:54 +01:00
Raúl Marín
7074bdb3bb Merge remote-tracking branch 'blessed/master' into access-control-and-named-collections-by-default 2023-11-17 14:53:02 +01:00
Kruglov Pavel
4e6f265e3e
Fix tests 2023-11-17 14:36:48 +01:00
Alexey Milovidov
5f9db3b248
Merge pull request #56794 from ClickHouse/early-exit-interserver-authentication-failure
Early disconnect if there is authentication failure with interserver secret
2023-11-17 13:36:09 +01:00
Antonio Andelic
803d9668e2 Merge branch 'master' into keeper-map-backup-restore 2023-11-17 10:11:59 +00:00
Antonio Andelic
f8794e72e7
Merge pull request #56858 from ClickHouse/better-test_keeper_broken_logs
Fix `test_keeper_broken_logs`
2023-11-17 10:36:05 +01:00
Antonio Andelic
8b30a54b48
Merge pull request #56835 from ClickHouse/better-test_keeper_reconfig_replace_leader
More stable `test_keeper_reconfig_replace_leader`
2023-11-17 10:35:26 +01:00
Aleksei Filatov
163b8a814e Fix clang-tidy issues and disable parallel execution of tests 2023-11-17 11:02:07 +03:00
Jianfei Hu
ef79bf6467 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into keeper-az-fix 2023-11-16 21:58:02 +00:00
Bharat Nallan Chakravarthy
7434dd9e0a Merge remote-tracking branch 'upstream/master' into ncb/hostname-system-log-tables 2023-11-16 11:17:45 -08:00
Kseniia Sumarokova
2a1467b8da
Update test_storage_s3_queue/test.py 2023-11-16 19:33:17 +01:00
Alexey Milovidov
2941f30678 Merge branch 'master' of github.com:ClickHouse/ClickHouse into early-exit-interserver-authentication-failure 2023-11-16 18:15:09 +01:00
Kseniia Sumarokova
a2ed756eec
Merge branch 'master' into fs-cache-improvement 2023-11-16 17:49:26 +01:00
robot-clickhouse
d03a1aab7b Automatic style fix 2023-11-16 16:39:57 +00:00
Igor Nikonov
34f8bf8da5 Merge remote-tracking branch 'origin/master' into test-rabbitmq-with-secure-connection 2023-11-16 16:27:25 +00:00
Igor Nikonov
49c58e7609 Disable RabbitMQ secure connection test in intergatiion test with TSAN 2023-11-16 16:24:09 +00:00
Konstantin Bogdanov
4cc2d6baa5
Merge pull request #56694 from ClickHouse/fix-configuration-merge-issue
Fix configuration merge for nodes with substitution attributes
2023-11-16 15:19:09 +01:00
Antonio Andelic
bb68321fc1 More stable test_keeper_reconfig_replace_leader 2023-11-16 13:31:39 +00:00
Antonio Andelic
bdf038191a better test_keeper_broken_logs 2023-11-16 13:08:48 +00:00
Igor Nikonov
b9d083e68d Merge remote-tracking branch 'origin/master' into test-rabbitmq-with-secure-connection 2023-11-16 13:05:10 +00:00
Igor Nikonov
4d16c096a1 Use ports from cluster 2023-11-16 13:03:20 +00:00
alesapin
437a911d7b
Merge pull request #56809 from ClickHouse/add_async_moves_execution
Implement asynchronous execute of `ALTER TABLE ... MOVE ... TO DISK|VOLUME`
2023-11-16 12:47:31 +01:00
Antonio Andelic
203f83c2c2 Merge branch 'master' into keeper-map-backup-restore 2023-11-16 11:31:11 +00:00
Sema Checherinda
7d37c0e070
Update tests/integration/test_merge_tree_s3_failover/configs/config.d/storage_conf.xml
Co-authored-by: Nikita Taranov <nickita.taranov@gmail.com>
2023-11-16 12:29:21 +01:00
Sema Checherinda
ebd42187ad
Update tests/integration/test_merge_tree_s3_failover/configs/config.d/storage_conf.xml
Co-authored-by: Nikita Taranov <nickita.taranov@gmail.com>
2023-11-16 12:29:15 +01:00
Vitaly Baranov
342e6c7546
Merge pull request #56782 from vitlibar/add-config-setting-wait_dictionaries_load_at_startup
Add config setting "wait_dictionaries_load_at_startup"
2023-11-16 11:51:29 +01:00
Aleksei Filatov
21a17f8334 Add cancellation hook for moving background operation 2023-11-16 13:37:51 +03:00
Vitaly Baranov
946e02ecbc
Merge pull request #56268 from vitlibar/add-script-backup-view
Add an external utility to view backups and to extract information from them
2023-11-16 11:17:16 +01:00
Robert Schulze
44bd0598a3
Merge pull request #56799 from slvrtrn/fix-binary-mysql-nullable-columns
Fix incorrect nullable columns reporting in MySQL binary protocol
2023-11-16 01:12:46 +01:00
Alexey Milovidov
3067ca64df
Merge pull request #56573 from mkmkme/mkmkme/reload-config
Update `query_masking_rules` when reloading the config
2023-11-16 00:27:01 +01:00
Alexey Milovidov
09bec3c754 Fix integration test 2023-11-16 00:16:02 +01:00
Konstantin Bogdanov
67796b18bb
Add missing test config files 2023-11-15 19:36:23 +01:00
robot-clickhouse
af2f986215 Automatic style fix 2023-11-15 17:48:54 +00:00
avogar
808c3f36ed Remove prints from test 2023-11-15 17:47:10 +00:00
avogar
28522d56b7 Support Iceberg metadata files for metastore tables 2023-11-15 17:45:07 +00:00
Alexander Sapin
3e26161eaa Implement asynchronous excute of ALTER TABLE ... MOVE ... TO DISK|VOLUME 2023-11-15 18:33:59 +01:00
Vitaly Baranov
21a283049c Add the test for clickhouse_backupview to CI. 2023-11-15 17:40:53 +01:00
Alexander Tokmakov
bf10989515
Fix unexpected parts handling (#56693)
* fix unexpected parts handling

* Automatic style fix

* fix

---------

Co-authored-by: robot-clickhouse <robot-clickhouse@users.noreply.github.com>
2023-11-15 16:54:47 +01:00
slvrtrn
9e3ba55070 Fix incorrect nullable columns reporting in MySQL binary protocol 2023-11-15 16:53:38 +01:00
Mikhail Koviazin
08d0a160ae
test_reload_query_masking_rules: don't rely on num of events
Due to the fact that integration tests are run in parallel and there
might be no ClickHouse restart between several tests, relying on
`system.events` in the test is fragile as it can be affected by previous
tests. This commit removes any assumptions regarding `system.events`
from the test and tries to keep it robust.
2023-11-15 17:48:24 +02:00
Antonio Andelic
c3f7c8792a
Merge pull request #56763 from ClickHouse/disable-compress-logs-default
Disable compressed logs by default in Keeper
2023-11-15 16:17:14 +01:00
Antonio Andelic
d567e718eb Merge branch 'master' into keeper-map-backup-restore 2023-11-15 11:51:27 +00:00
Vitaly Baranov
308654195c Add config setting "wait_dictionaries_load_at_startup". 2023-11-15 11:48:50 +01:00
Antonio Andelic
5309dc05ef Fix test 2023-11-15 07:42:19 +00:00
Alexey Milovidov
37e48e76fe
Merge pull request #56768 from vitlibar/fix-flaky-test_replicated_merge_tree_encryption_codec
Fix flaky test_replicated_merge_tree_encryption_codec.
2023-11-15 07:46:06 +01:00