Commit Graph

8852 Commits

Author SHA1 Message Date
Alexey Milovidov
1161dbbd54
Merge pull request #57993 from mkmkme/mkmkme/reload-config-again
Update `query_masking_rules` when reloading the config, attempt 2
2024-01-17 17:09:36 +01:00
robot-clickhouse
c2ccfba8ef Automatic style fix 2024-01-17 15:02:37 +00:00
kssenii
dc5d09fc55 Fix tests 2024-01-17 15:52:07 +01:00
kssenii
1e9de73bf5 Fix style check 2024-01-17 15:40:46 +01:00
kssenii
4d9c173008 Merge remote-tracking branch 'origin/broken-projections-better-handling' into broken-projections-better-handling 2024-01-17 15:40:27 +01:00
kssenii
dea8b10ae9 Fix test 2024-01-17 15:39:57 +01:00
Igor Nikonov
9e17877da5 Fix: use cluster() instead of defining distributed table 2024-01-17 14:11:09 +00:00
Kruglov Pavel
6d064512e1
Merge pull request #58614 from Blargian/58363_disable_ansi_pretty_automatically
58363 Automatically disable ANSI escape sequences in Pretty formats if the output is not a terminal
2024-01-17 13:45:41 +01:00
Кирилл Гарбар
dfbc2b142c GetTableDataPath and new docs 2024-01-17 12:50:09 +03:00
kssenii
1f384dd8a8 Fix 2024-01-17 10:32:56 +01:00
Mikhail Koviazin
6e6197019c
Reintroduce commit 3067ca6
Update `query_masking_rules` when reloading the config

This reverts commit 7e55eab1c3.
2024-01-17 08:10:49 +02:00
Alexander Tokmakov
38b85e6409
Merge pull request #58649 from azat/tests/test_replicated_database
Fix test_replicated_database::test_startup_without_zk flakiness
2024-01-16 21:08:33 +01:00
robot-clickhouse
216769f43e Automatic style fix 2024-01-16 18:55:23 +00:00
kssenii
7bc6a858c7 Rewrite bash test to integration test 2024-01-16 19:44:55 +01:00
Blargian
1b81efc337 fix failing 02421 stateless and (hopefully) fix failing integration tests 2024-01-16 19:48:23 +02:00
robot-clickhouse
7262e93a27 Automatic style fix 2024-01-16 16:26:28 +00:00
János Benjamin Antal
5a178ce1ef Add proper cleanup to kafka test 2024-01-16 16:03:44 +00:00
Vitaly Baranov
085833aaae Use special statuses BACKUP_CANCELLED and RESTORE_CANCELLED in system.backups 2024-01-16 15:07:24 +01:00
Vitaly Baranov
185f6cc37c Add tests for cancelling backup/restore queries. 2024-01-16 15:07:20 +01:00
kssenii
8decf5039a Fix for custom cached disk 2024-01-16 14:41:12 +01:00
vdimir
54a79c1f3d
Merge pull request #58123 from ClickHouse/vdimir/cluster_discovery_password
ClusterDiscovery supports setting username and password
2024-01-16 11:05:54 +01:00
zhongyuankai
cca2fa6976 fix test 2024-01-16 15:39:33 +08:00
Alexander Tokmakov
8bdd6542ce
Merge pull request #58647 from ClickHouse/tavplubix-patch-10
Update test_replicated_database/test.py
2024-01-16 00:32:08 +01:00
János Benjamin Antal
3e1d7bf685
Merge pull request #58477 from ClickHouse/multiple-read-kafka-storage-in-mv
Delay reading from StorageKafka to allow multiple reads in materialized views
2024-01-15 12:31:59 +01:00
vdimir
1f0a29cf10
Fix test_cluster_discovery/test_password.py 2024-01-15 11:31:22 +00:00
vdimir
53e2a97d33
Add missing configs to test_cluster_discovery 2024-01-15 11:22:54 +00:00
vdimir
8dc1756532
Support <secret> in cluster discovery 2024-01-15 11:22:53 +00:00
vdimir
6ab507e722
Add test case for cluster discovery with wrong password 2024-01-15 11:22:52 +00:00
vdimir
012713fc31
ClusterDiscovery supports setting username and password 2024-01-15 11:22:48 +00:00
zhongyuankai
826b29ef4d Max materialized views size for table 2024-01-15 10:02:57 +08:00
Kseniia Sumarokova
77f4cd8dd3
Merge pull request #58721 from ClickHouse/support-point-datatype-for-mysql
Support Point data type for MySQL
2024-01-12 17:02:01 +01:00
kssenii
8b1d2805ae Add one more test case to the test 2024-01-12 10:41:04 +01:00
Dmitry Novik
e3aed6ba4e
Fix integration test 2024-01-12 09:40:15 +01:00
kssenii
fada92173b Support Point data type for MySQL 2024-01-11 19:19:08 +01:00
Yakov Olkhovskiy
84809b36c3
Merge pull request #58475 from ClickHouse/revert-58450-revert-56064-feature-server-iface-metrics
Second attempt: Use CH Buffer for HTTP out stream, add metrics for interfaces
2024-01-11 12:06:59 -05:00
Igor Nikonov
8ba9b4a7ef
Merge pull request #57235 from ClickHouse/pr-custom-key-failover
Parallel replicas custom key: skip unavailable replicas
2024-01-11 12:22:32 +01:00
Konstantin Bogdanov
7f4ccebea4
Merge pull request #58550 from danipozo/hide-credentials-in-base-backup-name
Hide credentials in system.backup_log base_backup_name column
2024-01-11 03:42:17 +03:00
Igor Nikonov
8d4a395afa Merge remote-tracking branch 'origin/master' into pr-custom-key-failover 2024-01-10 20:28:04 +00:00
Igor Nikonov
e9fc869254 Fix test flakiness with hedge connections enabled 2024-01-10 15:59:48 +00:00
Yakov Olkhovskiy
2e124b19b8
Merge branch 'master' into revert-58450-revert-56064-feature-server-iface-metrics 2024-01-10 09:05:18 -05:00
Azat Khuzhin
2cf74aea5c Fix test_replicated_database::test_startup_without_zk flakiness
CI founds [1]

    E           assert 'ZooKeeper' in "Received exception from server (version 24.1.1):\nCode: 999. DB::Exception: Received from 172.16.1.6:9000. Coordination::Exception. Coordination::Exception: Coordination error: Connection loss, path /clickhouse/databases/startup. Stack trace:\n\n0. ? @ 0x000000000c734f1b in /usr/bin/clickhouse\n1. ? @ 0x000000000fbef3f4 in /usr/bin/clickhouse\n2. ? @ 0x000000000fbeebc8 in /usr/bin/clickhouse\n3. ? @ 0x0000000012ded0f9 in /usr/bin/clickhouse\n4. ? @ 0x0000000012deaa71 in /usr/bin/clickhouse\n5. ? @ 0x00000000107e51e8 in /usr/bin/clickhouse\n6. ? @ 0x00000000107eb87a in /usr/bin/clickhouse\n7. ? @ 0x0000000011213ec3 in /usr/bin/clickhouse\n8. ? @ 0x00000000111e9acf in /usr/bin/clickhouse\n9. ? @ 0x000000001120e85e in /usr/bin/clickhouse\n10. ? @ 0x000000001174f0a2 in /usr/bin/clickhouse\n11. ? @ 0x0000000011748c9a in /usr/bin/clickhouse\n12. ? @ 0x00000000126fb5a9 in /usr/bin/clickhouse\n13. ? @ 0x00000000127130f9 in /usr/bin/clickhouse\n14. ? @ 0x000000001517bf12 in /usr/bin/clickhouse\n15. ? @ 0x000000001517cd11 in /usr/bin/clickhouse\n16. ? @ 0x00000000152740c7 in /usr/bin/clickhouse\n17. ? @ 0x00000000152726bc in /usr/bin/clickhouse\n18. ? @ 0x00007f7f36485ac3\n19. ? @ 0x00007f7f36517660\n. (KEEPER_EXCEPTION)\n(query: CREATE DATABASE startup ENGINE = Replicated('/clickhouse/databases/startup', 'shard1', 'replica1');)\n"

  [1]: https://s3.amazonaws.com/clickhouse-test-reports/58610/441a3fbb9b1c088f528841727f013b829f63fc8a/integration_tests__release__[3_4]/integration_run_test_replicated_database_test_py_0.log

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2024-01-10 14:44:29 +03:00
Alexander Tokmakov
b954667ae9
Update test.py 2024-01-10 11:07:13 +01:00
Robert Schulze
4cacfa6b43
Merge pull request #57509 from jinjunzh/qatzstd_main
Add Intel-QAT-based `ZSTD_QAT` compression codec
2024-01-10 10:31:39 +01:00
Kseniia Sumarokova
1b517105db
Merge pull request #57323 from valbok/mysql-binlog-client
MaterializedMySQL: Introduce MySQL Binlog Client
2024-01-10 09:08:13 +01:00
Robert Schulze
ced9e93ac6
Merge remote-tracking branch 'rschu1ze/master' into qatzstd_main 2024-01-09 20:36:02 +00:00
Sema Checherinda
7c7e72c4b7
Merge pull request #58573 from ClickHouse/chesema-s3-client-creation
fix and test that S3Clients are reused
2024-01-09 13:28:20 +01:00
Sema Checherinda
6dbf7c967c
Merge pull request #57663 from ClickHouse/CheSema-remote-blobs-naming-2
generate object storage key by template
2024-01-09 13:26:56 +01:00
Кирилл Гарбар
85163994dc Merge remote-tracking branch 'ClickHouse/master' 2024-01-09 04:51:51 +03:00
Кирилл Гарбар
bebb2c9dca Common test functions, check flags are removed 2024-01-09 04:22:40 +03:00
Igor Nikonov
7fd851b4e9 Merge remote-tracking branch 'origin/master' into pr-custom-key-failover 2024-01-08 20:43:20 +00:00
Dani Pozo
d8f4b5fd13 Fix black formatting 2024-01-08 12:47:58 +01:00
Dani Pozo
b42eaf6c62 Add test for hiding credentials in system.backups table 2024-01-08 12:20:32 +01:00
Кирилл Гарбар
c7b14d190e Remove flags directory 2024-01-08 07:09:38 +03:00
robot-clickhouse
29173a602e Automatic style fix 2024-01-08 02:39:19 +00:00
Yakov Olkhovskiy
45ac4d2b67 Merge remote-tracking branch 'origin/master' into revert-58450-revert-56064-feature-server-iface-metrics 2024-01-07 15:23:46 +00:00
Robert Schulze
53965bb9f1
Merge remote-tracking branch 'rschu1ze/master' into qatzstd_main 2024-01-07 14:11:22 +00:00
Sema Checherinda
d5f86f671d fix and test that S3Clients reused 2024-01-07 02:19:06 +01:00
robot-clickhouse
4a473c37ec Automatic style fix 2024-01-06 15:55:20 +00:00
Alexey Milovidov
2751b351cb Fix test 2024-01-06 16:44:38 +01:00
Yakov Olkhovskiy
9ca4be5d91 Merge remote-tracking branch 'origin/master' into revert-58450-revert-56064-feature-server-iface-metrics 2024-01-06 00:24:31 +00:00
robot-clickhouse
630ee67262 Automatic style fix 2024-01-05 23:02:58 +00:00
Yakov Olkhovskiy
0964bf3638 add clickhouse-library-bridge logging for test_catboost_evaluate 2024-01-05 22:51:17 +00:00
Nikolay Degterinsky
56aa835954
Merge pull request #58444 from bharatnc/ncb/support-hints-for-db-engines
support hints for database engines
2024-01-05 17:12:44 +01:00
Sergei Trifonov
8d6b012223
Merge pull request #57877 from azat/merges-mutations-throttling
Add ability to throttle merges/mutations
2024-01-05 17:08:50 +01:00
Sema Checherinda
2861cc70ba fix style 2024-01-05 13:42:30 +00:00
Sema Checherinda
8ab4aa3423 enable randomization setting 2024-01-05 13:42:30 +00:00
Sema Checherinda
53f36f8b21 object storage key template 2024-01-05 13:42:30 +00:00
Alexander Tokmakov
d2752f4b66
Merge pull request #57603 from evillique/allow-not-resolving-hostnames
Allow avoiding resolving hostnames in DDLWorker
2024-01-04 15:46:28 +01:00
Alexander Tokmakov
972756486a
Merge pull request #58350 from ClickHouse/rdb_dont_wait_inactive
An option to avoid waiting for inactive Replicated database replicas
2024-01-04 14:50:11 +01:00
Alexander Tokmakov
10e3ae9e68 Merge branch 'master' into allow-not-resolving-hostnames 2024-01-04 13:22:15 +01:00
robot-clickhouse
224a58ddc5 Automatic style fix 2024-01-04 10:30:44 +00:00
János Benjamin Antal
1a1fa07128 Add missing asserts 2024-01-04 10:18:04 +00:00
János Benjamin Antal
2517d23e27 Add tests 2024-01-04 10:14:41 +00:00
Nikita Taranov
66d2db5283
New parallel replicas coordinator implementation (#57968) 2024-01-04 00:27:04 +01:00
Bharat Nallan Chakravarthy
d63765fb19 Merge upstream/master into ncb/support-hints-for-db-engines 2024-01-03 08:44:11 -08:00
Alexander Tokmakov
b1688b1953 Merge branch 'master' into rdb_dont_wait_inactive 2024-01-03 17:03:01 +01:00
Val Doroshchuk
5c221d123d MaterializedMySQL: Introduce MySQL Binlog Client
One binlog connection for many databases.

Suggesting to disable this feature by default for now. It should be explicitly enabled by SETTINGS use_binlog_client=1.
But if you would permanently enable it in MaterializedMySQLSettings, it should keep old behavior and all tests should pass too.

1. Introduced `IBinlog` and its impl to read the binlog events from socket - `BinlogFromSocket`, or file - `BinlogFromFile`. Based on prev impl of `EventBase` and the same old binlog parsers. It fully keeps BC with old version. Fixed `./check-mysql-binlog` to test new impl.
2. Introduced `BinlogEventsDispatcher`, it reads the event from the source `IBinlog` and sends it to currently attached `IBinlog` instances.
3. Introduced `BinlogClient`, which is used to group a list of `BinlogEventsDispatcher` by MySQL binlog connection which is defined by `user:password@host:port`. All dispatchers with the same binlog position should be merged to one.
4. Introduced `BinlogClientFactory`, which is a singleton and it is used to track all binlogs created over the instance.
5. Introduced `use_binlog_client` setting to `MaterializedMySQL`, which forces to reuse a `BinlogClient` if it already exists in `BinlogClientCatalog` or create new one. By default, it is disabled.
6. Introduced `max_bytes_in_binlog_queue` setting to define the limit of bytes in binlog's queue of events. If bytes in the queue increases this limit, `BinlogEventsDispatcher` will stop reading new events from source `IBinlog` until the space for new events will be freed.
7. Introduced `max_milliseconds_to_wait_in_binlog_queue` setting to define max ms to wait when the max bytes exceeded.
7. Introduced `max_milliseconds_to_wait_in_binlog_queue` setting to define max ms to wait when the max bytes exceeded.
8. Introduced `max_bytes_in_binlog_dispatcher_buffer` setting to define max bytes in the binlog dispatcher's buffer before it is flushed to attached binlogs.
9. Introduced `max_flush_milliseconds_in_binlog_dispatcher` setting to define max milliseconds in the binlog dispatcher's buffer to wait before it is flushed to attached binlogs.
10. Introduced `system.mysql_binlogs` system table, which shows a list of active binlogs.
11. Introduced `UnparsedRowsEvent` and `MYSQL_UNPARSED_ROWS_EVENT`, which defines that an event is not parsed and should be explicitly parsed later.
12. Fixed bug when not possible to apply DDL since syntax error or unsupported SQL.

@larspars is the author of following:
`GTIDSets::contains()`
`ReplicationHelper`
`shouldReconnectOnException()`
2024-01-03 15:26:09 +01:00
Kruglov Pavel
570d1c013b
Merge pull request #57053 from Avogar/better-parsing-exceptions
Better exception messages in input formats
2024-01-03 14:47:51 +01:00
Bharat Nallan Chakravarthy
2da0a30626 add a test 2024-01-02 13:52:52 -08:00
Smita Kulkarni
4122de9721 Updated tests and added settings 2024-01-02 20:19:01 +01:00
Nikolay Degterinsky
debbeb505d
Merge pull request #58409 from ClickHouse/evillique-patch-1
Fix test_user_valid_until
2024-01-02 17:06:11 +01:00
avogar
5f6a806d1c Fix style 2024-01-02 15:43:44 +00:00
avogar
0844528ec1 Fix tests 2024-01-02 15:22:42 +00:00
Kruglov Pavel
e69bda7f08
Merge pull request #57438 from Avogar/fix-resetting-buffers
Fix working with read buffers in StreamingFormatExecutor
2024-01-02 15:34:35 +01:00
Nikolay Degterinsky
004730fe87
Fix test_user_valid_until 2024-01-02 14:50:28 +01:00
Alexey Milovidov
78523639e0 Remove function 2023-12-30 14:52:54 +01:00
avogar
f20917048e Fix kafka tests 2023-12-29 17:02:46 +00:00
Alexander Tokmakov
bfc10bd234 an option to avoid waiting for inactive Replicated db replicas 2023-12-29 15:19:16 +01:00
Igor Nikonov
208a9193f6 Merge remote-tracking branch 'origin/master' into pr-custom-key-failover 2023-12-28 21:28:36 +00:00
Kruglov Pavel
9389ea8745 Fix style 2023-12-28 18:47:57 +00:00
Kruglov Pavel
69f6147499 Fix style 2023-12-28 18:47:57 +00:00
avogar
c6085b3eb9 Fix working with read buffers in StreamingFormatExecutor 2023-12-28 18:47:57 +00:00
Michael Kolupaev
609b2c216c Fix some of the CI 2023-12-28 17:56:05 +00:00
koloshmet
49367186e3 fix fix fix 2023-12-28 17:56:04 +00:00
Kseniia Sumarokova
8e8fd84cb7
Merge pull request #58293 from ClickHouse/fix-s3-queue-test
Fix test_storage_s3_queue/test.py::test_drop_table
2023-12-28 17:18:11 +01:00
Alexander Tokmakov
38fe70c68a
Revert "Refreshable materialized views (takeover)" 2023-12-28 13:12:20 +01:00
kssenii
529a1d67cb Fix 2023-12-28 12:48:56 +01:00
Smita Kulkarni
bf60a06a11 Merge branch 'master' into Azure_backup 2023-12-28 11:51:49 +01:00
Alexey Milovidov
40a5dbdeba
Merge branch 'master' into mv 2023-12-28 03:16:27 +01:00
Кирилл Гарбар
764791809b Test for unusual zk path 2023-12-28 03:33:56 +03:00
Alexey Milovidov
30f6d03b37
Merge pull request #57977 from bianpengyuan/add-s3-read-only-setting
Add a setting to specify s3 disk is read only.
2023-12-28 01:24:50 +01:00
Кирилл Гарбар
697b4d5fa1 No need to expand uuid macro 2023-12-28 02:35:13 +03:00
Alexey Milovidov
5406c79c89
Merge pull request #58264 from ClickHouse/fs-cache-better-multi-cache-configuration
Make fs caches with the same path use the same cache objects
2023-12-28 00:10:15 +01:00
Кирилл Гарбар
6f79f98719 Throw on ordinary db engine 2023-12-28 00:54:10 +03:00
Michael Kolupaev
673743e2ac Fix some of the CI 2023-12-27 20:24:55 +00:00
koloshmet
808cb0fa05 fix fix fix 2023-12-27 20:24:54 +00:00
Sema Checherinda
617ee5abaf
Merge pull request #57551 from bharatnc/ncb/async-metrics-pri-key-size
add asynchronous metrics for total primary key size in memory and allocated
2023-12-27 20:24:20 +01:00
Pengyuan Bian
26b553c3b6 format. 2023-12-27 18:27:36 +00:00
kssenii
041c5d4219 Make caches with the same path use the same cache objects 2023-12-27 18:37:43 +01:00
Pengyuan Bian
3dcc1f5448 update. 2023-12-27 07:54:18 +00:00
Igor Nikonov
bee15325fc Merge remote-tracking branch 'origin/master' into pr-custom-key-failover 2023-12-26 21:56:46 +00:00
Pengyuan Bian
2765acfe45 wip test. 2023-12-26 07:50:46 +00:00
Azat Khuzhin
837f4ea676 Add ability to throttle merges/mutations
Main motivation was to has an ability to throttle background tasks, to
avoid affecting queries.

To new server settings had been added for this:
- max_mutations_bandwidth_for_server
- max_merges_bandwidth_for_server

Note, that they limit only reading, since usually you will not write
more data then you read, but sometimes it is possible in case of ALTER
UPDATE.

But for now, to keep things simple, I decided to limit this with only
2 settings instead of 4.

Note, that if the write throttling will be needed, then they can use the
same settings, and just create new throttler for write.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-12-24 22:31:49 +01:00
Azat Khuzhin
6ed9b53d1f Refactor test_throttling slightly for upcoming tests
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-12-24 22:29:43 +01:00
Alexey Milovidov
817d0c6553
Merge pull request #58202 from vitlibar/explicit-finalize-in-zip-archive-writer
Explicit finalize in ZipArchiveWriter
2023-12-24 03:30:21 +01:00
Vitaly Baranov
e1a136b791 Explicit finalize() function in ZipArchiveWriter.
Simplify too complicated code in ZipArchiveWriter.
2023-12-24 00:33:59 +01:00
Smita Kulkarni
3e22f29b45 Fixed parameters 2023-12-23 11:40:58 +01:00
jinjunzh
aaf98f0ed7 refine setting with enable_zstd_qat_codec 2023-12-22 14:24:15 -05:00
Bharat Nallan Chakravarthy
d2f0fe28dc fix test query 2023-12-22 08:48:14 -08:00
Raúl Marín
24f952b2b8
Merge pull request #57986 from Algunenano/zk_retries_quorum
Support keeper failures in quorum check
2023-12-22 11:10:04 +01:00
Bharat Nallan Chakravarthy
62b28d365c Merge with upstream/master 2023-12-21 17:04:25 -08:00
Bharat Nallan Chakravarthy
929bc01281 better test with retries 2023-12-21 11:20:16 -08:00
Raúl Marín
6d9da8edd5 Merge remote-tracking branch 'blessed/master' into zk_retries_quorum 2023-12-21 17:03:29 +01:00
Mikhail f. Shiryaev
177f8f7d0d
Create a single point of true for integration tests images 2023-12-21 16:09:56 +01:00
Igor Nikonov
db87ea0b86 Merge remote-tracking branch 'origin/master' into pr-custom-key-failover 2023-12-20 20:48:51 +00:00
Kruglov Pavel
ca7001d70f
Merge pull request #57850 from shanielh/feature/s3-session-token
Feature/s3 session token
2023-12-20 17:22:09 +01:00
jsc0218
0a9475abc7
Merge pull request #57874 from joelynch/joelynch/host-psql
bugfix: addresses_expr ignored for psql named collections
2023-12-19 15:24:23 -05:00
Кирилл Гарбар
64073f09b6 Support converting tables with explicit engine arguments 2023-12-19 20:57:25 +03:00
Bharat Nallan Chakravarthy
6273c5920a fix tests 2023-12-19 08:26:54 -08:00
Raúl Marín
b458c77340 Fix integration test 2023-12-19 14:36:03 +01:00
Shani Elharrar
5f38e1d944 S3 Session Tokens: Added tests 2023-12-19 10:50:19 +02:00
robot-clickhouse
2be806618e Automatic style fix 2023-12-18 22:07:00 +00:00
Nikita Mikhaylov
6360b76792 Merge branch 'master' of github.com:ClickHouse/ClickHouse into remove-the-limit-for-connections-per-endpoint 2023-12-18 21:49:31 +00:00
Nikita Mikhaylov
ddf5da225d Fix integration test 2023-12-18 21:39:59 +00:00
pufit
36d1905881
Merge pull request #57752 from chizhonkova/refactor_sql_udf_loader
Refactoring for UserDefinedSQLObjectsLoader
2023-12-18 16:08:46 -05:00
Bharat Nallan Chakravarthy
73f71cc9fc add a test 2023-12-18 12:39:43 -08:00
Raúl Marín
c4f2511596
Merge pull request #57895 from Algunenano/fix_azure
Fix test_unset_skip_unavailable_shards
2023-12-18 12:01:57 +01:00
Кирилл Гарбар
f2a1ab522e Merge remote-tracking branch 'ClickHouse/master' 2023-12-18 13:45:28 +03:00
joelynch
403d9c809d
fix test for postgresql addresses_expr 2023-12-18 10:41:12 +01:00
Igor Nikonov
ec1539728d Fix load balancing for hedged connections 2023-12-16 21:17:01 +00:00
Кирилл Гарбар
1ca059594f Format tests 2023-12-16 13:48:25 +03:00
Igor Nikonov
a735820400 Merge remote-tracking branch 'origin/master' into pr-custom-key-failover 2023-12-15 21:14:43 +00:00
Igor Nikonov
da2dc30056 Fix test: use different log_comment for each test instance 2023-12-15 21:02:55 +00:00
Alexey Milovidov
7e55eab1c3 Revert "Merge pull request #56573 from mkmkme/mkmkme/reload-config"
This reverts commit 3067ca64df, reversing
changes made to 1e46460910.
2023-12-15 19:28:25 +01:00
Sema Checherinda
0dfe530a7f
Merge pull request #56744 from MikhailBurdukov/native_copy_for_s3_disks
Enabled s3 `copyObject` for copy between s3 disks.
2023-12-15 16:05:16 +01:00
Nikita Mikhaylov
1780671443
ClickHouse Cloud promotion (#57638)
Co-authored-by: Alexander Sapin <alesapin@gmail.com>
2023-12-15 15:23:23 +01:00
Yarik Briukhovetskyi
325374c68b
Merge pull request #57342 from bharatnc/ncb/better-hint-if-table-doesnt-exist
provide a better hint if a table doesn't exist
2023-12-15 12:35:13 +01:00
Raúl Marín
1e8397f437 Fix test_unset_skip_unavailable_shards 2023-12-15 12:18:40 +01:00
Кирилл Гарбар
8d1dfebb4f Refactor tests 2023-12-14 23:01:50 +03:00
Bharat Nallan Chakravarthy
daef2febf3 Merge with 'upstream/master' 2023-12-14 11:19:44 -08:00
robot-clickhouse
217570a165 Automatic style fix 2023-12-14 18:49:32 +00:00
Igor Nikonov
1370a8c6c4 Merge remote-tracking branch 'origin/master' into pr-custom-key-failover 2023-12-14 18:29:38 +00:00
Igor Nikonov
f8dc6e127e Update tests 2023-12-14 18:28:25 +00:00
joelynch
b4bef78e63
bugfix: addresses_expr ignored for psql named collections 2023-12-14 17:17:14 +01:00
robot-clickhouse
b05c73b3ae Automatic style fix 2023-12-14 13:19:46 +00:00
kssenii
a4a264c785 Fix 2023-12-14 14:04:17 +01:00
Igor Nikonov
42d0afef1f
Merge pull request #57749 from ClickHouse/pr-perf-improve-hope
Parallel replicas: announcement response handling improvement
2023-12-14 11:48:51 +01:00
Konstantin Bogdanov
808b4a0213
Merge pull request #57628 from azat/config-nodes-override
Allow nodes in config with from_env/from_zk and non empty element with replace=1
2023-12-14 13:20:46 +03:00
jsc0218
78773157b6
Merge pull request #57436 from ryanmjacobs/fix_issue_43186_pg_quote_array_ndims
Fix several issues regarding PostgreSQL `array_ndims` usage.
2023-12-13 13:41:43 -05:00
Nikita Mikhaylov
8372c70958 Merge branch 'master' of github.com:ClickHouse/ClickHouse into remove-the-limit-for-connections-per-endpoint 2023-12-13 18:29:56 +00:00
alesapin
230e1fe6c7
Merge pull request #57809 from ClickHouse/fix-flaky-test_parallel_replicas_over_distributed
Fix flaky test_parallel_replicas_over_distributed
2023-12-13 17:11:41 +01:00
Alexey Milovidov
54698cd2de
Merge pull request #55876 from ClickHouse/keeper-http-readiness-endpoint
Keeper HTTP readiness endpoint
2023-12-13 17:07:57 +01:00
Igor Nikonov
72f1c415f1 Merge remote-tracking branch 'origin/master' into pr-custom-key-failover 2023-12-13 15:03:05 +00:00
Igor Nikonov
c165be76ab
Parallel replicas: friendly settings (#57542) 2023-12-13 14:42:06 +01:00
Igor Nikonov
2a0a5f755c Comment to about possible query results 2023-12-13 12:29:57 +00:00
robot-clickhouse
7d9e9fd42e Automatic style fix 2023-12-13 11:38:41 +00:00
Igor Nikonov
7762beaf6c Fix: w/o replicas sync query result can vary 2023-12-13 11:23:53 +00:00
Kseniia Sumarokova
b1325450e9
Merge pull request #57754 from ClickHouse/fix-loading-dependent-table-materialized-postgresql
Table engine MaterializedPostgreSQL fix dependency loading
2023-12-13 12:17:01 +01:00
Igor Nikonov
5abeadf20f Fix: stop merges, otherwise test can be flaky 2023-12-13 08:54:44 +00:00
robot-clickhouse
ea86b33d3b Automatic style fix 2023-12-12 21:59:08 +00:00
Igor Nikonov
3333a7f219 Test cleanup 2023-12-12 21:39:08 +00:00
Igor Nikonov
616d054dee Merge remote-tracking branch 'origin/master' into pr-perf-improve-hope 2023-12-12 21:27:31 +00:00
Igor Nikonov
7f4a028196 Test and fix 2023-12-12 21:26:58 +00:00
Alexander Tokmakov
3482224f83 Merge branch 'master' into keeper-http-readiness-endpoint 2023-12-12 21:19:57 +01:00
Кирилл Гарбар
a6549c106a Tests for engine convertation 2023-12-12 21:45:00 +03:00
robot-clickhouse
f2336ff025 Automatic style fix 2023-12-12 14:43:44 +00:00
kssenii
e77cb18d18 Better test 2023-12-12 15:30:17 +01:00
Natalya Chizhonkova
3d51192bad Refactoring for UserDefinedSQLObjectsLoader 2023-12-12 17:03:49 +03:00
Kruglov Pavel
8a447bf57c
Merge pull request #55892 from Avogar/schema-inference-union
Add 'union' mode for schema inference
2023-12-12 15:02:06 +01:00
Kruglov Pavel
b33f124555
Merge pull request #57538 from MikhailBurdukov/ignore_on_cluster_for_grants
Ignore ON CLUSTER clause in grant/revoke queries for management of replicated access entities.
2023-12-12 14:59:58 +01:00
Konstantin Bogdanov
511cfb393d
Remove time.sleep from test 2023-12-12 12:29:00 +01:00
robot-clickhouse
b49452fb45 Automatic style fix 2023-12-12 10:44:55 +00:00
Konstantin Bogdanov
8fe2cd1a7e
Update tests 2023-12-12 11:35:17 +01:00
Igor Nikonov
fd92cb4c32
Merge pull request #57757 from ClickHouse/fix-flaky-test_parallel_replicas_distributed
Fix flaky test_parallel_replicas_distributed_read_from_all
2023-12-12 10:43:19 +01:00
Konstantin Bogdanov
1f9c7336a9
Fix test helper naming 2023-12-12 09:32:16 +01:00
robot-clickhouse
e95041a3af Automatic style fix 2023-12-11 21:40:23 +00:00
Igor Nikonov
da43d49ffc Fix: sync is failed with stopped merges 2023-12-11 21:30:25 +00:00
alesapin
cbae29d26a
Merge pull request #57699 from hanfei1991/hanfei/keeper_soft_limit_ratio
support keeper memory soft limit ratio
2023-12-11 21:43:24 +01:00
Igor Nikonov
c007025ee3 Fix flaky test_parallel_replicas_distributed_read_from_all 2023-12-11 20:05:26 +00:00
Ryan Jacobs
40da568ef9
Merge branch 'master' into fix_issue_43186_pg_quote_array_ndims 2023-12-11 11:17:38 -08:00
kssenii
05e56f76ed Merge remote-tracking branch 'origin/master' into fix-loading-dependent-table-materialized-postgresql 2023-12-11 20:17:26 +01:00
Kseniia Sumarokova
907d0a79eb
Merge pull request #57568 from ClickHouse/fix-materialized-pg-issue-with-generated-columns
MaterializedPostgreSQL: Support "generated columns" and default values
2023-12-11 19:57:41 +01:00
Kruglov Pavel
3b1ffbd77c
Merge branch 'master' into schema-inference-union 2023-12-11 18:59:15 +01:00
kssenii
48ce040621 Fix 2023-12-11 17:30:23 +01:00
kssenii
a06179da00 Merge remote-tracking branch 'origin/master' into fix-materialized-pg-issue-with-generated-columns 2023-12-11 11:15:00 +01:00
robot-clickhouse
3adb83e406
Automatic style fix 2023-12-11 10:19:49 +01:00
Konstantin Bogdanov
89ba4a8452
Add HTTP readiness endpoint 2023-12-11 10:19:42 +01:00
MikhailBurdukov
119e451967 Merge branch 'master' into native_copy_for_s3_disks 2023-12-11 07:25:20 +00:00
Vitaly Baranov
4aba3ec685 Add some comments. 2023-12-11 00:56:40 +01:00
Vitaly Baranov
e2a7f4004f Fix test test_create_or_drop_tables_during_backup. 2023-12-11 00:56:28 +01:00
Alexey Milovidov
644ef7b63f
Revert "Fix RWLock inconsistency after write lock timeout" 2023-12-10 23:51:59 +03:00
Igor Nikonov
f7e6a17052 Merge remote-tracking branch 'origin/master' into pr-custom-key-failover 2023-12-10 20:21:47 +00:00
Alexey Milovidov
98f9d2a5fc
Merge pull request #57714 from azat/tests/fix-test_dictionaries_update_and_reload
Fix test_dictionaries_update_and_reload/test.py::test_reload_while_loading flakiness
2023-12-10 17:55:50 +01:00
Alexey Milovidov
d119052895
Merge pull request #51858 from ClickHouse/CurtizJ-fix-ttl-non-deterministic
Merging #37286
2023-12-10 17:50:36 +01:00
Vitaly Baranov
a058a26b7e
Merge pull request #57454 from vitlibar/fix-rwlock
Fix RWLock inconsistency after write lock timeout
2023-12-10 14:09:29 +01:00
Azat Khuzhin
ce9fd2c57a Fix test_dictionaries_update_and_reload/test.py::test_reload_while_loading flakiness
On CI this test fails due to start_time == prev_start_time:

            start_time, duration = get_loading_start_time("slow"), get_loading_duration("slow")
    >       assert start_time > prev_start_time
    E       assert time.struct_time(tm_year=2023, tm_mon=12, tm_mday=9, tm_hour=23, tm_min=42, tm_sec=2, tm_wday=5, tm_yday=343, tm_isdst=-1) > time.struct_time(tm_year=2023, tm_mon=12, tm_mday=9, tm_hour=23, tm_min=42, tm_sec=2, tm_wday=5, tm_yday=343, tm_isdst=-1)

The reason I guess is that there is sleep(0.5), while the
loading_start_time is DateTime not DateTime64, so you cannot distinguish
values if the difference is less then one second

CI: https://s3.amazonaws.com/clickhouse-test-reports/57710/7af1c0885daaf1e41470c5fdd92abfc7b6b2befc/integration_tests__asan__[3_4].html
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-12-10 08:36:10 +01:00
Alexey Milovidov
33b0ec7d55 Merge branch 'master' into CurtizJ-fix-ttl-non-deterministic 2023-12-10 01:21:43 +01:00
Kseniia Sumarokova
69adfffa02
Merge pull request #57588 from ClickHouse/materialized-postgres-escape-publication
Add a test for #55251
2023-12-09 12:14:28 +01:00
Han Fei
881e58dfcb support soft limit ratio 2023-12-08 21:48:27 +01:00
avogar
ee7af95bc0 Merge branch 'master' of github.com:ClickHouse/ClickHouse into schema-inference-union 2023-12-08 20:29:28 +00:00
Nikolay Degterinsky
10bfd054d8 Fix style & review 2023-12-08 19:40:09 +00:00
Nikita Mikhaylov
5251548753 Fix tests and build 2023-12-08 17:28:26 +00:00
alesapin
dc7347e0ba
Merge pull request #57271 from hanfei1991/hanfei/keeper_soft_limit
support memory soft limit for keeper
2023-12-08 16:12:47 +01:00
Kseniia Sumarokova
928993f2bd
Merge branch 'master' into fix-materialized-pg-issue-with-generated-columns 2023-12-08 15:37:29 +01:00
Kseniia Sumarokova
d0c5dd3b87
Merge branch 'master' into materialized-postgres-escape-publication 2023-12-08 15:29:10 +01:00
zhongyuankai
7b0f8d44e8
Make DirectoryMonitor handle cluster node list change (#42826) 2023-12-08 14:41:51 +01:00
kssenii
0c40465fec Test 2023-12-08 12:46:59 +01:00
Kseniia Sumarokova
f41d3be4b2
Fix bad merge 2023-12-08 11:28:14 +01:00
Kseniia Sumarokova
de7abe2341
Merge branch 'master' into materialized-postgres-escape-publication 2023-12-08 10:53:39 +01:00
Kseniia Sumarokova
1ec2d29302
Merge pull request #57567 from ClickHouse/kssenii-patch-8
Fix possible segfault in PostgreSQLSource
2023-12-08 10:51:08 +01:00
vdimir
6df715f5d7
Merge pull request #57627 from ClickHouse/prometheus-all-errors
Expose the total number of errors to Prometheus endpoint
2023-12-08 10:42:01 +01:00
robot-clickhouse
129e0349b7 Automatic style fix 2023-12-08 02:33:04 +00:00
Alexey Milovidov
d810bea8e3 Fix test 2023-12-08 03:17:13 +01:00
Alexey Milovidov
01135211e9 Fix test 2023-12-08 03:12:54 +01:00
Alexey Milovidov
ce9e134043 Merge branch 'master' into CurtizJ-fix-ttl-non-deterministic 2023-12-08 03:10:09 +01:00
Vitaly Baranov
b4cac92e28 Add some comments. 2023-12-07 22:18:49 +01:00
robot-clickhouse
f7c3304045 Automatic style fix 2023-12-07 19:42:33 +00:00
Nikita Mikhaylov
0b5b042153 Add test 2023-12-07 20:09:44 +01:00
Azat Khuzhin
2c6604ec24 Allow nodes in config with from_env/from_zk and non empty element with replace=1
Such nodes in config are useful as a nodes with default values, that can
be overwritten from ZooKeeper/env.

So after this patch the following is valid, and is interpreted as
default value 86400, and can be overwritten via env:

```xml
<asynchronous_metrics_update_period_s replace="1" from_env="CH_ASYNCHRONOUS_METRICS_UPDATE_PERIOD_S">86400</asynchronous_metrics_update_period_s>
```

While the following is not:

```xml
<asynchronous_metrics_update_period_s             from_env="CH_ASYNCHRONOUS_METRICS_UPDATE_PERIOD_S">86400</asynchronous_metrics_update_period_s>
```

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-12-07 18:01:40 +01:00
Igor Nikonov
94fb23af0b Merge remote-tracking branch 'origin/master' into pr-custom-key-failover 2023-12-07 11:52:40 +00:00
kssenii
188d4d66c9 Merge with master 2023-12-07 12:45:36 +01:00
kssenii
0486631f4c Fxi 2023-12-07 12:44:00 +01:00
robot-clickhouse
6cf7f75fb4 Automatic style fix 2023-12-07 10:11:44 +00:00
kssenii
246eeee2df Merge remote-tracking branch 'origin/master' into materialized-postgres-escape-publication 2023-12-07 10:48:15 +01:00
kssenii
56b20b6d36 Merge remote-tracking branch 'origin/master' into add-a-test-for-issue-with-table-override 2023-12-07 10:46:08 +01:00
kssenii
bbddd330c6 Add forgotten change 2023-12-07 10:46:00 +01:00
Kseniia Sumarokova
b3f22449f6
Merge pull request #57515 from ClickHouse/fix-materialized-pg-issue-with-incorrect-connection-options
MaterializedPostgreSQL: fix issue #41922, add test for #41923
2023-12-07 10:42:14 +01:00
Nikolay Degterinsky
995b51ef73 Allow avoiding resolving hostnames in DDLWorker 2023-12-07 03:43:26 +00:00
Ryan Jacobs
ca790b6eec
apply python style check 2023-12-06 19:26:50 -08:00
Ryan Jacobs
7d41426935
test_storage_postgresql: Add test_postgres_array_ndim_error_messges()
Tests:

1. View with array column cannot be empty --> error message
2. View cannot have empty array --> error message
3. View cannot have NULL array value --> error message
4. Ensures PG identifiers that require quoting do not crash ClickHouse.

These apply to views that contain arrays.
2023-12-06 18:39:09 -08:00
kssenii
f461b5b2d0 Add a test 2023-12-06 23:39:34 +01:00
Alexey Milovidov
05bc8ef1e0
Merge pull request #57548 from ClickHouse/s3queue_production_ready
`S3Queue` is production ready
2023-12-06 22:21:08 +01:00
Ryan Jacobs
119c2864a0
test_storage_postgresql: mixed-case identifier on array column 2023-12-06 12:59:58 -08:00
robot-clickhouse
4dab2515dc Automatic style fix 2023-12-06 19:46:41 +00:00
kssenii
6321576ff8 Add a test 2023-12-06 20:27:41 +01:00
robot-clickhouse
ec5726c1e6 Automatic style fix 2023-12-06 18:58:03 +00:00
kssenii
6368d060b7 Add a test 2023-12-06 19:39:18 +01:00
vdimir
8f32293104
Merge pull request #57416 from ClickHouse/vdimir/fix_test_encrypted_disk
Change type of s3_cache in test_encrypted_disk
2023-12-06 14:37:40 +01:00
Alexey Milovidov
f8de7f6bc2 Merge branch 'master' into s3queue_production_ready 2023-12-06 12:25:29 +01:00
Kseniia Sumarokova
e02883cd4a
Merge branch 'master' into fix-materialized-pg-issue-with-incorrect-connection-options 2023-12-06 12:16:04 +01:00
Igor Nikonov
b4922eece4 Merge remote-tracking branch 'origin/master' into pr-custom-key-failover 2023-12-06 10:44:30 +00:00
Alexey Milovidov
76bbeaa676 S3Queue is production ready 2023-12-05 21:10:19 +01:00
Kseniia Sumarokova
f56fe785e8
Update test.py 2023-12-05 18:40:56 +01:00
MikhailBurdukov
c41511e51c ignore_on_cluster_for_grants 2023-12-05 15:11:32 +00:00
Nikolay Degterinsky
a776a5cac5
Merge pull request #57483 from joelynch/stop-listen-tcp-secure
bugfix: correctly parse SYSTEM STOP LISTEN TCP SECURE
2023-12-05 15:25:34 +01:00
robot-clickhouse
fee17abf7a Automatic style fix 2023-12-05 12:40:11 +00:00
kssenii
0d313acd03 Fix 2023-12-05 13:22:35 +01:00
Igor Nikonov
71204c2a82 Merge remote-tracking branch 'origin/master' into pr-custom-key-failover 2023-12-05 12:00:45 +00:00
Alexey Milovidov
1f752b79db
Merge pull request #55894 from bharatnc/ncb/hostname-system-log-tables
add hostname column to system log tables
2023-12-05 06:42:43 +01:00
Alexey Milovidov
10d65a1ade
Merge pull request #55559 from azat/s3-fix-excessive-reads
Add ability to disable checksums for S3 to avoid excessive input file read
2023-12-05 06:34:21 +01:00
Bharat Nallan Chakravarthy
942802f308 few more test cases 2023-12-04 14:01:43 -08:00
joelynch
b312a9f4ee
bugfix: parse system stop listen tcp secure 2023-12-04 19:56:35 +01:00
Alexey Milovidov
02439eee58
Merge pull request #49351 from ClickHouse/async-loader-integration
Asynchronous loading of tables
2023-12-04 17:16:12 +01:00
vdimir
a4ae90de0d
Merge pull request #57275 from ClickHouse/vdimir/merge_task_tmp_data
Background merges correctly use temporary data storage in the cache
2023-12-04 14:52:20 +01:00
vdimir
27589f5b0e
Enable temporary_data_in_cache in test_encrypted_disk 2023-12-04 10:34:17 +00:00
vdimir
b2e29d1416
Change type of s3_cache in test_encrypted_disk 2023-12-04 10:34:13 +00:00
Bharat Nallan Chakravarthy
25d1d59c6d fix tests 2023-12-03 20:16:16 -08:00
Bharat Nallan Chakravarthy
440dc66a5c Merge remote-tracking branch 'upstream/master' into ncb/hostname-system-log-tables 2023-12-03 15:19:47 -08:00
Vitaly Baranov
366095affc Fix test test_create_or_drop_tables_during_backup. 2023-12-04 00:15:31 +01:00
Sema Checherinda
32e3b2657b
Merge pull request #57205 from CheSema/zero-locks-leak
fix zero-copy locks leaking
2023-12-02 15:54:12 +01:00
Han Fei
fc0f080cb4 fix black 2023-12-01 20:39:38 +01:00
JackyWoo
8a68d51261
Add index to table system.numbers (#50909) 2023-12-01 19:59:25 +01:00
Han Fei
e4da317f05 fix test 2023-12-01 18:07:45 +01:00
Nikolai Kochetov
0b4131546a
Merge pull request #56813 from jsc0218/SystemTablesFilterEngine
Able to Filter Engine When Scanning System Tables
2023-12-01 16:02:27 +01:00
alesapin
fb98b212c5
Merge pull request #57311 from ClickHouse/make_alter_mat_view_non_experimental
Make alter materialized view non experimental
2023-12-01 12:18:43 +01:00
robot-clickhouse
a1219d0efd Automatic style fix 2023-11-30 21:12:29 +00:00
Igor Nikonov
b67bccd374 Integration test: custom key with failover 2023-11-30 20:54:53 +00:00
Sema Checherinda
60bf17ce8a fix style 2023-11-30 21:13:56 +01:00
Sema Checherinda
42f648f0d2 relock block when sessin expired 2023-11-30 21:13:56 +01:00
alesapin
17ac9e3409 Fix test 2023-11-30 19:42:12 +01:00
Igor Nikonov
7ab093c5d4 Fix existing intergation test 2023-11-30 18:42:03 +00:00
vdimir
b9860648c1
Remove temporary_data_in_cache from test_merge_tree_s3 2023-11-30 13:27:32 +00:00
Kruglov Pavel
575ee1e665
Merge pull request #56868 from zvonand/zvonand-fileCluster
Introduce `fileCluster` table function
2023-11-30 13:49:53 +01:00
Han Fei
a316b70917 Merge branch 'master' into hanfei/keeper_soft_limit 2023-11-30 12:29:40 +01:00
Alexander Tokmakov
a2dc2d9a53
Mark a setting obsolete (#57336)
* Update Settings.h

* Update InterpreterUndropQuery.cpp

* Update 02888_obsolete_settings.reference

* Update test.py

* Update 02681_undrop_query.sql

* Update 02681_undrop_query_uuid.sh
2023-11-29 19:24:11 +01:00
Alexander Tokmakov
97b63bbbca
Merge pull request #57339 from ClickHouse/fix_ddlworker_hostname_change
Ignore finished ON CLUSTER tasks if hostname changed
2023-11-29 17:46:17 +01:00
Nikolai Kochetov
823ba2db46
Merge pull request #57075 from yariks5s/s3_links_fix
S3-style links bug fix
2023-11-29 17:41:08 +01:00
vdimir
b5babe1692
MergeTask uses temporary data storage 2023-11-29 16:18:32 +00:00
zvonand
c329e382ab resolve conflicts 2023-11-29 16:03:07 +01:00
Han Fei
4c0efb0262
Merge pull request #53240 from hanfei1991/hanfei/statistic
use statistic to order prewhere conditions better
2023-11-29 12:26:44 +01:00
Nikolai Kochetov
731a1bfcf8 Cleanup 2023-11-29 11:11:58 +00:00
Alexander Tokmakov
84abbafcc5 ignore finished ddl tasks if hostname changed 2023-11-29 02:06:52 +01:00
Nikolai Kochetov
4d906d6e3c Cleanup 2023-11-28 18:35:19 +00:00
Yarik Briukhovetskyi
ad9583d6e9
style fix 2023-11-28 19:13:37 +01:00
Yarik Briukhovetskyi
a83a046d54 added tests 2023-11-28 17:33:46 +00:00
Nikolai Kochetov
08a7575984 Re-implement filtering a bit. 2023-11-28 16:17:35 +00:00
Han Fei
6733438adc Merge branch 'master' into hanfei/statistic 2023-11-28 16:35:00 +01:00
zvonand
c306d21b54 merge master + resolve conflicts 2023-11-28 15:51:21 +01:00
Han Fei
3aa4a6ae2c check integration test 2023-11-28 14:47:31 +01:00
Antonio Andelic
9dd4783675 Merge branch 'master' into fix-readonly-replica 2023-11-28 10:43:29 +00:00
vdimir
ae09d04578
Merge pull request #57154 from ClickHouse/vdimir/params_test_check_table
Minor changes in test_check_table
2023-11-28 11:10:26 +01:00
Han Fei
7bd2ea3add Merge branch 'master' into hanfei/statistic 2023-11-27 23:31:23 +01:00
Han Fei
8e1b61cc13 fix style 2023-11-27 22:03:05 +01:00
Nikolai Kochetov
e1b00b3ee0 Merge branch 'master' into SystemTablesFilterEngine 2023-11-27 18:54:21 +00:00
Han Fei
347efc669e support memory soft limit for keeper 2023-11-27 18:28:10 +01:00
Antonio Andelic
2ea44adefd Add test 2023-11-27 15:05:56 +00:00
Antonio Andelic
9707796869 Ignore comments when comparing column descriptions 2023-11-27 12:25:41 +00:00
MikhailBurdukov
6f19e8ebd1
Merge branch 'master' into native_copy_for_s3_disks 2023-11-27 14:25:36 +03:00
MikhailBurdukov
c10c30832c Review fix 2023-11-27 10:58:30 +00:00
Kruglov Pavel
b10e46b2bc
Merge pull request #57126 from Avogar/size-virtual-column
Add _size virtual column to s3/file/hdfs/url/azureBlobStorage engines
2023-11-27 10:12:18 +01:00
Bharat Nallan Chakravarthy
6454687fa6 Merge remote-tracking branch 'upstream/master' into ncb/hostname-system-log-tables 2023-11-26 16:24:18 -08:00
Azat Khuzhin
1168b7910b Update test_throttling with enabled s3_disable_checksum
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-11-26 20:03:33 +01:00
Vitaly Baranov
4fed61e8ca
Merge pull request #57146 from vitlibar/fix-test_replicated_merge_tree_encryption_codec_different_keys_2
Fix test test_replicated_merge_tree_encryption_codec/test.py::test_different_keys
2023-11-25 09:36:33 +01:00
Alexey Milovidov
c478acab42
Merge pull request #57206 from azat/tests/test_distributed_storage_configuration
Fix test_distributed_storage_configuration flakiness
2023-11-25 02:57:51 +01:00
Sema Checherinda
abd399705e
Merge pull request #57025 from CheSema/s3_engine_heavy_write_check_mem
test_s3_engine_heavy_write_check_mem: turn test off
2023-11-24 21:07:35 +01:00
Azat Khuzhin
d0b4917fd3 Fix test_distributed_storage_configuration flakiness
It fails [1] due to extra reservation on disk2 for some system.*_log
tables, sure we can turn them off, but better to fix it explicitly with
a separate policy.

  [1]: https://s3.amazonaws.com/clickhouse-test-reports/0/98cddf5312722e403dcea429639ac13dc6cada33/integration_tests__tsan__[2_6].html

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-11-24 20:28:18 +01:00
Alexey Milovidov
411660c267 Fix integration tests 2023-11-24 17:35:56 +01:00
Vitaly Baranov
5769a88b92
Merge pull request #57133 from vitlibar/change-default-for-wait_dictionaries_load_at_startup
Change the default for wait_dictionaries_load_at_startup to true
2023-11-24 17:09:05 +01:00
Vitaly Baranov
683827c6df Add comment. 2023-11-24 14:01:52 +01:00
vdimir
3b4f70a811
Minor changes in test_check_table 2023-11-23 16:24:11 +00:00
Vitaly Baranov
eabdc76e7c Fix test test_replicated_merge_tree_encryption_codec/test.py::test_different_keys 2023-11-23 15:36:11 +01:00
Vitaly Baranov
5f461ff780 Change the default for "wait_dictionaries_load_at_startup" to true,
and use this setting only if "dictionaries_lazy_load" is false.
2023-11-23 14:45:42 +01:00
Aleksei Filatov
1a03f5f7f4 Merge remote-tracking branch 'upstream/master' into add_cancellation_point_for_moving_background_operation 2023-11-23 16:43:33 +03:00
avogar
e3afb4182b Fix tests 2023-11-23 11:10:41 +00:00
serxa
0fc25fbda8 Merge branch 'master' into async-loader-integration 2023-11-23 09:32:43 +00:00
jsc0218
aba2f0e4b2 Fix Style 2023-11-22 20:02:15 +00:00
jsc0218
a1bde09c9c move the test from stateless to integration 2023-11-22 19:13:13 +00:00
Kruglov Pavel
b84e3cf683
Merge branch 'master' into size-virtual-column 2023-11-22 19:25:00 +01:00
avogar
193f8a5d23 Add test for azure 2023-11-22 18:12:50 +00:00
zvonand
5153798aeb Introduced fileCluster table function
Added fileCluster function
Added test and docs
2023-11-22 15:06:04 +01:00
Kruglov Pavel
9fb1acc947
Merge pull request #56810 from Avogar/iceberg-metadata-files
Iceberg metadata files
2023-11-22 14:40:04 +01:00
Sema Checherinda
adf4ade00a
Update tests/integration/test_merge_tree_s3/test.py 2023-11-22 13:38:15 +01:00
Kseniia Sumarokova
e4f66b8469
Merge pull request #55158 from kssenii/fs-cache-improvement
fs cache improvement for big reads
2023-11-21 21:50:00 +01:00
Nikita Mikhaylov
087a0a6043
Merge pull request #57040 from ClickHouse/fix-rabbitmq-supress-tsan-in-test
Fix: suppress TSAN in RabbitMQ test
2023-11-21 17:52:36 +01:00
Sema Checherinda
63bd45fa5f
Update tests/integration/test_merge_tree_s3/test.py 2023-11-21 17:42:37 +01:00
vdimir
a139ae97eb
Merge pull request #52918 from ClickHouse/vdimir/s3_blob_log
Add system table with blob storage operations log
2023-11-21 17:40:42 +01:00
Vitaly Baranov
4052ae8bfd
Merge pull request #57007 from vitlibar/fix-dropping-tables-in-test_create_or_drop_tables_during_backup
Fix dropping tables in test_create_or_drop_tables_during_backup
2023-11-21 17:16:16 +01:00
Sema Checherinda
2e1c142d6d black style 2023-11-21 14:03:49 +01:00
Igor Nikonov
08b2740d0f Fix: supress test with tsan 2023-11-21 11:26:14 +00:00
Kseniia Sumarokova
d384762123
Merge branch 'master' into fs-cache-improvement 2023-11-21 11:24:52 +01:00
Kseniia Sumarokova
f960036d76
Merge pull request #56874 from ClickHouse/kssenii-patch-8
Update test_storage_s3_queue/test.py
2023-11-21 11:13:00 +01:00
vdimir
15234474d7
Implement system table blob_storage_log 2023-11-21 09:18:25 +00:00
Sema Checherinda
524edd8460 turn test off 2023-11-21 00:08:20 +01:00
Alexey Milovidov
6a1955cb34 Merge branch 'master' of github.com:ClickHouse/ClickHouse into remove-useless-install 2023-11-20 21:34:31 +01:00
Sema Checherinda
485f1834d8
Merge pull request #56938 from CheSema/lz4-buffering
Lz4 compression: buffer block in a rare case
2023-11-20 20:33:30 +01:00
Kseniia Sumarokova
9454b203e1
Merge branch 'master' into kssenii-patch-8 2023-11-20 20:19:21 +01:00
Vitaly Baranov
897cd06bcf Fix dropping tables in test "test_create_or_drop_tables_during_backup". 2023-11-20 19:31:53 +01:00
avogar
f537bad469 Merge branch 'master' of github.com:ClickHouse/ClickHouse into schema-inference-union 2023-11-20 14:32:50 +00:00
Sema Checherinda
a7fc8d4b99 test_merge_tree_s3 counts errors, turn off s3_use_adaptive_timeouts 2023-11-20 15:04:14 +01:00
avogar
872556a5d4 Merge branch 'master' of github.com:ClickHouse/ClickHouse into schema-inference-union 2023-11-20 14:03:36 +00:00
Kseniia Sumarokova
4c7daf5167
Fix 2023-11-20 14:55:23 +01:00
Sema Checherinda
f999337dae
Revert "Revert "s3 adaptive timeouts"" 2023-11-20 14:53:22 +01:00
Alexander Tokmakov
5031f239c3
Revert "s3 adaptive timeouts" 2023-11-20 14:28:59 +01:00
robot-clickhouse
05b608cd76 Automatic style fix 2023-11-20 10:12:45 +00:00
Smita Kulkarni
961bf074da Initial draft version of adding backup support to AzureBlobStorage 2023-11-20 10:56:10 +01:00
Alexey Milovidov
241cc2abf4 Merge branch 'master' into remove-useless-install 2023-11-20 01:12:08 +01:00
Bharat Nallan Chakravarthy
d08512d8c0 Merge remote-tracking branch 'upstream/master' into ncb/hostname-system-log-tables 2023-11-19 14:06:40 -08:00
Igor Nikonov
b9a28028e4
Merge branch 'master' into test-rabbitmq-with-secure-connection 2023-11-19 21:33:09 +01:00
Alexey Milovidov
9367dcc276
Merge pull request #56975 from rschu1ze/unflake-test_backward_compat
Add exclude for tryBase64Decode to backward compat test (follow-up to #56913)
2023-11-19 16:41:42 +01:00
Robert Schulze
a3c9f13ac9
Add exclude for tryBase64Decode to backward compat test (follow-up to #56913)
Fixes #56969
2023-11-19 15:33:58 +00:00
Antonio Andelic
ab234a87a8
Merge pull request #56960 from ClickHouse/fix-test_keeper_auth
Fix `test_keeper_auth`
2023-11-19 16:09:04 +01:00
Sema Checherinda
a950595c24
Merge pull request #56314 from CheSema/s3-aggressive-timeouts
s3 adaptive timeouts
2023-11-19 14:12:14 +01:00
serxa
9023f61349 Merge branch 'master' into async-loader-integration 2023-11-19 13:05:58 +00:00
Antonio Andelic
7c81d9b2b7 Fix test_keeper_auth 2023-11-18 19:38:33 +00:00
Antonio Andelic
274f2c71f2
Merge pull request #56460 from ClickHouse/keeper-map-backup-restore
Backup/Restore for KeeperMap tables
2023-11-18 19:27:36 +01:00
Antonio Andelic
9eaafff3d2
Merge pull request #56715 from incfly/keeper-az-fix
Fix keeper availability zone configuration.
2023-11-18 19:24:24 +01:00
Alexey Milovidov
74319b5cbd
Merge pull request #56082 from Algunenano/less_diff_images
See what happens if we use less different docker images in integration tests
2023-11-18 17:24:28 +01:00
Sema Checherinda
6d5a5f9fcd buffer result if out copacity is not enough 2023-11-17 17:31:00 +01:00
Raúl Marín
4d5becb4de Adapt test_storage_s3/test.py::test_predefined_connection_configuration 2023-11-17 15:01:54 +01:00
Raúl Marín
7074bdb3bb Merge remote-tracking branch 'blessed/master' into access-control-and-named-collections-by-default 2023-11-17 14:53:02 +01:00
Kruglov Pavel
4e6f265e3e
Fix tests 2023-11-17 14:36:48 +01:00
Alexey Milovidov
5f9db3b248
Merge pull request #56794 from ClickHouse/early-exit-interserver-authentication-failure
Early disconnect if there is authentication failure with interserver secret
2023-11-17 13:36:09 +01:00
Antonio Andelic
803d9668e2 Merge branch 'master' into keeper-map-backup-restore 2023-11-17 10:11:59 +00:00
Antonio Andelic
f8794e72e7
Merge pull request #56858 from ClickHouse/better-test_keeper_broken_logs
Fix `test_keeper_broken_logs`
2023-11-17 10:36:05 +01:00
Antonio Andelic
8b30a54b48
Merge pull request #56835 from ClickHouse/better-test_keeper_reconfig_replace_leader
More stable `test_keeper_reconfig_replace_leader`
2023-11-17 10:35:26 +01:00
Aleksei Filatov
163b8a814e Fix clang-tidy issues and disable parallel execution of tests 2023-11-17 11:02:07 +03:00
Jianfei Hu
ef79bf6467 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into keeper-az-fix 2023-11-16 21:58:02 +00:00
Bharat Nallan Chakravarthy
7434dd9e0a Merge remote-tracking branch 'upstream/master' into ncb/hostname-system-log-tables 2023-11-16 11:17:45 -08:00
Kseniia Sumarokova
2a1467b8da
Update test_storage_s3_queue/test.py 2023-11-16 19:33:17 +01:00
Alexey Milovidov
2941f30678 Merge branch 'master' of github.com:ClickHouse/ClickHouse into early-exit-interserver-authentication-failure 2023-11-16 18:15:09 +01:00
Kseniia Sumarokova
a2ed756eec
Merge branch 'master' into fs-cache-improvement 2023-11-16 17:49:26 +01:00
robot-clickhouse
d03a1aab7b Automatic style fix 2023-11-16 16:39:57 +00:00
Igor Nikonov
34f8bf8da5 Merge remote-tracking branch 'origin/master' into test-rabbitmq-with-secure-connection 2023-11-16 16:27:25 +00:00
Igor Nikonov
49c58e7609 Disable RabbitMQ secure connection test in intergatiion test with TSAN 2023-11-16 16:24:09 +00:00
Konstantin Bogdanov
4cc2d6baa5
Merge pull request #56694 from ClickHouse/fix-configuration-merge-issue
Fix configuration merge for nodes with substitution attributes
2023-11-16 15:19:09 +01:00
Antonio Andelic
bb68321fc1 More stable test_keeper_reconfig_replace_leader 2023-11-16 13:31:39 +00:00
Antonio Andelic
bdf038191a better test_keeper_broken_logs 2023-11-16 13:08:48 +00:00
Igor Nikonov
b9d083e68d Merge remote-tracking branch 'origin/master' into test-rabbitmq-with-secure-connection 2023-11-16 13:05:10 +00:00
Igor Nikonov
4d16c096a1 Use ports from cluster 2023-11-16 13:03:20 +00:00
alesapin
437a911d7b
Merge pull request #56809 from ClickHouse/add_async_moves_execution
Implement asynchronous execute of `ALTER TABLE ... MOVE ... TO DISK|VOLUME`
2023-11-16 12:47:31 +01:00
Antonio Andelic
203f83c2c2 Merge branch 'master' into keeper-map-backup-restore 2023-11-16 11:31:11 +00:00
Sema Checherinda
7d37c0e070
Update tests/integration/test_merge_tree_s3_failover/configs/config.d/storage_conf.xml
Co-authored-by: Nikita Taranov <nickita.taranov@gmail.com>
2023-11-16 12:29:21 +01:00
Sema Checherinda
ebd42187ad
Update tests/integration/test_merge_tree_s3_failover/configs/config.d/storage_conf.xml
Co-authored-by: Nikita Taranov <nickita.taranov@gmail.com>
2023-11-16 12:29:15 +01:00
Vitaly Baranov
342e6c7546
Merge pull request #56782 from vitlibar/add-config-setting-wait_dictionaries_load_at_startup
Add config setting "wait_dictionaries_load_at_startup"
2023-11-16 11:51:29 +01:00
Aleksei Filatov
21a17f8334 Add cancellation hook for moving background operation 2023-11-16 13:37:51 +03:00
Vitaly Baranov
946e02ecbc
Merge pull request #56268 from vitlibar/add-script-backup-view
Add an external utility to view backups and to extract information from them
2023-11-16 11:17:16 +01:00
Robert Schulze
44bd0598a3
Merge pull request #56799 from slvrtrn/fix-binary-mysql-nullable-columns
Fix incorrect nullable columns reporting in MySQL binary protocol
2023-11-16 01:12:46 +01:00
Alexey Milovidov
3067ca64df
Merge pull request #56573 from mkmkme/mkmkme/reload-config
Update `query_masking_rules` when reloading the config
2023-11-16 00:27:01 +01:00
Alexey Milovidov
09bec3c754 Fix integration test 2023-11-16 00:16:02 +01:00
Konstantin Bogdanov
67796b18bb
Add missing test config files 2023-11-15 19:36:23 +01:00
robot-clickhouse
af2f986215 Automatic style fix 2023-11-15 17:48:54 +00:00
avogar
808c3f36ed Remove prints from test 2023-11-15 17:47:10 +00:00
avogar
28522d56b7 Support Iceberg metadata files for metastore tables 2023-11-15 17:45:07 +00:00
Alexander Sapin
3e26161eaa Implement asynchronous excute of ALTER TABLE ... MOVE ... TO DISK|VOLUME 2023-11-15 18:33:59 +01:00
Vitaly Baranov
21a283049c Add the test for clickhouse_backupview to CI. 2023-11-15 17:40:53 +01:00
Alexander Tokmakov
bf10989515
Fix unexpected parts handling (#56693)
* fix unexpected parts handling

* Automatic style fix

* fix

---------

Co-authored-by: robot-clickhouse <robot-clickhouse@users.noreply.github.com>
2023-11-15 16:54:47 +01:00
slvrtrn
9e3ba55070 Fix incorrect nullable columns reporting in MySQL binary protocol 2023-11-15 16:53:38 +01:00
Mikhail Koviazin
08d0a160ae
test_reload_query_masking_rules: don't rely on num of events
Due to the fact that integration tests are run in parallel and there
might be no ClickHouse restart between several tests, relying on
`system.events` in the test is fragile as it can be affected by previous
tests. This commit removes any assumptions regarding `system.events`
from the test and tries to keep it robust.
2023-11-15 17:48:24 +02:00
Antonio Andelic
c3f7c8792a
Merge pull request #56763 from ClickHouse/disable-compress-logs-default
Disable compressed logs by default in Keeper
2023-11-15 16:17:14 +01:00
Antonio Andelic
d567e718eb Merge branch 'master' into keeper-map-backup-restore 2023-11-15 11:51:27 +00:00
Vitaly Baranov
308654195c Add config setting "wait_dictionaries_load_at_startup". 2023-11-15 11:48:50 +01:00
Antonio Andelic
5309dc05ef Fix test 2023-11-15 07:42:19 +00:00
Alexey Milovidov
37e48e76fe
Merge pull request #56768 from vitlibar/fix-flaky-test_replicated_merge_tree_encryption_codec
Fix flaky test_replicated_merge_tree_encryption_codec.
2023-11-15 07:46:06 +01:00
Vitaly Baranov
3d95ac5f2a Fix flaky test_replicated_merge_tree_encryption_codec. 2023-11-14 21:53:23 +01:00
robot-clickhouse
369aaef92f Automatic style fix 2023-11-14 20:26:32 +00:00
Igor Nikonov
cd909ffc48 Test RabbitMQ with secure connection 2023-11-14 20:13:09 +00:00
Jianfei Hu
9df2775f08 reduce timeout and setTimeout earlier.
Signed-off-by: Jianfei Hu <hujianfei258@gmail.com>
2023-11-14 17:58:16 +00:00
Sema Checherinda
b9cc1580e0
Merge pull request #56367 from canhld94/ch_table_reinit_new_disk
Adding new disk to storage configuration without restart
2023-11-14 15:54:22 +01:00
Antonio Andelic
2f9ac9b49c Address comments 2023-11-14 14:33:34 +00:00
MikhailBurdukov
1aea8186c7 Enable s3 native copy for scenario: from s3 to s3. 2023-11-14 13:53:48 +00:00
robot-clickhouse
cc3dfafd7b
Automatic style fix 2023-11-14 13:07:45 +01:00
Konstantin Bogdanov
2fddc17642
Add test 2023-11-14 13:07:38 +01:00
Mikhail Koviazin
9e1357dd7e
Update query_masking_rules when reloading the config
Fixes #56449
2023-11-14 14:03:28 +02:00
Sema Checherinda
3075bd9745 track clickhouse high level retries 2023-11-14 11:34:12 +01:00
Sema Checherinda
bb0b6afe14 reduce cuncurrent request number to the minio in test_storage_s3 2023-11-14 11:34:12 +01:00
Sema Checherinda
1a7be21a66 adjust tests 2023-11-14 11:34:12 +01:00
Sema Checherinda
338c51745f fix style 2023-11-14 11:34:12 +01:00
Sema Checherinda
ab2594154e add tests 2023-11-14 11:34:12 +01:00
Jianfei Hu
554d907189 Fix the keeper_server availability zone configuration.
Signed-off-by: Jianfei Hu <hujianfei258@gmail.com>
2023-11-13 23:42:51 +00:00
Jianfei Hu
d3517140d9 Revert "Revert "Add /keeper/availability-zone node to allow server load balancing within AZ.""
This reverts commit e5aad4a6fd.
2023-11-13 19:03:27 +00:00
Bharat Nallan Chakravarthy
d90b8b6fba Merge remote-tracking branch 'upstream/master' into ncb/hostname-system-log-tables 2023-11-13 08:43:37 -08:00
robot-clickhouse
a9725f22e1 Automatic style fix 2023-11-13 12:38:02 +00:00
Antonio Andelic
59480205d4 Merge branch 'master' into keeper-map-backup-restore 2023-11-13 12:21:26 +00:00
Antonio Andelic
e3c66f8a63 Merge branch 'master' into more-reliable-log-handling-keeper 2023-11-13 12:14:57 +00:00
Antonio Andelic
99d521becf More reliable log handling in Keeper 2023-11-13 12:09:13 +00:00
Alexey Milovidov
5099859ccd
Merge pull request #56656 from ClickHouse/delete-a-file-2
Delete a file (2)
2023-11-13 10:17:13 +01:00
Alexey Milovidov
3807dcc469 Delete a file (2) 2023-11-13 07:08:21 +01:00
Alexey Milovidov
f48527b53a Merge branch 'concurrency' of github.com:ClickHouse/ClickHouse into concurrency 2023-11-13 02:57:14 +01:00
Alexey Milovidov
7ac931c29d Merge branch 'master' into concurrency 2023-11-13 02:56:56 +01:00
Alexey Milovidov
812b8c2bc1 Fix test; delete strange file 2023-11-12 00:56:57 +01:00
Pradeep Chhetri
2ec76f2140 Fix test 2023-11-11 16:47:43 +08:00
Pradeep Chhetri
d0922782ed Revert "Revert "Add 4-letter command for yielding/resigning leadership""
This reverts commit e1766b96d9.
2023-11-11 15:37:49 +08:00
Alexey Milovidov
e1766b96d9
Revert "Add 4-letter command for yielding/resigning leadership" 2023-11-11 05:45:02 +03:00
Alexey Milovidov
e5aad4a6fd
Revert "Add /keeper/availability-zone node to allow server load balancing within AZ." 2023-11-11 05:43:58 +03:00
Alexey Milovidov
28c89ad1c8
Merge pull request #56544 from jmaicher/fix/flaky-ldap-integration-test
Fix flaky LDAP integration tests
2023-11-10 17:00:54 +01:00
Alexey Milovidov
461b80a781
Merge pull request #56570 from ClickHouse/fix-keeper-test_cmd_crst
Fix `test_keeper_four_word_command/test.py::test_cmd_crst`
2023-11-10 16:57:42 +01:00
Nikolay Degterinsky
a6ebeb8422
Merge pull request #56484 from evillique/fix-on-cluster
Fix ON CLUSTER queries without database on initial node
2023-11-10 13:40:41 +01:00
Nikolay Degterinsky
07b2cdb614
Merge pull request #56491 from evillique/fix-alter-comment-on-cluster
Fix ALTER COMMENT queries ON CLUSTER
2023-11-10 13:35:26 +01:00
vdimir
051efccf6c
Delete unnecessary file
<!---
A technical comment, you are free to remove or leave it as it is when PR is created
The following categories are used in the next scripts, update them accordingly
utils/changelog/changelog.py
tests/ci/cancel_and_rerun_workflow_lambda/app.py
-->
### Changelog category (leave one):

- Not for changelog (changelog entry is not required)
2023-11-10 11:31:05 +01:00
Kseniia Sumarokova
e48df94772
Merge pull request #56377 from ClickHouse/minor-improvements-for-s3-queue
Minor improvements for S3Queue
2023-11-10 11:28:09 +01:00