Commit Graph

8405 Commits

Author SHA1 Message Date
János Benjamin Antal
3e1d7bf685
Merge pull request #58477 from ClickHouse/multiple-read-kafka-storage-in-mv
Delay reading from StorageKafka to allow multiple reads in materialized views
2024-01-15 12:31:59 +01:00
Kseniia Sumarokova
77f4cd8dd3
Merge pull request #58721 from ClickHouse/support-point-datatype-for-mysql
Support Point data type for MySQL
2024-01-12 17:02:01 +01:00
kssenii
8b1d2805ae Add one more test case to the test 2024-01-12 10:41:04 +01:00
Dmitry Novik
e3aed6ba4e
Fix integration test 2024-01-12 09:40:15 +01:00
kssenii
fada92173b Support Point data type for MySQL 2024-01-11 19:19:08 +01:00
Yakov Olkhovskiy
84809b36c3
Merge pull request #58475 from ClickHouse/revert-58450-revert-56064-feature-server-iface-metrics
Second attempt: Use CH Buffer for HTTP out stream, add metrics for interfaces
2024-01-11 12:06:59 -05:00
Igor Nikonov
8ba9b4a7ef
Merge pull request #57235 from ClickHouse/pr-custom-key-failover
Parallel replicas custom key: skip unavailable replicas
2024-01-11 12:22:32 +01:00
Konstantin Bogdanov
7f4ccebea4
Merge pull request #58550 from danipozo/hide-credentials-in-base-backup-name
Hide credentials in system.backup_log base_backup_name column
2024-01-11 03:42:17 +03:00
Igor Nikonov
8d4a395afa Merge remote-tracking branch 'origin/master' into pr-custom-key-failover 2024-01-10 20:28:04 +00:00
Igor Nikonov
e9fc869254 Fix test flakiness with hedge connections enabled 2024-01-10 15:59:48 +00:00
Yakov Olkhovskiy
2e124b19b8
Merge branch 'master' into revert-58450-revert-56064-feature-server-iface-metrics 2024-01-10 09:05:18 -05:00
Robert Schulze
4cacfa6b43
Merge pull request #57509 from jinjunzh/qatzstd_main
Add Intel-QAT-based `ZSTD_QAT` compression codec
2024-01-10 10:31:39 +01:00
Kseniia Sumarokova
1b517105db
Merge pull request #57323 from valbok/mysql-binlog-client
MaterializedMySQL: Introduce MySQL Binlog Client
2024-01-10 09:08:13 +01:00
Robert Schulze
ced9e93ac6
Merge remote-tracking branch 'rschu1ze/master' into qatzstd_main 2024-01-09 20:36:02 +00:00
Sema Checherinda
7c7e72c4b7
Merge pull request #58573 from ClickHouse/chesema-s3-client-creation
fix and test that S3Clients are reused
2024-01-09 13:28:20 +01:00
Sema Checherinda
6dbf7c967c
Merge pull request #57663 from ClickHouse/CheSema-remote-blobs-naming-2
generate object storage key by template
2024-01-09 13:26:56 +01:00
Igor Nikonov
7fd851b4e9 Merge remote-tracking branch 'origin/master' into pr-custom-key-failover 2024-01-08 20:43:20 +00:00
Dani Pozo
d8f4b5fd13 Fix black formatting 2024-01-08 12:47:58 +01:00
Dani Pozo
b42eaf6c62 Add test for hiding credentials in system.backups table 2024-01-08 12:20:32 +01:00
robot-clickhouse
29173a602e Automatic style fix 2024-01-08 02:39:19 +00:00
Yakov Olkhovskiy
45ac4d2b67 Merge remote-tracking branch 'origin/master' into revert-58450-revert-56064-feature-server-iface-metrics 2024-01-07 15:23:46 +00:00
Robert Schulze
53965bb9f1
Merge remote-tracking branch 'rschu1ze/master' into qatzstd_main 2024-01-07 14:11:22 +00:00
Sema Checherinda
d5f86f671d fix and test that S3Clients reused 2024-01-07 02:19:06 +01:00
robot-clickhouse
4a473c37ec Automatic style fix 2024-01-06 15:55:20 +00:00
Alexey Milovidov
2751b351cb Fix test 2024-01-06 16:44:38 +01:00
Yakov Olkhovskiy
9ca4be5d91 Merge remote-tracking branch 'origin/master' into revert-58450-revert-56064-feature-server-iface-metrics 2024-01-06 00:24:31 +00:00
robot-clickhouse
630ee67262 Automatic style fix 2024-01-05 23:02:58 +00:00
Yakov Olkhovskiy
0964bf3638 add clickhouse-library-bridge logging for test_catboost_evaluate 2024-01-05 22:51:17 +00:00
Nikolay Degterinsky
56aa835954
Merge pull request #58444 from bharatnc/ncb/support-hints-for-db-engines
support hints for database engines
2024-01-05 17:12:44 +01:00
Sergei Trifonov
8d6b012223
Merge pull request #57877 from azat/merges-mutations-throttling
Add ability to throttle merges/mutations
2024-01-05 17:08:50 +01:00
Sema Checherinda
2861cc70ba fix style 2024-01-05 13:42:30 +00:00
Sema Checherinda
8ab4aa3423 enable randomization setting 2024-01-05 13:42:30 +00:00
Sema Checherinda
53f36f8b21 object storage key template 2024-01-05 13:42:30 +00:00
Alexander Tokmakov
d2752f4b66
Merge pull request #57603 from evillique/allow-not-resolving-hostnames
Allow avoiding resolving hostnames in DDLWorker
2024-01-04 15:46:28 +01:00
Alexander Tokmakov
972756486a
Merge pull request #58350 from ClickHouse/rdb_dont_wait_inactive
An option to avoid waiting for inactive Replicated database replicas
2024-01-04 14:50:11 +01:00
Alexander Tokmakov
10e3ae9e68 Merge branch 'master' into allow-not-resolving-hostnames 2024-01-04 13:22:15 +01:00
robot-clickhouse
224a58ddc5 Automatic style fix 2024-01-04 10:30:44 +00:00
János Benjamin Antal
1a1fa07128 Add missing asserts 2024-01-04 10:18:04 +00:00
János Benjamin Antal
2517d23e27 Add tests 2024-01-04 10:14:41 +00:00
Nikita Taranov
66d2db5283
New parallel replicas coordinator implementation (#57968) 2024-01-04 00:27:04 +01:00
Bharat Nallan Chakravarthy
d63765fb19 Merge upstream/master into ncb/support-hints-for-db-engines 2024-01-03 08:44:11 -08:00
Alexander Tokmakov
b1688b1953 Merge branch 'master' into rdb_dont_wait_inactive 2024-01-03 17:03:01 +01:00
Val Doroshchuk
5c221d123d MaterializedMySQL: Introduce MySQL Binlog Client
One binlog connection for many databases.

Suggesting to disable this feature by default for now. It should be explicitly enabled by SETTINGS use_binlog_client=1.
But if you would permanently enable it in MaterializedMySQLSettings, it should keep old behavior and all tests should pass too.

1. Introduced `IBinlog` and its impl to read the binlog events from socket - `BinlogFromSocket`, or file - `BinlogFromFile`. Based on prev impl of `EventBase` and the same old binlog parsers. It fully keeps BC with old version. Fixed `./check-mysql-binlog` to test new impl.
2. Introduced `BinlogEventsDispatcher`, it reads the event from the source `IBinlog` and sends it to currently attached `IBinlog` instances.
3. Introduced `BinlogClient`, which is used to group a list of `BinlogEventsDispatcher` by MySQL binlog connection which is defined by `user:password@host:port`. All dispatchers with the same binlog position should be merged to one.
4. Introduced `BinlogClientFactory`, which is a singleton and it is used to track all binlogs created over the instance.
5. Introduced `use_binlog_client` setting to `MaterializedMySQL`, which forces to reuse a `BinlogClient` if it already exists in `BinlogClientCatalog` or create new one. By default, it is disabled.
6. Introduced `max_bytes_in_binlog_queue` setting to define the limit of bytes in binlog's queue of events. If bytes in the queue increases this limit, `BinlogEventsDispatcher` will stop reading new events from source `IBinlog` until the space for new events will be freed.
7. Introduced `max_milliseconds_to_wait_in_binlog_queue` setting to define max ms to wait when the max bytes exceeded.
7. Introduced `max_milliseconds_to_wait_in_binlog_queue` setting to define max ms to wait when the max bytes exceeded.
8. Introduced `max_bytes_in_binlog_dispatcher_buffer` setting to define max bytes in the binlog dispatcher's buffer before it is flushed to attached binlogs.
9. Introduced `max_flush_milliseconds_in_binlog_dispatcher` setting to define max milliseconds in the binlog dispatcher's buffer to wait before it is flushed to attached binlogs.
10. Introduced `system.mysql_binlogs` system table, which shows a list of active binlogs.
11. Introduced `UnparsedRowsEvent` and `MYSQL_UNPARSED_ROWS_EVENT`, which defines that an event is not parsed and should be explicitly parsed later.
12. Fixed bug when not possible to apply DDL since syntax error or unsupported SQL.

@larspars is the author of following:
`GTIDSets::contains()`
`ReplicationHelper`
`shouldReconnectOnException()`
2024-01-03 15:26:09 +01:00
Kruglov Pavel
570d1c013b
Merge pull request #57053 from Avogar/better-parsing-exceptions
Better exception messages in input formats
2024-01-03 14:47:51 +01:00
Bharat Nallan Chakravarthy
2da0a30626 add a test 2024-01-02 13:52:52 -08:00
Nikolay Degterinsky
debbeb505d
Merge pull request #58409 from ClickHouse/evillique-patch-1
Fix test_user_valid_until
2024-01-02 17:06:11 +01:00
avogar
5f6a806d1c Fix style 2024-01-02 15:43:44 +00:00
avogar
0844528ec1 Fix tests 2024-01-02 15:22:42 +00:00
Kruglov Pavel
e69bda7f08
Merge pull request #57438 from Avogar/fix-resetting-buffers
Fix working with read buffers in StreamingFormatExecutor
2024-01-02 15:34:35 +01:00
Nikolay Degterinsky
004730fe87
Fix test_user_valid_until 2024-01-02 14:50:28 +01:00