mirror of
https://github.com/ClickHouse/ClickHouse.git
synced 2024-11-22 07:31:57 +00:00
5c221d123d
One binlog connection for many databases. Suggesting to disable this feature by default for now. It should be explicitly enabled by SETTINGS use_binlog_client=1. But if you would permanently enable it in MaterializedMySQLSettings, it should keep old behavior and all tests should pass too. 1. Introduced `IBinlog` and its impl to read the binlog events from socket - `BinlogFromSocket`, or file - `BinlogFromFile`. Based on prev impl of `EventBase` and the same old binlog parsers. It fully keeps BC with old version. Fixed `./check-mysql-binlog` to test new impl. 2. Introduced `BinlogEventsDispatcher`, it reads the event from the source `IBinlog` and sends it to currently attached `IBinlog` instances. 3. Introduced `BinlogClient`, which is used to group a list of `BinlogEventsDispatcher` by MySQL binlog connection which is defined by `user:password@host:port`. All dispatchers with the same binlog position should be merged to one. 4. Introduced `BinlogClientFactory`, which is a singleton and it is used to track all binlogs created over the instance. 5. Introduced `use_binlog_client` setting to `MaterializedMySQL`, which forces to reuse a `BinlogClient` if it already exists in `BinlogClientCatalog` or create new one. By default, it is disabled. 6. Introduced `max_bytes_in_binlog_queue` setting to define the limit of bytes in binlog's queue of events. If bytes in the queue increases this limit, `BinlogEventsDispatcher` will stop reading new events from source `IBinlog` until the space for new events will be freed. 7. Introduced `max_milliseconds_to_wait_in_binlog_queue` setting to define max ms to wait when the max bytes exceeded. 7. Introduced `max_milliseconds_to_wait_in_binlog_queue` setting to define max ms to wait when the max bytes exceeded. 8. Introduced `max_bytes_in_binlog_dispatcher_buffer` setting to define max bytes in the binlog dispatcher's buffer before it is flushed to attached binlogs. 9. Introduced `max_flush_milliseconds_in_binlog_dispatcher` setting to define max milliseconds in the binlog dispatcher's buffer to wait before it is flushed to attached binlogs. 10. Introduced `system.mysql_binlogs` system table, which shows a list of active binlogs. 11. Introduced `UnparsedRowsEvent` and `MYSQL_UNPARSED_ROWS_EVENT`, which defines that an event is not parsed and should be explicitly parsed later. 12. Fixed bug when not possible to apply DDL since syntax error or unsupported SQL. @larspars is the author of following: `GTIDSets::contains()` `ReplicationHelper` `shouldReconnectOnException()` |
||
|---|---|---|
| .. | ||
| antlr | ||
| backupview | ||
| changelog | ||
| check-marks | ||
| check-mysql-binlog | ||
| check-style | ||
| checksum-for-compressed-block | ||
| clickhouse-diagnostics | ||
| compressor | ||
| corrector_utf8 | ||
| debug_remote_fs_performance | ||
| durability-test | ||
| grpc-client | ||
| kafka | ||
| keeper-bench | ||
| keeper-data-dumper | ||
| keeper-overload | ||
| list-licenses | ||
| list-versions | ||
| memcpy-bench | ||
| postprocess-traces | ||
| prepare-time-trace | ||
| report | ||
| s3tools | ||
| security-generator | ||
| self-extracting-executable | ||
| test_history | ||
| tests-visualizer | ||
| trace-visualizer | ||
| zero_copy | ||
| zookeeper-cli | ||
| zookeeper-dump-tree | ||
| zookeeper-remove-by-list | ||
| async_loader_graph | ||
| c++expr | ||
| clickhouse-docker | ||
| CMakeLists.txt | ||
| data-lakes-importer.py | ||