mirror of
https://github.com/ClickHouse/ClickHouse.git
synced 2024-11-23 16:12:01 +00:00
81bf43157f
The Kafka table engine allows global configuration and per-Kafka-topic configuration. The latter uses syntax <kafka_TOPIC>, e.g. for topic "football": <kafka_football> <retry_backoff_ms>250</retry_backoff_ms> <fetch_min_bytes>100000</fetch_min_bytes> </kafka_football> Some users had to find out the hard way that such configuration doesn't take effect if the topic name contains a period, e.g. "sports.football". The reason is that ClickHouse configuration framework already uses periods as level separators to descend the configuration hierarchy. (Besides that, per-topic configuration at the same level as global configuration could be considered ugly.) Note that Kafka topics may contain characters "a-zA-Z0-9._-" (*) and a tree-like topic organization using periods is quite common in practice. This PR deprecates the existing per-topic configuration syntax (but continues to support it for backward compat) and introduces a new per-topic configuration syntax below the global Kafka configuration of the form: <kafka> <topic name="football"> <retry_backoff_ms>250</retry_backoff_ms> <fetch_min_bytes>100000</fetch_min_bytes> </topic> </kafka> The period restriction doesn't apply to XML attributes, so <topic name="sports.football"> will work. Also, everything Kafka-related is below <kafka>. Considered but rejected alternatives: - Extending Poco ConfigurationView with custom separators (e.g."/" instead of "."). Won't work easily because ConfigurationView only builds a path but defers descending the configuration tree to the normal configuration classes. - Reloading the configuration file in StorageKafka (instead of reading the loaded file) but with a custom separator. This mode is supported by XML configuration. Too ugly and error-prone since the true configuration is composed from multiple configuration files. (*) https://stackoverflow.com/a/37067544 |
||
---|---|---|
.. | ||
database-engines | ||
table-engines | ||
_category_.yml |