Commit Graph

294 Commits

Author SHA1 Message Date
Robert Schulze
ff8fc0634a
Docs: Small cleanups after Kafka fix #47138 2023-03-07 19:50:42 +00:00
AVMusorin
0f2ae72141
prohibit DEFAULT/EPHEMERAL/ALIAS in KafkaEngine 2023-03-03 16:25:42 +01:00
Robert Schulze
cca0bead44
Slighly improved example 2023-02-23 20:07:06 +00:00
Robert Schulze
b3eacf0894
Whitespace for the win 2023-02-23 20:06:24 +00:00
Robert Schulze
6906cafa99
Change to S3 cfg syntax 2023-02-23 20:04:41 +00:00
Robert Schulze
81bf43157f
Allow configuration of Kafka topics with periods
The Kafka table engine allows global configuration and per-Kafka-topic
configuration. The latter uses syntax <kafka_TOPIC>, e.g. for topic
"football":

  <kafka_football>
      <retry_backoff_ms>250</retry_backoff_ms>
      <fetch_min_bytes>100000</fetch_min_bytes>
  </kafka_football>

Some users had to find out the hard way that such configuration doesn't
take effect if the topic name contains a period, e.g. "sports.football".
The reason is that ClickHouse configuration framework already uses
periods as level separators to descend the configuration hierarchy.
(Besides that, per-topic configuration at the same level as global
configuration could be considered ugly.)

Note that Kafka topics may contain characters "a-zA-Z0-9._-" (*) and
a tree-like topic organization using periods is quite common in
practice.

This PR deprecates the existing per-topic configuration syntax (but
continues to support it for backward compat) and introduces a new
per-topic configuration syntax below the global Kafka configuration of
the form:

<kafka>
   <topic name="football">
       <retry_backoff_ms>250</retry_backoff_ms>
       <fetch_min_bytes>100000</fetch_min_bytes>
   </topic>
</kafka>

The period restriction doesn't apply to XML attributes, so <topic
name="sports.football"> will work. Also, everything Kafka-related is
below <kafka>.

Considered but rejected alternatives:
- Extending Poco ConfigurationView with custom separators (e.g."/"
  instead of "."). Won't work easily because ConfigurationView only
  builds a path but defers descending the configuration tree to the
  normal configuration classes.
- Reloading the configuration file in StorageKafka (instead of reading
  the loaded file) but with a custom separator. This mode is supported
  by XML configuration. Too ugly and error-prone since the true
  configuration is composed from multiple configuration files.

(*) https://stackoverflow.com/a/37067544
2023-02-22 20:35:09 +00:00
kssenii
bac464f89b Fix 2023-02-22 14:25:08 +01:00
Kseniia Sumarokova
c242fe3e5e
Update docs/en/engines/table-engines/integrations/hudi.md
Co-authored-by: flynn <fenglv15@mails.ucas.ac.cn>
2023-02-22 12:11:42 +01:00
Kseniia Sumarokova
ef15d64895
Update docs/en/engines/table-engines/integrations/deltalake.md
Co-authored-by: flynn <fenglv15@mails.ucas.ac.cn>
2023-02-22 12:11:23 +01:00
kssenii
21fcc3b69c Add iceberg doc 2023-02-22 12:04:24 +01:00
Dan Roscigno
10c8f31811
Update docs/en/engines/table-engines/integrations/embedded-rocksdb.md 2023-02-15 13:29:33 -05:00
Dan Roscigno
0496b55003
Update docs/en/engines/table-engines/integrations/embedded-rocksdb.md 2023-02-15 13:29:26 -05:00
Antonio Andelic
da4389a763 Add docs for KeeperMap 2023-02-15 15:29:11 +01:00
Ryadh DAHIMENE
82a36f50c9
Adding missing integrations to the list 2023-02-08 15:11:18 +01:00
kssenii
bbf060bf89 Fix test 2023-02-01 18:22:05 +01:00
DanRoscigno
917e3d4244 add PARTITION BY to s3 and hdfs docs 2023-01-25 09:09:28 -05:00
Tanya Bragin
b5773caeb2
Update postgresql.md
Adding related Postgres blog here. 

cc @gingerwizard @DanRoscigno
2023-01-14 14:34:33 -08:00
Dan Roscigno
0ad969171e
Merge pull request #45127 from DanRoscigno/add-deltalake-docs
Add deltalake docs
2023-01-11 08:07:42 -05:00
Dan Roscigno
6e9669cfae
Apply suggestions from code review 2023-01-11 07:53:37 -05:00
Dan Roscigno
02261b0e2a
Update docs/en/engines/table-engines/integrations/hudi.md 2023-01-11 07:42:15 -05:00
Dan Roscigno
73ef2657dd
Update docs/en/engines/table-engines/integrations/hudi.md 2023-01-11 07:40:10 -05:00
Dan Roscigno
21573028ea
Update docs/en/engines/table-engines/integrations/deltalake.md 2023-01-11 07:38:41 -05:00
DanRoscigno
563e0e76f9 init 2023-01-10 16:59:34 -05:00
DanRoscigno
5cf1b1f61d feedback 2023-01-10 16:09:59 -05:00
DanRoscigno
879ee05218 fix case of names 2023-01-10 11:18:33 -05:00
Dan Roscigno
f8a5d5cb18
Merge branch 'ClickHouse:master' into add-deltalake-docs 2023-01-09 15:45:08 -05:00
Ivan Blinkov
61c2f23713 Remove leftover empty lines at the end of markdown files 2023-01-09 15:15:18 +01:00
Ivan Blinkov
b7e082d033 Remove "Original article links" 2023-01-09 15:13:36 +01:00
DanRoscigno
5cde7762ad WIP 2023-01-05 15:54:44 -05:00
Kruglov Pavel
c5b2e4cc23
Merge branch 'master' into improve-streaming-engines 2022-12-15 18:44:35 +01:00
Sergei Trifonov
f2f0676bcc
Revert "Revert "S3 request per second rate throttling"" 2022-11-17 17:35:04 +01:00
Alexander Tokmakov
9011a18234
Revert "S3 request per second rate throttling" 2022-11-16 22:33:48 +03:00
Sergei Trifonov
159743edd6
Merge pull request #43014 from ClickHouse/disk-s3-throttler
S3 request per second rate throttling
2022-11-16 18:51:06 +01:00
serxa
7448d1bb5d add docs 2022-11-09 14:58:41 +00:00
DanRoscigno
34f90ff6ef update pages that refer to dictionaries 2022-11-07 09:26:50 -05:00
Kruglov Pavel
b124875257
Merge branch 'master' into improve-streaming-engines 2022-11-03 13:22:06 +01:00
Alexander Yakovlev
70eaf69df1
Fix a typo in table-engines/integrations/s3.md 2022-11-01 02:57:55 +03:00
avogar
8e13d1f1ec Improve and refactor Kafka/StorageMQ/NATS and data formats 2022-10-28 16:41:10 +00:00
Lloyd-Pottiger
5a6b2106b5 Merge branch 'master' of github.com:ClickHouse/ClickHouse into feat/support-read-only-for-embeddedrocksdb 2022-09-02 18:10:00 +08:00
Lloyd-Pottiger
59dccd6e49 fix test
Signed-off-by: Lloyd-Pottiger <yan1579196623@gamil.com>
2022-09-02 11:14:49 +08:00
DanRoscigno
c4b8137d31 replace symlinks 2022-08-29 12:19:50 -04:00
DanRoscigno
5b5fcc56aa add slugs 2022-08-28 10:53:34 -04:00
Lloyd-Pottiger
ac3cf0beb4 add read_only option
Signed-off-by: Lloyd-Pottiger <yan1579196623@gamil.com>
2022-08-23 20:58:57 +08:00
Antonio Andelic
0765495be1
Merge pull request #39986 from Lloyd-Pottiger/feat/support-ttl-for-embeddedrocksdb
Support TTL for EmbeddedRocksDB
2022-08-23 08:27:47 +02:00
Lloyd-Pottiger
85a1a36167 remove read_only option
Signed-off-by: Lloyd-Pottiger <yan1579196623@gamil.com>
2022-08-22 23:31:55 +08:00
DanRoscigno
23b054617e update for setting change 2022-08-18 16:05:44 -04:00
Lloyd-Pottiger
c5d81e160c add functionality test
Signed-off-by: Lloyd-Pottiger <yan1579196623@gamil.com>
2022-08-10 23:10:52 +08:00
Lloyd-Pottiger
5488a392ab address comments
Signed-off-by: Lloyd-Pottiger <yan1579196623@gamil.com>
2022-08-10 21:20:25 +08:00
Lloyd-Pottiger
7ecaf58981 revise docs
Signed-off-by: Lloyd-Pottiger <yan1579196623@gamil.com>
2022-08-09 21:48:49 +08:00
Dan Roscigno
2a6765ce8c
Merge pull request #38603 from den-crane/patch-17
Doc. More settings / mysql
2022-07-06 15:24:13 -04:00