Commit Graph

108663 Commits

Author SHA1 Message Date
Robert Schulze
6906cafa99
Change to S3 cfg syntax 2023-02-23 20:04:41 +00:00
Robert Schulze
81bf43157f
Allow configuration of Kafka topics with periods
The Kafka table engine allows global configuration and per-Kafka-topic
configuration. The latter uses syntax <kafka_TOPIC>, e.g. for topic
"football":

  <kafka_football>
      <retry_backoff_ms>250</retry_backoff_ms>
      <fetch_min_bytes>100000</fetch_min_bytes>
  </kafka_football>

Some users had to find out the hard way that such configuration doesn't
take effect if the topic name contains a period, e.g. "sports.football".
The reason is that ClickHouse configuration framework already uses
periods as level separators to descend the configuration hierarchy.
(Besides that, per-topic configuration at the same level as global
configuration could be considered ugly.)

Note that Kafka topics may contain characters "a-zA-Z0-9._-" (*) and
a tree-like topic organization using periods is quite common in
practice.

This PR deprecates the existing per-topic configuration syntax (but
continues to support it for backward compat) and introduces a new
per-topic configuration syntax below the global Kafka configuration of
the form:

<kafka>
   <topic name="football">
       <retry_backoff_ms>250</retry_backoff_ms>
       <fetch_min_bytes>100000</fetch_min_bytes>
   </topic>
</kafka>

The period restriction doesn't apply to XML attributes, so <topic
name="sports.football"> will work. Also, everything Kafka-related is
below <kafka>.

Considered but rejected alternatives:
- Extending Poco ConfigurationView with custom separators (e.g."/"
  instead of "."). Won't work easily because ConfigurationView only
  builds a path but defers descending the configuration tree to the
  normal configuration classes.
- Reloading the configuration file in StorageKafka (instead of reading
  the loaded file) but with a custom separator. This mode is supported
  by XML configuration. Too ugly and error-prone since the true
  configuration is composed from multiple configuration files.

(*) https://stackoverflow.com/a/37067544
2023-02-22 20:35:09 +00:00
pufit
8e7533fa57
Merge pull request #46564 from AVMusorin/update-time-distribution-queue
Added `last_exception_time` column into distribution_queue table
2023-02-22 10:43:35 -05:00
Alexander Gololobov
b0427c2e3c
Merge pull request #46660 from ClickHouse/fix_backup_test
Fix integration test: terminate old version without wait
2023-02-22 15:20:26 +01:00
Antonio Andelic
7f5fb77ed5
Increase table retries in cluster copier tests (#46590) 2023-02-22 15:09:48 +01:00
Kseniia Sumarokova
bec094cd79
Merge pull request #46712 from kssenii/add-iceberg-doc
Add iceberg engine doc
2023-02-22 14:49:03 +01:00
kssenii
bac464f89b Fix 2023-02-22 14:25:08 +01:00
Robert Schulze
9d116e6f5c
Merge pull request #46710 from ClickHouse/rs/bump-clang
Bump minimum required Clang from 12 to 15
2023-02-22 13:38:21 +01:00
Kruglov Pavel
3ba3fdbfa3
Merge pull request #46607 from kssenii/delay-loading-of-named-collections
Do not load named collections on server startup (on first access instead)
2023-02-22 13:22:34 +01:00
Dmitry Novik
67469ad46b
Merge pull request #46622 from ClickHouse/async-insert-memory-fix
Fix MemoryTracker counters for async inserts
2023-02-22 12:27:05 +01:00
Kseniia Sumarokova
c242fe3e5e
Update docs/en/engines/table-engines/integrations/hudi.md
Co-authored-by: flynn <fenglv15@mails.ucas.ac.cn>
2023-02-22 12:11:42 +01:00
Kseniia Sumarokova
ef15d64895
Update docs/en/engines/table-engines/integrations/deltalake.md
Co-authored-by: flynn <fenglv15@mails.ucas.ac.cn>
2023-02-22 12:11:23 +01:00
kssenii
21fcc3b69c Add iceberg doc 2023-02-22 12:04:24 +01:00
flynn
678e4250cd
Fix incorrect predicate push down with grouping sets (#46151) 2023-02-22 11:54:19 +01:00
robot-clickhouse-ci-2
2df52af445
Merge pull request #46711 from ClickHouse/vdimir/tmp-data-in-fs-cache-doc
Add doc for temporary_data_in_cache
2023-02-22 11:41:03 +01:00
Kseniia Sumarokova
3f0d93d6e6
Merge pull request #46656 from ClickHouse/kssenii-patch-6
Update postgres_utility.py
2023-02-22 11:35:03 +01:00
vdimir
a4919ce3a2
Add doc for temporary_data_in_cache 2023-02-22 10:19:28 +00:00
Robert Schulze
16d61832fb
Bump minimum required Clang from 12 to 15
Needed due to https://github.com/ClickHouse/ClickHouse/pull/46247#discussion_r1109855435
2023-02-22 10:03:08 +00:00
Alexey Milovidov
5788deeadd
Merge pull request #46308 from ClickHouse/keeper-retries-by-default
Enable retries for INSERT by default in case of ZooKeeper session loss
2023-02-22 07:57:40 +03:00
Alexey Milovidov
2ae0b43570
Merge pull request #46626 from ClickHouse/fix-tests
Inhibit `index_granularity_bytes` randomization in some tests
2023-02-22 04:55:23 +03:00
AVMusorin
0bf0fe488e
added last_exception_time column into distribution_queue table 2023-02-21 23:46:57 +01:00
Dmitry Novik
ad1e5f3918 Review fixes 2023-02-21 22:06:17 +00:00
Dmitry Novik
8232966b9e Add a comment 2023-02-21 22:02:23 +00:00
robot-clickhouse
e010af8be8
Merge pull request #46649 from xiedeyantu/patch-2
Update s3Cluster.md
2023-02-21 22:48:38 +01:00
Alexey Milovidov
4adbcd4beb
Merge pull request #46665 from ClickHouse/clean-docs-release
Get rid of legacy DocsReleaseChecks
2023-02-22 00:31:17 +03:00
Alexey Milovidov
8eae18616f
Merge pull request #46671 from Kinzeng/dashboard-improvements
fix layout issues in dashboard.html
2023-02-22 00:30:18 +03:00
Alexander Tokmakov
6439743de6
Merge pull request #46683 from ClickHouse/tavplubix-patch-5
Fix Stress tests
2023-02-22 00:01:58 +03:00
Alexander Gololobov
b61bb56a5a
Merge pull request #46611 from ClickHouse/prewhere_progress
Fix reporting progress when rows are filtered in PREWHERE
2023-02-21 21:27:56 +01:00
Alexander Tokmakov
dcf8aeab8f
Update stress 2023-02-21 22:16:05 +03:00
Kevin Zhang
d9cff3a5e8 fix layout issues in dashboard.html 2023-02-21 13:52:28 -05:00
Alexey Milovidov
3ab54ac0ad Fix test output 2023-02-21 19:24:10 +01:00
Vitaly Baranov
49330b373c
Merge pull request #45982 from ClickHouse/Cluster_state_for_disallow_concurrent_backup_restore
Use cluster state data to check concurrent backup/restore
2023-02-21 19:18:18 +01:00
Alexey Milovidov
bf5640a597
Merge pull request #46629 from ClickHouse/fix-coverity
Fix coverity
2023-02-21 21:04:20 +03:00
Han Fei
fa5d06c4da
Merge pull request #46666 from hanfei1991/hanfei/add-doc-async-insert-max-query-number
add docs for setting async_insert_max_query_number
2023-02-21 19:02:37 +01:00
Dan Roscigno
efea3cbc5c
Update docs/en/operations/settings/settings.md 2023-02-21 12:19:56 -05:00
robot-ch-test-poll3
8a4b073485
Merge pull request #46664 from ClickHouse/DanRoscigno-patch-4
fix anchor link
2023-02-21 18:13:18 +01:00
Alexey Milovidov
290e10623d
Merge pull request #46630 from ClickHouse/alexey-milovidov-patch-6
Installation instructions in README
2023-02-21 20:02:26 +03:00
Alexey Milovidov
6822ad67e2
Update README.md 2023-02-21 20:02:12 +03:00
Igor Nikonov
bf05082984
Merge branch 'master' into keeper-retries-by-default 2023-02-21 17:59:55 +01:00
Alexey Milovidov
848286eca7
Merge pull request #40945 from arthurpassos/add_select_final_mt_setting
Add user setting to force select final at query level
2023-02-21 19:58:45 +03:00
Han Fei
eeac1abaa5 add docs for setting async_insert_max_query_number 2023-02-21 17:49:52 +01:00
Mikhail f. Shiryaev
cd2ab02f22
Get rid of legacy DocsReleaseChecks 2023-02-21 17:34:16 +01:00
Dan Roscigno
b6612d2c18
fix anchor link 2023-02-21 11:24:39 -05:00
Alexander Tokmakov
9e2126fc1a
Merge pull request #46663 from ClickHouse/tavplubix-patch-5
Break Stress tests
2023-02-21 19:18:16 +03:00
Alexander Tokmakov
97e9df0148
Update stress 2023-02-21 19:10:44 +03:00
robot-clickhouse
af677c7dcd Automatic style fix 2023-02-21 16:08:13 +00:00
alesapin
1d4352d82a Fix integration test: terminate old version without wait 2023-02-21 17:01:19 +01:00
Kseniia Sumarokova
3df7a10ac7
Update postgres_utility.py 2023-02-21 16:25:11 +01:00
Kseniia Sumarokova
655e7f29f7
Merge pull request #46595 from kssenii/update-rabbit-producer
Fix rabbitmq test
2023-02-21 16:18:55 +01:00
chen
c1e611334a
Update s3Cluster.md 2023-02-21 21:53:44 +08:00