thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-13 09:52:38 +00:00
Code Issues Packages Projects Releases Wiki Activity
109,875 Commits 9,324 Branches 1,783 Tags 2.1 GiB
a170a909a4
Commit Graph

160 Commits

Author SHA1 Message Date
AVMusorin
0f2ae72141
prohibit DEFAULT/EPHEMERAL/ALIAS in KafkaEngine 2023-03-03 16:25:42 +01:00
Antonio Andelic
890f094829 Set retention policy for Kafka topic 2023-01-04 12:04:00 +00:00
Kruglov Pavel
c3405be272
Increase the number of attempts in tests 2022-12-27 16:49:32 +01:00
avogar
8e13d1f1ec Improve and refactor Kafka/StorageMQ/NATS and data formats 2022-10-28 16:41:10 +00:00
Kruglov Pavel
d4d0ec1c95
Fix style 2022-08-26 16:44:54 +02:00
avogar
b8061532ac Add setting to disable limit on kafka_num_consumers 2022-08-26 12:49:39 +00:00
Anton Popov
e911900054 remove last mentions of data streams 2022-05-09 19:15:24 +00:00
Nikolai Kochetov
ebfdadd6f8 Fix firmatting. 2022-04-07 12:05:21 +00:00
Mikhail Filimonov
f9a4cd12d4
Make parallel reading of kafka consumers work again (issue #35153) 2022-04-05 21:46:56 +02:00
Mikhail f. Shiryaev
e6f5a3f98b
Apply black formatter to all *.py files in the repo 2022-03-22 17:39:58 +01:00
alesapin
1eefdc8df6 Fix more test 2022-03-15 17:31:33 +01:00
alesapin
ace8169732 Fix kafka test 2022-01-31 19:18:34 +03:00
Azat Khuzhin
63eac4b0cf tests/integration/test_storage_kafka: cover CustomSeparated 2021-12-31 12:02:21 +03:00
Azat Khuzhin
f1c726a675 tests/integration/test_storage_kafka: cover Template format 2021-12-31 11:25:08 +03:00
Azat Khuzhin
52ad7a4538 Fix test_storage_kafka failures by adjusting retention.ms 
This test uses predefined timestamps, and default retention.ms is too
small:

    kafka1_1           | [2021-12-28 21:40:21,842] INFO Created log for partition virt2_0-0 in /var/lib/kafka/data with properties {compression.type -> producer, message.format.version -> 2.2-IV1, file.delete.delay.ms -> 60000, max.message.bytes -> 1000012, min.compaction.lag.ms -> 0, message.timestamp.type -> CreateTime, message.downconversion.enable -> true, min.insync.replicas -> 1, segment.jitter.ms -> 0, preallocate -> false, min.cleanable.dirty.ratio -> 0.5, index.interval.bytes -> 4096, unclean.leader.election.enable -> false, retention.bytes -> -1, delete.retention.ms -> 86400000, cleanup.policy -> [delete], flush.ms -> 9223372036854775807, segment.ms -> 604800000, segment.bytes -> 1073741824, retention.ms -> 604800000, message.timestamp.difference.max.ms -> 9223372036854775807, segment.index.bytes -> 10485760, flush.messages -> 9223372036854775807}. (kafka.log.LogManager)

    kafka1_1           | [2021-12-28 21:40:24,540] INFO [Log partition=virt2_0-0, dir=/var/lib/kafka/data] Found deletable segments with base offsets [0] due to retention time 604800000ms breach (kafka.log.Log)

v2: fix tests with 0 timestamp_ms
    and this should be all tests that was left:

    $ fgrep 'Found deletable segments with base offsets' test_storage_kafka/_instances_0/docker.log
    kafka1_1           | [2021-12-29 09:46:15,610] INFO [Log partition=avro1-0, dir=/var/lib/kafka/data] Found deletable segments with base offsets [0] due to retention time 604800000ms breach (kafka.log.Log)
    kafka1_1           | [2021-12-29 09:51:15,609] INFO [Log partition=virt1-0, dir=/var/lib/kafka/data] Found deletable segments with base offsets [0] due to retention time 604800000ms breach (kafka.log.Log)
 2021-12-30 10:25:10 +03:00
Vitaly Baranov
b5b195f4e2
Merge branch 'master' into kafka_protobuf_issue26643 2021-12-10 23:22:35 +03:00
kssenii
2ea4e7c4e7 Better 2021-11-29 15:05:02 +00:00
kssenii
2f0fe4a21f Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into rabbit-kafka-select-commit 2021-11-29 10:08:14 +00:00
Kruglov Pavel
b39c19399f
Merge pull request #31691 from kssenii/testing-1 
Allow named_collections for kafka / rabbit
 2021-11-25 14:06:31 +03:00
kssenii
980c87c466 Add named collection for kafka / rabbit 2021-11-24 14:53:01 +03:00
kssenii
35a4be601a Select query commit only with settings 2021-11-03 15:43:58 +03:00
Yatsishin Ilya
d5138e1427 Merge remote-tracking branch 'origin' into integration--7 2021-11-02 17:41:40 +03:00
Kruglov Pavel
901ebcede6
Merge pull request #30351 from arenadata/ADQM-335 
output_format_avro_rows_in_file
 2021-11-02 12:25:27 +03:00
Ilya Golshtein
8efa1743cf output_format_avro_rows_in_file fix and test for _timestamp 2021-11-01 13:09:18 +03:00
avogar
0ce893cb10 Fix kafka test 2021-10-28 14:20:29 +03:00
Kruglov Pavel
11c0d08d1c
Fix test again 2021-10-21 16:46:27 +03:00
Kruglov Pavel
bd016f384e
Try fix tests 2021-10-21 12:54:10 +03:00
avogar
cd17643eb8 Fix tests 2021-10-21 09:22:42 +03:00
Ilya Golshtein
d90302aa3b output_format_avro_rows_in_file 2021-10-18 19:01:06 +03:00
Yatsishin Ilya
0efeaf5dc4 Merge remote-tracking branch 'origin' into integration--7 2021-10-08 11:22:55 +03:00
Nikolai Kochetov
78e1db209f
Remove more data streams (#29491) 
* Remove more streams.

* Fixing build.

* Fixing build.

* Rename files.

* Fix fast test.

* Fix StorageKafka.

* Try fix kafka test.

* Move createBuffer to KafkaSource ctor.

* Revert "Move createBuffer to KafkaSource ctor."

This reverts commit 81fa94d27e.

* Revert "Try fix kafka test."

This reverts commit 2107e54969.

* Comment some rows in test.

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2021-10-07 11:26:08 +03:00
Yatsishin Ilya
c32e833f25 better 2021-10-06 16:23:29 +03:00
Yatsishin Ilya
b0a5a8532f improve test_storage_kafka test 2021-10-06 16:08:25 +03:00
Mikhail Filimonov
ea320c96d5 Test for issue #26643 2021-10-04 14:38:10 +00:00
Nikolai Kochetov
193f566568 Fix exception message. 2021-09-24 14:54:39 +03:00
Nikolai Kochetov
24dac111b7 Try fix kafka test. 2021-09-24 13:30:57 +03:00
alesapin
36abf9f2f4 Fix broken kafka test 2021-09-03 10:51:37 +03:00
Yatsishin Ilya
da4e688293 Improve kafka topic creation and deletion. Improve mongodb tests 2021-08-30 15:58:21 +03:00
Yatsishin Ilya
d9e782d379 Merge remote-tracking branch 'origin' into mute-integration-debug-fails 2021-08-20 18:59:16 +03:00
Yatsishin Ilya
812a6ffb80 kafka assertion was fixed in master 2021-08-20 18:56:15 +03:00
Yatsishin Ilya
9c22d07dea Mute current failures 2021-08-19 14:32:32 +03:00
Mikhail Filimonov
05d77d2873
Safer processing of NULL messages in Kafka for some formats 2021-08-18 11:26:57 +02:00
Kseniia Sumarokova
0917488cec
Merge pull request #27382 from kssenii/cast-internal 
add Cast internal function
 2021-08-13 13:13:51 +03:00
Raúl Marín
d1d47658f5 Merge remote-tracking branch 'blessed/master' into materialization_log 2021-08-10 10:49:05 +02:00
kssenii
484c3a5d2e Update kafka test 2021-08-09 11:55:38 +00:00
Raúl Marín
b4e417e507 test_kafka_virtual_columns_with_materialized_view: Limit iterations and show better error 2021-08-06 13:39:33 +02:00
Raúl Marín
4f1b02426a test_storage_kafka: Remove duplicated code 2021-08-06 13:38:56 +02:00
Raúl Marín
145699d0c2 test_kafka_no_holes_when_write_suffix_failed: Fix expected error message 
The old error message was incorrect (it said a failure happened when
pushing the prefix when it was happening while pushing the suffix.
 2021-07-27 16:29:34 +02:00
Raúl Marín
ff249a14af Fix test_kafka_no_holes_when_write_suffix_failed 2021-07-27 16:29:34 +02:00
vdimir
dd3e1a8a4d
Adjust exception message in test_replicated_fetches_timeouts 2021-07-15 11:58:21 +03:00