Commit Graph

252 Commits

Author SHA1 Message Date
Mikhail Filimonov
1871de3620 fix Kafka test 2024-10-04 00:51:54 +02:00
Alexey Milovidov
0356287344
Merge pull request #70038 from ClickHouse/isort
Add isort to the style checking for python files
2024-09-28 01:47:37 +00:00
robot-clickhouse
7c6ce8611c Automatic style fix 2024-09-27 10:19:49 +00:00
Tuan Pham Anh
608c686573 Reformat test_storage_kafka 2024-09-27 02:09:39 +00:00
Tuan Pham Anh
9101beb79e Add check_callback in query_with_retry in test_storage_kafka test 2024-09-27 01:08:15 +00:00
Tuan Pham Anh
29ecba1a35 Use query_with_retry in test_storage_kafka test 2024-09-26 15:29:39 +00:00
Nikita Mikhaylov
9667c19b6e
Revert "Speed up some Kafka tests with multiprocessing" 2024-09-07 09:16:38 +02:00
Antonio Andelic
86f11e221e Speed up more tests 2024-09-06 17:08:35 +02:00
Antonio Andelic
06507190d4 Speed up some Kafka tests with multiprocessing 2024-09-06 14:29:26 +02:00
János Benjamin Antal
ac6826392d
Merge pull request #67554 from ClickHouse/fix-message-queue-sink-from-http-interface
Fix message queue sink from http interface
2024-08-12 21:29:14 +00:00
János Benjamin Antal
737948470d Fix syntax 2024-08-12 11:54:34 +00:00
János Benjamin Antal
8f124710ef Remove Log engine from Kafka integration tests
It doesn't work well when `thread_per_consumer` is used as writer can make readers starve when `shared_time_mutex` prefers writes over reads.
2024-08-12 10:27:21 +00:00
János Benjamin Antal
556f669878 Separate test into separate file to pass bugfix check 2024-08-10 21:53:22 +00:00
János Benjamin Antal
21be195dd8 Revert unnecessary change 2024-08-09 13:14:09 +00:00
János Benjamin Antal
2900c7006e Merge remote-tracking branch 'origin/master' into fix-message-queue-sink-from-http-interface 2024-08-09 13:06:58 +00:00
robot-clickhouse
94398996b2 Automatic style fix 2024-08-07 17:47:22 +00:00
János Benjamin Antal
d992431f96 Make tests more sturdier for flaky test check 2024-08-07 17:27:24 +00:00
János Benjamin Antal
b3c30b05d6 Make test_kafka_insert repeatable 2024-08-07 13:22:22 +00:00
János Benjamin Antal
139fe624c2 Fix typos 2024-08-07 13:16:47 +00:00
János Benjamin Antal
8b5c8e5361 Merge remote-tracking branch 'origin/master' into kafka-zookeeper 2024-08-07 11:33:21 +00:00
János Benjamin Antal
f816158dbc Address review comments 2024-08-05 09:11:55 +00:00
János Benjamin Antal
0e36db5437 Do not check the value of num_messages_read because it is not reliable
librdkafka usually polls both messages on the first SELECT, but sometimes only one. If it polls only one message first, then it will read two messages before rebalancing at the second SELECT from the table. This means it usually reads a single message twice (thus num_messages_read = 4 is usually fine as 1 discarded message + 3 actually consumed message). But when only one message is read in the first SELECT, then 2 messages are discarded, thus num_messages_read will be 5 as 2 discarded message + 3 actually consumed messages.
2024-07-31 22:09:04 +00:00
robot-clickhouse
bada9ea9be Automatic style fix 2024-07-31 17:30:32 +00:00
János Benjamin Antal
e7fc206069 Add test 2024-07-31 17:20:01 +00:00
robot-clickhouse
87e9b7c5bb Automatic style fix 2024-07-15 13:13:56 +00:00
János Benjamin Antal
123fd6b750 Remove seemingly unnecessary nullptr check 2024-07-15 12:40:44 +00:00
János Benjamin Antal
1ecfba837e Rename experimental flag to allow_experimental_kafka_offsets_storage_in_keeper 2024-07-15 09:03:05 +00:00
robot-clickhouse
f2e4ec28cb Automatic style fix 2024-06-18 11:51:51 +00:00
János Benjamin Antal
03500bbe2f Style fix 2024-06-18 11:43:05 +00:00
János Benjamin Antal
6e1a901510 Make big chunk of integration tests to work with new storage kafka 2024-06-18 11:42:58 +00:00
robot-clickhouse
7b90eccd57 Automatic style fix 2024-06-08 22:26:09 +00:00
János Benjamin Antal
be64d1bd32 Fix tests 2024-06-08 22:16:52 +00:00
János Benjamin Antal
419660d1b0 Fix one more test 2024-06-08 22:00:58 +00:00
robot-clickhouse
26851f1d34 Automatic style fix 2024-06-07 22:56:36 +00:00
János Benjamin Antal
1eec201777 Make most of the tests work 2024-06-07 22:39:17 +00:00
János Benjamin Antal
c0eea71ab3 Make some tests work with the new storage kafka too 2024-06-04 15:52:59 +00:00
Aleksandr Musorin
f9d81bc774 refactore: improve reading several configurations for kafka
Simplify and do some refactoring for kafka client settings.

Allows to set up separate
settings for consumer and producer like:

```
<consumer>
    ...
</consumer>

<producer>
    <kafka_topic>
        <name>topic_name</name>
        ...
    </kafka_topic>
</producer>
```

Moreover, this fixes warnings from kafka client like:
`Configuration property session.timeout.ms is a consumer property and
will be ignored by this producer instance`
2024-03-27 14:53:33 +01:00
Alexey Milovidov
06a9ac7126 Fix tests 2024-03-24 23:23:13 +01:00
Max Kainov
55737de0e6 disable more hdfs and kafka inttests
#ci_set_arm
2024-03-20 17:24:55 +00:00
Pervakov Grigorii
d3b82ab7c1 Support sql created named collections in Kafka Storage for librdkafka settings 2024-02-27 18:36:36 +00:00
Kseniia Sumarokova
aec76d6653
Merge pull request #58877 from ClickHouse/multiple-read-file-log-storage-in-mv
Multiple read file log storage in mv
2024-01-22 12:02:41 +01:00
Kruglov Pavel
6d064512e1
Merge pull request #58614 from Blargian/58363_disable_ansi_pretty_automatically
58363 Automatically disable ANSI escape sequences in Pretty formats if the output is not a terminal
2024-01-17 13:45:41 +01:00
Blargian
1b81efc337 fix failing 02421 stateless and (hopefully) fix failing integration tests 2024-01-16 19:48:23 +02:00
robot-clickhouse
7262e93a27 Automatic style fix 2024-01-16 16:26:28 +00:00
János Benjamin Antal
5a178ce1ef Add proper cleanup to kafka test 2024-01-16 16:03:44 +00:00
robot-clickhouse
224a58ddc5 Automatic style fix 2024-01-04 10:30:44 +00:00
János Benjamin Antal
1a1fa07128 Add missing asserts 2024-01-04 10:18:04 +00:00
János Benjamin Antal
2517d23e27 Add tests 2024-01-04 10:14:41 +00:00
avogar
f20917048e Fix kafka tests 2023-12-29 17:02:46 +00:00
Kruglov Pavel
9389ea8745 Fix style 2023-12-28 18:47:57 +00:00