Commit Graph

222 Commits

Author SHA1 Message Date
robot-clickhouse
7b90eccd57 Automatic style fix 2024-06-08 22:26:09 +00:00
János Benjamin Antal
be64d1bd32 Fix tests 2024-06-08 22:16:52 +00:00
János Benjamin Antal
419660d1b0 Fix one more test 2024-06-08 22:00:58 +00:00
robot-clickhouse
26851f1d34 Automatic style fix 2024-06-07 22:56:36 +00:00
János Benjamin Antal
1eec201777 Make most of the tests work 2024-06-07 22:39:17 +00:00
János Benjamin Antal
c0eea71ab3 Make some tests work with the new storage kafka too 2024-06-04 15:52:59 +00:00
Aleksandr Musorin
f9d81bc774 refactore: improve reading several configurations for kafka
Simplify and do some refactoring for kafka client settings.

Allows to set up separate
settings for consumer and producer like:

```
<consumer>
    ...
</consumer>

<producer>
    <kafka_topic>
        <name>topic_name</name>
        ...
    </kafka_topic>
</producer>
```

Moreover, this fixes warnings from kafka client like:
`Configuration property session.timeout.ms is a consumer property and
will be ignored by this producer instance`
2024-03-27 14:53:33 +01:00
Alexey Milovidov
06a9ac7126 Fix tests 2024-03-24 23:23:13 +01:00
Max Kainov
55737de0e6 disable more hdfs and kafka inttests
#ci_set_arm
2024-03-20 17:24:55 +00:00
Pervakov Grigorii
d3b82ab7c1 Support sql created named collections in Kafka Storage for librdkafka settings 2024-02-27 18:36:36 +00:00
Kseniia Sumarokova
aec76d6653
Merge pull request #58877 from ClickHouse/multiple-read-file-log-storage-in-mv
Multiple read file log storage in mv
2024-01-22 12:02:41 +01:00
Kruglov Pavel
6d064512e1
Merge pull request #58614 from Blargian/58363_disable_ansi_pretty_automatically
58363 Automatically disable ANSI escape sequences in Pretty formats if the output is not a terminal
2024-01-17 13:45:41 +01:00
Blargian
1b81efc337 fix failing 02421 stateless and (hopefully) fix failing integration tests 2024-01-16 19:48:23 +02:00
robot-clickhouse
7262e93a27 Automatic style fix 2024-01-16 16:26:28 +00:00
János Benjamin Antal
5a178ce1ef Add proper cleanup to kafka test 2024-01-16 16:03:44 +00:00
robot-clickhouse
224a58ddc5 Automatic style fix 2024-01-04 10:30:44 +00:00
János Benjamin Antal
1a1fa07128 Add missing asserts 2024-01-04 10:18:04 +00:00
János Benjamin Antal
2517d23e27 Add tests 2024-01-04 10:14:41 +00:00
avogar
f20917048e Fix kafka tests 2023-12-29 17:02:46 +00:00
Kruglov Pavel
9389ea8745 Fix style 2023-12-28 18:47:57 +00:00
Kruglov Pavel
69f6147499 Fix style 2023-12-28 18:47:57 +00:00
avogar
c6085b3eb9 Fix working with read buffers in StreamingFormatExecutor 2023-12-28 18:47:57 +00:00
Raúl Marín
6c76ad1ea4 Cleanup 2023-08-22 11:57:56 +02:00
Raúl Marín
f2d3c9c281 Merge remote-tracking branch 'blessed/master' into better_zk_retries_on_insert 2023-08-21 12:14:31 +02:00
Ilya Golshtein
978d8bf9a6 system_kafka_consumers: cleanup with exceptions 2023-08-10 22:35:05 +00:00
Ilya Golshtein
7dab2dc041 system_kafka_consumers: exceptions seem to work 2023-08-10 22:35:05 +00:00
Ilya Golshtein
e8b10f0126 system_kafka_consumers: black formatter thing 2023-08-10 22:35:05 +00:00
Ilya Golshtein
80829a6bf7 system_kafka_consumers: no offset_committed, new test with MV 2023-08-10 22:35:05 +00:00
Ilya Golshtein
ae3e6f0fac system_kafka_consumers: getMemberId introduced to avoid leak 2023-08-10 22:35:05 +00:00
Ilya Golshtein
2cc4bb355c system_kafka_consumers: make black formatter happy 2023-08-10 22:35:05 +00:00
Ilya Golshtein
46be5e5782 system_kafka_consumers: cleanup, test should be more stable 2023-08-10 22:35:05 +00:00
Ilya Golshtein
1f03e320ee system_kafka_consumers: test fix + black formatter 2023-08-10 22:35:05 +00:00
Ilya Golshtein
b71c2bedfc system_kafka_consumers: rebalance test is ok, statistics_interval_ms 2023-08-10 22:35:05 +00:00
Ilya Golshtein
ae6e0469ad system_kafka_consumers: rdfakfa stat with attempt to use json 2023-08-10 22:35:05 +00:00
Ilya Golshtein
40fd53c3e3 system_kafka_consumers: style check 2023-08-10 22:35:05 +00:00
Ilya Golshtein
88f5dd5db8 system_kafka_consumers: integration tests passed 2023-08-10 22:35:05 +00:00
Ilya Golshtein
5e1c72a0d0 system_kafka_consumers: test with rebalance 2023-08-10 22:35:05 +00:00
Ilya Golshtein
957787d96a system_kafka_consumers: kind of works, test added 2023-08-10 22:35:05 +00:00
Yury Bogomolov
ba081ed402
Merge branch 'master' into support-predefined-macro-in-kafka-settings 2023-07-29 15:00:21 +04:00
Yury Bogomolov
b2ee4505fb review fixes 2023-07-28 16:06:03 +00:00
Raúl Marín
e2a3a0e37a Adapt test 2023-07-28 13:33:14 +02:00
Alexey Milovidov
a1e9a76ead
Merge branch 'master' into 51844-fix-test_storage_kafkatestpytest_kafka_formats_with_broken_message-test 2023-07-28 13:18:10 +03:00
Alexey Milovidov
91fd286165 Merge branch 'master' into sleep-each-row-max-time 2023-07-22 02:25:03 +02:00
Yury Bogomolov
c8b128aad4 add predefined macros support in kafka engine settings 2023-07-21 01:06:49 +04:00
János Benjamin Antal
16cc007842 Fix table ad variable name 2023-07-20 09:21:18 +00:00
Ilya Golshtein
aa2b7e248d test_for_basic_auth_registry - fix port in test_kafka_formats 2023-07-19 08:32:45 +00:00
Alexey Milovidov
b2164ab97e Fix integration tests 2023-07-18 22:13:28 +02:00
János Benjamin Antal
6a104cc3f7 Add checked waits to improve kafka tests
It might take a few minutes to receive all the messages for the last
materialized view. By waiting to the number of expected results the
happy path of execution takes minimal time while becoming more stable,
while the erroneous path might take a bit longer.
2023-07-18 16:04:41 +00:00
kssenii
8073e0bad1 Fix tests 2023-06-14 13:45:53 +02:00
AVMusorin
418a61a68c
Allow using Alias column type for KafkaEngine
```
create table kafka
(
 a UInt32,
 a_str String Alias toString(a)
) engine = Kafka;

create table data
(
  a UInt32;
  a_str String
) engine = MergeTree
order by tuple();

create materialized view data_mv to data
(
  a UInt32,
  a_str String
) as
select a, a_str from kafka;
```
Alias type works as expected in comparison with MATERIALIZED/EPHEMERAL
or column with default expression.

Ref: https://github.com/ClickHouse/ClickHouse/pull/47138

Co-authored-by: Azat Khuzhin <a3at.mail@gmail.com>
2023-05-15 15:39:58 +02:00