Commit Graph

64 Commits

Author SHA1 Message Date
filimonov
bd8c3eb117
avoid race in librdkafka
see https://github.com/edenhill/librdkafka/issues/3279
2021-03-04 15:01:10 +01:00
Mikhail Filimonov
9e3547681b
try to address python kafka client disconnects better 2021-02-26 07:17:31 +01:00
Mikhail Filimonov
60ede36068
simple fix for CI failure 2021-02-26 06:41:11 +01:00
Mikhail Filimonov
c4470ff283
proper placement for test 2021-02-26 06:38:14 +01:00
Mikhail Filimonov
2bcfff257c
Remove most of sleeps in tests 2021-02-25 17:02:59 +01:00
Mikhail Filimonov
6c6eaf2a60
Fixes 2021-02-25 17:02:57 +01:00
Mikhail Filimonov
a4c9e62d6d
Integration tests: introduce wait_for_log_line.
It uses tail -f executed in container, this way i don't need to pull
the file many times, rememeber positions etc.
2021-02-25 17:02:56 +01:00
Yatsishin Ilya
70963e4396 Fixed test_kafka_flush_by_time test. 2021-01-28 18:03:23 +03:00
Nikita Mikhaylov
736cc6959a
remove kafka flaky fix 2020-12-30 17:24:39 +03:00
filimonov
e81a99156f
Attempt to prevent test flap
That extra SELECT is executed synchronously, after that all further actions will have a more stable timing.
2020-12-29 18:51:00 +01:00
Nikita Mikhailov
fbf2ac35e8 fix tsan 2020-12-17 18:14:09 +03:00
Azat Khuzhin
2e569899d1 Add a test for kafka with snappy compression method (regression for UB in snappy)
I've checked it manually and it works with clang build and does not with
gcc (there are NULL bytes in the middle of value).
2020-12-14 00:38:07 +03:00
filimonov
8e2ca9518b
Update test.py
adjusting timeouts a bit, in the good hope that it will prevent flakiness of the test.
2020-12-11 10:13:03 +01:00
Mikhail Filimonov
b072bc7d2b
Test for the issue #12615 2020-11-06 17:54:44 +01:00
Mikhail Filimonov
e17d4e1eeb
Tests and some docs 2020-10-12 07:51:46 +02:00
Mikhail Filimonov
b602d18dd8
ProtobufSingle input format
Allows to read Single protobuf message at once (w/o length-delemeters).
2020-10-12 07:51:36 +02:00
filimonov
1ce0d4dbc2
Update test.py
similar change for test_kafka_duplicates_when_commit_failed
2020-10-10 23:36:59 +02:00
Mikhail Filimonov
60f4e561fb
fix test_kafka_produce_key_timestamp 2020-10-10 17:01:22 +02:00
Mikhail Filimonov
5fb549ca24
fix flap in no_holes_when_write_suffix_failed 2020-10-09 17:06:36 +02:00
Azat Khuzhin
9cb3c743bd
Convert to python3 (#15007) 2020-10-02 19:54:07 +03:00
bharatnc
b75963d370 Format and optimize imports in integration test files
This PR formats all the `*.py` files found under the `tests/integration`
folder. It also reorders the imports and cleans up a bunch of unused
imports.

The formatting also takes care of other things like wrapping lines and
fixing spaces and indents such that the tests look more readable.
2020-09-15 23:10:54 -07:00
Peng Jian
77420c82ea Add test case for Kafka engine with kafka_thread_per_consumer enabled. 2020-09-06 11:17:19 +08:00
Yatsishin Ilya
6902bbce50 Merge remote-tracking branch 'origin' into integration-explicit-configs 2020-09-02 12:48:07 +03:00
Dongdong Yang
9651c36ceb testcase for issue 14202 2020-08-31 15:42:01 +08:00
Yatsishin Ilya
adc2c117c8 fixes 2020-08-18 17:34:04 +03:00
alesapin
d387b97a69
Merge pull request #13285 from filimonov/kafka_fix_test_rdkafka_15
Fix for test_kafka_flush_by_block_size after rdkafka 1.5
2020-08-06 11:48:21 +03:00
Andrew Onyshchuk
a46049f86d test_storage_kafka: test empty message when supported by format 2020-08-04 02:43:42 -05:00
Andrew Onyshchuk
e472c83601 Add AvroConfluent kafka test 2020-08-04 02:43:42 -05:00
Mikhail Filimonov
991c673870 Fix for test_kafka_flush_by_block_size after rdkafka 1.5
Since rdkafka 1.5 (https://github.com/edenhill/librdkafka/releases/tag/v1.5.0)
don't auto-create topics when subscribing, so test start failing
because it was intentionally time-limited to check if flushing by block size worked.

Since 20.5 it's possible to increase kafka_flush_interval_ms per table, so i've
adjusted the test using that feature to test if data is flushed by block size,
not by time.
2020-08-03 18:43:37 +02:00
Alexander Kuzmenkov
4049f23188
Merge pull request #13103 from filimonov/test_for_macros_in_kafka
Add test for macros usage in some kafka settings.
2020-08-03 17:16:12 +03:00
alexey-milovidov
5e0437eeac
Merge pull request #12442 from ClickHouse/deprecate-special-zero-datetime
Deprecate special printing of zero Date/DateTime
2020-08-02 00:36:34 +03:00
alexey-milovidov
dc733e7c45
Fix Python 2020-07-31 01:23:16 +03:00
Alexey Milovidov
cfd4e9da3d Update tests 2020-07-31 00:24:14 +03:00
Alexey Milovidov
83c9d9ef45 Merge branch 'master' into deprecate-special-zero-datetime 2020-07-31 00:17:04 +03:00
Alexey Milovidov
362655fa43 Fix some bad tests 2020-07-30 22:59:49 +03:00
Mikhail Filimonov
0d55d364da Add test for macros usage in certain kafka setting.
https://github.com/ClickHouse/ClickHouse/issues/2249
918dbc2902
2020-07-30 11:28:52 +02:00
Azat Khuzhin
cdb959ec8e Fix test_storage_kafka::test_kafka_unavailable 2020-07-22 11:42:30 +03:00
Alexey Milovidov
b0b0690e1c Fix Python 2020-07-16 06:56:46 +03:00
Alexey Milovidov
126a94ab69 Update integration tests 2020-07-16 04:48:50 +03:00
Anton Popov
890bb2f582 add failing test with kafka 2020-07-14 12:24:12 +03:00
Mikhail Filimonov
9609bd9dee Kafka better states, formats based on PeekableReadBuffer, and other minor fixes.
Add formats tests, fixes for JSONCompactEachRowWithNamesAndTypes, TSVWithNamesAndTypes. Some CR fixes
Add sanitizing for kafka_max_block_size and kafka_poll_max_batch_size
2020-06-19 10:49:05 +02:00
Ivan
c8808e89fd
Update test.py 2020-06-06 19:48:03 +03:00
Mikhail Filimonov
26d93fdfba Fix the test, adjust default for queued.min.messages 2020-06-04 14:14:24 +02:00
Mikhail Filimonov
bf1e5db0ee Tests from issue #11308, from issue #4116 2020-06-03 19:01:45 +02:00
alesapin
c22e8f6b9b
Merge pull request #11283 from filimonov/kafka_virtual_column_headers
Virtual columns for Kafka headers
2020-06-01 12:06:00 +03:00
alesapin
364ebe9246
Merge pull request #11260 from filimonov/kafka_virtual_column_timestamp_ms
Add _timestamp_ms virtual columns to Kafka engine
2020-06-01 12:04:17 +03:00
alexey-milovidov
9a362eb02e
Merge pull request #11252 from filimonov/kafka-clientid
Kafka clientid (finishing #11073)
2020-05-31 18:37:30 +03:00
Mikhail Filimonov
b885337454 Virtual columns for accessing kafka message headers 2020-05-29 10:44:10 +02:00
Mikhail Filimonov
03430b1763 Change the test, accept new librdkafka behaviour 2020-05-28 23:11:29 +02:00
Mikhail Filimonov
e39c4cdf71 Add _timestamp_ms virtual columns to Kafka engine 2020-05-28 19:41:57 +02:00