Mikhail Filimonov
e17d4e1eeb
Tests and some docs
2020-10-12 07:51:46 +02:00
Mikhail Filimonov
b602d18dd8
ProtobufSingle input format
...
Allows to read Single protobuf message at once (w/o length-delemeters).
2020-10-12 07:51:36 +02:00
filimonov
1ce0d4dbc2
Update test.py
...
similar change for test_kafka_duplicates_when_commit_failed
2020-10-10 23:36:59 +02:00
Mikhail Filimonov
60f4e561fb
fix test_kafka_produce_key_timestamp
2020-10-10 17:01:22 +02:00
Mikhail Filimonov
5fb549ca24
fix flap in no_holes_when_write_suffix_failed
2020-10-09 17:06:36 +02:00
Azat Khuzhin
9cb3c743bd
Convert to python3 ( #15007 )
2020-10-02 19:54:07 +03:00
bharatnc
b75963d370
Format and optimize imports in integration test files
...
This PR formats all the `*.py` files found under the `tests/integration`
folder. It also reorders the imports and cleans up a bunch of unused
imports.
The formatting also takes care of other things like wrapping lines and
fixing spaces and indents such that the tests look more readable.
2020-09-15 23:10:54 -07:00
Peng Jian
77420c82ea
Add test case for Kafka engine with kafka_thread_per_consumer enabled.
2020-09-06 11:17:19 +08:00
Yatsishin Ilya
6902bbce50
Merge remote-tracking branch 'origin' into integration-explicit-configs
2020-09-02 12:48:07 +03:00
Dongdong Yang
9651c36ceb
testcase for issue 14202
2020-08-31 15:42:01 +08:00
Yatsishin Ilya
adc2c117c8
fixes
2020-08-18 17:34:04 +03:00
alesapin
d387b97a69
Merge pull request #13285 from filimonov/kafka_fix_test_rdkafka_15
...
Fix for test_kafka_flush_by_block_size after rdkafka 1.5
2020-08-06 11:48:21 +03:00
Andrew Onyshchuk
a46049f86d
test_storage_kafka: test empty message when supported by format
2020-08-04 02:43:42 -05:00
Andrew Onyshchuk
e472c83601
Add AvroConfluent kafka test
2020-08-04 02:43:42 -05:00
Mikhail Filimonov
991c673870
Fix for test_kafka_flush_by_block_size after rdkafka 1.5
...
Since rdkafka 1.5 (https://github.com/edenhill/librdkafka/releases/tag/v1.5.0 )
don't auto-create topics when subscribing, so test start failing
because it was intentionally time-limited to check if flushing by block size worked.
Since 20.5 it's possible to increase kafka_flush_interval_ms per table, so i've
adjusted the test using that feature to test if data is flushed by block size,
not by time.
2020-08-03 18:43:37 +02:00
Alexander Kuzmenkov
4049f23188
Merge pull request #13103 from filimonov/test_for_macros_in_kafka
...
Add test for macros usage in some kafka settings.
2020-08-03 17:16:12 +03:00
alexey-milovidov
5e0437eeac
Merge pull request #12442 from ClickHouse/deprecate-special-zero-datetime
...
Deprecate special printing of zero Date/DateTime
2020-08-02 00:36:34 +03:00
alexey-milovidov
dc733e7c45
Fix Python
2020-07-31 01:23:16 +03:00
Alexey Milovidov
cfd4e9da3d
Update tests
2020-07-31 00:24:14 +03:00
Alexey Milovidov
83c9d9ef45
Merge branch 'master' into deprecate-special-zero-datetime
2020-07-31 00:17:04 +03:00
Alexey Milovidov
362655fa43
Fix some bad tests
2020-07-30 22:59:49 +03:00
Mikhail Filimonov
0d55d364da
Add test for macros usage in certain kafka setting.
...
https://github.com/ClickHouse/ClickHouse/issues/2249
918dbc2902
2020-07-30 11:28:52 +02:00
Azat Khuzhin
cdb959ec8e
Fix test_storage_kafka::test_kafka_unavailable
2020-07-22 11:42:30 +03:00
Alexey Milovidov
b0b0690e1c
Fix Python
2020-07-16 06:56:46 +03:00
Alexey Milovidov
126a94ab69
Update integration tests
2020-07-16 04:48:50 +03:00
Anton Popov
890bb2f582
add failing test with kafka
2020-07-14 12:24:12 +03:00
Mikhail Filimonov
9609bd9dee
Kafka better states, formats based on PeekableReadBuffer, and other minor fixes.
...
Add formats tests, fixes for JSONCompactEachRowWithNamesAndTypes, TSVWithNamesAndTypes. Some CR fixes
Add sanitizing for kafka_max_block_size and kafka_poll_max_batch_size
2020-06-19 10:49:05 +02:00
Ivan
c8808e89fd
Update test.py
2020-06-06 19:48:03 +03:00
Mikhail Filimonov
26d93fdfba
Fix the test, adjust default for queued.min.messages
2020-06-04 14:14:24 +02:00
Mikhail Filimonov
bf1e5db0ee
Tests from issue #11308 , from issue #4116
2020-06-03 19:01:45 +02:00
alesapin
c22e8f6b9b
Merge pull request #11283 from filimonov/kafka_virtual_column_headers
...
Virtual columns for Kafka headers
2020-06-01 12:06:00 +03:00
alesapin
364ebe9246
Merge pull request #11260 from filimonov/kafka_virtual_column_timestamp_ms
...
Add _timestamp_ms virtual columns to Kafka engine
2020-06-01 12:04:17 +03:00
alexey-milovidov
9a362eb02e
Merge pull request #11252 from filimonov/kafka-clientid
...
Kafka clientid (finishing #11073 )
2020-05-31 18:37:30 +03:00
Mikhail Filimonov
b885337454
Virtual columns for accessing kafka message headers
2020-05-29 10:44:10 +02:00
Mikhail Filimonov
03430b1763
Change the test, accept new librdkafka behaviour
2020-05-28 23:11:29 +02:00
Mikhail Filimonov
e39c4cdf71
Add _timestamp_ms virtual columns to Kafka engine
2020-05-28 19:41:57 +02:00
Mikhail Filimonov
6b694d54b4
Better kafka client.id
2020-05-28 15:39:55 +02:00
filimonov
c73b8372e7
Update test.py
2020-05-27 07:44:13 +02:00
Mikhail Filimonov
efc0b97707
Fix Kafka performance issue related to reschedules based on limits, which were always applied.
2020-05-27 07:39:06 +02:00
filimonov
73a0ef6c0e
Fix for the hang during deletion of engine=Kafka (one more time) ( #11145 )
...
* Added flag for safer rdkafka destruction, but more testing detected another hang case, which has no straigt solutions and can be workarounded currenly only by draining the consumer queue, so destructor is back
* After review fixes
* After review fixes2
* After review fixes3
2020-05-26 00:13:10 +03:00
Mikhail Filimonov
b50480cbde
Fixes the potential missed data during termination of Kafka engine table
2020-05-19 18:55:37 +02:00
Mikhail Filimonov
791f3a6c90
Merge branch 'kafka_drop_hang_fix' of github.com:filimonov/ClickHouse into kafka_drop_hang_fix
2020-05-15 22:20:45 +02:00
Mikhail Filimonov
405cce2c53
Another variant of fix. It looks like we do not need destructor at all
2020-05-15 22:19:21 +02:00
filimonov
f97c14f7be
bit better comment
2020-05-14 12:18:17 +02:00
Mikhail Filimonov
6499d702f9
After review fixes
2020-05-14 12:15:38 +02:00
Mikhail Filimonov
a1013f1e71
Fix for the hang during deletion of engine=Kafka
2020-05-14 10:38:02 +02:00
Azat Khuzhin
7ab0c3493c
Add integration test for kafka consumer hang
...
This test triggers heartbeat failrues which leads to
REBALANCE_IN_PROGRESS which causes hang.
2020-05-04 18:46:30 +03:00
Azat Khuzhin
fc5f0bfe52
Enable librdkafka debugging in test_storage_kafka
2020-05-04 17:23:09 +03:00
Azat Khuzhin
12871bf26e
Drop resolved TODO from test_storage_kafka
2020-05-04 17:23:09 +03:00
Ivan
97f2a2213e
Move all folders inside /dbms one level up ( #9974 )
...
* Move some code outside dbms/src folder
* Fix paths
2020-04-02 02:51:21 +03:00
