Mikhail Filimonov
0d25208237
Prevent loosing data in kafka when execption happen after read suffix, but before commit. Fixes #9378
2020-03-04 07:58:44 +01:00
Mikhail Filimonov
babd3ca34e
Fixing test build, more review fixes
2020-02-05 08:05:55 +01:00
Mikhail Filimonov
cf35392433
Test for producer with _timestamp and _key
2020-02-04 23:12:26 +01:00
Mikhail Filimonov
a6e3265960
Kafka proper rebalances processing, controlled assignment, faster tests
2020-01-30 20:30:45 +01:00
Mikhail Filimonov
dc43728dea
Test showing the problem with partial commit of polled batch + fix
2020-01-28 15:21:36 +01:00
Mikhail Filimonov
5dafccf410
Test for flushing by time and by block size (+fix), one extra test for virual columns
2020-01-28 14:24:37 +01:00
Mikhail Filimonov
f3fa71c707
Init query context for Kafka to make subqueries work
2019-12-13 02:21:40 +01:00
Nikita Mikhaylov
2bd79325d7
test kafka limit
2019-10-28 17:41:24 +03:00
Nikita Mikhaylov
206800b8fc
add stderr configs to kafka test
2019-10-28 13:51:43 +03:00
Ivan
7ba4e36e73
Don't push to MVs when inserting into Kafka table ( #7265 )
...
* Do not insert values into MV when inserting directly to Kafka
* Add method noPushingToViews() to IStorage interface
To separate InterpreterInsertQuery and StorageKafka
2019-10-22 13:31:28 +03:00
Ivan Lezhankin
622ab139ee
Add test
2019-09-11 16:13:09 +03:00
Ivan
ab7df6b6dd
Fix kafka tests ( #6805 )
...
* Commit offsets more precisely
* Get rid of DelimitedReadBuffer since read buffers don't line up well
* Increase timeouts
2019-09-05 00:25:33 +03:00
Ivan
4d21f97d6a
Increase test timeout
2019-09-01 16:03:38 +03:00
Ivan
cd5c0fc9ac
Fix build issues ( #6744 )
...
* libcxxabi uses exception handling library as public
* Don't set -stdlib for internal libc++ - it poisons the checks.
* Enable capnproto in unbundled build back
2019-08-30 20:40:27 +03:00
Ivan
5851316742
Don't interrupt copyData() used inside Kafka materialized views ( #6597 )
...
* Print better offsets on commit
* Stop consumption on the read buffer level
* Don't use cancellation in the middle of copyData()
* Add test
* Disable squashing stream for Kafka materialized views
* Commit last read message, since rdkafka auto-modifies offset
2019-08-29 18:36:07 +03:00
Ivan
a502424c33
Implement support for insertion into Kafka tables ( #6012 )
...
* Add write-callback on each row for RowOutputStream
* Fix build of new rdkafka library
* Poll messages if Kafka outgoing queue is full
* Add test
* Add test producer-consumer
* Truncate delimiter from last row in message
2019-08-20 14:17:57 +03:00
Ivan Lezhankin
c52a34fd15
Add test for multiple mat. views
2019-08-15 17:19:12 +03:00
Ivan Lezhankin
e90bf74ce8
Merge branch 'master' into issue-5286
2019-08-08 12:57:00 +03:00
Ivan
1bf6e034e8
Fix infinite loop when reading Kafka messages ( #6354 )
...
* Do not pause/resume consumer at all
* Fix kafka tests
* Try to ensure the subscription
* Set timeout for kafka tests and return 'while True'
* Update cluster.py
* When doing a raw select from kafka, ignore client errors.
They may rise due to 'Local: Timed out' while subscribing.
2019-08-07 19:10:14 +03:00
Ivan Lezhankin
b55da1888a
Add "_timestamp" virtual column
2019-08-06 17:18:37 +03:00
Ivan Lezhankin
4bc2b32c3e
Merge branch 'master' into issue-5286
2019-08-06 16:25:51 +03:00
alesapin
363fca3895
More integration fixes and disable kafka test test_kafka_settings_new_syntax
2019-07-26 17:37:37 +03:00
alesapin
21ef261702
Fix some integration tests and make kafka tests endless
2019-07-25 23:11:26 +03:00
Ivan Lezhankin
b14356ba11
Try to debug failing kafka test.
2019-07-24 12:53:51 +03:00
Ivan Lezhankin
969cb2f826
Append _partition virtual column
2019-07-22 14:32:11 +03:00
alexey-milovidov
c43dfce041
Merge pull request #5382 from abyss7/issue-5286
...
Add virtual columns to Kafka Engine tables
2019-07-01 21:30:09 +03:00
Ivan Lezhankin
1a7cb519fe
Final test fix
2019-06-28 23:09:15 +03:00
alesapin
d3a5759c73
Disable tests and add global timeout
2019-06-25 16:20:04 +03:00
Ivan Lezhankin
d6d10120c8
Refactor tests
2019-06-23 17:48:58 +03:00
Ivan Lezhankin
f34e4b53ce
Fix tests again
2019-06-21 21:34:24 +03:00
Ivan Lezhankin
892a82e5ff
Add test on virtual columns and materialized view
2019-06-21 20:43:39 +03:00
Ivan Lezhankin
b989d45818
Fix tests
2019-06-21 20:02:48 +03:00
Ivan Lezhankin
9fd048cdbd
Allow to select virtual columns in materialized view
2019-06-18 19:32:37 +03:00
Ivan Lezhankin
1985caed8d
Add test on virtual columns
2019-05-23 17:32:06 +03:00
Ivan Lezhankin
d479836362
Add test on lost messages
2019-04-22 14:53:42 +03:00
Ivan Lezhankin
a4dfa0d58d
Fix new test
2019-04-03 20:46:54 +03:00
Ivan Lezhankin
a03f3d9f41
Use python bindings in Kafka tests.
2019-04-02 20:34:04 +03:00
Vitaly Baranov
ba58fbbbec
Add a kafka test using protobuf format.
2019-03-29 22:36:19 +03:00
Ivan Lezhankin
15ad409875
Tests are now passing locally
2019-02-11 14:54:30 +03:00
Ivan Lezhankin
c67ec3ddde
Fix running integration tests locally on Fedora (with selinux)
...
Finally!
2019-02-08 17:20:25 +03:00
Ivan Lezhankin
c9bc09ab7f
Rewrite tests one more time
2019-02-07 19:40:16 +03:00
Ivan Lezhankin
c5eff12015
Another attempt to fix test
2019-01-31 14:19:06 +03:00
Ivan Lezhankin
20b94a635c
Select the materialized view only once
2019-01-24 14:58:25 +03:00
Ivan Lezhankin
c50583126e
Fix tests
2019-01-24 14:58:25 +03:00
Ivan Lezhankin
425d483cfb
Add integration tests
2019-01-24 14:58:25 +03:00
alesapin
1f0f1ecad3
CLICKHOUSE-3894: Fix Kafka test
2018-08-27 19:15:39 +03:00
Alexander Marshalov
154d7e3228
Added SETTINGS clause for Kafka storage engine
2018-08-03 21:51:28 +03:00
Amos Bird
54a08ed018
Use docker exec to operate on kafka.
2018-07-27 01:24:35 +08:00
Alexey Zatelepin
02946d2bc1
Revert "Set up dns server for kafka integration tests."
...
This reverts commit 1c75c19106
2018-07-26 18:52:50 +03:00
Alexey Milovidov
ec202a7fd0
Allow to start cluster with Kafka without pytest #2725
2018-07-26 17:40:33 +03:00
