Commit Graph

43 Commits

Author SHA1 Message Date
Ivan
7ba4e36e73
Don't push to MVs when inserting into Kafka table (#7265)
* Do not insert values into MV when inserting directly to Kafka
* Add method noPushingToViews() to IStorage interface
  To separate InterpreterInsertQuery and StorageKafka
2019-10-22 13:31:28 +03:00
Ivan Lezhankin
622ab139ee Add test 2019-09-11 16:13:09 +03:00
Ivan
ab7df6b6dd
Fix kafka tests (#6805)
* Commit offsets more precisely
* Get rid of DelimitedReadBuffer since read buffers don't line up well
* Increase timeouts
2019-09-05 00:25:33 +03:00
Ivan
4d21f97d6a
Increase test timeout 2019-09-01 16:03:38 +03:00
Ivan
cd5c0fc9ac
Fix build issues (#6744)
* libcxxabi uses exception handling library as public
* Don't set -stdlib for internal libc++ - it poisons the checks.
* Enable capnproto in unbundled build back
2019-08-30 20:40:27 +03:00
Ivan
5851316742
Don't interrupt copyData() used inside Kafka materialized views (#6597)
* Print better offsets on commit
* Stop consumption on the read buffer level
* Don't use cancellation in the middle of copyData()
* Add test
* Disable squashing stream for Kafka materialized views
* Commit last read message, since rdkafka auto-modifies offset
2019-08-29 18:36:07 +03:00
Ivan
a502424c33
Implement support for insertion into Kafka tables (#6012)
* Add write-callback on each row for RowOutputStream
* Fix build of new rdkafka library
* Poll messages if Kafka outgoing queue is full
* Add test
* Add test producer-consumer
* Truncate delimiter from last row in message
2019-08-20 14:17:57 +03:00
Ivan Lezhankin
c52a34fd15 Add test for multiple mat. views 2019-08-15 17:19:12 +03:00
Ivan Lezhankin
e90bf74ce8 Merge branch 'master' into issue-5286 2019-08-08 12:57:00 +03:00
Ivan
1bf6e034e8
Fix infinite loop when reading Kafka messages (#6354)
* Do not pause/resume consumer at all
* Fix kafka tests
* Try to ensure the subscription
* Set timeout for kafka tests and return 'while True'
* Update cluster.py
* When doing a raw select from kafka, ignore client errors.
  They may rise due to 'Local: Timed out' while subscribing.
2019-08-07 19:10:14 +03:00
Ivan Lezhankin
b55da1888a Add "_timestamp" virtual column 2019-08-06 17:18:37 +03:00
Ivan Lezhankin
4bc2b32c3e Merge branch 'master' into issue-5286 2019-08-06 16:25:51 +03:00
alesapin
363fca3895 More integration fixes and disable kafka test test_kafka_settings_new_syntax 2019-07-26 17:37:37 +03:00
alesapin
21ef261702 Fix some integration tests and make kafka tests endless 2019-07-25 23:11:26 +03:00
Ivan Lezhankin
b14356ba11 Try to debug failing kafka test. 2019-07-24 12:53:51 +03:00
Ivan Lezhankin
969cb2f826 Append _partition virtual column 2019-07-22 14:32:11 +03:00
alexey-milovidov
c43dfce041
Merge pull request #5382 from abyss7/issue-5286
Add virtual columns to Kafka Engine tables
2019-07-01 21:30:09 +03:00
Ivan Lezhankin
1a7cb519fe Final test fix 2019-06-28 23:09:15 +03:00
alesapin
d3a5759c73 Disable tests and add global timeout 2019-06-25 16:20:04 +03:00
Ivan Lezhankin
d6d10120c8 Refactor tests 2019-06-23 17:48:58 +03:00
Ivan Lezhankin
f34e4b53ce Fix tests again 2019-06-21 21:34:24 +03:00
Ivan Lezhankin
892a82e5ff Add test on virtual columns and materialized view 2019-06-21 20:43:39 +03:00
Ivan Lezhankin
b989d45818 Fix tests 2019-06-21 20:02:48 +03:00
Ivan Lezhankin
9fd048cdbd Allow to select virtual columns in materialized view 2019-06-18 19:32:37 +03:00
Ivan Lezhankin
1985caed8d Add test on virtual columns 2019-05-23 17:32:06 +03:00
Ivan Lezhankin
d479836362 Add test on lost messages 2019-04-22 14:53:42 +03:00
Ivan Lezhankin
a4dfa0d58d Fix new test 2019-04-03 20:46:54 +03:00
Ivan Lezhankin
a03f3d9f41 Use python bindings in Kafka tests. 2019-04-02 20:34:04 +03:00
Vitaly Baranov
ba58fbbbec Add a kafka test using protobuf format. 2019-03-29 22:36:19 +03:00
Ivan Lezhankin
15ad409875 Tests are now passing locally 2019-02-11 14:54:30 +03:00
Ivan Lezhankin
c67ec3ddde Fix running integration tests locally on Fedora (with selinux)
Finally!
2019-02-08 17:20:25 +03:00
Ivan Lezhankin
c9bc09ab7f Rewrite tests one more time 2019-02-07 19:40:16 +03:00
Ivan Lezhankin
c5eff12015 Another attempt to fix test 2019-01-31 14:19:06 +03:00
Ivan Lezhankin
20b94a635c Select the materialized view only once 2019-01-24 14:58:25 +03:00
Ivan Lezhankin
c50583126e Fix tests 2019-01-24 14:58:25 +03:00
Ivan Lezhankin
425d483cfb Add integration tests 2019-01-24 14:58:25 +03:00
alesapin
1f0f1ecad3 CLICKHOUSE-3894: Fix Kafka test 2018-08-27 19:15:39 +03:00
Alexander Marshalov
154d7e3228 Added SETTINGS clause for Kafka storage engine 2018-08-03 21:51:28 +03:00
Amos Bird
54a08ed018 Use docker exec to operate on kafka. 2018-07-27 01:24:35 +08:00
Alexey Zatelepin
02946d2bc1 Revert "Set up dns server for kafka integration tests."
This reverts commit 1c75c19106.
2018-07-26 18:52:50 +03:00
Alexey Milovidov
ec202a7fd0 Allow to start cluster with Kafka without pytest #2725 2018-07-26 17:40:33 +03:00
Amos Bird
1c75c19106 Set up dns server for kafka integration tests. 2018-07-26 12:53:48 +08:00
Amos Bird
0965f991b3 Add row_delimiter argument to StorageKafka.
There are common cases where a message doesn't end with a row delimiter.
This patch allows specifying a row_delimiter char to compensate that.

https://github.com/yandex/ClickHouse/issues/2298
2018-07-25 18:54:01 +03:00