Alexey Milovidov
d1c268c2ed
Added setting "kafka_max_wait_ms"
2019-08-28 18:24:23 +03:00
Ivan
a502424c33
Implement support for insertion into Kafka tables ( #6012 )
...
* Add write-callback on each row for RowOutputStream
* Fix build of new rdkafka library
* Poll messages if Kafka outgoing queue is full
* Add test
* Add test producer-consumer
* Truncate delimiter from last row in message
2019-08-20 14:17:57 +03:00
Ivan Lezhankin
fd27ac9e9e
Merge remote-tracking branch 'upstream/master' into issue-5286
2019-08-11 00:11:44 +03:00
kreuzerkrieg
112fc71276
adding -Wshadow for GCC
2019-08-09 23:58:16 +03:00
Ivan Lezhankin
b55da1888a
Add "_timestamp" virtual column
2019-08-06 17:18:37 +03:00
Ivan Lezhankin
4bc2b32c3e
Merge branch 'master' into issue-5286
2019-08-06 16:25:51 +03:00
Ivan Lezhankin
969cb2f826
Append _partition virtual column
2019-07-22 14:32:11 +03:00
Nik
4a30eba32f
Fix style check
2019-07-16 22:34:47 +04:00
Nik
922c3eb22e
Clear Kafka's buffer if an invalid message is found.
2019-07-16 19:27:42 +04:00
Ivan Lezhankin
436052c67b
Merge branch 'master' into issue-5286
2019-06-18 16:03:29 +03:00
proller
fc7ae399c4
Use system rapidjson; Fix types for macos;
2019-06-14 13:28:30 +03:00
Ivan Lezhankin
1eccbc39c5
Don't add virtual column to empty block
2019-05-28 00:01:24 +03:00
Ivan Lezhankin
19a850ad75
Use read callback to populate virtual columns in Kafka Engine
2019-05-23 16:20:25 +03:00
Ivan Lezhankin
800854119e
Add buffer callback to fill in virtual columns
2019-05-23 14:15:18 +03:00
Ivan Lezhankin
dd906eabdc
[WIP] refactoring
2019-05-22 22:38:43 +03:00
Ivan
4a5832b18a
Check the time limit every (flush_interval / poll_timeout) number of rows from Kafka ( #5249 )
2019-05-14 18:52:03 +03:00
Ivan Lezhankin
94bba1918f
Do not drop Kafka consumer buffers after closing stream
2019-04-22 16:23:05 +03:00
proller
8227f5fe63
Arcadia fix
2019-03-12 15:06:17 +03:00
Ivan Lezhankin
15ad409875
Tests are now passing locally
2019-02-11 14:54:30 +03:00
Ivan Lezhankin
39e2563f93
Create new consumers on-demand
...
and give them some time for graceful assignment.
2019-01-30 20:58:49 +03:00
Ivan Lezhankin
681266abd0
Poll messages in batches of max_block_size
size.
2019-01-25 15:48:59 +03:00
Ivan Lezhankin
9636bdcd67
Fix build
2019-01-24 15:44:58 +03:00
Ivan Lezhankin
862d419f31
Fix losing the next message after a broken one.
2019-01-24 14:58:25 +03:00
Ivan Lezhankin
c8e605327d
Split StorageKafka.cpp on smaller files per class
2019-01-24 14:58:24 +03:00