alesapin
13e4581317
More general
2019-08-07 18:21:45 +03:00
Ivan Lezhankin
0917573493
Merge remote-tracking branch 'upstream/master' into issue-5286
2019-06-24 14:42:58 +03:00
Ivan Lezhankin
61750d09d2
Commit offsets for SELECTing from Kafka table too
2019-06-20 15:34:25 +03:00
Ivan Lezhankin
fdd3f07860
Don't do premature subscription to topics
2019-06-19 19:15:30 +03:00
Ivan Lezhankin
9fd048cdbd
Allow to select virtual columns in materialized view
2019-06-18 19:32:37 +03:00
Ivan Lezhankin
436052c67b
Merge branch 'master' into issue-5286
2019-06-18 16:03:29 +03:00
Ivan Lezhankin
d907d8e8ca
Commit offsets for SELECTing from Kafka table too
2019-06-17 19:27:18 +03:00
Alexey Milovidov
c846d0f6af
Namespace comments are unneeded according to the code style.
...
find dbms -name '*.h' -or -name '*.cpp' | xargs grep -l -P '}\s*//+\s*namespace\s*' | xargs sed -i -r -e 's/}\s*\/\/+\s*namespace\s*.*$/}/'
2019-06-13 13:43:37 +03:00
Ivan Lezhankin
97558c1b41
Merge remote-tracking branch 'upstream/master' into issue-5286
2019-06-10 14:15:14 +03:00
Ivan
f70eed78d8
Don't do intermediate commits in Kafka tables by default ( #5445 )
...
* Don't do intermediate commits in Kafka tables by default
* Use default settings' values as defaults in code
2019-06-03 19:40:22 +03:00
Ivan Lezhankin
b311984879
Merge remote-tracking branch 'upstream/master' into issue-5286
2019-06-03 17:19:09 +03:00
alexey-milovidov
81ed764d6a
Merge pull request #5285 from abyss7/issue-5085
...
Don't crash the server when Kafka consumers have failed to start
2019-05-25 16:09:26 +03:00
Ivan Lezhankin
dd906eabdc
[WIP] refactoring
2019-05-22 22:38:43 +03:00
Ivan Lezhankin
cf39c4cc47
Embed virtual columns into IStorage
2019-05-21 14:24:32 +03:00
Ivan Lezhankin
a14013dff7
Add setting to regulate Kafka committing policy.
2019-05-16 18:20:30 +03:00
alexey-milovidov
8e2bdde93b
Update StorageKafka.cpp
2019-05-15 23:57:24 +03:00
Ivan Lezhankin
337ddf5047
Catch cppkafka exceptions about consumer creation failures
2019-05-15 17:23:52 +03:00
Ivan
4a5832b18a
Check the time limit every (flush_interval / poll_timeout) number of rows from Kafka ( #5249 )
2019-05-14 18:52:03 +03:00
Ivan Lezhankin
94bba1918f
Do not drop Kafka consumer buffers after closing stream
2019-04-22 16:23:05 +03:00
Ivan Lezhankin
4f6a494af8
Revert references where they were before refactoring.
2019-03-15 19:14:13 +03:00
Ivan Lezhankin
d1ead22073
Rename As<…> → as<…>
2019-03-11 16:22:51 +03:00
Ivan Lezhankin
a7e838e328
Replace all other casts to method IAST::As<Derived>
...
Also remove some redundant methods like IsIdentifier() et. al.
2019-03-11 15:49:39 +03:00
Ivan Lezhankin
0e325c7a4a
Replace const typeid_casts with As<Derived>() method invocations
2019-03-08 12:48:20 +03:00
proller
4078a62573
Fix build without contrib/cppkafka submodule
2019-02-11 19:34:32 +03:00
Ivan Lezhankin
2bd14117f4
Merge remote-tracking branch 'upstream/master' into issue-2675
2019-02-11 13:57:40 +03:00
Alexey Milovidov
58a6accee5
Updated test #4246
2019-02-10 19:55:12 +03:00
Ivan Lezhankin
39e2563f93
Create new consumers on-demand
...
and give them some time for graceful assignment.
2019-01-30 20:58:49 +03:00
Ivan Lezhankin
681266abd0
Poll messages in batches of max_block_size size.
2019-01-25 15:48:59 +03:00
Ivan Lezhankin
425d483cfb
Add integration tests
2019-01-24 14:58:25 +03:00
Ivan Lezhankin
c8e605327d
Split StorageKafka.cpp on smaller files per class
2019-01-24 14:58:24 +03:00
Ivan Lezhankin
05ab15cc84
Merge IBlockInputStream and IProfilingBlockInputStream into one
2019-01-23 17:48:50 +03:00
maiha
6c83beafdd
dbms: Fixed misspells in comments
2019-01-23 04:56:53 +09:00
Ivan Lezhankin
89ffb577b1
Implement additional Kafka engine setting for allowed parsing errors
2019-01-18 15:48:38 +03:00
Ivan Lezhankin
337c092c7e
Use cppkafka instead of raw C interface
2019-01-14 14:15:57 +03:00
Ivan Lezhankin
9c35598373
Add cppkafka to contrib
2019-01-14 14:15:57 +03:00
Alexey Milovidov
19f4652959
Fixed bugs found by PVS-Studio
2019-01-09 18:44:20 +03:00
Alexey Milovidov
1d4701b0bc
Added some warnings from clang's -Weverything
2019-01-04 15:10:00 +03:00
Alexey Milovidov
e33e5150b7
Miscellaneous [#CLICKHOUSE-2]
2019-01-02 09:44:36 +03:00
Alexey Milovidov
2c6e49c3f3
More checks [#CLICKHOUSE-2]
2018-12-21 19:24:47 +03:00
Alexey Milovidov
661a117b91
Added assertion [#CLICKHOUSE-2]
2018-12-21 19:03:40 +03:00
Alexey Milovidov
52fea90a86
Removed useless code [#CLICKHOUSE-2]
2018-11-28 17:33:40 +03:00
Ivan Lezhankin
74fe350451
Refactor partition commands handling into separate interface method.
2018-11-20 17:59:34 +03:00
alexey-milovidov
b1db9ec23c
Merge pull request #3436 from amosbird/master
...
Flexible fields assignment
2018-11-01 20:22:28 +03:00
chertus
d2518fdb3f
more refactoring: funcs to extract database_and_table from AST
2018-10-29 22:04:28 +03:00
Amos Bird
3cf2177268
Flexible fields assignment
...
Field f;
before:
f = Int64(-42);
after:
f = -42;
2018-10-24 08:47:33 +08:00
Marek Vavruša
37e03bd938
StorageKafka: check dependencies recursively, add max_block setting
...
Changes:
* Check table dependencies recursively before starting streaming
* Use actual number of created consumers when streaming
* Add size limits hint to Kafka consumer stream
* Configurable `kafka_max_block_size` per table
2018-10-18 09:20:34 -07:00
proller
ca03cd6606
Sync with arcadia
2018-10-01 18:43:48 +03:00
alexey-milovidov
bb0f76dd25
Update StorageKafka.cpp
2018-10-01 04:29:17 +03:00
alexey-milovidov
b9e6b02719
Update StorageKafka.cpp
2018-10-01 04:26:52 +03:00
Marek Vavruša
ea32434888
Storages/Kafka: fixed deadlocks, moved to background pool
...
This fixes two deadlocks in Kafka engine found previously:
* When exception is thrown before starting reading, consumer
was never returned to the storage. Now it is claimed only
when actually starting reading.
* Fixed lockup on deinitialization when consumer only unsubscribed,
but didn't close, and the endine then timeouted when waiting for
consumer destruction.
This also moves the stream thread to background worker pool.
The reason for that is that it will compete with other tasks for
time, so it will form a backpressure on insertion when the system
is busy.
2018-09-25 22:51:07 -07:00
