thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-09-29 21:20:49 +00:00
Code Issues Packages Projects Releases Wiki Activity
c6e959d1e9
ClickHouse/dbms/src/DataStreams
History
Marek Vavruša c6e959d1e9 StorageKafka: extended configuration, parallel consumers, offset tracking 
This contains many fixes and corrections for the Kafka engine.
Most notably it now supports extended configuration similarly to GraphiteMergeTree.
Now it also allows specification of consumer count to parallelize consumption of
multiple partitions both in materialized views and in SELECT queries.

The offsets are now committed in the insertSuffix() method after all rows
are successfully read. If an exception is thrown during reading, affected consumer
unsubscribes from all assignments and rejoins the consumer group to rewind offsets.
This means that the consumer won't lose messages in case of write failures.
2017-12-21 00:46:50 +03:00
..
tests Better semantic of sharing columns: development [#CLICKHOUSE-2]. 2017-12-16 09:10:05 +03:00
AddingConstColumnBlockInputStream.h Miscellaneous [#CLICKHOUSE-2]. 2017-12-11 01:44:04 +03:00
AddingDefaultBlockOutputStream.cpp Miscellaneous [#CLICKHOUSE-2]. 2017-12-18 07:07:26 +03:00
AddingDefaultBlockOutputStream.h Implemented some suggestions from clang-tidy (part 3) [#CLICKHOUSE-3301]. 2017-09-08 06:47:27 +03:00
AggregatingBlockInputStream.cpp fix handling of overflow data during external aggregation [#CLICKHOUSE-3133] 2017-07-25 19:56:48 +03:00
AggregatingBlockInputStream.h Implemented some suggestions from clang-tidy (part 1) [#CLICKHOUSE-3301]. 2017-09-08 05:29:47 +03:00
AggregatingSortedBlockInputStream.cpp Better semantic of sharing columns: development [#CLICKHOUSE-2]. 2017-12-15 03:06:56 +03:00
AggregatingSortedBlockInputStream.h Better semantic of sharing columns: development [#CLICKHOUSE-2]. 2017-12-15 03:06:56 +03:00
AsynchronousBlockInputStream.h Revert "Simplification [#CLICKHOUSE-2]." 2017-09-14 23:16:31 +03:00
BinaryRowInputStream.cpp Better semantic of sharing columns: development [#CLICKHOUSE-2]. 2017-12-14 23:58:18 +03:00
BinaryRowInputStream.h Better semantic of sharing columns: development [#CLICKHOUSE-2]. 2017-12-18 05:43:40 +03:00
BinaryRowOutputStream.cpp Moved headers and sources to same place [#CLICKHOUSE-3]. 2017-04-01 12:22:42 +03:00
BinaryRowOutputStream.h translate comments 2017-05-14 19:03:55 +04:00
BlockExtraInfoInputStream.h Implemented some suggestions from clang-tidy (part 1) [#CLICKHOUSE-3301]. 2017-09-08 05:29:47 +03:00
BlockInputStreamFromRowInputStream.cpp Better semantic of sharing columns: development [#CLICKHOUSE-2]. 2017-12-18 05:43:40 +03:00
BlockInputStreamFromRowInputStream.h Better semantic of sharing columns: development [#CLICKHOUSE-2]. 2017-12-18 05:43:40 +03:00
BlockIO.cpp Moved headers and sources to same place [#CLICKHOUSE-3]. 2017-04-01 12:22:42 +03:00
BlockIO.h Workable version for CREATE DROP w/o ZK tests. [#CLICKHOUSE-5] 2017-06-01 13:49:20 +03:00
BlockOutputStreamFromRowOutputStream.cpp Removed obsolete code [#CLICKHOUSE-3270]. 2017-09-01 21:21:01 +03:00
BlockOutputStreamFromRowOutputStream.h translate comments 2017-05-14 19:03:55 +04:00
BlocksListBlockInputStream.h translate comments 2017-05-14 19:03:55 +04:00
BlockStreamProfileInfo.cpp fix race condition while initializing BlockStreamProfileInfo [#CLICKHOUSE-3099] 2017-06-26 15:30:35 +03:00
BlockStreamProfileInfo.h fix race condition while initializing BlockStreamProfileInfo [#CLICKHOUSE-3099] 2017-06-26 15:30:35 +03:00
CapnProtoRowInputStream.cpp Better semantic of sharing columns: development [#CLICKHOUSE-2]. 2017-12-14 23:58:18 +03:00
CapnProtoRowInputStream.h StorageKafka: extended configuration, parallel consumers, offset tracking 2017-12-21 00:46:50 +03:00
CastTypeBlockInputStream.cpp Fixed error [#CLICKHOUSE-2] 2017-12-09 19:17:37 +03:00
CastTypeBlockInputStream.h Cleanup semantics of data types [#CLICKHOUSE-2]. 2017-12-09 19:17:37 +03:00
CMakeLists.txt Debian: simpler package (#662) 2017-04-10 20:43:30 +03:00
CollapsingFinalBlockInputStream.cpp removed typeid_cast from IColumn.h 2017-07-13 23:58:19 +03:00
CollapsingFinalBlockInputStream.h Better semantic of sharing columns: development [#CLICKHOUSE-2]. 2017-12-13 04:27:53 +03:00
CollapsingSortedBlockInputStream.cpp Better semantic of sharing columns: development [#CLICKHOUSE-2]. 2017-12-15 03:06:56 +03:00
CollapsingSortedBlockInputStream.h Better semantic of sharing columns: development [#CLICKHOUSE-2]. 2017-12-15 03:01:59 +03:00
ColumnGathererStream.cpp Make DataTypes totally immutable [#CLICKHOUSE-2]. 2017-12-18 04:11:48 +03:00
ColumnGathererStream.h Unification [#CLICKHOUSE-2]. 2017-09-15 15:16:12 +03:00
ConcatBlockInputStream.h translate comments 2017-05-14 19:03:55 +04:00
copyData.cpp Moved headers and sources to same place [#CLICKHOUSE-3]. 2017-04-01 12:22:42 +03:00
copyData.h translate comments 2017-05-14 19:03:55 +04:00
CountingBlockOutputStream.cpp Moved headers and sources to same place [#CLICKHOUSE-3]. 2017-04-01 12:22:42 +03:00
CountingBlockOutputStream.h Implemented some suggestions from clang-tidy (part 3) [#CLICKHOUSE-3301]. 2017-09-08 06:47:27 +03:00
CreatingSetsBlockInputStream.cpp Implemented some suggestions from clang-tidy (part 3) [#CLICKHOUSE-3301]. 2017-09-08 06:47:27 +03:00
CreatingSetsBlockInputStream.h Implemented some suggestions from clang-tidy (part 3) [#CLICKHOUSE-3301]. 2017-09-08 06:47:27 +03:00
CSVRowInputStream.cpp Better semantic of sharing columns: development [#CLICKHOUSE-2]. 2017-12-15 00:20:14 +03:00
CSVRowInputStream.h Better semantic of sharing columns: development [#CLICKHOUSE-2]. 2017-12-18 05:43:40 +03:00
CSVRowOutputStream.cpp Moved headers and sources to same place [#CLICKHOUSE-3]. 2017-04-01 12:22:42 +03:00
CSVRowOutputStream.h translate comments 2017-05-14 19:03:55 +04:00
DistinctBlockInputStream.cpp Better semantic of sharing columns: development [#CLICKHOUSE-2]. 2017-12-13 04:27:53 +03:00
DistinctBlockInputStream.h Better semantic of sharing columns: development [#CLICKHOUSE-2]. 2017-12-13 04:27:53 +03:00
DistinctSortedBlockInputStream.cpp Better semantic of sharing columns: development [#CLICKHOUSE-2]. 2017-12-13 04:27:53 +03:00
DistinctSortedBlockInputStream.h Better semantic of sharing columns: development [#CLICKHOUSE-2]. 2017-12-13 04:27:53 +03:00
EmptyBlockOutputStream.h translate comments 2017-05-14 19:03:55 +04:00
ExpressionBlockInputStream.cpp Implemented some suggestions from clang-tidy (part 3) [#CLICKHOUSE-3301]. 2017-09-08 06:47:27 +03:00
ExpressionBlockInputStream.h Implemented some suggestions from clang-tidy (part 3) [#CLICKHOUSE-3301]. 2017-09-08 06:47:27 +03:00
FilterBlockInputStream.cpp Better semantic of sharing columns: development [#CLICKHOUSE-2]. 2017-12-15 03:01:59 +03:00
FilterBlockInputStream.h Better semantic of sharing columns: development [#CLICKHOUSE-2]. 2017-12-15 03:01:59 +03:00
FilterColumnsBlockInputStream.cpp added throw_if_column_not_found to FilterColumnsBlockInputStream [#CLICKHOUSE-3111] 2017-07-11 20:58:49 +03:00
FilterColumnsBlockInputStream.h Implemented some suggestions from clang-tidy (part 3) [#CLICKHOUSE-3301]. 2017-09-08 06:47:27 +03:00
ForkBlockInputStreams.h Implemented some suggestions from clang-tidy (part 1) [#CLICKHOUSE-3301]. 2017-09-08 05:29:47 +03:00
FormatFactory.cpp Better semantic of sharing columns: development [#CLICKHOUSE-2]. 2017-12-15 03:01:59 +03:00
FormatFactory.h translate comments 2017-05-14 19:03:55 +04:00
glueBlockInputStreams.cpp Moved headers and sources to same place [#CLICKHOUSE-3]. 2017-04-01 12:22:42 +03:00
glueBlockInputStreams.h translate comments 2017-05-14 19:03:55 +04:00
GraphiteRollupSortedBlockInputStream.cpp Better semantic of sharing columns: development [#CLICKHOUSE-2]. 2017-12-15 03:06:56 +03:00
GraphiteRollupSortedBlockInputStream.h Better semantic of sharing columns: development [#CLICKHOUSE-2]. 2017-12-15 03:01:59 +03:00
IBlockInputStream.cpp Implemented some suggestions from clang-tidy (part 3) [#CLICKHOUSE-3301]. 2017-09-08 06:47:27 +03:00
IBlockInputStream.h use enum class 2017-10-03 22:12:44 +03:00
IBlockOutputStream.h Better [#CLICKHOUSE-2]. 2017-12-01 21:36:55 +03:00
InputStreamFromASTInsertQuery.cpp Moved headers and sources to same place [#CLICKHOUSE-3]. 2017-04-01 12:22:42 +03:00
InputStreamFromASTInsertQuery.h Miscellaneous [#CLICKHOUSE-2]. 2017-11-20 07:46:00 +03:00
IProfilingBlockInputStream.cpp Miscellaneous [#CLICKHOUSE-2]. 2017-12-18 07:07:26 +03:00
IProfilingBlockInputStream.h Core/FieldVisitors.h -> Common/FieldVisitors.h 2017-11-24 16:55:31 +03:00
IRowInputStream.cpp Moved headers and sources to same place [#CLICKHOUSE-3]. 2017-04-01 12:22:42 +03:00
IRowInputStream.h Better semantic of sharing columns: development [#CLICKHOUSE-2]. 2017-12-14 23:58:18 +03:00
IRowOutputStream.cpp Removed obsolete code [#CLICKHOUSE-3270]. 2017-09-01 21:21:01 +03:00
IRowOutputStream.h Better [#CLICKHOUSE-2]. 2017-12-01 22:34:51 +03:00
JSONCompactRowOutputStream.cpp updated FormatSettingsJSON 2017-07-09 20:16:26 +03:00
JSONCompactRowOutputStream.h Fixed build after merge [#CLICKHOUSE-3]. 2017-07-10 07:10:25 +03:00
JSONEachRowRowInputStream.cpp Better semantic of sharing columns: development [#CLICKHOUSE-2]. 2017-12-14 23:58:18 +03:00
JSONEachRowRowInputStream.h Better semantic of sharing columns: development [#CLICKHOUSE-2]. 2017-12-18 05:43:40 +03:00
JSONEachRowRowOutputStream.cpp updated FormatSettingsJSON 2017-07-09 20:16:26 +03:00
JSONEachRowRowOutputStream.h Fixed build after merge [#CLICKHOUSE-3]. 2017-07-10 07:10:39 +03:00
JSONRowOutputStream.cpp Better semantic of sharing columns: development [#CLICKHOUSE-2]. 2017-12-18 05:37:08 +03:00
JSONRowOutputStream.h Core/FieldVisitors.h -> Common/FieldVisitors.h 2017-11-24 16:55:31 +03:00
LazyBlockInputStream.h Added description of possible race condition [#CLICKHOUSE-2]. 2017-09-10 06:17:05 +03:00
LimitBlockInputStream.cpp Implemented some suggestions from clang-tidy (part 1) [#CLICKHOUSE-3301]. 2017-09-08 05:29:47 +03:00
LimitBlockInputStream.h Implemented some suggestions from clang-tidy (part 1) [#CLICKHOUSE-3301]. 2017-09-08 05:29:47 +03:00
LimitByBlockInputStream.cpp Better semantic of sharing columns: development [#CLICKHOUSE-2]. 2017-12-13 04:27:53 +03:00
LimitByBlockInputStream.h Better semantic of sharing columns: development [#CLICKHOUSE-2]. 2017-12-13 04:27:53 +03:00
MarkInCompressedFile.h translate comments 2017-05-14 19:03:55 +04:00
materializeBlock.cpp Cleanup semantics of data types [#CLICKHOUSE-2]. 2017-12-09 19:17:37 +03:00
materializeBlock.h Removed copy-paste [#CLICKHOUSE-2]. 2017-11-01 06:09:06 +03:00
MaterializingBlockInputStream.cpp Removed copy-paste [#CLICKHOUSE-2]. 2017-11-01 06:09:06 +03:00
MaterializingBlockInputStream.h Implemented some suggestions from clang-tidy (part 1) [#CLICKHOUSE-3301]. 2017-09-08 05:29:47 +03:00
MaterializingBlockOutputStream.h Removed copy-paste [#CLICKHOUSE-2]. 2017-11-01 06:09:06 +03:00
MergeSortingBlockInputStream.cpp Better semantic of sharing columns: development [#CLICKHOUSE-2]. 2017-12-17 11:44:06 +03:00
MergeSortingBlockInputStream.h Implemented some suggestions from clang-tidy (part 1) [#CLICKHOUSE-3301]. 2017-09-08 05:29:47 +03:00
MergingAggregatedBlockInputStream.cpp Moved headers and sources to same place [#CLICKHOUSE-3]. 2017-04-01 12:22:42 +03:00
MergingAggregatedBlockInputStream.h Implemented some suggestions from clang-tidy (part 1) [#CLICKHOUSE-3301]. 2017-09-08 05:29:47 +03:00
MergingAggregatedMemoryEfficientBlockInputStream.cpp Moved code to cpp [#CLICKHOUSE-2]. 2017-04-08 04:32:05 +03:00
MergingAggregatedMemoryEfficientBlockInputStream.h Implemented some suggestions from clang-tidy (part 3) [#CLICKHOUSE-3301]. 2017-09-08 06:47:27 +03:00
MergingSortedBlockInputStream.cpp Better semantic of sharing columns: development [#CLICKHOUSE-2]. 2017-12-15 03:06:56 +03:00
MergingSortedBlockInputStream.h Better semantic of sharing columns: development [#CLICKHOUSE-2]. 2017-12-15 03:06:56 +03:00
narrowBlockInputStreams.cpp Replaced Mersenne Twister to PCG-Random [#CLICKHOUSE-2]. 2017-09-10 02:17:38 +03:00
narrowBlockInputStreams.h translate comments 2017-05-14 19:03:55 +04:00
NativeBlockInputStream.cpp Better semantic of sharing columns: development [#CLICKHOUSE-2]. 2017-12-15 03:06:56 +03:00
NativeBlockInputStream.h Removed very old protocol compatibility features [#CLICKHOUSE-2]. 2017-08-16 23:27:35 +03:00
NativeBlockOutputStream.cpp Better semantic of sharing columns: development [#CLICKHOUSE-2]. 2017-12-15 00:51:30 +03:00
NativeBlockOutputStream.h Removed very old protocol compatibility features [#CLICKHOUSE-2]. 2017-08-16 23:27:35 +03:00
NullableAdapterBlockInputStream.cpp Better semantic of sharing columns: development [#CLICKHOUSE-2]. 2017-12-15 04:34:30 +03:00
NullableAdapterBlockInputStream.h Removed "experimental" namespace [#CLICKHOUSE-2]. 2017-11-20 07:15:43 +03:00
NullAndDoCopyBlockInputStream.h Implemented some suggestions from clang-tidy (part 1) [#CLICKHOUSE-3301]. 2017-09-08 05:29:47 +03:00
NullBlockInputStream.h LibDictionarySource : First lib dictionary version with only uint64 type support (#1150) 2017-09-05 04:08:26 +03:00
NullBlockOutputStream.h Better [#CLICKHOUSE-2]. 2017-12-01 22:34:51 +03:00
ODBCDriverBlockOutputStream.cpp Removed obsolete code [#CLICKHOUSE-3270]. 2017-09-01 21:21:01 +03:00
ODBCDriverBlockOutputStream.h translate comments 2017-05-14 19:03:55 +04:00
OneBlockInputStream.h LibDictionarySource : First lib dictionary version with only uint64 type support (#1150) 2017-09-05 04:08:26 +03:00
OwningBlockInputStream.h Executable dictionaries: fail if program returns non zero exit code (#CLICKHOUSE-3171) 2017-10-12 22:36:27 +03:00
ParallelAggregatingBlockInputStream.cpp Implemented some suggestions from clang-tidy (part 1) [#CLICKHOUSE-3301]. 2017-09-08 05:29:47 +03:00
ParallelAggregatingBlockInputStream.h Better semantic of sharing columns: development [#CLICKHOUSE-2]. 2017-12-13 04:27:53 +03:00
ParallelInputsProcessor.h Clarifications [#CLICKHOUSE-2]. 2017-12-04 23:29:13 +03:00
PartialSortingBlockInputStream.cpp Moved headers and sources to same place [#CLICKHOUSE-3]. 2017-04-01 12:22:42 +03:00
PartialSortingBlockInputStream.h Implemented some suggestions from clang-tidy (part 1) [#CLICKHOUSE-3301]. 2017-09-08 05:29:47 +03:00
PrettyBlockOutputStream.cpp Cleanup semantics of data types [#CLICKHOUSE-2]. 2017-12-09 19:17:37 +03:00
PrettyBlockOutputStream.h Removing dependency on functions from client: bug fixes [#CLICKHOUSE-2]. 2017-07-12 03:42:50 +03:00
PrettyCompactBlockOutputStream.cpp Cleanup semantics of data types [#CLICKHOUSE-2]. 2017-12-09 19:17:37 +03:00
PrettyCompactBlockOutputStream.h Better [#CLICKHOUSE-2]. 2017-12-01 22:34:51 +03:00
PrettySpaceBlockOutputStream.cpp Cleanup semantics of data types [#CLICKHOUSE-2]. 2017-12-09 19:17:37 +03:00
PrettySpaceBlockOutputStream.h translate comments 2017-05-14 19:03:55 +04:00
ProhibitColumnsBlockOutputStream.cpp Miscellaneous [#CLICKHOUSE-2]. 2017-07-11 04:20:40 +03:00
ProhibitColumnsBlockOutputStream.h Miscellaneous [#CLICKHOUSE-2]. 2017-07-11 04:20:40 +03:00
PushingToViewsBlockOutputStream.cpp merged with master 2017-12-01 16:33:47 +03:00
PushingToViewsBlockOutputStream.h merged with master 2017-12-01 16:33:47 +03:00
QueueBlockIOStream.h translate comments 2017-05-14 19:03:55 +04:00
RemoteBlockInputStream.cpp Fixed linking [#CLICKHOUSE-2]. 2017-11-20 08:22:54 +03:00
RemoteBlockInputStream.h Better [#CLICKHOUSE-2]. 2017-12-01 21:36:55 +03:00
RemoteBlockOutputStream.cpp Moved headers and sources to same place [#CLICKHOUSE-3]. 2017-04-01 12:22:42 +03:00
RemoteBlockOutputStream.h Moved headers and sources to same place [#CLICKHOUSE-3]. 2017-04-01 12:22:42 +03:00
RemoveColumnsBlockInputStream.h Miscellaneous [#CLICKHOUSE-2]. 2017-07-17 01:41:15 +03:00
ReplacingSortedBlockInputStream.cpp Better semantic of sharing columns: development [#CLICKHOUSE-2]. 2017-12-15 03:06:56 +03:00
ReplacingSortedBlockInputStream.h Better semantic of sharing columns: development [#CLICKHOUSE-2]. 2017-12-15 03:01:59 +03:00
SquashingBlockInputStream.cpp Moved headers and sources to same place [#CLICKHOUSE-3]. 2017-04-01 12:22:42 +03:00
SquashingBlockInputStream.h Moved headers and sources to same place [#CLICKHOUSE-3]. 2017-04-01 12:22:42 +03:00
SquashingBlockOutputStream.cpp Removing dependency on functions from client: bug fixes [#CLICKHOUSE-2]. 2017-07-12 03:42:50 +03:00
SquashingBlockOutputStream.h Removing dependency on functions from client: bug fixes [#CLICKHOUSE-2]. 2017-07-12 03:42:50 +03:00
SquashingTransform.cpp Better semantic of sharing columns: development [#CLICKHOUSE-2]. 2017-12-15 04:34:30 +03:00
SquashingTransform.h Moved headers and sources to same place [#CLICKHOUSE-3]. 2017-04-01 12:22:42 +03:00
SummingSortedBlockInputStream.cpp Better semantic of sharing columns: development [#CLICKHOUSE-2]. 2017-12-17 08:21:04 +03:00
SummingSortedBlockInputStream.h Better semantic of sharing columns: development [#CLICKHOUSE-2]. 2017-12-15 04:34:30 +03:00
TabSeparatedRawRowOutputStream.h translate comments 2017-05-14 19:03:55 +04:00
TabSeparatedRowInputStream.cpp Better semantic of sharing columns: development [#CLICKHOUSE-2]. 2017-12-15 04:34:30 +03:00
TabSeparatedRowInputStream.h Better semantic of sharing columns: development [#CLICKHOUSE-2]. 2017-12-18 05:43:40 +03:00
TabSeparatedRowOutputStream.cpp Moved headers and sources to same place [#CLICKHOUSE-3]. 2017-04-01 12:22:42 +03:00
TabSeparatedRowOutputStream.h translate comments 2017-05-14 19:03:55 +04:00
TotalsHavingBlockInputStream.cpp Better semantic of sharing columns: development [#CLICKHOUSE-2]. 2017-12-16 00:32:25 +03:00
TotalsHavingBlockInputStream.h Better semantic of sharing columns: development [#CLICKHOUSE-2]. 2017-12-15 05:04:28 +03:00
TSKVRowInputStream.cpp Better semantic of sharing columns: development [#CLICKHOUSE-2]. 2017-12-15 04:34:30 +03:00
TSKVRowInputStream.h Better semantic of sharing columns: development [#CLICKHOUSE-2]. 2017-12-18 05:43:40 +03:00
TSKVRowOutputStream.cpp Better semantic of sharing columns: development [#CLICKHOUSE-2]. 2017-12-18 05:37:08 +03:00
TSKVRowOutputStream.h translate comments 2017-05-14 19:03:55 +04:00
UnionBlockInputStream.h Better [#CLICKHOUSE-2]. 2017-12-02 06:42:21 +03:00
ValuesRowInputStream.cpp Make DataTypes totally immutable [#CLICKHOUSE-2]. 2017-12-18 04:11:48 +03:00
ValuesRowInputStream.h Better semantic of sharing columns: development [#CLICKHOUSE-2]. 2017-12-18 05:43:40 +03:00
ValuesRowOutputStream.cpp Moved headers and sources to same place [#CLICKHOUSE-3]. 2017-04-01 12:22:42 +03:00
ValuesRowOutputStream.h translate comments 2017-05-14 19:03:55 +04:00
verbosePrintString.cpp Attempt to improve performance [#CLICKHOUSE-3]. 2017-07-23 09:53:28 +03:00
verbosePrintString.h Small refinements [#CLICKHOUSE-2]. 2017-06-01 16:41:58 +03:00
VerticalRowOutputStream.cpp Cleanup semantics of data types [#CLICKHOUSE-2]. 2017-12-09 19:17:37 +03:00
VerticalRowOutputStream.h Better [#CLICKHOUSE-2]. 2017-12-01 23:21:35 +03:00
XMLRowOutputStream.cpp Better semantic of sharing columns: development [#CLICKHOUSE-2]. 2017-12-18 05:37:08 +03:00
XMLRowOutputStream.h Core/FieldVisitors.h -> Common/FieldVisitors.h 2017-11-24 16:55:31 +03:00