thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-10-01 22:20:50 +00:00
Code Issues Packages Projects Releases Wiki Activity
25,619 Commits 8,973 Branches 1,767 Tags 1.8 GiB
e3277c191b
Commit Graph

172 Commits

Author SHA1 Message Date
Vitaly Baranov
147a7a1458 Remove unused function ProtobufSchemas::getMessageTypeForColumns(). 2019-02-27 14:37:17 +03:00
Vitaly Baranov
e712a78cce Minor improvements of ProtobufReader. 2019-02-27 14:37:17 +03:00
Vitaly Baranov
85ab57f179 Implement support of nested messages in Protobuf output format. 2019-02-27 14:37:17 +03:00
Vitaly Baranov
4f46052f2e Optimize the class ProtobufWriter and prepare for implementing support of nested messages. 2019-02-27 14:37:14 +03:00
Vitaly Baranov
1cce1feac1 Move IConverter to header for allow inlining ProtobufWriter functions. 2019-02-27 00:54:25 +03:00
Vitaly Baranov
2f0276503a Move ProtobufSimpleWriter to the internals of ProtobufWriter. 2019-02-27 00:54:25 +03:00
Vitaly Baranov
65d997db94 Simplify excluding ProtobufWriter from compilation when USE_PROTOBUF=0. 2019-02-27 00:54:25 +03:00
Vitaly Baranov
ba2efbae01 Make the interface IRowOutputStream more generic and use it for writing protobufs. 2019-02-27 00:53:13 +03:00
Alexey Milovidov
ae28c46a46 Added workaround for compiler bug 2019-02-26 13:37:47 +03:00
Vitaly Baranov
dee7da9eb4
Merge branch 'master' into improve-performance-of-input-from-protobuf 2019-02-22 01:43:55 +03:00
proller
00e50f1502
Build fixes (#4471) 2019-02-21 23:11:21 +03:00
Vitaly Baranov
4793ccb3fc Improve performance of ProtobufReader. 2019-02-21 18:49:42 +03:00
Vitaly Baranov
eef02034b2 Implement a new row input stream for reading protobuf messages. 2019-02-20 07:28:18 +03:00
Vitaly Baranov
630012dfb7 Fix error text about format schema. 2019-02-20 07:28:18 +03:00
Vitaly Baranov
9cd4c7495f Implement an utility class for reading protobuf messages. 2019-02-20 07:28:02 +03:00
Vitaly Baranov
74cc6f285c Implement an utility class to match columns and protobuf fields. 2019-02-20 05:32:29 +03:00
proller
e20c77e4c6 Add parquet support. Fixes, tests, ... 2019-02-19 23:51:44 +03:00
proller
263e69e861
Build fixes (FreeBSD port) (#4397) 2019-02-15 14:46:07 +03:00
Alexey Milovidov
c345ded839 Added comment #4200 2019-02-11 03:17:49 +03:00
alexey-milovidov
f58e0c7c97
Merge pull request #4200 from DarkWanderer/feature/row-binary-with-headers 
Added RowBinaryWithNamesAndTypes format
 2019-02-11 03:14:56 +03:00
Alexey Milovidov
58a6accee5 Updated test #4246 2019-02-10 19:55:12 +03:00
DarkWanderer
6e92183854 Fixed format export test, fixed clang compilation 2019-02-04 22:59:59 +03:00
Vasily Nemkov
2716df8ab2 IPv4 and IPv6 Domain implementation. 
Added:
 * IDataTypeDomain interface;
 * method DataTypeFactory::registerDataTypeDomain for registering domains;
 * DataTypeDomainWithSimpleSerialization domain base class with simple serialization/deserialization;
 * Concrete IPv4 and IPv6 domain implementations: DataTypeDomanIPv6 and DataTypeDomanIPv4;

Updated:
 * IDataType text serialization/deserialization methods;
 * IDataType implementation to use domain for text serialization/deserialization;
 * Refactored implementation of the IPv4/IPv6 functions to use formatIPv4/v6 and parseIPv4/v6 from Common/formatIPv6.h;

Tests:
 * Added test cases for IPv4 and IPv6 domains.
 * Updated IPv4/v6 functions tests to validate more cases;
 * Added performance tests for IPv4 and IPv6 related functions;
 2019-02-04 12:37:46 +03:00
DarkWanderer
e2fe15aafa Build fix: round 2 2019-01-31 22:30:36 +03:00
DarkWanderer
c8cff57a62 Fix compile errors, added output format test 2019-01-30 23:38:47 +03:00
DarkWanderer
bcac2722d1 Added RowBinaryWithNamesAndTypes format 
* Extended BinaryRowOutputStream with flags allowing to output names/types of columns
* Added one (for now) new output format - RowBinaryWithNamesAndTypes
* Updated docs
 2019-01-30 14:30:43 +03:00
Vitaly Baranov
e9721c9548 Add support for absolute format schema paths. 2019-01-27 12:15:32 +03:00
proller
5328251ff1
Fix build without protobuf, gtest, cppkafka (#4152) 2019-01-25 23:02:03 +03:00
Vitaly Baranov
ed1136e49e Use more IO/WriteHelpers.h functions in the class ProtobufSimpleWriter, 
rename variant=>varint.
 2019-01-25 17:15:44 +03:00
Alexey Milovidov
304c7b0386 Style #4005 2019-01-24 22:50:16 +03:00
alexey-milovidov
f96b8e4ed6
Merge pull request #4005 from vitlibar/add-protobuf-output-format 
Add protobuf output format
 2019-01-24 22:05:23 +03:00
Vitaly Baranov
2886f42c0e Implemented a new block output stream using protobuf output format. 2019-01-24 17:54:57 +03:00
Vitaly Baranov
c241a4ea6d Added function serializeProtobuf() to each data type. 2019-01-24 17:54:57 +03:00
Vitaly Baranov
1e7a231c56 Implemented main class for writing protobufs. 2019-01-24 17:54:52 +03:00
Vitaly Baranov
42d9d4e81d Implemented storage for parsed protobuf schemas. 2019-01-24 17:32:10 +03:00
Vitaly Baranov
28e9837819 Implemented utility class for writing protobufs. 2019-01-24 17:32:01 +03:00
Vitaly Baranov
ec37ef5a89 No escaping for the format schema file's name anymore, 
added an utility class for parsing and checking the format schema.
 2019-01-24 17:31:41 +03:00
Ivan Lezhankin
05ab15cc84 Merge IBlockInputStream and IProfilingBlockInputStream into one 2019-01-23 17:48:50 +03:00
Alexey Milovidov
c3aae60aed Trying to understand the code #4063 2019-01-17 21:49:56 +03:00
Alexey Milovidov
736e1e2d71 Fixed build #4063 2019-01-17 18:21:21 +03:00
alexey-milovidov
eeac860e0a
Update CapnProtoRowInputStream.cpp 2019-01-16 16:14:54 +03:00
alexey-milovidov
fb68929b10
Update CapnProtoRowInputStream.cpp 2019-01-16 16:14:34 +03:00
Odin Hultgren Van Der Horst
3ff868917e Changed tabs to spaces 2019-01-15 09:41:48 +01:00
Odin Hultgren Van Der Horst
3378bf58f5 Comments and changes to parent check 
- Added some comments.
 - Immplemented a recurisve parent check so fields with parents by
the same name but grandparents with diffrent names dont cause problems
 2019-01-15 09:27:38 +01:00
Odin Hultgren Van Der Horst
8bff8d4e2d Merge branch 'master' of https://github.com/yandex/ClickHouse 2019-01-15 08:13:51 +01:00
proller
72df7ceee6 Macos build fix 2019-01-14 18:20:09 +03:00
Odin Hultgren Van Der Horst
b844bdef15 Tabs are now spaces 2019-01-10 09:18:13 +01:00
Odin Hultgren Van Der Horst
23c93b8c79 Removed include statemnets 2019-01-10 08:55:59 +01:00
Odin Hultgren Van Der Horst
ffdf565528 Merge branch 'master' of https://github.com/Miniwoffer/ClickHouse 2019-01-09 15:27:11 +00:00
Odin Hultgren Van Der Horst
96352ce78d Added _ as delim 
Added _ as delim
This makes it so you do not have to use . when addresing structures
and since Capnproto can not use _ in its naming its fine to use.
 2019-01-09 15:24:17 +00:00
Odin Hultgren Van Der Horst
68ed4d5c78 Removed console logs used in debuging 2019-01-08 15:02:41 +01:00
Odin Hultgren Van Der Horst
3875265721 CapnProto should now load jagged structures 
Changed how actions are created for CapnProto so it supports jagged
structures.
 2019-01-08 14:57:52 +01:00
Alexey Milovidov
1c7ccbf108 Miscellaneous [#CLICKHOUSE-2] 2019-01-07 22:56:53 +03:00
Alexey Milovidov
0dfa553fca Added more warnings from clang's -Weverything, that are available in clang 8 2019-01-07 13:40:58 +03:00
Alexey Milovidov
1d4701b0bc Added some warnings from clang's -Weverything 2019-01-04 15:10:00 +03:00
Alexey Milovidov
af31ff24b9 Better code; added a test #3835 2018-12-14 23:21:47 +03:00
alexey-milovidov
b6feafea41
Update ODBCDriver2BlockOutputStream.cpp 2018-12-14 22:31:38 +03:00
Max
4c145a9535 #3835 Add totals row for WITH TOTALS query 2018-12-14 20:43:04 +03:00
chertus
60dc8fcac2 fixes for review CLICKHOUSE-3578 2018-12-06 20:20:17 +03:00
chertus
3db00b7b98 Merge branch 'master' into issue-3578 2018-12-05 14:43:14 +03:00
chertus
933c055104 CLICKHOUSE-3578 review proress 2018-12-04 17:09:47 +03:00
Alexey Milovidov
7d6fffff48 More style checks [#CLICKHOUSE-3] 2018-11-24 04:48:06 +03:00
alexey-milovidov
2c2d75cceb
Don't allow exceptions without code (#3645) 
* Don't allow exceptions without code #3633

* Don't allow exceptions without code #3633
 2018-11-23 00:19:58 +03:00
proller
1d40c2c871 Better link and deps (#3631) 
* try fix 1

* Better deps

* fix

* fix

* fix

* Update CMakeLists.txt

* Update CMakeLists.txt

* fix

* fix

* fix

* disable old

* Update find_odbc.cmake
 2018-11-21 22:20:27 +03:00
proller
2deb523265 Fix build with old capnp 2018-11-20 18:10:28 +03:00
chertus
13646eb4c4 move BlockMissingValues from Block to IBlockInputStream::getMissingValues() 2018-11-15 18:57:20 +03:00
chertus
ebf3d60186 column defaults without CapNProto [CLICKHOUSE-3578] 2018-11-14 21:20:16 +03:00
chertus
9d3325f56e one more fix for bad merge of branches 2018-11-13 16:36:53 +03:00
chertus
2fe3cdb3f5 restore lost commits [CLICKHOUSE-3578] 2018-11-12 17:52:30 +03:00
chertus
cc005c3ac9 Merge branch 'master' into issue-3578 2018-11-09 13:12:34 +03:00
alexey-milovidov
b1db9ec23c
Merge pull request #3436 from amosbird/master 
Flexible fields assignment
 2018-11-01 20:22:28 +03:00
alesapin
892ab4ce8a Better code for check on null values inside array IN expression 2018-10-31 12:17:47 +03:00
alesapin
444b6e1e43 Add validation of array data 2018-10-29 16:18:41 +03:00
Amos Bird
3cf2177268 Flexible fields assignment 
Field f;

before:

f = Int64(-42);

after:

f = -42;
 2018-10-24 08:47:33 +08:00
Alexey Milovidov
7b24ad570c Inline some functions #3223 2018-10-10 00:38:06 +03:00
Alexey Milovidov
c014168c5f Attempt to fix performance regression while parsing JSONEachRow #3223 2018-10-10 00:21:52 +03:00
Alexey Milovidov
a432ff335e Attempt to fix performance regression while parsing JSONEachRow #3223 2018-10-10 00:09:48 +03:00
Alexey Milovidov
4a31c1c18a Attempt to fix performance regression while parsing JSONEachRow #3223 2018-10-09 22:46:35 +03:00
Alexey Milovidov
9a50300e42 Minor modifications #3223 2018-10-09 21:14:30 +03:00
Alexey Milovidov
e4b4b5a4fc Better diagnostic for JSONEachRow format #3223 2018-10-09 21:06:52 +03:00
alexey-milovidov
052fc2b674
Merge pull request #3223 from veloman-yunkan/jsoneachrow_bugfix 
Fix for issue #3297
 2018-10-09 21:05:51 +03:00
Marek Vavruša
5d5c99ebee Formats/CapnProtoRowInputStream: fix column mismatch in list of structures 
The fields are lexicographically sorted to make traversal easier,
but their order must be preserved when collecting fields from structures.

For example, a list with a structure like `{b @0 :Text, a @1 :Text}` would
read `a` first despite being second, which would cause a mismatch.
 2018-10-04 15:59:27 -07:00
alexey-milovidov
147a2a13c2
Merge pull request #3257 from amosbird/master 
Correct wcwidth computation for pretty outputs.
 2018-10-02 16:23:41 +03:00
Amos Bird
7df73088ab Correct wcwidth computation for pretty outputs. 2018-10-02 10:06:18 +08:00
alexey-milovidov
ff53aa4947
Update CapnProtoRowInputStream.cpp 2018-10-01 04:33:36 +03:00
alexey-milovidov
fac040c39a
Update CapnProtoRowInputStream.cpp 2018-10-01 04:31:48 +03:00
Veloman Yunkan
3e96f32833 Fixed the bug introduced by PR#3144 
The pre-existing optimization in the function
JSONEachRowRowInputStream::readColumnName() (that was extracted
during the refactoring step of PR#3144) imposed a restriction
on its usage - reading from the input stream might invalidate the return
value of that function, and this is what happenned in readJSONObject()
after the call to skipColonDelimiter().

One way of fixing the problem while preserving the original optimization
intact would be to defer the call to skipColonDelimiter() until the
variable name_ref was fully consumed, however that would result in worse
code (skipColonDelimiter() would need to be called in three different
places where it doesn't really belong).

Therefore I preferred to slightly weaken the optimization by always
copying the key name into the current_column_name data member.
 2018-09-26 11:27:04 +00:00
Marek Vavruša
cbbcb6d9df Formats/CapnProtoRowInputStream: support Nested and Tuple, fix alignment issues 
This updated contrib/capnproto to a newer version that fixes problems with
unaligned access to message frames.

It also adds support for parsing Struct types as Tuple (named or unnamed),
and Nested array types.

The `struct X { a @0 :UInt64; b @1 :Text }` in Cap'nProto is equivalent to
`x Tuple(a UInt64, b String)` in ClickHouse.

Arrays of Struct types such as `y List(X)` are equivalent to `y Nested(a UInt64, b String)`.
 2018-09-25 22:51:35 -07:00
alesapin
b87a084f38 Strict order of evaluated and added columns 2018-09-20 14:16:15 +03:00
alesapin
e1c8045055 Fix error with header 2018-09-19 15:56:56 +03:00
alesapin
c2f3453fda Reorder Squashing and adding default output stream in insert 2018-09-19 14:06:42 +03:00
Alexey Milovidov
46ef387ce3 Style #3144 2018-09-17 23:35:21 +03:00
Veloman Yunkan
5fe127e4c3 Added --input_format_import_nested_json option 
By default mapping of nested json data to nested tables is disabled. To
enable the import of nested json data (into corresponding nested tables)
clickhouse must be run with the --input_format_import_nested_json=1
option.
 2018-09-14 13:43:57 +00:00
Veloman Yunkan
47eb0e28b3 Nested JSON data is mapped to nested table data 2018-09-14 12:15:32 +00:00
Veloman Yunkan
8d427dd09d Extracted JSONEachRowRowInputStream::readJSONObject 
Also changed the names of previously extracted functions to camel case.
 2018-09-14 10:47:26 +00:00
Veloman Yunkan
4cf2cd9586 Some more refactoring 2018-09-14 08:27:49 +00:00
Veloman Yunkan
270f13fe99 Some refactoring 2018-09-14 07:28:49 +00:00
chertus
0b35cdae69 refactoring: add some sugar in type checks [CLICKHOUSE-0002] 2018-09-07 17:37:26 +03:00
Alexey Milovidov
0c6c1e83a3 Avoid excessive padding in Pretty formats [#CLICKHOUSE-2] 2018-08-31 02:34:12 +03:00
BlahGeek
739119d355 skip object fields on json input 2018-08-27 12:01:47 +08:00