thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-02 20:42:04 +00:00
Code Issues Packages Projects Releases Wiki Activity
32,371 Commits 9,183 Branches 1,782 Tags 2.2 GiB
9882c7c5ad
Commit Graph

369 Commits

Author SHA1 Message Date
Vitaly Baranov
65d997db94 Simplify excluding ProtobufWriter from compilation when USE_PROTOBUF=0. 2019-02-27 00:54:25 +03:00
Vitaly Baranov
ba2efbae01 Make the interface IRowOutputStream more generic and use it for writing protobufs. 2019-02-27 00:53:13 +03:00
Alexey Milovidov
ae28c46a46 Added workaround for compiler bug 2019-02-26 13:37:47 +03:00
Vitaly Baranov
dee7da9eb4
Merge branch 'master' into improve-performance-of-input-from-protobuf 2019-02-22 01:43:55 +03:00
proller
00e50f1502
Build fixes (#4471) 2019-02-21 23:11:21 +03:00
Nikolai Kochetov
25f6b97486 Added processors for Parquet and Protobuf. 2019-02-21 21:36:46 +03:00
Nikolai Kochetov
e91a0a31b7 Merge branch 'master' into processors 2019-02-21 19:19:35 +03:00
Nikolai Kochetov
aba56e171c Formats processorss. 2019-02-21 19:05:47 +03:00
Vitaly Baranov
4793ccb3fc Improve performance of ProtobufReader. 2019-02-21 18:49:42 +03:00
Vitaly Baranov
eef02034b2 Implement a new row input stream for reading protobuf messages. 2019-02-20 07:28:18 +03:00
Vitaly Baranov
630012dfb7 Fix error text about format schema. 2019-02-20 07:28:18 +03:00
Vitaly Baranov
9cd4c7495f Implement an utility class for reading protobuf messages. 2019-02-20 07:28:02 +03:00
Vitaly Baranov
74cc6f285c Implement an utility class to match columns and protobuf fields. 2019-02-20 05:32:29 +03:00
proller
e20c77e4c6 Add parquet support. Fixes, tests, ... 2019-02-19 23:51:44 +03:00
proller
263e69e861
Build fixes (FreeBSD port) (#4397) 2019-02-15 14:46:07 +03:00
Alexey Milovidov
c345ded839 Added comment #4200 2019-02-11 03:17:49 +03:00
alexey-milovidov
f58e0c7c97
Merge pull request #4200 from DarkWanderer/feature/row-binary-with-headers 
Added RowBinaryWithNamesAndTypes format
 2019-02-11 03:14:56 +03:00
Alexey Milovidov
58a6accee5 Updated test #4246 2019-02-10 19:55:12 +03:00
DarkWanderer
6e92183854 Fixed format export test, fixed clang compilation 2019-02-04 22:59:59 +03:00
Vasily Nemkov
2716df8ab2 IPv4 and IPv6 Domain implementation. 
Added:
 * IDataTypeDomain interface;
 * method DataTypeFactory::registerDataTypeDomain for registering domains;
 * DataTypeDomainWithSimpleSerialization domain base class with simple serialization/deserialization;
 * Concrete IPv4 and IPv6 domain implementations: DataTypeDomanIPv6 and DataTypeDomanIPv4;

Updated:
 * IDataType text serialization/deserialization methods;
 * IDataType implementation to use domain for text serialization/deserialization;
 * Refactored implementation of the IPv4/IPv6 functions to use formatIPv4/v6 and parseIPv4/v6 from Common/formatIPv6.h;

Tests:
 * Added test cases for IPv4 and IPv6 domains.
 * Updated IPv4/v6 functions tests to validate more cases;
 * Added performance tests for IPv4 and IPv6 related functions;
 2019-02-04 12:37:46 +03:00
DarkWanderer
e2fe15aafa Build fix: round 2 2019-01-31 22:30:36 +03:00
DarkWanderer
c8cff57a62 Fix compile errors, added output format test 2019-01-30 23:38:47 +03:00
DarkWanderer
bcac2722d1 Added RowBinaryWithNamesAndTypes format 
* Extended BinaryRowOutputStream with flags allowing to output names/types of columns
* Added one (for now) new output format - RowBinaryWithNamesAndTypes
* Updated docs
 2019-01-30 14:30:43 +03:00
Vitaly Baranov
e9721c9548 Add support for absolute format schema paths. 2019-01-27 12:15:32 +03:00
proller
5328251ff1
Fix build without protobuf, gtest, cppkafka (#4152) 2019-01-25 23:02:03 +03:00
Vitaly Baranov
ed1136e49e Use more IO/WriteHelpers.h functions in the class ProtobufSimpleWriter, 
rename variant=>varint.
 2019-01-25 17:15:44 +03:00
Alexey Milovidov
304c7b0386 Style #4005 2019-01-24 22:50:16 +03:00
alexey-milovidov
f96b8e4ed6
Merge pull request #4005 from vitlibar/add-protobuf-output-format 
Add protobuf output format
 2019-01-24 22:05:23 +03:00
Vitaly Baranov
2886f42c0e Implemented a new block output stream using protobuf output format. 2019-01-24 17:54:57 +03:00
Vitaly Baranov
c241a4ea6d Added function serializeProtobuf() to each data type. 2019-01-24 17:54:57 +03:00
Vitaly Baranov
1e7a231c56 Implemented main class for writing protobufs. 2019-01-24 17:54:52 +03:00
Vitaly Baranov
42d9d4e81d Implemented storage for parsed protobuf schemas. 2019-01-24 17:32:10 +03:00
Vitaly Baranov
28e9837819 Implemented utility class for writing protobufs. 2019-01-24 17:32:01 +03:00
Vitaly Baranov
ec37ef5a89 No escaping for the format schema file's name anymore, 
added an utility class for parsing and checking the format schema.
 2019-01-24 17:31:41 +03:00
Ivan Lezhankin
05ab15cc84 Merge IBlockInputStream and IProfilingBlockInputStream into one 2019-01-23 17:48:50 +03:00
Alexey Milovidov
c3aae60aed Trying to understand the code #4063 2019-01-17 21:49:56 +03:00
Alexey Milovidov
736e1e2d71 Fixed build #4063 2019-01-17 18:21:21 +03:00
alexey-milovidov
eeac860e0a
Update CapnProtoRowInputStream.cpp 2019-01-16 16:14:54 +03:00
alexey-milovidov
fb68929b10
Update CapnProtoRowInputStream.cpp 2019-01-16 16:14:34 +03:00
Odin Hultgren Van Der Horst
3ff868917e Changed tabs to spaces 2019-01-15 09:41:48 +01:00
Odin Hultgren Van Der Horst
3378bf58f5 Comments and changes to parent check 
- Added some comments.
 - Immplemented a recurisve parent check so fields with parents by
the same name but grandparents with diffrent names dont cause problems
 2019-01-15 09:27:38 +01:00
Odin Hultgren Van Der Horst
8bff8d4e2d Merge branch 'master' of https://github.com/yandex/ClickHouse 2019-01-15 08:13:51 +01:00
proller
72df7ceee6 Macos build fix 2019-01-14 18:20:09 +03:00
Odin Hultgren Van Der Horst
b844bdef15 Tabs are now spaces 2019-01-10 09:18:13 +01:00
Odin Hultgren Van Der Horst
23c93b8c79 Removed include statemnets 2019-01-10 08:55:59 +01:00
Odin Hultgren Van Der Horst
ffdf565528 Merge branch 'master' of https://github.com/Miniwoffer/ClickHouse 2019-01-09 15:27:11 +00:00
Odin Hultgren Van Der Horst
96352ce78d Added _ as delim 
Added _ as delim
This makes it so you do not have to use . when addresing structures
and since Capnproto can not use _ in its naming its fine to use.
 2019-01-09 15:24:17 +00:00
Odin Hultgren Van Der Horst
68ed4d5c78 Removed console logs used in debuging 2019-01-08 15:02:41 +01:00
Odin Hultgren Van Der Horst
3875265721 CapnProto should now load jagged structures 
Changed how actions are created for CapnProto so it supports jagged
structures.
 2019-01-08 14:57:52 +01:00
Alexey Milovidov
1c7ccbf108 Miscellaneous [#CLICKHOUSE-2] 2019-01-07 22:56:53 +03:00
Alexey Milovidov
0dfa553fca Added more warnings from clang's -Weverything, that are available in clang 8 2019-01-07 13:40:58 +03:00
Alexey Milovidov
1d4701b0bc Added some warnings from clang's -Weverything 2019-01-04 15:10:00 +03:00
Alexey Milovidov
af31ff24b9 Better code; added a test #3835 2018-12-14 23:21:47 +03:00
alexey-milovidov
b6feafea41
Update ODBCDriver2BlockOutputStream.cpp 2018-12-14 22:31:38 +03:00
Max
4c145a9535 #3835 Add totals row for WITH TOTALS query 2018-12-14 20:43:04 +03:00
chertus
60dc8fcac2 fixes for review CLICKHOUSE-3578 2018-12-06 20:20:17 +03:00
chertus
3db00b7b98 Merge branch 'master' into issue-3578 2018-12-05 14:43:14 +03:00
chertus
933c055104 CLICKHOUSE-3578 review proress 2018-12-04 17:09:47 +03:00
Alexey Milovidov
7d6fffff48 More style checks [#CLICKHOUSE-3] 2018-11-24 04:48:06 +03:00
alexey-milovidov
2c2d75cceb
Don't allow exceptions without code (#3645) 
* Don't allow exceptions without code #3633

* Don't allow exceptions without code #3633
 2018-11-23 00:19:58 +03:00
proller
1d40c2c871 Better link and deps (#3631) 
* try fix 1

* Better deps

* fix

* fix

* fix

* Update CMakeLists.txt

* Update CMakeLists.txt

* fix

* fix

* fix

* disable old

* Update find_odbc.cmake
 2018-11-21 22:20:27 +03:00
proller
2deb523265 Fix build with old capnp 2018-11-20 18:10:28 +03:00
chertus
13646eb4c4 move BlockMissingValues from Block to IBlockInputStream::getMissingValues() 2018-11-15 18:57:20 +03:00
chertus
ebf3d60186 column defaults without CapNProto [CLICKHOUSE-3578] 2018-11-14 21:20:16 +03:00
chertus
9d3325f56e one more fix for bad merge of branches 2018-11-13 16:36:53 +03:00
chertus
2fe3cdb3f5 restore lost commits [CLICKHOUSE-3578] 2018-11-12 17:52:30 +03:00
chertus
cc005c3ac9 Merge branch 'master' into issue-3578 2018-11-09 13:12:34 +03:00
alexey-milovidov
b1db9ec23c
Merge pull request #3436 from amosbird/master 
Flexible fields assignment
 2018-11-01 20:22:28 +03:00
alesapin
892ab4ce8a Better code for check on null values inside array IN expression 2018-10-31 12:17:47 +03:00
alesapin
444b6e1e43 Add validation of array data 2018-10-29 16:18:41 +03:00
Amos Bird
3cf2177268 Flexible fields assignment 
Field f;

before:

f = Int64(-42);

after:

f = -42;
 2018-10-24 08:47:33 +08:00
Alexey Milovidov
7b24ad570c Inline some functions #3223 2018-10-10 00:38:06 +03:00
Alexey Milovidov
c014168c5f Attempt to fix performance regression while parsing JSONEachRow #3223 2018-10-10 00:21:52 +03:00
Alexey Milovidov
a432ff335e Attempt to fix performance regression while parsing JSONEachRow #3223 2018-10-10 00:09:48 +03:00
Alexey Milovidov
4a31c1c18a Attempt to fix performance regression while parsing JSONEachRow #3223 2018-10-09 22:46:35 +03:00
Alexey Milovidov
9a50300e42 Minor modifications #3223 2018-10-09 21:14:30 +03:00
Alexey Milovidov
e4b4b5a4fc Better diagnostic for JSONEachRow format #3223 2018-10-09 21:06:52 +03:00
alexey-milovidov
052fc2b674
Merge pull request #3223 from veloman-yunkan/jsoneachrow_bugfix 
Fix for issue #3297
 2018-10-09 21:05:51 +03:00
Marek Vavruša
5d5c99ebee Formats/CapnProtoRowInputStream: fix column mismatch in list of structures 
The fields are lexicographically sorted to make traversal easier,
but their order must be preserved when collecting fields from structures.

For example, a list with a structure like `{b @0 :Text, a @1 :Text}` would
read `a` first despite being second, which would cause a mismatch.
 2018-10-04 15:59:27 -07:00
alexey-milovidov
147a2a13c2
Merge pull request #3257 from amosbird/master 
Correct wcwidth computation for pretty outputs.
 2018-10-02 16:23:41 +03:00
Amos Bird
7df73088ab Correct wcwidth computation for pretty outputs. 2018-10-02 10:06:18 +08:00
alexey-milovidov
ff53aa4947
Update CapnProtoRowInputStream.cpp 2018-10-01 04:33:36 +03:00
alexey-milovidov
fac040c39a
Update CapnProtoRowInputStream.cpp 2018-10-01 04:31:48 +03:00
Veloman Yunkan
3e96f32833 Fixed the bug introduced by PR#3144 
The pre-existing optimization in the function
JSONEachRowRowInputStream::readColumnName() (that was extracted
during the refactoring step of PR#3144) imposed a restriction
on its usage - reading from the input stream might invalidate the return
value of that function, and this is what happenned in readJSONObject()
after the call to skipColonDelimiter().

One way of fixing the problem while preserving the original optimization
intact would be to defer the call to skipColonDelimiter() until the
variable name_ref was fully consumed, however that would result in worse
code (skipColonDelimiter() would need to be called in three different
places where it doesn't really belong).

Therefore I preferred to slightly weaken the optimization by always
copying the key name into the current_column_name data member.
 2018-09-26 11:27:04 +00:00
Marek Vavruša
cbbcb6d9df Formats/CapnProtoRowInputStream: support Nested and Tuple, fix alignment issues 
This updated contrib/capnproto to a newer version that fixes problems with
unaligned access to message frames.

It also adds support for parsing Struct types as Tuple (named or unnamed),
and Nested array types.

The `struct X { a @0 :UInt64; b @1 :Text }` in Cap'nProto is equivalent to
`x Tuple(a UInt64, b String)` in ClickHouse.

Arrays of Struct types such as `y List(X)` are equivalent to `y Nested(a UInt64, b String)`.
 2018-09-25 22:51:35 -07:00
alesapin
b87a084f38 Strict order of evaluated and added columns 2018-09-20 14:16:15 +03:00
alesapin
e1c8045055 Fix error with header 2018-09-19 15:56:56 +03:00
alesapin
c2f3453fda Reorder Squashing and adding default output stream in insert 2018-09-19 14:06:42 +03:00
Alexey Milovidov
46ef387ce3 Style #3144 2018-09-17 23:35:21 +03:00
Veloman Yunkan
5fe127e4c3 Added --input_format_import_nested_json option 
By default mapping of nested json data to nested tables is disabled. To
enable the import of nested json data (into corresponding nested tables)
clickhouse must be run with the --input_format_import_nested_json=1
option.
 2018-09-14 13:43:57 +00:00
Veloman Yunkan
47eb0e28b3 Nested JSON data is mapped to nested table data 2018-09-14 12:15:32 +00:00
Veloman Yunkan
8d427dd09d Extracted JSONEachRowRowInputStream::readJSONObject 
Also changed the names of previously extracted functions to camel case.
 2018-09-14 10:47:26 +00:00
Veloman Yunkan
4cf2cd9586 Some more refactoring 2018-09-14 08:27:49 +00:00
Veloman Yunkan
270f13fe99 Some refactoring 2018-09-14 07:28:49 +00:00
chertus
0b35cdae69 refactoring: add some sugar in type checks [CLICKHOUSE-0002] 2018-09-07 17:37:26 +03:00
Alexey Milovidov
0c6c1e83a3 Avoid excessive padding in Pretty formats [#CLICKHOUSE-2] 2018-08-31 02:34:12 +03:00
BlahGeek
739119d355 skip object fields on json input 2018-08-27 12:01:47 +08:00
Alexey Milovidov
d8ca4345b6 Fixed style violations [#CLICKHOUSE-2] 2018-08-10 04:41:54 +03:00
proller
973bdab77f Format ODBCDriver2 with NULL support (#2834) 
* Format ODBCDriver2 with NULL support

* Fix comment

* Update ODBCDriver2BlockOutputStream.cpp

* clean
 2018-08-10 04:20:10 +03:00
Alex Bocharov
e8d4654419 Remove FormatSettings default value for writeJSONString and add tests for output_format_json_escape_forward_slashes. 2018-08-08 02:35:32 +03:00
alexey-milovidov
bc16e8ca69 Update FormatSettings.h 2018-08-08 02:35:32 +03:00
alexey-milovidov
31cff11fee Update FormatFactory.cpp 2018-08-08 02:35:32 +03:00
Alex Bocharov
a8818bb8af Add output_format_json_escape_slashes setting - controls escaping slashes for string outputs in JSON output format. 2018-08-08 02:35:32 +03:00
Alexey Milovidov
a106c65dec Miscellaneous #2702 2018-07-24 21:50:28 +03:00
Amos Bird
77ce910077 Fix invalid Context reference. 2018-07-24 21:47:47 +03:00
Alexey Milovidov
09d22da9b9 Miscellaneous #2687 2018-07-20 18:59:11 +03:00
alesapin
2915a58c8f Merge remote-tracking branch 'upstream/master' 2018-07-20 13:13:50 +03:00
alesapin
ec156e9f2b CLICKHOUSE-3772: Add system tables formats, data_type_families, table_functions, aggregate_function_combinators 2018-07-20 13:00:56 +03:00
Alexey Milovidov
7f04f593ed Moved code to .cpp; log only when skipping of errors is enabled #2669 2018-07-18 22:02:40 +03:00
Amos Bird
8b2e68bfa0 Function try in destructor rethrows exceptions 
According to https://en.cppreference.com/w/cpp/language/function-try-block , function try block used in destructor will rethrow exceptions if there isn't a return statement specified.
 2018-07-18 16:29:12 +03:00
Leonardo Cecchi
6d8d59bd1b Log number of skipped rows during a bulk insert 
When in a bulk insert the rows are parsed server-side and there are
skipped rows, log the number of these rows at the TRACE level.
 2018-07-18 03:04:01 +03:00
chertus
07a782e637 fix wrong defaults at columns tail and some build fixes 2018-07-16 22:28:07 +03:00
chertus
c8f71cb539 Merge branch 'master' into issue-3578 2018-07-13 14:30:42 +03:00
chertus
82d22574d4 fix empty defaults mask issue and add some comments 2018-07-11 17:11:47 +03:00
Alexey Milovidov
84fd35da41 Adaptations #2574 2018-07-05 00:00:50 +03:00
chertus
88c4081a82 3578: defaults for input streams (in progress) 2018-07-04 20:02:47 +03:00
Amos Bird
900b046943 Add options to enable parsing CSV with single/double quote. 2018-06-29 15:34:55 +08:00
proller
e1ae5233df Add // Y_IGNORE tags 2018-06-21 18:24:36 +03:00
alexey-milovidov
fafecb3c25
Better modularity of formats. (#2492) 
* Formats: better modularity (development) #2447

* Formats: better modularity (development) #2447

* Formats: better modularity (development) #2447

* Formats: better modularity (development) #2447

* Formats: better modularity (development) #2447

* Formats: better modularity (development): removed very old tests #2447

* Formats: better modularity (development) #2447

* Formats: better modularity (development) #2447

* Formats: better modularity (development) #2447

* Formats: better modularity (development) #2447

* Formats: better modularity (development) #2447

* Formats: better modularity (development) #2447

* Formats: better modularity (development) #2447

* Formats: better modularity (development) #2447
 2018-06-10 22:22:49 +03:00