Commit Graph

203 Commits

Author SHA1 Message Date
Alexander Kazakov
c7a4b2ec75 Fixed style 2019-04-22 19:53:54 +03:00
Alexander Kazakov
bde1845bdc Better naming 2019-04-22 17:00:51 +03:00
Alexander Kazakov
d90bab5e33 In [T|C]SVWithNames formats start using column information from header 2019-04-22 16:31:17 +03:00
Alexey Milovidov
b0b369b30f Fixed typo #5010 2019-04-18 22:46:59 +03:00
Alexey Milovidov
642b09c8d6 Added clarifications #5010 2019-04-18 22:45:46 +03:00
Alexey Milovidov
f2cf2ffc80 Cleanups 2019-04-11 00:20:22 +03:00
Alexey Milovidov
36d016d331 Added [[noreturn]] attributes for functions that throw unconditionally 2019-04-10 23:27:14 +03:00
Nikolai Kochetov
ec65b4c229 Check num columns in Block::cloneWithColumns. 2019-04-05 14:39:07 +03:00
Nikolai Kochetov
7626b1b267 Multiple fixes. 2019-04-05 13:52:07 +03:00
Nikolai Kochetov
5e39ec610c Merged with master 2019-04-03 14:21:38 +03:00
Nikolai Kochetov
94ff7f694a QueryPipeline [in progress]. 2019-04-03 13:14:36 +03:00
Vitaly Baranov
37b904b8f5
Merge pull request #4808 from vitlibar/kafka-with-protobuf-format
Kafka with protobuf format
2019-03-30 19:00:57 +03:00
Vitaly Baranov
378c6f7a23 Improve the error message when format schema isn't set. 2019-03-29 22:36:10 +03:00
Vladislav Smirnov
7a02d320ff rewrite capnproto reading 2019-03-25 20:17:58 +03:00
Vladislav Smirnov
08b7c48286 Merge remote-tracking branch 'upstream/master' into fix-capnproto 2019-03-25 19:24:30 +03:00
proller
fce9db12b9 Move Interpreters/Settings* -> Core/Settings* 2019-03-22 15:08:30 +03:00
proller
faeca57d9a
Try fix Default timezone regression #4658 (#4718) 2019-03-20 16:29:20 +03:00
Vladislav Smirnov
59d473e215 style 2019-03-13 18:28:29 +03:00
Vladislav Smirnov
a916b62f33 ignore instead append 2019-03-13 14:18:18 +03:00
Vladislav Smirnov
7d822d8081 tabs -> spaces 2019-03-13 14:05:52 +03:00
Vladislav Smirnov
22f4852830 fix reading capnproto 2019-03-13 13:50:49 +03:00
Alexey Milovidov
9d89cc0fc1 Debugging PODArray 2019-03-08 06:23:02 +03:00
proller
482d153dcd clickhouse-odbc #157 : Remove LowCardinality from type (#4381)
* clickhouse-odbc #157 : Remove LowCardinality from type

* Update ODBCDriver2BlockOutputStream.cpp

* fix
2019-03-06 21:37:16 +03:00
proller
ffc9fbf769 Build fixes (#4600)
* Add Y_IGNORE

* Build fixes
2019-03-06 02:53:35 +03:00
Amos Bird
26ab5dd7a7 A Proper lookup table that uses HashTable's API
This is the first step of allowing heterogeneous cells in hash tables.

performance test results are

```

1. HashMap<UInt16, UInt8, TrivialHash, HashTableFixedGrower<16>>;
2. NewLookupMap<UInt16, UInt8>

ResolutionWidth 30000 1 .................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................223550276.46
ResolutionWidth 30000 2 .................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................248772721.24
Best: 2 - 24877272124

ResolutionWidth 100000 1 ..........................................................................................................................................................................................................................................................238498413.99
ResolutionWidth 100000 2 ..........................................................................................................................................................................................................................................................261808889.98
Best: 2 - 26180888998

ResolutionWidth 300000 1 ...................................................................................239307348.81
ResolutionWidth 300000 2 ...................................................................................257592761.30
Best: 2 - 25759276130

ResolutionWidth 1000000 1 .........................240144759.26
ResolutionWidth 1000000 2 .........................257093531.91
Best: 2 - 25709353191

ResolutionWidth 5000000 1 .....241573260.35
ResolutionWidth 5000000 2 .....259314162.79
Best: 2 - 25931416279

ResolutionDepth 30000 1 .................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................217108119.84
ResolutionDepth 30000 2 .................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................249459504.41
Best: 2 - 24945950441

ResolutionDepth 100000 1 ..........................................................................................................................................................................................................................................................229065162.17
ResolutionDepth 100000 2 ..........................................................................................................................................................................................................................................................253769105.64
Best: 2 - 25376910564

ResolutionDepth 300000 1 ...................................................................................233079225.18
ResolutionDepth 300000 2 ...................................................................................256316273.78
Best: 2 - 25631627378

ResolutionDepth 1000000 1 .........................234184633.51
ResolutionDepth 1000000 2 .........................261100491.57
Best: 2 - 26110049157

ResolutionDepth 5000000 1 .....233118795.66
ResolutionDepth 5000000 2 .....252436160.41
Best: 2 - 25243616041

```
2019-03-01 16:47:13 +08:00
proller
23d70b2e04
Add Y_IGNORE (#4521) 2019-02-27 19:52:08 +03:00
Vitaly Baranov
6fbcc12674
Merge pull request #4493 from vitlibar/support-for-nested-messages-in-protobuf-output
Added support for nested messages in protobuf output format
2019-02-27 18:16:45 +03:00
proller
211ec436c5
Build fixes (#4516) 2019-02-27 15:52:24 +03:00
Vitaly Baranov
147a7a1458 Remove unused function ProtobufSchemas::getMessageTypeForColumns(). 2019-02-27 14:37:17 +03:00
Vitaly Baranov
e712a78cce Minor improvements of ProtobufReader. 2019-02-27 14:37:17 +03:00
Vitaly Baranov
85ab57f179 Implement support of nested messages in Protobuf output format. 2019-02-27 14:37:17 +03:00
Vitaly Baranov
4f46052f2e Optimize the class ProtobufWriter and prepare for implementing support of nested messages. 2019-02-27 14:37:14 +03:00
Vitaly Baranov
1cce1feac1 Move IConverter to header for allow inlining ProtobufWriter functions. 2019-02-27 00:54:25 +03:00
Vitaly Baranov
2f0276503a Move ProtobufSimpleWriter to the internals of ProtobufWriter. 2019-02-27 00:54:25 +03:00
Vitaly Baranov
65d997db94 Simplify excluding ProtobufWriter from compilation when USE_PROTOBUF=0. 2019-02-27 00:54:25 +03:00
Vitaly Baranov
ba2efbae01 Make the interface IRowOutputStream more generic and use it for writing protobufs. 2019-02-27 00:53:13 +03:00
Alexey Milovidov
ae28c46a46 Added workaround for compiler bug 2019-02-26 13:37:47 +03:00
Vitaly Baranov
dee7da9eb4
Merge branch 'master' into improve-performance-of-input-from-protobuf 2019-02-22 01:43:55 +03:00
proller
00e50f1502
Build fixes (#4471) 2019-02-21 23:11:21 +03:00
Nikolai Kochetov
25f6b97486 Added processors for Parquet and Protobuf. 2019-02-21 21:36:46 +03:00
Nikolai Kochetov
e91a0a31b7 Merge branch 'master' into processors 2019-02-21 19:19:35 +03:00
Nikolai Kochetov
aba56e171c Formats processorss. 2019-02-21 19:05:47 +03:00
Vitaly Baranov
4793ccb3fc Improve performance of ProtobufReader. 2019-02-21 18:49:42 +03:00
Vitaly Baranov
eef02034b2 Implement a new row input stream for reading protobuf messages. 2019-02-20 07:28:18 +03:00
Vitaly Baranov
630012dfb7 Fix error text about format schema. 2019-02-20 07:28:18 +03:00
Vitaly Baranov
9cd4c7495f Implement an utility class for reading protobuf messages. 2019-02-20 07:28:02 +03:00
Vitaly Baranov
74cc6f285c Implement an utility class to match columns and protobuf fields. 2019-02-20 05:32:29 +03:00
proller
e20c77e4c6 Add parquet support. Fixes, tests, ... 2019-02-19 23:51:44 +03:00
proller
263e69e861
Build fixes (FreeBSD port) (#4397) 2019-02-15 14:46:07 +03:00
Alexey Milovidov
c345ded839 Added comment #4200 2019-02-11 03:17:49 +03:00
alexey-milovidov
f58e0c7c97
Merge pull request #4200 from DarkWanderer/feature/row-binary-with-headers
Added RowBinaryWithNamesAndTypes format
2019-02-11 03:14:56 +03:00
Alexey Milovidov
58a6accee5 Updated test #4246 2019-02-10 19:55:12 +03:00
DarkWanderer
6e92183854 Fixed format export test, fixed clang compilation 2019-02-04 22:59:59 +03:00
Vasily Nemkov
2716df8ab2 IPv4 and IPv6 Domain implementation.
Added:
 * IDataTypeDomain interface;
 * method DataTypeFactory::registerDataTypeDomain for registering domains;
 * DataTypeDomainWithSimpleSerialization domain base class with simple serialization/deserialization;
 * Concrete IPv4 and IPv6 domain implementations: DataTypeDomanIPv6 and DataTypeDomanIPv4;

Updated:
 * IDataType text serialization/deserialization methods;
 * IDataType implementation to use domain for text serialization/deserialization;
 * Refactored implementation of the IPv4/IPv6 functions to use formatIPv4/v6 and parseIPv4/v6 from Common/formatIPv6.h;

Tests:
 * Added test cases for IPv4 and IPv6 domains.
 * Updated IPv4/v6 functions tests to validate more cases;
 * Added performance tests for IPv4 and IPv6 related functions;
2019-02-04 12:37:46 +03:00
DarkWanderer
e2fe15aafa Build fix: round 2 2019-01-31 22:30:36 +03:00
DarkWanderer
c8cff57a62 Fix compile errors, added output format test 2019-01-30 23:38:47 +03:00
DarkWanderer
bcac2722d1 Added RowBinaryWithNamesAndTypes format
* Extended BinaryRowOutputStream with flags allowing to output names/types of columns
* Added one (for now) new output format - RowBinaryWithNamesAndTypes
* Updated docs
2019-01-30 14:30:43 +03:00
Vitaly Baranov
e9721c9548 Add support for absolute format schema paths. 2019-01-27 12:15:32 +03:00
proller
5328251ff1
Fix build without protobuf, gtest, cppkafka (#4152) 2019-01-25 23:02:03 +03:00
Vitaly Baranov
ed1136e49e Use more IO/WriteHelpers.h functions in the class ProtobufSimpleWriter,
rename variant=>varint.
2019-01-25 17:15:44 +03:00
Alexey Milovidov
304c7b0386 Style #4005 2019-01-24 22:50:16 +03:00
alexey-milovidov
f96b8e4ed6
Merge pull request #4005 from vitlibar/add-protobuf-output-format
Add protobuf output format
2019-01-24 22:05:23 +03:00
Vitaly Baranov
2886f42c0e Implemented a new block output stream using protobuf output format. 2019-01-24 17:54:57 +03:00
Vitaly Baranov
c241a4ea6d Added function serializeProtobuf() to each data type. 2019-01-24 17:54:57 +03:00
Vitaly Baranov
1e7a231c56 Implemented main class for writing protobufs. 2019-01-24 17:54:52 +03:00
Vitaly Baranov
42d9d4e81d Implemented storage for parsed protobuf schemas. 2019-01-24 17:32:10 +03:00
Vitaly Baranov
28e9837819 Implemented utility class for writing protobufs. 2019-01-24 17:32:01 +03:00
Vitaly Baranov
ec37ef5a89 No escaping for the format schema file's name anymore,
added an utility class for parsing and checking the format schema.
2019-01-24 17:31:41 +03:00
Ivan Lezhankin
05ab15cc84 Merge IBlockInputStream and IProfilingBlockInputStream into one 2019-01-23 17:48:50 +03:00
Alexey Milovidov
c3aae60aed Trying to understand the code #4063 2019-01-17 21:49:56 +03:00
Alexey Milovidov
736e1e2d71 Fixed build #4063 2019-01-17 18:21:21 +03:00
alexey-milovidov
eeac860e0a
Update CapnProtoRowInputStream.cpp 2019-01-16 16:14:54 +03:00
alexey-milovidov
fb68929b10
Update CapnProtoRowInputStream.cpp 2019-01-16 16:14:34 +03:00
Odin Hultgren Van Der Horst
3ff868917e Changed tabs to spaces 2019-01-15 09:41:48 +01:00
Odin Hultgren Van Der Horst
3378bf58f5 Comments and changes to parent check
- Added some comments.
 - Immplemented a recurisve parent check so fields with parents by
the same name but grandparents with diffrent names dont cause problems
2019-01-15 09:27:38 +01:00
Odin Hultgren Van Der Horst
8bff8d4e2d Merge branch 'master' of https://github.com/yandex/ClickHouse 2019-01-15 08:13:51 +01:00
proller
72df7ceee6 Macos build fix 2019-01-14 18:20:09 +03:00
Odin Hultgren Van Der Horst
b844bdef15 Tabs are now spaces 2019-01-10 09:18:13 +01:00
Odin Hultgren Van Der Horst
23c93b8c79 Removed include statemnets 2019-01-10 08:55:59 +01:00
Odin Hultgren Van Der Horst
ffdf565528 Merge branch 'master' of https://github.com/Miniwoffer/ClickHouse 2019-01-09 15:27:11 +00:00
Odin Hultgren Van Der Horst
96352ce78d Added _ as delim
Added _ as delim
This makes it so you do not have to use . when addresing structures
and since Capnproto can not use _ in its naming its fine to use.
2019-01-09 15:24:17 +00:00
Odin Hultgren Van Der Horst
68ed4d5c78 Removed console logs used in debuging 2019-01-08 15:02:41 +01:00
Odin Hultgren Van Der Horst
3875265721 CapnProto should now load jagged structures
Changed how actions are created for CapnProto so it supports jagged
structures.
2019-01-08 14:57:52 +01:00
Alexey Milovidov
1c7ccbf108 Miscellaneous [#CLICKHOUSE-2] 2019-01-07 22:56:53 +03:00
Alexey Milovidov
0dfa553fca Added more warnings from clang's -Weverything, that are available in clang 8 2019-01-07 13:40:58 +03:00
Alexey Milovidov
1d4701b0bc Added some warnings from clang's -Weverything 2019-01-04 15:10:00 +03:00
Alexey Milovidov
af31ff24b9 Better code; added a test #3835 2018-12-14 23:21:47 +03:00
alexey-milovidov
b6feafea41
Update ODBCDriver2BlockOutputStream.cpp 2018-12-14 22:31:38 +03:00
Max
4c145a9535 #3835 Add totals row for WITH TOTALS query 2018-12-14 20:43:04 +03:00
chertus
60dc8fcac2 fixes for review CLICKHOUSE-3578 2018-12-06 20:20:17 +03:00
chertus
3db00b7b98 Merge branch 'master' into issue-3578 2018-12-05 14:43:14 +03:00
chertus
933c055104 CLICKHOUSE-3578 review proress 2018-12-04 17:09:47 +03:00
Alexey Milovidov
7d6fffff48 More style checks [#CLICKHOUSE-3] 2018-11-24 04:48:06 +03:00
alexey-milovidov
2c2d75cceb
Don't allow exceptions without code (#3645)
* Don't allow exceptions without code #3633

* Don't allow exceptions without code #3633
2018-11-23 00:19:58 +03:00
proller
1d40c2c871 Better link and deps (#3631)
* try fix 1

* Better deps

* fix

* fix

* fix

* Update CMakeLists.txt

* Update CMakeLists.txt

* fix

* fix

* fix

* disable old

* Update find_odbc.cmake
2018-11-21 22:20:27 +03:00
proller
2deb523265 Fix build with old capnp 2018-11-20 18:10:28 +03:00
chertus
13646eb4c4 move BlockMissingValues from Block to IBlockInputStream::getMissingValues() 2018-11-15 18:57:20 +03:00
chertus
ebf3d60186 column defaults without CapNProto [CLICKHOUSE-3578] 2018-11-14 21:20:16 +03:00
chertus
9d3325f56e one more fix for bad merge of branches 2018-11-13 16:36:53 +03:00
chertus
2fe3cdb3f5 restore lost commits [CLICKHOUSE-3578] 2018-11-12 17:52:30 +03:00