thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-10-09 18:10:48 +00:00
Code Issues Packages Projects Releases Wiki Activity
35,603 Commits 8,985 Branches 1,770 Tags 2.1 GiB
fc9fe91331
Commit Graph

386 Commits

Author SHA1 Message Date
Alexey Milovidov
413b780b65 Various fixes for PVS-Studio 2019-06-16 21:13:39 +03:00
Ivan Lezhankin
facdd966cc Fixes due to review 2019-06-14 20:19:02 +03:00
Vitaliy Zakaznikov
d50586d5a5 Merge branch 'master' of https://github.com/vzakaznikov/ClickHouse into liveview 2019-06-12 15:40:58 -04:00
Ivan Lezhankin
97558c1b41 Merge remote-tracking branch 'upstream/master' into issue-5286 2019-06-10 14:15:14 +03:00
alexey-milovidov
4c7d71af1c
Merge pull request #5560 from yandex/perftests-file-formats 
Fix #5439 bug PrettySpace format
 2019-06-09 17:22:47 +03:00
alexey-milovidov
7324e5e8d4
Update PrettySpaceBlockOutputStream.cpp 2019-06-09 17:22:16 +03:00
stavrolia
6c487a253b Fix #5439 bug pretty space 2019-06-08 00:04:06 +03:00
Vitaliy Zakaznikov
aa3ef47aab * Adding JSONEachRowWithProgress format 
that can be used with WATCH queries when HTTP connection is used
  in this case progress events are used as a heartbeat
* Adding new tests that use JSONEachRowWithProgress format
* Adding tests for WATCH query heartbeats when HTTP connection is used
* Small fixes to uexpect.py
 2019-06-07 09:05:14 -04:00
Orivej Desh
5ec179377a DEVTOOLS-5170 Delete Y_IGNORE markers (#5533) 
They have been superseded by a new include resolution configuration that lives
outside clickhouse source tree.
 2019-06-05 14:52:39 +03:00
Ivan Lezhankin
b311984879 Merge remote-tracking branch 'upstream/master' into issue-5286 2019-06-03 17:19:09 +03:00
Alexey Milovidov
05737678e2 Remove debug logging from MySQL protocol 2019-05-30 11:25:16 +03:00
Ivan Lezhankin
d2fd7a449f Fix build 2019-05-26 23:10:43 +03:00
Yuriy
96c2bb383f using logging level from config.xml 2019-05-26 22:30:23 +03:00
Yuriy
96e3574a06 Improvements of MySQL Wire Protocol 2019-05-26 09:52:29 +03:00
alexey-milovidov
461c4919d3
Merge pull request #5116 from PerformanceVision/send_header 
Send a X-ClickHouse summary on the header for HTTP client with number of rows inserted
 2019-05-25 20:20:35 +03:00
Alexey Milovidov
b94f2be154 Merge branch 'mysql' of https://github.com/yurriy/ClickHouse into yurriy-mysql 2019-05-25 17:14:57 +03:00
alexey-milovidov
b79582e15a
Merge pull request #5395 from clemrodriguez/master 
Close MySQL connections opened by external MySQL dictionaries
 2019-05-25 14:13:51 +03:00
Clément Rodriguez
8e8a563e30 Code style fixes 2019-05-24 14:30:12 +02:00
chertus
9eca696e82 actualize isNumber for Decimal 2019-05-24 15:11:03 +03:00
Ivan Lezhankin
19a850ad75 Use read callback to populate virtual columns in Kafka Engine 2019-05-23 16:20:25 +03:00
Clément Rodriguez
7b40d37212 Close MySQL connections after usage in MySQLDictionarySource 2019-05-23 15:09:07 +02:00
Ivan Lezhankin
800854119e Add buffer callback to fill in virtual columns 2019-05-23 14:15:18 +03:00
Guillaume Tassery
77b7158cd8 Progress rename for total_rows and write_[rows|bytes] 2019-05-20 18:37:41 +07:00
Ivan Lezhankin
0821d62516 Merge ITableDeclaration into IStorage. 
Also a couple of massive include-refactorings with better forwarding
 2019-05-17 17:34:25 +03:00
Yuriy
a51c293ec6 merged 'origin/master' into mysql 2019-05-16 14:19:05 +03:00
Yuriy
4ee84810b0 including information about query execution in response 2019-05-16 08:36:08 +03:00
Yuriy
ff4937859e added MySQL wire protocol presentational format 2019-05-16 06:34:04 +03:00
Ivan
1ea9e3019d
Freeze the Kafka buffer after first empty response (#5283) 
* Check inside inferior streams for cancellation while reading.
* Stop reading from Kafka buffer after first empty read.
 2019-05-15 19:11:50 +03:00
Ivan
4a5832b18a
Check the time limit every (flush_interval / poll_timeout) number of rows from Kafka (#5249) 2019-05-14 18:52:03 +03:00
Nikolai Kochetov
808d0f3b08 Merged with master. 2019-05-14 13:31:19 +03:00
Martijn Bakker
f76a2e2735 Merge branch 'master' into martijn-datetime64 2019-05-01 20:00:53 +01:00
Martijn Bakker
a602a6c79c add datetime64 definition 2019-04-23 14:05:45 +01:00
Alexander Kazakov
ecc6da1429 Fixed missing includes 2019-04-22 20:02:34 +03:00
Alexander Kazakov
c7a4b2ec75 Fixed style 2019-04-22 19:53:54 +03:00
Alexander Kazakov
bde1845bdc Better naming 2019-04-22 17:00:51 +03:00
Alexander Kazakov
d90bab5e33 In [T|C]SVWithNames formats start using column information from header 2019-04-22 16:31:17 +03:00
Alexey Milovidov
b0b369b30f Fixed typo #5010 2019-04-18 22:46:59 +03:00
Alexey Milovidov
642b09c8d6 Added clarifications #5010 2019-04-18 22:45:46 +03:00
Alexey Milovidov
f2cf2ffc80 Cleanups 2019-04-11 00:20:22 +03:00
Alexey Milovidov
36d016d331 Added [[noreturn]] attributes for functions that throw unconditionally 2019-04-10 23:27:14 +03:00
Nikolai Kochetov
ec65b4c229 Check num columns in Block::cloneWithColumns. 2019-04-05 14:39:07 +03:00
Nikolai Kochetov
7626b1b267 Multiple fixes. 2019-04-05 13:52:07 +03:00
Nikolai Kochetov
5e39ec610c Merged with master 2019-04-03 14:21:38 +03:00
Nikolai Kochetov
94ff7f694a QueryPipeline [in progress]. 2019-04-03 13:14:36 +03:00
Vitaly Baranov
37b904b8f5
Merge pull request #4808 from vitlibar/kafka-with-protobuf-format 
Kafka with protobuf format
 2019-03-30 19:00:57 +03:00
Vitaly Baranov
378c6f7a23 Improve the error message when format schema isn't set. 2019-03-29 22:36:10 +03:00
Vladislav Smirnov
7a02d320ff rewrite capnproto reading 2019-03-25 20:17:58 +03:00
Vladislav Smirnov
08b7c48286 Merge remote-tracking branch 'upstream/master' into fix-capnproto 2019-03-25 19:24:30 +03:00
proller
fce9db12b9 Move Interpreters/Settings* -> Core/Settings* 2019-03-22 15:08:30 +03:00
proller
faeca57d9a
Try fix Default timezone regression #4658 (#4718) 2019-03-20 16:29:20 +03:00
Vladislav Smirnov
59d473e215 style 2019-03-13 18:28:29 +03:00
Vladislav Smirnov
a916b62f33 ignore instead append 2019-03-13 14:18:18 +03:00
Vladislav Smirnov
7d822d8081 tabs -> spaces 2019-03-13 14:05:52 +03:00
Vladislav Smirnov
22f4852830 fix reading capnproto 2019-03-13 13:50:49 +03:00
Alexey Milovidov
9d89cc0fc1 Debugging PODArray 2019-03-08 06:23:02 +03:00
proller
482d153dcd clickhouse-odbc #157 : Remove LowCardinality from type (#4381) 
* clickhouse-odbc #157 : Remove LowCardinality from type

* Update ODBCDriver2BlockOutputStream.cpp

* fix
 2019-03-06 21:37:16 +03:00
proller
ffc9fbf769 Build fixes (#4600) 
* Add Y_IGNORE

* Build fixes
 2019-03-06 02:53:35 +03:00
Amos Bird
26ab5dd7a7 A Proper lookup table that uses HashTable's API 
This is the first step of allowing heterogeneous cells in hash tables.

performance test results are

```

1. HashMap<UInt16, UInt8, TrivialHash, HashTableFixedGrower<16>>;
2. NewLookupMap<UInt16, UInt8>

ResolutionWidth 30000 1 .................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................223550276.46
ResolutionWidth 30000 2 .................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................248772721.24
Best: 2 - 24877272124

ResolutionWidth 100000 1 ..........................................................................................................................................................................................................................................................238498413.99
ResolutionWidth 100000 2 ..........................................................................................................................................................................................................................................................261808889.98
Best: 2 - 26180888998

ResolutionWidth 300000 1 ...................................................................................239307348.81
ResolutionWidth 300000 2 ...................................................................................257592761.30
Best: 2 - 25759276130

ResolutionWidth 1000000 1 .........................240144759.26
ResolutionWidth 1000000 2 .........................257093531.91
Best: 2 - 25709353191

ResolutionWidth 5000000 1 .....241573260.35
ResolutionWidth 5000000 2 .....259314162.79
Best: 2 - 25931416279

ResolutionDepth 30000 1 .................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................217108119.84
ResolutionDepth 30000 2 .................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................249459504.41
Best: 2 - 24945950441

ResolutionDepth 100000 1 ..........................................................................................................................................................................................................................................................229065162.17
ResolutionDepth 100000 2 ..........................................................................................................................................................................................................................................................253769105.64
Best: 2 - 25376910564

ResolutionDepth 300000 1 ...................................................................................233079225.18
ResolutionDepth 300000 2 ...................................................................................256316273.78
Best: 2 - 25631627378

ResolutionDepth 1000000 1 .........................234184633.51
ResolutionDepth 1000000 2 .........................261100491.57
Best: 2 - 26110049157

ResolutionDepth 5000000 1 .....233118795.66
ResolutionDepth 5000000 2 .....252436160.41
Best: 2 - 25243616041

```
 2019-03-01 16:47:13 +08:00
proller
23d70b2e04
Add Y_IGNORE (#4521) 2019-02-27 19:52:08 +03:00
Vitaly Baranov
6fbcc12674
Merge pull request #4493 from vitlibar/support-for-nested-messages-in-protobuf-output 
Added support for nested messages in protobuf output format
 2019-02-27 18:16:45 +03:00
proller
211ec436c5
Build fixes (#4516) 2019-02-27 15:52:24 +03:00
Vitaly Baranov
147a7a1458 Remove unused function ProtobufSchemas::getMessageTypeForColumns(). 2019-02-27 14:37:17 +03:00
Vitaly Baranov
e712a78cce Minor improvements of ProtobufReader. 2019-02-27 14:37:17 +03:00
Vitaly Baranov
85ab57f179 Implement support of nested messages in Protobuf output format. 2019-02-27 14:37:17 +03:00
Vitaly Baranov
4f46052f2e Optimize the class ProtobufWriter and prepare for implementing support of nested messages. 2019-02-27 14:37:14 +03:00
Vitaly Baranov
1cce1feac1 Move IConverter to header for allow inlining ProtobufWriter functions. 2019-02-27 00:54:25 +03:00
Vitaly Baranov
2f0276503a Move ProtobufSimpleWriter to the internals of ProtobufWriter. 2019-02-27 00:54:25 +03:00
Vitaly Baranov
65d997db94 Simplify excluding ProtobufWriter from compilation when USE_PROTOBUF=0. 2019-02-27 00:54:25 +03:00
Vitaly Baranov
ba2efbae01 Make the interface IRowOutputStream more generic and use it for writing protobufs. 2019-02-27 00:53:13 +03:00
Alexey Milovidov
ae28c46a46 Added workaround for compiler bug 2019-02-26 13:37:47 +03:00
Vitaly Baranov
dee7da9eb4
Merge branch 'master' into improve-performance-of-input-from-protobuf 2019-02-22 01:43:55 +03:00
proller
00e50f1502
Build fixes (#4471) 2019-02-21 23:11:21 +03:00
Nikolai Kochetov
25f6b97486 Added processors for Parquet and Protobuf. 2019-02-21 21:36:46 +03:00
Nikolai Kochetov
e91a0a31b7 Merge branch 'master' into processors 2019-02-21 19:19:35 +03:00
Nikolai Kochetov
aba56e171c Formats processorss. 2019-02-21 19:05:47 +03:00
Vitaly Baranov
4793ccb3fc Improve performance of ProtobufReader. 2019-02-21 18:49:42 +03:00
Vitaly Baranov
eef02034b2 Implement a new row input stream for reading protobuf messages. 2019-02-20 07:28:18 +03:00
Vitaly Baranov
630012dfb7 Fix error text about format schema. 2019-02-20 07:28:18 +03:00
Vitaly Baranov
9cd4c7495f Implement an utility class for reading protobuf messages. 2019-02-20 07:28:02 +03:00
Vitaly Baranov
74cc6f285c Implement an utility class to match columns and protobuf fields. 2019-02-20 05:32:29 +03:00
proller
e20c77e4c6 Add parquet support. Fixes, tests, ... 2019-02-19 23:51:44 +03:00
proller
263e69e861
Build fixes (FreeBSD port) (#4397) 2019-02-15 14:46:07 +03:00
Alexey Milovidov
c345ded839 Added comment #4200 2019-02-11 03:17:49 +03:00
alexey-milovidov
f58e0c7c97
Merge pull request #4200 from DarkWanderer/feature/row-binary-with-headers 
Added RowBinaryWithNamesAndTypes format
 2019-02-11 03:14:56 +03:00
Alexey Milovidov
58a6accee5 Updated test #4246 2019-02-10 19:55:12 +03:00
DarkWanderer
6e92183854 Fixed format export test, fixed clang compilation 2019-02-04 22:59:59 +03:00
Vasily Nemkov
2716df8ab2 IPv4 and IPv6 Domain implementation. 
Added:
 * IDataTypeDomain interface;
 * method DataTypeFactory::registerDataTypeDomain for registering domains;
 * DataTypeDomainWithSimpleSerialization domain base class with simple serialization/deserialization;
 * Concrete IPv4 and IPv6 domain implementations: DataTypeDomanIPv6 and DataTypeDomanIPv4;

Updated:
 * IDataType text serialization/deserialization methods;
 * IDataType implementation to use domain for text serialization/deserialization;
 * Refactored implementation of the IPv4/IPv6 functions to use formatIPv4/v6 and parseIPv4/v6 from Common/formatIPv6.h;

Tests:
 * Added test cases for IPv4 and IPv6 domains.
 * Updated IPv4/v6 functions tests to validate more cases;
 * Added performance tests for IPv4 and IPv6 related functions;
 2019-02-04 12:37:46 +03:00
DarkWanderer
e2fe15aafa Build fix: round 2 2019-01-31 22:30:36 +03:00
DarkWanderer
c8cff57a62 Fix compile errors, added output format test 2019-01-30 23:38:47 +03:00
DarkWanderer
bcac2722d1 Added RowBinaryWithNamesAndTypes format 
* Extended BinaryRowOutputStream with flags allowing to output names/types of columns
* Added one (for now) new output format - RowBinaryWithNamesAndTypes
* Updated docs
 2019-01-30 14:30:43 +03:00
Vitaly Baranov
e9721c9548 Add support for absolute format schema paths. 2019-01-27 12:15:32 +03:00
proller
5328251ff1
Fix build without protobuf, gtest, cppkafka (#4152) 2019-01-25 23:02:03 +03:00
Vitaly Baranov
ed1136e49e Use more IO/WriteHelpers.h functions in the class ProtobufSimpleWriter, 
rename variant=>varint.
 2019-01-25 17:15:44 +03:00
Alexey Milovidov
304c7b0386 Style #4005 2019-01-24 22:50:16 +03:00
alexey-milovidov
f96b8e4ed6
Merge pull request #4005 from vitlibar/add-protobuf-output-format 
Add protobuf output format
 2019-01-24 22:05:23 +03:00
Vitaly Baranov
2886f42c0e Implemented a new block output stream using protobuf output format. 2019-01-24 17:54:57 +03:00
Vitaly Baranov
c241a4ea6d Added function serializeProtobuf() to each data type. 2019-01-24 17:54:57 +03:00
Vitaly Baranov
1e7a231c56 Implemented main class for writing protobufs. 2019-01-24 17:54:52 +03:00
Vitaly Baranov
42d9d4e81d Implemented storage for parsed protobuf schemas. 2019-01-24 17:32:10 +03:00
Vitaly Baranov
28e9837819 Implemented utility class for writing protobufs. 2019-01-24 17:32:01 +03:00
Vitaly Baranov
ec37ef5a89 No escaping for the format schema file's name anymore, 
added an utility class for parsing and checking the format schema.
 2019-01-24 17:31:41 +03:00
Ivan Lezhankin
05ab15cc84 Merge IBlockInputStream and IProfilingBlockInputStream into one 2019-01-23 17:48:50 +03:00
Alexey Milovidov
c3aae60aed Trying to understand the code #4063 2019-01-17 21:49:56 +03:00
Alexey Milovidov
736e1e2d71 Fixed build #4063 2019-01-17 18:21:21 +03:00
alexey-milovidov
eeac860e0a
Update CapnProtoRowInputStream.cpp 2019-01-16 16:14:54 +03:00
alexey-milovidov
fb68929b10
Update CapnProtoRowInputStream.cpp 2019-01-16 16:14:34 +03:00
Odin Hultgren Van Der Horst
3ff868917e Changed tabs to spaces 2019-01-15 09:41:48 +01:00
Odin Hultgren Van Der Horst
3378bf58f5 Comments and changes to parent check 
- Added some comments.
 - Immplemented a recurisve parent check so fields with parents by
the same name but grandparents with diffrent names dont cause problems
 2019-01-15 09:27:38 +01:00
Odin Hultgren Van Der Horst
8bff8d4e2d Merge branch 'master' of https://github.com/yandex/ClickHouse 2019-01-15 08:13:51 +01:00
proller
72df7ceee6 Macos build fix 2019-01-14 18:20:09 +03:00
Odin Hultgren Van Der Horst
b844bdef15 Tabs are now spaces 2019-01-10 09:18:13 +01:00
Odin Hultgren Van Der Horst
23c93b8c79 Removed include statemnets 2019-01-10 08:55:59 +01:00
Odin Hultgren Van Der Horst
ffdf565528 Merge branch 'master' of https://github.com/Miniwoffer/ClickHouse 2019-01-09 15:27:11 +00:00
Odin Hultgren Van Der Horst
96352ce78d Added _ as delim 
Added _ as delim
This makes it so you do not have to use . when addresing structures
and since Capnproto can not use _ in its naming its fine to use.
 2019-01-09 15:24:17 +00:00
Odin Hultgren Van Der Horst
68ed4d5c78 Removed console logs used in debuging 2019-01-08 15:02:41 +01:00
Odin Hultgren Van Der Horst
3875265721 CapnProto should now load jagged structures 
Changed how actions are created for CapnProto so it supports jagged
structures.
 2019-01-08 14:57:52 +01:00
Alexey Milovidov
1c7ccbf108 Miscellaneous [#CLICKHOUSE-2] 2019-01-07 22:56:53 +03:00
Alexey Milovidov
0dfa553fca Added more warnings from clang's -Weverything, that are available in clang 8 2019-01-07 13:40:58 +03:00
Alexey Milovidov
1d4701b0bc Added some warnings from clang's -Weverything 2019-01-04 15:10:00 +03:00
Alexey Milovidov
af31ff24b9 Better code; added a test #3835 2018-12-14 23:21:47 +03:00
alexey-milovidov
b6feafea41
Update ODBCDriver2BlockOutputStream.cpp 2018-12-14 22:31:38 +03:00
Max
4c145a9535 #3835 Add totals row for WITH TOTALS query 2018-12-14 20:43:04 +03:00
chertus
60dc8fcac2 fixes for review CLICKHOUSE-3578 2018-12-06 20:20:17 +03:00
chertus
3db00b7b98 Merge branch 'master' into issue-3578 2018-12-05 14:43:14 +03:00
chertus
933c055104 CLICKHOUSE-3578 review proress 2018-12-04 17:09:47 +03:00
Alexey Milovidov
7d6fffff48 More style checks [#CLICKHOUSE-3] 2018-11-24 04:48:06 +03:00
alexey-milovidov
2c2d75cceb
Don't allow exceptions without code (#3645) 
* Don't allow exceptions without code #3633

* Don't allow exceptions without code #3633
 2018-11-23 00:19:58 +03:00
proller
1d40c2c871 Better link and deps (#3631) 
* try fix 1

* Better deps

* fix

* fix

* fix

* Update CMakeLists.txt

* Update CMakeLists.txt

* fix

* fix

* fix

* disable old

* Update find_odbc.cmake
 2018-11-21 22:20:27 +03:00
proller
2deb523265 Fix build with old capnp 2018-11-20 18:10:28 +03:00
chertus
13646eb4c4 move BlockMissingValues from Block to IBlockInputStream::getMissingValues() 2018-11-15 18:57:20 +03:00
chertus
ebf3d60186 column defaults without CapNProto [CLICKHOUSE-3578] 2018-11-14 21:20:16 +03:00
chertus
9d3325f56e one more fix for bad merge of branches 2018-11-13 16:36:53 +03:00
chertus
2fe3cdb3f5 restore lost commits [CLICKHOUSE-3578] 2018-11-12 17:52:30 +03:00
chertus
cc005c3ac9 Merge branch 'master' into issue-3578 2018-11-09 13:12:34 +03:00
alexey-milovidov
b1db9ec23c
Merge pull request #3436 from amosbird/master 
Flexible fields assignment
 2018-11-01 20:22:28 +03:00
alesapin
892ab4ce8a Better code for check on null values inside array IN expression 2018-10-31 12:17:47 +03:00
alesapin
444b6e1e43 Add validation of array data 2018-10-29 16:18:41 +03:00
Amos Bird
3cf2177268 Flexible fields assignment 
Field f;

before:

f = Int64(-42);

after:

f = -42;
 2018-10-24 08:47:33 +08:00
Alexey Milovidov
7b24ad570c Inline some functions #3223 2018-10-10 00:38:06 +03:00
Alexey Milovidov
c014168c5f Attempt to fix performance regression while parsing JSONEachRow #3223 2018-10-10 00:21:52 +03:00
Alexey Milovidov
a432ff335e Attempt to fix performance regression while parsing JSONEachRow #3223 2018-10-10 00:09:48 +03:00
Alexey Milovidov
4a31c1c18a Attempt to fix performance regression while parsing JSONEachRow #3223 2018-10-09 22:46:35 +03:00
Alexey Milovidov
9a50300e42 Minor modifications #3223 2018-10-09 21:14:30 +03:00
Alexey Milovidov
e4b4b5a4fc Better diagnostic for JSONEachRow format #3223 2018-10-09 21:06:52 +03:00
alexey-milovidov
052fc2b674
Merge pull request #3223 from veloman-yunkan/jsoneachrow_bugfix 
Fix for issue #3297
 2018-10-09 21:05:51 +03:00
Marek Vavruša
5d5c99ebee Formats/CapnProtoRowInputStream: fix column mismatch in list of structures 
The fields are lexicographically sorted to make traversal easier,
but their order must be preserved when collecting fields from structures.

For example, a list with a structure like `{b @0 :Text, a @1 :Text}` would
read `a` first despite being second, which would cause a mismatch.
 2018-10-04 15:59:27 -07:00
alexey-milovidov
147a2a13c2
Merge pull request #3257 from amosbird/master 
Correct wcwidth computation for pretty outputs.
 2018-10-02 16:23:41 +03:00
Amos Bird
7df73088ab Correct wcwidth computation for pretty outputs. 2018-10-02 10:06:18 +08:00
alexey-milovidov
ff53aa4947
Update CapnProtoRowInputStream.cpp 2018-10-01 04:33:36 +03:00
alexey-milovidov
fac040c39a
Update CapnProtoRowInputStream.cpp 2018-10-01 04:31:48 +03:00
Veloman Yunkan
3e96f32833 Fixed the bug introduced by PR#3144 
The pre-existing optimization in the function
JSONEachRowRowInputStream::readColumnName() (that was extracted
during the refactoring step of PR#3144) imposed a restriction
on its usage - reading from the input stream might invalidate the return
value of that function, and this is what happenned in readJSONObject()
after the call to skipColonDelimiter().

One way of fixing the problem while preserving the original optimization
intact would be to defer the call to skipColonDelimiter() until the
variable name_ref was fully consumed, however that would result in worse
code (skipColonDelimiter() would need to be called in three different
places where it doesn't really belong).

Therefore I preferred to slightly weaken the optimization by always
copying the key name into the current_column_name data member.
 2018-09-26 11:27:04 +00:00
Marek Vavruša
cbbcb6d9df Formats/CapnProtoRowInputStream: support Nested and Tuple, fix alignment issues 
This updated contrib/capnproto to a newer version that fixes problems with
unaligned access to message frames.

It also adds support for parsing Struct types as Tuple (named or unnamed),
and Nested array types.

The `struct X { a @0 :UInt64; b @1 :Text }` in Cap'nProto is equivalent to
`x Tuple(a UInt64, b String)` in ClickHouse.

Arrays of Struct types such as `y List(X)` are equivalent to `y Nested(a UInt64, b String)`.
 2018-09-25 22:51:35 -07:00
alesapin
b87a084f38 Strict order of evaluated and added columns 2018-09-20 14:16:15 +03:00
alesapin
e1c8045055 Fix error with header 2018-09-19 15:56:56 +03:00
alesapin
c2f3453fda Reorder Squashing and adding default output stream in insert 2018-09-19 14:06:42 +03:00
Alexey Milovidov
46ef387ce3 Style #3144 2018-09-17 23:35:21 +03:00
Veloman Yunkan
5fe127e4c3 Added --input_format_import_nested_json option 
By default mapping of nested json data to nested tables is disabled. To
enable the import of nested json data (into corresponding nested tables)
clickhouse must be run with the --input_format_import_nested_json=1
option.
 2018-09-14 13:43:57 +00:00
Veloman Yunkan
47eb0e28b3 Nested JSON data is mapped to nested table data 2018-09-14 12:15:32 +00:00
Veloman Yunkan
8d427dd09d Extracted JSONEachRowRowInputStream::readJSONObject 
Also changed the names of previously extracted functions to camel case.
 2018-09-14 10:47:26 +00:00
Veloman Yunkan
4cf2cd9586 Some more refactoring 2018-09-14 08:27:49 +00:00
Veloman Yunkan
270f13fe99 Some refactoring 2018-09-14 07:28:49 +00:00
chertus
0b35cdae69 refactoring: add some sugar in type checks [CLICKHOUSE-0002] 2018-09-07 17:37:26 +03:00
Alexey Milovidov
0c6c1e83a3 Avoid excessive padding in Pretty formats [#CLICKHOUSE-2] 2018-08-31 02:34:12 +03:00
BlahGeek
739119d355 skip object fields on json input 2018-08-27 12:01:47 +08:00
Alexey Milovidov
d8ca4345b6 Fixed style violations [#CLICKHOUSE-2] 2018-08-10 04:41:54 +03:00
proller
973bdab77f Format ODBCDriver2 with NULL support (#2834) 
* Format ODBCDriver2 with NULL support

* Fix comment

* Update ODBCDriver2BlockOutputStream.cpp

* clean
 2018-08-10 04:20:10 +03:00
Alex Bocharov
e8d4654419 Remove FormatSettings default value for writeJSONString and add tests for output_format_json_escape_forward_slashes. 2018-08-08 02:35:32 +03:00
alexey-milovidov
bc16e8ca69 Update FormatSettings.h 2018-08-08 02:35:32 +03:00
alexey-milovidov
31cff11fee Update FormatFactory.cpp 2018-08-08 02:35:32 +03:00
Alex Bocharov
a8818bb8af Add output_format_json_escape_slashes setting - controls escaping slashes for string outputs in JSON output format. 2018-08-08 02:35:32 +03:00
Alexey Milovidov
a106c65dec Miscellaneous #2702 2018-07-24 21:50:28 +03:00
Amos Bird
77ce910077 Fix invalid Context reference. 2018-07-24 21:47:47 +03:00
Alexey Milovidov
09d22da9b9 Miscellaneous #2687 2018-07-20 18:59:11 +03:00
alesapin
2915a58c8f Merge remote-tracking branch 'upstream/master' 2018-07-20 13:13:50 +03:00
alesapin
ec156e9f2b CLICKHOUSE-3772: Add system tables formats, data_type_families, table_functions, aggregate_function_combinators 2018-07-20 13:00:56 +03:00
Alexey Milovidov
7f04f593ed Moved code to .cpp; log only when skipping of errors is enabled #2669 2018-07-18 22:02:40 +03:00
Amos Bird
8b2e68bfa0 Function try in destructor rethrows exceptions 
According to https://en.cppreference.com/w/cpp/language/function-try-block , function try block used in destructor will rethrow exceptions if there isn't a return statement specified.
 2018-07-18 16:29:12 +03:00
Leonardo Cecchi
6d8d59bd1b Log number of skipped rows during a bulk insert 
When in a bulk insert the rows are parsed server-side and there are
skipped rows, log the number of these rows at the TRACE level.
 2018-07-18 03:04:01 +03:00
chertus
07a782e637 fix wrong defaults at columns tail and some build fixes 2018-07-16 22:28:07 +03:00
chertus
c8f71cb539 Merge branch 'master' into issue-3578 2018-07-13 14:30:42 +03:00
chertus
82d22574d4 fix empty defaults mask issue and add some comments 2018-07-11 17:11:47 +03:00
Alexey Milovidov
84fd35da41 Adaptations #2574 2018-07-05 00:00:50 +03:00
chertus
88c4081a82 3578: defaults for input streams (in progress) 2018-07-04 20:02:47 +03:00
Amos Bird
900b046943 Add options to enable parsing CSV with single/double quote. 2018-06-29 15:34:55 +08:00
proller
e1ae5233df Add // Y_IGNORE tags 2018-06-21 18:24:36 +03:00
alexey-milovidov
fafecb3c25
Better modularity of formats. (#2492) 
* Formats: better modularity (development) #2447

* Formats: better modularity (development) #2447

* Formats: better modularity (development) #2447

* Formats: better modularity (development) #2447

* Formats: better modularity (development) #2447

* Formats: better modularity (development): removed very old tests #2447

* Formats: better modularity (development) #2447

* Formats: better modularity (development) #2447

* Formats: better modularity (development) #2447

* Formats: better modularity (development) #2447

* Formats: better modularity (development) #2447

* Formats: better modularity (development) #2447

* Formats: better modularity (development) #2447

* Formats: better modularity (development) #2447
 2018-06-10 22:22:49 +03:00