Commit Graph

371 Commits

Author SHA1 Message Date
Avogar
2a923f434f Fix build errors 2020-06-20 01:32:57 +03:00
Avogar
f6ab431f2f Change String type to Binary 2020-06-19 17:50:44 +03:00
Avogar
07ad947c1d Add arrays support 2020-06-19 17:21:35 +03:00
Mikhail Filimonov
9609bd9dee Kafka better states, formats based on PeekableReadBuffer, and other minor fixes.
Add formats tests, fixes for JSONCompactEachRowWithNamesAndTypes, TSVWithNamesAndTypes. Some CR fixes
Add sanitizing for kafka_max_block_size and kafka_poll_max_batch_size
2020-06-19 10:49:05 +02:00
Avogar
71b5d267ce Set compression to None 2020-06-18 21:02:13 +03:00
Avogar
75a66fbba3 Fix errors 2 2020-06-18 15:52:46 +03:00
Avogar
4fd3bcd823 Fix build errors 2020-06-18 03:17:08 +03:00
Avogar
7780a74bd8 Update ORCBlockOutputFormat and add orc include directory in include path 2020-06-17 22:32:11 +03:00
Alexander Tokmakov
3938b051c2 fix type duduction in values format 2020-06-17 16:21:06 +03:00
Avogar
93aee32ae4 Add ORCBlockOutputFormat 2020-06-14 18:35:32 +03:00
Alexander Kuzmenkov
435f53e05a Improve error messages
Also support fmt for exception messages, the same way we use it in
logging macros.
2020-06-04 20:50:29 +03:00
Alexey Milovidov
6fea843d88 Fix Mac OS build 2020-06-02 16:56:03 +03:00
Alexey Milovidov
933644b8e4 Fix clang-tidy 2020-06-02 16:53:34 +03:00
Alexey Milovidov
cd07d61ce7 Updated tests 2020-06-02 05:39:21 +03:00
Alexey Milovidov
d34b1a8791 Merge branch 'master' into improve-performance-of-client-interactive-mode 2020-06-02 04:24:17 +03:00
Alexey Milovidov
c901a657f8 Improvement 2020-06-01 01:17:40 +03:00
Alexey Milovidov
a76bd99e84 Added setting "output_format_pretty_max_value_width" 2020-06-01 01:12:13 +03:00
Alexey Milovidov
2895cfb480 Limit value width in Pretty formats 2020-05-31 22:22:59 +03:00
Alexey Milovidov
25f941020b Remove namespace pollution 2020-05-31 00:57:37 +03:00
Alexey Milovidov
5aff138956 Preparation for structured logging 2020-05-31 00:35:52 +03:00
MovElb
b4b5c90343 squash 2020-05-30 20:05:05 +03:00
Alexander Kuzmenkov
ac263f921e Merge remote-tracking branch 'origin/master' into HEAD 2020-05-27 15:51:03 +03:00
Alexander Kuzmenkov
ad93644254 make single if 2020-05-27 15:50:12 +03:00
Alexander Kuzmenkov
c1cfc68cd6 revert accidental changes 2020-05-27 15:46:49 +03:00
Alexander Kuzmenkov
f54c892b49
Update src/Processors/Formats/Impl/ArrowBlockInputFormat.h 2020-05-27 15:41:59 +03:00
Alexander Kuzmenkov
61f833efc1
Update src/Processors/Formats/Impl/ArrowBlockInputFormat.h 2020-05-27 15:41:52 +03:00
Alexander Kuzmenkov
81a5af1938
Update src/Processors/Formats/Impl/ArrowBlockInputFormat.h 2020-05-27 15:41:45 +03:00
hcz
f2e3bcdd15 Improve code style in src and test files of Arrow formats 2020-05-25 10:50:55 +08:00
Alexey Milovidov
7e1813825b Return old names of macros 2020-05-24 01:24:01 +03:00
Alexey Milovidov
9d2a0d2dd7 Apply all transformations again 2020-05-23 21:59:49 +03:00
Alexey Milovidov
a2ad11897f Remove duplicate whitespaces (preparation) 2020-05-23 21:53:58 +03:00
Alexey Milovidov
1f13515a65 Make all LOG in single line (preparation) 2020-05-23 21:31:37 +03:00
Alexey Milovidov
ee4ffbc332 find {base,src,programs} -name '*.h' -or -name '*.cpp' | xargs grep -l -P 'LOG_\w+\([^,]+, "[^"]+" << [^<]+\);' | xargs sed -i -r -e 's/(LOG_\w+)\(([^,]+), "([^"]+)" << ([^<]+)\);/\1_FORMATTED(\2, "\3{}", \4);/' 2020-05-23 19:47:56 +03:00
hcz
ab37791134 Fix arrow stream termination 2020-05-21 15:28:10 +08:00
hcz
e11fa03bdd Add ArrowStream input and output format 2020-05-21 12:07:47 +08:00
Azat Khuzhin
d93b9a57f6 Forward declaration for Context as much as possible.
Now after changing Context.h 488 modules will be recompiled instead of 582.
2020-05-21 01:53:18 +03:00
Nikolai Kochetov
4d0503f021 Make IColumn::mutate() static. 2020-05-14 11:30:18 +03:00
Alexey Milovidov
fd26550a70 Remove commented code before merge #10580 2020-05-10 03:41:52 +03:00
Alexey Milovidov
d9257a8682 Merge branch 'arrow-io-format' of https://github.com/FawnD2/ClickHouse into FawnD2-arrow-io-format 2020-05-10 03:26:10 +03:00
Ivan
85d783c247
Poco contrib refactoring (#10396)
* Remove config_common.h
* Refactor libcpuid contrib
* Remove support for libcpuinfo
* Define USE_CPUID in Arcadia
* Refactor Poco libraries
2020-05-08 17:11:19 +03:00
FawnD2
19022cb670 Remove redundant includes. Style fix 2020-05-05 15:56:54 +03:00
FawnD2
aaf4f4ad61 Add missing Float16 for parquet format 2020-05-05 04:46:04 +03:00
FawnD2
2f8435813c Remove unused stream 2020-05-04 17:49:13 +03:00
FawnD2
da0a908d27 Remove unused stream 2020-05-04 17:47:09 +03:00
FawnD2
b492e92d41 Remove extra check 2020-05-04 17:32:36 +03:00
FawnD2
182de47012 Use file reader for Arrow format 2020-05-04 17:30:42 +03:00
FawnD2
6d6d04903f Remove check 2020-05-04 16:33:20 +03:00
FawnD2
a044766d58 Remove check 2020-05-04 16:26:20 +03:00
FawnD2
17450811d4 Move getHeader at the beginning of generate() 2020-05-04 16:19:25 +03:00
FawnD2
83148289d8 Rename arg 2020-05-04 16:01:37 +03:00
FawnD2
7b23844b32 Common order of precompiled checks 2020-05-04 03:58:10 +03:00
FawnD2
02e12215e7 Apply reducing memory usage optimization for seekable files to ORC format 2020-05-04 03:52:28 +03:00
FawnD2
a554177724 Simplify ORC format 2020-05-04 02:23:20 +03:00
FawnD2
112758b99d Merge branch 'master' into arrow-io-format 2020-05-04 00:53:17 +03:00
FawnD2
2359ba3497 Correct stop condition 2020-05-03 22:59:19 +03:00
FawnD2
6b2e97fd56 Increate total read length while reading 2020-05-03 22:54:28 +03:00
FawnD2
b4898bdd19 Read single batch 2020-05-03 22:49:06 +03:00
FawnD2
a8d914a811 Read full table 2020-05-03 22:22:41 +03:00
FawnD2
b86562b8bc Style fixes 2020-05-03 21:29:04 +03:00
FawnD2
7cc7a87f9f Simplify interfaces 2020-05-03 21:12:14 +03:00
FawnD2
a590826fbb Format settings for Arrow 2020-05-03 15:26:39 +03:00
FawnD2
e5b59616cd Style fix 2020-05-03 15:25:53 +03:00
Alexey Milovidov
469cdc315b Added suggestion from @azat #10607 2020-05-03 14:40:37 +03:00
FawnD2
f62f1a5924 Add missed include 2020-05-03 06:08:37 +03:00
FawnD2
6e0b31fb2a Arrow output format 2020-05-03 05:46:36 +03:00
FawnD2
aeed62d5e7 Remove unused error codes 2020-05-03 04:20:03 +03:00
FawnD2
4d0225f5a9 std::string -> String 2020-05-03 04:15:09 +03:00
FawnD2
f4fe23e3fe Add ptr wrapper. Fix style 2020-05-03 04:01:05 +03:00
FawnD2
fa1c35753b Move CH -> Arrow routine converter into separated class 2020-05-03 03:54:39 +03:00
Alexey Milovidov
554e7a0dd2 Preparations 2020-05-02 22:59:07 +03:00
Alexey Milovidov
e6ab4d655b Fix bad code 2020-05-02 22:54:29 +03:00
Alexey Milovidov
f0e80fec68 Whitespaces 2020-05-02 22:41:01 +03:00
alexey-milovidov
330627f597
Update ArrowBlockInputFormat.h 2020-05-02 22:35:58 +03:00
alexey-milovidov
47e3f83bc8
Update ArrowBlockInputFormat.h 2020-05-02 22:34:34 +03:00
alexey-milovidov
07f824c746
Update ArrowBlockInputFormat.cpp 2020-05-02 22:33:47 +03:00
alexey-milovidov
dfeb76038d
Update ArrowBlockInputFormat.h 2020-05-02 22:32:33 +03:00
alexey-milovidov
fbb280b5ac
Update ArrowBlockInputFormat.cpp 2020-05-02 22:32:21 +03:00
alexey-milovidov
250a44a47f
Merge pull request #10607 from Avogar/json_as_string_format
Add JSONAsStringRowInputFormat
2020-05-02 22:17:22 +03:00
Matwey V. Kornilov
d7627fa4fb Fix capnproto version check for capnp::UnalignedFlatArrayMessageReader
capnp::UnalignedFlatArrayMessageReader was introduced in
3aa2b2aa02
(which is a part of 0.7.0 release). Unfortunately,
capnp::UnalignedFlatArrayMessageReader was removed in
3f0fee61c6
(which is a part of 0.8.0 release)

So change CAPNP_VERSION check accordingly.
2020-05-01 22:32:06 +03:00
Avogar
697d5b07ce Fix build and style errors. Update test. 2020-05-01 19:41:14 +03:00
Avogar
6464a1a902 Add JSONAsStringRowInputFormat 2020-04-30 23:16:09 +03:00
FawnD2
538e6c39da Arrow input format 2020-04-28 22:52:22 +03:00
alexey-milovidov
436e4e3f5e
Merge pull request #10502 from oandrew/avro-nested2
Avro nested fields - support complex types
2020-04-27 13:44:58 +03:00
Andrew Onyshchuk
570be94a48 Use correct type for DateTime64 (ubsan) 2020-04-26 21:10:30 -05:00
alexey-milovidov
c6ae6a7ba8
Merge pull request #10418 from ClickHouse/fix_logical_error_in_diagnostic_info
Better diagnostic info in input formats
2020-04-26 20:26:59 +03:00
alexey-milovidov
8820b43606
Merge pull request #10479 from Avogar/markdown_format
Markdown format
2020-04-26 03:25:48 +03:00
Andrew Onyshchuk
e7ba75ce98 Avro nested fields - support complex types 2020-04-25 17:42:49 -05:00
Alexander Tokmakov
53dcce55e9 better diagnostic info in input formats 2020-04-24 15:03:48 +03:00
Avogar
778f1aa1e7 Merge branch 'markdown_format' of github.com:avogar/ClickHouse into markdown_format 2020-04-24 14:07:04 +03:00
Avogar
2458da802c Update MarkdownRowOutputFormat 2020-04-24 14:06:41 +03:00
Alexey Milovidov
03e17405cd Checkpoint 2020-04-22 10:03:43 +03:00
Alexey Milovidov
603d132d45 Checkpoint 2020-04-22 09:34:20 +03:00
Alexey Milovidov
0a7edce036 Checkpoint 2020-04-22 09:01:33 +03:00
alexey-milovidov
7690c34e5b
Merge pull request #10424 from ClickHouse/tsv-bom-extra-case
Allow to parse BOM in TSV if the first column cannot contain BOM in its value
2020-04-22 06:56:43 +03:00
Alexander Kuzmenkov
d1c5e38a21
Merge pull request #10385 from CurtizJ/tuple-literal
Parse tuples of tuples as literals
2020-04-22 05:49:14 +03:00
Alexey Milovidov
e8a9c7d518 Allow to parse BOM in TSV if the first column cannot contain BOM in its value 2020-04-22 02:46:42 +03:00
Anton Popov
3ccff833c1 fix ConstantExpressionTemplate with arrays 2020-04-21 15:54:36 +03:00
alexey-milovidov
3d4696d4ef
Update AvroRowInputFormat.cpp 2020-04-20 23:34:48 +03:00
Andrew Onyshchuk
68f2e93443 Fix style 2020-04-18 22:54:22 -05:00
Andrew Onyshchuk
14647ed03b Add Avro nested fields insert support 2020-04-18 15:46:24 -05:00
alexey-milovidov
d1b72f10d5
Merge pull request #10325 from Avogar/msgpack_format
Improve MsgPackRowInputFormat
2020-04-18 14:27:11 +03:00
alexey-milovidov
ef5f83eef8
Update MsgPackRowInputFormat.cpp 2020-04-18 14:26:57 +03:00
Kruglov Pavel
96f4d5b7c6
Fix build error 2020-04-17 23:51:53 +03:00
Kruglov Pavel
0f11526f24
Fix style error 2020-04-17 22:19:03 +03:00
Avogar
b056dbce1c Improve MsgPackRowInputFormat 2020-04-17 12:35:38 +03:00
alexey-milovidov
17e7d4d88a
Merge pull request #10307 from abyss7/arcadia-4
Changes for auto-sync with Arcadia
2020-04-17 05:08:34 +03:00
alexey-milovidov
390bfe9a4c
Merge pull request #10317 from Avogar/markdown_format
Markdown format
2020-04-17 05:05:13 +03:00
Kruglov Pavel
2672b71b2d
Delete extra spaces 2020-04-16 23:47:01 +03:00
Avogar
89d787dae3 Add new line at the end of file 2020-04-16 23:14:19 +03:00
Avogar
831c48c514 Add MarkdownRowOutput format 2020-04-16 23:10:42 +03:00
Ivan Lezhankin
e230632645 Changes required for auto-sync with Arcadia 2020-04-16 15:31:57 +03:00
alexey-milovidov
d296e1b6d7
Merge pull request #10216 from Avogar/msgpack_format
Update MsgPack format
2020-04-16 00:44:08 +03:00
Avogar
93c322bc30 Fix style error 2020-04-15 02:58:36 +03:00
Avogar
b8061356b0 Update parsing method in MsgPackRowInputFormat 2020-04-15 02:08:55 +03:00
Avogar
e46322fcef Update MsgPack input format. 2020-04-13 22:33:02 +03:00
Avogar
b9931863ef Fix FixedString packing 2020-04-13 00:01:17 +03:00
alexey-milovidov
0782052768
Merge pull request #10131 from ClickHouse/fix-extremes-for-processors
Fix extremes for processors
2020-04-12 02:31:39 +03:00
Nikolai Kochetov
458c7f516d Fix extremes for Processors. 2020-04-08 21:50:14 +03:00
Avogar
e635b0e9eb Fix build error 2020-04-04 20:04:41 +03:00
Avogar
fbc11d3bf9 Move files 2020-04-04 17:04:20 +03:00
Ivan Lezhankin
06446b4f08 dbms/ → src/ 2020-04-03 18:14:31 +03:00