thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-14 18:32:29 +00:00
Code Issues Packages Projects Releases Wiki Activity
77,837 Commits 9,328 Branches 1,783 Tags 2.1 GiB
4aeadf3967
Commit Graph

687 Commits

Author SHA1 Message Date
FawnD2
26488467f8 Merge branch 'master' into switch-upstream-for-arrow-submodule 2020-11-29 15:53:28 +03:00
hexiaoting
5b3a7f5c98 Merge remote-tracking branch 'origin/master' into dev_map2 2020-11-26 16:43:27 +08:00
hexiaoting
74d875c837 Merge remote-tracking branch 'origin/master' into dev_map2 2020-11-26 15:13:52 +08:00
Nico Piderman
f1b8906418
Minor grammar correction in error messages 
Noticed this when I got an error loading some JSONEachRow data.
'while read the value of' -> 'while reading the value of'.
 2020-11-19 15:44:58 +01:00
Alexander Kuzmenkov
8cde88440b Write rows as JSON array in JSONEachRow output format 2020-11-17 22:50:47 +03:00
Nikita Mikhaylov
68b6f08e36
Merge pull request #16721 from nikitamikhaylov/parsing-constraints 
Turn off parallel parsing when memory limit is small.
 2020-11-17 17:37:14 +03:00
alexey-milovidov
9ae14d8f19
Update AvroRowInputFormat.cpp 2020-11-14 17:45:47 +03:00
Mikhail Filimonov
e78010c28c
Attempt to fix #16926 2020-11-13 15:21:04 +01:00
Nikita Mikhaylov
33bada767c
Merge branch 'master' into parsing-constraints 2020-11-12 23:25:39 +03:00
Alexander Tokmakov
b94cc5c4e5 remove more stringstreams 2020-11-10 21:22:26 +03:00
Alexander Tokmakov
5cdfcfb307 remove other stringstreams 2020-11-09 22:12:44 +03:00
tavplubix
67099f28ac
Merge pull request #16591 from ClickHouse/aku/create-file 
Support `SETTINGS` clause for File engine
 2020-11-09 14:15:42 +03:00
Alexey Milovidov
fd84d16387 Fix "server failed to start" error 2020-11-07 03:14:53 +03:00
nikitamikhaylov
9803565af7 style 2020-11-06 21:59:16 +03:00
nikitamikhaylov
f1709f9a80 better 2020-11-06 21:55:13 +03:00
nikitamikhaylov
2febfd43e5 rewrite format line as string 2020-11-06 21:55:13 +03:00
hexiaoting
28e87da976 remove map literal and Merge remote-tracking branch 'origin/master' into dev_map2 2020-11-06 15:07:41 +08:00
FawnD2
069e3dec0d Do not use deprecated API 2020-11-05 20:10:11 +03:00
Alexander Kuzmenkov
2efbcbaa14 Merge remote-tracking branch 'origin/master' into HEAD 2020-11-05 14:28:56 +03:00
Alexey Milovidov
0b1c96db51 Provide row number when "Cannot read all data" during parsing input 2020-11-03 23:32:18 +03:00
Alexander Kuzmenkov
0f22a9dfcd fix build 2020-11-02 19:13:42 +03:00
Alexander Kuzmenkov
99ee127620 Support SETTINGS clause for File engine 
Accept the usual user settings related to file formats.

Most of the diff are the mechanistic code changes required to allow
providing the required FormatSettings to the format factory. The File
engine then extracts these settings from the `CREATE` query, and specifies
them when creating the format parser.
 2020-11-02 10:50:38 +03:00
hexiaoting
3a70cf3773 Merge remote-tracking branch 'origin/master' into dev_map2 2020-11-02 14:46:56 +08:00
alesapin
e06f65ee73
Merge pull request #16521 from myrrc/improvement/avro-format-lc 
Fixing the inability to deserialize AVRO into table if it contains LowCardinality columns
 2020-10-30 11:31:22 +03:00
myrrc
9564a7f6d1 simplified the cast 2020-10-29 20:30:24 +03:00
myrrc
68b47f5f61 removing LC in AVRO input format if needed 2020-10-29 20:24:54 +03:00
Pavel Kruglov
dbfef325e6 Rename NullSink to NullOutput 2020-10-29 20:22:48 +03:00
hexiaoting
483be134b2 Support MergeTree Engine 2020-10-23 16:36:17 +08:00
hexiaoting
fcd35b0619 Merge branch 'master' into dev_map2 2020-10-18 16:18:13 +08:00
Vitaly Baranov
5f261cec3c
Merge pull request #15199 from filimonov/non_delimited_protobuf 
ProtobufSingle format
 2020-10-15 16:58:39 +03:00
Amos Bird
1bcacde8d5
Eagerly move data out of sink 2020-10-14 21:05:20 +08:00
hexiaoting
4f3f221987 Fix build error 2020-10-14 14:35:14 +08:00
hexiaoting
cdb4dd57a6 Add new DataType Map(key,value) 2020-10-14 14:35:14 +08:00
Mikhail Filimonov
87a8ba63f7
Few more CR fixes 2020-10-12 15:05:40 +02:00
Mikhail Filimonov
7fb329713a
Avoid shadowing class member, set first_row flag after write 2020-10-12 07:52:38 +02:00
Mikhail Filimonov
f6b00f2cb6
After CR fixes 2020-10-12 07:52:09 +02:00
Mikhail Filimonov
c37a456c49
Refactoring WriteCallback->RowOutputFormatParams 2020-10-12 07:52:06 +02:00
Mikhail Filimonov
e17d4e1eeb
Tests and some docs 2020-10-12 07:51:46 +02:00
Mikhail Filimonov
6df42c580b
ProtobufSingle output format 2020-10-12 07:51:45 +02:00
Mikhail Filimonov
b602d18dd8
ProtobufSingle input format 
Allows to read Single protobuf message at once (w/o length-delemeters).
 2020-10-12 07:51:36 +02:00
alexey-milovidov
2fcf2e0fbe
Merge pull request #15443 from ucasFL/pretty_row_numbers 
Add setting `output_format_pretty_row_numbers`
 2020-10-05 11:49:55 +03:00
feng lv
d2f75f8508 fix 2020-10-02 23:13:28 +08:00
feng lv
c857b5dd5d fix 2020-09-30 09:50:23 +08:00
alexey-milovidov
9f944424cf
Merge pull request #15364 from ClickHouse/format-raw-blob 
Add format RawBLOB
 2020-09-30 01:14:40 +03:00
feng lv
4f000388a7 add setting output_format_pretty_row_numbers 2020-09-29 20:30:36 +08:00
Alexey Milovidov
e4800e7b35 Fix comment 2020-09-29 00:10:36 +03:00
Alexander Kuzmenkov
51b9aaf4d8
Merge pull request #15363 from ClickHouse/format-regexp-raw 
Add support for "Raw" column format for Regexp format
 2020-09-28 20:03:32 +03:00
Nikita Mikhaylov
72e4b10c00
Merge pull request #14842 from hexiaoting/hxt_dev 
Fix bug for LineAsString Format
 2020-09-28 14:02:23 +03:00
Alexey Milovidov
a7ddd8489a Added format RawBLOB 2020-09-28 03:11:19 +03:00
Alexey Milovidov
b800a7fdf1 Add support for "Raw" column format for Regexp format 2020-09-28 01:09:32 +03:00
hexiaoting
023daddbea Fix bug for LineAsString Format 2020-09-15 18:13:39 +08:00
Artem Zuikov
51ba12c2c3
Try speedup build (#14809) 2020-09-15 12:55:57 +03:00
nikitamikhaylov
d0c259974d style check 2020-09-10 19:35:18 +03:00
Nikita Mikhaylov
a39ba57e8c
Merge branch 'master' into dev-nf 2020-09-10 18:10:47 +04:00
Nikita Mikhaylov
f0a65522cc
Merge pull request #14333 from hczhcz/patch-0901 
Add JSONStrings formats
 2020-09-10 17:54:42 +04:00
hcz
d8fce448a2 Implement null_as_default for JSONStrings formats 2020-09-09 15:47:32 +08:00
Nikolai Kochetov
d771ce5609
Merge pull request #14500 from ClickHouse/many-rows-in-totals 
Many rows in totals
 2020-09-08 10:40:14 +03:00
Nikita Mikhaylov
a56d42de67 fix arcadia 2020-09-07 20:55:06 +03:00
Nikolai Kochetov
73de00f4df Review fix. 2020-09-07 20:49:23 +03:00
Nikita Mikhaylov
2c04b0a8e6 comment added 2020-09-07 19:07:34 +03:00
Alexey Milovidov
4a336e3814 More consistent invocation of skipBOMIfExists 2020-09-07 07:21:55 +03:00
Nikolai Kochetov
41a3fad1ea Allow many rows in totals. 2020-09-04 15:34:36 +03:00
hcz
a80c1adee8 Add JSONCompactStrings formats 2020-09-02 12:15:58 +08:00
hcz
86fa185bb6 Add JSONStrings formats 2020-09-01 17:06:36 +08:00
alexey-milovidov
65ee7dcb18
Merge pull request #13941 from gervarela/avro_confluent_skip_missing_magic_or_schema_id 
Skip abnormaly small messages in AvroConfluent format
 2020-08-26 19:34:42 +03:00
alexey-milovidov
3c6675f7f4
Update AvroRowInputFormat.cpp 2020-08-26 02:26:27 +03:00
alexey-milovidov
5d4507907d
Update AvroRowInputFormat.cpp 2020-08-26 00:19:51 +03:00
alexey-milovidov
efa5b76605
Update AvroRowInputFormat.cpp 2020-08-26 00:17:39 +03:00
Alexander Kuzmenkov
e3c919ec19
Merge pull request #13847 from hexiaoting/dev_rmcode 
Deprecate ODBCDriver format
 2020-08-24 22:34:52 +03:00
alexey-milovidov
b71ef6db22
Update AvroRowInputFormat.cpp 2020-08-21 18:34:15 +03:00
Gervasio Varela
a6a18b62f9 AvroConfluent + Kafla: Skip malformed messages that do not contain at least the AvroConfluent magic number and the schema id definition. 2020-08-21 11:11:41 +02:00
tao jiang
294aa985e0 fix typo exception name 2020-08-21 08:53:00 +08:00
alexey-milovidov
c5a7b1c456
Update RowInputFormatWithDiagnosticInfo.cpp 2020-08-20 20:34:55 +03:00
alexey-milovidov
911946b517
Update TabSeparatedRowInputFormat.cpp 2020-08-20 20:32:49 +03:00
tao jiang
189c284597 fix if statement code style check 2020-08-20 18:42:14 +08:00
tao jiang
70ef84ee4b fix code style check 2020-08-20 18:06:41 +08:00
Jiang Tao
613e3f7247 optimize error msg for null value 2020-08-20 11:25:28 +08:00
hexiaoting
e9be2f14ea fix implementation for \n separated lines 2020-08-19 11:50:43 +08:00
hexiaoting
ddbfe0ce21 Deprecate ODBCDriver format 2020-08-18 11:29:40 +08:00
hexiaoting
0451d57293 Add new feature: LineAsString Format 2020-08-18 10:35:08 +08:00
hexiaoting
405a6fb08f New feature: LineAsString format. #13630 2020-08-17 18:20:23 +08:00
tavplubix
18ef911fe8
Merge pull request #13672 from zhang2014/refactor/materialize_mysql_database 
ISSUES-4006 split mysql protocol to multiple packets
 2020-08-16 13:55:11 +03:00
alexey-milovidov
d95b9f73a4
Update TabSeparatedRowInputFormat.cpp 2020-08-15 11:11:10 +03:00
zhang2014
96bd3ac34b ISSUES-4006 split mysql protocol 2020-08-13 20:41:36 +08:00
zhang2014
34f4c8972e ISSUES-4006 split replication packets 2020-08-13 20:07:02 +08:00
zhang2014
c76d7e3121 ISSUES-4006 split generic packets 2020-08-13 18:17:13 +08:00
zhang2014
c0d42b764e ISSUES-4006 split protocol text packet 2020-08-13 17:20:45 +08:00
zhang2014
4fcc178deb ISSUES-4006 split mysql protocol 2020-08-13 16:17:33 +08:00
zhang2014
688836cdc4 ISSUES-4006 split msql protocol & fix build 2020-08-13 14:30:29 +08:00
Azat Khuzhin
6097f769c7 Fix assertion for carriage return in the header in TSVWithNames* format. 
TabSeparatedRowInputFormat::readPrefix() tries to check for carriage
return (via checkForCarriageReturn()), however it does the check after
checking the column is exists, reorder the codepath a little bit to
run checkForCarriageReturn() before.
 2020-08-12 11:13:19 +03:00
Azat Khuzhin
d5a2e1e61c Call doWritePrefix() if IOutputFormat works as Processor (i.e. clickhouse-local) 
This fixes the *WithNames output format (lack of header before this
patch).
 2020-08-11 00:52:35 +03:00
Azat Khuzhin
ba92213d21 Write header only once in TSVWithNames* 2020-08-11 00:52:35 +03:00
Alexey Milovidov
edd89a8610 Fix half of typos 2020-08-08 03:47:03 +03:00
Nikolai Kochetov
79f30f9446
Merge pull request #13399 from hczhcz/patch-0806 
Fix empty output of arrow related formats
 2020-08-07 15:31:45 +03:00
hcz
898e19e18d Fix function names 2020-08-07 15:40:05 +08:00
Nikolai Kochetov
539be71635 Fix tests. 2020-08-06 13:05:27 +03:00
Azat Khuzhin
15be6a0dd5 Fix quirks for PrettyCompactMonoBlock 
- Fix PrettyCompactMonoBlock for clickhouse-local (broken in 20.3+,
  fails with an error `Unknown format PrettyCompactMonoBlock`, after #6239)
- Fix extremes/totals with PrettyCompactMonoBlock (even before 20.3
  breakage they were simply ignored)
 2020-08-06 11:59:45 +03:00
hcz
dadcca6c08 Fix empty output of arrow related formats 2020-08-06 16:32:31 +08:00
Andrew Onyshchuk
2883831564 AvroConfluent: skip broken messages 2020-08-04 02:43:42 -05:00
Andrew Onyshchuk
99c183da0f AvroConfluent: skip tombstone records 2020-08-04 02:43:42 -05:00
Artem Zuikov
2afd123eda
Refactoring: extract TreeOptimizer from SyntaxAnalyzer (#12645) 2020-07-22 20:13:05 +03:00
alesapin
21d3a79794 Fix arcadia build for msgpack 2020-07-16 14:08:26 +03:00
alesapin
0740b88d6b Merge branch 'master' into fasttest 2020-07-14 15:19:05 +03:00
alexey-milovidov
7e679b731d
Merge pull request #12477 from azat/arrow-deprecated-fixes 
arrow deprecated API usage build fixes
 2020-07-14 09:23:52 +03:00
alexey-milovidov
1893d89ce3
Merge pull request #12448 from ClickHouse/fix-trash-rabbitmq 
Fix trash from RabbitMQ
 2020-07-14 01:11:37 +03:00
Azat Khuzhin
4ec5e79ade Do not suppress -Wdeprecated* for arrow 2020-07-13 21:25:51 +03:00
Azat Khuzhin
5f6fdb89bb Fix arrow deprecated API usage 
And also use fmt-style for exceptions
 2020-07-13 21:25:49 +03:00
Alexey Milovidov
6edf43d112 Fix build 2020-07-13 19:51:37 +03:00
tavplubix
3314a97b99
Merge pull request #12009 from hczhcz/patch-0628 
Allow TabSeparatedRaw as an input format
 2020-07-13 19:43:31 +03:00
hcz
3683891ebc Fix TSVRaw on long text 2020-07-13 12:51:43 +08:00
Alexey Milovidov
8f2055b0a0 Fix trash from RabbitMQ 2020-07-13 04:11:48 +03:00
alexey-milovidov
4b2da605da
Update PrettyCompactBlockOutputFormat.cpp 2020-07-11 00:32:21 +03:00
alexey-milovidov
ca0591320d
Update PrettyBlockOutputFormat.cpp 2020-07-11 00:31:25 +03:00
alesapin
d2fcbe1f4e Fix build 2020-07-10 22:27:12 +03:00
Maxim Sabyanin
40f7ec71d3 add setting output_format_pretty_grid_charset 
This setting allows to chose charset for printing grids (either UTF-8 or
ASCII).
 2020-07-10 22:25:49 +03:00
alesapin
a9824fd650 Better cmake and less libraries 2020-07-10 22:08:18 +03:00
alexey-milovidov
36205e3ddf
Merge pull request #12181 from azat/bump-arrow-to-0.17 
Bump arrow to 0.17 (and flatbuffers to v1.12, required by arrow)
 2020-07-09 04:24:47 +03:00
Azat Khuzhin
d2d49972f1 Bump CI (after non-restartable inner CI issue) 
This reverts commit d199961e6e.
 2020-07-08 21:18:49 +03:00
tavplubix
33c3545125
Merge pull request #12197 from nikitamikhaylov/csv-file-bugfix 
Headers for CSVWithNames in StorageFile
 2020-07-08 20:42:49 +03:00
alexey-milovidov
d199961e6e
Rerun tests 2020-07-08 15:12:23 +03:00
Azat Khuzhin
fd5e08884a Bump arrow to 0.17 (and flatbuffers to v1.12, required by arrow) 
MOTIVATION:
- remove double-conversion external dependency
- remove flatc (but flatbuffers is still required, arrow just shipped
  with generated files and that's it)

CHANGED:
- remove pre-generated headers, it is shipped with the arrow
- remove flatc (see above)

NOTES (see tests changes):
- and snappy error is reported as unsupported compression.
 2020-07-07 23:54:32 +03:00
Nikita Mikhaylov
270551e599 done 2020-07-07 14:45:20 +03:00
Nikita Mikhaylov
53522c728b
Merge pull request #11662 from Avogar/orc_output_format 
Add ORCBlockOutputFormat
 2020-07-06 18:51:35 +04:00
Nikita Mikhaylov
dac7a39256 fix ubsan final 2020-07-06 12:29:22 +03:00
alexey-milovidov
cf1967bb29
Merge pull request #12089 from ClickHouse/fix-11595 
Fix result_rows and result_bytes metrics for selects.
 2020-07-04 23:41:51 +03:00
Nikita Mikhaylov
fba02badcb fix test with UBSan 2020-07-03 17:53:54 +03:00
Nikolai Kochetov
6d08b7b45a Remove LC converting to Arrow. 2020-07-03 14:04:36 +03:00
Nikolai Kochetov
3d37a63247 Remove LC converting to Arrow. 2020-07-03 13:37:55 +03:00
Nikolai Kochetov
b3f1842035 Remove LC converting to Arrow. 2020-07-03 13:35:44 +03:00
Nikolai Kochetov
e7ab4df148 Fix result_rows and result_bytes metrics for selects. 2020-07-02 18:00:37 +03:00
Nikolai Kochetov
57d727d078 Fix result_rows and result_bytes metrics for selects. 2020-07-02 17:51:10 +03:00
hcz
e284bf83d0 Optimize and fix 2020-07-01 11:21:53 +08:00
Nikita Mikhaylov
c45a054f65 fix clang-tidy 2020-06-30 16:06:35 +03:00
Nikita Mikhaylov
b74b27854a yet another try 2020-06-30 14:38:09 +03:00
Nikita Mikhaylov
7c828861da fix arcadia 2020-06-30 13:18:38 +03:00
Nikita Mikhaylov
b2712aa242 disable orc in arcadia at all 2020-06-29 23:48:18 +03:00
Nikita Mikhaylov
33e627777b Merge branch 'master' of github.com:ClickHouse/ClickHouse into Avogar_orc_output_format 2020-06-29 17:25:33 +03:00
hcz
0a8a29272b Add TabSeparatedRawRowInputFormat 2020-06-29 15:06:43 +08:00
Andrew Onyshchuk
9054862dde Avro: allow missing fields 2020-06-27 21:23:21 -05:00
Avogar
81d459a5a8 Small update ORCBlockOutputFormat and add ORC output format to performance test. 2020-06-26 23:17:28 +03:00
alexey-milovidov
751d36f7c7
Merge pull request #11954 from oandrew/avro-uuid 
Avro UUID support
 2020-06-26 14:09:54 +03:00
Andrew Onyshchuk
9364d42eb0 Add CANNOT_PARSE_UUID extern const 2020-06-25 14:50:07 -05:00
Andrew Onyshchuk
fc1ae85600 Avro UUID support 2020-06-25 14:25:45 -05:00
Nikita Mikhaylov
717f63923a
Merge branch 'master' into orc_output_format 2020-06-23 15:07:05 +04:00
Nikita Mikhaylov
ec3d101768
style fix + bump tests 2020-06-23 13:50:56 +03:00
Alexey Milovidov
199a6576a5 Remove dependency 2020-06-21 19:37:12 +03:00
Alexey Milovidov
3f96bdaa24 Minor changes after merge 2020-06-21 15:16:08 +03:00
alexey-milovidov
4ee623ccac
Merge pull request #10242 from MovElb/movelb-postgresql-wire-protocol-impl 
PostgreSQL wire protocol implementation
 2020-06-21 14:39:22 +03:00
Avogar
2a923f434f Fix build errors 2020-06-20 01:32:57 +03:00
Avogar
f6ab431f2f Change String type to Binary 2020-06-19 17:50:44 +03:00
Avogar
07ad947c1d Add arrays support 2020-06-19 17:21:35 +03:00
Mikhail Filimonov
9609bd9dee Kafka better states, formats based on PeekableReadBuffer, and other minor fixes. 
Add formats tests, fixes for JSONCompactEachRowWithNamesAndTypes, TSVWithNamesAndTypes. Some CR fixes
Add sanitizing for kafka_max_block_size and kafka_poll_max_batch_size
 2020-06-19 10:49:05 +02:00
Avogar
71b5d267ce Set compression to None 2020-06-18 21:02:13 +03:00
Avogar
75a66fbba3 Fix errors 2 2020-06-18 15:52:46 +03:00
Avogar
4fd3bcd823 Fix build errors 2020-06-18 03:17:08 +03:00
Avogar
7780a74bd8 Update ORCBlockOutputFormat and add orc include directory in include path 2020-06-17 22:32:11 +03:00
Alexander Tokmakov
3938b051c2 fix type duduction in values format 2020-06-17 16:21:06 +03:00
Avogar
93aee32ae4 Add ORCBlockOutputFormat 2020-06-14 18:35:32 +03:00
Alexander Kuzmenkov
435f53e05a Improve error messages 
Also support fmt for exception messages, the same way we use it in
logging macros.
 2020-06-04 20:50:29 +03:00
Alexey Milovidov
6fea843d88 Fix Mac OS build 2020-06-02 16:56:03 +03:00
Alexey Milovidov
933644b8e4 Fix clang-tidy 2020-06-02 16:53:34 +03:00
Alexey Milovidov
cd07d61ce7 Updated tests 2020-06-02 05:39:21 +03:00
Alexey Milovidov
d34b1a8791 Merge branch 'master' into improve-performance-of-client-interactive-mode 2020-06-02 04:24:17 +03:00
Alexey Milovidov
c901a657f8 Improvement 2020-06-01 01:17:40 +03:00
Alexey Milovidov
a76bd99e84 Added setting "output_format_pretty_max_value_width" 2020-06-01 01:12:13 +03:00
Alexey Milovidov
2895cfb480 Limit value width in Pretty formats 2020-05-31 22:22:59 +03:00
Alexey Milovidov
25f941020b Remove namespace pollution 2020-05-31 00:57:37 +03:00
Alexey Milovidov
5aff138956 Preparation for structured logging 2020-05-31 00:35:52 +03:00
MovElb
b4b5c90343 squash 2020-05-30 20:05:05 +03:00
Alexander Kuzmenkov
ac263f921e Merge remote-tracking branch 'origin/master' into HEAD 2020-05-27 15:51:03 +03:00
Alexander Kuzmenkov
ad93644254 make single if 2020-05-27 15:50:12 +03:00
Alexander Kuzmenkov
c1cfc68cd6 revert accidental changes 2020-05-27 15:46:49 +03:00
Alexander Kuzmenkov
f54c892b49
Update src/Processors/Formats/Impl/ArrowBlockInputFormat.h 2020-05-27 15:41:59 +03:00
Alexander Kuzmenkov
61f833efc1
Update src/Processors/Formats/Impl/ArrowBlockInputFormat.h 2020-05-27 15:41:52 +03:00
Alexander Kuzmenkov
81a5af1938
Update src/Processors/Formats/Impl/ArrowBlockInputFormat.h 2020-05-27 15:41:45 +03:00
hcz
f2e3bcdd15 Improve code style in src and test files of Arrow formats 2020-05-25 10:50:55 +08:00
Alexey Milovidov
7e1813825b Return old names of macros 2020-05-24 01:24:01 +03:00
Alexey Milovidov
9d2a0d2dd7 Apply all transformations again 2020-05-23 21:59:49 +03:00
Alexey Milovidov
a2ad11897f Remove duplicate whitespaces (preparation) 2020-05-23 21:53:58 +03:00
Alexey Milovidov
1f13515a65 Make all LOG in single line (preparation) 2020-05-23 21:31:37 +03:00
Alexey Milovidov
e391b77d81 find {base,src,programs} -name '*.h' -or -name '*.cpp' | xargs grep -l -P 'LOG_\w+\([^,]+, "[^"]+" << [^<]+ << "[^"]+"\);' | xargs sed -i -r -e 's/(LOG_\w+)\(([^,]+), "([^"]+)" << ([^<]+) << "([^"]+)"\);/\1_FORMATTED(\2, "\3{}\5", \4);/' 2020-05-23 19:56:05 +03:00
Alexey Milovidov
ee4ffbc332 find {base,src,programs} -name '*.h' -or -name '*.cpp' | xargs grep -l -P 'LOG_\w+\([^,]+, "[^"]+" << [^<]+\);' | xargs sed -i -r -e 's/(LOG_\w+)\(([^,]+), "([^"]+)" << ([^<]+)\);/\1_FORMATTED(\2, "\3{}", \4);/' 2020-05-23 19:47:56 +03:00
Nikolai Kochetov
f7456f8320
Merge pull request #11004 from ClickHouse/remove-experimental-use-processors-flag-2 
Remove allow_processors flag from executeQuery()
 2020-05-22 10:47:15 +03:00
hcz
ab37791134 Fix arrow stream termination 2020-05-21 15:28:10 +08:00
hcz
e11fa03bdd Add ArrowStream input and output format 2020-05-21 12:07:47 +08:00
Azat Khuzhin
d93b9a57f6 Forward declaration for Context as much as possible. 
Now after changing Context.h 488 modules will be recompiled instead of 582.
 2020-05-21 01:53:18 +03:00
Nikolai Kochetov
bcd38ac207 Fix build. 2020-05-20 22:08:20 +03:00
Nikolai Kochetov
6f50700f35 Add PullingOutputFormat and PullingPipelineExecutor. 2020-05-20 21:53:18 +03:00
alexey-milovidov
a3346e9331
Merge pull request #10912 from ClickHouse/static-mutate 
Static mutate
 2020-05-17 09:05:35 +03:00
Nikolai Kochetov
8c2d0aa7c0 Fix comment. 2020-05-15 22:30:41 +03:00
Nikolai Kochetov
681955661f Add PullingPipelineExecutor. 2020-05-15 00:03:38 +03:00
Nikolai Kochetov
4d0503f021 Make IColumn::mutate() static. 2020-05-14 11:30:18 +03:00
Alexey Milovidov
fd26550a70 Remove commented code before merge #10580 2020-05-10 03:41:52 +03:00
Alexey Milovidov
d9257a8682 Merge branch 'arrow-io-format' of https://github.com/FawnD2/ClickHouse into FawnD2-arrow-io-format 2020-05-10 03:26:10 +03:00
Ivan
85d783c247
Poco contrib refactoring (#10396) 
* Remove config_common.h
* Refactor libcpuid contrib
* Remove support for libcpuinfo
* Define USE_CPUID in Arcadia
* Refactor Poco libraries
 2020-05-08 17:11:19 +03:00
FawnD2
19022cb670 Remove redundant includes. Style fix 2020-05-05 15:56:54 +03:00
FawnD2
aaf4f4ad61 Add missing Float16 for parquet format 2020-05-05 04:46:04 +03:00
FawnD2
2f8435813c Remove unused stream 2020-05-04 17:49:13 +03:00
FawnD2
da0a908d27 Remove unused stream 2020-05-04 17:47:09 +03:00
FawnD2
b492e92d41 Remove extra check 2020-05-04 17:32:36 +03:00
FawnD2
182de47012 Use file reader for Arrow format 2020-05-04 17:30:42 +03:00
FawnD2
6d6d04903f Remove check 2020-05-04 16:33:20 +03:00
FawnD2
a044766d58 Remove check 2020-05-04 16:26:20 +03:00
FawnD2
17450811d4 Move getHeader at the beginning of generate() 2020-05-04 16:19:25 +03:00
FawnD2
83148289d8 Rename arg 2020-05-04 16:01:37 +03:00
FawnD2
7b23844b32 Common order of precompiled checks 2020-05-04 03:58:10 +03:00
FawnD2
02e12215e7 Apply reducing memory usage optimization for seekable files to ORC format 2020-05-04 03:52:28 +03:00
FawnD2
a554177724 Simplify ORC format 2020-05-04 02:23:20 +03:00
FawnD2
112758b99d Merge branch 'master' into arrow-io-format 2020-05-04 00:53:17 +03:00
FawnD2
2359ba3497 Correct stop condition 2020-05-03 22:59:19 +03:00
FawnD2
6b2e97fd56 Increate total read length while reading 2020-05-03 22:54:28 +03:00
FawnD2
b4898bdd19 Read single batch 2020-05-03 22:49:06 +03:00
FawnD2
a8d914a811 Read full table 2020-05-03 22:22:41 +03:00
FawnD2
b86562b8bc Style fixes 2020-05-03 21:29:04 +03:00
FawnD2
7cc7a87f9f Simplify interfaces 2020-05-03 21:12:14 +03:00
FawnD2
a590826fbb Format settings for Arrow 2020-05-03 15:26:39 +03:00
FawnD2
e5b59616cd Style fix 2020-05-03 15:25:53 +03:00
Alexey Milovidov
469cdc315b Added suggestion from @azat #10607 2020-05-03 14:40:37 +03:00
FawnD2
f62f1a5924 Add missed include 2020-05-03 06:08:37 +03:00
FawnD2
6e0b31fb2a Arrow output format 2020-05-03 05:46:36 +03:00
FawnD2
aeed62d5e7 Remove unused error codes 2020-05-03 04:20:03 +03:00
FawnD2
4d0225f5a9 std::string -> String 2020-05-03 04:15:09 +03:00
FawnD2
f4fe23e3fe Add ptr wrapper. Fix style 2020-05-03 04:01:05 +03:00
FawnD2
fa1c35753b Move CH -> Arrow routine converter into separated class 2020-05-03 03:54:39 +03:00
Alexey Milovidov
554e7a0dd2 Preparations 2020-05-02 22:59:07 +03:00
Alexey Milovidov
e6ab4d655b Fix bad code 2020-05-02 22:54:29 +03:00
Alexey Milovidov
f0e80fec68 Whitespaces 2020-05-02 22:41:01 +03:00
alexey-milovidov
330627f597
Update ArrowBlockInputFormat.h 2020-05-02 22:35:58 +03:00
alexey-milovidov
47e3f83bc8
Update ArrowBlockInputFormat.h 2020-05-02 22:34:34 +03:00
alexey-milovidov
07f824c746
Update ArrowBlockInputFormat.cpp 2020-05-02 22:33:47 +03:00
alexey-milovidov
dfeb76038d
Update ArrowBlockInputFormat.h 2020-05-02 22:32:33 +03:00
alexey-milovidov
fbb280b5ac
Update ArrowBlockInputFormat.cpp 2020-05-02 22:32:21 +03:00
alexey-milovidov
250a44a47f
Merge pull request #10607 from Avogar/json_as_string_format 
Add JSONAsStringRowInputFormat
 2020-05-02 22:17:22 +03:00
Matwey V. Kornilov
d7627fa4fb Fix capnproto version check for capnp::UnalignedFlatArrayMessageReader 
capnp::UnalignedFlatArrayMessageReader was introduced in
3aa2b2aa02
(which is a part of 0.7.0 release). Unfortunately,
capnp::UnalignedFlatArrayMessageReader was removed in
3f0fee61c6
(which is a part of 0.8.0 release)

So change CAPNP_VERSION check accordingly.
 2020-05-01 22:32:06 +03:00
Avogar
697d5b07ce Fix build and style errors. Update test. 2020-05-01 19:41:14 +03:00
Avogar
6464a1a902 Add JSONAsStringRowInputFormat 2020-04-30 23:16:09 +03:00
FawnD2
538e6c39da Arrow input format 2020-04-28 22:52:22 +03:00
Nikolai Kochetov
de6d2d01b9 Try to fix tests. 2020-04-27 21:15:55 +03:00
Nikolai Kochetov
2f06180c5e Revert changes for CSVRowOutputFormat. 2020-04-27 18:21:53 +03:00
Nikolai Kochetov
201a184d31 Enable auto flust for output formats if enable_streaming was set. 2020-04-27 18:00:10 +03:00
Nikolai Kochetov
32baa073b5 Merge remote-tracking branch 'upstream/master' into insert_into_watch_from_live_view 2020-04-27 17:49:50 +03:00
alexey-milovidov
436e4e3f5e
Merge pull request #10502 from oandrew/avro-nested2 
Avro nested fields - support complex types
 2020-04-27 13:44:58 +03:00
Andrew Onyshchuk
570be94a48 Use correct type for DateTime64 (ubsan) 2020-04-26 21:10:30 -05:00
alexey-milovidov
c6ae6a7ba8
Merge pull request #10418 from ClickHouse/fix_logical_error_in_diagnostic_info 
Better diagnostic info in input formats
 2020-04-26 20:26:59 +03:00
alexey-milovidov
d1464a441f
Merge pull request #10503 from oandrew/read-columns-perf-fix 
IRowInputFormat performance fix
 2020-04-26 17:36:54 +03:00
Vitaliy Zakaznikov
626468d255 Adding support for streaming mode to JSONEachRowRowOutputFormat 2020-04-26 15:56:45 +02:00
Vitaliy Zakaznikov
369b4d53ef Adding support for output_format_enable_streaming format setting. 2020-04-26 15:44:11 +02:00
Andrew Onyshchuk
4a5497498e IRowInputFormat performance fix 2020-04-25 19:33:36 -05:00
Alexey Milovidov
d01bd44f14 Merge branch 'master' into execute-query-flush-format-2 2020-04-26 03:31:46 +03:00
alexey-milovidov
8820b43606
Merge pull request #10479 from Avogar/markdown_format 
Markdown format
 2020-04-26 03:25:48 +03:00
Andrew Onyshchuk
e7ba75ce98 Avro nested fields - support complex types 2020-04-25 17:42:49 -05:00
Azat Khuzhin
40c65f7bee IRowInputFormat: omit "Skipped 0 rows with errors while reading the input stream" 
This will just pollute the output log.

And this is significant for kafka streaming, since it will print this
message for each message (due to how
ReadBufferFromKafkaConsumer/KafkaBlockInputStream works).

You can see this in the server log of the test_storage_kafka
(integration test), for the table with kafka_skip_broken_messages > 0,
it will have 52 lines of such message, 2 with "Skipped 1 rows" and 50
with "Skipped 0 rows" (since there are 52 messages in summary, 2
malformed and 50 correct).
 2020-04-25 15:16:29 +03:00
Alexander Tokmakov
53dcce55e9 better diagnostic info in input formats 2020-04-24 15:03:48 +03:00
Avogar
778f1aa1e7 Merge branch 'markdown_format' of github.com:avogar/ClickHouse into markdown_format 2020-04-24 14:07:04 +03:00
Avogar
2458da802c Update MarkdownRowOutputFormat 2020-04-24 14:06:41 +03:00
Alexey Milovidov
03e17405cd Checkpoint 2020-04-22 10:03:43 +03:00
Alexey Milovidov
603d132d45 Checkpoint 2020-04-22 09:34:20 +03:00
Alexey Milovidov
0a7edce036 Checkpoint 2020-04-22 09:01:33 +03:00
alexey-milovidov
7690c34e5b
Merge pull request #10424 from ClickHouse/tsv-bom-extra-case 
Allow to parse BOM in TSV if the first column cannot contain BOM in its value
 2020-04-22 06:56:43 +03:00
Alexander Kuzmenkov
d1c5e38a21
Merge pull request #10385 from CurtizJ/tuple-literal 
Parse tuples of tuples as literals
 2020-04-22 05:49:14 +03:00
Alexey Milovidov
e8a9c7d518 Allow to parse BOM in TSV if the first column cannot contain BOM in its value 2020-04-22 02:46:42 +03:00
Anton Popov
3ccff833c1 fix ConstantExpressionTemplate with arrays 2020-04-21 15:54:36 +03:00
alexey-milovidov
3d4696d4ef
Update AvroRowInputFormat.cpp 2020-04-20 23:34:48 +03:00
Alexey Milovidov
5262512f0c Flush output for interactive applications 2020-04-19 22:02:36 +03:00
Andrew Onyshchuk
68f2e93443 Fix style 2020-04-18 22:54:22 -05:00
Andrew Onyshchuk
14647ed03b Add Avro nested fields insert support 2020-04-18 15:46:24 -05:00
alexey-milovidov
d1b72f10d5
Merge pull request #10325 from Avogar/msgpack_format 
Improve MsgPackRowInputFormat
 2020-04-18 14:27:11 +03:00
alexey-milovidov
ef5f83eef8
Update MsgPackRowInputFormat.cpp 2020-04-18 14:26:57 +03:00
Kruglov Pavel
96f4d5b7c6
Fix build error 2020-04-17 23:51:53 +03:00
Kruglov Pavel
0f11526f24
Fix style error 2020-04-17 22:19:03 +03:00
Avogar
b056dbce1c Improve MsgPackRowInputFormat 2020-04-17 12:35:38 +03:00
alexey-milovidov
17e7d4d88a
Merge pull request #10307 from abyss7/arcadia-4 
Changes for auto-sync with Arcadia
 2020-04-17 05:08:34 +03:00
alexey-milovidov
390bfe9a4c
Merge pull request #10317 from Avogar/markdown_format 
Markdown format
 2020-04-17 05:05:13 +03:00
Kruglov Pavel
2672b71b2d
Delete extra spaces 2020-04-16 23:47:01 +03:00
Avogar
89d787dae3 Add new line at the end of file 2020-04-16 23:14:19 +03:00
Avogar
831c48c514 Add MarkdownRowOutput format 2020-04-16 23:10:42 +03:00
Ivan Lezhankin
e230632645 Changes required for auto-sync with Arcadia 2020-04-16 15:31:57 +03:00
alexey-milovidov
d296e1b6d7
Merge pull request #10216 from Avogar/msgpack_format 
Update MsgPack format
 2020-04-16 00:44:08 +03:00
Avogar
93c322bc30 Fix style error 2020-04-15 02:58:36 +03:00
Avogar
b8061356b0 Update parsing method in MsgPackRowInputFormat 2020-04-15 02:08:55 +03:00
Avogar
e46322fcef Update MsgPack input format. 2020-04-13 22:33:02 +03:00
Avogar
b9931863ef Fix FixedString packing 2020-04-13 00:01:17 +03:00
alexey-milovidov
0782052768
Merge pull request #10131 from ClickHouse/fix-extremes-for-processors 
Fix extremes for processors
 2020-04-12 02:31:39 +03:00
Nikolai Kochetov
458c7f516d Fix extremes for Processors. 2020-04-08 21:50:14 +03:00
Avogar
e635b0e9eb Fix build error 2020-04-04 20:04:41 +03:00
Avogar
fbc11d3bf9 Move files 2020-04-04 17:04:20 +03:00
Ivan Lezhankin
06446b4f08 dbms/ → src/ 2020-04-03 18:14:31 +03:00