Commit Graph

541 Commits

Author SHA1 Message Date
Pavel Kruglov
556ec9e533 Fix segfault in arrow and support Decimal256 2021-06-21 12:06:28 +03:00
Anton Popov
567043113c Merge remote-tracking branch 'upstream/master' into HEAD 2021-06-21 01:36:06 +03:00
mergify[bot]
c723dd7d40
Merge branch 'master' into arrow 2021-06-18 12:18:57 +00:00
Ivan Lezhankin
37365589ed Merge branch 'master' into async-insert 2021-06-17 16:57:45 +03:00
Maksim Kita
67e9b85951 Merge ext into common 2021-06-16 23:28:41 +03:00
Pavel Kruglov
48cd8f2207 Fix build 2021-06-15 16:15:27 +03:00
Pavel Kruglov
692150b578 Fix tests 2021-06-15 16:15:27 +03:00
Pavel Kruglov
931e05ab04 Minor refactoring 2021-06-15 16:15:27 +03:00
Pavel Kruglov
a4ef60e230 Remove Impl including from .h file 2021-06-15 16:15:27 +03:00
Pavel Kruglov
eb86721d14 Fix style, output strings as binary instead of utf8 2021-06-15 16:15:27 +03:00
Pavel Kruglov
787c800064 Fix build, tests, style 2021-06-15 16:15:27 +03:00
Pavel Kruglov
b120841b57 Small changes 2021-06-15 16:15:27 +03:00
Pavel Kruglov
a4decd0848 Support Map type, fix and add tests 2021-06-15 16:15:27 +03:00
Pavel Kruglov
c8b37977da Fix bugs, support dictionary for Arrow format 2021-06-15 16:15:27 +03:00
Pavel Kruglov
540c494492 Fix selecting indexes in ORC and Parquet formats 2021-06-15 16:15:27 +03:00
Kruglov Pavel
d74c5ef62d Minor change 2021-06-15 16:15:27 +03:00
Pavel Kruglov
235e3e2f5b Support structs in Arrow/Parquet/ORC 2021-06-15 16:15:27 +03:00
alexey-milovidov
05d1af153c
Merge branch 'master' into rename-const-context-ptr 2021-06-12 03:25:09 +03:00
Nikita Mikhaylov
395e0d6756 done 2021-06-10 13:57:19 +00:00
Ivan Lezhankin
973cea73b7 Merge branch 'master' into async-insert 2021-06-08 15:10:30 +03:00
Anton Popov
b3b29be02f Merge remote-tracking branch 'upstream/master' into HEAD 2021-06-07 19:03:24 +03:00
mergify[bot]
d68d5e0ef1
Merge branch 'master' into rename-const-context-ptr 2021-06-04 14:41:41 +00:00
Pavel Kruglov
af5d0e2487 Fix possible heap-buffer-overflow in Arrow 2021-06-03 17:11:05 +03:00
Ivan Lezhankin
791cbcf0b1 Merge remote-tracking branch 'upstream/master' into async-insert 2021-06-02 18:50:37 +03:00
Ivan Lezhankin
84c23dc060 Merge branch 'master' into async-insert 2021-06-02 18:06:21 +03:00
Nikolai Kochetov
dbaa6ffc62 Rename ContextConstPtr to ContextPtr. 2021-06-01 15:20:52 +03:00
Anton Popov
018a303387 Merge remote-tracking branch 'upstream/master' into HEAD 2021-05-31 23:08:04 +03:00
Ivan
ce2a809773
Consider last CRLF as part of boundary (#24399)
* Consider last CRLF as part of boundary
* Update HTMLForm.cpp
* Don't use xxd
2021-05-28 14:59:11 +03:00
Anton Popov
3e92c7f61a Merge remote-tracking branch 'upstream/master' into HEAD 2021-05-25 21:45:19 +03:00
Alexey Milovidov
ea30bb7abf Improve implementation 2021-05-21 04:17:18 +03:00
Maksim Kita
196ca14c3a
Merge pull request #24150 from azat/whitespaces
Fix a few trailing whitespaces in output
2021-05-15 22:39:16 +03:00
Azat Khuzhin
1cdeafe473 Remove superfluous trailing whitespace in Markdown format 2021-05-15 16:08:17 +03:00
mergify[bot]
d31021ea4f
Merge branch 'master' into fix-kafka-with-parquet 2021-05-14 21:10:12 +00:00
Anton Popov
d8df0903b9 Merge remote-tracking branch 'upstream/master' into HEAD 2021-05-14 23:38:16 +03:00
Pavel Kruglov
484cac6193 Return include_indices, fix arrays and add more tests 2021-05-14 13:50:10 +03:00
Pavel Kruglov
46a7cc5f1d Remove include_indices 2021-05-14 00:52:29 +03:00
Pavel Kruglov
bbe470a670 Remove fillArrowArray from .h file 2021-05-13 21:15:55 +03:00
Pavel Kruglov
25ceb1df65 Final fixes 2021-05-13 20:00:13 +03:00
mergify[bot]
1b181606e8
Merge branch 'master' into arrow_array 2021-05-13 12:23:48 +00:00
Pavel Kruglov
ff6e4bf91a Remove debug output 2021-05-13 15:21:39 +03:00
Pavel Kruglov
ec214bb04c Try fix tests again 2021-05-13 12:23:59 +03:00
Pavel Kruglov
86f061976e Some fixes 2021-05-12 22:02:56 +03:00
Pavel Kruglov
8ed6ad7c55 Refactor, support all types in array, add nested arrays support, more tests 2021-05-12 19:06:08 +03:00
chou.fan
8990120eb7 add array nullable datatype support in ArrowColumnToCHColumn and fix select from array datatype misplaced bug 2021-05-10 14:17:52 +01:00
Alexey Milovidov
45dea86395 More fixes for PVS-Studio 2021-05-08 18:35:09 +03:00
Chao Ma
ec8ca5bf5e Fix kafka storage does not work with parquet format messages 2021-05-06 11:32:58 +08:00
Alexey Milovidov
02ce9cc725 Untangle UUID 2021-05-04 01:46:51 +03:00
Alexey Milovidov
1fda866881 Merge branch 'master' into normalize-bigint 2021-05-02 21:24:45 +03:00
Benjamin Naecker
57fd59b73b Adds support for building on Solaris-derived systems
This commit adds a number of changes to platform-detection and
compile-time directives required to support building ClickHouse on
Solaris-derived systems, most notably illumos.
2021-04-28 16:32:41 -07:00
Nikita Mikhaylov
175ae42bae better 2021-04-27 01:56:47 +03:00
Nikita Mikhaylov
9a70136bcf better 2021-04-26 18:50:40 +03:00
Alexey Milovidov
8f01af62d9 Merge branch 'master' into normalize-bigint 2021-04-25 06:57:44 +03:00
Ivan Lezhankin
d9626e73c3 Revert "CC"
This reverts commit ee194928d2.
2021-04-22 16:55:18 +03:00
Ivan Lezhankin
ee194928d2 CC 2021-04-21 16:19:28 +03:00
Chao Ma
713eb9486c Fix kafka storage does not work with arrow and arrowstream format messages 2021-04-21 19:44:09 +08:00
Nikita Mikhaylov
16e7715039
Merge pull request #23202 from nikitamikhaylov/with_names_use_header_bug
`CSVWithNames` with `input_format_with_names_use_header=0`
2021-04-20 20:00:57 +03:00
Anton Popov
aea93d9ae5 Merge remote-tracking branch 'upstream/master' into HEAD 2021-04-20 15:16:12 +03:00
Ivan Lezhankin
567fb44b07 Works with trivial inserts 2021-04-19 20:49:02 +03:00
Ivan Lezhankin
7a88ca31cc Merge branch 'master' into async-insert 2021-04-19 18:33:43 +03:00
Chao Ma
6238028225 Fix integration test test_storage_kafka failed error 2021-04-19 01:38:29 +00:00
Chao Ma
8d7a53efda Fix test 00163_column_oriented_formats failed error 2021-04-19 01:38:29 +00:00
Chao Ma
2776772ac6 Read ORC file by stripe to reduce memory cost 2021-04-19 01:38:29 +00:00
Nikita Mikhaylov
7a23d8292f done 2021-04-16 23:53:26 +00:00
Ivan Lezhankin
7f43dddf0b CC 2021-04-16 20:49:38 +03:00
Anton Popov
6ce875175b Merge remote-tracking branch 'upstream/master' into HEAD 2021-04-16 02:08:20 +03:00
Alexey Milovidov
15153e504a Fix unpleasant behaviour of Markdown format 2021-04-15 02:08:43 +03:00
Anton Popov
236ab68724 ColumnSparse: several fixes 2021-04-14 16:20:05 +03:00
Nikolai Kochetov
7019a9a659
Merge pull request #22673 from nvartolomei/nv/exp-arrow-stream
Experiment with ArrowStream streaming
2021-04-12 13:29:25 +03:00
chou.fan
4a6345c210 add array nullable datatype support 2021-04-11 15:05:51 +01:00
Ivan
495c6e03aa
Replace all Context references with std::weak_ptr (#22297)
* Replace all Context references with std::weak_ptr

* Fix shared context captured by value

* Fix build

* Fix Context with named sessions

* Fix copy context

* Fix gcc build

* Merge with master and fix build

* Fix gcc-9 build
2021-04-11 02:33:54 +03:00
Maksim Kita
942399983c DataTypeLowCardinality format tsv parsing issue 2021-04-08 20:12:41 +03:00
Nicolae Vartolomei
7259834fad More Arrow like code style 2021-04-07 23:33:25 +03:00
Nicolae Vartolomei
4ea363006b Experiment with ArrowStream streaming 2021-04-07 23:33:25 +03:00
Nikita Mikhaylov
48af7a898c
Merge pull request #22299 from nikitamikhaylov/keen-wolf-storagefile-column-oriented
Merging #21302
2021-04-07 16:01:12 +03:00
Nikita Mikhailov
37f48d13b4 add test 2021-04-06 22:23:16 +03:00
Azat Khuzhin
f157278b72 Safer SCOPE_EXIT
It executes the code in the dtor, that should never throw.
2021-04-06 21:10:37 +03:00
Anton Popov
d46958a8d2 Merge remote-tracking branch 'upstream/master' into HEAD 2021-04-06 00:54:49 +03:00
chou.fan
111a0c26a0 fix list_nested_type uninitialized compile error 2021-04-02 10:59:32 +01:00
Alexey Milovidov
14f2cf9b80 Add memory accounting in parallel formatting 2021-04-01 06:17:23 +03:00
Anton Popov
372a1b1fe7 Merge remote-tracking branch 'upstream/master' into HEAD 2021-03-29 19:57:49 +03:00
Anton Popov
577d571300 ColumnSparse: initial implementation 2021-03-29 19:54:24 +03:00
Anton Popov
6a15431be7 Merge remote-tracking branch 'upstream/master' into HEAD 2021-03-25 15:57:35 +03:00
Vladimir
c8b5be636f
Merge pull request #21438 from arenadata/ADQM-170 2021-03-19 17:06:55 +03:00
chou.fan
8b5a989564 Merge branch 'master' into arrow_array 2021-03-18 10:12:03 +00:00
chou.fan
99e594e704 parquet data type support array format read and write 2021-03-16 15:29:21 +00:00
Ilya Golshtein
ab670a31dc AvroRowInputFormat::readPrefix inroduced 2021-03-16 17:40:59 +03:00
Anton Popov
6247cd5503 Merge remote-tracking branch 'upstream/master' into HEAD 2021-03-16 16:50:15 +03:00
Nikita Mikhaylov
2e13c767a0
Merge pull request #21149 from nikitamikhaylov/pp-for-csv-with-names
Support parallel parsing for *WithNames types
2021-03-16 14:38:03 +03:00
Nikita Mikhaylov
7d7a2bc724
Merge pull request #21630 from nikitamikhaylov/pf-local
Parallel formatting for clickhouse-local
2021-03-16 14:37:40 +03:00
Nikita Mikhaylov
588f3ee11e better[2] 2021-03-15 14:43:57 +03:00
Ilya Golshtein
abcf581005 avro bug with format_setting reference and a typo in kafka test 2021-03-15 12:19:33 +03:00
Ilya Golshtein
a579fcb424 kafka/avro works against table 2021-03-14 15:55:44 +03:00
Anton Popov
ed42437219 Merge remote-tracking branch 'upstream/master' into HEAD 2021-03-13 01:41:26 +03:00
Nikita Mikhaylov
e4b4665ff5 better 2021-03-12 13:58:03 +03:00
Nikita Mikhaylov
b434a2f0bd disable parallel parsing for grpc tests 2021-03-11 18:22:24 +03:00
Nikita Mikhaylov
761b369fa2 move to poco::event 2021-03-11 18:22:24 +03:00
Nikita Mikhaylov
f3ee129074 fix grpc 2021-03-11 18:22:24 +03:00
Nikita Mikhaylov
e7a1398def try fix tests 2021-03-11 18:22:24 +03:00
Nikita Mikhaylov
3372dd7b6a fix stupid bug 2021-03-11 18:22:24 +03:00
Nikita Mikhailov
3a7d48f3a4 style and tsan fix 2021-03-11 18:22:24 +03:00
Nikita Mikhailov
369c9da161 better 2021-03-11 18:22:24 +03:00
alexey-milovidov
eab35bfc7f Update ParallelParsingInputFormat.h 2021-03-11 18:22:24 +03:00
Nikita Mikhailov
91a418b3a9 better 2021-03-11 18:22:24 +03:00
Nikita Mikhailov
8b30bb0fb5 style 2021-03-11 18:22:24 +03:00
Nikita Mikhailov
61fdeb5897 better 2021-03-11 18:22:24 +03:00
Nikita Mikhailov
138dedf2df done 2021-03-11 18:22:24 +03:00
Nikita Mikhaylov
38e1cb41cc done 2021-03-11 17:04:59 +03:00
Nikita Mikhaylov
5ac5ae3b27
Merge pull request #21157 from CurtizJ/fix-type-map
Improve support of integer keys in data type Map
2021-03-11 15:49:57 +03:00
Anton Popov
bc417cf54a refactoring of serializations 2021-03-09 17:46:52 +03:00
chou.fan
c97e734d93 Merge branch 'master' of github.com:ClickHouse/ClickHouse into arrow_array 2021-03-04 10:17:26 +00:00
Nikolai Kochetov
f82e00ff3c Fix inconsistent column type for Values input format. 2021-03-01 23:10:50 +03:00
Anton Popov
b6a519cd04 Merge remote-tracking branch 'upstream/master' into HEAD 2021-02-24 19:45:38 +03:00
Maksim Kita
ae79fb22be
Merge pull request #21121 from amosbird/enumdefaultfix
disable null_as_default when type is Nullable
2021-02-24 19:31:30 +03:00
Amos Bird
abf26d4935
disable null_as_default when type is Nullable 2021-02-24 14:45:14 +08:00
chou.fan
2c48be3cfa Merge branch 'master' of github.com:ClickHouse/ClickHouse into arrow_array 2021-02-22 23:47:58 +00:00
Maksim Kita
f001a6755e Updated for map 2021-02-22 23:32:43 +03:00
Maksim Kita
3b80395b1b Fixed style check 2021-02-22 23:32:43 +03:00
Maksim Kita
5eec754fd6 Use standard checkStackSize 2021-02-22 23:32:43 +03:00
Maksim Kita
992747e5ec Updated for Array 2021-02-22 23:32:43 +03:00
Maksim Kita
3f6050cee9 Fixed error code include 2021-02-22 23:32:43 +03:00
Maksim Kita
8520fc3e4c Replace null fields in tuple during parsing with default values 2021-02-22 23:32:43 +03:00
Amos Bird
77fd060665
Normalize function names 2021-02-18 20:20:28 +08:00
Vitaly Baranov
3cbb325913
Merge pull request #20506 from vitlibar/refactor-protobuf-format-io
Improved serialization in Protobuf format.
2021-02-18 11:31:37 +03:00
Anton Popov
f1fa110d48 fix type map with integer keys 2021-02-18 01:05:31 +03:00
Vitaly Baranov
18e036d19b Improved serialization for data types combined of Arrays and Tuples.
Improved matching enum data types to protobuf enum type.
Fixed serialization of the Map data type.
Omitted values are now set by default.
2021-02-17 20:50:09 +03:00
Nikita Mikhailov
d615b8e516 more checks
(cherry picked from commit b45168ecaf37d0061edfd12c67a8c5300d45d2e3)
2021-02-15 16:11:16 +03:00
Azat Khuzhin
ca071381e1 Merge remote-tracking branch 'upstream/master' into writer-partial-buffer 2021-02-03 23:46:39 +03:00
Alexey Milovidov
12485eee6b Fix some of the issues found by Coverity 2021-02-02 22:07:23 +03:00
Azat Khuzhin
8b92749e1c Fix ParallelFormattingOutputFormat after allowed partial writes
The problem was that ParallelFormattingOutputFormat creats
BufferWithOutsideMemory with Memory<> whos size is 0, and this create
WriteBuffer with zero size, which will have endless loop.
2021-02-02 01:52:19 +03:00
Alexander Tokmakov
accf4d262e fix logical error 2021-02-01 15:40:57 +03:00
Alexey Milovidov
093108bf81 Normalize BigInt implementation 2021-01-27 03:54:57 +03:00
chou.fan
5e8426d627 Merge branch 'master' of github.com:ClickHouse/ClickHouse into arrow_array 2021-01-26 14:33:17 +00:00
Nikolai Kochetov
76495124cd Fix readign from fd for ORCBlockInputFormat. 2021-01-15 18:45:29 +03:00
chou.fan
c442f89fbd update 2021-01-15 15:07:43 +00:00
Alexey Milovidov
d88d3de0d1 Fix UBSan report: applying non-zero offset to nullptr 2021-01-04 02:31:47 +03:00
Nikita Mikhailov
fa48e062eb fix build 2020-12-30 21:51:41 +03:00
Nikita Mikhailov
d4ec52f732 shellchek 2020-12-30 18:21:58 +03:00
Nikita Mikhailov
b94a654715 build fix 2020-12-30 16:55:31 +03:00
Nikita Mikhailov
60b4a36c4a arcadia fix + live view fix + cleanup 2020-12-30 07:50:58 +03:00
Nikita Mikhailov
c5f92e5096 better formatfactory 2020-12-30 06:07:30 +03:00
Nikita Mikhailov
ffd73082ba fix memory and add comment 2020-12-30 01:34:26 +03:00
Nikita Mikhailov
def9c81779 disable values for parallel_formatting 2020-12-29 17:51:42 +03:00
Nikita Mikhailov
c0eef84c50 better 2020-12-29 04:15:26 +03:00
Nikita Mikhailov
2dde73f700 better 2020-12-28 19:52:54 +03:00
Nikita Mikhailov
964e12d8b7 Fix 2020-12-28 15:53:58 +03:00
Nikita Mikhailov
c3288c3fbf Merge branch 'master' of github.com:ClickHouse/ClickHouse into parallel-parsing-input-format 2020-12-28 15:09:37 +03:00
Nikita Mikhailov
dcfbe782c6 Merge branch 'master' of github.com:ClickHouse/ClickHouse into parallel-parsing-input-format 2020-12-23 05:20:22 +03:00
nikitamikhaylov
a285cb83df style 2020-12-23 01:02:01 +03:00
nikitamikhaylov
c60c161168 add ParsingException 2020-12-23 01:02:01 +03:00
nikitamikhaylov
27f647f93d done 2020-12-23 01:01:05 +03:00