thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-12 17:32:32 +00:00
Code Issues Packages Projects Releases Wiki Activity
131,051 Commits 9,328 Branches 1,783 Tags 2.1 GiB
e09c68db32
Commit Graph

1939 Commits

Author SHA1 Message Date
Kruglov Pavel
0d34e97dbe
Merge branch 'master' into formats-with-subcolumns 2023-07-26 13:30:35 +02:00
Kruglov Pavel
15cc046883
Merge branch 'master' into better-progress-bar-2 2023-07-26 13:12:24 +02:00
Michael Kolupaev
5ee71bd643 Work around the clang bug 2023-07-25 10:26:26 +00:00
Michael Kolupaev
dfdf5de972 Fixes 2023-07-25 10:16:28 +00:00
Michael Kolupaev
db5cb96050 Start over when falling back to non-dictionary encoding 2023-07-25 10:16:28 +00:00
Michael Kolupaev
8184a289e5 Partially reimplement Parquet encoder to make it faster and parallelizable 2023-07-25 10:16:28 +00:00
alexX512
1fe2f052e3 Merge branch 'master' of github.com:ClickHouse/ClickHouse 2023-07-25 09:36:44 +00:00
Alexey Milovidov
21382afa2b Check for punctuation 2023-07-25 06:10:04 +02:00
Alexey Milovidov
ecdafeaf83
Merge pull request #52175 from bigo-sg/comment_improve_ch_to_arrow 
Add comments for https://github.com/ClickHouse/ClickHouse/pull/52112
 2023-07-25 06:47:48 +03:00
Kruglov Pavel
fec5675cd4
Merge branch 'master' into better-progress-bar-2 2023-07-24 19:59:38 +02:00
李扬
e8fe485330
Merge branch 'master' into comment_improve_ch_to_arrow 2023-07-22 11:43:32 +08:00
Alexey Milovidov
9ae685975e
Merge branch 'master' into avro-fix 2023-07-22 04:56:48 +03:00
alexX512
c403f56e09 Merge branch 'master' of github.com:ClickHouse/ClickHouse 2023-07-21 17:56:53 +00:00
Alexander Tokmakov
b45c2c939b
disable expression templates for time intervals (#52335) 2023-07-21 15:17:07 +03:00
ltrk2
90a2c460c6
Merge branch 'master' into feature/mergetree-checksum-big-endian-support 2023-07-21 08:07:18 -04:00
Kruglov Pavel
342400d0b3
Merge branch 'master' into revert-52322-revert-51716-bug_fix_csv_field_type_not_match 2023-07-20 12:39:38 +02:00
Nikolay Degterinsky
209429d0e3
Merge pull request #49664 from ilejn/test_for_basic_auth_registry 
Basic auth to fetch Avro schema in Kafka
 2023-07-20 10:58:11 +02:00
李扬
68bf4c3590
Merge branch 'master' into comment_improve_ch_to_arrow 2023-07-20 10:10:47 +08:00
ltrk2
a753c3c6ad
Merge branch 'master' into feature/mergetree-checksum-big-endian-support 2023-07-19 16:22:58 -04:00
Kruglov Pavel
0fca64ced4
Merge pull request #51695 from Avogar/row-binary-with-defaults 
Add RowBinaryWithDefaults format
 2023-07-19 22:10:30 +02:00
ltrk2
ba4072f049 Adapt changes around SipHash 2023-07-19 10:01:58 -07:00
ltrk2
51e2c58a53 Implement endianness-independent SipHash and MergeTree checksum serialization 2023-07-19 10:01:55 -07:00
Kruglov Pavel
f0026af189
Revert "Revert "Improve CSVInputFormat to check and set default value to column if deserialize failed"" 2023-07-19 14:51:11 +02:00
Kruglov Pavel
7b3564f96a
Revert "Improve CSVInputFormat to check and set default value to column if deserialize failed" 2023-07-19 14:44:59 +02:00
robot-ch-test-poll4
63d0616a22
Merge pull request #51716 from KevinyhZou/bug_fix_csv_field_type_not_match 
Improve CSVInputFormat to check and set default value to column if deserialize failed
 2023-07-19 14:41:05 +02:00
kevinyhzou
dcf7ba2534 remove unuseful code 2023-07-19 19:36:19 +08:00
kevinyhzou
95424177d5 review fix 2023-07-19 18:26:54 +08:00
Ilya Golshtein
c1c5ffa309 test_for_basic_auth_registry - cpp code small improvement 2023-07-19 08:32:45 +00:00
dheerajathrey
8e1de7897a indentation fix 2023-07-19 08:32:44 +00:00
dheerajathrey
1564eace38 enable url-encoded basic auth to fetch avro schema in kafka 2023-07-19 08:32:44 +00:00
Alexey Milovidov
0789f388c3
Update ArrowFieldIndexUtil.h 2023-07-19 02:45:56 +03:00
Alexey Milovidov
6d915042a2 Fix ugly code 2023-07-19 01:44:20 +02:00
avogar
67f340b501 Merge branch 'master' of github.com:ClickHouse/ClickHouse into structure-to-schema 2023-07-18 13:52:15 +00:00
Kruglov Pavel
64e88cde21
Merge branch 'master' into better-progress-bar-2 2023-07-18 13:37:53 +02:00
Kruglov Pavel
1e616e17ab
Merge branch 'master' into row-binary-with-defaults 2023-07-17 19:13:57 +02:00
Kruglov Pavel
1dd05319b5
Merge branch 'master' into formats-with-subcolumns 2023-07-17 19:13:42 +02:00
kevinyhzou
355faa4251 ci fix 2023-07-17 20:08:32 +08:00
flynn
d6709ded53
Merge branch 'master' into avro-fix 2023-07-17 14:51:34 +08:00
taiyang-li
8ea335aca7 update style 2023-07-17 10:43:13 +08:00
taiyang-li
7716479a37 add comments for https://github.com/ClickHouse/ClickHouse/pull/52112 2023-07-17 10:33:38 +08:00
flynn
386adfad33 Avro input format support Union with single type 2023-07-15 16:21:58 +00:00
taiyang-li
8ea3bf4ade improve ch to arrow 2023-07-14 16:09:22 +08:00
kevinyhzou
c6b8097090 rebase main 2023-07-14 11:24:38 +08:00
kevinyhzou
b2665031dc review fix 2023-07-13 20:27:14 +08:00
kevinyhzou
ba57c84db3 bug fix csv input field type mismatch 2023-07-13 20:24:10 +08:00
Dmitry Kardymon
385a210fee Merge remote-tracking branch 'origin/master' into ADQM-870 2023-07-10 13:19:21 +00:00
Alexey Milovidov
3d4800995f
Merge pull request #49732 from nickitat/impr_prefetch 
Improve reading with prefetch
 2023-07-09 06:10:58 +03:00
Kruglov Pavel
06de25451a
Merge branch 'master' into formats-with-subcolumns 2023-07-06 16:21:52 +02:00
avogar
810d1ee069 Fix tests 2023-07-06 13:48:57 +00:00
Nikita Taranov
aec7205636 rework pool usage 2023-07-06 14:41:09 +02:00
Dmitry Kardymon
86fc702236
Add skipWhitespacesAndTabs() 
Co-authored-by: Kruglov Pavel <48961922+Avogar@users.noreply.github.com>
 2023-07-06 15:14:18 +03:00
Dmitry Kardymon
32f5a78302 Fix setting name 2023-07-06 07:32:46 +00:00
Dmitry Kardymon
24b5c9c204 Use one setting input_format_csv_allow_variable_number_of_colums and code in RowInput 2023-07-06 06:05:43 +00:00
avogar
d11cd0dc30 Fix tests 2023-07-05 17:56:03 +00:00
Dmitry Kardymon
86014a60a3 Fixed case with spaces before delimiter 2023-07-05 11:42:02 +00:00
avogar
98aa6b317f Support reading subcolumns from file/s3/hdfs/url/azureBlobStorage table functions 2023-07-04 21:17:26 +00:00
Robert Schulze
fe49e98455
Follow-up to re2 update 2023-06-02 (#50949) 2023-07-03 08:28:25 +00:00
avogar
34bf0284ad Add RowBinaryWithDefaults format 2023-06-30 16:18:30 +00:00
avogar
03f820bc4a Merge branch 'master' of github.com:ClickHouse/ClickHouse into structure-to-schema 2023-06-22 18:46:01 +00:00
avogar
4060beae49 Structure to CapnProto/Protobuf schema take 1 2023-06-22 18:00:00 +00:00
Dmitry Kardymon
19d0214ac1 Merge remote-tracking branch 'origin/master' into ADQM-870 2023-06-22 13:02:31 +00:00
Dmitry Kardymon
a0fde6a55b Style fix 2023-06-22 10:50:14 +00:00
Dmitry Kardymon
2c3a4cb90d Style fix 2023-06-22 10:47:07 +00:00
Sema Checherinda
01de36f1fa
Merge pull request #50395 from CheSema/better-log 
require `finalize()` call before d-tor for all writes buffers
 2023-06-21 21:12:02 +02:00
Dmitry Kardymon
fff0c8da92 Merge remote-tracking branch 'origin/master' into ADQM-870 2023-06-21 10:56:50 +00:00
Kruglov Pavel
8f8cd97fd8
Merge pull request #51088 from Avogar/better-progress-bar 
Improve progress bar for file/s3/hdfs/url table functions. Step 1
 2023-06-21 12:42:25 +02:00
Sema Checherinda
9b0c3359cf
Merge branch 'master' into better-log 2023-06-20 20:37:36 +02:00
Sema Checherinda
fd292dc730 work with comment on the PR 2023-06-20 20:02:04 +02:00
Kruglov Pavel
0edfbb45ad
Merge pull request #50873 from Avogar/parquet-big-integers 
Fallback to parsing big integer from String instead of exception in Parquet format
 2023-06-20 16:10:46 +02:00
avogar
d492acbcd2 Fix tests 2023-06-19 13:36:29 +00:00
Dmitry Kardymon
f81401db99 Add empty line test 2023-06-19 10:48:38 +00:00
Dmitry Kardymon
dd43a186ad Minor edit docs / add int256 test 2023-06-19 09:51:29 +00:00
Dmitry Kardymon
30bea857fd Merge remote-tracking branch 'origin/master' into ADQM-870 2023-06-19 07:19:07 +00:00
avogar
3209ebe34b Improve progress bar for file/s3/hdfs/url table functions. Step 1 2023-06-16 15:51:18 +00:00
Sema Checherinda
1cb02e2710 do call finalize for all buffers 2023-06-16 16:38:18 +02:00
Dmitry Kardymon
0eeee11dc4 Style fix, add comment 2023-06-15 12:36:18 +00:00
Dmitry Kardymon
806176d88e Add input_format_csv_missing_as_default setting and tests 2023-06-15 11:23:08 +00:00
KevinyhZou
953f40aa3b
Merge branch 'master' into bug_fix_csv_parse_by_tab_delimiter 2023-06-15 10:25:19 +08:00
Dmitry Kardymon
a91fc3ddb3 Add docs/ add more cases in test 2023-06-14 16:44:31 +00:00
Dmitry Kardymon
ed318d1035 Add input_format_csv_ignore_extra_columns setting (prototype) 2023-06-14 10:35:36 +00:00
kevinyhzou
f3b99156ac review fix 2023-06-14 10:48:21 +08:00
Kruglov Pavel
607f337d67
Merge pull request #50592 from Avogar/max-bytes-to-read-in-schema-inference 
Add setting to limit the number of bytes to read in schema inference
 2023-06-13 16:47:57 +02:00
Kruglov Pavel
8fdcd91c38
Merge pull request #49752 from Avogar/better-capnproto-3 
Refactor CapnProto format to improve input/output performance
 2023-06-13 16:20:38 +02:00
Kruglov Pavel
edd47a2281
Merge branch 'master' into skip-trailing-empty-lines 2023-06-12 13:57:15 +02:00
Kruglov Pavel
e03cd725b0
Merge pull request #50602 from Avogar/null-as-default-schema-inference 
Respect setting input_format_as_default in schema inference
 2023-06-12 13:45:52 +02:00
Kruglov Pavel
da68980b8d
Merge branch 'master' into max-bytes-to-read-in-schema-inference 2023-06-12 13:45:31 +02:00
avogar
5cec4c3161 Fallback to parsing big integer from String instead of exception in Parquet format 2023-06-12 11:34:40 +00:00
kevinyhzou
911f8ad8dc use whitespace or tab as field delimiter 2023-06-12 11:57:52 +08:00
Hongbin Ma
41c34aaf5e optimize parquet write performance for parallel threads 
fix CI

fix review comments and CI
 2023-06-09 19:09:58 -07:00
kevinyhzou
48e1b21aab Add feature to support read csv by space & tab delimiter 2023-06-08 20:34:30 +08:00
avogar
cc036528fe Merge branch 'master' of github.com:ClickHouse/ClickHouse into better-capnproto-3 2023-06-08 11:16:13 +00:00
Kruglov Pavel
a714c1662e
Merge branch 'master' into max-bytes-to-read-in-schema-inference 2023-06-08 12:55:31 +02:00
Kruglov Pavel
4727c85e1f
Merge branch 'master' into null-as-default-schema-inference 2023-06-08 12:54:18 +02:00
Kruglov Pavel
dc24599525
Merge branch 'master' into skip-trailing-empty-lines 2023-06-08 12:39:23 +02:00
avogar
cf947e6e01 Fix typo 2023-06-07 12:50:16 +00:00
avogar
87ac6b8b63 Fix reading negative decimals in avro format 2023-06-07 12:49:28 +00:00
Kruglov Pavel
5af1819143
Merge pull request #50586 from Avogar/better-avro-decimal 
Better support for avro decimals
 2023-06-06 19:40:59 +02:00
Kruglov Pavel
1baa6404e6
Merge branch 'master' into skip-trailing-empty-lines 2023-06-06 19:39:34 +02:00
avogar
df50833b70 Allow to skip trailing empty lines in CSV/TSV/CustomeSeparated formats 2023-06-06 17:33:05 +00:00
Kruglov Pavel
af880a6f3b
Merge branch 'master' into max-bytes-to-read-in-schema-inference 2023-06-06 14:47:58 +02:00
Robert Schulze
2e16b497f5
Merge pull request #50519 from ClibMouse/feature/uuid-serialization 
Implement endianness-independent serialization for UUID
 2023-06-06 09:18:19 +02:00
avogar
67af505ed6 Respect setting input_format_as_default in schema inference 2023-06-05 17:04:55 +00:00
avogar
33e51d4f3b Add setting to limit the number of bytes to read in schema inference 2023-06-05 15:22:04 +00:00
ltrk2
3938309374 Implement review comments 2023-06-05 08:18:03 -07:00
avogar
aa20935cb9 Better 2023-06-05 12:45:14 +00:00
avogar
55345d5a25 Fix exception message 2023-06-05 12:43:38 +00:00
avogar
4f0adf5f61 Better support for avro decimals 2023-06-05 12:40:54 +00:00
Alexey Gerasimchuk
9958731c27
Merge branch 'master' into ADQM-830 2023-06-05 07:46:47 +10:00
ltrk2
50654435dc Implement endianness-independent serialization for UUID 2023-06-02 19:36:37 +00:00
Michael Kolupaev
b51064a508 Get rid of SeekableReadBufferFactory, add SeekableReadBuffer::readBigAt() instead 2023-06-01 18:48:30 -07:00
Kruglov Pavel
898d1f34db
Merge branch 'master' into better-capnproto-3 2023-05-31 21:44:00 +02:00
Alexey Gerasimchuk
44ba35d2c1
Merge branch 'master' into ADQM-830 2023-05-31 15:07:37 +10:00
Michael Kolupaev
536c4a99c8 Fix clickhouse-local crashing when writing empty Arrow or Parquet output 2023-05-30 10:45:49 -07:00
Alexey Milovidov
1875a93328
Merge pull request #50224 from Avogar/fix-custom-separated-ignore-spaces 
Fix skipping spaces at end of row in CustomSeparatedIgnoreSpaces format
 2023-05-29 02:42:38 +03:00
Alexey Gerasimchuck
01f3a46cf0 fixed wrong case in removeNullable 2023-05-25 22:49:36 +00:00
Alexey Gerasimchuk
613568423d
Update src/Processors/Formats/Impl/CSVRowInputFormat.cpp 
Co-authored-by: Kruglov Pavel <48961922+Avogar@users.noreply.github.com>
 2023-05-26 07:49:45 +10:00
avogar
ce99825200 Fix skipping spaces at end of row in CustomSeparatedIgnoreSpaces format 2023-05-25 11:19:15 +00:00
Alexey Gerasimchuck
75791d7a63 Added input_format_csv_trim_whitespaces parameter 2023-05-25 07:51:32 +00:00
Kruglov Pavel
1347dc4ede Fix style 2023-05-24 17:19:04 +00:00
Kruglov Pavel
cc7cfa050f Fix style 2023-05-24 17:19:04 +00:00
avogar
e66f6272d1 Refactor CapnProto format to improve input/output performance 2023-05-24 17:19:04 +00:00
avogar
bf19765c9b Fix possible use-of-uninitialized-value 2023-05-22 19:34:19 +00:00
Michael Kolupaev
6fd5d8e8ba Add setting output_format_parquet_compliant_nested_types to produce more compatible Parquet files 2023-05-19 18:39:50 +00:00
Kruglov Pavel
558eda4146
Merge pull request #49412 from azat/block-use-dense-hash-map 
Switch Block::NameMap to google::dense_hash_map over HashMap
 2023-05-15 12:22:55 +02:00
Alexey Milovidov
f6144ee32b
Revert "Make Pretty formats even prettier." 2023-05-13 02:45:07 +03:00
Azat Khuzhin
2c40dd6a4c Switch Block::NameMap to google::dense_hash_map over HashMap 
Since HashMap creates 2^8 elements by default, while dense_hash_map
should be good here.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-05-12 05:52:57 +02:00
Robert Schulze
9db78792d0
Fix MsgPackRowInputFormat.cpp build 2023-05-11 10:00:32 +00:00
Azat Khuzhin
d8dd50a9c6 Fix misc-misplaced-const clang-tidy warning 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-05-09 21:27:21 +02:00
Azat Khuzhin
d03ae2abfa Fix modernize-loop-convert clang-tidy warning 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-05-09 21:19:37 +02:00
Azat Khuzhin
833652b7c9 Revert "Suppress clang-analyzer-cplusplus.NewDelete in MsgPackRowInputFormat" 
Let's try to revert this quirk during upgrading to clang 16.

This reverts commit c1e70169d2.
 2023-05-09 20:36:39 +02:00
alexX512
f3f9e29509 Merge branch 'master' of github.com:ClickHouse/ClickHouse 2023-05-06 14:35:29 +00:00
Alexey Milovidov
90b0de5677 Make Pretty prettier 2023-05-05 06:36:53 +02:00
Alexey Milovidov
179eddee01 Remove garbage from Pretty format 2023-05-05 04:44:47 +02:00
Michael Kolupaev
eb3b774ad0 Better control over Parquet row group size 2023-05-04 14:59:55 -07:00
taiyang-li
5c67c5fd26 add tests for hive null_as_default 2023-05-04 14:40:22 +08:00
alexX512
2cd1b37f9b Add support for connection of partial result processors with multiple ports 2023-05-02 07:26:36 +00:00
alexX512
e94285d468 Send partial result with the new architectuin the special pipeline 2023-05-01 09:25:15 +00:00
Raúl Marín
f0e045bb3d Merge remote-tracking branch 'blessed/master' into arenita 2023-04-24 10:42:56 +02:00
Alexey Milovidov
b08f6b9dcc
Update LineAsStringRowInputFormat.cpp 2023-04-23 08:32:58 +03:00
Alexey Milovidov
54d10f87f2 Consistency of the LineAsString format 2023-04-23 05:50:46 +02:00
Alexander Gololobov
e6d34a9f8b
Merge pull request #48987 from Avogar/avoid-logical-error 
Don't throw logical error when column is not found in Parquet/Arrow schema
 2023-04-21 09:46:16 +02:00
robot-ch-test-poll1
f466c89621
Merge pull request #48911 from Avogar/parquet-metadata-format 
Add ParquetMetadata input format to read Parquet file metadata
 2023-04-21 03:46:26 +02:00
avogar
8a3e813ecd Don't throw logical error when column is not found in Parquet/Arrow schema 2023-04-20 19:09:40 +00:00
Kruglov Pavel
9bc95bed85
Merge pull request #48898 from Avogar/pretty-json 
Add PrettyJSONEachRow format to output pretty JSON
 2023-04-19 12:27:24 +02:00
Kruglov Pavel
8053b18c05
Merge pull request #48361 from Avogar/fix-arrow-dict-2 
Fix serializing LowCardinality as Arrow dictionary
 2023-04-19 12:23:27 +02:00
Kruglov Pavel
21dddf8c4c
Merge pull request #48864 from Avogar/fix-parquet-date32 
Fix reading Date32 Parquet/Arrow column into not Date32 column
 2023-04-19 09:16:42 +02:00
avogar
0878ab8443 Fix build 2023-04-18 19:51:53 +00:00
Kruglov Pavel
a5c52d3bc3
Merge branch 'master' into parquet-metadata-format 2023-04-18 21:51:14 +02:00
avogar
7a67951f64 Add more fields, fix style 2023-04-18 17:59:01 +00:00
avogar
b0e5f7069e Update exception message 2023-04-18 17:15:16 +00:00
avogar
c5efa4dc01 Add comment 2023-04-18 17:10:37 +00:00
avogar
f7f609dfb9 Better 2023-04-18 16:57:55 +00:00
avogar
b277a5c943 Add ParquetMetadata input format to read Parquet file metadata 2023-04-18 16:46:26 +00:00
Kruglov Pavel
8710c15c85
Apply suggestion 2023-04-18 18:25:54 +02:00
avogar
e356f92b77 Add PrettyJSONEachRow format to output pretty JSON 2023-04-18 13:28:59 +00:00
Kruglov Pavel
3bbc347901
Fix build 2023-04-17 22:22:26 +02:00
Kruglov Pavel
be0b0e7921
Fix build 2023-04-17 20:58:19 +02:00
avogar
527572e7bd Fix reading Date32 Parquet/Arrow column into not Date32 column 2023-04-17 16:51:22 +00:00
Kruglov Pavel
5c9b404c6e
Update src/Processors/Formats/Impl/CHColumnToArrowColumn.cpp 
Co-authored-by: Yakov Olkhovskiy <99031427+yakov-olkhovskiy@users.noreply.github.com>
 2023-04-17 14:02:07 +02:00
Raúl Marín
39f8c43a60 Merge remote-tracking branch 'blessed/master' into arenita 2023-04-17 10:33:38 +02:00
Michael Kolupaev
87be78e6de Better 2023-04-17 04:58:32 +00:00
Michael Kolupaev
e133633359 Parallel decoding with one row group per thread 2023-04-17 04:58:32 +00:00
Michael Kolupaev
2d4fe85513 Something 2023-04-17 04:58:32 +00:00
Michael Kolupaev
dc6e34075e Read less unnecessary data from Parquet files 2023-04-17 04:58:32 +00:00
Dmitry Novik
5cc9b46f78 Merge remote-tracking branch 'origin/master' into optimize-compilation 2023-04-13 16:04:09 +02:00
Raúl Marín
da9a539cf7 Reduce the usage of Arena.h 2023-04-13 10:31:32 +02:00
Robert Schulze
3f7ce60e03
Merge branch 'master' into rs/gcc-removal 2023-04-12 22:17:04 +02:00
Robert Schulze
05606a8835
Clean up GCC warning pragmas 2023-04-11 18:21:08 +00:00
Kruglov Pavel
0b17ac318f
Merge branch 'master' into fix-arrow-dict-2 2023-04-11 14:32:27 +02:00
alexX512
a953a1f8c3 Merge branch 'master' of github.com:ClickHouse/ClickHouse 2023-04-10 13:57:01 +00:00
alexX512
af66d44106 Add settings for real-time updates during query execution 2023-04-10 13:51:39 +00:00
Azat Khuzhin
5b2b20a0b0 Rename ThreadGroupStatus to ThreadGroup 
There are methods like getThreadGroup() and ThreadGroupSwitcher class,
so seems that this is logical.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-04-07 15:31:48 +02:00
Kruglov Pavel
fb33d6bedb
Merge pull request #48126 from Avogar/orc-arrow-parquet-more-types 
Support more ClickHouse types in ORC/Arrow/Parquet formats
 2023-04-06 13:32:46 +02:00
Kruglov Pavel
11d3d01e50
Fix special build 2023-04-05 18:29:59 +02:00
Kruglov Pavel
9331c6c260
Merge pull request #48122 from Avogar/bson-more-types 
Support Enum output/input in BSONEachRow, allow all map key types and avoid extra calculations
 2023-04-05 18:26:19 +02:00
avogar
9d3790a4c8 Fix tests 2023-04-05 11:27:38 +00:00
Kruglov Pavel
031ab94636
Merge branch 'master' into fix-arrow-dict-2 2023-04-05 11:57:57 +02:00
Kruglov Pavel
8d6b709f1a
Merge pull request #48257 from Avogar/capnp-more-types 
Support more types in CapnProto format, allow integer conversions
 2023-04-05 11:56:46 +02:00
Kruglov Pavel
fad053c4fe
Make better 2023-04-04 16:10:47 +02:00
Kruglov Pavel
a3ef50c2c0
Update src/Processors/Formats/Impl/ArrowColumnToCHColumn.cpp 2023-04-04 16:06:21 +02:00
avogar
4894f47d95 Fix tests 2023-04-04 13:34:02 +00:00
Yakov Olkhovskiy
e1e508f8ec
Merge branch 'master' into bson-more-types 2023-04-03 17:21:50 -04:00
avogar
b2e52f80cd Fix serializing LowCardinality as Arrow dictionary 2023-04-03 19:36:43 +00:00
Kruglov Pavel
e2c3c3accc
Merge branch 'master' into orc-arrow-parquet-more-types 2023-04-03 18:33:20 +02:00
avogar
a7efb4a247 Fix tests, update working with column indexes in ORC format 2023-04-03 14:02:45 +00:00
avogar
6a66c3bc23 Fix tests 2023-03-31 18:37:12 +00:00
avogar
70fb4ad0b8 Fix build 2023-03-31 15:56:54 +00:00
Sergei Trifonov
01ed4ffca4
Merge pull request #47870 from ClickHouse/segmentator-fix 
Fix CPU usage counters update logic
 2023-03-30 22:55:58 +02:00
Kruglov Pavel
a3908b54c1
Merge pull request #48124 from Avogar/msgpack-more-types 
Support more ClickHouse types in MsgPack format
 2023-03-30 22:03:47 +02:00
Kruglov Pavel
d096da0cfd
Merge branch 'master' into bson-more-types 2023-03-30 21:18:10 +02:00
avogar
35937adcaa Support more types in CapnProto format 2023-03-30 19:15:28 +00:00
Alexey Milovidov
5ca488d70e
Update src/Processors/Formats/Impl/MsgPackRowInputFormat.cpp 
Co-authored-by: Antonio Andelic <antonio2368@users.noreply.github.com>
 2023-03-30 09:14:52 +03:00
Alexey Milovidov
e982fb9f1c
Merge pull request #47880 from azat/threadpool-introspection 
ThreadPool metrics introspection
 2023-03-30 01:27:31 +03:00
Alexey Milovidov
c00a2fc851
Merge pull request #48184 from Avogar/fix-avro 
Fix possible member call on null pointer in Avro format
 2023-03-30 01:19:44 +03:00
Alexey Milovidov
13761fe891
Merge pull request #48133 from ClickHouse/allow-to-skip-errors-of-enums 
Allow skipping errors related to unknown enum values in row input formats
 2023-03-29 23:20:05 +03:00
avogar
db83dd4e52 Fix possible member call on null pointer in Avro format 2023-03-29 17:33:22 +00:00
robot-ch-test-poll
04d30a2afb
Merge pull request #48119 from Avogar/avro-more-types 
Support more ClickHouse types in Avro format
 2023-03-29 13:07:35 +02:00
Azat Khuzhin
f38a7aeabe ThreadPool metrics introspection 
There are lots of thread pools and simple local-vs-global is not enough
already, it is good to know which one in particular uses threads.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-03-29 10:46:59 +02:00
Sergei Trifonov
1ff6519c25
Merge branch 'master' into segmentator-fix 2023-03-29 02:29:43 +02:00
Alexey Milovidov
cfc4e40811 Allow skipping errors related to unknown enum values in row input formats 2023-03-29 00:31:53 +02:00
avogar
8740b201f1 Fix 2023-03-28 20:07:59 +00:00
avogar
bc56c02858 Support more ClickHouse types in ORC/Arrow/Parquet formats 2023-03-28 20:02:42 +00:00
avogar
2321530bb0 Fix 2023-03-28 18:50:30 +00:00
Kruglov Pavel
3fb1a10a16
Fix style 2023-03-28 20:49:10 +02:00
avogar
a0fcf81abf Support more ClickHouse types in MsgPack format 2023-03-28 18:25:52 +00:00
avogar
04770cd444 Better 2023-03-28 17:58:29 +00:00
avogar
e7ff6e85c2 Support Enum output/input in BSONEachRow, allow all map key types and avoid extra calculations 2023-03-28 17:57:23 +00:00
avogar
06a6f5cf76 Support more ClickHouse types in Avro format 2023-03-28 17:36:20 +00:00
Kruglov Pavel
f3c594a160
Merge pull request #48057 from Avogar/fix-avro-records 
Use uniq names for Records in Avro to avoid reusing its schema
 2023-03-28 19:12:41 +02:00
avogar
61d41e8152 Use uniq names for Records in Avro to avoid reusing its schema 2023-03-27 14:41:36 +00:00
Nikolay Degterinsky
3218dce174
Merge pull request #46659 from ClibMouse/tiny-improvements 
Make tiny improvements
 2023-03-24 23:27:25 +01:00
Sergei Trifonov
a05aa5c1c6
Merge branch 'master' into segmentator-fix 2023-03-24 16:19:57 +01:00
Dmitry Novik
3488a4d84b Improve file includes 2023-03-24 03:44:52 +01:00
ltrk2
3bf2411be1 Make tiny improvements 2023-03-23 10:59:07 -04:00
Amos Bird
02c5d1f364
Correct exact_rows_before_limit in all senarios 2023-03-22 23:26:31 +08:00
serxa
f3c12b8543 fix per-thread perf counters update 2023-03-22 13:51:07 +00:00
serxa
7504e107ac Fix CPU usage counters in segmentator thread 2023-03-21 22:11:19 +00:00
Alexey Milovidov
2a077f11f6 Merge branch 'master' into fuzzer-of-data-formats 2023-03-19 01:07:31 +01:00
Alexey Milovidov
5f5e6d28c7
Merge pull request #47690 from Avogar/bson-fix-3 
Fix bug in tuple as array serialization in BSONEachRow format
 2023-03-18 22:18:40 +03:00
Alexey Milovidov
93a3962f45
Merge pull request #47609 from Avogar/fix-bson-2 
Check for duplicate column in BSONEachRow format
 2023-03-18 22:17:27 +03:00
Alexey Milovidov
26c17b61fb
Merge pull request #47434 from Avogar/avro-improvements 
Support Decimals and Date32 in Avro format
 2023-03-18 22:16:34 +03:00
Sema Checherinda
0b641fcead
Merge pull request #47564 from CheSema/remove-redundant 
remove counters for threads, fix negative counters
 2023-03-18 02:23:52 +01:00
Han Fei
722e6bb2e8
Merge pull request #47680 from ClickHouse/rs/uncruftify-build 
CMake: Remove further cruft from build
 2023-03-17 22:58:46 +01:00
Alexey Milovidov
c11915d35b Merge branch 'fix-bson-2' of github.com:Avogar/ClickHouse into fuzzer-of-data-formats 2023-03-17 22:05:46 +01:00
Kruglov Pavel
d72bf0af71
Fix style 2023-03-17 19:35:20 +01:00
avogar
facb978830 Fix bug in tuple as array serialization in BSONEachRow format 2023-03-17 17:59:29 +00:00
Kruglov Pavel
c376712007
Fix 2023-03-17 16:56:24 +01:00
Kruglov Pavel
5d0d7695aa
Fix 2023-03-17 16:55:13 +01:00
Robert Schulze
f72a337074
Remove cruft from build 
No need to check compiler flags, clang >= 15 supports all of them.
 2023-03-17 13:44:04 +00:00
Kruglov Pavel
4522b74fc6 Fix typo 2023-03-17 13:05:41 +00:00
avogar
f8dab850a9 Check for duplicate column in BSONEachRow format 2023-03-17 13:05:41 +00:00
Alexander Tokmakov
7719acad72
Revert "Fix BSONEachRow parallel parsing when document size is invalid" 2023-03-17 14:00:19 +03:00
Alexey Milovidov
651aba0d1b Slightly better 2023-03-17 00:19:49 +01:00
Alexey Milovidov
31b46df85c Fix an error 2023-03-17 00:13:52 +01:00
Sema Checherinda
3c6deddd1d work with comments on PR 2023-03-16 19:55:58 +01:00
Kruglov Pavel
ffa8af2861
Merge pull request #47540 from Avogar/fix-bson-bug 
Fix BSONEachRow parallel parsing when document size is invalid
 2023-03-16 16:25:25 +01:00
Alexey Milovidov
92a5b81222 Merge branch 'master' of github.com:ClickHouse/ClickHouse into fuzzer-of-data-formats 2023-03-15 23:59:16 +01:00
avogar
c4ac4c596f Fix comments 2023-03-15 13:17:13 +00:00
Alexey Milovidov
bb6b775884 Merge branch 'master' into fuzzer-of-data-formats 2023-03-15 12:42:00 +01:00
Alexey Milovidov
0631381161 Merge branch 'fix-bson-2' of github.com:Avogar/ClickHouse into fuzzer-of-data-formats 2023-03-15 12:41:54 +01:00
avogar
26e15b288b Check for duplicate column in BSONEachRow format 2023-03-15 11:37:25 +00:00
Robert Schulze
5cee479f0c
Merge pull request #47424 from ClickHouse/rs/compile-with-cpp23 
Compile with C++23
 2023-03-15 10:55:42 +01:00
avogar
637d8d0f09 Fix BSONEachRow parallel parsing when document size is invalid 2023-03-13 18:52:29 +00:00
avogar
278c76f85a Fix 2023-03-13 18:23:51 +00:00
avogar
4213ec609f Proper fix for bug in parquet, revert reverted #45878 2023-03-13 18:22:09 +00:00
avogar
5a18acde90 Revert #45878 and add a test 2023-03-11 21:15:14 +00:00
Alexey Milovidov
6ba54ab421
Merge branch 'master' into rs/compile-with-cpp23 2023-03-11 06:33:14 +03:00
Robert Schulze
348a40aa5d
Compile with C++23 2023-03-10 19:11:58 +00:00
avogar
b6c91d9ace Support Decimals and Date32 in Avro format 2023-03-10 15:48:18 +00:00
Kruglov Pavel
fe973f3d6f
Merge branch 'master' into native-types-conversions 2023-03-09 13:03:25 +01:00
Kruglov Pavel
71b6d6c6ae
Merge pull request #47114 from Avogar/parquet-compression 
Improve working with compression methods in Parquet/ORC/Arrow formats
 2023-03-09 13:02:18 +01:00
avogar
79ef95bc0c Fix tests and builds 2023-03-08 12:54:22 +00:00
Kruglov Pavel
69a1309ade
Merge branch 'master' into native-types-conversions 2023-03-07 20:06:17 +01:00
Kruglov Pavel
ba0cc95441
Fix typo 2023-03-07 20:04:52 +01:00
Kruglov Pavel
479cd9b90b
Merge pull request #46972 from Avogar/json-date-int-inference 
Fix date and int inference from string in JSON
 2023-03-06 20:40:38 +01:00
Kruglov Pavel
3de905bb7c
Merge pull request #46616 from Avogar/fix-ipv4-ipv6-formats 
Fix IPv4/IPv6 serialization/deserialization in binary formats
 2023-03-06 19:40:29 +01:00
Kruglov Pavel
fed645a494
Don't use switch 2023-03-06 18:03:54 +01:00
Kruglov Pavel
df34ab240d
Don't use switch 2023-03-06 18:03:17 +01:00
avogar
d8e5fb5195 Use switch, use lz4 by default 2023-03-06 15:12:15 +00:00
Kruglov Pavel
3b034e7c35
Merge branch 'master' into parquet-compression 2023-03-02 15:55:48 +01:00
Robert Schulze
740aeaba1f
Apply some CTAD 2023-03-02 13:36:47 +00:00
Kruglov Pavel
ce8e49a9a0
Merge pull request #45693 from Avogar/fix-kafka-protobuf 
Fix formats parser resetting, test processing bad messages in kafka
 2023-03-02 11:50:45 +01:00
avogar
a6cf2cdab8 Fix style, add docs 2023-03-02 10:36:07 +00:00
avogar
5ab5902f38 Allow control compression in Parquet/ORC/Arrow output formats, support more compression for input formats 2023-03-01 21:27:46 +00:00
Kruglov Pavel
65f06fc9b1
Merge branch 'master' into json-date-int-inference 2023-02-28 14:31:57 +01:00
Kruglov Pavel
545e5c0f5b
Merge pull request #46617 from Avogar/use-parquet-2 
Change default parquet format version to v2_latest, add a setting to controle the version
 2023-02-28 14:30:59 +01:00
avogar
ab899bf2f3 Allow types conversion in Native input format 2023-02-27 19:28:19 +00:00
avogar
2e921e3d6b Fix date and int inference from string in JSON 2023-02-27 16:00:19 +00:00
avogar
12a9ff36b2 Better exception messages when schema_inference_hints is ill-formatted 2023-02-27 15:20:56 +00:00
Kruglov Pavel
443dedddca
Merge branch 'master' into use-parquet-2 2023-02-27 14:31:43 +01:00
avogar
12b38ece1d Fix build 2023-02-24 13:07:09 +00:00
Kruglov Pavel
2150b836b1
Merge branch 'master' into fix-kafka-protobuf 2023-02-24 12:48:07 +01:00
Kruglov Pavel
47f9ca2166
Merge branch 'master' into fix-ipv4-ipv6-formats 2023-02-23 20:32:43 +01:00
avogar
54622566df Add setting to change parquet version 2023-02-23 16:14:10 +00:00
Kruglov Pavel
ef0d6becba
Merge branch 'master' into null-as-default-all-formats 2023-02-21 16:52:39 +01:00
Kruglov Pavel
828a06b511
Merge branch 'master' into fix-kafka-protobuf 2023-02-20 21:45:43 +01:00
Kruglov Pavel
b0424c1021
Merge pull request #46171 from Avogar/insert-null-as-default 
Use default of column type in `insert_null_as_default` if column DEFAULT values is not specified
 2023-02-20 21:45:02 +01:00
avogar
a568704d63 Fix avro 2023-02-20 20:43:28 +00:00
Kruglov Pavel
9866ecfe8b
Merge branch 'master' into null-as-default-all-formats 2023-02-20 20:49:30 +01:00
avogar
42ae0582d3 Try to use parquet v2 instead of v1 in output format 2023-02-20 17:46:54 +00:00
avogar
8da3594cd8 Fix IPv4/IPv6 serialization/deserialization in binary formats 2023-02-20 17:42:56 +00:00
Alexey Milovidov
d8cda3dbb8 Remove PVS-Studio 2023-02-19 23:30:05 +01:00
flynn
ebd88aab41
Merge branch 'master' into iceberg 2023-02-17 21:53:29 +08:00
Kruglov Pavel
e56dead5aa
Merge branch 'master' into fix-kafka-protobuf 2023-02-15 15:16:25 +01:00
flynn
e05b4e9745 fix conflict 2023-02-14 13:49:28 +00:00
Geoff Genz
2857b16562
Merge branch 'ClickHouse:master' into http_client_version 2023-02-14 03:41:17 -07:00
robot-ch-test-poll4
7c420f0982
Merge pull request #46350 from CurtizJ/fix-format-values 
Fix `LOGICAL_ERROR` in async inserts with invalid data in format `VALUES`
 2023-02-14 10:12:08 +01:00
Alexey Milovidov
307551f857
Update NativeFormat.cpp 2023-02-14 11:08:33 +03:00
Geoff Genz
be8bf3a6a3
Merge branch 'master' into http_client_version 2023-02-13 08:43:59 -07:00
Anton Popov
7f31196606 fix LOGICAL_ERROR in async inserts 2023-02-13 14:59:14 +00:00
Kruglov Pavel
e139070043
Merge branch 'master' into fix-kafka-protobuf 2023-02-13 12:38:18 +01:00
Alexander Gololobov
f2a975f548
Merge pull request #46270 from Avogar/fix-arrow-lc 
Fix possible out of bounds error while reading LowCardinality(Nullable) in Arrow format
 2023-02-11 10:03:00 +01:00
Kruglov Pavel
8bbf62508f
Merge branch 'master' into fix-kafka-protobuf 2023-02-10 18:32:25 +01:00
avogar
04cf144edc Fix TSKV, update docs 2023-02-10 17:20:51 +00:00
avogar
d1efd02480 Extend setting input_format_null_as_default for more formats 2023-02-10 16:41:09 +00:00
avogar
deda940d87 Fix possible out of bounds error while reading LowCardinality(Nullable) in Arrow format 2023-02-10 12:57:47 +00:00
Geoff Genz
99c3ff53c5 Merge remote-tracking branch 'origin/master' into http_client_version 
# Conflicts:
#	src/Interpreters/Context.cpp
#	src/Interpreters/Context.h
 2023-02-10 04:35:53 -07:00
Geoff Genz
db982695fa Simplify NativeOutputFormat constructors 2023-02-10 04:30:08 -07:00
Geoff Genz
7ed8ed0284 Add support for client_protocol_version sent with HTTP 2023-02-10 03:47:06 -07:00
flynn
db15634a01 fix conflict 2023-02-10 08:41:04 +00:00