Commit Graph

393 Commits

Author SHA1 Message Date
Alexey Milovidov
69e6e59e96 Add a test for #49708 2023-12-18 08:34:53 +01:00
avogar
ee7af95bc0 Merge branch 'master' of github.com:ClickHouse/ClickHouse into schema-inference-union 2023-12-08 20:29:28 +00:00
Nikolay Degterinsky
ceed0f73d3
Merge pull request #57391 from evillique/better-partitioned-write-to-file
Allow wildcards in directories for partitioned write with File storage
2023-12-01 17:23:54 +01:00
Nikolay Degterinsky
95b5b9eb35 Fix partitioned write for File storage 2023-11-30 16:18:12 +00:00
zvonand
6730a29a0b upd 2023-11-30 01:28:25 +01:00
zvonand
c329e382ab resolve conflicts 2023-11-29 16:03:07 +01:00
zvonand
a4d9fbdfc2 Fix indents, fix style, fix test 2023-11-29 15:55:24 +01:00
avogar
4d9a1b50f9 Add information about new _size virtual column in file/s3/url/hdfs/azure table functions 2023-11-28 18:15:07 +00:00
zvonand
c306d21b54 merge master + resolve conflicts 2023-11-28 15:51:21 +01:00
avogar
007353a2dd Add _size virtual column to s3/file/hdfs/url/azureBlobStorage engines 2023-11-22 18:12:36 +00:00
zvonand
5153798aeb Introduced fileCluster table function
Added fileCluster function
Added test and docs
2023-11-22 15:06:04 +01:00
avogar
872556a5d4 Merge branch 'master' of github.com:ClickHouse/ClickHouse into schema-inference-union 2023-11-20 14:03:36 +00:00
zvonand
4ff82528f1 small fixes after review 2023-11-16 00:01:36 +01:00
zvonand
a8aa206cfd Update after review
expandSelector is now returning vector<string> directly
minor changes
2023-11-15 16:37:24 +01:00
zvonand
70aa6e2672 Added a shortcut for no-wildcard globs in HDFS 2023-11-15 16:37:24 +01:00
zvonand
3e47a54a94 Added a shortcut for no-wildcard globs in file storage 2023-11-15 16:37:24 +01:00
Kruglov Pavel
570b66f027
Merge branch 'master' into schema-inference-union 2023-10-26 19:26:00 +02:00
李扬
465962df7f
Support orc filter push down (file + stripe + rowgroup level) (#55330)
* support orc filter push down

* update orc lib version

* replace setqueryinfo with setkeycondition

* fix issue https://github.com/ClickHouse/ClickHouse/issues/53536

* refactor source with key condition

* fix building error

* remove std::cout

* update orc

* update orc version

* fix bugs

* improve code

* upgrade orc lib

* fix code style

* change as requested

* add performance tests for orc filter push down

* add performance tests for orc filter push down

* fix all bugs

* fix default as null issue

* add uts for null as default issues

* upgrade orc lib

* fix failed orc lib uts and fix typo

* fix failed uts

* fix failed uts

* fix ast fuzzer tests

* fix bug of uint64 overflow in https://s3.amazonaws.com/clickhouse-test-reports/55330/de22fdcaea2e12c96f300e95f59beba84401712d/fuzzer_astfuzzerubsan/report.html

* fix asan fatal caused by reused column vector batch in native orc input format. refer to https://s3.amazonaws.com/clickhouse-test-reports/55330/be39d23af2d7e27f5ec7f168947cf75aeaabf674/stateless_tests__asan__[4_4].htm

* fix wrong performance tests

* disable 02892_orc_filter_pushdown on aarch64. https://s3.amazonaws.com/clickhouse-test-reports/55330/be39d23af2d7e27f5ec7f168947cf75aeaabf674/stateless_tests__aarch64_.html

* add some comments

* add some comments

* inline range::equals and range::less

* fix data race of key condition

* trigger ci
2023-10-24 12:08:17 -07:00
avogar
6934e27e8b Add union mode for schema inference to infer union schema of files with different schemas 2023-10-20 20:46:41 +00:00
Michael Kolupaev
ce7eca0615
DWARF input format (#55450)
* Add ReadBufferFromFileBase::isRegularLocalFile()

* DWARF input format

* Review comments

* Changed things around ENABLE_EMBEDDED_COMPILER build setting

* Added 'ranges' column

* no-msan no-ubsan
2023-10-16 17:00:07 -07:00
robot-clickhouse-ci-2
d98234dc9d
Merge pull request #54803 from Avogar/ephemeral-columns-from-files
Forbid special columns for file/s3/url/... storages, fix insert into ephemeral columns from files
2023-09-22 23:24:42 +02:00
zvonand
2eb63649bd fix old workflows 2023-09-22 00:30:46 +02:00
zvonand
bc5f141a6c fix {..} for File 2023-09-21 16:40:38 +02:00
zvonand
a05bb020d4 do all the same for HDFS + remove setting 2023-09-21 01:50:41 +02:00
zvonand
5f67788c03 update docs and comments 2023-09-21 00:16:17 +02:00
zvonand
9eccb12183 small refactor of storageFile globs 2023-09-20 23:59:10 +02:00
avogar
3e08800cb5 Forbid special columns for file/s3/url/... storages, fix insert into ephemeral columns from files 2023-09-20 16:25:55 +00:00
Robert Schulze
f5e8028bb1
Merge pull request #54642 from rschu1ze/broken-re2st
Remove broken lockless variant of re2
2023-09-17 15:30:57 +02:00
Robert Schulze
7b378dbad3
Remove broken lockless variant of re2 2023-09-14 16:40:42 +00:00
Andrey Zvonov
ff16ddbbb3
Merge branch 'ClickHouse:master' into zvonand-globs-small-fix 2023-09-14 11:31:49 +03:00
avogar
2d8f33bfa2 Fix parsing error in WithNames formats while reading subset of columns with disabled input_format_with_names_use_header 2023-09-11 14:55:37 +00:00
Andrey Zvonov
ff4586e663
Merge branch 'ClickHouse:master' into zvonand-globs-small-fix 2023-09-08 18:17:06 +02:00
zvonand
c36a3fbd39 renamed setting 2023-09-07 12:00:25 +03:00
Kruglov Pavel
de801ce563
Merge pull request #54293 from ClickHouse/pasha-returned-from-vacation
Code improvement for reading from archives
2023-09-05 21:09:09 +02:00
Antonio Andelic
88930a335c Apply comments 2023-09-05 12:32:07 +00:00
Antonio Andelic
a51c333c5f Improve code for reading from archives 2023-09-05 08:37:02 +00:00
avogar
e9456082c1 Fix using count from cache and improve progress bar for reading from archives 2023-09-04 17:33:59 +00:00
Andrey Zvonov
83405a8808
Merge branch 'master' into zvonand-globs-small-fix 2023-08-31 08:42:48 +02:00
Antonio Andelic
f406019413 Apply PR comments 2023-08-30 09:26:01 +00:00
Antonio Andelic
a0dd350e5e Fix assert 2023-08-29 09:59:16 +00:00
Antonio Andelic
8708ebd7fb Correctly check files 2023-08-29 07:40:08 +00:00
Antonio Andelic
9b99f25d75 Improve schema inference 2023-08-28 13:11:52 +00:00
Antonio Andelic
5a0c2ca108 Merge branch 'master' into archive-improvements-2 2023-08-28 08:34:42 +00:00
Andrey Zvonov
911bf94fbd
Merge branch 'master' into zvonand-globs-small-fix 2023-08-25 23:50:07 +05:00
Antonio Andelic
8e1d38d377 Merge branch 'master' into archive-improvements-2 2023-08-24 13:03:36 +00:00
Kruglov Pavel
f7e1abd774
Merge branch 'master' into cache-count 2023-08-23 22:31:49 +02:00
Kruglov Pavel
592fa77987
Merge branch 'master' into cache-count 2023-08-23 15:18:02 +02:00
Kruglov Pavel
7e362a2110
Merge branch 'master' into fast-count-from-files 2023-08-23 15:13:20 +02:00
robot-ch-test-poll1
c22ffa6195
Merge pull request #53529 from Avogar/filter-files-all-table-functions
Use filter by file/path before reading in url/file/hdfs table functins
2023-08-23 14:21:23 +02:00
Antonio Andelic
83d4b819f3 Better support for reading from archives 2023-08-23 08:10:30 +00:00