Commit Graph

85 Commits

Author SHA1 Message Date
李扬
465962df7f
Support orc filter push down (file + stripe + rowgroup level) (#55330)
* support orc filter push down

* update orc lib version

* replace setqueryinfo with setkeycondition

* fix issue https://github.com/ClickHouse/ClickHouse/issues/53536

* refactor source with key condition

* fix building error

* remove std::cout

* update orc

* update orc version

* fix bugs

* improve code

* upgrade orc lib

* fix code style

* change as requested

* add performance tests for orc filter push down

* add performance tests for orc filter push down

* fix all bugs

* fix default as null issue

* add uts for null as default issues

* upgrade orc lib

* fix failed orc lib uts and fix typo

* fix failed uts

* fix failed uts

* fix ast fuzzer tests

* fix bug of uint64 overflow in https://s3.amazonaws.com/clickhouse-test-reports/55330/de22fdcaea2e12c96f300e95f59beba84401712d/fuzzer_astfuzzerubsan/report.html

* fix asan fatal caused by reused column vector batch in native orc input format. refer to https://s3.amazonaws.com/clickhouse-test-reports/55330/be39d23af2d7e27f5ec7f168947cf75aeaabf674/stateless_tests__asan__[4_4].htm

* fix wrong performance tests

* disable 02892_orc_filter_pushdown on aarch64. https://s3.amazonaws.com/clickhouse-test-reports/55330/be39d23af2d7e27f5ec7f168947cf75aeaabf674/stateless_tests__aarch64_.html

* add some comments

* add some comments

* inline range::equals and range::less

* fix data race of key condition

* trigger ci
2023-10-24 12:08:17 -07:00
kssenii
f90e31eb31 Enable log by default 2023-10-18 15:23:16 +02:00
kssenii
c4bad255b7 Fix 2023-10-18 14:37:33 +02:00
kssenii
eb4519b9d6 Fix 2023-10-17 19:02:04 +02:00
kssenii
c549083e2c Fix 2023-10-17 14:23:59 +02:00
kssenii
42ed249954 Fix build 2023-10-17 12:03:49 +02:00
kssenii
8744cd9c0d Fix 2023-10-17 11:15:53 +02:00
kssenii
0a6a4b3894 Review fixes 2023-10-16 18:15:17 +02:00
kssenii
0ddee6ad6c Fix 2023-10-15 14:44:57 +02:00
kssenii
6fde98a33f Minor improvement 2023-10-13 15:56:05 +02:00
kssenii
44ea61e57f Improve shutdown 2023-10-13 15:23:36 +02:00
kssenii
b6b124f5a9 Usability improvement 2023-10-13 14:16:56 +02:00
kssenii
bfe174f71b Fix test 2023-10-13 12:57:57 +02:00
kssenii
d64b990712 Merge remote-tracking branch 'origin/master' into s3-queue-fixes 2023-10-13 12:13:56 +02:00
kssenii
1ef21bab3d Fix data race 2023-09-29 16:02:51 +02:00
kssenii
4a7922507b Minor changes 2023-09-28 16:18:00 +02:00
kssenii
7d91ba109a Update doc 2023-09-28 13:58:59 +02:00
kssenii
57cfb88cb6 Fix typos check, fix build wihtout s3 2023-09-27 22:22:49 +02:00
kssenii
c579f5b321 Fix style check 2023-09-27 20:10:05 +02:00
kssenii
42783894b7 Add comments 2023-09-27 20:07:19 +02:00
kssenii
e0ff76a7f9 Fix 2023-09-27 18:44:53 +02:00
kssenii
f753b91a3b Better maintenance of processing node 2023-09-27 17:17:52 +02:00
kssenii
6b191a1afe Better 2023-09-27 14:54:31 +02:00
kssenii
d77452c561 Fix style check 2023-09-27 12:06:41 +02:00
kssenii
6c778d4b12 Fix build without s3 2023-09-27 12:05:13 +02:00
kssenii
14b09d3cdc Add caching 2023-09-26 16:23:24 +02:00
kssenii
4d78dbacfe Fix style check, fix build without s3 2023-09-25 23:38:16 +02:00
kssenii
251b0165ca Better 2023-09-25 22:55:41 +02:00
kssenii
3b54b6be88 Add system tables 2023-09-25 20:04:15 +02:00
Kruglov Pavel
af8b1e4978
Merge pull request #54855 from Avogar/obsolete-format-settings
Move obsolete format settings to separate section
2023-09-25 18:00:31 +02:00
kssenii
e4256eb268 Better 2023-09-25 13:44:45 +02:00
avogar
54cee196bd Move obsolete format settings to separate section 2023-09-20 16:00:28 +00:00
pufit
a1c98bc8c9 fix build 2023-09-20 09:50:35 -04:00
kssenii
6846fe3c58 Fxi 2023-09-15 14:21:08 +02:00
kssenii
342755d35e Better 2023-09-14 18:41:31 +02:00
kssenii
afcb0b2b9a Merge remote-tracking branch 'upstream/master' into s3-queue-fixes 2023-09-14 17:16:29 +02:00
avogar
2d8f33bfa2 Fix parsing error in WithNames formats while reading subset of columns with disabled input_format_with_names_use_header 2023-09-11 14:55:37 +00:00
kssenii
220a67eca7 Better 2023-09-08 16:38:05 +02:00
kssenii
7338b560a8 S3Queue fixes 2023-09-07 19:17:21 +02:00
Alexey Milovidov
282636fd74 S3Queue is experimental 2023-09-03 00:00:37 +02:00
robot-clickhouse-ci-2
ae4f6c9683
Merge pull request #54137 from kssenii/s3-queue-better-default-path
Better path for storage `S3Queue` metadata
2023-09-01 16:01:38 +02:00
Kseniia Sumarokova
c51b6ca60c
Update StorageS3Queue.cpp 2023-08-31 18:29:06 +02:00
kssenii
da185fc8b1 Better paths for s3 queue metadata 2023-08-31 18:16:07 +02:00
Kruglov Pavel
592fa77987
Merge branch 'master' into cache-count 2023-08-23 15:18:02 +02:00
Kruglov Pavel
7e362a2110
Merge branch 'master' into fast-count-from-files 2023-08-23 15:13:20 +02:00
robot-ch-test-poll1
c22ffa6195
Merge pull request #53529 from Avogar/filter-files-all-table-functions
Use filter by file/path before reading in url/file/hdfs table functins
2023-08-23 14:21:23 +02:00
Kruglov Pavel
67c5c0203b
Merge branch 'master' into fast-count-from-files 2023-08-22 15:03:48 +02:00
Kruglov Pavel
c0bdd0e00b
Merge branch 'master' into cache-count 2023-08-22 14:42:22 +02:00
avogar
b4145aeddc Cache number of rows in files for count in file/s3/url/hdfs/azure functions 2023-08-22 11:59:59 +00:00
Michael Kolupaev
2f4d433e69 Parquet filter pushdown 2023-08-21 14:15:52 -07:00