thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-12 17:32:32 +00:00
Code Issues Packages Projects Releases Wiki Activity
128,960 Commits 9,328 Branches 1,783 Tags 2.1 GiB
3319934055
Commit Graph

476 Commits

Author SHA1 Message Date
zvonand
a8aa206cfd Update after review 
expandSelector is now returning vector<string> directly
minor changes
 2023-11-15 16:37:24 +01:00
zvonand
70aa6e2672 Added a shortcut for no-wildcard globs in HDFS 2023-11-15 16:37:24 +01:00
Alexey Milovidov
3bec4dce8e Merge branch 'master' into remove-cpp-templates-2 2023-11-11 00:50:32 +01:00
Alexey Milovidov
8c253b9e3e Remove C++ templates 2023-11-10 05:25:02 +01:00
kssenii
a5b9ba51cd Better test 2023-11-08 12:55:14 +01:00
李扬
465962df7f
Support orc filter push down (file + stripe + rowgroup level) (#55330) 
* support orc filter push down

* update orc lib version

* replace setqueryinfo with setkeycondition

* fix issue https://github.com/ClickHouse/ClickHouse/issues/53536

* refactor source with key condition

* fix building error

* remove std::cout

* update orc

* update orc version

* fix bugs

* improve code

* upgrade orc lib

* fix code style

* change as requested

* add performance tests for orc filter push down

* add performance tests for orc filter push down

* fix all bugs

* fix default as null issue

* add uts for null as default issues

* upgrade orc lib

* fix failed orc lib uts and fix typo

* fix failed uts

* fix failed uts

* fix ast fuzzer tests

* fix bug of uint64 overflow in https://s3.amazonaws.com/clickhouse-test-reports/55330/de22fdcaea2e12c96f300e95f59beba84401712d/fuzzer_astfuzzerubsan/report.html

* fix asan fatal caused by reused column vector batch in native orc input format. refer to https://s3.amazonaws.com/clickhouse-test-reports/55330/be39d23af2d7e27f5ec7f168947cf75aeaabf674/stateless_tests__asan__[4_4].htm

* fix wrong performance tests

* disable 02892_orc_filter_pushdown on aarch64. https://s3.amazonaws.com/clickhouse-test-reports/55330/be39d23af2d7e27f5ec7f168947cf75aeaabf674/stateless_tests__aarch64_.html

* add some comments

* add some comments

* inline range::equals and range::less

* fix data race of key condition

* trigger ci
 2023-10-24 12:08:17 -07:00
Azat Khuzhin
2ecd9ddabd Fix leak in StorageHDFS 
LSan report:

    Direct leak of 400 byte(s) in 5 object(s) allocated from:
        0 0x564c44d5308d in operator new[](unsigned long) (/usr/bin/clickhouse+0xbdf908d) (BuildId: 9bd2befe9c5f1d960238405d4a3fc9860273b81a)
        1 0x564c7b1b83f2 in hdfsGetPathInfo build_docker/./contrib/libhdfs3/src/client/Hdfs.cpp:1096:18
        2 0x564c670a8fed in DB::HDFSSource::URISIterator::Impl::next() build_docker/./src/Storages/HDFS/StorageHDFS.cpp:471:25
        3 0x564c670a54e9 in DB::HDFSSource::URISIterator::next() build_docker/./src/Storages/HDFS/StorageHDFS.cpp:510:19

CI: https://s3.amazonaws.com/clickhouse-test-reports/55261/de503f75dcbc5a4d0e7fbb2e6b08c2106d62848a/integration_tests__asan__[3_6].html
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-10-09 10:46:00 +02:00
robot-clickhouse-ci-2
d98234dc9d
Merge pull request #54803 from Avogar/ephemeral-columns-from-files 
Forbid special columns for file/s3/url/... storages, fix insert into ephemeral columns from files
 2023-09-22 23:24:42 +02:00
zvonand
2eb63649bd fix old workflows 2023-09-22 00:30:46 +02:00
zvonand
e2b40749a6 the same for HDFS 2023-09-21 16:45:47 +02:00
zvonand
bc5f141a6c fix {..} for File 2023-09-21 16:40:38 +02:00
zvonand
a05bb020d4 do all the same for HDFS + remove setting 2023-09-21 01:50:41 +02:00
avogar
3e08800cb5 Forbid special columns for file/s3/url/... storages, fix insert into ephemeral columns from files 2023-09-20 16:25:55 +00:00
Robert Schulze
f5e8028bb1
Merge pull request #54642 from rschu1ze/broken-re2st 
Remove broken lockless variant of re2
 2023-09-17 15:30:57 +02:00
Robert Schulze
7b378dbad3
Remove broken lockless variant of re2 2023-09-14 16:40:42 +00:00
Andrey Zvonov
ff16ddbbb3
Merge branch 'ClickHouse:master' into zvonand-globs-small-fix 2023-09-14 11:31:49 +03:00
avogar
2d8f33bfa2 Fix parsing error in WithNames formats while reading subset of columns with disabled input_format_with_names_use_header 2023-09-11 14:55:37 +00:00
Andrey Zvonov
ff4586e663
Merge branch 'ClickHouse:master' into zvonand-globs-small-fix 2023-09-08 18:17:06 +02:00
zvonand
c36a3fbd39 renamed setting 2023-09-07 12:00:25 +03:00
alesapin
095bb37bc4
Merge pull request #54244 from bigo-sg/hdfs-config 
Minor improvement, load from config
 2023-09-05 12:36:38 +02:00
zhanglistar
a2fd3a4de8 remove unnecessary check for HDFS load config 2023-09-04 17:52:43 +08:00
kssenii
9eb1dfcd12 Refactor buffers reading from object storage 2023-09-01 14:03:07 +02:00
Andrey Zvonov
2447eb027c
Update StorageHDFS.cpp 2023-08-28 15:12:44 +03:00
Andrey Zvonov
911bf94fbd
Merge branch 'master' into zvonand-globs-small-fix 2023-08-25 23:50:07 +05:00
avogar
c4f8113a78 Fix tests 2023-08-24 15:05:45 +00:00
Kruglov Pavel
f7e1abd774
Merge branch 'master' into cache-count 2023-08-23 22:31:49 +02:00
avogar
894513f6cd Fix tests 2023-08-23 18:43:08 +00:00
Kruglov Pavel
592fa77987
Merge branch 'master' into cache-count 2023-08-23 15:18:02 +02:00
Kruglov Pavel
7e362a2110
Merge branch 'master' into fast-count-from-files 2023-08-23 15:13:20 +02:00
robot-ch-test-poll1
c22ffa6195
Merge pull request #53529 from Avogar/filter-files-all-table-functions 
Use filter by file/path before reading in url/file/hdfs table functins
 2023-08-23 14:21:23 +02:00
Kruglov Pavel
a83a868acd
Update src/Storages/HDFS/StorageHDFS.cpp 
Co-authored-by: Dmitry Novik <mrnovikd@gmail.com>
 2023-08-22 19:08:04 +02:00
Kruglov Pavel
67c5c0203b
Merge branch 'master' into fast-count-from-files 2023-08-22 15:03:48 +02:00
avogar
7f9e81d504 Clean up 2023-08-22 12:55:00 +00:00
Kruglov Pavel
c0bdd0e00b
Merge branch 'master' into cache-count 2023-08-22 14:42:22 +02:00
avogar
b4145aeddc Cache number of rows in files for count in file/s3/url/hdfs/azure functions 2023-08-22 11:59:59 +00:00
Michael Kolupaev
2f4d433e69 Parquet filter pushdown 2023-08-21 14:15:52 -07:00
avogar
47304bf7aa Optimize count from files in most input formats 2023-08-21 12:30:52 +00:00
avogar
8e445b5461 Fixes 2023-08-18 17:49:40 +00:00
avogar
4c32097df3 Use filter by file/path before reading in url/file/hdfs table functions, reduce code duplication 2023-08-17 16:54:43 +00:00
zvonand
086bf6cb8c added ignore_eacces_multidirectory_globs 2023-08-11 15:49:21 +03:00
zvonand
e4a1780bd7 update docs according to the PR + fix the same for HDFS 2023-08-10 15:29:27 +03:00
Kruglov Pavel
b1fe9ce8f7
Merge branch 'master' into formats-with-subcolumns 2023-08-04 13:01:12 +02:00
Kseniia Sumarokova
68b48a0bc0
Merge pull request #50063 from KevinyhZou/minor_improve_hdfs_read_buffer 
Minor improve HDFS ReadBuffer for read end of file
 2023-08-03 17:55:48 +02:00
Anton Popov
ff137773e7
Merge branch 'master' into formats-with-subcolumns 2023-08-02 15:24:56 +02:00
Kseniia Sumarokova
ccf9fa9641
Update ReadBufferFromHDFS.cpp 2023-08-01 11:55:08 +02:00
Kseniia Sumarokova
5880a0a372
Update ReadBufferFromHDFS.cpp 2023-08-01 11:50:31 +02:00
Kseniia Sumarokova
74c0574d51
Update ReadBufferFromHDFS.cpp 2023-07-31 15:22:16 +02:00
Kseniia Sumarokova
dd85c3c114
Update ReadBufferFromHDFS.cpp 2023-07-30 22:40:11 +02:00
Kseniia Sumarokova
30b6cf0713
Merge branch 'master' into minor_improve_hdfs_read_buffer 2023-07-30 13:33:24 +02:00
Al Korgun
ebbd662071
Merge branch 'ClickHouse:master' into fix-named-collections-on-cluster-23.7 2023-07-27 23:31:35 +03:00