Commit Graph

215 Commits

Author SHA1 Message Date
Alexey Milovidov
d56cbda185 Add metrics for the number of queued jobs, which is useful for the IO thread pool 2023-11-18 19:07:59 +01:00
李扬
465962df7f
Support orc filter push down (file + stripe + rowgroup level) (#55330)
* support orc filter push down

* update orc lib version

* replace setqueryinfo with setkeycondition

* fix issue https://github.com/ClickHouse/ClickHouse/issues/53536

* refactor source with key condition

* fix building error

* remove std::cout

* update orc

* update orc version

* fix bugs

* improve code

* upgrade orc lib

* fix code style

* change as requested

* add performance tests for orc filter push down

* add performance tests for orc filter push down

* fix all bugs

* fix default as null issue

* add uts for null as default issues

* upgrade orc lib

* fix failed orc lib uts and fix typo

* fix failed uts

* fix failed uts

* fix ast fuzzer tests

* fix bug of uint64 overflow in https://s3.amazonaws.com/clickhouse-test-reports/55330/de22fdcaea2e12c96f300e95f59beba84401712d/fuzzer_astfuzzerubsan/report.html

* fix asan fatal caused by reused column vector batch in native orc input format. refer to https://s3.amazonaws.com/clickhouse-test-reports/55330/be39d23af2d7e27f5ec7f168947cf75aeaabf674/stateless_tests__asan__[4_4].htm

* fix wrong performance tests

* disable 02892_orc_filter_pushdown on aarch64. https://s3.amazonaws.com/clickhouse-test-reports/55330/be39d23af2d7e27f5ec7f168947cf75aeaabf674/stateless_tests__aarch64_.html

* add some comments

* add some comments

* inline range::equals and range::less

* fix data race of key condition

* trigger ci
2023-10-24 12:08:17 -07:00
avogar
54cee196bd Move obsolete format settings to separate section 2023-09-20 16:00:28 +00:00
avogar
2d8f33bfa2 Fix parsing error in WithNames formats while reading subset of columns with disabled input_format_with_names_use_header 2023-09-11 14:55:37 +00:00
Michael Kolupaev
2f4d433e69 Parquet filter pushdown 2023-08-21 14:15:52 -07:00
Antonio Andelic
b11f744252
Correctly disable async insert with deduplication when it's not needed (#50663)
* Correctly disable async insert when it's not used

* Better

* Add comment

* Better

* Fix tests

---------

Co-authored-by: Nikita Mikhaylov <mikhaylovnikitka@gmail.com>
2023-06-07 20:33:08 +02:00
kssenii
ecfbf1e304 Remove dependency from DB::Context in readers 2023-05-02 21:45:27 +02:00
Michael Kolupaev
683077890f Highly questionable refactoring (getInputMultistream() nonsense) 2023-04-17 04:58:32 +00:00
Azat Khuzhin
79b83c4fd2 Remove superfluous includes of logger_userful.h from headers
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-04-10 17:59:30 +02:00
Azat Khuzhin
f38a7aeabe ThreadPool metrics introspection
There are lots of thread pools and simple local-vs-global is not enough
already, it is good to know which one in particular uses threads.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-03-29 10:46:59 +02:00
Robert Schulze
740aeaba1f
Apply some CTAD 2023-03-02 13:36:47 +00:00
Antonio Andelic
8d16fe5793
Merge branch 'master' into add-support-for-settings-alias 2023-02-13 08:46:00 +01:00
kssenii
b0b865c32e Resubmit prefetches 2023-02-08 21:26:24 +01:00
Antonio Andelic
85cfee4bb9 Better alias definition 2023-02-01 13:54:03 +00:00
Antonio Andelic
ec04cf6876 Merge branch 'master' into add-support-for-settings-alias 2023-02-01 09:01:01 +00:00
Antonio Andelic
714fad1529 Add support for settings alias 2023-01-26 14:06:46 +00:00
Alexander Tokmakov
3f6594f4c6 forbid old ctor of Exception 2023-01-23 22:18:05 +01:00
Alexander Tokmakov
70d1adfe4b
Better formatting for exception messages (#45449)
* save format string for NetException

* format exceptions

* format exceptions 2

* format exceptions 3

* format exceptions 4

* format exceptions 5

* format exceptions 6

* fix

* format exceptions 7

* format exceptions 8

* Update MergeTreeIndexGin.cpp

* Update AggregateFunctionMap.cpp

* Update AggregateFunctionMap.cpp

* fix
2023-01-24 00:13:58 +03:00
xiedeyantu
304b6ebf3a s3 table function can support select nested column using {column_name}.{subcolumn_name} 2022-11-23 23:36:12 +08:00
Amos Bird
661c541e57
Fix Nullable keys in hyperrectangle. 2022-11-11 11:14:05 +08:00
Azat Khuzhin
905a95e166 Review fixes
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-10-21 22:40:13 +02:00
Azat Khuzhin
4e76629aaf Fixes for -Wshorten-64-to-32
- lots of static_cast
- add safe_cast
- types adjustments
  - config
  - IStorage::read/watch
  - ...
- some TODO's (to convert types in future)

P.S. That was quite a journey...

v2: fixes after rebase
v3: fix conflicts after #42308 merged
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-10-21 13:25:19 +02:00
Robert Schulze
78fc36ca49
Generate config.h into ${CONFIG_INCLUDE_PATH}
This makes the target location consistent with other auto-generated
files like config_formats.h, config_core.h, and config_functions.h and
simplifies the build of clickhouse_common.
2022-09-28 12:48:26 +00:00
Kseniia Sumarokova
a6cfc7bc3b
Merge pull request #34651 from alexX512/master
New caching strategies
2022-08-12 17:23:37 +02:00
vdimir
b7c5c54181
Fix build 2022-08-10 13:43:55 +00:00
Alexey Milovidov
78d3079f0c Non-significant changes 2022-08-09 21:32:42 +02:00
alexX512
6bf29cb610 Change class LRUCache to class CachBase. Check running CacheBase with default pcahce policy SLRU 2022-08-07 19:59:30 +00:00
Nikolai Kochetov
b70be40804
Merge branch 'master' into use-dag-in-key-condition 2022-07-25 14:30:22 +02:00
Alexander Tokmakov
a8da5d96fc remove some dead and commented code 2022-07-21 15:05:48 +02:00
Nikolai Kochetov
4e8cd70b1d
Merge branch 'master' into use-dag-in-key-condition 2022-07-20 17:38:33 +02:00
Vladimir C
7694c72154
Merge pull request #38887 from bigo-sg/hive_file_crash
fixed: crash caused by IHiveFile be shared among threads
2022-07-20 12:15:57 +02:00
lgbo-ustc
4b46d80169 update codes 2022-07-20 18:03:34 +08:00
Nikolai Kochetov
f570cde815 Fixing build. 2022-07-19 20:19:57 +00:00
Sergei Trifonov
af4a5a5f3a fix conflicts 2022-07-13 19:44:29 +02:00
Sergei Trifonov
f854507729 merge master and resolve conflicts 2022-07-13 19:07:28 +02:00
kssenii
6f29eada80 Merge master 2022-07-12 19:28:18 +02:00
Sergei Trifonov
add0d1698e respect new limits in HDFS 2022-07-11 17:30:23 +02:00
kssenii
bf1c99daab Refactoring of object storages, add LocalObjectStorage 2022-07-09 12:39:31 +02:00
lgbo-ustc
e01b6cf8a5 update codes 2022-07-08 09:25:45 +08:00
Kseniia Sumarokova
48220f64d6
Merge pull request #38214 from bigo-sg/storagehive_alter
support alter on `StorageHive` table
2022-07-07 17:06:43 +02:00
lgbo-ustc
337f379bc3 update codes 2022-07-07 09:45:49 +08:00
lgbo-ustc
cda081dc57 update codes 2022-07-07 09:41:52 +08:00
lgbo-ustc
aeb4f9c6be fixed: crash caused by IHiveFile be shared among threads 2022-07-06 16:56:17 +08:00
lgbo-ustc
946a9b8c63 update codes 2022-07-06 09:26:02 +08:00
lgbo-ustc
76813ee3c0 update codes 2022-07-05 11:49:55 +08:00
Kruglov Pavel
6c4b6fa834
Merge branch 'master' into better-exception-messages 2022-07-04 15:07:31 +02:00
lgbo-ustc
932e596689 Merge remote-tracking branch 'ck/master' into storagehive_alter 2022-07-04 09:56:00 +08:00
avogar
59c1c472cb Better exception messages on wrong table engines/functions argument types 2022-06-23 20:04:06 +00:00
lgbo-ustc
cd8e5c7c49 update headers 2022-06-23 17:43:54 +08:00
lgbo-ustc
fc641d9ce4 some changes 2022-06-21 11:29:55 +08:00