Commit Graph

54 Commits

Author SHA1 Message Date
kssenii
f2c0434c4d Merge remote-tracking branch 'origin/master' into minor-improvements-for-s3-queue 2023-11-06 15:51:11 +01:00
kssenii
da21413354 Better shutdown 2023-11-06 15:47:57 +01:00
Kruglov Pavel
5533258091
Merge pull request #55477 from Avogar/streaming-errors
Allow to save unparsed records and errors in RabbitMQ, NATS and FileLog engines
2023-11-01 14:35:35 +01:00
李扬
465962df7f
Support orc filter push down (file + stripe + rowgroup level) (#55330)
* support orc filter push down

* update orc lib version

* replace setqueryinfo with setkeycondition

* fix issue https://github.com/ClickHouse/ClickHouse/issues/53536

* refactor source with key condition

* fix building error

* remove std::cout

* update orc

* update orc version

* fix bugs

* improve code

* upgrade orc lib

* fix code style

* change as requested

* add performance tests for orc filter push down

* add performance tests for orc filter push down

* fix all bugs

* fix default as null issue

* add uts for null as default issues

* upgrade orc lib

* fix failed orc lib uts and fix typo

* fix failed uts

* fix failed uts

* fix ast fuzzer tests

* fix bug of uint64 overflow in https://s3.amazonaws.com/clickhouse-test-reports/55330/de22fdcaea2e12c96f300e95f59beba84401712d/fuzzer_astfuzzerubsan/report.html

* fix asan fatal caused by reused column vector batch in native orc input format. refer to https://s3.amazonaws.com/clickhouse-test-reports/55330/be39d23af2d7e27f5ec7f168947cf75aeaabf674/stateless_tests__asan__[4_4].htm

* fix wrong performance tests

* disable 02892_orc_filter_pushdown on aarch64. https://s3.amazonaws.com/clickhouse-test-reports/55330/be39d23af2d7e27f5ec7f168947cf75aeaabf674/stateless_tests__aarch64_.html

* add some comments

* add some comments

* inline range::equals and range::less

* fix data race of key condition

* trigger ci
2023-10-24 12:08:17 -07:00
avogar
1afd655bbc Allow to save unparsed records and errors in RabbitMQ, NATS and FileLog engines 2023-10-10 16:21:33 +00:00
Nikolay Degterinsky
987906680f
Revert "Fix NATS high cpu usage" 2023-09-26 03:36:14 +02:00
Kruglov Pavel
af8b1e4978
Merge pull request #54855 from Avogar/obsolete-format-settings
Move obsolete format settings to separate section
2023-09-25 18:00:31 +02:00
avogar
54cee196bd Move obsolete format settings to separate section 2023-09-20 16:00:28 +00:00
Petr Vasilev
cb969aa54a Remove empty line 2023-09-07 18:47:31 +03:00
Petr Vasilev
4ef241445f Remove received_mutex and received_cv 2023-09-07 18:44:18 +03:00
Petr Vasilev
c0333263a0 Rollback do_reschedule flag 2023-09-07 15:07:21 +03:00
Petr Vasilev
9c61ceb051 Fix nats high cpu usage 2023-09-07 13:59:32 +03:00
Nikita Mikhaylov
96cabcae3a Change interface 2023-08-29 16:26:48 +02:00
Antonio Andelic
b11f744252
Correctly disable async insert with deduplication when it's not needed (#50663)
* Correctly disable async insert when it's not used

* Better

* Add comment

* Better

* Fix tests

---------

Co-authored-by: Nikita Mikhaylov <mikhaylovnikitka@gmail.com>
2023-06-07 20:33:08 +02:00
Michael Kolupaev
683077890f Highly questionable refactoring (getInputMultistream() nonsense) 2023-04-17 04:58:32 +00:00
kssenii
8f2d75cef8 Fix tests 2023-03-05 12:56:00 +01:00
kssenii
b3ca976f5f Finish for streaming engines 2023-02-21 14:50:55 +01:00
Kseniia Sumarokova
0b55021c68
Merge pull request #46337 from kssenii/rabbitmq-tests-fix-2
Fix flaky test_storage_rabbitmq::test_rabbitmq_address
2023-02-14 11:44:34 +01:00
kssenii
614d57d245 Fix race 2023-02-13 16:02:41 +01:00
Antonio Andelic
85cfee4bb9 Better alias definition 2023-02-01 13:54:03 +00:00
Antonio Andelic
714fad1529 Add support for settings alias 2023-01-26 14:06:46 +00:00
Alexander Tokmakov
70d1adfe4b
Better formatting for exception messages (#45449)
* save format string for NetException

* format exceptions

* format exceptions 2

* format exceptions 3

* format exceptions 4

* format exceptions 5

* format exceptions 6

* fix

* format exceptions 7

* format exceptions 8

* Update MergeTreeIndexGin.cpp

* Update AggregateFunctionMap.cpp

* Update AggregateFunctionMap.cpp

* fix
2023-01-24 00:13:58 +03:00
avogar
755b08a49e Fix comments 2022-12-15 19:47:10 +00:00
Kruglov Pavel
c5b2e4cc23
Merge branch 'master' into improve-streaming-engines 2022-12-15 18:44:35 +01:00
Vitaly Baranov
e1f7f04752
Referential dependencies for RESTORE (#43834)
* Rename DDLDependencyVisitor -> DDLLoadingDependencyVisitor.

* Move building a loading graph to TablesLoader.

* Implement referential dependencies for tables and use them
when restoring tables from a backup.

* Remove StorageID::operator < (because of its inconsistency with ==).

* Add new tests.

* Fix test.

* Fix memory leak.

Co-authored-by: Nikita Mikhaylov <mikhaylovnikitka@gmail.com>
2022-12-02 15:05:46 +01:00
Kruglov Pavel
b124875257
Merge branch 'master' into improve-streaming-engines 2022-11-03 13:22:06 +01:00
avogar
e39e61fc71 Fix heap-use-after-free in PeekableReadBuffer 2022-11-01 12:58:20 +00:00
Kruglov Pavel
32c2fb378f
Fix build 2022-10-29 00:02:44 +02:00
avogar
8e13d1f1ec Improve and refactor Kafka/StorageMQ/NATS and data formats 2022-10-28 16:41:10 +00:00
Azat Khuzhin
4e76629aaf Fixes for -Wshorten-64-to-32
- lots of static_cast
- add safe_cast
- types adjustments
  - config
  - IStorage::read/watch
  - ...
- some TODO's (to convert types in future)

P.S. That was quite a journey...

v2: fixes after rebase
v3: fix conflicts after #42308 merged
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-10-21 13:25:19 +02:00
kssenii
f8665c83da Add retries for nats 2022-09-19 14:27:11 +02:00
vdimir
473c9106a5 Revert "Create new query context in StorageRabbitMQ/Nat"
This reverts commit 2d6ac7344a25134a625e879da63b9a4aa20d9a69.
2022-08-31 11:57:41 +02:00
vdimir
63f7301ee0 Create new query context in StorageRabbitMQ/Nat 2022-08-31 11:57:41 +02:00
vdimir
c829a1914d Disable max_analyze_depth for StorageRabbitMQ/NATS 2022-08-31 11:57:41 +02:00
Robert Schulze
e8b3f56733
Limit suppression to a specific warning 2022-08-21 18:24:17 +00:00
Alexey Milovidov
74e1f4dc61 Fix clang-tidy 2022-08-20 17:09:20 +02:00
Constantine Peresypkin
10c76917ea fix nats-io TLS support
nats-io library needs `NATS_HAS_TLS` define to correctly compile-in TLS
support

fixes #39525
2022-08-05 11:20:05 +02:00
kssenii
7a4be3b748 Fix build, disable for freebsd 2022-07-05 12:10:49 +02:00
tchepavel
75db37cb57 Small engine overhaul 2022-06-01 21:30:08 +03:00
tchepavel
531bda5a65 Spelling and more black python 2022-05-25 22:13:26 +03:00
tchepavel
d4afc36f8e Style+build fixes; make long tests time limited 2022-05-25 22:02:47 +03:00
tchepavel
87217ce6cc NATS. Add new setting to doc; clean up code. 2022-05-23 14:57:39 +03:00
tchepavel
a3af94d49e Documentation; integration tests; subscribe on startup 2022-05-17 16:58:09 +03:00
tchepavel
2f00139347 Fix review issues; add insert queue parameter 2022-05-16 16:35:17 +03:00
tchepavel
35cbe48ca3 Fixing; preparing test 2022-05-16 13:03:28 +03:00
tchepavel
1295ee852c NATS. Style check 2022-05-12 22:19:11 +03:00
tchepavel
fe1cdf8ef1 NATS. Final functional improvements 2022-05-12 22:11:59 +03:00
tchepavel
cc94378698 Add nats-io cmake; finally fix publishing 2022-05-12 19:11:32 +03:00
tchepavel
e1a7ccd069 NATS. Fix some reconnection issues; add settings 2022-05-12 16:52:14 +03:00
tchepavel
d9a19910a8 Add nats-io cmake; move subscribe 2022-05-12 15:32:51 +03:00