Commit Graph

2203 Commits

Author SHA1 Message Date
Antonio Andelic
59f9abcf58 Merge branch 'master' into keeper-reduce-memory 2024-01-22 08:04:17 +00:00
Antonio Andelic
0132455b39 Reduce Keeper memory usage 2024-01-19 17:00:53 +00:00
Robert Schulze
316669b90f
Merge pull request #58396 from zenmiao7/u/zhengmiao/AddArrayFunctionShingles
Add function `arrayShingles()`
2024-01-18 11:38:56 +01:00
Srikanth Chekuri
c23c7e0b6f
feat: add DDSketch quantile (#56342)
* feat: add DDSketch quantile

* Fix StyleCheck

* Move quantileddsketch.md under right dir

* Update stateless test number and add stateful test

* Throw exception when relative accuracy is too low

* Update test number

* Fix undefined behaviour for empty store

* Fix quantileGK test

* Update test numbers

* Update src/AggregateFunctions/

* Throw exception on out of range values

* Update relative accuracy docs and add encoding details

* Address review comments and suggestions

* Remove unused function

* Address alexey review comments

* Remove unused function isFloat64FieldType

* Throw error on invalid relative accuracy

* Simplify mapping

* Address remaining review comments

* Add effective memory usage suggestions

* Fix Stylecheck

* Fix fast test

* Incorporate bins capacity suggestion

* Fix fuzzer

* Remove inheritance in Mapping, Store and DDSketch

* Add checks for bin resizing

* Add note about the binary compatible implementation
2024-01-17 19:51:32 -08:00
Robert Schulze
226f1804b7
Fix spelling and style 2024-01-17 15:26:18 +00:00
Antonio Andelic
76c9bf42d7 Add docs for allocation profiling 2024-01-15 13:57:16 +01:00
Alexey Milovidov
c132e0141c
Merge pull request #58791 from ClickHouse/better-stateful-tests
Simplify stateful tests
2024-01-14 21:39:13 +01:00
Alexander Tokmakov
5128607995
Merge pull request #58393 from jaymebrd/feat/add-from-modifier-sync-replica-lightweight
add `from srcreplicas` modifier  for `SYSTEM SYNC REPLICA LIGHTWEIGHT`
2024-01-14 20:53:16 +01:00
Alexey Milovidov
7ca8ea9d23 Simplify stateful tests 2024-01-13 23:53:01 +01:00
Robert Schulze
52d75ab68f
Merge pull request #58458 from rschu1ze/replace-std_regexp-by-re2
Replace `std::regex` by re2
2024-01-10 17:46:54 +01:00
Robert Schulze
582a9cc2a7
Merge pull request #58655 from rschu1ze/docs-match
Document that `match()` can use `ngrambf_v1` and `tokenbf_v1` indexes
2024-01-10 15:43:52 +01:00
Robert Schulze
d7fbb1a4d9
Fix style 2024-01-10 13:54:52 +00:00
Robert Schulze
a0128e0f51
Merge pull request #58544 from rschu1ze/sqid-decode
Add `sqidDecode()` function
2024-01-10 13:47:13 +01:00
Robert Schulze
499227b9cf
Merge remote-tracking branch 'rschu1ze/master' into replace-std_regexp-by-re2 2024-01-10 10:02:53 +00:00
Robert Schulze
4cacfa6b43
Merge pull request #57509 from jinjunzh/qatzstd_main
Add Intel-QAT-based `ZSTD_QAT` compression codec
2024-01-10 10:31:39 +01:00
Kseniia Sumarokova
1b517105db
Merge pull request #57323 from valbok/mysql-binlog-client
MaterializedMySQL: Introduce MySQL Binlog Client
2024-01-10 09:08:13 +01:00
Robert Schulze
ced9e93ac6
Merge remote-tracking branch 'rschu1ze/master' into qatzstd_main 2024-01-09 20:36:02 +00:00
Robert Schulze
69ba1a706a
Merge remote-tracking branch 'rschu1ze/master' into string-similarity 2024-01-09 15:22:52 +00:00
Robert Schulze
64ab30f446
Merge pull request #57078 from ClibMouse/seriesDecompose
Seasonal decompose of time series into seasonal, trend and residue components
2024-01-09 11:18:13 +01:00
Robert Schulze
a2a86caf0f
Fix style, pt. II 2024-01-08 13:45:46 +00:00
Jayme Bird
4f7e4fef40 ignore spelling srcReplicas 2024-01-08 12:58:26 +00:00
Robert Schulze
f553b55e3a
Merge remote-tracking branch 'rschu1ze/master' into regex-std-re2 2024-01-07 22:31:35 +00:00
Robert Schulze
8e804487f3
Some fixups 2024-01-07 22:28:08 +00:00
Robert Schulze
14cda082d2
Fix style 2024-01-07 18:08:52 +00:00
Robert Schulze
0f51e32bb5
Rename from ZSTDQPL to ZSTD_QPL 2024-01-07 17:47:31 +00:00
Robert Schulze
53965bb9f1
Merge remote-tracking branch 'rschu1ze/master' into qatzstd_main 2024-01-07 14:11:22 +00:00
Robert Schulze
5f105a56af
Fix style 2024-01-07 08:34:41 +00:00
Robert Schulze
dd2d9ff168
Minor fixes 2024-01-07 08:25:19 +00:00
Robert Schulze
d54e500832
Merge remote-tracking branch 'rschu1ze/master' into punycode-revert-revert 2024-01-07 08:08:46 +00:00
Robert Schulze
1e5f7ff80d
Merge remote-tracking branch 'rschu1ze/master' into sqid-decode 2024-01-07 07:55:06 +00:00
Robert Schulze
44b9547fba
Rename decode/encodeSqid() to sqidDecode/Encode() 2024-01-05 21:40:35 +00:00
robot-clickhouse
5b9cc914db Update version_date.tsv and changelogs after v23.8.9.54-lts 2024-01-05 14:11:11 +00:00
Robert Schulze
a813431fcb
Fix spelling 2024-01-03 20:09:12 +00:00
Robert Schulze
8d4b519bb1
Replace std::regex by re2 2024-01-03 15:06:20 +00:00
Val Doroshchuk
5c221d123d MaterializedMySQL: Introduce MySQL Binlog Client
One binlog connection for many databases.

Suggesting to disable this feature by default for now. It should be explicitly enabled by SETTINGS use_binlog_client=1.
But if you would permanently enable it in MaterializedMySQLSettings, it should keep old behavior and all tests should pass too.

1. Introduced `IBinlog` and its impl to read the binlog events from socket - `BinlogFromSocket`, or file - `BinlogFromFile`. Based on prev impl of `EventBase` and the same old binlog parsers. It fully keeps BC with old version. Fixed `./check-mysql-binlog` to test new impl.
2. Introduced `BinlogEventsDispatcher`, it reads the event from the source `IBinlog` and sends it to currently attached `IBinlog` instances.
3. Introduced `BinlogClient`, which is used to group a list of `BinlogEventsDispatcher` by MySQL binlog connection which is defined by `user:password@host:port`. All dispatchers with the same binlog position should be merged to one.
4. Introduced `BinlogClientFactory`, which is a singleton and it is used to track all binlogs created over the instance.
5. Introduced `use_binlog_client` setting to `MaterializedMySQL`, which forces to reuse a `BinlogClient` if it already exists in `BinlogClientCatalog` or create new one. By default, it is disabled.
6. Introduced `max_bytes_in_binlog_queue` setting to define the limit of bytes in binlog's queue of events. If bytes in the queue increases this limit, `BinlogEventsDispatcher` will stop reading new events from source `IBinlog` until the space for new events will be freed.
7. Introduced `max_milliseconds_to_wait_in_binlog_queue` setting to define max ms to wait when the max bytes exceeded.
7. Introduced `max_milliseconds_to_wait_in_binlog_queue` setting to define max ms to wait when the max bytes exceeded.
8. Introduced `max_bytes_in_binlog_dispatcher_buffer` setting to define max bytes in the binlog dispatcher's buffer before it is flushed to attached binlogs.
9. Introduced `max_flush_milliseconds_in_binlog_dispatcher` setting to define max milliseconds in the binlog dispatcher's buffer to wait before it is flushed to attached binlogs.
10. Introduced `system.mysql_binlogs` system table, which shows a list of active binlogs.
11. Introduced `UnparsedRowsEvent` and `MYSQL_UNPARSED_ROWS_EVENT`, which defines that an event is not parsed and should be explicitly parsed later.
12. Fixed bug when not possible to apply DDL since syntax error or unsupported SQL.

@larspars is the author of following:
`GTIDSets::contains()`
`ReplicationHelper`
`shouldReconnectOnException()`
2024-01-03 15:26:09 +01:00
Robert Schulze
2186aa8f21
Revert "Revert "Implement punycode encoding/decoding""
This reverts commit 345d29a3c1.
2024-01-03 08:03:01 +00:00
Bhavna Jindal
6d58c99408 fixed the failing fasttest and style-check 2024-01-02 08:52:28 -08:00
Bhavna Jindal
a407cd270d Added new library stl-cpp for sesonal decompose of time series data 2024-01-02 08:48:10 -08:00
Alexey Milovidov
78523639e0 Remove function 2023-12-30 14:52:54 +01:00
Alexey Milovidov
96876aa4f4
Merge pull request #58345 from ClickHouse/dos-windows
Add a style check against DOS and Windows
2023-12-29 15:02:46 +01:00
Alexey Milovidov
4843d5d099
Update check-style 2023-12-29 16:29:30 +03:00
Alexey Milovidov
48a397577c Add a style check against DOS and Windows 2023-12-29 13:10:40 +01:00
Michael Kolupaev
609b2c216c Fix some of the CI 2023-12-28 17:56:05 +00:00
Alexander Tokmakov
38fe70c68a
Revert "Refreshable materialized views (takeover)" 2023-12-28 13:12:20 +01:00
robot-clickhouse
27657a3eb6 Update version_date.tsv and changelogs after v23.12.1.1368-stable 2023-12-28 10:32:09 +00:00
Alexey Milovidov
a2faa65b08
Merge pull request #56946 from ClickHouse/mv
Refreshable materialized views (takeover)
2023-12-28 09:29:44 +01:00
Alexey Milovidov
345d29a3c1
Revert "Implement punycode encoding/decoding" 2023-12-28 09:12:22 +03:00
Michael Kolupaev
673743e2ac Fix some of the CI 2023-12-27 20:24:55 +00:00
Sema Checherinda
617ee5abaf
Merge pull request #57551 from bharatnc/ncb/async-metrics-pri-key-size
add asynchronous metrics for total primary key size in memory and allocated
2023-12-27 20:24:20 +01:00
Alexey Milovidov
d0ca383bca
Merge pull request #58095 from azat/metrics/max-rss
Add max/peak RSS into system.asynchronous_metrics
2023-12-23 13:13:44 +01:00