mirror of
https://github.com/ClickHouse/ClickHouse.git
synced 2024-11-27 18:12:02 +00:00
72 KiB
72 KiB
sidebar_position | sidebar_label |
---|---|
1 | 2022 |
2022 Changelog
ClickHouse release v22.8.1.2097-lts (09a2ff8843
) FIXME as compared to v22.7.1.2484-stable (f4f05ec786
)
Backward Incompatible Change
- Make cache composable, allow not to evict certain files (regarding idx, mrk, ..), delete old cache version. Now it is possible to configure cache over Azure blob storage disk, over Local disk, over StaticWeb disk, etc. This PR is marked backward incompatible because cache configuration changes and in order for cache to work need to update the config file. Old cache will still be used with new configuration. The server will startup fine with the old cache configuration. Closes #36140. Closes #37889. #36171 (Kseniia Sumarokova).
- Now, all relevant dictionary sources respect
remote_url_allow_hosts
setting. It was already done for HTTP, Cassandra, Redis. Added ClickHouse, MongoDB, MySQL, PostgreSQL. Host is checked only for dictionaries created from DDL. #39184 (Nikolai Kochetov). - Extended range of Date32 and DateTime64 to support dates from the year 1900 to 2299. In previous versions, the supported interval was only from the year 1925 to 2283. The implementation is using the proleptic Gregorian calendar (which is conformant with ISO 8601:2004 (clause 3.2.1 The Gregorian calendar)) instead of accounting for historical transitions from the Julian to the Gregorian calendar. This change affects implementation-specific behavior for out-of-range arguments. E.g. if in previous versions the value of
1899-01-01
was clamped to1925-01-01
, in the new version it will be clamped to1900-01-01
. It changes the behavior of rounding withtoStartOfInterval
if you passINTERVAL 3 QUARTER
up to one quarter because the intervals are counted from an implementation-specific point of time. Closes #28216, improves #38393. #39425 (Roman Vasin).
New Feature
- Added a setting
exact_rows_before_limit
(0/1). When enabled, ClickHouse will provide exact value forrows_before_limit_at_least
statistic, but with the cost that the data before limit will have to be read completely. This closes #6613. #25333 (kevin wan). - Add SLRU cache policy for uncompressed cache and marks cache. #34651 (alexX512).
- Intel® In-Memory Analytics Accelerator (Intel® IAA) is a hardware accelerator available in the upcoming generation of Intel® Xeon® Scalable processors ("Sapphire Rapids"). Its goal is to speed up common operations in analytics like data (de)compression and filtering. ClickHouse gained the new "DeflateQpl" compression codec which utilizes the Intel® IAA offloading technology to provide a high-performance DEFLATE implementation. The codec uses the Intel® Query Processing Library (QPL) which abstracts access to the hardware accelerator, respectively to a software fallback in case the hardware accelerator is not available. DEFLATE provides in general higher compression rates than ClickHouse's LZ4 default codec, and as a result, offers less disk I/O and lower main memory consumption. #36654 (jasperzhu).
- Add concurrent_threads_soft_limit parameter to increase performance in case of high RPS by means of limiting total number of threads for all queries. #37285 (Roman Vasin).
- Added concurrency control logic to limit total number of concurrent threads created by queries. #37558 (Sergei Trifonov).
- Added support for parallel distributed insert select into tables with Distributed and Replicated engine #34670. #39107 (Nikita Mikhaylov).
- Add new settings to control schema inference from text formats: -
input_format_try_infer_dates
- try infer dates from strings. -input_format_try_infer_datetimes
- try infer datetimes from strings. -input_format_try_infer_integers
- try inferInt64
instead ofFloat64
. -input_format_json_try_infer_numbers_from_strings
- try infer numbers from json strings in JSON formats. #39186 (Kruglov Pavel). - This feature will provide JSON formatted log output in console. The purpose is to allow easier ingestion and query in log analysis tools. #39277 (Mallik Hassan).
- Intel® In-Memory Analytics Accelerator (Intel® IAA) is a hardware accelerator available in the upcoming generation of Intel® Xeon® Scalable processors ("Sapphire Rapids"). Its goal is to speed up common operations in analytics like data (de)compression and filtering. ClickHouse gained the new "DeflateQpl" compression codec which utilizes the Intel® IAA offloading technology to provide a high-performance DEFLATE implementation. The codec uses the Intel® Query Processing Library (QPL) which abstracts access to the hardware accelerator, respectively to a software fallback in case the hardware accelerator is not available. DEFLATE provides in general higher compression rates than ClickHouse's LZ4 default codec, and as a result, offers less disk I/O and lower main memory consumption. #39494 (Robert Schulze).
- Add function
nowInBlock
which allows getting the current time during long-running and continuous queries. Closes #39522. Notes: there are no functionsnow64InBlock
neithertodayInBlock
. #39533 (Alexey Milovidov). -
- Add result_rows and result_bytes to progress reports (
X-ClickHouse-Summary
). #39567 (Raúl Marín).
- Add result_rows and result_bytes to progress reports (
- adds ability to specify settings for an
executable()
table function. #39681 (Constantine Peresypkin). - Implemented automatic conversion of database engine from
Ordinary
toAtomic
. Create emptyconvert_ordinary_to_atomic
file inflags
directory and allOrdinary
databases will be converted automatically on next server start. Resolves #39546. #39933 (Alexander Tokmakov). - Add new setting
schema_inference_hints
that allows to specify structure hints in schema inference for specific columns. Closes #39569. #40068 (Kruglov Pavel).
Performance Improvement
-
- Break on analyze stuck on complex query. #38185 (Vladimir C).
- Deduce way to sort based on input stream sort description. Skip sorting if input stream is already sorted. #38719 (Igor Nikonov).
DISTINCT
in order withORDER BY
improves memory usage (significantly) and query execution time ifDISTINCT
columns match (or form a prefix of)ORDER BY
columns. #39432 (Igor Nikonov).- Use local node as first priority to get structure of remote table when executing
cluster
and similar table functions. #39440 (Mingliang Pan). - Use
DistinctSortedTransform
only when sort description is applicable to DISTINCT columns, otherwise fall back to ordinary DISTINCT implementation. It allows making less checks duringDistinctSortedTransform
execution. #39528 (Igor Nikonov). DistinctSortedTransform
didn't take advantage of sorting, i.e. it worked like ordinaryDISTINCT
implementation. The fix reduces memory usage significantly. #39538 (Igor Nikonov).- ColumnVector: optimize filter with AVX512VBMI2 compress store. #39633 (Guo Wangyang).
- KeyCondition: optimize applyFunction in multi-thread scenario. #39812 (Guo Wangyang).
- For systems with AVX512 VBMI2, this PR improves performance by ca. 6% for SSB benchmark queries queries 3.1, 3.2 and 3.3 (SF=100). Tested on Intel Icelake Xeon 8380 * 2 socket. #40033 (Robert Schulze).
-
- Don't visit the AST for UDFs if none are registered. #40069 (Raúl Marín).
-
- Optimize CurrentMemoryTracker alloc and free. #40078 (Raúl Marín).
Improvement
- Change the way how PK is analyzed for MergeTree. #25563 (Nikolai Kochetov).
-
- Improved structure of DDL query result table for
Replicated
database (separate columns with shard and replica name, more clear status) -CREATE TABLE ... ON CLUSTER
queries can be normalized on initiator first ifdistributed_ddl_entry_format_version
is set to 3 (default value). It means thatON CLUSTER
queries may not work if initiator does not belong to the cluster that specified in query. Fixes #37318, #39500 - IgnoreON CLUSTER
clause if database isReplicated
and cluster name equals to database name. Related to #35570 - Miscellaneous minor fixes forReplicated
database engine - Check metadata consistency when starting upReplicated
database, start replica recovery in case of mismatch of local metadata and metadata in Keeper. Resolves #24880. #37198 (Alexander Tokmakov).
- Improved structure of DDL query result table for
- Support SQL standard DELETE FROM syntax on merge tree tables and lightweight delete implementation for merge tree families. #37893 (Jianmei Zhang).
timeSlots
now works with DateTime64; subsecond duration and slot size available when working with DateTime64. #37951 (Andrey Zvonov).- Add cache for schema inference for file/s3/hdfs/url table functions. Now, schema inference will be performed only on the first query to the file, all subsequent queries to the same file will use the schema from cache if data wasn't changed. Add system table
system.schema_inference_cache
with all current schemas in cache and system queriesSYSTEM DROP SCHEMA CACHE [FOR FILE/S3/HDFS/URL]
to drop schemas from cache. #38286 (Kruglov Pavel). -
- Added support of
LEFT SEMI
andLEFT ANTI
direct join with rocksdb. #38956 (Vladimir C).
- Added support of
- resolves #37490. #39054 (SmitaRKulkarni).
- Store Keeper API version inside a predefined path. #39096 (Antonio Andelic).
- Now entrypoint.sh in docker image creates and executes chown for all folders it found in config for multidisk setup #17717. #39121 (Nikita Mikhaylov).
- Add profile events for fsync. #39179 (Azat Khuzhin).
- Add the second argument to the ordinary function
file(path[, default])
, which function returns in the case when a file does not exists. #39218 (Nikolay Degterinsky). - Some small fixes for reading via http, allow to retry partial content in case if got 200OK. #39244 (Kseniia Sumarokova).
- Improved Base58 encoding/decoding. #39292 (Andrey Zvonov).
- Normalize
AggregateFunction
types and state representations because optimizations like https://github.com/ClickHouse/ClickHouse/pull/35788 will treatcount(not null columns)
ascount()
, which might confuses distributed interpreters with the following error :Conversion from AggregateFunction(count) to AggregateFunction(count, Int64) is not supported
. #39420 (Amos Bird). - Improved memory usage during memory efficient merging of aggregation results. #39429 (Nikita Taranov).
- Support queries
CREATE TEMPORARY TABLE ... (<list of columns>) AS ...
. #39462 (Kruglov Pavel). - Add support of
!
/*
(exclamation/asterisk) in custom TLDs (cutToFirstSignificantSubdomainCustom()
/cutToFirstSignificantSubdomainCustomWithWWW()
/firstSignificantSubdomainCustom()
). #39496 (Azat Khuzhin). - Rework and simplify the
system.backups
table, remove theinternal
column, allow user to set ID of operation, add columnsnum_files
,uncompressed_size
,compressed_size
,start_time
,end_time
. #39503 (Vitaly Baranov). - Refactored a little code, removed duplicate code. #39509 (Simon Liu).
- Add support for TLS connections to NATS. Implements #39525. #39527 (Constantine Peresypkin).
clickhouse-obfuscator
(a tool for database obfuscation for testing and load generation) now has the new--save
and--load
parameters to work with pre-trained models. This closes #39534. #39541 (Alexey Milovidov).- Fix incorrect behavior of log rotation during restart. #39558 (Nikolay Degterinsky).
- Improve bytes to bits mask transform for SSE/AVX/AVX512. #39586 (Guo Wangyang).
- Add formats
PrettyMonoBlock
,PrettyNoEscapesMonoBlock
,PrettyCompactNoEscapes
,PrettyCompactNoEscapesMonoBlock
,PrettySpaceNoEscapes
,PrettySpaceMonoBlock
,PrettySpaceNoEscapesMonoBlock
. #39646 (Kruglov Pavel). - Fix building aggregate projections when external aggregation is on. Mark as improvement because the case is rare and there exists easy workaround to fix it via changing settings. This fixes #39667 . #39671 (Amos Bird).
- Allow to execute hash functions with arguments of type
Map
. #39685 (Anton Popov). - Add a configuration parameter to hide addresses in stack traces. It may improve security a little but generally, it is harmful and should not be used. #39690 (Alexey Milovidov).
- change the prefix size of AggregateFunctionDistinct to make sure nested function data memory aligned. #39696 (Pxl).
- Properly escape credentials passed to the
clickhouse-diagnostic
tool. #39707 (Dale McDiarmid). - keeper-improvement: create a snapshot on exit. It can be controlled with the config
keeper_server.create_snapshot_on_exit
,true
by default. #39755 (Antonio Andelic). - Support primary key analysis for
row_policy_filter
andadditional_filter
. It also helps fix issues like #37454 . #39826 (Amos Bird). - Parameters are now transferred in
Query
packets right after the query text in the same serialisation format as the settings. #39906 (Nikita Taranov). - Fix two usability issues in Play UI: - it was non-pixel-perfect on iPad due to parasitic border radius and margins; - the progress indication did not display after the first query. This closes #39957. This closes #39960. #39961 (Alexey Milovidov).
- Play UI: add row numbers; add cell selection on click; add hysteresis for table cells. #39962 (Alexey Milovidov).
- The client will show server-side elapsed time. This is important for the performance comparison of ClickHouse services in remote datacenters. This closes #38070. See also this for motivation. #39968 (Alexey Milovidov).
- Adds
parseDateTime64BestEffortUS
,parseDateTime64BestEffortUSOrNull
,parseDateTime64BestEffortUSOrZero
functions, closing #37492. #40015 (Tanya Bragin). -
- Add observer mode to (zoo)keeper cluster discovery feature. In this mode node itself doesn't belong to cluster. #40035 (Vladimir C).
- Play UI: recognize tab key in textarea, but at the same time don't mess up with tab navigation. #40053 (Alexey Milovidov).
- Extend processors_profile_log with more information such as input rows. #40121 (Amos Bird).
- Update tzdata to 2022b to support the new timezone changes. See https://github.com/google/cctz/pull/226. Chile's 2022 DST start is delayed from September 4 to September 11. Iran plans to stop observing DST permanently, after it falls back on 2022-09-21. There are corrections of the historical time zone of Asia/Tehran in the year 1977: Iran adopted standard time in 1935, not 1946. In 1977 it observed DST from 03-21 23:00 to 10-20 24:00; its 1978 transitions were on 03-24 and 08-05, not 03-20 and 10-20; and its spring 1979 transition was on 05-27, not 03-21 (https://data.iana.org/time-zones/tzdb/NEWS). #40184 (Alexey Milovidov).
- Display server-side time in
clickhouse-benchmark
by default if it is available (since ClickHouse version 22.8). This is needed to correctly compare the performance of clouds. This behavior can be changed with the new--client-side-time
command line option. Change the--randomize
command line option from--randomize 1
to the form without argument. #40193 (Alexey Milovidov). - Add counters (ProfileEvents) for cases when query complexity limitation has been set and has reached (a separate counter for
overflow_mode
=break
andthrow
). For example, if you have set upmax_rows_to_read
withread_overflow_mode = 'break'
, looking at the value ofOverflowBreak
counter will allow distinguishing incomplete results. #40205 (Alexey Milovidov). - Fix memory accounting in case of MEMORY_LIMIT_EXCEEDED errors (previously [peak] memory usage was takes failed allocations into account). #40249 (Azat Khuzhin).
- Add current metrics for fs cache:
FilesystemCacheSize
andFilesystemCacheElements
. #40260 (Kseniia Sumarokova). - Add support for LARGE_BINARY/LARGE_STRING with Arrow (Closes #32401). #40293 (Josh Taylor).
Bug Fix
- Support hadoop secure rpc transfer(hadoop.rpc.protection=privacy and hadoop.rpc.protection=integrity). #39411 (michael1589).
- Fix seeking while reading from encrypted disk. This PR fixes #38381. #39687 (Vitaly Baranov).
-
- Fix duplicate columns in join plan. Finally, solve #26809. #40009 (Vladimir C).
Build/Testing/Packaging Improvement
- Prebuild ClickHouse x86 binaries now require support for AVX instructions, i.e. a CPU not older than Intel Sandy Bridge / AMD Bulldozer, both released in 2011. #39000 (Robert Schulze).
- Former packages used to install systemd.service file to
/etc
. The files there are marked asconf
and are not cleaned out, and not updated automatically. This PR cleans them out. #39323 (Mikhail f. Shiryaev). - Fix LSan by fixing getauxval(). #39430 (Azat Khuzhin).
- TSAN has issues with clang-14 (https://github.com/google/sanitizers/issues/1552, https://github.com/google/sanitizers/issues/1540), so here we temporary build the TSAN binaries with clang-13. #39450 (Mikhail f. Shiryaev).
- Remove the option to build ClickHouse tools as separate executable programs. This fixes #37847. #39520 (Alexey Milovidov).
- Fixed Unit tests for wide integers on s390x. #39627 (Harry Lee).
- Increase max cache size for clang-tidy builds. Try to avoid flushing it out between builds. #39652 (Nikita Taranov).
- No need to use fixed IP when you are using cluster with SSL. Using the same fixed IP could trigger collision between tests. At this change the server's certificate is generated for a designated host name (see server-ext.cnf at each test). The client should check server's certificate against that name accordingly. #40007 (Sema Checherinda).
- Support build with
clang-16
(trunk). This closes #39949. #40181 (Alexey Milovidov). - Prepare RISC-V 64 build to run in CI. This is for #40141. #40197 (Alexey Milovidov).
Bug Fix (user-visible misbehavior in official stable release)
- Fixed query hanging for SELECT with ORDER BY WITH FILL with different date/time types. #37849 (Yakov Olkhovskiy).
- Fix ORDER BY that matches projections ORDER BY (before it simply returns unsorted result). #38725 (Azat Khuzhin).
- Do not optimise functions in GROUP BY statements if they shadow one of the table columns or expressions. Fixes #37032. #39103 (Anton Kozlov).
- Fix wrong table name in logs after RENAME TABLE. This fixes #38018. #39227 (Amos Bird).
- Fix positional arguments in case of columns pruning when optimising the query. Closes #38433. #39293 (Kseniia Sumarokova).
- Fix bug in schema inference in case of empty messages in Protobuf/CapnProto formats that allowed to create column with empty
Tuple
type. Closes #39051 Add 2 new settingsinput_format_{protobuf/capnproto}_skip_fields_with_unsupported_types_in_schema_inference
that allow to skip fields with unsupported types while schema inference for Protobuf and CapnProto formats. #39357 (Kruglov Pavel). - Fix segmentation fault on
CREATE WINDOW VIEW .. ON CLUSTER ... INNER
. Closes #39363. #39384 (Kseniia Sumarokova). - Fix WriteBuffer finalize when cancel insert into function. Proper version of https://github.com/ClickHouse/ClickHouse/pull/39396 that was reverted. #39458 (Kruglov Pavel).
- Fix storing of columns of type
Object
in sparse serialization. #39464 (Anton Popov). - Fix possible "Not found column in block" exception when using projections. This closes #39469. #39470 (小路).
- Fix LOGICAL_ERROR on race between DROP and INSERT with materialized views. #39477 (Azat Khuzhin).
- Fix data race and possible heap-buffer-overflow in Avro format. Closes #39094 Closes #33652. #39498 (Kruglov Pavel).
- Fix rare bug in asynchronous reading (with setting
local_filesystem_read_method='pread_threadpool'
) with enabledO_DIRECT
(enabled by settingmin_bytes_to_use_direct_io
). #39506 (Anton Popov). - Fixes "Code: 49. DB::Exception: FunctionFactory: the function name '' is not unique. (LOGICAL_ERROR)" observed on FreeBSD when starting clickhouse. #39551 (Alexander Gololobov).
- Fix bug with maxsplit argument for splitByChar, which was not working correctly. #39552 (filimonov).
-
- Fix bug in ASOF JOIN with
enable_optimize_predicate_expression
, close #37813. #39556 (Vladimir C).
- Fix bug in ASOF JOIN with
- Fixed
CREATE/DROP INDEX
query withON CLUSTER
orReplicated
database andReplicatedMergeTree
. It used to be executed on all replicas (causing error or DDL queue stuck). Fixes #39511. #39565 (Alexander Tokmakov). - Fix "column not found" error for push down with join, close #39505. #39575 (Vladimir C).
- Fix the wrong
REGEXP_REPLACE
alias. This fixes https://github.com/ClickHouse/ClickBench/issues/9. #39592 (Alexey Milovidov). - Fixed point of origin for exponential decay window functions to the last value in window. Previously, decay was calculated by formula
exp((t - curr_row_t) / decay_length)
, which is incorrect when right boundary of window is notCURRENT ROW
. It was changed to:exp((t - last_row_t) / decay_length)
. There is no change in results for windows withROWS BETWEEN (smth) AND CURRENT ROW
. #39593 (Vladimir Chebotaryov). - Fix Decimal division overflow, which can be detected based on operands scale. #39600 (Andrey Zvonov).
- Fix settings
output_format_arrow_string_as_string
andoutput_format_arrow_low_cardinality_as_dictionary
work in combination. Closes #39624. #39647 (Kruglov Pavel). - Fixed a bug in default database resolution in distributed table reads. #39674 (Anton Kozlov).
- Select might read data of dropped table if cache for mmap IO is used and database engine is Ordinary and new tables was created with the same name as dropped one had. It's fixed. #39708 (Alexander Tokmakov).
- Fix possible error
Invalid column type for ColumnUnique::insertRangeFrom. Expected String, got ColumnLowCardinality
Fixes #38460. #39716 (Arthur Passos). - Field names in the
meta
section of JSON format were erroneously double escaped. This closes #39693. #39747 (Alexey Milovidov). - Fix wrong index analysis with tuples and operator
IN
, which could lead to wrong query result. #39752 (Anton Popov). - Fix EmbeddedRocksDB filtering by key using params. #39757 (Antonio Andelic).
- Fix error
Invalid number of columns in chunk pushed to OutputPort
which was cause by ARRAY JOIN optimization. Fixes #39164. #39799 (Nikolai Kochetov). - Fix
CANNOT_READ_ALL_DATA
exception withlocal_filesystem_read_method=pread_threadpool
. This bug affected only Linux kernel version 5.9 and 5.10 according to man. #39800 (Anton Popov). - Fix quota_key application on connect. #39874 (Yakov Olkhovskiy).
- we meeted query exceptions: DB::Exception: Cannot open file /media/ssd1/fordata/clickhouse/data/data/perf/perf_log_local_v3_1/20220618_17233_17238_1/namespace.dict.bin, errno: 24, strerror: Too many open files. #39886 (Fangyuan Deng).
- fix broken NFS mkdir for root-squashed volumes. #39898 (Constantine Peresypkin).
- Remove dictionaries from prometheus metrics on DETACH/DROP. #39926 (Azat Khuzhin).
- Fix read of StorageFile with virtual columns. Closes #39907. #39943 (flynn).
- Fix big memory usage during fetches. Fixes #39915. #39990 (Nikolai Kochetov).
-
- Fix hashId crash and salt parameter not being used. #40002 (Raúl Marín).
- fix HashMethodOneNumber get wrong key value when column is const. #40020 (Duc Canh Le).
- Fixed "Part directory doesn't exist" and "
tmp_<part_name>
... No such file or directory" errors during too slow INSERT or too long merge/mutation. Also fixed issue that may cause some replication queue entries to stuck without any errors or warnings in logs if previous attempt to fetch part failed, buttmp-fetch_<part_name>
directory was not cleaned up. #40031 (Alexander Tokmakov). - Fix rare cases of parsing of arrays of tuples in format
Values
. #40034 (Anton Popov). - Fixes ArrowColumn format Dictionary(X) & Dictionary(Nullable(X)) conversion to ClickHouse LowCardinality(X) & LowCardinality(Nullable(X)) respectively. #40037 (Arthur Passos).
- Fix potential deadlock in WriteBufferFromS3 during task scheduling failure. #40070 (Maksim Kita).
- Fix bug in collectFilesToSkip() by adding correct file extension(.idx or idx2) for indexes to be recalculated, avoid wrong hard links. Fixed #39896. #40095 (Jianmei Zhang).
- A segmentation fault that has CaresPTRResolver::resolve in the stack trace has been reported:. #40134 (Arthur Passos).
- Fix a very rare case of incorrect behavior of array subscript operator. This closes #28720. #40185 (Alexey Milovidov).
- Fix insufficient argument check for encryption functions (found by query fuzzer). This closes #39987. #40194 (Alexey Milovidov).
-
- Fix unexpected result arrayDifference of Array(UInt32). #40211 (Duc Canh Le).
- Fix the case when the order of columns can be incorrect if the
IN
operator is used with a table withENGINE = Set
containing multiple columns. This fixes #13014. #40225 (Alexey Milovidov). - Fix possible segfault in CapnProto input format. This bug was found and send through ClickHouse bug-bounty program by kiojj. #40241 (Kruglov Pavel).
-
- Avoid continuously growing memory consumption of pattern cache when using functions multi(Fuzzy)Match(Any|AllIndices|AnyIndex)(). #40264 (Robert Schulze).
Build
- Fix build error:
[ 69%] Building CXX object src/CMakeFiles/clickhouse_common_io.dir/Common/waitForPid.cpp.o /CLionProjects/clickhouse-yandex/src/Common/waitForPid.cpp:112:5: error: identifier '__kevp__' is reserved because it starts with '__' [-Werror,-Wreserved-identifier] EV_SET(&change, pid, EVFILT_PROC, EV_ADD, NOTE_EXIT, 0, NULL); ^ /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/sys/event.h:108:17: note: expanded from macro 'EV_SET' struct kevent *__kevp__ = (kevp); \ ^
. #39493 (小路).
Build Improvement
- Fixed Endian issue in BitHelpers for s390x. #39656 (Harry Lee).
- Implement a piece of code related to SipHash for s390x architecture (which is not supported by ClickHouse). #39732 (Harry Lee).
- Fixed an Endian issue in Coordination snapshot code for s390x architecture (which is not supported by ClickHouse). #39931 (Harry Lee).
- Fixed Endian issues in Codec code for s390x architecture (which is not supported by ClickHouse). #40008 (Harry Lee).
- Fixed Endian issues in reading/writing BigEndian binary data in ReadHelpers and WriteHelpers code for s390x architecture (which is not supported by ClickHouse). #40179 (Harry Lee).
NO CL ENTRY
- NO CL ENTRY: 'Revert "tests: enable back 02232_dist_insert_send_logs_level_hung"'. #39788 (Alexander Tokmakov).
- NO CL ENTRY: 'Revert "Update arrow to fix possible data race"'. #39804 (Alexander Tokmakov).
- NO CL ENTRY: 'Revert "Revert "Update arrow to fix possible data race""'. #39811 (Kruglov Pavel).
- NO CL ENTRY: 'Revert "Limit number of analyze for one query"'. #39816 (Alexander Tokmakov).
- NO CL ENTRY: 'Revert "Revert "tests: enable back 02232_dist_insert_send_logs_level_hung""'. #39817 (Alexander Tokmakov).
- NO CL ENTRY: 'Prepare library-bridge for catboost integration'. #39904 (Robert Schulze).
- NO CL ENTRY: 'Revert "ColumnVector: optimize filter with AVX512VBMI2 compress store"'. #39963 (Alexey Milovidov).
- NO CL ENTRY: 'Revert "copy self-extracting to output"'. #40005 (Alexander Tokmakov).
- NO CL ENTRY: 'Revert "Use separate counter for RSS in global memory tracker."'. #40199 (Alexey Milovidov).
- NO CL ENTRY: 'Revert "tests/performance: cover sparse_hashed dictionary"'. #40268 (Alexander Tokmakov).
NOT FOR CHANGELOG / INSIGNIFICANT
- Test/insert deduplication token materialized views #34662 (Denny Crane).
- Merging #34372 #35968 (Nikita Mikhaylov).
- Decoupling local cache function and cache algorithm #38048 (Han Shukai).
- Use separate counter for RSS in global memory tracker. #38682 (Nikolai Kochetov).
- Build self-extracting-executable utils #38936 (Yakov Olkhovskiy).
- Improvements in integration tests #38978 (Ilya Yatsishin).
- More readable regexp in
test_quota
#39084 (Vladimir Chebotaryov). - Fixed regexp in
test_match_process_uid_against_data_owner
#39085 (Vladimir Chebotaryov). - tests: enable back 02232_dist_insert_send_logs_level_hung #39124 (Azat Khuzhin).
- Add connection info for Distributed sends log message #39178 (Azat Khuzhin).
- Forbid defining non-default disk with default path from #39183 (Azat Khuzhin).
- Fix LZ4 decompression issue for s390x #39195 (Harry Lee).
- Do not report "Failed communicating with" on and on for parts exchange #39222 (Azat Khuzhin).
- Improve logging around replicated merges #39230 (Raúl Marín).
- Cleanup logic around join_algorithm setting, add docs #39271 (Vladimir C).
- Possible fix for flaky
test_keeper_force_recovery
#39321 (Antonio Andelic). - tests/performance: improve parallel_mv test #39325 (Azat Khuzhin).
- Update azure library (removed "harmful" function) #39327 (Kseniia Sumarokova).
- Refactor PreparedSets/SubqueryForSet #39343 (Vladimir C).
- Small doc updates #39362 (Robert Schulze).
- Even less usage of StringRef #39364 (Robert Schulze).
- Automatic fixes for black formatting for domestic repo PRs #39390 (Mikhail f. Shiryaev).
- Clickhouse-local fixes #39404 (Heena Bansal).
- Uppercase
ROWS
,GROUPS
,RANGE
in queries with windows #39410 (Vladimir Chebotaryov). - GitHub helper #39421 (Mikhail f. Shiryaev).
- ShellCommand wait pid refactoring #39426 (Maksim Kita).
- Require clear style check to continue building #39428 (Mikhail f. Shiryaev).
- DirectDictionary improve performance of dictHas with duplicate keys #39449 (Maksim Kita).
- Commit status names: remove "actions" #39454 (Mikhail f. Shiryaev).
- Improve synchronization between hosts in distributed backup and fix locks #39455 (Vitaly Baranov).
- Remove some dead and commented code #39460 (Alexander Tokmakov).
- Add Build Check and Special Build Check to SimpleCheck #39467 (Yakov Olkhovskiy).
- Update version after release #39474 (Mikhail f. Shiryaev).
- Update version_date.tsv and changelogs after v22.7.1.2484-stable #39475 (github-actions[bot]).
- Update README.md #39478 (Dan Roscigno).
- Remove unused constructor #39491 (alesapin).
- Mark new codec DEFLATE_QPL as experimental + cosmetics #39495 (Robert Schulze).
- Update arrow to fix possible data race #39510 (Kruglov Pavel).
- fix
-DENABLE_EXAMPLES=1
in master #39517 (Constantine Peresypkin). - LZ4_decompress_faster.cpp: remove endianness-dependent code #39523 (Ignat Loskutov).
- Fix 02286_parallel_final #39524 (Nikita Taranov).
- add Equinix metal N3 Xlarge #39532 (Tyler Hannan).
- Less usage of StringRef #39535 (Robert Schulze).
- Follow up to #37827 #39557 (Alexander Tokmakov).
- Temporarily disable all tests with MaterializedPostgreSQL #39564 (Alexander Tokmakov).
- Update version_date.tsv after v22.3.9.19-lts #39576 (github-actions[bot]).
- free compression and decompression contexts #39578 (Yakov Olkhovskiy).
- Update version_date.tsv and changelogs after v22.6.4.35-stable #39579 (github-actions[bot]).
- Merge Woboq code browser page into "Getting Started" document #39596 (Robert Schulze).
- Fix Chain::addSink #39601 (Nikolai Kochetov).
- Update NuRaft to latest master #39609 (Antonio Andelic).
- copy self-extracting to output #39617 (Yakov Olkhovskiy).
- Replace MemoryTrackerBlockerInThread to LockMemoryExceptionInThread #39619 (Nikolai Kochetov).
- Combining sumIf->countIf and multiIf->if opt. #39621 (Amos Bird).
- Update README.md #39622 (Ivan Blinkov).
- Disable 02327_capnproto_protobuf_empty_messages with Ordinary #39623 (Alexander Tokmakov).
- add Dell PowerEdge R740XD results #39625 (Tyler Hannan).
- Attempt to fix wrong workflow_run data for rerun #39630 (Mikhail f. Shiryaev).
- Run tests with Replicated database in master #39653 (Alexander Tokmakov).
- Rollback request in Keeper if storing log fails #39673 (Antonio Andelic).
- Fix utils build on CI #39679 (Azat Khuzhin).
- Add duration_ms into system.zookeeper_log #39686 (Azat Khuzhin).
- Fix DISTINCT: handle all const columns case correctly #39688 (Igor Nikonov).
- Update README.md #39692 (Yuko Takagi).
- Update Keeper version for digest #39698 (Antonio Andelic).
- Change mysql-odbc url #39702 (Mikhail f. Shiryaev).
- Avoid recursive destruction of AST. #39705 (Nikolai Kochetov).
- Update ccache to the latest available version #39709 (Mikhail f. Shiryaev).
- Join enums refactoring #39718 (Maksim Kita).
- Fix flaky test
02360_send_logs_level_colors
#39720 (Anton Popov). - Fix cherry-pick for cases, when assignee is not set for PR #39723 (Mikhail f. Shiryaev).
- Jepsen label #39730 (Mikhail f. Shiryaev).
- Fix redirecting of logs to stdout in clickhouse-client #39731 (Anton Popov).
- CI: refactor Simple Check, use statuses to make it stateful #39735 (Yakov Olkhovskiy).
- Use different root path for total-queue Jepsen test #39738 (Antonio Andelic).
- Simple refactoring: ordinary DISTINCT implementation #39740 (Igor Nikonov).
- Cleanup usages of
allow_experimental_projection_optimization
setting, part 1 #39746 (Alexey Milovidov). - Enable SQL function getOSKernelVersion() on all platforms #39751 (Robert Schulze).
- Try clang-15 for build with tsan #39758 (Alexander Tokmakov).
- Rename "splitted build" to "shared libraries build" in CI tools #39759 (Robert Schulze).
- Use std::popcount, ::countl_zero, ::countr_zero functions #39760 (Robert Schulze).
- Self-extracting - run resulting executable with execvp #39763 (Yakov Olkhovskiy).
- Fix non-deterministic queries in distinct_in_order test #39772 (Igor Nikonov).
- Fix some flaky integration tests #39775 (Alexander Tokmakov).
- Retry inserts with ClickHouseHelper #39780 (Mikhail f. Shiryaev).
- Add cloudflare DNS as a fallback #39795 (Mikhail f. Shiryaev).
- Update README.md #39796 (Yuko Takagi).
- Minor fix for Stress Tests #39798 (Alexander Tokmakov).
- Typos #39813 (Robert Schulze).
- Update settings changes history #39839 (Kruglov Pavel).
- Fix post-build script for building utils/self-extracting-executable/compressor #39843 (Yakov Olkhovskiy).
- Add hasJoin method into ASTSelectQuery #39850 (Maksim Kita).
- Update tweak on version part update #39853 (Mikhail f. Shiryaev).
- Update version_date.tsv and changelogs after v22.7.2.15-stable #39854 (github-actions[bot]).
- Fix typo and extra dots in exception messages from OverCommitTracker #39858 (Nikita Mikhaylov).
- Fix flaky integration test test_async_backups_to_same_destination. #39859 (Vitaly Baranov).
- Better total part size calculation on mutation #39860 (alesapin).
- typo: PostgerSQL -> PostgreSQL #39861 (nathanbegbie).
- Remove prefer_localhost_replica from test #39862 (Igor Nikonov).
- Block memory tracker in Keeper during commit #39867 (Antonio Andelic).
- Update version_date.tsv after v22.3.10.22-lts #39868 (github-actions[bot]).
- fix incorrect format for functions with settings #39869 (Constantine Peresypkin).
- Get api url from event, not from const/ENV #39871 (Mikhail f. Shiryaev).
- Cleanup unused dirs from
store/
on all disks #39872 (Alexander Tokmakov). - Update 02354_distributed_with_external_aggregation_memory_usage.sql #39893 (Alexander Tokmakov).
- Fix the race between waitMutation and updating local queue from ZK #39900 (Alexander Gololobov).
- Improve 02354_distributed_with_external_aggregation_memory_usage #39908 (Nikita Taranov).
- Move username and password from URL parameters to Basic Authentication #39910 (San).
- Remove cache flush from the Docs Check #39911 (Alexey Milovidov).
- Fix flaky tests (
Tried to commit obsolete part
) #39922 (Alexander Tokmakov). - Add logging to debug flaky tests #39925 (Alexander Tokmakov).
- Fix flaky test
02360_send_logs_level_colors
#39927 (Anton Popov). - Don't create self-extracting clickhouse for split build #39936 (Yakov Olkhovskiy).
- tests/stress: add dmesg output (to see OOM details) #39939 (Azat Khuzhin).
- Create metadata directory on CREATE for FileLog engine #39940 (Azat Khuzhin).
- tests: fix 02352_rwlock flakiness #39941 (Azat Khuzhin).
- Remove old code from the website #39947 (Alexey Milovidov).
- Remove debug trace from DistinctStep #39955 (Igor Nikonov).
- IAST destructor intrusive list #39956 (Maksim Kita).
- Remove old code from the website (part 2) #39959 (Alexey Milovidov).
- Add Stateful tests (release), Stateless tests (release) to Mergeable Check #39967 (Yakov Olkhovskiy).
- Change font in CI reports #39969 (Alexey Milovidov).
- Add setting type to support special 'auto' value #39974 (Vladimir C).
- Update 02354_distributed_with_external_aggregation_memory_usage.sql #39979 (Nikita Taranov).
- tests/stress: fix dmesg reading #39980 (Azat Khuzhin).
- Disable 02380_insert_mv_race.sh with Ordinary #39985 (Alexander Tokmakov).
- Mention how the build can be speed up by disabling self-extraction #39988 (Robert Schulze).
- Use different root path for Jepsen Counter test #39992 (Antonio Andelic).
- ActionsDAG rename index to outputs #39998 (Maksim Kita).
- Added H literal for Hour IntervalKind #39999 (Heena Bansal).
- Try to avoid timeouts when checking for replication consistency #40001 (Alexander Tokmakov).
- More generic check for MergeTree table family #40004 (Alexander Gololobov).
- Further preparation for catboost integration into library-bridge #40010 (Robert Schulze).
- Self-extracting: decompressor, extract real path of executable instead of argv[0] #40011 (Yakov Olkhovskiy).
- copy self-extracting to output #40017 (Yakov Olkhovskiy).
- Update 02354_distributed_with_external_aggregation_memory_usage.sql #40024 (Nikita Taranov).
- Fix segfault in
DataTypeAggregateFunction
#40025 (Anton Popov). - tests/performance: cover sparse_hashed dictionary #40027 (Azat Khuzhin).
- Cleanup docs of parseDateTime*() function family #40030 (Robert Schulze).
- Job url #40032 (Mikhail f. Shiryaev).
- Update version_date.tsv and changelogs after v22.6.5.22-stable #40036 (github-actions[bot]).
- Non-significant changes #40038 (Alexey Milovidov).
- tests: attempt to make 02293_part_log_has_merge_reason less flaky #40047 (Azat Khuzhin).
- Remove documentation templates #40048 (Alexey Milovidov).
- Move images to clickhouse-presentations repository. #40049 (Alexey Milovidov).
- Fix broken image in test-visualizer #40050 (Alexey Milovidov).
- Add a test for query parameters in HTTP POST #40055 (Alexey Milovidov).
- Fix clickhouse-test hang in case of CREATE DATABASE fails #40057 (Azat Khuzhin).
- tests: fix 02380_insert_mv_race for Ordinary database #40058 (Azat Khuzhin).
- Skip newlines before Tags in clickhouse-test #40061 (Vladimir C).
- Replace S3 URLs by parameter #40066 (Mikhail f. Shiryaev).
- Finally fix
_csv.Error: field larger than field limit
#40072 (Alexander Tokmakov). - tests: fix 00926_adaptive_index_granularity_pk/00489_pk_subexpression flakiness #40075 (Azat Khuzhin).
- Changelogs and versions #40090 (Mikhail f. Shiryaev).
- A test for counting resources in subqueries #40104 (Alexey Milovidov).
- Use a job ID as ref text #40112 (Mikhail f. Shiryaev).
- Delete files DictionaryJoinAdapter.h/cpp #40113 (Vladimir C).
- Rework S3Helper a little bit #40127 (Mikhail f. Shiryaev).
- PODArray assign empty array fix #40129 (Maksim Kita).
- Disable 02390_prometheus_ClickHouseStatusInfo_DictionaryStatus with Ordinary database #40136 (Alexander Tokmakov).
- Add tests with Ordinary database to flaky check #40137 (Alexander Tokmakov).
- fs cache: minor change #40138 (Kseniia Sumarokova).
- Fix typo #40139 (Robert Schulze).
- Fix keeper-bench in case of error during scheduling a thread #40147 (Azat Khuzhin).
- Fix "Cannot quickly remove directory" #40151 (Alexander Tokmakov).
- Set sync_request_timeout to 10 to avoid reconnections in tests #40158 (Alexander Tokmakov).
- Disable zero-copy replication by default #40175 (Alexey Milovidov).
- Improve assignment and logging for cherry-pick and backport steps #40177 (Mikhail f. Shiryaev).
- test for Decimal aggregateFunction normalization #39420 #40178 (Denny Crane).
- Minor build changes #40182 (Alexey Milovidov).
- clickhouse-test: enable ZooKeeper tests by default #40191 (Alexey Milovidov).
- Remove old code #40196 (Alexey Milovidov).
- Update README.md #40198 (clickhouse-robot-curie).
- Fix a bug with symlinks detection #40232 (Alexander Tokmakov).
- Better error message when restoring covered parts #40234 (Alexander Tokmakov).
- Try to print stacktraces if query timeouts in integration tests #40248 (Alexander Tokmakov).
- Add Unit tests to Mergeable #40250 (Yakov Olkhovskiy).
- Extract common KV storage logic #40261 (Antonio Andelic).
- Add update_mergeable_check trigger for Unit tests #40269 (Yakov Olkhovskiy).
- CVE-2021-3520: (negligible) rdkafka library: update lz4.c from upstream #40272 (Suzy Wang).
- Fix build #40297 (Alexander Tokmakov).