ClickHouse/docs/changelogs/v22.7.1.2484-stable.md

sidebar_position	sidebar_label
1	2022

2022 Changelog

ClickHouse release v22.7.1.2484-stable (f4f05ec786) FIXME as compared to v22.6.1.1985-stable (7000c4e003)

Backward Incompatible Change

• Enable setting enable_positional_arguments by default. It allows queries like SELECT ... ORDER BY 1, 2 where 1, 2 are the references to the select clause. If you need to return the old behavior, disable this setting. #38204 (Alexey Milovidov).
• Ordinary database engine and old storage definition syntax for *MergeTree tables are deprecated. By default it's not possible to create new ones. If system database has Ordinary engine it will be automatically converted to Atomic on server startup. There are settings to keep old behavior (allow_deprecated_database_ordinary and allow_deprecated_syntax_for_merge_tree), but these settings may be removed in future releases. #38335 (Alexander Tokmakov).
• • Force rewriting comma join to inner by default (set default value cross_to_inner_join_rewrite = 2). To have old behavior set cross_to_inner_join_rewrite = 1. #39326 (Vladimir C).
• Disable format_csv_allow_single_quotes by default. #37096. #39423 (Kruglov Pavel).

New Feature

• Add new direct join algorithm for RocksDB, ref #33582. #35363 (Vladimir C).
• • Added full sorting merge join algorithm. #35796 (Vladimir C).
• Add a setting zstd_window_log_max to configure max memory usage on zstd decoding when importing external files. Closes #35693. #37015 (wuxiaobai24).
• Implement NatsStorage - table engine, which allows to pub/sub to NATS. Closes #32388. #37171 (tchepavel).
• Implement table function MongoDB. Allow writes into MongoDB storage / table function. #37213 (aaapetrenko).
• clickhouse-keeper new feature: add support for real-time digest calculation and verification. #37555 (Antonio Andelic).
• In #17202 was reported that host_regexp was being tested against only one of the possible PTR responses. This PR makes the necessary changes so that host_regexp is applied against all possible PTR responses and validate if any matches. #37827 (Arthur Passos).
• Support hadoop secure rpc transfer(hadoop.rpc.protection=privacy and hadoop.rpc.protection=integrity). #37852 (Peng Liu).
• Add struct type support in StorageHive. #38118 (lgbo).
• Added Base58 encoding/decoding. #38159 (Andrey Zvonov).
• Add chart visualization to Play UI. #38197 (Alexey Milovidov).
• support alter command on StorageHive table. #38214 (lgbo).
• Added CREATE TABLE ... EMPTY AS SELECT query. It automatically deduces table structure from the SELECT query, but does not fill the table after creation. Resolves #38049. #38272 (Alexander Tokmakov).
• Adds new setting implicit_transaction to run standalone queries inside a transaction. It handles both creation and closing (via COMMIT if the query succeeded or ROLLBACK if it didn't) of the transaction automatically. #38344 (Raúl Marín).
• Allow trailing comma in columns list. closes #38425. #38440 (chen).
• Compress clickhouse into self-extracting executable (path programs/self-extracting). New build target 'self-extracting' is added. #38447 (Yakov Olkhovskiy).
• Introduced settings additional_table_filters. Using this setting, you can specify additional filtering condition for a table which will be applied directly after reading. Example: select number, x, y from (select number from system.numbers limit 5) f any left join (select x, y from table_1) s on f.number = s.x settings additional_table_filters={'system.numbers : 'number != 3', 'table_1' : 'x != 2'}. Introduced setting additional_result_filter which specifies additional filtering condition for query result. Closes #37918. #38475 (Nikolai Kochetov).
• Add SQLInsert output format. Closes #38441. #38477 (Kruglov Pavel).
• Downloadable clickhouse executable is compressed self-extracting. #38653 (Yakov Olkhovskiy).
• Support isNullable function. This function checks whether it's argument is nullable and return true(1) or false(0). Closes #38611. #38841 (lokax).
• Add functions translate(string, from_string, to_string) and translateUTF8(string, from_string, to_string). #38935 (Nikolay Degterinsky).
• Add compatibility setting and system.settings_changes system table that contains information about changes in settings through ClickHouse versions. Closes #35972. #38957 (Kruglov Pavel).
• Add the 3rd parameter to the tupleElement function and return it if tuple doesn't have a member. Only works if the 2nd parameter is of type String. Closes #38872. #38989 (lokax).
• Support parseTimedelta function. It can be used like ```sql # ' ', ';', '-', '+', ',', ':' can be used as separators, eg. "1yr-2mo", "2m:6s" SELECT parseTimeDelta('1yr-2mo-4w + 12 days, 3 hours : 1 minute ; 33 seconds');. #39071 (jiahui-97).
• Added options to limit IO operations with remote storage: max_remote_read_network_bandwidth_for_server and max_remote_write_network_bandwidth_for_server. #39095 (Sergei Trifonov).
• Add send_logs_source_regexp setting. Send server text logs with specified regexp to match log source name. Empty means all sources. #39161 (Amos Bird).
• OpenTelemetry now collects traces without Processors spans by default. To enable Processors spans collection opentelemetry_trace_processors setting. #39170 (Ilya Yatsishin).

Performance Improvement

• Add new local_filesystem_read_method method io_uring based on the asynchronous Linux io_uring subsystem, improving read performance almost universally compared to the default pread method. #36103 (Saulius Valatka).
• Distinct optimization for sorted columns. Use specialized distinct transformation in case input stream is sorted by column(s) in distinct. Optimization can be applied to pre-distinct, final distinct, or both. Initial implementation by @dimarub2000. #37803 (Igor Nikonov).
• Add VBMI optimized copyOverlap32Shuffle for LZ4 decompress. #37891 (Guo Wangyang).
• Improve performance of ORDER BY, MergeTree merges, window functions using batch version of BinaryHeap. #38022 (Maksim Kita).
• Fix significant join performance regression which was introduced in https://github.com/ClickHouse/ClickHouse/pull/35616 . It's interesting that common join queries such as ssb queries have been 10 times slower for almost 3 months while no one complains. #38052 (Amos Bird).
• Migrate from the Intel hyperscan library to vectorscan, this speeds up many string matching on non-x86 platforms. #38171 (Robert Schulze).
• Increased parallelism of query plan steps executed after aggregation. #38295 (Nikita Taranov).
• Improve performance of insertion to columns of type JSON. #38320 (Anton Popov).
• Optimized insertion and lookups in the HashTable. #38413 (Nikita Taranov).
• Fix performance degradation from #32493. #38417 (Alexey Milovidov).
• Improve performance of column vector replicate using SIMD instructions. Author @zzachimed. #38565 (Maksim Kita).
• Norm and Distance functions for arrays speed up 1.2-2 times. #38740 (Alexander Gololobov).
• A less efficient execution plan can be generated for query with ORDER BY (a, b) than for ORDER BY a, b. #38873 (Igor Nikonov).
• Executable UDF, Executable Dictionary, Executable Storage poll subprocess fix 1 second subprocess wait during subprocess termination. #38929 (Constantine Peresypkin).
• • Pushdown filter to the right side of sorting join. #39123 (Vladimir C).
• Optimize accesses to system.stack_trace. #39177 (Azat Khuzhin).

Improvement

• Optimized processing of ORDER BY in window functions. #34632 (Vladimir Chebotarev).
• Support SQL standard create index and drop index syntax. #35166 (Jianmei Zhang).
• use simd to re-write the current column replicate funcion and got 2x performance boost in our unit benchmark test. #37235 (zzachimed).
• Send profile events for INSERT queries (previously only SELECT was supported). #37391 (Azat Khuzhin).
• Implement in order aggregation (optimize_aggregation_in_order) for fully materialized projections. #37469 (Azat Khuzhin).
• • Bugfixes and performance improvements for parallel_hash. #37648 (Vladimir C).
• Support expressions with window functions. Closes #19857. #37848 (Dmitry Novik).
• S3 single objects are now removed with RemoveObjectRequest (sic). Fixed a bug with S3ObjectStorage on GCP which did not allow to use removeFileIfExists effectively breaking approximately half of remove functionality. Automatic detection for DeleteObjects S3 API, that is not supported by GCS. This will allow to use GCS without explicit support_batch_delete=0 in configuration. #37882 (Vladimir Chebotarev).
• Fix refcnt for unused MergeTree parts in SELECT queries (may defer parts removal). #37913 (Azat Khuzhin).
• Expose basic Keeper related monitoring data (via ProfileEvents and CurrentMetrics). #38072 (lingpeng0314).
• Added kerberosInit function and corresponding KerberosInit class as a replacement for kinit executable. Replaced all calls of kinit in Kafka and HDFS code by call of kerberosInit function. Added new integration test. Closes #27651. #38105 (Roman Vasin).
• • Add setting multiple_joins_try_to_keep_original_names to not rewrite identifier name on multiple JOINs rewrite, close #34697. #38149 (Vladimir C).
• improved trace-visualizer UX. #38169 (Sergei Trifonov).
• Add ability to pass headers to url table function / storage via sql. Closes #37897. #38176 (Kseniia Sumarokova).
• Enable trace collection for AArch64. #38181 (Maksim Kita).
• Do not skip symlinks in user_defined directory during SQL user defined functions loading. Closes #38042. #38184 (Maksim Kita).
• Improve the stability for hive storage integration test. Move the data prepare step into test.py. #38260 (lgbo).
• Added background cleanup of subdirectories in store/. In some cases clickhouse-server might left garbage subdirectories in store/ (for example, on unsuccessful table creation) and those dirs were never been removed. Fixes #33710. #38265 (Alexander Tokmakov).
• Add DESCRIBE CACHE query to show cache settings from config. Add SHOW CACHES query to show available filesystem caches list. #38279 (Kseniia Sumarokova).
• Add access check for system drop fs cache. Support ON CLUSTER. #38319 (Kseniia Sumarokova).
• Support auto_close option for postgres engine connection. Closes #31486. #38363 (Kseniia Sumarokova).
• Fix PostgreSQL database engine incompatibility on upgrade from 21.3 to 22.3. Closes #36659. #38369 (Kseniia Sumarokova).
• filesystemAvailable and similar functions now work in clickhouse-local. This closes #38423. #38424 (Alexey Milovidov).
• Hardware benchmark now has support for automatic results uploading. #38427 (Alexey Milovidov).
• The table system.asynchronous_metric_log is further optimized for storage space. This closes #38134. See the YouTube video. #38428 (Alexey Milovidov).
• Functions multiMatchAny(), multiMatchAnyIndex(), multiMatchAllIndices() and their fuzzy variants now accept non-const pattern array argument. #38485 (Robert Schulze).
• Added L2 Squared distance and norm for both arrays and tuples. #38545 (Julian Gilyadov).
• Add revision() function. #38555 (Azat Khuzhin).
• Add group_by_use_nulls setting to make aggregation key columns nullable in the case of ROLLUP, CUBE and GROUPING SETS. Closes #37359. #38642 (Dmitry Novik).
• Fix GCS via proxy tunnel usage. #38726 (Azat Khuzhin).
• Support \i file in clickhouse client / local (similar to psql \i). #38813 (Kseniia Sumarokova).
• Allow null modifier in columns declaration for table functions. #38816 (Kruglov Pavel).
• • Deactivate mutations_finalizing_task before shutdown to avoid TABLE_IS_READ_ONLY errors. #38851 (Raúl Marín).
• Fix waiting of shared lock after exclusive lock failure. #38864 (Azat Khuzhin).
• Add the ability to specify compression level during data export. #38907 (Nikolay Degterinsky).
• New option rewrite in EXPLAIN AST. If enabled, it shows AST after it's rewritten, otherwise AST of original query. Disabled by default. #38910 (Igor Nikonov).
• • Stop reporting Zookeeper "Node exists" exceptions in system.errors when they are expected. #38961 (Raúl Marín).
• Allow to specify globs * or {expr1, expr2, expr3} inside a key for clickhouse-extract-from-config tool. #38966 (Nikita Mikhaylov).
• Add option enabling that SELECT from the system database requires grant. Details:. #38970 (Vitaly Baranov).
• • clearOldLogs: Don't report KEEPER_EXCEPTION on concurrent deletes. #39016 (Raúl Marín).
• clickhouse-keeper improvement: persist metainformation about keeper servers to disk. #39069 (Antonio Andelic).
• Continue without exception when running out of disk space when using filesystem cache. #39106 (Kseniia Sumarokova).
• Handling SIGTERM signals from k8s. #39130 (Timur Solodovnikov).
• SQL function multiStringAllPositions() now accepts non-const needle arguments. #39167 (Robert Schulze).
• Add merge_algorithm (Undecided, Horizontal, Vertical) to system.part_log. #39181 (Azat Khuzhin).
• Improve isNullable/isConstant/isNull/isNotNull performance for LowCardinality argument. #39192 (Kruglov Pavel).
• • Don't report system.errors when the disk is not rotational. #39216 (Raúl Marín).
• Metric result_bytes for INSERT queries in system.query_log shows number of bytes inserted. Previously value was incorrect and stored the same value as result_rows. #39225 (Ilya Yatsishin).
• The CPU usage metric in clickhouse-client will be displayed in a better way. Fixes #38756. #39280 (Sergei Trifonov).
• Rethrow exception on filesystem cache initialisation on server startup, better error message. #39386 (Kseniia Sumarokova).
• Support milliseconds, microseconds and nanoseconds in parseTimeDelta function. #39447 (Kruglov Pavel).

Bug Fix

• Fix crash when executing GRANT ALL ON . with ON CLUSTER. It was broken in https://github.com/ClickHouse/ClickHouse/pull/35767. This closes #38618. #38674 (Vitaly Baranov).
• • Fixed crash caused by IHiveFile be shared among threads. #38887 (lgbo).

Build/Testing/Packaging Improvement

• • Apply Clang Thread Safety Analysis (TSA) annotations to ClickHouse. #38068 (Robert Schulze).
• • System table "system.licenses" is now correctly populated on Mac (Darwin). #38294 (Robert Schulze).
• Handle full queue exception in clickhouse-test. If it happened we need to collect debug info to understand what queries didn't finish. #38490 (Dmitry Novik).
• • Change all|noarch packages to architecture-dependent - Fix some documentation for it - Push aarch64|arm64 packages to artifactory and release assets - Fixes #36443. #38580 (Mikhail f. Shiryaev).
• Add clickhouse-diagnostics binary to the packages. #38647 (Mikhail f. Shiryaev).
• Always print stacktraces if test queue is full. Follow up #38490 cc @tavplubix. #38662 (Dmitry Novik).
• Align branches within a 32B boundary to make benchmark more stable. #38988 (Guo Wangyang).
• Fix LSan by fixing getauxval(). #39299 (Azat Khuzhin).
• Adapt universal installation script for FreeBSD. #39302 (Alexey Milovidov).

Bug Fix (user-visible misbehavior in official stable release)

• Fix projection exception when aggregation keys are wrapped inside other functions. This fixes #37151. #37155 (Amos Bird).
• Fix possible logical error ... with argument with type Nothing and default implementation for Nothing is expected to return result with type Nothing, got ... in some functions. Closes: #37610 Closes: #37741. #37759 (Kruglov Pavel).
• Fix incorrect columns order in subqueries of UNION (in case of duplicated columns in subselects may produce incorrect result). #37887 (Azat Khuzhin).
• Fix incorrect work of MODIFY ALTER Column with column names that contain dots. Closes #37907. #37971 (Kruglov Pavel).
• Fix reading of sparse columns from MergeTree tables that store their data in S3. #37978 (Anton Popov).
• Fix rounding for Decimal128/Decimal256 with more than 19-digits long scale. #38027 (Igor Nikonov).
• Fix possible crash in Distributed async insert in case of removing a replica from config. #38029 (Nikolai Kochetov).
• Fix "Missing columns" for GLOBAL JOIN with CTE w/o alias. #38056 (Azat Khuzhin).
• Rewrite tuple functions as literals in backwards-compatibility mode. #38096 (Anton Kozlov).
• • Fix redundant memory reservation for output block during ORDER BY. #38127 (iyupeng).
• Fix possible logical error Bad cast from type DB::IColumn* to DB::ColumnNullable* in array mapped functions. Closes #38006. #38132 (Kruglov Pavel).
• • Fix temporary name clash in partial merge join, close #37928. #38135 (Vladimir C).
• With table SQL CREATE TABLE nested_name_tuples ( `a` Tuple(x String, y Tuple(i Int32, j String)) ) ENGINE = Memory; . #38136 (lgbo).
• Fix bug with nested short-circuit functions that led to execution of arguments even if condition is false. Closes #38040. #38173 (Kruglov Pavel).
• (Window View is a experimental feature) Fix LOGICAL_ERROR for WINDOW VIEW with incorrect structure. #38205 (Azat Khuzhin).
• Update librdkafka submodule to fix crash when an OAUTHBEARER refresh callback is set. #38225 (Rafael Acevedo).
• Do not allow recursive usage of OvercommitTracker during logging. Fixes #37794 cc @tavplubix @davenger. #38246 (Dmitry Novik).
• Fix INSERT into Distributed hung due to ProfileEvents. #38307 (Azat Khuzhin).
• Fix retries in PostgreSQL engine. #38310 (Kseniia Sumarokova).
• Fix optimization in PartialSortingTransform (SIGSEGV and possible incorrect result). #38324 (Azat Khuzhin).
• Fix RabbitMQ with formats based on PeekableReadBuffer. Closes #38061. #38356 (Kseniia Sumarokova).
• Fix possible Invalid number of rows in Chunk in materialised pg. Closes #37323. #38360 (Kseniia Sumarokova).
• Fix RabbitMQ configuration with connection string setting. Closes #36531. #38365 (Kseniia Sumarokova).
• Fix PostgreSQL engine not using PostgreSQL schema when retrieving array dimension size. Closes #36755. Closes #36772. #38366 (Kseniia Sumarokova).
• Fix incorrect result of distributed queries with DISTINCT and LIMIT. Fixes #38282. #38371 (Anton Popov).
• fix: expose new CH keeper port in Dockerfile clickhouse/clickhouse-keeper fix: use correct KEEPER_CONFIG filename in clickhouse/clickhouse-keeper docker image. #38462 (Evgeny Kruglov).
• Fix parts removal (will be left forever if they had not been removed on server shutdown) after incorrect server shutdown. #38486 (Azat Khuzhin).
• Fixes #38498 Current Implementation is similar to what shell does mentiond by @rschu1ze here. #38502 (Heena Bansal).
• Fix table creation to avoid replication issues with pre-22.4 replicas. #38541 (Raúl Marín).
• Fix crash for mapUpdate, mapFilter functions when using with constant map argument. Closes #38547. #38553 (hexiaoting).
• Fix wrong results of countSubstrings() & position() on patterns with 0-bytes. #38589 (Robert Schulze).
• Now it's possible to start a clickhouse-server and attach/detach tables even for tables with the incorrect values of IPv4/IPv6 representation. Proper fix for issue #35156. #38590 (alesapin).
• Adapt some more nodes to avoid issues with pre-22.4 replicas. #38627 (Raúl Marín).
• Fix toHour() monotonicity which can lead to incorrect query result (incorrect index analysis). This fixes #38333. #38675 (Amos Bird).
• rankCorr function will work correctly if some arguments are NaNs. This closes #38396. #38722 (Alexey Milovidov).
• Fix parallel_view_processing=1 with optimize_trivial_insert_select=1. Fix max_insert_threads while pushing to views. #38731 (Azat Khuzhin).
• Fix use-after-free for Map combinator that leads to incorrect result. #38748 (Azat Khuzhin).
• Fix throwing exception for seekable read from s3 (exception was not thrown). #38773 (Kseniia Sumarokova).
• Fix checking whether s3 storage support parallel writes. It resulted in s3 parallel writes not working. #38792 (chen).
• Fix s3 seekable reads with parallel read buffer. (Affected memory usage during query). Closes #38258. #38802 (Kseniia Sumarokova).
• Update simdjson. This fixes #38621. #38838 (Alexey Milovidov).
• MergeTree fix possible logical error for Vertical merges. #38859 (Maksim Kita).
• • Fix settings profile with seconds unit. #38896 (Raúl Marín).
• Fix incorrect partition pruning when there is a nullable partition. This fixes #38941. #38946 (Amos Bird).
• Fix fsync_part_directory for fetches. #38993 (Azat Khuzhin).
• Functions multiMatchFuzzy() no throw a logical error if the needle argument is empty. #39012 (Robert Schulze).
• Any allocations inside OvercommitTracker may lead to deadlock. Logging was not very informative so it's easier just to remove logging. Fixes #37794. #39030 (Dmitry Novik).
• Fix toHour() monotonicity which can lead to incorrect query result (incorrect index analysis). This fixes #38333. #39037 (Amos Bird).
• Fix bug in filesystem cache that could happen in some corner case which coincided with cache capacity hitting the limit. Closes #39066. #39070 (Kseniia Sumarokova).
• Fix ActionsDAG construction for arguments of window expressions. Fixes #38538 Allow using of higher-order functions in window expressions. #39112 (Dmitry Novik).
• Keep LowCardinality type in tuple() function. Previously LowCardinality type was dropped and elements of created tuple had underlying type of LowCardinality. #39113 (Anton Popov).
• Fix error Block structure mismatch which could happen for INSERT into table with attached MATERIALIZED VIEW and enabled setting extremes = 1. Closes #29759 and #38729. #39125 (Nikolai Kochetov).
• Fix unexpected query result when both optimize_trivial_count_query and empty_result_for_aggregation_by_empty_set are set to true. This fixes #39140. #39155 (Amos Bird).
• Fixed error Not found column Type in block in selects with PREWHERE and read-in-order optimizations. #39157 (Yakov Olkhovskiy).
• Fix extremely rare race condition in during hardnlinks for remote fs. The only way to reproduce it is concurrent run of backups. #39190 (alesapin).
• Fix fetch of in-memory part with allow_remote_fs_zero_copy_replication. #39214 (Azat Khuzhin).
• Fix NOEXCEPT_SCOPE (before it calls std::terminate and looses the exception). #39229 (Azat Khuzhin).
• Declare RabbitMQ queue without default arguments x-max-length and x-overflow. #39259 (rnbondarenko).
• Fix segmentation fault in MaterializedPostgreSQL database engine, which could happen if some exception occurred at replication initialisation. Closes #36939. #39272 (Kseniia Sumarokova).
• Fix incorrect fetch postgresql tables query fro PostgreSQL database engine. Closes #33502. #39283 (Kseniia Sumarokova).
• Fix possible UB in MergeTreeBackgroundExecutor (leads to SIGSEGV on race with DROP/DETACH). #39342 (Azat Khuzhin).
• Avoid possible abort() in CapnProto on exception descruction. Closes #30706. #39365 (Kruglov Pavel).
• Fix behaviour of dictHas for direct dictionaries when multiple lookups to the same key are made in a single action. #39385 (James Morrison).
• Fix crash which may happen while reading from dictionary with DateTime64 attribute. Fixes #38930. #39391 (Nikolai Kochetov).
• Fix WriteBuffer finalize in destructor when cacnel query that could lead to stuck query or even terminate. Closes #38199. #39396 (Kruglov Pavel).
• Fix UB (stack-use-after-scope) in extactAll(). #39397 (Azat Khuzhin).
• Fix incorrect query result when trivial count optimization is in effect with array join. This fixes #39431. #39444 (Amos Bird).

Bug Fix (user-visible misbehaviour in official stable release)

• Disable send_logs_level for INSERT into Distributed to avoid possible hung. #35075 (Azat Khuzhin).

NO CL ENTRY

• NO CL ENTRY: 'Revert "Add a setting to use more memory for zstd decompression"'. #38194 (alesapin).
• NO CL ENTRY: 'Revert "Revert "Add a setting to use more memory for zstd decompression""'. #38196 (Alexey Milovidov).
• NO CL ENTRY: 'Revert "ClickHouse's boringssl module updated to the official version of the FIPS compliant."'. #38201 (Alexey Milovidov).
• NO CL ENTRY: 'Revert "Fix optimization in PartialSortingTransform (SIGSEGV and possible incorrect result)"'. #38361 (Alexander Tokmakov).
• NO CL ENTRY: 'Revert "Add support for io_uring read method"'. #38377 (Alexander Tokmakov).
• NO CL ENTRY: 'Revert "Revert "Fix optimization in PartialSortingTransform (SIGSEGV and possible incorrect result)""'. #38449 (Maksim Kita).
• NO CL ENTRY: 'Don't spoil return code of integration tests runner with redundant tee'. #38548 (Vladimir Chebotarev).
• NO CL ENTRY: 'Revert "Non Negative Derivative window function"'. #38551 (Alexey Milovidov).
• NO CL ENTRY: 'Revert "Upload to S3 compressed self-extracting clickhouse"'. #38788 (Mikhail f. Shiryaev).
• NO CL ENTRY: 'Revert "Smallish updates of dev guide"'. #38848 (Alexander Tokmakov).
• NO CL ENTRY: 'Revert "Fix toHour() monotonicity which can lead to incorrect query result (incorrect index analysis)"'. #39001 (Alexander Tokmakov).
• NO CL ENTRY: 'Revert "Fix WriteBuffer finalize in destructor when cacnel query"'. #39433 (Kruglov Pavel).
• NO CL ENTRY: 'Revert "[RFC] Fix LSan by fixing getauxval()"'. #39434 (Alexey Milovidov).
• NO CL ENTRY: 'Revert "Remove broken optimisation in Direct dictionary dictHas implementation"'. #39461 (Alexander Tokmakov).
• NO CL ENTRY: 'Revert "Fix trivial count optimization with array join"'. #39466 (Nikolai Kochetov).

NOT FOR CHANGELOG / INSIGNIFICANT

• Separate data storage abstraction for MergeTree #36555 (Nikolai Kochetov).
• Randomize settings related to in-order read/aggregation #36914 (Azat Khuzhin).
• Merge tree reader support for multiple read/filter steps: row level filter, prewhere, ... #37165 (Alexander Gololobov).
• Backup Improvements 6 #37358 (Vitaly Baranov).
• Move updateInputStream to ITransformingStep #37393 (Nikita Taranov).
• Proper wait of the clickhouse-server in tests #37560 (Azat Khuzhin).
• Upgrade curl to 7.83.1 #37795 (Suzy Wang).
• Try fix flaky tests with transactions #37822 (Alexander Tokmakov).
• Send perf tests results to ci database #37841 (Vladimir C).
• Remove duplicate peak mem log #37860 (Amos Bird).
• tests: fix log_comment (extra quotes) #37932 (Azat Khuzhin).
• Throw exception when xml user profile does not exist #38024 (nvartolomei).
• Add SYNC command to internal ZooKeeper client #38047 (Antonio Andelic).
• Better support of GCP storage #38069 (Anton Popov).
• Build artifacts upload #38086 (Mikhail f. Shiryaev).
• Extract some diff from pr #36171 #38088 (Kseniia Sumarokova).
• Aggregate functions added restrict into batch methods #38090 (Maksim Kita).
• Add perf checkers to all Jepsen tests #38091 (Antonio Andelic).
• Some fixes for tests with tsan #38106 (Alexander Tokmakov).
• Bring back #36396 #38110 (Nikita Taranov).
• More suppressions for backward compatibility check #38131 (Alexander Tokmakov).
• Cherry pick #38137 (Mikhail f. Shiryaev).
• Disable parameters for non direct executable user defined functions #38142 (Maksim Kita).
• SortDescription compile fix typo #38144 (Maksim Kita).
• Update version after release #38147 (Mikhail f. Shiryaev).
• New changelog and versions updated #38148 (Mikhail f. Shiryaev).
• Some fixes for clickhouse-disks #38150 (Kseniia Sumarokova).
• Remove processor description from span attributes - it is not working #38157 (Ilya Yatsishin).
• Bump minimum / maximum LLVM to 12 / 14 #38170 (Robert Schulze).
• Disk transaction #38182 (alesapin).
• Check row size to avoid out of bounds access in PostgreSQLSource #38190 (Alexander Gololobov).
• tests: add no-backward-compatibility-check for 02067_lost_part_s3 #38195 (Azat Khuzhin).
• tests/stress: fix TSan detection (enables thread fuzzer for non-TSan builds) #38207 (Azat Khuzhin).
• tests: disable 01646_system_restart_replicas_smoke under stress tests #38212 (Azat Khuzhin).
• tests/stress: fix TSan detection #38213 (Azat Khuzhin).
• buffer's getFileSize small changes #38227 (Kseniia Sumarokova).
• Fix test for system table count in diag tool #38236 (Dale McDiarmid).
• Update version_date.tsv after v22.3.7.28-lts #38237 (github-actions[bot]).
• Changelog attrs #38238 (Mikhail f. Shiryaev).
• Fix reading from s3 in some corner cases #38239 (Anton Popov).
• use utility methods to access x509 struct fields. #38251 (larryluogit).
• Don't try to kill empty list of containers in integration/runner II #38269 (Vladimir Chebotarev).
• Improve runners AMI and init scripts #38273 (Mikhail f. Shiryaev).
• Update thrift to 0.16.0 #38280 (Suzy Wang).
• Extract some diff from #36171 #38285 (Kseniia Sumarokova).
• fix trace-viz zoom anomalies #38287 (Sergei Trifonov).
• Integration tests volume #38291 (Mikhail f. Shiryaev).
• fix typo in view.md #38292 (Anton Petrov).
• Backup improvements 7 #38299 (Vitaly Baranov).
• Document why the submodule check does not halt the configuration #38304 (Robert Schulze).
• Fix misleading error message while s3 schema inference #38306 (Kruglov Pavel).
• Update README.md #38313 (Yuko Takagi).
• Ban projections for zero-copy replication in a right way #38322 (alesapin).
• Checkout full repositories for performance tests #38327 (Mikhail f. Shiryaev).
• Fixed comments #38331 (Vladimir Chebotarev).
• Try to fix 02305_schema_inference_with_globs #38337 (Kruglov Pavel).
• Extend ZooKeeper list request with support for filtering persistent or ephemeral nodes only #38338 (Antonio Andelic).
• Upload logs for getting all tests command #38343 (Mikhail f. Shiryaev).
• Followup test fix for ban projections #38351 (alesapin).
• Added --recursive to clickhouse-disks list #38354 (Alexander Gololobov).
• Adding TLS V13 Test #38355 (larryluogit).
• Better exception messages on wrong table engines/functions argument types #38362 (Kruglov Pavel).
• Better error message for failed odbc query #38364 (Kseniia Sumarokova).
• Simplify parts commit methods #38380 (alesapin).
• Update docker-compose to try get rid of v1 errors #38394 (Mikhail f. Shiryaev).
• Correct submodule after "base-x" commit #38414 (Alexey Milovidov).
• Better hardware benchmark #38419 (Alexey Milovidov).
• Refactoring to enable multi-match functions with non-const needles #38434 (Robert Schulze).
• more consistent work with paths in object storages #38436 (Kseniia Sumarokova).
• Distinct sorted: calculate column positions once #38438 (Igor Nikonov).
• Small improvement of the error message to hint at possible issue #38458 (Miel Donkers).
• Fix comment #38465 (Kseniia Sumarokova).
• Follow up for #38436 #38466 (Kseniia Sumarokova).
• Add 22.7 release webinar. #38481 (Yuko Takagi).
• Add some TSA annotations #38487 (Alexander Tokmakov).
• tests: cleanup tmp data in 02335_column_ttl_expired_column_optimization #38488 (Azat Khuzhin).
• Cleanup: local clang-tidy warnings founded during review #38489 (Igor Nikonov).
• Fix some clang-tidy warnings in headers #38491 (Robert Schulze).
• A tiny improvement in report logging #38507 (Mikhail f. Shiryaev).
• tests: fix 02305_schema_inference_with_globs flakiness #38511 (Azat Khuzhin).
• Try to fix flaky test #38516 (Alexander Tokmakov).
• Fix _csv.Error: field larger than field limit #38518 (Alexander Tokmakov).
• Fix application errors grep in stress test #38520 (Kruglov Pavel).
• Use of disk batch operations in MergeTree #38531 (alesapin).
• Backup Improvements 8 #38537 (Vitaly Baranov).
• Update poco #38540 (Yakov Olkhovskiy).
• Don't use std::unique_lock unless we need to #38542 (Robert Schulze).
• Rename slightly weirdly named "BuilderBinTidy" to "BuilderBinClangTidy" #38546 (Robert Schulze).
• Don't rollback SessionID request in Keeper #38556 (Antonio Andelic).
• Add logging in Epoll and TimerDescriptor in case of EINTR #38559 (Kruglov Pavel).
• SQL create drop index minor fixes #38561 (Maksim Kita).
• Update version_date.tsv and changelogs after v22.6.2.12-stable #38563 (github-actions[bot]).
• Allow Ordinary database in Stress Tests #38568 (Alexander Tokmakov).
• Make postgres integration test great again #38582 (Ilya Yatsishin).
• Add check for empty proccessors in AggregatingTransform::expandPipeline #38584 (filimonov).
• quick fix for 02112_with_fill_interval #38587 (Nikita Taranov).
• Remove zlib in mariadb-connector-c #38599 (Suzy Wang).
• Dictionaries added TSA annotations #38601 (Maksim Kita).
• CacheDictionary simplify update queue #38602 (Maksim Kita).
• Add separate option to omit symbols from heavy contrib #38617 (Azat Khuzhin).
• Fix exception messages in clickhouse su #38619 (filimonov).
• Added Greenplum benchmark #38622 (Dmitry Pavlov).
• Fix typo #38623 (tiegen).
• Better diagnostics in ReplicatedMergeTreeQueue #38641 (Alexander Tokmakov).
• Clean out randomized integration volumes each run #38644 (Mikhail f. Shiryaev).
• Update README.md #38651 (Yuko Takagi).
• Better naming for stuff related to splitted debug symbols #38654 (Robert Schulze).
• Add test for keeper mntr command #38656 (alesapin).
• Update hardware benchmark script #38672 (Filatenkov Artur).
• Fix strange backport titles issues #38679 (Mikhail f. Shiryaev).
• Follow-up to #38568 #38680 (Alexander Tokmakov).
• Fix possible deadlocks with MergeTreeData::Transaction #38702 (alesapin).
• Fix backports diff #38703 (Mikhail f. Shiryaev).
• Fix FillingTransform #38705 (Nikita Taranov).
• Try to improve backward compatibility check #38717 (Kruglov Pavel).
• SQL create drop index fix formatting #38720 (Maksim Kita).
• Provide sort description for output stream in ReadFromMergeTree step #38721 (Igor Nikonov).
• Add exp_internal for expect tests #38728 (Azat Khuzhin).
• Fix CLICKHOUSE_TMP in tests (fixes broken CI) #38733 (Azat Khuzhin).
• Add SimpleCheck #38744 (Yakov Olkhovskiy).
• Tiny tests cleanup #38749 (Azat Khuzhin).
• Fix replication after improper merge process #38752 (Raúl Marín).
• tests: make aggregate_state_exception_memory_leak deterministic #38754 (Azat Khuzhin).
• Bump jemalloc to fix possible assertion #38757 (Azat Khuzhin).
• Reintroduce nonNegativeDerivative() #38774 (Andrey Zvonov).
• Temporarily disable 01710_projection_fetch_long in BC check #38798 (Alexander Tokmakov).
• Use native Map type for OpenTelemetry attributes #38814 (Ilya Yatsishin).
• Add test for segfault in Map combinator #38831 (Kruglov Pavel).
• Update libprotobuf-mutator + fix build #38834 (Nikita Mikhaylov).
• Get files changed in master since release is branched #38836 (Mikhail f. Shiryaev).
• update integration tests doc #38837 (Bharat Nallan).
• Revert of revert of smallish devguide update #38850 (Robert Schulze).
• Do not override compiler if it had been already set #38856 (Azat Khuzhin).
• Move check for denied allocations #38858 (Alexander Tokmakov).
• Refactoring of code around object storages, added LocalObjectStorage (extracted this diff from PR #36171) #38860 (Kseniia Sumarokova).
• Backup Improvements 9 #38861 (Vitaly Baranov).
• Simple cleanup: interpreters and parsers #38876 (Igor Nikonov).
• Remove unnecessary log #38892 (Raúl Marín).
• Update version_date.tsv and changelogs after v22.6.3.35-stable #38894 (github-actions[bot]).
• Retry docker buildx commands with progressive sleep in between #38898 (Mikhail f. Shiryaev).
• Avoid false positive red sanitizer asserts check in stress test #38901 (Kruglov Pavel).
• Interpreter cleanup: ContextPtr -> const ContextPtr & in parameters #38902 (Igor Nikonov).
• Add a test for simdjson #38933 (Alexey Milovidov).
• Fix a typo #38938 (Nikolay Degterinsky).
• Avoid redundant join block transformation during planning. #38943 (Amos Bird).
• Rename NUMBER_OF_DIMENSIONS_MISMATHED const to NUMBER_OF_DIMENSIONS_MISMATCHED #38947 (Vladimir Galunshchikov).
• More careful destructor in BackupImpl #38949 (Vitaly Baranov).
• Avoid weird exception in Keeper #38963 (Alexander Tokmakov).
• Update version_date.tsv after v22.3.8.39-lts #38969 (github-actions[bot]).
• Remove tag no-backward-compatibility-check for specific versions #38971 (Kruglov Pavel).
• add Hetzner benchmark #38974 (Tyler Hannan).
• Update version_date.tsv after v22.4.6.53-stable #38975 (github-actions[bot]).
• Disable instrumentation of sanitizer death callback #38977 (Alexander Tokmakov).
• add ryzen 9 5950 benchmark #38979 (Tyler Hannan).
• EXPLAIN AST rewrite: rename to optimize #38980 (Igor Nikonov).
• add macbook pro core i7 2014 benchmark #38981 (Tyler Hannan).
• add Huawei TaiShan 920 Benchmark #38982 (Tyler Hannan).
• tests: unique ZooKeeper path for Replicated.*MergeTree tables #38999 (Azat Khuzhin).
• Try another suppression for #38629 #39009 (Alexander Tokmakov).
• Add docker_server.py running to backport and release CIs #39011 (Mikhail f. Shiryaev).
• Fix flaky test_system_merges/test.py::test_mutation_simple #39013 (Alexander Tokmakov).
• Fix assertion in full soring merge join #39014 (Vladimir C).
• Fix flaky 00620_optimize_on_nonleader_replica_zookeeper #39019 (Alexander Tokmakov).
• Suppress #38643 #39024 (Alexander Tokmakov).
• Update url.md #39025 (Ilya Yatsishin).
• Fix 'Tried to lock part ... for removal second time' #39036 (Alexander Tokmakov).
• Add more settings for randomization #39039 (Anton Popov).
• add ScaleFlux CSD3000 Benchmark #39040 (Tyler Hannan).
• BACKUP/RESTORE ON CLUSTER use async mode on replicas now. #39046 (Vitaly Baranov).
• More stable test_s3_zero_copy_ttl, weakened requirement to move data to S3 in 0-5 seconds #39064 (Vladimir Chebotaryov).
• Parameter --decompressor added to utils/self-extracting-executable/compressor #39065 (Yakov Olkhovskiy).
• Follow-up to #39036 #39091 (Alexander Tokmakov).
• Update registerDiskS3.cpp #39092 (Kseniia Sumarokova).
• Fix code in getLeastSupertype function #39101 (Kruglov Pavel).
• Remove some debug logging #39102 (Kseniia Sumarokova).
• Prefix overridden add_executable() command with "clickhouse_" #39108 (Robert Schulze).
• Fix meilisearch tests #39110 (Kseniia Sumarokova).
• Properly remove projection from part in case it was removed from table metadata. #39119 (Nikolai Kochetov).
• Update cluster.py #39120 (Kseniia Sumarokova).
• Tiny updates for tests. #39127 (Nikolai Kochetov).
• Fix data race in CompletedPipelineExecutor. #39132 (Nikolai Kochetov).
• Fix style again #39133 (Nikolai Kochetov).
• Fix path retrieval for Keeper's state #39148 (Nikita Mikhaylov).
• Slightly better interface of waitForMutation #39154 (Amos Bird).
• ThreadPool fixes #39160 (Azat Khuzhin).
• Add test for #39132 #39173 (Nikolai Kochetov).
• Suppression for BC check (Cannot parse string 'Hello' as UInt64) #39176 (Alexander Tokmakov).
• Fix 01961_roaring_memory_tracking test #39187 (Dmitry Novik).
• Cleanup: done during #38719 (SortingStep: deduce way to sort based on ... #39191 (Igor Nikonov).
• Fix exception in AsynchronousMetrics for s390x #39193 (Harry Lee).
• Optimize accesses to system.stack_trace (filter by name before sending signal) #39212 (Azat Khuzhin).
• Enable warning "-Wdeprecated-dynamic-exception-spec" #39213 (Robert Schulze).
• Remove specialization global lock/unlock from ActionLocksManager #39215 (Azat Khuzhin).
• Turn some warnings on #39223 (Robert Schulze).
• Pass const std::string_view by value, not by reference #39224 (Kruglov Pavel).
• Minor fix for BC check #39231 (Alexander Tokmakov).
• Backport script #39235 (Mikhail f. Shiryaev).
• Do not color logs on client if they are redirected to file #39243 (Anton Popov).
• Remove incorrect assertion #39245 (Alexander Tokmakov).
• Add X86 prefix to x86 performance tests #39251 (Robert Schulze).
• Check that the destination for a backup is not in use. #39254 (Vitaly Baranov).
• Fix stacktraces in gdb in BC check #39256 (Alexander Tokmakov).
• Disable flaky test test_s3_zero_copy_on_hybrid_storage #39258 (Alexander Tokmakov).
• Enabled Wc99-extensions + Wsign-conversion #39261 (Robert Schulze).
• Pass const StringRef by value, not by reference #39262 (Robert Schulze).
• Fix assertion in transactions #39263 (Alexander Tokmakov).
• Fix macosx compilation due to endian.h #39265 (Jordi Villar).
• Another supression for BC check #39276 (Alexander Tokmakov).
• Fix builder image for releases w/o diagnostics tool #39281 (Azat Khuzhin).
• [RFC] Remove superior atomic from MergeTreeBackgroundExecutor and annotations for TSA #39285 (Azat Khuzhin).
• Fix clang tidy #39288 (Kseniia Sumarokova).
• Fix running cmake with predefined cache (for development only) #39295 (Azat Khuzhin).
• Fix googletest contrib compilation (due to GTEST_HAS_POSIX_RE=0) #39298 (Azat Khuzhin).
• First try at reducing the use of StringRef #39300 (Robert Schulze).
• Whitespaces #39303 (Alexey Milovidov).
• Add test for window function inside CASE #39305 (Dmitry Novik).
• Simple Check should be updated on rerun #39307 (Yakov Olkhovskiy).
• Fix leaking of logger in clickhouse-disks #39314 (Azat Khuzhin).
• Update exception message #39315 (Alexander Tokmakov).
• Fix build clang-13 #39318 (alesapin).
• Auto set test name in integration tests #39322 (Vitaly Baranov).
• Try fix flaky test_store_cleanup #39334 (Alexander Tokmakov).
• Do not start on unexpected Ordinary metadata #39337 (Alexander Tokmakov).
• switch from mkdocs to Docusaurus #39338 (Dan Roscigno).
• Fix flaky 01174_select_insert_isolation #39339 (Alexander Tokmakov).
• Better exception messages in schema inference #39340 (Kruglov Pavel).
• Fix memory exceptions with transactions #39341 (Alexander Tokmakov).
• Fix typo #39360 (Kruglov Pavel).
• Fix typo #39361 (Kruglov Pavel).
• Do not enqueue uneeded parts for check #39366 (Alexander Tokmakov).
• Avoid loading toolchain file multiple times to avoid confusing ccache #39387 (Azat Khuzhin).
• Fix make clean (due to crosscompile of llvm) #39392 (Azat Khuzhin).
• Disable real-time digest in Keeper by default #39393 (Antonio Andelic).
• Fix interactive client with older server #39413 (Vitaly Baranov).
• Fix BC check #39414 (Alexander Tokmakov).
• Fix flaky test test_login_as_dropped_user_xml. #39415 (Vitaly Baranov).
• Introduce a dependency to libuv when building NATS #39427 (ltrk2).
• Set default value cross_to_inner_join_rewrite = 1 #39443 (Vladimir C).
• Respect table alias for additional_table_filters. #39456 (Nikolai Kochetov).

Performance optimization and Bug Fix

• Enabled pread_threadpool read method by default. It will increase read performance. Bug fix: if direct IO is enabled and the number of threads is large and pread_threadpool is used, it may cause a logical error. #33653 (Alexey Milovidov).