ClickHouse/docs/changelogs/v24.2.1.2248-stable.md

sidebar_position	sidebar_label
1	2024

2024 Changelog

ClickHouse release v24.2.1.2248-stable (891689a415) FIXME as compared to v24.1.1.2048-stable (5a024dfc09)

Backward Incompatible Change

• Validate suspicious/experimental types in nested types. Previously we didn't validate such types (except JSON) in nested types like Array/Tuple/Map. #59385 (Kruglov Pavel).
• The sort clause ORDER BY ALL (introduced with v23.12) is replaced by ORDER BY *. The previous syntax was too error-prone for tables with a column all. #59450 (Robert Schulze).
• Rename the setting extract_kvp_max_pairs_per_row to extract_key_value_pairs_max_pairs_per_row. The bug (unnecessary abbreviation in the setting name) was introduced in https://github.com/ClickHouse/ClickHouse/pull/43606. Fix the documentation of this setting. #59683 (Alexey Milovidov).
• Rename the setting extract_kvp_max_pairs_per_row to extract_key_value_pairs_max_pairs_per_row. The bug (unnecessary abbreviation in the setting name) was introduced in https://github.com/ClickHouse/ClickHouse/pull/43606. Fix the documentation of this setting. #59960 (jsc0218).
• Add sanity check for number of threads and block sizes. #60138 (Raúl Marín).

New Feature

• Added maximum sequential login failures to the quota. #54737 (Alexey Gerasimchuck).
• Added new syntax which allows to specify definer user in View/Materialized View. This allows to execute selects/inserts from views without explicit grants for underlying tables. #54901 (pufit).
• Backup & Restore support for AzureBlobStorage resolves #50747. #56988 (SmitaRKulkarni).
• Implemented automatic conversion of merge tree tables of different kinds to replicated engine. Create empty convert_to_replicated file in table's data directory (/clickhouse/store/xxx/xxxyyyyy-yyyy-yyyy-yyyy-yyyyyyyyyyyy/) and that table will be converted automatically on next server start. #57798 (Kirill).
• Added table function mergeTreeIndex. It represents the contents of index and marks files of MergeTree tables. It can be used for introspection. Syntax: mergeTreeIndex(database, table, [with_marks = true]) where database.table is an existing table with MergeTree engine. #58140 (Anton Popov).
• Added function seriesOutliersTukey to detect outliers in series data using Tukey's fences algorithm. #58632 (Bhavna Jindal).
• The user can now specify the template string directly in the query using format_schema_rows_template as an alternative to format_template_row. Closes #31363. #59088 (Shaun Struwig).
• Try to detect file format automatically during schema inference if it's unknown in file/s3/hdfs/url/azureBlobStorage engines. Closes #50576. #59092 (Kruglov Pavel).
• Add function variantType that returns Enum with variant type name for each row. #59398 (Kruglov Pavel).
• Added query ALTER TABLE table FORGET PARTITION partition that removes ZooKeeper nodes, related to an empty partition. #59507 (Sergei Trifonov).
• Support JWT credentials file for the NATS table engine. #59543 (Nickolaj Jepsen).
• Provides new aggregate function ‘groupArrayIntersect’. Follows up: #49862. #59598 (Yarik Briukhovetskyi).
• Implemented system.dns_cache table, which can be useful for debugging DNS issues. #59856 (Kirill Nikiforov).
• The codec LZ4HC will accept a new level 2, which is faster than the previous minimum level 3, at the expense of less compression. In previous versions, LZ4HC(2) and less was the same as LZ4HC(3). Author: Cyan4973. #60090 (Alexey Milovidov).
• Implemented system.dns_cache table, which can be useful for debugging DNS issues. New server setting dns_cache_max_size. #60257 (Kirill Nikiforov).
• Support single-argument version for the merge table function, as merge(['db_name', ] 'tables_regexp'). #60372 (豪肥肥).
• Added new syntax which allows to specify definer user in View/Materialized View. This allows to execute selects/inserts from views without explicit grants for underlying tables. #60439 (pufit).

Performance Improvement

• Eliminates min/max/any/anyLast aggregators of GROUP BY keys in SELECT section. #52230 (JackyWoo).
• Vectorized distance functions used in vector search. #58866 (Robert Schulze).
• Continue optimizing branch miss of if function when result type is float*/decimal*/int* , follow up of https://github.com/ClickHouse/ClickHouse/pull/57885. #59148 (李扬).
• Optimize if function when input type is map, speed up by ~10x. #59413 (李扬).
• Improve performance of Int8 type by implementing strict aliasing. #59485 (Raúl Marín).
• Optimize performance of sum/avg conditionally for bigint and big decimal types by reducing branch miss. #59504 (李扬).
• Improve performance of SELECTs with active mutations. #59531 (Azat Khuzhin).
• Optimized function isNotNull with AVX2. #59621 (李扬).
• Reuse the result of FunctionFactory::instance().get("isNotNull", context) and FunctionFactory::instance().get("assumeNotNull", context). Make sure it is called once during the lifetime of FunctionCoalesce. #59627 (李扬).
• Improve ASOF JOIN performance for sorted or almost sorted data. #59731 (Maksim Kita).
• Primary key will use less amount of memory. #60049 (Alexey Milovidov).
• Improve memory usage for primary key and some other operations. #60050 (Alexey Milovidov).
• The tables' primary keys will be loaded in memory lazily on first access. This is controlled by the new MergeTree setting primary_key_lazy_load, which is on by default. This provides several advantages: - it will not be loaded for tables that are not used; - if there is not enough memory, an exception will be thrown on first use instead of at server startup. This provides several disadvantages: - the latency of loading the primary key will be paid on the first query rather than before accepting connections; this theoretically may introduce a thundering-herd problem. This closes #11188. #60093 (Alexey Milovidov).
• Vectorized function dotProduct which is useful for vector search. #60202 (Robert Schulze).
• As is shown in Fig 1, the replacement of "&&" with "&" could generate the SIMD code. Fig 1. Code compiled from '&&' (left) and '&' (right). #60498 (Zhiguo Zhou).

Improvement

• Added support for parameterized view with analyzer to not analyze create parameterized view. Refactor existing parameterized view logic to not analyze create parameterized view. #54211 (SmitaRKulkarni).
• Do not consider data part as broken if projection is broken. Closes #56593. #56864 (Kseniia Sumarokova).
• Add short-circuit ability for dictGetOrDefault function. Closes #52098. #57767 (jsc0218).
• Running ALTER COLUMN MATERIALIZE on a column with DEFAULT or MATERIALIZED expression now writes the correct values: The default value for existing parts with default value or the non-default value for existing parts with non-default value. Previously, the default value was written for all existing parts. #58023 (Duc Canh Le).
• Enabled a backoff logic (e.g. exponential). Will provide an ability for reduced CPU usage, memory usage and log file sizes. #58036 (MikhailBurdukov).
• Add improvement to count InitialQuery. #58195 (Unalian).
• Support negative positional arguments. Closes #57736. #58292 (flynn).
• Implement auto-adjustment for asynchronous insert timeouts. The following settings are introduced: async_insert_poll_timeout_ms, async_insert_use_adaptive_busy_timeout, async_insert_busy_timeout_min_ms, async_insert_busy_timeout_max_ms, async_insert_busy_timeout_increase_rate, async_insert_busy_timeout_decrease_rate. #58486 (Julia Kartseva).
• Allow to define volume_priority in storage_configuration. #58533 (Andrey Zvonov).
• Add support for Date32 type in T64 codec. #58738 (Hongbin Ma (Mahone)).
• Support LEFT JOIN, ALL INNER JOIN, and simple subqueries for parallel replicas (only with analyzer). New setting parallel_replicas_prefer_local_join chooses local JOIN execution (by default) vs GLOBAL JOIN. All tables should exist on every replica from cluster_for_parallel_replicas. New settings min_external_table_block_size_rows and min_external_table_block_size_bytes are used to squash small blocks that are sent for temporary tables (only with analyzer). #58916 (Nikolai Kochetov).
• Allow trailing commas in types with several items. #59119 (Aleksandr Musorin).
• Allow parallel and distributed processing for S3Queue table engine. For distributed processing use setting s3queue_total_shards_num (by default 1). Setting s3queue_processing_threads_num previously was not allowed for Ordered processing mode, now it is allowed. Warning: settings s3queue_processing_threads_num(processing threads per each shard) and s3queue_total_shards_num for ordered mode change how metadata is stored (make the number of max_processed_file nodes equal to s3queue_processing_threads_num * s3queue_total_shards_num), so they must be the same for all shards and cannot be changed once at least one shard is created. #59167 (Kseniia Sumarokova).
• Allow concurrent table creation in DatabaseReplicated during recoverLostReplica. #59277 (Konstantin Bogdanov).
• Settings for the Distributed table engine can now be specified in the server configuration file (similar to MergeTree settings), e.g. <distributed> <flush_on_detach>false</flush_on_detach> </distributed>. #59291 (Azat Khuzhin).
• Use MergeTree as a default table engine. It makes the usability much better, and closer to ClickHouse Cloud. #59316 (Alexey Milovidov).
• Retry disconnects and expired sessions when reading system.zookeeper. This is helpful when reading many rows from system.zookeeper table especially in the presence of fault-injected disconnects. #59388 (Alexander Gololobov).
• Do not interpret numbers with leading zeroes as octals when input_format_values_interpret_expressions=0. #59403 (Joanna Hulboj).
• At startup and whenever config files are changed, ClickHouse updates the hard memory limits of its total memory tracker. These limits are computed based on various server settings and cgroups limits (on Linux). Previously, setting /sys/fs/cgroup/memory.max (for cgroups v2) was hard-coded. As a result, cgroup v2 memory limits configured for nested groups (hierarchies), e.g. /sys/fs/cgroup/my/nested/group/memory.max were ignored. This is now fixed. The behavior of v1 memory limits remains unchanged. #59435 (Robert Schulze).
• New profile events added to observe the time spent on calculating PK/projections/secondary indices during INSERT-s. #59436 (Nikita Taranov).
• Allow to define a starting point for S3Queue with Ordered mode at creation using setting s3queue_last_processed_path. #59446 (Kseniia Sumarokova).
• Keeper improvement: cache only a certain amount of logs in-memory controlled by latest_logs_cache_size_threshold and commit_logs_cache_size_threshold. #59460 (Antonio Andelic).
• Made comments for system tables also available in system.tables in clickhouse-local. #59493 (Nikita Mikhaylov).
• Don't infer floats in exponential notation by default. Add a setting input_format_try_infer_exponent_floats that will restore previous behaviour (disabled by default). Closes #59476. #59500 (Kruglov Pavel).
• Allow alter operations to be surrounded by parenthesis. The emission of parentheses can be controlled by the format_alter_operations_with_parentheses config. By default in formatted queries the parentheses are emitted as we store the formatted alter operations in some places as metadata (e.g.: mutations). The new syntax clarifies some of the queries where alter operations end in a list. E.g.: ALTER TABLE x MODIFY TTL date GROUP BY a, b, DROP COLUMN c cannot be parsed properly with the old syntax. In the new syntax the query ALTER TABLE x (MODIFY TTL date GROUP BY a, b), (DROP COLUMN c) is obvious. Older versions are not able to read the new syntax, therefore using the new syntax might cause issues if newer and older version of ClickHouse are mixed in a single cluster. #59532 (János Benjamin Antal).
• The previous default value equals to 1 MB for async_insert_max_data_size appeared to be too small. The new one would be 10 MiB. #59536 (Nikita Mikhaylov).
• Previously the whole result was accumulated in memory and returned as one big chunk. This change should help to reduce memory consumption when reading many rows from system.zookeeper, allow showing intermediate progress (how many rows have been read so far) and avoid hitting connection timeout when result set is big. #59545 (Alexander Gololobov).
• Now dashboard understands both compressed and uncompressed state of URL's #hash (backward compatibility). Continuation of #59124 . #59548 (Amos Bird).
• Bumped Intel QPL (used by codec DEFLATE_QPL) from v1.3.1 to v1.4.0 . Also fixed a bug for polling timeout mechanism, as we observed in same cases timeout won't work properly, if timeout happen, IAA and CPU may process buffer concurrently. So far, we'd better make sure IAA codec status is not QPL_STS_BEING_PROCESSED, then fallback to SW codec. #59551 (jasperzhu).
• Keeper improvement: reduce size of data node even more. #59592 (Antonio Andelic).
• Do not show a warning about the server version in ClickHouse Cloud because ClickHouse Cloud handles seamless upgrades automatically. #59657 (Alexey Milovidov).
• After self-extraction temporary binary is moved instead copying. #59661 (Yakov Olkhovskiy).
• Fix stack unwinding on MacOS. This closes #53653. #59690 (Nikita Mikhaylov).
• Check for stack overflow in parsers even if the user misconfigured the max_parser_depth setting to a very high value. This closes #59622. #59697 (Alexey Milovidov).
• Perform synchronous inserts if dependent MV deduplication is enabled through deduplicate_blocks_in_dependent_materialized_views=1. #59699 (Julia Kartseva).
• Added settings split_parts_ranges_into_intersecting_and_non_intersecting_final and split_intersecting_parts_ranges_into_layers_final. This settings are needed to disable optimizations for queries with FINAL and needed for debug only. #59705 (Maksim Kita).
• Unify xml and sql created named collection behaviour in kafka storage. #59710 (Pervakov Grigorii).
• In case when merge_max_block_size_bytes is small enough and tables contain wide rows (strings or tuples) background merges may stuck in an endless loop. This behaviour is fixed. Follow-up for https://github.com/ClickHouse/ClickHouse/pull/59340. #59812 (Nikita Mikhaylov).
• Allow uuid in replica_path if CREATE TABLE explicitly has it. #59908 (Azat Khuzhin).
• Add column metadata_version of ReplicatedMergeTree table in system.tables system table. #59942 (Maksim Kita).
• Keeper improvement: send only Keeper related metrics/events for Prometheus. #59945 (Antonio Andelic).
• The dashboard will display metrics across different ClickHouse versions even if the structure of system tables has changed after the upgrade. #59967 (Alexey Milovidov).
• Allow loading AZ info from a file. #59976 (Konstantin Bogdanov).
• Keeper improvement: add retries on failures for Disk related operations. #59980 (Antonio Andelic).
• Add new config setting backups.remove_backup_files_after_failure: <clickhouse> <backups> <remove_backup_files_after_failure>true</remove_backup_files_after_failure> </backups> </clickhouse>. #60002 (Vitaly Baranov).
• Use multiple threads while reading the metadata of tables from a backup while executing the RESTORE command. #60040 (Vitaly Baranov).
• Now if StorageBuffer has more than 1 shard (num_layers > 1) background flush will happen simultaneously for all shards in multiple threads. #60111 (alesapin).
• Support specifying users for specific S3 settings in config using user key. #60144 (Antonio Andelic).
• Copy S3 file GCP fallback to buffer copy in case GCP returned Internal Error with GATEWAY_TIMEOUT HTTP error code. #60164 (Maksim Kita).
• Implement comparison operator for Variant values and proper Field inserting into Variant column. Don't allow creating Variant type with similar variant types by default (allow uder a setting allow_suspicious_variant_types) Closes #59996. Closes #59850. #60198 (Kruglov Pavel).
• Short circuit execution for ULIDStringToDateTime. #60211 (Juan Madurga).
• Added query_id column for tables system.backups and system.backup_log. Added error stacktrace to error column. #60220 (Maksim Kita).
• Connections through the MySQL port now automatically run with setting prefer_column_name_to_alias = 1 to support QuickSight out-of-the-box. Also, settings mysql_map_string_to_text_in_show_columns and mysql_map_fixed_string_to_text_in_show_columns are now enabled by default, affecting also only MySQL connections. This increases compatibility with more BI tools. #60365 (Robert Schulze).
• When output format is Pretty format and a block consists of a single numeric value which exceeds one million, A readable number will be printed on table right. e.g. ┌──────count()─┐ │ 233765663884 │ -- 233.77 billion └──────────────┘. #60379 (rogeryk).
• Fix a race condition in JavaScript code leading to duplicate charts on top of each other. #60392 (Alexey Milovidov).
• Check for stack overflow in parsers even if the user misconfigured the max_parser_depth setting to a very high value. This closes #59622. #60434 (Alexey Milovidov).

Build/Testing/Packaging Improvement

• Added builds and tests with coverage collection with introspection. Continuation of #56102. #58792 (Alexey Milovidov).
• Include pytest-reportlog in integration test CI runner Dockerfile to enable JSON test reports. #58926 (MyroTk).
• Update the rust toolchain in corrosion-cmake when the CMake cross-compilation toolchain variable is set. #59309 (Aris Tritas).
• Add some fuzzing to ASTLiterals. #59383 (Raúl Marín).
• If you want to run initdb scripts every time when ClickHouse container is starting you shoud initialize environment varible CLICKHOUSE_ALWAYS_RUN_INITDB_SCRIPTS. #59808 (Alexander Nikolaev).
• Remove ability to disable generic clickhouse components (like server/client/...), but keep some that requires extra libraries (like ODBC or keeper). #59857 (Azat Khuzhin).
• Query fuzzer will fuzz SETTINGS inside queries. #60087 (Alexey Milovidov).
• Add test that validates projections still work after attach partition. #60415 (Arthur Passos).
• Add test that validates attach partition fails if structure differs because of materialized column. #60418 (Arthur Passos).
• Add support for building ClickHouse with clang-19 (master). #60448 (Alexey Milovidov).
• Speedup check-whitespaces check. #60496 (Raúl Marín).

Bug Fix (user-visible misbehavior in an official stable release)

• Support IN (subquery) in table TTL expression. Initially, it was allowed to create such a TTL expression, but any TTL merge would fail with Not-ready Set error in the background. Now, TTL is correctly applied. Subquery is executed for every TTL merge, and its result is not cached or reused by other merges. Use such configuration with special care, because subqueries in TTL may lead to high memory consumption and, possibly, a non-deterministic result of TTL merge on different replicas (which is correctly handled by replication, however). #57430 (Nikolai Kochetov).
• Fix quantilesGK bug, close #57683. #58216 (李扬).
• Disable parallel replicas JOIN with CTE (not analyzer). #59239 (Raúl Marín).
• Fixes bug with for function intDiv with decimal arguments. Fixes #56414. #59243 (Yarik Briukhovetskyi).
• Fix translate() with FixedString input. Could lead to crashes as it'd return a String column (vs the expected FixedString). This issue was found through ClickHouse Bug Bounty Program YohannJardin. #59356 (Raúl Marín).
• Keeper fix: fix digest calculation for nodes. #59439 (Antonio Andelic).
• Fix stacktraces for binaries without debug symbols. #59444 (Azat Khuzhin).
• Fix formatting of alter commands in case of column specific settings. #59445 (János Benjamin Antal).
• SELECT * FROM [...] ORDER BY ALL SETTINGS allow_experimental_analyzer = 1 now works. #59462 (zhongyuankai).
• Fix possible uncaught exception during distributed query cancellation. Closes #59169. #59487 (Azat Khuzhin).
• Make MAX use the same rules as permutation for complex types. #59498 (Raúl Marín).
• Fix a corner case when passing update_insert_deduplication_token_in_dependent_materialized_views setting. There is one corner case not covered due to the absence of tables in the path:. #59544 (Jordi Villar).
• Fix incorrect result of arrayElement / map[] on empty value. #59594 (Raúl Marín).
• Fix crash in topK when merging empty states. #59603 (Raúl Marín).
• Fix distributed table with a constant sharding key. #59606 (Vitaly Baranov).
• Fix segmentation fault in KQL parser when the input query exceeds the max_query_size. Also re-enable the KQL dialect. Fixes #59036 and #59037. #59626 (Yong Wang).
• Fix error Read beyond last offset for AsynchronousBoundedReadBuffer. #59630 (Vitaly Baranov).
• Maintain function alias in RewriteSumFunctionWithSumAndCountVisitor. #59658 (Raúl Marín).
• Fix query start time on non initial queries. #59662 (Raúl Marín).
• Validate types of arguments for minmax skipping index. #59733 (Anton Popov).
• Fix leftPad / rightPad function with FixedString input. #59739 (Raúl Marín).
• Fixed an exception in function countMatches with non-const FixedString haystack arguments, e.g. SELECT countMatches(materialize(toFixedString('foobarfoo', 9)), 'foo');. #59752 (Robert Schulze).
• Fix having neigher acked nor nacked messages. If exception happens during read-write phase, messages will be nacked. #59775 (Kseniia Sumarokova).
• Fixed queries that read a Parquet file over HTTP (url()/URL()) executing in one thread instead of max_threads. #59833 (Michael Kolupaev).
• Fixed uninitialized value in s3 queue, which happened during upgrade to a new version if table had Ordered mode and resulted in an error "Existing table metadata in ZooKeeper differs in s3queue_processing_threads_num setting". #59897 (Kseniia Sumarokova).
• Fix parsing of partition expressions that are surrounded by parentheses, e.g.: ALTER TABLE test DROP PARTITION ('2023-10-19'). #59901 (János Benjamin Antal).
• Fix crash in JSONColumnsWithMetadata format over http. Closes #59853. #59925 (Kruglov Pavel).
• Do not rewrite sum() to count() if return value differs in analyzer. #59926 (Azat Khuzhin).
• Fix crash during deserialization of aggregation function states that internally use UniqExactSet. Introduced https://github.com/ClickHouse/ClickHouse/pull/59009. #59928 (Maksim Kita).
• ReplicatedMergeTree fix invalid metadata_version node initialization in Zookeeper during creation of non first replica. Closes #54902. #59946 (Maksim Kita).
• Fixed data race on cluster object between StorageDistributed and Context::reloadClusterConfig(). Former held const reference to its member while the latter destroyed the object (in process of replacing it with a new one). #59987 (Nikita Taranov).
• Fixes #59989: runs init scripts when force-enabled or when no database exists, rather than the inverse. #59991 (jktng).
• This PR fixes scale conversion for DateTime64 values (for example, DateTime64(6)->DateTime64(3)). ```SQL create table test (result DateTime64(3)) engine=Memory;. #60004 (Yarik Briukhovetskyi).
• Fix INSERT into SQLite with single quote (by properly escaping single quotes with a quote instead of backslash). #60015 (Azat Khuzhin).
• Fix several logical errors in arrayFold. Fixes support for Nullable and LowCardinality. #60022 (Raúl Marín).
• Fix optimize_uniq_to_count removing the column alias. #60026 (Raúl Marín).
• Fix possible error while dropping s3queue table, like "no node shard0". #60036 (Kseniia Sumarokova).
• Fix formatting of NOT with single literals. #60042 (Raúl Marín).
• Use max_query_size from context in parsing changed settings in DDLWorker. Previously with large number of changed settings DDLWorker could fail with Max query size exceeded error and don't process log entries. #60083 (Kruglov Pavel).
• Fix inconsistent formatting of queries containing tables named table. Fix wrong formatting of queries with UNION ALL, INTERSECT, and EXCEPT when their structure wasn't linear. This closes #52349. Fix wrong formatting of SYSTEM queries, including SYSTEM ... DROP FILESYSTEM CACHE, SYSTEM ... REFRESH/START/STOP/CANCEL/TEST VIEW, SYSTEM ENABLE/DISABLE FAILPOINT. Fix formatting of parameterized DDL queries. Fix the formatting of the DESCRIBE FILESYSTEM CACHE query. Fix incorrect formatting of the SET param_... (a query setting a parameter). Fix incorrect formatting of CREATE INDEX queries. Fix inconsistent formatting of CREATE USER and similar queries. Fix inconsistent formatting of CREATE SETTINGS PROFILE. Fix incorrect formatting of ALTER ... MODIFY REFRESH. Fix inconsistent formatting of window functions if frame offsets were expressions. Fix inconsistent formatting of RESPECT NULLS and IGNORE NULLS if they were used after a function that implements an operator (such as plus). Fix idiotic formatting of SYSTEM SYNC REPLICA ... LIGHTWEIGHT FROM .... Fix inconsistent formatting of invalid queries with GROUP BY GROUPING SETS ... WITH ROLLUP/CUBE/TOTALS. Fix inconsistent formatting of GRANT CURRENT GRANTS. Fix inconsistent formatting of CREATE TABLE (... COLLATE). Additionally, I fixed the incorrect formatting of EXPLAIN in subqueries (#60102). Fixed incorrect formatting of lambda functions (#60012). Added a check so there is no way to miss these abominations in the future. #60095 (Alexey Milovidov).
• Queries like SELECT * FROM (EXPLAIN ...) were formatted incorrectly. #60102 (Alexey Milovidov).
• Fix cosineDistance crash with Nullable. #60150 (Raúl Marín).
• Boolean values in string representation now cast to true bools. E.g. this query previously threw an exception but now works: SELECT true = 'true'. #60160 (Robert Schulze).
• Fix non-filled column table_uuid in system.s3queue_log. Added columns database and table. Renamed table_uuid to uuid. #60166 (Kseniia Sumarokova).
• Fix arrayReduce with nullable aggregate function name. #60188 (Raúl Marín).
• Fix actions execution during preliminary filtering (PK, partition pruning). #60196 (Azat Khuzhin).
• Hide sensitive info for S3Queue table engine. #60233 (Kseniia Sumarokova).
• Restore the previous syntax ORDER BY ALL which has temporarily (for a few days) been replaced by ORDER BY *. #60248 (Robert Schulze).
• Fixed a minor bug that caused all http return codes to be 200 (success) instead of a relevant code on exception. #60252 (Austin Kothig).
• Fix bug in S3Queue table engine with ordered parallel mode. #60282 (Kseniia Sumarokova).
• Fix use-of-uninitialized-value and invalid result in hashing functions with IPv6. #60359 (Kruglov Pavel).
• Fix OptimizeDateOrDateTimeConverterWithPreimageVisitor with null arguments. #60453 (Raúl Marín).
• Fixed a minor bug that prevented distributed table queries sent from either KQL or PRQL dialect clients to be executed on replicas. #60470 (Alexey Milovidov).
• Fix incomplete results with s3Cluster when multiple threads are used. #60477 (Antonio Andelic).

CI Fix or Improvement (changelog entry is not required)

• ... #60457 (Max K.).
• ... #60512 (Max K.).
• Arm and amd docker build jobs use similar job names and thus overwrite job reports - aarch64 and amd64 suffixes added to fix this. #60554 (Max K.).
• ... #60557 (Max K.).
• BUG: build job can report success cache record on failed build Add a check relying on job report fail. #60587 (Max K.).

NO CL ENTRY

• NO CL ENTRY: 'Revert "Revert "Add new aggregation function groupArraySorted()""'. #59003 (Maksim Kita).
• NO CL ENTRY: 'Revert "Update libxml2 version to address some bogus security issues"'. #59479 (Raúl Marín).
• NO CL ENTRY: 'Revert "Poco Logger small refactoring"'. #59509 (Raúl Marín).
• NO CL ENTRY: 'Revert "Revert "Poco Logger small refactoring""'. #59564 (Maksim Kita).
• NO CL ENTRY: 'Revert "MergeTree FINAL optimization diagnostics and settings"'. #59702 (Raúl Marín).
• NO CL ENTRY: 'Revert "Use MergeTree as a default table engine"'. #59711 (Raúl Marín).
• NO CL ENTRY: 'Revert "Rename a setting"'. #59754 (Raúl Marín).
• NO CL ENTRY: 'Revert "Less error prone interface of read buffers"'. #59911 (Nikita Mikhaylov).
• NO CL ENTRY: 'Revert "Update version_date.tsv and changelogs after v24.1.4.19-stable"'. #59973 (Mikhail f. Shiryaev).
• NO CL ENTRY: 'Revert "ReplicatedMergeTree invalid metadata_version fix"'. #60058 (Raúl Marín).
• NO CL ENTRY: 'Revert "Revert "ReplicatedMergeTree invalid metadata_version fix""'. #60078 (Maksim Kita).
• NO CL ENTRY: 'Revert "Implement system.dns_cache table"'. #60085 (Alexey Milovidov).
• NO CL ENTRY: 'Revert "Restriction for the access key id for s3."'. #60181 (Alexey Milovidov).
• NO CL ENTRY: 'Revert "Do not retry queries if container is down in integration tests"'. #60215 (Antonio Andelic).
• NO CL ENTRY: 'Revert "Check stack size in Parser"'. #60216 (Antonio Andelic).
• NO CL ENTRY: 'Revert "Support resource request canceling"'. #60253 (Raúl Marín).
• NO CL ENTRY: 'Revert "Add definers for views"'. #60350 (Raúl Marín).
• NO CL ENTRY: 'Update build-osx.md'. #60380 (rogeryk).
• NO CL ENTRY: 'Revert "Fix: IAST::clone() for RENAME"'. #60398 (Antonio Andelic).
• NO CL ENTRY: 'Revert "Add table function mergeTreeIndex"'. #60428 (Alexander Tokmakov).
• NO CL ENTRY: 'Revert "Userspace page cache"'. #60550 (Alexander Tokmakov).
• NO CL ENTRY: 'Revert "Analyzer: compute ALIAS columns right after reading"'. #60570 (Alexander Tokmakov).

NOT FOR CHANGELOG / INSIGNIFICANT

• Analyzer: support aliases and distributed JOINs in StorageMerge #50894 (Dmitry Novik).
• Userspace page cache #53770 (Michael Kolupaev).
• Simplify optimize-push-to-prewhere from query plan #58554 (Nikolai Kochetov).
• Create ch/chc/chl symlinks by cmake as well (for develop mode) #58609 (Azat Khuzhin).
• CI: ci cache. step 1 #58664 (Max K.).
• Enable building JIT with UBSAN #58952 (Raúl Marín).
• Support resource request canceling #59032 (Sergei Trifonov).
• Analyzer: Do not resolve remote table id on initiator #59073 (Dmitry Novik).
• Analyzer: Add cast for ConstantNode from constant folding #59121 (Dmitry Novik).
• Fix the default value of async_insert_max_data_size in EN document #59161 (Alex Cheng).
• CI: Add ARM integration tests #59241 (Max K.).
• Fix getting filename from read buffer wrappers #59298 (Kruglov Pavel).
• Update AWS SDK to 1.11.234 #59299 (Nikita Taranov).
• Split ISlotControl from ConcurrencyControl #59313 (Sergei Trifonov).
• Some small fixes for docker images #59337 (Mikhail f. Shiryaev).
• CI: bugfix-validate, integration, functional test scripts updates #59348 (Max K.).
• MaterializedMySQL: Fix gtid_after_attach_test to retry on detach #59370 (Val Doroshchuk).
• Poco Logger small refactoring #59375 (Maksim Kita).
• Add sanity checks for function return types #59379 (Raúl Marín).
• Cleanup connection pool surroundings #59380 (Igor Nikonov).
• Fix ARRAY JOIN with subcolumns #59382 (vdimir).
• Update curl submodule to be version 8.50 to address the irrelevant CVE-2023-46218 and CVE-2023-49219, which we don't care about at all. #59384 (josh-hildred).
• Update libxml2 version to address some bogus security issues #59386 (josh-hildred).
• Update version after release #59393 (Alexey Milovidov).
• Job names #59395 (Mikhail f. Shiryaev).
• CI: fix status and report for docker server jobs #59396 (Max K.).
• Update version_date.tsv and changelogs after v24.1.1.2048-stable #59397 (robot-clickhouse).
• Forward declaration for PeekableReadBuffer #59399 (Azat Khuzhin).
• Progress bar: use FQDN to differentiate metrics from different hosts #59404 (Igor Nikonov).
• Fix test test_stop_other_host_during_backup #59432 (Vitaly Baranov).
• Update run.sh #59433 (Kseniia Sumarokova).
• Post a failure status if can not run the CI #59440 (Mikhail f. Shiryaev).
• Safer Rust (catch panic with catch_unwind()) #59447 (Azat Khuzhin).
• More parallel insert-select pipeline #59448 (Nikita Taranov).
• CLion says these headers are unused #59451 (Alexey Milovidov).
• Fix 02720_row_policy_column_with_dots #59453 (Duc Canh Le).
• Fix problem detected by UBSAN #59461 (Raúl Marín).
• Analyzer: Fix denny_crane #59483 (vdimir).
• Fix 00191_aggregating_merge_tree_and_final #59494 (Nikita Taranov).
• Avoid running all checks when aspell-dict.txt was changed #59496 (Aleksandr Musorin).
• Fixes for binary.html #59499 (Alexey Milovidov).
• Parallel replicas: better initial replicas failover (2) #59501 (Igor Nikonov).
• Update version_date.tsv and changelogs after v24.1.2.5-stable #59510 (robot-clickhouse).
• Update version_date.tsv and changelogs after v23.12.3.40-stable #59511 (robot-clickhouse).
• Update version_date.tsv and changelogs after v23.11.5.29-stable #59515 (robot-clickhouse).
• Update memory tracker periodically with cgroup memory usage #59516 (Robert Schulze).
• Remove a scary message if an error is retryable #59517 (alesapin).
• Update the peter-evans/create-pull-request action to v6 #59520 (Mikhail f. Shiryaev).
• Fix usage of StatusType #59527 (Mikhail f. Shiryaev).
• Analyzer fix test_select_access_rights/test_main.py::test_select_count #59528 (vdimir).
• GRPCServer: do not call value() on empty optional query_info #59533 (Sema Checherinda).
• Use ConnectionPoolPtr instead of raw pointer #59534 (Igor Nikonov).
• Fix data race with AggregatedDataVariants #59537 (Nikita Taranov).
• Refactoring of dashboard state encoding #59554 (Sergei Trifonov).
• CI: ci_cache, enable await #59555 (Max K.).
• Bump libssh to 0.9.8 #59563 (Robert Schulze).
• MultiVersion use mutex #59565 (Maksim Kita).
• Fix aws submodule reference #59566 (Raúl Marín).
• Add missed #include and #59567 (Mikhnenko Sasha).
• CI: nightly job to update latest docker tag only #59586 (Max K.).
• Analyzer: compute ALIAS columns right after reading #59595 (vdimir).
• Add another sanity check for function return types #59605 (Raúl Marín).
• Update README.md #59610 (Tyler Hannan).
• Updated a list of trusted contributors #59616 (Nikita Mikhaylov).
• CI: fix ast fuzzer job report (slack bot issue) #59629 (Max K.).
• MergeTree FINAL optimization diagnostics and settings #59650 (Maksim Kita).
• Fix default path when path is not specified in config #59654 (Alexander Tokmakov).
• Follow up for #59277 #59659 (Konstantin Bogdanov).
• Pin python dependencies in stateless tests #59663 (Raúl Marín).
• Unquote FLAG_LATEST to fix issue with empty argument #59672 (Mikhail f. Shiryaev).
• Temporarily remove a feature that doesn't work #59688 (Alexander Tokmakov).
• ConnectionEstablisher: remove unused is_finished #59706 (Igor Nikonov).
• Add test for increase-always autoscaling lambda #59709 (Mikhail f. Shiryaev).
• Remove SourceWithKeyCondition from ReadFromStorageStep #59720 (Nikolai Kochetov).
• Make ZooKeeper actually sequentialy consistent #59735 (Alexander Tokmakov).
• Add assertions around FixedString code #59737 (Raúl Marín).
• Fix skipping unused shards with analyzer #59741 (Vitaly Baranov).
• Fix DB type check - now it'll refuse to create in Replicated databases #59743 (Michael Kolupaev).
• Analyzer: Fix test_replicating_constants/test.py::test_different_versions #59750 (Dmitry Novik).
• Fix dashboard params default values #59753 (Sergei Trifonov).
• Fix logical optimizer with LowCardinality in new analyzer #59766 (Antonio Andelic).
• Update libuv #59773 (Kseniia Sumarokova).
• Followup #50894 #59774 (Dmitry Novik).
• CI: ci test await #59778 (Max K.).
• Better logging for adaptive async timeouts #59781 (Julia Kartseva).
• Fix broken youtube embedding in ne-tormozit.md #59782 (Shaun Struwig).
• Hide URL/S3 'headers' argument in SHOW CREATE #59787 (Michael Kolupaev).
• Fix special build reports in release branches #59797 (Mikhail f. Shiryaev).
• CI: do not reuse builds on release branches #59798 (Max K.).
• Update version_date.tsv and changelogs after v24.1.3.31-stable #59799 (robot-clickhouse).
• Update version_date.tsv and changelogs after v23.12.4.15-stable #59800 (robot-clickhouse).
• Analyzer: fix test_access_for_functions/test.py::test_access_rights_for_function #59801 (Dmitry Novik).
• Analyzer: Fix test_wrong_db_or_table_name/test.py::test_wrong_table_name #59806 (Dmitry Novik).
• CI: await tune ups #59807 (Max K.).
• Enforce tests with enabled analyzer in CI #59814 (Dmitry Novik).
• Handle different timestamp related aspects of zip-files #59815 (Mikhail f. Shiryaev).
• Fix settings history azure_max_single_part_copy_size #59819 (SmitaRKulkarni).
• Updated a list of trusted contributors #59844 (Maksim Kita).
• Add check for recursiveRemoveLowCardinality() #59845 (Vitaly Baranov).
• Better warning for disabled kernel.task_delayacct #59846 (Azat Khuzhin).
• Reintroduce 02590_interserver_mode_client_info_initial_query_start_time #59851 (Azat Khuzhin).
• Respect CMAKE_OSX_DEPLOYMENT_TARGET for Rust targets #59852 (Azat Khuzhin).
• Do not reinitialize ZooKeeperWithFaultInjection on each chunk #59854 (Alexander Gololobov).
• Fix: check if std::function is set before calling it #59858 (Igor Nikonov).
• Fix long shutdown of FileLog storage #59873 (Azat Khuzhin).
• tests: fix 02322_sql_insert_format flakiness #59874 (Azat Khuzhin).
• Follow up for #58554. Cleanup. #59889 (Nikolai Kochetov).
• CI: Fix job failures due to jepsen artifacts #59890 (Max K.).
• Add test 02988_join_using_prewhere_pushdown #59892 (vdimir).
• Do not pull mutations if pulling replication log had been stopped #59895 (Azat Khuzhin).
• Fix 02982_comments_in_system_tables #59896 (János Benjamin Antal).
• Refactor Squashing for inserts. #59899 (Nikolai Kochetov).
• Do not rebuild a lambda package if it is updated #59902 (Mikhail f. Shiryaev).
• Fix self-extracting: macOS doesn't allow to run renamed executable - copy instead #59906 (Yakov Olkhovskiy).
• Update tests with indexHint for analyzer. #59907 (Nikolai Kochetov).
• Petite cleanup around macros and ReplicatedMergeTree #59909 (Azat Khuzhin).
• Fix: absence of closing record in query_log for failed insert over http #59910 (Igor Nikonov).
• Decrease logging level for http retriable errors to Warning (and fix 00157_cache_dictionary flakiness) #59920 (Azat Khuzhin).
• Remove test_distributed_backward_compatability #59921 (Dmitry Novik).
• Commands node args should add rvalue to push_back to reduce object copy cost #59922 (xuzifu666).
• tests: fix 02981_vertical_merges_memory_usage flakiness #59923 (Azat Khuzhin).
• Analyzer: Update broken integration tests list #59924 (Dmitry Novik).
• CI: integration tests to mysql80 #59939 (Max K.).
• Register StorageMergeTree exception message fix #59941 (Maksim Kita).
• Replace lambdas with pointers to members to simplify stacks #59944 (Alexander Gololobov).
• Analyzer: Fix test_user_defined_object_persistence #59948 (Dmitry Novik).
• Analyzer: Fix test_mutations_with_merge_tree #59951 (Dmitry Novik).
• Cleanups #59964 (Alexey Milovidov).
• Update version_date.tsv and changelogs after v24.1.4.19-stable #59966 (robot-clickhouse).
• Less conflicts #59968 (Alexey Milovidov).
• We don't have external dictionaries from Aerospike #59969 (Alexey Milovidov).
• Fix max num to warn message #59972 (Jordi Villar).
• Analyzer: Fix test_settings_profile #59975 (Dmitry Novik).
• Update version_date.tsv and changelogs after v24.1.4.20-stable #59978 (robot-clickhouse).
• Analyzer: Fix test_storage_rabbitmq #59981 (Dmitry Novik).
• Analyzer: Fix test_shard_level_const_function #59983 (Dmitry Novik).
• Add newlines to SettingsChangesHistory to maybe have less conflicts #59984 (Nikolai Kochetov).
• Remove context from comparison functions. #59985 (Nikolai Kochetov).
• Update version_date.tsv and changelogs after v24.1.5.6-stable #59993 (robot-clickhouse).
• Fix mark release ready #59994 (Mikhail f. Shiryaev).
• Revert "Insert synchronously if dependent MV deduplication is enabled" #59998 (Julia Kartseva).
• Fix obviously wrong (but non significant) error in dictionaries #60005 (Alexey Milovidov).
• Inhibit randomization in some tests #60009 (Alexey Milovidov).
• The code should not be complex #60010 (Alexey Milovidov).
• Exclude test run from a slow build #60011 (Alexey Milovidov).
• Fix broken lambdas formatting #60012 (Alexey Milovidov).
• Verify formatting consistency on the server-side #60013 (Alexey Milovidov).
• Analyzer: Fix test_sql_user_defined_functions_on_cluster #60019 (Dmitry Novik).
• Fix 02981_vertical_merges_memory_usage with SharedMergeTree #60028 (Raúl Marín).
• Fix 01656_test_query_log_factories_info with analyzer. #60037 (Nikolai Kochetov).
• Ability to detect undead ZooKeeper sessions #60044 (Alexander Tokmakov).
• Disable tests with coverage #60047 (Alexey Milovidov).
• Print CPU flags at startup #60075 (Alexey Milovidov).
• Cleanup: less confusion between config priority and balancing priority in connection pools #60077 (Igor Nikonov).
• Temporary table already exists exception message fix #60080 (Maksim Kita).
• Refactor prewhere and primary key optimization #60082 (Amos Bird).
• Bump curl to version 4.6.0 #60084 (josh-hildred).
• Check wrong abbreviations #60086 (Alexey Milovidov).
• Remove the check for formatting consistency from the Fuzzer #60088 (Alexey Milovidov).
• Avoid overflow in settings #60089 (Alexey Milovidov).
• A small preparation for better handling of primary key in memory #60092 (Alexey Milovidov).
• Move threadPoolCallbackRunner to the "Common" folder #60097 (Vitaly Baranov).
• Speed up the CI #60106 (Alexey Milovidov).
• Insignificant changes #60108 (Alexey Milovidov).
• Do not retry queries if container is down in integration tests #60109 (Azat Khuzhin).
• Better check for inconsistent formatting #60110 (Alexey Milovidov).
• skip printing meaningless log #60123 (conic).
• Implement TODO #60124 (Alexey Milovidov).
• Fix bad log message #60125 (Alexey Milovidov).
• Fix data race in IMergeTreeDataPart #60139 (Antonio Andelic).
• Add new setting to changes history #60141 (Antonio Andelic).
• Analyzer: fix row level filters with PREWHERE + additional filters #60142 (vdimir).
• Tests: query log for inserts over http #60143 (Igor Nikonov).
• Fix build in master #60151 (Raúl Marín).
• Add setting history check to stateless tests #60154 (Raúl Marín).
• Mini cleanup of CPUID.h #60155 (Robert Schulze).
• Fix: custom key failover test flakiness #60158 (Igor Nikonov).
• Skip sanity checks on secondary CREATE query #60159 (Alexander Tokmakov).
• Remove extensively aggressive check #60162 (Alexey Milovidov).
• Fix wrong message during compilation #60178 (Alexey Milovidov).
• Add a test for #44318 #60179 (Alexey Milovidov).
• Add test for 59437 #60191 (Raúl Marín).
• CI: hot fix for gh statuses #60201 (Max K.).
• Limit libarchive format to what we use #60203 (San).
• Fix bucket region discovery #60204 (Nikita Taranov).
• Fix test_backup_restore_s3/test.py::test_user_specific_auth #60210 (Antonio Andelic).
• CI: combine analyzer, s3, dbreplicated into one job #60224 (Max K.).
• Slightly better Keeper loading from snapshot #60226 (Antonio Andelic).
• Fix: IAST::clone() for RENAME #60227 (Igor Nikonov).
• Treat 2+ in allow_experimental_parallel_reading_from_replicas as 2 #60228 (Raúl Marín).
• CI: random job pick support #60229 (Max K.).
• Fix analyzer - hide arguments for secret functions #60230 (Yakov Olkhovskiy).
• Backups delete suspicious file #60231 (Maksim Kita).
• CI: random sanitizer for parallel repl in PR wf #60234 (Max K.).
• CI: use aarch runner for runconfig job #60236 (Max K.).
• Add test for 60232 #60244 (Raúl Marín).
• Make cloud sync required #60245 (Raúl Marín).
• Tests from #60094 #60256 (Alexey Milovidov).
• Remove bad check in Keeper #60266 (Antonio Andelic).
• Fix integration test_backup_restore_s3 #60269 (Alexey Milovidov).
• Ignore valid 'No such key errors' in stress tests #60270 (Raúl Marín).
• Stress test: Include the first sanitizer block message in the report #60283 (Raúl Marín).
• Update analyzer_tech_debt.txt #60303 (Alexander Tokmakov).
• Minor fixes for hashed dictionary #60310 (vdimir).
• Install tailscale during AMI build and set it up on runners #60316 (Mikhail f. Shiryaev).
• CI: remove Integration tests asan and release from PR wf #60327 (Max K.).
• Fix - analyzer related - "executable" function subquery arguments. #60339 (Yakov Olkhovskiy).
• Update settings.md to correct the description for setting max_concurrent_queries_for_user #60343 (Alex Cheng).
• Fix rapidjson submodule #60346 (Raúl Marín).
• Validate experimental and suspicious types inside nested types under a setting #60353 (Kruglov Pavel).
• Update 01158_zookeeper_log_long.sql #60357 (Alexander Tokmakov).
• Add missed #include #60358 (Mikhnenko Sasha).
• Follow up #60082 #60360 (Nikolai Kochetov).
• Remove ALTER LIVE VIEW #60370 (Alexey Milovidov).
• Detect io_uring in tests #60373 (Azat Khuzhin).
• Expose fatal.log separately for fuzzer #60374 (Azat Khuzhin).
• Minor changes for dashboard #60387 (Alexey Milovidov).
• Remove unused method #60388 (Alexey Milovidov).
• Allow to map UI handlers to different paths #60389 (Alexey Milovidov).
• Remove old tags from integration tests #60407 (Raúl Marín).
• Update liburing to 2.5 #60409 (Nikita Taranov).
• Fix undefined-behavior in case of too big max_execution_time setting #60419 (Kruglov Pavel).
• Fix wrong log message in Fuzzer #60425 (Alexey Milovidov).
• Fix unrestricted reads from keeper #60429 (Raúl Marín).
• Split update_mergeable_check into two functions to force trigger the status #60431 (Mikhail f. Shiryaev).
• Revert "Revert "Add table function mergeTreeIndex"" #60435 (Anton Popov).
• Revert "Merge pull request #56864 from ClickHouse/broken-projections-better-handling" #60436 (Nikita Taranov).
• Keeper: fix moving changelog files between disks #60442 (Antonio Andelic).
• Replace deprecated distutils by vendored packaging #60444 (Mikhail f. Shiryaev).
• Do not fail the build if ci-logs is not healthy #60445 (Mikhail f. Shiryaev).
• Move setting enable_order_by_all out of the experimental setting section #60449 (Robert Schulze).
• Minor: Replace boost::algorithm::starts_with() by std::string::starts_with() #60450 (Robert Schulze).
• Minor: Replace boost::algorithm::ends_with() by std::string::ends_with() #60454 (Robert Schulze).
• CI: remove input params for job scripts #60455 (Max K.).
• Fix: 02496_remove_redundant_sorting_analyzer #60456 (Igor Nikonov).
• PR template fix to include ci fix category #60461 (Max K.).
• Reduce iterations in 01383_log_broken_table #60465 (Raúl Marín).
• Merge #57434 #60466 (Alexey Milovidov).
• Fix bad test: looks like an obvious race condition, but I didn't check in detail. #60471 (Alexey Milovidov).
• Make test slower #60472 (Alexey Milovidov).
• Fix cgroups v1 rss parsing in CgroupsMemoryUsageObserver #60481 (Maksim Kita).
• CI: fix pr check status to not fail mergeable check #60483 (Max K.).
• Report respects skipped builds #60488 (Mikhail f. Shiryaev).
• CI: quick style fix #60490 (Max K.).
• Decrease logging level for http retriable errors to Info #60508 (Raúl Marín).
• Remove broken test while we fix it #60547 (Raúl Marín).