ClickHouse/docs/changelogs/v23.2.1.2537-stable.md

87 KiB

sidebar_position sidebar_label
1 2023

2023 Changelog

ClickHouse release v23.2.1.2537-stable (52bf836e03) FIXME as compared to v23.1.1.3077-stable (dcaac47702)

Backward Incompatible Change

  • Extend function "toDayOfWeek()" (alias: "DAYOFWEEK") with a mode argument that encodes whether the week starts on Monday or Sunday and whether counting starts at 0 or 1. For consistency with other date time functions, the mode argument was inserted between the time and the time zone arguments. This breaks existing usage of the (previously undocumented) 2-argument syntax "toDayOfWeek(time, time_zone)". A fix is to rewrite the function into "toDayOfWeek(time, 0, time_zone)". #45233 (Robert Schulze).
  • Rename setting max_query_cache_size to filesystem_cache_max_download_size. #45614 (Kseniia Sumarokova).
  • Fix applying settings for FORMAT on the client. #46003 (Azat Khuzhin).
  • Default user will not have permissions for access type SHOW NAMED COLLECTION by default (e.g. by default, default user will not longer be able to do grant ALL to other users as it was before, therefore this PR is backward incompatible). #46010 (Kseniia Sumarokova).
  • Remove support for setting materialized_postgresql_allow_automatic_update (which was by default turned off). Fix integration tests. #46106 (Kseniia Sumarokova).
  • Slightly improve performance of countDigits on realistic datasets. This closed #44518. In previous versions, countDigits(0) returned 0; now it returns 1, which is more correct, and follows the existing documentation. #46187 (Alexey Milovidov).

New Feature

  • Expose ProfileEvents counters in system.part_log. #38614 (Bharat Nallan).
  • Enrichment of the existing ReplacingMergeTree engine to allow duplicates insertion. It leverages the power of both ReplacingMergeTree and CollapsingMergeTree in one mergeTree engine. Deleted data are not returned when queried, but not removed from disk neither. #41005 (youennL-cs).
  • Add generateULID() function. Closes #36536. #44662 (Nikolay Degterinsky).
  • Add corrMatrix Aggregatefunction, calculating each two columns. In addition, since Aggregatefunctions covarSamp and covarPop are similar to corr, I add covarSampMatrix, covarPopMatrix by the way. @alexey-milovidov closes #44587. #44680 (FFFFFFFHHHHHHH).
  • Rewrite aggregate functions with if expression as argument when logically equivalent. For example, avg(if(cond, col, null)) can be rewritten to avgIf(cond, col). It is helpful in performance. #44730 (李扬).
  • Introduce arrayShuffle function for random array permutations. #45271 (Joanna Hulboj).
  • Support types FIXED_SIZE_BINARY type in Arrow, FIXED_LENGTH_BYTE_ARRAY in Parquet and match them to FixedString. Add settings output_format_parquet_fixed_string_as_fixed_byte_array/output_format_arrow_fixed_string_as_fixed_byte_array to control default output type for FixedString. Closes #45326. #45340 (Kruglov Pavel).
  • Add StorageIceberg and table function iceberg to access iceberg table store on S3. #45384 (flynn).
  • Add a new column last_exception_time to system.replication_queue. #45457 (Frank Chen).
  • Add two new functions which allow for user-defined keys/seeds with SipHash{64,128}. #45513 (Salvatore Mesoraca).
  • Allow a three-argument version for table function format. close #45808. #45873 (FFFFFFFHHHHHHH).
  • add joda format support for 'x','w','S'.Refer to https://joda-time.sourceforge.net/apidocs/org/joda/time/format/DateTimeFormat.html. #46073 (zk_kiger).
  • ... Support window function ntile. ``` insert into test_data values(1,2), (1,3), (1,4), (2,5),(2,6); select a, b, ntile(2) over (partition by a order by b rows between unbounded preceding and unbounded following ) from test_data;. #46256 (lgbo).
  • Added arrayPartialSort and arrayPartialReverseSort functions. #46296 (Joanna Hulboj).
  • The new http parameter client_protocol_version allows setting a client protocol version for HTTP responses using the Native format. #40397. #46360 (Geoff Genz).
  • Add new function regexpExtract, like spark function REGEXP_EXTRACT. #46469 (李扬).
  • Author: taiyang-li Add new function regexpExtract, like spark function REGEXP_EXTRACT. #46529 (Alexander Gololobov).
  • Add new function JSONArrayLength, which returns the number of elements in the outermost JSON array. The function returns NULL if input JSON string is invalid. #46631 (李扬).

Performance Improvement

  • Improve lower/upper function performance with avx512 instructions. #37894 (yaqi-zhao).
  • Add new local_filesystem_read_method method io_uring based on the asynchronous Linux io_uring subsystem, improving read performance almost universally compared to the default pread method. #38456 (Saulius Valatka).
  • Remove the limitation that on systems with >=32 cores and SMT disabled ClickHouse uses only half of the cores. #44973 (Robert Schulze).
  • Improve performance of function multiIf by columnar executing, speed up by 2.3x. #45296 (李扬).
  • An option added to aggregate partitions independently if table partition key and group by key are compatible. Controlled by the setting allow_aggregate_partitions_independently. Disabled by default because of limited applicability (please refer to the docs). #45364 (Nikita Taranov).
  • Add fastpath for function position when needle is empty. #45382 (李扬).
  • Enable query_plan_remove_redundant_sorting optimization by default. Optimization implemented in #45420. #45567 (Igor Nikonov).
  • Increased HTTP Transfer Encoding chunk size to improve performance of large queries using the HTTP interface. #45593 (Geoff Genz).
  • Fixed performance of short SELECT queries that read from tables with large number ofArray/Map/Nested columns. #45630 (Anton Popov).
  • Allow using Vertical merge algorithm with parts in Compact format. This will allow ClickHouse server to use much less memory for background operations. This closes #46084. #45681 (Anton Popov).
  • Optimize Parquet reader by using batch reader. #45878 (LiuNeng).
  • Improve performance of ColumnArray::filter for big int and decimal. #45949 (李扬).
  • This change could effectively reduce the overhead of obtaining the filter from ColumnNullable(UInt8) and improve the overall query performance. To evaluate the impact of this change, we adopted TPC-H benchmark but revised the column types from non-nullable to nullable, and we measured the QPS of its queries as the performance indicator. #45962 (Zhiguo Zhou).
  • Make the _part and _partition_id virtual column be LowCardinality(String) type. Closes #45964. #45975 (flynn).
  • Improve the performance of Decimal conversion when the scale does not change. #46095 (Alexey Milovidov).
  • The introduced logic works if PREWHERE condition is a conjunction of multiple conditions (cond1 AND cond2 AND ... ). It groups those conditions that require reading the same columns into steps. After each step the corresponding part of the full condition is computed and the result rows might be filtered. This allows to read fewer rows in the next steps thus saving IO bandwidth and doing less computation. This logic is disabled by default for now. It will be enabled by default in one of the future releases once it is known to not have any regressions, so it is highly encouraged to be used for testing. It can be controlled by 2 settings: "enable_multiple_prewhere_read_steps" and "move_all_conditions_to_prewhere". #46140 (Alexander Gololobov).
  • Allow to increase prefetching for read data. #46168 (Kseniia Sumarokova).
  • Rewrite arrayExists(x -> x = 1, arr) -> has(arr, 1), which improve performance by 1.34x. #46188 (李扬).
  • Fix too big memory usage for vertical merges on non-remote disk. Respect max_insert_delayed_streams_for_parallel_write for the remote disk. #46275 (Nikolai Kochetov).
  • Update zstd to v1.5.4. It has some minor improvements in performance and compression ratio. If you run replicas with different versions of ClickHouse you may see reasonable error messages Data after merge/mutation is not byte-identical to data on another replicas. with explanation. These messages are Ok and you should not worry. #46280 (Raúl Marín).
  • Allow using Vertical merge algorithm with parts in Compact format. This will allow ClickHouse server to use much less memory for background operations. This closes #46084. #46282 (Anton Popov).
  • Fix performance degradation caused by #39737. #46309 (Alexey Milovidov).
  • The replicas_status handle will answer quickly even in case of a large replication queue. #46310 (Alexey Milovidov).

Improvement

  • Add avx512 support for Aggregate Sum, function unary arithmetic, function comparison. #37870 (zhao zhou).
  • close issue: #38893. #38950 (hexiaoting).
  • Migration from other databases and updates/deletes are mimicked by Collapsing/Replacing. Want to use the same SELECT queries without adding FINAL to all the existing queries. #40945 (Arthur Passos).
  • Allow configuring storage as SETTINGS disk='<disk_name>' (instead of storage_policy) and with explicit disk creation SETTINGS disk=disk(type=s3, ...). #41976 (Kseniia Sumarokova).
  • Add new metrics for backups: num_processed_files and processed_files_size described actual number of processed files. #42244 (Aleksandr).
  • Added retries on interserver DNS errors. #43179 (Anton Kozlov).
  • Rewrote the code around marks distribution and the overall coordination of the reading in order to achieve the maximum performance improvement. This closes #34527. #43772 (Nikita Mikhaylov).
  • Remove redundant DISTINCT clauses in query (subqueries). Implemented on top of query plan. It does similar optimization as optimize_duplicate_order_by_and_distinct regarding DISTINCT clauses. Can be enabled via query_plan_remove_redundant_distinct setting. Related to #42648. #44176 (Igor Nikonov).
  • Keeper improvement: try preallocating space on the disk to avoid undefined out-of-space issues. Introduce setting max_log_file_size for the maximum size of Keeper's Raft log files. #44370 (Antonio Andelic).
  • sumIf(123, cond) -> 123 * countIf(1, cond) sum(if(cond, 123, 0)) -> 123 * countIf(cond) sum(if(cond, 0, 123)) -> 123 * countIf(not(cond)). #44728 (李扬).
  • Optimize behavior for a replica delay api logic in case the replica is read-only. #45148 (mateng915).
  • Introduce gwp-asan implemented by llvm runtime. This closes #27039. #45226 (Han Fei).
  • ... in the case key casted from uint64 to uint32, small impact for little endian platform but key value becomes zero in big endian case. ### Documentation entry for user-facing changes. #45375 (Suzy Wang).
  • Mark Gorilla compression on columns of non-Float* type as suspicious. #45376 (Robert Schulze).
  • Allow removing redundant aggregation keys with constants (e.g., simplify GROUP BY a, a + 1 to GROUP BY a). #45415 (Dmitry Novik).
  • Show replica name that is executing a merge in the postpone_reason. #45458 (Frank Chen).
  • Save exception stack trace in part_log. #45459 (Frank Chen).
  • Make RegExpTreeDictionary a ua parser which is compatible with https://github.com/ua-parser/uap-core. #45631 (Han Fei).
  • Enable ICU data support on s390x platform. #45632 (Suzy Wang).
  • Updated checking of SYSTEM SYNC REPLICA resolves #45508 Implementation: * Updated to wait for current last entry to be processed (after pulling shared log) instead of queue size becoming 0. * Updated Subscriber to notify both queue size and removed log_entry_id. #45648 (SmitaRKulkarni).
  • Disallow creation of new columns compressed by a combination of codecs "Delta" or "DoubleDelta" followed by codecs "Gorilla" or "FPC". This can be bypassed using setting "allow_suspicious_codecs = true". #45652 (Robert Schulze).
  • Rename setting replication_alter_partitions_sync to alter_sync. #45659 (Antonio Andelic).
  • The generateRandom table function and the engine now support LowCardinality data types. This is useful for testing, for example you can write INSERT INTO table SELECT * FROM generateRandom() LIMIT 1000. This is needed to debug #45590. #45661 (Alexey Milovidov).
  • Add ability to ignore unknown keys in JSON object for named tuples (input_format_json_ignore_unknown_keys_in_named_tuple). #45678 (Azat Khuzhin).
    • The experimental query result cache now provides more modular configuration settings. #45679 (Robert Schulze).
  • Renamed "query result cache" to "query cache". #45682 (Robert Schulze).
  • add SYSTEM SYNC FILE CACHE command. It will call sync syscall. It achieve #8921. #45685 (DR).
  • Add new S3 setting allow_head_object_request. This PR makes usage of GetObjectAttributes request instead of HeadObject introduced in https://github.com/ClickHouse/ClickHouse/pull/45288 optional (and disabled by default). #45701 (Vitaly Baranov).
  • Add ability to override connection settings based on connection names (that said that now you can forget about storing password for each connection, you can simply put everything into ~/.clickhouse-client/config.xml and even use different history files for them, which can be also useful). #45715 (Azat Khuzhin).
  • Arrow format support duration type. Closes #45669. #45750 (flynn).
  • Extend the logging in the Query Cache to improve investigations of the caching behavior. #45751 (Robert Schulze).
  • The query cache's server-level settings are now reconfigurable at runtime. #45758 (Robert Schulze).
  • Hide password in logs when a table function's arguments are specified with a named collection:. #45774 (Vitaly Baranov).
  • Improve internal S3 client to correctly deduce regions and redirections for different types of URLs. #45783 (Antonio Andelic).
    • Add support for Map, IPv4 and IPv6 types in generateRandom. Mostly useful for testing. #45785 (Raúl Marín).
  • Support empty/notEmpty for IP types. #45799 (Yakov Olkhovskiy).
  • The column num_processed_files was splitted into two columns: num_files (for BACKUP) and files_read (for RESTORE). The column processed_files_size was splitted into two columns: total_size (for BACKUP) and bytes_read (for RESTORE). #45800 (Vitaly Baranov).
    1. Upgrade Intel QPL from v0.3.0 to v1.0.0 2. Build libaccel-config and link it statically to QPL library instead of dynamically. #45809 (jasperzhu).
  • Add support for SHOW ENGINES query. #45859 (Filatenkov Artur).
  • Improved how memory bound merging and aggregation in order on top query plan interact. Previously we fell back to explicit sorting for AIO in some cases when it wasn't actually needed. So it is a perf issue, not a correctness one. #45892 (Nikita Taranov).
  • Improve behaviour of conversion into Date for boundary value 65535 (2149-06-06). #45914 (Joanna Hulboj).
  • Add setting check_referential_table_dependencies to check referential dependencies on DROP TABLE. This PR solves #38326. #45936 (Vitaly Baranov).
  • Fix tupleElement return Null when having Null argument. Closes #45894. #45952 (flynn).
  • Throw an error on no files satisfying S3 wildcard. Closes #45587. #45957 (chen).
  • Use cluster state data to check concurrent backup/restore. #45982 (SmitaRKulkarni).
  • Use "exact" matching for fuzzy search, which has correct case ignorance and more appropriate algorithm for matching SQL queries. #46000 (Azat Khuzhin).
  • Improve behaviour of conversion into Date for boundary value 65535 (2149-06-06). #46042 (Joanna Hulboj).
  • Forbid wrong create View syntax CREATE View X TO Y AS SELECT. Closes #4331. #46043 (flynn).
  • Storage Log family support settings storage_policy. Closes #43421. #46044 (flynn).
  • Improve format JSONColumns when result is empty. Closes #46024. #46053 (flynn).
  • Add reference implementation for SipHash128. #46065 (Salvatore Mesoraca).
  • Add new metric to record allocations times and bytes using mmap. #46068 (李扬).
  • Currently for functions like leftPad, rightPad, leftPadUTF8, rightPadUTF8, the second argument length must be UInt8|16|32|64|128|256. Which is too strict for clickhouse users, besides, it is not consistent with other similar functions like arrayResize, substring and so on. #46103 (李扬).
  • Update CapnProto to v0.10.3 to avoid CVE-2022-46149 ### Documentation entry for user-facing changes. #46139 (Mallik Hassan).
  • Fix assertion in the welchTTest function in debug build when the resulting statistics is NaN. Unified the behavior with other similar functions. Change the behavior of studentTTest to return NaN instead of throwing an exception because the previous behavior was inconvenient. This closes #41176 This closes #42162. #46141 (Alexey Milovidov).
  • More convenient usage of big integers and ORDER BY WITH FILL. Allow using plain integers for start and end points in WITH FILL when ORDER BY big (128-bit and 256-bit) integers. Fix the wrong result for big integers with negative start or end points. This closes #16733. #46152 (Alexey Milovidov).
  • Add parts, active_parts and total_marks columns to system.tables on issue. #46161 (attack204).
  • Functions "multi[Fuzzy]Match(Any|AnyIndex|AllIndices}" now reject regexes which will likely evaluate very slowly in vectorscan. #46167 (Robert Schulze).
  • When insert_null_as_default is enabled and column doesn't have defined default value, the default of column type will be used. Also this PR fixes using default values on nulls in case of LowCardinality columns. #46171 (Kruglov Pavel).
  • Prefer explicitly defined access keys for S3 clients. If use_environment_credentials is set to true, and the user has provided the access key through query or config, they will be used instead of the ones from the environment variable. #46191 (Antonio Andelic).
  • Concurrent merges are scheduled using round-robin by default to ensure fair and starvation-free operation. Previously in heavily overloaded shards, big merges could possibly be starved by smaller merges due to the use of strict priority scheduling. Added background_merges_mutations_scheduling_policy server config option to select scheduling algorithm (round_robin or shortest_task_first). #46247 (Sergei Trifonov).
  • Extend setting input_format_null_as_default for more formats. Fix setting input_format_defaults_for_omitted_fields with Native and TSKV formats. #46284 (Kruglov Pavel).
    • Add an alias "DATE_FORMAT()" for function "formatDateTime()" to improve compatibility with MySQL's SQL dialect, extend functionformatDateTime() with substitutions "a", "b", "c", "h", "i", "k", "l" "r", "s", "W". ### Documentation entry for user-facing changes User-readable short description: DATE_FORMAT is an alias of formatDateTime. Formats a Time according to the given Format string. Format is a constant expression, so you cannot have multiple formats for a single result column. (Provide link to formatDateTime). #46302 (Jake Bamrah).
  • not for changelog - part of #42648. #46306 (Yakov Olkhovskiy).
  • Enable retries for INSERT by default in case of ZooKeeper session loss. We already use it in production. #46308 (Alexey Milovidov).
  • Add ProfileEvents and CurrentMetrics about the callback tasks for parallel replicas (s3Cluster and MergeTree tables). #46313 (Alexey Milovidov).
  • Add support for DELETE and UPDATE for tables using KeeperMap storage engine. #46330 (Antonio Andelic).
  • Update unixodbc to v2.3.11 to mitigate CVE-2011-1145 ### Documentation entry for user-facing changes. #46363 (Mallik Hassan).
    • Apply ALTER TABLE table_name ON CLUSTER cluster MOVE PARTITION|PART partition_expr TO DISK|VOLUME 'disk_name' to all replicas. Because ALTER TABLE t MOVE is not replicated. #46402 (lizhuoyu5).
  • Allow writing RENAME queries with query parameters. Resolves #45778. #46407 (Nikolay Degterinsky).
  • Fix parameterized SELECT queries with REPLACE transformer. Resolves #33002. #46420 (Nikolay Degterinsky).
  • Exclude the internal database used for temporary/external tables from the calculation of asynchronous metric "NumberOfDatabases". This makes the behavior consistent with system table "system.databases". #46435 (Robert Schulze).
  • Added last_exception_time column into distribution_queue table. #46564 (Aleksandr).
  • Support for IN clause in parameterized views Implementation: * In case of parameterized views, the IN clause cannot be evaluated as constant expression during CREATE VIEW, added a check to ignore this step in case of parameterized view. * If parmeters are not in IN clause, we continue to evaluate it as constant expression. #46583 (SmitaRKulkarni).
  • Do not load named collections on server startup (load them on first access instead). #46607 (Kseniia Sumarokova).
  • Add separate access type SHOW_NAMED_COLLECTIONS_SECRETS to allow to see named collections and their keys, but making values hidden. Nevertheless, access type SHOW NAMED COLLECTIONS is still required. #46667 (Kseniia Sumarokova).
  • Hide arguments of custom disk merge tree setting. #46670 (Kseniia Sumarokova).
  • Ask for the password in clickhouse-client interactively in a case when the empty password is wrong. Closes #46702. #46730 (Nikolay Degterinsky).
  • Backward compatibility for T64 codec support for IPv4. #46747 (Yakov Olkhovskiy).
  • Allow to fallback from asynchronous insert to synchronous in case of large amount of data (more than async_insert_max_data_size bytes in single insert). #46753 (Anton Popov).

Bug Fix

  • Fix wiping sensitive info in logs. #45603 (Vitaly Baranov).
  • There is a check in format "time_check() || ptr ? ptr->finished() : data->is_finished()". Operator "||" will be executed before operator "?", but expected that there should be separated time and ptr checks. Also it's unexpected to run "ptr->finished()" in case of nullptr, but with current expression it's possible. #46054 (Alexey Perevyshin).

Build/Testing/Packaging Improvement

  • Allow to randomize merge tree settings in tests. #38983 (Anton Popov).
  • Enable the HDFS support in PowerPC and which helps to fixes the following functional tests 02113_hdfs_assert.sh, 02244_hdfs_cluster.sql and 02368_cancel_write_into_hdfs.sh. #44949 (MeenaRenganathan22).
  • Add systemd.service file for clickhouse-keeper. Fixes #44293. #45568 (Mikhail f. Shiryaev).
  • ClickHouse's fork of poco was moved from "contrib/" to "base/poco/". #46075 (Robert Schulze).
  • Remove excessive license notices from preciseExp10.cpp. #46163 (DimasKovas).
  • Add an option for clickhouse-watchdog to restart the child process. This does not make a lot of use. #46312 (Alexey Milovidov).
  • Get rid of unnecessary build for standalone clickhouse-keeper. #46367 (Mikhail f. Shiryaev).
  • If the environment variable CLICKHOUSE_DOCKER_RESTART_ON_EXIT is set to 1, the Docker container will run clickhouse-server as a child instead of the first process, and restart it when it exited. #46391 (Alexey Milovidov).
  • Some time ago the ccache compression was changed to zst, but gz archives are downloaded by default. It fixes it by prioritizing zst archive. #46490 (Mikhail f. Shiryaev).
  • Raised the minimum Clang version needed to build ClickHouse from 12 to 15. #46710 (Robert Schulze).

Bug Fix (user-visible misbehavior in official stable or prestable release)

  • Flush data exactly by rabbitmq_flush_interval_ms or by rabbitmq_max_block_size in StorageRabbitMQ. Closes #42389. Closes #45160. #44404 (Kseniia Sumarokova).
    • Use PODArray to render in sparkBar function, so we can control the memory usage. Close #44467. #44489 (Duc Canh Le).
  • Fix functions (quantilesExactExclusive, quantilesExactInclusive) return unsorted array element. #45379 (wujunfu).
  • Fix uncaught exception in HTTPHandler when open telemetry is enabled. #45456 (Frank Chen).
  • Don't infer Dates from 8 digit numbers. It could lead to wrong data to be read. #45581 (Kruglov Pavel).
  • Fixes to correctly use odbc_bridge_use_connection_pooling setting. #45591 (Bharat Nallan).
  • when the callback in the cache is called, it is possible that this cache is destructed. To keep it safe, we capture members by value. It's also safe for task schedule because it will be deactivated before storage is destroyed. Resolve #45548. #45601 (Han Fei).
    • Fix data corruption when codecs Delta or DoubleDelta are combined with codec Gorilla. #45615 (Robert Schulze).
  • Correctly check types when using N-gram bloom filter index to avoid invalid reads. #45617 (Antonio Andelic).
  • A couple of seg faults have been reported around c-ares. All of the recent stack traces observed fail on inserting into std::unodered_set<>. I believe I have found the root cause of this, it seems to be unprocessed queries. Prior to this PR, CH calls poll to wait on the file descriptors in the c-ares channel. According to the poll docs, a negative return value means an error has ocurred. Because of this, we would abort the execution and return failure. The problem is that poll will also return a negative value if a system interrupt occurs. A system interrupt does not mean the processing has failed or ended, but we would abort it anyways because we were checking for negative values. Once the execution is aborted, the whole stack is destroyed, which includes the std::unordered_set<std::string> passed to the void * parameter of the c-ares callback. Once c-ares completed the request, the callback would be invoked and would access an invalid memory address causing a segfault. #45629 (Arthur Passos).
  • Fix key description when encountering duplicate primary keys. This can happen in projections. See #45590 for details. #45686 (Amos Bird).
  • Set compression method and level for backup Closes #45690. #45737 (Pradeep Chhetri).
  • Should use select_query_typed.limitByOffset() instead of select_query_typed.limitOffset(). #45817 (刘陶峰).
  • When use experimental analyzer, queries like SELECT number FROM numbers(100) LIMIT 10 OFFSET 10; get wrong results (empty result for this sql). That is caused by an unnecessary offset step added by planner. #45822 (刘陶峰).
  • Backward compatibility - allow implicit narrowing conversion from UInt64 to IPv4 - required for "INSERT ... VALUES ..." expression. #45865 (Yakov Olkhovskiy).
  • Bugfix IPv6 parser for mixed ip4 address with missed first octet (like ::.1.2.3). #45871 (Yakov Olkhovskiy).
  • Add the query_kind column to the system.processes table and the SHOW PROCESSLIST query. Remove duplicate code. It fixes a bug: the global configuration parameter max_concurrent_select_queries was not respected to queries with INTERSECT or EXCEPT chains. #45872 (Alexey Milovidov).
  • Fix crash in a function stochasticLinearRegression. Found by WingFuzz. #45985 (Nikolai Kochetov).
  • Fix crash in SELECT queries with INTERSECT and EXCEPT modifiers that read data from tables with enabled sparse columns (controlled by setting `ratio_of_defaults_for_sparse_serialization). #45987 (Anton Popov).
  • Fix reading of non existing nested columns with multiple level in compact parts. #46045 (Azat Khuzhin).
  • Fix elapsed column in system.processes (10x error). #46047 (Azat Khuzhin).
  • Follow-up fix for Replace domain IP types (IPv4, IPv6) with native https://github.com/ClickHouse/ClickHouse/pull/43221. #46087 (Yakov Olkhovskiy).
  • Fix environment variable substitution in the configuration when a parameter already has a value. This closes #46131. This closes #9547. #46144 (pufit).
  • Fix incorrect predicate push down with grouping sets. Closes #45947. #46151 (flynn).
  • Fix possible pipeline stuck error on fulls_sorting_join with constant keys. #46175 (Vladimir C).
  • Never rewrite tuple functions as literals during formatting to avoid incorrect results. #46232 (Salvatore Mesoraca).
  • Fix possible out of bounds error while reading LowCardinality(Nullable) in Arrow format. #46270 (Kruglov Pavel).
  • Fix SYSTEM UNFREEZE queries failing with the exception CANNOT_PARSE_INPUT_ASSERTION_FAILED. #46325 (Aleksei Filatov).
  • Fix possible crash which can be caused by an integer overflow while deserializing aggregating state of a function that stores HashTable. #46349 (Nikolai Kochetov).
  • Fix possible LOGICAL_ERROR in asynchronous inserts with invalid data sent in format VALUES. #46350 (Anton Popov).
  • Fixed a LOGICAL_ERROR on an attempt to execute ALTER ... MOVE PART ... TO TABLE. This type of query was never actually supported. #46359 (Alexander Tokmakov).
  • Fix s3Cluster schema inference in parallel distributed insert select when parallel_distributed_insert_select is enabled. #46381 (Kruglov Pavel).
  • Fix queries like ALTER TABLE ... UPDATE nested.arr1 = nested.arr2 ..., where arr1 and arr2 are fields of the same Nested column. #46387 (Anton Popov).
  • Scheduler may fail to schedule a task. If it happens, the whole MulityPartUpload should be aborted and UploadHelper must wait for already scheduled tasks. #46451 (Dmitry Novik).
  • Fix PREWHERE for Merge with different default types (fixes some NOT_FOUND_COLUMN_IN_BLOCK when the default type for the column differs, also allow PREWHERE when the type of column is the same across tables, and prohibit it, only if it differs). #46454 (Azat Khuzhin).
  • Fix a crash that could happen when constant values are used in ORDER BY. Fixes #46466. #46493 (Nikolai Kochetov).
  • Do not throw exception if disk setting was specified on query level, but storage_policy was specified in config merge tree settings section. disk will override setting from config. #46533 (Kseniia Sumarokova).
  • Fix an invalid processing of constant LowCardinality argument in function arrayMap. This bug could lead to a segfault in release, and logical error Bad cast in debug build. #46569 (Alexey Milovidov).
  • fixes #46557. #46611 (Alexander Gololobov).
  • Fix endless restarts of clickhouse-server systemd unit if server cannot start within 1m30sec (Disable timeout logic for starting clickhouse-server from systemd service). #46613 (Azat Khuzhin).
  • Allocated during asynchronous inserts memory buffers were deallocated in the global context and MemoryTracker counters for corresponding user and query were not updated correctly. That led to false positive OOM exceptions. #46622 (Dmitry Novik).
  • Fix totals and extremes with constants in clickhouse-local. Closes #43831. #46669 (Kruglov Pavel).
  • Handle input_format_null_as_default for nested types. #46725 (Azat Khuzhin).

Bug-fix

Build Improvement

NO CL ENTRY

NOT FOR CHANGELOG / INSIGNIFICANT

Testing Improvement

  • Fixed functional test 00304_http_external_data for s390x. #45807 (Harry Lee).