ClickHouse/docs/changelogs/v20.9.2.20-stable.md

ClickHouse release v20.9.2.20-stable FIXME as compared to v20.8.1.4513-prestable

New Feature

• Add table function view which turns an subquery into a table object. This helps passing queries around. For instance, it can be used in remote/cluster table functions. #12567 (Amos Bird).
• Now we can write select * apply(length) apply(max) from wide_string_table to find out the maxium length of all string columns. And the follow two variants are provided too:. #14233 (Amos Bird).
• Added an aggregate function RankCorrelationSpearman which simply computes a rank correlation coefficient. Continuation of #11769. #14411 (Nikita Mikhaylov).
• Added database generation by query util. Continuation of #10973. #14442 (Nikita Mikhaylov).

Performance Improvement

• Optimize queries with LIMIT/LIMIT BY/ORDER BY for distributed with GROUP BY sharding_key (under optimize_skip_unused_shards and optimize_distributed_group_by_sharding_key). #10373 (Azat Khuzhin).

Improvement

• Improvements in StorageRabbitMQ: Added connection and channels failure handling, proper commits, insert failures handling, better exchanges, queue durability and queue resume opportunity, new queue settings. Fixed tests. #12761 (Kseniia Sumarokova).
• Added Redis requirepass authorization. #13688 (Ivan Torgashov).
• Add precision argument for DateTime type. #13761 (Winter Zhang).
• Improve the Kafka engine performance by providing independent thread for each consumer. Separate thread pool for streaming engines (like Kafka). #13939 (fastio).
• Add default compression codec for parts in system.part_log with name default_compression_codec. #14116 (alesapin).
• Replace wide integers from boost multiprecision with implementation from https://github.com/cerevra/int. #14229 (Artem Zuikov).
• Implicitly convert primary key to not null in MaterializeMySQL(Same as MySQL). Fixes #14114. #14397 (Winter Zhang).
• Added new setting system_events_show_zero_values as proposed in #11384. #14404 (Dmitry Rubashkin).
• Now obfuscator supports UUID type as proposed in #13163. #14409 (Dmitry Rubashkin).
• Creating sets for multiple JOIN and IN in parallel. It may slightly improve performance for queries with several different IN subquery expressions. #14412 (Nikolai Kochetov).
• Now TTLs will be applied during merge if they were not previously materialized. #14438 (alesapin).
• MySQL handler returns OK for queries like SET @@var = value. Such statement is ignored. It is needed because some MySQL drivers send SET @@ query for setup after handshake #9336#issuecomment-686222422 . #14469 (BohuTANG).
• Disallow empty time_zone argument in toStartOf* type of functions. #14509 (Bharat Nallan).
• ... #14523 (BohuTANG).
• Backported in #14873: Allow using multi-volume storage configuration in storage Distributed. #14839 (Pavel Kovalenko).

Bug Fix

• Fix arrayJoin() capturing in lambda (LOGICAL_ERROR). #13792 (Azat Khuzhin).
• Fix GRANT ALL statement when executed on a non-global level. #13987 (Vitaly Baranov).
• Disallows CODEC on ALIAS column type. Fixes #13911. #14263 (Bharat Nallan).
• Better check for tuple size in SSD cache complex key external dictionaries. This fixes #13981. #14313 (Alexey Milovidov).
• Fix QueryPlan lifetime (for EXPLAIN PIPELINE graph=1) for queries with nested interpreter. #14315 (Azat Khuzhin).
• Fix exception during ALTER LIVE VIEW query with REFRESH command. #14320 (Bharat Nallan).
• Fix crash during ALTER query for table which was created AS table_function. Fixes #14212. #14326 (alesapin).
• Stop query execution if exception happened in PipelineExecutor itself. This could prevent rare possible query hung. #14334 (Nikolai Kochetov).
• Stop query execution if exception happened in PipelineExecutor itself. This could prevent rare possible query hung. Continuation of #14334. #14402 (Nikolai Kochetov).
• Fix bug which leads to wrong merges assignment if table has partitions with a single part. #14444 (alesapin).
• Proxy restart/start/stop/reload of SysVinit to systemd (if it is used). #14460 (Azat Khuzhin).
• Check for array size overflow in topK aggregate function. Without this check the user may send a query with carefully crafter parameters that will lead to server crash. This closes #14452. #14467 (Alexey Milovidov).
• Backported in #15020: Fixed the incorrect sorting order of Nullable column. This fixes #14344. #14495 (Nikita Mikhaylov).
• Backported in #14599: Fix rare segfaults in functions with combinator -Resample, which could appear in result of overflow with very large parameters. #14562 (Anton Popov).
• Backported in #14729: Cleanup data directory after Zookeeper exceptions during CreateQuery for StorageReplicatedMergeTree Engine. #14563 (Bharat Nallan).
• Backported in #14654: Added the checker as neither calling lc->isNullable() nor calling ls->getDictionaryPtr()->isNullable() would return the correct result. #14591 (Mike Kot).
• Backported in #14663: Fix wrong Decimal multiplication result caused wrong decimal scale of result column. #14603 (Artem Zuikov).
• Backported in #14662: Stuff the query into ASTFunction's argument list so that we don't break the presumptions of some AST visitors. This fixes #14608. #14611 (Amos Bird).
• Backported in #14792: Fix bug when ALTER UPDATE mutation with Nullable column in assignment expression and constant value (like UPDATE x = 42) leads to incorrect value in column or segfault. Fixes #13634, #14045. #14646 (alesapin).
• Backported in #14721: Fixed missed default database name in metadata of materialized view when executing ALTER ... MODIFY QUERY. #14664 (Alexander Tokmakov).
• Backported in #14770: Replace column transformer should replace identifiers with cloned ASTs. This fixes #14695 . #14734 (Amos Bird).
• Backported in #14825: Fix wrong monotonicity detection for shrunk Int -> Int cast of signed types. It might lead to incorrect query result. This bug is unveiled in #14513. #14783 (Amos Bird).
• Backported in #14912: Fix SIGSEGV for an attempt to INSERT into StorageFile(fd). #14887 (Azat Khuzhin).
• Backported in #14944: Fix the issue when some invocations of extractAllGroups function may trigger "Memory limit exceeded" error. This fixes #13383. #14889 (Alexey Milovidov).
• Backported in #14956: Fixed .metadata.tmp File exists error when using MaterializeMySQL database engine. #14898 (Winter Zhang).
• Backported in #14989: Publish CPU frequencies per logical core in system.asynchronous_metrics. This fixes #14923. #14924 (Alexander Kuzmenkov).
• Backported in #15055: Now settings number_of_free_entries_in_pool_to_execute_mutation and number_of_free_entries_in_pool_to_lower_max_size_of_merge can be equal to background_pool_size. #14975 (alesapin).
• Backported in #15079: Fix crash in RIGHT or FULL JOIN with join_algorith='auto' when memory limit exceeded and we should change HashJoin with MergeJoin. #15002 (Artem Zuikov).
• Backported in #15059: If function bar was called with specifically crafted arguments, buffer overflow was possible. This closes #13926. #15028 (Alexey Milovidov).
• Backported in #15052: We already use padded comparison between String and FixedString (https://github.com/ClickHouse/ClickHouse/blob/master/src/Functions/FunctionsComparison.h#L333). This PR applies the same logic to field comparison which corrects the usage of FixedString as primary keys. This fixes #14908. #15033 (Amos Bird).

Build/Testing/Packaging Improvement

• Integration tests use default base config. All config changes are explicit with main_configs, user_configs and dictionaries parameters for instance. #13647 (Ilya Yatsishin).
• ... #14368 (BohuTANG).
• Fix the logic in backport script. In previous versions it was triggered for any labels of 100% red color. It was strange. #14433 (Alexey Milovidov).
• Fix missed #include <atomic>. #14440 (Matwey V. Kornilov).
• Prepare for build with clang 11. #14455 (Alexey Milovidov).
• Lower binary size in debug build by removing debug info from Functions. This is needed only for one internal project in Yandex who is using very old linker. #14549 (Alexey Milovidov).

Other

• Changelog for 20.7 #13499. #14420 (Alexander Kazakov).

NO CL ENTRY

• NO CL ENTRY: 'Revert "Less number of threads in builder"'. #14421 (Alexey Milovidov).