thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-11-10 01:25:21 +00:00
Code Issues Packages Projects Releases Wiki Activity
f2e1d6d402
ClickHouse/docs/changelogs/v21.10.1.8013-prestable.md

46 KiB
Raw Blame History

sidebar_position sidebar_label
1 2022

2022 Changelog

ClickHouse release v21.10.1.8013-prestable FIXME as compared to v21.9.1.7770-prestable

Backward Incompatible Change

  • Fix the issue that in case of some sophisticated query with column aliases identical to the names of expressions, bad cast may happen. This fixes #25447. This fixes #26914. This fix may introduce backward incompatibility: if there are different expressions with identical names, exception will be thrown. It may break some rare cases when enable_optimize_predicate_expression is set. #26639 (Alexey Milovidov).
  • Do not output trailing zeros in text representation of Decimal types. Example: 1.23 will be printed instead of 1.230000 for decimal with scale 6. This closes #15794. It may introduce slight incompatibility if your applications somehow relied on the trailing zeros. Serialization in output formats can be controlled with the setting output_format_decimal_trailing_zeros. Implementation of toString and casting to String is changed unconditionally. #27680 (Alexey Milovidov).
  • Now MergeTreeSettings replicated_max_parallel_sends, replicated_max_parallel_sends_for_table, replicated_max_parallel_fetches, replicated_max_parallel_fetches_for_table do nothing. They never worked well and were replaced with max_replicated_fetches_network_bandwidth, max_replicated_sends_network_bandwidth and background_fetches_pool_size. #28404 (alesapin).

New Feature

  • Generate a unique server uuid when server starts. #20089 (Bharat Nallan).
  • Added new commands BACKUP and RESTORE. #21945 (Vitaly Baranov).
  • Partitioned write into s3 table function. #23051 (Vladimir Chebotarev).
  • Implementation of short circuit function evaluation, closes #12587. Add settings short_circuit_function_evaluation to configure short circuit function evaluation. #23367 (Kruglov Pavel).
  • Add feature for creating user-defined functions. #23978 (Realist007).
  • Add support for INTERSECT, EXCEPT, ANY, ALL operators. #24757 (Kirill Ershov).
  • IDisk interface to store data on web server of static files. Closes #23982. #25251 (Kseniia Sumarokova).
  • Introduce lz4 compression for import / export. #25310 (Bharat Nallan).
  • Support the case when the data is enclosed in array in JSONAsString input format. Closes #25517. #25633 (Kruglov Pavel).
  • Add new column last_queue_update_exception to system.replicas table. #26843 (nvartolomei).
  • ALTER TABLE ... MATERIALIZE COLUMN. #27038 (Vladimir Chebotarev).
    • Add replicated storage of user, roles, row policies, quotas and settings profiles through ZooKeeper (experimental). #27426 (Kevin Michel).
  • Allow positional arguments under setting enable_positional_arguments. Closes #2592. #27530 (Kseniia Sumarokova).
  • Added ComplexKeyRangeHashed dictionary. Closes #22029. #27629 (Maksim Kita).
  • add conversion functions between snowflake id and dateTime(dateTime64) Close #27058. #27704 (jasine).
  • Add feature for creating user-defined functions as lambda expressions. Syntax CREATE FUNCTION {function_name} as ({parameters}) -> {function core}. Example CREATE FUNCTION plus_one as (a) -> a + 1. Authors @Realist007. #27796 (Maksim Kita).
  • Add getServerPort function to allow getting server port. When the port is not used by the server, throw an exception. #27900 (Amos Bird).
  • Accept user settings related to file formats in SETTINGS clause in CREATE query. This closes #27580. #28037 (Nikita Mikhaylov).
  • Add a system table of table_views, convenient to query the dependency relationship between tables and views. #28082 (zhongyuankai).
  • Added executable storage engine and table function. Authors @ruct. #28102 (Maksim Kita).
  • Added ExecutablePool storage. #28518 (Maksim Kita).

Performance Improvement

  • Introducing two checks in sequenceMatch and sequenceCount that allow for early exit when some deterministic part of the sequence pattern is missing from the events list. This change unlocks many queries that would previously fail due to reaching operations cap, and generally speeds up the pipeline. #27729 (Jakub Kuklis).
  • Make hasAll filter condition leverage bloom filter data-skipping indexes. #27984 (Braulio Valdivielso Martínez).
  • Speed up sumIf and countIf aggregation functions. #28272 (Raúl Marín).
  • Enhance primary key analysis with always monotonic information of binary functions, notably non-zero constant division. #28302 (Amos Bird).

Improvement

  • Create virtual projection for min_max indices. Now, when allow_experimental_projection_optimization is enabled, queries will use minmax index instead of reading a part when possible. #26286 (Amos Bird).
  • improve Materialize TTL by recalculating ttl.txt only without actual ttl action. #27019 (lthaooo).
  • Improved the existence condition judgment and empty string node judgment when clickhouse-keeper creates znode. #27125 (小路).
  • Don't silently ignore errors and don't count delays in ReadBufferFromS3. #27484 (Vladimir Chebotarev).
  • Add log_queries_probability setting that allows user to write to query_log only a sample of queries. Closes #16609. #27527 (Nikolay Degterinsky).
  • Disable arrayJoin on partition expressions. #27648 (Raúl Marín).
  • Enables query parameters to be passed in the body of http requests. #27706 (Hermano Lustosa).
  • Remove duplicate index analysis and avoid possible invalid limit checks during projection analysis. #27742 (Amos Bird).
  • Add aggregate function quantileBFloat16Weighted similarly to other quantile...Weighted functions. This closes #27745. #27758 (Ivan Novitskiy).
  • Now ALTER MODIFY COLUM DataType to Nullable(DataType) doesn't require mutation. #27787 (victorgao).
  • Allow symlinks for library dictionaty path. #27815 (Kseniia Sumarokova).
  • Add function SHA512. #27830 (zhanglistar).
  • Use Multipart copy upload for large S3 objects. #27858 (ianton-ru).
  • Improve remote query cancelation (in case of remote server abnormaly terminated). #27881 (Azat Khuzhin).
  • Enable tcp_keep_alive_timeout by default. #27882 (Azat Khuzhin).
  • Fix incorrect assertion during writing to StorageKafka. #27885 (Azat Khuzhin).
  • Support lambda argument for APPLY column transformer which allows applying functions with more than one argument. This is for #27877. #27901 (Amos Bird).
  • Add interactive documentation in clickhouse-client about how to reset the password. This is useful in scenario when user has installed ClickHouse, set up the password and instantly forget it. See #27750. #27903 (Alexey Milovidov).
  • Allow to create dictionaries with empty attributes list. #27905 (Maksim Kita).
  • Added replication_wait_for_inactive_replica_timeout setting. It allows to specify how long to wait for inactive replicas to execute ALTER/OPTIMZE/TRUNCATE query (default is 120 seconds). If replication_alter_partitions_sync is 2 and some replicas are not active for more than replication_wait_for_inactive_replica_timeout seconds, then UNFINISHED will be thrown. #27931 (Alexander Tokmakov).
  • Add a setting empty_result_for_aggregation_by_constant_keys_on_empty_set to control the behavior of grouping by constant keys on empty set. This is to bring back the old baviour of #6842. #27932 (Amos Bird).
  • Lower restrictions for Enum data type to allow attaching compatible data. Closes #26672. #28028 (Dmitry Novik).
  • Support ON CONFLICT clause when inserting into PostgreSQL table engine or table function. Closes #27727. #28081 (Kseniia Sumarokova).
  • Support implicit conversions between index in operator [] and key of type Map (e.g. different Int types, String and FixedString). #28096 (Anton Popov).
  • Enable optimize_distributed_group_by_sharding_key by default. #28105 (Azat Khuzhin).
  • Fix zookeeper_log.address (before the first patch in this PR the address was always ::) and reduce number of calls getpeername(2) for this column (since each time entry for zookeeper_log is added getpeername() is called, cache this address in the zookeeper client to avoid this). #28212 (Azat Khuzhin).
  • Fix removing of parts in a Temporary state which can lead to an unexpected exception (Part %name% doesn't exist). Fixes #23661. #28221 (Azat Khuzhin).
  • Added libhdfs3_conf in server config instead of export env LIBHDFS3_CONF in clickhouse-server.service. #28268 (Zhichang Yu).
  • Use real tmp file instead of predefined "rows_sources" for vertical merges. This avoids generating garbage directories in tmp disks. #28299 (Amos Bird).
  • Speed up data parts loading by delaying table startup process. #28313 (Amos Bird).
  • Allow ssl connection for RabbitMQ engine. #28365 (Kseniia Sumarokova).
  • Fix removing of parts in a Temporary state (follow up for #28221). #28366 (Azat Khuzhin).
  • Do not allow creating StorageMaterializedPostgreSQL with bad arguments. Closes #28423. #28430 (Kseniia Sumarokova).
  • Introduce connection_wait_timeout (default to 5 seconds, 0 - do not wait) setting for MySQL engine. #28474 (Azat Khuzhin).
  • Fix strange sessions expiration logic in Keeper. Probably it should help in CI: https://clickhouse-test-reports.s3.yandex.net/0/6bd9b82141c98dcd7796fd9d08326831095ba519/stress_test_(debug).html#fail1. #28519 (alesapin).
  • To be added. Closes #28529. #28614 (Kseniia Sumarokova).

Bug Fix

Build/Testing/Packaging Improvement

Bug Fix (user-visible misbehaviour in official stable or prestable release)

  • Fix handling null value with type of Nullable(String) in function JSONExtract. This fixes #27929 and #27930 . This was introduced in https://github.com/ClickHouse/ClickHouse/pull/25452 . #27939 (Amos Bird).
  • Fix extremely rare segfaults on shutdown due to incorrect order of context/config reloader shutdown. #28088 (nvartolomei).
  • Fixed possible excessive number of conditions moved from WHERE to PREWHERE (optimization controlled by settings optimize_move_to_prewhere). #28139 (lthaooo).
  • Fix bug in clickhouse-keeper which can lead to endless logs when rotate_logs_interval decreased. #28152 (alesapin).
  • Multiple small fixes for projections. See detailed description in pr. #28178 (Amos Bird).
  • Fix incorrect behavior in clickhouse-keeper when list watches (getChildren) triggered with set requests for children. #28190 (alesapin).
  • Fix a rare bug in clickhouse-keeper when the client can receive a watch response before request-response. #28197 (alesapin).
  • Fix possible read of uninitialized memory for queries with Nullable(LowCardinality) type and extremes. Fixes #28165. #28205 (Nikolai Kochetov).
  • Fix reading of custom TLD w/o new line at EOF. #28213 (Azat Khuzhin).
  • Fix inconsistent result in queries with ORDER BY and Merge tables with enabled setting optimize_read_in_order. #28266 (Anton Popov).
  • Fix intersecting parts due to new part had been replaced with an empty part. #28310 (Azat Khuzhin).
  • Fix NOT-IN index optimization when not all key columns are used. This fixes #28120. #28315 (Amos Bird).
  • Fix non joined rows from nullable column. Close #27691. #28349 (Vladimir C).
  • Fix rare case when changes of clickhouse-keeper settings may lead to lost logs and server hung. #28360 (alesapin).
  • Fix lack of quotes for table names in MaterializedPostgreSQL engine. Closes #28316. #28433 (Kseniia Sumarokova).
  • Fixed possible ZooKeeper watches leak on background processing of distributed DDL queue. Closes #26036. #28446 (Alexander Tokmakov).
  • Fix bug which can lead to error Existing table metadata in ZooKeeper differs in sorting key expression. after alter of ReplicatedVersionedCollapsingMergeTree. Fixes #28515. #28528 (alesapin).
  • Fix There is no subcolumn error, while select from tables, which have Nested columns and scalar columns with dot in name and the same prefix as Nested (e.g. n.id UInt32, n.arr1 Array(UInt64), n.arr2 Array(UInt64)). #28531 (Anton Popov).
  • Fix UUID overlap in DROP TABLE for internal DDL from MaterializeMySQL. #28533 (Azat Khuzhin).
  • Fix endless loop for truncated bzip2 archive. #28543 (Azat Khuzhin).

NO CL ENTRY

NOT FOR CHANGELOG / INSIGNIFICANT