mirror of
https://github.com/ClickHouse/ClickHouse.git
synced 2024-11-21 23:21:59 +00:00
26 KiB
26 KiB
sidebar_position | sidebar_label |
---|---|
1 | 2022 |
2022 Changelog
ClickHouse release v21.8.1.7409-prestable FIXME as compared to v21.7.1.7283-prestable
Backward Incompatible Change
-
- Backward Incompatible Change:. #23934 (hexiaoting).
New Feature
- Add an ability to reset custom setting to default and remove it from table's metadata. This will allow to rollback the change without knowing the system/config's default. Closes #14449. #17769 (xjewer).
- Add MaterializedPostgreSQL table engine and database engine. Database engine allows to replicate a whole database or any subset of database tables. #20470 (Kseniia Sumarokova).
- Adding support for a part of SQLJSON standard. #24148 (l1tsolaiki).
- Collect common system metrics (in
system.asynchronous_metrics
andsystem.asynchronous_metric_log
) about CPU usage, disk usage, memory usage, IO, network, files, load average, CPU frequencies, thermal sensors, EDAC counters, system uptime; also added metrics about the scheduling jitter and the time spent collecting the metrics. It works likeatop
in ClickHouse and allows to get monitoring data even if you have no additional tools installed. This closes #9430. #24416 (Yegor Levankov). - Add support
DISTINCT ON (columns)
expression, close #25404. #25589 (Zijie Lu). -
- Support Map type in
mapAdd
andmapSubtract
functions * Support (U)Int128, U(Int256) types inmapAdd
andmapSubtract
functions. #25596 (Ildus Kurbangaliev).
- Support Map type in
- Add bin/unbin functions support. #25609 (zhaoyu).
- Introduce
system.data_skipping_indices
table containing information about existing data skipping indices. Closes #7659. #25693 (Dmitry Novik). - in addition to https://github.com/ClickHouse/ClickHouse/pull/12073 add the FIRST keyword to the ADD INDEX command to be able to add index in the beginning of the indices list. #25904 (xjewer).
- Render pipelines as graphs in Web UI if
EXPLAIN PIPELINE graph = 1
query is given. #26067 (Alexey Milovidov). - Add new functions
leftPad()
,rightPad()
,leftPadUTF8()
,rightPadUTF8()
. #26075 (Vitaly Baranov).
Performance Improvement
- Added option to compile aggregate functions if
compile_aggregate_expressions
settings is on. #24789 (Maksim Kita).
Improvement
Database
argument forStorageMerge
support regular expression. This closes #776. #25064 (flynn).- Allow extract non-string element as string using JSONExtract. This is for #25414. #25452 (Amos Bird).
- Support for dynamic reloading of config to change number of threads in pool for background jobs execution (merges, mutations, fetches). #25548 (Nikita Mikhaylov).
- Support TRUNCATE TABLE for StorageS3 and StorageHDFS. Closes #25530. #25550 (Kseniia Sumarokova).
- Make
NetworkReceiveElapsedMicroseconds
metric to correctly include the time spent waiting for data from the client to INSERT. This closes #9958. #25602 (Alexey Milovidov). - Fix possible logical race condition between
ALTER TABLE ... DETACH
and background merges. #25605 (Azat Khuzhin). - Support materialized and aliased columns in joins, close #13274. #25634 (Vladimir C).
- MaterializeMySQL now supports
ENUM
data type. #25676 (Storozhuk Kostiantyn). - Cancel already running merges in partition on
DROP PARTITION
andTRUNCATE
forReplicatedMergeTree
. Resolves #17151. #25684 (Alexander Tokmakov). - Use
Map
data type for key-value dictionaries in system logs tables (system.query_log
,system.query_thread_log
,system.processes
,system.opentelemetry_span_log
). Virtual columns are created to support old queries. Closes #18698. Authors @hexiaoting, @sundy-li. #25773 (Maksim Kita). - Fix inconsistent behaviour of GROUP BY constant on empty set. Closes #6842. #25786 (Kseniia Sumarokova).
- MySQL Engine now supports the exchange of column comments between MySQL and ClickHouse. #25795 (Storozhuk Kostiantyn).
- Fix "No available columns" for Merge() storage. #25801 (Azat Khuzhin).
- Allow to start clickhouse-client with unreadable working directory. #25817 (ianton-ru).
- Better handling of lost parts for ReplicatedMergeTree tables. Fixes rare inconsistencies in ReplicationQueue. Nothing should be visible to the user. Fixes #10368. #25820 (alesapin).
- Fix an extremely rare bug which can lead to intersecting parts after
DROP PART
or background deletion of an empty part. #25884 (alesapin). - Convert history file from readline format to replxx format. #25888 (Azat Khuzhin).
- Support LowCardinality, Decimal and UUID for JSON extract. Closes #24606. #25900 (Kseniia Sumarokova).
- Add support for queries with a column named
"null"
(it must be specified in backticks or double quotes) and ON CLUSTER. This closes #24035. #25907 (Alexey Milovidov). - Correctly throw exception on attempt to parse invalid Date. This closes #6481. #25909 (Alexey Milovidov).
- Allow parameters for parametric aggregate functions to be arbitrary constant expressions (e.g.
1 + 2
), not just literals. This also allows to use query parameters (in parametrized queries like{param:UInt8}
) for parameters of parametric aggregate functions. This closes #11607. #25910 (Alexey Milovidov). - Allow
quantiles*
functions to work withaggregate_functions_null_for_empty
. This closes #25892. #25919 (Alexey Milovidov). - Make
sudo service clickhouse-server start
to work on systems withsystemd
like Centos 8. This closes #14298. This closes #17799. #25921 (Alexey Milovidov). - Add support for unicode (e.g. Chinese, Cyrillic) components in
Nested
data types. This closes #25594. #25923 (Alexey Milovidov). - Allow complex quoted identifiers of JOINed tables. This closes #17861. #25924 (Alexey Milovidov).
- Added setting
optimize_move_to_prewhere_if_final
. If query hasFINAL
, the optimizationmove_to_prewhere
will be enabled only if bothoptimize_move_to_prewhere
andoptimize_move_to_prewhere_if_final
are enabled. Closes #8684. #25940 (Kseniia Sumarokova). - More instrumentation for network interaction: add counters for recv/send bytes; add gauges for recvs/sends. Added missing documentation. This closes #5897. #25962 (Alexey Milovidov).
- Web UI: if value looks like an URL, automatically generate a link. #25965 (Alexey Milovidov).
- Fix error with query
SET SQL_SELECT_LIMIT
in mysql protocol. Closes #17115. #25972 (Kseniia Sumarokova). - Add support for argument of
UUID
type forempty
andnotEmpty
functions.UUID
is empty if it is all zeros (nil UUID). This closes #3446. #25974 (zhaoyu). - Add support for argument of AggregateFunction type for bin and hex functions. #26094 (zhaoyu).
- For dictionary with complex key if complex key contains only one attribute allow to not wrap key expression in tuple for functions
dictGet
,dictHas
. #26130 (Maksim Kita).
Bug Fix
CAST
fromDate
toDateTime
(orDateTime64
) was not using the timezone of theDateTime
type. It can also affect the comparison betweenDate
andDateTime
. Inference of the common type forDate
andDateTime
also was not using the corresponding timezone. It affected the results of functionif
and array construction. Closes #24128. #24129 (Maksim Kita).- Fix assertion in PREWHERE with non-uint8 type, close #19589. #25484 (Vladimir C).
- Fix
ALTER MODIFY COLUMN
of columns, which participates in TTL expressions. #25554 (Anton Popov). - Fix slow dict join in some cases, close #24209. #25618 (Vladimir C).
- Allow StorageMerge to access tables with aliases. Closes #6051. #25694 (Kseniia Sumarokova).
- Fix bug in
TTL
withGROUP BY
expression which refuses to executeTTL
after first execution in part. #25743 (alesapin). - Fix rare bug with
DROP PART
query forReplicatedMergeTree
tables which can lead to error messageUnexpected merged part intersecting drop range
. #25783 (alesapin). - Fix ARM exception handling with non default page size. Fixes #25512. Fixes #25044. Fixes #24901. Fixes #23183. Fixes #20221. Fixes #19703. Fixes #19028. Fixes #18391. Fixes #18121. Fixes #17994. Fixes #12483. #25854 (Maksim Kita).
- Fix extremely long backoff for background tasks when the background pool is full. Fixes #25836. #25893 (alesapin).
- Fixed
scram-sha-256
authentication for PostgreSQL engines. Closes #24516. #25906 (Kseniia Sumarokova). - Fix crash on call dictGet() with bad arguments. #25913 (Vitaly Baranov).
- Fix possible deadlock during query profiler stack unwinding. Fixes #25968. #25970 (Maksim Kita).
- Fix formatting of type
Map
with integer keys toJSON
. #25982 (Anton Popov). - Fix wrong thread estimation for right subquery join in some cases. Close #24075. #26052 (Vladimir C).
- Fix rare server crash because of
abort
in ZooKeeper client. Fixes #25813. #26079 (alesapin). - Fix throwing exception when iterate over non existing remote directory. #26087 (ianton-ru).
- Fix possible crash in
pointInPolygon
if the settingvalidate_polygons
is turned off. #26113 (Alexey Milovidov). - Fix
joinGet
with LowCarinality columns, close #25993. #26118 (Vladimir C).
Build/Testing/Packaging Improvement
- Add new tests for checking access rights for columns used in filters (WHERE / PREWHERE / row policy) of the
SELECT
statement after changes in https://github.com/ClickHouse/ClickHouse/pull/24405. #25619 (Vitaly Baranov). - Enabling all TestFlows modules and fixing some tests. #26011 (vzakaznikov).
- Disabling TestFlows LDAP module due to test fails. #26065 (vzakaznikov).
Other
- Add
clickhouse-keeper-converter
tool which allows converting zookeeper logs and snapshots intoclickhouse-keeper
snapshot format. #25428 (alesapin).
NO CL ENTRY
- NO CL ENTRY: '[ImgBot] Optimize images'. #26054 (imgbot[bot]).
NOT FOR CHANGELOG / INSIGNIFICANT
- Fix hang and incorrect exit code returned from clickhouse-test #25537 (nvartolomei).
- Remove PrewhereDAGInfo. #25719 (Nikolai Kochetov).
- Fix 01641_memory_tracking_insert_optimize #25731 (Azat Khuzhin).
- Separate log files for separate runs in stress test #25741 (Alexey Milovidov).
- Fix slow performance test #25742 (Alexey Milovidov).
- DatabaseAtomic EXCHANGE DICTIONARIES fix test #25753 (Maksim Kita).
- Try fix flaky rabbitmq test #25756 (Kseniia Sumarokova).
- Add a test for #13993 #25758 (Alexey Milovidov).
- Set follow-fork-mode child for gdb in stress/fasttest/fuzzer #25769 (Azat Khuzhin).
- Ignore TOO_DEEP_RECURSION server exception during fuzzing #25770 (Azat Khuzhin).
- Add comments for VERSION_REVISION vs DBMS_TCP_PROTOCOL_VERSION #25771 (Azat Khuzhin).
- Fix flaky test and wrong message #25772 (alesapin).
- MaterializeMySQL: Improved column comments support #25781 (Storozhuk Kostiantyn).
- Fix ANTRL merge_prewhere_table test #25782 (Maksim Kita).
- Remove dead code from VirtualColumnUtils.cpp #25787 (Alexander Tokmakov).
- Fix some typos in Storage classes #25796 (Raúl Marín).
- Fix DateLUT on Darwin #25803 (Alexey Milovidov).
- Better test_version_update_after_mutation #25810 (alesapin).
- Collect stderr.log and stdout.log in all integration tests by default. #25816 (Vitaly Baranov).
- METR-41529 #25819 (egatov).
- tests/integration: use iptables --wait #25823 (Azat Khuzhin).
- Add a test for #25611 #25831 (Alexey Milovidov).
- Fix odbc test #25834 (Kseniia Sumarokova).
- ExpressionCache destruction fix #25835 (Maksim Kita).
- Fix performance tests after converting ProfileEvents to Map type #25837 (Azat Khuzhin).
- Correct messages in integration tests. #25861 (Vitaly Baranov).
- Fix typo in the test 00900_orc_arrow_parquet_maps #25874 (Kruglov Pavel).
- Improve logging in integration tests. #25899 (Vitaly Baranov).
- kerberized HDFS test fix if run in parallel #25908 (Ilya Golshtein).
- fix special build on clang 11 #25912 (flynn).
- Remove obsolete code from init script #25920 (Alexey Milovidov).
- FunctionInitializeAggregation build fix #25922 (Maksim Kita).
- Fix clang tidy build check #25939 (Kseniia Sumarokova).
- Build subquery in replaceJoinedTable without parsing #25941 (Vladimir C).
- Remove experimental ANTLR parser #25942 (Alexey Milovidov).
- Follow-up for #20470 #25975 (Kseniia Sumarokova).
- Correct test #25984 (Kseniia Sumarokova).
- Fix parallel execution of integration tests #25986 (Vitaly Baranov).
- Compile aggregate functions perf tests fix #25989 (Maksim Kita).
- Fix alter of settings in
MergeTree
tables #25995 (Anton Popov). - Fix arcadia #26002 (Kseniia Sumarokova).
- Remove old code #26014 (Alexey Milovidov).
- FunctionsLogical const result for non const arguments fix #26018 (Maksim Kita).
- FunctionSQLJSON ContextPtr build fix #26022 (Maksim Kita).
- Replace print() with logging.debug() in integration tests. #26023 (Vitaly Baranov).
- Try to fix some flaky tests #26032 (Anton Popov).
- Fix for ZK watch metric drift in rare conditions #26034 (nvartolomei).
- AsynchronousMetrics: Don't assume temperature is always positive #26045 (Raúl Marín).
- Proper fix of serialization of type Map to JSON #26048 (Anton Popov).
- ClickHouse dictionary source secure setting added documentation #26055 (Maksim Kita).
- Add changelog for 21.7 #26057 (Maksim Kita).
- Fix 01791_dist_INSERT_block_structure_mismatch flakiness #26058 (Azat Khuzhin).
- Fix logical error with signed and unsigned offset in WindowFrame::checkValid #26072 (Vladimir C).
- Remove unused code #26077 (Anton Popov).
- Disabling annoying copier tests #26099 (Nikita Mikhaylov).
- Make graph pipeline rendering compatible with Dagre.JS #26114 (Alexey Milovidov).