mirror of
https://github.com/ClickHouse/ClickHouse.git
synced 2024-12-05 22:12:21 +00:00
28 KiB
28 KiB
ClickHouse release v20.12.2.1-stable FIXME as compared to v20.11.1.5109-prestable
Backward Incompatible Change
- Accept user settings related to file formats (e.g.
format_csv_delimiter
) in theSETTINGS
clause when creating a table that usesFile
engine, and use these settings in allINSERT
s andSELECT
s. The file format settings changed in the current user session, or in theSETTINGS
clause of a DML query itself, no longer affect the query. #16591 (Alexander Kuzmenkov). - Enable
use_compact_format_in_distributed_parts_names
by default (see the documentation for the reference). #16728 (Azat Khuzhin).
New Feature
- Added new ALTER UPDATE/DELETE IN PARTITION syntax. #13403 (Vladimir Chebotarev).
- Add StorageEmbeddedRocksdb Engine. #15073 (sundyli).
- Introduce the query
ALTER TABLE ... DROP|DETACH PART 'part_name'
. #15511 (nvartolomei). - Make it possible to change the path to history file in
clickhouse-client
using the--history_file
parameter. #15960 (Maksim Kita). - Updated DateTime, DateTime64 formatting to accept string Date literal format. #16040 (Maksim Kita).
- Add setting
aggregate_functions_null_for_empty
, this option will rewrite all aggregate functions in a query, adding -OrNull suffix to them. fix 10273. #16123 (flynn). - Add COLLATE support for Nullable, LowCardinality, Array and Tuple, where nested type is String. Also refactor the code associated with collations in ColumnString.cpp. #16273 (Kruglov Pavel).
- Possibility to distribute the merges between different replicas. Introduces the
execute_merges_on_single_replica_time_threshold
mergetree setting. #16424 (filimonov). - add
*.xz
compression/decompression support.It enables using*.xz
infile()
function.This closes #8828. #16578 (Abi Palagashvili). - Add new
cmath
functions: - acosh - asinh - atan2 - atanh - cosh - hypot - log1p - sinh. #16636 (Konstantin Malanchev). - Add a possibility to input enum value as it's id in TSV and CSV formats by default. #16834 (Kruglov Pavel).
- New tcpPort() function returns TCP port listened by this server. #17134 (Ivan).
Performance Improvement
- Now we can safely prune partitions with exact match. Useful case: Suppose table is partitioned by intHash64(x) % 100 and the query has condition on intHash64(x) % 100 verbatim, not on x. #16253 (Amos Bird).
- Use Floyd-Rivest algorithm, it should be the best for the ClickHouse use case of partial sorting. Bechmarks are in https://github.com/danlark1/miniselect and here. #16825 (Daniel Kutenin).
- Backported in #17589: Fix performance of reading from
Merge
tables over huge number ofMergeTree
tables. Fixes #7748. #16988 (Anton Popov).
Improvement
- Add
VIEW
subquery description toEXPLAIN
. Limit push down optimisation forVIEW
. Add local replicas ofDistributed
to query plan. #14936 (Nikolai Kochetov). - Made
indexOf()
use BloomFilter. #14977 (achimbab). - Throw exception about right sync privileges when MySQL sync user has error privileges. #15977 (TCeason).
- Fix possible stack overflow if a loop of materialized views is created. This closes #15732. #16048 (Alexey Milovidov).
- Support SNI in https connections to remote resources. This will allow to connect to Cloudflare servers that require SNI. This fixes #10055. #16252 (Alexey Milovidov).
- Now
ReplicatedMergeTree
tree engines family uses a separate thread pool for replicated fetches. Size of the pool limited by settingbackground_fetches_pool_size
which can be tuned with a server restart. The default value of the setting is 3 and it means that the maximum amount of parallel fetches is equal to 3 (and it allows to utilize 10G network). Fixes #520. #16390 (alesapin). - Now,
<auxiliary_zookeepers>
configuration can be changed inconfig.xml
and reloaded without server startup. #16627 (Amos Bird). - Allow reinterpret between integers and floats of the same size. fix 16640. #16657 (flynn).
- Workaround for use S3 with nginx server as proxy. Nginx currenty does not accept urls with empty path like
http://domain.com?delete
, but vanilla aws-sdk-cpp produces this kind of urls. This commit uses patched aws-sdk-cpp version, which makes urls with "/" as path in this cases, likehttp://domain.com/?delete
. #16709 (ianton-ru). - Remove empty directories for async INSERT at start of Distributed engine. #16729 (Azat Khuzhin).
- Usability improvement: better suggestions in syntax error message when
CODEC
expression is misplaced inCREATE TABLE
query. This fixes #12493. #16768 (Alexey Milovidov). - Better exception message when configuration for distributed DDL is absent. This fixes #5075. #16769 (Nikita Mikhaylov).
- Apply
use_compact_format_in_distributed_parts_names
for each INSERT (with internal_replication). #16788 (Azat Khuzhin). - Server refused to startup with exception message if wrong config is given (
metric_log
.collect_interval_milliseconds
is missing). #16815 (Ivan). - Add cutToFirstSignificantSubdomainWithWWW(). #16845 (Azat Khuzhin).
- Throw an informative error message when doing ATTACH/DETACH TABLE . Before this PR,
detach table <dict>
works but leads to an ill-formed in-memory metadata. #16885 (Amos Bird). - Remove empty parts after they were pruned by TTL, mutation, or collapsing merge algorithm. #16895 (Anton Popov).
- Make it possible to connect to
clickhouse-server
secure endpoint which requires SNI. This is possible whenclickhouse-server
is hosted behind TLS proxy. #16938 (filimonov). - Set default
host
andport
parameters forSOURCE(CLICKHOUSE(...))
to current instance and set defaultuser
value to'default'
. #16997 (Vladimir C). - Add ability to output all rows as a JSON array in the
JSONEachRow
format, controlled by theoutput_format_json_array_of_rows
setting. #17152 (Alexander Kuzmenkov). - Allow formatting named tuples as JSON objects when using JSON input/output formats, controlled by the
output_format_json_named_tuples_as_objects
setting, disabled by default. #17175 (Alexander Kuzmenkov). - Correct grammar in error message in JSONEachRow, JSONCompactEachRow, and RegexpRow input formats. #17205 (nico piderman).
Bug Fix
- Backported in #17620: Throw error when use ColumnTransformer replace non exist column. #16183 (hexiaoting).
- fixes #16574 fixes #16231 fix remote query failure when using 'if' suffix aggregate function. #16610 (Winter Zhang).
- Fixed #16081. #16613 (Nikita Mikhaylov).
- This will fix optimize_read_in_order/optimize_aggregation_in_order with max_threads>0 and expression in ORDER BY. #16637 (Azat Khuzhin).
- Fixed uncontrolled growth of TDigest. #16680 (hrissan).
- Backported in #17315: Return number of affected rows for INSERT queries via MySQL protocol. Previously ClickHouse used to always return 0, it's fixed. Fixes #16605. #16715 (Winter Zhang).
- Turn off parallel parsing when there is no enough memory for all threads to work simultaneously. Also there could be exceptions like "Memory limit exceeded" when somebody will try to insert extremely huge rows (> min_chunk_bytes_for_parallel_parsing), because each piece to parse has to be independent set of strings (one or more). #16721 (Nikita Mikhaylov).
- Fix
IN
operator over several columns and tuples with enabledtransform_null_in
setting. Fixes #15310. #16722 (Anton Popov). - Mask password in data_path in the system.distribution_queue. #16727 (Azat Khuzhin).
- Not for changelog. #16757 (Alexander Tokmakov).
- Backported in #17520: Fix optimize_trivial_count_query with partition predicate. #16767 (Azat Khuzhin).
- If no memory can be allocated while writing table metadata on disk, broken metadata file can be written. #16772 (Alexey Milovidov).
- Fix crash when using
any
without any arguments. This is for #16803 . cc @azat. #16826 (Amos Bird). - Abort multipart upload if no data was written to WriteBufferFromS3. #16840 (Pavel Kovalenko).
- Fix rare silent crashes when query profiler is on and ClickHouse is installed on OS with glibc version that has (supposedly) broken asynchronous unwind tables for some functions. This fixes #15301. This fixes #13098. #16846 (Alexey Milovidov).
- Prevent clickhouse server crashes when using TimeSeriesGroupSum. #16865 (filimonov).
- Backported in #17340: TODO. #16866 (Alexander Tokmakov).
- Fix possible error
Illegal type of argument
for queries withORDER BY
. Fixes #16580. #16928 (Nikolai Kochetov). - Install script should always create subdirs in config folders. This is only relevant for Docker build with custom config. #16936 (filimonov).
- Backported in #17586: Fix optimization of group by with enabled setting
optimize_aggregators_of_group_by_keys
and joins. Fixes #12604. #16951 (Anton Popov). - Backported in #17593: Fix order by optimization with monotonous functions. Fixes #16107. #16956 (Anton Popov).
- Blame info was not calculated correctly in
clickhouse-git-import
. #16959 (Alexey Milovidov). - Fix possible server crash after
ALTER TABLE ... MODIFY COLUMN ... NewType
whenSELECT
haveWHERE
expression on altering column and alter doesn't finished yet. #16968 (Amos Bird). - Reresolve the IP of the
format_avro_schema_registry_url
in case of errors. #16985 (filimonov). - Fixed wrong result in big integers (128, 256 bit) when casting from double. #16986 (Mike Kot).
- Backported in #17338: Fix Merge(Distributed()) with JOIN. #16993 (Azat Khuzhin).
- Backported in #17746: - Fix optimize_distributed_group_by_sharding_key for query with OFFSET only. #16996 (Azat Khuzhin).
- Avoid unnecessary network errors for remote queries which may be cancelled while execution, like queries with
LIMIT
. #17006 (Azat Khuzhin). - Fix LLVM's libunwind in the case when CFA register is RAX. This is the bug in LLVM's libunwind. We already have workarounds for this bug. #17046 (Alexey Milovidov).
- Backported in #17429: Bug fix for funciton fuzzBits, related issue: #16980. #17051 (hexiaoting).
- Fixed crash on
CREATE TABLE ... AS some_table
query whensome_table
was createdAS table_function()
Fixes #16944. #17072 (Alexander Tokmakov). - Fix bug when
ON CLUSTER
queries may hang forever for non-leader ReplicatedMergeTreeTables. #17089 (alesapin). - fixes #16923 fixes #15883 Fix MaterializeMySQL SYNC failure when the modify MySQL binlog_checksum. #17091 (Winter Zhang).
- Improve adaptive index granularity calculation when incoming blocks of data differ in bytes size a lot. #17120 (alesapin).
- Fix ColumnConst comparison which leads to crash. This fixed #17088 . #17135 (Amos Bird).
- Backported in #17560: Fix possible wrong index analysis when the types of the index comparison are different. This fixes #17122. #17145 (Amos Bird).
- Backported in #17394: Fix #15235. When clickhouse-copier handle non-partitioned table, throws segfault error. #17248 (Qi Chen).
- Backported in #17406: Fix set index invalidation when there are const columns in the subquery. This fixes #17246 . #17249 (Amos Bird).
- Backported in #17422: Fix possible
Unexpected packet Data received from client
error for Distributed queries withLIMIT
. #17254 (Azat Khuzhin). - fix
toInt256(inf)
stack overflow. close #17235. #17257 (flynn). - Backported in #17489: Fix crash while reading from
JOIN
table withLowCardinality
types. Fixes #17228. #17397 (Nikolai Kochetov). - Backported in #17451: Fixed high CPU usage in background tasks of *MergeTree tables. #17416 (Alexander Tokmakov).
- Backported in #17493: Fix duplicates after
DISTINCT
which were possible because of incorrect optimization. Fixes #17294. #17296 (li chengxiang). #17439 (Nikolai Kochetov). - Backported in #17523: Fix
ORDER BY
with enabled settingoptimize_redundant_functions_in_order_by
. #17471 (Anton Popov). - Backported in #17533: Fix bug when mark cache size was underestimated by clickhouse. It may happen when there are a lot of tiny files with marks. #17496 (alesapin).
- Backported in #17625: Fix alter query hang when the corresponding mutation was killed on the different replica. Fixes #16953. #17499 (alesapin).
- Backported in #17611: Fix the issue when server can stop accepting connections in very rare cases. #17542 (Alexey Milovidov).
- Backported in #17607: When clickhouse-client is used in interactive mode with multiline queries, single line comment was erronously extended till the end of query. This fixes #13654. #17565 (Alexey Milovidov).
- Backported in #17683: Exception
fmt::v7::format_error
can be logged in background for MergeTree tables. This fixes #17613. #17615 (Alexey Milovidov). - Backported in #17697: In might be determined incorrectly if cluster is circular- (cross-) replicated or not when executing
ON CLUSTER
query due to race condition whenpool_size
> 1. It's fixed. #17640 (Alexander Tokmakov). - Backported in #17727: Fixed
Function not implemented
error when executingRENAME
query inAtomic
database with ClickHouse running on Windows Subsystem for Linux. Fixes #17661. #17664 (Alexander Tokmakov). - Backported in #17784: Fixed problem when ClickHouse fails to resume connection to MySQL servers. #17681 (Alexander Kazakov).
- Backported in #17780: Exception message about max table size to drop was displayed incorrectly. #17764 (Alexey Milovidov).
- Backported in #17817: Do not restore parts from WAL if
in_memory_parts_enable_wal
is disabled. #17802 (detailyang). - Backported in #17842: fix incorrect initialize
max_compress_block_size
of MergeTreeWriterSettings withmin_compress_block_size
. #17833 (flynn).
Build/Testing/Packaging Improvement
- Fix UBSan report when trying to convert infinite floating point number to integer. This closes #14190. #16677 (Alexey Milovidov).
- Fix UBSan report in cache dictionaries. This closes #12641. #16763 (Alexey Milovidov).
- Do not instrument 3rd-party libraries with UBSan. #16764 (Alexey Milovidov).
- Fix UBSan report in Poco. This closes #12719. #16765 (Alexey Milovidov).
- fix query_db_generate build error. #16859 (hhhhhzhen).
- Try fix fasttest submodule clone #16132 https://clickhouse-test-reports.s3.yandex.net/16132/ad569f6d1bd2ce545db280daf7fbb9b8335de87b/fast_test.html#fail1. #16908 (Winter Zhang).
- Fixing unstable test in tests/testflows/ldap/external_user_directory/tests/authentications.py. #17161 (vzakaznikov).
- bump up rocksdb version to v6.14.5. #17179 (sundyli).
- Update embedded timezone data to version 2020d (also update cctz to the latest master). #17204 (filimonov).
- Improvements in coverage building images. #17233 (alesapin).
std::logic_error
is used at line 294 ofbase/common/StringRef.h
, so the appropriate<stdexcept>
header is required. #17256 (Matwey V. Kornilov).
NO CL ENTRY
- NO CL ENTRY: 'sync MySQL DDL atomicly'. #16704 (TCeason).
- NO CL ENTRY: 'RBAC Testflows - Server log intrumentation for debug and new ALTER tests'. #16719 (MyroTk).
- NO CL ENTRY: 'Enabling existing testflows RBAC tests.'. #16773 (MyroTk).
- NO CL ENTRY: 'Bump protobuf from 3.13.0 to 3.14.0 in /docs/tools'. #17056 (dependabot-preview[bot]).
- NO CL ENTRY: 'Fixed a problem with the translation of the document'. #17218 (qianmoQ).