thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-09-19 16:20:50 +00:00
Code Issues Packages Projects Releases Wiki Activity
master
ClickHouse/docs/changelogs/v20.8.1.4513-prestable.md

27 KiB
Raw Permalink Blame History

ClickHouse release v20.8.1.4513-prestable FIXME as compared to v20.7.1.4310-prestable

Backward Incompatible Change

  • Remove support for the ODBCDriver input/output format. This was a deprecated format once used for communication with the ClickHouse ODBC driver, now long superseded by the ODBCDriver2 format. Resolves #13629. #13847 (hexiaoting).
  • Extend parallel_distributed_insert_select setting, adding an option to run INSERT into local table. The setting changes type from Bool to UInt64, so the values false and true are no longer supported. If you have these values in server configuration, the server will not start. Please replace them with 0 and 1, respectively. #14060 (Azat Khuzhin).
  • Now OPTIMIZE FINAL query doesn't recalculate TTL for parts that were added before TTL was created. Use ALTER TABLE ... MATERIALIZE TTL once to calculate them, after that OPTIMIZE FINAL will evaluate TTL's properly. This behavior never worked for replicated tables. #14220 (alesapin).

New Feature

  • Support MaterializeMySQL database engine. Implements #4006. #10851 (Winter Zhang).
  • Support Kerberos authentication in Kafka, using krb5 and cyrus-sasl libraries. #12771 (Ilya Golshtein).
  • Add types Int128, Int256, UInt256 and related functions for them. Extend Decimals with Decimal256 (precision up to 76 digits). New types are under the setting allow_experimental_bigint_types. #13097 (Artem Zuikov).
  • Function position now supports optional start_pos argument. #13237 (Vladimir C).
  • Add ALTER SAMPLE BY statement that allows to change table sample clause. #13280 (Amos Bird).
  • Add new optional section <user_directories> to the main config. #13425 (Vitaly Baranov).
  • Add function normalizeQuery that replaces literals, sequences of literals and complex aliases with placeholders. Add function normalizedQueryHash that returns identical 64bit hash values for similar queries. It helps to analyze query log. This closes #11271. #13816 (Alexey Milovidov).
  • Add quantileExactLow and quantileExactHigh implementations with respective aliases for medianExactLow and medianExactHigh. #13818 (Bharat Nallan).
  • Add function defaultValueOfTypeName that returns the default value for a given type. #13877 (hcz).
  • Added date_trunc function that truncates a date/time value to a specified date/time part. #13888 (vladimir-golovchenko).
  • Add the ability to specify Default compression codec for columns that correspond to settings specified in config.xml. Implements: #9074. #14049 (alesapin).
  • Add setting min_index_granularity_bytes that protects against accidentally creating a table with very low index_granularity_bytes setting. #14139 (Bharat Nallan).
  • Add countDigits(x) function that count number of decimal digits in integer or decimal column. Add isDecimalOverflow(d, [p]) function that checks if the value in Decimal column is out of its (or specified) precision. #14151 (Artem Zuikov).

Performance Improvement

  • When performing trivial INSERT SELECT queries, automatically set max_threads to 1 or max_insert_threads, and set max_block_size to min_insert_block_size_rows. Related to #5907. #12195 (flynn).
  • Optimize has(), indexOf() and countEqual() functions for Array(LowCardinality(T)) and constant right arguments. #12550 (Mike Kot).
  • Slightly improve performance of aggregation by UInt8/UInt16 keys. #13099 (Alexey Milovidov).
  • Fail fast if max_rows_to_read limit is exceeded on parts scan. The motivation behind this change is to skip ranges scan for all selected parts if it is clear that max_rows_to_read is already exceeded. The change is quite noticeable for queries over big number of parts. #13677 (Roman Khavronenko).
  • Enable parallel INSERTs for table engines Null, Memory, Distributed and Buffer. #14120 (Alexey Milovidov).
  • Slightly optimize very short queries with LowCardinality. #14129 (Anton Popov).

Improvement

  • Added cache layer for DiskS3 (cache to local disk mark and index files). #13076 (Pavel Kovalenko).
  • Fix assert when decimal has too large negative exponent. Fixes #13188. #13228 (Alexey Milovidov).
  • Add QueryTimeMicroseconds, SelectQueryTimeMicroseconds and InsertQueryTimeMicroseconds to system.events. #13336 (ianton-ru).
  • Proper remote host checking in S3 redirects (security-related thing). #13404 (Vladimir Chebotarev).
  • Enable mixed granularity parts by default. #13449 (alesapin).
  • Move parts from DIskLocal to DiskS3 in parallel. #13459 (Pavel Kovalenko).
  • Support compound identifiers for custom settings. #13496 (Vitaly Baranov).
  • Provide monotonicity for toDate/toDateTime functions in more cases. Now the input arguments are saturated more naturally and provides better monotonicity. #13497 (Amos Bird).
  • In previous versions lcm function may produce assertion violation in debug build if called with specifically crafted arguments. This fixes #13368. #13510 (Alexey Milovidov).
  • Add MergeTree Write-Ahead-Log(WAL) dump tool. #13640 (BohuTANG).
  • Avoid too slow queries when arrays are manipulated as fields. Throw exception instead. #13753 (Alexey Milovidov).
    1. Add GTID-Based Replication, it works even when replication topology changes, and supported/prefered in MySQL 5.6/5.7/8.0 2. Add BIT/SET filed type supports 3. Fix up varchar type meta length bug. #13820 (BohuTANG).
  • Fix data race in lgamma function. This race was caught only in tsan, no side effects a really happened. #13842 (Nikolai Kochetov).
  • Function arrayCompact will compare NaNs bitwise if the type of array elements is Float32/Float64. In previous versions NaNs were always not equal if the type of array elements is Float32/Float64 and were always equal if the type is more complex, like Nullable(Float64). This closes #13857. #13868 (Alexey Milovidov).
  • Better error message for null value of TabSeparatedRow format. #13906 (jiang tao).
  • Fix wrong error for long queries. It was possible to get syntax error other than Max query size exceeded for correct query. #13928 (Nikolai Kochetov).
  • Corrected an error in AvroConfluent format that caused the Kafka table engine to stop processing messages when an abnormally small, malformed, message was received. #13941 (Gervasio Varela).
  • Increase limit in -Resample combinator to 1M. #13947 (Mikhail f. Shiryaev).
  • Conditional aggregate functions (for example: avgIf, sumIf, maxIf) should return NULL when miss rows and use nullable arguments. #13964 (Winter Zhang).
  • Slightly better performance of Memory table if it was constructed from a huge number of very small blocks (that's unlikely). Author of the idea: Mark Papadakis. Closes #14043. #14056 (Alexey Milovidov).
  • Fix toDeicmal256() and attempt to fix potential perf regression in Decimal to Float conversion. #14110 (Artem Zuikov).
  • Now it's possible to ALTER TABLE table_name FETCH PARTITION partition_expr FROM 'zk://<host>:<port>/path-in-zookeeper'. It's useful for shipping data to new clusters. #14155 (Amos Bird).

Bug Fix

Build/Testing/Packaging Improvement

  • Move Dockerfiles from integration tests to docker/test directory. docker_compose files are available in runner docker container. Docker images are built in CI and not in integration tests. #13448 (Ilya Yatsishin).
  • Skip PR's from robot-clickhouse. #13489 (Nikita Mikhaylov).
  • Fix typos in code with codespell. #13511 (Alexey Milovidov).
  • Enable check for typos in code with codespell. #13513 (Alexey Milovidov).
  • Ensure that there is no copy-pasted GPL code. #13514 (Alexey Milovidov).
  • Allow to run clickhouse binary without configuration. #13515 (Alexey Milovidov).
  • Added clickhouse install script, that is useful if you only have a single binary. #13528 (Alexey Milovidov).
  • Fix the remaining shellcheck notices. A preparation to enable Shellcheck. #13529 (Alexey Milovidov).
  • Enable Shellcheck in CI as a linter of .sh tests. This closes #13168. #13530 (Alexey Milovidov).
  • Make sure #10977 is invalid. #13539 (Amos Bird).
  • Increasing health-check timeouts for ClickHouse nodes and adding support to dump docker-compose logs if unhealthy containers found. #13612 (vzakaznikov).
  • Build ClickHouse with the most fresh tzdata from package repository. #13623 (Alexey Milovidov).
  • Removed -DENABLE_CURL_CLIENT for contrib/aws. #13628 (Vladimir Chebotarev).
  • Expose version of embedded tzdata via TZDATA_VERSION in system.build_options. #13648 (filimonov).
  • Updating LDAP user authentication suite to check that it works with RBAC. #13656 (vzakaznikov).
  • Add a CMake option to fail configuration instead of auto-reconfiguration, enabled by default. #13687 (Konstantin).
  • Fix link error in shared build. #13700 (Amos Bird).
  • FIx cassandra build on Mac OS. #13708 (Ilya Yatsishin).
  • Added docker image for style check. Added style check that all docker and docker compose files are located in docker directory. #13724 (Ilya Yatsishin).
  • ZooKeeper cannot work reliably in unit tests in CI infrastructure. Using unit tests for ZooKeeper interaction with real ZooKeeper is bad idea from the start (unit tests are not supposed to verify complex distributed systems). We already using integration tests for this purpose and they are better suited. #13745 (Alexey Milovidov).
  • Testflows LDAP module: adding missing certificates and dhparam.pem for openldap4. #13780 (vzakaznikov).
  • Enabled text-log in stress test to find more bugs. #13855 (Nikita Mikhaylov).
    • Adding retry logic when bringing up docker-compose cluster * Increasing COMPOSE_HTTP_TIMEOUT. #14112 (vzakaznikov).
  • Add the ability to write js-style comments in skip_list.json. #14159 (alesapin).
  • Switch tests docker images to use test-base parent. #14167 (Ilya Yatsishin).
  • Actually there are no symlinks there, so -type f is enough ~/workspace/ClickHouse/contrib/cctz/testdata/zoneinfo$ find . -type l -ls | wc -l 0 Closes #14209. #14215 (filimonov).

Other

NO CL ENTRY