mirror of
https://github.com/ClickHouse/ClickHouse.git
synced 2024-12-13 01:41:59 +00:00
69 KiB
69 KiB
sidebar_position | sidebar_label |
---|---|
1 | 2023 |
2023 Changelog
ClickHouse release v23.9.1.1854-stable (8f9a227de1
) FIXME as compared to v23.8.1.2992-lts (ebc7d9a9f3
)
Backward Incompatible Change
- Remove the
status_info
configuration option and dictionaries status from the default Prometheus handler. #54090 (Alexey Milovidov). - The experimental parts metadata cache is removed from the codebase. #54215 (Alexey Milovidov).
- Disable setting
input_format_json_try_infer_numbers_from_strings
by default, so we don't try to infer numbers from strings in JSON formats by default to avoid possible parsing errors when sample data contains strings that looks like a number. #55099 (Kruglov Pavel). - IPv6 bloom filter indexes created prior to March 2023 are not compatible with current version and have to be rebuilt. #54200 (Yakov Olkhovskiy).
New Feature
- Added new type of authentication based on SSH keys. It works only for Native TCP protocol. #41109 (George Gamezardashvili).
- Added IO Scheduling support for remote disks. Storage configuration for disk types
s3
,s3_plain
,hdfs
andazure_blob_storage
can now containread_resource
andwrite_resource
elements holding resource names. Scheduling policies for these resources can be configured in a separate server configuration sectionresources
. Queries can be marked using settingworkload
and classified using server configuration sectionworkload_classifiers
to achieve diverse resource scheduling goals. More details in docs/en/operations/workload-scheduling.md. #47009 (Sergei Trifonov). - Added a new column _block_number resolves #44532. #47532 (SmitaRKulkarni).
- Add options
partial_result_update_duration_ms
andmax_rows_in_partial_result
to show updates of a partial result of output table in real-time during query execution. #48607 (Alexey Perevyshin). - Support case-insensitive and dot-all matching modes in RegExpTree dictionaries. #50906 (Johann Gan).
- Add support for
ALTER TABLE MODIFY COMMENT
. Note: something similar was added by an external contributor a long time ago, but the feature did not work at all and only confused users. This closes #36377. #51304 (Alexey Milovidov). - Added "GCD" aka. "greatest common denominator" as a new data compression codec. The codec computes the GCD of all column values, and then divides each value by the GCD. The GCD codec is a data preparation codec (similar to Delta and DoubleDelta) and cannot be used stand-alone. It works with data integer, decimal and date/time type. A viable use case for the GCD codec are column values that change (increase/decrease) in multiples of the GCD, e.g. 24 - 28 - 16 - 24 - 8 - 24 (assuming GCD = 4). #53149 (Alexander Nam).
- Two new type aliases "DECIMAL(P)" (as shortcut for "DECIMAL(P, 0") and "DECIMAL" (as shortcut for "DECIMAL(10, 0)") were added. This makes ClickHouse more compatible with MySQL's SQL dialect. #53328 (Val Doroshchuk).
- Added a new system log table
backup_log
to track allBACKUP
andRESTORE
operations. #53638 (Victor Krasnov). - Added a format setting "output_format_markdown_escape_special_characters" (default: false). The setting controls whether special characters like "!", "#", "$" etc. are escaped (i.e. prefixed by a backslash) in the "Markdown" output format. #53860 (irenjj).
- Add function
decodeHTMLComponent
. #54097 (Bharat Nallan). - Added peak_threads_usage to query_log table. #54335 (Alexey Gerasimchuck).
- Add SHOW FUNCTIONS support to clickhouse-client. #54337 (Julia Kartseva).
- This PRs improves schema inference from JSON formats: 1) Now it's possible to infer named Tuples from JSON objects without experimantal JSON type under a setting
input_format_json_try_infer_named_tuples_from_objects
in JSON formats. Previously without experimantal type JSON we could only infer JSON objects as Strings or Maps, now we can infer named Tuple. Resulting Tuple type will conain all keys of objects that were read in data sample during schema inference. It can be useful for reading structured JSON data without sparse objects. The setting is enabled by default. 2) Allow parsing JSON array into a column with type String under settinginput_format_json_read_arrays_as_strings
. It can help reading arrays with values with different types. 3) Allow to use type String for JSON keys with unkown types (null
/[]
/{}
) in sample data under settinginput_format_json_infer_incomplete_types_as_strings
. Now in JSON formats we can read any value into String column and we can avoid getting errorCannot determine type for column 'column_name' by first 25000 rows of data, most likely this column contains only Nulls or empty Arrays/Maps
during schema inference by using type String for unknown types, so the data will be read successfully. #54427 (Kruglov Pavel). - Added function "toDaysSinceYearZero" with alias "TO_DAYS()" (for compatibility with MySQL) which returns the number of days passed since 0001-01-01. #54479 (Robert Schulze).
- Added functions YYYYMMDDtoDate(), YYYYMMDDtoDate32(), YYYYMMDDhhmmssToDateTime() and YYYYMMDDhhmmssToDateTime64(). They convert a date or date with time encoded as integer (e.g. 20230911) into a native date or date with time. As such, they provide the opposite functionality of existing functions YYYYMMDDToDate(), YYYYMMDDToDateTime(), YYYYMMDDhhmmddToDateTime(), YYYYMMDDhhmmddToDateTime64(). #54509 (Robert Schulze).
- Added "bandwidth_limit" IO scheduling node type. It allows you to specify
max_speed
andmax_burst
constraints on traffic passing though this node. More details in docs/en/operations/workload-scheduling.md. #54618 (Sergei Trifonov). - Function
toDaysSinceYearZero()
now supports arguments of typeDateTime
andDateTime64
. #54856 (Serge Klochkov). - Allow S3-style URLs for table functions
s3
,gcs
,oss
. URL is automatically converted to HTTP. Example:'s3://clickhouse-public-datasets/hits.csv'
is converted to'https://clickhouse-public-datasets.s3.amazonaws.com/hits.csv'
. #54931 (Yarik Briukhovetskyi). - Add several string distance functions, include
byteHammingDistance
,byteJaccardIndex
,byteEditDistance
. ### Documentation entry for user-facing changes. #54935 (flynn). - Add new setting
print_pretty_type_names
to print pretty deep nested types like Tuple/Maps/Arrays. #55095 (Kruglov Pavel).
Performance Improvement
- Improve performance of sorting for decimal columns. Improve performance of insertion into MergeTree if ORDER BY contains Decimal column. Improve performance of sorting when data is already sorted or almost sorted. #35961 (Maksim Kita).
- Improve performance for huge query analysis. Fixes #51224. #51469 (frinkr).
- Remove manual calls to
mmap/mremap/munmap
and delegate all this work tojemalloc
. #52792 (Nikita Taranov). - Now roaringBitmaps being optimized before serialization. #52842 (UnamedRus).
- Optimize group by constant keys. Will optimize queries with group by
_file/_path
after https://github.com/ClickHouse/ClickHouse/pull/53529. #53549 (Kruglov Pavel). - Speed up reading from S3 by enabling prefetches by default. #53709 (Alexey Milovidov).
- Do not implicitly read pk and version columns in lonely parts if unnecessary. #53919 (Duc Canh Le).
- Fixed high in CPU consumption when working with NATS. #54399 (Vasilev Pyotr).
- Since we use separate instructions for executing
toString()
with datetime argument, it is possible to improve performance a bit for non-datetime arguments and have some parts of the code cleaner. Follows up #53680. #54443 (Yarik Briukhovetskyi). - Instead of serializing json elements into a
std::stringstream
, this PR try to put the serialization result intoColumnString
direclty. #54613 (lgbo). - Enable ORDER BY optimization for reading data in corresponding order from a MergeTree table in case that the table is behind a view. #54628 (Vitaly Baranov).
- Improve JSON SQL functions by reusing
GeneratorJSONPath
. Since there are severalmake_shared
inGenerateorJSONPath
's constructor, it has bad performance. #54735 (lgbo).
Improvement
- Keeper improvement: Add a
createIfNotExists
Keeper command. #48855 (Konstantin Bogdanov). - Add IF EMPTY clause for DROP TABLE queries. #48915 (Pavel Novitskiy).
- The Keeper dynamically adjusts log levels. #50372 (helifu).
- Allow to replace long names of files of columns in
MergeTree
data parts to hashes of names. It helps to avoidFile name too long
error in some cases. #50612 (Anton Popov). - Allow specifying the expiration date and, optionally, the time for user credentials with
VALID UNTIL datetime
clause. #51261 (Nikolay Degterinsky). - Add setting
ignore_access_denied_multidirectory_globs
. #52839 (Andrey Zvonov). - Output valid JSON/XML on excetpion during HTTP query execution. Add setting
http_write_exception_in_output_format
to enable/disable this behaviour (enabled by default). #52853 (Kruglov Pavel). - More precise Integer type inference, fix #51236. #53003 (Chen768959).
- Keeper tries to batch flush requests for better performance. #53049 (Antonio Andelic).
- Introduced resolving of charsets in the string literals for MaterializedMySQL. #53220 (Val Doroshchuk).
- Fix a subtle issue with a rarely used
EmbeddedRocksDB
table engine in an extremely rare scenario: sometimes theEmbeddedRocksDB
table engine does not close files correctly in NFS after runningDROP TABLE
. #53502 (Mingliang Pan). - SQL functions "toString(datetime)" and "formatDateTime()" now support non-constant timezone arguments. #53680 (Yarik Briukhovetskyi).
RESTORE TABLE ON CLUSTER
must create replicated tables with a matching UUID on hosts. Otherwise the macro{uuid}
in ZooKeeper path can't work correctly after RESTORE. This PR implements that. #53765 (Vitaly Baranov).- Added restore setting
restore_broken_parts_as_detached
: if it's true the RESTORE process won't stop on broken parts while restoring, instead all the broken parts will be copied to thedetached
folder with the prefix `broken-from-backup'. If it's false the RESTORE process will stop on the first broken part (if any). The default value is false. #53877 (Vitaly Baranov). - The creation of Annoy indexes can now be parallelized using setting
max_threads_for_annoy_index_creation
. #54047 (Robert Schulze). - The MySQL interface gained a minimal implementation of prepared statements, just enough to allow a connection from Tableau Online to ClickHouse via the MySQL connector. #54115 (Serge Klochkov).
- Replaced the library to handle (encode/decode) base64 values from Turbo-Base64 to aklomp-base64. Both are SIMD-accelerated on x86 and ARM but 1. the license of the latter (BSD-2) is more favorable for ClickHouse, Turbo64 switched in the meantime to GPL-3, 2. with more GitHub stars, aklomp-base64 seems more future-proof, 3. aklomp-base64 has a slightly nicer API (which is arguably subjective), and 4. aklomp-base64 does not require us to hack around bugs (like non-threadsafe initialization). Note: aklomp-base64 rejects unpadded base64 values whereas Turbo-Base64 decodes them on a best-effort basis. RFC-4648 leaves it open whether padding is mandatory or not, but depending on the context this may be a behavioral change to be aware of. #54119 (Mikhail Koviazin).
- Add elapsed_ns to HTTP headers X-ClickHouse-Progress and X-ClickHouse-Summary. #54179 (joelynch).
- Implementation of
reconfig
(https://github.com/ClickHouse/ClickHouse/pull/49450),sync
, andexists
commands for keeper-client. #54201 (pufit). - "clickhouse-local" and "clickhouse-client" now allow to specify the "--query" parameter multiple times, e.g. './clickhouse-client --query "SELECT 1" --query "SELECT 2"'. This syntax is slightly more intuitive than `./clickhouse-client --multiquery "SELECT 1;SELECT2", a bit easier to script (e.g. "queries.push_back('--query "$q"')") and more consistent with the behavior of existing parameter "--queries-file" (e.g. "./clickhouse client --queries-file queries1.sql --queries-file queries2.sql"). #54249 (Robert Schulze).
- Add sub-second precision to
formatReadableTimeDelta
. #54250 (Andrey Zvonov). - Fix wrong reallocation in HashedArrayDictionary:. #54254 (Vitaly Baranov).
- Enable allow_remove_stale_moving_parts by default. #54260 (vdimir).
- Fix using count from cache and improve progress bar for reading from archives. #54271 (Kruglov Pavel).
- Add support for S3 credentials using SSO. To define a profile to be used with SSO, set
AWS_PROFILE
environment variable. #54347 (Antonio Andelic). - Support NULL as default for nested types Array/Tuple/Map for input formats. Closes #51100. #54351 (Kruglov Pavel).
- This is actually a bug fix, but not sure I'll be able to add a test to support the case, so I have put it as an improvement. This issue was introduced in https://github.com/ClickHouse/ClickHouse/pull/45878, which is when CH started reading arrow in batches. #54370 (Arthur Passos).
- Add STD alias to stddevPop function for MySQL compatibility. Closes #54274. #54382 (Nikolay Degterinsky).
- Add
addDate
function for compatibility with MySQL andsubDate
for consistency. Reference #54275. #54400 (Nikolay Degterinsky). - Parse data in JSON format as JSONEachRow if failed to parse metadata. It will allow to read files with
.json
extension even if real format is JSONEachRow. Closes #45740. #54405 (Kruglov Pavel). - Pass http retry timeout as milliseconds. #54438 (Duc Canh Le).
- Support SAMPLE BY for VIEW. #54477 (Azat Khuzhin).
- Add modification_time into system.detached_parts. #54506 (Azat Khuzhin).
- Added a setting "splitby_max_substrings_includes_remaining_string" which controls if functions "splitBy*()" with argument "max_substring" > 0 include the remaining string (if any) in the result array (Python/Spark semantics) or not. The default behavior does not change. #54518 (Robert Schulze).
- Now clickhouse-client process files in parallel in case of
INFILE 'glob_expression'
. Closes #54218. #54533 (Max K.). - Allow to use primary key for IN function where primary key column types are different from
IN
function right side column types. Example:SELECT id FROM test_table WHERE id IN (SELECT '5')
. Closes #48936. #54544 (Maksim Kita). - Better integer types inference for Int64/UInt64 fields. Continuation of https://github.com/ClickHouse/ClickHouse/pull/53003. Now it works also for nested types like Arrays of Arrays anf for functions like
map/tuple
. Issue: #51236. #54553 (Kruglov Pavel). - HashJoin tries to shrink internal buffers consuming half of maximal available memory (set by
max_bytes_in_join
). #54584 (vdimir). - Added array operations for multiplying, dividing and modulo on scalar. Works in each way, for example
5 * [5, 5]
and[5, 5] * 5
- both cases are possible. #54608 (Yarik Briukhovetskyi). - Added function
timestamp
for compatibility with MySQL. Closes #54275. #54639 (Nikolay Degterinsky). - Respect max_block_size for array join to avoid possible OOM. Close #54290. #54664 (李扬).
- Add optional
version
argument torm
command inkeeper-client
to support safer deletes. #54708 (János Benjamin Antal). - Disable killing the server by systemd (that may lead to data loss when using Buffer tables). #54744 (Azat Khuzhin).
- Added field "is_deterministic" to system table "system.functions" which indicates whether the result of a function is stable between two invocations (given exactly the same inputs) or not. #54766 (Robert Schulze).
- Made the views in schema "information_schema" more compatible with the equivalent views in MySQL (i.e. modified and extended them) up to a point where Tableau Online is able to connect to ClickHouse. More specifically: 1. The type of field "information_schema.tables.table_type" changed from Enum8 to String. 2. Added fields "table_comment" and "table_collation" to view "information_schema.table". 3. Added views "information_schema.key_column_usage" and "referential_constraints". 4. Replaced uppercase aliases in "information_schema" views with concrete uppercase columns. #54773 (Serge Klochkov).
- The query cache now returns an error if the user tries to cache the result of a query with a non-deterministic function such as "now()", "randomString()" and "dictGet()". Compared to the previous behavior (silently don't cache the result), this reduces confusion and surprise for users. #54801 (Robert Schulze).
- Forbid special columns for file/s3/url/... storages, fix insert into ephemeral columns from files. Closes #53477. #54803 (Kruglov Pavel).
- More configurable collecting metadata for backup. #54804 (Vitaly Baranov).
clickhouse-local
's log file (if enabled with --server_logs_file flag) will now prefix each line with timestamp, thread id, etc, just likeclickhouse-server
. #54807 (Michael Kolupaev).- Reuse HTTP connections in s3 table function. #54812 (Michael Kolupaev).
- Avoid excessive calls to getifaddrs in isLocalAddress. #54819 (Duc Canh Le).
- Field "is_obsolete" in system.merge_tree_settings is now 1 for obsolete merge tree settings. Previously, only the description indicated that the setting is obsolete. #54837 (Robert Schulze).
- Make it possible to use plural when using interval literals.
INTERVAL 2 HOURS
should be equivalent toINTERVAL 2 HOUR
. #54860 (Jordi Villar). - Replace the linear method in
MergeTreeRangeReader::Stream::ceilRowsToCompleteGranules
with a binary search. #54869 (usurai). - Always allow the creation of a projection with
Nullable
PK. This fixes #54814. #54895 (Amos Bird). - Retry backup S3 operations after connection reset failure. #54900 (Vitaly Baranov).
- Make the exception message exact in case of the maximum value of a settings is less than the minimum value. #54925 (János Benjamin Antal).
- LIKE, match, and other regular expressions matching functions now allow matching with patterns containing non-UTF-8 substrings by falling back to binary matching. Example: you can use
string LIKE '\xFE\xFF%'
to detect BOM. This closes #54486. #54942 (Alexey Milovidov). - ProfileEvents added ContextLockWaitMicroseconds event. #55029 (Maksim Kita).
- Added field "is_deterministic" to system table "system.functions" which indicates whether the result of a function is stable between two invocations (given exactly the same inputs) or not. #55035 (Robert Schulze).
- View information_schema.tables now has a new field
data_length
which shows the approximate size of the data on disk. Required to run queries generated by Amazon QuickSight. #55037 (Robert Schulze).
Build/Testing/Packaging Improvement
- ClickHouse is built with Musl instead of GLibc by default. #52550 (Alexey Milovidov).
- ClickHouse is built with Musl instead of GLibc. #52721 (Azat Khuzhin).
- Bumped the compiler of official and continuous integration builds of ClickHouse from Clang 16 to 17. #53831 (Robert Schulze).
- Fix flaky test.
wait_resolver
function was asserting the response to be == proxy1, but it might actually return proxy2. Account for that as well. #54191 (Arthur Passos). - Regenerated tld data for lookups (
tldLookup.generated.cpp
). #54269 (Bharat Nallan). - Report properly timeout for check itself in
fast_test_check
/stress_check
. #54278 (Igor Nikonov). - Suddenly,
test_host_regexp_multiple_ptr_records_concurrent
became flaky. #54307 (Arthur Passos). - Fixed precise float parsing issue on s390x. #54330 (Harry Lee).
- Enrich
changed_images.json
with the latest tag from master for images that are not changed in the pull request. #54369 (János Benjamin Antal). - Fixed endian issue in jemalloc_bins system table for s390x. #54517 (Harry Lee).
- Fixed random generation issue for UInt256 and IPv4 on s390x. #54576 (Harry Lee).
- Remove redundant
clickhouse-keeper-client
symlink. #54587 (Tomas Barton). - Use
/usr/bin/env
to resolve bash. #54603 (Fionera). - Move all
tests/ci/*.lib files
tostateless-tests
image. Closes #54540. #54668 (Kruglov Pavel). - We build and upload them for every push, which isn't worth it. #54675 (Mikhail f. Shiryaev).
- Fixed SimHash function endian issue for s390x. #54793 (Harry Lee).
- Do not clone the fast tests repo twice; parallelize submodules checkout; use the current user in the fast-tests container. #54849 (Mikhail f. Shiryaev).
- Avoid running pull request ci workflow for fixes touching .md files only. #54914 (Max K.).
- CMake added
PROFILE_CPU
option needed to performperf record
without using DWARF call graph. #54917 (Maksim Kita). - Use
--gtest_output='json:'
to parse unit test results. #54922 (Mikhail f. Shiryaev). - Added support for additional scripts (you need to mound a volume) to extend build process. #55000 (Nikita Mikhaylov).
- If the linker is different than LLD, stop with a fatal error. #55036 (Alexey Milovidov).
Bug Fix (user-visible misbehavior in an official stable release)
- Store NULL in scalar result map for empty subquery result #52240 (vdimir).
- Fix misleading error message in OUTFILE with CapnProto/Protobuf #52870 (Kruglov Pavel).
- Fix summary reporting with parallel replicas with LIMIT #53050 (Raúl Marín).
- Fix throttling of BACKUPs from/to S3 (in case native copy was not used) and in some other places as well #53336 (Azat Khuzhin).
- Fix IO throttling during copying whole directories #53338 (Azat Khuzhin).
- Fix: moved to prewhere condition actions can lose column #53492 (Yakov Olkhovskiy).
- 37737 fixed internal error when replacing with byte-equal parts #53735 (Pedro Riera).
- Fix: require columns participating in interpolate expression #53754 (Yakov Olkhovskiy).
- Fix cluster discovery initialization + setting up fail points in config #54113 (vdimir).
- Fix issues in accurateCastOrNull #54136 (Salvatore Mesoraca).
- Fix nullable primary key in final #54164 (Amos Bird).
- Inserting only non-duplicate chunks in MV #54184 (Pedro Riera).
- Fix REPLACE/MOVE PARTITION with zero-copy replication #54193 (Alexander Tokmakov).
- Fix: parallel replicas over distributed don't read from all replicas #54199 (Igor Nikonov).
- Fix: allow IPv6 for bloom filter #54200 (Yakov Olkhovskiy).
- fix possible type mismatch with IPv4 #54212 (Bharat Nallan).
- Fix system.data_skipping_indices for recreated indices #54225 (Artur Malchanau).
- fix name clash for multiple join rewriter v2 #54240 (Tao Wang).
- Fix unexpected errors in system.errors after join #54306 (vdimir).
- Fix isZeroOrNull(NULL) #54316 (flynn).
- Fix: parallel replicas over distributed with prefer_localhost_replica=1 #54334 (Igor Nikonov).
- Fix logical error in vertical merge + replacing merge tree + optimize cleanup #54368 (alesapin).
- Fix possible error 'URI contains invalid characters' in s3 table function #54373 (Kruglov Pavel).
- Fix segfault in AST optimization of
arrayExists
function #54379 (Nikolay Degterinsky). - Check for overflow before addition in
analysisOfVariance
function #54385 (Antonio Andelic). - reproduce and fix the bug in removeSharedRecursive #54430 (Sema Checherinda).
- Fix possible incorrect result with SimpleAggregateFunction in PREWHERE and FINAL #54436 (Azat Khuzhin).
- Fix filtering parts with indexHint for non analyzer #54449 (Azat Khuzhin).
- Fix aggregate projections with normalized states #54480 (Amos Bird).
- Bugfix/local multiquery parameter #54498 (CuiShuoGuo).
- clickhouse-local support --database command line argument #54503 (vdimir).
- Fix possible parsing error in WithNames formats with disabled input_format_with_names_use_header #54513 (Kruglov Pavel).
- Fix rare case of CHECKSUM_DOESNT_MATCH error #54549 (alesapin).
- Fix zero copy garbage #54550 (Alexander Tokmakov).
- Fix sorting of UNION ALL of already sorted results #54564 (Vitaly Baranov).
- Fix snapshot install in Keeper #54572 (Antonio Andelic).
- Fix race in
ColumnUnique
#54575 (Nikita Taranov). - Annoy/Usearch index: Fix LOGICAL_ERROR during build-up with default values #54600 (Robert Schulze).
- Fix serialization of
ColumnDecimal
#54601 (Nikita Taranov). - Fix schema inference for *Cluster functions for column names with spaces #54635 (Kruglov Pavel).
- Fix using structure from insertion tables in case of defaults and explicit insert columns #54655 (Kruglov Pavel).
- Fix: avoid using regex match, possibly containing alternation, as a key condition. #54696 (Yakov Olkhovskiy).
- Fix ReplacingMergeTree with vertical merge and cleanup #54706 (SmitaRKulkarni).
- Fix virtual columns having incorrect values after ORDER BY #54811 (Michael Kolupaev).
- Fix filtering parts with indexHint for non analyzer (resubmit) #54825 (Azat Khuzhin).
- Fix Keeper segfault during shutdown #54841 (Antonio Andelic).
- Fix "Invalid number of rows in Chunk" in MaterializedPostgreSQL #54844 (Kseniia Sumarokova).
- Move obsolete format settings to separate section #54855 (Kruglov Pavel).
- Fix zero copy locks with hardlinks #54859 (Alexander Tokmakov).
- Fix
FINAL
produces invalid read ranges in a rare case #54934 (Nikita Taranov). - Rebuild minmax_count_projection when partition key gets modified #54943 (Amos Bird).
- Fix bad cast to ColumnVector in function if #55019 (Kruglov Pavel).
- Fix: insert quorum w/o keeper retries #55026 (Igor Nikonov).
- Fix simple state with nullable #55030 (Pedro Riera).
- Prevent attaching parts from tables with different projections or indices #55062 (János Benjamin Antal).
NO CL ENTRY
- NO CL ENTRY: 'Revert "Revert "Fixed wrong python test name pattern""'. #54043 (Mikhail f. Shiryaev).
- NO CL ENTRY: 'Revert "Fix: respect skip_unavailable_shards with parallel replicas"'. #54189 (Alexander Tokmakov).
- NO CL ENTRY: 'Revert "Add settings for real-time updates during query execution"'. #54470 (Alexey Milovidov).
- NO CL ENTRY: 'Revert "Fix issues in accurateCastOrNull"'. #54472 (Alexey Milovidov).
- NO CL ENTRY: 'Revert "Revert "Add settings for real-time updates during query execution""'. #54476 (Nikolai Kochetov).
- NO CL ENTRY: 'Revert "add runOptimize call in bitmap write method"'. #54528 (Alexey Milovidov).
- NO CL ENTRY: 'Revert "Optimize uniq to count"'. #54566 (Alexey Milovidov).
- NO CL ENTRY: 'Revert "Add stateless test for clickhouse keeper-client --no-confirmation"'. #54616 (Alexey Milovidov).
- NO CL ENTRY: 'Revert "Remove flaky tests for the experimental
UNDROP
feature"'. #54671 (Alexander Tokmakov). - NO CL ENTRY: 'Revert "Fix filtering parts with indexHint for non analyzer"'. #54806 (Azat Khuzhin).
- NO CL ENTRY: 'Revert "refine error code of duplicated index in create query"'. #54840 (Alexander Tokmakov).
- NO CL ENTRY: 'Revert "Avoid excessive calls to getifaddrs in isLocalAddress"'. #54893 (Igor Nikonov).
- NO CL ENTRY: 'Revert "Fix NATS high cpu usage"'. #55005 (Nikolay Degterinsky).
NOT FOR CHANGELOG / INSIGNIFICANT
- libFuzzer: add CI fuzzers build, add tcp protocol fuzzer, fix other fuzzers. #42599 (Yakov Olkhovskiy).
- Add new exceptions to 4xx error #50722 (Boris Kuschel).
- Test libunwind changes. #51436 (Nikolai Kochetov).
- Fix data race in copyFromIStreamWithProgressCallback #51449 (Michael Kolupaev).
- Abort on
std::logic_error
in CI #51907 (Alexander Tokmakov). - Unify setting http keep-alive timeout, increase default to 30s #53068 (Nikita Taranov).
- Add a regression test for broken Vertical merge after ADD+DROP COLUMN #53214 (Azat Khuzhin).
- Revert "Revert "dateDiff: add support for plural units."" #53803 (Han Fei).
- Fix some tests #53892 (Alexander Tokmakov).
- Refactoring of reading from
MergeTree
tables #53931 (Anton Popov). - Use pathlib.Path in S3Helper, rewrite build reports, improve small things #54010 (Mikhail f. Shiryaev).
- Correct UniquesHashSet to be endianness-independent. #54045 (Austin Kothig).
- Increase retries for test_merge_tree_azure_blob_storage #54069 (SmitaRKulkarni).
- Fix SipHash128 reference for big-endian platforms #54095 (ltrk2).
- Small usearch index improvements: metrics and configurable internal data type #54103 (Michael Kolupaev).
- Small refactoring for read from object storage #54134 (Kseniia Sumarokova).
- Minor changes #54171 (Kseniia Sumarokova).
- Fix hostname and co result constness in new analyzer #54174 (vdimir).
- Amend a confusing line of code in Loggers.cpp #54183 (Victor Krasnov).
- Fix partition id pruning for analyzer. #54185 (Nikolai Kochetov).
- Update version after release #54186 (Alexey Milovidov).
- Update version_date.tsv and changelogs after v23.8.1.2992-lts #54188 (robot-clickhouse).
- Fix pager in client/local interactive mode when not all data had been read #54190 (Azat Khuzhin).
- Fix flaky test
01099_operators_date_and_timestamp
#54195 (Alexey Milovidov). - Save system tables from s3_disk in the report #54198 (Alexander Tokmakov).
- Fix timezones in the CI Logs database #54210 (Alexey Milovidov).
- 2R: Fix: respect skip_unavailable_shards with parallel replicas #54213 (Igor Nikonov).
- S3Queue is experimental #54214 (Alexey Milovidov).
- Improve vars with refernce in Settings cpps #54220 (xuzifu666).
- Add ProfileEvents::Timer class #54221 (Stig Bakken).
- Test: extend cluster_all_replicas integration test with skip_unavailable_shards #54223 (Igor Nikonov).
- remove semicolon #54236 (YinZheng-Sun).
- Fix bad code in the
system.filesystem_cache
: catching exceptions #54237 (Alexey Milovidov). - Merge #54236 #54238 (Alexey Milovidov).
- Minor improvement, load from config #54244 (zhanglistar).
- Follow-up to #54198 #54246 (Alexander Tokmakov).
- Properly re-initialize ZooKeeper fault injection #54251 (Alexander Gololobov).
- Update ci-slack-bot.py #54253 (Alexander Tokmakov).
- Fix clickhouse-test --no-drop-if-fail on reference mismatch #54256 (Azat Khuzhin).
- Improve slack-bot-ci lambda #54258 (Mikhail f. Shiryaev).
- Update version_date.tsv and changelogs after v23.3.12.11-lts #54259 (robot-clickhouse).
- Minor change #54261 (flynn).
- Add a note of where the lambda is deployed #54268 (Mikhail f. Shiryaev).
- Query cache: Log caching of entries #54270 (Robert Schulze).
- Update version_date.tsv and changelogs after v23.8.2.7-lts #54273 (robot-clickhouse).
- Fix test
02783_parsedatetimebesteffort_syslog
#54279 (Alexey Milovidov). - Fix
test_keeper_disks
#54291 (Antonio Andelic). - Code improvement for reading from archives #54293 (Antonio Andelic).
- Rollback testing part from #42599 #54301 (Mikhail f. Shiryaev).
- CI: libFuzzer integration #54310 (Yakov Olkhovskiy).
- Update version_date.tsv and changelogs after v23.3.13.6-lts #54313 (robot-clickhouse).
- Add logs for parallel replica over distributed #54315 (Igor Nikonov).
- Increase timeout for system.stack_trace in 01051_system_stack_trace #54321 (Azat Khuzhin).
- Fix replace_partition test #54322 (Pedro Riera).
- Fix segfault in system.zookeeper #54326 (Alexander Tokmakov).
- Fixed flaky test
02841_parallel_replicas_summary
#54331 (Nikita Mikhaylov). - Consolidate GCD codec tests (Follow up to #53149) #54332 (Robert Schulze).
- Fixed wrong dereference problem in Context::setTemporaryStorageInCache #54333 (Alexey Gerasimchuck).
- Used assert_cast instead of dynamic_cast in ExternalDataSourceCache #54336 (Alexey Gerasimchuck).
- Fix bad punctuation in Keeper's logs #54338 (Alexey Milovidov).
- Improved protection from dereferencing of nullptr #54339 (Alexey Gerasimchuck).
- Fix filesystem cache test #54343 (Salvatore Mesoraca).
- Parallel replicas: remove unused code #54354 (Igor Nikonov).
- Fix flaky test test_storage_azure_blob_storage/test.py::test_schema_iference_cache #54367 (Kruglov Pavel).
- Enable hedged requests integration tests with tsan, use max_distributed_connections=1 to fix possible flakiness #54371 (Kruglov Pavel).
- Use abiv2 when generating OpenSSL .s files for powerpc64le #54375 (Boris Kuschel).
- Disable prefer_localhost_replica in test for parallel replicas #54377 (Igor Nikonov).
- Fix incorrect formatting of CREATE query with PRIMARY KEY #54403 (Nikolay Degterinsky).
- Fix failed assert in attach thread during startup retries #54408 (Antonio Andelic).
- Hashtable order fix on big endian platform #54409 (Suzy Wang).
- Small fine-tune for using ColumnNullable pointer #54435 (Alex Cheng).
- Update automated commit status comment #54441 (vdimir).
- Remove useless line #54466 (Alexey Milovidov).
- Add a log message on replicated table drop #54467 (Dmitry Novik).
- Cleanup: unnecessary SelectQueryInfo usage around distributed #54468 (Igor Nikonov).
- Add
instance_type
column to CI Logs and thechecks
table #54469 (Alexey Milovidov). - Refactor IHints #54481 (flynn).
- Fix strange message #54489 (Alexey Milovidov).
- Bump re2 to latest main #54492 (Robert Schulze).
- S3 artifacts #54504 (Mikhail f. Shiryaev).
- Flush logs for system.backup_log test. #54507 (Nikolai Kochetov).
- Fix use-after-free in
MergeTreePrefetchedReadPool
#54512 (Anton Popov). - Disable parallel replicas on shards with not enough nodes #54519 (Igor Nikonov).
- Parallel replicas: cleanup unused params #54520 (Igor Nikonov).
- FunctionHelpers remove areTypesEqual function #54546 (Maksim Kita).
- Add stateless test for clickhouse keeper-client --no-confirmation #54547 (Azat Khuzhin).
- Increase default timeout in tests for keeper-client #54551 (pufit).
- clang-format: Disable namespace indentation and omit {} in if/for/while #54554 (Robert Schulze).
- ngramDistance* queries fix for big endian platform #54555 (Suzy Wang).
- Fix AST fuzzer crash in MergeTreeIndex{FullText|Inverted} #54563 (Robert Schulze).
- Remove output_format_markdown_escape_special_characters from settings changes history #54585 (Kruglov Pavel).
- Add basic logic to find releasable commits #54604 (János Benjamin Antal).
- Fix reading of virtual columns in reverse order #54610 (Anton Popov).
- Fix possible CANNOT_READ_ALL_DATA during ZooKeeper client finalization and add some tests #54632 (Azat Khuzhin).
- Fix a bug in addData and subData functions #54636 (Nikolay Degterinsky).
- Follow-up to #54550 #54641 (Alexander Tokmakov).
- Remove broken lockless variant of re2 #54642 (Robert Schulze).
- Bump abseil #54646 (Robert Schulze).
- limit the delay before next try in S3 #54651 (Sema Checherinda).
- Fix parser unit tests #54670 (János Benjamin Antal).
- Fix: Log engine Mark file to read and write in little Endian for s390x #54677 (bhavnajindal).
- Update WebObjectStorage.cpp #54695 (Kseniia Sumarokova).
- add cancelation point to s3 retries #54697 (Sema Checherinda).
- Revert default batch size for Keeper #54745 (Antonio Andelic).
- Enable
allow_experimental_undrop_table_query
#54754 (Alexander Tokmakov). - Fix 02882_clickhouse_keeper_client_no_confirmation test #54761 (Azat Khuzhin).
- Better exception message in checkDataPart #54768 (alesapin).
- Don't use default move assignment in TimerDescriptor #54769 (Kruglov Pavel).
- Add retries to rests test_async_query_sending/test_async_connect #54772 (Kruglov Pavel).
- update comment #54780 (flynn).
- Fix broken tests for clickhouse-diagnostics #54790 (Mikhail f. Shiryaev).
- refine error code of duplicated index in create query #54791 (Han Fei).
- Do not set PR status label #54799 (vdimir).
- Prevent parquet schema inference reading the first 1 MB of the file unnecessarily #54808 (Michael Kolupaev).
- Prevent ParquetMetadata reading 40 MB from each file unnecessarily #54809 (Michael Kolupaev).
- Use appropriate error code instead of LOGICAL_ERROR #54810 (Yakov Olkhovskiy).
- Adjusting
num_streams
by expected work in StorageS3 #54815 (pufit). - Fix test_backup_restore_on_cluster/test.py::test_stop_other_host_during_backup flakiness #54816 (Azat Khuzhin).
- Remove config files sizes check #54824 (Igor Nikonov).
- Set correct size for signal pipe buffer #54836 (Antonio Andelic).
- Refactor and split up vector search tests #54839 (Robert Schulze).
- Add some logging to StorageRabbitMQ #54842 (Kseniia Sumarokova).
- Update CHANGELOG.md #54843 (Ilya Yatsishin).
- Refactor and simplify multi-directory globs #54863 (Andrey Zvonov).
- KeeperTCPHandler.cpp: Fix clang-17 build #54874 (Robert Schulze).
- Decrease timeout for fast tests with a commit #54878 (Mikhail f. Shiryaev).
- More stable
02703_keeper_map_concurrent_create_drop
#54879 (Antonio Andelic). - Fix division by zero in StorageS3 #54904 (pufit).
- Set exception for promise in
CreatingSetsTransform
#54920 (Antonio Andelic). - Fix an exception message in Pipe::addTransform #54926 (Alex Cheng).
- Fix data race during BackupsWorker::backup_log initialization #54928 (Victor Krasnov).
- Provide support for BSON on BE #54933 (Austin Kothig).
- Set a minimum limit of
num_streams
in StorageS3 #54936 (pufit). - Ipv4 read big endian #54938 (Suzy Wang).
- Fix data race in SYSTEM STOP LISTEN #54939 (Nikolay Degterinsky).
- Add desperate instrumentation for debugging deadlock in MultiplexedConnections #54940 (Michael Kolupaev).
- Respect max_block_size while generating rows for system.stack_trace (will fix flakiness of the test) #54946 (Azat Khuzhin).
- Remove test
01051_system_stack_trace
#54951 (Alexey Milovidov). - Add a test for compatibility #54960 (Alexey Milovidov).
- Remove test
02151_hash_table_sizes_stats
#54961 (Alexey Milovidov). - Remove 02151_hash_table_sizes_stats_distributed (fixes broken CI) #54969 (Azat Khuzhin).
- Use pregenerated gRPC protocol pb2 files to fix test flakyness. #54976 (Vitaly Baranov).
- Delete a test #54984 (Kseniia Sumarokova).
- Add assertion #54985 (Kseniia Sumarokova).
- Fix test parallel replicas over distributed #54987 (Igor Nikonov).
- Update README.md #54990 (Tyler Hannan).
- Re-enable clang-tidy checks disabled in the Clang 17 update #54999 (Robert Schulze).
- Print more information about one logical error in MergeTreeDataWriter #55001 (Michael Kolupaev).
- Add a test #55003 (Kseniia Sumarokova).
- Lower log levels for
SSOCredentialsProvider
#55012 (Antonio Andelic). - Set exception for promise in
CreatingSetsTransform
in more cases #55013 (Antonio Andelic). - Setting compile_aggregate_expressions comment fix #55020 (Maksim Kita).
- Revert "Added field "is_deterministic" to system.functions" #55022 (Alexander Tokmakov).
- Get rid of the most of
os.path
stuff #55028 (Mikhail f. Shiryaev). - Fix pre-build scripts for old branches #55032 (Nikita Mikhaylov).
- Review fix for #54935 #55042 (flynn).
- Update gtest_lru_file_cache.cpp #55053 (Kseniia Sumarokova).
- Fix prebuild scripts one more time #55059 (Nikita Mikhaylov).
- Use different names for variables inside build.sh #55067 (Nikita Mikhaylov).
- Remove String Jaccard Index #55080 (Alexey Milovidov).
- I don't understand why backup log is not enabled by default #55081 (Alexey Milovidov).
- Fix typo in packager when ccache is used #55104 (Ilya Yatsishin).
- Reduce flakiness of 01455_opentelemetry_distributed #55111 (Michael Kolupaev).
- Fix build #55113 (Alexey Milovidov).