ClickHouse/docs/changelogs/v24.9.3.128-stable.md

sidebar_position	sidebar_label
1	2024

2024 Changelog

ClickHouse release v24.9.3.128-stable (9a816c73dd) FIXME as compared to v24.9.2.42-stable (de7c791a2e)

Backward Incompatible Change

• Backported in #71361: Fix possible error No such file or directory due to unescaped special symbols in files for JSON subcolumns. #71182 (Pavel Kruglov).

New Feature

• Backported in #71318: Allow each authentication method to have its own expiration date, remove from user entity. #70090 (Arthur Passos).

Improvement

• Backported in #70681: Don't do validation when synchronizing user_directories from keeper. #70644 (Raúl Marín).
• Backported in #71668: When user/group is given as ID, the clickhouse su fails. This patch fixes it to accept UID:GID as well. ### Documentation entry for user-facing changes. #71626 (Mikhail f. Shiryaev).
• Backported in #71938: Update HostResolver 3 times in a history period. #71863 (Sema Checherinda).

Bug Fix (user-visible misbehavior in an official stable release)

• Backported in #70936: Fix incorrect JOIN ON section optimization in case of IS NULL check under any other function (like NOT) that may lead to wrong results. Closes #67915. #68049 (Vladimir Cherkasov).
• Backported in #70441: Fix vrash during insertion into FixedString column in PostgreSQL engine. #69584 (Pavel Kruglov).
• Backported in #70563: Fix getSubcolumn with LowCardinality columns by overriding useDefaultImplementationForLowCardinalityColumns to return true. #69831 (Miсhael Stetsyuk).
• Backported in #70871: Avoid reusing columns among different named tuples when evaluating tuple functions. This fixes #70022. #70103 (Amos Bird).
• Backported in #70434: Fix possible crash in JSON column. #70172 (Pavel Kruglov).
• Backported in #70309: Fix multiple issues with arrayMin and arrayMax. #70207 (Raúl Marín).
• Backported in #70625: Fix server segfault on creating a materialized view with two selects and an INTERSECT, e.g. CREATE MATERIALIZED VIEW v0 AS (SELECT 1) INTERSECT (SELECT 1);. #70264 (Konstantin Bogdanov).
• Backported in #70347: Don't modify global settings with startup scripts. Previously, changing a setting in a startup script would change it globally. #70310 (Antonio Andelic).
• Backported in #70428: Fix ALTER of Dynamic type with reducing max_types parameter that could lead to server crash. #70328 (Pavel Kruglov).
• Backported in #70373: Fix crash when using WITH FILL incorrectly. #70338 (Raúl Marín).
• Backported in #70690: Fix possible use-after-free in SYSTEM DROP FORMAT SCHEMA CACHE FOR Protobuf. #70358 (Azat Khuzhin).
• Backported in #70496: Fix crash during GROUP BY JSON sub-object subcolumn. #70374 (Pavel Kruglov).
• Backported in #70484: Don't prefetch parts for vertical merges if part has no rows. #70452 (Antonio Andelic).
• Backported in #70558: Fix crash in WHERE with lambda functions. #70464 (Raúl Marín).
• Backported in #70922: Fix table creation with CREATE ... AS table_function() with database Replicated and unavailable table function source on secondary replica. #70511 (Kseniia Sumarokova).
• Backported in #70577: Ignore all output on async insert with wait_for_async_insert=1. Closes #62644. #70530 (Konstantin Bogdanov).
• Backported in #71062: Ignore frozen_metadata.txt while traversing shadow directory from system.remote_data_paths. #70590 (Aleksei Filatov).
• Backported in #70653: Fix creation of stateful window functions on misaligned memory. #70631 (Raúl Marín).
• Backported in #70759: Fixed rare crashes in SELECT-s and merges after adding a column of Array type with non-empty default expression. #70695 (Anton Popov).
• Backported in #70764: Fix infinite recursion when infering a proto schema with skip unsupported fields enabled. #70697 (Raúl Marín).
• Backported in #71120: GroupArraySortedData uses a PODArray with non-POD elements, manually calling constructors and destructors for the elements as needed. But it wasn't careful enough: in two places it forgot to call destructor, in one place it left elements uninitialized if an exception is thrown when deserializing previous elements. Then GroupArraySortedData's destructor called destructors on uninitialized elements and crashed: 2024.10.17 22:58:23.523790 [ 5233 ] {} <Fatal> BaseDaemon: ########## Short fault info ############ 2024.10.17 22:58:23.523834 [ 5233 ] {} <Fatal> BaseDaemon: (version 24.6.1.4609 (official build), build id: 5423339A6571004018D55BBE05D464AFA35E6718, git hash: fa6cdfda8a94890eb19bc7f22f8b0b56292f7a26) (from thread 682) Received signal 11 2024.10.17 22:58:23.523862 [ 5233 ] {} <Fatal> BaseDaemon: Signal description: Segmentation fault 2024.10.17 22:58:23.523883 [ 5233 ] {} <Fatal> BaseDaemon: Address: 0x8f. Access: . Address not mapped to object. 2024.10.17 22:58:23.523908 [ 5233 ] {} <Fatal> BaseDaemon: Stack trace: 0x0000aaaac4b78308 0x0000ffffb7701850 0x0000aaaac0104855 0x0000aaaac01048a0 0x0000aaaac501e84c 0x0000aaaac7c510d0 0x0000aaaac7c4ba20 0x0000aaaac968bbfc 0x0000aaaac968fab0 0x0000aaaac969bf50 0x0000aaaac9b7520c 0x0000aaaac9b74c74 0x0000aaaac9b8a150 0x0000aaaac9b809f0 0x0000aaaac9b80574 0x0000aaaac9b8e364 0x0000aaaac9b8e4fc 0x0000aaaac94f4328 0x0000aaaac94f428c 0x0000aaaac94f7df0 0x0000aaaac98b5a3c 0x0000aaaac950b234 0x0000aaaac49ae264 0x0000aaaac49b1dd0 0x0000aaaac49b0a80 0x0000ffffb755d5c8 0x0000ffffb75c5edc 2024.10.17 22:58:23.523936 [ 5233 ] {} <Fatal> BaseDaemon: ######################################## 2024.10.17 22:58:23.523959 [ 5233 ] {} <Fatal> BaseDaemon: (version 24.6.1.4609 (official build), build id: 5423339A6571004018D55BBE05D464AFA35E6718, git hash: fa6cdfda8a94890eb19bc7f22f8b0b56292f7a26) (from thread 682) (query_id: 6c8a33a2-f45a-4a3b-bd71-ded6a1c9ccd3::202410_534066_534078_2) (query: ) Received signal Segmentation fault (11) 2024.10.17 22:58:23.523977 [ 5233 ] {} <Fatal> BaseDaemon: Address: 0x8f. Access: . Address not mapped to object. 2024.10.17 22:58:23.523993 [ 5233 ] {} <Fatal> BaseDaemon: Stack trace: 0x0000aaaac4b78308 0x0000ffffb7701850 0x0000aaaac0104855 0x0000aaaac01048a0 0x0000aaaac501e84c 0x0000aaaac7c510d0 0x0000aaaac7c4ba20 0x0000aaaac968bbfc 0x0000aaaac968fab0 0x0000aaaac969bf50 0x0000aaaac9b7520c 0x0000aaaac9b74c74 0x0000aaaac9b8a150 0x0000aaaac9b809f0 0x0000aaaac9b80574 0x0000aaaac9b8e364 0x0000aaaac9b8e4fc 0x0000aaaac94f4328 0x0000aaaac94f428c 0x0000aaaac94f7df0 0x0000aaaac98b5a3c 0x0000aaaac950b234 0x0000aaaac49ae264 0x0000aaaac49b1dd0 0x0000aaaac49b0a80 0x0000ffffb755d5c8 0x0000ffffb75c5edc 2024.10.17 22:58:23.524817 [ 5233 ] {} <Fatal> BaseDaemon: 0. signalHandler(int, siginfo_t*, void*) @ 0x000000000c6f8308 2024.10.17 22:58:23.524917 [ 5233 ] {} <Fatal> BaseDaemon: 1. ? @ 0x0000ffffb7701850 2024.10.17 22:58:23.524962 [ 5233 ] {} <Fatal> BaseDaemon: 2. DB::Field::~Field() @ 0x0000000007c84855 2024.10.17 22:58:23.525012 [ 5233 ] {} <Fatal> BaseDaemon: 3. DB::Field::~Field() @ 0x0000000007c848a0 2024.10.17 22:58:23.526626 [ 5233 ] {} <Fatal> BaseDaemon: 4. DB::IAggregateFunctionDataHelper<DB::(anonymous namespace)::GroupArraySortedData<DB::Field, (DB::(anonymous namespace)::GroupArraySortedStrategy)0>, DB::(anonymous namespace)::GroupArraySorted<DB::(anonymous namespace)::GroupArraySortedData<DB::Field, (DB::(anonymous namespace)::GroupArraySortedStrategy)0>, DB::Field>>::destroy(char*) const (.5a6a451027f732f9fd91c13f4a13200c) @ 0x000000000cb9e84c 2024.10.17 22:58:23.527322 [ 5233 ] {} <Fatal> BaseDaemon: 5. DB::SerializationAggregateFunction::deserializeBinaryBulk(DB::IColumn&, DB::ReadBuffer&, unsigned long, double) const @ 0x000000000f7d10d0 2024.10.17 22:58:23.528470 [ 5233 ] {} <Fatal> BaseDaemon: 6. DB::ISerialization::deserializeBinaryBulkWithMultipleStreams(COW<DB::IColumn>::immutable_ptr<DB::IColumn>&, unsigned long, DB::ISerialization::DeserializeBinaryBulkSettings&, std::shared_ptr<DB::ISerialization::DeserializeBinaryBulkState>&, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>*) const @ 0x000000000f7cba20 2024.10.17 22:58:23.529213 [ 5233 ] {} <Fatal> BaseDaemon: 7. DB::MergeTreeReaderCompact::readData(DB::NameAndTypePair const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn>&, unsigned long, std::function<DB::ReadBuffer* (DB::ISerialization::SubstreamPath const&)> const&) @ 0x000000001120bbfc 2024.10.17 22:58:23.529277 [ 5233 ] {} <Fatal> BaseDaemon: 8. DB::MergeTreeReaderCompactSingleBuffer::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000001120fab0 2024.10.17 22:58:23.529319 [ 5233 ] {} <Fatal> BaseDaemon: 9. DB::MergeTreeSequentialSource::generate() @ 0x000000001121bf50 2024.10.17 22:58:23.529346 [ 5233 ] {} <Fatal> BaseDaemon: 10. DB::ISource::tryGenerate() @ 0x00000000116f520c 2024.10.17 22:58:23.529653 [ 5233 ] {} <Fatal> BaseDaemon: 11. DB::ISource::work() @ 0x00000000116f4c74 2024.10.17 22:58:23.529679 [ 5233 ] {} <Fatal> BaseDaemon: 12. DB::ExecutionThreadContext::executeTask() @ 0x000000001170a150 2024.10.17 22:58:23.529733 [ 5233 ] {} <Fatal> BaseDaemon: 13. DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x00000000117009f0 2024.10.17 22:58:23.529763 [ 5233 ] {} <Fatal> BaseDaemon: 14. DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x0000000011700574 2024.10.17 22:58:23.530089 [ 5233 ] {} <Fatal> BaseDaemon: 15. DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000001170e364 2024.10.17 22:58:23.530277 [ 5233 ] {} <Fatal> BaseDaemon: 16. DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000001170e4fc 2024.10.17 22:58:23.530295 [ 5233 ] {} <Fatal> BaseDaemon: 17. DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x0000000011074328 2024.10.17 22:58:23.530318 [ 5233 ] {} <Fatal> BaseDaemon: 18. DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x000000001107428c 2024.10.17 22:58:23.530339 [ 5233 ] {} <Fatal> BaseDaemon: 19. DB::MergeTask::execute() @ 0x0000000011077df0 2024.10.17 22:58:23.530362 [ 5233 ] {} <Fatal> BaseDaemon: 20. DB::SharedMergeMutateTaskBase::executeStep() @ 0x0000000011435a3c 2024.10.17 22:58:23.530384 [ 5233 ] {} <Fatal> BaseDaemon: 21. DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x000000001108b234 2024.10.17 22:58:23.530410 [ 5233 ] {} <Fatal> BaseDaemon: 22. ThreadPoolImpl<ThreadFromGlobalPoolImpl<false, true>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false, true>, void*>) @ 0x000000000c52e264 2024.10.17 22:58:23.530448 [ 5233 ] {} <Fatal> BaseDaemon: 23. void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false, true>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false, true>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000000c531dd0 2024.10.17 22:58:23.530476 [ 5233 ] {} <Fatal> BaseDaemon: 24. void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000c530a80 2024.10.17 22:58:23.530514 [ 5233 ] {} <Fatal> BaseDaemon: 25. ? @ 0x000000000007d5c8 2024.10.17 22:58:23.530534 [ 5233 ] {} <Fatal> BaseDaemon: 26. ? @ 0x00000000000e5edc 2024.10.17 22:58:23.530551 [ 5233 ] {} <Fatal> BaseDaemon: Integrity check of the executable skipped because the reference checksum could not be read. 2024.10.17 22:58:23.531083 [ 5233 ] {} <Fatal> BaseDaemon: Report this error to https://github.com/ClickHouse/ClickHouse/issues 2024.10.17 22:58:23.531294 [ 5233 ] {} <Fatal> BaseDaemon: Changed settings: max_insert_threads = 4, max_threads = 42, use_hedged_requests = false, distributed_foreground_insert = true, alter_sync = 0, enable_memory_bound_merging_of_aggregation_results = true, cluster_for_parallel_replicas = 'default', do_not_merge_across_partitions_select_final = false, log_queries = true, log_queries_probability = 1., max_http_get_redirects = 10, enable_deflate_qpl_codec = false, enable_zstd_qat_codec = false, query_profiler_real_time_period_ns = 0, query_profiler_cpu_time_period_ns = 0, max_bytes_before_external_group_by = 90194313216, max_bytes_before_external_sort = 90194313216, max_memory_usage = 180388626432, backup_restore_keeper_retry_max_backoff_ms = 60000, cancel_http_readonly_queries_on_client_close = true, max_table_size_to_drop = 1000000000000, max_partition_size_to_drop = 1000000000000, default_table_engine = 'ReplicatedMergeTree', mutations_sync = 0, optimize_trivial_insert_select = false, database_replicated_allow_only_replicated_engine = true, cloud_mode = true, cloud_mode_engine = 2, distributed_ddl_output_mode = 'none_only_active', distributed_ddl_entry_format_version = 6, async_insert_max_data_size = 10485760, async_insert_busy_timeout_max_ms = 1000, enable_filesystem_cache_on_write_operations = true, load_marks_asynchronously = true, allow_prefetched_read_pool_for_remote_filesystem = true, filesystem_prefetch_max_memory_usage = 18038862643, filesystem_prefetches_limit = 200, compatibility = '24.6', insert_keeper_max_retries = 20, allow_experimental_materialized_postgresql_table = false, date_time_input_format = 'best_effort'. #70820 (Michael Kolupaev).
• Backported in #70897: Disable enable_named_columns_in_function_tuple by default. #70833 (Raúl Marín).
• Backported in #70996: Fix a logical error due to negative zeros in the two-level hash table. This closes #70973. #70979 (Alexey Milovidov).
• Backported in #71212: Fix logical error in StorageS3Queue "Cannot create a persistent node in /processed since it already exists". #70984 (Kseniia Sumarokova).
• Backported in #71249: Fixed named sessions not being closed and hanging on forever under certain circumstances. #70998 (Márcio Martins).
• Backported in #71155: Fix the bug that didn't consider _row_exists column in rebuild option of projection lightweight delete. #71089 (Shichao Jin).
• Backported in #71353: Fix possible error "Cannot read all data" erros during deserialization of LowCardinality dictionary from Dynamic column. #71299 (Pavel Kruglov).
• Backported in #71464: Added missing unescaping in named collections. Without fix clickhouse-server can't start. #71308 (MikhailBurdukov).
• Backported in #71329: Fix async inserts with empty blocks via native protocol. #71312 (Anton Popov).
• Backported in #71377: Add try/catch to data parts destructors to avoid terminate. #71364 (alesapin).
• Backported in #71749: Check suspicious and experimental types in JSON type hints. #71369 (Pavel Kruglov).
• Backported in #71449: Start memory worker thread on non-Linux OS too (fixes #71051). #71384 (Alexandre Snarskii).
• Backported in #71606: Fix error Invalid number of rows in Chunk with Variant column. #71388 (Pavel Kruglov).
• Backported in #71823: Fix crash with optimize_rewrite_array_exists_to_has. #71432 (Raúl Marín).
• Backported in #71519: Fix possible error Argument for function must be constant (old analyzer) in case when arrayJoin can apparently appear in WHERE condition. Regression after https://github.com/ClickHouse/ClickHouse/pull/65414. #71476 (Nikolai Kochetov).
• Backported in #71553: Prevent crash in SortCursor with 0 columns (old analyzer). #71494 (Raúl Marín).
• Backported in #71568: Avoid crash when using a UDF in a constraint. #71541 (Raúl Marín).
• Backported in #71728: Return 0 or default char instead of throwing an error in bitShift functions in case of out of bounds. #71580 (Pablo Marcos).
• Backported in #71878: Fix LOGICAL_ERROR when doing ALTER with empty tuple. This fixes #71647. #71679 (Amos Bird).
• Backported in #71739: Don't transform constant set in predicates over partition columns in case of NOT IN operator. #71695 (Eduard Karacharov).
• Backported in #72004: Fix a crash in clickhouse-client syntax highlighting. Closes #71864. #71949 (Nikolay Degterinsky).

Build/Testing/Packaging Improvement

• Backported in #71691: Improve clickhouse-server Dockerfile.ubuntu. Deprecate CLICKHOUSE_UID/CLICKHOUSE_GID envs. Remove CLICKHOUSE_DOCKER_RESTART_ON_EXIT processing to complien requirements. Consistent clickhouse/clickhouse-server/clickhouse-keeper execution to not have it plain in one place and /usr/bin/clickhouse* in another. #71573 (Mikhail f. Shiryaev).

NO CL ENTRY

• NO CL ENTRY: 'Revert "Backport #70146 to 24.9: Upgrade integration-runner image"'. #70321 (Max Kainov).

NOT FOR CHANGELOG / INSIGNIFICANT

• Backported in #70316: CI: Remove await feature from release branches. #70294 (Max Kainov).
• Backported in #70390: CI: Enable Integration Tests for backport PRs. #70329 (Max Kainov).
• Backported in #70705: Don't fail the stateless check script if we can't collect minio logs. #70350 (Raúl Marín).
• Backported in #70382: Fix tiny mistake, responsible for some of kafka test flaps. Example report. #70352 (filimonov).
• Backported in #70407: Closes #69634. #70354 (pufit).
• Backported in #70703: Fix order in 03249_dynamic_alter_consistency. #70453 (Alexander Gololobov).
• Backported in #70544: Remove slow poll() logs in keeper. #70508 (Raúl Marín).
• Backported in #70692: Fix 24.9 setting compatibility database_replicated_allow_explicit_uuid. #70565 (Nikita Fomichev).
• Backported in #70683: Increase max_rows_to_read limit in some tests. #70617 (Raúl Marín).
• Backported in #70805: When the PR Check status is set, it's a valid RunConfig job failure. #70643 (Mikhail f. Shiryaev).
• Backported in #71762: Remove bad test test_system_replicated_fetches. #71071 (Alexey Milovidov).
• Backported in #71230: Maybe not GWPAsan by default. #71174 (Antonio Andelic).
• Backported in #71435: Ignore No such key exceptions in some cases. #71236 (Antonio Andelic).
• Backported in #71803: Fix issues we face on orphane backport branches and closed release PRs, when fake-master events are sent to the check DB. #71782 (Mikhail f. Shiryaev).
• Backported in #71838: The change has already been applied to https://github.com/docker-library/official-images/pull/17876. Backport it to every branch to have a proper Dockerfile.ubuntu there. #71825 (Mikhail f. Shiryaev).