thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-02 12:32:04 +00:00
Code Issues Packages Projects Releases Wiki Activity
133,563 Commits 9,194 Branches 1,782 Tags 2.2 GiB
50532c485f
Commit Graph

52451 Commits

Author SHA1 Message Date
Alexey Milovidov
50532c485f
Merge pull request #59340 from CurtizJ/lower-memory-usage-vertical-merges 
Lower memory usage in vertical merges
 2024-01-30 03:42:28 +01:00
Raúl Marín
dfc761ce5f
Merge pull request #59343 from ClickHouse/Fix_new_setting_for_azure 
Fix upgrade check for new setting
 2024-01-29 23:22:06 +01:00
Yarik Briukhovetskyi
5ba82eed81
Merge pull request #58841 from yariks5s/dictionary_table_exception_fix 
Fixing exceptions when table/dictionary already exist
 2024-01-29 22:41:06 +01:00
Raúl Marín
1580f73795
Update SettingsChangesHistory.h 2024-01-29 21:53:05 +01:00
Raúl Marín
45776ea035
Update SettingsChangesHistory.h 2024-01-29 21:24:20 +01:00
Raúl Marín
5108393157 Merge remote-tracking branch 'blessed/master' into Fix_new_setting_for_azure 2024-01-29 21:09:00 +01:00
Alexey Milovidov
69a1935a3e
Merge pull request #59345 from Algunenano/sccache_fix 
Fix sccache when building without coverage
 2024-01-29 21:06:29 +01:00
Robert Schulze
7683b44bf6
Merge pull request #58961 from ClibMouse/seriesDecompose 
Return baseline component from function `seriesDecomposeSTL()`
 2024-01-29 20:41:09 +01:00
Robert Schulze
2cfcaf8ff4
Merge pull request #59040 from MochiXu/fix_remove_inverted_index_files 
`DROP INDEX` of inverted index now removes all relevant files from persistence
 2024-01-29 20:39:56 +01:00
Robert Schulze
8ddda0caf0
Merge pull request #59229 from rschu1ze/cleanup-msan-usage 
Minor cleanup of msan usage
 2024-01-29 20:39:06 +01:00
Robert Schulze
80066118d9
Merge pull request #59132 from HarryLeeIBM/hlee-arm64-hash 
Fix aggregation issue in mixed x86_64 and ARM clusters
 2024-01-29 20:37:52 +01:00
Raúl Marín
925582bec2
Merge branch 'master' into Fix_new_setting_for_azure 2024-01-29 18:27:22 +01:00
Raúl Marín
f22762fcdc
Merge pull request #59335 from kitaisreal/volume-constructor-improve-exception-message 
IVolume constructor improve exception message
 2024-01-29 18:25:57 +01:00
Maksim Kita
7892adc18b Add setting update_insert_deduplication_token_in_dependent_materialized_views to settings changes history 2024-01-29 20:18:25 +03:00
Raúl Marín
b88d0a2b15 Don't apply O2 on release builds 2024-01-29 17:34:11 +01:00
Raúl Marín
468c7cf835
Merge pull request #59329 from Algunenano/prefertch 
Fix typo
 2024-01-29 17:02:01 +01:00
Smita Kulkarni
621c3e8596 Fix upgrade check for new setting 2024-01-29 16:50:01 +01:00
Nikolai Kochetov
5b2648929b
Merge pull request #59273 from kitaisreal/atomic-logger 
Added AtomicLogger
 2024-01-29 16:49:18 +01:00
Anton Popov
d3cdc8826e lower memory usage in vertical merges 2024-01-29 14:06:46 +00:00
Maksim Kita
5affd6af2e IVolume constructor improve exception message 2024-01-29 15:32:22 +03:00
Nikolai Kochetov
a51aa7b668
Merge pull request #59238 from kitaisreal/update-insert-deduplication-token-in-dependent-materialized-views 
Added setting update_insert_deduplication_token_in_dependent_materialized_views
 2024-01-29 12:52:06 +01:00
Raúl Marín
ab2b12ba9f Fix typo 2024-01-29 11:40:12 +01:00
Kruglov Pavel
6858d2f4ca
Merge pull request #58047 from Avogar/variant-data-type 
Implement Variant data type
 2024-01-29 11:36:08 +01:00
Kseniia Sumarokova
6903957414
Merge pull request #59001 from ClickHouse/Add_configurable_write_retry_azure 
Add settings max_unexpected_write_error_retries for Azure Blob Storage
 2024-01-29 10:48:17 +01:00
Kseniia Sumarokova
a29ab8f29c
Merge pull request #59170 from azat/disks/s3-plain-improvements 
Small improvements for tables on write-once (s3_plain)/read-only (web) disks
 2024-01-29 10:44:37 +01:00
Kseniia Sumarokova
0f88da5fd4
Merge pull request #59275 from ClickHouse/Avogar-patch-1 
Fix abort in iceberg metadata on bad file paths
 2024-01-29 10:35:34 +01:00
Alexey Milovidov
2cb2bcfbc3
Merge pull request #59315 from ClickHouse/support-backups-for-compressed-memory-tables 
Support backups for compressed in-memory tables
 2024-01-29 01:50:41 +01:00
Alexey Milovidov
a7483ec10b
Merge pull request #59242 from rschu1ze/dont-force-libssh-on-me 
Make libssh build dependent on `-DENABLE_LIBRARIES`
 2024-01-29 00:02:03 +01:00
Alexey Milovidov
caded18be5
Merge pull request #59290 from CurtizJ/lower-memory-usage 
Lower memory usage after reading from `MergeTree` tables
 2024-01-29 00:00:02 +01:00
Alexey Milovidov
aec3f28ccb Support backups for compressed in-memory tables 2024-01-28 23:06:50 +01:00
Alexey Milovidov
8326f5ffb7
Merge pull request #59308 from ClickHouse/fix-async-loader-stop 
Fix deadlock in `AsyncLoader::stop()`
 2024-01-28 22:18:00 +01:00
serxa
aa6c7e78be fix deadlock in AsyncLoader::stop() 2024-01-28 15:47:17 +00:00
Anton Popov
ef133bfd47 Merge remote-tracking branch 'upstream/master' into lower-memory-usage 2024-01-28 15:07:44 +00:00
avogar
92db86b829 Merge branch 'master' of github.com:ClickHouse/ClickHouse into variant-data-type 2024-01-28 14:33:49 +00:00
Kruglov Pavel
29700fbad1
Fix special build 2024-01-28 15:33:27 +01:00
Alexey Milovidov
267b077038 Merge remote-tracking branch 'origin/master' into remove-kql 2024-01-28 14:24:26 +01:00
Alexey Milovidov
e0f0b21e90 Remove KQL 2024-01-28 14:24:13 +01:00
Alexey Milovidov
29d54dab55
Merge pull request #59288 from azat/rmt/drop-update-zk 
Use fresh ZooKeeper client on DROP (to have higher chances on success)
 2024-01-28 13:16:21 +01:00
Maksim Kita
5165fb7286 Fixed build 2024-01-28 13:55:44 +03:00
Maksim Kita
50309dfd90 Fixed tests 2024-01-28 13:55:44 +03:00
Maksim Kita
07750ee949 Added AtomicLogger 2024-01-28 13:55:44 +03:00
Smita Kulkarni
b7a6dbfbb9 Check engine to avoid possible seg fault 2024-01-27 18:22:37 +01:00
Anton Popov
9d3b4e5308 lower memory usage 2024-01-27 16:40:11 +00:00
Azat Khuzhin
d94547d338 Use fresh ZooKeeper client on DROP (to have higher chances on success) 
In case of DROP the client can be expired, and even though
StorageReplicatedMergeTree::drop() uses getZooKeeperIfTableShutDown(),
which creates new client if current is expired, it is not enough, since
current_zookeeper (cached client) will be used implicitly from
dropAllData().

This had been found by stress tests, that leads to DROP query hang [1].

  [1]: https://s3.amazonaws.com/clickhouse-test-reports/59255/94eb33ef27a9ab7c4a99af40772ea287e67efcbf/stress_test__tsan_.html

<details>

<summary>stacktrace</summary>

    2024.01.26 17:41:41.138577 [ 3319 ] {} <Error> DatabaseCatalog: Cannot drop table test_vzfk1xoc.alter_table1 (455a674c-161d-44d8-abc2-fd205bad1116). Will retry later.: Code: 999. Coordination::Exception: Session expired. (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below):

    0. ./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x000000001e06a5e3 in /usr/bin/clickhouse
    1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000f697c74 in /usr/bin/clickhouse
    2. ./src/Common/Exception.h:0: Coordination::Exception::Exception<char const (&) [16]>(char const (&) [16], Coordination::Error) @ 0x000000001adc298b in /usr/bin/clickhouse
    3. ./src/Common/ZooKeeper/IKeeper.h:0: Coordination::ZooKeeper::pushRequest(Coordination::ZooKeeper::RequestInfo&&) @ 0x000000001adfb418 in /usr/bin/clickhouse
    4. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:1343: Coordination::ZooKeeper::get(String const&, std::function<void (Coordination::GetResponse const&)>, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>) @ 0x000000001adfd8e5 in /usr/bin/clickhouse
    5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001ad890bd in /usr/bin/clickhouse
    6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001ad88a36 in /usr/bin/clickhouse
    7. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:580: zkutil::ZooKeeper::tryGetWatch(String const&, String&, Coordination::Stat*, std::function<void (Coordination::WatchResponse const&)>, Coordination::Error*) @ 0x000000001ad898bf in /usr/bin/clickhouse
    8. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:570: zkutil::ZooKeeper::tryGet(String const&, String&, Coordination::Stat*, std::shared_ptr<Poco::Event> const&, Coordination::Error*) @ 0x000000001ad89554 in /usr/bin/clickhouse
    9. ./build_docker/./src/Common/ZooKeeper/ZooKeeperWithFaultInjection.cpp:0: DB::ZooKeeperWithFaultInjection::tryGet(String const&, String&, Coordination::Stat*, std::shared_ptr<Poco::Event> const&, Coordination::Error*) @ 0x000000001ae110dc in /usr/bin/clickhouse
    10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::StorageReplicatedMergeTree::unlockSharedDataByID(String, String const&, DB::MergeTreePartInfo const&, String const&, String const&, std::shared_ptr<DB::ZooKeeperWithFaultInjection> const&, DB::MergeTreeSettings const&, std::shared_ptr<Poco::Logger>, String const&, StrongTypedef<unsigned int, DB::MergeTreeDataFormatVersionTag>) @ 0x000000001975527e in /usr/bin/clickhouse
    11. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::removeSharedDetachedPart(std::shared_ptr<DB::IDisk>, String const&, String const&, String const&, String const&, String const&, std::shared_ptr<DB::Context const> const&, std::shared_ptr<zkutil::ZooKeeper> const&) @ 0x000000001976c980 in /usr/bin/clickhouse
    12. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:10171: DB::StorageReplicatedMergeTree::removeDetachedPart(std::shared_ptr<DB::IDisk>, String const&, String const&) @ 0x000000001976bdc6 in /usr/bin/clickhouse
    13. ./build_docker/./src/Storages/MergeTree/MergeTreeData.cpp:2798: DB::MergeTreeData::dropAllData() @ 0x0000000019c643c8 in /usr/bin/clickhouse
    14. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:1117: DB::StorageReplicatedMergeTree::drop() @ 0x0000000019664e0a in /usr/bin/clickhouse
    15. ./build_docker/./src/Interpreters/DatabaseCatalog.cpp:0: DB::DatabaseCatalog::dropTableFinally(DB::DatabaseCatalog::TableMarkedAsDropped const&) @ 0x0000000017bf1fac in /usr/bin/clickhouse
    16. ./build_docker/./src/Interpreters/DatabaseCatalog.cpp:0: DB::DatabaseCatalog::dropTableDataTask() @ 0x0000000017bf175e in /usr/bin/clickhouse
    17. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000017bf94c2 in /usr/bin/clickhouse
    18. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x00000000170bd0b8 in /usr/bin/clickhouse
    19. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000170c06ce in /usr/bin/clickhouse
    20. ./build_docker/./src/Core/BackgroundSchedulePool.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000170c0feb in /usr/bin/clickhouse
    21. ./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000000f7c6a2f in /usr/bin/clickhouse
    22. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000f7cc8f2 in /usr/bin/clickhouse
    23. ? @ 0x000000000733904f in /usr/bin/clickhouse
    24. ? @ 0x00007f28b2319ac3
    25. ? @ 0x00007f28b23ab850

</details>

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2024-01-27 16:04:18 +01:00
Maksim Kita
0f46200d5a Fixed tests 2024-01-27 15:57:49 +03:00
Maksim Kita
5d62cdd58c Added setting update_insert_deduplication_token_in_dependent_materialized_views 2024-01-27 15:57:48 +03:00
Kruglov Pavel
1e61521dcc
Add missing setting optimize_injective_functions_in_group_by to SettingsChangesHistory 2024-01-26 23:33:04 +01:00
avogar
5833641fa5 Merge branch 'master' of github.com:ClickHouse/ClickHouse into variant-data-type 2024-01-26 16:54:02 +00:00
Kruglov Pavel
8fcedddcda
Fix abort in iceberg metadata on bad file paths 2024-01-26 17:46:49 +01:00
Azat Khuzhin
90ab986be6 Revert "Prohibit CREATE on static storages (ATTACH should be used instead)" 
CREATE is used by Replicated database even for ATTACH, and anyway ATTACH
creates format_version.txt as well (without previuos patch).

This reverts commit 323f1f320ebe7e588d443abf78efa2c60193e7a9.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2024-01-26 16:47:36 +01:00