thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-11-13 02:53:38 +00:00
Code Issues Packages Projects Releases Wiki Activity
133,616 Commits 9,116 Branches 1,774 Tags 2 GiB
2ad7607bad
Commit Graph

133616 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Alexey Milovidov
9dd6362ae7 Play UI: improve rendering of errors from JSON formats 2024-01-28 13:44:56 +01:00
Kseniia Sumarokova
febd4ce8d7
Update run.sh 2024-01-28 13:41:33 +01:00
Alexey Milovidov
29d54dab55
Merge pull request #59288 from azat/rmt/drop-update-zk 
Use fresh ZooKeeper client on DROP (to have higher chances on success)
 2024-01-28 13:16:21 +01:00
Alexey Milovidov
7a5dddf202
Merge pull request #59292 from ClickHouse/Fix_possible_seg_fault_check_engine 
Fix possible seg fault
 2024-01-28 13:14:15 +01:00
alesapin
d70fe79a1e Revert "Disable randomization of allow_experimental_block_number_column flag (#56474)" 
This reverts commit 951d9ac144.
 2024-01-28 12:28:08 +01:00
Maksim Kita
5165fb7286 Fixed build 2024-01-28 13:55:44 +03:00
Maksim Kita
bccd8b5e27 Fixed tests 2024-01-28 13:55:44 +03:00
Maksim Kita
50309dfd90 Fixed tests 2024-01-28 13:55:44 +03:00
Maksim Kita
07750ee949 Added AtomicLogger 2024-01-28 13:55:44 +03:00
robot-ch-test-poll4
c0fc804797
Merge pull request #59289 from ClickHouse/melvynator-exp-rmv 
Update view.md
 2024-01-28 00:17:56 +01:00
robot-clickhouse
ac6d2c4f34
Merge pull request #59281 from ClickHouse/fix-doc-remove-ex-flag 
Update view.md
 2024-01-28 00:17:16 +01:00
Alexey Milovidov
c7289f69ed
Merge pull request #59287 from azat/tests/fix-perf 
Fix perf tests (after sumMap starts to filter out -0.)
 2024-01-27 22:56:32 +01:00
Kseniia Sumarokova
24ab61932e
Update run.sh 2024-01-27 22:42:43 +01:00
alesapin
7022ca9ece Fix _block_number column with non-default serializations and mutation 2024-01-27 20:24:03 +01:00
slvrtrn
7d785c47e2 Add drop table 2024-01-27 18:45:57 +01:00
slvrtrn
eb909a5db4 Reapply add MySQL net write/read settings 2024-01-27 18:36:38 +01:00
Smita Kulkarni
b7a6dbfbb9 Check engine to avoid possible seg fault 2024-01-27 18:22:37 +01:00
Anton Popov
9d3b4e5308 lower memory usage 2024-01-27 16:40:11 +00:00
Peignon Melvyn
cf1df96cc1
Update view.md 
Improved documentation.
 2024-01-27 17:27:00 +01:00
Azat Khuzhin
d94547d338 Use fresh ZooKeeper client on DROP (to have higher chances on success) 
In case of DROP the client can be expired, and even though
StorageReplicatedMergeTree::drop() uses getZooKeeperIfTableShutDown(),
which creates new client if current is expired, it is not enough, since
current_zookeeper (cached client) will be used implicitly from
dropAllData().

This had been found by stress tests, that leads to DROP query hang [1].

  [1]: https://s3.amazonaws.com/clickhouse-test-reports/59255/94eb33ef27a9ab7c4a99af40772ea287e67efcbf/stress_test__tsan_.html

<details>

<summary>stacktrace</summary>

    2024.01.26 17:41:41.138577 [ 3319 ] {} <Error> DatabaseCatalog: Cannot drop table test_vzfk1xoc.alter_table1 (455a674c-161d-44d8-abc2-fd205bad1116). Will retry later.: Code: 999. Coordination::Exception: Session expired. (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below):

    0. ./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x000000001e06a5e3 in /usr/bin/clickhouse
    1. ./build_docker/./src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000f697c74 in /usr/bin/clickhouse
    2. ./src/Common/Exception.h:0: Coordination::Exception::Exception<char const (&) [16]>(char const (&) [16], Coordination::Error) @ 0x000000001adc298b in /usr/bin/clickhouse
    3. ./src/Common/ZooKeeper/IKeeper.h:0: Coordination::ZooKeeper::pushRequest(Coordination::ZooKeeper::RequestInfo&&) @ 0x000000001adfb418 in /usr/bin/clickhouse
    4. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:1343: Coordination::ZooKeeper::get(String const&, std::function<void (Coordination::GetResponse const&)>, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>) @ 0x000000001adfd8e5 in /usr/bin/clickhouse
    5. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001ad890bd in /usr/bin/clickhouse
    6. ./contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001ad88a36 in /usr/bin/clickhouse
    7. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:580: zkutil::ZooKeeper::tryGetWatch(String const&, String&, Coordination::Stat*, std::function<void (Coordination::WatchResponse const&)>, Coordination::Error*) @ 0x000000001ad898bf in /usr/bin/clickhouse
    8. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:570: zkutil::ZooKeeper::tryGet(String const&, String&, Coordination::Stat*, std::shared_ptr<Poco::Event> const&, Coordination::Error*) @ 0x000000001ad89554 in /usr/bin/clickhouse
    9. ./build_docker/./src/Common/ZooKeeper/ZooKeeperWithFaultInjection.cpp:0: DB::ZooKeeperWithFaultInjection::tryGet(String const&, String&, Coordination::Stat*, std::shared_ptr<Poco::Event> const&, Coordination::Error*) @ 0x000000001ae110dc in /usr/bin/clickhouse
    10. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::StorageReplicatedMergeTree::unlockSharedDataByID(String, String const&, DB::MergeTreePartInfo const&, String const&, String const&, std::shared_ptr<DB::ZooKeeperWithFaultInjection> const&, DB::MergeTreeSettings const&, std::shared_ptr<Poco::Logger>, String const&, StrongTypedef<unsigned int, DB::MergeTreeDataFormatVersionTag>) @ 0x000000001975527e in /usr/bin/clickhouse
    11. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:0: DB::StorageReplicatedMergeTree::removeSharedDetachedPart(std::shared_ptr<DB::IDisk>, String const&, String const&, String const&, String const&, String const&, std::shared_ptr<DB::Context const> const&, std::shared_ptr<zkutil::ZooKeeper> const&) @ 0x000000001976c980 in /usr/bin/clickhouse
    12. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:10171: DB::StorageReplicatedMergeTree::removeDetachedPart(std::shared_ptr<DB::IDisk>, String const&, String const&) @ 0x000000001976bdc6 in /usr/bin/clickhouse
    13. ./build_docker/./src/Storages/MergeTree/MergeTreeData.cpp:2798: DB::MergeTreeData::dropAllData() @ 0x0000000019c643c8 in /usr/bin/clickhouse
    14. ./build_docker/./src/Storages/StorageReplicatedMergeTree.cpp:1117: DB::StorageReplicatedMergeTree::drop() @ 0x0000000019664e0a in /usr/bin/clickhouse
    15. ./build_docker/./src/Interpreters/DatabaseCatalog.cpp:0: DB::DatabaseCatalog::dropTableFinally(DB::DatabaseCatalog::TableMarkedAsDropped const&) @ 0x0000000017bf1fac in /usr/bin/clickhouse
    16. ./build_docker/./src/Interpreters/DatabaseCatalog.cpp:0: DB::DatabaseCatalog::dropTableDataTask() @ 0x0000000017bf175e in /usr/bin/clickhouse
    17. ./contrib/llvm-project/libcxx/include/__functional/function.h:717: ? @ 0x0000000017bf94c2 in /usr/bin/clickhouse
    18. ./contrib/llvm-project/libcxx/include/__functional/function.h:0: ? @ 0x00000000170bd0b8 in /usr/bin/clickhouse
    19. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000170c06ce in /usr/bin/clickhouse
    20. ./build_docker/./src/Core/BackgroundSchedulePool.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000170c0feb in /usr/bin/clickhouse
    21. ./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000000f7c6a2f in /usr/bin/clickhouse
    22. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000f7cc8f2 in /usr/bin/clickhouse
    23. ? @ 0x000000000733904f in /usr/bin/clickhouse
    24. ? @ 0x00007f28b2319ac3
    25. ? @ 0x00007f28b23ab850

</details>

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2024-01-27 16:04:18 +01:00
Azat Khuzhin
71cd1efd8b tests: fix 02980_s3_plain_DROP_TABLE_ReplicatedMergeTree flakiness 
In case of commit failed (likely due to fault injection in tests) part
name will have different name, not all_0_0_0, so replace numbers with X.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2024-01-27 15:19:21 +01:00
Alexey Milovidov
661d4a827b
Merge pull request #59272 from azat/osx-rust 
Enable Rust on OSX ARM64
 2024-01-27 14:29:32 +01:00
Maksim Kita
7d48d0da72 Fixed tests 2024-01-27 15:57:49 +03:00
Maksim Kita
0f46200d5a Fixed tests 2024-01-27 15:57:49 +03:00
Maksim Kita
4a10b71252 Updated documentation 2024-01-27 15:57:49 +03:00
Maksim Kita
4ae425461f Added documentation 2024-01-27 15:57:49 +03:00
Maksim Kita
5d62cdd58c Added setting update_insert_deduplication_token_in_dependent_materialized_views 2024-01-27 15:57:48 +03:00
Azat Khuzhin
44e42052b1 Fix perf tests after sumMap starts to filter out -0. 
Before perf tests was relying on the following:

    SELECT sumMap(['foo', 'bar'], [-0., -0.])

    ┌─sumMap(['foo', 'bar'], [-0., -0.])─┐
    │ (['bar','foo'],[-0,-0])            │
    └────────────────────────────────────┘

While it got changed, and now:

    ┌─sumMap(['foo', 'bar'], [-0., -0.])─┐
    │ ([],[])                            │
    └────────────────────────────────────┘

But it works for nan:

    SELECT sumMap(['foo', 'bar'], [nan, nan])

    ┌─sumMap(['foo', 'bar'], [nan, nan])─┐
    │ (['bar','foo'],[nan,nan])          │
    └────────────────────────────────────┘

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2024-01-27 13:14:07 +01:00
Kruglov Pavel
875efc54b8
Merge pull request #59283 from ClickHouse/Avogar-patch-2 
Add missing setting optimize_injective_functions_in_group_by to SettingsChangesHistory
 2024-01-27 00:12:10 +01:00
Kruglov Pavel
1e61521dcc
Add missing setting optimize_injective_functions_in_group_by to SettingsChangesHistory 2024-01-26 23:33:04 +01:00
Peignon Melvyn
4273ae4f4f
Update view.md 2024-01-26 21:27:20 +01:00
Kseniia Sumarokova
163a3e44d3
Update run.sh 2024-01-26 21:22:39 +01:00
robot-ch-test-poll2
930ce11ae4
Merge pull request #59261 from artbeglaryan/patch-3 
Fixed typo in documentation
 2024-01-26 22:41:04 +04:00
avogar
5833641fa5 Merge branch 'master' of github.com:ClickHouse/ClickHouse into variant-data-type 2024-01-26 16:54:02 +00:00
Kruglov Pavel
8fcedddcda
Fix abort in iceberg metadata on bad file paths 2024-01-26 17:46:49 +01:00
vdimir
6795d0207f
Update test_reload_after_fail_in_cache_dictionary for analyzer 2024-01-26 16:37:54 +00:00
Arthur Passos
d8deebaaba
Merge branch 'ClickHouse:master' into bring_back_attach_partition_pr 2024-01-26 13:27:57 -03:00
Azat Khuzhin
50f0a34ecb Enable Rust on OSX 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2024-01-26 17:20:00 +01:00
Azat Khuzhin
5227d7c9bc Fix 02475_bson_each_row_format flakiness (due to small parsing block) 
CI: https://s3.amazonaws.com/clickhouse-test-reports/59170/c82050d1925439f0ede2b32acb5b1b8df4acae5d/stateless_tests__release_/run.log
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2024-01-26 16:51:37 +01:00
Azat Khuzhin
332924a2b4 tests: adjust 02980_s3_plain_DROP_TABLE tests 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2024-01-26 16:47:36 +01:00
Azat Khuzhin
90ab986be6 Revert "Prohibit CREATE on static storages (ATTACH should be used instead)" 
CREATE is used by Replicated database even for ATTACH, and anyway ATTACH
creates format_version.txt as well (without previuos patch).

This reverts commit 323f1f320ebe7e588d443abf78efa2c60193e7a9.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2024-01-26 16:47:36 +01:00
Azat Khuzhin
7ba31bf55c Do not create format_version.txt on write-once storages 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2024-01-26 16:47:36 +01:00
Azat Khuzhin
7cc7fb892c tests/test_disk_over_web_server: update expected assertion for static storage 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2024-01-26 16:47:36 +01:00
Azat Khuzhin
66a40bda0a Revert "Prohibit DROP from static storages" 
It is actually OK to DROP at least from read-only disk, as for the
write-once it should be OK as well, since CREATE is prohibited anyway,
so there should be no leftovers

CI: https://s3.amazonaws.com/clickhouse-test-reports/59170/c82050d1925439f0ede2b32acb5b1b8df4acae5d/stress_test__asan_.html
CI: https://s3.amazonaws.com/clickhouse-test-reports/59170/c82050d1925439f0ede2b32acb5b1b8df4acae5d/integration_tests__asan__[3_4].html

This reverts commit 082e62712c5e2f4c48d1abc6141ccfd1249d26b4.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2024-01-26 16:47:35 +01:00
Azat Khuzhin
d99fb4ce5b tests: rewrite test for MergeTree engines in s3_plain using ATTACH 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2024-01-26 16:47:35 +01:00
Azat Khuzhin
aafc149c1e Prohibit CREATE on static storages (ATTACH should be used instead) 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2024-01-26 16:47:35 +01:00
Azat Khuzhin
1ee723d50d Prohibit DROP from static storages 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2024-01-26 16:47:35 +01:00
Azat Khuzhin
046a7a5df5 tests: cover DROP TABLE on s3_plain disk 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2024-01-26 16:47:35 +01:00
Azat Khuzhin
d79c1f7e1c Fix MetadataStorageFromPlainObjectStorage::iterateDirectory() (add prefix) 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2024-01-26 16:47:35 +01:00
Azat Khuzhin
9ce18bbf43 Implement MetadataStorageFromPlainObjectStorageTransaction::removeDirectory (for DROP TABLE) 
This is required to remove format_version.txt for MergeTree after DROP

    (lldb) bt
    * thread 192, name = 'BgSchPool', stop reason = breakpoint 1.1
        frame 0: 0x0000000016c7f452 clickhouse`__cxa_throw(thrown_object=0x00007ffd7a53c480, tinfo=0x00000000011602a0, dest=(clickhouse`DB::Exception::~Exception() at Exception.h:40)) + 18 at cxa_exception.cpp:258
        frame 1: 0x00000000109bf709 clickhouse`DB::IMetadataTransaction::throwNotImplemented() + 137 at IMetadataStorage.h:151
        frame 2: 0x00000000109bf126 clickhouse`DB::IMetadataTransaction::removeDirectory(this=<unavailable>, (null)=<unavailable>) + 6 at IMetadataStorage.h:96
        frame 3: 0x0000000010968978 clickhouse`DB::(anonymous namespace)::RemoveRecursiveObjectStorageOperation::removeMetadataRecursive(this=0x00007ffd7a7327a0, tx=std::__1::shared_ptr<DB::IMetadataTransaction>::element_type @ 0x00007ffd7a6f6018 strong=4 weak=1, path_to_remove="tmp_insert_all_2_2_0") + 376 at DiskObjectStorageTransaction.cpp:364
        frame 4: 0x000000001096893d clickhouse`DB::(anonymous namespace)::RemoveRecursiveObjectStorageOperation::removeMetadataRecursive(this=0x00007ffd7a7327a0, tx=std::__1::shared_ptr<DB::IMetadataTransaction>::element_type @ 0x00007ffd7a6f6018 strong=4 weak=1, path_to_remove="store/27f/27f6f9d2-ab2e-4817-b771-0e706e485b15/") + 317 at DiskObjectStorageTransaction.cpp:362
        frame 5: 0x0000000010965a7f clickhouse`DB::(anonymous namespace)::RemoveRecursiveObjectStorageOperation::execute(this=0x00007ffd7a7327a0, tx=std::__1::shared_ptr<DB::IMetadataTransaction>::element_type @ 0x00007ffd7a6f6018 strong=4 weak=1) + 95 at DiskObjectStorageTransaction.cpp:372
        frame 6: 0x0000000010963a2f clickhouse`DB::DiskObjectStorageTransaction::commit(this=0x00007ffea22b6168) + 79 at DiskObjectStorageTransaction.cpp:903
        frame 7: 0x00000000109577e5 clickhouse`DB::DiskObjectStorage::removeSharedRecursive(this=<unavailable>, path="store/27f/27f6f9d2-ab2e-4817-b771-0e706e485b15/", keep_all_batch_data=false, file_names_remove_metadata_only=size=0) + 133 at DiskObjectStorage.cpp:426
        frame 8: 0x000000001095af79 clickhouse`DB::DiskObjectStorage::removeRecursive(this=<unavailable>, path=<unavailable>) + 57 at DiskObjectStorage.h:79
      * frame 9: 0x0000000010b5c3d5 clickhouse`DB::DatabaseCatalog::dropTableFinally(this=0x00007ffff7096800, table=0x00007ffe853e1bd0) + 1365 at DatabaseCatalog.cpp:1317
        frame 10: 0x0000000010b5b7e7 clickhouse`DB::DatabaseCatalog::dropTableDataTask(this=0x00007ffff7096800) + 2375 at DatabaseCatalog.cpp:1270

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2024-01-26 16:47:35 +01:00