thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-09-20 08:40:50 +00:00
Code Issues Packages Projects Releases Wiki Activity
149,319 Commits 8,927 Branches 1,765 Tags 2 GiB
995187006a
Commit Graph

149319 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Sema Checherinda
995187006a rework custom table's disk usage 2024-08-06 13:05:23 +02:00
robot-clickhouse
5d9d5bf919 Automatic style fix 2024-08-05 16:34:55 +02:00
Sema Checherinda
462fe8b7ff repro for #67478 2024-08-05 16:34:55 +02:00
Alexey Milovidov
a3b73e94aa
Merge pull request #67756 from ClickHouse/fix-stackoverflow-json-merge-patch 
Fix stack overflow in JSONMergePatch function
 2024-08-05 12:28:16 +00:00
Alexey Milovidov
ba3099015f
Merge pull request #67779 from ClickHouse/fix-session-log-tests 
Fix test `02833_concurrent_sessions`, Fix test `02835_drop_user_during_session`
 2024-08-05 12:28:02 +00:00
Alexey Milovidov
5875900189
Merge pull request #67780 from ClickHouse/fix-small-diagnostic-in-test 
Fix diagnostics in the test script
 2024-08-05 12:27:37 +00:00
Alexey Milovidov
e1e17b91c5
Merge pull request #67776 from azat/debug_ast_formatting_print_original_query 
[RFC] Print original query for AST formatting check on CI
 2024-08-05 12:26:54 +00:00
Alexey Milovidov
f601b2870e
Merge pull request #65978 from azat/keeper-config-cleanup 
Remove default values for certificateFile/privateKeyFile/dhParamsFile in keeper config (to avoid annoying errors in logs)
 2024-08-05 12:25:48 +00:00
Alexey Milovidov
6b4353b910
Merge pull request #66495 from azat/ci-images-llvm-cleanup 
Ensure that llvm-symbolizer is used for symbolizing sanitizer reports
 2024-08-05 12:25:32 +00:00
Alexey Milovidov
243eb515d6
Merge pull request #66671 from azat/mongodb-uuid-type 
Support true UUID type for MongoDB engine
 2024-08-05 12:25:17 +00:00
János Benjamin Antal
0da44ad047
Merge pull request #67394 from ClickHouse/add-debug-to-s3-queue-shards-distributed 
Print debug info in `test_storage_s3_queue/test.py::test_shards_distributed`
 2024-08-05 12:23:40 +00:00
Alexey Milovidov
07f039677e
Merge pull request #67771 from ClickHouse/whitespaces 
Whitespaces
 2024-08-05 12:21:20 +00:00
Alexey Milovidov
78e26ac15f
Merge pull request #67784 from ClickHouse/fix-02231_bloom_filter_sizing 
Fix test `02231_bloom_filter_sizing`
 2024-08-05 12:21:10 +00:00
Kseniia Sumarokova
3111f0f32f
Merge pull request #67540 from ClickHouse/fix-busy-loop 
Fix (incorrectly) immediate reschedule of FileCache::freeSpaceRatioKeepingThreadFunc() in case of non-evictable cache
 2024-08-05 11:33:26 +00:00
János Benjamin Antal
8fece64f33
Merge pull request #66214 from sguerrini97/use_same_password_for_base_backup 
Allow to create incremental password protected backups
 2024-08-05 11:18:21 +00:00
Raúl Marín
079b8e1db2
Merge pull request #67405 from rschu1ze/input_row_count_remainders 
Follow-up to #67294
 2024-08-05 11:09:26 +00:00
Kseniia Sumarokova
2f1259dbfd
Merge pull request #67654 from shiyer7474/parameterized_view_serialization_fix 
Fix serialization of parameterized view parameters
 2024-08-05 10:56:13 +00:00
Kruglov Pavel
2c5687f3a4
Merge pull request #67122 from Avogar/variant-aggregate-column 
Fix crash with Variant + AggregateFunction type
 2024-08-05 10:49:22 +00:00
vdimir
2894b4339d
Merge pull request #67538 from ClickHouse/vdimir/join_on_tuple_comparison_elimination_bug 
Fix join on tuple with NULLs
 2024-08-05 10:22:32 +00:00
vdimir
908f3fc937
Merge pull request #66257 from bigo-sg/ch_support_printf 
Add function printf for spark compatiability
 2024-08-05 09:53:22 +00:00
vdimir
7add8ecc7a
Merge pull request #67135 from azat/local/config-from-home 
Read configuration for clickhouse-local from ~/.clickhouse-local
 2024-08-05 09:44:22 +00:00
Alexey Katsman
5006c571f3
Merge pull request #67674 from Blokje5/use-functionargumentdescriptors-in-bitslice 
Use FunctionArgumentDescriptors to check bitSlice function + add test case
 2024-08-05 09:36:27 +00:00
Azat Khuzhin
6b1e184e12 Print original query for AST formatting check on CI 
It may be tricky to understand the root cause of the AST formatting
issue in case of syntax error, so add one knob to control this -
debug_ast_formatting_print_original_query.

And CI contains core dumps anyway, so let's enable for CI.

P.S. There was concern from @al13n321 that printing original query even
in debug build is not a good idea [1], hence a knob for this.

  [1]: https://github.com/ClickHouse/ClickHouse/pull/63357/files#r1674809348

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2024-08-05 08:37:45 +02:00
Alexey Milovidov
681eafef79
Merge pull request #67752 from azat/ci/stateless 
ci: better stateless runner (correctly collect artifacts and also some basic errors capturing)
 2024-08-05 02:53:11 +00:00
Alexey Milovidov
5a860fcc3a
Update 02099_tsv_raw_format.sh 2024-08-05 00:25:56 +02:00
Alexey Milovidov
270dddc00f Fix test 02231_bloom_filter_sizing 2024-08-05 00:22:48 +02:00
Alexey Milovidov
0bdaa57023 Fix diagnostics in the test script 2024-08-04 23:53:54 +02:00
Alexey Milovidov
83be27cdf5 Fix test 02833_concurrent_sessions, Fix test 02835_drop_user_during_session 2024-08-04 23:34:01 +02:00
Azat Khuzhin
43cf85ef28 ci: collect basic issues in stateless tests (dmesg, fatal and similar) 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2024-08-04 23:19:33 +02:00
Azat Khuzhin
c200f43774 ci: fix stateless runner to correctly gather artifacts when server failed 
Right now it simply does not work due to "set -e", with it you cannot
use "foo=$(false)" since bash will break execution after, rewrite it to
a plain shell, with "if".

Also use ZSTD everywhere (ugh)

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2024-08-04 23:19:33 +02:00
Alexey Milovidov
1e19459610
Merge pull request #67334 from petern48/aliasFeature 
New Feature: Add camelCase aliases denseRank() and percentRank()
 2024-08-04 20:52:31 +00:00
Alexey Milovidov
16e8321d48
Merge pull request #67751 from azat/dict-fix-async-loading 
Fix dictionary hang in case of CANNOT_SCHEDULE_TASK while loading (also fixes possible stress tests hung)
 2024-08-04 20:46:57 +00:00
Azat Khuzhin
a49924b383 tests: avoid leaving leftovers after test_storage_mongodb (fixes flaky chec) 
CI: https://s3.amazonaws.com/clickhouse-test-reports/66671/2f00c962711e13ca00af324366421fe4593b4ce6/integration_tests_flaky_check__asan_.html
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2024-08-04 22:01:08 +02:00
Alexey Milovidov
bd50224329 Whitespaces 2024-08-04 20:57:22 +02:00
Alexey Milovidov
ce58549592
Merge pull request #67557 from rschu1ze/icu-s390x 
Re-enable ICU on s390/x
 2024-08-04 17:51:41 +00:00
Alexey Milovidov
6f9d78d873
Merge pull request #67743 from azat/tests/test_storage_rabbitmq 
Try to fix RabbitMQ test failures
 2024-08-04 19:26:23 +02:00
Nikita Fomichev
c4a0e7665c
Merge pull request #67747 from azat/tests/test_ttl_move 
Fix test_ttl_move::test_alter_with_merge_work flakiness
 2024-08-04 16:47:24 +00:00
Alexey Milovidov
37da0b3c30 Add a test 2024-08-04 18:27:37 +02:00
Alexey Milovidov
87e0cf6b76 Fix stack overflow in JSONMergePatch 2024-08-04 18:25:36 +02:00
Alexey Milovidov
0e013ba62e
Merge pull request #67306 from ucasfl/uniq-crash 
Fix crash of uniq and uniqTheta with empty tuple argument
 2024-08-04 16:12:00 +00:00
Alexey Milovidov
98c791160e
Merge pull request #67746 from ClickHouse/fix-flaky-testss 
Stateless tests: attempt to fix timeouts of `02473_multistep_prewhere* 00411_long_accurate_number_comparison*`
 2024-08-04 15:49:05 +00:00
Alexey Milovidov
fbd3e122e1
Merge pull request #67512 from ClickHouse/divanik/fix_retry_integration_tests_decorator 
[Green CI] Test `test_storage_azure_blob_storage/test.py` is flaky
 2024-08-04 15:46:55 +00:00
Alexey Milovidov
623665dc45
Merge pull request #67739 from azat/system.clusters-mask 
Fill only selected columns from system.clusters
 2024-08-04 15:46:01 +00:00
Alexey Milovidov
592f966c2e
Merge pull request #67740 from azat/bump-nuraft 
Bump NuRaft (to properly catch thread exceptions)
 2024-08-04 15:45:41 +00:00
Alexey Milovidov
7073022f11
Merge pull request #67623 from ClickHouse/fix-logical-error-in-cached-about-expected-read-size 
Fix `Logical error: 'file_offset_of_buffer_end <= read_until_position'` in fs cache
 2024-08-04 15:44:46 +00:00
Azat Khuzhin
eed2edd7db Comment certificateFile/privateKeyFile/dhParamsFile in keeper config 
Otherwise you will get annoying messages at startup:

    2024.07.02 10:03:38.331593 [ 1 ] {} <Error> CertificateReloader: Cannot obtain modification time for certificate file /etc/clickhouse-keeper/server.crt, skipping update. errno: 2, strerror: 0
    2024.07.02 10:03:38.331658 [ 1 ] {} <Error> CertificateReloader: Cannot obtain modification time for key file /etc/clickhouse-keeper/server.key, skipping update. errno: 2, strerror: 0
    2024.07.02 10:03:38.341085 [ 1 ] {} <Error> CertificateReloader: Poco::Exception. Code: 1000, e.code() = 0, SSL context exception: Error loading private key from file /etc/clickhouse-keeper/server.key: error:80000002:system library::No such file or directory

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2024-08-04 16:20:42 +02:00
Azat Khuzhin
1f2bb6676f Remove outdated comments about workaround for #64086 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2024-08-04 16:14:05 +02:00
Azat Khuzhin
dea95e6c33 Ensure that llvm-symbolizer is used for symbolizing sanitizer reports 
Since you don't want to fallback to addr2line:

    # addr2line
    $ time /bin/test
    set_flag_impl: Success
    set_flag_if: Success
    /usr/bin/addr2line: DWARF error: invalid or unhandled FORM value: 0x23
    ==================
    WARNING: ThreadSanitizer: data race (pid=18)
    ...
    real    3m8.580s
    user    0m21.967s
    sys     0m40.628s

    # llvm-symbolizer
    $ time ./test
    set_flag_impl: Success
    set_flag_if: Success
    ==================
    WARNING: ThreadSanitizer: data race (pid=24884)
    real    0m0.028s
    user    0m0.003s
    sys     0m0.006s

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2024-08-04 16:14:05 +02:00
Azat Khuzhin
9f31488e50 Fix dictionary hang in case of CANNOT_SCHEDULE_TASK while loading 
On CI you can find that 01747_executable_pool_dictionary_implicit_key
can hang [1], it is possible due to after CANNOT_SCHEDULE_TASK the async
loading will hang:

    2024.07.18 03:56:32.365226 [ 6138 ] {6206a18f-668c-4a5c-a5ad-07f577220762} <Trace> ExternalDictionariesLoader: Will load the object 'executable_pool_simple_implicit_key' in background, force = false, loading_id = 2
    2024.07.18 03:56:32.368005 [ 6138 ] {6206a18f-668c-4a5c-a5ad-07f577220762} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=766, jobs=746): In scope SELECT dictGet('executable_pool_simple_implicit_key', 'a', toUInt64(1)). (CANNOT_SCHEDULE_TASK) (version 24.7.1.2241) (from [::1]:56446) (comment: 01747_executable_pool_dictionary_implicit_key.sql) (in query: SELECT dictGet('executable_pool_simple_implicit_key', 'a', toUInt64(1));), Stack trace (when copying this message, always include the lines below):
    0. /build/contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000015f8a292
    1. /build/src/Common/Exception.cpp:110: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000c3df6b9
    2. /build/contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(PreformattedMessage&&, int) @ 0x0000000006de714c
    3. /build/contrib/llvm-project/libcxx/include/vector:438: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000000c4838eb
    4. /build/src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000000c4832d3
    5. /build/src/Common/ThreadPool.cpp:186: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000000c47e7db
    6. /build/contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000000c47ec8d
    7. /build/contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001114b16e
    8. /build/contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::ExternalLoader::LoadingDispatcher::startLoading(DB::ExternalLoader::LoadingDispatcher::Info&, bool, unsigned long) @ 0x0000000011147733
    9. /build/src/Interpreters/ExternalLoader.cpp:837: DB::ExternalLoader::LoadingDispatcher::loadImpl(String const&, std::chrono::duration<long long, std::ratio<1l, 1000l>>, bool, std::unique_lock<std::mutex>&)::'lambda'()::operator()() const @ 0x0000000011158bf9
    10. /build/contrib/llvm-project/libcxx/include/__mutex_base:397: DB::ExternalLoader::LoadingDispatcher::loadImpl(String const&, std::chrono::duration<long long, std::ratio<1l, 1000l>>, bool, std::unique_lock<std::mutex>&) @ 0x00000000111588bc
    11. /build/src/Interpreters/ExternalLoader.cpp:604: DB::ExternalLoader::LoadResult DB::ExternalLoader::LoadingDispatcher::tryLoad<DB::ExternalLoader::LoadResult>(String const&, std::chrono::duration<long long, std::ratio<1l, 1000l>>) @ 0x00000000111440bf
    12. /build/src/Interpreters/ExternalLoader.cpp:1381: std::shared_ptr<DB::IExternalLoadable const> DB::ExternalLoader::load<std::shared_ptr<DB::IExternalLoadable const>, void>(String const&) const @ 0x00000000111442f5
    13. /build/contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:587: DB::ExternalDictionariesLoader::getDictionary(String const&, std::shared_ptr<DB::Context const>) const @ 0x0000000011141028
    14. /build/src/Functions/FunctionsExternalDictionaries.h:76: DB::FunctionDictHelper::getDictionary(String const&) @ 0x00000000071d28ec
    ...
    2024.07.18 03:58:29.000900 [ 48468 ] {8cf63d7e-dcbf-4af6-bd7c-0e1789ddce3b} <Debug> executeQuery: (from [::1]:40410) (comment: 01747_executable_pool_dictionary_implicit_key.sql) SELECT dictGet('executable_pool_simple_implicit_key', 'a', toUInt64(1)); (stage: Complete)
    # and no more rows for 8cf63d7e-dcbf-4af6-bd7c-0e1789ddce3b

  [1]: https://s3.amazonaws.com/clickhouse-test-reports/66495/bc029ed8207ac75e96e9cb48cb79d27a9ffa4e2f/stress_test__debug_.html

The problem that it should be properly cancelled, otherwise it will not
be loaded in loadImpl(), but will be waited.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2024-08-04 15:48:59 +02:00
Alexey Milovidov
c629e2af50
Merge pull request #67741 from ClickHouse/arrayWithConstant-safety-thresholds 
Better safety thresholds in `arrayWithConstant` function
 2024-08-04 13:22:06 +00:00