thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-09-20 08:40:50 +00:00
Code Issues Packages Projects Releases Wiki Activity
149,374 Commits 8,927 Branches 1,765 Tags 2 GiB
ef811fd25c
Commit Graph

149374 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Alexey Milovidov
538761b43d Fix flaky check 2024-08-05 00:02:32 +02:00
Alexey Milovidov
a573b2926e Fixes for the script 2024-08-04 23:59:44 +02:00
Alexey Milovidov
0bdaa57023 Fix diagnostics in the test script 2024-08-04 23:53:54 +02:00
Alexey Milovidov
83be27cdf5 Fix test 02833_concurrent_sessions, Fix test 02835_drop_user_during_session 2024-08-04 23:34:01 +02:00
Azat Khuzhin
43cf85ef28 ci: collect basic issues in stateless tests (dmesg, fatal and similar) 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2024-08-04 23:19:33 +02:00
Azat Khuzhin
c200f43774 ci: fix stateless runner to correctly gather artifacts when server failed 
Right now it simply does not work due to "set -e", with it you cannot
use "foo=$(false)" since bash will break execution after, rewrite it to
a plain shell, with "if".

Also use ZSTD everywhere (ugh)

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2024-08-04 23:19:33 +02:00
Alexey Milovidov
1e19459610
Merge pull request #67334 from petern48/aliasFeature 
New Feature: Add camelCase aliases denseRank() and percentRank()
 2024-08-04 20:52:31 +00:00
Alexey Milovidov
16e8321d48
Merge pull request #67751 from azat/dict-fix-async-loading 
Fix dictionary hang in case of CANNOT_SCHEDULE_TASK while loading (also fixes possible stress tests hung)
 2024-08-04 20:46:57 +00:00
Azat Khuzhin
a49924b383 tests: avoid leaving leftovers after test_storage_mongodb (fixes flaky chec) 
CI: https://s3.amazonaws.com/clickhouse-test-reports/66671/2f00c962711e13ca00af324366421fe4593b4ce6/integration_tests_flaky_check__asan_.html
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2024-08-04 22:01:08 +02:00
Alexey Milovidov
bd50224329 Whitespaces 2024-08-04 20:57:22 +02:00
Alexey Milovidov
576e0952b5 Merge branch 'master' into add-assert-into-timer-descriptor 2024-08-04 19:53:39 +02:00
Alexey Milovidov
ce58549592
Merge pull request #67557 from rschu1ze/icu-s390x 
Re-enable ICU on s390/x
 2024-08-04 17:51:41 +00:00
Alexey Milovidov
6f9d78d873
Merge pull request #67743 from azat/tests/test_storage_rabbitmq 
Try to fix RabbitMQ test failures
 2024-08-04 19:26:23 +02:00
Alexey Milovidov
a433115434 Fix typo 2024-08-04 19:18:00 +02:00
Alexey Milovidov
334eaf201f Merge branch 'master' into no-flaky-check 2024-08-04 19:17:32 +02:00
Nikita Fomichev
c4a0e7665c
Merge pull request #67747 from azat/tests/test_ttl_move 
Fix test_ttl_move::test_alter_with_merge_work flakiness
 2024-08-04 16:47:24 +00:00
Alexey Milovidov
37da0b3c30 Add a test 2024-08-04 18:27:37 +02:00
Alexey Milovidov
87e0cf6b76 Fix stack overflow in JSONMergePatch 2024-08-04 18:25:36 +02:00
Alexey Milovidov
0e013ba62e
Merge pull request #67306 from ucasfl/uniq-crash 
Fix crash of uniq and uniqTheta with empty tuple argument
 2024-08-04 16:12:00 +00:00
Alexey Milovidov
701db12687 Merge branch 'no-flaky-check' into fix-log-and-exception-messages-formatting 2024-08-04 18:08:31 +02:00
Alexey Milovidov
6fd4552fad Merge branch 'master' into fix-log-and-exception-messages-formatting 2024-08-04 18:08:28 +02:00
Alexey Milovidov
a75c4b9f9f Introduce no-flaky-check tag 2024-08-04 18:07:40 +02:00
Alexey Milovidov
98c791160e
Merge pull request #67746 from ClickHouse/fix-flaky-testss 
Stateless tests: attempt to fix timeouts of `02473_multistep_prewhere* 00411_long_accurate_number_comparison*`
 2024-08-04 15:49:05 +00:00
Alexey Milovidov
fbd3e122e1
Merge pull request #67512 from ClickHouse/divanik/fix_retry_integration_tests_decorator 
[Green CI] Test `test_storage_azure_blob_storage/test.py` is flaky
 2024-08-04 15:46:55 +00:00
Alexey Milovidov
623665dc45
Merge pull request #67739 from azat/system.clusters-mask 
Fill only selected columns from system.clusters
 2024-08-04 15:46:01 +00:00
Alexey Milovidov
592f966c2e
Merge pull request #67740 from azat/bump-nuraft 
Bump NuRaft (to properly catch thread exceptions)
 2024-08-04 15:45:41 +00:00
Alexey Milovidov
7073022f11
Merge pull request #67623 from ClickHouse/fix-logical-error-in-cached-about-expected-read-size 
Fix `Logical error: 'file_offset_of_buffer_end <= read_until_position'` in fs cache
 2024-08-04 15:44:46 +00:00
Azat Khuzhin
eed2edd7db Comment certificateFile/privateKeyFile/dhParamsFile in keeper config 
Otherwise you will get annoying messages at startup:

    2024.07.02 10:03:38.331593 [ 1 ] {} <Error> CertificateReloader: Cannot obtain modification time for certificate file /etc/clickhouse-keeper/server.crt, skipping update. errno: 2, strerror: 0
    2024.07.02 10:03:38.331658 [ 1 ] {} <Error> CertificateReloader: Cannot obtain modification time for key file /etc/clickhouse-keeper/server.key, skipping update. errno: 2, strerror: 0
    2024.07.02 10:03:38.341085 [ 1 ] {} <Error> CertificateReloader: Poco::Exception. Code: 1000, e.code() = 0, SSL context exception: Error loading private key from file /etc/clickhouse-keeper/server.key: error:80000002:system library::No such file or directory

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2024-08-04 16:20:42 +02:00
Azat Khuzhin
1f2bb6676f Remove outdated comments about workaround for #64086 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2024-08-04 16:14:05 +02:00
Azat Khuzhin
dea95e6c33 Ensure that llvm-symbolizer is used for symbolizing sanitizer reports 
Since you don't want to fallback to addr2line:

    # addr2line
    $ time /bin/test
    set_flag_impl: Success
    set_flag_if: Success
    /usr/bin/addr2line: DWARF error: invalid or unhandled FORM value: 0x23
    ==================
    WARNING: ThreadSanitizer: data race (pid=18)
    ...
    real    3m8.580s
    user    0m21.967s
    sys     0m40.628s

    # llvm-symbolizer
    $ time ./test
    set_flag_impl: Success
    set_flag_if: Success
    ==================
    WARNING: ThreadSanitizer: data race (pid=24884)
    real    0m0.028s
    user    0m0.003s
    sys     0m0.006s

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2024-08-04 16:14:05 +02:00
Azat Khuzhin
9f31488e50 Fix dictionary hang in case of CANNOT_SCHEDULE_TASK while loading 
On CI you can find that 01747_executable_pool_dictionary_implicit_key
can hang [1], it is possible due to after CANNOT_SCHEDULE_TASK the async
loading will hang:

    2024.07.18 03:56:32.365226 [ 6138 ] {6206a18f-668c-4a5c-a5ad-07f577220762} <Trace> ExternalDictionariesLoader: Will load the object 'executable_pool_simple_implicit_key' in background, force = false, loading_id = 2
    2024.07.18 03:56:32.368005 [ 6138 ] {6206a18f-668c-4a5c-a5ad-07f577220762} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=766, jobs=746): In scope SELECT dictGet('executable_pool_simple_implicit_key', 'a', toUInt64(1)). (CANNOT_SCHEDULE_TASK) (version 24.7.1.2241) (from [::1]:56446) (comment: 01747_executable_pool_dictionary_implicit_key.sql) (in query: SELECT dictGet('executable_pool_simple_implicit_key', 'a', toUInt64(1));), Stack trace (when copying this message, always include the lines below):
    0. /build/contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000015f8a292
    1. /build/src/Common/Exception.cpp:110: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000c3df6b9
    2. /build/contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(PreformattedMessage&&, int) @ 0x0000000006de714c
    3. /build/contrib/llvm-project/libcxx/include/vector:438: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000000c4838eb
    4. /build/src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000000c4832d3
    5. /build/src/Common/ThreadPool.cpp:186: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000000c47e7db
    6. /build/contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000000c47ec8d
    7. /build/contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001114b16e
    8. /build/contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::ExternalLoader::LoadingDispatcher::startLoading(DB::ExternalLoader::LoadingDispatcher::Info&, bool, unsigned long) @ 0x0000000011147733
    9. /build/src/Interpreters/ExternalLoader.cpp:837: DB::ExternalLoader::LoadingDispatcher::loadImpl(String const&, std::chrono::duration<long long, std::ratio<1l, 1000l>>, bool, std::unique_lock<std::mutex>&)::'lambda'()::operator()() const @ 0x0000000011158bf9
    10. /build/contrib/llvm-project/libcxx/include/__mutex_base:397: DB::ExternalLoader::LoadingDispatcher::loadImpl(String const&, std::chrono::duration<long long, std::ratio<1l, 1000l>>, bool, std::unique_lock<std::mutex>&) @ 0x00000000111588bc
    11. /build/src/Interpreters/ExternalLoader.cpp:604: DB::ExternalLoader::LoadResult DB::ExternalLoader::LoadingDispatcher::tryLoad<DB::ExternalLoader::LoadResult>(String const&, std::chrono::duration<long long, std::ratio<1l, 1000l>>) @ 0x00000000111440bf
    12. /build/src/Interpreters/ExternalLoader.cpp:1381: std::shared_ptr<DB::IExternalLoadable const> DB::ExternalLoader::load<std::shared_ptr<DB::IExternalLoadable const>, void>(String const&) const @ 0x00000000111442f5
    13. /build/contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:587: DB::ExternalDictionariesLoader::getDictionary(String const&, std::shared_ptr<DB::Context const>) const @ 0x0000000011141028
    14. /build/src/Functions/FunctionsExternalDictionaries.h:76: DB::FunctionDictHelper::getDictionary(String const&) @ 0x00000000071d28ec
    ...
    2024.07.18 03:58:29.000900 [ 48468 ] {8cf63d7e-dcbf-4af6-bd7c-0e1789ddce3b} <Debug> executeQuery: (from [::1]:40410) (comment: 01747_executable_pool_dictionary_implicit_key.sql) SELECT dictGet('executable_pool_simple_implicit_key', 'a', toUInt64(1)); (stage: Complete)
    # and no more rows for 8cf63d7e-dcbf-4af6-bd7c-0e1789ddce3b

  [1]: https://s3.amazonaws.com/clickhouse-test-reports/66495/bc029ed8207ac75e96e9cb48cb79d27a9ffa4e2f/stress_test__debug_.html

The problem that it should be properly cancelled, otherwise it will not
be loaded in loadImpl(), but will be waited.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2024-08-04 15:48:59 +02:00
Alexey Milovidov
c629e2af50
Merge pull request #67741 from ClickHouse/arrayWithConstant-safety-thresholds 
Better safety thresholds in `arrayWithConstant` function
 2024-08-04 13:22:06 +00:00
Alexey Milovidov
06f83022da
Merge pull request #67324 from ClickHouse/fix-json-recursion 
Fix stack overflow in too deep JSON
 2024-08-04 13:21:44 +00:00
Kseniia Sumarokova
881ac7208e
Merge pull request #67361 from ClickHouse/add-no-distributed-cache-tag 
Add no-distributed-cache tag in tests
 2024-08-04 12:36:31 +00:00
Azat Khuzhin
062490e1b4 tests: fix test_ttl_move::test_alter_with_merge_work flakiness 
Increase timeout for TTL DELETE, since otherwise if other routines will
take too long, the part will be removed when it should be still be on
"external" disk:

    2024.08.04 03:48:53.803032 [ 622 ] {} <Information> default.mt_test_alter_with_merge_work_1722743323 (9dc6904a-f082-4f06-be7a-efe4733e811c): Will drop empty part all_1_3_4_4

And this is how part_log looks like:

    SELECT
        event_time,
        event_type,
        rows,
        part_name,
        error,
        database,
        disk_name
    FROM system.part_log
    WHERE `table` = 'mt_test_alter_with_merge_work_1722743323'
    ORDER BY event_time ASC

    Query id: a118b3cd-e4fe-45a5-b675-d73bdd887d79

        ┌──────────event_time─┬─event_type─┬─rows─┬─part_name───┬─error─┬─database─┬─disk_name─┐
     1. │ 2024-08-04 03:48:44 │ NewPart    │    2 │ all_1_1_0   │     0 │ default  │ jbod1     │
     2. │ 2024-08-04 03:48:44 │ NewPart    │    2 │ all_2_2_0   │     0 │ default  │ jbod2     │
     3. │ 2024-08-04 03:48:45 │ NewPart    │    2 │ all_3_3_0   │     0 │ default  │ jbod1     │
     4. │ 2024-08-04 03:48:46 │ MutatePart │    2 │ all_1_1_0_4 │     0 │ default  │ jbod1     │
     5. │ 2024-08-04 03:48:46 │ MutatePart │    2 │ all_2_2_0_4 │     0 │ default  │ jbod2     │
     6. │ 2024-08-04 03:48:46 │ MutatePart │    2 │ all_3_3_0_4 │     0 │ default  │ jbod1     │
     7. │ 2024-08-04 03:48:47 │ MovePart   │    2 │ all_1_1_0_4 │     0 │ default  │ external  │
     8. │ 2024-08-04 03:48:47 │ MovePart   │    2 │ all_3_3_0_4 │     0 │ default  │ jbod2     │
     9. │ 2024-08-04 03:48:47 │ MergeParts │    6 │ all_1_3_1_4 │     0 │ default  │ jbod2     │
    10. │ 2024-08-04 03:48:48 │ MovePart   │    6 │ all_1_3_1_4 │     0 │ default  │ external  │
    11. │ 2024-08-04 03:48:52 │ MergeParts │    4 │ all_1_3_2_4 │     0 │ default  │ external  │
    12. │ 2024-08-04 03:48:53 │ MergeParts │    0 │ all_1_3_3_4 │     0 │ default  │ external  │ # rows==0
    13. │ 2024-08-04 03:48:53 │ MergeParts │    0 │ all_1_3_4_4 │     0 │ default  │ external  │
        └─────────────────────┴────────────┴──────┴─────────────┴───────┴──────────┴───────────┘

CI: https://s3.amazonaws.com/clickhouse-test-reports/66671/2f00c962711e13ca00af324366421fe4593b4ce6/integration_tests__tsan__[5_6].html
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2024-08-04 14:25:18 +02:00
Azat Khuzhin
47dbc5e05b tests: add debug info into test_ttl_move::test_alter_with_merge_work 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2024-08-04 14:16:40 +02:00
Azat Khuzhin
f0aaac3bd1 tests: remove useless retries from test_ttl_move::test_alter_with_merge_work 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2024-08-04 14:14:42 +02:00
Robert Schulze
e4c165e919
Merge pull request #67103 from bigo-sg/fix_map_from_arrays 
Function `mapFromArrays`: Allow `Map(K, V)` and `Array(Nullable(T))` as 1st arguments
 2024-08-04 09:54:50 +00:00
Robert Schulze
80b41a3edd
Merge remote-tracking branch 'ClickHouse/master' into input_row_count_remainders 2024-08-04 09:53:00 +00:00
Robert Schulze
27db715761
Incorporate review feedback 2024-08-04 09:52:36 +00:00
Robert Schulze
ee8079b039
Merge remote-tracking branch 'ClickHouse/master' into icu-s390x 2024-08-04 09:38:36 +00:00
Kseniia Sumarokova
4f1c2ad277
Merge pull request #67459 from ClickHouse/write-buffer-from-s3-fix-dtor 
Fix wait for tasks in ~WriteBufferFromS3 in case WriteBuffer was cancelled
 2024-08-04 09:27:24 +00:00
Kseniia Sumarokova
15e1217beb
Merge branch 'master' into fix-busy-loop 2024-08-04 11:26:39 +02:00
Azat Khuzhin
dc527b6fd1 tests: detailed errors for RabbitMQ startup 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2024-08-04 09:07:05 +02:00
Azat Khuzhin
264be9c598 tests: increase timeout for RabbitMQ startup 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2024-08-04 09:06:50 +02:00
Azat Khuzhin
59e5a09b49 Use RabbitMQ without management in tests (attempt to improve startup) 
Sometimes startup can take ~90 seconds [1]:

    2024-08-03 23:11:38.756067+00:00 [info] <0.9.0> Time to start RabbitMQ: 94651980 us

Unlike normally ~10 [2]:

    2024-08-04 00:33:07.016137+00:00 [info] <0.9.0> Time to start RabbitMQ: 10082489 us

  [1]: https://s3.amazonaws.com/clickhouse-test-reports/67737/b4e3bbcb82158bea4f5db1d9f5c28cfb741d1d51/integration_tests__asan__old_analyzer__[4_6].html
  [2]: https://s3.amazonaws.com/clickhouse-test-reports/66671/2f00c962711e13ca00af324366421fe4593b4ce6/integration_tests__asan__old_analyzer__[4_6].html

I've tried locally, the difference is very small, 3135665 us
(+management) vs (2740747 us), but still something, and who knows how it
works under pressure.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2024-08-04 09:04:41 +02:00
Nikita Fomichev
c6c4b313e6
Merge pull request #67738 from azat/tests/fix-retries 
Fix test retries
 2024-08-04 06:33:48 +00:00
Peter Nguyen
a31e2a4a5a
Merge branch 'ClickHouse:master' into aliasFeature 2024-08-03 21:38:42 -06:00
Alexey Milovidov
3efa208680
Merge pull request #67729 from ClickHouse/fix-typo 
Fix a typo
 2024-08-03 23:04:41 +00:00
Alexey Milovidov
6a20f777f6
Merge pull request #67428 from ClickHouse/enable-text-log 
Enable `text_log` by default
 2024-08-03 23:02:40 +00:00